2023-12-26T21:18:29.251 INFO:root:teuthology version: 0.0.1.dev226+g9680d2a 2023-12-26T21:18:29.251 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --archive /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800 --name yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/orig.config.yaml 2023-12-26T21:18:29.296 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-26T21:18:29.378 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800 branch: wip-yuri-testing-2023-12-26-0957-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7501800' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri-testing-2023-12-26-0957-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 5b5a60d2d510bfaa9213c41e32227fb008245a30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 5b5a60d2d510bfaa9213c41e32227fb008245a30 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-12-26-0957-quincy sha1: 5b5a60d2d510bfaa9213c41e32227fb008245a30 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6259 sha1: 5b5a60d2d510bfaa9213c41e32227fb008245a30 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-12-26-0957-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_5b5a60d2d510bfaa9213c41e32227fb008245a30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 5b5a60d2d510bfaa9213c41e32227fb008245a30 targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvWSIYGN42nYXiZs+QFU5n975Xqni3CVSzG4cs89fCCP7TwSr65VsCey5e0Y0vLDmCeAeWE8TtPLR38WHbqHJ8= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBElAKLCJzJNC0gYvfEhSyz+aEWg0uonEKHK9wzdREnEf7++OXkZtvR7zKpar+37n0Ax4xVUToUYupuA29ji5gfA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9680d2aba1e76f64ade295c81599c97907457848 timestamp: 2023-12-26_19:48:51 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078 2023-12-26T21:18:29.378 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_5b5a60d2d510bfaa9213c41e32227fb008245a30/qa; will attempt to use it 2023-12-26T21:18:29.379 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_5b5a60d2d510bfaa9213c41e32227fb008245a30/qa/tasks 2023-12-26T21:18:29.379 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-26T21:18:29.381 INFO:teuthology.task.internal:Checking packages... 2023-12-26T21:18:29.404 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '5b5a60d2d510bfaa9213c41e32227fb008245a30' 2023-12-26T21:18:29.404 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-26T21:18:29.404 INFO:teuthology.packaging:ref: None 2023-12-26T21:18:29.404 INFO:teuthology.packaging:tag: None 2023-12-26T21:18:29.404 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-12-26-0957-quincy 2023-12-26T21:18:29.404 INFO:teuthology.packaging:sha1: 5b5a60d2d510bfaa9213c41e32227fb008245a30 2023-12-26T21:18:29.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-12-26-0957-quincy 2023-12-26T21:18:29.663 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-201.g5b5a60d2 2023-12-26T21:18:29.664 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-26T21:18:29.671 INFO:teuthology.task.internal:no buildpackages task found 2023-12-26T21:18:29.671 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-26T21:18:29.678 INFO:teuthology.task.internal:Saving configuration 2023-12-26T21:18:29.694 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-26T21:18:29.701 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-26T21:18:29.723 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-26 21:13:34.091589', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvWSIYGN42nYXiZs+QFU5n975Xqni3CVSzG4cs89fCCP7TwSr65VsCey5e0Y0vLDmCeAeWE8TtPLR38WHbqHJ8='} 2023-12-26T21:18:29.746 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-26 21:13:34.089700', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBElAKLCJzJNC0gYvfEhSyz+aEWg0uonEKHK9wzdREnEf7++OXkZtvR7zKpar+37n0Ax4xVUToUYupuA29ji5gfA='} 2023-12-26T21:18:29.746 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-26T21:18:29.753 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-12-26T21:18:29.753 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-12-26T21:18:29.753 INFO:teuthology.run_tasks:Running task console_log... 2023-12-26T21:18:29.830 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fafddd4de50>, signals=[15]) 2023-12-26T21:18:29.831 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-26T21:18:29.836 INFO:teuthology.task.internal:Opening connections... 2023-12-26T21:18:29.837 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:18:29.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:18:29.908 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T21:18:29.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:18:29.980 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-26T21:18:29.987 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2023-12-26T21:18:30.008 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2023-12-26T21:18:30.008 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2023-12-26T21:18:30.068 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="Red Hat Enterprise Linux" 2023-12-26T21:18:30.068 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8.6 (Ootpa)" 2023-12-26T21:18:30.068 INFO:teuthology.orchestra.run.smithi131.stdout:ID="rhel" 2023-12-26T21:18:30.068 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="fedora" 2023-12-26T21:18:30.068 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8.6" 2023-12-26T21:18:30.068 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2023-12-26T21:18:30.068 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://www.redhat.com/" 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-26T21:18:30.069 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-26T21:18:30.070 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2023-12-26T21:18:30.094 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2023-12-26T21:18:30.114 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2023-12-26T21:18:30.114 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-12-26T21:18:30.175 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Red Hat Enterprise Linux" 2023-12-26T21:18:30.175 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8.6 (Ootpa)" 2023-12-26T21:18:30.175 INFO:teuthology.orchestra.run.smithi190.stdout:ID="rhel" 2023-12-26T21:18:30.175 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="fedora" 2023-12-26T21:18:30.175 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8.6" 2023-12-26T21:18:30.175 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2023-12-26T21:18:30.175 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-26T21:18:30.175 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2023-12-26T21:18:30.175 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-26T21:18:30.176 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.redhat.com/" 2023-12-26T21:18:30.176 INFO:teuthology.orchestra.run.smithi190.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-26T21:18:30.176 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-26T21:18:30.176 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:18:30.176 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-26T21:18:30.176 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-26T21:18:30.176 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-26T21:18:30.176 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-26T21:18:30.176 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-12-26T21:18:30.210 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-26T21:18:30.220 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-26T21:18:30.226 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-26T21:18:30.226 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2023-12-26T21:18:30.228 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2023-12-26T21:18:30.247 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-26T21:18:30.254 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-26T21:18:30.254 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2023-12-26T21:18:30.288 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2023-12-26T21:18:30.319 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-26T21:18:30.375 INFO:teuthology.run_tasks:Running task kernel... 2023-12-26T21:18:30.389 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-12-26T21:18:30.389 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-12-26T21:18:30.389 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-12-26T21:18:30.390 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-26T21:18:30.391 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-26T21:18:30.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T21:18:30.409 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-12-26T21:18:30.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T21:18:30.413 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-12-26T21:18:30.585 DEBUG:teuthology.task.kernel:sha1 for client.1 is 623393c9d50c04c7ea1d88824b8e92712e3e5611 2023-12-26T21:18:30.585 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "623393c9d50c04c7ea1d88824b8e92712e3e5611"... 2023-12-26T21:18:30.585 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2023-12-26T21:18:30.592 DEBUG:teuthology.task.kernel:sha1 for client.0 is 623393c9d50c04c7ea1d88824b8e92712e3e5611 2023-12-26T21:18:30.592 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "623393c9d50c04c7ea1d88824b8e92712e3e5611"... 2023-12-26T21:18:30.593 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2023-12-26T21:18:30.606 INFO:teuthology.orchestra.run.smithi190.stdout:4.18.0-513.9.1.el8_9.x86_64 2023-12-26T21:18:30.606 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 623393c9d50c04c7ea1d88824b8e92712e3e5611 2023-12-26T21:18:30.606 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-26T21:18:30.607 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum upgrade -y linux-firmware 2023-12-26T21:18:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:4.18.0-513.9.1.el8_9.x86_64 2023-12-26T21:18:30.613 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 623393c9d50c04c7ea1d88824b8e92712e3e5611 2023-12-26T21:18:30.613 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-26T21:18:30.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum upgrade -y linux-firmware 2023-12-26T21:18:32.149 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-12-26T21:18:32.149 INFO:teuthology.orchestra.run.smithi131.stdout:Unable to read consumer identity 2023-12-26T21:18:32.201 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-12-26T21:18:32.201 INFO:teuthology.orchestra.run.smithi190.stdout:Unable to read consumer identity 2023-12-26T21:18:32.425 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:18:32.425 INFO:teuthology.orchestra.run.smithi131.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-26T21:18:32.425 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:18:32.484 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:18:32.484 INFO:teuthology.orchestra.run.smithi190.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-26T21:18:32.484 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:18:34.213 INFO:teuthology.orchestra.run.smithi131.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-12-26T21:18:34.232 INFO:teuthology.orchestra.run.smithi190.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-12-26T21:18:36.032 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 16 MB 00:01 2023-12-26T21:18:39.633 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 509 kB/s | 24 kB 00:00 2023-12-26T21:18:40.464 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-12-26T21:18:40.465 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2023-12-26T21:18:40.465 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-12-26T21:18:40.533 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 2.6 MB/s | 16 MB 00:06 2023-12-26T21:18:40.589 INFO:teuthology.task.kernel:Downloading kernel 623393c9d50c04c7ea1d88824b8e92712e3e5611 on client.1... 2023-12-26T21:18:40.590 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=623393c9d50c04c7ea1d88824b8e92712e3e5611 2023-12-26T21:18:40.809 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/623393c9d50c04c7ea1d88824b8e92712e3e5611/centos/8/flavors/default/x86_64/ 2023-12-26T21:18:40.809 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g623393c9d50c-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/623393c9d50c04c7ea1d88824b8e92712e3e5611/centos/8/flavors/default/x86_64/ --input-file=- 2023-12-26T21:18:40.812 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi190.front.sepia.ceph.com to complete... 2023-12-26T21:18:44.138 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 452 kB/s | 24 kB 00:00 2023-12-26T21:18:44.981 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2023-12-26T21:18:44.982 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2023-12-26T21:18:44.982 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2023-12-26T21:18:45.124 INFO:teuthology.task.kernel:Downloading kernel 623393c9d50c04c7ea1d88824b8e92712e3e5611 on client.0... 2023-12-26T21:18:45.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=623393c9d50c04c7ea1d88824b8e92712e3e5611 2023-12-26T21:18:45.368 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/623393c9d50c04c7ea1d88824b8e92712e3e5611/centos/8/flavors/default/x86_64/ 2023-12-26T21:18:45.368 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g623393c9d50c-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/623393c9d50c04c7ea1d88824b8e92712e3e5611/centos/8/flavors/default/x86_64/ --input-file=- 2023-12-26T21:18:45.370 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi131.front.sepia.ceph.com to complete... 2023-12-26T21:18:55.446 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26 21:18:55 URL:https://3.chacra.ceph.com/r/kernel/testing/623393c9d50c04c7ea1d88824b8e92712e3e5611/centos/8/flavors/default/x86_64/kernel-6.6.0_g623393c9d50c-1.x86_64.rpm [383182428/383182428] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-26T21:18:55.446 INFO:teuthology.orchestra.run.smithi190.stderr:FINISHED --2023-12-26 21:18:55-- 2023-12-26T21:18:55.446 INFO:teuthology.orchestra.run.smithi190.stderr:Total wall clock time: 15s 2023-12-26T21:18:55.446 INFO:teuthology.orchestra.run.smithi190.stderr:Downloaded: 1 files, 365M in 14s (25.4 MB/s) 2023-12-26T21:18:55.449 INFO:teuthology.task.kernel:Installing kernel 623393c9d50c04c7ea1d88824b8e92712e3e5611 on client.1... 2023-12-26T21:18:55.449 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-26T21:18:57.200 INFO:teuthology.orchestra.run.smithi190.stdout:Verifying... ######################################## 2023-12-26T21:18:57.299 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing... ######################################## 2023-12-26T21:18:57.463 INFO:teuthology.orchestra.run.smithi190.stdout:Updating / installing... 2023-12-26T21:19:18.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26 21:19:18 URL:https://3.chacra.ceph.com/r/kernel/testing/623393c9d50c04c7ea1d88824b8e92712e3e5611/centos/8/flavors/default/x86_64/kernel-6.6.0_g623393c9d50c-1.x86_64.rpm [383182428/383182428] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-26T21:19:18.440 INFO:teuthology.orchestra.run.smithi131.stderr:FINISHED --2023-12-26 21:19:18-- 2023-12-26T21:19:18.440 INFO:teuthology.orchestra.run.smithi131.stderr:Total wall clock time: 33s 2023-12-26T21:19:18.440 INFO:teuthology.orchestra.run.smithi131.stderr:Downloaded: 1 files, 365M in 33s (11.2 MB/s) 2023-12-26T21:19:18.443 INFO:teuthology.task.kernel:Installing kernel 623393c9d50c04c7ea1d88824b8e92712e3e5611 on client.0... 2023-12-26T21:19:18.443 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-26T21:19:20.219 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying... ######################################## 2023-12-26T21:19:20.488 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing... ######################################## 2023-12-26T21:19:21.222 INFO:teuthology.orchestra.run.smithi131.stdout:Updating / installing... 2023-12-26T21:19:35.780 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-6.6.0_g623393c9d50c-1 ######################################## 2023-12-26T21:19:37.214 INFO:teuthology.orchestra.run.smithi190.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-26T21:19:58.739 INFO:teuthology.orchestra.run.smithi190.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-26T21:20:00.012 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-6.6.0_g623393c9d50c-1 ######################################## 2023-12-26T21:20:04.743 INFO:teuthology.orchestra.run.smithi131.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-26T21:20:23.772 INFO:teuthology.orchestra.run.smithi131.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-26T21:22:04.150 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-26T21:22:04.151 DEBUG:teuthology.orchestra.run.smithi190:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-26T21:22:04.209 INFO:teuthology.orchestra.run.smithi190.stdout:/boot/System.map-6.6.0-g623393c9d50c 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/boot/config-6.6.0-g623393c9d50c 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/boot/vmlinuz-6.6.0-g623393c9d50c 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-26T21:22:04.210 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-26T21:22:04.211 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kernel 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kernel/cpu 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kernel/cpu/mce 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kvm 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kvm/kvm.ko 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/ansi_cprng.ko 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/anubis.ko 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/arc4.ko 2023-12-26T21:22:04.212 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_memcpy.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_pq.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_tx.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_xor.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/raid6test.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/authenc.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/authencesn.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/blake2b_generic.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/blowfish_common.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/blowfish_generic.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/camellia_generic.ko 2023-12-26T21:22:04.213 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cast5_generic.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cast6_generic.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cast_common.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/ccm.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cmac.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/crc32_generic.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/crypto_engine.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/crypto_user.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cts.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/des_generic.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/dh_generic.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/echainiv.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/essiv.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/fcrypt.ko 2023-12-26T21:22:04.214 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/khazad.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/lz4.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/lz4hc.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/md4.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/michael_mic.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/pcbc.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/pcrypt.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/rmd160.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/seed.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/serpent_generic.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/tcrypt.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/tea.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/twofish_common.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/twofish_generic.ko 2023-12-26T21:22:04.215 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/vmac.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/wp512.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/xcbc.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/xor.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/xxhash_generic.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/acpi_pad.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/custom_method.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/ec_sys.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/platform_profile.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/sbs.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/sbshc.ko 2023-12-26T21:22:04.216 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/video.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/acard-ahci.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/ahci_platform.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/ata_generic.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/libahci_platform.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_acpi.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_ali.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_amd.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_artop.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_atiixp.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_atp867x.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_cmd640.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_cmd64x.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_cypress.ko 2023-12-26T21:22:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_efar.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_hpt366.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_hpt37x.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_it8213.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_it821x.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_jmicron.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_marvell.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_mpiix.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_netcell.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_ninja32.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_ns87410.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_ns87415.ko 2023-12-26T21:22:04.218 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_oldpiix.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_opti.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_optidma.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_pcmcia.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_piccolo.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_rdc.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_sch.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_serverworks.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_sil680.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_sis.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_sl82c105.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_triflex.ko 2023-12-26T21:22:04.219 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_via.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pdc_adma.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_inic162x.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_mv.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_nv.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_promise.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_qstor.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_sil.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_sil24.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_sis.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_svw.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_sx4.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_uli.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_via.ko 2023-12-26T21:22:04.220 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_vsc.ko 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/auxdisplay 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/auxdisplay/ks0108.ko 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/base 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/base/regmap 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/bcma 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/bcma/bcma.ko 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/aoe 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/aoe/aoe.ko 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/brd.ko 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/drbd 2023-12-26T21:22:04.221 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/drbd/drbd.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/floppy.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/loop.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/mtip32xx 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/nbd.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/null_blk 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/null_blk/null_blk.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/pktcdvd.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/rbd.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/virtio_blk.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/xen-blkback 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/xen-blkfront.ko 2023-12-26T21:22:04.222 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/zram 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/zram/zram.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hangcheck-timer.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/via-rng.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-26T21:22:04.223 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/lp.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/mwave 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/mwave/mwave.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ppdev.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tlclk.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-26T21:22:04.224 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/uv_mmtimer.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/virtio_console.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/ccp 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/ccp/ccp.ko 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel 2023-12-26T21:22:04.225 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat/qat_common 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/padlock-aes.ko 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/padlock-sha.ko 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/virtio 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dca 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dca/dca.ko 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/devfreq 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma 2023-12-26T21:22:04.226 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma/dw 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma/ioat 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/amd64_edac.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/e752x_edac.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/edac_mce_amd.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i3000_edac.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i3200_edac.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i5100_edac.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i5400_edac.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i7300_edac.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i7core_edac.ko 2023-12-26T21:22:04.227 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i82975x_edac.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/ie31200_edac.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/sb_edac.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/x38_edac.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/firmware 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/firmware/edd.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/gpio 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/gpio/gpio-ich.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-appleir.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-aureal.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-axff.ko 2023-12-26T21:22:04.228 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-dr.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-elecom.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-elo.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-emsff.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-gaff.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-gt683r.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-gyration.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-holtekff.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-hyperv.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-icade.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-keytouch.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-kye.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-lcpower.ko 2023-12-26T21:22:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-led.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-lenovo.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-multitouch.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-ortek.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-penmount.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-petalynx.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-picolcd.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-pl.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-plantronics.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-primax.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-rmi.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-26T21:22:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-common.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-saitek.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-samsung.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-sjoy.ko 2023-12-26T21:22:04.231 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-sony.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-speedlink.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-steelseries.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-sunplus.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-tivo.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-tmff.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-topseed.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-twinhan.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-uclogic.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-waltop.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-wiimote.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-xinmo.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-zpff.ko 2023-12-26T21:22:04.232 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-zydacron.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/uhid.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/wacom.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hv 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hv/hv_balloon.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hv/hv_utils.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hv/hv_vmbus.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/abituguru.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/abituguru3.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ad7414.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ad7418.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adc128d818.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm1025.ko 2023-12-26T21:22:04.233 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm1026.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm1029.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm1031.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm9240.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ads7828.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7410.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7411.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7462.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7470.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7475.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7x10.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/amc6821.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/applesmc.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/asb100.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/asc7621.ko 2023-12-26T21:22:04.234 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/atxp1.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/coretemp.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/dme1737.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ds1621.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ds620.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/emc1403.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/emc6w201.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/f71805f.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/f71882fg.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/f75375s.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/fam15h_power.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/fschmd.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/g760a.ko 2023-12-26T21:22:04.235 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/g762.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/gl518sm.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/gl520sm.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/i5500_temp.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/i5k_amb.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ibmaem.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ibmpex.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ina209.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ina2xx.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/it87.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/k10temp.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/k8temp.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lineage-pem.ko 2023-12-26T21:22:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm63.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm73.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm75.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm77.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm78.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm80.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm83.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm85.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm87.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm90.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm92.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm93.ko 2023-12-26T21:22:04.237 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm95234.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm95241.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm95245.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc2945.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4151.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4215.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4222.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4245.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4260.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4261.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max16065.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max1619.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max1668.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max197.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max6639.ko 2023-12-26T21:22:04.238 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max6650.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max6697.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/mcp3021.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/nct6683.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/nct6775-core.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/nct6775.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/nct7802.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pc87360.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pc87427.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pcf8591.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-26T21:22:04.239 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/powr1220.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sch5627.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sch5636.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sht15.ko 2023-12-26T21:22:04.240 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sht21.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/shtc1.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sis5595.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/smsc47b397.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/smsc47m1.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/smsc47m192.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/thmc50.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/tmp102.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/tmp103.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/tmp401.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/tmp421.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/via-cputemp.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/via686a.ko 2023-12-26T21:22:04.241 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/vt1211.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/vt8231.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83627ehf.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83627hf.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83781d.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83791d.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83792d.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83793.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83795.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83l785ts.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83l786ng.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/algos 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-26T21:22:04.242 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-26T21:22:04.243 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-dev.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-mux.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-smbus.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-stub.ko 2023-12-26T21:22:04.244 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/ib_core.ko 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/cxgb4 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mlx4 2023-12-26T21:22:04.245 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mlx5 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mthca 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/ipoib 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/iser 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/isert 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/srp 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-26T21:22:04.246 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/srpt 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/gameport 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/gameport/gameport.ko 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/joydev.ko 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/keyboard 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/appletouch.ko 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/bcm5974.ko 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/cyapatp.ko 2023-12-26T21:22:04.247 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/sermouse.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/rmi4 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio/altera_ps2.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio/arc_ps2.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio/serio_raw.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/sparse-keymap.ko 2023-12-26T21:22:04.248 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/iommu 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/iommu/amd 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/leds-blinkm.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/leds-lm3530.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/leds-lp3944.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/leds-ss4200.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-26T21:22:04.249 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/bcache 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/bcache/bcache.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-bio-prison.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-cache-smq.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-cache.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-crypt.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-delay.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-flakey.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-log-userspace.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-multipath.ko 2023-12-26T21:22:04.250 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-queue-length.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-raid.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-round-robin.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-service-time.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-switch.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-thin-pool.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-verity.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/faulty.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/linear.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/multipath.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/persistent-data 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/raid0.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/raid1.ko 2023-12-26T21:22:04.251 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/raid10.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/raid456.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptbase.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptctl.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptfc.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptlan.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptsas.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptscsih.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptspi.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/lpc_ich.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/lpc_sch.ko 2023-12-26T21:22:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/mfd-core.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/sm501.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/viperboard.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/vx855.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/wl1273-core.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/altera-stapl 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/apds9802als.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/apds990x.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/bh1770glc.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/cb710 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/cb710/cb710.ko 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom 2023-12-26T21:22:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom/at24.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom/max6875.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/enclosure.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/hpilo.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/isl29003.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/isl29020.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/lis3lv02d 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/mei 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/mei/mei-me.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/mei/mei-txe.ko 2023-12-26T21:22:04.254 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/mei/mei.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-gru 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-xp 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/tifm_7xx1.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/tifm_core.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/tsl2550.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/vmw_balloon.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/vmw_vmci 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net 2023-12-26T21:22:04.255 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/bonding 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/bonding/bonding.ko 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/b53 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/mv88e6xxx 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dummy.ko 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/eql.ko 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-26T21:22:04.256 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/adaptec 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/agere 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/alteon 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/altera 2023-12-26T21:22:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/amd 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/alx 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atlx 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-26T21:22:04.258 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/genet 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/brocade 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/brocade/bna 2023-12-26T21:22:04.259 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/cisco 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/cisco/enic 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-26T21:22:04.260 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dlink 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dnet.ko 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/emulex 2023-12-26T21:22:04.261 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/emulex/benet 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/ethoc.ko 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/fealnx.ko 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e1000 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e1000e 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/fm10k 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/i40e 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/iavf 2023-12-26T21:22:04.262 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/igb 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/igbvf 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/jme.ko 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/marvell 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-26T21:22:04.263 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/micrel 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/myricom 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-26T21:22:04.264 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/natsemi 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/neterion 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/nvidia 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/packetengines 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-26T21:22:04.265 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/rdc 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/rocker 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sfc 2023-12-26T21:22:04.266 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/silan 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sis 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/stmicro 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-26T21:22:04.267 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/tehuti 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/ti 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/via 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/wiznet 2023-12-26T21:22:04.268 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/xircom 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/geneve.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/hyperv 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ieee802154 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ifb.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ipvlan 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/macvlan.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/macvtap.ko 2023-12-26T21:22:04.269 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mii.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/net_failover.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/netconsole.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/nlmon.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/pcs 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-26T21:22:04.270 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/amd.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/bcm87xx.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/broadcom.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/cicada.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/davicom.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/dp83640.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/et1011c.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/icplus.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/lxt.ko 2023-12-26T21:22:04.271 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/marvell.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/micrel.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/national.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/phylink.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/qsemi.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/realtek.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/smsc.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/ste10Xp.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/vitesse.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_async.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-26T21:22:04.272 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/pppoe.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/pppox.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/pptp.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/slip 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/slip/slhc.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/slip/slip.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/sungem_phy.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/tap.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-26T21:22:04.273 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_random.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/tun.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/veth.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/virtio_net.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/vmxnet3 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/vxlan 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/vxlan/vxlan.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/xen-netback 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/xen-netfront.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ntb 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ntb/ntb.ko 2023-12-26T21:22:04.274 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/nvme 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/nvme/host 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/nvme/host/nvme-core.ko 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/nvme/host/nvme.ko 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport/parport.ko 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport/parport_cs.ko 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport/parport_pc.ko 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport/parport_serial.ko 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/hotplug 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/pcie 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/xen-pcifront.ko 2023-12-26T21:22:04.275 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pcmcia 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pcmcia/i82092.ko 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pcmcia/pd6729.ko 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pinctrl 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pinctrl/intel 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/acerhdf.ko 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-26T21:22:04.276 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/hdaps.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/intel 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/intel_ips.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-26T21:22:04.277 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/wmi.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/powercap 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/clients 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/clients/pps_parport.ko 2023-12-26T21:22:04.278 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/pps_core.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ptp 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ptp/ptp.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ptp/ptp_kvm.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-26T21:22:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-em3027.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-max6900.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-26T21:22:04.280 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-x1205.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/3w-9xxx.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/3w-sas.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/3w-xxxx.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/BusLogic.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/a100u2w.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aacraid 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-26T21:22:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/advansys.ko 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aic7xxx 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/am53c974.ko 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/arcmsr 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/atp870u.ko 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/be2iscsi 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bfa 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bfa/bfa.ko 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bnx2fc 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-26T21:22:04.282 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bnx2i 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/ch.ko 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/csiostor 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/dc395x.ko 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-26T21:22:04.283 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/dmx3191d.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/esas2r 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/esp_scsi.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fcoe 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fnic 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fnic/fnic.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/hpsa.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/hptiop.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/hv_storvsc.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/initio.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/ipr.ko 2023-12-26T21:22:04.284 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/ips.ko 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/isci 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/isci/isci.ko 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libfc 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libfc/libfc.ko 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libiscsi.ko 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libsas 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libsas/libsas.ko 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/lpfc 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid 2023-12-26T21:22:04.285 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mpt3sas 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mvsas 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mvumi.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/pm8001 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/pmcraid.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla1280.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla2xxx 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-26T21:22:04.286 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla4xxx 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/raid_class.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_debug.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/ses.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/st.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/stex.ko 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/sym53c8xx_2 2023-12-26T21:22:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/virtio_scsi.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/wd719x.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ssb 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ssb/ssb.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/iscsi 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/loopback 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_file.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_iblock.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_mod.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_pscsi.ko 2023-12-26T21:22:04.288 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_user.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/tcm_fc 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-26T21:22:04.289 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/ipwireless 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/n_gsm.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/n_hdlc.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/nozomi.ko 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/8250 2023-12-26T21:22:04.290 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/arc_uart.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/jsm 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs/core 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs/host 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio_aec.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio_cif.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio_pci_generic.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio_sercos3.ko 2023-12-26T21:22:04.291 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class/cdc-acm.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class/usblp.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class/usbtmc.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/host 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/image 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/image/mdc800.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/image/microtek.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/adutux.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/appledisplay.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/emi26.ko 2023-12-26T21:22:04.292 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/emi62.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/ezusb.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/idmouse.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/iowarrior.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/ldusb.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/legousbtower.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/sisusbvga 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/usb3503.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/usblcd.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/uss720.ko 2023-12-26T21:22:04.293 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/yurex.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/phy 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/phy/phy-generic.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/aircable.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ark3116.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ch341.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/cp210x.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/cyberjack.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/empeg.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-26T21:22:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/io_ti.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ipaq.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ipw.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ir-usb.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/keyspan.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/mct_u232.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/mos7720.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/mos7840.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/navman.ko 2023-12-26T21:22:04.295 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/omninet.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/opticon.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/option.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/oti6858.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/pl2303.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/qcaux.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/qcserial.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/quatech2.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/safe_serial.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/sierra.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ssu100.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/symbolserial.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-26T21:22:04.296 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/usb_debug.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/visor.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/whiteheat.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/uas.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-26T21:22:04.297 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-karma.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/usb-storage.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/usbip 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/pci 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-26T21:22:04.298 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/vfio.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost/vhost.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost/vhost_net.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost/vhost_scsi.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video/backlight 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video/backlight/apple_bl.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video/backlight/lcd.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio.ko 2023-12-26T21:22:04.299 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_balloon.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_mmio.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_pci.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_ring.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/masters 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/masters/ds2482.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/masters/ds2490.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-26T21:22:04.300 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/wire.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/hpwdt.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/i6300esb.ko 2023-12-26T21:22:04.301 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/ib700wdt.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/ibmasr.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/it87_wdt.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/machzwd.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/nv_tco.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/softdog.ko 2023-12-26T21:22:04.302 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/via_wdt.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/wdt_pci.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/xen_wdt.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-evtchn.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-gntalloc.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-gntdev.ko 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-pciback 2023-12-26T21:22:04.303 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-privcmd.ko 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xenfs 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/9p 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/9p/9p.ko 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/affs 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/affs/affs.ko 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/befs 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/befs/befs.ko 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/binfmt_misc.ko 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/btrfs 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/btrfs/btrfs.ko 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/cachefiles 2023-12-26T21:22:04.304 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/cachefiles/cachefiles.ko 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ceph 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ceph/ceph.ko 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/coda 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/coda/coda.ko 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/cramfs 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/cramfs/cramfs.ko 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/dlm 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/dlm/dlm.ko 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ecryptfs 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/f2fs 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/f2fs/f2fs.ko 2023-12-26T21:22:04.305 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fat 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fat/fat.ko 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fat/msdos.ko 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fat/vfat.ko 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fscache 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fscache/fscache.ko 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fuse 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fuse/cuse.ko 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fuse/fuse.ko 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/gfs2 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/gfs2/gfs2.ko 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/hfs 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/hfs/hfs.ko 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/hfsplus 2023-12-26T21:22:04.306 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/hfsplus/hfsplus.ko 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/isofs 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/isofs/isofs.ko 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/jfs 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/jfs/jfs.ko 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/lockd 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/lockd/lockd.ko 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/minix 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/minix/minix.ko 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/netfs 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/netfs/netfs.ko 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/blocklayout 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-26T21:22:04.307 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/filelayout 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/flexfilelayout 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/nfs.ko 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/nfsv3.ko 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/nfsv4.ko 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs_common 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs_common/grace.ko 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs_common/nfs_acl.ko 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfsd 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfsd/nfsd.ko 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nilfs2 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nilfs2/nilfs2.ko 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls 2023-12-26T21:22:04.308 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-celtic.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-centeuro.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-croatian.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-cyrillic.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-gaelic.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-greek.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-iceland.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-inuit.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-roman.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-romanian.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-turkish.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp1250.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp1251.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp1255.ko 2023-12-26T21:22:04.309 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp737.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp775.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp850.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp852.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp855.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp857.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp860.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp861.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp862.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp863.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp864.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp865.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp866.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp869.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp874.ko 2023-12-26T21:22:04.310 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp932.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp936.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp949.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp950.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_euc-jp.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-1.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-13.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-14.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-15.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-2.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-3.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-4.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-5.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-6.ko 2023-12-26T21:22:04.311 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-7.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-9.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_koi8-r.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_koi8-ru.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_koi8-u.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_utf8.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/cluster 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/dlm 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/dlmfs 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-26T21:22:04.312 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/ocfs2.ko 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/overlayfs 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/overlayfs/overlay.ko 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/pstore 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/pstore/ramoops.ko 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/reiserfs 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/reiserfs/reiserfs.ko 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/romfs 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/romfs/romfs.ko 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/client 2023-12-26T21:22:04.313 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/client/cifs.ko 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/common 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/common/cifs_arc4.ko 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/common/cifs_md4.ko 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/squashfs 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/squashfs/squashfs.ko 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/sysv 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/sysv/sysv.ko 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/udf 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/udf/udf.ko 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ufs 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ufs/ufs.ko 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/xfs 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/xfs/xfs.ko 2023-12-26T21:22:04.314 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/rcu 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/rcu/rcutorture.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/torture.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/trace 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/asn1_encoder.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crc-itu-t.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crc8.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crypto 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crypto/libarc4.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crypto/libdes.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/libcrc32c.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/lru_cache.ko 2023-12-26T21:22:04.315 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/lz4 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/lz4/lz4_compress.ko 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/lz4/lz4hc_compress.ko 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/math 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/math/cordic.ko 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/raid6 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/raid6/raid6_pq.ko 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/reed_solomon 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/ts_bm.ko 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/ts_fsm.ko 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/ts_kmp.ko 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/zstd 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/zstd/zstd_compress.ko 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/mm 2023-12-26T21:22:04.316 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/mm/hwpoison-inject.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/6lowpan.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_dest.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_fragment.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_hop.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_mobility.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_routing.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_udp.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/garp.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/mrp.ko 2023-12-26T21:22:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/p8022.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/psnap.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/stp.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/8021q 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/8021q/8021q.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p/9pnet.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p/9pnet_fd.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p/9pnet_rdma.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p/9pnet_virtio.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/batman-adv 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/batman-adv/batman-adv.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/br_netfilter.ko 2023-12-26T21:22:04.318 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/bridge.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-26T21:22:04.319 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebtables.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ceph 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ceph/libceph.ko 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/core 2023-12-26T21:22:04.320 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/core/failover.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/core/pktgen.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp/dccp.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp/dccp_diag.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp/dccp_ipv4.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp/dccp_ipv6.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dns_resolver 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dns_resolver/dns_resolver.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/dsa_core.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/tag_brcm.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/tag_dsa.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/tag_none.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/tag_trailer.ko 2023-12-26T21:22:04.321 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154/6lowpan 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154/ieee802154.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ah4.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/esp4.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/fou.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/gre.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/inet_diag.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ip_gre.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ip_tunnel.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ip_vti.ko 2023-12-26T21:22:04.322 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ipcomp.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ipip.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-26T21:22:04.323 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_bic.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_dctcp.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_diag.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_highspeed.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_htcp.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_hybla.ko 2023-12-26T21:22:04.324 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_illinois.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_lp.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_scalable.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_vegas.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_veno.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_westwood.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_yeah.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tunnel4.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/udp_diag.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/udp_tunnel.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ah6.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/esp6.ko 2023-12-26T21:22:04.325 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/fou6.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ip6_tunnel.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ip6_vti.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ipcomp6.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-26T21:22:04.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-26T21:22:04.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/sit.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/tunnel6.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/key 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/key/af_key.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_core.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_eth.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_ip.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_ip6.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_netlink.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_ppp.ko 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/llc 2023-12-26T21:22:04.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/llc/llc.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/mac802154 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/mac802154/mac802154.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/mpls 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/mpls/mpls_gso.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-26T21:22:04.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-26T21:22:04.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conncount.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-26T21:22:04.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_flow_table.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_log_syslog.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-26T21:22:04.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_irc.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_sip.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_tables.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink_log.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_chain_nat.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_compat.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_ct.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-26T21:22:04.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_fib.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_hash.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_limit.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_log.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_masq.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_nat.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_numgen.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_queue.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_quota.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_redir.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_reject.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_reject_inet.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_socket.ko 2023-12-26T21:22:04.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_tunnel.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_AUDIT.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_CT.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_DSCP.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_HL.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_HMARK.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_LED.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_LOG.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_NETMAP.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_NFLOG.ko 2023-12-26T21:22:04.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_RATEEST.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_SECMARK.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TEE.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TPROXY.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TRACE.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_addrtype.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_bpf.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_cgroup.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_cluster.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_comment.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_connbytes.ko 2023-12-26T21:22:04.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_connlabel.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_connlimit.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_connmark.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_conntrack.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_cpu.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_dccp.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_devgroup.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_dscp.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_ecn.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_esp.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_hashlimit.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_helper.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_hl.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_ipcomp.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_iprange.ko 2023-12-26T21:22:04.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_ipvs.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_l2tp.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_length.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_limit.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_mac.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_mark.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_multiport.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_nat.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_nfacct.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_osf.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_owner.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_physdev.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_pkttype.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_policy.ko 2023-12-26T21:22:04.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_quota.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_rateest.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_realm.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_recent.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_sctp.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_set.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_socket.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_state.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_statistic.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_string.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_tcpmss.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_time.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_u32.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netlink 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netlink/netlink_diag.ko 2023-12-26T21:22:04.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/nsh 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/nsh/nsh.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch/openvswitch.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch/vport-geneve.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch/vport-gre.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch/vport-vxlan.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/packet 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/packet/af_packet_diag.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/rds 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/rds/rds.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/rds/rds_rdma.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/rds/rds_tcp.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_csum.ko 2023-12-26T21:22:04.340 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_gact.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_ipt.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_mirred.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_nat.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_pedit.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_police.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_simple.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_skbedit.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_vlan.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_basic.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_bpf.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_flow.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_fw.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_route.ko 2023-12-26T21:22:04.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_u32.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_cmp.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_ipset.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_meta.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_nbyte.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_text.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_u32.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_choke.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_codel.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_drr.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_fq.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_gred.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_hfsc.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_hhf.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_htb.ko 2023-12-26T21:22:04.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_ingress.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_mqprio.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_mqprio_lib.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_multiq.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_netem.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_pie.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_plug.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_prio.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_qfq.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_red.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_sfb.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_sfq.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_tbf.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_teql.ko 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sctp 2023-12-26T21:22:04.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sctp/sctp.ko 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sctp/sctp_diag.ko 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/auth_gss 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/sunrpc.ko 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/xprtrdma 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/tipc 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/tipc/diag.ko 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/tipc/tipc.ko 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/unix 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/unix/unix_diag.ko 2023-12-26T21:22:04.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vsock.ko 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/xfrm 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys/encrypted-keys 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys/trusted-keys 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys/trusted-keys/trusted.ko 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/virt 2023-12-26T21:22:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/virt/lib 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/virt/lib/irqbypass.ko 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.alias 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.alias.bin 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.builtin 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.builtin.bin 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.builtin.modinfo 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.dep 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.dep.bin 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.devname 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.order 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.softdep 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.symbols 2023-12-26T21:22:04.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.symbols.bin 2023-12-26T21:22:04.347 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g623393c9d50c 2023-12-26T21:22:04.347 DEBUG:teuthology.orchestra.run.smithi190:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-26T21:22:04.371 INFO:teuthology.orchestra.run.smithi190.stdout:postinstall scriptlet (using /bin/sh): 2023-12-26T21:22:04.371 INFO:teuthology.orchestra.run.smithi190.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g623393c9d50c -a -r /boot/System.map-6.6.0-g623393c9d50c ]; then 2023-12-26T21:22:04.371 INFO:teuthology.orchestra.run.smithi190.stdout:cp /boot/vmlinuz-6.6.0-g623393c9d50c /boot/.vmlinuz-6.6.0-g623393c9d50c-rpm 2023-12-26T21:22:04.371 INFO:teuthology.orchestra.run.smithi190.stdout:cp /boot/System.map-6.6.0-g623393c9d50c /boot/.System.map-6.6.0-g623393c9d50c-rpm 2023-12-26T21:22:04.371 INFO:teuthology.orchestra.run.smithi190.stdout:rm -f /boot/vmlinuz-6.6.0-g623393c9d50c /boot/System.map-6.6.0-g623393c9d50c 2023-12-26T21:22:04.371 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/installkernel 6.6.0-g623393c9d50c /boot/.vmlinuz-6.6.0-g623393c9d50c-rpm /boot/.System.map-6.6.0-g623393c9d50c-rpm 2023-12-26T21:22:04.371 INFO:teuthology.orchestra.run.smithi190.stdout:rm -f /boot/.vmlinuz-6.6.0-g623393c9d50c-rpm /boot/.System.map-6.6.0-g623393c9d50c-rpm 2023-12-26T21:22:04.371 INFO:teuthology.orchestra.run.smithi190.stdout:fi 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:new-kernel-pkg --remove 6.6.0-g623393c9d50c --rminitrd --initrdfile=/boot/initramfs-6.6.0-g623393c9d50c.img 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-install remove 6.6.0-g623393c9d50c 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:fi 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/update-bootloader --remove 6.6.0-g623393c9d50c 2023-12-26T21:22:04.372 INFO:teuthology.orchestra.run.smithi190.stdout:fi 2023-12-26T21:22:04.373 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2023-12-26T21:22:04.446 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.02 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 123.el8 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Group : System Environment/Base 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 9265450 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Relocations : (not relocatable) 2023-12-26T21:22:04.447 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : Red Hat, Inc. 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : Red Hat, Inc. 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:22:04.448 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2023-12-26T21:22:04.450 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-26T21:22:04.450 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 6.6.0-g623393c9d50c 2023-12-26T21:22:04.450 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-26T21:22:05.231 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2023-12-26T21:22:07.360 INFO:teuthology.orchestra.run.smithi190.stderr:done 2023-12-26T21:22:07.362 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-12-26T21:22:07.379 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.kt4HodgGeP 2023-12-26T21:22:07.379 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kt4HodgGeP 2023-12-26T21:22:07.452 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.kt4HodgGeP 2023-12-26T21:22:07.658 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.kt4HodgGeP is 6KB 2023-12-26T21:22:07.706 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.kt4HodgGeP 2023-12-26T21:22:07.723 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2023-12-26T21:22:07.787 INFO:teuthology.orchestra.run.smithi190.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2023-12-26T21:22:07.787 INFO:teuthology.orchestra.run.smithi190.stdout:2db68f91c8f90721a6596efe657a5324-6.6.0-g623393c9d50c.conf 2023-12-26T21:22:07.787 INFO:teuthology.orchestra.run.smithi190.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2023-12-26T21:22:07.787 INFO:teuthology.orchestra.run.smithi190.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2023-12-26T21:22:07.787 INFO:teuthology.orchestra.run.smithi190.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2023-12-26T21:22:07.788 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.6.0-g623393c9d50c 2023-12-26T21:22:07.937 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2023-12-26T21:22:30.448 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-26T21:22:30.448 DEBUG:teuthology.orchestra.run.smithi131:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-26T21:22:30.502 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/System.map-6.6.0-g623393c9d50c 2023-12-26T21:22:30.502 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/config-6.6.0-g623393c9d50c 2023-12-26T21:22:30.502 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/vmlinuz-6.6.0-g623393c9d50c 2023-12-26T21:22:30.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c 2023-12-26T21:22:30.502 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-26T21:22:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kernel 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kernel/cpu 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kernel/cpu/mce 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kvm 2023-12-26T21:22:30.504 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/arch/x86/kvm/kvm.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/ansi_cprng.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/anubis.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/arc4.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_memcpy.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_pq.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_tx.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/async_xor.ko 2023-12-26T21:22:30.505 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/async_tx/raid6test.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/authenc.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/authencesn.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/blake2b_generic.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/blowfish_common.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/blowfish_generic.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/camellia_generic.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cast5_generic.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cast6_generic.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cast_common.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/ccm.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cmac.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/crc32_generic.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/crypto_engine.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/crypto_user.ko 2023-12-26T21:22:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/cts.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/des_generic.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/dh_generic.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/echainiv.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/essiv.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/fcrypt.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/khazad.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/lz4.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/lz4hc.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/md4.ko 2023-12-26T21:22:30.507 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/michael_mic.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/pcbc.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/pcrypt.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/rmd160.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/seed.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/serpent_generic.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/tcrypt.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/tea.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/twofish_common.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/twofish_generic.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/vmac.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/wp512.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/xcbc.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/xor.ko 2023-12-26T21:22:30.508 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/crypto/xxhash_generic.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/acpi_pad.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/custom_method.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/ec_sys.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/platform_profile.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/sbs.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/sbshc.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/acpi/video.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/acard-ahci.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/ahci_platform.ko 2023-12-26T21:22:30.509 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/ata_generic.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/libahci_platform.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_acpi.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_ali.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_amd.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_artop.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_atiixp.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_atp867x.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_cmd640.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_cmd64x.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_cypress.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_efar.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_hpt366.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_hpt37x.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-26T21:22:30.510 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_it8213.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_it821x.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_jmicron.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_marvell.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_mpiix.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_netcell.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_ninja32.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_ns87410.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_ns87415.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_oldpiix.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_opti.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_optidma.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_pcmcia.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-26T21:22:30.511 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_piccolo.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_rdc.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_sch.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_serverworks.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_sil680.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_sis.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_sl82c105.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_triflex.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pata_via.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/pdc_adma.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_inic162x.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_mv.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_nv.ko 2023-12-26T21:22:30.512 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_promise.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_qstor.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_sil.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_sil24.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_sis.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_svw.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_sx4.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_uli.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_via.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ata/sata_vsc.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/auxdisplay 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/auxdisplay/ks0108.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/base 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/base/regmap 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-26T21:22:30.513 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/bcma 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/bcma/bcma.ko 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/aoe 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/aoe/aoe.ko 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/brd.ko 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/drbd 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/drbd/drbd.ko 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/floppy.ko 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/loop.ko 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/mtip32xx 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/nbd.ko 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/null_blk 2023-12-26T21:22:30.514 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/null_blk/null_blk.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/pktcdvd.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/rbd.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/virtio_blk.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/xen-blkback 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/xen-blkfront.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/zram 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/block/zram/zram.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hangcheck-timer.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-26T21:22:30.515 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/via-rng.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/lp.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/mwave 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/mwave/mwave.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/ppdev.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tlclk.ko 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm 2023-12-26T21:22:30.516 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm.ko 2023-12-26T21:22:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-26T21:22:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-26T21:22:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-26T21:22:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-26T21:22:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-26T21:22:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/uv_mmtimer.ko 2023-12-26T21:22:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/char/virtio_console.ko 2023-12-26T21:22:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq 2023-12-26T21:22:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-26T21:22:30.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-26T21:22:30.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-26T21:22:30.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-26T21:22:30.518 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/ccp 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/ccp/ccp.ko 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat/qat_common 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-26T21:22:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/padlock-aes.ko 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/padlock-sha.ko 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/virtio 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dca 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dca/dca.ko 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/devfreq 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma/dw 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma/ioat 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/amd64_edac.ko 2023-12-26T21:22:30.520 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/e752x_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/edac_mce_amd.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i3000_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i3200_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i5100_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i5400_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i7300_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i7core_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/i82975x_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/ie31200_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/sb_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/edac/x38_edac.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/firmware 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/firmware/edd.ko 2023-12-26T21:22:30.521 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/gpio 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/gpio/gpio-ich.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-appleir.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-aureal.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-axff.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-dr.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-elecom.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-elo.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-emsff.ko 2023-12-26T21:22:30.522 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-gaff.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-gt683r.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-gyration.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-holtekff.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-hyperv.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-icade.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-keytouch.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-kye.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-lcpower.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-led.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-lenovo.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-26T21:22:30.523 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-multitouch.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-ortek.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-penmount.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-petalynx.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-picolcd.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-pl.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-plantronics.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-primax.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-rmi.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-common.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-26T21:22:30.524 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-roccat.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-saitek.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-samsung.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-sjoy.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-sony.ko 2023-12-26T21:22:30.525 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-speedlink.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-steelseries.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-sunplus.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-tivo.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-tmff.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-topseed.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-twinhan.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-uclogic.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-waltop.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-wiimote.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-xinmo.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-zpff.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/hid-zydacron.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/uhid.ko 2023-12-26T21:22:30.526 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hid/wacom.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hv 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hv/hv_balloon.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hv/hv_utils.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hv/hv_vmbus.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/abituguru.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/abituguru3.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ad7414.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ad7418.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adc128d818.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm1025.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm1026.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm1029.ko 2023-12-26T21:22:30.527 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm1031.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adm9240.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ads7828.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7410.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7411.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7462.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7470.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7475.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/adt7x10.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/amc6821.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/applesmc.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/asb100.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/asc7621.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/atxp1.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/coretemp.ko 2023-12-26T21:22:30.528 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/dme1737.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ds1621.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ds620.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/emc1403.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/emc6w201.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/f71805f.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/f71882fg.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/f75375s.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/fam15h_power.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/fschmd.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/g760a.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/g762.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/gl518sm.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/gl520sm.ko 2023-12-26T21:22:30.529 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/i5500_temp.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/i5k_amb.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ibmaem.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ibmpex.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ina209.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ina2xx.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/it87.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/k10temp.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/k8temp.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lineage-pem.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm63.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm73.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm75.ko 2023-12-26T21:22:30.530 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm77.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm78.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm80.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm83.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm85.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm87.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm90.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm92.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm93.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm95234.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm95241.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/lm95245.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc2945.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4151.ko 2023-12-26T21:22:30.531 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4215.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4222.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4245.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4260.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/ltc4261.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max16065.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max1619.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max1668.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max197.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max6639.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max6650.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/max6697.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/mcp3021.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/nct6683.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/nct6775-core.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/nct6775.ko 2023-12-26T21:22:30.532 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/nct7802.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pc87360.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pc87427.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pcf8591.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-26T21:22:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/powr1220.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sch5627.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sch5636.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sht15.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sht21.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/shtc1.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/sis5595.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/smsc47b397.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/smsc47m1.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/smsc47m192.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/thmc50.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/tmp102.ko 2023-12-26T21:22:30.534 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/tmp103.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/tmp401.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/tmp421.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/via-cputemp.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/via686a.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/vt1211.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/vt8231.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83627ehf.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83627hf.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83781d.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83791d.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83792d.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83793.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83795.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83l785ts.ko 2023-12-26T21:22:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/hwmon/w83l786ng.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/algos 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-26T21:22:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-26T21:22:30.537 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-dev.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-mux.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-smbus.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/i2c/i2c-stub.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/ib_core.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-26T21:22:30.538 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/cxgb4 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mlx4 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mlx5 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mthca 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/ipoib 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/iser 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/isert 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-26T21:22:30.539 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/srp 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/srpt 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/gameport 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/gameport/gameport.ko 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/joydev.ko 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/keyboard 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/appletouch.ko 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/bcm5974.ko 2023-12-26T21:22:30.540 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/cyapatp.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/sermouse.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/rmi4 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio/altera_ps2.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio/arc_ps2.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/serio/serio_raw.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/input/sparse-keymap.ko 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/iommu 2023-12-26T21:22:30.541 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/iommu/amd 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/leds-blinkm.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/leds-lm3530.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/leds-lp3944.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/leds-ss4200.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-26T21:22:30.542 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/bcache 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/bcache/bcache.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-bio-prison.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-cache-smq.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-cache.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-crypt.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-delay.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-flakey.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-log-userspace.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-multipath.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-queue-length.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-raid.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-round-robin.ko 2023-12-26T21:22:30.543 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-service-time.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-switch.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-thin-pool.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/dm-verity.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/faulty.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/linear.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/multipath.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/persistent-data 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/raid0.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/raid1.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/raid10.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/md/raid456.ko 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion 2023-12-26T21:22:30.544 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptbase.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptctl.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptfc.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptlan.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptsas.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptscsih.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/message/fusion/mptspi.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/lpc_ich.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/lpc_sch.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/mfd-core.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/sm501.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/viperboard.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/vx855.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/mfd/wl1273-core.ko 2023-12-26T21:22:30.545 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/altera-stapl 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/apds9802als.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/apds990x.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/bh1770glc.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/cb710 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/cb710/cb710.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom/at24.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/eeprom/max6875.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/enclosure.ko 2023-12-26T21:22:30.546 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/hpilo.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/isl29003.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/isl29020.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/lis3lv02d 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/mei 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/mei/mei-me.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/mei/mei-txe.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/mei/mei.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-gru 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-xp 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-26T21:22:30.547 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/tifm_7xx1.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/tifm_core.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/tsl2550.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/vmw_balloon.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/vmw_vmci 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/bonding 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/bonding/bonding.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/b53 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-26T21:22:30.548 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/mv88e6xxx 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/dummy.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/eql.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-26T21:22:30.549 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/adaptec 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/agere 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/alteon 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/altera 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/amd 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/alx 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-26T21:22:30.550 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atlx 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/genet 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-26T21:22:30.551 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/brocade 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/brocade/bna 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-26T21:22:30.552 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/cisco 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/cisco/enic 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dlink 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/dnet.ko 2023-12-26T21:22:30.553 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/emulex 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/emulex/benet 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/ethoc.ko 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/fealnx.ko 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e1000 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e1000e 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/fm10k 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/i40e 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-26T21:22:30.554 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/iavf 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/igb 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/igbvf 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/jme.ko 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/marvell 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox 2023-12-26T21:22:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/micrel 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/myricom 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/natsemi 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-26T21:22:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/neterion 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/nvidia 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/packetengines 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/rdc 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-26T21:22:30.557 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/rocker 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sfc 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/silan 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sis 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-26T21:22:30.558 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/stmicro 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/tehuti 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-26T21:22:30.559 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/ti 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/via 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/wiznet 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/xircom 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/geneve.ko 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/hyperv 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ieee802154 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-26T21:22:30.560 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ifb.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ipvlan 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/macvlan.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/macvtap.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/mii.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/net_failover.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/netconsole.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/nlmon.ko 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/pcs 2023-12-26T21:22:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/amd.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/bcm87xx.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/broadcom.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/cicada.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/davicom.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/dp83640.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/et1011c.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/icplus.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/lxt.ko 2023-12-26T21:22:30.562 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/marvell.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/micrel.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/national.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/phylink.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/qsemi.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/realtek.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/smsc.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/ste10Xp.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/phy/vitesse.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_async.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-26T21:22:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/pppoe.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/pppox.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/ppp/pptp.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/slip 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/slip/slhc.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/slip/slip.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/sungem_phy.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/tap.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_random.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-26T21:22:30.564 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/tun.ko 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/veth.ko 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/virtio_net.ko 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/vmxnet3 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/vxlan 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/vxlan/vxlan.ko 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/xen-netback 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/net/xen-netfront.ko 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ntb 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ntb/ntb.ko 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/nvme 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/nvme/host 2023-12-26T21:22:30.565 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/nvme/host/nvme-core.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/nvme/host/nvme.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport/parport.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport/parport_cs.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport/parport_pc.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/parport/parport_serial.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/hotplug 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/pcie 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pci/xen-pcifront.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pcmcia 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pcmcia/i82092.ko 2023-12-26T21:22:30.566 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pcmcia/pd6729.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pinctrl 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pinctrl/intel 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/acerhdf.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-26T21:22:30.567 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/hdaps.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/intel 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/intel_ips.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-26T21:22:30.568 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/platform/x86/wmi.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/powercap 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/clients 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/clients/pps_parport.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/pps/pps_core.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ptp 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ptp/ptp.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ptp/ptp_kvm.ko 2023-12-26T21:22:30.569 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-em3027.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-26T21:22:30.570 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-max6900.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-26T21:22:30.571 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/rtc/rtc-x1205.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/3w-9xxx.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/3w-sas.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/3w-xxxx.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/BusLogic.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/a100u2w.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aacraid 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/advansys.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aic7xxx 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/am53c974.ko 2023-12-26T21:22:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/arcmsr 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/atp870u.ko 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/be2iscsi 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bfa 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bfa/bfa.ko 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bnx2fc 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bnx2i 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/ch.ko 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/csiostor 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi 2023-12-26T21:22:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/dc395x.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/dmx3191d.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/esas2r 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/esp_scsi.ko 2023-12-26T21:22:30.574 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fcoe 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fnic 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/fnic/fnic.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/hpsa.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/hptiop.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/hv_storvsc.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/initio.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/ipr.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/ips.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/isci 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/isci/isci.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libfc 2023-12-26T21:22:30.575 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libfc/libfc.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libiscsi.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libsas 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/libsas/libsas.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/lpfc 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mpt3sas 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mvsas 2023-12-26T21:22:30.576 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/mvumi.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/pm8001 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/pmcraid.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla1280.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla2xxx 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla4xxx 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/raid_class.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_debug.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-26T21:22:30.577 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/ses.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/st.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/stex.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/sym53c8xx_2 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/virtio_scsi.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/scsi/wd719x.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ssb 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ssb/ssb.ko 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target 2023-12-26T21:22:30.578 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/iscsi 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/loopback 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_file.ko 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_iblock.ko 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_mod.ko 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_pscsi.ko 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/target_core_user.ko 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/tcm_fc 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-26T21:22:30.579 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty 2023-12-26T21:22:30.580 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/ipwireless 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/n_gsm.ko 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/n_hdlc.ko 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/nozomi.ko 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/8250 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/arc_uart.ko 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/jsm 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs 2023-12-26T21:22:30.581 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs/core 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs/host 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio_aec.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio_cif.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio_pci_generic.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/uio/uio_sercos3.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class/cdc-acm.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class/usblp.ko 2023-12-26T21:22:30.582 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/class/usbtmc.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/host 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/image 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/image/mdc800.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/image/microtek.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/adutux.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/appledisplay.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/emi26.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/emi62.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/ezusb.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/idmouse.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/iowarrior.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-26T21:22:30.583 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/ldusb.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/legousbtower.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/sisusbvga 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/usb3503.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/usblcd.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/uss720.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/misc/yurex.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/phy 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/phy/phy-generic.ko 2023-12-26T21:22:30.584 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/aircable.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ark3116.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ch341.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/cp210x.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/cyberjack.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/empeg.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/io_ti.ko 2023-12-26T21:22:30.585 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ipaq.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ipw.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ir-usb.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/keyspan.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/mct_u232.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/mos7720.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/mos7840.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/navman.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/omninet.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/opticon.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/option.ko 2023-12-26T21:22:30.586 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/oti6858.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/pl2303.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/qcaux.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/qcserial.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/quatech2.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/safe_serial.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/sierra.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ssu100.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/symbolserial.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/usb_debug.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/visor.ko 2023-12-26T21:22:30.587 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/whiteheat.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/uas.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-karma.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-26T21:22:30.588 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/storage/usb-storage.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/usbip 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/pci 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/vfio.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost/vhost.ko 2023-12-26T21:22:30.589 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-26T21:22:30.590 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost/vhost_net.ko 2023-12-26T21:22:30.590 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/vhost/vhost_scsi.ko 2023-12-26T21:22:30.590 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video 2023-12-26T21:22:30.590 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video/backlight 2023-12-26T21:22:30.590 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video/backlight/apple_bl.ko 2023-12-26T21:22:30.590 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video/backlight/lcd.ko 2023-12-26T21:22:30.590 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-26T21:22:30.592 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio 2023-12-26T21:22:30.592 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio.ko 2023-12-26T21:22:30.592 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_balloon.ko 2023-12-26T21:22:30.592 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_mmio.ko 2023-12-26T21:22:30.592 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_pci.ko 2023-12-26T21:22:30.592 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-26T21:22:30.592 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-26T21:22:30.592 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/virtio/virtio_ring.ko 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/masters 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/masters/ds2482.ko 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/masters/ds2490.ko 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-26T21:22:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/w1/wire.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/hpwdt.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/i6300esb.ko 2023-12-26T21:22:30.594 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/ib700wdt.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/ibmasr.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/it87_wdt.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/machzwd.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/nv_tco.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/softdog.ko 2023-12-26T21:22:30.595 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/via_wdt.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/wdt_pci.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/watchdog/xen_wdt.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-evtchn.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-gntalloc.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-gntdev.ko 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-pciback 2023-12-26T21:22:30.596 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xen-privcmd.ko 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xenfs 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/9p 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/9p/9p.ko 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/affs 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/affs/affs.ko 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/befs 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/befs/befs.ko 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/binfmt_misc.ko 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/btrfs 2023-12-26T21:22:30.597 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/btrfs/btrfs.ko 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/cachefiles 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/cachefiles/cachefiles.ko 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ceph 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ceph/ceph.ko 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/coda 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/coda/coda.ko 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/cramfs 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/cramfs/cramfs.ko 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/dlm 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/dlm/dlm.ko 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ecryptfs 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/f2fs 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/f2fs/f2fs.ko 2023-12-26T21:22:30.598 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fat 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fat/fat.ko 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fat/msdos.ko 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fat/vfat.ko 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fscache 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fscache/fscache.ko 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fuse 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fuse/cuse.ko 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/fuse/fuse.ko 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/gfs2 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/gfs2/gfs2.ko 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/hfs 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/hfs/hfs.ko 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/hfsplus 2023-12-26T21:22:30.599 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/hfsplus/hfsplus.ko 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/isofs 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/isofs/isofs.ko 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/jfs 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/jfs/jfs.ko 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/lockd 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/lockd/lockd.ko 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/minix 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/minix/minix.ko 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/netfs 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/netfs/netfs.ko 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/blocklayout 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-26T21:22:30.600 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/filelayout 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/flexfilelayout 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/nfs.ko 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/nfsv3.ko 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs/nfsv4.ko 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs_common 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs_common/grace.ko 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfs_common/nfs_acl.ko 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfsd 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nfsd/nfsd.ko 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nilfs2 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nilfs2/nilfs2.ko 2023-12-26T21:22:30.601 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-celtic.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-centeuro.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-croatian.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-cyrillic.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-gaelic.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-greek.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-iceland.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-inuit.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-roman.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-romanian.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/mac-turkish.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp1250.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp1251.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp1255.ko 2023-12-26T21:22:30.602 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp737.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp775.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp850.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp852.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp855.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp857.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp860.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp861.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp862.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp863.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp864.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp865.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp866.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp869.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp874.ko 2023-12-26T21:22:30.603 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp932.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp936.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp949.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_cp950.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_euc-jp.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-1.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-13.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-14.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-15.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-2.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-3.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-4.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-5.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-6.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-7.ko 2023-12-26T21:22:30.604 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_iso8859-9.ko 2023-12-26T21:22:30.605 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_koi8-r.ko 2023-12-26T21:22:30.605 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_koi8-ru.ko 2023-12-26T21:22:30.605 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_koi8-u.ko 2023-12-26T21:22:30.605 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-26T21:22:30.605 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/nls/nls_utf8.ko 2023-12-26T21:22:30.605 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2 2023-12-26T21:22:30.605 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/cluster 2023-12-26T21:22:30.605 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-26T21:22:30.605 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/dlm 2023-12-26T21:22:30.606 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-26T21:22:30.606 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/dlmfs 2023-12-26T21:22:30.606 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-26T21:22:30.606 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/ocfs2.ko 2023-12-26T21:22:30.606 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-26T21:22:30.606 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-26T21:22:30.606 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-26T21:22:30.606 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/overlayfs 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/overlayfs/overlay.ko 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/pstore 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/pstore/ramoops.ko 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/reiserfs 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/reiserfs/reiserfs.ko 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/romfs 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/romfs/romfs.ko 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/client 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/client/cifs.ko 2023-12-26T21:22:30.607 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/common 2023-12-26T21:22:30.608 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/common/cifs_arc4.ko 2023-12-26T21:22:30.608 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/smb/common/cifs_md4.ko 2023-12-26T21:22:30.608 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/squashfs 2023-12-26T21:22:30.608 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/squashfs/squashfs.ko 2023-12-26T21:22:30.608 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/sysv 2023-12-26T21:22:30.608 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/sysv/sysv.ko 2023-12-26T21:22:30.609 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/udf 2023-12-26T21:22:30.609 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/udf/udf.ko 2023-12-26T21:22:30.609 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ufs 2023-12-26T21:22:30.609 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/ufs/ufs.ko 2023-12-26T21:22:30.609 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/xfs 2023-12-26T21:22:30.609 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/fs/xfs/xfs.ko 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/rcu 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/rcu/rcutorture.ko 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/torture.ko 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/trace 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/asn1_encoder.ko 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crc-itu-t.ko 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crc8.ko 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crypto 2023-12-26T21:22:30.610 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crypto/libarc4.ko 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/crypto/libdes.ko 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/libcrc32c.ko 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/lru_cache.ko 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/lz4 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/lz4/lz4_compress.ko 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/lz4/lz4hc_compress.ko 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/math 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/math/cordic.ko 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/raid6 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/raid6/raid6_pq.ko 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/reed_solomon 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-26T21:22:30.611 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/ts_bm.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/ts_fsm.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/ts_kmp.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/zstd 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/lib/zstd/zstd_compress.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/mm 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/mm/hwpoison-inject.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/6lowpan.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_dest.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_fragment.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_hop.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_mobility.ko 2023-12-26T21:22:30.612 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_routing.ko 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/6lowpan/nhc_udp.ko 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/garp.ko 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/mrp.ko 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/p8022.ko 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/psnap.ko 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/802/stp.ko 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/8021q 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/8021q/8021q.ko 2023-12-26T21:22:30.613 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p 2023-12-26T21:22:30.614 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p/9pnet.ko 2023-12-26T21:22:30.614 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p/9pnet_fd.ko 2023-12-26T21:22:30.614 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p/9pnet_rdma.ko 2023-12-26T21:22:30.614 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/9p/9pnet_virtio.ko 2023-12-26T21:22:30.614 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/batman-adv 2023-12-26T21:22:30.614 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/batman-adv/batman-adv.ko 2023-12-26T21:22:30.614 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge 2023-12-26T21:22:30.614 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/br_netfilter.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/bridge.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-26T21:22:30.615 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/ebtables.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ceph 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ceph/libceph.ko 2023-12-26T21:22:30.616 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/core 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/core/failover.ko 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/core/pktgen.ko 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp/dccp.ko 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp/dccp_diag.ko 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp/dccp_ipv4.ko 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dccp/dccp_ipv6.ko 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dns_resolver 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dns_resolver/dns_resolver.ko 2023-12-26T21:22:30.617 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa 2023-12-26T21:22:30.618 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/dsa_core.ko 2023-12-26T21:22:30.618 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/tag_brcm.ko 2023-12-26T21:22:30.618 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/tag_dsa.ko 2023-12-26T21:22:30.618 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/tag_none.ko 2023-12-26T21:22:30.618 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/dsa/tag_trailer.ko 2023-12-26T21:22:30.618 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154 2023-12-26T21:22:30.619 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154/6lowpan 2023-12-26T21:22:30.619 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-26T21:22:30.619 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154/ieee802154.ko 2023-12-26T21:22:30.619 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-26T21:22:30.620 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4 2023-12-26T21:22:30.620 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ah4.ko 2023-12-26T21:22:30.620 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/esp4.ko 2023-12-26T21:22:30.620 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/fou.ko 2023-12-26T21:22:30.620 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/gre.ko 2023-12-26T21:22:30.620 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/inet_diag.ko 2023-12-26T21:22:30.620 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ip_gre.ko 2023-12-26T21:22:30.621 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ip_tunnel.ko 2023-12-26T21:22:30.621 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ip_vti.ko 2023-12-26T21:22:30.621 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ipcomp.ko 2023-12-26T21:22:30.621 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/ipip.ko 2023-12-26T21:22:30.621 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter 2023-12-26T21:22:30.621 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-26T21:22:30.622 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-26T21:22:30.622 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-26T21:22:30.622 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-26T21:22:30.622 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-26T21:22:30.622 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-26T21:22:30.622 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-26T21:22:30.622 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-26T21:22:30.622 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-26T21:22:30.622 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_bic.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_dctcp.ko 2023-12-26T21:22:30.623 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_diag.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_highspeed.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_htcp.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_hybla.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_illinois.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_lp.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_scalable.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_vegas.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_veno.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_westwood.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tcp_yeah.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/tunnel4.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/udp_diag.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/udp_tunnel.ko 2023-12-26T21:22:30.624 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ah6.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/esp6.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/fou6.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ip6_tunnel.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ip6_vti.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/ipcomp6.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-26T21:22:30.625 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-26T21:22:30.626 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/sit.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/tunnel6.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/key 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/key/af_key.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_core.ko 2023-12-26T21:22:30.627 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_eth.ko 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_ip.ko 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_ip6.ko 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_netlink.ko 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/l2tp/l2tp_ppp.ko 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/llc 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/llc/llc.ko 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/mac802154 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/mac802154/mac802154.ko 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/mpls 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/mpls/mpls_gso.ko 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset 2023-12-26T21:22:30.628 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-26T21:22:30.629 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-26T21:22:30.630 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conncount.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-26T21:22:30.631 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_flow_table.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_log_syslog.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_irc.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_sip.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-26T21:22:30.632 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nf_tables.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink_log.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_chain_nat.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_compat.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_ct.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_fib.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_hash.ko 2023-12-26T21:22:30.633 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_limit.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_log.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_masq.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_nat.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_numgen.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_queue.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_quota.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_redir.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_reject.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_reject_inet.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_socket.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/nft_tunnel.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_AUDIT.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-26T21:22:30.634 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_CT.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_DSCP.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_HL.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_HMARK.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_LED.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_LOG.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_NETMAP.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_NFLOG.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_RATEEST.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_SECMARK.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-26T21:22:30.635 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TEE.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TPROXY.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_TRACE.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_addrtype.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_bpf.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_cgroup.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_cluster.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_comment.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_connbytes.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_connlabel.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_connlimit.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_connmark.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_conntrack.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_cpu.ko 2023-12-26T21:22:30.636 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_dccp.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_devgroup.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_dscp.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_ecn.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_esp.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_hashlimit.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_helper.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_hl.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_ipcomp.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_iprange.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_ipvs.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_l2tp.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_length.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_limit.ko 2023-12-26T21:22:30.637 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_mac.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_mark.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_multiport.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_nat.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_nfacct.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_osf.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_owner.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_physdev.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_pkttype.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_policy.ko 2023-12-26T21:22:30.638 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_quota.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_rateest.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_realm.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_recent.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_sctp.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_set.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_socket.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_state.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_statistic.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_string.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_tcpmss.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_time.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netfilter/xt_u32.ko 2023-12-26T21:22:30.639 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netlink 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/netlink/netlink_diag.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/nsh 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/nsh/nsh.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch/openvswitch.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch/vport-geneve.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch/vport-gre.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/openvswitch/vport-vxlan.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/packet 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/packet/af_packet_diag.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/rds 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/rds/rds.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/rds/rds_rdma.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/rds/rds_tcp.ko 2023-12-26T21:22:30.640 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_csum.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_gact.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_ipt.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_mirred.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_nat.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_pedit.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_police.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_simple.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_skbedit.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/act_vlan.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_basic.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_bpf.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_flow.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_fw.ko 2023-12-26T21:22:30.641 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_route.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/cls_u32.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_cmp.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_ipset.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_meta.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_nbyte.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_text.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/em_u32.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_choke.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_codel.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_drr.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_fq.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_gred.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_hfsc.ko 2023-12-26T21:22:30.642 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_hhf.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_htb.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_ingress.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_mqprio.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_mqprio_lib.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_multiq.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_netem.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_pie.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_plug.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_prio.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_qfq.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_red.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_sfb.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_sfq.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_tbf.ko 2023-12-26T21:22:30.643 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sched/sch_teql.ko 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sctp 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sctp/sctp.ko 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sctp/sctp_diag.ko 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/auth_gss 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/sunrpc.ko 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/xprtrdma 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/tipc 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/tipc/diag.ko 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/tipc/tipc.ko 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/unix 2023-12-26T21:22:30.644 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/unix/unix_diag.ko 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vsock.ko 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/xfrm 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys/encrypted-keys 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys/trusted-keys 2023-12-26T21:22:30.645 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/security/keys/trusted-keys/trusted.ko 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/virt 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/virt/lib 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/kernel/virt/lib/irqbypass.ko 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.alias 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.alias.bin 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.builtin 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.builtin.bin 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.builtin.modinfo 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.dep 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.dep.bin 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.devname 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.order 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.softdep 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.symbols 2023-12-26T21:22:30.646 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.6.0-g623393c9d50c/modules.symbols.bin 2023-12-26T21:22:30.647 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g623393c9d50c 2023-12-26T21:22:30.647 DEBUG:teuthology.orchestra.run.smithi131:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-26T21:22:30.670 INFO:teuthology.orchestra.run.smithi131.stdout:postinstall scriptlet (using /bin/sh): 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g623393c9d50c -a -r /boot/System.map-6.6.0-g623393c9d50c ]; then 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:cp /boot/vmlinuz-6.6.0-g623393c9d50c /boot/.vmlinuz-6.6.0-g623393c9d50c-rpm 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:cp /boot/System.map-6.6.0-g623393c9d50c /boot/.System.map-6.6.0-g623393c9d50c-rpm 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:rm -f /boot/vmlinuz-6.6.0-g623393c9d50c /boot/System.map-6.6.0-g623393c9d50c 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/installkernel 6.6.0-g623393c9d50c /boot/.vmlinuz-6.6.0-g623393c9d50c-rpm /boot/.System.map-6.6.0-g623393c9d50c-rpm 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:rm -f /boot/.vmlinuz-6.6.0-g623393c9d50c-rpm /boot/.System.map-6.6.0-g623393c9d50c-rpm 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:new-kernel-pkg --remove 6.6.0-g623393c9d50c --rminitrd --initrdfile=/boot/initramfs-6.6.0-g623393c9d50c.img 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-install remove 6.6.0-g623393c9d50c 2023-12-26T21:22:30.671 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2023-12-26T21:22:30.672 INFO:teuthology.orchestra.run.smithi131.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-26T21:22:30.672 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-26T21:22:30.672 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/update-bootloader --remove 6.6.0-g623393c9d50c 2023-12-26T21:22:30.672 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2023-12-26T21:22:30.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2023-12-26T21:22:30.746 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.02 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 123.el8 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Group : System Environment/Base 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 9265450 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Relocations : (not relocatable) 2023-12-26T21:22:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : Red Hat, Inc. 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : Red Hat, Inc. 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:22:30.748 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2023-12-26T21:22:30.749 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-26T21:22:30.749 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 6.6.0-g623393c9d50c 2023-12-26T21:22:30.749 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-26T21:22:31.475 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2023-12-26T21:22:33.819 INFO:teuthology.orchestra.run.smithi131.stderr:done 2023-12-26T21:22:33.821 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2023-12-26T21:22:33.838 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.wc1LZ0xJ3A 2023-12-26T21:22:33.838 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wc1LZ0xJ3A 2023-12-26T21:22:33.913 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.wc1LZ0xJ3A 2023-12-26T21:22:34.064 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.wc1LZ0xJ3A is 6KB 2023-12-26T21:22:34.112 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.wc1LZ0xJ3A 2023-12-26T21:22:34.128 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2023-12-26T21:22:34.207 INFO:teuthology.orchestra.run.smithi131.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2023-12-26T21:22:34.207 INFO:teuthology.orchestra.run.smithi131.stdout:2db68f91c8f90721a6596efe657a5324-6.6.0-g623393c9d50c.conf 2023-12-26T21:22:34.207 INFO:teuthology.orchestra.run.smithi131.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2023-12-26T21:22:34.207 INFO:teuthology.orchestra.run.smithi131.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2023-12-26T21:22:34.207 INFO:teuthology.orchestra.run.smithi131.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2023-12-26T21:22:34.208 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.6.0-g623393c9d50c 2023-12-26T21:22:34.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2023-12-26T21:22:37.941 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-12-26T21:22:37.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-12-26T21:22:37.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:22:56.402 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-26T21:23:04.377 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-12-26T21:23:04.378 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-12-26T21:23:04.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:23:05.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-12-26T21:23:05.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:23:08.469 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-26T21:23:20.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-12-26T21:23:20.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:23:22.769 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-12-26T21:23:31.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-12-26T21:23:31.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:23:52.392 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-12-26T21:23:52.805 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-12-26T21:23:52.805 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-g623393c9d50c"... 2023-12-26T21:23:52.805 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2023-12-26T21:23:52.846 INFO:teuthology.orchestra.run.smithi190.stdout:6.6.0-g623393c9d50c 2023-12-26T21:23:52.847 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g623393c9d50c vs 6.6.0-g623393c9d50c 2023-12-26T21:23:52.847 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-26T21:23:52.847 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-12-26T21:23:53.848 DEBUG:teuthology.parallel:result is None 2023-12-26T21:24:31.780 DEBUG:teuthology.orchestra.remote:timed out 2023-12-26T21:24:43.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-12-26T21:24:43.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:24:44.176 DEBUG:teuthology.orchestra.run.smithi131:> true 2023-12-26T21:24:44.598 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-12-26T21:24:44.598 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g623393c9d50c"... 2023-12-26T21:24:44.598 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2023-12-26T21:24:44.640 INFO:teuthology.orchestra.run.smithi131.stdout:6.6.0-g623393c9d50c 2023-12-26T21:24:44.641 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g623393c9d50c vs 6.6.0-g623393c9d50c 2023-12-26T21:24:44.641 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-26T21:24:44.641 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-12-26T21:24:45.642 DEBUG:teuthology.parallel:result is None 2023-12-26T21:24:45.642 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-26T21:24:45.650 INFO:teuthology.task.internal:Creating test directory... 2023-12-26T21:24:45.651 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-26T21:24:45.656 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-26T21:24:45.706 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-26T21:24:45.712 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-26T21:24:45.800 INFO:teuthology.task.internal:Creating archive directory... 2023-12-26T21:24:45.800 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-26T21:24:45.804 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-26T21:24:45.955 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-26T21:24:45.962 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-26T21:24:45.962 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-26T21:24:45.966 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-26T21:24:46.105 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-26T21:24:46.135 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-26T21:24:46.142 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-26T21:24:46.173 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-26T21:24:46.176 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-26T21:24:46.183 INFO:teuthology.task.internal:Configuring sudo... 2023-12-26T21:24:46.183 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-26T21:24:46.187 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-26T21:24:46.256 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-26T21:24:46.267 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-26T21:24:46.267 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-26T21:24:46.271 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-26T21:24:46.313 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-26T21:24:46.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-26T21:24:46.504 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-26T21:24:46.576 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-26T21:24:46.643 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:24:46.643 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-26T21:24:46.711 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-26T21:24:46.784 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-26T21:24:46.866 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-26T21:24:46.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-26T21:24:47.019 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:24:47.019 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-26T21:24:47.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2023-12-26T21:24:47.091 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2023-12-26T21:24:47.193 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-26T21:24:47.234 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-26T21:24:47.682 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-26T21:24:47.690 INFO:teuthology.task.internal:Starting timer... 2023-12-26T21:24:47.690 INFO:teuthology.run_tasks:Running task pcp... 2023-12-26T21:24:47.700 INFO:teuthology.run_tasks:Running task selinux... 2023-12-26T21:24:47.708 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-12-26T21:24:47.709 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2023-12-26T21:24:47.846 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2023-12-26T21:24:47.849 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2023-12-26T21:24:47.984 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2023-12-26T21:24:47.987 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-26T21:24:47.987 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2023-12-26T21:24:48.035 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2023-12-26T21:24:48.035 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2023-12-26T21:24:48.081 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2023-12-26T21:24:48.081 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2023-12-26T21:24:48.081 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-26T21:24:48.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T21:24:48.151 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-26T21:24:48.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T21:24:48.219 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-26T21:24:48.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2023-12-26T21:24:48.287 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2023-12-26T21:24:48.354 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-26T21:24:48.365 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-26T21:24:48.370 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2023-12-26T21:24:48.496 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-26T21:24:48.513 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-26T21:24:48.514 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-26T21:32:39.002 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2023-12-26T21:32:39.003 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-12-26T21:32:39.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:32:39.088 DEBUG:teuthology.orchestra.run.smithi131:> true 2023-12-26T21:32:39.203 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-12-26T21:32:39.203 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-12-26T21:32:39.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-26T21:32:39.286 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-12-26T21:32:39.407 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-12-26T21:32:39.407 INFO:teuthology.run_tasks:Running task clock... 2023-12-26T21:32:39.418 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-26T21:32:39.418 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-26T21:32:39.418 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-26T21:32:39.422 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-26T21:32:39.423 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-26T21:32:39.491 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-26T21:32:39.518 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-26T21:32:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-26T21:32:39.552 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-26T21:32:39.619 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2023-12-26T21:32:39.639 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2023-12-26T21:32:39.647 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2023-12-26T21:32:39.668 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2023-12-26T21:32:39.681 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-26T21:32:39.702 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-26T21:32:39.714 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-26T21:32:39.735 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-26T21:32:39.801 INFO:teuthology.orchestra.run.smithi131.stderr:bash: ntpq: command not found 2023-12-26T21:32:39.808 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-26T21:32:39.808 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2023-12-26T21:32:39.808 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T21:32:39.808 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T21:32:39.809 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T21:32:39.809 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T21:32:39.827 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2023-12-26T21:32:39.834 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-26T21:32:39.835 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-12-26T21:32:39.835 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T21:32:39.835 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T21:32:39.835 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T21:32:39.835 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T21:32:39.836 INFO:teuthology.run_tasks:Running task install... 2023-12-26T21:32:39.845 DEBUG:teuthology.task.install:project ceph 2023-12-26T21:32:39.845 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30'}} 2023-12-26T21:32:39.845 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30'} 2023-12-26T21:32:39.845 INFO:teuthology.task.install:Using flavor: default 2023-12-26T21:32:39.851 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-26T21:32:39.851 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-26T21:32:39.852 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30', 'tag': None, 'wait_for_package': False} 2023-12-26T21:32:39.852 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5b5a60d2d510bfaa9213c41e32227fb008245a30 2023-12-26T21:32:39.854 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30', 'tag': None, 'wait_for_package': False} 2023-12-26T21:32:39.854 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5b5a60d2d510bfaa9213c41e32227fb008245a30 2023-12-26T21:32:40.000 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-12-26-0957-quincy/5b5a60d2d510bfaa9213c41e32227fb008245a30/centos/8/flavors/default/ 2023-12-26T21:32:40.000 INFO:teuthology.task.install.rpm:Package version is 17.2.7-201.g5b5a60d2 2023-12-26T21:32:40.004 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-12-26-0957-quincy/5b5a60d2d510bfaa9213c41e32227fb008245a30/centos/8/flavors/default/ 2023-12-26T21:32:40.004 INFO:teuthology.task.install.rpm:Package version is 17.2.7-201.g5b5a60d2 2023-12-26T21:32:40.175 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-12-26-0957-quincy/5b5a60d2d510bfaa9213c41e32227fb008245a30/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-12-26-0957-quincy/5b5a60d2d510bfaa9213c41e32227fb008245a30/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-12-26-0957-quincy/5b5a60d2d510bfaa9213c41e32227fb008245a30/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-26T21:32:40.176 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:32:40.176 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-26T21:32:40.179 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-12-26-0957-quincy/5b5a60d2d510bfaa9213c41e32227fb008245a30/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-12-26-0957-quincy/5b5a60d2d510bfaa9213c41e32227fb008245a30/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-12-26-0957-quincy/5b5a60d2d510bfaa9213c41e32227fb008245a30/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-26T21:32:40.179 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:32:40.179 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-26T21:32:40.245 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-12-26T21:32:40.245 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5b5a60d2d510bfaa9213c41e32227fb008245a30/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-26T21:32:40.247 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-12-26T21:32:40.247 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5b5a60d2d510bfaa9213c41e32227fb008245a30/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-26T21:32:40.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-26T21:32:40.356 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-26T21:32:40.454 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-26T21:32:40.454 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-26T21:32:40.521 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2023-12-26T21:32:40.521 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2023-12-26T21:32:40.525 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2023-12-26T21:32:40.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2023-12-26T21:32:40.974 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-12-26T21:32:40.979 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-12-26T21:32:42.395 INFO:teuthology.orchestra.run.smithi131.stdout:52 files removed 2023-12-26T21:32:42.425 INFO:teuthology.orchestra.run.smithi190.stdout:52 files removed 2023-12-26T21:32:42.452 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-12-26T21:32:42.481 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-12-26T21:32:42.905 INFO:teuthology.orchestra.run.smithi131.stdout:Updating Subscription Management repositories. 2023-12-26T21:32:42.933 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-12-26T21:32:44.767 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 260 kB/s | 78 kB 00:00 2023-12-26T21:32:44.911 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 195 kB/s | 78 kB 00:00 2023-12-26T21:32:45.076 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 34 kB/s | 9.2 kB 00:00 2023-12-26T21:32:45.236 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 32 kB/s | 9.2 kB 00:00 2023-12-26T21:32:45.401 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 4.8 kB/s | 1.4 kB 00:00 2023-12-26T21:32:45.529 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 5.4 kB/s | 1.4 kB 00:00 2023-12-26T21:32:45.586 INFO:teuthology.orchestra.run.smithi131.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-12-26T21:32:45.671 INFO:teuthology.orchestra.run.smithi190.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-12-26T21:32:49.397 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 16 MB 00:03 2023-12-26T21:32:53.697 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 122 kB/s | 24 kB 00:00 2023-12-26T21:32:55.791 INFO:teuthology.orchestra.run.smithi190.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 MB/s | 58 MB 00:02 2023-12-26T21:32:56.627 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 16 MB 00:11 2023-12-26T21:33:00.752 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 401 kB/s | 24 kB 00:00 2023-12-26T21:33:02.488 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 MB/s | 58 MB 00:01 2023-12-26T21:33:06.824 INFO:teuthology.orchestra.run.smithi190.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2023-12-26T21:33:10.043 INFO:teuthology.orchestra.run.smithi190.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 64 MB 00:01 2023-12-26T21:33:13.446 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2023-12-26T21:33:16.773 INFO:teuthology.orchestra.run.smithi131.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 MB/s | 64 MB 00:01 2023-12-26T21:33:25.059 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-26T21:33:25.060 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-26T21:33:25.078 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-26T21:33:25.080 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-26T21:33:25.082 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-26T21:33:25.083 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-26T21:33:25.086 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-26T21:33:25.091 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-26T21:33:25.092 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-26T21:33:25.302 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-12-26T21:33:25.316 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-12-26T21:33:25.316 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2023-12-26T21:33:25.316 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-12-26T21:33:25.316 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2023-12-26T21:33:25.316 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-26T21:33:25.316 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-26T21:33:25.317 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 6.4 k 2023-12-26T21:33:25.317 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 5.2 M 2023-12-26T21:33:25.317 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 875 k 2023-12-26T21:33:25.317 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 162 k 2023-12-26T21:33:25.317 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 1.6 M 2023-12-26T21:33:25.317 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 121 k 2023-12-26T21:33:25.317 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 1.7 M 2023-12-26T21:33:25.317 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 7.4 M 2023-12-26T21:33:25.317 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 47 k 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 17 M 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 52 M 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 267 k 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 80 k 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 255 k 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 18 k 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 25 k 2023-12-26T21:33:25.318 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 796 k 2023-12-26T21:33:25.319 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 130 k 2023-12-26T21:33:25.319 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-26T21:33:25.319 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-26T21:33:25.319 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-26T21:33:25.319 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 190 k 2023-12-26T21:33:25.319 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 370 k 2023-12-26T21:33:25.319 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 358 k 2023-12-26T21:33:25.319 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 107 k 2023-12-26T21:33:25.319 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 90 k 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 3.5 M 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 174 k 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-26T21:33:25.320 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 3.5 M 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 3.6 M 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2023-12-26T21:33:25.321 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 23 M 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 23 k 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 2.3 M 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 238 k 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 3.9 M 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 17 M 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 14 k 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 24 k 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-26T21:33:25.322 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 174 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 507 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 5.6 M 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-26T21:33:25.323 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-26T21:33:25.324 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-26T21:33:25.324 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-26T21:33:25.324 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-26T21:33:25.324 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-26T21:33:25.324 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-26T21:33:25.324 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-26T21:33:25.324 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-26T21:33:25.324 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-26T21:33:25.324 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 45 k 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 115 k 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-26T21:33:25.325 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-26T21:33:25.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-26T21:33:25.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-26T21:33:25.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-26T21:33:25.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-26T21:33:25.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-26T21:33:25.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-26T21:33:25.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-26T21:33:25.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-26T21:33:25.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-26T21:33:25.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-26T21:33:25.328 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-26T21:33:25.328 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-26T21:33:25.328 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-26T21:33:25.328 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-26T21:33:25.328 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-26T21:33:25.328 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-26T21:33:25.328 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-26T21:33:25.328 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-26T21:33:25.328 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-26T21:33:25.329 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-26T21:33:25.329 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-26T21:33:25.329 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-26T21:33:25.329 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-26T21:33:25.329 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-26T21:33:25.329 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-26T21:33:25.329 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-26T21:33:25.329 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-26T21:33:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-26T21:33:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-26T21:33:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-26T21:33:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-26T21:33:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-26T21:33:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-26T21:33:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-26T21:33:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-26T21:33:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-26T21:33:25.331 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-26T21:33:25.331 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:33:25.331 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2023-12-26T21:33:25.331 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-12-26T21:33:25.331 INFO:teuthology.orchestra.run.smithi190.stdout:Install 124 Packages 2023-12-26T21:33:25.331 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 16 Packages 2023-12-26T21:33:25.331 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:33:25.332 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 212 M 2023-12-26T21:33:25.332 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2023-12-26T21:33:25.884 INFO:teuthology.orchestra.run.smithi190.stdout:(1/140): ceph-17.2.7-201.g5b5a60d2.el8.x86_64.r 32 kB/s | 6.4 kB 00:00 2023-12-26T21:33:26.218 INFO:teuthology.orchestra.run.smithi190.stdout:(2/140): ceph-fuse-17.2.7-201.g5b5a60d2.el8.x86 2.6 MB/s | 875 kB 00:00 2023-12-26T21:33:26.326 INFO:teuthology.orchestra.run.smithi190.stdout:(3/140): ceph-immutable-object-cache-17.2.7-201 1.5 MB/s | 162 kB 00:00 2023-12-26T21:33:26.660 INFO:teuthology.orchestra.run.smithi190.stdout:(4/140): ceph-base-17.2.7-201.g5b5a60d2.el8.x86 5.3 MB/s | 5.2 MB 00:00 2023-12-26T21:33:26.710 INFO:teuthology.orchestra.run.smithi190.stdout:(5/140): ceph-mds-17.2.7-201.g5b5a60d2.el8.x86_ 6.0 MB/s | 2.3 MB 00:00 2023-12-26T21:33:26.877 INFO:teuthology.orchestra.run.smithi190.stdout:(6/140): ceph-mgr-17.2.7-201.g5b5a60d2.el8.x86_ 7.2 MB/s | 1.6 MB 00:00 2023-12-26T21:33:27.252 INFO:teuthology.orchestra.run.smithi190.stdout:(7/140): ceph-mon-17.2.7-201.g5b5a60d2.el8.x86_ 7.3 MB/s | 3.9 MB 00:00 2023-12-26T21:33:28.495 INFO:teuthology.orchestra.run.smithi190.stdout:(8/140): ceph-common-17.2.7-201.g5b5a60d2.el8.x 8.3 MB/s | 23 MB 00:02 2023-12-26T21:33:28.695 INFO:teuthology.orchestra.run.smithi190.stdout:(9/140): ceph-osd-17.2.7-201.g5b5a60d2.el8.x86_ 9.2 MB/s | 17 MB 00:01 2023-12-26T21:33:28.720 INFO:teuthology.orchestra.run.smithi190.stdout:(10/140): ceph-selinux-17.2.7-201.g5b5a60d2.el8 108 kB/s | 24 kB 00:00 2023-12-26T21:33:28.804 INFO:teuthology.orchestra.run.smithi190.stdout:(11/140): cephfs-mirror-17.2.7-201.g5b5a60d2.el 3.0 MB/s | 255 kB 00:00 2023-12-26T21:33:28.879 INFO:teuthology.orchestra.run.smithi190.stdout:(12/140): libcephfs-devel-17.2.7-201.g5b5a60d2. 332 kB/s | 25 kB 00:00 2023-12-26T21:33:29.029 INFO:teuthology.orchestra.run.smithi190.stdout:(13/140): libcephfs2-17.2.7-201.g5b5a60d2.el8.x 5.2 MB/s | 796 kB 00:00 2023-12-26T21:33:29.121 INFO:teuthology.orchestra.run.smithi190.stdout:(14/140): libcephsqlite-17.2.7-201.g5b5a60d2.el 1.9 MB/s | 174 kB 00:00 2023-12-26T21:33:29.213 INFO:teuthology.orchestra.run.smithi190.stdout:(15/140): librados-devel-17.2.7-201.g5b5a60d2.e 1.4 MB/s | 130 kB 00:00 2023-12-26T21:33:29.405 INFO:teuthology.orchestra.run.smithi190.stdout:(16/140): ceph-radosgw-17.2.7-201.g5b5a60d2.el8 7.8 MB/s | 17 MB 00:02 2023-12-26T21:33:29.439 INFO:teuthology.orchestra.run.smithi190.stdout:(17/140): libradosstriper1-17.2.7-201.g5b5a60d2 2.2 MB/s | 507 kB 00:00 2023-12-26T21:33:29.497 INFO:teuthology.orchestra.run.smithi190.stdout:(18/140): python3-ceph-argparse-17.2.7-201.g5b5 771 kB/s | 45 kB 00:00 2023-12-26T21:33:29.581 INFO:teuthology.orchestra.run.smithi190.stdout:(19/140): python3-ceph-common-17.2.7-201.g5b5a6 1.4 MB/s | 115 kB 00:00 2023-12-26T21:33:29.664 INFO:teuthology.orchestra.run.smithi190.stdout:(20/140): python3-cephfs-17.2.7-201.g5b5a60d2.e 2.2 MB/s | 190 kB 00:00 2023-12-26T21:33:29.765 INFO:teuthology.orchestra.run.smithi190.stdout:(21/140): python3-rados-17.2.7-201.g5b5a60d2.el 3.6 MB/s | 370 kB 00:00 2023-12-26T21:33:29.865 INFO:teuthology.orchestra.run.smithi190.stdout:(22/140): python3-rbd-17.2.7-201.g5b5a60d2.el8. 3.5 MB/s | 358 kB 00:00 2023-12-26T21:33:29.966 INFO:teuthology.orchestra.run.smithi190.stdout:(23/140): python3-rgw-17.2.7-201.g5b5a60d2.el8. 1.0 MB/s | 107 kB 00:00 2023-12-26T21:33:30.099 INFO:teuthology.orchestra.run.smithi190.stdout:(24/140): librgw2-17.2.7-201.g5b5a60d2.el8.x86_ 8.0 MB/s | 5.6 MB 00:00 2023-12-26T21:33:30.125 INFO:teuthology.orchestra.run.smithi190.stdout:(25/140): rbd-fuse-17.2.7-201.g5b5a60d2.el8.x86 569 kB/s | 90 kB 00:00 2023-12-26T21:33:30.208 INFO:teuthology.orchestra.run.smithi190.stdout:(26/140): rbd-nbd-17.2.7-201.g5b5a60d2.el8.x86_ 2.0 MB/s | 174 kB 00:00 2023-12-26T21:33:30.292 INFO:teuthology.orchestra.run.smithi190.stdout:(27/140): ceph-grafana-dashboards-17.2.7-201.g5 277 kB/s | 23 kB 00:00 2023-12-26T21:33:30.375 INFO:teuthology.orchestra.run.smithi190.stdout:(28/140): ceph-mgr-cephadm-17.2.7-201.g5b5a60d2 1.4 MB/s | 121 kB 00:00 2023-12-26T21:33:30.559 INFO:teuthology.orchestra.run.smithi190.stdout:(29/140): rbd-mirror-17.2.7-201.g5b5a60d2.el8.x 7.5 MB/s | 3.5 MB 00:00 2023-12-26T21:33:30.610 INFO:teuthology.orchestra.run.smithi190.stdout:(30/140): ceph-mgr-dashboard-17.2.7-201.g5b5a60 7.3 MB/s | 1.7 MB 00:00 2023-12-26T21:33:30.710 INFO:teuthology.orchestra.run.smithi190.stdout:(31/140): ceph-mgr-modules-core-17.2.7-201.g5b5 2.3 MB/s | 238 kB 00:00 2023-12-26T21:33:30.810 INFO:teuthology.orchestra.run.smithi190.stdout:(32/140): ceph-mgr-rook-17.2.7-201.g5b5a60d2.el 474 kB/s | 47 kB 00:00 2023-12-26T21:33:30.894 INFO:teuthology.orchestra.run.smithi190.stdout:(33/140): ceph-prometheus-alerts-17.2.7-201.g5b 174 kB/s | 14 kB 00:00 2023-12-26T21:33:30.986 INFO:teuthology.orchestra.run.smithi190.stdout:(34/140): ceph-volume-17.2.7-201.g5b5a60d2.el8. 2.9 MB/s | 267 kB 00:00 2023-12-26T21:33:31.070 INFO:teuthology.orchestra.run.smithi190.stdout:(35/140): cephadm-17.2.7-201.g5b5a60d2.el8.noar 964 kB/s | 80 kB 00:00 2023-12-26T21:33:31.153 INFO:teuthology.orchestra.run.smithi190.stdout:(36/140): cephfs-top-17.2.7-201.g5b5a60d2.el8.n 217 kB/s | 18 kB 00:00 2023-12-26T21:33:31.328 INFO:teuthology.orchestra.run.smithi190.stdout:(37/140): ceph-mgr-diskprediction-local-17.2.7- 9.6 MB/s | 7.4 MB 00:00 2023-12-26T21:33:31.437 INFO:teuthology.orchestra.run.smithi190.stdout:(38/140): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-12-26T21:33:31.462 INFO:teuthology.orchestra.run.smithi190.stdout:(39/140): liboath-2.6.2-3.el8.x86_64.rpm 441 kB/s | 59 kB 00:00 2023-12-26T21:33:31.487 INFO:teuthology.orchestra.run.smithi190.stdout:(40/140): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2023-12-26T21:33:31.529 INFO:teuthology.orchestra.run.smithi190.stdout:(41/140): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-12-26T21:33:31.605 INFO:teuthology.orchestra.run.smithi190.stdout:(42/140): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2023-12-26T21:33:31.639 INFO:teuthology.orchestra.run.smithi190.stdout:(43/140): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-12-26T21:33:31.689 INFO:teuthology.orchestra.run.smithi190.stdout:(44/140): python3-cachetools-3.1.1-4.el8.noarch 394 kB/s | 33 kB 00:00 2023-12-26T21:33:31.706 INFO:teuthology.orchestra.run.smithi190.stdout:(45/140): python3-certifi-2018.10.15-7.el8.noar 243 kB/s | 16 kB 00:00 2023-12-26T21:33:31.748 INFO:teuthology.orchestra.run.smithi190.stdout:(46/140): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-12-26T21:33:31.806 INFO:teuthology.orchestra.run.smithi190.stdout:(47/140): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-12-26T21:33:31.840 INFO:teuthology.orchestra.run.smithi190.stdout:(48/140): python3-jaraco-6.2-6.el8.noarch.rpm 334 kB/s | 11 kB 00:00 2023-12-26T21:33:31.865 INFO:teuthology.orchestra.run.smithi190.stdout:(49/140): python3-cherrypy-18.4.0-1.el8.noarch. 2.4 MB/s | 384 kB 00:00 2023-12-26T21:33:31.891 INFO:teuthology.orchestra.run.smithi190.stdout:(50/140): python3-jaraco-functools-2.0-4.el8.no 349 kB/s | 18 kB 00:00 2023-12-26T21:33:32.131 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-26T21:33:32.132 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-26T21:33:32.151 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-26T21:33:32.153 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-26T21:33:32.155 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-26T21:33:32.156 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-26T21:33:32.159 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-26T21:33:32.164 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-26T21:33:32.165 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-26T21:33:32.358 INFO:teuthology.orchestra.run.smithi190.stdout:(51/140): ceph-test-17.2.7-201.g5b5a60d2.el8.x8 14 MB/s | 52 MB 00:03 2023-12-26T21:33:32.380 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2023-12-26T21:33:32.383 INFO:teuthology.orchestra.run.smithi190.stdout:(52/140): python3-logutils-0.3.5-11.el8.noarch. 99 kB/s | 49 kB 00:00 2023-12-26T21:33:32.394 INFO:teuthology.orchestra.run.smithi131.stdout:=========================================================================================================================== 2023-12-26T21:33:32.394 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2023-12-26T21:33:32.394 INFO:teuthology.orchestra.run.smithi131.stdout:=========================================================================================================================== 2023-12-26T21:33:32.394 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2023-12-26T21:33:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-26T21:33:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-26T21:33:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 6.4 k 2023-12-26T21:33:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 5.2 M 2023-12-26T21:33:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 875 k 2023-12-26T21:33:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 162 k 2023-12-26T21:33:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 1.6 M 2023-12-26T21:33:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 121 k 2023-12-26T21:33:32.395 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 1.7 M 2023-12-26T21:33:32.396 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 7.4 M 2023-12-26T21:33:32.396 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 47 k 2023-12-26T21:33:32.396 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 17 M 2023-12-26T21:33:32.396 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 52 M 2023-12-26T21:33:32.396 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 267 k 2023-12-26T21:33:32.396 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 80 k 2023-12-26T21:33:32.396 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 255 k 2023-12-26T21:33:32.396 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 18 k 2023-12-26T21:33:32.396 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-26T21:33:32.397 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-26T21:33:32.397 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-26T21:33:32.397 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 25 k 2023-12-26T21:33:32.397 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 796 k 2023-12-26T21:33:32.397 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 130 k 2023-12-26T21:33:32.397 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-26T21:33:32.397 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-26T21:33:32.397 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-26T21:33:32.398 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 190 k 2023-12-26T21:33:32.398 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 370 k 2023-12-26T21:33:32.398 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 358 k 2023-12-26T21:33:32.398 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 107 k 2023-12-26T21:33:32.398 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 90 k 2023-12-26T21:33:32.398 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 3.5 M 2023-12-26T21:33:32.398 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 174 k 2023-12-26T21:33:32.398 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 3.5 M 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 3.6 M 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2023-12-26T21:33:32.399 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 23 M 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 23 k 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 2.3 M 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 238 k 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 3.9 M 2023-12-26T21:33:32.400 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 17 M 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:17.2.7-201.g5b5a60d2.el8 ceph-noarch 14 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 24 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 174 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-26T21:33:32.401 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 507 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 5.6 M 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-26T21:33:32.402 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 45 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:17.2.7-201.g5b5a60d2.el8 ceph 115 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-26T21:33:32.403 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-26T21:33:32.404 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-26T21:33:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-26T21:33:32.406 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout:=========================================================================================================================== 2023-12-26T21:33:32.407 INFO:teuthology.orchestra.run.smithi131.stdout:Install 124 Packages 2023-12-26T21:33:32.408 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 16 Packages 2023-12-26T21:33:32.408 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:33:32.408 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 212 M 2023-12-26T21:33:32.408 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2023-12-26T21:33:32.442 INFO:teuthology.orchestra.run.smithi190.stdout:(53/140): python3-kubernetes-11.0.0-6.el8.noarc 2.0 MB/s | 1.2 MB 00:00 2023-12-26T21:33:32.467 INFO:teuthology.orchestra.run.smithi190.stdout:(54/140): python3-natsort-7.1.1-2.el8.noarch.rp 764 kB/s | 60 kB 00:00 2023-12-26T21:33:32.509 INFO:teuthology.orchestra.run.smithi190.stdout:(55/140): python3-portend-2.6-1.el8.noarch.rpm 376 kB/s | 16 kB 00:00 2023-12-26T21:33:32.543 INFO:teuthology.orchestra.run.smithi190.stdout:(56/140): python3-more-itertools-7.2.0-3.el8.no 319 kB/s | 59 kB 00:00 2023-12-26T21:33:32.576 INFO:teuthology.orchestra.run.smithi190.stdout:(57/140): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-12-26T21:33:32.602 INFO:teuthology.orchestra.run.smithi190.stdout:(58/140): python3-repoze-lru-0.7-6.el8.noarch.r 365 kB/s | 34 kB 00:00 2023-12-26T21:33:32.605 INFO:teuthology.orchestra.run.smithi131.stdout:(1/140): ceph-17.2.7-201.g5b5a60d2.el8.x86_64.r 49 kB/s | 6.4 kB 00:00 2023-12-26T21:33:32.627 INFO:teuthology.orchestra.run.smithi190.stdout:(59/140): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2023-12-26T21:33:32.652 INFO:teuthology.orchestra.run.smithi190.stdout:(60/140): python3-rsa-4.9-2.el8.noarch.rpm 855 kB/s | 64 kB 00:00 2023-12-26T21:33:32.677 INFO:teuthology.orchestra.run.smithi190.stdout:(61/140): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-12-26T21:33:32.703 INFO:teuthology.orchestra.run.smithi190.stdout:(62/140): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2023-12-26T21:33:32.728 INFO:teuthology.orchestra.run.smithi190.stdout:(63/140): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2023-12-26T21:33:32.753 INFO:teuthology.orchestra.run.smithi190.stdout:(64/140): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-12-26T21:33:32.796 INFO:teuthology.orchestra.run.smithi190.stdout:(65/140): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-12-26T21:33:32.817 INFO:teuthology.orchestra.run.smithi131.stdout:(2/140): ceph-fuse-17.2.7-201.g5b5a60d2.el8.x86 4.1 MB/s | 875 kB 00:00 2023-12-26T21:33:32.846 INFO:teuthology.orchestra.run.smithi190.stdout:(66/140): python3-websocket-client-0.56.0-5.el8 667 kB/s | 61 kB 00:00 2023-12-26T21:33:32.871 INFO:teuthology.orchestra.run.smithi190.stdout:(67/140): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2023-12-26T21:33:32.873 INFO:teuthology.orchestra.run.smithi131.stdout:(3/140): ceph-immutable-object-cache-17.2.7-201 2.7 MB/s | 162 kB 00:00 2023-12-26T21:33:32.896 INFO:teuthology.orchestra.run.smithi190.stdout:(68/140): python3-zc-lockfile-2.0-2.el8.noarch. 459 kB/s | 23 kB 00:00 2023-12-26T21:33:32.922 INFO:teuthology.orchestra.run.smithi190.stdout:(69/140): python3-webtest-2.0.33-1.el8.noarch.r 679 kB/s | 85 kB 00:00 2023-12-26T21:33:32.947 INFO:teuthology.orchestra.run.smithi190.stdout:(70/140): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-12-26T21:33:33.014 INFO:teuthology.orchestra.run.smithi190.stdout:(71/140): python3-jsonpointer-1.10-11.el8.noarc 218 kB/s | 20 kB 00:00 2023-12-26T21:33:33.114 INFO:teuthology.orchestra.run.smithi190.stdout:(72/140): python3-prettytable-0.7.2-14.el8.noar 273 kB/s | 44 kB 00:00 2023-12-26T21:33:33.123 INFO:teuthology.orchestra.run.smithi131.stdout:(4/140): ceph-mds-17.2.7-201.g5b5a60d2.el8.x86_ 9.2 MB/s | 2.3 MB 00:00 2023-12-26T21:33:33.223 INFO:teuthology.orchestra.run.smithi190.stdout:(73/140): python3-pyasn1-modules-0.3.7-6.el8.no 527 kB/s | 110 kB 00:00 2023-12-26T21:33:33.265 INFO:teuthology.orchestra.run.smithi190.stdout:(74/140): python3-jsonpatch-1.21-2.el8.noarch.r 176 kB/s | 27 kB 00:00 2023-12-26T21:33:33.307 INFO:teuthology.orchestra.run.smithi190.stdout:(75/140): thrift-0.13.0-2.el8.x86_64.rpm 4.0 MB/s | 1.7 MB 00:00 2023-12-26T21:33:33.315 INFO:teuthology.orchestra.run.smithi131.stdout:(5/140): ceph-mgr-17.2.7-201.g5b5a60d2.el8.x86_ 8.2 MB/s | 1.6 MB 00:00 2023-12-26T21:33:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:(76/140): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-12-26T21:33:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:(77/140): python3-werkzeug-0.12.2-4.el8.noarch. 3.3 MB/s | 457 kB 00:00 2023-12-26T21:33:33.424 INFO:teuthology.orchestra.run.smithi131.stdout:(6/140): ceph-base-17.2.7-201.g5b5a60d2.el8.x86 5.4 MB/s | 5.2 MB 00:00 2023-12-26T21:33:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:(78/140): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2023-12-26T21:33:33.508 INFO:teuthology.orchestra.run.smithi190.stdout:(79/140): bison-3.0.4-10.el8.x86_64.rpm 3.4 MB/s | 688 kB 00:00 2023-12-26T21:33:33.575 INFO:teuthology.orchestra.run.smithi190.stdout:(80/140): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2023-12-26T21:33:33.601 INFO:teuthology.orchestra.run.smithi190.stdout:(81/140): python3-markupsafe-0.23-19.el8.x86_64 309 kB/s | 39 kB 00:00 2023-12-26T21:33:33.626 INFO:teuthology.orchestra.run.smithi190.stdout:(82/140): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2023-12-26T21:33:33.733 INFO:teuthology.orchestra.run.smithi131.stdout:(7/140): ceph-mon-17.2.7-201.g5b5a60d2.el8.x86_ 9.4 MB/s | 3.9 MB 00:00 2023-12-26T21:33:33.734 INFO:teuthology.orchestra.run.smithi190.stdout:(83/140): python3-pyOpenSSL-19.0.0-1.el8.noarch 776 kB/s | 103 kB 00:00 2023-12-26T21:33:33.768 INFO:teuthology.orchestra.run.smithi190.stdout:(84/140): oniguruma-6.8.2-2.el8.x86_64.rpm 975 kB/s | 187 kB 00:00 2023-12-26T21:33:33.852 INFO:teuthology.orchestra.run.smithi190.stdout:(85/140): python3-jinja2-2.10.1-3.el8.noarch.rp 4.5 MB/s | 538 kB 00:00 2023-12-26T21:33:33.973 INFO:teuthology.orchestra.run.smithi190.stdout:(86/140): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-12-26T21:33:34.011 INFO:teuthology.orchestra.run.smithi190.stdout:(87/140): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2023-12-26T21:33:34.086 INFO:teuthology.orchestra.run.smithi190.stdout:(88/140): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-12-26T21:33:34.119 INFO:teuthology.orchestra.run.smithi190.stdout:(89/140): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2023-12-26T21:33:34.145 INFO:teuthology.orchestra.run.smithi190.stdout:(90/140): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-12-26T21:33:34.237 INFO:teuthology.orchestra.run.smithi190.stdout:(91/140): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2023-12-26T21:33:34.362 INFO:teuthology.orchestra.run.smithi190.stdout:(92/140): openblas-threads-0.3.15-6.el8.x86_64. 19 MB/s | 5.2 MB 00:00 2023-12-26T21:33:34.404 INFO:teuthology.orchestra.run.smithi190.stdout:(93/140): automake-1.16.1-8.el8.noarch.rpm 4.2 MB/s | 714 kB 00:00 2023-12-26T21:33:34.479 INFO:teuthology.orchestra.run.smithi190.stdout:(94/140): jq-1.6-7.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2023-12-26T21:33:34.555 INFO:teuthology.orchestra.run.smithi190.stdout:(95/140): openblas-0.3.15-6.el8.x86_64.rpm 12 MB/s | 5.0 MB 00:00 2023-12-26T21:33:34.580 INFO:teuthology.orchestra.run.smithi190.stdout:(96/140): python3-pytz-2017.2-11.el8.noarch.rpm 309 kB/s | 54 kB 00:00 2023-12-26T21:33:34.714 INFO:teuthology.orchestra.run.smithi190.stdout:(97/140): libverto-devel-0.3.0-5.el8.x86_64.rpm 113 kB/s | 18 kB 00:00 2023-12-26T21:33:34.739 INFO:teuthology.orchestra.run.smithi190.stdout:(98/140): python3-jwt-1.6.1-2.el8.noarch.rpm 268 kB/s | 43 kB 00:00 2023-12-26T21:33:34.931 INFO:teuthology.orchestra.run.smithi190.stdout:(99/140): python3-scipy-1.0.0-21.module+el8.9.0 30 MB/s | 14 MB 00:00 2023-12-26T21:33:34.956 INFO:teuthology.orchestra.run.smithi190.stdout:(100/140): python3-pycparser-2.14-14.el8.noarch 503 kB/s | 109 kB 00:00 2023-12-26T21:33:34.981 INFO:teuthology.orchestra.run.smithi190.stdout:(101/140): python3-oauthlib-2.1.0-1.el8.noarch. 579 kB/s | 155 kB 00:00 2023-12-26T21:33:35.074 INFO:teuthology.orchestra.run.smithi190.stdout:(102/140): python3-requests-oauthlib-1.0.0-1.el 300 kB/s | 43 kB 00:00 2023-12-26T21:33:35.090 INFO:teuthology.orchestra.run.smithi190.stdout:(103/140): libaio-devel-0.3.112-1.el8.x86_64.rp 139 kB/s | 19 kB 00:00 2023-12-26T21:33:35.137 INFO:teuthology.orchestra.run.smithi190.stdout:(104/140): libzstd-devel-1.4.4-1.el8.x86_64.rpm 283 kB/s | 44 kB 00:00 2023-12-26T21:33:35.225 INFO:teuthology.orchestra.run.smithi190.stdout:(105/140): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2023-12-26T21:33:35.283 INFO:teuthology.orchestra.run.smithi190.stdout:(106/140): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.2 MB/s | 229 kB 00:00 2023-12-26T21:33:35.308 INFO:teuthology.orchestra.run.smithi190.stdout:(107/140): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2023-12-26T21:33:35.367 INFO:teuthology.orchestra.run.smithi190.stdout:(108/140): libselinux-devel-2.9-5.el8.x86_64.rp 1.4 MB/s | 200 kB 00:00 2023-12-26T21:33:35.399 INFO:teuthology.orchestra.run.smithi131.stdout:(8/140): ceph-radosgw-17.2.7-201.g5b5a60d2.el8. 10 MB/s | 17 MB 00:01 2023-12-26T21:33:35.417 INFO:teuthology.orchestra.run.smithi190.stdout:(109/140): libsepol-devel-2.9-3.el8.x86_64.rpm 800 kB/s | 87 kB 00:00 2023-12-26T21:33:35.443 INFO:teuthology.orchestra.run.smithi190.stdout:(110/140): keyutils-libs-devel-1.5.10-9.el8.x86 302 kB/s | 48 kB 00:00 2023-12-26T21:33:35.509 INFO:teuthology.orchestra.run.smithi190.stdout:(111/140): zlib-devel-1.2.11-18.el8_5.x86_64.rp 405 kB/s | 58 kB 00:00 2023-12-26T21:33:35.560 INFO:teuthology.orchestra.run.smithi190.stdout:(112/140): libcom_err-devel-1.45.6-4.el8.x86_64 271 kB/s | 39 kB 00:00 2023-12-26T21:33:35.641 INFO:teuthology.orchestra.run.smithi131.stdout:(9/140): ceph-common-17.2.7-201.g5b5a60d2.el8.x 7.4 MB/s | 23 MB 00:03 2023-12-26T21:33:35.643 INFO:teuthology.orchestra.run.smithi190.stdout:(113/140): fuse-2.9.7-17.el8.x86_64.rpm 414 kB/s | 83 kB 00:00 2023-12-26T21:33:35.658 INFO:teuthology.orchestra.run.smithi131.stdout:(10/140): ceph-selinux-17.2.7-201.g5b5a60d2.el8 94 kB/s | 24 kB 00:00 2023-12-26T21:33:35.685 INFO:teuthology.orchestra.run.smithi190.stdout:(114/140): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2023-12-26T21:33:35.727 INFO:teuthology.orchestra.run.smithi190.stdout:(115/140): libgfortran-8.5.0-20.el8.x86_64.rpm 2.9 MB/s | 645 kB 00:00 2023-12-26T21:33:35.778 INFO:teuthology.orchestra.run.smithi190.stdout:(116/140): libuuid-devel-2.32.1-43.el8.x86_64.r 745 kB/s | 100 kB 00:00 2023-12-26T21:33:35.834 INFO:teuthology.orchestra.run.smithi131.stdout:(11/140): ceph-osd-17.2.7-201.g5b5a60d2.el8.x86 6.9 MB/s | 17 MB 00:02 2023-12-26T21:33:35.853 INFO:teuthology.orchestra.run.smithi190.stdout:(117/140): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.4 MB/s | 238 kB 00:00 2023-12-26T21:33:35.859 INFO:teuthology.orchestra.run.smithi131.stdout:(12/140): cephfs-mirror-17.2.7-201.g5b5a60d2.el 1.2 MB/s | 255 kB 00:00 2023-12-26T21:33:35.878 INFO:teuthology.orchestra.run.smithi190.stdout:(118/140): python3-cryptography-3.2.1-6.el8.x86 3.6 MB/s | 559 kB 00:00 2023-12-26T21:33:35.892 INFO:teuthology.orchestra.run.smithi131.stdout:(13/140): libcephfs-devel-17.2.7-201.g5b5a60d2. 428 kB/s | 25 kB 00:00 2023-12-26T21:33:35.929 INFO:teuthology.orchestra.run.smithi190.stdout:(119/140): xfsprogs-devel-5.0.0-12.el8.x86_64.r 482 kB/s | 73 kB 00:00 2023-12-26T21:33:35.951 INFO:teuthology.orchestra.run.smithi131.stdout:(14/140): libcephfs2-17.2.7-201.g5b5a60d2.el8.x 8.5 MB/s | 796 kB 00:00 2023-12-26T21:33:35.976 INFO:teuthology.orchestra.run.smithi131.stdout:(15/140): libcephsqlite-17.2.7-201.g5b5a60d2.el 2.0 MB/s | 174 kB 00:00 2023-12-26T21:33:35.979 INFO:teuthology.orchestra.run.smithi190.stdout:(120/140): NetworkManager-initscripts-updown-1. 1.1 MB/s | 143 kB 00:00 2023-12-26T21:33:36.010 INFO:teuthology.orchestra.run.smithi131.stdout:(16/140): librados-devel-17.2.7-201.g5b5a60d2.e 2.2 MB/s | 130 kB 00:00 2023-12-26T21:33:36.029 INFO:teuthology.orchestra.run.smithi190.stdout:(121/140): elfutils-libelf-devel-0.189-3.el8.x8 410 kB/s | 62 kB 00:00 2023-12-26T21:33:36.071 INFO:teuthology.orchestra.run.smithi190.stdout:(122/140): librabbitmq-0.9.0-4.el8.x86_64.rpm 331 kB/s | 47 kB 00:00 2023-12-26T21:33:36.113 INFO:teuthology.orchestra.run.smithi190.stdout:(123/140): krb5-devel-1.18.2-26.el8_9.x86_64.rp 4.1 MB/s | 562 kB 00:00 2023-12-26T21:33:36.127 INFO:teuthology.orchestra.run.smithi131.stdout:(17/140): libradosstriper1-17.2.7-201.g5b5a60d2 3.3 MB/s | 507 kB 00:00 2023-12-26T21:33:36.219 INFO:teuthology.orchestra.run.smithi131.stdout:(18/140): python3-ceph-argparse-17.2.7-201.g5b5 489 kB/s | 45 kB 00:00 2023-12-26T21:33:36.230 INFO:teuthology.orchestra.run.smithi190.stdout:(124/140): openssl-devel-1.1.1k-12.el8_9.x86_64 12 MB/s | 2.3 MB 00:00 2023-12-26T21:33:36.311 INFO:teuthology.orchestra.run.smithi131.stdout:(19/140): python3-ceph-common-17.2.7-201.g5b5a6 1.2 MB/s | 115 kB 00:00 2023-12-26T21:33:36.388 INFO:teuthology.orchestra.run.smithi190.stdout:(125/140): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2023-12-26T21:33:36.411 INFO:teuthology.orchestra.run.smithi131.stdout:(20/140): python3-cephfs-17.2.7-201.g5b5a60d2.e 1.9 MB/s | 190 kB 00:00 2023-12-26T21:33:36.537 INFO:teuthology.orchestra.run.smithi131.stdout:(21/140): librgw2-17.2.7-201.g5b5a60d2.el8.x86_ 11 MB/s | 5.6 MB 00:00 2023-12-26T21:33:36.555 INFO:teuthology.orchestra.run.smithi190.stdout:(126/140): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2023-12-26T21:33:36.570 INFO:teuthology.orchestra.run.smithi131.stdout:(22/140): python3-rados-17.2.7-201.g5b5a60d2.el 2.3 MB/s | 370 kB 00:00 2023-12-26T21:33:36.620 INFO:teuthology.orchestra.run.smithi131.stdout:(23/140): python3-rbd-17.2.7-201.g5b5a60d2.el8. 4.2 MB/s | 358 kB 00:00 2023-12-26T21:33:36.664 INFO:teuthology.orchestra.run.smithi190.stdout:(127/140): libmount-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 237 kB 00:00 2023-12-26T21:33:36.679 INFO:teuthology.orchestra.run.smithi131.stdout:(24/140): python3-rgw-17.2.7-201.g5b5a60d2.el8. 985 kB/s | 107 kB 00:00 2023-12-26T21:33:36.713 INFO:teuthology.orchestra.run.smithi131.stdout:(25/140): rbd-fuse-17.2.7-201.g5b5a60d2.el8.x86 979 kB/s | 90 kB 00:00 2023-12-26T21:33:36.806 INFO:teuthology.orchestra.run.smithi190.stdout:(128/140): librados2-17.2.7-201.g5b5a60d2.el8.x 4.7 MB/s | 3.5 MB 00:00 2023-12-26T21:33:36.821 INFO:teuthology.orchestra.run.smithi131.stdout:(26/140): rbd-nbd-17.2.7-201.g5b5a60d2.el8.x86_ 1.6 MB/s | 174 kB 00:00 2023-12-26T21:33:36.832 INFO:teuthology.orchestra.run.smithi190.stdout:(129/140): libsmartcols-2.32.1-43.el8.x86_64.rp 1.0 MB/s | 179 kB 00:00 2023-12-26T21:33:36.915 INFO:teuthology.orchestra.run.smithi190.stdout:(130/140): libuuid-2.32.1-43.el8.x86_64.rpm 909 kB/s | 99 kB 00:00 2023-12-26T21:33:36.930 INFO:teuthology.orchestra.run.smithi131.stdout:(27/140): ceph-grafana-dashboards-17.2.7-201.g5 212 kB/s | 23 kB 00:00 2023-12-26T21:33:36.990 INFO:teuthology.orchestra.run.smithi190.stdout:(131/140): librbd1-17.2.7-201.g5b5a60d2.el8.x86 4.1 MB/s | 3.6 MB 00:00 2023-12-26T21:33:36.997 INFO:teuthology.orchestra.run.smithi131.stdout:(28/140): ceph-mgr-cephadm-17.2.7-201.g5b5a60d2 1.8 MB/s | 121 kB 00:00 2023-12-26T21:33:37.057 INFO:teuthology.orchestra.run.smithi190.stdout:(132/140): util-linux-2.32.1-43.el8.x86_64.rpm 11 MB/s | 2.5 MB 00:00 2023-12-26T21:33:37.083 INFO:teuthology.orchestra.run.smithi190.stdout:(133/140): util-linux-user-2.32.1-43.el8.x86_64 615 kB/s | 103 kB 00:00 2023-12-26T21:33:37.116 INFO:teuthology.orchestra.run.smithi190.stdout:(134/140): xfsdump-3.1.8-6.el8.x86_64.rpm 2.6 MB/s | 333 kB 00:00 2023-12-26T21:33:37.192 INFO:teuthology.orchestra.run.smithi190.stdout:(135/140): iproute-6.2.0-5.el8_9.x86_64.rpm 6.2 MB/s | 854 kB 00:00 2023-12-26T21:33:37.214 INFO:teuthology.orchestra.run.smithi131.stdout:(29/140): ceph-mgr-dashboard-17.2.7-201.g5b5a60 7.8 MB/s | 1.7 MB 00:00 2023-12-26T21:33:37.267 INFO:teuthology.orchestra.run.smithi190.stdout:(136/140): libblkid-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-12-26T21:33:37.273 INFO:teuthology.orchestra.run.smithi131.stdout:(30/140): rbd-mirror-17.2.7-201.g5b5a60d2.el8.x 5.8 MB/s | 3.5 MB 00:00 2023-12-26T21:33:37.317 INFO:teuthology.orchestra.run.smithi190.stdout:(137/140): libfdisk-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 254 kB 00:00 2023-12-26T21:33:37.359 INFO:teuthology.orchestra.run.smithi190.stdout:(138/140): openssl-1.1.1k-12.el8_9.x86_64.rpm 7.5 MB/s | 711 kB 00:00 2023-12-26T21:33:37.381 INFO:teuthology.orchestra.run.smithi131.stdout:(31/140): ceph-mgr-modules-core-17.2.7-201.g5b5 2.2 MB/s | 238 kB 00:00 2023-12-26T21:33:37.393 INFO:teuthology.orchestra.run.smithi190.stdout:(139/140): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.3 MB/s | 1.1 MB 00:00 2023-12-26T21:33:37.460 INFO:teuthology.orchestra.run.smithi190.stdout:(140/140): openssl-libs-1.1.1k-12.el8_9.x86_64. 10 MB/s | 1.5 MB 00:00 2023-12-26T21:33:37.464 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2023-12-26T21:33:37.465 INFO:teuthology.orchestra.run.smithi190.stdout:Total 17 MB/s | 212 MB 00:12 2023-12-26T21:33:37.473 INFO:teuthology.orchestra.run.smithi131.stdout:(32/140): ceph-mgr-rook-17.2.7-201.g5b5a60d2.el 516 kB/s | 47 kB 00:00 2023-12-26T21:33:37.565 INFO:teuthology.orchestra.run.smithi131.stdout:(33/140): ceph-prometheus-alerts-17.2.7-201.g5b 158 kB/s | 14 kB 00:00 2023-12-26T21:33:37.682 INFO:teuthology.orchestra.run.smithi131.stdout:(34/140): ceph-volume-17.2.7-201.g5b5a60d2.el8. 2.2 MB/s | 267 kB 00:00 2023-12-26T21:33:37.774 INFO:teuthology.orchestra.run.smithi131.stdout:(35/140): cephadm-17.2.7-201.g5b5a60d2.el8.noar 878 kB/s | 80 kB 00:00 2023-12-26T21:33:37.867 INFO:teuthology.orchestra.run.smithi131.stdout:(36/140): ceph-mgr-diskprediction-local-17.2.7- 11 MB/s | 7.4 MB 00:00 2023-12-26T21:33:37.892 INFO:teuthology.orchestra.run.smithi131.stdout:(37/140): cephfs-top-17.2.7-201.g5b5a60d2.el8.n 154 kB/s | 18 kB 00:00 2023-12-26T21:33:38.184 INFO:teuthology.orchestra.run.smithi131.stdout:(38/140): liboath-2.6.2-3.el8.x86_64.rpm 202 kB/s | 59 kB 00:00 2023-12-26T21:33:38.292 INFO:teuthology.orchestra.run.smithi131.stdout:(39/140): gperftools-libs-2.7-9.el8.x86_64.rpm 720 kB/s | 306 kB 00:00 2023-12-26T21:33:38.317 INFO:teuthology.orchestra.run.smithi131.stdout:(40/140): libunwind-1.3.1-3.el8.x86_64.rpm 563 kB/s | 75 kB 00:00 2023-12-26T21:33:38.418 INFO:teuthology.orchestra.run.smithi131.stdout:(41/140): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 436 kB/s | 44 kB 00:00 2023-12-26T21:33:38.443 INFO:teuthology.orchestra.run.smithi131.stdout:(42/140): python3-asyncssh-2.7.0-2.el8.noarch.r 3.1 MB/s | 476 kB 00:00 2023-12-26T21:33:38.518 INFO:teuthology.orchestra.run.smithi131.stdout:(43/140): python3-cachetools-3.1.1-4.el8.noarch 440 kB/s | 33 kB 00:00 2023-12-26T21:33:38.577 INFO:teuthology.orchestra.run.smithi131.stdout:(44/140): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-12-26T21:33:38.594 INFO:teuthology.orchestra.run.smithi131.stdout:(45/140): python3-certifi-2018.10.15-7.el8.noar 217 kB/s | 16 kB 00:00 2023-12-26T21:33:38.686 INFO:teuthology.orchestra.run.smithi131.stdout:(46/140): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-12-26T21:33:38.712 INFO:teuthology.orchestra.run.smithi131.stdout:(47/140): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-12-26T21:33:38.770 INFO:teuthology.orchestra.run.smithi131.stdout:(48/140): python3-google-auth-1.1.1-10.el8.noar 969 kB/s | 81 kB 00:00 2023-12-26T21:33:38.787 INFO:teuthology.orchestra.run.smithi131.stdout:(49/140): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2023-12-26T21:33:38.838 INFO:teuthology.orchestra.run.smithi131.stdout:(50/140): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-12-26T21:33:38.913 INFO:teuthology.orchestra.run.smithi131.stdout:(51/140): python3-logutils-0.3.5-11.el8.noarch. 652 kB/s | 49 kB 00:00 2023-12-26T21:33:39.005 INFO:teuthology.orchestra.run.smithi131.stdout:(52/140): python3-kubernetes-11.0.0-6.el8.noarc 5.4 MB/s | 1.2 MB 00:00 2023-12-26T21:33:39.022 INFO:teuthology.orchestra.run.smithi131.stdout:(53/140): python3-more-itertools-7.2.0-3.el8.no 539 kB/s | 59 kB 00:00 2023-12-26T21:33:39.424 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2023-12-26T21:33:39.498 INFO:teuthology.orchestra.run.smithi131.stdout:(54/140): ceph-test-17.2.7-201.g5b5a60d2.el8.x8 13 MB/s | 52 MB 00:03 2023-12-26T21:33:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:(55/140): python3-natsort-7.1.1-2.el8.noarch.rp 116 kB/s | 60 kB 00:00 2023-12-26T21:33:39.548 INFO:teuthology.orchestra.run.smithi131.stdout:(56/140): python3-pecan-1.3.2-9.el8.noarch.rpm 537 kB/s | 283 kB 00:00 2023-12-26T21:33:39.599 INFO:teuthology.orchestra.run.smithi131.stdout:(57/140): python3-repoze-lru-0.7-6.el8.noarch.r 447 kB/s | 34 kB 00:00 2023-12-26T21:33:39.641 INFO:teuthology.orchestra.run.smithi131.stdout:(58/140): python3-routes-2.4.1-12.el8.noarch.rp 2.2 MB/s | 196 kB 00:00 2023-12-26T21:33:39.674 INFO:teuthology.orchestra.run.smithi131.stdout:(59/140): python3-rsa-4.9-2.el8.noarch.rpm 856 kB/s | 64 kB 00:00 2023-12-26T21:33:39.716 INFO:teuthology.orchestra.run.smithi131.stdout:(60/140): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-12-26T21:33:39.741 INFO:teuthology.orchestra.run.smithi131.stdout:(61/140): python3-portend-2.6-1.el8.noarch.rpm 64 kB/s | 16 kB 00:00 2023-12-26T21:33:39.767 INFO:teuthology.orchestra.run.smithi131.stdout:(62/140): python3-singledispatch-3.4.0.3-18.el8 259 kB/s | 24 kB 00:00 2023-12-26T21:33:39.792 INFO:teuthology.orchestra.run.smithi131.stdout:(63/140): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2023-12-26T21:33:39.842 INFO:teuthology.orchestra.run.smithi131.stdout:(64/140): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-12-26T21:33:39.868 INFO:teuthology.orchestra.run.smithi131.stdout:(65/140): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-12-26T21:33:39.871 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2023-12-26T21:33:39.871 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2023-12-26T21:33:39.893 INFO:teuthology.orchestra.run.smithi131.stdout:(66/140): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-12-26T21:33:39.943 INFO:teuthology.orchestra.run.smithi131.stdout:(67/140): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-12-26T21:33:39.969 INFO:teuthology.orchestra.run.smithi131.stdout:(68/140): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-12-26T21:33:39.994 INFO:teuthology.orchestra.run.smithi131.stdout:(69/140): python3-websocket-client-0.56.0-5.el8 405 kB/s | 61 kB 00:00 2023-12-26T21:33:40.019 INFO:teuthology.orchestra.run.smithi131.stdout:(70/140): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-12-26T21:33:40.094 INFO:teuthology.orchestra.run.smithi131.stdout:(71/140): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-12-26T21:33:40.120 INFO:teuthology.orchestra.run.smithi131.stdout:(72/140): python3-jsonpointer-1.10-11.el8.noarc 159 kB/s | 20 kB 00:00 2023-12-26T21:33:40.192 INFO:teuthology.orchestra.run.smithi131.stdout:(73/140): python3-prettytable-0.7.2-14.el8.noar 263 kB/s | 44 kB 00:00 2023-12-26T21:33:40.237 INFO:teuthology.orchestra.run.smithi131.stdout:(74/140): python3-pyasn1-modules-0.3.7-6.el8.no 775 kB/s | 110 kB 00:00 2023-12-26T21:33:40.262 INFO:teuthology.orchestra.run.smithi131.stdout:(75/140): python3-jsonpatch-1.21-2.el8.noarch.r 195 kB/s | 27 kB 00:00 2023-12-26T21:33:40.296 INFO:teuthology.orchestra.run.smithi131.stdout:(76/140): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.2 MB/s | 126 kB 00:00 2023-12-26T21:33:40.371 INFO:teuthology.orchestra.run.smithi131.stdout:(77/140): python3-werkzeug-0.12.2-4.el8.noarch. 3.3 MB/s | 457 kB 00:00 2023-12-26T21:33:40.422 INFO:teuthology.orchestra.run.smithi131.stdout:(78/140): bison-3.0.4-10.el8.x86_64.rpm 4.2 MB/s | 688 kB 00:00 2023-12-26T21:33:40.455 INFO:teuthology.orchestra.run.smithi131.stdout:(79/140): flex-2.6.1-9.el8.x86_64.rpm 2.0 MB/s | 320 kB 00:00 2023-12-26T21:33:40.514 INFO:teuthology.orchestra.run.smithi131.stdout:(80/140): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2023-12-26T21:33:40.539 INFO:teuthology.orchestra.run.smithi131.stdout:(81/140): python3-markupsafe-0.23-19.el8.x86_64 330 kB/s | 39 kB 00:00 2023-12-26T21:33:40.565 INFO:teuthology.orchestra.run.smithi131.stdout:(82/140): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.3 MB/s | 259 kB 00:00 2023-12-26T21:33:40.665 INFO:teuthology.orchestra.run.smithi131.stdout:(83/140): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-12-26T21:33:40.707 INFO:teuthology.orchestra.run.smithi131.stdout:(84/140): python3-pyOpenSSL-19.0.0-1.el8.noarch 618 kB/s | 103 kB 00:00 2023-12-26T21:33:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:(85/140): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-12-26T21:33:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:(86/140): python3-jinja2-2.10.1-3.el8.noarch.rp 2.2 MB/s | 538 kB 00:00 2023-12-26T21:33:41.000 INFO:teuthology.orchestra.run.smithi131.stdout:(87/140): python3-numpy-1.14.3-10.el8.x86_64.rp 13 MB/s | 3.7 MB 00:00 2023-12-26T21:33:41.041 INFO:teuthology.orchestra.run.smithi131.stdout:(88/140): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2023-12-26T21:33:41.067 INFO:teuthology.orchestra.run.smithi131.stdout:(89/140): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2023-12-26T21:33:41.126 INFO:teuthology.orchestra.run.smithi131.stdout:(90/140): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-12-26T21:33:41.151 INFO:teuthology.orchestra.run.smithi131.stdout:(91/140): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2023-12-26T21:33:41.335 INFO:teuthology.orchestra.run.smithi131.stdout:(92/140): openblas-0.3.15-6.el8.x86_64.rpm 24 MB/s | 5.0 MB 00:00 2023-12-26T21:33:41.376 INFO:teuthology.orchestra.run.smithi131.stdout:(93/140): automake-1.16.1-8.el8.noarch.rpm 3.1 MB/s | 714 kB 00:00 2023-12-26T21:33:41.460 INFO:teuthology.orchestra.run.smithi131.stdout:(94/140): openblas-threads-0.3.15-6.el8.x86_64. 12 MB/s | 5.2 MB 00:00 2023-12-26T21:33:41.485 INFO:teuthology.orchestra.run.smithi131.stdout:(95/140): jq-1.6-7.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2023-12-26T21:33:41.519 INFO:teuthology.orchestra.run.smithi131.stdout:(96/140): python3-pytz-2017.2-11.el8.noarch.rpm 381 kB/s | 54 kB 00:00 2023-12-26T21:33:41.644 INFO:teuthology.orchestra.run.smithi131.stdout:(97/140): libverto-devel-0.3.0-5.el8.x86_64.rpm 113 kB/s | 18 kB 00:00 2023-12-26T21:33:41.669 INFO:teuthology.orchestra.run.smithi131.stdout:(98/140): python3-jwt-1.6.1-2.el8.noarch.rpm 283 kB/s | 43 kB 00:00 2023-12-26T21:33:41.791 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2023-12-26T21:33:41.803 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2023-12-26T21:33:41.811 INFO:teuthology.orchestra.run.smithi131.stdout:(99/140): python3-oauthlib-2.1.0-1.el8.noarch.r 929 kB/s | 155 kB 00:00 2023-12-26T21:33:42.254 INFO:teuthology.orchestra.run.smithi131.stdout:(100/140): python3-scipy-1.0.0-21.module+el8.9. 17 MB/s | 14 MB 00:00 2023-12-26T21:33:42.329 INFO:teuthology.orchestra.run.smithi131.stdout:(101/140): python3-requests-oauthlib-1.0.0-1.el 83 kB/s | 43 kB 00:00 2023-12-26T21:33:42.354 INFO:teuthology.orchestra.run.smithi131.stdout:(102/140): python3-pycparser-2.14-14.el8.noarch 159 kB/s | 109 kB 00:00 2023-12-26T21:33:42.379 INFO:teuthology.orchestra.run.smithi131.stdout:(103/140): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2023-12-26T21:33:42.430 INFO:teuthology.orchestra.run.smithi131.stdout:(104/140): libzstd-devel-1.4.4-1.el8.x86_64.rpm 437 kB/s | 44 kB 00:00 2023-12-26T21:33:42.472 INFO:teuthology.orchestra.run.smithi131.stdout:(105/140): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-12-26T21:33:42.514 INFO:teuthology.orchestra.run.smithi131.stdout:(106/140): pcre2-devel-10.32-2.el8.x86_64.rpm 3.7 MB/s | 605 kB 00:00 2023-12-26T21:33:42.539 INFO:teuthology.orchestra.run.smithi131.stdout:(107/140): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.0 MB/s | 220 kB 00:00 2023-12-26T21:33:42.606 INFO:teuthology.orchestra.run.smithi131.stdout:(108/140): libselinux-devel-2.9-5.el8.x86_64.rp 1.5 MB/s | 200 kB 00:00 2023-12-26T21:33:42.631 INFO:teuthology.orchestra.run.smithi131.stdout:(109/140): keyutils-libs-devel-1.5.10-9.el8.x86 410 kB/s | 48 kB 00:00 2023-12-26T21:33:42.665 INFO:teuthology.orchestra.run.smithi131.stdout:(110/140): libsepol-devel-2.9-3.el8.x86_64.rpm 694 kB/s | 87 kB 00:00 2023-12-26T21:33:42.740 INFO:teuthology.orchestra.run.smithi131.stdout:(111/140): zlib-devel-1.2.11-18.el8_5.x86_64.rp 430 kB/s | 58 kB 00:00 2023-12-26T21:33:42.766 INFO:teuthology.orchestra.run.smithi131.stdout:(112/140): libcom_err-devel-1.45.6-4.el8.x86_64 288 kB/s | 39 kB 00:00 2023-12-26T21:33:42.834 INFO:teuthology.orchestra.run.smithi131.stdout:(113/140): fuse-2.9.7-17.el8.x86_64.rpm 505 kB/s | 83 kB 00:00 2023-12-26T21:33:42.949 INFO:teuthology.orchestra.run.smithi131.stdout:(114/140): libquadmath-8.5.0-20.el8.x86_64.rpm 939 kB/s | 172 kB 00:00 2023-12-26T21:33:42.975 INFO:teuthology.orchestra.run.smithi131.stdout:(115/140): libuuid-devel-2.32.1-43.el8.x86_64.r 703 kB/s | 100 kB 00:00 2023-12-26T21:33:43.017 INFO:teuthology.orchestra.run.smithi131.stdout:(116/140): libgfortran-8.5.0-20.el8.x86_64.rpm 2.3 MB/s | 645 kB 00:00 2023-12-26T21:33:43.042 INFO:teuthology.orchestra.run.smithi131.stdout:(117/140): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-12-26T21:33:43.117 INFO:teuthology.orchestra.run.smithi131.stdout:(118/140): python3-cryptography-3.2.1-6.el8.x86 3.8 MB/s | 559 kB 00:00 2023-12-26T21:33:43.143 INFO:teuthology.orchestra.run.smithi131.stdout:(119/140): xfsprogs-devel-5.0.0-12.el8.x86_64.r 579 kB/s | 73 kB 00:00 2023-12-26T21:33:43.168 INFO:teuthology.orchestra.run.smithi131.stdout:(120/140): NetworkManager-initscripts-updown-1. 1.1 MB/s | 143 kB 00:00 2023-12-26T21:33:43.268 INFO:teuthology.orchestra.run.smithi131.stdout:(121/140): elfutils-libelf-devel-0.189-3.el8.x8 411 kB/s | 62 kB 00:00 2023-12-26T21:33:43.293 INFO:teuthology.orchestra.run.smithi131.stdout:(122/140): librabbitmq-0.9.0-4.el8.x86_64.rpm 312 kB/s | 47 kB 00:00 2023-12-26T21:33:43.344 INFO:teuthology.orchestra.run.smithi131.stdout:(123/140): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.1 MB/s | 562 kB 00:00 2023-12-26T21:33:43.519 INFO:teuthology.orchestra.run.smithi131.stdout:(124/140): openssl-devel-1.1.1k-12.el8_9.x86_64 9.3 MB/s | 2.3 MB 00:00 2023-12-26T21:33:43.636 INFO:teuthology.orchestra.run.smithi131.stdout:(125/140): librados2-17.2.7-201.g5b5a60d2.el8.x 10 MB/s | 3.5 MB 00:00 2023-12-26T21:33:43.670 INFO:teuthology.orchestra.run.smithi131.stdout:(126/140): elfutils-libelf-0.189-3.el8.x86_64.r 1.5 MB/s | 232 kB 00:00 2023-12-26T21:33:43.812 INFO:teuthology.orchestra.run.smithi131.stdout:(127/140): libmount-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 237 kB 00:00 2023-12-26T21:33:43.837 INFO:teuthology.orchestra.run.smithi131.stdout:(128/140): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2023-12-26T21:33:43.938 INFO:teuthology.orchestra.run.smithi131.stdout:(129/140): librbd1-17.2.7-201.g5b5a60d2.el8.x86 6.0 MB/s | 3.6 MB 00:00 2023-12-26T21:33:43.971 INFO:teuthology.orchestra.run.smithi131.stdout:(130/140): libuuid-2.32.1-43.el8.x86_64.rpm 741 kB/s | 99 kB 00:00 2023-12-26T21:33:43.996 INFO:teuthology.orchestra.run.smithi131.stdout:(131/140): libsmartcols-2.32.1-43.el8.x86_64.rp 976 kB/s | 179 kB 00:00 2023-12-26T21:33:44.130 INFO:teuthology.orchestra.run.smithi131.stdout:(132/140): util-linux-user-2.32.1-43.el8.x86_64 648 kB/s | 103 kB 00:00 2023-12-26T21:33:44.163 INFO:teuthology.orchestra.run.smithi131.stdout:(133/140): xfsdump-3.1.8-6.el8.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2023-12-26T21:33:44.222 INFO:teuthology.orchestra.run.smithi131.stdout:(134/140): util-linux-2.32.1-43.el8.x86_64.rpm 8.7 MB/s | 2.5 MB 00:00 2023-12-26T21:33:44.264 INFO:teuthology.orchestra.run.smithi131.stdout:(135/140): iproute-6.2.0-5.el8_9.x86_64.rpm 6.2 MB/s | 854 kB 00:00 2023-12-26T21:33:44.298 INFO:teuthology.orchestra.run.smithi131.stdout:(136/140): libblkid-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2023-12-26T21:33:44.365 INFO:teuthology.orchestra.run.smithi131.stdout:(137/140): libfdisk-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 254 kB 00:00 2023-12-26T21:33:44.407 INFO:teuthology.orchestra.run.smithi131.stdout:(138/140): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.5 MB/s | 1.1 MB 00:00 2023-12-26T21:33:44.457 INFO:teuthology.orchestra.run.smithi131.stdout:(139/140): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.4 MB/s | 711 kB 00:00 2023-12-26T21:33:44.549 INFO:teuthology.orchestra.run.smithi131.stdout:(140/140): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.0 MB/s | 1.5 MB 00:00 2023-12-26T21:33:44.552 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2023-12-26T21:33:44.552 INFO:teuthology.orchestra.run.smithi131.stdout:Total 17 MB/s | 212 MB 00:12 2023-12-26T21:33:46.504 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2023-12-26T21:33:46.959 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2023-12-26T21:33:46.959 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2023-12-26T21:33:47.260 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2023-12-26T21:33:47.465 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2023-12-26T21:33:47.727 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2023-12-26T21:33:47.818 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2023-12-26T21:33:48.017 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 2/156 2023-12-26T21:33:48.099 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 2/156 2023-12-26T21:33:48.359 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 3/156 2023-12-26T21:33:48.437 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 3/156 2023-12-26T21:33:48.817 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/156 2023-12-26T21:33:48.837 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2023-12-26T21:33:48.849 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2023-12-26T21:33:49.069 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/156 2023-12-26T21:33:49.267 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/156 2023-12-26T21:33:49.343 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/156 2023-12-26T21:33:49.610 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/156 2023-12-26T21:33:49.833 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/156 2023-12-26T21:33:50.256 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/156 2023-12-26T21:33:50.459 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:17.2.7-201.g5b5a60d2.el8.x86_64 9/156 2023-12-26T21:33:50.925 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:17.2.7-201.g5b5a60d2.el8.x86_64 9/156 2023-12-26T21:33:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:17.2.7-201.g5b5a60d2.el8.x86_64 10/156 2023-12-26T21:33:51.266 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:17.2.7-201.g5b5a60d2.el8.x86_64 10/156 2023-12-26T21:33:51.501 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:17.2.7-201.g5b5a60d2.el8.x86_64 11/156 2023-12-26T21:33:51.619 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:17.2.7-201.g5b5a60d2.el8.x86_64 11/156 2023-12-26T21:33:52.318 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:17.2.7-201.g5b5a60d2.el8.x86_64 12/156 2023-12-26T21:33:52.525 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:17.2.7-201.g5b5a60d2.el8.x86_64 13/156 2023-12-26T21:33:52.796 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:17.2.7-201.g5b5a60d2.el8.x86_64 13/156 2023-12-26T21:33:53.033 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/156 2023-12-26T21:33:53.132 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/156 2023-12-26T21:33:53.559 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/156 2023-12-26T21:33:53.814 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/156 2023-12-26T21:33:54.009 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/156 2023-12-26T21:33:54.158 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:17.2.7-201.g5b5a60d2.el8.x86_64 18/156 2023-12-26T21:33:54.267 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2023-12-26T21:33:54.295 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:17.2.7-201.g5b5a60d2.el8.x86_64 18/156 2023-12-26T21:33:54.466 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:17.2.7-201.g5b5a60d2.el8.x86_ 19/156 2023-12-26T21:33:54.520 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2023-12-26T21:33:54.578 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:17.2.7-201.g5b5a60d2.el8.x86_ 19/156 2023-12-26T21:33:54.733 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/156 2023-12-26T21:33:54.825 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2023-12-26T21:33:54.913 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/156 2023-12-26T21:33:54.922 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2023-12-26T21:33:55.050 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/156 2023-12-26T21:33:55.081 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 2/156 2023-12-26T21:33:55.169 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 2/156 2023-12-26T21:33:55.357 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 3/156 2023-12-26T21:33:55.434 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 3/156 2023-12-26T21:33:55.747 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/156 2023-12-26T21:33:55.817 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/156 2023-12-26T21:33:55.941 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 22/156 2023-12-26T21:33:56.041 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/156 2023-12-26T21:33:56.223 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/156 2023-12-26T21:33:56.307 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/156 2023-12-26T21:33:56.560 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/156 2023-12-26T21:33:56.597 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 22/156 2023-12-26T21:33:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.15-6.el8.x86_64 23/156 2023-12-26T21:33:56.806 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/156 2023-12-26T21:33:57.243 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/156 2023-12-26T21:33:57.569 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 23/156 2023-12-26T21:33:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:17.2.7-201.g5b5a60d2.el8.x86_64 9/156 2023-12-26T21:33:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/156 2023-12-26T21:33:58.007 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/156 2023-12-26T21:33:58.154 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:17.2.7-201.g5b5a60d2.el8.x86_64 9/156 2023-12-26T21:33:58.172 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 26/156 2023-12-26T21:33:58.356 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:17.2.7-201.g5b5a60d2.el8.x86_64 10/156 2023-12-26T21:33:58.472 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 27/156 2023-12-26T21:33:58.513 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:17.2.7-201.g5b5a60d2.el8.x86_64 10/156 2023-12-26T21:33:58.706 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:17.2.7-201.g5b5a60d2.el8.x86_64 11/156 2023-12-26T21:33:58.757 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 28/156 2023-12-26T21:33:58.823 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:17.2.7-201.g5b5a60d2.el8.x86_64 11/156 2023-12-26T21:33:58.983 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 29/156 2023-12-26T21:33:59.190 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:17.2.7-201.g5b5a60d2.el8.x 30/156 2023-12-26T21:33:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:17.2.7-201.g5b5a60d2.el8 31/156 2023-12-26T21:33:59.521 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:17.2.7-201.g5b5a60d2.el8.x86_64 12/156 2023-12-26T21:33:59.605 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:17.2.7-201.g5b5a60d2.el8.x86_64 32/156 2023-12-26T21:33:59.740 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:17.2.7-201.g5b5a60d2.el8.x86_64 13/156 2023-12-26T21:34:00.021 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:17.2.7-201.g5b5a60d2.el8.x86_64 13/156 2023-12-26T21:34:00.256 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/156 2023-12-26T21:34:00.371 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/156 2023-12-26T21:34:00.612 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/156 2023-12-26T21:34:00.729 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/156 2023-12-26T21:34:00.861 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/156 2023-12-26T21:34:01.056 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/156 2023-12-26T21:34:01.065 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/156 2023-12-26T21:34:01.214 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:17.2.7-201.g5b5a60d2.el8.x86_64 18/156 2023-12-26T21:34:01.331 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/156 2023-12-26T21:34:01.350 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:17.2.7-201.g5b5a60d2.el8.x86_64 18/156 2023-12-26T21:34:01.507 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2023-12-26T21:34:01.531 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:17.2.7-201.g5b5a60d2.el8.x86_ 19/156 2023-12-26T21:34:01.621 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:17.2.7-201.g5b5a60d2.el8.x86_ 19/156 2023-12-26T21:34:01.822 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/156 2023-12-26T21:34:01.992 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/156 2023-12-26T21:34:02.164 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/156 2023-12-26T21:34:02.857 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/156 2023-12-26T21:34:03.039 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 22/156 2023-12-26T21:34:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2023-12-26T21:34:03.700 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 22/156 2023-12-26T21:34:03.888 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 37/156 2023-12-26T21:34:03.930 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.15-6.el8.x86_64 23/156 2023-12-26T21:34:04.150 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/156 2023-12-26T21:34:04.336 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 39/156 2023-12-26T21:34:04.594 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 40/156 2023-12-26T21:34:04.736 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 23/156 2023-12-26T21:34:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 41/156 2023-12-26T21:34:04.978 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:17.2.7-201.g5b5a60d2.el8.x86_64 42/156 2023-12-26T21:34:05.063 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/156 2023-12-26T21:34:05.223 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/156 2023-12-26T21:34:05.231 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/156 2023-12-26T21:34:05.336 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/156 2023-12-26T21:34:05.403 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 26/156 2023-12-26T21:34:05.540 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/156 2023-12-26T21:34:05.647 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/156 2023-12-26T21:34:05.697 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 27/156 2023-12-26T21:34:05.853 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/156 2023-12-26T21:34:06.030 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 28/156 2023-12-26T21:34:06.078 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/156 2023-12-26T21:34:06.306 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/156 2023-12-26T21:34:06.333 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 29/156 2023-12-26T21:34:06.496 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:17.2.7-201.g5b5a60d2.el8.x 30/156 2023-12-26T21:34:06.651 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:17.2.7-201.g5b5a60d2.el8 31/156 2023-12-26T21:34:06.901 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/156 2023-12-26T21:34:06.929 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:17.2.7-201.g5b5a60d2.el8.x86_64 32/156 2023-12-26T21:34:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/156 2023-12-26T21:34:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/156 2023-12-26T21:34:07.486 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/156 2023-12-26T21:34:07.691 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/156 2023-12-26T21:34:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/156 2023-12-26T21:34:08.051 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/156 2023-12-26T21:34:08.077 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/156 2023-12-26T21:34:08.286 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/156 2023-12-26T21:34:08.397 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/156 2023-12-26T21:34:08.614 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/156 2023-12-26T21:34:08.688 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/156 2023-12-26T21:34:08.879 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2023-12-26T21:34:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/156 2023-12-26T21:34:09.162 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/156 2023-12-26T21:34:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/156 2023-12-26T21:34:09.548 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/156 2023-12-26T21:34:09.972 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/156 2023-12-26T21:34:10.237 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 60/156 2023-12-26T21:34:10.437 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/156 2023-12-26T21:34:10.706 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 62/156 2023-12-26T21:34:10.932 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2023-12-26T21:34:10.974 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 63/156 2023-12-26T21:34:11.303 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 37/156 2023-12-26T21:34:11.309 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 64/156 2023-12-26T21:34:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 65/156 2023-12-26T21:34:11.539 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/156 2023-12-26T21:34:11.716 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 66/156 2023-12-26T21:34:11.734 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 39/156 2023-12-26T21:34:11.925 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/156 2023-12-26T21:34:12.050 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 40/156 2023-12-26T21:34:12.169 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/156 2023-12-26T21:34:12.277 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 41/156 2023-12-26T21:34:12.472 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/156 2023-12-26T21:34:12.483 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:17.2.7-201.g5b5a60d2.el8.x86_64 42/156 2023-12-26T21:34:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.1-8.el8.noarch 70/156 2023-12-26T21:34:12.712 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/156 2023-12-26T21:34:12.808 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/156 2023-12-26T21:34:12.847 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 71/156 2023-12-26T21:34:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 71/156 2023-12-26T21:34:13.054 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/156 2023-12-26T21:34:13.139 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : jq-1.6-7.el8.x86_64 72/156 2023-12-26T21:34:13.177 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/156 2023-12-26T21:34:13.330 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 73/156 2023-12-26T21:34:13.392 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/156 2023-12-26T21:34:13.513 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 74/156 2023-12-26T21:34:13.652 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/156 2023-12-26T21:34:13.699 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 75/156 2023-12-26T21:34:13.879 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/156 2023-12-26T21:34:13.889 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 76/156 2023-12-26T21:34:14.062 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 77/156 2023-12-26T21:34:14.302 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 78/156 2023-12-26T21:34:14.521 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 79/156 2023-12-26T21:34:14.526 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/156 2023-12-26T21:34:14.676 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 80/156 2023-12-26T21:34:14.846 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/156 2023-12-26T21:34:14.867 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 81/156 2023-12-26T21:34:14.879 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/156 2023-12-26T21:34:15.083 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 82/156 2023-12-26T21:34:15.142 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/156 2023-12-26T21:34:15.294 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 83/156 2023-12-26T21:34:15.365 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/156 2023-12-26T21:34:15.557 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/156 2023-12-26T21:34:15.561 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 84/156 2023-12-26T21:34:15.725 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 85/156 2023-12-26T21:34:15.759 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/156 2023-12-26T21:34:15.911 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 86/156 2023-12-26T21:34:15.959 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/156 2023-12-26T21:34:16.134 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 87/156 2023-12-26T21:34:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/156 2023-12-26T21:34:16.406 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 88/156 2023-12-26T21:34:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 89/156 2023-12-26T21:34:16.654 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/156 2023-12-26T21:34:16.839 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-2.6-1.el8.noarch 90/156 2023-12-26T21:34:16.884 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/156 2023-12-26T21:34:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 91/156 2023-12-26T21:34:17.063 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/156 2023-12-26T21:34:17.264 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 92/156 2023-12-26T21:34:17.303 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/156 2023-12-26T21:34:17.476 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 93/156 2023-12-26T21:34:17.719 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/156 2023-12-26T21:34:18.076 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 60/156 2023-12-26T21:34:18.326 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/156 2023-12-26T21:34:18.615 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 62/156 2023-12-26T21:34:18.693 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 94/156 2023-12-26T21:34:18.871 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 63/156 2023-12-26T21:34:19.233 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 64/156 2023-12-26T21:34:19.449 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 95/156 2023-12-26T21:34:19.454 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 65/156 2023-12-26T21:34:19.639 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 66/156 2023-12-26T21:34:19.684 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 96/156 2023-12-26T21:34:19.856 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/156 2023-12-26T21:34:20.048 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 97/156 2023-12-26T21:34:20.054 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/156 2023-12-26T21:34:20.353 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 98/156 2023-12-26T21:34:20.401 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/156 2023-12-26T21:34:20.559 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 99/156 2023-12-26T21:34:20.625 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.1-8.el8.noarch 70/156 2023-12-26T21:34:20.738 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 100/156 2023-12-26T21:34:20.778 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 71/156 2023-12-26T21:34:20.877 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 71/156 2023-12-26T21:34:21.087 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : jq-1.6-7.el8.x86_64 72/156 2023-12-26T21:34:21.244 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 73/156 2023-12-26T21:34:21.481 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 74/156 2023-12-26T21:34:21.663 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 75/156 2023-12-26T21:34:21.829 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 76/156 2023-12-26T21:34:22.018 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 77/156 2023-12-26T21:34:22.268 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 78/156 2023-12-26T21:34:22.519 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 79/156 2023-12-26T21:34:22.683 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 80/156 2023-12-26T21:34:22.874 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 81/156 2023-12-26T21:34:23.510 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 82/156 2023-12-26T21:34:23.745 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 83/156 2023-12-26T21:34:23.963 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 84/156 2023-12-26T21:34:24.123 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 85/156 2023-12-26T21:34:24.318 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 86/156 2023-12-26T21:34:24.566 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 87/156 2023-12-26T21:34:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 88/156 2023-12-26T21:34:24.976 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 89/156 2023-12-26T21:34:25.238 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-2.6-1.el8.noarch 90/156 2023-12-26T21:34:25.503 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 91/156 2023-12-26T21:34:25.705 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 92/156 2023-12-26T21:34:25.927 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 93/156 2023-12-26T21:34:26.139 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 101/156 2023-12-26T21:34:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 101/156 2023-12-26T21:34:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 101/156 2023-12-26T21:34:27.173 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 94/156 2023-12-26T21:34:27.339 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:17.2.7-201.g5b5a60d2.el8.x86_64 102/156 2023-12-26T21:34:27.473 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:17.2.7-201.g5b5a60d2.el8.x86_64 102/156 2023-12-26T21:34:27.528 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 95/156 2023-12-26T21:34:27.653 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:17.2.7-201.g5b5a60d2.el8.x86_64 103/156 2023-12-26T21:34:27.789 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 96/156 2023-12-26T21:34:28.161 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 97/156 2023-12-26T21:34:28.487 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 98/156 2023-12-26T21:34:29.057 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 99/156 2023-12-26T21:34:29.286 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 100/156 2023-12-26T21:34:34.469 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 101/156 2023-12-26T21:34:34.743 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 101/156 2023-12-26T21:34:35.405 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 101/156 2023-12-26T21:34:35.752 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:17.2.7-201.g5b5a60d2.el8.x86_64 102/156 2023-12-26T21:34:35.892 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:17.2.7-201.g5b5a60d2.el8.x86_64 102/156 2023-12-26T21:34:36.084 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:17.2.7-201.g5b5a60d2.el8.x86_64 103/156 2023-12-26T21:34:48.521 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:17.2.7-201.g5b5a60d2.el8.x86_64 103/156 2023-12-26T21:34:48.521 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2023-12-26T21:34:48.521 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2023-12-26T21:34:48.521 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2023-12-26T21:34:48.521 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /run 2023-12-26T21:34:48.521 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2023-12-26T21:34:48.521 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2023-12-26T21:34:48.521 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2023-12-26T21:34:48.521 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2023-12-26T21:34:48.522 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2023-12-26T21:34:48.522 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:34:48.900 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:17.2.7-201.g5b5a60d2.el8.noarch 104/156 2023-12-26T21:34:50.504 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:17.2.7-201.g5b5a60d2.el8.noarch 104/156 2023-12-26T21:34:50.693 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:17.2.7-201.g5b5a60d2.el8.x86_64 105/156 2023-12-26T21:34:50.961 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:17.2.7-201.g5b5a60d2.el8.x86_64 105/156 2023-12-26T21:34:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:17.2.7-201.g5b5a60d2.el8.x86_64 106/156 2023-12-26T21:34:51.572 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:17.2.7-201.g5b5a60d2.el8.x86_64 106/156 2023-12-26T21:34:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:17.2.7-201.g5b5a60d2.el8.x86_64 107/156 2023-12-26T21:34:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:17.2.7-201.g5b5a60d2.el8.x86_64 107/156 2023-12-26T21:34:54.551 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 108/156 2023-12-26T21:34:54.721 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 108/156 2023-12-26T21:34:54.823 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-201.g5b5a60d2.el 109/156 2023-12-26T21:34:55.008 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-201.g5b5a60d2.e 110/156 2023-12-26T21:34:55.215 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-201.g5b5a60d2.el8.noar 111/156 2023-12-26T21:34:55.480 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-201.g5b5a60d2.el8.noar 111/156 2023-12-26T21:34:55.784 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-201.g5b5a60d2.el8.no 112/156 2023-12-26T21:34:56.867 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-201.g5b5a60d2.el8.no 112/156 2023-12-26T21:34:57.048 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-201.g5b5a 113/156 2023-12-26T21:34:57.080 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:17.2.7-201.g5b5a60d2.el8.x86_64 103/156 2023-12-26T21:34:57.080 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2023-12-26T21:34:57.080 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2023-12-26T21:34:57.080 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2023-12-26T21:34:57.080 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /run 2023-12-26T21:34:57.080 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2023-12-26T21:34:57.080 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2023-12-26T21:34:57.081 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2023-12-26T21:34:57.081 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2023-12-26T21:34:57.081 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /dev 2023-12-26T21:34:57.081 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:34:57.178 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-201.g5b5a 113/156 2023-12-26T21:34:57.490 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:17.2.7-201.g5b5a60d2.el8.noarch 104/156 2023-12-26T21:34:57.591 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-201.g5b5a60d2.el8 114/156 2023-12-26T21:34:57.775 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:17.2.7-201.g5b5a60d2.el8.x86_64 115/156 2023-12-26T21:34:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:17.2.7-201.g5b5a60d2.el8.x86_64 115/156 2023-12-26T21:34:57.999 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:17.2.7-201.g5b5a60d2.el8.noarch 116/156 2023-12-26T21:34:58.029 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-201.g5b5a60d2.el8.noarch 116/156 2023-12-26T21:34:59.090 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:17.2.7-201.g5b5a60d2.el8.noarch 104/156 2023-12-26T21:34:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:17.2.7-201.g5b5a60d2.el8.x86_64 105/156 2023-12-26T21:34:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:17.2.7-201.g5b5a60d2.el8.x86_64 105/156 2023-12-26T21:34:59.703 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:17.2.7-201.g5b5a60d2.el8.x86_64 117/156 2023-12-26T21:34:59.714 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:17.2.7-201.g5b5a60d2.el8.x86_64 106/156 2023-12-26T21:34:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 118/156 2023-12-26T21:35:00.013 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 118/156 2023-12-26T21:35:00.167 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:17.2.7-201.g5b5a60d2.el8.x86_64 106/156 2023-12-26T21:35:00.174 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-201.g5b5a60 119/156 2023-12-26T21:35:00.246 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-201.g5b5a60 119/156 2023-12-26T21:35:00.330 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:17.2.7-201.g5b5a60d2.el8.x86_64 107/156 2023-12-26T21:35:00.374 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:17.2.7-201.g5b5a60d2.el8.x86_64 107/156 2023-12-26T21:35:00.382 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 120/156 2023-12-26T21:35:00.785 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 120/156 2023-12-26T21:35:00.956 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 121/156 2023-12-26T21:35:03.161 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 108/156 2023-12-26T21:35:03.345 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 108/156 2023-12-26T21:35:03.470 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-201.g5b5a60d2.el 109/156 2023-12-26T21:35:03.655 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-201.g5b5a60d2.e 110/156 2023-12-26T21:35:03.905 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-201.g5b5a60d2.el8.noar 111/156 2023-12-26T21:35:04.168 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-201.g5b5a60d2.el8.noar 111/156 2023-12-26T21:35:04.430 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-201.g5b5a60d2.el8.no 112/156 2023-12-26T21:35:05.521 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-201.g5b5a60d2.el8.no 112/156 2023-12-26T21:35:05.712 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-201.g5b5a 113/156 2023-12-26T21:35:05.842 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-201.g5b5a 113/156 2023-12-26T21:35:06.235 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 121/156 2023-12-26T21:35:06.263 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-201.g5b5a60d2.el8 114/156 2023-12-26T21:35:06.456 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:17.2.7-201.g5b5a60d2.el8.x86_64 115/156 2023-12-26T21:35:06.524 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:17.2.7-201.g5b5a60d2.el8.x86_64 115/156 2023-12-26T21:35:06.537 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:17.2.7-201.g5b5a60d2.el8.x86_64 122/156 2023-12-26T21:35:06.696 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:17.2.7-201.g5b5a60d2.el8.noarch 116/156 2023-12-26T21:35:06.705 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-25.el8.x86_64 123/156 2023-12-26T21:35:06.726 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-201.g5b5a60d2.el8.noarch 116/156 2023-12-26T21:35:07.716 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 123/156 2023-12-26T21:35:08.019 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 124/156 2023-12-26T21:35:08.287 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 125/156 2023-12-26T21:35:08.419 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:17.2.7-201.g5b5a60d2.el8.x86_64 117/156 2023-12-26T21:35:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 126/156 2023-12-26T21:35:08.614 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 118/156 2023-12-26T21:35:08.661 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 127/156 2023-12-26T21:35:08.741 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 118/156 2023-12-26T21:35:08.842 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 128/156 2023-12-26T21:35:08.921 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-201.g5b5a60 119/156 2023-12-26T21:35:08.993 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-201.g5b5a60 119/156 2023-12-26T21:35:09.082 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:17.2.7-201.g5b5a60d2.el8.x86_6 129/156 2023-12-26T21:35:09.179 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 120/156 2023-12-26T21:35:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 130/156 2023-12-26T21:35:09.581 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 120/156 2023-12-26T21:35:09.737 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 121/156 2023-12-26T21:35:09.843 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 131/156 2023-12-26T21:35:10.085 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 132/156 2023-12-26T21:35:10.238 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:17.2.7-201.g5b5a60d2.el8.noarch 133/156 2023-12-26T21:35:10.472 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 134/156 2023-12-26T21:35:10.772 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 135/156 2023-12-26T21:35:11.023 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 136/156 2023-12-26T21:35:11.204 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 137/156 2023-12-26T21:35:11.229 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 137/156 2023-12-26T21:35:11.429 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 138/156 2023-12-26T21:35:11.654 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.1-9.el8.x86_64 139/156 2023-12-26T21:35:11.767 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 139/156 2023-12-26T21:35:11.996 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.0.4-10.el8.x86_64 140/156 2023-12-26T21:35:12.025 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 140/156 2023-12-26T21:35:12.202 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 141/156 2023-12-26T21:35:12.381 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 142/156 2023-12-26T21:35:12.561 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 143/156 2023-12-26T21:35:12.561 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 144/156 2023-12-26T21:35:12.834 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 144/156 2023-12-26T21:35:12.834 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 145/156 2023-12-26T21:35:13.069 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 145/156 2023-12-26T21:35:13.070 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/156 2023-12-26T21:35:13.305 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/156 2023-12-26T21:35:13.506 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 147/156 2023-12-26T21:35:13.667 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 148/156 2023-12-26T21:35:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 149/156 2023-12-26T21:35:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 150/156 2023-12-26T21:35:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 150/156 2023-12-26T21:35:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 151/156 2023-12-26T21:35:14.407 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 151/156 2023-12-26T21:35:14.407 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 152/156 2023-12-26T21:35:14.640 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 152/156 2023-12-26T21:35:14.640 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 153/156 2023-12-26T21:35:14.881 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 153/156 2023-12-26T21:35:14.882 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 154/156 2023-12-26T21:35:15.006 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 121/156 2023-12-26T21:35:15.132 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 154/156 2023-12-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 155/156 2023-12-26T21:35:15.273 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2023-12-26T21:35:15.373 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:17.2.7-201.g5b5a60d2.el8.x86_64 122/156 2023-12-26T21:35:15.569 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-25.el8.x86_64 123/156 2023-12-26T21:35:15.829 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2023-12-26T21:35:15.829 INFO:teuthology.orchestra.run.smithi190.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-26T21:35:15.829 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:17.2.7-201.g5b5a60d2.el8.x86_64 1/156 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:17.2.7-201.g5b5a60d2.el8.x86_64 2/156 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 3/156 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 4/156 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-201.g5b5a60 5/156 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:17.2.7-201.g5b5a60d2.el8.x86_64 6/156 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:17.2.7-201.g5b5a60d2.el8.x86_64 7/156 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:17.2.7-201.g5b5a60d2.el8.x86_64 8/156 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:17.2.7-201.g5b5a60d2.el8.x86_64 9/156 2023-12-26T21:35:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 10/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:17.2.7-201.g5b5a60d2.el8.x86_64 11/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:17.2.7-201.g5b5a60d2.el8.x86_64 12/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 13/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:17.2.7-201.g5b5a60d2.el8.x86_6 14/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:17.2.7-201.g5b5a60d2.el8.x86_64 15/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:17.2.7-201.g5b5a60d2.el8.x86_64 16/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:17.2.7-201.g5b5a60d2.el8.x86_64 17/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:17.2.7-201.g5b5a60d2.el8.x86_ 18/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:17.2.7-201.g5b5a60d2.el8.x86_64 19/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:17.2.7-201.g5b5a60d2.el8 20/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:17.2.7-201.g5b5a60d2.el8.x 21/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:17.2.7-201.g5b5a60d2.el8.x86_64 22/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:17.2.7-201.g5b5a60d2.el8.x86_64 23/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 24/156 2023-12-26T21:35:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 25/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 26/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 27/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 28/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-201.g5b5a60d2.e 29/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-201.g5b5a60d2.el8.noar 30/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-201.g5b5a60d2.el8.no 31/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-201.g5b5a 32/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-201.g5b5a60d2.el8 33/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:17.2.7-201.g5b5a60d2.el8.noarch 34/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-201.g5b5a60d2.el 35/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:17.2.7-201.g5b5a60d2.el8.noarch 36/156 2023-12-26T21:35:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 37/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:17.2.7-201.g5b5a60d2.el8.noarch 38/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/156 2023-12-26T21:35:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/156 2023-12-26T21:35:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/156 2023-12-26T21:35:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/156 2023-12-26T21:35:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/156 2023-12-26T21:35:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/156 2023-12-26T21:35:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/156 2023-12-26T21:35:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/156 2023-12-26T21:35:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/156 2023-12-26T21:35:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/156 2023-12-26T21:35:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/156 2023-12-26T21:35:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/156 2023-12-26T21:35:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 83/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/156 2023-12-26T21:35:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : jq-1.6-7.el8.x86_64 95/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 96/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 97/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 98/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 99/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 100/156 2023-12-26T21:35:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 101/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 102/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 103/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 104/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 105/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 106/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 107/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 108/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 109/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 110/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 111/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 112/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 113/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 114/156 2023-12-26T21:35:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 115/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 116/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 117/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 118/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 119/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 121/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 122/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 123/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 124/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:17.2.7-201.g5b5a60d2.el8.x86_64 125/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 126/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:17.2.7-201.g5b5a60d2.el8.x86_64 127/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 128/156 2023-12-26T21:35:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 129/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 130/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 131/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 132/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 133/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 134/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 135/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 136/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 137/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 138/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 139/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 140/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 141/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 142/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 143/156 2023-12-26T21:35:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 144/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 145/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 146/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 147/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 148/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 149/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 150/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 151/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 152/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 153/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 154/156 2023-12-26T21:35:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 155/156 2023-12-26T21:35:16.588 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 123/156 2023-12-26T21:35:16.883 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 124/156 2023-12-26T21:35:17.126 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 125/156 2023-12-26T21:35:17.344 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 126/156 2023-12-26T21:35:17.566 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 127/156 2023-12-26T21:35:17.790 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 128/156 2023-12-26T21:35:18.010 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:17.2.7-201.g5b5a60d2.el8.x86_6 129/156 2023-12-26T21:35:18.427 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 130/156 2023-12-26T21:35:18.786 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 131/156 2023-12-26T21:35:19.040 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 132/156 2023-12-26T21:35:19.229 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:17.2.7-201.g5b5a60d2.el8.noarch 133/156 2023-12-26T21:35:19.469 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 134/156 2023-12-26T21:35:19.783 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 135/156 2023-12-26T21:35:20.046 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 136/156 2023-12-26T21:35:20.210 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 137/156 2023-12-26T21:35:20.238 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 137/156 2023-12-26T21:35:20.459 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 138/156 2023-12-26T21:35:20.685 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.1-9.el8.x86_64 139/156 2023-12-26T21:35:20.794 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 139/156 2023-12-26T21:35:21.035 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.0.4-10.el8.x86_64 140/156 2023-12-26T21:35:21.063 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 140/156 2023-12-26T21:35:21.258 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 141/156 2023-12-26T21:35:21.486 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 142/156 2023-12-26T21:35:21.690 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 143/156 2023-12-26T21:35:21.690 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 144/156 2023-12-26T21:35:21.964 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 144/156 2023-12-26T21:35:21.964 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 145/156 2023-12-26T21:35:22.208 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 145/156 2023-12-26T21:35:22.208 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/156 2023-12-26T21:35:22.535 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/156 2023-12-26T21:35:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 147/156 2023-12-26T21:35:22.971 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 148/156 2023-12-26T21:35:23.228 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 149/156 2023-12-26T21:35:23.228 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 150/156 2023-12-26T21:35:23.530 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 150/156 2023-12-26T21:35:23.530 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 151/156 2023-12-26T21:35:23.804 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 151/156 2023-12-26T21:35:23.804 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 152/156 2023-12-26T21:35:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 152/156 2023-12-26T21:35:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 153/156 2023-12-26T21:35:24.329 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 153/156 2023-12-26T21:35:24.329 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 154/156 2023-12-26T21:35:24.579 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 154/156 2023-12-26T21:35:24.779 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 155/156 2023-12-26T21:35:24.779 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2023-12-26T21:35:25.360 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2023-12-26T21:35:25.360 INFO:teuthology.orchestra.run.smithi131.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-26T21:35:25.361 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:17.2.7-201.g5b5a60d2.el8.x86_64 1/156 2023-12-26T21:35:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:17.2.7-201.g5b5a60d2.el8.x86_64 2/156 2023-12-26T21:35:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 3/156 2023-12-26T21:35:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 4/156 2023-12-26T21:35:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-201.g5b5a60 5/156 2023-12-26T21:35:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:17.2.7-201.g5b5a60d2.el8.x86_64 6/156 2023-12-26T21:35:26.029 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:17.2.7-201.g5b5a60d2.el8.x86_64 7/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:17.2.7-201.g5b5a60d2.el8.x86_64 8/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:17.2.7-201.g5b5a60d2.el8.x86_64 9/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 10/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:17.2.7-201.g5b5a60d2.el8.x86_64 11/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:17.2.7-201.g5b5a60d2.el8.x86_64 12/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 13/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:17.2.7-201.g5b5a60d2.el8.x86_6 14/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:17.2.7-201.g5b5a60d2.el8.x86_64 15/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:17.2.7-201.g5b5a60d2.el8.x86_64 16/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:17.2.7-201.g5b5a60d2.el8.x86_64 17/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:17.2.7-201.g5b5a60d2.el8.x86_ 18/156 2023-12-26T21:35:26.030 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:17.2.7-201.g5b5a60d2.el8.x86_64 19/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:17.2.7-201.g5b5a60d2.el8 20/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:17.2.7-201.g5b5a60d2.el8.x 21/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:17.2.7-201.g5b5a60d2.el8.x86_64 22/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:17.2.7-201.g5b5a60d2.el8.x86_64 23/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 24/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 25/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 26/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 27/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 28/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-201.g5b5a60d2.e 29/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-201.g5b5a60d2.el8.noar 30/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-201.g5b5a60d2.el8.no 31/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-201.g5b5a 32/156 2023-12-26T21:35:26.031 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-201.g5b5a60d2.el8 33/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:17.2.7-201.g5b5a60d2.el8.noarch 34/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-201.g5b5a60d2.el 35/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:17.2.7-201.g5b5a60d2.el8.noarch 36/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 37/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:17.2.7-201.g5b5a60d2.el8.noarch 38/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/156 2023-12-26T21:35:26.032 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/156 2023-12-26T21:35:26.033 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/156 2023-12-26T21:35:26.034 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/156 2023-12-26T21:35:26.035 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 83/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/156 2023-12-26T21:35:26.036 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : jq-1.6-7.el8.x86_64 95/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 96/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 97/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 98/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 99/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 100/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 101/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 102/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 103/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 104/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 105/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 106/156 2023-12-26T21:35:26.037 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 107/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 108/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 109/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 110/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 111/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 112/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 113/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 114/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 115/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 116/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 117/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 118/156 2023-12-26T21:35:26.038 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 119/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 121/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 122/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 123/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 124/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:17.2.7-201.g5b5a60d2.el8.x86_64 125/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 126/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:17.2.7-201.g5b5a60d2.el8.x86_64 127/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 128/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 129/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 130/156 2023-12-26T21:35:26.039 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 131/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 132/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 133/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 134/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 135/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 136/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 137/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 138/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 139/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 140/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 141/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 142/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 143/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 144/156 2023-12-26T21:35:26.040 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 145/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 146/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 147/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 148/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 149/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 150/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 151/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 152/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 153/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 154/156 2023-12-26T21:35:26.041 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 155/156 2023-12-26T21:35:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2023-12-26T21:35:27.098 INFO:teuthology.orchestra.run.smithi190.stdout:Installed products updated. 2023-12-26T21:35:27.759 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:27.759 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2023-12-26T21:35:27.759 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-26T21:35:27.760 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.1-8.el8.noarch 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.0.4-10.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.761 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.762 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.762 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.762 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.762 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.762 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.763 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.763 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.763 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.763 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.763 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.763 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.763 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.763 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.1-9.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: jq-1.6-7.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-26T21:35:27.764 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-26T21:35:27.765 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-26T21:35:27.766 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-26T21:35:27.766 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-26T21:35:27.766 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-26T21:35:27.766 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-26T21:35:27.766 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2023-12-26T21:35:27.766 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-26T21:35:27.766 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-26T21:35:27.767 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-26T21:35:27.768 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-26T21:35:27.768 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-26T21:35:27.768 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-26T21:35:27.769 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-26T21:35:27.769 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-26T21:35:27.769 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-26T21:35:27.769 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-26T21:35:27.769 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-26T21:35:27.769 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-26T21:35:27.769 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-26T21:35:27.769 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-26T21:35:27.769 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-2.6-1.el8.noarch 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-26T21:35:27.770 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-26T21:35:27.771 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-26T21:35:27.772 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-26T21:35:27.773 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-26T21:35:27.773 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:27.773 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-12-26T21:35:28.106 DEBUG:teuthology.parallel:result is None 2023-12-26T21:35:39.497 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2023-12-26T21:35:39.497 INFO:teuthology.orchestra.run.smithi131.stdout:Installed products updated. 2023-12-26T21:35:40.148 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:40.148 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2023-12-26T21:35:40.149 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.1-8.el8.noarch 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.0.4-10.el8.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.150 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.151 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:17.2.7-201.g5b5a60d2.el8.noarch 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.1-9.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: jq-1.6-7.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.152 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-26T21:35:40.153 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-26T21:35:40.154 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-26T21:35:40.155 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-26T21:35:40.156 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-2.6-1.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-26T21:35:40.157 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:17.2.7-201.g5b5a60d2.el8.x86_64 2023-12-26T21:35:40.158 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-26T21:35:40.159 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-26T21:35:40.159 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-26T21:35:40.159 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-26T21:35:40.159 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-26T21:35:40.159 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:40.159 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2023-12-26T21:35:40.467 DEBUG:teuthology.parallel:result is None 2023-12-26T21:35:40.468 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5b5a60d2d510bfaa9213c41e32227fb008245a30 2023-12-26T21:35:40.696 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-26T21:35:40.750 INFO:teuthology.orchestra.run.smithi131.stdout:17.2.7-201.g5b5a60d2.el8 2023-12-26T21:35:40.751 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-201.g5b5a60d2.el8 2023-12-26T21:35:40.751 INFO:teuthology.task.install:The correct ceph version 17.2.7-201.g5b5a60d2 is installed. 2023-12-26T21:35:40.754 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5b5a60d2d510bfaa9213c41e32227fb008245a30 2023-12-26T21:35:40.899 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-26T21:35:40.962 INFO:teuthology.orchestra.run.smithi190.stdout:17.2.7-201.g5b5a60d2.el8 2023-12-26T21:35:40.962 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-201.g5b5a60d2.el8 2023-12-26T21:35:40.962 INFO:teuthology.task.install:The correct ceph version 17.2.7-201.g5b5a60d2 is installed. 2023-12-26T21:35:40.964 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-26T21:35:40.964 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:40.964 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-26T21:35:41.051 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:35:41.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-26T21:35:41.143 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-26T21:35:41.144 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:41.144 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2023-12-26T21:35:41.211 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-26T21:35:41.318 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:35:41.318 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2023-12-26T21:35:41.387 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-26T21:35:41.495 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-26T21:35:41.495 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:41.495 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-26T21:35:41.563 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-26T21:35:41.671 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:35:41.671 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-26T21:35:41.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-26T21:35:41.849 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-26T21:35:41.849 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:41.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2023-12-26T21:35:41.917 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-26T21:35:42.024 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:35:42.024 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2023-12-26T21:35:42.092 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-26T21:35:42.201 INFO:teuthology.run_tasks:Running task ceph... 2023-12-26T21:35:42.277 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-26T21:35:42.277 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /var/log/ceph 2023-12-26T21:35:42.282 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2023-12-26T21:35:42.349 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-26T21:35:42.349 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-26T21:35:42.354 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-26T21:35:42.420 INFO:tasks.ceph:Creating extra log directories... 2023-12-26T21:35:42.420 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-26T21:35:42.425 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-26T21:35:42.498 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-26T21:35:42.498 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-26T21:35:42.499 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-26T21:35:42.499 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:42.499 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-26T21:35:42.541 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-26T21:35:42.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-26T21:35:42.825 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:35:42.825 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-26T21:35:42.868 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-26T21:35:43.081 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-26T21:35:43.154 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-26T21:35:43.154 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-26T21:35:43.155 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800', 'branch': 'wip-yuri-testing-2023-12-26-0957-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7501800', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-12-26-0957-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-12-26-0957-quincy', 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6259, 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-12-26-0957-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_5b5a60d2d510bfaa9213c41e32227fb008245a30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30', 'targets': {'smithi131.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvWSIYGN42nYXiZs+QFU5n975Xqni3CVSzG4cs89fCCP7TwSr65VsCey5e0Y0vLDmCeAeWE8TtPLR38WHbqHJ8=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBElAKLCJzJNC0gYvfEhSyz+aEWg0uonEKHK9wzdREnEf7++OXkZtvR7zKpar+37n0Ax4xVUToUYupuA29ji5gfA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '5b5a60d2d510bfaa9213c41e32227fb008245a30', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9680d2aba1e76f64ade295c81599c97907457848', 'timestamp': '2023-12-26_19:48:51', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078'} 2023-12-26T21:35:43.155 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-26T21:35:43.160 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-26T21:35:43.203 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/run/ceph 2023-12-26T21:35:43.248 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2023-12-26T21:35:43.316 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:43.316 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2023-12-26T21:35:43.358 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-26T21:35:43.358 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2023-12-26T21:35:43.441 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-26T21:35:43.442 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-26T21:35:43.442 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-12-26T21:35:43.442 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-26T21:35:43.442 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2023-12-26T21:35:43.442 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-12-26 21:35:25.490040128 +0000 2023-12-26T21:35:43.442 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-12-26 21:31:54.225438998 +0000 2023-12-26T21:35:43.442 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-12-26 21:31:54.225438998 +0000 2023-12-26T21:35:43.442 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2023-12-26 21:31:54.198439562 +0000 2023-12-26T21:35:43.442 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-26T21:35:43.545 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-12-26T21:35:43.545 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-12-26T21:35:43.545 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000328793 s, 1.6 MB/s 2023-12-26T21:35:43.548 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-26T21:35:43.592 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2023-12-26T21:35:43.675 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-26T21:35:43.676 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-26T21:35:43.676 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-12-26T21:35:43.676 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-26T21:35:43.676 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2023-12-26T21:35:43.676 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-12-26 21:35:25.490040128 +0000 2023-12-26T21:35:43.676 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-12-26 21:31:55.060421560 +0000 2023-12-26T21:35:43.676 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-12-26 21:31:55.060421560 +0000 2023-12-26T21:35:43.676 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2023-12-26 21:31:55.032422145 +0000 2023-12-26T21:35:43.676 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-26T21:35:43.782 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-12-26T21:35:43.782 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-12-26T21:35:43.782 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000468819 s, 1.1 MB/s 2023-12-26T21:35:43.785 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-26T21:35:43.828 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2023-12-26T21:35:43.911 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-26T21:35:43.911 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-26T21:35:43.911 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-12-26T21:35:43.911 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-26T21:35:43.911 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2023-12-26T21:35:43.911 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-12-26 21:35:25.490040128 +0000 2023-12-26T21:35:43.911 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-12-26 21:31:55.864404769 +0000 2023-12-26T21:35:43.911 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-12-26 21:31:55.864404769 +0000 2023-12-26T21:35:43.911 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2023-12-26 21:31:55.840405270 +0000 2023-12-26T21:35:43.911 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-26T21:35:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-12-26T21:35:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-12-26T21:35:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000304621 s, 1.7 MB/s 2023-12-26T21:35:44.018 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-26T21:35:44.062 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2023-12-26T21:35:44.145 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-26T21:35:44.146 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-26T21:35:44.146 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-12-26T21:35:44.146 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-26T21:35:44.146 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2023-12-26T21:35:44.146 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2023-12-26 21:35:25.490040128 +0000 2023-12-26T21:35:44.146 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2023-12-26 21:31:56.675387831 +0000 2023-12-26T21:35:44.146 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2023-12-26 21:31:56.675387831 +0000 2023-12-26T21:35:44.146 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2023-12-26 21:31:56.651388332 +0000 2023-12-26T21:35:44.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-26T21:35:44.248 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2023-12-26T21:35:44.248 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2023-12-26T21:35:44.248 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000326454 s, 1.6 MB/s 2023-12-26T21:35:44.251 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-26T21:35:44.295 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-26T21:35:44.295 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:35:44.295 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2023-12-26T21:35:44.343 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-26T21:35:44.343 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2023-12-26T21:35:44.430 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-26T21:35:44.430 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-26T21:35:44.430 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-12-26T21:35:44.430 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-26T21:35:44.430 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-12-26T21:35:44.430 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-12-26 21:35:15.956497901 +0000 2023-12-26T21:35:44.430 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-12-26 21:31:46.546669608 +0000 2023-12-26T21:35:44.430 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-12-26 21:31:46.546669608 +0000 2023-12-26T21:35:44.430 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2023-12-26 21:31:46.517670421 +0000 2023-12-26T21:35:44.430 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-26T21:35:44.535 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-12-26T21:35:44.535 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-12-26T21:35:44.536 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000342656 s, 1.5 MB/s 2023-12-26T21:35:44.538 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-26T21:35:44.583 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2023-12-26T21:35:44.667 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-26T21:35:44.667 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-26T21:35:44.667 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-12-26T21:35:44.667 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-26T21:35:44.667 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-12-26T21:35:44.667 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-12-26 21:35:15.956497901 +0000 2023-12-26T21:35:44.668 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-12-26 21:31:47.373646428 +0000 2023-12-26T21:35:44.668 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-12-26 21:31:47.373646428 +0000 2023-12-26T21:35:44.668 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2023-12-26 21:31:47.345647213 +0000 2023-12-26T21:35:44.668 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-26T21:35:44.772 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-12-26T21:35:44.772 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-12-26T21:35:44.772 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000498512 s, 1.0 MB/s 2023-12-26T21:35:44.775 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-26T21:35:44.820 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2023-12-26T21:35:44.904 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-26T21:35:44.904 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-26T21:35:44.904 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-12-26T21:35:44.904 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-26T21:35:44.904 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-12-26T21:35:44.905 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-12-26 21:35:15.956497901 +0000 2023-12-26T21:35:44.905 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-12-26 21:31:48.198623304 +0000 2023-12-26T21:35:44.905 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-12-26 21:31:48.198623304 +0000 2023-12-26T21:35:44.905 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2023-12-26 21:31:48.173624005 +0000 2023-12-26T21:35:44.905 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-26T21:35:45.008 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-12-26T21:35:45.009 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-12-26T21:35:45.009 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.00030161 s, 1.7 MB/s 2023-12-26T21:35:45.011 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-26T21:35:45.056 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2023-12-26T21:35:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-26T21:35:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-26T21:35:45.139 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-12-26T21:35:45.140 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-26T21:35:45.140 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-12-26T21:35:45.140 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-12-26 21:35:15.956497901 +0000 2023-12-26T21:35:45.140 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-12-26 21:31:49.014600432 +0000 2023-12-26T21:35:45.140 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-12-26 21:31:49.014600432 +0000 2023-12-26T21:35:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2023-12-26 21:31:48.989601133 +0000 2023-12-26T21:35:45.140 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-26T21:35:45.244 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-12-26T21:35:45.244 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-12-26T21:35:45.244 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000330388 s, 1.5 MB/s 2023-12-26T21:35:45.246 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-26T21:35:45.291 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-26T21:35:45.291 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-26T21:35:45.291 INFO:tasks.ceph:Generating config... 2023-12-26T21:35:45.292 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-26T21:35:45.293 INFO:tasks.ceph:[client] debug client = 20 2023-12-26T21:35:45.293 INFO:tasks.ceph:[client] debug ms = 1 2023-12-26T21:35:45.293 INFO:tasks.ceph:[client] pid file = 2023-12-26T21:35:45.293 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-26T21:35:45.293 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-26T21:35:45.293 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-12-26T21:35:45.293 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-26T21:35:45.293 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-26T21:35:45.293 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-26T21:35:45.293 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-26T21:35:45.293 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-26T21:35:45.293 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-26T21:35:45.293 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-26T21:35:45.293 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mgr] debug client = 20 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-26T21:35:45.294 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-26T21:35:45.294 INFO:tasks.ceph:[osd] bdev async discard = True 2023-12-26T21:35:45.294 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-12-26T21:35:45.294 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-12-26T21:35:45.294 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-26T21:35:45.294 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-26T21:35:45.295 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-26T21:35:45.295 INFO:tasks.ceph:Setting up mon.a... 2023-12-26T21:35:45.296 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-26T21:35:45.398 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.keyring 2023-12-26T21:35:45.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-26T21:35:45.498 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-26T21:35:45.565 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.131'), ('mon.b', '172.21.15.190'), ('mon.c', '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]')] 2023-12-26T21:35:45.565 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.131,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-26T21:35:45.566 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:45.566 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-26T21:35:45.608 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.131 --add b 172.21.15.190 --addv c '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:35:45.728 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: generated fsid ab603302-d88b-429a-be64-92b3c298adad 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = octopus 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 0 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:fsid ab603302-d88b-429a-be64-92b3c298adad 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:last_changed 2023-12-26T21:35:45.729599+0000 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:created 2023-12-26T21:35:45.729599+0000 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:min_mon_release 15 (octopus) 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:election_strategy: 1 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:2: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.c 2023-12-26T21:35:45.729 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-26T21:35:45.731 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-26T21:35:45.773 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ab603302-d88b-429a-be64-92b3c298adad... 2023-12-26T21:35:45.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-26T21:35:45.819 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-26T21:35:45.932 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2023-12-26T21:35:45.933 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2023-12-26T21:35:45.934 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.935 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on skipped message = False 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = ab603302-d88b-429a-be64-92b3c298adad 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2023-12-26T21:35:45.936 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.937 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: bdev async discard = True 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: bdev enable discard = True 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore allocator = bitmap 2023-12-26T21:35:45.938 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 1/20 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 1/20 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 4/10 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: osd op complaint time = 180 2023-12-26T21:35:45.939 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout: debug client = 20 2023-12-26T21:35:45.940 INFO:teuthology.orchestra.run.smithi131.stdout: debug finisher = 20 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.941 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: mon op complaint time = 120 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:35:45.942 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: client mount timeout = 600 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: debug client = 20 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = "" 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: rados mon op timeout = 900 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout: rados osd op timeout = 900 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2023-12-26T21:35:45.943 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout:[mds] 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: debug mds = 20 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: debug mds balancer = 20 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: mds debug frag = True 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: mds debug scatterstat = True 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: mds op complaint time = 180 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: mds verify scatter = True 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: osd op complaint time = 180 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: rados mon op timeout = 900 2023-12-26T21:35:45.944 INFO:teuthology.orchestra.run.smithi131.stdout: rados osd op timeout = 900 2023-12-26T21:35:45.945 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-12-26T21:35:45.945 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-12-26T21:35:45.945 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-26T21:35:45.945 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-12-26T21:35:45.945 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.945 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-12-26T21:35:45.945 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.946 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-26T21:35:45.947 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-12-26T21:35:45.948 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-12-26T21:35:45.949 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.131,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-12-26T21:35:45.949 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on skipped message = False 2023-12-26T21:35:45.949 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = ab603302-d88b-429a-be64-92b3c298adad 2023-12-26T21:35:45.949 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.949 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-12-26T21:35:45.949 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-12-26T21:35:45.949 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.949 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.950 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-12-26T21:35:45.951 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: bdev async discard = True 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: bdev enable discard = True 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore allocator = bitmap 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 1/20 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 1/20 2023-12-26T21:35:45.952 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-12-26T21:35:45.953 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-12-26T21:35:45.953 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 4/10 2023-12-26T21:35:45.953 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-12-26T21:35:45.953 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-12-26T21:35:45.953 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-12-26T21:35:45.953 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-12-26T21:35:45.953 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-12-26T21:35:45.953 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-12-26T21:35:45.953 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-12-26T21:35:45.954 INFO:teuthology.orchestra.run.smithi190.stdout: debug finisher = 20 2023-12-26T21:35:45.955 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.955 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-12-26T21:35:45.955 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-12-26T21:35:45.955 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-12-26T21:35:45.955 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-12-26T21:35:45.955 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-12-26T21:35:45.955 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-12-26T21:35:45.955 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-12-26T21:35:45.955 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-26T21:35:45.956 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-26T21:35:45.957 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.957 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-12-26T21:35:45.957 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-12-26T21:35:45.957 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.957 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-12-26T21:35:45.957 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-12-26T21:35:45.957 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-26T21:35:45.957 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-12-26T21:35:45.958 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-12-26T21:35:45.959 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-12-26T21:35:45.960 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-12-26T21:35:45.960 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-12-26T21:35:45.960 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2023-12-26T21:35:45.960 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-12-26T21:35:45.960 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2023-12-26T21:35:45.960 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-12-26T21:35:45.961 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-12-26T21:35:45.961 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-12-26T21:35:45.968 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-26T21:35:45.968 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-26T21:35:46.073 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-26T21:35:46.073 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:46.074 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-26T21:35:46.115 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:46.116 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-26T21:35:46.197 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:35:46.198 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:46.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-26T21:35:46.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-26T21:35:46.331 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:35:46.331 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:35:46.414 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T21:35:46.414 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:35:46.414 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-26T21:35:46.414 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-26T21:35:46.508 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:35:46.508 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:35:46.590 INFO:tasks.ceph:Setting up mon nodes... 2023-12-26T21:35:46.590 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-26T21:35:46.590 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-26T21:35:46.706 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-26T21:35:46.712 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-26T21:35:46.865 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-26T21:35:46.870 INFO:tasks.ceph:Setting up mds nodes... 2023-12-26T21:35:46.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-26T21:35:46.987 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-26T21:35:46.992 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-26T21:35:47.060 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-12-26T21:35:47.175 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-12-26T21:35:47.180 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-12-26T21:35:47.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-26T21:35:47.365 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-26T21:35:47.370 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-26T21:35:47.438 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-12-26T21:35:47.553 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-12-26T21:35:47.558 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-12-26T21:35:47.625 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-26T21:35:47.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-26T21:35:47.716 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-26T21:35:47.748 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-12-26T21:35:47.836 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-12-26T21:35:47.867 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-26T21:35:47.958 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-26T21:35:47.988 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-12-26T21:35:48.077 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-12-26T21:35:48.108 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-26T21:35:48.108 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-26T21:35:48.109 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-26T21:35:48.176 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-26T21:35:48.177 INFO:tasks.ceph:role: osd.0 2023-12-26T21:35:48.177 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:35:48.177 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-26T21:35:48.251 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-26T21:35:48.683 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2023-12-26T21:35:48.688 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2023-12-26T21:35:48.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-26T21:35:48.878 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-26T21:35:48.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-26T21:35:49.025 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-26T21:35:49.026 INFO:tasks.ceph:role: osd.1 2023-12-26T21:35:49.026 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:35:49.026 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-26T21:35:49.100 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-26T21:35:49.100 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-26T21:35:49.100 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-26T21:35:49.100 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-26T21:35:49.101 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-26T21:35:49.101 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2023-12-26T21:35:49.101 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-26T21:35:49.101 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-26T21:35:49.101 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-26T21:35:49.101 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-26T21:35:49.508 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2023-12-26T21:35:49.513 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2023-12-26T21:35:49.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-26T21:35:49.650 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-26T21:35:49.727 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-26T21:35:49.794 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-26T21:35:49.794 INFO:tasks.ceph:role: osd.2 2023-12-26T21:35:49.794 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:35:49.794 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-26T21:35:49.870 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-26T21:35:49.870 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-26T21:35:49.870 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-26T21:35:49.870 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-26T21:35:49.870 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-26T21:35:49.871 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2023-12-26T21:35:49.871 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-26T21:35:49.871 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-26T21:35:49.871 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-26T21:35:49.871 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-26T21:35:50.321 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2023-12-26T21:35:50.325 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2023-12-26T21:35:50.325 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-26T21:35:50.530 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-26T21:35:50.604 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-26T21:35:50.671 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-26T21:35:50.672 INFO:tasks.ceph:role: osd.3 2023-12-26T21:35:50.672 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:35:50.672 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-26T21:35:50.746 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-26T21:35:50.746 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-26T21:35:50.746 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-26T21:35:50.747 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-26T21:35:50.747 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-26T21:35:50.747 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2023-12-26T21:35:50.747 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-26T21:35:50.747 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-26T21:35:50.747 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-26T21:35:50.747 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-26T21:35:51.181 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2023-12-26T21:35:51.185 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2023-12-26T21:35:51.185 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-26T21:35:51.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-26T21:35:51.386 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:35:51.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:51.504+0000 7f41c73b9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-26T21:35:51.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:51.504+0000 7f41c73b9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-26T21:35:51.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:51.504+0000 7f41c73b9540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-26T21:35:51.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:51.504+0000 7f41c73b9540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-26T21:35:51.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:51.504+0000 7f41c73b9540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-26T21:35:51.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:51.506+0000 7f41c73b9540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-26T21:35:52.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:52.047+0000 7f41c73b9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:35:52.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:52.809+0000 7f41c73b9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:35:53.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:53.575+0000 7f41c73b9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:35:54.402 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-26T21:35:54.475 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:35:54.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:54.575+0000 7fbe5e0f0540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-26T21:35:54.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:54.575+0000 7fbe5e0f0540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-26T21:35:54.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:54.575+0000 7fbe5e0f0540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-26T21:35:54.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:54.575+0000 7fbe5e0f0540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-26T21:35:54.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:54.575+0000 7fbe5e0f0540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-26T21:35:54.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:54.577+0000 7fbe5e0f0540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-26T21:35:55.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:55.116+0000 7fbe5e0f0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:35:55.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:55.878+0000 7fbe5e0f0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:35:56.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:56.642+0000 7fbe5e0f0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:35:57.460 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-26T21:35:57.532 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:35:57.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:57.630+0000 7f0751eed540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-26T21:35:57.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:57.630+0000 7f0751eed540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-26T21:35:57.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:57.630+0000 7f0751eed540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-26T21:35:57.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:57.630+0000 7f0751eed540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-26T21:35:57.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:57.630+0000 7f0751eed540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-26T21:35:57.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:57.632+0000 7f0751eed540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-26T21:35:58.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:58.178+0000 7f0751eed540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:35:58.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:58.939+0000 7f0751eed540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:35:59.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:35:59.707+0000 7f0751eed540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:00.522 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-26T21:36:00.592 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:36:00.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:00.690+0000 7fb9bdfa1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-26T21:36:00.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:00.691+0000 7fb9bdfa1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-26T21:36:00.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:00.691+0000 7fb9bdfa1540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-26T21:36:00.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:00.691+0000 7fb9bdfa1540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-26T21:36:00.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:00.691+0000 7fb9bdfa1540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-26T21:36:00.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:00.692+0000 7fb9bdfa1540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-26T21:36:01.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:01.233+0000 7fb9bdfa1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:01.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:01.993+0000 7fb9bdfa1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:02.766+0000 7fb9bdfa1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:03.588 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-26T21:36:03.658 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-26T21:36:03.733 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-26T21:36:03.733 INFO:tasks.ceph:role: osd.4 2023-12-26T21:36:03.734 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T21:36:03.734 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-26T21:36:03.809 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-26T21:36:03.810 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-26T21:36:03.810 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-26T21:36:03.810 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-26T21:36:03.810 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-26T21:36:03.810 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-12-26T21:36:03.810 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-26T21:36:03.810 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-26T21:36:03.810 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-26T21:36:03.810 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-26T21:36:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-12-26T21:36:04.239 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-12-26T21:36:04.240 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-26T21:36:04.437 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-26T21:36:04.514 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-26T21:36:04.582 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-26T21:36:04.582 INFO:tasks.ceph:role: osd.5 2023-12-26T21:36:04.582 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T21:36:04.583 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-26T21:36:04.657 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-26T21:36:04.658 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-26T21:36:04.658 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-26T21:36:04.658 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-26T21:36:04.658 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-26T21:36:04.658 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-12-26T21:36:04.659 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-26T21:36:04.659 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-26T21:36:04.659 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-26T21:36:04.659 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-26T21:36:05.070 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-12-26T21:36:05.076 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-12-26T21:36:05.076 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-26T21:36:05.213 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-26T21:36:05.287 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-26T21:36:05.355 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-26T21:36:05.355 INFO:tasks.ceph:role: osd.6 2023-12-26T21:36:05.356 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T21:36:05.356 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-26T21:36:05.430 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-26T21:36:05.431 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-26T21:36:05.431 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-26T21:36:05.431 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-26T21:36:05.431 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-26T21:36:05.431 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-12-26T21:36:05.431 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-26T21:36:05.431 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-26T21:36:05.431 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-26T21:36:05.431 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-26T21:36:05.880 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-12-26T21:36:05.884 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-12-26T21:36:05.884 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-26T21:36:06.084 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-26T21:36:06.160 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-26T21:36:06.228 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-26T21:36:06.229 INFO:tasks.ceph:role: osd.7 2023-12-26T21:36:06.229 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T21:36:06.229 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-26T21:36:06.303 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-26T21:36:06.303 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-26T21:36:06.303 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-26T21:36:06.304 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-26T21:36:06.304 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-26T21:36:06.304 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-12-26T21:36:06.304 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-26T21:36:06.304 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-26T21:36:06.304 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-26T21:36:06.304 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-26T21:36:06.748 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-12-26T21:36:06.752 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-12-26T21:36:06.752 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-26T21:36:06.872 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-26T21:36:06.945 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:36:07.058 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:07.058+0000 7fa7f7bc7540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-26T21:36:07.059 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:07.059+0000 7fa7f7bc7540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-26T21:36:07.059 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:07.059+0000 7fa7f7bc7540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-26T21:36:07.059 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:07.059+0000 7fa7f7bc7540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-26T21:36:07.059 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:07.059+0000 7fa7f7bc7540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-26T21:36:07.061 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:07.061+0000 7fa7f7bc7540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-26T21:36:07.610 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:07.610+0000 7fa7f7bc7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:08.376 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:08.376+0000 7fa7f7bc7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:09.142 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:09.141+0000 7fa7f7bc7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:09.962 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-26T21:36:10.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:36:10.135 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:10.132+0000 7f7640521540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-26T21:36:10.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:10.133+0000 7f7640521540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-26T21:36:10.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:10.133+0000 7f7640521540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-26T21:36:10.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:10.133+0000 7f7640521540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-26T21:36:10.137 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:10.133+0000 7f7640521540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-26T21:36:10.137 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:10.135+0000 7f7640521540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-26T21:36:10.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:10.675+0000 7f7640521540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:11.437 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:11.436+0000 7f7640521540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:12.199 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:12.199+0000 7f7640521540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:13.019 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-26T21:36:13.087 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:36:13.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:36:13.162 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:36:13.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:13.185+0000 7ff6533c2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-26T21:36:13.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:13.185+0000 7ff6533c2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-26T21:36:13.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:13.185+0000 7ff6533c2540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-26T21:36:13.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:13.186+0000 7ff6533c2540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-26T21:36:13.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:13.186+0000 7ff6533c2540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-26T21:36:13.187 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:13.187+0000 7ff6533c2540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-26T21:36:13.730 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:13.730+0000 7ff6533c2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:14.493 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:14.493+0000 7ff6533c2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:15.254 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:15.254+0000 7ff6533c2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:16.072 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-26T21:36:16.140 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:36:16.238 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:16.237+0000 7fc9806f8540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-26T21:36:16.238 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:16.238+0000 7fc9806f8540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-26T21:36:16.238 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:16.238+0000 7fc9806f8540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-26T21:36:16.238 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:16.238+0000 7fc9806f8540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-26T21:36:16.238 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:16.238+0000 7fc9806f8540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-26T21:36:16.239 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:16.239+0000 7fc9806f8540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-26T21:36:16.778 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:16.778+0000 7fc9806f8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:17.540 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:17.540+0000 7fc9806f8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:18.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:18.307+0000 7fc9806f8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-12-26T21:36:19.118 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-26T21:36:19.186 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-26T21:36:19.186 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:19.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-26T21:36:19.261 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:19.261 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-26T21:36:19.371 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:19.371 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-12-26T21:36:19.479 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:19.479 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-26T21:36:19.587 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:19.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-26T21:36:19.695 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:19.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-26T21:36:19.802 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:19.802 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-26T21:36:19.909 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:19.909 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-26T21:36:19.980 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:19.980 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-26T21:36:20.087 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:20.087 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-12-26T21:36:20.194 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:20.194 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-26T21:36:20.301 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:20.302 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-26T21:36:20.409 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:20.409 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-26T21:36:20.515 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:20.515 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-26T21:36:20.622 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:20.622 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-26T21:36:20.666 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:20.666 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-12-26T21:36:20.749 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:20.749 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-26T21:36:20.792 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:20.792 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-12-26T21:36:20.874 INFO:tasks.ceph:Adding keys to all mons... 2023-12-26T21:36:20.874 DEBUG:teuthology.orchestra.run.smithi131:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-26T21:36:20.879 DEBUG:teuthology.orchestra.run.smithi190:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.y] 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ== 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw== 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ== 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.0] 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg== 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.1] 2023-12-26T21:36:20.939 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw== 2023-12-26T21:36:20.940 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.2] 2023-12-26T21:36:20.940 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA== 2023-12-26T21:36:20.940 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.3] 2023-12-26T21:36:20.940 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ== 2023-12-26T21:36:20.940 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.x] 2023-12-26T21:36:20.940 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw== 2023-12-26T21:36:20.940 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2023-12-26T21:36:20.940 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ== 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw== 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.4] 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw== 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.5] 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A== 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.6] 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA== 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.7] 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA== 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA== 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout:[client.2] 2023-12-26T21:36:20.941 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw== 2023-12-26T21:36:20.942 INFO:teuthology.orchestra.run.smithi131.stdout:[client.1] 2023-12-26T21:36:20.942 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA== 2023-12-26T21:36:20.942 INFO:teuthology.orchestra.run.smithi131.stdout:[client.3] 2023-12-26T21:36:20.942 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA== 2023-12-26T21:36:20.980 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.y] 2023-12-26T21:36:20.980 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ== 2023-12-26T21:36:20.980 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-12-26T21:36:20.980 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw== 2023-12-26T21:36:20.980 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ== 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg== 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw== 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA== 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ== 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.x] 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw== 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ== 2023-12-26T21:36:20.981 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw== 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.4] 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw== 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.5] 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A== 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.6] 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA== 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.7] 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA== 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA== 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout:[client.2] 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw== 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2023-12-26T21:36:20.982 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA== 2023-12-26T21:36:20.983 INFO:teuthology.orchestra.run.smithi190.stdout:[client.3] 2023-12-26T21:36:20.983 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA== 2023-12-26T21:36:20.984 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-26T21:36:20.988 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-26T21:36:21.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-26T21:36:21.130 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-26T21:36:21.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-26T21:36:21.238 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-26T21:36:21.342 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:21.346 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:21.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:21.450 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:21.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:21.555 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:21.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:21.661 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:21.763 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-26T21:36:21.767 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-26T21:36:21.870 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-26T21:36:21.875 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-26T21:36:21.979 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-26T21:36:21.983 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-26T21:36:22.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:22.093 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:22.194 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:22.199 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:22.300 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:22.329 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:22.430 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:22.435 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-26T21:36:22.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-26T21:36:22.541 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-26T21:36:22.645 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-26T21:36:22.650 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-26T21:36:22.754 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-26T21:36:22.758 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-26T21:36:22.863 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-26T21:36:22.867 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-26T21:36:22.973 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-26T21:36:22.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-26T21:36:23.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-26T21:36:23.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-26T21:36:23.224 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-26T21:36:23.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-26T21:36:23.408 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-26T21:36:23.475 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-26T21:36:23.543 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-26T21:36:23.656 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-26T21:36:23.725 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:36:23.730 DEBUG:teuthology.orchestra.run.smithi190:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-26T21:36:23.774 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-26T21:36:23.774 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-26T21:36:23.774 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-26T21:36:23.814 INFO:tasks.ceph.mon.a:Started 2023-12-26T21:36:23.814 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-26T21:36:23.814 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-26T21:36:23.818 INFO:tasks.ceph.mon.b:Started 2023-12-26T21:36:23.819 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-26T21:36:23.819 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-26T21:36:23.823 INFO:tasks.ceph.mon.c:Started 2023-12-26T21:36:23.823 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-26T21:36:23.823 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-26T21:36:23.823 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-26T21:36:23.827 INFO:tasks.ceph.mgr.y:Started 2023-12-26T21:36:23.827 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-26T21:36:23.827 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-26T21:36:23.832 INFO:tasks.ceph.mgr.x:Started 2023-12-26T21:36:23.832 INFO:tasks.ceph:Setting crush tunables to default 2023-12-26T21:36:23.832 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd crush tunables default 2023-12-26T21:36:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.023+0000 7f7ad93ff700 1 Processor -- start 2023-12-26T21:36:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.023+0000 7f7ad93ff700 1 -- start start 2023-12-26T21:36:24.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad93ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ad413c510 0x7f7ad40c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:24.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad93ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7ad406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:24.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad93ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad40686c0 con 0x7f7ad40cc100 2023-12-26T21:36:24.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad93ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad4069520 con 0x7f7ad413c510 2023-12-26T21:36:24.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad93ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad406c250 con 0x7f7ad406b8c0 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad27fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7ad406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad2ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ad413c510 0x7f7ad40c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad27fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7ad406bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:60192/0 (socket says 172.21.15.131:60192) 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad2ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ad413c510 0x7f7ad40c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:54804/0 (socket says 172.21.15.131:54804) 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.025+0000 7f7ad27fc700 1 -- 172.21.15.131:0/3010678666 learned_addr learned my addr 172.21.15.131:0/3010678666 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.026+0000 7f7ad2ffd700 1 -- 172.21.15.131:0/3010678666 >> v2:172.21.15.190:3300/0 conn(0x7f7ad413c510 msgr2=0x7f7ad40c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.026+0000 7f7ad2ffd700 1 -- 172.21.15.131:0/3010678666 >> v2:172.21.15.190:3300/0 conn(0x7f7ad413c510 msgr2=0x7f7ad40c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.026+0000 7f7ad2ffd700 1 --2- 172.21.15.131:0/3010678666 >> v2:172.21.15.190:3300/0 conn(0x7f7ad413c510 0x7f7ad40c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.026+0000 7f7ad2ffd700 1 --2- 172.21.15.131:0/3010678666 >> v2:172.21.15.190:3300/0 conn(0x7f7ad413c510 0x7f7ad40c8a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:36:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.026+0000 7f7ad27fc700 1 -- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 msgr2=0x7f7ad406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-26T21:36:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.026+0000 7f7ad27fc700 1 -- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 msgr2=0x7f7ad406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-26T21:36:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.026+0000 7f7ad27fc700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7ad406bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-12-26T21:36:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.026+0000 7f7ad27fc700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7ad406bd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:36:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.221+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323765316 0 0) 0x7f7ad40686c0 con 0x7f7ad40cc100 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.221+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7abc003570 con 0x7f7ad40cc100 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.221+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913123360 0 0) 0x7f7abc003570 con 0x7f7ad40cc100 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.221+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 msgr2=0x7f7ad406bd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.221+0000 7f7ad1ffb700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7ad406bd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.221+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 >> v2:172.21.15.190:3300/0 conn(0x7f7ad413c510 msgr2=0x7f7ad40c8a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.221+0000 7f7ad1ffb700 1 --2- 172.21.15.131:0/3010678666 >> v2:172.21.15.190:3300/0 conn(0x7f7ad413c510 0x7f7ad40c8a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.221+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad4068910 con 0x7f7ad40cc100 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.221+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f7acc0017f0 con 0x7f7ad40cc100 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 >> v1:172.21.15.131:6789/0 conn(0x7f7ad40cc100 legacy=0x7f7ad40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad1ffb700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7abc003ae0 0x7f7abc003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad1ffb700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7abc004810 0x7f7abc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad1ffb700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7abc005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad4069520 con 0x7f7abc003ae0 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad406c250 con 0x7f7abc004810 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc00bc70 con 0x7f7ad406b8c0 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad2ffd700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7abc003ae0 0x7f7abc003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad27fc700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7abc004810 0x7f7abc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad37fe700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7abc005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad2ffd700 1 -- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 msgr2=0x7f7abc005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad2ffd700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7abc005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad2ffd700 1 -- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7abc004810 msgr2=0x7f7abc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad2ffd700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7abc004810 0x7f7abc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad2ffd700 1 -- 172.21.15.131:0/3010678666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc003570 con 0x7f7abc003ae0 2023-12-26T21:36:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad27fc700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7abc004810 0x7f7abc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.222+0000 7f7ad37fe700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad406b8c0 0x7f7abc005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.223+0000 7f7ad2ffd700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7abc003ae0 0x7f7abc003ef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400bb40 tx=0x7f7ac4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed385bb1e93a5ceb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.223+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4005a00 con 0x7f7abc003ae0 2023-12-26T21:36:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.223+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac4005b80 con 0x7f7abc003ae0 2023-12-26T21:36:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.223+0000 7f7ad1ffb700 1 -- 172.21.15.131:0/3010678666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400d5d0 con 0x7f7abc003ae0 2023-12-26T21:36:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.224+0000 7f7ad93ff700 1 -- 172.21.15.131:0/3010678666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7abc003ae0 msgr2=0x7f7abc003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.224+0000 7f7ad93ff700 1 --2- 172.21.15.131:0/3010678666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7abc003ae0 0x7f7abc003ef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400bb40 tx=0x7f7ac4008040 comp rx=0 tx=0).stop 2023-12-26T21:36:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.224+0000 7f7ad37fe700 1 -- 172.21.15.131:0/3010678666 reap_dead start 2023-12-26T21:36:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.224+0000 7f7ad93ff700 1 -- 172.21.15.131:0/3010678666 shutdown_connections 2023-12-26T21:36:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.224+0000 7f7ad93ff700 1 -- 172.21.15.131:0/3010678666 >> 172.21.15.131:0/3010678666 conn(0x7f7ad40ce3e0 msgr2=0x7f7ad40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.224+0000 7f7ad93ff700 1 -- 172.21.15.131:0/3010678666 shutdown_connections 2023-12-26T21:36:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.224+0000 7f7ad93ff700 1 -- 172.21.15.131:0/3010678666 wait complete. 2023-12-26T21:36:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.226+0000 7f7ad93ff700 1 Processor -- start 2023-12-26T21:36:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.227+0000 7f7ad93ff700 1 -- start start 2023-12-26T21:36:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.228+0000 7f7ad93ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc003ae0 0x7f7ad4156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.228+0000 7f7ad93ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad40b2150 0x7f7ad40b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.228+0000 7f7ad93ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad413b670 0x7f7ad413bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.228+0000 7f7ad93ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad413c000 con 0x7f7ad413b670 2023-12-26T21:36:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.228+0000 7f7ad93ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad415b380 con 0x7f7ad40b2150 2023-12-26T21:36:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.228+0000 7f7ad93ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40b2cc0 con 0x7f7abc003ae0 2023-12-26T21:36:24.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.228+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad413b670 0x7f7ad413bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.228+0000 7f7ad2ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc003ae0 0x7f7ad4156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.228+0000 7f7ad27fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad40b2150 0x7f7ad40b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad27fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad40b2150 0x7f7ad40b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:54870/0 (socket says 172.21.15.131:54870) 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad2ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc003ae0 0x7f7ad4156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:60268/0 (socket says 172.21.15.131:60268) 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad27fc700 1 -- 172.21.15.131:0/4112454593 learned_addr learned my addr 172.21.15.131:0/4112454593 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad413b670 0x7f7ad413bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45456/0 (socket says 172.21.15.131:45456) 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad37fe700 1 -- 172.21.15.131:0/4112454593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc003ae0 msgr2=0x7f7ad4156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad37fe700 1 --2- 172.21.15.131:0/4112454593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc003ae0 0x7f7ad4156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad37fe700 1 -- 172.21.15.131:0/4112454593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad40b2150 msgr2=0x7f7ad40b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad37fe700 1 --2- 172.21.15.131:0/4112454593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad40b2150 0x7f7ad40b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad37fe700 1 -- 172.21.15.131:0/4112454593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad40b2fa0 con 0x7f7ad413b670 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7ad37fe700 1 --2- 172.21.15.131:0/4112454593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad413b670 0x7f7ad413bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4143300 tx=0x7f7acc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7abaffd700 1 -- 172.21.15.131:0/4112454593 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc00c750 con 0x7f7ad413b670 2023-12-26T21:36:24.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.229+0000 7f7abaffd700 1 -- 172.21.15.131:0/4112454593 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc00c8b0 con 0x7f7ad413b670 2023-12-26T21:36:24.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.230+0000 7f7ad93ff700 1 -- 172.21.15.131:0/4112454593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad40b3280 con 0x7f7ad413b670 2023-12-26T21:36:24.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.230+0000 7f7abaffd700 1 -- 172.21.15.131:0/4112454593 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc00cc70 con 0x7f7ad413b670 2023-12-26T21:36:24.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.230+0000 7f7ad93ff700 1 -- 172.21.15.131:0/4112454593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad40c5c50 con 0x7f7ad413b670 2023-12-26T21:36:24.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.230+0000 7f7abaffd700 1 -- 172.21.15.131:0/4112454593 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7acc01f460 con 0x7f7ad413b670 2023-12-26T21:36:24.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.230+0000 7f7abaffd700 1 -- 172.21.15.131:0/4112454593 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7acc015780 con 0x7f7ad413b670 2023-12-26T21:36:24.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.231+0000 7f7ad93ff700 1 -- 172.21.15.131:0/4112454593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad40c5ec0 con 0x7f7ad413b670 2023-12-26T21:36:24.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.234+0000 7f7abaffd700 1 -- 172.21.15.131:0/4112454593 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f7ad40c5ec0 con 0x7f7ad413b670 2023-12-26T21:36:24.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.312+0000 7f7ad93ff700 1 -- 172.21.15.131:0/4112454593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7ad40cd5c0 con 0x7f7ad413b670 2023-12-26T21:36:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.319+0000 7f7abaffd700 1 -- 172.21.15.131:0/4112454593 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7acc006ba0 con 0x7f7ad413b670 2023-12-26T21:36:24.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.321+0000 7f7ad93ff700 1 -- 172.21.15.131:0/4112454593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad413b670 msgr2=0x7f7ad413bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:24.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.321+0000 7f7ad93ff700 1 --2- 172.21.15.131:0/4112454593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad413b670 0x7f7ad413bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4143300 tx=0x7f7acc009040 comp rx=0 tx=0).stop 2023-12-26T21:36:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.321+0000 7f7ad93ff700 1 -- 172.21.15.131:0/4112454593 shutdown_connections 2023-12-26T21:36:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.321+0000 7f7ad93ff700 1 --2- 172.21.15.131:0/4112454593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad413b670 0x7f7ad413bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.321+0000 7f7ad93ff700 1 --2- 172.21.15.131:0/4112454593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc003ae0 0x7f7ad4156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.321+0000 7f7ad93ff700 1 --2- 172.21.15.131:0/4112454593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad40b2150 0x7f7ad40b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.321+0000 7f7ad93ff700 1 -- 172.21.15.131:0/4112454593 >> 172.21.15.131:0/4112454593 conn(0x7f7ad40ce3e0 msgr2=0x7f7ad413a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.321+0000 7f7ad93ff700 1 -- 172.21.15.131:0/4112454593 shutdown_connections 2023-12-26T21:36:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:24.321+0000 7f7ad93ff700 1 -- 172.21.15.131:0/4112454593 wait complete. 2023-12-26T21:36:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2023-12-26T21:36:24.339 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-26T21:36:24.339 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:24.339 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-26T21:36:24.407 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:24.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-26T21:36:24.515 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:24.515 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-26T21:36:24.625 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2023-12-26T21:36:24.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-26T21:36:24.668 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:24.668+0000 7f21292781c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:36:24.689 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:24.689+0000 7f442e0ca1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:36:24.739 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:24.739 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-26T21:36:24.765 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:24.765+0000 7f21292781c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:36:24.793 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:24.792+0000 7f442e0ca1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:36:24.807 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:24.807 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-26T21:36:24.859 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:24.859+0000 7f21292781c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:36:24.882 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:24.882+0000 7f442e0ca1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:36:24.915 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:24.915 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-26T21:36:24.951 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:24.951+0000 7f21292781c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:36:24.970 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:24.969+0000 7f442e0ca1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:36:25.019 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-12-26T21:36:25.019 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-26T21:36:25.128 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 81a77a72-fe92-4844-a340-b892cc90cd2d 0 2023-12-26T21:36:25.174 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:25.170+0000 7f21292781c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:36:25.175 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:25.175+0000 7f442e0ca1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:36:25.355 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.355+0000 7f9a975ff700 1 Processor -- start 2023-12-26T21:36:25.355 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.355+0000 7f9a975ff700 1 -- start start 2023-12-26T21:36:25.356 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.356+0000 7f9a975ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a90159920 0x7f9a90159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.356 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.356+0000 7f9a975ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900c5420 0x7f9a900c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.356 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.356+0000 7f9a975ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a900c6810 con 0x7f9a900ced90 2023-12-26T21:36:25.356 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.356+0000 7f9a975ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9015a230 con 0x7f9a90159920 2023-12-26T21:36:25.357 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.356+0000 7f9a975ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900c5c00 con 0x7f9a900c5420 2023-12-26T21:36:25.357 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:25.356+0000 7f442e0ca1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.357+0000 7f9a9539b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a90159920 0x7f9a90159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.357+0000 7f9a9539b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a90159920 0x7f9a90159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.190:51094/0 (socket says 172.21.15.190:51094) 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.357+0000 7f9a9539b700 1 -- 172.21.15.190:0/2334068441 learned_addr learned my addr 172.21.15.190:0/2334068441 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.357+0000 7f9a94b9a700 1 --2- 172.21.15.190:0/2334068441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900c5420 0x7f9a900c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.357+0000 7f9a9539b700 1 -- 172.21.15.190:0/2334068441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900c5420 msgr2=0x7f9a900c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.357+0000 7f9a9539b700 1 --2- 172.21.15.190:0/2334068441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900c5420 0x7f9a900c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.357+0000 7f9a9539b700 1 -- 172.21.15.190:0/2334068441 >> v1:172.21.15.131:6789/0 conn(0x7f9a900ced90 legacy=0x7f9a9015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.357+0000 7f9a9539b700 1 -- 172.21.15.190:0/2334068441 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900c5ee0 con 0x7f9a90159920 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.357+0000 7f9a87fff700 1 -- 172.21.15.190:0/2334068441 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516561911 0 0) 0x7f9a900c6810 con 0x7f9a900ced90 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a9539b700 1 --2- 172.21.15.190:0/2334068441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a90159920 0x7f9a90159cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009990 tx=0x7f9a8c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f213026bb57917 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a87fff700 1 -- 172.21.15.190:0/2334068441 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c009e80 con 0x7f9a90159920 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a87fff700 1 -- 172.21.15.190:0/2334068441 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c005c10 con 0x7f9a90159920 2023-12-26T21:36:25.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a87fff700 1 -- 172.21.15.190:0/2334068441 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00b650 con 0x7f9a90159920 2023-12-26T21:36:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a975ff700 1 -- 172.21.15.190:0/2334068441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a90159920 msgr2=0x7f9a90159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a975ff700 1 --2- 172.21.15.190:0/2334068441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a90159920 0x7f9a90159cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009990 tx=0x7f9a8c009220 comp rx=0 tx=0).stop 2023-12-26T21:36:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a975ff700 1 -- 172.21.15.190:0/2334068441 shutdown_connections 2023-12-26T21:36:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a975ff700 1 --2- 172.21.15.190:0/2334068441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a90159920 0x7f9a90159cf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a975ff700 1 --2- 172.21.15.190:0/2334068441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900c5420 0x7f9a900c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.358+0000 7f9a975ff700 1 -- 172.21.15.190:0/2334068441 >> 172.21.15.190:0/2334068441 conn(0x7f9a90099980 msgr2=0x7f9a9006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.359+0000 7f9a975ff700 1 -- 172.21.15.190:0/2334068441 shutdown_connections 2023-12-26T21:36:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.359+0000 7f9a975ff700 1 -- 172.21.15.190:0/2334068441 wait complete. 2023-12-26T21:36:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.361+0000 7f9a975ff700 1 Processor -- start 2023-12-26T21:36:25.361 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.361+0000 7f9a975ff700 1 -- start start 2023-12-26T21:36:25.361 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:25.361+0000 7f21292781c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a975ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a900c5420 0x7f9a900c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a975ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900ced90 0x7f9a90154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a975ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a90159920 0x7f9a900c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a975ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900c8e00 con 0x7f9a90159920 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a975ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a90159010 con 0x7f9a900c5420 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a975ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900c81f0 con 0x7f9a900ced90 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a94b9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900ced90 0x7f9a90154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a94b9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900ced90 0x7f9a90154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:34092/0 (socket says 172.21.15.190:34092) 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a94b9a700 1 -- 172.21.15.190:0/113782432 learned_addr learned my addr 172.21.15.190:0/113782432 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a9539b700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a900c5420 0x7f9a900c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.362+0000 7f9a95b9c700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a90159920 0x7f9a900c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a94b9a700 1 -- 172.21.15.190:0/113782432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a900c5420 msgr2=0x7f9a900c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a94b9a700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a900c5420 0x7f9a900c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a94b9a700 1 -- 172.21.15.190:0/113782432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a90159920 msgr2=0x7f9a900c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a94b9a700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a90159920 0x7f9a900c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a94b9a700 1 -- 172.21.15.190:0/113782432 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c009480 con 0x7f9a900ced90 2023-12-26T21:36:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a9539b700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a900c5420 0x7f9a900c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a94b9a700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900ced90 0x7f9a90154f50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000e930 tx=0x7f9a800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a857fa700 1 -- 172.21.15.190:0/113782432 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80013680 con 0x7f9a900ced90 2023-12-26T21:36:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a857fa700 1 -- 172.21.15.190:0/113782432 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a800137e0 con 0x7f9a900ced90 2023-12-26T21:36:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a975ff700 1 -- 172.21.15.190:0/113782432 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a900c84d0 con 0x7f9a900ced90 2023-12-26T21:36:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a857fa700 1 -- 172.21.15.190:0/113782432 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80013b80 con 0x7f9a900ced90 2023-12-26T21:36:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.363+0000 7f9a975ff700 1 -- 172.21.15.190:0/113782432 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a900c8a90 con 0x7f9a900ced90 2023-12-26T21:36:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.364+0000 7f9a857fa700 1 -- 172.21.15.190:0/113782432 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9a80005c80 con 0x7f9a900ced90 2023-12-26T21:36:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.364+0000 7f9a857fa700 1 -- 172.21.15.190:0/113782432 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9a8001c450 con 0x7f9a900ced90 2023-12-26T21:36:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.364+0000 7f9a975ff700 1 -- 172.21.15.190:0/113782432 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a900657e0 con 0x7f9a900ced90 2023-12-26T21:36:25.367 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.367+0000 7f9a857fa700 1 -- 172.21.15.190:0/113782432 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9a80021060 con 0x7f9a900ced90 2023-12-26T21:36:25.444 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:25.444+0000 7f442e0ca1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:36:25.451 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:25.451+0000 7f21292781c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:36:25.455 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.455+0000 7f9a975ff700 1 -- 172.21.15.190:0/113782432 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "81a77a72-fe92-4844-a340-b892cc90cd2d", "id": 0} v 0) v1 -- 0x7f9a900c8f70 con 0x7f9a900ced90 2023-12-26T21:36:25.463 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.463+0000 7f9a857fa700 1 -- 172.21.15.190:0/113782432 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "81a77a72-fe92-4844-a340-b892cc90cd2d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9a80017070 con 0x7f9a900ced90 2023-12-26T21:36:25.463 INFO:teuthology.orchestra.run.smithi190.stdout:0 2023-12-26T21:36:25.464 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.464+0000 7f9a975ff700 1 -- 172.21.15.190:0/113782432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900ced90 msgr2=0x7f9a90154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.464 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.464+0000 7f9a975ff700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900ced90 0x7f9a90154f50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000e930 tx=0x7f9a800091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.464 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.464+0000 7f9a975ff700 1 -- 172.21.15.190:0/113782432 shutdown_connections 2023-12-26T21:36:25.464 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.464+0000 7f9a975ff700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a90159920 0x7f9a900c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.464 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.464+0000 7f9a975ff700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a900c5420 0x7f9a900c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.464+0000 7f9a975ff700 1 --2- 172.21.15.190:0/113782432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a900ced90 0x7f9a90154f50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.464+0000 7f9a975ff700 1 -- 172.21.15.190:0/113782432 >> 172.21.15.190:0/113782432 conn(0x7f9a90099980 msgr2=0x7f9a90068670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:25.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.464+0000 7f9a975ff700 1 -- 172.21.15.190:0/113782432 shutdown_connections 2023-12-26T21:36:25.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.464+0000 7f9a975ff700 1 -- 172.21.15.190:0/113782432 wait complete. 2023-12-26T21:36:25.482 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc 1 2023-12-26T21:36:25.605 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:25.605+0000 7f442e0ca1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:36:25.624 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:25.624+0000 7f21292781c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:36:25.668 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.668+0000 7f4b8edff700 1 Processor -- start 2023-12-26T21:36:25.668 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.668+0000 7f4b8edff700 1 -- start start 2023-12-26T21:36:25.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b8edff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b8815d240 0x7f4b880ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b8edff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b880ad390 0x7f4b880aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b8edff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b880a98b0 con 0x7f4b881595e0 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b8edff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880add40 con 0x7f4b8815d240 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b8edff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880ad790 con 0x7f4b880ad390 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b8d39c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b880ad390 0x7f4b880aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b87fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b8815d240 0x7f4b880ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b8d39c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b880ad390 0x7f4b880aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49146/0 (socket says 172.21.15.190:49146) 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b8d39c700 1 -- 172.21.15.190:0/192388889 learned_addr learned my addr 172.21.15.190:0/192388889 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.669+0000 7f4b8cb9b700 1 --1- 172.21.15.190:0/192388889 >> v1:172.21.15.131:6789/0 conn(0x7f4b881595e0 0x7f4b881599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.190:41996/0 (socket says 172.21.15.190:41996) 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b877fe700 1 -- 172.21.15.190:0/192388889 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5017324 0 0) 0x7f4b880a98b0 con 0x7f4b881595e0 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b877fe700 1 -- 172.21.15.190:0/192388889 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b78003570 con 0x7f4b881595e0 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b8d39c700 1 -- 172.21.15.190:0/192388889 >> v2:172.21.15.131:3300/0 conn(0x7f4b8815d240 msgr2=0x7f4b880ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b8d39c700 1 --2- 172.21.15.190:0/192388889 >> v2:172.21.15.131:3300/0 conn(0x7f4b8815d240 0x7f4b880ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b8d39c700 1 -- 172.21.15.190:0/192388889 >> v1:172.21.15.131:6789/0 conn(0x7f4b881595e0 legacy=0x7f4b881599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b8d39c700 1 -- 172.21.15.190:0/192388889 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8815a500 con 0x7f4b880ad390 2023-12-26T21:36:25.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b8d39c700 1 --2- 172.21.15.190:0/192388889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b880ad390 0x7f4b880aa020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b8806b450 tx=0x7f4b80009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b929d371ed64d7c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:25.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b877fe700 1 -- 172.21.15.190:0/192388889 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000e070 con 0x7f4b880ad390 2023-12-26T21:36:25.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b877fe700 1 -- 172.21.15.190:0/192388889 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8000b970 con 0x7f4b880ad390 2023-12-26T21:36:25.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.670+0000 7f4b877fe700 1 -- 172.21.15.190:0/192388889 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b800092e0 con 0x7f4b880ad390 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.671+0000 7f4b8edff700 1 -- 172.21.15.190:0/192388889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b880ad390 msgr2=0x7f4b880aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.671+0000 7f4b8edff700 1 --2- 172.21.15.190:0/192388889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b880ad390 0x7f4b880aa020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b8806b450 tx=0x7f4b80009840 comp rx=0 tx=0).stop 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.671+0000 7f4b8edff700 1 -- 172.21.15.190:0/192388889 shutdown_connections 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.671+0000 7f4b8edff700 1 --2- 172.21.15.190:0/192388889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b880ad390 0x7f4b880aa020 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.671+0000 7f4b8edff700 1 --2- 172.21.15.190:0/192388889 >> v2:172.21.15.131:3300/0 conn(0x7f4b8815d240 0x7f4b880ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.671+0000 7f4b8edff700 1 -- 172.21.15.190:0/192388889 >> 172.21.15.190:0/192388889 conn(0x7f4b88068040 msgr2=0x7f4b88069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.671+0000 7f4b8edff700 1 -- 172.21.15.190:0/192388889 shutdown_connections 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.671+0000 7f4b8edff700 1 -- 172.21.15.190:0/192388889 wait complete. 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.673+0000 7f4b8edff700 1 Processor -- start 2023-12-26T21:36:25.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.673+0000 7f4b8edff700 1 -- start start 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8edff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b880ad390 0x7f4b880cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8edff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b881595e0 0x7f4b880b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8edff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8815d240 0x7f4b8816d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8edff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8806f440 con 0x7f4b881595e0 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8edff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8806e830 con 0x7f4b8815d240 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8edff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8806eb10 con 0x7f4b880ad390 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8cb9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b880ad390 0x7f4b880cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8d39c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8815d240 0x7f4b8816d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8cb9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b880ad390 0x7f4b880cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:34104/0 (socket says 172.21.15.190:34104) 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8cb9b700 1 -- 172.21.15.190:0/1367346436 learned_addr learned my addr 172.21.15.190:0/1367346436 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b87fff700 1 --2- 172.21.15.190:0/1367346436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b881595e0 0x7f4b880b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8cb9b700 1 -- 172.21.15.190:0/1367346436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8815d240 msgr2=0x7f4b8816d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8cb9b700 1 --2- 172.21.15.190:0/1367346436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8815d240 0x7f4b8816d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8cb9b700 1 -- 172.21.15.190:0/1367346436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b881595e0 msgr2=0x7f4b880b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8cb9b700 1 --2- 172.21.15.190:0/1367346436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b881595e0 0x7f4b880b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8cb9b700 1 -- 172.21.15.190:0/1367346436 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80009ae0 con 0x7f4b880ad390 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.674+0000 7f4b8d39c700 1 --2- 172.21.15.190:0/1367346436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8815d240 0x7f4b8816d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.675+0000 7f4b8cb9b700 1 --2- 172.21.15.190:0/1367346436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b880ad390 0x7f4b880cb880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c7c0 tx=0x7f4b7c007e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.675+0000 7f4b84ff9700 1 -- 172.21.15.190:0/1367346436 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c005a70 con 0x7f4b880ad390 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.675+0000 7f4b84ff9700 1 -- 172.21.15.190:0/1367346436 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c005bd0 con 0x7f4b880ad390 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.675+0000 7f4b8edff700 1 -- 172.21.15.190:0/1367346436 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8806edf0 con 0x7f4b880ad390 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.675+0000 7f4b84ff9700 1 -- 172.21.15.190:0/1367346436 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0070d0 con 0x7f4b880ad390 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.675+0000 7f4b8edff700 1 -- 172.21.15.190:0/1367346436 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b880cbdc0 con 0x7f4b880ad390 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.675+0000 7f4b84ff9700 1 -- 172.21.15.190:0/1367346436 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4b7c01c440 con 0x7f4b880ad390 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.675+0000 7f4b84ff9700 1 -- 172.21.15.190:0/1367346436 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f4b7c0127c0 con 0x7f4b880ad390 2023-12-26T21:36:25.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.676+0000 7f4b8edff700 1 -- 172.21.15.190:0/1367346436 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b880cc040 con 0x7f4b880ad390 2023-12-26T21:36:25.680 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.680+0000 7f4b84ff9700 1 -- 172.21.15.190:0/1367346436 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4b7c017070 con 0x7f4b880ad390 2023-12-26T21:36:25.698 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:25.697+0000 7f442e0ca1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:36:25.723 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:25.723+0000 7f21292781c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:36:25.768 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.768+0000 7f4b8edff700 1 -- 172.21.15.190:0/1367346436 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc", "id": 1} v 0) v1 -- 0x7f4b880657e0 con 0x7f4b880ad390 2023-12-26T21:36:25.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.776+0000 7f4b84ff9700 1 -- 172.21.15.190:0/1367346436 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4b7c01c8d0 con 0x7f4b880ad390 2023-12-26T21:36:25.776 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-12-26T21:36:25.777 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.777+0000 7f4b8edff700 1 -- 172.21.15.190:0/1367346436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b880ad390 msgr2=0x7f4b880cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:25.777 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.777+0000 7f4b8edff700 1 --2- 172.21.15.190:0/1367346436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b880ad390 0x7f4b880cb880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c7c0 tx=0x7f4b7c007e00 comp rx=0 tx=0).stop 2023-12-26T21:36:25.777 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.777+0000 7f4b8edff700 1 -- 172.21.15.190:0/1367346436 shutdown_connections 2023-12-26T21:36:25.777 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.777+0000 7f4b8edff700 1 --2- 172.21.15.190:0/1367346436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8815d240 0x7f4b8816d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.778 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.777+0000 7f4b8edff700 1 --2- 172.21.15.190:0/1367346436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b880ad390 0x7f4b880cb880 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.778 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.777+0000 7f4b8edff700 1 --2- 172.21.15.190:0/1367346436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b881595e0 0x7f4b880b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:25.778 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.777+0000 7f4b8edff700 1 -- 172.21.15.190:0/1367346436 >> 172.21.15.190:0/1367346436 conn(0x7f4b88068040 msgr2=0x7f4b880ab020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:25.778 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.777+0000 7f4b8edff700 1 -- 172.21.15.190:0/1367346436 shutdown_connections 2023-12-26T21:36:25.778 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:25.777+0000 7f4b8edff700 1 -- 172.21.15.190:0/1367346436 wait complete. 2023-12-26T21:36:25.795 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 49cabd4f-9407-4b82-9bea-e959f127e69f 2 2023-12-26T21:36:25.902 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:25.902+0000 7f442e0ca1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:36:25.938 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:25.938+0000 7f21292781c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:36:26.018 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.018+0000 7fe90bfff700 1 Processor -- start 2023-12-26T21:36:26.018 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.018+0000 7fe90bfff700 1 -- start start 2023-12-26T21:36:26.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.019+0000 7fe90bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe90415b260 0x7fe9040caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.019+0000 7fe90bfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe904067c30 con 0x7fe9040b2360 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.019+0000 7fe90bfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe904067650 con 0x7fe9040aecc0 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.019+0000 7fe90bfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90415bcd0 con 0x7fe90415b260 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.020+0000 7fe909d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe90415b260 0x7fe9040caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.020+0000 7fe909d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe90415b260 0x7fe9040caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49154/0 (socket says 172.21.15.190:49154) 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.020+0000 7fe90959a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe9040aecc0 0x7fe9040b1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:42216/0 (socket says 172.21.15.190:42216) 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.020+0000 7fe909d9b700 1 -- 172.21.15.190:0/3936418569 learned_addr learned my addr 172.21.15.190:0/3936418569 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.020+0000 7fe909d9b700 1 -- 172.21.15.190:0/3936418569 >> v1:172.21.15.190:6789/0 conn(0x7fe9040aecc0 legacy=0x7fe9040b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.020+0000 7fe909d9b700 1 -- 172.21.15.190:0/3936418569 >> v1:172.21.15.131:6789/0 conn(0x7fe9040b2360 legacy=0x7fe9040680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.020+0000 7fe909d9b700 1 -- 172.21.15.190:0/3936418569 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040637b0 con 0x7fe90415b260 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.020+0000 7fe909d9b700 1 --2- 172.21.15.190:0/3936418569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe90415b260 0x7fe9040caae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9000099b0 tx=0x7fe900009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51069d4c4e68f753 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe908d99700 1 -- 172.21.15.190:0/3936418569 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90000b8f0 con 0x7fe90415b260 2023-12-26T21:36:26.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe908d99700 1 -- 172.21.15.190:0/3936418569 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90000ba50 con 0x7fe90415b260 2023-12-26T21:36:26.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe908d99700 1 -- 172.21.15.190:0/3936418569 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90000bdf0 con 0x7fe90415b260 2023-12-26T21:36:26.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe90bfff700 1 -- 172.21.15.190:0/3936418569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe90415b260 msgr2=0x7fe9040caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe90bfff700 1 --2- 172.21.15.190:0/3936418569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe90415b260 0x7fe9040caae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9000099b0 tx=0x7fe900009250 comp rx=0 tx=0).stop 2023-12-26T21:36:26.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe90bfff700 1 -- 172.21.15.190:0/3936418569 shutdown_connections 2023-12-26T21:36:26.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe90bfff700 1 --2- 172.21.15.190:0/3936418569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe90415b260 0x7fe9040caae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe90bfff700 1 -- 172.21.15.190:0/3936418569 >> 172.21.15.190:0/3936418569 conn(0x7fe90406c290 msgr2=0x7fe9040ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:26.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe90bfff700 1 -- 172.21.15.190:0/3936418569 shutdown_connections 2023-12-26T21:36:26.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.021+0000 7fe90bfff700 1 -- 172.21.15.190:0/3936418569 wait complete. 2023-12-26T21:36:26.026 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.023+0000 7fe90bfff700 1 Processor -- start 2023-12-26T21:36:26.026 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.024+0000 7fe90bfff700 1 -- start start 2023-12-26T21:36:26.026 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9040aecc0 0x7fe9040b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.026 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9040b2360 0x7fe90416d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe90415b260 0x7fe9040adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90bfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90416e5d0 con 0x7fe90415b260 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90bfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe904154f00 con 0x7fe9040b2360 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90416a020 con 0x7fe9040aecc0 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90959a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9040b2360 0x7fe90416d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90959a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9040b2360 0x7fe90416d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49170/0 (socket says 172.21.15.190:49170) 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90959a700 1 -- 172.21.15.190:0/1792116735 learned_addr learned my addr 172.21.15.190:0/1792116735 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe909d9b700 1 --2- 172.21.15.190:0/1792116735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9040aecc0 0x7fe9040b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.025+0000 7fe90a59c700 1 --2- 172.21.15.190:0/1792116735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe90415b260 0x7fe9040adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe90959a700 1 -- 172.21.15.190:0/1792116735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9040aecc0 msgr2=0x7fe9040b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe90959a700 1 --2- 172.21.15.190:0/1792116735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9040aecc0 0x7fe9040b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe90959a700 1 -- 172.21.15.190:0/1792116735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe90415b260 msgr2=0x7fe9040adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe90959a700 1 --2- 172.21.15.190:0/1792116735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe90415b260 0x7fe9040adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.027 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe90959a700 1 -- 172.21.15.190:0/1792116735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000094f0 con 0x7fe9040b2360 2023-12-26T21:36:26.028 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe90959a700 1 --2- 172.21.15.190:0/1792116735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9040b2360 0x7fe90416d620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400c7c0 tx=0x7fe8f4007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:26.028 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe8f9ffb700 1 -- 172.21.15.190:0/1792116735 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4005a70 con 0x7fe9040b2360 2023-12-26T21:36:26.028 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe8f9ffb700 1 -- 172.21.15.190:0/1792116735 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f4005bd0 con 0x7fe9040b2360 2023-12-26T21:36:26.028 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe8f9ffb700 1 -- 172.21.15.190:0/1792116735 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40070d0 con 0x7fe9040b2360 2023-12-26T21:36:26.028 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe90bfff700 1 -- 172.21.15.190:0/1792116735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90416dd30 con 0x7fe9040b2360 2023-12-26T21:36:26.028 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe90bfff700 1 -- 172.21.15.190:0/1792116735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90416e2f0 con 0x7fe9040b2360 2023-12-26T21:36:26.028 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.026+0000 7fe8f9ffb700 1 -- 172.21.15.190:0/1792116735 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe8f401c440 con 0x7fe9040b2360 2023-12-26T21:36:26.029 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.027+0000 7fe8f9ffb700 1 -- 172.21.15.190:0/1792116735 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fe8f4007700 con 0x7fe9040b2360 2023-12-26T21:36:26.029 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.027+0000 7fe90bfff700 1 -- 172.21.15.190:0/1792116735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9040657e0 con 0x7fe9040b2360 2023-12-26T21:36:26.031 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.030+0000 7fe8f9ffb700 1 -- 172.21.15.190:0/1792116735 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe8f4017070 con 0x7fe9040b2360 2023-12-26T21:36:26.051 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:26.051+0000 7f442e0ca1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:36:26.095 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:26.095+0000 7f21292781c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:36:26.115 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.115+0000 7fe90bfff700 1 -- 172.21.15.190:0/1792116735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "49cabd4f-9407-4b82-9bea-e959f127e69f", "id": 2} v 0) v1 -- 0x7fe9040678c0 con 0x7fe9040b2360 2023-12-26T21:36:26.119 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.119+0000 7fe8f9ffb700 1 -- 172.21.15.190:0/1792116735 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "49cabd4f-9407-4b82-9bea-e959f127e69f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe8f401c6f0 con 0x7fe9040b2360 2023-12-26T21:36:26.120 INFO:teuthology.orchestra.run.smithi190.stdout:2 2023-12-26T21:36:26.121 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.120+0000 7fe90bfff700 1 -- 172.21.15.190:0/1792116735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9040b2360 msgr2=0x7fe90416d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.121 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.120+0000 7fe90bfff700 1 --2- 172.21.15.190:0/1792116735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9040b2360 0x7fe90416d620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400c7c0 tx=0x7fe8f4007f70 comp rx=0 tx=0).stop 2023-12-26T21:36:26.121 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.121+0000 7fe90bfff700 1 -- 172.21.15.190:0/1792116735 shutdown_connections 2023-12-26T21:36:26.121 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.121+0000 7fe90bfff700 1 --2- 172.21.15.190:0/1792116735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe90415b260 0x7fe9040adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.121 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.121+0000 7fe90bfff700 1 --2- 172.21.15.190:0/1792116735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9040aecc0 0x7fe9040b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.121 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.121+0000 7fe90bfff700 1 --2- 172.21.15.190:0/1792116735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9040b2360 0x7fe90416d620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.121 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.121+0000 7fe90bfff700 1 -- 172.21.15.190:0/1792116735 >> 172.21.15.190:0/1792116735 conn(0x7fe90406c290 msgr2=0x7fe90406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:26.121 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.121+0000 7fe90bfff700 1 -- 172.21.15.190:0/1792116735 shutdown_connections 2023-12-26T21:36:26.121 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.121+0000 7fe90bfff700 1 -- 172.21.15.190:0/1792116735 wait complete. 2023-12-26T21:36:26.137 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 43c1b543-b3c5-40ad-b1df-34fcd9cd23e1 3 2023-12-26T21:36:26.323 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.322+0000 7ff55cbff700 1 Processor -- start 2023-12-26T21:36:26.323 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.323+0000 7ff55cbff700 1 -- start start 2023-12-26T21:36:26.324 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff55cbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.324 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff55cbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff5580ced90 0x7ff5580cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.324 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff55cbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5580c6810 con 0x7ff558159920 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff55cbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5580cf750 con 0x7ff5580ced90 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff55cbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580c5c00 con 0x7ff558159cf0 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff556d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff5580ced90 0x7ff5580cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff555d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff556d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff5580ced90 0x7ff5580cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49180/0 (socket says 172.21.15.190:49180) 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff555d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:34134/0 (socket says 172.21.15.190:34134) 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff556d9d700 1 -- 172.21.15.190:0/2243123959 learned_addr learned my addr 172.21.15.190:0/2243123959 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff55659c700 1 --1- 172.21.15.190:0/2243123959 >> v1:172.21.15.190:6789/0 conn(0x7ff558159920 0x7ff5580c5420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:42224/0 (socket says 172.21.15.190:42224) 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff555d9b700 1 -- 172.21.15.190:0/2243123959 >> v2:172.21.15.190:3300/0 conn(0x7ff5580ced90 msgr2=0x7ff5580cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff555d9b700 1 --2- 172.21.15.190:0/2243123959 >> v2:172.21.15.190:3300/0 conn(0x7ff5580ced90 0x7ff5580cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff555d9b700 1 -- 172.21.15.190:0/2243123959 >> v1:172.21.15.190:6789/0 conn(0x7ff558159920 legacy=0x7ff5580c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.324+0000 7ff55559a700 1 -- 172.21.15.190:0/2243123959 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410989369 0 0) 0x7ff5580c6810 con 0x7ff558159920 2023-12-26T21:36:26.325 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.325+0000 7ff555d9b700 1 -- 172.21.15.190:0/2243123959 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580c5ee0 con 0x7ff558159cf0 2023-12-26T21:36:26.326 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.325+0000 7ff556d9d700 1 --2- 172.21.15.190:0/2243123959 >> v2:172.21.15.190:3300/0 conn(0x7ff5580ced90 0x7ff5580cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.325+0000 7ff555d9b700 1 --2- 172.21.15.190:0/2243123959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580c6ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0098e0 tx=0x7ff54c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245192a9bda1ca0c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.325+0000 7ff55559a700 1 -- 172.21.15.190:0/2243123959 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00b640 con 0x7ff558159cf0 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.325+0000 7ff55559a700 1 -- 172.21.15.190:0/2243123959 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c00b7a0 con 0x7ff558159cf0 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.325+0000 7ff55559a700 1 -- 172.21.15.190:0/2243123959 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00bb40 con 0x7ff558159cf0 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.326+0000 7ff55cbff700 1 -- 172.21.15.190:0/2243123959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 msgr2=0x7ff5580c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.326+0000 7ff55cbff700 1 --2- 172.21.15.190:0/2243123959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580c6ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0098e0 tx=0x7ff54c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.326+0000 7ff55cbff700 1 -- 172.21.15.190:0/2243123959 shutdown_connections 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.326+0000 7ff55cbff700 1 --2- 172.21.15.190:0/2243123959 >> v2:172.21.15.190:3300/0 conn(0x7ff5580ced90 0x7ff5580cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.326+0000 7ff55cbff700 1 --2- 172.21.15.190:0/2243123959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580c6ca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.326+0000 7ff55cbff700 1 -- 172.21.15.190:0/2243123959 >> 172.21.15.190:0/2243123959 conn(0x7ff558099980 msgr2=0x7ff55806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.326+0000 7ff55cbff700 1 -- 172.21.15.190:0/2243123959 shutdown_connections 2023-12-26T21:36:26.327 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.326+0000 7ff55cbff700 1 -- 172.21.15.190:0/2243123959 wait complete. 2023-12-26T21:36:26.328 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.327+0000 7ff55cbff700 1 Processor -- start 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.327+0000 7ff55cbff700 1 -- start start 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.328+0000 7ff55cbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5580ced90 0x7ff558154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.328+0000 7ff55cbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff558159920 0x7ff558156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.328+0000 7ff55cbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.328+0000 7ff55cbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff558157cb0 con 0x7ff5580ced90 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.328+0000 7ff55cbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5581570a0 con 0x7ff558159920 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.328+0000 7ff55cbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff558157380 con 0x7ff558159cf0 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.328+0000 7ff555d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff558159920 0x7ff558156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff556d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff555d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff558159920 0x7ff558156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49184/0 (socket says 172.21.15.190:49184) 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff555d9b700 1 -- 172.21.15.190:0/397540022 learned_addr learned my addr 172.21.15.190:0/397540022 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:26.329 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff55659c700 1 --2- 172.21.15.190:0/397540022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5580ced90 0x7ff558154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.331 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff555d9b700 1 -- 172.21.15.190:0/397540022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 msgr2=0x7ff5580cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.331 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff555d9b700 1 --2- 172.21.15.190:0/397540022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.331 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff555d9b700 1 -- 172.21.15.190:0/397540022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5580ced90 msgr2=0x7ff558154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.331 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff555d9b700 1 --2- 172.21.15.190:0/397540022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5580ced90 0x7ff558154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff555d9b700 1 -- 172.21.15.190:0/397540022 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff550009ae0 con 0x7ff558159920 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff55659c700 1 --2- 172.21.15.190:0/397540022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5580ced90 0x7ff558154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff555d9b700 1 --2- 172.21.15.190:0/397540022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff558159920 0x7ff558156670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0093f0 tx=0x7ff54c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff5467fc700 1 -- 172.21.15.190:0/397540022 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c01f070 con 0x7ff558159920 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff5467fc700 1 -- 172.21.15.190:0/397540022 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c000bc0 con 0x7ff558159920 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.329+0000 7ff55cbff700 1 -- 172.21.15.190:0/397540022 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c009420 con 0x7ff558159920 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.330+0000 7ff5467fc700 1 -- 172.21.15.190:0/397540022 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c025420 con 0x7ff558159920 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.330+0000 7ff55cbff700 1 -- 172.21.15.190:0/397540022 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff558157940 con 0x7ff558159920 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.330+0000 7ff5467fc700 1 -- 172.21.15.190:0/397540022 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff54c023070 con 0x7ff558159920 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.330+0000 7ff5467fc700 1 -- 172.21.15.190:0/397540022 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7ff54c011420 con 0x7ff558159920 2023-12-26T21:36:26.332 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.331+0000 7ff55cbff700 1 -- 172.21.15.190:0/397540022 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff558157e20 con 0x7ff558159920 2023-12-26T21:36:26.333 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.333+0000 7ff5467fc700 1 -- 172.21.15.190:0/397540022 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff54c02a070 con 0x7ff558159920 2023-12-26T21:36:26.459 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.459+0000 7ff55cbff700 1 -- 172.21.15.190:0/397540022 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "43c1b543-b3c5-40ad-b1df-34fcd9cd23e1", "id": 3} v 0) v1 -- 0x7ff5580657e0 con 0x7ff558159920 2023-12-26T21:36:26.464 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.464+0000 7ff5467fc700 1 -- 172.21.15.190:0/397540022 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "43c1b543-b3c5-40ad-b1df-34fcd9cd23e1", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff54c015020 con 0x7ff558159920 2023-12-26T21:36:26.464 INFO:teuthology.orchestra.run.smithi190.stdout:3 2023-12-26T21:36:26.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.465+0000 7ff55cbff700 1 -- 172.21.15.190:0/397540022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff558159920 msgr2=0x7ff558156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.465+0000 7ff55cbff700 1 --2- 172.21.15.190:0/397540022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff558159920 0x7ff558156670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0093f0 tx=0x7ff54c017000 comp rx=0 tx=0).stop 2023-12-26T21:36:26.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.465+0000 7ff55cbff700 1 -- 172.21.15.190:0/397540022 shutdown_connections 2023-12-26T21:36:26.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.465+0000 7ff55cbff700 1 --2- 172.21.15.190:0/397540022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff558159cf0 0x7ff5580cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.465+0000 7ff55cbff700 1 --2- 172.21.15.190:0/397540022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5580ced90 0x7ff558154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.465+0000 7ff55cbff700 1 --2- 172.21.15.190:0/397540022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff558159920 0x7ff558156670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.465 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.465+0000 7ff55cbff700 1 -- 172.21.15.190:0/397540022 >> 172.21.15.190:0/397540022 conn(0x7ff558099980 msgr2=0x7ff5580c5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:26.466 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.465+0000 7ff55cbff700 1 -- 172.21.15.190:0/397540022 shutdown_connections 2023-12-26T21:36:26.466 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.465+0000 7ff55cbff700 1 -- 172.21.15.190:0/397540022 wait complete. 2023-12-26T21:36:26.483 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new bd6a60f8-0595-4fdb-91e3-cd1321c248a0 4 2023-12-26T21:36:26.492 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:26.491+0000 7f442e0ca1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:36:26.616 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:26.616+0000 7f21292781c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:36:26.668 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.668+0000 7fd28a3ff700 1 Processor -- start 2023-12-26T21:36:26.669 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.668+0000 7fd28a3ff700 1 -- start start 2023-12-26T21:36:26.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd28a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd284159a70 0x7fd2840b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd28a3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2840afcd0 con 0x7fd2841596a0 2023-12-26T21:36:26.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd28a3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2840b3e70 con 0x7fd2840b2a90 2023-12-26T21:36:26.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd28a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840b0f20 con 0x7fd284159a70 2023-12-26T21:36:26.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd2837fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd284159a70 0x7fd2840b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd28899c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd2840b2a90 0x7fd2840b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:42226/0 (socket says 172.21.15.190:42226) 2023-12-26T21:36:26.670 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd28899c700 1 -- 172.21.15.190:0/4144491783 learned_addr learned my addr 172.21.15.190:0/4144491783 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-12-26T21:36:26.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd282ffd700 1 -- 172.21.15.190:0/4144491783 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616147744 0 0) 0x7fd2840b3e70 con 0x7fd2840b2a90 2023-12-26T21:36:26.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd282ffd700 1 -- 172.21.15.190:0/4144491783 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd26c003570 con 0x7fd2840b2a90 2023-12-26T21:36:26.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd2837fe700 1 -- 172.21.15.190:0/4144491783 >> v1:172.21.15.190:6789/0 conn(0x7fd2840b2a90 legacy=0x7fd2840b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd282ffd700 1 -- 172.21.15.190:0/4144491783 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913424354 0 0) 0x7fd2840afcd0 con 0x7fd2841596a0 2023-12-26T21:36:26.671 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd2837fe700 1 -- 172.21.15.190:0/4144491783 >> v1:172.21.15.131:6789/0 conn(0x7fd2841596a0 legacy=0x7fd2840b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.670+0000 7fd2837fe700 1 -- 172.21.15.190:0/4144491783 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2840b3540 con 0x7fd284159a70 2023-12-26T21:36:26.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.671+0000 7fd2837fe700 1 --2- 172.21.15.190:0/4144491783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd284159a70 0x7fd2840b0160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd278009910 tx=0x7fd2780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f004f339e9e8c51a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:26.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.671+0000 7fd282ffd700 1 -- 172.21.15.190:0/4144491783 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800b680 con 0x7fd284159a70 2023-12-26T21:36:26.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.671+0000 7fd282ffd700 1 -- 172.21.15.190:0/4144491783 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27800b7e0 con 0x7fd284159a70 2023-12-26T21:36:26.672 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.671+0000 7fd282ffd700 1 -- 172.21.15.190:0/4144491783 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800bba0 con 0x7fd284159a70 2023-12-26T21:36:26.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.672+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4144491783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd284159a70 msgr2=0x7fd2840b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.672+0000 7fd28a3ff700 1 --2- 172.21.15.190:0/4144491783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd284159a70 0x7fd2840b0160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd278009910 tx=0x7fd2780091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.672+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4144491783 shutdown_connections 2023-12-26T21:36:26.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.672+0000 7fd28a3ff700 1 --2- 172.21.15.190:0/4144491783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd284159a70 0x7fd2840b0160 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.672+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4144491783 >> 172.21.15.190:0/4144491783 conn(0x7fd28406ece0 msgr2=0x7fd28406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:26.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.672+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4144491783 shutdown_connections 2023-12-26T21:36:26.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.672+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4144491783 wait complete. 2023-12-26T21:36:26.673 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.673+0000 7fd28a3ff700 1 Processor -- start 2023-12-26T21:36:26.674 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.673+0000 7fd28a3ff700 1 -- start start 2023-12-26T21:36:26.675 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.674+0000 7fd28a3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2840b2a90 0x7fd2840c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.675 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.674+0000 7fd28a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2841596a0 0x7fd28414a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.675 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.674+0000 7fd28a3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd284159a70 0x7fd28416d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.675 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.674+0000 7fd28a3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28416def0 con 0x7fd284159a70 2023-12-26T21:36:26.675 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.674+0000 7fd28a3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840c9540 con 0x7fd2840b2a90 2023-12-26T21:36:26.675 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.674+0000 7fd28a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840c5400 con 0x7fd2841596a0 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.674+0000 7fd283fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2840b2a90 0x7fd2840c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd283fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2840b2a90 0x7fd2840c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49192/0 (socket says 172.21.15.190:49192) 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd283fff700 1 -- 172.21.15.190:0/4163321362 learned_addr learned my addr 172.21.15.190:0/4163321362 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd2837fe700 1 --2- 172.21.15.190:0/4163321362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2841596a0 0x7fd28414a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd28899c700 1 --2- 172.21.15.190:0/4163321362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd284159a70 0x7fd28416d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd2837fe700 1 -- 172.21.15.190:0/4163321362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2840b2a90 msgr2=0x7fd2840c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd2837fe700 1 --2- 172.21.15.190:0/4163321362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2840b2a90 0x7fd2840c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd2837fe700 1 -- 172.21.15.190:0/4163321362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd284159a70 msgr2=0x7fd28416d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd2837fe700 1 --2- 172.21.15.190:0/4163321362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd284159a70 0x7fd28416d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd2837fe700 1 -- 172.21.15.190:0/4163321362 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd278009420 con 0x7fd2841596a0 2023-12-26T21:36:26.676 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.675+0000 7fd2837fe700 1 --2- 172.21.15.190:0/4163321362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2841596a0 0x7fd28414a9a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd2840892a0 tx=0x7fd278017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:26.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.676+0000 7fd26bfff700 1 -- 172.21.15.190:0/4163321362 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800bb60 con 0x7fd2841596a0 2023-12-26T21:36:26.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.676+0000 7fd26bfff700 1 -- 172.21.15.190:0/4163321362 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd278027450 con 0x7fd2841596a0 2023-12-26T21:36:26.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.676+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4163321362 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2840c47f0 con 0x7fd2841596a0 2023-12-26T21:36:26.677 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.676+0000 7fd26bfff700 1 -- 172.21.15.190:0/4163321362 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780277f0 con 0x7fd2841596a0 2023-12-26T21:36:26.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.676+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4163321362 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2840c4db0 con 0x7fd2841596a0 2023-12-26T21:36:26.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.676+0000 7fd26bfff700 1 -- 172.21.15.190:0/4163321362 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd278027b60 con 0x7fd2841596a0 2023-12-26T21:36:26.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.676+0000 7fd26bfff700 1 -- 172.21.15.190:0/4163321362 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fd278011b60 con 0x7fd2841596a0 2023-12-26T21:36:26.678 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.677+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4163321362 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2840657e0 con 0x7fd2841596a0 2023-12-26T21:36:26.679 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.679+0000 7fd26bfff700 1 -- 172.21.15.190:0/4163321362 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd27802c020 con 0x7fd2841596a0 2023-12-26T21:36:26.762 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:26.761+0000 7f442e0ca1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:36:26.766 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.766+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4163321362 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bd6a60f8-0595-4fdb-91e3-cd1321c248a0", "id": 4} v 0) v1 -- 0x7fd2840c5540 con 0x7fd2841596a0 2023-12-26T21:36:26.774 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.774+0000 7fd26bfff700 1 -- 172.21.15.190:0/4163321362 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bd6a60f8-0595-4fdb-91e3-cd1321c248a0", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd278015020 con 0x7fd2841596a0 2023-12-26T21:36:26.775 INFO:teuthology.orchestra.run.smithi190.stdout:4 2023-12-26T21:36:26.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.775+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4163321362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2841596a0 msgr2=0x7fd28414a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.775+0000 7fd28a3ff700 1 --2- 172.21.15.190:0/4163321362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2841596a0 0x7fd28414a9a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd2840892a0 tx=0x7fd278017000 comp rx=0 tx=0).stop 2023-12-26T21:36:26.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.775+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4163321362 shutdown_connections 2023-12-26T21:36:26.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.775+0000 7fd28a3ff700 1 --2- 172.21.15.190:0/4163321362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd284159a70 0x7fd28416d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.775+0000 7fd28a3ff700 1 --2- 172.21.15.190:0/4163321362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2840b2a90 0x7fd2840c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.775+0000 7fd28a3ff700 1 --2- 172.21.15.190:0/4163321362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2841596a0 0x7fd28414a9a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.776+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4163321362 >> 172.21.15.190:0/4163321362 conn(0x7fd28406ece0 msgr2=0x7fd284156320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:26.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.776+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4163321362 shutdown_connections 2023-12-26T21:36:26.776 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.776+0000 7fd28a3ff700 1 -- 172.21.15.190:0/4163321362 wait complete. 2023-12-26T21:36:26.792 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new ceb3f886-e96a-4cca-8136-58614b96b5d4 5 2023-12-26T21:36:26.899 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:26.899+0000 7f21292781c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:36:26.978 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.977+0000 7f7bbf5ff700 1 Processor -- start 2023-12-26T21:36:26.978 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.977+0000 7f7bbf5ff700 1 -- start start 2023-12-26T21:36:26.983 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.978+0000 7f7bbf5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7bb813c510 0x7f7bb80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.983 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.978+0000 7f7bbf5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7bb806b8c0 0x7f7bb806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.983 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbf5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb80686c0 con 0x7f7bb80cc100 2023-12-26T21:36:26.983 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbf5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb806c280 con 0x7f7bb813c510 2023-12-26T21:36:26.983 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbf5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb8069520 con 0x7f7bb806b8c0 2023-12-26T21:36:26.984 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbcb9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7bb806b8c0 0x7f7bb806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.984 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbdb9c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f7bb80cc100 0x7f7bb80cc520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:42228/0 (socket says 172.21.15.190:42228) 2023-12-26T21:36:26.984 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbdb9c700 1 -- 172.21.15.190:0/563466423 learned_addr learned my addr 172.21.15.190:0/563466423 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-12-26T21:36:26.984 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbd39b700 1 --2- 172.21.15.190:0/563466423 >> v2:172.21.15.131:3300/0 conn(0x7f7bb813c510 0x7f7bb80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.984 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbcb9a700 1 -- 172.21.15.190:0/563466423 >> v2:172.21.15.131:3300/0 conn(0x7f7bb813c510 msgr2=0x7f7bb80c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbcb9a700 1 --2- 172.21.15.190:0/563466423 >> v2:172.21.15.131:3300/0 conn(0x7f7bb813c510 0x7f7bb80c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbcb9a700 1 -- 172.21.15.190:0/563466423 >> v1:172.21.15.190:6789/0 conn(0x7f7bb80cc100 legacy=0x7f7bb80cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7bbcb9a700 1 -- 172.21.15.190:0/563466423 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8068910 con 0x7f7bb806b8c0 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.979+0000 7f7baffff700 1 -- 172.21.15.190:0/563466423 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751592281 0 0) 0x7f7bb80686c0 con 0x7f7bb80cc100 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.980+0000 7f7bbcb9a700 1 --2- 172.21.15.190:0/563466423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb806b8c0 0x7f7bb806bd40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80b43e0 tx=0x7f7ba80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ab6908d84f6657 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.980+0000 7f7baffff700 1 -- 172.21.15.190:0/563466423 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011630 con 0x7f7bb806b8c0 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.980+0000 7f7baffff700 1 -- 172.21.15.190:0/563466423 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8011790 con 0x7f7bb806b8c0 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.980+0000 7f7baffff700 1 -- 172.21.15.190:0/563466423 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011b30 con 0x7f7bb806b8c0 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.980+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/563466423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb806b8c0 msgr2=0x7f7bb806bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.985 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.980+0000 7f7bbf5ff700 1 --2- 172.21.15.190:0/563466423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb806b8c0 0x7f7bb806bd40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80b43e0 tx=0x7f7ba80091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.986 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.981+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/563466423 shutdown_connections 2023-12-26T21:36:26.986 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.981+0000 7f7bbf5ff700 1 --2- 172.21.15.190:0/563466423 >> v2:172.21.15.131:3300/0 conn(0x7f7bb813c510 0x7f7bb80c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.986 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.981+0000 7f7bbf5ff700 1 --2- 172.21.15.190:0/563466423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb806b8c0 0x7f7bb806bd40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.986 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.981+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/563466423 >> 172.21.15.190:0/563466423 conn(0x7f7bb80ce3e0 msgr2=0x7f7bb80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:26.986 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.981+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/563466423 shutdown_connections 2023-12-26T21:36:26.986 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.981+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/563466423 wait complete. 2023-12-26T21:36:26.986 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.982+0000 7f7bbf5ff700 1 Processor -- start 2023-12-26T21:36:26.986 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.982+0000 7f7bbf5ff700 1 -- start start 2023-12-26T21:36:26.989 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbf5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb806b8c0 0x7f7bb80c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbf5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb80cc100 0x7f7bb813b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbf5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb813c510 0x7f7bb81569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbf5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8127fd0 con 0x7f7bb806b8c0 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbf5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb81273c0 con 0x7f7bb80cc100 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbf5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb81276a0 con 0x7f7bb813c510 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbcb9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb80cc100 0x7f7bb813b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbd39b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb806b8c0 0x7f7bb80c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbcb9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb80cc100 0x7f7bb813b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49214/0 (socket says 172.21.15.190:49214) 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbcb9a700 1 -- 172.21.15.190:0/2832469244 learned_addr learned my addr 172.21.15.190:0/2832469244 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.983+0000 7f7bbdb9c700 1 --2- 172.21.15.190:0/2832469244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb813c510 0x7f7bb81569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.984+0000 7f7bbcb9a700 1 -- 172.21.15.190:0/2832469244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb813c510 msgr2=0x7f7bb81569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.984+0000 7f7bbcb9a700 1 --2- 172.21.15.190:0/2832469244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb813c510 0x7f7bb81569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.984+0000 7f7bbcb9a700 1 -- 172.21.15.190:0/2832469244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb806b8c0 msgr2=0x7f7bb80c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:26.990 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.984+0000 7f7bbcb9a700 1 --2- 172.21.15.190:0/2832469244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb806b8c0 0x7f7bb80c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.984+0000 7f7bbcb9a700 1 -- 172.21.15.190:0/2832469244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8009420 con 0x7f7bb80cc100 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.984+0000 7f7bbcb9a700 1 --2- 172.21.15.190:0/2832469244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb80cc100 0x7f7bb813b460 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7bb81578a0 tx=0x7f7ba8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.984+0000 7f7bad7fa700 1 -- 172.21.15.190:0/2832469244 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8000d10 con 0x7f7bb80cc100 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.984+0000 7f7bad7fa700 1 -- 172.21.15.190:0/2832469244 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8011b10 con 0x7f7bb80cc100 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.984+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/2832469244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb8127980 con 0x7f7bb80cc100 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.985+0000 7f7bad7fa700 1 -- 172.21.15.190:0/2832469244 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8025640 con 0x7f7bb80cc100 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.985+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/2832469244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb80c8170 con 0x7f7bb80cc100 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.985+0000 7f7bad7fa700 1 -- 172.21.15.190:0/2832469244 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7ba8025cb0 con 0x7f7bb80cc100 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.985+0000 7f7bad7fa700 1 -- 172.21.15.190:0/2832469244 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f7ba802ee20 con 0x7f7bb80cc100 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.986+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/2832469244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb80657e0 con 0x7f7bb80cc100 2023-12-26T21:36:26.991 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:26.988+0000 7f7bad7fa700 1 -- 172.21.15.190:0/2832469244 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f7ba8033060 con 0x7f7bb80cc100 2023-12-26T21:36:27.054 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:27.054+0000 7f442e0ca1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:36:27.074 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.074+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/2832469244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ceb3f886-e96a-4cca-8136-58614b96b5d4", "id": 5} v 0) v1 -- 0x7f7bb8128110 con 0x7f7bb80cc100 2023-12-26T21:36:27.083 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.083+0000 7f7bad7fa700 1 -- 172.21.15.190:0/2832469244 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ceb3f886-e96a-4cca-8136-58614b96b5d4", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7ba8015020 con 0x7f7bb80cc100 2023-12-26T21:36:27.083 INFO:teuthology.orchestra.run.smithi190.stdout:5 2023-12-26T21:36:27.084 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.084+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/2832469244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb80cc100 msgr2=0x7f7bb813b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.084 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.084+0000 7f7bbf5ff700 1 --2- 172.21.15.190:0/2832469244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb80cc100 0x7f7bb813b460 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7bb81578a0 tx=0x7f7ba8017000 comp rx=0 tx=0).stop 2023-12-26T21:36:27.084 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.084+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/2832469244 shutdown_connections 2023-12-26T21:36:27.084 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.084+0000 7f7bbf5ff700 1 --2- 172.21.15.190:0/2832469244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb813c510 0x7f7bb81569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.084 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.084+0000 7f7bbf5ff700 1 --2- 172.21.15.190:0/2832469244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb806b8c0 0x7f7bb80c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.084 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.084+0000 7f7bbf5ff700 1 --2- 172.21.15.190:0/2832469244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb80cc100 0x7f7bb813b460 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.085 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.084+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/2832469244 >> 172.21.15.190:0/2832469244 conn(0x7f7bb80ce3e0 msgr2=0x7f7bb81558c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:27.085 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.084+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/2832469244 shutdown_connections 2023-12-26T21:36:27.085 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.084+0000 7f7bbf5ff700 1 -- 172.21.15.190:0/2832469244 wait complete. 2023-12-26T21:36:27.101 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new a3915ff5-2514-4fcf-92e0-8f9144eab84c 6 2023-12-26T21:36:27.152 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:27.151+0000 7f442e0ca1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:36:27.193 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:27.193+0000 7f21292781c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:36:27.288 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.288+0000 7f8847bff700 1 Processor -- start 2023-12-26T21:36:27.288 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.288+0000 7f8847bff700 1 -- start start 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.289+0000 7f8847bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8840159920 0x7f8840159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.289+0000 7f8847bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88400c5420 0x7f88400c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.289+0000 7f8847bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f88400ced90 0x7f88400cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.289+0000 7f8847bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884015a230 con 0x7f88400ced90 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.289+0000 7f8847bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88400cf730 con 0x7f8840159920 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.289+0000 7f8847bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88400c6810 con 0x7f88400c5420 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.289+0000 7f884519a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88400c5420 0x7f88400c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.289+0000 7f884599b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8840159920 0x7f8840159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884519a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88400c5420 0x7f88400c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:34174/0 (socket says 172.21.15.190:34174) 2023-12-26T21:36:27.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884599b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8840159920 0x7f8840159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49222/0 (socket says 172.21.15.190:49222) 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884519a700 1 -- 172.21.15.190:0/2770514969 learned_addr learned my addr 172.21.15.190:0/2770514969 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884619c700 1 --2- 172.21.15.190:0/2770514969 >> v2:172.21.15.131:3300/0 conn(0x7f88400ced90 0x7f88400cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884519a700 1 -- 172.21.15.190:0/2770514969 >> v2:172.21.15.190:3300/0 conn(0x7f8840159920 msgr2=0x7f8840159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884519a700 1 --2- 172.21.15.190:0/2770514969 >> v2:172.21.15.190:3300/0 conn(0x7f8840159920 0x7f8840159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884519a700 1 -- 172.21.15.190:0/2770514969 >> v2:172.21.15.131:3300/0 conn(0x7f88400ced90 msgr2=0x7f88400cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884519a700 1 --2- 172.21.15.190:0/2770514969 >> v2:172.21.15.131:3300/0 conn(0x7f88400ced90 0x7f88400cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884519a700 1 -- 172.21.15.190:0/2770514969 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400c5c00 con 0x7f88400c5420 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f884519a700 1 --2- 172.21.15.190:0/2770514969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88400c5420 0x7f88400c6ca0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8840088ec0 tx=0x7f8830009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff05f69dc508db5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.290+0000 7f8844999700 1 -- 172.21.15.190:0/2770514969 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000b640 con 0x7f88400c5420 2023-12-26T21:36:27.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8844999700 1 -- 172.21.15.190:0/2770514969 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883000b7a0 con 0x7f88400c5420 2023-12-26T21:36:27.293 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8844999700 1 -- 172.21.15.190:0/2770514969 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000bb60 con 0x7f88400c5420 2023-12-26T21:36:27.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8847bff700 1 -- 172.21.15.190:0/2770514969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88400c5420 msgr2=0x7f88400c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8847bff700 1 --2- 172.21.15.190:0/2770514969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88400c5420 0x7f88400c6ca0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8840088ec0 tx=0x7f8830009250 comp rx=0 tx=0).stop 2023-12-26T21:36:27.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8847bff700 1 -- 172.21.15.190:0/2770514969 shutdown_connections 2023-12-26T21:36:27.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8847bff700 1 --2- 172.21.15.190:0/2770514969 >> v2:172.21.15.131:3300/0 conn(0x7f88400ced90 0x7f88400cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8847bff700 1 --2- 172.21.15.190:0/2770514969 >> v2:172.21.15.190:3300/0 conn(0x7f8840159920 0x7f8840159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8847bff700 1 --2- 172.21.15.190:0/2770514969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88400c5420 0x7f88400c6ca0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8847bff700 1 -- 172.21.15.190:0/2770514969 >> 172.21.15.190:0/2770514969 conn(0x7f8840099980 msgr2=0x7f884006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:27.296 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8847bff700 1 -- 172.21.15.190:0/2770514969 shutdown_connections 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.291+0000 7f8847bff700 1 -- 172.21.15.190:0/2770514969 wait complete. 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.293+0000 7f8847bff700 1 Processor -- start 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.293+0000 7f8847bff700 1 -- start start 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f8847bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88400c5420 0x7f88400b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f8847bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400ced90 0x7f88400c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f8847bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8840159920 0x7f8840154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f8847bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400b4410 con 0x7f88400ced90 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f8847bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400b46f0 con 0x7f88400c5420 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f8847bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88400b49d0 con 0x7f8840159920 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f884599b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88400c5420 0x7f88400b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f884599b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88400c5420 0x7f88400b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49234/0 (socket says 172.21.15.190:49234) 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f884619c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8840159920 0x7f8840154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.297 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f884599b700 1 -- 172.21.15.190:0/994988575 learned_addr learned my addr 172.21.15.190:0/994988575 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.294+0000 7f884519a700 1 --2- 172.21.15.190:0/994988575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400ced90 0x7f88400c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f884599b700 1 -- 172.21.15.190:0/994988575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8840159920 msgr2=0x7f8840154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f884599b700 1 --2- 172.21.15.190:0/994988575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8840159920 0x7f8840154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f884599b700 1 -- 172.21.15.190:0/994988575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400ced90 msgr2=0x7f88400c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f884599b700 1 --2- 172.21.15.190:0/994988575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400ced90 0x7f88400c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f884599b700 1 -- 172.21.15.190:0/994988575 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88300094f0 con 0x7f88400c5420 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f884599b700 1 --2- 172.21.15.190:0/994988575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88400c5420 0x7f88400b3e30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f883c00ec80 tx=0x7f883c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f8835ffb700 1 -- 172.21.15.190:0/994988575 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00c850 con 0x7f88400c5420 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f8835ffb700 1 -- 172.21.15.190:0/994988575 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c00c9b0 con 0x7f88400c5420 2023-12-26T21:36:27.298 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f8835ffb700 1 -- 172.21.15.190:0/994988575 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00cd30 con 0x7f88400c5420 2023-12-26T21:36:27.299 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f8847bff700 1 -- 172.21.15.190:0/994988575 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88400b4cb0 con 0x7f88400c5420 2023-12-26T21:36:27.299 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.295+0000 7f8847bff700 1 -- 172.21.15.190:0/994988575 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88401567d0 con 0x7f88400c5420 2023-12-26T21:36:27.299 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.296+0000 7f8835ffb700 1 -- 172.21.15.190:0/994988575 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f883c013900 con 0x7f88400c5420 2023-12-26T21:36:27.299 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.296+0000 7f8835ffb700 1 -- 172.21.15.190:0/994988575 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f883c01cb90 con 0x7f88400c5420 2023-12-26T21:36:27.299 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.296+0000 7f8847bff700 1 -- 172.21.15.190:0/994988575 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88400b4e40 con 0x7f88400c5420 2023-12-26T21:36:27.300 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:27.298+0000 7f21292781c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:36:27.300 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.299+0000 7f8835ffb700 1 -- 172.21.15.190:0/994988575 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f88400b4e40 con 0x7f88400c5420 2023-12-26T21:36:27.385 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.385+0000 7f8847bff700 1 -- 172.21.15.190:0/994988575 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a3915ff5-2514-4fcf-92e0-8f9144eab84c", "id": 6} v 0) v1 -- 0x7f88400635f0 con 0x7f88400c5420 2023-12-26T21:36:27.393 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.393+0000 7f8835ffb700 1 -- 172.21.15.190:0/994988575 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a3915ff5-2514-4fcf-92e0-8f9144eab84c", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f883c021060 con 0x7f88400c5420 2023-12-26T21:36:27.394 INFO:teuthology.orchestra.run.smithi190.stdout:6 2023-12-26T21:36:27.395 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.394+0000 7f8847bff700 1 -- 172.21.15.190:0/994988575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88400c5420 msgr2=0x7f88400b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.395 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.394+0000 7f8847bff700 1 --2- 172.21.15.190:0/994988575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88400c5420 0x7f88400b3e30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f883c00ec80 tx=0x7f883c009120 comp rx=0 tx=0).stop 2023-12-26T21:36:27.395 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.395+0000 7f8847bff700 1 -- 172.21.15.190:0/994988575 shutdown_connections 2023-12-26T21:36:27.395 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.395+0000 7f8847bff700 1 --2- 172.21.15.190:0/994988575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8840159920 0x7f8840154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.395 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.395+0000 7f8847bff700 1 --2- 172.21.15.190:0/994988575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88400c5420 0x7f88400b3e30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.395 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.395+0000 7f8847bff700 1 --2- 172.21.15.190:0/994988575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88400ced90 0x7f88400c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.395 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.395+0000 7f8847bff700 1 -- 172.21.15.190:0/994988575 >> 172.21.15.190:0/994988575 conn(0x7f8840099980 msgr2=0x7f884015a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:27.396 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.395+0000 7f8847bff700 1 -- 172.21.15.190:0/994988575 shutdown_connections 2023-12-26T21:36:27.396 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.395+0000 7f8847bff700 1 -- 172.21.15.190:0/994988575 wait complete. 2023-12-26T21:36:27.412 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 2e7abc9f-f98a-4741-b738-321bf1e85a40 7 2023-12-26T21:36:27.597 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.596+0000 7f3014dff700 1 Processor -- start 2023-12-26T21:36:27.597 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.597+0000 7f3014dff700 1 -- start start 2023-12-26T21:36:27.599 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.598+0000 7f3014dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30100b0750 0x7f30100b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.599 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.598+0000 7f3014dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3010156940 con 0x7f30100cb320 2023-12-26T21:36:27.599 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.598+0000 7f3014dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30100ae500 con 0x7f30100c9400 2023-12-26T21:36:27.599 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.598+0000 7f3014dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30100c8cc0 con 0x7f30100b0750 2023-12-26T21:36:27.599 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.598+0000 7f300dd9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30100b0750 0x7f30100b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.599 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.598+0000 7f300dd9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30100b0750 0x7f30100b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49240/0 (socket says 172.21.15.190:49240) 2023-12-26T21:36:27.599 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.598+0000 7f300dd9b700 1 -- 172.21.15.190:0/778557778 learned_addr learned my addr 172.21.15.190:0/778557778 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:27.599 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.598+0000 7f300ed9d700 1 --1- 172.21.15.190:0/778557778 >> v1:172.21.15.190:6789/0 conn(0x7f30100c9400 0x7f30100c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:42230/0 (socket says 172.21.15.190:42230) 2023-12-26T21:36:27.600 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.599+0000 7f300dd9b700 1 -- 172.21.15.190:0/778557778 >> v1:172.21.15.190:6789/0 conn(0x7f30100c9400 legacy=0x7f30100c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.600 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.599+0000 7f300dd9b700 1 -- 172.21.15.190:0/778557778 >> v1:172.21.15.131:6789/0 conn(0x7f30100cb320 legacy=0x7f3010069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.600 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.599+0000 7f300dd9b700 1 -- 172.21.15.190:0/778557778 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30100b0280 con 0x7f30100b0750 2023-12-26T21:36:27.600 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.599+0000 7f300d59a700 1 -- 172.21.15.190:0/778557778 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725353163 0 0) 0x7f3010156940 con 0x7f30100cb320 2023-12-26T21:36:27.600 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.599+0000 7f300dd9b700 1 --2- 172.21.15.190:0/778557778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100b0750 0x7f30100b0bf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f30040099b0 tx=0x7f3004009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da34e997a6d7be41 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:27.601 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.599+0000 7f300d59a700 1 -- 172.21.15.190:0/778557778 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300400b860 con 0x7f30100b0750 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.599+0000 7f300d59a700 1 -- 172.21.15.190:0/778557778 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300400b9c0 con 0x7f30100b0750 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.599+0000 7f300d59a700 1 -- 172.21.15.190:0/778557778 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300400bd60 con 0x7f30100b0750 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.600+0000 7f3014dff700 1 -- 172.21.15.190:0/778557778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100b0750 msgr2=0x7f30100b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.600+0000 7f3014dff700 1 --2- 172.21.15.190:0/778557778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100b0750 0x7f30100b0bf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f30040099b0 tx=0x7f3004009250 comp rx=0 tx=0).stop 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.600+0000 7f3014dff700 1 -- 172.21.15.190:0/778557778 shutdown_connections 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.600+0000 7f3014dff700 1 --2- 172.21.15.190:0/778557778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100b0750 0x7f30100b0bf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.600+0000 7f3014dff700 1 -- 172.21.15.190:0/778557778 >> 172.21.15.190:0/778557778 conn(0x7f3010068040 msgr2=0x7f3010069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.600+0000 7f3014dff700 1 -- 172.21.15.190:0/778557778 shutdown_connections 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.600+0000 7f3014dff700 1 -- 172.21.15.190:0/778557778 wait complete. 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.601+0000 7f3014dff700 1 Processor -- start 2023-12-26T21:36:27.602 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.601+0000 7f3014dff700 1 -- start start 2023-12-26T21:36:27.604 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.602+0000 7f3014dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30100b0750 0x7f3010155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.602+0000 7f3014dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100c9400 0x7f30100c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.602+0000 7f3014dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30100cb320 0x7f3010158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:27.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.602+0000 7f3014dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30100c6290 con 0x7f30100b0750 2023-12-26T21:36:27.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.602+0000 7f3014dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30100c53a0 con 0x7f30100c9400 2023-12-26T21:36:27.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.602+0000 7f3014dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30100c5680 con 0x7f30100cb320 2023-12-26T21:36:27.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300dd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100c9400 0x7f30100c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30100b0750 0x7f3010155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30100cb320 0x7f3010158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:27.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30100b0750 0x7f3010155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.190:51182/0 (socket says 172.21.15.190:51182) 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300e59c700 1 -- 172.21.15.190:0/246240447 learned_addr learned my addr 172.21.15.190:0/246240447 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300e59c700 1 -- 172.21.15.190:0/246240447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30100cb320 msgr2=0x7f3010158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300e59c700 1 --2- 172.21.15.190:0/246240447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30100cb320 0x7f3010158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300e59c700 1 -- 172.21.15.190:0/246240447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100c9400 msgr2=0x7f30100c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300e59c700 1 --2- 172.21.15.190:0/246240447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100c9400 0x7f30100c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300e59c700 1 -- 172.21.15.190:0/246240447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30040094f0 con 0x7f30100b0750 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300dd9b700 1 --2- 172.21.15.190:0/246240447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100c9400 0x7f30100c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.603+0000 7f300e59c700 1 --2- 172.21.15.190:0/246240447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30100b0750 0x7f3010155060 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f300000c530 tx=0x7f300000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.604+0000 7f2ffe7fc700 1 -- 172.21.15.190:0/246240447 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000011590 con 0x7f30100b0750 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.604+0000 7f2ffe7fc700 1 -- 172.21.15.190:0/246240447 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000011710 con 0x7f30100b0750 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.604+0000 7f3014dff700 1 -- 172.21.15.190:0/246240447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30100c5960 con 0x7f30100b0750 2023-12-26T21:36:27.606 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.604+0000 7f2ffe7fc700 1 -- 172.21.15.190:0/246240447 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000011ad0 con 0x7f30100b0750 2023-12-26T21:36:27.607 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.604+0000 7f3014dff700 1 -- 172.21.15.190:0/246240447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30100c5f20 con 0x7f30100b0750 2023-12-26T21:36:27.607 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.604+0000 7f2ffe7fc700 1 -- 172.21.15.190:0/246240447 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f300001c6c0 con 0x7f30100b0750 2023-12-26T21:36:27.607 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.604+0000 7f2ffe7fc700 1 -- 172.21.15.190:0/246240447 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f3000012a90 con 0x7f30100b0750 2023-12-26T21:36:27.607 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.605+0000 7f3014dff700 1 -- 172.21.15.190:0/246240447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30100657e0 con 0x7f30100b0750 2023-12-26T21:36:27.607 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.607+0000 7f2ffe7fc700 1 -- 172.21.15.190:0/246240447 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3000017070 con 0x7f30100b0750 2023-12-26T21:36:27.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.693+0000 7f3014dff700 1 -- 172.21.15.190:0/246240447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2e7abc9f-f98a-4741-b738-321bf1e85a40", "id": 7} v 0) v1 -- 0x7f30101563c0 con 0x7f30100b0750 2023-12-26T21:36:27.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.697+0000 7f2ffe7fc700 1 -- 172.21.15.190:0/246240447 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2e7abc9f-f98a-4741-b738-321bf1e85a40", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f300001cba0 con 0x7f30100b0750 2023-12-26T21:36:27.698 INFO:teuthology.orchestra.run.smithi190.stdout:7 2023-12-26T21:36:27.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.698+0000 7f3014dff700 1 -- 172.21.15.190:0/246240447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30100b0750 msgr2=0x7f3010155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:27.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.698+0000 7f3014dff700 1 --2- 172.21.15.190:0/246240447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30100b0750 0x7f3010155060 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f300000c530 tx=0x7f300000c840 comp rx=0 tx=0).stop 2023-12-26T21:36:27.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.698+0000 7f3014dff700 1 -- 172.21.15.190:0/246240447 shutdown_connections 2023-12-26T21:36:27.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.698+0000 7f3014dff700 1 --2- 172.21.15.190:0/246240447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30100cb320 0x7f3010158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.698+0000 7f3014dff700 1 --2- 172.21.15.190:0/246240447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30100b0750 0x7f3010155060 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.698+0000 7f3014dff700 1 --2- 172.21.15.190:0/246240447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30100c9400 0x7f30100c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:27.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.699+0000 7f3014dff700 1 -- 172.21.15.190:0/246240447 >> 172.21.15.190:0/246240447 conn(0x7f3010068040 msgr2=0x7f30101559c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:27.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.699+0000 7f3014dff700 1 -- 172.21.15.190:0/246240447 shutdown_connections 2023-12-26T21:36:27.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-12-26T21:36:27.699+0000 7f3014dff700 1 -- 172.21.15.190:0/246240447 wait complete. 2023-12-26T21:36:27.715 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-26T21:36:27.715 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-26T21:36:27.720 INFO:tasks.ceph.osd.0:Started 2023-12-26T21:36:27.720 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-26T21:36:27.720 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-26T21:36:27.727 INFO:tasks.ceph.osd.1:Started 2023-12-26T21:36:27.727 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-26T21:36:27.727 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-26T21:36:27.731 INFO:tasks.ceph.osd.2:Started 2023-12-26T21:36:27.731 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-26T21:36:27.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-26T21:36:27.737 INFO:tasks.ceph.osd.3:Started 2023-12-26T21:36:27.737 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-26T21:36:27.738 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-26T21:36:27.742 INFO:tasks.ceph.osd.4:Started 2023-12-26T21:36:27.742 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-26T21:36:27.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-26T21:36:27.746 INFO:tasks.ceph.osd.5:Started 2023-12-26T21:36:27.746 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-26T21:36:27.746 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-26T21:36:27.751 INFO:tasks.ceph.osd.6:Started 2023-12-26T21:36:27.751 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-26T21:36:27.751 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-26T21:36:27.757 INFO:tasks.ceph.osd.7:Started 2023-12-26T21:36:27.758 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:28.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.016+0000 7fdc829ff700 1 Processor -- start 2023-12-26T21:36:28.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.016+0000 7fdc829ff700 1 -- start start 2023-12-26T21:36:28.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.017+0000 7fdc829ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdc7c1596a0 0x7fdc7c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.017+0000 7fdc829ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7c0afcd0 con 0x7fdc7c0b0870 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.017+0000 7fdc829ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7c0b3e70 con 0x7fdc7c159a70 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.017+0000 7fdc829ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c0b0cb0 con 0x7fdc7c1596a0 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.017+0000 7fdc7bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdc7c1596a0 0x7fdc7c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.017+0000 7fdc80f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdc7c0b0870 0x7fdc7c0b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50500/0 (socket says 172.21.15.131:50500) 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.017+0000 7fdc80f9c700 1 -- 172.21.15.131:0/1368661439 learned_addr learned my addr 172.21.15.131:0/1368661439 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.018+0000 7fdc7bfff700 1 -- 172.21.15.131:0/1368661439 >> v1:172.21.15.190:6789/0 conn(0x7fdc7c159a70 legacy=0x7fdc7c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.018+0000 7fdc7bfff700 1 -- 172.21.15.131:0/1368661439 >> v1:172.21.15.131:6789/0 conn(0x7fdc7c0b0870 legacy=0x7fdc7c0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.018+0000 7fdc7bfff700 1 -- 172.21.15.131:0/1368661439 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c0b0f20 con 0x7fdc7c1596a0 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.018+0000 7fdc7affd700 1 -- 172.21.15.131:0/1368661439 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328850305 0 0) 0x7fdc7c0afcd0 con 0x7fdc7c0b0870 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.018+0000 7fdc7bfff700 1 --2- 172.21.15.131:0/1368661439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c1596a0 0x7fdc7c0b2380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0098e0 tx=0x7fdc6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2144deccdbebb209 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.018+0000 7fdc7affd700 1 -- 172.21.15.131:0/1368661439 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c009e80 con 0x7fdc7c1596a0 2023-12-26T21:36:28.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.018+0000 7fdc7affd700 1 -- 172.21.15.131:0/1368661439 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6c00b860 con 0x7fdc7c1596a0 2023-12-26T21:36:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.018+0000 7fdc7affd700 1 -- 172.21.15.131:0/1368661439 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c00bbe0 con 0x7fdc7c1596a0 2023-12-26T21:36:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.019+0000 7fdc829ff700 1 -- 172.21.15.131:0/1368661439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c1596a0 msgr2=0x7fdc7c0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.019+0000 7fdc829ff700 1 --2- 172.21.15.131:0/1368661439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c1596a0 0x7fdc7c0b2380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0098e0 tx=0x7fdc6c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.019+0000 7fdc829ff700 1 -- 172.21.15.131:0/1368661439 shutdown_connections 2023-12-26T21:36:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.019+0000 7fdc829ff700 1 --2- 172.21.15.131:0/1368661439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c1596a0 0x7fdc7c0b2380 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.019+0000 7fdc829ff700 1 -- 172.21.15.131:0/1368661439 >> 172.21.15.131:0/1368661439 conn(0x7fdc7c06ece0 msgr2=0x7fdc7c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.019+0000 7fdc829ff700 1 -- 172.21.15.131:0/1368661439 shutdown_connections 2023-12-26T21:36:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.019+0000 7fdc829ff700 1 -- 172.21.15.131:0/1368661439 wait complete. 2023-12-26T21:36:28.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.020+0000 7fdc829ff700 1 Processor -- start 2023-12-26T21:36:28.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.021+0000 7fdc829ff700 1 -- start start 2023-12-26T21:36:28.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.021+0000 7fdc829ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c0b0870 0x7fdc7c16d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:28.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.021+0000 7fdc829ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc7c1596a0 0x7fdc7c0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:28.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.021+0000 7fdc829ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c159a70 0x7fdc7c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:28.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.021+0000 7fdc829ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0c4140 con 0x7fdc7c0b0870 2023-12-26T21:36:28.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.021+0000 7fdc829ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c15cc20 con 0x7fdc7c159a70 2023-12-26T21:36:28.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.021+0000 7fdc829ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c16dad0 con 0x7fdc7c1596a0 2023-12-26T21:36:28.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c0b0870 0x7fdc7c16d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c0b0870 0x7fdc7c16d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53656/0 (socket says 172.21.15.131:53656) 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7bfff700 1 -- 172.21.15.131:0/649131832 learned_addr learned my addr 172.21.15.131:0/649131832 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7b7fe700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc7c1596a0 0x7fdc7c0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc80f9c700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c159a70 0x7fdc7c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7bfff700 1 -- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc7c1596a0 msgr2=0x7fdc7c0c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7bfff700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc7c1596a0 0x7fdc7c0c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7bfff700 1 -- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c159a70 msgr2=0x7fdc7c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7bfff700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c159a70 0x7fdc7c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7bfff700 1 -- 172.21.15.131:0/649131832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c009420 con 0x7fdc7c0b0870 2023-12-26T21:36:28.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7b7fe700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc7c1596a0 0x7fdc7c0c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:28.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc80f9c700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c159a70 0x7fdc7c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:28.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.022+0000 7fdc7bfff700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c0b0870 0x7fdc7c16d4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c089710 tx=0x7fdc6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:28.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.023+0000 7fdc63fff700 1 -- 172.21.15.131:0/649131832 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c00bba0 con 0x7fdc7c0b0870 2023-12-26T21:36:28.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.023+0000 7fdc63fff700 1 -- 172.21.15.131:0/649131832 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6c011630 con 0x7fdc7c0b0870 2023-12-26T21:36:28.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.023+0000 7fdc829ff700 1 -- 172.21.15.131:0/649131832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c16ddb0 con 0x7fdc7c0b0870 2023-12-26T21:36:28.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.023+0000 7fdc63fff700 1 -- 172.21.15.131:0/649131832 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0119d0 con 0x7fdc7c0b0870 2023-12-26T21:36:28.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.023+0000 7fdc829ff700 1 -- 172.21.15.131:0/649131832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c0c5730 con 0x7fdc7c0b0870 2023-12-26T21:36:28.026 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:28.023+0000 7f442e0ca1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:36:28.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.023+0000 7fdc63fff700 1 -- 172.21.15.131:0/649131832 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fdc6c02e3f0 con 0x7fdc7c0b0870 2023-12-26T21:36:28.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.023+0000 7fdc63fff700 1 -- 172.21.15.131:0/649131832 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fdc6c025ad0 con 0x7fdc7c0b0870 2023-12-26T21:36:28.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.024+0000 7fdc829ff700 1 -- 172.21.15.131:0/649131832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c0657e0 con 0x7fdc7c0b0870 2023-12-26T21:36:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.026+0000 7fdc63fff700 1 -- 172.21.15.131:0/649131832 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fdc6c033060 con 0x7fdc7c0b0870 2023-12-26T21:36:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.099+0000 7fdc829ff700 1 -- 172.21.15.131:0/649131832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc7c0b3540 con 0x7fdc7c0b0870 2023-12-26T21:36:28.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.099+0000 7fdc63fff700 1 -- 172.21.15.131:0/649131832 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fdc6c015020 con 0x7fdc7c0b0870 2023-12-26T21:36:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:28.100 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":10,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:27.695410+0000","last_up_change":"0.000000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.100+0000 7fdc829ff700 1 -- 172.21.15.131:0/649131832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c0b0870 msgr2=0x7fdc7c16d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.100+0000 7fdc829ff700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c0b0870 0x7fdc7c16d4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c089710 tx=0x7fdc6c017000 comp rx=0 tx=0).stop 2023-12-26T21:36:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.100+0000 7fdc829ff700 1 -- 172.21.15.131:0/649131832 shutdown_connections 2023-12-26T21:36:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.100+0000 7fdc829ff700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc7c159a70 0x7fdc7c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.100+0000 7fdc829ff700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc7c0b0870 0x7fdc7c16d4f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.100+0000 7fdc829ff700 1 --2- 172.21.15.131:0/649131832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc7c1596a0 0x7fdc7c0c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.100+0000 7fdc829ff700 1 -- 172.21.15.131:0/649131832 >> 172.21.15.131:0/649131832 conn(0x7fdc7c06ece0 msgr2=0x7fdc7c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.100+0000 7fdc829ff700 1 -- 172.21.15.131:0/649131832 shutdown_connections 2023-12-26T21:36:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:28.100+0000 7fdc829ff700 1 -- 172.21.15.131:0/649131832 wait complete. 2023-12-26T21:36:28.118 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-26T21:36:28.118 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-26T21:36:28.127 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:28.126+0000 7f442e0ca1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:36:28.180 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:28.180+0000 7f21292781c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:36:28.282 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:28.282+0000 7f21292781c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:36:28.289 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:28.289+0000 7f442e0ca1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:36:28.431 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-12-26T21:36:28.431+0000 7f1584fff540 -1 Falling back to public interface 2023-12-26T21:36:28.432 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-12-26T21:36:28.431+0000 7f60088f1540 -1 Falling back to public interface 2023-12-26T21:36:28.432 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-12-26T21:36:28.431+0000 7f157c1fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-12-26T21:36:28.439 INFO:tasks.ceph.osd.2.smithi131.stderr:2023-12-26T21:36:28.439+0000 7ff402674540 -1 Falling back to public interface 2023-12-26T21:36:28.444 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:28.444+0000 7f21292781c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:36:28.449 INFO:tasks.ceph.osd.0.smithi131.stderr:2023-12-26T21:36:28.449+0000 7f981718a540 -1 Falling back to public interface 2023-12-26T21:36:28.468 INFO:tasks.ceph.osd.1.smithi131.stderr:2023-12-26T21:36:28.467+0000 7f3cd7bce540 -1 Falling back to public interface 2023-12-26T21:36:28.468 INFO:tasks.ceph.osd.3.smithi131.stderr:2023-12-26T21:36:28.468+0000 7f8364602540 -1 Falling back to public interface 2023-12-26T21:36:28.472 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-12-26T21:36:28.472+0000 7f904dc01540 -1 Falling back to public interface 2023-12-26T21:36:28.473 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-12-26T21:36:28.473+0000 7f86cfda5540 -1 Falling back to public interface 2023-12-26T21:36:28.478 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-12-26T21:36:28.478+0000 7f86c6ffd700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6824/0: (98) Address already in use 2023-12-26T21:36:28.925 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:28.998 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:28.996+0000 7f442e0ca1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:36:29.094 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.094+0000 7f1a195ff700 1 Processor -- start 2023-12-26T21:36:29.094 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.094+0000 7f1a195ff700 1 -- start start 2023-12-26T21:36:29.098 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.095+0000 7f1a195ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1a1415c940 0x7f1a140d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:29.098 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.095+0000 7f1a195ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1a1415cd10 0x7f1a140ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:29.098 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.095+0000 7f1a195ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415dce0 0x7f1a140ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:29.098 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.095+0000 7f1a195ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a140cca30 con 0x7f1a1415cd10 2023-12-26T21:36:29.098 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.095+0000 7f1a195ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a140cfc10 con 0x7f1a1415c940 2023-12-26T21:36:29.098 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.095+0000 7f1a195ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140b1040 con 0x7f1a1415dce0 2023-12-26T21:36:29.098 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a127fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1a1415cd10 0x7f1a140ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a127fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1a1415cd10 0x7f1a140ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53670/0 (socket says 172.21.15.131:53670) 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a127fc700 1 -- 172.21.15.131:0/4008049323 learned_addr learned my addr 172.21.15.131:0/4008049323 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a12ffd700 1 --2- 172.21.15.131:0/4008049323 >> v2:172.21.15.190:3300/0 conn(0x7f1a1415c940 0x7f1a140d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a137fe700 1 --2- 172.21.15.131:0/4008049323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415dce0 0x7f1a140ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a127fc700 1 -- 172.21.15.131:0/4008049323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415dce0 msgr2=0x7f1a140ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a127fc700 1 --2- 172.21.15.131:0/4008049323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415dce0 0x7f1a140ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a127fc700 1 -- 172.21.15.131:0/4008049323 >> v2:172.21.15.190:3300/0 conn(0x7f1a1415c940 msgr2=0x7f1a140d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a127fc700 1 --2- 172.21.15.131:0/4008049323 >> v2:172.21.15.190:3300/0 conn(0x7f1a1415c940 0x7f1a140d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a127fc700 1 -- 172.21.15.131:0/4008049323 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a140cf000 con 0x7f1a1415cd10 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a12ffd700 1 --2- 172.21.15.131:0/4008049323 >> v2:172.21.15.190:3300/0 conn(0x7f1a1415c940 0x7f1a140d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a127fc700 1 --2- 172.21.15.131:0/4008049323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415cd10 0x7f1a140ce9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1a140ce0b0 tx=0x7f1a08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178a8a1432b4c13a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.096+0000 7f1a11ffb700 1 -- 172.21.15.131:0/4008049323 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800b640 con 0x7f1a1415cd10 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a11ffb700 1 -- 172.21.15.131:0/4008049323 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0800b7a0 con 0x7f1a1415cd10 2023-12-26T21:36:29.099 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a11ffb700 1 -- 172.21.15.131:0/4008049323 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800bb60 con 0x7f1a1415cd10 2023-12-26T21:36:29.100 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a195ff700 1 -- 172.21.15.131:0/4008049323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415cd10 msgr2=0x7f1a140ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:29.100 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a195ff700 1 --2- 172.21.15.131:0/4008049323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415cd10 0x7f1a140ce9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1a140ce0b0 tx=0x7f1a08009250 comp rx=0 tx=0).stop 2023-12-26T21:36:29.100 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a195ff700 1 -- 172.21.15.131:0/4008049323 shutdown_connections 2023-12-26T21:36:29.100 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a195ff700 1 --2- 172.21.15.131:0/4008049323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415dce0 0x7f1a140ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.100 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a195ff700 1 --2- 172.21.15.131:0/4008049323 >> v2:172.21.15.190:3300/0 conn(0x7f1a1415c940 0x7f1a140d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.100 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a195ff700 1 --2- 172.21.15.131:0/4008049323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415cd10 0x7f1a140ce9b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.100 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a195ff700 1 -- 172.21.15.131:0/4008049323 >> 172.21.15.131:0/4008049323 conn(0x7f1a1406ece0 msgr2=0x7f1a1406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:29.100 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a195ff700 1 -- 172.21.15.131:0/4008049323 shutdown_connections 2023-12-26T21:36:29.100 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.097+0000 7f1a195ff700 1 -- 172.21.15.131:0/4008049323 wait complete. 2023-12-26T21:36:29.102 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.099+0000 7f1a195ff700 1 Processor -- start 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.099+0000 7f1a195ff700 1 -- start start 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a195ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a1415c940 0x7f1a1416a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a195ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415cd10 0x7f1a140a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a195ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415dce0 0x7f1a1416ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a195ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140a7c20 con 0x7f1a1415dce0 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a195ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140c9b10 con 0x7f1a1415c940 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a195ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140a7ea0 con 0x7f1a1415cd10 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a137fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415dce0 0x7f1a1416ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a137fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415dce0 0x7f1a1416ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53686/0 (socket says 172.21.15.131:53686) 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a137fe700 1 -- 172.21.15.131:0/1298648960 learned_addr learned my addr 172.21.15.131:0/1298648960 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a127fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415cd10 0x7f1a140a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a12ffd700 1 --2- 172.21.15.131:0/1298648960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a1415c940 0x7f1a1416a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a137fe700 1 -- 172.21.15.131:0/1298648960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415cd10 msgr2=0x7f1a140a76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:29.103 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a137fe700 1 --2- 172.21.15.131:0/1298648960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415cd10 0x7f1a140a76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a137fe700 1 -- 172.21.15.131:0/1298648960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a1415c940 msgr2=0x7f1a1416a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a137fe700 1 --2- 172.21.15.131:0/1298648960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a1415c940 0x7f1a1416a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a137fe700 1 -- 172.21.15.131:0/1298648960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a04009420 con 0x7f1a1415dce0 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.100+0000 7f1a127fc700 1 --2- 172.21.15.131:0/1298648960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415cd10 0x7f1a140a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.101+0000 7f1a137fe700 1 --2- 172.21.15.131:0/1298648960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415dce0 0x7f1a1416ad40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009e90 tx=0x7f1a0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.101+0000 7f19faffd700 1 -- 172.21.15.131:0/1298648960 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c00cb70 con 0x7f1a1415dce0 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.101+0000 7f19faffd700 1 -- 172.21.15.131:0/1298648960 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c00f040 con 0x7f1a1415dce0 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.101+0000 7f1a195ff700 1 -- 172.21.15.131:0/1298648960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a080094f0 con 0x7f1a1415dce0 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.101+0000 7f19faffd700 1 -- 172.21.15.131:0/1298648960 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c014480 con 0x7f1a1415dce0 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.101+0000 7f1a195ff700 1 -- 172.21.15.131:0/1298648960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1416b280 con 0x7f1a1415dce0 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.101+0000 7f19faffd700 1 -- 172.21.15.131:0/1298648960 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1a0c014830 con 0x7f1a1415dce0 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.101+0000 7f19faffd700 1 -- 172.21.15.131:0/1298648960 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f1a0c014ae0 con 0x7f1a1415dce0 2023-12-26T21:36:29.104 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.102+0000 7f1a195ff700 1 -- 172.21.15.131:0/1298648960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a140c9960 con 0x7f1a1415dce0 2023-12-26T21:36:29.105 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.104+0000 7f19faffd700 1 -- 172.21.15.131:0/1298648960 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1a0c018020 con 0x7f1a1415dce0 2023-12-26T21:36:29.147 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:29.147+0000 7f21292781c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:36:29.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.176+0000 7f1a195ff700 1 -- 172.21.15.131:0/1298648960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a140717f0 con 0x7f1a1415dce0 2023-12-26T21:36:29.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.177+0000 7f19faffd700 1 -- 172.21.15.131:0/1298648960 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f1a0c014d60 con 0x7f1a1415dce0 2023-12-26T21:36:29.178 INFO:teuthology.misc.health.smithi131.stdout: 2023-12-26T21:36:29.178 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":10,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:27.695410+0000","last_up_change":"0.000000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:29.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.178+0000 7f1a195ff700 1 -- 172.21.15.131:0/1298648960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415dce0 msgr2=0x7f1a1416ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:29.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.178+0000 7f1a195ff700 1 --2- 172.21.15.131:0/1298648960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415dce0 0x7f1a1416ad40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009e90 tx=0x7f1a0c00b040 comp rx=0 tx=0).stop 2023-12-26T21:36:29.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.178+0000 7f1a195ff700 1 -- 172.21.15.131:0/1298648960 shutdown_connections 2023-12-26T21:36:29.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.178+0000 7f1a195ff700 1 --2- 172.21.15.131:0/1298648960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a1415dce0 0x7f1a1416ad40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.178+0000 7f1a195ff700 1 --2- 172.21.15.131:0/1298648960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a1415c940 0x7f1a1416a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.178+0000 7f1a195ff700 1 --2- 172.21.15.131:0/1298648960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a1415cd10 0x7f1a140a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:29.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.178+0000 7f1a195ff700 1 -- 172.21.15.131:0/1298648960 >> 172.21.15.131:0/1298648960 conn(0x7f1a1406ece0 msgr2=0x7f1a140b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:29.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.178+0000 7f1a195ff700 1 -- 172.21.15.131:0/1298648960 shutdown_connections 2023-12-26T21:36:29.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:29.178+0000 7f1a195ff700 1 -- 172.21.15.131:0/1298648960 wait complete. 2023-12-26T21:36:29.194 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-26T21:36:29.205 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:29.204+0000 7f442e0ca1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:36:29.357 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:29.356+0000 7f21292781c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:36:29.415 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:29.415+0000 7f442e0ca1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:36:29.564 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:29.563+0000 7f21292781c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:36:30.390 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:30.390+0000 7f442e0ca1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:36:30.525 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:30.525+0000 7f21292781c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:36:30.668 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:30.667+0000 7f442e0ca1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:36:30.764 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:30.764+0000 7f442e0ca1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:36:30.798 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:30.798+0000 7f21292781c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:36:30.890 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:30.890+0000 7f21292781c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:36:30.977 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:30.977+0000 7f442e0ca1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:36:31.124 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:36:31.124+0000 7f21292781c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:36:31.786 INFO:tasks.ceph.osd.2.smithi131.stderr:2023-12-26T21:36:31.786+0000 7ff402674540 -1 osd.2 0 log_to_monitors true 2023-12-26T21:36:31.969 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-12-26T21:36:31.969+0000 7f1584fff540 -1 osd.4 0 log_to_monitors true 2023-12-26T21:36:31.994 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-12-26T21:36:31.994+0000 7f60088f1540 -1 osd.7 0 log_to_monitors true 2023-12-26T21:36:32.005 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:32.004+0000 7f43e8bfa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-26T21:36:32.006 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:36:32.005+0000 7f43e8bfa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-26T21:36:32.041 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-12-26T21:36:32.041+0000 7f86cfda5540 -1 osd.5 0 log_to_monitors true 2023-12-26T21:36:32.042 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-12-26T21:36:32.042+0000 7f904dc01540 -1 osd.6 0 log_to_monitors true 2023-12-26T21:36:32.047 INFO:tasks.ceph.osd.0.smithi131.stderr:2023-12-26T21:36:32.047+0000 7f981718a540 -1 osd.0 0 log_to_monitors true 2023-12-26T21:36:32.059 INFO:tasks.ceph.osd.1.smithi131.stderr:2023-12-26T21:36:32.058+0000 7f3cd7bce540 -1 osd.1 0 log_to_monitors true 2023-12-26T21:36:32.082 INFO:tasks.ceph.osd.3.smithi131.stderr:2023-12-26T21:36:32.081+0000 7f8364602540 -1 osd.3 0 log_to_monitors true 2023-12-26T21:36:34.147 INFO:tasks.ceph.osd.2.smithi131.stderr:2023-12-26T21:36:34.146+0000 7ff3f61f6700 -1 osd.2 0 waiting for initial osdmap 2023-12-26T21:36:34.202 INFO:tasks.ceph.osd.2.smithi131.stderr:2023-12-26T21:36:34.201+0000 7ff3ef1e8700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-26T21:36:34.667 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-12-26T21:36:34.667+0000 7f5ffc3f6700 -1 osd.7 0 waiting for initial osdmap 2023-12-26T21:36:34.698 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-12-26T21:36:34.698+0000 7f157a1f9700 -1 osd.4 0 waiting for initial osdmap 2023-12-26T21:36:34.701 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-12-26T21:36:34.701+0000 7f5ff53e8700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-26T21:36:34.725 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-12-26T21:36:34.725+0000 7f15719e8700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-26T21:36:35.559 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-12-26T21:36:35.559+0000 7f86c37f6700 -1 osd.5 0 waiting for initial osdmap 2023-12-26T21:36:35.567 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-12-26T21:36:35.567+0000 7f86bc7e8700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-26T21:36:35.592 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-12-26T21:36:35.592+0000 7f90415f6700 -1 osd.6 0 waiting for initial osdmap 2023-12-26T21:36:35.599 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-12-26T21:36:35.600+0000 7f903a5e8700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-26T21:36:35.839 INFO:tasks.ceph.osd.3.smithi131.stderr:2023-12-26T21:36:35.839+0000 7f8357ff6700 -1 osd.3 0 waiting for initial osdmap 2023-12-26T21:36:35.848 INFO:tasks.ceph.osd.3.smithi131.stderr:2023-12-26T21:36:35.848+0000 7f8350fe8700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-26T21:36:35.891 INFO:tasks.ceph.osd.1.smithi131.stderr:2023-12-26T21:36:35.890+0000 7f3ccb5f6700 -1 osd.1 0 waiting for initial osdmap 2023-12-26T21:36:35.898 INFO:tasks.ceph.osd.1.smithi131.stderr:2023-12-26T21:36:35.898+0000 7f3cc45e8700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-26T21:36:35.907 INFO:tasks.ceph.osd.0.smithi131.stderr:2023-12-26T21:36:35.906+0000 7f980abf6700 -1 osd.0 0 waiting for initial osdmap 2023-12-26T21:36:35.914 INFO:tasks.ceph.osd.0.smithi131.stderr:2023-12-26T21:36:35.913+0000 7f9803be8700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-26T21:36:36.000 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:36.169 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.169+0000 7f7b981ff700 1 Processor -- start 2023-12-26T21:36:36.169 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.169+0000 7f7b981ff700 1 -- start start 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b981ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b90159460 0x7f7b90159830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b981ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7b900b2400 0x7f7b900b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b981ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b900b4470 0x7f7b900b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b981ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b900cb280 con 0x7f7b90159460 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b981ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b900b51e0 con 0x7f7b900b2400 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b981ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900b54c0 con 0x7f7b900b4470 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b95f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b90159460 0x7f7b90159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b95f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b90159460 0x7f7b90159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44118/0 (socket says 172.21.15.131:44118) 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b95f9b700 1 -- 172.21.15.131:0/243094898 learned_addr learned my addr 172.21.15.131:0/243094898 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b9579a700 1 --2- 172.21.15.131:0/243094898 >> v2:172.21.15.190:3300/0 conn(0x7f7b900b2400 0x7f7b900b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.170+0000 7f7b9679c700 1 --2- 172.21.15.131:0/243094898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b900b4470 0x7f7b900b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.171+0000 7f7b95f9b700 1 -- 172.21.15.131:0/243094898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b900b4470 msgr2=0x7f7b900b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:36.172 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.171+0000 7f7b95f9b700 1 --2- 172.21.15.131:0/243094898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b900b4470 0x7f7b900b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.173 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.171+0000 7f7b95f9b700 1 -- 172.21.15.131:0/243094898 >> v2:172.21.15.190:3300/0 conn(0x7f7b900b2400 msgr2=0x7f7b900b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:36.173 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.171+0000 7f7b95f9b700 1 --2- 172.21.15.131:0/243094898 >> v2:172.21.15.190:3300/0 conn(0x7f7b900b2400 0x7f7b900b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.173 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.171+0000 7f7b95f9b700 1 -- 172.21.15.131:0/243094898 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900b57a0 con 0x7f7b90159460 2023-12-26T21:36:36.173 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.171+0000 7f7b95f9b700 1 --2- 172.21.15.131:0/243094898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b90159460 0x7f7b90159830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7b900cb040 tx=0x7f7b8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f93344b74740a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:36.173 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.171+0000 7f7b94f99700 1 -- 172.21.15.131:0/243094898 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00b640 con 0x7f7b90159460 2023-12-26T21:36:36.173 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.171+0000 7f7b94f99700 1 -- 172.21.15.131:0/243094898 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00b7a0 con 0x7f7b90159460 2023-12-26T21:36:36.173 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.171+0000 7f7b94f99700 1 -- 172.21.15.131:0/243094898 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00bb60 con 0x7f7b90159460 2023-12-26T21:36:36.174 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.172+0000 7f7b981ff700 1 -- 172.21.15.131:0/243094898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b90159460 msgr2=0x7f7b90159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:36.174 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.172+0000 7f7b981ff700 1 --2- 172.21.15.131:0/243094898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b90159460 0x7f7b90159830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7b900cb040 tx=0x7f7b8c009250 comp rx=0 tx=0).stop 2023-12-26T21:36:36.174 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.172+0000 7f7b981ff700 1 -- 172.21.15.131:0/243094898 shutdown_connections 2023-12-26T21:36:36.174 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.172+0000 7f7b981ff700 1 --2- 172.21.15.131:0/243094898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b90159460 0x7f7b90159830 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.174 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.172+0000 7f7b981ff700 1 --2- 172.21.15.131:0/243094898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b900b4470 0x7f7b900b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.174 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.172+0000 7f7b981ff700 1 --2- 172.21.15.131:0/243094898 >> v2:172.21.15.190:3300/0 conn(0x7f7b900b2400 0x7f7b900b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.174 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.172+0000 7f7b981ff700 1 -- 172.21.15.131:0/243094898 >> 172.21.15.131:0/243094898 conn(0x7f7b9006ece0 msgr2=0x7f7b9006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:36.174 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.172+0000 7f7b981ff700 1 -- 172.21.15.131:0/243094898 shutdown_connections 2023-12-26T21:36:36.175 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.172+0000 7f7b981ff700 1 -- 172.21.15.131:0/243094898 wait complete. 2023-12-26T21:36:36.175 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.173+0000 7f7b981ff700 1 Processor -- start 2023-12-26T21:36:36.175 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.173+0000 7f7b981ff700 1 -- start start 2023-12-26T21:36:36.176 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.174+0000 7f7b981ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b900b2400 0x7f7b901489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.174+0000 7f7b981ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b900b4470 0x7f7b90169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.174+0000 7f7b981ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b90159460 0x7f7b9015e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.174+0000 7f7b981ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9016d1d0 con 0x7f7b900b4470 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.174+0000 7f7b981ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9016a690 con 0x7f7b900b2400 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.174+0000 7f7b981ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9016a970 con 0x7f7b90159460 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.174+0000 7f7b9579a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b900b4470 0x7f7b90169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b9579a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b900b4470 0x7f7b90169bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44120/0 (socket says 172.21.15.131:44120) 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b9579a700 1 -- 172.21.15.131:0/367661060 learned_addr learned my addr 172.21.15.131:0/367661060 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b95f9b700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b900b2400 0x7f7b901489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b9679c700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b90159460 0x7f7b9015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:36.177 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b9579a700 1 -- 172.21.15.131:0/367661060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b90159460 msgr2=0x7f7b9015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b9579a700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b90159460 0x7f7b9015e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b9579a700 1 -- 172.21.15.131:0/367661060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b900b2400 msgr2=0x7f7b901489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b9579a700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b900b2400 0x7f7b901489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b9579a700 1 -- 172.21.15.131:0/367661060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c0094f0 con 0x7f7b900b4470 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b9579a700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b900b4470 0x7f7b90169bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000ec80 tx=0x7f7b800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b85ffb700 1 -- 172.21.15.131:0/367661060 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000c8e0 con 0x7f7b900b4470 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b85ffb700 1 -- 172.21.15.131:0/367661060 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8000ca40 con 0x7f7b900b4470 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9016d420 con 0x7f7b900b4470 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.175+0000 7f7b85ffb700 1 -- 172.21.15.131:0/367661060 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000cde0 con 0x7f7b900b4470 2023-12-26T21:36:36.178 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.176+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b90148f10 con 0x7f7b900b4470 2023-12-26T21:36:36.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.177+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b900717f0 con 0x7f7b900b4470 2023-12-26T21:36:36.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.177+0000 7f7b85ffb700 1 -- 172.21.15.131:0/367661060 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7f7b80013680 con 0x7f7b900b4470 2023-12-26T21:36:36.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.178+0000 7f7b85ffb700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b74066ea0 0x7f7b74069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:36.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.179+0000 7f7b85ffb700 1 -- 172.21.15.131:0/367661060 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3067+0+0 (secure 0 0 0) 0x7f7b80083c60 con 0x7f7b900b4470 2023-12-26T21:36:36.179 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.179+0000 7f7b95f9b700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b74066ea0 0x7f7b74069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:36.180 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.179+0000 7f7b95f9b700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b74066ea0 0x7f7b74069350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b900cb040 tx=0x7f7b8c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:36.184 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.184+0000 7f7b85ffb700 1 -- 172.21.15.131:0/367661060 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b800519e0 con 0x7f7b900b4470 2023-12-26T21:36:36.378 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.377+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b900b5a80 con 0x7f7b900b4470 2023-12-26T21:36:36.378 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.378+0000 7f7b85ffb700 1 -- 172.21.15.131:0/367661060 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8314 (secure 0 0 0) 0x7f7b80055000 con 0x7f7b900b4470 2023-12-26T21:36:36.379 INFO:teuthology.misc.health.smithi131.stdout: 2023-12-26T21:36:36.379 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":14,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:35.182978+0000","last_up_change":"2023-12-26T21:36:35.182978+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:36.381 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b74066ea0 msgr2=0x7f7b74069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:36.381 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b74066ea0 0x7f7b74069350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b900cb040 tx=0x7f7b8c017000 comp rx=0 tx=0).stop 2023-12-26T21:36:36.381 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b900b4470 msgr2=0x7f7b90169bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:36.381 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b900b4470 0x7f7b90169bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000ec80 tx=0x7f7b800091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.381 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 shutdown_connections 2023-12-26T21:36:36.381 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b74066ea0 0x7f7b74069350 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.382 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b90159460 0x7f7b9015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.382 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b900b2400 0x7f7b901489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.382 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 --2- 172.21.15.131:0/367661060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b900b4470 0x7f7b90169bd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:36.382 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 >> 172.21.15.131:0/367661060 conn(0x7f7b9006ece0 msgr2=0x7f7b900cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:36.382 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.380+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 shutdown_connections 2023-12-26T21:36:36.382 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:36.381+0000 7f7b981ff700 1 -- 172.21.15.131:0/367661060 wait complete. 2023-12-26T21:36:36.396 DEBUG:teuthology.misc:3 of 8 OSDs are up 2023-12-26T21:36:43.202 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:43.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:36:43.299 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:36:43.381 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.380+0000 7fec8c1ff700 1 Processor -- start 2023-12-26T21:36:43.381 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.380+0000 7fec8c1ff700 1 -- start start 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.381+0000 7fec8c1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec840cd460 0x7fec840d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec8c1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec840cfc10 con 0x7fec8415c940 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec8c1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec840b1040 con 0x7fec8415cd10 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec8c1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec840cca30 con 0x7fec840cd460 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec89f9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fec8415c940 0x7fec8415dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58070/0 (socket says 172.21.15.131:58070) 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec89f9b700 1 -- 172.21.15.131:0/1572324748 learned_addr learned my addr 172.21.15.131:0/1572324748 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec8a79c700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec840cd460 0x7fec840d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844451986 0 0) 0x7fec840cfc10 con 0x7fec8415c940 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec70003570 con 0x7fec8415c940 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2754498539 0 0) 0x7fec70003570 con 0x7fec8415c940 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec840cd460 msgr2=0x7fec840d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:43.385 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec88f99700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec840cd460 0x7fec840d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 >> v1:172.21.15.190:6789/0 conn(0x7fec8415cd10 legacy=0x7fec840ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840cf000 con 0x7fec8415c940 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.382+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fec800017f0 con 0x7fec8415c940 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 >> v1:172.21.15.131:6789/0 conn(0x7fec8415c940 legacy=0x7fec8415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec88f99700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec70003a90 0x7fec70003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec88f99700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec840cd460 0x7fec70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec88f99700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec70004d50 0x7fec700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7000bb10 con 0x7fec70004d50 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7000bc80 con 0x7fec70003a90 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7000be40 con 0x7fec840cd460 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec8a79c700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec70004d50 0x7fec700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec89f9b700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec70003a90 0x7fec70003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.386 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec8979a700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec840cd460 0x7fec70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec8a79c700 1 -- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec840cd460 msgr2=0x7fec70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec8a79c700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec840cd460 0x7fec70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec8a79c700 1 -- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec70003a90 msgr2=0x7fec70003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec8a79c700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec70003a90 0x7fec70003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec8a79c700 1 -- 172.21.15.131:0/1572324748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec70003570 con 0x7fec70004d50 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.383+0000 7fec8a79c700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec70004d50 0x7fec700051c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009c00 tx=0x7fec7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b09bb8a2c0b10 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00e710 con 0x7fec70004d50 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c00e870 con 0x7fec70004d50 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec88f99700 1 -- 172.21.15.131:0/1572324748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00ec30 con 0x7fec70004d50 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec8c1ff700 1 -- 172.21.15.131:0/1572324748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec70004d50 msgr2=0x7fec700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec8c1ff700 1 --2- 172.21.15.131:0/1572324748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec70004d50 0x7fec700051c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009c00 tx=0x7fec7c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec8a79c700 1 -- 172.21.15.131:0/1572324748 reap_dead start 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec8c1ff700 1 -- 172.21.15.131:0/1572324748 shutdown_connections 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec8c1ff700 1 -- 172.21.15.131:0/1572324748 >> 172.21.15.131:0/1572324748 conn(0x7fec8406ece0 msgr2=0x7fec8406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:43.387 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec8c1ff700 1 -- 172.21.15.131:0/1572324748 shutdown_connections 2023-12-26T21:36:43.388 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.384+0000 7fec8c1ff700 1 -- 172.21.15.131:0/1572324748 wait complete. 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.386+0000 7fec8c1ff700 1 Processor -- start 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.386+0000 7fec8c1ff700 1 -- start start 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8c1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec70004d50 0x7fec84158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8c1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec840c7980 0x7fec841585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8c1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec840b43a0 0x7fec840b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8c1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec84158b10 con 0x7fec840c7980 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8c1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8415d560 con 0x7fec840b43a0 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8c1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec841695d0 con 0x7fec70004d50 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8979a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec840c7980 0x7fec841585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8979a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec840c7980 0x7fec841585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44160/0 (socket says 172.21.15.131:44160) 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8979a700 1 -- 172.21.15.131:0/4250026412 learned_addr learned my addr 172.21.15.131:0/4250026412 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:43.390 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8a79c700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec840b43a0 0x7fec840b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec89f9b700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec70004d50 0x7fec84158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8979a700 1 -- 172.21.15.131:0/4250026412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec70004d50 msgr2=0x7fec84158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8979a700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec70004d50 0x7fec84158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8979a700 1 -- 172.21.15.131:0/4250026412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec840b43a0 msgr2=0x7fec840b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8979a700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec840b43a0 0x7fec840b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.387+0000 7fec8979a700 1 -- 172.21.15.131:0/4250026412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840ca6a0 con 0x7fec840c7980 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.388+0000 7fec8979a700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec840c7980 0x7fec841585d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fec74002a60 tx=0x7fec74007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.388+0000 7fec79ffb700 1 -- 172.21.15.131:0/4250026412 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74002e40 con 0x7fec840c7980 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.388+0000 7fec79ffb700 1 -- 172.21.15.131:0/4250026412 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7400f8d0 con 0x7fec840c7980 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.388+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec840ca980 con 0x7fec840c7980 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.388+0000 7fec79ffb700 1 -- 172.21.15.131:0/4250026412 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400fc70 con 0x7fec840c7980 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.388+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8415da90 con 0x7fec840c7980 2023-12-26T21:36:43.391 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.389+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8415dcb0 con 0x7fec840c7980 2023-12-26T21:36:43.392 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.390+0000 7fec79ffb700 1 -- 172.21.15.131:0/4250026412 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fec74013600 con 0x7fec840c7980 2023-12-26T21:36:43.392 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.390+0000 7fec79ffb700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec68066f30 0x7fec680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.392 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.391+0000 7fec79ffb700 1 -- 172.21.15.131:0/4250026412 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fec74084800 con 0x7fec840c7980 2023-12-26T21:36:43.392 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.391+0000 7fec89f9b700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec68066f30 0x7fec680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.392 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.391+0000 7fec89f9b700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec68066f30 0x7fec680693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fec800011d0 tx=0x7fec840cfb80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:43.396 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.396+0000 7fec79ffb700 1 -- 172.21.15.131:0/4250026412 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec74051dd0 con 0x7fec840c7980 2023-12-26T21:36:43.592 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.591+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec840717f0 con 0x7fec840c7980 2023-12-26T21:36:43.592 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.592+0000 7fec79ffb700 1 -- 172.21.15.131:0/4250026412 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12175 (secure 0 0 0) 0x7fec740553f0 con 0x7fec840c7980 2023-12-26T21:36:43.593 INFO:teuthology.misc.health.smithi131.stdout: 2023-12-26T21:36:43.593 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":18,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:39.577205+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:43.595 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec68066f30 msgr2=0x7fec680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.595 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec68066f30 0x7fec680693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fec800011d0 tx=0x7fec840cfb80 comp rx=0 tx=0).stop 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec840c7980 msgr2=0x7fec841585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec840c7980 0x7fec841585d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fec74002a60 tx=0x7fec74007d10 comp rx=0 tx=0).stop 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 shutdown_connections 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec840b43a0 0x7fec840b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec68066f30 0x7fec680693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec70004d50 0x7fec84158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 --2- 172.21.15.131:0/4250026412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec840c7980 0x7fec841585d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 >> 172.21.15.131:0/4250026412 conn(0x7fec8406ece0 msgr2=0x7fec840cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 shutdown_connections 2023-12-26T21:36:43.596 INFO:teuthology.misc.health.smithi131.stderr:2023-12-26T21:36:43.595+0000 7fec8c1ff700 1 -- 172.21.15.131:0/4250026412 wait complete. 2023-12-26T21:36:43.610 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-26T21:36:43.611 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-26T21:36:43.611 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T21:36:43.611 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T21:36:43.616 INFO:tasks.ceph.mds.a:Started 2023-12-26T21:36:43.617 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T21:36:43.617 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T21:36:43.621 INFO:tasks.ceph.mds.c:Started 2023-12-26T21:36:43.621 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T21:36:43.621 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T21:36:43.625 INFO:tasks.ceph.mds.b:Started 2023-12-26T21:36:43.625 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T21:36:43.626 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T21:36:43.633 INFO:tasks.ceph.mds.d:Started 2023-12-26T21:36:43.633 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-26T21:36:43.634 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:43.760 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T21:36:43.761 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:36:43.771 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T21:36:43.775 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:36:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.838+0000 7feecc9ff700 1 Processor -- start 2023-12-26T21:36:43.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.838+0000 7feecc9ff700 1 -- start start 2023-12-26T21:36:43.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.839+0000 7feecc9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feec813c510 0x7feec80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.839+0000 7feecc9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec806b8c0 0x7feec806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.839+0000 7feecc9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feec80cc100 0x7feec80cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.839+0000 7feecc9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec80686c0 con 0x7feec80cc100 2023-12-26T21:36:43.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.839+0000 7feecc9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec8069520 con 0x7feec813c510 2023-12-26T21:36:43.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.839+0000 7feecc9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec806c250 con 0x7feec806b8c0 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec6d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feec80cc100 0x7feec80cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec6d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feec80cc100 0x7feec80cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44178/0 (socket says 172.21.15.131:44178) 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec6d9d700 1 -- 172.21.15.131:0/1173157396 learned_addr learned my addr 172.21.15.131:0/1173157396 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feebffff700 1 --2- 172.21.15.131:0/1173157396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec806b8c0 0x7feec806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec659c700 1 --2- 172.21.15.131:0/1173157396 >> v2:172.21.15.190:3300/0 conn(0x7feec813c510 0x7feec80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec6d9d700 1 -- 172.21.15.131:0/1173157396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec806b8c0 msgr2=0x7feec806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec6d9d700 1 --2- 172.21.15.131:0/1173157396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec806b8c0 0x7feec806bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec6d9d700 1 -- 172.21.15.131:0/1173157396 >> v2:172.21.15.190:3300/0 conn(0x7feec813c510 msgr2=0x7feec80c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec6d9d700 1 --2- 172.21.15.131:0/1173157396 >> v2:172.21.15.190:3300/0 conn(0x7feec813c510 0x7feec80c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec6d9d700 1 -- 172.21.15.131:0/1173157396 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8068910 con 0x7feec80cc100 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.840+0000 7feec6d9d700 1 --2- 172.21.15.131:0/1173157396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec80cc100 0x7feec80cc560 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feec806fcc0 tx=0x7feec0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59aec1edd9c23b63 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feec5d9b700 1 -- 172.21.15.131:0/1173157396 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec000e070 con 0x7feec80cc100 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feec5d9b700 1 -- 172.21.15.131:0/1173157396 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec000b970 con 0x7feec80cc100 2023-12-26T21:36:43.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feec5d9b700 1 -- 172.21.15.131:0/1173157396 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec00092e0 con 0x7feec80cc100 2023-12-26T21:36:43.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feecc9ff700 1 -- 172.21.15.131:0/1173157396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec80cc100 msgr2=0x7feec80cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feecc9ff700 1 --2- 172.21.15.131:0/1173157396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec80cc100 0x7feec80cc560 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feec806fcc0 tx=0x7feec0009840 comp rx=0 tx=0).stop 2023-12-26T21:36:43.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feecc9ff700 1 -- 172.21.15.131:0/1173157396 shutdown_connections 2023-12-26T21:36:43.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feecc9ff700 1 --2- 172.21.15.131:0/1173157396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec80cc100 0x7feec80cc560 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feecc9ff700 1 --2- 172.21.15.131:0/1173157396 >> v2:172.21.15.190:3300/0 conn(0x7feec813c510 0x7feec80c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feecc9ff700 1 --2- 172.21.15.131:0/1173157396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec806b8c0 0x7feec806bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feecc9ff700 1 -- 172.21.15.131:0/1173157396 >> 172.21.15.131:0/1173157396 conn(0x7feec80ce3e0 msgr2=0x7feec80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:43.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feecc9ff700 1 -- 172.21.15.131:0/1173157396 shutdown_connections 2023-12-26T21:36:43.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.841+0000 7feecc9ff700 1 -- 172.21.15.131:0/1173157396 wait complete. 2023-12-26T21:36:43.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.843+0000 7feecc9ff700 1 Processor -- start 2023-12-26T21:36:43.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.844+0000 7feecc9ff700 1 -- start start 2023-12-26T21:36:43.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feecc9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec806b8c0 0x7feec8148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feecc9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec80cc100 0x7feec815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feecc9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec813c510 0x7feec815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feecc9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec815caa0 con 0x7feec813c510 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feecc9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec80c56c0 con 0x7feec806b8c0 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feecc9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec815c200 con 0x7feec80cc100 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feec6d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec813c510 0x7feec815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feebffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec80cc100 0x7feec815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feec6d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec813c510 0x7feec815e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44192/0 (socket says 172.21.15.131:44192) 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feec6d9d700 1 -- 172.21.15.131:0/261282375 learned_addr learned my addr 172.21.15.131:0/261282375 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feec659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec806b8c0 0x7feec8148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feec6d9d700 1 -- 172.21.15.131:0/261282375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec80cc100 msgr2=0x7feec815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feec6d9d700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec80cc100 0x7feec815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feec6d9d700 1 -- 172.21.15.131:0/261282375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec806b8c0 msgr2=0x7feec8148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:43.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feec6d9d700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec806b8c0 0x7feec8148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.845+0000 7feec6d9d700 1 -- 172.21.15.131:0/261282375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec0009ae0 con 0x7feec813c510 2023-12-26T21:36:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.846+0000 7feec6d9d700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec813c510 0x7feec815e000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feec806fe40 tx=0x7feec0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.846+0000 7feebe7fc700 1 -- 172.21.15.131:0/261282375 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0008ad0 con 0x7feec813c510 2023-12-26T21:36:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.846+0000 7feebe7fc700 1 -- 172.21.15.131:0/261282375 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0008c50 con 0x7feec813c510 2023-12-26T21:36:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.846+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec815c4e0 con 0x7feec813c510 2023-12-26T21:36:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.846+0000 7feebe7fc700 1 -- 172.21.15.131:0/261282375 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0026460 con 0x7feec813c510 2023-12-26T21:36:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.846+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec8148680 con 0x7feec813c510 2023-12-26T21:36:43.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.847+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec80c5490 con 0x7feec813c510 2023-12-26T21:36:43.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.849+0000 7feebe7fc700 1 -- 172.21.15.131:0/261282375 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feec0003040 con 0x7feec813c510 2023-12-26T21:36:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.849+0000 7feebe7fc700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feea8066f30 0x7feea80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.850+0000 7feebe7fc700 1 -- 172.21.15.131:0/261282375 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7feec00862b0 con 0x7feec813c510 2023-12-26T21:36:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.850+0000 7feec659c700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feea8066f30 0x7feea80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:43.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.850+0000 7feec659c700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feea8066f30 0x7feea80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009da0 tx=0x7feeb8009120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:43.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:43.853+0000 7feebe7fc700 1 -- 172.21.15.131:0/261282375 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feec0053850 con 0x7feec813c510 2023-12-26T21:36:44.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.048+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feec80c5490 con 0x7feec813c510 2023-12-26T21:36:44.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.049+0000 7feebe7fc700 1 -- 172.21.15.131:0/261282375 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12175 (secure 0 0 0) 0x7feec80c5490 con 0x7feec813c510 2023-12-26T21:36:44.049 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:44.050 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:39.577205+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:44.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feea8066f30 msgr2=0x7feea80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feea8066f30 0x7feea80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009da0 tx=0x7feeb8009120 comp rx=0 tx=0).stop 2023-12-26T21:36:44.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec813c510 msgr2=0x7feec815e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec813c510 0x7feec815e000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feec806fe40 tx=0x7feec0018000 comp rx=0 tx=0).stop 2023-12-26T21:36:44.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 shutdown_connections 2023-12-26T21:36:44.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec813c510 0x7feec815e000 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec806b8c0 0x7feec8148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feea8066f30 0x7feea80693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 --2- 172.21.15.131:0/261282375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feec80cc100 0x7feec815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 >> 172.21.15.131:0/261282375 conn(0x7feec80ce3e0 msgr2=0x7feec80cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 shutdown_connections 2023-12-26T21:36:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.051+0000 7feecc9ff700 1 -- 172.21.15.131:0/261282375 wait complete. 2023-12-26T21:36:44.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T21:36:44.070 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:36:44.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.263+0000 7fe2ec1ff700 1 Processor -- start 2023-12-26T21:36:44.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.263+0000 7fe2ec1ff700 1 -- start start 2023-12-26T21:36:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.264+0000 7fe2ec1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2e4159cf0 0x7fe2e40c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.264+0000 7fe2ec1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e40ced90 0x7fe2e40cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.264+0000 7fe2ec1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e40cf730 con 0x7fe2e4159920 2023-12-26T21:36:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.264+0000 7fe2ec1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e40c6810 con 0x7fe2e4159cf0 2023-12-26T21:36:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.264+0000 7fe2ec1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40c5c00 con 0x7fe2e40ced90 2023-12-26T21:36:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2ea79c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e40ced90 0x7fe2e40cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2ea79c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e40ced90 0x7fe2e40cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:35302/0 (socket says 172.21.15.131:35302) 2023-12-26T21:36:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2e9f9b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe2e4159920 0x7fe2e40c5420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:40396/0 (socket says 172.21.15.131:40396) 2023-12-26T21:36:44.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2e979a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2e4159cf0 0x7fe2e40c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2ea79c700 1 -- 172.21.15.131:0/2532263098 learned_addr learned my addr 172.21.15.131:0/2532263098 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:44.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2ea79c700 1 -- 172.21.15.131:0/2532263098 >> v2:172.21.15.190:3300/0 conn(0x7fe2e4159cf0 msgr2=0x7fe2e40c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:44.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2ea79c700 1 --2- 172.21.15.131:0/2532263098 >> v2:172.21.15.190:3300/0 conn(0x7fe2e4159cf0 0x7fe2e40c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2e8f99700 1 -- 172.21.15.131:0/2532263098 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12554415 0 0) 0x7fe2e0001160 con 0x7fe2e4159920 2023-12-26T21:36:44.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2ea79c700 1 -- 172.21.15.131:0/2532263098 >> v1:172.21.15.190:6789/0 conn(0x7fe2e4159920 legacy=0x7fe2e40c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2ea79c700 1 -- 172.21.15.131:0/2532263098 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e40c5ee0 con 0x7fe2e40ced90 2023-12-26T21:36:44.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2ea79c700 1 --2- 172.21.15.131:0/2532263098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e40ced90 0x7fe2e40cf1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009f20 tx=0x7fe2dc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a67ede4b0efc40 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:44.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.265+0000 7fe2e8f99700 1 -- 172.21.15.131:0/2532263098 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00e070 con 0x7fe2e40ced90 2023-12-26T21:36:44.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2e8f99700 1 -- 172.21.15.131:0/2532263098 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc00b9d0 con 0x7fe2e40ced90 2023-12-26T21:36:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2e8f99700 1 -- 172.21.15.131:0/2532263098 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0092e0 con 0x7fe2e40ced90 2023-12-26T21:36:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/2532263098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e40ced90 msgr2=0x7fe2e40cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2ec1ff700 1 --2- 172.21.15.131:0/2532263098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e40ced90 0x7fe2e40cf1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009f20 tx=0x7fe2dc009840 comp rx=0 tx=0).stop 2023-12-26T21:36:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/2532263098 shutdown_connections 2023-12-26T21:36:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2ec1ff700 1 --2- 172.21.15.131:0/2532263098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e40ced90 0x7fe2e40cf1f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2ec1ff700 1 --2- 172.21.15.131:0/2532263098 >> v2:172.21.15.190:3300/0 conn(0x7fe2e4159cf0 0x7fe2e40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/2532263098 >> 172.21.15.131:0/2532263098 conn(0x7fe2e4099980 msgr2=0x7fe2e406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/2532263098 shutdown_connections 2023-12-26T21:36:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.266+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/2532263098 wait complete. 2023-12-26T21:36:44.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.268+0000 7fe2ec1ff700 1 Processor -- start 2023-12-26T21:36:44.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.268+0000 7fe2ec1ff700 1 -- start start 2023-12-26T21:36:44.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2ec1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e40ced90 0x7fe2e4154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2ec1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e4159920 0x7fe2e40b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2ec1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e4159cf0 0x7fe2e40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2ec1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e416cb60 con 0x7fe2e40ced90 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2ec1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e416d6a0 con 0x7fe2e4159920 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2ec1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e416cd70 con 0x7fe2e4159cf0 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e9f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e40ced90 0x7fe2e4154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e9f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e40ced90 0x7fe2e4154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44204/0 (socket says 172.21.15.131:44204) 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e9f9b700 1 -- 172.21.15.131:0/1150347629 learned_addr learned my addr 172.21.15.131:0/1150347629 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2ea79c700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e4159cf0 0x7fe2e40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e979a700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e4159920 0x7fe2e40b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e9f9b700 1 -- 172.21.15.131:0/1150347629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e4159cf0 msgr2=0x7fe2e40b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e9f9b700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e4159cf0 0x7fe2e40b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e9f9b700 1 -- 172.21.15.131:0/1150347629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e4159920 msgr2=0x7fe2e40b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e9f9b700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e4159920 0x7fe2e40b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e9f9b700 1 -- 172.21.15.131:0/1150347629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc009ae0 con 0x7fe2e40ced90 2023-12-26T21:36:44.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.270+0000 7fe2e9f9b700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e40ced90 0x7fe2e4154240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c560 tx=0x7fe2e0001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:44.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.271+0000 7fe2d9ffb700 1 -- 172.21.15.131:0/1150347629 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0005ac0 con 0x7fe2e40ced90 2023-12-26T21:36:44.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.271+0000 7fe2d9ffb700 1 -- 172.21.15.131:0/1150347629 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e0005c20 con 0x7fe2e40ced90 2023-12-26T21:36:44.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.271+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e416d050 con 0x7fe2e40ced90 2023-12-26T21:36:44.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.271+0000 7fe2d9ffb700 1 -- 172.21.15.131:0/1150347629 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00070d0 con 0x7fe2e40ced90 2023-12-26T21:36:44.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.271+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e4154780 con 0x7fe2e40ced90 2023-12-26T21:36:44.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.272+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e40657e0 con 0x7fe2e40ced90 2023-12-26T21:36:44.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.273+0000 7fe2d9ffb700 1 -- 172.21.15.131:0/1150347629 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe2e0007480 con 0x7fe2e40ced90 2023-12-26T21:36:44.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.274+0000 7fe2d9ffb700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2c8066f30 0x7fe2c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.274+0000 7fe2d9ffb700 1 -- 172.21.15.131:0/1150347629 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe2e0083480 con 0x7fe2e40ced90 2023-12-26T21:36:44.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.274+0000 7fe2e979a700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2c8066f30 0x7fe2c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.274+0000 7fe2e979a700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2c8066f30 0x7fe2c80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe2e416cac0 tx=0x7fe2d40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:44.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.278+0000 7fe2d9ffb700 1 -- 172.21.15.131:0/1150347629 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2e00509a0 con 0x7fe2e40ced90 2023-12-26T21:36:44.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.469+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2e40cf730 con 0x7fe2e40ced90 2023-12-26T21:36:44.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.470+0000 7fe2d9ffb700 1 -- 172.21.15.131:0/1150347629 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2e40cf730 con 0x7fe2e40ced90 2023-12-26T21:36:44.471 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:36:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2c8066f30 msgr2=0x7fe2c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2c8066f30 0x7fe2c80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe2e416cac0 tx=0x7fe2d40091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e40ced90 msgr2=0x7fe2e4154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e40ced90 0x7fe2e4154240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c560 tx=0x7fe2e0001030 comp rx=0 tx=0).stop 2023-12-26T21:36:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 shutdown_connections 2023-12-26T21:36:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2c8066f30 0x7fe2c80693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e4159cf0 0x7fe2e40b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e40ced90 0x7fe2e4154240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 --2- 172.21.15.131:0/1150347629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e4159920 0x7fe2e40b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 >> 172.21.15.131:0/1150347629 conn(0x7fe2e4099980 msgr2=0x7fe2e40696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:44.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.472+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 shutdown_connections 2023-12-26T21:36:44.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.473+0000 7fe2ec1ff700 1 -- 172.21.15.131:0/1150347629 wait complete. 2023-12-26T21:36:44.491 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:36:44.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.679+0000 7f6293fff700 1 Processor -- start 2023-12-26T21:36:44.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.679+0000 7f6293fff700 1 -- start start 2023-12-26T21:36:44.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6293fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f628c13c510 0x7f628c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6293fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f628c06b8c0 0x7f628c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6293fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0686c0 con 0x7f628c0cc100 2023-12-26T21:36:44.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6293fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c06c280 con 0x7f628c13c510 2023-12-26T21:36:44.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6293fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c069520 con 0x7f628c06b8c0 2023-12-26T21:36:44.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6291d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f628c13c510 0x7f628c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6291d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f628c13c510 0x7f628c0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44214/0 (socket says 172.21.15.131:44214) 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6291d9b700 1 -- 172.21.15.131:0/76021938 learned_addr learned my addr 172.21.15.131:0/76021938 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f629159a700 1 --2- 172.21.15.131:0/76021938 >> v2:172.21.15.190:3300/0 conn(0x7f628c06b8c0 0x7f628c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6291d9b700 1 -- 172.21.15.131:0/76021938 >> v2:172.21.15.190:3300/0 conn(0x7f628c06b8c0 msgr2=0x7f628c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6291d9b700 1 --2- 172.21.15.131:0/76021938 >> v2:172.21.15.190:3300/0 conn(0x7f628c06b8c0 0x7f628c06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6291d9b700 1 -- 172.21.15.131:0/76021938 >> v1:172.21.15.190:6789/0 conn(0x7f628c0cc100 legacy=0x7f628c0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f6291d9b700 1 -- 172.21.15.131:0/76021938 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c068910 con 0x7f628c13c510 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.681+0000 7f629159a700 1 --2- 172.21.15.131:0/76021938 >> v2:172.21.15.190:3300/0 conn(0x7f628c06b8c0 0x7f628c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.682+0000 7f6291d9b700 1 --2- 172.21.15.131:0/76021938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c13c510 0x7f628c0c8a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f628c0b43e0 tx=0x7f6288009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad45a1c0b7e90211 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.682+0000 7f6290d99700 1 -- 172.21.15.131:0/76021938 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288005c10 con 0x7f628c13c510 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.682+0000 7f6290d99700 1 -- 172.21.15.131:0/76021938 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288005d70 con 0x7f628c13c510 2023-12-26T21:36:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.682+0000 7f6290d99700 1 -- 172.21.15.131:0/76021938 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800b880 con 0x7f628c13c510 2023-12-26T21:36:44.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.682+0000 7f6293fff700 1 -- 172.21.15.131:0/76021938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c13c510 msgr2=0x7f628c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.682+0000 7f6293fff700 1 --2- 172.21.15.131:0/76021938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c13c510 0x7f628c0c8a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f628c0b43e0 tx=0x7f6288009220 comp rx=0 tx=0).stop 2023-12-26T21:36:44.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.682+0000 7f6293fff700 1 -- 172.21.15.131:0/76021938 shutdown_connections 2023-12-26T21:36:44.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.682+0000 7f6293fff700 1 --2- 172.21.15.131:0/76021938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c13c510 0x7f628c0c8a20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.683+0000 7f6293fff700 1 --2- 172.21.15.131:0/76021938 >> v2:172.21.15.190:3300/0 conn(0x7f628c06b8c0 0x7f628c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.683+0000 7f6293fff700 1 -- 172.21.15.131:0/76021938 >> 172.21.15.131:0/76021938 conn(0x7f628c0ce3e0 msgr2=0x7f628c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:44.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.683+0000 7f6293fff700 1 -- 172.21.15.131:0/76021938 shutdown_connections 2023-12-26T21:36:44.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.683+0000 7f6293fff700 1 -- 172.21.15.131:0/76021938 wait complete. 2023-12-26T21:36:44.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.684+0000 7f6293fff700 1 Processor -- start 2023-12-26T21:36:44.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.684+0000 7f6293fff700 1 -- start start 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.685+0000 7f6293fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c06b8c0 0x7f628c153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.685+0000 7f6293fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f628c0cc100 0x7f628c154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.685+0000 7f6293fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c13c510 0x7f628c154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.685+0000 7f6293fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b1180 con 0x7f628c06b8c0 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.685+0000 7f6293fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b13d0 con 0x7f628c13c510 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.685+0000 7f6293fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b16b0 con 0x7f628c0cc100 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.685+0000 7f6291d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c06b8c0 0x7f628c153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.685+0000 7f6291d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c06b8c0 0x7f628c153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44230/0 (socket says 172.21.15.131:44230) 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.685+0000 7f6291d9b700 1 -- 172.21.15.131:0/2482347093 learned_addr learned my addr 172.21.15.131:0/2482347093 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f629159a700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f628c0cc100 0x7f628c154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f629259c700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c13c510 0x7f628c154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6291d9b700 1 -- 172.21.15.131:0/2482347093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f628c0cc100 msgr2=0x7f628c154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6291d9b700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f628c0cc100 0x7f628c154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6291d9b700 1 -- 172.21.15.131:0/2482347093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c13c510 msgr2=0x7f628c154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6291d9b700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c13c510 0x7f628c154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6291d9b700 1 -- 172.21.15.131:0/2482347093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288009480 con 0x7f628c06b8c0 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6291d9b700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c06b8c0 0x7f628c153fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f628c06fe40 tx=0x7f6288016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6281ffb700 1 -- 172.21.15.131:0/2482347093 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880118e0 con 0x7f628c06b8c0 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6281ffb700 1 -- 172.21.15.131:0/2482347093 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288020070 con 0x7f628c06b8c0 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0b1990 con 0x7f628c06b8c0 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6281ffb700 1 -- 172.21.15.131:0/2482347093 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288011cb0 con 0x7f628c06b8c0 2023-12-26T21:36:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.686+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0c7040 con 0x7f628c06b8c0 2023-12-26T21:36:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.687+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c0b1e30 con 0x7f628c06b8c0 2023-12-26T21:36:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.688+0000 7f6281ffb700 1 -- 172.21.15.131:0/2482347093 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6288022030 con 0x7f628c06b8c0 2023-12-26T21:36:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.689+0000 7f6281ffb700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6270066f30 0x7f62700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.689+0000 7f6281ffb700 1 -- 172.21.15.131:0/2482347093 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6288085f20 con 0x7f628c06b8c0 2023-12-26T21:36:44.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.689+0000 7f629159a700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6270066f30 0x7f62700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:44.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.690+0000 7f629159a700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6270066f30 0x7f62700693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f628c157630 tx=0x7f627c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:44.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.693+0000 7f6281ffb700 1 -- 172.21.15.131:0/2482347093 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6288053440 con 0x7f628c06b8c0 2023-12-26T21:36:44.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.936+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f628c0657e0 con 0x7f628c06b8c0 2023-12-26T21:36:44.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.936+0000 7f6281ffb700 1 -- 172.21.15.131:0/2482347093 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7f6288056a60 con 0x7f628c06b8c0 2023-12-26T21:36:44.937 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:44.937 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1127181197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1127181197},{"type":"v1","addr":"172.21.15.190:6835","nonce":1127181197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1425514628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1425514628},{"type":"v1","addr":"172.21.15.131:6837","nonce":1425514628}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3223324183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3223324183},{"type":"v1","addr":"172.21.15.190:6833","nonce":3223324183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/3852718531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3852718531},{"type":"v1","addr":"172.21.15.131:6836","nonce":3852718531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-26T21:36:44.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.938+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6270066f30 msgr2=0x7f62700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.938+0000 7f6293fff700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6270066f30 0x7f62700693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f628c157630 tx=0x7f627c008040 comp rx=0 tx=0).stop 2023-12-26T21:36:44.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.938+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c06b8c0 msgr2=0x7f628c153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.938+0000 7f6293fff700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c06b8c0 0x7f628c153fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f628c06fe40 tx=0x7f6288016000 comp rx=0 tx=0).stop 2023-12-26T21:36:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.938+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 shutdown_connections 2023-12-26T21:36:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.938+0000 7f6293fff700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f628c13c510 0x7f628c154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.938+0000 7f6293fff700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c06b8c0 0x7f628c153fd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.938+0000 7f6293fff700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6270066f30 0x7f62700693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.938+0000 7f6293fff700 1 --2- 172.21.15.131:0/2482347093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f628c0cc100 0x7f628c154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.939+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 >> 172.21.15.131:0/2482347093 conn(0x7f628c0ce3e0 msgr2=0x7f628c13b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.939+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 shutdown_connections 2023-12-26T21:36:44.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:44.939+0000 7f6293fff700 1 -- 172.21.15.131:0/2482347093 wait complete. 2023-12-26T21:36:44.943 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 3 2023-12-26T21:36:44.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:45.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.148+0000 7f592dfff700 1 Processor -- start 2023-12-26T21:36:45.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.148+0000 7f592dfff700 1 -- start start 2023-12-26T21:36:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f592dfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59280b5310 0x7f59280b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f592dfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59280afcd0 con 0x7f5928159a70 2023-12-26T21:36:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f592dfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59280b3e70 con 0x7f59281596a0 2023-12-26T21:36:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f592dfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59280b0f20 con 0x7f59280b5310 2023-12-26T21:36:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f5926ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5928159a70 0x7f59280b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47774/0 (socket says 172.21.15.131:47774) 2023-12-26T21:36:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f5926ffd700 1 -- 172.21.15.131:0/800557047 learned_addr learned my addr 172.21.15.131:0/800557047 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f5927fff700 1 --2- 172.21.15.131:0/800557047 >> v2:172.21.15.190:3300/0 conn(0x7f59280b5310 0x7f59280b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f59267fc700 1 -- 172.21.15.131:0/800557047 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870522945 0 0) 0x7f59280afcd0 con 0x7f5928159a70 2023-12-26T21:36:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f59267fc700 1 -- 172.21.15.131:0/800557047 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5910003570 con 0x7f5928159a70 2023-12-26T21:36:45.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f59267fc700 1 -- 172.21.15.131:0/800557047 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501766896 0 0) 0x7f59280b3e70 con 0x7f59281596a0 2023-12-26T21:36:45.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f59267fc700 1 -- 172.21.15.131:0/800557047 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59280afcd0 con 0x7f59281596a0 2023-12-26T21:36:45.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f59267fc700 1 -- 172.21.15.131:0/800557047 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996132167 0 0) 0x7f5910003570 con 0x7f5928159a70 2023-12-26T21:36:45.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.150+0000 7f5927fff700 1 -- 172.21.15.131:0/800557047 >> v1:172.21.15.190:6789/0 conn(0x7f59281596a0 legacy=0x7f59280b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.151+0000 7f5927fff700 1 -- 172.21.15.131:0/800557047 >> v1:172.21.15.131:6789/0 conn(0x7f5928159a70 legacy=0x7f59280b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.151+0000 7f5927fff700 1 -- 172.21.15.131:0/800557047 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59280b3540 con 0x7f59280b5310 2023-12-26T21:36:45.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.151+0000 7f5927fff700 1 --2- 172.21.15.131:0/800557047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59280b5310 0x7f59280b0870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5920009e60 tx=0x7f5920009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadab50b8de09864 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:45.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.151+0000 7f59267fc700 1 -- 172.21.15.131:0/800557047 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592000e070 con 0x7f59280b5310 2023-12-26T21:36:45.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.151+0000 7f59267fc700 1 -- 172.21.15.131:0/800557047 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592000b7a0 con 0x7f59280b5310 2023-12-26T21:36:45.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.151+0000 7f59267fc700 1 -- 172.21.15.131:0/800557047 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c001470 con 0x7f59280b5310 2023-12-26T21:36:45.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.152+0000 7f592dfff700 1 -- 172.21.15.131:0/800557047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59280b5310 msgr2=0x7f59280b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.152+0000 7f592dfff700 1 --2- 172.21.15.131:0/800557047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59280b5310 0x7f59280b0870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5920009e60 tx=0x7f5920009780 comp rx=0 tx=0).stop 2023-12-26T21:36:45.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.152+0000 7f592dfff700 1 -- 172.21.15.131:0/800557047 shutdown_connections 2023-12-26T21:36:45.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.152+0000 7f592dfff700 1 --2- 172.21.15.131:0/800557047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59280b5310 0x7f59280b0870 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.152+0000 7f592dfff700 1 -- 172.21.15.131:0/800557047 >> 172.21.15.131:0/800557047 conn(0x7f592806ece0 msgr2=0x7f592806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:45.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.152+0000 7f592dfff700 1 -- 172.21.15.131:0/800557047 shutdown_connections 2023-12-26T21:36:45.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.152+0000 7f592dfff700 1 -- 172.21.15.131:0/800557047 wait complete. 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.153+0000 7f592dfff700 1 Processor -- start 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.153+0000 7f592dfff700 1 -- start start 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.154+0000 7f592dfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280b5310 0x7f5928169f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.154+0000 7f592dfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59281596a0 0x7f59280c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.154+0000 7f592dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5928159a70 0x7f59280cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.154+0000 7f592dfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928159e60 con 0x7f59280b5310 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.154+0000 7f592dfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592815a6c0 con 0x7f59281596a0 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.154+0000 7f592dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59280c6e90 con 0x7f5928159a70 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f59277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280b5310 0x7f5928169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f59277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280b5310 0x7f5928169f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56878/0 (socket says 172.21.15.131:56878) 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f59277fe700 1 -- 172.21.15.131:0/3507896207 learned_addr learned my addr 172.21.15.131:0/3507896207 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f5926ffd700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59281596a0 0x7f59280c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f5927fff700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5928159a70 0x7f59280cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f59277fe700 1 -- 172.21.15.131:0/3507896207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5928159a70 msgr2=0x7f59280cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f59277fe700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5928159a70 0x7f59280cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f59277fe700 1 -- 172.21.15.131:0/3507896207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59281596a0 msgr2=0x7f59280c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f59277fe700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59281596a0 0x7f59280c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f59277fe700 1 -- 172.21.15.131:0/3507896207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5920009a20 con 0x7f59280b5310 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f5927fff700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5928159a70 0x7f59280cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f59277fe700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280b5310 0x7f5928169f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f591800c790 tx=0x7f5918007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f590f7fe700 1 -- 172.21.15.131:0/3507896207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918010510 con 0x7f59280b5310 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f590f7fe700 1 -- 172.21.15.131:0/3507896207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5918010670 con 0x7f59280b5310 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.155+0000 7f590f7fe700 1 -- 172.21.15.131:0/3507896207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918010a10 con 0x7f59280b5310 2023-12-26T21:36:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.156+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592815a070 con 0x7f59280b5310 2023-12-26T21:36:45.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.156+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59280c6b60 con 0x7f59280b5310 2023-12-26T21:36:45.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.157+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592815a9a0 con 0x7f59280b5310 2023-12-26T21:36:45.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.158+0000 7f590f7fe700 1 -- 172.21.15.131:0/3507896207 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f591801b400 con 0x7f59280b5310 2023-12-26T21:36:45.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.159+0000 7f590f7fe700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5914066f30 0x7f59140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.159+0000 7f590f7fe700 1 -- 172.21.15.131:0/3507896207 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f59180840d0 con 0x7f59280b5310 2023-12-26T21:36:45.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.159+0000 7f5926ffd700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5914066f30 0x7f59140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.160+0000 7f5926ffd700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5914066f30 0x7f59140693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f591c001ed0 tx=0x7f591c000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:45.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.163+0000 7f590f7fe700 1 -- 172.21.15.131:0/3507896207 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59180515f0 con 0x7f59280b5310 2023-12-26T21:36:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.357+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59280657e0 con 0x7f59280b5310 2023-12-26T21:36:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.358+0000 7f590f7fe700 1 -- 172.21.15.131:0/3507896207 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f5918054c10 con 0x7f59280b5310 2023-12-26T21:36:45.359 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:45.359 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:39.577205+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5914066f30 msgr2=0x7f59140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5914066f30 0x7f59140693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f591c001ed0 tx=0x7f591c000bc0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280b5310 msgr2=0x7f5928169f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280b5310 0x7f5928169f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f591800c790 tx=0x7f5918007e10 comp rx=0 tx=0).stop 2023-12-26T21:36:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 shutdown_connections 2023-12-26T21:36:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5914066f30 0x7f59140693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5928159a70 0x7f59280cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280b5310 0x7f5928169f50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 --2- 172.21.15.131:0/3507896207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59281596a0 0x7f59280c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 >> 172.21.15.131:0/3507896207 conn(0x7f592806ece0 msgr2=0x7f592815d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.360+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 shutdown_connections 2023-12-26T21:36:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.361+0000 7f592dfff700 1 -- 172.21.15.131:0/3507896207 wait complete. 2023-12-26T21:36:45.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T21:36:45.379 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:36:45.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.569+0000 7f9adf1ff700 1 Processor -- start 2023-12-26T21:36:45.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.569+0000 7f9adf1ff700 1 -- start start 2023-12-26T21:36:45.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.570+0000 7f9adf1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9ad81596a0 0x7f9ad80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.570+0000 7f9adf1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad80afcd0 con 0x7f9ad80b0870 2023-12-26T21:36:45.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.570+0000 7f9adf1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad80b3e70 con 0x7f9ad8159a70 2023-12-26T21:36:45.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.570+0000 7f9adf1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad80b0cb0 con 0x7f9ad81596a0 2023-12-26T21:36:45.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9add79c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9ad80b0870 0x7f9ad80b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47782/0 (socket says 172.21.15.131:47782) 2023-12-26T21:36:45.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9add79c700 1 -- 172.21.15.131:0/3370100131 learned_addr learned my addr 172.21.15.131:0/3370100131 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9adcf9b700 1 --2- 172.21.15.131:0/3370100131 >> v2:172.21.15.190:3300/0 conn(0x7f9ad81596a0 0x7f9ad80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9ad77fe700 1 -- 172.21.15.131:0/3370100131 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882197460 0 0) 0x7f9ad80afcd0 con 0x7f9ad80b0870 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9ad77fe700 1 -- 172.21.15.131:0/3370100131 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ac8003570 con 0x7f9ad80b0870 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9ad77fe700 1 -- 172.21.15.131:0/3370100131 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948858139 0 0) 0x7f9ad80b3e70 con 0x7f9ad8159a70 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9ad77fe700 1 -- 172.21.15.131:0/3370100131 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ad80afcd0 con 0x7f9ad8159a70 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9adcf9b700 1 -- 172.21.15.131:0/3370100131 >> v1:172.21.15.190:6789/0 conn(0x7f9ad8159a70 legacy=0x7f9ad80b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9adcf9b700 1 -- 172.21.15.131:0/3370100131 >> v1:172.21.15.131:6789/0 conn(0x7f9ad80b0870 legacy=0x7f9ad80b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9adcf9b700 1 -- 172.21.15.131:0/3370100131 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80b0f20 con 0x7f9ad81596a0 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.571+0000 7f9adcf9b700 1 --2- 172.21.15.131:0/3370100131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ad81596a0 0x7f9ad80b2380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0098e0 tx=0x7f9acc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ade1c5fccb9671af server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9ad77fe700 1 -- 172.21.15.131:0/3370100131 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc011630 con 0x7f9ad81596a0 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9ad77fe700 1 -- 172.21.15.131:0/3370100131 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc011790 con 0x7f9ad81596a0 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9ad77fe700 1 -- 172.21.15.131:0/3370100131 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc011b30 con 0x7f9ad81596a0 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9adf1ff700 1 -- 172.21.15.131:0/3370100131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ad81596a0 msgr2=0x7f9ad80b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9adf1ff700 1 --2- 172.21.15.131:0/3370100131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ad81596a0 0x7f9ad80b2380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0098e0 tx=0x7f9acc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9adf1ff700 1 -- 172.21.15.131:0/3370100131 shutdown_connections 2023-12-26T21:36:45.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9adf1ff700 1 --2- 172.21.15.131:0/3370100131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ad81596a0 0x7f9ad80b2380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9adf1ff700 1 -- 172.21.15.131:0/3370100131 >> 172.21.15.131:0/3370100131 conn(0x7f9ad806ece0 msgr2=0x7f9ad806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:45.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9adf1ff700 1 -- 172.21.15.131:0/3370100131 shutdown_connections 2023-12-26T21:36:45.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.572+0000 7f9adf1ff700 1 -- 172.21.15.131:0/3370100131 wait complete. 2023-12-26T21:36:45.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.574+0000 7f9adf1ff700 1 Processor -- start 2023-12-26T21:36:45.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.574+0000 7f9adf1ff700 1 -- start start 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9adf1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ad80b0870 0x7f9ad80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9adf1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ad81596a0 0x7f9ad815a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9adf1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8159a70 0x7f9ad80ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9adf1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80cca20 con 0x7f9ad8159a70 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9adf1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad815ad80 con 0x7f9ad81596a0 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9adf1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80cbe10 con 0x7f9ad80b0870 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9add79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8159a70 0x7f9ad80ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9add79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8159a70 0x7f9ad80ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56880/0 (socket says 172.21.15.131:56880) 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9add79c700 1 -- 172.21.15.131:0/2103887281 learned_addr learned my addr 172.21.15.131:0/2103887281 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9ad7fff700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ad81596a0 0x7f9ad815a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.575+0000 7f9adcf9b700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ad80b0870 0x7f9ad80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9add79c700 1 -- 172.21.15.131:0/2103887281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ad80b0870 msgr2=0x7f9ad80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9add79c700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ad80b0870 0x7f9ad80cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9add79c700 1 -- 172.21.15.131:0/2103887281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ad81596a0 msgr2=0x7f9ad815a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9add79c700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ad81596a0 0x7f9ad815a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9add79c700 1 -- 172.21.15.131:0/2103887281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc009420 con 0x7f9ad8159a70 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9add79c700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8159a70 0x7f9ad80ca1c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000cc60 tx=0x7f9ad0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9ad4ff9700 1 -- 172.21.15.131:0/2103887281 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000ce70 con 0x7f9ad8159a70 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9ad4ff9700 1 -- 172.21.15.131:0/2103887281 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad000ab70 con 0x7f9ad8159a70 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9ad4ff9700 1 -- 172.21.15.131:0/2103887281 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00070f0 con 0x7f9ad8159a70 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad80cc0f0 con 0x7f9ad8159a70 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.576+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad80cc6b0 con 0x7f9ad8159a70 2023-12-26T21:36:45.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.577+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad80657e0 con 0x7f9ad8159a70 2023-12-26T21:36:45.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.578+0000 7f9ad4ff9700 1 -- 172.21.15.131:0/2103887281 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9ad00074a0 con 0x7f9ad8159a70 2023-12-26T21:36:45.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.579+0000 7f9ad4ff9700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9abc066f30 0x7f9abc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.579+0000 7f9ad4ff9700 1 -- 172.21.15.131:0/2103887281 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9ad0084770 con 0x7f9ad8159a70 2023-12-26T21:36:45.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.579+0000 7f9adcf9b700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9abc066f30 0x7f9abc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.580+0000 7f9adcf9b700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9abc066f30 0x7f9abc0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0093f0 tx=0x7f9acc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:45.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.583+0000 7f9ad4ff9700 1 -- 172.21.15.131:0/2103887281 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ad0051c90 con 0x7f9ad8159a70 2023-12-26T21:36:45.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.773+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ad80b3540 con 0x7f9ad8159a70 2023-12-26T21:36:45.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.774+0000 7f9ad4ff9700 1 -- 172.21.15.131:0/2103887281 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ad00552b0 con 0x7f9ad8159a70 2023-12-26T21:36:45.774 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:36:45.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9abc066f30 msgr2=0x7f9abc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9abc066f30 0x7f9abc0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0093f0 tx=0x7f9acc017000 comp rx=0 tx=0).stop 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8159a70 msgr2=0x7f9ad80ca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8159a70 0x7f9ad80ca1c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000cc60 tx=0x7f9ad0001ae0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 shutdown_connections 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8159a70 0x7f9ad80ca1c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9abc066f30 0x7f9abc0693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ad80b0870 0x7f9ad80cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 --2- 172.21.15.131:0/2103887281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ad81596a0 0x7f9ad815a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 >> 172.21.15.131:0/2103887281 conn(0x7f9ad806ece0 msgr2=0x7f9ad80cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 shutdown_connections 2023-12-26T21:36:45.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.776+0000 7f9adf1ff700 1 -- 172.21.15.131:0/2103887281 wait complete. 2023-12-26T21:36:45.795 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T21:36:45.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.983+0000 7fd1cedff700 1 Processor -- start 2023-12-26T21:36:45.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.983+0000 7fd1cedff700 1 -- start start 2023-12-26T21:36:45.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.984+0000 7fd1cedff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 0x7fd1c80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.984+0000 7fd1cedff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1c8159a70 0x7fd1c80b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.984+0000 7fd1cedff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c80afcd0 con 0x7fd1c80b0160 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.984+0000 7fd1cedff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c80b3e70 con 0x7fd1c8159a70 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.984+0000 7fd1cedff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c80b0cd0 con 0x7fd1c81596a0 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1cd39c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd1c80b0160 0x7fd1c80b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47790/0 (socket says 172.21.15.131:47790) 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1cd39c700 1 -- 172.21.15.131:0/2380611663 learned_addr learned my addr 172.21.15.131:0/2380611663 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1ccb9b700 1 --2- 172.21.15.131:0/2380611663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 0x7fd1c80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1c7fff700 1 --2- 172.21.15.131:0/2380611663 >> v2:172.21.15.190:3300/0 conn(0x7fd1c8159a70 0x7fd1c80b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1c77fe700 1 -- 172.21.15.131:0/2380611663 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202266189 0 0) 0x7fd1c80afcd0 con 0x7fd1c80b0160 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1c77fe700 1 -- 172.21.15.131:0/2380611663 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b8003570 con 0x7fd1c80b0160 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1ccb9b700 1 -- 172.21.15.131:0/2380611663 >> v2:172.21.15.190:3300/0 conn(0x7fd1c8159a70 msgr2=0x7fd1c80b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1ccb9b700 1 --2- 172.21.15.131:0/2380611663 >> v2:172.21.15.190:3300/0 conn(0x7fd1c8159a70 0x7fd1c80b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1ccb9b700 1 -- 172.21.15.131:0/2380611663 >> v1:172.21.15.131:6789/0 conn(0x7fd1c80b0160 legacy=0x7fd1c80b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1ccb9b700 1 -- 172.21.15.131:0/2380611663 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80b2e00 con 0x7fd1c81596a0 2023-12-26T21:36:45.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.985+0000 7fd1ccb9b700 1 --2- 172.21.15.131:0/2380611663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 0x7fd1c80b2380 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0098e0 tx=0x7fd1bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15f6c745a185e6e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:45.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1c77fe700 1 -- 172.21.15.131:0/2380611663 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc011630 con 0x7fd1c81596a0 2023-12-26T21:36:45.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1c77fe700 1 -- 172.21.15.131:0/2380611663 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc011790 con 0x7fd1c81596a0 2023-12-26T21:36:45.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1c77fe700 1 -- 172.21.15.131:0/2380611663 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc011b10 con 0x7fd1c81596a0 2023-12-26T21:36:45.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1cedff700 1 -- 172.21.15.131:0/2380611663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 msgr2=0x7fd1c80b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1cedff700 1 --2- 172.21.15.131:0/2380611663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 0x7fd1c80b2380 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0098e0 tx=0x7fd1bc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1cedff700 1 -- 172.21.15.131:0/2380611663 shutdown_connections 2023-12-26T21:36:45.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1cedff700 1 --2- 172.21.15.131:0/2380611663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 0x7fd1c80b2380 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1cedff700 1 --2- 172.21.15.131:0/2380611663 >> v2:172.21.15.190:3300/0 conn(0x7fd1c8159a70 0x7fd1c80b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1cedff700 1 -- 172.21.15.131:0/2380611663 >> 172.21.15.131:0/2380611663 conn(0x7fd1c806ece0 msgr2=0x7fd1c806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1cedff700 1 -- 172.21.15.131:0/2380611663 shutdown_connections 2023-12-26T21:36:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.986+0000 7fd1cedff700 1 -- 172.21.15.131:0/2380611663 wait complete. 2023-12-26T21:36:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.988+0000 7fd1cedff700 1 Processor -- start 2023-12-26T21:36:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.988+0000 7fd1cedff700 1 -- start start 2023-12-26T21:36:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1cedff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c80b0160 0x7fd1c8150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1cedff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 0x7fd1c81572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1cedff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c8159a70 0x7fd1c80d0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1cedff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c816a7a0 con 0x7fd1c80b0160 2023-12-26T21:36:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1cedff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c81577e0 con 0x7fd1c8159a70 2023-12-26T21:36:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1cedff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c81698e0 con 0x7fd1c81596a0 2023-12-26T21:36:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1ccb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c80b0160 0x7fd1c8150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1ccb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c80b0160 0x7fd1c8150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56894/0 (socket says 172.21.15.131:56894) 2023-12-26T21:36:45.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1ccb9b700 1 -- 172.21.15.131:0/2857678655 learned_addr learned my addr 172.21.15.131:0/2857678655 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:45.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1c7fff700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 0x7fd1c81572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1cd39c700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c8159a70 0x7fd1c80d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1ccb9b700 1 -- 172.21.15.131:0/2857678655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 msgr2=0x7fd1c81572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1ccb9b700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 0x7fd1c81572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1ccb9b700 1 -- 172.21.15.131:0/2857678655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c8159a70 msgr2=0x7fd1c80d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1ccb9b700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c8159a70 0x7fd1c80d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1ccb9b700 1 -- 172.21.15.131:0/2857678655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc009420 con 0x7fd1c80b0160 2023-12-26T21:36:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.989+0000 7fd1cd39c700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c8159a70 0x7fd1c80d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.990+0000 7fd1ccb9b700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c80b0160 0x7fd1c8150760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd1c815e440 tx=0x7fd1bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.990+0000 7fd1c4ff9700 1 -- 172.21.15.131:0/2857678655 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc011b10 con 0x7fd1c80b0160 2023-12-26T21:36:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.990+0000 7fd1c4ff9700 1 -- 172.21.15.131:0/2857678655 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc000c90 con 0x7fd1c80b0160 2023-12-26T21:36:45.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.990+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c815ace0 con 0x7fd1c80b0160 2023-12-26T21:36:45.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.990+0000 7fd1c4ff9700 1 -- 172.21.15.131:0/2857678655 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc01a660 con 0x7fd1c80b0160 2023-12-26T21:36:45.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.990+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c8169e70 con 0x7fd1c80b0160 2023-12-26T21:36:45.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.991+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c80657e0 con 0x7fd1c80b0160 2023-12-26T21:36:45.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.992+0000 7fd1c4ff9700 1 -- 172.21.15.131:0/2857678655 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd1bc01aa50 con 0x7fd1c80b0160 2023-12-26T21:36:45.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.993+0000 7fd1c4ff9700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd1ac066f30 0x7fd1ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:45.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.993+0000 7fd1c4ff9700 1 -- 172.21.15.131:0/2857678655 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd1bc085e80 con 0x7fd1c80b0160 2023-12-26T21:36:45.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.993+0000 7fd1c7fff700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd1ac066f30 0x7fd1ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:45.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.994+0000 7fd1c7fff700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd1ac066f30 0x7fd1ac0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8154630 tx=0x7fd1b40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:45.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:45.997+0000 7fd1c4ff9700 1 -- 172.21.15.131:0/2857678655 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1bc053440 con 0x7fd1c80b0160 2023-12-26T21:36:46.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.194+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd1c80b2820 con 0x7fd1c80b0160 2023-12-26T21:36:46.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.195+0000 7fd1c4ff9700 1 -- 172.21.15.131:0/2857678655 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd1bc015090 con 0x7fd1c80b0160 2023-12-26T21:36:46.195 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:46.195 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:36:46.195 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:36:46.196 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T21:36:46.196 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T21:36:46.196 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:36:46.196 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd1ac066f30 msgr2=0x7fd1ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd1ac066f30 0x7fd1ac0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8154630 tx=0x7fd1b40091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c80b0160 msgr2=0x7fd1c8150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c80b0160 0x7fd1c8150760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd1c815e440 tx=0x7fd1bc017000 comp rx=0 tx=0).stop 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 shutdown_connections 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c8159a70 0x7fd1c80d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c80b0160 0x7fd1c8150760 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd1ac066f30 0x7fd1ac0693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 --2- 172.21.15.131:0/2857678655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c81596a0 0x7fd1c81572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 >> 172.21.15.131:0/2857678655 conn(0x7fd1c806ece0 msgr2=0x7fd1c80cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 shutdown_connections 2023-12-26T21:36:46.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.197+0000 7fd1cedff700 1 -- 172.21.15.131:0/2857678655 wait complete. 2023-12-26T21:36:46.215 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T21:36:46.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T21:36:46.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.407+0000 7fc6571ff700 1 Processor -- start 2023-12-26T21:36:46.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.407+0000 7fc6571ff700 1 -- start start 2023-12-26T21:36:46.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.408+0000 7fc6571ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc650159810 0x7fc6500c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.408+0000 7fc6571ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6500cd7b0 0x7fc6500cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.408+0000 7fc6571ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6500cbb00 con 0x7fc6500cec80 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.408+0000 7fc6571ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6500cb2a0 con 0x7fc650159810 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.408+0000 7fc6571ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6500cf640 con 0x7fc6500cd7b0 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.408+0000 7fc654f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc650159810 0x7fc6500c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.408+0000 7fc654f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc650159810 0x7fc6500c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56900/0 (socket says 172.21.15.131:56900) 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.408+0000 7fc654f9b700 1 -- 172.21.15.131:0/251196495 learned_addr learned my addr 172.21.15.131:0/251196495 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.409+0000 7fc65579c700 1 --2- 172.21.15.131:0/251196495 >> v2:172.21.15.190:3300/0 conn(0x7fc6500cd7b0 0x7fc6500cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.409+0000 7fc654f9b700 1 -- 172.21.15.131:0/251196495 >> v2:172.21.15.190:3300/0 conn(0x7fc6500cd7b0 msgr2=0x7fc6500cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.409+0000 7fc654f9b700 1 --2- 172.21.15.131:0/251196495 >> v2:172.21.15.190:3300/0 conn(0x7fc6500cd7b0 0x7fc6500cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.409+0000 7fc654f9b700 1 -- 172.21.15.131:0/251196495 >> v1:172.21.15.190:6789/0 conn(0x7fc6500cec80 legacy=0x7fc6500cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.409+0000 7fc654f9b700 1 -- 172.21.15.131:0/251196495 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500ce040 con 0x7fc650159810 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.409+0000 7fc654f9b700 1 --2- 172.21.15.131:0/251196495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc650159810 0x7fc6500c5440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6500cec10 tx=0x7fc644009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b92a70d4412f45 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:46.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.409+0000 7fc64f7fe700 1 -- 172.21.15.131:0/251196495 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644005c10 con 0x7fc650159810 2023-12-26T21:36:46.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.409+0000 7fc64f7fe700 1 -- 172.21.15.131:0/251196495 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644005d70 con 0x7fc650159810 2023-12-26T21:36:46.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.409+0000 7fc64f7fe700 1 -- 172.21.15.131:0/251196495 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64400b880 con 0x7fc650159810 2023-12-26T21:36:46.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.410+0000 7fc6571ff700 1 -- 172.21.15.131:0/251196495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc650159810 msgr2=0x7fc6500c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:46.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.410+0000 7fc6571ff700 1 --2- 172.21.15.131:0/251196495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc650159810 0x7fc6500c5440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6500cec10 tx=0x7fc644009220 comp rx=0 tx=0).stop 2023-12-26T21:36:46.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.410+0000 7fc6571ff700 1 -- 172.21.15.131:0/251196495 shutdown_connections 2023-12-26T21:36:46.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.410+0000 7fc6571ff700 1 --2- 172.21.15.131:0/251196495 >> v2:172.21.15.190:3300/0 conn(0x7fc6500cd7b0 0x7fc6500cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.410+0000 7fc6571ff700 1 --2- 172.21.15.131:0/251196495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc650159810 0x7fc6500c5440 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.410+0000 7fc6571ff700 1 -- 172.21.15.131:0/251196495 >> 172.21.15.131:0/251196495 conn(0x7fc650099950 msgr2=0x7fc65006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:46.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.410+0000 7fc6571ff700 1 -- 172.21.15.131:0/251196495 shutdown_connections 2023-12-26T21:36:46.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.410+0000 7fc6571ff700 1 -- 172.21.15.131:0/251196495 wait complete. 2023-12-26T21:36:46.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.411+0000 7fc6571ff700 1 Processor -- start 2023-12-26T21:36:46.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.412+0000 7fc6571ff700 1 -- start start 2023-12-26T21:36:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.412+0000 7fc6571ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cd7b0 0x7fc6500b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.412+0000 7fc6571ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6500cec80 0x7fc6501471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.412+0000 7fc6571ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc650159810 0x7fc6500686f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.412+0000 7fc6571ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc650169b30 con 0x7fc6500cd7b0 2023-12-26T21:36:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.412+0000 7fc6571ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6501476f0 con 0x7fc6500cec80 2023-12-26T21:36:46.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.412+0000 7fc6571ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500b4cf0 con 0x7fc650159810 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc654f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cd7b0 0x7fc6500b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc654f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cd7b0 0x7fc6500b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56910/0 (socket says 172.21.15.131:56910) 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc654f9b700 1 -- 172.21.15.131:0/4209633905 learned_addr learned my addr 172.21.15.131:0/4209633905 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc64ffff700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6500cec80 0x7fc6501471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc65579c700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc650159810 0x7fc6500686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc654f9b700 1 -- 172.21.15.131:0/4209633905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc650159810 msgr2=0x7fc6500686f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc654f9b700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc650159810 0x7fc6500686f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc654f9b700 1 -- 172.21.15.131:0/4209633905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6500cec80 msgr2=0x7fc6501471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc654f9b700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6500cec80 0x7fc6501471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc654f9b700 1 -- 172.21.15.131:0/4209633905 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644009480 con 0x7fc6500cd7b0 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc654f9b700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cd7b0 0x7fc6500b45f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc650088b20 tx=0x7fc644016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:46.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.413+0000 7fc64cff9700 1 -- 172.21.15.131:0/4209633905 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644011740 con 0x7fc6500cd7b0 2023-12-26T21:36:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.414+0000 7fc64cff9700 1 -- 172.21.15.131:0/4209633905 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644020070 con 0x7fc6500cd7b0 2023-12-26T21:36:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.414+0000 7fc64cff9700 1 -- 172.21.15.131:0/4209633905 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644011af0 con 0x7fc6500cd7b0 2023-12-26T21:36:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.414+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65016a640 con 0x7fc6500cd7b0 2023-12-26T21:36:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.414+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65016aad0 con 0x7fc6500cd7b0 2023-12-26T21:36:46.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.415+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6500717f0 con 0x7fc6500cd7b0 2023-12-26T21:36:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.415+0000 7fc64cff9700 1 -- 172.21.15.131:0/4209633905 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc644022030 con 0x7fc6500cd7b0 2023-12-26T21:36:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.416+0000 7fc64cff9700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc634066f30 0x7fc6340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.416+0000 7fc64cff9700 1 -- 172.21.15.131:0/4209633905 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc644085cc0 con 0x7fc6500cd7b0 2023-12-26T21:36:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.416+0000 7fc64ffff700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc634066f30 0x7fc6340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:46.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.417+0000 7fc64ffff700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc634066f30 0x7fc6340693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc6500c7c80 tx=0x7fc63c007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:46.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.420+0000 7fc64cff9700 1 -- 172.21.15.131:0/4209633905 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc644053280 con 0x7fc6500cd7b0 2023-12-26T21:36:46.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.617+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc650159db0 con 0x7fc6500cd7b0 2023-12-26T21:36:46.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.789+0000 7fc64cff9700 1 -- 172.21.15.131:0/4209633905 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc6440568a0 con 0x7fc6500cd7b0 2023-12-26T21:36:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.792+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc634066f30 msgr2=0x7fc6340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.792+0000 7fc6571ff700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc634066f30 0x7fc6340693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc6500c7c80 tx=0x7fc63c007ec0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.792+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cd7b0 msgr2=0x7fc6500b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:46.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.792+0000 7fc6571ff700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cd7b0 0x7fc6500b45f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc650088b20 tx=0x7fc644016000 comp rx=0 tx=0).stop 2023-12-26T21:36:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.794+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 shutdown_connections 2023-12-26T21:36:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.794+0000 7fc6571ff700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc634066f30 0x7fc6340693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.794+0000 7fc6571ff700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc650159810 0x7fc6500686f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.794+0000 7fc6571ff700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cd7b0 0x7fc6500b45f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.794+0000 7fc6571ff700 1 --2- 172.21.15.131:0/4209633905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6500cec80 0x7fc6501471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.794+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 >> 172.21.15.131:0/4209633905 conn(0x7fc650099950 msgr2=0x7fc650154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.794+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 shutdown_connections 2023-12-26T21:36:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:46.794+0000 7fc6571ff700 1 -- 172.21.15.131:0/4209633905 wait complete. 2023-12-26T21:36:46.798 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T21:36:46.817 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T21:36:47.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.014+0000 7fe719dff700 1 Processor -- start 2023-12-26T21:36:47.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.014+0000 7fe719dff700 1 -- start start 2023-12-26T21:36:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.015+0000 7fe719dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe714159650 0x7fe7140cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.015+0000 7fe719dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7140cf430 0x7fe7140cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.015+0000 7fe719dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7140ccbc0 con 0x7fe714159a20 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.015+0000 7fe719dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7140cbfb0 con 0x7fe714159650 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.015+0000 7fe719dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7140cc290 con 0x7fe7140cf430 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.015+0000 7fe7137fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe714159650 0x7fe7140cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.016+0000 7fe7137fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe714159650 0x7fe7140cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56920/0 (socket says 172.21.15.131:56920) 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.016+0000 7fe7137fe700 1 -- 172.21.15.131:0/2167937783 learned_addr learned my addr 172.21.15.131:0/2167937783 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.016+0000 7fe713fff700 1 --2- 172.21.15.131:0/2167937783 >> v2:172.21.15.190:3300/0 conn(0x7fe7140cf430 0x7fe7140cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.016+0000 7fe7137fe700 1 -- 172.21.15.131:0/2167937783 >> v2:172.21.15.190:3300/0 conn(0x7fe7140cf430 msgr2=0x7fe7140cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.016+0000 7fe7137fe700 1 --2- 172.21.15.131:0/2167937783 >> v2:172.21.15.190:3300/0 conn(0x7fe7140cf430 0x7fe7140cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.016+0000 7fe7137fe700 1 -- 172.21.15.131:0/2167937783 >> v1:172.21.15.190:6789/0 conn(0x7fe714159a20 legacy=0x7fe7140ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.016+0000 7fe7137fe700 1 -- 172.21.15.131:0/2167937783 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7140cc570 con 0x7fe714159650 2023-12-26T21:36:47.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.016+0000 7fe7137fe700 1 --2- 172.21.15.131:0/2167937783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159650 0x7fe7140cb460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe7140cacc0 tx=0x7fe704009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e36b63c3b130a3a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe7127fc700 1 -- 172.21.15.131:0/2167937783 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704005c10 con 0x7fe714159650 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe7127fc700 1 -- 172.21.15.131:0/2167937783 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704005d70 con 0x7fe714159650 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe7127fc700 1 -- 172.21.15.131:0/2167937783 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70400b880 con 0x7fe714159650 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe719dff700 1 -- 172.21.15.131:0/2167937783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159650 msgr2=0x7fe7140cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe719dff700 1 --2- 172.21.15.131:0/2167937783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159650 0x7fe7140cb460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe7140cacc0 tx=0x7fe704009220 comp rx=0 tx=0).stop 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe719dff700 1 -- 172.21.15.131:0/2167937783 shutdown_connections 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe719dff700 1 --2- 172.21.15.131:0/2167937783 >> v2:172.21.15.190:3300/0 conn(0x7fe7140cf430 0x7fe7140cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe719dff700 1 --2- 172.21.15.131:0/2167937783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159650 0x7fe7140cb460 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe719dff700 1 -- 172.21.15.131:0/2167937783 >> 172.21.15.131:0/2167937783 conn(0x7fe714099980 msgr2=0x7fe71406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe719dff700 1 -- 172.21.15.131:0/2167937783 shutdown_connections 2023-12-26T21:36:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.017+0000 7fe719dff700 1 -- 172.21.15.131:0/2167937783 wait complete. 2023-12-26T21:36:47.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.019+0000 7fe719dff700 1 Processor -- start 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.019+0000 7fe719dff700 1 -- start start 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe719dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7140cf430 0x7fe71415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe719dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe714159650 0x7fe71416da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe719dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159a20 0x7fe71416df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe719dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71415bb90 con 0x7fe714159a20 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe719dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71415af10 con 0x7fe714159650 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe719dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71415b1f0 con 0x7fe7140cf430 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe713fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159a20 0x7fe71416df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe713fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159a20 0x7fe71416df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56932/0 (socket says 172.21.15.131:56932) 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe713fff700 1 -- 172.21.15.131:0/3674006260 learned_addr learned my addr 172.21.15.131:0/3674006260 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe7137fe700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7140cf430 0x7fe71415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe712ffd700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe714159650 0x7fe71416da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe713fff700 1 -- 172.21.15.131:0/3674006260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7140cf430 msgr2=0x7fe71415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe713fff700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7140cf430 0x7fe71415a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe713fff700 1 -- 172.21.15.131:0/3674006260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe714159650 msgr2=0x7fe71416da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe713fff700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe714159650 0x7fe71416da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe713fff700 1 -- 172.21.15.131:0/3674006260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704009480 con 0x7fe714159a20 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.020+0000 7fe712ffd700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe714159650 0x7fe71416da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.021+0000 7fe713fff700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159a20 0x7fe71416df90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007f20 tx=0x7fe70c0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.021+0000 7fe6fb7fe700 1 -- 172.21.15.131:0/3674006260 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00ebf0 con 0x7fe714159a20 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.021+0000 7fe6fb7fe700 1 -- 172.21.15.131:0/3674006260 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c00ed70 con 0x7fe714159a20 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.021+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71415b4d0 con 0x7fe714159a20 2023-12-26T21:36:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.021+0000 7fe6fb7fe700 1 -- 172.21.15.131:0/3674006260 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c014670 con 0x7fe714159a20 2023-12-26T21:36:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.021+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71416a700 con 0x7fe714159a20 2023-12-26T21:36:47.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.022+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71415bcd0 con 0x7fe714159a20 2023-12-26T21:36:47.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.023+0000 7fe6fb7fe700 1 -- 172.21.15.131:0/3674006260 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe70c014a60 con 0x7fe714159a20 2023-12-26T21:36:47.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.024+0000 7fe6fb7fe700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe700066f30 0x7fe7000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:47.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.024+0000 7fe6fb7fe700 1 -- 172.21.15.131:0/3674006260 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fe70c084a80 con 0x7fe714159a20 2023-12-26T21:36:47.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.024+0000 7fe7137fe700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe700066f30 0x7fe7000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:47.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.025+0000 7fe7137fe700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe700066f30 0x7fe7000693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe704007fd0 tx=0x7fe704016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:47.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.028+0000 7fe6fb7fe700 1 -- 172.21.15.131:0/3674006260 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe71415bcd0 con 0x7fe714159a20 2023-12-26T21:36:47.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.226+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe71415bcd0 con 0x7fe714159a20 2023-12-26T21:36:47.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.805+0000 7fe6fb7fe700 1 -- 172.21.15.131:0/3674006260 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe71415bcd0 con 0x7fe714159a20 2023-12-26T21:36:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.811+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe700066f30 msgr2=0x7fe7000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.811+0000 7fe719dff700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe700066f30 0x7fe7000693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe704007fd0 tx=0x7fe704016000 comp rx=0 tx=0).stop 2023-12-26T21:36:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.811+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159a20 msgr2=0x7fe71416df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.811+0000 7fe719dff700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159a20 0x7fe71416df90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007f20 tx=0x7fe70c0098a0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.814+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 shutdown_connections 2023-12-26T21:36:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.814+0000 7fe719dff700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe714159a20 0x7fe71416df90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.814+0000 7fe719dff700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe700066f30 0x7fe7000693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.814+0000 7fe719dff700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7140cf430 0x7fe71415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.814+0000 7fe719dff700 1 --2- 172.21.15.131:0/3674006260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe714159650 0x7fe71416da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.814+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 >> 172.21.15.131:0/3674006260 conn(0x7fe714099980 msgr2=0x7fe714152e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:47.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.815+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 shutdown_connections 2023-12-26T21:36:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:47.818+0000 7fe719dff700 1 -- 172.21.15.131:0/3674006260 wait complete. 2023-12-26T21:36:47.823 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T21:36:47.843 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T21:36:48.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.035+0000 7f48eabff700 1 Processor -- start 2023-12-26T21:36:48.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.035+0000 7f48eabff700 1 -- start start 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.036+0000 7f48eabff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48e41596a0 0x7f48e40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48eabff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48e40b0870 0x7f48e40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48eabff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e40afcd0 con 0x7f48e4159a70 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48eabff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e40b3e70 con 0x7f48e40b0870 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48eabff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e40b2e00 con 0x7f48e41596a0 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48e919c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48e40b0870 0x7f48e40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48e919c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48e40b0870 0x7f48e40b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56946/0 (socket says 172.21.15.131:56946) 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48e919c700 1 -- 172.21.15.131:0/3751372901 learned_addr learned my addr 172.21.15.131:0/3751372901 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48e899b700 1 --2- 172.21.15.131:0/3751372901 >> v2:172.21.15.190:3300/0 conn(0x7f48e41596a0 0x7f48e40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48e919c700 1 -- 172.21.15.131:0/3751372901 >> v2:172.21.15.190:3300/0 conn(0x7f48e41596a0 msgr2=0x7f48e40b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48e919c700 1 --2- 172.21.15.131:0/3751372901 >> v2:172.21.15.190:3300/0 conn(0x7f48e41596a0 0x7f48e40b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48e919c700 1 -- 172.21.15.131:0/3751372901 >> v1:172.21.15.190:6789/0 conn(0x7f48e4159a70 legacy=0x7f48e40b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.037+0000 7f48e919c700 1 -- 172.21.15.131:0/3751372901 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e40b0cb0 con 0x7f48e40b0870 2023-12-26T21:36:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.038+0000 7f48e919c700 1 --2- 172.21.15.131:0/3751372901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e40b0870 0x7f48e40b28c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48e40892a0 tx=0x7f48dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ea41ca0164bbd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.038+0000 7f48e37fe700 1 -- 172.21.15.131:0/3751372901 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00e070 con 0x7f48e40b0870 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.038+0000 7f48e37fe700 1 -- 172.21.15.131:0/3751372901 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc00ba20 con 0x7f48e40b0870 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.038+0000 7f48e37fe700 1 -- 172.21.15.131:0/3751372901 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc0092e0 con 0x7f48e40b0870 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.038+0000 7f48eabff700 1 -- 172.21.15.131:0/3751372901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e40b0870 msgr2=0x7f48e40b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.038+0000 7f48eabff700 1 --2- 172.21.15.131:0/3751372901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e40b0870 0x7f48e40b28c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48e40892a0 tx=0x7f48dc009840 comp rx=0 tx=0).stop 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.039+0000 7f48eabff700 1 -- 172.21.15.131:0/3751372901 shutdown_connections 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.039+0000 7f48eabff700 1 --2- 172.21.15.131:0/3751372901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e40b0870 0x7f48e40b28c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.039+0000 7f48eabff700 1 --2- 172.21.15.131:0/3751372901 >> v2:172.21.15.190:3300/0 conn(0x7f48e41596a0 0x7f48e40b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.039+0000 7f48eabff700 1 -- 172.21.15.131:0/3751372901 >> 172.21.15.131:0/3751372901 conn(0x7f48e406ece0 msgr2=0x7f48e406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.039+0000 7f48eabff700 1 -- 172.21.15.131:0/3751372901 shutdown_connections 2023-12-26T21:36:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.039+0000 7f48eabff700 1 -- 172.21.15.131:0/3751372901 wait complete. 2023-12-26T21:36:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.040+0000 7f48eabff700 1 Processor -- start 2023-12-26T21:36:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.040+0000 7f48eabff700 1 -- start start 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.041+0000 7f48eabff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e40b0870 0x7f48e40cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.041+0000 7f48eabff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48e41596a0 0x7f48e4151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.041+0000 7f48eabff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4159a70 0x7f48e415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.041+0000 7f48eabff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e415afd0 con 0x7f48e4159a70 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.041+0000 7f48eabff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e415a3e0 con 0x7f48e40b0870 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.041+0000 7f48eabff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e415a6a0 con 0x7f48e41596a0 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.041+0000 7f48e919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4159a70 0x7f48e415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.041+0000 7f48e919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4159a70 0x7f48e415b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56954/0 (socket says 172.21.15.131:56954) 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.041+0000 7f48e919c700 1 -- 172.21.15.131:0/477091298 learned_addr learned my addr 172.21.15.131:0/477091298 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e899b700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e40b0870 0x7f48e40cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e3fff700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48e41596a0 0x7f48e4151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e919c700 1 -- 172.21.15.131:0/477091298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48e41596a0 msgr2=0x7f48e4151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e919c700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48e41596a0 0x7f48e4151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e919c700 1 -- 172.21.15.131:0/477091298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e40b0870 msgr2=0x7f48e40cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e919c700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e40b0870 0x7f48e40cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e919c700 1 -- 172.21.15.131:0/477091298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc009ae0 con 0x7f48e4159a70 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e3fff700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48e41596a0 0x7f48e4151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e919c700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4159a70 0x7f48e415b460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009880 tx=0x7f48dc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e0ff9700 1 -- 172.21.15.131:0/477091298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc008b30 con 0x7f48e4159a70 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e0ff9700 1 -- 172.21.15.131:0/477091298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc008cb0 con 0x7f48e4159a70 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e415a980 con 0x7f48e4159a70 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48e0ff9700 1 -- 172.21.15.131:0/477091298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc0256b0 con 0x7f48e4159a70 2023-12-26T21:36:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.042+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e40cdaa0 con 0x7f48e4159a70 2023-12-26T21:36:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.043+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e415b180 con 0x7f48e4159a70 2023-12-26T21:36:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.044+0000 7f48e0ff9700 1 -- 172.21.15.131:0/477091298 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f48dc003040 con 0x7f48e4159a70 2023-12-26T21:36:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.045+0000 7f48e0ff9700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48c8066f60 0x7f48c8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.045+0000 7f48e0ff9700 1 -- 172.21.15.131:0/477091298 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f48dc086090 con 0x7f48e4159a70 2023-12-26T21:36:48.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.045+0000 7f48e899b700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48c8066f60 0x7f48c8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:48.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.046+0000 7f48e899b700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48c8066f60 0x7f48c8069410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009db0 tx=0x7f48d80091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:48.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.050+0000 7f48e0ff9700 1 -- 172.21.15.131:0/477091298 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48e415b180 con 0x7f48e4159a70 2023-12-26T21:36:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.286+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f48e40ce9d0 con 0x7f48e4159a70 2023-12-26T21:36:48.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.803+0000 7f48e0ff9700 1 -- 172.21.15.131:0/477091298 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f48dc053360 con 0x7f48e4159a70 2023-12-26T21:36:48.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48c8066f60 msgr2=0x7f48c8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:48.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48c8066f60 0x7f48c8069410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009db0 tx=0x7f48d80091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4159a70 msgr2=0x7f48e415b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:48.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4159a70 0x7f48e415b460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009880 tx=0x7f48dc018000 comp rx=0 tx=0).stop 2023-12-26T21:36:48.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 shutdown_connections 2023-12-26T21:36:48.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4159a70 0x7f48e415b460 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e40b0870 0x7f48e40cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48c8066f60 0x7f48c8069410 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009db0 tx=0x7f48d80091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 --2- 172.21.15.131:0/477091298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48e41596a0 0x7f48e4151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:48.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.816+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 >> 172.21.15.131:0/477091298 conn(0x7f48e406ece0 msgr2=0x7f48e40ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:48.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.824+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 shutdown_connections 2023-12-26T21:36:48.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:48.824+0000 7f48eabff700 1 -- 172.21.15.131:0/477091298 wait complete. 2023-12-26T21:36:48.833 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 2 and data pool 3 2023-12-26T21:36:48.855 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.046+0000 7f17183ff700 1 Processor -- start 2023-12-26T21:36:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.047+0000 7f17183ff700 1 -- start start 2023-12-26T21:36:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f17183ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17101596a0 0x7f17100b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f17183ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17100b0870 0x7f17100b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f17183ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17100afcd0 con 0x7f1710159a70 2023-12-26T21:36:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f17183ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17100b3e70 con 0x7f17101596a0 2023-12-26T21:36:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f17183ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17100b2e00 con 0x7f17100b0870 2023-12-26T21:36:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f171699c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17100b0870 0x7f17100b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f171699c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17100b0870 0x7f17100b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:34962/0 (socket says 172.21.15.131:34962) 2023-12-26T21:36:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f171599a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1710159a70 0x7f17100b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:47382/0 (socket says 172.21.15.131:47382) 2023-12-26T21:36:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f171699c700 1 -- 172.21.15.131:0/3267572293 learned_addr learned my addr 172.21.15.131:0/3267572293 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.048+0000 7f171619b700 1 --2- 172.21.15.131:0/3267572293 >> v2:172.21.15.190:3300/0 conn(0x7f17101596a0 0x7f17100b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.049+0000 7f171699c700 1 -- 172.21.15.131:0/3267572293 >> v2:172.21.15.190:3300/0 conn(0x7f17101596a0 msgr2=0x7f17100b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.049+0000 7f171699c700 1 --2- 172.21.15.131:0/3267572293 >> v2:172.21.15.190:3300/0 conn(0x7f17101596a0 0x7f17100b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.049+0000 7f171699c700 1 -- 172.21.15.131:0/3267572293 >> v1:172.21.15.190:6789/0 conn(0x7f1710159a70 legacy=0x7f17100b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.049+0000 7f1715199700 1 -- 172.21.15.131:0/3267572293 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477348495 0 0) 0x7f17100afcd0 con 0x7f1710159a70 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.049+0000 7f171699c700 1 -- 172.21.15.131:0/3267572293 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17100b0cb0 con 0x7f17100b0870 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.049+0000 7f171699c700 1 --2- 172.21.15.131:0/3267572293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17100b0870 0x7f17100b28c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1708009f20 tx=0x7f1708009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd1346b984f59f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.049+0000 7f1715199700 1 -- 172.21.15.131:0/3267572293 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170800e070 con 0x7f17100b0870 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.049+0000 7f1715199700 1 -- 172.21.15.131:0/3267572293 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170800b9d0 con 0x7f17100b0870 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.049+0000 7f1715199700 1 -- 172.21.15.131:0/3267572293 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080092e0 con 0x7f17100b0870 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.050+0000 7f17183ff700 1 -- 172.21.15.131:0/3267572293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17100b0870 msgr2=0x7f17100b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.050+0000 7f17183ff700 1 --2- 172.21.15.131:0/3267572293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17100b0870 0x7f17100b28c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1708009f20 tx=0x7f1708009840 comp rx=0 tx=0).stop 2023-12-26T21:36:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.050+0000 7f17183ff700 1 -- 172.21.15.131:0/3267572293 shutdown_connections 2023-12-26T21:36:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.050+0000 7f17183ff700 1 --2- 172.21.15.131:0/3267572293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17100b0870 0x7f17100b28c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.050+0000 7f17183ff700 1 --2- 172.21.15.131:0/3267572293 >> v2:172.21.15.190:3300/0 conn(0x7f17101596a0 0x7f17100b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.050+0000 7f17183ff700 1 -- 172.21.15.131:0/3267572293 >> 172.21.15.131:0/3267572293 conn(0x7f171006ece0 msgr2=0x7f171006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.050+0000 7f17183ff700 1 -- 172.21.15.131:0/3267572293 shutdown_connections 2023-12-26T21:36:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.050+0000 7f17183ff700 1 -- 172.21.15.131:0/3267572293 wait complete. 2023-12-26T21:36:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.051+0000 7f17183ff700 1 Processor -- start 2023-12-26T21:36:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.051+0000 7f17183ff700 1 -- start start 2023-12-26T21:36:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.052+0000 7f17183ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17100b0870 0x7f1710158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.052+0000 7f17183ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17101596a0 0x7f17100c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.052+0000 7f17183ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1710159a70 0x7f17100c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.052+0000 7f17183ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171015ab10 con 0x7f1710159a70 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.052+0000 7f17183ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1710159270 con 0x7f17100b0870 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.052+0000 7f17183ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1710159f00 con 0x7f17101596a0 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1710159a70 0x7f17100c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1710159a70 0x7f17100c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56960/0 (socket says 172.21.15.131:56960) 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171699c700 1 -- 172.21.15.131:0/132889234 learned_addr learned my addr 172.21.15.131:0/132889234 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171599a700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17101596a0 0x7f17100c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171619b700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17100b0870 0x7f1710158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171699c700 1 -- 172.21.15.131:0/132889234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17101596a0 msgr2=0x7f17100c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171699c700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17101596a0 0x7f17100c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171699c700 1 -- 172.21.15.131:0/132889234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17100b0870 msgr2=0x7f1710158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171699c700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17100b0870 0x7f1710158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171699c700 1 -- 172.21.15.131:0/132889234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708009ae0 con 0x7f1710159a70 2023-12-26T21:36:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f171699c700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1710159a70 0x7f17100c7b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f170800b610 tx=0x7f1708018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f17067fc700 1 -- 172.21.15.131:0/132889234 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708008c30 con 0x7f1710159a70 2023-12-26T21:36:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f17067fc700 1 -- 172.21.15.131:0/132889234 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708008dd0 con 0x7f1710159a70 2023-12-26T21:36:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f17067fc700 1 -- 172.21.15.131:0/132889234 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708026660 con 0x7f1710159a70 2023-12-26T21:36:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.053+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171015a1e0 con 0x7f1710159a70 2023-12-26T21:36:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.054+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171015a7a0 con 0x7f1710159a70 2023-12-26T21:36:49.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.055+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171015acc0 con 0x7f1710159a70 2023-12-26T21:36:49.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.055+0000 7f17067fc700 1 -- 172.21.15.131:0/132889234 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1708003040 con 0x7f1710159a70 2023-12-26T21:36:49.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.056+0000 7f17067fc700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066f30 0x7f16f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.056+0000 7f17067fc700 1 -- 172.21.15.131:0/132889234 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1708085ed0 con 0x7f1710159a70 2023-12-26T21:36:49.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.056+0000 7f171619b700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066f30 0x7f16f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.057+0000 7f171619b700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066f30 0x7f16f40693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f170c009ca0 tx=0x7f170c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.061+0000 7f17067fc700 1 -- 172.21.15.131:0/132889234 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f171015acc0 con 0x7f1710159a70 2023-12-26T21:36:49.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.258+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f171015acc0 con 0x7f1710159a70 2023-12-26T21:36:49.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.259+0000 7f17067fc700 1 -- 172.21.15.131:0/132889234 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f17080530a0 con 0x7f1710159a70 2023-12-26T21:36:49.260 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:49.260 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":21,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:48.288539+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066f30 msgr2=0x7f16f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066f30 0x7f16f40693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f170c009ca0 tx=0x7f170c008040 comp rx=0 tx=0).stop 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1710159a70 msgr2=0x7f17100c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1710159a70 0x7f17100c7b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f170800b610 tx=0x7f1708018000 comp rx=0 tx=0).stop 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 shutdown_connections 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1710159a70 0x7f17100c7b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17100b0870 0x7f1710158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066f30 0x7f16f40693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 --2- 172.21.15.131:0/132889234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17101596a0 0x7f17100c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 >> 172.21.15.131:0/132889234 conn(0x7f171006ece0 msgr2=0x7f17100d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 shutdown_connections 2023-12-26T21:36:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.262+0000 7f17183ff700 1 -- 172.21.15.131:0/132889234 wait complete. 2023-12-26T21:36:49.280 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:49.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.470+0000 7f5d85dff700 1 Processor -- start 2023-12-26T21:36:49.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.470+0000 7f5d85dff700 1 -- start start 2023-12-26T21:36:49.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d85dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5d8013c510 0x7f5d800c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d85dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d8006b8c0 0x7f5d8006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d85dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d800686c0 con 0x7f5d800cc100 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d85dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8006c280 con 0x7f5d8013c510 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d85dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d80069520 con 0x7f5d8006b8c0 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d7f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5d8013c510 0x7f5d800c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d7f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5d8013c510 0x7f5d800c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56964/0 (socket says 172.21.15.131:56964) 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d7f7fe700 1 -- 172.21.15.131:0/2507722489 learned_addr learned my addr 172.21.15.131:0/2507722489 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d7ffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5d800cc100 0x7f5d800cc520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47800/0 (socket says 172.21.15.131:47800) 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.471+0000 7f5d7effd700 1 --2- 172.21.15.131:0/2507722489 >> v2:172.21.15.190:3300/0 conn(0x7f5d8006b8c0 0x7f5d8006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.472+0000 7f5d7f7fe700 1 -- 172.21.15.131:0/2507722489 >> v2:172.21.15.190:3300/0 conn(0x7f5d8006b8c0 msgr2=0x7f5d8006bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.472+0000 7f5d7f7fe700 1 --2- 172.21.15.131:0/2507722489 >> v2:172.21.15.190:3300/0 conn(0x7f5d8006b8c0 0x7f5d8006bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.472+0000 7f5d7f7fe700 1 -- 172.21.15.131:0/2507722489 >> v1:172.21.15.131:6789/0 conn(0x7f5d800cc100 legacy=0x7f5d800cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.472+0000 7f5d7f7fe700 1 -- 172.21.15.131:0/2507722489 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d80068910 con 0x7f5d8013c510 2023-12-26T21:36:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.472+0000 7f5d7f7fe700 1 --2- 172.21.15.131:0/2507722489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8013c510 0x7f5d800c8a20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5d800b43e0 tx=0x7f5d70009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfda1e70d2a415c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:49.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.472+0000 7f5d7e7fc700 1 -- 172.21.15.131:0/2507722489 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70005c10 con 0x7f5d8013c510 2023-12-26T21:36:49.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.472+0000 7f5d7e7fc700 1 -- 172.21.15.131:0/2507722489 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70005d70 con 0x7f5d8013c510 2023-12-26T21:36:49.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.472+0000 7f5d7e7fc700 1 -- 172.21.15.131:0/2507722489 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70006260 con 0x7f5d8013c510 2023-12-26T21:36:49.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.473+0000 7f5d85dff700 1 -- 172.21.15.131:0/2507722489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8013c510 msgr2=0x7f5d800c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.473+0000 7f5d85dff700 1 --2- 172.21.15.131:0/2507722489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8013c510 0x7f5d800c8a20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5d800b43e0 tx=0x7f5d70009220 comp rx=0 tx=0).stop 2023-12-26T21:36:49.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.473+0000 7f5d85dff700 1 -- 172.21.15.131:0/2507722489 shutdown_connections 2023-12-26T21:36:49.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.473+0000 7f5d85dff700 1 --2- 172.21.15.131:0/2507722489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d8013c510 0x7f5d800c8a20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.473+0000 7f5d85dff700 1 --2- 172.21.15.131:0/2507722489 >> v2:172.21.15.190:3300/0 conn(0x7f5d8006b8c0 0x7f5d8006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.473+0000 7f5d85dff700 1 -- 172.21.15.131:0/2507722489 >> 172.21.15.131:0/2507722489 conn(0x7f5d800ce3e0 msgr2=0x7f5d800ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:49.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.473+0000 7f5d85dff700 1 -- 172.21.15.131:0/2507722489 shutdown_connections 2023-12-26T21:36:49.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.473+0000 7f5d85dff700 1 -- 172.21.15.131:0/2507722489 wait complete. 2023-12-26T21:36:49.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.474+0000 7f5d85dff700 1 Processor -- start 2023-12-26T21:36:49.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.474+0000 7f5d85dff700 1 -- start start 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.475+0000 7f5d85dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d8006b8c0 0x7f5d80155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.475+0000 7f5d85dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d800cc100 0x7f5d800b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.475+0000 7f5d85dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d8013c510 0x7f5d8014fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.475+0000 7f5d85dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80155560 con 0x7f5d800cc100 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.475+0000 7f5d85dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d800b16c0 con 0x7f5d8013c510 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.475+0000 7f5d85dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8016e0b0 con 0x7f5d8006b8c0 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.475+0000 7f5d7effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d800cc100 0x7f5d800b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.475+0000 7f5d7effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d800cc100 0x7f5d800b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56966/0 (socket says 172.21.15.131:56966) 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d7effd700 1 -- 172.21.15.131:0/2168442453 learned_addr learned my addr 172.21.15.131:0/2168442453 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d7f7fe700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d8006b8c0 0x7f5d80155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d7ffff700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d8013c510 0x7f5d8014fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d7effd700 1 -- 172.21.15.131:0/2168442453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d8006b8c0 msgr2=0x7f5d80155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d7effd700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d8006b8c0 0x7f5d80155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d7effd700 1 -- 172.21.15.131:0/2168442453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d8013c510 msgr2=0x7f5d8014fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d7effd700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d8013c510 0x7f5d8014fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d7effd700 1 -- 172.21.15.131:0/2168442453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70009480 con 0x7f5d800cc100 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d7effd700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d800cc100 0x7f5d800b1180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400e930 tx=0x7f5d740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d677fe700 1 -- 172.21.15.131:0/2168442453 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74013680 con 0x7f5d800cc100 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d677fe700 1 -- 172.21.15.131:0/2168442453 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d740137e0 con 0x7f5d800cc100 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8016d200 con 0x7f5d800cc100 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.476+0000 7f5d677fe700 1 -- 172.21.15.131:0/2168442453 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74013b80 con 0x7f5d800cc100 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.477+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8016d550 con 0x7f5d800cc100 2023-12-26T21:36:49.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.477+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d800657e0 con 0x7f5d800cc100 2023-12-26T21:36:49.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.478+0000 7f5d677fe700 1 -- 172.21.15.131:0/2168442453 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5d7400c3e0 con 0x7f5d800cc100 2023-12-26T21:36:49.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.480+0000 7f5d677fe700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5d6c066f30 0x7f5d6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.480+0000 7f5d677fe700 1 -- 172.21.15.131:0/2168442453 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5d740845e0 con 0x7f5d800cc100 2023-12-26T21:36:49.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.480+0000 7f5d7f7fe700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5d6c066f30 0x7f5d6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.480+0000 7f5d7f7fe700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5d6c066f30 0x7f5d6c0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009260 tx=0x7f5d70016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:49.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.484+0000 7f5d677fe700 1 -- 172.21.15.131:0/2168442453 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d74051850 con 0x7f5d800cc100 2023-12-26T21:36:49.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.676+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d80068bf0 con 0x7f5d800cc100 2023-12-26T21:36:49.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.677+0000 7f5d677fe700 1 -- 172.21.15.131:0/2168442453 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f5d74054e70 con 0x7f5d800cc100 2023-12-26T21:36:49.678 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:49.678 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":21,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:48.288539+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5d6c066f30 msgr2=0x7f5d6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5d6c066f30 0x7f5d6c0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009260 tx=0x7f5d70016000 comp rx=0 tx=0).stop 2023-12-26T21:36:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d800cc100 msgr2=0x7f5d800b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d800cc100 0x7f5d800b1180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400e930 tx=0x7f5d740091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 shutdown_connections 2023-12-26T21:36:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d8013c510 0x7f5d8014fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5d6c066f30 0x7f5d6c0693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d8006b8c0 0x7f5d80155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 --2- 172.21.15.131:0/2168442453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d800cc100 0x7f5d800b1180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 >> 172.21.15.131:0/2168442453 conn(0x7f5d800ce3e0 msgr2=0x7f5d8013bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.679+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 shutdown_connections 2023-12-26T21:36:49.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.680+0000 7f5d85dff700 1 -- 172.21.15.131:0/2168442453 wait complete. 2023-12-26T21:36:49.698 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T21:36:49.778 INFO:tasks.ceph.mds.c.smithi131.stderr:2023-12-26T21:36:49.778+0000 7f70f7df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:36:49.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.892+0000 7f16c25ff700 1 Processor -- start 2023-12-26T21:36:49.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.892+0000 7f16c25ff700 1 -- start start 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.893+0000 7f16c25ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f16bc0cd100 0x7f16bc0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.893+0000 7f16c25ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f16bc0cd4d0 0x7f16bc0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.893+0000 7f16c25ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16bc15db90 con 0x7f16bc1591f0 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.893+0000 7f16c25ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16bc0cbe50 con 0x7f16bc0cd100 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.893+0000 7f16c25ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16bc15d330 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f16bc0cd100 0x7f16bc0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f16bc0cd100 0x7f16bc0cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56968/0 (socket says 172.21.15.131:56968) 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bbfff700 1 -- 172.21.15.131:0/1650961266 learned_addr learned my addr 172.21.15.131:0/1650961266 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bb7fe700 1 --2- 172.21.15.131:0/1650961266 >> v2:172.21.15.190:3300/0 conn(0x7f16bc0cd4d0 0x7f16bc0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bbfff700 1 -- 172.21.15.131:0/1650961266 >> v2:172.21.15.190:3300/0 conn(0x7f16bc0cd4d0 msgr2=0x7f16bc0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bbfff700 1 --2- 172.21.15.131:0/1650961266 >> v2:172.21.15.190:3300/0 conn(0x7f16bc0cd4d0 0x7f16bc0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bbfff700 1 -- 172.21.15.131:0/1650961266 >> v1:172.21.15.131:6789/0 conn(0x7f16bc1591f0 legacy=0x7f16bc159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bbfff700 1 -- 172.21.15.131:0/1650961266 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0cbb70 con 0x7f16bc0cd100 2023-12-26T21:36:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bb7fe700 1 --2- 172.21.15.131:0/1650961266 >> v2:172.21.15.190:3300/0 conn(0x7f16bc0cd4d0 0x7f16bc0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16baffd700 1 -- 172.21.15.131:0/1650961266 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022898545 0 0) 0x7f16bc15db90 con 0x7f16bc1591f0 2023-12-26T21:36:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16bbfff700 1 --2- 172.21.15.131:0/1650961266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd100 0x7f16bc0cb590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0cf900 tx=0x7f16ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6733fac26e59b5d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.894+0000 7f16baffd700 1 -- 172.21.15.131:0/1650961266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac00b860 con 0x7f16bc0cd100 2023-12-26T21:36:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16baffd700 1 -- 172.21.15.131:0/1650961266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac00b9c0 con 0x7f16bc0cd100 2023-12-26T21:36:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16baffd700 1 -- 172.21.15.131:0/1650961266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac00bd60 con 0x7f16bc0cd100 2023-12-26T21:36:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16c25ff700 1 -- 172.21.15.131:0/1650961266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd100 msgr2=0x7f16bc0cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16c25ff700 1 --2- 172.21.15.131:0/1650961266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd100 0x7f16bc0cb590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0cf900 tx=0x7f16ac009250 comp rx=0 tx=0).stop 2023-12-26T21:36:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16c25ff700 1 -- 172.21.15.131:0/1650961266 shutdown_connections 2023-12-26T21:36:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16c25ff700 1 --2- 172.21.15.131:0/1650961266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd100 0x7f16bc0cb590 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16c25ff700 1 --2- 172.21.15.131:0/1650961266 >> v2:172.21.15.190:3300/0 conn(0x7f16bc0cd4d0 0x7f16bc0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16c25ff700 1 -- 172.21.15.131:0/1650961266 >> 172.21.15.131:0/1650961266 conn(0x7f16bc06ece0 msgr2=0x7f16bc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16c25ff700 1 -- 172.21.15.131:0/1650961266 shutdown_connections 2023-12-26T21:36:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.895+0000 7f16c25ff700 1 -- 172.21.15.131:0/1650961266 wait complete. 2023-12-26T21:36:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.897+0000 7f16c25ff700 1 Processor -- start 2023-12-26T21:36:49.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.897+0000 7f16c25ff700 1 -- start start 2023-12-26T21:36:49.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16c25ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16bc0cd100 0x7f16bc15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16c25ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd4d0 0x7f16bc0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16c25ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16bc1591f0 0x7f16bc0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16c25ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc16da10 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16c25ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc16dea0 con 0x7f16bc1591f0 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16c25ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc15c650 con 0x7f16bc0cd100 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16bb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd4d0 0x7f16bc0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16bb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd4d0 0x7f16bc0b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56978/0 (socket says 172.21.15.131:56978) 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16bb7fe700 1 -- 172.21.15.131:0/490807875 learned_addr learned my addr 172.21.15.131:0/490807875 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16c0b9c700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16bc1591f0 0x7f16bc0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16bbfff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16bc0cd100 0x7f16bc15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16bb7fe700 1 -- 172.21.15.131:0/490807875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16bc0cd100 msgr2=0x7f16bc15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16bb7fe700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16bc0cd100 0x7f16bc15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16bb7fe700 1 -- 172.21.15.131:0/490807875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16bc1591f0 msgr2=0x7f16bc0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16bb7fe700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16bc1591f0 0x7f16bc0b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.898+0000 7f16bb7fe700 1 -- 172.21.15.131:0/490807875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac0094f0 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.899+0000 7f16bb7fe700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd4d0 0x7f16bc0b2200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f16b000c830 tx=0x7f16b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.899+0000 7f16a3fff700 1 -- 172.21.15.131:0/490807875 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0011600 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.899+0000 7f16a3fff700 1 -- 172.21.15.131:0/490807875 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b0011760 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.899+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc16d120 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.899+0000 7f16a3fff700 1 -- 172.21.15.131:0/490807875 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0011b20 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.899+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc16d3c0 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.900+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc16db80 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.901+0000 7f16a3fff700 1 -- 172.21.15.131:0/490807875 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f16b0018360 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.902+0000 7f16a3fff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16a8066f30 0x7f16a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.902+0000 7f16a3fff700 1 -- 172.21.15.131:0/490807875 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f16b0084510 con 0x7f16bc0cd4d0 2023-12-26T21:36:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.902+0000 7f16bbfff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16a8066f30 0x7f16a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.902+0000 7f16bbfff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16a8066f30 0x7f16a80693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0cf900 tx=0x7f16ac017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:49.905+0000 7f16a3fff700 1 -- 172.21.15.131:0/490807875 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16b00516e0 con 0x7f16bc0cd4d0 2023-12-26T21:36:50.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.147+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f16bc0657e0 con 0x7f16bc0cd4d0 2023-12-26T21:36:50.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.988+0000 7f16a3fff700 1 -- 172.21.15.131:0/490807875 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f16b0054d00 con 0x7f16bc0cd4d0 2023-12-26T21:36:50.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16a8066f30 msgr2=0x7f16a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:50.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16a8066f30 0x7f16a80693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0cf900 tx=0x7f16ac017000 comp rx=0 tx=0).stop 2023-12-26T21:36:50.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd4d0 msgr2=0x7f16bc0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:50.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd4d0 0x7f16bc0b2200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f16b000c830 tx=0x7f16b0008040 comp rx=0 tx=0).stop 2023-12-26T21:36:50.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 shutdown_connections 2023-12-26T21:36:50.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16bc1591f0 0x7f16bc0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:50.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16a8066f30 0x7f16a80693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:50.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16bc0cd100 0x7f16bc15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:50.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 --2- 172.21.15.131:0/490807875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16bc0cd4d0 0x7f16bc0b2200 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:50.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 >> 172.21.15.131:0/490807875 conn(0x7f16bc06ece0 msgr2=0x7f16bc159670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:50.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 shutdown_connections 2023-12-26T21:36:50.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:50.991+0000 7f16c25ff700 1 -- 172.21.15.131:0/490807875 wait complete. 2023-12-26T21:36:51.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T21:36:51.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.203+0000 7f36531ff700 1 Processor -- start 2023-12-26T21:36:51.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.203+0000 7f36531ff700 1 -- start start 2023-12-26T21:36:51.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.204+0000 7f36531ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f364c159820 0x7f364c0abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:51.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.204+0000 7f36531ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c0aa480 0x7f364c06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:51.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.204+0000 7f36531ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c15a8c0 con 0x7f364c06e2b0 2023-12-26T21:36:51.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.204+0000 7f36531ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c15ad50 con 0x7f364c159820 2023-12-26T21:36:51.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.204+0000 7f36531ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c06e730 con 0x7f364c0aa480 2023-12-26T21:36:51.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f365179c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f364c06e2b0 0x7f364c15d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47816/0 (socket says 172.21.15.131:47816) 2023-12-26T21:36:51.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f365179c700 1 -- 172.21.15.131:0/1143181840 learned_addr learned my addr 172.21.15.131:0/1143181840 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:51.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364bfff700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c0aa480 0x7f364c06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:51.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f3650f9b700 1 --2- 172.21.15.131:0/1143181840 >> v2:172.21.15.190:3300/0 conn(0x7f364c159820 0x7f364c0abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:51.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268619927 0 0) 0x7f364c15a8c0 con 0x7f364c06e2b0 2023-12-26T21:36:51.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f363c003570 con 0x7f364c06e2b0 2023-12-26T21:36:51.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 667437169 0 0) 0x7f363c003570 con 0x7f364c06e2b0 2023-12-26T21:36:51.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c0aa480 msgr2=0x7f364c06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c0aa480 0x7f364c06dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 >> v2:172.21.15.190:3300/0 conn(0x7f364c159820 msgr2=0x7f364c0abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 --2- 172.21.15.131:0/1143181840 >> v2:172.21.15.190:3300/0 conn(0x7f364c159820 0x7f364c0abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c159f90 con 0x7f364c06e2b0 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f36440017f0 con 0x7f364c06e2b0 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 >> v1:172.21.15.131:6789/0 conn(0x7f364c06e2b0 legacy=0x7f364c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c0aa480 0x7f363c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.205+0000 7f364b7fe700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f363c004010 0x7f363c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364b7fe700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c004d50 0x7f363c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c06e730 con 0x7f363c004010 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c00ba60 con 0x7f363c004d50 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c00bbf0 con 0x7f364c0aa480 2023-12-26T21:36:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364bfff700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f363c004010 0x7f363c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f3650f9b700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c0aa480 0x7f363c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f365179c700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c004d50 0x7f363c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364bfff700 1 -- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c0aa480 msgr2=0x7f363c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364bfff700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c0aa480 0x7f363c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364bfff700 1 -- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c004d50 msgr2=0x7f363c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364bfff700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c004d50 0x7f363c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364bfff700 1 -- 172.21.15.131:0/1143181840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c003570 con 0x7f363c004010 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.206+0000 7f364bfff700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f363c004010 0x7f363c004810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3638009c00 tx=0x7f36380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd29557b74b44b22 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363800e710 con 0x7f363c004010 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363800e870 con 0x7f363c004010 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f364b7fe700 1 -- 172.21.15.131:0/1143181840 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363800ec30 con 0x7f363c004010 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f36531ff700 1 -- 172.21.15.131:0/1143181840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f363c004010 msgr2=0x7f363c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:51.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f36531ff700 1 --2- 172.21.15.131:0/1143181840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f363c004010 0x7f363c004810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3638009c00 tx=0x7f36380091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f365179c700 1 -- 172.21.15.131:0/1143181840 reap_dead start 2023-12-26T21:36:51.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f36531ff700 1 -- 172.21.15.131:0/1143181840 shutdown_connections 2023-12-26T21:36:51.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f36531ff700 1 -- 172.21.15.131:0/1143181840 >> 172.21.15.131:0/1143181840 conn(0x7f364c06c290 msgr2=0x7f364c0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:51.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f36531ff700 1 -- 172.21.15.131:0/1143181840 shutdown_connections 2023-12-26T21:36:51.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.207+0000 7f36531ff700 1 -- 172.21.15.131:0/1143181840 wait complete. 2023-12-26T21:36:51.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.209+0000 7f36531ff700 1 Processor -- start 2023-12-26T21:36:51.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.209+0000 7f36531ff700 1 -- start start 2023-12-26T21:36:51.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f36531ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c004010 0x7f364c14c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:51.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f36531ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f364c16a6d0 0x7f364c067ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:51.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f36531ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c068000 0x7f364c16cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:51.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f36531ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c157c80 con 0x7f364c16a6d0 2023-12-26T21:36:51.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f36531ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c1699e0 con 0x7f363c004010 2023-12-26T21:36:51.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f36531ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c068480 con 0x7f364c068000 2023-12-26T21:36:51.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f364bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f364c16a6d0 0x7f364c067ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:51.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f364bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f364c16a6d0 0x7f364c067ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56984/0 (socket says 172.21.15.131:56984) 2023-12-26T21:36:51.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f364bfff700 1 -- 172.21.15.131:0/2787045356 learned_addr learned my addr 172.21.15.131:0/2787045356 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:51.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f365179c700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c068000 0x7f364c16cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f3650f9b700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c004010 0x7f364c14c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f364bfff700 1 -- 172.21.15.131:0/2787045356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c068000 msgr2=0x7f364c16cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f364bfff700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c068000 0x7f364c16cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f364bfff700 1 -- 172.21.15.131:0/2787045356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c004010 msgr2=0x7f364c14c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f364bfff700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c004010 0x7f364c14c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.210+0000 7f364bfff700 1 -- 172.21.15.131:0/2787045356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c16a240 con 0x7f364c16a6d0 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.211+0000 7f364bfff700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f364c16a6d0 0x7f364c067ac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f363800e4b0 tx=0x7f363801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.211+0000 7f3648ff9700 1 -- 172.21.15.131:0/2787045356 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638023070 con 0x7f364c16a6d0 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.211+0000 7f3648ff9700 1 -- 172.21.15.131:0/2787045356 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363800ec10 con 0x7f364c16a6d0 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.211+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c16dc30 con 0x7f364c16a6d0 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.211+0000 7f3648ff9700 1 -- 172.21.15.131:0/2787045356 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363800e710 con 0x7f364c16a6d0 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.211+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c169ed0 con 0x7f364c16a6d0 2023-12-26T21:36:51.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.212+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c0657e0 con 0x7f364c16a6d0 2023-12-26T21:36:51.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.213+0000 7f3648ff9700 1 -- 172.21.15.131:0/2787045356 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3638013680 con 0x7f364c16a6d0 2023-12-26T21:36:51.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.214+0000 7f3648ff9700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3630066f30 0x7f36300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:51.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.214+0000 7f3648ff9700 1 -- 172.21.15.131:0/2787045356 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3638085c70 con 0x7f364c16a6d0 2023-12-26T21:36:51.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.214+0000 7f3650f9b700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3630066f30 0x7f36300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:51.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.214+0000 7f3650f9b700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3630066f30 0x7f36300693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3640006f00 tx=0x7f36400091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:51.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.218+0000 7f3648ff9700 1 -- 172.21.15.131:0/2787045356 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3638052e40 con 0x7f364c16a6d0 2023-12-26T21:36:51.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.456+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f364c15a270 con 0x7f364c16a6d0 2023-12-26T21:36:51.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.994+0000 7f3648ff9700 1 -- 172.21.15.131:0/2787045356 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f36380170c0 con 0x7f364c16a6d0 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3630066f30 msgr2=0x7f36300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3630066f30 0x7f36300693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3640006f00 tx=0x7f36400091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f364c16a6d0 msgr2=0x7f364c067ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f364c16a6d0 0x7f364c067ac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f363800e4b0 tx=0x7f363801a000 comp rx=0 tx=0).stop 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 shutdown_connections 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3630066f30 0x7f36300693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c068000 0x7f364c16cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f363c004010 0x7f364c14c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 --2- 172.21.15.131:0/2787045356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f364c16a6d0 0x7f364c067ac0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 >> 172.21.15.131:0/2787045356 conn(0x7f364c06c290 msgr2=0x7f364c158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:51.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 shutdown_connections 2023-12-26T21:36:51.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:51.997+0000 7f36531ff700 1 -- 172.21.15.131:0/2787045356 wait complete. 2023-12-26T21:36:52.018 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:36:52.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.206+0000 7f7e12dff700 1 Processor -- start 2023-12-26T21:36:52.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.206+0000 7f7e12dff700 1 -- start start 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.207+0000 7f7e12dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e0c159a70 0x7f7e0c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.207+0000 7f7e12dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e0c0b2a90 0x7f7e0c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.207+0000 7f7e12dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c0afcd0 con 0x7f7e0c1596a0 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.207+0000 7f7e12dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b3e70 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.207+0000 7f7e12dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b0f20 con 0x7f7e0c159a70 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.208+0000 7f7e1139c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e0c0b2a90 0x7f7e0c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.208+0000 7f7e1139c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e0c0b2a90 0x7f7e0c0b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56986/0 (socket says 172.21.15.131:56986) 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.208+0000 7f7e1139c700 1 -- 172.21.15.131:0/1613866443 learned_addr learned my addr 172.21.15.131:0/1613866443 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.208+0000 7f7e0bfff700 1 --2- 172.21.15.131:0/1613866443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e0c159a70 0x7f7e0c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.208+0000 7f7e1139c700 1 -- 172.21.15.131:0/1613866443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e0c159a70 msgr2=0x7f7e0c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.208+0000 7f7e1139c700 1 --2- 172.21.15.131:0/1613866443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e0c159a70 0x7f7e0c0b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.208+0000 7f7e1139c700 1 -- 172.21.15.131:0/1613866443 >> v1:172.21.15.190:6789/0 conn(0x7f7e0c1596a0 legacy=0x7f7e0c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.208+0000 7f7e1139c700 1 -- 172.21.15.131:0/1613866443 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c0b3540 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.208+0000 7f7e1139c700 1 --2- 172.21.15.131:0/1613866443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 0x7f7e0c0b06a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009f20 tx=0x7f7e04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26976c5824370452 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e0b7fe700 1 -- 172.21.15.131:0/1613866443 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0400e070 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e0b7fe700 1 -- 172.21.15.131:0/1613866443 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0400b9d0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e0b7fe700 1 -- 172.21.15.131:0/1613866443 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040092e0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e12dff700 1 -- 172.21.15.131:0/1613866443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 msgr2=0x7f7e0c0b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e12dff700 1 --2- 172.21.15.131:0/1613866443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 0x7f7e0c0b06a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009f20 tx=0x7f7e04009840 comp rx=0 tx=0).stop 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e12dff700 1 -- 172.21.15.131:0/1613866443 shutdown_connections 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e12dff700 1 --2- 172.21.15.131:0/1613866443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 0x7f7e0c0b06a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e12dff700 1 --2- 172.21.15.131:0/1613866443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e0c159a70 0x7f7e0c0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e12dff700 1 -- 172.21.15.131:0/1613866443 >> 172.21.15.131:0/1613866443 conn(0x7f7e0c06ece0 msgr2=0x7f7e0c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e12dff700 1 -- 172.21.15.131:0/1613866443 shutdown_connections 2023-12-26T21:36:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.209+0000 7f7e12dff700 1 -- 172.21.15.131:0/1613866443 wait complete. 2023-12-26T21:36:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.211+0000 7f7e12dff700 1 Processor -- start 2023-12-26T21:36:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.211+0000 7f7e12dff700 1 -- start start 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e12dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 0x7f7e0c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e12dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e0c1596a0 0x7f7e0c15a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e12dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e0c159a70 0x7f7e0c0b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e12dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c15a160 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e12dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0c7d70 con 0x7f7e0c1596a0 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e12dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c15ce50 con 0x7f7e0c159a70 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e10b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 0x7f7e0c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e10b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 0x7f7e0c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56994/0 (socket says 172.21.15.131:56994) 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e10b9b700 1 -- 172.21.15.131:0/2954832832 learned_addr learned my addr 172.21.15.131:0/2954832832 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e10b9b700 1 -- 172.21.15.131:0/2954832832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e0c159a70 msgr2=0x7f7e0c0b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e10b9b700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e0c159a70 0x7f7e0c0b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e10b9b700 1 -- 172.21.15.131:0/2954832832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e0c1596a0 msgr2=0x7f7e0c15a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e10b9b700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e0c1596a0 0x7f7e0c15a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.212+0000 7f7e10b9b700 1 -- 172.21.15.131:0/2954832832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e04009ae0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.213+0000 7f7e10b9b700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 0x7f7e0c15c910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000c590 tx=0x7f7e0000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.213+0000 7f7e08ff9700 1 -- 172.21.15.131:0/2954832832 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00011590 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.213+0000 7f7e08ff9700 1 -- 172.21.15.131:0/2954832832 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e000116f0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.213+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c16d0f0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.213+0000 7f7e08ff9700 1 -- 172.21.15.131:0/2954832832 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00011ab0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.213+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c16d6b0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.214+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c06f920 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.215+0000 7f7e08ff9700 1 -- 172.21.15.131:0/2954832832 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7e0001b3b0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.215+0000 7f7e08ff9700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7df0066f30 0x7f7df00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.215+0000 7f7e08ff9700 1 -- 172.21.15.131:0/2954832832 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e00083ae0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.216+0000 7f7e0bfff700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7df0066f30 0x7f7df00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.216+0000 7f7e0bfff700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7df0066f30 0x7f7df00693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0b5d70 tx=0x7f7df80091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:52.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.219+0000 7f7e08ff9700 1 -- 172.21.15.131:0/2954832832 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e00050d50 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.458+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e0c0657e0 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.459+0000 7f7e08ff9700 1 -- 172.21.15.131:0/2954832832 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f7e00017070 con 0x7f7e0c0b2a90 2023-12-26T21:36:52.458 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:52.458 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1127181197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1127181197},{"type":"v1","addr":"172.21.15.190:6835","nonce":1127181197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1425514628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1425514628},{"type":"v1","addr":"172.21.15.131:6837","nonce":1425514628}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3223324183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3223324183},{"type":"v1","addr":"172.21.15.190:6833","nonce":3223324183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:36:48.288910+0000","modified":"2023-12-26T21:36:51.988691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3852718531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3852718531},{"type":"v1","addr":"172.21.15.131:6836","nonce":3852718531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-12-26T21:36:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7df0066f30 msgr2=0x7f7df00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7df0066f30 0x7f7df00693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0b5d70 tx=0x7f7df80091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 msgr2=0x7f7e0c15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 0x7f7e0c15c910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000c590 tx=0x7f7e0000c950 comp rx=0 tx=0).stop 2023-12-26T21:36:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 shutdown_connections 2023-12-26T21:36:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7df0066f30 0x7f7df00693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e0c159a70 0x7f7e0c0b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e0c0b2a90 0x7f7e0c15c910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 --2- 172.21.15.131:0/2954832832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e0c1596a0 0x7f7e0c15a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 >> 172.21.15.131:0/2954832832 conn(0x7f7e0c06ece0 msgr2=0x7f7e0c151af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:52.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 shutdown_connections 2023-12-26T21:36:52.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.461+0000 7f7e12dff700 1 -- 172.21.15.131:0/2954832832 wait complete. 2023-12-26T21:36:52.462 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2023-12-26T21:36:52.479 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:52.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.667+0000 7f89623ff700 1 Processor -- start 2023-12-26T21:36:52.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.667+0000 7f89623ff700 1 -- start start 2023-12-26T21:36:52.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.668+0000 7f89623ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c13c510 0x7f895c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.668+0000 7f89623ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f895c0cc810 0x7f895c06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.668+0000 7f89623ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c06c270 con 0x7f895c06b8c0 2023-12-26T21:36:52.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.668+0000 7f89623ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895c0686c0 con 0x7f895c0cc810 2023-12-26T21:36:52.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.668+0000 7f89623ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c069520 con 0x7f895c13c510 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.668+0000 7f895bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c13c510 0x7f895c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.668+0000 7f896099c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f895c0cc810 0x7f895c06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f895b7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f895c06b8c0 0x7f895c0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:47412/0 (socket says 172.21.15.131:47412) 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f895b7fe700 1 -- 172.21.15.131:0/4180481283 learned_addr learned my addr 172.21.15.131:0/4180481283 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f896099c700 1 -- 172.21.15.131:0/4180481283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c13c510 msgr2=0x7f895c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f896099c700 1 --2- 172.21.15.131:0/4180481283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c13c510 0x7f895c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f896099c700 1 -- 172.21.15.131:0/4180481283 >> v1:172.21.15.190:6789/0 conn(0x7f895c06b8c0 legacy=0x7f895c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f896099c700 1 -- 172.21.15.131:0/4180481283 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c068910 con 0x7f895c0cc810 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f896099c700 1 --2- 172.21.15.131:0/4180481283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c0cc810 0x7f895c06bd30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8954009e60 tx=0x7f8954009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=250a69ff0940ca01 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:52.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f895affd700 1 -- 172.21.15.131:0/4180481283 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895400e070 con 0x7f895c0cc810 2023-12-26T21:36:52.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f895affd700 1 -- 172.21.15.131:0/4180481283 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895400bb60 con 0x7f895c0cc810 2023-12-26T21:36:52.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.669+0000 7f895affd700 1 -- 172.21.15.131:0/4180481283 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954012430 con 0x7f895c0cc810 2023-12-26T21:36:52.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.670+0000 7f89623ff700 1 -- 172.21.15.131:0/4180481283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c0cc810 msgr2=0x7f895c06bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.670+0000 7f89623ff700 1 --2- 172.21.15.131:0/4180481283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c0cc810 0x7f895c06bd30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8954009e60 tx=0x7f8954009780 comp rx=0 tx=0).stop 2023-12-26T21:36:52.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.670+0000 7f89623ff700 1 -- 172.21.15.131:0/4180481283 shutdown_connections 2023-12-26T21:36:52.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.670+0000 7f89623ff700 1 --2- 172.21.15.131:0/4180481283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c0cc810 0x7f895c06bd30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.670+0000 7f89623ff700 1 --2- 172.21.15.131:0/4180481283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c13c510 0x7f895c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.670+0000 7f89623ff700 1 -- 172.21.15.131:0/4180481283 >> 172.21.15.131:0/4180481283 conn(0x7f895c0ce3e0 msgr2=0x7f895c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:52.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.670+0000 7f89623ff700 1 -- 172.21.15.131:0/4180481283 shutdown_connections 2023-12-26T21:36:52.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.670+0000 7f89623ff700 1 -- 172.21.15.131:0/4180481283 wait complete. 2023-12-26T21:36:52.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.672+0000 7f89623ff700 1 Processor -- start 2023-12-26T21:36:52.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.672+0000 7f89623ff700 1 -- start start 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f89623ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06b8c0 0x7f895c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f89623ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0cc810 0x7f895c154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f89623ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f895c13c510 0x7f895c0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f89623ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c155930 con 0x7f895c13c510 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f89623ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c153d00 con 0x7f895c06b8c0 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f89623ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c154d20 con 0x7f895c0cc810 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f896099c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f895c13c510 0x7f895c0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f896099c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f895c13c510 0x7f895c0b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57002/0 (socket says 172.21.15.131:57002) 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f896099c700 1 -- 172.21.15.131:0/3874026009 learned_addr learned my addr 172.21.15.131:0/3874026009 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f895bfff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06b8c0 0x7f895c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f895b7fe700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0cc810 0x7f895c154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f896099c700 1 -- 172.21.15.131:0/3874026009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0cc810 msgr2=0x7f895c154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f896099c700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0cc810 0x7f895c154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f896099c700 1 -- 172.21.15.131:0/3874026009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06b8c0 msgr2=0x7f895c1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f896099c700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06b8c0 0x7f895c1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.673+0000 7f896099c700 1 -- 172.21.15.131:0/3874026009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954009a20 con 0x7f895c13c510 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.674+0000 7f896099c700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f895c13c510 0x7f895c0b1180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f895c06fe40 tx=0x7f8954019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.674+0000 7f8943fff700 1 -- 172.21.15.131:0/3874026009 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954027470 con 0x7f895c13c510 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.674+0000 7f8943fff700 1 -- 172.21.15.131:0/3874026009 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89540275f0 con 0x7f895c13c510 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.674+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c155000 con 0x7f895c13c510 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.674+0000 7f8943fff700 1 -- 172.21.15.131:0/3874026009 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540279b0 con 0x7f895c13c510 2023-12-26T21:36:52.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.674+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c1555c0 con 0x7f895c13c510 2023-12-26T21:36:52.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.675+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c0657e0 con 0x7f895c13c510 2023-12-26T21:36:52.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.676+0000 7f8943fff700 1 -- 172.21.15.131:0/3874026009 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8954024070 con 0x7f895c13c510 2023-12-26T21:36:52.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.677+0000 7f8943fff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8948066f30 0x7f89480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:52.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.677+0000 7f8943fff700 1 -- 172.21.15.131:0/3874026009 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8954086fe0 con 0x7f895c13c510 2023-12-26T21:36:52.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.677+0000 7f895bfff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8948066f30 0x7f89480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:52.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.677+0000 7f895bfff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8948066f30 0x7f89480693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f894c009d30 tx=0x7f894c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:52.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.681+0000 7f8943fff700 1 -- 172.21.15.131:0/3874026009 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89540541b0 con 0x7f895c13c510 2023-12-26T21:36:52.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.877+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f895c068bf0 con 0x7f895c13c510 2023-12-26T21:36:52.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.878+0000 7f8943fff700 1 -- 172.21.15.131:0/3874026009 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f89540577d0 con 0x7f895c13c510 2023-12-26T21:36:52.878 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:52.878 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:52.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8948066f30 msgr2=0x7f89480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8948066f30 0x7f89480693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f894c009d30 tx=0x7f894c008040 comp rx=0 tx=0).stop 2023-12-26T21:36:52.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f895c13c510 msgr2=0x7f895c0b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:52.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f895c13c510 0x7f895c0b1180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f895c06fe40 tx=0x7f8954019000 comp rx=0 tx=0).stop 2023-12-26T21:36:52.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 shutdown_connections 2023-12-26T21:36:52.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f895c13c510 0x7f895c0b1180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06b8c0 0x7f895c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8948066f30 0x7f89480693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 --2- 172.21.15.131:0/3874026009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0cc810 0x7f895c154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:52.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 >> 172.21.15.131:0/3874026009 conn(0x7f895c0ce3e0 msgr2=0x7f895c15eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:52.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 shutdown_connections 2023-12-26T21:36:52.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:52.880+0000 7f89623ff700 1 -- 172.21.15.131:0/3874026009 wait complete. 2023-12-26T21:36:52.898 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T21:36:52.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:36:53.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.089+0000 7ff78bfff700 1 Processor -- start 2023-12-26T21:36:53.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.089+0000 7ff78bfff700 1 -- start start 2023-12-26T21:36:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.090+0000 7ff78bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78413c510 0x7ff7840c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.090+0000 7ff78bfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7840686c0 con 0x7ff78406b8c0 2023-12-26T21:36:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.090+0000 7ff78bfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff784069520 con 0x7ff7840cc810 2023-12-26T21:36:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.090+0000 7ff78bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff784068910 con 0x7ff78413c510 2023-12-26T21:36:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.090+0000 7ff78959a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff78406b8c0 0x7ff7840cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47822/0 (socket says 172.21.15.131:47822) 2023-12-26T21:36:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.090+0000 7ff78959a700 1 -- 172.21.15.131:0/2557244620 learned_addr learned my addr 172.21.15.131:0/2557244620 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff789d9b700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78413c510 0x7ff7840c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033110381 0 0) 0x7ff7840686c0 con 0x7ff78406b8c0 2023-12-26T21:36:53.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff770003570 con 0x7ff78406b8c0 2023-12-26T21:36:53.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491409966 0 0) 0x7ff784069520 con 0x7ff7840cc810 2023-12-26T21:36:53.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7840686c0 con 0x7ff7840cc810 2023-12-26T21:36:53.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1322387818 0 0) 0x7ff770003570 con 0x7ff78406b8c0 2023-12-26T21:36:53.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78413c510 msgr2=0x7ff7840c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:53.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78413c510 0x7ff7840c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 >> v1:172.21.15.190:6789/0 conn(0x7ff7840cc810 legacy=0x7ff7840699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff784068bf0 con 0x7ff78406b8c0 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7ff774001470 con 0x7ff78406b8c0 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 >> v1:172.21.15.131:6789/0 conn(0x7ff78406b8c0 legacy=0x7ff7840cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff770003c20 0x7ff770004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78413c510 0x7ff770004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff770004d50 0x7ff7700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff784068910 con 0x7ff770003c20 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77000bb40 con 0x7ff770004d50 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77000bd00 con 0x7ff78413c510 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff789d9b700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff770003c20 0x7ff770004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.091+0000 7ff78959a700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78413c510 0x7ff770004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff78a59c700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff770004d50 0x7ff7700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff789d9b700 1 -- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78413c510 msgr2=0x7ff770004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff789d9b700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78413c510 0x7ff770004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff789d9b700 1 -- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff770004d50 msgr2=0x7ff7700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff789d9b700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff770004d50 0x7ff7700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff789d9b700 1 -- 172.21.15.131:0/2557244620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770003570 con 0x7ff770003c20 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff789d9b700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff770003c20 0x7ff770004010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff780009bf0 tx=0x7ff780009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d190a9f31acbe0b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780005a00 con 0x7ff770003c20 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff780005b60 con 0x7ff770003c20 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.092+0000 7ff788d99700 1 -- 172.21.15.131:0/2557244620 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000e710 con 0x7ff770003c20 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.093+0000 7ff78bfff700 1 -- 172.21.15.131:0/2557244620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff770003c20 msgr2=0x7ff770004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.093+0000 7ff78bfff700 1 --2- 172.21.15.131:0/2557244620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff770003c20 0x7ff770004010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff780009bf0 tx=0x7ff780009120 comp rx=0 tx=0).stop 2023-12-26T21:36:53.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.093+0000 7ff78a59c700 1 -- 172.21.15.131:0/2557244620 reap_dead start 2023-12-26T21:36:53.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.093+0000 7ff78bfff700 1 -- 172.21.15.131:0/2557244620 shutdown_connections 2023-12-26T21:36:53.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.093+0000 7ff78bfff700 1 -- 172.21.15.131:0/2557244620 >> 172.21.15.131:0/2557244620 conn(0x7ff7840ce3e0 msgr2=0x7ff7840ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:53.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.093+0000 7ff78bfff700 1 -- 172.21.15.131:0/2557244620 shutdown_connections 2023-12-26T21:36:53.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.093+0000 7ff78bfff700 1 -- 172.21.15.131:0/2557244620 wait complete. 2023-12-26T21:36:53.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.094+0000 7ff78bfff700 1 Processor -- start 2023-12-26T21:36:53.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.094+0000 7ff78bfff700 1 -- start start 2023-12-26T21:36:53.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff770003c20 0x7ff78413bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78415b500 0x7ff78415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7841552c0 0x7ff784155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78bfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840b2eb0 con 0x7ff7841552c0 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78bfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff784154e30 con 0x7ff770003c20 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff784154500 con 0x7ff78415b500 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7841552c0 0x7ff784155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7841552c0 0x7ff784155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57026/0 (socket says 172.21.15.131:57026) 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78a59c700 1 -- 172.21.15.131:0/3634692272 learned_addr learned my addr 172.21.15.131:0/3634692272 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff789d9b700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff770003c20 0x7ff78413bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.095+0000 7ff78959a700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78415b500 0x7ff78415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff78a59c700 1 -- 172.21.15.131:0/3634692272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78415b500 msgr2=0x7ff78415b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff78a59c700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78415b500 0x7ff78415b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff78a59c700 1 -- 172.21.15.131:0/3634692272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff770003c20 msgr2=0x7ff78413bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff78a59c700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff770003c20 0x7ff78413bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff78a59c700 1 -- 172.21.15.131:0/3634692272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7841547e0 con 0x7ff7841552c0 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff78a59c700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7841552c0 0x7ff784155730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff78406fcc0 tx=0x7ff77c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:53.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff779ffb700 1 -- 172.21.15.131:0/3634692272 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c017070 con 0x7ff7841552c0 2023-12-26T21:36:53.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff779ffb700 1 -- 172.21.15.131:0/3634692272 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c006ba0 con 0x7ff7841552c0 2023-12-26T21:36:53.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff779ffb700 1 -- 172.21.15.131:0/3634692272 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00c820 con 0x7ff7841552c0 2023-12-26T21:36:53.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff784154ac0 con 0x7ff7841552c0 2023-12-26T21:36:53.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.096+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff784159eb0 con 0x7ff7841552c0 2023-12-26T21:36:53.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.097+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff784154fc0 con 0x7ff7841552c0 2023-12-26T21:36:53.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.099+0000 7ff779ffb700 1 -- 172.21.15.131:0/3634692272 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff77c00cb90 con 0x7ff7841552c0 2023-12-26T21:36:53.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.099+0000 7ff779ffb700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff768066f60 0x7ff768069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.100+0000 7ff779ffb700 1 -- 172.21.15.131:0/3634692272 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff77c0840d0 con 0x7ff7841552c0 2023-12-26T21:36:53.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.100+0000 7ff789d9b700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff768066f60 0x7ff768069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.100+0000 7ff789d9b700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff768066f60 0x7ff768069410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff780009160 tx=0x7ff780019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:53.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.104+0000 7ff779ffb700 1 -- 172.21.15.131:0/3634692272 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff77c0512a0 con 0x7ff7841552c0 2023-12-26T21:36:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.295+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7840657e0 con 0x7ff768066f60 2023-12-26T21:36:53.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.304+0000 7ff779ffb700 1 -- 172.21.15.131:0/3634692272 <== mgr.4100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264605 (secure 0 0 0) 0x7ff7840657e0 con 0x7ff768066f60 2023-12-26T21:36:53.308 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:53.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.308+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff768066f60 msgr2=0x7ff768069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff768066f60 0x7ff768069410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff780009160 tx=0x7ff780019000 comp rx=0 tx=0).stop 2023-12-26T21:36:53.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7841552c0 msgr2=0x7ff784155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7841552c0 0x7ff784155730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff78406fcc0 tx=0x7ff77c0017a0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 shutdown_connections 2023-12-26T21:36:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7841552c0 0x7ff784155730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff770003c20 0x7ff78413bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff768066f60 0x7ff768069410 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 --2- 172.21.15.131:0/3634692272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff78415b500 0x7ff78415b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 >> 172.21.15.131:0/3634692272 conn(0x7ff7840ce3e0 msgr2=0x7ff784153660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 shutdown_connections 2023-12-26T21:36:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.309+0000 7ff78bfff700 1 -- 172.21.15.131:0/3634692272 wait complete. 2023-12-26T21:36:53.311 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:36:53.327 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-26T21:36:52.998659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381256,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593464,"statfs":{"total":773094113280,"available":770655707136,"internally_reserved":0,"allocated":3022848,"data_stored":1800548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001048"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845569+0000","last_change":"2023-12-26T21:36:48.838074+0000","last_active":"2023-12-26T21:36:48.845569+0000","last_peered":"2023-12-26T21:36:48.845569+0000","last_clean":"2023-12-26T21:36:48.845569+0000","last_became_active":"2023-12-26T21:36:48.837495+0000","last_became_peered":"2023-12-26T21:36:48.837495+0000","last_unstale":"2023-12-26T21:36:48.845569+0000","last_undegraded":"2023-12-26T21:36:48.845569+0000","last_fullsized":"2023-12-26T21:36:48.845569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841349+0000","last_change":"2023-12-26T21:36:48.830149+0000","last_active":"2023-12-26T21:36:48.841349+0000","last_peered":"2023-12-26T21:36:48.841349+0000","last_clean":"2023-12-26T21:36:48.841349+0000","last_became_active":"2023-12-26T21:36:48.829771+0000","last_became_peered":"2023-12-26T21:36:48.829771+0000","last_unstale":"2023-12-26T21:36:48.841349+0000","last_undegraded":"2023-12-26T21:36:48.841349+0000","last_fullsized":"2023-12-26T21:36:48.841349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:59.578493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841417+0000","last_change":"2023-12-26T21:36:48.831124+0000","last_active":"2023-12-26T21:36:48.841417+0000","last_peered":"2023-12-26T21:36:48.841417+0000","last_clean":"2023-12-26T21:36:48.841417+0000","last_became_active":"2023-12-26T21:36:48.830817+0000","last_became_peered":"2023-12-26T21:36:48.830817+0000","last_unstale":"2023-12-26T21:36:48.841417+0000","last_undegraded":"2023-12-26T21:36:48.841417+0000","last_fullsized":"2023-12-26T21:36:48.841417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:47.183606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845126+0000","last_change":"2023-12-26T21:36:48.842229+0000","last_active":"2023-12-26T21:36:48.845126+0000","last_peered":"2023-12-26T21:36:48.845126+0000","last_clean":"2023-12-26T21:36:48.845126+0000","last_became_active":"2023-12-26T21:36:48.839269+0000","last_became_peered":"2023-12-26T21:36:48.839269+0000","last_unstale":"2023-12-26T21:36:48.845126+0000","last_undegraded":"2023-12-26T21:36:48.845126+0000","last_fullsized":"2023-12-26T21:36:48.845126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:01:49.095423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845558+0000","last_change":"2023-12-26T21:36:48.838113+0000","last_active":"2023-12-26T21:36:48.845558+0000","last_peered":"2023-12-26T21:36:48.845558+0000","last_clean":"2023-12-26T21:36:48.845558+0000","last_became_active":"2023-12-26T21:36:48.837882+0000","last_became_peered":"2023-12-26T21:36:48.837882+0000","last_unstale":"2023-12-26T21:36:48.845558+0000","last_undegraded":"2023-12-26T21:36:48.845558+0000","last_fullsized":"2023-12-26T21:36:48.845558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:40.692719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839762+0000","last_change":"2023-12-26T21:36:48.835731+0000","last_active":"2023-12-26T21:36:48.839762+0000","last_peered":"2023-12-26T21:36:48.839762+0000","last_clean":"2023-12-26T21:36:48.839762+0000","last_became_active":"2023-12-26T21:36:48.830672+0000","last_became_peered":"2023-12-26T21:36:48.830672+0000","last_unstale":"2023-12-26T21:36:48.839762+0000","last_undegraded":"2023-12-26T21:36:48.839762+0000","last_fullsized":"2023-12-26T21:36:48.839762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:06:58.960255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835557+0000","last_change":"2023-12-26T21:36:48.831504+0000","last_active":"2023-12-26T21:36:48.835557+0000","last_peered":"2023-12-26T21:36:48.835557+0000","last_clean":"2023-12-26T21:36:48.835557+0000","last_became_active":"2023-12-26T21:36:48.831173+0000","last_became_peered":"2023-12-26T21:36:48.831173+0000","last_unstale":"2023-12-26T21:36:48.835557+0000","last_undegraded":"2023-12-26T21:36:48.835557+0000","last_fullsized":"2023-12-26T21:36:48.835557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:37:13.353768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841944+0000","last_change":"2023-12-26T21:36:48.838249+0000","last_active":"2023-12-26T21:36:48.841944+0000","last_peered":"2023-12-26T21:36:48.841944+0000","last_clean":"2023-12-26T21:36:48.841944+0000","last_became_active":"2023-12-26T21:36:48.837983+0000","last_became_peered":"2023-12-26T21:36:48.837983+0000","last_unstale":"2023-12-26T21:36:48.841944+0000","last_undegraded":"2023-12-26T21:36:48.841944+0000","last_fullsized":"2023-12-26T21:36:48.841944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:56.706835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831690+0000","last_change":"2023-12-26T21:36:48.829328+0000","last_active":"2023-12-26T21:36:48.831690+0000","last_peered":"2023-12-26T21:36:48.831690+0000","last_clean":"2023-12-26T21:36:48.831690+0000","last_became_active":"2023-12-26T21:36:48.829033+0000","last_became_peered":"2023-12-26T21:36:48.829033+0000","last_unstale":"2023-12-26T21:36:48.831690+0000","last_undegraded":"2023-12-26T21:36:48.831690+0000","last_fullsized":"2023-12-26T21:36:48.831690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:31.058793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.837727+0000","last_change":"2023-12-26T21:36:48.834507+0000","last_active":"2023-12-26T21:36:48.837727+0000","last_peered":"2023-12-26T21:36:48.837727+0000","last_clean":"2023-12-26T21:36:48.837727+0000","last_became_active":"2023-12-26T21:36:48.833751+0000","last_became_peered":"2023-12-26T21:36:48.833751+0000","last_unstale":"2023-12-26T21:36:48.837727+0000","last_undegraded":"2023-12-26T21:36:48.837727+0000","last_fullsized":"2023-12-26T21:36:48.837727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:23.349206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846618+0000","last_change":"2023-12-26T21:36:48.839909+0000","last_active":"2023-12-26T21:36:48.846618+0000","last_peered":"2023-12-26T21:36:48.846618+0000","last_clean":"2023-12-26T21:36:48.846618+0000","last_became_active":"2023-12-26T21:36:48.839611+0000","last_became_peered":"2023-12-26T21:36:48.839611+0000","last_unstale":"2023-12-26T21:36:48.846618+0000","last_undegraded":"2023-12-26T21:36:48.846618+0000","last_fullsized":"2023-12-26T21:36:48.846618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:49.385696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841298+0000","last_change":"2023-12-26T21:36:48.836176+0000","last_active":"2023-12-26T21:36:48.841298+0000","last_peered":"2023-12-26T21:36:48.841298+0000","last_clean":"2023-12-26T21:36:48.841298+0000","last_became_active":"2023-12-26T21:36:48.835911+0000","last_became_peered":"2023-12-26T21:36:48.835911+0000","last_unstale":"2023-12-26T21:36:48.841298+0000","last_undegraded":"2023-12-26T21:36:48.841298+0000","last_fullsized":"2023-12-26T21:36:48.841298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:32.390648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828384+0000","last_change":"2023-12-26T21:36:48.825560+0000","last_active":"2023-12-26T21:36:48.828384+0000","last_peered":"2023-12-26T21:36:48.828384+0000","last_clean":"2023-12-26T21:36:48.828384+0000","last_became_active":"2023-12-26T21:36:48.825203+0000","last_became_peered":"2023-12-26T21:36:48.825203+0000","last_unstale":"2023-12-26T21:36:48.828384+0000","last_undegraded":"2023-12-26T21:36:48.828384+0000","last_fullsized":"2023-12-26T21:36:48.828384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:24.061374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828296+0000","last_change":"2023-12-26T21:36:48.823931+0000","last_active":"2023-12-26T21:36:48.828296+0000","last_peered":"2023-12-26T21:36:48.828296+0000","last_clean":"2023-12-26T21:36:48.828296+0000","last_became_active":"2023-12-26T21:36:48.823564+0000","last_became_peered":"2023-12-26T21:36:48.823564+0000","last_unstale":"2023-12-26T21:36:48.828296+0000","last_undegraded":"2023-12-26T21:36:48.828296+0000","last_fullsized":"2023-12-26T21:36:48.828296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:42.053931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833245+0000","last_change":"2023-12-26T21:36:48.823836+0000","last_active":"2023-12-26T21:36:48.833245+0000","last_peered":"2023-12-26T21:36:48.833245+0000","last_clean":"2023-12-26T21:36:48.833245+0000","last_became_active":"2023-12-26T21:36:48.823053+0000","last_became_peered":"2023-12-26T21:36:48.823053+0000","last_unstale":"2023-12-26T21:36:48.833245+0000","last_undegraded":"2023-12-26T21:36:48.833245+0000","last_fullsized":"2023-12-26T21:36:48.833245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:28.883031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838446+0000","last_change":"2023-12-26T21:36:48.829032+0000","last_active":"2023-12-26T21:36:48.838446+0000","last_peered":"2023-12-26T21:36:48.838446+0000","last_clean":"2023-12-26T21:36:48.838446+0000","last_became_active":"2023-12-26T21:36:48.826097+0000","last_became_peered":"2023-12-26T21:36:48.826097+0000","last_unstale":"2023-12-26T21:36:48.838446+0000","last_undegraded":"2023-12-26T21:36:48.838446+0000","last_fullsized":"2023-12-26T21:36:48.838446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:49.385696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844638+0000","last_change":"2023-12-26T21:36:48.832781+0000","last_active":"2023-12-26T21:36:48.844638+0000","last_peered":"2023-12-26T21:36:48.844638+0000","last_clean":"2023-12-26T21:36:48.844638+0000","last_became_active":"2023-12-26T21:36:48.832414+0000","last_became_peered":"2023-12-26T21:36:48.832414+0000","last_unstale":"2023-12-26T21:36:48.844638+0000","last_undegraded":"2023-12-26T21:36:48.844638+0000","last_fullsized":"2023-12-26T21:36:48.844638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:05:36.844680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.825827+0000","last_change":"2023-12-26T21:36:48.817787+0000","last_active":"2023-12-26T21:36:48.825827+0000","last_peered":"2023-12-26T21:36:48.825827+0000","last_clean":"2023-12-26T21:36:48.825827+0000","last_became_active":"2023-12-26T21:36:48.817432+0000","last_became_peered":"2023-12-26T21:36:48.817432+0000","last_unstale":"2023-12-26T21:36:48.825827+0000","last_undegraded":"2023-12-26T21:36:48.825827+0000","last_fullsized":"2023-12-26T21:36:48.825827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:51.402901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844696+0000","last_change":"2023-12-26T21:36:48.834659+0000","last_active":"2023-12-26T21:36:48.844696+0000","last_peered":"2023-12-26T21:36:48.844696+0000","last_clean":"2023-12-26T21:36:48.844696+0000","last_became_active":"2023-12-26T21:36:48.833967+0000","last_became_peered":"2023-12-26T21:36:48.833967+0000","last_unstale":"2023-12-26T21:36:48.844696+0000","last_undegraded":"2023-12-26T21:36:48.844696+0000","last_fullsized":"2023-12-26T21:36:48.844696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:12:41.152256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839315+0000","last_change":"2023-12-26T21:36:47.830031+0000","last_active":"2023-12-26T21:36:48.839315+0000","last_peered":"2023-12-26T21:36:48.839315+0000","last_clean":"2023-12-26T21:36:48.839315+0000","last_became_active":"2023-12-26T21:36:47.829681+0000","last_became_peered":"2023-12-26T21:36:47.829681+0000","last_unstale":"2023-12-26T21:36:48.839315+0000","last_undegraded":"2023-12-26T21:36:48.839315+0000","last_fullsized":"2023-12-26T21:36:48.839315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:25.200664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841270+0000","last_change":"2023-12-26T21:36:48.827382+0000","last_active":"2023-12-26T21:36:48.841270+0000","last_peered":"2023-12-26T21:36:48.841270+0000","last_clean":"2023-12-26T21:36:48.841270+0000","last_became_active":"2023-12-26T21:36:48.825125+0000","last_became_peered":"2023-12-26T21:36:48.825125+0000","last_unstale":"2023-12-26T21:36:48.841270+0000","last_undegraded":"2023-12-26T21:36:48.841270+0000","last_fullsized":"2023-12-26T21:36:48.841270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:02:27.671653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843042+0000","last_change":"2023-12-26T21:36:47.825862+0000","last_active":"2023-12-26T21:36:48.843042+0000","last_peered":"2023-12-26T21:36:48.843042+0000","last_clean":"2023-12-26T21:36:48.843042+0000","last_became_active":"2023-12-26T21:36:47.825603+0000","last_became_peered":"2023-12-26T21:36:47.825603+0000","last_unstale":"2023-12-26T21:36:48.843042+0000","last_undegraded":"2023-12-26T21:36:48.843042+0000","last_fullsized":"2023-12-26T21:36:48.843042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:18:58.593197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833411+0000","last_change":"2023-12-26T21:36:48.819527+0000","last_active":"2023-12-26T21:36:48.833411+0000","last_peered":"2023-12-26T21:36:48.833411+0000","last_clean":"2023-12-26T21:36:48.833411+0000","last_became_active":"2023-12-26T21:36:48.819208+0000","last_became_peered":"2023-12-26T21:36:48.819208+0000","last_unstale":"2023-12-26T21:36:48.833411+0000","last_undegraded":"2023-12-26T21:36:48.833411+0000","last_fullsized":"2023-12-26T21:36:48.833411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:32.390648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841778+0000","last_change":"2023-12-26T21:36:47.824557+0000","last_active":"2023-12-26T21:36:48.841778+0000","last_peered":"2023-12-26T21:36:48.841778+0000","last_clean":"2023-12-26T21:36:48.841778+0000","last_became_active":"2023-12-26T21:36:47.824200+0000","last_became_peered":"2023-12-26T21:36:47.824200+0000","last_unstale":"2023-12-26T21:36:48.841778+0000","last_undegraded":"2023-12-26T21:36:48.841778+0000","last_fullsized":"2023-12-26T21:36:48.841778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:13.097046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.827346+0000","last_change":"2023-12-26T21:36:48.823425+0000","last_active":"2023-12-26T21:36:48.827346+0000","last_peered":"2023-12-26T21:36:48.827346+0000","last_clean":"2023-12-26T21:36:48.827346+0000","last_became_active":"2023-12-26T21:36:48.823148+0000","last_became_peered":"2023-12-26T21:36:48.823148+0000","last_unstale":"2023-12-26T21:36:48.827346+0000","last_undegraded":"2023-12-26T21:36:48.827346+0000","last_fullsized":"2023-12-26T21:36:48.827346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:35.691705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841375+0000","last_change":"2023-12-26T21:36:47.826954+0000","last_active":"2023-12-26T21:36:48.841375+0000","last_peered":"2023-12-26T21:36:48.841375+0000","last_clean":"2023-12-26T21:36:48.841375+0000","last_became_active":"2023-12-26T21:36:47.826691+0000","last_became_peered":"2023-12-26T21:36:47.826691+0000","last_unstale":"2023-12-26T21:36:48.841375+0000","last_undegraded":"2023-12-26T21:36:48.841375+0000","last_fullsized":"2023-12-26T21:36:48.841375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:24.145739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843898+0000","last_change":"2023-12-26T21:36:48.830893+0000","last_active":"2023-12-26T21:36:48.843898+0000","last_peered":"2023-12-26T21:36:48.843898+0000","last_clean":"2023-12-26T21:36:48.843898+0000","last_became_active":"2023-12-26T21:36:48.830553+0000","last_became_peered":"2023-12-26T21:36:48.830553+0000","last_unstale":"2023-12-26T21:36:48.843898+0000","last_undegraded":"2023-12-26T21:36:48.843898+0000","last_fullsized":"2023-12-26T21:36:48.843898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:40:07.698471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841849+0000","last_change":"2023-12-26T21:36:47.828216+0000","last_active":"2023-12-26T21:36:48.841849+0000","last_peered":"2023-12-26T21:36:48.841849+0000","last_clean":"2023-12-26T21:36:48.841849+0000","last_became_active":"2023-12-26T21:36:47.827950+0000","last_became_peered":"2023-12-26T21:36:47.827950+0000","last_unstale":"2023-12-26T21:36:48.841849+0000","last_undegraded":"2023-12-26T21:36:48.841849+0000","last_fullsized":"2023-12-26T21:36:48.841849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:23.069341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828328+0000","last_change":"2023-12-26T21:36:48.820490+0000","last_active":"2023-12-26T21:36:48.828328+0000","last_peered":"2023-12-26T21:36:48.828328+0000","last_clean":"2023-12-26T21:36:48.828328+0000","last_became_active":"2023-12-26T21:36:48.820298+0000","last_became_peered":"2023-12-26T21:36:48.820298+0000","last_unstale":"2023-12-26T21:36:48.828328+0000","last_undegraded":"2023-12-26T21:36:48.828328+0000","last_fullsized":"2023-12-26T21:36:48.828328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:25.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.830175+0000","last_change":"2023-12-26T21:36:47.830245+0000","last_active":"2023-12-26T21:36:48.830175+0000","last_peered":"2023-12-26T21:36:48.830175+0000","last_clean":"2023-12-26T21:36:48.830175+0000","last_became_active":"2023-12-26T21:36:47.829970+0000","last_became_peered":"2023-12-26T21:36:47.829970+0000","last_unstale":"2023-12-26T21:36:48.830175+0000","last_undegraded":"2023-12-26T21:36:48.830175+0000","last_fullsized":"2023-12-26T21:36:48.830175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:13.329724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843818+0000","last_change":"2023-12-26T21:36:48.839949+0000","last_active":"2023-12-26T21:36:48.843818+0000","last_peered":"2023-12-26T21:36:48.843818+0000","last_clean":"2023-12-26T21:36:48.843818+0000","last_became_active":"2023-12-26T21:36:48.839655+0000","last_became_peered":"2023-12-26T21:36:48.839655+0000","last_unstale":"2023-12-26T21:36:48.843818+0000","last_undegraded":"2023-12-26T21:36:48.843818+0000","last_fullsized":"2023-12-26T21:36:48.843818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:43:57.839515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.809369+0000","last_change":"2023-12-26T21:36:47.828451+0000","last_active":"2023-12-26T21:36:48.809369+0000","last_peered":"2023-12-26T21:36:48.809369+0000","last_clean":"2023-12-26T21:36:48.809369+0000","last_became_active":"2023-12-26T21:36:47.828158+0000","last_became_peered":"2023-12-26T21:36:47.828158+0000","last_unstale":"2023-12-26T21:36:48.809369+0000","last_undegraded":"2023-12-26T21:36:48.809369+0000","last_fullsized":"2023-12-26T21:36:48.809369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:23.069341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843628+0000","last_change":"2023-12-26T21:36:48.840894+0000","last_active":"2023-12-26T21:36:48.843628+0000","last_peered":"2023-12-26T21:36:48.843628+0000","last_clean":"2023-12-26T21:36:48.843628+0000","last_became_active":"2023-12-26T21:36:48.840284+0000","last_became_peered":"2023-12-26T21:36:48.840284+0000","last_unstale":"2023-12-26T21:36:48.843628+0000","last_undegraded":"2023-12-26T21:36:48.843628+0000","last_fullsized":"2023-12-26T21:36:48.843628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:23.349206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816543+0000","last_change":"2023-12-26T21:36:47.821233+0000","last_active":"2023-12-26T21:36:48.816543+0000","last_peered":"2023-12-26T21:36:48.816543+0000","last_clean":"2023-12-26T21:36:48.816543+0000","last_became_active":"2023-12-26T21:36:47.820129+0000","last_became_peered":"2023-12-26T21:36:47.820129+0000","last_unstale":"2023-12-26T21:36:48.816543+0000","last_undegraded":"2023-12-26T21:36:48.816543+0000","last_fullsized":"2023-12-26T21:36:48.816543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:30.052367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838583+0000","last_change":"2023-12-26T21:36:48.829638+0000","last_active":"2023-12-26T21:36:48.838583+0000","last_peered":"2023-12-26T21:36:48.838583+0000","last_clean":"2023-12-26T21:36:48.838583+0000","last_became_active":"2023-12-26T21:36:48.829154+0000","last_became_peered":"2023-12-26T21:36:48.829154+0000","last_unstale":"2023-12-26T21:36:48.838583+0000","last_undegraded":"2023-12-26T21:36:48.838583+0000","last_fullsized":"2023-12-26T21:36:48.838583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:45:55.685017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.803440+0000","last_change":"2023-12-26T21:36:47.818993+0000","last_active":"2023-12-26T21:36:48.803440+0000","last_peered":"2023-12-26T21:36:48.803440+0000","last_clean":"2023-12-26T21:36:48.803440+0000","last_became_active":"2023-12-26T21:36:47.818731+0000","last_became_peered":"2023-12-26T21:36:47.818731+0000","last_unstale":"2023-12-26T21:36:48.803440+0000","last_undegraded":"2023-12-26T21:36:48.803440+0000","last_fullsized":"2023-12-26T21:36:48.803440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:14.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.840833+0000","last_change":"2023-12-26T21:36:48.837014+0000","last_active":"2023-12-26T21:36:48.840833+0000","last_peered":"2023-12-26T21:36:48.840833+0000","last_clean":"2023-12-26T21:36:48.840833+0000","last_became_active":"2023-12-26T21:36:48.830649+0000","last_became_peered":"2023-12-26T21:36:48.830649+0000","last_unstale":"2023-12-26T21:36:48.840833+0000","last_undegraded":"2023-12-26T21:36:48.840833+0000","last_fullsized":"2023-12-26T21:36:48.840833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:07.755765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829625+0000","last_change":"2023-12-26T21:36:47.827036+0000","last_active":"2023-12-26T21:36:48.829625+0000","last_peered":"2023-12-26T21:36:48.829625+0000","last_clean":"2023-12-26T21:36:48.829625+0000","last_became_active":"2023-12-26T21:36:47.826746+0000","last_became_peered":"2023-12-26T21:36:47.826746+0000","last_unstale":"2023-12-26T21:36:48.829625+0000","last_undegraded":"2023-12-26T21:36:48.829625+0000","last_fullsized":"2023-12-26T21:36:48.829625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:28.026360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841896+0000","last_change":"2023-12-26T21:36:48.837290+0000","last_active":"2023-12-26T21:36:48.841896+0000","last_peered":"2023-12-26T21:36:48.841896+0000","last_clean":"2023-12-26T21:36:48.841896+0000","last_became_active":"2023-12-26T21:36:48.836991+0000","last_became_peered":"2023-12-26T21:36:48.836991+0000","last_unstale":"2023-12-26T21:36:48.841896+0000","last_undegraded":"2023-12-26T21:36:48.841896+0000","last_fullsized":"2023-12-26T21:36:48.841896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835423+0000","last_change":"2023-12-26T21:36:47.823376+0000","last_active":"2023-12-26T21:36:48.835423+0000","last_peered":"2023-12-26T21:36:48.835423+0000","last_clean":"2023-12-26T21:36:48.835423+0000","last_became_active":"2023-12-26T21:36:47.822986+0000","last_became_peered":"2023-12-26T21:36:47.822986+0000","last_unstale":"2023-12-26T21:36:48.835423+0000","last_undegraded":"2023-12-26T21:36:48.835423+0000","last_fullsized":"2023-12-26T21:36:48.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:45:26.314194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829780+0000","last_change":"2023-12-26T21:36:48.825556+0000","last_active":"2023-12-26T21:36:48.829780+0000","last_peered":"2023-12-26T21:36:48.829780+0000","last_clean":"2023-12-26T21:36:48.829780+0000","last_became_active":"2023-12-26T21:36:48.825233+0000","last_became_peered":"2023-12-26T21:36:48.825233+0000","last_unstale":"2023-12-26T21:36:48.829780+0000","last_undegraded":"2023-12-26T21:36:48.829780+0000","last_fullsized":"2023-12-26T21:36:48.829780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:40.692719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.848461+0000","last_change":"2023-12-26T21:36:47.821098+0000","last_active":"2023-12-26T21:36:48.848461+0000","last_peered":"2023-12-26T21:36:48.848461+0000","last_clean":"2023-12-26T21:36:48.848461+0000","last_became_active":"2023-12-26T21:36:47.820639+0000","last_became_peered":"2023-12-26T21:36:47.820639+0000","last_unstale":"2023-12-26T21:36:48.848461+0000","last_undegraded":"2023-12-26T21:36:48.848461+0000","last_fullsized":"2023-12-26T21:36:48.848461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:26.330467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833737+0000","last_change":"2023-12-26T21:36:48.828635+0000","last_active":"2023-12-26T21:36:48.833737+0000","last_peered":"2023-12-26T21:36:48.833737+0000","last_clean":"2023-12-26T21:36:48.833737+0000","last_became_active":"2023-12-26T21:36:48.828382+0000","last_became_peered":"2023-12-26T21:36:48.828382+0000","last_unstale":"2023-12-26T21:36:48.833737+0000","last_undegraded":"2023-12-26T21:36:48.833737+0000","last_fullsized":"2023-12-26T21:36:48.833737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:59.578493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835877+0000","last_change":"2023-12-26T21:36:47.830348+0000","last_active":"2023-12-26T21:36:48.835877+0000","last_peered":"2023-12-26T21:36:48.835877+0000","last_clean":"2023-12-26T21:36:48.835877+0000","last_became_active":"2023-12-26T21:36:47.830053+0000","last_became_peered":"2023-12-26T21:36:47.830053+0000","last_unstale":"2023-12-26T21:36:48.835877+0000","last_undegraded":"2023-12-26T21:36:48.835877+0000","last_fullsized":"2023-12-26T21:36:48.835877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:26.330467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.832805+0000","last_change":"2023-12-26T21:36:48.830118+0000","last_active":"2023-12-26T21:36:48.832805+0000","last_peered":"2023-12-26T21:36:48.832805+0000","last_clean":"2023-12-26T21:36:48.832805+0000","last_became_active":"2023-12-26T21:36:48.829843+0000","last_became_peered":"2023-12-26T21:36:48.829843+0000","last_unstale":"2023-12-26T21:36:48.832805+0000","last_undegraded":"2023-12-26T21:36:48.832805+0000","last_fullsized":"2023-12-26T21:36:48.832805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817442+0000","last_change":"2023-12-26T21:36:47.822624+0000","last_active":"2023-12-26T21:36:48.817442+0000","last_peered":"2023-12-26T21:36:48.817442+0000","last_clean":"2023-12-26T21:36:48.817442+0000","last_became_active":"2023-12-26T21:36:47.822048+0000","last_became_peered":"2023-12-26T21:36:47.822048+0000","last_unstale":"2023-12-26T21:36:48.817442+0000","last_undegraded":"2023-12-26T21:36:48.817442+0000","last_fullsized":"2023-12-26T21:36:48.817442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:24.145739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846368+0000","last_change":"2023-12-26T21:36:48.837022+0000","last_active":"2023-12-26T21:36:48.846368+0000","last_peered":"2023-12-26T21:36:48.846368+0000","last_clean":"2023-12-26T21:36:48.846368+0000","last_became_active":"2023-12-26T21:36:48.836736+0000","last_became_peered":"2023-12-26T21:36:48.836736+0000","last_unstale":"2023-12-26T21:36:48.846368+0000","last_undegraded":"2023-12-26T21:36:48.846368+0000","last_fullsized":"2023-12-26T21:36:48.846368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:57.196069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826918+0000","last_change":"2023-12-26T21:36:47.829979+0000","last_active":"2023-12-26T21:36:48.826918+0000","last_peered":"2023-12-26T21:36:48.826918+0000","last_clean":"2023-12-26T21:36:48.826918+0000","last_became_active":"2023-12-26T21:36:47.829685+0000","last_became_peered":"2023-12-26T21:36:47.829685+0000","last_unstale":"2023-12-26T21:36:48.826918+0000","last_undegraded":"2023-12-26T21:36:48.826918+0000","last_fullsized":"2023-12-26T21:36:48.826918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831660+0000","last_change":"2023-12-26T21:36:48.828008+0000","last_active":"2023-12-26T21:36:48.831660+0000","last_peered":"2023-12-26T21:36:48.831660+0000","last_clean":"2023-12-26T21:36:48.831660+0000","last_became_active":"2023-12-26T21:36:48.827743+0000","last_became_peered":"2023-12-26T21:36:48.827743+0000","last_unstale":"2023-12-26T21:36:48.831660+0000","last_undegraded":"2023-12-26T21:36:48.831660+0000","last_fullsized":"2023-12-26T21:36:48.831660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:14.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839801+0000","last_change":"2023-12-26T21:36:47.815814+0000","last_active":"2023-12-26T21:36:48.839801+0000","last_peered":"2023-12-26T21:36:48.839801+0000","last_clean":"2023-12-26T21:36:48.839801+0000","last_became_active":"2023-12-26T21:36:47.815480+0000","last_became_peered":"2023-12-26T21:36:47.815480+0000","last_unstale":"2023-12-26T21:36:48.839801+0000","last_undegraded":"2023-12-26T21:36:48.839801+0000","last_fullsized":"2023-12-26T21:36:48.839801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:18:28.266978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843897+0000","last_change":"2023-12-26T21:36:48.833364+0000","last_active":"2023-12-26T21:36:48.843897+0000","last_peered":"2023-12-26T21:36:48.843897+0000","last_clean":"2023-12-26T21:36:48.843897+0000","last_became_active":"2023-12-26T21:36:48.833087+0000","last_became_peered":"2023-12-26T21:36:48.833087+0000","last_unstale":"2023-12-26T21:36:48.843897+0000","last_undegraded":"2023-12-26T21:36:48.843897+0000","last_fullsized":"2023-12-26T21:36:48.843897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:05:49.639640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846496+0000","last_change":"2023-12-26T21:36:47.823499+0000","last_active":"2023-12-26T21:36:48.846496+0000","last_peered":"2023-12-26T21:36:48.846496+0000","last_clean":"2023-12-26T21:36:48.846496+0000","last_became_active":"2023-12-26T21:36:47.819416+0000","last_became_peered":"2023-12-26T21:36:47.819416+0000","last_unstale":"2023-12-26T21:36:48.846496+0000","last_undegraded":"2023-12-26T21:36:48.846496+0000","last_fullsized":"2023-12-26T21:36:48.846496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:13.329724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839537+0000","last_change":"2023-12-26T21:36:48.824260+0000","last_active":"2023-12-26T21:36:48.839537+0000","last_peered":"2023-12-26T21:36:48.839537+0000","last_clean":"2023-12-26T21:36:48.839537+0000","last_became_active":"2023-12-26T21:36:48.824000+0000","last_became_peered":"2023-12-26T21:36:48.824000+0000","last_unstale":"2023-12-26T21:36:48.839537+0000","last_undegraded":"2023-12-26T21:36:48.839537+0000","last_fullsized":"2023-12-26T21:36:48.839537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:24.151016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.812438+0000","last_change":"2023-12-26T21:36:47.829031+0000","last_active":"2023-12-26T21:36:48.812438+0000","last_peered":"2023-12-26T21:36:48.812438+0000","last_clean":"2023-12-26T21:36:48.812438+0000","last_became_active":"2023-12-26T21:36:47.828711+0000","last_became_peered":"2023-12-26T21:36:47.828711+0000","last_unstale":"2023-12-26T21:36:48.812438+0000","last_undegraded":"2023-12-26T21:36:48.812438+0000","last_fullsized":"2023-12-26T21:36:48.812438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.822529+0000","last_change":"2023-12-26T21:36:48.817001+0000","last_active":"2023-12-26T21:36:48.822529+0000","last_peered":"2023-12-26T21:36:48.822529+0000","last_clean":"2023-12-26T21:36:48.822529+0000","last_became_active":"2023-12-26T21:36:48.816746+0000","last_became_peered":"2023-12-26T21:36:48.816746+0000","last_unstale":"2023-12-26T21:36:48.822529+0000","last_undegraded":"2023-12-26T21:36:48.822529+0000","last_fullsized":"2023-12-26T21:36:48.822529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:56.619479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817332+0000","last_change":"2023-12-26T21:36:47.828501+0000","last_active":"2023-12-26T21:36:48.817332+0000","last_peered":"2023-12-26T21:36:48.817332+0000","last_clean":"2023-12-26T21:36:48.817332+0000","last_became_active":"2023-12-26T21:36:47.828257+0000","last_became_peered":"2023-12-26T21:36:47.828257+0000","last_unstale":"2023-12-26T21:36:48.817332+0000","last_undegraded":"2023-12-26T21:36:48.817332+0000","last_fullsized":"2023-12-26T21:36:48.817332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:17.677425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.837624+0000","last_change":"2023-12-26T21:36:48.834334+0000","last_active":"2023-12-26T21:36:48.837624+0000","last_peered":"2023-12-26T21:36:48.837624+0000","last_clean":"2023-12-26T21:36:48.837624+0000","last_became_active":"2023-12-26T21:36:48.834070+0000","last_became_peered":"2023-12-26T21:36:48.834070+0000","last_unstale":"2023-12-26T21:36:48.837624+0000","last_undegraded":"2023-12-26T21:36:48.837624+0000","last_fullsized":"2023-12-26T21:36:48.837624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:21:25.931065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843647+0000","last_change":"2023-12-26T21:36:48.841064+0000","last_active":"2023-12-26T21:36:48.843647+0000","last_peered":"2023-12-26T21:36:48.843647+0000","last_clean":"2023-12-26T21:36:48.843647+0000","last_became_active":"2023-12-26T21:36:48.840654+0000","last_became_peered":"2023-12-26T21:36:48.840654+0000","last_unstale":"2023-12-26T21:36:48.843647+0000","last_undegraded":"2023-12-26T21:36:48.843647+0000","last_fullsized":"2023-12-26T21:36:48.843647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.804661+0000","last_change":"2023-12-26T21:36:47.822491+0000","last_active":"2023-12-26T21:36:48.804661+0000","last_peered":"2023-12-26T21:36:48.804661+0000","last_clean":"2023-12-26T21:36:48.804661+0000","last_became_active":"2023-12-26T21:36:47.822150+0000","last_became_peered":"2023-12-26T21:36:47.822150+0000","last_unstale":"2023-12-26T21:36:48.804661+0000","last_undegraded":"2023-12-26T21:36:48.804661+0000","last_fullsized":"2023-12-26T21:36:48.804661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:08:16.186961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.834826+0000","last_change":"2023-12-26T21:36:48.828798+0000","last_active":"2023-12-26T21:36:48.834826+0000","last_peered":"2023-12-26T21:36:48.834826+0000","last_clean":"2023-12-26T21:36:48.834826+0000","last_became_active":"2023-12-26T21:36:48.828093+0000","last_became_peered":"2023-12-26T21:36:48.828093+0000","last_unstale":"2023-12-26T21:36:48.834826+0000","last_undegraded":"2023-12-26T21:36:48.834826+0000","last_fullsized":"2023-12-26T21:36:48.834826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:19:55.017056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817551+0000","last_change":"2023-12-26T21:36:47.825144+0000","last_active":"2023-12-26T21:36:48.817551+0000","last_peered":"2023-12-26T21:36:48.817551+0000","last_clean":"2023-12-26T21:36:48.817551+0000","last_became_active":"2023-12-26T21:36:47.824896+0000","last_became_peered":"2023-12-26T21:36:47.824896+0000","last_unstale":"2023-12-26T21:36:48.817551+0000","last_undegraded":"2023-12-26T21:36:48.817551+0000","last_fullsized":"2023-12-26T21:36:48.817551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:03:48.097670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.803182+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:36:48.803182+0000","last_peered":"2023-12-26T21:36:48.803182+0000","last_clean":"2023-12-26T21:36:48.803182+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:36:48.803182+0000","last_undegraded":"2023-12-26T21:36:48.803182+0000","last_fullsized":"2023-12-26T21:36:48.803182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.818212+0000","last_change":"2023-12-26T21:36:48.816469+0000","last_active":"2023-12-26T21:36:48.818212+0000","last_peered":"2023-12-26T21:36:48.818212+0000","last_clean":"2023-12-26T21:36:48.818212+0000","last_became_active":"2023-12-26T21:36:48.816190+0000","last_became_peered":"2023-12-26T21:36:48.816190+0000","last_unstale":"2023-12-26T21:36:48.818212+0000","last_undegraded":"2023-12-26T21:36:48.818212+0000","last_fullsized":"2023-12-26T21:36:48.818212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:35.691705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816437+0000","last_change":"2023-12-26T21:36:47.822572+0000","last_active":"2023-12-26T21:36:48.816437+0000","last_peered":"2023-12-26T21:36:48.816437+0000","last_clean":"2023-12-26T21:36:48.816437+0000","last_became_active":"2023-12-26T21:36:47.822214+0000","last_became_peered":"2023-12-26T21:36:47.822214+0000","last_unstale":"2023-12-26T21:36:48.816437+0000","last_undegraded":"2023-12-26T21:36:48.816437+0000","last_fullsized":"2023-12-26T21:36:48.816437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:34.685280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816507+0000","last_change":"2023-12-26T21:36:47.826782+0000","last_active":"2023-12-26T21:36:48.816507+0000","last_peered":"2023-12-26T21:36:48.816507+0000","last_clean":"2023-12-26T21:36:48.816507+0000","last_became_active":"2023-12-26T21:36:47.825248+0000","last_became_peered":"2023-12-26T21:36:47.825248+0000","last_unstale":"2023-12-26T21:36:48.816507+0000","last_undegraded":"2023-12-26T21:36:48.816507+0000","last_fullsized":"2023-12-26T21:36:48.816507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:28.026360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831901+0000","last_change":"2023-12-26T21:36:48.827788+0000","last_active":"2023-12-26T21:36:48.831901+0000","last_peered":"2023-12-26T21:36:48.831901+0000","last_clean":"2023-12-26T21:36:48.831901+0000","last_became_active":"2023-12-26T21:36:48.827487+0000","last_became_peered":"2023-12-26T21:36:48.827487+0000","last_unstale":"2023-12-26T21:36:48.831901+0000","last_undegraded":"2023-12-26T21:36:48.831901+0000","last_fullsized":"2023-12-26T21:36:48.831901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:22.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.818519+0000","last_change":"2023-12-26T21:36:47.830347+0000","last_active":"2023-12-26T21:36:48.818519+0000","last_peered":"2023-12-26T21:36:48.818519+0000","last_clean":"2023-12-26T21:36:48.818519+0000","last_became_active":"2023-12-26T21:36:47.830037+0000","last_became_peered":"2023-12-26T21:36:47.830037+0000","last_unstale":"2023-12-26T21:36:48.818519+0000","last_undegraded":"2023-12-26T21:36:48.818519+0000","last_fullsized":"2023-12-26T21:36:48.818519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:55.613054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838484+0000","last_change":"2023-12-26T21:36:48.829585+0000","last_active":"2023-12-26T21:36:48.838484+0000","last_peered":"2023-12-26T21:36:48.838484+0000","last_clean":"2023-12-26T21:36:48.838484+0000","last_became_active":"2023-12-26T21:36:48.829170+0000","last_became_peered":"2023-12-26T21:36:48.829170+0000","last_unstale":"2023-12-26T21:36:48.838484+0000","last_undegraded":"2023-12-26T21:36:48.838484+0000","last_fullsized":"2023-12-26T21:36:48.838484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:25:31.563553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844919+0000","last_change":"2023-12-26T21:36:47.832588+0000","last_active":"2023-12-26T21:36:48.844919+0000","last_peered":"2023-12-26T21:36:48.844919+0000","last_clean":"2023-12-26T21:36:48.844919+0000","last_became_active":"2023-12-26T21:36:47.832310+0000","last_became_peered":"2023-12-26T21:36:47.832310+0000","last_unstale":"2023-12-26T21:36:48.844919+0000","last_undegraded":"2023-12-26T21:36:48.844919+0000","last_fullsized":"2023-12-26T21:36:48.844919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:22:54.126845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839479+0000","last_change":"2023-12-26T21:36:48.830882+0000","last_active":"2023-12-26T21:36:48.839479+0000","last_peered":"2023-12-26T21:36:48.839479+0000","last_clean":"2023-12-26T21:36:48.839479+0000","last_became_active":"2023-12-26T21:36:48.830587+0000","last_became_peered":"2023-12-26T21:36:48.830587+0000","last_unstale":"2023-12-26T21:36:48.839479+0000","last_undegraded":"2023-12-26T21:36:48.839479+0000","last_fullsized":"2023-12-26T21:36:48.839479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:54.746974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826954+0000","last_change":"2023-12-26T21:36:47.826141+0000","last_active":"2023-12-26T21:36:48.826954+0000","last_peered":"2023-12-26T21:36:48.826954+0000","last_clean":"2023-12-26T21:36:48.826954+0000","last_became_active":"2023-12-26T21:36:47.825876+0000","last_became_peered":"2023-12-26T21:36:47.825876+0000","last_unstale":"2023-12-26T21:36:48.826954+0000","last_undegraded":"2023-12-26T21:36:48.826954+0000","last_fullsized":"2023-12-26T21:36:48.826954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:12.184779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843719+0000","last_change":"2023-12-26T21:36:48.829405+0000","last_active":"2023-12-26T21:36:48.843719+0000","last_peered":"2023-12-26T21:36:48.843719+0000","last_clean":"2023-12-26T21:36:48.843719+0000","last_became_active":"2023-12-26T21:36:48.829169+0000","last_became_peered":"2023-12-26T21:36:48.829169+0000","last_unstale":"2023-12-26T21:36:48.843719+0000","last_undegraded":"2023-12-26T21:36:48.843719+0000","last_fullsized":"2023-12-26T21:36:48.843719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:42:15.750967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.819202+0000","last_change":"2023-12-26T21:36:47.830820+0000","last_active":"2023-12-26T21:36:48.819202+0000","last_peered":"2023-12-26T21:36:48.819202+0000","last_clean":"2023-12-26T21:36:48.819202+0000","last_became_active":"2023-12-26T21:36:47.830514+0000","last_became_peered":"2023-12-26T21:36:47.830514+0000","last_unstale":"2023-12-26T21:36:48.819202+0000","last_undegraded":"2023-12-26T21:36:48.819202+0000","last_fullsized":"2023-12-26T21:36:48.819202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:53.472268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839858+0000","last_change":"2023-12-26T21:36:48.832982+0000","last_active":"2023-12-26T21:36:48.839858+0000","last_peered":"2023-12-26T21:36:48.839858+0000","last_clean":"2023-12-26T21:36:48.839858+0000","last_became_active":"2023-12-26T21:36:48.831969+0000","last_became_peered":"2023-12-26T21:36:48.831969+0000","last_unstale":"2023-12-26T21:36:48.839858+0000","last_undegraded":"2023-12-26T21:36:48.839858+0000","last_fullsized":"2023-12-26T21:36:48.839858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:54.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.812495+0000","last_change":"2023-12-26T21:36:47.826837+0000","last_active":"2023-12-26T21:36:48.812495+0000","last_peered":"2023-12-26T21:36:48.812495+0000","last_clean":"2023-12-26T21:36:48.812495+0000","last_became_active":"2023-12-26T21:36:47.826479+0000","last_became_peered":"2023-12-26T21:36:47.826479+0000","last_unstale":"2023-12-26T21:36:48.812495+0000","last_undegraded":"2023-12-26T21:36:48.812495+0000","last_fullsized":"2023-12-26T21:36:48.812495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:05.573200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817414+0000","last_change":"2023-12-26T21:36:48.813654+0000","last_active":"2023-12-26T21:36:48.817414+0000","last_peered":"2023-12-26T21:36:48.817414+0000","last_clean":"2023-12-26T21:36:48.817414+0000","last_became_active":"2023-12-26T21:36:48.813344+0000","last_became_peered":"2023-12-26T21:36:48.813344+0000","last_unstale":"2023-12-26T21:36:48.817414+0000","last_undegraded":"2023-12-26T21:36:48.817414+0000","last_fullsized":"2023-12-26T21:36:48.817414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:54.478693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839092+0000","last_change":"2023-12-26T21:36:47.820913+0000","last_active":"2023-12-26T21:36:48.839092+0000","last_peered":"2023-12-26T21:36:48.839092+0000","last_clean":"2023-12-26T21:36:48.839092+0000","last_became_active":"2023-12-26T21:36:47.820617+0000","last_became_peered":"2023-12-26T21:36:47.820617+0000","last_unstale":"2023-12-26T21:36:48.839092+0000","last_undegraded":"2023-12-26T21:36:48.839092+0000","last_fullsized":"2023-12-26T21:36:48.839092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:41.047506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829801+0000","last_change":"2023-12-26T21:36:48.826060+0000","last_active":"2023-12-26T21:36:48.829801+0000","last_peered":"2023-12-26T21:36:48.829801+0000","last_clean":"2023-12-26T21:36:48.829801+0000","last_became_active":"2023-12-26T21:36:48.825817+0000","last_became_peered":"2023-12-26T21:36:48.825817+0000","last_unstale":"2023-12-26T21:36:48.829801+0000","last_undegraded":"2023-12-26T21:36:48.829801+0000","last_fullsized":"2023-12-26T21:36:48.829801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.809730+0000","last_change":"2023-12-26T21:36:47.827562+0000","last_active":"2023-12-26T21:36:48.809730+0000","last_peered":"2023-12-26T21:36:48.809730+0000","last_clean":"2023-12-26T21:36:48.809730+0000","last_became_active":"2023-12-26T21:36:47.827284+0000","last_became_peered":"2023-12-26T21:36:47.827284+0000","last_unstale":"2023-12-26T21:36:48.809730+0000","last_undegraded":"2023-12-26T21:36:48.809730+0000","last_fullsized":"2023-12-26T21:36:48.809730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843878+0000","last_change":"2023-12-26T21:36:48.829818+0000","last_active":"2023-12-26T21:36:48.843878+0000","last_peered":"2023-12-26T21:36:48.843878+0000","last_clean":"2023-12-26T21:36:48.843878+0000","last_became_active":"2023-12-26T21:36:48.829087+0000","last_became_peered":"2023-12-26T21:36:48.829087+0000","last_unstale":"2023-12-26T21:36:48.843878+0000","last_undegraded":"2023-12-26T21:36:48.843878+0000","last_fullsized":"2023-12-26T21:36:48.843878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:37:13.353768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.842747+0000","last_change":"2023-12-26T21:36:47.815095+0000","last_active":"2023-12-26T21:36:48.842747+0000","last_peered":"2023-12-26T21:36:48.842747+0000","last_clean":"2023-12-26T21:36:48.842747+0000","last_became_active":"2023-12-26T21:36:47.814728+0000","last_became_peered":"2023-12-26T21:36:47.814728+0000","last_unstale":"2023-12-26T21:36:48.842747+0000","last_undegraded":"2023-12-26T21:36:48.842747+0000","last_fullsized":"2023-12-26T21:36:48.842747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:14.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846566+0000","last_change":"2023-12-26T21:36:48.837315+0000","last_active":"2023-12-26T21:36:48.846566+0000","last_peered":"2023-12-26T21:36:48.846566+0000","last_clean":"2023-12-26T21:36:48.846566+0000","last_became_active":"2023-12-26T21:36:48.837067+0000","last_became_peered":"2023-12-26T21:36:48.837067+0000","last_unstale":"2023-12-26T21:36:48.846566+0000","last_undegraded":"2023-12-26T21:36:48.846566+0000","last_fullsized":"2023-12-26T21:36:48.846566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:28.883031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.834582+0000","last_change":"2023-12-26T21:36:47.825290+0000","last_active":"2023-12-26T21:36:48.834582+0000","last_peered":"2023-12-26T21:36:48.834582+0000","last_clean":"2023-12-26T21:36:48.834582+0000","last_became_active":"2023-12-26T21:36:47.824594+0000","last_became_peered":"2023-12-26T21:36:47.824594+0000","last_unstale":"2023-12-26T21:36:48.834582+0000","last_undegraded":"2023-12-26T21:36:48.834582+0000","last_fullsized":"2023-12-26T21:36:48.834582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:06.749340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843529+0000","last_change":"2023-12-26T21:36:48.835342+0000","last_active":"2023-12-26T21:36:48.843529+0000","last_peered":"2023-12-26T21:36:48.843529+0000","last_clean":"2023-12-26T21:36:48.843529+0000","last_became_active":"2023-12-26T21:36:48.830463+0000","last_became_peered":"2023-12-26T21:36:48.830463+0000","last_unstale":"2023-12-26T21:36:48.843529+0000","last_undegraded":"2023-12-26T21:36:48.843529+0000","last_fullsized":"2023-12-26T21:36:48.843529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.824113+0000","last_change":"2023-12-26T21:36:48.817508+0000","last_active":"2023-12-26T21:36:48.824113+0000","last_peered":"2023-12-26T21:36:48.824113+0000","last_clean":"2023-12-26T21:36:48.824113+0000","last_became_active":"2023-12-26T21:36:48.817269+0000","last_became_peered":"2023-12-26T21:36:48.817269+0000","last_unstale":"2023-12-26T21:36:48.824113+0000","last_undegraded":"2023-12-26T21:36:48.824113+0000","last_fullsized":"2023-12-26T21:36:48.824113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:54.478693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831298+0000","last_change":"2023-12-26T21:36:48.820601+0000","last_active":"2023-12-26T21:36:48.831298+0000","last_peered":"2023-12-26T21:36:48.831298+0000","last_clean":"2023-12-26T21:36:48.831298+0000","last_became_active":"2023-12-26T21:36:48.819293+0000","last_became_peered":"2023-12-26T21:36:48.819293+0000","last_unstale":"2023-12-26T21:36:48.831298+0000","last_undegraded":"2023-12-26T21:36:48.831298+0000","last_fullsized":"2023-12-26T21:36:48.831298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:26.207089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844324+0000","last_change":"2023-12-26T21:36:48.834258+0000","last_active":"2023-12-26T21:36:48.844324+0000","last_peered":"2023-12-26T21:36:48.844324+0000","last_clean":"2023-12-26T21:36:48.844324+0000","last_became_active":"2023-12-26T21:36:48.834005+0000","last_became_peered":"2023-12-26T21:36:48.834005+0000","last_unstale":"2023-12-26T21:36:48.844324+0000","last_undegraded":"2023-12-26T21:36:48.844324+0000","last_fullsized":"2023-12-26T21:36:48.844324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:15.953197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841827+0000","last_change":"2023-12-26T21:36:48.834061+0000","last_active":"2023-12-26T21:36:48.841827+0000","last_peered":"2023-12-26T21:36:48.841827+0000","last_clean":"2023-12-26T21:36:48.841827+0000","last_became_active":"2023-12-26T21:36:48.821177+0000","last_became_peered":"2023-12-26T21:36:48.821177+0000","last_unstale":"2023-12-26T21:36:48.841827+0000","last_undegraded":"2023-12-26T21:36:48.841827+0000","last_fullsized":"2023-12-26T21:36:48.841827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:21:25.931065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828251+0000","last_change":"2023-12-26T21:36:48.823718+0000","last_active":"2023-12-26T21:36:48.828251+0000","last_peered":"2023-12-26T21:36:48.828251+0000","last_clean":"2023-12-26T21:36:48.828251+0000","last_became_active":"2023-12-26T21:36:48.823422+0000","last_became_peered":"2023-12-26T21:36:48.823422+0000","last_unstale":"2023-12-26T21:36:48.828251+0000","last_undegraded":"2023-12-26T21:36:48.828251+0000","last_fullsized":"2023-12-26T21:36:48.828251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:42:23.347556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826781+0000","last_change":"2023-12-26T21:36:48.821819+0000","last_active":"2023-12-26T21:36:48.826781+0000","last_peered":"2023-12-26T21:36:48.826781+0000","last_clean":"2023-12-26T21:36:48.826781+0000","last_became_active":"2023-12-26T21:36:48.821527+0000","last_became_peered":"2023-12-26T21:36:48.821527+0000","last_unstale":"2023-12-26T21:36:48.826781+0000","last_undegraded":"2023-12-26T21:36:48.826781+0000","last_fullsized":"2023-12-26T21:36:48.826781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:31.058793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844139+0000","last_change":"2023-12-26T21:36:48.841186+0000","last_active":"2023-12-26T21:36:48.844139+0000","last_peered":"2023-12-26T21:36:48.844139+0000","last_clean":"2023-12-26T21:36:48.844139+0000","last_became_active":"2023-12-26T21:36:48.839955+0000","last_became_peered":"2023-12-26T21:36:48.839955+0000","last_unstale":"2023-12-26T21:36:48.844139+0000","last_undegraded":"2023-12-26T21:36:48.844139+0000","last_fullsized":"2023-12-26T21:36:48.844139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:35.307950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841419+0000","last_change":"2023-12-26T21:36:48.832556+0000","last_active":"2023-12-26T21:36:48.841419+0000","last_peered":"2023-12-26T21:36:48.841419+0000","last_clean":"2023-12-26T21:36:48.841419+0000","last_became_active":"2023-12-26T21:36:48.830061+0000","last_became_peered":"2023-12-26T21:36:48.830061+0000","last_unstale":"2023-12-26T21:36:48.841419+0000","last_undegraded":"2023-12-26T21:36:48.841419+0000","last_fullsized":"2023-12-26T21:36:48.841419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:22.938456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846674+0000","last_change":"2023-12-26T21:36:48.842574+0000","last_active":"2023-12-26T21:36:48.846674+0000","last_peered":"2023-12-26T21:36:48.846674+0000","last_clean":"2023-12-26T21:36:48.846674+0000","last_became_active":"2023-12-26T21:36:48.842273+0000","last_became_peered":"2023-12-26T21:36:48.842273+0000","last_unstale":"2023-12-26T21:36:48.846674+0000","last_undegraded":"2023-12-26T21:36:48.846674+0000","last_fullsized":"2023-12-26T21:36:48.846674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:17:59.482430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.840028+0000","last_change":"2023-12-26T21:36:48.826270+0000","last_active":"2023-12-26T21:36:48.840028+0000","last_peered":"2023-12-26T21:36:48.840028+0000","last_clean":"2023-12-26T21:36:48.840028+0000","last_became_active":"2023-12-26T21:36:48.825303+0000","last_became_peered":"2023-12-26T21:36:48.825303+0000","last_unstale":"2023-12-26T21:36:48.840028+0000","last_undegraded":"2023-12-26T21:36:48.840028+0000","last_fullsized":"2023-12-26T21:36:48.840028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:22.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828348+0000","last_change":"2023-12-26T21:36:48.823561+0000","last_active":"2023-12-26T21:36:48.828348+0000","last_peered":"2023-12-26T21:36:48.828348+0000","last_clean":"2023-12-26T21:36:48.828348+0000","last_became_active":"2023-12-26T21:36:48.823270+0000","last_became_peered":"2023-12-26T21:36:48.823270+0000","last_unstale":"2023-12-26T21:36:48.828348+0000","last_undegraded":"2023-12-26T21:36:48.828348+0000","last_fullsized":"2023-12-26T21:36:48.828348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:29.032786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839419+0000","last_change":"2023-12-26T21:36:48.828687+0000","last_active":"2023-12-26T21:36:48.839419+0000","last_peered":"2023-12-26T21:36:48.839419+0000","last_clean":"2023-12-26T21:36:48.839419+0000","last_became_active":"2023-12-26T21:36:48.828405+0000","last_became_peered":"2023-12-26T21:36:48.828405+0000","last_unstale":"2023-12-26T21:36:48.839419+0000","last_undegraded":"2023-12-26T21:36:48.839419+0000","last_fullsized":"2023-12-26T21:36:48.839419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:37.832638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841288+0000","last_change":"2023-12-26T21:36:48.835378+0000","last_active":"2023-12-26T21:36:48.841288+0000","last_peered":"2023-12-26T21:36:48.841288+0000","last_clean":"2023-12-26T21:36:48.841288+0000","last_became_active":"2023-12-26T21:36:48.835049+0000","last_became_peered":"2023-12-26T21:36:48.835049+0000","last_unstale":"2023-12-26T21:36:48.841288+0000","last_undegraded":"2023-12-26T21:36:48.841288+0000","last_fullsized":"2023-12-26T21:36:48.841288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:22.938456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":76900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":667360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":79080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":78436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":77024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298088,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073752,"statfs":{"total":96636764160,"available":96331522048,"internally_reserved":0,"allocated":819200,"data_stored":667338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":77454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:36:53.332 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:36:53.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.556+0000 7f82f5fff700 1 Processor -- start 2023-12-26T21:36:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.556+0000 7f82f5fff700 1 -- start start 2023-12-26T21:36:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.558+0000 7f82f5fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f0159a70 0x7f82f00b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.558+0000 7f82f5fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82f00b2a90 0x7f82f00b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.558+0000 7f82f5fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f00afcd0 con 0x7f82f01596a0 2023-12-26T21:36:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.558+0000 7f82f5fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f00b3e70 con 0x7f82f00b2a90 2023-12-26T21:36:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.558+0000 7f82f5fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00b0f20 con 0x7f82f0159a70 2023-12-26T21:36:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.558+0000 7f82effff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82f00b2a90 0x7f82f00b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.558+0000 7f82e7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f0159a70 0x7f82f00b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.558+0000 7f82ef7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f82f01596a0 0x7f82f00b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:47420/0 (socket says 172.21.15.131:47420) 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.558+0000 7f82ef7fe700 1 -- 172.21.15.131:0/2924741361 learned_addr learned my addr 172.21.15.131:0/2924741361 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.559+0000 7f82effff700 1 -- 172.21.15.131:0/2924741361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f0159a70 msgr2=0x7f82f00b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.559+0000 7f82effff700 1 --2- 172.21.15.131:0/2924741361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f0159a70 0x7f82f00b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.559+0000 7f82effff700 1 -- 172.21.15.131:0/2924741361 >> v1:172.21.15.190:6789/0 conn(0x7f82f01596a0 legacy=0x7f82f00b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.559+0000 7f82effff700 1 -- 172.21.15.131:0/2924741361 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f00b3540 con 0x7f82f00b2a90 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.559+0000 7f82effff700 1 --2- 172.21.15.131:0/2924741361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 0x7f82f00b06a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009e60 tx=0x7f82e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f12022f5a53184ef server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.559+0000 7f82eeffd700 1 -- 172.21.15.131:0/2924741361 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800e070 con 0x7f82f00b2a90 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.559+0000 7f82eeffd700 1 -- 172.21.15.131:0/2924741361 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e800bb60 con 0x7f82f00b2a90 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.559+0000 7f82eeffd700 1 -- 172.21.15.131:0/2924741361 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8012430 con 0x7f82f00b2a90 2023-12-26T21:36:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.560+0000 7f82f5fff700 1 -- 172.21.15.131:0/2924741361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 msgr2=0x7f82f00b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.560+0000 7f82f5fff700 1 --2- 172.21.15.131:0/2924741361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 0x7f82f00b06a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009e60 tx=0x7f82e8009780 comp rx=0 tx=0).stop 2023-12-26T21:36:53.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.560+0000 7f82f5fff700 1 -- 172.21.15.131:0/2924741361 shutdown_connections 2023-12-26T21:36:53.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.560+0000 7f82f5fff700 1 --2- 172.21.15.131:0/2924741361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 0x7f82f00b06a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.560+0000 7f82f5fff700 1 --2- 172.21.15.131:0/2924741361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f0159a70 0x7f82f00b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.560+0000 7f82f5fff700 1 -- 172.21.15.131:0/2924741361 >> 172.21.15.131:0/2924741361 conn(0x7f82f006ece0 msgr2=0x7f82f006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:53.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.560+0000 7f82f5fff700 1 -- 172.21.15.131:0/2924741361 shutdown_connections 2023-12-26T21:36:53.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.560+0000 7f82f5fff700 1 -- 172.21.15.131:0/2924741361 wait complete. 2023-12-26T21:36:53.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.561+0000 7f82f5fff700 1 Processor -- start 2023-12-26T21:36:53.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.561+0000 7f82f5fff700 1 -- start start 2023-12-26T21:36:53.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.562+0000 7f82f5fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 0x7f82f00c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.562+0000 7f82f5fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f01596a0 0x7f82f00cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.562+0000 7f82f5fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f0159a70 0x7f82f00cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.562+0000 7f82f5fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00ca330 con 0x7f82f0159a70 2023-12-26T21:36:53.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.562+0000 7f82f5fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00c9030 con 0x7f82f00b2a90 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.562+0000 7f82f5fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00c9310 con 0x7f82f01596a0 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.562+0000 7f82effff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f0159a70 0x7f82f00cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.562+0000 7f82ef7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 0x7f82f00c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82e7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f01596a0 0x7f82f00cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82ef7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 0x7f82f00c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53208/0 (socket says 172.21.15.131:53208) 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82ef7fe700 1 -- 172.21.15.131:0/2466876799 learned_addr learned my addr 172.21.15.131:0/2466876799 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82effff700 1 -- 172.21.15.131:0/2466876799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f01596a0 msgr2=0x7f82f00cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82effff700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f01596a0 0x7f82f00cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82effff700 1 -- 172.21.15.131:0/2466876799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 msgr2=0x7f82f00c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82effff700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 0x7f82f00c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82effff700 1 -- 172.21.15.131:0/2466876799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8009a20 con 0x7f82f0159a70 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82ef7fe700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 0x7f82f00c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82effff700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f0159a70 0x7f82f00cb740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f82e8016000 tx=0x7f82e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:53.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82e77fe700 1 -- 172.21.15.131:0/2466876799 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8027470 con 0x7f82f0159a70 2023-12-26T21:36:53.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.563+0000 7f82e77fe700 1 -- 172.21.15.131:0/2466876799 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e80275f0 con 0x7f82f0159a70 2023-12-26T21:36:53.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.564+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f00c95f0 con 0x7f82f0159a70 2023-12-26T21:36:53.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.564+0000 7f82e77fe700 1 -- 172.21.15.131:0/2466876799 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e80279b0 con 0x7f82f0159a70 2023-12-26T21:36:53.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.564+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f00c8ce0 con 0x7f82f0159a70 2023-12-26T21:36:53.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.565+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f00657e0 con 0x7f82f0159a70 2023-12-26T21:36:53.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.565+0000 7f82e77fe700 1 -- 172.21.15.131:0/2466876799 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f82e8024070 con 0x7f82f0159a70 2023-12-26T21:36:53.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.566+0000 7f82e77fe700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f82d0066f30 0x7f82d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:53.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.566+0000 7f82e77fe700 1 -- 172.21.15.131:0/2466876799 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f82e8086f90 con 0x7f82f0159a70 2023-12-26T21:36:53.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.566+0000 7f82ef7fe700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f82d0066f30 0x7f82d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:53.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.567+0000 7f82ef7fe700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f82d0066f30 0x7f82d00693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82e0004900 tx=0x7f82e0007f70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:53.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.570+0000 7f82e77fe700 1 -- 172.21.15.131:0/2466876799 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82e8054160 con 0x7f82f0159a70 2023-12-26T21:36:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.760+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82f00b3820 con 0x7f82d0066f30 2023-12-26T21:36:53.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.768+0000 7f82e77fe700 1 -- 172.21.15.131:0/2466876799 <== mgr.4100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264605 (secure 0 0 0) 0x7f82f00b3820 con 0x7f82d0066f30 2023-12-26T21:36:53.771 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f82d0066f30 msgr2=0x7f82d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f82d0066f30 0x7f82d00693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82e0004900 tx=0x7f82e0007f70 comp rx=0 tx=0).stop 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f0159a70 msgr2=0x7f82f00cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f0159a70 0x7f82f00cb740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f82e8016000 tx=0x7f82e8019000 comp rx=0 tx=0).stop 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 shutdown_connections 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82f0159a70 0x7f82f00cb740 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f00b2a90 0x7f82f00c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f82d0066f30 0x7f82d00693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 --2- 172.21.15.131:0/2466876799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82f01596a0 0x7f82f00cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.772+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 >> 172.21.15.131:0/2466876799 conn(0x7f82f006ece0 msgr2=0x7f82f015e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.773+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 shutdown_connections 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:53.773+0000 7f82f5fff700 1 -- 172.21.15.131:0/2466876799 wait complete. 2023-12-26T21:36:53.774 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:36:53.790 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-26T21:36:52.998659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381256,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593464,"statfs":{"total":773094113280,"available":770655707136,"internally_reserved":0,"allocated":3022848,"data_stored":1800548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001048"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845569+0000","last_change":"2023-12-26T21:36:48.838074+0000","last_active":"2023-12-26T21:36:48.845569+0000","last_peered":"2023-12-26T21:36:48.845569+0000","last_clean":"2023-12-26T21:36:48.845569+0000","last_became_active":"2023-12-26T21:36:48.837495+0000","last_became_peered":"2023-12-26T21:36:48.837495+0000","last_unstale":"2023-12-26T21:36:48.845569+0000","last_undegraded":"2023-12-26T21:36:48.845569+0000","last_fullsized":"2023-12-26T21:36:48.845569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841349+0000","last_change":"2023-12-26T21:36:48.830149+0000","last_active":"2023-12-26T21:36:48.841349+0000","last_peered":"2023-12-26T21:36:48.841349+0000","last_clean":"2023-12-26T21:36:48.841349+0000","last_became_active":"2023-12-26T21:36:48.829771+0000","last_became_peered":"2023-12-26T21:36:48.829771+0000","last_unstale":"2023-12-26T21:36:48.841349+0000","last_undegraded":"2023-12-26T21:36:48.841349+0000","last_fullsized":"2023-12-26T21:36:48.841349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:59.578493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841417+0000","last_change":"2023-12-26T21:36:48.831124+0000","last_active":"2023-12-26T21:36:48.841417+0000","last_peered":"2023-12-26T21:36:48.841417+0000","last_clean":"2023-12-26T21:36:48.841417+0000","last_became_active":"2023-12-26T21:36:48.830817+0000","last_became_peered":"2023-12-26T21:36:48.830817+0000","last_unstale":"2023-12-26T21:36:48.841417+0000","last_undegraded":"2023-12-26T21:36:48.841417+0000","last_fullsized":"2023-12-26T21:36:48.841417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:47.183606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845126+0000","last_change":"2023-12-26T21:36:48.842229+0000","last_active":"2023-12-26T21:36:48.845126+0000","last_peered":"2023-12-26T21:36:48.845126+0000","last_clean":"2023-12-26T21:36:48.845126+0000","last_became_active":"2023-12-26T21:36:48.839269+0000","last_became_peered":"2023-12-26T21:36:48.839269+0000","last_unstale":"2023-12-26T21:36:48.845126+0000","last_undegraded":"2023-12-26T21:36:48.845126+0000","last_fullsized":"2023-12-26T21:36:48.845126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:01:49.095423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845558+0000","last_change":"2023-12-26T21:36:48.838113+0000","last_active":"2023-12-26T21:36:48.845558+0000","last_peered":"2023-12-26T21:36:48.845558+0000","last_clean":"2023-12-26T21:36:48.845558+0000","last_became_active":"2023-12-26T21:36:48.837882+0000","last_became_peered":"2023-12-26T21:36:48.837882+0000","last_unstale":"2023-12-26T21:36:48.845558+0000","last_undegraded":"2023-12-26T21:36:48.845558+0000","last_fullsized":"2023-12-26T21:36:48.845558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:40.692719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839762+0000","last_change":"2023-12-26T21:36:48.835731+0000","last_active":"2023-12-26T21:36:48.839762+0000","last_peered":"2023-12-26T21:36:48.839762+0000","last_clean":"2023-12-26T21:36:48.839762+0000","last_became_active":"2023-12-26T21:36:48.830672+0000","last_became_peered":"2023-12-26T21:36:48.830672+0000","last_unstale":"2023-12-26T21:36:48.839762+0000","last_undegraded":"2023-12-26T21:36:48.839762+0000","last_fullsized":"2023-12-26T21:36:48.839762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:06:58.960255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835557+0000","last_change":"2023-12-26T21:36:48.831504+0000","last_active":"2023-12-26T21:36:48.835557+0000","last_peered":"2023-12-26T21:36:48.835557+0000","last_clean":"2023-12-26T21:36:48.835557+0000","last_became_active":"2023-12-26T21:36:48.831173+0000","last_became_peered":"2023-12-26T21:36:48.831173+0000","last_unstale":"2023-12-26T21:36:48.835557+0000","last_undegraded":"2023-12-26T21:36:48.835557+0000","last_fullsized":"2023-12-26T21:36:48.835557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:37:13.353768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841944+0000","last_change":"2023-12-26T21:36:48.838249+0000","last_active":"2023-12-26T21:36:48.841944+0000","last_peered":"2023-12-26T21:36:48.841944+0000","last_clean":"2023-12-26T21:36:48.841944+0000","last_became_active":"2023-12-26T21:36:48.837983+0000","last_became_peered":"2023-12-26T21:36:48.837983+0000","last_unstale":"2023-12-26T21:36:48.841944+0000","last_undegraded":"2023-12-26T21:36:48.841944+0000","last_fullsized":"2023-12-26T21:36:48.841944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:56.706835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831690+0000","last_change":"2023-12-26T21:36:48.829328+0000","last_active":"2023-12-26T21:36:48.831690+0000","last_peered":"2023-12-26T21:36:48.831690+0000","last_clean":"2023-12-26T21:36:48.831690+0000","last_became_active":"2023-12-26T21:36:48.829033+0000","last_became_peered":"2023-12-26T21:36:48.829033+0000","last_unstale":"2023-12-26T21:36:48.831690+0000","last_undegraded":"2023-12-26T21:36:48.831690+0000","last_fullsized":"2023-12-26T21:36:48.831690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:31.058793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.837727+0000","last_change":"2023-12-26T21:36:48.834507+0000","last_active":"2023-12-26T21:36:48.837727+0000","last_peered":"2023-12-26T21:36:48.837727+0000","last_clean":"2023-12-26T21:36:48.837727+0000","last_became_active":"2023-12-26T21:36:48.833751+0000","last_became_peered":"2023-12-26T21:36:48.833751+0000","last_unstale":"2023-12-26T21:36:48.837727+0000","last_undegraded":"2023-12-26T21:36:48.837727+0000","last_fullsized":"2023-12-26T21:36:48.837727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:23.349206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846618+0000","last_change":"2023-12-26T21:36:48.839909+0000","last_active":"2023-12-26T21:36:48.846618+0000","last_peered":"2023-12-26T21:36:48.846618+0000","last_clean":"2023-12-26T21:36:48.846618+0000","last_became_active":"2023-12-26T21:36:48.839611+0000","last_became_peered":"2023-12-26T21:36:48.839611+0000","last_unstale":"2023-12-26T21:36:48.846618+0000","last_undegraded":"2023-12-26T21:36:48.846618+0000","last_fullsized":"2023-12-26T21:36:48.846618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:49.385696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841298+0000","last_change":"2023-12-26T21:36:48.836176+0000","last_active":"2023-12-26T21:36:48.841298+0000","last_peered":"2023-12-26T21:36:48.841298+0000","last_clean":"2023-12-26T21:36:48.841298+0000","last_became_active":"2023-12-26T21:36:48.835911+0000","last_became_peered":"2023-12-26T21:36:48.835911+0000","last_unstale":"2023-12-26T21:36:48.841298+0000","last_undegraded":"2023-12-26T21:36:48.841298+0000","last_fullsized":"2023-12-26T21:36:48.841298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:32.390648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828384+0000","last_change":"2023-12-26T21:36:48.825560+0000","last_active":"2023-12-26T21:36:48.828384+0000","last_peered":"2023-12-26T21:36:48.828384+0000","last_clean":"2023-12-26T21:36:48.828384+0000","last_became_active":"2023-12-26T21:36:48.825203+0000","last_became_peered":"2023-12-26T21:36:48.825203+0000","last_unstale":"2023-12-26T21:36:48.828384+0000","last_undegraded":"2023-12-26T21:36:48.828384+0000","last_fullsized":"2023-12-26T21:36:48.828384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:24.061374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828296+0000","last_change":"2023-12-26T21:36:48.823931+0000","last_active":"2023-12-26T21:36:48.828296+0000","last_peered":"2023-12-26T21:36:48.828296+0000","last_clean":"2023-12-26T21:36:48.828296+0000","last_became_active":"2023-12-26T21:36:48.823564+0000","last_became_peered":"2023-12-26T21:36:48.823564+0000","last_unstale":"2023-12-26T21:36:48.828296+0000","last_undegraded":"2023-12-26T21:36:48.828296+0000","last_fullsized":"2023-12-26T21:36:48.828296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:42.053931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833245+0000","last_change":"2023-12-26T21:36:48.823836+0000","last_active":"2023-12-26T21:36:48.833245+0000","last_peered":"2023-12-26T21:36:48.833245+0000","last_clean":"2023-12-26T21:36:48.833245+0000","last_became_active":"2023-12-26T21:36:48.823053+0000","last_became_peered":"2023-12-26T21:36:48.823053+0000","last_unstale":"2023-12-26T21:36:48.833245+0000","last_undegraded":"2023-12-26T21:36:48.833245+0000","last_fullsized":"2023-12-26T21:36:48.833245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:28.883031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838446+0000","last_change":"2023-12-26T21:36:48.829032+0000","last_active":"2023-12-26T21:36:48.838446+0000","last_peered":"2023-12-26T21:36:48.838446+0000","last_clean":"2023-12-26T21:36:48.838446+0000","last_became_active":"2023-12-26T21:36:48.826097+0000","last_became_peered":"2023-12-26T21:36:48.826097+0000","last_unstale":"2023-12-26T21:36:48.838446+0000","last_undegraded":"2023-12-26T21:36:48.838446+0000","last_fullsized":"2023-12-26T21:36:48.838446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:49.385696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844638+0000","last_change":"2023-12-26T21:36:48.832781+0000","last_active":"2023-12-26T21:36:48.844638+0000","last_peered":"2023-12-26T21:36:48.844638+0000","last_clean":"2023-12-26T21:36:48.844638+0000","last_became_active":"2023-12-26T21:36:48.832414+0000","last_became_peered":"2023-12-26T21:36:48.832414+0000","last_unstale":"2023-12-26T21:36:48.844638+0000","last_undegraded":"2023-12-26T21:36:48.844638+0000","last_fullsized":"2023-12-26T21:36:48.844638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:05:36.844680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.825827+0000","last_change":"2023-12-26T21:36:48.817787+0000","last_active":"2023-12-26T21:36:48.825827+0000","last_peered":"2023-12-26T21:36:48.825827+0000","last_clean":"2023-12-26T21:36:48.825827+0000","last_became_active":"2023-12-26T21:36:48.817432+0000","last_became_peered":"2023-12-26T21:36:48.817432+0000","last_unstale":"2023-12-26T21:36:48.825827+0000","last_undegraded":"2023-12-26T21:36:48.825827+0000","last_fullsized":"2023-12-26T21:36:48.825827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:51.402901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844696+0000","last_change":"2023-12-26T21:36:48.834659+0000","last_active":"2023-12-26T21:36:48.844696+0000","last_peered":"2023-12-26T21:36:48.844696+0000","last_clean":"2023-12-26T21:36:48.844696+0000","last_became_active":"2023-12-26T21:36:48.833967+0000","last_became_peered":"2023-12-26T21:36:48.833967+0000","last_unstale":"2023-12-26T21:36:48.844696+0000","last_undegraded":"2023-12-26T21:36:48.844696+0000","last_fullsized":"2023-12-26T21:36:48.844696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:12:41.152256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839315+0000","last_change":"2023-12-26T21:36:47.830031+0000","last_active":"2023-12-26T21:36:48.839315+0000","last_peered":"2023-12-26T21:36:48.839315+0000","last_clean":"2023-12-26T21:36:48.839315+0000","last_became_active":"2023-12-26T21:36:47.829681+0000","last_became_peered":"2023-12-26T21:36:47.829681+0000","last_unstale":"2023-12-26T21:36:48.839315+0000","last_undegraded":"2023-12-26T21:36:48.839315+0000","last_fullsized":"2023-12-26T21:36:48.839315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:25.200664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841270+0000","last_change":"2023-12-26T21:36:48.827382+0000","last_active":"2023-12-26T21:36:48.841270+0000","last_peered":"2023-12-26T21:36:48.841270+0000","last_clean":"2023-12-26T21:36:48.841270+0000","last_became_active":"2023-12-26T21:36:48.825125+0000","last_became_peered":"2023-12-26T21:36:48.825125+0000","last_unstale":"2023-12-26T21:36:48.841270+0000","last_undegraded":"2023-12-26T21:36:48.841270+0000","last_fullsized":"2023-12-26T21:36:48.841270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:02:27.671653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843042+0000","last_change":"2023-12-26T21:36:47.825862+0000","last_active":"2023-12-26T21:36:48.843042+0000","last_peered":"2023-12-26T21:36:48.843042+0000","last_clean":"2023-12-26T21:36:48.843042+0000","last_became_active":"2023-12-26T21:36:47.825603+0000","last_became_peered":"2023-12-26T21:36:47.825603+0000","last_unstale":"2023-12-26T21:36:48.843042+0000","last_undegraded":"2023-12-26T21:36:48.843042+0000","last_fullsized":"2023-12-26T21:36:48.843042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:18:58.593197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833411+0000","last_change":"2023-12-26T21:36:48.819527+0000","last_active":"2023-12-26T21:36:48.833411+0000","last_peered":"2023-12-26T21:36:48.833411+0000","last_clean":"2023-12-26T21:36:48.833411+0000","last_became_active":"2023-12-26T21:36:48.819208+0000","last_became_peered":"2023-12-26T21:36:48.819208+0000","last_unstale":"2023-12-26T21:36:48.833411+0000","last_undegraded":"2023-12-26T21:36:48.833411+0000","last_fullsized":"2023-12-26T21:36:48.833411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:32.390648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841778+0000","last_change":"2023-12-26T21:36:47.824557+0000","last_active":"2023-12-26T21:36:48.841778+0000","last_peered":"2023-12-26T21:36:48.841778+0000","last_clean":"2023-12-26T21:36:48.841778+0000","last_became_active":"2023-12-26T21:36:47.824200+0000","last_became_peered":"2023-12-26T21:36:47.824200+0000","last_unstale":"2023-12-26T21:36:48.841778+0000","last_undegraded":"2023-12-26T21:36:48.841778+0000","last_fullsized":"2023-12-26T21:36:48.841778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:13.097046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.827346+0000","last_change":"2023-12-26T21:36:48.823425+0000","last_active":"2023-12-26T21:36:48.827346+0000","last_peered":"2023-12-26T21:36:48.827346+0000","last_clean":"2023-12-26T21:36:48.827346+0000","last_became_active":"2023-12-26T21:36:48.823148+0000","last_became_peered":"2023-12-26T21:36:48.823148+0000","last_unstale":"2023-12-26T21:36:48.827346+0000","last_undegraded":"2023-12-26T21:36:48.827346+0000","last_fullsized":"2023-12-26T21:36:48.827346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:35.691705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841375+0000","last_change":"2023-12-26T21:36:47.826954+0000","last_active":"2023-12-26T21:36:48.841375+0000","last_peered":"2023-12-26T21:36:48.841375+0000","last_clean":"2023-12-26T21:36:48.841375+0000","last_became_active":"2023-12-26T21:36:47.826691+0000","last_became_peered":"2023-12-26T21:36:47.826691+0000","last_unstale":"2023-12-26T21:36:48.841375+0000","last_undegraded":"2023-12-26T21:36:48.841375+0000","last_fullsized":"2023-12-26T21:36:48.841375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:24.145739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843898+0000","last_change":"2023-12-26T21:36:48.830893+0000","last_active":"2023-12-26T21:36:48.843898+0000","last_peered":"2023-12-26T21:36:48.843898+0000","last_clean":"2023-12-26T21:36:48.843898+0000","last_became_active":"2023-12-26T21:36:48.830553+0000","last_became_peered":"2023-12-26T21:36:48.830553+0000","last_unstale":"2023-12-26T21:36:48.843898+0000","last_undegraded":"2023-12-26T21:36:48.843898+0000","last_fullsized":"2023-12-26T21:36:48.843898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:40:07.698471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841849+0000","last_change":"2023-12-26T21:36:47.828216+0000","last_active":"2023-12-26T21:36:48.841849+0000","last_peered":"2023-12-26T21:36:48.841849+0000","last_clean":"2023-12-26T21:36:48.841849+0000","last_became_active":"2023-12-26T21:36:47.827950+0000","last_became_peered":"2023-12-26T21:36:47.827950+0000","last_unstale":"2023-12-26T21:36:48.841849+0000","last_undegraded":"2023-12-26T21:36:48.841849+0000","last_fullsized":"2023-12-26T21:36:48.841849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:23.069341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828328+0000","last_change":"2023-12-26T21:36:48.820490+0000","last_active":"2023-12-26T21:36:48.828328+0000","last_peered":"2023-12-26T21:36:48.828328+0000","last_clean":"2023-12-26T21:36:48.828328+0000","last_became_active":"2023-12-26T21:36:48.820298+0000","last_became_peered":"2023-12-26T21:36:48.820298+0000","last_unstale":"2023-12-26T21:36:48.828328+0000","last_undegraded":"2023-12-26T21:36:48.828328+0000","last_fullsized":"2023-12-26T21:36:48.828328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:25.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.830175+0000","last_change":"2023-12-26T21:36:47.830245+0000","last_active":"2023-12-26T21:36:48.830175+0000","last_peered":"2023-12-26T21:36:48.830175+0000","last_clean":"2023-12-26T21:36:48.830175+0000","last_became_active":"2023-12-26T21:36:47.829970+0000","last_became_peered":"2023-12-26T21:36:47.829970+0000","last_unstale":"2023-12-26T21:36:48.830175+0000","last_undegraded":"2023-12-26T21:36:48.830175+0000","last_fullsized":"2023-12-26T21:36:48.830175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:13.329724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843818+0000","last_change":"2023-12-26T21:36:48.839949+0000","last_active":"2023-12-26T21:36:48.843818+0000","last_peered":"2023-12-26T21:36:48.843818+0000","last_clean":"2023-12-26T21:36:48.843818+0000","last_became_active":"2023-12-26T21:36:48.839655+0000","last_became_peered":"2023-12-26T21:36:48.839655+0000","last_unstale":"2023-12-26T21:36:48.843818+0000","last_undegraded":"2023-12-26T21:36:48.843818+0000","last_fullsized":"2023-12-26T21:36:48.843818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:43:57.839515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.809369+0000","last_change":"2023-12-26T21:36:47.828451+0000","last_active":"2023-12-26T21:36:48.809369+0000","last_peered":"2023-12-26T21:36:48.809369+0000","last_clean":"2023-12-26T21:36:48.809369+0000","last_became_active":"2023-12-26T21:36:47.828158+0000","last_became_peered":"2023-12-26T21:36:47.828158+0000","last_unstale":"2023-12-26T21:36:48.809369+0000","last_undegraded":"2023-12-26T21:36:48.809369+0000","last_fullsized":"2023-12-26T21:36:48.809369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:23.069341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843628+0000","last_change":"2023-12-26T21:36:48.840894+0000","last_active":"2023-12-26T21:36:48.843628+0000","last_peered":"2023-12-26T21:36:48.843628+0000","last_clean":"2023-12-26T21:36:48.843628+0000","last_became_active":"2023-12-26T21:36:48.840284+0000","last_became_peered":"2023-12-26T21:36:48.840284+0000","last_unstale":"2023-12-26T21:36:48.843628+0000","last_undegraded":"2023-12-26T21:36:48.843628+0000","last_fullsized":"2023-12-26T21:36:48.843628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:23.349206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816543+0000","last_change":"2023-12-26T21:36:47.821233+0000","last_active":"2023-12-26T21:36:48.816543+0000","last_peered":"2023-12-26T21:36:48.816543+0000","last_clean":"2023-12-26T21:36:48.816543+0000","last_became_active":"2023-12-26T21:36:47.820129+0000","last_became_peered":"2023-12-26T21:36:47.820129+0000","last_unstale":"2023-12-26T21:36:48.816543+0000","last_undegraded":"2023-12-26T21:36:48.816543+0000","last_fullsized":"2023-12-26T21:36:48.816543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:30.052367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838583+0000","last_change":"2023-12-26T21:36:48.829638+0000","last_active":"2023-12-26T21:36:48.838583+0000","last_peered":"2023-12-26T21:36:48.838583+0000","last_clean":"2023-12-26T21:36:48.838583+0000","last_became_active":"2023-12-26T21:36:48.829154+0000","last_became_peered":"2023-12-26T21:36:48.829154+0000","last_unstale":"2023-12-26T21:36:48.838583+0000","last_undegraded":"2023-12-26T21:36:48.838583+0000","last_fullsized":"2023-12-26T21:36:48.838583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:45:55.685017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.803440+0000","last_change":"2023-12-26T21:36:47.818993+0000","last_active":"2023-12-26T21:36:48.803440+0000","last_peered":"2023-12-26T21:36:48.803440+0000","last_clean":"2023-12-26T21:36:48.803440+0000","last_became_active":"2023-12-26T21:36:47.818731+0000","last_became_peered":"2023-12-26T21:36:47.818731+0000","last_unstale":"2023-12-26T21:36:48.803440+0000","last_undegraded":"2023-12-26T21:36:48.803440+0000","last_fullsized":"2023-12-26T21:36:48.803440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:14.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.840833+0000","last_change":"2023-12-26T21:36:48.837014+0000","last_active":"2023-12-26T21:36:48.840833+0000","last_peered":"2023-12-26T21:36:48.840833+0000","last_clean":"2023-12-26T21:36:48.840833+0000","last_became_active":"2023-12-26T21:36:48.830649+0000","last_became_peered":"2023-12-26T21:36:48.830649+0000","last_unstale":"2023-12-26T21:36:48.840833+0000","last_undegraded":"2023-12-26T21:36:48.840833+0000","last_fullsized":"2023-12-26T21:36:48.840833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:07.755765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829625+0000","last_change":"2023-12-26T21:36:47.827036+0000","last_active":"2023-12-26T21:36:48.829625+0000","last_peered":"2023-12-26T21:36:48.829625+0000","last_clean":"2023-12-26T21:36:48.829625+0000","last_became_active":"2023-12-26T21:36:47.826746+0000","last_became_peered":"2023-12-26T21:36:47.826746+0000","last_unstale":"2023-12-26T21:36:48.829625+0000","last_undegraded":"2023-12-26T21:36:48.829625+0000","last_fullsized":"2023-12-26T21:36:48.829625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:28.026360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841896+0000","last_change":"2023-12-26T21:36:48.837290+0000","last_active":"2023-12-26T21:36:48.841896+0000","last_peered":"2023-12-26T21:36:48.841896+0000","last_clean":"2023-12-26T21:36:48.841896+0000","last_became_active":"2023-12-26T21:36:48.836991+0000","last_became_peered":"2023-12-26T21:36:48.836991+0000","last_unstale":"2023-12-26T21:36:48.841896+0000","last_undegraded":"2023-12-26T21:36:48.841896+0000","last_fullsized":"2023-12-26T21:36:48.841896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835423+0000","last_change":"2023-12-26T21:36:47.823376+0000","last_active":"2023-12-26T21:36:48.835423+0000","last_peered":"2023-12-26T21:36:48.835423+0000","last_clean":"2023-12-26T21:36:48.835423+0000","last_became_active":"2023-12-26T21:36:47.822986+0000","last_became_peered":"2023-12-26T21:36:47.822986+0000","last_unstale":"2023-12-26T21:36:48.835423+0000","last_undegraded":"2023-12-26T21:36:48.835423+0000","last_fullsized":"2023-12-26T21:36:48.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:45:26.314194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829780+0000","last_change":"2023-12-26T21:36:48.825556+0000","last_active":"2023-12-26T21:36:48.829780+0000","last_peered":"2023-12-26T21:36:48.829780+0000","last_clean":"2023-12-26T21:36:48.829780+0000","last_became_active":"2023-12-26T21:36:48.825233+0000","last_became_peered":"2023-12-26T21:36:48.825233+0000","last_unstale":"2023-12-26T21:36:48.829780+0000","last_undegraded":"2023-12-26T21:36:48.829780+0000","last_fullsized":"2023-12-26T21:36:48.829780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:40.692719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.848461+0000","last_change":"2023-12-26T21:36:47.821098+0000","last_active":"2023-12-26T21:36:48.848461+0000","last_peered":"2023-12-26T21:36:48.848461+0000","last_clean":"2023-12-26T21:36:48.848461+0000","last_became_active":"2023-12-26T21:36:47.820639+0000","last_became_peered":"2023-12-26T21:36:47.820639+0000","last_unstale":"2023-12-26T21:36:48.848461+0000","last_undegraded":"2023-12-26T21:36:48.848461+0000","last_fullsized":"2023-12-26T21:36:48.848461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:26.330467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833737+0000","last_change":"2023-12-26T21:36:48.828635+0000","last_active":"2023-12-26T21:36:48.833737+0000","last_peered":"2023-12-26T21:36:48.833737+0000","last_clean":"2023-12-26T21:36:48.833737+0000","last_became_active":"2023-12-26T21:36:48.828382+0000","last_became_peered":"2023-12-26T21:36:48.828382+0000","last_unstale":"2023-12-26T21:36:48.833737+0000","last_undegraded":"2023-12-26T21:36:48.833737+0000","last_fullsized":"2023-12-26T21:36:48.833737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:59.578493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835877+0000","last_change":"2023-12-26T21:36:47.830348+0000","last_active":"2023-12-26T21:36:48.835877+0000","last_peered":"2023-12-26T21:36:48.835877+0000","last_clean":"2023-12-26T21:36:48.835877+0000","last_became_active":"2023-12-26T21:36:47.830053+0000","last_became_peered":"2023-12-26T21:36:47.830053+0000","last_unstale":"2023-12-26T21:36:48.835877+0000","last_undegraded":"2023-12-26T21:36:48.835877+0000","last_fullsized":"2023-12-26T21:36:48.835877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:26.330467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.832805+0000","last_change":"2023-12-26T21:36:48.830118+0000","last_active":"2023-12-26T21:36:48.832805+0000","last_peered":"2023-12-26T21:36:48.832805+0000","last_clean":"2023-12-26T21:36:48.832805+0000","last_became_active":"2023-12-26T21:36:48.829843+0000","last_became_peered":"2023-12-26T21:36:48.829843+0000","last_unstale":"2023-12-26T21:36:48.832805+0000","last_undegraded":"2023-12-26T21:36:48.832805+0000","last_fullsized":"2023-12-26T21:36:48.832805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817442+0000","last_change":"2023-12-26T21:36:47.822624+0000","last_active":"2023-12-26T21:36:48.817442+0000","last_peered":"2023-12-26T21:36:48.817442+0000","last_clean":"2023-12-26T21:36:48.817442+0000","last_became_active":"2023-12-26T21:36:47.822048+0000","last_became_peered":"2023-12-26T21:36:47.822048+0000","last_unstale":"2023-12-26T21:36:48.817442+0000","last_undegraded":"2023-12-26T21:36:48.817442+0000","last_fullsized":"2023-12-26T21:36:48.817442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:24.145739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846368+0000","last_change":"2023-12-26T21:36:48.837022+0000","last_active":"2023-12-26T21:36:48.846368+0000","last_peered":"2023-12-26T21:36:48.846368+0000","last_clean":"2023-12-26T21:36:48.846368+0000","last_became_active":"2023-12-26T21:36:48.836736+0000","last_became_peered":"2023-12-26T21:36:48.836736+0000","last_unstale":"2023-12-26T21:36:48.846368+0000","last_undegraded":"2023-12-26T21:36:48.846368+0000","last_fullsized":"2023-12-26T21:36:48.846368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:57.196069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826918+0000","last_change":"2023-12-26T21:36:47.829979+0000","last_active":"2023-12-26T21:36:48.826918+0000","last_peered":"2023-12-26T21:36:48.826918+0000","last_clean":"2023-12-26T21:36:48.826918+0000","last_became_active":"2023-12-26T21:36:47.829685+0000","last_became_peered":"2023-12-26T21:36:47.829685+0000","last_unstale":"2023-12-26T21:36:48.826918+0000","last_undegraded":"2023-12-26T21:36:48.826918+0000","last_fullsized":"2023-12-26T21:36:48.826918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831660+0000","last_change":"2023-12-26T21:36:48.828008+0000","last_active":"2023-12-26T21:36:48.831660+0000","last_peered":"2023-12-26T21:36:48.831660+0000","last_clean":"2023-12-26T21:36:48.831660+0000","last_became_active":"2023-12-26T21:36:48.827743+0000","last_became_peered":"2023-12-26T21:36:48.827743+0000","last_unstale":"2023-12-26T21:36:48.831660+0000","last_undegraded":"2023-12-26T21:36:48.831660+0000","last_fullsized":"2023-12-26T21:36:48.831660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:14.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839801+0000","last_change":"2023-12-26T21:36:47.815814+0000","last_active":"2023-12-26T21:36:48.839801+0000","last_peered":"2023-12-26T21:36:48.839801+0000","last_clean":"2023-12-26T21:36:48.839801+0000","last_became_active":"2023-12-26T21:36:47.815480+0000","last_became_peered":"2023-12-26T21:36:47.815480+0000","last_unstale":"2023-12-26T21:36:48.839801+0000","last_undegraded":"2023-12-26T21:36:48.839801+0000","last_fullsized":"2023-12-26T21:36:48.839801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:18:28.266978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843897+0000","last_change":"2023-12-26T21:36:48.833364+0000","last_active":"2023-12-26T21:36:48.843897+0000","last_peered":"2023-12-26T21:36:48.843897+0000","last_clean":"2023-12-26T21:36:48.843897+0000","last_became_active":"2023-12-26T21:36:48.833087+0000","last_became_peered":"2023-12-26T21:36:48.833087+0000","last_unstale":"2023-12-26T21:36:48.843897+0000","last_undegraded":"2023-12-26T21:36:48.843897+0000","last_fullsized":"2023-12-26T21:36:48.843897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:05:49.639640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846496+0000","last_change":"2023-12-26T21:36:47.823499+0000","last_active":"2023-12-26T21:36:48.846496+0000","last_peered":"2023-12-26T21:36:48.846496+0000","last_clean":"2023-12-26T21:36:48.846496+0000","last_became_active":"2023-12-26T21:36:47.819416+0000","last_became_peered":"2023-12-26T21:36:47.819416+0000","last_unstale":"2023-12-26T21:36:48.846496+0000","last_undegraded":"2023-12-26T21:36:48.846496+0000","last_fullsized":"2023-12-26T21:36:48.846496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:13.329724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839537+0000","last_change":"2023-12-26T21:36:48.824260+0000","last_active":"2023-12-26T21:36:48.839537+0000","last_peered":"2023-12-26T21:36:48.839537+0000","last_clean":"2023-12-26T21:36:48.839537+0000","last_became_active":"2023-12-26T21:36:48.824000+0000","last_became_peered":"2023-12-26T21:36:48.824000+0000","last_unstale":"2023-12-26T21:36:48.839537+0000","last_undegraded":"2023-12-26T21:36:48.839537+0000","last_fullsized":"2023-12-26T21:36:48.839537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:24.151016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.812438+0000","last_change":"2023-12-26T21:36:47.829031+0000","last_active":"2023-12-26T21:36:48.812438+0000","last_peered":"2023-12-26T21:36:48.812438+0000","last_clean":"2023-12-26T21:36:48.812438+0000","last_became_active":"2023-12-26T21:36:47.828711+0000","last_became_peered":"2023-12-26T21:36:47.828711+0000","last_unstale":"2023-12-26T21:36:48.812438+0000","last_undegraded":"2023-12-26T21:36:48.812438+0000","last_fullsized":"2023-12-26T21:36:48.812438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.822529+0000","last_change":"2023-12-26T21:36:48.817001+0000","last_active":"2023-12-26T21:36:48.822529+0000","last_peered":"2023-12-26T21:36:48.822529+0000","last_clean":"2023-12-26T21:36:48.822529+0000","last_became_active":"2023-12-26T21:36:48.816746+0000","last_became_peered":"2023-12-26T21:36:48.816746+0000","last_unstale":"2023-12-26T21:36:48.822529+0000","last_undegraded":"2023-12-26T21:36:48.822529+0000","last_fullsized":"2023-12-26T21:36:48.822529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:56.619479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817332+0000","last_change":"2023-12-26T21:36:47.828501+0000","last_active":"2023-12-26T21:36:48.817332+0000","last_peered":"2023-12-26T21:36:48.817332+0000","last_clean":"2023-12-26T21:36:48.817332+0000","last_became_active":"2023-12-26T21:36:47.828257+0000","last_became_peered":"2023-12-26T21:36:47.828257+0000","last_unstale":"2023-12-26T21:36:48.817332+0000","last_undegraded":"2023-12-26T21:36:48.817332+0000","last_fullsized":"2023-12-26T21:36:48.817332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:17.677425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.837624+0000","last_change":"2023-12-26T21:36:48.834334+0000","last_active":"2023-12-26T21:36:48.837624+0000","last_peered":"2023-12-26T21:36:48.837624+0000","last_clean":"2023-12-26T21:36:48.837624+0000","last_became_active":"2023-12-26T21:36:48.834070+0000","last_became_peered":"2023-12-26T21:36:48.834070+0000","last_unstale":"2023-12-26T21:36:48.837624+0000","last_undegraded":"2023-12-26T21:36:48.837624+0000","last_fullsized":"2023-12-26T21:36:48.837624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:21:25.931065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843647+0000","last_change":"2023-12-26T21:36:48.841064+0000","last_active":"2023-12-26T21:36:48.843647+0000","last_peered":"2023-12-26T21:36:48.843647+0000","last_clean":"2023-12-26T21:36:48.843647+0000","last_became_active":"2023-12-26T21:36:48.840654+0000","last_became_peered":"2023-12-26T21:36:48.840654+0000","last_unstale":"2023-12-26T21:36:48.843647+0000","last_undegraded":"2023-12-26T21:36:48.843647+0000","last_fullsized":"2023-12-26T21:36:48.843647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.804661+0000","last_change":"2023-12-26T21:36:47.822491+0000","last_active":"2023-12-26T21:36:48.804661+0000","last_peered":"2023-12-26T21:36:48.804661+0000","last_clean":"2023-12-26T21:36:48.804661+0000","last_became_active":"2023-12-26T21:36:47.822150+0000","last_became_peered":"2023-12-26T21:36:47.822150+0000","last_unstale":"2023-12-26T21:36:48.804661+0000","last_undegraded":"2023-12-26T21:36:48.804661+0000","last_fullsized":"2023-12-26T21:36:48.804661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:08:16.186961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.834826+0000","last_change":"2023-12-26T21:36:48.828798+0000","last_active":"2023-12-26T21:36:48.834826+0000","last_peered":"2023-12-26T21:36:48.834826+0000","last_clean":"2023-12-26T21:36:48.834826+0000","last_became_active":"2023-12-26T21:36:48.828093+0000","last_became_peered":"2023-12-26T21:36:48.828093+0000","last_unstale":"2023-12-26T21:36:48.834826+0000","last_undegraded":"2023-12-26T21:36:48.834826+0000","last_fullsized":"2023-12-26T21:36:48.834826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:19:55.017056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817551+0000","last_change":"2023-12-26T21:36:47.825144+0000","last_active":"2023-12-26T21:36:48.817551+0000","last_peered":"2023-12-26T21:36:48.817551+0000","last_clean":"2023-12-26T21:36:48.817551+0000","last_became_active":"2023-12-26T21:36:47.824896+0000","last_became_peered":"2023-12-26T21:36:47.824896+0000","last_unstale":"2023-12-26T21:36:48.817551+0000","last_undegraded":"2023-12-26T21:36:48.817551+0000","last_fullsized":"2023-12-26T21:36:48.817551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:03:48.097670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.803182+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:36:48.803182+0000","last_peered":"2023-12-26T21:36:48.803182+0000","last_clean":"2023-12-26T21:36:48.803182+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:36:48.803182+0000","last_undegraded":"2023-12-26T21:36:48.803182+0000","last_fullsized":"2023-12-26T21:36:48.803182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.818212+0000","last_change":"2023-12-26T21:36:48.816469+0000","last_active":"2023-12-26T21:36:48.818212+0000","last_peered":"2023-12-26T21:36:48.818212+0000","last_clean":"2023-12-26T21:36:48.818212+0000","last_became_active":"2023-12-26T21:36:48.816190+0000","last_became_peered":"2023-12-26T21:36:48.816190+0000","last_unstale":"2023-12-26T21:36:48.818212+0000","last_undegraded":"2023-12-26T21:36:48.818212+0000","last_fullsized":"2023-12-26T21:36:48.818212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:35.691705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816437+0000","last_change":"2023-12-26T21:36:47.822572+0000","last_active":"2023-12-26T21:36:48.816437+0000","last_peered":"2023-12-26T21:36:48.816437+0000","last_clean":"2023-12-26T21:36:48.816437+0000","last_became_active":"2023-12-26T21:36:47.822214+0000","last_became_peered":"2023-12-26T21:36:47.822214+0000","last_unstale":"2023-12-26T21:36:48.816437+0000","last_undegraded":"2023-12-26T21:36:48.816437+0000","last_fullsized":"2023-12-26T21:36:48.816437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:34.685280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816507+0000","last_change":"2023-12-26T21:36:47.826782+0000","last_active":"2023-12-26T21:36:48.816507+0000","last_peered":"2023-12-26T21:36:48.816507+0000","last_clean":"2023-12-26T21:36:48.816507+0000","last_became_active":"2023-12-26T21:36:47.825248+0000","last_became_peered":"2023-12-26T21:36:47.825248+0000","last_unstale":"2023-12-26T21:36:48.816507+0000","last_undegraded":"2023-12-26T21:36:48.816507+0000","last_fullsized":"2023-12-26T21:36:48.816507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:28.026360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831901+0000","last_change":"2023-12-26T21:36:48.827788+0000","last_active":"2023-12-26T21:36:48.831901+0000","last_peered":"2023-12-26T21:36:48.831901+0000","last_clean":"2023-12-26T21:36:48.831901+0000","last_became_active":"2023-12-26T21:36:48.827487+0000","last_became_peered":"2023-12-26T21:36:48.827487+0000","last_unstale":"2023-12-26T21:36:48.831901+0000","last_undegraded":"2023-12-26T21:36:48.831901+0000","last_fullsized":"2023-12-26T21:36:48.831901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:22.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.818519+0000","last_change":"2023-12-26T21:36:47.830347+0000","last_active":"2023-12-26T21:36:48.818519+0000","last_peered":"2023-12-26T21:36:48.818519+0000","last_clean":"2023-12-26T21:36:48.818519+0000","last_became_active":"2023-12-26T21:36:47.830037+0000","last_became_peered":"2023-12-26T21:36:47.830037+0000","last_unstale":"2023-12-26T21:36:48.818519+0000","last_undegraded":"2023-12-26T21:36:48.818519+0000","last_fullsized":"2023-12-26T21:36:48.818519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:55.613054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838484+0000","last_change":"2023-12-26T21:36:48.829585+0000","last_active":"2023-12-26T21:36:48.838484+0000","last_peered":"2023-12-26T21:36:48.838484+0000","last_clean":"2023-12-26T21:36:48.838484+0000","last_became_active":"2023-12-26T21:36:48.829170+0000","last_became_peered":"2023-12-26T21:36:48.829170+0000","last_unstale":"2023-12-26T21:36:48.838484+0000","last_undegraded":"2023-12-26T21:36:48.838484+0000","last_fullsized":"2023-12-26T21:36:48.838484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:25:31.563553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844919+0000","last_change":"2023-12-26T21:36:47.832588+0000","last_active":"2023-12-26T21:36:48.844919+0000","last_peered":"2023-12-26T21:36:48.844919+0000","last_clean":"2023-12-26T21:36:48.844919+0000","last_became_active":"2023-12-26T21:36:47.832310+0000","last_became_peered":"2023-12-26T21:36:47.832310+0000","last_unstale":"2023-12-26T21:36:48.844919+0000","last_undegraded":"2023-12-26T21:36:48.844919+0000","last_fullsized":"2023-12-26T21:36:48.844919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:22:54.126845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839479+0000","last_change":"2023-12-26T21:36:48.830882+0000","last_active":"2023-12-26T21:36:48.839479+0000","last_peered":"2023-12-26T21:36:48.839479+0000","last_clean":"2023-12-26T21:36:48.839479+0000","last_became_active":"2023-12-26T21:36:48.830587+0000","last_became_peered":"2023-12-26T21:36:48.830587+0000","last_unstale":"2023-12-26T21:36:48.839479+0000","last_undegraded":"2023-12-26T21:36:48.839479+0000","last_fullsized":"2023-12-26T21:36:48.839479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:54.746974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826954+0000","last_change":"2023-12-26T21:36:47.826141+0000","last_active":"2023-12-26T21:36:48.826954+0000","last_peered":"2023-12-26T21:36:48.826954+0000","last_clean":"2023-12-26T21:36:48.826954+0000","last_became_active":"2023-12-26T21:36:47.825876+0000","last_became_peered":"2023-12-26T21:36:47.825876+0000","last_unstale":"2023-12-26T21:36:48.826954+0000","last_undegraded":"2023-12-26T21:36:48.826954+0000","last_fullsized":"2023-12-26T21:36:48.826954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:12.184779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843719+0000","last_change":"2023-12-26T21:36:48.829405+0000","last_active":"2023-12-26T21:36:48.843719+0000","last_peered":"2023-12-26T21:36:48.843719+0000","last_clean":"2023-12-26T21:36:48.843719+0000","last_became_active":"2023-12-26T21:36:48.829169+0000","last_became_peered":"2023-12-26T21:36:48.829169+0000","last_unstale":"2023-12-26T21:36:48.843719+0000","last_undegraded":"2023-12-26T21:36:48.843719+0000","last_fullsized":"2023-12-26T21:36:48.843719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:42:15.750967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.819202+0000","last_change":"2023-12-26T21:36:47.830820+0000","last_active":"2023-12-26T21:36:48.819202+0000","last_peered":"2023-12-26T21:36:48.819202+0000","last_clean":"2023-12-26T21:36:48.819202+0000","last_became_active":"2023-12-26T21:36:47.830514+0000","last_became_peered":"2023-12-26T21:36:47.830514+0000","last_unstale":"2023-12-26T21:36:48.819202+0000","last_undegraded":"2023-12-26T21:36:48.819202+0000","last_fullsized":"2023-12-26T21:36:48.819202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:53.472268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839858+0000","last_change":"2023-12-26T21:36:48.832982+0000","last_active":"2023-12-26T21:36:48.839858+0000","last_peered":"2023-12-26T21:36:48.839858+0000","last_clean":"2023-12-26T21:36:48.839858+0000","last_became_active":"2023-12-26T21:36:48.831969+0000","last_became_peered":"2023-12-26T21:36:48.831969+0000","last_unstale":"2023-12-26T21:36:48.839858+0000","last_undegraded":"2023-12-26T21:36:48.839858+0000","last_fullsized":"2023-12-26T21:36:48.839858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:54.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.812495+0000","last_change":"2023-12-26T21:36:47.826837+0000","last_active":"2023-12-26T21:36:48.812495+0000","last_peered":"2023-12-26T21:36:48.812495+0000","last_clean":"2023-12-26T21:36:48.812495+0000","last_became_active":"2023-12-26T21:36:47.826479+0000","last_became_peered":"2023-12-26T21:36:47.826479+0000","last_unstale":"2023-12-26T21:36:48.812495+0000","last_undegraded":"2023-12-26T21:36:48.812495+0000","last_fullsized":"2023-12-26T21:36:48.812495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:05.573200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817414+0000","last_change":"2023-12-26T21:36:48.813654+0000","last_active":"2023-12-26T21:36:48.817414+0000","last_peered":"2023-12-26T21:36:48.817414+0000","last_clean":"2023-12-26T21:36:48.817414+0000","last_became_active":"2023-12-26T21:36:48.813344+0000","last_became_peered":"2023-12-26T21:36:48.813344+0000","last_unstale":"2023-12-26T21:36:48.817414+0000","last_undegraded":"2023-12-26T21:36:48.817414+0000","last_fullsized":"2023-12-26T21:36:48.817414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:54.478693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839092+0000","last_change":"2023-12-26T21:36:47.820913+0000","last_active":"2023-12-26T21:36:48.839092+0000","last_peered":"2023-12-26T21:36:48.839092+0000","last_clean":"2023-12-26T21:36:48.839092+0000","last_became_active":"2023-12-26T21:36:47.820617+0000","last_became_peered":"2023-12-26T21:36:47.820617+0000","last_unstale":"2023-12-26T21:36:48.839092+0000","last_undegraded":"2023-12-26T21:36:48.839092+0000","last_fullsized":"2023-12-26T21:36:48.839092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:41.047506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829801+0000","last_change":"2023-12-26T21:36:48.826060+0000","last_active":"2023-12-26T21:36:48.829801+0000","last_peered":"2023-12-26T21:36:48.829801+0000","last_clean":"2023-12-26T21:36:48.829801+0000","last_became_active":"2023-12-26T21:36:48.825817+0000","last_became_peered":"2023-12-26T21:36:48.825817+0000","last_unstale":"2023-12-26T21:36:48.829801+0000","last_undegraded":"2023-12-26T21:36:48.829801+0000","last_fullsized":"2023-12-26T21:36:48.829801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.809730+0000","last_change":"2023-12-26T21:36:47.827562+0000","last_active":"2023-12-26T21:36:48.809730+0000","last_peered":"2023-12-26T21:36:48.809730+0000","last_clean":"2023-12-26T21:36:48.809730+0000","last_became_active":"2023-12-26T21:36:47.827284+0000","last_became_peered":"2023-12-26T21:36:47.827284+0000","last_unstale":"2023-12-26T21:36:48.809730+0000","last_undegraded":"2023-12-26T21:36:48.809730+0000","last_fullsized":"2023-12-26T21:36:48.809730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843878+0000","last_change":"2023-12-26T21:36:48.829818+0000","last_active":"2023-12-26T21:36:48.843878+0000","last_peered":"2023-12-26T21:36:48.843878+0000","last_clean":"2023-12-26T21:36:48.843878+0000","last_became_active":"2023-12-26T21:36:48.829087+0000","last_became_peered":"2023-12-26T21:36:48.829087+0000","last_unstale":"2023-12-26T21:36:48.843878+0000","last_undegraded":"2023-12-26T21:36:48.843878+0000","last_fullsized":"2023-12-26T21:36:48.843878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:37:13.353768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.842747+0000","last_change":"2023-12-26T21:36:47.815095+0000","last_active":"2023-12-26T21:36:48.842747+0000","last_peered":"2023-12-26T21:36:48.842747+0000","last_clean":"2023-12-26T21:36:48.842747+0000","last_became_active":"2023-12-26T21:36:47.814728+0000","last_became_peered":"2023-12-26T21:36:47.814728+0000","last_unstale":"2023-12-26T21:36:48.842747+0000","last_undegraded":"2023-12-26T21:36:48.842747+0000","last_fullsized":"2023-12-26T21:36:48.842747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:14.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846566+0000","last_change":"2023-12-26T21:36:48.837315+0000","last_active":"2023-12-26T21:36:48.846566+0000","last_peered":"2023-12-26T21:36:48.846566+0000","last_clean":"2023-12-26T21:36:48.846566+0000","last_became_active":"2023-12-26T21:36:48.837067+0000","last_became_peered":"2023-12-26T21:36:48.837067+0000","last_unstale":"2023-12-26T21:36:48.846566+0000","last_undegraded":"2023-12-26T21:36:48.846566+0000","last_fullsized":"2023-12-26T21:36:48.846566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:28.883031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.834582+0000","last_change":"2023-12-26T21:36:47.825290+0000","last_active":"2023-12-26T21:36:48.834582+0000","last_peered":"2023-12-26T21:36:48.834582+0000","last_clean":"2023-12-26T21:36:48.834582+0000","last_became_active":"2023-12-26T21:36:47.824594+0000","last_became_peered":"2023-12-26T21:36:47.824594+0000","last_unstale":"2023-12-26T21:36:48.834582+0000","last_undegraded":"2023-12-26T21:36:48.834582+0000","last_fullsized":"2023-12-26T21:36:48.834582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:06.749340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843529+0000","last_change":"2023-12-26T21:36:48.835342+0000","last_active":"2023-12-26T21:36:48.843529+0000","last_peered":"2023-12-26T21:36:48.843529+0000","last_clean":"2023-12-26T21:36:48.843529+0000","last_became_active":"2023-12-26T21:36:48.830463+0000","last_became_peered":"2023-12-26T21:36:48.830463+0000","last_unstale":"2023-12-26T21:36:48.843529+0000","last_undegraded":"2023-12-26T21:36:48.843529+0000","last_fullsized":"2023-12-26T21:36:48.843529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.824113+0000","last_change":"2023-12-26T21:36:48.817508+0000","last_active":"2023-12-26T21:36:48.824113+0000","last_peered":"2023-12-26T21:36:48.824113+0000","last_clean":"2023-12-26T21:36:48.824113+0000","last_became_active":"2023-12-26T21:36:48.817269+0000","last_became_peered":"2023-12-26T21:36:48.817269+0000","last_unstale":"2023-12-26T21:36:48.824113+0000","last_undegraded":"2023-12-26T21:36:48.824113+0000","last_fullsized":"2023-12-26T21:36:48.824113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:54.478693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831298+0000","last_change":"2023-12-26T21:36:48.820601+0000","last_active":"2023-12-26T21:36:48.831298+0000","last_peered":"2023-12-26T21:36:48.831298+0000","last_clean":"2023-12-26T21:36:48.831298+0000","last_became_active":"2023-12-26T21:36:48.819293+0000","last_became_peered":"2023-12-26T21:36:48.819293+0000","last_unstale":"2023-12-26T21:36:48.831298+0000","last_undegraded":"2023-12-26T21:36:48.831298+0000","last_fullsized":"2023-12-26T21:36:48.831298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:26.207089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844324+0000","last_change":"2023-12-26T21:36:48.834258+0000","last_active":"2023-12-26T21:36:48.844324+0000","last_peered":"2023-12-26T21:36:48.844324+0000","last_clean":"2023-12-26T21:36:48.844324+0000","last_became_active":"2023-12-26T21:36:48.834005+0000","last_became_peered":"2023-12-26T21:36:48.834005+0000","last_unstale":"2023-12-26T21:36:48.844324+0000","last_undegraded":"2023-12-26T21:36:48.844324+0000","last_fullsized":"2023-12-26T21:36:48.844324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:15.953197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841827+0000","last_change":"2023-12-26T21:36:48.834061+0000","last_active":"2023-12-26T21:36:48.841827+0000","last_peered":"2023-12-26T21:36:48.841827+0000","last_clean":"2023-12-26T21:36:48.841827+0000","last_became_active":"2023-12-26T21:36:48.821177+0000","last_became_peered":"2023-12-26T21:36:48.821177+0000","last_unstale":"2023-12-26T21:36:48.841827+0000","last_undegraded":"2023-12-26T21:36:48.841827+0000","last_fullsized":"2023-12-26T21:36:48.841827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:21:25.931065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828251+0000","last_change":"2023-12-26T21:36:48.823718+0000","last_active":"2023-12-26T21:36:48.828251+0000","last_peered":"2023-12-26T21:36:48.828251+0000","last_clean":"2023-12-26T21:36:48.828251+0000","last_became_active":"2023-12-26T21:36:48.823422+0000","last_became_peered":"2023-12-26T21:36:48.823422+0000","last_unstale":"2023-12-26T21:36:48.828251+0000","last_undegraded":"2023-12-26T21:36:48.828251+0000","last_fullsized":"2023-12-26T21:36:48.828251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:42:23.347556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826781+0000","last_change":"2023-12-26T21:36:48.821819+0000","last_active":"2023-12-26T21:36:48.826781+0000","last_peered":"2023-12-26T21:36:48.826781+0000","last_clean":"2023-12-26T21:36:48.826781+0000","last_became_active":"2023-12-26T21:36:48.821527+0000","last_became_peered":"2023-12-26T21:36:48.821527+0000","last_unstale":"2023-12-26T21:36:48.826781+0000","last_undegraded":"2023-12-26T21:36:48.826781+0000","last_fullsized":"2023-12-26T21:36:48.826781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:31.058793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844139+0000","last_change":"2023-12-26T21:36:48.841186+0000","last_active":"2023-12-26T21:36:48.844139+0000","last_peered":"2023-12-26T21:36:48.844139+0000","last_clean":"2023-12-26T21:36:48.844139+0000","last_became_active":"2023-12-26T21:36:48.839955+0000","last_became_peered":"2023-12-26T21:36:48.839955+0000","last_unstale":"2023-12-26T21:36:48.844139+0000","last_undegraded":"2023-12-26T21:36:48.844139+0000","last_fullsized":"2023-12-26T21:36:48.844139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:35.307950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841419+0000","last_change":"2023-12-26T21:36:48.832556+0000","last_active":"2023-12-26T21:36:48.841419+0000","last_peered":"2023-12-26T21:36:48.841419+0000","last_clean":"2023-12-26T21:36:48.841419+0000","last_became_active":"2023-12-26T21:36:48.830061+0000","last_became_peered":"2023-12-26T21:36:48.830061+0000","last_unstale":"2023-12-26T21:36:48.841419+0000","last_undegraded":"2023-12-26T21:36:48.841419+0000","last_fullsized":"2023-12-26T21:36:48.841419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:22.938456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846674+0000","last_change":"2023-12-26T21:36:48.842574+0000","last_active":"2023-12-26T21:36:48.846674+0000","last_peered":"2023-12-26T21:36:48.846674+0000","last_clean":"2023-12-26T21:36:48.846674+0000","last_became_active":"2023-12-26T21:36:48.842273+0000","last_became_peered":"2023-12-26T21:36:48.842273+0000","last_unstale":"2023-12-26T21:36:48.846674+0000","last_undegraded":"2023-12-26T21:36:48.846674+0000","last_fullsized":"2023-12-26T21:36:48.846674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:17:59.482430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.840028+0000","last_change":"2023-12-26T21:36:48.826270+0000","last_active":"2023-12-26T21:36:48.840028+0000","last_peered":"2023-12-26T21:36:48.840028+0000","last_clean":"2023-12-26T21:36:48.840028+0000","last_became_active":"2023-12-26T21:36:48.825303+0000","last_became_peered":"2023-12-26T21:36:48.825303+0000","last_unstale":"2023-12-26T21:36:48.840028+0000","last_undegraded":"2023-12-26T21:36:48.840028+0000","last_fullsized":"2023-12-26T21:36:48.840028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:22.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828348+0000","last_change":"2023-12-26T21:36:48.823561+0000","last_active":"2023-12-26T21:36:48.828348+0000","last_peered":"2023-12-26T21:36:48.828348+0000","last_clean":"2023-12-26T21:36:48.828348+0000","last_became_active":"2023-12-26T21:36:48.823270+0000","last_became_peered":"2023-12-26T21:36:48.823270+0000","last_unstale":"2023-12-26T21:36:48.828348+0000","last_undegraded":"2023-12-26T21:36:48.828348+0000","last_fullsized":"2023-12-26T21:36:48.828348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:29.032786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839419+0000","last_change":"2023-12-26T21:36:48.828687+0000","last_active":"2023-12-26T21:36:48.839419+0000","last_peered":"2023-12-26T21:36:48.839419+0000","last_clean":"2023-12-26T21:36:48.839419+0000","last_became_active":"2023-12-26T21:36:48.828405+0000","last_became_peered":"2023-12-26T21:36:48.828405+0000","last_unstale":"2023-12-26T21:36:48.839419+0000","last_undegraded":"2023-12-26T21:36:48.839419+0000","last_fullsized":"2023-12-26T21:36:48.839419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:37.832638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841288+0000","last_change":"2023-12-26T21:36:48.835378+0000","last_active":"2023-12-26T21:36:48.841288+0000","last_peered":"2023-12-26T21:36:48.841288+0000","last_clean":"2023-12-26T21:36:48.841288+0000","last_became_active":"2023-12-26T21:36:48.835049+0000","last_became_peered":"2023-12-26T21:36:48.835049+0000","last_unstale":"2023-12-26T21:36:48.841288+0000","last_undegraded":"2023-12-26T21:36:48.841288+0000","last_fullsized":"2023-12-26T21:36:48.841288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:22.938456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":76900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":667360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":79080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":78436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":77024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298088,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073752,"statfs":{"total":96636764160,"available":96331522048,"internally_reserved":0,"allocated":819200,"data_stored":667338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":77454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:36:53.794 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T21:36:53.794 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-26T21:36:53.795 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-26T21:36:53.795 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-26T21:36:53.795 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-26T21:36:54.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.023+0000 7f3b75bff700 1 Processor -- start 2023-12-26T21:36:54.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.023+0000 7f3b75bff700 1 -- start start 2023-12-26T21:36:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.024+0000 7f3b75bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b7013c510 0x7f3b700c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.024+0000 7f3b75bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b7006b8c0 0x7f3b7006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b75bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3b700cc100 0x7f3b700cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b75bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7006c270 con 0x7f3b7006b8c0 2023-12-26T21:36:54.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b75bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b700686c0 con 0x7f3b700cc100 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b75bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b70069520 con 0x7f3b7013c510 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b67fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b7006b8c0 0x7f3b7006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b67fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b7006b8c0 0x7f3b7006bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57038/0 (socket says 172.21.15.131:57038) 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b67fff700 1 -- 172.21.15.131:0/2805520955 learned_addr learned my addr 172.21.15.131:0/2805520955 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b6ffff700 1 --2- 172.21.15.131:0/2805520955 >> v2:172.21.15.190:3300/0 conn(0x7f3b700cc100 0x7f3b700cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b6f7fe700 1 --2- 172.21.15.131:0/2805520955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b7013c510 0x7f3b700c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b67fff700 1 -- 172.21.15.131:0/2805520955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b7013c510 msgr2=0x7f3b700c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b67fff700 1 --2- 172.21.15.131:0/2805520955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b7013c510 0x7f3b700c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b67fff700 1 -- 172.21.15.131:0/2805520955 >> v2:172.21.15.190:3300/0 conn(0x7f3b700cc100 msgr2=0x7f3b700cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b67fff700 1 --2- 172.21.15.131:0/2805520955 >> v2:172.21.15.190:3300/0 conn(0x7f3b700cc100 0x7f3b700cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b67fff700 1 -- 172.21.15.131:0/2805520955 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70068910 con 0x7f3b7006b8c0 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.025+0000 7f3b67fff700 1 --2- 172.21.15.131:0/2805520955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 0x7f3b7006bd30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b7006f130 tx=0x7f3b58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aca1859a916cc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:54.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b6effd700 1 -- 172.21.15.131:0/2805520955 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800b860 con 0x7f3b7006b8c0 2023-12-26T21:36:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b6effd700 1 -- 172.21.15.131:0/2805520955 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5800b9c0 con 0x7f3b7006b8c0 2023-12-26T21:36:54.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b6effd700 1 -- 172.21.15.131:0/2805520955 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800bd60 con 0x7f3b7006b8c0 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b75bff700 1 -- 172.21.15.131:0/2805520955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 msgr2=0x7f3b7006bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b75bff700 1 --2- 172.21.15.131:0/2805520955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 0x7f3b7006bd30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b7006f130 tx=0x7f3b58009250 comp rx=0 tx=0).stop 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b75bff700 1 -- 172.21.15.131:0/2805520955 shutdown_connections 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b75bff700 1 --2- 172.21.15.131:0/2805520955 >> v2:172.21.15.190:3300/0 conn(0x7f3b700cc100 0x7f3b700cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b75bff700 1 --2- 172.21.15.131:0/2805520955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b7013c510 0x7f3b700c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b75bff700 1 --2- 172.21.15.131:0/2805520955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 0x7f3b7006bd30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b75bff700 1 -- 172.21.15.131:0/2805520955 >> 172.21.15.131:0/2805520955 conn(0x7f3b700ce3e0 msgr2=0x7f3b700ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b75bff700 1 -- 172.21.15.131:0/2805520955 shutdown_connections 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.026+0000 7f3b75bff700 1 -- 172.21.15.131:0/2805520955 wait complete. 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.028+0000 7f3b75bff700 1 Processor -- start 2023-12-26T21:36:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.028+0000 7f3b75bff700 1 -- start start 2023-12-26T21:36:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b75bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 0x7f3b700b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b75bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b700cc100 0x7f3b7013af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b75bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b7013c510 0x7f3b7015b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b75bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b700b3280 con 0x7f3b7006b8c0 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b75bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b700b16c0 con 0x7f3b7013c510 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b75bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b700b24e0 con 0x7f3b700cc100 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 0x7f3b700b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 0x7f3b700b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57050/0 (socket says 172.21.15.131:57050) 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b6f7fe700 1 -- 172.21.15.131:0/3222722908 learned_addr learned my addr 172.21.15.131:0/3222722908 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b6ffff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b7013c510 0x7f3b7015b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b67fff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b700cc100 0x7f3b7013af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b6f7fe700 1 -- 172.21.15.131:0/3222722908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b700cc100 msgr2=0x7f3b7013af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b6f7fe700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b700cc100 0x7f3b7013af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b6f7fe700 1 -- 172.21.15.131:0/3222722908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b7013c510 msgr2=0x7f3b7015b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b6f7fe700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b7013c510 0x7f3b7015b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.029+0000 7f3b6f7fe700 1 -- 172.21.15.131:0/3222722908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b580094f0 con 0x7f3b7006b8c0 2023-12-26T21:36:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.030+0000 7f3b6f7fe700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 0x7f3b700b1180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000d890 tx=0x7f3b60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.030+0000 7f3b677fe700 1 -- 172.21.15.131:0/3222722908 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60018070 con 0x7f3b7006b8c0 2023-12-26T21:36:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.030+0000 7f3b677fe700 1 -- 172.21.15.131:0/3222722908 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b60005a00 con 0x7f3b7006b8c0 2023-12-26T21:36:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.030+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7013b4b0 con 0x7f3b7006b8c0 2023-12-26T21:36:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.030+0000 7f3b677fe700 1 -- 172.21.15.131:0/3222722908 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60005dc0 con 0x7f3b7006b8c0 2023-12-26T21:36:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.030+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b700b1980 con 0x7f3b7006b8c0 2023-12-26T21:36:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.031+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b700657e0 con 0x7f3b7006b8c0 2023-12-26T21:36:54.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.032+0000 7f3b677fe700 1 -- 172.21.15.131:0/3222722908 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3b6000b810 con 0x7f3b7006b8c0 2023-12-26T21:36:54.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.033+0000 7f3b677fe700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3b50066f30 0x7f3b500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.033+0000 7f3b677fe700 1 -- 172.21.15.131:0/3222722908 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3b60084790 con 0x7f3b7006b8c0 2023-12-26T21:36:54.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.033+0000 7f3b67fff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3b50066f30 0x7f3b500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.034+0000 7f3b67fff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3b50066f30 0x7f3b500693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b7006f130 tx=0x7f3b58017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:54.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.037+0000 7f3b677fe700 1 -- 172.21.15.131:0/3222722908 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b60051960 con 0x7f3b7006b8c0 2023-12-26T21:36:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.283+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3b70068bf0 con 0x7f3b7006b8c0 2023-12-26T21:36:54.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.286+0000 7f3b677fe700 1 -- 172.21.15.131:0/3222722908 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163706 (secure 0 0 0) 0x7f3b60054f80 con 0x7f3b7006b8c0 2023-12-26T21:36:54.288 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:54.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3b50066f30 msgr2=0x7f3b500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3b50066f30 0x7f3b500693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b7006f130 tx=0x7f3b58017000 comp rx=0 tx=0).stop 2023-12-26T21:36:54.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 msgr2=0x7f3b700b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 0x7f3b700b1180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000d890 tx=0x7f3b60008040 comp rx=0 tx=0).stop 2023-12-26T21:36:54.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 shutdown_connections 2023-12-26T21:36:54.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b7013c510 0x7f3b7015b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b7006b8c0 0x7f3b700b1180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3b50066f30 0x7f3b500693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 --2- 172.21.15.131:0/3222722908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b700cc100 0x7f3b7013af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 >> 172.21.15.131:0/3222722908 conn(0x7f3b700ce3e0 msgr2=0x7f3b700cad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:54.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 shutdown_connections 2023-12-26T21:36:54.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.289+0000 7f3b75bff700 1 -- 172.21.15.131:0/3222722908 wait complete. 2023-12-26T21:36:54.307 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":5,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":39288},{"type":"v1","addr":"172.21.15.131:6833","nonce":39288}]},"active_addr":"172.21.15.131:6833/39288","active_change":"2023-12-26T21:36:30.981656+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":2098522634}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":509990189}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1288038854}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":518003331}]}]} 2023-12-26T21:36:54.310 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-26T21:36:54.310 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-26T21:36:54.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:54.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.500+0000 7fec2c7ff700 1 Processor -- start 2023-12-26T21:36:54.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.500+0000 7fec2c7ff700 1 -- start start 2023-12-26T21:36:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.501+0000 7fec2c7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fec240cb320 0x7fec24069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.501+0000 7fec2c7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec24156940 con 0x7fec240b0e60 2023-12-26T21:36:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.501+0000 7fec2c7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec240ae500 con 0x7fec2406a1e0 2023-12-26T21:36:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.501+0000 7fec2c7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240c8cc0 con 0x7fec240cb320 2023-12-26T21:36:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.501+0000 7fec2ad9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fec240b0e60 0x7fec240c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47824/0 (socket says 172.21.15.131:47824) 2023-12-26T21:36:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.501+0000 7fec2ad9c700 1 -- 172.21.15.131:0/3593861246 learned_addr learned my addr 172.21.15.131:0/3593861246 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec2a59b700 1 --2- 172.21.15.131:0/3593861246 >> v2:172.21.15.190:3300/0 conn(0x7fec240cb320 0x7fec24069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558728325 0 0) 0x7fec24156940 con 0x7fec240b0e60 2023-12-26T21:36:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec10003570 con 0x7fec240b0e60 2023-12-26T21:36:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3400571709 0 0) 0x7fec10003570 con 0x7fec240b0e60 2023-12-26T21:36:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 >> v2:172.21.15.190:3300/0 conn(0x7fec240cb320 msgr2=0x7fec24069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 --2- 172.21.15.131:0/3593861246 >> v2:172.21.15.190:3300/0 conn(0x7fec240cb320 0x7fec24069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 >> v1:172.21.15.190:6789/0 conn(0x7fec2406a1e0 legacy=0x7fec240b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240b0280 con 0x7fec240b0e60 2023-12-26T21:36:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fec1c0017f0 con 0x7fec240b0e60 2023-12-26T21:36:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 >> v1:172.21.15.131:6789/0 conn(0x7fec240b0e60 legacy=0x7fec240c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec10003a60 0x7fec10003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec10004810 0x7fec10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec100051c0 0x7fec1000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240c8cc0 con 0x7fec10004810 2023-12-26T21:36:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1000c050 con 0x7fec10003a60 2023-12-26T21:36:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1000e420 con 0x7fec100051c0 2023-12-26T21:36:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec2a59b700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec10003a60 0x7fec10003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec29d9a700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec10004810 0x7fec10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.502+0000 7fec2ad9c700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec100051c0 0x7fec1000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec2a59b700 1 -- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec100051c0 msgr2=0x7fec1000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec2a59b700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec100051c0 0x7fec1000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec2a59b700 1 -- 172.21.15.131:0/3593861246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec10004810 msgr2=0x7fec10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec2a59b700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec10004810 0x7fec10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec2a59b700 1 -- 172.21.15.131:0/3593861246 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec10003570 con 0x7fec10003a60 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec29d9a700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec10004810 0x7fec10004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec2ad9c700 1 -- 172.21.15.131:0/3593861246 reap_dead start 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec2a59b700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec10003a60 0x7fec10003e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fec20009c30 tx=0x7fec200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3227241406c76086 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec200095b0 con 0x7fec10003a60 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2000e710 con 0x7fec10003a60 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.503+0000 7fec29599700 1 -- 172.21.15.131:0/3593861246 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2000ead0 con 0x7fec10003a60 2023-12-26T21:36:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.504+0000 7fec2c7ff700 1 -- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec10003a60 msgr2=0x7fec10003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.504+0000 7fec2c7ff700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec10003a60 0x7fec10003e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fec20009c30 tx=0x7fec200091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.504+0000 7fec2c7ff700 1 -- 172.21.15.131:0/3593861246 shutdown_connections 2023-12-26T21:36:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.504+0000 7fec2c7ff700 1 --2- 172.21.15.131:0/3593861246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec10003a60 0x7fec10003e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.504+0000 7fec2c7ff700 1 -- 172.21.15.131:0/3593861246 >> 172.21.15.131:0/3593861246 conn(0x7fec24068040 msgr2=0x7fec24069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.504+0000 7fec2c7ff700 1 -- 172.21.15.131:0/3593861246 shutdown_connections 2023-12-26T21:36:54.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.504+0000 7fec2c7ff700 1 -- 172.21.15.131:0/3593861246 wait complete. 2023-12-26T21:36:54.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.505+0000 7fec2c7ff700 1 Processor -- start 2023-12-26T21:36:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.506+0000 7fec2c7ff700 1 -- start start 2023-12-26T21:36:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.506+0000 7fec2c7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec10003a60 0x7fec241522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec2c7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec240aaba0 0x7fec241578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec2c7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec240ab010 0x7fec240a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec2c7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240a64f0 con 0x7fec240aaba0 2023-12-26T21:36:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec2c7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240a58e0 con 0x7fec240ab010 2023-12-26T21:36:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec2c7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec240a5bc0 con 0x7fec10003a60 2023-12-26T21:36:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec29d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec240aaba0 0x7fec241578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec29d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec240aaba0 0x7fec241578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57064/0 (socket says 172.21.15.131:57064) 2023-12-26T21:36:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec29d9a700 1 -- 172.21.15.131:0/326461222 learned_addr learned my addr 172.21.15.131:0/326461222 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec2ad9c700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec240ab010 0x7fec240a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec2a59b700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec10003a60 0x7fec241522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec29d9a700 1 -- 172.21.15.131:0/326461222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec10003a60 msgr2=0x7fec241522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec29d9a700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec10003a60 0x7fec241522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec29d9a700 1 -- 172.21.15.131:0/326461222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec240ab010 msgr2=0x7fec240a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec29d9a700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec240ab010 0x7fec240a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec29d9a700 1 -- 172.21.15.131:0/326461222 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240a5ea0 con 0x7fec240aaba0 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec2ad9c700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec240ab010 0x7fec240a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.507+0000 7fec29d9a700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec240aaba0 0x7fec241578d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec14007880 tx=0x7fec14009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.508+0000 7fec1a7fc700 1 -- 172.21.15.131:0/326461222 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1400d850 con 0x7fec240aaba0 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.508+0000 7fec1a7fc700 1 -- 172.21.15.131:0/326461222 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1400d9d0 con 0x7fec240aaba0 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.508+0000 7fec1a7fc700 1 -- 172.21.15.131:0/326461222 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1400dd90 con 0x7fec240aaba0 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.508+0000 7fec2c7ff700 1 -- 172.21.15.131:0/326461222 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec240a6180 con 0x7fec240aaba0 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.508+0000 7fec2c7ff700 1 -- 172.21.15.131:0/326461222 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec241531d0 con 0x7fec240aaba0 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.509+0000 7fec07fff700 1 -- 172.21.15.131:0/326461222 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec240657e0 con 0x7fec240aaba0 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.510+0000 7fec1a7fc700 1 -- 172.21.15.131:0/326461222 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fec14013600 con 0x7fec240aaba0 2023-12-26T21:36:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.511+0000 7fec1a7fc700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec08066f30 0x7fec080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.511+0000 7fec1a7fc700 1 -- 172.21.15.131:0/326461222 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec140846d0 con 0x7fec240aaba0 2023-12-26T21:36:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.511+0000 7fec2a59b700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec08066f30 0x7fec080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.512+0000 7fec2a59b700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec08066f30 0x7fec080693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec200091e0 tx=0x7fec2001a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.515+0000 7fec1a7fc700 1 -- 172.21.15.131:0/326461222 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec140518a0 con 0x7fec240aaba0 2023-12-26T21:36:54.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.706+0000 7fec07fff700 1 -- 172.21.15.131:0/326461222 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec241531d0 con 0x7fec240aaba0 2023-12-26T21:36:54.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.707+0000 7fec1a7fc700 1 -- 172.21.15.131:0/326461222 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fec241531d0 con 0x7fec240aaba0 2023-12-26T21:36:54.707 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:54.707 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 -- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec08066f30 msgr2=0x7fec080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec08066f30 0x7fec080693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec200091e0 tx=0x7fec2001a000 comp rx=0 tx=0).stop 2023-12-26T21:36:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 -- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec240aaba0 msgr2=0x7fec241578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec240aaba0 0x7fec241578d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec14007880 tx=0x7fec14009040 comp rx=0 tx=0).stop 2023-12-26T21:36:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 -- 172.21.15.131:0/326461222 shutdown_connections 2023-12-26T21:36:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec240ab010 0x7fec240a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec08066f30 0x7fec080693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec10003a60 0x7fec241522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 --2- 172.21.15.131:0/326461222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec240aaba0 0x7fec241578d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 -- 172.21.15.131:0/326461222 >> 172.21.15.131:0/326461222 conn(0x7fec24068040 msgr2=0x7fec240ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 -- 172.21.15.131:0/326461222 shutdown_connections 2023-12-26T21:36:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.709+0000 7fec07fff700 1 -- 172.21.15.131:0/326461222 wait complete. 2023-12-26T21:36:54.727 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-26T21:36:54.727 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:36:54.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.916+0000 7f1be39ff700 1 Processor -- start 2023-12-26T21:36:54.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.916+0000 7f1be39ff700 1 -- start start 2023-12-26T21:36:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.917+0000 7f1be39ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc1596a0 0x7f1bdc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.917+0000 7f1be39ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1bdc0b0870 0x7f1bdc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.917+0000 7f1be39ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bdc0afcd0 con 0x7f1bdc159a70 2023-12-26T21:36:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.917+0000 7f1be39ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bdc0b3e70 con 0x7f1bdc0b0870 2023-12-26T21:36:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.917+0000 7f1be39ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc0b0cd0 con 0x7f1bdc1596a0 2023-12-26T21:36:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be1f9c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1bdc0b0870 0x7f1bdc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be179b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc1596a0 0x7f1bdc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be179b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc1596a0 0x7f1bdc0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57234/0 (socket says 172.21.15.131:57234) 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be0f9a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1bdc159a70 0x7f1bdc0b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39708/0 (socket says 172.21.15.131:39708) 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be179b700 1 -- 172.21.15.131:0/3765059864 learned_addr learned my addr 172.21.15.131:0/3765059864 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be1f9c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1bdc0b0870 0x7f1bdc0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53308/0 (socket says 172.21.15.131:53308) 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be179b700 1 -- 172.21.15.131:0/3765059864 >> v2:172.21.15.190:3300/0 conn(0x7f1bdc0b0870 msgr2=0x7f1bdc0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be179b700 1 --2- 172.21.15.131:0/3765059864 >> v2:172.21.15.190:3300/0 conn(0x7f1bdc0b0870 0x7f1bdc0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be179b700 1 -- 172.21.15.131:0/3765059864 >> v1:172.21.15.190:6789/0 conn(0x7f1bdc159a70 legacy=0x7f1bdc0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be179b700 1 -- 172.21.15.131:0/3765059864 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc0b2e00 con 0x7f1bdc1596a0 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.918+0000 7f1be179b700 1 --2- 172.21.15.131:0/3765059864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc1596a0 0x7f1bdc0b2380 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80099b0 tx=0x7f1bd8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a6165ec375141a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1bd3fff700 1 -- 172.21.15.131:0/3765059864 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8009ea0 con 0x7f1bdc1596a0 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1bd3fff700 1 -- 172.21.15.131:0/3765059864 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd800b860 con 0x7f1bdc1596a0 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1bd3fff700 1 -- 172.21.15.131:0/3765059864 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800bc00 con 0x7f1bdc1596a0 2023-12-26T21:36:54.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1be39ff700 1 -- 172.21.15.131:0/3765059864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc1596a0 msgr2=0x7f1bdc0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1be39ff700 1 --2- 172.21.15.131:0/3765059864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc1596a0 0x7f1bdc0b2380 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80099b0 tx=0x7f1bd8009250 comp rx=0 tx=0).stop 2023-12-26T21:36:54.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1be39ff700 1 -- 172.21.15.131:0/3765059864 shutdown_connections 2023-12-26T21:36:54.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1be39ff700 1 --2- 172.21.15.131:0/3765059864 >> v2:172.21.15.190:3300/0 conn(0x7f1bdc0b0870 0x7f1bdc0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1be39ff700 1 --2- 172.21.15.131:0/3765059864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc1596a0 0x7f1bdc0b2380 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1be39ff700 1 -- 172.21.15.131:0/3765059864 >> 172.21.15.131:0/3765059864 conn(0x7f1bdc06ece0 msgr2=0x7f1bdc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:54.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1be39ff700 1 -- 172.21.15.131:0/3765059864 shutdown_connections 2023-12-26T21:36:54.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.919+0000 7f1be39ff700 1 -- 172.21.15.131:0/3765059864 wait complete. 2023-12-26T21:36:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.921+0000 7f1be39ff700 1 Processor -- start 2023-12-26T21:36:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.921+0000 7f1be39ff700 1 -- start start 2023-12-26T21:36:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be39ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bdc0b0870 0x7f1bdc0c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be39ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc1596a0 0x7f1bdc0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be39ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc159a70 0x7f1bdc0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be39ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc0cbb10 con 0x7f1bdc0b0870 2023-12-26T21:36:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be39ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc0cb200 con 0x7f1bdc1596a0 2023-12-26T21:36:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be39ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc0c5cd0 con 0x7f1bdc159a70 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be179b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bdc0b0870 0x7f1bdc0c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be179b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bdc0b0870 0x7f1bdc0c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44708/0 (socket says 172.21.15.131:44708) 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be179b700 1 -- 172.21.15.131:0/3433583042 learned_addr learned my addr 172.21.15.131:0/3433583042 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be0f9a700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc1596a0 0x7f1bdc0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be179b700 1 -- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc159a70 msgr2=0x7f1bdc0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be1f9c700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc159a70 0x7f1bdc0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be179b700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc159a70 0x7f1bdc0d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be179b700 1 -- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc1596a0 msgr2=0x7f1bdc0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be179b700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc1596a0 0x7f1bdc0d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be179b700 1 -- 172.21.15.131:0/3433583042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd80094f0 con 0x7f1bdc0b0870 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be1f9c700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc159a70 0x7f1bdc0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.922+0000 7f1be0f9a700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc1596a0 0x7f1bdc0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.923+0000 7f1be179b700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bdc0b0870 0x7f1bdc0c5230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc15e440 tx=0x7f1bd8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.923+0000 7f1bd17fa700 1 -- 172.21.15.131:0/3433583042 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800bbe0 con 0x7f1bdc0b0870 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.923+0000 7f1bd17fa700 1 -- 172.21.15.131:0/3433583042 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd8011420 con 0x7f1bdc0b0870 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.923+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bdc0cb4c0 con 0x7f1bdc0b0870 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.923+0000 7f1bd17fa700 1 -- 172.21.15.131:0/3433583042 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80117c0 con 0x7f1bdc0b0870 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.923+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bdc0cd260 con 0x7f1bdc0b0870 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.924+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bdc0cb960 con 0x7f1bdc0b0870 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.925+0000 7f1bd17fa700 1 -- 172.21.15.131:0/3433583042 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1bd8011bf0 con 0x7f1bdc0b0870 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.926+0000 7f1bd17fa700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1bc0066f30 0x7f1bc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.926+0000 7f1bd17fa700 1 -- 172.21.15.131:0/3433583042 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1bd8085f60 con 0x7f1bdc0b0870 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.926+0000 7f1be0f9a700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1bc0066f30 0x7f1bc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:54.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.927+0000 7f1be0f9a700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1bc0066f30 0x7f1bc00693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0cd040 tx=0x7f1bcc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:54.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:54.930+0000 7f1bd17fa700 1 -- 172.21.15.131:0/3433583042 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1bd8053130 con 0x7f1bdc0b0870 2023-12-26T21:36:55.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.126+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bdc0cbdf0 con 0x7f1bdc0b0870 2023-12-26T21:36:55.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.127+0000 7f1bd17fa700 1 -- 172.21.15.131:0/3433583042 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f1bd8056750 con 0x7f1bdc0b0870 2023-12-26T21:36:55.126 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:55.127 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:36:55.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.129+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1bc0066f30 msgr2=0x7f1bc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.129+0000 7f1be39ff700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1bc0066f30 0x7f1bc00693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0cd040 tx=0x7f1bcc009040 comp rx=0 tx=0).stop 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.129+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bdc0b0870 msgr2=0x7f1bdc0c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.129+0000 7f1be39ff700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bdc0b0870 0x7f1bdc0c5230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc15e440 tx=0x7f1bd8017000 comp rx=0 tx=0).stop 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.130+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 shutdown_connections 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.130+0000 7f1be39ff700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1bc0066f30 0x7f1bc00693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.130+0000 7f1be39ff700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bdc159a70 0x7f1bdc0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.130+0000 7f1be39ff700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bdc0b0870 0x7f1bdc0c5230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.130+0000 7f1be39ff700 1 --2- 172.21.15.131:0/3433583042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bdc1596a0 0x7f1bdc0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.130+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 >> 172.21.15.131:0/3433583042 conn(0x7f1bdc06ece0 msgr2=0x7f1bdc0b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.130+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 shutdown_connections 2023-12-26T21:36:55.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.130+0000 7f1be39ff700 1 -- 172.21.15.131:0/3433583042 wait complete. 2023-12-26T21:36:55.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-26T21:36:55.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-26T21:36:55.149 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-26T21:36:55.149 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-26T21:36:55.149 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-26T21:36:55.150 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-26T21:36:55.150 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-26T21:36:55.150 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-26T21:36:55.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.486+0000 7f316ddff700 1 Processor -- start 2023-12-26T21:36:55.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.486+0000 7f316ddff700 1 -- start start 2023-12-26T21:36:55.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.489+0000 7f316ddff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3160050430 0x7f3160006d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.489+0000 7f316ddff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3160054690 con 0x7f316004ffb0 2023-12-26T21:36:55.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.489+0000 7f316ddff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31600061f0 con 0x7f316000c250 2023-12-26T21:36:55.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.489+0000 7f316ddff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3160053df0 con 0x7f3160050430 2023-12-26T21:36:55.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.489+0000 7f31677fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f316000c250 0x7f3160053330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39714/0 (socket says 172.21.15.131:39714) 2023-12-26T21:36:55.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.490+0000 7f31677fe700 1 -- 172.21.15.131:0/3264823224 learned_addr learned my addr 172.21.15.131:0/3264823224 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.490+0000 7f3167fff700 1 --2- 172.21.15.131:0/3264823224 >> v2:172.21.15.190:3300/0 conn(0x7f3160050430 0x7f3160006d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.490+0000 7f2f49fff700 1 Processor -- start 2023-12-26T21:36:55.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.490+0000 7f2f49fff700 1 -- start start 2023-12-26T21:36:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f31667fc700 1 -- 172.21.15.131:0/3264823224 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162905978 0 0) 0x7f31600061f0 con 0x7f316000c250 2023-12-26T21:36:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f31667fc700 1 -- 172.21.15.131:0/3264823224 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3158004680 con 0x7f316000c250 2023-12-26T21:36:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f2f49fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f44159400 0x7f2f441597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f2f49fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f440b3dc0 0x7f2f440b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f2f49fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f440cea50 con 0x7f2f440b6180 2023-12-26T21:36:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f2f49fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f440cecf0 con 0x7f2f440b3dc0 2023-12-26T21:36:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f2f49fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f440cefd0 con 0x7f2f44159400 2023-12-26T21:36:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f3167fff700 1 -- 172.21.15.131:0/3264823224 >> v1:172.21.15.190:6789/0 conn(0x7f316000c250 legacy=0x7f3160053330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f31667fc700 1 -- 172.21.15.131:0/3264823224 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2745058283 0 0) 0x7f3158004680 con 0x7f316000c250 2023-12-26T21:36:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f3167fff700 1 -- 172.21.15.131:0/3264823224 >> v1:172.21.15.131:6789/0 conn(0x7f316004ffb0 legacy=0x7f3160006680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f3167fff700 1 -- 172.21.15.131:0/3264823224 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31600540d0 con 0x7f3160050430 2023-12-26T21:36:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.493+0000 7f2f43fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f440b3dc0 0x7f2f440b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f2f42ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2f440b6180 0x7f2f440cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49366/0 (socket says 172.21.15.131:49366) 2023-12-26T21:36:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f2f42ffd700 1 -- 172.21.15.131:0/62506251 learned_addr learned my addr 172.21.15.131:0/62506251 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f3167fff700 1 --2- 172.21.15.131:0/3264823224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3160050430 0x7f3160006d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3154009860 tx=0x7f31540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e0b0dd8b75c9e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f31667fc700 1 -- 172.21.15.131:0/3264823224 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c001470 con 0x7f3160050430 2023-12-26T21:36:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f31667fc700 1 -- 172.21.15.131:0/3264823224 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3154011420 con 0x7f3160050430 2023-12-26T21:36:55.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f2f437fe700 1 --2- 172.21.15.131:0/62506251 >> v2:172.21.15.190:3300/0 conn(0x7f2f44159400 0x7f2f441597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f31667fc700 1 -- 172.21.15.131:0/3264823224 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31540117c0 con 0x7f3160050430 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f2f43fff700 1 -- 172.21.15.131:0/62506251 >> v2:172.21.15.190:3300/0 conn(0x7f2f44159400 msgr2=0x7f2f441597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f2f427fc700 1 -- 172.21.15.131:0/62506251 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006531495 0 0) 0x7f2f440cea50 con 0x7f2f440b6180 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f2f43fff700 1 --2- 172.21.15.131:0/62506251 >> v2:172.21.15.190:3300/0 conn(0x7f2f44159400 0x7f2f441597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f2f43fff700 1 -- 172.21.15.131:0/62506251 >> v1:172.21.15.131:6789/0 conn(0x7f2f440b6180 legacy=0x7f2f440cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f2f43fff700 1 -- 172.21.15.131:0/62506251 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f440cf2b0 con 0x7f2f440b3dc0 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.494+0000 7f2f437fe700 1 --2- 172.21.15.131:0/62506251 >> v2:172.21.15.190:3300/0 conn(0x7f2f44159400 0x7f2f441597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.495+0000 7f2f43fff700 1 --2- 172.21.15.131:0/62506251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f440b3dc0 0x7f2f440b4260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f440d1720 tx=0x7f2f380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596668b1bf2ebbc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.495+0000 7f2f427fc700 1 -- 172.21.15.131:0/62506251 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800b6b0 con 0x7f2f440b3dc0 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.495+0000 7f316ddff700 1 -- 172.21.15.131:0/3264823224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3160050430 msgr2=0x7f3160006d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.495+0000 7f316ddff700 1 --2- 172.21.15.131:0/3264823224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3160050430 0x7f3160006d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3154009860 tx=0x7f31540091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.495+0000 7f2f427fc700 1 -- 172.21.15.131:0/62506251 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3800b810 con 0x7f2f440b3dc0 2023-12-26T21:36:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.495+0000 7f2f427fc700 1 -- 172.21.15.131:0/62506251 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800bbb0 con 0x7f2f440b3dc0 2023-12-26T21:36:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.496+0000 7f2f49fff700 1 -- 172.21.15.131:0/62506251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f440b3dc0 msgr2=0x7f2f440b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.496+0000 7f2f49fff700 1 --2- 172.21.15.131:0/62506251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f440b3dc0 0x7f2f440b4260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f440d1720 tx=0x7f2f380091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.497+0000 7f316ddff700 1 -- 172.21.15.131:0/3264823224 shutdown_connections 2023-12-26T21:36:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.497+0000 7f316ddff700 1 --2- 172.21.15.131:0/3264823224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3160050430 0x7f3160006d90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.497+0000 7f316ddff700 1 -- 172.21.15.131:0/3264823224 >> 172.21.15.131:0/3264823224 conn(0x7f3160054da0 msgr2=0x7f31600551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.497+0000 7f316ddff700 1 -- 172.21.15.131:0/3264823224 shutdown_connections 2023-12-26T21:36:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.497+0000 7f316ddff700 1 -- 172.21.15.131:0/3264823224 wait complete. 2023-12-26T21:36:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.499+0000 7f316ddff700 1 Processor -- start 2023-12-26T21:36:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.499+0000 7f2f49fff700 1 -- 172.21.15.131:0/62506251 shutdown_connections 2023-12-26T21:36:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.499+0000 7f2f49fff700 1 --2- 172.21.15.131:0/62506251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f440b3dc0 0x7f2f440b4260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.499+0000 7f2f49fff700 1 --2- 172.21.15.131:0/62506251 >> v2:172.21.15.190:3300/0 conn(0x7f2f44159400 0x7f2f441597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.500+0000 7f2f49fff700 1 -- 172.21.15.131:0/62506251 >> 172.21.15.131:0/62506251 conn(0x7f2f4406ece0 msgr2=0x7f2f4406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.500+0000 7f316ddff700 1 -- start start 2023-12-26T21:36:55.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.501+0000 7f316ddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316000c250 0x7f31600dedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.501+0000 7f316ddff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f316004ffb0 0x7f31600d2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.501+0000 7f316ddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3160050430 0x7f31600d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f316ddff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600d2bb0 con 0x7f316004ffb0 2023-12-26T21:36:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f316ddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600d2e90 con 0x7f3160050430 2023-12-26T21:36:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f316ddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316004cec0 con 0x7f316000c250 2023-12-26T21:36:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f31677fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316000c250 0x7f31600dedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f31677fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316000c250 0x7f31600dedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57250/0 (socket says 172.21.15.131:57250) 2023-12-26T21:36:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f3167fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3160050430 0x7f31600d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f31677fe700 1 -- 172.21.15.131:0/2271878467 learned_addr learned my addr 172.21.15.131:0/2271878467 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f31677fe700 1 -- 172.21.15.131:0/2271878467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3160050430 msgr2=0x7f31600d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.503+0000 7f2f49fff700 1 -- 172.21.15.131:0/62506251 shutdown_connections 2023-12-26T21:36:55.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f31677fe700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3160050430 0x7f31600d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f31677fe700 1 -- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f316004ffb0 msgr2=0x7f31600d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f31677fe700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f316004ffb0 0x7f31600d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.502+0000 7f31677fe700 1 -- 172.21.15.131:0/2271878467 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3154009420 con 0x7f316000c250 2023-12-26T21:36:55.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.503+0000 7f3167fff700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3160050430 0x7f31600d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:55.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.503+0000 7f31677fe700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316000c250 0x7f31600dedc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f315c00cb90 tx=0x7f315c001ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.503+0000 7f31537fe700 1 -- 172.21.15.131:0/2271878467 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c007150 con 0x7f316000c250 2023-12-26T21:36:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.503+0000 7f31537fe700 1 -- 172.21.15.131:0/2271878467 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c0072b0 con 0x7f316000c250 2023-12-26T21:36:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.503+0000 7f31537fe700 1 -- 172.21.15.131:0/2271878467 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c007670 con 0x7f316000c250 2023-12-26T21:36:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.503+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316004d1a0 con 0x7f316000c250 2023-12-26T21:36:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.503+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31600d2840 con 0x7f316000c250 2023-12-26T21:36:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.503+0000 7f2f49fff700 1 -- 172.21.15.131:0/62506251 wait complete. 2023-12-26T21:36:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.505+0000 7f2f49fff700 1 Processor -- start 2023-12-26T21:36:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.505+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31600054a0 con 0x7f316000c250 2023-12-26T21:36:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.506+0000 7f31537fe700 1 -- 172.21.15.131:0/2271878467 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f315c004a40 con 0x7f316000c250 2023-12-26T21:36:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.505+0000 7f2f49fff700 1 -- start start 2023-12-26T21:36:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.507+0000 7f2f49fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f440b3dc0 0x7f2f441678b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.507+0000 7f2f49fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44159400 0x7f2f4416ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.507+0000 7f2f49fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f4416f010 0x7f2f4414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.507+0000 7f2f49fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440b0f80 con 0x7f2f44159400 2023-12-26T21:36:55.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.507+0000 7f2f49fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440b1ac0 con 0x7f2f4416f010 2023-12-26T21:36:55.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.507+0000 7f2f49fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440b1190 con 0x7f2f440b3dc0 2023-12-26T21:36:55.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.508+0000 7f2f42ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44159400 0x7f2f4416ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.508+0000 7f2f43fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f4416f010 0x7f2f4414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.508+0000 7f2f43fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f4416f010 0x7f2f4414fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53342/0 (socket says 172.21.15.131:53342) 2023-12-26T21:36:55.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.508+0000 7f2f43fff700 1 -- 172.21.15.131:0/2874598560 learned_addr learned my addr 172.21.15.131:0/2874598560 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.508+0000 7f2f43fff700 1 -- 172.21.15.131:0/2874598560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f440b3dc0 msgr2=0x7f2f441678b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-26T21:36:55.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.508+0000 7f2f43fff700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f440b3dc0 0x7f2f441678b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.508+0000 7f31537fe700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f314c066f30 0x7f314c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.508+0000 7f31537fe700 1 -- 172.21.15.131:0/2271878467 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f315c083930 con 0x7f316000c250 2023-12-26T21:36:55.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.508+0000 7f31537fe700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7f314c06f140 0x7f314c071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.509+0000 7f2f43fff700 1 -- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44159400 msgr2=0x7f2f4416ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.509+0000 7f2f43fff700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44159400 0x7f2f4416ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.509+0000 7f2f43fff700 1 -- 172.21.15.131:0/2874598560 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38009420 con 0x7f2f4416f010 2023-12-26T21:36:55.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.509+0000 7f2f42ffd700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f44159400 0x7f2f4416ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:55.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.509+0000 7f3167fff700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7f314c06f140 0x7f314c071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.509+0000 7f2f43fff700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f4416f010 0x7f2f4414fee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f440892d0 tx=0x7f2f38017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.509+0000 7f3166ffd700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f314c066f30 0x7f314c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.510+0000 7f3167fff700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7f314c06f140 0x7f314c071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.510+0000 7f31537fe700 1 -- 172.21.15.131:0/2271878467 --> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f314c071c00 con 0x7f314c06f140 2023-12-26T21:36:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.510+0000 7f3166ffd700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f314c066f30 0x7f314c0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f316806fed0 tx=0x7f3168063120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.511+0000 7f31537fe700 1 -- 172.21.15.131:0/2271878467 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f315c088240 con 0x7f316000c250 2023-12-26T21:36:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.512+0000 7f31537fe700 1 -- 172.21.15.131:0/2271878467 <== osd.0 v2:172.21.15.131:6808/39607 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f314c071c00 con 0x7f314c06f140 2023-12-26T21:36:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.512+0000 7f2f2b7fe700 1 -- 172.21.15.131:0/2874598560 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38011630 con 0x7f2f4416f010 2023-12-26T21:36:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.512+0000 7f2f2b7fe700 1 -- 172.21.15.131:0/2874598560 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38011790 con 0x7f2f4416f010 2023-12-26T21:36:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.512+0000 7f2f2b7fe700 1 -- 172.21.15.131:0/2874598560 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38011b50 con 0x7f2f4416f010 2023-12-26T21:36:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.512+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f440b1470 con 0x7f2f4416f010 2023-12-26T21:36:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.512+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f44167df0 con 0x7f2f4416f010 2023-12-26T21:36:55.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.515+0000 7f2f2b7fe700 1 -- 172.21.15.131:0/2874598560 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2f380253f0 con 0x7f2f4416f010 2023-12-26T21:36:55.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.516+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f24000fc0 con 0x7f2f4416f010 2023-12-26T21:36:55.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.516+0000 7f2f2b7fe700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2f30067080 0x7f2f30069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.517+0000 7f2f2b7fe700 1 -- 172.21.15.131:0/2874598560 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f380861b0 con 0x7f2f4416f010 2023-12-26T21:36:55.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.517+0000 7f2f437fe700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2f30067080 0x7f2f30069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.517+0000 7f2f2b7fe700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7f2f3006f210 0x7f2f30071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.517+0000 7f2f2b7fe700 1 -- 172.21.15.131:0/2874598560 --> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f30071cd0 con 0x7f2f3006f210 2023-12-26T21:36:55.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.518+0000 7f2f42ffd700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7f2f3006f210 0x7f2f30071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.518+0000 7f2f42ffd700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7f2f3006f210 0x7f2f30071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.518+0000 7ff9773ff700 1 Processor -- start 2023-12-26T21:36:55.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.518+0000 7f2f2b7fe700 1 -- 172.21.15.131:0/2874598560 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f38056ed0 con 0x7f2f4416f010 2023-12-26T21:36:55.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.520+0000 7f2f437fe700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2f30067080 0x7f2f30069530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f4406ecb0 tx=0x7f2f3c007480 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.520+0000 7f2f2b7fe700 1 -- 172.21.15.131:0/2874598560 <== osd.1 v2:172.21.15.131:6816/39610 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2f30071cd0 con 0x7f2f3006f210 2023-12-26T21:36:55.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.518+0000 7f6e36bff700 1 Processor -- start 2023-12-26T21:36:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.519+0000 7f6e36bff700 1 -- start start 2023-12-26T21:36:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.519+0000 7ff9773ff700 1 -- start start 2023-12-26T21:36:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7ff9773ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9700af970 0x7ff9700afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7ff9773ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9700aea80 con 0x7ff9701578e0 2023-12-26T21:36:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7ff9773ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9700b2100 con 0x7ff970155630 2023-12-26T21:36:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7ff9773ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9700b0350 con 0x7ff9700af970 2023-12-26T21:36:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7f6e36bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6e30160810 0x7f6e300aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7ff97519b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff970155630 0x7ff9700b19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39716/0 (socket says 172.21.15.131:39716) 2023-12-26T21:36:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7ff97519b700 1 -- 172.21.15.131:0/4005060240 learned_addr learned my addr 172.21.15.131:0/4005060240 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7ff97499a700 1 --2- 172.21.15.131:0/4005060240 >> v2:172.21.15.190:3300/0 conn(0x7ff9700af970 0x7ff9700afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.522+0000 7f6e3499b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6e3015d950 0x7f6e301616d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49370/0 (socket says 172.21.15.131:49370) 2023-12-26T21:36:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.522+0000 7f6e3499b700 1 -- 172.21.15.131:0/1051174720 learned_addr learned my addr 172.21.15.131:0/1051174720 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7f170e99d700 1 Processor -- start 2023-12-26T21:36:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.521+0000 7f170e99d700 1 -- start start 2023-12-26T21:36:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.522+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288823897 0 0) 0x7ff9700aea80 con 0x7ff9701578e0 2023-12-26T21:36:55.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.522+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff95c003570 con 0x7ff9701578e0 2023-12-26T21:36:55.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.522+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689612995 0 0) 0x7ff960001110 con 0x7ff970155630 2023-12-26T21:36:55.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.522+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9700aea80 con 0x7ff970155630 2023-12-26T21:36:55.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7f6e36bff700 1 -- 172.21.15.131:0/1051174720 --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e300af6d0 con 0x7f6e3015d950 2023-12-26T21:36:55.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7f6e36bff700 1 -- 172.21.15.131:0/1051174720 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3015f370 con 0x7f6e30161de0 2023-12-26T21:36:55.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7f6e2ffff700 1 --2- 172.21.15.131:0/1051174720 >> v2:172.21.15.131:3300/0 conn(0x7f6e30160810 0x7f6e300aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3002970854 0 0) 0x7ff9700aea80 con 0x7ff970155630 2023-12-26T21:36:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 >> v2:172.21.15.190:3300/0 conn(0x7ff9700af970 msgr2=0x7ff9700afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 --2- 172.21.15.131:0/4005060240 >> v2:172.21.15.190:3300/0 conn(0x7ff9700af970 0x7ff9700afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 >> v1:172.21.15.131:6789/0 conn(0x7ff9701578e0 legacy=0x7ff970157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970156840 con 0x7ff970155630 2023-12-26T21:36:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7f6e36bff700 1 -- 172.21.15.131:0/1051174720 --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e300afb90 con 0x7f6e30160810 2023-12-26T21:36:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7ff960001580 con 0x7ff970155630 2023-12-26T21:36:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 >> v1:172.21.15.190:6789/0 conn(0x7ff970155630 legacy=0x7ff9700b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff95c004c50 0x7ff95c005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0037b0 0x7ff95c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff95c010070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9700b0350 con 0x7ff95c004c50 2023-12-26T21:36:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0105b0 con 0x7ff95c0037b0 2023-12-26T21:36:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.523+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c010720 con 0x7ff95c004160 2023-12-26T21:36:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.524+0000 7f6e2f7fe700 1 -- 172.21.15.131:0/1051174720 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496992349 0 0) 0x7f6e300af6d0 con 0x7f6e3015d950 2023-12-26T21:36:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.524+0000 7f170e99d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f170806e670 0x7f1708168a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.524+0000 7f4fff3ff700 1 Processor -- start 2023-12-26T21:36:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.524+0000 7f83745ff700 1 Processor -- start 2023-12-26T21:36:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.524+0000 7f6e2f7fe700 1 -- 172.21.15.131:0/1051174720 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e18003570 con 0x7f6e3015d950 2023-12-26T21:36:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.524+0000 7f170e99d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170806abb0 con 0x7f170806f480 2023-12-26T21:36:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.525+0000 7f170e99d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1708069fa0 con 0x7f170806f0b0 2023-12-26T21:36:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.525+0000 7f170e99d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170815e0a0 con 0x7f170806e670 2023-12-26T21:36:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.525+0000 7f6e2ffff700 1 -- 172.21.15.131:0/1051174720 >> v1:172.21.15.190:6789/0 conn(0x7f6e30161de0 legacy=0x7f6e3015dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.524+0000 7f170d99b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f170806f0b0 0x7f1708069520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39722/0 (socket says 172.21.15.131:39722) 2023-12-26T21:36:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.525+0000 7f170d99b700 1 -- 172.21.15.131:0/2393695029 learned_addr learned my addr 172.21.15.131:0/2393695029 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.525+0000 7f6e2f7fe700 1 -- 172.21.15.131:0/1051174720 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3464944307 0 0) 0x7f6e18003570 con 0x7f6e3015d950 2023-12-26T21:36:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.525+0000 7f6e2ffff700 1 -- 172.21.15.131:0/1051174720 >> v1:172.21.15.131:6789/0 conn(0x7f6e3015d950 legacy=0x7f6e301616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.525+0000 7f6e2ffff700 1 -- 172.21.15.131:0/1051174720 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e300afe70 con 0x7f6e30160810 2023-12-26T21:36:55.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.525+0000 7f170e19c700 1 --2- 172.21.15.131:0/2393695029 >> v2:172.21.15.131:3300/0 conn(0x7f170806e670 0x7f1708168a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.526+0000 7f6e2ffff700 1 --2- 172.21.15.131:0/1051174720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e30160810 0x7f6e300aefd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6e200098e0 tx=0x7f6e200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc714247edaacf4c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.526+0000 7f6e2f7fe700 1 -- 172.21.15.131:0/1051174720 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280017f0 con 0x7f6e30160810 2023-12-26T21:36:55.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.526+0000 7f170e19c700 1 -- 172.21.15.131:0/2393695029 >> v1:172.21.15.190:6789/0 conn(0x7f170806f0b0 legacy=0x7f1708069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.526+0000 7f6e2f7fe700 1 -- 172.21.15.131:0/1051174720 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e20011420 con 0x7f6e30160810 2023-12-26T21:36:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.526+0000 7f170e19c700 1 -- 172.21.15.131:0/2393695029 >> v1:172.21.15.131:6789/0 conn(0x7f170806f480 legacy=0x7f17080b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.526+0000 7f170c999700 1 -- 172.21.15.131:0/2393695029 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635012270 0 0) 0x7f1708069fa0 con 0x7f170806f0b0 2023-12-26T21:36:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f6e2f7fe700 1 -- 172.21.15.131:0/1051174720 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280017f0 con 0x7f6e30160810 2023-12-26T21:36:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.526+0000 7f170e19c700 1 -- 172.21.15.131:0/2393695029 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170806a280 con 0x7f170806e670 2023-12-26T21:36:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.524+0000 7f83745ff700 1 -- start start 2023-12-26T21:36:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.526+0000 7f83745ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f836c159920 0x7f836c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f83745ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c0ced90 0x7f836c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f83745ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836c0c6810 con 0x7f836c0c5420 2023-12-26T21:36:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f83745ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836c15a230 con 0x7f836c159920 2023-12-26T21:36:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f83745ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0cf710 con 0x7f836c0ced90 2023-12-26T21:36:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f837239b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f836c159920 0x7f836c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f8371b9a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f836c0c5420 0x7f836c0c6ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39738/0 (socket says 172.21.15.131:39738) 2023-12-26T21:36:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f8371b9a700 1 -- 172.21.15.131:0/2009474005 learned_addr learned my addr 172.21.15.131:0/2009474005 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f170e19c700 1 --2- 172.21.15.131:0/2393695029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170806e670 0x7f1708168a50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17000098e0 tx=0x7f17000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168d2089bc725ead server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f8372b9c700 1 --2- 172.21.15.131:0/2009474005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c0ced90 0x7f836c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f6e36bff700 1 -- 172.21.15.131:0/1051174720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e30160810 msgr2=0x7f6e300aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.524+0000 7f4fff3ff700 1 -- start start 2023-12-26T21:36:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.527+0000 7f6e36bff700 1 --2- 172.21.15.131:0/1051174720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e30160810 0x7f6e300aefd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6e200098e0 tx=0x7f6e200091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7f170c999700 1 -- 172.21.15.131:0/2393695029 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700009e80 con 0x7f170806e670 2023-12-26T21:36:55.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7f8371399700 1 -- 172.21.15.131:0/2009474005 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432982473 0 0) 0x7f836c0c6810 con 0x7f836c0c5420 2023-12-26T21:36:55.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7f8371399700 1 -- 172.21.15.131:0/2009474005 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f835c003570 con 0x7f836c0c5420 2023-12-26T21:36:55.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7ff97599c700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff95c010070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7ff97519b700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff95c004c50 0x7ff95c005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7ff97499a700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0037b0 0x7ff95c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7f170c999700 1 -- 172.21.15.131:0/2393695029 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170000b760 con 0x7f170806e670 2023-12-26T21:36:55.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7f6e36bff700 1 -- 172.21.15.131:0/1051174720 shutdown_connections 2023-12-26T21:36:55.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7f6e36bff700 1 --2- 172.21.15.131:0/1051174720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e30160810 0x7f6e300aefd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7f6e36bff700 1 -- 172.21.15.131:0/1051174720 >> 172.21.15.131:0/1051174720 conn(0x7f6e3006a060 msgr2=0x7f6e300cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.528+0000 7f6e36bff700 1 -- 172.21.15.131:0/1051174720 shutdown_connections 2023-12-26T21:36:55.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7ff97599c700 1 -- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0037b0 msgr2=0x7ff95c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:55.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7ff97599c700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0037b0 0x7ff95c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7ff97599c700 1 -- 172.21.15.131:0/4005060240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff95c004c50 msgr2=0x7ff95c005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f837239b700 1 -- 172.21.15.131:0/2009474005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c0ced90 msgr2=0x7f836c0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7ff97599c700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff95c004c50 0x7ff95c005040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7ff97599c700 1 -- 172.21.15.131:0/4005060240 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970156840 con 0x7ff95c004160 2023-12-26T21:36:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7ff97599c700 1 -- 172.21.15.131:0/4005060240 reap_dead start 2023-12-26T21:36:55.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f837239b700 1 --2- 172.21.15.131:0/2009474005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c0ced90 0x7f836c0cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f837239b700 1 -- 172.21.15.131:0/2009474005 >> v1:172.21.15.190:6789/0 conn(0x7f836c0c5420 legacy=0x7f836c0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f837239b700 1 -- 172.21.15.131:0/2009474005 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c0c5c00 con 0x7f836c159920 2023-12-26T21:36:55.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f8372b9c700 1 --2- 172.21.15.131:0/2009474005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c0ced90 0x7f836c0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:55.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f170e99d700 1 -- 172.21.15.131:0/2393695029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170806e670 msgr2=0x7f1708168a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7ff97519b700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff95c004c50 0x7ff95c005040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:55.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f170c999700 1 -- 172.21.15.131:0/2393695029 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700009e80 con 0x7f170806e670 2023-12-26T21:36:55.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f170e99d700 1 --2- 172.21.15.131:0/2393695029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170806e670 0x7f1708168a50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17000098e0 tx=0x7f17000091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7ff97599c700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff95c010070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff968002170 tx=0x7ff9680017f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699bab6aafb8c0db server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f837239b700 1 --2- 172.21.15.131:0/2009474005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c159920 0x7f836c159cf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8360009eb0 tx=0x7f8360009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f1230546a3bf87 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.529+0000 7f6e36bff700 1 -- 172.21.15.131:0/1051174720 wait complete. 2023-12-26T21:36:55.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968001b00 con 0x7ff95c004160 2023-12-26T21:36:55.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7f170e99d700 1 -- 172.21.15.131:0/2393695029 shutdown_connections 2023-12-26T21:36:55.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7f170e99d700 1 --2- 172.21.15.131:0/2393695029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170806e670 0x7f1708168a50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7f170e99d700 1 -- 172.21.15.131:0/2393695029 >> 172.21.15.131:0/2393695029 conn(0x7f1708068040 msgr2=0x7f17080d4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7ff967fff700 1 -- 172.21.15.131:0/4005060240 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96800c980 con 0x7ff95c004160 2023-12-26T21:36:55.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7f170e99d700 1 -- 172.21.15.131:0/2393695029 shutdown_connections 2023-12-26T21:36:55.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7f8371399700 1 -- 172.21.15.131:0/2009474005 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836000e070 con 0x7f836c159920 2023-12-26T21:36:55.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7f8371399700 1 -- 172.21.15.131:0/2009474005 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836000bb80 con 0x7f836c159920 2023-12-26T21:36:55.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7f8371399700 1 -- 172.21.15.131:0/2009474005 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836000e070 con 0x7f836c159920 2023-12-26T21:36:55.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.531+0000 7f6e36bff700 1 Processor -- start 2023-12-26T21:36:55.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.531+0000 7f83745ff700 1 -- 172.21.15.131:0/2009474005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c159920 msgr2=0x7f836c159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.531+0000 7ff9773ff700 1 -- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 msgr2=0x7ff95c010070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.531+0000 7ff9773ff700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff95c010070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff968002170 tx=0x7ff9680017f0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.531+0000 7f83745ff700 1 --2- 172.21.15.131:0/2009474005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c159920 0x7f836c159cf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8360009eb0 tx=0x7f8360009780 comp rx=0 tx=0).stop 2023-12-26T21:36:55.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.531+0000 7f6e36bff700 1 -- start start 2023-12-26T21:36:55.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7ff9773ff700 1 -- 172.21.15.131:0/4005060240 shutdown_connections 2023-12-26T21:36:55.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7ff9773ff700 1 --2- 172.21.15.131:0/4005060240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff95c010070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f83745ff700 1 -- 172.21.15.131:0/2009474005 shutdown_connections 2023-12-26T21:36:55.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f83745ff700 1 --2- 172.21.15.131:0/2009474005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c0ced90 0x7f836c0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f4fff3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ff80aefd0 0x7f4ff8168a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f4fff3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff815f320 con 0x7f4ff81618c0 2023-12-26T21:36:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f83745ff700 1 --2- 172.21.15.131:0/2009474005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c159920 0x7f836c159cf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f4fff3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff80afc60 con 0x7f4ff81607c0 2023-12-26T21:36:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f4fff3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff80b0220 con 0x7f4ff80aefd0 2023-12-26T21:36:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f83745ff700 1 -- 172.21.15.131:0/2009474005 >> 172.21.15.131:0/2009474005 conn(0x7f836c099980 msgr2=0x7f836c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7ff9773ff700 1 -- 172.21.15.131:0/4005060240 >> 172.21.15.131:0/4005060240 conn(0x7ff97006a060 msgr2=0x7ff9700cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f4ffd19b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4ff81618c0 0x7f4ff8161c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49398/0 (socket says 172.21.15.131:49398) 2023-12-26T21:36:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.532+0000 7f4ffd19b700 1 -- 172.21.15.131:0/4069955195 learned_addr learned my addr 172.21.15.131:0/4069955195 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.530+0000 7f170e99d700 1 -- 172.21.15.131:0/2393695029 wait complete. 2023-12-26T21:36:55.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7f170e99d700 1 Processor -- start 2023-12-26T21:36:55.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7f4ffd99c700 1 --2- 172.21.15.131:0/4069955195 >> v2:172.21.15.190:3300/0 conn(0x7f4ff80aefd0 0x7f4ff8168a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7ff9773ff700 1 -- 172.21.15.131:0/4005060240 shutdown_connections 2023-12-26T21:36:55.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7f6e36bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3015d950 0x7f6e300acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7f4ffd99c700 1 -- 172.21.15.131:0/4069955195 >> v1:172.21.15.190:6789/0 conn(0x7f4ff81607c0 legacy=0x7f4ff815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7f4ffd99c700 1 -- 172.21.15.131:0/4069955195 >> v1:172.21.15.131:6789/0 conn(0x7f4ff81618c0 legacy=0x7f4ff8161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7f4ffd99c700 1 -- 172.21.15.131:0/4069955195 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff80aff40 con 0x7f4ff80aefd0 2023-12-26T21:36:55.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7ff9773ff700 1 -- 172.21.15.131:0/4005060240 wait complete. 2023-12-26T21:36:55.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f6e36bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e30160810 0x7f6e3016df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f6e3499b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3015d950 0x7f6e300acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f6e3499b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3015d950 0x7f6e300acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57284/0 (socket says 172.21.15.131:57284) 2023-12-26T21:36:55.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f6e3499b700 1 -- 172.21.15.131:0/356492275 learned_addr learned my addr 172.21.15.131:0/356492275 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f4ffd99c700 1 --2- 172.21.15.131:0/4069955195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 0x7f4ff8168a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009e60 tx=0x7f4ff0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7399511c5ed4c68f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7f83745ff700 1 -- 172.21.15.131:0/2009474005 shutdown_connections 2023-12-26T21:36:55.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f83745ff700 1 -- 172.21.15.131:0/2009474005 wait complete. 2023-12-26T21:36:55.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f6e36bff700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e30161de0 0x7f6e3015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f6e2ffff700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e30160810 0x7f6e3016df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f4febfff700 1 -- 172.21.15.131:0/4069955195 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000e070 con 0x7f4ff80aefd0 2023-12-26T21:36:55.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.534+0000 7f4febfff700 1 -- 172.21.15.131:0/4069955195 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff000bc30 con 0x7f4ff80aefd0 2023-12-26T21:36:55.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f6e3519c700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e30161de0 0x7f6e3015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.533+0000 7f170e99d700 1 -- start start 2023-12-26T21:36:55.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f170e99d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170806e670 0x7f1708160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f170e99d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170806f0b0 0x7f17081593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f6e36bff700 1 -- 172.21.15.131:0/356492275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300a86b0 con 0x7f6e30161de0 2023-12-26T21:36:55.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f6e36bff700 1 -- 172.21.15.131:0/356492275 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3016e450 con 0x7f6e30160810 2023-12-26T21:36:55.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f6e36bff700 1 -- 172.21.15.131:0/356492275 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300a9d00 con 0x7f6e3015d950 2023-12-26T21:36:55.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f170d99b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170806e670 0x7f1708160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f4fff3ff700 1 -- 172.21.15.131:0/4069955195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 msgr2=0x7f4ff8168a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f4fff3ff700 1 --2- 172.21.15.131:0/4069955195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 0x7f4ff8168a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009e60 tx=0x7f4ff0009780 comp rx=0 tx=0).stop 2023-12-26T21:36:55.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7ff9773ff700 1 Processor -- start 2023-12-26T21:36:55.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f170d99b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170806e670 0x7f1708160ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44808/0 (socket says 172.21.15.131:44808) 2023-12-26T21:36:55.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.535+0000 7f4fff3ff700 1 -- 172.21.15.131:0/4069955195 shutdown_connections 2023-12-26T21:36:55.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f170d99b700 1 -- 172.21.15.131:0/767464352 learned_addr learned my addr 172.21.15.131:0/767464352 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7ff9773ff700 1 -- start start 2023-12-26T21:36:55.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f170e99d700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170806f480 0x7f17080af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f4fff3ff700 1 --2- 172.21.15.131:0/4069955195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 0x7f4ff8168a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f4fff3ff700 1 -- 172.21.15.131:0/4069955195 >> 172.21.15.131:0/4069955195 conn(0x7f4ff8068040 msgr2=0x7f4ff8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f170d19a700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170806f0b0 0x7f17081593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170815e280 con 0x7f170806e670 2023-12-26T21:36:55.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1708161400 con 0x7f170806f0b0 2023-12-26T21:36:55.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f170e19c700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170806f480 0x7f17080af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17080b1380 con 0x7f170806f480 2023-12-26T21:36:55.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f6e3499b700 1 -- 172.21.15.131:0/356492275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e30160810 msgr2=0x7f6e3016df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f6e3499b700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e30160810 0x7f6e3016df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f6e3499b700 1 -- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e30161de0 msgr2=0x7f6e3015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f6e3499b700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e30161de0 0x7f6e3015a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.536+0000 7f6e3499b700 1 -- 172.21.15.131:0/356492275 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e20009420 con 0x7f6e3015d950 2023-12-26T21:36:55.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.537+0000 7f170e19c700 1 -- 172.21.15.131:0/767464352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170806f0b0 msgr2=0x7f17081593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.537+0000 7f170e19c700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170806f0b0 0x7f17081593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.537+0000 7f170e19c700 1 -- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170806e670 msgr2=0x7f1708160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.537+0000 7f170e19c700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170806e670 0x7f1708160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.537+0000 7f170e19c700 1 -- 172.21.15.131:0/767464352 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1700009420 con 0x7f170806f480 2023-12-26T21:36:55.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.537+0000 7ff9773ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff97006af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.537+0000 7f6e3499b700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3015d950 0x7f6e300acb00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6e28001700 tx=0x7f6e28007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7ff9773ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff970144e80 0x7ff970139a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7f170e19c700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170806f480 0x7f17080af2d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1700015000 tx=0x7f1700016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7f6e2cff9700 1 -- 172.21.15.131:0/356492275 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28004dc0 con 0x7f6e3015d950 2023-12-26T21:36:55.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7ff9773ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff97006ed80 0x7ff97006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7f6e2cff9700 1 -- 172.21.15.131:0/356492275 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2800a5d0 con 0x7f6e3015d950 2023-12-26T21:36:55.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7f6e36bff700 1 -- 172.21.15.131:0/356492275 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3016eee0 con 0x7f6e3015d950 2023-12-26T21:36:55.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7f1705ffb700 1 -- 172.21.15.131:0/767464352 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170001c460 con 0x7f170806f480 2023-12-26T21:36:55.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7f1705ffb700 1 -- 172.21.15.131:0/767464352 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170001c600 con 0x7f170806f480 2023-12-26T21:36:55.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7f1705ffb700 1 -- 172.21.15.131:0/767464352 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170001c9c0 con 0x7f170806f480 2023-12-26T21:36:55.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7f4fff3ff700 1 -- 172.21.15.131:0/4069955195 shutdown_connections 2023-12-26T21:36:55.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7f4fff3ff700 1 -- 172.21.15.131:0/4069955195 wait complete. 2023-12-26T21:36:55.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7ff97519b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff97006af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7ff97499a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff970144e80 0x7ff970139a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7ff97519b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff97006af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57300/0 (socket says 172.21.15.131:57300) 2023-12-26T21:36:55.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7ff97499a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff970144e80 0x7ff970139a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53392/0 (socket says 172.21.15.131:53392) 2023-12-26T21:36:55.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7ff97599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff97006ed80 0x7ff97006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7ff97519b700 1 -- 172.21.15.131:0/4023696793 learned_addr learned my addr 172.21.15.131:0/4023696793 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7ff9773ff700 1 -- 172.21.15.131:0/4023696793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97006f710 con 0x7ff97006ed80 2023-12-26T21:36:55.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7ff9773ff700 1 -- 172.21.15.131:0/4023696793 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970145380 con 0x7ff970144e80 2023-12-26T21:36:55.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.538+0000 7f83745ff700 1 Processor -- start 2023-12-26T21:36:55.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7ff9773ff700 1 -- 172.21.15.131:0/4023696793 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff970145660 con 0x7ff95c004160 2023-12-26T21:36:55.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17080aec40 con 0x7f170806f480 2023-12-26T21:36:55.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170815e7d0 con 0x7f170806f480 2023-12-26T21:36:55.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.540+0000 7f6e2cff9700 1 -- 172.21.15.131:0/356492275 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800a990 con 0x7f6e3015d950 2023-12-26T21:36:55.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7ff97499a700 1 -- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 msgr2=0x7ff97006af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7ff97499a700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff97006af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7ff97499a700 1 -- 172.21.15.131:0/4023696793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff97006ed80 msgr2=0x7ff97006f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.540+0000 7ff97499a700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff97006ed80 0x7ff97006f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.540+0000 7ff97499a700 1 -- 172.21.15.131:0/4023696793 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970145940 con 0x7ff970144e80 2023-12-26T21:36:55.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.540+0000 7ff97519b700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c004160 0x7ff97006af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:55.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.539+0000 7f83745ff700 1 -- start start 2023-12-26T21:36:55.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.540+0000 7f83745ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0c5420 0x7f836c154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.540+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f170804c3c0 con 0x7f170806f480 2023-12-26T21:36:55.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.541+0000 7fceef59e700 1 Processor -- start 2023-12-26T21:36:55.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.541+0000 7f6e36bff700 1 -- 172.21.15.131:0/356492275 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3016ec00 con 0x7f6e3015d950 2023-12-26T21:36:55.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.541+0000 7ff97499a700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff970144e80 0x7ff970139a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0119f0 tx=0x7ff96c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.541+0000 7f837239b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0c5420 0x7f836c154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.541+0000 7f837239b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0c5420 0x7f836c154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44814/0 (socket says 172.21.15.131:44814) 2023-12-26T21:36:55.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.541+0000 7f83745ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836c0ced90 0x7f836c0b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.541+0000 7ff9657fa700 1 -- 172.21.15.131:0/4023696793 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c00f790 con 0x7ff970144e80 2023-12-26T21:36:55.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.542+0000 7f837239b700 1 -- 172.21.15.131:0/3176897483 learned_addr learned my addr 172.21.15.131:0/3176897483 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.542+0000 7f83745ff700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c159920 0x7f836c0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.542+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c156580 con 0x7f836c0c5420 2023-12-26T21:36:55.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.542+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0c96d0 con 0x7f836c0ced90 2023-12-26T21:36:55.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.542+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c155c50 con 0x7f836c159920 2023-12-26T21:36:55.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7ff9657fa700 1 -- 172.21.15.131:0/4023696793 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96c00f8f0 con 0x7ff970144e80 2023-12-26T21:36:55.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7ff9657fa700 1 -- 172.21.15.131:0/4023696793 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c00fc90 con 0x7ff970144e80 2023-12-26T21:36:55.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f4fff3ff700 1 Processor -- start 2023-12-26T21:36:55.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.544+0000 7f6e2cff9700 1 -- 172.21.15.131:0/356492275 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6e28007150 con 0x7f6e3015d950 2023-12-26T21:36:55.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f1705ffb700 1 -- 172.21.15.131:0/767464352 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f170001a450 con 0x7f170806f480 2023-12-26T21:36:55.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.544+0000 7f1705ffb700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066fe0 0x7f16f4069490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.544+0000 7f1705ffb700 1 -- 172.21.15.131:0/767464352 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1700086c20 con 0x7f170806f480 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f4fff3ff700 1 -- start start 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.542+0000 7f8371b9a700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836c0ced90 0x7f836c0b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.542+0000 7f8372b9c700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c159920 0x7f836c0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f8371b9a700 1 -- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c159920 msgr2=0x7f836c0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f8371b9a700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f836c159920 0x7f836c0b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f8371b9a700 1 -- 172.21.15.131:0/3176897483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0c5420 msgr2=0x7f836c154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f8371b9a700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0c5420 0x7f836c154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f8371b9a700 1 -- 172.21.15.131:0/3176897483 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364009420 con 0x7f836c0ced90 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f8371b9a700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836c0ced90 0x7f836c0b55b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f835800c560 tx=0x7f8358007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f836a7fc700 1 -- 172.21.15.131:0/3176897483 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358005c00 con 0x7f836c0ced90 2023-12-26T21:36:55.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.541+0000 7fceef59e700 1 -- start start 2023-12-26T21:36:55.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7fceef59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7fceef59e700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee8006300 con 0x7fcee8053ea0 2023-12-26T21:36:55.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7fceef59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee8054460 con 0x7fcee8033960 2023-12-26T21:36:55.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7fceef59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8054d00 con 0x7fcee80dde90 2023-12-26T21:36:55.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.544+0000 7fcef4fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fcee8033960 0x7fcee8006790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39754/0 (socket says 172.21.15.131:39754) 2023-12-26T21:36:55.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.544+0000 7fcef4fff700 1 -- 172.21.15.131:0/2150593401 learned_addr learned my addr 172.21.15.131:0/2150593401 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.544+0000 7fceee59c700 1 --2- 172.21.15.131:0/2150593401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7f6e36bff700 1 -- 172.21.15.131:0/356492275 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e300657e0 con 0x7f6e3015d950 2023-12-26T21:36:55.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7f4fff3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 0x7f4ff8163110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7fceed59a700 1 -- 172.21.15.131:0/2150593401 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375654342 0 0) 0x7fcee8054460 con 0x7fcee8033960 2023-12-26T21:36:55.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7fceed59a700 1 -- 172.21.15.131:0/2150593401 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fced8005780 con 0x7fcee8033960 2023-12-26T21:36:55.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7fceee59c700 1 -- 172.21.15.131:0/2150593401 >> v1:172.21.15.190:6789/0 conn(0x7fcee8033960 legacy=0x7fcee8006790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7f4fff3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff81607c0 0x7f4ff80a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7f4fff3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ff81618c0 0x7f4ff80a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7fceee59c700 1 -- 172.21.15.131:0/2150593401 >> v1:172.21.15.131:6789/0 conn(0x7fcee8053ea0 legacy=0x7fcee80503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7fceee59c700 1 -- 172.21.15.131:0/2150593401 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee8054740 con 0x7fcee80dde90 2023-12-26T21:36:55.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f4ffd19b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 0x7f4ff8163110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7f1705ffb700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7f16f406f170 0x7f16f4071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7f1705ffb700 1 -- 172.21.15.131:0/767464352 --> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16f4071c30 con 0x7f16f406f170 2023-12-26T21:36:55.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7f170d19a700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7f16f406f170 0x7f16f4071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f4ffd19b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 0x7f4ff8163110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53410/0 (socket says 172.21.15.131:53410) 2023-12-26T21:36:55.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7fceee59c700 1 --2- 172.21.15.131:0/2150593401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80de260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcef004d1b0 tx=0x7fcef0063120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997d37ddf1cb2928 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7ff9773ff700 1 -- 172.21.15.131:0/4023696793 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff970145c20 con 0x7ff970144e80 2023-12-26T21:36:55.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f4ffd19b700 1 -- 172.21.15.131:0/366163841 learned_addr learned my addr 172.21.15.131:0/366163841 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f4ffc99a700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff81607c0 0x7f4ff80a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8162070 con 0x7f4ff81618c0 2023-12-26T21:36:55.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f6e2cff9700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e1c066f30 0x7f6e1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7fceed59a700 1 -- 172.21.15.131:0/2150593401 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0064770 con 0x7fcee80dde90 2023-12-26T21:36:55.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7fceed59a700 1 -- 172.21.15.131:0/2150593401 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef00648d0 con 0x7fcee80dde90 2023-12-26T21:36:55.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7fceed59a700 1 -- 172.21.15.131:0/2150593401 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0067800 con 0x7fcee80dde90 2023-12-26T21:36:55.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8162350 con 0x7f4ff80aefd0 2023-12-26T21:36:55.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8162630 con 0x7f4ff81607c0 2023-12-26T21:36:55.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f170d99b700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066fe0 0x7f16f4069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7f6e2cff9700 1 -- 172.21.15.131:0/356492275 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e28083d90 con 0x7f6e3015d950 2023-12-26T21:36:55.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7ff9773ff700 1 -- 172.21.15.131:0/4023696793 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97006b4b0 con 0x7ff970144e80 2023-12-26T21:36:55.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7f6e2cff9700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f6e1c06f140 0x7f6e1c071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7f1705ffb700 1 -- 172.21.15.131:0/767464352 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1700053df0 con 0x7f170806f480 2023-12-26T21:36:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7fceef59e700 1 -- 172.21.15.131:0/2150593401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 msgr2=0x7fcee80de260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7fceef59e700 1 --2- 172.21.15.131:0/2150593401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80de260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcef004d1b0 tx=0x7fcef0063120 comp rx=0 tx=0).stop 2023-12-26T21:36:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.548+0000 7f170d99b700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066fe0 0x7f16f4069490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16fc001a00 tx=0x7f16fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7f4ffd19b700 1 -- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff81607c0 msgr2=0x7f4ff80a7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7f4ffd19b700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff81607c0 0x7f4ff80a7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7f4ffd19b700 1 -- 172.21.15.131:0/366163841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ff81618c0 msgr2=0x7f4ff80a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f836a7fc700 1 -- 172.21.15.131:0/3176897483 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8358005d60 con 0x7f836c0ced90 2023-12-26T21:36:55.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.543+0000 7f836a7fc700 1 -- 172.21.15.131:0/3176897483 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358007300 con 0x7f836c0ced90 2023-12-26T21:36:55.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.548+0000 7fceef59e700 1 -- 172.21.15.131:0/2150593401 shutdown_connections 2023-12-26T21:36:55.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.545+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8360009a20 con 0x7f836c0ced90 2023-12-26T21:36:55.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.546+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836c156210 con 0x7f836c0ced90 2023-12-26T21:36:55.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.548+0000 7fceef59e700 1 --2- 172.21.15.131:0/2150593401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80de260 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7f4ffd19b700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ff81618c0 0x7f4ff80a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7f4ffd19b700 1 -- 172.21.15.131:0/366163841 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0009a20 con 0x7f4ff80aefd0 2023-12-26T21:36:55.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.548+0000 7fceef59e700 1 -- 172.21.15.131:0/2150593401 >> 172.21.15.131:0/2150593401 conn(0x7fcee80518b0 msgr2=0x7fcee8051cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.547+0000 7f6e2ffff700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e1c066f30 0x7f6e1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.548+0000 7f6e3519c700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f6e1c06f140 0x7f6e1c071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.548+0000 7ff95affd700 1 -- 172.21.15.131:0/4023696793 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff970148c40 con 0x7ff970144e80 2023-12-26T21:36:55.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.548+0000 7f6e2ffff700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e1c066f30 0x7f6e1c0693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e200098e0 tx=0x7f6e20016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.549+0000 7fceef59e700 1 -- 172.21.15.131:0/2150593401 shutdown_connections 2023-12-26T21:36:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.549+0000 7f4ffd19b700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 0x7f4ff8163110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00c560 tx=0x7f4fec001020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.549+0000 7f6e2cff9700 1 -- 172.21.15.131:0/356492275 --> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e1c071c00 con 0x7f6e1c06f140 2023-12-26T21:36:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.549+0000 7f6e2cff9700 1 -- 172.21.15.131:0/356492275 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e28083fd0 con 0x7f6e3015d950 2023-12-26T21:36:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.549+0000 7f6e3519c700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f6e1c06f140 0x7f6e1c071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.549+0000 7f4fe97fa700 1 -- 172.21.15.131:0/366163841 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec005b30 con 0x7f4ff80aefd0 2023-12-26T21:36:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.549+0000 7f4fe97fa700 1 -- 172.21.15.131:0/366163841 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec005c90 con 0x7f4ff80aefd0 2023-12-26T21:36:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.551+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 --> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f316004d000 con 0x7f314c06f140 2023-12-26T21:36:55.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7f6e2cff9700 1 -- 172.21.15.131:0/356492275 <== osd.2 v2:172.21.15.131:6800/39608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6e1c071c00 con 0x7f6e1c06f140 2023-12-26T21:36:55.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7f170d19a700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7f16f406f170 0x7f16f4071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7f4fe97fa700 1 -- 172.21.15.131:0/366163841 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec007320 con 0x7f4ff80aefd0 2023-12-26T21:36:55.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff8162910 con 0x7f4ff80aefd0 2023-12-26T21:36:55.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7f4ffc99a700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff81607c0 0x7f4ff80a7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:55.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff8163650 con 0x7f4ff80aefd0 2023-12-26T21:36:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7ff9657fa700 1 -- 172.21.15.131:0/4023696793 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff96c014650 con 0x7ff970144e80 2023-12-26T21:36:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.549+0000 7f837239b700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0c5420 0x7f836c154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-12-26T21:36:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.549+0000 7f836a7fc700 1 -- 172.21.15.131:0/3176897483 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f835801b3b0 con 0x7f836c0ced90 2023-12-26T21:36:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f836c0cd4c0 con 0x7f836c0ced90 2023-12-26T21:36:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7f836a7fc700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8350067000 0x7f83500694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7f837239b700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8350067000 0x7f83500694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.550+0000 7fceef59e700 1 -- 172.21.15.131:0/2150593401 wait complete. 2023-12-26T21:36:55.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.551+0000 7f31537fe700 1 -- 172.21.15.131:0/2271878467 <== osd.0 v2:172.21.15.131:6808/39607 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f316004d000 con 0x7f314c06f140 2023-12-26T21:36:55.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.551+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ff80657e0 con 0x7f4ff80aefd0 2023-12-26T21:36:55.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.552+0000 7f837239b700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8350067000 0x7f83500694b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f836c088f70 tx=0x7f8360019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.552+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7f314c06f140 msgr2=0x7f314c071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.552+0000 7f316ddff700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7f314c06f140 0x7f314c071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.552+0000 7ff9657fa700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff954067000 0x7ff9540694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.552+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f314c066f30 msgr2=0x7f314c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.552+0000 7f316ddff700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f314c066f30 0x7f314c0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f316806fed0 tx=0x7f3168063120 comp rx=0 tx=0).stop 2023-12-26T21:36:55.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316000c250 msgr2=0x7f31600dedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7f316ddff700 1 --2- 172.21.15.131:0/2271878467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316000c250 0x7f31600dedc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f315c00cb90 tx=0x7f315c001ad0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.552+0000 7ff9657fa700 1 -- 172.21.15.131:0/4023696793 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff96c018030 con 0x7ff970144e80 2023-12-26T21:36:55.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.552+0000 7fceef59e700 1 Processor -- start 2023-12-26T21:36:55.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7fceef59e700 1 -- start start 2023-12-26T21:36:55.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7f836a7fc700 1 -- 172.21.15.131:0/3176897483 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8358054de0 con 0x7f836c0ced90 2023-12-26T21:36:55.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.552+0000 7ff97519b700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff954067000 0x7ff9540694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7ff9657fa700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7ff95406f210 0x7ff954071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7ff9657fa700 1 -- 172.21.15.131:0/4023696793 --> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff954071cd0 con 0x7ff95406f210 2023-12-26T21:36:55.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7ff9657fa700 1 -- 172.21.15.131:0/4023696793 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff96c084140 con 0x7ff970144e80 2023-12-26T21:36:55.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7ff97599c700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7ff95406f210 0x7ff954071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7f3167fff700 1 -- 172.21.15.131:0/2271878467 reap_dead start 2023-12-26T21:36:55.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7f836a7fc700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7f835006f190 0x7f83500715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.554+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 shutdown_connections 2023-12-26T21:36:55.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.554+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 >> 172.21.15.131:0/2271878467 conn(0x7f3160054da0 msgr2=0x7f3160035020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.554+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 shutdown_connections 2023-12-26T21:36:55.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.554+0000 7f316ddff700 1 -- 172.21.15.131:0/2271878467 wait complete. 2023-12-26T21:36:55.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.554+0000 7ff97519b700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff954067000 0x7ff9540694b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff960007fd0 tx=0x7ff9600079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.554+0000 7ff97599c700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7ff95406f210 0x7ff954071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.553+0000 7f836a7fc700 1 -- 172.21.15.131:0/3176897483 --> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8350071c50 con 0x7f835006f190 2023-12-26T21:36:55.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.554+0000 7f836a7fc700 1 -- 172.21.15.131:0/3176897483 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8358007e90 con 0x7f836c0ced90 2023-12-26T21:36:55.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.554+0000 7f8372b9c700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7f835006f190 0x7f83500715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.555+0000 7fceef59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee8033960 0x7fcee803a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.555+0000 7f1705ffb700 1 -- 172.21.15.131:0/767464352 <== osd.3 v2:172.21.15.131:6817/39609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f16f4071c30 con 0x7f16f406f170 2023-12-26T21:36:55.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.555+0000 7f8372b9c700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7f835006f190 0x7f83500715a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.555+0000 7fceef59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee8053ea0 0x7fcee8035720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.555+0000 7ff9657fa700 1 -- 172.21.15.131:0/4023696793 <== osd.5 v2:172.21.15.190:6817/40012 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff954071cd0 con 0x7ff95406f210 2023-12-26T21:36:55.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.555+0000 7fceee59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee8033960 0x7fcee803a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.555+0000 7fceee59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee8033960 0x7fcee803a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53412/0 (socket says 172.21.15.131:53412) 2023-12-26T21:36:55.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.556+0000 7fceedd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee8053ea0 0x7fcee8035720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.556+0000 7fceedd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee8053ea0 0x7fcee8035720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44826/0 (socket says 172.21.15.131:44826) 2023-12-26T21:36:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.556+0000 7fceef59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.556+0000 7fceef59e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8036ca0 con 0x7fcee8053ea0 2023-12-26T21:36:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.556+0000 7fceef59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8037130 con 0x7fcee8033960 2023-12-26T21:36:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.556+0000 7fceef59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8036090 con 0x7fcee80dde90 2023-12-26T21:36:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.556+0000 7fcef4fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.556+0000 7fcef4fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80e0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57336/0 (socket says 172.21.15.131:57336) 2023-12-26T21:36:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.556+0000 7fcef4fff700 1 -- 172.21.15.131:0/203243830 learned_addr learned my addr 172.21.15.131:0/203243830 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.555+0000 7f4fe97fa700 1 -- 172.21.15.131:0/366163841 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4fec01b3b0 con 0x7f4ff80aefd0 2023-12-26T21:36:55.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.557+0000 7f4fe97fa700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fdc067080 0x7f4fdc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.557+0000 7fcef4fff700 1 -- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee8033960 msgr2=0x7fcee803a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.557+0000 7fcef4fff700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee8033960 0x7fcee803a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.557+0000 7fcef4fff700 1 -- 172.21.15.131:0/203243830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee8053ea0 msgr2=0x7fcee8035720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.557+0000 7fcef4fff700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee8053ea0 0x7fcee8035720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.557+0000 7fcef4fff700 1 -- 172.21.15.131:0/203243830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef004c0e0 con 0x7fcee80dde90 2023-12-26T21:36:55.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.557+0000 7fcef4fff700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80e0a00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcee00010e0 tx=0x7fcee0007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.557+0000 7f836a7fc700 1 -- 172.21.15.131:0/3176897483 <== osd.6 v2:172.21.15.190:6816/40035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8350071c50 con 0x7f835006f190 2023-12-26T21:36:55.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.559+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 --> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2f24002cf0 con 0x7f2f3006f210 2023-12-26T21:36:55.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.560+0000 7f2f2b7fe700 1 -- 172.21.15.131:0/2874598560 <== osd.1 v2:172.21.15.131:6816/39610 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2f24002cf0 con 0x7f2f3006f210 2023-12-26T21:36:55.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.558+0000 7fcede7fc700 1 -- 172.21.15.131:0/203243830 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0005c00 con 0x7fcee80dde90 2023-12-26T21:36:55.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.558+0000 7fcede7fc700 1 -- 172.21.15.131:0/203243830 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee0005d60 con 0x7fcee80dde90 2023-12-26T21:36:55.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.559+0000 7fceee59c700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee8033960 0x7fcee803a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-12-26T21:36:55.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.559+0000 7fceedd9b700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee8053ea0 0x7fcee8035720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-12-26T21:36:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.559+0000 7fcede7fc700 1 -- 172.21.15.131:0/203243830 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee00072e0 con 0x7fcee80dde90 2023-12-26T21:36:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.559+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee8036370 con 0x7fcee80dde90 2023-12-26T21:36:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.559+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee8036930 con 0x7fcee80dde90 2023-12-26T21:36:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.560+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcee8005540 con 0x7fcee80dde90 2023-12-26T21:36:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.561+0000 7f4fe97fa700 1 -- 172.21.15.131:0/366163841 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4fec083b00 con 0x7f4ff80aefd0 2023-12-26T21:36:55.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.561+0000 7f4fe97fa700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f4fdc06f290 0x7f4fdc0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.561+0000 7f4fe97fa700 1 -- 172.21.15.131:0/366163841 --> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fdc071d50 con 0x7f4fdc06f290 2023-12-26T21:36:55.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.561+0000 7f4fe97fa700 1 -- 172.21.15.131:0/366163841 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fec0881a0 con 0x7f4ff80aefd0 2023-12-26T21:36:55.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.561+0000 7fcede7fc700 1 -- 172.21.15.131:0/203243830 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcee0007690 con 0x7fcee80dde90 2023-12-26T21:36:55.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.562+0000 7f4ffd99c700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f4fdc06f290 0x7f4fdc0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.562+0000 7f4ffc99a700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fdc067080 0x7f4fdc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.562+0000 7f4ffd99c700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f4fdc06f290 0x7f4fdc0716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7f2f3006f210 msgr2=0x7f2f30071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7f2f49fff700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7f2f3006f210 0x7f2f30071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2f30067080 msgr2=0x7f2f30069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7f2f49fff700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2f30067080 0x7f2f30069530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f4406ecb0 tx=0x7f2f3c007480 comp rx=0 tx=0).stop 2023-12-26T21:36:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f4416f010 msgr2=0x7f2f4414fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7f2f49fff700 1 --2- 172.21.15.131:0/2874598560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f4416f010 0x7f2f4414fee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f440892d0 tx=0x7f2f38017000 comp rx=0 tx=0).stop 2023-12-26T21:36:55.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7f4fe97fa700 1 -- 172.21.15.131:0/366163841 <== osd.4 v2:172.21.15.190:6801/40010 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4fdc071d50 con 0x7f4fdc06f290 2023-12-26T21:36:55.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7fcede7fc700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fced0066f80 0x7fced0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7fcede7fc700 1 -- 172.21.15.131:0/203243830 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcee0083970 con 0x7fcee80dde90 2023-12-26T21:36:55.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7fcede7fc700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7fced006f020 0x7fced0071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7fcede7fc700 1 -- 172.21.15.131:0/203243830 --> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fced0071ae0 con 0x7fced006f020 2023-12-26T21:36:55.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7fcede7fc700 1 -- 172.21.15.131:0/203243830 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee0088220 con 0x7fcee80dde90 2023-12-26T21:36:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.563+0000 7f2f43fff700 1 -- 172.21.15.131:0/2874598560 reap_dead start 2023-12-26T21:36:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.564+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 shutdown_connections 2023-12-26T21:36:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.564+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 >> 172.21.15.131:0/2874598560 conn(0x7f2f4406ece0 msgr2=0x7f2f44158270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.564+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 shutdown_connections 2023-12-26T21:36:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.564+0000 7fceee59c700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fced0066f80 0x7fced0069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.564+0000 7fceedd9b700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7fced006f020 0x7fced0071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.564+0000 7f4ffc99a700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fdc067080 0x7f4fdc069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80cb670 tx=0x7f4ff4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.564+0000 7f2f49fff700 1 -- 172.21.15.131:0/2874598560 wait complete. 2023-12-26T21:36:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.565+0000 7fceedd9b700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7fced006f020 0x7fced0071430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.565+0000 7fceee59c700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fced0066f80 0x7fced0069430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcee800d7c0 tx=0x7fcef008e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.566+0000 7fcede7fc700 1 -- 172.21.15.131:0/203243830 <== osd.7 v2:172.21.15.190:6800/40011 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fced0071ae0 con 0x7fced006f020 2023-12-26T21:36:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.585+0000 7ff95affd700 1 -- 172.21.15.131:0/4023696793 --> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9700657e0 con 0x7ff95406f210 2023-12-26T21:36:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.587+0000 7ff9657fa700 1 -- 172.21.15.131:0/4023696793 <== osd.5 v2:172.21.15.190:6817/40012 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff9700657e0 con 0x7ff95406f210 2023-12-26T21:36:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.587+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 --> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f17080717f0 con 0x7f16f406f170 2023-12-26T21:36:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7f1705ffb700 1 -- 172.21.15.131:0/767464352 <== osd.3 v2:172.21.15.131:6817/39609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f17080717f0 con 0x7f16f406f170 2023-12-26T21:36:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7ff95affd700 1 -- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7ff95406f210 msgr2=0x7ff954071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7ff95affd700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7ff95406f210 0x7ff954071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7f6e36bff700 1 -- 172.21.15.131:0/356492275 --> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e3006bde0 con 0x7f6e1c06f140 2023-12-26T21:36:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7ff95affd700 1 -- 172.21.15.131:0/4023696793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff954067000 msgr2=0x7ff9540694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7ff95affd700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff954067000 0x7ff9540694b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff960007fd0 tx=0x7ff9600079d0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7ff95affd700 1 -- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff970144e80 msgr2=0x7ff970139a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7ff95affd700 1 --2- 172.21.15.131:0/4023696793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff970144e80 0x7ff970139a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0119f0 tx=0x7ff96c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7ff97599c700 1 -- 172.21.15.131:0/4023696793 reap_dead start 2023-12-26T21:36:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7f16f406f170 msgr2=0x7f16f4071580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.588+0000 7f170e99d700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7f16f406f170 0x7f16f4071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 --> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f836c0657e0 con 0x7f835006f190 2023-12-26T21:36:55.609 INFO:teuthology.orchestra.run.smithi131.stdout:64424509447 2023-12-26T21:36:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f6e2cff9700 1 -- 172.21.15.131:0/356492275 <== osd.2 v2:172.21.15.131:6800/39608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6e3006bde0 con 0x7f6e1c06f140 2023-12-26T21:36:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066fe0 msgr2=0x7f16f4069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f170e99d700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16f4066fe0 0x7f16f4069490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16fc001a00 tx=0x7f16fc009040 comp rx=0 tx=0).stop 2023-12-26T21:36:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170806f480 msgr2=0x7f17080af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f170e99d700 1 --2- 172.21.15.131:0/767464352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170806f480 0x7f17080af2d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1700015000 tx=0x7f1700016000 comp rx=0 tx=0).stop 2023-12-26T21:36:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f170e19c700 1 -- 172.21.15.131:0/767464352 reap_dead start 2023-12-26T21:36:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 shutdown_connections 2023-12-26T21:36:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 >> 172.21.15.131:0/767464352 conn(0x7f1708068040 msgr2=0x7f17080d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 shutdown_connections 2023-12-26T21:36:55.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.590+0000 7f170e99d700 1 -- 172.21.15.131:0/767464352 wait complete. 2023-12-26T21:36:55.611 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-26T21:36:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7ff95affd700 1 -- 172.21.15.131:0/4023696793 shutdown_connections 2023-12-26T21:36:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7ff95affd700 1 -- 172.21.15.131:0/4023696793 >> 172.21.15.131:0/4023696793 conn(0x7ff97006a060 msgr2=0x7ff970150c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7ff95affd700 1 -- 172.21.15.131:0/4023696793 shutdown_connections 2023-12-26T21:36:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.589+0000 7ff95affd700 1 -- 172.21.15.131:0/4023696793 wait complete. 2023-12-26T21:36:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.590+0000 7f836a7fc700 1 -- 172.21.15.131:0/3176897483 <== osd.6 v2:172.21.15.190:6816/40035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f836c0657e0 con 0x7f835006f190 2023-12-26T21:36:55.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.590+0000 7f6e167fc700 1 -- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f6e1c06f140 msgr2=0x7f6e1c071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.590+0000 7f6e167fc700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f6e1c06f140 0x7f6e1c071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f6e167fc700 1 -- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e1c066f30 msgr2=0x7f6e1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f6e167fc700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e1c066f30 0x7f6e1c0693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e200098e0 tx=0x7f6e20016000 comp rx=0 tx=0).stop 2023-12-26T21:36:55.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f6e167fc700 1 -- 172.21.15.131:0/356492275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3015d950 msgr2=0x7f6e300acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f6e167fc700 1 --2- 172.21.15.131:0/356492275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3015d950 0x7f6e300acb00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6e28001700 tx=0x7f6e28007d50 comp rx=0 tx=0).stop 2023-12-26T21:36:55.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f6e3519c700 1 -- 172.21.15.131:0/356492275 reap_dead start 2023-12-26T21:36:55.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f6e167fc700 1 -- 172.21.15.131:0/356492275 shutdown_connections 2023-12-26T21:36:55.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f6e167fc700 1 -- 172.21.15.131:0/356492275 >> 172.21.15.131:0/356492275 conn(0x7f6e3006a060 msgr2=0x7f6e3015e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7f835006f190 msgr2=0x7f83500715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f83745ff700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7f835006f190 0x7f83500715a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f6e167fc700 1 -- 172.21.15.131:0/356492275 shutdown_connections 2023-12-26T21:36:55.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f6e167fc700 1 -- 172.21.15.131:0/356492275 wait complete. 2023-12-26T21:36:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8350067000 msgr2=0x7f83500694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f83745ff700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8350067000 0x7f83500694b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f836c088f70 tx=0x7f8360019000 comp rx=0 tx=0).stop 2023-12-26T21:36:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836c0ced90 msgr2=0x7f836c0b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.591+0000 7f83745ff700 1 --2- 172.21.15.131:0/3176897483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836c0ced90 0x7f836c0b55b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f835800c560 tx=0x7f8358007d50 comp rx=0 tx=0).stop 2023-12-26T21:36:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.592+0000 7f8372b9c700 1 -- 172.21.15.131:0/3176897483 reap_dead start 2023-12-26T21:36:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.592+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 shutdown_connections 2023-12-26T21:36:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.592+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 >> 172.21.15.131:0/3176897483 conn(0x7f836c099980 msgr2=0x7f836c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.592+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 shutdown_connections 2023-12-26T21:36:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.592+0000 7f83745ff700 1 -- 172.21.15.131:0/3176897483 wait complete. 2023-12-26T21:36:55.616 INFO:teuthology.orchestra.run.smithi131.stdout:64424509447 2023-12-26T21:36:55.617 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-26T21:36:55.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.596+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 --> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcee80377b0 con 0x7fced006f020 2023-12-26T21:36:55.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.597+0000 7fcede7fc700 1 -- 172.21.15.131:0/203243830 <== osd.7 v2:172.21.15.190:6800/40011 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcee80377b0 con 0x7fced006f020 2023-12-26T21:36:55.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.597+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 --> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4ff815ef50 con 0x7f4fdc06f290 2023-12-26T21:36:55.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7fced006f020 msgr2=0x7fced0071430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7fced006f020 0x7fced0071430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7f4fe97fa700 1 -- 172.21.15.131:0/366163841 <== osd.4 v2:172.21.15.190:6801/40010 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4ff815ef50 con 0x7f4fdc06f290 2023-12-26T21:36:55.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fced0066f80 msgr2=0x7fced0069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fced0066f80 0x7fced0069430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcee800d7c0 tx=0x7fcef008e000 comp rx=0 tx=0).stop 2023-12-26T21:36:55.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 msgr2=0x7fcee80e0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 --2- 172.21.15.131:0/203243830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcee80dde90 0x7fcee80e0a00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcee00010e0 tx=0x7fcee0007dd0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fcef4fff700 1 -- 172.21.15.131:0/203243830 reap_dead start 2023-12-26T21:36:55.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 shutdown_connections 2023-12-26T21:36:55.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 >> 172.21.15.131:0/203243830 conn(0x7fcee80518b0 msgr2=0x7fcee80d8300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 shutdown_connections 2023-12-26T21:36:55.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.598+0000 7fceef59e700 1 -- 172.21.15.131:0/203243830 wait complete. 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.599+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f4fdc06f290 msgr2=0x7f4fdc0716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.599+0000 7f4fff3ff700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f4fdc06f290 0x7f4fdc0716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.599+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fdc067080 msgr2=0x7f4fdc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.599+0000 7f4fff3ff700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fdc067080 0x7f4fdc069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80cb670 tx=0x7f4ff4009040 comp rx=0 tx=0).stop 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.599+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 msgr2=0x7f4ff8163110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.599+0000 7f4fff3ff700 1 --2- 172.21.15.131:0/366163841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff80aefd0 0x7f4ff8163110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00c560 tx=0x7f4fec001020 comp rx=0 tx=0).stop 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.599+0000 7f4ffd99c700 1 -- 172.21.15.131:0/366163841 reap_dead start 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.599+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 shutdown_connections 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.599+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 >> 172.21.15.131:0/366163841 conn(0x7f4ff8068040 msgr2=0x7f4ff815b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.601+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 shutdown_connections 2023-12-26T21:36:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.601+0000 7f4fff3ff700 1 -- 172.21.15.131:0/366163841 wait complete. 2023-12-26T21:36:55.622 INFO:teuthology.orchestra.run.smithi131.stdout:60129542152 2023-12-26T21:36:55.622 INFO:teuthology.orchestra.run.smithi131.stdout:64424509446 2023-12-26T21:36:55.623 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-26T21:36:55.623 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-26T21:36:55.628 INFO:teuthology.orchestra.run.smithi131.stdout:64424509448 2023-12-26T21:36:55.628 INFO:teuthology.orchestra.run.smithi131.stdout:64424509447 2023-12-26T21:36:55.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-26T21:36:55.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-26T21:36:55.633 INFO:teuthology.orchestra.run.smithi131.stdout:60129542151 2023-12-26T21:36:55.633 INFO:teuthology.orchestra.run.smithi131.stdout:60129542151 2023-12-26T21:36:55.633 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-26T21:36:55.634 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-26T21:36:55.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.950+0000 7f5bdb39d700 1 Processor -- start 2023-12-26T21:36:55.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.950+0000 7f5bdb39d700 1 -- start start 2023-12-26T21:36:55.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.953+0000 7f5bdb39d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 0x7f5bcc033970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.954+0000 7f5bdb39d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bcc053ac0 con 0x7f5bcc033eb0 2023-12-26T21:36:55.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.954+0000 7f5bdb39d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bcc050b00 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.954+0000 7f5bdb39d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc034330 con 0x7f5bcc006060 2023-12-26T21:36:55.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.955+0000 7f5bd9b9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 0x7f5bcc033970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.955+0000 7f5bdab9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5bcc033eb0 0x7f5bcc053f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49412/0 (socket says 172.21.15.131:49412) 2023-12-26T21:36:55.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.955+0000 7f5bdab9c700 1 -- 172.21.15.131:0/2574986681 learned_addr learned my addr 172.21.15.131:0/2574986681 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.955+0000 7f5bd9b9a700 1 -- 172.21.15.131:0/2574986681 >> v1:172.21.15.190:6789/0 conn(0x7f5bcc0d51e0 legacy=0x7f5bcc04fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.955+0000 7f5bd9b9a700 1 -- 172.21.15.131:0/2574986681 >> v1:172.21.15.131:6789/0 conn(0x7f5bcc033eb0 legacy=0x7f5bcc053f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.955+0000 7f5bd9b9a700 1 -- 172.21.15.131:0/2574986681 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc0064b0 con 0x7f5bcc006060 2023-12-26T21:36:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.956+0000 7f5bd9b9a700 1 --2- 172.21.15.131:0/2574986681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 0x7f5bcc033970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5bd404c860 tx=0x7f5bd406a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c007ff8d9a189cde server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.957+0000 7f5bd9399700 1 -- 172.21.15.131:0/2574986681 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4064630 con 0x7f5bcc006060 2023-12-26T21:36:55.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.957+0000 7ff728f9d700 1 Processor -- start 2023-12-26T21:36:55.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.957+0000 7f5bd9399700 1 -- 172.21.15.131:0/2574986681 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd4064790 con 0x7f5bcc006060 2023-12-26T21:36:55.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.957+0000 7f5bd9399700 1 -- 172.21.15.131:0/2574986681 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd406f6d0 con 0x7f5bcc006060 2023-12-26T21:36:55.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.957+0000 7ff728f9d700 1 -- start start 2023-12-26T21:36:55.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.959+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2574986681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 msgr2=0x7f5bcc033970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.959+0000 7f5bdb39d700 1 --2- 172.21.15.131:0/2574986681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 0x7f5bcc033970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5bd404c860 tx=0x7f5bd406a050 comp rx=0 tx=0).stop 2023-12-26T21:36:55.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.959+0000 7ff728f9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0e6320 0x7ff71c0e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.959+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2574986681 shutdown_connections 2023-12-26T21:36:55.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.959+0000 7f5bdb39d700 1 --2- 172.21.15.131:0/2574986681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 0x7f5bcc033970 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.959+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2574986681 >> 172.21.15.131:0/2574986681 conn(0x7f5bcc051840 msgr2=0x7f5bcc051c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7ff728f9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff71c034b20 0x7ff71c034f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7ff728f9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71c053bb0 con 0x7ff71c0ea270 2023-12-26T21:36:55.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7ff728f9d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71c0eb0f0 con 0x7ff71c034b20 2023-12-26T21:36:55.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7ff728f9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c052fa0 con 0x7ff71c0e6320 2023-12-26T21:36:55.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7fb35119d700 1 Processor -- start 2023-12-26T21:36:55.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7fb35119d700 1 -- start start 2023-12-26T21:36:55.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7ff722ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff71c034b20 0x7ff71c034f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7ff722ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff71c034b20 0x7ff71c034f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44834/0 (socket says 172.21.15.131:44834) 2023-12-26T21:36:55.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7ff722ffd700 1 -- 172.21.15.131:0/2995574435 learned_addr learned my addr 172.21.15.131:0/2995574435 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.961+0000 7ff722ffd700 1 -- 172.21.15.131:0/2995574435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0e6320 msgr2=0x7ff71c0e66f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-26T21:36:55.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.961+0000 7ff722ffd700 1 --2- 172.21.15.131:0/2995574435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0e6320 0x7ff71c0e66f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.961+0000 7ff722ffd700 1 -- 172.21.15.131:0/2995574435 >> v1:172.21.15.190:6789/0 conn(0x7ff71c0ea270 legacy=0x7ff71c054040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.961+0000 7ff722ffd700 1 -- 172.21.15.131:0/2995574435 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c053280 con 0x7ff71c034b20 2023-12-26T21:36:55.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2574986681 shutdown_connections 2023-12-26T21:36:55.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.960+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2574986681 wait complete. 2023-12-26T21:36:55.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.962+0000 7ff722ffd700 1 --2- 172.21.15.131:0/2995574435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 0x7ff71c034f90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff7100099b0 tx=0x7ff710009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444c3e23c1d76d6a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.962+0000 7f5bdb39d700 1 Processor -- start 2023-12-26T21:36:55.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7fb35119d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb34c0b02a0 0x7fb34c069590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7fb35119d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb34c0b06f0 0x7fb34c069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7fb35119d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34c0ad430 con 0x7fb34c0afc60 2023-12-26T21:36:55.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7fb35119d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c0b0ef0 con 0x7fb34c0b06f0 2023-12-26T21:36:55.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7fb35119d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c0c8c70 con 0x7fb34c0b02a0 2023-12-26T21:36:55.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7fb34b7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb34c0b02a0 0x7fb34c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7fb34b7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb34c0b02a0 0x7fb34c069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53414/0 (socket says 172.21.15.131:53414) 2023-12-26T21:36:55.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7fb34b7fe700 1 -- 172.21.15.131:0/2745189647 learned_addr learned my addr 172.21.15.131:0/2745189647 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7fb35099c700 1 --2- 172.21.15.131:0/2745189647 >> v2:172.21.15.131:3300/0 conn(0x7fb34c0b06f0 0x7fb34c069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.963+0000 7f5bdb39d700 1 -- start start 2023-12-26T21:36:55.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7f5bdb39d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 0x7f5bcc039f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7f5bdb39d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc033eb0 0x7f5bcc04e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7f5bdb39d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bcc0d51e0 0x7f5bcc0d3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7f5bdb39d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc04c290 con 0x7f5bcc033eb0 2023-12-26T21:36:55.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7f5bdb39d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc03a5a0 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.964+0000 7f5bdb39d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc03a880 con 0x7f5bcc006060 2023-12-26T21:36:55.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7fb34b7fe700 1 -- 172.21.15.131:0/2745189647 >> v2:172.21.15.131:3300/0 conn(0x7fb34c0b06f0 msgr2=0x7fb34c069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:55.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7fb34b7fe700 1 --2- 172.21.15.131:0/2745189647 >> v2:172.21.15.131:3300/0 conn(0x7fb34c0b06f0 0x7fb34c069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7fb34b7fe700 1 -- 172.21.15.131:0/2745189647 >> v1:172.21.15.131:6789/0 conn(0x7fb34c0afc60 legacy=0x7fb34c156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:55.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7fb34b7fe700 1 -- 172.21.15.131:0/2745189647 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c0acb10 con 0x7fb34c0b02a0 2023-12-26T21:36:55.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7fb35099c700 1 --2- 172.21.15.131:0/2745189647 >> v2:172.21.15.131:3300/0 conn(0x7fb34c0b06f0 0x7fb34c069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:55.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7fb34b7fe700 1 --2- 172.21.15.131:0/2745189647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0b02a0 0x7fb34c069590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb34c06d0a0 tx=0x7fb33c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f05e0d0d9bf4712 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7f5bdab9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bcc0d51e0 0x7f5bcc0d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7f5bdab9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bcc0d51e0 0x7f5bcc0d3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53422/0 (socket says 172.21.15.131:53422) 2023-12-26T21:36:55.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.965+0000 7f5bdab9c700 1 -- 172.21.15.131:0/2601230441 learned_addr learned my addr 172.21.15.131:0/2601230441 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.966+0000 7f5bdab9c700 1 -- 172.21.15.131:0/2601230441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 msgr2=0x7f5bcc039f90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-26T21:36:55.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.966+0000 7f5bdab9c700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 0x7f5bcc039f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.966+0000 7f5bdab9c700 1 -- 172.21.15.131:0/2601230441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc033eb0 msgr2=0x7f5bcc04e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.966+0000 7f5bdab9c700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc033eb0 0x7f5bcc04e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.966+0000 7f5bdab9c700 1 -- 172.21.15.131:0/2601230441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd404c9a0 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.966+0000 7f5bdab9c700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bcc0d51e0 0x7f5bcc0d3790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c9b0 tx=0x7f5bd000a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.966+0000 7fb34affd700 1 -- 172.21.15.131:0/2745189647 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00b820 con 0x7fb34c0b02a0 2023-12-26T21:36:55.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.967+0000 7fb34affd700 1 -- 172.21.15.131:0/2745189647 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c00b980 con 0x7fb34c0b02a0 2023-12-26T21:36:55.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.967+0000 7fb34affd700 1 -- 172.21.15.131:0/2745189647 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00bd20 con 0x7fb34c0b02a0 2023-12-26T21:36:55.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.967+0000 7ff7227fc700 1 -- 172.21.15.131:0/2995574435 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71000b7a0 con 0x7ff71c034b20 2023-12-26T21:36:55.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.967+0000 7ff7227fc700 1 -- 172.21.15.131:0/2995574435 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71000b900 con 0x7ff71c034b20 2023-12-26T21:36:55.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.967+0000 7ff7227fc700 1 -- 172.21.15.131:0/2995574435 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71000bca0 con 0x7ff71c034b20 2023-12-26T21:36:55.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.967+0000 7f5bca7fc700 1 -- 172.21.15.131:0/2601230441 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0007940 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.967+0000 7f5bca7fc700 1 -- 172.21.15.131:0/2601230441 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd0007ac0 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.967+0000 7f5bca7fc700 1 -- 172.21.15.131:0/2601230441 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0007e80 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7ff728f9d700 1 -- 172.21.15.131:0/2995574435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 msgr2=0x7ff71c034f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc03ab60 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7ff728f9d700 1 --2- 172.21.15.131:0/2995574435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 0x7ff71c034f90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff7100099b0 tx=0x7ff710009250 comp rx=0 tx=0).stop 2023-12-26T21:36:55.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc0df490 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7fb35119d700 1 -- 172.21.15.131:0/2745189647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0b02a0 msgr2=0x7fb34c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7fb35119d700 1 --2- 172.21.15.131:0/2745189647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0b02a0 0x7fb34c069590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb34c06d0a0 tx=0x7fb33c009250 comp rx=0 tx=0).stop 2023-12-26T21:36:55.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7fb35119d700 1 -- 172.21.15.131:0/2745189647 shutdown_connections 2023-12-26T21:36:55.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7fb35119d700 1 --2- 172.21.15.131:0/2745189647 >> v2:172.21.15.131:3300/0 conn(0x7fb34c0b06f0 0x7fb34c069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7fb35119d700 1 --2- 172.21.15.131:0/2745189647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0b02a0 0x7fb34c069590 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.968+0000 7fb35119d700 1 -- 172.21.15.131:0/2745189647 >> 172.21.15.131:0/2745189647 conn(0x7fb34c0675b0 msgr2=0x7fb34c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.969+0000 7fb35119d700 1 -- 172.21.15.131:0/2745189647 shutdown_connections 2023-12-26T21:36:55.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.969+0000 7fb35119d700 1 -- 172.21.15.131:0/2745189647 wait complete. 2023-12-26T21:36:55.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.971+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc005480 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.971+0000 7ff728f9d700 1 -- 172.21.15.131:0/2995574435 shutdown_connections 2023-12-26T21:36:55.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.971+0000 7ff728f9d700 1 --2- 172.21.15.131:0/2995574435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0e6320 0x7ff71c0e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.971+0000 7ff728f9d700 1 --2- 172.21.15.131:0/2995574435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 0x7ff71c034f90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.971+0000 7ff728f9d700 1 -- 172.21.15.131:0/2995574435 >> 172.21.15.131:0/2995574435 conn(0x7ff71c054dd0 msgr2=0x7ff71c0551d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.971+0000 7fb35119d700 1 Processor -- start 2023-12-26T21:36:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.972+0000 7ff728f9d700 1 -- 172.21.15.131:0/2995574435 shutdown_connections 2023-12-26T21:36:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.972+0000 7fb35119d700 1 -- start start 2023-12-26T21:36:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.973+0000 7fb35119d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0afc60 0x7fb34c145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.973+0000 7fb35119d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0b02a0 0x7fb34c136300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.973+0000 7fb35119d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb34c0b06f0 0x7fb34c137470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.973+0000 7fb35119d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c136ce0 con 0x7fb34c0b06f0 2023-12-26T21:36:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.973+0000 7fb35119d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c1445c0 con 0x7fb34c0b02a0 2023-12-26T21:36:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.973+0000 7fb35119d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c1448a0 con 0x7fb34c0afc60 2023-12-26T21:36:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.972+0000 7f5bca7fc700 1 -- 172.21.15.131:0/2601230441 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5bd000f450 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.973+0000 7f5bca7fc700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5bb8067000 0x7f5bb80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.973+0000 7ff728f9d700 1 -- 172.21.15.131:0/2995574435 wait complete. 2023-12-26T21:36:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb34bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0afc60 0x7fb34c145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb34bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0afc60 0x7fb34c145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57352/0 (socket says 172.21.15.131:57352) 2023-12-26T21:36:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb34bfff700 1 -- 172.21.15.131:0/326098211 learned_addr learned my addr 172.21.15.131:0/326098211 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7f5bda39b700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5bb8067000 0x7f5bb80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb34b7fe700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0b02a0 0x7fb34c136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb35099c700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb34c0b06f0 0x7fb34c137470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb34bfff700 1 -- 172.21.15.131:0/326098211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0b02a0 msgr2=0x7fb34c136300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb34bfff700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0b02a0 0x7fb34c136300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb34bfff700 1 -- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb34c0b06f0 msgr2=0x7fb34c137470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb34bfff700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb34c0b06f0 0x7fb34c137470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.974+0000 7fb34bfff700 1 -- 172.21.15.131:0/326098211 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0094f0 con 0x7fb34c0afc60 2023-12-26T21:36:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.975+0000 7fb34bfff700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0afc60 0x7fb34c145380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb34000b600 tx=0x7fb3400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.975+0000 7fb333fff700 1 -- 172.21.15.131:0/326098211 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34000f510 con 0x7fb34c0afc60 2023-12-26T21:36:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.975+0000 7fb333fff700 1 -- 172.21.15.131:0/326098211 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34000f670 con 0x7fb34c0afc60 2023-12-26T21:36:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.975+0000 7fb333fff700 1 -- 172.21.15.131:0/326098211 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34000fa30 con 0x7fb34c0afc60 2023-12-26T21:36:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.975+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c144b80 con 0x7fb34c0afc60 2023-12-26T21:36:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.975+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c1458c0 con 0x7fb34c0afc60 2023-12-26T21:36:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.975+0000 7f5bca7fc700 1 -- 172.21.15.131:0/2601230441 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5bd0013070 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.975+0000 7f5bda39b700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5bb8067000 0x7f5bb80694b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4000f60 tx=0x7f5bc4007b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.977+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c0052b0 con 0x7fb34c0afc60 2023-12-26T21:36:55.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.977+0000 7ff728f9d700 1 Processor -- start 2023-12-26T21:36:55.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.977+0000 7ff728f9d700 1 -- start start 2023-12-26T21:36:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.978+0000 7fb333fff700 1 -- 172.21.15.131:0/326098211 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb34001a400 con 0x7fb34c0afc60 2023-12-26T21:36:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.979+0000 7fb333fff700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb338067000 0x7fb3380694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.979+0000 7fb333fff700 1 -- 172.21.15.131:0/326098211 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb340083d70 con 0x7fb34c0afc60 2023-12-26T21:36:55.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.979+0000 7fb34b7fe700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb338067000 0x7fb3380694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.979+0000 7ff728f9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 0x7ff71c0e9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.979+0000 7ff728f9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0e6320 0x7ff71c0e11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.979+0000 7ff728f9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0ea270 0x7ff71c0058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.979+0000 7ff728f9d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c039620 con 0x7ff71c034b20 2023-12-26T21:36:55.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.979+0000 7ff728f9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0fa5c0 con 0x7ff71c0e6320 2023-12-26T21:36:55.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.979+0000 7ff728f9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0fa8a0 con 0x7ff71c0ea270 2023-12-26T21:36:55.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.980+0000 7fb34b7fe700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb338067000 0x7fb3380694b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb34c06d0a0 tx=0x7fb33c016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.980+0000 7ff7237fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 0x7ff71c0e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.980+0000 7ff7237fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 0x7ff71c0e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44864/0 (socket says 172.21.15.131:44864) 2023-12-26T21:36:55.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.980+0000 7ff7237fe700 1 -- 172.21.15.131:0/3829607145 learned_addr learned my addr 172.21.15.131:0/3829607145 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.980+0000 7ff722ffd700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0e6320 0x7ff71c0e11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff722ffd700 1 -- 172.21.15.131:0/3829607145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0ea270 msgr2=0x7ff71c0058a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff722ffd700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0ea270 0x7ff71c0058a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff722ffd700 1 -- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 msgr2=0x7ff71c0e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff722ffd700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 0x7ff71c0e9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff722ffd700 1 -- 172.21.15.131:0/3829607145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100094f0 con 0x7ff71c0e6320 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff7237fe700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 0x7ff71c0e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff722ffd700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0e6320 0x7ff71c0e11d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0354d0 tx=0x7ff710016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3829607145 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71001a460 con 0x7ff71c0e6320 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3829607145 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71001a5e0 con 0x7ff71c0e6320 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.981+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3829607145 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71001a9a0 con 0x7ff71c0e6320 2023-12-26T21:36:55.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.982+0000 7f5bca7fc700 1 -- 172.21.15.131:0/2601230441 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bd0051d60 con 0x7f5bcc0d51e0 2023-12-26T21:36:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.982+0000 7ff728f9d700 1 -- 172.21.15.131:0/3829607145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71c0fab80 con 0x7ff71c0e6320 2023-12-26T21:36:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.982+0000 7ff728f9d700 1 -- 172.21.15.131:0/3829607145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c0e1710 con 0x7ff71c0e6320 2023-12-26T21:36:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.984+0000 7ff70d7fa700 1 -- 172.21.15.131:0/3829607145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc0052b0 con 0x7ff71c0e6320 2023-12-26T21:36:55.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.984+0000 7f90c9fff700 1 Processor -- start 2023-12-26T21:36:55.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.985+0000 7fb333fff700 1 -- 172.21.15.131:0/326098211 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb340051140 con 0x7fb34c0afc60 2023-12-26T21:36:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.984+0000 7f90c9fff700 1 -- start start 2023-12-26T21:36:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.986+0000 7f90c9fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f90c40af970 0x7f90c40afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.986+0000 7f90c9fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c40aea80 con 0x7f90c4155630 2023-12-26T21:36:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.986+0000 7f90c9fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c40b2100 con 0x7f90c41578e0 2023-12-26T21:36:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.986+0000 7f90c9fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c40b0350 con 0x7f90c40af970 2023-12-26T21:36:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.984+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3829607145 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff710025440 con 0x7ff71c0e6320 2023-12-26T21:36:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.986+0000 7ff70f7fe700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff708066f30 0x7ff7080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.986+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3829607145 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff710086290 con 0x7ff71c0e6320 2023-12-26T21:36:55.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.987+0000 7ff7237fe700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff708066f30 0x7ff7080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.987+0000 7f90c3fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f90c41578e0 0x7f90c4157d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39796/0 (socket says 172.21.15.131:39796) 2023-12-26T21:36:55.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.987+0000 7f90c3fff700 1 -- 172.21.15.131:0/3963316976 learned_addr learned my addr 172.21.15.131:0/3963316976 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:55.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.988+0000 7ff7237fe700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff708066f30 0x7ff7080693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff71800e1d0 tx=0x7ff718000f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.988+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543694418 0 0) 0x7f90c40aea80 con 0x7f90c4155630 2023-12-26T21:36:55.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.988+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90ac003570 con 0x7f90c4155630 2023-12-26T21:36:55.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c2ffd700 1 --2- 172.21.15.131:0/3963316976 >> v2:172.21.15.131:3300/0 conn(0x7f90c40af970 0x7f90c40afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3321009713 0 0) 0x7f90ac003570 con 0x7f90c4155630 2023-12-26T21:36:55.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 >> v2:172.21.15.131:3300/0 conn(0x7f90c40af970 msgr2=0x7f90c40afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 --2- 172.21.15.131:0/3963316976 >> v2:172.21.15.131:3300/0 conn(0x7f90c40af970 0x7f90c40afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 >> v1:172.21.15.190:6789/0 conn(0x7f90c41578e0 legacy=0x7f90c4157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c4156840 con 0x7f90c4155630 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f90b8001470 con 0x7f90c4155630 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 >> v1:172.21.15.131:6789/0 conn(0x7f90c4155630 legacy=0x7f90c40b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90ac003ab0 0x7f90ac003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90ac004810 0x7f90ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90ac0052e0 0x7f90ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40b0350 con 0x7f90ac004810 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac00c260 con 0x7f90ac003ab0 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac00c420 con 0x7f90ac0052e0 2023-12-26T21:36:55.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c37fe700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90ac003ab0 0x7f90ac003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.989+0000 7f90c3fff700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90ac0052e0 0x7f90ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.990+0000 7f90c37fe700 1 -- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90ac0052e0 msgr2=0x7f90ac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.990+0000 7f90c37fe700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90ac0052e0 0x7f90ac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.990+0000 7f90c37fe700 1 -- 172.21.15.131:0/3963316976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90ac004810 msgr2=0x7f90ac004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.990+0000 7f90c37fe700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90ac004810 0x7f90ac004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.990+0000 7f90c37fe700 1 -- 172.21.15.131:0/3963316976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac003570 con 0x7f90ac003ab0 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.990+0000 7f90c3fff700 1 -- 172.21.15.131:0/3963316976 reap_dead start 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.990+0000 7f90c37fe700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90ac003ab0 0x7f90ac003ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f90b801d1f0 tx=0x7f90b8002e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f60f2e9de5422234 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.991+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b801db20 con 0x7f90ac003ab0 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.991+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b801dca0 con 0x7f90ac003ab0 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.991+0000 7f90c27fc700 1 -- 172.21.15.131:0/3963316976 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8001b90 con 0x7f90ac003ab0 2023-12-26T21:36:55.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.991+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3829607145 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff710053510 con 0x7ff71c0e6320 2023-12-26T21:36:55.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.992+0000 7f90c9fff700 1 -- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90ac003ab0 msgr2=0x7f90ac003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:55.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.992+0000 7f90c9fff700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90ac003ab0 0x7f90ac003ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f90b801d1f0 tx=0x7f90b8002e80 comp rx=0 tx=0).stop 2023-12-26T21:36:55.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.992+0000 7f90c9fff700 1 -- 172.21.15.131:0/3963316976 shutdown_connections 2023-12-26T21:36:55.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.992+0000 7f90c9fff700 1 --2- 172.21.15.131:0/3963316976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90ac003ab0 0x7f90ac003ec0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:55.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.992+0000 7f90c9fff700 1 -- 172.21.15.131:0/3963316976 >> 172.21.15.131:0/3963316976 conn(0x7f90c406a060 msgr2=0x7f90c40cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:55.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.992+0000 7f90c9fff700 1 -- 172.21.15.131:0/3963316976 shutdown_connections 2023-12-26T21:36:55.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.992+0000 7f90c9fff700 1 -- 172.21.15.131:0/3963316976 wait complete. 2023-12-26T21:36:55.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.995+0000 7f90c9fff700 1 Processor -- start 2023-12-26T21:36:55.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.995+0000 7f90c9fff700 1 -- start start 2023-12-26T21:36:55.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.997+0000 7f90c9fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90ac003ab0 0x7f90c4142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f90c9fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40a7e80 0x7f90c40ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f90c9fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40a82f0 0x7f90c40ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:55.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f90c9fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40aa1f0 con 0x7f90c40a7e80 2023-12-26T21:36:55.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f4822dff700 1 Processor -- start 2023-12-26T21:36:55.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f90c9fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40a87f0 con 0x7f90c40a82f0 2023-12-26T21:36:55.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f90c9fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40a98c0 con 0x7f90ac003ab0 2023-12-26T21:36:55.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f90c3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40a82f0 0x7f90c40ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:55.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f90c3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40a82f0 0x7f90c40ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53460/0 (socket says 172.21.15.131:53460) 2023-12-26T21:36:55.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f90c3fff700 1 -- 172.21.15.131:0/2791441470 learned_addr learned my addr 172.21.15.131:0/2791441470 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:55.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f4822dff700 1 -- start start 2023-12-26T21:36:56.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.998+0000 7f90c37fe700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90ac003ab0 0x7f90c4142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.999+0000 7f90c3fff700 1 -- 172.21.15.131:0/2791441470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90ac003ab0 msgr2=0x7f90c4142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.999+0000 7f90c3fff700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90ac003ab0 0x7f90c4142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.999+0000 7f90c3fff700 1 -- 172.21.15.131:0/2791441470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40a7e80 msgr2=0x7f90c40ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.999+0000 7f90c3fff700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40a7e80 0x7f90c40ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.999+0000 7f90c3fff700 1 -- 172.21.15.131:0/2791441470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c40a9ba0 con 0x7f90c40a82f0 2023-12-26T21:36:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.999+0000 7f90c3fff700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40a82f0 0x7f90c40ac650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90bc000f80 tx=0x7f90bc0052a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:55.999+0000 7f90ab7fe700 1 -- 172.21.15.131:0/2791441470 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc019030 con 0x7f90c40a82f0 2023-12-26T21:36:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.000+0000 7f90ab7fe700 1 -- 172.21.15.131:0/2791441470 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc010610 con 0x7f90c40a82f0 2023-12-26T21:36:56.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.000+0000 7f90ab7fe700 1 -- 172.21.15.131:0/2791441470 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0109d0 con 0x7f90c40a82f0 2023-12-26T21:36:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.000+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c40a9e80 con 0x7f90c40a82f0 2023-12-26T21:36:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.000+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c40a70b0 con 0x7f90c40a82f0 2023-12-26T21:36:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.001+0000 7f4822dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48140e3e70 0x7f48140e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.001+0000 7f4822dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48140ea1a0 0x7f48140ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.001+0000 7f4822dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48140eab20 con 0x7f4814006930 2023-12-26T21:36:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.001+0000 7f4822dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4814006ea0 con 0x7f48140ea1a0 2023-12-26T21:36:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.001+0000 7f4822dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4814007180 con 0x7f48140e3e70 2023-12-26T21:36:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.002+0000 7f482139c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48140ea1a0 0x7f48140ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.002+0000 7f482139c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48140ea1a0 0x7f48140ea5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44886/0 (socket says 172.21.15.131:44886) 2023-12-26T21:36:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.002+0000 7f482139c700 1 -- 172.21.15.131:0/1648334498 learned_addr learned my addr 172.21.15.131:0/1648334498 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.002+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c40aa010 con 0x7f90c40a82f0 2023-12-26T21:36:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.004+0000 7f482139c700 1 -- 172.21.15.131:0/1648334498 >> v2:172.21.15.190:3300/0 conn(0x7f48140e3e70 msgr2=0x7f48140e4240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-26T21:36:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.004+0000 7f482139c700 1 --2- 172.21.15.131:0/1648334498 >> v2:172.21.15.190:3300/0 conn(0x7f48140e3e70 0x7f48140e4240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.004+0000 7f482139c700 1 -- 172.21.15.131:0/1648334498 >> v1:172.21.15.131:6789/0 conn(0x7f4814006930 legacy=0x7f481404fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.004+0000 7f90ab7fe700 1 -- 172.21.15.131:0/2791441470 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f90bc01c070 con 0x7f90c40a82f0 2023-12-26T21:36:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.005+0000 7f482139c700 1 -- 172.21.15.131:0/1648334498 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4814007460 con 0x7f48140ea1a0 2023-12-26T21:36:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.005+0000 7f482139c700 1 --2- 172.21.15.131:0/1648334498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48140ea1a0 0x7f48140ea5e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f481c04c860 tx=0x7f481c063120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47fd158151551c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.006+0000 7f90ab7fe700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90b0066fb0 0x7f90b0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.006+0000 7f90ab7fe700 1 -- 172.21.15.131:0/2791441470 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f90bc01a070 con 0x7f90c40a82f0 2023-12-26T21:36:56.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.006+0000 7f481b7fe700 1 -- 172.21.15.131:0/1648334498 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c064770 con 0x7f48140ea1a0 2023-12-26T21:36:56.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.006+0000 7f481b7fe700 1 -- 172.21.15.131:0/1648334498 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c0648d0 con 0x7f48140ea1a0 2023-12-26T21:36:56.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.006+0000 7f481b7fe700 1 -- 172.21.15.131:0/1648334498 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c064770 con 0x7f48140ea1a0 2023-12-26T21:36:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.008+0000 7f317d1ff700 1 Processor -- start 2023-12-26T21:36:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.008+0000 7f4822dff700 1 -- 172.21.15.131:0/1648334498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48140ea1a0 msgr2=0x7f48140ea5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.008+0000 7f317d1ff700 1 -- start start 2023-12-26T21:36:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.008+0000 7f4822dff700 1 --2- 172.21.15.131:0/1648334498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48140ea1a0 0x7f48140ea5e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f481c04c860 tx=0x7f481c063120 comp rx=0 tx=0).stop 2023-12-26T21:36:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.006+0000 7f90c37fe700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90b0066fb0 0x7f90b0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.009+0000 7f4822dff700 1 -- 172.21.15.131:0/1648334498 shutdown_connections 2023-12-26T21:36:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.009+0000 7f90c37fe700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90b0066fb0 0x7f90b0069460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f90b8007bc0 tx=0x7f90b8028000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.009+0000 7f4822dff700 1 --2- 172.21.15.131:0/1648334498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48140ea1a0 0x7f48140ea5e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.009+0000 7f4822dff700 1 --2- 172.21.15.131:0/1648334498 >> v2:172.21.15.190:3300/0 conn(0x7f48140e3e70 0x7f48140e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.009+0000 7f4822dff700 1 -- 172.21.15.131:0/1648334498 >> 172.21.15.131:0/1648334498 conn(0x7f4814054c90 msgr2=0x7f4814055090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.010+0000 7f4822dff700 1 -- 172.21.15.131:0/1648334498 shutdown_connections 2023-12-26T21:36:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.011+0000 7f317d1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3178155630 0x7f31780b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.010+0000 7f4822dff700 1 -- 172.21.15.131:0/1648334498 wait complete. 2023-12-26T21:36:56.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.012+0000 7f90ab7fe700 1 -- 172.21.15.131:0/2791441470 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90bc051330 con 0x7f90c40a82f0 2023-12-26T21:36:56.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.012+0000 7f317d1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31781588f0 0x7f31780afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.012+0000 7f317d1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31780aea80 con 0x7f31780af970 2023-12-26T21:36:56.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.012+0000 7f317d1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31780b0350 con 0x7f3178155630 2023-12-26T21:36:56.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.012+0000 7f317d1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3178156840 con 0x7f31781588f0 2023-12-26T21:36:56.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.012+0000 7f317759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31781588f0 0x7f31780afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.012+0000 7f317759e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31781588f0 0x7f31780afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57392/0 (socket says 172.21.15.131:57392) 2023-12-26T21:36:56.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.012+0000 7f317759e700 1 -- 172.21.15.131:0/431696898 learned_addr learned my addr 172.21.15.131:0/431696898 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:56.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.013+0000 7f317759e700 1 -- 172.21.15.131:0/431696898 >> v2:172.21.15.131:3300/0 conn(0x7f3178155630 msgr2=0x7f31780b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:56.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.013+0000 7f317759e700 1 --2- 172.21.15.131:0/431696898 >> v2:172.21.15.131:3300/0 conn(0x7f3178155630 0x7f31780b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.013+0000 7f317759e700 1 -- 172.21.15.131:0/431696898 >> v1:172.21.15.131:6789/0 conn(0x7f31780af970 legacy=0x7f31781592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:56.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.013+0000 7f317759e700 1 -- 172.21.15.131:0/431696898 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3178156b20 con 0x7f31781588f0 2023-12-26T21:36:56.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.013+0000 7f4822dff700 1 Processor -- start 2023-12-26T21:36:56.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.013+0000 7f317759e700 1 --2- 172.21.15.131:0/431696898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31781588f0 0x7f31780afe10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f316c0089b0 tx=0x7f316c0082d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb5e247e55e61573 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.014+0000 7f4822dff700 1 -- start start 2023-12-26T21:36:56.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.014+0000 7f3175d9b700 1 -- 172.21.15.131:0/431696898 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c010070 con 0x7f31781588f0 2023-12-26T21:36:56.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.014+0000 7f3175d9b700 1 -- 172.21.15.131:0/431696898 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c00daf0 con 0x7f31781588f0 2023-12-26T21:36:56.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.014+0000 7f3175d9b700 1 -- 172.21.15.131:0/431696898 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c00de70 con 0x7f31781588f0 2023-12-26T21:36:56.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.015+0000 7f317d1ff700 1 -- 172.21.15.131:0/431696898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31781588f0 msgr2=0x7f31780afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.015+0000 7f317d1ff700 1 --2- 172.21.15.131:0/431696898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31781588f0 0x7f31780afe10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f316c0089b0 tx=0x7f316c0082d0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f4822dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4814006930 0x7f481404e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f4822dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48140e3e70 0x7f481403a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f4820b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4814006930 0x7f481404e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f4822dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48140ea1a0 0x7f48140359d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f4820b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4814006930 0x7f481404e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44908/0 (socket says 172.21.15.131:44908) 2023-12-26T21:36:56.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f4820b9b700 1 -- 172.21.15.131:0/2008472112 learned_addr learned my addr 172.21.15.131:0/2008472112 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:56.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48140dfae0 con 0x7f4814006930 2023-12-26T21:36:56.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48140f6d20 con 0x7f48140e3e70 2023-12-26T21:36:56.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48140f7000 con 0x7f48140ea1a0 2023-12-26T21:36:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.015+0000 7f317d1ff700 1 -- 172.21.15.131:0/431696898 shutdown_connections 2023-12-26T21:36:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.015+0000 7f317d1ff700 1 --2- 172.21.15.131:0/431696898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31781588f0 0x7f31780afe10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.015+0000 7f317d1ff700 1 --2- 172.21.15.131:0/431696898 >> v2:172.21.15.131:3300/0 conn(0x7f3178155630 0x7f31780b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.015+0000 7f317d1ff700 1 -- 172.21.15.131:0/431696898 >> 172.21.15.131:0/431696898 conn(0x7f317806a060 msgr2=0x7f31780cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.016+0000 7f481bfff700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48140e3e70 0x7f481403a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.017+0000 7f482139c700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48140ea1a0 0x7f48140359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.017+0000 7f482139c700 1 -- 172.21.15.131:0/2008472112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48140e3e70 msgr2=0x7f481403a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.017+0000 7f482139c700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48140e3e70 0x7f481403a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.017+0000 7f482139c700 1 -- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4814006930 msgr2=0x7f481404e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.017+0000 7f482139c700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4814006930 0x7f481404e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.017+0000 7f482139c700 1 -- 172.21.15.131:0/2008472112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c04c0e0 con 0x7f48140ea1a0 2023-12-26T21:36:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.017+0000 7f4820b9b700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4814006930 0x7f481404e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.018+0000 7f482139c700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48140ea1a0 0x7f48140359d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f481c04cc40 tx=0x7f481c08b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.018+0000 7f4818ff9700 1 -- 172.21.15.131:0/2008472112 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c064770 con 0x7f48140ea1a0 2023-12-26T21:36:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.018+0000 7f4818ff9700 1 -- 172.21.15.131:0/2008472112 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c067e10 con 0x7f48140ea1a0 2023-12-26T21:36:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.018+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48140f72e0 con 0x7f48140ea1a0 2023-12-26T21:36:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.018+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481404e890 con 0x7f48140ea1a0 2023-12-26T21:36:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.019+0000 7f4818ff9700 1 -- 172.21.15.131:0/2008472112 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c064770 con 0x7f48140ea1a0 2023-12-26T21:36:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.019+0000 7f317d1ff700 1 -- 172.21.15.131:0/431696898 shutdown_connections 2023-12-26T21:36:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.019+0000 7f317d1ff700 1 -- 172.21.15.131:0/431696898 wait complete. 2023-12-26T21:36:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.021+0000 7f4818ff9700 1 -- 172.21.15.131:0/2008472112 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f481c0a3350 con 0x7f48140ea1a0 2023-12-26T21:36:56.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.022+0000 7f317d1ff700 1 Processor -- start 2023-12-26T21:36:56.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.022+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4814005390 con 0x7f48140ea1a0 2023-12-26T21:36:56.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.023+0000 7f317d1ff700 1 -- start start 2023-12-26T21:36:56.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.023+0000 7f4818ff9700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4800066ff0 0x7f48000694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.023+0000 7f4818ff9700 1 -- 172.21.15.131:0/2008472112 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f481c0fc2a0 con 0x7f48140ea1a0 2023-12-26T21:36:56.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.024+0000 7f317d1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31780af970 0x7f317813ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.024+0000 7f317d1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3178155630 0x7f31781534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.024+0000 7f317d1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31781588f0 0x7f3178153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.024+0000 7f317659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3178155630 0x7f31781534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.024+0000 7f317659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3178155630 0x7f31781534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53498/0 (socket says 172.21.15.131:53498) 2023-12-26T21:36:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.024+0000 7f317659c700 1 -- 172.21.15.131:0/2464952975 learned_addr learned my addr 172.21.15.131:0/2464952975 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.024+0000 7f317759e700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31781588f0 0x7f3178153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.025+0000 7f317d1ff700 1 -- 172.21.15.131:0/2464952975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317806efb0 con 0x7f31781588f0 2023-12-26T21:36:56.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.025+0000 7f317d1ff700 1 -- 172.21.15.131:0/2464952975 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317813f180 con 0x7f3178155630 2023-12-26T21:36:56.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.025+0000 7f317d1ff700 1 -- 172.21.15.131:0/2464952975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317806e3a0 con 0x7f31780af970 2023-12-26T21:36:56.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.027+0000 7f4820b9b700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4800066ff0 0x7f48000694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.024+0000 7f3176d9d700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31780af970 0x7f317813ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.027+0000 7f3176d9d700 1 -- 172.21.15.131:0/2464952975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3178155630 msgr2=0x7f31781534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.027+0000 7f3176d9d700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3178155630 0x7f31781534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.027+0000 7f3176d9d700 1 -- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31781588f0 msgr2=0x7f3178153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.027+0000 7f3176d9d700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31781588f0 0x7f3178153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.027+0000 7f3176d9d700 1 -- 172.21.15.131:0/2464952975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c008570 con 0x7f31780af970 2023-12-26T21:36:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.027+0000 7f317759e700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31781588f0 0x7f3178153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:56.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.028+0000 7f3176d9d700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31780af970 0x7f317813ec40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f317000bad0 tx=0x7f3170009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.028+0000 7f4820b9b700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4800066ff0 0x7f48000694a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4810009f60 tx=0x7f48100091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.028+0000 7f315effd700 1 -- 172.21.15.131:0/2464952975 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317000c8c0 con 0x7f31780af970 2023-12-26T21:36:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.028+0000 7f315effd700 1 -- 172.21.15.131:0/2464952975 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317000ca40 con 0x7f31780af970 2023-12-26T21:36:56.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.028+0000 7f315effd700 1 -- 172.21.15.131:0/2464952975 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317000ce00 con 0x7f31780af970 2023-12-26T21:36:56.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.029+0000 7f317d1ff700 1 -- 172.21.15.131:0/2464952975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317806e680 con 0x7f31780af970 2023-12-26T21:36:56.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.029+0000 7f317d1ff700 1 -- 172.21.15.131:0/2464952975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317806ec40 con 0x7f31780af970 2023-12-26T21:36:56.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.029+0000 7f4818ff9700 1 -- 172.21.15.131:0/2008472112 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f481c0c9470 con 0x7f48140ea1a0 2023-12-26T21:36:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.033+0000 7f315effd700 1 -- 172.21.15.131:0/2464952975 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3170014620 con 0x7f31780af970 2023-12-26T21:36:56.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.034+0000 7f315cff9700 1 -- 172.21.15.131:0/2464952975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31500052a0 con 0x7f31780af970 2023-12-26T21:36:56.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.036+0000 7f315effd700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3164067000 0x7f31640694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.036+0000 7f315effd700 1 -- 172.21.15.131:0/2464952975 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3170084440 con 0x7f31780af970 2023-12-26T21:36:56.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.036+0000 7f317659c700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3164067000 0x7f31640694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.037+0000 7f317659c700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3164067000 0x7f31640694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f317806c040 tx=0x7f3168008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.039+0000 7f315effd700 1 -- 172.21.15.131:0/2464952975 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f317004c5a0 con 0x7f31780af970 2023-12-26T21:36:56.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.073+0000 7f6e9f59d700 1 Processor -- start 2023-12-26T21:36:56.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.073+0000 7f6e9f59d700 1 -- start start 2023-12-26T21:36:56.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.074+0000 7ffb087ff700 1 Processor -- start 2023-12-26T21:36:56.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.074+0000 7ffb087ff700 1 -- start start 2023-12-26T21:36:56.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.076+0000 7f6e9f59d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e9806b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7f6e9f59d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e9806bae0 0x7f6e9815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7f6e9f59d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6e9815d810 0x7f6e9806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7f6e9f59d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e980c9180 con 0x7f6e9815d810 2023-12-26T21:36:56.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7f6e9f59d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9815aa30 con 0x7f6e9806bae0 2023-12-26T21:36:56.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7f6e9f59d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9806bf50 con 0x7f6e98153a20 2023-12-26T21:36:56.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7ffb087ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffaf80073c0 0x7ffaf8033a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7ffb087ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf8005f20 con 0x7ffaf804f340 2023-12-26T21:36:56.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7ffb087ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf8053bb0 con 0x7ffaf8052210 2023-12-26T21:36:56.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7ffb087ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf80de520 con 0x7ffaf80073c0 2023-12-26T21:36:56.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7f6e9e59b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e9806b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7f6e9e59b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e9806b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57418/0 (socket says 172.21.15.131:57418) 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7f6e9e59b700 1 -- 172.21.15.131:0/1123489698 learned_addr learned my addr 172.21.15.131:0/1123489698 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.077+0000 7f6e9dd9a700 1 --2- 172.21.15.131:0/1123489698 >> v2:172.21.15.190:3300/0 conn(0x7f6e9806bae0 0x7f6e9815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7f6e9e59b700 1 -- 172.21.15.131:0/1123489698 >> v2:172.21.15.190:3300/0 conn(0x7f6e9806bae0 msgr2=0x7f6e9815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7f6e9e59b700 1 --2- 172.21.15.131:0/1123489698 >> v2:172.21.15.190:3300/0 conn(0x7f6e9806bae0 0x7f6e9815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7f6e9e59b700 1 -- 172.21.15.131:0/1123489698 >> v2:172.21.15.131:3300/0 conn(0x7f6e9815d810 msgr2=0x7f6e9806ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7f6e9e59b700 1 --2- 172.21.15.131:0/1123489698 >> v2:172.21.15.131:3300/0 conn(0x7f6e9815d810 0x7f6e9806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7f6e9e59b700 1 -- 172.21.15.131:0/1123489698 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9815ce50 con 0x7f6e98153a20 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7f6e9e59b700 1 --2- 172.21.15.131:0/1123489698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e9806b5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e9806f270 tx=0x7f6e88009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c50b02791775fc95 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7ffb05d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ffaf804f340 0x7ffaf80dde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49456/0 (socket says 172.21.15.131:49456) 2023-12-26T21:36:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7ffb05d9a700 1 -- 172.21.15.131:0/567553979 learned_addr learned my addr 172.21.15.131:0/567553979 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7ffb06d9c700 1 --2- 172.21.15.131:0/567553979 >> v2:172.21.15.131:3300/0 conn(0x7ffaf80073c0 0x7ffaf8033a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7f6e9d599700 1 -- 172.21.15.131:0/1123489698 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8800b860 con 0x7f6e98153a20 2023-12-26T21:36:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7f6e9d599700 1 -- 172.21.15.131:0/1123489698 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8800b9c0 con 0x7f6e98153a20 2023-12-26T21:36:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7f6e9d599700 1 -- 172.21.15.131:0/1123489698 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8800bd60 con 0x7f6e98153a20 2023-12-26T21:36:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7ffb05599700 1 -- 172.21.15.131:0/567553979 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649638003 0 0) 0x7ffaf8005f20 con 0x7ffaf804f340 2023-12-26T21:36:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.078+0000 7ffb05599700 1 -- 172.21.15.131:0/567553979 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaec003570 con 0x7ffaf804f340 2023-12-26T21:36:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.079+0000 7ffb06d9c700 1 -- 172.21.15.131:0/567553979 >> v1:172.21.15.190:6789/0 conn(0x7ffaf8052210 legacy=0x7ffaf8053360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-26T21:36:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.079+0000 7ffb06d9c700 1 -- 172.21.15.131:0/567553979 >> v1:172.21.15.131:6789/0 conn(0x7ffaf804f340 legacy=0x7ffaf80dde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.079+0000 7ffb06d9c700 1 -- 172.21.15.131:0/567553979 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf80e51d0 con 0x7ffaf80073c0 2023-12-26T21:36:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.079+0000 7f6e9f59d700 1 -- 172.21.15.131:0/1123489698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 msgr2=0x7f6e9806b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.079+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/1123489698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e9806b5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e9806f270 tx=0x7f6e88009250 comp rx=0 tx=0).stop 2023-12-26T21:36:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.080+0000 7ffb06d9c700 1 --2- 172.21.15.131:0/567553979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf80073c0 0x7ffaf8033a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffb0004d1b0 tx=0x7ffb00063120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b32cbf1db45da7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.080+0000 7f6e9f59d700 1 -- 172.21.15.131:0/1123489698 shutdown_connections 2023-12-26T21:36:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.080+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/1123489698 >> v2:172.21.15.131:3300/0 conn(0x7f6e9815d810 0x7f6e9806ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.081+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/1123489698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e9806b5a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.081+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/1123489698 >> v2:172.21.15.190:3300/0 conn(0x7f6e9806bae0 0x7f6e9815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.081+0000 7f6e9f59d700 1 -- 172.21.15.131:0/1123489698 >> 172.21.15.131:0/1123489698 conn(0x7f6e980ce220 msgr2=0x7f6e980ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.081+0000 7f6e9f59d700 1 -- 172.21.15.131:0/1123489698 shutdown_connections 2023-12-26T21:36:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.081+0000 7ffb05599700 1 -- 172.21.15.131:0/567553979 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00064770 con 0x7ffaf80073c0 2023-12-26T21:36:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.081+0000 7ffb05599700 1 -- 172.21.15.131:0/567553979 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb000648f0 con 0x7ffaf80073c0 2023-12-26T21:36:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.081+0000 7ffb05599700 1 -- 172.21.15.131:0/567553979 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00067800 con 0x7ffaf80073c0 2023-12-26T21:36:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.082+0000 7ffb087ff700 1 -- 172.21.15.131:0/567553979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf80073c0 msgr2=0x7ffaf8033a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.082+0000 7ffb087ff700 1 --2- 172.21.15.131:0/567553979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf80073c0 0x7ffaf8033a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffb0004d1b0 tx=0x7ffb00063120 comp rx=0 tx=0).stop 2023-12-26T21:36:56.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.081+0000 7f6e9f59d700 1 -- 172.21.15.131:0/1123489698 wait complete. 2023-12-26T21:36:56.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.083+0000 7f6e9f59d700 1 Processor -- start 2023-12-26T21:36:56.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.084+0000 7ffb087ff700 1 -- 172.21.15.131:0/567553979 shutdown_connections 2023-12-26T21:36:56.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.084+0000 7ffb087ff700 1 --2- 172.21.15.131:0/567553979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf80073c0 0x7ffaf8033a70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.084+0000 7f6e9f59d700 1 -- start start 2023-12-26T21:36:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.084+0000 7ffb087ff700 1 -- 172.21.15.131:0/567553979 >> 172.21.15.131:0/567553979 conn(0x7ffaf8051700 msgr2=0x7ffaf8051b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.084+0000 7ffb087ff700 1 -- 172.21.15.131:0/567553979 shutdown_connections 2023-12-26T21:36:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.084+0000 7ffb087ff700 1 -- 172.21.15.131:0/567553979 wait complete. 2023-12-26T21:36:56.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.086+0000 7f6e9f59d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e9806bae0 0x7f6e9816a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.086+0000 7f6e9f59d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e98152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.086+0000 7f6e9f59d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9815d810 0x7f6e980c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.087+0000 7f6e9dd9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e98152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.087+0000 7f6e9dd9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e98152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57430/0 (socket says 172.21.15.131:57430) 2023-12-26T21:36:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.087+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980c4650 con 0x7f6e9806bae0 2023-12-26T21:36:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.087+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980c4930 con 0x7f6e9815d810 2023-12-26T21:36:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.087+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e98169fc0 con 0x7f6e98153a20 2023-12-26T21:36:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.087+0000 7f6e9e59b700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e9806bae0 0x7f6e9816a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.087+0000 7f6e9ed9c700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9815d810 0x7f6e980c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.087+0000 7f6e9dd9a700 1 -- 172.21.15.131:0/2090091326 learned_addr learned my addr 172.21.15.131:0/2090091326 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.087+0000 7ffb087ff700 1 Processor -- start 2023-12-26T21:36:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.088+0000 7ffb087ff700 1 -- start start 2023-12-26T21:36:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.088+0000 7f6e9dd9a700 1 -- 172.21.15.131:0/2090091326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9815d810 msgr2=0x7f6e980c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.088+0000 7f6e9dd9a700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9815d810 0x7f6e980c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.088+0000 7f6e9dd9a700 1 -- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e9806bae0 msgr2=0x7f6e9816a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.088+0000 7f6e9dd9a700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e9806bae0 0x7f6e9816a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.088+0000 7f6e9dd9a700 1 -- 172.21.15.131:0/2090091326 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e880094f0 con 0x7f6e98153a20 2023-12-26T21:36:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.088+0000 7f6e9e59b700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e9806bae0 0x7f6e9816a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:56.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.088+0000 7f6e9ed9c700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9815d810 0x7f6e980c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:56.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7f6e9dd9a700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e98152670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400d860 tx=0x7f6e94008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7f6e8e7fc700 1 -- 172.21.15.131:0/2090091326 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e940125f0 con 0x7f6e98153a20 2023-12-26T21:36:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.091+0000 7f6e8e7fc700 1 -- 172.21.15.131:0/2090091326 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e94012750 con 0x7f6e98153a20 2023-12-26T21:36:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.091+0000 7f6e8e7fc700 1 -- 172.21.15.131:0/2090091326 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94012b10 con 0x7f6e98153a20 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7ffb087ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffaf80073c0 0x7ffaf804c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7ffb087ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf804f340 0x7ffaf8033a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7ffb087ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf8052210 0x7ffaf8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7ffb087ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf8103560 con 0x7ffaf8052210 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7ffb087ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf80e0480 con 0x7ffaf804f340 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7ffb087ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf8103280 con 0x7ffaf80073c0 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7ffb0659b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffaf80073c0 0x7ffaf804c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.091+0000 7ffb06d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf8052210 0x7ffaf8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.090+0000 7ffb0659b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffaf80073c0 0x7ffaf804c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57440/0 (socket says 172.21.15.131:57440) 2023-12-26T21:36:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.091+0000 7ffb0659b700 1 -- 172.21.15.131:0/1404792492 learned_addr learned my addr 172.21.15.131:0/1404792492 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.091+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9816a2a0 con 0x7f6e98153a20 2023-12-26T21:36:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.091+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e98169c70 con 0x7f6e98153a20 2023-12-26T21:36:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.092+0000 7ffb05d9a700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf804f340 0x7ffaf8033a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.092+0000 7ffb05d9a700 1 -- 172.21.15.131:0/1404792492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffaf80073c0 msgr2=0x7ffaf804c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.092+0000 7ffb05d9a700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffaf80073c0 0x7ffaf804c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.092+0000 7ffb05d9a700 1 -- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf8052210 msgr2=0x7ffaf8102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.092+0000 7ffb05d9a700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf8052210 0x7ffaf8102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.092+0000 7ffb05d9a700 1 -- 172.21.15.131:0/1404792492 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0004c0e0 con 0x7ffaf804f340 2023-12-26T21:36:56.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.092+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e980c4790 con 0x7f6e98153a20 2023-12-26T21:36:56.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.092+0000 7ffb06d9c700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf8052210 0x7ffaf8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:56.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.094+0000 7f6e8e7fc700 1 -- 172.21.15.131:0/2090091326 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6e9401e460 con 0x7f6e98153a20 2023-12-26T21:36:56.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.095+0000 7f6e8e7fc700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e7c066f30 0x7f6e7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.095+0000 7f6e8e7fc700 1 -- 172.21.15.131:0/2090091326 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e94084b00 con 0x7f6e98153a20 2023-12-26T21:36:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.097+0000 7ffb05d9a700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf804f340 0x7ffaf8033a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000c580 tx=0x7ffaf0007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.097+0000 7f6e9e59b700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e7c066f30 0x7f6e7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.097+0000 7ffaf67fc700 1 -- 172.21.15.131:0/1404792492 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0005c00 con 0x7ffaf804f340 2023-12-26T21:36:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.097+0000 7ffaf67fc700 1 -- 172.21.15.131:0/1404792492 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf0005d80 con 0x7ffaf804f340 2023-12-26T21:36:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.097+0000 7ffaf67fc700 1 -- 172.21.15.131:0/1404792492 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0007320 con 0x7ffaf804f340 2023-12-26T21:36:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.098+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf80e0720 con 0x7ffaf804f340 2023-12-26T21:36:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.098+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf804c660 con 0x7ffaf804f340 2023-12-26T21:36:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.099+0000 7f6e9e59b700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e7c066f30 0x7f6e7c0693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e9806f270 tx=0x7f6e88017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.100+0000 7f6e8e7fc700 1 -- 172.21.15.131:0/2090091326 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e980c4790 con 0x7f6e98153a20 2023-12-26T21:36:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.101+0000 7ffaf67fc700 1 -- 172.21.15.131:0/1404792492 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffaf001b3b0 con 0x7ffaf804f340 2023-12-26T21:36:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.102+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf80e02e0 con 0x7ffaf804f340 2023-12-26T21:36:56.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.103+0000 7ffaf67fc700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffae4067000 0x7ffae40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:56.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.103+0000 7ffaf67fc700 1 -- 172.21.15.131:0/1404792492 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffaf0083a70 con 0x7ffaf804f340 2023-12-26T21:36:56.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.104+0000 7ffb0659b700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffae4067000 0x7ffae40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:56.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.105+0000 7ffb0659b700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffae4067000 0x7ffae40694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0099f0 tx=0x7ffafc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:56.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.107+0000 7ffaf67fc700 1 -- 172.21.15.131:0/1404792492 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffaf80e02e0 con 0x7ffaf804f340 2023-12-26T21:36:56.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.408+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb32c005830 con 0x7fb34c0afc60 2023-12-26T21:36:56.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.409+0000 7fb333fff700 1 -- 172.21.15.131:0/326098211 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb340054760 con 0x7fb34c0afc60 2023-12-26T21:36:56.409 INFO:teuthology.orchestra.run.smithi131.stdout:64424509446 2023-12-26T21:36:56.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.413+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb338067000 msgr2=0x7fb3380694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.413+0000 7fb35119d700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb338067000 0x7fb3380694b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb34c06d0a0 tx=0x7fb33c016000 comp rx=0 tx=0).stop 2023-12-26T21:36:56.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.414+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0afc60 msgr2=0x7fb34c145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.414+0000 7fb35119d700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0afc60 0x7fb34c145380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb34000b600 tx=0x7fb3400091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.414+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 shutdown_connections 2023-12-26T21:36:56.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.414+0000 7fb35119d700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb34c0b06f0 0x7fb34c137470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.414+0000 7fb35119d700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb338067000 0x7fb3380694b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.414+0000 7fb35119d700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0afc60 0x7fb34c145380 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.414+0000 7fb35119d700 1 --2- 172.21.15.131:0/326098211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0b02a0 0x7fb34c136300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.415+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 >> 172.21.15.131:0/326098211 conn(0x7fb34c0675b0 msgr2=0x7fb34c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.415+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 shutdown_connections 2023-12-26T21:36:56.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.415+0000 7fb35119d700 1 -- 172.21.15.131:0/326098211 wait complete. 2023-12-26T21:36:56.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.426+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5bcc050820 con 0x7f5bcc0d51e0 2023-12-26T21:36:56.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.426+0000 7f5bca7fc700 1 -- 172.21.15.131:0/2601230441 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5bd001d070 con 0x7f5bcc0d51e0 2023-12-26T21:36:56.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.427+0000 7ff70d7fa700 1 -- 172.21.15.131:0/3829607145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff6fc005560 con 0x7ff71c0e6320 2023-12-26T21:36:56.427 INFO:teuthology.orchestra.run.smithi131.stdout:64424509446 2023-12-26T21:36:56.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.428+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3829607145 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff710056b30 con 0x7ff71c0e6320 2023-12-26T21:36:56.430 INFO:teuthology.orchestra.run.smithi131.stdout:64424509445 2023-12-26T21:36:56.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.431+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5bb8067000 msgr2=0x7f5bb80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7f5bdb39d700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5bb8067000 0x7f5bb80694b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4000f60 tx=0x7f5bc4007b40 comp rx=0 tx=0).stop 2023-12-26T21:36:56.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bcc0d51e0 msgr2=0x7f5bcc0d3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7f5bdb39d700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bcc0d51e0 0x7f5bcc0d3790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c9b0 tx=0x7f5bd000a250 comp rx=0 tx=0).stop 2023-12-26T21:36:56.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 shutdown_connections 2023-12-26T21:36:56.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7f5bdb39d700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bcc0d51e0 0x7f5bcc0d3790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7f5bdb39d700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5bb8067000 0x7f5bb80694b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7f5bdb39d700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bcc006060 0x7f5bcc039f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7ff70d7fa700 1 -- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff708066f30 msgr2=0x7ff7080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7ff70d7fa700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff708066f30 0x7ff7080693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff71800e1d0 tx=0x7ff718000f90 comp rx=0 tx=0).stop 2023-12-26T21:36:56.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7ff70d7fa700 1 -- 172.21.15.131:0/3829607145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0e6320 msgr2=0x7ff71c0e11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7ff70d7fa700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0e6320 0x7ff71c0e11d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0354d0 tx=0x7ff710016000 comp rx=0 tx=0).stop 2023-12-26T21:36:56.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7f5bdb39d700 1 --2- 172.21.15.131:0/2601230441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc033eb0 0x7f5bcc04e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.432+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 >> 172.21.15.131:0/2601230441 conn(0x7f5bcc051840 msgr2=0x7f5bcc036860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.433+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 shutdown_connections 2023-12-26T21:36:56.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.433+0000 7f5bdb39d700 1 -- 172.21.15.131:0/2601230441 wait complete. 2023-12-26T21:36:56.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.433+0000 7ff70d7fa700 1 -- 172.21.15.131:0/3829607145 shutdown_connections 2023-12-26T21:36:56.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.433+0000 7ff70d7fa700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff708066f30 0x7ff7080693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.433+0000 7ff70d7fa700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0ea270 0x7ff71c0058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.433+0000 7ff70d7fa700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c034b20 0x7ff71c0e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.433+0000 7ff70d7fa700 1 --2- 172.21.15.131:0/3829607145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0e6320 0x7ff71c0e11d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.433+0000 7ff70d7fa700 1 -- 172.21.15.131:0/3829607145 >> 172.21.15.131:0/3829607145 conn(0x7ff71c054dd0 msgr2=0x7ff71c0e40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.434+0000 7ff70d7fa700 1 -- 172.21.15.131:0/3829607145 shutdown_connections 2023-12-26T21:36:56.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.434+0000 7ff70d7fa700 1 -- 172.21.15.131:0/3829607145 wait complete. 2023-12-26T21:36:56.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2023-12-26T21:36:56.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.458+0000 7f315cff9700 1 -- 172.21.15.131:0/2464952975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3150005820 con 0x7f31780af970 2023-12-26T21:36:56.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.459+0000 7f315effd700 1 -- 172.21.15.131:0/2464952975 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3170051710 con 0x7f31780af970 2023-12-26T21:36:56.459 INFO:teuthology.orchestra.run.smithi131.stdout:60129542151 2023-12-26T21:36:56.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.462+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f90c40b2100 con 0x7f90c40a82f0 2023-12-26T21:36:56.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.462+0000 7f90ab7fe700 1 -- 172.21.15.131:0/2791441470 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f90c40b2100 con 0x7f90c40a82f0 2023-12-26T21:36:56.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 -- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3164067000 msgr2=0x7f31640694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3164067000 0x7f31640694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f317806c040 tx=0x7f3168008040 comp rx=0 tx=0).stop 2023-12-26T21:36:56.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 -- 172.21.15.131:0/2464952975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31780af970 msgr2=0x7f317813ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31780af970 0x7f317813ec40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f317000bad0 tx=0x7f3170009780 comp rx=0 tx=0).stop 2023-12-26T21:36:56.463 INFO:teuthology.orchestra.run.smithi131.stdout:64424509446 2023-12-26T21:36:56.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 -- 172.21.15.131:0/2464952975 shutdown_connections 2023-12-26T21:36:56.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31781588f0 0x7f3178153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3164067000 0x7f31640694b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31780af970 0x7f317813ec40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 --2- 172.21.15.131:0/2464952975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3178155630 0x7f31781534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.463+0000 7f315cff9700 1 -- 172.21.15.131:0/2464952975 >> 172.21.15.131:0/2464952975 conn(0x7f317806a060 msgr2=0x7f3178154660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.464+0000 7f315cff9700 1 -- 172.21.15.131:0/2464952975 shutdown_connections 2023-12-26T21:36:56.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.464+0000 7f315cff9700 1 -- 172.21.15.131:0/2464952975 wait complete. 2023-12-26T21:36:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90b0066fb0 msgr2=0x7f90b0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90b0066fb0 0x7f90b0069460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f90b8007bc0 tx=0x7f90b8028000 comp rx=0 tx=0).stop 2023-12-26T21:36:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40a82f0 msgr2=0x7f90c40ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40a82f0 0x7f90c40ac650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90bc000f80 tx=0x7f90bc0052a0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 shutdown_connections 2023-12-26T21:36:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40a82f0 0x7f90c40ac650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90b0066fb0 0x7f90b0069460 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90ac003ab0 0x7f90c4142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 --2- 172.21.15.131:0/2791441470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40a7e80 0x7f90c40ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.466+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 >> 172.21.15.131:0/2791441470 conn(0x7f90c406a060 msgr2=0x7f90c406ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.467+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 shutdown_connections 2023-12-26T21:36:56.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.467+0000 7f90c9fff700 1 -- 172.21.15.131:0/2791441470 wait complete. 2023-12-26T21:36:56.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2023-12-26T21:36:56.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2023-12-26T21:36:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.483+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4814007740 con 0x7f48140ea1a0 2023-12-26T21:36:56.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.484+0000 7f4818ff9700 1 -- 172.21.15.131:0/2008472112 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f481c0a0090 con 0x7f48140ea1a0 2023-12-26T21:36:56.483 INFO:teuthology.orchestra.run.smithi131.stdout:64424509447 2023-12-26T21:36:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.486+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4800066ff0 msgr2=0x7f48000694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.486+0000 7f4822dff700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4800066ff0 0x7f48000694a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4810009f60 tx=0x7f48100091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.486+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48140ea1a0 msgr2=0x7f48140359d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.486+0000 7f4822dff700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48140ea1a0 0x7f48140359d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f481c04cc40 tx=0x7f481c08b000 comp rx=0 tx=0).stop 2023-12-26T21:36:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.487+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 shutdown_connections 2023-12-26T21:36:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.487+0000 7f4822dff700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4800066ff0 0x7f48000694a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.487+0000 7f4822dff700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48140ea1a0 0x7f48140359d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.487+0000 7f4822dff700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4814006930 0x7f481404e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.487+0000 7f4822dff700 1 --2- 172.21.15.131:0/2008472112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48140e3e70 0x7f481403a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.487+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 >> 172.21.15.131:0/2008472112 conn(0x7f4814054c90 msgr2=0x7f48140e0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.487+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 shutdown_connections 2023-12-26T21:36:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.487+0000 7f4822dff700 1 -- 172.21.15.131:0/2008472112 wait complete. 2023-12-26T21:36:56.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.2 2023-12-26T21:36:56.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2023-12-26T21:36:56.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.3 2023-12-26T21:36:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.508+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ffaf80e02e0 con 0x7ffaf804f340 2023-12-26T21:36:56.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.509+0000 7ffaf67fc700 1 -- 172.21.15.131:0/1404792492 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffaf80e02e0 con 0x7ffaf804f340 2023-12-26T21:36:56.509 INFO:teuthology.orchestra.run.smithi131.stdout:60129542150 2023-12-26T21:36:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.512+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffae4067000 msgr2=0x7ffae40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.512+0000 7ffb087ff700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffae4067000 0x7ffae40694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0099f0 tx=0x7ffafc008040 comp rx=0 tx=0).stop 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf804f340 msgr2=0x7ffaf8033a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf804f340 0x7ffaf8033a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000c580 tx=0x7ffaf0007d50 comp rx=0 tx=0).stop 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 shutdown_connections 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaf8052210 0x7ffaf8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffae4067000 0x7ffae40694b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffaf80073c0 0x7ffaf804c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 --2- 172.21.15.131:0/1404792492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffaf804f340 0x7ffaf8033a10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 >> 172.21.15.131:0/1404792492 conn(0x7ffaf8051700 msgr2=0x7ffaf804b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 shutdown_connections 2023-12-26T21:36:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.513+0000 7ffb087ff700 1 -- 172.21.15.131:0/1404792492 wait complete. 2023-12-26T21:36:56.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.514+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6e9816a100 con 0x7f6e98153a20 2023-12-26T21:36:56.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.515+0000 7f6e8e7fc700 1 -- 172.21.15.131:0/2090091326 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e9816a100 con 0x7f6e98153a20 2023-12-26T21:36:56.514 INFO:teuthology.orchestra.run.smithi131.stdout:60129542150 2023-12-26T21:36:56.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.517+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e7c066f30 msgr2=0x7f6e7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.517+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e7c066f30 0x7f6e7c0693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e9806f270 tx=0x7f6e88017000 comp rx=0 tx=0).stop 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.517+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 msgr2=0x7f6e98152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.517+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e98152670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400d860 tx=0x7f6e94008040 comp rx=0 tx=0).stop 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.518+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 shutdown_connections 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.518+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9815d810 0x7f6e980c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.518+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e9806bae0 0x7f6e9816a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.518+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6e7c066f30 0x7f6e7c0693e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.518+0000 7f6e9f59d700 1 --2- 172.21.15.131:0/2090091326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e98153a20 0x7f6e98152670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.518+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 >> 172.21.15.131:0/2090091326 conn(0x7f6e980ce220 msgr2=0x7f6e980cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.518+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 shutdown_connections 2023-12-26T21:36:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:56.518+0000 7f6e9f59d700 1 -- 172.21.15.131:0/2090091326 wait complete. 2023-12-26T21:36:56.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.7 2023-12-26T21:36:56.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.4 2023-12-26T21:36:57.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-26T21:36:57.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-26T21:36:57.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-26T21:36:57.489 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-26T21:36:57.490 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-26T21:36:57.507 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-26T21:36:57.533 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-26T21:36:57.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-26T21:36:57.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.751+0000 7fda5519d700 1 Processor -- start 2023-12-26T21:36:57.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.751+0000 7fda5519d700 1 -- start start 2023-12-26T21:36:57.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.753+0000 7fda5519d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fda500ae8b0 0x7fda501604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.754+0000 7fda4f7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fda501557e0 0x7fda500ab8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49464/0 (socket says 172.21.15.131:49464) 2023-12-26T21:36:57.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.754+0000 7fda4f7fe700 1 -- 172.21.15.131:0/2158489532 learned_addr learned my addr 172.21.15.131:0/2158489532 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:57.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.754+0000 7fda5499c700 1 --2- 172.21.15.131:0/2158489532 >> v2:172.21.15.190:3300/0 conn(0x7fda500ae8b0 0x7fda501604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.754+0000 7fda5519d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda500ad410 con 0x7fda501557e0 2023-12-26T21:36:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.755+0000 7fda5519d700 1 -- 172.21.15.131:0/2158489532 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda500ac560 con 0x7fda50155410 2023-12-26T21:36:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.755+0000 7fda5519d700 1 -- 172.21.15.131:0/2158489532 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda500ac840 con 0x7fda500ae8b0 2023-12-26T21:36:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.755+0000 7fda4effd700 1 -- 172.21.15.131:0/2158489532 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747799953 0 0) 0x7fda500ad410 con 0x7fda501557e0 2023-12-26T21:36:57.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.755+0000 7fda4effd700 1 -- 172.21.15.131:0/2158489532 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda38003570 con 0x7fda501557e0 2023-12-26T21:36:57.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.756+0000 7fda5499c700 1 -- 172.21.15.131:0/2158489532 >> v1:172.21.15.190:6789/0 conn(0x7fda50155410 legacy=0x7fda5006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.756+0000 7fda5499c700 1 -- 172.21.15.131:0/2158489532 >> v1:172.21.15.131:6789/0 conn(0x7fda501557e0 legacy=0x7fda500ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.756+0000 7fda5499c700 1 -- 172.21.15.131:0/2158489532 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda500acb20 con 0x7fda500ae8b0 2023-12-26T21:36:57.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.758+0000 7fda5499c700 1 --2- 172.21.15.131:0/2158489532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda500ae8b0 0x7fda501604b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fda4000b1e0 tx=0x7fda40009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=934194391fd06611 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.758+0000 7fda4effd700 1 -- 172.21.15.131:0/2158489532 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4000e070 con 0x7fda500ae8b0 2023-12-26T21:36:57.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.758+0000 7fda4effd700 1 -- 172.21.15.131:0/2158489532 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4000bb40 con 0x7fda500ae8b0 2023-12-26T21:36:57.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.759+0000 7fda5519d700 1 -- 172.21.15.131:0/2158489532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda500ae8b0 msgr2=0x7fda501604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.759+0000 7fda5519d700 1 --2- 172.21.15.131:0/2158489532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda500ae8b0 0x7fda501604b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fda4000b1e0 tx=0x7fda40009ab0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.760+0000 7fda5519d700 1 -- 172.21.15.131:0/2158489532 shutdown_connections 2023-12-26T21:36:57.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.760+0000 7fda5519d700 1 --2- 172.21.15.131:0/2158489532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda500ae8b0 0x7fda501604b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.760+0000 7fda5519d700 1 -- 172.21.15.131:0/2158489532 >> 172.21.15.131:0/2158489532 conn(0x7fda500cc040 msgr2=0x7fda500cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.761+0000 7fda5519d700 1 -- 172.21.15.131:0/2158489532 shutdown_connections 2023-12-26T21:36:57.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.761+0000 7fda5519d700 1 -- 172.21.15.131:0/2158489532 wait complete. 2023-12-26T21:36:57.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.763+0000 7fda5519d700 1 Processor -- start 2023-12-26T21:36:57.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.764+0000 7fda5519d700 1 -- start start 2023-12-26T21:36:57.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.766+0000 7fda5519d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda50155410 0x7fda50156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.766+0000 7fda5519d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda501557e0 0x7fda50158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.766+0000 7fda5519d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5014a690 0x7fda501760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.767+0000 7fda5519d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda50158a70 con 0x7fda501557e0 2023-12-26T21:36:57.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.767+0000 7fda5519d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda50157330 con 0x7fda5014a690 2023-12-26T21:36:57.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.767+0000 7fda5519d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda50157610 con 0x7fda50155410 2023-12-26T21:36:57.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.767+0000 7fda5499c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5014a690 0x7fda501760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.767+0000 7fda5499c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5014a690 0x7fda501760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53548/0 (socket says 172.21.15.131:53548) 2023-12-26T21:36:57.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.767+0000 7fda5499c700 1 -- 172.21.15.131:0/2178945402 learned_addr learned my addr 172.21.15.131:0/2178945402 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.767+0000 7fda4ffff700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda50155410 0x7fda50156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.767+0000 7fda5499c700 1 -- 172.21.15.131:0/2178945402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda50155410 msgr2=0x7fda50156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.768+0000 7fda5499c700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda50155410 0x7fda50156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.768+0000 7fda5499c700 1 -- 172.21.15.131:0/2178945402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda501557e0 msgr2=0x7fda50158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.768+0000 7fda5499c700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda501557e0 0x7fda50158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.768+0000 7fda5499c700 1 -- 172.21.15.131:0/2178945402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda40009d50 con 0x7fda5014a690 2023-12-26T21:36:57.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.768+0000 7fda5499c700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5014a690 0x7fda501760b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda40009af0 tx=0x7fda40016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.768+0000 7fda4ffff700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda50155410 0x7fda50156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:57.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.768+0000 7fda37fff700 1 -- 172.21.15.131:0/2178945402 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44001470 con 0x7fda5014a690 2023-12-26T21:36:57.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.768+0000 7fda37fff700 1 -- 172.21.15.131:0/2178945402 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda40005ce0 con 0x7fda5014a690 2023-12-26T21:36:57.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.768+0000 7fda37fff700 1 -- 172.21.15.131:0/2178945402 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40012980 con 0x7fda5014a690 2023-12-26T21:36:57.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.769+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda501578f0 con 0x7fda5014a690 2023-12-26T21:36:57.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.769+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda50157eb0 con 0x7fda5014a690 2023-12-26T21:36:57.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.772+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda300052b0 con 0x7fda5014a690 2023-12-26T21:36:57.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.773+0000 7fda37fff700 1 -- 172.21.15.131:0/2178945402 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fda40012d30 con 0x7fda5014a690 2023-12-26T21:36:57.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.774+0000 7fda37fff700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fda3c066fb0 0x7fda3c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.774+0000 7fda37fff700 1 -- 172.21.15.131:0/2178945402 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fda40084ad0 con 0x7fda5014a690 2023-12-26T21:36:57.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.775+0000 7fda4ffff700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fda3c066fb0 0x7fda3c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.777+0000 7fda4ffff700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fda3c066fb0 0x7fda3c069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda48007fd0 tx=0x7fda48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.780+0000 7fda37fff700 1 -- 172.21.15.131:0/2178945402 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda40022080 con 0x7fda5014a690 2023-12-26T21:36:57.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.805+0000 7fba8ed9d700 1 Processor -- start 2023-12-26T21:36:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.805+0000 7fba8ed9d700 1 -- start start 2023-12-26T21:36:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.807+0000 7fba8ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806af40 0x7fba880afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.807+0000 7fba8ed9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fba880c8a40 0x7fba88142650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.807+0000 7fba8ed9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba880b0170 con 0x7fba8806ab70 2023-12-26T21:36:57.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.807+0000 7fba8ed9d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba880b0450 con 0x7fba880c8a40 2023-12-26T21:36:57.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.807+0000 7fba8ed9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba880b0730 con 0x7fba8806af40 2023-12-26T21:36:57.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.808+0000 7fba8d59a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806af40 0x7fba880afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.808+0000 7fba8d59a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806af40 0x7fba880afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57450/0 (socket says 172.21.15.131:57450) 2023-12-26T21:36:57.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.808+0000 7fba8d59a700 1 -- 172.21.15.131:0/996480672 learned_addr learned my addr 172.21.15.131:0/996480672 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.808+0000 7fba8e59c700 1 --2- 172.21.15.131:0/996480672 >> v2:172.21.15.131:3300/0 conn(0x7fba880c8a40 0x7fba88142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.808+0000 7fba8d59a700 1 -- 172.21.15.131:0/996480672 >> v2:172.21.15.131:3300/0 conn(0x7fba880c8a40 msgr2=0x7fba88142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.808+0000 7fba8d59a700 1 --2- 172.21.15.131:0/996480672 >> v2:172.21.15.131:3300/0 conn(0x7fba880c8a40 0x7fba88142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.808+0000 7fba8d59a700 1 -- 172.21.15.131:0/996480672 >> v1:172.21.15.190:6789/0 conn(0x7fba8806ab70 legacy=0x7fba880ad260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.808+0000 7fba8d59a700 1 -- 172.21.15.131:0/996480672 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880b0a10 con 0x7fba8806af40 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.809+0000 7fba8e59c700 1 --2- 172.21.15.131:0/996480672 >> v2:172.21.15.131:3300/0 conn(0x7fba880c8a40 0x7fba88142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.810+0000 7fba8d59a700 1 --2- 172.21.15.131:0/996480672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806af40 0x7fba880afb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba780099b0 tx=0x7fba78009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=354167a55cf26044 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.810+0000 7fba8cd99700 1 -- 172.21.15.131:0/996480672 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7800b8b0 con 0x7fba8806af40 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.811+0000 7fba8cd99700 1 -- 172.21.15.131:0/996480672 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7800ba10 con 0x7fba8806af40 2023-12-26T21:36:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.811+0000 7fba8cd99700 1 -- 172.21.15.131:0/996480672 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7800bdb0 con 0x7fba8806af40 2023-12-26T21:36:57.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.812+0000 7fba8ed9d700 1 -- 172.21.15.131:0/996480672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806af40 msgr2=0x7fba880afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.812+0000 7fba8ed9d700 1 --2- 172.21.15.131:0/996480672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806af40 0x7fba880afb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba780099b0 tx=0x7fba78009250 comp rx=0 tx=0).stop 2023-12-26T21:36:57.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.812+0000 7fba8ed9d700 1 -- 172.21.15.131:0/996480672 shutdown_connections 2023-12-26T21:36:57.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.812+0000 7fba8ed9d700 1 --2- 172.21.15.131:0/996480672 >> v2:172.21.15.131:3300/0 conn(0x7fba880c8a40 0x7fba88142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.813+0000 7fba8ed9d700 1 --2- 172.21.15.131:0/996480672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806af40 0x7fba880afb60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.813+0000 7fba8ed9d700 1 -- 172.21.15.131:0/996480672 >> 172.21.15.131:0/996480672 conn(0x7fba880cc040 msgr2=0x7fba880cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:57.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.814+0000 7fba8ed9d700 1 -- 172.21.15.131:0/996480672 shutdown_connections 2023-12-26T21:36:57.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.814+0000 7fba8ed9d700 1 -- 172.21.15.131:0/996480672 wait complete. 2023-12-26T21:36:57.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.817+0000 7fba8ed9d700 1 Processor -- start 2023-12-26T21:36:57.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.818+0000 7fba8ed9d700 1 -- start start 2023-12-26T21:36:57.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.818+0000 7fb48f59e700 1 Processor -- start 2023-12-26T21:36:57.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.819+0000 7fb48f59e700 1 -- start start 2023-12-26T21:36:57.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.820+0000 7fba8ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806ab70 0x7fba8815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.820+0000 7fba8ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba8806af40 0x7fba88158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.820+0000 7fba8ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba880c8a40 0x7fba8813ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.821+0000 7fba8ed9d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba881588b0 con 0x7fba8806af40 2023-12-26T21:36:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.821+0000 7fba8ed9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba88153460 con 0x7fba880c8a40 2023-12-26T21:36:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.821+0000 7fba8dd9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806ab70 0x7fba8815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.821+0000 7fba8dd9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806ab70 0x7fba8815d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57462/0 (socket says 172.21.15.131:57462) 2023-12-26T21:36:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.821+0000 7fba8dd9b700 1 -- 172.21.15.131:0/1394165104 learned_addr learned my addr 172.21.15.131:0/1394165104 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.821+0000 7fba8e59c700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba880c8a40 0x7fba8813ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.821+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba88159340 con 0x7fba8806ab70 2023-12-26T21:36:57.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fba8e59c700 1 -- 172.21.15.131:0/1394165104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806ab70 msgr2=0x7fba8815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fba8e59c700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806ab70 0x7fba8815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fb48f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880dc620 0x7fb4880dc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fb48f59e700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb4880e28b0 0x7fb488006d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fb48f59e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb488053e20 con 0x7fb48800d940 2023-12-26T21:36:57.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fb48f59e700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4880e3170 con 0x7fb4880e28b0 2023-12-26T21:36:57.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fb48f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4880e3450 con 0x7fb4880dc620 2023-12-26T21:36:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fb48dd9b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fb48800d940 0x7fb488051b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39828/0 (socket says 172.21.15.131:39828) 2023-12-26T21:36:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fba8e59c700 1 -- 172.21.15.131:0/1394165104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba8806af40 msgr2=0x7fba88158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fba8e59c700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba8806af40 0x7fba88158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fba8e59c700 1 -- 172.21.15.131:0/1394165104 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780094f0 con 0x7fba880c8a40 2023-12-26T21:36:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fba8dd9b700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806ab70 0x7fba8815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.822+0000 7fb48dd9b700 1 -- 172.21.15.131:0/2776766233 learned_addr learned my addr 172.21.15.131:0/2776766233 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.823+0000 7fb48e59c700 1 --2- 172.21.15.131:0/2776766233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880dc620 0x7fb4880dc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.823+0000 7fb48ed9d700 1 --2- 172.21.15.131:0/2776766233 >> v2:172.21.15.131:3300/0 conn(0x7fb4880e28b0 0x7fb488006d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.823+0000 7fb48d59a700 1 -- 172.21.15.131:0/2776766233 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993375769 0 0) 0x7fb488053e20 con 0x7fb48800d940 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.823+0000 7fb48d59a700 1 -- 172.21.15.131:0/2776766233 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb478003570 con 0x7fb48800d940 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.823+0000 7fb48e59c700 1 -- 172.21.15.131:0/2776766233 >> v2:172.21.15.131:3300/0 conn(0x7fb4880e28b0 msgr2=0x7fb488006d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.823+0000 7fb48e59c700 1 --2- 172.21.15.131:0/2776766233 >> v2:172.21.15.131:3300/0 conn(0x7fb4880e28b0 0x7fb488006d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.823+0000 7fb48e59c700 1 -- 172.21.15.131:0/2776766233 >> v1:172.21.15.190:6789/0 conn(0x7fb48800d940 legacy=0x7fb488051b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.823+0000 7fb48e59c700 1 -- 172.21.15.131:0/2776766233 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4880e3730 con 0x7fb4880dc620 2023-12-26T21:36:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.824+0000 7fba8e59c700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba880c8a40 0x7fba8813ab50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba8400d890 tx=0x7fba84008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.824+0000 7fb48e59c700 1 --2- 172.21.15.131:0/2776766233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880dc620 0x7fb4880dc9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb4840098e0 tx=0x7fb4840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af87b3a6ae2ae2de server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.824+0000 7fb48d59a700 1 -- 172.21.15.131:0/2776766233 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400b640 con 0x7fb4880dc620 2023-12-26T21:36:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.824+0000 7fb48d59a700 1 -- 172.21.15.131:0/2776766233 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48400b7a0 con 0x7fb4880dc620 2023-12-26T21:36:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.824+0000 7fb48d59a700 1 -- 172.21.15.131:0/2776766233 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400bb40 con 0x7fb4880dc620 2023-12-26T21:36:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.825+0000 7fb48f59e700 1 -- 172.21.15.131:0/2776766233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880dc620 msgr2=0x7fb4880dc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.825+0000 7fb48f59e700 1 --2- 172.21.15.131:0/2776766233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880dc620 0x7fb4880dc9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb4840098e0 tx=0x7fb4840091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.824+0000 7fba7dffb700 1 -- 172.21.15.131:0/1394165104 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba840125f0 con 0x7fba880c8a40 2023-12-26T21:36:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.824+0000 7fba7dffb700 1 -- 172.21.15.131:0/1394165104 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba84012750 con 0x7fba880c8a40 2023-12-26T21:36:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.824+0000 7fba7dffb700 1 -- 172.21.15.131:0/1394165104 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84012b10 con 0x7fba880c8a40 2023-12-26T21:36:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.824+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba88153720 con 0x7fba880c8a40 2023-12-26T21:36:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.825+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba88158d80 con 0x7fba880c8a40 2023-12-26T21:36:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.827+0000 7fb48f59e700 1 -- 172.21.15.131:0/2776766233 shutdown_connections 2023-12-26T21:36:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.827+0000 7fb48f59e700 1 --2- 172.21.15.131:0/2776766233 >> v2:172.21.15.131:3300/0 conn(0x7fb4880e28b0 0x7fb488006d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.827+0000 7fb48f59e700 1 --2- 172.21.15.131:0/2776766233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880dc620 0x7fb4880dc9f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.827+0000 7fb48f59e700 1 -- 172.21.15.131:0/2776766233 >> 172.21.15.131:0/2776766233 conn(0x7fb48800b320 msgr2=0x7fb48800b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.827+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba881538b0 con 0x7fba880c8a40 2023-12-26T21:36:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.828+0000 7fba7dffb700 1 -- 172.21.15.131:0/1394165104 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fba84013480 con 0x7fba880c8a40 2023-12-26T21:36:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.827+0000 7fb48f59e700 1 -- 172.21.15.131:0/2776766233 shutdown_connections 2023-12-26T21:36:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.827+0000 7fb48f59e700 1 -- 172.21.15.131:0/2776766233 wait complete. 2023-12-26T21:36:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.829+0000 7fb48f59e700 1 Processor -- start 2023-12-26T21:36:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.830+0000 7fba7dffb700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba6c067080 0x7fba6c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.830+0000 7fba7dffb700 1 -- 172.21.15.131:0/1394165104 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba840846e0 con 0x7fba880c8a40 2023-12-26T21:36:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.831+0000 7fba8dd9b700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba6c067080 0x7fba6c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.833+0000 7fba8dd9b700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba6c067080 0x7fba6c069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba80007ad0 tx=0x7fba80009820 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.833+0000 7fb48f59e700 1 -- start start 2023-12-26T21:36:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.834+0000 7fba7dffb700 1 -- 172.21.15.131:0/1394165104 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba84055330 con 0x7fba880c8a40 2023-12-26T21:36:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48f59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb48800d940 0x7fb48804ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48f59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4880dc620 0x7fb4880cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880e28b0 0x7fb4880e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48f59e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48804ee60 con 0x7fb48800d940 2023-12-26T21:36:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48f59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48804f140 con 0x7fb4880dc620 2023-12-26T21:36:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48804f420 con 0x7fb4880e28b0 2023-12-26T21:36:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb48800d940 0x7fb48804ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48dd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4880dc620 0x7fb4880cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb48800d940 0x7fb48804ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45002/0 (socket says 172.21.15.131:45002) 2023-12-26T21:36:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48e59c700 1 -- 172.21.15.131:0/3891613765 learned_addr learned my addr 172.21.15.131:0/3891613765 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.835+0000 7fb48ed9d700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880e28b0 0x7fb4880e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.836+0000 7fb48e59c700 1 -- 172.21.15.131:0/3891613765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880e28b0 msgr2=0x7fb4880e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.836+0000 7fb48e59c700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880e28b0 0x7fb4880e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.836+0000 7fb48e59c700 1 -- 172.21.15.131:0/3891613765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4880dc620 msgr2=0x7fb4880cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.836+0000 7fb48e59c700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4880dc620 0x7fb4880cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.836+0000 7fb48e59c700 1 -- 172.21.15.131:0/3891613765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484009420 con 0x7fb48800d940 2023-12-26T21:36:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.836+0000 7fb48e59c700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb48800d940 0x7fb48804ff00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb4880e6fe0 tx=0x7fb484017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.836+0000 7fb47e7fc700 1 -- 172.21.15.131:0/3891613765 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48401bc40 con 0x7fb48800d940 2023-12-26T21:36:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.837+0000 7fb47e7fc700 1 -- 172.21.15.131:0/3891613765 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48401bda0 con 0x7fb48800d940 2023-12-26T21:36:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.837+0000 7fb47e7fc700 1 -- 172.21.15.131:0/3891613765 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48401bc40 con 0x7fb48800d940 2023-12-26T21:36:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.837+0000 7f87e8fff700 1 Processor -- start 2023-12-26T21:36:57.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.837+0000 7f87e8fff700 1 -- start start 2023-12-26T21:36:57.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.837+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48804f700 con 0x7fb48800d940 2023-12-26T21:36:57.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.838+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4880ce390 con 0x7fb48800d940 2023-12-26T21:36:57.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.839+0000 7f87e8fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e41558f0 0x7f87e40afb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.840+0000 7f87e8fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e4158fd0 con 0x7f87e406b5d0 2023-12-26T21:36:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.840+0000 7f87e8fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e4157b30 con 0x7f87e40b1b40 2023-12-26T21:36:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.840+0000 7f87e8fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40b1fb0 con 0x7f87e41558f0 2023-12-26T21:36:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.840+0000 7f87e1d9b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f87e40b1b40 0x7f87e41551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39832/0 (socket says 172.21.15.131:39832) 2023-12-26T21:36:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.840+0000 7f87e1d9b700 1 -- 172.21.15.131:0/4168502992 learned_addr learned my addr 172.21.15.131:0/4168502992 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.840+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/4168502992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e41558f0 0x7f87e40afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.841+0000 7fb47e7fc700 1 -- 172.21.15.131:0/3891613765 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb484011630 con 0x7fb48800d940 2023-12-26T21:36:57.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.842+0000 7fb47e7fc700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb470066f90 0x7fb470069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.843+0000 7fb48dd9b700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb470066f90 0x7fb470069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.841+0000 7f87e2d9d700 1 -- 172.21.15.131:0/4168502992 >> v1:172.21.15.190:6789/0 conn(0x7f87e40b1b40 legacy=0x7f87e41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.841+0000 7f87e2d9d700 1 -- 172.21.15.131:0/4168502992 >> v1:172.21.15.131:6789/0 conn(0x7f87e406b5d0 legacy=0x7f87e41566a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-26T21:36:57.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.841+0000 7f87e2d9d700 1 -- 172.21.15.131:0/4168502992 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e4157200 con 0x7f87e41558f0 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.841+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/4168502992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e41558f0 0x7f87e40afb00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009910 tx=0x7f87d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b88e24a017cacc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.842+0000 7f87e159a700 1 -- 172.21.15.131:0/4168502992 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800b8b0 con 0x7f87e41558f0 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.842+0000 7f87e159a700 1 -- 172.21.15.131:0/4168502992 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d800ba10 con 0x7f87e41558f0 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.842+0000 7f87e159a700 1 -- 172.21.15.131:0/4168502992 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800bdb0 con 0x7f87e41558f0 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.843+0000 7f87e8fff700 1 -- 172.21.15.131:0/4168502992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e41558f0 msgr2=0x7f87e40afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.843+0000 7f87e8fff700 1 --2- 172.21.15.131:0/4168502992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e41558f0 0x7f87e40afb00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009910 tx=0x7f87d80091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.843+0000 7f87e8fff700 1 -- 172.21.15.131:0/4168502992 shutdown_connections 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.843+0000 7f87e8fff700 1 --2- 172.21.15.131:0/4168502992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e41558f0 0x7f87e40afb00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.843+0000 7f87e8fff700 1 -- 172.21.15.131:0/4168502992 >> 172.21.15.131:0/4168502992 conn(0x7f87e4068040 msgr2=0x7f87e4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.843+0000 7f87e8fff700 1 -- 172.21.15.131:0/4168502992 shutdown_connections 2023-12-26T21:36:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.843+0000 7f87e8fff700 1 -- 172.21.15.131:0/4168502992 wait complete. 2023-12-26T21:36:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.844+0000 7fb48dd9b700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb470066f90 0x7fb470069440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb4880e1b60 tx=0x7fb480007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.844+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4880ce580 con 0x7fb48800d940 2023-12-26T21:36:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.844+0000 7fb47e7fc700 1 -- 172.21.15.131:0/3891613765 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb484056cb0 con 0x7fb48800d940 2023-12-26T21:36:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.846+0000 7f87e8fff700 1 Processor -- start 2023-12-26T21:36:57.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.849+0000 7f87e8fff700 1 -- start start 2023-12-26T21:36:57.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.851+0000 7f87e8fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406b5d0 0x7f87e4176680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.851+0000 7f87e8fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b1b40 0x7f87e413a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.852+0000 7f87e259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406b5d0 0x7f87e4176680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.852+0000 7f87e259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406b5d0 0x7f87e4176680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57496/0 (socket says 172.21.15.131:57496) 2023-12-26T21:36:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.852+0000 7f87e1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b1b40 0x7f87e413a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.853+0000 7f87e259c700 1 -- 172.21.15.131:0/1896317700 learned_addr learned my addr 172.21.15.131:0/1896317700 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.853+0000 7f87e8fff700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e41558f0 0x7f87e414ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.853+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e414fb50 con 0x7f87e40b1b40 2023-12-26T21:36:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.853+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e414ef40 con 0x7f87e41558f0 2023-12-26T21:36:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.853+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e414f220 con 0x7f87e406b5d0 2023-12-26T21:36:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.853+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e41558f0 0x7f87e414ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.854+0000 7fb47e7fc700 1 -- 172.21.15.131:0/3891613765 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb484053080 con 0x7fb48800d940 2023-12-26T21:36:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.854+0000 7f87e2d9d700 1 -- 172.21.15.131:0/1896317700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406b5d0 msgr2=0x7f87e4176680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.854+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406b5d0 0x7f87e4176680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.854+0000 7f87e2d9d700 1 -- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b1b40 msgr2=0x7f87e413a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.854+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b1b40 0x7f87e413a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.854+0000 7f87e2d9d700 1 -- 172.21.15.131:0/1896317700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8009420 con 0x7f87e41558f0 2023-12-26T21:36:57.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.854+0000 7f87e1d9b700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b1b40 0x7f87e413a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:57.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.855+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e41558f0 0x7f87e414ffe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87d8015000 tx=0x7f87d8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.855+0000 7f87d27fc700 1 -- 172.21.15.131:0/1896317700 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d801c670 con 0x7f87e41558f0 2023-12-26T21:36:57.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.855+0000 7f87d27fc700 1 -- 172.21.15.131:0/1896317700 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d801c830 con 0x7f87e41558f0 2023-12-26T21:36:57.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.855+0000 7f87d27fc700 1 -- 172.21.15.131:0/1896317700 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d801cbf0 con 0x7f87e41558f0 2023-12-26T21:36:57.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.855+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e414f500 con 0x7f87e41558f0 2023-12-26T21:36:57.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.857+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e41399a0 con 0x7f87e41558f0 2023-12-26T21:36:57.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.860+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e40657e0 con 0x7f87e41558f0 2023-12-26T21:36:57.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.860+0000 7f87d27fc700 1 -- 172.21.15.131:0/1896317700 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f87d801a450 con 0x7f87e41558f0 2023-12-26T21:36:57.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.862+0000 7f87d27fc700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f87c4067080 0x7f87c4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.862+0000 7f87d27fc700 1 -- 172.21.15.131:0/1896317700 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f87d8086c30 con 0x7f87e41558f0 2023-12-26T21:36:57.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.863+0000 7f87e259c700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f87c4067080 0x7f87c4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.864+0000 7f87e259c700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f87c4067080 0x7f87c4069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87d4007fd0 tx=0x7f87d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.864+0000 7f4754b9d700 1 Processor -- start 2023-12-26T21:36:57.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.864+0000 7f4754b9d700 1 -- start start 2023-12-26T21:36:57.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.866+0000 7f87d27fc700 1 -- 172.21.15.131:0/1896317700 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87d8053f30 con 0x7f87e41558f0 2023-12-26T21:36:57.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.867+0000 7f4754b9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f475006f430 0x7f47501551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.867+0000 7f4754b9d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500ad410 con 0x7f4750155700 2023-12-26T21:36:57.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.867+0000 7f4754b9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500ac840 con 0x7f475006b030 2023-12-26T21:36:57.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.867+0000 7f4754b9d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475015c4a0 con 0x7f475006f430 2023-12-26T21:36:57.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.868+0000 7f474ffff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f475006b030 0x7f47500ab8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39838/0 (socket says 172.21.15.131:39838) 2023-12-26T21:36:57.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.868+0000 7f474f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f475006f430 0x7f47501551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.869+0000 7f474f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f475006f430 0x7f47501551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53582/0 (socket says 172.21.15.131:53582) 2023-12-26T21:36:57.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.869+0000 7f474ffff700 1 -- 172.21.15.131:0/700705167 learned_addr learned my addr 172.21.15.131:0/700705167 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:57.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.869+0000 7f474e7fc700 1 -- 172.21.15.131:0/700705167 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055790528 0 0) 0x7f47500ac840 con 0x7f475006b030 2023-12-26T21:36:57.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.870+0000 7f474e7fc700 1 -- 172.21.15.131:0/700705167 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f473c003570 con 0x7f475006b030 2023-12-26T21:36:57.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.870+0000 7f474f7fe700 1 -- 172.21.15.131:0/700705167 >> v1:172.21.15.190:6789/0 conn(0x7f475006b030 legacy=0x7f47500ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.870+0000 7f474f7fe700 1 -- 172.21.15.131:0/700705167 >> v1:172.21.15.131:6789/0 conn(0x7f4750155700 legacy=0x7f475006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.870+0000 7f474f7fe700 1 -- 172.21.15.131:0/700705167 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500acb20 con 0x7f475006f430 2023-12-26T21:36:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.870+0000 7f474f7fe700 1 --2- 172.21.15.131:0/700705167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f475006f430 0x7f47501551c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4748009e60 tx=0x7f4748009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=662da927e6c3ccf8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.871+0000 7f474e7fc700 1 -- 172.21.15.131:0/700705167 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474800e070 con 0x7f475006f430 2023-12-26T21:36:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.871+0000 7f474e7fc700 1 -- 172.21.15.131:0/700705167 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474800bb60 con 0x7f475006f430 2023-12-26T21:36:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.871+0000 7f474e7fc700 1 -- 172.21.15.131:0/700705167 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748012430 con 0x7f475006f430 2023-12-26T21:36:57.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.874+0000 7f4754b9d700 1 -- 172.21.15.131:0/700705167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f475006f430 msgr2=0x7f47501551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.874+0000 7f4754b9d700 1 --2- 172.21.15.131:0/700705167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f475006f430 0x7f47501551c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4748009e60 tx=0x7f4748009780 comp rx=0 tx=0).stop 2023-12-26T21:36:57.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.874+0000 7f4754b9d700 1 -- 172.21.15.131:0/700705167 shutdown_connections 2023-12-26T21:36:57.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.874+0000 7f4754b9d700 1 --2- 172.21.15.131:0/700705167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f475006f430 0x7f47501551c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.874+0000 7f4754b9d700 1 -- 172.21.15.131:0/700705167 >> 172.21.15.131:0/700705167 conn(0x7f47500cc040 msgr2=0x7f47500cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:57.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.877+0000 7f4754b9d700 1 -- 172.21.15.131:0/700705167 shutdown_connections 2023-12-26T21:36:57.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.877+0000 7f4754b9d700 1 -- 172.21.15.131:0/700705167 wait complete. 2023-12-26T21:36:57.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.879+0000 7f4754b9d700 1 Processor -- start 2023-12-26T21:36:57.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.879+0000 7f4754b9d700 1 -- start start 2023-12-26T21:36:57.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.881+0000 7f4754b9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f475006b030 0x7f475006e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.881+0000 7f4754b9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475006f430 0x7f475006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.881+0000 7f4754b9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750155700 0x7f4750143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.881+0000 7f4754b9d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475016ae60 con 0x7f475006f430 2023-12-26T21:36:57.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.881+0000 7f4754b9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475016a200 con 0x7f475006b030 2023-12-26T21:36:57.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.881+0000 7f4754b9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475016a4e0 con 0x7f4750155700 2023-12-26T21:36:57.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.882+0000 7f474f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f475006b030 0x7f475006e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.882+0000 7f474ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750155700 0x7f4750143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.882+0000 7f474ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750155700 0x7f4750143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57506/0 (socket says 172.21.15.131:57506) 2023-12-26T21:36:57.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.882+0000 7f474ffff700 1 -- 172.21.15.131:0/4072241145 learned_addr learned my addr 172.21.15.131:0/4072241145 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.882+0000 7f474effd700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475006f430 0x7f475006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.883+0000 7f474ffff700 1 -- 172.21.15.131:0/4072241145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f475006b030 msgr2=0x7f475006e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.883+0000 7f474ffff700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f475006b030 0x7f475006e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.883+0000 7f474ffff700 1 -- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475006f430 msgr2=0x7f475006eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.883+0000 7f474ffff700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475006f430 0x7f475006eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.883+0000 7f474ffff700 1 -- 172.21.15.131:0/4072241145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4748009a20 con 0x7f4750155700 2023-12-26T21:36:57.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.884+0000 7f474ffff700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750155700 0x7f4750143200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f47380010e0 tx=0x7f4738007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.884+0000 7f47477fe700 1 -- 172.21.15.131:0/4072241145 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738015d10 con 0x7f4750155700 2023-12-26T21:36:57.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.884+0000 7f47477fe700 1 -- 172.21.15.131:0/4072241145 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4738015e70 con 0x7f4750155700 2023-12-26T21:36:57.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.884+0000 7f47477fe700 1 -- 172.21.15.131:0/4072241145 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738005900 con 0x7f4750155700 2023-12-26T21:36:57.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.884+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475016a7c0 con 0x7f4750155700 2023-12-26T21:36:57.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.884+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475015d1d0 con 0x7f4750155700 2023-12-26T21:36:57.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.888+0000 7f47477fe700 1 -- 172.21.15.131:0/4072241145 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f473801b020 con 0x7f4750155700 2023-12-26T21:36:57.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.889+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47340052b0 con 0x7f4750155700 2023-12-26T21:36:57.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.889+0000 7f47477fe700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.890+0000 7f47477fe700 1 -- 172.21.15.131:0/4072241145 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f473808b140 con 0x7f4750155700 2023-12-26T21:36:57.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.890+0000 7f474f7fe700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.891+0000 7f474f7fe700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47480099f0 tx=0x7f4748019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.898+0000 7f47477fe700 1 -- 172.21.15.131:0/4072241145 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f473804d5a0 con 0x7f4750155700 2023-12-26T21:36:57.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.910+0000 7fd54a39d700 1 Processor -- start 2023-12-26T21:36:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.911+0000 7fd54a39d700 1 -- start start 2023-12-26T21:36:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.913+0000 7fd54a39d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406ab70 0x7fd5440afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.913+0000 7fd54a39d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5440b0170 con 0x7fd5440c8a40 2023-12-26T21:36:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.913+0000 7fd54a39d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5440b0450 con 0x7fd54406af40 2023-12-26T21:36:57.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.913+0000 7fd54a39d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440b0730 con 0x7fd54406ab70 2023-12-26T21:36:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.914+0000 7fd549b9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd5440c8a40 0x7fd544142650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49480/0 (socket says 172.21.15.131:49480) 2023-12-26T21:36:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.918+0000 7fd549b9c700 1 -- 172.21.15.131:0/465031596 learned_addr learned my addr 172.21.15.131:0/465031596 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.917+0000 7fd54939b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406ab70 0x7fd5440afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.917+0000 7fd548b9a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd54406af40 0x7fd5440ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39846/0 (socket says 172.21.15.131:39846) 2023-12-26T21:36:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.918+0000 7fd53bfff700 1 -- 172.21.15.131:0/465031596 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494990559 0 0) 0x7fd5440b0170 con 0x7fd5440c8a40 2023-12-26T21:36:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.918+0000 7fd53bfff700 1 -- 172.21.15.131:0/465031596 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd530003570 con 0x7fd5440c8a40 2023-12-26T21:36:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.918+0000 7fd54939b700 1 -- 172.21.15.131:0/465031596 >> v1:172.21.15.190:6789/0 conn(0x7fd54406af40 legacy=0x7fd5440ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.918+0000 7fd53bfff700 1 -- 172.21.15.131:0/465031596 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169860139 0 0) 0x7fd5440b0450 con 0x7fd54406af40 2023-12-26T21:36:57.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.918+0000 7fd54939b700 1 -- 172.21.15.131:0/465031596 >> v1:172.21.15.131:6789/0 conn(0x7fd5440c8a40 legacy=0x7fd544142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.918+0000 7fd54939b700 1 -- 172.21.15.131:0/465031596 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440b0a10 con 0x7fd54406ab70 2023-12-26T21:36:57.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.920+0000 7fd54939b700 1 --2- 172.21.15.131:0/465031596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406ab70 0x7fd5440afb60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd540009910 tx=0x7fd5400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400da69f0036cf43 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.920+0000 7fd53bfff700 1 -- 172.21.15.131:0/465031596 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54000b710 con 0x7fd54406ab70 2023-12-26T21:36:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.920+0000 7fd53bfff700 1 -- 172.21.15.131:0/465031596 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54000b870 con 0x7fd54406ab70 2023-12-26T21:36:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.920+0000 7fd53bfff700 1 -- 172.21.15.131:0/465031596 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54000bc30 con 0x7fd54406ab70 2023-12-26T21:36:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.921+0000 7fd54a39d700 1 -- 172.21.15.131:0/465031596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406ab70 msgr2=0x7fd5440afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.921+0000 7fd54a39d700 1 --2- 172.21.15.131:0/465031596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406ab70 0x7fd5440afb60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd540009910 tx=0x7fd5400091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.922+0000 7ff462b9d700 1 Processor -- start 2023-12-26T21:36:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.922+0000 7fd54a39d700 1 -- 172.21.15.131:0/465031596 shutdown_connections 2023-12-26T21:36:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.922+0000 7ff462b9d700 1 -- start start 2023-12-26T21:36:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.922+0000 7fd54a39d700 1 --2- 172.21.15.131:0/465031596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406ab70 0x7fd5440afb60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.922+0000 7fd54a39d700 1 -- 172.21.15.131:0/465031596 >> 172.21.15.131:0/465031596 conn(0x7fd5440cc040 msgr2=0x7fd5440cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:57.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.923+0000 7f125bfff700 1 Processor -- start 2023-12-26T21:36:57.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.923+0000 7f125bfff700 1 -- start start 2023-12-26T21:36:57.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.925+0000 7fd54a39d700 1 -- 172.21.15.131:0/465031596 shutdown_connections 2023-12-26T21:36:57.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.924+0000 7ff462b9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff45c159420 0x7ff45c0ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.924+0000 7ff462b9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c0c64b0 0x7ff45c0ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.924+0000 7ff462b9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45c0cd130 con 0x7ff45c1597f0 2023-12-26T21:36:57.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.924+0000 7ff462b9d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45c0cbf10 con 0x7ff45c159420 2023-12-26T21:36:57.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.924+0000 7ff462b9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0cc1f0 con 0x7ff45c0c64b0 2023-12-26T21:36:57.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.925+0000 7fd54a39d700 1 -- 172.21.15.131:0/465031596 wait complete. 2023-12-26T21:36:57.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.925+0000 7ff461b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff45c159420 0x7ff45c0ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.925+0000 7ff461b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff45c159420 0x7ff45c0ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45018/0 (socket says 172.21.15.131:45018) 2023-12-26T21:36:57.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.925+0000 7ff46239c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c0c64b0 0x7ff45c0ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.925+0000 7ff461b9b700 1 -- 172.21.15.131:0/849049588 learned_addr learned my addr 172.21.15.131:0/849049588 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.925+0000 7f125affd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f12540dfd40 0x7f12540e1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49492/0 (socket says 172.21.15.131:49492) 2023-12-26T21:36:57.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.926+0000 7f125affd700 1 -- 172.21.15.131:0/3185006569 learned_addr learned my addr 172.21.15.131:0/3185006569 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.926+0000 7f125bfff700 1 --2- 172.21.15.131:0/3185006569 >> v2:172.21.15.131:3300/0 conn(0x7f12540338f0 0x7f12540e0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.927+0000 7f125bfff700 1 -- 172.21.15.131:0/3185006569 --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12540e0bd0 con 0x7f12540dfd40 2023-12-26T21:36:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.927+0000 7fd54a39d700 1 Processor -- start 2023-12-26T21:36:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.927+0000 7f1259ffb700 1 -- 172.21.15.131:0/3185006569 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596933996 0 0) 0x7f12540e0bd0 con 0x7f12540dfd40 2023-12-26T21:36:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.927+0000 7f125bfff700 1 -- 172.21.15.131:0/3185006569 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1254033f80 con 0x7f12540351d0 2023-12-26T21:36:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.928+0000 7f125bfff700 1 -- 172.21.15.131:0/3185006569 --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1254033d90 con 0x7f12540338f0 2023-12-26T21:36:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.928+0000 7f125a7fc700 1 --2- 172.21.15.131:0/3185006569 >> v2:172.21.15.131:3300/0 conn(0x7f12540338f0 0x7f12540e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.928+0000 7f1259ffb700 1 -- 172.21.15.131:0/3185006569 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1244003570 con 0x7f12540dfd40 2023-12-26T21:36:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.928+0000 7f125a7fc700 1 -- 172.21.15.131:0/3185006569 >> v1:172.21.15.190:6789/0 conn(0x7f12540351d0 legacy=0x7f1254036c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.928+0000 7f1259ffb700 1 -- 172.21.15.131:0/3185006569 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2128579251 0 0) 0x7f1244003570 con 0x7f12540dfd40 2023-12-26T21:36:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.929+0000 7f125a7fc700 1 -- 172.21.15.131:0/3185006569 >> v1:172.21.15.131:6789/0 conn(0x7f12540dfd40 legacy=0x7f12540e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.929+0000 7f125a7fc700 1 -- 172.21.15.131:0/3185006569 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1254034260 con 0x7f12540338f0 2023-12-26T21:36:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.929+0000 7f125a7fc700 1 --2- 172.21.15.131:0/3185006569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12540338f0 0x7f12540e0110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f124c009b40 tx=0x7f124c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bacb4174039920 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.926+0000 7ff461b9b700 1 -- 172.21.15.131:0/849049588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c0c64b0 msgr2=0x7ff45c0ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.926+0000 7ff461b9b700 1 --2- 172.21.15.131:0/849049588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c0c64b0 0x7ff45c0ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.926+0000 7ff461b9b700 1 -- 172.21.15.131:0/849049588 >> v1:172.21.15.190:6789/0 conn(0x7ff45c1597f0 legacy=0x7ff45c0cb2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-26T21:36:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.926+0000 7ff461b9b700 1 -- 172.21.15.131:0/849049588 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0cc4d0 con 0x7ff45c159420 2023-12-26T21:36:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.929+0000 7ff461b9b700 1 --2- 172.21.15.131:0/849049588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c159420 0x7ff45c0ce9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff454009f70 tx=0x7ff454009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef8460cdf9e3e5c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.929+0000 7ff460b99700 1 -- 172.21.15.131:0/849049588 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45400e070 con 0x7ff45c159420 2023-12-26T21:36:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.929+0000 7fd54a39d700 1 -- start start 2023-12-26T21:36:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.931+0000 7ff460b99700 1 -- 172.21.15.131:0/849049588 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45400b9e0 con 0x7ff45c159420 2023-12-26T21:36:57.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.931+0000 7ff460b99700 1 -- 172.21.15.131:0/849049588 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4540092e0 con 0x7ff45c159420 2023-12-26T21:36:57.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.931+0000 7fd54a39d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd54406ab70 0x7fd5440a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.931+0000 7fd54a39d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406af40 0x7fd54415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.931+0000 7fd54a39d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5440c8a40 0x7fd544175500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.931+0000 7fd54a39d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440a7310 con 0x7fd5440c8a40 2023-12-26T21:36:57.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7fd54a39d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440a75f0 con 0x7fd54406ab70 2023-12-26T21:36:57.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7fd54a39d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440aa070 con 0x7fd54406af40 2023-12-26T21:36:57.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7ff462b9d700 1 -- 172.21.15.131:0/849049588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c159420 msgr2=0x7ff45c0ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7ff462b9d700 1 --2- 172.21.15.131:0/849049588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c159420 0x7ff45c0ce9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff454009f70 tx=0x7ff454009840 comp rx=0 tx=0).stop 2023-12-26T21:36:57.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7ff462b9d700 1 -- 172.21.15.131:0/849049588 shutdown_connections 2023-12-26T21:36:57.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7ff462b9d700 1 --2- 172.21.15.131:0/849049588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c0c64b0 0x7ff45c0ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7ff462b9d700 1 --2- 172.21.15.131:0/849049588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c159420 0x7ff45c0ce9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7ff462b9d700 1 -- 172.21.15.131:0/849049588 >> 172.21.15.131:0/849049588 conn(0x7ff45c099780 msgr2=0x7ff45c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:57.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7fd548b9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406af40 0x7fd54415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7fd548b9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406af40 0x7fd54415c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57526/0 (socket says 172.21.15.131:57526) 2023-12-26T21:36:57.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7fd548b9a700 1 -- 172.21.15.131:0/366887284 learned_addr learned my addr 172.21.15.131:0/366887284 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7fd549b9c700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5440c8a40 0x7fd544175500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.932+0000 7fd548b9a700 1 -- 172.21.15.131:0/366887284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd54406ab70 msgr2=0x7fd5440a9700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:36:57.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.931+0000 7f1259ffb700 1 -- 172.21.15.131:0/3185006569 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250001470 con 0x7f12540338f0 2023-12-26T21:36:57.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7ff462b9d700 1 -- 172.21.15.131:0/849049588 shutdown_connections 2023-12-26T21:36:57.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7fd54939b700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd54406ab70 0x7fd5440a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7f1259ffb700 1 -- 172.21.15.131:0/3185006569 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c00b620 con 0x7f12540338f0 2023-12-26T21:36:57.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7ff462b9d700 1 -- 172.21.15.131:0/849049588 wait complete. 2023-12-26T21:36:57.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7f1259ffb700 1 -- 172.21.15.131:0/3185006569 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00b7d0 con 0x7f12540338f0 2023-12-26T21:36:57.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7fd548b9a700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd54406ab70 0x7fd5440a9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7fd548b9a700 1 -- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5440c8a40 msgr2=0x7fd544175500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7fd548b9a700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5440c8a40 0x7fd544175500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7fd548b9a700 1 -- 172.21.15.131:0/366887284 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd540009420 con 0x7fd54406af40 2023-12-26T21:36:57.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7fd549b9c700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5440c8a40 0x7fd544175500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:57.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.933+0000 7fd548b9a700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406af40 0x7fd54415c210 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd5340011e0 tx=0x7fd534007e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.934+0000 7fd5397fa700 1 -- 172.21.15.131:0/366887284 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534007300 con 0x7fd54406af40 2023-12-26T21:36:57.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.934+0000 7fd5397fa700 1 -- 172.21.15.131:0/366887284 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534007460 con 0x7fd54406af40 2023-12-26T21:36:57.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.934+0000 7fd5397fa700 1 -- 172.21.15.131:0/366887284 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534007300 con 0x7fd54406af40 2023-12-26T21:36:57.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.935+0000 7fd54a39d700 1 -- 172.21.15.131:0/366887284 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5440aa350 con 0x7fd54406af40 2023-12-26T21:36:57.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.935+0000 7ff462b9d700 1 Processor -- start 2023-12-26T21:36:57.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.935+0000 7f125bfff700 1 -- 172.21.15.131:0/3185006569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12540338f0 msgr2=0x7f12540e0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.935+0000 7f125bfff700 1 --2- 172.21.15.131:0/3185006569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12540338f0 0x7f12540e0110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f124c009b40 tx=0x7f124c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.936+0000 7ff462b9d700 1 -- start start 2023-12-26T21:36:57.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.936+0000 7fd54a39d700 1 -- 172.21.15.131:0/366887284 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5440a9c40 con 0x7fd54406af40 2023-12-26T21:36:57.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7f125bfff700 1 -- 172.21.15.131:0/3185006569 shutdown_connections 2023-12-26T21:36:57.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7f125bfff700 1 --2- 172.21.15.131:0/3185006569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12540338f0 0x7f12540e0110 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7f125bfff700 1 -- 172.21.15.131:0/3185006569 >> 172.21.15.131:0/3185006569 conn(0x7f1254054dd0 msgr2=0x7f12540551d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:57.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7ff462b9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c0c64b0 0x7ff45c1491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7f125bfff700 1 -- 172.21.15.131:0/3185006569 shutdown_connections 2023-12-26T21:36:57.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7ff462b9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c159420 0x7ff45c0d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7ff462b9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c1597f0 0x7ff45c1409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7ff462b9d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c141580 con 0x7ff45c159420 2023-12-26T21:36:57.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7ff462b9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0b4d00 con 0x7ff45c0c64b0 2023-12-26T21:36:57.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.937+0000 7ff462b9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0b4410 con 0x7ff45c1597f0 2023-12-26T21:36:57.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff46239c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c1597f0 0x7ff45c1409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff46239c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c1597f0 0x7ff45c1409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57540/0 (socket says 172.21.15.131:57540) 2023-12-26T21:36:57.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff46239c700 1 -- 172.21.15.131:0/4175238644 learned_addr learned my addr 172.21.15.131:0/4175238644 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7fd52effd700 1 -- 172.21.15.131:0/366887284 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5440cb180 con 0x7fd54406af40 2023-12-26T21:36:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff461b9b700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c0c64b0 0x7ff45c1491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff46239c700 1 -- 172.21.15.131:0/4175238644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c0c64b0 msgr2=0x7ff45c1491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff46239c700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c0c64b0 0x7ff45c1491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff46239c700 1 -- 172.21.15.131:0/4175238644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c159420 msgr2=0x7ff45c0d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff46239c700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c159420 0x7ff45c0d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff46239c700 1 -- 172.21.15.131:0/4175238644 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff454009ae0 con 0x7ff45c1597f0 2023-12-26T21:36:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7ff461b9b700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c0c64b0 0x7ff45c1491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:36:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.939+0000 7ff46239c700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c1597f0 0x7ff45c1409c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff45800ea80 tx=0x7ff4580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.938+0000 7f125bfff700 1 -- 172.21.15.131:0/3185006569 wait complete. 2023-12-26T21:36:57.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.939+0000 7ff451ffb700 1 -- 172.21.15.131:0/4175238644 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4580096b0 con 0x7ff45c1597f0 2023-12-26T21:36:57.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.939+0000 7ff451ffb700 1 -- 172.21.15.131:0/4175238644 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800c920 con 0x7ff45c1597f0 2023-12-26T21:36:57.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.940+0000 7ff451ffb700 1 -- 172.21.15.131:0/4175238644 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800ccc0 con 0x7ff45c1597f0 2023-12-26T21:36:57.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.940+0000 7f125bfff700 1 Processor -- start 2023-12-26T21:36:57.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.940+0000 7fd5397fa700 1 -- 172.21.15.131:0/366887284 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd5340075d0 con 0x7fd54406af40 2023-12-26T21:36:57.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.941+0000 7f125bfff700 1 -- start start 2023-12-26T21:36:57.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.940+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c0b46b0 con 0x7ff45c1597f0 2023-12-26T21:36:57.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.940+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c13dd60 con 0x7ff45c1597f0 2023-12-26T21:36:57.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.942+0000 7fd5397fa700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd528066ff0 0x7fd5280694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.942+0000 7fd5397fa700 1 -- 172.21.15.131:0/366887284 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd534083350 con 0x7fd54406af40 2023-12-26T21:36:57.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.942+0000 7f125bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12540338f0 0x7f1254038b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.942+0000 7f125bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12540351d0 0x7f12540f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.942+0000 7f125bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12540dfd40 0x7f12540f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.943+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45c14c760 con 0x7ff45c1597f0 2023-12-26T21:36:57.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.943+0000 7f125affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12540338f0 0x7f1254038b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.943+0000 7f125bfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12540f5db0 con 0x7f12540338f0 2023-12-26T21:36:57.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.943+0000 7f125bfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125403a0c0 con 0x7f12540dfd40 2023-12-26T21:36:57.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.943+0000 7f125bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12540394b0 con 0x7f12540351d0 2023-12-26T21:36:57.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.943+0000 7fd54939b700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd528066ff0 0x7fd5280694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.943+0000 7f125b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12540dfd40 0x7f12540f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.943+0000 7f125b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12540dfd40 0x7f12540f2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53614/0 (socket says 172.21.15.131:53614) 2023-12-26T21:36:57.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.943+0000 7f125b7fe700 1 -- 172.21.15.131:0/539730686 learned_addr learned my addr 172.21.15.131:0/539730686 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:57.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.944+0000 7f125a7fc700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12540351d0 0x7f12540f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.944+0000 7f125b7fe700 1 -- 172.21.15.131:0/539730686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12540351d0 msgr2=0x7f12540f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.944+0000 7f125b7fe700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12540351d0 0x7f12540f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.944+0000 7f125b7fe700 1 -- 172.21.15.131:0/539730686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12540338f0 msgr2=0x7f1254038b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:57.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.944+0000 7fd54939b700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd528066ff0 0x7fd5280694a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd540009860 tx=0x7fd540017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.944+0000 7f125b7fe700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12540338f0 0x7f1254038b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:57.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.944+0000 7f125b7fe700 1 -- 172.21.15.131:0/539730686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c009420 con 0x7f12540dfd40 2023-12-26T21:36:57.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.944+0000 7f125a7fc700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12540351d0 0x7f12540f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:57.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.945+0000 7f125b7fe700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12540dfd40 0x7f12540f2eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f125c04d1b0 tx=0x7f125c064240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.945+0000 7f1242ffd700 1 -- 172.21.15.131:0/539730686 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c064910 con 0x7f12540dfd40 2023-12-26T21:36:57.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.945+0000 7fd5397fa700 1 -- 172.21.15.131:0/366887284 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd53404c5a0 con 0x7fd54406af40 2023-12-26T21:36:57.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.945+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1254039790 con 0x7f12540dfd40 2023-12-26T21:36:57.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.946+0000 7f1242ffd700 1 -- 172.21.15.131:0/539730686 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c068cb0 con 0x7f12540dfd40 2023-12-26T21:36:57.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.946+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1254039d50 con 0x7f12540dfd40 2023-12-26T21:36:57.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.947+0000 7f1242ffd700 1 -- 172.21.15.131:0/539730686 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c064910 con 0x7f12540dfd40 2023-12-26T21:36:57.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.948+0000 7ff451ffb700 1 -- 172.21.15.131:0/4175238644 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff458013680 con 0x7ff45c1597f0 2023-12-26T21:36:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.949+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c0052b0 con 0x7f12540dfd40 2023-12-26T21:36:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.949+0000 7f1242ffd700 1 -- 172.21.15.131:0/539730686 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f125c0983f0 con 0x7f12540dfd40 2023-12-26T21:36:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.949+0000 7ff451ffb700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff440067080 0x7ff440069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.950+0000 7ff451ffb700 1 -- 172.21.15.131:0/4175238644 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff458084fe0 con 0x7ff45c1597f0 2023-12-26T21:36:57.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.951+0000 7ff451ffb700 1 -- 172.21.15.131:0/4175238644 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4580522b0 con 0x7ff45c1597f0 2023-12-26T21:36:57.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.951+0000 7f1242ffd700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1248067010 0x7f12480694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:57.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.951+0000 7f1242ffd700 1 -- 172.21.15.131:0/539730686 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f125c0fae50 con 0x7f12540dfd40 2023-12-26T21:36:57.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.952+0000 7f125affd700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1248067010 0x7f12480694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.953+0000 7ff461b9b700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff440067080 0x7ff440069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:57.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.954+0000 7ff461b9b700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff440067080 0x7ff440069530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff454009880 tx=0x7ff454017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.954+0000 7f125affd700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1248067010 0x7f12480694c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1250001ed0 tx=0x7f1250000d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:57.956+0000 7f1242ffd700 1 -- 172.21.15.131:0/539730686 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f125c0cbab0 con 0x7f12540dfd40 2023-12-26T21:36:58.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.203+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fda30005560 con 0x7fda5014a690 2023-12-26T21:36:58.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.204+0000 7fda37fff700 1 -- 172.21.15.131:0/2178945402 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda40051ca0 con 0x7fda5014a690 2023-12-26T21:36:58.204 INFO:teuthology.orchestra.run.smithi131.stdout:64424509447 2023-12-26T21:36:58.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fda3c066fb0 msgr2=0x7fda3c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fda3c066fb0 0x7fda3c069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda48007fd0 tx=0x7fda48009040 comp rx=0 tx=0).stop 2023-12-26T21:36:58.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5014a690 msgr2=0x7fda501760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5014a690 0x7fda501760b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda40009af0 tx=0x7fda40016000 comp rx=0 tx=0).stop 2023-12-26T21:36:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 shutdown_connections 2023-12-26T21:36:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5014a690 0x7fda501760b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fda3c066fb0 0x7fda3c069460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda50155410 0x7fda50156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 --2- 172.21.15.131:0/2178945402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda501557e0 0x7fda50158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.208+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 >> 172.21.15.131:0/2178945402 conn(0x7fda500cc040 msgr2=0x7fda500a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.209+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 shutdown_connections 2023-12-26T21:36:58.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.209+0000 7fda5519d700 1 -- 172.21.15.131:0/2178945402 wait complete. 2023-12-26T21:36:58.240 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2023-12-26T21:36:58.240 DEBUG:teuthology.parallel:result is None 2023-12-26T21:36:58.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.276+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb48804fc20 con 0x7fb48800d940 2023-12-26T21:36:58.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.277+0000 7fb47e7fc700 1 -- 172.21.15.131:0/3891613765 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb48804fc20 con 0x7fb48800d940 2023-12-26T21:36:58.277 INFO:teuthology.orchestra.run.smithi131.stdout:64424509447 2023-12-26T21:36:58.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.278+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fba8804c3c0 con 0x7fba880c8a40 2023-12-26T21:36:58.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.279+0000 7fba7dffb700 1 -- 172.21.15.131:0/1394165104 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba84054ec0 con 0x7fba880c8a40 2023-12-26T21:36:58.279 INFO:teuthology.orchestra.run.smithi131.stdout:64424509446 2023-12-26T21:36:58.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.281+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb470066f90 msgr2=0x7fb470069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.281+0000 7fb48f59e700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb470066f90 0x7fb470069440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb4880e1b60 tx=0x7fb480007de0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.281+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb48800d940 msgr2=0x7fb48804ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.281+0000 7fb48f59e700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb48800d940 0x7fb48804ff00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb4880e6fe0 tx=0x7fb484017000 comp rx=0 tx=0).stop 2023-12-26T21:36:58.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.282+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 shutdown_connections 2023-12-26T21:36:58.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.282+0000 7fb48f59e700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb470066f90 0x7fb470069440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.282+0000 7fb48f59e700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4880e28b0 0x7fb4880e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.282+0000 7fb48f59e700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb48800d940 0x7fb48804ff00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.282+0000 7fb48f59e700 1 --2- 172.21.15.131:0/3891613765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4880dc620 0x7fb4880cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.282+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 >> 172.21.15.131:0/3891613765 conn(0x7fb48800b320 msgr2=0x7fb4880e1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.282+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 shutdown_connections 2023-12-26T21:36:58.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.282+0000 7fb48f59e700 1 -- 172.21.15.131:0/3891613765 wait complete. 2023-12-26T21:36:58.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba6c067080 msgr2=0x7fba6c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba6c067080 0x7fba6c069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba80007ad0 tx=0x7fba80009820 comp rx=0 tx=0).stop 2023-12-26T21:36:58.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba880c8a40 msgr2=0x7fba8813ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba880c8a40 0x7fba8813ab50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba8400d890 tx=0x7fba84008040 comp rx=0 tx=0).stop 2023-12-26T21:36:58.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 shutdown_connections 2023-12-26T21:36:58.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba880c8a40 0x7fba8813ab50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba6c067080 0x7fba6c069530 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba80007ad0 tx=0x7fba80009820 comp rx=0 tx=0).stop 2023-12-26T21:36:58.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8806ab70 0x7fba8815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 --2- 172.21.15.131:0/1394165104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba8806af40 0x7fba88158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.284+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 >> 172.21.15.131:0/1394165104 conn(0x7fba880cc040 msgr2=0x7fba880c8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.285+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 shutdown_connections 2023-12-26T21:36:58.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.285+0000 7fba8ed9d700 1 -- 172.21.15.131:0/1394165104 wait complete. 2023-12-26T21:36:58.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.295+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f87e40657e0 con 0x7f87e41558f0 2023-12-26T21:36:58.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.296+0000 7f87d27fc700 1 -- 172.21.15.131:0/1896317700 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87d8057550 con 0x7f87e41558f0 2023-12-26T21:36:58.296 INFO:teuthology.orchestra.run.smithi131.stdout:60129542153 2023-12-26T21:36:58.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f87c4067080 msgr2=0x7f87c4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f87c4067080 0x7f87c4069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87d4007fd0 tx=0x7f87d4009040 comp rx=0 tx=0).stop 2023-12-26T21:36:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e41558f0 msgr2=0x7f87e414ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e41558f0 0x7f87e414ffe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87d8015000 tx=0x7f87d8016000 comp rx=0 tx=0).stop 2023-12-26T21:36:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 shutdown_connections 2023-12-26T21:36:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e41558f0 0x7f87e414ffe0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f87c4067080 0x7f87c4069530 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406b5d0 0x7f87e4176680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 --2- 172.21.15.131:0/1896317700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b1b40 0x7f87e413a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.300+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 >> 172.21.15.131:0/1896317700 conn(0x7f87e4068040 msgr2=0x7f87e4157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.302+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 shutdown_connections 2023-12-26T21:36:58.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.302+0000 7f87e8fff700 1 -- 172.21.15.131:0/1896317700 wait complete. 2023-12-26T21:36:58.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2023-12-26T21:36:58.316 DEBUG:teuthology.parallel:result is None 2023-12-26T21:36:58.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2023-12-26T21:36:58.316 DEBUG:teuthology.parallel:result is None 2023-12-26T21:36:58.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.318+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff45c04c3c0 con 0x7ff45c1597f0 2023-12-26T21:36:58.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.319+0000 7ff451ffb700 1 -- 172.21.15.131:0/4175238644 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4580558d0 con 0x7ff45c1597f0 2023-12-26T21:36:58.320 INFO:teuthology.orchestra.run.smithi131.stdout:60129542152 2023-12-26T21:36:58.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff440067080 msgr2=0x7ff440069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff440067080 0x7ff440069530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff454009880 tx=0x7ff454017000 comp rx=0 tx=0).stop 2023-12-26T21:36:58.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c1597f0 msgr2=0x7ff45c1409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c1597f0 0x7ff45c1409c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff45800ea80 tx=0x7ff4580091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 shutdown_connections 2023-12-26T21:36:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff440067080 0x7ff440069530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff45c1597f0 0x7ff45c1409c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff45c0c64b0 0x7ff45c1491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 --2- 172.21.15.131:0/4175238644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c159420 0x7ff45c0d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 >> 172.21.15.131:0/4175238644 conn(0x7ff45c099780 msgr2=0x7ff45c14c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.322+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 shutdown_connections 2023-12-26T21:36:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.323+0000 7ff462b9d700 1 -- 172.21.15.131:0/4175238644 wait complete. 2023-12-26T21:36:58.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542153 for osd.2 2023-12-26T21:36:58.326 DEBUG:teuthology.parallel:result is None 2023-12-26T21:36:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.341+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4734005830 con 0x7f4750155700 2023-12-26T21:36:58.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.4 2023-12-26T21:36:58.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.342+0000 7f47477fe700 1 -- 172.21.15.131:0/4072241145 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4738090060 con 0x7f4750155700 2023-12-26T21:36:58.342 INFO:teuthology.orchestra.run.smithi131.stdout:64424509447 2023-12-26T21:36:58.342 DEBUG:teuthology.parallel:result is None 2023-12-26T21:36:58.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.344+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 msgr2=0x7f47300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.344+0000 7f4754b9d700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47480099f0 tx=0x7f4748019000 comp rx=0 tx=0).stop 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.344+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750155700 msgr2=0x7f4750143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.344+0000 7f4754b9d700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750155700 0x7f4750143200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f47380010e0 tx=0x7f4738007dd0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.345+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 shutdown_connections 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.345+0000 7f4754b9d700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.345+0000 7f4754b9d700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4750155700 0x7f4750143200 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.345+0000 7f4754b9d700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f475006b030 0x7f475006e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.345+0000 7f4754b9d700 1 --2- 172.21.15.131:0/4072241145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475006f430 0x7f475006eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.345+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 >> 172.21.15.131:0/4072241145 conn(0x7f47500cc040 msgr2=0x7f47500ab880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.345+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 shutdown_connections 2023-12-26T21:36:58.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.345+0000 7f4754b9d700 1 -- 172.21.15.131:0/4072241145 wait complete. 2023-12-26T21:36:58.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.352+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f123c005560 con 0x7f12540dfd40 2023-12-26T21:36:58.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.352+0000 7f1242ffd700 1 -- 172.21.15.131:0/539730686 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f125c0cb640 con 0x7f12540dfd40 2023-12-26T21:36:58.352 INFO:teuthology.orchestra.run.smithi131.stdout:60129542152 2023-12-26T21:36:58.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1248067010 msgr2=0x7f12480694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1248067010 0x7f12480694c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1250001ed0 tx=0x7f1250000d00 comp rx=0 tx=0).stop 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12540dfd40 msgr2=0x7f12540f2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12540dfd40 0x7f12540f2eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f125c04d1b0 tx=0x7f125c064240 comp rx=0 tx=0).stop 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 shutdown_connections 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12540dfd40 0x7f12540f2eb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12540338f0 0x7f1254038b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1248067010 0x7f12480694c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 --2- 172.21.15.131:0/539730686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12540351d0 0x7f12540f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 >> 172.21.15.131:0/539730686 conn(0x7f1254054dd0 msgr2=0x7f1254036be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 shutdown_connections 2023-12-26T21:36:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.355+0000 7f125bfff700 1 -- 172.21.15.131:0/539730686 wait complete. 2023-12-26T21:36:58.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2023-12-26T21:36:58.367 DEBUG:teuthology.parallel:result is None 2023-12-26T21:36:58.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.369+0000 7fd52effd700 1 -- 172.21.15.131:0/366887284 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd54404c3c0 con 0x7fd54406af40 2023-12-26T21:36:58.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.370+0000 7fd5397fa700 1 -- 172.21.15.131:0/366887284 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd534018090 con 0x7fd54406af40 2023-12-26T21:36:58.369 INFO:teuthology.orchestra.run.smithi131.stdout:64424509448 2023-12-26T21:36:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 -- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd528066ff0 msgr2=0x7fd5280694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd528066ff0 0x7fd5280694a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd540009860 tx=0x7fd540017000 comp rx=0 tx=0).stop 2023-12-26T21:36:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 -- 172.21.15.131:0/366887284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406af40 msgr2=0x7fd54415c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406af40 0x7fd54415c210 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd5340011e0 tx=0x7fd534007e10 comp rx=0 tx=0).stop 2023-12-26T21:36:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 -- 172.21.15.131:0/366887284 shutdown_connections 2023-12-26T21:36:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5440c8a40 0x7fd544175500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd54406ab70 0x7fd5440a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd528066ff0 0x7fd5280694a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 --2- 172.21.15.131:0/366887284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd54406af40 0x7fd54415c210 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 -- 172.21.15.131:0/366887284 >> 172.21.15.131:0/366887284 conn(0x7fd5440cc040 msgr2=0x7fd5440704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 -- 172.21.15.131:0/366887284 shutdown_connections 2023-12-26T21:36:58.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.372+0000 7fd52effd700 1 -- 172.21.15.131:0/366887284 wait complete. 2023-12-26T21:36:58.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.7 2023-12-26T21:36:58.374 DEBUG:teuthology.parallel:result is None 2023-12-26T21:36:58.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.3 2023-12-26T21:36:58.390 DEBUG:teuthology.parallel:result is None 2023-12-26T21:36:58.390 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-26T21:36:58.391 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:36:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.621+0000 7fb710dff700 1 Processor -- start 2023-12-26T21:36:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.621+0000 7fb710dff700 1 -- start start 2023-12-26T21:36:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.622+0000 7fb710dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c1596a0 0x7fb70c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.622+0000 7fb710dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb70c159a70 0x7fb70c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.622+0000 7fb710dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c0afcd0 con 0x7fb70c0b0160 2023-12-26T21:36:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.622+0000 7fb710dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c0b3e70 con 0x7fb70c159a70 2023-12-26T21:36:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.622+0000 7fb710dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0b0cd0 con 0x7fb70c1596a0 2023-12-26T21:36:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb70a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c1596a0 0x7fb70c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb709d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb70c159a70 0x7fb70c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb70ad9d700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fb70c0b0160 0x7fb70c0b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39858/0 (socket says 172.21.15.131:39858) 2023-12-26T21:36:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb70ad9d700 1 -- 172.21.15.131:0/2633685100 learned_addr learned my addr 172.21.15.131:0/2633685100 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb70959a700 1 -- 172.21.15.131:0/2633685100 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304472595 0 0) 0x7fb70c0afcd0 con 0x7fb70c0b0160 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb70959a700 1 -- 172.21.15.131:0/2633685100 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6f4003570 con 0x7fb70c0b0160 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb709d9b700 1 -- 172.21.15.131:0/2633685100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c1596a0 msgr2=0x7fb70c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb709d9b700 1 --2- 172.21.15.131:0/2633685100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c1596a0 0x7fb70c0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb709d9b700 1 -- 172.21.15.131:0/2633685100 >> v1:172.21.15.190:6789/0 conn(0x7fb70c0b0160 legacy=0x7fb70c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb709d9b700 1 -- 172.21.15.131:0/2633685100 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0b2e00 con 0x7fb70c159a70 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb70a59c700 1 --2- 172.21.15.131:0/2633685100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c1596a0 0x7fb70c0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.623+0000 7fb709d9b700 1 --2- 172.21.15.131:0/2633685100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 0x7fb70c0b28c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb7000098e0 tx=0x7fb7000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e85d1ff5d2499df2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb70959a700 1 -- 172.21.15.131:0/2633685100 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700011630 con 0x7fb70c159a70 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb70959a700 1 -- 172.21.15.131:0/2633685100 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700011790 con 0x7fb70c159a70 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb70959a700 1 -- 172.21.15.131:0/2633685100 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700011b30 con 0x7fb70c159a70 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb710dff700 1 -- 172.21.15.131:0/2633685100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 msgr2=0x7fb70c0b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb710dff700 1 --2- 172.21.15.131:0/2633685100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 0x7fb70c0b28c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb7000098e0 tx=0x7fb7000091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb710dff700 1 -- 172.21.15.131:0/2633685100 shutdown_connections 2023-12-26T21:36:58.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb710dff700 1 --2- 172.21.15.131:0/2633685100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c1596a0 0x7fb70c0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb710dff700 1 --2- 172.21.15.131:0/2633685100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 0x7fb70c0b28c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb710dff700 1 -- 172.21.15.131:0/2633685100 >> 172.21.15.131:0/2633685100 conn(0x7fb70c06ece0 msgr2=0x7fb70c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb710dff700 1 -- 172.21.15.131:0/2633685100 shutdown_connections 2023-12-26T21:36:58.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.624+0000 7fb710dff700 1 -- 172.21.15.131:0/2633685100 wait complete. 2023-12-26T21:36:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.626+0000 7fb710dff700 1 Processor -- start 2023-12-26T21:36:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.626+0000 7fb710dff700 1 -- start start 2023-12-26T21:36:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb710dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c0b0160 0x7fb70c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb710dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb70c1596a0 0x7fb70c15b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb710dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 0x7fb70c15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb710dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c15bdd0 con 0x7fb70c1596a0 2023-12-26T21:36:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb710dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0c84d0 con 0x7fb70c159a70 2023-12-26T21:36:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb710dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c15a000 con 0x7fb70c0b0160 2023-12-26T21:36:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb709d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb70c1596a0 0x7fb70c15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb709d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb70c1596a0 0x7fb70c15b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45078/0 (socket says 172.21.15.131:45078) 2023-12-26T21:36:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb709d9b700 1 -- 172.21.15.131:0/74190540 learned_addr learned my addr 172.21.15.131:0/74190540 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb70ad9d700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 0x7fb70c15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb70a59c700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c0b0160 0x7fb70c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb709d9b700 1 -- 172.21.15.131:0/74190540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c0b0160 msgr2=0x7fb70c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb709d9b700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c0b0160 0x7fb70c156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb709d9b700 1 -- 172.21.15.131:0/74190540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 msgr2=0x7fb70c15b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb709d9b700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 0x7fb70c15b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb709d9b700 1 -- 172.21.15.131:0/74190540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc009420 con 0x7fb70c1596a0 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.627+0000 7fb70ad9d700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 0x7fb70c15b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.628+0000 7fb709d9b700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb70c1596a0 0x7fb70c15b260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb70c089710 tx=0x7fb700017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.628+0000 7fb6fa7fc700 1 -- 172.21.15.131:0/74190540 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700011b10 con 0x7fb70c1596a0 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.628+0000 7fb6fa7fc700 1 -- 172.21.15.131:0/74190540 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7000253f0 con 0x7fb70c1596a0 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.628+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb700009420 con 0x7fb70c1596a0 2023-12-26T21:36:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.628+0000 7fb6fa7fc700 1 -- 172.21.15.131:0/74190540 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7000257b0 con 0x7fb70c1596a0 2023-12-26T21:36:58.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.628+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c15a5c0 con 0x7fb70c1596a0 2023-12-26T21:36:58.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.629+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c0657e0 con 0x7fb70c1596a0 2023-12-26T21:36:58.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.630+0000 7fb6fa7fc700 1 -- 172.21.15.131:0/74190540 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb700025ba0 con 0x7fb70c1596a0 2023-12-26T21:36:58.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.631+0000 7fb6fa7fc700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb6ec066f30 0x7fb6ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:58.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.631+0000 7fb6fa7fc700 1 -- 172.21.15.131:0/74190540 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb700085f20 con 0x7fb70c1596a0 2023-12-26T21:36:58.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.631+0000 7fb70a59c700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb6ec066f30 0x7fb6ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:58.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.631+0000 7fb70a59c700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb6ec066f30 0x7fb6ec0693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009d60 tx=0x7fb6fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:58.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.635+0000 7fb6fa7fc700 1 -- 172.21.15.131:0/74190540 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7000530f0 con 0x7fb70c1596a0 2023-12-26T21:36:58.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.823+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb70c0cac70 con 0x7fb6ec066f30 2023-12-26T21:36:58.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.830+0000 7fb6fa7fc700 1 -- 172.21.15.131:0/74190540 <== mgr.4100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264445 (secure 0 0 0) 0x7fb70c0cac70 con 0x7fb6ec066f30 2023-12-26T21:36:58.834 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:58.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb6ec066f30 msgr2=0x7fb6ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb6ec066f30 0x7fb6ec0693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009d60 tx=0x7fb6fc008040 comp rx=0 tx=0).stop 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb70c1596a0 msgr2=0x7fb70c15b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb70c1596a0 0x7fb70c15b260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb70c089710 tx=0x7fb700017000 comp rx=0 tx=0).stop 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 shutdown_connections 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb70c159a70 0x7fb70c15b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb6ec066f30 0x7fb6ec0693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb70c0b0160 0x7fb70c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 --2- 172.21.15.131:0/74190540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb70c1596a0 0x7fb70c15b260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 >> 172.21.15.131:0/74190540 conn(0x7fb70c06ece0 msgr2=0x7fb70c0cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 shutdown_connections 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:58.834+0000 7fb710dff700 1 -- 172.21.15.131:0/74190540 wait complete. 2023-12-26T21:36:58.837 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:36:58.851 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-26T21:36:56.999658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381256,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593464,"statfs":{"total":773094113280,"available":770655707136,"internally_reserved":0,"allocated":3022848,"data_stored":1800548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.199253"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845569+0000","last_change":"2023-12-26T21:36:48.838074+0000","last_active":"2023-12-26T21:36:48.845569+0000","last_peered":"2023-12-26T21:36:48.845569+0000","last_clean":"2023-12-26T21:36:48.845569+0000","last_became_active":"2023-12-26T21:36:48.837495+0000","last_became_peered":"2023-12-26T21:36:48.837495+0000","last_unstale":"2023-12-26T21:36:48.845569+0000","last_undegraded":"2023-12-26T21:36:48.845569+0000","last_fullsized":"2023-12-26T21:36:48.845569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841349+0000","last_change":"2023-12-26T21:36:48.830149+0000","last_active":"2023-12-26T21:36:48.841349+0000","last_peered":"2023-12-26T21:36:48.841349+0000","last_clean":"2023-12-26T21:36:48.841349+0000","last_became_active":"2023-12-26T21:36:48.829771+0000","last_became_peered":"2023-12-26T21:36:48.829771+0000","last_unstale":"2023-12-26T21:36:48.841349+0000","last_undegraded":"2023-12-26T21:36:48.841349+0000","last_fullsized":"2023-12-26T21:36:48.841349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:59.578493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841417+0000","last_change":"2023-12-26T21:36:48.831124+0000","last_active":"2023-12-26T21:36:48.841417+0000","last_peered":"2023-12-26T21:36:48.841417+0000","last_clean":"2023-12-26T21:36:48.841417+0000","last_became_active":"2023-12-26T21:36:48.830817+0000","last_became_peered":"2023-12-26T21:36:48.830817+0000","last_unstale":"2023-12-26T21:36:48.841417+0000","last_undegraded":"2023-12-26T21:36:48.841417+0000","last_fullsized":"2023-12-26T21:36:48.841417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:47.183606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845126+0000","last_change":"2023-12-26T21:36:48.842229+0000","last_active":"2023-12-26T21:36:48.845126+0000","last_peered":"2023-12-26T21:36:48.845126+0000","last_clean":"2023-12-26T21:36:48.845126+0000","last_became_active":"2023-12-26T21:36:48.839269+0000","last_became_peered":"2023-12-26T21:36:48.839269+0000","last_unstale":"2023-12-26T21:36:48.845126+0000","last_undegraded":"2023-12-26T21:36:48.845126+0000","last_fullsized":"2023-12-26T21:36:48.845126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:01:49.095423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845558+0000","last_change":"2023-12-26T21:36:48.838113+0000","last_active":"2023-12-26T21:36:48.845558+0000","last_peered":"2023-12-26T21:36:48.845558+0000","last_clean":"2023-12-26T21:36:48.845558+0000","last_became_active":"2023-12-26T21:36:48.837882+0000","last_became_peered":"2023-12-26T21:36:48.837882+0000","last_unstale":"2023-12-26T21:36:48.845558+0000","last_undegraded":"2023-12-26T21:36:48.845558+0000","last_fullsized":"2023-12-26T21:36:48.845558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:40.692719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839762+0000","last_change":"2023-12-26T21:36:48.835731+0000","last_active":"2023-12-26T21:36:48.839762+0000","last_peered":"2023-12-26T21:36:48.839762+0000","last_clean":"2023-12-26T21:36:48.839762+0000","last_became_active":"2023-12-26T21:36:48.830672+0000","last_became_peered":"2023-12-26T21:36:48.830672+0000","last_unstale":"2023-12-26T21:36:48.839762+0000","last_undegraded":"2023-12-26T21:36:48.839762+0000","last_fullsized":"2023-12-26T21:36:48.839762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:06:58.960255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835557+0000","last_change":"2023-12-26T21:36:48.831504+0000","last_active":"2023-12-26T21:36:48.835557+0000","last_peered":"2023-12-26T21:36:48.835557+0000","last_clean":"2023-12-26T21:36:48.835557+0000","last_became_active":"2023-12-26T21:36:48.831173+0000","last_became_peered":"2023-12-26T21:36:48.831173+0000","last_unstale":"2023-12-26T21:36:48.835557+0000","last_undegraded":"2023-12-26T21:36:48.835557+0000","last_fullsized":"2023-12-26T21:36:48.835557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:37:13.353768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841944+0000","last_change":"2023-12-26T21:36:48.838249+0000","last_active":"2023-12-26T21:36:48.841944+0000","last_peered":"2023-12-26T21:36:48.841944+0000","last_clean":"2023-12-26T21:36:48.841944+0000","last_became_active":"2023-12-26T21:36:48.837983+0000","last_became_peered":"2023-12-26T21:36:48.837983+0000","last_unstale":"2023-12-26T21:36:48.841944+0000","last_undegraded":"2023-12-26T21:36:48.841944+0000","last_fullsized":"2023-12-26T21:36:48.841944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:56.706835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831690+0000","last_change":"2023-12-26T21:36:48.829328+0000","last_active":"2023-12-26T21:36:48.831690+0000","last_peered":"2023-12-26T21:36:48.831690+0000","last_clean":"2023-12-26T21:36:48.831690+0000","last_became_active":"2023-12-26T21:36:48.829033+0000","last_became_peered":"2023-12-26T21:36:48.829033+0000","last_unstale":"2023-12-26T21:36:48.831690+0000","last_undegraded":"2023-12-26T21:36:48.831690+0000","last_fullsized":"2023-12-26T21:36:48.831690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:31.058793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.837727+0000","last_change":"2023-12-26T21:36:48.834507+0000","last_active":"2023-12-26T21:36:48.837727+0000","last_peered":"2023-12-26T21:36:48.837727+0000","last_clean":"2023-12-26T21:36:48.837727+0000","last_became_active":"2023-12-26T21:36:48.833751+0000","last_became_peered":"2023-12-26T21:36:48.833751+0000","last_unstale":"2023-12-26T21:36:48.837727+0000","last_undegraded":"2023-12-26T21:36:48.837727+0000","last_fullsized":"2023-12-26T21:36:48.837727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:23.349206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846618+0000","last_change":"2023-12-26T21:36:48.839909+0000","last_active":"2023-12-26T21:36:48.846618+0000","last_peered":"2023-12-26T21:36:48.846618+0000","last_clean":"2023-12-26T21:36:48.846618+0000","last_became_active":"2023-12-26T21:36:48.839611+0000","last_became_peered":"2023-12-26T21:36:48.839611+0000","last_unstale":"2023-12-26T21:36:48.846618+0000","last_undegraded":"2023-12-26T21:36:48.846618+0000","last_fullsized":"2023-12-26T21:36:48.846618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:49.385696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841298+0000","last_change":"2023-12-26T21:36:48.836176+0000","last_active":"2023-12-26T21:36:48.841298+0000","last_peered":"2023-12-26T21:36:48.841298+0000","last_clean":"2023-12-26T21:36:48.841298+0000","last_became_active":"2023-12-26T21:36:48.835911+0000","last_became_peered":"2023-12-26T21:36:48.835911+0000","last_unstale":"2023-12-26T21:36:48.841298+0000","last_undegraded":"2023-12-26T21:36:48.841298+0000","last_fullsized":"2023-12-26T21:36:48.841298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:32.390648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828384+0000","last_change":"2023-12-26T21:36:48.825560+0000","last_active":"2023-12-26T21:36:48.828384+0000","last_peered":"2023-12-26T21:36:48.828384+0000","last_clean":"2023-12-26T21:36:48.828384+0000","last_became_active":"2023-12-26T21:36:48.825203+0000","last_became_peered":"2023-12-26T21:36:48.825203+0000","last_unstale":"2023-12-26T21:36:48.828384+0000","last_undegraded":"2023-12-26T21:36:48.828384+0000","last_fullsized":"2023-12-26T21:36:48.828384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:24.061374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828296+0000","last_change":"2023-12-26T21:36:48.823931+0000","last_active":"2023-12-26T21:36:48.828296+0000","last_peered":"2023-12-26T21:36:48.828296+0000","last_clean":"2023-12-26T21:36:48.828296+0000","last_became_active":"2023-12-26T21:36:48.823564+0000","last_became_peered":"2023-12-26T21:36:48.823564+0000","last_unstale":"2023-12-26T21:36:48.828296+0000","last_undegraded":"2023-12-26T21:36:48.828296+0000","last_fullsized":"2023-12-26T21:36:48.828296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:42.053931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833245+0000","last_change":"2023-12-26T21:36:48.823836+0000","last_active":"2023-12-26T21:36:48.833245+0000","last_peered":"2023-12-26T21:36:48.833245+0000","last_clean":"2023-12-26T21:36:48.833245+0000","last_became_active":"2023-12-26T21:36:48.823053+0000","last_became_peered":"2023-12-26T21:36:48.823053+0000","last_unstale":"2023-12-26T21:36:48.833245+0000","last_undegraded":"2023-12-26T21:36:48.833245+0000","last_fullsized":"2023-12-26T21:36:48.833245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:28.883031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838446+0000","last_change":"2023-12-26T21:36:48.829032+0000","last_active":"2023-12-26T21:36:48.838446+0000","last_peered":"2023-12-26T21:36:48.838446+0000","last_clean":"2023-12-26T21:36:48.838446+0000","last_became_active":"2023-12-26T21:36:48.826097+0000","last_became_peered":"2023-12-26T21:36:48.826097+0000","last_unstale":"2023-12-26T21:36:48.838446+0000","last_undegraded":"2023-12-26T21:36:48.838446+0000","last_fullsized":"2023-12-26T21:36:48.838446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:49.385696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844638+0000","last_change":"2023-12-26T21:36:48.832781+0000","last_active":"2023-12-26T21:36:48.844638+0000","last_peered":"2023-12-26T21:36:48.844638+0000","last_clean":"2023-12-26T21:36:48.844638+0000","last_became_active":"2023-12-26T21:36:48.832414+0000","last_became_peered":"2023-12-26T21:36:48.832414+0000","last_unstale":"2023-12-26T21:36:48.844638+0000","last_undegraded":"2023-12-26T21:36:48.844638+0000","last_fullsized":"2023-12-26T21:36:48.844638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:05:36.844680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.825827+0000","last_change":"2023-12-26T21:36:48.817787+0000","last_active":"2023-12-26T21:36:48.825827+0000","last_peered":"2023-12-26T21:36:48.825827+0000","last_clean":"2023-12-26T21:36:48.825827+0000","last_became_active":"2023-12-26T21:36:48.817432+0000","last_became_peered":"2023-12-26T21:36:48.817432+0000","last_unstale":"2023-12-26T21:36:48.825827+0000","last_undegraded":"2023-12-26T21:36:48.825827+0000","last_fullsized":"2023-12-26T21:36:48.825827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:51.402901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844696+0000","last_change":"2023-12-26T21:36:48.834659+0000","last_active":"2023-12-26T21:36:48.844696+0000","last_peered":"2023-12-26T21:36:48.844696+0000","last_clean":"2023-12-26T21:36:48.844696+0000","last_became_active":"2023-12-26T21:36:48.833967+0000","last_became_peered":"2023-12-26T21:36:48.833967+0000","last_unstale":"2023-12-26T21:36:48.844696+0000","last_undegraded":"2023-12-26T21:36:48.844696+0000","last_fullsized":"2023-12-26T21:36:48.844696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:12:41.152256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839315+0000","last_change":"2023-12-26T21:36:47.830031+0000","last_active":"2023-12-26T21:36:48.839315+0000","last_peered":"2023-12-26T21:36:48.839315+0000","last_clean":"2023-12-26T21:36:48.839315+0000","last_became_active":"2023-12-26T21:36:47.829681+0000","last_became_peered":"2023-12-26T21:36:47.829681+0000","last_unstale":"2023-12-26T21:36:48.839315+0000","last_undegraded":"2023-12-26T21:36:48.839315+0000","last_fullsized":"2023-12-26T21:36:48.839315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:25.200664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841270+0000","last_change":"2023-12-26T21:36:48.827382+0000","last_active":"2023-12-26T21:36:48.841270+0000","last_peered":"2023-12-26T21:36:48.841270+0000","last_clean":"2023-12-26T21:36:48.841270+0000","last_became_active":"2023-12-26T21:36:48.825125+0000","last_became_peered":"2023-12-26T21:36:48.825125+0000","last_unstale":"2023-12-26T21:36:48.841270+0000","last_undegraded":"2023-12-26T21:36:48.841270+0000","last_fullsized":"2023-12-26T21:36:48.841270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:02:27.671653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843042+0000","last_change":"2023-12-26T21:36:47.825862+0000","last_active":"2023-12-26T21:36:48.843042+0000","last_peered":"2023-12-26T21:36:48.843042+0000","last_clean":"2023-12-26T21:36:48.843042+0000","last_became_active":"2023-12-26T21:36:47.825603+0000","last_became_peered":"2023-12-26T21:36:47.825603+0000","last_unstale":"2023-12-26T21:36:48.843042+0000","last_undegraded":"2023-12-26T21:36:48.843042+0000","last_fullsized":"2023-12-26T21:36:48.843042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:18:58.593197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833411+0000","last_change":"2023-12-26T21:36:48.819527+0000","last_active":"2023-12-26T21:36:48.833411+0000","last_peered":"2023-12-26T21:36:48.833411+0000","last_clean":"2023-12-26T21:36:48.833411+0000","last_became_active":"2023-12-26T21:36:48.819208+0000","last_became_peered":"2023-12-26T21:36:48.819208+0000","last_unstale":"2023-12-26T21:36:48.833411+0000","last_undegraded":"2023-12-26T21:36:48.833411+0000","last_fullsized":"2023-12-26T21:36:48.833411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:32.390648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841778+0000","last_change":"2023-12-26T21:36:47.824557+0000","last_active":"2023-12-26T21:36:48.841778+0000","last_peered":"2023-12-26T21:36:48.841778+0000","last_clean":"2023-12-26T21:36:48.841778+0000","last_became_active":"2023-12-26T21:36:47.824200+0000","last_became_peered":"2023-12-26T21:36:47.824200+0000","last_unstale":"2023-12-26T21:36:48.841778+0000","last_undegraded":"2023-12-26T21:36:48.841778+0000","last_fullsized":"2023-12-26T21:36:48.841778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:13.097046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.827346+0000","last_change":"2023-12-26T21:36:48.823425+0000","last_active":"2023-12-26T21:36:48.827346+0000","last_peered":"2023-12-26T21:36:48.827346+0000","last_clean":"2023-12-26T21:36:48.827346+0000","last_became_active":"2023-12-26T21:36:48.823148+0000","last_became_peered":"2023-12-26T21:36:48.823148+0000","last_unstale":"2023-12-26T21:36:48.827346+0000","last_undegraded":"2023-12-26T21:36:48.827346+0000","last_fullsized":"2023-12-26T21:36:48.827346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:35.691705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841375+0000","last_change":"2023-12-26T21:36:47.826954+0000","last_active":"2023-12-26T21:36:48.841375+0000","last_peered":"2023-12-26T21:36:48.841375+0000","last_clean":"2023-12-26T21:36:48.841375+0000","last_became_active":"2023-12-26T21:36:47.826691+0000","last_became_peered":"2023-12-26T21:36:47.826691+0000","last_unstale":"2023-12-26T21:36:48.841375+0000","last_undegraded":"2023-12-26T21:36:48.841375+0000","last_fullsized":"2023-12-26T21:36:48.841375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:24.145739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843898+0000","last_change":"2023-12-26T21:36:48.830893+0000","last_active":"2023-12-26T21:36:48.843898+0000","last_peered":"2023-12-26T21:36:48.843898+0000","last_clean":"2023-12-26T21:36:48.843898+0000","last_became_active":"2023-12-26T21:36:48.830553+0000","last_became_peered":"2023-12-26T21:36:48.830553+0000","last_unstale":"2023-12-26T21:36:48.843898+0000","last_undegraded":"2023-12-26T21:36:48.843898+0000","last_fullsized":"2023-12-26T21:36:48.843898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:40:07.698471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841849+0000","last_change":"2023-12-26T21:36:47.828216+0000","last_active":"2023-12-26T21:36:48.841849+0000","last_peered":"2023-12-26T21:36:48.841849+0000","last_clean":"2023-12-26T21:36:48.841849+0000","last_became_active":"2023-12-26T21:36:47.827950+0000","last_became_peered":"2023-12-26T21:36:47.827950+0000","last_unstale":"2023-12-26T21:36:48.841849+0000","last_undegraded":"2023-12-26T21:36:48.841849+0000","last_fullsized":"2023-12-26T21:36:48.841849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:23.069341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828328+0000","last_change":"2023-12-26T21:36:48.820490+0000","last_active":"2023-12-26T21:36:48.828328+0000","last_peered":"2023-12-26T21:36:48.828328+0000","last_clean":"2023-12-26T21:36:48.828328+0000","last_became_active":"2023-12-26T21:36:48.820298+0000","last_became_peered":"2023-12-26T21:36:48.820298+0000","last_unstale":"2023-12-26T21:36:48.828328+0000","last_undegraded":"2023-12-26T21:36:48.828328+0000","last_fullsized":"2023-12-26T21:36:48.828328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:25.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.830175+0000","last_change":"2023-12-26T21:36:47.830245+0000","last_active":"2023-12-26T21:36:48.830175+0000","last_peered":"2023-12-26T21:36:48.830175+0000","last_clean":"2023-12-26T21:36:48.830175+0000","last_became_active":"2023-12-26T21:36:47.829970+0000","last_became_peered":"2023-12-26T21:36:47.829970+0000","last_unstale":"2023-12-26T21:36:48.830175+0000","last_undegraded":"2023-12-26T21:36:48.830175+0000","last_fullsized":"2023-12-26T21:36:48.830175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:13.329724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843818+0000","last_change":"2023-12-26T21:36:48.839949+0000","last_active":"2023-12-26T21:36:48.843818+0000","last_peered":"2023-12-26T21:36:48.843818+0000","last_clean":"2023-12-26T21:36:48.843818+0000","last_became_active":"2023-12-26T21:36:48.839655+0000","last_became_peered":"2023-12-26T21:36:48.839655+0000","last_unstale":"2023-12-26T21:36:48.843818+0000","last_undegraded":"2023-12-26T21:36:48.843818+0000","last_fullsized":"2023-12-26T21:36:48.843818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:43:57.839515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.809369+0000","last_change":"2023-12-26T21:36:47.828451+0000","last_active":"2023-12-26T21:36:48.809369+0000","last_peered":"2023-12-26T21:36:48.809369+0000","last_clean":"2023-12-26T21:36:48.809369+0000","last_became_active":"2023-12-26T21:36:47.828158+0000","last_became_peered":"2023-12-26T21:36:47.828158+0000","last_unstale":"2023-12-26T21:36:48.809369+0000","last_undegraded":"2023-12-26T21:36:48.809369+0000","last_fullsized":"2023-12-26T21:36:48.809369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:23.069341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843628+0000","last_change":"2023-12-26T21:36:48.840894+0000","last_active":"2023-12-26T21:36:48.843628+0000","last_peered":"2023-12-26T21:36:48.843628+0000","last_clean":"2023-12-26T21:36:48.843628+0000","last_became_active":"2023-12-26T21:36:48.840284+0000","last_became_peered":"2023-12-26T21:36:48.840284+0000","last_unstale":"2023-12-26T21:36:48.843628+0000","last_undegraded":"2023-12-26T21:36:48.843628+0000","last_fullsized":"2023-12-26T21:36:48.843628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:23.349206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816543+0000","last_change":"2023-12-26T21:36:47.821233+0000","last_active":"2023-12-26T21:36:48.816543+0000","last_peered":"2023-12-26T21:36:48.816543+0000","last_clean":"2023-12-26T21:36:48.816543+0000","last_became_active":"2023-12-26T21:36:47.820129+0000","last_became_peered":"2023-12-26T21:36:47.820129+0000","last_unstale":"2023-12-26T21:36:48.816543+0000","last_undegraded":"2023-12-26T21:36:48.816543+0000","last_fullsized":"2023-12-26T21:36:48.816543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:30.052367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838583+0000","last_change":"2023-12-26T21:36:48.829638+0000","last_active":"2023-12-26T21:36:48.838583+0000","last_peered":"2023-12-26T21:36:48.838583+0000","last_clean":"2023-12-26T21:36:48.838583+0000","last_became_active":"2023-12-26T21:36:48.829154+0000","last_became_peered":"2023-12-26T21:36:48.829154+0000","last_unstale":"2023-12-26T21:36:48.838583+0000","last_undegraded":"2023-12-26T21:36:48.838583+0000","last_fullsized":"2023-12-26T21:36:48.838583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:45:55.685017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.803440+0000","last_change":"2023-12-26T21:36:47.818993+0000","last_active":"2023-12-26T21:36:48.803440+0000","last_peered":"2023-12-26T21:36:48.803440+0000","last_clean":"2023-12-26T21:36:48.803440+0000","last_became_active":"2023-12-26T21:36:47.818731+0000","last_became_peered":"2023-12-26T21:36:47.818731+0000","last_unstale":"2023-12-26T21:36:48.803440+0000","last_undegraded":"2023-12-26T21:36:48.803440+0000","last_fullsized":"2023-12-26T21:36:48.803440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:14.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.840833+0000","last_change":"2023-12-26T21:36:48.837014+0000","last_active":"2023-12-26T21:36:48.840833+0000","last_peered":"2023-12-26T21:36:48.840833+0000","last_clean":"2023-12-26T21:36:48.840833+0000","last_became_active":"2023-12-26T21:36:48.830649+0000","last_became_peered":"2023-12-26T21:36:48.830649+0000","last_unstale":"2023-12-26T21:36:48.840833+0000","last_undegraded":"2023-12-26T21:36:48.840833+0000","last_fullsized":"2023-12-26T21:36:48.840833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:07.755765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829625+0000","last_change":"2023-12-26T21:36:47.827036+0000","last_active":"2023-12-26T21:36:48.829625+0000","last_peered":"2023-12-26T21:36:48.829625+0000","last_clean":"2023-12-26T21:36:48.829625+0000","last_became_active":"2023-12-26T21:36:47.826746+0000","last_became_peered":"2023-12-26T21:36:47.826746+0000","last_unstale":"2023-12-26T21:36:48.829625+0000","last_undegraded":"2023-12-26T21:36:48.829625+0000","last_fullsized":"2023-12-26T21:36:48.829625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:28.026360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841896+0000","last_change":"2023-12-26T21:36:48.837290+0000","last_active":"2023-12-26T21:36:48.841896+0000","last_peered":"2023-12-26T21:36:48.841896+0000","last_clean":"2023-12-26T21:36:48.841896+0000","last_became_active":"2023-12-26T21:36:48.836991+0000","last_became_peered":"2023-12-26T21:36:48.836991+0000","last_unstale":"2023-12-26T21:36:48.841896+0000","last_undegraded":"2023-12-26T21:36:48.841896+0000","last_fullsized":"2023-12-26T21:36:48.841896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835423+0000","last_change":"2023-12-26T21:36:47.823376+0000","last_active":"2023-12-26T21:36:48.835423+0000","last_peered":"2023-12-26T21:36:48.835423+0000","last_clean":"2023-12-26T21:36:48.835423+0000","last_became_active":"2023-12-26T21:36:47.822986+0000","last_became_peered":"2023-12-26T21:36:47.822986+0000","last_unstale":"2023-12-26T21:36:48.835423+0000","last_undegraded":"2023-12-26T21:36:48.835423+0000","last_fullsized":"2023-12-26T21:36:48.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:45:26.314194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829780+0000","last_change":"2023-12-26T21:36:48.825556+0000","last_active":"2023-12-26T21:36:48.829780+0000","last_peered":"2023-12-26T21:36:48.829780+0000","last_clean":"2023-12-26T21:36:48.829780+0000","last_became_active":"2023-12-26T21:36:48.825233+0000","last_became_peered":"2023-12-26T21:36:48.825233+0000","last_unstale":"2023-12-26T21:36:48.829780+0000","last_undegraded":"2023-12-26T21:36:48.829780+0000","last_fullsized":"2023-12-26T21:36:48.829780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:40.692719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.848461+0000","last_change":"2023-12-26T21:36:47.821098+0000","last_active":"2023-12-26T21:36:48.848461+0000","last_peered":"2023-12-26T21:36:48.848461+0000","last_clean":"2023-12-26T21:36:48.848461+0000","last_became_active":"2023-12-26T21:36:47.820639+0000","last_became_peered":"2023-12-26T21:36:47.820639+0000","last_unstale":"2023-12-26T21:36:48.848461+0000","last_undegraded":"2023-12-26T21:36:48.848461+0000","last_fullsized":"2023-12-26T21:36:48.848461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:26.330467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833737+0000","last_change":"2023-12-26T21:36:48.828635+0000","last_active":"2023-12-26T21:36:48.833737+0000","last_peered":"2023-12-26T21:36:48.833737+0000","last_clean":"2023-12-26T21:36:48.833737+0000","last_became_active":"2023-12-26T21:36:48.828382+0000","last_became_peered":"2023-12-26T21:36:48.828382+0000","last_unstale":"2023-12-26T21:36:48.833737+0000","last_undegraded":"2023-12-26T21:36:48.833737+0000","last_fullsized":"2023-12-26T21:36:48.833737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:59.578493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835877+0000","last_change":"2023-12-26T21:36:47.830348+0000","last_active":"2023-12-26T21:36:48.835877+0000","last_peered":"2023-12-26T21:36:48.835877+0000","last_clean":"2023-12-26T21:36:48.835877+0000","last_became_active":"2023-12-26T21:36:47.830053+0000","last_became_peered":"2023-12-26T21:36:47.830053+0000","last_unstale":"2023-12-26T21:36:48.835877+0000","last_undegraded":"2023-12-26T21:36:48.835877+0000","last_fullsized":"2023-12-26T21:36:48.835877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:26.330467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.832805+0000","last_change":"2023-12-26T21:36:48.830118+0000","last_active":"2023-12-26T21:36:48.832805+0000","last_peered":"2023-12-26T21:36:48.832805+0000","last_clean":"2023-12-26T21:36:48.832805+0000","last_became_active":"2023-12-26T21:36:48.829843+0000","last_became_peered":"2023-12-26T21:36:48.829843+0000","last_unstale":"2023-12-26T21:36:48.832805+0000","last_undegraded":"2023-12-26T21:36:48.832805+0000","last_fullsized":"2023-12-26T21:36:48.832805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817442+0000","last_change":"2023-12-26T21:36:47.822624+0000","last_active":"2023-12-26T21:36:48.817442+0000","last_peered":"2023-12-26T21:36:48.817442+0000","last_clean":"2023-12-26T21:36:48.817442+0000","last_became_active":"2023-12-26T21:36:47.822048+0000","last_became_peered":"2023-12-26T21:36:47.822048+0000","last_unstale":"2023-12-26T21:36:48.817442+0000","last_undegraded":"2023-12-26T21:36:48.817442+0000","last_fullsized":"2023-12-26T21:36:48.817442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:24.145739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846368+0000","last_change":"2023-12-26T21:36:48.837022+0000","last_active":"2023-12-26T21:36:48.846368+0000","last_peered":"2023-12-26T21:36:48.846368+0000","last_clean":"2023-12-26T21:36:48.846368+0000","last_became_active":"2023-12-26T21:36:48.836736+0000","last_became_peered":"2023-12-26T21:36:48.836736+0000","last_unstale":"2023-12-26T21:36:48.846368+0000","last_undegraded":"2023-12-26T21:36:48.846368+0000","last_fullsized":"2023-12-26T21:36:48.846368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:57.196069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826918+0000","last_change":"2023-12-26T21:36:47.829979+0000","last_active":"2023-12-26T21:36:48.826918+0000","last_peered":"2023-12-26T21:36:48.826918+0000","last_clean":"2023-12-26T21:36:48.826918+0000","last_became_active":"2023-12-26T21:36:47.829685+0000","last_became_peered":"2023-12-26T21:36:47.829685+0000","last_unstale":"2023-12-26T21:36:48.826918+0000","last_undegraded":"2023-12-26T21:36:48.826918+0000","last_fullsized":"2023-12-26T21:36:48.826918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831660+0000","last_change":"2023-12-26T21:36:48.828008+0000","last_active":"2023-12-26T21:36:48.831660+0000","last_peered":"2023-12-26T21:36:48.831660+0000","last_clean":"2023-12-26T21:36:48.831660+0000","last_became_active":"2023-12-26T21:36:48.827743+0000","last_became_peered":"2023-12-26T21:36:48.827743+0000","last_unstale":"2023-12-26T21:36:48.831660+0000","last_undegraded":"2023-12-26T21:36:48.831660+0000","last_fullsized":"2023-12-26T21:36:48.831660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:14.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839801+0000","last_change":"2023-12-26T21:36:47.815814+0000","last_active":"2023-12-26T21:36:48.839801+0000","last_peered":"2023-12-26T21:36:48.839801+0000","last_clean":"2023-12-26T21:36:48.839801+0000","last_became_active":"2023-12-26T21:36:47.815480+0000","last_became_peered":"2023-12-26T21:36:47.815480+0000","last_unstale":"2023-12-26T21:36:48.839801+0000","last_undegraded":"2023-12-26T21:36:48.839801+0000","last_fullsized":"2023-12-26T21:36:48.839801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:18:28.266978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843897+0000","last_change":"2023-12-26T21:36:48.833364+0000","last_active":"2023-12-26T21:36:48.843897+0000","last_peered":"2023-12-26T21:36:48.843897+0000","last_clean":"2023-12-26T21:36:48.843897+0000","last_became_active":"2023-12-26T21:36:48.833087+0000","last_became_peered":"2023-12-26T21:36:48.833087+0000","last_unstale":"2023-12-26T21:36:48.843897+0000","last_undegraded":"2023-12-26T21:36:48.843897+0000","last_fullsized":"2023-12-26T21:36:48.843897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:05:49.639640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846496+0000","last_change":"2023-12-26T21:36:47.823499+0000","last_active":"2023-12-26T21:36:48.846496+0000","last_peered":"2023-12-26T21:36:48.846496+0000","last_clean":"2023-12-26T21:36:48.846496+0000","last_became_active":"2023-12-26T21:36:47.819416+0000","last_became_peered":"2023-12-26T21:36:47.819416+0000","last_unstale":"2023-12-26T21:36:48.846496+0000","last_undegraded":"2023-12-26T21:36:48.846496+0000","last_fullsized":"2023-12-26T21:36:48.846496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:13.329724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839537+0000","last_change":"2023-12-26T21:36:48.824260+0000","last_active":"2023-12-26T21:36:48.839537+0000","last_peered":"2023-12-26T21:36:48.839537+0000","last_clean":"2023-12-26T21:36:48.839537+0000","last_became_active":"2023-12-26T21:36:48.824000+0000","last_became_peered":"2023-12-26T21:36:48.824000+0000","last_unstale":"2023-12-26T21:36:48.839537+0000","last_undegraded":"2023-12-26T21:36:48.839537+0000","last_fullsized":"2023-12-26T21:36:48.839537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:24.151016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.812438+0000","last_change":"2023-12-26T21:36:47.829031+0000","last_active":"2023-12-26T21:36:48.812438+0000","last_peered":"2023-12-26T21:36:48.812438+0000","last_clean":"2023-12-26T21:36:48.812438+0000","last_became_active":"2023-12-26T21:36:47.828711+0000","last_became_peered":"2023-12-26T21:36:47.828711+0000","last_unstale":"2023-12-26T21:36:48.812438+0000","last_undegraded":"2023-12-26T21:36:48.812438+0000","last_fullsized":"2023-12-26T21:36:48.812438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.822529+0000","last_change":"2023-12-26T21:36:48.817001+0000","last_active":"2023-12-26T21:36:48.822529+0000","last_peered":"2023-12-26T21:36:48.822529+0000","last_clean":"2023-12-26T21:36:48.822529+0000","last_became_active":"2023-12-26T21:36:48.816746+0000","last_became_peered":"2023-12-26T21:36:48.816746+0000","last_unstale":"2023-12-26T21:36:48.822529+0000","last_undegraded":"2023-12-26T21:36:48.822529+0000","last_fullsized":"2023-12-26T21:36:48.822529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:56.619479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817332+0000","last_change":"2023-12-26T21:36:47.828501+0000","last_active":"2023-12-26T21:36:48.817332+0000","last_peered":"2023-12-26T21:36:48.817332+0000","last_clean":"2023-12-26T21:36:48.817332+0000","last_became_active":"2023-12-26T21:36:47.828257+0000","last_became_peered":"2023-12-26T21:36:47.828257+0000","last_unstale":"2023-12-26T21:36:48.817332+0000","last_undegraded":"2023-12-26T21:36:48.817332+0000","last_fullsized":"2023-12-26T21:36:48.817332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:17.677425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.837624+0000","last_change":"2023-12-26T21:36:48.834334+0000","last_active":"2023-12-26T21:36:48.837624+0000","last_peered":"2023-12-26T21:36:48.837624+0000","last_clean":"2023-12-26T21:36:48.837624+0000","last_became_active":"2023-12-26T21:36:48.834070+0000","last_became_peered":"2023-12-26T21:36:48.834070+0000","last_unstale":"2023-12-26T21:36:48.837624+0000","last_undegraded":"2023-12-26T21:36:48.837624+0000","last_fullsized":"2023-12-26T21:36:48.837624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:21:25.931065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843647+0000","last_change":"2023-12-26T21:36:48.841064+0000","last_active":"2023-12-26T21:36:48.843647+0000","last_peered":"2023-12-26T21:36:48.843647+0000","last_clean":"2023-12-26T21:36:48.843647+0000","last_became_active":"2023-12-26T21:36:48.840654+0000","last_became_peered":"2023-12-26T21:36:48.840654+0000","last_unstale":"2023-12-26T21:36:48.843647+0000","last_undegraded":"2023-12-26T21:36:48.843647+0000","last_fullsized":"2023-12-26T21:36:48.843647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.804661+0000","last_change":"2023-12-26T21:36:47.822491+0000","last_active":"2023-12-26T21:36:48.804661+0000","last_peered":"2023-12-26T21:36:48.804661+0000","last_clean":"2023-12-26T21:36:48.804661+0000","last_became_active":"2023-12-26T21:36:47.822150+0000","last_became_peered":"2023-12-26T21:36:47.822150+0000","last_unstale":"2023-12-26T21:36:48.804661+0000","last_undegraded":"2023-12-26T21:36:48.804661+0000","last_fullsized":"2023-12-26T21:36:48.804661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:08:16.186961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.834826+0000","last_change":"2023-12-26T21:36:48.828798+0000","last_active":"2023-12-26T21:36:48.834826+0000","last_peered":"2023-12-26T21:36:48.834826+0000","last_clean":"2023-12-26T21:36:48.834826+0000","last_became_active":"2023-12-26T21:36:48.828093+0000","last_became_peered":"2023-12-26T21:36:48.828093+0000","last_unstale":"2023-12-26T21:36:48.834826+0000","last_undegraded":"2023-12-26T21:36:48.834826+0000","last_fullsized":"2023-12-26T21:36:48.834826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:19:55.017056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817551+0000","last_change":"2023-12-26T21:36:47.825144+0000","last_active":"2023-12-26T21:36:48.817551+0000","last_peered":"2023-12-26T21:36:48.817551+0000","last_clean":"2023-12-26T21:36:48.817551+0000","last_became_active":"2023-12-26T21:36:47.824896+0000","last_became_peered":"2023-12-26T21:36:47.824896+0000","last_unstale":"2023-12-26T21:36:48.817551+0000","last_undegraded":"2023-12-26T21:36:48.817551+0000","last_fullsized":"2023-12-26T21:36:48.817551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:03:48.097670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.803182+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:36:48.803182+0000","last_peered":"2023-12-26T21:36:48.803182+0000","last_clean":"2023-12-26T21:36:48.803182+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:36:48.803182+0000","last_undegraded":"2023-12-26T21:36:48.803182+0000","last_fullsized":"2023-12-26T21:36:48.803182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.818212+0000","last_change":"2023-12-26T21:36:48.816469+0000","last_active":"2023-12-26T21:36:48.818212+0000","last_peered":"2023-12-26T21:36:48.818212+0000","last_clean":"2023-12-26T21:36:48.818212+0000","last_became_active":"2023-12-26T21:36:48.816190+0000","last_became_peered":"2023-12-26T21:36:48.816190+0000","last_unstale":"2023-12-26T21:36:48.818212+0000","last_undegraded":"2023-12-26T21:36:48.818212+0000","last_fullsized":"2023-12-26T21:36:48.818212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:35.691705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816437+0000","last_change":"2023-12-26T21:36:47.822572+0000","last_active":"2023-12-26T21:36:48.816437+0000","last_peered":"2023-12-26T21:36:48.816437+0000","last_clean":"2023-12-26T21:36:48.816437+0000","last_became_active":"2023-12-26T21:36:47.822214+0000","last_became_peered":"2023-12-26T21:36:47.822214+0000","last_unstale":"2023-12-26T21:36:48.816437+0000","last_undegraded":"2023-12-26T21:36:48.816437+0000","last_fullsized":"2023-12-26T21:36:48.816437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:34.685280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816507+0000","last_change":"2023-12-26T21:36:47.826782+0000","last_active":"2023-12-26T21:36:48.816507+0000","last_peered":"2023-12-26T21:36:48.816507+0000","last_clean":"2023-12-26T21:36:48.816507+0000","last_became_active":"2023-12-26T21:36:47.825248+0000","last_became_peered":"2023-12-26T21:36:47.825248+0000","last_unstale":"2023-12-26T21:36:48.816507+0000","last_undegraded":"2023-12-26T21:36:48.816507+0000","last_fullsized":"2023-12-26T21:36:48.816507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:28.026360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831901+0000","last_change":"2023-12-26T21:36:48.827788+0000","last_active":"2023-12-26T21:36:48.831901+0000","last_peered":"2023-12-26T21:36:48.831901+0000","last_clean":"2023-12-26T21:36:48.831901+0000","last_became_active":"2023-12-26T21:36:48.827487+0000","last_became_peered":"2023-12-26T21:36:48.827487+0000","last_unstale":"2023-12-26T21:36:48.831901+0000","last_undegraded":"2023-12-26T21:36:48.831901+0000","last_fullsized":"2023-12-26T21:36:48.831901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:22.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-26T21:36:49.819679+0000","last_change":"2023-12-26T21:36:47.830347+0000","last_active":"2023-12-26T21:36:49.819679+0000","last_peered":"2023-12-26T21:36:49.819679+0000","last_clean":"2023-12-26T21:36:49.819679+0000","last_became_active":"2023-12-26T21:36:47.830037+0000","last_became_peered":"2023-12-26T21:36:47.830037+0000","last_unstale":"2023-12-26T21:36:49.819679+0000","last_undegraded":"2023-12-26T21:36:49.819679+0000","last_fullsized":"2023-12-26T21:36:49.819679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:55.613054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838484+0000","last_change":"2023-12-26T21:36:48.829585+0000","last_active":"2023-12-26T21:36:48.838484+0000","last_peered":"2023-12-26T21:36:48.838484+0000","last_clean":"2023-12-26T21:36:48.838484+0000","last_became_active":"2023-12-26T21:36:48.829170+0000","last_became_peered":"2023-12-26T21:36:48.829170+0000","last_unstale":"2023-12-26T21:36:48.838484+0000","last_undegraded":"2023-12-26T21:36:48.838484+0000","last_fullsized":"2023-12-26T21:36:48.838484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:25:31.563553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844919+0000","last_change":"2023-12-26T21:36:47.832588+0000","last_active":"2023-12-26T21:36:48.844919+0000","last_peered":"2023-12-26T21:36:48.844919+0000","last_clean":"2023-12-26T21:36:48.844919+0000","last_became_active":"2023-12-26T21:36:47.832310+0000","last_became_peered":"2023-12-26T21:36:47.832310+0000","last_unstale":"2023-12-26T21:36:48.844919+0000","last_undegraded":"2023-12-26T21:36:48.844919+0000","last_fullsized":"2023-12-26T21:36:48.844919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:22:54.126845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839479+0000","last_change":"2023-12-26T21:36:48.830882+0000","last_active":"2023-12-26T21:36:48.839479+0000","last_peered":"2023-12-26T21:36:48.839479+0000","last_clean":"2023-12-26T21:36:48.839479+0000","last_became_active":"2023-12-26T21:36:48.830587+0000","last_became_peered":"2023-12-26T21:36:48.830587+0000","last_unstale":"2023-12-26T21:36:48.839479+0000","last_undegraded":"2023-12-26T21:36:48.839479+0000","last_fullsized":"2023-12-26T21:36:48.839479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:54.746974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826954+0000","last_change":"2023-12-26T21:36:47.826141+0000","last_active":"2023-12-26T21:36:48.826954+0000","last_peered":"2023-12-26T21:36:48.826954+0000","last_clean":"2023-12-26T21:36:48.826954+0000","last_became_active":"2023-12-26T21:36:47.825876+0000","last_became_peered":"2023-12-26T21:36:47.825876+0000","last_unstale":"2023-12-26T21:36:48.826954+0000","last_undegraded":"2023-12-26T21:36:48.826954+0000","last_fullsized":"2023-12-26T21:36:48.826954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:12.184779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843719+0000","last_change":"2023-12-26T21:36:48.829405+0000","last_active":"2023-12-26T21:36:48.843719+0000","last_peered":"2023-12-26T21:36:48.843719+0000","last_clean":"2023-12-26T21:36:48.843719+0000","last_became_active":"2023-12-26T21:36:48.829169+0000","last_became_peered":"2023-12-26T21:36:48.829169+0000","last_unstale":"2023-12-26T21:36:48.843719+0000","last_undegraded":"2023-12-26T21:36:48.843719+0000","last_fullsized":"2023-12-26T21:36:48.843719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:42:15.750967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.819202+0000","last_change":"2023-12-26T21:36:47.830820+0000","last_active":"2023-12-26T21:36:48.819202+0000","last_peered":"2023-12-26T21:36:48.819202+0000","last_clean":"2023-12-26T21:36:48.819202+0000","last_became_active":"2023-12-26T21:36:47.830514+0000","last_became_peered":"2023-12-26T21:36:47.830514+0000","last_unstale":"2023-12-26T21:36:48.819202+0000","last_undegraded":"2023-12-26T21:36:48.819202+0000","last_fullsized":"2023-12-26T21:36:48.819202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:53.472268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839858+0000","last_change":"2023-12-26T21:36:48.832982+0000","last_active":"2023-12-26T21:36:48.839858+0000","last_peered":"2023-12-26T21:36:48.839858+0000","last_clean":"2023-12-26T21:36:48.839858+0000","last_became_active":"2023-12-26T21:36:48.831969+0000","last_became_peered":"2023-12-26T21:36:48.831969+0000","last_unstale":"2023-12-26T21:36:48.839858+0000","last_undegraded":"2023-12-26T21:36:48.839858+0000","last_fullsized":"2023-12-26T21:36:48.839858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:54.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.812495+0000","last_change":"2023-12-26T21:36:47.826837+0000","last_active":"2023-12-26T21:36:48.812495+0000","last_peered":"2023-12-26T21:36:48.812495+0000","last_clean":"2023-12-26T21:36:48.812495+0000","last_became_active":"2023-12-26T21:36:47.826479+0000","last_became_peered":"2023-12-26T21:36:47.826479+0000","last_unstale":"2023-12-26T21:36:48.812495+0000","last_undegraded":"2023-12-26T21:36:48.812495+0000","last_fullsized":"2023-12-26T21:36:48.812495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:05.573200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817414+0000","last_change":"2023-12-26T21:36:48.813654+0000","last_active":"2023-12-26T21:36:48.817414+0000","last_peered":"2023-12-26T21:36:48.817414+0000","last_clean":"2023-12-26T21:36:48.817414+0000","last_became_active":"2023-12-26T21:36:48.813344+0000","last_became_peered":"2023-12-26T21:36:48.813344+0000","last_unstale":"2023-12-26T21:36:48.817414+0000","last_undegraded":"2023-12-26T21:36:48.817414+0000","last_fullsized":"2023-12-26T21:36:48.817414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:54.478693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839092+0000","last_change":"2023-12-26T21:36:47.820913+0000","last_active":"2023-12-26T21:36:48.839092+0000","last_peered":"2023-12-26T21:36:48.839092+0000","last_clean":"2023-12-26T21:36:48.839092+0000","last_became_active":"2023-12-26T21:36:47.820617+0000","last_became_peered":"2023-12-26T21:36:47.820617+0000","last_unstale":"2023-12-26T21:36:48.839092+0000","last_undegraded":"2023-12-26T21:36:48.839092+0000","last_fullsized":"2023-12-26T21:36:48.839092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:41.047506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829801+0000","last_change":"2023-12-26T21:36:48.826060+0000","last_active":"2023-12-26T21:36:48.829801+0000","last_peered":"2023-12-26T21:36:48.829801+0000","last_clean":"2023-12-26T21:36:48.829801+0000","last_became_active":"2023-12-26T21:36:48.825817+0000","last_became_peered":"2023-12-26T21:36:48.825817+0000","last_unstale":"2023-12-26T21:36:48.829801+0000","last_undegraded":"2023-12-26T21:36:48.829801+0000","last_fullsized":"2023-12-26T21:36:48.829801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.809730+0000","last_change":"2023-12-26T21:36:47.827562+0000","last_active":"2023-12-26T21:36:48.809730+0000","last_peered":"2023-12-26T21:36:48.809730+0000","last_clean":"2023-12-26T21:36:48.809730+0000","last_became_active":"2023-12-26T21:36:47.827284+0000","last_became_peered":"2023-12-26T21:36:47.827284+0000","last_unstale":"2023-12-26T21:36:48.809730+0000","last_undegraded":"2023-12-26T21:36:48.809730+0000","last_fullsized":"2023-12-26T21:36:48.809730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843878+0000","last_change":"2023-12-26T21:36:48.829818+0000","last_active":"2023-12-26T21:36:48.843878+0000","last_peered":"2023-12-26T21:36:48.843878+0000","last_clean":"2023-12-26T21:36:48.843878+0000","last_became_active":"2023-12-26T21:36:48.829087+0000","last_became_peered":"2023-12-26T21:36:48.829087+0000","last_unstale":"2023-12-26T21:36:48.843878+0000","last_undegraded":"2023-12-26T21:36:48.843878+0000","last_fullsized":"2023-12-26T21:36:48.843878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:37:13.353768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.842747+0000","last_change":"2023-12-26T21:36:47.815095+0000","last_active":"2023-12-26T21:36:48.842747+0000","last_peered":"2023-12-26T21:36:48.842747+0000","last_clean":"2023-12-26T21:36:48.842747+0000","last_became_active":"2023-12-26T21:36:47.814728+0000","last_became_peered":"2023-12-26T21:36:47.814728+0000","last_unstale":"2023-12-26T21:36:48.842747+0000","last_undegraded":"2023-12-26T21:36:48.842747+0000","last_fullsized":"2023-12-26T21:36:48.842747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:14.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846566+0000","last_change":"2023-12-26T21:36:48.837315+0000","last_active":"2023-12-26T21:36:48.846566+0000","last_peered":"2023-12-26T21:36:48.846566+0000","last_clean":"2023-12-26T21:36:48.846566+0000","last_became_active":"2023-12-26T21:36:48.837067+0000","last_became_peered":"2023-12-26T21:36:48.837067+0000","last_unstale":"2023-12-26T21:36:48.846566+0000","last_undegraded":"2023-12-26T21:36:48.846566+0000","last_fullsized":"2023-12-26T21:36:48.846566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:28.883031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.834582+0000","last_change":"2023-12-26T21:36:47.825290+0000","last_active":"2023-12-26T21:36:48.834582+0000","last_peered":"2023-12-26T21:36:48.834582+0000","last_clean":"2023-12-26T21:36:48.834582+0000","last_became_active":"2023-12-26T21:36:47.824594+0000","last_became_peered":"2023-12-26T21:36:47.824594+0000","last_unstale":"2023-12-26T21:36:48.834582+0000","last_undegraded":"2023-12-26T21:36:48.834582+0000","last_fullsized":"2023-12-26T21:36:48.834582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:06.749340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843529+0000","last_change":"2023-12-26T21:36:48.835342+0000","last_active":"2023-12-26T21:36:48.843529+0000","last_peered":"2023-12-26T21:36:48.843529+0000","last_clean":"2023-12-26T21:36:48.843529+0000","last_became_active":"2023-12-26T21:36:48.830463+0000","last_became_peered":"2023-12-26T21:36:48.830463+0000","last_unstale":"2023-12-26T21:36:48.843529+0000","last_undegraded":"2023-12-26T21:36:48.843529+0000","last_fullsized":"2023-12-26T21:36:48.843529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.824113+0000","last_change":"2023-12-26T21:36:48.817508+0000","last_active":"2023-12-26T21:36:48.824113+0000","last_peered":"2023-12-26T21:36:48.824113+0000","last_clean":"2023-12-26T21:36:48.824113+0000","last_became_active":"2023-12-26T21:36:48.817269+0000","last_became_peered":"2023-12-26T21:36:48.817269+0000","last_unstale":"2023-12-26T21:36:48.824113+0000","last_undegraded":"2023-12-26T21:36:48.824113+0000","last_fullsized":"2023-12-26T21:36:48.824113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:54.478693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831298+0000","last_change":"2023-12-26T21:36:48.820601+0000","last_active":"2023-12-26T21:36:48.831298+0000","last_peered":"2023-12-26T21:36:48.831298+0000","last_clean":"2023-12-26T21:36:48.831298+0000","last_became_active":"2023-12-26T21:36:48.819293+0000","last_became_peered":"2023-12-26T21:36:48.819293+0000","last_unstale":"2023-12-26T21:36:48.831298+0000","last_undegraded":"2023-12-26T21:36:48.831298+0000","last_fullsized":"2023-12-26T21:36:48.831298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:26.207089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844324+0000","last_change":"2023-12-26T21:36:48.834258+0000","last_active":"2023-12-26T21:36:48.844324+0000","last_peered":"2023-12-26T21:36:48.844324+0000","last_clean":"2023-12-26T21:36:48.844324+0000","last_became_active":"2023-12-26T21:36:48.834005+0000","last_became_peered":"2023-12-26T21:36:48.834005+0000","last_unstale":"2023-12-26T21:36:48.844324+0000","last_undegraded":"2023-12-26T21:36:48.844324+0000","last_fullsized":"2023-12-26T21:36:48.844324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:15.953197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841827+0000","last_change":"2023-12-26T21:36:48.834061+0000","last_active":"2023-12-26T21:36:48.841827+0000","last_peered":"2023-12-26T21:36:48.841827+0000","last_clean":"2023-12-26T21:36:48.841827+0000","last_became_active":"2023-12-26T21:36:48.821177+0000","last_became_peered":"2023-12-26T21:36:48.821177+0000","last_unstale":"2023-12-26T21:36:48.841827+0000","last_undegraded":"2023-12-26T21:36:48.841827+0000","last_fullsized":"2023-12-26T21:36:48.841827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:21:25.931065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828251+0000","last_change":"2023-12-26T21:36:48.823718+0000","last_active":"2023-12-26T21:36:48.828251+0000","last_peered":"2023-12-26T21:36:48.828251+0000","last_clean":"2023-12-26T21:36:48.828251+0000","last_became_active":"2023-12-26T21:36:48.823422+0000","last_became_peered":"2023-12-26T21:36:48.823422+0000","last_unstale":"2023-12-26T21:36:48.828251+0000","last_undegraded":"2023-12-26T21:36:48.828251+0000","last_fullsized":"2023-12-26T21:36:48.828251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:42:23.347556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826781+0000","last_change":"2023-12-26T21:36:48.821819+0000","last_active":"2023-12-26T21:36:48.826781+0000","last_peered":"2023-12-26T21:36:48.826781+0000","last_clean":"2023-12-26T21:36:48.826781+0000","last_became_active":"2023-12-26T21:36:48.821527+0000","last_became_peered":"2023-12-26T21:36:48.821527+0000","last_unstale":"2023-12-26T21:36:48.826781+0000","last_undegraded":"2023-12-26T21:36:48.826781+0000","last_fullsized":"2023-12-26T21:36:48.826781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:31.058793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844139+0000","last_change":"2023-12-26T21:36:48.841186+0000","last_active":"2023-12-26T21:36:48.844139+0000","last_peered":"2023-12-26T21:36:48.844139+0000","last_clean":"2023-12-26T21:36:48.844139+0000","last_became_active":"2023-12-26T21:36:48.839955+0000","last_became_peered":"2023-12-26T21:36:48.839955+0000","last_unstale":"2023-12-26T21:36:48.844139+0000","last_undegraded":"2023-12-26T21:36:48.844139+0000","last_fullsized":"2023-12-26T21:36:48.844139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:35.307950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841419+0000","last_change":"2023-12-26T21:36:48.832556+0000","last_active":"2023-12-26T21:36:48.841419+0000","last_peered":"2023-12-26T21:36:48.841419+0000","last_clean":"2023-12-26T21:36:48.841419+0000","last_became_active":"2023-12-26T21:36:48.830061+0000","last_became_peered":"2023-12-26T21:36:48.830061+0000","last_unstale":"2023-12-26T21:36:48.841419+0000","last_undegraded":"2023-12-26T21:36:48.841419+0000","last_fullsized":"2023-12-26T21:36:48.841419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:22.938456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846674+0000","last_change":"2023-12-26T21:36:48.842574+0000","last_active":"2023-12-26T21:36:48.846674+0000","last_peered":"2023-12-26T21:36:48.846674+0000","last_clean":"2023-12-26T21:36:48.846674+0000","last_became_active":"2023-12-26T21:36:48.842273+0000","last_became_peered":"2023-12-26T21:36:48.842273+0000","last_unstale":"2023-12-26T21:36:48.846674+0000","last_undegraded":"2023-12-26T21:36:48.846674+0000","last_fullsized":"2023-12-26T21:36:48.846674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:17:59.482430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.840028+0000","last_change":"2023-12-26T21:36:48.826270+0000","last_active":"2023-12-26T21:36:48.840028+0000","last_peered":"2023-12-26T21:36:48.840028+0000","last_clean":"2023-12-26T21:36:48.840028+0000","last_became_active":"2023-12-26T21:36:48.825303+0000","last_became_peered":"2023-12-26T21:36:48.825303+0000","last_unstale":"2023-12-26T21:36:48.840028+0000","last_undegraded":"2023-12-26T21:36:48.840028+0000","last_fullsized":"2023-12-26T21:36:48.840028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:22.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828348+0000","last_change":"2023-12-26T21:36:48.823561+0000","last_active":"2023-12-26T21:36:48.828348+0000","last_peered":"2023-12-26T21:36:48.828348+0000","last_clean":"2023-12-26T21:36:48.828348+0000","last_became_active":"2023-12-26T21:36:48.823270+0000","last_became_peered":"2023-12-26T21:36:48.823270+0000","last_unstale":"2023-12-26T21:36:48.828348+0000","last_undegraded":"2023-12-26T21:36:48.828348+0000","last_fullsized":"2023-12-26T21:36:48.828348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:29.032786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839419+0000","last_change":"2023-12-26T21:36:48.828687+0000","last_active":"2023-12-26T21:36:48.839419+0000","last_peered":"2023-12-26T21:36:48.839419+0000","last_clean":"2023-12-26T21:36:48.839419+0000","last_became_active":"2023-12-26T21:36:48.828405+0000","last_became_peered":"2023-12-26T21:36:48.828405+0000","last_unstale":"2023-12-26T21:36:48.839419+0000","last_undegraded":"2023-12-26T21:36:48.839419+0000","last_fullsized":"2023-12-26T21:36:48.839419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:37.832638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841288+0000","last_change":"2023-12-26T21:36:48.835378+0000","last_active":"2023-12-26T21:36:48.841288+0000","last_peered":"2023-12-26T21:36:48.841288+0000","last_clean":"2023-12-26T21:36:48.841288+0000","last_became_active":"2023-12-26T21:36:48.835049+0000","last_became_peered":"2023-12-26T21:36:48.835049+0000","last_unstale":"2023-12-26T21:36:48.841288+0000","last_undegraded":"2023-12-26T21:36:48.841288+0000","last_fullsized":"2023-12-26T21:36:48.841288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:22.938456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":76900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":667360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":79080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":78436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]}]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":77024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0150000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.012}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2610000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.423}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3650000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6070000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298088,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073752,"statfs":{"total":96636764160,"available":96331522048,"internally_reserved":0,"allocated":819200,"data_stored":667338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":77454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:36:58.856 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:36:59.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.084+0000 7f44f9dff700 1 Processor -- start 2023-12-26T21:36:59.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.084+0000 7f44f9dff700 1 -- start start 2023-12-26T21:36:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f9dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f4159820 0x7f44f40abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f9dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f44f40aa480 0x7f44f406dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f9dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f415a8c0 con 0x7f44f406e2b0 2023-12-26T21:36:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f9dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f415ad50 con 0x7f44f40aa480 2023-12-26T21:36:59.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f9dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f406e730 con 0x7f44f4159820 2023-12-26T21:36:59.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f3fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f44f406e2b0 0x7f44f415d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49494/0 (socket says 172.21.15.131:49494) 2023-12-26T21:36:59.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f3fff700 1 -- 172.21.15.131:0/3825016357 learned_addr learned my addr 172.21.15.131:0/3825016357 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:36:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f2ffd700 1 --2- 172.21.15.131:0/3825016357 >> v2:172.21.15.190:3300/0 conn(0x7f44f40aa480 0x7f44f406dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f37fe700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f4159820 0x7f44f40abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425629474 0 0) 0x7f44f415a8c0 con 0x7f44f406e2b0 2023-12-26T21:36:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.086+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44dc003570 con 0x7f44f406e2b0 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676625839 0 0) 0x7f44dc003570 con 0x7f44f406e2b0 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f4159820 msgr2=0x7f44f40abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f4159820 0x7f44f40abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 >> v2:172.21.15.190:3300/0 conn(0x7f44f40aa480 msgr2=0x7f44f406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 --2- 172.21.15.131:0/3825016357 >> v2:172.21.15.190:3300/0 conn(0x7f44f40aa480 0x7f44f406dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f4159f90 con 0x7f44f406e2b0 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f44ec0017f0 con 0x7f44f406e2b0 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 >> v1:172.21.15.131:6789/0 conn(0x7f44f406e2b0 legacy=0x7f44f415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f4159820 0x7f44dc003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44dc0040e0 0x7f44dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44dc004d50 0x7f44dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f406e730 con 0x7f44dc004d50 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc00bb10 con 0x7f44dc0040e0 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc00bcd0 con 0x7f44f4159820 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f3fff700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44dc004d50 0x7f44dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f37fe700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f4159820 0x7f44dc003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f2ffd700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44dc0040e0 0x7f44dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f3fff700 1 -- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f4159820 msgr2=0x7f44dc003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f3fff700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f4159820 0x7f44dc003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f3fff700 1 -- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44dc0040e0 msgr2=0x7f44dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f3fff700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44dc0040e0 0x7f44dc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.087+0000 7f44f3fff700 1 -- 172.21.15.131:0/3825016357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc003570 con 0x7f44dc004d50 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.088+0000 7f44f3fff700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44dc004d50 0x7f44dc0051c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00a830 tx=0x7f44ec002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aefb64d33962140 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.088+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0070f0 con 0x7f44dc004d50 2023-12-26T21:36:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.088+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec007290 con 0x7f44dc004d50 2023-12-26T21:36:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.088+0000 7f44f27fc700 1 -- 172.21.15.131:0/3825016357 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec007650 con 0x7f44dc004d50 2023-12-26T21:36:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.088+0000 7f44f9dff700 1 -- 172.21.15.131:0/3825016357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44dc004d50 msgr2=0x7f44dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.088+0000 7f44f9dff700 1 --2- 172.21.15.131:0/3825016357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44dc004d50 0x7f44dc0051c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00a830 tx=0x7f44ec002100 comp rx=0 tx=0).stop 2023-12-26T21:36:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.088+0000 7f44f3fff700 1 -- 172.21.15.131:0/3825016357 reap_dead start 2023-12-26T21:36:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.088+0000 7f44f9dff700 1 -- 172.21.15.131:0/3825016357 shutdown_connections 2023-12-26T21:36:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.089+0000 7f44f9dff700 1 -- 172.21.15.131:0/3825016357 >> 172.21.15.131:0/3825016357 conn(0x7f44f406c290 msgr2=0x7f44f40ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.089+0000 7f44f9dff700 1 -- 172.21.15.131:0/3825016357 shutdown_connections 2023-12-26T21:36:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.089+0000 7f44f9dff700 1 -- 172.21.15.131:0/3825016357 wait complete. 2023-12-26T21:36:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.090+0000 7f44f9dff700 1 Processor -- start 2023-12-26T21:36:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.090+0000 7f44f9dff700 1 -- start start 2023-12-26T21:36:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f9dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44dc004d50 0x7f44f4152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f9dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f40aee60 0x7f44f416ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f9dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f4169660 0x7f44f4169b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f9dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40abdb0 con 0x7f44f4169660 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f9dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40ac000 con 0x7f44f40aee60 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f9dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40ac2e0 con 0x7f44dc004d50 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f4169660 0x7f44f4169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f37fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44dc004d50 0x7f44f4152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f4169660 0x7f44f4169b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45096/0 (socket says 172.21.15.131:45096) 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f3fff700 1 -- 172.21.15.131:0/1166126538 learned_addr learned my addr 172.21.15.131:0/1166126538 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.091+0000 7f44f2ffd700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f40aee60 0x7f44f416ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44f3fff700 1 -- 172.21.15.131:0/1166126538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44dc004d50 msgr2=0x7f44f4152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44f3fff700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44dc004d50 0x7f44f4152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44f3fff700 1 -- 172.21.15.131:0/1166126538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f40aee60 msgr2=0x7f44f416ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44f3fff700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f40aee60 0x7f44f416ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44f3fff700 1 -- 172.21.15.131:0/1166126538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f40ac5c0 con 0x7f44f4169660 2023-12-26T21:36:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44f3fff700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f4169660 0x7f44f4169b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44ec001220 tx=0x7f44ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44db7fe700 1 -- 172.21.15.131:0/1166126538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec001ee0 con 0x7f44f4169660 2023-12-26T21:36:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44db7fe700 1 -- 172.21.15.131:0/1166126538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec007630 con 0x7f44f4169660 2023-12-26T21:36:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f40ac8a0 con 0x7f44f4169660 2023-12-26T21:36:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44db7fe700 1 -- 172.21.15.131:0/1166126538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec007070 con 0x7f44f4169660 2023-12-26T21:36:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.092+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f416d3b0 con 0x7f44f4169660 2023-12-26T21:36:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.094+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f40657e0 con 0x7f44f4169660 2023-12-26T21:36:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.094+0000 7f44db7fe700 1 -- 172.21.15.131:0/1166126538 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f44ec004bb0 con 0x7f44f4169660 2023-12-26T21:36:59.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.095+0000 7f44db7fe700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44e0066f30 0x7f44e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.095+0000 7f44db7fe700 1 -- 172.21.15.131:0/1166126538 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f44ec086b80 con 0x7f44f4169660 2023-12-26T21:36:59.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.095+0000 7f44f37fe700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44e0066f30 0x7f44e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.096+0000 7f44f37fe700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44e0066f30 0x7f44e00693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44e4006f00 tx=0x7f44e40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:59.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.100+0000 7f44db7fe700 1 -- 172.21.15.131:0/1166126538 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44ec053df0 con 0x7f44f4169660 2023-12-26T21:36:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.289+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44f415a270 con 0x7f44e0066f30 2023-12-26T21:36:59.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.296+0000 7f44db7fe700 1 -- 172.21.15.131:0/1166126538 <== mgr.4100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264464 (secure 0 0 0) 0x7f44f415a270 con 0x7f44e0066f30 2023-12-26T21:36:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.300+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44e0066f30 msgr2=0x7f44e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.300+0000 7f44f9dff700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44e0066f30 0x7f44e00693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44e4006f00 tx=0x7f44e40091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f4169660 msgr2=0x7f44f4169b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f4169660 0x7f44f4169b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44ec001220 tx=0x7f44ec016000 comp rx=0 tx=0).stop 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 shutdown_connections 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f4169660 0x7f44f4169b00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44e0066f30 0x7f44e00693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44dc004d50 0x7f44f4152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 --2- 172.21.15.131:0/1166126538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f40aee60 0x7f44f416ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 >> 172.21.15.131:0/1166126538 conn(0x7f44f406c290 msgr2=0x7f44f4151b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:59.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 shutdown_connections 2023-12-26T21:36:59.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.301+0000 7f44f9dff700 1 -- 172.21.15.131:0/1166126538 wait complete. 2023-12-26T21:36:59.303 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:36:59.318 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-26T21:36:59.000319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381256,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593464,"statfs":{"total":773094113280,"available":770655707136,"internally_reserved":0,"allocated":3022848,"data_stored":1800548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002398"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845569+0000","last_change":"2023-12-26T21:36:48.838074+0000","last_active":"2023-12-26T21:36:48.845569+0000","last_peered":"2023-12-26T21:36:48.845569+0000","last_clean":"2023-12-26T21:36:48.845569+0000","last_became_active":"2023-12-26T21:36:48.837495+0000","last_became_peered":"2023-12-26T21:36:48.837495+0000","last_unstale":"2023-12-26T21:36:48.845569+0000","last_undegraded":"2023-12-26T21:36:48.845569+0000","last_fullsized":"2023-12-26T21:36:48.845569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841349+0000","last_change":"2023-12-26T21:36:48.830149+0000","last_active":"2023-12-26T21:36:48.841349+0000","last_peered":"2023-12-26T21:36:48.841349+0000","last_clean":"2023-12-26T21:36:48.841349+0000","last_became_active":"2023-12-26T21:36:48.829771+0000","last_became_peered":"2023-12-26T21:36:48.829771+0000","last_unstale":"2023-12-26T21:36:48.841349+0000","last_undegraded":"2023-12-26T21:36:48.841349+0000","last_fullsized":"2023-12-26T21:36:48.841349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:59.578493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841417+0000","last_change":"2023-12-26T21:36:48.831124+0000","last_active":"2023-12-26T21:36:48.841417+0000","last_peered":"2023-12-26T21:36:48.841417+0000","last_clean":"2023-12-26T21:36:48.841417+0000","last_became_active":"2023-12-26T21:36:48.830817+0000","last_became_peered":"2023-12-26T21:36:48.830817+0000","last_unstale":"2023-12-26T21:36:48.841417+0000","last_undegraded":"2023-12-26T21:36:48.841417+0000","last_fullsized":"2023-12-26T21:36:48.841417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:47.183606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845126+0000","last_change":"2023-12-26T21:36:48.842229+0000","last_active":"2023-12-26T21:36:48.845126+0000","last_peered":"2023-12-26T21:36:48.845126+0000","last_clean":"2023-12-26T21:36:48.845126+0000","last_became_active":"2023-12-26T21:36:48.839269+0000","last_became_peered":"2023-12-26T21:36:48.839269+0000","last_unstale":"2023-12-26T21:36:48.845126+0000","last_undegraded":"2023-12-26T21:36:48.845126+0000","last_fullsized":"2023-12-26T21:36:48.845126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:01:49.095423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.845558+0000","last_change":"2023-12-26T21:36:48.838113+0000","last_active":"2023-12-26T21:36:48.845558+0000","last_peered":"2023-12-26T21:36:48.845558+0000","last_clean":"2023-12-26T21:36:48.845558+0000","last_became_active":"2023-12-26T21:36:48.837882+0000","last_became_peered":"2023-12-26T21:36:48.837882+0000","last_unstale":"2023-12-26T21:36:48.845558+0000","last_undegraded":"2023-12-26T21:36:48.845558+0000","last_fullsized":"2023-12-26T21:36:48.845558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:40.692719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839762+0000","last_change":"2023-12-26T21:36:48.835731+0000","last_active":"2023-12-26T21:36:48.839762+0000","last_peered":"2023-12-26T21:36:48.839762+0000","last_clean":"2023-12-26T21:36:48.839762+0000","last_became_active":"2023-12-26T21:36:48.830672+0000","last_became_peered":"2023-12-26T21:36:48.830672+0000","last_unstale":"2023-12-26T21:36:48.839762+0000","last_undegraded":"2023-12-26T21:36:48.839762+0000","last_fullsized":"2023-12-26T21:36:48.839762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:06:58.960255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835557+0000","last_change":"2023-12-26T21:36:48.831504+0000","last_active":"2023-12-26T21:36:48.835557+0000","last_peered":"2023-12-26T21:36:48.835557+0000","last_clean":"2023-12-26T21:36:48.835557+0000","last_became_active":"2023-12-26T21:36:48.831173+0000","last_became_peered":"2023-12-26T21:36:48.831173+0000","last_unstale":"2023-12-26T21:36:48.835557+0000","last_undegraded":"2023-12-26T21:36:48.835557+0000","last_fullsized":"2023-12-26T21:36:48.835557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:37:13.353768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841944+0000","last_change":"2023-12-26T21:36:48.838249+0000","last_active":"2023-12-26T21:36:48.841944+0000","last_peered":"2023-12-26T21:36:48.841944+0000","last_clean":"2023-12-26T21:36:48.841944+0000","last_became_active":"2023-12-26T21:36:48.837983+0000","last_became_peered":"2023-12-26T21:36:48.837983+0000","last_unstale":"2023-12-26T21:36:48.841944+0000","last_undegraded":"2023-12-26T21:36:48.841944+0000","last_fullsized":"2023-12-26T21:36:48.841944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:56.706835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831690+0000","last_change":"2023-12-26T21:36:48.829328+0000","last_active":"2023-12-26T21:36:48.831690+0000","last_peered":"2023-12-26T21:36:48.831690+0000","last_clean":"2023-12-26T21:36:48.831690+0000","last_became_active":"2023-12-26T21:36:48.829033+0000","last_became_peered":"2023-12-26T21:36:48.829033+0000","last_unstale":"2023-12-26T21:36:48.831690+0000","last_undegraded":"2023-12-26T21:36:48.831690+0000","last_fullsized":"2023-12-26T21:36:48.831690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:31.058793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.837727+0000","last_change":"2023-12-26T21:36:48.834507+0000","last_active":"2023-12-26T21:36:48.837727+0000","last_peered":"2023-12-26T21:36:48.837727+0000","last_clean":"2023-12-26T21:36:48.837727+0000","last_became_active":"2023-12-26T21:36:48.833751+0000","last_became_peered":"2023-12-26T21:36:48.833751+0000","last_unstale":"2023-12-26T21:36:48.837727+0000","last_undegraded":"2023-12-26T21:36:48.837727+0000","last_fullsized":"2023-12-26T21:36:48.837727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:23.349206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846618+0000","last_change":"2023-12-26T21:36:48.839909+0000","last_active":"2023-12-26T21:36:48.846618+0000","last_peered":"2023-12-26T21:36:48.846618+0000","last_clean":"2023-12-26T21:36:48.846618+0000","last_became_active":"2023-12-26T21:36:48.839611+0000","last_became_peered":"2023-12-26T21:36:48.839611+0000","last_unstale":"2023-12-26T21:36:48.846618+0000","last_undegraded":"2023-12-26T21:36:48.846618+0000","last_fullsized":"2023-12-26T21:36:48.846618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:49.385696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841298+0000","last_change":"2023-12-26T21:36:48.836176+0000","last_active":"2023-12-26T21:36:48.841298+0000","last_peered":"2023-12-26T21:36:48.841298+0000","last_clean":"2023-12-26T21:36:48.841298+0000","last_became_active":"2023-12-26T21:36:48.835911+0000","last_became_peered":"2023-12-26T21:36:48.835911+0000","last_unstale":"2023-12-26T21:36:48.841298+0000","last_undegraded":"2023-12-26T21:36:48.841298+0000","last_fullsized":"2023-12-26T21:36:48.841298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:32.390648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828384+0000","last_change":"2023-12-26T21:36:48.825560+0000","last_active":"2023-12-26T21:36:48.828384+0000","last_peered":"2023-12-26T21:36:48.828384+0000","last_clean":"2023-12-26T21:36:48.828384+0000","last_became_active":"2023-12-26T21:36:48.825203+0000","last_became_peered":"2023-12-26T21:36:48.825203+0000","last_unstale":"2023-12-26T21:36:48.828384+0000","last_undegraded":"2023-12-26T21:36:48.828384+0000","last_fullsized":"2023-12-26T21:36:48.828384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:24.061374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828296+0000","last_change":"2023-12-26T21:36:48.823931+0000","last_active":"2023-12-26T21:36:48.828296+0000","last_peered":"2023-12-26T21:36:48.828296+0000","last_clean":"2023-12-26T21:36:48.828296+0000","last_became_active":"2023-12-26T21:36:48.823564+0000","last_became_peered":"2023-12-26T21:36:48.823564+0000","last_unstale":"2023-12-26T21:36:48.828296+0000","last_undegraded":"2023-12-26T21:36:48.828296+0000","last_fullsized":"2023-12-26T21:36:48.828296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:42.053931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833245+0000","last_change":"2023-12-26T21:36:48.823836+0000","last_active":"2023-12-26T21:36:48.833245+0000","last_peered":"2023-12-26T21:36:48.833245+0000","last_clean":"2023-12-26T21:36:48.833245+0000","last_became_active":"2023-12-26T21:36:48.823053+0000","last_became_peered":"2023-12-26T21:36:48.823053+0000","last_unstale":"2023-12-26T21:36:48.833245+0000","last_undegraded":"2023-12-26T21:36:48.833245+0000","last_fullsized":"2023-12-26T21:36:48.833245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:28.883031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838446+0000","last_change":"2023-12-26T21:36:48.829032+0000","last_active":"2023-12-26T21:36:48.838446+0000","last_peered":"2023-12-26T21:36:48.838446+0000","last_clean":"2023-12-26T21:36:48.838446+0000","last_became_active":"2023-12-26T21:36:48.826097+0000","last_became_peered":"2023-12-26T21:36:48.826097+0000","last_unstale":"2023-12-26T21:36:48.838446+0000","last_undegraded":"2023-12-26T21:36:48.838446+0000","last_fullsized":"2023-12-26T21:36:48.838446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:49.385696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844638+0000","last_change":"2023-12-26T21:36:48.832781+0000","last_active":"2023-12-26T21:36:48.844638+0000","last_peered":"2023-12-26T21:36:48.844638+0000","last_clean":"2023-12-26T21:36:48.844638+0000","last_became_active":"2023-12-26T21:36:48.832414+0000","last_became_peered":"2023-12-26T21:36:48.832414+0000","last_unstale":"2023-12-26T21:36:48.844638+0000","last_undegraded":"2023-12-26T21:36:48.844638+0000","last_fullsized":"2023-12-26T21:36:48.844638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:05:36.844680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.825827+0000","last_change":"2023-12-26T21:36:48.817787+0000","last_active":"2023-12-26T21:36:48.825827+0000","last_peered":"2023-12-26T21:36:48.825827+0000","last_clean":"2023-12-26T21:36:48.825827+0000","last_became_active":"2023-12-26T21:36:48.817432+0000","last_became_peered":"2023-12-26T21:36:48.817432+0000","last_unstale":"2023-12-26T21:36:48.825827+0000","last_undegraded":"2023-12-26T21:36:48.825827+0000","last_fullsized":"2023-12-26T21:36:48.825827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:51.402901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844696+0000","last_change":"2023-12-26T21:36:48.834659+0000","last_active":"2023-12-26T21:36:48.844696+0000","last_peered":"2023-12-26T21:36:48.844696+0000","last_clean":"2023-12-26T21:36:48.844696+0000","last_became_active":"2023-12-26T21:36:48.833967+0000","last_became_peered":"2023-12-26T21:36:48.833967+0000","last_unstale":"2023-12-26T21:36:48.844696+0000","last_undegraded":"2023-12-26T21:36:48.844696+0000","last_fullsized":"2023-12-26T21:36:48.844696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:12:41.152256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839315+0000","last_change":"2023-12-26T21:36:47.830031+0000","last_active":"2023-12-26T21:36:48.839315+0000","last_peered":"2023-12-26T21:36:48.839315+0000","last_clean":"2023-12-26T21:36:48.839315+0000","last_became_active":"2023-12-26T21:36:47.829681+0000","last_became_peered":"2023-12-26T21:36:47.829681+0000","last_unstale":"2023-12-26T21:36:48.839315+0000","last_undegraded":"2023-12-26T21:36:48.839315+0000","last_fullsized":"2023-12-26T21:36:48.839315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:25.200664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841270+0000","last_change":"2023-12-26T21:36:48.827382+0000","last_active":"2023-12-26T21:36:48.841270+0000","last_peered":"2023-12-26T21:36:48.841270+0000","last_clean":"2023-12-26T21:36:48.841270+0000","last_became_active":"2023-12-26T21:36:48.825125+0000","last_became_peered":"2023-12-26T21:36:48.825125+0000","last_unstale":"2023-12-26T21:36:48.841270+0000","last_undegraded":"2023-12-26T21:36:48.841270+0000","last_fullsized":"2023-12-26T21:36:48.841270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:02:27.671653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843042+0000","last_change":"2023-12-26T21:36:47.825862+0000","last_active":"2023-12-26T21:36:48.843042+0000","last_peered":"2023-12-26T21:36:48.843042+0000","last_clean":"2023-12-26T21:36:48.843042+0000","last_became_active":"2023-12-26T21:36:47.825603+0000","last_became_peered":"2023-12-26T21:36:47.825603+0000","last_unstale":"2023-12-26T21:36:48.843042+0000","last_undegraded":"2023-12-26T21:36:48.843042+0000","last_fullsized":"2023-12-26T21:36:48.843042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:18:58.593197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833411+0000","last_change":"2023-12-26T21:36:48.819527+0000","last_active":"2023-12-26T21:36:48.833411+0000","last_peered":"2023-12-26T21:36:48.833411+0000","last_clean":"2023-12-26T21:36:48.833411+0000","last_became_active":"2023-12-26T21:36:48.819208+0000","last_became_peered":"2023-12-26T21:36:48.819208+0000","last_unstale":"2023-12-26T21:36:48.833411+0000","last_undegraded":"2023-12-26T21:36:48.833411+0000","last_fullsized":"2023-12-26T21:36:48.833411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:32.390648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841778+0000","last_change":"2023-12-26T21:36:47.824557+0000","last_active":"2023-12-26T21:36:48.841778+0000","last_peered":"2023-12-26T21:36:48.841778+0000","last_clean":"2023-12-26T21:36:48.841778+0000","last_became_active":"2023-12-26T21:36:47.824200+0000","last_became_peered":"2023-12-26T21:36:47.824200+0000","last_unstale":"2023-12-26T21:36:48.841778+0000","last_undegraded":"2023-12-26T21:36:48.841778+0000","last_fullsized":"2023-12-26T21:36:48.841778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:13.097046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.827346+0000","last_change":"2023-12-26T21:36:48.823425+0000","last_active":"2023-12-26T21:36:48.827346+0000","last_peered":"2023-12-26T21:36:48.827346+0000","last_clean":"2023-12-26T21:36:48.827346+0000","last_became_active":"2023-12-26T21:36:48.823148+0000","last_became_peered":"2023-12-26T21:36:48.823148+0000","last_unstale":"2023-12-26T21:36:48.827346+0000","last_undegraded":"2023-12-26T21:36:48.827346+0000","last_fullsized":"2023-12-26T21:36:48.827346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:35.691705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841375+0000","last_change":"2023-12-26T21:36:47.826954+0000","last_active":"2023-12-26T21:36:48.841375+0000","last_peered":"2023-12-26T21:36:48.841375+0000","last_clean":"2023-12-26T21:36:48.841375+0000","last_became_active":"2023-12-26T21:36:47.826691+0000","last_became_peered":"2023-12-26T21:36:47.826691+0000","last_unstale":"2023-12-26T21:36:48.841375+0000","last_undegraded":"2023-12-26T21:36:48.841375+0000","last_fullsized":"2023-12-26T21:36:48.841375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:24.145739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843898+0000","last_change":"2023-12-26T21:36:48.830893+0000","last_active":"2023-12-26T21:36:48.843898+0000","last_peered":"2023-12-26T21:36:48.843898+0000","last_clean":"2023-12-26T21:36:48.843898+0000","last_became_active":"2023-12-26T21:36:48.830553+0000","last_became_peered":"2023-12-26T21:36:48.830553+0000","last_unstale":"2023-12-26T21:36:48.843898+0000","last_undegraded":"2023-12-26T21:36:48.843898+0000","last_fullsized":"2023-12-26T21:36:48.843898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:40:07.698471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841849+0000","last_change":"2023-12-26T21:36:47.828216+0000","last_active":"2023-12-26T21:36:48.841849+0000","last_peered":"2023-12-26T21:36:48.841849+0000","last_clean":"2023-12-26T21:36:48.841849+0000","last_became_active":"2023-12-26T21:36:47.827950+0000","last_became_peered":"2023-12-26T21:36:47.827950+0000","last_unstale":"2023-12-26T21:36:48.841849+0000","last_undegraded":"2023-12-26T21:36:48.841849+0000","last_fullsized":"2023-12-26T21:36:48.841849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:23.069341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828328+0000","last_change":"2023-12-26T21:36:48.820490+0000","last_active":"2023-12-26T21:36:48.828328+0000","last_peered":"2023-12-26T21:36:48.828328+0000","last_clean":"2023-12-26T21:36:48.828328+0000","last_became_active":"2023-12-26T21:36:48.820298+0000","last_became_peered":"2023-12-26T21:36:48.820298+0000","last_unstale":"2023-12-26T21:36:48.828328+0000","last_undegraded":"2023-12-26T21:36:48.828328+0000","last_fullsized":"2023-12-26T21:36:48.828328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:25.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.830175+0000","last_change":"2023-12-26T21:36:47.830245+0000","last_active":"2023-12-26T21:36:48.830175+0000","last_peered":"2023-12-26T21:36:48.830175+0000","last_clean":"2023-12-26T21:36:48.830175+0000","last_became_active":"2023-12-26T21:36:47.829970+0000","last_became_peered":"2023-12-26T21:36:47.829970+0000","last_unstale":"2023-12-26T21:36:48.830175+0000","last_undegraded":"2023-12-26T21:36:48.830175+0000","last_fullsized":"2023-12-26T21:36:48.830175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:13.329724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843818+0000","last_change":"2023-12-26T21:36:48.839949+0000","last_active":"2023-12-26T21:36:48.843818+0000","last_peered":"2023-12-26T21:36:48.843818+0000","last_clean":"2023-12-26T21:36:48.843818+0000","last_became_active":"2023-12-26T21:36:48.839655+0000","last_became_peered":"2023-12-26T21:36:48.839655+0000","last_unstale":"2023-12-26T21:36:48.843818+0000","last_undegraded":"2023-12-26T21:36:48.843818+0000","last_fullsized":"2023-12-26T21:36:48.843818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:43:57.839515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.809369+0000","last_change":"2023-12-26T21:36:47.828451+0000","last_active":"2023-12-26T21:36:48.809369+0000","last_peered":"2023-12-26T21:36:48.809369+0000","last_clean":"2023-12-26T21:36:48.809369+0000","last_became_active":"2023-12-26T21:36:47.828158+0000","last_became_peered":"2023-12-26T21:36:47.828158+0000","last_unstale":"2023-12-26T21:36:48.809369+0000","last_undegraded":"2023-12-26T21:36:48.809369+0000","last_fullsized":"2023-12-26T21:36:48.809369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:23.069341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843628+0000","last_change":"2023-12-26T21:36:48.840894+0000","last_active":"2023-12-26T21:36:48.843628+0000","last_peered":"2023-12-26T21:36:48.843628+0000","last_clean":"2023-12-26T21:36:48.843628+0000","last_became_active":"2023-12-26T21:36:48.840284+0000","last_became_peered":"2023-12-26T21:36:48.840284+0000","last_unstale":"2023-12-26T21:36:48.843628+0000","last_undegraded":"2023-12-26T21:36:48.843628+0000","last_fullsized":"2023-12-26T21:36:48.843628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:37:23.349206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816543+0000","last_change":"2023-12-26T21:36:47.821233+0000","last_active":"2023-12-26T21:36:48.816543+0000","last_peered":"2023-12-26T21:36:48.816543+0000","last_clean":"2023-12-26T21:36:48.816543+0000","last_became_active":"2023-12-26T21:36:47.820129+0000","last_became_peered":"2023-12-26T21:36:47.820129+0000","last_unstale":"2023-12-26T21:36:48.816543+0000","last_undegraded":"2023-12-26T21:36:48.816543+0000","last_fullsized":"2023-12-26T21:36:48.816543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:30.052367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838583+0000","last_change":"2023-12-26T21:36:48.829638+0000","last_active":"2023-12-26T21:36:48.838583+0000","last_peered":"2023-12-26T21:36:48.838583+0000","last_clean":"2023-12-26T21:36:48.838583+0000","last_became_active":"2023-12-26T21:36:48.829154+0000","last_became_peered":"2023-12-26T21:36:48.829154+0000","last_unstale":"2023-12-26T21:36:48.838583+0000","last_undegraded":"2023-12-26T21:36:48.838583+0000","last_fullsized":"2023-12-26T21:36:48.838583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:45:55.685017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.803440+0000","last_change":"2023-12-26T21:36:47.818993+0000","last_active":"2023-12-26T21:36:48.803440+0000","last_peered":"2023-12-26T21:36:48.803440+0000","last_clean":"2023-12-26T21:36:48.803440+0000","last_became_active":"2023-12-26T21:36:47.818731+0000","last_became_peered":"2023-12-26T21:36:47.818731+0000","last_unstale":"2023-12-26T21:36:48.803440+0000","last_undegraded":"2023-12-26T21:36:48.803440+0000","last_fullsized":"2023-12-26T21:36:48.803440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:14.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.840833+0000","last_change":"2023-12-26T21:36:48.837014+0000","last_active":"2023-12-26T21:36:48.840833+0000","last_peered":"2023-12-26T21:36:48.840833+0000","last_clean":"2023-12-26T21:36:48.840833+0000","last_became_active":"2023-12-26T21:36:48.830649+0000","last_became_peered":"2023-12-26T21:36:48.830649+0000","last_unstale":"2023-12-26T21:36:48.840833+0000","last_undegraded":"2023-12-26T21:36:48.840833+0000","last_fullsized":"2023-12-26T21:36:48.840833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:07.755765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829625+0000","last_change":"2023-12-26T21:36:47.827036+0000","last_active":"2023-12-26T21:36:48.829625+0000","last_peered":"2023-12-26T21:36:48.829625+0000","last_clean":"2023-12-26T21:36:48.829625+0000","last_became_active":"2023-12-26T21:36:47.826746+0000","last_became_peered":"2023-12-26T21:36:47.826746+0000","last_unstale":"2023-12-26T21:36:48.829625+0000","last_undegraded":"2023-12-26T21:36:48.829625+0000","last_fullsized":"2023-12-26T21:36:48.829625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:28.026360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841896+0000","last_change":"2023-12-26T21:36:48.837290+0000","last_active":"2023-12-26T21:36:48.841896+0000","last_peered":"2023-12-26T21:36:48.841896+0000","last_clean":"2023-12-26T21:36:48.841896+0000","last_became_active":"2023-12-26T21:36:48.836991+0000","last_became_peered":"2023-12-26T21:36:48.836991+0000","last_unstale":"2023-12-26T21:36:48.841896+0000","last_undegraded":"2023-12-26T21:36:48.841896+0000","last_fullsized":"2023-12-26T21:36:48.841896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835423+0000","last_change":"2023-12-26T21:36:47.823376+0000","last_active":"2023-12-26T21:36:48.835423+0000","last_peered":"2023-12-26T21:36:48.835423+0000","last_clean":"2023-12-26T21:36:48.835423+0000","last_became_active":"2023-12-26T21:36:47.822986+0000","last_became_peered":"2023-12-26T21:36:47.822986+0000","last_unstale":"2023-12-26T21:36:48.835423+0000","last_undegraded":"2023-12-26T21:36:48.835423+0000","last_fullsized":"2023-12-26T21:36:48.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:45:26.314194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829780+0000","last_change":"2023-12-26T21:36:48.825556+0000","last_active":"2023-12-26T21:36:48.829780+0000","last_peered":"2023-12-26T21:36:48.829780+0000","last_clean":"2023-12-26T21:36:48.829780+0000","last_became_active":"2023-12-26T21:36:48.825233+0000","last_became_peered":"2023-12-26T21:36:48.825233+0000","last_unstale":"2023-12-26T21:36:48.829780+0000","last_undegraded":"2023-12-26T21:36:48.829780+0000","last_fullsized":"2023-12-26T21:36:48.829780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:40.692719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.848461+0000","last_change":"2023-12-26T21:36:47.821098+0000","last_active":"2023-12-26T21:36:48.848461+0000","last_peered":"2023-12-26T21:36:48.848461+0000","last_clean":"2023-12-26T21:36:48.848461+0000","last_became_active":"2023-12-26T21:36:47.820639+0000","last_became_peered":"2023-12-26T21:36:47.820639+0000","last_unstale":"2023-12-26T21:36:48.848461+0000","last_undegraded":"2023-12-26T21:36:48.848461+0000","last_fullsized":"2023-12-26T21:36:48.848461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:26.330467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.833737+0000","last_change":"2023-12-26T21:36:48.828635+0000","last_active":"2023-12-26T21:36:48.833737+0000","last_peered":"2023-12-26T21:36:48.833737+0000","last_clean":"2023-12-26T21:36:48.833737+0000","last_became_active":"2023-12-26T21:36:48.828382+0000","last_became_peered":"2023-12-26T21:36:48.828382+0000","last_unstale":"2023-12-26T21:36:48.833737+0000","last_undegraded":"2023-12-26T21:36:48.833737+0000","last_fullsized":"2023-12-26T21:36:48.833737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:59.578493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.835877+0000","last_change":"2023-12-26T21:36:47.830348+0000","last_active":"2023-12-26T21:36:48.835877+0000","last_peered":"2023-12-26T21:36:48.835877+0000","last_clean":"2023-12-26T21:36:48.835877+0000","last_became_active":"2023-12-26T21:36:47.830053+0000","last_became_peered":"2023-12-26T21:36:47.830053+0000","last_unstale":"2023-12-26T21:36:48.835877+0000","last_undegraded":"2023-12-26T21:36:48.835877+0000","last_fullsized":"2023-12-26T21:36:48.835877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:26.330467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.832805+0000","last_change":"2023-12-26T21:36:48.830118+0000","last_active":"2023-12-26T21:36:48.832805+0000","last_peered":"2023-12-26T21:36:48.832805+0000","last_clean":"2023-12-26T21:36:48.832805+0000","last_became_active":"2023-12-26T21:36:48.829843+0000","last_became_peered":"2023-12-26T21:36:48.829843+0000","last_unstale":"2023-12-26T21:36:48.832805+0000","last_undegraded":"2023-12-26T21:36:48.832805+0000","last_fullsized":"2023-12-26T21:36:48.832805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817442+0000","last_change":"2023-12-26T21:36:47.822624+0000","last_active":"2023-12-26T21:36:48.817442+0000","last_peered":"2023-12-26T21:36:48.817442+0000","last_clean":"2023-12-26T21:36:48.817442+0000","last_became_active":"2023-12-26T21:36:47.822048+0000","last_became_peered":"2023-12-26T21:36:47.822048+0000","last_unstale":"2023-12-26T21:36:48.817442+0000","last_undegraded":"2023-12-26T21:36:48.817442+0000","last_fullsized":"2023-12-26T21:36:48.817442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:24.145739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846368+0000","last_change":"2023-12-26T21:36:48.837022+0000","last_active":"2023-12-26T21:36:48.846368+0000","last_peered":"2023-12-26T21:36:48.846368+0000","last_clean":"2023-12-26T21:36:48.846368+0000","last_became_active":"2023-12-26T21:36:48.836736+0000","last_became_peered":"2023-12-26T21:36:48.836736+0000","last_unstale":"2023-12-26T21:36:48.846368+0000","last_undegraded":"2023-12-26T21:36:48.846368+0000","last_fullsized":"2023-12-26T21:36:48.846368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:57.196069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826918+0000","last_change":"2023-12-26T21:36:47.829979+0000","last_active":"2023-12-26T21:36:48.826918+0000","last_peered":"2023-12-26T21:36:48.826918+0000","last_clean":"2023-12-26T21:36:48.826918+0000","last_became_active":"2023-12-26T21:36:47.829685+0000","last_became_peered":"2023-12-26T21:36:47.829685+0000","last_unstale":"2023-12-26T21:36:48.826918+0000","last_undegraded":"2023-12-26T21:36:48.826918+0000","last_fullsized":"2023-12-26T21:36:48.826918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831660+0000","last_change":"2023-12-26T21:36:48.828008+0000","last_active":"2023-12-26T21:36:48.831660+0000","last_peered":"2023-12-26T21:36:48.831660+0000","last_clean":"2023-12-26T21:36:48.831660+0000","last_became_active":"2023-12-26T21:36:48.827743+0000","last_became_peered":"2023-12-26T21:36:48.827743+0000","last_unstale":"2023-12-26T21:36:48.831660+0000","last_undegraded":"2023-12-26T21:36:48.831660+0000","last_fullsized":"2023-12-26T21:36:48.831660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:14.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839801+0000","last_change":"2023-12-26T21:36:47.815814+0000","last_active":"2023-12-26T21:36:48.839801+0000","last_peered":"2023-12-26T21:36:48.839801+0000","last_clean":"2023-12-26T21:36:48.839801+0000","last_became_active":"2023-12-26T21:36:47.815480+0000","last_became_peered":"2023-12-26T21:36:47.815480+0000","last_unstale":"2023-12-26T21:36:48.839801+0000","last_undegraded":"2023-12-26T21:36:48.839801+0000","last_fullsized":"2023-12-26T21:36:48.839801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:18:28.266978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843897+0000","last_change":"2023-12-26T21:36:48.833364+0000","last_active":"2023-12-26T21:36:48.843897+0000","last_peered":"2023-12-26T21:36:48.843897+0000","last_clean":"2023-12-26T21:36:48.843897+0000","last_became_active":"2023-12-26T21:36:48.833087+0000","last_became_peered":"2023-12-26T21:36:48.833087+0000","last_unstale":"2023-12-26T21:36:48.843897+0000","last_undegraded":"2023-12-26T21:36:48.843897+0000","last_fullsized":"2023-12-26T21:36:48.843897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:05:49.639640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846496+0000","last_change":"2023-12-26T21:36:47.823499+0000","last_active":"2023-12-26T21:36:48.846496+0000","last_peered":"2023-12-26T21:36:48.846496+0000","last_clean":"2023-12-26T21:36:48.846496+0000","last_became_active":"2023-12-26T21:36:47.819416+0000","last_became_peered":"2023-12-26T21:36:47.819416+0000","last_unstale":"2023-12-26T21:36:48.846496+0000","last_undegraded":"2023-12-26T21:36:48.846496+0000","last_fullsized":"2023-12-26T21:36:48.846496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:04:13.329724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839537+0000","last_change":"2023-12-26T21:36:48.824260+0000","last_active":"2023-12-26T21:36:48.839537+0000","last_peered":"2023-12-26T21:36:48.839537+0000","last_clean":"2023-12-26T21:36:48.839537+0000","last_became_active":"2023-12-26T21:36:48.824000+0000","last_became_peered":"2023-12-26T21:36:48.824000+0000","last_unstale":"2023-12-26T21:36:48.839537+0000","last_undegraded":"2023-12-26T21:36:48.839537+0000","last_fullsized":"2023-12-26T21:36:48.839537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:24.151016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.812438+0000","last_change":"2023-12-26T21:36:47.829031+0000","last_active":"2023-12-26T21:36:48.812438+0000","last_peered":"2023-12-26T21:36:48.812438+0000","last_clean":"2023-12-26T21:36:48.812438+0000","last_became_active":"2023-12-26T21:36:47.828711+0000","last_became_peered":"2023-12-26T21:36:47.828711+0000","last_unstale":"2023-12-26T21:36:48.812438+0000","last_undegraded":"2023-12-26T21:36:48.812438+0000","last_fullsized":"2023-12-26T21:36:48.812438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.822529+0000","last_change":"2023-12-26T21:36:48.817001+0000","last_active":"2023-12-26T21:36:48.822529+0000","last_peered":"2023-12-26T21:36:48.822529+0000","last_clean":"2023-12-26T21:36:48.822529+0000","last_became_active":"2023-12-26T21:36:48.816746+0000","last_became_peered":"2023-12-26T21:36:48.816746+0000","last_unstale":"2023-12-26T21:36:48.822529+0000","last_undegraded":"2023-12-26T21:36:48.822529+0000","last_fullsized":"2023-12-26T21:36:48.822529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:56.619479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817332+0000","last_change":"2023-12-26T21:36:47.828501+0000","last_active":"2023-12-26T21:36:48.817332+0000","last_peered":"2023-12-26T21:36:48.817332+0000","last_clean":"2023-12-26T21:36:48.817332+0000","last_became_active":"2023-12-26T21:36:47.828257+0000","last_became_peered":"2023-12-26T21:36:47.828257+0000","last_unstale":"2023-12-26T21:36:48.817332+0000","last_undegraded":"2023-12-26T21:36:48.817332+0000","last_fullsized":"2023-12-26T21:36:48.817332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:55:17.677425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.837624+0000","last_change":"2023-12-26T21:36:48.834334+0000","last_active":"2023-12-26T21:36:48.837624+0000","last_peered":"2023-12-26T21:36:48.837624+0000","last_clean":"2023-12-26T21:36:48.837624+0000","last_became_active":"2023-12-26T21:36:48.834070+0000","last_became_peered":"2023-12-26T21:36:48.834070+0000","last_unstale":"2023-12-26T21:36:48.837624+0000","last_undegraded":"2023-12-26T21:36:48.837624+0000","last_fullsized":"2023-12-26T21:36:48.837624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:21:25.931065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843647+0000","last_change":"2023-12-26T21:36:48.841064+0000","last_active":"2023-12-26T21:36:48.843647+0000","last_peered":"2023-12-26T21:36:48.843647+0000","last_clean":"2023-12-26T21:36:48.843647+0000","last_became_active":"2023-12-26T21:36:48.840654+0000","last_became_peered":"2023-12-26T21:36:48.840654+0000","last_unstale":"2023-12-26T21:36:48.843647+0000","last_undegraded":"2023-12-26T21:36:48.843647+0000","last_fullsized":"2023-12-26T21:36:48.843647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.804661+0000","last_change":"2023-12-26T21:36:47.822491+0000","last_active":"2023-12-26T21:36:48.804661+0000","last_peered":"2023-12-26T21:36:48.804661+0000","last_clean":"2023-12-26T21:36:48.804661+0000","last_became_active":"2023-12-26T21:36:47.822150+0000","last_became_peered":"2023-12-26T21:36:47.822150+0000","last_unstale":"2023-12-26T21:36:48.804661+0000","last_undegraded":"2023-12-26T21:36:48.804661+0000","last_fullsized":"2023-12-26T21:36:48.804661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:08:16.186961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.834826+0000","last_change":"2023-12-26T21:36:48.828798+0000","last_active":"2023-12-26T21:36:48.834826+0000","last_peered":"2023-12-26T21:36:48.834826+0000","last_clean":"2023-12-26T21:36:48.834826+0000","last_became_active":"2023-12-26T21:36:48.828093+0000","last_became_peered":"2023-12-26T21:36:48.828093+0000","last_unstale":"2023-12-26T21:36:48.834826+0000","last_undegraded":"2023-12-26T21:36:48.834826+0000","last_fullsized":"2023-12-26T21:36:48.834826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:19:55.017056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817551+0000","last_change":"2023-12-26T21:36:47.825144+0000","last_active":"2023-12-26T21:36:48.817551+0000","last_peered":"2023-12-26T21:36:48.817551+0000","last_clean":"2023-12-26T21:36:48.817551+0000","last_became_active":"2023-12-26T21:36:47.824896+0000","last_became_peered":"2023-12-26T21:36:47.824896+0000","last_unstale":"2023-12-26T21:36:48.817551+0000","last_undegraded":"2023-12-26T21:36:48.817551+0000","last_fullsized":"2023-12-26T21:36:48.817551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:03:48.097670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.803182+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:36:48.803182+0000","last_peered":"2023-12-26T21:36:48.803182+0000","last_clean":"2023-12-26T21:36:48.803182+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:36:48.803182+0000","last_undegraded":"2023-12-26T21:36:48.803182+0000","last_fullsized":"2023-12-26T21:36:48.803182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.818212+0000","last_change":"2023-12-26T21:36:48.816469+0000","last_active":"2023-12-26T21:36:48.818212+0000","last_peered":"2023-12-26T21:36:48.818212+0000","last_clean":"2023-12-26T21:36:48.818212+0000","last_became_active":"2023-12-26T21:36:48.816190+0000","last_became_peered":"2023-12-26T21:36:48.816190+0000","last_unstale":"2023-12-26T21:36:48.818212+0000","last_undegraded":"2023-12-26T21:36:48.818212+0000","last_fullsized":"2023-12-26T21:36:48.818212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:35.691705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816437+0000","last_change":"2023-12-26T21:36:47.822572+0000","last_active":"2023-12-26T21:36:48.816437+0000","last_peered":"2023-12-26T21:36:48.816437+0000","last_clean":"2023-12-26T21:36:48.816437+0000","last_became_active":"2023-12-26T21:36:47.822214+0000","last_became_peered":"2023-12-26T21:36:47.822214+0000","last_unstale":"2023-12-26T21:36:48.816437+0000","last_undegraded":"2023-12-26T21:36:48.816437+0000","last_fullsized":"2023-12-26T21:36:48.816437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:34.685280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.816507+0000","last_change":"2023-12-26T21:36:47.826782+0000","last_active":"2023-12-26T21:36:48.816507+0000","last_peered":"2023-12-26T21:36:48.816507+0000","last_clean":"2023-12-26T21:36:48.816507+0000","last_became_active":"2023-12-26T21:36:47.825248+0000","last_became_peered":"2023-12-26T21:36:47.825248+0000","last_unstale":"2023-12-26T21:36:48.816507+0000","last_undegraded":"2023-12-26T21:36:48.816507+0000","last_fullsized":"2023-12-26T21:36:48.816507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:28.026360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831901+0000","last_change":"2023-12-26T21:36:48.827788+0000","last_active":"2023-12-26T21:36:48.831901+0000","last_peered":"2023-12-26T21:36:48.831901+0000","last_clean":"2023-12-26T21:36:48.831901+0000","last_became_active":"2023-12-26T21:36:48.827487+0000","last_became_peered":"2023-12-26T21:36:48.827487+0000","last_unstale":"2023-12-26T21:36:48.831901+0000","last_undegraded":"2023-12-26T21:36:48.831901+0000","last_fullsized":"2023-12-26T21:36:48.831901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:22.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-26T21:36:49.819679+0000","last_change":"2023-12-26T21:36:47.830347+0000","last_active":"2023-12-26T21:36:49.819679+0000","last_peered":"2023-12-26T21:36:49.819679+0000","last_clean":"2023-12-26T21:36:49.819679+0000","last_became_active":"2023-12-26T21:36:47.830037+0000","last_became_peered":"2023-12-26T21:36:47.830037+0000","last_unstale":"2023-12-26T21:36:49.819679+0000","last_undegraded":"2023-12-26T21:36:49.819679+0000","last_fullsized":"2023-12-26T21:36:49.819679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:55.613054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.838484+0000","last_change":"2023-12-26T21:36:48.829585+0000","last_active":"2023-12-26T21:36:48.838484+0000","last_peered":"2023-12-26T21:36:48.838484+0000","last_clean":"2023-12-26T21:36:48.838484+0000","last_became_active":"2023-12-26T21:36:48.829170+0000","last_became_peered":"2023-12-26T21:36:48.829170+0000","last_unstale":"2023-12-26T21:36:48.838484+0000","last_undegraded":"2023-12-26T21:36:48.838484+0000","last_fullsized":"2023-12-26T21:36:48.838484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:25:31.563553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844919+0000","last_change":"2023-12-26T21:36:47.832588+0000","last_active":"2023-12-26T21:36:48.844919+0000","last_peered":"2023-12-26T21:36:48.844919+0000","last_clean":"2023-12-26T21:36:48.844919+0000","last_became_active":"2023-12-26T21:36:47.832310+0000","last_became_peered":"2023-12-26T21:36:47.832310+0000","last_unstale":"2023-12-26T21:36:48.844919+0000","last_undegraded":"2023-12-26T21:36:48.844919+0000","last_fullsized":"2023-12-26T21:36:48.844919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:22:54.126845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839479+0000","last_change":"2023-12-26T21:36:48.830882+0000","last_active":"2023-12-26T21:36:48.839479+0000","last_peered":"2023-12-26T21:36:48.839479+0000","last_clean":"2023-12-26T21:36:48.839479+0000","last_became_active":"2023-12-26T21:36:48.830587+0000","last_became_peered":"2023-12-26T21:36:48.830587+0000","last_unstale":"2023-12-26T21:36:48.839479+0000","last_undegraded":"2023-12-26T21:36:48.839479+0000","last_fullsized":"2023-12-26T21:36:48.839479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:54.746974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826954+0000","last_change":"2023-12-26T21:36:47.826141+0000","last_active":"2023-12-26T21:36:48.826954+0000","last_peered":"2023-12-26T21:36:48.826954+0000","last_clean":"2023-12-26T21:36:48.826954+0000","last_became_active":"2023-12-26T21:36:47.825876+0000","last_became_peered":"2023-12-26T21:36:47.825876+0000","last_unstale":"2023-12-26T21:36:48.826954+0000","last_undegraded":"2023-12-26T21:36:48.826954+0000","last_fullsized":"2023-12-26T21:36:48.826954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:12.184779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843719+0000","last_change":"2023-12-26T21:36:48.829405+0000","last_active":"2023-12-26T21:36:48.843719+0000","last_peered":"2023-12-26T21:36:48.843719+0000","last_clean":"2023-12-26T21:36:48.843719+0000","last_became_active":"2023-12-26T21:36:48.829169+0000","last_became_peered":"2023-12-26T21:36:48.829169+0000","last_unstale":"2023-12-26T21:36:48.843719+0000","last_undegraded":"2023-12-26T21:36:48.843719+0000","last_fullsized":"2023-12-26T21:36:48.843719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:42:15.750967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.819202+0000","last_change":"2023-12-26T21:36:47.830820+0000","last_active":"2023-12-26T21:36:48.819202+0000","last_peered":"2023-12-26T21:36:48.819202+0000","last_clean":"2023-12-26T21:36:48.819202+0000","last_became_active":"2023-12-26T21:36:47.830514+0000","last_became_peered":"2023-12-26T21:36:47.830514+0000","last_unstale":"2023-12-26T21:36:48.819202+0000","last_undegraded":"2023-12-26T21:36:48.819202+0000","last_fullsized":"2023-12-26T21:36:48.819202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:53.472268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839858+0000","last_change":"2023-12-26T21:36:48.832982+0000","last_active":"2023-12-26T21:36:48.839858+0000","last_peered":"2023-12-26T21:36:48.839858+0000","last_clean":"2023-12-26T21:36:48.839858+0000","last_became_active":"2023-12-26T21:36:48.831969+0000","last_became_peered":"2023-12-26T21:36:48.831969+0000","last_unstale":"2023-12-26T21:36:48.839858+0000","last_undegraded":"2023-12-26T21:36:48.839858+0000","last_fullsized":"2023-12-26T21:36:48.839858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:54.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.812495+0000","last_change":"2023-12-26T21:36:47.826837+0000","last_active":"2023-12-26T21:36:48.812495+0000","last_peered":"2023-12-26T21:36:48.812495+0000","last_clean":"2023-12-26T21:36:48.812495+0000","last_became_active":"2023-12-26T21:36:47.826479+0000","last_became_peered":"2023-12-26T21:36:47.826479+0000","last_unstale":"2023-12-26T21:36:48.812495+0000","last_undegraded":"2023-12-26T21:36:48.812495+0000","last_fullsized":"2023-12-26T21:36:48.812495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:05.573200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.817414+0000","last_change":"2023-12-26T21:36:48.813654+0000","last_active":"2023-12-26T21:36:48.817414+0000","last_peered":"2023-12-26T21:36:48.817414+0000","last_clean":"2023-12-26T21:36:48.817414+0000","last_became_active":"2023-12-26T21:36:48.813344+0000","last_became_peered":"2023-12-26T21:36:48.813344+0000","last_unstale":"2023-12-26T21:36:48.817414+0000","last_undegraded":"2023-12-26T21:36:48.817414+0000","last_fullsized":"2023-12-26T21:36:48.817414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:54.478693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839092+0000","last_change":"2023-12-26T21:36:47.820913+0000","last_active":"2023-12-26T21:36:48.839092+0000","last_peered":"2023-12-26T21:36:48.839092+0000","last_clean":"2023-12-26T21:36:48.839092+0000","last_became_active":"2023-12-26T21:36:47.820617+0000","last_became_peered":"2023-12-26T21:36:47.820617+0000","last_unstale":"2023-12-26T21:36:48.839092+0000","last_undegraded":"2023-12-26T21:36:48.839092+0000","last_fullsized":"2023-12-26T21:36:48.839092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:41.047506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.829801+0000","last_change":"2023-12-26T21:36:48.826060+0000","last_active":"2023-12-26T21:36:48.829801+0000","last_peered":"2023-12-26T21:36:48.829801+0000","last_clean":"2023-12-26T21:36:48.829801+0000","last_became_active":"2023-12-26T21:36:48.825817+0000","last_became_peered":"2023-12-26T21:36:48.825817+0000","last_unstale":"2023-12-26T21:36:48.829801+0000","last_undegraded":"2023-12-26T21:36:48.829801+0000","last_fullsized":"2023-12-26T21:36:48.829801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:21.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.809730+0000","last_change":"2023-12-26T21:36:47.827562+0000","last_active":"2023-12-26T21:36:48.809730+0000","last_peered":"2023-12-26T21:36:48.809730+0000","last_clean":"2023-12-26T21:36:48.809730+0000","last_became_active":"2023-12-26T21:36:47.827284+0000","last_became_peered":"2023-12-26T21:36:47.827284+0000","last_unstale":"2023-12-26T21:36:48.809730+0000","last_undegraded":"2023-12-26T21:36:48.809730+0000","last_fullsized":"2023-12-26T21:36:48.809730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:13.148760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843878+0000","last_change":"2023-12-26T21:36:48.829818+0000","last_active":"2023-12-26T21:36:48.843878+0000","last_peered":"2023-12-26T21:36:48.843878+0000","last_clean":"2023-12-26T21:36:48.843878+0000","last_became_active":"2023-12-26T21:36:48.829087+0000","last_became_peered":"2023-12-26T21:36:48.829087+0000","last_unstale":"2023-12-26T21:36:48.843878+0000","last_undegraded":"2023-12-26T21:36:48.843878+0000","last_fullsized":"2023-12-26T21:36:48.843878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:37:13.353768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.842747+0000","last_change":"2023-12-26T21:36:47.815095+0000","last_active":"2023-12-26T21:36:48.842747+0000","last_peered":"2023-12-26T21:36:48.842747+0000","last_clean":"2023-12-26T21:36:48.842747+0000","last_became_active":"2023-12-26T21:36:47.814728+0000","last_became_peered":"2023-12-26T21:36:47.814728+0000","last_unstale":"2023-12-26T21:36:48.842747+0000","last_undegraded":"2023-12-26T21:36:48.842747+0000","last_fullsized":"2023-12-26T21:36:48.842747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:14.650975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846566+0000","last_change":"2023-12-26T21:36:48.837315+0000","last_active":"2023-12-26T21:36:48.846566+0000","last_peered":"2023-12-26T21:36:48.846566+0000","last_clean":"2023-12-26T21:36:48.846566+0000","last_became_active":"2023-12-26T21:36:48.837067+0000","last_became_peered":"2023-12-26T21:36:48.837067+0000","last_unstale":"2023-12-26T21:36:48.846566+0000","last_undegraded":"2023-12-26T21:36:48.846566+0000","last_fullsized":"2023-12-26T21:36:48.846566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:28.883031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.834582+0000","last_change":"2023-12-26T21:36:47.825290+0000","last_active":"2023-12-26T21:36:48.834582+0000","last_peered":"2023-12-26T21:36:48.834582+0000","last_clean":"2023-12-26T21:36:48.834582+0000","last_became_active":"2023-12-26T21:36:47.824594+0000","last_became_peered":"2023-12-26T21:36:47.824594+0000","last_unstale":"2023-12-26T21:36:48.834582+0000","last_undegraded":"2023-12-26T21:36:48.834582+0000","last_fullsized":"2023-12-26T21:36:48.834582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:46.783489+0000","last_clean_scrub_stamp":"2023-12-26T21:36:46.783489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:06.749340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.843529+0000","last_change":"2023-12-26T21:36:48.835342+0000","last_active":"2023-12-26T21:36:48.843529+0000","last_peered":"2023-12-26T21:36:48.843529+0000","last_clean":"2023-12-26T21:36:48.843529+0000","last_became_active":"2023-12-26T21:36:48.830463+0000","last_became_peered":"2023-12-26T21:36:48.830463+0000","last_unstale":"2023-12-26T21:36:48.843529+0000","last_undegraded":"2023-12-26T21:36:48.843529+0000","last_fullsized":"2023-12-26T21:36:48.843529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:50.422360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.824113+0000","last_change":"2023-12-26T21:36:48.817508+0000","last_active":"2023-12-26T21:36:48.824113+0000","last_peered":"2023-12-26T21:36:48.824113+0000","last_clean":"2023-12-26T21:36:48.824113+0000","last_became_active":"2023-12-26T21:36:48.817269+0000","last_became_peered":"2023-12-26T21:36:48.817269+0000","last_unstale":"2023-12-26T21:36:48.824113+0000","last_undegraded":"2023-12-26T21:36:48.824113+0000","last_fullsized":"2023-12-26T21:36:48.824113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:54.478693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.831298+0000","last_change":"2023-12-26T21:36:48.820601+0000","last_active":"2023-12-26T21:36:48.831298+0000","last_peered":"2023-12-26T21:36:48.831298+0000","last_clean":"2023-12-26T21:36:48.831298+0000","last_became_active":"2023-12-26T21:36:48.819293+0000","last_became_peered":"2023-12-26T21:36:48.819293+0000","last_unstale":"2023-12-26T21:36:48.831298+0000","last_undegraded":"2023-12-26T21:36:48.831298+0000","last_fullsized":"2023-12-26T21:36:48.831298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:26.207089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844324+0000","last_change":"2023-12-26T21:36:48.834258+0000","last_active":"2023-12-26T21:36:48.844324+0000","last_peered":"2023-12-26T21:36:48.844324+0000","last_clean":"2023-12-26T21:36:48.844324+0000","last_became_active":"2023-12-26T21:36:48.834005+0000","last_became_peered":"2023-12-26T21:36:48.834005+0000","last_unstale":"2023-12-26T21:36:48.844324+0000","last_undegraded":"2023-12-26T21:36:48.844324+0000","last_fullsized":"2023-12-26T21:36:48.844324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:15.953197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841827+0000","last_change":"2023-12-26T21:36:48.834061+0000","last_active":"2023-12-26T21:36:48.841827+0000","last_peered":"2023-12-26T21:36:48.841827+0000","last_clean":"2023-12-26T21:36:48.841827+0000","last_became_active":"2023-12-26T21:36:48.821177+0000","last_became_peered":"2023-12-26T21:36:48.821177+0000","last_unstale":"2023-12-26T21:36:48.841827+0000","last_undegraded":"2023-12-26T21:36:48.841827+0000","last_fullsized":"2023-12-26T21:36:48.841827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:21:25.931065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828251+0000","last_change":"2023-12-26T21:36:48.823718+0000","last_active":"2023-12-26T21:36:48.828251+0000","last_peered":"2023-12-26T21:36:48.828251+0000","last_clean":"2023-12-26T21:36:48.828251+0000","last_became_active":"2023-12-26T21:36:48.823422+0000","last_became_peered":"2023-12-26T21:36:48.823422+0000","last_unstale":"2023-12-26T21:36:48.828251+0000","last_undegraded":"2023-12-26T21:36:48.828251+0000","last_fullsized":"2023-12-26T21:36:48.828251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:42:23.347556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.826781+0000","last_change":"2023-12-26T21:36:48.821819+0000","last_active":"2023-12-26T21:36:48.826781+0000","last_peered":"2023-12-26T21:36:48.826781+0000","last_clean":"2023-12-26T21:36:48.826781+0000","last_became_active":"2023-12-26T21:36:48.821527+0000","last_became_peered":"2023-12-26T21:36:48.821527+0000","last_unstale":"2023-12-26T21:36:48.826781+0000","last_undegraded":"2023-12-26T21:36:48.826781+0000","last_fullsized":"2023-12-26T21:36:48.826781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:31.058793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.844139+0000","last_change":"2023-12-26T21:36:48.841186+0000","last_active":"2023-12-26T21:36:48.844139+0000","last_peered":"2023-12-26T21:36:48.844139+0000","last_clean":"2023-12-26T21:36:48.844139+0000","last_became_active":"2023-12-26T21:36:48.839955+0000","last_became_peered":"2023-12-26T21:36:48.839955+0000","last_unstale":"2023-12-26T21:36:48.844139+0000","last_undegraded":"2023-12-26T21:36:48.844139+0000","last_fullsized":"2023-12-26T21:36:48.844139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:35.307950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841419+0000","last_change":"2023-12-26T21:36:48.832556+0000","last_active":"2023-12-26T21:36:48.841419+0000","last_peered":"2023-12-26T21:36:48.841419+0000","last_clean":"2023-12-26T21:36:48.841419+0000","last_became_active":"2023-12-26T21:36:48.830061+0000","last_became_peered":"2023-12-26T21:36:48.830061+0000","last_unstale":"2023-12-26T21:36:48.841419+0000","last_undegraded":"2023-12-26T21:36:48.841419+0000","last_fullsized":"2023-12-26T21:36:48.841419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:22.938456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.846674+0000","last_change":"2023-12-26T21:36:48.842574+0000","last_active":"2023-12-26T21:36:48.846674+0000","last_peered":"2023-12-26T21:36:48.846674+0000","last_clean":"2023-12-26T21:36:48.846674+0000","last_became_active":"2023-12-26T21:36:48.842273+0000","last_became_peered":"2023-12-26T21:36:48.842273+0000","last_unstale":"2023-12-26T21:36:48.846674+0000","last_undegraded":"2023-12-26T21:36:48.846674+0000","last_fullsized":"2023-12-26T21:36:48.846674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:17:59.482430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.840028+0000","last_change":"2023-12-26T21:36:48.826270+0000","last_active":"2023-12-26T21:36:48.840028+0000","last_peered":"2023-12-26T21:36:48.840028+0000","last_clean":"2023-12-26T21:36:48.840028+0000","last_became_active":"2023-12-26T21:36:48.825303+0000","last_became_peered":"2023-12-26T21:36:48.825303+0000","last_unstale":"2023-12-26T21:36:48.840028+0000","last_undegraded":"2023-12-26T21:36:48.840028+0000","last_fullsized":"2023-12-26T21:36:48.840028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:22.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.828348+0000","last_change":"2023-12-26T21:36:48.823561+0000","last_active":"2023-12-26T21:36:48.828348+0000","last_peered":"2023-12-26T21:36:48.828348+0000","last_clean":"2023-12-26T21:36:48.828348+0000","last_became_active":"2023-12-26T21:36:48.823270+0000","last_became_peered":"2023-12-26T21:36:48.823270+0000","last_unstale":"2023-12-26T21:36:48.828348+0000","last_undegraded":"2023-12-26T21:36:48.828348+0000","last_fullsized":"2023-12-26T21:36:48.828348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:19:29.032786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.839419+0000","last_change":"2023-12-26T21:36:48.828687+0000","last_active":"2023-12-26T21:36:48.839419+0000","last_peered":"2023-12-26T21:36:48.839419+0000","last_clean":"2023-12-26T21:36:48.839419+0000","last_became_active":"2023-12-26T21:36:48.828405+0000","last_became_peered":"2023-12-26T21:36:48.828405+0000","last_unstale":"2023-12-26T21:36:48.839419+0000","last_undegraded":"2023-12-26T21:36:48.839419+0000","last_fullsized":"2023-12-26T21:36:48.839419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:37.832638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-26T21:36:48.841288+0000","last_change":"2023-12-26T21:36:48.835378+0000","last_active":"2023-12-26T21:36:48.841288+0000","last_peered":"2023-12-26T21:36:48.841288+0000","last_clean":"2023-12-26T21:36:48.841288+0000","last_became_active":"2023-12-26T21:36:48.835049+0000","last_became_peered":"2023-12-26T21:36:48.835049+0000","last_unstale":"2023-12-26T21:36:48.841288+0000","last_undegraded":"2023-12-26T21:36:48.841288+0000","last_fullsized":"2023-12-26T21:36:48.841288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:47.789914+0000","last_clean_scrub_stamp":"2023-12-26T21:36:47.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:22.938456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":76900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":667360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":79080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":78436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]}]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":77024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0150000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298088,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073752,"statfs":{"total":96636764160,"available":96331522048,"internally_reserved":0,"allocated":819200,"data_stored":667338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":77454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:36:59.323 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-26T21:36:59.323 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-26T21:36:59.323 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-26T21:36:59.324 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-26T21:36:59.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.547+0000 7f4ed0fff700 1 Processor -- start 2023-12-26T21:36:59.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.547+0000 7f4ed0fff700 1 -- start start 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.548+0000 7f4ed0fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ecc13c510 0x7f4ecc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.548+0000 7f4ed0fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ecc06b8c0 0x7f4ecc06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.548+0000 7f4ed0fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0686c0 con 0x7f4ecc0cc100 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.548+0000 7f4ed0fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc06c280 con 0x7f4ecc13c510 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.548+0000 7f4ed0fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc069520 con 0x7f4ecc06b8c0 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.548+0000 7f4eca59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ecc13c510 0x7f4ecc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4eca59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ecc13c510 0x7f4ecc0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45100/0 (socket says 172.21.15.131:45100) 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4eca59c700 1 -- 172.21.15.131:0/2757282497 learned_addr learned my addr 172.21.15.131:0/2757282497 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4ec9d9b700 1 --2- 172.21.15.131:0/2757282497 >> v2:172.21.15.190:3300/0 conn(0x7f4ecc06b8c0 0x7f4ecc06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4eca59c700 1 -- 172.21.15.131:0/2757282497 >> v2:172.21.15.190:3300/0 conn(0x7f4ecc06b8c0 msgr2=0x7f4ecc06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:36:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4eca59c700 1 --2- 172.21.15.131:0/2757282497 >> v2:172.21.15.190:3300/0 conn(0x7f4ecc06b8c0 0x7f4ecc06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4eca59c700 1 -- 172.21.15.131:0/2757282497 >> v1:172.21.15.190:6789/0 conn(0x7f4ecc0cc100 legacy=0x7f4ecc0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4eca59c700 1 -- 172.21.15.131:0/2757282497 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc068910 con 0x7f4ecc13c510 2023-12-26T21:36:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4eca59c700 1 --2- 172.21.15.131:0/2757282497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc13c510 0x7f4ecc0c8a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0b43e0 tx=0x7f4ebc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55590a0cbbe2ece7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4ec959a700 1 -- 172.21.15.131:0/2757282497 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc005c10 con 0x7f4ecc13c510 2023-12-26T21:36:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4ec959a700 1 -- 172.21.15.131:0/2757282497 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc005d70 con 0x7f4ecc13c510 2023-12-26T21:36:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.549+0000 7f4ec959a700 1 -- 172.21.15.131:0/2757282497 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc006260 con 0x7f4ecc13c510 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.550+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2757282497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc13c510 msgr2=0x7f4ecc0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.550+0000 7f4ed0fff700 1 --2- 172.21.15.131:0/2757282497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc13c510 0x7f4ecc0c8a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0b43e0 tx=0x7f4ebc009220 comp rx=0 tx=0).stop 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.550+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2757282497 shutdown_connections 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.550+0000 7f4ed0fff700 1 --2- 172.21.15.131:0/2757282497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc13c510 0x7f4ecc0c8a20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.550+0000 7f4ed0fff700 1 --2- 172.21.15.131:0/2757282497 >> v2:172.21.15.190:3300/0 conn(0x7f4ecc06b8c0 0x7f4ecc06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.550+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2757282497 >> 172.21.15.131:0/2757282497 conn(0x7f4ecc0ce3e0 msgr2=0x7f4ecc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.550+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2757282497 shutdown_connections 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.550+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2757282497 wait complete. 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.552+0000 7f4ed0fff700 1 Processor -- start 2023-12-26T21:36:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.552+0000 7f4ed0fff700 1 -- start start 2023-12-26T21:36:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ed0fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc06b8c0 0x7f4ecc155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ed0fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cc100 0x7f4ecc0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ed0fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc13c510 0x7f4ecc14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ed0fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc155560 con 0x7f4ecc0cc100 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ed0fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0b16c0 con 0x7f4ecc06b8c0 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ed0fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc16e0b0 con 0x7f4ecc13c510 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ec9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cc100 0x7f4ecc0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ec9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cc100 0x7f4ecc0b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45104/0 (socket says 172.21.15.131:45104) 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ec9d9b700 1 -- 172.21.15.131:0/2184420569 learned_addr learned my addr 172.21.15.131:0/2184420569 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4eca59c700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc06b8c0 0x7f4ecc155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ecad9d700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc13c510 0x7f4ecc14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ec9d9b700 1 -- 172.21.15.131:0/2184420569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc13c510 msgr2=0x7f4ecc14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ec9d9b700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc13c510 0x7f4ecc14fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ec9d9b700 1 -- 172.21.15.131:0/2184420569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc06b8c0 msgr2=0x7f4ecc155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ec9d9b700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc06b8c0 0x7f4ecc155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ec9d9b700 1 -- 172.21.15.131:0/2184420569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc009480 con 0x7f4ecc0cc100 2023-12-26T21:36:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.553+0000 7f4ec9d9b700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cc100 0x7f4ecc0b1180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e930 tx=0x7f4ec00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.554+0000 7f4eba7fc700 1 -- 172.21.15.131:0/2184420569 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0013680 con 0x7f4ecc0cc100 2023-12-26T21:36:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.554+0000 7f4eba7fc700 1 -- 172.21.15.131:0/2184420569 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec00137e0 con 0x7f4ecc0cc100 2023-12-26T21:36:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.554+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc16d200 con 0x7f4ecc0cc100 2023-12-26T21:36:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.554+0000 7f4eba7fc700 1 -- 172.21.15.131:0/2184420569 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0013b80 con 0x7f4ecc0cc100 2023-12-26T21:36:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.554+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc16d550 con 0x7f4ecc0cc100 2023-12-26T21:36:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.555+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc0657e0 con 0x7f4ecc0cc100 2023-12-26T21:36:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.556+0000 7f4eba7fc700 1 -- 172.21.15.131:0/2184420569 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4ec000c460 con 0x7f4ecc0cc100 2023-12-26T21:36:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.557+0000 7f4eba7fc700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4eac066f30 0x7f4eac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:36:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.557+0000 7f4eba7fc700 1 -- 172.21.15.131:0/2184420569 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ec0084740 con 0x7f4ecc0cc100 2023-12-26T21:36:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.557+0000 7f4eca59c700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4eac066f30 0x7f4eac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:36:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.558+0000 7f4eca59c700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4eac066f30 0x7f4eac0693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009260 tx=0x7f4ebc016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:36:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.561+0000 7f4eba7fc700 1 -- 172.21.15.131:0/2184420569 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ec0051910 con 0x7f4ecc0cc100 2023-12-26T21:36:59.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.793+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4ecc068c50 con 0x7f4ecc0cc100 2023-12-26T21:36:59.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.794+0000 7f4eba7fc700 1 -- 172.21.15.131:0/2184420569 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4ec0054f30 con 0x7f4ecc0cc100 2023-12-26T21:36:59.793 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:36:59.794 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4eac066f30 msgr2=0x7f4eac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4eac066f30 0x7f4eac0693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009260 tx=0x7f4ebc016000 comp rx=0 tx=0).stop 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cc100 msgr2=0x7f4ecc0b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cc100 0x7f4ecc0b1180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e930 tx=0x7f4ec00091a0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 shutdown_connections 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4eac066f30 0x7f4eac0693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc13c510 0x7f4ecc14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc06b8c0 0x7f4ecc155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 --2- 172.21.15.131:0/2184420569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cc100 0x7f4ecc0b1180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 >> 172.21.15.131:0/2184420569 conn(0x7f4ecc0ce3e0 msgr2=0x7f4ecc13bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 shutdown_connections 2023-12-26T21:36:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:36:59.796+0000 7f4ed0fff700 1 -- 172.21.15.131:0/2184420569 wait complete. 2023-12-26T21:36:59.813 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-26T21:36:59.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.005+0000 7fd40a3ff700 1 Processor -- start 2023-12-26T21:37:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.005+0000 7fd40a3ff700 1 -- start start 2023-12-26T21:37:00.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd40a3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd404159a70 0x7fd4040b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd40a3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4040b2a90 0x7fd4040b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd40a3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4040afcd0 con 0x7fd4041596a0 2023-12-26T21:37:00.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd40a3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4040b3e70 con 0x7fd404159a70 2023-12-26T21:37:00.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd40a3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4040b0f20 con 0x7fd4040b2a90 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd4037fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd404159a70 0x7fd4040b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd4037fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd404159a70 0x7fd4040b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45106/0 (socket says 172.21.15.131:45106) 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd4037fe700 1 -- 172.21.15.131:0/141434024 learned_addr learned my addr 172.21.15.131:0/141434024 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd403fff700 1 --1- 172.21.15.131:0/141434024 >> v1:172.21.15.131:6789/0 conn(0x7fd4041596a0 0x7fd4040b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49496/0 (socket says 172.21.15.131:49496) 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.006+0000 7fd40899c700 1 --2- 172.21.15.131:0/141434024 >> v2:172.21.15.190:3300/0 conn(0x7fd4040b2a90 0x7fd4040b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.007+0000 7fd4037fe700 1 -- 172.21.15.131:0/141434024 >> v2:172.21.15.190:3300/0 conn(0x7fd4040b2a90 msgr2=0x7fd4040b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.007+0000 7fd4037fe700 1 --2- 172.21.15.131:0/141434024 >> v2:172.21.15.190:3300/0 conn(0x7fd4040b2a90 0x7fd4040b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.007+0000 7fd4037fe700 1 -- 172.21.15.131:0/141434024 >> v1:172.21.15.131:6789/0 conn(0x7fd4041596a0 legacy=0x7fd4040b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.007+0000 7fd4037fe700 1 -- 172.21.15.131:0/141434024 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4040b3540 con 0x7fd404159a70 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.007+0000 7fd4037fe700 1 --2- 172.21.15.131:0/141434024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd404159a70 0x7fd4040b0160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd4040b66b0 tx=0x7fd3f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73666f4fc693dbbe server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.007+0000 7fd402ffd700 1 -- 172.21.15.131:0/141434024 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8009e70 con 0x7fd404159a70 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.007+0000 7fd402ffd700 1 -- 172.21.15.131:0/141434024 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f800b860 con 0x7fd404159a70 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.007+0000 7fd402ffd700 1 -- 172.21.15.131:0/141434024 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800bc00 con 0x7fd404159a70 2023-12-26T21:37:00.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.008+0000 7fd40a3ff700 1 -- 172.21.15.131:0/141434024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd404159a70 msgr2=0x7fd4040b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.008+0000 7fd40a3ff700 1 --2- 172.21.15.131:0/141434024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd404159a70 0x7fd4040b0160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd4040b66b0 tx=0x7fd3f8009250 comp rx=0 tx=0).stop 2023-12-26T21:37:00.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.008+0000 7fd40a3ff700 1 -- 172.21.15.131:0/141434024 shutdown_connections 2023-12-26T21:37:00.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.008+0000 7fd40a3ff700 1 --2- 172.21.15.131:0/141434024 >> v2:172.21.15.190:3300/0 conn(0x7fd4040b2a90 0x7fd4040b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.008+0000 7fd40a3ff700 1 --2- 172.21.15.131:0/141434024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd404159a70 0x7fd4040b0160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.008+0000 7fd40a3ff700 1 -- 172.21.15.131:0/141434024 >> 172.21.15.131:0/141434024 conn(0x7fd40406ece0 msgr2=0x7fd40406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:00.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.008+0000 7fd40a3ff700 1 -- 172.21.15.131:0/141434024 shutdown_connections 2023-12-26T21:37:00.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.008+0000 7fd40a3ff700 1 -- 172.21.15.131:0/141434024 wait complete. 2023-12-26T21:37:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.009+0000 7fd40a3ff700 1 Processor -- start 2023-12-26T21:37:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.009+0000 7fd40a3ff700 1 -- start start 2023-12-26T21:37:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.010+0000 7fd40a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4040b2a90 0x7fd4040cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.010+0000 7fd40a3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4041596a0 0x7fd4040c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.010+0000 7fd40a3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd404159a70 0x7fd4040c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.010+0000 7fd40a3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040c6f10 con 0x7fd4041596a0 2023-12-26T21:37:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.010+0000 7fd40a3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040cc460 con 0x7fd404159a70 2023-12-26T21:37:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.010+0000 7fd40a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040c4e90 con 0x7fd4040b2a90 2023-12-26T21:37:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd4037fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4041596a0 0x7fd4040c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd4037fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4041596a0 0x7fd4040c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45108/0 (socket says 172.21.15.131:45108) 2023-12-26T21:37:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd4037fe700 1 -- 172.21.15.131:0/1112943589 learned_addr learned my addr 172.21.15.131:0/1112943589 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd40899c700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd404159a70 0x7fd4040c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd403fff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4040b2a90 0x7fd4040cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd4037fe700 1 -- 172.21.15.131:0/1112943589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4040b2a90 msgr2=0x7fd4040cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd4037fe700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4040b2a90 0x7fd4040cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd4037fe700 1 -- 172.21.15.131:0/1112943589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd404159a70 msgr2=0x7fd4040c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd4037fe700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd404159a70 0x7fd4040c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd4037fe700 1 -- 172.21.15.131:0/1112943589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80094f0 con 0x7fd4041596a0 2023-12-26T21:37:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd40899c700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd404159a70 0x7fd4040c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd4037fe700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4041596a0 0x7fd4040c69d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd404089710 tx=0x7fd3f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd3ebfff700 1 -- 172.21.15.131:0/1112943589 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8011420 con 0x7fd4041596a0 2023-12-26T21:37:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd3ebfff700 1 -- 172.21.15.131:0/1112943589 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f80115a0 con 0x7fd4041596a0 2023-12-26T21:37:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4040cb850 con 0x7fd4041596a0 2023-12-26T21:37:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.011+0000 7fd3ebfff700 1 -- 172.21.15.131:0/1112943589 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8011960 con 0x7fd4041596a0 2023-12-26T21:37:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.012+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4040cbe10 con 0x7fd4041596a0 2023-12-26T21:37:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.012+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4040657e0 con 0x7fd4041596a0 2023-12-26T21:37:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.013+0000 7fd3ebfff700 1 -- 172.21.15.131:0/1112943589 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd3f8011d50 con 0x7fd4041596a0 2023-12-26T21:37:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.015+0000 7fd3ebfff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd3f0066f30 0x7fd3f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.015+0000 7fd3ebfff700 1 -- 172.21.15.131:0/1112943589 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd3f8085b90 con 0x7fd4041596a0 2023-12-26T21:37:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.015+0000 7fd403fff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd3f0066f30 0x7fd3f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.015+0000 7fd403fff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd3f0066f30 0x7fd3f00693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4007fd0 tx=0x7fd3f4007d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.019+0000 7fd3ebfff700 1 -- 172.21.15.131:0/1112943589 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3f8052d60 con 0x7fd4041596a0 2023-12-26T21:37:00.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.217+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4040b3820 con 0x7fd4041596a0 2023-12-26T21:37:00.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.218+0000 7fd3ebfff700 1 -- 172.21.15.131:0/1112943589 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fd3f8056380 con 0x7fd4041596a0 2023-12-26T21:37:00.217 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:00.218 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd3f0066f30 msgr2=0x7fd3f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd3f0066f30 0x7fd3f00693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4007fd0 tx=0x7fd3f4007d70 comp rx=0 tx=0).stop 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4041596a0 msgr2=0x7fd4040c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4041596a0 0x7fd4040c69d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd404089710 tx=0x7fd3f8017000 comp rx=0 tx=0).stop 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 shutdown_connections 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd404159a70 0x7fd4040c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd3f0066f30 0x7fd3f00693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4040b2a90 0x7fd4040cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 --2- 172.21.15.131:0/1112943589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4041596a0 0x7fd4040c69d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 >> 172.21.15.131:0/1112943589 conn(0x7fd40406ece0 msgr2=0x7fd4040ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:00.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 shutdown_connections 2023-12-26T21:37:00.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.220+0000 7fd40a3ff700 1 -- 172.21.15.131:0/1112943589 wait complete. 2023-12-26T21:37:00.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:36:46.654972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:36:47.264880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:37:00.239 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:37:00.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.430+0000 7ff59b3ff700 1 Processor -- start 2023-12-26T21:37:00.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.430+0000 7ff59b3ff700 1 -- start start 2023-12-26T21:37:00.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.431+0000 7ff59b3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff5940c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.431+0000 7ff59b3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff59406b8c0 0x7ff59406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.431+0000 7ff59b3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59406c270 con 0x7ff5940cc100 2023-12-26T21:37:00.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.431+0000 7ff59b3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5940686c0 con 0x7ff59406b8c0 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.431+0000 7ff59b3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff594069520 con 0x7ff59413c510 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59899a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff59406b8c0 0x7ff59406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59919b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff5940c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59999c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff5940cc100 0x7ff5940cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39878/0 (socket says 172.21.15.131:39878) 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59919b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff5940c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57632/0 (socket says 172.21.15.131:57632) 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59999c700 1 -- 172.21.15.131:0/2325254362 learned_addr learned my addr 172.21.15.131:0/2325254362 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59899a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff59406b8c0 0x7ff59406bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45120/0 (socket says 172.21.15.131:45120) 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59899a700 1 -- 172.21.15.131:0/2325254362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 msgr2=0x7ff5940c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59899a700 1 --2- 172.21.15.131:0/2325254362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff5940c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59899a700 1 -- 172.21.15.131:0/2325254362 >> v1:172.21.15.190:6789/0 conn(0x7ff5940cc100 legacy=0x7ff5940cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59899a700 1 -- 172.21.15.131:0/2325254362 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594068910 con 0x7ff59406b8c0 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59919b700 1 --2- 172.21.15.131:0/2325254362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff5940c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff58bfff700 1 -- 172.21.15.131:0/2325254362 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048783091 0 0) 0x7ff59406c270 con 0x7ff5940cc100 2023-12-26T21:37:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.432+0000 7ff59899a700 1 --2- 172.21.15.131:0/2325254362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59406b8c0 0x7ff59406bd30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5840098e0 tx=0x7ff5840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2e8e8aac67cccc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff58bfff700 1 -- 172.21.15.131:0/2325254362 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400b640 con 0x7ff59406b8c0 2023-12-26T21:37:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff58bfff700 1 -- 172.21.15.131:0/2325254362 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58400b7a0 con 0x7ff59406b8c0 2023-12-26T21:37:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff58bfff700 1 -- 172.21.15.131:0/2325254362 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400bb40 con 0x7ff59406b8c0 2023-12-26T21:37:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff59b3ff700 1 -- 172.21.15.131:0/2325254362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59406b8c0 msgr2=0x7ff59406bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff59b3ff700 1 --2- 172.21.15.131:0/2325254362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59406b8c0 0x7ff59406bd30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5840098e0 tx=0x7ff5840091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff59b3ff700 1 -- 172.21.15.131:0/2325254362 shutdown_connections 2023-12-26T21:37:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff59b3ff700 1 --2- 172.21.15.131:0/2325254362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff5940c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff59b3ff700 1 --2- 172.21.15.131:0/2325254362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59406b8c0 0x7ff59406bd30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff59b3ff700 1 -- 172.21.15.131:0/2325254362 >> 172.21.15.131:0/2325254362 conn(0x7ff5940ce3e0 msgr2=0x7ff5940ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff59b3ff700 1 -- 172.21.15.131:0/2325254362 shutdown_connections 2023-12-26T21:37:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.433+0000 7ff59b3ff700 1 -- 172.21.15.131:0/2325254362 wait complete. 2023-12-26T21:37:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.435+0000 7ff59b3ff700 1 Processor -- start 2023-12-26T21:37:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.435+0000 7ff59b3ff700 1 -- start start 2023-12-26T21:37:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59b3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59406b8c0 0x7ff59413f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59b3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5940cc100 0x7ff594169c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59b3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff59416a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59b3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59416a710 con 0x7ff5940cc100 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59b3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff594157f80 con 0x7ff59406b8c0 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59b3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff594157970 con 0x7ff59413c510 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59899a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5940cc100 0x7ff594169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59899a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5940cc100 0x7ff594169c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45136/0 (socket says 172.21.15.131:45136) 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59899a700 1 -- 172.21.15.131:0/4022678800 learned_addr learned my addr 172.21.15.131:0/4022678800 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59919b700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59406b8c0 0x7ff59413f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59899a700 1 -- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 msgr2=0x7ff59416a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59999c700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff59416a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59899a700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff59416a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59899a700 1 -- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59406b8c0 msgr2=0x7ff59413f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59899a700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59406b8c0 0x7ff59413f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59899a700 1 -- 172.21.15.131:0/4022678800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5900093c0 con 0x7ff5940cc100 2023-12-26T21:37:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59919b700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59406b8c0 0x7ff59413f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.436+0000 7ff59999c700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff59416a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.437+0000 7ff59899a700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5940cc100 0x7ff594169c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff59406fe40 tx=0x7ff584017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.437+0000 7ff5897fa700 1 -- 172.21.15.131:0/4022678800 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400bb20 con 0x7ff5940cc100 2023-12-26T21:37:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.437+0000 7ff5897fa700 1 -- 172.21.15.131:0/4022678800 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff584011420 con 0x7ff5940cc100 2023-12-26T21:37:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.437+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff584009420 con 0x7ff5940cc100 2023-12-26T21:37:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.437+0000 7ff5897fa700 1 -- 172.21.15.131:0/4022678800 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5840117a0 con 0x7ff5940cc100 2023-12-26T21:37:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.437+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5940b3d20 con 0x7ff5940cc100 2023-12-26T21:37:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.438+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5941580f0 con 0x7ff5940cc100 2023-12-26T21:37:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.439+0000 7ff5897fa700 1 -- 172.21.15.131:0/4022678800 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff584011b50 con 0x7ff5940cc100 2023-12-26T21:37:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.440+0000 7ff5897fa700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff578066f30 0x7ff5780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.440+0000 7ff5897fa700 1 -- 172.21.15.131:0/4022678800 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff584085c80 con 0x7ff5940cc100 2023-12-26T21:37:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.440+0000 7ff59919b700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff578066f30 0x7ff5780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.440+0000 7ff59919b700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff578066f30 0x7ff5780693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff590009390 tx=0x7ff590008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.444+0000 7ff5897fa700 1 -- 172.21.15.131:0/4022678800 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff584052e50 con 0x7ff5940cc100 2023-12-26T21:37:00.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.633+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5940657e0 con 0x7ff5940cc100 2023-12-26T21:37:00.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.633+0000 7ff5897fa700 1 -- 172.21.15.131:0/4022678800 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff584056470 con 0x7ff5940cc100 2023-12-26T21:37:00.633 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:37:00.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff578066f30 msgr2=0x7ff5780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff578066f30 0x7ff5780693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff590009390 tx=0x7ff590008040 comp rx=0 tx=0).stop 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5940cc100 msgr2=0x7ff594169c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5940cc100 0x7ff594169c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff59406fe40 tx=0x7ff584017000 comp rx=0 tx=0).stop 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 shutdown_connections 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff578066f30 0x7ff5780693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59413c510 0x7ff59416a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59406b8c0 0x7ff59413f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 --2- 172.21.15.131:0/4022678800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5940cc100 0x7ff594169c90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 >> 172.21.15.131:0/4022678800 conn(0x7ff5940ce3e0 msgr2=0x7ff594153830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 shutdown_connections 2023-12-26T21:37:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.636+0000 7ff59b3ff700 1 -- 172.21.15.131:0/4022678800 wait complete. 2023-12-26T21:37:00.654 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:37:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.846+0000 7fafa15ff700 1 Processor -- start 2023-12-26T21:37:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.846+0000 7fafa15ff700 1 -- start start 2023-12-26T21:37:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7fafa15ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c0b07f0 0x7faf9c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7fafa15ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faf9c159cf0 0x7faf9c155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7fafa15ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c0c8bf0 con 0x7faf9c159920 2023-12-26T21:37:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7fafa15ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c155f50 con 0x7faf9c159cf0 2023-12-26T21:37:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7fafa15ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0c8e60 con 0x7faf9c0b07f0 2023-12-26T21:37:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7faf9b7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faf9c159cf0 0x7faf9c155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7faf9affd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7faf9c159920 0x7faf9c0b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:39882/0 (socket says 172.21.15.131:39882) 2023-12-26T21:37:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7faf9affd700 1 -- 172.21.15.131:0/68493451 learned_addr learned my addr 172.21.15.131:0/68493451 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7faf9a7fc700 1 --2- 172.21.15.131:0/68493451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c0b07f0 0x7faf9c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7faf9b7fe700 1 -- 172.21.15.131:0/68493451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c0b07f0 msgr2=0x7faf9c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7faf9b7fe700 1 --2- 172.21.15.131:0/68493451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c0b07f0 0x7faf9c1554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7faf9b7fe700 1 -- 172.21.15.131:0/68493451 >> v1:172.21.15.190:6789/0 conn(0x7faf9c159920 legacy=0x7faf9c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7faf9b7fe700 1 -- 172.21.15.131:0/68493451 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0c9140 con 0x7faf9c159cf0 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.848+0000 7faf99ffb700 1 -- 172.21.15.131:0/68493451 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191316952 0 0) 0x7faf9c0c8bf0 con 0x7faf9c159920 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.849+0000 7faf9b7fe700 1 --2- 172.21.15.131:0/68493451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159cf0 0x7faf9c155a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf9400af70 tx=0x7faf9400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8f6a76d8f26ba5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.849+0000 7faf99ffb700 1 -- 172.21.15.131:0/68493451 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9400f070 con 0x7faf9c159cf0 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.849+0000 7faf99ffb700 1 -- 172.21.15.131:0/68493451 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9400a2e0 con 0x7faf9c159cf0 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.849+0000 7faf99ffb700 1 -- 172.21.15.131:0/68493451 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9400cc70 con 0x7faf9c159cf0 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.849+0000 7fafa15ff700 1 -- 172.21.15.131:0/68493451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159cf0 msgr2=0x7faf9c155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.849+0000 7fafa15ff700 1 --2- 172.21.15.131:0/68493451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159cf0 0x7faf9c155a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf9400af70 tx=0x7faf9400a890 comp rx=0 tx=0).stop 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.850+0000 7fafa15ff700 1 -- 172.21.15.131:0/68493451 shutdown_connections 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.850+0000 7fafa15ff700 1 --2- 172.21.15.131:0/68493451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159cf0 0x7faf9c155a10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.850+0000 7fafa15ff700 1 --2- 172.21.15.131:0/68493451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c0b07f0 0x7faf9c1554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.850+0000 7fafa15ff700 1 -- 172.21.15.131:0/68493451 >> 172.21.15.131:0/68493451 conn(0x7faf9c068040 msgr2=0x7faf9c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.850+0000 7fafa15ff700 1 -- 172.21.15.131:0/68493451 shutdown_connections 2023-12-26T21:37:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.850+0000 7fafa15ff700 1 -- 172.21.15.131:0/68493451 wait complete. 2023-12-26T21:37:00.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.851+0000 7fafa15ff700 1 Processor -- start 2023-12-26T21:37:00.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.851+0000 7fafa15ff700 1 -- start start 2023-12-26T21:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.852+0000 7fafa15ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159920 0x7faf9c06ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.852+0000 7fafa15ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf9c159cf0 0x7faf9c0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.852+0000 7fafa15ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c06f1d0 0x7faf9c15d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.852+0000 7fafa15ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0c5440 con 0x7faf9c159920 2023-12-26T21:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.852+0000 7fafa15ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0c5720 con 0x7faf9c159cf0 2023-12-26T21:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.852+0000 7fafa15ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0c5a00 con 0x7faf9c06f1d0 2023-12-26T21:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.852+0000 7faf9affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159920 0x7faf9c06ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.852+0000 7faf9affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159920 0x7faf9c06ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45156/0 (socket says 172.21.15.131:45156) 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.852+0000 7faf9affd700 1 -- 172.21.15.131:0/1240113375 learned_addr learned my addr 172.21.15.131:0/1240113375 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf9a7fc700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf9c159cf0 0x7faf9c0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf9b7fe700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c06f1d0 0x7faf9c15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf9affd700 1 -- 172.21.15.131:0/1240113375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c06f1d0 msgr2=0x7faf9c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf9affd700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c06f1d0 0x7faf9c15d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf9affd700 1 -- 172.21.15.131:0/1240113375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf9c159cf0 msgr2=0x7faf9c0c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf9affd700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf9c159cf0 0x7faf9c0c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf9affd700 1 -- 172.21.15.131:0/1240113375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9400ab30 con 0x7faf9c159920 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf9affd700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159920 0x7faf9c06ea00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c7e0 tx=0x7faf8c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf82ffd700 1 -- 172.21.15.131:0/1240113375 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c005b30 con 0x7faf9c159920 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf82ffd700 1 -- 172.21.15.131:0/1240113375 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c005c90 con 0x7faf9c159920 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c0c5ce0 con 0x7faf9c159920 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7faf82ffd700 1 -- 172.21.15.131:0/1240113375 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c007300 con 0x7faf9c159920 2023-12-26T21:37:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.853+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c06ef40 con 0x7faf9c159920 2023-12-26T21:37:00.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.854+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c0657e0 con 0x7faf9c159920 2023-12-26T21:37:00.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.855+0000 7faf82ffd700 1 -- 172.21.15.131:0/1240113375 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faf8c01b3b0 con 0x7faf9c159920 2023-12-26T21:37:00.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.856+0000 7faf82ffd700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf88066f30 0x7faf880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:00.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.856+0000 7faf82ffd700 1 -- 172.21.15.131:0/1240113375 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faf8c083af0 con 0x7faf9c159920 2023-12-26T21:37:00.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.856+0000 7faf9a7fc700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf88066f30 0x7faf880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:00.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.857+0000 7faf9a7fc700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf88066f30 0x7faf880693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf9c16cef0 tx=0x7faf900091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:00.860+0000 7faf82ffd700 1 -- 172.21.15.131:0/1240113375 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf8c050cc0 con 0x7faf9c159920 2023-12-26T21:37:01.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.060+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf9c0c6200 con 0x7faf9c159920 2023-12-26T21:37:01.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.060+0000 7faf82ffd700 1 -- 172.21.15.131:0/1240113375 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf9c0c6200 con 0x7faf9c159920 2023-12-26T21:37:01.060 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf88066f30 msgr2=0x7faf880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf88066f30 0x7faf880693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf9c16cef0 tx=0x7faf900091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159920 msgr2=0x7faf9c06ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159920 0x7faf9c06ea00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c7e0 tx=0x7faf8c007df0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 shutdown_connections 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf88066f30 0x7faf880693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9c06f1d0 0x7faf9c15d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9c159920 0x7faf9c06ea00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 --2- 172.21.15.131:0/1240113375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf9c159cf0 0x7faf9c0c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 >> 172.21.15.131:0/1240113375 conn(0x7faf9c068040 msgr2=0x7faf9c158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 shutdown_connections 2023-12-26T21:37:01.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.063+0000 7fafa15ff700 1 -- 172.21.15.131:0/1240113375 wait complete. 2023-12-26T21:37:01.080 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:37:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.273+0000 7f2ff29ff700 1 Processor -- start 2023-12-26T21:37:01.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.274+0000 7f2ff29ff700 1 -- start start 2023-12-26T21:37:01.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.275+0000 7f2ff29ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec13c510 0x7f2fec0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.275+0000 7f2ff29ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2fec06b8c0 0x7f2fec06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.275+0000 7f2ff29ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fec06c270 con 0x7f2fec0cc100 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.275+0000 7f2ff29ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fec0686c0 con 0x7f2fec06b8c0 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.275+0000 7f2ff29ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec069520 con 0x7f2fec13c510 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.275+0000 7f2febfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec13c510 0x7f2fec0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.275+0000 7f2febfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec13c510 0x7f2fec0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57676/0 (socket says 172.21.15.131:57676) 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.275+0000 7f2febfff700 1 -- 172.21.15.131:0/93619677 learned_addr learned my addr 172.21.15.131:0/93619677 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.275+0000 7f2feb7fe700 1 --2- 172.21.15.131:0/93619677 >> v2:172.21.15.190:3300/0 conn(0x7f2fec06b8c0 0x7f2fec06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.276+0000 7f2febfff700 1 -- 172.21.15.131:0/93619677 >> v2:172.21.15.190:3300/0 conn(0x7f2fec06b8c0 msgr2=0x7f2fec06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.276+0000 7f2febfff700 1 --2- 172.21.15.131:0/93619677 >> v2:172.21.15.190:3300/0 conn(0x7f2fec06b8c0 0x7f2fec06bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.276+0000 7f2febfff700 1 -- 172.21.15.131:0/93619677 >> v1:172.21.15.190:6789/0 conn(0x7f2fec0cc100 legacy=0x7f2fec0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.276+0000 7f2febfff700 1 -- 172.21.15.131:0/93619677 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec068910 con 0x7f2fec13c510 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.276+0000 7f2feb7fe700 1 --2- 172.21.15.131:0/93619677 >> v2:172.21.15.190:3300/0 conn(0x7f2fec06b8c0 0x7f2fec06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:01.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.276+0000 7f2febfff700 1 --2- 172.21.15.131:0/93619677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec13c510 0x7f2fec0c8a20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009990 tx=0x7f2fdc009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdef2955c94bc4e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:01.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.276+0000 7f2feaffd700 1 -- 172.21.15.131:0/93619677 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc009e80 con 0x7f2fec13c510 2023-12-26T21:37:01.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.276+0000 7f2feaffd700 1 -- 172.21.15.131:0/93619677 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc005c10 con 0x7f2fec13c510 2023-12-26T21:37:01.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.276+0000 7f2feaffd700 1 -- 172.21.15.131:0/93619677 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00b650 con 0x7f2fec13c510 2023-12-26T21:37:01.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.277+0000 7f2ff29ff700 1 -- 172.21.15.131:0/93619677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec13c510 msgr2=0x7f2fec0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.277+0000 7f2ff29ff700 1 --2- 172.21.15.131:0/93619677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec13c510 0x7f2fec0c8a20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009990 tx=0x7f2fdc009220 comp rx=0 tx=0).stop 2023-12-26T21:37:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.277+0000 7f2ff29ff700 1 -- 172.21.15.131:0/93619677 shutdown_connections 2023-12-26T21:37:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.277+0000 7f2ff29ff700 1 --2- 172.21.15.131:0/93619677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec13c510 0x7f2fec0c8a20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.277+0000 7f2ff29ff700 1 --2- 172.21.15.131:0/93619677 >> v2:172.21.15.190:3300/0 conn(0x7f2fec06b8c0 0x7f2fec06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.277+0000 7f2ff29ff700 1 -- 172.21.15.131:0/93619677 >> 172.21.15.131:0/93619677 conn(0x7f2fec0ce3e0 msgr2=0x7f2fec0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.277+0000 7f2ff29ff700 1 -- 172.21.15.131:0/93619677 shutdown_connections 2023-12-26T21:37:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.277+0000 7f2ff29ff700 1 -- 172.21.15.131:0/93619677 wait complete. 2023-12-26T21:37:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.279+0000 7f2ff29ff700 1 Processor -- start 2023-12-26T21:37:01.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.279+0000 7f2ff29ff700 1 -- start start 2023-12-26T21:37:01.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff29ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec06b8c0 0x7f2fec13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff29ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0cc100 0x7f2fec14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff29ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fec13c510 0x7f2fec15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff29ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0b1180 con 0x7f2fec13c510 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff29ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec14c840 con 0x7f2fec0cc100 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff29ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec14cb20 con 0x7f2fec06b8c0 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff0f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fec13c510 0x7f2fec15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff0f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fec13c510 0x7f2fec15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45172/0 (socket says 172.21.15.131:45172) 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff0f9c700 1 -- 172.21.15.131:0/3012773620 learned_addr learned my addr 172.21.15.131:0/3012773620 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2febfff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec06b8c0 0x7f2fec13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2feb7fe700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0cc100 0x7f2fec14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff0f9c700 1 -- 172.21.15.131:0/3012773620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec06b8c0 msgr2=0x7f2fec13bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff0f9c700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec06b8c0 0x7f2fec13bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff0f9c700 1 -- 172.21.15.131:0/3012773620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0cc100 msgr2=0x7f2fec14bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff0f9c700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0cc100 0x7f2fec14bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.280+0000 7f2ff0f9c700 1 -- 172.21.15.131:0/3012773620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0009420 con 0x7f2fec13c510 2023-12-26T21:37:01.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.281+0000 7f2ff0f9c700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fec13c510 0x7f2fec15e210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c8e0 tx=0x7f2fe4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:01.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.281+0000 7f2fd3fff700 1 -- 172.21.15.131:0/3012773620 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4012630 con 0x7f2fec13c510 2023-12-26T21:37:01.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.281+0000 7f2fd3fff700 1 -- 172.21.15.131:0/3012773620 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe4012790 con 0x7f2fec13c510 2023-12-26T21:37:01.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.281+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc009480 con 0x7f2fec13c510 2023-12-26T21:37:01.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.281+0000 7f2fd3fff700 1 -- 172.21.15.131:0/3012773620 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4012b50 con 0x7f2fec13c510 2023-12-26T21:37:01.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.281+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec14d0e0 con 0x7f2fec13c510 2023-12-26T21:37:01.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.282+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fec0b12f0 con 0x7f2fec13c510 2023-12-26T21:37:01.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.283+0000 7f2fd3fff700 1 -- 172.21.15.131:0/3012773620 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2fe401a3a0 con 0x7f2fec13c510 2023-12-26T21:37:01.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.284+0000 7f2fd3fff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2fd8066fb0 0x7f2fd8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.284+0000 7f2fd3fff700 1 -- 172.21.15.131:0/3012773620 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fe4084710 con 0x7f2fec13c510 2023-12-26T21:37:01.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.285+0000 7f2febfff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2fd8066fb0 0x7f2fd8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.285+0000 7f2febfff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2fd8066fb0 0x7f2fd8069460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009260 tx=0x7f2fdc016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:01.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.288+0000 7f2fd3fff700 1 -- 172.21.15.131:0/3012773620 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fec0b12f0 con 0x7f2fec13c510 2023-12-26T21:37:01.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.480+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2fec0b12c0 con 0x7f2fec13c510 2023-12-26T21:37:01.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.481+0000 7f2fd3fff700 1 -- 172.21.15.131:0/3012773620 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2fe4055370 con 0x7f2fec13c510 2023-12-26T21:37:01.480 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2fd8066fb0 msgr2=0x7f2fd8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2fd8066fb0 0x7f2fd8069460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009260 tx=0x7f2fdc016000 comp rx=0 tx=0).stop 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fec13c510 msgr2=0x7f2fec15e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fec13c510 0x7f2fec15e210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c8e0 tx=0x7f2fe4001710 comp rx=0 tx=0).stop 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 shutdown_connections 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fec13c510 0x7f2fec15e210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2fd8066fb0 0x7f2fd8069460 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fec06b8c0 0x7f2fec13bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 --2- 172.21.15.131:0/3012773620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fec0cc100 0x7f2fec14bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 >> 172.21.15.131:0/3012773620 conn(0x7f2fec0ce3e0 msgr2=0x7f2fec0c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 shutdown_connections 2023-12-26T21:37:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.483+0000 7f2ff29ff700 1 -- 172.21.15.131:0/3012773620 wait complete. 2023-12-26T21:37:01.500 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:37:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.689+0000 7fec6bdff700 1 Processor -- start 2023-12-26T21:37:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.690+0000 7fec6bdff700 1 -- start start 2023-12-26T21:37:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6bdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fec64159a70 0x7fec640b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6bdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fec640b2a90 0x7fec640b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6bdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec640afcd0 con 0x7fec641596a0 2023-12-26T21:37:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6bdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec640b3e70 con 0x7fec640b2a90 2023-12-26T21:37:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6bdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec640b0f20 con 0x7fec64159a70 2023-12-26T21:37:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6a39c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fec640b2a90 0x7fec640b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec69b9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fec641596a0 0x7fec640b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49506/0 (socket says 172.21.15.131:49506) 2023-12-26T21:37:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec69b9b700 1 -- 172.21.15.131:0/1024897069 learned_addr learned my addr 172.21.15.131:0/1024897069 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6939a700 1 --2- 172.21.15.131:0/1024897069 >> v2:172.21.15.190:3300/0 conn(0x7fec64159a70 0x7fec640b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6a39c700 1 -- 172.21.15.131:0/1024897069 >> v2:172.21.15.190:3300/0 conn(0x7fec64159a70 msgr2=0x7fec640b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6a39c700 1 --2- 172.21.15.131:0/1024897069 >> v2:172.21.15.190:3300/0 conn(0x7fec64159a70 0x7fec640b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6a39c700 1 -- 172.21.15.131:0/1024897069 >> v1:172.21.15.131:6789/0 conn(0x7fec641596a0 legacy=0x7fec640b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec6a39c700 1 -- 172.21.15.131:0/1024897069 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640b3540 con 0x7fec640b2a90 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.691+0000 7fec68b99700 1 -- 172.21.15.131:0/1024897069 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680839602 0 0) 0x7fec640afcd0 con 0x7fec641596a0 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.692+0000 7fec6a39c700 1 --2- 172.21.15.131:0/1024897069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec640b2a90 0x7fec640b06a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec640642f0 tx=0x7fec5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12712c213e86b2a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.692+0000 7fec68b99700 1 -- 172.21.15.131:0/1024897069 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00e070 con 0x7fec640b2a90 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.692+0000 7fec68b99700 1 -- 172.21.15.131:0/1024897069 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c00bb60 con 0x7fec640b2a90 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.692+0000 7fec68b99700 1 -- 172.21.15.131:0/1024897069 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c012430 con 0x7fec640b2a90 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.692+0000 7fec6bdff700 1 -- 172.21.15.131:0/1024897069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec640b2a90 msgr2=0x7fec640b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.692+0000 7fec6bdff700 1 --2- 172.21.15.131:0/1024897069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec640b2a90 0x7fec640b06a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec640642f0 tx=0x7fec5c009780 comp rx=0 tx=0).stop 2023-12-26T21:37:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.693+0000 7fec6bdff700 1 -- 172.21.15.131:0/1024897069 shutdown_connections 2023-12-26T21:37:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.693+0000 7fec6bdff700 1 --2- 172.21.15.131:0/1024897069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec640b2a90 0x7fec640b06a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.693+0000 7fec6bdff700 1 --2- 172.21.15.131:0/1024897069 >> v2:172.21.15.190:3300/0 conn(0x7fec64159a70 0x7fec640b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.693+0000 7fec6bdff700 1 -- 172.21.15.131:0/1024897069 >> 172.21.15.131:0/1024897069 conn(0x7fec6406ece0 msgr2=0x7fec6406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.693+0000 7fec6bdff700 1 -- 172.21.15.131:0/1024897069 shutdown_connections 2023-12-26T21:37:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.693+0000 7fec6bdff700 1 -- 172.21.15.131:0/1024897069 wait complete. 2023-12-26T21:37:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.695+0000 7fec6bdff700 1 Processor -- start 2023-12-26T21:37:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.695+0000 7fec6bdff700 1 -- start start 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.696+0000 7fec6bdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec640b2a90 0x7fec641563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.696+0000 7fec6bdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec641596a0 0x7fec640cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.696+0000 7fec6bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec64159a70 0x7fec64150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.696+0000 7fec6bdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64156a20 con 0x7fec641596a0 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.696+0000 7fec6bdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec641510e0 con 0x7fec640b2a90 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.696+0000 7fec6bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec640cc340 con 0x7fec64159a70 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.696+0000 7fec6939a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec641596a0 0x7fec640cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.696+0000 7fec6939a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec641596a0 0x7fec640cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45194/0 (socket says 172.21.15.131:45194) 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.696+0000 7fec6939a700 1 -- 172.21.15.131:0/987876183 learned_addr learned my addr 172.21.15.131:0/987876183 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec69b9b700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec640b2a90 0x7fec641563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec6a39c700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec64159a70 0x7fec64150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec6939a700 1 -- 172.21.15.131:0/987876183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec64159a70 msgr2=0x7fec64150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec6939a700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec64159a70 0x7fec64150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec6939a700 1 -- 172.21.15.131:0/987876183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec640b2a90 msgr2=0x7fec641563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec6939a700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec640b2a90 0x7fec641563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec6939a700 1 -- 172.21.15.131:0/987876183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c009a20 con 0x7fec641596a0 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec6939a700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec641596a0 0x7fec640cc790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fec5400e930 tx=0x7fec540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec59ffb700 1 -- 172.21.15.131:0/987876183 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400eed0 con 0x7fec641596a0 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec59ffb700 1 -- 172.21.15.131:0/987876183 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5400c850 con 0x7fec641596a0 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec64159e40 con 0x7fec641596a0 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec59ffb700 1 -- 172.21.15.131:0/987876183 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400cbf0 con 0x7fec641596a0 2023-12-26T21:37:01.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.697+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec640c4490 con 0x7fec641596a0 2023-12-26T21:37:01.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.698+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec640cc4b0 con 0x7fec641596a0 2023-12-26T21:37:01.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.699+0000 7fec59ffb700 1 -- 172.21.15.131:0/987876183 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fec54013470 con 0x7fec641596a0 2023-12-26T21:37:01.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.700+0000 7fec59ffb700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec48066f30 0x7fec480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:01.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.700+0000 7fec59ffb700 1 -- 172.21.15.131:0/987876183 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec54084790 con 0x7fec641596a0 2023-12-26T21:37:01.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.700+0000 7fec69b9b700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec48066f30 0x7fec480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:01.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.701+0000 7fec69b9b700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec48066f30 0x7fec480693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec600011e0 tx=0x7fec60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:01.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.704+0000 7fec59ffb700 1 -- 172.21.15.131:0/987876183 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec640cc4b0 con 0x7fec641596a0 2023-12-26T21:37:01.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.940+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec640cc4b0 con 0x7fec641596a0 2023-12-26T21:37:01.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.941+0000 7fec59ffb700 1 -- 172.21.15.131:0/987876183 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fec640cc4b0 con 0x7fec641596a0 2023-12-26T21:37:01.940 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:01.941 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1127181197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1127181197},{"type":"v1","addr":"172.21.15.190:6835","nonce":1127181197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1425514628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1425514628},{"type":"v1","addr":"172.21.15.131:6837","nonce":1425514628}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3223324183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3223324183},{"type":"v1","addr":"172.21.15.190:6833","nonce":3223324183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:36:48.288910+0000","modified":"2023-12-26T21:36:51.988691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3852718531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3852718531},{"type":"v1","addr":"172.21.15.131:6836","nonce":3852718531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-12-26T21:37:01.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec48066f30 msgr2=0x7fec480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec48066f30 0x7fec480693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec600011e0 tx=0x7fec60009040 comp rx=0 tx=0).stop 2023-12-26T21:37:01.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec641596a0 msgr2=0x7fec640cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:01.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec641596a0 0x7fec640cc790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fec5400e930 tx=0x7fec540091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 shutdown_connections 2023-12-26T21:37:01.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fec48066f30 0x7fec480693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec64159a70 0x7fec64150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec640b2a90 0x7fec641563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 --2- 172.21.15.131:0/987876183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec641596a0 0x7fec640cc790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:01.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 >> 172.21.15.131:0/987876183 conn(0x7fec6406ece0 msgr2=0x7fec64156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:01.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 shutdown_connections 2023-12-26T21:37:01.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:01.943+0000 7fec6bdff700 1 -- 172.21.15.131:0/987876183 wait complete. 2023-12-26T21:37:01.944 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2023-12-26T21:37:01.961 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:02.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.150+0000 7ff6a85ff700 1 Processor -- start 2023-12-26T21:37:02.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.150+0000 7ff6a85ff700 1 -- start start 2023-12-26T21:37:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a85ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a00aa480 0x7ff6a00abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a85ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a015a8c0 con 0x7ff6a006e480 2023-12-26T21:37:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a85ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a0159f90 con 0x7ff6a0159820 2023-12-26T21:37:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a85ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a015ad50 con 0x7ff6a00aa480 2023-12-26T21:37:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a5b9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff6a006e480 0x7ff6a00675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49514/0 (socket says 172.21.15.131:49514) 2023-12-26T21:37:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a5b9a700 1 -- 172.21.15.131:0/2528946080 learned_addr learned my addr 172.21.15.131:0/2528946080 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a6b9c700 1 --2- 172.21.15.131:0/2528946080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a00aa480 0x7ff6a00abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a5399700 1 -- 172.21.15.131:0/2528946080 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312299275 0 0) 0x7ff6a015a8c0 con 0x7ff6a006e480 2023-12-26T21:37:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a5399700 1 -- 172.21.15.131:0/2528946080 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff68c003570 con 0x7ff6a006e480 2023-12-26T21:37:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a5399700 1 -- 172.21.15.131:0/2528946080 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986339444 0 0) 0x7ff6a0159f90 con 0x7ff6a0159820 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.152+0000 7ff6a5399700 1 -- 172.21.15.131:0/2528946080 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a015a8c0 con 0x7ff6a0159820 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.153+0000 7ff6a5399700 1 -- 172.21.15.131:0/2528946080 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1998249188 0 0) 0x7ff68c003570 con 0x7ff6a006e480 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.153+0000 7ff6a6b9c700 1 -- 172.21.15.131:0/2528946080 >> v1:172.21.15.190:6789/0 conn(0x7ff6a0159820 legacy=0x7ff6a006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.153+0000 7ff6a6b9c700 1 -- 172.21.15.131:0/2528946080 >> v1:172.21.15.131:6789/0 conn(0x7ff6a006e480 legacy=0x7ff6a00675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.153+0000 7ff6a6b9c700 1 -- 172.21.15.131:0/2528946080 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a015a270 con 0x7ff6a00aa480 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.153+0000 7ff6a6b9c700 1 --2- 172.21.15.131:0/2528946080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a00aa480 0x7ff6a00abdb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff698009e90 tx=0x7ff698009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d27fa4d02cbaf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.153+0000 7ff6a5399700 1 -- 172.21.15.131:0/2528946080 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800e070 con 0x7ff6a00aa480 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.153+0000 7ff6a5399700 1 -- 172.21.15.131:0/2528946080 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69800b870 con 0x7ff6a00aa480 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.153+0000 7ff6a5399700 1 -- 172.21.15.131:0/2528946080 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690001470 con 0x7ff6a00aa480 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.154+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2528946080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a00aa480 msgr2=0x7ff6a00abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.154+0000 7ff6a85ff700 1 --2- 172.21.15.131:0/2528946080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a00aa480 0x7ff6a00abdb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff698009e90 tx=0x7ff698009780 comp rx=0 tx=0).stop 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.154+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2528946080 shutdown_connections 2023-12-26T21:37:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.154+0000 7ff6a85ff700 1 --2- 172.21.15.131:0/2528946080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a00aa480 0x7ff6a00abdb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.154+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2528946080 >> 172.21.15.131:0/2528946080 conn(0x7ff6a006c290 msgr2=0x7ff6a00ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.154+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2528946080 shutdown_connections 2023-12-26T21:37:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.154+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2528946080 wait complete. 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.155+0000 7ff6a85ff700 1 Processor -- start 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.155+0000 7ff6a85ff700 1 -- start start 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.156+0000 7ff6a85ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6a006e480 0x7ff6a016cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.156+0000 7ff6a85ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a00aa480 0x7ff6a0169b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.156+0000 7ff6a85ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a0159820 0x7ff6a00c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.156+0000 7ff6a85ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c2e40 con 0x7ff6a00aa480 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.156+0000 7ff6a85ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c2230 con 0x7ff6a006e480 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.156+0000 7ff6a85ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c2510 con 0x7ff6a0159820 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.156+0000 7ff6a5b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a00aa480 0x7ff6a0169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.156+0000 7ff6a5b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a00aa480 0x7ff6a0169b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45200/0 (socket says 172.21.15.131:45200) 2023-12-26T21:37:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.156+0000 7ff6a5b9a700 1 -- 172.21.15.131:0/2750084167 learned_addr learned my addr 172.21.15.131:0/2750084167 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a6b9c700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a0159820 0x7ff6a00c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a639b700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6a006e480 0x7ff6a016cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a5b9a700 1 -- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a0159820 msgr2=0x7ff6a00c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a5b9a700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a0159820 0x7ff6a00c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a5b9a700 1 -- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6a006e480 msgr2=0x7ff6a016cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a5b9a700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6a006e480 0x7ff6a016cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a5b9a700 1 -- 172.21.15.131:0/2750084167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698009a20 con 0x7ff6a00aa480 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a6b9c700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a0159820 0x7ff6a00c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a639b700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6a006e480 0x7ff6a016cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a5b9a700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a00aa480 0x7ff6a0169b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff69000cb90 tx=0x7ff690001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6967fc700 1 -- 172.21.15.131:0/2750084167 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690007360 con 0x7ff6a00aa480 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6967fc700 1 -- 172.21.15.131:0/2750084167 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6900074e0 con 0x7ff6a00aa480 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a00c27f0 con 0x7ff6a00aa480 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6967fc700 1 -- 172.21.15.131:0/2750084167 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690004830 con 0x7ff6a00aa480 2023-12-26T21:37:02.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.157+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a00c1e00 con 0x7ff6a00aa480 2023-12-26T21:37:02.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.158+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a00c2ff0 con 0x7ff6a00aa480 2023-12-26T21:37:02.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.159+0000 7ff6967fc700 1 -- 172.21.15.131:0/2750084167 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff690004ba0 con 0x7ff6a00aa480 2023-12-26T21:37:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.160+0000 7ff6967fc700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff684066f30 0x7ff6840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.160+0000 7ff6967fc700 1 -- 172.21.15.131:0/2750084167 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff690083dc0 con 0x7ff6a00aa480 2023-12-26T21:37:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.160+0000 7ff6a639b700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff684066f30 0x7ff6840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.161+0000 7ff6a639b700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff684066f30 0x7ff6840693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff69c007fd0 tx=0x7ff69c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:02.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.164+0000 7ff6967fc700 1 -- 172.21.15.131:0/2750084167 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6a00c2ff0 con 0x7ff6a00aa480 2023-12-26T21:37:02.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.359+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6a00c2f80 con 0x7ff6a00aa480 2023-12-26T21:37:02.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.360+0000 7ff6967fc700 1 -- 172.21.15.131:0/2750084167 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7ff690051010 con 0x7ff6a00aa480 2023-12-26T21:37:02.360 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:02.360 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff684066f30 msgr2=0x7ff6840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff684066f30 0x7ff6840693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff69c007fd0 tx=0x7ff69c009040 comp rx=0 tx=0).stop 2023-12-26T21:37:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a00aa480 msgr2=0x7ff6a0169b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a00aa480 0x7ff6a0169b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff69000cb90 tx=0x7ff690001b50 comp rx=0 tx=0).stop 2023-12-26T21:37:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 shutdown_connections 2023-12-26T21:37:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff684066f30 0x7ff6840693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6a0159820 0x7ff6a00c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6a006e480 0x7ff6a016cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 --2- 172.21.15.131:0/2750084167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6a00aa480 0x7ff6a0169b50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 >> 172.21.15.131:0/2750084167 conn(0x7ff6a006c290 msgr2=0x7ff6a0069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 shutdown_connections 2023-12-26T21:37:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.363+0000 7ff6a85ff700 1 -- 172.21.15.131:0/2750084167 wait complete. 2023-12-26T21:37:02.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:36:46.654972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:36:47.264880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:37:02.381 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:37:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.573+0000 7f4f8bfff700 1 Processor -- start 2023-12-26T21:37:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.573+0000 7f4f8bfff700 1 -- start start 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.574+0000 7f4f8bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f8406bfd0 0x7f4f840c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.574+0000 7f4f8bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 0x7f4f840cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.574+0000 7f4f8bfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f840686c0 con 0x7f4f8413c510 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.574+0000 7f4f8bfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f84069520 con 0x7f4f8406bfd0 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.574+0000 7f4f8bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84068910 con 0x7f4f840cc100 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.574+0000 7f4f89d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4f8413c510 0x7f4f8406b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49516/0 (socket says 172.21.15.131:49516) 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.574+0000 7f4f89d9b700 1 -- 172.21.15.131:0/1867364772 learned_addr learned my addr 172.21.15.131:0/1867364772 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.574+0000 7f4f8a59c700 1 --2- 172.21.15.131:0/1867364772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 0x7f4f840cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.574+0000 7f4f8959a700 1 --2- 172.21.15.131:0/1867364772 >> v2:172.21.15.190:3300/0 conn(0x7f4f8406bfd0 0x7f4f840c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f88d99700 1 -- 172.21.15.131:0/1867364772 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485619093 0 0) 0x7f4f840686c0 con 0x7f4f8413c510 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f88d99700 1 -- 172.21.15.131:0/1867364772 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f70003570 con 0x7f4f8413c510 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f8a59c700 1 -- 172.21.15.131:0/1867364772 >> v2:172.21.15.190:3300/0 conn(0x7f4f8406bfd0 msgr2=0x7f4f840c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f8a59c700 1 --2- 172.21.15.131:0/1867364772 >> v2:172.21.15.190:3300/0 conn(0x7f4f8406bfd0 0x7f4f840c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f8a59c700 1 -- 172.21.15.131:0/1867364772 >> v1:172.21.15.131:6789/0 conn(0x7f4f8413c510 legacy=0x7f4f8406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f8a59c700 1 -- 172.21.15.131:0/1867364772 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f84068bf0 con 0x7f4f840cc100 2023-12-26T21:37:02.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f8a59c700 1 --2- 172.21.15.131:0/1867364772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 0x7f4f840cc560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009e60 tx=0x7f4f7c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96af6aa9d24c3940 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:02.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f88d99700 1 -- 172.21.15.131:0/1867364772 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c00e070 con 0x7f4f840cc100 2023-12-26T21:37:02.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f88d99700 1 -- 172.21.15.131:0/1867364772 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c00bab0 con 0x7f4f840cc100 2023-12-26T21:37:02.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.575+0000 7f4f88d99700 1 -- 172.21.15.131:0/1867364772 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c00be70 con 0x7f4f840cc100 2023-12-26T21:37:02.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.576+0000 7f4f8bfff700 1 -- 172.21.15.131:0/1867364772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 msgr2=0x7f4f840cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.576+0000 7f4f8bfff700 1 --2- 172.21.15.131:0/1867364772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 0x7f4f840cc560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009e60 tx=0x7f4f7c009780 comp rx=0 tx=0).stop 2023-12-26T21:37:02.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.576+0000 7f4f8bfff700 1 -- 172.21.15.131:0/1867364772 shutdown_connections 2023-12-26T21:37:02.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.576+0000 7f4f8bfff700 1 --2- 172.21.15.131:0/1867364772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 0x7f4f840cc560 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.576+0000 7f4f8bfff700 1 --2- 172.21.15.131:0/1867364772 >> v2:172.21.15.190:3300/0 conn(0x7f4f8406bfd0 0x7f4f840c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.576+0000 7f4f8bfff700 1 -- 172.21.15.131:0/1867364772 >> 172.21.15.131:0/1867364772 conn(0x7f4f840ce3e0 msgr2=0x7f4f840ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:02.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.576+0000 7f4f8bfff700 1 -- 172.21.15.131:0/1867364772 shutdown_connections 2023-12-26T21:37:02.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.576+0000 7f4f8bfff700 1 -- 172.21.15.131:0/1867364772 wait complete. 2023-12-26T21:37:02.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.578+0000 7f4f8bfff700 1 Processor -- start 2023-12-26T21:37:02.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.578+0000 7f4f8bfff700 1 -- start start 2023-12-26T21:37:02.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.579+0000 7f4f8bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8406bfd0 0x7f4f8415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.579+0000 7f4f8bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 0x7f4f8415b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.579+0000 7f4f8bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f8413c510 0x7f4f840b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.579+0000 7f4f8bfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8413c140 con 0x7f4f8413c510 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.579+0000 7f4f8bfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8415b760 con 0x7f4f8406bfd0 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.579+0000 7f4f8bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8415ba40 con 0x7f4f840cc100 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f8413c510 0x7f4f840b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f8413c510 0x7f4f840b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45208/0 (socket says 172.21.15.131:45208) 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8a59c700 1 -- 172.21.15.131:0/3484596786 learned_addr learned my addr 172.21.15.131:0/3484596786 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f89d9b700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8406bfd0 0x7f4f8415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8959a700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 0x7f4f8415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8a59c700 1 -- 172.21.15.131:0/3484596786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 msgr2=0x7f4f8415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8a59c700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 0x7f4f8415b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8a59c700 1 -- 172.21.15.131:0/3484596786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8406bfd0 msgr2=0x7f4f8415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8a59c700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8406bfd0 0x7f4f8415a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8a59c700 1 -- 172.21.15.131:0/3484596786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c009a20 con 0x7f4f8413c510 2023-12-26T21:37:02.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.580+0000 7f4f8a59c700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f8413c510 0x7f4f840b2730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f84068a50 tx=0x7f4f7c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:02.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.581+0000 7f4f79ffb700 1 -- 172.21.15.131:0/3484596786 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c012e10 con 0x7f4f8413c510 2023-12-26T21:37:02.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.581+0000 7f4f79ffb700 1 -- 172.21.15.131:0/3484596786 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c027470 con 0x7f4f8413c510 2023-12-26T21:37:02.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.581+0000 7f4f79ffb700 1 -- 172.21.15.131:0/3484596786 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c027830 con 0x7f4f8413c510 2023-12-26T21:37:02.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.581+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8415bd20 con 0x7f4f8413c510 2023-12-26T21:37:02.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.581+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8415c2e0 con 0x7f4f8413c510 2023-12-26T21:37:02.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.582+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f840657e0 con 0x7f4f8413c510 2023-12-26T21:37:02.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.583+0000 7f4f79ffb700 1 -- 172.21.15.131:0/3484596786 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4f7c024070 con 0x7f4f8413c510 2023-12-26T21:37:02.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.584+0000 7f4f79ffb700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4f68066f30 0x7f4f680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.584+0000 7f4f79ffb700 1 -- 172.21.15.131:0/3484596786 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4f7c086f50 con 0x7f4f8413c510 2023-12-26T21:37:02.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.584+0000 7f4f89d9b700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4f68066f30 0x7f4f680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.585+0000 7f4f89d9b700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4f68066f30 0x7f4f680693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4f80001360 tx=0x7f4f80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:02.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.588+0000 7f4f79ffb700 1 -- 172.21.15.131:0/3484596786 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f7c0541c0 con 0x7f4f8413c510 2023-12-26T21:37:02.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.777+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f84068ed0 con 0x7f4f8413c510 2023-12-26T21:37:02.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.778+0000 7f4f79ffb700 1 -- 172.21.15.131:0/3484596786 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f7c02c0a0 con 0x7f4f8413c510 2023-12-26T21:37:02.778 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:37:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4f68066f30 msgr2=0x7f4f680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4f68066f30 0x7f4f680693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4f80001360 tx=0x7f4f80009040 comp rx=0 tx=0).stop 2023-12-26T21:37:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f8413c510 msgr2=0x7f4f840b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f8413c510 0x7f4f840b2730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f84068a50 tx=0x7f4f7c019000 comp rx=0 tx=0).stop 2023-12-26T21:37:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 shutdown_connections 2023-12-26T21:37:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f8413c510 0x7f4f840b2730 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8406bfd0 0x7f4f8415a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4f68066f30 0x7f4f680693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 --2- 172.21.15.131:0/3484596786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f840cc100 0x7f4f8415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 >> 172.21.15.131:0/3484596786 conn(0x7f4f840ce3e0 msgr2=0x7f4f840c3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 shutdown_connections 2023-12-26T21:37:02.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.780+0000 7f4f8bfff700 1 -- 172.21.15.131:0/3484596786 wait complete. 2023-12-26T21:37:02.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:37:02.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.991+0000 7f4b54dff700 1 Processor -- start 2023-12-26T21:37:02.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.991+0000 7f4b54dff700 1 -- start start 2023-12-26T21:37:02.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b54dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b501596a0 0x7f4b500b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b54dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b50159a70 0x7f4b500b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b54dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500b0160 0x7f4b500b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b54dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500afcd0 con 0x7f4b50159a70 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b54dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500b3e70 con 0x7f4b501596a0 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b54dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500b2e00 con 0x7f4b500b0160 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4dd9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b50159a70 0x7f4b500b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4dd9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b50159a70 0x7f4b500b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45218/0 (socket says 172.21.15.131:45218) 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4dd9b700 1 -- 172.21.15.131:0/1910611495 learned_addr learned my addr 172.21.15.131:0/1910611495 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4ed9d700 1 --2- 172.21.15.131:0/1910611495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500b0160 0x7f4b500b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4e59c700 1 --2- 172.21.15.131:0/1910611495 >> v2:172.21.15.190:3300/0 conn(0x7f4b501596a0 0x7f4b500b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4dd9b700 1 -- 172.21.15.131:0/1910611495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500b0160 msgr2=0x7f4b500b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4dd9b700 1 --2- 172.21.15.131:0/1910611495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500b0160 0x7f4b500b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4dd9b700 1 -- 172.21.15.131:0/1910611495 >> v2:172.21.15.190:3300/0 conn(0x7f4b501596a0 msgr2=0x7f4b500b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4dd9b700 1 --2- 172.21.15.131:0/1910611495 >> v2:172.21.15.190:3300/0 conn(0x7f4b501596a0 0x7f4b500b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4dd9b700 1 -- 172.21.15.131:0/1910611495 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500b0f20 con 0x7f4b50159a70 2023-12-26T21:37:02.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.993+0000 7f4b4e59c700 1 --2- 172.21.15.131:0/1910611495 >> v2:172.21.15.190:3300/0 conn(0x7f4b501596a0 0x7f4b500b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:02.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.994+0000 7f4b4dd9b700 1 --2- 172.21.15.131:0/1910611495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b50159a70 0x7f4b500b28c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4b500892a0 tx=0x7f4b44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8735ddd7a64d8a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:02.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.994+0000 7f4b4d59a700 1 -- 172.21.15.131:0/1910611495 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44009e70 con 0x7f4b50159a70 2023-12-26T21:37:02.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.994+0000 7f4b4d59a700 1 -- 172.21.15.131:0/1910611495 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4400b860 con 0x7f4b50159a70 2023-12-26T21:37:02.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.994+0000 7f4b4d59a700 1 -- 172.21.15.131:0/1910611495 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4400bc00 con 0x7f4b50159a70 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.994+0000 7f4b54dff700 1 -- 172.21.15.131:0/1910611495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b50159a70 msgr2=0x7f4b500b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.994+0000 7f4b54dff700 1 --2- 172.21.15.131:0/1910611495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b50159a70 0x7f4b500b28c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4b500892a0 tx=0x7f4b44009250 comp rx=0 tx=0).stop 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.995+0000 7f4b54dff700 1 -- 172.21.15.131:0/1910611495 shutdown_connections 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.995+0000 7f4b54dff700 1 --2- 172.21.15.131:0/1910611495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500b0160 0x7f4b500b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.995+0000 7f4b54dff700 1 --2- 172.21.15.131:0/1910611495 >> v2:172.21.15.190:3300/0 conn(0x7f4b501596a0 0x7f4b500b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.995+0000 7f4b54dff700 1 --2- 172.21.15.131:0/1910611495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b50159a70 0x7f4b500b28c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.995+0000 7f4b54dff700 1 -- 172.21.15.131:0/1910611495 >> 172.21.15.131:0/1910611495 conn(0x7f4b5006ece0 msgr2=0x7f4b5006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.995+0000 7f4b54dff700 1 -- 172.21.15.131:0/1910611495 shutdown_connections 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.995+0000 7f4b54dff700 1 -- 172.21.15.131:0/1910611495 wait complete. 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.996+0000 7f4b54dff700 1 Processor -- start 2023-12-26T21:37:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.996+0000 7f4b54dff700 1 -- start start 2023-12-26T21:37:02.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b54dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b500b0160 0x7f4b50156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b54dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b501596a0 0x7f4b50154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b54dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b50159a70 0x7f4b50154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:02.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b54dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50156170 con 0x7f4b500b0160 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b54dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50155560 con 0x7f4b501596a0 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b54dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50155840 con 0x7f4b50159a70 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b4e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b500b0160 0x7f4b50156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b4e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b500b0160 0x7f4b50156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45228/0 (socket says 172.21.15.131:45228) 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b4e59c700 1 -- 172.21.15.131:0/3630395422 learned_addr learned my addr 172.21.15.131:0/3630395422 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b4ed9d700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b50159a70 0x7f4b50154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.997+0000 7f4b4dd9b700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b501596a0 0x7f4b50154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b4e59c700 1 -- 172.21.15.131:0/3630395422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b50159a70 msgr2=0x7f4b50154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b4e59c700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b50159a70 0x7f4b50154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b4e59c700 1 -- 172.21.15.131:0/3630395422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b501596a0 msgr2=0x7f4b50154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b4e59c700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b501596a0 0x7f4b50154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b4e59c700 1 -- 172.21.15.131:0/3630395422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b440094f0 con 0x7f4b500b0160 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b4e59c700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b500b0160 0x7f4b50156600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e8d0 tx=0x7f4b40008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:03.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b3e7fc700 1 -- 172.21.15.131:0/3630395422 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40013410 con 0x7f4b500b0160 2023-12-26T21:37:03.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b3e7fc700 1 -- 172.21.15.131:0/3630395422 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b40013570 con 0x7f4b500b0160 2023-12-26T21:37:03.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b50155b20 con 0x7f4b500b0160 2023-12-26T21:37:03.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b3e7fc700 1 -- 172.21.15.131:0/3630395422 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40013910 con 0x7f4b500b0160 2023-12-26T21:37:03.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.998+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b50156b40 con 0x7f4b500b0160 2023-12-26T21:37:03.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:02.999+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b500657e0 con 0x7f4b500b0160 2023-12-26T21:37:03.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.000+0000 7f4b3e7fc700 1 -- 172.21.15.131:0/3630395422 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4b40013d00 con 0x7f4b500b0160 2023-12-26T21:37:03.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.001+0000 7f4b3e7fc700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b30066f30 0x7f4b300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.001+0000 7f4b3e7fc700 1 -- 172.21.15.131:0/3630395422 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4b400842f0 con 0x7f4b500b0160 2023-12-26T21:37:03.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.001+0000 7f4b4dd9b700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b30066f30 0x7f4b300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.002+0000 7f4b4dd9b700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b30066f30 0x7f4b300693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b500892a0 tx=0x7f4b44017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:03.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.005+0000 7f4b3e7fc700 1 -- 172.21.15.131:0/3630395422 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b400514c0 con 0x7f4b500b0160 2023-12-26T21:37:03.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.200+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b500b3540 con 0x7f4b500b0160 2023-12-26T21:37:03.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.201+0000 7f4b3e7fc700 1 -- 172.21.15.131:0/3630395422 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b40054ae0 con 0x7f4b500b0160 2023-12-26T21:37:03.201 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b30066f30 msgr2=0x7f4b300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b30066f30 0x7f4b300693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b500892a0 tx=0x7f4b44017000 comp rx=0 tx=0).stop 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b500b0160 msgr2=0x7f4b50156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b500b0160 0x7f4b50156600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e8d0 tx=0x7f4b40008040 comp rx=0 tx=0).stop 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 shutdown_connections 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b30066f30 0x7f4b300693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b50159a70 0x7f4b50154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b500b0160 0x7f4b50156600 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 --2- 172.21.15.131:0/3630395422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b501596a0 0x7f4b50154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 >> 172.21.15.131:0/3630395422 conn(0x7f4b5006ece0 msgr2=0x7f4b500cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:03.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 shutdown_connections 2023-12-26T21:37:03.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.203+0000 7f4b54dff700 1 -- 172.21.15.131:0/3630395422 wait complete. 2023-12-26T21:37:03.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:37:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.414+0000 7f4baafff700 1 Processor -- start 2023-12-26T21:37:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.414+0000 7f4baafff700 1 -- start start 2023-12-26T21:37:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.415+0000 7f4baafff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ba41596a0 0x7f4ba40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.415+0000 7f4baafff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.415+0000 7f4baafff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba40afcd0 con 0x7f4ba40b0160 2023-12-26T21:37:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.415+0000 7f4baafff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba40b3e70 con 0x7f4ba41596a0 2023-12-26T21:37:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.415+0000 7f4baafff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba40b0cd0 con 0x7f4ba4159a70 2023-12-26T21:37:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.415+0000 7f4ba8d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ba41596a0 0x7f4ba40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.415+0000 7f4ba8d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ba41596a0 0x7f4ba40b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45238/0 (socket says 172.21.15.131:45238) 2023-12-26T21:37:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba8d9b700 1 -- 172.21.15.131:0/3105601520 learned_addr learned my addr 172.21.15.131:0/3105601520 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba3fff700 1 --2- 172.21.15.131:0/3105601520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba8d9b700 1 -- 172.21.15.131:0/3105601520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 msgr2=0x7f4ba40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba8d9b700 1 --2- 172.21.15.131:0/3105601520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba40b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba8d9b700 1 -- 172.21.15.131:0/3105601520 >> v1:172.21.15.190:6789/0 conn(0x7f4ba40b0160 legacy=0x7f4ba40b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba8d9b700 1 -- 172.21.15.131:0/3105601520 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba40b2e00 con 0x7f4ba41596a0 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba3fff700 1 --2- 172.21.15.131:0/3105601520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba40b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba8d9b700 1 --2- 172.21.15.131:0/3105601520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba41596a0 0x7f4ba40b2380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b980099b0 tx=0x7f4b98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20aff61dfdcf2380 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba37fe700 1 -- 172.21.15.131:0/3105601520 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800b860 con 0x7f4ba41596a0 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba37fe700 1 -- 172.21.15.131:0/3105601520 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800b9c0 con 0x7f4ba41596a0 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.416+0000 7f4ba37fe700 1 -- 172.21.15.131:0/3105601520 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800bd60 con 0x7f4ba41596a0 2023-12-26T21:37:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.417+0000 7f4baafff700 1 -- 172.21.15.131:0/3105601520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba41596a0 msgr2=0x7f4ba40b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.417+0000 7f4baafff700 1 --2- 172.21.15.131:0/3105601520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba41596a0 0x7f4ba40b2380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b980099b0 tx=0x7f4b98009250 comp rx=0 tx=0).stop 2023-12-26T21:37:03.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.417+0000 7f4baafff700 1 -- 172.21.15.131:0/3105601520 shutdown_connections 2023-12-26T21:37:03.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.417+0000 7f4baafff700 1 --2- 172.21.15.131:0/3105601520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba41596a0 0x7f4ba40b2380 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.417+0000 7f4baafff700 1 --2- 172.21.15.131:0/3105601520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba40b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.417+0000 7f4baafff700 1 -- 172.21.15.131:0/3105601520 >> 172.21.15.131:0/3105601520 conn(0x7f4ba406ece0 msgr2=0x7f4ba406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:03.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.417+0000 7f4baafff700 1 -- 172.21.15.131:0/3105601520 shutdown_connections 2023-12-26T21:37:03.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.417+0000 7f4baafff700 1 -- 172.21.15.131:0/3105601520 wait complete. 2023-12-26T21:37:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.418+0000 7f4baafff700 1 Processor -- start 2023-12-26T21:37:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.419+0000 7f4baafff700 1 -- start start 2023-12-26T21:37:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.419+0000 7f4baafff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba40b0160 0x7f4ba416dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.419+0000 7f4baafff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba41596a0 0x7f4ba4159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4baafff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba4157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4baafff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba415a3a0 con 0x7f4ba40b0160 2023-12-26T21:37:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4baafff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba415b150 con 0x7f4ba41596a0 2023-12-26T21:37:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4baafff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba415a5d0 con 0x7f4ba4159a70 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba8d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba40b0160 0x7f4ba416dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba8d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba40b0160 0x7f4ba416dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45248/0 (socket says 172.21.15.131:45248) 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba8d9b700 1 -- 172.21.15.131:0/1851493750 learned_addr learned my addr 172.21.15.131:0/1851493750 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba3fff700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba41596a0 0x7f4ba4159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba959c700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba4157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba8d9b700 1 -- 172.21.15.131:0/1851493750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 msgr2=0x7f4ba4157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba8d9b700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba4157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba8d9b700 1 -- 172.21.15.131:0/1851493750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba41596a0 msgr2=0x7f4ba4159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba8d9b700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba41596a0 0x7f4ba4159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba8d9b700 1 -- 172.21.15.131:0/1851493750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980094f0 con 0x7f4ba40b0160 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba959c700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba4157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.420+0000 7f4ba8d9b700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba40b0160 0x7f4ba416dbc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009290 tx=0x7f4b98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.421+0000 7f4ba0ff9700 1 -- 172.21.15.131:0/1851493750 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98026480 con 0x7f4ba40b0160 2023-12-26T21:37:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.421+0000 7f4ba0ff9700 1 -- 172.21.15.131:0/1851493750 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9801f070 con 0x7f4ba40b0160 2023-12-26T21:37:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.421+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba415a8b0 con 0x7f4ba40b0160 2023-12-26T21:37:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.421+0000 7f4ba0ff9700 1 -- 172.21.15.131:0/1851493750 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98026830 con 0x7f4ba40b0160 2023-12-26T21:37:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.421+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba415ae70 con 0x7f4ba40b0160 2023-12-26T21:37:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.422+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba40c8850 con 0x7f4ba40b0160 2023-12-26T21:37:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.422+0000 7f4ba0ff9700 1 -- 172.21.15.131:0/1851493750 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4b98026ba0 con 0x7f4ba40b0160 2023-12-26T21:37:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.423+0000 7f4ba0ff9700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b88066f30 0x7f4b880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.423+0000 7f4ba0ff9700 1 -- 172.21.15.131:0/1851493750 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4b98021070 con 0x7f4ba40b0160 2023-12-26T21:37:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.423+0000 7f4ba3fff700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b88066f30 0x7f4b880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.424+0000 7f4ba3fff700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b88066f30 0x7f4b880693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40c6110 tx=0x7f4b90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:03.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.427+0000 7f4ba0ff9700 1 -- 172.21.15.131:0/1851493750 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b98053540 con 0x7f4ba40b0160 2023-12-26T21:37:03.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.618+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ba40657e0 con 0x7f4ba40b0160 2023-12-26T21:37:03.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.618+0000 7f4ba0ff9700 1 -- 172.21.15.131:0/1851493750 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b98056b60 con 0x7f4ba40b0160 2023-12-26T21:37:03.619 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:37:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b88066f30 msgr2=0x7f4b880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b88066f30 0x7f4b880693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40c6110 tx=0x7f4b90008040 comp rx=0 tx=0).stop 2023-12-26T21:37:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba40b0160 msgr2=0x7f4ba416dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba40b0160 0x7f4ba416dbc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009290 tx=0x7f4b98017000 comp rx=0 tx=0).stop 2023-12-26T21:37:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 shutdown_connections 2023-12-26T21:37:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b88066f30 0x7f4b880693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba4159a70 0x7f4ba4157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ba40b0160 0x7f4ba416dbc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 --2- 172.21.15.131:0/1851493750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba41596a0 0x7f4ba4159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.620+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 >> 172.21.15.131:0/1851493750 conn(0x7f4ba406ece0 msgr2=0x7f4ba40cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.621+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 shutdown_connections 2023-12-26T21:37:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.621+0000 7f4baafff700 1 -- 172.21.15.131:0/1851493750 wait complete. 2023-12-26T21:37:03.639 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:37:03.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.833+0000 7f18b63ff700 1 Processor -- start 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.833+0000 7f18b63ff700 1 -- start start 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.834+0000 7f18b63ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f18b00c8820 0x7f18b00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.834+0000 7f18b63ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18b00b19f0 0x7f18b0161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.834+0000 7f18b63ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b0157a80 con 0x7f18b00c8bf0 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.834+0000 7f18b63ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00cac50 con 0x7f18b00b19f0 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.834+0000 7f18b63ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00caf30 con 0x7f18b00c8820 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.834+0000 7f18b499c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18b00b19f0 0x7f18b0161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18b499c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18b00b19f0 0x7f18b0161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45250/0 (socket says 172.21.15.131:45250) 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18b499c700 1 -- 172.21.15.131:0/472762121 learned_addr learned my addr 172.21.15.131:0/472762121 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:03.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.834+0000 7f18af7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f18b00c8bf0 0x7f18b00b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49528/0 (socket says 172.21.15.131:49528) 2023-12-26T21:37:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18affff700 1 --2- 172.21.15.131:0/472762121 >> v2:172.21.15.190:3300/0 conn(0x7f18b00c8820 0x7f18b00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18b499c700 1 -- 172.21.15.131:0/472762121 >> v2:172.21.15.190:3300/0 conn(0x7f18b00c8820 msgr2=0x7f18b00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18b499c700 1 --2- 172.21.15.131:0/472762121 >> v2:172.21.15.190:3300/0 conn(0x7f18b00c8820 0x7f18b00ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18b499c700 1 -- 172.21.15.131:0/472762121 >> v1:172.21.15.131:6789/0 conn(0x7f18b00c8bf0 legacy=0x7f18b00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18b499c700 1 -- 172.21.15.131:0/472762121 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00cb210 con 0x7f18b00b19f0 2023-12-26T21:37:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18b499c700 1 --2- 172.21.15.131:0/472762121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 0x7f18b0161090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f18b0064ea0 tx=0x7f18a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74afbdefa934860 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18aeffd700 1 -- 172.21.15.131:0/472762121 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a800e070 con 0x7f18b00b19f0 2023-12-26T21:37:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18aeffd700 1 -- 172.21.15.131:0/472762121 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a800ba20 con 0x7f18b00b19f0 2023-12-26T21:37:03.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.835+0000 7f18aeffd700 1 -- 172.21.15.131:0/472762121 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80092e0 con 0x7f18b00b19f0 2023-12-26T21:37:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.836+0000 7f18b63ff700 1 -- 172.21.15.131:0/472762121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 msgr2=0x7f18b0161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.836+0000 7f18b63ff700 1 --2- 172.21.15.131:0/472762121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 0x7f18b0161090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f18b0064ea0 tx=0x7f18a8009840 comp rx=0 tx=0).stop 2023-12-26T21:37:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.836+0000 7f18b63ff700 1 -- 172.21.15.131:0/472762121 shutdown_connections 2023-12-26T21:37:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.836+0000 7f18b63ff700 1 --2- 172.21.15.131:0/472762121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 0x7f18b0161090 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.836+0000 7f18b63ff700 1 --2- 172.21.15.131:0/472762121 >> v2:172.21.15.190:3300/0 conn(0x7f18b00c8820 0x7f18b00ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.836+0000 7f18b63ff700 1 -- 172.21.15.131:0/472762121 >> 172.21.15.131:0/472762121 conn(0x7f18b0068040 msgr2=0x7f18b0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.836+0000 7f18b63ff700 1 -- 172.21.15.131:0/472762121 shutdown_connections 2023-12-26T21:37:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.836+0000 7f18b63ff700 1 -- 172.21.15.131:0/472762121 wait complete. 2023-12-26T21:37:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.837+0000 7f18b63ff700 1 Processor -- start 2023-12-26T21:37:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.838+0000 7f18b63ff700 1 -- start start 2023-12-26T21:37:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.838+0000 7f18b63ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 0x7f18b00c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.838+0000 7f18b63ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b00c8820 0x7f18b00c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.838+0000 7f18b63ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b00c8bf0 0x7f18b0144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18b63ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00c5050 con 0x7f18b00b19f0 2023-12-26T21:37:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18b63ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00c4440 con 0x7f18b00c8820 2023-12-26T21:37:03.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18b63ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00c4720 con 0x7f18b00c8bf0 2023-12-26T21:37:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18affff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 0x7f18b00c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18affff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 0x7f18b00c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45252/0 (socket says 172.21.15.131:45252) 2023-12-26T21:37:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18affff700 1 -- 172.21.15.131:0/4255852341 learned_addr learned my addr 172.21.15.131:0/4255852341 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18af7fe700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b00c8820 0x7f18b00c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18b499c700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b00c8bf0 0x7f18b0144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18affff700 1 -- 172.21.15.131:0/4255852341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b00c8bf0 msgr2=0x7f18b0144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18affff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b00c8bf0 0x7f18b0144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18affff700 1 -- 172.21.15.131:0/4255852341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b00c8820 msgr2=0x7f18b00c2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18affff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b00c8820 0x7f18b00c2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18affff700 1 -- 172.21.15.131:0/4255852341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8009ae0 con 0x7f18b00b19f0 2023-12-26T21:37:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.839+0000 7f18affff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 0x7f18b00c6cd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f18a000ec80 tx=0x7f18a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.840+0000 7f1897fff700 1 -- 172.21.15.131:0/4255852341 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a000c850 con 0x7f18b00b19f0 2023-12-26T21:37:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.840+0000 7f1897fff700 1 -- 172.21.15.131:0/4255852341 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a000c9b0 con 0x7f18b00b19f0 2023-12-26T21:37:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.840+0000 7f1897fff700 1 -- 172.21.15.131:0/4255852341 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a000cd50 con 0x7f18b00b19f0 2023-12-26T21:37:03.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.840+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b00c4a00 con 0x7f18b00b19f0 2023-12-26T21:37:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.840+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b0145700 con 0x7f18b00b19f0 2023-12-26T21:37:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.841+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b00657e0 con 0x7f18b00b19f0 2023-12-26T21:37:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.842+0000 7f1897fff700 1 -- 172.21.15.131:0/4255852341 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f18a0013680 con 0x7f18b00b19f0 2023-12-26T21:37:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.842+0000 7f1897fff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f189c066f30 0x7f189c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.842+0000 7f1897fff700 1 -- 172.21.15.131:0/4255852341 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f18a0084800 con 0x7f18b00b19f0 2023-12-26T21:37:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.843+0000 7f18af7fe700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f189c066f30 0x7f189c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:03.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.843+0000 7f18af7fe700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f189c066f30 0x7f189c0693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f18b00c6930 tx=0x7f18a4007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:03.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:03.846+0000 7f1897fff700 1 -- 172.21.15.131:0/4255852341 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18a00519d0 con 0x7f18b00b19f0 2023-12-26T21:37:04.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.086+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18b00cb4f0 con 0x7f18b00b19f0 2023-12-26T21:37:04.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.087+0000 7f1897fff700 1 -- 172.21.15.131:0/4255852341 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f18a0054ff0 con 0x7f18b00b19f0 2023-12-26T21:37:04.088 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:04.088 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1127181197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1127181197},{"type":"v1","addr":"172.21.15.190:6835","nonce":1127181197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1425514628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1425514628},{"type":"v1","addr":"172.21.15.131:6837","nonce":1425514628}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3223324183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3223324183},{"type":"v1","addr":"172.21.15.190:6833","nonce":3223324183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:36:48.288910+0000","modified":"2023-12-26T21:36:51.988691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3852718531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3852718531},{"type":"v1","addr":"172.21.15.131:6836","nonce":3852718531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-12-26T21:37:04.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f189c066f30 msgr2=0x7f189c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f189c066f30 0x7f189c0693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f18b00c6930 tx=0x7f18a4007eb0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 msgr2=0x7f18b00c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 0x7f18b00c6cd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f18a000ec80 tx=0x7f18a00091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 shutdown_connections 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f189c066f30 0x7f189c0693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b00c8bf0 0x7f18b0144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18b00b19f0 0x7f18b00c6cd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 --2- 172.21.15.131:0/4255852341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b00c8820 0x7f18b00c2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 >> 172.21.15.131:0/4255852341 conn(0x7f18b0068040 msgr2=0x7f18b00b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 shutdown_connections 2023-12-26T21:37:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.090+0000 7f18b63ff700 1 -- 172.21.15.131:0/4255852341 wait complete. 2023-12-26T21:37:04.092 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2023-12-26T21:37:04.109 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:04.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.299+0000 7f48dcdff700 1 Processor -- start 2023-12-26T21:37:04.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.299+0000 7f48dcdff700 1 -- start start 2023-12-26T21:37:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.300+0000 7f48dcdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48d8159820 0x7f48d80abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.300+0000 7f48dcdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d80aa480 0x7f48d806dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.300+0000 7f48dcdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d815a8c0 con 0x7f48d806e2b0 2023-12-26T21:37:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.300+0000 7f48dcdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d815ad50 con 0x7f48d8159820 2023-12-26T21:37:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.300+0000 7f48dcdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d806e730 con 0x7f48d80aa480 2023-12-26T21:37:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d659c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48d8159820 0x7f48d80abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d659c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48d8159820 0x7f48d80abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45264/0 (socket says 172.21.15.131:45264) 2023-12-26T21:37:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d659c700 1 -- 172.21.15.131:0/2425893873 learned_addr learned my addr 172.21.15.131:0/2425893873 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d6d9d700 1 --1- 172.21.15.131:0/2425893873 >> v1:172.21.15.131:6789/0 conn(0x7f48d806e2b0 0x7f48d815d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49540/0 (socket says 172.21.15.131:49540) 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d5d9b700 1 --2- 172.21.15.131:0/2425893873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d80aa480 0x7f48d806dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d659c700 1 -- 172.21.15.131:0/2425893873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d80aa480 msgr2=0x7f48d806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d659c700 1 --2- 172.21.15.131:0/2425893873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d80aa480 0x7f48d806dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d659c700 1 -- 172.21.15.131:0/2425893873 >> v1:172.21.15.131:6789/0 conn(0x7f48d806e2b0 legacy=0x7f48d815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d659c700 1 -- 172.21.15.131:0/2425893873 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d8159f90 con 0x7f48d8159820 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d559a700 1 -- 172.21.15.131:0/2425893873 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092183781 0 0) 0x7f48d815a8c0 con 0x7f48d806e2b0 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d659c700 1 --2- 172.21.15.131:0/2425893873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d8159820 0x7f48d80abdb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48c80099e0 tx=0x7f48c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f155d08bac897482 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.301+0000 7f48d559a700 1 -- 172.21.15.131:0/2425893873 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800b820 con 0x7f48d8159820 2023-12-26T21:37:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48d559a700 1 -- 172.21.15.131:0/2425893873 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c800b980 con 0x7f48d8159820 2023-12-26T21:37:04.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48d559a700 1 -- 172.21.15.131:0/2425893873 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800bd20 con 0x7f48d8159820 2023-12-26T21:37:04.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48dcdff700 1 -- 172.21.15.131:0/2425893873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d8159820 msgr2=0x7f48d80abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48dcdff700 1 --2- 172.21.15.131:0/2425893873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d8159820 0x7f48d80abdb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48c80099e0 tx=0x7f48c8009250 comp rx=0 tx=0).stop 2023-12-26T21:37:04.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48dcdff700 1 -- 172.21.15.131:0/2425893873 shutdown_connections 2023-12-26T21:37:04.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48dcdff700 1 --2- 172.21.15.131:0/2425893873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d8159820 0x7f48d80abdb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48dcdff700 1 --2- 172.21.15.131:0/2425893873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d80aa480 0x7f48d806dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48dcdff700 1 -- 172.21.15.131:0/2425893873 >> 172.21.15.131:0/2425893873 conn(0x7f48d806c290 msgr2=0x7f48d80ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:04.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48dcdff700 1 -- 172.21.15.131:0/2425893873 shutdown_connections 2023-12-26T21:37:04.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.302+0000 7f48dcdff700 1 -- 172.21.15.131:0/2425893873 wait complete. 2023-12-26T21:37:04.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.304+0000 7f48dcdff700 1 Processor -- start 2023-12-26T21:37:04.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.304+0000 7f48dcdff700 1 -- start start 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48dcdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d806e2b0 0x7f48d816d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48dcdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d80aa480 0x7f48d80c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48dcdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d8159820 0x7f48d80c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48dcdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d816deb0 con 0x7f48d80aa480 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48dcdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80c29b0 con 0x7f48d8159820 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48dcdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80c2c90 con 0x7f48d806e2b0 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d80aa480 0x7f48d80c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d80aa480 0x7f48d80c1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45280/0 (socket says 172.21.15.131:45280) 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d5d9b700 1 -- 172.21.15.131:0/4215658146 learned_addr learned my addr 172.21.15.131:0/4215658146 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d6d9d700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d8159820 0x7f48d80c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d659c700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d806e2b0 0x7f48d816d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d5d9b700 1 -- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d806e2b0 msgr2=0x7f48d816d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d5d9b700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d806e2b0 0x7f48d816d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d5d9b700 1 -- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d8159820 msgr2=0x7f48d80c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d5d9b700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d8159820 0x7f48d80c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d5d9b700 1 -- 172.21.15.131:0/4215658146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c80094f0 con 0x7f48d80aa480 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d6d9d700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d8159820 0x7f48d80c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d659c700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d806e2b0 0x7f48d816d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.306+0000 7f48d5d9b700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d80aa480 0x7f48d80c1dd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00d890 tx=0x7f48cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.307+0000 7f48c67fc700 1 -- 172.21.15.131:0/4215658146 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00b600 con 0x7f48d80aa480 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.307+0000 7f48c67fc700 1 -- 172.21.15.131:0/4215658146 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc00b760 con 0x7f48d80aa480 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.307+0000 7f48c67fc700 1 -- 172.21.15.131:0/4215658146 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00bae0 con 0x7f48d80aa480 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.307+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d80c2f70 con 0x7f48d80aa480 2023-12-26T21:37:04.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.307+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d80c3530 con 0x7f48d80aa480 2023-12-26T21:37:04.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.308+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d80c36f0 con 0x7f48d80aa480 2023-12-26T21:37:04.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.309+0000 7f48c67fc700 1 -- 172.21.15.131:0/4215658146 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f48cc01e460 con 0x7f48d80aa480 2023-12-26T21:37:04.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.309+0000 7f48c67fc700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48b8066f30 0x7f48b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.310+0000 7f48c67fc700 1 -- 172.21.15.131:0/4215658146 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f48cc084be0 con 0x7f48d80aa480 2023-12-26T21:37:04.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.310+0000 7f48d659c700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48b8066f30 0x7f48b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.310+0000 7f48d659c700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48b8066f30 0x7f48b80693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f48c80099e0 tx=0x7f48c8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:04.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.314+0000 7f48c67fc700 1 -- 172.21.15.131:0/4215658146 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48cc051e50 con 0x7f48d80aa480 2023-12-26T21:37:04.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.508+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48d815a270 con 0x7f48d80aa480 2023-12-26T21:37:04.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.509+0000 7f48c67fc700 1 -- 172.21.15.131:0/4215658146 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f48cc055470 con 0x7f48d80aa480 2023-12-26T21:37:04.509 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:04.509 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:04.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48b8066f30 msgr2=0x7f48b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48b8066f30 0x7f48b80693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f48c80099e0 tx=0x7f48c8017000 comp rx=0 tx=0).stop 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d80aa480 msgr2=0x7f48d80c1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d80aa480 0x7f48d80c1dd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00d890 tx=0x7f48cc008040 comp rx=0 tx=0).stop 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 shutdown_connections 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d8159820 0x7f48d80c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48b8066f30 0x7f48b80693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d806e2b0 0x7f48d816d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 --2- 172.21.15.131:0/4215658146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d80aa480 0x7f48d80c1dd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 >> 172.21.15.131:0/4215658146 conn(0x7f48d806c290 msgr2=0x7f48d815aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 shutdown_connections 2023-12-26T21:37:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.511+0000 7f48dcdff700 1 -- 172.21.15.131:0/4215658146 wait complete. 2023-12-26T21:37:04.530 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:37:04.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.717+0000 7f2952fff700 1 Processor -- start 2023-12-26T21:37:04.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.718+0000 7f2952fff700 1 -- start start 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2952fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f294c1596a0 0x7f294c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2952fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f294c0b0870 0x7f294c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2952fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c0afcd0 con 0x7f294c159a70 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2952fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0b3e70 con 0x7f294c1596a0 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2952fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0b2e00 con 0x7f294c0b0870 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2950d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f294c1596a0 0x7f294c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2950d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f294c1596a0 0x7f294c0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45292/0 (socket says 172.21.15.131:45292) 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2950d9b700 1 -- 172.21.15.131:0/3471797117 learned_addr learned my addr 172.21.15.131:0/3471797117 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2950d9b700 1 -- 172.21.15.131:0/3471797117 >> v2:172.21.15.190:3300/0 conn(0x7f294c0b0870 msgr2=0x7f294c0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2950d9b700 1 --2- 172.21.15.131:0/3471797117 >> v2:172.21.15.190:3300/0 conn(0x7f294c0b0870 0x7f294c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2950d9b700 1 -- 172.21.15.131:0/3471797117 >> v1:172.21.15.190:6789/0 conn(0x7f294c159a70 legacy=0x7f294c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:04.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.719+0000 7f2950d9b700 1 -- 172.21.15.131:0/3471797117 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c0b0cb0 con 0x7f294c1596a0 2023-12-26T21:37:04.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f2950d9b700 1 --2- 172.21.15.131:0/3471797117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c1596a0 0x7f294c0b2380 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f294c0b66b0 tx=0x7f2940009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92a0f7f6c197411 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:04.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f294b7fe700 1 -- 172.21.15.131:0/3471797117 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000b820 con 0x7f294c1596a0 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f294b7fe700 1 -- 172.21.15.131:0/3471797117 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294000b980 con 0x7f294c1596a0 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f294b7fe700 1 -- 172.21.15.131:0/3471797117 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000bd20 con 0x7f294c1596a0 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f2952fff700 1 -- 172.21.15.131:0/3471797117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c1596a0 msgr2=0x7f294c0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f2952fff700 1 --2- 172.21.15.131:0/3471797117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c1596a0 0x7f294c0b2380 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f294c0b66b0 tx=0x7f2940009250 comp rx=0 tx=0).stop 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f2952fff700 1 -- 172.21.15.131:0/3471797117 shutdown_connections 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f2952fff700 1 --2- 172.21.15.131:0/3471797117 >> v2:172.21.15.190:3300/0 conn(0x7f294c0b0870 0x7f294c0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f2952fff700 1 --2- 172.21.15.131:0/3471797117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c1596a0 0x7f294c0b2380 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.720+0000 7f2952fff700 1 -- 172.21.15.131:0/3471797117 >> 172.21.15.131:0/3471797117 conn(0x7f294c06ece0 msgr2=0x7f294c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.721+0000 7f2952fff700 1 -- 172.21.15.131:0/3471797117 shutdown_connections 2023-12-26T21:37:04.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.721+0000 7f2952fff700 1 -- 172.21.15.131:0/3471797117 wait complete. 2023-12-26T21:37:04.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.722+0000 7f2952fff700 1 Processor -- start 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.722+0000 7f2952fff700 1 -- start start 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.723+0000 7f2952fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c0b0870 0x7f294c156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.723+0000 7f2952fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f294c1596a0 0x7f294c0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.723+0000 7f2952fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f294c159a70 0x7f294c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.723+0000 7f2952fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c16ac70 con 0x7f294c0b0870 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.723+0000 7f2952fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c158250 con 0x7f294c1596a0 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.723+0000 7f2952fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c157490 con 0x7f294c159a70 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.723+0000 7f2950d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c0b0870 0x7f294c156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.723+0000 7f2950d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c0b0870 0x7f294c156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45296/0 (socket says 172.21.15.131:45296) 2023-12-26T21:37:04.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.723+0000 7f2950d9b700 1 -- 172.21.15.131:0/4026347003 learned_addr learned my addr 172.21.15.131:0/4026347003 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:04.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f294bfff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f294c1596a0 0x7f294c0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2950d9b700 1 -- 172.21.15.131:0/4026347003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f294c159a70 msgr2=0x7f294c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2950d9b700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f294c159a70 0x7f294c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2950d9b700 1 -- 172.21.15.131:0/4026347003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f294c1596a0 msgr2=0x7f294c0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2950d9b700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f294c1596a0 0x7f294c0d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2950d9b700 1 -- 172.21.15.131:0/4026347003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29400094f0 con 0x7f294c0b0870 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f294bfff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f294c1596a0 0x7f294c0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2950d9b700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c0b0870 0x7f294c156670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2940009290 tx=0x7f2940016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2948ff9700 1 -- 172.21.15.131:0/4026347003 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940026480 con 0x7f294c0b0870 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2948ff9700 1 -- 172.21.15.131:0/4026347003 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294001f070 con 0x7f294c0b0870 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2948ff9700 1 -- 172.21.15.131:0/4026347003 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940026850 con 0x7f294c0b0870 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c157770 con 0x7f294c0b0870 2023-12-26T21:37:04.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.724+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c157d30 con 0x7f294c0b0870 2023-12-26T21:37:04.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.725+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c0657e0 con 0x7f294c0b0870 2023-12-26T21:37:04.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.726+0000 7f2948ff9700 1 -- 172.21.15.131:0/4026347003 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f294001a440 con 0x7f294c0b0870 2023-12-26T21:37:04.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.727+0000 7f2948ff9700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2930066f30 0x7f29300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:04.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.727+0000 7f2948ff9700 1 -- 172.21.15.131:0/4026347003 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2940021070 con 0x7f294c0b0870 2023-12-26T21:37:04.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.727+0000 7f294bfff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2930066f30 0x7f29300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:04.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.728+0000 7f294bfff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2930066f30 0x7f29300693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2938006fd0 tx=0x7f2938008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.731+0000 7f2948ff9700 1 -- 172.21.15.131:0/4026347003 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2940053d60 con 0x7f294c0b0870 2023-12-26T21:37:04.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.975+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f294c157f20 con 0x7f294c0b0870 2023-12-26T21:37:04.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.976+0000 7f2948ff9700 1 -- 172.21.15.131:0/4026347003 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f294002b020 con 0x7f294c0b0870 2023-12-26T21:37:04.977 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:04.977 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1127181197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1127181197},{"type":"v1","addr":"172.21.15.190:6835","nonce":1127181197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1425514628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1425514628},{"type":"v1","addr":"172.21.15.131:6837","nonce":1425514628}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3223324183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3223324183},{"type":"v1","addr":"172.21.15.190:6833","nonce":3223324183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:36:48.288910+0000","modified":"2023-12-26T21:36:51.988691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3852718531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3852718531},{"type":"v1","addr":"172.21.15.131:6836","nonce":3852718531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-12-26T21:37:04.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2930066f30 msgr2=0x7f29300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2930066f30 0x7f29300693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2938006fd0 tx=0x7f2938008040 comp rx=0 tx=0).stop 2023-12-26T21:37:04.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c0b0870 msgr2=0x7f294c156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:04.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c0b0870 0x7f294c156670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2940009290 tx=0x7f2940016000 comp rx=0 tx=0).stop 2023-12-26T21:37:04.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 shutdown_connections 2023-12-26T21:37:04.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2930066f30 0x7f29300693e0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2938006fd0 tx=0x7f2938008040 comp rx=0 tx=0).stop 2023-12-26T21:37:04.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f294c159a70 0x7f294c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f294c0b0870 0x7f294c156670 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 --2- 172.21.15.131:0/4026347003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f294c1596a0 0x7f294c0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:04.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 >> 172.21.15.131:0/4026347003 conn(0x7f294c06ece0 msgr2=0x7f294c0b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:04.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 shutdown_connections 2023-12-26T21:37:04.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:04.978+0000 7f2952fff700 1 -- 172.21.15.131:0/4026347003 wait complete. 2023-12-26T21:37:04.980 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2023-12-26T21:37:05.400 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T21:36:48.288910+0000', 'modified': '2023-12-26T21:36:51.988691+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/3852718531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 3852718531}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 3852718531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T21:37:05.400 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T21:37:05.400 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-26T21:37:05.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.589+0000 7f63655ff700 1 Processor -- start 2023-12-26T21:37:05.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.590+0000 7f63655ff700 1 -- start start 2023-12-26T21:37:05.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.591+0000 7f63655ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6360159a70 0x7f63600b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.591+0000 7f63655ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63600afcd0 con 0x7f63600b2a90 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.591+0000 7f63655ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63600b3e70 con 0x7f63601596a0 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.591+0000 7f63655ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600b0f20 con 0x7f6360159a70 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.591+0000 7f635f7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f63600b2a90 0x7f63600b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55618/0 (socket says 172.21.15.131:55618) 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.591+0000 7f635f7fe700 1 -- 172.21.15.131:0/2641901004 learned_addr learned my addr 172.21.15.131:0/2641901004 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.591+0000 7f635e7fc700 1 --2- 172.21.15.131:0/2641901004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6360159a70 0x7f63600b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.591+0000 7f635dffb700 1 -- 172.21.15.131:0/2641901004 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581057309 0 0) 0x7f63600b3e70 con 0x7f63601596a0 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.591+0000 7f635dffb700 1 -- 172.21.15.131:0/2641901004 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6348003570 con 0x7f63601596a0 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.592+0000 7f635dffb700 1 -- 172.21.15.131:0/2641901004 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936419411 0 0) 0x7f63600afcd0 con 0x7f63600b2a90 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.592+0000 7f635dffb700 1 -- 172.21.15.131:0/2641901004 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63600b3e70 con 0x7f63600b2a90 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.592+0000 7f635e7fc700 1 -- 172.21.15.131:0/2641901004 >> v1:172.21.15.190:6789/0 conn(0x7f63601596a0 legacy=0x7f63600b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.592+0000 7f635e7fc700 1 -- 172.21.15.131:0/2641901004 >> v1:172.21.15.131:6789/0 conn(0x7f63600b2a90 legacy=0x7f63600b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.592+0000 7f635e7fc700 1 -- 172.21.15.131:0/2641901004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600b3540 con 0x7f6360159a70 2023-12-26T21:37:05.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.592+0000 7f635e7fc700 1 --2- 172.21.15.131:0/2641901004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6360159a70 0x7f63600b0160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6354009910 tx=0x7f63540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e7c4edc6c71801 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:05.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.592+0000 7f635dffb700 1 -- 172.21.15.131:0/2641901004 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354011630 con 0x7f6360159a70 2023-12-26T21:37:05.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.592+0000 7f635dffb700 1 -- 172.21.15.131:0/2641901004 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6354011790 con 0x7f6360159a70 2023-12-26T21:37:05.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.592+0000 7f635dffb700 1 -- 172.21.15.131:0/2641901004 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354011b30 con 0x7f6360159a70 2023-12-26T21:37:05.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.593+0000 7f63655ff700 1 -- 172.21.15.131:0/2641901004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6360159a70 msgr2=0x7f63600b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.593+0000 7f63655ff700 1 --2- 172.21.15.131:0/2641901004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6360159a70 0x7f63600b0160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6354009910 tx=0x7f63540091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.593+0000 7f63655ff700 1 -- 172.21.15.131:0/2641901004 shutdown_connections 2023-12-26T21:37:05.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.593+0000 7f63655ff700 1 --2- 172.21.15.131:0/2641901004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6360159a70 0x7f63600b0160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.593+0000 7f63655ff700 1 -- 172.21.15.131:0/2641901004 >> 172.21.15.131:0/2641901004 conn(0x7f636006ece0 msgr2=0x7f636006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:05.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.593+0000 7f63655ff700 1 -- 172.21.15.131:0/2641901004 shutdown_connections 2023-12-26T21:37:05.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.593+0000 7f63655ff700 1 -- 172.21.15.131:0/2641901004 wait complete. 2023-12-26T21:37:05.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.594+0000 7f63655ff700 1 Processor -- start 2023-12-26T21:37:05.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.594+0000 7f63655ff700 1 -- start start 2023-12-26T21:37:05.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.595+0000 7f63655ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b2a90 0x7f6360155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.595+0000 7f63655ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63601596a0 0x7f636015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.595+0000 7f63655ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6360159a70 0x7f63600c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.595+0000 7f63655ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600c8da0 con 0x7f6360159a70 2023-12-26T21:37:05.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.595+0000 7f63655ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636016a820 con 0x7f63601596a0 2023-12-26T21:37:05.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.595+0000 7f63655ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600c8ac0 con 0x7f63600b2a90 2023-12-26T21:37:05.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6360159a70 0x7f63600c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6360159a70 0x7f63600c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39464/0 (socket says 172.21.15.131:39464) 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635f7fe700 1 -- 172.21.15.131:0/2722016927 learned_addr learned my addr 172.21.15.131:0/2722016927 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635e7fc700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63601596a0 0x7f636015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635effd700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b2a90 0x7f6360155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635f7fe700 1 -- 172.21.15.131:0/2722016927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b2a90 msgr2=0x7f6360155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635f7fe700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b2a90 0x7f6360155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635f7fe700 1 -- 172.21.15.131:0/2722016927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63601596a0 msgr2=0x7f636015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635f7fe700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63601596a0 0x7f636015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635f7fe700 1 -- 172.21.15.131:0/2722016927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354009420 con 0x7f6360159a70 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f635f7fe700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6360159a70 0x7f63600c9c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6358001460 tx=0x7f6358001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f6346ffd700 1 -- 172.21.15.131:0/2722016927 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800ce80 con 0x7f6360159a70 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f6346ffd700 1 -- 172.21.15.131:0/2722016927 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635800ab70 con 0x7f6360159a70 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f6346ffd700 1 -- 172.21.15.131:0/2722016927 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63580070f0 con 0x7f6360159a70 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.596+0000 7f63655ff700 1 -- 172.21.15.131:0/2722016927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63600c91a0 con 0x7f6360159a70 2023-12-26T21:37:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.597+0000 7f63655ff700 1 -- 172.21.15.131:0/2722016927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63600c9760 con 0x7f6360159a70 2023-12-26T21:37:05.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.599+0000 7f6346ffd700 1 -- 172.21.15.131:0/2722016927 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6358007520 con 0x7f6360159a70 2023-12-26T21:37:05.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.599+0000 7f6346ffd700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f634c066f30 0x7f634c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.600+0000 7f6346ffd700 1 -- 172.21.15.131:0/2722016927 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6358084a60 con 0x7f6360159a70 2023-12-26T21:37:05.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.600+0000 7f635effd700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f634c066f30 0x7f634c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.600+0000 7f635effd700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f634c066f30 0x7f634c0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6350003870 tx=0x7f6350009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:05.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.601+0000 7f6373cb7b80 1 Processor -- start 2023-12-26T21:37:05.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.602+0000 7f6373cb7b80 1 -- start start 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.602+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b6fa0 0x558ea08b7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.602+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea08b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.602+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b3170 0x558ea08d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.602+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea07e53e0 con 0x558ea08b6fa0 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.602+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea07e2be0 con 0x558ea08b3170 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.602+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ea07de9d0 con 0x558ea08b78b0 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b6fa0 0x558ea08b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b6fa0 0x558ea08b7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39468/0 (socket says 172.21.15.131:39468) 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635effd700 1 -- 172.21.15.131:0/3944042531 learned_addr learned my addr 172.21.15.131:0/3944042531 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635e7fc700 1 --2- 172.21.15.131:0/3944042531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea08b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635f7fe700 1 --2- 172.21.15.131:0/3944042531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b3170 0x558ea08d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635effd700 1 -- 172.21.15.131:0/3944042531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 msgr2=0x558ea08b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635effd700 1 --2- 172.21.15.131:0/3944042531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea08b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635effd700 1 -- 172.21.15.131:0/3944042531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b3170 msgr2=0x558ea08d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635effd700 1 --2- 172.21.15.131:0/3944042531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b3170 0x558ea08d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635effd700 1 -- 172.21.15.131:0/3944042531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ea0672360 con 0x558ea08b6fa0 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f635effd700 1 --2- 172.21.15.131:0/3944042531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b6fa0 0x558ea08b7370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f635000e8b0 tx=0x7f6350014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40078d2c5fab26a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.603+0000 7f633ffff700 1 -- 172.21.15.131:0/3944042531 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350003550 con 0x558ea08b6fa0 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f633ffff700 1 -- 172.21.15.131:0/3944042531 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63500136b0 con 0x558ea08b6fa0 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f633ffff700 1 -- 172.21.15.131:0/3944042531 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350013a70 con 0x558ea08b6fa0 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3944042531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b6fa0 msgr2=0x558ea08b7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3944042531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b6fa0 0x558ea08b7370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f635000e8b0 tx=0x7f6350014000 comp rx=0 tx=0).stop 2023-12-26T21:37:05.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3944042531 shutdown_connections 2023-12-26T21:37:05.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3944042531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b3170 0x558ea08d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3944042531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b6fa0 0x558ea08b7370 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3944042531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea08b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3944042531 >> 172.21.15.131:0/3944042531 conn(0x558ea088d440 msgr2=0x558ea08d8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3944042531 shutdown_connections 2023-12-26T21:37:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.604+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3944042531 wait complete. 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.605+0000 7f6373cb7b80 1 -- start start 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.605+0000 7f6373cb7b80 1 Processor -- start 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 0x558ea08b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea0890490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea078b1b0 con 0x558ea08b3170 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea078ae50 con 0x558ea08b6fa0 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ea06d7e40 con 0x558ea08b78b0 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f6373cb7b80 10 client.0 _ll_register_callbacks cb 0x558ea06da7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f635effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f635effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39480/0 (socket says 172.21.15.131:39480) 2023-12-26T21:37:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.606+0000 7f635effd700 1 -- 172.21.15.131:0/114008821 learned_addr learned my addr 172.21.15.131:0/114008821 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:05.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f635e7fc700 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 0x558ea08b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f635f7fe700 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea0890490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f635effd700 1 -- 172.21.15.131:0/114008821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 msgr2=0x558ea0890490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:05.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f635effd700 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea0890490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f635effd700 1 -- 172.21.15.131:0/114008821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 msgr2=0x558ea08b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:05.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f635effd700 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 0x558ea08b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f635effd700 1 -- 172.21.15.131:0/114008821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635000f040 con 0x558ea08b3170 2023-12-26T21:37:05.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f635e7fc700 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 0x558ea08b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:05.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f635effd700 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08d0460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f635000f010 tx=0x7f6350019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7915efa6b9fb7fb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f633d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f633d7fa700 1 -- 172.21.15.131:0/114008821 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635002d430 con 0x558ea08b3170 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f633d7fa700 1 -- 172.21.15.131:0/114008821 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f635001b050 con 0x558ea08b3170 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f6373cb7b80 1 -- 172.21.15.131:0/114008821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ea085feb0 con 0x558ea08b3170 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f633d7fa700 1 -- 172.21.15.131:0/114008821 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f635002d7a0 con 0x558ea08b3170 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f6373cb7b80 10 client.4602 fetch_fsmap learned FSMap version 7 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.607+0000 7f6373cb7b80 1 -- 172.21.15.131:0/114008821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x558ea061eb10 con 0x558ea08b3170 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f633d7fa700 1 -- 172.21.15.131:0/114008821 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ea085feb0 con 0x558ea08b3170 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f633d7fa700 1 -- 172.21.15.131:0/114008821 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f635000e8e0 con 0x558ea08b3170 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f6373cb7b80 10 client.4602 fetch_fsmap finished waiting for FSMap version 7 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f6373cb7b80 10 client.4602 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f6373cb7b80 20 client.4602 populate_metadata read hostname 'smithi131' 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] conn(0x558ea08e6170 0x558ea08e8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f6373cb7b80 4 client.4602 mds_command: new command op to 4288 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f6373cb7b80 1 -- 172.21.15.131:0/114008821 --> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ea07b5880 con 0x558ea08e6170 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f635e7fc700 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] conn(0x558ea08e6170 0x558ea08e8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f633d7fa700 10 client.4602 ms_handle_connect on v2:172.21.15.131:6835/3852718531 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.608+0000 7f635e7fc700 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] conn(0x558ea08e6170 0x558ea08e8600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c26306fadd358bca server_cookie=3df65bd67a36125c in_seq=0 out_seq=0 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.609+0000 7f633d7fa700 1 -- 172.21.15.131:0/114008821 <== mds.0 v2:172.21.15.131:6835/3852718531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558ea07b5880 con 0x558ea08e6170 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.609+0000 7f633d7fa700 10 client.4602 handle_command_reply: tid=0 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.609+0000 7f6373cb7b80 1 client.4602 shutdown 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.609+0000 7f6373cb7b80 1 -- 172.21.15.131:0/114008821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 msgr2=0x558ea08d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.609+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08d0460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f635000f010 tx=0x7f6350019000 comp rx=0 tx=0).stop 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.609+0000 7f6373cb7b80 1 -- 172.21.15.131:0/114008821 shutdown_connections 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.609+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea0890490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.609+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] conn(0x558ea08e6170 0x558ea08e8600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.610+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08d0460 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.610+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/114008821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 0x558ea08b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.610+0000 7f633d7fa700 0 client.4602 ms_handle_reset on v2:172.21.15.131:6835/3852718531 2023-12-26T21:37:05.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.610+0000 7f6373cb7b80 1 -- 172.21.15.131:0/114008821 >> 172.21.15.131:0/114008821 conn(0x558ea088d440 msgr2=0x558ea088d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:05.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.610+0000 7f6373cb7b80 1 -- 172.21.15.131:0/114008821 shutdown_connections 2023-12-26T21:37:05.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.610+0000 7f6373cb7b80 1 -- 172.21.15.131:0/114008821 wait complete. 2023-12-26T21:37:05.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.610+0000 7f6373cb7b80 20 client.4602 trim_cache size 0 max 16384 2023-12-26T21:37:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.627+0000 7f6373cb7b80 1 Processor -- start 2023-12-26T21:37:05.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.627+0000 7f6373cb7b80 1 -- start start 2023-12-26T21:37:05.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b6fa0 0x558ea08b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b78b0 0x558ea08be080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea08cfdd0 con 0x558ea08b3170 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea088eb80 con 0x558ea08b78b0 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ea08e59a0 con 0x558ea08b6fa0 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39488/0 (socket says 172.21.15.131:39488) 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635effd700 1 -- 172.21.15.131:0/315152386 learned_addr learned my addr 172.21.15.131:0/315152386 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635f7fe700 1 --2- 172.21.15.131:0/315152386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b78b0 0x558ea08be080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635e7fc700 1 --2- 172.21.15.131:0/315152386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b6fa0 0x558ea08b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635effd700 1 -- 172.21.15.131:0/315152386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b6fa0 msgr2=0x558ea08b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635effd700 1 --2- 172.21.15.131:0/315152386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b6fa0 0x558ea08b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635effd700 1 -- 172.21.15.131:0/315152386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b78b0 msgr2=0x558ea08be080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635effd700 1 --2- 172.21.15.131:0/315152386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b78b0 0x558ea08be080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.628+0000 7f635effd700 1 -- 172.21.15.131:0/315152386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ea07ad290 con 0x558ea08b3170 2023-12-26T21:37:05.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.629+0000 7f635effd700 1 --2- 172.21.15.131:0/315152386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08b3d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6350003f10 tx=0x7f635002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07323294c48d69f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:05.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.629+0000 7f6344ff9700 1 -- 172.21.15.131:0/315152386 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635002d7a0 con 0x558ea08b3170 2023-12-26T21:37:05.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.629+0000 7f6344ff9700 1 -- 172.21.15.131:0/315152386 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350013760 con 0x558ea08b3170 2023-12-26T21:37:05.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.629+0000 7f6344ff9700 1 -- 172.21.15.131:0/315152386 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350013420 con 0x558ea08b3170 2023-12-26T21:37:05.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.629+0000 7f6373cb7b80 1 -- 172.21.15.131:0/315152386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 msgr2=0x558ea08b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.629+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/315152386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08b3d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6350003f10 tx=0x7f635002d000 comp rx=0 tx=0).stop 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.630+0000 7f6373cb7b80 1 -- 172.21.15.131:0/315152386 shutdown_connections 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.630+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/315152386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b78b0 0x558ea08be080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.630+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/315152386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea08b3d80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.630+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/315152386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b6fa0 0x558ea08b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.630+0000 7f6373cb7b80 1 -- 172.21.15.131:0/315152386 >> 172.21.15.131:0/315152386 conn(0x558ea088d440 msgr2=0x558ea08b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.630+0000 7f6373cb7b80 1 -- 172.21.15.131:0/315152386 shutdown_connections 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.630+0000 7f6373cb7b80 1 -- 172.21.15.131:0/315152386 wait complete. 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.630+0000 7f6373cb7b80 1 -- start start 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.630+0000 7f6373cb7b80 1 Processor -- start 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.631+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea0938980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.631+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 0x558ea088a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.631+0000 7f6373cb7b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea088ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.631+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea08be420 con 0x558ea08b3170 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.631+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ea093d5a0 con 0x558ea08b6fa0 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.631+0000 7f6373cb7b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ea093d110 con 0x558ea08b78b0 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.631+0000 7f6373cb7b80 10 client.0 _ll_register_callbacks cb 0x7f6350024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea0938980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea0938980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39496/0 (socket says 172.21.15.131:39496) 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635effd700 1 -- 172.21.15.131:0/3405026382 learned_addr learned my addr 172.21.15.131:0/3405026382 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635e7fc700 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 0x558ea088a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635f7fe700 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea088ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635effd700 1 -- 172.21.15.131:0/3405026382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 msgr2=0x558ea088ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635effd700 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea088ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635effd700 1 -- 172.21.15.131:0/3405026382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 msgr2=0x558ea088a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:05.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635effd700 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 0x558ea088a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635effd700 1 -- 172.21.15.131:0/3405026382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ea061eb10 con 0x558ea08b3170 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f635effd700 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea0938980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f635000e4c0 tx=0x7f635003e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4817947be7cdb064 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f633d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.632+0000 7f633d7fa700 1 -- 172.21.15.131:0/3405026382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350000fe0 con 0x558ea08b3170 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f633d7fa700 1 -- 172.21.15.131:0/3405026382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f635004c070 con 0x558ea08b3170 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3405026382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ea0883490 con 0x558ea08b3170 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f633d7fa700 1 -- 172.21.15.131:0/3405026382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f635002d7a0 con 0x558ea08b3170 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f6373cb7b80 10 client.4608 fetch_fsmap learned FSMap version 7 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3405026382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x558ea0676c80 con 0x558ea08b3170 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f633d7fa700 1 -- 172.21.15.131:0/3405026382 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ea0883490 con 0x558ea08b3170 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f633d7fa700 1 -- 172.21.15.131:0/3405026382 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f635002d9f0 con 0x558ea08b3170 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f6373cb7b80 10 client.4608 fetch_fsmap finished waiting for FSMap version 7 2023-12-26T21:37:05.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f6373cb7b80 10 client.4608 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f6373cb7b80 20 client.4608 populate_metadata read hostname 'smithi131' 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] conn(0x558ea08e67f0 0x558ea08e6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f6373cb7b80 4 client.4608 mds_command: new command op to 4288 tid=0[{"prefix": "status"}] 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3405026382 --> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558ea0781910 con 0x558ea08e67f0 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.633+0000 7f635e7fc700 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] conn(0x558ea08e67f0 0x558ea08e6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f633d7fa700 10 client.4608 ms_handle_connect on v2:172.21.15.131:6835/3852718531 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f635e7fc700 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] conn(0x558ea08e67f0 0x558ea08e6bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e01d0355a3f750 server_cookie=f166d269656898c0 in_seq=0 out_seq=0 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f633d7fa700 1 -- 172.21.15.131:0/3405026382 <== mds.0 v2:172.21.15.131:6835/3852718531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x558ea0781910 con 0x558ea08e67f0 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f633d7fa700 10 client.4608 handle_command_reply: tid=0 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 client.4608 shutdown 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3405026382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 msgr2=0x558ea0938980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea0938980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f635000e4c0 tx=0x7f635003e070 comp rx=0 tx=0).stop 2023-12-26T21:37:05.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3405026382 shutdown_connections 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558ea08b78b0 0x558ea088ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.131:6835/3852718531,v1:172.21.15.131:6836/3852718531] conn(0x558ea08e67f0 0x558ea08e6bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ea08b3170 0x558ea0938980 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 --2- 172.21.15.131:0/3405026382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ea08b6fa0 0x558ea088a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f633d7fa700 0 client.4608 ms_handle_reset on v2:172.21.15.131:6835/3852718531 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3405026382 >> 172.21.15.131:0/3405026382 conn(0x558ea088d440 msgr2=0x558ea08b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3405026382 shutdown_connections 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.634+0000 7f6373cb7b80 1 -- 172.21.15.131:0/3405026382 wait complete. 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f6373cb7b80 20 client.4608 trim_cache size 0 max 16384 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:37:05.640 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4288, 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 16.835267353999999, 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 7, 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 22, 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 22, 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 21.869945626 2023-12-26T21:37:05.641 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:37:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 -- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f634c066f30 msgr2=0x7f634c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f634c066f30 0x7f634c0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6350003870 tx=0x7f6350009040 comp rx=0 tx=0).stop 2023-12-26T21:37:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 -- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6360159a70 msgr2=0x7f63600c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6360159a70 0x7f63600c9c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6358001460 tx=0x7f6358001ae0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 -- 172.21.15.131:0/2722016927 shutdown_connections 2023-12-26T21:37:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6360159a70 0x7f63600c9c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f634c066f30 0x7f634c0693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b2a90 0x7f6360155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 --2- 172.21.15.131:0/2722016927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63601596a0 0x7f636015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 -- 172.21.15.131:0/2722016927 >> 172.21.15.131:0/2722016927 conn(0x7f636006ece0 msgr2=0x7f63601564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 -- 172.21.15.131:0/2722016927 shutdown_connections 2023-12-26T21:37:05.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:05.635+0000 7f63655ff700 1 -- 172.21.15.131:0/2722016927 wait complete. 2023-12-26T21:37:05.653 INFO:teuthology.run_tasks:Running task kclient... 2023-12-26T21:37:05.678 INFO:tasks.kclient:Mounting kernel clients... 2023-12-26T21:37:05.679 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-12-26T21:37:05.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:05.679 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:37:05.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:05.724 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2023-12-26T21:37:05.835 INFO:teuthology.orchestra.run.smithi131.stderr:Cannot find device "ceph-brx" 2023-12-26T21:37:05.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T21:37:05.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:05.839 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:37:05.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:05.883 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2023-12-26T21:37:05.993 INFO:teuthology.orchestra.run.smithi131.stderr:Cannot find device "ceph-brx" 2023-12-26T21:37:05.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T21:37:05.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:05.997 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-12-26T21:37:06.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:06.046 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-12-26T21:37:06.121 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-12-26T21:37:06.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T21:37:06.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:06.124 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-12-26T21:37:06.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:06.168 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-12-26T21:37:06.276 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-12-26T21:37:06.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T21:37:06.279 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-12-26T21:37:06.280 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:37:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.468+0000 7f7dae1ff700 1 Processor -- start 2023-12-26T21:37:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.468+0000 7f7dae1ff700 1 -- start start 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.469+0000 7f7dae1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7da806bfd0 0x7f7da80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.469+0000 7f7dae1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da80cc100 0x7f7da80cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.469+0000 7f7dae1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da80686c0 con 0x7f7da813c510 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.469+0000 7f7dae1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da8069520 con 0x7f7da806bfd0 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.469+0000 7f7dae1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da8068910 con 0x7f7da80cc100 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.469+0000 7f7da6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7da806bfd0 0x7f7da80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7da806bfd0 0x7f7da80c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39498/0 (socket says 172.21.15.131:39498) 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da6ffd700 1 -- 172.21.15.131:0/4019043255 learned_addr learned my addr 172.21.15.131:0/4019043255 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da7fff700 1 --2- 172.21.15.131:0/4019043255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da80cc100 0x7f7da80cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da6ffd700 1 -- 172.21.15.131:0/4019043255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da80cc100 msgr2=0x7f7da80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da6ffd700 1 --2- 172.21.15.131:0/4019043255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da80cc100 0x7f7da80cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da6ffd700 1 -- 172.21.15.131:0/4019043255 >> v1:172.21.15.190:6789/0 conn(0x7f7da813c510 legacy=0x7f7da806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da6ffd700 1 -- 172.21.15.131:0/4019043255 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da8068bf0 con 0x7f7da806bfd0 2023-12-26T21:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da67fc700 1 -- 172.21.15.131:0/4019043255 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884508916 0 0) 0x7f7da80686c0 con 0x7f7da813c510 2023-12-26T21:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da6ffd700 1 --2- 172.21.15.131:0/4019043255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da806bfd0 0x7f7da80c8a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0099b0 tx=0x7f7d9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697121cfdf02ea87 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da67fc700 1 -- 172.21.15.131:0/4019043255 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c009ea0 con 0x7f7da806bfd0 2023-12-26T21:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da67fc700 1 -- 172.21.15.131:0/4019043255 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c00b860 con 0x7f7da806bfd0 2023-12-26T21:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.470+0000 7f7da67fc700 1 -- 172.21.15.131:0/4019043255 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00bc00 con 0x7f7da806bfd0 2023-12-26T21:37:06.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.471+0000 7f7dae1ff700 1 -- 172.21.15.131:0/4019043255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da806bfd0 msgr2=0x7f7da80c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:06.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.471+0000 7f7dae1ff700 1 --2- 172.21.15.131:0/4019043255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da806bfd0 0x7f7da80c8a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0099b0 tx=0x7f7d9c009250 comp rx=0 tx=0).stop 2023-12-26T21:37:06.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.471+0000 7f7dae1ff700 1 -- 172.21.15.131:0/4019043255 shutdown_connections 2023-12-26T21:37:06.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.471+0000 7f7dae1ff700 1 --2- 172.21.15.131:0/4019043255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da80cc100 0x7f7da80cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:06.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.471+0000 7f7dae1ff700 1 --2- 172.21.15.131:0/4019043255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da806bfd0 0x7f7da80c8a20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:06.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.471+0000 7f7dae1ff700 1 -- 172.21.15.131:0/4019043255 >> 172.21.15.131:0/4019043255 conn(0x7f7da80ce3e0 msgr2=0x7f7da80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:06.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.471+0000 7f7dae1ff700 1 -- 172.21.15.131:0/4019043255 shutdown_connections 2023-12-26T21:37:06.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.471+0000 7f7dae1ff700 1 -- 172.21.15.131:0/4019043255 wait complete. 2023-12-26T21:37:06.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.473+0000 7f7dae1ff700 1 Processor -- start 2023-12-26T21:37:06.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.473+0000 7f7dae1ff700 1 -- start start 2023-12-26T21:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7dae1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da806bfd0 0x7f7da80b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7dae1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da80cc100 0x7f7da8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7dae1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da813c510 0x7f7da816da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7dae1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da8155780 con 0x7f7da80cc100 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7dae1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b19e0 con 0x7f7da813c510 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7dae1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da8155240 con 0x7f7da806bfd0 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7da6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da80cc100 0x7f7da8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7da6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da80cc100 0x7f7da8154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39504/0 (socket says 172.21.15.131:39504) 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7da6ffd700 1 -- 172.21.15.131:0/3264958834 learned_addr learned my addr 172.21.15.131:0/3264958834 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7da6ffd700 1 -- 172.21.15.131:0/3264958834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da806bfd0 msgr2=0x7f7da80b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7da6ffd700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da806bfd0 0x7f7da80b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7da6ffd700 1 -- 172.21.15.131:0/3264958834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da813c510 msgr2=0x7f7da816da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7da6ffd700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da813c510 0x7f7da816da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.474+0000 7f7da6ffd700 1 -- 172.21.15.131:0/3264958834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c0094f0 con 0x7f7da80cc100 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.475+0000 7f7da6ffd700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da80cc100 0x7f7da8154d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7da806fe40 tx=0x7f7d9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.475+0000 7f7d8f7fe700 1 -- 172.21.15.131:0/3264958834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00bbe0 con 0x7f7da80cc100 2023-12-26T21:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.475+0000 7f7d8f7fe700 1 -- 172.21.15.131:0/3264958834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c011420 con 0x7f7da80cc100 2023-12-26T21:37:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.475+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da81554a0 con 0x7f7da80cc100 2023-12-26T21:37:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.475+0000 7f7d8f7fe700 1 -- 172.21.15.131:0/3264958834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0117e0 con 0x7f7da80cc100 2023-12-26T21:37:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.475+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da80b1600 con 0x7f7da80cc100 2023-12-26T21:37:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.476+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da80657e0 con 0x7f7da80cc100 2023-12-26T21:37:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.477+0000 7f7d8f7fe700 1 -- 172.21.15.131:0/3264958834 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7d9c011c10 con 0x7f7da80cc100 2023-12-26T21:37:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.478+0000 7f7d8f7fe700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7d94066f30 0x7f7d940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.478+0000 7f7d8f7fe700 1 -- 172.21.15.131:0/3264958834 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7d9c085ea0 con 0x7f7da80cc100 2023-12-26T21:37:06.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.478+0000 7f7da77fe700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7d94066f30 0x7f7d940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:06.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.479+0000 7f7da77fe700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7d94066f30 0x7f7d940693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7d98001720 tx=0x7f7d98007990 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:06.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.482+0000 7f7d8f7fe700 1 -- 172.21.15.131:0/3264958834 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d9c053070 con 0x7f7da80cc100 2023-12-26T21:37:06.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.719+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7da8068ed0 con 0x7f7da80cc100 2023-12-26T21:37:06.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.720+0000 7f7d8f7fe700 1 -- 172.21.15.131:0/3264958834 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7d9c015060 con 0x7f7da80cc100 2023-12-26T21:37:06.720 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7d94066f30 msgr2=0x7f7d940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7d94066f30 0x7f7d940693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7d98001720 tx=0x7f7d98007990 comp rx=0 tx=0).stop 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da80cc100 msgr2=0x7f7da8154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da80cc100 0x7f7da8154d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7da806fe40 tx=0x7f7d9c017000 comp rx=0 tx=0).stop 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 shutdown_connections 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7da813c510 0x7f7da816da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7d94066f30 0x7f7d940693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7da806bfd0 0x7f7da80b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 --2- 172.21.15.131:0/3264958834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7da80cc100 0x7f7da8154d00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 >> 172.21.15.131:0/3264958834 conn(0x7f7da80ce3e0 msgr2=0x7f7da813aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 shutdown_connections 2023-12-26T21:37:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:06.722+0000 7f7dae1ff700 1 -- 172.21.15.131:0/3264958834 wait complete. 2023-12-26T21:37:06.740 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-26T21:37:06.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:37:06.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T21:37:06.741 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T21:37:06.741 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T21:37:06.741 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-26T21:37:06.741 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-26T21:37:06.741 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:37:06.741 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:37:06.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:06.741 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T21:37:06.785 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:37:06.785 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:37:06.785 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:37:06.785 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:06.785 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:37:06.785 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:06.785 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42429sec preferred_lft 42429sec 2023-12-26T21:37:06.786 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T21:37:06.787 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:06.788 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-26T21:37:06.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:06.788 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:06.788 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:06.788 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2023-12-26T21:37:06.788 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2023-12-26T21:37:06.788 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2023-12-26T21:37:06.789 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-26T21:37:06.789 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:07.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:07.255 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:37:07.325 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T21:37:07.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:07.329 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T21:37:07.402 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T21:37:07.402 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:37:07.402 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:37:07.402 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:37:07.402 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:37:07.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:07.403 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:07.403 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:07.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:37:07.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:37:07.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:37:07.403 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:08.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:08.067 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:37:08.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:08.112 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2023-12-26T21:37:08.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:08.195 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:08.196 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:08.196 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T21:37:08.196 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-26T21:37:08.196 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:08.378 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-26T21:37:08.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:08.378 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:08.378 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:08.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-26T21:37:08.379 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-26T21:37:08.379 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-26T21:37:08.379 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-26T21:37:08.379 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-26T21:37:08.379 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:08.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:08.651 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:08.651 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:08.651 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T21:37:08.651 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2023-12-26T21:37:08.651 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:08.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:08.823 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T21:37:08.866 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:37:08.867 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:08.867 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:37:08.949 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T21:37:08.949 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:37:08.949 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:09.095 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:09.096 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:09.551 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:09.551 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:09.551 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:37:09.551 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:37:09.551 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:37:09.556 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:37:09.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:09.557 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:37:09.627 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-12-26T21:37:09.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:37:09.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.815+0000 7f29d59ff700 1 Processor -- start 2023-12-26T21:37:09.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.815+0000 7f29d59ff700 1 -- start start 2023-12-26T21:37:09.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29d59ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f29d00cb320 0x7f29d0069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:09.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29d59ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 0x7f29d00c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:09.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29d59ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d0156940 con 0x7f29d006a1e0 2023-12-26T21:37:09.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29d59ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d00c8cc0 con 0x7f29d00cb320 2023-12-26T21:37:09.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29d59ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00ad700 con 0x7f29d00b0e60 2023-12-26T21:37:09.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29ceffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f29d00cb320 0x7f29d0069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29ceffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f29d00cb320 0x7f29d0069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39516/0 (socket says 172.21.15.131:39516) 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29ceffd700 1 -- 172.21.15.131:0/3096353154 learned_addr learned my addr 172.21.15.131:0/3096353154 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29ce7fc700 1 --1- 172.21.15.131:0/3096353154 >> v1:172.21.15.131:6789/0 conn(0x7f29d006a1e0 0x7f29d00b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55624/0 (socket says 172.21.15.131:55624) 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29cf7fe700 1 --2- 172.21.15.131:0/3096353154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 0x7f29d00c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29ceffd700 1 -- 172.21.15.131:0/3096353154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 msgr2=0x7f29d00c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29ceffd700 1 --2- 172.21.15.131:0/3096353154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 0x7f29d00c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29ceffd700 1 -- 172.21.15.131:0/3096353154 >> v1:172.21.15.131:6789/0 conn(0x7f29d006a1e0 legacy=0x7f29d00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29ceffd700 1 -- 172.21.15.131:0/3096353154 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d00ae500 con 0x7f29d00cb320 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.817+0000 7f29cdffb700 1 -- 172.21.15.131:0/3096353154 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555804060 0 0) 0x7f29d0156940 con 0x7f29d006a1e0 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.818+0000 7f29ceffd700 1 --2- 172.21.15.131:0/3096353154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d00cb320 0x7f29d0069ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f29c00099b0 tx=0x7f29c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4755952d6ba2f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.818+0000 7f29cdffb700 1 -- 172.21.15.131:0/3096353154 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0009ea0 con 0x7f29d00cb320 2023-12-26T21:37:09.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.818+0000 7f29cdffb700 1 -- 172.21.15.131:0/3096353154 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c000b860 con 0x7f29d00cb320 2023-12-26T21:37:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.818+0000 7f29cdffb700 1 -- 172.21.15.131:0/3096353154 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c000bc00 con 0x7f29d00cb320 2023-12-26T21:37:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.818+0000 7f29d59ff700 1 -- 172.21.15.131:0/3096353154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d00cb320 msgr2=0x7f29d0069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.818+0000 7f29d59ff700 1 --2- 172.21.15.131:0/3096353154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d00cb320 0x7f29d0069ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f29c00099b0 tx=0x7f29c0009250 comp rx=0 tx=0).stop 2023-12-26T21:37:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.818+0000 7f29d59ff700 1 -- 172.21.15.131:0/3096353154 shutdown_connections 2023-12-26T21:37:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.818+0000 7f29d59ff700 1 --2- 172.21.15.131:0/3096353154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 0x7f29d00c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.819+0000 7f29d59ff700 1 --2- 172.21.15.131:0/3096353154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d00cb320 0x7f29d0069ca0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.819+0000 7f29d59ff700 1 -- 172.21.15.131:0/3096353154 >> 172.21.15.131:0/3096353154 conn(0x7f29d0068040 msgr2=0x7f29d0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.819+0000 7f29d59ff700 1 -- 172.21.15.131:0/3096353154 shutdown_connections 2023-12-26T21:37:09.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.819+0000 7f29d59ff700 1 -- 172.21.15.131:0/3096353154 wait complete. 2023-12-26T21:37:09.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.820+0000 7f29d59ff700 1 Processor -- start 2023-12-26T21:37:09.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.820+0000 7f29d59ff700 1 -- start start 2023-12-26T21:37:09.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29d59ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d006a1e0 0x7f29d0150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:09.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29d59ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 0x7f29d00c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:09.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29d59ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d00cb320 0x7f29d00ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:09.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29d59ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00ac890 con 0x7f29d006a1e0 2023-12-26T21:37:09.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29d59ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0156ec0 con 0x7f29d00cb320 2023-12-26T21:37:09.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29d59ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01568b0 con 0x7f29d00b0e60 2023-12-26T21:37:09.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29ceffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d006a1e0 0x7f29d0150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:09.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29ceffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d006a1e0 0x7f29d0150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39528/0 (socket says 172.21.15.131:39528) 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29ceffd700 1 -- 172.21.15.131:0/624834789 learned_addr learned my addr 172.21.15.131:0/624834789 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29ce7fc700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 0x7f29d00c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.821+0000 7f29cf7fe700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d00cb320 0x7f29d00ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29ceffd700 1 -- 172.21.15.131:0/624834789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 msgr2=0x7f29d00c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29ceffd700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 0x7f29d00c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29ceffd700 1 -- 172.21.15.131:0/624834789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d00cb320 msgr2=0x7f29d00ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29ceffd700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d00cb320 0x7f29d00ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29ceffd700 1 -- 172.21.15.131:0/624834789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00094f0 con 0x7f29d006a1e0 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29cf7fe700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d00cb320 0x7f29d00ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29ceffd700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d006a1e0 0x7f29d0150020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006fd0 tx=0x7f29c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29b6ffd700 1 -- 172.21.15.131:0/624834789 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c000b9d0 con 0x7f29d006a1e0 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29b6ffd700 1 -- 172.21.15.131:0/624834789 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c000bb50 con 0x7f29d006a1e0 2023-12-26T21:37:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d00ac5b0 con 0x7f29d006a1e0 2023-12-26T21:37:09.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29b6ffd700 1 -- 172.21.15.131:0/624834789 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0025670 con 0x7f29d006a1e0 2023-12-26T21:37:09.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.822+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d0150560 con 0x7f29d006a1e0 2023-12-26T21:37:09.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.823+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d01571a0 con 0x7f29d006a1e0 2023-12-26T21:37:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.824+0000 7f29b6ffd700 1 -- 172.21.15.131:0/624834789 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f29c0023070 con 0x7f29d006a1e0 2023-12-26T21:37:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.825+0000 7f29b6ffd700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f29bc066f30 0x7f29bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.825+0000 7f29b6ffd700 1 -- 172.21.15.131:0/624834789 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f29c0085a80 con 0x7f29d006a1e0 2023-12-26T21:37:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.825+0000 7f29ce7fc700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f29bc066f30 0x7f29bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.826+0000 7f29ce7fc700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f29bc066f30 0x7f29bc0693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f29d0155e40 tx=0x7f29c4007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:09.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:09.829+0000 7f29b6ffd700 1 -- 172.21.15.131:0/624834789 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29c0052c50 con 0x7f29d006a1e0 2023-12-26T21:37:10.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.069+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f29d0156710 con 0x7f29d006a1e0 2023-12-26T21:37:10.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.069+0000 7f29b6ffd700 1 -- 172.21.15.131:0/624834789 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29d0156710 con 0x7f29d006a1e0 2023-12-26T21:37:10.070 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:37:10.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f29bc066f30 msgr2=0x7f29bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:10.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f29bc066f30 0x7f29bc0693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f29d0155e40 tx=0x7f29c4007d50 comp rx=0 tx=0).stop 2023-12-26T21:37:10.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d006a1e0 msgr2=0x7f29d0150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:10.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d006a1e0 0x7f29d0150020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006fd0 tx=0x7f29c0017000 comp rx=0 tx=0).stop 2023-12-26T21:37:10.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 shutdown_connections 2023-12-26T21:37:10.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29d00cb320 0x7f29d00ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:10.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29d006a1e0 0x7f29d0150020 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:10.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f29bc066f30 0x7f29bc0693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:10.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 --2- 172.21.15.131:0/624834789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29d00b0e60 0x7f29d00c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:10.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 >> 172.21.15.131:0/624834789 conn(0x7f29d0068040 msgr2=0x7f29d0151ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:10.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.071+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 shutdown_connections 2023-12-26T21:37:10.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:10.072+0000 7f29d59ff700 1 -- 172.21.15.131:0/624834789 wait complete. 2023-12-26T21:37:10.090 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-26T21:37:10.090 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:37:10.090 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T21:37:10.091 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-12-26T21:37:10.091 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-26T21:37:10.091 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-26T21:37:10.091 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-26T21:37:10.091 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:37:10.091 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-12-26T21:37:10.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.091 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T21:37:10.136 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:37:10.136 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:37:10.136 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:37:10.136 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.136 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:37:10.136 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.136 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:10.137 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:10.137 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T21:37:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:10.137 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:10.137 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T21:37:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:10.137 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:37:10.137 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:10.138 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:37:10.138 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42426sec preferred_lft 42426sec 2023-12-26T21:37:10.138 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T21:37:10.138 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:37:10.138 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:10.138 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T21:37:10.138 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.139 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T21:37:10.139 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:37:10.139 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T21:37:10.139 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link tentative 2023-12-26T21:37:10.139 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.140 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:37:10.242 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T21:37:10.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.245 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T21:37:10.291 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T21:37:10.291 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:37:10.291 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:37:10.291 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:37:10.291 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:37:10.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.292 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:10.292 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:10.292 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:37:10.292 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:37:10.292 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:37:10.292 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:10.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.502 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:37:10.545 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T21:37:10.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.546 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2023-12-26T21:37:10.629 INFO:teuthology.orchestra.run.smithi131.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-12-26T21:37:10.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.630 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:10.630 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:10.630 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T21:37:10.630 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-12-26T21:37:10.630 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:10.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.812 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-26T21:37:10.881 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:37:10.881 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:37:10.881 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:37:10.881 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.881 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:37:10.882 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.882 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:37:10.882 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether f2:83:e1:4e:1e:b9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-26T21:37:10.882 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-26T21:37:10.882 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.882 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::f083:e1ff:fe4e:1eb9/64 scope link 2023-12-26T21:37:10.882 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.884 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-26T21:37:10.953 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:37:10.953 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:37:10.953 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:37:10.953 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.954 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:37:10.954 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.954 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:37:10.954 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether f2:83:e1:4e:1e:b9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-26T21:37:10.954 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-26T21:37:10.954 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.954 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::f083:e1ff:fe4e:1eb9/64 scope link 2023-12-26T21:37:10.954 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:10.957 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-12-26T21:37:10.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:10.957 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:10.957 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:10.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-12-26T21:37:10.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-26T21:37:10.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-12-26T21:37:10.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-12-26T21:37:10.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-12-26T21:37:10.957 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:11.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:11.209 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:11.209 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:11.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.1 up 2023-12-26T21:37:11.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.1 master ceph-brx 2023-12-26T21:37:11.209 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:11.381 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:11.381 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-12-26T21:37:11.424 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-12-26T21:37:11.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:11.425 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-12-26T21:37:11.507 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-12-26T21:37:11.507 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:37:11.507 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:11.645 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:11.645 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:11.674 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:11.675 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:11.675 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:37:11.675 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 2@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:37:11.675 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:37:11.679 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:37:11.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:11.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-12-26T21:37:11.747 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-12-26T21:37:11.747 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:37:11.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.937+0000 7f8ec5dff700 1 Processor -- start 2023-12-26T21:37:11.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.938+0000 7f8ec5dff700 1 -- start start 2023-12-26T21:37:11.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.939+0000 7f8ec5dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ec006e480 0x7f8ec00abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:11.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.939+0000 7f8ec5dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec015a8c0 con 0x7f8ec00aa480 2023-12-26T21:37:11.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.939+0000 7f8ec5dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec0159f90 con 0x7f8ec0159820 2023-12-26T21:37:11.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.939+0000 7f8ec5dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec015ad50 con 0x7f8ec006e480 2023-12-26T21:37:11.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.939+0000 7f8ebffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8ec00aa480 0x7f8ec00675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55640/0 (socket says 172.21.15.131:55640) 2023-12-26T21:37:11.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.939+0000 7f8ebffff700 1 -- 172.21.15.131:0/3082510466 learned_addr learned my addr 172.21.15.131:0/3082510466 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:11.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.939+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352733435 0 0) 0x7f8ec015a8c0 con 0x7f8ec00aa480 2023-12-26T21:37:11.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.939+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea8003570 con 0x7f8ec00aa480 2023-12-26T21:37:11.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebeffd700 1 --2- 172.21.15.131:0/3082510466 >> v2:172.21.15.190:3300/0 conn(0x7f8ec006e480 0x7f8ec00abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:11.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987558091 0 0) 0x7f8ea8003570 con 0x7f8ec00aa480 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 >> v2:172.21.15.190:3300/0 conn(0x7f8ec006e480 msgr2=0x7f8ec00abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 --2- 172.21.15.131:0/3082510466 >> v2:172.21.15.190:3300/0 conn(0x7f8ec006e480 0x7f8ec00abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 >> v1:172.21.15.190:6789/0 conn(0x7f8ec0159820 legacy=0x7f8ec006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec015a270 con 0x7f8ec00aa480 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f8eb80017f0 con 0x7f8ec00aa480 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 >> v1:172.21.15.131:6789/0 conn(0x7f8ec00aa480 legacy=0x7f8ec00675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ea8003ab0 0x7f8ea8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ea8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ea80052e0 0x7f8ea800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea800c230 con 0x7f8ea8004810 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea800c3a0 con 0x7f8ea80052e0 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea800c560 con 0x7f8ea8003ab0 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebeffd700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ea8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:11.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebffff700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ea80052e0 0x7f8ea800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.940+0000 7f8ebf7fe700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ea8003ab0 0x7f8ea8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebeffd700 1 -- 172.21.15.131:0/3082510466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ea8003ab0 msgr2=0x7f8ea8003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebeffd700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ea8003ab0 0x7f8ea8003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebeffd700 1 -- 172.21.15.131:0/3082510466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ea80052e0 msgr2=0x7f8ea800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebeffd700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ea80052e0 0x7f8ea800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebeffd700 1 -- 172.21.15.131:0/3082510466 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8003570 con 0x7f8ea8004810 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebffff700 1 -- 172.21.15.131:0/3082510466 reap_dead start 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebeffd700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ea8004c80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400bb90 tx=0x7f8eb4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec1cc332f86e6c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400d5d0 con 0x7f8ea8004810 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb400d730 con 0x7f8ea8004810 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.941+0000 7f8ebe7fc700 1 -- 172.21.15.131:0/3082510466 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400daf0 con 0x7f8ea8004810 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.942+0000 7f8ec5dff700 1 -- 172.21.15.131:0/3082510466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 msgr2=0x7f8ea8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.942+0000 7f8ec5dff700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ea8004c80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400bb90 tx=0x7f8eb4008040 comp rx=0 tx=0).stop 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.942+0000 7f8ec5dff700 1 -- 172.21.15.131:0/3082510466 shutdown_connections 2023-12-26T21:37:11.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.942+0000 7f8ec5dff700 1 --2- 172.21.15.131:0/3082510466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ea8004c80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:11.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.942+0000 7f8ec5dff700 1 -- 172.21.15.131:0/3082510466 >> 172.21.15.131:0/3082510466 conn(0x7f8ec006c290 msgr2=0x7f8ec00ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:11.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.942+0000 7f8ec5dff700 1 -- 172.21.15.131:0/3082510466 shutdown_connections 2023-12-26T21:37:11.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.942+0000 7f8ec5dff700 1 -- 172.21.15.131:0/3082510466 wait complete. 2023-12-26T21:37:11.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.943+0000 7f8ec5dff700 1 Processor -- start 2023-12-26T21:37:11.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.943+0000 7f8ec5dff700 1 -- start start 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.944+0000 7f8ec5dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ec01497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.944+0000 7f8ec5dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ec00c3890 0x7f8ec00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.944+0000 7f8ec5dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ec00b1f50 0x7f8ec00c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.944+0000 7f8ec5dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec00b23c0 con 0x7f8ea8004810 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.944+0000 7f8ec5dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec00c3400 con 0x7f8ec00c3890 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.944+0000 7f8ec5dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec00c27f0 con 0x7f8ec00b1f50 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.944+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ec01497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ec01497b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39538/0 (socket says 172.21.15.131:39538) 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebf7fe700 1 -- 172.21.15.131:0/1689316811 learned_addr learned my addr 172.21.15.131:0/1689316811 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebeffd700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ec00c3890 0x7f8ec00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebffff700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ec00b1f50 0x7f8ec00c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebf7fe700 1 -- 172.21.15.131:0/1689316811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ec00b1f50 msgr2=0x7f8ec00c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebf7fe700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ec00b1f50 0x7f8ec00c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebf7fe700 1 -- 172.21.15.131:0/1689316811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ec00c3890 msgr2=0x7f8ec00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:11.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebf7fe700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ec00c3890 0x7f8ec00b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:11.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebf7fe700 1 -- 172.21.15.131:0/1689316811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec00c2ad0 con 0x7f8ea8004810 2023-12-26T21:37:11.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ebf7fe700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ec01497b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0002950 tx=0x7f8eb0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:11.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ea77fe700 1 -- 172.21.15.131:0/1689316811 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00033b0 con 0x7f8ea8004810 2023-12-26T21:37:11.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ea77fe700 1 -- 172.21.15.131:0/1689316811 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb0006a30 con 0x7f8ea8004810 2023-12-26T21:37:11.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec00c2db0 con 0x7f8ea8004810 2023-12-26T21:37:11.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.945+0000 7f8ea77fe700 1 -- 172.21.15.131:0/1689316811 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0006df0 con 0x7f8ea8004810 2023-12-26T21:37:11.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.946+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec00c1fc0 con 0x7f8ea8004810 2023-12-26T21:37:11.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.947+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec00c36e0 con 0x7f8ea8004810 2023-12-26T21:37:11.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.947+0000 7f8ea77fe700 1 -- 172.21.15.131:0/1689316811 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8eb0007260 con 0x7f8ea8004810 2023-12-26T21:37:11.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.948+0000 7f8ea77fe700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8eac066f30 0x7f8eac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:11.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.948+0000 7f8ea77fe700 1 -- 172.21.15.131:0/1689316811 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8eb00844e0 con 0x7f8ea8004810 2023-12-26T21:37:11.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.949+0000 7f8ebeffd700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8eac066f30 0x7f8eac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:11.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.949+0000 7f8ebeffd700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8eac066f30 0x7f8eac0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400bb90 tx=0x7f8eb401b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:11.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:11.952+0000 7f8ea77fe700 1 -- 172.21.15.131:0/1689316811 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8eb00551e0 con 0x7f8ea8004810 2023-12-26T21:37:12.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.195+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8ec00657e0 con 0x7f8ea8004810 2023-12-26T21:37:12.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.195+0000 7f8ea77fe700 1 -- 172.21.15.131:0/1689316811 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8eb0054d70 con 0x7f8ea8004810 2023-12-26T21:37:12.196 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:37:12.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8eac066f30 msgr2=0x7f8eac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:12.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8eac066f30 0x7f8eac0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400bb90 tx=0x7f8eb401b000 comp rx=0 tx=0).stop 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 msgr2=0x7f8ec01497b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ec01497b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0002950 tx=0x7f8eb0007d30 comp rx=0 tx=0).stop 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 shutdown_connections 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8eac066f30 0x7f8eac0693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ec00b1f50 0x7f8ec00c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea8004810 0x7f8ec01497b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 --2- 172.21.15.131:0/1689316811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ec00c3890 0x7f8ec00b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 >> 172.21.15.131:0/1689316811 conn(0x7f8ec006c290 msgr2=0x7f8ec0156370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 shutdown_connections 2023-12-26T21:37:12.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:12.198+0000 7f8ec5dff700 1 -- 172.21.15.131:0/1689316811 wait complete. 2023-12-26T21:37:12.216 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-26T21:37:12.216 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:37:12.216 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2023-12-26T21:37:12.216 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-26T21:37:12.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-26T21:37:12.217 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-26T21:37:12.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-26T21:37:12.217 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:37:12.217 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-26T21:37:12.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:12.217 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2023-12-26T21:37:12.266 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:37:12.266 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:37:12.266 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:37:12.266 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:12.266 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-12-26T21:37:12.266 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:12.266 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42401sec preferred_lft 42401sec 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2023-12-26T21:37:12.267 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:12.268 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-26T21:37:12.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:12.269 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-12-26T21:37:12.269 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-12-26T21:37:12.269 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2023-12-26T21:37:12.269 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2023-12-26T21:37:12.269 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2023-12-26T21:37:12.269 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-26T21:37:12.269 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-12-26T21:37:12.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:12.696 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:37:12.768 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-12-26T21:37:12.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:12.772 DEBUG:teuthology.orchestra.run.smithi190:> route 2023-12-26T21:37:12.832 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2023-12-26T21:37:12.833 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:37:12.833 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:37:12.833 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:37:12.833 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:37:12.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:12.834 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-12-26T21:37:12.834 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-12-26T21:37:12.834 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:37:12.834 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:37:12.834 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:37:12.834 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-12-26T21:37:13.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:37:13.411 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:37:13.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:13.546 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-12-26T21:37:13.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:13.589 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2023-12-26T21:37:13.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:13.673 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-12-26T21:37:13.673 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-12-26T21:37:13.673 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-26T21:37:13.673 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-12-26T21:37:13.673 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-12-26T21:37:13.855 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-12-26T21:37:13.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:13.856 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-12-26T21:37:13.856 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-12-26T21:37:13.856 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-12-26T21:37:13.856 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-26T21:37:13.856 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-12-26T21:37:13.856 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-12-26T21:37:13.856 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-12-26T21:37:13.856 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-12-26T21:37:14.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:14.132 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-12-26T21:37:14.132 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-12-26T21:37:14.132 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2023-12-26T21:37:14.133 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2023-12-26T21:37:14.133 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-12-26T21:37:14.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:14.304 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-26T21:37:14.347 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-26T21:37:14.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:14.348 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-26T21:37:14.431 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-12-26T21:37:14.431 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:37:14.431 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:14.569 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:14.570 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:14.997 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:14.997 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:14.997 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:37:14.997 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with new device syntax: 1@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:37:14.997 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:37:15.003 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:37:15.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:15.003 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-26T21:37:15.072 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-12-26T21:37:15.073 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:37:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.272+0000 7f47ae9ff700 1 Processor -- start 2023-12-26T21:37:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.272+0000 7f47ae9ff700 1 -- start start 2023-12-26T21:37:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.273+0000 7f47ae9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47a813c510 0x7f47a80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.273+0000 7f47ae9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a80cc810 0x7f47a806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.273+0000 7f47ae9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a80686c0 con 0x7f47a806b8c0 2023-12-26T21:37:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47ae9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a806c280 con 0x7f47a813c510 2023-12-26T21:37:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47ae9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8069520 con 0x7f47a80cc810 2023-12-26T21:37:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47a7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47a813c510 0x7f47a80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47a77fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f47a806b8c0 0x7f47a80cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35360/0 (socket says 172.21.15.131:35360) 2023-12-26T21:37:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47a77fe700 1 -- 172.21.15.131:0/529454461 learned_addr learned my addr 172.21.15.131:0/529454461 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47acf9c700 1 --2- 172.21.15.131:0/529454461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a80cc810 0x7f47a806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:15.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47a7fff700 1 -- 172.21.15.131:0/529454461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a80cc810 msgr2=0x7f47a806bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:15.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47a7fff700 1 --2- 172.21.15.131:0/529454461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a80cc810 0x7f47a806bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47a7fff700 1 -- 172.21.15.131:0/529454461 >> v1:172.21.15.131:6789/0 conn(0x7f47a806b8c0 legacy=0x7f47a80cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:15.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47a7fff700 1 -- 172.21.15.131:0/529454461 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8068910 con 0x7f47a813c510 2023-12-26T21:37:15.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.274+0000 7f47a6ffd700 1 -- 172.21.15.131:0/529454461 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156528592 0 0) 0x7f47a80686c0 con 0x7f47a806b8c0 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47a7fff700 1 --2- 172.21.15.131:0/529454461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 0x7f47a80c8a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47980098d0 tx=0x7f47980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65a6c63db9cf716 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47a6ffd700 1 -- 172.21.15.131:0/529454461 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479800b640 con 0x7f47a813c510 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47a6ffd700 1 -- 172.21.15.131:0/529454461 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479800b7a0 con 0x7f47a813c510 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47a6ffd700 1 -- 172.21.15.131:0/529454461 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479800bb60 con 0x7f47a813c510 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47ae9ff700 1 -- 172.21.15.131:0/529454461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 msgr2=0x7f47a80c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47ae9ff700 1 --2- 172.21.15.131:0/529454461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 0x7f47a80c8a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47980098d0 tx=0x7f47980091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47ae9ff700 1 -- 172.21.15.131:0/529454461 shutdown_connections 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47ae9ff700 1 --2- 172.21.15.131:0/529454461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a80cc810 0x7f47a806bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47ae9ff700 1 --2- 172.21.15.131:0/529454461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 0x7f47a80c8a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47ae9ff700 1 -- 172.21.15.131:0/529454461 >> 172.21.15.131:0/529454461 conn(0x7f47a80ce3e0 msgr2=0x7f47a80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47ae9ff700 1 -- 172.21.15.131:0/529454461 shutdown_connections 2023-12-26T21:37:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.275+0000 7f47ae9ff700 1 -- 172.21.15.131:0/529454461 wait complete. 2023-12-26T21:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.277+0000 7f47ae9ff700 1 Processor -- start 2023-12-26T21:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.277+0000 7f47ae9ff700 1 -- start start 2023-12-26T21:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47ae9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a806b8c0 0x7f47a80b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47ae9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a80cc810 0x7f47a80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47ae9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 0x7f47a80b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47ae9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80b3b10 con 0x7f47a813c510 2023-12-26T21:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47ae9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a815b250 con 0x7f47a80cc810 2023-12-26T21:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47ae9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a815a680 con 0x7f47a806b8c0 2023-12-26T21:37:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47acf9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 0x7f47a80b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47acf9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 0x7f47a80b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55408/0 (socket says 172.21.15.131:55408) 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47acf9c700 1 -- 172.21.15.131:0/3942559509 learned_addr learned my addr 172.21.15.131:0/3942559509 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47a7fff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a806b8c0 0x7f47a80b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47a77fe700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a80cc810 0x7f47a80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47acf9c700 1 -- 172.21.15.131:0/3942559509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a806b8c0 msgr2=0x7f47a80b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.278+0000 7f47acf9c700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a806b8c0 0x7f47a80b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f47acf9c700 1 -- 172.21.15.131:0/3942559509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a80cc810 msgr2=0x7f47a80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f47acf9c700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a80cc810 0x7f47a80b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f47acf9c700 1 -- 172.21.15.131:0/3942559509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980093c0 con 0x7f47a813c510 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f47a77fe700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a80cc810 0x7f47a80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f47acf9c700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 0x7f47a80b35d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009e90 tx=0x7f47a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f478ffff700 1 -- 172.21.15.131:0/3942559509 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a000cc20 con 0x7f47a813c510 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f478ffff700 1 -- 172.21.15.131:0/3942559509 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a000f040 con 0x7f47a813c510 2023-12-26T21:37:15.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a815a920 con 0x7f47a813c510 2023-12-26T21:37:15.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f478ffff700 1 -- 172.21.15.131:0/3942559509 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0014480 con 0x7f47a813c510 2023-12-26T21:37:15.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.279+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a815aee0 con 0x7f47a813c510 2023-12-26T21:37:15.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.280+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a815b3c0 con 0x7f47a813c510 2023-12-26T21:37:15.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.281+0000 7f478ffff700 1 -- 172.21.15.131:0/3942559509 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f47a0014830 con 0x7f47a813c510 2023-12-26T21:37:15.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.282+0000 7f478ffff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4794066f30 0x7f47940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:15.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.282+0000 7f478ffff700 1 -- 172.21.15.131:0/3942559509 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f47a00852b0 con 0x7f47a813c510 2023-12-26T21:37:15.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.283+0000 7f47a7fff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4794066f30 0x7f47940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:15.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.283+0000 7f47a7fff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4794066f30 0x7f47940693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f47980091e0 tx=0x7f4798017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:15.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.287+0000 7f478ffff700 1 -- 172.21.15.131:0/3942559509 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47a0055f10 con 0x7f47a813c510 2023-12-26T21:37:15.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.522+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f47a80657e0 con 0x7f47a813c510 2023-12-26T21:37:15.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.522+0000 7f478ffff700 1 -- 172.21.15.131:0/3942559509 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f47a0055aa0 con 0x7f47a813c510 2023-12-26T21:37:15.523 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:37:15.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.524+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4794066f30 msgr2=0x7f47940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:15.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.524+0000 7f47ae9ff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4794066f30 0x7f47940693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f47980091e0 tx=0x7f4798017000 comp rx=0 tx=0).stop 2023-12-26T21:37:15.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.524+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 msgr2=0x7f47a80b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:15.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.524+0000 7f47ae9ff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 0x7f47a80b35d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009e90 tx=0x7f47a000b040 comp rx=0 tx=0).stop 2023-12-26T21:37:15.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.525+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 shutdown_connections 2023-12-26T21:37:15.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.525+0000 7f47ae9ff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a813c510 0x7f47a80b35d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.525+0000 7f47ae9ff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4794066f30 0x7f47940693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.525+0000 7f47ae9ff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a806b8c0 0x7f47a80b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.525+0000 7f47ae9ff700 1 --2- 172.21.15.131:0/3942559509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a80cc810 0x7f47a80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:15.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.525+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 >> 172.21.15.131:0/3942559509 conn(0x7f47a80ce3e0 msgr2=0x7f47a815e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:15.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.525+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 shutdown_connections 2023-12-26T21:37:15.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:15.525+0000 7f47ae9ff700 1 -- 172.21.15.131:0/3942559509 wait complete. 2023-12-26T21:37:15.543 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-26T21:37:15.543 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:37:15.543 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2023-12-26T21:37:15.543 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-12-26T21:37:15.543 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-26T21:37:15.543 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-26T21:37:15.544 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-26T21:37:15.544 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:37:15.544 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-12-26T21:37:15.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:15.544 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2023-12-26T21:37:15.589 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42397sec preferred_lft 42397sec 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 92:6b:79:b9:63:16 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:15.590 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::9cf6:ccff:fefd:8109/64 scope link 2023-12-26T21:37:15.591 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:15.591 INFO:teuthology.orchestra.run.smithi190.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:37:15.591 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 92:6b:79:b9:63:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-26T21:37:15.591 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::906b:79ff:feb9:6316/64 scope link tentative 2023-12-26T21:37:15.591 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:15.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:15.591 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:37:15.695 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-12-26T21:37:15.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:15.698 DEBUG:teuthology.orchestra.run.smithi190:> route 2023-12-26T21:37:15.743 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2023-12-26T21:37:15.744 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:37:15.744 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:37:15.744 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:37:15.744 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:37:15.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:15.745 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-12-26T21:37:15.745 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-12-26T21:37:15.745 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:37:15.745 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:37:15.745 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:37:15.745 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-12-26T21:37:15.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:15.952 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-12-26T21:37:15.995 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-26T21:37:15.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:15.996 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2023-12-26T21:37:16.079 INFO:teuthology.orchestra.run.smithi190.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-12-26T21:37:16.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:16.081 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-12-26T21:37:16.081 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-12-26T21:37:16.081 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-26T21:37:16.081 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-12-26T21:37:16.081 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-12-26T21:37:16.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:16.261 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-26T21:37:16.329 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:37:16.329 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:37:16.329 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:37:16.329 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:16.329 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-12-26T21:37:16.329 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:16.329 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:37:16.330 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 8e:ba:31:ac:6c:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-26T21:37:16.330 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-26T21:37:16.330 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:16.330 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::8cba:31ff:feac:6cdb/64 scope link 2023-12-26T21:37:16.330 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:16.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:16.332 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-26T21:37:16.399 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 8e:ba:31:ac:6c:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::8cba:31ff:feac:6cdb/64 scope link 2023-12-26T21:37:16.400 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:16.403 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-12-26T21:37:16.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:16.403 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-12-26T21:37:16.403 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-12-26T21:37:16.403 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-12-26T21:37:16.403 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-26T21:37:16.403 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-12-26T21:37:16.403 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-12-26T21:37:16.403 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-12-26T21:37:16.403 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-12-26T21:37:16.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:16.653 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-12-26T21:37:16.653 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-12-26T21:37:16.653 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.1 up 2023-12-26T21:37:16.653 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.1 master ceph-brx 2023-12-26T21:37:16.653 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-12-26T21:37:16.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:16.823 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-12-26T21:37:16.866 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-12-26T21:37:16.866 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:16.866 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-12-26T21:37:16.948 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-12-26T21:37:16.948 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:37:16.948 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:17.087 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:17.088 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:17.116 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:17.116 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:17.116 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:37:17.116 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with new device syntax: 3@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:37:17.116 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:37:17.120 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:37:17.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:17.120 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-12-26T21:37:17.187 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-26T21:37:17.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.406+0000 7f7b61fff700 1 Processor -- start 2023-12-26T21:37:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.406+0000 7f7b61fff700 1 -- start start 2023-12-26T21:37:17.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.407+0000 7f7b61fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c1596a0 0x7f7b5c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.407+0000 7f7b61fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b5c159a70 0x7f7b5c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.407+0000 7f7b61fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5c0afcd0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.407+0000 7f7b61fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5c0b3e70 con 0x7f7b5c159a70 2023-12-26T21:37:17.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.407+0000 7f7b61fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c0b0cd0 con 0x7f7b5c1596a0 2023-12-26T21:37:17.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.407+0000 7f7b5affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b5c159a70 0x7f7b5c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c1596a0 0x7f7b5c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b5c159a70 0x7f7b5c0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55418/0 (socket says 172.21.15.131:55418) 2023-12-26T21:37:17.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5affd700 1 -- 172.21.15.131:0/1406081836 learned_addr learned my addr 172.21.15.131:0/1406081836 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:17.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5affd700 1 -- 172.21.15.131:0/1406081836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c1596a0 msgr2=0x7f7b5c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:17.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5affd700 1 --2- 172.21.15.131:0/1406081836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c1596a0 0x7f7b5c0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5affd700 1 -- 172.21.15.131:0/1406081836 >> v1:172.21.15.131:6789/0 conn(0x7f7b5c0b0160 legacy=0x7f7b5c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5affd700 1 -- 172.21.15.131:0/1406081836 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c0b2e00 con 0x7f7b5c159a70 2023-12-26T21:37:17.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5a7fc700 1 -- 172.21.15.131:0/1406081836 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657635157 0 0) 0x7f7b5c0afcd0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5affd700 1 --2- 172.21.15.131:0/1406081836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c159a70 0x7f7b5c0b28c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b500099b0 tx=0x7f7b50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c756a2acf44ce9f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:17.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5a7fc700 1 -- 172.21.15.131:0/1406081836 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50009ea0 con 0x7f7b5c159a70 2023-12-26T21:37:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5a7fc700 1 -- 172.21.15.131:0/1406081836 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5000b860 con 0x7f7b5c159a70 2023-12-26T21:37:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.408+0000 7f7b5a7fc700 1 -- 172.21.15.131:0/1406081836 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000bc00 con 0x7f7b5c159a70 2023-12-26T21:37:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.409+0000 7f7b61fff700 1 -- 172.21.15.131:0/1406081836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c159a70 msgr2=0x7f7b5c0b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.409+0000 7f7b61fff700 1 --2- 172.21.15.131:0/1406081836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c159a70 0x7f7b5c0b28c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b500099b0 tx=0x7f7b50009250 comp rx=0 tx=0).stop 2023-12-26T21:37:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.409+0000 7f7b61fff700 1 -- 172.21.15.131:0/1406081836 shutdown_connections 2023-12-26T21:37:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.409+0000 7f7b61fff700 1 --2- 172.21.15.131:0/1406081836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c1596a0 0x7f7b5c0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.409+0000 7f7b61fff700 1 --2- 172.21.15.131:0/1406081836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c159a70 0x7f7b5c0b28c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.409+0000 7f7b61fff700 1 -- 172.21.15.131:0/1406081836 >> 172.21.15.131:0/1406081836 conn(0x7f7b5c06ece0 msgr2=0x7f7b5c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.409+0000 7f7b61fff700 1 -- 172.21.15.131:0/1406081836 shutdown_connections 2023-12-26T21:37:17.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.409+0000 7f7b61fff700 1 -- 172.21.15.131:0/1406081836 wait complete. 2023-12-26T21:37:17.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.411+0000 7f7b61fff700 1 Processor -- start 2023-12-26T21:37:17.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.411+0000 7f7b61fff700 1 -- start start 2023-12-26T21:37:17.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b61fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c0b0160 0x7f7b5c16a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b61fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c1596a0 0x7f7b5c15c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b61fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c159a70 0x7f7b5c16cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b61fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c15cec0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b61fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c16a110 con 0x7f7b5c1596a0 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b61fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c16db00 con 0x7f7b5c159a70 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b5b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c0b0160 0x7f7b5c16a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b5b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c0b0160 0x7f7b5c16a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55420/0 (socket says 172.21.15.131:55420) 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b5b7fe700 1 -- 172.21.15.131:0/980429167 learned_addr learned my addr 172.21.15.131:0/980429167 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.412+0000 7f7b5bfff700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c159a70 0x7f7b5c16cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b5affd700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c1596a0 0x7f7b5c15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b5b7fe700 1 -- 172.21.15.131:0/980429167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c159a70 msgr2=0x7f7b5c16cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b5b7fe700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c159a70 0x7f7b5c16cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b5b7fe700 1 -- 172.21.15.131:0/980429167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c1596a0 msgr2=0x7f7b5c15c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b5b7fe700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c1596a0 0x7f7b5c15c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b5b7fe700 1 -- 172.21.15.131:0/980429167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b500094f0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b5affd700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c1596a0 0x7f7b5c15c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b5b7fe700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c0b0160 0x7f7b5c16a5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00ec80 tx=0x7f7b4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:17.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b437fe700 1 -- 172.21.15.131:0/980429167 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00c850 con 0x7f7b5c0b0160 2023-12-26T21:37:17.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b437fe700 1 -- 172.21.15.131:0/980429167 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c00c9b0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b437fe700 1 -- 172.21.15.131:0/980429167 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00cd50 con 0x7f7b5c0b0160 2023-12-26T21:37:17.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5c16dde0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.413+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c169da0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.414+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5c0657e0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.415+0000 7f7b437fe700 1 -- 172.21.15.131:0/980429167 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7b4c013680 con 0x7f7b5c0b0160 2023-12-26T21:37:17.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.416+0000 7f7b437fe700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b48066f30 0x7f7b480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.416+0000 7f7b437fe700 1 -- 172.21.15.131:0/980429167 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7b4c0847f0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.417+0000 7f7b5affd700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b48066f30 0x7f7b480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.417+0000 7f7b5affd700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b48066f30 0x7f7b480693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b500099b0 tx=0x7f7b50017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:17.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.421+0000 7f7b437fe700 1 -- 172.21.15.131:0/980429167 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b4c051a60 con 0x7f7b5c0b0160 2023-12-26T21:37:17.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.617+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b5c0c3db0 con 0x7f7b5c0b0160 2023-12-26T21:37:17.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.617+0000 7f7b437fe700 1 -- 172.21.15.131:0/980429167 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7b4c055080 con 0x7f7b5c0b0160 2023-12-26T21:37:17.618 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:17.618 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b48066f30 msgr2=0x7f7b480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b48066f30 0x7f7b480693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b500099b0 tx=0x7f7b50017000 comp rx=0 tx=0).stop 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c0b0160 msgr2=0x7f7b5c16a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c0b0160 0x7f7b5c16a5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00ec80 tx=0x7f7b4c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 shutdown_connections 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7b48066f30 0x7f7b480693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c159a70 0x7f7b5c16cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b5c0b0160 0x7f7b5c16a5a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 --2- 172.21.15.131:0/980429167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c1596a0 0x7f7b5c15c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 >> 172.21.15.131:0/980429167 conn(0x7f7b5c06ece0 msgr2=0x7f7b5c15d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 shutdown_connections 2023-12-26T21:37:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.620+0000 7f7b61fff700 1 -- 172.21.15.131:0/980429167 wait complete. 2023-12-26T21:37:17.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:36:46.654972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:36:47.264880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:37:17.639 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:37:17.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.830+0000 7fa2e2fff700 1 Processor -- start 2023-12-26T21:37:17.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.831+0000 7fa2e2fff700 1 -- start start 2023-12-26T21:37:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.832+0000 7fa2e2fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 0x7fa2dc0abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.832+0000 7fa2e2fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc15a8c0 con 0x7fa2dc06e480 2023-12-26T21:37:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.832+0000 7fa2e2fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc159f90 con 0x7fa2dc159820 2023-12-26T21:37:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.832+0000 7fa2e2fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc15ad50 con 0x7fa2dc0aa480 2023-12-26T21:37:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.832+0000 7fa2dbfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa2dc06e480 0x7fa2dc0675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35372/0 (socket says 172.21.15.131:35372) 2023-12-26T21:37:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.832+0000 7fa2dbfff700 1 -- 172.21.15.131:0/2740664800 learned_addr learned my addr 172.21.15.131:0/2740664800 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:17.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.832+0000 7fa2e159c700 1 --2- 172.21.15.131:0/2740664800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 0x7fa2dc0abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2db7fe700 1 -- 172.21.15.131:0/2740664800 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523035520 0 0) 0x7fa2dc15a8c0 con 0x7fa2dc06e480 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2db7fe700 1 -- 172.21.15.131:0/2740664800 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2cc003570 con 0x7fa2dc06e480 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2db7fe700 1 -- 172.21.15.131:0/2740664800 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288554563 0 0) 0x7fa2dc159f90 con 0x7fa2dc159820 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2db7fe700 1 -- 172.21.15.131:0/2740664800 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2dc15a8c0 con 0x7fa2dc159820 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2e159c700 1 -- 172.21.15.131:0/2740664800 >> v1:172.21.15.190:6789/0 conn(0x7fa2dc159820 legacy=0x7fa2dc06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2e159c700 1 -- 172.21.15.131:0/2740664800 >> v1:172.21.15.131:6789/0 conn(0x7fa2dc06e480 legacy=0x7fa2dc0675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2e159c700 1 -- 172.21.15.131:0/2740664800 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc15a270 con 0x7fa2dc0aa480 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2e159c700 1 --2- 172.21.15.131:0/2740664800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 0x7fa2dc0abdb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009e90 tx=0x7fa2d4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ed86f72ab997821 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2db7fe700 1 -- 172.21.15.131:0/2740664800 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400e070 con 0x7fa2dc0aa480 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2db7fe700 1 -- 172.21.15.131:0/2740664800 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d400bb80 con 0x7fa2dc0aa480 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.833+0000 7fa2db7fe700 1 -- 172.21.15.131:0/2740664800 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4012430 con 0x7fa2dc0aa480 2023-12-26T21:37:17.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.834+0000 7fa2e2fff700 1 -- 172.21.15.131:0/2740664800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 msgr2=0x7fa2dc0abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.834+0000 7fa2e2fff700 1 --2- 172.21.15.131:0/2740664800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 0x7fa2dc0abdb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009e90 tx=0x7fa2d4009780 comp rx=0 tx=0).stop 2023-12-26T21:37:17.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.834+0000 7fa2e2fff700 1 -- 172.21.15.131:0/2740664800 shutdown_connections 2023-12-26T21:37:17.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.834+0000 7fa2e2fff700 1 --2- 172.21.15.131:0/2740664800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 0x7fa2dc0abdb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.834+0000 7fa2e2fff700 1 -- 172.21.15.131:0/2740664800 >> 172.21.15.131:0/2740664800 conn(0x7fa2dc06c290 msgr2=0x7fa2dc0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:17.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.834+0000 7fa2e2fff700 1 -- 172.21.15.131:0/2740664800 shutdown_connections 2023-12-26T21:37:17.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.834+0000 7fa2e2fff700 1 -- 172.21.15.131:0/2740664800 wait complete. 2023-12-26T21:37:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.835+0000 7fa2e2fff700 1 Processor -- start 2023-12-26T21:37:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.836+0000 7fa2e2fff700 1 -- start start 2023-12-26T21:37:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.836+0000 7fa2e2fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2dc06e480 0x7fa2dc157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.836+0000 7fa2e2fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 0x7fa2dc0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.836+0000 7fa2e2fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2dc159820 0x7fa2dc0c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.836+0000 7fa2e2fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc1696c0 con 0x7fa2dc159820 2023-12-26T21:37:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.836+0000 7fa2e2fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc1699a0 con 0x7fa2dc06e480 2023-12-26T21:37:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.836+0000 7fa2e2fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc169c80 con 0x7fa2dc0aa480 2023-12-26T21:37:17.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e159c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2dc159820 0x7fa2dc0c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e159c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2dc159820 0x7fa2dc0c3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55424/0 (socket says 172.21.15.131:55424) 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e159c700 1 -- 172.21.15.131:0/4235206813 learned_addr learned my addr 172.21.15.131:0/4235206813 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2dbfff700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 0x7fa2dc0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e0d9b700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2dc06e480 0x7fa2dc157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e159c700 1 -- 172.21.15.131:0/4235206813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 msgr2=0x7fa2dc0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e159c700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 0x7fa2dc0cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e159c700 1 -- 172.21.15.131:0/4235206813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2dc06e480 msgr2=0x7fa2dc157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e159c700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2dc06e480 0x7fa2dc157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e159c700 1 -- 172.21.15.131:0/4235206813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d4009a20 con 0x7fa2dc159820 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.837+0000 7fa2e159c700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2dc159820 0x7fa2dc0c3440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc06d0a0 tx=0x7fa2d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.838+0000 7fa2d8ff9700 1 -- 172.21.15.131:0/4235206813 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d401b420 con 0x7fa2dc159820 2023-12-26T21:37:17.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.838+0000 7fa2d8ff9700 1 -- 172.21.15.131:0/4235206813 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d401b5a0 con 0x7fa2dc159820 2023-12-26T21:37:17.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.838+0000 7fa2d8ff9700 1 -- 172.21.15.131:0/4235206813 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d401b960 con 0x7fa2dc159820 2023-12-26T21:37:17.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.838+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc169f60 con 0x7fa2dc159820 2023-12-26T21:37:17.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.838+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc0cc640 con 0x7fa2dc159820 2023-12-26T21:37:17.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.839+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc0657e0 con 0x7fa2dc159820 2023-12-26T21:37:17.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.840+0000 7fa2d8ff9700 1 -- 172.21.15.131:0/4235206813 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa2d401bd50 con 0x7fa2dc159820 2023-12-26T21:37:17.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.840+0000 7fa2d8ff9700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa2c0066f30 0x7fa2c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:17.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.840+0000 7fa2d8ff9700 1 -- 172.21.15.131:0/4235206813 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa2d4085e70 con 0x7fa2dc159820 2023-12-26T21:37:17.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.841+0000 7fa2e0d9b700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa2c0066f30 0x7fa2c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:17.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.841+0000 7fa2e0d9b700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa2c0066f30 0x7fa2c00693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0007fd0 tx=0x7fa2d0007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:17.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:17.844+0000 7fa2d8ff9700 1 -- 172.21.15.131:0/4235206813 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2d40530f0 con 0x7fa2dc159820 2023-12-26T21:37:18.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.039+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2dc169800 con 0x7fa2dc159820 2023-12-26T21:37:18.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.039+0000 7fa2d8ff9700 1 -- 172.21.15.131:0/4235206813 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2d402a020 con 0x7fa2dc159820 2023-12-26T21:37:18.040 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa2c0066f30 msgr2=0x7fa2c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa2c0066f30 0x7fa2c00693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0007fd0 tx=0x7fa2d0007d50 comp rx=0 tx=0).stop 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2dc159820 msgr2=0x7fa2dc0c3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2dc159820 0x7fa2dc0c3440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc06d0a0 tx=0x7fa2d4019000 comp rx=0 tx=0).stop 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 shutdown_connections 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2dc159820 0x7fa2dc0c3440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2dc06e480 0x7fa2dc157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa2c0066f30 0x7fa2c00693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 --2- 172.21.15.131:0/4235206813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2dc0aa480 0x7fa2dc0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 >> 172.21.15.131:0/4235206813 conn(0x7fa2dc06c290 msgr2=0x7fa2dc15aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:18.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 shutdown_connections 2023-12-26T21:37:18.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.042+0000 7fa2e2fff700 1 -- 172.21.15.131:0/4235206813 wait complete. 2023-12-26T21:37:18.060 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:37:18.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.252+0000 7f8dafdff700 1 Processor -- start 2023-12-26T21:37:18.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.252+0000 7f8dafdff700 1 -- start start 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.253+0000 7f8dafdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8da81596a0 0x7f8da80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.253+0000 7f8dafdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8da8159a70 0x7f8da80b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.253+0000 7f8dafdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da80b0160 0x7f8da80b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.253+0000 7f8dafdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da80afcd0 con 0x7f8da81596a0 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.253+0000 7f8dafdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da80b3e70 con 0x7f8da8159a70 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.253+0000 7f8dafdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80b2e00 con 0x7f8da80b0160 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dadb9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8da81596a0 0x7f8da80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dadb9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8da81596a0 0x7f8da80b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55432/0 (socket says 172.21.15.131:55432) 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dadb9b700 1 -- 172.21.15.131:0/2797664701 learned_addr learned my addr 172.21.15.131:0/2797664701 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dae39c700 1 --2- 172.21.15.131:0/2797664701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da80b0160 0x7f8da80b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dad39a700 1 --2- 172.21.15.131:0/2797664701 >> v2:172.21.15.190:3300/0 conn(0x7f8da8159a70 0x7f8da80b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dadb9b700 1 -- 172.21.15.131:0/2797664701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da80b0160 msgr2=0x7f8da80b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dadb9b700 1 --2- 172.21.15.131:0/2797664701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da80b0160 0x7f8da80b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dadb9b700 1 -- 172.21.15.131:0/2797664701 >> v2:172.21.15.190:3300/0 conn(0x7f8da8159a70 msgr2=0x7f8da80b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dadb9b700 1 --2- 172.21.15.131:0/2797664701 >> v2:172.21.15.190:3300/0 conn(0x7f8da8159a70 0x7f8da80b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dadb9b700 1 -- 172.21.15.131:0/2797664701 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80b0f20 con 0x7f8da81596a0 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dad39a700 1 --2- 172.21.15.131:0/2797664701 >> v2:172.21.15.190:3300/0 conn(0x7f8da8159a70 0x7f8da80b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.254+0000 7f8dadb9b700 1 --2- 172.21.15.131:0/2797664701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 0x7f8da80b2380 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8da80642f0 tx=0x7f8da4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdffbe32ca44ea6b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dacb99700 1 -- 172.21.15.131:0/2797664701 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400b860 con 0x7f8da81596a0 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dacb99700 1 -- 172.21.15.131:0/2797664701 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da400b9c0 con 0x7f8da81596a0 2023-12-26T21:37:18.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dacb99700 1 -- 172.21.15.131:0/2797664701 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400bd60 con 0x7f8da81596a0 2023-12-26T21:37:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dafdff700 1 -- 172.21.15.131:0/2797664701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 msgr2=0x7f8da80b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2797664701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 0x7f8da80b2380 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8da80642f0 tx=0x7f8da4009250 comp rx=0 tx=0).stop 2023-12-26T21:37:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dafdff700 1 -- 172.21.15.131:0/2797664701 shutdown_connections 2023-12-26T21:37:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2797664701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 0x7f8da80b2380 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2797664701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da80b0160 0x7f8da80b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2797664701 >> v2:172.21.15.190:3300/0 conn(0x7f8da8159a70 0x7f8da80b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dafdff700 1 -- 172.21.15.131:0/2797664701 >> 172.21.15.131:0/2797664701 conn(0x7f8da806ece0 msgr2=0x7f8da806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dafdff700 1 -- 172.21.15.131:0/2797664701 shutdown_connections 2023-12-26T21:37:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.255+0000 7f8dafdff700 1 -- 172.21.15.131:0/2797664701 wait complete. 2023-12-26T21:37:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.257+0000 7f8dafdff700 1 Processor -- start 2023-12-26T21:37:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.257+0000 7f8dafdff700 1 -- start start 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dafdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da80b0160 0x7f8da80c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dafdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 0x7f8da8148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dafdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da8159a70 0x7f8da8158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dafdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8148fc0 con 0x7f8da81596a0 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dafdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80c5ed0 con 0x7f8da80b0160 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dafdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80c4fe0 con 0x7f8da8159a70 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dad39a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 0x7f8da8148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dad39a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 0x7f8da8148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55438/0 (socket says 172.21.15.131:55438) 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dad39a700 1 -- 172.21.15.131:0/2607693244 learned_addr learned my addr 172.21.15.131:0/2607693244 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dae39c700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da8159a70 0x7f8da8158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.258+0000 7f8dadb9b700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da80b0160 0x7f8da80c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8dad39a700 1 -- 172.21.15.131:0/2607693244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da8159a70 msgr2=0x7f8da8158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8dad39a700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da8159a70 0x7f8da8158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8dad39a700 1 -- 172.21.15.131:0/2607693244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da80b0160 msgr2=0x7f8da80c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8dad39a700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da80b0160 0x7f8da80c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8dad39a700 1 -- 172.21.15.131:0/2607693244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da40094f0 con 0x7f8da81596a0 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8dad39a700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 0x7f8da8148a80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800d890 tx=0x7f8d98008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8d9dffb700 1 -- 172.21.15.131:0/2607693244 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d980125f0 con 0x7f8da81596a0 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8d9dffb700 1 -- 172.21.15.131:0/2607693244 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d98012750 con 0x7f8da81596a0 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8d9dffb700 1 -- 172.21.15.131:0/2607693244 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98012af0 con 0x7f8da81596a0 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da80c52c0 con 0x7f8da81596a0 2023-12-26T21:37:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.259+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da80c5880 con 0x7f8da81596a0 2023-12-26T21:37:18.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.260+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8159e60 con 0x7f8da81596a0 2023-12-26T21:37:18.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.261+0000 7f8d9dffb700 1 -- 172.21.15.131:0/2607693244 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8d98014430 con 0x7f8da81596a0 2023-12-26T21:37:18.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.262+0000 7f8d9dffb700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8d8c066f30 0x7f8d8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.262+0000 7f8d9dffb700 1 -- 172.21.15.131:0/2607693244 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d980847e0 con 0x7f8da81596a0 2023-12-26T21:37:18.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.262+0000 7f8dadb9b700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8d8c066f30 0x7f8d8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.263+0000 7f8dadb9b700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8d8c066f30 0x7f8d8c0693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8da80642f0 tx=0x7f8da4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:18.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.266+0000 7f8d9dffb700 1 -- 172.21.15.131:0/2607693244 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d98051a50 con 0x7f8da81596a0 2023-12-26T21:37:18.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.457+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8da80c6010 con 0x7f8da81596a0 2023-12-26T21:37:18.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.457+0000 7f8d9dffb700 1 -- 172.21.15.131:0/2607693244 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8da80c6010 con 0x7f8da81596a0 2023-12-26T21:37:18.458 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:37:18.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8d8c066f30 msgr2=0x7f8d8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8d8c066f30 0x7f8d8c0693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8da80642f0 tx=0x7f8da4017000 comp rx=0 tx=0).stop 2023-12-26T21:37:18.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 msgr2=0x7f8da8148a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 0x7f8da8148a80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800d890 tx=0x7f8d98008040 comp rx=0 tx=0).stop 2023-12-26T21:37:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 shutdown_connections 2023-12-26T21:37:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8d8c066f30 0x7f8d8c0693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8da8159a70 0x7f8da8158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da80b0160 0x7f8da80c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 --2- 172.21.15.131:0/2607693244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da81596a0 0x7f8da8148a80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 >> 172.21.15.131:0/2607693244 conn(0x7f8da806ece0 msgr2=0x7f8da80cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 shutdown_connections 2023-12-26T21:37:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.460+0000 7f8dafdff700 1 -- 172.21.15.131:0/2607693244 wait complete. 2023-12-26T21:37:18.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:37:18.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.670+0000 7fdcf25ff700 1 Processor -- start 2023-12-26T21:37:18.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.670+0000 7fdcf25ff700 1 -- start start 2023-12-26T21:37:18.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.672+0000 7fdcf25ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec0ced90 0x7fdcec0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.672+0000 7fdcf25ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec0cf730 con 0x7fdcec159cf0 2023-12-26T21:37:18.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.672+0000 7fdcf25ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec0c6810 con 0x7fdcec159920 2023-12-26T21:37:18.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.672+0000 7fdcf25ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0c5c00 con 0x7fdcec0ced90 2023-12-26T21:37:18.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.672+0000 7fdce3fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdcec159cf0 0x7fdcec0c6ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35382/0 (socket says 172.21.15.131:35382) 2023-12-26T21:37:18.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.672+0000 7fdce3fff700 1 -- 172.21.15.131:0/536282330 learned_addr learned my addr 172.21.15.131:0/536282330 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:18.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.672+0000 7fdcf0b9c700 1 --2- 172.21.15.131:0/536282330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec0ced90 0x7fdcec0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdceb7fe700 1 -- 172.21.15.131:0/536282330 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858926913 0 0) 0x7fdcd4001110 con 0x7fdcec159cf0 2023-12-26T21:37:18.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdceb7fe700 1 -- 172.21.15.131:0/536282330 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd8003570 con 0x7fdcec159cf0 2023-12-26T21:37:18.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdceb7fe700 1 -- 172.21.15.131:0/536282330 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479290166 0 0) 0x7fdcec0c6810 con 0x7fdcec159920 2023-12-26T21:37:18.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdceb7fe700 1 -- 172.21.15.131:0/536282330 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd4001110 con 0x7fdcec159920 2023-12-26T21:37:18.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdcf0b9c700 1 -- 172.21.15.131:0/536282330 >> v1:172.21.15.190:6789/0 conn(0x7fdcec159920 legacy=0x7fdcec0c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdcf0b9c700 1 -- 172.21.15.131:0/536282330 >> v1:172.21.15.131:6789/0 conn(0x7fdcec159cf0 legacy=0x7fdcec0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdcf0b9c700 1 -- 172.21.15.131:0/536282330 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec0c5ee0 con 0x7fdcec0ced90 2023-12-26T21:37:18.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdcf0b9c700 1 --2- 172.21.15.131:0/536282330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec0ced90 0x7fdcec0cf1f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009e90 tx=0x7fdce4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae2233e9d67dada server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:18.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdceb7fe700 1 -- 172.21.15.131:0/536282330 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400e070 con 0x7fdcec0ced90 2023-12-26T21:37:18.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdceb7fe700 1 -- 172.21.15.131:0/536282330 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce400bb80 con 0x7fdcec0ced90 2023-12-26T21:37:18.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.673+0000 7fdceb7fe700 1 -- 172.21.15.131:0/536282330 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4012430 con 0x7fdcec0ced90 2023-12-26T21:37:18.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.674+0000 7fdcf25ff700 1 -- 172.21.15.131:0/536282330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec0ced90 msgr2=0x7fdcec0cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.674+0000 7fdcf25ff700 1 --2- 172.21.15.131:0/536282330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec0ced90 0x7fdcec0cf1f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009e90 tx=0x7fdce4009780 comp rx=0 tx=0).stop 2023-12-26T21:37:18.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.674+0000 7fdcf25ff700 1 -- 172.21.15.131:0/536282330 shutdown_connections 2023-12-26T21:37:18.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.674+0000 7fdcf25ff700 1 --2- 172.21.15.131:0/536282330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec0ced90 0x7fdcec0cf1f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.674+0000 7fdcf25ff700 1 -- 172.21.15.131:0/536282330 >> 172.21.15.131:0/536282330 conn(0x7fdcec099980 msgr2=0x7fdcec06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:18.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.674+0000 7fdcf25ff700 1 -- 172.21.15.131:0/536282330 shutdown_connections 2023-12-26T21:37:18.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.674+0000 7fdcf25ff700 1 -- 172.21.15.131:0/536282330 wait complete. 2023-12-26T21:37:18.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.675+0000 7fdcf25ff700 1 Processor -- start 2023-12-26T21:37:18.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.676+0000 7fdcf25ff700 1 -- start start 2023-12-26T21:37:18.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.676+0000 7fdcf25ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcec0ced90 0x7fdcec15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.676+0000 7fdcf25ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcec159920 0x7fdcec068d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.676+0000 7fdcf25ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec159cf0 0x7fdcec0692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.676+0000 7fdcf25ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec15f740 con 0x7fdcec159920 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.676+0000 7fdcf25ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec15fa20 con 0x7fdcec0ced90 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.676+0000 7fdcf25ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec16ae00 con 0x7fdcec159cf0 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdce3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcec159920 0x7fdcec068d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdce3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcec159920 0x7fdcec068d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55452/0 (socket says 172.21.15.131:55452) 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdce3fff700 1 -- 172.21.15.131:0/362234594 learned_addr learned my addr 172.21.15.131:0/362234594 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdcebfff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcec0ced90 0x7fdcec15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdcf0b9c700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec159cf0 0x7fdcec0692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdce3fff700 1 -- 172.21.15.131:0/362234594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec159cf0 msgr2=0x7fdcec0692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdce3fff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec159cf0 0x7fdcec0692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdce3fff700 1 -- 172.21.15.131:0/362234594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcec0ced90 msgr2=0x7fdcec15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdce3fff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcec0ced90 0x7fdcec15ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdce3fff700 1 -- 172.21.15.131:0/362234594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce4009a20 con 0x7fdcec159920 2023-12-26T21:37:18.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.677+0000 7fdce3fff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcec159920 0x7fdcec068d60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4001ed0 tx=0x7fdcd4007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:18.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.678+0000 7fdce8ff9700 1 -- 172.21.15.131:0/362234594 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4005b30 con 0x7fdcec159920 2023-12-26T21:37:18.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.678+0000 7fdce8ff9700 1 -- 172.21.15.131:0/362234594 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd4005cd0 con 0x7fdcec159920 2023-12-26T21:37:18.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.678+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec16b0e0 con 0x7fdcec159920 2023-12-26T21:37:18.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.678+0000 7fdce8ff9700 1 -- 172.21.15.131:0/362234594 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4007320 con 0x7fdcec159920 2023-12-26T21:37:18.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.678+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec15f1c0 con 0x7fdcec159920 2023-12-26T21:37:18.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.679+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec0657e0 con 0x7fdcec159920 2023-12-26T21:37:18.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.680+0000 7fdce8ff9700 1 -- 172.21.15.131:0/362234594 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdcd4012400 con 0x7fdcec159920 2023-12-26T21:37:18.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.680+0000 7fdce8ff9700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdccc066f30 0x7fdccc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:18.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.681+0000 7fdce8ff9700 1 -- 172.21.15.131:0/362234594 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdcd40838f0 con 0x7fdcec159920 2023-12-26T21:37:18.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.681+0000 7fdcebfff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdccc066f30 0x7fdccc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:18.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.681+0000 7fdcebfff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdccc066f30 0x7fdccc0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0698f0 tx=0x7fdcdc007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:18.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.685+0000 7fdce8ff9700 1 -- 172.21.15.131:0/362234594 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcd404b5a0 con 0x7fdcec159920 2023-12-26T21:37:18.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.877+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcec0cf730 con 0x7fdcec159920 2023-12-26T21:37:18.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.877+0000 7fdce8ff9700 1 -- 172.21.15.131:0/362234594 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcec0cf730 con 0x7fdcec159920 2023-12-26T21:37:18.878 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:37:18.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdccc066f30 msgr2=0x7fdccc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdccc066f30 0x7fdccc0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0698f0 tx=0x7fdcdc007d30 comp rx=0 tx=0).stop 2023-12-26T21:37:18.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcec159920 msgr2=0x7fdcec068d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:18.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcec159920 0x7fdcec068d60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4001ed0 tx=0x7fdcd4007e80 comp rx=0 tx=0).stop 2023-12-26T21:37:18.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 shutdown_connections 2023-12-26T21:37:18.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdccc066f30 0x7fdccc0693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcec159cf0 0x7fdcec0692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcec0ced90 0x7fdcec15ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 --2- 172.21.15.131:0/362234594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcec159920 0x7fdcec068d60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:18.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 >> 172.21.15.131:0/362234594 conn(0x7fdcec099980 msgr2=0x7fdcec0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:18.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 shutdown_connections 2023-12-26T21:37:18.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:18.880+0000 7fdcf25ff700 1 -- 172.21.15.131:0/362234594 wait complete. 2023-12-26T21:37:18.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:19.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.091+0000 7fca2b1ff700 1 Processor -- start 2023-12-26T21:37:19.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.091+0000 7fca2b1ff700 1 -- start start 2023-12-26T21:37:19.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.092+0000 7fca2b1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca24159a70 0x7fca240b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.092+0000 7fca2b1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fca240b2a90 0x7fca240b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.092+0000 7fca2b1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca240afcd0 con 0x7fca241596a0 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.092+0000 7fca2b1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca240b3e70 con 0x7fca240b2a90 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.092+0000 7fca2b1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca240b0f20 con 0x7fca24159a70 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.092+0000 7fca2979c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fca240b2a90 0x7fca240b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.092+0000 7fca2979c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fca240b2a90 0x7fca240b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55466/0 (socket says 172.21.15.131:55466) 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.092+0000 7fca2979c700 1 -- 172.21.15.131:0/2022484702 learned_addr learned my addr 172.21.15.131:0/2022484702 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.092+0000 7fca23fff700 1 --2- 172.21.15.131:0/2022484702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca24159a70 0x7fca240b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.093+0000 7fca2979c700 1 -- 172.21.15.131:0/2022484702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca24159a70 msgr2=0x7fca240b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.093+0000 7fca2979c700 1 --2- 172.21.15.131:0/2022484702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca24159a70 0x7fca240b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.093+0000 7fca2979c700 1 -- 172.21.15.131:0/2022484702 >> v1:172.21.15.190:6789/0 conn(0x7fca241596a0 legacy=0x7fca240b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.093+0000 7fca2979c700 1 -- 172.21.15.131:0/2022484702 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca240b3540 con 0x7fca240b2a90 2023-12-26T21:37:19.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.093+0000 7fca2979c700 1 --2- 172.21.15.131:0/2022484702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca240b2a90 0x7fca240b06a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009f20 tx=0x7fca1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a822a9a2597795 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:19.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.093+0000 7fca237fe700 1 -- 172.21.15.131:0/2022484702 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c00e070 con 0x7fca240b2a90 2023-12-26T21:37:19.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.093+0000 7fca237fe700 1 -- 172.21.15.131:0/2022484702 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c00b9d0 con 0x7fca240b2a90 2023-12-26T21:37:19.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.093+0000 7fca237fe700 1 -- 172.21.15.131:0/2022484702 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0092e0 con 0x7fca240b2a90 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.094+0000 7fca2b1ff700 1 -- 172.21.15.131:0/2022484702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca240b2a90 msgr2=0x7fca240b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.094+0000 7fca2b1ff700 1 --2- 172.21.15.131:0/2022484702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca240b2a90 0x7fca240b06a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009f20 tx=0x7fca1c009840 comp rx=0 tx=0).stop 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.094+0000 7fca2b1ff700 1 -- 172.21.15.131:0/2022484702 shutdown_connections 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.094+0000 7fca2b1ff700 1 --2- 172.21.15.131:0/2022484702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca240b2a90 0x7fca240b06a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.094+0000 7fca2b1ff700 1 --2- 172.21.15.131:0/2022484702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca24159a70 0x7fca240b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.094+0000 7fca2b1ff700 1 -- 172.21.15.131:0/2022484702 >> 172.21.15.131:0/2022484702 conn(0x7fca2406ece0 msgr2=0x7fca2406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.094+0000 7fca2b1ff700 1 -- 172.21.15.131:0/2022484702 shutdown_connections 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.094+0000 7fca2b1ff700 1 -- 172.21.15.131:0/2022484702 wait complete. 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.095+0000 7fca2b1ff700 1 Processor -- start 2023-12-26T21:37:19.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.096+0000 7fca2b1ff700 1 -- start start 2023-12-26T21:37:19.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.096+0000 7fca2b1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca240b2a90 0x7fca2415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.096+0000 7fca2b1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca241596a0 0x7fca2415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.096+0000 7fca2b1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca24159a70 0x7fca240b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.096+0000 7fca2b1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2415a160 con 0x7fca24159a70 2023-12-26T21:37:19.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.096+0000 7fca2b1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca240c7d70 con 0x7fca241596a0 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.096+0000 7fca2b1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2415ce50 con 0x7fca240b2a90 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca2979c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca24159a70 0x7fca240b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca2979c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca24159a70 0x7fca240b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55478/0 (socket says 172.21.15.131:55478) 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca2979c700 1 -- 172.21.15.131:0/862012302 learned_addr learned my addr 172.21.15.131:0/862012302 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca28f9b700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca240b2a90 0x7fca2415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca23fff700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca241596a0 0x7fca2415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca2979c700 1 -- 172.21.15.131:0/862012302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca240b2a90 msgr2=0x7fca2415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca2979c700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca240b2a90 0x7fca2415c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca2979c700 1 -- 172.21.15.131:0/862012302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca241596a0 msgr2=0x7fca2415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca2979c700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca241596a0 0x7fca2415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca2979c700 1 -- 172.21.15.131:0/862012302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c009ae0 con 0x7fca24159a70 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.097+0000 7fca2979c700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca24159a70 0x7fca240b50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fca24089710 tx=0x7fca1c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.098+0000 7fca20ff9700 1 -- 172.21.15.131:0/862012302 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c008b10 con 0x7fca24159a70 2023-12-26T21:37:19.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.098+0000 7fca20ff9700 1 -- 172.21.15.131:0/862012302 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c008cb0 con 0x7fca24159a70 2023-12-26T21:37:19.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.098+0000 7fca20ff9700 1 -- 172.21.15.131:0/862012302 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0266b0 con 0x7fca24159a70 2023-12-26T21:37:19.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.098+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2416d0f0 con 0x7fca24159a70 2023-12-26T21:37:19.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.098+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2416d6b0 con 0x7fca24159a70 2023-12-26T21:37:19.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.099+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2406f920 con 0x7fca24159a70 2023-12-26T21:37:19.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.100+0000 7fca20ff9700 1 -- 172.21.15.131:0/862012302 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fca1c003040 con 0x7fca24159a70 2023-12-26T21:37:19.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.101+0000 7fca20ff9700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca08066f30 0x7fca080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.101+0000 7fca20ff9700 1 -- 172.21.15.131:0/862012302 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca1c086110 con 0x7fca24159a70 2023-12-26T21:37:19.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.101+0000 7fca28f9b700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca08066f30 0x7fca080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.101+0000 7fca28f9b700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca08066f30 0x7fca080693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca18003a80 tx=0x7fca18007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:19.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.105+0000 7fca20ff9700 1 -- 172.21.15.131:0/862012302 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca1c053380 con 0x7fca24159a70 2023-12-26T21:37:19.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.300+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca240657e0 con 0x7fca24159a70 2023-12-26T21:37:19.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.301+0000 7fca20ff9700 1 -- 172.21.15.131:0/862012302 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fca1c0569a0 con 0x7fca24159a70 2023-12-26T21:37:19.302 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:19.302 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca08066f30 msgr2=0x7fca080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca08066f30 0x7fca080693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca18003a80 tx=0x7fca18007ec0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca24159a70 msgr2=0x7fca240b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca24159a70 0x7fca240b50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fca24089710 tx=0x7fca1c018000 comp rx=0 tx=0).stop 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 shutdown_connections 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca24159a70 0x7fca240b50e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca08066f30 0x7fca080693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca240b2a90 0x7fca2415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 --2- 172.21.15.131:0/862012302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca241596a0 0x7fca2415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.303+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 >> 172.21.15.131:0/862012302 conn(0x7fca2406ece0 msgr2=0x7fca24151af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.304+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 shutdown_connections 2023-12-26T21:37:19.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.304+0000 7fca2b1ff700 1 -- 172.21.15.131:0/862012302 wait complete. 2023-12-26T21:37:19.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:36:46.654972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:36:47.264880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:37:19.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:37:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.512+0000 7faf487ff700 1 Processor -- start 2023-12-26T21:37:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.512+0000 7faf487ff700 1 -- start start 2023-12-26T21:37:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf487ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faf400cc810 0x7faf400c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf487ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf400686c0 con 0x7faf4013c510 2023-12-26T21:37:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf487ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf40069520 con 0x7faf4006bfd0 2023-12-26T21:37:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf487ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf40068910 con 0x7faf400cc810 2023-12-26T21:37:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf4659b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7faf4013c510 0x7faf4006b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35398/0 (socket says 172.21.15.131:35398) 2023-12-26T21:37:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf46d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faf400cc810 0x7faf400c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf4659b700 1 -- 172.21.15.131:0/1467731679 learned_addr learned my addr 172.21.15.131:0/1467731679 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:19.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf45599700 1 -- 172.21.15.131:0/1467731679 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073980042 0 0) 0x7faf400686c0 con 0x7faf4013c510 2023-12-26T21:37:19.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf45599700 1 -- 172.21.15.131:0/1467731679 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf2c003570 con 0x7faf4013c510 2023-12-26T21:37:19.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf46d9c700 1 -- 172.21.15.131:0/1467731679 >> v1:172.21.15.190:6789/0 conn(0x7faf4006bfd0 legacy=0x7faf400cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf46d9c700 1 -- 172.21.15.131:0/1467731679 >> v1:172.21.15.131:6789/0 conn(0x7faf4013c510 legacy=0x7faf4006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.514+0000 7faf46d9c700 1 -- 172.21.15.131:0/1467731679 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf40068bf0 con 0x7faf400cc810 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.515+0000 7faf46d9c700 1 --2- 172.21.15.131:0/1467731679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf400cc810 0x7faf400c8a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf38009e60 tx=0x7faf38009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c78af9f1cddc59 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.515+0000 7faf45599700 1 -- 172.21.15.131:0/1467731679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3800e070 con 0x7faf400cc810 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.515+0000 7faf45599700 1 -- 172.21.15.131:0/1467731679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3800bb60 con 0x7faf400cc810 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.515+0000 7faf45599700 1 -- 172.21.15.131:0/1467731679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38012430 con 0x7faf400cc810 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.515+0000 7faf487ff700 1 -- 172.21.15.131:0/1467731679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf400cc810 msgr2=0x7faf400c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.515+0000 7faf487ff700 1 --2- 172.21.15.131:0/1467731679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf400cc810 0x7faf400c8a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf38009e60 tx=0x7faf38009780 comp rx=0 tx=0).stop 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.515+0000 7faf487ff700 1 -- 172.21.15.131:0/1467731679 shutdown_connections 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.516+0000 7faf487ff700 1 --2- 172.21.15.131:0/1467731679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf400cc810 0x7faf400c8a20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.516+0000 7faf487ff700 1 -- 172.21.15.131:0/1467731679 >> 172.21.15.131:0/1467731679 conn(0x7faf400ce3e0 msgr2=0x7faf400ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.516+0000 7faf487ff700 1 -- 172.21.15.131:0/1467731679 shutdown_connections 2023-12-26T21:37:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.516+0000 7faf487ff700 1 -- 172.21.15.131:0/1467731679 wait complete. 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.517+0000 7faf487ff700 1 Processor -- start 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.517+0000 7faf487ff700 1 -- start start 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf487ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4006bfd0 0x7faf40153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf487ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400cc810 0x7faf401588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf487ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4013c510 0x7faf40158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf487ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4013ac70 con 0x7faf4006bfd0 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf487ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf400b3380 con 0x7faf400cc810 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf487ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf400b2770 con 0x7faf4013c510 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf4659b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4006bfd0 0x7faf40153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf4659b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4006bfd0 0x7faf40153850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55486/0 (socket says 172.21.15.131:55486) 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf4659b700 1 -- 172.21.15.131:0/3067949853 learned_addr learned my addr 172.21.15.131:0/3067949853 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf45d9a700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400cc810 0x7faf401588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.518+0000 7faf46d9c700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4013c510 0x7faf40158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf4659b700 1 -- 172.21.15.131:0/3067949853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4013c510 msgr2=0x7faf40158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf4659b700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4013c510 0x7faf40158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf4659b700 1 -- 172.21.15.131:0/3067949853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400cc810 msgr2=0x7faf401588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf4659b700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400cc810 0x7faf401588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf4659b700 1 -- 172.21.15.131:0/3067949853 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf38009a20 con 0x7faf4006bfd0 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf4659b700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4006bfd0 0x7faf40153850 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00c790 tx=0x7faf3c007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf367fc700 1 -- 172.21.15.131:0/3067949853 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0072e0 con 0x7faf4006bfd0 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf367fc700 1 -- 172.21.15.131:0/3067949853 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c007440 con 0x7faf4006bfd0 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf400b2a50 con 0x7faf4006bfd0 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf367fc700 1 -- 172.21.15.131:0/3067949853 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0057f0 con 0x7faf4006bfd0 2023-12-26T21:37:19.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.519+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf400b3010 con 0x7faf4006bfd0 2023-12-26T21:37:19.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.520+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4015a5c0 con 0x7faf4006bfd0 2023-12-26T21:37:19.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.521+0000 7faf367fc700 1 -- 172.21.15.131:0/3067949853 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faf3c005c20 con 0x7faf4006bfd0 2023-12-26T21:37:19.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.522+0000 7faf367fc700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf24066f30 0x7faf240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.523+0000 7faf367fc700 1 -- 172.21.15.131:0/3067949853 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faf3c083a70 con 0x7faf4006bfd0 2023-12-26T21:37:19.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.523+0000 7faf45d9a700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf24066f30 0x7faf240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.523+0000 7faf45d9a700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf24066f30 0x7faf240693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faf40138970 tx=0x7faf30007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:19.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.527+0000 7faf367fc700 1 -- 172.21.15.131:0/3067949853 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf3c04b5a0 con 0x7faf4006bfd0 2023-12-26T21:37:19.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.716+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf400657e0 con 0x7faf4006bfd0 2023-12-26T21:37:19.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.717+0000 7faf367fc700 1 -- 172.21.15.131:0/3067949853 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf3c017070 con 0x7faf4006bfd0 2023-12-26T21:37:19.718 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf24066f30 msgr2=0x7faf240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf24066f30 0x7faf240693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faf40138970 tx=0x7faf30007ec0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4006bfd0 msgr2=0x7faf40153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4006bfd0 0x7faf40153850 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00c790 tx=0x7faf3c007c80 comp rx=0 tx=0).stop 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 shutdown_connections 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7faf24066f30 0x7faf240693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4013c510 0x7faf40158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4006bfd0 0x7faf40153850 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 --2- 172.21.15.131:0/3067949853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400cc810 0x7faf401588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 >> 172.21.15.131:0/3067949853 conn(0x7faf400ce3e0 msgr2=0x7faf400ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 shutdown_connections 2023-12-26T21:37:19.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.719+0000 7faf487ff700 1 -- 172.21.15.131:0/3067949853 wait complete. 2023-12-26T21:37:19.737 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:37:19.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.930+0000 7f1e12bff700 1 Processor -- start 2023-12-26T21:37:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.930+0000 7f1e12bff700 1 -- start start 2023-12-26T21:37:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.931+0000 7f1e12bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1e0c0b0bf0 0x7f1e0c0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.931+0000 7f1e12bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c156940 con 0x7f1e0c0b0750 2023-12-26T21:37:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.931+0000 7f1e12bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0ae500 con 0x7f1e0c0cb320 2023-12-26T21:37:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.931+0000 7f1e12bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0c0c8cc0 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.932+0000 7f1e1119c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1e0c0b0bf0 0x7f1e0c0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.932+0000 7f1e1119c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1e0c0b0bf0 0x7f1e0c0c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55502/0 (socket says 172.21.15.131:55502) 2023-12-26T21:37:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.932+0000 7f1e0bfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1e0c0b0750 0x7f1e0c0c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35400/0 (socket says 172.21.15.131:35400) 2023-12-26T21:37:19.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.932+0000 7f1e1119c700 1 -- 172.21.15.131:0/3555091049 learned_addr learned my addr 172.21.15.131:0/3555091049 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.932+0000 7f1e1119c700 1 -- 172.21.15.131:0/3555091049 >> v1:172.21.15.190:6789/0 conn(0x7f1e0c0cb320 legacy=0x7f1e0c069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.932+0000 7f1e1119c700 1 -- 172.21.15.131:0/3555091049 >> v1:172.21.15.131:6789/0 conn(0x7f1e0c0b0750 legacy=0x7f1e0c0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.932+0000 7f1e1119c700 1 -- 172.21.15.131:0/3555091049 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0b0280 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.932+0000 7f1e0b7fe700 1 -- 172.21.15.131:0/3555091049 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973509462 0 0) 0x7f1e0c156940 con 0x7f1e0c0b0750 2023-12-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.932+0000 7f1e1119c700 1 --2- 172.21.15.131:0/3555091049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 0x7f1e0c0c9b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009f20 tx=0x7f1e04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d04094cf3ef83dc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e0b7fe700 1 -- 172.21.15.131:0/3555091049 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400e070 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e0b7fe700 1 -- 172.21.15.131:0/3555091049 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400ba80 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e0b7fe700 1 -- 172.21.15.131:0/3555091049 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040092e0 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e12bff700 1 -- 172.21.15.131:0/3555091049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 msgr2=0x7f1e0c0c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e12bff700 1 --2- 172.21.15.131:0/3555091049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 0x7f1e0c0c9b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009f20 tx=0x7f1e04009840 comp rx=0 tx=0).stop 2023-12-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e12bff700 1 -- 172.21.15.131:0/3555091049 shutdown_connections 2023-12-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e12bff700 1 --2- 172.21.15.131:0/3555091049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 0x7f1e0c0c9b10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e12bff700 1 -- 172.21.15.131:0/3555091049 >> 172.21.15.131:0/3555091049 conn(0x7f1e0c068040 msgr2=0x7f1e0c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e12bff700 1 -- 172.21.15.131:0/3555091049 shutdown_connections 2023-12-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.933+0000 7f1e12bff700 1 -- 172.21.15.131:0/3555091049 wait complete. 2023-12-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.935+0000 7f1e12bff700 1 Processor -- start 2023-12-26T21:37:19.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.935+0000 7f1e12bff700 1 -- start start 2023-12-26T21:37:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e12bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e0c0b0750 0x7f1e0c1693a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e12bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 0x7f1e0c159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e12bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c0cb320 0x7f1e0c159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e12bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0a9940 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e12bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c1699a0 con 0x7f1e0c0cb320 2023-12-26T21:37:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e12bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c169c80 con 0x7f1e0c0b0750 2023-12-26T21:37:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 0x7f1e0c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 0x7f1e0c159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55506/0 (socket says 172.21.15.131:55506) 2023-12-26T21:37:19.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e0bfff700 1 -- 172.21.15.131:0/1052057340 learned_addr learned my addr 172.21.15.131:0/1052057340 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:19.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e1119c700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c0cb320 0x7f1e0c159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.936+0000 7f1e1099b700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e0c0b0750 0x7f1e0c1693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e0bfff700 1 -- 172.21.15.131:0/1052057340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e0c0b0750 msgr2=0x7f1e0c1693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e0bfff700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e0c0b0750 0x7f1e0c1693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e0bfff700 1 -- 172.21.15.131:0/1052057340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c0cb320 msgr2=0x7f1e0c159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e0bfff700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c0cb320 0x7f1e0c159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e0bfff700 1 -- 172.21.15.131:0/1052057340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04009ae0 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e0bfff700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 0x7f1e0c159150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1df800c810 tx=0x7f1df8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e08ff9700 1 -- 172.21.15.131:0/1052057340 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8005b40 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e08ff9700 1 -- 172.21.15.131:0/1052057340 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df8005ca0 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c169f60 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e08ff9700 1 -- 172.21.15.131:0/1052057340 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8007320 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.937+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c16a590 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.938+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c0a9af0 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.939+0000 7f1e08ff9700 1 -- 172.21.15.131:0/1052057340 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1df801b3b0 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.940+0000 7f1e08ff9700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1df0066f30 0x7f1df00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:19.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.940+0000 7f1e08ff9700 1 -- 172.21.15.131:0/1052057340 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1df8083a20 con 0x7f1e0c0b0bf0 2023-12-26T21:37:19.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.940+0000 7f1e1099b700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1df0066f30 0x7f1df00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:19.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.941+0000 7f1e1099b700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1df0066f30 0x7f1df00693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e000039e0 tx=0x7f1e00007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:19.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:19.944+0000 7f1e08ff9700 1 -- 172.21.15.131:0/1052057340 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e0c0a9af0 con 0x7f1e0c0b0bf0 2023-12-26T21:37:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.138+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e0c0657e0 con 0x7f1e0c0b0bf0 2023-12-26T21:37:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.139+0000 7f1e08ff9700 1 -- 172.21.15.131:0/1052057340 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1df80170a0 con 0x7f1e0c0b0bf0 2023-12-26T21:37:20.140 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1df0066f30 msgr2=0x7f1df00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1df0066f30 0x7f1df00693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e000039e0 tx=0x7f1e00007ec0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 msgr2=0x7f1e0c159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 0x7f1e0c159150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1df800c810 tx=0x7f1df8007d90 comp rx=0 tx=0).stop 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 shutdown_connections 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e0c0cb320 0x7f1e0c159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1df0066f30 0x7f1df00693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e0c0b0750 0x7f1e0c1693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 --2- 172.21.15.131:0/1052057340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e0c0b0bf0 0x7f1e0c159150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 >> 172.21.15.131:0/1052057340 conn(0x7f1e0c068040 msgr2=0x7f1e0c069510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:20.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 shutdown_connections 2023-12-26T21:37:20.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.141+0000 7f1e12bff700 1 -- 172.21.15.131:0/1052057340 wait complete. 2023-12-26T21:37:20.160 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:37:20.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.351+0000 7f0d485ff700 1 Processor -- start 2023-12-26T21:37:20.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.351+0000 7f0d485ff700 1 -- start start 2023-12-26T21:37:20.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.352+0000 7f0d485ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d400c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d485ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d4006b8c0 0x7f0d4006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d485ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4006c270 con 0x7f0d400cc100 2023-12-26T21:37:20.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d485ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d400686c0 con 0x7f0d4006b8c0 2023-12-26T21:37:20.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d485ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40069520 con 0x7f0d4013c510 2023-12-26T21:37:20.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d4639b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d400c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d4639b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d400c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:49592/0 (socket says 172.21.15.131:49592) 2023-12-26T21:37:20.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d45b9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d4006b8c0 0x7f0d4006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d4639b700 1 -- 172.21.15.131:0/2176795653 learned_addr learned my addr 172.21.15.131:0/2176795653 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:20.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d46b9c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f0d400cc100 0x7f0d400cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:41612/0 (socket says 172.21.15.131:41612) 2023-12-26T21:37:20.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.353+0000 7f0d45399700 1 -- 172.21.15.131:0/2176795653 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516176530 0 0) 0x7f0d4006c270 con 0x7f0d400cc100 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d45399700 1 -- 172.21.15.131:0/2176795653 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d2c003570 con 0x7f0d400cc100 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d45b9a700 1 -- 172.21.15.131:0/2176795653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 msgr2=0x7f0d400c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d45b9a700 1 --2- 172.21.15.131:0/2176795653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d400c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d45b9a700 1 -- 172.21.15.131:0/2176795653 >> v1:172.21.15.190:6789/0 conn(0x7f0d400cc100 legacy=0x7f0d400cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d45b9a700 1 -- 172.21.15.131:0/2176795653 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d40068910 con 0x7f0d4006b8c0 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d4639b700 1 --2- 172.21.15.131:0/2176795653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d400c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d45b9a700 1 --2- 172.21.15.131:0/2176795653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4006b8c0 0x7f0d4006bd30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d300098e0 tx=0x7f0d300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=419889c28497a39b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d45399700 1 -- 172.21.15.131:0/2176795653 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30011630 con 0x7f0d4006b8c0 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d45399700 1 -- 172.21.15.131:0/2176795653 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d30011790 con 0x7f0d4006b8c0 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.354+0000 7f0d45399700 1 -- 172.21.15.131:0/2176795653 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30011b30 con 0x7f0d4006b8c0 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.355+0000 7f0d485ff700 1 -- 172.21.15.131:0/2176795653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4006b8c0 msgr2=0x7f0d4006bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.355+0000 7f0d485ff700 1 --2- 172.21.15.131:0/2176795653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4006b8c0 0x7f0d4006bd30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d300098e0 tx=0x7f0d300091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.355+0000 7f0d485ff700 1 -- 172.21.15.131:0/2176795653 shutdown_connections 2023-12-26T21:37:20.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.355+0000 7f0d485ff700 1 --2- 172.21.15.131:0/2176795653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d400c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.355+0000 7f0d485ff700 1 --2- 172.21.15.131:0/2176795653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4006b8c0 0x7f0d4006bd30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.355+0000 7f0d485ff700 1 -- 172.21.15.131:0/2176795653 >> 172.21.15.131:0/2176795653 conn(0x7f0d400ce3e0 msgr2=0x7f0d400ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:20.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.355+0000 7f0d485ff700 1 -- 172.21.15.131:0/2176795653 shutdown_connections 2023-12-26T21:37:20.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.355+0000 7f0d485ff700 1 -- 172.21.15.131:0/2176795653 wait complete. 2023-12-26T21:37:20.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.356+0000 7f0d485ff700 1 Processor -- start 2023-12-26T21:37:20.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.356+0000 7f0d485ff700 1 -- start start 2023-12-26T21:37:20.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.357+0000 7f0d485ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4006b8c0 0x7f0d40138c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.357+0000 7f0d485ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d400cc100 0x7f0d4015c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.357+0000 7f0d485ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d40156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.357+0000 7f0d485ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4015cc20 con 0x7f0d400cc100 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.357+0000 7f0d485ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40139240 con 0x7f0d4006b8c0 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.357+0000 7f0d485ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40139520 con 0x7f0d4013c510 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.357+0000 7f0d45b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d400cc100 0x7f0d4015c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d45b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d400cc100 0x7f0d4015c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55516/0 (socket says 172.21.15.131:55516) 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d45b9a700 1 -- 172.21.15.131:0/867009834 learned_addr learned my addr 172.21.15.131:0/867009834 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d46b9c700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d40156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d4639b700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4006b8c0 0x7f0d40138c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d45b9a700 1 -- 172.21.15.131:0/867009834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 msgr2=0x7f0d40156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d45b9a700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d40156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d45b9a700 1 -- 172.21.15.131:0/867009834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4006b8c0 msgr2=0x7f0d40138c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d45b9a700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4006b8c0 0x7f0d40138c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d45b9a700 1 -- 172.21.15.131:0/867009834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c009480 con 0x7f0d400cc100 2023-12-26T21:37:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.358+0000 7f0d45b9a700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d400cc100 0x7f0d4015c6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0d4006fe40 tx=0x7f0d30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.359+0000 7f0d367fc700 1 -- 172.21.15.131:0/867009834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30011900 con 0x7f0d400cc100 2023-12-26T21:37:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.359+0000 7f0d367fc700 1 -- 172.21.15.131:0/867009834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d30011a60 con 0x7f0d400cc100 2023-12-26T21:37:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.359+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d30009420 con 0x7f0d400cc100 2023-12-26T21:37:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.359+0000 7f0d367fc700 1 -- 172.21.15.131:0/867009834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30025640 con 0x7f0d400cc100 2023-12-26T21:37:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.359+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d40139ae0 con 0x7f0d400cc100 2023-12-26T21:37:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.360+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4013a130 con 0x7f0d400cc100 2023-12-26T21:37:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.361+0000 7f0d367fc700 1 -- 172.21.15.131:0/867009834 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0d30025a30 con 0x7f0d400cc100 2023-12-26T21:37:20.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.361+0000 7f0d367fc700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d24066f30 0x7f0d240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.361+0000 7f0d367fc700 1 -- 172.21.15.131:0/867009834 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d30085c50 con 0x7f0d400cc100 2023-12-26T21:37:20.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.362+0000 7f0d4639b700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d24066f30 0x7f0d240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.362+0000 7f0d4639b700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d24066f30 0x7f0d240693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009f30 tx=0x7f0d3c009220 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:20.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.366+0000 7f0d367fc700 1 -- 172.21.15.131:0/867009834 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d30052e20 con 0x7f0d400cc100 2023-12-26T21:37:20.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.555+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d400657e0 con 0x7f0d400cc100 2023-12-26T21:37:20.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.556+0000 7f0d367fc700 1 -- 172.21.15.131:0/867009834 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d30015020 con 0x7f0d400cc100 2023-12-26T21:37:20.556 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d24066f30 msgr2=0x7f0d240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d24066f30 0x7f0d240693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009f30 tx=0x7f0d3c009220 comp rx=0 tx=0).stop 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d400cc100 msgr2=0x7f0d4015c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d400cc100 0x7f0d4015c6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0d4006fe40 tx=0x7f0d30017000 comp rx=0 tx=0).stop 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 shutdown_connections 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d24066f30 0x7f0d240693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4013c510 0x7f0d40156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4006b8c0 0x7f0d40138c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 --2- 172.21.15.131:0/867009834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d400cc100 0x7f0d4015c6e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 >> 172.21.15.131:0/867009834 conn(0x7f0d400ce3e0 msgr2=0x7f0d400cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 shutdown_connections 2023-12-26T21:37:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.558+0000 7f0d485ff700 1 -- 172.21.15.131:0/867009834 wait complete. 2023-12-26T21:37:20.576 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:20.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.765+0000 7f02967ff700 1 Processor -- start 2023-12-26T21:37:20.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.765+0000 7f02967ff700 1 -- start start 2023-12-26T21:37:20.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.767+0000 7f02967ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f029013c510 0x7f02900c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.767+0000 7f02967ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 0x7f029006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.767+0000 7f02967ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02900686c0 con 0x7f029006b8c0 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.767+0000 7f02967ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029006c280 con 0x7f029013c510 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.767+0000 7f02967ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0290069520 con 0x7f02900cc810 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.767+0000 7f028ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f029013c510 0x7f02900c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.767+0000 7f028ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f029013c510 0x7f02900c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55530/0 (socket says 172.21.15.131:55530) 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.767+0000 7f028ffff700 1 -- 172.21.15.131:0/842513396 learned_addr learned my addr 172.21.15.131:0/842513396 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.767+0000 7f0294d9c700 1 --2- 172.21.15.131:0/842513396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 0x7f029006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.768+0000 7f028ffff700 1 -- 172.21.15.131:0/842513396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 msgr2=0x7f029006bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.768+0000 7f028ffff700 1 --2- 172.21.15.131:0/842513396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 0x7f029006bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.768+0000 7f028ffff700 1 -- 172.21.15.131:0/842513396 >> v1:172.21.15.190:6789/0 conn(0x7f029006b8c0 legacy=0x7f02900cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.768+0000 7f028ffff700 1 -- 172.21.15.131:0/842513396 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290068910 con 0x7f029013c510 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.768+0000 7f028ffff700 1 --2- 172.21.15.131:0/842513396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029013c510 0x7f02900c8a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0280009990 tx=0x7f0280009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465e8a2e380aca13 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:20.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.768+0000 7f028effd700 1 -- 172.21.15.131:0/842513396 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280005c10 con 0x7f029013c510 2023-12-26T21:37:20.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.768+0000 7f028effd700 1 -- 172.21.15.131:0/842513396 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280005d70 con 0x7f029013c510 2023-12-26T21:37:20.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.768+0000 7f028effd700 1 -- 172.21.15.131:0/842513396 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000b880 con 0x7f029013c510 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.769+0000 7f02967ff700 1 -- 172.21.15.131:0/842513396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029013c510 msgr2=0x7f02900c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.769+0000 7f02967ff700 1 --2- 172.21.15.131:0/842513396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029013c510 0x7f02900c8a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0280009990 tx=0x7f0280009220 comp rx=0 tx=0).stop 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.769+0000 7f02967ff700 1 -- 172.21.15.131:0/842513396 shutdown_connections 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.769+0000 7f02967ff700 1 --2- 172.21.15.131:0/842513396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 0x7f029006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.769+0000 7f02967ff700 1 --2- 172.21.15.131:0/842513396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029013c510 0x7f02900c8a20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.769+0000 7f02967ff700 1 -- 172.21.15.131:0/842513396 >> 172.21.15.131:0/842513396 conn(0x7f02900ce3e0 msgr2=0x7f02900ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.769+0000 7f02967ff700 1 -- 172.21.15.131:0/842513396 shutdown_connections 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.769+0000 7f02967ff700 1 -- 172.21.15.131:0/842513396 wait complete. 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.770+0000 7f02967ff700 1 Processor -- start 2023-12-26T21:37:20.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.770+0000 7f02967ff700 1 -- start start 2023-12-26T21:37:20.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.771+0000 7f02967ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029006b8c0 0x7f029015b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.771+0000 7f02967ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 0x7f029015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.771+0000 7f02967ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029013c510 0x7f02900c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.771+0000 7f02967ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029016aaa0 con 0x7f029006b8c0 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.771+0000 7f02967ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0290169bd0 con 0x7f029013c510 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.771+0000 7f02967ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0290169e90 con 0x7f02900cc810 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029006b8c0 0x7f029015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029006b8c0 0x7f029015b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55536/0 (socket says 172.21.15.131:55536) 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028ffff700 1 -- 172.21.15.131:0/2733878800 learned_addr learned my addr 172.21.15.131:0/2733878800 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f0294d9c700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029013c510 0x7f02900c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028f7fe700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 0x7f029015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028ffff700 1 -- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 msgr2=0x7f029015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028ffff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 0x7f029015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028ffff700 1 -- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029013c510 msgr2=0x7f02900c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028ffff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029013c510 0x7f02900c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028ffff700 1 -- 172.21.15.131:0/2733878800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280009480 con 0x7f029006b8c0 2023-12-26T21:37:20.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028f7fe700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 0x7f029015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f0294d9c700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029013c510 0x7f02900c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f028ffff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029006b8c0 0x7f029015b490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0280005d40 tx=0x7f0280016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.772+0000 7f0277fff700 1 -- 172.21.15.131:0/2733878800 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280011a60 con 0x7f029006b8c0 2023-12-26T21:37:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.773+0000 7f0277fff700 1 -- 172.21.15.131:0/2733878800 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280011bc0 con 0x7f029006b8c0 2023-12-26T21:37:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.773+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029016a170 con 0x7f029006b8c0 2023-12-26T21:37:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.773+0000 7f0277fff700 1 -- 172.21.15.131:0/2733878800 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028001b430 con 0x7f029006b8c0 2023-12-26T21:37:20.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.773+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029016a730 con 0x7f029006b8c0 2023-12-26T21:37:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.774+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0290150ac0 con 0x7f029006b8c0 2023-12-26T21:37:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.775+0000 7f0277fff700 1 -- 172.21.15.131:0/2733878800 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f028001b860 con 0x7f029006b8c0 2023-12-26T21:37:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.775+0000 7f0277fff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f027c066f30 0x7f027c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.776+0000 7f0277fff700 1 -- 172.21.15.131:0/2733878800 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0280085f90 con 0x7f029006b8c0 2023-12-26T21:37:20.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.776+0000 7f028f7fe700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f027c066f30 0x7f027c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:20.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.776+0000 7f028f7fe700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f027c066f30 0x7f027c0693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0284007fd0 tx=0x7f0284009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:20.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.780+0000 7f0277fff700 1 -- 172.21.15.131:0/2733878800 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02800b75a0 con 0x7f029006b8c0 2023-12-26T21:37:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.981+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02900657e0 con 0x7f029006b8c0 2023-12-26T21:37:20.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.982+0000 7f0277fff700 1 -- 172.21.15.131:0/2733878800 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f028002a090 con 0x7f029006b8c0 2023-12-26T21:37:20.982 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:20.982 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f027c066f30 msgr2=0x7f027c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f027c066f30 0x7f027c0693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0284007fd0 tx=0x7f0284009040 comp rx=0 tx=0).stop 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029006b8c0 msgr2=0x7f029015b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029006b8c0 0x7f029015b490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0280005d40 tx=0x7f0280016000 comp rx=0 tx=0).stop 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 shutdown_connections 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f029013c510 0x7f02900c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f029006b8c0 0x7f029015b490 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f027c066f30 0x7f027c0693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 --2- 172.21.15.131:0/2733878800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02900cc810 0x7f029015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 >> 172.21.15.131:0/2733878800 conn(0x7f02900ce3e0 msgr2=0x7f029013b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:20.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 shutdown_connections 2023-12-26T21:37:20.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:20.984+0000 7f02967ff700 1 -- 172.21.15.131:0/2733878800 wait complete. 2023-12-26T21:37:21.003 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:36:46.654972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:36:47.264880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:37:21.003 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:37:21.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.196+0000 7fe9db1ff700 1 Processor -- start 2023-12-26T21:37:21.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.196+0000 7fe9db1ff700 1 -- start start 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.197+0000 7fe9db1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe9d41596a0 0x7fe9d40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.197+0000 7fe9db1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9d4159a70 0x7fe9d40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.197+0000 7fe9db1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 0x7fe9d40b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9db1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d40afcd0 con 0x7fe9d4159a70 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9db1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d40b3e70 con 0x7fe9d41596a0 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9db1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40b2e00 con 0x7fe9d40b0160 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9d4159a70 0x7fe9d40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9d4159a70 0x7fe9d40b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55544/0 (socket says 172.21.15.131:55544) 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d3fff700 1 -- 172.21.15.131:0/3960108992 learned_addr learned my addr 172.21.15.131:0/3960108992 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d8f9b700 1 --2- 172.21.15.131:0/3960108992 >> v2:172.21.15.190:3300/0 conn(0x7fe9d41596a0 0x7fe9d40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d979c700 1 --2- 172.21.15.131:0/3960108992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 0x7fe9d40b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d3fff700 1 -- 172.21.15.131:0/3960108992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 msgr2=0x7fe9d40b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:21.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d3fff700 1 --2- 172.21.15.131:0/3960108992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 0x7fe9d40b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d3fff700 1 -- 172.21.15.131:0/3960108992 >> v2:172.21.15.190:3300/0 conn(0x7fe9d41596a0 msgr2=0x7fe9d40b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:21.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d3fff700 1 --2- 172.21.15.131:0/3960108992 >> v2:172.21.15.190:3300/0 conn(0x7fe9d41596a0 0x7fe9d40b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d3fff700 1 -- 172.21.15.131:0/3960108992 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d40b0f20 con 0x7fe9d4159a70 2023-12-26T21:37:21.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d8f9b700 1 --2- 172.21.15.131:0/3960108992 >> v2:172.21.15.190:3300/0 conn(0x7fe9d41596a0 0x7fe9d40b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:21.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.198+0000 7fe9d979c700 1 --2- 172.21.15.131:0/3960108992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 0x7fe9d40b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:21.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9d3fff700 1 --2- 172.21.15.131:0/3960108992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d4159a70 0x7fe9d40b28c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40892a0 tx=0x7fe9c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e898d1ab6f90260 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9d37fe700 1 -- 172.21.15.131:0/3960108992 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000b860 con 0x7fe9d4159a70 2023-12-26T21:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9d37fe700 1 -- 172.21.15.131:0/3960108992 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c000b9c0 con 0x7fe9d4159a70 2023-12-26T21:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9d37fe700 1 -- 172.21.15.131:0/3960108992 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000bd60 con 0x7fe9d4159a70 2023-12-26T21:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3960108992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d4159a70 msgr2=0x7fe9d40b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3960108992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d4159a70 0x7fe9d40b28c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40892a0 tx=0x7fe9c0009250 comp rx=0 tx=0).stop 2023-12-26T21:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3960108992 shutdown_connections 2023-12-26T21:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3960108992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 0x7fe9d40b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3960108992 >> v2:172.21.15.190:3300/0 conn(0x7fe9d41596a0 0x7fe9d40b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3960108992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d4159a70 0x7fe9d40b28c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3960108992 >> 172.21.15.131:0/3960108992 conn(0x7fe9d406ece0 msgr2=0x7fe9d406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:21.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.199+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3960108992 shutdown_connections 2023-12-26T21:37:21.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.200+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3960108992 wait complete. 2023-12-26T21:37:21.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.202+0000 7fe9db1ff700 1 Processor -- start 2023-12-26T21:37:21.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.202+0000 7fe9db1ff700 1 -- start start 2023-12-26T21:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9db1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 0x7fe9d414a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9db1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d41596a0 0x7fe9d415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9db1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d4159a70 0x7fe9d415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9db1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40c7780 con 0x7fe9d41596a0 2023-12-26T21:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9db1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40ca420 con 0x7fe9d4159a70 2023-12-26T21:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9db1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40c9b80 con 0x7fe9d40b0160 2023-12-26T21:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9d3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d41596a0 0x7fe9d415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9d3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d41596a0 0x7fe9d415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55548/0 (socket says 172.21.15.131:55548) 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9d3fff700 1 -- 172.21.15.131:0/3732161461 learned_addr learned my addr 172.21.15.131:0/3732161461 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.203+0000 7fe9d8f9b700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 0x7fe9d414a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d979c700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d4159a70 0x7fe9d415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d3fff700 1 -- 172.21.15.131:0/3732161461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 msgr2=0x7fe9d414a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d3fff700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 0x7fe9d414a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d3fff700 1 -- 172.21.15.131:0/3732161461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d4159a70 msgr2=0x7fe9d415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d3fff700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d4159a70 0x7fe9d415eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d3fff700 1 -- 172.21.15.131:0/3732161461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00094f0 con 0x7fe9d41596a0 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d3fff700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d41596a0 0x7fe9d415e580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00090b0 tx=0x7fe9c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d0ff9700 1 -- 172.21.15.131:0/3732161461 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0011e00 con 0x7fe9d41596a0 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d0ff9700 1 -- 172.21.15.131:0/3732161461 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c001f070 con 0x7fe9d41596a0 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9d0ff9700 1 -- 172.21.15.131:0/3732161461 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00266d0 con 0x7fe9d41596a0 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.204+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d40c9e60 con 0x7fe9d41596a0 2023-12-26T21:37:21.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.205+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d41698c0 con 0x7fe9d41596a0 2023-12-26T21:37:21.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.205+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d40c7930 con 0x7fe9d41596a0 2023-12-26T21:37:21.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.206+0000 7fe9d0ff9700 1 -- 172.21.15.131:0/3732161461 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe9c0026a40 con 0x7fe9d41596a0 2023-12-26T21:37:21.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.207+0000 7fe9d0ff9700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe9b8066f30 0x7fe9b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.208+0000 7fe9d0ff9700 1 -- 172.21.15.131:0/3732161461 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe9c0021070 con 0x7fe9d41596a0 2023-12-26T21:37:21.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.208+0000 7fe9d8f9b700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe9b8066f30 0x7fe9b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.208+0000 7fe9d8f9b700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe9b8066f30 0x7fe9b80693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:21.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.212+0000 7fe9d0ff9700 1 -- 172.21.15.131:0/3732161461 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9d40c7930 con 0x7fe9d41596a0 2023-12-26T21:37:21.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.407+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9d40c7930 con 0x7fe9d41596a0 2023-12-26T21:37:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.407+0000 7fe9d0ff9700 1 -- 172.21.15.131:0/3732161461 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9d40c7930 con 0x7fe9d41596a0 2023-12-26T21:37:21.408 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:37:21.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe9b8066f30 msgr2=0x7fe9b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe9b8066f30 0x7fe9b80693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8008040 comp rx=0 tx=0).stop 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d41596a0 msgr2=0x7fe9d415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d41596a0 0x7fe9d415e580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00090b0 tx=0x7fe9c0017000 comp rx=0 tx=0).stop 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 shutdown_connections 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d4159a70 0x7fe9d415eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe9b8066f30 0x7fe9b80693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d40b0160 0x7fe9d414a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 --2- 172.21.15.131:0/3732161461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9d41596a0 0x7fe9d415e580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 >> 172.21.15.131:0/3732161461 conn(0x7fe9d406ece0 msgr2=0x7fe9d40cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 shutdown_connections 2023-12-26T21:37:21.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.410+0000 7fe9db1ff700 1 -- 172.21.15.131:0/3732161461 wait complete. 2023-12-26T21:37:21.428 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:37:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.619+0000 7fe3fa9ff700 1 Processor -- start 2023-12-26T21:37:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.619+0000 7fe3fa9ff700 1 -- start start 2023-12-26T21:37:21.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.621+0000 7fe3fa9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3f4159820 0x7fe3f40abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.621+0000 7fe3fa9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f415a8c0 con 0x7fe3f406e480 2023-12-26T21:37:21.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.621+0000 7fe3fa9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f4159f90 con 0x7fe3f40aa480 2023-12-26T21:37:21.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.621+0000 7fe3fa9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f415ad50 con 0x7fe3f4159820 2023-12-26T21:37:21.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.621+0000 7fe3f8f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe3f406e480 0x7fe3f40675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35416/0 (socket says 172.21.15.131:35416) 2023-12-26T21:37:21.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.621+0000 7fe3f8f9c700 1 -- 172.21.15.131:0/3188511854 learned_addr learned my addr 172.21.15.131:0/3188511854 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:21.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.621+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/3188511854 >> v2:172.21.15.190:3300/0 conn(0x7fe3f4159820 0x7fe3f40abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.621+0000 7fe3f2ffd700 1 -- 172.21.15.131:0/3188511854 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103301708 0 0) 0x7fe3f415a8c0 con 0x7fe3f406e480 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.621+0000 7fe3f2ffd700 1 -- 172.21.15.131:0/3188511854 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3dc003570 con 0x7fe3f406e480 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.622+0000 7fe3f2ffd700 1 -- 172.21.15.131:0/3188511854 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153834366 0 0) 0x7fe3f4159f90 con 0x7fe3f40aa480 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.622+0000 7fe3f2ffd700 1 -- 172.21.15.131:0/3188511854 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f415a8c0 con 0x7fe3f40aa480 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.622+0000 7fe3f3fff700 1 -- 172.21.15.131:0/3188511854 >> v1:172.21.15.190:6789/0 conn(0x7fe3f40aa480 legacy=0x7fe3f406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.622+0000 7fe3f3fff700 1 -- 172.21.15.131:0/3188511854 >> v1:172.21.15.131:6789/0 conn(0x7fe3f406e480 legacy=0x7fe3f40675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.622+0000 7fe3f3fff700 1 -- 172.21.15.131:0/3188511854 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f415a270 con 0x7fe3f4159820 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.622+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/3188511854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3f4159820 0x7fe3f40abdb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40098e0 tx=0x7fe3e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eccd8f79a3da6aa8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.622+0000 7fe3f2ffd700 1 -- 172.21.15.131:0/3188511854 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4011630 con 0x7fe3f4159820 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.622+0000 7fe3f2ffd700 1 -- 172.21.15.131:0/3188511854 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4011790 con 0x7fe3f4159820 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.622+0000 7fe3f2ffd700 1 -- 172.21.15.131:0/3188511854 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4011b30 con 0x7fe3f4159820 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.623+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/3188511854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3f4159820 msgr2=0x7fe3f40abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.623+0000 7fe3fa9ff700 1 --2- 172.21.15.131:0/3188511854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3f4159820 0x7fe3f40abdb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40098e0 tx=0x7fe3e40091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.623+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/3188511854 shutdown_connections 2023-12-26T21:37:21.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.623+0000 7fe3fa9ff700 1 --2- 172.21.15.131:0/3188511854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3f4159820 0x7fe3f40abdb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.623+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/3188511854 >> 172.21.15.131:0/3188511854 conn(0x7fe3f406c290 msgr2=0x7fe3f40ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:21.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.623+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/3188511854 shutdown_connections 2023-12-26T21:37:21.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.623+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/3188511854 wait complete. 2023-12-26T21:37:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.624+0000 7fe3fa9ff700 1 Processor -- start 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.624+0000 7fe3fa9ff700 1 -- start start 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.625+0000 7fe3fa9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f406e480 0x7fe3f414b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.625+0000 7fe3fa9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3f40aa480 0x7fe3f40c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.625+0000 7fe3fa9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3f4159820 0x7fe3f40c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.625+0000 7fe3fa9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f416b0f0 con 0x7fe3f406e480 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.625+0000 7fe3fa9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f416b3d0 con 0x7fe3f40aa480 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.625+0000 7fe3fa9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f416b6b0 con 0x7fe3f4159820 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f406e480 0x7fe3f414b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f406e480 0x7fe3f414b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55562/0 (socket says 172.21.15.131:55562) 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f3fff700 1 -- 172.21.15.131:0/924004048 learned_addr learned my addr 172.21.15.131:0/924004048 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f37fe700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3f40aa480 0x7fe3f40c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f8f9c700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3f4159820 0x7fe3f40c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f3fff700 1 -- 172.21.15.131:0/924004048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3f4159820 msgr2=0x7fe3f40c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3f4159820 0x7fe3f40c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f3fff700 1 -- 172.21.15.131:0/924004048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3f40aa480 msgr2=0x7fe3f40c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3f40aa480 0x7fe3f40c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f3fff700 1 -- 172.21.15.131:0/924004048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e4009420 con 0x7fe3f406e480 2023-12-26T21:37:21.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.626+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f406e480 0x7fe3f414b5b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40af700 tx=0x7fe3e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:21.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.627+0000 7fe3dbfff700 1 -- 172.21.15.131:0/924004048 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e401f070 con 0x7fe3f406e480 2023-12-26T21:37:21.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.627+0000 7fe3dbfff700 1 -- 172.21.15.131:0/924004048 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4000bc0 con 0x7fe3f406e480 2023-12-26T21:37:21.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.627+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f416b990 con 0x7fe3f406e480 2023-12-26T21:37:21.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.627+0000 7fe3dbfff700 1 -- 172.21.15.131:0/924004048 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4025420 con 0x7fe3f406e480 2023-12-26T21:37:21.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.627+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f414baf0 con 0x7fe3f406e480 2023-12-26T21:37:21.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.628+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f416be30 con 0x7fe3f406e480 2023-12-26T21:37:21.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.629+0000 7fe3dbfff700 1 -- 172.21.15.131:0/924004048 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe3e4023070 con 0x7fe3f406e480 2023-12-26T21:37:21.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.629+0000 7fe3dbfff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3e0066f30 0x7fe3e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:21.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.630+0000 7fe3dbfff700 1 -- 172.21.15.131:0/924004048 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe3e40856f0 con 0x7fe3f406e480 2023-12-26T21:37:21.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.630+0000 7fe3f37fe700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3e0066f30 0x7fe3e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:21.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.630+0000 7fe3f37fe700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3e0066f30 0x7fe3e00693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40c4c20 tx=0x7fe3e8007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:21.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.633+0000 7fe3dbfff700 1 -- 172.21.15.131:0/924004048 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3e4052960 con 0x7fe3f406e480 2023-12-26T21:37:21.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.828+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3f415a550 con 0x7fe3f406e480 2023-12-26T21:37:21.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.829+0000 7fe3dbfff700 1 -- 172.21.15.131:0/924004048 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3e4055f80 con 0x7fe3f406e480 2023-12-26T21:37:21.829 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3e0066f30 msgr2=0x7fe3e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3e0066f30 0x7fe3e00693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40c4c20 tx=0x7fe3e8007d50 comp rx=0 tx=0).stop 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f406e480 msgr2=0x7fe3f414b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f406e480 0x7fe3f414b5b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40af700 tx=0x7fe3e4017000 comp rx=0 tx=0).stop 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 shutdown_connections 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3e0066f30 0x7fe3e00693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3f4159820 0x7fe3f40c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3f406e480 0x7fe3f414b5b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 --2- 172.21.15.131:0/924004048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3f40aa480 0x7fe3f40c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 >> 172.21.15.131:0/924004048 conn(0x7fe3f406c290 msgr2=0x7fe3f40adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:21.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 shutdown_connections 2023-12-26T21:37:21.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:21.831+0000 7fe3fa9ff700 1 -- 172.21.15.131:0/924004048 wait complete. 2023-12-26T21:37:21.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:37:22.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.042+0000 7f046e7ff700 1 Processor -- start 2023-12-26T21:37:22.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.042+0000 7f046e7ff700 1 -- start start 2023-12-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.043+0000 7f046e7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0468159920 0x7f04680b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f046e7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04680c8bf0 con 0x7f0468159cf0 2023-12-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f046e7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04680c8e60 con 0x7f0468155be0 2023-12-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f046e7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04680c9140 con 0x7f0468159920 2023-12-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f0467fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0468159920 0x7f04680b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f045ffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0468159cf0 0x7f04681554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35420/0 (socket says 172.21.15.131:35420) 2023-12-26T21:37:22.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f045ffff700 1 -- 172.21.15.131:0/848571765 learned_addr learned my addr 172.21.15.131:0/848571765 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:22.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f0467fff700 1 -- 172.21.15.131:0/848571765 >> v1:172.21.15.190:6789/0 conn(0x7f0468155be0 legacy=0x7f04680c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f0467fff700 1 -- 172.21.15.131:0/848571765 >> v1:172.21.15.131:6789/0 conn(0x7f0468159cf0 legacy=0x7f04681554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f0467fff700 1 -- 172.21.15.131:0/848571765 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04680c9420 con 0x7f0468159920 2023-12-26T21:37:22.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.044+0000 7f04677fe700 1 -- 172.21.15.131:0/848571765 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219997026 0 0) 0x7f04680c8bf0 con 0x7f0468159cf0 2023-12-26T21:37:22.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f0467fff700 1 --2- 172.21.15.131:0/848571765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468159920 0x7f04680b00e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f045800bb90 tx=0x7f045800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2628655eb38d5341 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:22.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f04677fe700 1 -- 172.21.15.131:0/848571765 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045800dba0 con 0x7f0468159920 2023-12-26T21:37:22.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f04677fe700 1 -- 172.21.15.131:0/848571765 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045800dd00 con 0x7f0468159920 2023-12-26T21:37:22.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f04677fe700 1 -- 172.21.15.131:0/848571765 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04580196b0 con 0x7f0468159920 2023-12-26T21:37:22.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f046e7ff700 1 -- 172.21.15.131:0/848571765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468159920 msgr2=0x7f04680b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f046e7ff700 1 --2- 172.21.15.131:0/848571765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468159920 0x7f04680b00e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f045800bb90 tx=0x7f045800b430 comp rx=0 tx=0).stop 2023-12-26T21:37:22.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f046e7ff700 1 -- 172.21.15.131:0/848571765 shutdown_connections 2023-12-26T21:37:22.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f046e7ff700 1 --2- 172.21.15.131:0/848571765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468159920 0x7f04680b00e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f046e7ff700 1 -- 172.21.15.131:0/848571765 >> 172.21.15.131:0/848571765 conn(0x7f0468068040 msgr2=0x7f0468069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:22.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f046e7ff700 1 -- 172.21.15.131:0/848571765 shutdown_connections 2023-12-26T21:37:22.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.045+0000 7f046e7ff700 1 -- 172.21.15.131:0/848571765 wait complete. 2023-12-26T21:37:22.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.047+0000 7f046e7ff700 1 Processor -- start 2023-12-26T21:37:22.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.047+0000 7f046e7ff700 1 -- start start 2023-12-26T21:37:22.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f046e7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468155be0 0x7f0468153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f046e7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468159920 0x7f04681576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f046e7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0468159cf0 0x7f0468140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f046e7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0468157220 con 0x7f0468155be0 2023-12-26T21:37:22.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f046e7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0468156610 con 0x7f0468159cf0 2023-12-26T21:37:22.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f046e7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04681568f0 con 0x7f0468159920 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f0467fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468155be0 0x7f0468153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f0467fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468155be0 0x7f0468153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55580/0 (socket says 172.21.15.131:55580) 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f0467fff700 1 -- 172.21.15.131:0/2309755742 learned_addr learned my addr 172.21.15.131:0/2309755742 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f0467fff700 1 -- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468159920 msgr2=0x7f04681576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f045ffff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468159920 0x7f04681576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f046cd9c700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0468159cf0 0x7f0468140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f0467fff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468159920 0x7f04681576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f0467fff700 1 -- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0468159cf0 msgr2=0x7f0468140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f0467fff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0468159cf0 0x7f0468140b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f0467fff700 1 -- 172.21.15.131:0/2309755742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045800b6d0 con 0x7f0468155be0 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f045ffff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468159920 0x7f04681576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.048+0000 7f046cd9c700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0468159cf0 0x7f0468140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.049+0000 7f0467fff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468155be0 0x7f0468153660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f045800b2c0 tx=0x7f045801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.049+0000 7f0464ff9700 1 -- 172.21.15.131:0/2309755742 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045801d070 con 0x7f0468155be0 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.049+0000 7f0464ff9700 1 -- 172.21.15.131:0/2309755742 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04580107f0 con 0x7f0468155be0 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.049+0000 7f0464ff9700 1 -- 172.21.15.131:0/2309755742 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458010bb0 con 0x7f0468155be0 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.049+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0468156bd0 con 0x7f0468155be0 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.049+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0468153ba0 con 0x7f0468155be0 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.050+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04680657e0 con 0x7f0468155be0 2023-12-26T21:37:22.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.051+0000 7f0464ff9700 1 -- 172.21.15.131:0/2309755742 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0458021070 con 0x7f0468155be0 2023-12-26T21:37:22.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.052+0000 7f0464ff9700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0448066f30 0x7f04480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.052+0000 7f0464ff9700 1 -- 172.21.15.131:0/2309755742 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0458014070 con 0x7f0468155be0 2023-12-26T21:37:22.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.052+0000 7f045ffff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0448066f30 0x7f04480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.052+0000 7f045ffff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0448066f30 0x7f04480693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f04680c8050 tx=0x7f0450009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:22.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.056+0000 7f0464ff9700 1 -- 172.21.15.131:0/2309755742 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0458053180 con 0x7f0468155be0 2023-12-26T21:37:22.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.250+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0468157500 con 0x7f0468155be0 2023-12-26T21:37:22.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.250+0000 7f0464ff9700 1 -- 172.21.15.131:0/2309755742 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0458056760 con 0x7f0468155be0 2023-12-26T21:37:22.251 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:37:22.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0448066f30 msgr2=0x7f04480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0448066f30 0x7f04480693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f04680c8050 tx=0x7f0450009040 comp rx=0 tx=0).stop 2023-12-26T21:37:22.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468155be0 msgr2=0x7f0468153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468155be0 0x7f0468153660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f045800b2c0 tx=0x7f045801b000 comp rx=0 tx=0).stop 2023-12-26T21:37:22.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 shutdown_connections 2023-12-26T21:37:22.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0468159cf0 0x7f0468140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0468155be0 0x7f0468153660 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0448066f30 0x7f04480693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 --2- 172.21.15.131:0/2309755742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468159920 0x7f04681576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.252+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 >> 172.21.15.131:0/2309755742 conn(0x7f0468068040 msgr2=0x7f04680ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:22.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.253+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 shutdown_connections 2023-12-26T21:37:22.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.253+0000 7f046e7ff700 1 -- 172.21.15.131:0/2309755742 wait complete. 2023-12-26T21:37:22.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:22.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.460+0000 7f243b1ff700 1 Processor -- start 2023-12-26T21:37:22.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.460+0000 7f243b1ff700 1 -- start start 2023-12-26T21:37:22.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.461+0000 7f243b1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24341596a0 0x7f24340b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.461+0000 7f243b1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2434159a70 0x7f24340b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.461+0000 7f243b1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24340afcd0 con 0x7f24340b0160 2023-12-26T21:37:22.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.461+0000 7f243b1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24340b3e70 con 0x7f24341596a0 2023-12-26T21:37:22.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.461+0000 7f243b1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24340b0cd0 con 0x7f2434159a70 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f2438f9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24341596a0 0x7f24340b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f2433fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2434159a70 0x7f24340b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f243979c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f24340b0160 0x7f24340b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:41656/0 (socket says 172.21.15.131:41656) 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f243979c700 1 -- 172.21.15.131:0/3137373591 learned_addr learned my addr 172.21.15.131:0/3137373591 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f2438f9b700 1 -- 172.21.15.131:0/3137373591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2434159a70 msgr2=0x7f24340b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f2438f9b700 1 --2- 172.21.15.131:0/3137373591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2434159a70 0x7f24340b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f2438f9b700 1 -- 172.21.15.131:0/3137373591 >> v1:172.21.15.190:6789/0 conn(0x7f24340b0160 legacy=0x7f24340b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f2438f9b700 1 -- 172.21.15.131:0/3137373591 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24340b2e00 con 0x7f24341596a0 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f24337fe700 1 -- 172.21.15.131:0/3137373591 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020829477 0 0) 0x7f24340afcd0 con 0x7f24340b0160 2023-12-26T21:37:22.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.462+0000 7f2438f9b700 1 --2- 172.21.15.131:0/3137373591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24341596a0 0x7f24340b2380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2428009e60 tx=0x7f2428009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=904885aed164635 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:22.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.463+0000 7f24337fe700 1 -- 172.21.15.131:0/3137373591 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242800e070 con 0x7f24341596a0 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.463+0000 7f24337fe700 1 -- 172.21.15.131:0/3137373591 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242800bb60 con 0x7f24341596a0 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.463+0000 7f24337fe700 1 -- 172.21.15.131:0/3137373591 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428012430 con 0x7f24341596a0 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.464+0000 7f243b1ff700 1 -- 172.21.15.131:0/3137373591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24341596a0 msgr2=0x7f24340b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.464+0000 7f243b1ff700 1 --2- 172.21.15.131:0/3137373591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24341596a0 0x7f24340b2380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2428009e60 tx=0x7f2428009780 comp rx=0 tx=0).stop 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.464+0000 7f243b1ff700 1 -- 172.21.15.131:0/3137373591 shutdown_connections 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.464+0000 7f243b1ff700 1 --2- 172.21.15.131:0/3137373591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24341596a0 0x7f24340b2380 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.464+0000 7f243b1ff700 1 --2- 172.21.15.131:0/3137373591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2434159a70 0x7f24340b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.464+0000 7f243b1ff700 1 -- 172.21.15.131:0/3137373591 >> 172.21.15.131:0/3137373591 conn(0x7f243406ece0 msgr2=0x7f243406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.464+0000 7f243b1ff700 1 -- 172.21.15.131:0/3137373591 shutdown_connections 2023-12-26T21:37:22.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.464+0000 7f243b1ff700 1 -- 172.21.15.131:0/3137373591 wait complete. 2023-12-26T21:37:22.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.465+0000 7f243b1ff700 1 Processor -- start 2023-12-26T21:37:22.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.465+0000 7f243b1ff700 1 -- start start 2023-12-26T21:37:22.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.466+0000 7f243b1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24340b0160 0x7f2434156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.466+0000 7f243b1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24341596a0 0x7f2434159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.466+0000 7f243b1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434159a70 0x7f24340c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.466+0000 7f243b1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243415b0c0 con 0x7f24341596a0 2023-12-26T21:37:22.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.466+0000 7f243b1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24340c41d0 con 0x7f2434159a70 2023-12-26T21:37:22.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.466+0000 7f243b1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243415a820 con 0x7f24340b0160 2023-12-26T21:37:22.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.466+0000 7f2433fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24341596a0 0x7f2434159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.466+0000 7f2433fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24341596a0 0x7f2434159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55590/0 (socket says 172.21.15.131:55590) 2023-12-26T21:37:22.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.466+0000 7f2433fff700 1 -- 172.21.15.131:0/2209547701 learned_addr learned my addr 172.21.15.131:0/2209547701 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:22.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f243979c700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434159a70 0x7f24340c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2438f9b700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24340b0160 0x7f2434156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2433fff700 1 -- 172.21.15.131:0/2209547701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24340b0160 msgr2=0x7f2434156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2433fff700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24340b0160 0x7f2434156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2433fff700 1 -- 172.21.15.131:0/2209547701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434159a70 msgr2=0x7f24340c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2433fff700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434159a70 0x7f24340c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2433fff700 1 -- 172.21.15.131:0/2209547701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428009a20 con 0x7f24341596a0 2023-12-26T21:37:22.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f243979c700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434159a70 0x7f24340c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:22.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2433fff700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24341596a0 0x7f2434159e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f242000e930 tx=0x7f24200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:22.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2430ff9700 1 -- 172.21.15.131:0/2209547701 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420013680 con 0x7f24341596a0 2023-12-26T21:37:22.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2430ff9700 1 -- 172.21.15.131:0/2209547701 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24200137e0 con 0x7f24341596a0 2023-12-26T21:37:22.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f2430ff9700 1 -- 172.21.15.131:0/2209547701 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420013ba0 con 0x7f24341596a0 2023-12-26T21:37:22.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.467+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243415ab00 con 0x7f24341596a0 2023-12-26T21:37:22.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.468+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2434157240 con 0x7f24341596a0 2023-12-26T21:37:22.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.469+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24340c4340 con 0x7f24341596a0 2023-12-26T21:37:22.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.469+0000 7f2430ff9700 1 -- 172.21.15.131:0/2209547701 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f242000c460 con 0x7f24341596a0 2023-12-26T21:37:22.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.470+0000 7f2430ff9700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2418066f30 0x7f24180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.470+0000 7f2430ff9700 1 -- 172.21.15.131:0/2209547701 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f24200845b0 con 0x7f24341596a0 2023-12-26T21:37:22.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.470+0000 7f2438f9b700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2418066f30 0x7f24180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.471+0000 7f2438f9b700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2418066f30 0x7f24180693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24280097c0 tx=0x7f2428019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:22.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.474+0000 7f2430ff9700 1 -- 172.21.15.131:0/2209547701 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2420051820 con 0x7f24341596a0 2023-12-26T21:37:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.665+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24340657e0 con 0x7f24341596a0 2023-12-26T21:37:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.666+0000 7f2430ff9700 1 -- 172.21.15.131:0/2209547701 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2420054e40 con 0x7f24341596a0 2023-12-26T21:37:22.666 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:22.667 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.668+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2418066f30 msgr2=0x7f24180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.668+0000 7f243b1ff700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2418066f30 0x7f24180693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24280097c0 tx=0x7f2428019000 comp rx=0 tx=0).stop 2023-12-26T21:37:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.668+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24341596a0 msgr2=0x7f2434159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.668+0000 7f243b1ff700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24341596a0 0x7f2434159e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f242000e930 tx=0x7f24200091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.669+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 shutdown_connections 2023-12-26T21:37:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.669+0000 7f243b1ff700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2434159a70 0x7f24340c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.669+0000 7f243b1ff700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2418066f30 0x7f24180693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.669+0000 7f243b1ff700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24340b0160 0x7f2434156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.669+0000 7f243b1ff700 1 --2- 172.21.15.131:0/2209547701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24341596a0 0x7f2434159e40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.669+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 >> 172.21.15.131:0/2209547701 conn(0x7f243406ece0 msgr2=0x7f2434158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.669+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 shutdown_connections 2023-12-26T21:37:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.669+0000 7f243b1ff700 1 -- 172.21.15.131:0/2209547701 wait complete. 2023-12-26T21:37:22.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:36:46.654972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:36:47.264880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:37:22.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:37:22.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.878+0000 7fcbf2bff700 1 Processor -- start 2023-12-26T21:37:22.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.878+0000 7fcbf2bff700 1 -- start start 2023-12-26T21:37:22.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.879+0000 7fcbf2bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbec0b0bf0 0x7fcbec0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.879+0000 7fcbf2bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbec156940 con 0x7fcbec0cb320 2023-12-26T21:37:22.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.879+0000 7fcbf2bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbec0ae500 con 0x7fcbec0b0750 2023-12-26T21:37:22.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.879+0000 7fcbf2bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec0c8cc0 con 0x7fcbec0b0bf0 2023-12-26T21:37:22.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbf099b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcbec0cb320 0x7fcbec069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35430/0 (socket says 172.21.15.131:35430) 2023-12-26T21:37:22.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbf099b700 1 -- 172.21.15.131:0/3464720034 learned_addr learned my addr 172.21.15.131:0/3464720034 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:22.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775349256 0 0) 0x7fcbec156940 con 0x7fcbec0cb320 2023-12-26T21:37:22.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbdc003570 con 0x7fcbec0cb320 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbf119c700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbec0b0bf0 0x7fcbec0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 707703621 0 0) 0x7fcbdc003570 con 0x7fcbec0cb320 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbec0b0bf0 msgr2=0x7fcbec0c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbeb7fe700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbec0b0bf0 0x7fcbec0c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 >> v1:172.21.15.190:6789/0 conn(0x7fcbec0b0750 legacy=0x7fcbec0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec0b0280 con 0x7fcbec0cb320 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fcbe0001470 con 0x7fcbec0cb320 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.880+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 >> v1:172.21.15.131:6789/0 conn(0x7fcbec0cb320 legacy=0x7fcbec069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbeb7fe700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbdc003a90 0x7fcbdc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbeb7fe700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbdc004810 0x7fcbdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbeb7fe700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbec0b0bf0 0x7fcbdc0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec0c8cc0 con 0x7fcbdc004810 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc00bb60 con 0x7fcbdc003a90 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc00df30 con 0x7fcbec0b0bf0 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbebfff700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbdc004810 0x7fcbdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbf099b700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbdc003a90 0x7fcbdc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbf119c700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbec0b0bf0 0x7fcbdc0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbebfff700 1 -- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbec0b0bf0 msgr2=0x7fcbdc0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbebfff700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbec0b0bf0 0x7fcbdc0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbebfff700 1 -- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbdc003a90 msgr2=0x7fcbdc003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbebfff700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbdc003a90 0x7fcbdc003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbebfff700 1 -- 172.21.15.131:0/3464720034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc003570 con 0x7fcbdc004810 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.881+0000 7fcbebfff700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbdc004810 0x7fcbdc004c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80018e0 tx=0x7fcbd8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7612eb284fcf7491 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8012420 con 0x7fcbdc004810 2023-12-26T21:37:22.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd8012580 con 0x7fcbdc004810 2023-12-26T21:37:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbeb7fe700 1 -- 172.21.15.131:0/3464720034 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8012920 con 0x7fcbdc004810 2023-12-26T21:37:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbf2bff700 1 -- 172.21.15.131:0/3464720034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbdc004810 msgr2=0x7fcbdc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbf2bff700 1 --2- 172.21.15.131:0/3464720034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbdc004810 0x7fcbdc004c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80018e0 tx=0x7fcbd8007d30 comp rx=0 tx=0).stop 2023-12-26T21:37:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbf119c700 1 -- 172.21.15.131:0/3464720034 reap_dead start 2023-12-26T21:37:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbf2bff700 1 -- 172.21.15.131:0/3464720034 shutdown_connections 2023-12-26T21:37:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbf2bff700 1 -- 172.21.15.131:0/3464720034 >> 172.21.15.131:0/3464720034 conn(0x7fcbec068040 msgr2=0x7fcbec069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbf2bff700 1 -- 172.21.15.131:0/3464720034 shutdown_connections 2023-12-26T21:37:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.882+0000 7fcbf2bff700 1 -- 172.21.15.131:0/3464720034 wait complete. 2023-12-26T21:37:22.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.884+0000 7fcbf2bff700 1 Processor -- start 2023-12-26T21:37:22.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.884+0000 7fcbf2bff700 1 -- start start 2023-12-26T21:37:22.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbf2bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbdc004810 0x7fcbec169aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbf2bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec16c850 0x7fcbec0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbf2bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbec0ac170 0x7fcbec159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbf2bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec0c10a0 con 0x7fcbec16c850 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbf2bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec0ac5e0 con 0x7fcbec0ac170 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbf2bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec15b930 con 0x7fcbdc004810 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec16c850 0x7fcbec0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec16c850 0x7fcbec0abc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55618/0 (socket says 172.21.15.131:55618) 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbebfff700 1 -- 172.21.15.131:0/728728213 learned_addr learned my addr 172.21.15.131:0/728728213 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbf119c700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbec0ac170 0x7fcbec159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbf099b700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbdc004810 0x7fcbec169aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbebfff700 1 -- 172.21.15.131:0/728728213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbdc004810 msgr2=0x7fcbec169aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbebfff700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbdc004810 0x7fcbec169aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbebfff700 1 -- 172.21.15.131:0/728728213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbec0ac170 msgr2=0x7fcbec159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbebfff700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbec0ac170 0x7fcbec159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.885+0000 7fcbebfff700 1 -- 172.21.15.131:0/728728213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec15ad10 con 0x7fcbec16c850 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.886+0000 7fcbebfff700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec16c850 0x7fcbec0abc30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800c0e0 tx=0x7fcbd8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.886+0000 7fcbe8ff9700 1 -- 172.21.15.131:0/728728213 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8023070 con 0x7fcbec16c850 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.886+0000 7fcbe8ff9700 1 -- 172.21.15.131:0/728728213 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd8012900 con 0x7fcbec16c850 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.886+0000 7fcbe8ff9700 1 -- 172.21.15.131:0/728728213 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8012420 con 0x7fcbec16c850 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.886+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbec15aff0 con 0x7fcbec16c850 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.886+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec16ccc0 con 0x7fcbec16c850 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.887+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbec15b420 con 0x7fcbec16c850 2023-12-26T21:37:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.888+0000 7fcbe8ff9700 1 -- 172.21.15.131:0/728728213 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcbd8019070 con 0x7fcbec16c850 2023-12-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.889+0000 7fcbe8ff9700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbd0066f30 0x7fcbd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.889+0000 7fcbe8ff9700 1 -- 172.21.15.131:0/728728213 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcbd8085820 con 0x7fcbec16c850 2023-12-26T21:37:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.889+0000 7fcbf099b700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbd0066f30 0x7fcbd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.889+0000 7fcbf099b700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbd0066f30 0x7fcbd00693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0001ce0 tx=0x7fcbe0001e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:22.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:22.893+0000 7fcbe8ff9700 1 -- 172.21.15.131:0/728728213 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbec15b420 con 0x7fcbec16c850 2023-12-26T21:37:23.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.089+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcbec15b420 con 0x7fcbec16c850 2023-12-26T21:37:23.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.089+0000 7fcbe8ff9700 1 -- 172.21.15.131:0/728728213 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcbec15b420 con 0x7fcbec16c850 2023-12-26T21:37:23.090 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.091+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbd0066f30 msgr2=0x7fcbd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.091+0000 7fcbf2bff700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbd0066f30 0x7fcbd00693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0001ce0 tx=0x7fcbe0001e50 comp rx=0 tx=0).stop 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.091+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec16c850 msgr2=0x7fcbec0abc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.092+0000 7fcbf2bff700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec16c850 0x7fcbec0abc30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800c0e0 tx=0x7fcbd8015000 comp rx=0 tx=0).stop 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.092+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 shutdown_connections 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.092+0000 7fcbf2bff700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbec0ac170 0x7fcbec159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.092+0000 7fcbf2bff700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbd0066f30 0x7fcbd00693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.092+0000 7fcbf2bff700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbdc004810 0x7fcbec169aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.092+0000 7fcbf2bff700 1 --2- 172.21.15.131:0/728728213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbec16c850 0x7fcbec0abc30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.092+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 >> 172.21.15.131:0/728728213 conn(0x7fcbec068040 msgr2=0x7fcbec0acec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.092+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 shutdown_connections 2023-12-26T21:37:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.092+0000 7fcbf2bff700 1 -- 172.21.15.131:0/728728213 wait complete. 2023-12-26T21:37:23.110 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:37:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.301+0000 7fa6d7dff700 1 Processor -- start 2023-12-26T21:37:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.301+0000 7fa6d7dff700 1 -- start start 2023-12-26T21:37:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.302+0000 7fa6d7dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa6d0159920 0x7fa6d0159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.302+0000 7fa6d7dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa6d00c5420 0x7fa6d00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d7dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d00c6810 con 0x7fa6d00ced90 2023-12-26T21:37:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d7dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d015a230 con 0x7fa6d00c5420 2023-12-26T21:37:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d7dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d00c5c00 con 0x7fa6d0159920 2023-12-26T21:37:23.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d539a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa6d00c5420 0x7fa6d00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d539a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa6d00c5420 0x7fa6d00c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55630/0 (socket says 172.21.15.131:55630) 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d539a700 1 -- 172.21.15.131:0/1024576752 learned_addr learned my addr 172.21.15.131:0/1024576752 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d5b9b700 1 --2- 172.21.15.131:0/1024576752 >> v2:172.21.15.190:3300/0 conn(0x7fa6d0159920 0x7fa6d0159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d539a700 1 -- 172.21.15.131:0/1024576752 >> v2:172.21.15.190:3300/0 conn(0x7fa6d0159920 msgr2=0x7fa6d0159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d539a700 1 --2- 172.21.15.131:0/1024576752 >> v2:172.21.15.190:3300/0 conn(0x7fa6d0159920 0x7fa6d0159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d539a700 1 -- 172.21.15.131:0/1024576752 >> v1:172.21.15.190:6789/0 conn(0x7fa6d00ced90 legacy=0x7fa6d00cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d539a700 1 -- 172.21.15.131:0/1024576752 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d00c5ee0 con 0x7fa6d00c5420 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.303+0000 7fa6d539a700 1 --2- 172.21.15.131:0/1024576752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 0x7fa6d00c6ca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0088ec0 tx=0x7fa6c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9171695493b110a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d4b99700 1 -- 172.21.15.131:0/1024576752 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000b820 con 0x7fa6d00c5420 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d4b99700 1 -- 172.21.15.131:0/1024576752 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c000b980 con 0x7fa6d00c5420 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d4b99700 1 -- 172.21.15.131:0/1024576752 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000bd20 con 0x7fa6d00c5420 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d7dff700 1 -- 172.21.15.131:0/1024576752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 msgr2=0x7fa6d00c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d7dff700 1 --2- 172.21.15.131:0/1024576752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 0x7fa6d00c6ca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0088ec0 tx=0x7fa6c0009250 comp rx=0 tx=0).stop 2023-12-26T21:37:23.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d7dff700 1 -- 172.21.15.131:0/1024576752 shutdown_connections 2023-12-26T21:37:23.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d7dff700 1 --2- 172.21.15.131:0/1024576752 >> v2:172.21.15.190:3300/0 conn(0x7fa6d0159920 0x7fa6d0159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d7dff700 1 --2- 172.21.15.131:0/1024576752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 0x7fa6d00c6ca0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d7dff700 1 -- 172.21.15.131:0/1024576752 >> 172.21.15.131:0/1024576752 conn(0x7fa6d0099980 msgr2=0x7fa6d006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:23.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d7dff700 1 -- 172.21.15.131:0/1024576752 shutdown_connections 2023-12-26T21:37:23.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.304+0000 7fa6d7dff700 1 -- 172.21.15.131:0/1024576752 wait complete. 2023-12-26T21:37:23.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.306+0000 7fa6d7dff700 1 Processor -- start 2023-12-26T21:37:23.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.306+0000 7fa6d7dff700 1 -- start start 2023-12-26T21:37:23.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d7dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 0x7fa6d00ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d7dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00ced90 0x7fa6d01585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d7dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d0159920 0x7fa6d0149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d7dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d016d3f0 con 0x7fa6d00c5420 2023-12-26T21:37:23.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d7dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d016d6d0 con 0x7fa6d00ced90 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d7dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d016d9b0 con 0x7fa6d0159920 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d5b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 0x7fa6d00ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d5b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 0x7fa6d00ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55636/0 (socket says 172.21.15.131:55636) 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d5b9b700 1 -- 172.21.15.131:0/3262593317 learned_addr learned my addr 172.21.15.131:0/3262593317 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d539a700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00ced90 0x7fa6d01585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.307+0000 7fa6d639c700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d0159920 0x7fa6d0149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6d5b9b700 1 -- 172.21.15.131:0/3262593317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d0159920 msgr2=0x7fa6d0149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6d5b9b700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d0159920 0x7fa6d0149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6d5b9b700 1 -- 172.21.15.131:0/3262593317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00ced90 msgr2=0x7fa6d01585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6d5b9b700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00ced90 0x7fa6d01585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6d5b9b700 1 -- 172.21.15.131:0/3262593317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00094f0 con 0x7fa6d00c5420 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6d5b9b700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 0x7fa6d00ca1e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00d890 tx=0x7fa6cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6c5ffb700 1 -- 172.21.15.131:0/3262593317 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc018070 con 0x7fa6d00c5420 2023-12-26T21:37:23.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6c5ffb700 1 -- 172.21.15.131:0/3262593317 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc005a00 con 0x7fa6d00c5420 2023-12-26T21:37:23.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d016dc90 con 0x7fa6d00c5420 2023-12-26T21:37:23.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6c5ffb700 1 -- 172.21.15.131:0/3262593317 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc005dc0 con 0x7fa6d00c5420 2023-12-26T21:37:23.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.308+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d0158af0 con 0x7fa6d00c5420 2023-12-26T21:37:23.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.309+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d0158d10 con 0x7fa6d00c5420 2023-12-26T21:37:23.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.310+0000 7fa6c5ffb700 1 -- 172.21.15.131:0/3262593317 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa6cc012800 con 0x7fa6d00c5420 2023-12-26T21:37:23.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.311+0000 7fa6c5ffb700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa6b4066f30 0x7fa6b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.311+0000 7fa6c5ffb700 1 -- 172.21.15.131:0/3262593317 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa6cc084740 con 0x7fa6d00c5420 2023-12-26T21:37:23.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.311+0000 7fa6d539a700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa6b4066f30 0x7fa6b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.312+0000 7fa6d539a700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa6b4066f30 0x7fa6b40693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0088ec0 tx=0x7fa6c0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:23.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.315+0000 7fa6c5ffb700 1 -- 172.21.15.131:0/3262593317 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6cc051910 con 0x7fa6d00c5420 2023-12-26T21:37:23.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.508+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6d00657e0 con 0x7fa6d00c5420 2023-12-26T21:37:23.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.508+0000 7fa6c5ffb700 1 -- 172.21.15.131:0/3262593317 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6cc054f30 con 0x7fa6d00c5420 2023-12-26T21:37:23.509 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa6b4066f30 msgr2=0x7fa6b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa6b4066f30 0x7fa6b40693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0088ec0 tx=0x7fa6c0016000 comp rx=0 tx=0).stop 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 msgr2=0x7fa6d00ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 0x7fa6d00ca1e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00d890 tx=0x7fa6cc008040 comp rx=0 tx=0).stop 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 shutdown_connections 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa6b4066f30 0x7fa6b40693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d0159920 0x7fa6d0149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d00c5420 0x7fa6d00ca1e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 --2- 172.21.15.131:0/3262593317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00ced90 0x7fa6d01585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 >> 172.21.15.131:0/3262593317 conn(0x7fa6d0099980 msgr2=0x7fa6d0069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 shutdown_connections 2023-12-26T21:37:23.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.511+0000 7fa6d7dff700 1 -- 172.21.15.131:0/3262593317 wait complete. 2023-12-26T21:37:23.530 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:37:23.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.722+0000 7fd6659ff700 1 Processor -- start 2023-12-26T21:37:23.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.722+0000 7fd6659ff700 1 -- start start 2023-12-26T21:37:23.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.723+0000 7fd6659ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd660159a70 0x7fd6600b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.723+0000 7fd6659ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6600b2a90 0x7fd6600b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.723+0000 7fd6659ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6600afcd0 con 0x7fd6601596a0 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.723+0000 7fd6659ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6600b3e70 con 0x7fd6600b2a90 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.723+0000 7fd6659ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6600b0f20 con 0x7fd660159a70 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.723+0000 7fd65f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6600b2a90 0x7fd6600b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6600b2a90 0x7fd6600b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55638/0 (socket says 172.21.15.131:55638) 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65f7fe700 1 -- 172.21.15.131:0/3922208911 learned_addr learned my addr 172.21.15.131:0/3922208911 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65e7fc700 1 --2- 172.21.15.131:0/3922208911 >> v2:172.21.15.190:3300/0 conn(0x7fd660159a70 0x7fd6600b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65f7fe700 1 -- 172.21.15.131:0/3922208911 >> v2:172.21.15.190:3300/0 conn(0x7fd660159a70 msgr2=0x7fd6600b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65f7fe700 1 --2- 172.21.15.131:0/3922208911 >> v2:172.21.15.190:3300/0 conn(0x7fd660159a70 0x7fd6600b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65f7fe700 1 -- 172.21.15.131:0/3922208911 >> v1:172.21.15.190:6789/0 conn(0x7fd6601596a0 legacy=0x7fd6600b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65f7fe700 1 -- 172.21.15.131:0/3922208911 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6600b3540 con 0x7fd6600b2a90 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65e7fc700 1 --2- 172.21.15.131:0/3922208911 >> v2:172.21.15.190:3300/0 conn(0x7fd660159a70 0x7fd6600b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:23.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65f7fe700 1 --2- 172.21.15.131:0/3922208911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6600b2a90 0x7fd6600b06a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd6600642f0 tx=0x7fd658009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bfc9fbf5ceafaf server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65dffb700 1 -- 172.21.15.131:0/3922208911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65800e070 con 0x7fd6600b2a90 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65dffb700 1 -- 172.21.15.131:0/3922208911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65800ba50 con 0x7fd6600b2a90 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.724+0000 7fd65dffb700 1 -- 172.21.15.131:0/3922208911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6580092e0 con 0x7fd6600b2a90 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.725+0000 7fd6659ff700 1 -- 172.21.15.131:0/3922208911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6600b2a90 msgr2=0x7fd6600b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.725+0000 7fd6659ff700 1 --2- 172.21.15.131:0/3922208911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6600b2a90 0x7fd6600b06a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd6600642f0 tx=0x7fd658009840 comp rx=0 tx=0).stop 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.725+0000 7fd6659ff700 1 -- 172.21.15.131:0/3922208911 shutdown_connections 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.725+0000 7fd6659ff700 1 --2- 172.21.15.131:0/3922208911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6600b2a90 0x7fd6600b06a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.725+0000 7fd6659ff700 1 --2- 172.21.15.131:0/3922208911 >> v2:172.21.15.190:3300/0 conn(0x7fd660159a70 0x7fd6600b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.725+0000 7fd6659ff700 1 -- 172.21.15.131:0/3922208911 >> 172.21.15.131:0/3922208911 conn(0x7fd66006ece0 msgr2=0x7fd66006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.725+0000 7fd6659ff700 1 -- 172.21.15.131:0/3922208911 shutdown_connections 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.725+0000 7fd6659ff700 1 -- 172.21.15.131:0/3922208911 wait complete. 2023-12-26T21:37:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.727+0000 7fd6659ff700 1 Processor -- start 2023-12-26T21:37:23.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.727+0000 7fd6659ff700 1 -- start start 2023-12-26T21:37:23.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd6659ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6600b2a90 0x7fd6600cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd6659ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6601596a0 0x7fd6600c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd6659ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd660159a70 0x7fd660155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd6659ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600c8500 con 0x7fd660159a70 2023-12-26T21:37:23.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd6659ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600c87d0 con 0x7fd6600b2a90 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd6659ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600c8ab0 con 0x7fd6601596a0 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd65f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd660159a70 0x7fd660155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd65f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd660159a70 0x7fd660155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55640/0 (socket says 172.21.15.131:55640) 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd65f7fe700 1 -- 172.21.15.131:0/1595267058 learned_addr learned my addr 172.21.15.131:0/1595267058 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd65f7fe700 1 -- 172.21.15.131:0/1595267058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6601596a0 msgr2=0x7fd6600c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd65f7fe700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6601596a0 0x7fd6600c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd65f7fe700 1 -- 172.21.15.131:0/1595267058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6600b2a90 msgr2=0x7fd6600cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd65f7fe700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6600b2a90 0x7fd6600cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd65f7fe700 1 -- 172.21.15.131:0/1595267058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd658009ae0 con 0x7fd660159a70 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.728+0000 7fd65f7fe700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd660159a70 0x7fd660155320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd658009880 tx=0x7fd658018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.729+0000 7fd646ffd700 1 -- 172.21.15.131:0/1595267058 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658008e80 con 0x7fd660159a70 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.729+0000 7fd646ffd700 1 -- 172.21.15.131:0/1595267058 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65800be60 con 0x7fd660159a70 2023-12-26T21:37:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.729+0000 7fd646ffd700 1 -- 172.21.15.131:0/1595267058 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658026690 con 0x7fd660159a70 2023-12-26T21:37:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.729+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6600c8d90 con 0x7fd660159a70 2023-12-26T21:37:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.729+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6600c9070 con 0x7fd660159a70 2023-12-26T21:37:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.730+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd660159e60 con 0x7fd660159a70 2023-12-26T21:37:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.731+0000 7fd646ffd700 1 -- 172.21.15.131:0/1595267058 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd658003040 con 0x7fd660159a70 2023-12-26T21:37:23.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.731+0000 7fd646ffd700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd64c066f30 0x7fd64c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:23.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.732+0000 7fd646ffd700 1 -- 172.21.15.131:0/1595267058 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd6580865c0 con 0x7fd660159a70 2023-12-26T21:37:23.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.732+0000 7fd65effd700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd64c066f30 0x7fd64c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:23.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.732+0000 7fd65effd700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd64c066f30 0x7fd64c0693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd650001730 tx=0x7fd650007b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:23.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.736+0000 7fd646ffd700 1 -- 172.21.15.131:0/1595267058 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd660159e60 con 0x7fd660159a70 2023-12-26T21:37:23.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.929+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd660159e60 con 0x7fd660159a70 2023-12-26T21:37:23.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.929+0000 7fd646ffd700 1 -- 172.21.15.131:0/1595267058 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd660159e60 con 0x7fd660159a70 2023-12-26T21:37:23.930 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:37:23.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.931+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd64c066f30 msgr2=0x7fd64c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.931+0000 7fd6659ff700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd64c066f30 0x7fd64c0693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd650001730 tx=0x7fd650007b40 comp rx=0 tx=0).stop 2023-12-26T21:37:23.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.931+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd660159a70 msgr2=0x7fd660155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:23.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.931+0000 7fd6659ff700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd660159a70 0x7fd660155320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd658009880 tx=0x7fd658018000 comp rx=0 tx=0).stop 2023-12-26T21:37:23.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.932+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 shutdown_connections 2023-12-26T21:37:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.932+0000 7fd6659ff700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd660159a70 0x7fd660155320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.932+0000 7fd6659ff700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6600b2a90 0x7fd6600cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.932+0000 7fd6659ff700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd64c066f30 0x7fd64c0693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.932+0000 7fd6659ff700 1 --2- 172.21.15.131:0/1595267058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6601596a0 0x7fd6600c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.932+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 >> 172.21.15.131:0/1595267058 conn(0x7fd66006ece0 msgr2=0x7fd6600ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.932+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 shutdown_connections 2023-12-26T21:37:23.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:23.932+0000 7fd6659ff700 1 -- 172.21.15.131:0/1595267058 wait complete. 2023-12-26T21:37:23.977 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-12-26T21:37:23.977 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T21:37:23.977 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-12-26T21:37:24.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.167+0000 7fa7ef1ff700 1 Processor -- start 2023-12-26T21:37:24.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.167+0000 7fa7ef1ff700 1 -- start start 2023-12-26T21:37:24.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.168+0000 7fa7ef1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e8159a70 0x7fa7e80b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:24.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.168+0000 7fa7ef1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa7e80b2a90 0x7fa7e80b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:24.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.168+0000 7fa7ef1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e80afcd0 con 0x7fa7e81596a0 2023-12-26T21:37:24.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.168+0000 7fa7ef1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e80b3e70 con 0x7fa7e80b2a90 2023-12-26T21:37:24.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.168+0000 7fa7ef1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80b0f20 con 0x7fa7e8159a70 2023-12-26T21:37:24.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.168+0000 7fa7ed79c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa7e80b2a90 0x7fa7e80b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7ecf9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa7e81596a0 0x7fa7e80b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35432/0 (socket says 172.21.15.131:35432) 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7ed79c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa7e80b2a90 0x7fa7e80b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55642/0 (socket says 172.21.15.131:55642) 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7ecf9b700 1 -- 172.21.15.131:0/2583973647 learned_addr learned my addr 172.21.15.131:0/2583973647 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7e7fff700 1 --2- 172.21.15.131:0/2583973647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e8159a70 0x7fa7e80b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7ed79c700 1 -- 172.21.15.131:0/2583973647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e8159a70 msgr2=0x7fa7e80b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7ed79c700 1 --2- 172.21.15.131:0/2583973647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e8159a70 0x7fa7e80b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7ed79c700 1 -- 172.21.15.131:0/2583973647 >> v1:172.21.15.131:6789/0 conn(0x7fa7e81596a0 legacy=0x7fa7e80b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7ed79c700 1 -- 172.21.15.131:0/2583973647 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80b3540 con 0x7fa7e80b2a90 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7e77fe700 1 -- 172.21.15.131:0/2583973647 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677833941 0 0) 0x7fa7e80afcd0 con 0x7fa7e81596a0 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7ed79c700 1 --2- 172.21.15.131:0/2583973647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e80b2a90 0x7fa7e80b06a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009e60 tx=0x7fa7e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29e62d2ae5dd1c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.169+0000 7fa7e77fe700 1 -- 172.21.15.131:0/2583973647 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000e070 con 0x7fa7e80b2a90 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7e77fe700 1 -- 172.21.15.131:0/2583973647 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e000bc30 con 0x7fa7e80b2a90 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7e77fe700 1 -- 172.21.15.131:0/2583973647 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0012430 con 0x7fa7e80b2a90 2023-12-26T21:37:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/2583973647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e80b2a90 msgr2=0x7fa7e80b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7ef1ff700 1 --2- 172.21.15.131:0/2583973647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e80b2a90 0x7fa7e80b06a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009e60 tx=0x7fa7e0009780 comp rx=0 tx=0).stop 2023-12-26T21:37:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/2583973647 shutdown_connections 2023-12-26T21:37:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7ef1ff700 1 --2- 172.21.15.131:0/2583973647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e80b2a90 0x7fa7e80b06a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7ef1ff700 1 --2- 172.21.15.131:0/2583973647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e8159a70 0x7fa7e80b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/2583973647 >> 172.21.15.131:0/2583973647 conn(0x7fa7e806ece0 msgr2=0x7fa7e806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/2583973647 shutdown_connections 2023-12-26T21:37:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.170+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/2583973647 wait complete. 2023-12-26T21:37:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.172+0000 7fa7ef1ff700 1 Processor -- start 2023-12-26T21:37:24.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.172+0000 7fa7ef1ff700 1 -- start start 2023-12-26T21:37:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ef1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e80b2a90 0x7fa7e8151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ef1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7e81596a0 0x7fa7e80ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ef1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e8159a70 0x7fa7e80cf410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ef1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80c6ec0 con 0x7fa7e8159a70 2023-12-26T21:37:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ef1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80c6be0 con 0x7fa7e81596a0 2023-12-26T21:37:24.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ef1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e8159f70 con 0x7fa7e80b2a90 2023-12-26T21:37:24.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ed79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e8159a70 0x7fa7e80cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:24.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ed79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e8159a70 0x7fa7e80cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55658/0 (socket says 172.21.15.131:55658) 2023-12-26T21:37:24.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ed79c700 1 -- 172.21.15.131:0/3759879275 learned_addr learned my addr 172.21.15.131:0/3759879275 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:24.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7e7fff700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7e81596a0 0x7fa7e80ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:24.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ecf9b700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e80b2a90 0x7fa7e8151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:24.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ed79c700 1 -- 172.21.15.131:0/3759879275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e80b2a90 msgr2=0x7fa7e8151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:24.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ed79c700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e80b2a90 0x7fa7e8151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:24.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ed79c700 1 -- 172.21.15.131:0/3759879275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7e81596a0 msgr2=0x7fa7e80ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:24.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ed79c700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7e81596a0 0x7fa7e80ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:24.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.174+0000 7fa7ed79c700 1 -- 172.21.15.131:0/3759879275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e0009a20 con 0x7fa7e8159a70 2023-12-26T21:37:24.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.175+0000 7fa7ed79c700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e8159a70 0x7fa7e80cf410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0013000 tx=0x7fa7e0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:24.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.175+0000 7fa7e4ff9700 1 -- 172.21.15.131:0/3759879275 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0027470 con 0x7fa7e8159a70 2023-12-26T21:37:24.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.175+0000 7fa7e4ff9700 1 -- 172.21.15.131:0/3759879275 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e00275d0 con 0x7fa7e8159a70 2023-12-26T21:37:24.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.175+0000 7fa7e4ff9700 1 -- 172.21.15.131:0/3759879275 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0027990 con 0x7fa7e8159a70 2023-12-26T21:37:24.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.175+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e815a250 con 0x7fa7e8159a70 2023-12-26T21:37:24.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.175+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e8151da0 con 0x7fa7e8159a70 2023-12-26T21:37:24.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.176+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e80657e0 con 0x7fa7e8159a70 2023-12-26T21:37:24.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.177+0000 7fa7e4ff9700 1 -- 172.21.15.131:0/3759879275 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa7e0024070 con 0x7fa7e8159a70 2023-12-26T21:37:24.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.178+0000 7fa7e4ff9700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa7cc066f30 0x7fa7cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:24.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.178+0000 7fa7e4ff9700 1 -- 172.21.15.131:0/3759879275 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa7e0086d30 con 0x7fa7e8159a70 2023-12-26T21:37:24.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.178+0000 7fa7ecf9b700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa7cc066f30 0x7fa7cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:24.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.178+0000 7fa7ecf9b700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa7cc066f30 0x7fa7cc0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc007fd0 tx=0x7fa7dc007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:24.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.182+0000 7fa7e4ff9700 1 -- 172.21.15.131:0/3759879275 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7e0053f00 con 0x7fa7e8159a70 2023-12-26T21:37:24.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:24.449+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fa7e815a770 con 0x7fa7e8159a70 2023-12-26T21:37:25.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.042+0000 7fa7e4ff9700 1 -- 172.21.15.131:0/3759879275 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa7e0057520 con 0x7fa7e8159a70 2023-12-26T21:37:25.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.044+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa7cc066f30 msgr2=0x7fa7cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.044+0000 7fa7ef1ff700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa7cc066f30 0x7fa7cc0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc007fd0 tx=0x7fa7dc007d30 comp rx=0 tx=0).stop 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.044+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e8159a70 msgr2=0x7fa7e80cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.044+0000 7fa7ef1ff700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e8159a70 0x7fa7e80cf410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0013000 tx=0x7fa7e0019000 comp rx=0 tx=0).stop 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.045+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 shutdown_connections 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.045+0000 7fa7ef1ff700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7e8159a70 0x7fa7e80cf410 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.045+0000 7fa7ef1ff700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa7cc066f30 0x7fa7cc0693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.045+0000 7fa7ef1ff700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7e80b2a90 0x7fa7e8151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.045+0000 7fa7ef1ff700 1 --2- 172.21.15.131:0/3759879275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7e81596a0 0x7fa7e80ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.045+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 >> 172.21.15.131:0/3759879275 conn(0x7fa7e806ece0 msgr2=0x7fa7e80ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.045+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 shutdown_connections 2023-12-26T21:37:25.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.045+0000 7fa7ef1ff700 1 -- 172.21.15.131:0/3759879275 wait complete. 2023-12-26T21:37:25.064 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T21:37:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.254+0000 7fdd00fff700 1 Processor -- start 2023-12-26T21:37:25.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.255+0000 7fdd00fff700 1 -- start start 2023-12-26T21:37:25.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.256+0000 7fdd00fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdcfc0c8bf0 0x7fdcfc0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:25.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.256+0000 7fdd00fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc069d10 0x7fdcfc0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:25.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.256+0000 7fdd00fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc0caa10 con 0x7fdcfc0c8820 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.256+0000 7fdd00fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc157a80 con 0x7fdcfc0c8bf0 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.256+0000 7fdd00fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0cac50 con 0x7fdcfc069d10 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.256+0000 7fdcf3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdcfc0c8bf0 0x7fdcfc0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.256+0000 7fdcf3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdcfc0c8bf0 0x7fdcfc0b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55842/0 (socket says 172.21.15.131:55842) 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.256+0000 7fdcf3fff700 1 -- 172.21.15.131:0/2584859379 learned_addr learned my addr 172.21.15.131:0/2584859379 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.256+0000 7fdcfad9d700 1 --2- 172.21.15.131:0/2584859379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc069d10 0x7fdcfc0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.257+0000 7fdcf3fff700 1 -- 172.21.15.131:0/2584859379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc069d10 msgr2=0x7fdcfc0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.257+0000 7fdcf3fff700 1 --2- 172.21.15.131:0/2584859379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc069d10 0x7fdcfc0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.257+0000 7fdcf3fff700 1 -- 172.21.15.131:0/2584859379 >> v1:172.21.15.190:6789/0 conn(0x7fdcfc0c8820 legacy=0x7fdcfc0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.257+0000 7fdcf3fff700 1 -- 172.21.15.131:0/2584859379 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc0caf30 con 0x7fdcfc0c8bf0 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.257+0000 7fdcf3fff700 1 --2- 172.21.15.131:0/2584859379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc0c8bf0 0x7fdcfc0b12e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdce40099b0 tx=0x7fdce4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8661995149ad19cc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.257+0000 7fdcf9d9b700 1 -- 172.21.15.131:0/2584859379 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400b860 con 0x7fdcfc0c8bf0 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.257+0000 7fdcf9d9b700 1 -- 172.21.15.131:0/2584859379 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce400b9c0 con 0x7fdcfc0c8bf0 2023-12-26T21:37:25.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.257+0000 7fdcf9d9b700 1 -- 172.21.15.131:0/2584859379 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400bd60 con 0x7fdcfc0c8bf0 2023-12-26T21:37:25.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.258+0000 7fdd00fff700 1 -- 172.21.15.131:0/2584859379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc0c8bf0 msgr2=0x7fdcfc0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:25.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.258+0000 7fdd00fff700 1 --2- 172.21.15.131:0/2584859379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc0c8bf0 0x7fdcfc0b12e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdce40099b0 tx=0x7fdce4009250 comp rx=0 tx=0).stop 2023-12-26T21:37:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.258+0000 7fdd00fff700 1 -- 172.21.15.131:0/2584859379 shutdown_connections 2023-12-26T21:37:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.258+0000 7fdd00fff700 1 --2- 172.21.15.131:0/2584859379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc069d10 0x7fdcfc0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.258+0000 7fdd00fff700 1 --2- 172.21.15.131:0/2584859379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc0c8bf0 0x7fdcfc0b12e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.258+0000 7fdd00fff700 1 -- 172.21.15.131:0/2584859379 >> 172.21.15.131:0/2584859379 conn(0x7fdcfc068040 msgr2=0x7fdcfc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.258+0000 7fdd00fff700 1 -- 172.21.15.131:0/2584859379 shutdown_connections 2023-12-26T21:37:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.258+0000 7fdd00fff700 1 -- 172.21.15.131:0/2584859379 wait complete. 2023-12-26T21:37:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.259+0000 7fdd00fff700 1 Processor -- start 2023-12-26T21:37:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.259+0000 7fdd00fff700 1 -- start start 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.260+0000 7fdd00fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc069d10 0x7fdcfc145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.260+0000 7fdd00fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcfc0c8820 0x7fdcfc0c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.260+0000 7fdd00fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0c8bf0 0x7fdcfc0c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.260+0000 7fdd00fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0c4a00 con 0x7fdcfc069d10 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.260+0000 7fdd00fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0c1c20 con 0x7fdcfc0c8820 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.260+0000 7fdd00fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc155320 con 0x7fdcfc0c8bf0 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.260+0000 7fdcfa59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc069d10 0x7fdcfc145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcfa59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc069d10 0x7fdcfc145db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55848/0 (socket says 172.21.15.131:55848) 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcfa59c700 1 -- 172.21.15.131:0/1259477605 learned_addr learned my addr 172.21.15.131:0/1259477605 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcf3fff700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcfc0c8820 0x7fdcfc0c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcfad9d700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0c8bf0 0x7fdcfc0c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcfa59c700 1 -- 172.21.15.131:0/1259477605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0c8bf0 msgr2=0x7fdcfc0c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcfa59c700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0c8bf0 0x7fdcfc0c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcfa59c700 1 -- 172.21.15.131:0/1259477605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcfc0c8820 msgr2=0x7fdcfc0c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcfa59c700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcfc0c8820 0x7fdcfc0c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcfa59c700 1 -- 172.21.15.131:0/1259477605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40094f0 con 0x7fdcfc069d10 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcfa59c700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc069d10 0x7fdcfc145db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00c590 tx=0x7fdcec00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcf27fc700 1 -- 172.21.15.131:0/1259477605 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec011590 con 0x7fdcfc069d10 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcf27fc700 1 -- 172.21.15.131:0/1259477605 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec011710 con 0x7fdcfc069d10 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdcf27fc700 1 -- 172.21.15.131:0/1259477605 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec011ad0 con 0x7fdcfc069d10 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.261+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc0c15d0 con 0x7fdcfc069d10 2023-12-26T21:37:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.262+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc154560 con 0x7fdcfc069d10 2023-12-26T21:37:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.263+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc0c1f00 con 0x7fdcfc069d10 2023-12-26T21:37:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.263+0000 7fdcf27fc700 1 -- 172.21.15.131:0/1259477605 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdcec01b3b0 con 0x7fdcfc069d10 2023-12-26T21:37:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.264+0000 7fdcf27fc700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdcdc066f30 0x7fdcdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.264+0000 7fdcf27fc700 1 -- 172.21.15.131:0/1259477605 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdcec083860 con 0x7fdcfc069d10 2023-12-26T21:37:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.265+0000 7fdcf3fff700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdcdc066f30 0x7fdcdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.265+0000 7fdcf3fff700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdcdc066f30 0x7fdcdc0693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdce40099b0 tx=0x7fdce4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.268+0000 7fdcf27fc700 1 -- 172.21.15.131:0/1259477605 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcec050ad0 con 0x7fdcfc069d10 2023-12-26T21:37:25.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.461+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdcfc0c1430 con 0x7fdcfc069d10 2023-12-26T21:37:25.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.465+0000 7fdcf27fc700 1 -- 172.21.15.131:0/1259477605 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdcfc0c1430 con 0x7fdcfc069d10 2023-12-26T21:37:25.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdcdc066f30 msgr2=0x7fdcdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:25.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdcdc066f30 0x7fdcdc0693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdce40099b0 tx=0x7fdce4017000 comp rx=0 tx=0).stop 2023-12-26T21:37:25.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc069d10 msgr2=0x7fdcfc145db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:25.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc069d10 0x7fdcfc145db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00c590 tx=0x7fdcec00c8a0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 shutdown_connections 2023-12-26T21:37:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdcdc066f30 0x7fdcdc0693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0c8bf0 0x7fdcfc0c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcfc069d10 0x7fdcfc145db0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 --2- 172.21.15.131:0/1259477605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcfc0c8820 0x7fdcfc0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 >> 172.21.15.131:0/1259477605 conn(0x7fdcfc068040 msgr2=0x7fdcfc0ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 shutdown_connections 2023-12-26T21:37:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:25.467+0000 7fdd00fff700 1 -- 172.21.15.131:0/1259477605 wait complete. 2023-12-26T21:37:25.486 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:37:25.486 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:37:25.579 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:37:25.682 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:37:25.683 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T21:37:25.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:37:25.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-12-26T21:37:25.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:37:25.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:25.757 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T21:37:25.815 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-12-26T21:37:25.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:37:25.815 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-12-26T21:37:25.892 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T21:37:25.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:25.892 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-26T21:37:26.132 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-12-26T21:37:26.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:37:26.132 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-12-26T21:37:26.205 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:37:26.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:26.206 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-26T21:37:26.250 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-12-26T21:37:26.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:37:26.250 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-12-26T21:37:26.322 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T21:37:26.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:26.323 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-26T21:37:26.367 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T21:37:26.367 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:37:26.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.556+0000 7f08879ff700 1 Processor -- start 2023-12-26T21:37:26.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.556+0000 7f08879ff700 1 -- start start 2023-12-26T21:37:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.557+0000 7f08879ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08800b0750 0x7f08800b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.557+0000 7f08879ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08800c9400 0x7f08800c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.557+0000 7f08879ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0880156940 con 0x7f08800cb320 2023-12-26T21:37:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.557+0000 7f08879ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08800c8cc0 con 0x7f08800c9400 2023-12-26T21:37:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.557+0000 7f08879ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08800ad700 con 0x7f08800b0750 2023-12-26T21:37:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f088579b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f08800cb320 0x7f0880069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39006/0 (socket says 172.21.15.131:39006) 2023-12-26T21:37:26.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f088579b700 1 -- 172.21.15.131:0/219122276 learned_addr learned my addr 172.21.15.131:0/219122276 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0885f9c700 1 --2- 172.21.15.131:0/219122276 >> v2:172.21.15.190:3300/0 conn(0x7f08800c9400 0x7f08800c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0884f9a700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08800b0750 0x7f08800b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286103455 0 0) 0x7f0880156940 con 0x7f08800cb320 2023-12-26T21:37:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f086c003570 con 0x7f08800cb320 2023-12-26T21:37:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 729154999 0 0) 0x7f086c003570 con 0x7f08800cb320 2023-12-26T21:37:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08800b0750 msgr2=0x7f08800b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08800b0750 0x7f08800b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 >> v2:172.21.15.190:3300/0 conn(0x7f08800c9400 msgr2=0x7f08800c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:26.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 --2- 172.21.15.131:0/219122276 >> v2:172.21.15.190:3300/0 conn(0x7f08800c9400 0x7f08800c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800b1100 con 0x7f08800cb320 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f087c001470 con 0x7f08800cb320 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 >> v1:172.21.15.131:6789/0 conn(0x7f08800cb320 legacy=0x7f0880069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08800b0750 0x7f086c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c003fe0 0x7f086c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f086c004d50 0x7f086c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c8cc0 con 0x7f086c003fe0 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.558+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c00ba60 con 0x7f086c004d50 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c00dde0 con 0x7f08800b0750 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0885f9c700 1 --2- 172.21.15.131:0/219122276 >> v2:172.21.15.190:3300/0 conn(0x7f08800c9400 0x7f08800c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0884f9a700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c003fe0 0x7f086c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f088579b700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08800b0750 0x7f086c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0885f9c700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f086c004d50 0x7f086c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:26.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0884f9a700 1 -- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08800b0750 msgr2=0x7f086c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0884f9a700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08800b0750 0x7f086c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0884f9a700 1 -- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f086c004d50 msgr2=0x7f086c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0884f9a700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f086c004d50 0x7f086c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0884f9a700 1 -- 172.21.15.131:0/219122276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c003570 con 0x7f086c003fe0 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0885f9c700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f086c004d50 0x7f086c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.559+0000 7f0884f9a700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c003fe0 0x7f086c004810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0870009c00 tx=0x7f08700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8a430a5448912c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000e6b0 con 0x7f086c003fe0 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000e810 con 0x7f086c003fe0 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f0877fff700 1 -- 172.21.15.131:0/219122276 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000ebd0 con 0x7f086c003fe0 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f08879ff700 1 -- 172.21.15.131:0/219122276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c003fe0 msgr2=0x7f086c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f08879ff700 1 --2- 172.21.15.131:0/219122276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c003fe0 0x7f086c004810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0870009c00 tx=0x7f08700091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f0885f9c700 1 -- 172.21.15.131:0/219122276 reap_dead start 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f08879ff700 1 -- 172.21.15.131:0/219122276 shutdown_connections 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f08879ff700 1 -- 172.21.15.131:0/219122276 >> 172.21.15.131:0/219122276 conn(0x7f0880068040 msgr2=0x7f0880069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:26.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f08879ff700 1 -- 172.21.15.131:0/219122276 shutdown_connections 2023-12-26T21:37:26.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.560+0000 7f08879ff700 1 -- 172.21.15.131:0/219122276 wait complete. 2023-12-26T21:37:26.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.562+0000 7f08879ff700 1 Processor -- start 2023-12-26T21:37:26.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.563+0000 7f08879ff700 1 -- start start 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f08879ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f086c003fe0 0x7f08801546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f08879ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08800a83b0 0x7f0880154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f08879ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0880165540 0x7f08801659c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f08879ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0880144560 con 0x7f08800a83b0 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f08879ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0880158da0 con 0x7f0880165540 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f08879ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0880165f00 con 0x7f086c003fe0 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f0884f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08800a83b0 0x7f0880154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f0884f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08800a83b0 0x7f0880154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55864/0 (socket says 172.21.15.131:55864) 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f0884f9a700 1 -- 172.21.15.131:0/384533558 learned_addr learned my addr 172.21.15.131:0/384533558 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f0885f9c700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0880165540 0x7f08801659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f088579b700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f086c003fe0 0x7f08801546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f0884f9a700 1 -- 172.21.15.131:0/384533558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f086c003fe0 msgr2=0x7f08801546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f0884f9a700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f086c003fe0 0x7f08801546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f0884f9a700 1 -- 172.21.15.131:0/384533558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0880165540 msgr2=0x7f08801659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f0884f9a700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0880165540 0x7f08801659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.564+0000 7f0884f9a700 1 -- 172.21.15.131:0/384533558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0880158190 con 0x7f08800a83b0 2023-12-26T21:37:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.565+0000 7f0884f9a700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08800a83b0 0x7f0880154c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f087000e4b0 tx=0x7f087001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.565+0000 7f08757fa700 1 -- 172.21.15.131:0/384533558 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870022070 con 0x7f08800a83b0 2023-12-26T21:37:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.565+0000 7f08757fa700 1 -- 172.21.15.131:0/384533558 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000e980 con 0x7f08800a83b0 2023-12-26T21:37:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.565+0000 7f08757fa700 1 -- 172.21.15.131:0/384533558 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870027470 con 0x7f08800a83b0 2023-12-26T21:37:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.565+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0880158470 con 0x7f08800a83b0 2023-12-26T21:37:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.565+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0880158a30 con 0x7f08800a83b0 2023-12-26T21:37:26.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.566+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0880158f50 con 0x7f08800a83b0 2023-12-26T21:37:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.567+0000 7f08757fa700 1 -- 172.21.15.131:0/384533558 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f08700278c0 con 0x7f08800a83b0 2023-12-26T21:37:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.568+0000 7f08757fa700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0864066f30 0x7f08640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.568+0000 7f08757fa700 1 -- 172.21.15.131:0/384533558 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0870085f50 con 0x7f08800a83b0 2023-12-26T21:37:26.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.568+0000 7f088579b700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0864066f30 0x7f08640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:26.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.569+0000 7f088579b700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0864066f30 0x7f08640693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f087c001200 tx=0x7f08800c8b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:26.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.572+0000 7f08757fa700 1 -- 172.21.15.131:0/384533558 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0880158f50 con 0x7f08800a83b0 2023-12-26T21:37:26.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.806+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0880144560 con 0x7f08800a83b0 2023-12-26T21:37:26.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.806+0000 7f08757fa700 1 -- 172.21.15.131:0/384533558 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f0870017090 con 0x7f08800a83b0 2023-12-26T21:37:26.807 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:26.807 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1127181197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1127181197},{"type":"v1","addr":"172.21.15.190:6835","nonce":1127181197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1425514628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1425514628},{"type":"v1","addr":"172.21.15.131:6837","nonce":1425514628}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3223324183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3223324183},{"type":"v1","addr":"172.21.15.190:6833","nonce":3223324183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:36:48.288910+0000","modified":"2023-12-26T21:36:51.988691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3852718531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3852718531},{"type":"v1","addr":"172.21.15.131:6836","nonce":3852718531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-12-26T21:37:26.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0864066f30 msgr2=0x7f08640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:26.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0864066f30 0x7f08640693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f087c001200 tx=0x7f08800c8b50 comp rx=0 tx=0).stop 2023-12-26T21:37:26.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08800a83b0 msgr2=0x7f0880154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08800a83b0 0x7f0880154c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f087000e4b0 tx=0x7f087001a000 comp rx=0 tx=0).stop 2023-12-26T21:37:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 shutdown_connections 2023-12-26T21:37:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0880165540 0x7f08801659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0864066f30 0x7f08640693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f086c003fe0 0x7f08801546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 --2- 172.21.15.131:0/384533558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08800a83b0 0x7f0880154c20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.808+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 >> 172.21.15.131:0/384533558 conn(0x7f0880068040 msgr2=0x7f08800ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.809+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 shutdown_connections 2023-12-26T21:37:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:26.809+0000 7f08879ff700 1 -- 172.21.15.131:0/384533558 wait complete. 2023-12-26T21:37:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2023-12-26T21:37:26.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:27.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.017+0000 7fb5049ff700 1 Processor -- start 2023-12-26T21:37:27.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.017+0000 7fb5049ff700 1 -- start start 2023-12-26T21:37:27.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.019+0000 7fb5049ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb500159600 0x7fb5000ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.019+0000 7fb5049ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5000cfd50 con 0x7fb5000d01e0 2023-12-26T21:37:27.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.019+0000 7fb5049ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5000b1280 con 0x7fb5001599d0 2023-12-26T21:37:27.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.019+0000 7fb5049ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5000cf140 con 0x7fb500159600 2023-12-26T21:37:27.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.019+0000 7fb4fe59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb500159600 0x7fb5000ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.019+0000 7fb4fed9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb5000d01e0 0x7fb5000ccf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39018/0 (socket says 172.21.15.131:39018) 2023-12-26T21:37:27.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.019+0000 7fb4fed9d700 1 -- 172.21.15.131:0/3337686577 learned_addr learned my addr 172.21.15.131:0/3337686577 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:27.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.020+0000 7fb4fe59c700 1 -- 172.21.15.131:0/3337686577 >> v1:172.21.15.190:6789/0 conn(0x7fb5001599d0 legacy=0x7fb50015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.020+0000 7fb4fe59c700 1 -- 172.21.15.131:0/3337686577 >> v1:172.21.15.131:6789/0 conn(0x7fb5000d01e0 legacy=0x7fb5000ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.020+0000 7fb4fe59c700 1 -- 172.21.15.131:0/3337686577 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5000cf420 con 0x7fb500159600 2023-12-26T21:37:27.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.020+0000 7fb4fd59a700 1 -- 172.21.15.131:0/3337686577 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302095333 0 0) 0x7fb5000b1280 con 0x7fb5001599d0 2023-12-26T21:37:27.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.020+0000 7fb4fe59c700 1 --2- 172.21.15.131:0/3337686577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb500159600 0x7fb5000ceb60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00098d0 tx=0x7fb4f0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bfd25f881d0869 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.020+0000 7fb4fd59a700 1 -- 172.21.15.131:0/3337686577 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0009e10 con 0x7fb500159600 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.020+0000 7fb4fd59a700 1 -- 172.21.15.131:0/3337686577 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f000b760 con 0x7fb500159600 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.020+0000 7fb4fd59a700 1 -- 172.21.15.131:0/3337686577 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000bb00 con 0x7fb500159600 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.021+0000 7fb5049ff700 1 -- 172.21.15.131:0/3337686577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb500159600 msgr2=0x7fb5000ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.021+0000 7fb5049ff700 1 --2- 172.21.15.131:0/3337686577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb500159600 0x7fb5000ceb60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00098d0 tx=0x7fb4f0009120 comp rx=0 tx=0).stop 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.021+0000 7fb5049ff700 1 -- 172.21.15.131:0/3337686577 shutdown_connections 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.021+0000 7fb5049ff700 1 --2- 172.21.15.131:0/3337686577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb500159600 0x7fb5000ceb60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.021+0000 7fb5049ff700 1 -- 172.21.15.131:0/3337686577 >> 172.21.15.131:0/3337686577 conn(0x7fb50006ece0 msgr2=0x7fb50006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.021+0000 7fb5049ff700 1 -- 172.21.15.131:0/3337686577 shutdown_connections 2023-12-26T21:37:27.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.021+0000 7fb5049ff700 1 -- 172.21.15.131:0/3337686577 wait complete. 2023-12-26T21:37:27.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.022+0000 7fb5049ff700 1 Processor -- start 2023-12-26T21:37:27.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.022+0000 7fb5049ff700 1 -- start start 2023-12-26T21:37:27.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.023+0000 7fb5049ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5000d01e0 0x7fb5000b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.023+0000 7fb5049ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb500159600 0x7fb5000b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.023+0000 7fb5049ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000b4830 0x7fb5000b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.023+0000 7fb5049ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5000b3ca0 con 0x7fb5000d01e0 2023-12-26T21:37:27.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.023+0000 7fb5049ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5000b3390 con 0x7fb500159600 2023-12-26T21:37:27.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.023+0000 7fb5049ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5000b5280 con 0x7fb5000b4830 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.023+0000 7fb4fe59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5000d01e0 0x7fb5000b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fe59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5000d01e0 0x7fb5000b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55878/0 (socket says 172.21.15.131:55878) 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fe59c700 1 -- 172.21.15.131:0/2433757422 learned_addr learned my addr 172.21.15.131:0/2433757422 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fdd9b700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb500159600 0x7fb5000b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fe59c700 1 -- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000b4830 msgr2=0x7fb5000b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fed9d700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000b4830 0x7fb5000b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fe59c700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000b4830 0x7fb5000b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fe59c700 1 -- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb500159600 msgr2=0x7fb5000b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fe59c700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb500159600 0x7fb5000b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fe59c700 1 -- 172.21.15.131:0/2433757422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f00093c0 con 0x7fb5000d01e0 2023-12-26T21:37:27.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fdd9b700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb500159600 0x7fb5000b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:27.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fed9d700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000b4830 0x7fb5000b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:27.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4fe59c700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5000d01e0 0x7fb5000b4130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009160 tx=0x7fb4f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:27.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4ee7fc700 1 -- 172.21.15.131:0/2433757422 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000bae0 con 0x7fb5000d01e0 2023-12-26T21:37:27.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4ee7fc700 1 -- 172.21.15.131:0/2433757422 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f001c480 con 0x7fb5000d01e0 2023-12-26T21:37:27.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5000b3650 con 0x7fb5000d01e0 2023-12-26T21:37:27.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.024+0000 7fb4ee7fc700 1 -- 172.21.15.131:0/2433757422 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f001c840 con 0x7fb5000d01e0 2023-12-26T21:37:27.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.025+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb500156460 con 0x7fb5000d01e0 2023-12-26T21:37:27.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.026+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb500088480 con 0x7fb5000d01e0 2023-12-26T21:37:27.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.026+0000 7fb4ee7fc700 1 -- 172.21.15.131:0/2433757422 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb4f001cbf0 con 0x7fb5000d01e0 2023-12-26T21:37:27.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.027+0000 7fb4ee7fc700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4e0066f30 0x7fb4e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.027+0000 7fb4ee7fc700 1 -- 172.21.15.131:0/2433757422 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb4f0086780 con 0x7fb5000d01e0 2023-12-26T21:37:27.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.028+0000 7fb4fdd9b700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4e0066f30 0x7fb4e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.028+0000 7fb4fdd9b700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4e0066f30 0x7fb4e00693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb5000cab50 tx=0x7fb4f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:27.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.032+0000 7fb4ee7fc700 1 -- 172.21.15.131:0/2433757422 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4f0053950 con 0x7fb5000d01e0 2023-12-26T21:37:27.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.231+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5000b3f80 con 0x7fb5000d01e0 2023-12-26T21:37:27.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.232+0000 7fb4ee7fc700 1 -- 172.21.15.131:0/2433757422 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fb4f0056f70 con 0x7fb5000d01e0 2023-12-26T21:37:27.232 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:27.232 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4e0066f30 msgr2=0x7fb4e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4e0066f30 0x7fb4e00693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb5000cab50 tx=0x7fb4f4009040 comp rx=0 tx=0).stop 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5000d01e0 msgr2=0x7fb5000b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5000d01e0 0x7fb5000b4130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009160 tx=0x7fb4f0016000 comp rx=0 tx=0).stop 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 shutdown_connections 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4e0066f30 0x7fb4e00693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000b4830 0x7fb5000b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5000d01e0 0x7fb5000b4130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 --2- 172.21.15.131:0/2433757422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb500159600 0x7fb5000b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 >> 172.21.15.131:0/2433757422 conn(0x7fb50006ece0 msgr2=0x7fb5000b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 shutdown_connections 2023-12-26T21:37:27.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.234+0000 7fb5049ff700 1 -- 172.21.15.131:0/2433757422 wait complete. 2023-12-26T21:37:27.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:36:46.654972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:36:47.264880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:37:27.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:37:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.444+0000 7f801c1ff700 1 Processor -- start 2023-12-26T21:37:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.445+0000 7f801c1ff700 1 -- start start 2023-12-26T21:37:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.446+0000 7f801c1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801413c510 0x7f80140c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.446+0000 7f801c1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80140cc810 0x7f801406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.446+0000 7f801c1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801406c270 con 0x7f801406b8c0 2023-12-26T21:37:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.446+0000 7f801c1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80140686c0 con 0x7f80140cc810 2023-12-26T21:37:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.446+0000 7f801c1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8014069520 con 0x7f801413c510 2023-12-26T21:37:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.446+0000 7f801979a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f801406b8c0 0x7f80140cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39034/0 (socket says 172.21.15.131:39034) 2023-12-26T21:37:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.446+0000 7f801979a700 1 -- 172.21.15.131:0/1177739557 learned_addr learned my addr 172.21.15.131:0/1177739557 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.446+0000 7f801a79c700 1 --2- 172.21.15.131:0/1177739557 >> v2:172.21.15.131:3300/0 conn(0x7f80140cc810 0x7f801406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.446+0000 7f8019f9b700 1 --2- 172.21.15.131:0/1177739557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801413c510 0x7f80140c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f8018f99700 1 -- 172.21.15.131:0/1177739557 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157814362 0 0) 0x7f801406c270 con 0x7f801406b8c0 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f8018f99700 1 -- 172.21.15.131:0/1177739557 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8000003570 con 0x7f801406b8c0 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f801a79c700 1 -- 172.21.15.131:0/1177739557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801413c510 msgr2=0x7f80140c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f801a79c700 1 --2- 172.21.15.131:0/1177739557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801413c510 0x7f80140c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f801a79c700 1 -- 172.21.15.131:0/1177739557 >> v1:172.21.15.131:6789/0 conn(0x7f801406b8c0 legacy=0x7f80140cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f801a79c700 1 -- 172.21.15.131:0/1177739557 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8014068910 con 0x7f80140cc810 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f801a79c700 1 --2- 172.21.15.131:0/1177739557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80140cc810 0x7f801406bd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f800c009e60 tx=0x7f800c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f52fb7084cc0bf server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f8018f99700 1 -- 172.21.15.131:0/1177739557 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c00e070 con 0x7f80140cc810 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f8018f99700 1 -- 172.21.15.131:0/1177739557 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c00bab0 con 0x7f80140cc810 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.447+0000 7f8018f99700 1 -- 172.21.15.131:0/1177739557 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c00be70 con 0x7f80140cc810 2023-12-26T21:37:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.448+0000 7f801c1ff700 1 -- 172.21.15.131:0/1177739557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80140cc810 msgr2=0x7f801406bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.448+0000 7f801c1ff700 1 --2- 172.21.15.131:0/1177739557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80140cc810 0x7f801406bd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f800c009e60 tx=0x7f800c009780 comp rx=0 tx=0).stop 2023-12-26T21:37:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.448+0000 7f801c1ff700 1 -- 172.21.15.131:0/1177739557 shutdown_connections 2023-12-26T21:37:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.448+0000 7f801c1ff700 1 --2- 172.21.15.131:0/1177739557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80140cc810 0x7f801406bd30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.448+0000 7f801c1ff700 1 --2- 172.21.15.131:0/1177739557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801413c510 0x7f80140c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.448+0000 7f801c1ff700 1 -- 172.21.15.131:0/1177739557 >> 172.21.15.131:0/1177739557 conn(0x7f80140ce3e0 msgr2=0x7f80140ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.448+0000 7f801c1ff700 1 -- 172.21.15.131:0/1177739557 shutdown_connections 2023-12-26T21:37:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.448+0000 7f801c1ff700 1 -- 172.21.15.131:0/1177739557 wait complete. 2023-12-26T21:37:27.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.449+0000 7f801c1ff700 1 Processor -- start 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.449+0000 7f801c1ff700 1 -- start start 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.450+0000 7f801c1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801406b8c0 0x7f801413bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.450+0000 7f801c1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80140cc810 0x7f80140b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.450+0000 7f801c1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f801413c510 0x7f8014158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.450+0000 7f801c1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801413c330 con 0x7f801413c510 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.450+0000 7f801c1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801416a580 con 0x7f80140cc810 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.450+0000 7f801c1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801416a860 con 0x7f801406b8c0 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801a79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f801413c510 0x7f8014158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801a79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f801413c510 0x7f8014158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55896/0 (socket says 172.21.15.131:55896) 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801a79c700 1 -- 172.21.15.131:0/232760456 learned_addr learned my addr 172.21.15.131:0/232760456 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f8019f9b700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801406b8c0 0x7f801413bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801979a700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80140cc810 0x7f80140b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801a79c700 1 -- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801406b8c0 msgr2=0x7f801413bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801a79c700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801406b8c0 0x7f801413bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801a79c700 1 -- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80140cc810 msgr2=0x7f80140b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801a79c700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80140cc810 0x7f80140b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801a79c700 1 -- 172.21.15.131:0/232760456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c009a20 con 0x7f801413c510 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f8019f9b700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801406b8c0 0x7f801413bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801979a700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80140cc810 0x7f80140b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f801a79c700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f801413c510 0x7f8014158e70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f801406fe40 tx=0x7f800c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.451+0000 7f8009ffb700 1 -- 172.21.15.131:0/232760456 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c027470 con 0x7f801413c510 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.452+0000 7f8009ffb700 1 -- 172.21.15.131:0/232760456 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c0275f0 con 0x7f801413c510 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.452+0000 7f8009ffb700 1 -- 172.21.15.131:0/232760456 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c0279b0 con 0x7f801413c510 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.452+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801416ab40 con 0x7f801413c510 2023-12-26T21:37:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.452+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801416b100 con 0x7f801413c510 2023-12-26T21:37:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.453+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801416b470 con 0x7f801413c510 2023-12-26T21:37:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.454+0000 7f8009ffb700 1 -- 172.21.15.131:0/232760456 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f800c024070 con 0x7f801413c510 2023-12-26T21:37:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.455+0000 7f8009ffb700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7ff8066f30 0x7f7ff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.455+0000 7f8009ffb700 1 -- 172.21.15.131:0/232760456 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f800c086f50 con 0x7f801413c510 2023-12-26T21:37:27.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.455+0000 7f8019f9b700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7ff8066f30 0x7f7ff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.455+0000 7f8019f9b700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7ff8066f30 0x7f7ff80693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8010006ea0 tx=0x7f8010008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:27.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.459+0000 7f8009ffb700 1 -- 172.21.15.131:0/232760456 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f800c054120 con 0x7f801413c510 2023-12-26T21:37:27.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.650+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80140657e0 con 0x7f801413c510 2023-12-26T21:37:27.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.650+0000 7f8009ffb700 1 -- 172.21.15.131:0/232760456 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f800c02c020 con 0x7f801413c510 2023-12-26T21:37:27.651 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:37:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7ff8066f30 msgr2=0x7f7ff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7ff8066f30 0x7f7ff80693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8010006ea0 tx=0x7f8010008040 comp rx=0 tx=0).stop 2023-12-26T21:37:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f801413c510 msgr2=0x7f8014158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f801413c510 0x7f8014158e70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f801406fe40 tx=0x7f800c019000 comp rx=0 tx=0).stop 2023-12-26T21:37:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 shutdown_connections 2023-12-26T21:37:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f801413c510 0x7f8014158e70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7ff8066f30 0x7f7ff80693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f801406b8c0 0x7f801413bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 --2- 172.21.15.131:0/232760456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80140cc810 0x7f80140b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.652+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 >> 172.21.15.131:0/232760456 conn(0x7f80140ce3e0 msgr2=0x7f8014152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.653+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 shutdown_connections 2023-12-26T21:37:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.653+0000 7f801c1ff700 1 -- 172.21.15.131:0/232760456 wait complete. 2023-12-26T21:37:27.670 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:37:27.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.864+0000 7f89697ff700 1 Processor -- start 2023-12-26T21:37:27.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.864+0000 7f89697ff700 1 -- start start 2023-12-26T21:37:27.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.865+0000 7f89697ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f89640cb320 0x7f8964069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.865+0000 7f89697ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89640b0e60 0x7f89640c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.865+0000 7f89697ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8964156940 con 0x7f896406a1e0 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.865+0000 7f89697ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89640c8cc0 con 0x7f89640cb320 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.865+0000 7f89697ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89640ad700 con 0x7f89640b0e60 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.865+0000 7f8962ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f89640cb320 0x7f8964069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.866+0000 7f89627fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f896406a1e0 0x7f89640b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39050/0 (socket says 172.21.15.131:39050) 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.866+0000 7f89627fc700 1 -- 172.21.15.131:0/3035481419 learned_addr learned my addr 172.21.15.131:0/3035481419 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.866+0000 7f89637fe700 1 --2- 172.21.15.131:0/3035481419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89640b0e60 0x7f89640c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.866+0000 7f8962ffd700 1 -- 172.21.15.131:0/3035481419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89640b0e60 msgr2=0x7f89640c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.866+0000 7f8962ffd700 1 --2- 172.21.15.131:0/3035481419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89640b0e60 0x7f89640c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.866+0000 7f8962ffd700 1 -- 172.21.15.131:0/3035481419 >> v1:172.21.15.131:6789/0 conn(0x7f896406a1e0 legacy=0x7f89640b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.866+0000 7f8962ffd700 1 -- 172.21.15.131:0/3035481419 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89640ae500 con 0x7f89640cb320 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.866+0000 7f8961ffb700 1 -- 172.21.15.131:0/3035481419 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441223522 0 0) 0x7f8964156940 con 0x7f896406a1e0 2023-12-26T21:37:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.866+0000 7f8962ffd700 1 --2- 172.21.15.131:0/3035481419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 0x7f8964069ca0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89540098e0 tx=0x7f89540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88055c7d0fe30a33 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:27.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f8961ffb700 1 -- 172.21.15.131:0/3035481419 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895400b640 con 0x7f89640cb320 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f8961ffb700 1 -- 172.21.15.131:0/3035481419 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895400b7a0 con 0x7f89640cb320 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f8961ffb700 1 -- 172.21.15.131:0/3035481419 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895400bb40 con 0x7f89640cb320 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f89697ff700 1 -- 172.21.15.131:0/3035481419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 msgr2=0x7f8964069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f89697ff700 1 --2- 172.21.15.131:0/3035481419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 0x7f8964069ca0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89540098e0 tx=0x7f89540091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f89697ff700 1 -- 172.21.15.131:0/3035481419 shutdown_connections 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f89697ff700 1 --2- 172.21.15.131:0/3035481419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89640b0e60 0x7f89640c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f89697ff700 1 --2- 172.21.15.131:0/3035481419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 0x7f8964069ca0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f89697ff700 1 -- 172.21.15.131:0/3035481419 >> 172.21.15.131:0/3035481419 conn(0x7f8964068040 msgr2=0x7f8964069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f89697ff700 1 -- 172.21.15.131:0/3035481419 shutdown_connections 2023-12-26T21:37:27.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.867+0000 7f89697ff700 1 -- 172.21.15.131:0/3035481419 wait complete. 2023-12-26T21:37:27.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.869+0000 7f89697ff700 1 Processor -- start 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.869+0000 7f89697ff700 1 -- start start 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89697ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f896406a1e0 0x7f896406e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89697ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89640b0e60 0x7f896414fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89697ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 0x7f8964169500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89697ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896406e500 con 0x7f89640cb320 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89697ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896406d8f0 con 0x7f89640b0e60 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89697ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896406dbd0 con 0x7f896406a1e0 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89637fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 0x7f8964169500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89637fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 0x7f8964169500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55916/0 (socket says 172.21.15.131:55916) 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89637fe700 1 -- 172.21.15.131:0/2907082310 learned_addr learned my addr 172.21.15.131:0/2907082310 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f8962ffd700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f896406a1e0 0x7f896406e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89627fc700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89640b0e60 0x7f896414fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89637fe700 1 -- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f896406a1e0 msgr2=0x7f896406e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89637fe700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f896406a1e0 0x7f896406e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89637fe700 1 -- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89640b0e60 msgr2=0x7f896414fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89637fe700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89640b0e60 0x7f896414fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89637fe700 1 -- 172.21.15.131:0/2907082310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954009420 con 0x7f89640cb320 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f89627fc700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89640b0e60 0x7f896414fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.870+0000 7f8962ffd700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f896406a1e0 0x7f896406e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.871+0000 7f89637fe700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 0x7f8964169500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f895c009e90 tx=0x7f895c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.871+0000 7f894affd700 1 -- 172.21.15.131:0/2907082310 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c00cc20 con 0x7f89640cb320 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.871+0000 7f894affd700 1 -- 172.21.15.131:0/2907082310 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895c00f040 con 0x7f89640cb320 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.871+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896406deb0 con 0x7f89640cb320 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.871+0000 7f894affd700 1 -- 172.21.15.131:0/2907082310 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c014480 con 0x7f89640cb320 2023-12-26T21:37:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.871+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896406eed0 con 0x7f89640cb320 2023-12-26T21:37:27.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.872+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89640657e0 con 0x7f89640cb320 2023-12-26T21:37:27.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.873+0000 7f894affd700 1 -- 172.21.15.131:0/2907082310 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f895c0147f0 con 0x7f89640cb320 2023-12-26T21:37:27.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.874+0000 7f894affd700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8950066f30 0x7f89500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:27.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.874+0000 7f894affd700 1 -- 172.21.15.131:0/2907082310 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f895c085380 con 0x7f89640cb320 2023-12-26T21:37:27.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.874+0000 7f8962ffd700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8950066f30 0x7f89500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:27.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.875+0000 7f8962ffd700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8950066f30 0x7f89500693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89540091e0 tx=0x7f8954017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:27.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:27.878+0000 7f894affd700 1 -- 172.21.15.131:0/2907082310 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f895c055fe0 con 0x7f89640cb320 2023-12-26T21:37:28.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.072+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89641592c0 con 0x7f89640cb320 2023-12-26T21:37:28.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.073+0000 7f894affd700 1 -- 172.21.15.131:0/2907082310 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f895c055b70 con 0x7f89640cb320 2023-12-26T21:37:28.073 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8950066f30 msgr2=0x7f89500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8950066f30 0x7f89500693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89540091e0 tx=0x7f8954017000 comp rx=0 tx=0).stop 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 msgr2=0x7f8964169500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 0x7f8964169500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f895c009e90 tx=0x7f895c00b040 comp rx=0 tx=0).stop 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 shutdown_connections 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89640cb320 0x7f8964169500 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8950066f30 0x7f89500693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f896406a1e0 0x7f896406e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 --2- 172.21.15.131:0/2907082310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89640b0e60 0x7f896414fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 >> 172.21.15.131:0/2907082310 conn(0x7f8964068040 msgr2=0x7f8964158200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 shutdown_connections 2023-12-26T21:37:28.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.075+0000 7f89697ff700 1 -- 172.21.15.131:0/2907082310 wait complete. 2023-12-26T21:37:28.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:37:28.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.285+0000 7fe7e99ff700 1 Processor -- start 2023-12-26T21:37:28.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.285+0000 7fe7e99ff700 1 -- start start 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e99ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7e41596a0 0x7fe7e40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e99ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7e4159a70 0x7fe7e40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e99ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 0x7fe7e40b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e99ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e40afcd0 con 0x7fe7e41596a0 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e99ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e40b3e70 con 0x7fe7e4159a70 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e99ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e40b2e00 con 0x7fe7e40b0160 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7e41596a0 0x7fe7e40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7e41596a0 0x7fe7e40b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55926/0 (socket says 172.21.15.131:55926) 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e2ffd700 1 -- 172.21.15.131:0/2892144914 learned_addr learned my addr 172.21.15.131:0/2892144914 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e27fc700 1 --2- 172.21.15.131:0/2892144914 >> v2:172.21.15.190:3300/0 conn(0x7fe7e4159a70 0x7fe7e40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.286+0000 7fe7e37fe700 1 --2- 172.21.15.131:0/2892144914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 0x7fe7e40b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.287+0000 7fe7e2ffd700 1 -- 172.21.15.131:0/2892144914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 msgr2=0x7fe7e40b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.287+0000 7fe7e2ffd700 1 --2- 172.21.15.131:0/2892144914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 0x7fe7e40b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.287+0000 7fe7e2ffd700 1 -- 172.21.15.131:0/2892144914 >> v2:172.21.15.190:3300/0 conn(0x7fe7e4159a70 msgr2=0x7fe7e40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.287+0000 7fe7e2ffd700 1 --2- 172.21.15.131:0/2892144914 >> v2:172.21.15.190:3300/0 conn(0x7fe7e4159a70 0x7fe7e40b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.287+0000 7fe7e2ffd700 1 -- 172.21.15.131:0/2892144914 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e40b0f20 con 0x7fe7e41596a0 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.287+0000 7fe7e2ffd700 1 --2- 172.21.15.131:0/2892144914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e41596a0 0x7fe7e40b2380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40642f0 tx=0x7fe7d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56ccf1cff7372da server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.287+0000 7fe7e1ffb700 1 -- 172.21.15.131:0/2892144914 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4009e70 con 0x7fe7e41596a0 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.287+0000 7fe7e1ffb700 1 -- 172.21.15.131:0/2892144914 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d400b860 con 0x7fe7e41596a0 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.287+0000 7fe7e1ffb700 1 -- 172.21.15.131:0/2892144914 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d400bc00 con 0x7fe7e41596a0 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.288+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2892144914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e41596a0 msgr2=0x7fe7e40b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.288+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2892144914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e41596a0 0x7fe7e40b2380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40642f0 tx=0x7fe7d4009250 comp rx=0 tx=0).stop 2023-12-26T21:37:28.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.288+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2892144914 shutdown_connections 2023-12-26T21:37:28.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.288+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2892144914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e41596a0 0x7fe7e40b2380 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.288+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2892144914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 0x7fe7e40b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.288+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2892144914 >> v2:172.21.15.190:3300/0 conn(0x7fe7e4159a70 0x7fe7e40b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.288+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2892144914 >> 172.21.15.131:0/2892144914 conn(0x7fe7e406ece0 msgr2=0x7fe7e406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:28.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.288+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2892144914 shutdown_connections 2023-12-26T21:37:28.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.288+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2892144914 wait complete. 2023-12-26T21:37:28.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.290+0000 7fe7e99ff700 1 Processor -- start 2023-12-26T21:37:28.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.290+0000 7fe7e99ff700 1 -- start start 2023-12-26T21:37:28.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.291+0000 7fe7e99ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 0x7fe7e40c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.291+0000 7fe7e99ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e41596a0 0x7fe7e4158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.291+0000 7fe7e99ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e4159a70 0x7fe7e415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.291+0000 7fe7e99ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e4158580 con 0x7fe7e4159a70 2023-12-26T21:37:28.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.291+0000 7fe7e99ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e415bf60 con 0x7fe7e41596a0 2023-12-26T21:37:28.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.291+0000 7fe7e99ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e415c240 con 0x7fe7e40b0160 2023-12-26T21:37:28.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e4159a70 0x7fe7e415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e2ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 0x7fe7e40c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e4159a70 0x7fe7e415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55936/0 (socket says 172.21.15.131:55936) 2023-12-26T21:37:28.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e27fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e41596a0 0x7fe7e4158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e2ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 0x7fe7e40c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:35492/0 (socket says 172.21.15.131:35492) 2023-12-26T21:37:28.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e37fe700 1 -- 172.21.15.131:0/2145813248 learned_addr learned my addr 172.21.15.131:0/2145813248 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:28.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e37fe700 1 -- 172.21.15.131:0/2145813248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 msgr2=0x7fe7e40c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e37fe700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 0x7fe7e40c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e37fe700 1 -- 172.21.15.131:0/2145813248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e41596a0 msgr2=0x7fe7e4158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e37fe700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e41596a0 0x7fe7e4158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e37fe700 1 -- 172.21.15.131:0/2145813248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d40094f0 con 0x7fe7e4159a70 2023-12-26T21:37:28.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e37fe700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e4159a70 0x7fe7e415b730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009fa0 tx=0x7fe7dc009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:28.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7caffd700 1 -- 172.21.15.131:0/2145813248 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00ebc0 con 0x7fe7e4159a70 2023-12-26T21:37:28.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7caffd700 1 -- 172.21.15.131:0/2145813248 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc00ed20 con 0x7fe7e4159a70 2023-12-26T21:37:28.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.292+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e415c520 con 0x7fe7e4159a70 2023-12-26T21:37:28.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.293+0000 7fe7caffd700 1 -- 172.21.15.131:0/2145813248 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc014670 con 0x7fe7e4159a70 2023-12-26T21:37:28.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.293+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e415cae0 con 0x7fe7e4159a70 2023-12-26T21:37:28.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.294+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e40657e0 con 0x7fe7e4159a70 2023-12-26T21:37:28.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.294+0000 7fe7caffd700 1 -- 172.21.15.131:0/2145813248 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe7dc0149e0 con 0x7fe7e4159a70 2023-12-26T21:37:28.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.295+0000 7fe7caffd700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7d0066f30 0x7fe7d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.295+0000 7fe7caffd700 1 -- 172.21.15.131:0/2145813248 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe7dc084c70 con 0x7fe7e4159a70 2023-12-26T21:37:28.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.296+0000 7fe7e2ffd700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7d0066f30 0x7fe7d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.296+0000 7fe7e2ffd700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7d0066f30 0x7fe7d00693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40642f0 tx=0x7fe7d4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:28.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.299+0000 7fe7caffd700 1 -- 172.21.15.131:0/2145813248 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7dc051e40 con 0x7fe7e4159a70 2023-12-26T21:37:28.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.492+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7e40b3540 con 0x7fe7e4159a70 2023-12-26T21:37:28.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.493+0000 7fe7caffd700 1 -- 172.21.15.131:0/2145813248 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7dc055460 con 0x7fe7e4159a70 2023-12-26T21:37:28.494 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:37:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7d0066f30 msgr2=0x7fe7d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7d0066f30 0x7fe7d00693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40642f0 tx=0x7fe7d4017000 comp rx=0 tx=0).stop 2023-12-26T21:37:28.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e4159a70 msgr2=0x7fe7e415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e4159a70 0x7fe7e415b730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009fa0 tx=0x7fe7dc009900 comp rx=0 tx=0).stop 2023-12-26T21:37:28.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 shutdown_connections 2023-12-26T21:37:28.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e4159a70 0x7fe7e415b730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7d0066f30 0x7fe7d00693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e40b0160 0x7fe7e40c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 --2- 172.21.15.131:0/2145813248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e41596a0 0x7fe7e4158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 >> 172.21.15.131:0/2145813248 conn(0x7fe7e406ece0 msgr2=0x7fe7e40b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:28.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 shutdown_connections 2023-12-26T21:37:28.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.495+0000 7fe7e99ff700 1 -- 172.21.15.131:0/2145813248 wait complete. 2023-12-26T21:37:28.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:37:28.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.702+0000 7fe3273ff700 1 Processor -- start 2023-12-26T21:37:28.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.702+0000 7fe3273ff700 1 -- start start 2023-12-26T21:37:28.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.703+0000 7fe3273ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe320159a70 0x7fe3200b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.703+0000 7fe3273ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 0x7fe3200b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.703+0000 7fe3273ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3200afcd0 con 0x7fe3201596a0 2023-12-26T21:37:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.703+0000 7fe3273ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3200b3e70 con 0x7fe320159a70 2023-12-26T21:37:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.703+0000 7fe3273ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3200b0f20 con 0x7fe3200b2a90 2023-12-26T21:37:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.703+0000 7fe32519b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe3201596a0 0x7fe3200b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39052/0 (socket says 172.21.15.131:39052) 2023-12-26T21:37:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.703+0000 7fe32519b700 1 -- 172.21.15.131:0/3111429267 learned_addr learned my addr 172.21.15.131:0/3111429267 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe32499a700 1 --2- 172.21.15.131:0/3111429267 >> v2:172.21.15.190:3300/0 conn(0x7fe320159a70 0x7fe3200b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe32599c700 1 --2- 172.21.15.131:0/3111429267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 0x7fe3200b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe317fff700 1 -- 172.21.15.131:0/3111429267 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506036581 0 0) 0x7fe3200afcd0 con 0x7fe3201596a0 2023-12-26T21:37:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe317fff700 1 -- 172.21.15.131:0/3111429267 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe30c003570 con 0x7fe3201596a0 2023-12-26T21:37:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe317fff700 1 -- 172.21.15.131:0/3111429267 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1007648483 0 0) 0x7fe30c003570 con 0x7fe3201596a0 2023-12-26T21:37:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe32599c700 1 -- 172.21.15.131:0/3111429267 >> v2:172.21.15.190:3300/0 conn(0x7fe320159a70 msgr2=0x7fe3200b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe32599c700 1 --2- 172.21.15.131:0/3111429267 >> v2:172.21.15.190:3300/0 conn(0x7fe320159a70 0x7fe3200b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe32599c700 1 -- 172.21.15.131:0/3111429267 >> v1:172.21.15.131:6789/0 conn(0x7fe3201596a0 legacy=0x7fe3200b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe32599c700 1 -- 172.21.15.131:0/3111429267 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3200b3540 con 0x7fe3200b2a90 2023-12-26T21:37:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe32499a700 1 --2- 172.21.15.131:0/3111429267 >> v2:172.21.15.190:3300/0 conn(0x7fe320159a70 0x7fe3200b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.704+0000 7fe32599c700 1 --2- 172.21.15.131:0/3111429267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 0x7fe3200b06a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe318009e60 tx=0x7fe318009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757b609f181d0bb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe317fff700 1 -- 172.21.15.131:0/3111429267 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31800e070 con 0x7fe3200b2a90 2023-12-26T21:37:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe317fff700 1 -- 172.21.15.131:0/3111429267 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31800b7a0 con 0x7fe3200b2a90 2023-12-26T21:37:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe317fff700 1 -- 172.21.15.131:0/3111429267 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c001470 con 0x7fe3200b2a90 2023-12-26T21:37:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe3273ff700 1 -- 172.21.15.131:0/3111429267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 msgr2=0x7fe3200b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe3273ff700 1 --2- 172.21.15.131:0/3111429267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 0x7fe3200b06a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe318009e60 tx=0x7fe318009780 comp rx=0 tx=0).stop 2023-12-26T21:37:28.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe3273ff700 1 -- 172.21.15.131:0/3111429267 shutdown_connections 2023-12-26T21:37:28.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe3273ff700 1 --2- 172.21.15.131:0/3111429267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 0x7fe3200b06a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe3273ff700 1 --2- 172.21.15.131:0/3111429267 >> v2:172.21.15.190:3300/0 conn(0x7fe320159a70 0x7fe3200b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe3273ff700 1 -- 172.21.15.131:0/3111429267 >> 172.21.15.131:0/3111429267 conn(0x7fe32006ece0 msgr2=0x7fe32006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:28.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe3273ff700 1 -- 172.21.15.131:0/3111429267 shutdown_connections 2023-12-26T21:37:28.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.705+0000 7fe3273ff700 1 -- 172.21.15.131:0/3111429267 wait complete. 2023-12-26T21:37:28.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.707+0000 7fe3273ff700 1 Processor -- start 2023-12-26T21:37:28.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.707+0000 7fe3273ff700 1 -- start start 2023-12-26T21:37:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe3273ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 0x7fe3201528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe3273ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3201596a0 0x7fe3200c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe3273ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320159a70 0x7fe3200c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe3273ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe320154be0 con 0x7fe3201596a0 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe3273ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe320156cb0 con 0x7fe320159a70 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe3273ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe320152e10 con 0x7fe3200b2a90 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32499a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3201596a0 0x7fe3200c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32499a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3201596a0 0x7fe3200c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55950/0 (socket says 172.21.15.131:55950) 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32499a700 1 -- 172.21.15.131:0/2673529379 learned_addr learned my addr 172.21.15.131:0/2673529379 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32599c700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320159a70 0x7fe3200c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32519b700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 0x7fe3201528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32499a700 1 -- 172.21.15.131:0/2673529379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 msgr2=0x7fe3201528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32499a700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 0x7fe3201528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32499a700 1 -- 172.21.15.131:0/2673529379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320159a70 msgr2=0x7fe3200c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32499a700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320159a70 0x7fe3200c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32499a700 1 -- 172.21.15.131:0/2673529379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310009420 con 0x7fe3201596a0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.708+0000 7fe32499a700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3201596a0 0x7fe3200c9220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe31000ecb0 tx=0x7fe3100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.709+0000 7fe3157fa700 1 -- 172.21.15.131:0/2673529379 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31000c9d0 con 0x7fe3201596a0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.709+0000 7fe3157fa700 1 -- 172.21.15.131:0/2673529379 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31000cb30 con 0x7fe3201596a0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.709+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe318009a20 con 0x7fe3201596a0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.709+0000 7fe3157fa700 1 -- 172.21.15.131:0/2673529379 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31000ceb0 con 0x7fe3201596a0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.709+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe320155110 con 0x7fe3201596a0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.710+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe320156e40 con 0x7fe3201596a0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.711+0000 7fe3157fa700 1 -- 172.21.15.131:0/2673529379 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe310013680 con 0x7fe3201596a0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.712+0000 7fe3157fa700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe304066f30 0x7fe3040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.712+0000 7fe3157fa700 1 -- 172.21.15.131:0/2673529379 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe310084d20 con 0x7fe3201596a0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.712+0000 7fe32519b700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe304066f30 0x7fe3040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.712+0000 7fe32519b700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe304066f30 0x7fe3040693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe32015d000 tx=0x7fe31c001540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:28.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.716+0000 7fe3157fa700 1 -- 172.21.15.131:0/2673529379 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe310055980 con 0x7fe3201596a0 2023-12-26T21:37:28.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.955+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3200657e0 con 0x7fe3201596a0 2023-12-26T21:37:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.956+0000 7fe3157fa700 1 -- 172.21.15.131:0/2673529379 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fe310055510 con 0x7fe3201596a0 2023-12-26T21:37:28.956 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:28.956 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1127181197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1127181197},{"type":"v1","addr":"172.21.15.190:6835","nonce":1127181197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1425514628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1425514628},{"type":"v1","addr":"172.21.15.131:6837","nonce":1425514628}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3223324183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3223324183},{"type":"v1","addr":"172.21.15.190:6833","nonce":3223324183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:36:48.288910+0000","modified":"2023-12-26T21:36:51.988691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3852718531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3852718531},{"type":"v1","addr":"172.21.15.131:6836","nonce":3852718531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-12-26T21:37:28.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe304066f30 msgr2=0x7fe3040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe304066f30 0x7fe3040693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe32015d000 tx=0x7fe31c001540 comp rx=0 tx=0).stop 2023-12-26T21:37:28.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3201596a0 msgr2=0x7fe3200c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:28.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3201596a0 0x7fe3200c9220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe31000ecb0 tx=0x7fe3100091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 shutdown_connections 2023-12-26T21:37:28.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320159a70 0x7fe3200c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe304066f30 0x7fe3040693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3200b2a90 0x7fe3201528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 --2- 172.21.15.131:0/2673529379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3201596a0 0x7fe3200c9220 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:28.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.958+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 >> 172.21.15.131:0/2673529379 conn(0x7fe32006ece0 msgr2=0x7fe3200cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:28.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.959+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 shutdown_connections 2023-12-26T21:37:28.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:28.959+0000 7fe3273ff700 1 -- 172.21.15.131:0/2673529379 wait complete. 2023-12-26T21:37:28.960 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2023-12-26T21:37:28.977 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:29.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.166+0000 7fbaafdff700 1 Processor -- start 2023-12-26T21:37:29.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.166+0000 7fbaafdff700 1 -- start start 2023-12-26T21:37:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.167+0000 7fbaafdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbaa8159820 0x7fbaa80abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.167+0000 7fbaafdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa815a8c0 con 0x7fbaa80aa480 2023-12-26T21:37:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.167+0000 7fbaafdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa8159f90 con 0x7fbaa806e480 2023-12-26T21:37:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.167+0000 7fbaafdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa815ad50 con 0x7fbaa8159820 2023-12-26T21:37:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.168+0000 7fbaadb9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbaa8159820 0x7fbaa80abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.168+0000 7fbaadb9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbaa8159820 0x7fbaa80abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55958/0 (socket says 172.21.15.131:55958) 2023-12-26T21:37:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.168+0000 7fbaadb9b700 1 -- 172.21.15.131:0/574114268 learned_addr learned my addr 172.21.15.131:0/574114268 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.168+0000 7fbaadb9b700 1 -- 172.21.15.131:0/574114268 >> v1:172.21.15.190:6789/0 conn(0x7fbaa806e480 legacy=0x7fbaa80675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.168+0000 7fbaadb9b700 1 -- 172.21.15.131:0/574114268 >> v1:172.21.15.131:6789/0 conn(0x7fbaa80aa480 legacy=0x7fbaa806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.168+0000 7fbaadb9b700 1 -- 172.21.15.131:0/574114268 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa815a270 con 0x7fbaa8159820 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.168+0000 7fbaadb9b700 1 --2- 172.21.15.131:0/574114268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa8159820 0x7fbaa80abdb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40099b0 tx=0x7fbaa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbf3fbb6ce6204f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.168+0000 7fbaacb99700 1 -- 172.21.15.131:0/574114268 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa400b860 con 0x7fbaa8159820 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.169+0000 7fbaacb99700 1 -- 172.21.15.131:0/574114268 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa400b9c0 con 0x7fbaa8159820 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.169+0000 7fbaacb99700 1 -- 172.21.15.131:0/574114268 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa400bd60 con 0x7fbaa8159820 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.169+0000 7fbaafdff700 1 -- 172.21.15.131:0/574114268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa8159820 msgr2=0x7fbaa80abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.169+0000 7fbaafdff700 1 --2- 172.21.15.131:0/574114268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa8159820 0x7fbaa80abdb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40099b0 tx=0x7fbaa4009250 comp rx=0 tx=0).stop 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.169+0000 7fbaafdff700 1 -- 172.21.15.131:0/574114268 shutdown_connections 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.169+0000 7fbaafdff700 1 --2- 172.21.15.131:0/574114268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa8159820 0x7fbaa80abdb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.169+0000 7fbaafdff700 1 -- 172.21.15.131:0/574114268 >> 172.21.15.131:0/574114268 conn(0x7fbaa806c290 msgr2=0x7fbaa80ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.169+0000 7fbaafdff700 1 -- 172.21.15.131:0/574114268 shutdown_connections 2023-12-26T21:37:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.169+0000 7fbaafdff700 1 -- 172.21.15.131:0/574114268 wait complete. 2023-12-26T21:37:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.171+0000 7fbaafdff700 1 Processor -- start 2023-12-26T21:37:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.171+0000 7fbaafdff700 1 -- start start 2023-12-26T21:37:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaafdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa806e480 0x7fbaa80c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaafdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80aa480 0x7fbaa80c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaafdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa8159820 0x7fbaa80ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaafdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa8169ff0 con 0x7fbaa806e480 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaafdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa80c3680 con 0x7fbaa80aa480 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaafdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa80c3960 con 0x7fbaa8159820 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaadb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa806e480 0x7fbaa80c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaadb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa806e480 0x7fbaa80c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55964/0 (socket says 172.21.15.131:55964) 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaadb9b700 1 -- 172.21.15.131:0/2538518280 learned_addr learned my addr 172.21.15.131:0/2538518280 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaad39a700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80aa480 0x7fbaa80c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaae39c700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa8159820 0x7fbaa80ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaadb9b700 1 -- 172.21.15.131:0/2538518280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa8159820 msgr2=0x7fbaa80ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaadb9b700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa8159820 0x7fbaa80ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaadb9b700 1 -- 172.21.15.131:0/2538518280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80aa480 msgr2=0x7fbaa80c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaadb9b700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80aa480 0x7fbaa80c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaadb9b700 1 -- 172.21.15.131:0/2538518280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa40094f0 con 0x7fbaa806e480 2023-12-26T21:37:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaae39c700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa8159820 0x7fbaa80ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.172+0000 7fbaadb9b700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa806e480 0x7fbaa80c2d80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80af700 tx=0x7fbaa4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.173+0000 7fba9dffb700 1 -- 172.21.15.131:0/2538518280 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4011ea0 con 0x7fbaa806e480 2023-12-26T21:37:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.173+0000 7fba9dffb700 1 -- 172.21.15.131:0/2538518280 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa401a630 con 0x7fbaa806e480 2023-12-26T21:37:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.173+0000 7fba9dffb700 1 -- 172.21.15.131:0/2538518280 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa401a9d0 con 0x7fbaa806e480 2023-12-26T21:37:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.173+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa80c3c40 con 0x7fbaa806e480 2023-12-26T21:37:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.173+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa80c32c0 con 0x7fbaa806e480 2023-12-26T21:37:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.174+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa80657e0 con 0x7fbaa806e480 2023-12-26T21:37:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.175+0000 7fba9dffb700 1 -- 172.21.15.131:0/2538518280 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbaa402d440 con 0x7fbaa806e480 2023-12-26T21:37:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.175+0000 7fba9dffb700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba8c066f30 0x7fba8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.176+0000 7fba9dffb700 1 -- 172.21.15.131:0/2538518280 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbaa4086040 con 0x7fbaa806e480 2023-12-26T21:37:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.176+0000 7fbaad39a700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba8c066f30 0x7fba8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.176+0000 7fbaad39a700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba8c066f30 0x7fba8c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80aedd0 tx=0x7fba98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:29.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.180+0000 7fba9dffb700 1 -- 172.21.15.131:0/2538518280 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbaa40532b0 con 0x7fbaa806e480 2023-12-26T21:37:29.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.376+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbaa815a550 con 0x7fbaa806e480 2023-12-26T21:37:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.377+0000 7fba9dffb700 1 -- 172.21.15.131:0/2538518280 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fbaa40568d0 con 0x7fbaa806e480 2023-12-26T21:37:29.378 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:29.378 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:29.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba8c066f30 msgr2=0x7fba8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba8c066f30 0x7fba8c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80aedd0 tx=0x7fba98009040 comp rx=0 tx=0).stop 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa806e480 msgr2=0x7fbaa80c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa806e480 0x7fbaa80c2d80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80af700 tx=0x7fbaa4017000 comp rx=0 tx=0).stop 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 shutdown_connections 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fba8c066f30 0x7fba8c0693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa8159820 0x7fbaa80ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbaa806e480 0x7fbaa80c2d80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 --2- 172.21.15.131:0/2538518280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80aa480 0x7fbaa80c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 >> 172.21.15.131:0/2538518280 conn(0x7fbaa806c290 msgr2=0x7fbaa815bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 shutdown_connections 2023-12-26T21:37:29.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.380+0000 7fbaafdff700 1 -- 172.21.15.131:0/2538518280 wait complete. 2023-12-26T21:37:29.399 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T21:37:29.399 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:37:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.591+0000 7fb6b19ff700 1 Processor -- start 2023-12-26T21:37:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.591+0000 7fb6b19ff700 1 -- start start 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.592+0000 7fb6b19ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6ac1596a0 0x7fb6ac0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.592+0000 7fb6b19ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac159a70 0x7fb6ac0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.592+0000 7fb6b19ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac0afcd0 con 0x7fb6ac0b0160 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.592+0000 7fb6b19ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ac0b3e70 con 0x7fb6ac1596a0 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.592+0000 7fb6b19ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0b0cd0 con 0x7fb6ac159a70 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6aaffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6ac1596a0 0x7fb6ac0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6ab7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb6ac0b0160 0x7fb6ac0b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39064/0 (socket says 172.21.15.131:39064) 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6ab7fe700 1 -- 172.21.15.131:0/4218633752 learned_addr learned my addr 172.21.15.131:0/4218633752 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6aa7fc700 1 --2- 172.21.15.131:0/4218633752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac159a70 0x7fb6ac0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6aaffd700 1 -- 172.21.15.131:0/4218633752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac159a70 msgr2=0x7fb6ac0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6aaffd700 1 --2- 172.21.15.131:0/4218633752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac159a70 0x7fb6ac0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6aaffd700 1 -- 172.21.15.131:0/4218633752 >> v1:172.21.15.131:6789/0 conn(0x7fb6ac0b0160 legacy=0x7fb6ac0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6aaffd700 1 -- 172.21.15.131:0/4218633752 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac0b2e00 con 0x7fb6ac1596a0 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6a9ffb700 1 -- 172.21.15.131:0/4218633752 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444736756 0 0) 0x7fb6ac0afcd0 con 0x7fb6ac0b0160 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6aaffd700 1 --2- 172.21.15.131:0/4218633752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac1596a0 0x7fb6ac0b2380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0098e0 tx=0x7fb69c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc39226a453fc7f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.593+0000 7fb6a9ffb700 1 -- 172.21.15.131:0/4218633752 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c009e80 con 0x7fb6ac1596a0 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6a9ffb700 1 -- 172.21.15.131:0/4218633752 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c00b860 con 0x7fb6ac1596a0 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6a9ffb700 1 -- 172.21.15.131:0/4218633752 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00bbe0 con 0x7fb6ac1596a0 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4218633752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac1596a0 msgr2=0x7fb6ac0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6b19ff700 1 --2- 172.21.15.131:0/4218633752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac1596a0 0x7fb6ac0b2380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0098e0 tx=0x7fb69c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4218633752 shutdown_connections 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6b19ff700 1 --2- 172.21.15.131:0/4218633752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac1596a0 0x7fb6ac0b2380 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6b19ff700 1 --2- 172.21.15.131:0/4218633752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac159a70 0x7fb6ac0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4218633752 >> 172.21.15.131:0/4218633752 conn(0x7fb6ac06ece0 msgr2=0x7fb6ac06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:29.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4218633752 shutdown_connections 2023-12-26T21:37:29.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.594+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4218633752 wait complete. 2023-12-26T21:37:29.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.596+0000 7fb6b19ff700 1 Processor -- start 2023-12-26T21:37:29.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.596+0000 7fb6b19ff700 1 -- start start 2023-12-26T21:37:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6b19ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6ac0b0160 0x7fb6ac156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6b19ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac1596a0 0x7fb6ac15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6b19ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac159a70 0x7fb6ac0c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6b19ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac1572b0 con 0x7fb6ac159a70 2023-12-26T21:37:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6b19ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac15bca0 con 0x7fb6ac0b0160 2023-12-26T21:37:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6b19ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0c5c70 con 0x7fb6ac1596a0 2023-12-26T21:37:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6ab7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac159a70 0x7fb6ac0c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6ab7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac159a70 0x7fb6ac0c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55990/0 (socket says 172.21.15.131:55990) 2023-12-26T21:37:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6ab7fe700 1 -- 172.21.15.131:0/4246441794 learned_addr learned my addr 172.21.15.131:0/4246441794 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6aa7fc700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac1596a0 0x7fb6ac15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6aaffd700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6ac0b0160 0x7fb6ac156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6ab7fe700 1 -- 172.21.15.131:0/4246441794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac1596a0 msgr2=0x7fb6ac15b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6ab7fe700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac1596a0 0x7fb6ac15b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6ab7fe700 1 -- 172.21.15.131:0/4246441794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6ac0b0160 msgr2=0x7fb6ac156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6ab7fe700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6ac0b0160 0x7fb6ac156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.597+0000 7fb6ab7fe700 1 -- 172.21.15.131:0/4246441794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c009420 con 0x7fb6ac159a70 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.598+0000 7fb6ab7fe700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac159a70 0x7fb6ac0c62c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400ccf0 tx=0x7fb6a4001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.598+0000 7fb692ffd700 1 -- 172.21.15.131:0/4246441794 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400a940 con 0x7fb6ac159a70 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.598+0000 7fb692ffd700 1 -- 172.21.15.131:0/4246441794 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a400aaa0 con 0x7fb6ac159a70 2023-12-26T21:37:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.598+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac15b2d0 con 0x7fb6ac159a70 2023-12-26T21:37:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.598+0000 7fb692ffd700 1 -- 172.21.15.131:0/4246441794 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400ae60 con 0x7fb6ac159a70 2023-12-26T21:37:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.598+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac15ac80 con 0x7fb6ac159a70 2023-12-26T21:37:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.599+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac15b460 con 0x7fb6ac159a70 2023-12-26T21:37:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.600+0000 7fb692ffd700 1 -- 172.21.15.131:0/4246441794 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb6a4007300 con 0x7fb6ac159a70 2023-12-26T21:37:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.601+0000 7fb692ffd700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb698066f30 0x7fb6980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.601+0000 7fb692ffd700 1 -- 172.21.15.131:0/4246441794 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6a4084980 con 0x7fb6ac159a70 2023-12-26T21:37:29.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.601+0000 7fb6aaffd700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb698066f30 0x7fb6980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:29.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.601+0000 7fb6aaffd700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb698066f30 0x7fb6980693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0091e0 tx=0x7fb69c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.605+0000 7fb692ffd700 1 -- 172.21.15.131:0/4246441794 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6a4051bd0 con 0x7fb6ac159a70 2023-12-26T21:37:29.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.838+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6ac0657e0 con 0x7fb6ac159a70 2023-12-26T21:37:29.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.839+0000 7fb692ffd700 1 -- 172.21.15.131:0/4246441794 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fb6a40551f0 con 0x7fb6ac159a70 2023-12-26T21:37:29.839 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:29.839 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1127181197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1127181197},{"type":"v1","addr":"172.21.15.190:6835","nonce":1127181197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1425514628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1425514628},{"type":"v1","addr":"172.21.15.131:6837","nonce":1425514628}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3223324183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3223324183},{"type":"v1","addr":"172.21.15.190:6833","nonce":3223324183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:36:48.288910+0000","modified":"2023-12-26T21:36:51.988691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3852718531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3852718531},{"type":"v1","addr":"172.21.15.131:6836","nonce":3852718531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb698066f30 msgr2=0x7fb6980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb698066f30 0x7fb6980693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0091e0 tx=0x7fb69c017000 comp rx=0 tx=0).stop 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac159a70 msgr2=0x7fb6ac0c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac159a70 0x7fb6ac0c62c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400ccf0 tx=0x7fb6a4001a80 comp rx=0 tx=0).stop 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 shutdown_connections 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6ac159a70 0x7fb6ac0c62c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6ac0b0160 0x7fb6ac156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb698066f30 0x7fb6980693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 --2- 172.21.15.131:0/4246441794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6ac1596a0 0x7fb6ac15b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 >> 172.21.15.131:0/4246441794 conn(0x7fb6ac06ece0 msgr2=0x7fb6ac0b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 shutdown_connections 2023-12-26T21:37:29.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:29.841+0000 7fb6b19ff700 1 -- 172.21.15.131:0/4246441794 wait complete. 2023-12-26T21:37:29.843 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2023-12-26T21:37:29.859 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:37:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.050+0000 7f1f54bff700 1 Processor -- start 2023-12-26T21:37:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.050+0000 7f1f54bff700 1 -- start start 2023-12-26T21:37:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.051+0000 7f1f54bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500b0750 0x7f1f500b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.051+0000 7f1f54bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f50156940 con 0x7f1f500cb320 2023-12-26T21:37:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.051+0000 7f1f54bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f500b1100 con 0x7f1f500c9400 2023-12-26T21:37:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.051+0000 7f1f54bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500c8cc0 con 0x7f1f500b0750 2023-12-26T21:37:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.051+0000 7f1f4e59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1f500cb320 0x7f1f50069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39066/0 (socket says 172.21.15.131:39066) 2023-12-26T21:37:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.051+0000 7f1f4e59c700 1 -- 172.21.15.131:0/2589054511 learned_addr learned my addr 172.21.15.131:0/2589054511 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4dd9b700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500b0750 0x7f1f500b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257863773 0 0) 0x7f1f50156940 con 0x7f1f500cb320 2023-12-26T21:37:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f38003570 con 0x7f1f500cb320 2023-12-26T21:37:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581290302 0 0) 0x7f1f500b1100 con 0x7f1f500c9400 2023-12-26T21:37:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f50156940 con 0x7f1f500c9400 2023-12-26T21:37:30.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 841911422 0 0) 0x7f1f38003570 con 0x7f1f500cb320 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500b0750 msgr2=0x7f1f500b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500b0750 0x7f1f500b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 >> v1:172.21.15.190:6789/0 conn(0x7f1f500c9400 legacy=0x7f1f500c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500ae500 con 0x7f1f500cb320 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f1f40001470 con 0x7f1f500cb320 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 >> v1:172.21.15.131:6789/0 conn(0x7f1f500cb320 legacy=0x7f1f50069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f38003c20 0x7f1f38004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f38004810 0x7f1f38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500b0750 0x7f1f38005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500c8cc0 con 0x7f1f38003c20 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3800bbf0 con 0x7f1f38004810 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.052+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3800bdb0 con 0x7f1f500b0750 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4e59c700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f38003c20 0x7f1f38004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4dd9b700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f38004810 0x7f1f38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4ed9d700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500b0750 0x7f1f38005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4e59c700 1 -- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500b0750 msgr2=0x7f1f38005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4e59c700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500b0750 0x7f1f38005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4e59c700 1 -- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f38004810 msgr2=0x7f1f38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4e59c700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f38004810 0x7f1f38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4e59c700 1 -- 172.21.15.131:0/2589054511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f38003570 con 0x7f1f38003c20 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4e59c700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f38003c20 0x7f1f38004010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c210 tx=0x7f1f40001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e0d84189a863a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.053+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4000cc10 con 0x7f1f38003c20 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.054+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4000cd70 con 0x7f1f38003c20 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.054+0000 7f1f4d59a700 1 -- 172.21.15.131:0/2589054511 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40005b70 con 0x7f1f38003c20 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.054+0000 7f1f54bff700 1 -- 172.21.15.131:0/2589054511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f38003c20 msgr2=0x7f1f38004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.054+0000 7f1f54bff700 1 --2- 172.21.15.131:0/2589054511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f38003c20 0x7f1f38004010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c210 tx=0x7f1f40001d40 comp rx=0 tx=0).stop 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.054+0000 7f1f4ed9d700 1 -- 172.21.15.131:0/2589054511 reap_dead start 2023-12-26T21:37:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.054+0000 7f1f54bff700 1 -- 172.21.15.131:0/2589054511 shutdown_connections 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.054+0000 7f1f54bff700 1 -- 172.21.15.131:0/2589054511 >> 172.21.15.131:0/2589054511 conn(0x7f1f50068040 msgr2=0x7f1f50069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.054+0000 7f1f54bff700 1 -- 172.21.15.131:0/2589054511 shutdown_connections 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.054+0000 7f1f54bff700 1 -- 172.21.15.131:0/2589054511 wait complete. 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.056+0000 7f1f54bff700 1 Processor -- start 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.056+0000 7f1f54bff700 1 -- start start 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f54bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f38003c20 0x7f1f50069fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f54bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c6b20 0x7f1f50151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f54bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f501519d0 0x7f1f500ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f54bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f50169940 con 0x7f1f501519d0 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f54bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500c6fb0 con 0x7f1f38003c20 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f54bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500c7290 con 0x7f1f500c6b20 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f501519d0 0x7f1f500ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f501519d0 0x7f1f500ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56012/0 (socket says 172.21.15.131:56012) 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4ed9d700 1 -- 172.21.15.131:0/925624010 learned_addr learned my addr 172.21.15.131:0/925624010 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4e59c700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f38003c20 0x7f1f50069fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4dd9b700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c6b20 0x7f1f50151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4ed9d700 1 -- 172.21.15.131:0/925624010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c6b20 msgr2=0x7f1f50151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4ed9d700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c6b20 0x7f1f50151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4ed9d700 1 -- 172.21.15.131:0/925624010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f38003c20 msgr2=0x7f1f50069fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4ed9d700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f38003c20 0x7f1f50069fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4ed9d700 1 -- 172.21.15.131:0/925624010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500c7570 con 0x7f1f501519d0 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.057+0000 7f1f4dd9b700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c6b20 0x7f1f50151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.058+0000 7f1f4ed9d700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f501519d0 0x7f1f500ca210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1f5006b9f0 tx=0x7f1f480017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:30.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.060+0000 7f1f3e7fc700 1 -- 172.21.15.131:0/925624010 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48006e80 con 0x7f1f501519d0 2023-12-26T21:37:30.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.060+0000 7f1f3e7fc700 1 -- 172.21.15.131:0/925624010 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4800c540 con 0x7f1f501519d0 2023-12-26T21:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.060+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f500c7850 con 0x7f1f501519d0 2023-12-26T21:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.060+0000 7f1f3e7fc700 1 -- 172.21.15.131:0/925624010 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4800c900 con 0x7f1f501519d0 2023-12-26T21:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.060+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f500ca750 con 0x7f1f501519d0 2023-12-26T21:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.061+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f500c7a40 con 0x7f1f501519d0 2023-12-26T21:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.062+0000 7f1f3e7fc700 1 -- 172.21.15.131:0/925624010 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f4800cc70 con 0x7f1f501519d0 2023-12-26T21:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.063+0000 7f1f3e7fc700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1f30066f20 0x7f1f300693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.063+0000 7f1f3e7fc700 1 -- 172.21.15.131:0/925624010 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f48084060 con 0x7f1f501519d0 2023-12-26T21:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.063+0000 7f1f4e59c700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1f30066f20 0x7f1f300693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.063+0000 7f1f4e59c700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1f30066f20 0x7f1f300693d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f40007ec0 tx=0x7f1f40015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.067+0000 7f1f3e7fc700 1 -- 172.21.15.131:0/925624010 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f480512d0 con 0x7f1f501519d0 2023-12-26T21:37:30.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.300+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f50169ad0 con 0x7f1f501519d0 2023-12-26T21:37:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.301+0000 7f1f3e7fc700 1 -- 172.21.15.131:0/925624010 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f1f480548f0 con 0x7f1f501519d0 2023-12-26T21:37:30.301 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:30.301 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1127181197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1127181197},{"type":"v1","addr":"172.21.15.190:6835","nonce":1127181197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1425514628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1425514628},{"type":"v1","addr":"172.21.15.131:6837","nonce":1425514628}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/3223324183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3223324183},{"type":"v1","addr":"172.21.15.190:6833","nonce":3223324183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:36:48.288910+0000","modified":"2023-12-26T21:36:51.988691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3852718531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3852718531},{"type":"v1","addr":"172.21.15.131:6836","nonce":3852718531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1f30066f20 msgr2=0x7f1f300693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1f30066f20 0x7f1f300693d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f40007ec0 tx=0x7f1f40015000 comp rx=0 tx=0).stop 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f501519d0 msgr2=0x7f1f500ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f501519d0 0x7f1f500ca210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1f5006b9f0 tx=0x7f1f480017a0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 shutdown_connections 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f501519d0 0x7f1f500ca210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f38003c20 0x7f1f50069fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1f30066f20 0x7f1f300693d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 --2- 172.21.15.131:0/925624010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f500c6b20 0x7f1f50151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 >> 172.21.15.131:0/925624010 conn(0x7f1f50068040 msgr2=0x7f1f5006e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 shutdown_connections 2023-12-26T21:37:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.303+0000 7f1f54bff700 1 -- 172.21.15.131:0/925624010 wait complete. 2023-12-26T21:37:30.305 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2023-12-26T21:37:30.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:30.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.510+0000 7f349e5ff700 1 Processor -- start 2023-12-26T21:37:30.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.510+0000 7f349e5ff700 1 -- start start 2023-12-26T21:37:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.512+0000 7f349e5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34980b0bf0 0x7f34980c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.512+0000 7f349e5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3498156940 con 0x7f34980cb320 2023-12-26T21:37:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.512+0000 7f349e5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34980ae500 con 0x7f34980b0750 2023-12-26T21:37:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.512+0000 7f349e5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34980c8cc0 con 0x7f34980b0bf0 2023-12-26T21:37:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.512+0000 7f3497fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f34980cb320 0x7f3498069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39078/0 (socket says 172.21.15.131:39078) 2023-12-26T21:37:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.512+0000 7f3497fff700 1 -- 172.21.15.131:0/1949771978 learned_addr learned my addr 172.21.15.131:0/1949771978 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.512+0000 7f349cb9c700 1 --2- 172.21.15.131:0/1949771978 >> v2:172.21.15.190:3300/0 conn(0x7f34980b0bf0 0x7f34980c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.512+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542945705 0 0) 0x7f3498156940 con 0x7f34980cb320 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.512+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3480003570 con 0x7f34980cb320 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1492164190 0 0) 0x7f3480003570 con 0x7f34980cb320 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 >> v2:172.21.15.190:3300/0 conn(0x7f34980b0bf0 msgr2=0x7f34980c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 --2- 172.21.15.131:0/1949771978 >> v2:172.21.15.190:3300/0 conn(0x7f34980b0bf0 0x7f34980c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 >> v1:172.21.15.190:6789/0 conn(0x7f34980b0750 legacy=0x7f34980c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34980b0280 con 0x7f34980cb320 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f3488001470 con 0x7f34980cb320 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f349cb9c700 1 --2- 172.21.15.131:0/1949771978 >> v2:172.21.15.190:3300/0 conn(0x7f34980b0bf0 0x7f34980c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 >> v1:172.21.15.131:6789/0 conn(0x7f34980cb320 legacy=0x7f3498069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3480003a60 0x7f3480003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3480004810 0x7f3480004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34800051c0 0x7f348000dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34980c8cc0 con 0x7f34800051c0 2023-12-26T21:37:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348000e2e0 con 0x7f3480004810 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348000e4a0 con 0x7f3480003a60 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f349cb9c700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34800051c0 0x7f348000dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f3497fff700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3480003a60 0x7f3480003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f34977fe700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3480004810 0x7f3480004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f349cb9c700 1 -- 172.21.15.131:0/1949771978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3480003a60 msgr2=0x7f3480003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f349cb9c700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3480003a60 0x7f3480003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f349cb9c700 1 -- 172.21.15.131:0/1949771978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3480004810 msgr2=0x7f3480004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f349cb9c700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3480004810 0x7f3480004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.513+0000 7f349cb9c700 1 -- 172.21.15.131:0/1949771978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480003570 con 0x7f34800051c0 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f349cb9c700 1 -- 172.21.15.131:0/1949771978 reap_dead start 2023-12-26T21:37:30.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f349cb9c700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34800051c0 0x7f348000dc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f349000e3c0 tx=0x7f349000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd49a3ea662e373b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490009c50 con 0x7f34800051c0 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349000f040 con 0x7f34800051c0 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f3496ffd700 1 -- 172.21.15.131:0/1949771978 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000eec0 con 0x7f34800051c0 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f349e5ff700 1 -- 172.21.15.131:0/1949771978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34800051c0 msgr2=0x7f348000dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f349e5ff700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34800051c0 0x7f348000dc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f349000e3c0 tx=0x7f349000b040 comp rx=0 tx=0).stop 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f349e5ff700 1 -- 172.21.15.131:0/1949771978 shutdown_connections 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f349e5ff700 1 --2- 172.21.15.131:0/1949771978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34800051c0 0x7f348000dc70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f349e5ff700 1 -- 172.21.15.131:0/1949771978 >> 172.21.15.131:0/1949771978 conn(0x7f3498068040 msgr2=0x7f3498069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.514+0000 7f349e5ff700 1 -- 172.21.15.131:0/1949771978 shutdown_connections 2023-12-26T21:37:30.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.515+0000 7f349e5ff700 1 -- 172.21.15.131:0/1949771978 wait complete. 2023-12-26T21:37:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.516+0000 7f349e5ff700 1 Processor -- start 2023-12-26T21:37:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.516+0000 7f349e5ff700 1 -- start start 2023-12-26T21:37:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f349e5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34800051c0 0x7f3498165740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f349e5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3498165c80 0x7f3498157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f349e5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3498157760 0x7f349814ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f349e5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498150b50 con 0x7f3498165c80 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f349e5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349814ff40 con 0x7f3498157760 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f349e5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3498150220 con 0x7f34800051c0 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f34977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3498165c80 0x7f3498157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f34977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3498165c80 0x7f3498157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56026/0 (socket says 172.21.15.131:56026) 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f34977fe700 1 -- 172.21.15.131:0/2364953363 learned_addr learned my addr 172.21.15.131:0/2364953363 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f349cb9c700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3498157760 0x7f349814ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.517+0000 7f3497fff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34800051c0 0x7f3498165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f34977fe700 1 -- 172.21.15.131:0/2364953363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34800051c0 msgr2=0x7f3498165740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f34977fe700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34800051c0 0x7f3498165740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f34977fe700 1 -- 172.21.15.131:0/2364953363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3498157760 msgr2=0x7f349814ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f34977fe700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3498157760 0x7f349814ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f34977fe700 1 -- 172.21.15.131:0/2364953363 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3498150500 con 0x7f3498165c80 2023-12-26T21:37:30.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f34977fe700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3498165c80 0x7f3498157220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f348c002950 tx=0x7f348c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:30.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f347ffff700 1 -- 172.21.15.131:0/2364953363 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c002e90 con 0x7f3498165c80 2023-12-26T21:37:30.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f347ffff700 1 -- 172.21.15.131:0/2364953363 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c00f8d0 con 0x7f3498165c80 2023-12-26T21:37:30.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f347ffff700 1 -- 172.21.15.131:0/2364953363 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c00fc90 con 0x7f3498165c80 2023-12-26T21:37:30.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34981507e0 con 0x7f3498165c80 2023-12-26T21:37:30.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.518+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349814f3d0 con 0x7f3498165c80 2023-12-26T21:37:30.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.519+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34980657e0 con 0x7f3498165c80 2023-12-26T21:37:30.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.521+0000 7f347ffff700 1 -- 172.21.15.131:0/2364953363 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f348c013600 con 0x7f3498165c80 2023-12-26T21:37:30.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.521+0000 7f347ffff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3484066f30 0x7f34840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.521+0000 7f347ffff700 1 -- 172.21.15.131:0/2364953363 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f348c084990 con 0x7f3498165c80 2023-12-26T21:37:30.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.522+0000 7f3497fff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3484066f30 0x7f34840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.522+0000 7f3497fff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3484066f30 0x7f34840693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f34880016e0 tx=0x7f3488009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:30.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.525+0000 7f347ffff700 1 -- 172.21.15.131:0/2364953363 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f348c051cb0 con 0x7f3498165c80 2023-12-26T21:37:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.720+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3498150c90 con 0x7f3498165c80 2023-12-26T21:37:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.721+0000 7f347ffff700 1 -- 172.21.15.131:0/2364953363 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f348c00fdf0 con 0x7f3498165c80 2023-12-26T21:37:30.721 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:30.721 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:36:49.810698+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:36:46.654972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-26T21:36:47.264880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3484066f30 msgr2=0x7f34840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3484066f30 0x7f34840693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f34880016e0 tx=0x7f3488009040 comp rx=0 tx=0).stop 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3498165c80 msgr2=0x7f3498157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3498165c80 0x7f3498157220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f348c002950 tx=0x7f348c007d30 comp rx=0 tx=0).stop 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 shutdown_connections 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3498157760 0x7f349814ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3484066f30 0x7f34840693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34800051c0 0x7f3498165740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 --2- 172.21.15.131:0/2364953363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3498165c80 0x7f3498157220 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 >> 172.21.15.131:0/2364953363 conn(0x7f3498068040 msgr2=0x7f349814dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:30.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 shutdown_connections 2023-12-26T21:37:30.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.723+0000 7f349e5ff700 1 -- 172.21.15.131:0/2364953363 wait complete. 2023-12-26T21:37:30.742 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T21:37:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.932+0000 7f8f69dff700 1 Processor -- start 2023-12-26T21:37:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.932+0000 7f8f69dff700 1 -- start start 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.933+0000 7f8f69dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8f6406bfd0 0x7f8f640c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.933+0000 7f8f69dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8f640cc100 0x7f8f640cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.933+0000 7f8f69dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f640686c0 con 0x7f8f6413c510 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.933+0000 7f8f69dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f64069520 con 0x7f8f6406bfd0 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.933+0000 7f8f69dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f64068910 con 0x7f8f640cc100 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.933+0000 7f8f62ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8f6406bfd0 0x7f8f640c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.933+0000 7f8f62ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8f6406bfd0 0x7f8f640c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56028/0 (socket says 172.21.15.131:56028) 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.933+0000 7f8f62ffd700 1 -- 172.21.15.131:0/2638193194 learned_addr learned my addr 172.21.15.131:0/2638193194 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.934+0000 7f8f63fff700 1 --2- 172.21.15.131:0/2638193194 >> v2:172.21.15.190:3300/0 conn(0x7f8f640cc100 0x7f8f640cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.934+0000 7f8f62ffd700 1 -- 172.21.15.131:0/2638193194 >> v2:172.21.15.190:3300/0 conn(0x7f8f640cc100 msgr2=0x7f8f640cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.934+0000 7f8f62ffd700 1 --2- 172.21.15.131:0/2638193194 >> v2:172.21.15.190:3300/0 conn(0x7f8f640cc100 0x7f8f640cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.934+0000 7f8f62ffd700 1 -- 172.21.15.131:0/2638193194 >> v1:172.21.15.190:6789/0 conn(0x7f8f6413c510 legacy=0x7f8f6406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.934+0000 7f8f62ffd700 1 -- 172.21.15.131:0/2638193194 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f64068bf0 con 0x7f8f6406bfd0 2023-12-26T21:37:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.934+0000 7f8f62ffd700 1 --2- 172.21.15.131:0/2638193194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f6406bfd0 0x7f8f640c8a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f640cf0a0 tx=0x7f8f58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8effa30326520617 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.934+0000 7f8f627fc700 1 -- 172.21.15.131:0/2638193194 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800b860 con 0x7f8f6406bfd0 2023-12-26T21:37:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.934+0000 7f8f627fc700 1 -- 172.21.15.131:0/2638193194 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800b9c0 con 0x7f8f6406bfd0 2023-12-26T21:37:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.934+0000 7f8f627fc700 1 -- 172.21.15.131:0/2638193194 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800bd60 con 0x7f8f6406bfd0 2023-12-26T21:37:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.935+0000 7f8f69dff700 1 -- 172.21.15.131:0/2638193194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f6406bfd0 msgr2=0x7f8f640c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.935+0000 7f8f69dff700 1 --2- 172.21.15.131:0/2638193194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f6406bfd0 0x7f8f640c8a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f640cf0a0 tx=0x7f8f58009250 comp rx=0 tx=0).stop 2023-12-26T21:37:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.935+0000 7f8f69dff700 1 -- 172.21.15.131:0/2638193194 shutdown_connections 2023-12-26T21:37:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.935+0000 7f8f69dff700 1 --2- 172.21.15.131:0/2638193194 >> v2:172.21.15.190:3300/0 conn(0x7f8f640cc100 0x7f8f640cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.935+0000 7f8f69dff700 1 --2- 172.21.15.131:0/2638193194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f6406bfd0 0x7f8f640c8a20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.935+0000 7f8f69dff700 1 -- 172.21.15.131:0/2638193194 >> 172.21.15.131:0/2638193194 conn(0x7f8f640ce3e0 msgr2=0x7f8f640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.935+0000 7f8f69dff700 1 -- 172.21.15.131:0/2638193194 shutdown_connections 2023-12-26T21:37:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.935+0000 7f8f69dff700 1 -- 172.21.15.131:0/2638193194 wait complete. 2023-12-26T21:37:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.937+0000 7f8f69dff700 1 Processor -- start 2023-12-26T21:37:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.937+0000 7f8f69dff700 1 -- start start 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f69dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6406bfd0 0x7f8f64153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f69dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f640cc100 0x7f8f64157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f69dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6413c510 0x7f8f64158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f69dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640b3b30 con 0x7f8f640cc100 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f69dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640b3e10 con 0x7f8f6413c510 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f69dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640b1e50 con 0x7f8f6406bfd0 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f640cc100 0x7f8f64157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f62ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f640cc100 0x7f8f64157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56034/0 (socket says 172.21.15.131:56034) 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f62ffd700 1 -- 172.21.15.131:0/114794294 learned_addr learned my addr 172.21.15.131:0/114794294 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f63fff700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6413c510 0x7f8f64158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f637fe700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6406bfd0 0x7f8f64153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f62ffd700 1 -- 172.21.15.131:0/114794294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6406bfd0 msgr2=0x7f8f64153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f62ffd700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6406bfd0 0x7f8f64153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f62ffd700 1 -- 172.21.15.131:0/114794294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6413c510 msgr2=0x7f8f64158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f62ffd700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6413c510 0x7f8f64158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.938+0000 7f8f62ffd700 1 -- 172.21.15.131:0/114794294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f580094f0 con 0x7f8f640cc100 2023-12-26T21:37:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.939+0000 7f8f62ffd700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f640cc100 0x7f8f64157b10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8f6406fe40 tx=0x7f8f58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.939+0000 7f8f4b7fe700 1 -- 172.21.15.131:0/114794294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58024440 con 0x7f8f640cc100 2023-12-26T21:37:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.939+0000 7f8f4b7fe700 1 -- 172.21.15.131:0/114794294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f580245a0 con 0x7f8f640cc100 2023-12-26T21:37:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.939+0000 7f8f4b7fe700 1 -- 172.21.15.131:0/114794294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58024960 con 0x7f8f640cc100 2023-12-26T21:37:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.939+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f640b1b50 con 0x7f8f640cc100 2023-12-26T21:37:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.939+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f640b3800 con 0x7f8f640cc100 2023-12-26T21:37:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.940+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f640657e0 con 0x7f8f640cc100 2023-12-26T21:37:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.941+0000 7f8f4b7fe700 1 -- 172.21.15.131:0/114794294 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8f58024d50 con 0x7f8f640cc100 2023-12-26T21:37:30.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.942+0000 7f8f4b7fe700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8f50066f30 0x7f8f500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:30.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.942+0000 7f8f4b7fe700 1 -- 172.21.15.131:0/114794294 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f58086070 con 0x7f8f640cc100 2023-12-26T21:37:30.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.942+0000 7f8f637fe700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8f50066f30 0x7f8f500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:30.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.943+0000 7f8f637fe700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8f50066f30 0x7f8f500693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f6416ce70 tx=0x7f8f54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:30.946+0000 7f8f4b7fe700 1 -- 172.21.15.131:0/114794294 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f580b75a0 con 0x7f8f640cc100 2023-12-26T21:37:31.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.190+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8f640b3c70 con 0x7f8f640cc100 2023-12-26T21:37:31.191 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T21:37:31.191+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T21:37:31.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.196+0000 7f8f4b7fe700 1 -- 172.21.15.131:0/114794294 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8f640b3c70 con 0x7f8f640cc100 2023-12-26T21:37:31.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.199+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8f50066f30 msgr2=0x7f8f500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:31.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.199+0000 7f8f69dff700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8f50066f30 0x7f8f500693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f6416ce70 tx=0x7f8f54009040 comp rx=0 tx=0).stop 2023-12-26T21:37:31.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.199+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f640cc100 msgr2=0x7f8f64157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:31.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.199+0000 7f8f69dff700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f640cc100 0x7f8f64157b10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8f6406fe40 tx=0x7f8f58017000 comp rx=0 tx=0).stop 2023-12-26T21:37:31.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.200+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 shutdown_connections 2023-12-26T21:37:31.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.200+0000 7f8f69dff700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6413c510 0x7f8f64158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:31.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.200+0000 7f8f69dff700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8f50066f30 0x7f8f500693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:31.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.200+0000 7f8f69dff700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6406bfd0 0x7f8f64153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:31.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.200+0000 7f8f69dff700 1 --2- 172.21.15.131:0/114794294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f640cc100 0x7f8f64157b10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:31.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.200+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 >> 172.21.15.131:0/114794294 conn(0x7f8f640ce3e0 msgr2=0x7f8f64155cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:31.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.200+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 shutdown_connections 2023-12-26T21:37:31.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.200+0000 7f8f69dff700 1 -- 172.21.15.131:0/114794294 wait complete. 2023-12-26T21:37:31.201 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T21:37:31.202 INFO:tasks.ceph.mds.c.smithi131.stderr: -2122> 2023-12-26T21:36:49.778+0000 7f70f7df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:37:31.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T21:37:31.244 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:37:31.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.412+0000 7fe7c4fff700 1 Processor -- start 2023-12-26T21:37:31.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.412+0000 7fe7c4fff700 1 -- start start 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.413+0000 7fe7c4fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7c0159820 0x7fe7c00abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.413+0000 7fe7c4fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 0x7fe7c00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.413+0000 7fe7c4fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c015a8c0 con 0x7fe7c00aa480 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.413+0000 7fe7c4fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c015ad50 con 0x7fe7c0159820 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.413+0000 7fe7c4fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0159f90 con 0x7fe7c006e480 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.413+0000 7fe7be59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7c0159820 0x7fe7c00abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.414+0000 7fe7be59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7c0159820 0x7fe7c00abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56052/0 (socket says 172.21.15.131:56052) 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.414+0000 7fe7be59c700 1 -- 172.21.15.131:0/1697370170 learned_addr learned my addr 172.21.15.131:0/1697370170 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.414+0000 7fe7bed9d700 1 --2- 172.21.15.131:0/1697370170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 0x7fe7c00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.414+0000 7fe7be59c700 1 -- 172.21.15.131:0/1697370170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 msgr2=0x7fe7c00675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.414+0000 7fe7be59c700 1 --2- 172.21.15.131:0/1697370170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 0x7fe7c00675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.414+0000 7fe7be59c700 1 -- 172.21.15.131:0/1697370170 >> v1:172.21.15.190:6789/0 conn(0x7fe7c00aa480 legacy=0x7fe7c006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.414+0000 7fe7be59c700 1 -- 172.21.15.131:0/1697370170 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c015a270 con 0x7fe7c0159820 2023-12-26T21:37:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.414+0000 7fe7be59c700 1 --2- 172.21.15.131:0/1697370170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c0159820 0x7fe7c00abdb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00099b0 tx=0x7fe7b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0735cde29e1c811 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.414+0000 7fe7bd59a700 1 -- 172.21.15.131:0/1697370170 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0009ea0 con 0x7fe7c0159820 2023-12-26T21:37:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7bd59a700 1 -- 172.21.15.131:0/1697370170 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b000b860 con 0x7fe7c0159820 2023-12-26T21:37:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7bd59a700 1 -- 172.21.15.131:0/1697370170 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000bc00 con 0x7fe7c0159820 2023-12-26T21:37:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7c4fff700 1 -- 172.21.15.131:0/1697370170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c0159820 msgr2=0x7fe7c00abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7c4fff700 1 --2- 172.21.15.131:0/1697370170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c0159820 0x7fe7c00abdb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00099b0 tx=0x7fe7b0009250 comp rx=0 tx=0).stop 2023-12-26T21:37:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7c4fff700 1 -- 172.21.15.131:0/1697370170 shutdown_connections 2023-12-26T21:37:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7c4fff700 1 --2- 172.21.15.131:0/1697370170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 0x7fe7c00675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7c4fff700 1 --2- 172.21.15.131:0/1697370170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c0159820 0x7fe7c00abdb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7c4fff700 1 -- 172.21.15.131:0/1697370170 >> 172.21.15.131:0/1697370170 conn(0x7fe7c006c290 msgr2=0x7fe7c00ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7c4fff700 1 -- 172.21.15.131:0/1697370170 shutdown_connections 2023-12-26T21:37:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.415+0000 7fe7c4fff700 1 -- 172.21.15.131:0/1697370170 wait complete. 2023-12-26T21:37:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.417+0000 7fe7c4fff700 1 Processor -- start 2023-12-26T21:37:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.417+0000 7fe7c4fff700 1 -- start start 2023-12-26T21:37:31.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7c4fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 0x7fe7c0155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:31.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7c4fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c00aa480 0x7fe7c00c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:31.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7c4fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c0159820 0x7fe7c00b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7c4fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00b22e0 con 0x7fe7c00aa480 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7c4fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00c18c0 con 0x7fe7c0159820 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7c4fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00c1e80 con 0x7fe7c006e480 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7bdd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c00aa480 0x7fe7c00c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7bdd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c00aa480 0x7fe7c00c2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56068/0 (socket says 172.21.15.131:56068) 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7bdd9b700 1 -- 172.21.15.131:0/3454471866 learned_addr learned my addr 172.21.15.131:0/3454471866 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7be59c700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 0x7fe7c0155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7bed9d700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c0159820 0x7fe7c00b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7bdd9b700 1 -- 172.21.15.131:0/3454471866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 msgr2=0x7fe7c0155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7bdd9b700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 0x7fe7c0155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7bdd9b700 1 -- 172.21.15.131:0/3454471866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c0159820 msgr2=0x7fe7c00b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7bdd9b700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c0159820 0x7fe7c00b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.418+0000 7fe7bdd9b700 1 -- 172.21.15.131:0/3454471866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00094f0 con 0x7fe7c00aa480 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.419+0000 7fe7bdd9b700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c00aa480 0x7fe7c00c2310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400c590 tx=0x7fe7b400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.419+0000 7fe7ae7fc700 1 -- 172.21.15.131:0/3454471866 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4011590 con 0x7fe7c00aa480 2023-12-26T21:37:31.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.419+0000 7fe7ae7fc700 1 -- 172.21.15.131:0/3454471866 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b4011710 con 0x7fe7c00aa480 2023-12-26T21:37:31.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.419+0000 7fe7ae7fc700 1 -- 172.21.15.131:0/3454471866 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4011ad0 con 0x7fe7c00aa480 2023-12-26T21:37:31.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.419+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c00c39f0 con 0x7fe7c00aa480 2023-12-26T21:37:31.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.419+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c00c1b10 con 0x7fe7c00aa480 2023-12-26T21:37:31.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.420+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c00c2010 con 0x7fe7c00aa480 2023-12-26T21:37:31.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.421+0000 7fe7ae7fc700 1 -- 172.21.15.131:0/3454471866 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe7b401b3b0 con 0x7fe7c00aa480 2023-12-26T21:37:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.423+0000 7fe7ae7fc700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7a0066f30 0x7fe7a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.423+0000 7fe7ae7fc700 1 -- 172.21.15.131:0/3454471866 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5820+0+0 (secure 0 0 0) 0x7fe7b40839b0 con 0x7fe7c00aa480 2023-12-26T21:37:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.423+0000 7fe7be59c700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7a0066f30 0x7fe7a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:31.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.423+0000 7fe7be59c700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7a0066f30 0x7fe7a00693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00099b0 tx=0x7fe7b0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:31.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.427+0000 7fe7ae7fc700 1 -- 172.21.15.131:0/3454471866 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7b4050bb0 con 0x7fe7c00aa480 2023-12-26T21:37:31.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:31.668+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe7c015a550 con 0x7fe7c00aa480 2023-12-26T21:37:32.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.197+0000 7fe7ae7fc700 1 -- 172.21.15.131:0/3454471866 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe7b4017020 con 0x7fe7c00aa480 2023-12-26T21:37:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7a0066f30 msgr2=0x7fe7a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7a0066f30 0x7fe7a00693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00099b0 tx=0x7fe7b0017000 comp rx=0 tx=0).stop 2023-12-26T21:37:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c00aa480 msgr2=0x7fe7c00c2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c00aa480 0x7fe7c00c2310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400c590 tx=0x7fe7b400c8a0 comp rx=0 tx=0).stop 2023-12-26T21:37:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 shutdown_connections 2023-12-26T21:37:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c0159820 0x7fe7c00b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe7a0066f30 0x7fe7a00693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c006e480 0x7fe7c0155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 --2- 172.21.15.131:0/3454471866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c00aa480 0x7fe7c00c2310 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.200+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 >> 172.21.15.131:0/3454471866 conn(0x7fe7c006c290 msgr2=0x7fe7c0069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.201+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 shutdown_connections 2023-12-26T21:37:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.201+0000 7fe7c4fff700 1 -- 172.21.15.131:0/3454471866 wait complete. 2023-12-26T21:37:32.221 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T21:37:32.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.458+0000 7f2a391ff700 1 Processor -- start 2023-12-26T21:37:32.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.458+0000 7f2a391ff700 1 -- start start 2023-12-26T21:37:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.459+0000 7f2a391ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a34159810 0x7f2a340c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.459+0000 7f2a391ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a340cbb00 con 0x7f2a340cec80 2023-12-26T21:37:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.459+0000 7f2a391ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a340ce040 con 0x7f2a340cd7b0 2023-12-26T21:37:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.459+0000 7f2a391ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340cb2a0 con 0x7f2a34159810 2023-12-26T21:37:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.460+0000 7f2a3259c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2a340cec80 0x7f2a340cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39080/0 (socket says 172.21.15.131:39080) 2023-12-26T21:37:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.460+0000 7f2a3259c700 1 -- 172.21.15.131:0/64693934 learned_addr learned my addr 172.21.15.131:0/64693934 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.460+0000 7f2a32d9d700 1 --2- 172.21.15.131:0/64693934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a34159810 0x7f2a340c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.460+0000 7f2a31d9b700 1 -- 172.21.15.131:0/64693934 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980553359 0 0) 0x7f2a340cbb00 con 0x7f2a340cec80 2023-12-26T21:37:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.460+0000 7f2a31d9b700 1 -- 172.21.15.131:0/64693934 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a1c003570 con 0x7f2a340cec80 2023-12-26T21:37:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.460+0000 7f2a31d9b700 1 -- 172.21.15.131:0/64693934 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953513381 0 0) 0x7f2a340ce040 con 0x7f2a340cd7b0 2023-12-26T21:37:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.460+0000 7f2a32d9d700 1 -- 172.21.15.131:0/64693934 >> v1:172.21.15.190:6789/0 conn(0x7f2a340cd7b0 legacy=0x7f2a340cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.460+0000 7f2a32d9d700 1 -- 172.21.15.131:0/64693934 >> v1:172.21.15.131:6789/0 conn(0x7f2a340cec80 legacy=0x7f2a340cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.460+0000 7f2a32d9d700 1 -- 172.21.15.131:0/64693934 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34159be0 con 0x7f2a34159810 2023-12-26T21:37:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.461+0000 7f2a32d9d700 1 --2- 172.21.15.131:0/64693934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a34159810 0x7f2a340c5440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009900 tx=0x7f2a24009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9175cd86d43bdec6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.461+0000 7f2a31d9b700 1 -- 172.21.15.131:0/64693934 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24005c10 con 0x7f2a34159810 2023-12-26T21:37:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.461+0000 7f2a31d9b700 1 -- 172.21.15.131:0/64693934 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a24005d70 con 0x7f2a34159810 2023-12-26T21:37:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.461+0000 7f2a31d9b700 1 -- 172.21.15.131:0/64693934 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24006260 con 0x7f2a34159810 2023-12-26T21:37:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.461+0000 7f2a391ff700 1 -- 172.21.15.131:0/64693934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a34159810 msgr2=0x7f2a340c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.461+0000 7f2a391ff700 1 --2- 172.21.15.131:0/64693934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a34159810 0x7f2a340c5440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009900 tx=0x7f2a24009120 comp rx=0 tx=0).stop 2023-12-26T21:37:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.461+0000 7f2a391ff700 1 -- 172.21.15.131:0/64693934 shutdown_connections 2023-12-26T21:37:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.461+0000 7f2a391ff700 1 --2- 172.21.15.131:0/64693934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a34159810 0x7f2a340c5440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.461+0000 7f2a391ff700 1 -- 172.21.15.131:0/64693934 >> 172.21.15.131:0/64693934 conn(0x7f2a34099950 msgr2=0x7f2a3406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.462+0000 7f2a391ff700 1 -- 172.21.15.131:0/64693934 shutdown_connections 2023-12-26T21:37:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.462+0000 7f2a391ff700 1 -- 172.21.15.131:0/64693934 wait complete. 2023-12-26T21:37:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.463+0000 7f2a391ff700 1 Processor -- start 2023-12-26T21:37:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.463+0000 7f2a391ff700 1 -- start start 2023-12-26T21:37:32.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.464+0000 7f2a391ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a340cd7b0 0x7f2a340ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:32.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.464+0000 7f2a391ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a340cec80 0x7f2a34153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:32.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.464+0000 7f2a391ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a34159810 0x7f2a3415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:32.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.464+0000 7f2a391ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a34155e40 con 0x7f2a340cd7b0 2023-12-26T21:37:32.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.464+0000 7f2a391ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340ca5b0 con 0x7f2a34159810 2023-12-26T21:37:32.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.464+0000 7f2a391ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340c7ac0 con 0x7f2a340cec80 2023-12-26T21:37:32.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.464+0000 7f2a32d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a340cd7b0 0x7f2a340ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:32.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.464+0000 7f2a32d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a340cd7b0 0x7f2a340ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56080/0 (socket says 172.21.15.131:56080) 2023-12-26T21:37:32.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.464+0000 7f2a32d9d700 1 -- 172.21.15.131:0/2814282230 learned_addr learned my addr 172.21.15.131:0/2814282230 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:32.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a32d9d700 1 -- 172.21.15.131:0/2814282230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a340cec80 msgr2=0x7f2a34153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:32.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a32d9d700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a340cec80 0x7f2a34153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:32.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a32d9d700 1 -- 172.21.15.131:0/2814282230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a34159810 msgr2=0x7f2a3415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:32.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a32d9d700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a34159810 0x7f2a3415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:32.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a32d9d700 1 -- 172.21.15.131:0/2814282230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240093c0 con 0x7f2a340cd7b0 2023-12-26T21:37:32.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a32d9d700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a340cd7b0 0x7f2a340ca070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009390 tx=0x7f2a24016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:32.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a1affd700 1 -- 172.21.15.131:0/2814282230 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400b940 con 0x7f2a340cd7b0 2023-12-26T21:37:32.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a1affd700 1 -- 172.21.15.131:0/2814282230 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2400bb00 con 0x7f2a340cd7b0 2023-12-26T21:37:32.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a340c7da0 con 0x7f2a340cd7b0 2023-12-26T21:37:32.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a1affd700 1 -- 172.21.15.131:0/2814282230 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400bec0 con 0x7f2a340cd7b0 2023-12-26T21:37:32.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.465+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a340c8360 con 0x7f2a340cd7b0 2023-12-26T21:37:32.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.466+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a340717f0 con 0x7f2a340cd7b0 2023-12-26T21:37:32.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.467+0000 7f2a1affd700 1 -- 172.21.15.131:0/2814282230 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2a2401b690 con 0x7f2a340cd7b0 2023-12-26T21:37:32.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.468+0000 7f2a1affd700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a20066f30 0x7f2a200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:32.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.468+0000 7f2a1affd700 1 -- 172.21.15.131:0/2814282230 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5820+0+0 (secure 0 0 0) 0x7f2a240868e0 con 0x7f2a340cd7b0 2023-12-26T21:37:32.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.468+0000 7f2a3259c700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a20066f30 0x7f2a200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:32.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.469+0000 7f2a3259c700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a20066f30 0x7f2a200693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a280058d0 tx=0x7f2a280079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:32.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.472+0000 7f2a1affd700 1 -- 172.21.15.131:0/2814282230 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a24053a40 con 0x7f2a340cd7b0 2023-12-26T21:37:32.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:32.662+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a340cdd60 con 0x7f2a340cd7b0 2023-12-26T21:37:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.205+0000 7f2a1affd700 1 -- 172.21.15.131:0/2814282230 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2a24057060 con 0x7f2a340cd7b0 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a20066f30 msgr2=0x7f2a200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a20066f30 0x7f2a200693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a280058d0 tx=0x7f2a280079d0 comp rx=0 tx=0).stop 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a340cd7b0 msgr2=0x7f2a340ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a340cd7b0 0x7f2a340ca070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009390 tx=0x7f2a24016000 comp rx=0 tx=0).stop 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 shutdown_connections 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a34159810 0x7f2a3415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a340cd7b0 0x7f2a340ca070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a20066f30 0x7f2a200693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 --2- 172.21.15.131:0/2814282230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a340cec80 0x7f2a34153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 >> 172.21.15.131:0/2814282230 conn(0x7f2a34099950 msgr2=0x7f2a34068c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 shutdown_connections 2023-12-26T21:37:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.209+0000 7f2a391ff700 1 -- 172.21.15.131:0/2814282230 wait complete. 2023-12-26T21:37:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T21:37:33.229 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T21:37:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.430+0000 7fdafedff700 1 Processor -- start 2023-12-26T21:37:33.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.430+0000 7fdafedff700 1 -- start start 2023-12-26T21:37:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.431+0000 7fdafedff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdaf80cd7b0 0x7fdaf80c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.431+0000 7fdafedff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf80cbb00 con 0x7fdaf80cf390 2023-12-26T21:37:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.431+0000 7fdafedff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf80ce040 con 0x7fdaf8159810 2023-12-26T21:37:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.431+0000 7fdafedff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf80cdb80 con 0x7fdaf80cd7b0 2023-12-26T21:37:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.432+0000 7fdafd39c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdaf80cd7b0 0x7fdaf80c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.432+0000 7fdaf7fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdaf80cf390 0x7fdaf80cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39090/0 (socket says 172.21.15.131:39090) 2023-12-26T21:37:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.432+0000 7fdafd39c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdaf80cd7b0 0x7fdaf80c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56090/0 (socket says 172.21.15.131:56090) 2023-12-26T21:37:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.432+0000 7fdaf7fff700 1 -- 172.21.15.131:0/1194023266 learned_addr learned my addr 172.21.15.131:0/1194023266 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.432+0000 7fdafd39c700 1 -- 172.21.15.131:0/1194023266 >> v1:172.21.15.190:6789/0 conn(0x7fdaf8159810 legacy=0x7fdaf80cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.432+0000 7fdafd39c700 1 -- 172.21.15.131:0/1194023266 >> v1:172.21.15.131:6789/0 conn(0x7fdaf80cf390 legacy=0x7fdaf80cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.432+0000 7fdafd39c700 1 -- 172.21.15.131:0/1194023266 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8159be0 con 0x7fdaf80cd7b0 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.432+0000 7fdafd39c700 1 --2- 172.21.15.131:0/1194023266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cd7b0 0x7fdaf80c5440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009e60 tx=0x7fdaf0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf8d736361185d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.433+0000 7fdaf77fe700 1 -- 172.21.15.131:0/1194023266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000e070 con 0x7fdaf80cd7b0 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.433+0000 7fdaf77fe700 1 -- 172.21.15.131:0/1194023266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf000bc30 con 0x7fdaf80cd7b0 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.433+0000 7fdaf77fe700 1 -- 172.21.15.131:0/1194023266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0012430 con 0x7fdaf80cd7b0 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.433+0000 7fdafedff700 1 -- 172.21.15.131:0/1194023266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cd7b0 msgr2=0x7fdaf80c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.433+0000 7fdafedff700 1 --2- 172.21.15.131:0/1194023266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cd7b0 0x7fdaf80c5440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009e60 tx=0x7fdaf0009780 comp rx=0 tx=0).stop 2023-12-26T21:37:33.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.433+0000 7fdafedff700 1 -- 172.21.15.131:0/1194023266 shutdown_connections 2023-12-26T21:37:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.433+0000 7fdafedff700 1 --2- 172.21.15.131:0/1194023266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cd7b0 0x7fdaf80c5440 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.433+0000 7fdafedff700 1 -- 172.21.15.131:0/1194023266 >> 172.21.15.131:0/1194023266 conn(0x7fdaf8099950 msgr2=0x7fdaf806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.434+0000 7fdafedff700 1 -- 172.21.15.131:0/1194023266 shutdown_connections 2023-12-26T21:37:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.434+0000 7fdafedff700 1 -- 172.21.15.131:0/1194023266 wait complete. 2023-12-26T21:37:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.435+0000 7fdafedff700 1 Processor -- start 2023-12-26T21:37:33.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.435+0000 7fdafedff700 1 -- start start 2023-12-26T21:37:33.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdafedff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf80cd7b0 0x7fdaf80b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdafedff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cf390 0x7fdaf81533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdafedff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf8159810 0x7fdaf80c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdafedff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf815d380 con 0x7fdaf80cf390 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdafedff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf815d5d0 con 0x7fdaf80cd7b0 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdafedff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf815d8b0 con 0x7fdaf8159810 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdaf7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cf390 0x7fdaf81533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdaf7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cf390 0x7fdaf81533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56104/0 (socket says 172.21.15.131:56104) 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdaf7fff700 1 -- 172.21.15.131:0/1795874556 learned_addr learned my addr 172.21.15.131:0/1795874556 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdafd39c700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf8159810 0x7fdaf80c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.436+0000 7fdafcb9b700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf80cd7b0 0x7fdaf80b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdaf7fff700 1 -- 172.21.15.131:0/1795874556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf8159810 msgr2=0x7fdaf80c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdaf7fff700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf8159810 0x7fdaf80c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdaf7fff700 1 -- 172.21.15.131:0/1795874556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf80cd7b0 msgr2=0x7fdaf80b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:33.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdaf7fff700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf80cd7b0 0x7fdaf80b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:33.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdaf7fff700 1 -- 172.21.15.131:0/1795874556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf0009a20 con 0x7fdaf80cf390 2023-12-26T21:37:33.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdaf7fff700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cf390 0x7fdaf81533b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdae400c5b0 tx=0x7fdae4007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:33.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdaf4ff9700 1 -- 172.21.15.131:0/1795874556 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4005b30 con 0x7fdaf80cf390 2023-12-26T21:37:33.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdaf4ff9700 1 -- 172.21.15.131:0/1795874556 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae4005cb0 con 0x7fdaf80cf390 2023-12-26T21:37:33.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf815db90 con 0x7fdaf80cf390 2023-12-26T21:37:33.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdaf4ff9700 1 -- 172.21.15.131:0/1795874556 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4004a80 con 0x7fdaf80cf390 2023-12-26T21:37:33.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.437+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf815e150 con 0x7fdaf80cf390 2023-12-26T21:37:33.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.438+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf80717f0 con 0x7fdaf80cf390 2023-12-26T21:37:33.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.439+0000 7fdaf4ff9700 1 -- 172.21.15.131:0/1795874556 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdae4005030 con 0x7fdaf80cf390 2023-12-26T21:37:33.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.440+0000 7fdaf4ff9700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdadc066f30 0x7fdadc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:33.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.440+0000 7fdaf4ff9700 1 -- 172.21.15.131:0/1795874556 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fdae4083df0 con 0x7fdaf80cf390 2023-12-26T21:37:33.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.440+0000 7fdafcb9b700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdadc066f30 0x7fdadc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:33.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.441+0000 7fdafcb9b700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdadc066f30 0x7fdadc0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000c00 tx=0x7fdaec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:33.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.444+0000 7fdaf4ff9700 1 -- 172.21.15.131:0/1795874556 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdae4051100 con 0x7fdaf80cf390 2023-12-26T21:37:33.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:33.636+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdaf80c68b0 con 0x7fdaf80cf390 2023-12-26T21:37:34.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.213+0000 7fdaf4ff9700 1 -- 172.21.15.131:0/1795874556 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdae4054720 con 0x7fdaf80cf390 2023-12-26T21:37:34.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdadc066f30 msgr2=0x7fdadc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdadc066f30 0x7fdadc0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000c00 tx=0x7fdaec009040 comp rx=0 tx=0).stop 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cf390 msgr2=0x7fdaf81533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cf390 0x7fdaf81533b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdae400c5b0 tx=0x7fdae4007f10 comp rx=0 tx=0).stop 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 shutdown_connections 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdadc066f30 0x7fdadc0693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf8159810 0x7fdaf80c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf80cd7b0 0x7fdaf80b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 --2- 172.21.15.131:0/1795874556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaf80cf390 0x7fdaf81533b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 >> 172.21.15.131:0/1795874556 conn(0x7fdaf8099950 msgr2=0x7fdaf80c6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 shutdown_connections 2023-12-26T21:37:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:34.217+0000 7fdafedff700 1 -- 172.21.15.131:0/1795874556 wait complete. 2023-12-26T21:37:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T21:37:34.237 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T21:37:34.238 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T21:37:34.238 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T21:37:34.238 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:37:34.238 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T21:37:34.238 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T21:37:34.238 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T21:37:34.238 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:37:34.239 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T21:37:34.239 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T21:37:34.239 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T21:37:34.239 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:37:34.239 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T21:37:34.239 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T21:37:34.239 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T21:37:34.240 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:37:34.252 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T21:37:34.253 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T21:37:34.253 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T21:37:34.253 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T21:37:34.257 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T21:37:34.257 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T21:37:34.298 INFO:tasks.ceph.mds.a:Started 2023-12-26T21:37:34.298 DEBUG:teuthology.parallel:result is None 2023-12-26T21:37:34.300 INFO:tasks.ceph.mds.c:Started 2023-12-26T21:37:34.300 DEBUG:teuthology.parallel:result is None 2023-12-26T21:37:34.302 INFO:tasks.ceph.mds.d:Started 2023-12-26T21:37:34.302 DEBUG:teuthology.parallel:result is None 2023-12-26T21:37:34.436 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T21:37:34.437 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:37:34.442 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:37:35.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-12-26T21:37:40.257 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T21:37:40.257 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T21:37:40.263 INFO:tasks.ceph.mds.b:Started 2023-12-26T21:37:40.263 DEBUG:teuthology.parallel:result is None 2023-12-26T21:37:40.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T21:37:40.406 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T21:37:40.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.466+0000 7f1d907ff700 1 Processor -- start 2023-12-26T21:37:40.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.466+0000 7f1d907ff700 1 -- start start 2023-12-26T21:37:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d907ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d8813c510 0x7f1d880c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d907ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d8806b8c0 0x7f1d8806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d907ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d880686c0 con 0x7f1d880cc100 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d907ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8806c280 con 0x7f1d8806b8c0 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d907ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d88069520 con 0x7f1d8813c510 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d8dd9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d8806b8c0 0x7f1d8806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d8ed9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1d880cc100 0x7f1d880cc520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43042/0 (socket says 172.21.15.131:43042) 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d8dd9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d8806b8c0 0x7f1d8806bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56360/0 (socket says 172.21.15.131:56360) 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d8ed9c700 1 -- 172.21.15.131:0/3074389531 learned_addr learned my addr 172.21.15.131:0/3074389531 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.467+0000 7f1d8e59b700 1 --2- 172.21.15.131:0/3074389531 >> v2:172.21.15.190:3300/0 conn(0x7f1d8813c510 0x7f1d880c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.468+0000 7f1d8dd9a700 1 -- 172.21.15.131:0/3074389531 >> v2:172.21.15.190:3300/0 conn(0x7f1d8813c510 msgr2=0x7f1d880c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.468+0000 7f1d8dd9a700 1 --2- 172.21.15.131:0/3074389531 >> v2:172.21.15.190:3300/0 conn(0x7f1d8813c510 0x7f1d880c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.468+0000 7f1d8dd9a700 1 -- 172.21.15.131:0/3074389531 >> v1:172.21.15.131:6789/0 conn(0x7f1d880cc100 legacy=0x7f1d880cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.468+0000 7f1d8dd9a700 1 -- 172.21.15.131:0/3074389531 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d88068910 con 0x7f1d8806b8c0 2023-12-26T21:37:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.468+0000 7f1d8e59b700 1 --2- 172.21.15.131:0/3074389531 >> v2:172.21.15.190:3300/0 conn(0x7f1d8813c510 0x7f1d880c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.468+0000 7f1d8dd9a700 1 --2- 172.21.15.131:0/3074389531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d8806b8c0 0x7f1d8806bd40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d8806fcc0 tx=0x7f1d780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172e6c4374d43cb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.468+0000 7f1d8d599700 1 -- 172.21.15.131:0/3074389531 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78011630 con 0x7f1d8806b8c0 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.468+0000 7f1d8d599700 1 -- 172.21.15.131:0/3074389531 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78011790 con 0x7f1d8806b8c0 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.468+0000 7f1d8d599700 1 -- 172.21.15.131:0/3074389531 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78011b30 con 0x7f1d8806b8c0 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.469+0000 7f1d907ff700 1 -- 172.21.15.131:0/3074389531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d8806b8c0 msgr2=0x7f1d8806bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.469+0000 7f1d907ff700 1 --2- 172.21.15.131:0/3074389531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d8806b8c0 0x7f1d8806bd40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d8806fcc0 tx=0x7f1d780091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.469+0000 7f1d907ff700 1 -- 172.21.15.131:0/3074389531 shutdown_connections 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.469+0000 7f1d907ff700 1 --2- 172.21.15.131:0/3074389531 >> v2:172.21.15.190:3300/0 conn(0x7f1d8813c510 0x7f1d880c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.469+0000 7f1d907ff700 1 --2- 172.21.15.131:0/3074389531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d8806b8c0 0x7f1d8806bd40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.469+0000 7f1d907ff700 1 -- 172.21.15.131:0/3074389531 >> 172.21.15.131:0/3074389531 conn(0x7f1d880ce3e0 msgr2=0x7f1d880ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.469+0000 7f1d907ff700 1 -- 172.21.15.131:0/3074389531 shutdown_connections 2023-12-26T21:37:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.469+0000 7f1d907ff700 1 -- 172.21.15.131:0/3074389531 wait complete. 2023-12-26T21:37:40.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.470+0000 7f1d907ff700 1 Processor -- start 2023-12-26T21:37:40.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.471+0000 7f1d907ff700 1 -- start start 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.471+0000 7f1d907ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8806b8c0 0x7f1d88154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.471+0000 7f1d907ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cc100 0x7f1d8815b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.471+0000 7f1d907ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8813c510 0x7f1d8815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.471+0000 7f1d907ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b1510 con 0x7f1d880cc100 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.471+0000 7f1d907ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b1230 con 0x7f1d8813c510 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.471+0000 7f1d907ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8813b660 con 0x7f1d8806b8c0 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8dd9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cc100 0x7f1d8815b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8e59b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8806b8c0 0x7f1d88154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8e59b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8806b8c0 0x7f1d88154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:38154/0 (socket says 172.21.15.131:38154) 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8dd9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cc100 0x7f1d8815b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56370/0 (socket says 172.21.15.131:56370) 2023-12-26T21:37:40.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8e59b700 1 -- 172.21.15.131:0/3988899616 learned_addr learned my addr 172.21.15.131:0/3988899616 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8ed9c700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8813c510 0x7f1d8815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8dd9a700 1 -- 172.21.15.131:0/3988899616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8806b8c0 msgr2=0x7f1d88154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8dd9a700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8806b8c0 0x7f1d88154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8dd9a700 1 -- 172.21.15.131:0/3988899616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8813c510 msgr2=0x7f1d8815bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8dd9a700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8813c510 0x7f1d8815bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8dd9a700 1 -- 172.21.15.131:0/3988899616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d840093c0 con 0x7f1d880cc100 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.472+0000 7f1d8dd9a700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cc100 0x7f1d8815b6f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d78016000 tx=0x7f1d78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.473+0000 7f1d7e7fc700 1 -- 172.21.15.131:0/3988899616 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78000d10 con 0x7f1d880cc100 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.473+0000 7f1d7e7fc700 1 -- 172.21.15.131:0/3988899616 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78011b10 con 0x7f1d880cc100 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.473+0000 7f1d7e7fc700 1 -- 172.21.15.131:0/3988899616 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78025640 con 0x7f1d880cc100 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.473+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d78009420 con 0x7f1d880cc100 2023-12-26T21:37:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.473+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d88155670 con 0x7f1d880cc100 2023-12-26T21:37:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.474+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8806c8d0 con 0x7f1d880cc100 2023-12-26T21:37:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.475+0000 7f1d7e7fc700 1 -- 172.21.15.131:0/3988899616 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1d7802e450 con 0x7f1d880cc100 2023-12-26T21:37:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.475+0000 7f1d7e7fc700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d6c066f30 0x7f1d6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.476+0000 7f1d7e7fc700 1 -- 172.21.15.131:0/3988899616 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1d780859f0 con 0x7f1d880cc100 2023-12-26T21:37:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.476+0000 7f1d8e59b700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d6c066f30 0x7f1d6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.476+0000 7f1d8e59b700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d6c066f30 0x7f1d6c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1d84011310 tx=0x7f1d84009120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:40.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.480+0000 7f1d7e7fc700 1 -- 172.21.15.131:0/3988899616 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d78052ec0 con 0x7f1d880cc100 2023-12-26T21:37:40.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:40.688+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1d880657e0 con 0x7f1d880cc100 2023-12-26T21:37:42.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.004+0000 7f1d7e7fc700 1 -- 172.21.15.131:0/3988899616 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1d78015020 con 0x7f1d880cc100 2023-12-26T21:37:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d6c066f30 msgr2=0x7f1d6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d6c066f30 0x7f1d6c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1d84011310 tx=0x7f1d84009120 comp rx=0 tx=0).stop 2023-12-26T21:37:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cc100 msgr2=0x7f1d8815b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cc100 0x7f1d8815b6f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d78016000 tx=0x7f1d78017000 comp rx=0 tx=0).stop 2023-12-26T21:37:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 shutdown_connections 2023-12-26T21:37:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d8813c510 0x7f1d8815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d6c066f30 0x7f1d6c0693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d8806b8c0 0x7f1d88154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 --2- 172.21.15.131:0/3988899616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d880cc100 0x7f1d8815b6f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 >> 172.21.15.131:0/3988899616 conn(0x7f1d880ce3e0 msgr2=0x7f1d880c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:42.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 shutdown_connections 2023-12-26T21:37:42.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.007+0000 7f1d907ff700 1 -- 172.21.15.131:0/3988899616 wait complete. 2023-12-26T21:37:42.008 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T21:37:42.028 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T21:37:42.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.221+0000 7fb110fff700 1 Processor -- start 2023-12-26T21:37:42.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.221+0000 7fb110fff700 1 -- start start 2023-12-26T21:37:42.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.222+0000 7fb110fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb10c0c8bf0 0x7fb10c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.222+0000 7fb110fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0caa10 con 0x7fb10c069d10 2023-12-26T21:37:42.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.222+0000 7fb110fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c157a80 con 0x7fb10c0c8820 2023-12-26T21:37:42.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.222+0000 7fb110fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0cac50 con 0x7fb10c0c8bf0 2023-12-26T21:37:42.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10ad9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb10c069d10 0x7fb10c161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43048/0 (socket says 172.21.15.131:43048) 2023-12-26T21:37:42.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10ad9d700 1 -- 172.21.15.131:0/3175689573 learned_addr learned my addr 172.21.15.131:0/3175689573 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb109d9b700 1 --2- 172.21.15.131:0/3175689573 >> v2:172.21.15.190:3300/0 conn(0x7fb10c0c8bf0 0x7fb10c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922656158 0 0) 0x7fb104001490 con 0x7fb10c069d10 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f4003570 con 0x7fb10c069d10 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1863417676 0 0) 0x7fb0f4003570 con 0x7fb10c069d10 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 >> v2:172.21.15.190:3300/0 conn(0x7fb10c0c8bf0 msgr2=0x7fb10c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10959a700 1 --2- 172.21.15.131:0/3175689573 >> v2:172.21.15.190:3300/0 conn(0x7fb10c0c8bf0 0x7fb10c0b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 >> v1:172.21.15.190:6789/0 conn(0x7fb10c0c8820 legacy=0x7fb10c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0caf30 con 0x7fb10c069d10 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.223+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fb104001980 con 0x7fb10c069d10 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 >> v1:172.21.15.131:6789/0 conn(0x7fb10c069d10 legacy=0x7fb10c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10959a700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4003a60 0x7fb0f4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10959a700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4004810 0x7fb0f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10959a700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f40051c0 0x7fb0f400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0cac50 con 0x7fb0f4003a60 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f400c050 con 0x7fb0f4004810 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f400c210 con 0x7fb0f40051c0 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10a59c700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4003a60 0x7fb0f4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb109d9b700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4004810 0x7fb0f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10ad9d700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f40051c0 0x7fb0f400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10a59c700 1 -- 172.21.15.131:0/3175689573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f40051c0 msgr2=0x7fb0f400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10a59c700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f40051c0 0x7fb0f400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10a59c700 1 -- 172.21.15.131:0/3175689573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4004810 msgr2=0x7fb0f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10a59c700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4004810 0x7fb0f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10a59c700 1 -- 172.21.15.131:0/3175689573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f4003570 con 0x7fb0f4003a60 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.224+0000 7fb10ad9d700 1 -- 172.21.15.131:0/3175689573 reap_dead start 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb10a59c700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4003a60 0x7fb0f4003e30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0019a0 tx=0x7fb0fc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17519c180077c082 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00c5d0 con 0x7fb0f4003a60 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00c750 con 0x7fb0f4003a60 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb10959a700 1 -- 172.21.15.131:0/3175689573 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00cb10 con 0x7fb0f4003a60 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb110fff700 1 -- 172.21.15.131:0/3175689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4003a60 msgr2=0x7fb0f4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb110fff700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4003a60 0x7fb0f4003e30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0019a0 tx=0x7fb0fc007d10 comp rx=0 tx=0).stop 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb110fff700 1 -- 172.21.15.131:0/3175689573 shutdown_connections 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb110fff700 1 --2- 172.21.15.131:0/3175689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0f4003a60 0x7fb0f4003e30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb110fff700 1 -- 172.21.15.131:0/3175689573 >> 172.21.15.131:0/3175689573 conn(0x7fb10c068040 msgr2=0x7fb10c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:42.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb110fff700 1 -- 172.21.15.131:0/3175689573 shutdown_connections 2023-12-26T21:37:42.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.225+0000 7fb110fff700 1 -- 172.21.15.131:0/3175689573 wait complete. 2023-12-26T21:37:42.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.227+0000 7fb110fff700 1 Processor -- start 2023-12-26T21:37:42.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.228+0000 7fb110fff700 1 -- start start 2023-12-26T21:37:42.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb110fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4003a60 0x7fb10c158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb110fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0c6bf0 0x7fb10c0c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb110fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb10c169480 0x7fb10c169920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb110fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c14f680 con 0x7fb10c169480 2023-12-26T21:37:42.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb110fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0c75a0 con 0x7fb0f4003a60 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb110fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c16a4c0 con 0x7fb10c0c6bf0 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb10ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb10c169480 0x7fb10c169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb10ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb10c169480 0x7fb10c169920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56382/0 (socket says 172.21.15.131:56382) 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb10ad9d700 1 -- 172.21.15.131:0/1474062742 learned_addr learned my addr 172.21.15.131:0/1474062742 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb10a59c700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4003a60 0x7fb10c158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb109d9b700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0c6bf0 0x7fb10c0c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.229+0000 7fb10ad9d700 1 -- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0c6bf0 msgr2=0x7fb10c0c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb10ad9d700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0c6bf0 0x7fb10c0c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb10ad9d700 1 -- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4003a60 msgr2=0x7fb10c158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb10ad9d700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4003a60 0x7fb10c158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb10ad9d700 1 -- 172.21.15.131:0/1474062742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c16a7a0 con 0x7fb10c169480 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb10a59c700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4003a60 0x7fb10c158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:42.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb109d9b700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0c6bf0 0x7fb10c0c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:42.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb10ad9d700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb10c169480 0x7fb10c169920 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0c8fc0 tx=0x7fb104001490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:42.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb0fa7fc700 1 -- 172.21.15.131:0/1474062742 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104001500 con 0x7fb10c169480 2023-12-26T21:37:42.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb0fa7fc700 1 -- 172.21.15.131:0/1474062742 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10400ebc0 con 0x7fb10c169480 2023-12-26T21:37:42.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb0fa7fc700 1 -- 172.21.15.131:0/1474062742 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104013420 con 0x7fb10c169480 2023-12-26T21:37:42.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c16aa80 con 0x7fb10c169480 2023-12-26T21:37:42.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.230+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c169e60 con 0x7fb10c169480 2023-12-26T21:37:42.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.231+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c0657e0 con 0x7fb10c169480 2023-12-26T21:37:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.232+0000 7fb0fa7fc700 1 -- 172.21.15.131:0/1474062742 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb104013850 con 0x7fb10c169480 2023-12-26T21:37:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.233+0000 7fb0fa7fc700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb0ec066f30 0x7fb0ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.233+0000 7fb0fa7fc700 1 -- 172.21.15.131:0/1474062742 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb104084750 con 0x7fb10c169480 2023-12-26T21:37:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.233+0000 7fb10a59c700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb0ec066f30 0x7fb0ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.234+0000 7fb10a59c700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb0ec066f30 0x7fb0ec0693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000c00 tx=0x7fb0fc015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:42.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.237+0000 7fb0fa7fc700 1 -- 172.21.15.131:0/1474062742 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb104051c70 con 0x7fb10c169480 2023-12-26T21:37:42.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-12-26T21:37:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.506+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb10c0caa10 con 0x7fb10c169480 2023-12-26T21:37:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.507+0000 7fb0fa7fc700 1 -- 172.21.15.131:0/1474062742 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fb10c0caa10 con 0x7fb10c169480 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.507 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T21:37:42.508 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:37:42.509 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.510 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:37:42.511 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T21:37:42.512 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.513 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:37:42.514 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.515 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.516 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:37:42.517 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mds", 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBYR4tlFWHGCxAA59/KZj2X/wsrlJ4wiSN72Q==", 2023-12-26T21:37:42.518 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mds" 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mgr", 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBYR4tlHW3GCxAAZX9Q5wm7XjAs63tlzW6XGg==", 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-osd", 2023-12-26T21:37:42.519 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBYR4tlqXXGCxAAwfJXp7J2Hv7NRi3SOd9AOg==", 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-osd" 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd", 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBYR4tlo33GCxAA+O5CZtypXTkRn4Ovi2GJlw==", 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-26T21:37:42.520 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBYR4tlJIbGCxAAq9kSEyEGojHW+DXFCsDpHQ==", 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rgw", 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBYR4tlsY7GCxAAxKrdJgqldpS3yAX7b1Py7w==", 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T21:37:42.521 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:37:42.522 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.523 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:42.523 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T21:37:42.523 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.509+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb0ec066f30 msgr2=0x7fb0ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.509+0000 7fb110fff700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb0ec066f30 0x7fb0ec0693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000c00 tx=0x7fb0fc015000 comp rx=0 tx=0).stop 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.509+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb10c169480 msgr2=0x7fb10c169920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.509+0000 7fb110fff700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb10c169480 0x7fb10c169920 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0c8fc0 tx=0x7fb104001490 comp rx=0 tx=0).stop 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.509+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 shutdown_connections 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.509+0000 7fb110fff700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb10c169480 0x7fb10c169920 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.509+0000 7fb110fff700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f4003a60 0x7fb10c158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.510+0000 7fb110fff700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb0ec066f30 0x7fb0ec0693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.510+0000 7fb110fff700 1 --2- 172.21.15.131:0/1474062742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0c6bf0 0x7fb10c0c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.510+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 >> 172.21.15.131:0/1474062742 conn(0x7fb10c068040 msgr2=0x7fb10c0cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.510+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 shutdown_connections 2023-12-26T21:37:42.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.510+0000 7fb110fff700 1 -- 172.21.15.131:0/1474062742 wait complete. 2023-12-26T21:37:42.527 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-26T21:37:42.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.717+0000 7fcf437ff700 1 Processor -- start 2023-12-26T21:37:42.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.717+0000 7fcf437ff700 1 -- start start 2023-12-26T21:37:42.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.718+0000 7fcf437ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c159920 0x7fcf3c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.718+0000 7fcf437ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf3c159cf0 0x7fcf3c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.718+0000 7fcf437ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3c0c8bf0 con 0x7fcf3c069ca0 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.718+0000 7fcf437ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3c0c8e60 con 0x7fcf3c159cf0 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.718+0000 7fcf437ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0c9140 con 0x7fcf3c159920 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.719+0000 7fcf40d9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf3c159cf0 0x7fcf3c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.719+0000 7fcf41d9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcf3c069ca0 0x7fcf3c1554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43060/0 (socket says 172.21.15.131:43060) 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.719+0000 7fcf41d9c700 1 -- 172.21.15.131:0/2215004968 learned_addr learned my addr 172.21.15.131:0/2215004968 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.719+0000 7fcf4159b700 1 --2- 172.21.15.131:0/2215004968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c159920 0x7fcf3c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.719+0000 7fcf40d9a700 1 -- 172.21.15.131:0/2215004968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c159920 msgr2=0x7fcf3c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.719+0000 7fcf40d9a700 1 --2- 172.21.15.131:0/2215004968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c159920 0x7fcf3c0b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.719+0000 7fcf40d9a700 1 -- 172.21.15.131:0/2215004968 >> v1:172.21.15.131:6789/0 conn(0x7fcf3c069ca0 legacy=0x7fcf3c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.719+0000 7fcf40d9a700 1 -- 172.21.15.131:0/2215004968 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0c9420 con 0x7fcf3c159cf0 2023-12-26T21:37:42.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.719+0000 7fcf33fff700 1 -- 172.21.15.131:0/2215004968 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932879855 0 0) 0x7fcf3c0c8bf0 con 0x7fcf3c069ca0 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf40d9a700 1 --2- 172.21.15.131:0/2215004968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c159cf0 0x7fcf3c0b0620 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0098e0 tx=0x7fcf2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cad66f485d045d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf33fff700 1 -- 172.21.15.131:0/2215004968 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00b640 con 0x7fcf3c159cf0 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf33fff700 1 -- 172.21.15.131:0/2215004968 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c00b7a0 con 0x7fcf3c159cf0 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf33fff700 1 -- 172.21.15.131:0/2215004968 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00bb40 con 0x7fcf3c159cf0 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf437ff700 1 -- 172.21.15.131:0/2215004968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c159cf0 msgr2=0x7fcf3c0b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf437ff700 1 --2- 172.21.15.131:0/2215004968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c159cf0 0x7fcf3c0b0620 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0098e0 tx=0x7fcf2c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf437ff700 1 -- 172.21.15.131:0/2215004968 shutdown_connections 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf437ff700 1 --2- 172.21.15.131:0/2215004968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c159920 0x7fcf3c0b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf437ff700 1 --2- 172.21.15.131:0/2215004968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c159cf0 0x7fcf3c0b0620 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf437ff700 1 -- 172.21.15.131:0/2215004968 >> 172.21.15.131:0/2215004968 conn(0x7fcf3c068040 msgr2=0x7fcf3c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:42.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf437ff700 1 -- 172.21.15.131:0/2215004968 shutdown_connections 2023-12-26T21:37:42.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.720+0000 7fcf437ff700 1 -- 172.21.15.131:0/2215004968 wait complete. 2023-12-26T21:37:42.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.722+0000 7fcf437ff700 1 Processor -- start 2023-12-26T21:37:42.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.722+0000 7fcf437ff700 1 -- start start 2023-12-26T21:37:42.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf437ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c069ca0 0x7fcf3c0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf437ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c159920 0x7fcf3c154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf437ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c159cf0 0x7fcf3c0c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf437ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0c70e0 con 0x7fcf3c069ca0 2023-12-26T21:37:42.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf437ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0ad480 con 0x7fcf3c159920 2023-12-26T21:37:42.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf437ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0c64d0 con 0x7fcf3c159cf0 2023-12-26T21:37:42.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf4159b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c069ca0 0x7fcf3c0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf4159b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c069ca0 0x7fcf3c0c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56398/0 (socket says 172.21.15.131:56398) 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf4159b700 1 -- 172.21.15.131:0/376152026 learned_addr learned my addr 172.21.15.131:0/376152026 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf4159b700 1 -- 172.21.15.131:0/376152026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c159cf0 msgr2=0x7fcf3c0c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf4159b700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c159cf0 0x7fcf3c0c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf4159b700 1 -- 172.21.15.131:0/376152026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c159920 msgr2=0x7fcf3c154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.723+0000 7fcf4159b700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c159920 0x7fcf3c154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.724+0000 7fcf4159b700 1 -- 172.21.15.131:0/376152026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c009420 con 0x7fcf3c069ca0 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.724+0000 7fcf4159b700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c069ca0 0x7fcf3c0c7570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800ec80 tx=0x7fcf380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.724+0000 7fcf317fa700 1 -- 172.21.15.131:0/376152026 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800c9d0 con 0x7fcf3c069ca0 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.724+0000 7fcf317fa700 1 -- 172.21.15.131:0/376152026 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3800cb30 con 0x7fcf3c069ca0 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.724+0000 7fcf317fa700 1 -- 172.21.15.131:0/376152026 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38013470 con 0x7fcf3c069ca0 2023-12-26T21:37:42.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.724+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c0c67b0 con 0x7fcf3c069ca0 2023-12-26T21:37:42.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.724+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c0c6d70 con 0x7fcf3c069ca0 2023-12-26T21:37:42.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.725+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c0657e0 con 0x7fcf3c069ca0 2023-12-26T21:37:42.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.726+0000 7fcf317fa700 1 -- 172.21.15.131:0/376152026 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcf38013820 con 0x7fcf3c069ca0 2023-12-26T21:37:42.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.727+0000 7fcf317fa700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcf20066f30 0x7fcf200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:42.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.727+0000 7fcf317fa700 1 -- 172.21.15.131:0/376152026 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcf38084610 con 0x7fcf3c069ca0 2023-12-26T21:37:42.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.727+0000 7fcf40d9a700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcf20066f30 0x7fcf200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:42.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.728+0000 7fcf40d9a700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcf20066f30 0x7fcf200693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0c8000 tx=0x7fcf2c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:42.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:42.731+0000 7fcf317fa700 1 -- 172.21.15.131:0/376152026 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf38051b30 con 0x7fcf3c069ca0 2023-12-26T21:37:43.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.005+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fcf3c0c9700 con 0x7fcf3c069ca0 2023-12-26T21:37:43.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.008+0000 7fcf317fa700 1 -- 172.21.15.131:0/376152026 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcf38055150 con 0x7fcf3c069ca0 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcf20066f30 msgr2=0x7fcf200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcf20066f30 0x7fcf200693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0c8000 tx=0x7fcf2c017000 comp rx=0 tx=0).stop 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c069ca0 msgr2=0x7fcf3c0c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c069ca0 0x7fcf3c0c7570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800ec80 tx=0x7fcf380091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 shutdown_connections 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcf20066f30 0x7fcf200693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c159cf0 0x7fcf3c0c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3c069ca0 0x7fcf3c0c7570 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 --2- 172.21.15.131:0/376152026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c159920 0x7fcf3c154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 >> 172.21.15.131:0/376152026 conn(0x7fcf3c068040 msgr2=0x7fcf3c0caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 shutdown_connections 2023-12-26T21:37:43.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.011+0000 7fcf437ff700 1 -- 172.21.15.131:0/376152026 wait complete. 2023-12-26T21:37:43.029 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-26T21:37:43.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.219+0000 7f344c9ff700 1 Processor -- start 2023-12-26T21:37:43.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.219+0000 7f344c9ff700 1 -- start start 2023-12-26T21:37:43.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.220+0000 7f344c9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34480cb320 0x7f3448069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.220+0000 7f344c9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f34480b0e60 0x7f34480c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.220+0000 7f344c9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3448156940 con 0x7f344806a1e0 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.220+0000 7f344c9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34480c8cc0 con 0x7f34480b0e60 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.220+0000 7f344c9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34480ad700 con 0x7f34480cb320 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.220+0000 7f3445d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f344806a1e0 0x7f34480b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43070/0 (socket says 172.21.15.131:43070) 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f3445d9b700 1 -- 172.21.15.131:0/940142058 learned_addr learned my addr 172.21.15.131:0/940142058 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f3446d9d700 1 --2- 172.21.15.131:0/940142058 >> v2:172.21.15.131:3300/0 conn(0x7f34480b0e60 0x7f34480c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f344659c700 1 --2- 172.21.15.131:0/940142058 >> v2:172.21.15.190:3300/0 conn(0x7f34480cb320 0x7f3448069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f3446d9d700 1 -- 172.21.15.131:0/940142058 >> v2:172.21.15.190:3300/0 conn(0x7f34480cb320 msgr2=0x7f3448069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f3446d9d700 1 --2- 172.21.15.131:0/940142058 >> v2:172.21.15.190:3300/0 conn(0x7f34480cb320 0x7f3448069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f344559a700 1 -- 172.21.15.131:0/940142058 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289313790 0 0) 0x7f3448156940 con 0x7f344806a1e0 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f3446d9d700 1 -- 172.21.15.131:0/940142058 >> v1:172.21.15.131:6789/0 conn(0x7f344806a1e0 legacy=0x7f34480b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f3446d9d700 1 -- 172.21.15.131:0/940142058 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34480ae500 con 0x7f34480b0e60 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f3446d9d700 1 --2- 172.21.15.131:0/940142058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34480b0e60 0x7f34480c9400 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3448064410 tx=0x7f3440009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7f8262e6950b96 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f344559a700 1 -- 172.21.15.131:0/940142058 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344000e070 con 0x7f34480b0e60 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f344559a700 1 -- 172.21.15.131:0/940142058 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344000bc30 con 0x7f34480b0e60 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.221+0000 7f344559a700 1 -- 172.21.15.131:0/940142058 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440012430 con 0x7f34480b0e60 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.222+0000 7f344c9ff700 1 -- 172.21.15.131:0/940142058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34480b0e60 msgr2=0x7f34480c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.222+0000 7f344c9ff700 1 --2- 172.21.15.131:0/940142058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34480b0e60 0x7f34480c9400 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3448064410 tx=0x7f3440009780 comp rx=0 tx=0).stop 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.222+0000 7f344c9ff700 1 -- 172.21.15.131:0/940142058 shutdown_connections 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.222+0000 7f344c9ff700 1 --2- 172.21.15.131:0/940142058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34480b0e60 0x7f34480c9400 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.222+0000 7f344c9ff700 1 --2- 172.21.15.131:0/940142058 >> v2:172.21.15.190:3300/0 conn(0x7f34480cb320 0x7f3448069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.222+0000 7f344c9ff700 1 -- 172.21.15.131:0/940142058 >> 172.21.15.131:0/940142058 conn(0x7f3448068040 msgr2=0x7f3448069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.222+0000 7f344c9ff700 1 -- 172.21.15.131:0/940142058 shutdown_connections 2023-12-26T21:37:43.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.222+0000 7f344c9ff700 1 -- 172.21.15.131:0/940142058 wait complete. 2023-12-26T21:37:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.224+0000 7f344c9ff700 1 Processor -- start 2023-12-26T21:37:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.224+0000 7f344c9ff700 1 -- start start 2023-12-26T21:37:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.225+0000 7f344c9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344806a1e0 0x7f34480a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344c9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34480b0e60 0x7f3448153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344c9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34480cb320 0x7f34480a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344c9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34480a79d0 con 0x7f344806a1e0 2023-12-26T21:37:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344c9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34480a7cb0 con 0x7f34480cb320 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344c9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34481567b0 con 0x7f34480b0e60 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344806a1e0 0x7f34480a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344806a1e0 0x7f34480a70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56414/0 (socket says 172.21.15.131:56414) 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344659c700 1 -- 172.21.15.131:0/2288610065 learned_addr learned my addr 172.21.15.131:0/2288610065 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344659c700 1 -- 172.21.15.131:0/2288610065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34480b0e60 msgr2=0x7f3448153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344659c700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34480b0e60 0x7f3448153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344659c700 1 -- 172.21.15.131:0/2288610065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34480cb320 msgr2=0x7f34480a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344659c700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34480cb320 0x7f34480a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344659c700 1 -- 172.21.15.131:0/2288610065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3440009a20 con 0x7f344806a1e0 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.226+0000 7f344659c700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344806a1e0 0x7f34480a70d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f343800d860 tx=0x7f3438008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.227+0000 7f34367fc700 1 -- 172.21.15.131:0/2288610065 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438009610 con 0x7f344806a1e0 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.227+0000 7f34367fc700 1 -- 172.21.15.131:0/2288610065 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3438009770 con 0x7f344806a1e0 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.227+0000 7f34367fc700 1 -- 172.21.15.131:0/2288610065 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438009b30 con 0x7f344806a1e0 2023-12-26T21:37:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.227+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3448156a90 con 0x7f344806a1e0 2023-12-26T21:37:43.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.227+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34480a7f90 con 0x7f344806a1e0 2023-12-26T21:37:43.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.228+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34480a7df0 con 0x7f344806a1e0 2023-12-26T21:37:43.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.229+0000 7f34367fc700 1 -- 172.21.15.131:0/2288610065 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f343801e460 con 0x7f344806a1e0 2023-12-26T21:37:43.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.229+0000 7f34367fc700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3428069140 0x7f342806b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.229+0000 7f34367fc700 1 -- 172.21.15.131:0/2288610065 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3438084850 con 0x7f344806a1e0 2023-12-26T21:37:43.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.230+0000 7f3445d9b700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3428069140 0x7f342806b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.230+0000 7f3445d9b700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3428069140 0x7f342806b5f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f343c0011e0 tx=0x7f343c0079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:43.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.233+0000 7f34367fc700 1 -- 172.21.15.131:0/2288610065 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34480a7df0 con 0x7f344806a1e0 2023-12-26T21:37:43.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.502+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f34480a7df0 con 0x7f344806a1e0 2023-12-26T21:37:43.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.506+0000 7f34367fc700 1 -- 172.21.15.131:0/2288610065 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f34480a7df0 con 0x7f344806a1e0 2023-12-26T21:37:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3428069140 msgr2=0x7f342806b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3428069140 0x7f342806b5f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f343c0011e0 tx=0x7f343c0079d0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344806a1e0 msgr2=0x7f34480a70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344806a1e0 0x7f34480a70d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f343800d860 tx=0x7f3438008040 comp rx=0 tx=0).stop 2023-12-26T21:37:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 shutdown_connections 2023-12-26T21:37:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34480cb320 0x7f34480a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f344806a1e0 0x7f34480a70d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3428069140 0x7f342806b5f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 --2- 172.21.15.131:0/2288610065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34480b0e60 0x7f3448153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.508+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 >> 172.21.15.131:0/2288610065 conn(0x7f3448068040 msgr2=0x7f344806d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.509+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 shutdown_connections 2023-12-26T21:37:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.509+0000 7f344c9ff700 1 -- 172.21.15.131:0/2288610065 wait complete. 2023-12-26T21:37:43.511 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:37:43.515 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:37:43.526 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-26T21:37:43.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.757+0000 7f1188dff700 1 Processor -- start 2023-12-26T21:37:43.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.757+0000 7f1188dff700 1 -- start start 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.758+0000 7f1188dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f11840cc580 0x7f11840cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.758+0000 7f1188dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11840b0560 0x7f118415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.758+0000 7f1188dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11840b0d70 con 0x7f11840b00e0 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.758+0000 7f1188dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11840680f0 con 0x7f11840cc580 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.758+0000 7f1188dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11840675b0 con 0x7f11840b0560 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.758+0000 7f118259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f11840cc580 0x7f11840cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.758+0000 7f118259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f11840cc580 0x7f11840cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56424/0 (socket says 172.21.15.131:56424) 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f118259c700 1 -- 172.21.15.131:0/3354011309 learned_addr learned my addr 172.21.15.131:0/3354011309 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f1182d9d700 1 --2- 172.21.15.131:0/3354011309 >> v2:172.21.15.190:3300/0 conn(0x7f11840b0560 0x7f118415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f118259c700 1 -- 172.21.15.131:0/3354011309 >> v2:172.21.15.190:3300/0 conn(0x7f11840b0560 msgr2=0x7f118415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f118259c700 1 --2- 172.21.15.131:0/3354011309 >> v2:172.21.15.190:3300/0 conn(0x7f11840b0560 0x7f118415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f118259c700 1 -- 172.21.15.131:0/3354011309 >> v1:172.21.15.190:6789/0 conn(0x7f11840b00e0 legacy=0x7f118406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f118259c700 1 -- 172.21.15.131:0/3354011309 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1184159c30 con 0x7f11840cc580 2023-12-26T21:37:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f1182d9d700 1 --2- 172.21.15.131:0/3354011309 >> v2:172.21.15.190:3300/0 conn(0x7f11840b0560 0x7f118415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f118259c700 1 --2- 172.21.15.131:0/3354011309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 0x7f11840cc950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11840b1330 tx=0x7f1174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e4885d71d902a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f118159a700 1 -- 172.21.15.131:0/3354011309 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400b820 con 0x7f11840cc580 2023-12-26T21:37:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f118159a700 1 -- 172.21.15.131:0/3354011309 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117400b980 con 0x7f11840cc580 2023-12-26T21:37:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.759+0000 7f118159a700 1 -- 172.21.15.131:0/3354011309 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400bd20 con 0x7f11840cc580 2023-12-26T21:37:43.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.760+0000 7f1188dff700 1 -- 172.21.15.131:0/3354011309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 msgr2=0x7f11840cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.760+0000 7f1188dff700 1 --2- 172.21.15.131:0/3354011309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 0x7f11840cc950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11840b1330 tx=0x7f1174009250 comp rx=0 tx=0).stop 2023-12-26T21:37:43.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.760+0000 7f1188dff700 1 -- 172.21.15.131:0/3354011309 shutdown_connections 2023-12-26T21:37:43.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.760+0000 7f1188dff700 1 --2- 172.21.15.131:0/3354011309 >> v2:172.21.15.190:3300/0 conn(0x7f11840b0560 0x7f118415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.760+0000 7f1188dff700 1 --2- 172.21.15.131:0/3354011309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 0x7f11840cc950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.760+0000 7f1188dff700 1 -- 172.21.15.131:0/3354011309 >> 172.21.15.131:0/3354011309 conn(0x7f11840686c0 msgr2=0x7f1184068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:43.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.760+0000 7f1188dff700 1 -- 172.21.15.131:0/3354011309 shutdown_connections 2023-12-26T21:37:43.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.760+0000 7f1188dff700 1 -- 172.21.15.131:0/3354011309 wait complete. 2023-12-26T21:37:43.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.762+0000 7f1188dff700 1 Processor -- start 2023-12-26T21:37:43.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.762+0000 7f1188dff700 1 -- start start 2023-12-26T21:37:43.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1188dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11840b00e0 0x7f118415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1188dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11840b0560 0x7f1184154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1188dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 0x7f118406ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1188dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1184159e50 con 0x7f11840cc580 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1188dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118415a6f0 con 0x7f11840b0560 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1188dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118415efb0 con 0x7f11840b00e0 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1182d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 0x7f118406ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1182d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 0x7f118406ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56426/0 (socket says 172.21.15.131:56426) 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1182d9d700 1 -- 172.21.15.131:0/438635646 learned_addr learned my addr 172.21.15.131:0/438635646 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f118259c700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11840b00e0 0x7f118415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1181d9b700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11840b0560 0x7f1184154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1182d9d700 1 -- 172.21.15.131:0/438635646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11840b00e0 msgr2=0x7f118415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1182d9d700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11840b00e0 0x7f118415e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1182d9d700 1 -- 172.21.15.131:0/438635646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11840b0560 msgr2=0x7f1184154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1182d9d700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11840b0560 0x7f1184154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1182d9d700 1 -- 172.21.15.131:0/438635646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740094f0 con 0x7f11840cc580 2023-12-26T21:37:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.763+0000 7f1181d9b700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11840b0560 0x7f1184154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.764+0000 7f1182d9d700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 0x7f118406ec00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f117c00cfb0 tx=0x7f117c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.764+0000 7f11727fc700 1 -- 172.21.15.131:0/438635646 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c015440 con 0x7f11840cc580 2023-12-26T21:37:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.764+0000 7f11727fc700 1 -- 172.21.15.131:0/438635646 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c0155a0 con 0x7f11840cc580 2023-12-26T21:37:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.764+0000 7f11727fc700 1 -- 172.21.15.131:0/438635646 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c015960 con 0x7f11840cc580 2023-12-26T21:37:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.764+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118415a0a0 con 0x7f11840cc580 2023-12-26T21:37:43.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.764+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840cbbd0 con 0x7f11840cc580 2023-12-26T21:37:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.765+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118415a860 con 0x7f11840cc580 2023-12-26T21:37:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.766+0000 7f11727fc700 1 -- 172.21.15.131:0/438635646 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f117c003020 con 0x7f11840cc580 2023-12-26T21:37:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.767+0000 7f11727fc700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1164066f60 0x7f1164069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.767+0000 7f11727fc700 1 -- 172.21.15.131:0/438635646 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f117c0849b0 con 0x7f11840cc580 2023-12-26T21:37:43.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.767+0000 7f118259c700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1164066f60 0x7f1164069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:43.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.767+0000 7f118259c700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1164066f60 0x7f1164069410 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11840b1330 tx=0x7f1174016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:43.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:43.770+0000 7f11727fc700 1 -- 172.21.15.131:0/438635646 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f117c04d5a0 con 0x7f11840cc580 2023-12-26T21:37:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.047+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f11840657e0 con 0x7f11840cc580 2023-12-26T21:37:44.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.050+0000 7f11727fc700 1 -- 172.21.15.131:0/438635646 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f117c051f50 con 0x7f11840cc580 2023-12-26T21:37:44.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1164066f60 msgr2=0x7f1164069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1164066f60 0x7f1164069410 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11840b1330 tx=0x7f1174016000 comp rx=0 tx=0).stop 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 msgr2=0x7f118406ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 0x7f118406ec00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f117c00cfb0 tx=0x7f117c0078c0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 shutdown_connections 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11840cc580 0x7f118406ec00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1164066f60 0x7f1164069410 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11840b00e0 0x7f118415e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 --2- 172.21.15.131:0/438635646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11840b0560 0x7f1184154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 >> 172.21.15.131:0/438635646 conn(0x7f11840686c0 msgr2=0x7f11840ab2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 shutdown_connections 2023-12-26T21:37:44.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.053+0000 7f1188dff700 1 -- 172.21.15.131:0/438635646 wait complete. 2023-12-26T21:37:44.070 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-26T21:37:44.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.261+0000 7f86405ff700 1 Processor -- start 2023-12-26T21:37:44.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.261+0000 7f86405ff700 1 -- start start 2023-12-26T21:37:44.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.262+0000 7f86405ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86380c8820 0x7f86380ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.262+0000 7f86405ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86380c8bf0 0x7f86380b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.262+0000 7f86405ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8638157a80 con 0x7f8638069d10 2023-12-26T21:37:44.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.262+0000 7f86405ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86380cac50 con 0x7f86380c8820 2023-12-26T21:37:44.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.262+0000 7f86405ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86380caf30 con 0x7f86380c8bf0 2023-12-26T21:37:44.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863db9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86380c8bf0 0x7f86380b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863eb9c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8638069d10 0x7f8638157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:53372/0 (socket says 172.21.15.131:53372) 2023-12-26T21:37:44.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863eb9c700 1 -- 172.21.15.131:0/3113760090 learned_addr learned my addr 172.21.15.131:0/3113760090 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863db9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86380c8bf0 0x7f86380b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:38216/0 (socket says 172.21.15.131:38216) 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863e39b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86380c8820 0x7f86380ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863d399700 1 -- 172.21.15.131:0/3113760090 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650415686 0 0) 0x7f8638157a80 con 0x7f8638069d10 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863d399700 1 -- 172.21.15.131:0/3113760090 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8624003570 con 0x7f8638069d10 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863db9a700 1 -- 172.21.15.131:0/3113760090 >> v2:172.21.15.190:3300/0 conn(0x7f86380c8820 msgr2=0x7f86380ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863db9a700 1 --2- 172.21.15.131:0/3113760090 >> v2:172.21.15.190:3300/0 conn(0x7f86380c8820 0x7f86380ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863db9a700 1 -- 172.21.15.131:0/3113760090 >> v1:172.21.15.190:6789/0 conn(0x7f8638069d10 legacy=0x7f8638157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863db9a700 1 -- 172.21.15.131:0/3113760090 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86380cb210 con 0x7f86380c8bf0 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.263+0000 7f863e39b700 1 --2- 172.21.15.131:0/3113760090 >> v2:172.21.15.190:3300/0 conn(0x7f86380c8820 0x7f86380ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.264+0000 7f863db9a700 1 --2- 172.21.15.131:0/3113760090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86380c8bf0 0x7f86380b12e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86280098e0 tx=0x7f86280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e27093668ffbc989 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.264+0000 7f863d399700 1 -- 172.21.15.131:0/3113760090 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628011630 con 0x7f86380c8bf0 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.264+0000 7f863d399700 1 -- 172.21.15.131:0/3113760090 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8628011790 con 0x7f86380c8bf0 2023-12-26T21:37:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.264+0000 7f863d399700 1 -- 172.21.15.131:0/3113760090 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628011b30 con 0x7f86380c8bf0 2023-12-26T21:37:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.264+0000 7f86405ff700 1 -- 172.21.15.131:0/3113760090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86380c8bf0 msgr2=0x7f86380b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.264+0000 7f86405ff700 1 --2- 172.21.15.131:0/3113760090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86380c8bf0 0x7f86380b12e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86280098e0 tx=0x7f86280091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.265+0000 7f86405ff700 1 -- 172.21.15.131:0/3113760090 shutdown_connections 2023-12-26T21:37:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.265+0000 7f86405ff700 1 --2- 172.21.15.131:0/3113760090 >> v2:172.21.15.190:3300/0 conn(0x7f86380c8820 0x7f86380ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.265+0000 7f86405ff700 1 --2- 172.21.15.131:0/3113760090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86380c8bf0 0x7f86380b12e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.265+0000 7f86405ff700 1 -- 172.21.15.131:0/3113760090 >> 172.21.15.131:0/3113760090 conn(0x7f8638068040 msgr2=0x7f8638069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:44.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.265+0000 7f86405ff700 1 -- 172.21.15.131:0/3113760090 shutdown_connections 2023-12-26T21:37:44.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.265+0000 7f86405ff700 1 -- 172.21.15.131:0/3113760090 wait complete. 2023-12-26T21:37:44.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.267+0000 7f86405ff700 1 Processor -- start 2023-12-26T21:37:44.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.267+0000 7f86405ff700 1 -- start start 2023-12-26T21:37:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f86405ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8638069d10 0x7f86381515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f86405ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86380c8820 0x7f8638152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f86405ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86380c8bf0 0x7f863815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f86405ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8638152850 con 0x7f86380c8bf0 2023-12-26T21:37:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f86405ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863813f2b0 con 0x7f86380c8820 2023-12-26T21:37:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f86405ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8638151c40 con 0x7f8638069d10 2023-12-26T21:37:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f863eb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86380c8bf0 0x7f863815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f863eb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86380c8bf0 0x7f863815d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56440/0 (socket says 172.21.15.131:56440) 2023-12-26T21:37:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f863eb9c700 1 -- 172.21.15.131:0/3240238713 learned_addr learned my addr 172.21.15.131:0/3240238713 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:44.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.268+0000 7f863e39b700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8638069d10 0x7f86381515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f863db9a700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86380c8820 0x7f8638152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f863eb9c700 1 -- 172.21.15.131:0/3240238713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8638069d10 msgr2=0x7f86381515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f863eb9c700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8638069d10 0x7f86381515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f863eb9c700 1 -- 172.21.15.131:0/3240238713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86380c8820 msgr2=0x7f8638152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f863eb9c700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86380c8820 0x7f8638152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f863eb9c700 1 -- 172.21.15.131:0/3240238713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634009420 con 0x7f86380c8bf0 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f863eb9c700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86380c8bf0 0x7f863815d550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8630001460 tx=0x7f8630001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f862e7fc700 1 -- 172.21.15.131:0/3240238713 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000a870 con 0x7f86380c8bf0 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f862e7fc700 1 -- 172.21.15.131:0/3240238713 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863000a9d0 con 0x7f86380c8bf0 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8628009420 con 0x7f86380c8bf0 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.269+0000 7f862e7fc700 1 -- 172.21.15.131:0/3240238713 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000ad50 con 0x7f86380c8bf0 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.270+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8638152200 con 0x7f86380c8bf0 2023-12-26T21:37:44.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.270+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8638152b30 con 0x7f86380c8bf0 2023-12-26T21:37:44.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.271+0000 7f862e7fc700 1 -- 172.21.15.131:0/3240238713 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8630007300 con 0x7f86380c8bf0 2023-12-26T21:37:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.272+0000 7f862e7fc700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f861c066f30 0x7f861c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.272+0000 7f862e7fc700 1 -- 172.21.15.131:0/3240238713 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8630084520 con 0x7f86380c8bf0 2023-12-26T21:37:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.273+0000 7f863e39b700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f861c066f30 0x7f861c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.273+0000 7f863e39b700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f861c066f30 0x7f861c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8634009ea0 tx=0x7f86340091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:44.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.277+0000 7f862e7fc700 1 -- 172.21.15.131:0/3240238713 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8630051ac0 con 0x7f86380c8bf0 2023-12-26T21:37:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.547+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f86380657e0 con 0x7f86380c8bf0 2023-12-26T21:37:44.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.551+0000 7f862e7fc700 1 -- 172.21.15.131:0/3240238713 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f86300550e0 con 0x7f86380c8bf0 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f861c066f30 msgr2=0x7f861c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f861c066f30 0x7f861c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8634009ea0 tx=0x7f86340091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86380c8bf0 msgr2=0x7f863815d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86380c8bf0 0x7f863815d550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8630001460 tx=0x7f8630001a40 comp rx=0 tx=0).stop 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 shutdown_connections 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86380c8bf0 0x7f863815d550 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f861c066f30 0x7f861c0693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8638069d10 0x7f86381515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 --2- 172.21.15.131:0/3240238713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86380c8820 0x7f8638152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 >> 172.21.15.131:0/3240238713 conn(0x7f8638068040 msgr2=0x7f86380cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 shutdown_connections 2023-12-26T21:37:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.554+0000 7f86405ff700 1 -- 172.21.15.131:0/3240238713 wait complete. 2023-12-26T21:37:44.571 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-26T21:37:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.761+0000 7f4e2bbff700 1 Processor -- start 2023-12-26T21:37:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.761+0000 7f4e2bbff700 1 -- start start 2023-12-26T21:37:44.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.762+0000 7f4e2bbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4e240b0750 0x7f4e240b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.762+0000 7f4e2bbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e24156940 con 0x7f4e240c9400 2023-12-26T21:37:44.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.762+0000 7f4e2bbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e240ae500 con 0x7f4e240cb320 2023-12-26T21:37:44.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.762+0000 7f4e2bbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e240c8cc0 con 0x7f4e240b0750 2023-12-26T21:37:44.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e2a19c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4e240c9400 0x7f4e240c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43078/0 (socket says 172.21.15.131:43078) 2023-12-26T21:37:44.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e2a19c700 1 -- 172.21.15.131:0/1274410283 learned_addr learned my addr 172.21.15.131:0/1274410283 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e2919a700 1 --2- 172.21.15.131:0/1274410283 >> v2:172.21.15.190:3300/0 conn(0x7f4e240b0750 0x7f4e240b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512288652 0 0) 0x7f4e24156940 con 0x7f4e240c9400 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e10003570 con 0x7f4e240c9400 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199830779 0 0) 0x7f4e240ae500 con 0x7f4e240cb320 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e24156940 con 0x7f4e240cb320 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3829203979 0 0) 0x7f4e10003570 con 0x7f4e240c9400 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 >> v2:172.21.15.190:3300/0 conn(0x7f4e240b0750 msgr2=0x7f4e240b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 --2- 172.21.15.131:0/1274410283 >> v2:172.21.15.190:3300/0 conn(0x7f4e240b0750 0x7f4e240b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 >> v1:172.21.15.190:6789/0 conn(0x7f4e240cb320 legacy=0x7f4e24069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240b0280 con 0x7f4e240c9400 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.763+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f4e1c0017f0 con 0x7f4e240c9400 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2919a700 1 --2- 172.21.15.131:0/1274410283 >> v2:172.21.15.190:3300/0 conn(0x7f4e240b0750 0x7f4e240b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 >> v1:172.21.15.131:6789/0 conn(0x7f4e240c9400 legacy=0x7f4e240c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e28999700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e10003bf0 0x7f4e10003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e28999700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e10004810 0x7f4e10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e28999700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e100051c0 0x7f4e1000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240c8cc0 con 0x7f4e10003bf0 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1000c0b0 con 0x7f4e10004810 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1000e480 con 0x7f4e100051c0 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2999b700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e10003bf0 0x7f4e10003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2919a700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e10004810 0x7f4e10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2a19c700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e100051c0 0x7f4e1000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2999b700 1 -- 172.21.15.131:0/1274410283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e100051c0 msgr2=0x7f4e1000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2999b700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e100051c0 0x7f4e1000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2999b700 1 -- 172.21.15.131:0/1274410283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e10004810 msgr2=0x7f4e10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2999b700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e10004810 0x7f4e10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2999b700 1 -- 172.21.15.131:0/1274410283 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e10003570 con 0x7f4e10003bf0 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.764+0000 7f4e2a19c700 1 -- 172.21.15.131:0/1274410283 reap_dead start 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e2999b700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e10003bf0 0x7f4e10003fe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e20001a10 tx=0x7f4e20007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b333113eee98fe37 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000c5d0 con 0x7f4e10003bf0 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2000c730 con 0x7f4e10003bf0 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e28999700 1 -- 172.21.15.131:0/1274410283 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000cad0 con 0x7f4e10003bf0 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1274410283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e10003bf0 msgr2=0x7f4e10003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e2bbff700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e10003bf0 0x7f4e10003fe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e20001a10 tx=0x7f4e20007d30 comp rx=0 tx=0).stop 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1274410283 shutdown_connections 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e2bbff700 1 --2- 172.21.15.131:0/1274410283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e10003bf0 0x7f4e10003fe0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1274410283 >> 172.21.15.131:0/1274410283 conn(0x7f4e24068040 msgr2=0x7f4e24069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1274410283 shutdown_connections 2023-12-26T21:37:44.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.765+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1274410283 wait complete. 2023-12-26T21:37:44.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.767+0000 7f4e2bbff700 1 Processor -- start 2023-12-26T21:37:44.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.767+0000 7f4e2bbff700 1 -- start start 2023-12-26T21:37:44.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e10003bf0 0x7f4e24158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2bbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e240ac710 0x7f4e2415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2bbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e240acb80 0x7f4e2415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2bbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2415bdd0 con 0x7f4e240acb80 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2bbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e24158d00 con 0x7f4e240ac710 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e24158fe0 con 0x7f4e10003bf0 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2a19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e240acb80 0x7f4e2415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2a19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e240acb80 0x7f4e2415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56458/0 (socket says 172.21.15.131:56458) 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2a19c700 1 -- 172.21.15.131:0/1967962544 learned_addr learned my addr 172.21.15.131:0/1967962544 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2a19c700 1 -- 172.21.15.131:0/1967962544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e10003bf0 msgr2=0x7f4e24158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2a19c700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e10003bf0 0x7f4e24158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2a19c700 1 -- 172.21.15.131:0/1967962544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e240ac710 msgr2=0x7f4e2415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2a19c700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e240ac710 0x7f4e2415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.768+0000 7f4e2a19c700 1 -- 172.21.15.131:0/1967962544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e241592c0 con 0x7f4e240acb80 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.769+0000 7f4e2a19c700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e240acb80 0x7f4e2415b890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c000f80 tx=0x7f4e240c8b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.769+0000 7f4e19ffb700 1 -- 172.21.15.131:0/1967962544 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00ebc0 con 0x7f4e240acb80 2023-12-26T21:37:44.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.769+0000 7f4e19ffb700 1 -- 172.21.15.131:0/1967962544 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c00ed40 con 0x7f4e240acb80 2023-12-26T21:37:44.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.769+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e241595a0 con 0x7f4e240acb80 2023-12-26T21:37:44.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.769+0000 7f4e19ffb700 1 -- 172.21.15.131:0/1967962544 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c013670 con 0x7f4e240acb80 2023-12-26T21:37:44.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.769+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e241588d0 con 0x7f4e240acb80 2023-12-26T21:37:44.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.770+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2415d1a0 con 0x7f4e240acb80 2023-12-26T21:37:44.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.771+0000 7f4e19ffb700 1 -- 172.21.15.131:0/1967962544 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4e1c0139e0 con 0x7f4e240acb80 2023-12-26T21:37:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.772+0000 7f4e19ffb700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4e08066f30 0x7f4e080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.773+0000 7f4e19ffb700 1 -- 172.21.15.131:0/1967962544 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4e1c0846a0 con 0x7f4e240acb80 2023-12-26T21:37:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.773+0000 7f4e2999b700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4e08066f30 0x7f4e080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.773+0000 7f4e2999b700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4e08066f30 0x7f4e080693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e20001a10 tx=0x7f4e20015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:44.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:44.776+0000 7f4e19ffb700 1 -- 172.21.15.131:0/1967962544 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e1c051c60 con 0x7f4e240acb80 2023-12-26T21:37:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.046+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f4e240657e0 con 0x7f4e240acb80 2023-12-26T21:37:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.050+0000 7f4e19ffb700 1 -- 172.21.15.131:0/1967962544 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4e1c055280 con 0x7f4e240acb80 2023-12-26T21:37:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4e08066f30 msgr2=0x7f4e080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4e08066f30 0x7f4e080693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e20001a10 tx=0x7f4e20015000 comp rx=0 tx=0).stop 2023-12-26T21:37:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e240acb80 msgr2=0x7f4e2415b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e240acb80 0x7f4e2415b890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c000f80 tx=0x7f4e240c8b50 comp rx=0 tx=0).stop 2023-12-26T21:37:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 shutdown_connections 2023-12-26T21:37:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e240acb80 0x7f4e2415b890 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4e08066f30 0x7f4e080693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e10003bf0 0x7f4e24158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 --2- 172.21.15.131:0/1967962544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e240ac710 0x7f4e2415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 >> 172.21.15.131:0/1967962544 conn(0x7f4e24068040 msgr2=0x7f4e241554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:45.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 shutdown_connections 2023-12-26T21:37:45.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.052+0000 7f4e2bbff700 1 -- 172.21.15.131:0/1967962544 wait complete. 2023-12-26T21:37:45.069 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-26T21:37:45.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.260+0000 7fadd0dff700 1 Processor -- start 2023-12-26T21:37:45.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.260+0000 7fadd0dff700 1 -- start start 2023-12-26T21:37:45.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.261+0000 7fadd0dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc159820 0x7fadcc0abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.261+0000 7fadd0dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fadcc0aa480 0x7fadcc06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.261+0000 7fadd0dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fadcc06e2b0 0x7fadcc15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.261+0000 7fadd0dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc15ad50 con 0x7fadcc06e2b0 2023-12-26T21:37:45.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.261+0000 7fadd0dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc06e730 con 0x7fadcc0aa480 2023-12-26T21:37:45.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.261+0000 7fadd0dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc15a8c0 con 0x7fadcc159820 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.261+0000 7fadcad9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fadcc06e2b0 0x7fadcc15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadcad9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fadcc06e2b0 0x7fadcc15d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60684/0 (socket says 172.21.15.131:60684) 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadcad9d700 1 -- 172.21.15.131:0/3186816066 learned_addr learned my addr 172.21.15.131:0/3186816066 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadc9d9b700 1 --2- 172.21.15.131:0/3186816066 >> v2:172.21.15.190:3300/0 conn(0x7fadcc0aa480 0x7fadcc06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadca59c700 1 --2- 172.21.15.131:0/3186816066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc159820 0x7fadcc0abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadcad9d700 1 -- 172.21.15.131:0/3186816066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc159820 msgr2=0x7fadcc0abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadcad9d700 1 --2- 172.21.15.131:0/3186816066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc159820 0x7fadcc0abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadcad9d700 1 -- 172.21.15.131:0/3186816066 >> v2:172.21.15.190:3300/0 conn(0x7fadcc0aa480 msgr2=0x7fadcc06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadcad9d700 1 --2- 172.21.15.131:0/3186816066 >> v2:172.21.15.190:3300/0 conn(0x7fadcc0aa480 0x7fadcc06dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadcad9d700 1 -- 172.21.15.131:0/3186816066 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc159f90 con 0x7fadcc06e2b0 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadc9d9b700 1 --2- 172.21.15.131:0/3186816066 >> v2:172.21.15.190:3300/0 conn(0x7fadcc0aa480 0x7fadcc06dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadcad9d700 1 --2- 172.21.15.131:0/3186816066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 0x7fadcc15d2d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fadcc06d4a0 tx=0x7fadc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b05f9da5e956608 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.262+0000 7fadc959a700 1 -- 172.21.15.131:0/3186816066 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc400e070 con 0x7fadcc06e2b0 2023-12-26T21:37:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadc959a700 1 -- 172.21.15.131:0/3186816066 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc400ba40 con 0x7fadcc06e2b0 2023-12-26T21:37:45.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadc959a700 1 -- 172.21.15.131:0/3186816066 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc40092e0 con 0x7fadcc06e2b0 2023-12-26T21:37:45.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadd0dff700 1 -- 172.21.15.131:0/3186816066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 msgr2=0x7fadcc15d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadd0dff700 1 --2- 172.21.15.131:0/3186816066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 0x7fadcc15d2d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fadcc06d4a0 tx=0x7fadc4009840 comp rx=0 tx=0).stop 2023-12-26T21:37:45.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadd0dff700 1 -- 172.21.15.131:0/3186816066 shutdown_connections 2023-12-26T21:37:45.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadd0dff700 1 --2- 172.21.15.131:0/3186816066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 0x7fadcc15d2d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadd0dff700 1 --2- 172.21.15.131:0/3186816066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc159820 0x7fadcc0abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadd0dff700 1 --2- 172.21.15.131:0/3186816066 >> v2:172.21.15.190:3300/0 conn(0x7fadcc0aa480 0x7fadcc06dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadd0dff700 1 -- 172.21.15.131:0/3186816066 >> 172.21.15.131:0/3186816066 conn(0x7fadcc06c290 msgr2=0x7fadcc0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:45.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadd0dff700 1 -- 172.21.15.131:0/3186816066 shutdown_connections 2023-12-26T21:37:45.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.263+0000 7fadd0dff700 1 -- 172.21.15.131:0/3186816066 wait complete. 2023-12-26T21:37:45.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.265+0000 7fadd0dff700 1 Processor -- start 2023-12-26T21:37:45.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.265+0000 7fadd0dff700 1 -- start start 2023-12-26T21:37:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadd0dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 0x7fadcc0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadd0dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc0aa480 0x7fadcc0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadd0dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadcc159820 0x7fadcc15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadd0dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0b15d0 con 0x7fadcc06e2b0 2023-12-26T21:37:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadd0dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0cc820 con 0x7fadcc159820 2023-12-26T21:37:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadd0dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0b23d0 con 0x7fadcc0aa480 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadca59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 0x7fadcc0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadca59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 0x7fadcc0b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60694/0 (socket says 172.21.15.131:60694) 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadca59c700 1 -- 172.21.15.131:0/544682836 learned_addr learned my addr 172.21.15.131:0/544682836 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadc9d9b700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc0aa480 0x7fadcc0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadcad9d700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadcc159820 0x7fadcc15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadca59c700 1 -- 172.21.15.131:0/544682836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc0aa480 msgr2=0x7fadcc0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadca59c700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc0aa480 0x7fadcc0cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadca59c700 1 -- 172.21.15.131:0/544682836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadcc159820 msgr2=0x7fadcc15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadca59c700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadcc159820 0x7fadcc15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.266+0000 7fadca59c700 1 -- 172.21.15.131:0/544682836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc4009ae0 con 0x7fadcc06e2b0 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.267+0000 7fadca59c700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 0x7fadcc0b1090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00d8c0 tx=0x7fadbc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.267+0000 7fadba7fc700 1 -- 172.21.15.131:0/544682836 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0125f0 con 0x7fadcc06e2b0 2023-12-26T21:37:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.267+0000 7fadba7fc700 1 -- 172.21.15.131:0/544682836 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc012770 con 0x7fadcc06e2b0 2023-12-26T21:37:45.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.267+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc0b17c0 con 0x7fadcc06e2b0 2023-12-26T21:37:45.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.267+0000 7fadba7fc700 1 -- 172.21.15.131:0/544682836 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc012b30 con 0x7fadcc06e2b0 2023-12-26T21:37:45.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.267+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc0b1d80 con 0x7fadcc06e2b0 2023-12-26T21:37:45.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.268+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc0657e0 con 0x7fadcc06e2b0 2023-12-26T21:37:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.269+0000 7fadba7fc700 1 -- 172.21.15.131:0/544682836 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fadbc01e460 con 0x7fadcc06e2b0 2023-12-26T21:37:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.270+0000 7fadba7fc700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fadac066f30 0x7fadac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.270+0000 7fadba7fc700 1 -- 172.21.15.131:0/544682836 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fadbc0847f0 con 0x7fadcc06e2b0 2023-12-26T21:37:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.270+0000 7fadc9d9b700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fadac066f30 0x7fadac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.271+0000 7fadc9d9b700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fadac066f30 0x7fadac0693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0abfd0 tx=0x7fadc0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:45.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.274+0000 7fadba7fc700 1 -- 172.21.15.131:0/544682836 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadbc051d10 con 0x7fadcc06e2b0 2023-12-26T21:37:45.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.543+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fadcc15a270 con 0x7fadcc06e2b0 2023-12-26T21:37:45.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.547+0000 7fadba7fc700 1 -- 172.21.15.131:0/544682836 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fadbc055330 con 0x7fadcc06e2b0 2023-12-26T21:37:45.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.549+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fadac066f30 msgr2=0x7fadac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.549+0000 7fadd0dff700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fadac066f30 0x7fadac0693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0abfd0 tx=0x7fadc0008040 comp rx=0 tx=0).stop 2023-12-26T21:37:45.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.549+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 msgr2=0x7fadcc0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.549+0000 7fadd0dff700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 0x7fadcc0b1090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00d8c0 tx=0x7fadbc008040 comp rx=0 tx=0).stop 2023-12-26T21:37:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.550+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 shutdown_connections 2023-12-26T21:37:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.550+0000 7fadd0dff700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadcc159820 0x7fadcc15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.550+0000 7fadd0dff700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadcc06e2b0 0x7fadcc0b1090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.550+0000 7fadd0dff700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fadac066f30 0x7fadac0693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.550+0000 7fadd0dff700 1 --2- 172.21.15.131:0/544682836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadcc0aa480 0x7fadcc0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.550+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 >> 172.21.15.131:0/544682836 conn(0x7fadcc06c290 msgr2=0x7fadcc157610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.550+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 shutdown_connections 2023-12-26T21:37:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.550+0000 7fadd0dff700 1 -- 172.21.15.131:0/544682836 wait complete. 2023-12-26T21:37:45.567 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:45.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.755+0000 7f5ba9bff700 1 Processor -- start 2023-12-26T21:37:45.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.756+0000 7f5ba9bff700 1 -- start start 2023-12-26T21:37:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba9bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ba41596a0 0x7f5ba40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba9bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba4159a70 0x7f5ba40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba9bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba40afcd0 con 0x7f5ba40b0160 2023-12-26T21:37:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba9bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba40b3e70 con 0x7f5ba41596a0 2023-12-26T21:37:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba9bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40b0cd0 con 0x7f5ba4159a70 2023-12-26T21:37:45.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba37fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ba41596a0 0x7f5ba40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba3fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5ba40b0160 0x7f5ba40b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35664/0 (socket says 172.21.15.131:35664) 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba3fff700 1 -- 172.21.15.131:0/3597921342 learned_addr learned my addr 172.21.15.131:0/3597921342 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba37fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ba41596a0 0x7f5ba40b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60704/0 (socket says 172.21.15.131:60704) 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba2ffd700 1 --2- 172.21.15.131:0/3597921342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba4159a70 0x7f5ba40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba37fe700 1 -- 172.21.15.131:0/3597921342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba4159a70 msgr2=0x7f5ba40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba37fe700 1 --2- 172.21.15.131:0/3597921342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba4159a70 0x7f5ba40b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba37fe700 1 -- 172.21.15.131:0/3597921342 >> v1:172.21.15.131:6789/0 conn(0x7f5ba40b0160 legacy=0x7f5ba40b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.757+0000 7f5ba37fe700 1 -- 172.21.15.131:0/3597921342 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba40b2e00 con 0x7f5ba41596a0 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.758+0000 7f5ba37fe700 1 --2- 172.21.15.131:0/3597921342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba41596a0 0x7f5ba40b2380 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b940098e0 tx=0x7f5b940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49271c4328c5c600 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.758+0000 7f5ba27fc700 1 -- 172.21.15.131:0/3597921342 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94009e80 con 0x7f5ba41596a0 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.758+0000 7f5ba27fc700 1 -- 172.21.15.131:0/3597921342 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9400b860 con 0x7f5ba41596a0 2023-12-26T21:37:45.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.758+0000 7f5ba27fc700 1 -- 172.21.15.131:0/3597921342 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400bbe0 con 0x7f5ba41596a0 2023-12-26T21:37:45.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.758+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3597921342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba41596a0 msgr2=0x7f5ba40b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.758+0000 7f5ba9bff700 1 --2- 172.21.15.131:0/3597921342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba41596a0 0x7f5ba40b2380 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b940098e0 tx=0x7f5b940091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.758+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3597921342 shutdown_connections 2023-12-26T21:37:45.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.758+0000 7f5ba9bff700 1 --2- 172.21.15.131:0/3597921342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba41596a0 0x7f5ba40b2380 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.759+0000 7f5ba9bff700 1 --2- 172.21.15.131:0/3597921342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba4159a70 0x7f5ba40b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.759+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3597921342 >> 172.21.15.131:0/3597921342 conn(0x7f5ba406ece0 msgr2=0x7f5ba406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:45.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.759+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3597921342 shutdown_connections 2023-12-26T21:37:45.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.759+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3597921342 wait complete. 2023-12-26T21:37:45.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.760+0000 7f5ba9bff700 1 Processor -- start 2023-12-26T21:37:45.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.760+0000 7f5ba9bff700 1 -- start start 2023-12-26T21:37:45.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.761+0000 7f5ba9bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba40b0160 0x7f5ba40cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.761+0000 7f5ba9bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba41596a0 0x7f5ba4159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.761+0000 7f5ba9bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba4159a70 0x7f5ba415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.761+0000 7f5ba9bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40ca220 con 0x7f5ba40b0160 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.761+0000 7f5ba9bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40cc340 con 0x7f5ba4159a70 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.761+0000 7f5ba9bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40cc620 con 0x7f5ba41596a0 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.761+0000 7f5ba37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba40b0160 0x7f5ba40cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.761+0000 7f5ba37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba40b0160 0x7f5ba40cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60718/0 (socket says 172.21.15.131:60718) 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.761+0000 7f5ba37fe700 1 -- 172.21.15.131:0/3837623017 learned_addr learned my addr 172.21.15.131:0/3837623017 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba2ffd700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba41596a0 0x7f5ba4159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba3fff700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba4159a70 0x7f5ba415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba37fe700 1 -- 172.21.15.131:0/3837623017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba41596a0 msgr2=0x7f5ba4159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba37fe700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba41596a0 0x7f5ba4159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba37fe700 1 -- 172.21.15.131:0/3837623017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba4159a70 msgr2=0x7f5ba415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba37fe700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba4159a70 0x7f5ba415c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba37fe700 1 -- 172.21.15.131:0/3837623017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94009420 con 0x7f5ba40b0160 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba2ffd700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba41596a0 0x7f5ba4159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba37fe700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba40b0160 0x7f5ba40cce20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4089710 tx=0x7f5b94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5b8b7fe700 1 -- 172.21.15.131:0/3837623017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400bbe0 con 0x7f5ba40b0160 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5b8b7fe700 1 -- 172.21.15.131:0/3837623017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b94011420 con 0x7f5ba40b0160 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5b8b7fe700 1 -- 172.21.15.131:0/3837623017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940117e0 con 0x7f5ba40b0160 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba40cc900 con 0x7f5ba40b0160 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.762+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba416e190 con 0x7f5ba40b0160 2023-12-26T21:37:45.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.763+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba40cd360 con 0x7f5ba40b0160 2023-12-26T21:37:45.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.764+0000 7f5b8b7fe700 1 -- 172.21.15.131:0/3837623017 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5b94011bd0 con 0x7f5ba40b0160 2023-12-26T21:37:45.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.765+0000 7f5b8b7fe700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5b90066f30 0x7f5b900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:45.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.765+0000 7f5b8b7fe700 1 -- 172.21.15.131:0/3837623017 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5b94085cb0 con 0x7f5ba40b0160 2023-12-26T21:37:45.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.765+0000 7f5ba2ffd700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5b90066f30 0x7f5b900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:45.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.766+0000 7f5ba2ffd700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5b90066f30 0x7f5b900693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006fd0 tx=0x7f5b98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:45.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.769+0000 7f5b8b7fe700 1 -- 172.21.15.131:0/3837623017 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b940531d0 con 0x7f5ba40b0160 2023-12-26T21:37:45.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.964+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ba40657e0 con 0x7f5ba40b0160 2023-12-26T21:37:45.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.964+0000 7f5b8b7fe700 1 -- 172.21.15.131:0/3837623017 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f5b940567f0 con 0x7f5ba40b0160 2023-12-26T21:37:45.964 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:45.964 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":31,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:37:41.999873+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5b90066f30 msgr2=0x7f5b900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5b90066f30 0x7f5b900693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006fd0 tx=0x7f5b98008040 comp rx=0 tx=0).stop 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba40b0160 msgr2=0x7f5ba40cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba40b0160 0x7f5ba40cce20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4089710 tx=0x7f5b94017000 comp rx=0 tx=0).stop 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 shutdown_connections 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba4159a70 0x7f5ba415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba40b0160 0x7f5ba40cce20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5b90066f30 0x7f5b900693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 --2- 172.21.15.131:0/3837623017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ba41596a0 0x7f5ba4159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 >> 172.21.15.131:0/3837623017 conn(0x7f5ba406ece0 msgr2=0x7f5ba4158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 shutdown_connections 2023-12-26T21:37:45.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:45.967+0000 7f5ba9bff700 1 -- 172.21.15.131:0/3837623017 wait complete. 2023-12-26T21:37:45.985 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T21:37:45.985 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:37:46.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.177+0000 7ff7dffff700 1 Processor -- start 2023-12-26T21:37:46.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.178+0000 7ff7dffff700 1 -- start start 2023-12-26T21:37:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.179+0000 7ff7dffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7d813c510 0x7ff7d80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.179+0000 7ff7dffff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d80686c0 con 0x7ff7d806b8c0 2023-12-26T21:37:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.179+0000 7ff7dffff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d8069520 con 0x7ff7d80cc810 2023-12-26T21:37:46.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.179+0000 7ff7dffff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d8068910 con 0x7ff7d813c510 2023-12-26T21:37:46.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.179+0000 7ff7dd59a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff7d806b8c0 0x7ff7d80cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35666/0 (socket says 172.21.15.131:35666) 2023-12-26T21:37:46.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.179+0000 7ff7dd59a700 1 -- 172.21.15.131:0/3811906696 learned_addr learned my addr 172.21.15.131:0/3811906696 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:46.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.179+0000 7ff7ddd9b700 1 --2- 172.21.15.131:0/3811906696 >> v2:172.21.15.190:3300/0 conn(0x7ff7d813c510 0x7ff7d80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7dcd99700 1 -- 172.21.15.131:0/3811906696 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597623632 0 0) 0x7ff7d80686c0 con 0x7ff7d806b8c0 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7dcd99700 1 -- 172.21.15.131:0/3811906696 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c4003570 con 0x7ff7d806b8c0 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7dcd99700 1 -- 172.21.15.131:0/3811906696 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319897966 0 0) 0x7ff7d8069520 con 0x7ff7d80cc810 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7dcd99700 1 -- 172.21.15.131:0/3811906696 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d80686c0 con 0x7ff7d80cc810 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7ddd9b700 1 -- 172.21.15.131:0/3811906696 >> v1:172.21.15.190:6789/0 conn(0x7ff7d80cc810 legacy=0x7ff7d80699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7ddd9b700 1 -- 172.21.15.131:0/3811906696 >> v1:172.21.15.131:6789/0 conn(0x7ff7d806b8c0 legacy=0x7ff7d80cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7dcd99700 1 -- 172.21.15.131:0/3811906696 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3103491331 0 0) 0x7ff7c4003570 con 0x7ff7d806b8c0 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7ddd9b700 1 -- 172.21.15.131:0/3811906696 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d8068bf0 con 0x7ff7d813c510 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7ddd9b700 1 --2- 172.21.15.131:0/3811906696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d813c510 0x7ff7d80c8a20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40098d0 tx=0x7ff7d4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7a289ae8175d48 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7dcd99700 1 -- 172.21.15.131:0/3811906696 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8001470 con 0x7ff7d813c510 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7dcd99700 1 -- 172.21.15.131:0/3811906696 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d4011420 con 0x7ff7d813c510 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.180+0000 7ff7dcd99700 1 -- 172.21.15.131:0/3811906696 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40117c0 con 0x7ff7d813c510 2023-12-26T21:37:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.181+0000 7ff7dffff700 1 -- 172.21.15.131:0/3811906696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d813c510 msgr2=0x7ff7d80c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.181+0000 7ff7dffff700 1 --2- 172.21.15.131:0/3811906696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d813c510 0x7ff7d80c8a20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40098d0 tx=0x7ff7d4009120 comp rx=0 tx=0).stop 2023-12-26T21:37:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.181+0000 7ff7dffff700 1 -- 172.21.15.131:0/3811906696 shutdown_connections 2023-12-26T21:37:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.181+0000 7ff7dffff700 1 --2- 172.21.15.131:0/3811906696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d813c510 0x7ff7d80c8a20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.181+0000 7ff7dffff700 1 -- 172.21.15.131:0/3811906696 >> 172.21.15.131:0/3811906696 conn(0x7ff7d80ce3e0 msgr2=0x7ff7d80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.181+0000 7ff7dffff700 1 -- 172.21.15.131:0/3811906696 shutdown_connections 2023-12-26T21:37:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.181+0000 7ff7dffff700 1 -- 172.21.15.131:0/3811906696 wait complete. 2023-12-26T21:37:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.183+0000 7ff7dffff700 1 Processor -- start 2023-12-26T21:37:46.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.183+0000 7ff7dffff700 1 -- start start 2023-12-26T21:37:46.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.184+0000 7ff7dffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d806b8c0 0x7ff7d815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.184+0000 7ff7dffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80cc810 0x7ff7d8169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7dffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d813c510 0x7ff7d80c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7dffff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d816ab70 con 0x7ff7d813c510 2023-12-26T21:37:46.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7dffff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d816b060 con 0x7ff7d806b8c0 2023-12-26T21:37:46.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7dffff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8157de0 con 0x7ff7d80cc810 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7de59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d813c510 0x7ff7d80c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7de59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d813c510 0x7ff7d80c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60720/0 (socket says 172.21.15.131:60720) 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7de59c700 1 -- 172.21.15.131:0/4169701596 learned_addr learned my addr 172.21.15.131:0/4169701596 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7ddd9b700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d806b8c0 0x7ff7d815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7dd59a700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80cc810 0x7ff7d8169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7de59c700 1 -- 172.21.15.131:0/4169701596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80cc810 msgr2=0x7ff7d8169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7de59c700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80cc810 0x7ff7d8169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7de59c700 1 -- 172.21.15.131:0/4169701596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d806b8c0 msgr2=0x7ff7d815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7de59c700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d806b8c0 0x7ff7d815c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7de59c700 1 -- 172.21.15.131:0/4169701596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40093c0 con 0x7ff7d813c510 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.185+0000 7ff7de59c700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d813c510 0x7ff7d80c7290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c910 tx=0x7ff7d00027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.186+0000 7ff7cdffb700 1 -- 172.21.15.131:0/4169701596 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0012630 con 0x7ff7d813c510 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.186+0000 7ff7cdffb700 1 -- 172.21.15.131:0/4169701596 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d00127b0 con 0x7ff7d813c510 2023-12-26T21:37:46.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.186+0000 7ff7cdffb700 1 -- 172.21.15.131:0/4169701596 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0012b70 con 0x7ff7d813c510 2023-12-26T21:37:46.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.186+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d816a5b0 con 0x7ff7d813c510 2023-12-26T21:37:46.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.186+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d8157a70 con 0x7ff7d813c510 2023-12-26T21:37:46.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.187+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d8157f70 con 0x7ff7d813c510 2023-12-26T21:37:46.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.188+0000 7ff7cdffb700 1 -- 172.21.15.131:0/4169701596 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff7d001b400 con 0x7ff7d813c510 2023-12-26T21:37:46.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.189+0000 7ff7cdffb700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7bc066f30 0x7ff7bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.189+0000 7ff7cdffb700 1 -- 172.21.15.131:0/4169701596 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff7d00845e0 con 0x7ff7d813c510 2023-12-26T21:37:46.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.189+0000 7ff7ddd9b700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7bc066f30 0x7ff7bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.190+0000 7ff7ddd9b700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7bc066f30 0x7ff7bc0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009390 tx=0x7ff7d4016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:46.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.193+0000 7ff7cdffb700 1 -- 172.21.15.131:0/4169701596 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7d0051b00 con 0x7ff7d813c510 2023-12-26T21:37:46.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.385+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7d80657e0 con 0x7ff7d813c510 2023-12-26T21:37:46.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.386+0000 7ff7cdffb700 1 -- 172.21.15.131:0/4169701596 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7d0055120 con 0x7ff7d813c510 2023-12-26T21:37:46.386 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7bc066f30 msgr2=0x7ff7bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7bc066f30 0x7ff7bc0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009390 tx=0x7ff7d4016000 comp rx=0 tx=0).stop 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d813c510 msgr2=0x7ff7d80c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d813c510 0x7ff7d80c7290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c910 tx=0x7ff7d00027a0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 shutdown_connections 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d813c510 0x7ff7d80c7290 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d806b8c0 0x7ff7d815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7bc066f30 0x7ff7bc0693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 --2- 172.21.15.131:0/4169701596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80cc810 0x7ff7d8169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 >> 172.21.15.131:0/4169701596 conn(0x7ff7d80ce3e0 msgr2=0x7ff7d8152720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:46.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 shutdown_connections 2023-12-26T21:37:46.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.388+0000 7ff7dffff700 1 -- 172.21.15.131:0/4169701596 wait complete. 2023-12-26T21:37:46.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T21:37:46.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.596+0000 7f24bfdff700 1 Processor -- start 2023-12-26T21:37:46.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.596+0000 7f24bfdff700 1 -- start start 2023-12-26T21:37:46.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.597+0000 7f24bfdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24b806bfd0 0x7f24b80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.597+0000 7f24bfdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b80686c0 con 0x7f24b80cc100 2023-12-26T21:37:46.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.597+0000 7f24bfdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b8069520 con 0x7f24b813c510 2023-12-26T21:37:46.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.597+0000 7f24bfdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b8068910 con 0x7f24b806bfd0 2023-12-26T21:37:46.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.598+0000 7f24bd39a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24b806bfd0 0x7f24b80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.598+0000 7f24be39c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f24b80cc100 0x7f24b80cc520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35670/0 (socket says 172.21.15.131:35670) 2023-12-26T21:37:46.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.598+0000 7f24be39c700 1 -- 172.21.15.131:0/572305910 learned_addr learned my addr 172.21.15.131:0/572305910 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:46.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.598+0000 7f24bd39a700 1 -- 172.21.15.131:0/572305910 >> v1:172.21.15.190:6789/0 conn(0x7f24b813c510 legacy=0x7f24b806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.598+0000 7f24bd39a700 1 -- 172.21.15.131:0/572305910 >> v1:172.21.15.131:6789/0 conn(0x7f24b80cc100 legacy=0x7f24b80cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.598+0000 7f24bd39a700 1 -- 172.21.15.131:0/572305910 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8068bf0 con 0x7f24b806bfd0 2023-12-26T21:37:46.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.598+0000 7f24bcb99700 1 -- 172.21.15.131:0/572305910 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787187723 0 0) 0x7f24b80686c0 con 0x7f24b80cc100 2023-12-26T21:37:46.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.598+0000 7f24bd39a700 1 --2- 172.21.15.131:0/572305910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b806bfd0 0x7f24b80c8a20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f24a80098e0 tx=0x7f24a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b3c529a3de95c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:46.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.598+0000 7f24bcb99700 1 -- 172.21.15.131:0/572305910 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8009e80 con 0x7f24b806bfd0 2023-12-26T21:37:46.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.599+0000 7f24bcb99700 1 -- 172.21.15.131:0/572305910 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a800b860 con 0x7f24b806bfd0 2023-12-26T21:37:46.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.599+0000 7f24bcb99700 1 -- 172.21.15.131:0/572305910 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a800bbe0 con 0x7f24b806bfd0 2023-12-26T21:37:46.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.599+0000 7f24bfdff700 1 -- 172.21.15.131:0/572305910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b806bfd0 msgr2=0x7f24b80c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.599+0000 7f24bfdff700 1 --2- 172.21.15.131:0/572305910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b806bfd0 0x7f24b80c8a20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f24a80098e0 tx=0x7f24a80091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.599+0000 7f24bfdff700 1 -- 172.21.15.131:0/572305910 shutdown_connections 2023-12-26T21:37:46.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.599+0000 7f24bfdff700 1 --2- 172.21.15.131:0/572305910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b806bfd0 0x7f24b80c8a20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.599+0000 7f24bfdff700 1 -- 172.21.15.131:0/572305910 >> 172.21.15.131:0/572305910 conn(0x7f24b80ce3e0 msgr2=0x7f24b80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:46.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.599+0000 7f24bfdff700 1 -- 172.21.15.131:0/572305910 shutdown_connections 2023-12-26T21:37:46.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.599+0000 7f24bfdff700 1 -- 172.21.15.131:0/572305910 wait complete. 2023-12-26T21:37:46.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.601+0000 7f24bfdff700 1 Processor -- start 2023-12-26T21:37:46.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.601+0000 7f24bfdff700 1 -- start start 2023-12-26T21:37:46.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bfdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b806bfd0 0x7f24b8153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bfdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b80cc100 0x7f24b80b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bfdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b813c510 0x7f24b8155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bfdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b8154330 con 0x7f24b80cc100 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bfdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b8154eb0 con 0x7f24b813c510 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bfdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b8154580 con 0x7f24b806bfd0 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bd39a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b80cc100 0x7f24b80b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bd39a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b80cc100 0x7f24b80b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60730/0 (socket says 172.21.15.131:60730) 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bd39a700 1 -- 172.21.15.131:0/3740154338 learned_addr learned my addr 172.21.15.131:0/3740154338 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24be39c700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b813c510 0x7f24b8155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bdb9b700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b806bfd0 0x7f24b8153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bd39a700 1 -- 172.21.15.131:0/3740154338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b806bfd0 msgr2=0x7f24b8153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bd39a700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b806bfd0 0x7f24b8153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bd39a700 1 -- 172.21.15.131:0/3740154338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b813c510 msgr2=0x7f24b8155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bd39a700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b813c510 0x7f24b8155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bd39a700 1 -- 172.21.15.131:0/3740154338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a8009420 con 0x7f24b80cc100 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.602+0000 7f24bdb9b700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b806bfd0 0x7f24b8153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.603+0000 7f24bd39a700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b80cc100 0x7f24b80b1ea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f24a8013000 tx=0x7f24a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.603+0000 7f24adffb700 1 -- 172.21.15.131:0/3740154338 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a801f070 con 0x7f24b80cc100 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.603+0000 7f24adffb700 1 -- 172.21.15.131:0/3740154338 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a800b9d0 con 0x7f24b80cc100 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.603+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b8154860 con 0x7f24b80cc100 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.603+0000 7f24adffb700 1 -- 172.21.15.131:0/3740154338 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8025670 con 0x7f24b80cc100 2023-12-26T21:37:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.603+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b8153fc0 con 0x7f24b80cc100 2023-12-26T21:37:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.604+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b8155020 con 0x7f24b80cc100 2023-12-26T21:37:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.605+0000 7f24adffb700 1 -- 172.21.15.131:0/3740154338 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f24a8023070 con 0x7f24b80cc100 2023-12-26T21:37:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.606+0000 7f24adffb700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f249c066f30 0x7f249c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.606+0000 7f24adffb700 1 -- 172.21.15.131:0/3740154338 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f24a8085820 con 0x7f24b80cc100 2023-12-26T21:37:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.606+0000 7f24bdb9b700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f249c066f30 0x7f249c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.607+0000 7f24bdb9b700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f249c066f30 0x7f249c0693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24b4001870 tx=0x7f24b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.610+0000 7f24adffb700 1 -- 172.21.15.131:0/3740154338 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24a8052d40 con 0x7f24b80cc100 2023-12-26T21:37:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.805+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f24b80657e0 con 0x7f24b80cc100 2023-12-26T21:37:46.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.805+0000 7f24adffb700 1 -- 172.21.15.131:0/3740154338 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7f24a802a060 con 0x7f24b80cc100 2023-12-26T21:37:46.805 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:46.805 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:37:46.805 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:46.806 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T21:37:46.806 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T21:37:46.806 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:46.806 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f249c066f30 msgr2=0x7f249c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f249c066f30 0x7f249c0693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24b4001870 tx=0x7f24b4009040 comp rx=0 tx=0).stop 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b80cc100 msgr2=0x7f24b80b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b80cc100 0x7f24b80b1ea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f24a8013000 tx=0x7f24a8017000 comp rx=0 tx=0).stop 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 shutdown_connections 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b813c510 0x7f24b8155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f249c066f30 0x7f249c0693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b806bfd0 0x7f24b8153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 --2- 172.21.15.131:0/3740154338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24b80cc100 0x7f24b80b1ea0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 >> 172.21.15.131:0/3740154338 conn(0x7f24b80ce3e0 msgr2=0x7f24b80c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 shutdown_connections 2023-12-26T21:37:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:46.808+0000 7f24bfdff700 1 -- 172.21.15.131:0/3740154338 wait complete. 2023-12-26T21:37:46.825 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T21:37:46.825 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T21:37:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.019+0000 7f208e9ff700 1 Processor -- start 2023-12-26T21:37:47.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.020+0000 7f208e9ff700 1 -- start start 2023-12-26T21:37:47.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f208e9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20880c8d40 0x7f208815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:47.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f208e9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2088156e50 0x7f20881572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:47.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f208e9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20880af040 con 0x7f20880c9110 2023-12-26T21:37:47.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f208e9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20880ae0e0 con 0x7f2088156e50 2023-12-26T21:37:47.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f208e9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20880ae3c0 con 0x7f20880c8d40 2023-12-26T21:37:47.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f20877fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f20880c9110 0x7f20880ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35684/0 (socket says 172.21.15.131:35684) 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f20877fe700 1 -- 172.21.15.131:0/1103002391 learned_addr learned my addr 172.21.15.131:0/1103002391 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f2087fff700 1 --2- 172.21.15.131:0/1103002391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20880c8d40 0x7f208815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f208cf9c700 1 --2- 172.21.15.131:0/1103002391 >> v2:172.21.15.190:3300/0 conn(0x7f2088156e50 0x7f20881572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f2086ffd700 1 -- 172.21.15.131:0/1103002391 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835332502 0 0) 0x7f20880af040 con 0x7f20880c9110 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.021+0000 7f2086ffd700 1 -- 172.21.15.131:0/1103002391 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2070003570 con 0x7f20880c9110 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.022+0000 7f2087fff700 1 -- 172.21.15.131:0/1103002391 >> v2:172.21.15.190:3300/0 conn(0x7f2088156e50 msgr2=0x7f20881572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.022+0000 7f2087fff700 1 --2- 172.21.15.131:0/1103002391 >> v2:172.21.15.190:3300/0 conn(0x7f2088156e50 0x7f20881572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.022+0000 7f2087fff700 1 -- 172.21.15.131:0/1103002391 >> v1:172.21.15.131:6789/0 conn(0x7f20880c9110 legacy=0x7f20880ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.022+0000 7f2087fff700 1 -- 172.21.15.131:0/1103002391 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880ae6a0 con 0x7f20880c8d40 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.022+0000 7f208cf9c700 1 --2- 172.21.15.131:0/1103002391 >> v2:172.21.15.190:3300/0 conn(0x7f2088156e50 0x7f20881572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:47.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.022+0000 7f2087fff700 1 --2- 172.21.15.131:0/1103002391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20880c8d40 0x7f208815bc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f20780098e0 tx=0x7f20780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbc8d1092b07691f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.022+0000 7f2086ffd700 1 -- 172.21.15.131:0/1103002391 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078011630 con 0x7f20880c8d40 2023-12-26T21:37:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.022+0000 7f2086ffd700 1 -- 172.21.15.131:0/1103002391 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2078011790 con 0x7f20880c8d40 2023-12-26T21:37:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.022+0000 7f2086ffd700 1 -- 172.21.15.131:0/1103002391 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078011b30 con 0x7f20880c8d40 2023-12-26T21:37:47.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.023+0000 7f208e9ff700 1 -- 172.21.15.131:0/1103002391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20880c8d40 msgr2=0x7f208815bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.023+0000 7f208e9ff700 1 --2- 172.21.15.131:0/1103002391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20880c8d40 0x7f208815bc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f20780098e0 tx=0x7f20780091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.023+0000 7f208e9ff700 1 -- 172.21.15.131:0/1103002391 shutdown_connections 2023-12-26T21:37:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.023+0000 7f208e9ff700 1 --2- 172.21.15.131:0/1103002391 >> v2:172.21.15.190:3300/0 conn(0x7f2088156e50 0x7f20881572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.023+0000 7f208e9ff700 1 --2- 172.21.15.131:0/1103002391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20880c8d40 0x7f208815bc30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.023+0000 7f208e9ff700 1 -- 172.21.15.131:0/1103002391 >> 172.21.15.131:0/1103002391 conn(0x7f20880675b0 msgr2=0x7f20880679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.023+0000 7f208e9ff700 1 -- 172.21.15.131:0/1103002391 shutdown_connections 2023-12-26T21:37:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.023+0000 7f208e9ff700 1 -- 172.21.15.131:0/1103002391 wait complete. 2023-12-26T21:37:47.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.025+0000 7f208e9ff700 1 Processor -- start 2023-12-26T21:37:47.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.025+0000 7f208e9ff700 1 -- start start 2023-12-26T21:37:47.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f208e9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20880c8d40 0x7f20880abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:47.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f208e9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20880c9110 0x7f20880c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f208e9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088156e50 0x7f208814a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f208e9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20880c7e00 con 0x7f20880c9110 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f208e9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208814ad40 con 0x7f20880c8d40 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f208e9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20880c7230 con 0x7f2088156e50 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f20877fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20880c9110 0x7f20880c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f2087fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20880c8d40 0x7f20880abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f2087fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20880c8d40 0x7f20880abdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53550/0 (socket says 172.21.15.131:53550) 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f2087fff700 1 -- 172.21.15.131:0/3254476849 learned_addr learned my addr 172.21.15.131:0/3254476849 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.026+0000 7f208cf9c700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088156e50 0x7f208814a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f20877fe700 1 -- 172.21.15.131:0/3254476849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088156e50 msgr2=0x7f208814a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f20877fe700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088156e50 0x7f208814a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f20877fe700 1 -- 172.21.15.131:0/3254476849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20880c8d40 msgr2=0x7f20880abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:47.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f20877fe700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20880c8d40 0x7f20880abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f20877fe700 1 -- 172.21.15.131:0/3254476849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2080009a20 con 0x7f20880c9110 2023-12-26T21:37:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f2087fff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20880c8d40 0x7f20880abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f20877fe700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20880c9110 0x7f20880c8290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f207c00c5b0 tx=0x7f207c007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f206ffff700 1 -- 172.21.15.131:0/3254476849 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c005c00 con 0x7f20880c9110 2023-12-26T21:37:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f206ffff700 1 -- 172.21.15.131:0/3254476849 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c005d80 con 0x7f20880c9110 2023-12-26T21:37:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2078009420 con 0x7f20880c9110 2023-12-26T21:37:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f206ffff700 1 -- 172.21.15.131:0/3254476849 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c007320 con 0x7f20880c9110 2023-12-26T21:37:47.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.027+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20880c77b0 con 0x7f20880c9110 2023-12-26T21:37:47.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.028+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20880c7fb0 con 0x7f20880c9110 2023-12-26T21:37:47.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.029+0000 7f206ffff700 1 -- 172.21.15.131:0/3254476849 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f207c01b3b0 con 0x7f20880c9110 2023-12-26T21:37:47.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.030+0000 7f206ffff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2074066f30 0x7f20740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:47.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.030+0000 7f206ffff700 1 -- 172.21.15.131:0/3254476849 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f207c0836a0 con 0x7f20880c9110 2023-12-26T21:37:47.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.030+0000 7f2087fff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2074066f30 0x7f20740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:47.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.031+0000 7f2087fff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2074066f30 0x7f20740693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20780091e0 tx=0x7f2078017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:47.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.034+0000 7f206ffff700 1 -- 172.21.15.131:0/3254476849 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20880c7fb0 con 0x7f20880c9110 2023-12-26T21:37:47.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:47.231+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f20880c7fb0 con 0x7f20880c9110 2023-12-26T21:37:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.033+0000 7f206ffff700 1 -- 172.21.15.131:0/3254476849 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f20880c7fb0 con 0x7f20880c9110 2023-12-26T21:37:48.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.036+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2074066f30 msgr2=0x7f20740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:48.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.036+0000 7f208e9ff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2074066f30 0x7f20740693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20780091e0 tx=0x7f2078017000 comp rx=0 tx=0).stop 2023-12-26T21:37:48.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.036+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20880c9110 msgr2=0x7f20880c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:48.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.036+0000 7f208e9ff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20880c9110 0x7f20880c8290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f207c00c5b0 tx=0x7f207c007d50 comp rx=0 tx=0).stop 2023-12-26T21:37:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.037+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 shutdown_connections 2023-12-26T21:37:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.037+0000 7f208e9ff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2074066f30 0x7f20740693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.037+0000 7f208e9ff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088156e50 0x7f208814a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.037+0000 7f208e9ff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20880c8d40 0x7f20880abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.038+0000 7f208e9ff700 1 --2- 172.21.15.131:0/3254476849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20880c9110 0x7f20880c8290 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.038+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 >> 172.21.15.131:0/3254476849 conn(0x7f20880675b0 msgr2=0x7f20880cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.038+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 shutdown_connections 2023-12-26T21:37:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.038+0000 7f208e9ff700 1 -- 172.21.15.131:0/3254476849 wait complete. 2023-12-26T21:37:48.042 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T21:37:48.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T21:37:48.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.254+0000 7f0b85f9d700 1 Processor -- start 2023-12-26T21:37:48.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.254+0000 7f0b85f9d700 1 -- start start 2023-12-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.256+0000 7f0b85f9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b800cc540 0x7f0b80068730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.256+0000 7f0b85f9d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8006a020 con 0x7f0b80068c70 2023-12-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.256+0000 7f0b85f9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b800694a0 con 0x7f0b80143010 2023-12-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.256+0000 7f0b85f9d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b80069780 con 0x7f0b800cc540 2023-12-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.256+0000 7f0b8579c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0b80068c70 0x7f0b800c8c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35694/0 (socket says 172.21.15.131:35694) 2023-12-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.256+0000 7f0b8579c700 1 -- 172.21.15.131:0/163361342 learned_addr learned my addr 172.21.15.131:0/163361342 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:48.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.257+0000 7f0b7ffff700 1 --2- 172.21.15.131:0/163361342 >> v2:172.21.15.190:3300/0 conn(0x7f0b800cc540 0x7f0b80068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.257+0000 7f0b7f7fe700 1 -- 172.21.15.131:0/163361342 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018210468 0 0) 0x7f0b8006a020 con 0x7f0b80068c70 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.257+0000 7f0b7f7fe700 1 -- 172.21.15.131:0/163361342 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b70003570 con 0x7f0b80068c70 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.257+0000 7f0b7f7fe700 1 -- 172.21.15.131:0/163361342 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12784315 0 0) 0x7f0b800694a0 con 0x7f0b80143010 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.257+0000 7f0b7f7fe700 1 -- 172.21.15.131:0/163361342 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b8006a020 con 0x7f0b80143010 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.257+0000 7f0b7ffff700 1 -- 172.21.15.131:0/163361342 >> v1:172.21.15.190:6789/0 conn(0x7f0b80143010 legacy=0x7f0b800cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.257+0000 7f0b7ffff700 1 -- 172.21.15.131:0/163361342 >> v1:172.21.15.131:6789/0 conn(0x7f0b80068c70 legacy=0x7f0b800c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.257+0000 7f0b7ffff700 1 -- 172.21.15.131:0/163361342 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b80069a60 con 0x7f0b800cc540 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b7ffff700 1 --2- 172.21.15.131:0/163361342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b800cc540 0x7f0b80068730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0098e0 tx=0x7f0b6c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=275fb1ad59958eb1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b7f7fe700 1 -- 172.21.15.131:0/163361342 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c011630 con 0x7f0b800cc540 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b7f7fe700 1 -- 172.21.15.131:0/163361342 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c011790 con 0x7f0b800cc540 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b7f7fe700 1 -- 172.21.15.131:0/163361342 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c011b30 con 0x7f0b800cc540 2023-12-26T21:37:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b85f9d700 1 -- 172.21.15.131:0/163361342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b800cc540 msgr2=0x7f0b80068730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b85f9d700 1 --2- 172.21.15.131:0/163361342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b800cc540 0x7f0b80068730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0098e0 tx=0x7f0b6c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b85f9d700 1 -- 172.21.15.131:0/163361342 shutdown_connections 2023-12-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b85f9d700 1 --2- 172.21.15.131:0/163361342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b800cc540 0x7f0b80068730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b85f9d700 1 -- 172.21.15.131:0/163361342 >> 172.21.15.131:0/163361342 conn(0x7f0b8006b5c0 msgr2=0x7f0b8006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b85f9d700 1 -- 172.21.15.131:0/163361342 shutdown_connections 2023-12-26T21:37:48.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.258+0000 7f0b85f9d700 1 -- 172.21.15.131:0/163361342 wait complete. 2023-12-26T21:37:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.260+0000 7f0b85f9d700 1 Processor -- start 2023-12-26T21:37:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.261+0000 7f0b85f9d700 1 -- start start 2023-12-26T21:37:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b85f9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80068c70 0x7f0b800c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b85f9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b800cc540 0x7f0b800c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b85f9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b80143010 0x7f0b800ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b85f9d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800c7d50 con 0x7f0b80068c70 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b85f9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800c6260 con 0x7f0b80143010 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b85f9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800aec00 con 0x7f0b800cc540 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b84f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80068c70 0x7f0b800c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b84f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80068c70 0x7f0b800c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60762/0 (socket says 172.21.15.131:60762) 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b84f9b700 1 -- 172.21.15.131:0/1504350289 learned_addr learned my addr 172.21.15.131:0/1504350289 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b84f9b700 1 -- 172.21.15.131:0/1504350289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b800cc540 msgr2=0x7f0b800c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b84f9b700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b800cc540 0x7f0b800c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b84f9b700 1 -- 172.21.15.131:0/1504350289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b80143010 msgr2=0x7f0b800ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b84f9b700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b80143010 0x7f0b800ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.262+0000 7f0b84f9b700 1 -- 172.21.15.131:0/1504350289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c009420 con 0x7f0b80068c70 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.263+0000 7f0b84f9b700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80068c70 0x7f0b800c7810 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400c5b0 tx=0x7f0b74007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.263+0000 7f0b7cff9700 1 -- 172.21.15.131:0/1504350289 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74010510 con 0x7f0b80068c70 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.263+0000 7f0b7cff9700 1 -- 172.21.15.131:0/1504350289 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74010670 con 0x7f0b80068c70 2023-12-26T21:37:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.263+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8013a5a0 con 0x7f0b80068c70 2023-12-26T21:37:48.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.263+0000 7f0b7cff9700 1 -- 172.21.15.131:0/1504350289 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74010a30 con 0x7f0b80068c70 2023-12-26T21:37:48.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.263+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8013a7f0 con 0x7f0b80068c70 2023-12-26T21:37:48.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.264+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8004c3c0 con 0x7f0b80068c70 2023-12-26T21:37:48.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.265+0000 7f0b7cff9700 1 -- 172.21.15.131:0/1504350289 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0b7401b400 con 0x7f0b80068c70 2023-12-26T21:37:48.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.266+0000 7f0b7cff9700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0b64066f30 0x7f0b640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:48.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.266+0000 7f0b7cff9700 1 -- 172.21.15.131:0/1504350289 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f0b74083fd0 con 0x7f0b80068c70 2023-12-26T21:37:48.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.266+0000 7f0b7ffff700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0b64066f30 0x7f0b640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:48.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.267+0000 7f0b7ffff700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0b64066f30 0x7f0b640693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0b800c3070 tx=0x7f0b6c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:48.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.270+0000 7f0b7cff9700 1 -- 172.21.15.131:0/1504350289 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b74051420 con 0x7f0b80068c70 2023-12-26T21:37:48.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:48.464+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0b80069d40 con 0x7f0b80068c70 2023-12-26T21:37:49.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.035+0000 7f0b7cff9700 1 -- 172.21.15.131:0/1504350289 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0b74054a40 con 0x7f0b80068c70 2023-12-26T21:37:49.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.040+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0b64066f30 msgr2=0x7f0b640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:49.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.040+0000 7f0b85f9d700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0b64066f30 0x7f0b640693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0b800c3070 tx=0x7f0b6c017000 comp rx=0 tx=0).stop 2023-12-26T21:37:49.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.040+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80068c70 msgr2=0x7f0b800c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:49.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.040+0000 7f0b85f9d700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80068c70 0x7f0b800c7810 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400c5b0 tx=0x7f0b74007e10 comp rx=0 tx=0).stop 2023-12-26T21:37:49.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.041+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 shutdown_connections 2023-12-26T21:37:49.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.041+0000 7f0b85f9d700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b80143010 0x7f0b800ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:49.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.041+0000 7f0b85f9d700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80068c70 0x7f0b800c7810 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:49.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.041+0000 7f0b85f9d700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0b64066f30 0x7f0b640693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:49.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.041+0000 7f0b85f9d700 1 --2- 172.21.15.131:0/1504350289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b800cc540 0x7f0b800c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:49.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.041+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 >> 172.21.15.131:0/1504350289 conn(0x7f0b8006b5c0 msgr2=0x7f0b8006b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:49.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.042+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 shutdown_connections 2023-12-26T21:37:49.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.042+0000 7f0b85f9d700 1 -- 172.21.15.131:0/1504350289 wait complete. 2023-12-26T21:37:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T21:37:49.074 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T21:37:49.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.266+0000 7f1adf5ff700 1 Processor -- start 2023-12-26T21:37:49.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.266+0000 7f1adf5ff700 1 -- start start 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.267+0000 7f1adf5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad80cc950 0x7f1ad806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.267+0000 7f1adf5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1ad806eb30 0x7f1ad815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1adf5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad80b07f0 con 0x7f1ad80cc580 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1adf5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad80680f0 con 0x7f1ad806eb30 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1adf5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80ccda0 con 0x7f1ad80cc950 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1addb9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1ad806eb30 0x7f1ad815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1addb9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1ad806eb30 0x7f1ad815d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60772/0 (socket says 172.21.15.131:60772) 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1addb9c700 1 -- 172.21.15.131:0/3707435989 learned_addr learned my addr 172.21.15.131:0/3707435989 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1adcb9a700 1 --2- 172.21.15.131:0/3707435989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad80cc950 0x7f1ad806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:49.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1addb9c700 1 -- 172.21.15.131:0/3707435989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad80cc950 msgr2=0x7f1ad806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:49.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1addb9c700 1 --2- 172.21.15.131:0/3707435989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad80cc950 0x7f1ad806e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:49.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1addb9c700 1 -- 172.21.15.131:0/3707435989 >> v1:172.21.15.190:6789/0 conn(0x7f1ad80cc580 legacy=0x7f1ad80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:49.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.268+0000 7f1addb9c700 1 -- 172.21.15.131:0/3707435989 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80b0d70 con 0x7f1ad806eb30 2023-12-26T21:37:49.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1addb9c700 1 --2- 172.21.15.131:0/3707435989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad806eb30 0x7f1ad815d9b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009f50 tx=0x7f1ad0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974de2b61789115a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:49.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1acffff700 1 -- 172.21.15.131:0/3707435989 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000e070 con 0x7f1ad806eb30 2023-12-26T21:37:49.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1acffff700 1 -- 172.21.15.131:0/3707435989 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad000ba70 con 0x7f1ad806eb30 2023-12-26T21:37:49.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1acffff700 1 -- 172.21.15.131:0/3707435989 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00092e0 con 0x7f1ad806eb30 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1adf5ff700 1 -- 172.21.15.131:0/3707435989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad806eb30 msgr2=0x7f1ad815d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1adf5ff700 1 --2- 172.21.15.131:0/3707435989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad806eb30 0x7f1ad815d9b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009f50 tx=0x7f1ad0009840 comp rx=0 tx=0).stop 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1adf5ff700 1 -- 172.21.15.131:0/3707435989 shutdown_connections 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1adf5ff700 1 --2- 172.21.15.131:0/3707435989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad806eb30 0x7f1ad815d9b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1adf5ff700 1 --2- 172.21.15.131:0/3707435989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad80cc950 0x7f1ad806e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1adf5ff700 1 -- 172.21.15.131:0/3707435989 >> 172.21.15.131:0/3707435989 conn(0x7f1ad80686c0 msgr2=0x7f1ad8068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1adf5ff700 1 -- 172.21.15.131:0/3707435989 shutdown_connections 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.269+0000 7f1adf5ff700 1 -- 172.21.15.131:0/3707435989 wait complete. 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.271+0000 7f1adf5ff700 1 Processor -- start 2023-12-26T21:37:49.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.271+0000 7f1adf5ff700 1 -- start start 2023-12-26T21:37:49.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.272+0000 7f1adf5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad806eb30 0x7f1ad8169580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:49.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.272+0000 7f1adf5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad80cc580 0x7f1ad80af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:49.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.272+0000 7f1adf5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad80cc950 0x7f1ad815aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:49.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.272+0000 7f1adf5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8169b90 con 0x7f1ad80cc950 2023-12-26T21:37:49.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.272+0000 7f1adf5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8169e70 con 0x7f1ad80cc580 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.272+0000 7f1adf5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad816a150 con 0x7f1ad806eb30 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.272+0000 7f1addb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad80cc950 0x7f1ad815aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1addb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad80cc950 0x7f1ad815aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60784/0 (socket says 172.21.15.131:60784) 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1addb9c700 1 -- 172.21.15.131:0/1399572721 learned_addr learned my addr 172.21.15.131:0/1399572721 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1adcb9a700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad80cc580 0x7f1ad80af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1add39b700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad806eb30 0x7f1ad8169580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1addb9c700 1 -- 172.21.15.131:0/1399572721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad806eb30 msgr2=0x7f1ad8169580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1addb9c700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad806eb30 0x7f1ad8169580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1addb9c700 1 -- 172.21.15.131:0/1399572721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad80cc580 msgr2=0x7f1ad80af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1addb9c700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad80cc580 0x7f1ad80af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1addb9c700 1 -- 172.21.15.131:0/1399572721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0009ae0 con 0x7f1ad80cc950 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1addb9c700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad80cc950 0x7f1ad815aff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ad806b770 tx=0x7f1ad0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:49.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.273+0000 7f1acd7fa700 1 -- 172.21.15.131:0/1399572721 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00032f0 con 0x7f1ad80cc950 2023-12-26T21:37:49.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.274+0000 7f1acd7fa700 1 -- 172.21.15.131:0/1399572721 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad0008ce0 con 0x7f1ad80cc950 2023-12-26T21:37:49.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.274+0000 7f1acd7fa700 1 -- 172.21.15.131:0/1399572721 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0025690 con 0x7f1ad80cc950 2023-12-26T21:37:49.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.274+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad816a430 con 0x7f1ad80cc950 2023-12-26T21:37:49.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.274+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad80afb80 con 0x7f1ad80cc950 2023-12-26T21:37:49.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.275+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad80657e0 con 0x7f1ad80cc950 2023-12-26T21:37:49.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.276+0000 7f1acd7fa700 1 -- 172.21.15.131:0/1399572721 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1ad0003040 con 0x7f1ad80cc950 2023-12-26T21:37:49.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.276+0000 7f1acd7fa700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1abc066f30 0x7f1abc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:49.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.276+0000 7f1acd7fa700 1 -- 172.21.15.131:0/1399572721 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f1ad0086150 con 0x7f1ad80cc950 2023-12-26T21:37:49.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.277+0000 7f1add39b700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1abc066f30 0x7f1abc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:49.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.277+0000 7f1add39b700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1abc066f30 0x7f1abc0693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4003a80 tx=0x7f1ad4007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:49.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.281+0000 7f1acd7fa700 1 -- 172.21.15.131:0/1399572721 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ad0053380 con 0x7f1ad80cc950 2023-12-26T21:37:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:49.515+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1ad80cdda0 con 0x7f1ad80cc950 2023-12-26T21:37:50.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.044+0000 7f1acd7fa700 1 -- 172.21.15.131:0/1399572721 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1ad00569a0 con 0x7f1ad80cc950 2023-12-26T21:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.046+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1abc066f30 msgr2=0x7f1abc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.046+0000 7f1adf5ff700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1abc066f30 0x7f1abc0693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4003a80 tx=0x7f1ad4007ec0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.046+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad80cc950 msgr2=0x7f1ad815aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.046+0000 7f1adf5ff700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad80cc950 0x7f1ad815aff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ad806b770 tx=0x7f1ad0018000 comp rx=0 tx=0).stop 2023-12-26T21:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.047+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 shutdown_connections 2023-12-26T21:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.047+0000 7f1adf5ff700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad80cc950 0x7f1ad815aff0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.047+0000 7f1adf5ff700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1abc066f30 0x7f1abc0693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.047+0000 7f1adf5ff700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad806eb30 0x7f1ad8169580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.047+0000 7f1adf5ff700 1 --2- 172.21.15.131:0/1399572721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad80cc580 0x7f1ad80af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.047+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 >> 172.21.15.131:0/1399572721 conn(0x7f1ad80686c0 msgr2=0x7f1ad80b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.047+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 shutdown_connections 2023-12-26T21:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.047+0000 7f1adf5ff700 1 -- 172.21.15.131:0/1399572721 wait complete. 2023-12-26T21:37:50.049 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 4 and data pool 5 2023-12-26T21:37:50.070 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:50.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.264+0000 7f9b123ff700 1 Processor -- start 2023-12-26T21:37:50.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.264+0000 7f9b123ff700 1 -- start start 2023-12-26T21:37:50.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.265+0000 7f9b123ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b0c0b0750 0x7f9b0c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b123ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9b0c0c9400 0x7f9b0c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b123ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0c156940 con 0x7f9b0c0cb320 2023-12-26T21:37:50.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b123ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0c0c8cc0 con 0x7f9b0c0b0750 2023-12-26T21:37:50.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b123ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0c0ad700 con 0x7f9b0c0c9400 2023-12-26T21:37:50.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b0b7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b0c0b0750 0x7f9b0c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:50.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b0b7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b0c0b0750 0x7f9b0c0b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60800/0 (socket says 172.21.15.131:60800) 2023-12-26T21:37:50.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b0b7fe700 1 -- 172.21.15.131:0/105952051 learned_addr learned my addr 172.21.15.131:0/105952051 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:50.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b1099c700 1 --2- 172.21.15.131:0/105952051 >> v2:172.21.15.190:3300/0 conn(0x7f9b0c0c9400 0x7f9b0c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:50.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b0b7fe700 1 -- 172.21.15.131:0/105952051 >> v2:172.21.15.190:3300/0 conn(0x7f9b0c0c9400 msgr2=0x7f9b0c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:50.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b0b7fe700 1 --2- 172.21.15.131:0/105952051 >> v2:172.21.15.190:3300/0 conn(0x7f9b0c0c9400 0x7f9b0c0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b0b7fe700 1 -- 172.21.15.131:0/105952051 >> v1:172.21.15.190:6789/0 conn(0x7f9b0c0cb320 legacy=0x7f9b0c069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b0b7fe700 1 -- 172.21.15.131:0/105952051 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0ae500 con 0x7f9b0c0b0750 2023-12-26T21:37:50.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.266+0000 7f9b1099c700 1 --2- 172.21.15.131:0/105952051 >> v2:172.21.15.190:3300/0 conn(0x7f9b0c0c9400 0x7f9b0c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:50.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b0b7fe700 1 --2- 172.21.15.131:0/105952051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0b0750 0x7f9b0c0b0bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c06fb50 tx=0x7f9b00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c01d6ae09a03d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:50.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b0affd700 1 -- 172.21.15.131:0/105952051 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000b820 con 0x7f9b0c0b0750 2023-12-26T21:37:50.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b0affd700 1 -- 172.21.15.131:0/105952051 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0000b980 con 0x7f9b0c0b0750 2023-12-26T21:37:50.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b0affd700 1 -- 172.21.15.131:0/105952051 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000bd20 con 0x7f9b0c0b0750 2023-12-26T21:37:50.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b123ff700 1 -- 172.21.15.131:0/105952051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0b0750 msgr2=0x7f9b0c0b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b123ff700 1 --2- 172.21.15.131:0/105952051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0b0750 0x7f9b0c0b0bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c06fb50 tx=0x7f9b00009250 comp rx=0 tx=0).stop 2023-12-26T21:37:50.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b123ff700 1 -- 172.21.15.131:0/105952051 shutdown_connections 2023-12-26T21:37:50.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b123ff700 1 --2- 172.21.15.131:0/105952051 >> v2:172.21.15.190:3300/0 conn(0x7f9b0c0c9400 0x7f9b0c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b123ff700 1 --2- 172.21.15.131:0/105952051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0b0750 0x7f9b0c0b0bf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.267+0000 7f9b123ff700 1 -- 172.21.15.131:0/105952051 >> 172.21.15.131:0/105952051 conn(0x7f9b0c068040 msgr2=0x7f9b0c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:50.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.268+0000 7f9b123ff700 1 -- 172.21.15.131:0/105952051 shutdown_connections 2023-12-26T21:37:50.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.268+0000 7f9b123ff700 1 -- 172.21.15.131:0/105952051 wait complete. 2023-12-26T21:37:50.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.270+0000 7f9b123ff700 1 Processor -- start 2023-12-26T21:37:50.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.270+0000 7f9b123ff700 1 -- start start 2023-12-26T21:37:50.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b123ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b0c0b0750 0x7f9b0c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b123ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0c9400 0x7f9b0c14ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b123ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b0c0cb320 0x7f9b0c0aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b123ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c15c2b0 con 0x7f9b0c0c9400 2023-12-26T21:37:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b123ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c15b6a0 con 0x7f9b0c0b0750 2023-12-26T21:37:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b123ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c15b980 con 0x7f9b0c0cb320 2023-12-26T21:37:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0c9400 0x7f9b0c14ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0c9400 0x7f9b0c14ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60816/0 (socket says 172.21.15.131:60816) 2023-12-26T21:37:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b0b7fe700 1 -- 172.21.15.131:0/1534415994 learned_addr learned my addr 172.21.15.131:0/1534415994 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b0bfff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b0c0b0750 0x7f9b0c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.271+0000 7f9b1099c700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b0c0cb320 0x7f9b0c0aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9b0b7fe700 1 -- 172.21.15.131:0/1534415994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b0c0cb320 msgr2=0x7f9b0c0aa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9b0b7fe700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b0c0cb320 0x7f9b0c0aa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9b0b7fe700 1 -- 172.21.15.131:0/1534415994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b0c0b0750 msgr2=0x7f9b0c15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9b0b7fe700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b0c0b0750 0x7f9b0c15adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9b0b7fe700 1 -- 172.21.15.131:0/1534415994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b000094f0 con 0x7f9b0c0c9400 2023-12-26T21:37:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9b1099c700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b0c0cb320 0x7f9b0c0aa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9b0b7fe700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0c9400 0x7f9b0c14ae60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b000094c0 tx=0x7f9b00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9af3fff700 1 -- 172.21.15.131:0/1534415994 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000bd00 con 0x7f9b0c0c9400 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9af3fff700 1 -- 172.21.15.131:0/1534415994 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0001f070 con 0x7f9b0c0c9400 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0c15bc60 con 0x7f9b0c0c9400 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9af3fff700 1 -- 172.21.15.131:0/1534415994 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00026690 con 0x7f9b0c0c9400 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.272+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c15b300 con 0x7f9b0c0c9400 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.273+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0c0657e0 con 0x7f9b0c0c9400 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.274+0000 7f9af3fff700 1 -- 172.21.15.131:0/1534415994 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9b00026a00 con 0x7f9b0c0c9400 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.275+0000 7f9af3fff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9af8066f30 0x7f9af80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.275+0000 7f9af3fff700 1 -- 172.21.15.131:0/1534415994 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9b00021070 con 0x7f9b0c0c9400 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.275+0000 7f9b0bfff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9af8066f30 0x7f9af80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.276+0000 7f9b0bfff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9af8066f30 0x7f9af80693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0039e0 tx=0x7f9afc007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.279+0000 7f9af3fff700 1 -- 172.21.15.131:0/1534415994 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b00053560 con 0x7f9b0c0c9400 2023-12-26T21:37:50.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.472+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b0c15c3f0 con 0x7f9b0c0c9400 2023-12-26T21:37:50.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.473+0000 7f9af3fff700 1 -- 172.21.15.131:0/1534415994 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f9b00026cb0 con 0x7f9b0c0c9400 2023-12-26T21:37:50.474 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:50.474 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:37:49.515659+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:37:47.268950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-26T21:37:48.501427+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:50.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9af8066f30 msgr2=0x7f9af80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9af8066f30 0x7f9af80693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0039e0 tx=0x7f9afc007ec0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0c9400 msgr2=0x7f9b0c14ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0c9400 0x7f9b0c14ae60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b000094c0 tx=0x7f9b00017000 comp rx=0 tx=0).stop 2023-12-26T21:37:50.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 shutdown_connections 2023-12-26T21:37:50.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9af8066f30 0x7f9af80693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b0c0cb320 0x7f9b0c0aa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b0c0b0750 0x7f9b0c15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 --2- 172.21.15.131:0/1534415994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b0c0c9400 0x7f9b0c14ae60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 >> 172.21.15.131:0/1534415994 conn(0x7f9b0c068040 msgr2=0x7f9b0c0ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:50.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 shutdown_connections 2023-12-26T21:37:50.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.476+0000 7f9b123ff700 1 -- 172.21.15.131:0/1534415994 wait complete. 2023-12-26T21:37:50.495 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:50.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.684+0000 7f3d01fff700 1 Processor -- start 2023-12-26T21:37:50.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.684+0000 7f3d01fff700 1 -- start start 2023-12-26T21:37:50.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.685+0000 7f3d01fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3cfc0b5310 0x7f3cfc0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.685+0000 7f3d01fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc0afcd0 con 0x7f3cfc1596a0 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.685+0000 7f3d01fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc0b3e70 con 0x7f3cfc159a70 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.685+0000 7f3d01fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cfc0b0f20 con 0x7f3cfc0b5310 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.686+0000 7f3cfbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3cfc0b5310 0x7f3cfc0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.686+0000 7f3cfb7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3cfc1596a0 0x7f3cfc0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35708/0 (socket says 172.21.15.131:35708) 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.686+0000 7f3cfbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3cfc0b5310 0x7f3cfc0b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60826/0 (socket says 172.21.15.131:60826) 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.686+0000 7f3cfb7fe700 1 -- 172.21.15.131:0/426592406 learned_addr learned my addr 172.21.15.131:0/426592406 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.686+0000 7f3cfbfff700 1 -- 172.21.15.131:0/426592406 >> v1:172.21.15.190:6789/0 conn(0x7f3cfc159a70 legacy=0x7f3cfc0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.686+0000 7f3cfbfff700 1 -- 172.21.15.131:0/426592406 >> v1:172.21.15.131:6789/0 conn(0x7f3cfc1596a0 legacy=0x7f3cfc0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.686+0000 7f3cfbfff700 1 -- 172.21.15.131:0/426592406 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0b3540 con 0x7f3cfc0b5310 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.686+0000 7f3cfa7fc700 1 -- 172.21.15.131:0/426592406 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937131034 0 0) 0x7f3cfc0afcd0 con 0x7f3cfc1596a0 2023-12-26T21:37:50.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.686+0000 7f3cfbfff700 1 --2- 172.21.15.131:0/426592406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 0x7f3cfc0b0870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009e60 tx=0x7f3cf4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdc228ad34fcd78 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:50.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3cfa7fc700 1 -- 172.21.15.131:0/426592406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400e070 con 0x7f3cfc0b5310 2023-12-26T21:37:50.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3cfa7fc700 1 -- 172.21.15.131:0/426592406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf400bc30 con 0x7f3cfc0b5310 2023-12-26T21:37:50.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3cfa7fc700 1 -- 172.21.15.131:0/426592406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4012430 con 0x7f3cfc0b5310 2023-12-26T21:37:50.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3d01fff700 1 -- 172.21.15.131:0/426592406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 msgr2=0x7f3cfc0b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3d01fff700 1 --2- 172.21.15.131:0/426592406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 0x7f3cfc0b0870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009e60 tx=0x7f3cf4009780 comp rx=0 tx=0).stop 2023-12-26T21:37:50.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3d01fff700 1 -- 172.21.15.131:0/426592406 shutdown_connections 2023-12-26T21:37:50.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3d01fff700 1 --2- 172.21.15.131:0/426592406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 0x7f3cfc0b0870 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3d01fff700 1 -- 172.21.15.131:0/426592406 >> 172.21.15.131:0/426592406 conn(0x7f3cfc06ece0 msgr2=0x7f3cfc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:50.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3d01fff700 1 -- 172.21.15.131:0/426592406 shutdown_connections 2023-12-26T21:37:50.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.687+0000 7f3d01fff700 1 -- 172.21.15.131:0/426592406 wait complete. 2023-12-26T21:37:50.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.689+0000 7f3d01fff700 1 Processor -- start 2023-12-26T21:37:50.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.689+0000 7f3d01fff700 1 -- start start 2023-12-26T21:37:50.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3d01fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 0x7f3cfc155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3d01fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cfc1596a0 0x7f3cfc0cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3d01fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cfc159a70 0x7f3cfc0cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3d01fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0cc2d0 con 0x7f3cfc0b5310 2023-12-26T21:37:50.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3d01fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc156070 con 0x7f3cfc1596a0 2023-12-26T21:37:50.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3d01fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0c8160 con 0x7f3cfc159a70 2023-12-26T21:37:50.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3cfb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 0x7f3cfc155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:50.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3cfb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 0x7f3cfc155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60830/0 (socket says 172.21.15.131:60830) 2023-12-26T21:37:50.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3cfb7fe700 1 -- 172.21.15.131:0/1869113469 learned_addr learned my addr 172.21.15.131:0/1869113469 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3cfb7fe700 1 -- 172.21.15.131:0/1869113469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cfc159a70 msgr2=0x7f3cfc0cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3cfb7fe700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cfc159a70 0x7f3cfc0cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3cfb7fe700 1 -- 172.21.15.131:0/1869113469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cfc1596a0 msgr2=0x7f3cfc0cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3cfb7fe700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cfc1596a0 0x7f3cfc0cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.690+0000 7f3cfb7fe700 1 -- 172.21.15.131:0/1869113469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf4009a20 con 0x7f3cfc0b5310 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.691+0000 7f3cfb7fe700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 0x7f3cfc155b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00c7e0 tx=0x7f3cec007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.691+0000 7f3ce37fe700 1 -- 172.21.15.131:0/1869113469 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec005b30 con 0x7f3cfc0b5310 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.691+0000 7f3ce37fe700 1 -- 172.21.15.131:0/1869113469 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec005c90 con 0x7f3cfc0b5310 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.691+0000 7f3ce37fe700 1 -- 172.21.15.131:0/1869113469 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec007300 con 0x7f3cfc0b5310 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.691+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc0cba30 con 0x7f3cfc0b5310 2023-12-26T21:37:50.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.691+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc0cbff0 con 0x7f3cfc0b5310 2023-12-26T21:37:50.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.692+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc0657e0 con 0x7f3cfc0b5310 2023-12-26T21:37:50.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.693+0000 7f3ce37fe700 1 -- 172.21.15.131:0/1869113469 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3cec01b400 con 0x7f3cfc0b5310 2023-12-26T21:37:50.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.694+0000 7f3ce37fe700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3ce8066f30 0x7f3ce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:50.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.694+0000 7f3ce37fe700 1 -- 172.21.15.131:0/1869113469 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3cec083f90 con 0x7f3cfc0b5310 2023-12-26T21:37:50.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.694+0000 7f3cfaffd700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3ce8066f30 0x7f3ce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:50.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.695+0000 7f3cfaffd700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3ce8066f30 0x7f3ce80693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc16cde0 tx=0x7f3cf0007b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:50.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.698+0000 7f3ce37fe700 1 -- 172.21.15.131:0/1869113469 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cec051280 con 0x7f3cfc0b5310 2023-12-26T21:37:50.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.896+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cfc0b3820 con 0x7f3cfc0b5310 2023-12-26T21:37:50.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.897+0000 7f3ce37fe700 1 -- 172.21.15.131:0/1869113469 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f3cec0548a0 con 0x7f3cfc0b5310 2023-12-26T21:37:50.897 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:50.897 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:37:49.515659+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:37:47.268950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-26T21:37:48.501427+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3ce8066f30 msgr2=0x7f3ce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3ce8066f30 0x7f3ce80693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc16cde0 tx=0x7f3cf0007b40 comp rx=0 tx=0).stop 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 msgr2=0x7f3cfc155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 0x7f3cfc155b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00c7e0 tx=0x7f3cec007df0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 shutdown_connections 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3ce8066f30 0x7f3ce80693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cfc159a70 0x7f3cfc0cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cfc0b5310 0x7f3cfc155b30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 --2- 172.21.15.131:0/1869113469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cfc1596a0 0x7f3cfc0cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 >> 172.21.15.131:0/1869113469 conn(0x7f3cfc06ece0 msgr2=0x7f3cfc158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 shutdown_connections 2023-12-26T21:37:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:50.899+0000 7f3d01fff700 1 -- 172.21.15.131:0/1869113469 wait complete. 2023-12-26T21:37:50.918 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T21:37:51.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.114+0000 7fa41bdff700 1 Processor -- start 2023-12-26T21:37:51.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.115+0000 7fa41bdff700 1 -- start start 2023-12-26T21:37:51.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41bdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa414159a70 0x7fa4140b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:51.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41bdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa4140b2a90 0x7fa4140b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:51.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41bdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140afcd0 con 0x7fa4141596a0 2023-12-26T21:37:51.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41bdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140b3e70 con 0x7fa414159a70 2023-12-26T21:37:51.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41bdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140b0f20 con 0x7fa4140b2a90 2023-12-26T21:37:51.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41939a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa414159a70 0x7fa4140b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:51.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41939a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa414159a70 0x7fa4140b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60842/0 (socket says 172.21.15.131:60842) 2023-12-26T21:37:51.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41939a700 1 -- 172.21.15.131:0/2371470915 learned_addr learned my addr 172.21.15.131:0/2371470915 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:51.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41a39c700 1 --2- 172.21.15.131:0/2371470915 >> v2:172.21.15.190:3300/0 conn(0x7fa4140b2a90 0x7fa4140b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:51.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41939a700 1 -- 172.21.15.131:0/2371470915 >> v2:172.21.15.190:3300/0 conn(0x7fa4140b2a90 msgr2=0x7fa4140b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:51.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41939a700 1 --2- 172.21.15.131:0/2371470915 >> v2:172.21.15.190:3300/0 conn(0x7fa4140b2a90 0x7fa4140b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:51.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41939a700 1 -- 172.21.15.131:0/2371470915 >> v1:172.21.15.190:6789/0 conn(0x7fa4141596a0 legacy=0x7fa4140b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:51.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.116+0000 7fa41939a700 1 -- 172.21.15.131:0/2371470915 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140b3540 con 0x7fa414159a70 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.117+0000 7fa41939a700 1 --2- 172.21.15.131:0/2371470915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 0x7fa4140b0160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4140b66b0 tx=0x7fa404009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888e0b45a8d62a34 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.117+0000 7fa418b99700 1 -- 172.21.15.131:0/2371470915 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404009e70 con 0x7fa414159a70 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.117+0000 7fa418b99700 1 -- 172.21.15.131:0/2371470915 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40400b860 con 0x7fa414159a70 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.117+0000 7fa418b99700 1 -- 172.21.15.131:0/2371470915 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40400bc00 con 0x7fa414159a70 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.117+0000 7fa41bdff700 1 -- 172.21.15.131:0/2371470915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 msgr2=0x7fa4140b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.117+0000 7fa41bdff700 1 --2- 172.21.15.131:0/2371470915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 0x7fa4140b0160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4140b66b0 tx=0x7fa404009250 comp rx=0 tx=0).stop 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.118+0000 7fa41bdff700 1 -- 172.21.15.131:0/2371470915 shutdown_connections 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.118+0000 7fa41bdff700 1 --2- 172.21.15.131:0/2371470915 >> v2:172.21.15.190:3300/0 conn(0x7fa4140b2a90 0x7fa4140b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.118+0000 7fa41bdff700 1 --2- 172.21.15.131:0/2371470915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 0x7fa4140b0160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.118+0000 7fa41bdff700 1 -- 172.21.15.131:0/2371470915 >> 172.21.15.131:0/2371470915 conn(0x7fa41406ece0 msgr2=0x7fa41406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.118+0000 7fa41bdff700 1 -- 172.21.15.131:0/2371470915 shutdown_connections 2023-12-26T21:37:51.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.118+0000 7fa41bdff700 1 -- 172.21.15.131:0/2371470915 wait complete. 2023-12-26T21:37:51.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.119+0000 7fa41bdff700 1 Processor -- start 2023-12-26T21:37:51.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.119+0000 7fa41bdff700 1 -- start start 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4140b2a90 0x7fa4140c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41bdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4141596a0 0x7fa4140c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41bdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 0x7fa4140c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41bdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41415a2b0 con 0x7fa414159a70 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41bdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140cc6b0 con 0x7fa4141596a0 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140cbe00 con 0x7fa4140b2a90 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41a39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 0x7fa4140c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41a39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 0x7fa4140c4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60858/0 (socket says 172.21.15.131:60858) 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41a39c700 1 -- 172.21.15.131:0/3302635064 learned_addr learned my addr 172.21.15.131:0/3302635064 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa419b9b700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4140b2a90 0x7fa4140c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.120+0000 7fa41939a700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4141596a0 0x7fa4140c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa41a39c700 1 -- 172.21.15.131:0/3302635064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4140b2a90 msgr2=0x7fa4140c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa41a39c700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4140b2a90 0x7fa4140c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa41a39c700 1 -- 172.21.15.131:0/3302635064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4141596a0 msgr2=0x7fa4140c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa41a39c700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4141596a0 0x7fa4140c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa41a39c700 1 -- 172.21.15.131:0/3302635064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040094f0 con 0x7fa414159a70 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa41a39c700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 0x7fa4140c4550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009e90 tx=0x7fa40c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa409ffb700 1 -- 172.21.15.131:0/3302635064 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00cb70 con 0x7fa414159a70 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa409ffb700 1 -- 172.21.15.131:0/3302635064 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c00f040 con 0x7fa414159a70 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa414155bc0 con 0x7fa414159a70 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa409ffb700 1 -- 172.21.15.131:0/3302635064 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c014480 con 0x7fa414159a70 2023-12-26T21:37:51.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.121+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41415cd80 con 0x7fa414159a70 2023-12-26T21:37:51.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.122+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4140cc840 con 0x7fa414159a70 2023-12-26T21:37:51.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.124+0000 7fa409ffb700 1 -- 172.21.15.131:0/3302635064 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa40c0147f0 con 0x7fa414159a70 2023-12-26T21:37:51.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.124+0000 7fa409ffb700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa3f8066f30 0x7fa3f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:51.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.125+0000 7fa409ffb700 1 -- 172.21.15.131:0/3302635064 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa40c0852b0 con 0x7fa414159a70 2023-12-26T21:37:51.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.125+0000 7fa419b9b700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa3f8066f30 0x7fa3f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:51.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.125+0000 7fa419b9b700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa3f8066f30 0x7fa3f80693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4140cd960 tx=0x7fa410009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:51.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.128+0000 7fa409ffb700 1 -- 172.21.15.131:0/3302635064 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa40c052480 con 0x7fa414159a70 2023-12-26T21:37:51.232 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-12-26T21:37:51.232+0000 7fe51f3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:37:51.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:51.365+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa4140657e0 con 0x7fa414159a70 2023-12-26T21:37:52.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.065+0000 7fa409ffb700 1 -- 172.21.15.131:0/3302635064 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa40c055aa0 con 0x7fa414159a70 2023-12-26T21:37:52.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.067+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa3f8066f30 msgr2=0x7fa3f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:52.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.067+0000 7fa41bdff700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa3f8066f30 0x7fa3f80693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4140cd960 tx=0x7fa410009040 comp rx=0 tx=0).stop 2023-12-26T21:37:52.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.067+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 msgr2=0x7fa4140c4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:52.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.068+0000 7fa41bdff700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 0x7fa4140c4550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009e90 tx=0x7fa40c00b040 comp rx=0 tx=0).stop 2023-12-26T21:37:52.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.068+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 shutdown_connections 2023-12-26T21:37:52.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.068+0000 7fa41bdff700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa414159a70 0x7fa4140c4550 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:52.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.068+0000 7fa41bdff700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa3f8066f30 0x7fa3f80693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:52.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.068+0000 7fa41bdff700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4140b2a90 0x7fa4140c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:52.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.068+0000 7fa41bdff700 1 --2- 172.21.15.131:0/3302635064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4141596a0 0x7fa4140c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:52.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.068+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 >> 172.21.15.131:0/3302635064 conn(0x7fa41406ece0 msgr2=0x7fa414157770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:52.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.068+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 shutdown_connections 2023-12-26T21:37:52.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.068+0000 7fa41bdff700 1 -- 172.21.15.131:0/3302635064 wait complete. 2023-12-26T21:37:52.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T21:37:52.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.281+0000 7ff0527ff700 1 Processor -- start 2023-12-26T21:37:52.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.281+0000 7ff0527ff700 1 -- start start 2023-12-26T21:37:52.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.282+0000 7ff0527ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff04c159560 0x7ff04c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:52.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.282+0000 7ff0527ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff04c06a1a0 0x7ff04c068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:52.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff0527ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04c068a80 con 0x7ff04c159930 2023-12-26T21:37:52.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff0527ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c0cac50 con 0x7ff04c159560 2023-12-26T21:37:52.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff0527ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c15da00 con 0x7ff04c06a1a0 2023-12-26T21:37:52.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff04bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff04c159560 0x7ff04c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:52.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff04bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff04c159560 0x7ff04c15d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60874/0 (socket says 172.21.15.131:60874) 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff04bfff700 1 -- 172.21.15.131:0/1024396883 learned_addr learned my addr 172.21.15.131:0/1024396883 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff04b7fe700 1 --1- 172.21.15.131:0/1024396883 >> v1:172.21.15.131:6789/0 conn(0x7ff04c159930 0x7ff04c0b11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35714/0 (socket says 172.21.15.131:35714) 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff050d9c700 1 --2- 172.21.15.131:0/1024396883 >> v2:172.21.15.190:3300/0 conn(0x7ff04c06a1a0 0x7ff04c068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff04bfff700 1 -- 172.21.15.131:0/1024396883 >> v2:172.21.15.190:3300/0 conn(0x7ff04c06a1a0 msgr2=0x7ff04c068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff04bfff700 1 --2- 172.21.15.131:0/1024396883 >> v2:172.21.15.190:3300/0 conn(0x7ff04c06a1a0 0x7ff04c068f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff04bfff700 1 -- 172.21.15.131:0/1024396883 >> v1:172.21.15.131:6789/0 conn(0x7ff04c159930 legacy=0x7ff04c0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff04bfff700 1 -- 172.21.15.131:0/1024396883 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c15dce0 con 0x7ff04c159560 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.283+0000 7ff04affd700 1 -- 172.21.15.131:0/1024396883 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856451247 0 0) 0x7ff04c068a80 con 0x7ff04c159930 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff04bfff700 1 --2- 172.21.15.131:0/1024396883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c159560 0x7ff04c15d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0c1dd0 tx=0x7ff03c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92db2d0c9b6acd4b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff04affd700 1 -- 172.21.15.131:0/1024396883 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c009e70 con 0x7ff04c159560 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff04affd700 1 -- 172.21.15.131:0/1024396883 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c00b860 con 0x7ff04c159560 2023-12-26T21:37:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff04affd700 1 -- 172.21.15.131:0/1024396883 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00bc00 con 0x7ff04c159560 2023-12-26T21:37:52.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff0527ff700 1 -- 172.21.15.131:0/1024396883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c159560 msgr2=0x7ff04c15d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:52.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff0527ff700 1 --2- 172.21.15.131:0/1024396883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c159560 0x7ff04c15d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0c1dd0 tx=0x7ff03c009250 comp rx=0 tx=0).stop 2023-12-26T21:37:52.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff0527ff700 1 -- 172.21.15.131:0/1024396883 shutdown_connections 2023-12-26T21:37:52.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff0527ff700 1 --2- 172.21.15.131:0/1024396883 >> v2:172.21.15.190:3300/0 conn(0x7ff04c06a1a0 0x7ff04c068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:52.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff0527ff700 1 --2- 172.21.15.131:0/1024396883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c159560 0x7ff04c15d300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.284+0000 7ff0527ff700 1 -- 172.21.15.131:0/1024396883 >> 172.21.15.131:0/1024396883 conn(0x7ff04c0ce3d0 msgr2=0x7ff04c0ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.285+0000 7ff0527ff700 1 -- 172.21.15.131:0/1024396883 shutdown_connections 2023-12-26T21:37:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.285+0000 7ff0527ff700 1 -- 172.21.15.131:0/1024396883 wait complete. 2023-12-26T21:37:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.287+0000 7ff0527ff700 1 Processor -- start 2023-12-26T21:37:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.287+0000 7ff0527ff700 1 -- start start 2023-12-26T21:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff0527ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c06a1a0 0x7ff04c145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff0527ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff04c159560 0x7ff04c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff0527ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff04c159930 0x7ff04c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff0527ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c16a5b0 con 0x7ff04c06a1a0 2023-12-26T21:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff0527ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c1699a0 con 0x7ff04c159930 2023-12-26T21:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff0527ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c169c80 con 0x7ff04c159560 2023-12-26T21:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c06a1a0 0x7ff04c145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c06a1a0 0x7ff04c145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60886/0 (socket says 172.21.15.131:60886) 2023-12-26T21:37:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff04bfff700 1 -- 172.21.15.131:0/2269409500 learned_addr learned my addr 172.21.15.131:0/2269409500 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff050d9c700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff04c159930 0x7ff04c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.288+0000 7ff04b7fe700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff04c159560 0x7ff04c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff04bfff700 1 -- 172.21.15.131:0/2269409500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff04c159560 msgr2=0x7ff04c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff04bfff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff04c159560 0x7ff04c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff04bfff700 1 -- 172.21.15.131:0/2269409500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff04c159930 msgr2=0x7ff04c0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff04bfff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff04c159930 0x7ff04c0cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff04bfff700 1 -- 172.21.15.131:0/2269409500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c0094f0 con 0x7ff04c06a1a0 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff050d9c700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff04c159930 0x7ff04c0cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff04bfff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c06a1a0 0x7ff04c145380 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff04c06efe0 tx=0x7ff03c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff033fff700 1 -- 172.21.15.131:0/2269409500 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00b9d0 con 0x7ff04c06a1a0 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff033fff700 1 -- 172.21.15.131:0/2269409500 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c00bb30 con 0x7ff04c06a1a0 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04c169f60 con 0x7ff04c06a1a0 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff033fff700 1 -- 172.21.15.131:0/2269409500 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c026640 con 0x7ff04c06a1a0 2023-12-26T21:37:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.289+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04c0c57a0 con 0x7ff04c06a1a0 2023-12-26T21:37:52.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.290+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04c0657e0 con 0x7ff04c06a1a0 2023-12-26T21:37:52.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.291+0000 7ff033fff700 1 -- 172.21.15.131:0/2269409500 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff03c023030 con 0x7ff04c06a1a0 2023-12-26T21:37:52.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.292+0000 7ff033fff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff038066f30 0x7ff0380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:52.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.292+0000 7ff033fff700 1 -- 172.21.15.131:0/2269409500 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff03c085e10 con 0x7ff04c06a1a0 2023-12-26T21:37:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.292+0000 7ff04b7fe700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff038066f30 0x7ff0380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.293+0000 7ff04b7fe700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff038066f30 0x7ff0380693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff040007fd0 tx=0x7ff040007e10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.296+0000 7ff033fff700 1 -- 172.21.15.131:0/2269409500 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff03c053100 con 0x7ff04c06a1a0 2023-12-26T21:37:52.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:52.533+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff04c0c9f40 con 0x7ff04c06a1a0 2023-12-26T21:37:53.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.074+0000 7ff033fff700 1 -- 172.21.15.131:0/2269409500 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff03c009350 con 0x7ff04c06a1a0 2023-12-26T21:37:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff038066f30 msgr2=0x7ff0380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff038066f30 0x7ff0380693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff040007fd0 tx=0x7ff040007e10 comp rx=0 tx=0).stop 2023-12-26T21:37:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c06a1a0 msgr2=0x7ff04c145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c06a1a0 0x7ff04c145380 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff04c06efe0 tx=0x7ff03c017000 comp rx=0 tx=0).stop 2023-12-26T21:37:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 shutdown_connections 2023-12-26T21:37:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff04c159930 0x7ff04c0cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff04c06a1a0 0x7ff04c145380 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff038066f30 0x7ff0380693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 --2- 172.21.15.131:0/2269409500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff04c159560 0x7ff04c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 >> 172.21.15.131:0/2269409500 conn(0x7ff04c0ce3d0 msgr2=0x7ff04c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:53.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 shutdown_connections 2023-12-26T21:37:53.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.077+0000 7ff0527ff700 1 -- 172.21.15.131:0/2269409500 wait complete. 2023-12-26T21:37:53.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:37:53.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.285+0000 7f16dc9ff700 1 Processor -- start 2023-12-26T21:37:53.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.285+0000 7f16dc9ff700 1 -- start start 2023-12-26T21:37:53.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.286+0000 7f16dc9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d80b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.286+0000 7f16dc9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f16d80b2a90 0x7f16d80b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.286+0000 7f16dc9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d80afcd0 con 0x7f16d81596a0 2023-12-26T21:37:53.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.286+0000 7f16dc9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d80b3e70 con 0x7f16d80b2a90 2023-12-26T21:37:53.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.286+0000 7f16dc9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80b0f20 con 0x7f16d8159a70 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d5d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d80b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d5d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d80b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:59852/0 (socket says 172.21.15.131:59852) 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d6d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f16d80b2a90 0x7f16d80b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d5d9b700 1 -- 172.21.15.131:0/1437704244 learned_addr learned my addr 172.21.15.131:0/1437704244 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d659c700 1 --1- 172.21.15.131:0/1437704244 >> v1:172.21.15.190:6789/0 conn(0x7f16d81596a0 0x7f16d80b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:60492/0 (socket says 172.21.15.131:60492) 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d559a700 1 -- 172.21.15.131:0/1437704244 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542461791 0 0) 0x7f16d80afcd0 con 0x7f16d81596a0 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d559a700 1 -- 172.21.15.131:0/1437704244 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c0003570 con 0x7f16d81596a0 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d5d9b700 1 -- 172.21.15.131:0/1437704244 >> v2:172.21.15.190:3300/0 conn(0x7f16d80b2a90 msgr2=0x7f16d80b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d5d9b700 1 --2- 172.21.15.131:0/1437704244 >> v2:172.21.15.190:3300/0 conn(0x7f16d80b2a90 0x7f16d80b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.287+0000 7f16d5d9b700 1 -- 172.21.15.131:0/1437704244 >> v1:172.21.15.190:6789/0 conn(0x7f16d81596a0 legacy=0x7f16d80b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.288+0000 7f16d5d9b700 1 -- 172.21.15.131:0/1437704244 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80b3540 con 0x7f16d8159a70 2023-12-26T21:37:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.288+0000 7f16d6d9d700 1 --2- 172.21.15.131:0/1437704244 >> v2:172.21.15.190:3300/0 conn(0x7f16d80b2a90 0x7f16d80b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:53.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.288+0000 7f16d5d9b700 1 --2- 172.21.15.131:0/1437704244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d80b0160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0099b0 tx=0x7f16cc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e66563f0fb76f521 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:53.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.288+0000 7f16d559a700 1 -- 172.21.15.131:0/1437704244 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc00b680 con 0x7f16d8159a70 2023-12-26T21:37:53.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.288+0000 7f16d559a700 1 -- 172.21.15.131:0/1437704244 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc00b7e0 con 0x7f16d8159a70 2023-12-26T21:37:53.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.288+0000 7f16d559a700 1 -- 172.21.15.131:0/1437704244 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc00bb80 con 0x7f16d8159a70 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.289+0000 7f16dc9ff700 1 -- 172.21.15.131:0/1437704244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 msgr2=0x7f16d80b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.289+0000 7f16dc9ff700 1 --2- 172.21.15.131:0/1437704244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d80b0160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0099b0 tx=0x7f16cc009250 comp rx=0 tx=0).stop 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.289+0000 7f16dc9ff700 1 -- 172.21.15.131:0/1437704244 shutdown_connections 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.289+0000 7f16dc9ff700 1 --2- 172.21.15.131:0/1437704244 >> v2:172.21.15.190:3300/0 conn(0x7f16d80b2a90 0x7f16d80b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.289+0000 7f16dc9ff700 1 --2- 172.21.15.131:0/1437704244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d80b0160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.289+0000 7f16dc9ff700 1 -- 172.21.15.131:0/1437704244 >> 172.21.15.131:0/1437704244 conn(0x7f16d806ece0 msgr2=0x7f16d806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.289+0000 7f16dc9ff700 1 -- 172.21.15.131:0/1437704244 shutdown_connections 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.289+0000 7f16dc9ff700 1 -- 172.21.15.131:0/1437704244 wait complete. 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.290+0000 7f16dc9ff700 1 Processor -- start 2023-12-26T21:37:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.290+0000 7f16dc9ff700 1 -- start start 2023-12-26T21:37:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.291+0000 7f16dc9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16d80b2a90 0x7f16d8150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.291+0000 7f16dc9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16d81596a0 0x7f16d814fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.291+0000 7f16dc9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d8150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.291+0000 7f16dc9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80cb200 con 0x7f16d80b2a90 2023-12-26T21:37:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.291+0000 7f16dc9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80c9630 con 0x7f16d81596a0 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.291+0000 7f16dc9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80c9910 con 0x7f16d8159a70 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16d80b2a90 0x7f16d8150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16d80b2a90 0x7f16d8150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60898/0 (socket says 172.21.15.131:60898) 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d659c700 1 -- 172.21.15.131:0/3716778927 learned_addr learned my addr 172.21.15.131:0/3716778927 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d5d9b700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16d81596a0 0x7f16d814fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d6d9d700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d8150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d659c700 1 -- 172.21.15.131:0/3716778927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 msgr2=0x7f16d8150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d659c700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d8150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d659c700 1 -- 172.21.15.131:0/3716778927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16d81596a0 msgr2=0x7f16d814fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d659c700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16d81596a0 0x7f16d814fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d659c700 1 -- 172.21.15.131:0/3716778927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d0009a20 con 0x7f16d80b2a90 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.292+0000 7f16d659c700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16d80b2a90 0x7f16d8150800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f16c800c7c0 tx=0x7f16c8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.293+0000 7f16c67fc700 1 -- 172.21.15.131:0/3716778927 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8005a70 con 0x7f16d80b2a90 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.293+0000 7f16c67fc700 1 -- 172.21.15.131:0/3716778927 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c8005bf0 con 0x7f16d80b2a90 2023-12-26T21:37:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.293+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc0094f0 con 0x7f16d80b2a90 2023-12-26T21:37:53.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.293+0000 7f16c67fc700 1 -- 172.21.15.131:0/3716778927 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80070d0 con 0x7f16d80b2a90 2023-12-26T21:37:53.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.293+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d80cb450 con 0x7f16d80b2a90 2023-12-26T21:37:53.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.294+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d80657e0 con 0x7f16d80b2a90 2023-12-26T21:37:53.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.295+0000 7f16c67fc700 1 -- 172.21.15.131:0/3716778927 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f16c8007480 con 0x7f16d80b2a90 2023-12-26T21:37:53.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.295+0000 7f16c67fc700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16b8066f30 0x7f16b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.295+0000 7f16c67fc700 1 -- 172.21.15.131:0/3716778927 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f16c80839a0 con 0x7f16d80b2a90 2023-12-26T21:37:53.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.296+0000 7f16d5d9b700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16b8066f30 0x7f16b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.296+0000 7f16d5d9b700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16b8066f30 0x7f16b80693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0099b0 tx=0x7f16cc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:53.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.299+0000 7f16c67fc700 1 -- 172.21.15.131:0/3716778927 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16c8050cd0 con 0x7f16d80b2a90 2023-12-26T21:37:53.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.534+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16d80b3820 con 0x7f16d80b2a90 2023-12-26T21:37:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.535+0000 7f16c67fc700 1 -- 172.21.15.131:0/3716778927 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f16c8017020 con 0x7f16d80b2a90 2023-12-26T21:37:53.535 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:53.535 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1410949426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1410949426},{"type":"v1","addr":"172.21.15.190:6835","nonce":1410949426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2210886653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2210886653},{"type":"v1","addr":"172.21.15.131:6836","nonce":2210886653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4878,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/241955638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":241955638},{"type":"v1","addr":"172.21.15.131:6837","nonce":241955638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:37:49.516027+0000","modified":"2023-12-26T21:37:53.065572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6833/3890149779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3890149779},{"type":"v1","addr":"172.21.15.190:6833","nonce":3890149779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-12-26T21:37:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16b8066f30 msgr2=0x7f16b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16b8066f30 0x7f16b80693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0099b0 tx=0x7f16cc017000 comp rx=0 tx=0).stop 2023-12-26T21:37:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16d80b2a90 msgr2=0x7f16d8150800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16d80b2a90 0x7f16d8150800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f16c800c7c0 tx=0x7f16c8007e00 comp rx=0 tx=0).stop 2023-12-26T21:37:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 shutdown_connections 2023-12-26T21:37:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f16b8066f30 0x7f16b80693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16d8159a70 0x7f16d8150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16d80b2a90 0x7f16d8150800 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 --2- 172.21.15.131:0/3716778927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16d81596a0 0x7f16d814fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 >> 172.21.15.131:0/3716778927 conn(0x7f16d806ece0 msgr2=0x7f16d8157700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:53.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 shutdown_connections 2023-12-26T21:37:53.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.537+0000 7f16dc9ff700 1 -- 172.21.15.131:0/3716778927 wait complete. 2023-12-26T21:37:53.539 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2023-12-26T21:37:53.555 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.744+0000 7f7ec5fff700 1 Processor -- start 2023-12-26T21:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.744+0000 7f7ec5fff700 1 -- start start 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.745+0000 7f7ec5fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ec0159600 0x7f7ec00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ec5fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7ec00d01e0 0x7f7ec00ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ec5fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00cfd50 con 0x7f7ec01599d0 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ec5fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec00b1280 con 0x7f7ec00d01e0 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ec5fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec00cf140 con 0x7f7ec0159600 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ebffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7ec00d01e0 0x7f7ec00ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ebffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7ec00d01e0 0x7f7ec00ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60908/0 (socket says 172.21.15.131:60908) 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ebffff700 1 -- 172.21.15.131:0/1941391715 learned_addr learned my addr 172.21.15.131:0/1941391715 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ebf7fe700 1 --2- 172.21.15.131:0/1941391715 >> v2:172.21.15.190:3300/0 conn(0x7f7ec0159600 0x7f7ec00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ebffff700 1 -- 172.21.15.131:0/1941391715 >> v2:172.21.15.190:3300/0 conn(0x7f7ec0159600 msgr2=0x7f7ec00ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ebffff700 1 --2- 172.21.15.131:0/1941391715 >> v2:172.21.15.190:3300/0 conn(0x7f7ec0159600 0x7f7ec00ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ebffff700 1 -- 172.21.15.131:0/1941391715 >> v1:172.21.15.190:6789/0 conn(0x7f7ec01599d0 legacy=0x7f7ec015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ebffff700 1 -- 172.21.15.131:0/1941391715 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00cf420 con 0x7f7ec00d01e0 2023-12-26T21:37:53.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.746+0000 7f7ebf7fe700 1 --2- 172.21.15.131:0/1941391715 >> v2:172.21.15.190:3300/0 conn(0x7f7ec0159600 0x7f7ec00ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:53.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ebffff700 1 --2- 172.21.15.131:0/1941391715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 0x7f7ec00ccf00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0089850 tx=0x7f7eb8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3210b79906c913e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ebe7fc700 1 -- 172.21.15.131:0/1941391715 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb800e070 con 0x7f7ec00d01e0 2023-12-26T21:37:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ebe7fc700 1 -- 172.21.15.131:0/1941391715 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb800ba50 con 0x7f7ec00d01e0 2023-12-26T21:37:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ebe7fc700 1 -- 172.21.15.131:0/1941391715 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80092e0 con 0x7f7ec00d01e0 2023-12-26T21:37:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1941391715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 msgr2=0x7f7ec00ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ec5fff700 1 --2- 172.21.15.131:0/1941391715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 0x7f7ec00ccf00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0089850 tx=0x7f7eb8009840 comp rx=0 tx=0).stop 2023-12-26T21:37:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1941391715 shutdown_connections 2023-12-26T21:37:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ec5fff700 1 --2- 172.21.15.131:0/1941391715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 0x7f7ec00ccf00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ec5fff700 1 --2- 172.21.15.131:0/1941391715 >> v2:172.21.15.190:3300/0 conn(0x7f7ec0159600 0x7f7ec00ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.747+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1941391715 >> 172.21.15.131:0/1941391715 conn(0x7f7ec006ece0 msgr2=0x7f7ec006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.748+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1941391715 shutdown_connections 2023-12-26T21:37:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.748+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1941391715 wait complete. 2023-12-26T21:37:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.749+0000 7f7ec5fff700 1 Processor -- start 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.749+0000 7f7ec5fff700 1 -- start start 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ec5fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 0x7f7ec015f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ec5fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ec0159600 0x7f7ec0146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ec5fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ec01602e0 0x7f7ec00c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ec5fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec015edb0 con 0x7f7ec00d01e0 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ec5fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec015e1a0 con 0x7f7ec01602e0 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ec5fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec015e480 con 0x7f7ec0159600 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ebf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 0x7f7ec015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ebf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 0x7f7ec015f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60922/0 (socket says 172.21.15.131:60922) 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ebf7fe700 1 -- 172.21.15.131:0/1639152583 learned_addr learned my addr 172.21.15.131:0/1639152583 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ebffff700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ec01602e0 0x7f7ec00c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.750+0000 7f7ebeffd700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ec0159600 0x7f7ec0146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ebf7fe700 1 -- 172.21.15.131:0/1639152583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ec0159600 msgr2=0x7f7ec0146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ebf7fe700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ec0159600 0x7f7ec0146000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ebf7fe700 1 -- 172.21.15.131:0/1639152583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ec01602e0 msgr2=0x7f7ec00c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ebf7fe700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ec01602e0 0x7f7ec00c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ebf7fe700 1 -- 172.21.15.131:0/1639152583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb8009ae0 con 0x7f7ec00d01e0 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ebf7fe700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 0x7f7ec015f720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000e920 tx=0x7f7eb0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ea77fe700 1 -- 172.21.15.131:0/1639152583 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0005a00 con 0x7f7ec00d01e0 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ea77fe700 1 -- 172.21.15.131:0/1639152583 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb0005b60 con 0x7f7ec00d01e0 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec015e760 con 0x7f7ec00d01e0 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ea77fe700 1 -- 172.21.15.131:0/1639152583 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0006020 con 0x7f7ec00d01e0 2023-12-26T21:37:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.751+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec015fc60 con 0x7f7ec00d01e0 2023-12-26T21:37:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.752+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec00657e0 con 0x7f7ec00d01e0 2023-12-26T21:37:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.753+0000 7f7ea77fe700 1 -- 172.21.15.131:0/1639152583 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7eb00063d0 con 0x7f7ec00d01e0 2023-12-26T21:37:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.754+0000 7f7ea77fe700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7eac066f60 0x7f7eac069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.754+0000 7f7ea77fe700 1 -- 172.21.15.131:0/1639152583 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7eb0084600 con 0x7f7ec00d01e0 2023-12-26T21:37:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.755+0000 7f7ebeffd700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7eac066f60 0x7f7eac069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.755+0000 7f7ebeffd700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7eac066f60 0x7f7eac069410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7ec016d190 tx=0x7f7eb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.758+0000 7f7ea77fe700 1 -- 172.21.15.131:0/1639152583 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7eb00517d0 con 0x7f7ec00d01e0 2023-12-26T21:37:53.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.952+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ec00cf700 con 0x7f7ec00d01e0 2023-12-26T21:37:53.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.953+0000 7f7ea77fe700 1 -- 172.21.15.131:0/1639152583 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7eb0054df0 con 0x7f7ec00d01e0 2023-12-26T21:37:53.953 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:53.953 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":35,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:37:50.996932+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:37:47.268950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-26T21:37:48.501427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:37:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7eac066f60 msgr2=0x7f7eac069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7eac066f60 0x7f7eac069410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7ec016d190 tx=0x7f7eb4009040 comp rx=0 tx=0).stop 2023-12-26T21:37:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 msgr2=0x7f7ec015f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 0x7f7ec015f720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000e920 tx=0x7f7eb0008040 comp rx=0 tx=0).stop 2023-12-26T21:37:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 shutdown_connections 2023-12-26T21:37:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ec01602e0 0x7f7ec00c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec00d01e0 0x7f7ec015f720 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7eac066f60 0x7f7eac069410 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 --2- 172.21.15.131:0/1639152583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ec0159600 0x7f7ec0146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:53.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 >> 172.21.15.131:0/1639152583 conn(0x7f7ec006ece0 msgr2=0x7f7ec00d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:53.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.955+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 shutdown_connections 2023-12-26T21:37:53.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:53.956+0000 7f7ec5fff700 1 -- 172.21.15.131:0/1639152583 wait complete. 2023-12-26T21:37:53.974 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T21:37:53.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:37:54.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.165+0000 7f296f7ff700 1 Processor -- start 2023-12-26T21:37:54.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.165+0000 7f296f7ff700 1 -- start start 2023-12-26T21:37:54.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.166+0000 7f296f7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f29681599d0 0x7f29680ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.166+0000 7f296f7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.166+0000 7f296f7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29680cfd50 con 0x7f2968159600 2023-12-26T21:37:54.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.166+0000 7f296f7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29680b1280 con 0x7f29681599d0 2023-12-26T21:37:54.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.166+0000 7f296f7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29680cf140 con 0x7f29680d01e0 2023-12-26T21:37:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f296dd9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f296dd9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:59884/0 (socket says 172.21.15.131:59884) 2023-12-26T21:37:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f296dd9c700 1 -- 172.21.15.131:0/487217463 learned_addr learned my addr 172.21.15.131:0/487217463 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f296d59b700 1 --1- 172.21.15.131:0/487217463 >> v1:172.21.15.190:6789/0 conn(0x7f2968159600 0x7f296815dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:60516/0 (socket says 172.21.15.131:60516) 2023-12-26T21:37:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f296cd9a700 1 --2- 172.21.15.131:0/487217463 >> v2:172.21.15.190:3300/0 conn(0x7f29681599d0 0x7f29680ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f295ffff700 1 -- 172.21.15.131:0/487217463 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010909365 0 0) 0x7f29680cfd50 con 0x7f2968159600 2023-12-26T21:37:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f296dd9c700 1 -- 172.21.15.131:0/487217463 >> v2:172.21.15.190:3300/0 conn(0x7f29681599d0 msgr2=0x7f29680ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f296dd9c700 1 --2- 172.21.15.131:0/487217463 >> v2:172.21.15.190:3300/0 conn(0x7f29681599d0 0x7f29680ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f296dd9c700 1 -- 172.21.15.131:0/487217463 >> v1:172.21.15.190:6789/0 conn(0x7f2968159600 legacy=0x7f296815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.167+0000 7f296dd9c700 1 -- 172.21.15.131:0/487217463 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29680cf420 con 0x7f29680d01e0 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f296dd9c700 1 --2- 172.21.15.131:0/487217463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680ccf00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2960009f20 tx=0x7f2960009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109bcfd708656bd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f295ffff700 1 -- 172.21.15.131:0/487217463 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296000e070 con 0x7f29680d01e0 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f295ffff700 1 -- 172.21.15.131:0/487217463 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296000b9d0 con 0x7f29680d01e0 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f295ffff700 1 -- 172.21.15.131:0/487217463 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29600092e0 con 0x7f29680d01e0 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f296f7ff700 1 -- 172.21.15.131:0/487217463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 msgr2=0x7f29680ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f296f7ff700 1 --2- 172.21.15.131:0/487217463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680ccf00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2960009f20 tx=0x7f2960009840 comp rx=0 tx=0).stop 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f296f7ff700 1 -- 172.21.15.131:0/487217463 shutdown_connections 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f296f7ff700 1 --2- 172.21.15.131:0/487217463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680ccf00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f296f7ff700 1 --2- 172.21.15.131:0/487217463 >> v2:172.21.15.190:3300/0 conn(0x7f29681599d0 0x7f29680ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.168+0000 7f296f7ff700 1 -- 172.21.15.131:0/487217463 >> 172.21.15.131:0/487217463 conn(0x7f296806ece0 msgr2=0x7f296806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.169+0000 7f296f7ff700 1 -- 172.21.15.131:0/487217463 shutdown_connections 2023-12-26T21:37:54.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.169+0000 7f296f7ff700 1 -- 172.21.15.131:0/487217463 wait complete. 2023-12-26T21:37:54.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.170+0000 7f296f7ff700 1 Processor -- start 2023-12-26T21:37:54.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.170+0000 7f296f7ff700 1 -- start start 2023-12-26T21:37:54.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.171+0000 7f296f7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.171+0000 7f296f7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968159600 0x7f29680b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.171+0000 7f296f7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29680b38d0 0x7f2968153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.171+0000 7f296f7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29680b3e50 con 0x7f2968159600 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.171+0000 7f296f7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29680b4130 con 0x7f29680b38d0 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.171+0000 7f296f7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29680b4410 con 0x7f29680d01e0 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.171+0000 7f296cd9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968159600 0x7f29680b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.171+0000 7f296cd9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968159600 0x7f29680b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60938/0 (socket says 172.21.15.131:60938) 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.171+0000 7f296cd9a700 1 -- 172.21.15.131:0/3335275026 learned_addr learned my addr 172.21.15.131:0/3335275026 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296dd9c700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29680b38d0 0x7f2968153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296d59b700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296cd9a700 1 -- 172.21.15.131:0/3335275026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 msgr2=0x7f29680b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296cd9a700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296cd9a700 1 -- 172.21.15.131:0/3335275026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29680b38d0 msgr2=0x7f2968153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296cd9a700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29680b38d0 0x7f2968153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296cd9a700 1 -- 172.21.15.131:0/3335275026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2960009ae0 con 0x7f2968159600 2023-12-26T21:37:54.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296d59b700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:54.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296cd9a700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968159600 0x7f29680b3390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f295800e930 tx=0x7f29580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f295d7fa700 1 -- 172.21.15.131:0/3335275026 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958013680 con 0x7f2968159600 2023-12-26T21:37:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f295d7fa700 1 -- 172.21.15.131:0/3335275026 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29580137e0 con 0x7f2968159600 2023-12-26T21:37:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296815af20 con 0x7f2968159600 2023-12-26T21:37:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f295d7fa700 1 -- 172.21.15.131:0/3335275026 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958013b80 con 0x7f2968159600 2023-12-26T21:37:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.172+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29680b5560 con 0x7f2968159600 2023-12-26T21:37:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.173+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296815ad80 con 0x7f2968159600 2023-12-26T21:37:54.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.174+0000 7f295d7fa700 1 -- 172.21.15.131:0/3335275026 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f295800c460 con 0x7f2968159600 2023-12-26T21:37:54.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.175+0000 7f295d7fa700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f294c066f30 0x7f294c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.175+0000 7f295d7fa700 1 -- 172.21.15.131:0/3335275026 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2958084710 con 0x7f2968159600 2023-12-26T21:37:54.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.175+0000 7f296d59b700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f294c066f30 0x7f294c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.176+0000 7f296d59b700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f294c066f30 0x7f294c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2964001230 tx=0x7f2964007c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:54.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.180+0000 7f295d7fa700 1 -- 172.21.15.131:0/3335275026 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f296815ad80 con 0x7f2968159600 2023-12-26T21:37:54.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.370+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29680b42a0 con 0x7f294c066f30 2023-12-26T21:37:54.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.380+0000 7f295d7fa700 1 -- 172.21.15.131:0/3335275026 <== mgr.4100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288665 (secure 0 0 0) 0x7f29680b42a0 con 0x7f294c066f30 2023-12-26T21:37:54.384 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f294c066f30 msgr2=0x7f294c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f294c066f30 0x7f294c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2964001230 tx=0x7f2964007c60 comp rx=0 tx=0).stop 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968159600 msgr2=0x7f29680b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968159600 0x7f29680b3390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f295800e930 tx=0x7f29580091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 shutdown_connections 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29680b38d0 0x7f2968153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f294c066f30 0x7f294c0693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29680d01e0 0x7f29680b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 --2- 172.21.15.131:0/3335275026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968159600 0x7f29680b3390 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 >> 172.21.15.131:0/3335275026 conn(0x7f296806ece0 msgr2=0x7f29680cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:54.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 shutdown_connections 2023-12-26T21:37:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.384+0000 7f296f7ff700 1 -- 172.21.15.131:0/3335275026 wait complete. 2023-12-26T21:37:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:37:54.403 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-12-26T21:37:53.011173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382496,"kb_used_data":4192,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592224,"statfs":{"total":773094113280,"available":770654437376,"internally_reserved":0,"allocated":4292608,"data_stored":2358092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.978802"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022566+0000","last_change":"2023-12-26T21:37:50.022566+0000","last_active":"2023-12-26T21:37:50.022566+0000","last_peered":"2023-12-26T21:37:50.022566+0000","last_clean":"2023-12-26T21:37:50.022566+0000","last_became_active":"2023-12-26T21:37:50.022341+0000","last_became_peered":"2023-12-26T21:37:50.022341+0000","last_unstale":"2023-12-26T21:37:50.022566+0000","last_undegraded":"2023-12-26T21:37:50.022566+0000","last_fullsized":"2023-12-26T21:37:50.022566+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:01.590388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029202+0000","last_change":"2023-12-26T21:37:50.023094+0000","last_active":"2023-12-26T21:37:50.029202+0000","last_peered":"2023-12-26T21:37:50.029202+0000","last_clean":"2023-12-26T21:37:50.029202+0000","last_became_active":"2023-12-26T21:37:50.022865+0000","last_became_peered":"2023-12-26T21:37:50.022865+0000","last_unstale":"2023-12-26T21:37:50.029202+0000","last_undegraded":"2023-12-26T21:37:50.029202+0000","last_fullsized":"2023-12-26T21:37:50.029202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:41:45.917944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015939+0000","last_change":"2023-12-26T21:37:50.014051+0000","last_active":"2023-12-26T21:37:50.015939+0000","last_peered":"2023-12-26T21:37:50.015939+0000","last_clean":"2023-12-26T21:37:50.015939+0000","last_became_active":"2023-12-26T21:37:50.013797+0000","last_became_peered":"2023-12-26T21:37:50.013797+0000","last_unstale":"2023-12-26T21:37:50.015939+0000","last_undegraded":"2023-12-26T21:37:50.015939+0000","last_fullsized":"2023-12-26T21:37:50.015939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:25:10.241763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.032029+0000","last_change":"2023-12-26T21:37:50.032029+0000","last_active":"2023-12-26T21:37:50.032029+0000","last_peered":"2023-12-26T21:37:50.032029+0000","last_clean":"2023-12-26T21:37:50.032029+0000","last_became_active":"2023-12-26T21:37:50.031732+0000","last_became_peered":"2023-12-26T21:37:50.031732+0000","last_unstale":"2023-12-26T21:37:50.032029+0000","last_undegraded":"2023-12-26T21:37:50.032029+0000","last_fullsized":"2023-12-26T21:37:50.032029+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:18:48.013567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.028101+0000","last_change":"2023-12-26T21:37:50.018511+0000","last_active":"2023-12-26T21:37:50.028101+0000","last_peered":"2023-12-26T21:37:50.028101+0000","last_clean":"2023-12-26T21:37:50.028101+0000","last_became_active":"2023-12-26T21:37:50.018249+0000","last_became_peered":"2023-12-26T21:37:50.018249+0000","last_unstale":"2023-12-26T21:37:50.028101+0000","last_undegraded":"2023-12-26T21:37:50.028101+0000","last_fullsized":"2023-12-26T21:37:50.028101+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:35.444063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.026118+0000","last_change":"2023-12-26T21:37:50.019338+0000","last_active":"2023-12-26T21:37:50.026118+0000","last_peered":"2023-12-26T21:37:50.026118+0000","last_clean":"2023-12-26T21:37:50.026118+0000","last_became_active":"2023-12-26T21:37:50.019083+0000","last_became_peered":"2023-12-26T21:37:50.019083+0000","last_unstale":"2023-12-26T21:37:50.026118+0000","last_undegraded":"2023-12-26T21:37:50.026118+0000","last_fullsized":"2023-12-26T21:37:50.026118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:53:29.786265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015805+0000","last_change":"2023-12-26T21:37:50.015805+0000","last_active":"2023-12-26T21:37:50.015805+0000","last_peered":"2023-12-26T21:37:50.015805+0000","last_clean":"2023-12-26T21:37:50.015805+0000","last_became_active":"2023-12-26T21:37:50.015330+0000","last_became_peered":"2023-12-26T21:37:50.015330+0000","last_unstale":"2023-12-26T21:37:50.015805+0000","last_undegraded":"2023-12-26T21:37:50.015805+0000","last_fullsized":"2023-12-26T21:37:50.015805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:32:16.784548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.020147+0000","last_change":"2023-12-26T21:37:50.020147+0000","last_active":"2023-12-26T21:37:50.020147+0000","last_peered":"2023-12-26T21:37:50.020147+0000","last_clean":"2023-12-26T21:37:50.020147+0000","last_became_active":"2023-12-26T21:37:50.019567+0000","last_became_peered":"2023-12-26T21:37:50.019567+0000","last_unstale":"2023-12-26T21:37:50.020147+0000","last_undegraded":"2023-12-26T21:37:50.020147+0000","last_fullsized":"2023-12-26T21:37:50.020147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:17:52.028697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.023970+0000","last_change":"2023-12-26T21:37:50.022267+0000","last_active":"2023-12-26T21:37:50.023970+0000","last_peered":"2023-12-26T21:37:50.023970+0000","last_clean":"2023-12-26T21:37:50.023970+0000","last_became_active":"2023-12-26T21:37:50.021937+0000","last_became_peered":"2023-12-26T21:37:50.021937+0000","last_unstale":"2023-12-26T21:37:50.023970+0000","last_undegraded":"2023-12-26T21:37:50.023970+0000","last_fullsized":"2023-12-26T21:37:50.023970+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:44.548035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.024140+0000","last_change":"2023-12-26T21:37:50.024140+0000","last_active":"2023-12-26T21:37:50.024140+0000","last_peered":"2023-12-26T21:37:50.024140+0000","last_clean":"2023-12-26T21:37:50.024140+0000","last_became_active":"2023-12-26T21:37:50.023412+0000","last_became_peered":"2023-12-26T21:37:50.023412+0000","last_unstale":"2023-12-26T21:37:50.024140+0000","last_undegraded":"2023-12-26T21:37:50.024140+0000","last_fullsized":"2023-12-26T21:37:50.024140+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:45:20.209845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029578+0000","last_change":"2023-12-26T21:37:50.027938+0000","last_active":"2023-12-26T21:37:50.029578+0000","last_peered":"2023-12-26T21:37:50.029578+0000","last_clean":"2023-12-26T21:37:50.029578+0000","last_became_active":"2023-12-26T21:37:50.027689+0000","last_became_peered":"2023-12-26T21:37:50.027689+0000","last_unstale":"2023-12-26T21:37:50.029578+0000","last_undegraded":"2023-12-26T21:37:50.029578+0000","last_fullsized":"2023-12-26T21:37:50.029578+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:57:50.330517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.014727+0000","last_change":"2023-12-26T21:37:50.011664+0000","last_active":"2023-12-26T21:37:50.014727+0000","last_peered":"2023-12-26T21:37:50.014727+0000","last_clean":"2023-12-26T21:37:50.014727+0000","last_became_active":"2023-12-26T21:37:50.011407+0000","last_became_peered":"2023-12-26T21:37:50.011407+0000","last_unstale":"2023-12-26T21:37:50.014727+0000","last_undegraded":"2023-12-26T21:37:50.014727+0000","last_fullsized":"2023-12-26T21:37:50.014727+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:57:45.742847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029851+0000","last_change":"2023-12-26T21:37:50.026678+0000","last_active":"2023-12-26T21:37:50.029851+0000","last_peered":"2023-12-26T21:37:50.029851+0000","last_clean":"2023-12-26T21:37:50.029851+0000","last_became_active":"2023-12-26T21:37:50.026387+0000","last_became_peered":"2023-12-26T21:37:50.026387+0000","last_unstale":"2023-12-26T21:37:50.029851+0000","last_undegraded":"2023-12-26T21:37:50.029851+0000","last_fullsized":"2023-12-26T21:37:50.029851+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:01:59.418380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021785+0000","last_change":"2023-12-26T21:37:50.021785+0000","last_active":"2023-12-26T21:37:50.021785+0000","last_peered":"2023-12-26T21:37:50.021785+0000","last_clean":"2023-12-26T21:37:50.021785+0000","last_became_active":"2023-12-26T21:37:50.020946+0000","last_became_peered":"2023-12-26T21:37:50.020946+0000","last_unstale":"2023-12-26T21:37:50.021785+0000","last_undegraded":"2023-12-26T21:37:50.021785+0000","last_fullsized":"2023-12-26T21:37:50.021785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:17:52.028697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021315+0000","last_change":"2023-12-26T21:37:50.014903+0000","last_active":"2023-12-26T21:37:50.021315+0000","last_peered":"2023-12-26T21:37:50.021315+0000","last_clean":"2023-12-26T21:37:50.021315+0000","last_became_active":"2023-12-26T21:37:50.014641+0000","last_became_peered":"2023-12-26T21:37:50.014641+0000","last_unstale":"2023-12-26T21:37:50.021315+0000","last_undegraded":"2023-12-26T21:37:50.021315+0000","last_fullsized":"2023-12-26T21:37:50.021315+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:31.976009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.018923+0000","last_change":"2023-12-26T21:37:50.009856+0000","last_active":"2023-12-26T21:37:50.018923+0000","last_peered":"2023-12-26T21:37:50.018923+0000","last_clean":"2023-12-26T21:37:50.018923+0000","last_became_active":"2023-12-26T21:37:50.009584+0000","last_became_peered":"2023-12-26T21:37:50.009584+0000","last_unstale":"2023-12-26T21:37:50.018923+0000","last_undegraded":"2023-12-26T21:37:50.018923+0000","last_fullsized":"2023-12-26T21:37:50.018923+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:35.444063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022461+0000","last_change":"2023-12-26T21:37:50.016920+0000","last_active":"2023-12-26T21:37:50.022461+0000","last_peered":"2023-12-26T21:37:50.022461+0000","last_clean":"2023-12-26T21:37:50.022461+0000","last_became_active":"2023-12-26T21:37:50.016668+0000","last_became_peered":"2023-12-26T21:37:50.016668+0000","last_unstale":"2023-12-26T21:37:50.022461+0000","last_undegraded":"2023-12-26T21:37:50.022461+0000","last_fullsized":"2023-12-26T21:37:50.022461+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:06:15.016173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.014665+0000","last_change":"2023-12-26T21:37:50.010453+0000","last_active":"2023-12-26T21:37:50.014665+0000","last_peered":"2023-12-26T21:37:50.014665+0000","last_clean":"2023-12-26T21:37:50.014665+0000","last_became_active":"2023-12-26T21:37:50.010137+0000","last_became_peered":"2023-12-26T21:37:50.010137+0000","last_unstale":"2023-12-26T21:37:50.014665+0000","last_undegraded":"2023-12-26T21:37:50.014665+0000","last_fullsized":"2023-12-26T21:37:50.014665+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:37:58.777798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.008907+0000","last_change":"2023-12-26T21:37:50.006668+0000","last_active":"2023-12-26T21:37:50.008907+0000","last_peered":"2023-12-26T21:37:50.008907+0000","last_clean":"2023-12-26T21:37:50.008907+0000","last_became_active":"2023-12-26T21:37:50.006120+0000","last_became_peered":"2023-12-26T21:37:50.006120+0000","last_unstale":"2023-12-26T21:37:50.008907+0000","last_undegraded":"2023-12-26T21:37:50.008907+0000","last_fullsized":"2023-12-26T21:37:50.008907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:42:06.643972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.024437+0000","last_change":"2023-12-26T21:37:50.019055+0000","last_active":"2023-12-26T21:37:50.024437+0000","last_peered":"2023-12-26T21:37:50.024437+0000","last_clean":"2023-12-26T21:37:50.024437+0000","last_became_active":"2023-12-26T21:37:50.018692+0000","last_became_peered":"2023-12-26T21:37:50.018692+0000","last_unstale":"2023-12-26T21:37:50.024437+0000","last_undegraded":"2023-12-26T21:37:50.024437+0000","last_fullsized":"2023-12-26T21:37:50.024437+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:45:36.463616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021818+0000","last_change":"2023-12-26T21:37:50.016617+0000","last_active":"2023-12-26T21:37:50.021818+0000","last_peered":"2023-12-26T21:37:50.021818+0000","last_clean":"2023-12-26T21:37:50.021818+0000","last_became_active":"2023-12-26T21:37:50.016326+0000","last_became_peered":"2023-12-26T21:37:50.016326+0000","last_unstale":"2023-12-26T21:37:50.021818+0000","last_undegraded":"2023-12-26T21:37:50.021818+0000","last_fullsized":"2023-12-26T21:37:50.021818+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:51:26.623941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.027933+0000","last_change":"2023-12-26T21:37:50.027933+0000","last_active":"2023-12-26T21:37:50.027933+0000","last_peered":"2023-12-26T21:37:50.027933+0000","last_clean":"2023-12-26T21:37:50.027933+0000","last_became_active":"2023-12-26T21:37:50.027647+0000","last_became_peered":"2023-12-26T21:37:50.027647+0000","last_unstale":"2023-12-26T21:37:50.027933+0000","last_undegraded":"2023-12-26T21:37:50.027933+0000","last_fullsized":"2023-12-26T21:37:50.027933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:45:20.209845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.016563+0000","last_change":"2023-12-26T21:37:50.012252+0000","last_active":"2023-12-26T21:37:50.016563+0000","last_peered":"2023-12-26T21:37:50.016563+0000","last_clean":"2023-12-26T21:37:50.016563+0000","last_became_active":"2023-12-26T21:37:50.011863+0000","last_became_peered":"2023-12-26T21:37:50.011863+0000","last_unstale":"2023-12-26T21:37:50.016563+0000","last_undegraded":"2023-12-26T21:37:50.016563+0000","last_fullsized":"2023-12-26T21:37:50.016563+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:46:52.556912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.025094+0000","last_change":"2023-12-26T21:37:50.025094+0000","last_active":"2023-12-26T21:37:50.025094+0000","last_peered":"2023-12-26T21:37:50.025094+0000","last_clean":"2023-12-26T21:37:50.025094+0000","last_became_active":"2023-12-26T21:37:50.024889+0000","last_became_peered":"2023-12-26T21:37:50.024889+0000","last_unstale":"2023-12-26T21:37:50.025094+0000","last_undegraded":"2023-12-26T21:37:50.025094+0000","last_fullsized":"2023-12-26T21:37:50.025094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:18:48.013567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.030699+0000","last_change":"2023-12-26T21:37:50.030699+0000","last_active":"2023-12-26T21:37:50.030699+0000","last_peered":"2023-12-26T21:37:50.030699+0000","last_clean":"2023-12-26T21:37:50.030699+0000","last_became_active":"2023-12-26T21:37:50.030294+0000","last_became_peered":"2023-12-26T21:37:50.030294+0000","last_unstale":"2023-12-26T21:37:50.030699+0000","last_undegraded":"2023-12-26T21:37:50.030699+0000","last_fullsized":"2023-12-26T21:37:50.030699+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:25:12.521941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.023255+0000","last_change":"2023-12-26T21:37:50.023255+0000","last_active":"2023-12-26T21:37:50.023255+0000","last_peered":"2023-12-26T21:37:50.023255+0000","last_clean":"2023-12-26T21:37:50.023255+0000","last_became_active":"2023-12-26T21:37:50.021901+0000","last_became_peered":"2023-12-26T21:37:50.021901+0000","last_unstale":"2023-12-26T21:37:50.023255+0000","last_undegraded":"2023-12-26T21:37:50.023255+0000","last_fullsized":"2023-12-26T21:37:50.023255+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:32:16.784548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.028123+0000","last_change":"2023-12-26T21:37:50.017265+0000","last_active":"2023-12-26T21:37:50.028123+0000","last_peered":"2023-12-26T21:37:50.028123+0000","last_clean":"2023-12-26T21:37:50.028123+0000","last_became_active":"2023-12-26T21:37:50.017015+0000","last_became_peered":"2023-12-26T21:37:50.017015+0000","last_unstale":"2023-12-26T21:37:50.028123+0000","last_undegraded":"2023-12-26T21:37:50.028123+0000","last_fullsized":"2023-12-26T21:37:50.028123+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:34:15.322150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.025524+0000","last_change":"2023-12-26T21:37:50.025524+0000","last_active":"2023-12-26T21:37:50.025524+0000","last_peered":"2023-12-26T21:37:50.025524+0000","last_clean":"2023-12-26T21:37:50.025524+0000","last_became_active":"2023-12-26T21:37:50.025332+0000","last_became_peered":"2023-12-26T21:37:50.025332+0000","last_unstale":"2023-12-26T21:37:50.025524+0000","last_undegraded":"2023-12-26T21:37:50.025524+0000","last_fullsized":"2023-12-26T21:37:50.025524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:13.643734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022695+0000","last_change":"2023-12-26T21:37:50.019399+0000","last_active":"2023-12-26T21:37:50.022695+0000","last_peered":"2023-12-26T21:37:50.022695+0000","last_clean":"2023-12-26T21:37:50.022695+0000","last_became_active":"2023-12-26T21:37:50.018744+0000","last_became_peered":"2023-12-26T21:37:50.018744+0000","last_unstale":"2023-12-26T21:37:50.022695+0000","last_undegraded":"2023-12-26T21:37:50.022695+0000","last_fullsized":"2023-12-26T21:37:50.022695+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:44:28.087202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021842+0000","last_change":"2023-12-26T21:37:50.016056+0000","last_active":"2023-12-26T21:37:50.021842+0000","last_peered":"2023-12-26T21:37:50.021842+0000","last_clean":"2023-12-26T21:37:50.021842+0000","last_became_active":"2023-12-26T21:37:50.015774+0000","last_became_peered":"2023-12-26T21:37:50.015774+0000","last_unstale":"2023-12-26T21:37:50.021842+0000","last_undegraded":"2023-12-26T21:37:50.021842+0000","last_fullsized":"2023-12-26T21:37:50.021842+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:38:16.973436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.020540+0000","last_change":"2023-12-26T21:37:50.020540+0000","last_active":"2023-12-26T21:37:50.020540+0000","last_peered":"2023-12-26T21:37:50.020540+0000","last_clean":"2023-12-26T21:37:50.020540+0000","last_became_active":"2023-12-26T21:37:50.020300+0000","last_became_peered":"2023-12-26T21:37:50.020300+0000","last_unstale":"2023-12-26T21:37:50.020540+0000","last_undegraded":"2023-12-26T21:37:50.020540+0000","last_fullsized":"2023-12-26T21:37:50.020540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:49.041763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.019426+0000","last_change":"2023-12-26T21:37:50.013636+0000","last_active":"2023-12-26T21:37:50.019426+0000","last_peered":"2023-12-26T21:37:50.019426+0000","last_clean":"2023-12-26T21:37:50.019426+0000","last_became_active":"2023-12-26T21:37:50.013369+0000","last_became_peered":"2023-12-26T21:37:50.013369+0000","last_unstale":"2023-12-26T21:37:50.019426+0000","last_undegraded":"2023-12-26T21:37:50.019426+0000","last_fullsized":"2023-12-26T21:37:50.019426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:37.762931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.027104+0000","last_change":"2023-12-26T21:37:50.021148+0000","last_active":"2023-12-26T21:37:50.027104+0000","last_peered":"2023-12-26T21:37:50.027104+0000","last_clean":"2023-12-26T21:37:50.027104+0000","last_became_active":"2023-12-26T21:37:50.020878+0000","last_became_peered":"2023-12-26T21:37:50.020878+0000","last_unstale":"2023-12-26T21:37:50.027104+0000","last_undegraded":"2023-12-26T21:37:50.027104+0000","last_fullsized":"2023-12-26T21:37:50.027104+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:19:30.444831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060466+0000","last_change":"2023-12-26T21:37:49.059929+0000","last_active":"2023-12-26T21:37:50.060466+0000","last_peered":"2023-12-26T21:37:50.060466+0000","last_clean":"2023-12-26T21:37:50.060466+0000","last_became_active":"2023-12-26T21:37:49.059601+0000","last_became_peered":"2023-12-26T21:37:49.059601+0000","last_unstale":"2023-12-26T21:37:50.060466+0000","last_undegraded":"2023-12-26T21:37:50.060466+0000","last_fullsized":"2023-12-26T21:37:50.060466+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:25:37.176835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.023473+0000","last_change":"2023-12-26T21:37:50.023473+0000","last_active":"2023-12-26T21:37:50.023473+0000","last_peered":"2023-12-26T21:37:50.023473+0000","last_clean":"2023-12-26T21:37:50.023473+0000","last_became_active":"2023-12-26T21:37:50.022916+0000","last_became_peered":"2023-12-26T21:37:50.022916+0000","last_unstale":"2023-12-26T21:37:50.023473+0000","last_undegraded":"2023-12-26T21:37:50.023473+0000","last_fullsized":"2023-12-26T21:37:50.023473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:20:03.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.011219+0000","last_change":"2023-12-26T21:37:49.051127+0000","last_active":"2023-12-26T21:37:50.011219+0000","last_peered":"2023-12-26T21:37:50.011219+0000","last_clean":"2023-12-26T21:37:50.011219+0000","last_became_active":"2023-12-26T21:37:49.050808+0000","last_became_peered":"2023-12-26T21:37:49.050808+0000","last_unstale":"2023-12-26T21:37:50.011219+0000","last_undegraded":"2023-12-26T21:37:50.011219+0000","last_fullsized":"2023-12-26T21:37:50.011219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:37:57.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.027906+0000","last_change":"2023-12-26T21:37:50.027906+0000","last_active":"2023-12-26T21:37:50.027906+0000","last_peered":"2023-12-26T21:37:50.027906+0000","last_clean":"2023-12-26T21:37:50.027906+0000","last_became_active":"2023-12-26T21:37:50.027649+0000","last_became_peered":"2023-12-26T21:37:50.027649+0000","last_unstale":"2023-12-26T21:37:50.027906+0000","last_undegraded":"2023-12-26T21:37:50.027906+0000","last_fullsized":"2023-12-26T21:37:50.027906+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:55:22.146030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060696+0000","last_change":"2023-12-26T21:37:49.057504+0000","last_active":"2023-12-26T21:37:50.060696+0000","last_peered":"2023-12-26T21:37:50.060696+0000","last_clean":"2023-12-26T21:37:50.060696+0000","last_became_active":"2023-12-26T21:37:49.057174+0000","last_became_peered":"2023-12-26T21:37:49.057174+0000","last_unstale":"2023-12-26T21:37:50.060696+0000","last_undegraded":"2023-12-26T21:37:50.060696+0000","last_fullsized":"2023-12-26T21:37:50.060696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:25:35.067327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.032108+0000","last_change":"2023-12-26T21:37:50.032108+0000","last_active":"2023-12-26T21:37:50.032108+0000","last_peered":"2023-12-26T21:37:50.032108+0000","last_clean":"2023-12-26T21:37:50.032108+0000","last_became_active":"2023-12-26T21:37:50.031800+0000","last_became_peered":"2023-12-26T21:37:50.031800+0000","last_unstale":"2023-12-26T21:37:50.032108+0000","last_undegraded":"2023-12-26T21:37:50.032108+0000","last_fullsized":"2023-12-26T21:37:50.032108+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:13.643734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060308+0000","last_change":"2023-12-26T21:37:49.059658+0000","last_active":"2023-12-26T21:37:50.060308+0000","last_peered":"2023-12-26T21:37:50.060308+0000","last_clean":"2023-12-26T21:37:50.060308+0000","last_became_active":"2023-12-26T21:37:49.059468+0000","last_became_peered":"2023-12-26T21:37:49.059468+0000","last_unstale":"2023-12-26T21:37:50.060308+0000","last_undegraded":"2023-12-26T21:37:50.060308+0000","last_fullsized":"2023-12-26T21:37:50.060308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:53:28.779103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015238+0000","last_change":"2023-12-26T21:37:50.013675+0000","last_active":"2023-12-26T21:37:50.015238+0000","last_peered":"2023-12-26T21:37:50.015238+0000","last_clean":"2023-12-26T21:37:50.015238+0000","last_became_active":"2023-12-26T21:37:50.013264+0000","last_became_peered":"2023-12-26T21:37:50.013264+0000","last_unstale":"2023-12-26T21:37:50.015238+0000","last_undegraded":"2023-12-26T21:37:50.015238+0000","last_fullsized":"2023-12-26T21:37:50.015238+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:01.152199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.008106+0000","last_change":"2023-12-26T21:37:49.065084+0000","last_active":"2023-12-26T21:37:50.008106+0000","last_peered":"2023-12-26T21:37:50.008106+0000","last_clean":"2023-12-26T21:37:50.008106+0000","last_became_active":"2023-12-26T21:37:49.064768+0000","last_became_peered":"2023-12-26T21:37:49.064768+0000","last_unstale":"2023-12-26T21:37:50.008106+0000","last_undegraded":"2023-12-26T21:37:50.008106+0000","last_fullsized":"2023-12-26T21:37:50.008106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:42:05.636810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.020162+0000","last_change":"2023-12-26T21:37:50.020162+0000","last_active":"2023-12-26T21:37:50.020162+0000","last_peered":"2023-12-26T21:37:50.020162+0000","last_clean":"2023-12-26T21:37:50.020162+0000","last_became_active":"2023-12-26T21:37:50.019885+0000","last_became_peered":"2023-12-26T21:37:50.019885+0000","last_unstale":"2023-12-26T21:37:50.020162+0000","last_undegraded":"2023-12-26T21:37:50.020162+0000","last_fullsized":"2023-12-26T21:37:50.020162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:31:29.275066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.008730+0000","last_change":"2023-12-26T21:37:49.057844+0000","last_active":"2023-12-26T21:37:50.008730+0000","last_peered":"2023-12-26T21:37:50.008730+0000","last_clean":"2023-12-26T21:37:50.008730+0000","last_became_active":"2023-12-26T21:37:49.057459+0000","last_became_peered":"2023-12-26T21:37:49.057459+0000","last_unstale":"2023-12-26T21:37:50.008730+0000","last_undegraded":"2023-12-26T21:37:50.008730+0000","last_fullsized":"2023-12-26T21:37:50.008730+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:37.235627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.012234+0000","last_change":"2023-12-26T21:37:50.010055+0000","last_active":"2023-12-26T21:37:50.012234+0000","last_peered":"2023-12-26T21:37:50.012234+0000","last_clean":"2023-12-26T21:37:50.012234+0000","last_became_active":"2023-12-26T21:37:50.009783+0000","last_became_peered":"2023-12-26T21:37:50.009783+0000","last_unstale":"2023-12-26T21:37:50.012234+0000","last_undegraded":"2023-12-26T21:37:50.012234+0000","last_fullsized":"2023-12-26T21:37:50.012234+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:58:41.674361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.000500+0000","last_change":"2023-12-26T21:37:49.059217+0000","last_active":"2023-12-26T21:37:50.000500+0000","last_peered":"2023-12-26T21:37:50.000500+0000","last_clean":"2023-12-26T21:37:50.000500+0000","last_became_active":"2023-12-26T21:37:49.056059+0000","last_became_peered":"2023-12-26T21:37:49.056059+0000","last_unstale":"2023-12-26T21:37:50.000500+0000","last_undegraded":"2023-12-26T21:37:50.000500+0000","last_fullsized":"2023-12-26T21:37:50.000500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:27:37.134010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.030718+0000","last_change":"2023-12-26T21:37:50.030718+0000","last_active":"2023-12-26T21:37:50.030718+0000","last_peered":"2023-12-26T21:37:50.030718+0000","last_clean":"2023-12-26T21:37:50.030718+0000","last_became_active":"2023-12-26T21:37:50.030134+0000","last_became_peered":"2023-12-26T21:37:50.030134+0000","last_unstale":"2023-12-26T21:37:50.030718+0000","last_undegraded":"2023-12-26T21:37:50.030718+0000","last_fullsized":"2023-12-26T21:37:50.030718+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:53:00.606274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.065971+0000","last_change":"2023-12-26T21:37:49.064610+0000","last_active":"2023-12-26T21:37:49.065971+0000","last_peered":"2023-12-26T21:37:49.065971+0000","last_clean":"2023-12-26T21:37:49.065971+0000","last_became_active":"2023-12-26T21:37:49.064374+0000","last_became_peered":"2023-12-26T21:37:49.064374+0000","last_unstale":"2023-12-26T21:37:49.065971+0000","last_undegraded":"2023-12-26T21:37:49.065971+0000","last_fullsized":"2023-12-26T21:37:49.065971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:37:57.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021884+0000","last_change":"2023-12-26T21:37:50.019779+0000","last_active":"2023-12-26T21:37:50.021884+0000","last_peered":"2023-12-26T21:37:50.021884+0000","last_clean":"2023-12-26T21:37:50.021884+0000","last_became_active":"2023-12-26T21:37:50.019495+0000","last_became_peered":"2023-12-26T21:37:50.019495+0000","last_unstale":"2023-12-26T21:37:50.021884+0000","last_undegraded":"2023-12-26T21:37:50.021884+0000","last_fullsized":"2023-12-26T21:37:50.021884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:37:29.638859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.006124+0000","last_change":"2023-12-26T21:37:49.057640+0000","last_active":"2023-12-26T21:37:50.006124+0000","last_peered":"2023-12-26T21:37:50.006124+0000","last_clean":"2023-12-26T21:37:50.006124+0000","last_became_active":"2023-12-26T21:37:49.057342+0000","last_became_peered":"2023-12-26T21:37:49.057342+0000","last_unstale":"2023-12-26T21:37:50.006124+0000","last_undegraded":"2023-12-26T21:37:50.006124+0000","last_fullsized":"2023-12-26T21:37:50.006124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:56:21.776351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.017719+0000","last_change":"2023-12-26T21:37:50.017719+0000","last_active":"2023-12-26T21:37:50.017719+0000","last_peered":"2023-12-26T21:37:50.017719+0000","last_clean":"2023-12-26T21:37:50.017719+0000","last_became_active":"2023-12-26T21:37:50.017423+0000","last_became_peered":"2023-12-26T21:37:50.017423+0000","last_unstale":"2023-12-26T21:37:50.017719+0000","last_undegraded":"2023-12-26T21:37:50.017719+0000","last_fullsized":"2023-12-26T21:37:50.017719+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:55:22.146030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.007001+0000","last_change":"2023-12-26T21:37:49.064273+0000","last_active":"2023-12-26T21:37:50.007001+0000","last_peered":"2023-12-26T21:37:50.007001+0000","last_clean":"2023-12-26T21:37:50.007001+0000","last_became_active":"2023-12-26T21:37:49.058591+0000","last_became_peered":"2023-12-26T21:37:49.058591+0000","last_unstale":"2023-12-26T21:37:50.007001+0000","last_undegraded":"2023-12-26T21:37:50.007001+0000","last_fullsized":"2023-12-26T21:37:50.007001+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:19:29.437669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.032142+0000","last_change":"2023-12-26T21:37:50.032142+0000","last_active":"2023-12-26T21:37:50.032142+0000","last_peered":"2023-12-26T21:37:50.032142+0000","last_clean":"2023-12-26T21:37:50.032142+0000","last_became_active":"2023-12-26T21:37:50.031369+0000","last_became_peered":"2023-12-26T21:37:50.031369+0000","last_unstale":"2023-12-26T21:37:50.032142+0000","last_undegraded":"2023-12-26T21:37:50.032142+0000","last_fullsized":"2023-12-26T21:37:50.032142+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:45.556024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.055312+0000","last_change":"2023-12-26T21:37:49.058851+0000","last_active":"2023-12-26T21:37:50.055312+0000","last_peered":"2023-12-26T21:37:50.055312+0000","last_clean":"2023-12-26T21:37:50.055312+0000","last_became_active":"2023-12-26T21:37:49.058558+0000","last_became_peered":"2023-12-26T21:37:49.058558+0000","last_unstale":"2023-12-26T21:37:50.055312+0000","last_undegraded":"2023-12-26T21:37:50.055312+0000","last_fullsized":"2023-12-26T21:37:50.055312+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:30.968847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029597+0000","last_change":"2023-12-26T21:37:50.028023+0000","last_active":"2023-12-26T21:37:50.029597+0000","last_peered":"2023-12-26T21:37:50.029597+0000","last_clean":"2023-12-26T21:37:50.029597+0000","last_became_active":"2023-12-26T21:37:50.027709+0000","last_became_peered":"2023-12-26T21:37:50.027709+0000","last_unstale":"2023-12-26T21:37:50.029597+0000","last_undegraded":"2023-12-26T21:37:50.029597+0000","last_fullsized":"2023-12-26T21:37:50.029597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:37.613373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.062877+0000","last_change":"2023-12-26T21:37:49.060492+0000","last_active":"2023-12-26T21:37:50.062877+0000","last_peered":"2023-12-26T21:37:50.062877+0000","last_clean":"2023-12-26T21:37:50.062877+0000","last_became_active":"2023-12-26T21:37:49.059949+0000","last_became_peered":"2023-12-26T21:37:49.059949+0000","last_unstale":"2023-12-26T21:37:50.062877+0000","last_undegraded":"2023-12-26T21:37:50.062877+0000","last_fullsized":"2023-12-26T21:37:50.062877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:21:06.881704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021921+0000","last_change":"2023-12-26T21:37:50.016184+0000","last_active":"2023-12-26T21:37:50.021921+0000","last_peered":"2023-12-26T21:37:50.021921+0000","last_clean":"2023-12-26T21:37:50.021921+0000","last_became_active":"2023-12-26T21:37:50.015896+0000","last_became_peered":"2023-12-26T21:37:50.015896+0000","last_unstale":"2023-12-26T21:37:50.021921+0000","last_undegraded":"2023-12-26T21:37:50.021921+0000","last_fullsized":"2023-12-26T21:37:50.021921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:36.781362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.057829+0000","last_change":"2023-12-26T21:37:49.056945+0000","last_active":"2023-12-26T21:37:50.057829+0000","last_peered":"2023-12-26T21:37:50.057829+0000","last_clean":"2023-12-26T21:37:50.057829+0000","last_became_active":"2023-12-26T21:37:49.056678+0000","last_became_peered":"2023-12-26T21:37:49.056678+0000","last_unstale":"2023-12-26T21:37:50.057829+0000","last_undegraded":"2023-12-26T21:37:50.057829+0000","last_fullsized":"2023-12-26T21:37:50.057829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:20.144180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029617+0000","last_change":"2023-12-26T21:37:50.029617+0000","last_active":"2023-12-26T21:37:50.029617+0000","last_peered":"2023-12-26T21:37:50.029617+0000","last_clean":"2023-12-26T21:37:50.029617+0000","last_became_active":"2023-12-26T21:37:50.029429+0000","last_became_peered":"2023-12-26T21:37:50.029429+0000","last_unstale":"2023-12-26T21:37:50.029617+0000","last_undegraded":"2023-12-26T21:37:50.029617+0000","last_fullsized":"2023-12-26T21:37:50.029617+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:01.590388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.059472+0000","last_change":"2023-12-26T21:37:49.055122+0000","last_active":"2023-12-26T21:37:49.059472+0000","last_peered":"2023-12-26T21:37:49.059472+0000","last_clean":"2023-12-26T21:37:49.059472+0000","last_became_active":"2023-12-26T21:37:49.054791+0000","last_became_peered":"2023-12-26T21:37:49.054791+0000","last_unstale":"2023-12-26T21:37:49.059472+0000","last_undegraded":"2023-12-26T21:37:49.059472+0000","last_fullsized":"2023-12-26T21:37:49.059472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:49:55.783980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.013164+0000","last_change":"2023-12-26T21:37:50.013164+0000","last_active":"2023-12-26T21:37:50.013164+0000","last_peered":"2023-12-26T21:37:50.013164+0000","last_clean":"2023-12-26T21:37:50.013164+0000","last_became_active":"2023-12-26T21:37:50.012793+0000","last_became_peered":"2023-12-26T21:37:50.012793+0000","last_unstale":"2023-12-26T21:37:50.013164+0000","last_undegraded":"2023-12-26T21:37:50.013164+0000","last_fullsized":"2023-12-26T21:37:50.013164+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:12:24.533764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.064861+0000","last_change":"2023-12-26T21:37:49.060744+0000","last_active":"2023-12-26T21:37:49.064861+0000","last_peered":"2023-12-26T21:37:49.064861+0000","last_clean":"2023-12-26T21:37:49.064861+0000","last_became_active":"2023-12-26T21:37:49.060388+0000","last_became_peered":"2023-12-26T21:37:49.060388+0000","last_unstale":"2023-12-26T21:37:49.064861+0000","last_undegraded":"2023-12-26T21:37:49.064861+0000","last_fullsized":"2023-12-26T21:37:49.064861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:37.235627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022415+0000","last_change":"2023-12-26T21:37:50.017245+0000","last_active":"2023-12-26T21:37:50.022415+0000","last_peered":"2023-12-26T21:37:50.022415+0000","last_clean":"2023-12-26T21:37:50.022415+0000","last_became_active":"2023-12-26T21:37:50.016887+0000","last_became_peered":"2023-12-26T21:37:50.016887+0000","last_unstale":"2023-12-26T21:37:50.022415+0000","last_undegraded":"2023-12-26T21:37:50.022415+0000","last_fullsized":"2023-12-26T21:37:50.022415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:12:20.128839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060866+0000","last_change":"2023-12-26T21:37:49.044678+0000","last_active":"2023-12-26T21:37:50.060866+0000","last_peered":"2023-12-26T21:37:50.060866+0000","last_clean":"2023-12-26T21:37:50.060866+0000","last_became_active":"2023-12-26T21:37:49.044404+0000","last_became_peered":"2023-12-26T21:37:49.044404+0000","last_unstale":"2023-12-26T21:37:50.060866+0000","last_undegraded":"2023-12-26T21:37:50.060866+0000","last_fullsized":"2023-12-26T21:37:50.060866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:10.777671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029541+0000","last_change":"2023-12-26T21:37:50.025483+0000","last_active":"2023-12-26T21:37:50.029541+0000","last_peered":"2023-12-26T21:37:50.029541+0000","last_clean":"2023-12-26T21:37:50.029541+0000","last_became_active":"2023-12-26T21:37:50.025172+0000","last_became_peered":"2023-12-26T21:37:50.025172+0000","last_unstale":"2023-12-26T21:37:50.029541+0000","last_undegraded":"2023-12-26T21:37:50.029541+0000","last_fullsized":"2023-12-26T21:37:50.029541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:53:46.215076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.059940+0000","last_change":"2023-12-26T21:37:49.057012+0000","last_active":"2023-12-26T21:37:50.059940+0000","last_peered":"2023-12-26T21:37:50.059940+0000","last_clean":"2023-12-26T21:37:50.059940+0000","last_became_active":"2023-12-26T21:37:49.056691+0000","last_became_peered":"2023-12-26T21:37:49.056691+0000","last_unstale":"2023-12-26T21:37:50.059940+0000","last_undegraded":"2023-12-26T21:37:50.059940+0000","last_fullsized":"2023-12-26T21:37:50.059940+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:20:15.766845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.009538+0000","last_change":"2023-12-26T21:37:50.005001+0000","last_active":"2023-12-26T21:37:50.009538+0000","last_peered":"2023-12-26T21:37:50.009538+0000","last_clean":"2023-12-26T21:37:50.009538+0000","last_became_active":"2023-12-26T21:37:50.004703+0000","last_became_peered":"2023-12-26T21:37:50.004703+0000","last_unstale":"2023-12-26T21:37:50.009538+0000","last_undegraded":"2023-12-26T21:37:50.009538+0000","last_fullsized":"2023-12-26T21:37:50.009538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:34:04.184026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.055641+0000","last_change":"2023-12-26T21:37:49.060695+0000","last_active":"2023-12-26T21:37:50.055641+0000","last_peered":"2023-12-26T21:37:50.055641+0000","last_clean":"2023-12-26T21:37:50.055641+0000","last_became_active":"2023-12-26T21:37:49.058055+0000","last_became_peered":"2023-12-26T21:37:49.058055+0000","last_unstale":"2023-12-26T21:37:50.055641+0000","last_undegraded":"2023-12-26T21:37:50.055641+0000","last_fullsized":"2023-12-26T21:37:50.055641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:32:11.434034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.020217+0000","last_change":"2023-12-26T21:37:50.010547+0000","last_active":"2023-12-26T21:37:50.020217+0000","last_peered":"2023-12-26T21:37:50.020217+0000","last_clean":"2023-12-26T21:37:50.020217+0000","last_became_active":"2023-12-26T21:37:50.010248+0000","last_became_peered":"2023-12-26T21:37:50.010248+0000","last_unstale":"2023-12-26T21:37:50.020217+0000","last_undegraded":"2023-12-26T21:37:50.020217+0000","last_fullsized":"2023-12-26T21:37:50.020217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:08:01.371579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.054817+0000","last_change":"2023-12-26T21:37:49.057397+0000","last_active":"2023-12-26T21:37:50.054817+0000","last_peered":"2023-12-26T21:37:50.054817+0000","last_clean":"2023-12-26T21:37:50.054817+0000","last_became_active":"2023-12-26T21:37:49.055492+0000","last_became_peered":"2023-12-26T21:37:49.055492+0000","last_unstale":"2023-12-26T21:37:50.054817+0000","last_undegraded":"2023-12-26T21:37:50.054817+0000","last_fullsized":"2023-12-26T21:37:50.054817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:25.672661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015786+0000","last_change":"2023-12-26T21:37:50.011523+0000","last_active":"2023-12-26T21:37:50.015786+0000","last_peered":"2023-12-26T21:37:50.015786+0000","last_clean":"2023-12-26T21:37:50.015786+0000","last_became_active":"2023-12-26T21:37:50.011228+0000","last_became_peered":"2023-12-26T21:37:50.011228+0000","last_unstale":"2023-12-26T21:37:50.015786+0000","last_undegraded":"2023-12-26T21:37:50.015786+0000","last_fullsized":"2023-12-26T21:37:50.015786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:49:56.791142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.001367+0000","last_change":"2023-12-26T21:37:49.056890+0000","last_active":"2023-12-26T21:37:50.001367+0000","last_peered":"2023-12-26T21:37:50.001367+0000","last_clean":"2023-12-26T21:37:50.001367+0000","last_became_active":"2023-12-26T21:37:49.056617+0000","last_became_peered":"2023-12-26T21:37:49.056617+0000","last_unstale":"2023-12-26T21:37:50.001367+0000","last_undegraded":"2023-12-26T21:37:50.001367+0000","last_fullsized":"2023-12-26T21:37:50.001367+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:35.774200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.030122+0000","last_change":"2023-12-26T21:37:50.028042+0000","last_active":"2023-12-26T21:37:50.030122+0000","last_peered":"2023-12-26T21:37:50.030122+0000","last_clean":"2023-12-26T21:37:50.030122+0000","last_became_active":"2023-12-26T21:37:50.027724+0000","last_became_peered":"2023-12-26T21:37:50.027724+0000","last_unstale":"2023-12-26T21:37:50.030122+0000","last_undegraded":"2023-12-26T21:37:50.030122+0000","last_fullsized":"2023-12-26T21:37:50.030122+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:56:22.944966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060611+0000","last_change":"2023-12-26T21:37:49.061864+0000","last_active":"2023-12-26T21:37:50.060611+0000","last_peered":"2023-12-26T21:37:50.060611+0000","last_clean":"2023-12-26T21:37:50.060611+0000","last_became_active":"2023-12-26T21:37:49.061569+0000","last_became_peered":"2023-12-26T21:37:49.061569+0000","last_unstale":"2023-12-26T21:37:50.060611+0000","last_undegraded":"2023-12-26T21:37:50.060611+0000","last_fullsized":"2023-12-26T21:37:50.060611+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:42:46.794558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021019+0000","last_change":"2023-12-26T21:37:50.015748+0000","last_active":"2023-12-26T21:37:50.021019+0000","last_peered":"2023-12-26T21:37:50.021019+0000","last_clean":"2023-12-26T21:37:50.021019+0000","last_became_active":"2023-12-26T21:37:50.015470+0000","last_became_peered":"2023-12-26T21:37:50.015470+0000","last_unstale":"2023-12-26T21:37:50.021019+0000","last_undegraded":"2023-12-26T21:37:50.021019+0000","last_fullsized":"2023-12-26T21:37:50.021019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:33.796968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.063588+0000","last_change":"2023-12-26T21:37:49.055204+0000","last_active":"2023-12-26T21:37:49.063588+0000","last_peered":"2023-12-26T21:37:49.063588+0000","last_clean":"2023-12-26T21:37:49.063588+0000","last_became_active":"2023-12-26T21:37:49.054942+0000","last_became_peered":"2023-12-26T21:37:49.054942+0000","last_unstale":"2023-12-26T21:37:49.063588+0000","last_undegraded":"2023-12-26T21:37:49.063588+0000","last_fullsized":"2023-12-26T21:37:49.063588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:42.166494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029958+0000","last_change":"2023-12-26T21:37:50.026788+0000","last_active":"2023-12-26T21:37:50.029958+0000","last_peered":"2023-12-26T21:37:50.029958+0000","last_clean":"2023-12-26T21:37:50.029958+0000","last_became_active":"2023-12-26T21:37:50.026521+0000","last_became_peered":"2023-12-26T21:37:50.026521+0000","last_unstale":"2023-12-26T21:37:50.029958+0000","last_undegraded":"2023-12-26T21:37:50.029958+0000","last_fullsized":"2023-12-26T21:37:50.029958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:20:26.434022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.002565+0000","last_change":"2023-12-26T21:37:49.062160+0000","last_active":"2023-12-26T21:37:50.002565+0000","last_peered":"2023-12-26T21:37:50.002565+0000","last_clean":"2023-12-26T21:37:50.002565+0000","last_became_active":"2023-12-26T21:37:49.061823+0000","last_became_peered":"2023-12-26T21:37:49.061823+0000","last_unstale":"2023-12-26T21:37:50.002565+0000","last_undegraded":"2023-12-26T21:37:50.002565+0000","last_fullsized":"2023-12-26T21:37:50.002565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:32:11.550625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.008639+0000","last_change":"2023-12-26T21:37:50.005148+0000","last_active":"2023-12-26T21:37:50.008639+0000","last_peered":"2023-12-26T21:37:50.008639+0000","last_clean":"2023-12-26T21:37:50.008639+0000","last_became_active":"2023-12-26T21:37:50.004668+0000","last_became_peered":"2023-12-26T21:37:50.004668+0000","last_unstale":"2023-12-26T21:37:50.008639+0000","last_undegraded":"2023-12-26T21:37:50.008639+0000","last_fullsized":"2023-12-26T21:37:50.008639+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:34:20.086714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.006407+0000","last_change":"2023-12-26T21:37:49.052833+0000","last_active":"2023-12-26T21:37:50.006407+0000","last_peered":"2023-12-26T21:37:50.006407+0000","last_clean":"2023-12-26T21:37:50.006407+0000","last_became_active":"2023-12-26T21:37:49.050159+0000","last_became_peered":"2023-12-26T21:37:49.050159+0000","last_unstale":"2023-12-26T21:37:50.006407+0000","last_undegraded":"2023-12-26T21:37:50.006407+0000","last_fullsized":"2023-12-26T21:37:50.006407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:41:44.910782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015036+0000","last_change":"2023-12-26T21:37:50.011396+0000","last_active":"2023-12-26T21:37:50.015036+0000","last_peered":"2023-12-26T21:37:50.015036+0000","last_clean":"2023-12-26T21:37:50.015036+0000","last_became_active":"2023-12-26T21:37:50.011065+0000","last_became_peered":"2023-12-26T21:37:50.011065+0000","last_unstale":"2023-12-26T21:37:50.015036+0000","last_undegraded":"2023-12-26T21:37:50.015036+0000","last_fullsized":"2023-12-26T21:37:50.015036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:38.242789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.055549+0000","last_change":"2023-12-26T21:37:49.060258+0000","last_active":"2023-12-26T21:37:50.055549+0000","last_peered":"2023-12-26T21:37:50.055549+0000","last_clean":"2023-12-26T21:37:50.055549+0000","last_became_active":"2023-12-26T21:37:49.059932+0000","last_became_peered":"2023-12-26T21:37:49.059932+0000","last_unstale":"2023-12-26T21:37:50.055549+0000","last_undegraded":"2023-12-26T21:37:50.055549+0000","last_fullsized":"2023-12-26T21:37:50.055549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:43.540873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022445+0000","last_change":"2023-12-26T21:37:50.016905+0000","last_active":"2023-12-26T21:37:50.022445+0000","last_peered":"2023-12-26T21:37:50.022445+0000","last_clean":"2023-12-26T21:37:50.022445+0000","last_became_active":"2023-12-26T21:37:50.016618+0000","last_became_peered":"2023-12-26T21:37:50.016618+0000","last_unstale":"2023-12-26T21:37:50.022445+0000","last_undegraded":"2023-12-26T21:37:50.022445+0000","last_fullsized":"2023-12-26T21:37:50.022445+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:29:35.827917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.007099+0000","last_change":"2023-12-26T21:37:49.060728+0000","last_active":"2023-12-26T21:37:50.007099+0000","last_peered":"2023-12-26T21:37:50.007099+0000","last_clean":"2023-12-26T21:37:50.007099+0000","last_became_active":"2023-12-26T21:37:49.058273+0000","last_became_peered":"2023-12-26T21:37:49.058273+0000","last_unstale":"2023-12-26T21:37:50.007099+0000","last_undegraded":"2023-12-26T21:37:50.007099+0000","last_fullsized":"2023-12-26T21:37:50.007099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:35.774200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.028678+0000","last_change":"2023-12-26T21:37:50.021450+0000","last_active":"2023-12-26T21:37:50.028678+0000","last_peered":"2023-12-26T21:37:50.028678+0000","last_clean":"2023-12-26T21:37:50.028678+0000","last_became_active":"2023-12-26T21:37:50.021194+0000","last_became_peered":"2023-12-26T21:37:50.021194+0000","last_unstale":"2023-12-26T21:37:50.028678+0000","last_undegraded":"2023-12-26T21:37:50.028678+0000","last_fullsized":"2023-12-26T21:37:50.028678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:43.173656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.013888+0000","last_change":"2023-12-26T21:37:49.058279+0000","last_active":"2023-12-26T21:37:50.013888+0000","last_peered":"2023-12-26T21:37:50.013888+0000","last_clean":"2023-12-26T21:37:50.013888+0000","last_became_active":"2023-12-26T21:37:49.057964+0000","last_became_peered":"2023-12-26T21:37:49.057964+0000","last_unstale":"2023-12-26T21:37:50.013888+0000","last_undegraded":"2023-12-26T21:37:50.013888+0000","last_fullsized":"2023-12-26T21:37:50.013888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:56:51.912847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.018954+0000","last_change":"2023-12-26T21:37:50.011822+0000","last_active":"2023-12-26T21:37:50.018954+0000","last_peered":"2023-12-26T21:37:50.018954+0000","last_clean":"2023-12-26T21:37:50.018954+0000","last_became_active":"2023-12-26T21:37:50.011581+0000","last_became_peered":"2023-12-26T21:37:50.011581+0000","last_unstale":"2023-12-26T21:37:50.018954+0000","last_undegraded":"2023-12-26T21:37:50.018954+0000","last_fullsized":"2023-12-26T21:37:50.018954+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:41:45.917944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.012679+0000","last_change":"2023-12-26T21:37:49.062314+0000","last_active":"2023-12-26T21:37:50.012679+0000","last_peered":"2023-12-26T21:37:50.012679+0000","last_clean":"2023-12-26T21:37:50.012679+0000","last_became_active":"2023-12-26T21:37:49.062067+0000","last_became_peered":"2023-12-26T21:37:49.062067+0000","last_unstale":"2023-12-26T21:37:50.012679+0000","last_undegraded":"2023-12-26T21:37:50.012679+0000","last_fullsized":"2023-12-26T21:37:50.012679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:34:03.176864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029562+0000","last_change":"2023-12-26T21:37:50.029562+0000","last_active":"2023-12-26T21:37:50.029562+0000","last_peered":"2023-12-26T21:37:50.029562+0000","last_clean":"2023-12-26T21:37:50.029562+0000","last_became_active":"2023-12-26T21:37:50.029381+0000","last_became_peered":"2023-12-26T21:37:50.029381+0000","last_unstale":"2023-12-26T21:37:50.029562+0000","last_undegraded":"2023-12-26T21:37:50.029562+0000","last_fullsized":"2023-12-26T21:37:50.029562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:20:03.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.012601+0000","last_change":"2023-12-26T21:37:49.054282+0000","last_active":"2023-12-26T21:37:50.012601+0000","last_peered":"2023-12-26T21:37:50.012601+0000","last_clean":"2023-12-26T21:37:50.012601+0000","last_became_active":"2023-12-26T21:37:49.053589+0000","last_became_peered":"2023-12-26T21:37:49.053589+0000","last_unstale":"2023-12-26T21:37:50.012601+0000","last_undegraded":"2023-12-26T21:37:50.012601+0000","last_fullsized":"2023-12-26T21:37:50.012601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:25:09.234601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.026982+0000","last_change":"2023-12-26T21:37:50.026982+0000","last_active":"2023-12-26T21:37:50.026982+0000","last_peered":"2023-12-26T21:37:50.026982+0000","last_clean":"2023-12-26T21:37:50.026982+0000","last_became_active":"2023-12-26T21:37:50.026522+0000","last_became_peered":"2023-12-26T21:37:50.026522+0000","last_unstale":"2023-12-26T21:37:50.026982+0000","last_undegraded":"2023-12-26T21:37:50.026982+0000","last_fullsized":"2023-12-26T21:37:50.026982+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:55:52.330005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.064248+0000","last_change":"2023-12-26T21:37:49.057261+0000","last_active":"2023-12-26T21:37:49.064248+0000","last_peered":"2023-12-26T21:37:49.064248+0000","last_clean":"2023-12-26T21:37:49.064248+0000","last_became_active":"2023-12-26T21:37:49.054556+0000","last_became_peered":"2023-12-26T21:37:49.054556+0000","last_unstale":"2023-12-26T21:37:49.064248+0000","last_undegraded":"2023-12-26T21:37:49.064248+0000","last_fullsized":"2023-12-26T21:37:49.064248+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:44:12.532434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022396+0000","last_change":"2023-12-26T21:37:50.015679+0000","last_active":"2023-12-26T21:37:50.022396+0000","last_peered":"2023-12-26T21:37:50.022396+0000","last_clean":"2023-12-26T21:37:50.022396+0000","last_became_active":"2023-12-26T21:37:50.015330+0000","last_became_peered":"2023-12-26T21:37:50.015330+0000","last_unstale":"2023-12-26T21:37:50.022396+0000","last_undegraded":"2023-12-26T21:37:50.022396+0000","last_fullsized":"2023-12-26T21:37:50.022396+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:48:46.804827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.054410+0000","last_change":"2023-12-26T21:37:49.051331+0000","last_active":"2023-12-26T21:37:49.054410+0000","last_peered":"2023-12-26T21:37:49.054410+0000","last_clean":"2023-12-26T21:37:49.054410+0000","last_became_active":"2023-12-26T21:37:49.050468+0000","last_became_peered":"2023-12-26T21:37:49.050468+0000","last_unstale":"2023-12-26T21:37:49.054410+0000","last_undegraded":"2023-12-26T21:37:49.054410+0000","last_fullsized":"2023-12-26T21:37:49.054410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:42.166494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.018737+0000","last_change":"2023-12-26T21:37:50.011289+0000","last_active":"2023-12-26T21:37:50.018737+0000","last_peered":"2023-12-26T21:37:50.018737+0000","last_clean":"2023-12-26T21:37:50.018737+0000","last_became_active":"2023-12-26T21:37:50.010959+0000","last_became_peered":"2023-12-26T21:37:50.010959+0000","last_unstale":"2023-12-26T21:37:50.018737+0000","last_undegraded":"2023-12-26T21:37:50.018737+0000","last_fullsized":"2023-12-26T21:37:50.018737+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:31:29.912297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.009851+0000","last_change":"2023-12-26T21:37:49.050353+0000","last_active":"2023-12-26T21:37:50.009851+0000","last_peered":"2023-12-26T21:37:50.009851+0000","last_clean":"2023-12-26T21:37:50.009851+0000","last_became_active":"2023-12-26T21:37:49.050075+0000","last_became_peered":"2023-12-26T21:37:49.050075+0000","last_unstale":"2023-12-26T21:37:50.009851+0000","last_undegraded":"2023-12-26T21:37:50.009851+0000","last_fullsized":"2023-12-26T21:37:50.009851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:38:15.966274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":172,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.003040+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:37:50.003040+0000","last_peered":"2023-12-26T21:37:50.003040+0000","last_clean":"2023-12-26T21:37:50.003040+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:37:50.003040+0000","last_undegraded":"2023-12-26T21:37:50.003040+0000","last_fullsized":"2023-12-26T21:37:50.003040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":146557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.91700000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.622},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.80700000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.872},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.84099999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.751}]},{"osd":7,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76500000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073588,"statfs":{"total":96636764160,"available":96331354112,"internally_reserved":0,"allocated":987136,"data_stored":742231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.59799999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.66000000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78600000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.627},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.86699999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.89100000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0049999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297672,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074168,"statfs":{"total":96636764160,"available":96331948032,"internally_reserved":0,"allocated":393216,"data_stored":146579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.50600000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.54600000000000004}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":0.59599999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.66500000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.74399999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542164,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":145055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.436},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.33900000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.374}]},{"osd":4,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70099999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.628},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.74299999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":145043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.61899999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.78100000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.81399999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.68000000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":145111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.3150000000000004,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.6479999999999997,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.39300000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.2610000000000001,"5min":4.2610000000000001,"15min":4.2610000000000001},"last":0.375},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.63300000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.81899999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.3650000000000002,"5min":4.3650000000000002,"15min":4.3650000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.5519999999999996,"5min":4.5519999999999996,"15min":4.5519999999999996},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.69099999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":4.6070000000000002,"5min":4.6070000000000002,"15min":4.6070000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":4.5069999999999997,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.58999999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073600,"statfs":{"total":96636764160,"available":96331366400,"internally_reserved":0,"allocated":974848,"data_stored":735459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.378},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.6600000000000001,"5min":4.6600000000000001,"15min":4.6600000000000001},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.3639999999999999,"5min":4.3639999999999999,"15min":4.3639999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":4.3259999999999996,"5min":4.3259999999999996,"15min":4.3259999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.548,"5min":4.548,"15min":4.548},"last":0.437}]},{"osd":4,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63400000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":152057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.496},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.62}]},{"osd":5,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:37:54.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:37:54.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.631+0000 7f09473ff700 1 Processor -- start 2023-12-26T21:37:54.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.631+0000 7f09473ff700 1 -- start start 2023-12-26T21:37:54.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f09473ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f094013c510 0x7f09400c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f09473ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f094006b8c0 0x7f094006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f09473ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09400686c0 con 0x7f09400cc100 2023-12-26T21:37:54.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f09473ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094006c280 con 0x7f094006b8c0 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f09473ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0940069520 con 0x7f094013c510 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f094499a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f094006b8c0 0x7f094006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f094499a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f094006b8c0 0x7f094006bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60942/0 (socket says 172.21.15.131:60942) 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f094499a700 1 -- 172.21.15.131:0/647140014 learned_addr learned my addr 172.21.15.131:0/647140014 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f094519b700 1 --2- 172.21.15.131:0/647140014 >> v2:172.21.15.190:3300/0 conn(0x7f094013c510 0x7f09400c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f094499a700 1 -- 172.21.15.131:0/647140014 >> v2:172.21.15.190:3300/0 conn(0x7f094013c510 msgr2=0x7f09400c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f094499a700 1 --2- 172.21.15.131:0/647140014 >> v2:172.21.15.190:3300/0 conn(0x7f094013c510 0x7f09400c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f094499a700 1 -- 172.21.15.131:0/647140014 >> v1:172.21.15.190:6789/0 conn(0x7f09400cc100 legacy=0x7f09400cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.633+0000 7f094499a700 1 -- 172.21.15.131:0/647140014 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0940068910 con 0x7f094006b8c0 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.634+0000 7f094499a700 1 --2- 172.21.15.131:0/647140014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094006b8c0 0x7f094006bd40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f094006fcc0 tx=0x7f0930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa826d5007a64537 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.634+0000 7f0937fff700 1 -- 172.21.15.131:0/647140014 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000b860 con 0x7f094006b8c0 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.634+0000 7f0937fff700 1 -- 172.21.15.131:0/647140014 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093000b9c0 con 0x7f094006b8c0 2023-12-26T21:37:54.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.634+0000 7f0937fff700 1 -- 172.21.15.131:0/647140014 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000bd60 con 0x7f094006b8c0 2023-12-26T21:37:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.634+0000 7f09473ff700 1 -- 172.21.15.131:0/647140014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094006b8c0 msgr2=0x7f094006bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.634+0000 7f09473ff700 1 --2- 172.21.15.131:0/647140014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094006b8c0 0x7f094006bd40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f094006fcc0 tx=0x7f0930009250 comp rx=0 tx=0).stop 2023-12-26T21:37:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.635+0000 7f09473ff700 1 -- 172.21.15.131:0/647140014 shutdown_connections 2023-12-26T21:37:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.635+0000 7f09473ff700 1 --2- 172.21.15.131:0/647140014 >> v2:172.21.15.190:3300/0 conn(0x7f094013c510 0x7f09400c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.635+0000 7f09473ff700 1 --2- 172.21.15.131:0/647140014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094006b8c0 0x7f094006bd40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.635+0000 7f09473ff700 1 -- 172.21.15.131:0/647140014 >> 172.21.15.131:0/647140014 conn(0x7f09400ce3e0 msgr2=0x7f09400ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:54.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.635+0000 7f09473ff700 1 -- 172.21.15.131:0/647140014 shutdown_connections 2023-12-26T21:37:54.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.635+0000 7f09473ff700 1 -- 172.21.15.131:0/647140014 wait complete. 2023-12-26T21:37:54.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.636+0000 7f09473ff700 1 Processor -- start 2023-12-26T21:37:54.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.636+0000 7f09473ff700 1 -- start start 2023-12-26T21:37:54.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f09473ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094006b8c0 0x7f09400b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f09473ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09400cc100 0x7f094012eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f09473ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094013c510 0x7f0940158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f09473ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094016d0b0 con 0x7f094013c510 2023-12-26T21:37:54.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f09473ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094012f3e0 con 0x7f09400cc100 2023-12-26T21:37:54.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f09473ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094016d8e0 con 0x7f094006b8c0 2023-12-26T21:37:54.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f094599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094013c510 0x7f0940158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f094499a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09400cc100 0x7f094012eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f094519b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094006b8c0 0x7f09400b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f094499a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09400cc100 0x7f094012eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53680/0 (socket says 172.21.15.131:53680) 2023-12-26T21:37:54.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f094599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094013c510 0x7f0940158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60958/0 (socket says 172.21.15.131:60958) 2023-12-26T21:37:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.637+0000 7f094499a700 1 -- 172.21.15.131:0/2684647436 learned_addr learned my addr 172.21.15.131:0/2684647436 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f094499a700 1 -- 172.21.15.131:0/2684647436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094006b8c0 msgr2=0x7f09400b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f094499a700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094006b8c0 0x7f09400b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f094499a700 1 -- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094013c510 msgr2=0x7f0940158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f094499a700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094013c510 0x7f0940158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f094499a700 1 -- 172.21.15.131:0/2684647436 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09300094f0 con 0x7f09400cc100 2023-12-26T21:37:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f094599c700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094013c510 0x7f0940158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f094499a700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09400cc100 0x7f094012eea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f094006fe40 tx=0x7f0930017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f09357fa700 1 -- 172.21.15.131:0/2684647436 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930024440 con 0x7f09400cc100 2023-12-26T21:37:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f09357fa700 1 -- 172.21.15.131:0/2684647436 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09300245a0 con 0x7f09400cc100 2023-12-26T21:37:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09400b3dc0 con 0x7f09400cc100 2023-12-26T21:37:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.638+0000 7f09357fa700 1 -- 172.21.15.131:0/2684647436 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930024960 con 0x7f09400cc100 2023-12-26T21:37:54.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.639+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094016d570 con 0x7f09400cc100 2023-12-26T21:37:54.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.639+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094016da70 con 0x7f09400cc100 2023-12-26T21:37:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.640+0000 7f09357fa700 1 -- 172.21.15.131:0/2684647436 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0930024d50 con 0x7f09400cc100 2023-12-26T21:37:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.641+0000 7f09357fa700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0924066f30 0x7f09240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.641+0000 7f09357fa700 1 -- 172.21.15.131:0/2684647436 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0930085fc0 con 0x7f09400cc100 2023-12-26T21:37:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.641+0000 7f094519b700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0924066f30 0x7f09240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:54.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.642+0000 7f094519b700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0924066f30 0x7f09240693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f093c006aa0 tx=0x7f093c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:54.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.645+0000 7f09357fa700 1 -- 172.21.15.131:0/2684647436 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0930053210 con 0x7f09400cc100 2023-12-26T21:37:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.834+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09400657e0 con 0x7f0924066f30 2023-12-26T21:37:54.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.842+0000 7f09357fa700 1 -- 172.21.15.131:0/2684647436 <== mgr.4100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288665 (secure 0 0 0) 0x7f09400657e0 con 0x7f0924066f30 2023-12-26T21:37:54.846 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0924066f30 msgr2=0x7f09240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0924066f30 0x7f09240693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f093c006aa0 tx=0x7f093c008040 comp rx=0 tx=0).stop 2023-12-26T21:37:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09400cc100 msgr2=0x7f094012eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09400cc100 0x7f094012eea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f094006fe40 tx=0x7f0930017000 comp rx=0 tx=0).stop 2023-12-26T21:37:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 shutdown_connections 2023-12-26T21:37:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f094013c510 0x7f0940158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0924066f30 0x7f09240693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094006b8c0 0x7f09400b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 --2- 172.21.15.131:0/2684647436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09400cc100 0x7f094012eea0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 >> 172.21.15.131:0/2684647436 conn(0x7f09400ce3e0 msgr2=0x7f09400c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.846+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 shutdown_connections 2023-12-26T21:37:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:54.847+0000 7f09473ff700 1 -- 172.21.15.131:0/2684647436 wait complete. 2023-12-26T21:37:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:37:54.864 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-12-26T21:37:53.011173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382496,"kb_used_data":4192,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592224,"statfs":{"total":773094113280,"available":770654437376,"internally_reserved":0,"allocated":4292608,"data_stored":2358092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.978802"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022566+0000","last_change":"2023-12-26T21:37:50.022566+0000","last_active":"2023-12-26T21:37:50.022566+0000","last_peered":"2023-12-26T21:37:50.022566+0000","last_clean":"2023-12-26T21:37:50.022566+0000","last_became_active":"2023-12-26T21:37:50.022341+0000","last_became_peered":"2023-12-26T21:37:50.022341+0000","last_unstale":"2023-12-26T21:37:50.022566+0000","last_undegraded":"2023-12-26T21:37:50.022566+0000","last_fullsized":"2023-12-26T21:37:50.022566+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:01.590388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029202+0000","last_change":"2023-12-26T21:37:50.023094+0000","last_active":"2023-12-26T21:37:50.029202+0000","last_peered":"2023-12-26T21:37:50.029202+0000","last_clean":"2023-12-26T21:37:50.029202+0000","last_became_active":"2023-12-26T21:37:50.022865+0000","last_became_peered":"2023-12-26T21:37:50.022865+0000","last_unstale":"2023-12-26T21:37:50.029202+0000","last_undegraded":"2023-12-26T21:37:50.029202+0000","last_fullsized":"2023-12-26T21:37:50.029202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:41:45.917944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015939+0000","last_change":"2023-12-26T21:37:50.014051+0000","last_active":"2023-12-26T21:37:50.015939+0000","last_peered":"2023-12-26T21:37:50.015939+0000","last_clean":"2023-12-26T21:37:50.015939+0000","last_became_active":"2023-12-26T21:37:50.013797+0000","last_became_peered":"2023-12-26T21:37:50.013797+0000","last_unstale":"2023-12-26T21:37:50.015939+0000","last_undegraded":"2023-12-26T21:37:50.015939+0000","last_fullsized":"2023-12-26T21:37:50.015939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:25:10.241763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.032029+0000","last_change":"2023-12-26T21:37:50.032029+0000","last_active":"2023-12-26T21:37:50.032029+0000","last_peered":"2023-12-26T21:37:50.032029+0000","last_clean":"2023-12-26T21:37:50.032029+0000","last_became_active":"2023-12-26T21:37:50.031732+0000","last_became_peered":"2023-12-26T21:37:50.031732+0000","last_unstale":"2023-12-26T21:37:50.032029+0000","last_undegraded":"2023-12-26T21:37:50.032029+0000","last_fullsized":"2023-12-26T21:37:50.032029+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:18:48.013567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.028101+0000","last_change":"2023-12-26T21:37:50.018511+0000","last_active":"2023-12-26T21:37:50.028101+0000","last_peered":"2023-12-26T21:37:50.028101+0000","last_clean":"2023-12-26T21:37:50.028101+0000","last_became_active":"2023-12-26T21:37:50.018249+0000","last_became_peered":"2023-12-26T21:37:50.018249+0000","last_unstale":"2023-12-26T21:37:50.028101+0000","last_undegraded":"2023-12-26T21:37:50.028101+0000","last_fullsized":"2023-12-26T21:37:50.028101+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:35.444063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.026118+0000","last_change":"2023-12-26T21:37:50.019338+0000","last_active":"2023-12-26T21:37:50.026118+0000","last_peered":"2023-12-26T21:37:50.026118+0000","last_clean":"2023-12-26T21:37:50.026118+0000","last_became_active":"2023-12-26T21:37:50.019083+0000","last_became_peered":"2023-12-26T21:37:50.019083+0000","last_unstale":"2023-12-26T21:37:50.026118+0000","last_undegraded":"2023-12-26T21:37:50.026118+0000","last_fullsized":"2023-12-26T21:37:50.026118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:53:29.786265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015805+0000","last_change":"2023-12-26T21:37:50.015805+0000","last_active":"2023-12-26T21:37:50.015805+0000","last_peered":"2023-12-26T21:37:50.015805+0000","last_clean":"2023-12-26T21:37:50.015805+0000","last_became_active":"2023-12-26T21:37:50.015330+0000","last_became_peered":"2023-12-26T21:37:50.015330+0000","last_unstale":"2023-12-26T21:37:50.015805+0000","last_undegraded":"2023-12-26T21:37:50.015805+0000","last_fullsized":"2023-12-26T21:37:50.015805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:32:16.784548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.020147+0000","last_change":"2023-12-26T21:37:50.020147+0000","last_active":"2023-12-26T21:37:50.020147+0000","last_peered":"2023-12-26T21:37:50.020147+0000","last_clean":"2023-12-26T21:37:50.020147+0000","last_became_active":"2023-12-26T21:37:50.019567+0000","last_became_peered":"2023-12-26T21:37:50.019567+0000","last_unstale":"2023-12-26T21:37:50.020147+0000","last_undegraded":"2023-12-26T21:37:50.020147+0000","last_fullsized":"2023-12-26T21:37:50.020147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:17:52.028697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.023970+0000","last_change":"2023-12-26T21:37:50.022267+0000","last_active":"2023-12-26T21:37:50.023970+0000","last_peered":"2023-12-26T21:37:50.023970+0000","last_clean":"2023-12-26T21:37:50.023970+0000","last_became_active":"2023-12-26T21:37:50.021937+0000","last_became_peered":"2023-12-26T21:37:50.021937+0000","last_unstale":"2023-12-26T21:37:50.023970+0000","last_undegraded":"2023-12-26T21:37:50.023970+0000","last_fullsized":"2023-12-26T21:37:50.023970+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:44.548035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.024140+0000","last_change":"2023-12-26T21:37:50.024140+0000","last_active":"2023-12-26T21:37:50.024140+0000","last_peered":"2023-12-26T21:37:50.024140+0000","last_clean":"2023-12-26T21:37:50.024140+0000","last_became_active":"2023-12-26T21:37:50.023412+0000","last_became_peered":"2023-12-26T21:37:50.023412+0000","last_unstale":"2023-12-26T21:37:50.024140+0000","last_undegraded":"2023-12-26T21:37:50.024140+0000","last_fullsized":"2023-12-26T21:37:50.024140+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:45:20.209845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029578+0000","last_change":"2023-12-26T21:37:50.027938+0000","last_active":"2023-12-26T21:37:50.029578+0000","last_peered":"2023-12-26T21:37:50.029578+0000","last_clean":"2023-12-26T21:37:50.029578+0000","last_became_active":"2023-12-26T21:37:50.027689+0000","last_became_peered":"2023-12-26T21:37:50.027689+0000","last_unstale":"2023-12-26T21:37:50.029578+0000","last_undegraded":"2023-12-26T21:37:50.029578+0000","last_fullsized":"2023-12-26T21:37:50.029578+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:57:50.330517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.014727+0000","last_change":"2023-12-26T21:37:50.011664+0000","last_active":"2023-12-26T21:37:50.014727+0000","last_peered":"2023-12-26T21:37:50.014727+0000","last_clean":"2023-12-26T21:37:50.014727+0000","last_became_active":"2023-12-26T21:37:50.011407+0000","last_became_peered":"2023-12-26T21:37:50.011407+0000","last_unstale":"2023-12-26T21:37:50.014727+0000","last_undegraded":"2023-12-26T21:37:50.014727+0000","last_fullsized":"2023-12-26T21:37:50.014727+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:57:45.742847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029851+0000","last_change":"2023-12-26T21:37:50.026678+0000","last_active":"2023-12-26T21:37:50.029851+0000","last_peered":"2023-12-26T21:37:50.029851+0000","last_clean":"2023-12-26T21:37:50.029851+0000","last_became_active":"2023-12-26T21:37:50.026387+0000","last_became_peered":"2023-12-26T21:37:50.026387+0000","last_unstale":"2023-12-26T21:37:50.029851+0000","last_undegraded":"2023-12-26T21:37:50.029851+0000","last_fullsized":"2023-12-26T21:37:50.029851+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:01:59.418380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021785+0000","last_change":"2023-12-26T21:37:50.021785+0000","last_active":"2023-12-26T21:37:50.021785+0000","last_peered":"2023-12-26T21:37:50.021785+0000","last_clean":"2023-12-26T21:37:50.021785+0000","last_became_active":"2023-12-26T21:37:50.020946+0000","last_became_peered":"2023-12-26T21:37:50.020946+0000","last_unstale":"2023-12-26T21:37:50.021785+0000","last_undegraded":"2023-12-26T21:37:50.021785+0000","last_fullsized":"2023-12-26T21:37:50.021785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:17:52.028697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021315+0000","last_change":"2023-12-26T21:37:50.014903+0000","last_active":"2023-12-26T21:37:50.021315+0000","last_peered":"2023-12-26T21:37:50.021315+0000","last_clean":"2023-12-26T21:37:50.021315+0000","last_became_active":"2023-12-26T21:37:50.014641+0000","last_became_peered":"2023-12-26T21:37:50.014641+0000","last_unstale":"2023-12-26T21:37:50.021315+0000","last_undegraded":"2023-12-26T21:37:50.021315+0000","last_fullsized":"2023-12-26T21:37:50.021315+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:31.976009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.018923+0000","last_change":"2023-12-26T21:37:50.009856+0000","last_active":"2023-12-26T21:37:50.018923+0000","last_peered":"2023-12-26T21:37:50.018923+0000","last_clean":"2023-12-26T21:37:50.018923+0000","last_became_active":"2023-12-26T21:37:50.009584+0000","last_became_peered":"2023-12-26T21:37:50.009584+0000","last_unstale":"2023-12-26T21:37:50.018923+0000","last_undegraded":"2023-12-26T21:37:50.018923+0000","last_fullsized":"2023-12-26T21:37:50.018923+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:35.444063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022461+0000","last_change":"2023-12-26T21:37:50.016920+0000","last_active":"2023-12-26T21:37:50.022461+0000","last_peered":"2023-12-26T21:37:50.022461+0000","last_clean":"2023-12-26T21:37:50.022461+0000","last_became_active":"2023-12-26T21:37:50.016668+0000","last_became_peered":"2023-12-26T21:37:50.016668+0000","last_unstale":"2023-12-26T21:37:50.022461+0000","last_undegraded":"2023-12-26T21:37:50.022461+0000","last_fullsized":"2023-12-26T21:37:50.022461+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:06:15.016173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.014665+0000","last_change":"2023-12-26T21:37:50.010453+0000","last_active":"2023-12-26T21:37:50.014665+0000","last_peered":"2023-12-26T21:37:50.014665+0000","last_clean":"2023-12-26T21:37:50.014665+0000","last_became_active":"2023-12-26T21:37:50.010137+0000","last_became_peered":"2023-12-26T21:37:50.010137+0000","last_unstale":"2023-12-26T21:37:50.014665+0000","last_undegraded":"2023-12-26T21:37:50.014665+0000","last_fullsized":"2023-12-26T21:37:50.014665+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:37:58.777798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.008907+0000","last_change":"2023-12-26T21:37:50.006668+0000","last_active":"2023-12-26T21:37:50.008907+0000","last_peered":"2023-12-26T21:37:50.008907+0000","last_clean":"2023-12-26T21:37:50.008907+0000","last_became_active":"2023-12-26T21:37:50.006120+0000","last_became_peered":"2023-12-26T21:37:50.006120+0000","last_unstale":"2023-12-26T21:37:50.008907+0000","last_undegraded":"2023-12-26T21:37:50.008907+0000","last_fullsized":"2023-12-26T21:37:50.008907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:42:06.643972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.024437+0000","last_change":"2023-12-26T21:37:50.019055+0000","last_active":"2023-12-26T21:37:50.024437+0000","last_peered":"2023-12-26T21:37:50.024437+0000","last_clean":"2023-12-26T21:37:50.024437+0000","last_became_active":"2023-12-26T21:37:50.018692+0000","last_became_peered":"2023-12-26T21:37:50.018692+0000","last_unstale":"2023-12-26T21:37:50.024437+0000","last_undegraded":"2023-12-26T21:37:50.024437+0000","last_fullsized":"2023-12-26T21:37:50.024437+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:45:36.463616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021818+0000","last_change":"2023-12-26T21:37:50.016617+0000","last_active":"2023-12-26T21:37:50.021818+0000","last_peered":"2023-12-26T21:37:50.021818+0000","last_clean":"2023-12-26T21:37:50.021818+0000","last_became_active":"2023-12-26T21:37:50.016326+0000","last_became_peered":"2023-12-26T21:37:50.016326+0000","last_unstale":"2023-12-26T21:37:50.021818+0000","last_undegraded":"2023-12-26T21:37:50.021818+0000","last_fullsized":"2023-12-26T21:37:50.021818+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:51:26.623941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.027933+0000","last_change":"2023-12-26T21:37:50.027933+0000","last_active":"2023-12-26T21:37:50.027933+0000","last_peered":"2023-12-26T21:37:50.027933+0000","last_clean":"2023-12-26T21:37:50.027933+0000","last_became_active":"2023-12-26T21:37:50.027647+0000","last_became_peered":"2023-12-26T21:37:50.027647+0000","last_unstale":"2023-12-26T21:37:50.027933+0000","last_undegraded":"2023-12-26T21:37:50.027933+0000","last_fullsized":"2023-12-26T21:37:50.027933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:45:20.209845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.016563+0000","last_change":"2023-12-26T21:37:50.012252+0000","last_active":"2023-12-26T21:37:50.016563+0000","last_peered":"2023-12-26T21:37:50.016563+0000","last_clean":"2023-12-26T21:37:50.016563+0000","last_became_active":"2023-12-26T21:37:50.011863+0000","last_became_peered":"2023-12-26T21:37:50.011863+0000","last_unstale":"2023-12-26T21:37:50.016563+0000","last_undegraded":"2023-12-26T21:37:50.016563+0000","last_fullsized":"2023-12-26T21:37:50.016563+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:46:52.556912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.025094+0000","last_change":"2023-12-26T21:37:50.025094+0000","last_active":"2023-12-26T21:37:50.025094+0000","last_peered":"2023-12-26T21:37:50.025094+0000","last_clean":"2023-12-26T21:37:50.025094+0000","last_became_active":"2023-12-26T21:37:50.024889+0000","last_became_peered":"2023-12-26T21:37:50.024889+0000","last_unstale":"2023-12-26T21:37:50.025094+0000","last_undegraded":"2023-12-26T21:37:50.025094+0000","last_fullsized":"2023-12-26T21:37:50.025094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:18:48.013567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.030699+0000","last_change":"2023-12-26T21:37:50.030699+0000","last_active":"2023-12-26T21:37:50.030699+0000","last_peered":"2023-12-26T21:37:50.030699+0000","last_clean":"2023-12-26T21:37:50.030699+0000","last_became_active":"2023-12-26T21:37:50.030294+0000","last_became_peered":"2023-12-26T21:37:50.030294+0000","last_unstale":"2023-12-26T21:37:50.030699+0000","last_undegraded":"2023-12-26T21:37:50.030699+0000","last_fullsized":"2023-12-26T21:37:50.030699+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:25:12.521941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.023255+0000","last_change":"2023-12-26T21:37:50.023255+0000","last_active":"2023-12-26T21:37:50.023255+0000","last_peered":"2023-12-26T21:37:50.023255+0000","last_clean":"2023-12-26T21:37:50.023255+0000","last_became_active":"2023-12-26T21:37:50.021901+0000","last_became_peered":"2023-12-26T21:37:50.021901+0000","last_unstale":"2023-12-26T21:37:50.023255+0000","last_undegraded":"2023-12-26T21:37:50.023255+0000","last_fullsized":"2023-12-26T21:37:50.023255+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:32:16.784548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.028123+0000","last_change":"2023-12-26T21:37:50.017265+0000","last_active":"2023-12-26T21:37:50.028123+0000","last_peered":"2023-12-26T21:37:50.028123+0000","last_clean":"2023-12-26T21:37:50.028123+0000","last_became_active":"2023-12-26T21:37:50.017015+0000","last_became_peered":"2023-12-26T21:37:50.017015+0000","last_unstale":"2023-12-26T21:37:50.028123+0000","last_undegraded":"2023-12-26T21:37:50.028123+0000","last_fullsized":"2023-12-26T21:37:50.028123+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:34:15.322150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.025524+0000","last_change":"2023-12-26T21:37:50.025524+0000","last_active":"2023-12-26T21:37:50.025524+0000","last_peered":"2023-12-26T21:37:50.025524+0000","last_clean":"2023-12-26T21:37:50.025524+0000","last_became_active":"2023-12-26T21:37:50.025332+0000","last_became_peered":"2023-12-26T21:37:50.025332+0000","last_unstale":"2023-12-26T21:37:50.025524+0000","last_undegraded":"2023-12-26T21:37:50.025524+0000","last_fullsized":"2023-12-26T21:37:50.025524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:13.643734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022695+0000","last_change":"2023-12-26T21:37:50.019399+0000","last_active":"2023-12-26T21:37:50.022695+0000","last_peered":"2023-12-26T21:37:50.022695+0000","last_clean":"2023-12-26T21:37:50.022695+0000","last_became_active":"2023-12-26T21:37:50.018744+0000","last_became_peered":"2023-12-26T21:37:50.018744+0000","last_unstale":"2023-12-26T21:37:50.022695+0000","last_undegraded":"2023-12-26T21:37:50.022695+0000","last_fullsized":"2023-12-26T21:37:50.022695+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:44:28.087202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021842+0000","last_change":"2023-12-26T21:37:50.016056+0000","last_active":"2023-12-26T21:37:50.021842+0000","last_peered":"2023-12-26T21:37:50.021842+0000","last_clean":"2023-12-26T21:37:50.021842+0000","last_became_active":"2023-12-26T21:37:50.015774+0000","last_became_peered":"2023-12-26T21:37:50.015774+0000","last_unstale":"2023-12-26T21:37:50.021842+0000","last_undegraded":"2023-12-26T21:37:50.021842+0000","last_fullsized":"2023-12-26T21:37:50.021842+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:38:16.973436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.020540+0000","last_change":"2023-12-26T21:37:50.020540+0000","last_active":"2023-12-26T21:37:50.020540+0000","last_peered":"2023-12-26T21:37:50.020540+0000","last_clean":"2023-12-26T21:37:50.020540+0000","last_became_active":"2023-12-26T21:37:50.020300+0000","last_became_peered":"2023-12-26T21:37:50.020300+0000","last_unstale":"2023-12-26T21:37:50.020540+0000","last_undegraded":"2023-12-26T21:37:50.020540+0000","last_fullsized":"2023-12-26T21:37:50.020540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:49.041763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.019426+0000","last_change":"2023-12-26T21:37:50.013636+0000","last_active":"2023-12-26T21:37:50.019426+0000","last_peered":"2023-12-26T21:37:50.019426+0000","last_clean":"2023-12-26T21:37:50.019426+0000","last_became_active":"2023-12-26T21:37:50.013369+0000","last_became_peered":"2023-12-26T21:37:50.013369+0000","last_unstale":"2023-12-26T21:37:50.019426+0000","last_undegraded":"2023-12-26T21:37:50.019426+0000","last_fullsized":"2023-12-26T21:37:50.019426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:37.762931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.027104+0000","last_change":"2023-12-26T21:37:50.021148+0000","last_active":"2023-12-26T21:37:50.027104+0000","last_peered":"2023-12-26T21:37:50.027104+0000","last_clean":"2023-12-26T21:37:50.027104+0000","last_became_active":"2023-12-26T21:37:50.020878+0000","last_became_peered":"2023-12-26T21:37:50.020878+0000","last_unstale":"2023-12-26T21:37:50.027104+0000","last_undegraded":"2023-12-26T21:37:50.027104+0000","last_fullsized":"2023-12-26T21:37:50.027104+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:19:30.444831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060466+0000","last_change":"2023-12-26T21:37:49.059929+0000","last_active":"2023-12-26T21:37:50.060466+0000","last_peered":"2023-12-26T21:37:50.060466+0000","last_clean":"2023-12-26T21:37:50.060466+0000","last_became_active":"2023-12-26T21:37:49.059601+0000","last_became_peered":"2023-12-26T21:37:49.059601+0000","last_unstale":"2023-12-26T21:37:50.060466+0000","last_undegraded":"2023-12-26T21:37:50.060466+0000","last_fullsized":"2023-12-26T21:37:50.060466+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:25:37.176835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.023473+0000","last_change":"2023-12-26T21:37:50.023473+0000","last_active":"2023-12-26T21:37:50.023473+0000","last_peered":"2023-12-26T21:37:50.023473+0000","last_clean":"2023-12-26T21:37:50.023473+0000","last_became_active":"2023-12-26T21:37:50.022916+0000","last_became_peered":"2023-12-26T21:37:50.022916+0000","last_unstale":"2023-12-26T21:37:50.023473+0000","last_undegraded":"2023-12-26T21:37:50.023473+0000","last_fullsized":"2023-12-26T21:37:50.023473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:20:03.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.011219+0000","last_change":"2023-12-26T21:37:49.051127+0000","last_active":"2023-12-26T21:37:50.011219+0000","last_peered":"2023-12-26T21:37:50.011219+0000","last_clean":"2023-12-26T21:37:50.011219+0000","last_became_active":"2023-12-26T21:37:49.050808+0000","last_became_peered":"2023-12-26T21:37:49.050808+0000","last_unstale":"2023-12-26T21:37:50.011219+0000","last_undegraded":"2023-12-26T21:37:50.011219+0000","last_fullsized":"2023-12-26T21:37:50.011219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:37:57.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.027906+0000","last_change":"2023-12-26T21:37:50.027906+0000","last_active":"2023-12-26T21:37:50.027906+0000","last_peered":"2023-12-26T21:37:50.027906+0000","last_clean":"2023-12-26T21:37:50.027906+0000","last_became_active":"2023-12-26T21:37:50.027649+0000","last_became_peered":"2023-12-26T21:37:50.027649+0000","last_unstale":"2023-12-26T21:37:50.027906+0000","last_undegraded":"2023-12-26T21:37:50.027906+0000","last_fullsized":"2023-12-26T21:37:50.027906+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:55:22.146030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060696+0000","last_change":"2023-12-26T21:37:49.057504+0000","last_active":"2023-12-26T21:37:50.060696+0000","last_peered":"2023-12-26T21:37:50.060696+0000","last_clean":"2023-12-26T21:37:50.060696+0000","last_became_active":"2023-12-26T21:37:49.057174+0000","last_became_peered":"2023-12-26T21:37:49.057174+0000","last_unstale":"2023-12-26T21:37:50.060696+0000","last_undegraded":"2023-12-26T21:37:50.060696+0000","last_fullsized":"2023-12-26T21:37:50.060696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:25:35.067327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.032108+0000","last_change":"2023-12-26T21:37:50.032108+0000","last_active":"2023-12-26T21:37:50.032108+0000","last_peered":"2023-12-26T21:37:50.032108+0000","last_clean":"2023-12-26T21:37:50.032108+0000","last_became_active":"2023-12-26T21:37:50.031800+0000","last_became_peered":"2023-12-26T21:37:50.031800+0000","last_unstale":"2023-12-26T21:37:50.032108+0000","last_undegraded":"2023-12-26T21:37:50.032108+0000","last_fullsized":"2023-12-26T21:37:50.032108+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:13.643734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060308+0000","last_change":"2023-12-26T21:37:49.059658+0000","last_active":"2023-12-26T21:37:50.060308+0000","last_peered":"2023-12-26T21:37:50.060308+0000","last_clean":"2023-12-26T21:37:50.060308+0000","last_became_active":"2023-12-26T21:37:49.059468+0000","last_became_peered":"2023-12-26T21:37:49.059468+0000","last_unstale":"2023-12-26T21:37:50.060308+0000","last_undegraded":"2023-12-26T21:37:50.060308+0000","last_fullsized":"2023-12-26T21:37:50.060308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:53:28.779103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015238+0000","last_change":"2023-12-26T21:37:50.013675+0000","last_active":"2023-12-26T21:37:50.015238+0000","last_peered":"2023-12-26T21:37:50.015238+0000","last_clean":"2023-12-26T21:37:50.015238+0000","last_became_active":"2023-12-26T21:37:50.013264+0000","last_became_peered":"2023-12-26T21:37:50.013264+0000","last_unstale":"2023-12-26T21:37:50.015238+0000","last_undegraded":"2023-12-26T21:37:50.015238+0000","last_fullsized":"2023-12-26T21:37:50.015238+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:01.152199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.008106+0000","last_change":"2023-12-26T21:37:49.065084+0000","last_active":"2023-12-26T21:37:50.008106+0000","last_peered":"2023-12-26T21:37:50.008106+0000","last_clean":"2023-12-26T21:37:50.008106+0000","last_became_active":"2023-12-26T21:37:49.064768+0000","last_became_peered":"2023-12-26T21:37:49.064768+0000","last_unstale":"2023-12-26T21:37:50.008106+0000","last_undegraded":"2023-12-26T21:37:50.008106+0000","last_fullsized":"2023-12-26T21:37:50.008106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:42:05.636810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.020162+0000","last_change":"2023-12-26T21:37:50.020162+0000","last_active":"2023-12-26T21:37:50.020162+0000","last_peered":"2023-12-26T21:37:50.020162+0000","last_clean":"2023-12-26T21:37:50.020162+0000","last_became_active":"2023-12-26T21:37:50.019885+0000","last_became_peered":"2023-12-26T21:37:50.019885+0000","last_unstale":"2023-12-26T21:37:50.020162+0000","last_undegraded":"2023-12-26T21:37:50.020162+0000","last_fullsized":"2023-12-26T21:37:50.020162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:31:29.275066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.008730+0000","last_change":"2023-12-26T21:37:49.057844+0000","last_active":"2023-12-26T21:37:50.008730+0000","last_peered":"2023-12-26T21:37:50.008730+0000","last_clean":"2023-12-26T21:37:50.008730+0000","last_became_active":"2023-12-26T21:37:49.057459+0000","last_became_peered":"2023-12-26T21:37:49.057459+0000","last_unstale":"2023-12-26T21:37:50.008730+0000","last_undegraded":"2023-12-26T21:37:50.008730+0000","last_fullsized":"2023-12-26T21:37:50.008730+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:37.235627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.012234+0000","last_change":"2023-12-26T21:37:50.010055+0000","last_active":"2023-12-26T21:37:50.012234+0000","last_peered":"2023-12-26T21:37:50.012234+0000","last_clean":"2023-12-26T21:37:50.012234+0000","last_became_active":"2023-12-26T21:37:50.009783+0000","last_became_peered":"2023-12-26T21:37:50.009783+0000","last_unstale":"2023-12-26T21:37:50.012234+0000","last_undegraded":"2023-12-26T21:37:50.012234+0000","last_fullsized":"2023-12-26T21:37:50.012234+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:58:41.674361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.000500+0000","last_change":"2023-12-26T21:37:49.059217+0000","last_active":"2023-12-26T21:37:50.000500+0000","last_peered":"2023-12-26T21:37:50.000500+0000","last_clean":"2023-12-26T21:37:50.000500+0000","last_became_active":"2023-12-26T21:37:49.056059+0000","last_became_peered":"2023-12-26T21:37:49.056059+0000","last_unstale":"2023-12-26T21:37:50.000500+0000","last_undegraded":"2023-12-26T21:37:50.000500+0000","last_fullsized":"2023-12-26T21:37:50.000500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:27:37.134010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.030718+0000","last_change":"2023-12-26T21:37:50.030718+0000","last_active":"2023-12-26T21:37:50.030718+0000","last_peered":"2023-12-26T21:37:50.030718+0000","last_clean":"2023-12-26T21:37:50.030718+0000","last_became_active":"2023-12-26T21:37:50.030134+0000","last_became_peered":"2023-12-26T21:37:50.030134+0000","last_unstale":"2023-12-26T21:37:50.030718+0000","last_undegraded":"2023-12-26T21:37:50.030718+0000","last_fullsized":"2023-12-26T21:37:50.030718+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:53:00.606274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.065971+0000","last_change":"2023-12-26T21:37:49.064610+0000","last_active":"2023-12-26T21:37:49.065971+0000","last_peered":"2023-12-26T21:37:49.065971+0000","last_clean":"2023-12-26T21:37:49.065971+0000","last_became_active":"2023-12-26T21:37:49.064374+0000","last_became_peered":"2023-12-26T21:37:49.064374+0000","last_unstale":"2023-12-26T21:37:49.065971+0000","last_undegraded":"2023-12-26T21:37:49.065971+0000","last_fullsized":"2023-12-26T21:37:49.065971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:37:57.770636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021884+0000","last_change":"2023-12-26T21:37:50.019779+0000","last_active":"2023-12-26T21:37:50.021884+0000","last_peered":"2023-12-26T21:37:50.021884+0000","last_clean":"2023-12-26T21:37:50.021884+0000","last_became_active":"2023-12-26T21:37:50.019495+0000","last_became_peered":"2023-12-26T21:37:50.019495+0000","last_unstale":"2023-12-26T21:37:50.021884+0000","last_undegraded":"2023-12-26T21:37:50.021884+0000","last_fullsized":"2023-12-26T21:37:50.021884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:37:29.638859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.006124+0000","last_change":"2023-12-26T21:37:49.057640+0000","last_active":"2023-12-26T21:37:50.006124+0000","last_peered":"2023-12-26T21:37:50.006124+0000","last_clean":"2023-12-26T21:37:50.006124+0000","last_became_active":"2023-12-26T21:37:49.057342+0000","last_became_peered":"2023-12-26T21:37:49.057342+0000","last_unstale":"2023-12-26T21:37:50.006124+0000","last_undegraded":"2023-12-26T21:37:50.006124+0000","last_fullsized":"2023-12-26T21:37:50.006124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:56:21.776351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.017719+0000","last_change":"2023-12-26T21:37:50.017719+0000","last_active":"2023-12-26T21:37:50.017719+0000","last_peered":"2023-12-26T21:37:50.017719+0000","last_clean":"2023-12-26T21:37:50.017719+0000","last_became_active":"2023-12-26T21:37:50.017423+0000","last_became_peered":"2023-12-26T21:37:50.017423+0000","last_unstale":"2023-12-26T21:37:50.017719+0000","last_undegraded":"2023-12-26T21:37:50.017719+0000","last_fullsized":"2023-12-26T21:37:50.017719+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:55:22.146030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.007001+0000","last_change":"2023-12-26T21:37:49.064273+0000","last_active":"2023-12-26T21:37:50.007001+0000","last_peered":"2023-12-26T21:37:50.007001+0000","last_clean":"2023-12-26T21:37:50.007001+0000","last_became_active":"2023-12-26T21:37:49.058591+0000","last_became_peered":"2023-12-26T21:37:49.058591+0000","last_unstale":"2023-12-26T21:37:50.007001+0000","last_undegraded":"2023-12-26T21:37:50.007001+0000","last_fullsized":"2023-12-26T21:37:50.007001+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:19:29.437669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.032142+0000","last_change":"2023-12-26T21:37:50.032142+0000","last_active":"2023-12-26T21:37:50.032142+0000","last_peered":"2023-12-26T21:37:50.032142+0000","last_clean":"2023-12-26T21:37:50.032142+0000","last_became_active":"2023-12-26T21:37:50.031369+0000","last_became_peered":"2023-12-26T21:37:50.031369+0000","last_unstale":"2023-12-26T21:37:50.032142+0000","last_undegraded":"2023-12-26T21:37:50.032142+0000","last_fullsized":"2023-12-26T21:37:50.032142+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:45.556024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.055312+0000","last_change":"2023-12-26T21:37:49.058851+0000","last_active":"2023-12-26T21:37:50.055312+0000","last_peered":"2023-12-26T21:37:50.055312+0000","last_clean":"2023-12-26T21:37:50.055312+0000","last_became_active":"2023-12-26T21:37:49.058558+0000","last_became_peered":"2023-12-26T21:37:49.058558+0000","last_unstale":"2023-12-26T21:37:50.055312+0000","last_undegraded":"2023-12-26T21:37:50.055312+0000","last_fullsized":"2023-12-26T21:37:50.055312+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:30.968847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029597+0000","last_change":"2023-12-26T21:37:50.028023+0000","last_active":"2023-12-26T21:37:50.029597+0000","last_peered":"2023-12-26T21:37:50.029597+0000","last_clean":"2023-12-26T21:37:50.029597+0000","last_became_active":"2023-12-26T21:37:50.027709+0000","last_became_peered":"2023-12-26T21:37:50.027709+0000","last_unstale":"2023-12-26T21:37:50.029597+0000","last_undegraded":"2023-12-26T21:37:50.029597+0000","last_fullsized":"2023-12-26T21:37:50.029597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:37.613373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.062877+0000","last_change":"2023-12-26T21:37:49.060492+0000","last_active":"2023-12-26T21:37:50.062877+0000","last_peered":"2023-12-26T21:37:50.062877+0000","last_clean":"2023-12-26T21:37:50.062877+0000","last_became_active":"2023-12-26T21:37:49.059949+0000","last_became_peered":"2023-12-26T21:37:49.059949+0000","last_unstale":"2023-12-26T21:37:50.062877+0000","last_undegraded":"2023-12-26T21:37:50.062877+0000","last_fullsized":"2023-12-26T21:37:50.062877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:21:06.881704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021921+0000","last_change":"2023-12-26T21:37:50.016184+0000","last_active":"2023-12-26T21:37:50.021921+0000","last_peered":"2023-12-26T21:37:50.021921+0000","last_clean":"2023-12-26T21:37:50.021921+0000","last_became_active":"2023-12-26T21:37:50.015896+0000","last_became_peered":"2023-12-26T21:37:50.015896+0000","last_unstale":"2023-12-26T21:37:50.021921+0000","last_undegraded":"2023-12-26T21:37:50.021921+0000","last_fullsized":"2023-12-26T21:37:50.021921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:36.781362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.057829+0000","last_change":"2023-12-26T21:37:49.056945+0000","last_active":"2023-12-26T21:37:50.057829+0000","last_peered":"2023-12-26T21:37:50.057829+0000","last_clean":"2023-12-26T21:37:50.057829+0000","last_became_active":"2023-12-26T21:37:49.056678+0000","last_became_peered":"2023-12-26T21:37:49.056678+0000","last_unstale":"2023-12-26T21:37:50.057829+0000","last_undegraded":"2023-12-26T21:37:50.057829+0000","last_fullsized":"2023-12-26T21:37:50.057829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:20.144180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029617+0000","last_change":"2023-12-26T21:37:50.029617+0000","last_active":"2023-12-26T21:37:50.029617+0000","last_peered":"2023-12-26T21:37:50.029617+0000","last_clean":"2023-12-26T21:37:50.029617+0000","last_became_active":"2023-12-26T21:37:50.029429+0000","last_became_peered":"2023-12-26T21:37:50.029429+0000","last_unstale":"2023-12-26T21:37:50.029617+0000","last_undegraded":"2023-12-26T21:37:50.029617+0000","last_fullsized":"2023-12-26T21:37:50.029617+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:01.590388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.059472+0000","last_change":"2023-12-26T21:37:49.055122+0000","last_active":"2023-12-26T21:37:49.059472+0000","last_peered":"2023-12-26T21:37:49.059472+0000","last_clean":"2023-12-26T21:37:49.059472+0000","last_became_active":"2023-12-26T21:37:49.054791+0000","last_became_peered":"2023-12-26T21:37:49.054791+0000","last_unstale":"2023-12-26T21:37:49.059472+0000","last_undegraded":"2023-12-26T21:37:49.059472+0000","last_fullsized":"2023-12-26T21:37:49.059472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:49:55.783980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.013164+0000","last_change":"2023-12-26T21:37:50.013164+0000","last_active":"2023-12-26T21:37:50.013164+0000","last_peered":"2023-12-26T21:37:50.013164+0000","last_clean":"2023-12-26T21:37:50.013164+0000","last_became_active":"2023-12-26T21:37:50.012793+0000","last_became_peered":"2023-12-26T21:37:50.012793+0000","last_unstale":"2023-12-26T21:37:50.013164+0000","last_undegraded":"2023-12-26T21:37:50.013164+0000","last_fullsized":"2023-12-26T21:37:50.013164+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:12:24.533764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.064861+0000","last_change":"2023-12-26T21:37:49.060744+0000","last_active":"2023-12-26T21:37:49.064861+0000","last_peered":"2023-12-26T21:37:49.064861+0000","last_clean":"2023-12-26T21:37:49.064861+0000","last_became_active":"2023-12-26T21:37:49.060388+0000","last_became_peered":"2023-12-26T21:37:49.060388+0000","last_unstale":"2023-12-26T21:37:49.064861+0000","last_undegraded":"2023-12-26T21:37:49.064861+0000","last_fullsized":"2023-12-26T21:37:49.064861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:37.235627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022415+0000","last_change":"2023-12-26T21:37:50.017245+0000","last_active":"2023-12-26T21:37:50.022415+0000","last_peered":"2023-12-26T21:37:50.022415+0000","last_clean":"2023-12-26T21:37:50.022415+0000","last_became_active":"2023-12-26T21:37:50.016887+0000","last_became_peered":"2023-12-26T21:37:50.016887+0000","last_unstale":"2023-12-26T21:37:50.022415+0000","last_undegraded":"2023-12-26T21:37:50.022415+0000","last_fullsized":"2023-12-26T21:37:50.022415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:12:20.128839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060866+0000","last_change":"2023-12-26T21:37:49.044678+0000","last_active":"2023-12-26T21:37:50.060866+0000","last_peered":"2023-12-26T21:37:50.060866+0000","last_clean":"2023-12-26T21:37:50.060866+0000","last_became_active":"2023-12-26T21:37:49.044404+0000","last_became_peered":"2023-12-26T21:37:49.044404+0000","last_unstale":"2023-12-26T21:37:50.060866+0000","last_undegraded":"2023-12-26T21:37:50.060866+0000","last_fullsized":"2023-12-26T21:37:50.060866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:10.777671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029541+0000","last_change":"2023-12-26T21:37:50.025483+0000","last_active":"2023-12-26T21:37:50.029541+0000","last_peered":"2023-12-26T21:37:50.029541+0000","last_clean":"2023-12-26T21:37:50.029541+0000","last_became_active":"2023-12-26T21:37:50.025172+0000","last_became_peered":"2023-12-26T21:37:50.025172+0000","last_unstale":"2023-12-26T21:37:50.029541+0000","last_undegraded":"2023-12-26T21:37:50.029541+0000","last_fullsized":"2023-12-26T21:37:50.029541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:53:46.215076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.059940+0000","last_change":"2023-12-26T21:37:49.057012+0000","last_active":"2023-12-26T21:37:50.059940+0000","last_peered":"2023-12-26T21:37:50.059940+0000","last_clean":"2023-12-26T21:37:50.059940+0000","last_became_active":"2023-12-26T21:37:49.056691+0000","last_became_peered":"2023-12-26T21:37:49.056691+0000","last_unstale":"2023-12-26T21:37:50.059940+0000","last_undegraded":"2023-12-26T21:37:50.059940+0000","last_fullsized":"2023-12-26T21:37:50.059940+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:20:15.766845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.009538+0000","last_change":"2023-12-26T21:37:50.005001+0000","last_active":"2023-12-26T21:37:50.009538+0000","last_peered":"2023-12-26T21:37:50.009538+0000","last_clean":"2023-12-26T21:37:50.009538+0000","last_became_active":"2023-12-26T21:37:50.004703+0000","last_became_peered":"2023-12-26T21:37:50.004703+0000","last_unstale":"2023-12-26T21:37:50.009538+0000","last_undegraded":"2023-12-26T21:37:50.009538+0000","last_fullsized":"2023-12-26T21:37:50.009538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:34:04.184026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.055641+0000","last_change":"2023-12-26T21:37:49.060695+0000","last_active":"2023-12-26T21:37:50.055641+0000","last_peered":"2023-12-26T21:37:50.055641+0000","last_clean":"2023-12-26T21:37:50.055641+0000","last_became_active":"2023-12-26T21:37:49.058055+0000","last_became_peered":"2023-12-26T21:37:49.058055+0000","last_unstale":"2023-12-26T21:37:50.055641+0000","last_undegraded":"2023-12-26T21:37:50.055641+0000","last_fullsized":"2023-12-26T21:37:50.055641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:32:11.434034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.020217+0000","last_change":"2023-12-26T21:37:50.010547+0000","last_active":"2023-12-26T21:37:50.020217+0000","last_peered":"2023-12-26T21:37:50.020217+0000","last_clean":"2023-12-26T21:37:50.020217+0000","last_became_active":"2023-12-26T21:37:50.010248+0000","last_became_peered":"2023-12-26T21:37:50.010248+0000","last_unstale":"2023-12-26T21:37:50.020217+0000","last_undegraded":"2023-12-26T21:37:50.020217+0000","last_fullsized":"2023-12-26T21:37:50.020217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:08:01.371579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.054817+0000","last_change":"2023-12-26T21:37:49.057397+0000","last_active":"2023-12-26T21:37:50.054817+0000","last_peered":"2023-12-26T21:37:50.054817+0000","last_clean":"2023-12-26T21:37:50.054817+0000","last_became_active":"2023-12-26T21:37:49.055492+0000","last_became_peered":"2023-12-26T21:37:49.055492+0000","last_unstale":"2023-12-26T21:37:50.054817+0000","last_undegraded":"2023-12-26T21:37:50.054817+0000","last_fullsized":"2023-12-26T21:37:50.054817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:25.672661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015786+0000","last_change":"2023-12-26T21:37:50.011523+0000","last_active":"2023-12-26T21:37:50.015786+0000","last_peered":"2023-12-26T21:37:50.015786+0000","last_clean":"2023-12-26T21:37:50.015786+0000","last_became_active":"2023-12-26T21:37:50.011228+0000","last_became_peered":"2023-12-26T21:37:50.011228+0000","last_unstale":"2023-12-26T21:37:50.015786+0000","last_undegraded":"2023-12-26T21:37:50.015786+0000","last_fullsized":"2023-12-26T21:37:50.015786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:49:56.791142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.001367+0000","last_change":"2023-12-26T21:37:49.056890+0000","last_active":"2023-12-26T21:37:50.001367+0000","last_peered":"2023-12-26T21:37:50.001367+0000","last_clean":"2023-12-26T21:37:50.001367+0000","last_became_active":"2023-12-26T21:37:49.056617+0000","last_became_peered":"2023-12-26T21:37:49.056617+0000","last_unstale":"2023-12-26T21:37:50.001367+0000","last_undegraded":"2023-12-26T21:37:50.001367+0000","last_fullsized":"2023-12-26T21:37:50.001367+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:35.774200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.030122+0000","last_change":"2023-12-26T21:37:50.028042+0000","last_active":"2023-12-26T21:37:50.030122+0000","last_peered":"2023-12-26T21:37:50.030122+0000","last_clean":"2023-12-26T21:37:50.030122+0000","last_became_active":"2023-12-26T21:37:50.027724+0000","last_became_peered":"2023-12-26T21:37:50.027724+0000","last_unstale":"2023-12-26T21:37:50.030122+0000","last_undegraded":"2023-12-26T21:37:50.030122+0000","last_fullsized":"2023-12-26T21:37:50.030122+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:56:22.944966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.060611+0000","last_change":"2023-12-26T21:37:49.061864+0000","last_active":"2023-12-26T21:37:50.060611+0000","last_peered":"2023-12-26T21:37:50.060611+0000","last_clean":"2023-12-26T21:37:50.060611+0000","last_became_active":"2023-12-26T21:37:49.061569+0000","last_became_peered":"2023-12-26T21:37:49.061569+0000","last_unstale":"2023-12-26T21:37:50.060611+0000","last_undegraded":"2023-12-26T21:37:50.060611+0000","last_fullsized":"2023-12-26T21:37:50.060611+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:42:46.794558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.021019+0000","last_change":"2023-12-26T21:37:50.015748+0000","last_active":"2023-12-26T21:37:50.021019+0000","last_peered":"2023-12-26T21:37:50.021019+0000","last_clean":"2023-12-26T21:37:50.021019+0000","last_became_active":"2023-12-26T21:37:50.015470+0000","last_became_peered":"2023-12-26T21:37:50.015470+0000","last_unstale":"2023-12-26T21:37:50.021019+0000","last_undegraded":"2023-12-26T21:37:50.021019+0000","last_fullsized":"2023-12-26T21:37:50.021019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:33.796968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.063588+0000","last_change":"2023-12-26T21:37:49.055204+0000","last_active":"2023-12-26T21:37:49.063588+0000","last_peered":"2023-12-26T21:37:49.063588+0000","last_clean":"2023-12-26T21:37:49.063588+0000","last_became_active":"2023-12-26T21:37:49.054942+0000","last_became_peered":"2023-12-26T21:37:49.054942+0000","last_unstale":"2023-12-26T21:37:49.063588+0000","last_undegraded":"2023-12-26T21:37:49.063588+0000","last_fullsized":"2023-12-26T21:37:49.063588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:42.166494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029958+0000","last_change":"2023-12-26T21:37:50.026788+0000","last_active":"2023-12-26T21:37:50.029958+0000","last_peered":"2023-12-26T21:37:50.029958+0000","last_clean":"2023-12-26T21:37:50.029958+0000","last_became_active":"2023-12-26T21:37:50.026521+0000","last_became_peered":"2023-12-26T21:37:50.026521+0000","last_unstale":"2023-12-26T21:37:50.029958+0000","last_undegraded":"2023-12-26T21:37:50.029958+0000","last_fullsized":"2023-12-26T21:37:50.029958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:20:26.434022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.002565+0000","last_change":"2023-12-26T21:37:49.062160+0000","last_active":"2023-12-26T21:37:50.002565+0000","last_peered":"2023-12-26T21:37:50.002565+0000","last_clean":"2023-12-26T21:37:50.002565+0000","last_became_active":"2023-12-26T21:37:49.061823+0000","last_became_peered":"2023-12-26T21:37:49.061823+0000","last_unstale":"2023-12-26T21:37:50.002565+0000","last_undegraded":"2023-12-26T21:37:50.002565+0000","last_fullsized":"2023-12-26T21:37:50.002565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:32:11.550625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.008639+0000","last_change":"2023-12-26T21:37:50.005148+0000","last_active":"2023-12-26T21:37:50.008639+0000","last_peered":"2023-12-26T21:37:50.008639+0000","last_clean":"2023-12-26T21:37:50.008639+0000","last_became_active":"2023-12-26T21:37:50.004668+0000","last_became_peered":"2023-12-26T21:37:50.004668+0000","last_unstale":"2023-12-26T21:37:50.008639+0000","last_undegraded":"2023-12-26T21:37:50.008639+0000","last_fullsized":"2023-12-26T21:37:50.008639+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:34:20.086714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.006407+0000","last_change":"2023-12-26T21:37:49.052833+0000","last_active":"2023-12-26T21:37:50.006407+0000","last_peered":"2023-12-26T21:37:50.006407+0000","last_clean":"2023-12-26T21:37:50.006407+0000","last_became_active":"2023-12-26T21:37:49.050159+0000","last_became_peered":"2023-12-26T21:37:49.050159+0000","last_unstale":"2023-12-26T21:37:50.006407+0000","last_undegraded":"2023-12-26T21:37:50.006407+0000","last_fullsized":"2023-12-26T21:37:50.006407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:41:44.910782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.015036+0000","last_change":"2023-12-26T21:37:50.011396+0000","last_active":"2023-12-26T21:37:50.015036+0000","last_peered":"2023-12-26T21:37:50.015036+0000","last_clean":"2023-12-26T21:37:50.015036+0000","last_became_active":"2023-12-26T21:37:50.011065+0000","last_became_peered":"2023-12-26T21:37:50.011065+0000","last_unstale":"2023-12-26T21:37:50.015036+0000","last_undegraded":"2023-12-26T21:37:50.015036+0000","last_fullsized":"2023-12-26T21:37:50.015036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:38.242789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.055549+0000","last_change":"2023-12-26T21:37:49.060258+0000","last_active":"2023-12-26T21:37:50.055549+0000","last_peered":"2023-12-26T21:37:50.055549+0000","last_clean":"2023-12-26T21:37:50.055549+0000","last_became_active":"2023-12-26T21:37:49.059932+0000","last_became_peered":"2023-12-26T21:37:49.059932+0000","last_unstale":"2023-12-26T21:37:50.055549+0000","last_undegraded":"2023-12-26T21:37:50.055549+0000","last_fullsized":"2023-12-26T21:37:50.055549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:43.540873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022445+0000","last_change":"2023-12-26T21:37:50.016905+0000","last_active":"2023-12-26T21:37:50.022445+0000","last_peered":"2023-12-26T21:37:50.022445+0000","last_clean":"2023-12-26T21:37:50.022445+0000","last_became_active":"2023-12-26T21:37:50.016618+0000","last_became_peered":"2023-12-26T21:37:50.016618+0000","last_unstale":"2023-12-26T21:37:50.022445+0000","last_undegraded":"2023-12-26T21:37:50.022445+0000","last_fullsized":"2023-12-26T21:37:50.022445+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:29:35.827917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.007099+0000","last_change":"2023-12-26T21:37:49.060728+0000","last_active":"2023-12-26T21:37:50.007099+0000","last_peered":"2023-12-26T21:37:50.007099+0000","last_clean":"2023-12-26T21:37:50.007099+0000","last_became_active":"2023-12-26T21:37:49.058273+0000","last_became_peered":"2023-12-26T21:37:49.058273+0000","last_unstale":"2023-12-26T21:37:50.007099+0000","last_undegraded":"2023-12-26T21:37:50.007099+0000","last_fullsized":"2023-12-26T21:37:50.007099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:35.774200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.028678+0000","last_change":"2023-12-26T21:37:50.021450+0000","last_active":"2023-12-26T21:37:50.028678+0000","last_peered":"2023-12-26T21:37:50.028678+0000","last_clean":"2023-12-26T21:37:50.028678+0000","last_became_active":"2023-12-26T21:37:50.021194+0000","last_became_peered":"2023-12-26T21:37:50.021194+0000","last_unstale":"2023-12-26T21:37:50.028678+0000","last_undegraded":"2023-12-26T21:37:50.028678+0000","last_fullsized":"2023-12-26T21:37:50.028678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:43.173656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.013888+0000","last_change":"2023-12-26T21:37:49.058279+0000","last_active":"2023-12-26T21:37:50.013888+0000","last_peered":"2023-12-26T21:37:50.013888+0000","last_clean":"2023-12-26T21:37:50.013888+0000","last_became_active":"2023-12-26T21:37:49.057964+0000","last_became_peered":"2023-12-26T21:37:49.057964+0000","last_unstale":"2023-12-26T21:37:50.013888+0000","last_undegraded":"2023-12-26T21:37:50.013888+0000","last_fullsized":"2023-12-26T21:37:50.013888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:56:51.912847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.018954+0000","last_change":"2023-12-26T21:37:50.011822+0000","last_active":"2023-12-26T21:37:50.018954+0000","last_peered":"2023-12-26T21:37:50.018954+0000","last_clean":"2023-12-26T21:37:50.018954+0000","last_became_active":"2023-12-26T21:37:50.011581+0000","last_became_peered":"2023-12-26T21:37:50.011581+0000","last_unstale":"2023-12-26T21:37:50.018954+0000","last_undegraded":"2023-12-26T21:37:50.018954+0000","last_fullsized":"2023-12-26T21:37:50.018954+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:41:45.917944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.012679+0000","last_change":"2023-12-26T21:37:49.062314+0000","last_active":"2023-12-26T21:37:50.012679+0000","last_peered":"2023-12-26T21:37:50.012679+0000","last_clean":"2023-12-26T21:37:50.012679+0000","last_became_active":"2023-12-26T21:37:49.062067+0000","last_became_peered":"2023-12-26T21:37:49.062067+0000","last_unstale":"2023-12-26T21:37:50.012679+0000","last_undegraded":"2023-12-26T21:37:50.012679+0000","last_fullsized":"2023-12-26T21:37:50.012679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:34:03.176864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.029562+0000","last_change":"2023-12-26T21:37:50.029562+0000","last_active":"2023-12-26T21:37:50.029562+0000","last_peered":"2023-12-26T21:37:50.029562+0000","last_clean":"2023-12-26T21:37:50.029562+0000","last_became_active":"2023-12-26T21:37:50.029381+0000","last_became_peered":"2023-12-26T21:37:50.029381+0000","last_unstale":"2023-12-26T21:37:50.029562+0000","last_undegraded":"2023-12-26T21:37:50.029562+0000","last_fullsized":"2023-12-26T21:37:50.029562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:20:03.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.012601+0000","last_change":"2023-12-26T21:37:49.054282+0000","last_active":"2023-12-26T21:37:50.012601+0000","last_peered":"2023-12-26T21:37:50.012601+0000","last_clean":"2023-12-26T21:37:50.012601+0000","last_became_active":"2023-12-26T21:37:49.053589+0000","last_became_peered":"2023-12-26T21:37:49.053589+0000","last_unstale":"2023-12-26T21:37:50.012601+0000","last_undegraded":"2023-12-26T21:37:50.012601+0000","last_fullsized":"2023-12-26T21:37:50.012601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:25:09.234601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.026982+0000","last_change":"2023-12-26T21:37:50.026982+0000","last_active":"2023-12-26T21:37:50.026982+0000","last_peered":"2023-12-26T21:37:50.026982+0000","last_clean":"2023-12-26T21:37:50.026982+0000","last_became_active":"2023-12-26T21:37:50.026522+0000","last_became_peered":"2023-12-26T21:37:50.026522+0000","last_unstale":"2023-12-26T21:37:50.026982+0000","last_undegraded":"2023-12-26T21:37:50.026982+0000","last_fullsized":"2023-12-26T21:37:50.026982+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:55:52.330005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.064248+0000","last_change":"2023-12-26T21:37:49.057261+0000","last_active":"2023-12-26T21:37:49.064248+0000","last_peered":"2023-12-26T21:37:49.064248+0000","last_clean":"2023-12-26T21:37:49.064248+0000","last_became_active":"2023-12-26T21:37:49.054556+0000","last_became_peered":"2023-12-26T21:37:49.054556+0000","last_unstale":"2023-12-26T21:37:49.064248+0000","last_undegraded":"2023-12-26T21:37:49.064248+0000","last_fullsized":"2023-12-26T21:37:49.064248+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:44:12.532434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.022396+0000","last_change":"2023-12-26T21:37:50.015679+0000","last_active":"2023-12-26T21:37:50.022396+0000","last_peered":"2023-12-26T21:37:50.022396+0000","last_clean":"2023-12-26T21:37:50.022396+0000","last_became_active":"2023-12-26T21:37:50.015330+0000","last_became_peered":"2023-12-26T21:37:50.015330+0000","last_unstale":"2023-12-26T21:37:50.022396+0000","last_undegraded":"2023-12-26T21:37:50.022396+0000","last_fullsized":"2023-12-26T21:37:50.022396+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:48:46.804827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-26T21:37:49.054410+0000","last_change":"2023-12-26T21:37:49.051331+0000","last_active":"2023-12-26T21:37:49.054410+0000","last_peered":"2023-12-26T21:37:49.054410+0000","last_clean":"2023-12-26T21:37:49.054410+0000","last_became_active":"2023-12-26T21:37:49.050468+0000","last_became_peered":"2023-12-26T21:37:49.050468+0000","last_unstale":"2023-12-26T21:37:49.054410+0000","last_undegraded":"2023-12-26T21:37:49.054410+0000","last_fullsized":"2023-12-26T21:37:49.054410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:42.166494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.018737+0000","last_change":"2023-12-26T21:37:50.011289+0000","last_active":"2023-12-26T21:37:50.018737+0000","last_peered":"2023-12-26T21:37:50.018737+0000","last_clean":"2023-12-26T21:37:50.018737+0000","last_became_active":"2023-12-26T21:37:50.010959+0000","last_became_peered":"2023-12-26T21:37:50.010959+0000","last_unstale":"2023-12-26T21:37:50.018737+0000","last_undegraded":"2023-12-26T21:37:50.018737+0000","last_fullsized":"2023-12-26T21:37:50.018737+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:49.031202+0000","last_clean_scrub_stamp":"2023-12-26T21:37:49.031202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:31:29.912297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.009851+0000","last_change":"2023-12-26T21:37:49.050353+0000","last_active":"2023-12-26T21:37:50.009851+0000","last_peered":"2023-12-26T21:37:50.009851+0000","last_clean":"2023-12-26T21:37:50.009851+0000","last_became_active":"2023-12-26T21:37:49.050075+0000","last_became_peered":"2023-12-26T21:37:49.050075+0000","last_unstale":"2023-12-26T21:37:50.009851+0000","last_undegraded":"2023-12-26T21:37:50.009851+0000","last_fullsized":"2023-12-26T21:37:50.009851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:37:48.024040+0000","last_clean_scrub_stamp":"2023-12-26T21:37:48.024040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:38:15.966274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":172,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-26T21:37:50.003040+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:37:50.003040+0000","last_peered":"2023-12-26T21:37:50.003040+0000","last_clean":"2023-12-26T21:37:50.003040+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:37:50.003040+0000","last_undegraded":"2023-12-26T21:37:50.003040+0000","last_fullsized":"2023-12-26T21:37:50.003040+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":146557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.91700000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.622},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.80700000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.872},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.84099999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.751}]},{"osd":7,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76500000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073588,"statfs":{"total":96636764160,"available":96331354112,"internally_reserved":0,"allocated":987136,"data_stored":742231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.59799999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.66000000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78600000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.627},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.86699999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.89100000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:37:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0049999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297672,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074168,"statfs":{"total":96636764160,"available":96331948032,"internally_reserved":0,"allocated":393216,"data_stored":146579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.50600000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.54600000000000004}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":0.59599999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.66500000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.74399999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542164,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":145055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.436},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.33900000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.374}]},{"osd":4,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70099999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.628},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.74299999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":145043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.61899999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.78100000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.81399999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.68000000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":145111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.3150000000000004,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.6479999999999997,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.39300000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.2610000000000001,"5min":4.2610000000000001,"15min":4.2610000000000001},"last":0.375},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.63300000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.81899999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":4.3650000000000002,"5min":4.3650000000000002,"15min":4.3650000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.5519999999999996,"5min":4.5519999999999996,"15min":4.5519999999999996},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.69099999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:37:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":4.6070000000000002,"5min":4.6070000000000002,"15min":4.6070000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":4.5069999999999997,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.58999999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073600,"statfs":{"total":96636764160,"available":96331366400,"internally_reserved":0,"allocated":974848,"data_stored":735459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.378},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.6600000000000001,"5min":4.6600000000000001,"15min":4.6600000000000001},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.3639999999999999,"5min":4.3639999999999999,"15min":4.3639999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":4.3259999999999996,"5min":4.3259999999999996,"15min":4.3259999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.548,"5min":4.548,"15min":4.548},"last":0.437}]},{"osd":4,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63400000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":152057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.496},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.62}]},{"osd":5,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:37:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:37:54.870 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T21:37:54.870 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T21:37:55.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.102+0000 7f64355ff700 1 Processor -- start 2023-12-26T21:37:55.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.102+0000 7f64355ff700 1 -- start start 2023-12-26T21:37:55.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.103+0000 7f64355ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157420 0x7f6430159c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.103+0000 7f64355ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f643015a1b0 0x7f64300cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.103+0000 7f64355ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64300caed0 con 0x7f6430157050 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.103+0000 7f64355ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64300c9520 con 0x7f643015a1b0 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.103+0000 7f64355ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64300ca8e0 con 0x7f6430157420 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.103+0000 7f642f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f643015a1b0 0x7f64300cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.103+0000 7f642f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f643015a1b0 0x7f64300cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58630/0 (socket says 172.21.15.131:58630) 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.103+0000 7f642f7fe700 1 -- 172.21.15.131:0/3998249425 learned_addr learned my addr 172.21.15.131:0/3998249425 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.103+0000 7f642e7fc700 1 --2- 172.21.15.131:0/3998249425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157420 0x7f6430159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.104+0000 7f642f7fe700 1 -- 172.21.15.131:0/3998249425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157420 msgr2=0x7f6430159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.104+0000 7f642f7fe700 1 --2- 172.21.15.131:0/3998249425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157420 0x7f6430159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.104+0000 7f642f7fe700 1 -- 172.21.15.131:0/3998249425 >> v1:172.21.15.190:6789/0 conn(0x7f6430157050 legacy=0x7f64300686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.104+0000 7f642f7fe700 1 -- 172.21.15.131:0/3998249425 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643015a6c0 con 0x7f643015a1b0 2023-12-26T21:37:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.104+0000 7f642f7fe700 1 --2- 172.21.15.131:0/3998249425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643015a1b0 0x7f64300cb360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6428009f20 tx=0x7f6428009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30aa88815560a56 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:55.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.104+0000 7f642dffb700 1 -- 172.21.15.131:0/3998249425 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642800e070 con 0x7f643015a1b0 2023-12-26T21:37:55.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.104+0000 7f642dffb700 1 -- 172.21.15.131:0/3998249425 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642800b9d0 con 0x7f643015a1b0 2023-12-26T21:37:55.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.104+0000 7f642dffb700 1 -- 172.21.15.131:0/3998249425 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280092e0 con 0x7f643015a1b0 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.105+0000 7f64355ff700 1 -- 172.21.15.131:0/3998249425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643015a1b0 msgr2=0x7f64300cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.105+0000 7f64355ff700 1 --2- 172.21.15.131:0/3998249425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643015a1b0 0x7f64300cb360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6428009f20 tx=0x7f6428009840 comp rx=0 tx=0).stop 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.105+0000 7f64355ff700 1 -- 172.21.15.131:0/3998249425 shutdown_connections 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.105+0000 7f64355ff700 1 --2- 172.21.15.131:0/3998249425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f643015a1b0 0x7f64300cb360 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.105+0000 7f64355ff700 1 --2- 172.21.15.131:0/3998249425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157420 0x7f6430159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.105+0000 7f64355ff700 1 -- 172.21.15.131:0/3998249425 >> 172.21.15.131:0/3998249425 conn(0x7f643006dc80 msgr2=0x7f64300675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.105+0000 7f64355ff700 1 -- 172.21.15.131:0/3998249425 shutdown_connections 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.105+0000 7f64355ff700 1 -- 172.21.15.131:0/3998249425 wait complete. 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.106+0000 7f64355ff700 1 Processor -- start 2023-12-26T21:37:55.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.107+0000 7f64355ff700 1 -- start start 2023-12-26T21:37:55.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.107+0000 7f64355ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157050 0x7f6430156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.107+0000 7f64355ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6430157420 0x7f64300b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.107+0000 7f64355ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643015a1b0 0x7f643015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.107+0000 7f64355ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64300c3530 con 0x7f6430157420 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.107+0000 7f64355ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6430156b40 con 0x7f643015a1b0 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.107+0000 7f64355ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64300c2920 con 0x7f6430157050 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6430157420 0x7f64300b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6430157420 0x7f64300b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58634/0 (socket says 172.21.15.131:58634) 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642e7fc700 1 -- 172.21.15.131:0/1415517088 learned_addr learned my addr 172.21.15.131:0/1415517088 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642f7fe700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643015a1b0 0x7f643015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642effd700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157050 0x7f6430156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642e7fc700 1 -- 172.21.15.131:0/1415517088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157050 msgr2=0x7f6430156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642e7fc700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157050 0x7f6430156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642e7fc700 1 -- 172.21.15.131:0/1415517088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643015a1b0 msgr2=0x7f643015bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642e7fc700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643015a1b0 0x7f643015bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642e7fc700 1 -- 172.21.15.131:0/1415517088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428009ae0 con 0x7f6430157420 2023-12-26T21:37:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.108+0000 7f642e7fc700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6430157420 0x7f64300b3b60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f642400e900 tx=0x7f64240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.109+0000 7f6416ffd700 1 -- 172.21.15.131:0/1415517088 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424013680 con 0x7f6430157420 2023-12-26T21:37:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.109+0000 7f6416ffd700 1 -- 172.21.15.131:0/1415517088 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64240137e0 con 0x7f6430157420 2023-12-26T21:37:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.109+0000 7f6416ffd700 1 -- 172.21.15.131:0/1415517088 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424013b80 con 0x7f6430157420 2023-12-26T21:37:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.109+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64300c2c00 con 0x7f6430157420 2023-12-26T21:37:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.109+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64300c31c0 con 0x7f6430157420 2023-12-26T21:37:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.110+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64300657e0 con 0x7f6430157420 2023-12-26T21:37:55.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.111+0000 7f6416ffd700 1 -- 172.21.15.131:0/1415517088 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f64240123d0 con 0x7f6430157420 2023-12-26T21:37:55.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.112+0000 7f6416ffd700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f641c066f30 0x7f641c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.112+0000 7f6416ffd700 1 -- 172.21.15.131:0/1415517088 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6424084620 con 0x7f6430157420 2023-12-26T21:37:55.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.112+0000 7f642effd700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f641c066f30 0x7f641c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.112+0000 7f642effd700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f641c066f30 0x7f641c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f64200039e0 tx=0x7f6420007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:55.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.116+0000 7f6416ffd700 1 -- 172.21.15.131:0/1415517088 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64240517f0 con 0x7f6430157420 2023-12-26T21:37:55.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.388+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f64300c3670 con 0x7f6430157420 2023-12-26T21:37:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.394+0000 7f6416ffd700 1 -- 172.21.15.131:0/1415517088 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6424054e10 con 0x7f6430157420 2023-12-26T21:37:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f641c066f30 msgr2=0x7f641c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f641c066f30 0x7f641c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f64200039e0 tx=0x7f6420007ec0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6430157420 msgr2=0x7f64300b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6430157420 0x7f64300b3b60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f642400e900 tx=0x7f64240091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 shutdown_connections 2023-12-26T21:37:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643015a1b0 0x7f643015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f641c066f30 0x7f641c0693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6430157050 0x7f6430156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 --2- 172.21.15.131:0/1415517088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6430157420 0x7f64300b3b60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 >> 172.21.15.131:0/1415517088 conn(0x7f643006dc80 msgr2=0x7f64300cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.396+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 shutdown_connections 2023-12-26T21:37:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.397+0000 7f64355ff700 1 -- 172.21.15.131:0/1415517088 wait complete. 2023-12-26T21:37:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T21:37:55.414 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:37:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.602+0000 7f2eeb9ff700 1 Processor -- start 2023-12-26T21:37:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.602+0000 7f2eeb9ff700 1 -- start start 2023-12-26T21:37:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.603+0000 7f2eeb9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ee413c510 0x7f2ee40c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.603+0000 7f2eeb9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee406b8c0 0x7f2ee406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.603+0000 7f2eeb9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40686c0 con 0x7f2ee40cc100 2023-12-26T21:37:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.603+0000 7f2eeb9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee4069520 con 0x7f2ee413c510 2023-12-26T21:37:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.603+0000 7f2eeb9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee406c250 con 0x7f2ee406b8c0 2023-12-26T21:37:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.603+0000 7f2ee979b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ee413c510 0x7f2ee40c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.603+0000 7f2ee979b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ee413c510 0x7f2ee40c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58642/0 (socket says 172.21.15.131:58642) 2023-12-26T21:37:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.603+0000 7f2ee979b700 1 -- 172.21.15.131:0/1355880815 learned_addr learned my addr 172.21.15.131:0/1355880815 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.604+0000 7f2ee8f9a700 1 --2- 172.21.15.131:0/1355880815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee406b8c0 0x7f2ee406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.604+0000 7f2ee979b700 1 -- 172.21.15.131:0/1355880815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee406b8c0 msgr2=0x7f2ee406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.604+0000 7f2ee979b700 1 --2- 172.21.15.131:0/1355880815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee406b8c0 0x7f2ee406bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.604+0000 7f2ee979b700 1 -- 172.21.15.131:0/1355880815 >> v1:172.21.15.190:6789/0 conn(0x7f2ee40cc100 legacy=0x7f2ee40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.604+0000 7f2ee979b700 1 -- 172.21.15.131:0/1355880815 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4068910 con 0x7f2ee413c510 2023-12-26T21:37:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.604+0000 7f2ee979b700 1 --2- 172.21.15.131:0/1355880815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 0x7f2ee40c8a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009990 tx=0x7f2ee0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da635eb7fa5756a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.604+0000 7f2edbfff700 1 -- 172.21.15.131:0/1355880815 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0005c10 con 0x7f2ee413c510 2023-12-26T21:37:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.604+0000 7f2edbfff700 1 -- 172.21.15.131:0/1355880815 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0005d70 con 0x7f2ee413c510 2023-12-26T21:37:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.604+0000 7f2edbfff700 1 -- 172.21.15.131:0/1355880815 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000b880 con 0x7f2ee413c510 2023-12-26T21:37:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.605+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/1355880815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 msgr2=0x7f2ee40c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.605+0000 7f2eeb9ff700 1 --2- 172.21.15.131:0/1355880815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 0x7f2ee40c8a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009990 tx=0x7f2ee0009220 comp rx=0 tx=0).stop 2023-12-26T21:37:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.605+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/1355880815 shutdown_connections 2023-12-26T21:37:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.605+0000 7f2eeb9ff700 1 --2- 172.21.15.131:0/1355880815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 0x7f2ee40c8a20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.605+0000 7f2eeb9ff700 1 --2- 172.21.15.131:0/1355880815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee406b8c0 0x7f2ee406bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.605+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/1355880815 >> 172.21.15.131:0/1355880815 conn(0x7f2ee40ce3e0 msgr2=0x7f2ee40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.605+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/1355880815 shutdown_connections 2023-12-26T21:37:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.605+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/1355880815 wait complete. 2023-12-26T21:37:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.606+0000 7f2eeb9ff700 1 Processor -- start 2023-12-26T21:37:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.607+0000 7f2eeb9ff700 1 -- start start 2023-12-26T21:37:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.607+0000 7f2eeb9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee406b8c0 0x7f2ee416a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.607+0000 7f2eeb9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee40cc100 0x7f2ee40b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.607+0000 7f2eeb9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 0x7f2ee4152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2eeb9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee4169f00 con 0x7f2ee413c510 2023-12-26T21:37:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2eeb9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40b3560 con 0x7f2ee406b8c0 2023-12-26T21:37:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2eeb9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40b3840 con 0x7f2ee40cc100 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee9f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 0x7f2ee4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee8f9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee40cc100 0x7f2ee40b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee9f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 0x7f2ee4152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58644/0 (socket says 172.21.15.131:58644) 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee979b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee406b8c0 0x7f2ee416a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee9f9c700 1 -- 172.21.15.131:0/4018171848 learned_addr learned my addr 172.21.15.131:0/4018171848 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee9f9c700 1 -- 172.21.15.131:0/4018171848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee40cc100 msgr2=0x7f2ee40b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee9f9c700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee40cc100 0x7f2ee40b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee9f9c700 1 -- 172.21.15.131:0/4018171848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee406b8c0 msgr2=0x7f2ee416a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee9f9c700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee406b8c0 0x7f2ee416a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.608+0000 7f2ee9f9c700 1 -- 172.21.15.131:0/4018171848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0009480 con 0x7f2ee413c510 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.609+0000 7f2ee9f9c700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 0x7f2ee4152740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00c8e0 tx=0x7f2edc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.609+0000 7f2ed97fa700 1 -- 172.21.15.131:0/4018171848 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc012630 con 0x7f2ee413c510 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.609+0000 7f2ed97fa700 1 -- 172.21.15.131:0/4018171848 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc012790 con 0x7f2ee413c510 2023-12-26T21:37:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.609+0000 7f2ed97fa700 1 -- 172.21.15.131:0/4018171848 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc012b50 con 0x7f2ee413c510 2023-12-26T21:37:55.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.609+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee4169b90 con 0x7f2ee413c510 2023-12-26T21:37:55.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.609+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee4169850 con 0x7f2ee413c510 2023-12-26T21:37:55.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.610+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee40657e0 con 0x7f2ee413c510 2023-12-26T21:37:55.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.611+0000 7f2ed97fa700 1 -- 172.21.15.131:0/4018171848 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2edc01a3a0 con 0x7f2ee413c510 2023-12-26T21:37:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.612+0000 7f2ed97fa700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2ec8066f30 0x7f2ec80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.612+0000 7f2ed97fa700 1 -- 172.21.15.131:0/4018171848 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2edc084710 con 0x7f2ee413c510 2023-12-26T21:37:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.612+0000 7f2ee979b700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2ec8066f30 0x7f2ec80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.612+0000 7f2ee979b700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2ec8066f30 0x7f2ec80693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40b41a0 tx=0x7f2ee0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.615+0000 7f2ed97fa700 1 -- 172.21.15.131:0/4018171848 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2edc055370 con 0x7f2ee413c510 2023-12-26T21:37:55.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.855+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ee4068bf0 con 0x7f2ee413c510 2023-12-26T21:37:55.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.856+0000 7f2ed97fa700 1 -- 172.21.15.131:0/4018171848 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f2edc054f00 con 0x7f2ee413c510 2023-12-26T21:37:55.856 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:37:55.857 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1410949426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1410949426},{"type":"v1","addr":"172.21.15.190:6835","nonce":1410949426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2210886653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2210886653},{"type":"v1","addr":"172.21.15.131:6836","nonce":2210886653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4878,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/241955638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":241955638},{"type":"v1","addr":"172.21.15.131:6837","nonce":241955638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:37:49.516027+0000","modified":"2023-12-26T21:37:53.065572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6833/3890149779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3890149779},{"type":"v1","addr":"172.21.15.190:6833","nonce":3890149779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2ec8066f30 msgr2=0x7f2ec80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2ec8066f30 0x7f2ec80693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40b41a0 tx=0x7f2ee0016000 comp rx=0 tx=0).stop 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 msgr2=0x7f2ee4152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 0x7f2ee4152740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00c8e0 tx=0x7f2edc001710 comp rx=0 tx=0).stop 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 shutdown_connections 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ee413c510 0x7f2ee4152740 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee406b8c0 0x7f2ee416a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2ec8066f30 0x7f2ec80693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 --2- 172.21.15.131:0/4018171848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ee40cc100 0x7f2ee40b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 >> 172.21.15.131:0/4018171848 conn(0x7f2ee40ce3e0 msgr2=0x7f2ee40c3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.858+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 shutdown_connections 2023-12-26T21:37:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:55.859+0000 7f2eeb9ff700 1 -- 172.21.15.131:0/4018171848 wait complete. 2023-12-26T21:37:55.861 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2023-12-26T21:37:56.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T21:37:49.516027+0000', 'modified': '2023-12-26T21:37:53.065572+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4888': {'gid': 4888, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.190:6833/3890149779', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 3890149779}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 3890149779}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T21:37:56.281 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T21:37:56.281 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-26T21:37:56.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.472+0000 7f50b5dff700 1 Processor -- start 2023-12-26T21:37:56.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.472+0000 7f50b5dff700 1 -- start start 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50b5dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b013c510 0x7f50b00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50b5dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f50b006b8c0 0x7f50b006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50b5dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50b00cc100 0x7f50b00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50b5dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b006c270 con 0x7f50b006b8c0 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50b5dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b00686c0 con 0x7f50b00cc100 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50b5dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b0069520 con 0x7f50b013c510 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50a7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f50b006b8c0 0x7f50b006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50a7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f50b006b8c0 0x7f50b006bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58648/0 (socket says 172.21.15.131:58648) 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50af7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b013c510 0x7f50b00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.473+0000 7f50a7fff700 1 -- 172.21.15.131:0/3551760996 learned_addr learned my addr 172.21.15.131:0/3551760996 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50affff700 1 --2- 172.21.15.131:0/3551760996 >> v2:172.21.15.190:3300/0 conn(0x7f50b00cc100 0x7f50b00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50a7fff700 1 -- 172.21.15.131:0/3551760996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b013c510 msgr2=0x7f50b00c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3551760996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b013c510 0x7f50b00c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50a7fff700 1 -- 172.21.15.131:0/3551760996 >> v2:172.21.15.190:3300/0 conn(0x7f50b00cc100 msgr2=0x7f50b00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3551760996 >> v2:172.21.15.190:3300/0 conn(0x7f50b00cc100 0x7f50b00cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50a7fff700 1 -- 172.21.15.131:0/3551760996 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b0068910 con 0x7f50b006b8c0 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3551760996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b006b8c0 0x7f50b006bd30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f50b006f130 tx=0x7f5098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda7167052cbe457 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50aeffd700 1 -- 172.21.15.131:0/3551760996 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800b640 con 0x7f50b006b8c0 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50aeffd700 1 -- 172.21.15.131:0/3551760996 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509800b7a0 con 0x7f50b006b8c0 2023-12-26T21:37:56.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.474+0000 7f50aeffd700 1 -- 172.21.15.131:0/3551760996 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800bb60 con 0x7f50b006b8c0 2023-12-26T21:37:56.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.475+0000 7f50b5dff700 1 -- 172.21.15.131:0/3551760996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b006b8c0 msgr2=0x7f50b006bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.475+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3551760996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b006b8c0 0x7f50b006bd30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f50b006f130 tx=0x7f5098009250 comp rx=0 tx=0).stop 2023-12-26T21:37:56.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.475+0000 7f50b5dff700 1 -- 172.21.15.131:0/3551760996 shutdown_connections 2023-12-26T21:37:56.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.475+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3551760996 >> v2:172.21.15.190:3300/0 conn(0x7f50b00cc100 0x7f50b00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.475+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3551760996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b013c510 0x7f50b00c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.475+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3551760996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b006b8c0 0x7f50b006bd30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.475+0000 7f50b5dff700 1 -- 172.21.15.131:0/3551760996 >> 172.21.15.131:0/3551760996 conn(0x7f50b00ce3e0 msgr2=0x7f50b00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:56.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.475+0000 7f50b5dff700 1 -- 172.21.15.131:0/3551760996 shutdown_connections 2023-12-26T21:37:56.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.475+0000 7f50b5dff700 1 -- 172.21.15.131:0/3551760996 wait complete. 2023-12-26T21:37:56.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.476+0000 7f50b5dff700 1 Processor -- start 2023-12-26T21:37:56.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.477+0000 7f50b5dff700 1 -- start start 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.477+0000 7f50b5dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b006b8c0 0x7f50b013bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.477+0000 7f50b5dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50b00cc100 0x7f50b015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.477+0000 7f50b5dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b013c510 0x7f50b016d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.477+0000 7f50b5dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b01535d0 con 0x7f50b013c510 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.477+0000 7f50b5dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b00b17d0 con 0x7f50b00cc100 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.477+0000 7f50b5dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b0159dc0 con 0x7f50b006b8c0 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50affff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b013c510 0x7f50b016d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50affff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b013c510 0x7f50b016d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58662/0 (socket says 172.21.15.131:58662) 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50affff700 1 -- 172.21.15.131:0/3693480394 learned_addr learned my addr 172.21.15.131:0/3693480394 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50af7fe700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b006b8c0 0x7f50b013bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50b00cc100 0x7f50b015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50affff700 1 -- 172.21.15.131:0/3693480394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b006b8c0 msgr2=0x7f50b013bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50affff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b006b8c0 0x7f50b013bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50affff700 1 -- 172.21.15.131:0/3693480394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50b00cc100 msgr2=0x7f50b015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50affff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50b00cc100 0x7f50b015c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50affff700 1 -- 172.21.15.131:0/3693480394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50980094f0 con 0x7f50b013c510 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50b00cc100 0x7f50b015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.478+0000 7f50affff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b013c510 0x7f50b016d0d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009e90 tx=0x7f50a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.479+0000 7f50a77fe700 1 -- 172.21.15.131:0/3693480394 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a800cc20 con 0x7f50b013c510 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.479+0000 7f50a77fe700 1 -- 172.21.15.131:0/3693480394 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a800f040 con 0x7f50b013c510 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.479+0000 7f50a77fe700 1 -- 172.21.15.131:0/3693480394 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8014480 con 0x7f50b013c510 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.479+0000 7f50b5dff700 1 -- 172.21.15.131:0/3693480394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b015a0a0 con 0x7f50b013c510 2023-12-26T21:37:56.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.479+0000 7f50b5dff700 1 -- 172.21.15.131:0/3693480394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b00b1a20 con 0x7f50b013c510 2023-12-26T21:37:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.481+0000 7f50a77fe700 1 -- 172.21.15.131:0/3693480394 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f50a8014830 con 0x7f50b013c510 2023-12-26T21:37:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.482+0000 7f50a77fe700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5090066f30 0x7f50900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.482+0000 7f50a77fe700 1 -- 172.21.15.131:0/3693480394 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50a8084fe0 con 0x7f50b013c510 2023-12-26T21:37:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.482+0000 7f50af7fe700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5090066f30 0x7f50900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.482+0000 7f50af7fe700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5090066f30 0x7f50900693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009d10 tx=0x7f50a0009120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.483+0000 7f50c4580b80 1 Processor -- start 2023-12-26T21:37:56.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.483+0000 7f50c4580b80 1 -- start start 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.484+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d51fa0 0x558767d52370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.484+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 0x558767d4dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.484+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d6e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.484+0000 7f50c4580b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767c803e0 con 0x558767d4e170 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.484+0000 7f50c4580b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767c7dbe0 con 0x558767d528b0 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.484+0000 7f50c4580b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558767c799d0 con 0x558767d51fa0 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50affff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50affff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d6e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58664/0 (socket says 172.21.15.131:58664) 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50affff700 1 -- 172.21.15.131:0/922637276 learned_addr learned my addr 172.21.15.131:0/922637276 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50a7fff700 1 --2- 172.21.15.131:0/922637276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 0x558767d4dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50af7fe700 1 --2- 172.21.15.131:0/922637276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d51fa0 0x558767d52370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50affff700 1 -- 172.21.15.131:0/922637276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d51fa0 msgr2=0x558767d52370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50affff700 1 --2- 172.21.15.131:0/922637276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d51fa0 0x558767d52370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50affff700 1 -- 172.21.15.131:0/922637276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 msgr2=0x558767d4dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50affff700 1 --2- 172.21.15.131:0/922637276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 0x558767d4dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50affff700 1 -- 172.21.15.131:0/922637276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558767b0d360 con 0x558767d4e170 2023-12-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50affff700 1 --2- 172.21.15.131:0/922637276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d6e360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f50a8055e60 tx=0x7f50a8083420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ebc8f82600a42b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.485+0000 7f50a4ff9700 1 -- 172.21.15.131:0/922637276 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8055f50 con 0x558767d4e170 2023-12-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50a4ff9700 1 -- 172.21.15.131:0/922637276 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a8059120 con 0x558767d4e170 2023-12-26T21:37:56.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50a4ff9700 1 -- 172.21.15.131:0/922637276 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a805ce70 con 0x558767d4e170 2023-12-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50c4580b80 1 -- 172.21.15.131:0/922637276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 msgr2=0x558767d6e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50c4580b80 1 --2- 172.21.15.131:0/922637276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d6e360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f50a8055e60 tx=0x7f50a8083420 comp rx=0 tx=0).stop 2023-12-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50c4580b80 1 -- 172.21.15.131:0/922637276 shutdown_connections 2023-12-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50c4580b80 1 --2- 172.21.15.131:0/922637276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d6e360 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50c4580b80 1 --2- 172.21.15.131:0/922637276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d51fa0 0x558767d52370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50c4580b80 1 --2- 172.21.15.131:0/922637276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 0x558767d4dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50c4580b80 1 -- 172.21.15.131:0/922637276 >> 172.21.15.131:0/922637276 conn(0x558767d28440 msgr2=0x558767d73dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50c4580b80 1 -- 172.21.15.131:0/922637276 shutdown_connections 2023-12-26T21:37:56.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.486+0000 7f50c4580b80 1 -- 172.21.15.131:0/922637276 wait complete. 2023-12-26T21:37:56.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.487+0000 7f50c4580b80 1 -- start start 2023-12-26T21:37:56.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.487+0000 7f50c4580b80 1 Processor -- start 2023-12-26T21:37:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.488+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d4fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.488+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d51fa0 0x558767d4d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.488+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d528b0 0x558767d6add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.488+0000 7f50c4580b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767c261b0 con 0x558767d4e170 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.488+0000 7f50c4580b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767c25e50 con 0x558767d51fa0 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.488+0000 7f50c4580b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558767b72e40 con 0x558767d528b0 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.488+0000 7f50c4580b80 10 client.0 _ll_register_callbacks cb 0x558767b757b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.488+0000 7f50af7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d4fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f50af7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d4fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58678/0 (socket says 172.21.15.131:58678) 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f50af7fe700 1 -- 172.21.15.131:0/3629069462 learned_addr learned my addr 172.21.15.131:0/3629069462 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f50af7fe700 1 -- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d528b0 msgr2=0x558767d6add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f50af7fe700 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d528b0 0x558767d6add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f50af7fe700 1 -- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d51fa0 msgr2=0x558767d4d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f50af7fe700 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d51fa0 0x558767d4d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f50af7fe700 1 -- 172.21.15.131:0/3629069462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b00b1a20 con 0x558767d4e170 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f50af7fe700 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d4fed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f50a0006430 tx=0x7f50a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9408855c26b703fb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f508dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f508dffb700 1 -- 172.21.15.131:0/3629069462 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0011bc0 con 0x558767d4e170 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f508dffb700 1 -- 172.21.15.131:0/3629069462 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50a00068a0 con 0x558767d4e170 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.489+0000 7f50c4580b80 1 -- 172.21.15.131:0/3629069462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558767cfaeb0 con 0x558767d4e170 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f508dffb700 1 -- 172.21.15.131:0/3629069462 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50a0014ae0 con 0x558767d4e170 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f50c4580b80 10 client.5055 fetch_fsmap learned FSMap version 17 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f50c4580b80 1 -- 172.21.15.131:0/3629069462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x558767ab9b10 con 0x558767d4e170 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f508dffb700 1 -- 172.21.15.131:0/3629069462 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558767cfaeb0 con 0x558767d4e170 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f508dffb700 1 -- 172.21.15.131:0/3629069462 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f50a0014c40 con 0x558767d4e170 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f50c4580b80 10 client.5055 fetch_fsmap finished waiting for FSMap version 17 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f50c4580b80 10 client.5055 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-26T21:37:56.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f50c4580b80 20 client.5055 populate_metadata read hostname 'smithi131' 2023-12-26T21:37:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] conn(0x558767d69e00 0x558767d6a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f50c4580b80 4 client.5055 mds_command: new command op to 4888 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T21:37:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.490+0000 7f50c4580b80 1 -- 172.21.15.131:0/3629069462 --> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558767c50880 con 0x558767d69e00 2023-12-26T21:37:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.491+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] conn(0x558767d69e00 0x558767d6a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.491+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] conn(0x558767d69e00 0x558767d6a1d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f82a757f79d689 server_cookie=f58a4a3c3dfa38fd in_seq=0 out_seq=0 2023-12-26T21:37:56.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.491+0000 7f508dffb700 10 client.5055 ms_handle_connect on v2:172.21.15.190:6832/3890149779 2023-12-26T21:37:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.492+0000 7f508dffb700 1 -- 172.21.15.131:0/3629069462 <== mds.0 v2:172.21.15.190:6832/3890149779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558767c50880 con 0x558767d69e00 2023-12-26T21:37:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.492+0000 7f508dffb700 10 client.5055 handle_command_reply: tid=0 2023-12-26T21:37:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.492+0000 7f50c4580b80 1 client.5055 shutdown 2023-12-26T21:37:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 -- 172.21.15.131:0/3629069462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 msgr2=0x558767d4fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d4fed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f50a0006430 tx=0x7f50a0018000 comp rx=0 tx=0).stop 2023-12-26T21:37:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 -- 172.21.15.131:0/3629069462 shutdown_connections 2023-12-26T21:37:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] conn(0x558767d69e00 0x558767d6a1d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d528b0 0x558767d6add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d4fed0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f508dffb700 0 client.5055 ms_handle_reset on v2:172.21.15.190:6832/3890149779 2023-12-26T21:37:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3629069462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d51fa0 0x558767d4d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 -- 172.21.15.131:0/3629069462 >> 172.21.15.131:0/3629069462 conn(0x558767d28440 msgr2=0x558767d2cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 -- 172.21.15.131:0/3629069462 shutdown_connections 2023-12-26T21:37:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 1 -- 172.21.15.131:0/3629069462 wait complete. 2023-12-26T21:37:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.493+0000 7f50c4580b80 20 client.5055 trim_cache size 0 max 16384 2023-12-26T21:37:56.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.510+0000 7f50c4580b80 1 Processor -- start 2023-12-26T21:37:56.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.510+0000 7f50c4580b80 1 -- start start 2023-12-26T21:37:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.511+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d4e170 0x558767d51190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.511+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d51fa0 0x558767d1fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.511+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 0x558767d20420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.511+0000 7f50c4580b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767d27360 con 0x558767d51fa0 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.511+0000 7f50c4580b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767d28a20 con 0x558767d528b0 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.511+0000 7f50c4580b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558767d91470 con 0x558767d4e170 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.511+0000 7f50a7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d51fa0 0x558767d1fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.511+0000 7f50a7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d51fa0 0x558767d1fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58686/0 (socket says 172.21.15.131:58686) 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.511+0000 7f50a7fff700 1 -- 172.21.15.131:0/3846159005 learned_addr learned my addr 172.21.15.131:0/3846159005 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50af7fe700 1 --2- 172.21.15.131:0/3846159005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d4e170 0x558767d51190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50affff700 1 --2- 172.21.15.131:0/3846159005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 0x558767d20420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50a7fff700 1 -- 172.21.15.131:0/3846159005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d4e170 msgr2=0x558767d51190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3846159005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d4e170 0x558767d51190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50a7fff700 1 -- 172.21.15.131:0/3846159005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 msgr2=0x558767d20420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3846159005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 0x558767d20420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50a7fff700 1 -- 172.21.15.131:0/3846159005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558767c48290 con 0x558767d51fa0 2023-12-26T21:37:56.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50a7fff700 1 --2- 172.21.15.131:0/3846159005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d51fa0 0x558767d1fee0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f50b006fe40 tx=0x7f5098014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7845b792f1190d1b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50a57fa700 1 -- 172.21.15.131:0/3846159005 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098020040 con 0x558767d51fa0 2023-12-26T21:37:56.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50a57fa700 1 -- 172.21.15.131:0/3846159005 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5098021040 con 0x558767d51fa0 2023-12-26T21:37:56.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.512+0000 7f50a57fa700 1 -- 172.21.15.131:0/3846159005 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800b910 con 0x558767d51fa0 2023-12-26T21:37:56.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.513+0000 7f50c4580b80 1 -- 172.21.15.131:0/3846159005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d51fa0 msgr2=0x558767d1fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.513+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3846159005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d51fa0 0x558767d1fee0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f50b006fe40 tx=0x7f5098014000 comp rx=0 tx=0).stop 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.513+0000 7f50c4580b80 1 -- 172.21.15.131:0/3846159005 shutdown_connections 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.513+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3846159005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d528b0 0x558767d20420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.513+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3846159005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d4e170 0x558767d51190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.513+0000 7f50c4580b80 1 --2- 172.21.15.131:0/3846159005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d51fa0 0x558767d1fee0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.513+0000 7f50c4580b80 1 -- 172.21.15.131:0/3846159005 >> 172.21.15.131:0/3846159005 conn(0x558767d28440 msgr2=0x558767d4d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.513+0000 7f50c4580b80 1 -- 172.21.15.131:0/3846159005 shutdown_connections 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.513+0000 7f50c4580b80 1 -- 172.21.15.131:0/3846159005 wait complete. 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.514+0000 7f50c4580b80 1 -- start start 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.514+0000 7f50c4580b80 1 Processor -- start 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d1fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d51fa0 0x558767d20420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50c4580b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d528b0 0x558767d73dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50c4580b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767d86300 con 0x558767d4e170 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50c4580b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767d4f920 con 0x558767d51fa0 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50c4580b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558767d6fca0 con 0x558767d528b0 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50c4580b80 10 client.0 _ll_register_callbacks cb 0x7f50a0005da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50af7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d1fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50af7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d1fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58700/0 (socket says 172.21.15.131:58700) 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50af7fe700 1 -- 172.21.15.131:0/2403226503 learned_addr learned my addr 172.21.15.131:0/2403226503 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50a7fff700 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d51fa0 0x558767d20420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50affff700 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d528b0 0x558767d73dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50af7fe700 1 -- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d528b0 msgr2=0x558767d73dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50af7fe700 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d528b0 0x558767d73dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50af7fe700 1 -- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d51fa0 msgr2=0x558767d20420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50af7fe700 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d51fa0 0x558767d20420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.515+0000 7f50af7fe700 1 -- 172.21.15.131:0/2403226503 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b0068910 con 0x558767d4e170 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50af7fe700 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d1fee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005f80 tx=0x7f50a0025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b1379932afb4fb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f508dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f508dffb700 1 -- 172.21.15.131:0/2403226503 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a00095a0 con 0x558767d4e170 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f508dffb700 1 -- 172.21.15.131:0/2403226503 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50a00068a0 con 0x558767d4e170 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50c4580b80 1 -- 172.21.15.131:0/2403226503 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558767d1e490 con 0x558767d4e170 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f508dffb700 1 -- 172.21.15.131:0/2403226503 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50a002a560 con 0x558767d4e170 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50c4580b80 10 client.5061 fetch_fsmap learned FSMap version 17 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50c4580b80 1 -- 172.21.15.131:0/2403226503 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x558767b11c80 con 0x558767d4e170 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f508dffb700 1 -- 172.21.15.131:0/2403226503 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558767d1e490 con 0x558767d4e170 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f508dffb700 1 -- 172.21.15.131:0/2403226503 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f50a002a6c0 con 0x558767d4e170 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50c4580b80 10 client.5061 fetch_fsmap finished waiting for FSMap version 17 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50c4580b80 10 client.5061 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50c4580b80 20 client.5061 populate_metadata read hostname 'smithi131' 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50c4580b80 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] conn(0x558767d74300 0x558767d746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50c4580b80 4 client.5061 mds_command: new command op to 4888 tid=0[{"prefix": "status"}] 2023-12-26T21:37:56.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.516+0000 7f50c4580b80 1 -- 172.21.15.131:0/2403226503 --> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558767c1c910 con 0x558767d74300 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.517+0000 7f50a7fff700 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] conn(0x558767d74300 0x558767d746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.517+0000 7f50a7fff700 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] conn(0x558767d74300 0x558767d746d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16a36ccb2be9ee33 server_cookie=5ad358ea6799170c in_seq=0 out_seq=0 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.517+0000 7f508dffb700 10 client.5061 ms_handle_connect on v2:172.21.15.190:6832/3890149779 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.517+0000 7f508dffb700 1 -- 172.21.15.131:0/2403226503 <== mds.0 v2:172.21.15.190:6832/3890149779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x558767c1c910 con 0x558767d74300 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f508dffb700 10 client.5061 handle_command_reply: tid=0 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 client.5061 shutdown 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 -- 172.21.15.131:0/2403226503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 msgr2=0x558767d1fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d1fee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005f80 tx=0x7f50a0025000 comp rx=0 tx=0).stop 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 -- 172.21.15.131:0/2403226503 shutdown_connections 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:6832/3890149779,v1:172.21.15.190:6833/3890149779] conn(0x558767d74300 0x558767d746d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558767d528b0 0x558767d73dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558767d4e170 0x558767d1fee0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f508dffb700 0 client.5061 ms_handle_reset on v2:172.21.15.190:6832/3890149779 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 --2- 172.21.15.131:0/2403226503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558767d51fa0 0x558767d20420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 -- 172.21.15.131:0/2403226503 >> 172.21.15.131:0/2403226503 conn(0x558767d28440 msgr2=0x558767d51230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 -- 172.21.15.131:0/2403226503 shutdown_connections 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 1 -- 172.21.15.131:0/2403226503 wait complete. 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.518+0000 7f50c4580b80 20 client.5061 trim_cache size 0 max 16384 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4888, 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T21:37:56.523 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T21:37:56.524 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.4754471870000003, 2023-12-26T21:37:56.524 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 17, 2023-12-26T21:37:56.524 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 35, 2023-12-26T21:37:56.524 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 35, 2023-12-26T21:37:56.524 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 22.072474989 2023-12-26T21:37:56.524 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:37:56.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 -- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5090066f30 msgr2=0x7f50900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5090066f30 0x7f50900693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009d10 tx=0x7f50a0009120 comp rx=0 tx=0).stop 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 -- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b013c510 msgr2=0x7f50b016d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b013c510 0x7f50b016d0d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009e90 tx=0x7f50a800b040 comp rx=0 tx=0).stop 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 -- 172.21.15.131:0/3693480394 shutdown_connections 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50b013c510 0x7f50b016d0d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5090066f30 0x7f50900693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50b006b8c0 0x7f50b013bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 --2- 172.21.15.131:0/3693480394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50b00cc100 0x7f50b015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 -- 172.21.15.131:0/3693480394 >> 172.21.15.131:0/3693480394 conn(0x7f50b00ce3e0 msgr2=0x7f50b00c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 -- 172.21.15.131:0/3693480394 shutdown_connections 2023-12-26T21:37:56.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.519+0000 7f50b5dff700 1 -- 172.21.15.131:0/3693480394 wait complete. 2023-12-26T21:37:56.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:37:56.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.725+0000 7fd377dff700 1 Processor -- start 2023-12-26T21:37:56.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.726+0000 7fd377dff700 1 -- start start 2023-12-26T21:37:56.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.727+0000 7fd377dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd3700c8820 0x7fd3700ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.727+0000 7fd377dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3700c8bf0 0x7fd3700b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.727+0000 7fd377dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd370157a80 con 0x7fd370069d10 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.727+0000 7fd377dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3700cac50 con 0x7fd3700c8820 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.727+0000 7fd377dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700caf30 con 0x7fd3700c8bf0 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.727+0000 7fd375b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd3700c8820 0x7fd3700ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.727+0000 7fd37639c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd370069d10 0x7fd370157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41426/0 (socket says 172.21.15.131:41426) 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.727+0000 7fd375b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd3700c8820 0x7fd3700ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58712/0 (socket says 172.21.15.131:58712) 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.727+0000 7fd37639c700 1 -- 172.21.15.131:0/3496831504 learned_addr learned my addr 172.21.15.131:0/3496831504 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.728+0000 7fd375b9b700 1 -- 172.21.15.131:0/3496831504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3700c8bf0 msgr2=0x7fd3700b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.728+0000 7fd375b9b700 1 --2- 172.21.15.131:0/3496831504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3700c8bf0 0x7fd3700b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.728+0000 7fd375b9b700 1 -- 172.21.15.131:0/3496831504 >> v1:172.21.15.131:6789/0 conn(0x7fd370069d10 legacy=0x7fd370157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.728+0000 7fd375b9b700 1 -- 172.21.15.131:0/3496831504 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3700cb210 con 0x7fd3700c8820 2023-12-26T21:37:56.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.728+0000 7fd374b99700 1 -- 172.21.15.131:0/3496831504 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861443089 0 0) 0x7fd370157a80 con 0x7fd370069d10 2023-12-26T21:37:56.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.728+0000 7fd375b9b700 1 --2- 172.21.15.131:0/3496831504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8820 0x7fd3700ca300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0098e0 tx=0x7fd36c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bfac9d16bd866c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.728+0000 7fd374b99700 1 -- 172.21.15.131:0/3496831504 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c009e80 con 0x7fd3700c8820 2023-12-26T21:37:56.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.728+0000 7fd374b99700 1 -- 172.21.15.131:0/3496831504 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c00b860 con 0x7fd3700c8820 2023-12-26T21:37:56.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.728+0000 7fd374b99700 1 -- 172.21.15.131:0/3496831504 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00bbe0 con 0x7fd3700c8820 2023-12-26T21:37:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.729+0000 7fd377dff700 1 -- 172.21.15.131:0/3496831504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8820 msgr2=0x7fd3700ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.729+0000 7fd377dff700 1 --2- 172.21.15.131:0/3496831504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8820 0x7fd3700ca300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0098e0 tx=0x7fd36c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.729+0000 7fd377dff700 1 -- 172.21.15.131:0/3496831504 shutdown_connections 2023-12-26T21:37:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.729+0000 7fd377dff700 1 --2- 172.21.15.131:0/3496831504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8820 0x7fd3700ca300 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.729+0000 7fd377dff700 1 --2- 172.21.15.131:0/3496831504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3700c8bf0 0x7fd3700b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.729+0000 7fd377dff700 1 -- 172.21.15.131:0/3496831504 >> 172.21.15.131:0/3496831504 conn(0x7fd370068040 msgr2=0x7fd370069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.729+0000 7fd377dff700 1 -- 172.21.15.131:0/3496831504 shutdown_connections 2023-12-26T21:37:56.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.729+0000 7fd377dff700 1 -- 172.21.15.131:0/3496831504 wait complete. 2023-12-26T21:37:56.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.731+0000 7fd377dff700 1 Processor -- start 2023-12-26T21:37:56.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.731+0000 7fd377dff700 1 -- start start 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.732+0000 7fd377dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd370069d10 0x7fd370154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.732+0000 7fd377dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3700c8820 0x7fd3700a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.732+0000 7fd377dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8bf0 0x7fd37015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.732+0000 7fd377dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd370169b00 con 0x7fd3700c8bf0 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.732+0000 7fd377dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700a71d0 con 0x7fd3700c8820 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.732+0000 7fd377dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd370169d80 con 0x7fd370069d10 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.732+0000 7fd37639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8bf0 0x7fd37015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd37639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8bf0 0x7fd37015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58716/0 (socket says 172.21.15.131:58716) 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd37639c700 1 -- 172.21.15.131:0/3767219744 learned_addr learned my addr 172.21.15.131:0/3767219744 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:56.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd375b9b700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd370069d10 0x7fd370154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd37539a700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3700c8820 0x7fd3700a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd37639c700 1 -- 172.21.15.131:0/3767219744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd370069d10 msgr2=0x7fd370154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd37639c700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd370069d10 0x7fd370154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd37639c700 1 -- 172.21.15.131:0/3767219744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3700c8820 msgr2=0x7fd3700a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd37639c700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3700c8820 0x7fd3700a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd37639c700 1 -- 172.21.15.131:0/3767219744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c009420 con 0x7fd3700c8bf0 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd375b9b700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd370069d10 0x7fd370154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd37639c700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8bf0 0x7fd37015c190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd36800cc10 tx=0x7fd368001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd365ffb700 1 -- 172.21.15.131:0/3767219744 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36800a930 con 0x7fd3700c8bf0 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd365ffb700 1 -- 172.21.15.131:0/3767219744 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36800aab0 con 0x7fd3700c8bf0 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3700a6930 con 0x7fd3700c8bf0 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.733+0000 7fd365ffb700 1 -- 172.21.15.131:0/3767219744 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36800ae70 con 0x7fd3700c8bf0 2023-12-26T21:37:56.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.734+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3700a6cc0 con 0x7fd3700c8bf0 2023-12-26T21:37:56.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.735+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd370157690 con 0x7fd3700c8bf0 2023-12-26T21:37:56.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.736+0000 7fd365ffb700 1 -- 172.21.15.131:0/3767219744 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd368007300 con 0x7fd3700c8bf0 2023-12-26T21:37:56.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.737+0000 7fd365ffb700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd354066f30 0x7fd3540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:56.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.737+0000 7fd365ffb700 1 -- 172.21.15.131:0/3767219744 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd3680849a0 con 0x7fd3700c8bf0 2023-12-26T21:37:56.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.737+0000 7fd375b9b700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd354066f30 0x7fd3540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:56.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.737+0000 7fd375b9b700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd354066f30 0x7fd3540693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0091e0 tx=0x7fd36c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:56.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.740+0000 7fd365ffb700 1 -- 172.21.15.131:0/3767219744 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd368051b60 con 0x7fd3700c8bf0 2023-12-26T21:37:56.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.973+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd3700657e0 con 0x7fd3700c8bf0 2023-12-26T21:37:56.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.974+0000 7fd365ffb700 1 -- 172.21.15.131:0/3767219744 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd368055180 con 0x7fd3700c8bf0 2023-12-26T21:37:56.974 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:37:56.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd354066f30 msgr2=0x7fd3540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd354066f30 0x7fd3540693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0091e0 tx=0x7fd36c017000 comp rx=0 tx=0).stop 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8bf0 msgr2=0x7fd37015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8bf0 0x7fd37015c190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd36800cc10 tx=0x7fd368001970 comp rx=0 tx=0).stop 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 shutdown_connections 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3700c8bf0 0x7fd37015c190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd354066f30 0x7fd3540693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd370069d10 0x7fd370154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 --2- 172.21.15.131:0/3767219744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3700c8820 0x7fd3700a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 >> 172.21.15.131:0/3767219744 conn(0x7fd370068040 msgr2=0x7fd3700c26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 shutdown_connections 2023-12-26T21:37:56.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:56.976+0000 7fd377dff700 1 -- 172.21.15.131:0/3767219744 wait complete. 2023-12-26T21:37:56.994 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T21:37:56.994 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:37:56.994 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T21:37:56.994 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T21:37:56.994 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T21:37:56.994 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T21:37:56.994 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T21:37:56.994 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:37:56.995 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:37:56.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:56.995 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T21:37:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:57.040 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42379sec preferred_lft 42379sec 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T21:37:57.041 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T21:37:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:57.042 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:37:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T21:37:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T21:37:57.042 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:37:57.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:57.043 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:37:57.150 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T21:37:57.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:57.153 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T21:37:57.199 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T21:37:57.199 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:37:57.199 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:37:57.199 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:37:57.199 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:37:57.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:57.200 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:37:57.200 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:37:57.200 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:37:57.200 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:37:57.200 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:37:57.200 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:37:57.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:57.410 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:37:57.453 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T21:37:57.453 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T21:37:57.454 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:37:57.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:57.454 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T21:37:57.564 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:57.564 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T21:37:57.607 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:37:57.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:37:57.607 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:37:57.689 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T21:37:57.689 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:37:57.690 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:57.827 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:57.827 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:57.856 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:37:57.856 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:37:57.856 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:37:57.856 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:37:57.856 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:37:57.861 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:37:57.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:37:57.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:37:57.931 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T21:37:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.123+0000 7f8428fff700 1 Processor -- start 2023-12-26T21:37:58.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.123+0000 7f8428fff700 1 -- start start 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.124+0000 7f8428fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f84240cb320 0x7f8424069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.124+0000 7f8428fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f84240b0e60 0x7f84240c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.124+0000 7f8428fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8424156940 con 0x7f842406a1e0 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.124+0000 7f8428fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84240c8cc0 con 0x7f84240b0e60 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.124+0000 7f8428fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84240ad700 con 0x7f84240cb320 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.124+0000 7f8422d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f84240b0e60 0x7f84240c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f8421d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f842406a1e0 0x7f84240b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41428/0 (socket says 172.21.15.131:41428) 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f8421d9b700 1 -- 172.21.15.131:0/2419561349 learned_addr learned my addr 172.21.15.131:0/2419561349 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f842259c700 1 --2- 172.21.15.131:0/2419561349 >> v2:172.21.15.190:3300/0 conn(0x7f84240cb320 0x7f8424069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f842159a700 1 -- 172.21.15.131:0/2419561349 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773668383 0 0) 0x7f8424156940 con 0x7f842406a1e0 2023-12-26T21:37:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f8422d9d700 1 -- 172.21.15.131:0/2419561349 >> v2:172.21.15.190:3300/0 conn(0x7f84240cb320 msgr2=0x7f8424069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f8422d9d700 1 --2- 172.21.15.131:0/2419561349 >> v2:172.21.15.190:3300/0 conn(0x7f84240cb320 0x7f8424069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f8422d9d700 1 -- 172.21.15.131:0/2419561349 >> v1:172.21.15.131:6789/0 conn(0x7f842406a1e0 legacy=0x7f84240b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f8422d9d700 1 -- 172.21.15.131:0/2419561349 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84240ae500 con 0x7f84240b0e60 2023-12-26T21:37:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f8422d9d700 1 --2- 172.21.15.131:0/2419561349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 0x7f84240c9400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8424064410 tx=0x7f841c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2e99cf323022b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.125+0000 7f842159a700 1 -- 172.21.15.131:0/2419561349 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00e070 con 0x7f84240b0e60 2023-12-26T21:37:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f842159a700 1 -- 172.21.15.131:0/2419561349 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c00bb60 con 0x7f84240b0e60 2023-12-26T21:37:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f842159a700 1 -- 172.21.15.131:0/2419561349 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c012430 con 0x7f84240b0e60 2023-12-26T21:37:58.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f8428fff700 1 -- 172.21.15.131:0/2419561349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 msgr2=0x7f84240c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:58.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f8428fff700 1 --2- 172.21.15.131:0/2419561349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 0x7f84240c9400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8424064410 tx=0x7f841c009780 comp rx=0 tx=0).stop 2023-12-26T21:37:58.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f8428fff700 1 -- 172.21.15.131:0/2419561349 shutdown_connections 2023-12-26T21:37:58.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f8428fff700 1 --2- 172.21.15.131:0/2419561349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 0x7f84240c9400 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f8428fff700 1 --2- 172.21.15.131:0/2419561349 >> v2:172.21.15.190:3300/0 conn(0x7f84240cb320 0x7f8424069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f8428fff700 1 -- 172.21.15.131:0/2419561349 >> 172.21.15.131:0/2419561349 conn(0x7f8424068040 msgr2=0x7f8424069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f8428fff700 1 -- 172.21.15.131:0/2419561349 shutdown_connections 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.126+0000 7f8428fff700 1 -- 172.21.15.131:0/2419561349 wait complete. 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.128+0000 7f8428fff700 1 Processor -- start 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.128+0000 7f8428fff700 1 -- start start 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8428fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842406a1e0 0x7f8424151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8428fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 0x7f842414e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8428fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84240cb320 0x7f842413eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8428fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842414eb00 con 0x7f84240b0e60 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8428fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8424159810 con 0x7f842406a1e0 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8428fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8424158c00 con 0x7f84240cb320 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8421d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 0x7f842414e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8421d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 0x7f842414e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58736/0 (socket says 172.21.15.131:58736) 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8421d9b700 1 -- 172.21.15.131:0/2942344007 learned_addr learned my addr 172.21.15.131:0/2942344007 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:58.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f842259c700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842406a1e0 0x7f8424151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8422d9d700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84240cb320 0x7f842413eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8421d9b700 1 -- 172.21.15.131:0/2942344007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84240cb320 msgr2=0x7f842413eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8421d9b700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84240cb320 0x7f842413eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8421d9b700 1 -- 172.21.15.131:0/2942344007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842406a1e0 msgr2=0x7f8424151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8421d9b700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842406a1e0 0x7f8424151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8421d9b700 1 -- 172.21.15.131:0/2942344007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c009a20 con 0x7f84240b0e60 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.129+0000 7f8421d9b700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 0x7f842414e5c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f841800c810 tx=0x7f8418007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.130+0000 7f84127fc700 1 -- 172.21.15.131:0/2942344007 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418005c00 con 0x7f84240b0e60 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.130+0000 7f84127fc700 1 -- 172.21.15.131:0/2942344007 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418005d60 con 0x7f84240b0e60 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.130+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8424158ee0 con 0x7f84240b0e60 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.130+0000 7f84127fc700 1 -- 172.21.15.131:0/2942344007 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180072e0 con 0x7f84240b0e60 2023-12-26T21:37:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.130+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84241594a0 con 0x7f84240b0e60 2023-12-26T21:37:58.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.131+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84240657e0 con 0x7f84240b0e60 2023-12-26T21:37:58.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.132+0000 7f84127fc700 1 -- 172.21.15.131:0/2942344007 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8418007650 con 0x7f84240b0e60 2023-12-26T21:37:58.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.133+0000 7f84127fc700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8404066f30 0x7f84040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.133+0000 7f84127fc700 1 -- 172.21.15.131:0/2942344007 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f84180839d0 con 0x7f84240b0e60 2023-12-26T21:37:58.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.133+0000 7f842259c700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8404066f30 0x7f84040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.133+0000 7f842259c700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8404066f30 0x7f84040693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8414009d10 tx=0x7f84140091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:58.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.137+0000 7f84127fc700 1 -- 172.21.15.131:0/2942344007 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f841804b5a0 con 0x7f84240b0e60 2023-12-26T21:37:58.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.353+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8424159950 con 0x7f8404066f30 2023-12-26T21:37:58.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.355+0000 7f84127fc700 1 -- 172.21.15.131:0/2942344007 <== mgr.4100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8424159950 con 0x7f8404066f30 2023-12-26T21:37:58.356 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:37:58.356 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:37:58.356 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T21:37:58.356 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:37:58.356 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:37:58.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8404066f30 msgr2=0x7f84040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:58.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8404066f30 0x7f84040693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8414009d10 tx=0x7f84140091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 msgr2=0x7f842414e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:58.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 0x7f842414e5c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f841800c810 tx=0x7f8418007db0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 shutdown_connections 2023-12-26T21:37:58.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8404066f30 0x7f84040693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84240cb320 0x7f842413eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f842406a1e0 0x7f8424151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 --2- 172.21.15.131:0/2942344007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84240b0e60 0x7f842414e5c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.357+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 >> 172.21.15.131:0/2942344007 conn(0x7f8424068040 msgr2=0x7f84240cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:58.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.358+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 shutdown_connections 2023-12-26T21:37:58.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.358+0000 7f8428fff700 1 -- 172.21.15.131:0/2942344007 wait complete. 2023-12-26T21:37:58.376 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T21:37:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.567+0000 7f28697ff700 1 Processor -- start 2023-12-26T21:37:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.567+0000 7f28697ff700 1 -- start start 2023-12-26T21:37:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.568+0000 7f28697ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f286413c510 0x7f28640c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.568+0000 7f28697ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f286406b8c0 0x7f286406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.568+0000 7f28697ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f28640cc100 0x7f28640cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.568+0000 7f28697ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28640686c0 con 0x7f286413c510 2023-12-26T21:37:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f28697ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2864069520 con 0x7f28640cc100 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f28697ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286406c250 con 0x7f286406b8c0 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f2862ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f286413c510 0x7f28640c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f2862ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f286413c510 0x7f28640c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58746/0 (socket says 172.21.15.131:58746) 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f2862ffd700 1 -- 172.21.15.131:0/4117361999 learned_addr learned my addr 172.21.15.131:0/4117361999 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f28637fe700 1 --2- 172.21.15.131:0/4117361999 >> v2:172.21.15.190:3300/0 conn(0x7f28640cc100 0x7f28640cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f28627fc700 1 --2- 172.21.15.131:0/4117361999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f286406b8c0 0x7f286406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f28637fe700 1 -- 172.21.15.131:0/4117361999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f286406b8c0 msgr2=0x7f286406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f28637fe700 1 --2- 172.21.15.131:0/4117361999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f286406b8c0 0x7f286406bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f28637fe700 1 -- 172.21.15.131:0/4117361999 >> v2:172.21.15.131:3300/0 conn(0x7f286413c510 msgr2=0x7f28640c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f28637fe700 1 --2- 172.21.15.131:0/4117361999 >> v2:172.21.15.131:3300/0 conn(0x7f286413c510 0x7f28640c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f28637fe700 1 -- 172.21.15.131:0/4117361999 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864068910 con 0x7f28640cc100 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.569+0000 7f2862ffd700 1 --2- 172.21.15.131:0/4117361999 >> v2:172.21.15.131:3300/0 conn(0x7f286413c510 0x7f28640c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f28637fe700 1 --2- 172.21.15.131:0/4117361999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28640cc100 0x7f28640cc560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f286406f130 tx=0x7f285c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7902406a219ee2a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f2861ffb700 1 -- 172.21.15.131:0/4117361999 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c00e070 con 0x7f28640cc100 2023-12-26T21:37:58.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f2861ffb700 1 -- 172.21.15.131:0/4117361999 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c00bab0 con 0x7f28640cc100 2023-12-26T21:37:58.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f2861ffb700 1 -- 172.21.15.131:0/4117361999 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c00be70 con 0x7f28640cc100 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f28697ff700 1 -- 172.21.15.131:0/4117361999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28640cc100 msgr2=0x7f28640cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f28697ff700 1 --2- 172.21.15.131:0/4117361999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28640cc100 0x7f28640cc560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f286406f130 tx=0x7f285c009780 comp rx=0 tx=0).stop 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f28697ff700 1 -- 172.21.15.131:0/4117361999 shutdown_connections 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f28697ff700 1 --2- 172.21.15.131:0/4117361999 >> v2:172.21.15.131:3300/0 conn(0x7f286413c510 0x7f28640c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f28697ff700 1 --2- 172.21.15.131:0/4117361999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28640cc100 0x7f28640cc560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.570+0000 7f28697ff700 1 --2- 172.21.15.131:0/4117361999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f286406b8c0 0x7f286406bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.571+0000 7f28697ff700 1 -- 172.21.15.131:0/4117361999 >> 172.21.15.131:0/4117361999 conn(0x7f28640ce3e0 msgr2=0x7f28640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.571+0000 7f28697ff700 1 -- 172.21.15.131:0/4117361999 shutdown_connections 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.571+0000 7f28697ff700 1 -- 172.21.15.131:0/4117361999 wait complete. 2023-12-26T21:37:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.572+0000 7f28697ff700 1 Processor -- start 2023-12-26T21:37:58.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.572+0000 7f28697ff700 1 -- start start 2023-12-26T21:37:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f28697ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f286406b8c0 0x7f286413a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f28697ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28640cc100 0x7f2864154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f28697ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f286413c510 0x7f2864154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f28697ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2864157200 con 0x7f286406b8c0 2023-12-26T21:37:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f28697ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28640c5f60 con 0x7f286413c510 2023-12-26T21:37:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f28697ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28640c5630 con 0x7f28640cc100 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f2862ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f286406b8c0 0x7f286413a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f2862ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f286406b8c0 0x7f286413a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58758/0 (socket says 172.21.15.131:58758) 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f2862ffd700 1 -- 172.21.15.131:0/515116475 learned_addr learned my addr 172.21.15.131:0/515116475 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.573+0000 7f28627fc700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28640cc100 0x7f2864154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f28637fe700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f286413c510 0x7f2864154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f2862ffd700 1 -- 172.21.15.131:0/515116475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28640cc100 msgr2=0x7f2864154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f2862ffd700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28640cc100 0x7f2864154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f2862ffd700 1 -- 172.21.15.131:0/515116475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f286413c510 msgr2=0x7f2864154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f2862ffd700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f286413c510 0x7f2864154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f2862ffd700 1 -- 172.21.15.131:0/515116475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854009490 con 0x7f286406b8c0 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f28637fe700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f286413c510 0x7f2864154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f2862ffd700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f286406b8c0 0x7f286413a970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f285400ec50 tx=0x7f2854009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f284affd700 1 -- 172.21.15.131:0/515116475 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285400c9b0 con 0x7f286406b8c0 2023-12-26T21:37:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f284affd700 1 -- 172.21.15.131:0/515116475 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285400cb10 con 0x7f286406b8c0 2023-12-26T21:37:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c009a20 con 0x7f286406b8c0 2023-12-26T21:37:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f284affd700 1 -- 172.21.15.131:0/515116475 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285400ce90 con 0x7f286406b8c0 2023-12-26T21:37:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.574+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28640c5bf0 con 0x7f286406b8c0 2023-12-26T21:37:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.575+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28640657e0 con 0x7f286406b8c0 2023-12-26T21:37:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.576+0000 7f284affd700 1 -- 172.21.15.131:0/515116475 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2854013680 con 0x7f286406b8c0 2023-12-26T21:37:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.577+0000 7f284affd700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2850066f30 0x7f28500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.577+0000 7f284affd700 1 -- 172.21.15.131:0/515116475 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2854084cc0 con 0x7f286406b8c0 2023-12-26T21:37:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.578+0000 7f28627fc700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2850066f30 0x7f28500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.578+0000 7f28627fc700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2850066f30 0x7f28500693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f286416cc50 tx=0x7f28580091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.581+0000 7f284affd700 1 -- 172.21.15.131:0/515116475 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2854051f30 con 0x7f286406b8c0 2023-12-26T21:37:58.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:58.826+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2864068bf0 con 0x7f286406b8c0 2023-12-26T21:37:59.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.095+0000 7f284affd700 1 -- 172.21.15.131:0/515116475 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 6) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f2854052300 con 0x7f286406b8c0 2023-12-26T21:37:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.097+0000 7f284affd700 1 -- 172.21.15.131:0/515116475 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2854051f30 con 0x7f286406b8c0 2023-12-26T21:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2850066f30 msgr2=0x7f28500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2850066f30 0x7f28500693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f286416cc50 tx=0x7f28580091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f286406b8c0 msgr2=0x7f286413a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f286406b8c0 0x7f286413a970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f285400ec50 tx=0x7f2854009250 comp rx=0 tx=0).stop 2023-12-26T21:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 shutdown_connections 2023-12-26T21:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f286413c510 0x7f2864154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f286406b8c0 0x7f286413a970 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2850066f30 0x7f28500693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 --2- 172.21.15.131:0/515116475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28640cc100 0x7f2864154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 >> 172.21.15.131:0/515116475 conn(0x7f28640ce3e0 msgr2=0x7f286413b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 shutdown_connections 2023-12-26T21:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.100+0000 7f28697ff700 1 -- 172.21.15.131:0/515116475 wait complete. 2023-12-26T21:37:59.119 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T21:37:59.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.322+0000 7f70843ff700 1 Processor -- start 2023-12-26T21:37:59.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.322+0000 7f70843ff700 1 -- start start 2023-12-26T21:37:59.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.324+0000 7f70843ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f707c159920 0x7f707c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f70843ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f707c155be0 0x7f707c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f70843ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707c0c8bf0 con 0x7f707c159cf0 2023-12-26T21:37:59.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f70843ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c0c8e60 con 0x7f707c159920 2023-12-26T21:37:59.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f70843ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c0c9140 con 0x7f707c155be0 2023-12-26T21:37:59.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f708219b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f707c159920 0x7f707c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f708199a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f707c159cf0 0x7f707c1554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41444/0 (socket says 172.21.15.131:41444) 2023-12-26T21:37:59.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f708199a700 1 -- 172.21.15.131:0/2569909522 learned_addr learned my addr 172.21.15.131:0/2569909522 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:59.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f708299c700 1 --2- 172.21.15.131:0/2569909522 >> v2:172.21.15.190:3300/0 conn(0x7f707c155be0 0x7f707c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f7081199700 1 -- 172.21.15.131:0/2569909522 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347323890 0 0) 0x7f707c0c8bf0 con 0x7f707c159cf0 2023-12-26T21:37:59.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f7081199700 1 -- 172.21.15.131:0/2569909522 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7068003570 con 0x7f707c159cf0 2023-12-26T21:37:59.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f708219b700 1 -- 172.21.15.131:0/2569909522 >> v2:172.21.15.190:3300/0 conn(0x7f707c155be0 msgr2=0x7f707c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:59.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f708219b700 1 --2- 172.21.15.131:0/2569909522 >> v2:172.21.15.190:3300/0 conn(0x7f707c155be0 0x7f707c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f708219b700 1 -- 172.21.15.131:0/2569909522 >> v1:172.21.15.131:6789/0 conn(0x7f707c159cf0 legacy=0x7f707c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.325+0000 7f708219b700 1 -- 172.21.15.131:0/2569909522 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0c9420 con 0x7f707c159920 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.326+0000 7f708219b700 1 --2- 172.21.15.131:0/2569909522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 0x7f707c0b00e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7078009910 tx=0x7f70780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f685af6b2b6c8695 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.326+0000 7f7081199700 1 -- 172.21.15.131:0/2569909522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078011630 con 0x7f707c159920 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.326+0000 7f7081199700 1 -- 172.21.15.131:0/2569909522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7078011790 con 0x7f707c159920 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.326+0000 7f7081199700 1 -- 172.21.15.131:0/2569909522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078011b10 con 0x7f707c159920 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.327+0000 7f70843ff700 1 -- 172.21.15.131:0/2569909522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 msgr2=0x7f707c0b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.327+0000 7f70843ff700 1 --2- 172.21.15.131:0/2569909522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 0x7f707c0b00e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7078009910 tx=0x7f70780091a0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.327+0000 7f70843ff700 1 -- 172.21.15.131:0/2569909522 shutdown_connections 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.327+0000 7f70843ff700 1 --2- 172.21.15.131:0/2569909522 >> v2:172.21.15.190:3300/0 conn(0x7f707c155be0 0x7f707c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.327+0000 7f70843ff700 1 --2- 172.21.15.131:0/2569909522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 0x7f707c0b00e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.327+0000 7f70843ff700 1 -- 172.21.15.131:0/2569909522 >> 172.21.15.131:0/2569909522 conn(0x7f707c068040 msgr2=0x7f707c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:59.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.327+0000 7f70843ff700 1 -- 172.21.15.131:0/2569909522 shutdown_connections 2023-12-26T21:37:59.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.327+0000 7f70843ff700 1 -- 172.21.15.131:0/2569909522 wait complete. 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.329+0000 7f70843ff700 1 Processor -- start 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.329+0000 7f70843ff700 1 -- start start 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f70843ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f707c155be0 0x7f707c0c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f70843ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 0x7f707c15aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f70843ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c159cf0 0x7f707c15b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f70843ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c15dfa0 con 0x7f707c159920 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f70843ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c15e280 con 0x7f707c159cf0 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f70843ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c0c6aa0 con 0x7f707c155be0 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f708199a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 0x7f707c15aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f708299c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c159cf0 0x7f707c15b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f708199a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 0x7f707c15aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58792/0 (socket says 172.21.15.131:58792) 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f708199a700 1 -- 172.21.15.131:0/3100235732 learned_addr learned my addr 172.21.15.131:0/3100235732 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f708219b700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f707c155be0 0x7f707c0c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f708199a700 1 -- 172.21.15.131:0/3100235732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f707c155be0 msgr2=0x7f707c0c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f708199a700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f707c155be0 0x7f707c0c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f708199a700 1 -- 172.21.15.131:0/3100235732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c159cf0 msgr2=0x7f707c15b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.330+0000 7f708199a700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c159cf0 0x7f707c15b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.331+0000 7f708199a700 1 -- 172.21.15.131:0/3100235732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078009420 con 0x7f707c159920 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.331+0000 7f708299c700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c159cf0 0x7f707c15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.331+0000 7f708199a700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 0x7f707c15aa40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f706c0011e0 tx=0x7f706c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.331+0000 7f70727fc700 1 -- 172.21.15.131:0/3100235732 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c005b30 con 0x7f707c159920 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.331+0000 7f70727fc700 1 -- 172.21.15.131:0/3100235732 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c005c90 con 0x7f707c159920 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.331+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c15b940 con 0x7f707c159920 2023-12-26T21:37:59.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.331+0000 7f70727fc700 1 -- 172.21.15.131:0/3100235732 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c007300 con 0x7f707c159920 2023-12-26T21:37:59.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.331+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c06d6a0 con 0x7f707c159920 2023-12-26T21:37:59.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.332+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707c0c6c60 con 0x7f707c159920 2023-12-26T21:37:59.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.333+0000 7f70727fc700 1 -- 172.21.15.131:0/3100235732 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 6) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f706c007670 con 0x7f707c159920 2023-12-26T21:37:59.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.334+0000 7f70727fc700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7060066f80 0x7f7060069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.334+0000 7f708219b700 1 -- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7060066f80 msgr2=0x7f7060069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:37:59.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.334+0000 7f70727fc700 1 -- 172.21.15.131:0/3100235732 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f706c0839a0 con 0x7f707c159920 2023-12-26T21:37:59.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.334+0000 7f708219b700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7060066f80 0x7f7060069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:37:59.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.339+0000 7f70727fc700 1 -- 172.21.15.131:0/3100235732 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f706c050c10 con 0x7f707c159920 2023-12-26T21:37:59.524 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:37:59.524+0000 7fa5f7ac61c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:37:59.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.535+0000 7f708219b700 1 -- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7060066f80 msgr2=0x7f7060069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:37:59.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.535+0000 7f708219b700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7060066f80 0x7f7060069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:37:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.557+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f707c0657e0 con 0x7f707c159920 2023-12-26T21:37:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.561+0000 7f70727fc700 1 -- 172.21.15.131:0/3100235732 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f706c017020 con 0x7f707c159920 2023-12-26T21:37:59.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7060066f80 msgr2=0x7f7060069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7060066f80 0x7f7060069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 msgr2=0x7f707c15aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 0x7f707c15aa40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f706c0011e0 tx=0x7f706c007df0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 shutdown_connections 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c159cf0 0x7f707c15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7060066f80 0x7f7060069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f707c155be0 0x7f707c0c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 --2- 172.21.15.131:0/3100235732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f707c159920 0x7f707c15aa40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 >> 172.21.15.131:0/3100235732 conn(0x7f707c068040 msgr2=0x7f707c0cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 shutdown_connections 2023-12-26T21:37:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.564+0000 7f70843ff700 1 -- 172.21.15.131:0/3100235732 wait complete. 2023-12-26T21:37:59.568 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:37:59.568+0000 7efefb08a1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:37:59.584 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T21:37:59.628 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:37:59.628+0000 7fa5f7ac61c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:37:59.684 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:37:59.684+0000 7efefb08a1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:37:59.717 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:37:59.716+0000 7fa5f7ac61c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:37:59.778 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:37:59.777+0000 7efefb08a1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:37:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.779+0000 7f479319d700 1 Processor -- start 2023-12-26T21:37:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.780+0000 7f479319d700 1 -- start start 2023-12-26T21:37:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.781+0000 7f479319d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c159490 0x7f478c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.781+0000 7f479319d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f478c0cf080 0x7f478c1631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.781+0000 7f479319d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478c0cca40 con 0x7f478c0cb7b0 2023-12-26T21:37:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.781+0000 7f479319d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478c0cbc20 con 0x7f478c0cf080 2023-12-26T21:37:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.781+0000 7f479319d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0cbe30 con 0x7f478c159490 2023-12-26T21:37:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.781+0000 7f479299c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f478c0cf080 0x7f478c1631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.781+0000 7f479199a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f478c0cb7b0 0x7f478c0ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41454/0 (socket says 172.21.15.131:41454) 2023-12-26T21:37:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.781+0000 7f479199a700 1 -- 172.21.15.131:0/3720800329 learned_addr learned my addr 172.21.15.131:0/3720800329 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:37:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.781+0000 7f479219b700 1 --2- 172.21.15.131:0/3720800329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c159490 0x7f478c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.782+0000 7f479299c700 1 -- 172.21.15.131:0/3720800329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c159490 msgr2=0x7f478c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:37:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.782+0000 7f479299c700 1 --2- 172.21.15.131:0/3720800329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c159490 0x7f478c0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.782+0000 7f479299c700 1 -- 172.21.15.131:0/3720800329 >> v1:172.21.15.131:6789/0 conn(0x7f478c0cb7b0 legacy=0x7f478c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.782+0000 7f479299c700 1 -- 172.21.15.131:0/3720800329 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c0cc110 con 0x7f478c0cf080 2023-12-26T21:37:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.782+0000 7f4791199700 1 -- 172.21.15.131:0/3720800329 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787710248 0 0) 0x7f478c0cca40 con 0x7f478c0cb7b0 2023-12-26T21:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.782+0000 7f479299c700 1 --2- 172.21.15.131:0/3720800329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cf080 0x7f478c1631a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4784009e60 tx=0x7f4784009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73899e50d1cf441 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.782+0000 7f4791199700 1 -- 172.21.15.131:0/3720800329 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478400e070 con 0x7f478c0cf080 2023-12-26T21:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.782+0000 7f4791199700 1 -- 172.21.15.131:0/3720800329 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478400bb60 con 0x7f478c0cf080 2023-12-26T21:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.782+0000 7f4791199700 1 -- 172.21.15.131:0/3720800329 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4784012430 con 0x7f478c0cf080 2023-12-26T21:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.783+0000 7f479319d700 1 -- 172.21.15.131:0/3720800329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cf080 msgr2=0x7f478c1631a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.783+0000 7f479319d700 1 --2- 172.21.15.131:0/3720800329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cf080 0x7f478c1631a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4784009e60 tx=0x7f4784009780 comp rx=0 tx=0).stop 2023-12-26T21:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.783+0000 7f479319d700 1 -- 172.21.15.131:0/3720800329 shutdown_connections 2023-12-26T21:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.783+0000 7f479319d700 1 --2- 172.21.15.131:0/3720800329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cf080 0x7f478c1631a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.783+0000 7f479319d700 1 --2- 172.21.15.131:0/3720800329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c159490 0x7f478c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.783+0000 7f479319d700 1 -- 172.21.15.131:0/3720800329 >> 172.21.15.131:0/3720800329 conn(0x7f478c099750 msgr2=0x7f478c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:37:59.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.783+0000 7f479319d700 1 -- 172.21.15.131:0/3720800329 shutdown_connections 2023-12-26T21:37:59.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.783+0000 7f479319d700 1 -- 172.21.15.131:0/3720800329 wait complete. 2023-12-26T21:37:59.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.784+0000 7f479319d700 1 Processor -- start 2023-12-26T21:37:59.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.785+0000 7f479319d700 1 -- start start 2023-12-26T21:37:59.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.785+0000 7f479319d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cb7b0 0x7f478c14aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479319d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c0cf080 0x7f478c09d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479319d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f478c159490 0x7f478c0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479319d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c09dfa0 con 0x7f478c0cb7b0 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479319d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c09dcc0 con 0x7f478c159490 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479319d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0c8710 con 0x7f478c0cf080 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479219b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cb7b0 0x7f478c14aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479219b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cb7b0 0x7f478c14aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58812/0 (socket says 172.21.15.131:58812) 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479219b700 1 -- 172.21.15.131:0/2947610408 learned_addr learned my addr 172.21.15.131:0/2947610408 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479199a700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c0cf080 0x7f478c09d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479299c700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f478c159490 0x7f478c0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479219b700 1 -- 172.21.15.131:0/2947610408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c0cf080 msgr2=0x7f478c09d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479219b700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c0cf080 0x7f478c09d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479219b700 1 -- 172.21.15.131:0/2947610408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f478c159490 msgr2=0x7f478c0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479219b700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f478c159490 0x7f478c0c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479219b700 1 -- 172.21.15.131:0/2947610408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4784009a20 con 0x7f478c0cb7b0 2023-12-26T21:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479299c700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f478c159490 0x7f478c0c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:37:59.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.786+0000 7f479219b700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cb7b0 0x7f478c14aa70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f478800ea00 tx=0x7f4788009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:37:59.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.787+0000 7f47827fc700 1 -- 172.21.15.131:0/2947610408 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478800c850 con 0x7f478c0cb7b0 2023-12-26T21:37:59.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.787+0000 7f47827fc700 1 -- 172.21.15.131:0/2947610408 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478800c9b0 con 0x7f478c0cb7b0 2023-12-26T21:37:59.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.787+0000 7f47827fc700 1 -- 172.21.15.131:0/2947610408 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478800cd30 con 0x7f478c0cb7b0 2023-12-26T21:37:59.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.787+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c0c89f0 con 0x7f478c0cb7b0 2023-12-26T21:37:59.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.787+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c14afb0 con 0x7f478c0cb7b0 2023-12-26T21:37:59.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.788+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478c0c8e90 con 0x7f478c0cb7b0 2023-12-26T21:37:59.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.790+0000 7f47827fc700 1 -- 172.21.15.131:0/2947610408 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 6) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f4788013680 con 0x7f478c0cb7b0 2023-12-26T21:37:59.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.791+0000 7f47827fc700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4770066f80 0x7f4770069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:37:59.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.791+0000 7f47827fc700 1 -- 172.21.15.131:0/2947610408 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4788084b70 con 0x7f478c0cb7b0 2023-12-26T21:37:59.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.791+0000 7f479199a700 1 -- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4770066f80 msgr2=0x7f4770069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:37:59.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.791+0000 7f479199a700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4770066f80 0x7f4770069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:37:59.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.795+0000 7f47827fc700 1 -- 172.21.15.131:0/2947610408 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4788051de0 con 0x7f478c0cb7b0 2023-12-26T21:37:59.799 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:37:59.799+0000 7fa5f7ac61c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:37:59.866 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:37:59.866+0000 7efefb08a1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:37:59.986 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:37:59.986+0000 7fa5f7ac61c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:37:59.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.992+0000 7f479199a700 1 -- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4770066f80 msgr2=0x7f4770069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:37:59.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.992+0000 7f479199a700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4770066f80 0x7f4770069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:37:59.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.994+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f478c04c3c0 con 0x7f478c0cb7b0 2023-12-26T21:37:59.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:37:59.998+0000 7f47827fc700 1 -- 172.21.15.131:0/2947610408 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f47880210c0 con 0x7f478c0cb7b0 2023-12-26T21:38:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4770066f80 msgr2=0x7f4770069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:38:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4770066f80 0x7f4770069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cb7b0 msgr2=0x7f478c14aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cb7b0 0x7f478c14aa70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f478800ea00 tx=0x7f4788009120 comp rx=0 tx=0).stop 2023-12-26T21:38:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 shutdown_connections 2023-12-26T21:38:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f478c159490 0x7f478c0c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f478c0cb7b0 0x7f478c14aa70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4770066f80 0x7f4770069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 --2- 172.21.15.131:0/2947610408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478c0cf080 0x7f478c09d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 >> 172.21.15.131:0/2947610408 conn(0x7f478c099750 msgr2=0x7f478c0b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:00.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 shutdown_connections 2023-12-26T21:38:00.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.001+0000 7f479319d700 1 -- 172.21.15.131:0/2947610408 wait complete. 2023-12-26T21:38:00.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:00.021 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-26T21:38:00.068 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:00.067+0000 7efefb08a1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:38:00.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:00.073 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-12-26T21:38:00.119 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-12-26T21:38:00.162 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:00.162+0000 7fa5f7ac61c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:38:00.246 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:00.246+0000 7fa5f7ac61c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:38:00.251 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:00.251+0000 7efefb08a1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:38:00.338 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:00.337+0000 7efefb08a1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:38:00.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.357+0000 7f93793ff700 1 Processor -- start 2023-12-26T21:38:00.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.357+0000 7f93793ff700 1 -- start start 2023-12-26T21:38:00.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f93793ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9374142f00 0x7f9374069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f93793ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f93740cc590 0x7f93740cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f93793ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9374069640 con 0x7f93740cb340 2023-12-26T21:38:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f93793ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93740cb7c0 con 0x7f93740cc590 2023-12-26T21:38:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f93793ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9374068750 con 0x7f9374142f00 2023-12-26T21:38:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f93727fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f93740cc590 0x7f93740cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f93727fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f93740cc590 0x7f93740cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58818/0 (socket says 172.21.15.131:58818) 2023-12-26T21:38:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f93727fc700 1 -- 172.21.15.131:0/3595078764 learned_addr learned my addr 172.21.15.131:0/3595078764 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f93737fe700 1 --1- 172.21.15.131:0/3595078764 >> v1:172.21.15.131:6789/0 conn(0x7f93740cb340 0x7f93740c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41468/0 (socket says 172.21.15.131:41468) 2023-12-26T21:38:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.359+0000 7f9372ffd700 1 --2- 172.21.15.131:0/3595078764 >> v2:172.21.15.190:3300/0 conn(0x7f9374142f00 0x7f9374069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.360+0000 7f93727fc700 1 -- 172.21.15.131:0/3595078764 >> v2:172.21.15.190:3300/0 conn(0x7f9374142f00 msgr2=0x7f9374069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.360+0000 7f93727fc700 1 --2- 172.21.15.131:0/3595078764 >> v2:172.21.15.190:3300/0 conn(0x7f9374142f00 0x7f9374069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.360+0000 7f93727fc700 1 -- 172.21.15.131:0/3595078764 >> v1:172.21.15.131:6789/0 conn(0x7f93740cb340 legacy=0x7f93740c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.360+0000 7f93727fc700 1 -- 172.21.15.131:0/3595078764 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9374068a30 con 0x7f93740cc590 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.360+0000 7f9371ffb700 1 -- 172.21.15.131:0/3595078764 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558406729 0 0) 0x7f9374069640 con 0x7f93740cb340 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.360+0000 7f93727fc700 1 --2- 172.21.15.131:0/3595078764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 0x7f93740cae00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f937406f5b0 tx=0x7f9368009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823e2564f31b8db0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.360+0000 7f9371ffb700 1 -- 172.21.15.131:0/3595078764 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368009e70 con 0x7f93740cc590 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.360+0000 7f9371ffb700 1 -- 172.21.15.131:0/3595078764 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800b860 con 0x7f93740cc590 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.360+0000 7f9371ffb700 1 -- 172.21.15.131:0/3595078764 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800bc00 con 0x7f93740cc590 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.361+0000 7f93793ff700 1 -- 172.21.15.131:0/3595078764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 msgr2=0x7f93740cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.361+0000 7f93793ff700 1 --2- 172.21.15.131:0/3595078764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 0x7f93740cae00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f937406f5b0 tx=0x7f9368009250 comp rx=0 tx=0).stop 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.361+0000 7f93793ff700 1 -- 172.21.15.131:0/3595078764 shutdown_connections 2023-12-26T21:38:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.361+0000 7f93793ff700 1 --2- 172.21.15.131:0/3595078764 >> v2:172.21.15.190:3300/0 conn(0x7f9374142f00 0x7f9374069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.361+0000 7f93793ff700 1 --2- 172.21.15.131:0/3595078764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 0x7f93740cae00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.361+0000 7f93793ff700 1 -- 172.21.15.131:0/3595078764 >> 172.21.15.131:0/3595078764 conn(0x7f9374063cd0 msgr2=0x7f937406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.361+0000 7f93793ff700 1 -- 172.21.15.131:0/3595078764 shutdown_connections 2023-12-26T21:38:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.361+0000 7f93793ff700 1 -- 172.21.15.131:0/3595078764 wait complete. 2023-12-26T21:38:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.362+0000 7f93793ff700 1 Processor -- start 2023-12-26T21:38:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.362+0000 7f93793ff700 1 -- start start 2023-12-26T21:38:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.363+0000 7f93793ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93740cb340 0x7f937413f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.363+0000 7f93793ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 0x7f93740c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.363+0000 7f93793ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9374142f00 0x7f937415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.363+0000 7f93793ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93740b1b80 con 0x7f93740cc590 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.363+0000 7f93793ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93740c7bf0 con 0x7f93740cb340 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.363+0000 7f93793ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93740b1e20 con 0x7f9374142f00 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93727fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 0x7f93740c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93727fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 0x7f93740c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58824/0 (socket says 172.21.15.131:58824) 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93727fc700 1 -- 172.21.15.131:0/3220161538 learned_addr learned my addr 172.21.15.131:0/3220161538 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f9372ffd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93740cb340 0x7f937413f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93737fe700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9374142f00 0x7f937415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93727fc700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9374142f00 msgr2=0x7f937415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93727fc700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9374142f00 0x7f937415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93727fc700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93740cb340 msgr2=0x7f937413f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93727fc700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93740cb340 0x7f937413f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93727fc700 1 -- 172.21.15.131:0/3220161538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93680094f0 con 0x7f93740cc590 2023-12-26T21:38:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93737fe700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9374142f00 0x7f937415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:38:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.364+0000 7f93727fc700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 0x7f93740c76b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f937406f3d0 tx=0x7f9368017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.365+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800b9d0 con 0x7f93740cc590 2023-12-26T21:38:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.365+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800bb50 con 0x7f93740cc590 2023-12-26T21:38:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.365+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368025670 con 0x7f93740cc590 2023-12-26T21:38:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.365+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937415e580 con 0x7f93740cc590 2023-12-26T21:38:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.365+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937415dfc0 con 0x7f93740cc590 2023-12-26T21:38:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.366+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93740b1fb0 con 0x7f93740cc590 2023-12-26T21:38:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.367+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 6) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f9368023070 con 0x7f93740cc590 2023-12-26T21:38:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.368+0000 7f935affd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f9360069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:00.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.368+0000 7f9372ffd700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 msgr2=0x7f9360069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:38:00.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.368+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9368085c60 con 0x7f93740cc590 2023-12-26T21:38:00.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.368+0000 7f9372ffd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f9360069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:38:00.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.372+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9368052e30 con 0x7f93740cc590 2023-12-26T21:38:00.391 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:00.391+0000 7fa5f7ac61c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:38:00.481 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:00.481+0000 7fa5f7ac61c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:38:00.489 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:00.489+0000 7efefb08a1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:38:00.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.569+0000 7f9372ffd700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 msgr2=0x7f9360069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:38:00.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.569+0000 7f9372ffd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f9360069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:38:00.580 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:00.580+0000 7efefb08a1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:38:00.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.595+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f93740717f0 con 0x7f9360066f80 2023-12-26T21:38:00.662 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:00.662+0000 7fa5f7ac61c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:38:00.763 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:00.762+0000 7efefb08a1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:38:00.809 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:00.809+0000 7fa5f7ac61c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:38:00.912 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:00.911+0000 7efefb08a1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:38:00.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.970+0000 7f9372ffd700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 msgr2=0x7f9360069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:38:00.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:00.970+0000 7f9372ffd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f9360069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:38:01.187 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:01.187+0000 7fa5f7ac61c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:38:01.294 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:01.293+0000 7efefb08a1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:38:01.449 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:01.448+0000 7fa5f7ac61c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:38:01.559 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:01.559+0000 7efefb08a1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:38:01.722 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:01.722+0000 7fa5f7ac61c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:38:01.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:01.771+0000 7f9372ffd700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 msgr2=0x7f9360069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:38:01.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:01.771+0000 7f9372ffd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f9360069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T21:38:01.809 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:01.809+0000 7fa5f7ac61c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:38:01.837 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:01.836+0000 7efefb08a1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:38:01.925 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:01.925+0000 7efefb08a1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:38:02.532 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:02.532+0000 7fa5f7ac61c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:38:02.625 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:02.624+0000 7fa5f7ac61c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:38:02.658 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:02.657+0000 7efefb08a1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:38:02.751 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:02.750+0000 7efefb08a1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:38:02.775 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:02.775+0000 7fa5f7ac61c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:38:02.901 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:02.901+0000 7efefb08a1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:38:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:03.373+0000 7f9372ffd700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 msgr2=0x7f9360069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:38:03.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:03.373+0000 7f9372ffd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f9360069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T21:38:03.395 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:03.395+0000 7fa5f7ac61c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:38:03.523 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:03.523+0000 7efefb08a1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:38:03.580 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:03.579+0000 7fa5f7ac61c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:38:03.710 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:03.710+0000 7efefb08a1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:38:03.764 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:03.763+0000 7fa5f7ac61c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:38:03.903 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:03.903+0000 7efefb08a1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:38:04.604 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:04.603+0000 7fa5f7ac61c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:38:04.742 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:04.741+0000 7efefb08a1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:38:04.874 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:04.873+0000 7fa5f7ac61c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:38:04.964 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:04.964+0000 7fa5f7ac61c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:38:05.013 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.012+0000 7efefb08a1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:38:05.106 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.106+0000 7efefb08a1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:38:05.159 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:38:05.159+0000 7fa5f7ac61c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:38:05.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:05.218+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 7) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f9368053200 con 0x7f93740cc590 2023-12-26T21:38:05.304 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.304+0000 7efefb08a1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:38:05.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:05.313+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 8) v1 ==== 84470+0+0 (secure 0 0 0) 0x7f93680548a0 con 0x7f93740cc590 2023-12-26T21:38:05.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:05.313+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 msgr2=0x7f9360069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:38:05.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:05.313+0000 7f935affd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f9360069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:05.457 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.456+0000 7efe6f9b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:05.457 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.456+0000 7efe6f9b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:05.457 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.456+0000 7efe6f9b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:05.457 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.456+0000 7efe6f9b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:05.457 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.456+0000 7efe6f9b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:05.458 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.458+0000 7efe6c9b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:05.458 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.458+0000 7efe6c9b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:05.458 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.458+0000 7efe6c9b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:05.458 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.458+0000 7efe6c9b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:05.458 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:38:05.458+0000 7efe6c9b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:38:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.317+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mon.0 v2:172.21.15.131:3300/0 9 ==== mgrmap(e 9) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f9368053ad0 con 0x7f93740cc590 2023-12-26T21:38:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.318+0000 7f935affd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f93600693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.318+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f93740717f0 con 0x7f9360066f80 2023-12-26T21:38:06.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.320+0000 7f9372ffd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f93600693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:06.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.322+0000 7f9372ffd700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f93600693b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9364009690 tx=0x7f93640091a0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:06.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.417+0000 7f935affd700 1 -- 172.21.15.131:0/3220161538 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f93740717f0 con 0x7f9360066f80 2023-12-26T21:38:06.418 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir0 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 msgr2=0x7f93600693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f93600693b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9364009690 tx=0x7f93640091a0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 msgr2=0x7f93740c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 0x7f93740c76b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f937406f3d0 tx=0x7f9368017000 comp rx=0 tx=0).stop 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 shutdown_connections 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9360066f80 0x7f93600693b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9374142f00 0x7f937415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93740cb340 0x7f937413f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 --2- 172.21.15.131:0/3220161538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cc590 0x7f93740c76b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 >> 172.21.15.131:0/3220161538 conn(0x7f9374063cd0 msgr2=0x7f93740c4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 shutdown_connections 2023-12-26T21:38:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.420+0000 7f93793ff700 1 -- 172.21.15.131:0/3220161538 wait complete. 2023-12-26T21:38:06.440 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-12-26T21:38:06.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:06.440 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-12-26T21:38:06.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-12-26T21:38:06.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.684+0000 7f512b5ff700 1 Processor -- start 2023-12-26T21:38:06.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.684+0000 7f512b5ff700 1 -- start start 2023-12-26T21:38:06.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.685+0000 7f512b5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f51240c9110 0x7f512415bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:06.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.685+0000 7f512b5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51240af040 con 0x7f51240c8d40 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.685+0000 7f512b5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51240ae0e0 con 0x7f5124156e50 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.685+0000 7f512b5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51241579c0 con 0x7f51240c9110 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.685+0000 7f512939b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f51240c8d40 0x7f51240ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47102/0 (socket says 172.21.15.131:47102) 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.685+0000 7f512939b700 1 -- 172.21.15.131:0/1112740388 learned_addr learned my addr 172.21.15.131:0/1112740388 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.686+0000 7f5128b9a700 1 --2- 172.21.15.131:0/1112740388 >> v2:172.21.15.190:3300/0 conn(0x7f51240c9110 0x7f512415bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.686+0000 7f511bfff700 1 -- 172.21.15.131:0/1112740388 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898692804 0 0) 0x7f51240af040 con 0x7f51240c8d40 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.686+0000 7f511bfff700 1 -- 172.21.15.131:0/1112740388 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5110003570 con 0x7f51240c8d40 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.686+0000 7f511bfff700 1 -- 172.21.15.131:0/1112740388 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847290463 0 0) 0x7f51240ae0e0 con 0x7f5124156e50 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.686+0000 7f511bfff700 1 -- 172.21.15.131:0/1112740388 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51240af040 con 0x7f5124156e50 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.686+0000 7f5128b9a700 1 -- 172.21.15.131:0/1112740388 >> v1:172.21.15.190:6789/0 conn(0x7f5124156e50 legacy=0x7f51241572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.686+0000 7f511bfff700 1 -- 172.21.15.131:0/1112740388 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4095355121 0 0) 0x7f5110003570 con 0x7f51240c8d40 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.686+0000 7f5128b9a700 1 -- 172.21.15.131:0/1112740388 >> v1:172.21.15.131:6789/0 conn(0x7f51240c8d40 legacy=0x7f51240ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.686+0000 7f5128b9a700 1 -- 172.21.15.131:0/1112740388 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240ae3c0 con 0x7f51240c9110 2023-12-26T21:38:06.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.687+0000 7f5128b9a700 1 --2- 172.21.15.131:0/1112740388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c9110 0x7f512415bc30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f51140098e0 tx=0x7f51140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba15c506ee205517 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.687+0000 7f511bfff700 1 -- 172.21.15.131:0/1112740388 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120001470 con 0x7f51240c9110 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.687+0000 7f511bfff700 1 -- 172.21.15.131:0/1112740388 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5114011420 con 0x7f51240c9110 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.687+0000 7f511bfff700 1 -- 172.21.15.131:0/1112740388 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51140117c0 con 0x7f51240c9110 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.688+0000 7f512b5ff700 1 -- 172.21.15.131:0/1112740388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c9110 msgr2=0x7f512415bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.688+0000 7f512b5ff700 1 --2- 172.21.15.131:0/1112740388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c9110 0x7f512415bc30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f51140098e0 tx=0x7f51140091a0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.688+0000 7f512b5ff700 1 -- 172.21.15.131:0/1112740388 shutdown_connections 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.688+0000 7f512b5ff700 1 --2- 172.21.15.131:0/1112740388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c9110 0x7f512415bc30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.688+0000 7f512b5ff700 1 -- 172.21.15.131:0/1112740388 >> 172.21.15.131:0/1112740388 conn(0x7f51240675b0 msgr2=0x7f51240679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.688+0000 7f512b5ff700 1 -- 172.21.15.131:0/1112740388 shutdown_connections 2023-12-26T21:38:06.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.688+0000 7f512b5ff700 1 -- 172.21.15.131:0/1112740388 wait complete. 2023-12-26T21:38:06.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.689+0000 7f512b5ff700 1 Processor -- start 2023-12-26T21:38:06.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.690+0000 7f512b5ff700 1 -- start start 2023-12-26T21:38:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.690+0000 7f512b5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c8d40 0x7f51240c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.690+0000 7f512b5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51240c9110 0x7f512415a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.690+0000 7f512b5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5124156e50 0x7f51240c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.690+0000 7f512b5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51241541f0 con 0x7f51240c9110 2023-12-26T21:38:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.690+0000 7f512b5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51240aaee0 con 0x7f51240c8d40 2023-12-26T21:38:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.690+0000 7f512b5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51240ac070 con 0x7f5124156e50 2023-12-26T21:38:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f5128b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51240c9110 0x7f512415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f5128b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51240c9110 0x7f512415a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52668/0 (socket says 172.21.15.131:52668) 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f5128b9a700 1 -- 172.21.15.131:0/3518782880 learned_addr learned my addr 172.21.15.131:0/3518782880 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f512939b700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c8d40 0x7f51240c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f5128b9a700 1 -- 172.21.15.131:0/3518782880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5124156e50 msgr2=0x7f51240c1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f5128b9a700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5124156e50 0x7f51240c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f5128b9a700 1 -- 172.21.15.131:0/3518782880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c8d40 msgr2=0x7f51240c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f5128b9a700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c8d40 0x7f51240c01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f5128b9a700 1 -- 172.21.15.131:0/3518782880 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114009420 con 0x7f51240c9110 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f512939b700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c8d40 0x7f51240c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.691+0000 7f5128b9a700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51240c9110 0x7f512415a130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f51240c8a80 tx=0x7f5114016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.692+0000 7f51197fa700 1 -- 172.21.15.131:0/3518782880 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511401cc90 con 0x7f51240c9110 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.692+0000 7f51197fa700 1 -- 172.21.15.131:0/3518782880 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511401cdf0 con 0x7f51240c9110 2023-12-26T21:38:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.692+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51240aa2d0 con 0x7f51240c9110 2023-12-26T21:38:06.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.692+0000 7f51197fa700 1 -- 172.21.15.131:0/3518782880 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511401a6a0 con 0x7f51240c9110 2023-12-26T21:38:06.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.692+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51240aa890 con 0x7f51240c9110 2023-12-26T21:38:06.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.693+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51240ab050 con 0x7f51240c9110 2023-12-26T21:38:06.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.694+0000 7f51197fa700 1 -- 172.21.15.131:0/3518782880 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f511401aa50 con 0x7f51240c9110 2023-12-26T21:38:06.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.694+0000 7f51197fa700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5108066dd0 0x7f5108069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:06.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.694+0000 7f51197fa700 1 -- 172.21.15.131:0/3518782880 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f5114085d90 con 0x7f51240c9110 2023-12-26T21:38:06.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.695+0000 7f512939b700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5108066dd0 0x7f5108069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:06.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.695+0000 7f512939b700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5108066dd0 0x7f5108069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51200015a0 tx=0x7f5120009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:06.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.699+0000 7f51197fa700 1 -- 172.21.15.131:0/3518782880 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51140531b0 con 0x7f51240c9110 2023-12-26T21:38:06.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.916+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f51240717f0 con 0x7f5108066dd0 2023-12-26T21:38:06.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.988+0000 7f51197fa700 1 -- 172.21.15.131:0/3518782880 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f51240717f0 con 0x7f5108066dd0 2023-12-26T21:38:06.989 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir1 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5108066dd0 msgr2=0x7f5108069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5108066dd0 0x7f5108069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51200015a0 tx=0x7f5120009040 comp rx=0 tx=0).stop 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51240c9110 msgr2=0x7f512415a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51240c9110 0x7f512415a130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f51240c8a80 tx=0x7f5114016000 comp rx=0 tx=0).stop 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 shutdown_connections 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5108066dd0 0x7f5108069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5124156e50 0x7f51240c1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51240c8d40 0x7f51240c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 --2- 172.21.15.131:0/3518782880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51240c9110 0x7f512415a130 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 >> 172.21.15.131:0/3518782880 conn(0x7f51240675b0 msgr2=0x7f51241562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 shutdown_connections 2023-12-26T21:38:06.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:06.991+0000 7f512b5ff700 1 -- 172.21.15.131:0/3518782880 wait complete. 2023-12-26T21:38:07.010 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-12-26T21:38:07.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:07.010 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-12-26T21:38:07.056 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-12-26T21:38:07.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.251+0000 7f6e12fff700 1 Processor -- start 2023-12-26T21:38:07.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.251+0000 7f6e12fff700 1 -- start start 2023-12-26T21:38:07.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.252+0000 7f6e12fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e0c1597e0 0x7f6e0c0675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.252+0000 7f6e12fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c0aa1f0 0x7f6e0c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.252+0000 7f6e12fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c15a750 con 0x7f6e0c067af0 2023-12-26T21:38:07.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.252+0000 7f6e12fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c159bb0 con 0x7f6e0c1597e0 2023-12-26T21:38:07.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.252+0000 7f6e12fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c068070 con 0x7f6e0c0aa1f0 2023-12-26T21:38:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e10d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e0c1597e0 0x7f6e0c0675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e1159c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c0aa1f0 0x7f6e0c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e10d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e0c1597e0 0x7f6e0c0675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:49772/0 (socket says 172.21.15.131:49772) 2023-12-26T21:38:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e10d9b700 1 -- 172.21.15.131:0/2163596353 learned_addr learned my addr 172.21.15.131:0/2163596353 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e0bfff700 1 --1- 172.21.15.131:0/2163596353 >> v1:172.21.15.190:6789/0 conn(0x7f6e0c067af0 0x7f6e0c0a9ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:35814/0 (socket says 172.21.15.131:35814) 2023-12-26T21:38:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e10d9b700 1 -- 172.21.15.131:0/2163596353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c0aa1f0 msgr2=0x7f6e0c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e10d9b700 1 --2- 172.21.15.131:0/2163596353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c0aa1f0 0x7f6e0c15d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e10d9b700 1 -- 172.21.15.131:0/2163596353 >> v1:172.21.15.190:6789/0 conn(0x7f6e0c067af0 legacy=0x7f6e0c0a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e10d9b700 1 -- 172.21.15.131:0/2163596353 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c159e20 con 0x7f6e0c1597e0 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.253+0000 7f6e0b7fe700 1 -- 172.21.15.131:0/2163596353 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715778099 0 0) 0x7f6e0c15a750 con 0x7f6e0c067af0 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.254+0000 7f6e10d9b700 1 --2- 172.21.15.131:0/2163596353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e0c1597e0 0x7f6e0c0675b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6e000099b0 tx=0x7f6e00009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=141ec94991ea35e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.254+0000 7f6e0b7fe700 1 -- 172.21.15.131:0/2163596353 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00009ea0 con 0x7f6e0c1597e0 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.254+0000 7f6e0b7fe700 1 -- 172.21.15.131:0/2163596353 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0000b860 con 0x7f6e0c1597e0 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.254+0000 7f6e0b7fe700 1 -- 172.21.15.131:0/2163596353 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000bc00 con 0x7f6e0c1597e0 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.254+0000 7f6e12fff700 1 -- 172.21.15.131:0/2163596353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e0c1597e0 msgr2=0x7f6e0c0675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.254+0000 7f6e12fff700 1 --2- 172.21.15.131:0/2163596353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e0c1597e0 0x7f6e0c0675b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6e000099b0 tx=0x7f6e00009250 comp rx=0 tx=0).stop 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.255+0000 7f6e12fff700 1 -- 172.21.15.131:0/2163596353 shutdown_connections 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.255+0000 7f6e12fff700 1 --2- 172.21.15.131:0/2163596353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c0aa1f0 0x7f6e0c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.255+0000 7f6e12fff700 1 --2- 172.21.15.131:0/2163596353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e0c1597e0 0x7f6e0c0675b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.255+0000 7f6e12fff700 1 -- 172.21.15.131:0/2163596353 >> 172.21.15.131:0/2163596353 conn(0x7f6e0c06b1c0 msgr2=0x7f6e0c0aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.255+0000 7f6e12fff700 1 -- 172.21.15.131:0/2163596353 shutdown_connections 2023-12-26T21:38:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.255+0000 7f6e12fff700 1 -- 172.21.15.131:0/2163596353 wait complete. 2023-12-26T21:38:07.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.256+0000 7f6e12fff700 1 Processor -- start 2023-12-26T21:38:07.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.256+0000 7f6e12fff700 1 -- start start 2023-12-26T21:38:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e12fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e0c067af0 0x7f6e0c152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e12fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e0c0aa1f0 0x7f6e0c1699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e12fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c1597e0 0x7f6e0c0ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e12fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0ad100 con 0x7f6e0c067af0 2023-12-26T21:38:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e12fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0acdd0 con 0x7f6e0c0aa1f0 2023-12-26T21:38:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e12fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c16a3f0 con 0x7f6e0c1597e0 2023-12-26T21:38:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e10d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e0c067af0 0x7f6e0c152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e10d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e0c067af0 0x7f6e0c152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52678/0 (socket says 172.21.15.131:52678) 2023-12-26T21:38:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e10d9b700 1 -- 172.21.15.131:0/2307436856 learned_addr learned my addr 172.21.15.131:0/2307436856 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.257+0000 7f6e0bfff700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e0c0aa1f0 0x7f6e0c1699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e1159c700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c1597e0 0x7f6e0c0ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e10d9b700 1 -- 172.21.15.131:0/2307436856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c1597e0 msgr2=0x7f6e0c0ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e10d9b700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c1597e0 0x7f6e0c0ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e10d9b700 1 -- 172.21.15.131:0/2307436856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e0c0aa1f0 msgr2=0x7f6e0c1699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e10d9b700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e0c0aa1f0 0x7f6e0c1699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e10d9b700 1 -- 172.21.15.131:0/2307436856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e000094f0 con 0x7f6e0c067af0 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e10d9b700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e0c067af0 0x7f6e0c152130 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c06e290 tx=0x7f6e00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e08ff9700 1 -- 172.21.15.131:0/2307436856 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00000d10 con 0x7f6e0c067af0 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e08ff9700 1 -- 172.21.15.131:0/2307436856 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0000bbe0 con 0x7f6e0c067af0 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c16a6d0 con 0x7f6e0c067af0 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.258+0000 7f6e08ff9700 1 -- 172.21.15.131:0/2307436856 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00011670 con 0x7f6e0c067af0 2023-12-26T21:38:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.259+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c1532c0 con 0x7f6e0c067af0 2023-12-26T21:38:07.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.259+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0c1528b0 con 0x7f6e0c067af0 2023-12-26T21:38:07.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.260+0000 7f6e08ff9700 1 -- 172.21.15.131:0/2307436856 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f6e00011aa0 con 0x7f6e0c067af0 2023-12-26T21:38:07.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.261+0000 7f6e08ff9700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6df0066dd0 0x7f6df0069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.261+0000 7f6e08ff9700 1 -- 172.21.15.131:0/2307436856 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f6e00085900 con 0x7f6e0c067af0 2023-12-26T21:38:07.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.261+0000 7f6e0bfff700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6df0066dd0 0x7f6df0069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.262+0000 7f6e0bfff700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6df0066dd0 0x7f6df0069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0ac660 tx=0x7f6df8007ef0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:07.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.265+0000 7f6e08ff9700 1 -- 172.21.15.131:0/2307436856 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e00052c50 con 0x7f6e0c067af0 2023-12-26T21:38:07.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.321+0000 7f6e08ff9700 1 -- 172.21.15.131:0/2307436856 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f6e00053020 con 0x7f6e0c067af0 2023-12-26T21:38:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.493+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e0c0ad240 con 0x7f6df0066dd0 2023-12-26T21:38:07.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.569+0000 7f6e08ff9700 1 -- 172.21.15.131:0/2307436856 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6df8004cc0 con 0x7f6df0066dd0 2023-12-26T21:38:07.569 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir2 2023-12-26T21:38:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6df0066dd0 msgr2=0x7f6df0069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6df0066dd0 0x7f6df0069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0ac660 tx=0x7f6df8007ef0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e0c067af0 msgr2=0x7f6e0c152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e0c067af0 0x7f6e0c152130 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c06e290 tx=0x7f6e00017000 comp rx=0 tx=0).stop 2023-12-26T21:38:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 shutdown_connections 2023-12-26T21:38:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6df0066dd0 0x7f6df0069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e0c1597e0 0x7f6e0c0ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e0c067af0 0x7f6e0c152130 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 --2- 172.21.15.131:0/2307436856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e0c0aa1f0 0x7f6e0c1699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 >> 172.21.15.131:0/2307436856 conn(0x7f6e0c06b1c0 msgr2=0x7f6e0c06c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:07.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 shutdown_connections 2023-12-26T21:38:07.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.571+0000 7f6e12fff700 1 -- 172.21.15.131:0/2307436856 wait complete. 2023-12-26T21:38:07.590 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-12-26T21:38:07.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:07.590 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-12-26T21:38:07.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-12-26T21:38:07.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.871+0000 7f83491ff700 1 Processor -- start 2023-12-26T21:38:07.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.871+0000 7f83491ff700 1 -- start start 2023-12-26T21:38:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.872+0000 7f83491ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 0x7f8344069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.873+0000 7f83491ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8344069640 con 0x7f83440cb510 2023-12-26T21:38:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.873+0000 7f83491ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8344068750 con 0x7f83440cc590 2023-12-26T21:38:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.873+0000 7f83491ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8344068a30 con 0x7f8344142f00 2023-12-26T21:38:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.873+0000 7f834359e700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f83440cb510 0x7f83440c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47104/0 (socket says 172.21.15.131:47104) 2023-12-26T21:38:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.873+0000 7f834359e700 1 -- 172.21.15.131:0/3336536208 learned_addr learned my addr 172.21.15.131:0/3336536208 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:38:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.873+0000 7f8342d9d700 1 --2- 172.21.15.131:0/3336536208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 0x7f8344069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.873+0000 7f8341d9b700 1 -- 172.21.15.131:0/3336536208 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259106409 0 0) 0x7f8344069640 con 0x7f83440cb510 2023-12-26T21:38:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.873+0000 7f8341d9b700 1 -- 172.21.15.131:0/3336536208 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f832c003570 con 0x7f83440cb510 2023-12-26T21:38:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.874+0000 7f8341d9b700 1 -- 172.21.15.131:0/3336536208 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222630619 0 0) 0x7f8344068750 con 0x7f83440cc590 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.874+0000 7f8341d9b700 1 -- 172.21.15.131:0/3336536208 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8344069640 con 0x7f83440cc590 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.874+0000 7f8342d9d700 1 -- 172.21.15.131:0/3336536208 >> v1:172.21.15.190:6789/0 conn(0x7f83440cc590 legacy=0x7f83440cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.874+0000 7f8342d9d700 1 -- 172.21.15.131:0/3336536208 >> v1:172.21.15.131:6789/0 conn(0x7f83440cb510 legacy=0x7f83440c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.874+0000 7f8342d9d700 1 -- 172.21.15.131:0/3336536208 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344068d10 con 0x7f8344142f00 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.874+0000 7f8342d9d700 1 --2- 172.21.15.131:0/3336536208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 0x7f8344069ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8334009910 tx=0x7f83340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57557c977d78ea79 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.874+0000 7f8341d9b700 1 -- 172.21.15.131:0/3336536208 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334011630 con 0x7f8344142f00 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.874+0000 7f8341d9b700 1 -- 172.21.15.131:0/3336536208 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334011790 con 0x7f8344142f00 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.874+0000 7f8341d9b700 1 -- 172.21.15.131:0/3336536208 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334011b30 con 0x7f8344142f00 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.875+0000 7f83491ff700 1 -- 172.21.15.131:0/3336536208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 msgr2=0x7f8344069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.875+0000 7f83491ff700 1 --2- 172.21.15.131:0/3336536208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 0x7f8344069ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8334009910 tx=0x7f83340091a0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.875+0000 7f83491ff700 1 -- 172.21.15.131:0/3336536208 shutdown_connections 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.875+0000 7f83491ff700 1 --2- 172.21.15.131:0/3336536208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 0x7f8344069ad0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.875+0000 7f83491ff700 1 -- 172.21.15.131:0/3336536208 >> 172.21.15.131:0/3336536208 conn(0x7f8344063cd0 msgr2=0x7f834406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:07.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.875+0000 7f83491ff700 1 -- 172.21.15.131:0/3336536208 shutdown_connections 2023-12-26T21:38:07.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.875+0000 7f83491ff700 1 -- 172.21.15.131:0/3336536208 wait complete. 2023-12-26T21:38:07.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.876+0000 7f83491ff700 1 Processor -- start 2023-12-26T21:38:07.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.877+0000 7f83491ff700 1 -- start start 2023-12-26T21:38:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.877+0000 7f83491ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83440cb510 0x7f834415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f83491ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83440cc590 0x7f83440ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f83491ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 0x7f83440ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f83491ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83440c7560 con 0x7f83440cb510 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f83491ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834416ade0 con 0x7f83440cc590 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f83491ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834415dca0 con 0x7f8344142f00 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f8342d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83440cb510 0x7f834415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f8342d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83440cb510 0x7f834415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52694/0 (socket says 172.21.15.131:52694) 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f8342d9d700 1 -- 172.21.15.131:0/321211707 learned_addr learned my addr 172.21.15.131:0/321211707 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f834359e700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 0x7f83440ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f834259c700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83440cc590 0x7f83440ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f8342d9d700 1 -- 172.21.15.131:0/321211707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 msgr2=0x7f83440ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f8342d9d700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 0x7f83440ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f8342d9d700 1 -- 172.21.15.131:0/321211707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83440cc590 msgr2=0x7f83440ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f8342d9d700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83440cc590 0x7f83440ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.878+0000 7f8342d9d700 1 -- 172.21.15.131:0/321211707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334009420 con 0x7f83440cb510 2023-12-26T21:38:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.879+0000 7f8342d9d700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83440cb510 0x7f834415e130 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f834406f3d0 tx=0x7f8334017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.879+0000 7f832affd700 1 -- 172.21.15.131:0/321211707 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334020070 con 0x7f83440cb510 2023-12-26T21:38:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.879+0000 7f832affd700 1 -- 172.21.15.131:0/321211707 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334000bc0 con 0x7f83440cb510 2023-12-26T21:38:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.879+0000 7f832affd700 1 -- 172.21.15.131:0/321211707 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334011b30 con 0x7f83440cb510 2023-12-26T21:38:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.879+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834416ab00 con 0x7f83440cb510 2023-12-26T21:38:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.879+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834415d930 con 0x7f83440cb510 2023-12-26T21:38:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.880+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83440717f0 con 0x7f83440cb510 2023-12-26T21:38:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.881+0000 7f832affd700 1 -- 172.21.15.131:0/321211707 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f8334024070 con 0x7f83440cb510 2023-12-26T21:38:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.882+0000 7f832affd700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8330067130 0x7f83300695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.882+0000 7f832affd700 1 -- 172.21.15.131:0/321211707 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f8334085dd0 con 0x7f83440cb510 2023-12-26T21:38:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.882+0000 7f834259c700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8330067130 0x7f83300695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:07.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.883+0000 7f834259c700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8330067130 0x7f83300695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f834415bb40 tx=0x7f8338009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:07.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:07.886+0000 7f832affd700 1 -- 172.21.15.131:0/321211707 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8334052ce0 con 0x7f83440cb510 2023-12-26T21:38:08.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.105+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f834415dde0 con 0x7f8330067130 2023-12-26T21:38:08.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.184+0000 7f832affd700 1 -- 172.21.15.131:0/321211707 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f83380034a0 con 0x7f8330067130 2023-12-26T21:38:08.185 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir3 2023-12-26T21:38:08.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8330067130 msgr2=0x7f83300695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:08.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8330067130 0x7f83300695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f834415bb40 tx=0x7f8338009040 comp rx=0 tx=0).stop 2023-12-26T21:38:08.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83440cb510 msgr2=0x7f834415e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:08.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83440cb510 0x7f834415e130 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f834406f3d0 tx=0x7f8334017000 comp rx=0 tx=0).stop 2023-12-26T21:38:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 shutdown_connections 2023-12-26T21:38:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8330067130 0x7f83300695e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8344142f00 0x7f83440ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83440cb510 0x7f834415e130 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 --2- 172.21.15.131:0/321211707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83440cc590 0x7f83440ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.186+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 >> 172.21.15.131:0/321211707 conn(0x7f8344063cd0 msgr2=0x7f83441408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.187+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 shutdown_connections 2023-12-26T21:38:08.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.187+0000 7f83491ff700 1 -- 172.21.15.131:0/321211707 wait complete. 2023-12-26T21:38:08.205 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-12-26T21:38:08.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:08.205 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-12-26T21:38:08.251 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-12-26T21:38:08.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.485+0000 7f9f049ff700 1 Processor -- start 2023-12-26T21:38:08.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.486+0000 7f9f049ff700 1 -- start start 2023-12-26T21:38:08.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9f049ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f00142f00 0x7f9f00069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:08.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9f049ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f000cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:08.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9f049ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f000cb340 0x7f9f000c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:08.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9f049ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f000cb7c0 con 0x7f9f000cb340 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9f049ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f00069640 con 0x7f9f00142f00 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9f049ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00068750 con 0x7f9f000cc590 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9efed9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f000cb340 0x7f9f000c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9efed9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f000cb340 0x7f9f000c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52698/0 (socket says 172.21.15.131:52698) 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9efed9d700 1 -- 172.21.15.131:0/4165320670 learned_addr learned my addr 172.21.15.131:0/4165320670 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9efe59c700 1 --2- 172.21.15.131:0/4165320670 >> v2:172.21.15.190:3300/0 conn(0x7f9f00142f00 0x7f9f00069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.487+0000 7f9efdd9b700 1 --2- 172.21.15.131:0/4165320670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f000cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efed9d700 1 -- 172.21.15.131:0/4165320670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 msgr2=0x7f9f000cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efed9d700 1 --2- 172.21.15.131:0/4165320670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f000cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efed9d700 1 -- 172.21.15.131:0/4165320670 >> v2:172.21.15.190:3300/0 conn(0x7f9f00142f00 msgr2=0x7f9f00069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efed9d700 1 --2- 172.21.15.131:0/4165320670 >> v2:172.21.15.190:3300/0 conn(0x7f9f00142f00 0x7f9f00069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efed9d700 1 -- 172.21.15.131:0/4165320670 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f00068a30 con 0x7f9f000cb340 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efe59c700 1 --2- 172.21.15.131:0/4165320670 >> v2:172.21.15.190:3300/0 conn(0x7f9f00142f00 0x7f9f00069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:38:08.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efdd9b700 1 --2- 172.21.15.131:0/4165320670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f000cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efed9d700 1 --2- 172.21.15.131:0/4165320670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f000cb340 0x7f9f000c8a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9f0006f5b0 tx=0x7f9ef8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c60d89d02f46e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efd59a700 1 -- 172.21.15.131:0/4165320670 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800e070 con 0x7f9f000cb340 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efd59a700 1 -- 172.21.15.131:0/4165320670 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef800ba50 con 0x7f9f000cb340 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.488+0000 7f9efd59a700 1 -- 172.21.15.131:0/4165320670 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80092e0 con 0x7f9f000cb340 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.489+0000 7f9f049ff700 1 -- 172.21.15.131:0/4165320670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f000cb340 msgr2=0x7f9f000c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.489+0000 7f9f049ff700 1 --2- 172.21.15.131:0/4165320670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f000cb340 0x7f9f000c8a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9f0006f5b0 tx=0x7f9ef8009840 comp rx=0 tx=0).stop 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.489+0000 7f9f049ff700 1 -- 172.21.15.131:0/4165320670 shutdown_connections 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.489+0000 7f9f049ff700 1 --2- 172.21.15.131:0/4165320670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f000cb340 0x7f9f000c8a40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.489+0000 7f9f049ff700 1 --2- 172.21.15.131:0/4165320670 >> v2:172.21.15.190:3300/0 conn(0x7f9f00142f00 0x7f9f00069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.489+0000 7f9f049ff700 1 --2- 172.21.15.131:0/4165320670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f000cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.489+0000 7f9f049ff700 1 -- 172.21.15.131:0/4165320670 >> 172.21.15.131:0/4165320670 conn(0x7f9f00063cd0 msgr2=0x7f9f0006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.489+0000 7f9f049ff700 1 -- 172.21.15.131:0/4165320670 shutdown_connections 2023-12-26T21:38:08.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.489+0000 7f9f049ff700 1 -- 172.21.15.131:0/4165320670 wait complete. 2023-12-26T21:38:08.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.491+0000 7f9f049ff700 1 Processor -- start 2023-12-26T21:38:08.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.491+0000 7f9f049ff700 1 -- start start 2023-12-26T21:38:08.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9f049ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f000cb340 0x7f9f00169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9f049ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f0016c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9f049ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f00142f00 0x7f9f000c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9f049ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0015e730 con 0x7f9f00142f00 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9f049ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000c5d60 con 0x7f9f000cb340 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9f049ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000c6020 con 0x7f9f000cc590 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9efed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f00142f00 0x7f9f000c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9efed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f00142f00 0x7f9f000c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52706/0 (socket says 172.21.15.131:52706) 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9efed9d700 1 -- 172.21.15.131:0/1207605442 learned_addr learned my addr 172.21.15.131:0/1207605442 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9efe59c700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f000cb340 0x7f9f00169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.492+0000 7f9efdd9b700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f0016c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:08.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9efed9d700 1 -- 172.21.15.131:0/1207605442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 msgr2=0x7f9f0016c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9efed9d700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f0016c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9efed9d700 1 -- 172.21.15.131:0/1207605442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f000cb340 msgr2=0x7f9f00169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9efed9d700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f000cb340 0x7f9f00169b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9efed9d700 1 -- 172.21.15.131:0/1207605442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef8009ae0 con 0x7f9f00142f00 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9efdd9b700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f0016c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9efed9d700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f00142f00 0x7f9f000c8710 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9f0006f3d0 tx=0x7f9ef8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9eee7fc700 1 -- 172.21.15.131:0/1207605442 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80032f0 con 0x7f9f00142f00 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9eee7fc700 1 -- 172.21.15.131:0/1207605442 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef8008d90 con 0x7f9f00142f00 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f000c6300 con 0x7f9f00142f00 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9eee7fc700 1 -- 172.21.15.131:0/1207605442 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80266b0 con 0x7f9f00142f00 2023-12-26T21:38:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.493+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f000c68c0 con 0x7f9f00142f00 2023-12-26T21:38:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.494+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f000717f0 con 0x7f9f00142f00 2023-12-26T21:38:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.495+0000 7f9eee7fc700 1 -- 172.21.15.131:0/1207605442 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f9ef8003040 con 0x7f9f00142f00 2023-12-26T21:38:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.496+0000 7f9eee7fc700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9ee0067160 0x7f9ee0069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.496+0000 7f9eee7fc700 1 -- 172.21.15.131:0/1207605442 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f9ef8086990 con 0x7f9f00142f00 2023-12-26T21:38:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.496+0000 7f9efe59c700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9ee0067160 0x7f9ee0069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.497+0000 7f9efe59c700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9ee0067160 0x7f9ee0069610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0006fd0 tx=0x7f9ef0008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.500+0000 7f9eee7fc700 1 -- 172.21.15.131:0/1207605442 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ef8053770 con 0x7f9f00142f00 2023-12-26T21:38:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.721+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f000c6190 con 0x7f9ee0067160 2023-12-26T21:38:08.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.791+0000 7f9eee7fc700 1 -- 172.21.15.131:0/1207605442 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9f000c6190 con 0x7f9ee0067160 2023-12-26T21:38:08.792 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir4 2023-12-26T21:38:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9ee0067160 msgr2=0x7f9ee0069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9ee0067160 0x7f9ee0069610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0006fd0 tx=0x7f9ef0008040 comp rx=0 tx=0).stop 2023-12-26T21:38:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f00142f00 msgr2=0x7f9f000c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f00142f00 0x7f9f000c8710 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9f0006f3d0 tx=0x7f9ef8018000 comp rx=0 tx=0).stop 2023-12-26T21:38:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 shutdown_connections 2023-12-26T21:38:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f00142f00 0x7f9f000c8710 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f000cb340 0x7f9f00169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9ee0067160 0x7f9ee0069610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 --2- 172.21.15.131:0/1207605442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f000cc590 0x7f9f0016c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:08.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 >> 172.21.15.131:0/1207605442 conn(0x7f9f00063cd0 msgr2=0x7f9f00141f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:08.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 shutdown_connections 2023-12-26T21:38:08.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:08.794+0000 7f9f049ff700 1 -- 172.21.15.131:0/1207605442 wait complete. 2023-12-26T21:38:08.813 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-12-26T21:38:08.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:08.813 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-12-26T21:38:08.859 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-12-26T21:38:09.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.093+0000 7f29a219d700 1 Processor -- start 2023-12-26T21:38:09.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.093+0000 7f29a219d700 1 -- start start 2023-12-26T21:38:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.095+0000 7f29a219d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f299c0ca0a0 0x7f299c0cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.095+0000 7f29a219d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299c06d770 con 0x7f299c0cc500 2023-12-26T21:38:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.095+0000 7f29a219d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299c06ce40 con 0x7f299c0ca470 2023-12-26T21:38:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.095+0000 7f29a219d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c06cc30 con 0x7f299c0ca0a0 2023-12-26T21:38:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.095+0000 7f29a119b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f299c0ca0a0 0x7f299c0cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.095+0000 7f29a119b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f299c0ca0a0 0x7f299c0cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52720/0 (socket says 172.21.15.131:52720) 2023-12-26T21:38:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.095+0000 7f29a119b700 1 -- 172.21.15.131:0/821142243 learned_addr learned my addr 172.21.15.131:0/821142243 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:09.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.095+0000 7f29a199c700 1 --1- 172.21.15.131:0/821142243 >> v1:172.21.15.131:6789/0 conn(0x7f299c0cc500 0x7f299c159a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47110/0 (socket says 172.21.15.131:47110) 2023-12-26T21:38:09.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.096+0000 7f29a119b700 1 -- 172.21.15.131:0/821142243 >> v1:172.21.15.190:6789/0 conn(0x7f299c0ca470 legacy=0x7f299c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.096+0000 7f29a119b700 1 -- 172.21.15.131:0/821142243 >> v1:172.21.15.131:6789/0 conn(0x7f299c0cc500 legacy=0x7f299c159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.096+0000 7f29a119b700 1 -- 172.21.15.131:0/821142243 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c06d120 con 0x7f299c0ca0a0 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.096+0000 7f2993fff700 1 -- 172.21.15.131:0/821142243 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268945133 0 0) 0x7f299c06d770 con 0x7f299c0cc500 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.096+0000 7f29a119b700 1 --2- 172.21.15.131:0/821142243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca0a0 0x7f299c0cab30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f29980099b0 tx=0x7f2998009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ed3f2715e58a50 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.096+0000 7f2993fff700 1 -- 172.21.15.131:0/821142243 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299800b860 con 0x7f299c0ca0a0 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.096+0000 7f2993fff700 1 -- 172.21.15.131:0/821142243 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299800b9c0 con 0x7f299c0ca0a0 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.096+0000 7f2993fff700 1 -- 172.21.15.131:0/821142243 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299800bd60 con 0x7f299c0ca0a0 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.097+0000 7f29a219d700 1 -- 172.21.15.131:0/821142243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca0a0 msgr2=0x7f299c0cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.097+0000 7f29a219d700 1 --2- 172.21.15.131:0/821142243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca0a0 0x7f299c0cab30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f29980099b0 tx=0x7f2998009250 comp rx=0 tx=0).stop 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.097+0000 7f29a219d700 1 -- 172.21.15.131:0/821142243 shutdown_connections 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.097+0000 7f29a219d700 1 --2- 172.21.15.131:0/821142243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca0a0 0x7f299c0cab30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.097+0000 7f29a219d700 1 -- 172.21.15.131:0/821142243 >> 172.21.15.131:0/821142243 conn(0x7f299c06b5f0 msgr2=0x7f299c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.097+0000 7f29a219d700 1 -- 172.21.15.131:0/821142243 shutdown_connections 2023-12-26T21:38:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.097+0000 7f29a219d700 1 -- 172.21.15.131:0/821142243 wait complete. 2023-12-26T21:38:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.098+0000 7f29a219d700 1 Processor -- start 2023-12-26T21:38:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.098+0000 7f29a219d700 1 -- start start 2023-12-26T21:38:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.099+0000 7f29a219d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0ca0a0 0x7f299c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.099+0000 7f29a219d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca470 0x7f299c1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.099+0000 7f29a219d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0cc500 0x7f299c151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.099+0000 7f29a219d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0b11a0 con 0x7f299c0ca470 2023-12-26T21:38:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.099+0000 7f29a219d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c151760 con 0x7f299c0ca0a0 2023-12-26T21:38:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.099+0000 7f29a219d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c154500 con 0x7f299c0cc500 2023-12-26T21:38:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca470 0x7f299c1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca470 0x7f299c1519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52726/0 (socket says 172.21.15.131:52726) 2023-12-26T21:38:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a099a700 1 -- 172.21.15.131:0/621738109 learned_addr learned my addr 172.21.15.131:0/621738109 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a119b700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0ca0a0 0x7f299c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a199c700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0cc500 0x7f299c151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a099a700 1 -- 172.21.15.131:0/621738109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0cc500 msgr2=0x7f299c151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a099a700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0cc500 0x7f299c151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a099a700 1 -- 172.21.15.131:0/621738109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0ca0a0 msgr2=0x7f299c151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a099a700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0ca0a0 0x7f299c151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a099a700 1 -- 172.21.15.131:0/621738109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29980094f0 con 0x7f299c0ca470 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.100+0000 7f29a099a700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca470 0x7f299c1519f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f298c00c7c0 tx=0x7f298c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.101+0000 7f29917fa700 1 -- 172.21.15.131:0/621738109 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c011590 con 0x7f299c0ca470 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.101+0000 7f29917fa700 1 -- 172.21.15.131:0/621738109 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c0116f0 con 0x7f299c0ca470 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.101+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299c1547e0 con 0x7f299c0ca470 2023-12-26T21:38:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.101+0000 7f29917fa700 1 -- 172.21.15.131:0/621738109 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c011ab0 con 0x7f299c0ca470 2023-12-26T21:38:09.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.101+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299c154da0 con 0x7f299c0ca470 2023-12-26T21:38:09.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.102+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299c154f30 con 0x7f299c0ca470 2023-12-26T21:38:09.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.103+0000 7f29917fa700 1 -- 172.21.15.131:0/621738109 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f298c01b3b0 con 0x7f299c0ca470 2023-12-26T21:38:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.104+0000 7f29917fa700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2980067130 0x7f29800695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.104+0000 7f29917fa700 1 -- 172.21.15.131:0/621738109 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f298c083e80 con 0x7f299c0ca470 2023-12-26T21:38:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.104+0000 7f29a119b700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2980067130 0x7f29800695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.105+0000 7f29a119b700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2980067130 0x7f29800695e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f29980099b0 tx=0x7f2998017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:09.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.108+0000 7f29917fa700 1 -- 172.21.15.131:0/621738109 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f298c050cd0 con 0x7f299c0ca470 2023-12-26T21:38:09.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.328+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f299c0b13a0 con 0x7f2980067130 2023-12-26T21:38:09.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.411+0000 7f29917fa700 1 -- 172.21.15.131:0/621738109 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f299c0b13a0 con 0x7f2980067130 2023-12-26T21:38:09.412 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir5 2023-12-26T21:38:09.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.413+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2980067130 msgr2=0x7f29800695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.413+0000 7f29a219d700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2980067130 0x7f29800695e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f29980099b0 tx=0x7f2998017000 comp rx=0 tx=0).stop 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.413+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca470 msgr2=0x7f299c1519f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.413+0000 7f29a219d700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca470 0x7f299c1519f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f298c00c7c0 tx=0x7f298c007f50 comp rx=0 tx=0).stop 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.414+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 shutdown_connections 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.414+0000 7f29a219d700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2980067130 0x7f29800695e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.414+0000 7f29a219d700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0cc500 0x7f299c151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.414+0000 7f29a219d700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0ca0a0 0x7f299c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.414+0000 7f29a219d700 1 --2- 172.21.15.131:0/621738109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c0ca470 0x7f299c1519f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.414+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 >> 172.21.15.131:0/621738109 conn(0x7f299c06b5f0 msgr2=0x7f299c068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.414+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 shutdown_connections 2023-12-26T21:38:09.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.414+0000 7f29a219d700 1 -- 172.21.15.131:0/621738109 wait complete. 2023-12-26T21:38:09.432 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-12-26T21:38:09.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:09.432 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-12-26T21:38:09.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-12-26T21:38:09.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.712+0000 7f376a1ff700 1 Processor -- start 2023-12-26T21:38:09.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.712+0000 7f376a1ff700 1 -- start start 2023-12-26T21:38:09.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.713+0000 7f376a1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37640cbdc0 0x7f37640ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f376a1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37640ce760 con 0x7f37640cce60 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f376a1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37640cdd80 con 0x7f37640cd230 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f376a1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37640cf5a0 con 0x7f37640cbdc0 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f375bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37640cbdc0 0x7f37640ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f375bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37640cbdc0 0x7f37640ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52734/0 (socket says 172.21.15.131:52734) 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f375bfff700 1 -- 172.21.15.131:0/329685560 learned_addr learned my addr 172.21.15.131:0/329685560 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f375bfff700 1 -- 172.21.15.131:0/329685560 >> v1:172.21.15.190:6789/0 conn(0x7f37640cd230 legacy=0x7f37640cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f375bfff700 1 -- 172.21.15.131:0/329685560 >> v1:172.21.15.131:6789/0 conn(0x7f37640cce60 legacy=0x7f37640cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f375bfff700 1 -- 172.21.15.131:0/329685560 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3764159dc0 con 0x7f37640cbdc0 2023-12-26T21:38:09.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.714+0000 7f375bfff700 1 --2- 172.21.15.131:0/329685560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cbdc0 0x7f37640ce220 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f374c0099e0 tx=0x7f374c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd263215b8f074 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:09.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f3762ffd700 1 -- 172.21.15.131:0/329685560 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c00b7e0 con 0x7f37640cbdc0 2023-12-26T21:38:09.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f3762ffd700 1 -- 172.21.15.131:0/329685560 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c00b940 con 0x7f37640cbdc0 2023-12-26T21:38:09.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f3762ffd700 1 -- 172.21.15.131:0/329685560 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c00bce0 con 0x7f37640cbdc0 2023-12-26T21:38:09.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f376a1ff700 1 -- 172.21.15.131:0/329685560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cbdc0 msgr2=0x7f37640ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f376a1ff700 1 --2- 172.21.15.131:0/329685560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cbdc0 0x7f37640ce220 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f374c0099e0 tx=0x7f374c009250 comp rx=0 tx=0).stop 2023-12-26T21:38:09.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f376a1ff700 1 -- 172.21.15.131:0/329685560 shutdown_connections 2023-12-26T21:38:09.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f376a1ff700 1 --2- 172.21.15.131:0/329685560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cbdc0 0x7f37640ce220 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f376a1ff700 1 -- 172.21.15.131:0/329685560 >> 172.21.15.131:0/329685560 conn(0x7f3764099960 msgr2=0x7f376406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:09.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f376a1ff700 1 -- 172.21.15.131:0/329685560 shutdown_connections 2023-12-26T21:38:09.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.715+0000 7f376a1ff700 1 -- 172.21.15.131:0/329685560 wait complete. 2023-12-26T21:38:09.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.717+0000 7f376a1ff700 1 Processor -- start 2023-12-26T21:38:09.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.717+0000 7f376a1ff700 1 -- start start 2023-12-26T21:38:09.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f376a1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37640cbdc0 0x7f3764158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f376a1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cce60 0x7f376416a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f376a1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37640cd230 0x7f376409c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f376a1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376416b120 con 0x7f37640cce60 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f376a1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376409e230 con 0x7f37640cd230 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f376a1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376416ab60 con 0x7f37640cbdc0 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f375bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cce60 0x7f376416a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f375bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cce60 0x7f376416a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52746/0 (socket says 172.21.15.131:52746) 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f375bfff700 1 -- 172.21.15.131:0/2691265227 learned_addr learned my addr 172.21.15.131:0/2691265227 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f3763fff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37640cd230 0x7f376409c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.718+0000 7f37637fe700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37640cbdc0 0x7f3764158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f375bfff700 1 -- 172.21.15.131:0/2691265227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37640cbdc0 msgr2=0x7f3764158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f375bfff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37640cbdc0 0x7f3764158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f375bfff700 1 -- 172.21.15.131:0/2691265227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37640cd230 msgr2=0x7f376409c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:09.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f375bfff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37640cd230 0x7f376409c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f375bfff700 1 -- 172.21.15.131:0/2691265227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0094f0 con 0x7f37640cce60 2023-12-26T21:38:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f375bfff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cce60 0x7f376416a110 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f374c009930 tx=0x7f374c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f375b7fe700 1 -- 172.21.15.131:0/2691265227 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c011d00 con 0x7f37640cce60 2023-12-26T21:38:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f375b7fe700 1 -- 172.21.15.131:0/2691265227 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c020070 con 0x7f37640cce60 2023-12-26T21:38:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f375b7fe700 1 -- 172.21.15.131:0/2691265227 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c01b6a0 con 0x7f37640cce60 2023-12-26T21:38:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376416ae40 con 0x7f37640cce60 2023-12-26T21:38:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.719+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37640c6080 con 0x7f37640cce60 2023-12-26T21:38:09.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.720+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37640717f0 con 0x7f37640cce60 2023-12-26T21:38:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.721+0000 7f375b7fe700 1 -- 172.21.15.131:0/2691265227 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f374c022030 con 0x7f37640cce60 2023-12-26T21:38:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.722+0000 7f375b7fe700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3744067130 0x7f37440695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.722+0000 7f375b7fe700 1 -- 172.21.15.131:0/2691265227 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f374c086900 con 0x7f37640cce60 2023-12-26T21:38:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.722+0000 7f37637fe700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3744067130 0x7f37440695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.723+0000 7f37637fe700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3744067130 0x7f37440695e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3754003790 tx=0x7f3754009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:09.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.726+0000 7f375b7fe700 1 -- 172.21.15.131:0/2691265227 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f374c053750 con 0x7f37640cce60 2023-12-26T21:38:09.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:09.941+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f37640717f0 con 0x7f3744067130 2023-12-26T21:38:10.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.021+0000 7f375b7fe700 1 -- 172.21.15.131:0/2691265227 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f37640717f0 con 0x7f3744067130 2023-12-26T21:38:10.022 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir6 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3744067130 msgr2=0x7f37440695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3744067130 0x7f37440695e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3754003790 tx=0x7f3754009040 comp rx=0 tx=0).stop 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cce60 msgr2=0x7f376416a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cce60 0x7f376416a110 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f374c009930 tx=0x7f374c016000 comp rx=0 tx=0).stop 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 shutdown_connections 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37640cd230 0x7f376409c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3744067130 0x7f37440695e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37640cbdc0 0x7f3764158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 --2- 172.21.15.131:0/2691265227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37640cce60 0x7f376416a110 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 >> 172.21.15.131:0/2691265227 conn(0x7f3764099960 msgr2=0x7f37640c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 shutdown_connections 2023-12-26T21:38:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.024+0000 7f376a1ff700 1 -- 172.21.15.131:0/2691265227 wait complete. 2023-12-26T21:38:10.042 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-12-26T21:38:10.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:10.042 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-12-26T21:38:10.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-12-26T21:38:10.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.323+0000 7f03b39ff700 1 Processor -- start 2023-12-26T21:38:10.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.323+0000 7f03b39ff700 1 -- start start 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.324+0000 7f03b39ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03ac156360 0x7f03ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.324+0000 7f03b39ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac0acfc0 con 0x7f03ac156730 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.324+0000 7f03b39ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac0c91f0 con 0x7f03ac0c8ba0 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.324+0000 7f03b39ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac0c9000 con 0x7f03ac156360 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.324+0000 7f03b179b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03ac156360 0x7f03ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.325+0000 7f03b179b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03ac156360 0x7f03ac06e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52760/0 (socket says 172.21.15.131:52760) 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.325+0000 7f03b179b700 1 -- 172.21.15.131:0/1005170951 learned_addr learned my addr 172.21.15.131:0/1005170951 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.325+0000 7f03b179b700 1 -- 172.21.15.131:0/1005170951 >> v1:172.21.15.190:6789/0 conn(0x7f03ac0c8ba0 legacy=0x7f03ac0ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.325+0000 7f03b179b700 1 -- 172.21.15.131:0/1005170951 >> v1:172.21.15.131:6789/0 conn(0x7f03ac156730 legacy=0x7f03ac0c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.325+0000 7f03b179b700 1 -- 172.21.15.131:0/1005170951 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac0c94d0 con 0x7f03ac156360 2023-12-26T21:38:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.325+0000 7f03b179b700 1 --2- 172.21.15.131:0/1005170951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 0x7f03ac06e770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f03a80099b0 tx=0x7f03a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470e2846c8b5075c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.325+0000 7f03a3fff700 1 -- 172.21.15.131:0/1005170951 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a800b860 con 0x7f03ac156360 2023-12-26T21:38:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.326+0000 7f03a3fff700 1 -- 172.21.15.131:0/1005170951 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a800b9c0 con 0x7f03ac156360 2023-12-26T21:38:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.326+0000 7f03a3fff700 1 -- 172.21.15.131:0/1005170951 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a800bd60 con 0x7f03ac156360 2023-12-26T21:38:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.326+0000 7f03b39ff700 1 -- 172.21.15.131:0/1005170951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 msgr2=0x7f03ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.326+0000 7f03b39ff700 1 --2- 172.21.15.131:0/1005170951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 0x7f03ac06e770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f03a80099b0 tx=0x7f03a8009250 comp rx=0 tx=0).stop 2023-12-26T21:38:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.326+0000 7f03b39ff700 1 -- 172.21.15.131:0/1005170951 shutdown_connections 2023-12-26T21:38:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.326+0000 7f03b39ff700 1 --2- 172.21.15.131:0/1005170951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 0x7f03ac06e770 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.326+0000 7f03b39ff700 1 -- 172.21.15.131:0/1005170951 >> 172.21.15.131:0/1005170951 conn(0x7f03ac0675b0 msgr2=0x7f03ac0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.326+0000 7f03b39ff700 1 -- 172.21.15.131:0/1005170951 shutdown_connections 2023-12-26T21:38:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.326+0000 7f03b39ff700 1 -- 172.21.15.131:0/1005170951 wait complete. 2023-12-26T21:38:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.328+0000 7f03b39ff700 1 Processor -- start 2023-12-26T21:38:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.328+0000 7f03b39ff700 1 -- start start 2023-12-26T21:38:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b39ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03ac0c8ba0 0x7f03ac0c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b39ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 0x7f03ac1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b39ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03ac156730 0x7f03ac15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b39ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac153d30 con 0x7f03ac156360 2023-12-26T21:38:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b39ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0c3290 con 0x7f03ac0c8ba0 2023-12-26T21:38:10.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b39ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0c3530 con 0x7f03ac156730 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b0f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 0x7f03ac1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b0f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 0x7f03ac1537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52772/0 (socket says 172.21.15.131:52772) 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b0f9a700 1 -- 172.21.15.131:0/780740320 learned_addr learned my addr 172.21.15.131:0/780740320 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b0f9a700 1 -- 172.21.15.131:0/780740320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03ac156730 msgr2=0x7f03ac15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b0f9a700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03ac156730 0x7f03ac15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b0f9a700 1 -- 172.21.15.131:0/780740320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03ac0c8ba0 msgr2=0x7f03ac0c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b0f9a700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03ac0c8ba0 0x7f03ac0c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b0f9a700 1 -- 172.21.15.131:0/780740320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a80094f0 con 0x7f03ac156360 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.329+0000 7f03b0f9a700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 0x7f03ac1537f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f039c00c590 tx=0x7f039c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.330+0000 7f03a17fa700 1 -- 172.21.15.131:0/780740320 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c011590 con 0x7f03ac156360 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.330+0000 7f03a17fa700 1 -- 172.21.15.131:0/780740320 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c011710 con 0x7f03ac156360 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.330+0000 7f03a17fa700 1 -- 172.21.15.131:0/780740320 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c011ad0 con 0x7f03ac156360 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.330+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac0c3810 con 0x7f03ac156360 2023-12-26T21:38:10.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.330+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac0c3dd0 con 0x7f03ac156360 2023-12-26T21:38:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.331+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac0c3f90 con 0x7f03ac156360 2023-12-26T21:38:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.332+0000 7f03a17fa700 1 -- 172.21.15.131:0/780740320 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f039c01a360 con 0x7f03ac156360 2023-12-26T21:38:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.333+0000 7f03a17fa700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0390067130 0x7f03900695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.333+0000 7f03a17fa700 1 -- 172.21.15.131:0/780740320 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f039c083a10 con 0x7f03ac156360 2023-12-26T21:38:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.333+0000 7f03b179b700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0390067130 0x7f03900695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:10.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.333+0000 7f03b179b700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0390067130 0x7f03900695e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f03a80099b0 tx=0x7f03a8017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:10.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.337+0000 7f03a17fa700 1 -- 172.21.15.131:0/780740320 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f039c050860 con 0x7f03ac156360 2023-12-26T21:38:10.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.555+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f03ac0717f0 con 0x7f0390067130 2023-12-26T21:38:10.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.628+0000 7f03a17fa700 1 -- 172.21.15.131:0/780740320 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f03ac0717f0 con 0x7f0390067130 2023-12-26T21:38:10.629 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir7 2023-12-26T21:38:10.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.630+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0390067130 msgr2=0x7f03900695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.630+0000 7f03b39ff700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0390067130 0x7f03900695e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f03a80099b0 tx=0x7f03a8017000 comp rx=0 tx=0).stop 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.630+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 msgr2=0x7f03ac1537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.630+0000 7f03b39ff700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 0x7f03ac1537f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f039c00c590 tx=0x7f039c007f50 comp rx=0 tx=0).stop 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.631+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 shutdown_connections 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.631+0000 7f03b39ff700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0390067130 0x7f03900695e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.631+0000 7f03b39ff700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03ac156730 0x7f03ac15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.631+0000 7f03b39ff700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03ac0c8ba0 0x7f03ac0c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.631+0000 7f03b39ff700 1 --2- 172.21.15.131:0/780740320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03ac156360 0x7f03ac1537f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.631+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 >> 172.21.15.131:0/780740320 conn(0x7f03ac0675b0 msgr2=0x7f03ac067980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.631+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 shutdown_connections 2023-12-26T21:38:10.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.631+0000 7f03b39ff700 1 -- 172.21.15.131:0/780740320 wait complete. 2023-12-26T21:38:10.649 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-12-26T21:38:10.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:10.650 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-12-26T21:38:10.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-12-26T21:38:10.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.930+0000 7f3f31dff700 1 Processor -- start 2023-12-26T21:38:10.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.930+0000 7f3f31dff700 1 -- start start 2023-12-26T21:38:10.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.931+0000 7f3f31dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3f2c156360 0x7f3f2c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.931+0000 7f3f31dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 0x7f3f2c06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.931+0000 7f3f31dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c0acfc0 con 0x7f3f2c0c8490 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.931+0000 7f3f31dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2c06f1f0 con 0x7f3f2c156360 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.931+0000 7f3f31dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0c91f0 con 0x7f3f2c156730 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.931+0000 7f3f2b7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3f2c156360 0x7f3f2c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.931+0000 7f3f2bfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3f2c0c8490 0x7f3f2c0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47128/0 (socket says 172.21.15.131:47128) 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.931+0000 7f3f2bfff700 1 -- 172.21.15.131:0/2461107505 learned_addr learned my addr 172.21.15.131:0/2461107505 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.931+0000 7f3f2affd700 1 --2- 172.21.15.131:0/2461107505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 0x7f3f2c06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.932+0000 7f3f2b7fe700 1 -- 172.21.15.131:0/2461107505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 msgr2=0x7f3f2c06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.932+0000 7f3f2b7fe700 1 --2- 172.21.15.131:0/2461107505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 0x7f3f2c06ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.932+0000 7f3f2b7fe700 1 -- 172.21.15.131:0/2461107505 >> v1:172.21.15.131:6789/0 conn(0x7f3f2c0c8490 legacy=0x7f3f2c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.932+0000 7f3f2b7fe700 1 -- 172.21.15.131:0/2461107505 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0c94d0 con 0x7f3f2c156360 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.932+0000 7f3f2b7fe700 1 --2- 172.21.15.131:0/2461107505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 0x7f3f2c06e770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0098e0 tx=0x7f3f1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3839673ca06ccb9b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:10.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.932+0000 7f3f2a7fc700 1 -- 172.21.15.131:0/2461107505 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c009e80 con 0x7f3f2c156360 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.932+0000 7f3f2a7fc700 1 -- 172.21.15.131:0/2461107505 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c00b860 con 0x7f3f2c156360 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.932+0000 7f3f2a7fc700 1 -- 172.21.15.131:0/2461107505 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c00bbe0 con 0x7f3f2c156360 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.933+0000 7f3f31dff700 1 -- 172.21.15.131:0/2461107505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 msgr2=0x7f3f2c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.933+0000 7f3f31dff700 1 --2- 172.21.15.131:0/2461107505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 0x7f3f2c06e770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0098e0 tx=0x7f3f1c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.933+0000 7f3f31dff700 1 -- 172.21.15.131:0/2461107505 shutdown_connections 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.933+0000 7f3f31dff700 1 --2- 172.21.15.131:0/2461107505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 0x7f3f2c06e770 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.933+0000 7f3f31dff700 1 --2- 172.21.15.131:0/2461107505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 0x7f3f2c06ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.933+0000 7f3f31dff700 1 -- 172.21.15.131:0/2461107505 >> 172.21.15.131:0/2461107505 conn(0x7f3f2c0675b0 msgr2=0x7f3f2c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.933+0000 7f3f31dff700 1 -- 172.21.15.131:0/2461107505 shutdown_connections 2023-12-26T21:38:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.933+0000 7f3f31dff700 1 -- 172.21.15.131:0/2461107505 wait complete. 2023-12-26T21:38:10.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.934+0000 7f3f31dff700 1 Processor -- start 2023-12-26T21:38:10.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.934+0000 7f3f31dff700 1 -- start start 2023-12-26T21:38:10.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.935+0000 7f3f31dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f2c0c8490 0x7f3f2c15e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.935+0000 7f3f31dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 0x7f3f2c15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.935+0000 7f3f31dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 0x7f3f2c15dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.935+0000 7f3f31dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0c0ce0 con 0x7f3f2c156360 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.935+0000 7f3f31dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0c0fc0 con 0x7f3f2c0c8490 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.935+0000 7f3f31dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0c12a0 con 0x7f3f2c156730 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 0x7f3f2c15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 0x7f3f2c15d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52798/0 (socket says 172.21.15.131:52798) 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2affd700 1 -- 172.21.15.131:0/4258249824 learned_addr learned my addr 172.21.15.131:0/4258249824 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2bfff700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 0x7f3f2c15dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2b7fe700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f2c0c8490 0x7f3f2c15e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2affd700 1 -- 172.21.15.131:0/4258249824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 msgr2=0x7f3f2c15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2affd700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 0x7f3f2c15dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2affd700 1 -- 172.21.15.131:0/4258249824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f2c0c8490 msgr2=0x7f3f2c15e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:10.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2affd700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f2c0c8490 0x7f3f2c15e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:10.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2affd700 1 -- 172.21.15.131:0/4258249824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c009420 con 0x7f3f2c156360 2023-12-26T21:38:10.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f2affd700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 0x7f3f2c15d7e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000d890 tx=0x7f3f20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:10.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.936+0000 7f3f137fe700 1 -- 172.21.15.131:0/4258249824 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f200125f0 con 0x7f3f2c156360 2023-12-26T21:38:10.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.937+0000 7f3f137fe700 1 -- 172.21.15.131:0/4258249824 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f20012750 con 0x7f3f2c156360 2023-12-26T21:38:10.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.937+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2c0c1580 con 0x7f3f2c156360 2023-12-26T21:38:10.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.937+0000 7f3f137fe700 1 -- 172.21.15.131:0/4258249824 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20012af0 con 0x7f3f2c156360 2023-12-26T21:38:10.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.937+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c15f0a0 con 0x7f3f2c156360 2023-12-26T21:38:10.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.938+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2c0c18f0 con 0x7f3f2c156360 2023-12-26T21:38:10.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.938+0000 7f3f137fe700 1 -- 172.21.15.131:0/4258249824 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f3f2001e460 con 0x7f3f2c156360 2023-12-26T21:38:10.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.939+0000 7f3f137fe700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3f18067130 0x7f3f180695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:10.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.939+0000 7f3f137fe700 1 -- 172.21.15.131:0/4258249824 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f3f20084d40 con 0x7f3f2c156360 2023-12-26T21:38:10.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.940+0000 7f3f2b7fe700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3f18067130 0x7f3f180695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:10.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.940+0000 7f3f2b7fe700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3f18067130 0x7f3f180695e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0091e0 tx=0x7f3f1c017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:10.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:10.943+0000 7f3f137fe700 1 -- 172.21.15.131:0/4258249824 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f20051c10 con 0x7f3f2c156360 2023-12-26T21:38:11.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.163+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f2c15f0a0 con 0x7f3f18067130 2023-12-26T21:38:11.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.236+0000 7f3f137fe700 1 -- 172.21.15.131:0/4258249824 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f3f2c15f0a0 con 0x7f3f18067130 2023-12-26T21:38:11.237 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir8 2023-12-26T21:38:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3f18067130 msgr2=0x7f3f180695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3f18067130 0x7f3f180695e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0091e0 tx=0x7f3f1c017000 comp rx=0 tx=0).stop 2023-12-26T21:38:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 msgr2=0x7f3f2c15d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 0x7f3f2c15d7e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000d890 tx=0x7f3f20008040 comp rx=0 tx=0).stop 2023-12-26T21:38:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 shutdown_connections 2023-12-26T21:38:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3f18067130 0x7f3f180695e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2c156730 0x7f3f2c15dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f2c0c8490 0x7f3f2c15e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 --2- 172.21.15.131:0/4258249824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f2c156360 0x7f3f2c15d7e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 >> 172.21.15.131:0/4258249824 conn(0x7f3f2c0675b0 msgr2=0x7f3f2c1554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:11.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 shutdown_connections 2023-12-26T21:38:11.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.238+0000 7f3f31dff700 1 -- 172.21.15.131:0/4258249824 wait complete. 2023-12-26T21:38:11.256 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-12-26T21:38:11.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:38:11.257 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-12-26T21:38:11.302 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-12-26T21:38:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.536+0000 7f36e8fff700 1 Processor -- start 2023-12-26T21:38:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.536+0000 7f36e8fff700 1 -- start start 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.537+0000 7f36e8fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f36e4156730 0x7f36e40c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.537+0000 7f36e8fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36e406ee80 0x7f36e40c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.537+0000 7f36e8fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e40acfc0 con 0x7f36e4156360 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.537+0000 7f36e8fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e40c91f0 con 0x7f36e406ee80 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.537+0000 7f36e8fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e40c94d0 con 0x7f36e4156730 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e2d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36e406ee80 0x7f36e40c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e2d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36e406ee80 0x7f36e40c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52808/0 (socket says 172.21.15.131:52808) 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e2d9d700 1 -- 172.21.15.131:0/2322059007 learned_addr learned my addr 172.21.15.131:0/2322059007 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e1d9b700 1 --2- 172.21.15.131:0/2322059007 >> v2:172.21.15.190:3300/0 conn(0x7f36e4156730 0x7f36e40c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e2d9d700 1 -- 172.21.15.131:0/2322059007 >> v2:172.21.15.190:3300/0 conn(0x7f36e4156730 msgr2=0x7f36e40c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e2d9d700 1 --2- 172.21.15.131:0/2322059007 >> v2:172.21.15.190:3300/0 conn(0x7f36e4156730 0x7f36e40c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e2d9d700 1 -- 172.21.15.131:0/2322059007 >> v1:172.21.15.190:6789/0 conn(0x7f36e4156360 legacy=0x7f36e406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e2d9d700 1 -- 172.21.15.131:0/2322059007 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40acc50 con 0x7f36e406ee80 2023-12-26T21:38:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e1d9b700 1 --2- 172.21.15.131:0/2322059007 >> v2:172.21.15.190:3300/0 conn(0x7f36e4156730 0x7f36e40c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:38:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.538+0000 7f36e2d9d700 1 --2- 172.21.15.131:0/2322059007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e406ee80 0x7f36e40c89d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f36e4071110 tx=0x7f36dc00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6c88b397540e3a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e159a700 1 -- 172.21.15.131:0/2322059007 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc010070 con 0x7f36e406ee80 2023-12-26T21:38:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e159a700 1 -- 172.21.15.131:0/2322059007 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc00de90 con 0x7f36e406ee80 2023-12-26T21:38:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e159a700 1 -- 172.21.15.131:0/2322059007 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc00b320 con 0x7f36e406ee80 2023-12-26T21:38:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e8fff700 1 -- 172.21.15.131:0/2322059007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e406ee80 msgr2=0x7f36e40c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e8fff700 1 --2- 172.21.15.131:0/2322059007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e406ee80 0x7f36e40c89d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f36e4071110 tx=0x7f36dc00ba50 comp rx=0 tx=0).stop 2023-12-26T21:38:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e8fff700 1 -- 172.21.15.131:0/2322059007 shutdown_connections 2023-12-26T21:38:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e8fff700 1 --2- 172.21.15.131:0/2322059007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e406ee80 0x7f36e40c89d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e8fff700 1 --2- 172.21.15.131:0/2322059007 >> v2:172.21.15.190:3300/0 conn(0x7f36e4156730 0x7f36e40c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e8fff700 1 -- 172.21.15.131:0/2322059007 >> 172.21.15.131:0/2322059007 conn(0x7f36e40675b0 msgr2=0x7f36e40679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:11.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e8fff700 1 -- 172.21.15.131:0/2322059007 shutdown_connections 2023-12-26T21:38:11.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.539+0000 7f36e8fff700 1 -- 172.21.15.131:0/2322059007 wait complete. 2023-12-26T21:38:11.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.541+0000 7f36e8fff700 1 Processor -- start 2023-12-26T21:38:11.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.541+0000 7f36e8fff700 1 -- start start 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e8fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36e406ee80 0x7f36e40cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e8fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e4156360 0x7f36e40b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e8fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36e4156730 0x7f36e40b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e8fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40afe10 con 0x7f36e4156360 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e8fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40b1d00 con 0x7f36e406ee80 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e8fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e416cfe0 con 0x7f36e4156730 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e4156360 0x7f36e40b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e1d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e4156360 0x7f36e40b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52814/0 (socket says 172.21.15.131:52814) 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e1d9b700 1 -- 172.21.15.131:0/592547323 learned_addr learned my addr 172.21.15.131:0/592547323 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e2d9d700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36e4156730 0x7f36e40b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e259c700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36e406ee80 0x7f36e40cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e1d9b700 1 -- 172.21.15.131:0/592547323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36e4156730 msgr2=0x7f36e40b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e1d9b700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36e4156730 0x7f36e40b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e1d9b700 1 -- 172.21.15.131:0/592547323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36e406ee80 msgr2=0x7f36e40cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e1d9b700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36e406ee80 0x7f36e40cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.542+0000 7f36e1d9b700 1 -- 172.21.15.131:0/592547323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc00bcf0 con 0x7f36e4156360 2023-12-26T21:38:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.543+0000 7f36e1d9b700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e4156360 0x7f36e40b1120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f36d800d890 tx=0x7f36d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.543+0000 7f36d27fc700 1 -- 172.21.15.131:0/592547323 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d80072b0 con 0x7f36e4156360 2023-12-26T21:38:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.543+0000 7f36d27fc700 1 -- 172.21.15.131:0/592547323 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d80125f0 con 0x7f36e4156360 2023-12-26T21:38:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.543+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e416d2c0 con 0x7f36e4156360 2023-12-26T21:38:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.543+0000 7f36d27fc700 1 -- 172.21.15.131:0/592547323 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8012990 con 0x7f36e4156360 2023-12-26T21:38:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.543+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e416d880 con 0x7f36e4156360 2023-12-26T21:38:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.544+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e40717f0 con 0x7f36e4156360 2023-12-26T21:38:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.545+0000 7f36d27fc700 1 -- 172.21.15.131:0/592547323 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f36d8014430 con 0x7f36e4156360 2023-12-26T21:38:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.546+0000 7f36d27fc700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f36c4067130 0x7f36c40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.546+0000 7f36d27fc700 1 -- 172.21.15.131:0/592547323 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f36d8084db0 con 0x7f36e4156360 2023-12-26T21:38:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.546+0000 7f36e259c700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f36c4067130 0x7f36c40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:11.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.546+0000 7f36e259c700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f36c4067130 0x7f36c40695e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f36d4003860 tx=0x7f36d4009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.550+0000 7f36d27fc700 1 -- 172.21.15.131:0/592547323 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36d8089070 con 0x7f36e4156360 2023-12-26T21:38:11.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.766+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f36e40c8f10 con 0x7f36c4067130 2023-12-26T21:38:11.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.846+0000 7f36d27fc700 1 -- 172.21.15.131:0/592547323 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f36e40c8f10 con 0x7f36c4067130 2023-12-26T21:38:11.847 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/dir9 2023-12-26T21:38:11.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f36c4067130 msgr2=0x7f36c40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f36c4067130 0x7f36c40695e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f36d4003860 tx=0x7f36d4009040 comp rx=0 tx=0).stop 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e4156360 msgr2=0x7f36e40b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e4156360 0x7f36e40b1120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f36d800d890 tx=0x7f36d8008040 comp rx=0 tx=0).stop 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 shutdown_connections 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f36c4067130 0x7f36c40695e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36e4156730 0x7f36e40b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36e406ee80 0x7f36e40cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 --2- 172.21.15.131:0/592547323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36e4156360 0x7f36e40b1120 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 >> 172.21.15.131:0/592547323 conn(0x7f36e40675b0 msgr2=0x7f36e4151820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 shutdown_connections 2023-12-26T21:38:11.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:11.849+0000 7f36e8fff700 1 -- 172.21.15.131:0/592547323 wait complete. 2023-12-26T21:38:11.867 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-12-26T21:38:11.867 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-12-26T21:38:12.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.062+0000 7f77e79ff700 1 Processor -- start 2023-12-26T21:38:12.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.062+0000 7f77e79ff700 1 -- start start 2023-12-26T21:38:12.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.063+0000 7f77e79ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f77e00cb510 0x7f77e0069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e79ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77e00cc590 0x7f77e00c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e79ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e0069640 con 0x7f77e0142f00 2023-12-26T21:38:12.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e79ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e0068750 con 0x7f77e00cb510 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e79ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e0068a30 con 0x7f77e00cc590 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e4f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f77e00cb510 0x7f77e0069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e4f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f77e00cb510 0x7f77e0069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52826/0 (socket says 172.21.15.131:52826) 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e4f9a700 1 -- 172.21.15.131:0/3348291055 learned_addr learned my addr 172.21.15.131:0/3348291055 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e579b700 1 --1- 172.21.15.131:0/3348291055 >> v1:172.21.15.131:6789/0 conn(0x7f77e0142f00 0x7f77e00cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47140/0 (socket says 172.21.15.131:47140) 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e5f9c700 1 --2- 172.21.15.131:0/3348291055 >> v2:172.21.15.190:3300/0 conn(0x7f77e00cc590 0x7f77e00c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e4f9a700 1 -- 172.21.15.131:0/3348291055 >> v2:172.21.15.190:3300/0 conn(0x7f77e00cc590 msgr2=0x7f77e00c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e4f9a700 1 --2- 172.21.15.131:0/3348291055 >> v2:172.21.15.190:3300/0 conn(0x7f77e00cc590 0x7f77e00c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e4f9a700 1 -- 172.21.15.131:0/3348291055 >> v1:172.21.15.131:6789/0 conn(0x7f77e0142f00 legacy=0x7f77e00cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e4f9a700 1 -- 172.21.15.131:0/3348291055 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0068d10 con 0x7f77e00cb510 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.064+0000 7f77e5f9c700 1 --2- 172.21.15.131:0/3348291055 >> v2:172.21.15.190:3300/0 conn(0x7f77e00cc590 0x7f77e00c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:38:12.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77e4f9a700 1 --2- 172.21.15.131:0/3348291055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e00cb510 0x7f77e0069ad0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f77e006c8b0 tx=0x7f77d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60cb63ee0f25743 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77d7fff700 1 -- 172.21.15.131:0/3348291055 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000b860 con 0x7f77e00cb510 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77d7fff700 1 -- 172.21.15.131:0/3348291055 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d000b9c0 con 0x7f77e00cb510 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77d7fff700 1 -- 172.21.15.131:0/3348291055 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000bd60 con 0x7f77e00cb510 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77e79ff700 1 -- 172.21.15.131:0/3348291055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e00cb510 msgr2=0x7f77e0069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77e79ff700 1 --2- 172.21.15.131:0/3348291055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e00cb510 0x7f77e0069ad0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f77e006c8b0 tx=0x7f77d0009250 comp rx=0 tx=0).stop 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77e79ff700 1 -- 172.21.15.131:0/3348291055 shutdown_connections 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77e79ff700 1 --2- 172.21.15.131:0/3348291055 >> v2:172.21.15.190:3300/0 conn(0x7f77e00cc590 0x7f77e00c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77e79ff700 1 --2- 172.21.15.131:0/3348291055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e00cb510 0x7f77e0069ad0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77e79ff700 1 -- 172.21.15.131:0/3348291055 >> 172.21.15.131:0/3348291055 conn(0x7f77e0063cd0 msgr2=0x7f77e006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.065+0000 7f77e79ff700 1 -- 172.21.15.131:0/3348291055 shutdown_connections 2023-12-26T21:38:12.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.066+0000 7f77e79ff700 1 -- 172.21.15.131:0/3348291055 wait complete. 2023-12-26T21:38:12.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.067+0000 7f77e79ff700 1 Processor -- start 2023-12-26T21:38:12.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.067+0000 7f77e79ff700 1 -- start start 2023-12-26T21:38:12.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e79ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77e00cb510 0x7f77e00c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e79ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77e00cc590 0x7f77e013fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e79ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e0142f00 0x7f77e00b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e79ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00c5900 con 0x7f77e0142f00 2023-12-26T21:38:12.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e79ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00c6760 con 0x7f77e00cb510 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e79ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00c5b50 con 0x7f77e00cc590 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e5f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e0142f00 0x7f77e00b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e5f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e0142f00 0x7f77e00b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52842/0 (socket says 172.21.15.131:52842) 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e5f9c700 1 -- 172.21.15.131:0/797103300 learned_addr learned my addr 172.21.15.131:0/797103300 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e579b700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77e00cb510 0x7f77e00c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.068+0000 7f77e4f9a700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77e00cc590 0x7f77e013fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e5f9c700 1 -- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77e00cc590 msgr2=0x7f77e013fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e5f9c700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77e00cc590 0x7f77e013fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e5f9c700 1 -- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77e00cb510 msgr2=0x7f77e00c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e5f9c700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77e00cb510 0x7f77e00c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e5f9c700 1 -- 172.21.15.131:0/797103300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00094f0 con 0x7f77e0142f00 2023-12-26T21:38:12.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e4f9a700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77e00cc590 0x7f77e013fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:38:12.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e579b700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77e00cb510 0x7f77e00c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:38:12.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e5f9c700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e0142f00 0x7f77e00b3490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d1b0 tx=0x7f77d8009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:12.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77d57fa700 1 -- 172.21.15.131:0/797103300 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d800de50 con 0x7f77e0142f00 2023-12-26T21:38:12.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77d57fa700 1 -- 172.21.15.131:0/797103300 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d800f040 con 0x7f77e0142f00 2023-12-26T21:38:12.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77d57fa700 1 -- 172.21.15.131:0/797103300 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8015690 con 0x7f77e0142f00 2023-12-26T21:38:12.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e00c5e30 con 0x7f77e0142f00 2023-12-26T21:38:12.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.069+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e00c63f0 con 0x7f77e0142f00 2023-12-26T21:38:12.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.070+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e00c6a40 con 0x7f77e0142f00 2023-12-26T21:38:12.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.071+0000 7f77d57fa700 1 -- 172.21.15.131:0/797103300 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f77d801e3b0 con 0x7f77e0142f00 2023-12-26T21:38:12.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.072+0000 7f77d57fa700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f77c4067130 0x7f77c40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.072+0000 7f77d57fa700 1 -- 172.21.15.131:0/797103300 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f77d8085160 con 0x7f77e0142f00 2023-12-26T21:38:12.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.072+0000 7f77e579b700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f77c4067130 0x7f77c40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.073+0000 7f77e579b700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f77c4067130 0x7f77c40695e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f77dc007fd0 tx=0x7f77dc007e10 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.076+0000 7f77d57fa700 1 -- 172.21.15.131:0/797103300 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77d8051ea0 con 0x7f77e0142f00 2023-12-26T21:38:12.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.296+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f77e00c68a0 con 0x7f77c4067130 2023-12-26T21:38:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.357+0000 7f77d57fa700 1 -- 172.21.15.131:0/797103300 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f77dc013420 con 0x7f77c4067130 2023-12-26T21:38:12.357 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-12-26T21:38:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f77c4067130 msgr2=0x7f77c40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f77c4067130 0x7f77c40695e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f77dc007fd0 tx=0x7f77dc007e10 comp rx=0 tx=0).stop 2023-12-26T21:38:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e0142f00 msgr2=0x7f77e00b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e0142f00 0x7f77e00b3490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d1b0 tx=0x7f77d8009a50 comp rx=0 tx=0).stop 2023-12-26T21:38:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 shutdown_connections 2023-12-26T21:38:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77e0142f00 0x7f77e00b3490 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77e00cb510 0x7f77e00c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f77c4067130 0x7f77c40695e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 --2- 172.21.15.131:0/797103300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77e00cc590 0x7f77e013fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 >> 172.21.15.131:0/797103300 conn(0x7f77e0063cd0 msgr2=0x7f77e00c4910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 shutdown_connections 2023-12-26T21:38:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.359+0000 7f77e79ff700 1 -- 172.21.15.131:0/797103300 wait complete. 2023-12-26T21:38:12.377 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-12-26T21:38:12.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-12-26T21:38:12.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.573+0000 7f61fd7ff700 1 Processor -- start 2023-12-26T21:38:12.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.573+0000 7f61fd7ff700 1 -- start start 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.574+0000 7f61fd7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f61f8159810 0x7f61f80c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.574+0000 7f61fd7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f61f80cec80 0x7f61f80cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.574+0000 7f61fd7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f80cbb00 con 0x7f61f80cd0a0 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.574+0000 7f61fd7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f80cb2a0 con 0x7f61f8159810 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.574+0000 7f61fd7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f80cf640 con 0x7f61f80cec80 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f61f8159810 0x7f61f80c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f61f8159810 0x7f61f80c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52848/0 (socket says 172.21.15.131:52848) 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f6ffd700 1 -- 172.21.15.131:0/2872259049 learned_addr learned my addr 172.21.15.131:0/2872259049 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f77fe700 1 --1- 172.21.15.131:0/2872259049 >> v1:172.21.15.131:6789/0 conn(0x7f61f80cd0a0 0x7f61f80cd4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47154/0 (socket says 172.21.15.131:47154) 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f67fc700 1 --2- 172.21.15.131:0/2872259049 >> v2:172.21.15.190:3300/0 conn(0x7f61f80cec80 0x7f61f80cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f6ffd700 1 -- 172.21.15.131:0/2872259049 >> v2:172.21.15.190:3300/0 conn(0x7f61f80cec80 msgr2=0x7f61f80cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:12.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f6ffd700 1 --2- 172.21.15.131:0/2872259049 >> v2:172.21.15.190:3300/0 conn(0x7f61f80cec80 0x7f61f80cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f6ffd700 1 -- 172.21.15.131:0/2872259049 >> v1:172.21.15.131:6789/0 conn(0x7f61f80cd0a0 legacy=0x7f61f80cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f6ffd700 1 -- 172.21.15.131:0/2872259049 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f80ce040 con 0x7f61f8159810 2023-12-26T21:38:12.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f6ffd700 1 --2- 172.21.15.131:0/2872259049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 0x7f61f80c5440 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f61f8089a80 tx=0x7f61e8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a270facac507be6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:12.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f5ffb700 1 -- 172.21.15.131:0/2872259049 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8005c10 con 0x7f61f8159810 2023-12-26T21:38:12.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f5ffb700 1 -- 172.21.15.131:0/2872259049 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e8005d70 con 0x7f61f8159810 2023-12-26T21:38:12.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.575+0000 7f61f5ffb700 1 -- 172.21.15.131:0/2872259049 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8006260 con 0x7f61f8159810 2023-12-26T21:38:12.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.576+0000 7f61fd7ff700 1 -- 172.21.15.131:0/2872259049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 msgr2=0x7f61f80c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.576+0000 7f61fd7ff700 1 --2- 172.21.15.131:0/2872259049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 0x7f61f80c5440 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f61f8089a80 tx=0x7f61e8009220 comp rx=0 tx=0).stop 2023-12-26T21:38:12.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.576+0000 7f61fd7ff700 1 -- 172.21.15.131:0/2872259049 shutdown_connections 2023-12-26T21:38:12.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.576+0000 7f61fd7ff700 1 --2- 172.21.15.131:0/2872259049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 0x7f61f80c5440 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.576+0000 7f61fd7ff700 1 --2- 172.21.15.131:0/2872259049 >> v2:172.21.15.190:3300/0 conn(0x7f61f80cec80 0x7f61f80cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.576+0000 7f61fd7ff700 1 -- 172.21.15.131:0/2872259049 >> 172.21.15.131:0/2872259049 conn(0x7f61f8099950 msgr2=0x7f61f806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:12.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.576+0000 7f61fd7ff700 1 -- 172.21.15.131:0/2872259049 shutdown_connections 2023-12-26T21:38:12.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.576+0000 7f61fd7ff700 1 -- 172.21.15.131:0/2872259049 wait complete. 2023-12-26T21:38:12.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.578+0000 7f61fd7ff700 1 Processor -- start 2023-12-26T21:38:12.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.578+0000 7f61fd7ff700 1 -- start start 2023-12-26T21:38:12.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61fd7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f80cd0a0 0x7f61f80b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61fd7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f80cec80 0x7f61f8157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61fd7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 0x7f61f8158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61fd7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80b4ca0 con 0x7f61f8159810 2023-12-26T21:38:12.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61fd7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80b3db0 con 0x7f61f80cd0a0 2023-12-26T21:38:12.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61fd7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80b4090 con 0x7f61f80cec80 2023-12-26T21:38:12.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 0x7f61f8158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 0x7f61f8158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52852/0 (socket says 172.21.15.131:52852) 2023-12-26T21:38:12.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f77fe700 1 -- 172.21.15.131:0/3941891323 learned_addr learned my addr 172.21.15.131:0/3941891323 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f6ffd700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f80cd0a0 0x7f61f80b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f67fc700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f80cec80 0x7f61f8157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f77fe700 1 -- 172.21.15.131:0/3941891323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f80cec80 msgr2=0x7f61f8157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f77fe700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f80cec80 0x7f61f8157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f77fe700 1 -- 172.21.15.131:0/3941891323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f80cd0a0 msgr2=0x7f61f80b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f77fe700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f80cd0a0 0x7f61f80b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.579+0000 7f61f77fe700 1 -- 172.21.15.131:0/3941891323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e8009480 con 0x7f61f8159810 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.580+0000 7f61f77fe700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 0x7f61f8158190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f61f000cc70 tx=0x7f61f0001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.580+0000 7f61deffd700 1 -- 172.21.15.131:0/3941891323 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0005df0 con 0x7f61f8159810 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.580+0000 7f61deffd700 1 -- 172.21.15.131:0/3941891323 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f000a5b0 con 0x7f61f8159810 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.580+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f80b4370 con 0x7f61f8159810 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.580+0000 7f61deffd700 1 -- 172.21.15.131:0/3941891323 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f000a930 con 0x7f61f8159810 2023-12-26T21:38:12.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.580+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f80b4930 con 0x7f61f8159810 2023-12-26T21:38:12.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.581+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f80717f0 con 0x7f61f8159810 2023-12-26T21:38:12.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.582+0000 7f61deffd700 1 -- 172.21.15.131:0/3941891323 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f61f000ace0 con 0x7f61f8159810 2023-12-26T21:38:12.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.583+0000 7f61deffd700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f61e4067130 0x7f61e40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:12.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.583+0000 7f61deffd700 1 -- 172.21.15.131:0/3941891323 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f61f00849e0 con 0x7f61f8159810 2023-12-26T21:38:12.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.583+0000 7f61f6ffd700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f61e4067130 0x7f61e40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:12.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.584+0000 7f61f6ffd700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f61e4067130 0x7f61e40695e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009260 tx=0x7f61e8016000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:12.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.586+0000 7f61deffd700 1 -- 172.21.15.131:0/3941891323 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61f00518b0 con 0x7f61f8159810 2023-12-26T21:38:12.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.803+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f61f80cfbf0 con 0x7f61e4067130 2023-12-26T21:38:12.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.855+0000 7f61deffd700 1 -- 172.21.15.131:0/3941891323 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f61f80cfbf0 con 0x7f61e4067130 2023-12-26T21:38:12.856 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-12-26T21:38:12.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f61e4067130 msgr2=0x7f61e40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f61e4067130 0x7f61e40695e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009260 tx=0x7f61e8016000 comp rx=0 tx=0).stop 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 msgr2=0x7f61f8158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 0x7f61f8158190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f61f000cc70 tx=0x7f61f0001a70 comp rx=0 tx=0).stop 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 shutdown_connections 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61f8159810 0x7f61f8158190 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f80cd0a0 0x7f61f80b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f61e4067130 0x7f61e40695e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 --2- 172.21.15.131:0/3941891323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f80cec80 0x7f61f8157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 >> 172.21.15.131:0/3941891323 conn(0x7f61f8099950 msgr2=0x7f61f8068650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 shutdown_connections 2023-12-26T21:38:12.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:12.858+0000 7f61fd7ff700 1 -- 172.21.15.131:0/3941891323 wait complete. 2023-12-26T21:38:12.876 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-12-26T21:38:12.877 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-12-26T21:38:13.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.073+0000 7fbae71ff700 1 Processor -- start 2023-12-26T21:38:13.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.074+0000 7fbae71ff700 1 -- start start 2023-12-26T21:38:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbae71ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae0069c30 0x7fbae00abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbae71ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbae00ae080 0x7fbae006df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbae71ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae00c7430 con 0x7fbae00adcb0 2023-12-26T21:38:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbae71ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae00ac460 con 0x7fbae00ae080 2023-12-26T21:38:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbae71ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00c6820 con 0x7fbae0069c30 2023-12-26T21:38:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbae4f9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbae00adcb0 0x7fbae0069520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47166/0 (socket says 172.21.15.131:47166) 2023-12-26T21:38:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbae4f9b700 1 -- 172.21.15.131:0/591224005 learned_addr learned my addr 172.21.15.131:0/591224005 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:38:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbadffff700 1 --2- 172.21.15.131:0/591224005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae0069c30 0x7fbae00abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbae579c700 1 --2- 172.21.15.131:0/591224005 >> v2:172.21.15.190:3300/0 conn(0x7fbae00ae080 0x7fbae006df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.075+0000 7fbadf7fe700 1 -- 172.21.15.131:0/591224005 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967112516 0 0) 0x7fbae00c7430 con 0x7fbae00adcb0 2023-12-26T21:38:13.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.076+0000 7fbadf7fe700 1 -- 172.21.15.131:0/591224005 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbad0003570 con 0x7fbae00adcb0 2023-12-26T21:38:13.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.076+0000 7fbadffff700 1 -- 172.21.15.131:0/591224005 >> v2:172.21.15.190:3300/0 conn(0x7fbae00ae080 msgr2=0x7fbae006df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:13.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.076+0000 7fbadffff700 1 --2- 172.21.15.131:0/591224005 >> v2:172.21.15.190:3300/0 conn(0x7fbae00ae080 0x7fbae006df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.076+0000 7fbadffff700 1 -- 172.21.15.131:0/591224005 >> v1:172.21.15.131:6789/0 conn(0x7fbae00adcb0 legacy=0x7fbae0069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.076+0000 7fbadffff700 1 -- 172.21.15.131:0/591224005 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00c6b00 con 0x7fbae0069c30 2023-12-26T21:38:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.076+0000 7fbadffff700 1 --2- 172.21.15.131:0/591224005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae0069c30 0x7fbae00abf20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00aa90 tx=0x7fbacc00a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3dd8a19fbb892ea server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.076+0000 7fbadf7fe700 1 -- 172.21.15.131:0/591224005 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00c970 con 0x7fbae0069c30 2023-12-26T21:38:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.076+0000 7fbadf7fe700 1 -- 172.21.15.131:0/591224005 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc00cad0 con 0x7fbae0069c30 2023-12-26T21:38:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.076+0000 7fbadf7fe700 1 -- 172.21.15.131:0/591224005 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00ce50 con 0x7fbae0069c30 2023-12-26T21:38:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.077+0000 7fbae71ff700 1 -- 172.21.15.131:0/591224005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae0069c30 msgr2=0x7fbae00abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.077+0000 7fbae71ff700 1 --2- 172.21.15.131:0/591224005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae0069c30 0x7fbae00abf20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00aa90 tx=0x7fbacc00a330 comp rx=0 tx=0).stop 2023-12-26T21:38:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.077+0000 7fbae71ff700 1 -- 172.21.15.131:0/591224005 shutdown_connections 2023-12-26T21:38:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.077+0000 7fbae71ff700 1 --2- 172.21.15.131:0/591224005 >> v2:172.21.15.190:3300/0 conn(0x7fbae00ae080 0x7fbae006df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.077+0000 7fbae71ff700 1 --2- 172.21.15.131:0/591224005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae0069c30 0x7fbae00abf20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.077+0000 7fbae71ff700 1 -- 172.21.15.131:0/591224005 >> 172.21.15.131:0/591224005 conn(0x7fbae00675b0 msgr2=0x7fbae00679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.077+0000 7fbae71ff700 1 -- 172.21.15.131:0/591224005 shutdown_connections 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.077+0000 7fbae71ff700 1 -- 172.21.15.131:0/591224005 wait complete. 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.078+0000 7fbae71ff700 1 Processor -- start 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.079+0000 7fbae71ff700 1 -- start start 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.079+0000 7fbae71ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae0069c30 0x7fbae01694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.079+0000 7fbae71ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbae00adcb0 0x7fbae00afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.079+0000 7fbae71ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae00ae080 0x7fbae00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.079+0000 7fbae71ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00c3280 con 0x7fbae00adcb0 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.079+0000 7fbae71ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00c4fd0 con 0x7fbae0069c30 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.079+0000 7fbae71ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00c5870 con 0x7fbae00ae080 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbae00adcb0 0x7fbae00afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbae00adcb0 0x7fbae00afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52862/0 (socket says 172.21.15.131:52862) 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadffff700 1 -- 172.21.15.131:0/496948582 learned_addr learned my addr 172.21.15.131:0/496948582 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbae4f9b700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae0069c30 0x7fbae01694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbae579c700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae00ae080 0x7fbae00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadffff700 1 -- 172.21.15.131:0/496948582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae00ae080 msgr2=0x7fbae00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadffff700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae00ae080 0x7fbae00c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadffff700 1 -- 172.21.15.131:0/496948582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae0069c30 msgr2=0x7fbae01694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadffff700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae0069c30 0x7fbae01694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadffff700 1 -- 172.21.15.131:0/496948582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc00a5d0 con 0x7fbae00adcb0 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadffff700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbae00adcb0 0x7fbae00afb10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbacc000e50 tx=0x7fbacc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.080+0000 7fbadcff9700 1 -- 172.21.15.131:0/496948582 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc01c040 con 0x7fbae00adcb0 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.081+0000 7fbadcff9700 1 -- 172.21.15.131:0/496948582 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc020400 con 0x7fbae00adcb0 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.081+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae0169b00 con 0x7fbae00adcb0 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.081+0000 7fbadcff9700 1 -- 172.21.15.131:0/496948582 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0207e0 con 0x7fbae00adcb0 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.081+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae016a0c0 con 0x7fbae00adcb0 2023-12-26T21:38:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.082+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae00717f0 con 0x7fbae00adcb0 2023-12-26T21:38:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.083+0000 7fbadcff9700 1 -- 172.21.15.131:0/496948582 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7fbacc0103f0 con 0x7fbae00adcb0 2023-12-26T21:38:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.083+0000 7fbadcff9700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fbac4067130 0x7fbac40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.083+0000 7fbadcff9700 1 -- 172.21.15.131:0/496948582 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fbacc085e40 con 0x7fbae00adcb0 2023-12-26T21:38:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.084+0000 7fbae4f9b700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fbac4067130 0x7fbac40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.084+0000 7fbae4f9b700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fbac4067130 0x7fbac40695e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbae00c61f0 tx=0x7fbad4009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:13.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.088+0000 7fbadcff9700 1 -- 172.21.15.131:0/496948582 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbacc052b80 con 0x7fbae00adcb0 2023-12-26T21:38:13.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.308+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbae00c6de0 con 0x7fbac4067130 2023-12-26T21:38:13.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.361+0000 7fbadcff9700 1 -- 172.21.15.131:0/496948582 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbae00c6de0 con 0x7fbac4067130 2023-12-26T21:38:13.362 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fbac4067130 msgr2=0x7fbac40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fbac4067130 0x7fbac40695e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbae00c61f0 tx=0x7fbad4009040 comp rx=0 tx=0).stop 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbae00adcb0 msgr2=0x7fbae00afb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbae00adcb0 0x7fbae00afb10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbacc000e50 tx=0x7fbacc01a000 comp rx=0 tx=0).stop 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 shutdown_connections 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fbac4067130 0x7fbac40695e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae00ae080 0x7fbae00c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae0069c30 0x7fbae01694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 --2- 172.21.15.131:0/496948582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbae00adcb0 0x7fbae00afb10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 >> 172.21.15.131:0/496948582 conn(0x7fbae00675b0 msgr2=0x7fbae006d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:13.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 shutdown_connections 2023-12-26T21:38:13.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.364+0000 7fbae71ff700 1 -- 172.21.15.131:0/496948582 wait complete. 2023-12-26T21:38:13.382 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-12-26T21:38:13.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-12-26T21:38:13.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.575+0000 7f4b673ff700 1 Processor -- start 2023-12-26T21:38:13.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.575+0000 7f4b673ff700 1 -- start start 2023-12-26T21:38:13.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b673ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b60156360 0x7f4b6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b673ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156730 0x7f4b6006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b673ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b600c8490 0x7f4b600c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b673ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6006f1f0 con 0x7f4b600c8490 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b673ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b600acfc0 con 0x7f4b60156360 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b673ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600c91f0 con 0x7f4b60156730 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b6599c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b600c8490 0x7f4b600c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b6599c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b600c8490 0x7f4b600c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52868/0 (socket says 172.21.15.131:52868) 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b6599c700 1 -- 172.21.15.131:0/578392352 learned_addr learned my addr 172.21.15.131:0/578392352 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b6499a700 1 --2- 172.21.15.131:0/578392352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156730 0x7f4b6006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.576+0000 7f4b6519b700 1 --2- 172.21.15.131:0/578392352 >> v2:172.21.15.190:3300/0 conn(0x7f4b60156360 0x7f4b6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.577+0000 7f4b6599c700 1 -- 172.21.15.131:0/578392352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156730 msgr2=0x7f4b6006ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.577+0000 7f4b6599c700 1 --2- 172.21.15.131:0/578392352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156730 0x7f4b6006ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.577+0000 7f4b6599c700 1 -- 172.21.15.131:0/578392352 >> v2:172.21.15.190:3300/0 conn(0x7f4b60156360 msgr2=0x7f4b6006e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:13.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.577+0000 7f4b6599c700 1 --2- 172.21.15.131:0/578392352 >> v2:172.21.15.190:3300/0 conn(0x7f4b60156360 0x7f4b6006e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.577+0000 7f4b6599c700 1 -- 172.21.15.131:0/578392352 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600c94d0 con 0x7f4b600c8490 2023-12-26T21:38:13.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.577+0000 7f4b6599c700 1 --2- 172.21.15.131:0/578392352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b600c8490 0x7f4b600c8930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4b60069410 tx=0x7f4b5800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28afeb85c860d72 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:13.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.577+0000 7f4b57fff700 1 -- 172.21.15.131:0/578392352 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58010070 con 0x7f4b600c8490 2023-12-26T21:38:13.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.577+0000 7f4b57fff700 1 -- 172.21.15.131:0/578392352 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5800dd30 con 0x7f4b600c8490 2023-12-26T21:38:13.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.577+0000 7f4b57fff700 1 -- 172.21.15.131:0/578392352 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800dea0 con 0x7f4b600c8490 2023-12-26T21:38:13.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.578+0000 7f4b673ff700 1 -- 172.21.15.131:0/578392352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b600c8490 msgr2=0x7f4b600c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.578+0000 7f4b673ff700 1 --2- 172.21.15.131:0/578392352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b600c8490 0x7f4b600c8930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4b60069410 tx=0x7f4b5800ba50 comp rx=0 tx=0).stop 2023-12-26T21:38:13.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.578+0000 7f4b673ff700 1 -- 172.21.15.131:0/578392352 shutdown_connections 2023-12-26T21:38:13.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.578+0000 7f4b673ff700 1 --2- 172.21.15.131:0/578392352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b600c8490 0x7f4b600c8930 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.578+0000 7f4b673ff700 1 --2- 172.21.15.131:0/578392352 >> v2:172.21.15.190:3300/0 conn(0x7f4b60156360 0x7f4b6006e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.578+0000 7f4b673ff700 1 --2- 172.21.15.131:0/578392352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156730 0x7f4b6006ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.578+0000 7f4b673ff700 1 -- 172.21.15.131:0/578392352 >> 172.21.15.131:0/578392352 conn(0x7f4b600675b0 msgr2=0x7f4b600679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:13.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.578+0000 7f4b673ff700 1 -- 172.21.15.131:0/578392352 shutdown_connections 2023-12-26T21:38:13.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.578+0000 7f4b673ff700 1 -- 172.21.15.131:0/578392352 wait complete. 2023-12-26T21:38:13.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.579+0000 7f4b673ff700 1 Processor -- start 2023-12-26T21:38:13.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.580+0000 7f4b673ff700 1 -- start start 2023-12-26T21:38:13.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.580+0000 7f4b673ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b600c8490 0x7f4b600c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b673ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156360 0x7f4b60153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b673ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b60156730 0x7f4b600c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b673ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600b0720 con 0x7f4b60156730 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b673ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600afe10 con 0x7f4b600c8490 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b673ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600c67e0 con 0x7f4b60156360 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b60156730 0x7f4b600c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b60156730 0x7f4b600c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52876/0 (socket says 172.21.15.131:52876) 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6599c700 1 -- 172.21.15.131:0/1953412469 learned_addr learned my addr 172.21.15.131:0/1953412469 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6499a700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156360 0x7f4b60153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6519b700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b600c8490 0x7f4b600c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6599c700 1 -- 172.21.15.131:0/1953412469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156360 msgr2=0x7f4b60153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6599c700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156360 0x7f4b60153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6599c700 1 -- 172.21.15.131:0/1953412469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b600c8490 msgr2=0x7f4b600c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6599c700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b600c8490 0x7f4b600c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.581+0000 7f4b6599c700 1 -- 172.21.15.131:0/1953412469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5800bcf0 con 0x7f4b60156730 2023-12-26T21:38:13.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.582+0000 7f4b6599c700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b60156730 0x7f4b600c5eb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800ba90 tx=0x7f4b5801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:13.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.582+0000 7f4b557fa700 1 -- 172.21.15.131:0/1953412469 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800aae0 con 0x7f4b60156730 2023-12-26T21:38:13.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.582+0000 7f4b557fa700 1 -- 172.21.15.131:0/1953412469 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5800ac60 con 0x7f4b60156730 2023-12-26T21:38:13.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.582+0000 7f4b557fa700 1 -- 172.21.15.131:0/1953412469 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5801e440 con 0x7f4b60156730 2023-12-26T21:38:13.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.582+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b600b00d0 con 0x7f4b60156730 2023-12-26T21:38:13.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.582+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b60153bc0 con 0x7f4b60156730 2023-12-26T21:38:13.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.583+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b600b08d0 con 0x7f4b60156730 2023-12-26T21:38:13.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.584+0000 7f4b557fa700 1 -- 172.21.15.131:0/1953412469 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f4b58003040 con 0x7f4b60156730 2023-12-26T21:38:13.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.584+0000 7f4b557fa700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b44067130 0x7f4b440695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:13.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.585+0000 7f4b557fa700 1 -- 172.21.15.131:0/1953412469 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f4b580865e0 con 0x7f4b60156730 2023-12-26T21:38:13.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.585+0000 7f4b6519b700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b44067130 0x7f4b440695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:13.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.585+0000 7f4b6519b700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b44067130 0x7f4b440695e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009d40 tx=0x7f4b5c008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:13.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.589+0000 7f4b557fa700 1 -- 172.21.15.131:0/1953412469 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b600b08d0 con 0x7f4b60156730 2023-12-26T21:38:13.630 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:38:13.635 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:38:13.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.814+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b600717f0 con 0x7f4b44067130 2023-12-26T21:38:13.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.869+0000 7f4b557fa700 1 -- 172.21.15.131:0/1953412469 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4b600717f0 con 0x7f4b44067130 2023-12-26T21:38:13.870 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-12-26T21:38:13.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b44067130 msgr2=0x7f4b440695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b44067130 0x7f4b440695e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009d40 tx=0x7f4b5c008040 comp rx=0 tx=0).stop 2023-12-26T21:38:13.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b60156730 msgr2=0x7f4b600c5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:13.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b60156730 0x7f4b600c5eb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800ba90 tx=0x7f4b5801a000 comp rx=0 tx=0).stop 2023-12-26T21:38:13.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 shutdown_connections 2023-12-26T21:38:13.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b60156730 0x7f4b600c5eb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b600c8490 0x7f4b600c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4b44067130 0x7f4b440695e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 --2- 172.21.15.131:0/1953412469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b60156360 0x7f4b60153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:13.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 >> 172.21.15.131:0/1953412469 conn(0x7f4b600675b0 msgr2=0x7f4b600c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:13.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 shutdown_connections 2023-12-26T21:38:13.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:13.871+0000 7f4b673ff700 1 -- 172.21.15.131:0/1953412469 wait complete. 2023-12-26T21:38:13.890 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-12-26T21:38:13.890 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-12-26T21:38:14.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.089+0000 7fd7849ff700 1 Processor -- start 2023-12-26T21:38:14.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.090+0000 7fd7849ff700 1 -- start start 2023-12-26T21:38:14.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.091+0000 7fd7849ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7800adcb0 0x7fd7800abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.091+0000 7fd7849ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 0x7fd78006df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.091+0000 7fd7849ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7800c7430 con 0x7fd7800ae080 2023-12-26T21:38:14.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.091+0000 7fd7849ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7800ac460 con 0x7fd7800adcb0 2023-12-26T21:38:14.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.091+0000 7fd7849ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800c6820 con 0x7fd780069c30 2023-12-26T21:38:14.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.091+0000 7fd77e59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7800adcb0 0x7fd7800abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.091+0000 7fd77ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 0x7fd78006df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.091+0000 7fd777fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd7800ae080 0x7fd780069520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:35860/0 (socket says 172.21.15.131:35860) 2023-12-26T21:38:14.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.091+0000 7fd777fff700 1 -- 172.21.15.131:0/2314317612 learned_addr learned my addr 172.21.15.131:0/2314317612 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:38:14.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.092+0000 7fd77dd9b700 1 -- 172.21.15.131:0/2314317612 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815914311 0 0) 0x7fd7800c7430 con 0x7fd7800ae080 2023-12-26T21:38:14.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.092+0000 7fd77dd9b700 1 -- 172.21.15.131:0/2314317612 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd76c003570 con 0x7fd7800ae080 2023-12-26T21:38:14.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.092+0000 7fd77ed9d700 1 -- 172.21.15.131:0/2314317612 >> v2:172.21.15.190:3300/0 conn(0x7fd7800adcb0 msgr2=0x7fd7800abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:14.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.092+0000 7fd77ed9d700 1 --2- 172.21.15.131:0/2314317612 >> v2:172.21.15.190:3300/0 conn(0x7fd7800adcb0 0x7fd7800abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.092+0000 7fd77ed9d700 1 -- 172.21.15.131:0/2314317612 >> v1:172.21.15.190:6789/0 conn(0x7fd7800ae080 legacy=0x7fd780069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.092+0000 7fd77ed9d700 1 -- 172.21.15.131:0/2314317612 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7800c6b00 con 0x7fd780069c30 2023-12-26T21:38:14.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.092+0000 7fd77e59c700 1 --2- 172.21.15.131:0/2314317612 >> v2:172.21.15.190:3300/0 conn(0x7fd7800adcb0 0x7fd7800abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:38:14.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.092+0000 7fd77ed9d700 1 --2- 172.21.15.131:0/2314317612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 0x7fd78006df30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd778009d50 tx=0x7fd778009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e4fb48dec7031f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:14.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd77dd9b700 1 -- 172.21.15.131:0/2314317612 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77800cbc0 con 0x7fd780069c30 2023-12-26T21:38:14.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd77dd9b700 1 -- 172.21.15.131:0/2314317612 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77800cd20 con 0x7fd780069c30 2023-12-26T21:38:14.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd77dd9b700 1 -- 172.21.15.131:0/2314317612 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd778011650 con 0x7fd780069c30 2023-12-26T21:38:14.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd7849ff700 1 -- 172.21.15.131:0/2314317612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 msgr2=0x7fd78006df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd7849ff700 1 --2- 172.21.15.131:0/2314317612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 0x7fd78006df30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd778009d50 tx=0x7fd778009780 comp rx=0 tx=0).stop 2023-12-26T21:38:14.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd7849ff700 1 -- 172.21.15.131:0/2314317612 shutdown_connections 2023-12-26T21:38:14.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd7849ff700 1 --2- 172.21.15.131:0/2314317612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 0x7fd78006df30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd7849ff700 1 --2- 172.21.15.131:0/2314317612 >> v2:172.21.15.190:3300/0 conn(0x7fd7800adcb0 0x7fd7800abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd7849ff700 1 -- 172.21.15.131:0/2314317612 >> 172.21.15.131:0/2314317612 conn(0x7fd7800675b0 msgr2=0x7fd7800679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:14.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd7849ff700 1 -- 172.21.15.131:0/2314317612 shutdown_connections 2023-12-26T21:38:14.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.093+0000 7fd7849ff700 1 -- 172.21.15.131:0/2314317612 wait complete. 2023-12-26T21:38:14.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.095+0000 7fd7849ff700 1 Processor -- start 2023-12-26T21:38:14.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.095+0000 7fd7849ff700 1 -- start start 2023-12-26T21:38:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd7849ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 0x7fd780153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd7849ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7800adcb0 0x7fd7800aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd7849ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7800ae080 0x7fd7800af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd7849ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800c3560 con 0x7fd7800ae080 2023-12-26T21:38:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd7849ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800c1d20 con 0x7fd7800adcb0 2023-12-26T21:38:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd7849ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800c2000 con 0x7fd780069c30 2023-12-26T21:38:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd77ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7800ae080 0x7fd7800af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd77ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7800ae080 0x7fd7800af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52878/0 (socket says 172.21.15.131:52878) 2023-12-26T21:38:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd77ed9d700 1 -- 172.21.15.131:0/640863639 learned_addr learned my addr 172.21.15.131:0/640863639 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd777fff700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7800adcb0 0x7fd7800aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.096+0000 7fd77e59c700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 0x7fd780153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd77ed9d700 1 -- 172.21.15.131:0/640863639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 msgr2=0x7fd780153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd77ed9d700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 0x7fd780153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd77ed9d700 1 -- 172.21.15.131:0/640863639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7800adcb0 msgr2=0x7fd7800aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd77ed9d700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7800adcb0 0x7fd7800aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd77ed9d700 1 -- 172.21.15.131:0/640863639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770009420 con 0x7fd7800ae080 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd77ed9d700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7800ae080 0x7fd7800af4d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd778014000 tx=0x7fd77801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd7767fc700 1 -- 172.21.15.131:0/640863639 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd778020070 con 0x7fd7800ae080 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd7767fc700 1 -- 172.21.15.131:0/640863639 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd778011e70 con 0x7fd7800ae080 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd7767fc700 1 -- 172.21.15.131:0/640863639 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77800cbc0 con 0x7fd7800ae080 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77800d040 con 0x7fd7800ae080 2023-12-26T21:38:14.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.097+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7800c31f0 con 0x7fd7800ae080 2023-12-26T21:38:14.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.098+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7800c3710 con 0x7fd7800ae080 2023-12-26T21:38:14.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.099+0000 7fd7767fc700 1 -- 172.21.15.131:0/640863639 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7fd778003070 con 0x7fd7800ae080 2023-12-26T21:38:14.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.100+0000 7fd7767fc700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd760067130 0x7fd7600695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.100+0000 7fd7767fc700 1 -- 172.21.15.131:0/640863639 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fd7780864d0 con 0x7fd7800ae080 2023-12-26T21:38:14.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.100+0000 7fd77e59c700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd760067130 0x7fd7600695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.101+0000 7fd77e59c700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd760067130 0x7fd7600695e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd770009f40 tx=0x7fd7700091a0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:14.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.104+0000 7fd7767fc700 1 -- 172.21.15.131:0/640863639 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7800c3710 con 0x7fd7800ae080 2023-12-26T21:38:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.322+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7800717f0 con 0x7fd760067130 2023-12-26T21:38:14.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.380+0000 7fd7767fc700 1 -- 172.21.15.131:0/640863639 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd7800717f0 con 0x7fd760067130 2023-12-26T21:38:14.381 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd760067130 msgr2=0x7fd7600695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd760067130 0x7fd7600695e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd770009f40 tx=0x7fd7700091a0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7800ae080 msgr2=0x7fd7800af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7800ae080 0x7fd7800af4d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd778014000 tx=0x7fd77801a000 comp rx=0 tx=0).stop 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 shutdown_connections 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7800ae080 0x7fd7800af4d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd760067130 0x7fd7600695e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd780069c30 0x7fd780153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 --2- 172.21.15.131:0/640863639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7800adcb0 0x7fd7800aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 >> 172.21.15.131:0/640863639 conn(0x7fd7800675b0 msgr2=0x7fd7800c80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:14.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 shutdown_connections 2023-12-26T21:38:14.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.383+0000 7fd7849ff700 1 -- 172.21.15.131:0/640863639 wait complete. 2023-12-26T21:38:14.401 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-12-26T21:38:14.402 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-12-26T21:38:14.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.596+0000 7f5a911ff700 1 Processor -- start 2023-12-26T21:38:14.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.596+0000 7f5a911ff700 1 -- start start 2023-12-26T21:38:14.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.597+0000 7f5a911ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5a8c0c9110 0x7f5a8c15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.597+0000 7f5a911ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5a8c156e50 0x7f5a8c1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.597+0000 7f5a911ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8c0af040 con 0x7f5a8c0c8d40 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.597+0000 7f5a911ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8c0ae0e0 con 0x7f5a8c156e50 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.597+0000 7f5a911ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8c0ae3c0 con 0x7f5a8c0c9110 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8b59e700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5a8c156e50 0x7f5a8c1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8b59e700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5a8c156e50 0x7f5a8c1572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52890/0 (socket says 172.21.15.131:52890) 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8b59e700 1 -- 172.21.15.131:0/2094046888 learned_addr learned my addr 172.21.15.131:0/2094046888 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8a59c700 1 --2- 172.21.15.131:0/2094046888 >> v2:172.21.15.190:3300/0 conn(0x7f5a8c0c9110 0x7f5a8c15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8b59e700 1 -- 172.21.15.131:0/2094046888 >> v2:172.21.15.190:3300/0 conn(0x7f5a8c0c9110 msgr2=0x7f5a8c15bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8b59e700 1 --2- 172.21.15.131:0/2094046888 >> v2:172.21.15.190:3300/0 conn(0x7f5a8c0c9110 0x7f5a8c15bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8b59e700 1 -- 172.21.15.131:0/2094046888 >> v1:172.21.15.190:6789/0 conn(0x7f5a8c0c8d40 legacy=0x7f5a8c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8b59e700 1 -- 172.21.15.131:0/2094046888 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0ae6a0 con 0x7f5a8c156e50 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8a59c700 1 --2- 172.21.15.131:0/2094046888 >> v2:172.21.15.190:3300/0 conn(0x7f5a8c0c9110 0x7f5a8c15bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:38:14.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.598+0000 7f5a8b59e700 1 --2- 172.21.15.131:0/2094046888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c156e50 0x7f5a8c1572b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c06fbb0 tx=0x7f5a84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba740a5b4712432 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a89d9b700 1 -- 172.21.15.131:0/2094046888 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400e070 con 0x7f5a8c156e50 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a89d9b700 1 -- 172.21.15.131:0/2094046888 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8400ba50 con 0x7f5a8c156e50 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a89d9b700 1 -- 172.21.15.131:0/2094046888 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a840092e0 con 0x7f5a8c156e50 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a911ff700 1 -- 172.21.15.131:0/2094046888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c156e50 msgr2=0x7f5a8c1572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a911ff700 1 --2- 172.21.15.131:0/2094046888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c156e50 0x7f5a8c1572b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c06fbb0 tx=0x7f5a84009840 comp rx=0 tx=0).stop 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a911ff700 1 -- 172.21.15.131:0/2094046888 shutdown_connections 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a911ff700 1 --2- 172.21.15.131:0/2094046888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c156e50 0x7f5a8c1572b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a911ff700 1 --2- 172.21.15.131:0/2094046888 >> v2:172.21.15.190:3300/0 conn(0x7f5a8c0c9110 0x7f5a8c15bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a911ff700 1 -- 172.21.15.131:0/2094046888 >> 172.21.15.131:0/2094046888 conn(0x7f5a8c0675b0 msgr2=0x7f5a8c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a911ff700 1 -- 172.21.15.131:0/2094046888 shutdown_connections 2023-12-26T21:38:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.599+0000 7f5a911ff700 1 -- 172.21.15.131:0/2094046888 wait complete. 2023-12-26T21:38:14.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.601+0000 7f5a911ff700 1 Processor -- start 2023-12-26T21:38:14.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.601+0000 7f5a911ff700 1 -- start start 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a911ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a8c0c8d40 0x7f5a8c15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a911ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c0c9110 0x7f5a8c0ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a911ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a8c156e50 0x7f5a8c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a911ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c15a800 con 0x7f5a8c0c9110 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a911ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c158080 con 0x7f5a8c156e50 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a911ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c157ac0 con 0x7f5a8c0c8d40 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a8a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c0c9110 0x7f5a8c0ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a8a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c0c9110 0x7f5a8c0ab3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52902/0 (socket says 172.21.15.131:52902) 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a8a59c700 1 -- 172.21.15.131:0/2549252316 learned_addr learned my addr 172.21.15.131:0/2549252316 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a8b59e700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a8c156e50 0x7f5a8c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.602+0000 7f5a8ad9d700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a8c0c8d40 0x7f5a8c15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a8a59c700 1 -- 172.21.15.131:0/2549252316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a8c0c8d40 msgr2=0x7f5a8c15a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a8a59c700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a8c0c8d40 0x7f5a8c15a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a8a59c700 1 -- 172.21.15.131:0/2549252316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a8c156e50 msgr2=0x7f5a8c1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a8a59c700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a8c156e50 0x7f5a8c1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a8a59c700 1 -- 172.21.15.131:0/2549252316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a84009ae0 con 0x7f5a8c0c9110 2023-12-26T21:38:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a8a59c700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c0c9110 0x7f5a8c0ab3f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000e8d0 tx=0x7f5a80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a72ffd700 1 -- 172.21.15.131:0/2549252316 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80013410 con 0x7f5a8c0c9110 2023-12-26T21:38:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a72ffd700 1 -- 172.21.15.131:0/2549252316 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a80013570 con 0x7f5a8c0c9110 2023-12-26T21:38:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c157da0 con 0x7f5a8c0c9110 2023-12-26T21:38:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a72ffd700 1 -- 172.21.15.131:0/2549252316 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80013910 con 0x7f5a8c0c9110 2023-12-26T21:38:14.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.603+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c156a30 con 0x7f5a8c0c9110 2023-12-26T21:38:14.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.604+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c0717f0 con 0x7f5a8c0c9110 2023-12-26T21:38:14.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.605+0000 7f5a72ffd700 1 -- 172.21.15.131:0/2549252316 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f5a80013cc0 con 0x7f5a8c0c9110 2023-12-26T21:38:14.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.606+0000 7f5a72ffd700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5a78067130 0x7f5a780695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:14.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.606+0000 7f5a72ffd700 1 -- 172.21.15.131:0/2549252316 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f5a80084910 con 0x7f5a8c0c9110 2023-12-26T21:38:14.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.606+0000 7f5a8ad9d700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5a78067130 0x7f5a780695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:14.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.607+0000 7f5a8ad9d700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5a78067130 0x7f5a780695e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c003860 tx=0x7f5a7c009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:14.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.610+0000 7f5a72ffd700 1 -- 172.21.15.131:0/2549252316 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a80051650 con 0x7f5a8c0c9110 2023-12-26T21:38:14.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.831+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a8c0ae980 con 0x7f5a78067130 2023-12-26T21:38:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.892+0000 7f5a72ffd700 1 -- 172.21.15.131:0/2549252316 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5a8c0ae980 con 0x7f5a78067130 2023-12-26T21:38:14.893 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-12-26T21:38:14.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5a78067130 msgr2=0x7f5a780695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5a78067130 0x7f5a780695e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c003860 tx=0x7f5a7c009040 comp rx=0 tx=0).stop 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c0c9110 msgr2=0x7f5a8c0ab3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c0c9110 0x7f5a8c0ab3f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000e8d0 tx=0x7f5a80008040 comp rx=0 tx=0).stop 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 shutdown_connections 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a8c156e50 0x7f5a8c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5a78067130 0x7f5a780695e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a8c0c8d40 0x7f5a8c15a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 --2- 172.21.15.131:0/2549252316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8c0c9110 0x7f5a8c0ab3f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 >> 172.21.15.131:0/2549252316 conn(0x7f5a8c0675b0 msgr2=0x7f5a8c15aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 shutdown_connections 2023-12-26T21:38:14.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:14.895+0000 7f5a911ff700 1 -- 172.21.15.131:0/2549252316 wait complete. 2023-12-26T21:38:14.913 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-12-26T21:38:14.913 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-12-26T21:38:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.108+0000 7fc0605ff700 1 Processor -- start 2023-12-26T21:38:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.108+0000 7fc0605ff700 1 -- start start 2023-12-26T21:38:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc0605ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb510 0x7fc058069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc0605ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc058069640 con 0x7fc0580cc590 2023-12-26T21:38:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc0605ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc058068750 con 0x7fc058142f00 2023-12-26T21:38:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc0605ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc058068a30 con 0x7fc0580cb510 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc05eb9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc0580cc590 0x7fc0580c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40530/0 (socket says 172.21.15.131:40530) 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc05eb9c700 1 -- 172.21.15.131:0/3569805823 learned_addr learned my addr 172.21.15.131:0/3569805823 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc05db9a700 1 --2- 172.21.15.131:0/3569805823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb510 0x7fc058069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc05d399700 1 -- 172.21.15.131:0/3569805823 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365921478 0 0) 0x7fc058069640 con 0x7fc0580cc590 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc05d399700 1 -- 172.21.15.131:0/3569805823 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc044003570 con 0x7fc0580cc590 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.110+0000 7fc05d399700 1 -- 172.21.15.131:0/3569805823 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857222139 0 0) 0x7fc058068750 con 0x7fc058142f00 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.111+0000 7fc05d399700 1 -- 172.21.15.131:0/3569805823 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc058069640 con 0x7fc058142f00 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.111+0000 7fc05db9a700 1 -- 172.21.15.131:0/3569805823 >> v1:172.21.15.190:6789/0 conn(0x7fc058142f00 legacy=0x7fc0580cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.111+0000 7fc05db9a700 1 -- 172.21.15.131:0/3569805823 >> v1:172.21.15.131:6789/0 conn(0x7fc0580cc590 legacy=0x7fc0580c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.111+0000 7fc05db9a700 1 -- 172.21.15.131:0/3569805823 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc058068d10 con 0x7fc0580cb510 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.111+0000 7fc05db9a700 1 --2- 172.21.15.131:0/3569805823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb510 0x7fc058069ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc048009910 tx=0x7fc0480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b971fcb96f48c588 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.111+0000 7fc05d399700 1 -- 172.21.15.131:0/3569805823 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048011630 con 0x7fc0580cb510 2023-12-26T21:38:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.111+0000 7fc05d399700 1 -- 172.21.15.131:0/3569805823 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc048011790 con 0x7fc0580cb510 2023-12-26T21:38:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.111+0000 7fc05d399700 1 -- 172.21.15.131:0/3569805823 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048011b30 con 0x7fc0580cb510 2023-12-26T21:38:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.112+0000 7fc0605ff700 1 -- 172.21.15.131:0/3569805823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb510 msgr2=0x7fc058069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.112+0000 7fc0605ff700 1 --2- 172.21.15.131:0/3569805823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb510 0x7fc058069ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc048009910 tx=0x7fc0480091a0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.112+0000 7fc0605ff700 1 -- 172.21.15.131:0/3569805823 shutdown_connections 2023-12-26T21:38:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.112+0000 7fc0605ff700 1 --2- 172.21.15.131:0/3569805823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb510 0x7fc058069ad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.112+0000 7fc0605ff700 1 -- 172.21.15.131:0/3569805823 >> 172.21.15.131:0/3569805823 conn(0x7fc058063cd0 msgr2=0x7fc05806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.112+0000 7fc0605ff700 1 -- 172.21.15.131:0/3569805823 shutdown_connections 2023-12-26T21:38:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.112+0000 7fc0605ff700 1 -- 172.21.15.131:0/3569805823 wait complete. 2023-12-26T21:38:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.113+0000 7fc0605ff700 1 Processor -- start 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.113+0000 7fc0605ff700 1 -- start start 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.114+0000 7fc0605ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580cb510 0x7fc0580c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.114+0000 7fc0605ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580cc590 0x7fc0580c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.114+0000 7fc0605ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc058142f00 0x7fc0580c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.114+0000 7fc0605ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580c8670 con 0x7fc0580cc590 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.114+0000 7fc0605ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05815f3a0 con 0x7fc0580cb510 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.114+0000 7fc0605ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05815ea70 con 0x7fc058142f00 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05db9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580cc590 0x7fc0580c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05db9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580cc590 0x7fc0580c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47250/0 (socket says 172.21.15.131:47250) 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05db9a700 1 -- 172.21.15.131:0/3806164587 learned_addr learned my addr 172.21.15.131:0/3806164587 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05eb9c700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc058142f00 0x7fc0580c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05e39b700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580cb510 0x7fc0580c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05db9a700 1 -- 172.21.15.131:0/3806164587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc058142f00 msgr2=0x7fc0580c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05db9a700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc058142f00 0x7fc0580c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05db9a700 1 -- 172.21.15.131:0/3806164587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580cb510 msgr2=0x7fc0580c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05db9a700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580cb510 0x7fc0580c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05db9a700 1 -- 172.21.15.131:0/3806164587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc048009420 con 0x7fc0580cc590 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc05db9a700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580cc590 0x7fc0580c8130 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc0480091e0 tx=0x7fc048017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.115+0000 7fc04e7fc700 1 -- 172.21.15.131:0/3806164587 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04801f070 con 0x7fc0580cc590 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.116+0000 7fc04e7fc700 1 -- 172.21.15.131:0/3806164587 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc048000c90 con 0x7fc0580cc590 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.116+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05815ed50 con 0x7fc0580cc590 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.116+0000 7fc04e7fc700 1 -- 172.21.15.131:0/3806164587 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048025420 con 0x7fc0580cc590 2023-12-26T21:38:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.116+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05815f830 con 0x7fc0580cc590 2023-12-26T21:38:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.117+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05815f510 con 0x7fc0580cc590 2023-12-26T21:38:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.117+0000 7fc04e7fc700 1 -- 172.21.15.131:0/3806164587 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7fc048023070 con 0x7fc0580cc590 2023-12-26T21:38:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.118+0000 7fc04e7fc700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc03c067130 0x7fc03c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.118+0000 7fc04e7fc700 1 -- 172.21.15.131:0/3806164587 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fc048085eb0 con 0x7fc0580cc590 2023-12-26T21:38:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.119+0000 7fc05e39b700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc03c067130 0x7fc03c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.119+0000 7fc05e39b700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc03c067130 0x7fc03c0695e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc0540037e0 tx=0x7fc054009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:15.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.123+0000 7fc04e7fc700 1 -- 172.21.15.131:0/3806164587 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc048052d00 con 0x7fc0580cc590 2023-12-26T21:38:15.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.344+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0580717f0 con 0x7fc03c067130 2023-12-26T21:38:15.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.396+0000 7fc04e7fc700 1 -- 172.21.15.131:0/3806164587 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc0580717f0 con 0x7fc03c067130 2023-12-26T21:38:15.397 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-12-26T21:38:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.398+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc03c067130 msgr2=0x7fc03c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc03c067130 0x7fc03c0695e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc0540037e0 tx=0x7fc054009040 comp rx=0 tx=0).stop 2023-12-26T21:38:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580cc590 msgr2=0x7fc0580c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580cc590 0x7fc0580c8130 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc0480091e0 tx=0x7fc048017000 comp rx=0 tx=0).stop 2023-12-26T21:38:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 shutdown_connections 2023-12-26T21:38:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc03c067130 0x7fc03c0695e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc058142f00 0x7fc0580c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580cb510 0x7fc0580c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 --2- 172.21.15.131:0/3806164587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0580cc590 0x7fc0580c8130 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 >> 172.21.15.131:0/3806164587 conn(0x7fc058063cd0 msgr2=0x7fc05806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 shutdown_connections 2023-12-26T21:38:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.399+0000 7fc0605ff700 1 -- 172.21.15.131:0/3806164587 wait complete. 2023-12-26T21:38:15.417 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-12-26T21:38:15.417 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-12-26T21:38:15.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.612+0000 7f33069ff700 1 Processor -- start 2023-12-26T21:38:15.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.612+0000 7f33069ff700 1 -- start start 2023-12-26T21:38:15.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.613+0000 7f33069ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3300156360 0x7f330006e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.613+0000 7f33069ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156730 0x7f330006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.613+0000 7f33069ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33000acfc0 con 0x7f33000c8490 2023-12-26T21:38:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.613+0000 7f33069ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330006f1f0 con 0x7f3300156360 2023-12-26T21:38:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.613+0000 7f33069ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33000c91f0 con 0x7f3300156730 2023-12-26T21:38:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f3304f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f33000c8490 0x7f33000c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40544/0 (socket says 172.21.15.131:40544) 2023-12-26T21:38:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f3304f9c700 1 -- 172.21.15.131:0/2620186574 learned_addr learned my addr 172.21.15.131:0/2620186574 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:38:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f32fffff700 1 --2- 172.21.15.131:0/2620186574 >> v2:172.21.15.190:3300/0 conn(0x7f3300156360 0x7f330006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f32ff7fe700 1 --2- 172.21.15.131:0/2620186574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156730 0x7f330006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f32feffd700 1 -- 172.21.15.131:0/2620186574 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493406333 0 0) 0x7f33000acfc0 con 0x7f33000c8490 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f32feffd700 1 -- 172.21.15.131:0/2620186574 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e8003570 con 0x7f33000c8490 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f32fffff700 1 -- 172.21.15.131:0/2620186574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156730 msgr2=0x7f330006ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f32fffff700 1 --2- 172.21.15.131:0/2620186574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156730 0x7f330006ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f32fffff700 1 -- 172.21.15.131:0/2620186574 >> v1:172.21.15.131:6789/0 conn(0x7f33000c8490 legacy=0x7f33000c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f32fffff700 1 -- 172.21.15.131:0/2620186574 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33000c94d0 con 0x7f3300156360 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.614+0000 7f32ff7fe700 1 --2- 172.21.15.131:0/2620186574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156730 0x7f330006ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.615+0000 7f32fffff700 1 --2- 172.21.15.131:0/2620186574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300156360 0x7f330006e770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32f00098e0 tx=0x7f32f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e137809680e5238 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.615+0000 7f32feffd700 1 -- 172.21.15.131:0/2620186574 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011630 con 0x7f3300156360 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.615+0000 7f32feffd700 1 -- 172.21.15.131:0/2620186574 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f0011790 con 0x7f3300156360 2023-12-26T21:38:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.615+0000 7f32feffd700 1 -- 172.21.15.131:0/2620186574 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011b30 con 0x7f3300156360 2023-12-26T21:38:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.615+0000 7f33069ff700 1 -- 172.21.15.131:0/2620186574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300156360 msgr2=0x7f330006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.616+0000 7f33069ff700 1 --2- 172.21.15.131:0/2620186574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300156360 0x7f330006e770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32f00098e0 tx=0x7f32f00091a0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.616+0000 7f33069ff700 1 -- 172.21.15.131:0/2620186574 shutdown_connections 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.616+0000 7f33069ff700 1 --2- 172.21.15.131:0/2620186574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300156360 0x7f330006e770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.616+0000 7f33069ff700 1 --2- 172.21.15.131:0/2620186574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156730 0x7f330006ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.616+0000 7f33069ff700 1 -- 172.21.15.131:0/2620186574 >> 172.21.15.131:0/2620186574 conn(0x7f33000675b0 msgr2=0x7f33000679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.616+0000 7f33069ff700 1 -- 172.21.15.131:0/2620186574 shutdown_connections 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.616+0000 7f33069ff700 1 -- 172.21.15.131:0/2620186574 wait complete. 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.617+0000 7f33069ff700 1 Processor -- start 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.617+0000 7f33069ff700 1 -- start start 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.618+0000 7f33069ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33000c8490 0x7f3300154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.618+0000 7f33069ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156360 0x7f33000c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.618+0000 7f33069ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3300156730 0x7f33000c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.618+0000 7f33069ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33000c4170 con 0x7f3300156730 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.618+0000 7f33069ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33000c3560 con 0x7f33000c8490 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.618+0000 7f33069ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33000c3840 con 0x7f3300156360 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f3304f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3300156730 0x7f33000c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32fffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33000c8490 0x7f3300154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32fffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33000c8490 0x7f3300154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:34802/0 (socket says 172.21.15.131:34802) 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32fffff700 1 -- 172.21.15.131:0/2414395334 learned_addr learned my addr 172.21.15.131:0/2414395334 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32ff7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156360 0x7f33000c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32fffff700 1 -- 172.21.15.131:0/2414395334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156360 msgr2=0x7f33000c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32fffff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156360 0x7f33000c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32fffff700 1 -- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3300156730 msgr2=0x7f33000c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32fffff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3300156730 0x7f33000c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32fffff700 1 -- 172.21.15.131:0/2414395334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f4009420 con 0x7f33000c8490 2023-12-26T21:38:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f3304f9c700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3300156730 0x7f33000c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:38:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32fffff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33000c8490 0x7f3300154aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f33000690d0 tx=0x7f32f0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.619+0000 7f32e7fff700 1 -- 172.21.15.131:0/2414395334 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f001f070 con 0x7f33000c8490 2023-12-26T21:38:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.620+0000 7f32e7fff700 1 -- 172.21.15.131:0/2414395334 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f0000bc0 con 0x7f33000c8490 2023-12-26T21:38:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.620+0000 7f32e7fff700 1 -- 172.21.15.131:0/2414395334 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011b30 con 0x7f33000c8490 2023-12-26T21:38:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.620+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f0009420 con 0x7f33000c8490 2023-12-26T21:38:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.620+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33000c3e00 con 0x7f33000c8490 2023-12-26T21:38:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.621+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33000717f0 con 0x7f33000c8490 2023-12-26T21:38:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.622+0000 7f32e7fff700 1 -- 172.21.15.131:0/2414395334 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f32f0023070 con 0x7f33000c8490 2023-12-26T21:38:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.622+0000 7f32e7fff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f32ec067130 0x7f32ec0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.622+0000 7f32e7fff700 1 -- 172.21.15.131:0/2414395334 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f32f0085d00 con 0x7f33000c8490 2023-12-26T21:38:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.623+0000 7f32ff7fe700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f32ec067130 0x7f32ec0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.623+0000 7f32ff7fe700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f32ec067130 0x7f32ec0695e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f330006e030 tx=0x7f32f40091a0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.628+0000 7f32e7fff700 1 -- 172.21.15.131:0/2414395334 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32f0052b50 con 0x7f33000c8490 2023-12-26T21:38:15.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.846+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3300067bd0 con 0x7f32ec067130 2023-12-26T21:38:15.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.898+0000 7f32e7fff700 1 -- 172.21.15.131:0/2414395334 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3300067bd0 con 0x7f32ec067130 2023-12-26T21:38:15.899 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-12-26T21:38:15.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f32ec067130 msgr2=0x7f32ec0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f32ec067130 0x7f32ec0695e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f330006e030 tx=0x7f32f40091a0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33000c8490 msgr2=0x7f3300154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:15.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33000c8490 0x7f3300154aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f33000690d0 tx=0x7f32f0017000 comp rx=0 tx=0).stop 2023-12-26T21:38:15.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 shutdown_connections 2023-12-26T21:38:15.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3300156730 0x7f33000c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33000c8490 0x7f3300154aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f32ec067130 0x7f32ec0695e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 --2- 172.21.15.131:0/2414395334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300156360 0x7f33000c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:15.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 >> 172.21.15.131:0/2414395334 conn(0x7f33000675b0 msgr2=0x7f3300155430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:15.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.900+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 shutdown_connections 2023-12-26T21:38:15.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:15.901+0000 7f33069ff700 1 -- 172.21.15.131:0/2414395334 wait complete. 2023-12-26T21:38:15.919 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-12-26T21:38:15.919 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-12-26T21:38:16.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.115+0000 7f813ebff700 1 Processor -- start 2023-12-26T21:38:16.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.115+0000 7f813ebff700 1 -- start start 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.116+0000 7f813ebff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81380a9ae0 0x7f81380a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.116+0000 7f813ebff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f813815d2b0 0x7f813815d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.116+0000 7f813ebff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813815a750 con 0x7f81381597e0 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.116+0000 7f813ebff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81380aa4c0 con 0x7f813815d2b0 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.116+0000 7f813ebff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8138159bb0 con 0x7f81380a9ae0 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.117+0000 7f813c99b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f81381597e0 0x7f81380675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40548/0 (socket says 172.21.15.131:40548) 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.117+0000 7f813c99b700 1 -- 172.21.15.131:0/3769986292 learned_addr learned my addr 172.21.15.131:0/3769986292 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.117+0000 7f813d19c700 1 --2- 172.21.15.131:0/3769986292 >> v2:172.21.15.131:3300/0 conn(0x7f813815d2b0 0x7f813815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.117+0000 7f8137fff700 1 --2- 172.21.15.131:0/3769986292 >> v2:172.21.15.190:3300/0 conn(0x7f81380a9ae0 0x7f81380a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.117+0000 7f813d19c700 1 -- 172.21.15.131:0/3769986292 >> v2:172.21.15.190:3300/0 conn(0x7f81380a9ae0 msgr2=0x7f81380a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.117+0000 7f813d19c700 1 --2- 172.21.15.131:0/3769986292 >> v2:172.21.15.190:3300/0 conn(0x7f81380a9ae0 0x7f81380a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.117+0000 7f813d19c700 1 -- 172.21.15.131:0/3769986292 >> v1:172.21.15.131:6789/0 conn(0x7f81381597e0 legacy=0x7f81380675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.117+0000 7f813d19c700 1 -- 172.21.15.131:0/3769986292 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8138068070 con 0x7f813815d2b0 2023-12-26T21:38:16.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.117+0000 7f813d19c700 1 --2- 172.21.15.131:0/3769986292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f813815d2b0 0x7f813815d680 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f813806a1b0 tx=0x7f8130009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35111a79977e8d5a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:16.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f81377fe700 1 -- 172.21.15.131:0/3769986292 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000e070 con 0x7f813815d2b0 2023-12-26T21:38:16.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f81377fe700 1 -- 172.21.15.131:0/3769986292 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813000bc30 con 0x7f813815d2b0 2023-12-26T21:38:16.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f81377fe700 1 -- 172.21.15.131:0/3769986292 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130012430 con 0x7f813815d2b0 2023-12-26T21:38:16.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f813ebff700 1 -- 172.21.15.131:0/3769986292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f813815d2b0 msgr2=0x7f813815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f813ebff700 1 --2- 172.21.15.131:0/3769986292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f813815d2b0 0x7f813815d680 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f813806a1b0 tx=0x7f8130009780 comp rx=0 tx=0).stop 2023-12-26T21:38:16.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f813ebff700 1 -- 172.21.15.131:0/3769986292 shutdown_connections 2023-12-26T21:38:16.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f813ebff700 1 --2- 172.21.15.131:0/3769986292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f813815d2b0 0x7f813815d680 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f813ebff700 1 --2- 172.21.15.131:0/3769986292 >> v2:172.21.15.190:3300/0 conn(0x7f81380a9ae0 0x7f81380a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f813ebff700 1 -- 172.21.15.131:0/3769986292 >> 172.21.15.131:0/3769986292 conn(0x7f813806b1c0 msgr2=0x7f81380aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:16.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f813ebff700 1 -- 172.21.15.131:0/3769986292 shutdown_connections 2023-12-26T21:38:16.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.118+0000 7f813ebff700 1 -- 172.21.15.131:0/3769986292 wait complete. 2023-12-26T21:38:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.120+0000 7f813ebff700 1 Processor -- start 2023-12-26T21:38:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.120+0000 7f813ebff700 1 -- start start 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813ebff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81380a9ae0 0x7f813816a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813ebff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81381597e0 0x7f81380b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813ebff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813815d2b0 0x7f8138150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813ebff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8138155c10 con 0x7f81380a9ae0 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813ebff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81380b05e0 con 0x7f813815d2b0 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813ebff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81380b0860 con 0x7f81381597e0 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813c99b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81380a9ae0 0x7f813816a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813c99b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81380a9ae0 0x7f813816a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47288/0 (socket says 172.21.15.131:47288) 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813c99b700 1 -- 172.21.15.131:0/3228610213 learned_addr learned my addr 172.21.15.131:0/3228610213 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813d19c700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813815d2b0 0x7f8138150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f8137fff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81381597e0 0x7f81380b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813c99b700 1 -- 172.21.15.131:0/3228610213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81381597e0 msgr2=0x7f81380b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813c99b700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81381597e0 0x7f81380b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813c99b700 1 -- 172.21.15.131:0/3228610213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813815d2b0 msgr2=0x7f8138150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813c99b700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813815d2b0 0x7f8138150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f813c99b700 1 -- 172.21.15.131:0/3228610213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8130009a20 con 0x7f81380a9ae0 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.121+0000 7f8137fff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81381597e0 0x7f81380b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.122+0000 7f813c99b700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81380a9ae0 0x7f813816a930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c560 tx=0x7f812c001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.122+0000 7f8134ff9700 1 -- 172.21.15.131:0/3228610213 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c005b30 con 0x7f81380a9ae0 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.122+0000 7f8134ff9700 1 -- 172.21.15.131:0/3228610213 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812c005cb0 con 0x7f81380a9ae0 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.122+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81380b0b40 con 0x7f81380a9ae0 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.122+0000 7f8134ff9700 1 -- 172.21.15.131:0/3228610213 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c007320 con 0x7f81380a9ae0 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.122+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81380b1100 con 0x7f81380a9ae0 2023-12-26T21:38:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.123+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813806dbf0 con 0x7f81380a9ae0 2023-12-26T21:38:16.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.124+0000 7f8134ff9700 1 -- 172.21.15.131:0/3228610213 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f812c01b3b0 con 0x7f81380a9ae0 2023-12-26T21:38:16.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.125+0000 7f8134ff9700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f811c067130 0x7f811c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.125+0000 7f8134ff9700 1 -- 172.21.15.131:0/3228610213 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f812c083ef0 con 0x7f81380a9ae0 2023-12-26T21:38:16.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.125+0000 7f8137fff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f811c067130 0x7f811c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.126+0000 7f8137fff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f811c067130 0x7f811c0695e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8138154770 tx=0x7f8124008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.129+0000 7f8134ff9700 1 -- 172.21.15.131:0/3228610213 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f812c050de0 con 0x7f81380a9ae0 2023-12-26T21:38:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.349+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8138155d50 con 0x7f811c067130 2023-12-26T21:38:16.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.408+0000 7f8134ff9700 1 -- 172.21.15.131:0/3228610213 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f81240096f0 con 0x7f811c067130 2023-12-26T21:38:16.408 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f811c067130 msgr2=0x7f811c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f811c067130 0x7f811c0695e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8138154770 tx=0x7f8124008040 comp rx=0 tx=0).stop 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81380a9ae0 msgr2=0x7f813816a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81380a9ae0 0x7f813816a930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c560 tx=0x7f812c001020 comp rx=0 tx=0).stop 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 shutdown_connections 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813815d2b0 0x7f8138150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81380a9ae0 0x7f813816a930 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f811c067130 0x7f811c0695e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 --2- 172.21.15.131:0/3228610213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81381597e0 0x7f81380b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 >> 172.21.15.131:0/3228610213 conn(0x7f813806b1c0 msgr2=0x7f8138151830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 shutdown_connections 2023-12-26T21:38:16.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.410+0000 7f813ebff700 1 -- 172.21.15.131:0/3228610213 wait complete. 2023-12-26T21:38:16.428 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-12-26T21:38:16.428 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-12-26T21:38:16.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.622+0000 7f24fbfff700 1 Processor -- start 2023-12-26T21:38:16.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.622+0000 7f24fbfff700 1 -- start start 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.623+0000 7f24fbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24f4142f00 0x7f24f4069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.623+0000 7f24fbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24f40cb510 0x7f24f40c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24fbfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f4069640 con 0x7f24f40cc590 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24fbfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f4068750 con 0x7f24f4142f00 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24fbfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f4068a30 con 0x7f24f40cb510 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24f9d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24f4142f00 0x7f24f4069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24f9d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24f4142f00 0x7f24f4069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47290/0 (socket says 172.21.15.131:47290) 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24f9d9b700 1 -- 172.21.15.131:0/121146830 learned_addr learned my addr 172.21.15.131:0/121146830 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24f9d9b700 1 -- 172.21.15.131:0/121146830 >> v2:172.21.15.190:3300/0 conn(0x7f24f40cb510 msgr2=0x7f24f40c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24f9d9b700 1 --2- 172.21.15.131:0/121146830 >> v2:172.21.15.190:3300/0 conn(0x7f24f40cb510 0x7f24f40c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24f9d9b700 1 -- 172.21.15.131:0/121146830 >> v1:172.21.15.190:6789/0 conn(0x7f24f40cc590 legacy=0x7f24f40cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24f9d9b700 1 -- 172.21.15.131:0/121146830 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4068d10 con 0x7f24f4142f00 2023-12-26T21:38:16.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.624+0000 7f24f9d9b700 1 --2- 172.21.15.131:0/121146830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f4142f00 0x7f24f4069ad0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24f406f230 tx=0x7f24f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a88b444ffc5c497 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24f8d99700 1 -- 172.21.15.131:0/121146830 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f000b820 con 0x7f24f4142f00 2023-12-26T21:38:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24f8d99700 1 -- 172.21.15.131:0/121146830 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f000b980 con 0x7f24f4142f00 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24f8d99700 1 -- 172.21.15.131:0/121146830 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f000bd20 con 0x7f24f4142f00 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24fbfff700 1 -- 172.21.15.131:0/121146830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f4142f00 msgr2=0x7f24f4069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24fbfff700 1 --2- 172.21.15.131:0/121146830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f4142f00 0x7f24f4069ad0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24f406f230 tx=0x7f24f0009250 comp rx=0 tx=0).stop 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24fbfff700 1 -- 172.21.15.131:0/121146830 shutdown_connections 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24fbfff700 1 --2- 172.21.15.131:0/121146830 >> v2:172.21.15.190:3300/0 conn(0x7f24f40cb510 0x7f24f40c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24fbfff700 1 --2- 172.21.15.131:0/121146830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f4142f00 0x7f24f4069ad0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24fbfff700 1 -- 172.21.15.131:0/121146830 >> 172.21.15.131:0/121146830 conn(0x7f24f4063cd0 msgr2=0x7f24f406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24fbfff700 1 -- 172.21.15.131:0/121146830 shutdown_connections 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.625+0000 7f24fbfff700 1 -- 172.21.15.131:0/121146830 wait complete. 2023-12-26T21:38:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.627+0000 7f24fbfff700 1 Processor -- start 2023-12-26T21:38:16.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.627+0000 7f24fbfff700 1 -- start start 2023-12-26T21:38:16.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24fbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f40cb510 0x7f24f40a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f40cc590 0x7f24f40c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24fbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4142f00 0x7f24f40b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24fbfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40c6ea0 con 0x7f24f40cc590 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24fbfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40c5db0 con 0x7f24f4142f00 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24fbfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40c6090 con 0x7f24f40cb510 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24f959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f40cc590 0x7f24f40c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24f959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f40cc590 0x7f24f40c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47302/0 (socket says 172.21.15.131:47302) 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24f959a700 1 -- 172.21.15.131:0/1502272299 learned_addr learned my addr 172.21.15.131:0/1502272299 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24f9d9b700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f40cb510 0x7f24f40a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24fa59c700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4142f00 0x7f24f40b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24f959a700 1 -- 172.21.15.131:0/1502272299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f40cb510 msgr2=0x7f24f40a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24f959a700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f40cb510 0x7f24f40a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24f959a700 1 -- 172.21.15.131:0/1502272299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4142f00 msgr2=0x7f24f40b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24f959a700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4142f00 0x7f24f40b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.628+0000 7f24f959a700 1 -- 172.21.15.131:0/1502272299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f00094f0 con 0x7f24f40cc590 2023-12-26T21:38:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.629+0000 7f24f959a700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f40cc590 0x7f24f40c54b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f24e400b600 tx=0x7f24e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.629+0000 7f24e9ffb700 1 -- 172.21.15.131:0/1502272299 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e400f510 con 0x7f24f40cc590 2023-12-26T21:38:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.629+0000 7f24e9ffb700 1 -- 172.21.15.131:0/1502272299 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e400f670 con 0x7f24f40cc590 2023-12-26T21:38:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.629+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f40c6370 con 0x7f24f40cc590 2023-12-26T21:38:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.629+0000 7f24e9ffb700 1 -- 172.21.15.131:0/1502272299 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e400fa30 con 0x7f24f40cc590 2023-12-26T21:38:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.629+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f40c59f0 con 0x7f24f40cc590 2023-12-26T21:38:16.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.630+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f40c7010 con 0x7f24f40cc590 2023-12-26T21:38:16.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.631+0000 7f24e9ffb700 1 -- 172.21.15.131:0/1502272299 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 10) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f24e401a400 con 0x7f24f40cc590 2023-12-26T21:38:16.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.632+0000 7f24e9ffb700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f24d8067130 0x7f24d80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:38:16.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.633+0000 7f24e9ffb700 1 -- 172.21.15.131:0/1502272299 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f24e4084510 con 0x7f24f40cc590 2023-12-26T21:38:16.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.633+0000 7f24f9d9b700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f24d8067130 0x7f24d80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:38:16.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.633+0000 7f24f9d9b700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f24d8067130 0x7f24d80695e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f24f406f230 tx=0x7f24f0016000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:38:16.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.636+0000 7f24e9ffb700 1 -- 172.21.15.131:0/1502272299 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24e40513e0 con 0x7f24f40cc590 2023-12-26T21:38:16.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.856+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f24f40717f0 con 0x7f24d8067130 2023-12-26T21:38:16.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.918+0000 7f24e9ffb700 1 -- 172.21.15.131:0/1502272299 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f24f40717f0 con 0x7f24d8067130 2023-12-26T21:38:16.919 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-12-26T21:38:16.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f24d8067130 msgr2=0x7f24d80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f24d8067130 0x7f24d80695e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f24f406f230 tx=0x7f24f0016000 comp rx=0 tx=0).stop 2023-12-26T21:38:16.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f40cc590 msgr2=0x7f24f40c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:38:16.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f40cc590 0x7f24f40c54b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f24e400b600 tx=0x7f24e40091a0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 shutdown_connections 2023-12-26T21:38:16.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4142f00 0x7f24f40b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f24d8067130 0x7f24d80695e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f40cb510 0x7f24f40a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 --2- 172.21.15.131:0/1502272299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24f40cc590 0x7f24f40c54b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:38:16.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 >> 172.21.15.131:0/1502272299 conn(0x7f24f4063cd0 msgr2=0x7f24f406d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:38:16.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 shutdown_connections 2023-12-26T21:38:16.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:38:16.920+0000 7f24fbfff700 1 -- 172.21.15.131:0/1502272299 wait complete. 2023-12-26T21:38:16.939 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-12-26T21:38:43.853 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:38:43.859 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:39:00.371 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:39:00.371+0000 7efe5a18c700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:39:00.372 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:39:00.371+0000 7efe5a18c700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:39:00.372 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:39:00.371+0000 7efe5a18c700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:39:00.372 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:39:00.371+0000 7efe5a18c700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:39:00.372 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:39:00.371+0000 7efe5a18c700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:39:13.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:39:13.981 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:39:44.099 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:39:44.105 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:40:14.398 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:40:14.405 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:40:44.508 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:40:44.514 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:41:14.646 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:41:14.652 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:41:44.768 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:41:44.774 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:42:14.889 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:42:14.895 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:42:45.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:42:45.025 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:43:00.120 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:43:00.119+0000 7efe5d993700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:43:00.120 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:43:00.119+0000 7efe5d993700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:43:00.120 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:43:00.119+0000 7efe5d993700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:43:00.120 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:43:00.119+0000 7efe5d993700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:43:00.120 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:43:00.119+0000 7efe5d993700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:43:15.133 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:43:15.138 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:43:45.246 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:43:45.253 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:44:15.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:44:15.375 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:44:45.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:44:45.505 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:45:15.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:45:15.627 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:45:45.743 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:45:45.748 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:46:15.873 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:46:15.878 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:46:45.985 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:46:46.000 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:47:16.251 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:47:16.257 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:47:46.599 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:47:46.605 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:48:16.711 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:48:16.716 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:48:46.832 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:48:46.838 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:49:16.946 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:49:16.952 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:49:20.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-12-26T21:49:21.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.144+0000 7f8258dff700 1 Processor -- start 2023-12-26T21:49:21.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.144+0000 7f8258dff700 1 -- start start 2023-12-26T21:49:21.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.145+0000 7f8258dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8254142f00 0x7f8254069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.145+0000 7f8258dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8254069640 con 0x7f82540cc590 2023-12-26T21:49:21.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.145+0000 7f8258dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8254068750 con 0x7f82540cb510 2023-12-26T21:49:21.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.145+0000 7f8258dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8254068a30 con 0x7f8254142f00 2023-12-26T21:49:21.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.145+0000 7f8251d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f82540cc590 0x7f82540cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37976/0 (socket says 172.21.15.131:37976) 2023-12-26T21:49:21.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.145+0000 7f8251d9b700 1 -- 172.21.15.131:0/157705717 learned_addr learned my addr 172.21.15.131:0/157705717 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:21.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.146+0000 7f825259c700 1 --2- 172.21.15.131:0/157705717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8254142f00 0x7f8254069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.146+0000 7f825159a700 1 -- 172.21.15.131:0/157705717 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110891935 0 0) 0x7f8254069640 con 0x7f82540cc590 2023-12-26T21:49:21.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.146+0000 7f825159a700 1 -- 172.21.15.131:0/157705717 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f823c003570 con 0x7f82540cc590 2023-12-26T21:49:21.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.146+0000 7f825159a700 1 -- 172.21.15.131:0/157705717 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281030632 0 0) 0x7f8254068750 con 0x7f82540cb510 2023-12-26T21:49:21.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.146+0000 7f825159a700 1 -- 172.21.15.131:0/157705717 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8254069640 con 0x7f82540cb510 2023-12-26T21:49:21.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.146+0000 7f825259c700 1 -- 172.21.15.131:0/157705717 >> v1:172.21.15.190:6789/0 conn(0x7f82540cb510 legacy=0x7f82540c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.146+0000 7f825259c700 1 -- 172.21.15.131:0/157705717 >> v1:172.21.15.131:6789/0 conn(0x7f82540cc590 legacy=0x7f82540cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.146+0000 7f825259c700 1 -- 172.21.15.131:0/157705717 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8254068d10 con 0x7f8254142f00 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.146+0000 7f825259c700 1 --2- 172.21.15.131:0/157705717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8254142f00 0x7f8254069ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8244009910 tx=0x7f82440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e825ceb0f3b6126d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f825159a700 1 -- 172.21.15.131:0/157705717 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244011630 con 0x7f8254142f00 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f825159a700 1 -- 172.21.15.131:0/157705717 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8244011790 con 0x7f8254142f00 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f825159a700 1 -- 172.21.15.131:0/157705717 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244011b10 con 0x7f8254142f00 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f8258dff700 1 -- 172.21.15.131:0/157705717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8254142f00 msgr2=0x7f8254069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f8258dff700 1 --2- 172.21.15.131:0/157705717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8254142f00 0x7f8254069ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8244009910 tx=0x7f82440091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f8258dff700 1 -- 172.21.15.131:0/157705717 shutdown_connections 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f8258dff700 1 --2- 172.21.15.131:0/157705717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8254142f00 0x7f8254069ad0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f8258dff700 1 -- 172.21.15.131:0/157705717 >> 172.21.15.131:0/157705717 conn(0x7f8254063cd0 msgr2=0x7f825406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f8258dff700 1 -- 172.21.15.131:0/157705717 shutdown_connections 2023-12-26T21:49:21.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.147+0000 7f8258dff700 1 -- 172.21.15.131:0/157705717 wait complete. 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.149+0000 7f8258dff700 1 Processor -- start 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.149+0000 7f8258dff700 1 -- start start 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f8258dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82540cb510 0x7f825416a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f8258dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82540cc590 0x7f8254157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f8258dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254142f00 0x7f8254152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f8258dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82541564b0 con 0x7f82540cb510 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f8258dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8254154cd0 con 0x7f8254142f00 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f8258dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82541581b0 con 0x7f82540cc590 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f825259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82540cb510 0x7f825416a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f825259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82540cb510 0x7f825416a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37704/0 (socket says 172.21.15.131:37704) 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f825259c700 1 -- 172.21.15.131:0/752002416 learned_addr learned my addr 172.21.15.131:0/752002416 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f8251d9b700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82540cc590 0x7f8254157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.150+0000 7f8252d9d700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254142f00 0x7f8254152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f825259c700 1 -- 172.21.15.131:0/752002416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82540cc590 msgr2=0x7f8254157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f825259c700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82540cc590 0x7f8254157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f825259c700 1 -- 172.21.15.131:0/752002416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254142f00 msgr2=0x7f8254152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f825259c700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254142f00 0x7f8254152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f825259c700 1 -- 172.21.15.131:0/752002416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8244009420 con 0x7f82540cb510 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f8252d9d700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254142f00 0x7f8254152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f825259c700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82540cb510 0x7f825416a190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8244009860 tx=0x7f8244017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f82427fc700 1 -- 172.21.15.131:0/752002416 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244011b10 con 0x7f82540cb510 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f82427fc700 1 -- 172.21.15.131:0/752002416 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8244027450 con 0x7f82540cb510 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8254153460 con 0x7f82540cb510 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f82427fc700 1 -- 172.21.15.131:0/752002416 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244027810 con 0x7f82540cb510 2023-12-26T21:49:21.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.151+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8254153a10 con 0x7f82540cb510 2023-12-26T21:49:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.152+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82540717f0 con 0x7f82540cb510 2023-12-26T21:49:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.153+0000 7f82427fc700 1 -- 172.21.15.131:0/752002416 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f8244027bc0 con 0x7f82540cb510 2023-12-26T21:49:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.154+0000 7f82427fc700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8234067130 0x7f82340695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.154+0000 7f82427fc700 1 -- 172.21.15.131:0/752002416 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f8244086910 con 0x7f82540cb510 2023-12-26T21:49:21.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.154+0000 7f8251d9b700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8234067130 0x7f82340695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.155+0000 7f8251d9b700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8234067130 0x7f82340695e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8254152bb0 tx=0x7f8248009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:21.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.159+0000 7f82427fc700 1 -- 172.21.15.131:0/752002416 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82440536f0 con 0x7f82540cb510 2023-12-26T21:49:21.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.384+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8254068ff0 con 0x7f8234067130 2023-12-26T21:49:21.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.462+0000 7f82427fc700 1 -- 172.21.15.131:0/752002416 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8254068ff0 con 0x7f8234067130 2023-12-26T21:49:21.463 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-12-26T21:49:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.464+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8234067130 msgr2=0x7f82340695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.464+0000 7f8258dff700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8234067130 0x7f82340695e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8254152bb0 tx=0x7f8248009040 comp rx=0 tx=0).stop 2023-12-26T21:49:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.464+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82540cb510 msgr2=0x7f825416a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.464+0000 7f8258dff700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82540cb510 0x7f825416a190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8244009860 tx=0x7f8244017000 comp rx=0 tx=0).stop 2023-12-26T21:49:21.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.465+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 shutdown_connections 2023-12-26T21:49:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.465+0000 7f8258dff700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254142f00 0x7f8254152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.465+0000 7f8258dff700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82540cb510 0x7f825416a190 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.465+0000 7f8258dff700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8234067130 0x7f82340695e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.465+0000 7f8258dff700 1 --2- 172.21.15.131:0/752002416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82540cc590 0x7f8254157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.465+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 >> 172.21.15.131:0/752002416 conn(0x7f8254063cd0 msgr2=0x7f82541408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.465+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 shutdown_connections 2023-12-26T21:49:21.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.465+0000 7f8258dff700 1 -- 172.21.15.131:0/752002416 wait complete. 2023-12-26T21:49:21.483 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-12-26T21:49:21.483 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-12-26T21:49:21.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.679+0000 7fca76fff700 1 Processor -- start 2023-12-26T21:49:21.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.679+0000 7fca76fff700 1 -- start start 2023-12-26T21:49:21.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.680+0000 7fca76fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fca70156730 0x7fca700c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.680+0000 7fca76fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca700acfc0 con 0x7fca70156360 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.680+0000 7fca76fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca700c91f0 con 0x7fca7006ee80 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.680+0000 7fca76fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca700c94d0 con 0x7fca70156730 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.680+0000 7fca74d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fca70156360 0x7fca7006e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37982/0 (socket says 172.21.15.131:37982) 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.680+0000 7fca74d9b700 1 -- 172.21.15.131:0/3351128830 learned_addr learned my addr 172.21.15.131:0/3351128830 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6ffff700 1 --2- 172.21.15.131:0/3351128830 >> v2:172.21.15.190:3300/0 conn(0x7fca70156730 0x7fca700c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101986799 0 0) 0x7fca700acfc0 con 0x7fca70156360 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca60003570 con 0x7fca70156360 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516737229 0 0) 0x7fca700c91f0 con 0x7fca7006ee80 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca700acfc0 con 0x7fca7006ee80 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1269310699 0 0) 0x7fca60003570 con 0x7fca70156360 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 >> v2:172.21.15.190:3300/0 conn(0x7fca70156730 msgr2=0x7fca700c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:21.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 --2- 172.21.15.131:0/3351128830 >> v2:172.21.15.190:3300/0 conn(0x7fca70156730 0x7fca700c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 >> v1:172.21.15.190:6789/0 conn(0x7fca7006ee80 legacy=0x7fca700c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700acc50 con 0x7fca70156360 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fca64001470 con 0x7fca70156360 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 >> v1:172.21.15.131:6789/0 conn(0x7fca70156360 legacy=0x7fca7006e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca60004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca60004810 0x7fca60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca60005210 0x7fca6000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca700c94d0 con 0x7fca60003c40 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6000e340 con 0x7fca60004810 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6000e4d0 con 0x7fca60005210 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.681+0000 7fca74d9b700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca60004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca6ffff700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca60004810 0x7fca60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca7559c700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca60005210 0x7fca6000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca74d9b700 1 -- 172.21.15.131:0/3351128830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca60005210 msgr2=0x7fca6000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca74d9b700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca60005210 0x7fca6000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca74d9b700 1 -- 172.21.15.131:0/3351128830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca60004810 msgr2=0x7fca60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca74d9b700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca60004810 0x7fca60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca7559c700 1 -- 172.21.15.131:0/3351128830 reap_dead start 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca74d9b700 1 -- 172.21.15.131:0/3351128830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca60003570 con 0x7fca60003c40 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca74d9b700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca60004030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c1e0 tx=0x7fca7006f2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4794cb99d8854d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6400cbd0 con 0x7fca60003c40 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6400cd30 con 0x7fca60003c40 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.682+0000 7fca6f7fe700 1 -- 172.21.15.131:0/3351128830 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6400cea0 con 0x7fca60003c40 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.683+0000 7fca76fff700 1 -- 172.21.15.131:0/3351128830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 msgr2=0x7fca60004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.683+0000 7fca76fff700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca60004030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c1e0 tx=0x7fca7006f2e0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.683+0000 7fca76fff700 1 -- 172.21.15.131:0/3351128830 shutdown_connections 2023-12-26T21:49:21.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.683+0000 7fca76fff700 1 --2- 172.21.15.131:0/3351128830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca60004030 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.683+0000 7fca76fff700 1 -- 172.21.15.131:0/3351128830 >> 172.21.15.131:0/3351128830 conn(0x7fca700675b0 msgr2=0x7fca700679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:21.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.683+0000 7fca76fff700 1 -- 172.21.15.131:0/3351128830 shutdown_connections 2023-12-26T21:49:21.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.683+0000 7fca76fff700 1 -- 172.21.15.131:0/3351128830 wait complete. 2023-12-26T21:49:21.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.684+0000 7fca76fff700 1 Processor -- start 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.685+0000 7fca76fff700 1 -- start start 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.685+0000 7fca76fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca7014b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.685+0000 7fca76fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca700cac70 0x7fca700c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.685+0000 7fca76fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca700c5470 0x7fca700c0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.685+0000 7fca76fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca700c58e0 con 0x7fca60003c40 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.685+0000 7fca76fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca700aaff0 con 0x7fca700cac70 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.685+0000 7fca76fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca700aa3e0 con 0x7fca700c5470 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca74d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca7014b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca74d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca7014b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37716/0 (socket says 172.21.15.131:37716) 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca74d9b700 1 -- 172.21.15.131:0/2238534310 learned_addr learned my addr 172.21.15.131:0/2238534310 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca6ffff700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca700cac70 0x7fca700c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca7559c700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca700c5470 0x7fca700c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca74d9b700 1 -- 172.21.15.131:0/2238534310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca700c5470 msgr2=0x7fca700c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca74d9b700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca700c5470 0x7fca700c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca74d9b700 1 -- 172.21.15.131:0/2238534310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca700cac70 msgr2=0x7fca700c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca74d9b700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca700cac70 0x7fca700c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca74d9b700 1 -- 172.21.15.131:0/2238534310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700aa6c0 con 0x7fca60003c40 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca7559c700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca700c5470 0x7fca700c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca74d9b700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca7014b3d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca64001ed0 tx=0x7fca64015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.686+0000 7fca6cff9700 1 -- 172.21.15.131:0/2238534310 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64022070 con 0x7fca60003c40 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.687+0000 7fca6cff9700 1 -- 172.21.15.131:0/2238534310 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca64004e80 con 0x7fca60003c40 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.687+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca700aa9a0 con 0x7fca60003c40 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.687+0000 7fca6cff9700 1 -- 172.21.15.131:0/2238534310 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6400cbd0 con 0x7fca60003c40 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.687+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7014b910 con 0x7fca60003c40 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.688+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca700aae40 con 0x7fca60003c40 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.688+0000 7fca6cff9700 1 -- 172.21.15.131:0/2238534310 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fca64005040 con 0x7fca60003c40 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.689+0000 7fca6cff9700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca54067130 0x7fca540695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:21.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.689+0000 7fca6cff9700 1 -- 172.21.15.131:0/2238534310 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fca640856b0 con 0x7fca60003c40 2023-12-26T21:49:21.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.690+0000 7fca6ffff700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca54067130 0x7fca540695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:21.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.690+0000 7fca6ffff700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca54067130 0x7fca540695e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca700c4ad0 tx=0x7fca5c008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:21.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.694+0000 7fca6cff9700 1 -- 172.21.15.131:0/2238534310 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca64052490 con 0x7fca60003c40 2023-12-26T21:49:21.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.915+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fca7006b660 con 0x7fca54067130 2023-12-26T21:49:21.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.992+0000 7fca6cff9700 1 -- 172.21.15.131:0/2238534310 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fca7006b660 con 0x7fca54067130 2023-12-26T21:49:21.993 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-12-26T21:49:21.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca54067130 msgr2=0x7fca540695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca54067130 0x7fca540695e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca700c4ad0 tx=0x7fca5c008040 comp rx=0 tx=0).stop 2023-12-26T21:49:21.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 msgr2=0x7fca7014b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:21.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca7014b3d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca64001ed0 tx=0x7fca64015000 comp rx=0 tx=0).stop 2023-12-26T21:49:21.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 shutdown_connections 2023-12-26T21:49:21.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fca54067130 0x7fca540695e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca700c5470 0x7fca700c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca60003c40 0x7fca7014b3d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 --2- 172.21.15.131:0/2238534310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca700cac70 0x7fca700c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:21.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 >> 172.21.15.131:0/2238534310 conn(0x7fca700675b0 msgr2=0x7fca70155330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:21.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 shutdown_connections 2023-12-26T21:49:21.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:21.995+0000 7fca76fff700 1 -- 172.21.15.131:0/2238534310 wait complete. 2023-12-26T21:49:22.013 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-12-26T21:49:22.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-12-26T21:49:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.206+0000 7f2ab7dff700 1 Processor -- start 2023-12-26T21:49:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.207+0000 7f2ab7dff700 1 -- start start 2023-12-26T21:49:22.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab7dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ab0156730 0x7f2ab00c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab7dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2ab006ee80 0x7f2ab00c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab7dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00acfc0 con 0x7f2ab0156360 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab7dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00c91f0 con 0x7f2ab0156730 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab7dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00c94d0 con 0x7f2ab006ee80 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab539a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ab0156730 0x7f2ab00c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab539a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ab0156730 0x7f2ab00c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37732/0 (socket says 172.21.15.131:37732) 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab5b9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2ab0156360 0x7f2ab006e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37986/0 (socket says 172.21.15.131:37986) 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab539a700 1 -- 172.21.15.131:0/1871306965 learned_addr learned my addr 172.21.15.131:0/1871306965 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.208+0000 7f2ab639c700 1 --2- 172.21.15.131:0/1871306965 >> v2:172.21.15.190:3300/0 conn(0x7f2ab006ee80 0x7f2ab00c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab539a700 1 -- 172.21.15.131:0/1871306965 >> v2:172.21.15.190:3300/0 conn(0x7f2ab006ee80 msgr2=0x7f2ab00c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab539a700 1 --2- 172.21.15.131:0/1871306965 >> v2:172.21.15.190:3300/0 conn(0x7f2ab006ee80 0x7f2ab00c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab539a700 1 -- 172.21.15.131:0/1871306965 >> v1:172.21.15.131:6789/0 conn(0x7f2ab0156360 legacy=0x7f2ab006e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab539a700 1 -- 172.21.15.131:0/1871306965 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00acc50 con 0x7f2ab0156730 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab4b99700 1 -- 172.21.15.131:0/1871306965 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002648454 0 0) 0x7f2ab00acfc0 con 0x7f2ab0156360 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab539a700 1 --2- 172.21.15.131:0/1871306965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 0x7f2ab00c8490 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00b22a0 tx=0x7f2aa000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a5ab13e871c95c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab4b99700 1 -- 172.21.15.131:0/1871306965 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000dc50 con 0x7f2ab0156730 2023-12-26T21:49:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab4b99700 1 -- 172.21.15.131:0/1871306965 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa000ddb0 con 0x7f2ab0156730 2023-12-26T21:49:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab4b99700 1 -- 172.21.15.131:0/1871306965 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00196d0 con 0x7f2ab0156730 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab7dff700 1 -- 172.21.15.131:0/1871306965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 msgr2=0x7f2ab00c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.209+0000 7f2ab7dff700 1 --2- 172.21.15.131:0/1871306965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 0x7f2ab00c8490 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00b22a0 tx=0x7f2aa000b4f0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.210+0000 7f2ab7dff700 1 -- 172.21.15.131:0/1871306965 shutdown_connections 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.210+0000 7f2ab7dff700 1 --2- 172.21.15.131:0/1871306965 >> v2:172.21.15.190:3300/0 conn(0x7f2ab006ee80 0x7f2ab00c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.210+0000 7f2ab7dff700 1 --2- 172.21.15.131:0/1871306965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 0x7f2ab00c8490 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.210+0000 7f2ab7dff700 1 -- 172.21.15.131:0/1871306965 >> 172.21.15.131:0/1871306965 conn(0x7f2ab00675b0 msgr2=0x7f2ab00679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.210+0000 7f2ab7dff700 1 -- 172.21.15.131:0/1871306965 shutdown_connections 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.210+0000 7f2ab7dff700 1 -- 172.21.15.131:0/1871306965 wait complete. 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.211+0000 7f2ab7dff700 1 Processor -- start 2023-12-26T21:49:22.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.211+0000 7f2ab7dff700 1 -- start start 2023-12-26T21:49:22.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.212+0000 7f2ab7dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ab006ee80 0x7f2ab0152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.212+0000 7f2ab7dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ab0156360 0x7f2ab00aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.212+0000 7f2ab7dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 0x7f2ab00b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.212+0000 7f2ab7dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0156b50 con 0x7f2ab0156730 2023-12-26T21:49:22.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.212+0000 7f2ab7dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0156e30 con 0x7f2ab0156360 2023-12-26T21:49:22.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.212+0000 7f2ab7dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00af890 con 0x7f2ab006ee80 2023-12-26T21:49:22.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.212+0000 7f2ab639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 0x7f2ab00b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.212+0000 7f2ab639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 0x7f2ab00b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37744/0 (socket says 172.21.15.131:37744) 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.212+0000 7f2ab639c700 1 -- 172.21.15.131:0/499850929 learned_addr learned my addr 172.21.15.131:0/499850929 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab539a700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ab0156360 0x7f2ab00aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab5b9b700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ab006ee80 0x7f2ab0152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab639c700 1 -- 172.21.15.131:0/499850929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ab006ee80 msgr2=0x7f2ab0152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab639c700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ab006ee80 0x7f2ab0152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab639c700 1 -- 172.21.15.131:0/499850929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ab0156360 msgr2=0x7f2ab00aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab639c700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ab0156360 0x7f2ab00aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab639c700 1 -- 172.21.15.131:0/499850929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa000b790 con 0x7f2ab0156730 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab539a700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ab0156360 0x7f2ab00aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab639c700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 0x7f2ab00b07c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009fa0 tx=0x7f2aa8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2aa5ffb700 1 -- 172.21.15.131:0/499850929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800ed20 con 0x7f2ab0156730 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2aa5ffb700 1 -- 172.21.15.131:0/499850929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa800ee80 con 0x7f2ab0156730 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab00afb70 con 0x7f2ab0156730 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2aa5ffb700 1 -- 172.21.15.131:0/499850929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8014670 con 0x7f2ab0156730 2023-12-26T21:49:22.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.213+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab0153420 con 0x7f2ab0156730 2023-12-26T21:49:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.214+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab00717f0 con 0x7f2ab0156730 2023-12-26T21:49:22.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.216+0000 7f2aa5ffb700 1 -- 172.21.15.131:0/499850929 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f2aa80149e0 con 0x7f2ab0156730 2023-12-26T21:49:22.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.216+0000 7f2aa5ffb700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a94067130 0x7f2a940695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.217+0000 7f2aa5ffb700 1 -- 172.21.15.131:0/499850929 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f2aa8085270 con 0x7f2ab0156730 2023-12-26T21:49:22.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.217+0000 7f2ab5b9b700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a94067130 0x7f2a940695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.217+0000 7f2ab5b9b700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a94067130 0x7f2a940695e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0011e0 tx=0x7f2aac007bf0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:22.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.221+0000 7f2aa5ffb700 1 -- 172.21.15.131:0/499850929 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2aa8052050 con 0x7f2ab0156730 2023-12-26T21:49:22.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.444+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ab0156f70 con 0x7f2a94067130 2023-12-26T21:49:22.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.522+0000 7f2aa5ffb700 1 -- 172.21.15.131:0/499850929 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f2ab0156f70 con 0x7f2a94067130 2023-12-26T21:49:22.523 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a94067130 msgr2=0x7f2a940695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a94067130 0x7f2a940695e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0011e0 tx=0x7f2aac007bf0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 msgr2=0x7f2ab00b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 0x7f2ab00b07c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009fa0 tx=0x7f2aa8009900 comp rx=0 tx=0).stop 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 shutdown_connections 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ab0156730 0x7f2ab00b07c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2a94067130 0x7f2a940695e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ab006ee80 0x7f2ab0152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 --2- 172.21.15.131:0/499850929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ab0156360 0x7f2ab00aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 >> 172.21.15.131:0/499850929 conn(0x7f2ab00675b0 msgr2=0x7f2ab00cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 shutdown_connections 2023-12-26T21:49:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.525+0000 7f2ab7dff700 1 -- 172.21.15.131:0/499850929 wait complete. 2023-12-26T21:49:22.543 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-12-26T21:49:22.543 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-12-26T21:49:22.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.738+0000 7f5e47bff700 1 Processor -- start 2023-12-26T21:49:22.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.738+0000 7f5e47bff700 1 -- start start 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.739+0000 7f5e47bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e40156730 0x7f5e400c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.739+0000 7f5e47bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5e4006ee80 0x7f5e400c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.739+0000 7f5e47bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e400acfc0 con 0x7f5e40156360 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.739+0000 7f5e47bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e400c91f0 con 0x7f5e4006ee80 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.739+0000 7f5e47bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e400c94d0 con 0x7f5e40156730 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.739+0000 7f5e4619c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5e4006ee80 0x7f5e400c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.739+0000 7f5e4619c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5e4006ee80 0x7f5e400c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37760/0 (socket says 172.21.15.131:37760) 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.739+0000 7f5e4599b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5e40156360 0x7f5e4006e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38002/0 (socket says 172.21.15.131:38002) 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.739+0000 7f5e4619c700 1 -- 172.21.15.131:0/586895421 learned_addr learned my addr 172.21.15.131:0/586895421 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e4519a700 1 --2- 172.21.15.131:0/586895421 >> v2:172.21.15.190:3300/0 conn(0x7f5e40156730 0x7f5e400c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e4619c700 1 -- 172.21.15.131:0/586895421 >> v2:172.21.15.190:3300/0 conn(0x7f5e40156730 msgr2=0x7f5e400c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e4619c700 1 --2- 172.21.15.131:0/586895421 >> v2:172.21.15.190:3300/0 conn(0x7f5e40156730 0x7f5e400c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e4619c700 1 -- 172.21.15.131:0/586895421 >> v1:172.21.15.131:6789/0 conn(0x7f5e40156360 legacy=0x7f5e4006e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e4619c700 1 -- 172.21.15.131:0/586895421 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e400acc50 con 0x7f5e4006ee80 2023-12-26T21:49:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e44999700 1 -- 172.21.15.131:0/586895421 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619741929 0 0) 0x7f5e400acfc0 con 0x7f5e40156360 2023-12-26T21:49:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e4619c700 1 --2- 172.21.15.131:0/586895421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4006ee80 0x7f5e400c89d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e40071110 tx=0x7f5e38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770a04d60960aa35 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e44999700 1 -- 172.21.15.131:0/586895421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800e070 con 0x7f5e4006ee80 2023-12-26T21:49:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e44999700 1 -- 172.21.15.131:0/586895421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3800ba50 con 0x7f5e4006ee80 2023-12-26T21:49:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.740+0000 7f5e44999700 1 -- 172.21.15.131:0/586895421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380092e0 con 0x7f5e4006ee80 2023-12-26T21:49:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.741+0000 7f5e47bff700 1 -- 172.21.15.131:0/586895421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4006ee80 msgr2=0x7f5e400c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.741+0000 7f5e47bff700 1 --2- 172.21.15.131:0/586895421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4006ee80 0x7f5e400c89d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e40071110 tx=0x7f5e38009840 comp rx=0 tx=0).stop 2023-12-26T21:49:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.741+0000 7f5e47bff700 1 -- 172.21.15.131:0/586895421 shutdown_connections 2023-12-26T21:49:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.741+0000 7f5e47bff700 1 --2- 172.21.15.131:0/586895421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e4006ee80 0x7f5e400c89d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.741+0000 7f5e47bff700 1 --2- 172.21.15.131:0/586895421 >> v2:172.21.15.190:3300/0 conn(0x7f5e40156730 0x7f5e400c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.741+0000 7f5e47bff700 1 -- 172.21.15.131:0/586895421 >> 172.21.15.131:0/586895421 conn(0x7f5e400675b0 msgr2=0x7f5e400679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.741+0000 7f5e47bff700 1 -- 172.21.15.131:0/586895421 shutdown_connections 2023-12-26T21:49:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.741+0000 7f5e47bff700 1 -- 172.21.15.131:0/586895421 wait complete. 2023-12-26T21:49:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.743+0000 7f5e47bff700 1 Processor -- start 2023-12-26T21:49:22.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.743+0000 7f5e47bff700 1 -- start start 2023-12-26T21:49:22.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e47bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e4006ee80 0x7f5e400caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e47bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e40156360 0x7f5e400c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e47bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e40156730 0x7f5e4016c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e47bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400c47d0 con 0x7f5e40156360 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e47bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400c41c0 con 0x7f5e40156730 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e47bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4016d0d0 con 0x7f5e4006ee80 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4519a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e40156360 0x7f5e400c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4519a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e40156360 0x7f5e400c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37764/0 (socket says 172.21.15.131:37764) 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4519a700 1 -- 172.21.15.131:0/2364422794 learned_addr learned my addr 172.21.15.131:0/2364422794 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4619c700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e40156730 0x7f5e4016c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4599b700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e4006ee80 0x7f5e400caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4519a700 1 -- 172.21.15.131:0/2364422794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e4006ee80 msgr2=0x7f5e400caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4519a700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e4006ee80 0x7f5e400caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4519a700 1 -- 172.21.15.131:0/2364422794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e40156730 msgr2=0x7f5e4016c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4519a700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e40156730 0x7f5e4016c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4519a700 1 -- 172.21.15.131:0/2364422794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38009ae0 con 0x7f5e40156360 2023-12-26T21:49:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.744+0000 7f5e4599b700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e4006ee80 0x7f5e400caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.745+0000 7f5e4519a700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e40156360 0x7f5e400c4c60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5e30006b00 tx=0x7f5e30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.745+0000 7f5e35ffb700 1 -- 172.21.15.131:0/2364422794 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e300125f0 con 0x7f5e40156360 2023-12-26T21:49:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.745+0000 7f5e35ffb700 1 -- 172.21.15.131:0/2364422794 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e30012770 con 0x7f5e40156360 2023-12-26T21:49:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.745+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4016d3b0 con 0x7f5e40156360 2023-12-26T21:49:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.745+0000 7f5e35ffb700 1 -- 172.21.15.131:0/2364422794 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30012b30 con 0x7f5e40156360 2023-12-26T21:49:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.745+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e400abab0 con 0x7f5e40156360 2023-12-26T21:49:22.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.746+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e400c4940 con 0x7f5e40156360 2023-12-26T21:49:22.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.747+0000 7f5e35ffb700 1 -- 172.21.15.131:0/2364422794 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f5e3001e460 con 0x7f5e40156360 2023-12-26T21:49:22.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.748+0000 7f5e35ffb700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5e24067160 0x7f5e24069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:22.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.748+0000 7f5e35ffb700 1 -- 172.21.15.131:0/2364422794 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f5e30084ff0 con 0x7f5e40156360 2023-12-26T21:49:22.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.748+0000 7f5e4599b700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5e24067160 0x7f5e24069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:22.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.749+0000 7f5e4599b700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5e24067160 0x7f5e24069610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0011e0 tx=0x7f5e3c007c90 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:22.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.752+0000 7f5e35ffb700 1 -- 172.21.15.131:0/2364422794 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e30055860 con 0x7f5e40156360 2023-12-26T21:49:22.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:22.974+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e400717f0 con 0x7f5e24067160 2023-12-26T21:49:23.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.048+0000 7f5e35ffb700 1 -- 172.21.15.131:0/2364422794 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5e400717f0 con 0x7f5e24067160 2023-12-26T21:49:23.049 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-12-26T21:49:23.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.050+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5e24067160 msgr2=0x7f5e24069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.050+0000 7f5e47bff700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5e24067160 0x7f5e24069610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0011e0 tx=0x7f5e3c007c90 comp rx=0 tx=0).stop 2023-12-26T21:49:23.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.050+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e40156360 msgr2=0x7f5e400c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.050+0000 7f5e47bff700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e40156360 0x7f5e400c4c60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5e30006b00 tx=0x7f5e30008040 comp rx=0 tx=0).stop 2023-12-26T21:49:23.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.050+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 shutdown_connections 2023-12-26T21:49:23.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.050+0000 7f5e47bff700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e40156730 0x7f5e4016c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.050+0000 7f5e47bff700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5e24067160 0x7f5e24069610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.050+0000 7f5e47bff700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e4006ee80 0x7f5e400caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.050+0000 7f5e47bff700 1 --2- 172.21.15.131:0/2364422794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e40156360 0x7f5e400c4c60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.051+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 >> 172.21.15.131:0/2364422794 conn(0x7f5e400675b0 msgr2=0x7f5e40067980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:23.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.051+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 shutdown_connections 2023-12-26T21:49:23.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.051+0000 7f5e47bff700 1 -- 172.21.15.131:0/2364422794 wait complete. 2023-12-26T21:49:23.069 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-12-26T21:49:23.069 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-12-26T21:49:23.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.263+0000 7fd54e3ff700 1 Processor -- start 2023-12-26T21:49:23.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.263+0000 7fd54e3ff700 1 -- start start 2023-12-26T21:49:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.264+0000 7fd54e3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5480b22e0 0x7fd5480cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.264+0000 7fd54e3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 0x7fd5480cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.264+0000 7fd54e3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5481521a0 con 0x7fd548068b40 2023-12-26T21:49:23.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.264+0000 7fd54e3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd548068f10 con 0x7fd5480b22e0 2023-12-26T21:49:23.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.264+0000 7fd54e3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd548158c20 con 0x7fd5481586c0 2023-12-26T21:49:23.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.265+0000 7fd5477fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5480b22e0 0x7fd5480cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.265+0000 7fd5477fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5480b22e0 0x7fd5480cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37766/0 (socket says 172.21.15.131:37766) 2023-12-26T21:49:23.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.265+0000 7fd5477fe700 1 -- 172.21.15.131:0/637500033 learned_addr learned my addr 172.21.15.131:0/637500033 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.265+0000 7fd54c99c700 1 --2- 172.21.15.131:0/637500033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 0x7fd5480cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.265+0000 7fd5477fe700 1 -- 172.21.15.131:0/637500033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 msgr2=0x7fd5480cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.265+0000 7fd5477fe700 1 --2- 172.21.15.131:0/637500033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 0x7fd5480cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.265+0000 7fd5477fe700 1 -- 172.21.15.131:0/637500033 >> v1:172.21.15.190:6789/0 conn(0x7fd548068b40 legacy=0x7fd5480b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.265+0000 7fd5477fe700 1 -- 172.21.15.131:0/637500033 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54806de50 con 0x7fd5480b22e0 2023-12-26T21:49:23.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.265+0000 7fd5477fe700 1 --2- 172.21.15.131:0/637500033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5480b22e0 0x7fd5480cc3b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0099b0 tx=0x7fd53c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cea38c316c28d74 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.267+0000 7fd546ffd700 1 -- 172.21.15.131:0/637500033 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c009ea0 con 0x7fd5480b22e0 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd546ffd700 1 -- 172.21.15.131:0/637500033 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c00b860 con 0x7fd5480b22e0 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd546ffd700 1 -- 172.21.15.131:0/637500033 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00bc00 con 0x7fd5480b22e0 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd54e3ff700 1 -- 172.21.15.131:0/637500033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5480b22e0 msgr2=0x7fd5480cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd54e3ff700 1 --2- 172.21.15.131:0/637500033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5480b22e0 0x7fd5480cc3b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0099b0 tx=0x7fd53c009250 comp rx=0 tx=0).stop 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd54e3ff700 1 -- 172.21.15.131:0/637500033 shutdown_connections 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd54e3ff700 1 --2- 172.21.15.131:0/637500033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 0x7fd5480cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd54e3ff700 1 --2- 172.21.15.131:0/637500033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5480b22e0 0x7fd5480cc3b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd54e3ff700 1 -- 172.21.15.131:0/637500033 >> 172.21.15.131:0/637500033 conn(0x7fd54806b710 msgr2=0x7fd54806bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd54e3ff700 1 -- 172.21.15.131:0/637500033 shutdown_connections 2023-12-26T21:49:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.268+0000 7fd54e3ff700 1 -- 172.21.15.131:0/637500033 wait complete. 2023-12-26T21:49:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.270+0000 7fd54e3ff700 1 Processor -- start 2023-12-26T21:49:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.270+0000 7fd54e3ff700 1 -- start start 2023-12-26T21:49:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd54e3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd548068b40 0x7fd54813be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd54e3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5480b22e0 0x7fd54815d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd54e3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 0x7fd54815d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd54e3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480b1340 con 0x7fd548068b40 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd54e3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54813c370 con 0x7fd5480b22e0 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd54e3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480b0730 con 0x7fd5481586c0 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd547fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd548068b40 0x7fd54813be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd547fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd548068b40 0x7fd54813be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37780/0 (socket says 172.21.15.131:37780) 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd547fff700 1 -- 172.21.15.131:0/693849122 learned_addr learned my addr 172.21.15.131:0/693849122 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd54c99c700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 0x7fd54815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd5477fe700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5480b22e0 0x7fd54815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd547fff700 1 -- 172.21.15.131:0/693849122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 msgr2=0x7fd54815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd547fff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 0x7fd54815d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd547fff700 1 -- 172.21.15.131:0/693849122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5480b22e0 msgr2=0x7fd54815d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd547fff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5480b22e0 0x7fd54815d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.271+0000 7fd547fff700 1 -- 172.21.15.131:0/693849122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c0094f0 con 0x7fd548068b40 2023-12-26T21:49:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.272+0000 7fd547fff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd548068b40 0x7fd54813be30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd53800c590 tx=0x7fd53800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.272+0000 7fd52ffff700 1 -- 172.21.15.131:0/693849122 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538011590 con 0x7fd548068b40 2023-12-26T21:49:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.272+0000 7fd52ffff700 1 -- 172.21.15.131:0/693849122 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5380116f0 con 0x7fd548068b40 2023-12-26T21:49:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.272+0000 7fd52ffff700 1 -- 172.21.15.131:0/693849122 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538011ab0 con 0x7fd548068b40 2023-12-26T21:49:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.272+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5480b0a10 con 0x7fd548068b40 2023-12-26T21:49:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.272+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5480b0fd0 con 0x7fd548068b40 2023-12-26T21:49:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.273+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5480717f0 con 0x7fd548068b40 2023-12-26T21:49:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.274+0000 7fd52ffff700 1 -- 172.21.15.131:0/693849122 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fd53801b3b0 con 0x7fd548068b40 2023-12-26T21:49:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.275+0000 7fd52ffff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd534067130 0x7fd5340695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.276+0000 7fd52ffff700 1 -- 172.21.15.131:0/693849122 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fd538083e60 con 0x7fd548068b40 2023-12-26T21:49:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.276+0000 7fd5477fe700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd534067130 0x7fd5340695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.276+0000 7fd5477fe700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd534067130 0x7fd5340695e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0099b0 tx=0x7fd53c017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:23.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.279+0000 7fd52ffff700 1 -- 172.21.15.131:0/693849122 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd538050c40 con 0x7fd548068b40 2023-12-26T21:49:23.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.498+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5481521a0 con 0x7fd534067130 2023-12-26T21:49:23.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.571+0000 7fd52ffff700 1 -- 172.21.15.131:0/693849122 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd5481521a0 con 0x7fd534067130 2023-12-26T21:49:23.572 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-12-26T21:49:23.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd534067130 msgr2=0x7fd5340695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd534067130 0x7fd5340695e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0099b0 tx=0x7fd53c017000 comp rx=0 tx=0).stop 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd548068b40 msgr2=0x7fd54813be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd548068b40 0x7fd54813be30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd53800c590 tx=0x7fd53800c950 comp rx=0 tx=0).stop 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 shutdown_connections 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd534067130 0x7fd5340695e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5481586c0 0x7fd54815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd548068b40 0x7fd54813be30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 --2- 172.21.15.131:0/693849122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5480b22e0 0x7fd54815d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 >> 172.21.15.131:0/693849122 conn(0x7fd54806b710 msgr2=0x7fd5480cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 shutdown_connections 2023-12-26T21:49:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.574+0000 7fd54e3ff700 1 -- 172.21.15.131:0/693849122 wait complete. 2023-12-26T21:49:23.593 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-12-26T21:49:23.593 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-12-26T21:49:23.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.788+0000 7f558dfff700 1 Processor -- start 2023-12-26T21:49:23.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.788+0000 7f558dfff700 1 -- start start 2023-12-26T21:49:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.789+0000 7f558dfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f55880c8d40 0x7f558815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.789+0000 7f558dfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55880c9110 0x7f55880ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.789+0000 7f558dfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55880af040 con 0x7f5588156e50 2023-12-26T21:49:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.789+0000 7f558dfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55881579c0 con 0x7f55880c8d40 2023-12-26T21:49:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.789+0000 7f558dfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55880ae0e0 con 0x7f55880c9110 2023-12-26T21:49:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f55877fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f55880c8d40 0x7f558815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f5587fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5588156e50 0x7f55881572b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38014/0 (socket says 172.21.15.131:38014) 2023-12-26T21:49:23.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f5587fff700 1 -- 172.21.15.131:0/402343193 learned_addr learned my addr 172.21.15.131:0/402343193 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:23.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f5586ffd700 1 --2- 172.21.15.131:0/402343193 >> v2:172.21.15.190:3300/0 conn(0x7f55880c9110 0x7f55880ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f55877fe700 1 -- 172.21.15.131:0/402343193 >> v2:172.21.15.190:3300/0 conn(0x7f55880c9110 msgr2=0x7f55880ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:23.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f55877fe700 1 --2- 172.21.15.131:0/402343193 >> v2:172.21.15.190:3300/0 conn(0x7f55880c9110 0x7f55880ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f55877fe700 1 -- 172.21.15.131:0/402343193 >> v1:172.21.15.131:6789/0 conn(0x7f5588156e50 legacy=0x7f55881572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f55877fe700 1 -- 172.21.15.131:0/402343193 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55880ae3c0 con 0x7f55880c8d40 2023-12-26T21:49:23.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f55867fc700 1 -- 172.21.15.131:0/402343193 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991350225 0 0) 0x7f55880af040 con 0x7f5588156e50 2023-12-26T21:49:23.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.790+0000 7f55877fe700 1 --2- 172.21.15.131:0/402343193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 0x7f558815bc30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f558806fbb0 tx=0x7f55780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ad784032afd830 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:23.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f55867fc700 1 -- 172.21.15.131:0/402343193 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557800b6b0 con 0x7f55880c8d40 2023-12-26T21:49:23.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f55867fc700 1 -- 172.21.15.131:0/402343193 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557800b810 con 0x7f55880c8d40 2023-12-26T21:49:23.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f55867fc700 1 -- 172.21.15.131:0/402343193 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557800bbb0 con 0x7f55880c8d40 2023-12-26T21:49:23.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f558dfff700 1 -- 172.21.15.131:0/402343193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 msgr2=0x7f558815bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f558dfff700 1 --2- 172.21.15.131:0/402343193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 0x7f558815bc30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f558806fbb0 tx=0x7f55780091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f558dfff700 1 -- 172.21.15.131:0/402343193 shutdown_connections 2023-12-26T21:49:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f558dfff700 1 --2- 172.21.15.131:0/402343193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 0x7f558815bc30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f558dfff700 1 --2- 172.21.15.131:0/402343193 >> v2:172.21.15.190:3300/0 conn(0x7f55880c9110 0x7f55880ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f558dfff700 1 -- 172.21.15.131:0/402343193 >> 172.21.15.131:0/402343193 conn(0x7f55880675b0 msgr2=0x7f55880679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f558dfff700 1 -- 172.21.15.131:0/402343193 shutdown_connections 2023-12-26T21:49:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.791+0000 7f558dfff700 1 -- 172.21.15.131:0/402343193 wait complete. 2023-12-26T21:49:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.793+0000 7f558dfff700 1 Processor -- start 2023-12-26T21:49:23.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.793+0000 7f558dfff700 1 -- start start 2023-12-26T21:49:23.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f558dfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 0x7f55880c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f558dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55880c9110 0x7f55881592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f558dfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5588156e50 0x7f558806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f558dfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5588159d50 con 0x7f55880c8d40 2023-12-26T21:49:23.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f558dfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558815a030 con 0x7f5588156e50 2023-12-26T21:49:23.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f558dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558815a310 con 0x7f55880c9110 2023-12-26T21:49:23.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f55877fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 0x7f55880c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f55877fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 0x7f55880c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37800/0 (socket says 172.21.15.131:37800) 2023-12-26T21:49:23.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f55877fe700 1 -- 172.21.15.131:0/3503181676 learned_addr learned my addr 172.21.15.131:0/3503181676 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:23.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f5586ffd700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55880c9110 0x7f55881592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f5587fff700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5588156e50 0x7f558806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f55877fe700 1 -- 172.21.15.131:0/3503181676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55880c9110 msgr2=0x7f55881592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f55877fe700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55880c9110 0x7f55881592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f55877fe700 1 -- 172.21.15.131:0/3503181676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5588156e50 msgr2=0x7f558806e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.794+0000 7f55877fe700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5588156e50 0x7f558806e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.795+0000 7f55877fe700 1 -- 172.21.15.131:0/3503181676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578009420 con 0x7f55880c8d40 2023-12-26T21:49:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.795+0000 7f55877fe700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 0x7f55880c7380 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f55880c8a80 tx=0x7f5578017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.795+0000 7f556f7fe700 1 -- 172.21.15.131:0/3503181676 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578011420 con 0x7f55880c8d40 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.795+0000 7f556f7fe700 1 -- 172.21.15.131:0/3503181676 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5578011580 con 0x7f55880c8d40 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.795+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558815a5f0 con 0x7f55880c8d40 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.795+0000 7f556f7fe700 1 -- 172.21.15.131:0/3503181676 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578011940 con 0x7f55880c8d40 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.795+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55880c78c0 con 0x7f55880c8d40 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.796+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55880717f0 con 0x7f55880c8d40 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.797+0000 7f556f7fe700 1 -- 172.21.15.131:0/3503181676 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f5578011d30 con 0x7f55880c8d40 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.798+0000 7f556f7fe700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5574067130 0x7f55740695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.798+0000 7f556f7fe700 1 -- 172.21.15.131:0/3503181676 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f5578086530 con 0x7f55880c8d40 2023-12-26T21:49:23.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.798+0000 7f5586ffd700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5574067130 0x7f55740695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:23.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.799+0000 7f5586ffd700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5574067130 0x7f55740695e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5588155600 tx=0x7f557c008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:23.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:23.802+0000 7f556f7fe700 1 -- 172.21.15.131:0/3503181676 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5578053310 con 0x7f55880c8d40 2023-12-26T21:49:24.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.020+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f55880a99f0 con 0x7f5574067130 2023-12-26T21:49:24.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.100+0000 7f556f7fe700 1 -- 172.21.15.131:0/3503181676 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f55880a99f0 con 0x7f5574067130 2023-12-26T21:49:24.100 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5574067130 msgr2=0x7f55740695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5574067130 0x7f55740695e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5588155600 tx=0x7f557c008040 comp rx=0 tx=0).stop 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 msgr2=0x7f55880c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 0x7f55880c7380 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f55880c8a80 tx=0x7f5578017000 comp rx=0 tx=0).stop 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 shutdown_connections 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5588156e50 0x7f558806e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55880c8d40 0x7f55880c7380 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5574067130 0x7f55740695e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 --2- 172.21.15.131:0/3503181676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55880c9110 0x7f55881592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 >> 172.21.15.131:0/3503181676 conn(0x7f55880675b0 msgr2=0x7f5588154220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:24.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 shutdown_connections 2023-12-26T21:49:24.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.102+0000 7f558dfff700 1 -- 172.21.15.131:0/3503181676 wait complete. 2023-12-26T21:49:24.120 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-12-26T21:49:24.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-12-26T21:49:24.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.317+0000 7f22ea7ff700 1 Processor -- start 2023-12-26T21:49:24.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.317+0000 7f22ea7ff700 1 -- start start 2023-12-26T21:49:24.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22ea7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f22e40c8d40 0x7f22e415bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22ea7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f22e40c9110 0x7f22e40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22ea7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40af040 con 0x7f22e4156e50 2023-12-26T21:49:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22ea7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e41579c0 con 0x7f22e40c8d40 2023-12-26T21:49:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22ea7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40ae0e0 con 0x7f22e40c9110 2023-12-26T21:49:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22e3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f22e40c8d40 0x7f22e415bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22e8d9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f22e4156e50 0x7f22e41572b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38016/0 (socket says 172.21.15.131:38016) 2023-12-26T21:49:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22e8d9c700 1 -- 172.21.15.131:0/885580085 learned_addr learned my addr 172.21.15.131:0/885580085 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22dbfff700 1 --2- 172.21.15.131:0/885580085 >> v2:172.21.15.190:3300/0 conn(0x7f22e40c9110 0x7f22e40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22e3fff700 1 -- 172.21.15.131:0/885580085 >> v2:172.21.15.190:3300/0 conn(0x7f22e40c9110 msgr2=0x7f22e40ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.319+0000 7f22e3fff700 1 --2- 172.21.15.131:0/885580085 >> v2:172.21.15.190:3300/0 conn(0x7f22e40c9110 0x7f22e40ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.320+0000 7f22e3fff700 1 -- 172.21.15.131:0/885580085 >> v1:172.21.15.131:6789/0 conn(0x7f22e4156e50 legacy=0x7f22e41572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.320+0000 7f22e3fff700 1 -- 172.21.15.131:0/885580085 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40ae3c0 con 0x7f22e40c8d40 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.320+0000 7f22e37fe700 1 -- 172.21.15.131:0/885580085 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668831459 0 0) 0x7f22e40af040 con 0x7f22e4156e50 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.320+0000 7f22e3fff700 1 --2- 172.21.15.131:0/885580085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e40c8d40 0x7f22e415bc30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22e406fbb0 tx=0x7f22d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7924da48605bf6f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.320+0000 7f22e37fe700 1 -- 172.21.15.131:0/885580085 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4011630 con 0x7f22e40c8d40 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.320+0000 7f22e37fe700 1 -- 172.21.15.131:0/885580085 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4011790 con 0x7f22e40c8d40 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.320+0000 7f22e37fe700 1 -- 172.21.15.131:0/885580085 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4011b30 con 0x7f22e40c8d40 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.321+0000 7f22ea7ff700 1 -- 172.21.15.131:0/885580085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e40c8d40 msgr2=0x7f22e415bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.321+0000 7f22ea7ff700 1 --2- 172.21.15.131:0/885580085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e40c8d40 0x7f22e415bc30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22e406fbb0 tx=0x7f22d40091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.321+0000 7f22ea7ff700 1 -- 172.21.15.131:0/885580085 shutdown_connections 2023-12-26T21:49:24.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.321+0000 7f22ea7ff700 1 --2- 172.21.15.131:0/885580085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e40c8d40 0x7f22e415bc30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.321+0000 7f22ea7ff700 1 --2- 172.21.15.131:0/885580085 >> v2:172.21.15.190:3300/0 conn(0x7f22e40c9110 0x7f22e40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.321+0000 7f22ea7ff700 1 -- 172.21.15.131:0/885580085 >> 172.21.15.131:0/885580085 conn(0x7f22e40675b0 msgr2=0x7f22e40679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:24.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.321+0000 7f22ea7ff700 1 -- 172.21.15.131:0/885580085 shutdown_connections 2023-12-26T21:49:24.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.321+0000 7f22ea7ff700 1 -- 172.21.15.131:0/885580085 wait complete. 2023-12-26T21:49:24.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.322+0000 7f22ea7ff700 1 Processor -- start 2023-12-26T21:49:24.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.323+0000 7f22ea7ff700 1 -- start start 2023-12-26T21:49:24.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.323+0000 7f22ea7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22e40c8d40 0x7f22e40a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.323+0000 7f22ea7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22e40c9110 0x7f22e415e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.323+0000 7f22ea7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e4156e50 0x7f22e406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.323+0000 7f22ea7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40c1670 con 0x7f22e4156e50 2023-12-26T21:49:24.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.323+0000 7f22ea7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40a9e00 con 0x7f22e40c8d40 2023-12-26T21:49:24.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.323+0000 7f22ea7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e416d460 con 0x7f22e40c9110 2023-12-26T21:49:24.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e8d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e4156e50 0x7f22e406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e8d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e4156e50 0x7f22e406ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37808/0 (socket says 172.21.15.131:37808) 2023-12-26T21:49:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e8d9c700 1 -- 172.21.15.131:0/4176252833 learned_addr learned my addr 172.21.15.131:0/4176252833 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e3fff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22e40c8d40 0x7f22e40a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22dbfff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22e40c9110 0x7f22e415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e8d9c700 1 -- 172.21.15.131:0/4176252833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22e40c9110 msgr2=0x7f22e415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e8d9c700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22e40c9110 0x7f22e415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e8d9c700 1 -- 172.21.15.131:0/4176252833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22e40c8d40 msgr2=0x7f22e40a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e8d9c700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22e40c8d40 0x7f22e40a98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e8d9c700 1 -- 172.21.15.131:0/4176252833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4009420 con 0x7f22e4156e50 2023-12-26T21:49:24.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.324+0000 7f22e8d9c700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e4156e50 0x7f22e406ecc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00cbf0 tx=0x7f22dc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:24.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.325+0000 7f22e0ff9700 1 -- 172.21.15.131:0/4176252833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00a9c0 con 0x7f22e4156e50 2023-12-26T21:49:24.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.325+0000 7f22e0ff9700 1 -- 172.21.15.131:0/4176252833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc00ab20 con 0x7f22e4156e50 2023-12-26T21:49:24.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.325+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e416d740 con 0x7f22e4156e50 2023-12-26T21:49:24.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.325+0000 7f22e0ff9700 1 -- 172.21.15.131:0/4176252833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0070f0 con 0x7f22e4156e50 2023-12-26T21:49:24.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.325+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e416dd00 con 0x7f22e4156e50 2023-12-26T21:49:24.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.326+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40c17e0 con 0x7f22e4156e50 2023-12-26T21:49:24.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.327+0000 7f22e0ff9700 1 -- 172.21.15.131:0/4176252833 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f22dc007520 con 0x7f22e4156e50 2023-12-26T21:49:24.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.328+0000 7f22e0ff9700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f22c4067130 0x7f22c40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.328+0000 7f22e0ff9700 1 -- 172.21.15.131:0/4176252833 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f22dc084f20 con 0x7f22e4156e50 2023-12-26T21:49:24.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.328+0000 7f22e3fff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f22c4067130 0x7f22c40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.329+0000 7f22e3fff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f22c4067130 0x7f22c40695e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f22d40093f0 tx=0x7f22d4017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:24.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.332+0000 7f22e0ff9700 1 -- 172.21.15.131:0/4176252833 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22e40c17e0 con 0x7f22e4156e50 2023-12-26T21:49:24.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.554+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f22e40c17b0 con 0x7f22c4067130 2023-12-26T21:49:24.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.624+0000 7f22e0ff9700 1 -- 172.21.15.131:0/4176252833 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f22e40c17b0 con 0x7f22c4067130 2023-12-26T21:49:24.624 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-12-26T21:49:24.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f22c4067130 msgr2=0x7f22c40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f22c4067130 0x7f22c40695e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f22d40093f0 tx=0x7f22d4017000 comp rx=0 tx=0).stop 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e4156e50 msgr2=0x7f22e406ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e4156e50 0x7f22e406ecc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00cbf0 tx=0x7f22dc0019b0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 shutdown_connections 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e4156e50 0x7f22e406ecc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22e40c8d40 0x7f22e40a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f22c4067130 0x7f22c40695e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 --2- 172.21.15.131:0/4176252833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22e40c9110 0x7f22e415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 >> 172.21.15.131:0/4176252833 conn(0x7f22e40675b0 msgr2=0x7f22e40c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 shutdown_connections 2023-12-26T21:49:24.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.627+0000 7f22ea7ff700 1 -- 172.21.15.131:0/4176252833 wait complete. 2023-12-26T21:49:24.645 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-12-26T21:49:24.645 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-12-26T21:49:24.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.837+0000 7f4adddff700 1 Processor -- start 2023-12-26T21:49:24.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.838+0000 7f4adddff700 1 -- start start 2023-12-26T21:49:24.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4adddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 0x7f4ad8069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4adddff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ad80cc590 0x7f4ad80cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4adddff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad8069640 con 0x7f4ad80cb340 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4adddff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad80cb7c0 con 0x7f4ad80cc590 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4adddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad8068750 con 0x7f4ad8142f00 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4ad6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ad80cc590 0x7f4ad80cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4ad7fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4ad80cb340 0x7f4ad80c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38020/0 (socket says 172.21.15.131:38020) 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4ad7fff700 1 -- 172.21.15.131:0/3120847473 learned_addr learned my addr 172.21.15.131:0/3120847473 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4ad6ffd700 1 -- 172.21.15.131:0/3120847473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 msgr2=0x7f4ad8069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.839+0000 7f4ad77fe700 1 --2- 172.21.15.131:0/3120847473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 0x7f4ad8069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.840+0000 7f4ad67fc700 1 -- 172.21.15.131:0/3120847473 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098332378 0 0) 0x7f4ad8069640 con 0x7f4ad80cb340 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.840+0000 7f4ad6ffd700 1 --2- 172.21.15.131:0/3120847473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 0x7f4ad8069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.840+0000 7f4ad6ffd700 1 -- 172.21.15.131:0/3120847473 >> v1:172.21.15.131:6789/0 conn(0x7f4ad80cb340 legacy=0x7f4ad80c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.840+0000 7f4ad6ffd700 1 -- 172.21.15.131:0/3120847473 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad8068a30 con 0x7f4ad80cc590 2023-12-26T21:49:24.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.840+0000 7f4ad77fe700 1 --2- 172.21.15.131:0/3120847473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 0x7f4ad8069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:49:24.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.840+0000 7f4ad6ffd700 1 --2- 172.21.15.131:0/3120847473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 0x7f4ad80cae00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0098e0 tx=0x7f4acc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5d303e38a05bb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:24.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.840+0000 7f4ad67fc700 1 -- 172.21.15.131:0/3120847473 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc009e80 con 0x7f4ad80cc590 2023-12-26T21:49:24.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.840+0000 7f4ad67fc700 1 -- 172.21.15.131:0/3120847473 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc00b860 con 0x7f4ad80cc590 2023-12-26T21:49:24.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.840+0000 7f4ad67fc700 1 -- 172.21.15.131:0/3120847473 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc00bbe0 con 0x7f4ad80cc590 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.841+0000 7f4adddff700 1 -- 172.21.15.131:0/3120847473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 msgr2=0x7f4ad80cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.841+0000 7f4adddff700 1 --2- 172.21.15.131:0/3120847473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 0x7f4ad80cae00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0098e0 tx=0x7f4acc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.841+0000 7f4adddff700 1 -- 172.21.15.131:0/3120847473 shutdown_connections 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.841+0000 7f4adddff700 1 --2- 172.21.15.131:0/3120847473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 0x7f4ad8069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.841+0000 7f4adddff700 1 --2- 172.21.15.131:0/3120847473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 0x7f4ad80cae00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.841+0000 7f4adddff700 1 -- 172.21.15.131:0/3120847473 >> 172.21.15.131:0/3120847473 conn(0x7f4ad8063cd0 msgr2=0x7f4ad806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.841+0000 7f4adddff700 1 -- 172.21.15.131:0/3120847473 shutdown_connections 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.841+0000 7f4adddff700 1 -- 172.21.15.131:0/3120847473 wait complete. 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.842+0000 7f4adddff700 1 Processor -- start 2023-12-26T21:49:24.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.842+0000 7f4adddff700 1 -- start start 2023-12-26T21:49:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.843+0000 7f4adddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad80cb340 0x7f4ad80c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.843+0000 7f4adddff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 0x7f4ad80b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.843+0000 7f4adddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 0x7f4ad80b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.843+0000 7f4adddff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad816ac70 con 0x7f4ad80cc590 2023-12-26T21:49:24.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.843+0000 7f4adddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80c6f40 con 0x7f4ad80cb340 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.843+0000 7f4adddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80c66a0 con 0x7f4ad8142f00 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 0x7f4ad80b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 0x7f4ad80b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37816/0 (socket says 172.21.15.131:37816) 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad6ffd700 1 -- 172.21.15.131:0/50767444 learned_addr learned my addr 172.21.15.131:0/50767444 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad7fff700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 0x7f4ad80b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad77fe700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad80cb340 0x7f4ad80c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad6ffd700 1 -- 172.21.15.131:0/50767444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 msgr2=0x7f4ad80b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad6ffd700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 0x7f4ad80b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad6ffd700 1 -- 172.21.15.131:0/50767444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad80cb340 msgr2=0x7f4ad80c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad6ffd700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad80cb340 0x7f4ad80c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad6ffd700 1 -- 172.21.15.131:0/50767444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc009420 con 0x7f4ad80cc590 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.844+0000 7f4ad6ffd700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 0x7f4ad80b2720 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4ad806f3d0 tx=0x7f4acc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.845+0000 7f4abf7fe700 1 -- 172.21.15.131:0/50767444 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc00bbe0 con 0x7f4ad80cc590 2023-12-26T21:49:24.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.845+0000 7f4abf7fe700 1 -- 172.21.15.131:0/50767444 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc011420 con 0x7f4ad80cc590 2023-12-26T21:49:24.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.845+0000 7f4abf7fe700 1 -- 172.21.15.131:0/50767444 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc011800 con 0x7f4ad80cc590 2023-12-26T21:49:24.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.845+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad80c6980 con 0x7f4ad80cc590 2023-12-26T21:49:24.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.845+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad80c6390 con 0x7f4ad80cc590 2023-12-26T21:49:24.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.846+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad816aa40 con 0x7f4ad80cc590 2023-12-26T21:49:24.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.847+0000 7f4abf7fe700 1 -- 172.21.15.131:0/50767444 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f4acc011c30 con 0x7f4ad80cc590 2023-12-26T21:49:24.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.848+0000 7f4abf7fe700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4ac4067130 0x7f4ac40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:24.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.848+0000 7f4abf7fe700 1 -- 172.21.15.131:0/50767444 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f4acc086450 con 0x7f4ad80cc590 2023-12-26T21:49:24.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.848+0000 7f4ad77fe700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4ac4067130 0x7f4ac40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:24.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.848+0000 7f4ad77fe700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4ac4067130 0x7f4ac40695e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8006fd0 tx=0x7f4ac8008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:24.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:24.852+0000 7f4abf7fe700 1 -- 172.21.15.131:0/50767444 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4acc0532d0 con 0x7f4ad80cc590 2023-12-26T21:49:25.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.076+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ad80717f0 con 0x7f4ac4067130 2023-12-26T21:49:25.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.147+0000 7f4abf7fe700 1 -- 172.21.15.131:0/50767444 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4ad80717f0 con 0x7f4ac4067130 2023-12-26T21:49:25.147 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4ac4067130 msgr2=0x7f4ac40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4ac4067130 0x7f4ac40695e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8006fd0 tx=0x7f4ac8008040 comp rx=0 tx=0).stop 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 msgr2=0x7f4ad80b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 0x7f4ad80b2720 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4ad806f3d0 tx=0x7f4acc017000 comp rx=0 tx=0).stop 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 shutdown_connections 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4ac4067130 0x7f4ac40695e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad8142f00 0x7f4ad80b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad80cb340 0x7f4ad80c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 --2- 172.21.15.131:0/50767444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ad80cc590 0x7f4ad80b2720 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 >> 172.21.15.131:0/50767444 conn(0x7f4ad8063cd0 msgr2=0x7f4ad81408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:25.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 shutdown_connections 2023-12-26T21:49:25.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.149+0000 7f4adddff700 1 -- 172.21.15.131:0/50767444 wait complete. 2023-12-26T21:49:25.167 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-12-26T21:49:25.168 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-12-26T21:49:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.363+0000 7fda0a3ff700 1 Processor -- start 2023-12-26T21:49:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.363+0000 7fda0a3ff700 1 -- start start 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.364+0000 7fda0a3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fda040ae460 0x7fda040c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.364+0000 7fda0a3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda040acfc0 con 0x7fda04156360 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.364+0000 7fda0a3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda040c91f0 con 0x7fda04156730 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.364+0000 7fda0a3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040c94d0 con 0x7fda040ae460 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.364+0000 7fda03fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fda04156360 0x7fda0406e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45758/0 (socket says 172.21.15.131:45758) 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.364+0000 7fda03fff700 1 -- 172.21.15.131:0/47810477 learned_addr learned my addr 172.21.15.131:0/47810477 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.364+0000 7fda0899c700 1 --2- 172.21.15.131:0/47810477 >> v2:172.21.15.190:3300/0 conn(0x7fda040ae460 0x7fda040c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570108543 0 0) 0x7fda040acfc0 con 0x7fda04156360 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f0003570 con 0x7fda04156360 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183446104 0 0) 0x7fda040c91f0 con 0x7fda04156730 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda040acfc0 con 0x7fda04156730 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1294090490 0 0) 0x7fd9f0003570 con 0x7fda04156360 2023-12-26T21:49:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 >> v2:172.21.15.190:3300/0 conn(0x7fda040ae460 msgr2=0x7fda040c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 --2- 172.21.15.131:0/47810477 >> v2:172.21.15.190:3300/0 conn(0x7fda040ae460 0x7fda040c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 >> v1:172.21.15.190:6789/0 conn(0x7fda04156730 legacy=0x7fda040c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040acc50 con 0x7fda04156360 2023-12-26T21:49:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fd9f4001470 con 0x7fda04156360 2023-12-26T21:49:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda0899c700 1 --2- 172.21.15.131:0/47810477 >> v2:172.21.15.190:3300/0 conn(0x7fda040ae460 0x7fda040c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 >> v1:172.21.15.131:6789/0 conn(0x7fda04156360 legacy=0x7fda0406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f0003bf0 0x7fd9f0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fd9f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9f00051c0 0x7fd9f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040c94d0 con 0x7fd9f0004810 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f000c0b0 con 0x7fd9f0003bf0 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f000c270 con 0x7fd9f00051c0 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fd9fbfff700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fd9f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.365+0000 7fda03fff700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f0003bf0 0x7fd9f0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fda0899c700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9f00051c0 0x7fd9f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fd9fbfff700 1 -- 172.21.15.131:0/47810477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9f00051c0 msgr2=0x7fd9f000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fd9fbfff700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9f00051c0 0x7fd9f000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fd9fbfff700 1 -- 172.21.15.131:0/47810477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f0003bf0 msgr2=0x7fd9f0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fd9fbfff700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f0003bf0 0x7fd9f0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fd9fbfff700 1 -- 172.21.15.131:0/47810477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0003570 con 0x7fd9f0004810 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fda0899c700 1 -- 172.21.15.131:0/47810477 reap_dead start 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fd9fbfff700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fd9f0004c80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0019e0 tx=0x7fd9ec007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce11cef12e0e8887 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00c5d0 con 0x7fd9f0004810 2023-12-26T21:49:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec00c730 con 0x7fd9f0004810 2023-12-26T21:49:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.366+0000 7fda037fe700 1 -- 172.21.15.131:0/47810477 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00caf0 con 0x7fd9f0004810 2023-12-26T21:49:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.367+0000 7fda0a3ff700 1 -- 172.21.15.131:0/47810477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 msgr2=0x7fd9f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.367+0000 7fda0a3ff700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fd9f0004c80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0019e0 tx=0x7fd9ec007d30 comp rx=0 tx=0).stop 2023-12-26T21:49:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.367+0000 7fda0a3ff700 1 -- 172.21.15.131:0/47810477 shutdown_connections 2023-12-26T21:49:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.367+0000 7fda0a3ff700 1 --2- 172.21.15.131:0/47810477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fd9f0004c80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.367+0000 7fda0a3ff700 1 -- 172.21.15.131:0/47810477 >> 172.21.15.131:0/47810477 conn(0x7fda040675b0 msgr2=0x7fda040679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.367+0000 7fda0a3ff700 1 -- 172.21.15.131:0/47810477 shutdown_connections 2023-12-26T21:49:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.367+0000 7fda0a3ff700 1 -- 172.21.15.131:0/47810477 wait complete. 2023-12-26T21:49:25.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.368+0000 7fda0a3ff700 1 Processor -- start 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.369+0000 7fda0a3ff700 1 -- start start 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.369+0000 7fda0a3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fda040b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.369+0000 7fda0a3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda040c5330 0x7fda04154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.369+0000 7fda0a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda041687c0 0x7fda04168c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.369+0000 7fda0a3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040c6190 con 0x7fd9f0004810 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.369+0000 7fda0a3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda041691a0 con 0x7fda040c5330 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.369+0000 7fda0a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda04157b20 con 0x7fda041687c0 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda03fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fda040b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda03fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fda040b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48548/0 (socket says 172.21.15.131:48548) 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda03fff700 1 -- 172.21.15.131:0/2265625403 learned_addr learned my addr 172.21.15.131:0/2265625403 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fd9fbfff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda040c5330 0x7fda04154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda0899c700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda041687c0 0x7fda04168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda03fff700 1 -- 172.21.15.131:0/2265625403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda041687c0 msgr2=0x7fda04168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda03fff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda041687c0 0x7fda04168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda03fff700 1 -- 172.21.15.131:0/2265625403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda040c5330 msgr2=0x7fda04154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda03fff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda040c5330 0x7fda04154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda03fff700 1 -- 172.21.15.131:0/2265625403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040c4550 con 0x7fd9f0004810 2023-12-26T21:49:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda0899c700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda041687c0 0x7fda04168c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.370+0000 7fda03fff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fda040b1a20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000ff0 tx=0x7fda040acf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.371+0000 7fda00ff9700 1 -- 172.21.15.131:0/2265625403 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4018030 con 0x7fd9f0004810 2023-12-26T21:49:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.371+0000 7fda00ff9700 1 -- 172.21.15.131:0/2265625403 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f400f660 con 0x7fd9f0004810 2023-12-26T21:49:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.371+0000 7fda00ff9700 1 -- 172.21.15.131:0/2265625403 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400fa20 con 0x7fd9f0004810 2023-12-26T21:49:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.371+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda040c58f0 con 0x7fd9f0004810 2023-12-26T21:49:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.371+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda040c5eb0 con 0x7fd9f0004810 2023-12-26T21:49:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.372+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda040717f0 con 0x7fd9f0004810 2023-12-26T21:49:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.373+0000 7fda00ff9700 1 -- 172.21.15.131:0/2265625403 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fd9f401b070 con 0x7fd9f0004810 2023-12-26T21:49:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.373+0000 7fda00ff9700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd9e4067120 0x7fd9e40695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.374+0000 7fda00ff9700 1 -- 172.21.15.131:0/2265625403 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fd9f4019070 con 0x7fd9f0004810 2023-12-26T21:49:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.374+0000 7fd9fbfff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd9e4067120 0x7fd9e40695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.374+0000 7fd9fbfff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd9e4067120 0x7fd9e40695d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec000c00 tx=0x7fd9ec015000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:25.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.378+0000 7fda00ff9700 1 -- 172.21.15.131:0/2265625403 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9f4051630 con 0x7fd9f0004810 2023-12-26T21:49:25.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.600+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fda040ac870 con 0x7fd9e4067120 2023-12-26T21:49:25.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.673+0000 7fda00ff9700 1 -- 172.21.15.131:0/2265625403 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fda040ac870 con 0x7fd9e4067120 2023-12-26T21:49:25.673 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-12-26T21:49:25.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd9e4067120 msgr2=0x7fd9e40695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd9e4067120 0x7fd9e40695d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec000c00 tx=0x7fd9ec015000 comp rx=0 tx=0).stop 2023-12-26T21:49:25.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 msgr2=0x7fda040b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fda040b1a20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000ff0 tx=0x7fda040acf30 comp rx=0 tx=0).stop 2023-12-26T21:49:25.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 shutdown_connections 2023-12-26T21:49:25.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd9e4067120 0x7fd9e40695d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda041687c0 0x7fda04168c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9f0004810 0x7fda040b1a20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 --2- 172.21.15.131:0/2265625403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda040c5330 0x7fda04154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 >> 172.21.15.131:0/2265625403 conn(0x7fda040675b0 msgr2=0x7fda040c7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 shutdown_connections 2023-12-26T21:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.675+0000 7fda0a3ff700 1 -- 172.21.15.131:0/2265625403 wait complete. 2023-12-26T21:49:25.693 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-12-26T21:49:25.694 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-12-26T21:49:25.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.886+0000 7fb3dd5ff700 1 Processor -- start 2023-12-26T21:49:25.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.887+0000 7fb3dd5ff700 1 -- start start 2023-12-26T21:49:25.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3dd5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb3d80cf390 0x7fb3d80c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3dd5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 0x7fb3d80cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3dd5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d80cbb00 con 0x7fb3d8159810 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3dd5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d80cb2a0 con 0x7fb3d80cf390 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3dd5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80ce040 con 0x7fb3d80cd0a0 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3d6ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb3d8159810 0x7fb3d80cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45768/0 (socket says 172.21.15.131:45768) 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3d6ffd700 1 -- 172.21.15.131:0/4140610624 learned_addr learned my addr 172.21.15.131:0/4140610624 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3d67fc700 1 --2- 172.21.15.131:0/4140610624 >> v2:172.21.15.131:3300/0 conn(0x7fb3d80cf390 0x7fb3d80c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3d77fe700 1 --2- 172.21.15.131:0/4140610624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 0x7fb3d80cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3d5ffb700 1 -- 172.21.15.131:0/4140610624 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186379863 0 0) 0x7fb3d80cbb00 con 0x7fb3d8159810 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.888+0000 7fb3d5ffb700 1 -- 172.21.15.131:0/4140610624 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3c0003570 con 0x7fb3d8159810 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3d67fc700 1 -- 172.21.15.131:0/4140610624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 msgr2=0x7fb3d80cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3d67fc700 1 --2- 172.21.15.131:0/4140610624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 0x7fb3d80cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3d67fc700 1 -- 172.21.15.131:0/4140610624 >> v1:172.21.15.131:6789/0 conn(0x7fb3d8159810 legacy=0x7fb3d80cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3d67fc700 1 -- 172.21.15.131:0/4140610624 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d8159be0 con 0x7fb3d80cf390 2023-12-26T21:49:25.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3d67fc700 1 --2- 172.21.15.131:0/4140610624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 0x7fb3d80c5440 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0098e0 tx=0x7fb3cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aded49e1b3687f47 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:25.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3d5ffb700 1 -- 172.21.15.131:0/4140610624 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc011630 con 0x7fb3d80cf390 2023-12-26T21:49:25.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3d5ffb700 1 -- 172.21.15.131:0/4140610624 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc011790 con 0x7fb3d80cf390 2023-12-26T21:49:25.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3d5ffb700 1 -- 172.21.15.131:0/4140610624 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc011b30 con 0x7fb3d80cf390 2023-12-26T21:49:25.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/4140610624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 msgr2=0x7fb3d80c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.889+0000 7fb3dd5ff700 1 --2- 172.21.15.131:0/4140610624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 0x7fb3d80c5440 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0098e0 tx=0x7fb3cc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.890+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/4140610624 shutdown_connections 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.890+0000 7fb3dd5ff700 1 --2- 172.21.15.131:0/4140610624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 0x7fb3d80cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.890+0000 7fb3dd5ff700 1 --2- 172.21.15.131:0/4140610624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 0x7fb3d80c5440 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.890+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/4140610624 >> 172.21.15.131:0/4140610624 conn(0x7fb3d8099950 msgr2=0x7fb3d806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.890+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/4140610624 shutdown_connections 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.890+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/4140610624 wait complete. 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.891+0000 7fb3dd5ff700 1 Processor -- start 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.891+0000 7fb3dd5ff700 1 -- start start 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.892+0000 7fb3dd5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 0x7fb3d80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.892+0000 7fb3dd5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 0x7fb3d80c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.892+0000 7fb3dd5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d8159810 0x7fb3d814b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.892+0000 7fb3dd5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8155bf0 con 0x7fb3d80cf390 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.892+0000 7fb3dd5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d814b390 con 0x7fb3d8159810 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.892+0000 7fb3dd5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c8cd0 con 0x7fb3d80cd0a0 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.892+0000 7fb3d67fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 0x7fb3d80c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.892+0000 7fb3d67fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 0x7fb3d80c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48570/0 (socket says 172.21.15.131:48570) 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.892+0000 7fb3d67fc700 1 -- 172.21.15.131:0/783577312 learned_addr learned my addr 172.21.15.131:0/783577312 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3d6ffd700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 0x7fb3d80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3d77fe700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d8159810 0x7fb3d814b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3d67fc700 1 -- 172.21.15.131:0/783577312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 msgr2=0x7fb3d80c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3d67fc700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 0x7fb3d80c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3d67fc700 1 -- 172.21.15.131:0/783577312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d8159810 msgr2=0x7fb3d814b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3d67fc700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d8159810 0x7fb3d814b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:25.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3d67fc700 1 -- 172.21.15.131:0/783577312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc009420 con 0x7fb3d80cf390 2023-12-26T21:49:25.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3d77fe700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d8159810 0x7fb3d814b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:25.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3d67fc700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 0x7fb3d80c7750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8088b20 tx=0x7fb3cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:25.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3beffd700 1 -- 172.21.15.131:0/783577312 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc000d10 con 0x7fb3d80cf390 2023-12-26T21:49:25.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3beffd700 1 -- 172.21.15.131:0/783577312 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc011b10 con 0x7fb3d80cf390 2023-12-26T21:49:25.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d80c80c0 con 0x7fb3d80cf390 2023-12-26T21:49:25.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.893+0000 7fb3beffd700 1 -- 172.21.15.131:0/783577312 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc025640 con 0x7fb3d80cf390 2023-12-26T21:49:25.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.894+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d80c8680 con 0x7fb3d80cf390 2023-12-26T21:49:25.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.895+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d80c8e60 con 0x7fb3d80cf390 2023-12-26T21:49:25.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.895+0000 7fb3beffd700 1 -- 172.21.15.131:0/783577312 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fb3cc025a30 con 0x7fb3d80cf390 2023-12-26T21:49:25.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.897+0000 7fb3beffd700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3c4067130 0x7fb3c40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:25.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.897+0000 7fb3beffd700 1 -- 172.21.15.131:0/783577312 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fb3cc086320 con 0x7fb3d80cf390 2023-12-26T21:49:25.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.897+0000 7fb3d6ffd700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3c4067130 0x7fb3c40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:25.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.897+0000 7fb3d6ffd700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3c4067130 0x7fb3c40695e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8001360 tx=0x7fb3c8009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:25.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:25.901+0000 7fb3beffd700 1 -- 172.21.15.131:0/783577312 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3cc053100 con 0x7fb3d80cf390 2023-12-26T21:49:26.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.122+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3d8071860 con 0x7fb3c4067130 2023-12-26T21:49:26.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.195+0000 7fb3beffd700 1 -- 172.21.15.131:0/783577312 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb3d8071860 con 0x7fb3c4067130 2023-12-26T21:49:26.196 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-12-26T21:49:26.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.197+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3c4067130 msgr2=0x7fb3c40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:26.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.197+0000 7fb3dd5ff700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3c4067130 0x7fb3c40695e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8001360 tx=0x7fb3c8009040 comp rx=0 tx=0).stop 2023-12-26T21:49:26.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.197+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 msgr2=0x7fb3d80c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:26.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.197+0000 7fb3dd5ff700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 0x7fb3d80c7750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8088b20 tx=0x7fb3cc017000 comp rx=0 tx=0).stop 2023-12-26T21:49:26.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.198+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 shutdown_connections 2023-12-26T21:49:26.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.198+0000 7fb3dd5ff700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d8159810 0x7fb3d814b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.198+0000 7fb3dd5ff700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3c4067130 0x7fb3c40695e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.198+0000 7fb3dd5ff700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d80cd0a0 0x7fb3d80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.198+0000 7fb3dd5ff700 1 --2- 172.21.15.131:0/783577312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d80cf390 0x7fb3d80c7750 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.198+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 >> 172.21.15.131:0/783577312 conn(0x7fb3d8099950 msgr2=0x7fb3d80c5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.198+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 shutdown_connections 2023-12-26T21:49:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.198+0000 7fb3dd5ff700 1 -- 172.21.15.131:0/783577312 wait complete. 2023-12-26T21:49:26.216 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-12-26T21:49:26.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:26.217 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-12-26T21:49:26.263 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:26.263 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:26.263 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:26.263 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:26.263 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:26.263 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:26.263 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:26.263 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:26.264 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:26.264 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:26.264 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:26.265 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_00_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:26.265 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-12-26T21:49:26.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.458+0000 7fee6a3ff700 1 Processor -- start 2023-12-26T21:49:26.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.458+0000 7fee6a3ff700 1 -- start start 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.459+0000 7fee6a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 0x7fee64069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.459+0000 7fee6a3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee640cc590 0x7fee640c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.459+0000 7fee6a3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee64069640 con 0x7fee64142f00 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.459+0000 7fee6a3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee64068750 con 0x7fee640cc590 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.459+0000 7fee6a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee64068a30 con 0x7fee640cb510 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee63fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fee64142f00 0x7fee640cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45770/0 (socket says 172.21.15.131:45770) 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee63fff700 1 -- 172.21.15.131:0/105598453 learned_addr learned my addr 172.21.15.131:0/105598453 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee6899c700 1 --2- 172.21.15.131:0/105598453 >> v2:172.21.15.131:3300/0 conn(0x7fee640cc590 0x7fee640c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee637fe700 1 --2- 172.21.15.131:0/105598453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 0x7fee64069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee62ffd700 1 -- 172.21.15.131:0/105598453 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315222511 0 0) 0x7fee64069640 con 0x7fee64142f00 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee6899c700 1 -- 172.21.15.131:0/105598453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 msgr2=0x7fee64069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee6899c700 1 --2- 172.21.15.131:0/105598453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 0x7fee64069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee6899c700 1 -- 172.21.15.131:0/105598453 >> v1:172.21.15.131:6789/0 conn(0x7fee64142f00 legacy=0x7fee640cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee6899c700 1 -- 172.21.15.131:0/105598453 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64068d10 con 0x7fee640cc590 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.460+0000 7fee6899c700 1 --2- 172.21.15.131:0/105598453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee640cc590 0x7fee640c8a40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009e60 tx=0x7fee5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f147489638c386fc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee62ffd700 1 -- 172.21.15.131:0/105598453 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00e070 con 0x7fee640cc590 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee62ffd700 1 -- 172.21.15.131:0/105598453 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c00bc30 con 0x7fee640cc590 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee62ffd700 1 -- 172.21.15.131:0/105598453 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c012430 con 0x7fee640cc590 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee6a3ff700 1 -- 172.21.15.131:0/105598453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee640cc590 msgr2=0x7fee640c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee6a3ff700 1 --2- 172.21.15.131:0/105598453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee640cc590 0x7fee640c8a40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009e60 tx=0x7fee5c009780 comp rx=0 tx=0).stop 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee6a3ff700 1 -- 172.21.15.131:0/105598453 shutdown_connections 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee6a3ff700 1 --2- 172.21.15.131:0/105598453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee640cc590 0x7fee640c8a40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee6a3ff700 1 --2- 172.21.15.131:0/105598453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 0x7fee64069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee6a3ff700 1 -- 172.21.15.131:0/105598453 >> 172.21.15.131:0/105598453 conn(0x7fee64063cd0 msgr2=0x7fee6406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee6a3ff700 1 -- 172.21.15.131:0/105598453 shutdown_connections 2023-12-26T21:49:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.461+0000 7fee6a3ff700 1 -- 172.21.15.131:0/105598453 wait complete. 2023-12-26T21:49:26.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.463+0000 7fee6a3ff700 1 Processor -- start 2023-12-26T21:49:26.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.463+0000 7fee6a3ff700 1 -- start start 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 0x7fee640aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6a3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee640cc590 0x7fee640c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6a3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee64142f00 0x7fee640c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6a3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640c5e10 con 0x7fee64142f00 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6a3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640c5550 con 0x7fee640cc590 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640c57c0 con 0x7fee640cb510 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6899c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee64142f00 0x7fee640c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6899c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee64142f00 0x7fee640c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48578/0 (socket says 172.21.15.131:48578) 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6899c700 1 -- 172.21.15.131:0/3481215896 learned_addr learned my addr 172.21.15.131:0/3481215896 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee63fff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 0x7fee640aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee637fe700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee640cc590 0x7fee640c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6899c700 1 -- 172.21.15.131:0/3481215896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 msgr2=0x7fee640aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6899c700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 0x7fee640aa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6899c700 1 -- 172.21.15.131:0/3481215896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee640cc590 msgr2=0x7fee640c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:26.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6899c700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee640cc590 0x7fee640c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.464+0000 7fee6899c700 1 -- 172.21.15.131:0/3481215896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c009a20 con 0x7fee64142f00 2023-12-26T21:49:26.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.465+0000 7fee6899c700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee64142f00 0x7fee640c9730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fee6406f3d0 tx=0x7fee5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:26.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.465+0000 7fee4bfff700 1 -- 172.21.15.131:0/3481215896 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c027470 con 0x7fee64142f00 2023-12-26T21:49:26.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.465+0000 7fee4bfff700 1 -- 172.21.15.131:0/3481215896 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c027630 con 0x7fee64142f00 2023-12-26T21:49:26.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.465+0000 7fee4bfff700 1 -- 172.21.15.131:0/3481215896 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0279f0 con 0x7fee64142f00 2023-12-26T21:49:26.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.465+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee640c5aa0 con 0x7fee64142f00 2023-12-26T21:49:26.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.465+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee640aae70 con 0x7fee64142f00 2023-12-26T21:49:26.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.466+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee640c5fc0 con 0x7fee64142f00 2023-12-26T21:49:26.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.467+0000 7fee4bfff700 1 -- 172.21.15.131:0/3481215896 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fee5c024070 con 0x7fee64142f00 2023-12-26T21:49:26.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.468+0000 7fee4bfff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fee50067130 0x7fee500695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:26.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.468+0000 7fee4bfff700 1 -- 172.21.15.131:0/3481215896 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fee5c087450 con 0x7fee64142f00 2023-12-26T21:49:26.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.468+0000 7fee63fff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fee50067130 0x7fee500695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:26.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.469+0000 7fee63fff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fee50067130 0x7fee500695e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee640ca760 tx=0x7fee54007f50 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:26.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.472+0000 7fee4bfff700 1 -- 172.21.15.131:0/3481215896 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee640c5fc0 con 0x7fee64142f00 2023-12-26T21:49:26.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.688+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee640c8890 con 0x7fee50067130 2023-12-26T21:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.697+0000 7fee4bfff700 1 -- 172.21.15.131:0/3481215896 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fee640c8890 con 0x7fee50067130 2023-12-26T21:49:26.697 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:26.698 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:06", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:26.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.699+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fee50067130 msgr2=0x7fee500695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:26.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.699+0000 7fee6a3ff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fee50067130 0x7fee500695e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee640ca760 tx=0x7fee54007f50 comp rx=0 tx=0).stop 2023-12-26T21:49:26.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.699+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee64142f00 msgr2=0x7fee640c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:26.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.699+0000 7fee6a3ff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee64142f00 0x7fee640c9730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fee6406f3d0 tx=0x7fee5c019000 comp rx=0 tx=0).stop 2023-12-26T21:49:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.699+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 shutdown_connections 2023-12-26T21:49:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.699+0000 7fee6a3ff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee64142f00 0x7fee640c9730 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.699+0000 7fee6a3ff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fee50067130 0x7fee500695e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.699+0000 7fee6a3ff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee640cb510 0x7fee640aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.700+0000 7fee6a3ff700 1 --2- 172.21.15.131:0/3481215896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee640cc590 0x7fee640c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.700+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 >> 172.21.15.131:0/3481215896 conn(0x7fee64063cd0 msgr2=0x7fee6406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.700+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 shutdown_connections 2023-12-26T21:49:26.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.700+0000 7fee6a3ff700 1 -- 172.21.15.131:0/3481215896 wait complete. 2023-12-26T21:49:26.718 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:06", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:26.718 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:06', 'first': '2023-12-26T21:39:00', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:26.718 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:26.718 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:26.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:26.719 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:26.764 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:26.765 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_00_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:26.765 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-12-26T21:49:26.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.999+0000 7fce111ff700 1 Processor -- start 2023-12-26T21:49:26.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:26.999+0000 7fce111ff700 1 -- start start 2023-12-26T21:49:27.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.000+0000 7fce111ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fce0c156730 0x7fce0c0c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.000+0000 7fce111ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fce0c06ee80 0x7fce0c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.000+0000 7fce111ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0c0acfc0 con 0x7fce0c156360 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.000+0000 7fce111ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0c0c91f0 con 0x7fce0c156730 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.000+0000 7fce111ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0c0c94d0 con 0x7fce0c06ee80 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.000+0000 7fce0a59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fce0c156730 0x7fce0c0c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.000+0000 7fce0a59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fce0c156730 0x7fce0c0c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48588/0 (socket says 172.21.15.131:48588) 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.000+0000 7fce0a59c700 1 -- 172.21.15.131:0/3842107711 learned_addr learned my addr 172.21.15.131:0/3842107711 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.001+0000 7fce0b59e700 1 --2- 172.21.15.131:0/3842107711 >> v2:172.21.15.190:3300/0 conn(0x7fce0c06ee80 0x7fce0c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.001+0000 7fce0a59c700 1 -- 172.21.15.131:0/3842107711 >> v2:172.21.15.190:3300/0 conn(0x7fce0c06ee80 msgr2=0x7fce0c0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.001+0000 7fce0a59c700 1 --2- 172.21.15.131:0/3842107711 >> v2:172.21.15.190:3300/0 conn(0x7fce0c06ee80 0x7fce0c0c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.001+0000 7fce0a59c700 1 -- 172.21.15.131:0/3842107711 >> v1:172.21.15.190:6789/0 conn(0x7fce0c156360 legacy=0x7fce0c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.001+0000 7fce0a59c700 1 -- 172.21.15.131:0/3842107711 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c0acc50 con 0x7fce0c156730 2023-12-26T21:49:27.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.001+0000 7fce0a59c700 1 --2- 172.21.15.131:0/3842107711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 0x7fce0c0c8490 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0b22a0 tx=0x7fce0000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce3701fbc223c51 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:27.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.001+0000 7fce09d9b700 1 -- 172.21.15.131:0/3842107711 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0000dc10 con 0x7fce0c156730 2023-12-26T21:49:27.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.001+0000 7fce09d9b700 1 -- 172.21.15.131:0/3842107711 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0000dd70 con 0x7fce0c156730 2023-12-26T21:49:27.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.001+0000 7fce09d9b700 1 -- 172.21.15.131:0/3842107711 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce000106c0 con 0x7fce0c156730 2023-12-26T21:49:27.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.002+0000 7fce111ff700 1 -- 172.21.15.131:0/3842107711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 msgr2=0x7fce0c0c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.002+0000 7fce111ff700 1 --2- 172.21.15.131:0/3842107711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 0x7fce0c0c8490 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0b22a0 tx=0x7fce0000b4f0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.002+0000 7fce111ff700 1 -- 172.21.15.131:0/3842107711 shutdown_connections 2023-12-26T21:49:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.002+0000 7fce111ff700 1 --2- 172.21.15.131:0/3842107711 >> v2:172.21.15.190:3300/0 conn(0x7fce0c06ee80 0x7fce0c0c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.002+0000 7fce111ff700 1 --2- 172.21.15.131:0/3842107711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 0x7fce0c0c8490 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.002+0000 7fce111ff700 1 -- 172.21.15.131:0/3842107711 >> 172.21.15.131:0/3842107711 conn(0x7fce0c0675b0 msgr2=0x7fce0c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.002+0000 7fce111ff700 1 -- 172.21.15.131:0/3842107711 shutdown_connections 2023-12-26T21:49:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.002+0000 7fce111ff700 1 -- 172.21.15.131:0/3842107711 wait complete. 2023-12-26T21:49:27.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.003+0000 7fce111ff700 1 Processor -- start 2023-12-26T21:49:27.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.004+0000 7fce111ff700 1 -- start start 2023-12-26T21:49:27.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.004+0000 7fce111ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce0c06ee80 0x7fce0c16c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.004+0000 7fce111ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce0c156360 0x7fce0c16a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.004+0000 7fce111ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 0x7fce0c1507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.004+0000 7fce111ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0b0180 con 0x7fce0c156730 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce111ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0b0fc0 con 0x7fce0c06ee80 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce111ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0b03b0 con 0x7fce0c156360 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0b59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 0x7fce0c1507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0b59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 0x7fce0c1507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48602/0 (socket says 172.21.15.131:48602) 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0b59e700 1 -- 172.21.15.131:0/1741421855 learned_addr learned my addr 172.21.15.131:0/1741421855 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0ad9d700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce0c06ee80 0x7fce0c16c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0a59c700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce0c156360 0x7fce0c16a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0b59e700 1 -- 172.21.15.131:0/1741421855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce0c156360 msgr2=0x7fce0c16a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0b59e700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce0c156360 0x7fce0c16a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0b59e700 1 -- 172.21.15.131:0/1741421855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce0c06ee80 msgr2=0x7fce0c16c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0b59e700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce0c06ee80 0x7fce0c16c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0b59e700 1 -- 172.21.15.131:0/1741421855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0000b790 con 0x7fce0c156730 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0a59c700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce0c156360 0x7fce0c16a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:27.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.005+0000 7fce0b59e700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 0x7fce0c1507c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce04009fa0 tx=0x7fce04009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:27.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.006+0000 7fcdf2ffd700 1 -- 172.21.15.131:0/1741421855 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0400ec70 con 0x7fce0c156730 2023-12-26T21:49:27.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.006+0000 7fcdf2ffd700 1 -- 172.21.15.131:0/1741421855 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0400edd0 con 0x7fce0c156730 2023-12-26T21:49:27.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.006+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c0b0690 con 0x7fce0c156730 2023-12-26T21:49:27.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.006+0000 7fcdf2ffd700 1 -- 172.21.15.131:0/1741421855 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04014670 con 0x7fce0c156730 2023-12-26T21:49:27.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.006+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c0b0c50 con 0x7fce0c156730 2023-12-26T21:49:27.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.007+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c0b1130 con 0x7fce0c156730 2023-12-26T21:49:27.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.008+0000 7fcdf2ffd700 1 -- 172.21.15.131:0/1741421855 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fce040149e0 con 0x7fce0c156730 2023-12-26T21:49:27.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.008+0000 7fcdf2ffd700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdf8067130 0x7fcdf80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.008+0000 7fcdf2ffd700 1 -- 172.21.15.131:0/1741421855 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fce04085170 con 0x7fce0c156730 2023-12-26T21:49:27.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.009+0000 7fce0ad9d700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdf8067130 0x7fcdf80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.009+0000 7fce0ad9d700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdf8067130 0x7fcdf80695e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc003a30 tx=0x7fcdfc007ea0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:27.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.012+0000 7fcdf2ffd700 1 -- 172.21.15.131:0/1741421855 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce04051f50 con 0x7fce0c156730 2023-12-26T21:49:27.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.228+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce0c0717f0 con 0x7fcdf8067130 2023-12-26T21:49:27.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.236+0000 7fcdf2ffd700 1 -- 172.21.15.131:0/1741421855 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fce0c0717f0 con 0x7fcdf8067130 2023-12-26T21:49:27.237 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:27.237 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:06", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:27.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdf8067130 msgr2=0x7fcdf80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdf8067130 0x7fcdf80695e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc003a30 tx=0x7fcdfc007ea0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 msgr2=0x7fce0c1507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 0x7fce0c1507c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce04009fa0 tx=0x7fce04009900 comp rx=0 tx=0).stop 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 shutdown_connections 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce0c156730 0x7fce0c1507c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce0c06ee80 0x7fce0c16c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdf8067130 0x7fcdf80695e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 --2- 172.21.15.131:0/1741421855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce0c156360 0x7fce0c16a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 >> 172.21.15.131:0/1741421855 conn(0x7fce0c0675b0 msgr2=0x7fce0c0c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 shutdown_connections 2023-12-26T21:49:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.239+0000 7fce111ff700 1 -- 172.21.15.131:0/1741421855 wait complete. 2023-12-26T21:49:27.257 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:06", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:27.257 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:06', 'first': '2023-12-26T21:39:00', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:27.257 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:27.257 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:27.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:27.258 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-12-26T21:49:27.304 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:27.304 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:27.304 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:27.304 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:27.305 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:27.305 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:27.305 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:27.305 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:27.305 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:27.305 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:27.305 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:27.306 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_00_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:27.306 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-12-26T21:49:27.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.539+0000 7fa9f61ff700 1 Processor -- start 2023-12-26T21:49:27.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.539+0000 7fa9f61ff700 1 -- start start 2023-12-26T21:49:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.541+0000 7fa9f61ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa9f00cb510 0x7fa9f0069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.541+0000 7fa9f61ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f0069640 con 0x7fa9f0142f00 2023-12-26T21:49:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.541+0000 7fa9f61ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f0068750 con 0x7fa9f00cc590 2023-12-26T21:49:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.541+0000 7fa9f61ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f0068a30 con 0x7fa9f00cb510 2023-12-26T21:49:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.541+0000 7fa9ef7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa9f0142f00 0x7fa9f00cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45784/0 (socket says 172.21.15.131:45784) 2023-12-26T21:49:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.541+0000 7fa9ef7fe700 1 -- 172.21.15.131:0/116625157 learned_addr learned my addr 172.21.15.131:0/116625157 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.541+0000 7fa9eeffd700 1 --2- 172.21.15.131:0/116625157 >> v2:172.21.15.190:3300/0 conn(0x7fa9f00cb510 0x7fa9f0069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.541+0000 7fa9ee7fc700 1 -- 172.21.15.131:0/116625157 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962022055 0 0) 0x7fa9f0069640 con 0x7fa9f0142f00 2023-12-26T21:49:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.542+0000 7fa9ee7fc700 1 -- 172.21.15.131:0/116625157 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d8003570 con 0x7fa9f0142f00 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.542+0000 7fa9ee7fc700 1 -- 172.21.15.131:0/116625157 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1899064857 0 0) 0x7fa9d8003570 con 0x7fa9f0142f00 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.542+0000 7fa9eeffd700 1 -- 172.21.15.131:0/116625157 >> v1:172.21.15.190:6789/0 conn(0x7fa9f00cc590 legacy=0x7fa9f00c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.542+0000 7fa9eeffd700 1 -- 172.21.15.131:0/116625157 >> v1:172.21.15.131:6789/0 conn(0x7fa9f0142f00 legacy=0x7fa9f00cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.542+0000 7fa9eeffd700 1 -- 172.21.15.131:0/116625157 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0068d10 con 0x7fa9f00cb510 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.542+0000 7fa9eeffd700 1 --2- 172.21.15.131:0/116625157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f00cb510 0x7fa9f0069ad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40098e0 tx=0x7fa9e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3271541392c8ff2d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.542+0000 7fa9ee7fc700 1 -- 172.21.15.131:0/116625157 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0001470 con 0x7fa9f00cb510 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.542+0000 7fa9ee7fc700 1 -- 172.21.15.131:0/116625157 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e4011420 con 0x7fa9f00cb510 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.542+0000 7fa9ee7fc700 1 -- 172.21.15.131:0/116625157 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40117c0 con 0x7fa9f00cb510 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.543+0000 7fa9f61ff700 1 -- 172.21.15.131:0/116625157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f00cb510 msgr2=0x7fa9f0069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.543+0000 7fa9f61ff700 1 --2- 172.21.15.131:0/116625157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f00cb510 0x7fa9f0069ad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40098e0 tx=0x7fa9e40091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.543+0000 7fa9f61ff700 1 -- 172.21.15.131:0/116625157 shutdown_connections 2023-12-26T21:49:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.543+0000 7fa9f61ff700 1 --2- 172.21.15.131:0/116625157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f00cb510 0x7fa9f0069ad0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.543+0000 7fa9f61ff700 1 -- 172.21.15.131:0/116625157 >> 172.21.15.131:0/116625157 conn(0x7fa9f0063cd0 msgr2=0x7fa9f006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:27.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.543+0000 7fa9f61ff700 1 -- 172.21.15.131:0/116625157 shutdown_connections 2023-12-26T21:49:27.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.543+0000 7fa9f61ff700 1 -- 172.21.15.131:0/116625157 wait complete. 2023-12-26T21:49:27.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.544+0000 7fa9f61ff700 1 Processor -- start 2023-12-26T21:49:27.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.545+0000 7fa9f61ff700 1 -- start start 2023-12-26T21:49:27.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.545+0000 7fa9f61ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00cb510 0x7fa9f00b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.545+0000 7fa9f61ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f00cc590 0x7fa9f015e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.545+0000 7fa9f61ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f0142f00 0x7fa9f015f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.545+0000 7fa9f61ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f015d840 con 0x7fa9f00cc590 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.545+0000 7fa9f61ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f015db20 con 0x7fa9f0142f00 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.545+0000 7fa9f61ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f015de00 con 0x7fa9f00cb510 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9eeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f00cc590 0x7fa9f015e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9eeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f00cc590 0x7fa9f015e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48604/0 (socket says 172.21.15.131:48604) 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9eeffd700 1 -- 172.21.15.131:0/3811736474 learned_addr learned my addr 172.21.15.131:0/3811736474 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9effff700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f0142f00 0x7fa9f015f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9ef7fe700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00cb510 0x7fa9f00b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9eeffd700 1 -- 172.21.15.131:0/3811736474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00cb510 msgr2=0x7fa9f00b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9eeffd700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00cb510 0x7fa9f00b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9eeffd700 1 -- 172.21.15.131:0/3811736474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f0142f00 msgr2=0x7fa9f015f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9eeffd700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f0142f00 0x7fa9f015f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9eeffd700 1 -- 172.21.15.131:0/3811736474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e4009420 con 0x7fa9f00cc590 2023-12-26T21:49:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9effff700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f0142f00 0x7fa9f015f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.546+0000 7fa9eeffd700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f00cc590 0x7fa9f015e8e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40091e0 tx=0x7fa9e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.547+0000 7fa9d77fe700 1 -- 172.21.15.131:0/3811736474 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e401cc90 con 0x7fa9f00cc590 2023-12-26T21:49:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.547+0000 7fa9d77fe700 1 -- 172.21.15.131:0/3811736474 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e401cdf0 con 0x7fa9f00cc590 2023-12-26T21:49:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.547+0000 7fa9d77fe700 1 -- 172.21.15.131:0/3811736474 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e401a6a0 con 0x7fa9f00cc590 2023-12-26T21:49:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.547+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f015e0e0 con 0x7fa9f00cc590 2023-12-26T21:49:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.547+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f00b3c00 con 0x7fa9f00cc590 2023-12-26T21:49:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.548+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f015e580 con 0x7fa9f00cc590 2023-12-26T21:49:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.549+0000 7fa9d77fe700 1 -- 172.21.15.131:0/3811736474 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fa9e401aa50 con 0x7fa9f00cc590 2023-12-26T21:49:27.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.550+0000 7fa9d77fe700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa9dc067130 0x7fa9dc0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:27.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.550+0000 7fa9d77fe700 1 -- 172.21.15.131:0/3811736474 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fa9e4086200 con 0x7fa9f00cc590 2023-12-26T21:49:27.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.551+0000 7fa9ef7fe700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa9dc067130 0x7fa9dc0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:27.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.551+0000 7fa9ef7fe700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa9dc067130 0x7fa9dc0695e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00011e0 tx=0x7fa9e0000bc0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:27.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.554+0000 7fa9d77fe700 1 -- 172.21.15.131:0/3811736474 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9e4052fe0 con 0x7fa9f00cc590 2023-12-26T21:49:27.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.772+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9f00717f0 con 0x7fa9dc067130 2023-12-26T21:49:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.782+0000 7fa9d77fe700 1 -- 172.21.15.131:0/3811736474 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fa9f00717f0 con 0x7fa9dc067130 2023-12-26T21:49:27.782 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:27.782 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:07", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:27.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa9dc067130 msgr2=0x7fa9dc0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa9dc067130 0x7fa9dc0695e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00011e0 tx=0x7fa9e0000bc0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f00cc590 msgr2=0x7fa9f015e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f00cc590 0x7fa9f015e8e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40091e0 tx=0x7fa9e4016000 comp rx=0 tx=0).stop 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 shutdown_connections 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f0142f00 0x7fa9f015f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa9dc067130 0x7fa9dc0695e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00cb510 0x7fa9f00b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 --2- 172.21.15.131:0/3811736474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9f00cc590 0x7fa9f015e8e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 >> 172.21.15.131:0/3811736474 conn(0x7fa9f0063cd0 msgr2=0x7fa9f006a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 shutdown_connections 2023-12-26T21:49:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:27.784+0000 7fa9f61ff700 1 -- 172.21.15.131:0/3811736474 wait complete. 2023-12-26T21:49:27.802 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:07", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:27.802 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:07', 'first': '2023-12-26T21:39:00', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:27.802 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:27.803 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:27.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:27.803 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:27.848 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:27.849 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_00_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:27.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-12-26T21:49:28.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.083+0000 7f17b85ff700 1 Processor -- start 2023-12-26T21:49:28.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.084+0000 7f17b85ff700 1 -- start start 2023-12-26T21:49:28.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.085+0000 7f17b85ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17b01593f0 0x7f17b00abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.085+0000 7f17b85ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b0067ff0 con 0x7f17b0069e60 2023-12-26T21:49:28.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.085+0000 7f17b85ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b00ac350 con 0x7f17b01597c0 2023-12-26T21:49:28.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.085+0000 7f17b85ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b0068480 con 0x7f17b01593f0 2023-12-26T21:49:28.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.085+0000 7f17b6b9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f17b0069e60 0x7f17b006a290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45800/0 (socket says 172.21.15.131:45800) 2023-12-26T21:49:28.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.085+0000 7f17b6b9c700 1 -- 172.21.15.131:0/805032942 learned_addr learned my addr 172.21.15.131:0/805032942 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:28.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.085+0000 7f17b639b700 1 --2- 172.21.15.131:0/805032942 >> v2:172.21.15.190:3300/0 conn(0x7f17b01593f0 0x7f17b00abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.086+0000 7f17b5399700 1 -- 172.21.15.131:0/805032942 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918752974 0 0) 0x7f17b0067ff0 con 0x7f17b0069e60 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.086+0000 7f17b5399700 1 -- 172.21.15.131:0/805032942 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f179c003570 con 0x7f17b0069e60 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.086+0000 7f17b639b700 1 -- 172.21.15.131:0/805032942 >> v1:172.21.15.190:6789/0 conn(0x7f17b01597c0 legacy=0x7f17b00aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.086+0000 7f17b639b700 1 -- 172.21.15.131:0/805032942 >> v1:172.21.15.131:6789/0 conn(0x7f17b0069e60 legacy=0x7f17b006a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.086+0000 7f17b639b700 1 -- 172.21.15.131:0/805032942 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00af8c0 con 0x7f17b01593f0 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.086+0000 7f17b639b700 1 --2- 172.21.15.131:0/805032942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b01593f0 0x7f17b00abc90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0098e0 tx=0x7f17ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb7f24a4e2b8b84 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.086+0000 7f17b5399700 1 -- 172.21.15.131:0/805032942 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac00b640 con 0x7f17b01593f0 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.086+0000 7f17b5399700 1 -- 172.21.15.131:0/805032942 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac00b7a0 con 0x7f17b01593f0 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.086+0000 7f17b5399700 1 -- 172.21.15.131:0/805032942 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac00bb40 con 0x7f17b01593f0 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.087+0000 7f17b85ff700 1 -- 172.21.15.131:0/805032942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b01593f0 msgr2=0x7f17b00abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.087+0000 7f17b85ff700 1 --2- 172.21.15.131:0/805032942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b01593f0 0x7f17b00abc90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0098e0 tx=0x7f17ac0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.087+0000 7f17b85ff700 1 -- 172.21.15.131:0/805032942 shutdown_connections 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.087+0000 7f17b85ff700 1 --2- 172.21.15.131:0/805032942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b01593f0 0x7f17b00abc90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.087+0000 7f17b85ff700 1 -- 172.21.15.131:0/805032942 >> 172.21.15.131:0/805032942 conn(0x7f17b006b710 msgr2=0x7f17b006bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:28.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.087+0000 7f17b85ff700 1 -- 172.21.15.131:0/805032942 shutdown_connections 2023-12-26T21:49:28.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.087+0000 7f17b85ff700 1 -- 172.21.15.131:0/805032942 wait complete. 2023-12-26T21:49:28.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.088+0000 7f17b85ff700 1 Processor -- start 2023-12-26T21:49:28.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.089+0000 7f17b85ff700 1 -- start start 2023-12-26T21:49:28.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.089+0000 7f17b85ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0069e60 0x7f17b015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.089+0000 7f17b85ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b01593f0 0x7f17b01444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.089+0000 7f17b85ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b01597c0 0x7f17b01699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.089+0000 7f17b85ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00adef0 con 0x7f17b01593f0 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.089+0000 7f17b85ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00ae720 con 0x7f17b0069e60 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.089+0000 7f17b85ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00c1c00 con 0x7f17b01597c0 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b5b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b01593f0 0x7f17b01444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b5b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b01593f0 0x7f17b01444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48612/0 (socket says 172.21.15.131:48612) 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b5b9a700 1 -- 172.21.15.131:0/4212229637 learned_addr learned my addr 172.21.15.131:0/4212229637 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b639b700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0069e60 0x7f17b015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b6b9c700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b01597c0 0x7f17b01699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b5b9a700 1 -- 172.21.15.131:0/4212229637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b01597c0 msgr2=0x7f17b01699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b5b9a700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b01597c0 0x7f17b01699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b5b9a700 1 -- 172.21.15.131:0/4212229637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0069e60 msgr2=0x7f17b015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b5b9a700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0069e60 0x7f17b015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b5b9a700 1 -- 172.21.15.131:0/4212229637 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac009420 con 0x7f17b01593f0 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b6b9c700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b01597c0 0x7f17b01699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.090+0000 7f17b5b9a700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b01593f0 0x7f17b01444b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c4e0 tx=0x7f17a000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:28.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.091+0000 7f17a67fc700 1 -- 172.21.15.131:0/4212229637 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011590 con 0x7f17b01593f0 2023-12-26T21:49:28.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.091+0000 7f17a67fc700 1 -- 172.21.15.131:0/4212229637 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a0011710 con 0x7f17b01593f0 2023-12-26T21:49:28.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.091+0000 7f17a67fc700 1 -- 172.21.15.131:0/4212229637 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011ad0 con 0x7f17b01593f0 2023-12-26T21:49:28.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.091+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b00ae0d0 con 0x7f17b01593f0 2023-12-26T21:49:28.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.091+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b01449f0 con 0x7f17b01593f0 2023-12-26T21:49:28.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.092+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b00ae8b0 con 0x7f17b01593f0 2023-12-26T21:49:28.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.093+0000 7f17a67fc700 1 -- 172.21.15.131:0/4212229637 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f17a001b3b0 con 0x7f17b01593f0 2023-12-26T21:49:28.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.094+0000 7f17a67fc700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1794067130 0x7f17940695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.094+0000 7f17a67fc700 1 -- 172.21.15.131:0/4212229637 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f17a0084090 con 0x7f17b01593f0 2023-12-26T21:49:28.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.094+0000 7f17b639b700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1794067130 0x7f17940695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.095+0000 7f17b639b700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1794067130 0x7f17940695e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0093f0 tx=0x7f17ac017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:28.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.098+0000 7f17a67fc700 1 -- 172.21.15.131:0/4212229637 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17a0050e70 con 0x7f17b01593f0 2023-12-26T21:49:28.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.318+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f17b00717f0 con 0x7f1794067130 2023-12-26T21:49:28.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.329+0000 7f17a67fc700 1 -- 172.21.15.131:0/4212229637 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f17b00717f0 con 0x7f1794067130 2023-12-26T21:49:28.329 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:28.329 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:08", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:28.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1794067130 msgr2=0x7f17940695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1794067130 0x7f17940695e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0093f0 tx=0x7f17ac017000 comp rx=0 tx=0).stop 2023-12-26T21:49:28.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b01593f0 msgr2=0x7f17b01444b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b01593f0 0x7f17b01444b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c4e0 tx=0x7f17a000c8a0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 shutdown_connections 2023-12-26T21:49:28.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1794067130 0x7f17940695e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b01597c0 0x7f17b01699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0069e60 0x7f17b015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 --2- 172.21.15.131:0/4212229637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b01593f0 0x7f17b01444b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.330+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 >> 172.21.15.131:0/4212229637 conn(0x7f17b006b710 msgr2=0x7f17b00b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:28.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.331+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 shutdown_connections 2023-12-26T21:49:28.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.331+0000 7f17b85ff700 1 -- 172.21.15.131:0/4212229637 wait complete. 2023-12-26T21:49:28.349 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:08", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:28.349 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:08', 'first': '2023-12-26T21:39:00', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:28.349 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:28.349 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:28.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:28.349 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-12-26T21:49:28.394 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:28.395 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:28.396 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_00_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:28.396 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-12-26T21:49:28.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.628+0000 7f7135fff700 1 Processor -- start 2023-12-26T21:49:28.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.629+0000 7f7135fff700 1 -- start start 2023-12-26T21:49:28.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f7135fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71300c8d40 0x7f713015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f7135fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7130156e50 0x7f71301572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f7135fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71300af040 con 0x7f71300c9110 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f7135fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71300ae0e0 con 0x7f71300c8d40 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f7135fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71300ae3c0 con 0x7f7130156e50 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f712f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71300c8d40 0x7f713015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f712f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71300c8d40 0x7f713015bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48620/0 (socket says 172.21.15.131:48620) 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f712f7fe700 1 -- 172.21.15.131:0/1606473309 learned_addr learned my addr 172.21.15.131:0/1606473309 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f712effd700 1 --1- 172.21.15.131:0/1606473309 >> v1:172.21.15.131:6789/0 conn(0x7f71300c9110 0x7f71300ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45804/0 (socket says 172.21.15.131:45804) 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f712ffff700 1 --2- 172.21.15.131:0/1606473309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7130156e50 0x7f71301572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.630+0000 7f712f7fe700 1 -- 172.21.15.131:0/1606473309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7130156e50 msgr2=0x7f71301572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.631+0000 7f712f7fe700 1 --2- 172.21.15.131:0/1606473309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7130156e50 0x7f71301572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.631+0000 7f712f7fe700 1 -- 172.21.15.131:0/1606473309 >> v1:172.21.15.131:6789/0 conn(0x7f71300c9110 legacy=0x7f71300ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.631+0000 7f712f7fe700 1 -- 172.21.15.131:0/1606473309 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300ae6a0 con 0x7f71300c8d40 2023-12-26T21:49:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.631+0000 7f712e7fc700 1 -- 172.21.15.131:0/1606473309 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504219719 0 0) 0x7f71300af040 con 0x7f71300c9110 2023-12-26T21:49:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.631+0000 7f712f7fe700 1 --2- 172.21.15.131:0/1606473309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71300c8d40 0x7f713015bc30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f71200099b0 tx=0x7f7120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8021c738d2b7cece server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.631+0000 7f712e7fc700 1 -- 172.21.15.131:0/1606473309 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000b860 con 0x7f71300c8d40 2023-12-26T21:49:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.631+0000 7f712e7fc700 1 -- 172.21.15.131:0/1606473309 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712000b9c0 con 0x7f71300c8d40 2023-12-26T21:49:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.631+0000 7f712e7fc700 1 -- 172.21.15.131:0/1606473309 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000bd60 con 0x7f71300c8d40 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.632+0000 7f7135fff700 1 -- 172.21.15.131:0/1606473309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71300c8d40 msgr2=0x7f713015bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.632+0000 7f7135fff700 1 --2- 172.21.15.131:0/1606473309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71300c8d40 0x7f713015bc30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f71200099b0 tx=0x7f7120009250 comp rx=0 tx=0).stop 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.632+0000 7f7135fff700 1 -- 172.21.15.131:0/1606473309 shutdown_connections 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.632+0000 7f7135fff700 1 --2- 172.21.15.131:0/1606473309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7130156e50 0x7f71301572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.632+0000 7f7135fff700 1 --2- 172.21.15.131:0/1606473309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71300c8d40 0x7f713015bc30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.632+0000 7f7135fff700 1 -- 172.21.15.131:0/1606473309 >> 172.21.15.131:0/1606473309 conn(0x7f71300675b0 msgr2=0x7f71300679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.632+0000 7f7135fff700 1 -- 172.21.15.131:0/1606473309 shutdown_connections 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.632+0000 7f7135fff700 1 -- 172.21.15.131:0/1606473309 wait complete. 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.633+0000 7f7135fff700 1 Processor -- start 2023-12-26T21:49:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.633+0000 7f7135fff700 1 -- start start 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f7135fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71300c8d40 0x7f71300c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f7135fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300c9110 0x7f7130169d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f7135fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7130156e50 0x7f713006e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f7135fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713016a260 con 0x7f7130156e50 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f7135fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713016ad30 con 0x7f71300c8d40 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f7135fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713016d550 con 0x7f71300c9110 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f712ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7130156e50 0x7f713006e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f712ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7130156e50 0x7f713006e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48628/0 (socket says 172.21.15.131:48628) 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f712ffff700 1 -- 172.21.15.131:0/3722567409 learned_addr learned my addr 172.21.15.131:0/3722567409 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.634+0000 7f712f7fe700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71300c8d40 0x7f71300c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f712effd700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300c9110 0x7f7130169d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f712ffff700 1 -- 172.21.15.131:0/3722567409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300c9110 msgr2=0x7f7130169d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f712ffff700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300c9110 0x7f7130169d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f712ffff700 1 -- 172.21.15.131:0/3722567409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71300c8d40 msgr2=0x7f71300c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f712ffff700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71300c8d40 0x7f71300c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f712ffff700 1 -- 172.21.15.131:0/3722567409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71200094f0 con 0x7f7130156e50 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f712ffff700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7130156e50 0x7f713006e690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7128007f20 tx=0x7f7128009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f71177fe700 1 -- 172.21.15.131:0/3722567409 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712800eb90 con 0x7f7130156e50 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f71177fe700 1 -- 172.21.15.131:0/3722567409 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712800ed10 con 0x7f7130156e50 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713016a490 con 0x7f7130156e50 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f71177fe700 1 -- 172.21.15.131:0/3722567409 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128014670 con 0x7f7130156e50 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.635+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713016aa50 con 0x7f7130156e50 2023-12-26T21:49:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.636+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71300717f0 con 0x7f7130156e50 2023-12-26T21:49:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.637+0000 7f71177fe700 1 -- 172.21.15.131:0/3722567409 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f71280149e0 con 0x7f7130156e50 2023-12-26T21:49:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.638+0000 7f71177fe700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f711c067130 0x7f711c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.638+0000 7f71177fe700 1 -- 172.21.15.131:0/3722567409 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f71280850d0 con 0x7f7130156e50 2023-12-26T21:49:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.638+0000 7f712f7fe700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f711c067130 0x7f711c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.639+0000 7f712f7fe700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f711c067130 0x7f711c0695e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71200099b0 tx=0x7f7120017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:28.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.642+0000 7f71177fe700 1 -- 172.21.15.131:0/3722567409 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7128051eb0 con 0x7f7130156e50 2023-12-26T21:49:28.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.857+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71300ae980 con 0x7f711c067130 2023-12-26T21:49:28.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.866+0000 7f71177fe700 1 -- 172.21.15.131:0/3722567409 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f71300ae980 con 0x7f711c067130 2023-12-26T21:49:28.867 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:28.867 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:08", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f711c067130 msgr2=0x7f711c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f711c067130 0x7f711c0695e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71200099b0 tx=0x7f7120017000 comp rx=0 tx=0).stop 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7130156e50 msgr2=0x7f713006e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7130156e50 0x7f713006e690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7128007f20 tx=0x7f7128009860 comp rx=0 tx=0).stop 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 shutdown_connections 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7130156e50 0x7f713006e690 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71300c8d40 0x7f71300c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f711c067130 0x7f711c0695e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 --2- 172.21.15.131:0/3722567409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300c9110 0x7f7130169d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 >> 172.21.15.131:0/3722567409 conn(0x7f71300675b0 msgr2=0x7f7130156110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:28.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 shutdown_connections 2023-12-26T21:49:28.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:28.869+0000 7f7135fff700 1 -- 172.21.15.131:0/3722567409 wait complete. 2023-12-26T21:49:28.887 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:08", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:28.887 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:08', 'first': '2023-12-26T21:39:00', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:28.887 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:28.887 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:28.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:28.888 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:28.933 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:28.934 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_00_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:28.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-12-26T21:49:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.167+0000 7f9f27bff700 1 Processor -- start 2023-12-26T21:49:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.168+0000 7f9f27bff700 1 -- start start 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f27bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f200cf390 0x7f9f200c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f27bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f200cd0a0 0x7f9f200cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f27bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f200cbb00 con 0x7f9f20159810 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f27bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f200cb2a0 con 0x7f9f200cd0a0 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f27bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f200ce040 con 0x7f9f200cf390 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f2599b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9f20159810 0x7f9f200cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45808/0 (socket says 172.21.15.131:45808) 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f2599b700 1 -- 172.21.15.131:0/2276858969 learned_addr learned my addr 172.21.15.131:0/2276858969 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f2619c700 1 --2- 172.21.15.131:0/2276858969 >> v2:172.21.15.131:3300/0 conn(0x7f9f200cd0a0 0x7f9f200cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f2519a700 1 --2- 172.21.15.131:0/2276858969 >> v2:172.21.15.190:3300/0 conn(0x7f9f200cf390 0x7f9f200c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.169+0000 7f9f24999700 1 -- 172.21.15.131:0/2276858969 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516158881 0 0) 0x7f9f200cbb00 con 0x7f9f20159810 2023-12-26T21:49:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f24999700 1 -- 172.21.15.131:0/2276858969 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f0c003570 con 0x7f9f20159810 2023-12-26T21:49:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f2619c700 1 -- 172.21.15.131:0/2276858969 >> v2:172.21.15.190:3300/0 conn(0x7f9f200cf390 msgr2=0x7f9f200c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f2619c700 1 --2- 172.21.15.131:0/2276858969 >> v2:172.21.15.190:3300/0 conn(0x7f9f200cf390 0x7f9f200c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f2619c700 1 -- 172.21.15.131:0/2276858969 >> v1:172.21.15.131:6789/0 conn(0x7f9f20159810 legacy=0x7f9f200cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f2619c700 1 -- 172.21.15.131:0/2276858969 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f20159be0 con 0x7f9f200cd0a0 2023-12-26T21:49:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f2519a700 1 --2- 172.21.15.131:0/2276858969 >> v2:172.21.15.190:3300/0 conn(0x7f9f200cf390 0x7f9f200c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f2619c700 1 --2- 172.21.15.131:0/2276858969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f200cd0a0 0x7f9f200cd470 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9f20089e20 tx=0x7f9f18009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f81eaea96c7e06 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f24999700 1 -- 172.21.15.131:0/2276858969 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800e070 con 0x7f9f200cd0a0 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f24999700 1 -- 172.21.15.131:0/2276858969 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1800bb60 con 0x7f9f200cd0a0 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.170+0000 7f9f24999700 1 -- 172.21.15.131:0/2276858969 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18012430 con 0x7f9f200cd0a0 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.171+0000 7f9f27bff700 1 -- 172.21.15.131:0/2276858969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f200cd0a0 msgr2=0x7f9f200cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.171+0000 7f9f27bff700 1 --2- 172.21.15.131:0/2276858969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f200cd0a0 0x7f9f200cd470 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9f20089e20 tx=0x7f9f18009780 comp rx=0 tx=0).stop 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.171+0000 7f9f27bff700 1 -- 172.21.15.131:0/2276858969 shutdown_connections 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.171+0000 7f9f27bff700 1 --2- 172.21.15.131:0/2276858969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f200cd0a0 0x7f9f200cd470 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.171+0000 7f9f27bff700 1 --2- 172.21.15.131:0/2276858969 >> v2:172.21.15.190:3300/0 conn(0x7f9f200cf390 0x7f9f200c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.171+0000 7f9f27bff700 1 -- 172.21.15.131:0/2276858969 >> 172.21.15.131:0/2276858969 conn(0x7f9f20099950 msgr2=0x7f9f2006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.171+0000 7f9f27bff700 1 -- 172.21.15.131:0/2276858969 shutdown_connections 2023-12-26T21:49:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.171+0000 7f9f27bff700 1 -- 172.21.15.131:0/2276858969 wait complete. 2023-12-26T21:49:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.172+0000 7f9f27bff700 1 Processor -- start 2023-12-26T21:49:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.172+0000 7f9f27bff700 1 -- start start 2023-12-26T21:49:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.173+0000 7f9f27bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f200cd0a0 0x7f9f20151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.173+0000 7f9f27bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f200cf390 0x7f9f200b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.173+0000 7f9f27bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f20159810 0x7f9f200b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.173+0000 7f9f27bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2015bcd0 con 0x7f9f20159810 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.173+0000 7f9f27bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2015c850 con 0x7f9f200cd0a0 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.173+0000 7f9f27bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2015bf20 con 0x7f9f200cf390 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.173+0000 7f9f2619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f20159810 0x7f9f200b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2599b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f200cd0a0 0x7f9f20151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f20159810 0x7f9f200b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48648/0 (socket says 172.21.15.131:48648) 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2619c700 1 -- 172.21.15.131:0/4079663345 learned_addr learned my addr 172.21.15.131:0/4079663345 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2519a700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f200cf390 0x7f9f200b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2619c700 1 -- 172.21.15.131:0/4079663345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f200cf390 msgr2=0x7f9f200b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2619c700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f200cf390 0x7f9f200b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2619c700 1 -- 172.21.15.131:0/4079663345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f200cd0a0 msgr2=0x7f9f20151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2619c700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f200cd0a0 0x7f9f20151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2619c700 1 -- 172.21.15.131:0/4079663345 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f18009a20 con 0x7f9f20159810 2023-12-26T21:49:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.174+0000 7f9f2619c700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f20159810 0x7f9f200b44e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9f20088b20 tx=0x7f9f18019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.175+0000 7f9f15ffb700 1 -- 172.21.15.131:0/4079663345 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18027470 con 0x7f9f20159810 2023-12-26T21:49:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.175+0000 7f9f15ffb700 1 -- 172.21.15.131:0/4079663345 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f180275f0 con 0x7f9f20159810 2023-12-26T21:49:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.175+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2015c200 con 0x7f9f20159810 2023-12-26T21:49:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.175+0000 7f9f15ffb700 1 -- 172.21.15.131:0/4079663345 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180279b0 con 0x7f9f20159810 2023-12-26T21:49:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.175+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f201519a0 con 0x7f9f20159810 2023-12-26T21:49:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.176+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2015cce0 con 0x7f9f20159810 2023-12-26T21:49:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.177+0000 7f9f15ffb700 1 -- 172.21.15.131:0/4079663345 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f9f18024070 con 0x7f9f20159810 2023-12-26T21:49:29.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.178+0000 7f9f15ffb700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9f04067130 0x7f9f040695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.178+0000 7f9f15ffb700 1 -- 172.21.15.131:0/4079663345 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f9f18087380 con 0x7f9f20159810 2023-12-26T21:49:29.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.178+0000 7f9f2599b700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9f04067130 0x7f9f040695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.178+0000 7f9f2599b700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9f04067130 0x7f9f040695e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c001360 tx=0x7f9f1c009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:29.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.182+0000 7f9f15ffb700 1 -- 172.21.15.131:0/4079663345 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f18054160 con 0x7f9f20159810 2023-12-26T21:49:29.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.404+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f200717f0 con 0x7f9f04067130 2023-12-26T21:49:29.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.414+0000 7f9f15ffb700 1 -- 172.21.15.131:0/4079663345 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f9f200717f0 con 0x7f9f04067130 2023-12-26T21:49:29.415 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:29.415 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:09", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9f04067130 msgr2=0x7f9f040695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9f04067130 0x7f9f040695e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c001360 tx=0x7f9f1c009040 comp rx=0 tx=0).stop 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f20159810 msgr2=0x7f9f200b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f20159810 0x7f9f200b44e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9f20088b20 tx=0x7f9f18019000 comp rx=0 tx=0).stop 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 shutdown_connections 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f20159810 0x7f9f200b44e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f200cd0a0 0x7f9f20151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9f04067130 0x7f9f040695e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 --2- 172.21.15.131:0/4079663345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f200cf390 0x7f9f200b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 >> 172.21.15.131:0/4079663345 conn(0x7f9f20099950 msgr2=0x7f9f20069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 shutdown_connections 2023-12-26T21:49:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.416+0000 7f9f27bff700 1 -- 172.21.15.131:0/4079663345 wait complete. 2023-12-26T21:49:29.434 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:09", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:29.435 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:09', 'first': '2023-12-26T21:39:00', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:29.435 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:29.435 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:29.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:29.435 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-12-26T21:49:29.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:29.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:29.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:29.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:29.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:29.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:29.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:29.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:29.481 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:29.481 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:29.481 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:29.482 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_00_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:29.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-12-26T21:49:29.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.715+0000 7f0e131ff700 1 Processor -- start 2023-12-26T21:49:29.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.716+0000 7f0e131ff700 1 -- start start 2023-12-26T21:49:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.717+0000 7f0e131ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e0c0cce60 0x7f0e0c0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.717+0000 7f0e131ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0e0c0cbdc0 0x7f0e0c0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.717+0000 7f0e131ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 0x7f0e0c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.717+0000 7f0e131ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0c0ce760 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.717+0000 7f0e131ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0c0cf5a0 con 0x7f0e0c0cce60 2023-12-26T21:49:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.717+0000 7f0e131ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c159dc0 con 0x7f0e0c0cd230 2023-12-26T21:49:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e0bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0e0c0cbdc0 0x7f0e0c0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e0bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0e0c0cbdc0 0x7f0e0c0cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48656/0 (socket says 172.21.15.131:48656) 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e0bfff700 1 -- 172.21.15.131:0/837249491 learned_addr learned my addr 172.21.15.131:0/837249491 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e10f9b700 1 --2- 172.21.15.131:0/837249491 >> v2:172.21.15.190:3300/0 conn(0x7f0e0c0cce60 0x7f0e0c0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e1179c700 1 --2- 172.21.15.131:0/837249491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 0x7f0e0c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e0bfff700 1 -- 172.21.15.131:0/837249491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 msgr2=0x7f0e0c0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e0bfff700 1 --2- 172.21.15.131:0/837249491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 0x7f0e0c0cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e0bfff700 1 -- 172.21.15.131:0/837249491 >> v2:172.21.15.190:3300/0 conn(0x7f0e0c0cce60 msgr2=0x7f0e0c0ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e0bfff700 1 --2- 172.21.15.131:0/837249491 >> v2:172.21.15.190:3300/0 conn(0x7f0e0c0cce60 0x7f0e0c0ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e0bfff700 1 -- 172.21.15.131:0/837249491 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c0ce990 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.718+0000 7f0e1179c700 1 --2- 172.21.15.131:0/837249491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 0x7f0e0c0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e0bfff700 1 --2- 172.21.15.131:0/837249491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 0x7f0e0c0cfa30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0df80099b0 tx=0x7f0df8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4338eb378ed2b018 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e0b7fe700 1 -- 172.21.15.131:0/837249491 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8009ea0 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e0b7fe700 1 -- 172.21.15.131:0/837249491 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df800b820 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e0b7fe700 1 -- 172.21.15.131:0/837249491 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df800bbc0 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e131ff700 1 -- 172.21.15.131:0/837249491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 msgr2=0x7f0e0c0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e131ff700 1 --2- 172.21.15.131:0/837249491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 0x7f0e0c0cfa30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0df80099b0 tx=0x7f0df8009250 comp rx=0 tx=0).stop 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e131ff700 1 -- 172.21.15.131:0/837249491 shutdown_connections 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e131ff700 1 --2- 172.21.15.131:0/837249491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 0x7f0e0c0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e131ff700 1 --2- 172.21.15.131:0/837249491 >> v2:172.21.15.190:3300/0 conn(0x7f0e0c0cce60 0x7f0e0c0ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e131ff700 1 --2- 172.21.15.131:0/837249491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 0x7f0e0c0cfa30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e131ff700 1 -- 172.21.15.131:0/837249491 >> 172.21.15.131:0/837249491 conn(0x7f0e0c099960 msgr2=0x7f0e0c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.719+0000 7f0e131ff700 1 -- 172.21.15.131:0/837249491 shutdown_connections 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.720+0000 7f0e131ff700 1 -- 172.21.15.131:0/837249491 wait complete. 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.721+0000 7f0e131ff700 1 Processor -- start 2023-12-26T21:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.721+0000 7f0e131ff700 1 -- start start 2023-12-26T21:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.722+0000 7f0e131ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 0x7f0e0c0c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.722+0000 7f0e131ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e0c0cce60 0x7f0e0c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.722+0000 7f0e131ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 0x7f0e0c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.722+0000 7f0e131ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0c6a00 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.722+0000 7f0e131ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0c4410 con 0x7f0e0c0cce60 2023-12-26T21:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.722+0000 7f0e131ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0c6e90 con 0x7f0e0c0cd230 2023-12-26T21:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.722+0000 7f0e10f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 0x7f0e0c0c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.722+0000 7f0e10f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 0x7f0e0c0c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48664/0 (socket says 172.21.15.131:48664) 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.722+0000 7f0e10f9b700 1 -- 172.21.15.131:0/4050312030 learned_addr learned my addr 172.21.15.131:0/4050312030 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e0bfff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e0c0cce60 0x7f0e0c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e1179c700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 0x7f0e0c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e10f9b700 1 -- 172.21.15.131:0/4050312030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 msgr2=0x7f0e0c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e10f9b700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 0x7f0e0c15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e10f9b700 1 -- 172.21.15.131:0/4050312030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e0c0cce60 msgr2=0x7f0e0c157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e10f9b700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e0c0cce60 0x7f0e0c157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e10f9b700 1 -- 172.21.15.131:0/4050312030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df80094f0 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e0bfff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e0c0cce60 0x7f0e0c157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e10f9b700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 0x7f0e0c0c5070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000ea00 tx=0x7f0e00009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e08ff9700 1 -- 172.21.15.131:0/4050312030 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000c850 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e08ff9700 1 -- 172.21.15.131:0/4050312030 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000c9b0 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e08ff9700 1 -- 172.21.15.131:0/4050312030 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000cd30 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c0c5df0 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.723+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c0c63b0 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.724+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c0c6ce0 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.725+0000 7f0e08ff9700 1 -- 172.21.15.131:0/4050312030 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f0e00013680 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.726+0000 7f0e08ff9700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0df0067130 0x7f0df00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.727+0000 7f0e08ff9700 1 -- 172.21.15.131:0/4050312030 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f0e00085170 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.727+0000 7f0e0bfff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0df0067130 0x7f0df00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.727+0000 7f0e0bfff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0df0067130 0x7f0df00695e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0df80099b0 tx=0x7f0df8017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.730+0000 7f0e08ff9700 1 -- 172.21.15.131:0/4050312030 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e00051f50 con 0x7f0e0c0cbdc0 2023-12-26T21:49:29.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.951+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e0c0c6b40 con 0x7f0df0067130 2023-12-26T21:49:29.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.961+0000 7f0e08ff9700 1 -- 172.21.15.131:0/4050312030 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f0df8000c50 con 0x7f0df0067130 2023-12-26T21:49:29.962 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:29.962 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:09", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.964+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0df0067130 msgr2=0x7f0df00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.964+0000 7f0e131ff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0df0067130 0x7f0df00695e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0df80099b0 tx=0x7f0df8017000 comp rx=0 tx=0).stop 2023-12-26T21:49:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 msgr2=0x7f0e0c0c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 0x7f0e0c0c5070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000ea00 tx=0x7f0e00009120 comp rx=0 tx=0).stop 2023-12-26T21:49:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 shutdown_connections 2023-12-26T21:49:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0df0067130 0x7f0df00695e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e0c0cd230 0x7f0e0c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e0c0cbdc0 0x7f0e0c0c5070 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 --2- 172.21.15.131:0/4050312030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e0c0cce60 0x7f0e0c157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 >> 172.21.15.131:0/4050312030 conn(0x7f0e0c099960 msgr2=0x7f0e0c158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 shutdown_connections 2023-12-26T21:49:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:29.965+0000 7f0e131ff700 1 -- 172.21.15.131:0/4050312030 wait complete. 2023-12-26T21:49:29.981 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:09", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:29.982 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:09', 'first': '2023-12-26T21:39:00', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:29.982 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:29.982 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:29.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:29.982 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-12-26T21:49:30.027 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:30.027 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:30.027 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:30.028 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:30.028 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:30.028 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:30.028 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:30.028 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:30.028 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:30.028 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:30.028 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:30.029 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_00_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:30.029 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-12-26T21:49:30.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.263+0000 7f8aa39ff700 1 Processor -- start 2023-12-26T21:49:30.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.263+0000 7f8aa39ff700 1 -- start start 2023-12-26T21:49:30.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.264+0000 7f8aa39ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a9c156360 0x7f8a9c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.264+0000 7f8aa39ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8a9c156730 0x7f8a9c06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.264+0000 7f8aa39ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9c0acfc0 con 0x7f8a9c0c8490 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.264+0000 7f8aa39ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c06f1f0 con 0x7f8a9c156730 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.264+0000 7f8aa39ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0c91f0 con 0x7f8a9c156360 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa0f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8a9c156730 0x7f8a9c06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa1f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8a9c0c8490 0x7f8a9c0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45816/0 (socket says 172.21.15.131:45816) 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa1f9c700 1 -- 172.21.15.131:0/875625576 learned_addr learned my addr 172.21.15.131:0/875625576 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa0f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8a9c156730 0x7f8a9c06ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48678/0 (socket says 172.21.15.131:48678) 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa179b700 1 --2- 172.21.15.131:0/875625576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a9c156360 0x7f8a9c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa0f9a700 1 -- 172.21.15.131:0/875625576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a9c156360 msgr2=0x7f8a9c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa0f9a700 1 --2- 172.21.15.131:0/875625576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a9c156360 0x7f8a9c06e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa0f9a700 1 -- 172.21.15.131:0/875625576 >> v1:172.21.15.131:6789/0 conn(0x7f8a9c0c8490 legacy=0x7f8a9c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa0f9a700 1 -- 172.21.15.131:0/875625576 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c0c94d0 con 0x7f8a9c156730 2023-12-26T21:49:30.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.265+0000 7f8aa0f9a700 1 --2- 172.21.15.131:0/875625576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156730 0x7f8a9c06ecb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0098e0 tx=0x7f8a8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b80f302cfa1da44 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:30.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8a93fff700 1 -- 172.21.15.131:0/875625576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c00b640 con 0x7f8a9c156730 2023-12-26T21:49:30.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8a93fff700 1 -- 172.21.15.131:0/875625576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c00b7a0 con 0x7f8a9c156730 2023-12-26T21:49:30.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8a93fff700 1 -- 172.21.15.131:0/875625576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c00bb40 con 0x7f8a9c156730 2023-12-26T21:49:30.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8aa39ff700 1 -- 172.21.15.131:0/875625576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156730 msgr2=0x7f8a9c06ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8aa39ff700 1 --2- 172.21.15.131:0/875625576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156730 0x7f8a9c06ecb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0098e0 tx=0x7f8a8c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8aa39ff700 1 -- 172.21.15.131:0/875625576 shutdown_connections 2023-12-26T21:49:30.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8aa39ff700 1 --2- 172.21.15.131:0/875625576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a9c156360 0x7f8a9c06e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8aa39ff700 1 --2- 172.21.15.131:0/875625576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156730 0x7f8a9c06ecb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8aa39ff700 1 -- 172.21.15.131:0/875625576 >> 172.21.15.131:0/875625576 conn(0x7f8a9c0675b0 msgr2=0x7f8a9c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:30.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8aa39ff700 1 -- 172.21.15.131:0/875625576 shutdown_connections 2023-12-26T21:49:30.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.266+0000 7f8aa39ff700 1 -- 172.21.15.131:0/875625576 wait complete. 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.268+0000 7f8aa39ff700 1 Processor -- start 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.268+0000 7f8aa39ff700 1 -- start start 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa39ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a9c0c8490 0x7f8a9c0c2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa39ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156360 0x7f8a9c0adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa39ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c156730 0x7f8a9c0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa39ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0c1520 con 0x7f8a9c156360 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa39ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0c0910 con 0x7f8a9c156730 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa39ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0c0bf0 con 0x7f8a9c0c8490 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa0f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156360 0x7f8a9c0adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa0f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156360 0x7f8a9c0adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48690/0 (socket says 172.21.15.131:48690) 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa0f9a700 1 -- 172.21.15.131:0/3347773137 learned_addr learned my addr 172.21.15.131:0/3347773137 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:30.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.269+0000 7f8aa1f9c700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c156730 0x7f8a9c0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8aa0f9a700 1 -- 172.21.15.131:0/3347773137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a9c0c8490 msgr2=0x7f8a9c0c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8aa0f9a700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a9c0c8490 0x7f8a9c0c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8aa0f9a700 1 -- 172.21.15.131:0/3347773137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c156730 msgr2=0x7f8a9c0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8aa0f9a700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c156730 0x7f8a9c0ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8aa0f9a700 1 -- 172.21.15.131:0/3347773137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c009420 con 0x7f8a9c156360 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8aa0f9a700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156360 0x7f8a9c0adcf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00a010 tx=0x7f8a8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8a917fa700 1 -- 172.21.15.131:0/3347773137 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c01f070 con 0x7f8a9c156360 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8a917fa700 1 -- 172.21.15.131:0/3347773137 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c01aab0 con 0x7f8a9c156360 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8a917fa700 1 -- 172.21.15.131:0/3347773137 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c01ae50 con 0x7f8a9c156360 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c0c0ed0 con 0x7f8a9c156360 2023-12-26T21:49:30.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.270+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c0c2db0 con 0x7f8a9c156360 2023-12-26T21:49:30.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.271+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9c0717f0 con 0x7f8a9c156360 2023-12-26T21:49:30.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.272+0000 7f8a917fa700 1 -- 172.21.15.131:0/3347773137 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f8a8c023070 con 0x7f8a9c156360 2023-12-26T21:49:30.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.273+0000 7f8a917fa700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8a800671b0 0x7f8a80069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.274+0000 7f8a917fa700 1 -- 172.21.15.131:0/3347773137 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f8a8c085c30 con 0x7f8a9c156360 2023-12-26T21:49:30.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.274+0000 7f8aa179b700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8a800671b0 0x7f8a80069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.274+0000 7f8aa179b700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8a800671b0 0x7f8a80069660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009bb0 tx=0x7f8a980091a0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:30.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.278+0000 7f8a917fa700 1 -- 172.21.15.131:0/3347773137 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a8c052a10 con 0x7f8a9c156360 2023-12-26T21:49:30.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.497+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a9c0c2fd0 con 0x7f8a800671b0 2023-12-26T21:49:30.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.506+0000 7f8a917fa700 1 -- 172.21.15.131:0/3347773137 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f8a9c0c2fd0 con 0x7f8a800671b0 2023-12-26T21:49:30.506 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:30.506 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:10", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:30.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8a800671b0 msgr2=0x7f8a80069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8a800671b0 0x7f8a80069660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009bb0 tx=0x7f8a980091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156360 msgr2=0x7f8a9c0adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156360 0x7f8a9c0adcf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00a010 tx=0x7f8a8c017000 comp rx=0 tx=0).stop 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 shutdown_connections 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a9c156730 0x7f8a9c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8a800671b0 0x7f8a80069660 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a9c0c8490 0x7f8a9c0c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 --2- 172.21.15.131:0/3347773137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a9c156360 0x7f8a9c0adcf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 >> 172.21.15.131:0/3347773137 conn(0x7f8a9c0675b0 msgr2=0x7f8a9c14f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 shutdown_connections 2023-12-26T21:49:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.509+0000 7f8aa39ff700 1 -- 172.21.15.131:0/3347773137 wait complete. 2023-12-26T21:49:30.526 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:10", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:30.527 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:10', 'first': '2023-12-26T21:39:00', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:30.527 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:30.527 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:30.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:30.527 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-12-26T21:49:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:30.572 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:30.574 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_00_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:30.574 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-12-26T21:49:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.807+0000 7f030c3ff700 1 Processor -- start 2023-12-26T21:49:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.807+0000 7f030c3ff700 1 -- start start 2023-12-26T21:49:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f030c3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0304156730 0x7f03040c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f030c3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f030406ee80 0x7f03040c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f030c3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03040acfc0 con 0x7f0304156360 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f030c3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03040c91f0 con 0x7f0304156730 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f030c3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03040c94d0 con 0x7f030406ee80 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f030a19b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0304156360 0x7f030406e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45830/0 (socket says 172.21.15.131:45830) 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f030a19b700 1 -- 172.21.15.131:0/3483391347 learned_addr learned my addr 172.21.15.131:0/3483391347 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f030a99c700 1 --2- 172.21.15.131:0/3483391347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f030406ee80 0x7f03040c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f030999a700 1 --2- 172.21.15.131:0/3483391347 >> v2:172.21.15.190:3300/0 conn(0x7f0304156730 0x7f03040c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f0309199700 1 -- 172.21.15.131:0/3483391347 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107077752 0 0) 0x7f03040acfc0 con 0x7f0304156360 2023-12-26T21:49:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.809+0000 7f0309199700 1 -- 172.21.15.131:0/3483391347 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02f0003570 con 0x7f0304156360 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f030999a700 1 -- 172.21.15.131:0/3483391347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f030406ee80 msgr2=0x7f03040c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f0309199700 1 -- 172.21.15.131:0/3483391347 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4153493873 0 0) 0x7f02f0003570 con 0x7f0304156360 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f030999a700 1 --2- 172.21.15.131:0/3483391347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f030406ee80 0x7f03040c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f030999a700 1 -- 172.21.15.131:0/3483391347 >> v1:172.21.15.131:6789/0 conn(0x7f0304156360 legacy=0x7f030406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f030999a700 1 -- 172.21.15.131:0/3483391347 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03040acc50 con 0x7f0304156730 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f030a99c700 1 --2- 172.21.15.131:0/3483391347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f030406ee80 0x7f03040c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f030999a700 1 --2- 172.21.15.131:0/3483391347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0304156730 0x7f03040c8490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f02f40098e0 tx=0x7f02f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59b5f99a0d88a0e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f0309199700 1 -- 172.21.15.131:0/3483391347 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300001470 con 0x7f0304156730 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f0309199700 1 -- 172.21.15.131:0/3483391347 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f4011420 con 0x7f0304156730 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.810+0000 7f0309199700 1 -- 172.21.15.131:0/3483391347 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f40117c0 con 0x7f0304156730 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.811+0000 7f030c3ff700 1 -- 172.21.15.131:0/3483391347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0304156730 msgr2=0x7f03040c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.811+0000 7f030c3ff700 1 --2- 172.21.15.131:0/3483391347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0304156730 0x7f03040c8490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f02f40098e0 tx=0x7f02f40091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.811+0000 7f030c3ff700 1 -- 172.21.15.131:0/3483391347 shutdown_connections 2023-12-26T21:49:30.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.811+0000 7f030c3ff700 1 --2- 172.21.15.131:0/3483391347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f030406ee80 0x7f03040c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.811+0000 7f030c3ff700 1 --2- 172.21.15.131:0/3483391347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0304156730 0x7f03040c8490 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.811+0000 7f030c3ff700 1 -- 172.21.15.131:0/3483391347 >> 172.21.15.131:0/3483391347 conn(0x7f03040675b0 msgr2=0x7f03040679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:30.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.811+0000 7f030c3ff700 1 -- 172.21.15.131:0/3483391347 shutdown_connections 2023-12-26T21:49:30.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.811+0000 7f030c3ff700 1 -- 172.21.15.131:0/3483391347 wait complete. 2023-12-26T21:49:30.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.812+0000 7f030c3ff700 1 Processor -- start 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.812+0000 7f030c3ff700 1 -- start start 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.813+0000 7f030c3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f030406ee80 0x7f0304155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.813+0000 7f030c3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0304156360 0x7f03040b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.813+0000 7f030c3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0304156730 0x7f03040c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.813+0000 7f030c3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030416df20 con 0x7f0304156730 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.813+0000 7f030c3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030416d310 con 0x7f030406ee80 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.813+0000 7f030c3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030416d5f0 con 0x7f0304156360 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0304156730 0x7f03040c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030999a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0304156360 0x7f03040b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a19b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f030406ee80 0x7f0304155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0304156730 0x7f03040c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48694/0 (socket says 172.21.15.131:48694) 2023-12-26T21:49:30.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a99c700 1 -- 172.21.15.131:0/1224154829 learned_addr learned my addr 172.21.15.131:0/1224154829 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a99c700 1 -- 172.21.15.131:0/1224154829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0304156360 msgr2=0x7f03040b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a99c700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0304156360 0x7f03040b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a99c700 1 -- 172.21.15.131:0/1224154829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f030406ee80 msgr2=0x7f0304155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a99c700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f030406ee80 0x7f0304155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a99c700 1 -- 172.21.15.131:0/1224154829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc009a20 con 0x7f0304156730 2023-12-26T21:49:30.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.814+0000 7f030a99c700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0304156730 0x7f03040c4b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009f90 tx=0x7f02fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.815+0000 7f02fa7fc700 1 -- 172.21.15.131:0/1224154829 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00cbd0 con 0x7f0304156730 2023-12-26T21:49:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.815+0000 7f02fa7fc700 1 -- 172.21.15.131:0/1224154829 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc00f040 con 0x7f0304156730 2023-12-26T21:49:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.815+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f4009420 con 0x7f0304156730 2023-12-26T21:49:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.815+0000 7f02fa7fc700 1 -- 172.21.15.131:0/1224154829 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc014480 con 0x7f0304156730 2023-12-26T21:49:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.815+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030416dbb0 con 0x7f0304156730 2023-12-26T21:49:30.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.816+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03040717f0 con 0x7f0304156730 2023-12-26T21:49:30.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.817+0000 7f02fa7fc700 1 -- 172.21.15.131:0/1224154829 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f02fc0147f0 con 0x7f0304156730 2023-12-26T21:49:30.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.818+0000 7f02fa7fc700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f02e8067130 0x7f02e80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:30.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.818+0000 7f02fa7fc700 1 -- 172.21.15.131:0/1224154829 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f02fc0858d0 con 0x7f0304156730 2023-12-26T21:49:30.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.818+0000 7f030a19b700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f02e8067130 0x7f02e80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:30.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.818+0000 7f030a19b700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f02e8067130 0x7f02e80695e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03000011e0 tx=0x7f0300001aa0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:30.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:30.822+0000 7f02fa7fc700 1 -- 172.21.15.131:0/1224154829 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02fc0525a0 con 0x7f0304156730 2023-12-26T21:49:31.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.039+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03040c8f10 con 0x7f02e8067130 2023-12-26T21:49:31.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.048+0000 7f02fa7fc700 1 -- 172.21.15.131:0/1224154829 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f03040c8f10 con 0x7f02e8067130 2023-12-26T21:49:31.048 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:31.048 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:11", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:31.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f02e8067130 msgr2=0x7f02e80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f02e8067130 0x7f02e80695e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03000011e0 tx=0x7f0300001aa0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0304156730 msgr2=0x7f03040c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0304156730 0x7f03040c4b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009f90 tx=0x7f02fc00b040 comp rx=0 tx=0).stop 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 shutdown_connections 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0304156730 0x7f03040c4b90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f030406ee80 0x7f0304155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f02e8067130 0x7f02e80695e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 --2- 172.21.15.131:0/1224154829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0304156360 0x7f03040b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 >> 172.21.15.131:0/1224154829 conn(0x7f03040675b0 msgr2=0x7f0304067980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 shutdown_connections 2023-12-26T21:49:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.051+0000 7f030c3ff700 1 -- 172.21.15.131:0/1224154829 wait complete. 2023-12-26T21:49:31.067 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:11", "first": "2023-12-26T21:39:00", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:31.068 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:11', 'first': '2023-12-26T21:39:00', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:31.068 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:31.068 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:31.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:31.068 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-12-26T21:49:31.113 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_01_UTC 2023-12-26T21:49:31.113 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:31.115 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_39_01_UTC', 'scheduled-2023-12-26-21_40_00_UTC', 'scheduled-2023-12-26-21_41_00_UTC', 'scheduled-2023-12-26-21_42_00_UTC', 'scheduled-2023-12-26-21_43_00_UTC', 'scheduled-2023-12-26-21_44_00_UTC', 'scheduled-2023-12-26-21_45_00_UTC', 'scheduled-2023-12-26-21_46_00_UTC', 'scheduled-2023-12-26-21_47_00_UTC', 'scheduled-2023-12-26-21_48_00_UTC', 'scheduled-2023-12-26-21_49_00_UTC'] 2023-12-26T21:49:31.115 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-12-26T21:49:31.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.349+0000 7f9578dff700 1 Processor -- start 2023-12-26T21:49:31.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.349+0000 7f9578dff700 1 -- start start 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.350+0000 7f9578dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574142f00 0x7f9574069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.350+0000 7f9578dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95740cb510 0x7f95740c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.350+0000 7f9578dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9574069640 con 0x7f95740cc590 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.350+0000 7f9578dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9574068750 con 0x7f95740cb510 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.350+0000 7f9578dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9574068a30 con 0x7f9574142f00 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f957259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574142f00 0x7f9574069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f9572d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95740cb510 0x7f95740c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f957259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574142f00 0x7f9574069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:53528/0 (socket says 172.21.15.131:53528) 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f957259c700 1 -- 172.21.15.131:0/1732738283 learned_addr learned my addr 172.21.15.131:0/1732738283 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:31.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f9572d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95740cb510 0x7f95740c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:43804/0 (socket says 172.21.15.131:43804) 2023-12-26T21:49:31.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f957259c700 1 -- 172.21.15.131:0/1732738283 >> v2:172.21.15.190:3300/0 conn(0x7f95740cb510 msgr2=0x7f95740c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:31.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f957259c700 1 --2- 172.21.15.131:0/1732738283 >> v2:172.21.15.190:3300/0 conn(0x7f95740cb510 0x7f95740c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f957259c700 1 -- 172.21.15.131:0/1732738283 >> v1:172.21.15.190:6789/0 conn(0x7f95740cc590 legacy=0x7f95740cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f957259c700 1 -- 172.21.15.131:0/1732738283 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9574068d10 con 0x7f9574142f00 2023-12-26T21:49:31.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f9572d9d700 1 --2- 172.21.15.131:0/1732738283 >> v2:172.21.15.190:3300/0 conn(0x7f95740cb510 0x7f95740c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:31.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.351+0000 7f957259c700 1 --2- 172.21.15.131:0/1732738283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574142f00 0x7f9574069ad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f95640099b0 tx=0x7f9564009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f293798f982189 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:31.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f957159a700 1 -- 172.21.15.131:0/1732738283 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564009ea0 con 0x7f9574142f00 2023-12-26T21:49:31.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f957159a700 1 -- 172.21.15.131:0/1732738283 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956400b860 con 0x7f9574142f00 2023-12-26T21:49:31.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f957159a700 1 -- 172.21.15.131:0/1732738283 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956400bc00 con 0x7f9574142f00 2023-12-26T21:49:31.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f9578dff700 1 -- 172.21.15.131:0/1732738283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574142f00 msgr2=0x7f9574069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f9578dff700 1 --2- 172.21.15.131:0/1732738283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574142f00 0x7f9574069ad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f95640099b0 tx=0x7f9564009250 comp rx=0 tx=0).stop 2023-12-26T21:49:31.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f9578dff700 1 -- 172.21.15.131:0/1732738283 shutdown_connections 2023-12-26T21:49:31.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f9578dff700 1 --2- 172.21.15.131:0/1732738283 >> v2:172.21.15.190:3300/0 conn(0x7f95740cb510 0x7f95740c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f9578dff700 1 --2- 172.21.15.131:0/1732738283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9574142f00 0x7f9574069ad0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f9578dff700 1 -- 172.21.15.131:0/1732738283 >> 172.21.15.131:0/1732738283 conn(0x7f9574063cd0 msgr2=0x7f957406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:31.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f9578dff700 1 -- 172.21.15.131:0/1732738283 shutdown_connections 2023-12-26T21:49:31.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.352+0000 7f9578dff700 1 -- 172.21.15.131:0/1732738283 wait complete. 2023-12-26T21:49:31.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.354+0000 7f9578dff700 1 Processor -- start 2023-12-26T21:49:31.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.354+0000 7f9578dff700 1 -- start start 2023-12-26T21:49:31.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.355+0000 7f9578dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740cb510 0x7f95740b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.355+0000 7f9578dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95740cc590 0x7f957415d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.355+0000 7f9578dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574142f00 0x7f95740b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.355+0000 7f9578dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957415eb20 con 0x7f95740cc590 2023-12-26T21:49:31.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.355+0000 7f9578dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95740b05a0 con 0x7f9574142f00 2023-12-26T21:49:31.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.355+0000 7f9578dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957415dea0 con 0x7f95740cb510 2023-12-26T21:49:31.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.355+0000 7f9571d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95740cc590 0x7f957415d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.355+0000 7f9571d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95740cc590 0x7f957415d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48702/0 (socket says 172.21.15.131:48702) 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.355+0000 7f9571d9b700 1 -- 172.21.15.131:0/631843646 learned_addr learned my addr 172.21.15.131:0/631843646 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9572d9d700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574142f00 0x7f95740b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f957259c700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740cb510 0x7f95740b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9571d9b700 1 -- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740cb510 msgr2=0x7f95740b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9571d9b700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740cb510 0x7f95740b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9571d9b700 1 -- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574142f00 msgr2=0x7f95740b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9571d9b700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574142f00 0x7f95740b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9571d9b700 1 -- 172.21.15.131:0/631843646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c009a20 con 0x7f95740cc590 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9572d9d700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574142f00 0x7f95740b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f957259c700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740cb510 0x7f95740b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9571d9b700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95740cc590 0x7f957415d7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f956800c590 tx=0x7f956800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f95627fc700 1 -- 172.21.15.131:0/631843646 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568011590 con 0x7f95740cc590 2023-12-26T21:49:31.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f95627fc700 1 -- 172.21.15.131:0/631843646 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95680116f0 con 0x7f95740cc590 2023-12-26T21:49:31.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95640094f0 con 0x7f95740cc590 2023-12-26T21:49:31.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f95627fc700 1 -- 172.21.15.131:0/631843646 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568011ab0 con 0x7f95740cc590 2023-12-26T21:49:31.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.356+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957415e460 con 0x7f95740cc590 2023-12-26T21:49:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.357+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957415ed90 con 0x7f95740cc590 2023-12-26T21:49:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.359+0000 7f95627fc700 1 -- 172.21.15.131:0/631843646 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f956801b3b0 con 0x7f95740cc590 2023-12-26T21:49:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.359+0000 7f95627fc700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9554067130 0x7f95540695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.359+0000 7f95627fc700 1 -- 172.21.15.131:0/631843646 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f9568084080 con 0x7f95740cc590 2023-12-26T21:49:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.360+0000 7f957259c700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9554067130 0x7f95540695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.360+0000 7f957259c700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9554067130 0x7f95540695e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95640099b0 tx=0x7f9564017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:31.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.364+0000 7f95627fc700 1 -- 172.21.15.131:0/631843646 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9568050e60 con 0x7f95740cc590 2023-12-26T21:49:31.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.582+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95740717f0 con 0x7f9554067130 2023-12-26T21:49:31.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.592+0000 7f95627fc700 1 -- 172.21.15.131:0/631843646 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f95740717f0 con 0x7f9554067130 2023-12-26T21:49:31.591 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:31.592 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:11", "first": "2023-12-26T21:39:01", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9554067130 msgr2=0x7f95540695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9554067130 0x7f95540695e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95640099b0 tx=0x7f9564017000 comp rx=0 tx=0).stop 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95740cc590 msgr2=0x7f957415d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95740cc590 0x7f957415d7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f956800c590 tx=0x7f956800c950 comp rx=0 tx=0).stop 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 shutdown_connections 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574142f00 0x7f95740b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9554067130 0x7f95540695e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740cb510 0x7f95740b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 --2- 172.21.15.131:0/631843646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95740cc590 0x7f957415d7a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 >> 172.21.15.131:0/631843646 conn(0x7f9574063cd0 msgr2=0x7f95740c40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:31.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 shutdown_connections 2023-12-26T21:49:31.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.594+0000 7f9578dff700 1 -- 172.21.15.131:0/631843646 wait complete. 2023-12-26T21:49:31.611 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:38:11", "first": "2023-12-26T21:39:01", "last": "2023-12-26T21:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-26T21:49:31.611 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:38:11', 'first': '2023-12-26T21:39:01', 'last': '2023-12-26T21:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-26T21:49:31.611 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T21:49:31.611 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T21:49:31.612 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-12-26T21:49:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.805+0000 7f8f091ff700 1 Processor -- start 2023-12-26T21:49:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.805+0000 7f8f091ff700 1 -- start start 2023-12-26T21:49:31.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.806+0000 7f8f091ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 0x7f8f040c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.806+0000 7f8f091ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8f040cd7b0 0x7f8f040cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.806+0000 7f8f091ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f040cbb00 con 0x7f8f040cec80 2023-12-26T21:49:31.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.806+0000 7f8f091ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f040cf660 con 0x7f8f040cd7b0 2023-12-26T21:49:31.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.806+0000 7f8f091ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040cb2a0 con 0x7f8f04159810 2023-12-26T21:49:31.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.807+0000 7f8f0359e700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8f040cd7b0 0x7f8f040cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.807+0000 7f8f0359e700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8f040cd7b0 0x7f8f040cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48708/0 (socket says 172.21.15.131:48708) 2023-12-26T21:49:31.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.807+0000 7f8f0359e700 1 -- 172.21.15.131:0/2555692100 learned_addr learned my addr 172.21.15.131:0/2555692100 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:31.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.807+0000 7f8f02d9d700 1 --2- 172.21.15.131:0/2555692100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 0x7f8f040c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.807+0000 7f8f0359e700 1 -- 172.21.15.131:0/2555692100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 msgr2=0x7f8f040c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:31.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.807+0000 7f8f0359e700 1 --2- 172.21.15.131:0/2555692100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 0x7f8f040c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.807+0000 7f8f0359e700 1 -- 172.21.15.131:0/2555692100 >> v1:172.21.15.190:6789/0 conn(0x7f8f040cec80 legacy=0x7f8f040cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:31.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.807+0000 7f8f0359e700 1 -- 172.21.15.131:0/2555692100 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f040ce040 con 0x7f8f040cd7b0 2023-12-26T21:49:31.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.807+0000 7f8f0359e700 1 --2- 172.21.15.131:0/2555692100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cd7b0 0x7f8f040cf120 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009f20 tx=0x7f8efc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b5141b37a5211 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:31.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f01d9b700 1 -- 172.21.15.131:0/2555692100 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00e070 con 0x7f8f040cd7b0 2023-12-26T21:49:31.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f01d9b700 1 -- 172.21.15.131:0/2555692100 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00ba80 con 0x7f8f040cd7b0 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f01d9b700 1 -- 172.21.15.131:0/2555692100 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0092e0 con 0x7f8f040cd7b0 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f091ff700 1 -- 172.21.15.131:0/2555692100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cd7b0 msgr2=0x7f8f040cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f091ff700 1 --2- 172.21.15.131:0/2555692100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cd7b0 0x7f8f040cf120 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009f20 tx=0x7f8efc009840 comp rx=0 tx=0).stop 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f091ff700 1 -- 172.21.15.131:0/2555692100 shutdown_connections 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f091ff700 1 --2- 172.21.15.131:0/2555692100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cd7b0 0x7f8f040cf120 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f091ff700 1 --2- 172.21.15.131:0/2555692100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 0x7f8f040c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f091ff700 1 -- 172.21.15.131:0/2555692100 >> 172.21.15.131:0/2555692100 conn(0x7f8f04099950 msgr2=0x7f8f0406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f091ff700 1 -- 172.21.15.131:0/2555692100 shutdown_connections 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.808+0000 7f8f091ff700 1 -- 172.21.15.131:0/2555692100 wait complete. 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.810+0000 7f8f091ff700 1 Processor -- start 2023-12-26T21:49:31.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.810+0000 7f8f091ff700 1 -- start start 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f091ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f040cd7b0 0x7f8f041562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f091ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cec80 0x7f8f04145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f091ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 0x7f8f041575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f091ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04154c30 con 0x7f8f040cec80 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f091ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04155a90 con 0x7f8f040cd7b0 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f091ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04154e80 con 0x7f8f04159810 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f0259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cec80 0x7f8f04145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f0259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cec80 0x7f8f04145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48724/0 (socket says 172.21.15.131:48724) 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f0259c700 1 -- 172.21.15.131:0/157467610 learned_addr learned my addr 172.21.15.131:0/157467610 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:31.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f02d9d700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f040cd7b0 0x7f8f041562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.811+0000 7f8f0359e700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 0x7f8f041575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8f0259c700 1 -- 172.21.15.131:0/157467610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 msgr2=0x7f8f041575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8f0259c700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 0x7f8f041575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8f0259c700 1 -- 172.21.15.131:0/157467610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f040cd7b0 msgr2=0x7f8f041562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8f0259c700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f040cd7b0 0x7f8f041562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8f0259c700 1 -- 172.21.15.131:0/157467610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc009ae0 con 0x7f8f040cec80 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8f0359e700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 0x7f8f041575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8f0259c700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cec80 0x7f8f04145000 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c510 tx=0x7f8ef800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8eeaffd700 1 -- 172.21.15.131:0/157467610 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8011590 con 0x7f8f040cec80 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8eeaffd700 1 -- 172.21.15.131:0/157467610 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef8011710 con 0x7f8f040cec80 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8eeaffd700 1 -- 172.21.15.131:0/157467610 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8011ad0 con 0x7f8f040cec80 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f04155160 con 0x7f8f040cec80 2023-12-26T21:49:31.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.812+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f04155720 con 0x7f8f040cec80 2023-12-26T21:49:31.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.813+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f040717f0 con 0x7f8f040cec80 2023-12-26T21:49:31.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.814+0000 7f8eeaffd700 1 -- 172.21.15.131:0/157467610 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f8ef801b3b0 con 0x7f8f040cec80 2023-12-26T21:49:31.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.815+0000 7f8eeaffd700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8ef0067130 0x7f8ef00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:31.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.815+0000 7f8eeaffd700 1 -- 172.21.15.131:0/157467610 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f8ef8083f50 con 0x7f8f040cec80 2023-12-26T21:49:31.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.815+0000 7f8f02d9d700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8ef0067130 0x7f8ef00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:31.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.816+0000 7f8f02d9d700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8ef0067130 0x7f8ef00695e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4006aa0 tx=0x7f8ef4008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:31.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:31.819+0000 7f8eeaffd700 1 -- 172.21.15.131:0/157467610 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ef8050d30 con 0x7f8f040cec80 2023-12-26T21:49:32.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.036+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f04155bd0 con 0x7f8ef0067130 2023-12-26T21:49:32.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.123+0000 7f8eeaffd700 1 -- 172.21.15.131:0/157467610 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f8f04155bd0 con 0x7f8ef0067130 2023-12-26T21:49:32.123 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-12-26T21:49:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8ef0067130 msgr2=0x7f8ef00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8ef0067130 0x7f8ef00695e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4006aa0 tx=0x7f8ef4008040 comp rx=0 tx=0).stop 2023-12-26T21:49:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cec80 msgr2=0x7f8f04145000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cec80 0x7f8f04145000 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c510 tx=0x7f8ef800c820 comp rx=0 tx=0).stop 2023-12-26T21:49:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 shutdown_connections 2023-12-26T21:49:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8ef0067130 0x7f8ef00695e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f04159810 0x7f8f041575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f040cd7b0 0x7f8f041562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 --2- 172.21.15.131:0/157467610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f040cec80 0x7f8f04145000 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 >> 172.21.15.131:0/157467610 conn(0x7f8f04099950 msgr2=0x7f8f04158720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 shutdown_connections 2023-12-26T21:49:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:32.126+0000 7f8f091ff700 1 -- 172.21.15.131:0/157467610 wait complete. 2023-12-26T21:49:32.143 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-12-26T21:49:32.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.143 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-12-26T21:49:32.188 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:32.188 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:32.188 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:32.188 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:32.188 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:32.188 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:32.188 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:32.189 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:32.189 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:32.189 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:32.189 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:32.190 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:32.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.190 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_39_00_UTC) 2023-12-26T21:49:32.282 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:32.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.283 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:32.333 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:32.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.334 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:32.384 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:32.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.385 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:32.435 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:32.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.436 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:32.487 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:32.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.487 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:32.538 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:32.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.538 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:32.591 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:32.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.591 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:32.641 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:32.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.642 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:32.691 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:32.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.691 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:32.741 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:32.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.742 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:32.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:32.792 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-12-26T21:49:32.839 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-12-26T21:49:33.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.034+0000 7fd5e49ff700 1 Processor -- start 2023-12-26T21:49:33.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.034+0000 7fd5e49ff700 1 -- start start 2023-12-26T21:49:33.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.035+0000 7fd5e49ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00adcb0 0x7fd5e00abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:33.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.035+0000 7fd5e49ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e00c7430 con 0x7fd5e0069c30 2023-12-26T21:49:33.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.035+0000 7fd5e49ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e00ac460 con 0x7fd5e00ae080 2023-12-26T21:49:33.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.035+0000 7fd5e49ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00c6820 con 0x7fd5e00adcb0 2023-12-26T21:49:33.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5ded9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd5e0069c30 0x7fd5e006df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45840/0 (socket says 172.21.15.131:45840) 2023-12-26T21:49:33.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5ded9d700 1 -- 172.21.15.131:0/3120757945 learned_addr learned my addr 172.21.15.131:0/3120757945 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5de59c700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00adcb0 0x7fd5e00abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460425169 0 0) 0x7fd5e00c7430 con 0x7fd5e0069c30 2023-12-26T21:49:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5c8003570 con 0x7fd5e0069c30 2023-12-26T21:49:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470739759 0 0) 0x7fd5e00ac460 con 0x7fd5e00ae080 2023-12-26T21:49:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e00c7430 con 0x7fd5e00ae080 2023-12-26T21:49:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148117843 0 0) 0x7fd5c8003570 con 0x7fd5e0069c30 2023-12-26T21:49:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00adcb0 msgr2=0x7fd5e00abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00adcb0 0x7fd5e00abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 >> v1:172.21.15.190:6789/0 conn(0x7fd5e00ae080 legacy=0x7fd5e0069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e00c6b00 con 0x7fd5e0069c30 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.036+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fd5d80017f0 con 0x7fd5e0069c30 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 >> v1:172.21.15.131:6789/0 conn(0x7fd5e0069c30 legacy=0x7fd5e006df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5dd59a700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c8003c20 0x7fd5c8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5dd59a700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5c8004810 0x7fd5c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5dd59a700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00adcb0 0x7fd5c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00c6820 con 0x7fd5c8003c20 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c800bb40 con 0x7fd5c8004810 2023-12-26T21:49:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c800df10 con 0x7fd5e00adcb0 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5de59c700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c8003c20 0x7fd5c8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5ded9d700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00adcb0 0x7fd5c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5ddd9b700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5c8004810 0x7fd5c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5de59c700 1 -- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00adcb0 msgr2=0x7fd5c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5de59c700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00adcb0 0x7fd5c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5de59c700 1 -- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5c8004810 msgr2=0x7fd5c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5de59c700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5c8004810 0x7fd5c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5de59c700 1 -- 172.21.15.131:0/3120757945 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c8003570 con 0x7fd5c8003c20 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.037+0000 7fd5de59c700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c8003c20 0x7fd5c8004010 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009c00 tx=0x7fd5d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ab17afae9443d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000e710 con 0x7fd5c8003c20 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d000e870 con 0x7fd5c8003c20 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5dd59a700 1 -- 172.21.15.131:0/3120757945 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000ec30 con 0x7fd5c8003c20 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5e49ff700 1 -- 172.21.15.131:0/3120757945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c8003c20 msgr2=0x7fd5c8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5e49ff700 1 --2- 172.21.15.131:0/3120757945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5c8003c20 0x7fd5c8004010 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009c00 tx=0x7fd5d00091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5ded9d700 1 -- 172.21.15.131:0/3120757945 reap_dead start 2023-12-26T21:49:33.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5e49ff700 1 -- 172.21.15.131:0/3120757945 shutdown_connections 2023-12-26T21:49:33.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5e49ff700 1 -- 172.21.15.131:0/3120757945 >> 172.21.15.131:0/3120757945 conn(0x7fd5e00675b0 msgr2=0x7fd5e00679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:33.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5e49ff700 1 -- 172.21.15.131:0/3120757945 shutdown_connections 2023-12-26T21:49:33.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.038+0000 7fd5e49ff700 1 -- 172.21.15.131:0/3120757945 wait complete. 2023-12-26T21:49:33.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.040+0000 7fd5e49ff700 1 Processor -- start 2023-12-26T21:49:33.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.040+0000 7fd5e49ff700 1 -- start start 2023-12-26T21:49:33.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5e49ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5c8003c20 0x7fd5e00afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:33.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5e49ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e00b1800 0x7fd5e0156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:33.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5e49ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00b1c50 0x7fd5e0157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:33.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5e49ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0166500 con 0x7fd5e00b1800 2023-12-26T21:49:33.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5e49ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0166220 con 0x7fd5c8003c20 2023-12-26T21:49:33.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5e49ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00af130 con 0x7fd5e00b1c50 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5ddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e00b1800 0x7fd5e0156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5ddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e00b1800 0x7fd5e0156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48742/0 (socket says 172.21.15.131:48742) 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5ddd9b700 1 -- 172.21.15.131:0/2556267793 learned_addr learned my addr 172.21.15.131:0/2556267793 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5ddd9b700 1 -- 172.21.15.131:0/2556267793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00b1c50 msgr2=0x7fd5e0157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5ddd9b700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00b1c50 0x7fd5e0157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5ddd9b700 1 -- 172.21.15.131:0/2556267793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5c8003c20 msgr2=0x7fd5e00afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5ddd9b700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5c8003c20 0x7fd5e00afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.041+0000 7fd5ddd9b700 1 -- 172.21.15.131:0/2556267793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e00af410 con 0x7fd5e00b1800 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.042+0000 7fd5ddd9b700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e00b1800 0x7fd5e0156d10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e810 tx=0x7fd5d4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.042+0000 7fd5ce7fc700 1 -- 172.21.15.131:0/2556267793 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40026a0 con 0x7fd5e00b1800 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.042+0000 7fd5ce7fc700 1 -- 172.21.15.131:0/2556267793 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d4002820 con 0x7fd5e00b1800 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.042+0000 7fd5ce7fc700 1 -- 172.21.15.131:0/2556267793 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4002be0 con 0x7fd5e00b1800 2023-12-26T21:49:33.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.042+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e00af6f0 con 0x7fd5e00b1800 2023-12-26T21:49:33.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.042+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e0165730 con 0x7fd5e00b1800 2023-12-26T21:49:33.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.043+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e00aef90 con 0x7fd5e00b1800 2023-12-26T21:49:33.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.044+0000 7fd5ce7fc700 1 -- 172.21.15.131:0/2556267793 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fd5d401d460 con 0x7fd5e00b1800 2023-12-26T21:49:33.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.044+0000 7fd5ce7fc700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd5c0067130 0x7fd5c00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:33.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.045+0000 7fd5ce7fc700 1 -- 172.21.15.131:0/2556267793 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fd5d4085110 con 0x7fd5e00b1800 2023-12-26T21:49:33.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.045+0000 7fd5de59c700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd5c0067130 0x7fd5c00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:33.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.045+0000 7fd5de59c700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd5c0067130 0x7fd5c00695e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5e01592d0 tx=0x7fd5d001a000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:33.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.049+0000 7fd5ce7fc700 1 -- 172.21.15.131:0/2556267793 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5e00aef90 con 0x7fd5e00b1800 2023-12-26T21:49:33.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.268+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5e00aef90 con 0x7fd5c0067130 2023-12-26T21:49:33.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.354+0000 7fd5ce7fc700 1 -- 172.21.15.131:0/2556267793 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fd5e00aef90 con 0x7fd5c0067130 2023-12-26T21:49:33.354 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-12-26T21:49:33.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd5c0067130 msgr2=0x7fd5c00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:33.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd5c0067130 0x7fd5c00695e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5e01592d0 tx=0x7fd5d001a000 comp rx=0 tx=0).stop 2023-12-26T21:49:33.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e00b1800 msgr2=0x7fd5e0156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e00b1800 0x7fd5e0156d10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e810 tx=0x7fd5d4009040 comp rx=0 tx=0).stop 2023-12-26T21:49:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 shutdown_connections 2023-12-26T21:49:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd5c0067130 0x7fd5c00695e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e00b1c50 0x7fd5e0157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5c8003c20 0x7fd5e00afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 --2- 172.21.15.131:0/2556267793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e00b1800 0x7fd5e0156d10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 >> 172.21.15.131:0/2556267793 conn(0x7fd5e00675b0 msgr2=0x7fd5e00ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 shutdown_connections 2023-12-26T21:49:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:33.357+0000 7fd5e49ff700 1 -- 172.21.15.131:0/2556267793 wait complete. 2023-12-26T21:49:33.375 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-12-26T21:49:33.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.376 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-12-26T21:49:33.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:33.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:33.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:33.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:33.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:33.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:33.421 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:33.421 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:33.421 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:33.421 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:33.421 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:33.422 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:33.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.422 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_39_00_UTC) 2023-12-26T21:49:33.516 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:33.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.516 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:33.568 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:33.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.569 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:33.620 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:33.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.620 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:33.671 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:33.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.672 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:33.723 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:33.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.724 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:33.775 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:33.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.776 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:33.826 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:33.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.826 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:33.877 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:33.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.877 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:33.928 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:33.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.928 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:33.980 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:33.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:33.980 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:34.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:34.031 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-12-26T21:49:34.079 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-12-26T21:49:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.274+0000 7fcbfe5ff700 1 Processor -- start 2023-12-26T21:49:34.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.274+0000 7fcbfe5ff700 1 -- start start 2023-12-26T21:49:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.275+0000 7fcbfe5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbf80c9110 0x7fcbf815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.275+0000 7fcbfe5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf80af040 con 0x7fcbf8156e50 2023-12-26T21:49:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.275+0000 7fcbfe5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf80ae0e0 con 0x7fcbf80c8d40 2023-12-26T21:49:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.275+0000 7fcbfe5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf81579c0 con 0x7fcbf80c9110 2023-12-26T21:49:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.275+0000 7fcbf77fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbf80c9110 0x7fcbf815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.275+0000 7fcbfcb9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcbf8156e50 0x7fcbf81572b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45844/0 (socket says 172.21.15.131:45844) 2023-12-26T21:49:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.275+0000 7fcbfcb9c700 1 -- 172.21.15.131:0/1643269079 learned_addr learned my addr 172.21.15.131:0/1643269079 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.276+0000 7fcbf77fe700 1 -- 172.21.15.131:0/1643269079 >> v1:172.21.15.190:6789/0 conn(0x7fcbf80c8d40 legacy=0x7fcbf80ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.276+0000 7fcbf77fe700 1 -- 172.21.15.131:0/1643269079 >> v1:172.21.15.131:6789/0 conn(0x7fcbf8156e50 legacy=0x7fcbf81572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.276+0000 7fcbf77fe700 1 -- 172.21.15.131:0/1643269079 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf80ae3c0 con 0x7fcbf80c9110 2023-12-26T21:49:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.276+0000 7fcbf6ffd700 1 -- 172.21.15.131:0/1643269079 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591428847 0 0) 0x7fcbf80af040 con 0x7fcbf8156e50 2023-12-26T21:49:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.276+0000 7fcbf77fe700 1 --2- 172.21.15.131:0/1643269079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c9110 0x7fcbf815bc30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0098e0 tx=0x7fcbec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d890e9c2c18aeb0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.276+0000 7fcbf6ffd700 1 -- 172.21.15.131:0/1643269079 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec009e80 con 0x7fcbf80c9110 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.276+0000 7fcbf6ffd700 1 -- 172.21.15.131:0/1643269079 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec00b860 con 0x7fcbf80c9110 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.276+0000 7fcbf6ffd700 1 -- 172.21.15.131:0/1643269079 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00bbe0 con 0x7fcbf80c9110 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.277+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/1643269079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c9110 msgr2=0x7fcbf815bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.277+0000 7fcbfe5ff700 1 --2- 172.21.15.131:0/1643269079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c9110 0x7fcbf815bc30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0098e0 tx=0x7fcbec0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.277+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/1643269079 shutdown_connections 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.277+0000 7fcbfe5ff700 1 --2- 172.21.15.131:0/1643269079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c9110 0x7fcbf815bc30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.277+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/1643269079 >> 172.21.15.131:0/1643269079 conn(0x7fcbf80675b0 msgr2=0x7fcbf80679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.277+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/1643269079 shutdown_connections 2023-12-26T21:49:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.277+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/1643269079 wait complete. 2023-12-26T21:49:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.278+0000 7fcbfe5ff700 1 Processor -- start 2023-12-26T21:49:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.278+0000 7fcbfe5ff700 1 -- start start 2023-12-26T21:49:34.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.279+0000 7fcbfe5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c8d40 0x7fcbf80abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:34.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.279+0000 7fcbfe5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf80c9110 0x7fcbf816a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.279+0000 7fcbfe5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf8156e50 0x7fcbf814a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.279+0000 7fcbfe5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf816ab90 con 0x7fcbf80c8d40 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.279+0000 7fcbfe5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf8159bb0 con 0x7fcbf80c9110 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.279+0000 7fcbfe5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80ac250 con 0x7fcbf8156e50 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c8d40 0x7fcbf80abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c8d40 0x7fcbf80abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48760/0 (socket says 172.21.15.131:48760) 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf7fff700 1 -- 172.21.15.131:0/2914776512 learned_addr learned my addr 172.21.15.131:0/2914776512 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbfcb9c700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf8156e50 0x7fcbf814a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf77fe700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf80c9110 0x7fcbf816a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf7fff700 1 -- 172.21.15.131:0/2914776512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf8156e50 msgr2=0x7fcbf814a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf7fff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf8156e50 0x7fcbf814a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf7fff700 1 -- 172.21.15.131:0/2914776512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf80c9110 msgr2=0x7fcbf816a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf7fff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf80c9110 0x7fcbf816a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf7fff700 1 -- 172.21.15.131:0/2914776512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec009420 con 0x7fcbf80c8d40 2023-12-26T21:49:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbfcb9c700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf8156e50 0x7fcbf814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.280+0000 7fcbf7fff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c8d40 0x7fcbf80abb70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800c590 tx=0x7fcbe800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.281+0000 7fcbdffff700 1 -- 172.21.15.131:0/2914776512 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8011590 con 0x7fcbf80c8d40 2023-12-26T21:49:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.281+0000 7fcbdffff700 1 -- 172.21.15.131:0/2914776512 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe80116f0 con 0x7fcbf80c8d40 2023-12-26T21:49:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.281+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf8159280 con 0x7fcbf80c8d40 2023-12-26T21:49:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.281+0000 7fcbdffff700 1 -- 172.21.15.131:0/2914776512 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8011ab0 con 0x7fcbf80c8d40 2023-12-26T21:49:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.281+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf8159840 con 0x7fcbf80c8d40 2023-12-26T21:49:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.282+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf80717f0 con 0x7fcbf80c8d40 2023-12-26T21:49:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.283+0000 7fcbdffff700 1 -- 172.21.15.131:0/2914776512 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fcbe801b3b0 con 0x7fcbf80c8d40 2023-12-26T21:49:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.284+0000 7fcbdffff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbe4067130 0x7fcbe40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.284+0000 7fcbdffff700 1 -- 172.21.15.131:0/2914776512 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fcbe8083fc0 con 0x7fcbf80c8d40 2023-12-26T21:49:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.284+0000 7fcbf77fe700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbe4067130 0x7fcbe40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:34.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.284+0000 7fcbf77fe700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbe4067130 0x7fcbe40695e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0093f0 tx=0x7fcbec017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:34.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.288+0000 7fcbdffff700 1 -- 172.21.15.131:0/2914776512 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbe8050e40 con 0x7fcbf80c8d40 2023-12-26T21:49:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.511+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbf8159cf0 con 0x7fcbe4067130 2023-12-26T21:49:34.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.597+0000 7fcbdffff700 1 -- 172.21.15.131:0/2914776512 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fcbec00b9d0 con 0x7fcbe4067130 2023-12-26T21:49:34.596 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-12-26T21:49:34.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbe4067130 msgr2=0x7fcbe40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbe4067130 0x7fcbe40695e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0093f0 tx=0x7fcbec017000 comp rx=0 tx=0).stop 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c8d40 msgr2=0x7fcbf80abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c8d40 0x7fcbf80abb70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800c590 tx=0x7fcbe800c950 comp rx=0 tx=0).stop 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 shutdown_connections 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcbe4067130 0x7fcbe40695e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf8156e50 0x7fcbf814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbf80c8d40 0x7fcbf80abb70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 --2- 172.21.15.131:0/2914776512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf80c9110 0x7fcbf816a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 >> 172.21.15.131:0/2914776512 conn(0x7fcbf80675b0 msgr2=0x7fcbf81558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 shutdown_connections 2023-12-26T21:49:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:34.599+0000 7fcbfe5ff700 1 -- 172.21.15.131:0/2914776512 wait complete. 2023-12-26T21:49:34.616 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-12-26T21:49:34.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:34.617 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-12-26T21:49:34.661 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:34.661 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:34.661 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:34.661 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:34.661 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:34.661 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:34.661 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:34.661 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:34.662 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:34.662 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:34.662 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:34.666 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:34.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:34.666 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_39_00_UTC) 2023-12-26T21:49:34.758 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:34.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:34.758 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:34.810 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:34.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:34.810 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:34.861 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:34.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:34.861 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:34.911 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:34.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:34.912 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:34.963 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:34.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:34.963 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:35.016 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:35.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:35.016 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:35.066 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:35.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:35.067 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:35.118 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:35.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:35.119 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:35.169 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:35.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:35.170 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:35.220 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:35.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:35.220 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:35.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:35.271 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-12-26T21:49:35.324 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-12-26T21:49:35.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.575+0000 7f91ccbff700 1 Processor -- start 2023-12-26T21:49:35.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.576+0000 7f91ccbff700 1 -- start start 2023-12-26T21:49:35.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91ccbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156360 0x7f91c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:35.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91ccbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f91c80c8ba0 0x7f91c806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:35.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91ccbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c80acfc0 con 0x7f91c8156730 2023-12-26T21:49:35.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91ccbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c806f1f0 con 0x7f91c80c8ba0 2023-12-26T21:49:35.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91ccbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80c91f0 con 0x7f91c8156360 2023-12-26T21:49:35.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91c6d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f91c80c8ba0 0x7f91c806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:35.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91c659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156360 0x7f91c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:35.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91c5d9b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f91c8156730 0x7f91c80c8490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:53922/0 (socket says 172.21.15.131:53922) 2023-12-26T21:49:35.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91c5d9b700 1 -- 172.21.15.131:0/996030139 learned_addr learned my addr 172.21.15.131:0/996030139 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.578+0000 7f91c6d9d700 1 -- 172.21.15.131:0/996030139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156360 msgr2=0x7f91c806e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.578+0000 7f91c6d9d700 1 --2- 172.21.15.131:0/996030139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156360 0x7f91c806e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.577+0000 7f91c559a700 1 -- 172.21.15.131:0/996030139 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067713953 0 0) 0x7f91c80acfc0 con 0x7f91c8156730 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.578+0000 7f91c6d9d700 1 -- 172.21.15.131:0/996030139 >> v1:172.21.15.190:6789/0 conn(0x7f91c8156730 legacy=0x7f91c80c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.578+0000 7f91c6d9d700 1 -- 172.21.15.131:0/996030139 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c80c94d0 con 0x7f91c80c8ba0 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.578+0000 7f91c659c700 1 --2- 172.21.15.131:0/996030139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156360 0x7f91c806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.578+0000 7f91c6d9d700 1 --2- 172.21.15.131:0/996030139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91c80c8ba0 0x7f91c806ecb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e60 tx=0x7f91c0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725aaf0b75cb395e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.578+0000 7f91c559a700 1 -- 172.21.15.131:0/996030139 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c000e070 con 0x7f91c80c8ba0 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.578+0000 7f91c559a700 1 -- 172.21.15.131:0/996030139 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c000bb60 con 0x7f91c80c8ba0 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.578+0000 7f91c559a700 1 -- 172.21.15.131:0/996030139 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0012430 con 0x7f91c80c8ba0 2023-12-26T21:49:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.579+0000 7f91ccbff700 1 -- 172.21.15.131:0/996030139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91c80c8ba0 msgr2=0x7f91c806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.579+0000 7f91ccbff700 1 --2- 172.21.15.131:0/996030139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91c80c8ba0 0x7f91c806ecb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e60 tx=0x7f91c0009780 comp rx=0 tx=0).stop 2023-12-26T21:49:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.579+0000 7f91ccbff700 1 -- 172.21.15.131:0/996030139 shutdown_connections 2023-12-26T21:49:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.579+0000 7f91ccbff700 1 --2- 172.21.15.131:0/996030139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91c80c8ba0 0x7f91c806ecb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.579+0000 7f91ccbff700 1 --2- 172.21.15.131:0/996030139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156360 0x7f91c806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.579+0000 7f91ccbff700 1 -- 172.21.15.131:0/996030139 >> 172.21.15.131:0/996030139 conn(0x7f91c80675b0 msgr2=0x7f91c80679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.579+0000 7f91ccbff700 1 -- 172.21.15.131:0/996030139 shutdown_connections 2023-12-26T21:49:35.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.579+0000 7f91ccbff700 1 -- 172.21.15.131:0/996030139 wait complete. 2023-12-26T21:49:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.580+0000 7f91ccbff700 1 Processor -- start 2023-12-26T21:49:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.580+0000 7f91ccbff700 1 -- start start 2023-12-26T21:49:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.581+0000 7f91ccbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91c80c8ba0 0x7f91c80bfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.581+0000 7f91ccbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c8156360 0x7f91c80ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.581+0000 7f91ccbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156730 0x7f91c80c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.581+0000 7f91ccbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80abba0 con 0x7f91c8156360 2023-12-26T21:49:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.581+0000 7f91ccbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80c0eb0 con 0x7f91c80c8ba0 2023-12-26T21:49:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.581+0000 7f91ccbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80c02a0 con 0x7f91c8156730 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.581+0000 7f91c5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c8156360 0x7f91c80ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.581+0000 7f91c5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c8156360 0x7f91c80ab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38586/0 (socket says 172.21.15.131:38586) 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.581+0000 7f91c5d9b700 1 -- 172.21.15.131:0/3877972094 learned_addr learned my addr 172.21.15.131:0/3877972094 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91c659c700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91c80c8ba0 0x7f91c80bfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91c6d9d700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156730 0x7f91c80c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91c5d9b700 1 -- 172.21.15.131:0/3877972094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156730 msgr2=0x7f91c80c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91c5d9b700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156730 0x7f91c80c20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91c5d9b700 1 -- 172.21.15.131:0/3877972094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91c80c8ba0 msgr2=0x7f91c80bfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91c5d9b700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91c80c8ba0 0x7f91c80bfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91c5d9b700 1 -- 172.21.15.131:0/3877972094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b8009420 con 0x7f91c8156360 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91c5d9b700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c8156360 0x7f91c80ab660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c7e0 tx=0x7f91bc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91b67fc700 1 -- 172.21.15.131:0/3877972094 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc005b30 con 0x7f91c8156360 2023-12-26T21:49:35.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91b67fc700 1 -- 172.21.15.131:0/3877972094 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc005c90 con 0x7f91c8156360 2023-12-26T21:49:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c0009a20 con 0x7f91c8156360 2023-12-26T21:49:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.582+0000 7f91b67fc700 1 -- 172.21.15.131:0/3877972094 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc007300 con 0x7f91c8156360 2023-12-26T21:49:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.583+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c80c0860 con 0x7f91c8156360 2023-12-26T21:49:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.584+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c80717f0 con 0x7f91c8156360 2023-12-26T21:49:35.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.584+0000 7f91b67fc700 1 -- 172.21.15.131:0/3877972094 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f91bc0076b0 con 0x7f91c8156360 2023-12-26T21:49:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.586+0000 7f91b67fc700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f91a8067130 0x7f91a80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.586+0000 7f91b67fc700 1 -- 172.21.15.131:0/3877972094 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f91bc084230 con 0x7f91c8156360 2023-12-26T21:49:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.586+0000 7f91c659c700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f91a8067130 0x7f91a80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:35.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.587+0000 7f91c659c700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f91a8067130 0x7f91a80695e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91c8159810 tx=0x7f91b80091a0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:35.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.590+0000 7f91b67fc700 1 -- 172.21.15.131:0/3877972094 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91bc050d70 con 0x7f91c8156360 2023-12-26T21:49:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.826+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f91c80cca60 con 0x7f91a8067130 2023-12-26T21:49:35.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.935+0000 7f91b67fc700 1 -- 172.21.15.131:0/3877972094 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f91c80cca60 con 0x7f91a8067130 2023-12-26T21:49:36.031 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-12-26T21:49:36.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.937+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f91a8067130 msgr2=0x7f91a80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:36.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.937+0000 7f91ccbff700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f91a8067130 0x7f91a80695e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91c8159810 tx=0x7f91b80091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:36.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.937+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c8156360 msgr2=0x7f91c80ab660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:36.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.937+0000 7f91ccbff700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c8156360 0x7f91c80ab660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c7e0 tx=0x7f91bc007df0 comp rx=0 tx=0).stop 2023-12-26T21:49:36.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.938+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 shutdown_connections 2023-12-26T21:49:36.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.938+0000 7f91ccbff700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f91a8067130 0x7f91a80695e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:36.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.938+0000 7f91ccbff700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91c8156730 0x7f91c80c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:36.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.938+0000 7f91ccbff700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91c80c8ba0 0x7f91c80bfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:36.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.938+0000 7f91ccbff700 1 --2- 172.21.15.131:0/3877972094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c8156360 0x7f91c80ab660 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:36.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.938+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 >> 172.21.15.131:0/3877972094 conn(0x7f91c80675b0 msgr2=0x7f91c80c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:36.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.938+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 shutdown_connections 2023-12-26T21:49:36.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:35.938+0000 7f91ccbff700 1 -- 172.21.15.131:0/3877972094 wait complete. 2023-12-26T21:49:36.033 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-12-26T21:49:36.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.033 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:36.079 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:36.080 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:36.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.081 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_39_00_UTC) 2023-12-26T21:49:36.174 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:36.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.174 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:36.230 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:36.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.230 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:36.288 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:36.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.289 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:36.368 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:36.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.368 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:36.429 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:36.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.430 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:36.480 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:36.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.481 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:36.530 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:36.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.530 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:36.580 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:36.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.581 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:36.630 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:36.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.630 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:36.683 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:36.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.684 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:36.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:36.755 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-12-26T21:49:36.808 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-12-26T21:49:37.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.002+0000 7f056b5ff700 1 Processor -- start 2023-12-26T21:49:37.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.002+0000 7f056b5ff700 1 -- start start 2023-12-26T21:49:37.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.003+0000 7f056b5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0564156360 0x7f056406e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:37.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.003+0000 7f056b5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0564156730 0x7f056406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:37.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.003+0000 7f056b5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f05640c8490 0x7f05640c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:37.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.003+0000 7f056b5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056406f1f0 con 0x7f05640c8490 2023-12-26T21:49:37.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.003+0000 7f056b5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05640acfc0 con 0x7f0564156360 2023-12-26T21:49:37.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f056b5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05640c91f0 con 0x7f0564156730 2023-12-26T21:49:37.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0569b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f05640c8490 0x7f05640c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:37.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0569b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f05640c8490 0x7f05640c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38596/0 (socket says 172.21.15.131:38596) 2023-12-26T21:49:37.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0569b9c700 1 -- 172.21.15.131:0/4066665805 learned_addr learned my addr 172.21.15.131:0/4066665805 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0568b9a700 1 --2- 172.21.15.131:0/4066665805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0564156730 0x7f056406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f056939b700 1 --2- 172.21.15.131:0/4066665805 >> v2:172.21.15.190:3300/0 conn(0x7f0564156360 0x7f056406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0569b9c700 1 -- 172.21.15.131:0/4066665805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0564156730 msgr2=0x7f056406ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0569b9c700 1 --2- 172.21.15.131:0/4066665805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0564156730 0x7f056406ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0569b9c700 1 -- 172.21.15.131:0/4066665805 >> v2:172.21.15.190:3300/0 conn(0x7f0564156360 msgr2=0x7f056406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0569b9c700 1 --2- 172.21.15.131:0/4066665805 >> v2:172.21.15.190:3300/0 conn(0x7f0564156360 0x7f056406e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0569b9c700 1 -- 172.21.15.131:0/4066665805 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640c94d0 con 0x7f05640c8490 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.004+0000 7f0569b9c700 1 --2- 172.21.15.131:0/4066665805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05640c8490 0x7f05640c8930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0564069410 tx=0x7f055c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6f74b6f6f4d379 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f055bfff700 1 -- 172.21.15.131:0/4066665805 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c00e070 con 0x7f05640c8490 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f055bfff700 1 -- 172.21.15.131:0/4066665805 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c00b970 con 0x7f05640c8490 2023-12-26T21:49:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f055bfff700 1 -- 172.21.15.131:0/4066665805 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0092e0 con 0x7f05640c8490 2023-12-26T21:49:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f056b5ff700 1 -- 172.21.15.131:0/4066665805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05640c8490 msgr2=0x7f05640c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f056b5ff700 1 --2- 172.21.15.131:0/4066665805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05640c8490 0x7f05640c8930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0564069410 tx=0x7f055c009840 comp rx=0 tx=0).stop 2023-12-26T21:49:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f056b5ff700 1 -- 172.21.15.131:0/4066665805 shutdown_connections 2023-12-26T21:49:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f056b5ff700 1 --2- 172.21.15.131:0/4066665805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05640c8490 0x7f05640c8930 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f056b5ff700 1 --2- 172.21.15.131:0/4066665805 >> v2:172.21.15.190:3300/0 conn(0x7f0564156360 0x7f056406e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f056b5ff700 1 --2- 172.21.15.131:0/4066665805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0564156730 0x7f056406ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f056b5ff700 1 -- 172.21.15.131:0/4066665805 >> 172.21.15.131:0/4066665805 conn(0x7f05640675b0 msgr2=0x7f05640679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f056b5ff700 1 -- 172.21.15.131:0/4066665805 shutdown_connections 2023-12-26T21:49:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.005+0000 7f056b5ff700 1 -- 172.21.15.131:0/4066665805 wait complete. 2023-12-26T21:49:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.007+0000 7f056b5ff700 1 Processor -- start 2023-12-26T21:49:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.007+0000 7f056b5ff700 1 -- start start 2023-12-26T21:49:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f056b5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05640c8490 0x7f0564143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f056b5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0564156360 0x7f0564154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f056b5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0564156730 0x7f05641554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f056b5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056414c240 con 0x7f0564156730 2023-12-26T21:49:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f056b5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640c1590 con 0x7f0564156360 2023-12-26T21:49:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f056b5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05641439e0 con 0x7f05640c8490 2023-12-26T21:49:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0569b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0564156730 0x7f05641554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0569b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0564156730 0x7f05641554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38606/0 (socket says 172.21.15.131:38606) 2023-12-26T21:49:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0569b9c700 1 -- 172.21.15.131:0/1625353092 learned_addr learned my addr 172.21.15.131:0/1625353092 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f056939b700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05640c8490 0x7f0564143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0568b9a700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0564156360 0x7f0564154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0569b9c700 1 -- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05640c8490 msgr2=0x7f0564143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0569b9c700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05640c8490 0x7f0564143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0569b9c700 1 -- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0564156360 msgr2=0x7f0564154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0569b9c700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0564156360 0x7f0564154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0569b9c700 1 -- 172.21.15.131:0/1625353092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c009ae0 con 0x7f0564156730 2023-12-26T21:49:37.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f0568b9a700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0564156360 0x7f0564154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:49:37.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.008+0000 7f056939b700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05640c8490 0x7f0564143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:49:37.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.009+0000 7f0569b9c700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0564156730 0x7f05641554b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f055c009880 tx=0x7f055c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:37.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.009+0000 7f05597fa700 1 -- 172.21.15.131:0/1625353092 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c008bc0 con 0x7f0564156730 2023-12-26T21:49:37.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.009+0000 7f05597fa700 1 -- 172.21.15.131:0/1625353092 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c008d40 con 0x7f0564156730 2023-12-26T21:49:37.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.009+0000 7f05597fa700 1 -- 172.21.15.131:0/1625353092 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0266b0 con 0x7f0564156730 2023-12-26T21:49:37.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.009+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056414f520 con 0x7f0564156730 2023-12-26T21:49:37.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.009+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05640c1220 con 0x7f0564156730 2023-12-26T21:49:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.010+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056414f800 con 0x7f0564156730 2023-12-26T21:49:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.011+0000 7f05597fa700 1 -- 172.21.15.131:0/1625353092 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f055c003040 con 0x7f0564156730 2023-12-26T21:49:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.012+0000 7f05597fa700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0548067130 0x7f05480695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.012+0000 7f05597fa700 1 -- 172.21.15.131:0/1625353092 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6615+0+0 (secure 0 0 0) 0x7f055c086d40 con 0x7f0564156730 2023-12-26T21:49:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.012+0000 7f056939b700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0548067130 0x7f05480695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.012+0000 7f056939b700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0548067130 0x7f05480695e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0560006f00 tx=0x7f0560008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:37.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.016+0000 7f05597fa700 1 -- 172.21.15.131:0/1625353092 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f055c0538a0 con 0x7f0564156730 2023-12-26T21:49:37.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.236+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f056406b8c0 con 0x7f0548067130 2023-12-26T21:49:37.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.414+0000 7f05597fa700 1 -- 172.21.15.131:0/1625353092 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f056406b8c0 con 0x7f0548067130 2023-12-26T21:49:37.414 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0548067130 msgr2=0x7f05480695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0548067130 0x7f05480695e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0560006f00 tx=0x7f0560008040 comp rx=0 tx=0).stop 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0564156730 msgr2=0x7f05641554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0564156730 0x7f05641554b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f055c009880 tx=0x7f055c018000 comp rx=0 tx=0).stop 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 shutdown_connections 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0564156730 0x7f05641554b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0548067130 0x7f05480695e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05640c8490 0x7f0564143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 --2- 172.21.15.131:0/1625353092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0564156360 0x7f0564154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 >> 172.21.15.131:0/1625353092 conn(0x7f05640675b0 msgr2=0x7f05640c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 shutdown_connections 2023-12-26T21:49:37.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:37.417+0000 7f056b5ff700 1 -- 172.21.15.131:0/1625353092 wait complete. 2023-12-26T21:49:37.433 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-12-26T21:49:37.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.434 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-12-26T21:49:37.479 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:37.479 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:37.479 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:37.479 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:37.479 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:37.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:37.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:37.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:37.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:37.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:37.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:37.481 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:37.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.481 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_39_00_UTC) 2023-12-26T21:49:37.574 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:37.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.574 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:37.624 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:37.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.624 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:37.676 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:37.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.676 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:37.727 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:37.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.727 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:37.777 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:37.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.777 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:37.828 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:37.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.828 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:37.878 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:37.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.879 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:37.929 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:37.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.930 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:37.988 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:37.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:37.988 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:38.038 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:38.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:38.039 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:38.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:38.090 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-12-26T21:49:38.136 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-12-26T21:49:38.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.332+0000 7fd0587ff700 1 Processor -- start 2023-12-26T21:49:38.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.332+0000 7fd0587ff700 1 -- start start 2023-12-26T21:49:38.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.333+0000 7fd0587ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd0500c9110 0x7fd05015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:38.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.333+0000 7fd0587ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd050156e50 0x7fd0501572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:38.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.333+0000 7fd0587ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0500af040 con 0x7fd0500c8d40 2023-12-26T21:49:38.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.333+0000 7fd0587ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0500ae0e0 con 0x7fd050156e50 2023-12-26T21:49:38.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.333+0000 7fd0587ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0500ae3c0 con 0x7fd0500c9110 2023-12-26T21:49:38.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.333+0000 7fd056d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd050156e50 0x7fd0501572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:38.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd056d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd050156e50 0x7fd0501572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38616/0 (socket says 172.21.15.131:38616) 2023-12-26T21:49:38.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd056d9c700 1 -- 172.21.15.131:0/3994449969 learned_addr learned my addr 172.21.15.131:0/3994449969 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:38.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd055d9a700 1 --2- 172.21.15.131:0/3994449969 >> v2:172.21.15.190:3300/0 conn(0x7fd0500c9110 0x7fd05015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd056d9c700 1 -- 172.21.15.131:0/3994449969 >> v2:172.21.15.190:3300/0 conn(0x7fd0500c9110 msgr2=0x7fd05015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd056d9c700 1 --2- 172.21.15.131:0/3994449969 >> v2:172.21.15.190:3300/0 conn(0x7fd0500c9110 0x7fd05015bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd056d9c700 1 -- 172.21.15.131:0/3994449969 >> v1:172.21.15.190:6789/0 conn(0x7fd0500c8d40 legacy=0x7fd0500ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd056d9c700 1 -- 172.21.15.131:0/3994449969 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0500ae6a0 con 0x7fd050156e50 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd056d9c700 1 --2- 172.21.15.131:0/3994449969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 0x7fd0501572b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd05006fbb0 tx=0x7fd048009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0945b3a359a178 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd055599700 1 -- 172.21.15.131:0/3994449969 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04800e070 con 0x7fd050156e50 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd055599700 1 -- 172.21.15.131:0/3994449969 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04800ba20 con 0x7fd050156e50 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.334+0000 7fd055599700 1 -- 172.21.15.131:0/3994449969 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480092e0 con 0x7fd050156e50 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.335+0000 7fd0587ff700 1 -- 172.21.15.131:0/3994449969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 msgr2=0x7fd0501572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:38.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.335+0000 7fd0587ff700 1 --2- 172.21.15.131:0/3994449969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 0x7fd0501572b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd05006fbb0 tx=0x7fd048009840 comp rx=0 tx=0).stop 2023-12-26T21:49:38.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.335+0000 7fd0587ff700 1 -- 172.21.15.131:0/3994449969 shutdown_connections 2023-12-26T21:49:38.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.335+0000 7fd0587ff700 1 --2- 172.21.15.131:0/3994449969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 0x7fd0501572b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.335+0000 7fd0587ff700 1 --2- 172.21.15.131:0/3994449969 >> v2:172.21.15.190:3300/0 conn(0x7fd0500c9110 0x7fd05015bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.335+0000 7fd0587ff700 1 -- 172.21.15.131:0/3994449969 >> 172.21.15.131:0/3994449969 conn(0x7fd0500675b0 msgr2=0x7fd0500679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:38.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.335+0000 7fd0587ff700 1 -- 172.21.15.131:0/3994449969 shutdown_connections 2023-12-26T21:49:38.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.335+0000 7fd0587ff700 1 -- 172.21.15.131:0/3994449969 wait complete. 2023-12-26T21:49:38.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.337+0000 7fd0587ff700 1 Processor -- start 2023-12-26T21:49:38.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.337+0000 7fd0587ff700 1 -- start start 2023-12-26T21:49:38.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd0587ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0500c8d40 0x7fd050155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:38.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd0587ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0500c9110 0x7fd0500c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:38.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd0587ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 0x7fd0500aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:38.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd0587ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd050158250 con 0x7fd050156e50 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd0587ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0500c6b60 con 0x7fd0500c9110 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd0587ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0500c6e40 con 0x7fd0500c8d40 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd056d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 0x7fd0500aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd056d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 0x7fd0500aade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38628/0 (socket says 172.21.15.131:38628) 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd056d9c700 1 -- 172.21.15.131:0/3235241987 learned_addr learned my addr 172.21.15.131:0/3235241987 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd055d9a700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0500c9110 0x7fd0500c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd05659b700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0500c8d40 0x7fd050155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd056d9c700 1 -- 172.21.15.131:0/3235241987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0500c8d40 msgr2=0x7fd050155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd056d9c700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0500c8d40 0x7fd050155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd056d9c700 1 -- 172.21.15.131:0/3235241987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0500c9110 msgr2=0x7fd0500c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd056d9c700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0500c9110 0x7fd0500c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.338+0000 7fd056d9c700 1 -- 172.21.15.131:0/3235241987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048009ae0 con 0x7fd050156e50 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.339+0000 7fd056d9c700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 0x7fd0500aade0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd048009880 tx=0x7fd048018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.339+0000 7fd0467fc700 1 -- 172.21.15.131:0/3235241987 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048008a80 con 0x7fd050156e50 2023-12-26T21:49:38.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.339+0000 7fd0467fc700 1 -- 172.21.15.131:0/3235241987 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd048008c00 con 0x7fd050156e50 2023-12-26T21:49:38.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.339+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0500c7120 con 0x7fd050156e50 2023-12-26T21:49:38.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.339+0000 7fd0467fc700 1 -- 172.21.15.131:0/3235241987 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04801c410 con 0x7fd050156e50 2023-12-26T21:49:38.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.339+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd050155c50 con 0x7fd050156e50 2023-12-26T21:49:38.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.340+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0501583e0 con 0x7fd050156e50 2023-12-26T21:49:38.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.341+0000 7fd0467fc700 1 -- 172.21.15.131:0/3235241987 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fd048003040 con 0x7fd050156e50 2023-12-26T21:49:38.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.342+0000 7fd0467fc700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd034067130 0x7fd0340695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:38.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.342+0000 7fd0467fc700 1 -- 172.21.15.131:0/3235241987 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fd0480864a0 con 0x7fd050156e50 2023-12-26T21:49:38.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.343+0000 7fd05659b700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd034067130 0x7fd0340695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:38.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.343+0000 7fd05659b700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd034067130 0x7fd0340695e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd04c003a80 tx=0x7fd04c007ec0 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:38.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.346+0000 7fd0467fc700 1 -- 172.21.15.131:0/3235241987 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0501583e0 con 0x7fd050156e50 2023-12-26T21:49:38.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.599+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0501583e0 con 0x7fd034067130 2023-12-26T21:49:38.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.700+0000 7fd0467fc700 1 -- 172.21.15.131:0/3235241987 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fd0501583e0 con 0x7fd034067130 2023-12-26T21:49:38.699 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-12-26T21:49:38.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.702+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd034067130 msgr2=0x7fd0340695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:38.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.702+0000 7fd0587ff700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd034067130 0x7fd0340695e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd04c003a80 tx=0x7fd04c007ec0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.702+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 msgr2=0x7fd0500aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:38.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.702+0000 7fd0587ff700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 0x7fd0500aade0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd048009880 tx=0x7fd048018000 comp rx=0 tx=0).stop 2023-12-26T21:49:38.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.703+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 shutdown_connections 2023-12-26T21:49:38.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.703+0000 7fd0587ff700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd050156e50 0x7fd0500aade0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.703+0000 7fd0587ff700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd034067130 0x7fd0340695e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.703+0000 7fd0587ff700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0500c8d40 0x7fd050155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.703+0000 7fd0587ff700 1 --2- 172.21.15.131:0/3235241987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0500c9110 0x7fd0500c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:38.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.703+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 >> 172.21.15.131:0/3235241987 conn(0x7fd0500675b0 msgr2=0x7fd050156440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:38.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.703+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 shutdown_connections 2023-12-26T21:49:38.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:38.703+0000 7fd0587ff700 1 -- 172.21.15.131:0/3235241987 wait complete. 2023-12-26T21:49:38.720 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-12-26T21:49:38.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:38.720 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-12-26T21:49:38.765 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:38.765 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:38.765 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:38.768 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:38.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:38.768 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_39_00_UTC) 2023-12-26T21:49:38.861 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:38.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:38.861 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:38.911 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:38.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:38.911 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:38.972 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:38.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:38.973 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:39.025 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:39.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:39.025 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:39.074 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:39.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:39.075 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:39.124 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:39.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:39.125 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:39.200 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:39.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:39.201 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:39.263 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:39.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:39.263 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:39.316 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:39.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:39.316 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:39.380 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:39.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:39.380 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:39.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:39.430 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-12-26T21:49:39.485 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-12-26T21:49:39.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.702+0000 7fe4029ff700 1 Processor -- start 2023-12-26T21:49:39.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.703+0000 7fe4029ff700 1 -- start start 2023-12-26T21:49:39.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe4029ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe3fc0c9110 0x7fe3fc15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:39.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe4029ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc156e50 0x7fe3fc1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:39.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe4029ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3fc0af040 con 0x7fe3fc0c8d40 2023-12-26T21:49:39.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe4029ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3fc0ae0e0 con 0x7fe3fc0c9110 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe4029ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0ae3c0 con 0x7fe3fc156e50 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe3f3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe3fc0c9110 0x7fe3fc15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe3fbfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe3fc0c8d40 0x7fe3fc0ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45220/0 (socket says 172.21.15.131:45220) 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe3fbfff700 1 -- 172.21.15.131:0/1360495107 learned_addr learned my addr 172.21.15.131:0/1360495107 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/1360495107 >> v2:172.21.15.131:3300/0 conn(0x7fe3fc0c9110 0x7fe3fc15bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38632/0 (socket says 172.21.15.131:38632) 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe400f9c700 1 --2- 172.21.15.131:0/1360495107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc156e50 0x7fe3fc1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe3f3fff700 1 -- 172.21.15.131:0/1360495107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc156e50 msgr2=0x7fe3fc1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.704+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/1360495107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc156e50 0x7fe3fc1572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.705+0000 7fe3f3fff700 1 -- 172.21.15.131:0/1360495107 >> v1:172.21.15.131:6789/0 conn(0x7fe3fc0c8d40 legacy=0x7fe3fc0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.705+0000 7fe3f3fff700 1 -- 172.21.15.131:0/1360495107 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc0ae6a0 con 0x7fe3fc0c9110 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.705+0000 7fe3fb7fe700 1 -- 172.21.15.131:0/1360495107 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888185668 0 0) 0x7fe3fc0af040 con 0x7fe3fc0c8d40 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.705+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/1360495107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 0x7fe3fc15bc30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40098e0 tx=0x7fe3e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17721072259da6e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:39.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.705+0000 7fe3fb7fe700 1 -- 172.21.15.131:0/1360495107 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4009e80 con 0x7fe3fc0c9110 2023-12-26T21:49:39.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.705+0000 7fe3fb7fe700 1 -- 172.21.15.131:0/1360495107 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e400b860 con 0x7fe3fc0c9110 2023-12-26T21:49:39.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.705+0000 7fe3fb7fe700 1 -- 172.21.15.131:0/1360495107 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400bbe0 con 0x7fe3fc0c9110 2023-12-26T21:49:39.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.706+0000 7fe4029ff700 1 -- 172.21.15.131:0/1360495107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 msgr2=0x7fe3fc15bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:39.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.706+0000 7fe4029ff700 1 --2- 172.21.15.131:0/1360495107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 0x7fe3fc15bc30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40098e0 tx=0x7fe3e40091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:39.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.706+0000 7fe4029ff700 1 -- 172.21.15.131:0/1360495107 shutdown_connections 2023-12-26T21:49:39.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.706+0000 7fe4029ff700 1 --2- 172.21.15.131:0/1360495107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc156e50 0x7fe3fc1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:39.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.706+0000 7fe4029ff700 1 --2- 172.21.15.131:0/1360495107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 0x7fe3fc15bc30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:39.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.706+0000 7fe4029ff700 1 -- 172.21.15.131:0/1360495107 >> 172.21.15.131:0/1360495107 conn(0x7fe3fc0675b0 msgr2=0x7fe3fc0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:39.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.706+0000 7fe4029ff700 1 -- 172.21.15.131:0/1360495107 shutdown_connections 2023-12-26T21:49:39.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.706+0000 7fe4029ff700 1 -- 172.21.15.131:0/1360495107 wait complete. 2023-12-26T21:49:39.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.708+0000 7fe4029ff700 1 Processor -- start 2023-12-26T21:49:39.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.708+0000 7fe4029ff700 1 -- start start 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.709+0000 7fe4029ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc0c8d40 0x7fe3fc0c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.709+0000 7fe4029ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 0x7fe3fc0a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.709+0000 7fe4029ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3fc156e50 0x7fe3fc069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.709+0000 7fe4029ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc16ab70 con 0x7fe3fc0c9110 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.709+0000 7fe4029ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0c5e70 con 0x7fe3fc156e50 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.709+0000 7fe4029ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc16a240 con 0x7fe3fc0c8d40 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.709+0000 7fe3f3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 0x7fe3fc0a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 0x7fe3fc0a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38634/0 (socket says 172.21.15.131:38634) 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f3fff700 1 -- 172.21.15.131:0/3443767257 learned_addr learned my addr 172.21.15.131:0/3443767257 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3fbfff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc0c8d40 0x7fe3fc0c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:39.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe400f9c700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3fc156e50 0x7fe3fc069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f3fff700 1 -- 172.21.15.131:0/3443767257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc0c8d40 msgr2=0x7fe3fc0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc0c8d40 0x7fe3fc0c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f3fff700 1 -- 172.21.15.131:0/3443767257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3fc156e50 msgr2=0x7fe3fc069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3fc156e50 0x7fe3fc069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f3fff700 1 -- 172.21.15.131:0/3443767257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e4009420 con 0x7fe3fc0c9110 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f3fff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 0x7fe3fc0a98c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0c8a80 tx=0x7fe3e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f8ff9700 1 -- 172.21.15.131:0/3443767257 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400bbe0 con 0x7fe3fc0c9110 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f8ff9700 1 -- 172.21.15.131:0/3443767257 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4011420 con 0x7fe3fc0c9110 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe3f8ff9700 1 -- 172.21.15.131:0/3443767257 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40117e0 con 0x7fe3fc0c9110 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.710+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc16a520 con 0x7fe3fc0c9110 2023-12-26T21:49:39.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.711+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc169ef0 con 0x7fe3fc0c9110 2023-12-26T21:49:39.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.712+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc16ace0 con 0x7fe3fc0c9110 2023-12-26T21:49:39.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.712+0000 7fe3f8ff9700 1 -- 172.21.15.131:0/3443767257 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fe3e4011bd0 con 0x7fe3fc0c9110 2023-12-26T21:49:39.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.713+0000 7fe3f8ff9700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3dc067130 0x7fe3dc0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:39.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.713+0000 7fe3f8ff9700 1 -- 172.21.15.131:0/3443767257 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6963+0+0 (secure 0 0 0) 0x7fe3e40868c0 con 0x7fe3fc0c9110 2023-12-26T21:49:39.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.714+0000 7fe3fbfff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3dc067130 0x7fe3dc0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:39.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.714+0000 7fe3fbfff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3dc067130 0x7fe3dc0695e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0011e0 tx=0x7fe3ec009040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:39.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.718+0000 7fe3f8ff9700 1 -- 172.21.15.131:0/3443767257 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3e4053320 con 0x7fe3fc0c9110 2023-12-26T21:49:39.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:39.935+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3fc0717f0 con 0x7fe3dc067130 2023-12-26T21:49:40.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.014+0000 7fe3f8ff9700 1 -- 172.21.15.131:0/3443767257 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fe3fc0717f0 con 0x7fe3dc067130 2023-12-26T21:49:40.013 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3dc067130 msgr2=0x7fe3dc0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3dc067130 0x7fe3dc0695e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0011e0 tx=0x7fe3ec009040 comp rx=0 tx=0).stop 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 msgr2=0x7fe3fc0a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 0x7fe3fc0a98c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0c8a80 tx=0x7fe3e4017000 comp rx=0 tx=0).stop 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 shutdown_connections 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3fc156e50 0x7fe3fc069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe3dc067130 0x7fe3dc0695e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3fc0c8d40 0x7fe3fc0c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 --2- 172.21.15.131:0/3443767257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3fc0c9110 0x7fe3fc0a98c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 >> 172.21.15.131:0/3443767257 conn(0x7fe3fc0675b0 msgr2=0x7fe3fc1542f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 shutdown_connections 2023-12-26T21:49:40.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:40.016+0000 7fe4029ff700 1 -- 172.21.15.131:0/3443767257 wait complete. 2023-12-26T21:49:40.033 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-12-26T21:49:40.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.033 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:40.079 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:40.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.079 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_39_00_UTC) 2023-12-26T21:49:40.181 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:40.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.185 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:40.266 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:40.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.267 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:40.318 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:40.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.318 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:40.368 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:40.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.368 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:40.420 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:40.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.420 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:40.486 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:40.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.486 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:40.535 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:40.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.536 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:40.585 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:40.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.586 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:40.650 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:40.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.650 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:40.700 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:40.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.700 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:40.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:40.750 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-12-26T21:49:40.796 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-12-26T21:49:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.029+0000 7fa652bff700 1 Processor -- start 2023-12-26T21:49:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.029+0000 7fa652bff700 1 -- start start 2023-12-26T21:49:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.030+0000 7fa652bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa64c156360 0x7fa64c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.030+0000 7fa652bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 0x7fa64c06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.030+0000 7fa652bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c0acfc0 con 0x7fa64c156730 2023-12-26T21:49:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.030+0000 7fa652bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c06f1f0 con 0x7fa64c156360 2023-12-26T21:49:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.030+0000 7fa652bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c91f0 con 0x7fa64c0c8ba0 2023-12-26T21:49:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.030+0000 7fa65099b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa64c156360 0x7fa64c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.030+0000 7fa65099b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa64c156360 0x7fa64c06e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38640/0 (socket says 172.21.15.131:38640) 2023-12-26T21:49:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.030+0000 7fa65099b700 1 -- 172.21.15.131:0/2271456301 learned_addr learned my addr 172.21.15.131:0/2271456301 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa65119c700 1 --2- 172.21.15.131:0/2271456301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 0x7fa64c06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:41.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa65099b700 1 -- 172.21.15.131:0/2271456301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 msgr2=0x7fa64c06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:41.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa65099b700 1 --2- 172.21.15.131:0/2271456301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 0x7fa64c06ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa65099b700 1 -- 172.21.15.131:0/2271456301 >> v1:172.21.15.190:6789/0 conn(0x7fa64c156730 legacy=0x7fa64c0c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:41.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa65099b700 1 -- 172.21.15.131:0/2271456301 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0c94d0 con 0x7fa64c156360 2023-12-26T21:49:41.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa6437fe700 1 -- 172.21.15.131:0/2271456301 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700950597 0 0) 0x7fa64c0acfc0 con 0x7fa64c156730 2023-12-26T21:49:41.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa65099b700 1 --2- 172.21.15.131:0/2271456301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156360 0x7fa64c06e770 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa648009f20 tx=0x7fa648009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e75b0f867afd09 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:41.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa6437fe700 1 -- 172.21.15.131:0/2271456301 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64800e070 con 0x7fa64c156360 2023-12-26T21:49:41.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa6437fe700 1 -- 172.21.15.131:0/2271456301 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64800b9d0 con 0x7fa64c156360 2023-12-26T21:49:41.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.031+0000 7fa6437fe700 1 -- 172.21.15.131:0/2271456301 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6480092e0 con 0x7fa64c156360 2023-12-26T21:49:41.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.032+0000 7fa652bff700 1 -- 172.21.15.131:0/2271456301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156360 msgr2=0x7fa64c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:41.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.032+0000 7fa652bff700 1 --2- 172.21.15.131:0/2271456301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156360 0x7fa64c06e770 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa648009f20 tx=0x7fa648009840 comp rx=0 tx=0).stop 2023-12-26T21:49:41.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.032+0000 7fa652bff700 1 -- 172.21.15.131:0/2271456301 shutdown_connections 2023-12-26T21:49:41.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.032+0000 7fa652bff700 1 --2- 172.21.15.131:0/2271456301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 0x7fa64c06ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.032+0000 7fa652bff700 1 --2- 172.21.15.131:0/2271456301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156360 0x7fa64c06e770 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.032+0000 7fa652bff700 1 -- 172.21.15.131:0/2271456301 >> 172.21.15.131:0/2271456301 conn(0x7fa64c0675b0 msgr2=0x7fa64c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:41.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.032+0000 7fa652bff700 1 -- 172.21.15.131:0/2271456301 shutdown_connections 2023-12-26T21:49:41.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.032+0000 7fa652bff700 1 -- 172.21.15.131:0/2271456301 wait complete. 2023-12-26T21:49:41.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.034+0000 7fa652bff700 1 Processor -- start 2023-12-26T21:49:41.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.034+0000 7fa652bff700 1 -- start start 2023-12-26T21:49:41.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa652bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 0x7fa64c0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:41.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa652bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c156360 0x7fa64c1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa652bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156730 0x7fa64c0aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa652bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0aa280 con 0x7fa64c156730 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa652bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c06e970 con 0x7fa64c156360 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa652bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c0cf0 con 0x7fa64c0c8ba0 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65119c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156730 0x7fa64c0aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65119c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156730 0x7fa64c0aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38656/0 (socket says 172.21.15.131:38656) 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65119c700 1 -- 172.21.15.131:0/2554713748 learned_addr learned my addr 172.21.15.131:0/2554713748 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65099b700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 0x7fa64c0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa643fff700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c156360 0x7fa64c1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65119c700 1 -- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 msgr2=0x7fa64c0aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65119c700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 0x7fa64c0aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65119c700 1 -- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c156360 msgr2=0x7fa64c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65119c700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c156360 0x7fa64c1536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65119c700 1 -- 172.21.15.131:0/2554713748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648009ae0 con 0x7fa64c156730 2023-12-26T21:49:41.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa65099b700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 0x7fa64c0aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:49:41.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.035+0000 7fa643fff700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c156360 0x7fa64c1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:49:41.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.036+0000 7fa65119c700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156730 0x7fa64c0aaff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa64400e8d0 tx=0x7fa6440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:41.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.036+0000 7fa640ff9700 1 -- 172.21.15.131:0/2554713748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400ee70 con 0x7fa64c156730 2023-12-26T21:49:41.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.036+0000 7fa640ff9700 1 -- 172.21.15.131:0/2554713748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64400c850 con 0x7fa64c156730 2023-12-26T21:49:41.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.036+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c0aa4e0 con 0x7fa64c156730 2023-12-26T21:49:41.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.036+0000 7fa640ff9700 1 -- 172.21.15.131:0/2554713748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400cc50 con 0x7fa64c156730 2023-12-26T21:49:41.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.036+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c0c0f90 con 0x7fa64c156730 2023-12-26T21:49:41.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.037+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c0c0b50 con 0x7fa64c156730 2023-12-26T21:49:41.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.038+0000 7fa640ff9700 1 -- 172.21.15.131:0/2554713748 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fa644013680 con 0x7fa64c156730 2023-12-26T21:49:41.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.039+0000 7fa640ff9700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa638067130 0x7fa6380695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:41.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.039+0000 7fa640ff9700 1 -- 172.21.15.131:0/2554713748 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6647+0+0 (secure 0 0 0) 0x7fa644084ed0 con 0x7fa64c156730 2023-12-26T21:49:41.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.039+0000 7fa65099b700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa638067130 0x7fa6380695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:41.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.039+0000 7fa65099b700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa638067130 0x7fa6380695e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa648009880 tx=0x7fa648018000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:41.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.043+0000 7fa640ff9700 1 -- 172.21.15.131:0/2554713748 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa64c0c0b50 con 0x7fa64c156730 2023-12-26T21:49:41.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.271+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa64c0c0b50 con 0x7fa638067130 2023-12-26T21:49:41.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.353+0000 7fa640ff9700 1 -- 172.21.15.131:0/2554713748 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fa64c0c0b50 con 0x7fa638067130 2023-12-26T21:49:41.354 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-12-26T21:49:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa638067130 msgr2=0x7fa6380695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa638067130 0x7fa6380695e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa648009880 tx=0x7fa648018000 comp rx=0 tx=0).stop 2023-12-26T21:49:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156730 msgr2=0x7fa64c0aaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156730 0x7fa64c0aaff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa64400e8d0 tx=0x7fa6440091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 shutdown_connections 2023-12-26T21:49:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c156730 0x7fa64c0aaff0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fa638067130 0x7fa6380695e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64c0c8ba0 0x7fa64c0aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 --2- 172.21.15.131:0/2554713748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64c156360 0x7fa64c1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 >> 172.21.15.131:0/2554713748 conn(0x7fa64c0675b0 msgr2=0x7fa64c0ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 shutdown_connections 2023-12-26T21:49:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:41.356+0000 7fa652bff700 1 -- 172.21.15.131:0/2554713748 wait complete. 2023-12-26T21:49:41.374 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-12-26T21:49:41.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.374 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-12-26T21:49:41.419 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:41.419 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:41.419 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:41.421 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:41.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.421 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_39_00_UTC) 2023-12-26T21:49:41.513 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:41.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.513 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:41.563 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:41.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.564 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:41.614 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:41.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.615 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:41.665 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:41.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.665 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:41.714 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:41.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.715 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:41.764 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:41.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.764 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:41.816 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:41.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.817 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:41.865 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:41.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.866 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:41.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:41.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.916 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:41.966 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:41.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:41.967 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:42.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:42.017 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-12-26T21:49:42.064 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-12-26T21:49:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.288+0000 7f636bbff700 1 Processor -- start 2023-12-26T21:49:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.288+0000 7f636bbff700 1 -- start start 2023-12-26T21:49:42.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.289+0000 7f636bbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63640adcb0 0x7f63640abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:42.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.289+0000 7f636bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640ae080 0x7f6364069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.289+0000 7f636bbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63640c7430 con 0x7f6364069a60 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.289+0000 7f636bbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63640ac460 con 0x7f63640adcb0 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.289+0000 7f636bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6364069ee0 con 0x7f63640ae080 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.290+0000 7f636999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63640adcb0 0x7f63640abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.290+0000 7f636999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63640adcb0 0x7f63640abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38666/0 (socket says 172.21.15.131:38666) 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.290+0000 7f636999b700 1 -- 172.21.15.131:0/991329256 learned_addr learned my addr 172.21.15.131:0/991329256 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.290+0000 7f636919a700 1 --2- 172.21.15.131:0/991329256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640ae080 0x7f6364069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.290+0000 7f636999b700 1 -- 172.21.15.131:0/991329256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640ae080 msgr2=0x7f6364069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.290+0000 7f636999b700 1 --2- 172.21.15.131:0/991329256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640ae080 0x7f6364069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.290+0000 7f636999b700 1 -- 172.21.15.131:0/991329256 >> v1:172.21.15.190:6789/0 conn(0x7f6364069a60 legacy=0x7f636406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.290+0000 7f636999b700 1 -- 172.21.15.131:0/991329256 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63640c6820 con 0x7f63640adcb0 2023-12-26T21:49:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.290+0000 7f636999b700 1 --2- 172.21.15.131:0/991329256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63640adcb0 0x7f63640abf20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f63600099b0 tx=0x7f6360009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e3be95a65e532b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f6368999700 1 -- 172.21.15.131:0/991329256 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360009ea0 con 0x7f63640adcb0 2023-12-26T21:49:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f6368999700 1 -- 172.21.15.131:0/991329256 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636000b860 con 0x7f63640adcb0 2023-12-26T21:49:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f6368999700 1 -- 172.21.15.131:0/991329256 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000bc00 con 0x7f63640adcb0 2023-12-26T21:49:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f636bbff700 1 -- 172.21.15.131:0/991329256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63640adcb0 msgr2=0x7f63640abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f636bbff700 1 --2- 172.21.15.131:0/991329256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63640adcb0 0x7f63640abf20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f63600099b0 tx=0x7f6360009250 comp rx=0 tx=0).stop 2023-12-26T21:49:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f636bbff700 1 -- 172.21.15.131:0/991329256 shutdown_connections 2023-12-26T21:49:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f636bbff700 1 --2- 172.21.15.131:0/991329256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63640adcb0 0x7f63640abf20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f636bbff700 1 --2- 172.21.15.131:0/991329256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640ae080 0x7f6364069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f636bbff700 1 -- 172.21.15.131:0/991329256 >> 172.21.15.131:0/991329256 conn(0x7f63640675b0 msgr2=0x7f63640679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f636bbff700 1 -- 172.21.15.131:0/991329256 shutdown_connections 2023-12-26T21:49:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.291+0000 7f636bbff700 1 -- 172.21.15.131:0/991329256 wait complete. 2023-12-26T21:49:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.293+0000 7f636bbff700 1 Processor -- start 2023-12-26T21:49:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.293+0000 7f636bbff700 1 -- start start 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636bbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6364069a60 0x7f6364166530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640adcb0 0x7f63640ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636bbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63640ae080 0x7f63640ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636bbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63640c3380 con 0x7f6364069a60 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636bbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63640c3f00 con 0x7f63640ae080 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63640c35d0 con 0x7f63640adcb0 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636999b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6364069a60 0x7f6364166530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636999b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6364069a60 0x7f6364166530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38670/0 (socket says 172.21.15.131:38670) 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636999b700 1 -- 172.21.15.131:0/794637529 learned_addr learned my addr 172.21.15.131:0/794637529 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636919a700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640adcb0 0x7f63640ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636a19c700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63640ae080 0x7f63640ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636999b700 1 -- 172.21.15.131:0/794637529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640adcb0 msgr2=0x7f63640ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636999b700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640adcb0 0x7f63640ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636999b700 1 -- 172.21.15.131:0/794637529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63640ae080 msgr2=0x7f63640ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636999b700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63640ae080 0x7f63640ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636999b700 1 -- 172.21.15.131:0/794637529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600094f0 con 0x7f6364069a60 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.295+0000 7f636919a700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640adcb0 0x7f63640ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.296+0000 7f636999b700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6364069a60 0x7f6364166530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6364069080 tx=0x7f6360017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.296+0000 7f6359ffb700 1 -- 172.21.15.131:0/794637529 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000bbe0 con 0x7f6364069a60 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.296+0000 7f6359ffb700 1 -- 172.21.15.131:0/794637529 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6360011420 con 0x7f6364069a60 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.296+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63640c38b0 con 0x7f6364069a60 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.296+0000 7f6359ffb700 1 -- 172.21.15.131:0/794637529 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63600117c0 con 0x7f6364069a60 2023-12-26T21:49:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.296+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6364166a70 con 0x7f6364069a60 2023-12-26T21:49:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.297+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63640717f0 con 0x7f6364069a60 2023-12-26T21:49:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.298+0000 7f6359ffb700 1 -- 172.21.15.131:0/794637529 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7f6360011bf0 con 0x7f6364069a60 2023-12-26T21:49:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.298+0000 7f6359ffb700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6348067130 0x7f63480695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.299+0000 7f6359ffb700 1 -- 172.21.15.131:0/794637529 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6615+0+0 (secure 0 0 0) 0x7f63600862a0 con 0x7f6364069a60 2023-12-26T21:49:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.299+0000 7f636919a700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6348067130 0x7f63480695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.299+0000 7f636919a700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6348067130 0x7f63480695e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63640b1b40 tx=0x7f6354008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:42.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.303+0000 7f6359ffb700 1 -- 172.21.15.131:0/794637529 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6360052ee0 con 0x7f6364069a60 2023-12-26T21:49:42.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.548+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f63640c6b00 con 0x7f6348067130 2023-12-26T21:49:42.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.645+0000 7f6359ffb700 1 -- 172.21.15.131:0/794637529 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f63640c6b00 con 0x7f6348067130 2023-12-26T21:49:42.646 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6348067130 msgr2=0x7f63480695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6348067130 0x7f63480695e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63640b1b40 tx=0x7f6354008040 comp rx=0 tx=0).stop 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6364069a60 msgr2=0x7f6364166530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6364069a60 0x7f6364166530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6364069080 tx=0x7f6360017000 comp rx=0 tx=0).stop 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 shutdown_connections 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63640ae080 0x7f63640ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6364069a60 0x7f6364166530 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6348067130 0x7f63480695e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 --2- 172.21.15.131:0/794637529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63640adcb0 0x7f63640ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 >> 172.21.15.131:0/794637529 conn(0x7f63640675b0 msgr2=0x7f63640c78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:42.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 shutdown_connections 2023-12-26T21:49:42.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:42.647+0000 7f636bbff700 1 -- 172.21.15.131:0/794637529 wait complete. 2023-12-26T21:49:42.665 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-12-26T21:49:42.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:42.666 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-12-26T21:49:42.711 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:42.711 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:42.711 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:42.711 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:42.711 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:42.711 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:42.711 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:42.711 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:42.712 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:42.712 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:42.712 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:42.713 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_39_00_UTC 2023-12-26T21:49:42.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:42.713 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_39_00_UTC) 2023-12-26T21:49:42.805 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:42.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:42.805 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:42.855 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:42.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:42.855 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:42.905 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:42.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:42.905 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:42.956 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:42.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:42.956 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:43.010 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:43.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:43.011 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:43.063 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:43.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:43.063 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:43.113 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:43.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:43.114 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:43.164 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:43.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:43.164 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:43.249 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:43.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:43.250 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:43.305 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:43.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:43.306 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:43.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:43.356 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-12-26T21:49:43.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-12-26T21:49:43.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.628+0000 7ffa82dff700 1 Processor -- start 2023-12-26T21:49:43.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.628+0000 7ffa82dff700 1 -- start start 2023-12-26T21:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa82dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa7c140d20 0x7ffa7c0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa82dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa7c1410f0 0x7ffa7c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa82dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 0x7ffa7c0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa82dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c0ccdf0 con 0x7ffa7c1410f0 2023-12-26T21:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa82dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c0c9400 con 0x7ffa7c140d20 2023-12-26T21:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa82dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0cc1e0 con 0x7ffa7c0c8f80 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa7bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa7c1410f0 0x7ffa7c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa7bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa7c1410f0 0x7ffa7c0c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38672/0 (socket says 172.21.15.131:38672) 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa7bfff700 1 -- 172.21.15.131:0/2831162390 learned_addr learned my addr 172.21.15.131:0/2831162390 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa80b9b700 1 --2- 172.21.15.131:0/2831162390 >> v2:172.21.15.190:3300/0 conn(0x7ffa7c140d20 0x7ffa7c0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa8139c700 1 --2- 172.21.15.131:0/2831162390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 0x7ffa7c0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa7bfff700 1 -- 172.21.15.131:0/2831162390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 msgr2=0x7ffa7c0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa7bfff700 1 --2- 172.21.15.131:0/2831162390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 0x7ffa7c0b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa7bfff700 1 -- 172.21.15.131:0/2831162390 >> v2:172.21.15.190:3300/0 conn(0x7ffa7c140d20 msgr2=0x7ffa7c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa7bfff700 1 --2- 172.21.15.131:0/2831162390 >> v2:172.21.15.190:3300/0 conn(0x7ffa7c140d20 0x7ffa7c0cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.630+0000 7ffa7bfff700 1 -- 172.21.15.131:0/2831162390 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c0cc4c0 con 0x7ffa7c1410f0 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa7bfff700 1 --2- 172.21.15.131:0/2831162390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c1410f0 0x7ffa7c0c8a40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0cf4a0 tx=0x7ffa68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcba5b58ea9fc181 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa7b7fe700 1 -- 172.21.15.131:0/2831162390 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800b860 con 0x7ffa7c1410f0 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa7b7fe700 1 -- 172.21.15.131:0/2831162390 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6800b9c0 con 0x7ffa7c1410f0 2023-12-26T21:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa7b7fe700 1 -- 172.21.15.131:0/2831162390 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800bd60 con 0x7ffa7c1410f0 2023-12-26T21:49:43.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa82dff700 1 -- 172.21.15.131:0/2831162390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c1410f0 msgr2=0x7ffa7c0c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:43.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa82dff700 1 --2- 172.21.15.131:0/2831162390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c1410f0 0x7ffa7c0c8a40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0cf4a0 tx=0x7ffa68009250 comp rx=0 tx=0).stop 2023-12-26T21:49:43.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa82dff700 1 -- 172.21.15.131:0/2831162390 shutdown_connections 2023-12-26T21:49:43.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa82dff700 1 --2- 172.21.15.131:0/2831162390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 0x7ffa7c0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa82dff700 1 --2- 172.21.15.131:0/2831162390 >> v2:172.21.15.190:3300/0 conn(0x7ffa7c140d20 0x7ffa7c0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa82dff700 1 --2- 172.21.15.131:0/2831162390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c1410f0 0x7ffa7c0c8a40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.631+0000 7ffa82dff700 1 -- 172.21.15.131:0/2831162390 >> 172.21.15.131:0/2831162390 conn(0x7ffa7c063cd0 msgr2=0x7ffa7c06b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:43.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.632+0000 7ffa82dff700 1 -- 172.21.15.131:0/2831162390 shutdown_connections 2023-12-26T21:49:43.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.632+0000 7ffa82dff700 1 -- 172.21.15.131:0/2831162390 wait complete. 2023-12-26T21:49:43.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.634+0000 7ffa82dff700 1 Processor -- start 2023-12-26T21:49:43.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.634+0000 7ffa82dff700 1 -- start start 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa82dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 0x7ffa7c0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa82dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c140d20 0x7ffa7c0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa82dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa7c1410f0 0x7ffa7c0c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa82dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0a5f70 con 0x7ffa7c140d20 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa82dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0a5640 con 0x7ffa7c1410f0 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa82dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0a5920 con 0x7ffa7c0c8f80 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa7bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c140d20 0x7ffa7c0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa7bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c140d20 0x7ffa7c0c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38684/0 (socket says 172.21.15.131:38684) 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa7bfff700 1 -- 172.21.15.131:0/1750513688 learned_addr learned my addr 172.21.15.131:0/1750513688 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa80b9b700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 0x7ffa7c0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:43.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.635+0000 7ffa8139c700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa7c1410f0 0x7ffa7c0c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:43.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa7bfff700 1 -- 172.21.15.131:0/1750513688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 msgr2=0x7ffa7c0c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa7bfff700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 0x7ffa7c0c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa7bfff700 1 -- 172.21.15.131:0/1750513688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa7c1410f0 msgr2=0x7ffa7c0c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa7bfff700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa7c1410f0 0x7ffa7c0c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa7bfff700 1 -- 172.21.15.131:0/1750513688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa680094f0 con 0x7ffa7c140d20 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa7bfff700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c140d20 0x7ffa7c0c8370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c06f3d0 tx=0x7ffa68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa78ff9700 1 -- 172.21.15.131:0/1750513688 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68011e40 con 0x7ffa7c140d20 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa78ff9700 1 -- 172.21.15.131:0/1750513688 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6801a450 con 0x7ffa7c140d20 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7c0a5c00 con 0x7ffa7c140d20 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.636+0000 7ffa78ff9700 1 -- 172.21.15.131:0/1750513688 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6801a810 con 0x7ffa7c140d20 2023-12-26T21:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.637+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c167d70 con 0x7ffa7c140d20 2023-12-26T21:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.638+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7c0717f0 con 0x7ffa7c140d20 2023-12-26T21:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.638+0000 7ffa78ff9700 1 -- 172.21.15.131:0/1750513688 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7ffa6801abc0 con 0x7ffa7c140d20 2023-12-26T21:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.639+0000 7ffa78ff9700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffa60067130 0x7ffa600695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.639+0000 7ffa78ff9700 1 -- 172.21.15.131:0/1750513688 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6647+0+0 (secure 0 0 0) 0x7ffa68086740 con 0x7ffa7c140d20 2023-12-26T21:49:43.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.639+0000 7ffa80b9b700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffa60067130 0x7ffa600695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:43.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.640+0000 7ffa80b9b700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffa60067130 0x7ffa600695e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa70006fd0 tx=0x7ffa70008040 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:43.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.644+0000 7ffa78ff9700 1 -- 172.21.15.131:0/1750513688 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa680532e0 con 0x7ffa7c140d20 2023-12-26T21:49:43.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.862+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa7c0a60b0 con 0x7ffa60067130 2023-12-26T21:49:43.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.943+0000 7ffa78ff9700 1 -- 172.21.15.131:0/1750513688 <== mgr.5100 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7ffa700108d0 con 0x7ffa60067130 2023-12-26T21:49:43.943 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffa60067130 msgr2=0x7ffa600695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffa60067130 0x7ffa600695e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa70006fd0 tx=0x7ffa70008040 comp rx=0 tx=0).stop 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c140d20 msgr2=0x7ffa7c0c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c140d20 0x7ffa7c0c8370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c06f3d0 tx=0x7ffa68017000 comp rx=0 tx=0).stop 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 shutdown_connections 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa7c1410f0 0x7ffa7c0c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ffa60067130 0x7ffa600695e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa7c0c8f80 0x7ffa7c0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 --2- 172.21.15.131:0/1750513688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c140d20 0x7ffa7c0c8370 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 >> 172.21.15.131:0/1750513688 conn(0x7ffa7c063cd0 msgr2=0x7ffa7c0cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 shutdown_connections 2023-12-26T21:49:43.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:43.945+0000 7ffa82dff700 1 -- 172.21.15.131:0/1750513688 wait complete. 2023-12-26T21:49:43.963 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-12-26T21:49:43.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:43.964 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_39_01_UTC 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:44.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:44.010 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:44.010 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_39_01_UTC 2023-12-26T21:49:44.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.011 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_39_01_UTC) 2023-12-26T21:49:44.105 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_40_00_UTC 2023-12-26T21:49:44.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.105 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_40_00_UTC) 2023-12-26T21:49:44.155 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_41_00_UTC 2023-12-26T21:49:44.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.156 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_41_00_UTC) 2023-12-26T21:49:44.240 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_42_00_UTC 2023-12-26T21:49:44.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.240 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_42_00_UTC) 2023-12-26T21:49:44.291 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_43_00_UTC 2023-12-26T21:49:44.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.292 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_43_00_UTC) 2023-12-26T21:49:44.341 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_44_00_UTC 2023-12-26T21:49:44.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.342 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_44_00_UTC) 2023-12-26T21:49:44.392 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_45_00_UTC 2023-12-26T21:49:44.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.393 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_45_00_UTC) 2023-12-26T21:49:44.444 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_46_00_UTC 2023-12-26T21:49:44.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.444 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_46_00_UTC) 2023-12-26T21:49:44.494 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_47_00_UTC 2023-12-26T21:49:44.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.495 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_47_00_UTC) 2023-12-26T21:49:44.546 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_48_00_UTC 2023-12-26T21:49:44.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.546 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_48_00_UTC) 2023-12-26T21:49:44.596 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_49_00_UTC 2023-12-26T21:49:44.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.596 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-26-21_49_00_UTC) 2023-12-26T21:49:44.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:44.657 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-12-26T21:49:44.711 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T21:49:44.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.909+0000 7fe010bff700 1 Processor -- start 2023-12-26T21:49:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.909+0000 7fe010bff700 1 -- start start 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.910+0000 7fe010bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe00c0cb320 0x7fe00c069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.910+0000 7fe010bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c06a1e0 0x7fe00c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.910+0000 7fe010bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe00c156940 con 0x7fe00c0b0c90 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.910+0000 7fe010bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c0c8cc0 con 0x7fe00c0cb320 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.910+0000 7fe010bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0ad700 con 0x7fe00c06a1e0 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.910+0000 7fe00a59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe00c0cb320 0x7fe00c069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe00ad9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe00c0b0c90 0x7fe00c0aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40118/0 (socket says 172.21.15.131:40118) 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe00ad9d700 1 -- 172.21.15.131:0/402883048 learned_addr learned my addr 172.21.15.131:0/402883048 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe009d9b700 1 --2- 172.21.15.131:0/402883048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c06a1e0 0x7fe00c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe00a59c700 1 -- 172.21.15.131:0/402883048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c06a1e0 msgr2=0x7fe00c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe00a59c700 1 --2- 172.21.15.131:0/402883048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c06a1e0 0x7fe00c0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe00a59c700 1 -- 172.21.15.131:0/402883048 >> v1:172.21.15.131:6789/0 conn(0x7fe00c0b0c90 legacy=0x7fe00c0aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe00a59c700 1 -- 172.21.15.131:0/402883048 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c0ae500 con 0x7fe00c0cb320 2023-12-26T21:49:44.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe00959a700 1 -- 172.21.15.131:0/402883048 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293742255 0 0) 0x7fe00c156940 con 0x7fe00c0b0c90 2023-12-26T21:49:44.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe00a59c700 1 --2- 172.21.15.131:0/402883048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0cb320 0x7fe00c069ca0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0098e0 tx=0x7fdffc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e73d2a8b283914 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:44.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.911+0000 7fe00959a700 1 -- 172.21.15.131:0/402883048 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc009e80 con 0x7fe00c0cb320 2023-12-26T21:49:44.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe00959a700 1 -- 172.21.15.131:0/402883048 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc00b860 con 0x7fe00c0cb320 2023-12-26T21:49:44.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe00959a700 1 -- 172.21.15.131:0/402883048 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc00bbe0 con 0x7fe00c0cb320 2023-12-26T21:49:44.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe010bff700 1 -- 172.21.15.131:0/402883048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0cb320 msgr2=0x7fe00c069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:44.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe010bff700 1 --2- 172.21.15.131:0/402883048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0cb320 0x7fe00c069ca0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0098e0 tx=0x7fdffc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:44.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe010bff700 1 -- 172.21.15.131:0/402883048 shutdown_connections 2023-12-26T21:49:44.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe010bff700 1 --2- 172.21.15.131:0/402883048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0cb320 0x7fe00c069ca0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:44.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe010bff700 1 --2- 172.21.15.131:0/402883048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c06a1e0 0x7fe00c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:44.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe010bff700 1 -- 172.21.15.131:0/402883048 >> 172.21.15.131:0/402883048 conn(0x7fe00c068040 msgr2=0x7fe00c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:44.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe010bff700 1 -- 172.21.15.131:0/402883048 shutdown_connections 2023-12-26T21:49:44.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.912+0000 7fe010bff700 1 -- 172.21.15.131:0/402883048 wait complete. 2023-12-26T21:49:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.914+0000 7fe010bff700 1 Processor -- start 2023-12-26T21:49:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.914+0000 7fe010bff700 1 -- start start 2023-12-26T21:49:44.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe010bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe00c06a1e0 0x7fe00c143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe010bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0b0c90 0x7fe00c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe010bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c0cb320 0x7fe00c15d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe010bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c1558b0 con 0x7fe00c0b0c90 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe010bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0c6370 con 0x7fe00c06a1e0 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe010bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c155340 con 0x7fe00c0cb320 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe009d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0b0c90 0x7fe00c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe009d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0b0c90 0x7fe00c06def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60496/0 (socket says 172.21.15.131:60496) 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe009d9b700 1 -- 172.21.15.131:0/2199326207 learned_addr learned my addr 172.21.15.131:0/2199326207 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe00a59c700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe00c06a1e0 0x7fe00c143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe00ad9d700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c0cb320 0x7fe00c15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe009d9b700 1 -- 172.21.15.131:0/2199326207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c0cb320 msgr2=0x7fe00c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe009d9b700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c0cb320 0x7fe00c15d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe009d9b700 1 -- 172.21.15.131:0/2199326207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe00c06a1e0 msgr2=0x7fe00c143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe009d9b700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe00c06a1e0 0x7fe00c143830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:44.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.916+0000 7fe009d9b700 1 -- 172.21.15.131:0/2199326207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc009420 con 0x7fe00c0b0c90 2023-12-26T21:49:44.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.917+0000 7fe009d9b700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0b0c90 0x7fe00c06def0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe00000d8c0 tx=0x7fe000008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.917+0000 7fdffa7fc700 1 -- 172.21.15.131:0/2199326207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0000125f0 con 0x7fe00c0b0c90 2023-12-26T21:49:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.917+0000 7fdffa7fc700 1 -- 172.21.15.131:0/2199326207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000012750 con 0x7fe00c0b0c90 2023-12-26T21:49:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.917+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c0c5760 con 0x7fe00c0b0c90 2023-12-26T21:49:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.917+0000 7fdffa7fc700 1 -- 172.21.15.131:0/2199326207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000012af0 con 0x7fe00c0b0c90 2023-12-26T21:49:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.917+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c0c5d20 con 0x7fe00c0b0c90 2023-12-26T21:49:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.918+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00c0657e0 con 0x7fe00c0b0c90 2023-12-26T21:49:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.919+0000 7fdffa7fc700 1 -- 172.21.15.131:0/2199326207 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 84930+0+0 (secure 0 0 0) 0x7fe00001e460 con 0x7fe00c0b0c90 2023-12-26T21:49:44.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.920+0000 7fdffa7fc700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdfec067130 0x7fdfec0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:44.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.920+0000 7fdffa7fc700 1 -- 172.21.15.131:0/2199326207 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6947+0+0 (secure 0 0 0) 0x7fe0000852c0 con 0x7fe00c0b0c90 2023-12-26T21:49:44.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.920+0000 7fe00a59c700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdfec067130 0x7fdfec0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:44.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.921+0000 7fe00a59c700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdfec067130 0x7fdfec0695e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0091e0 tx=0x7fdffc017000 comp rx=0 tx=0).ready entity=mgr.5100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:44.924+0000 7fdffa7fc700 1 -- 172.21.15.131:0/2199326207 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe000051dd0 con 0x7fe00c0b0c90 2023-12-26T21:49:45.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:45.181+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fe00c0c64b0 con 0x7fe00c0b0c90 2023-12-26T21:49:46.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.174+0000 7fdffa7fc700 1 -- 172.21.15.131:0/2199326207 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7fe0000521a0 con 0x7fe00c0b0c90 2023-12-26T21:49:46.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.175+0000 7fdffa7fc700 1 -- 172.21.15.131:0/2199326207 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v14) v1 ==== 93+0+0 (secure 0 0 0) 0x7fe000051dd0 con 0x7fe00c0b0c90 2023-12-26T21:49:46.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdfec067130 msgr2=0x7fdfec0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:46.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdfec067130 0x7fdfec0695e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0091e0 tx=0x7fdffc017000 comp rx=0 tx=0).stop 2023-12-26T21:49:46.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0b0c90 msgr2=0x7fe00c06def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:46.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0b0c90 0x7fe00c06def0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe00000d8c0 tx=0x7fe000008040 comp rx=0 tx=0).stop 2023-12-26T21:49:46.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 shutdown_connections 2023-12-26T21:49:46.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdfec067130 0x7fdfec0695e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:46.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe00c0cb320 0x7fe00c15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe00c06a1e0 0x7fe00c143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 --2- 172.21.15.131:0/2199326207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe00c0b0c90 0x7fe00c06def0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.178+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 >> 172.21.15.131:0/2199326207 conn(0x7fe00c068040 msgr2=0x7fe00c157b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.179+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 shutdown_connections 2023-12-26T21:49:46.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.179+0000 7fe010bff700 1 -- 172.21.15.131:0/2199326207 wait complete. 2023-12-26T21:49:46.211 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:49:46.211 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:49:46.316 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:49:46.413 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:49:46.413 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T21:49:46.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:49:46.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T21:49:46.508 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:49:46.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:49:46.508 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T21:49:46.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:49:46.647 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:46.647+0000 7f178a0751c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:49:46.672 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:46.671+0000 7ff21e09c1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:49:46.752 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:46.752+0000 7f178a0751c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:49:46.780 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:46.780+0000 7ff21e09c1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:49:46.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.788+0000 7f5162fff700 1 Processor -- start 2023-12-26T21:49:46.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.788+0000 7f5162fff700 1 -- start start 2023-12-26T21:49:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.790+0000 7f5162fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f515c0b07f0 0x7f515c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.790+0000 7f5162fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f515c159cf0 0x7f515c155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.790+0000 7f5162fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515c0c8bf0 con 0x7f515c159920 2023-12-26T21:49:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.790+0000 7f5162fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515c155f50 con 0x7f515c0b07f0 2023-12-26T21:49:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.790+0000 7f5162fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515c0c8e60 con 0x7f515c159cf0 2023-12-26T21:49:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.790+0000 7f515bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f515c0b07f0 0x7f515c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:46.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.790+0000 7f515bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f515c0b07f0 0x7f515c1554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60526/0 (socket says 172.21.15.131:60526) 2023-12-26T21:49:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.790+0000 7f515bfff700 1 -- 172.21.15.131:0/3874563769 learned_addr learned my addr 172.21.15.131:0/3874563769 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.791+0000 7f516159c700 1 --2- 172.21.15.131:0/3874563769 >> v2:172.21.15.190:3300/0 conn(0x7f515c159cf0 0x7f515c155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.791+0000 7f515bfff700 1 -- 172.21.15.131:0/3874563769 >> v2:172.21.15.190:3300/0 conn(0x7f515c159cf0 msgr2=0x7f515c155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.791+0000 7f515bfff700 1 --2- 172.21.15.131:0/3874563769 >> v2:172.21.15.190:3300/0 conn(0x7f515c159cf0 0x7f515c155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.791+0000 7f515bfff700 1 -- 172.21.15.131:0/3874563769 >> v1:172.21.15.190:6789/0 conn(0x7f515c159920 legacy=0x7f515c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.791+0000 7f515bfff700 1 -- 172.21.15.131:0/3874563769 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c0c9140 con 0x7f515c0b07f0 2023-12-26T21:49:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.791+0000 7f515bfff700 1 --2- 172.21.15.131:0/3874563769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c0b07f0 0x7f515c1554d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f515c0b0e40 tx=0x7f5148009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7557433b92cfc47 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.791+0000 7f515b7fe700 1 -- 172.21.15.131:0/3874563769 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5148009e70 con 0x7f515c0b07f0 2023-12-26T21:49:46.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.791+0000 7f515b7fe700 1 -- 172.21.15.131:0/3874563769 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514800b860 con 0x7f515c0b07f0 2023-12-26T21:49:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.791+0000 7f515b7fe700 1 -- 172.21.15.131:0/3874563769 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514800bc00 con 0x7f515c0b07f0 2023-12-26T21:49:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.792+0000 7f5162fff700 1 -- 172.21.15.131:0/3874563769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c0b07f0 msgr2=0x7f515c1554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.792+0000 7f5162fff700 1 --2- 172.21.15.131:0/3874563769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c0b07f0 0x7f515c1554d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f515c0b0e40 tx=0x7f5148009250 comp rx=0 tx=0).stop 2023-12-26T21:49:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.792+0000 7f5162fff700 1 -- 172.21.15.131:0/3874563769 shutdown_connections 2023-12-26T21:49:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.792+0000 7f5162fff700 1 --2- 172.21.15.131:0/3874563769 >> v2:172.21.15.190:3300/0 conn(0x7f515c159cf0 0x7f515c155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.792+0000 7f5162fff700 1 --2- 172.21.15.131:0/3874563769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c0b07f0 0x7f515c1554d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:46.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.792+0000 7f5162fff700 1 -- 172.21.15.131:0/3874563769 >> 172.21.15.131:0/3874563769 conn(0x7f515c068040 msgr2=0x7f515c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:46.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.792+0000 7f5162fff700 1 -- 172.21.15.131:0/3874563769 shutdown_connections 2023-12-26T21:49:46.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.792+0000 7f5162fff700 1 -- 172.21.15.131:0/3874563769 wait complete. 2023-12-26T21:49:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.794+0000 7f5162fff700 1 Processor -- start 2023-12-26T21:49:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.794+0000 7f5162fff700 1 -- start start 2023-12-26T21:49:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f5162fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c159920 0x7f515c154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f5162fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c159cf0 0x7f515c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f5162fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f515c15e080 0x7f515c169c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f5162fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c15e520 con 0x7f515c159cf0 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f5162fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c157c40 con 0x7f515c159920 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f5162fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c154ee0 con 0x7f515c15e080 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f515bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c159cf0 0x7f515c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f515bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c159cf0 0x7f515c15db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60534/0 (socket says 172.21.15.131:60534) 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f515bfff700 1 -- 172.21.15.131:0/3760694936 learned_addr learned my addr 172.21.15.131:0/3760694936 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f516159c700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f515c15e080 0x7f515c169c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f5160d9b700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c159920 0x7f515c154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f515bfff700 1 -- 172.21.15.131:0/3760694936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f515c15e080 msgr2=0x7f515c169c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f515bfff700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f515c15e080 0x7f515c169c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f515bfff700 1 -- 172.21.15.131:0/3760694936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c159920 msgr2=0x7f515c154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f515bfff700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c159920 0x7f515c154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.795+0000 7f515bfff700 1 -- 172.21.15.131:0/3760694936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480094f0 con 0x7f515c159cf0 2023-12-26T21:49:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.796+0000 7f515bfff700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c159cf0 0x7f515c15db40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f515c0688f0 tx=0x7f5148017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.796+0000 7f5158ff9700 1 -- 172.21.15.131:0/3760694936 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514800bbe0 con 0x7f515c159cf0 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.796+0000 7f5158ff9700 1 -- 172.21.15.131:0/3760694936 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5148011420 con 0x7f515c159cf0 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.796+0000 7f5158ff9700 1 -- 172.21.15.131:0/3760694936 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51480117c0 con 0x7f515c159cf0 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.796+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c16a860 con 0x7f515c159cf0 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.796+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c1575f0 con 0x7f515c159cf0 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.797+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515c157df0 con 0x7f515c159cf0 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.798+0000 7f5158ff9700 1 -- 172.21.15.131:0/3760694936 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f5148011bb0 con 0x7f515c159cf0 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.799+0000 7f5158ff9700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51400670e0 0x7f5140069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.799+0000 7f5158ff9700 1 -- 172.21.15.131:0/3760694936 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f51480863c0 con 0x7f515c159cf0 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.799+0000 7f5160d9b700 1 -- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51400670e0 msgr2=0x7f5140069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.799+0000 7f5160d9b700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51400670e0 0x7f5140069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:46.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:46.803+0000 7f5158ff9700 1 -- 172.21.15.131:0/3760694936 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f515c157df0 con 0x7f515c159cf0 2023-12-26T21:49:46.842 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:46.842+0000 7f178a0751c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:49:46.875 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:46.874+0000 7ff21e09c1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:49:46.925 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:46.925+0000 7f178a0751c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:49:46.960 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:46.960+0000 7ff21e09c1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:49:47.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.000+0000 7f5160d9b700 1 -- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51400670e0 msgr2=0x7f5140069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:47.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.000+0000 7f5160d9b700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51400670e0 0x7f5140069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:47.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.049+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f515c157db0 con 0x7f515c159cf0 2023-12-26T21:49:47.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.050+0000 7f5158ff9700 1 -- 172.21.15.131:0/3760694936 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f5148011e60 con 0x7f515c159cf0 2023-12-26T21:49:47.051 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:47.051 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1410949426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1410949426},{"type":"v1","addr":"172.21.15.190:6835","nonce":1410949426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2210886653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2210886653},{"type":"v1","addr":"172.21.15.131:6836","nonce":2210886653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4878,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/241955638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":241955638},{"type":"v1","addr":"172.21.15.131:6837","nonce":241955638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:37:49.516027+0000","modified":"2023-12-26T21:37:53.065572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6833/3890149779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3890149779},{"type":"v1","addr":"172.21.15.190:6833","nonce":3890149779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-12-26T21:49:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.052+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51400670e0 msgr2=0x7f5140069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.052+0000 7f5162fff700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51400670e0 0x7f5140069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.052+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c159cf0 msgr2=0x7f515c15db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.052+0000 7f5162fff700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c159cf0 0x7f515c15db40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f515c0688f0 tx=0x7f5148017000 comp rx=0 tx=0).stop 2023-12-26T21:49:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.053+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 shutdown_connections 2023-12-26T21:49:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.053+0000 7f5162fff700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51400670e0 0x7f5140069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.053+0000 7f5162fff700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f515c15e080 0x7f515c169c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.053+0000 7f5162fff700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f515c159920 0x7f515c154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.053+0000 7f5162fff700 1 --2- 172.21.15.131:0/3760694936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f515c159cf0 0x7f515c15db40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.053+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 >> 172.21.15.131:0/3760694936 conn(0x7f515c068040 msgr2=0x7f515c0caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.053+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 shutdown_connections 2023-12-26T21:49:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.053+0000 7f5162fff700 1 -- 172.21.15.131:0/3760694936 wait complete. 2023-12-26T21:49:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2023-12-26T21:49:47.062 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:49:47.069 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:49:47.071 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:49:47.120 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:47.120+0000 7f178a0751c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:49:47.168 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:47.168+0000 7ff21e09c1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:49:47.295 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:47.295+0000 7f178a0751c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:49:47.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.310+0000 7f740f9ff700 1 Processor -- start 2023-12-26T21:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.310+0000 7f740f9ff700 1 -- start start 2023-12-26T21:49:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.311+0000 7f740f9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f74080cc580 0x7f74080cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.311+0000 7f740f9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74080b00e0 0x7f740806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.311+0000 7f740f9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74080b0d70 con 0x7f74080b0560 2023-12-26T21:49:47.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.311+0000 7f740f9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74080680f0 con 0x7f74080cc580 2023-12-26T21:49:47.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.311+0000 7f740f9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74080675b0 con 0x7f74080b00e0 2023-12-26T21:49:47.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.312+0000 7f740d79b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f74080cc580 0x7f74080cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:47.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.312+0000 7f740d79b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f74080cc580 0x7f74080cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60542/0 (socket says 172.21.15.131:60542) 2023-12-26T21:49:47.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.312+0000 7f740d79b700 1 -- 172.21.15.131:0/732874310 learned_addr learned my addr 172.21.15.131:0/732874310 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:47.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.312+0000 7f740cf9a700 1 --2- 172.21.15.131:0/732874310 >> v2:172.21.15.190:3300/0 conn(0x7f74080b00e0 0x7f740806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.312+0000 7f740d79b700 1 -- 172.21.15.131:0/732874310 >> v2:172.21.15.190:3300/0 conn(0x7f74080b00e0 msgr2=0x7f740806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.312+0000 7f740d79b700 1 --2- 172.21.15.131:0/732874310 >> v2:172.21.15.190:3300/0 conn(0x7f74080b00e0 0x7f740806e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.312+0000 7f740d79b700 1 -- 172.21.15.131:0/732874310 >> v1:172.21.15.190:6789/0 conn(0x7f74080b0560 legacy=0x7f740815d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.312+0000 7f740d79b700 1 -- 172.21.15.131:0/732874310 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408159c30 con 0x7f74080cc580 2023-12-26T21:49:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.312+0000 7f740d79b700 1 --2- 172.21.15.131:0/732874310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080cc580 0x7f74080cc950 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f740806a140 tx=0x7f7404009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528bc29b60fd2eb8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f73fffff700 1 -- 172.21.15.131:0/732874310 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404009e70 con 0x7f74080cc580 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f73fffff700 1 -- 172.21.15.131:0/732874310 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740400b860 con 0x7f74080cc580 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f73fffff700 1 -- 172.21.15.131:0/732874310 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400bc00 con 0x7f74080cc580 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f740f9ff700 1 -- 172.21.15.131:0/732874310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080cc580 msgr2=0x7f74080cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f740f9ff700 1 --2- 172.21.15.131:0/732874310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080cc580 0x7f74080cc950 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f740806a140 tx=0x7f7404009250 comp rx=0 tx=0).stop 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f740f9ff700 1 -- 172.21.15.131:0/732874310 shutdown_connections 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f740f9ff700 1 --2- 172.21.15.131:0/732874310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080cc580 0x7f74080cc950 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f740f9ff700 1 --2- 172.21.15.131:0/732874310 >> v2:172.21.15.190:3300/0 conn(0x7f74080b00e0 0x7f740806e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f740f9ff700 1 -- 172.21.15.131:0/732874310 >> 172.21.15.131:0/732874310 conn(0x7f74080686c0 msgr2=0x7f7408068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f740f9ff700 1 -- 172.21.15.131:0/732874310 shutdown_connections 2023-12-26T21:49:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.313+0000 7f740f9ff700 1 -- 172.21.15.131:0/732874310 wait complete. 2023-12-26T21:49:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.315+0000 7f740f9ff700 1 Processor -- start 2023-12-26T21:49:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.315+0000 7f740f9ff700 1 -- start start 2023-12-26T21:49:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740f9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b00e0 0x7f74080c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740f9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74080b0560 0x7f74081570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740f9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080cc580 0x7f740815e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740f9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080c2680 con 0x7f74080b00e0 2023-12-26T21:49:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740f9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7408157630 con 0x7f74080cc580 2023-12-26T21:49:47.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740f9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740815ea00 con 0x7f74080b0560 2023-12-26T21:49:47.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740d79b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b00e0 0x7f74080c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:47.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740d79b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b00e0 0x7f74080c20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60552/0 (socket says 172.21.15.131:60552) 2023-12-26T21:49:47.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740d79b700 1 -- 172.21.15.131:0/1341752118 learned_addr learned my addr 172.21.15.131:0/1341752118 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:47.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740cf9a700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74080b0560 0x7f74081570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:47.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740df9c700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080cc580 0x7f740815e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740d79b700 1 -- 172.21.15.131:0/1341752118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74080b0560 msgr2=0x7f74081570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740d79b700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74080b0560 0x7f74081570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740d79b700 1 -- 172.21.15.131:0/1341752118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080cc580 msgr2=0x7f740815e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740d79b700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080cc580 0x7f740815e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.316+0000 7f740d79b700 1 -- 172.21.15.131:0/1341752118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74040094f0 con 0x7f74080b00e0 2023-12-26T21:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.317+0000 7f740d79b700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b00e0 0x7f74080c20a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f740806b770 tx=0x7f7404017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.317+0000 7f73fd7fa700 1 -- 172.21.15.131:0/1341752118 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400bbe0 con 0x7f74080b00e0 2023-12-26T21:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.317+0000 7f73fd7fa700 1 -- 172.21.15.131:0/1341752118 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7404011420 con 0x7f74080b00e0 2023-12-26T21:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.317+0000 7f73fd7fa700 1 -- 172.21.15.131:0/1341752118 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040117e0 con 0x7f74080b00e0 2023-12-26T21:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.317+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740816a800 con 0x7f74080b00e0 2023-12-26T21:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.317+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740816adc0 con 0x7f74080b00e0 2023-12-26T21:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.318+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74080657e0 con 0x7f74080b00e0 2023-12-26T21:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.320+0000 7f73fd7fa700 1 -- 172.21.15.131:0/1341752118 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f7404011bd0 con 0x7f74080b00e0 2023-12-26T21:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.321+0000 7f73fd7fa700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73ec0670e0 0x7f73ec069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.321+0000 7f73fd7fa700 1 -- 172.21.15.131:0/1341752118 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f7404085fd0 con 0x7f74080b00e0 2023-12-26T21:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.321+0000 7f740cf9a700 1 -- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73ec0670e0 msgr2=0x7f73ec069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.321+0000 7f740cf9a700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73ec0670e0 0x7f73ec069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:47.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.325+0000 7f73fd7fa700 1 -- 172.21.15.131:0/1341752118 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7404052c80 con 0x7f74080b00e0 2023-12-26T21:49:47.350 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:47.349+0000 7ff21e09c1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:49:47.379 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:47.379+0000 7f178a0751c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:49:47.437 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:47.437+0000 7ff21e09c1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.522+0000 7f740cf9a700 1 -- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73ec0670e0 msgr2=0x7f73ec069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.522+0000 7f740cf9a700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73ec0670e0 0x7f73ec069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:47.525 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:47.525+0000 7f178a0751c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:49:47.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.526+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7408067850 con 0x7f74080b00e0 2023-12-26T21:49:47.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.527+0000 7f73fd7fa700 1 -- 172.21.15.131:0/1341752118 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16677 (secure 0 0 0) 0x7f74040562a0 con 0x7f74080b00e0 2023-12-26T21:49:47.528 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:47.528 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:49:46.168964+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:37:47.268950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-26T21:37:48.501427+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.23","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1288038854":"2023-12-27T21:38:05.306953+0000","172.21.15.131:0/518003331":"2023-12-27T21:38:05.306953+0000","172.21.15.131:6833/39288":"2023-12-27T21:38:05.306953+0000","172.21.15.131:6832/39288":"2023-12-27T21:38:05.306953+0000","172.21.15.131:0/2098522634":"2023-12-27T21:38:05.306953+0000","172.21.15.131:0/509990189":"2023-12-27T21:38:05.306953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":2},{"begin":177,"length":44}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":70,"length":1},{"begin":81,"length":1},{"begin":91,"length":1},{"begin":101,"length":1},{"begin":217,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:49:47.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.529+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73ec0670e0 msgr2=0x7f73ec069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:47.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73ec0670e0 0x7f73ec069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b00e0 msgr2=0x7f74080c20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b00e0 0x7f74080c20a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f740806b770 tx=0x7f7404017000 comp rx=0 tx=0).stop 2023-12-26T21:49:47.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 shutdown_connections 2023-12-26T21:49:47.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080cc580 0x7f740815e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b00e0 0x7f74080c20a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73ec0670e0 0x7f73ec069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 --2- 172.21.15.131:0/1341752118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74080b0560 0x7f74081570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 >> 172.21.15.131:0/1341752118 conn(0x7f74080686c0 msgr2=0x7f74080aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:47.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 shutdown_connections 2023-12-26T21:49:47.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.530+0000 7f740f9ff700 1 -- 172.21.15.131:0/1341752118 wait complete. 2023-12-26T21:49:47.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:37:47.268950+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:37:48.501427+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 50, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:49:47.553 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:49:47.597 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:47.597+0000 7ff21e09c1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:49:47.617 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:47.616+0000 7f178a0751c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:49:47.694 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:47.694+0000 7ff21e09c1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:49:47.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.755+0000 7f21aefff700 1 Processor -- start 2023-12-26T21:49:47.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.755+0000 7f21aefff700 1 -- start start 2023-12-26T21:49:47.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.756+0000 7f21aefff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f21a80c8bf0 0x7f21a80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.756+0000 7f21aefff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a80caa10 con 0x7f21a8069d10 2023-12-26T21:49:47.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.756+0000 7f21aefff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a8157a80 con 0x7f21a80c8820 2023-12-26T21:49:47.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.756+0000 7f21aefff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a80cac50 con 0x7f21a80c8bf0 2023-12-26T21:49:47.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.756+0000 7f21a7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f21a80c8bf0 0x7f21a80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.757+0000 7f21a7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f21a80c8bf0 0x7f21a80b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60564/0 (socket says 172.21.15.131:60564) 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.757+0000 7f21a7fff700 1 -- 172.21.15.131:0/1887112724 learned_addr learned my addr 172.21.15.131:0/1887112724 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.757+0000 7f21ad59c700 1 --1- 172.21.15.131:0/1887112724 >> v1:172.21.15.131:6789/0 conn(0x7f21a8069d10 0x7f21a8161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40132/0 (socket says 172.21.15.131:40132) 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.757+0000 7f21a7fff700 1 -- 172.21.15.131:0/1887112724 >> v1:172.21.15.190:6789/0 conn(0x7f21a80c8820 legacy=0x7f21a80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.757+0000 7f21a7fff700 1 -- 172.21.15.131:0/1887112724 >> v1:172.21.15.131:6789/0 conn(0x7f21a8069d10 legacy=0x7f21a8161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.757+0000 7f21a7fff700 1 -- 172.21.15.131:0/1887112724 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a80caf30 con 0x7f21a80c8bf0 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.757+0000 7f21a7fff700 1 --2- 172.21.15.131:0/1887112724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8bf0 0x7f21a80b12e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f21940099b0 tx=0x7f2194009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1488e27ffccd9c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.757+0000 7f21a77fe700 1 -- 172.21.15.131:0/1887112724 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219400b860 con 0x7f21a80c8bf0 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.758+0000 7f21a77fe700 1 -- 172.21.15.131:0/1887112724 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219400b9c0 con 0x7f21a80c8bf0 2023-12-26T21:49:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.758+0000 7f21a77fe700 1 -- 172.21.15.131:0/1887112724 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219400bd60 con 0x7f21a80c8bf0 2023-12-26T21:49:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.758+0000 7f21aefff700 1 -- 172.21.15.131:0/1887112724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8bf0 msgr2=0x7f21a80b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.758+0000 7f21aefff700 1 --2- 172.21.15.131:0/1887112724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8bf0 0x7f21a80b12e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f21940099b0 tx=0x7f2194009250 comp rx=0 tx=0).stop 2023-12-26T21:49:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.758+0000 7f21aefff700 1 -- 172.21.15.131:0/1887112724 shutdown_connections 2023-12-26T21:49:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.758+0000 7f21aefff700 1 --2- 172.21.15.131:0/1887112724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8bf0 0x7f21a80b12e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.758+0000 7f21aefff700 1 -- 172.21.15.131:0/1887112724 >> 172.21.15.131:0/1887112724 conn(0x7f21a8068040 msgr2=0x7f21a8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.758+0000 7f21aefff700 1 -- 172.21.15.131:0/1887112724 shutdown_connections 2023-12-26T21:49:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.758+0000 7f21aefff700 1 -- 172.21.15.131:0/1887112724 wait complete. 2023-12-26T21:49:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.760+0000 7f21aefff700 1 Processor -- start 2023-12-26T21:49:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.760+0000 7f21aefff700 1 -- start start 2023-12-26T21:49:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21aefff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a8069d10 0x7f21a80c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21aefff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8820 0x7f21a816a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21aefff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21a80c8bf0 0x7f21a80ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21aefff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a80b0f10 con 0x7f21a80c8820 2023-12-26T21:49:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21aefff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a80c75b0 con 0x7f21a8069d10 2023-12-26T21:49:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21aefff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a80c7890 con 0x7f21a80c8bf0 2023-12-26T21:49:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21a7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8820 0x7f21a816a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21a7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8820 0x7f21a816a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60572/0 (socket says 172.21.15.131:60572) 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21a7fff700 1 -- 172.21.15.131:0/3274286089 learned_addr learned my addr 172.21.15.131:0/3274286089 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21ad59c700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21a80c8bf0 0x7f21a80ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.761+0000 7f21acd9b700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a8069d10 0x7f21a80c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21a7fff700 1 -- 172.21.15.131:0/3274286089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21a80c8bf0 msgr2=0x7f21a80ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21a7fff700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21a80c8bf0 0x7f21a80ad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21a7fff700 1 -- 172.21.15.131:0/3274286089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a8069d10 msgr2=0x7f21a80c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21a7fff700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a8069d10 0x7f21a80c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21a7fff700 1 -- 172.21.15.131:0/3274286089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940094f0 con 0x7f21a80c8820 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21a7fff700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8820 0x7f21a816a210 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f21a80ae030 tx=0x7f2194017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21a4ff9700 1 -- 172.21.15.131:0/3274286089 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194011ea0 con 0x7f21a80c8820 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21a4ff9700 1 -- 172.21.15.131:0/3274286089 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219401a660 con 0x7f21a80c8820 2023-12-26T21:49:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a80c7b70 con 0x7f21a80c8820 2023-12-26T21:49:47.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21a4ff9700 1 -- 172.21.15.131:0/3274286089 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219401aa20 con 0x7f21a80c8820 2023-12-26T21:49:47.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.762+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a816a750 con 0x7f21a80c8820 2023-12-26T21:49:47.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.763+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a80657e0 con 0x7f21a80c8820 2023-12-26T21:49:47.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.764+0000 7f21a4ff9700 1 -- 172.21.15.131:0/3274286089 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f2194025440 con 0x7f21a80c8820 2023-12-26T21:49:47.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.766+0000 7f21a4ff9700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f218c0670e0 0x7f218c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:47.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.766+0000 7f21acd9b700 1 -- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f218c0670e0 msgr2=0x7f218c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:47.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.766+0000 7f21a4ff9700 1 -- 172.21.15.131:0/3274286089 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f21940868a0 con 0x7f21a80c8820 2023-12-26T21:49:47.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.766+0000 7f21acd9b700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f218c0670e0 0x7f218c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:47.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.770+0000 7f21a4ff9700 1 -- 172.21.15.131:0/3274286089 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2194053550 con 0x7f21a80c8820 2023-12-26T21:49:47.800 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:47.800+0000 7f178a0751c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:49:47.889 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:47.889+0000 7ff21e09c1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:49:47.948 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:47.948+0000 7f178a0751c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:49:47.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.967+0000 7f21acd9b700 1 -- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f218c0670e0 msgr2=0x7f218c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:47.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:47.967+0000 7f21acd9b700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f218c0670e0 0x7f218c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:48.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.005+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21a80c8010 con 0x7f21a80c8820 2023-12-26T21:49:48.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.005+0000 7f21a4ff9700 1 -- 172.21.15.131:0/3274286089 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2194056b70 con 0x7f21a80c8820 2023-12-26T21:49:48.006 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:49:48.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f218c0670e0 msgr2=0x7f218c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:48.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f218c0670e0 0x7f218c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8820 msgr2=0x7f21a816a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8820 0x7f21a816a210 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f21a80ae030 tx=0x7f2194017000 comp rx=0 tx=0).stop 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 shutdown_connections 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f218c0670e0 0x7f218c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21a80c8bf0 0x7f21a80ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a8069d10 0x7f21a80c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 --2- 172.21.15.131:0/3274286089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21a80c8820 0x7f21a816a210 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 >> 172.21.15.131:0/3274286089 conn(0x7f21a8068040 msgr2=0x7f21a806d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 shutdown_connections 2023-12-26T21:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.008+0000 7f21aefff700 1 -- 172.21.15.131:0/3274286089 wait complete. 2023-12-26T21:49:48.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:49:48.044 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:48.044+0000 7ff21e09c1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:49:48.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.240+0000 7fdf749ff700 1 Processor -- start 2023-12-26T21:49:48.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.240+0000 7fdf749ff700 1 -- start start 2023-12-26T21:49:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.241+0000 7fdf749ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.241+0000 7fdf749ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdf700b2a90 0x7fdf700b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.241+0000 7fdf749ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf700afcd0 con 0x7fdf701596a0 2023-12-26T21:49:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.241+0000 7fdf749ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf700b3e70 con 0x7fdf700b2a90 2023-12-26T21:49:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.241+0000 7fdf749ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b0f20 con 0x7fdf70159a70 2023-12-26T21:49:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.241+0000 7fdf6dd9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6ed9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdf700b2a90 0x7fdf700b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6dd9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:39546/0 (socket says 172.21.15.131:39546) 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6dd9b700 1 -- 172.21.15.131:0/1992500638 learned_addr learned my addr 172.21.15.131:0/1992500638 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6e59c700 1 --1- 172.21.15.131:0/1992500638 >> v1:172.21.15.190:6789/0 conn(0x7fdf701596a0 0x7fdf700b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:57884/0 (socket says 172.21.15.131:57884) 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6dd9b700 1 -- 172.21.15.131:0/1992500638 >> v2:172.21.15.190:3300/0 conn(0x7fdf700b2a90 msgr2=0x7fdf700b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6dd9b700 1 --2- 172.21.15.131:0/1992500638 >> v2:172.21.15.190:3300/0 conn(0x7fdf700b2a90 0x7fdf700b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6dd9b700 1 -- 172.21.15.131:0/1992500638 >> v1:172.21.15.190:6789/0 conn(0x7fdf701596a0 legacy=0x7fdf700b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6dd9b700 1 -- 172.21.15.131:0/1992500638 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700b3540 con 0x7fdf70159a70 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6ed9d700 1 --2- 172.21.15.131:0/1992500638 >> v2:172.21.15.190:3300/0 conn(0x7fdf700b2a90 0x7fdf700b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6dd9b700 1 --2- 172.21.15.131:0/1992500638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700b0160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf640099b0 tx=0x7fdf64009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff34fe2be4542 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.242+0000 7fdf6d59a700 1 -- 172.21.15.131:0/1992500638 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64009ea0 con 0x7fdf70159a70 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf6d59a700 1 -- 172.21.15.131:0/1992500638 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6400b860 con 0x7fdf70159a70 2023-12-26T21:49:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf6d59a700 1 -- 172.21.15.131:0/1992500638 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6400bc00 con 0x7fdf70159a70 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf749ff700 1 -- 172.21.15.131:0/1992500638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 msgr2=0x7fdf700b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf749ff700 1 --2- 172.21.15.131:0/1992500638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700b0160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf640099b0 tx=0x7fdf64009250 comp rx=0 tx=0).stop 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf749ff700 1 -- 172.21.15.131:0/1992500638 shutdown_connections 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf749ff700 1 --2- 172.21.15.131:0/1992500638 >> v2:172.21.15.190:3300/0 conn(0x7fdf700b2a90 0x7fdf700b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf749ff700 1 --2- 172.21.15.131:0/1992500638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700b0160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf749ff700 1 -- 172.21.15.131:0/1992500638 >> 172.21.15.131:0/1992500638 conn(0x7fdf7006ece0 msgr2=0x7fdf7006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf749ff700 1 -- 172.21.15.131:0/1992500638 shutdown_connections 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.243+0000 7fdf749ff700 1 -- 172.21.15.131:0/1992500638 wait complete. 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.245+0000 7fdf749ff700 1 Processor -- start 2023-12-26T21:49:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.245+0000 7fdf749ff700 1 -- start start 2023-12-26T21:49:48.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf749ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf700b2a90 0x7fdf7016e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf749ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf701596a0 0x7fdf7015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf749ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf749ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7016e700 con 0x7fdf700b2a90 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf749ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700c7a20 con 0x7fdf701596a0 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf749ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7015a420 con 0x7fdf70159a70 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf6e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf700b2a90 0x7fdf7016e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf6e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf700b2a90 0x7fdf7016e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60582/0 (socket says 172.21.15.131:60582) 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf6e59c700 1 -- 172.21.15.131:0/34719091 learned_addr learned my addr 172.21.15.131:0/34719091 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf6dd9b700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf701596a0 0x7fdf7015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.246+0000 7fdf6ed9d700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf6e59c700 1 -- 172.21.15.131:0/34719091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 msgr2=0x7fdf700c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf6e59c700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf6e59c700 1 -- 172.21.15.131:0/34719091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf701596a0 msgr2=0x7fdf7015b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf6e59c700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf701596a0 0x7fdf7015b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf6e59c700 1 -- 172.21.15.131:0/34719091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf68009a20 con 0x7fdf700b2a90 2023-12-26T21:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf6e59c700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf700b2a90 0x7fdf7016e1c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c7c0 tx=0x7fdf60007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf5e7fc700 1 -- 172.21.15.131:0/34719091 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60005a70 con 0x7fdf700b2a90 2023-12-26T21:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf5e7fc700 1 -- 172.21.15.131:0/34719091 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60005bd0 con 0x7fdf700b2a90 2023-12-26T21:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf640094f0 con 0x7fdf700b2a90 2023-12-26T21:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf5e7fc700 1 -- 172.21.15.131:0/34719091 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf600070d0 con 0x7fdf700b2a90 2023-12-26T21:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.247+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7015a9e0 con 0x7fdf700b2a90 2023-12-26T21:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.248+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf700c7d70 con 0x7fdf700b2a90 2023-12-26T21:49:48.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.249+0000 7fdf5e7fc700 1 -- 172.21.15.131:0/34719091 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7fdf60007480 con 0x7fdf700b2a90 2023-12-26T21:49:48.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.250+0000 7fdf5e7fc700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdf500670e0 0x7fdf50069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.250+0000 7fdf6dd9b700 1 -- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdf500670e0 msgr2=0x7fdf50069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:48.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.250+0000 7fdf5e7fc700 1 -- 172.21.15.131:0/34719091 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6535+0+0 (secure 0 0 0) 0x7fdf60083ff0 con 0x7fdf700b2a90 2023-12-26T21:49:48.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.250+0000 7fdf6dd9b700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdf500670e0 0x7fdf50069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:48.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.254+0000 7fdf5e7fc700 1 -- 172.21.15.131:0/34719091 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf60050d20 con 0x7fdf700b2a90 2023-12-26T21:49:48.332 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:48.332+0000 7f178a0751c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:49:48.450 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:48.450+0000 7ff21e09c1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:49:48.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.451+0000 7fdf6dd9b700 1 -- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdf500670e0 msgr2=0x7fdf50069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:48.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.451+0000 7fdf6dd9b700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdf500670e0 0x7fdf50069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:48.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.454+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf700657e0 con 0x7fdf700b2a90 2023-12-26T21:49:48.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.455+0000 7fdf5e7fc700 1 -- 172.21.15.131:0/34719091 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf60017020 con 0x7fdf700b2a90 2023-12-26T21:49:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:49:48.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdf500670e0 msgr2=0x7fdf50069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:48.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdf500670e0 0x7fdf50069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf700b2a90 msgr2=0x7fdf7016e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf700b2a90 0x7fdf7016e1c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c7c0 tx=0x7fdf60007f70 comp rx=0 tx=0).stop 2023-12-26T21:49:48.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 shutdown_connections 2023-12-26T21:49:48.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdf500670e0 0x7fdf50069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf70159a70 0x7fdf700c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf700b2a90 0x7fdf7016e1c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 --2- 172.21.15.131:0/34719091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf701596a0 0x7fdf7015b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 >> 172.21.15.131:0/34719091 conn(0x7fdf7006ece0 msgr2=0x7fdf700cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:48.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 shutdown_connections 2023-12-26T21:49:48.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.457+0000 7fdf749ff700 1 -- 172.21.15.131:0/34719091 wait complete. 2023-12-26T21:49:48.477 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:49:48.596 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:48.596+0000 7f178a0751c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:49:48.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.673+0000 7f4377fff700 1 Processor -- start 2023-12-26T21:49:48.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.673+0000 7f4377fff700 1 -- start start 2023-12-26T21:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.674+0000 7f4377fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f43701596a0 0x7f43700b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.674+0000 7f4377fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4370159a70 0x7f43700b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.674+0000 7f4377fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43700afcd0 con 0x7f43700b0160 2023-12-26T21:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.674+0000 7f4377fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43700b3e70 con 0x7f4370159a70 2023-12-26T21:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.674+0000 7f4377fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43700b2e00 con 0x7f43701596a0 2023-12-26T21:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.675+0000 7f437559a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4370159a70 0x7f43700b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.675+0000 7f437559a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4370159a70 0x7f43700b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60592/0 (socket says 172.21.15.131:60592) 2023-12-26T21:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.675+0000 7f437559a700 1 -- 172.21.15.131:0/2301936578 learned_addr learned my addr 172.21.15.131:0/2301936578 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:48.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.675+0000 7f4375d9b700 1 --2- 172.21.15.131:0/2301936578 >> v2:172.21.15.190:3300/0 conn(0x7f43701596a0 0x7f43700b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.675+0000 7f437559a700 1 -- 172.21.15.131:0/2301936578 >> v2:172.21.15.190:3300/0 conn(0x7f43701596a0 msgr2=0x7f43700b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.675+0000 7f437559a700 1 --2- 172.21.15.131:0/2301936578 >> v2:172.21.15.190:3300/0 conn(0x7f43701596a0 0x7f43700b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.675+0000 7f437559a700 1 -- 172.21.15.131:0/2301936578 >> v1:172.21.15.190:6789/0 conn(0x7f43700b0160 legacy=0x7f43700b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.675+0000 7f437559a700 1 -- 172.21.15.131:0/2301936578 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700b0cb0 con 0x7f4370159a70 2023-12-26T21:49:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.675+0000 7f437559a700 1 --2- 172.21.15.131:0/2301936578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4370159a70 0x7f43700b28c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f43700892a0 tx=0x7f4360009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34648f5b9a6dfbfd server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.676+0000 7f4374d99700 1 -- 172.21.15.131:0/2301936578 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360009e70 con 0x7f4370159a70 2023-12-26T21:49:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.676+0000 7f4374d99700 1 -- 172.21.15.131:0/2301936578 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436000b860 con 0x7f4370159a70 2023-12-26T21:49:48.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.676+0000 7f4374d99700 1 -- 172.21.15.131:0/2301936578 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000bc00 con 0x7f4370159a70 2023-12-26T21:49:48.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.676+0000 7f4377fff700 1 -- 172.21.15.131:0/2301936578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4370159a70 msgr2=0x7f43700b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.676+0000 7f4377fff700 1 --2- 172.21.15.131:0/2301936578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4370159a70 0x7f43700b28c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f43700892a0 tx=0x7f4360009250 comp rx=0 tx=0).stop 2023-12-26T21:49:48.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.676+0000 7f4377fff700 1 -- 172.21.15.131:0/2301936578 shutdown_connections 2023-12-26T21:49:48.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.676+0000 7f4377fff700 1 --2- 172.21.15.131:0/2301936578 >> v2:172.21.15.190:3300/0 conn(0x7f43701596a0 0x7f43700b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.676+0000 7f4377fff700 1 --2- 172.21.15.131:0/2301936578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4370159a70 0x7f43700b28c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.676+0000 7f4377fff700 1 -- 172.21.15.131:0/2301936578 >> 172.21.15.131:0/2301936578 conn(0x7f437006ece0 msgr2=0x7f437006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:48.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.677+0000 7f4377fff700 1 -- 172.21.15.131:0/2301936578 shutdown_connections 2023-12-26T21:49:48.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.677+0000 7f4377fff700 1 -- 172.21.15.131:0/2301936578 wait complete. 2023-12-26T21:49:48.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.678+0000 7f4377fff700 1 Processor -- start 2023-12-26T21:49:48.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.678+0000 7f4377fff700 1 -- start start 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f4377fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43700b0160 0x7f43700ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f4377fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43701596a0 0x7f437015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f4377fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4370159a70 0x7f437015edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f4377fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4370155b20 con 0x7f43701596a0 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f4377fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437015be20 con 0x7f4370159a70 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f4377fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437015c100 con 0x7f43700b0160 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f437559a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43701596a0 0x7f437015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f437559a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43701596a0 0x7f437015e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60604/0 (socket says 172.21.15.131:60604) 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f437559a700 1 -- 172.21.15.131:0/3012887314 learned_addr learned my addr 172.21.15.131:0/3012887314 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f437659c700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4370159a70 0x7f437015edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.679+0000 7f4375d9b700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43700b0160 0x7f43700ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f437559a700 1 -- 172.21.15.131:0/3012887314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43700b0160 msgr2=0x7f43700ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f437559a700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43700b0160 0x7f43700ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f437559a700 1 -- 172.21.15.131:0/3012887314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4370159a70 msgr2=0x7f437015edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f437559a700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4370159a70 0x7f437015edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f437559a700 1 -- 172.21.15.131:0/3012887314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43600094f0 con 0x7f43701596a0 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f4375d9b700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43700b0160 0x7f43700ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f437559a700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43701596a0 0x7f437015e880 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4370089710 tx=0x7f4360017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f4365ffb700 1 -- 172.21.15.131:0/3012887314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000b9d0 con 0x7f43701596a0 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f4365ffb700 1 -- 172.21.15.131:0/3012887314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436000bb30 con 0x7f43701596a0 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f4365ffb700 1 -- 172.21.15.131:0/3012887314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360025650 con 0x7f43701596a0 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437015c3e0 con 0x7f43701596a0 2023-12-26T21:49:48.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.680+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437015c9a0 con 0x7f43701596a0 2023-12-26T21:49:48.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.682+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43700657e0 con 0x7f43701596a0 2023-12-26T21:49:48.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.682+0000 7f4365ffb700 1 -- 172.21.15.131:0/3012887314 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f4360023070 con 0x7f43701596a0 2023-12-26T21:49:48.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.683+0000 7f4365ffb700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f43540670e0 0x7f4354069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:48.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.683+0000 7f4365ffb700 1 -- 172.21.15.131:0/3012887314 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f4360086150 con 0x7f43701596a0 2023-12-26T21:49:48.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.683+0000 7f4375d9b700 1 -- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f43540670e0 msgr2=0x7f4354069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:48.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.683+0000 7f4375d9b700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f43540670e0 0x7f4354069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:48.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.687+0000 7f4365ffb700 1 -- 172.21.15.131:0/3012887314 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4360052d60 con 0x7f43701596a0 2023-12-26T21:49:48.738 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:48.738+0000 7ff21e09c1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:49:48.875 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:48.875+0000 7f178a0751c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:49:48.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.884+0000 7f4375d9b700 1 -- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f43540670e0 msgr2=0x7f4354069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:48.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.884+0000 7f4375d9b700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f43540670e0 0x7f4354069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:48.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.888+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f437015cbb0 con 0x7f43701596a0 2023-12-26T21:49:48.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.888+0000 7f4365ffb700 1 -- 172.21.15.131:0/3012887314 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4360056380 con 0x7f43701596a0 2023-12-26T21:49:48.889 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f43540670e0 msgr2=0x7f4354069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f43540670e0 0x7f4354069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43701596a0 msgr2=0x7f437015e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43701596a0 0x7f437015e880 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4370089710 tx=0x7f4360017000 comp rx=0 tx=0).stop 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 shutdown_connections 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4370159a70 0x7f437015edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f43540670e0 0x7f4354069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43700b0160 0x7f43700ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 --2- 172.21.15.131:0/3012887314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43701596a0 0x7f437015e880 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 >> 172.21.15.131:0/3012887314 conn(0x7f437006ece0 msgr2=0x7f437015d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 shutdown_connections 2023-12-26T21:49:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:48.891+0000 7f4377fff700 1 -- 172.21.15.131:0/3012887314 wait complete. 2023-12-26T21:49:48.911 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:49:48.963 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:48.963+0000 7f178a0751c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:49:49.028 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:49.028+0000 7ff21e09c1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:49:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.105+0000 7f3a3e5ff700 1 Processor -- start 2023-12-26T21:49:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.105+0000 7f3a3e5ff700 1 -- start start 2023-12-26T21:49:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a3e5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 0x7f3a3806f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a3e5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3a3815ba50 0x7f3a38157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a3e5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3a38157f90 0x7f3a380b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a3e5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a38158af0 con 0x7f3a38157f90 2023-12-26T21:49:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a3e5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3815b5c0 con 0x7f3a3815ba50 2023-12-26T21:49:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a3e5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a38158530 con 0x7f3a3806f0e0 2023-12-26T21:49:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a3cb9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3a38157f90 0x7f3a380b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a377fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3a3815ba50 0x7f3a38157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a37fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 0x7f3a3806f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a3cb9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3a38157f90 0x7f3a380b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60618/0 (socket says 172.21.15.131:60618) 2023-12-26T21:49:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a3cb9c700 1 -- 172.21.15.131:0/769961879 learned_addr learned my addr 172.21.15.131:0/769961879 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a377fe700 1 -- 172.21.15.131:0/769961879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 msgr2=0x7f3a3806f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a377fe700 1 --2- 172.21.15.131:0/769961879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 0x7f3a3806f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a377fe700 1 -- 172.21.15.131:0/769961879 >> v2:172.21.15.131:3300/0 conn(0x7f3a38157f90 msgr2=0x7f3a380b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a377fe700 1 --2- 172.21.15.131:0/769961879 >> v2:172.21.15.131:3300/0 conn(0x7f3a38157f90 0x7f3a380b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.107+0000 7f3a377fe700 1 -- 172.21.15.131:0/769961879 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a38158810 con 0x7f3a3815ba50 2023-12-26T21:49:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.108+0000 7f3a377fe700 1 --2- 172.21.15.131:0/769961879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a3815ba50 0x7f3a38157a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a3806e000 tx=0x7f3a2c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2822700b3985288 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.108+0000 7f3a36ffd700 1 -- 172.21.15.131:0/769961879 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c011630 con 0x7f3a3815ba50 2023-12-26T21:49:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.108+0000 7f3a36ffd700 1 -- 172.21.15.131:0/769961879 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2c0117b0 con 0x7f3a3815ba50 2023-12-26T21:49:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.108+0000 7f3a36ffd700 1 -- 172.21.15.131:0/769961879 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c011b70 con 0x7f3a3815ba50 2023-12-26T21:49:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.109+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/769961879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a3815ba50 msgr2=0x7f3a38157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.109+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/769961879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a3815ba50 0x7f3a38157a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a3806e000 tx=0x7f3a2c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.109+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/769961879 shutdown_connections 2023-12-26T21:49:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.109+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/769961879 >> v2:172.21.15.131:3300/0 conn(0x7f3a38157f90 0x7f3a380b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.109+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/769961879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 0x7f3a3806f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.109+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/769961879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a3815ba50 0x7f3a38157a50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.109+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/769961879 >> 172.21.15.131:0/769961879 conn(0x7f3a380686c0 msgr2=0x7f3a38068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.109+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/769961879 shutdown_connections 2023-12-26T21:49:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.109+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/769961879 wait complete. 2023-12-26T21:49:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.110+0000 7f3a3e5ff700 1 Processor -- start 2023-12-26T21:49:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.111+0000 7f3a3e5ff700 1 -- start start 2023-12-26T21:49:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.111+0000 7f3a3e5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 0x7f3a380ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.111+0000 7f3a3e5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38157f90 0x7f3a380ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.111+0000 7f3a3e5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a3815ba50 0x7f3a38146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.111+0000 7f3a3e5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380adcd0 con 0x7f3a38157f90 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.111+0000 7f3a3e5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380af3d0 con 0x7f3a3815ba50 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.111+0000 7f3a3e5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a381468c0 con 0x7f3a3806f0e0 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38157f90 0x7f3a380ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38157f90 0x7f3a380ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60624/0 (socket says 172.21.15.131:60624) 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a377fe700 1 -- 172.21.15.131:0/174405058 learned_addr learned my addr 172.21.15.131:0/174405058 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a37fff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 0x7f3a380ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a3cb9c700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a3815ba50 0x7f3a38146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a377fe700 1 -- 172.21.15.131:0/174405058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 msgr2=0x7f3a380ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a377fe700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 0x7f3a380ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a377fe700 1 -- 172.21.15.131:0/174405058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a3815ba50 msgr2=0x7f3a38146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a377fe700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a3815ba50 0x7f3a38146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a377fe700 1 -- 172.21.15.131:0/174405058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c009420 con 0x7f3a38157f90 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.112+0000 7f3a377fe700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38157f90 0x7f3a380ad790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c013000 tx=0x7f3a2c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.113+0000 7f3a1ffff700 1 -- 172.21.15.131:0/174405058 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c01f070 con 0x7f3a38157f90 2023-12-26T21:49:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.113+0000 7f3a1ffff700 1 -- 172.21.15.131:0/174405058 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2c000bc0 con 0x7f3a38157f90 2023-12-26T21:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.113+0000 7f3a1ffff700 1 -- 172.21.15.131:0/174405058 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c011b50 con 0x7f3a38157f90 2023-12-26T21:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.113+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a380ae7c0 con 0x7f3a38157f90 2023-12-26T21:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.113+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a380aed80 con 0x7f3a38157f90 2023-12-26T21:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.114+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a380657e0 con 0x7f3a38157f90 2023-12-26T21:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.115+0000 7f3a1ffff700 1 -- 172.21.15.131:0/174405058 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f3a2c023070 con 0x7f3a38157f90 2023-12-26T21:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.116+0000 7f3a1ffff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3a240670e0 0x7f3a24069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.116+0000 7f3a1ffff700 1 -- 172.21.15.131:0/174405058 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f3a2c085d40 con 0x7f3a38157f90 2023-12-26T21:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.116+0000 7f3a37fff700 1 -- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3a240670e0 msgr2=0x7f3a24069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.116+0000 7f3a37fff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3a240670e0 0x7f3a24069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:49.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.120+0000 7f3a1ffff700 1 -- 172.21.15.131:0/174405058 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a2c0529f0 con 0x7f3a38157f90 2023-12-26T21:49:49.124 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:49.124+0000 7ff21e09c1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:49:49.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.317+0000 7f3a37fff700 1 -- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3a240670e0 msgr2=0x7f3a24069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:49.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.317+0000 7f3a37fff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3a240670e0 0x7f3a24069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:49.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.370+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a3815b250 con 0x7f3a38157f90 2023-12-26T21:49:49.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.370+0000 7f3a1ffff700 1 -- 172.21.15.131:0/174405058 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f3a2c02a090 con 0x7f3a38157f90 2023-12-26T21:49:49.371 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:49.371 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1410949426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1410949426},{"type":"v1","addr":"172.21.15.190:6835","nonce":1410949426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2210886653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2210886653},{"type":"v1","addr":"172.21.15.131:6836","nonce":2210886653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4878,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/241955638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":241955638},{"type":"v1","addr":"172.21.15.131:6837","nonce":241955638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:37:49.516027+0000","modified":"2023-12-26T21:37:53.065572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6833/3890149779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3890149779},{"type":"v1","addr":"172.21.15.190:6833","nonce":3890149779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-12-26T21:49:49.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3a240670e0 msgr2=0x7f3a24069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:49.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3a240670e0 0x7f3a24069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38157f90 msgr2=0x7f3a380ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:49.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38157f90 0x7f3a380ad790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c013000 tx=0x7f3a2c017000 comp rx=0 tx=0).stop 2023-12-26T21:49:49.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 shutdown_connections 2023-12-26T21:49:49.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a3815ba50 0x7f3a38146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3a240670e0 0x7f3a24069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a3806f0e0 0x7f3a380ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 --2- 172.21.15.131:0/174405058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38157f90 0x7f3a380ad790 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 >> 172.21.15.131:0/174405058 conn(0x7f3a380686c0 msgr2=0x7f3a3814db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:49.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 shutdown_connections 2023-12-26T21:49:49.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.373+0000 7f3a3e5ff700 1 -- 172.21.15.131:0/174405058 wait complete. 2023-12-26T21:49:49.375 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2023-12-26T21:49:49.393 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:49:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.592+0000 7f6dd1dff700 1 Processor -- start 2023-12-26T21:49:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.592+0000 7f6dd1dff700 1 -- start start 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.593+0000 7f6dd1dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc159820 0x7f6dcc0abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.593+0000 7f6dd1dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6dcc0aa480 0x7f6dcc06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.593+0000 7f6dd1dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc15a8c0 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.593+0000 7f6dd1dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc15ad50 con 0x7f6dcc0aa480 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.593+0000 7f6dd1dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc06e730 con 0x7f6dcc159820 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.594+0000 7f6dcaffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6dcc0aa480 0x7f6dcc06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.594+0000 7f6dcaffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6dcc0aa480 0x7f6dcc06dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60628/0 (socket says 172.21.15.131:60628) 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.594+0000 7f6dcaffd700 1 -- 172.21.15.131:0/1028588867 learned_addr learned my addr 172.21.15.131:0/1028588867 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.594+0000 7f6dcb7fe700 1 --2- 172.21.15.131:0/1028588867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc159820 0x7f6dcc0abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.594+0000 7f6dcaffd700 1 -- 172.21.15.131:0/1028588867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc159820 msgr2=0x7f6dcc0abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.594+0000 7f6dcaffd700 1 --2- 172.21.15.131:0/1028588867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc159820 0x7f6dcc0abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.594+0000 7f6dcaffd700 1 -- 172.21.15.131:0/1028588867 >> v1:172.21.15.131:6789/0 conn(0x7f6dcc06e2b0 legacy=0x7f6dcc15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.594+0000 7f6dcaffd700 1 -- 172.21.15.131:0/1028588867 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc159f90 con 0x7f6dcc0aa480 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.594+0000 7f6dcaffd700 1 --2- 172.21.15.131:0/1028588867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0aa480 0x7f6dcc06dd70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00099e0 tx=0x7f6dc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26533a892b03b136 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dca7fc700 1 -- 172.21.15.131:0/1028588867 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000b820 con 0x7f6dcc0aa480 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dca7fc700 1 -- 172.21.15.131:0/1028588867 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc000b980 con 0x7f6dcc0aa480 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dca7fc700 1 -- 172.21.15.131:0/1028588867 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000bd20 con 0x7f6dcc0aa480 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dd1dff700 1 -- 172.21.15.131:0/1028588867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0aa480 msgr2=0x7f6dcc06dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dd1dff700 1 --2- 172.21.15.131:0/1028588867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0aa480 0x7f6dcc06dd70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00099e0 tx=0x7f6dc0009250 comp rx=0 tx=0).stop 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dd1dff700 1 -- 172.21.15.131:0/1028588867 shutdown_connections 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dd1dff700 1 --2- 172.21.15.131:0/1028588867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc159820 0x7f6dcc0abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dd1dff700 1 --2- 172.21.15.131:0/1028588867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0aa480 0x7f6dcc06dd70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dd1dff700 1 -- 172.21.15.131:0/1028588867 >> 172.21.15.131:0/1028588867 conn(0x7f6dcc06c290 msgr2=0x7f6dcc0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dd1dff700 1 -- 172.21.15.131:0/1028588867 shutdown_connections 2023-12-26T21:49:49.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.595+0000 7f6dd1dff700 1 -- 172.21.15.131:0/1028588867 wait complete. 2023-12-26T21:49:49.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.597+0000 7f6dd1dff700 1 Processor -- start 2023-12-26T21:49:49.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.598+0000 7f6dd1dff700 1 -- start start 2023-12-26T21:49:49.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dd1dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc06e2b0 0x7f6dcc0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dd1dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc0aa480 0x7f6dcc0c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dd1dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc159820 0x7f6dcc0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dd1dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0c2890 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dd1dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0c3a90 con 0x7f6dcc159820 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dd1dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0c30c0 con 0x7f6dcc0aa480 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dcb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc06e2b0 0x7f6dcc0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dcb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc06e2b0 0x7f6dcc0c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60638/0 (socket says 172.21.15.131:60638) 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dcb7fe700 1 -- 172.21.15.131:0/4174924331 learned_addr learned my addr 172.21.15.131:0/4174924331 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dcaffd700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc0aa480 0x7f6dcc0c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.599+0000 7f6dcbfff700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc159820 0x7f6dcc0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6dcb7fe700 1 -- 172.21.15.131:0/4174924331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc0aa480 msgr2=0x7f6dcc0c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6dcb7fe700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc0aa480 0x7f6dcc0c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6dcb7fe700 1 -- 172.21.15.131:0/4174924331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc159820 msgr2=0x7f6dcc0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6dcb7fe700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc159820 0x7f6dcc0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6dcb7fe700 1 -- 172.21.15.131:0/4174924331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00094f0 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6dcaffd700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc0aa480 0x7f6dcc0c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:49.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6dcb7fe700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc06e2b0 0x7f6dcc0c3550 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00ecb0 tx=0x7f6dbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:49.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6db37fe700 1 -- 172.21.15.131:0/4174924331 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00c850 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6db37fe700 1 -- 172.21.15.131:0/4174924331 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc00c9b0 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6db37fe700 1 -- 172.21.15.131:0/4174924331 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00cd50 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc16ae60 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.600+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc0c2d50 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.602+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc0657e0 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.603+0000 7f6db37fe700 1 -- 172.21.15.131:0/4174924331 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f6dbc013680 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.603+0000 7f6db37fe700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6db80670e0 0x7f6db8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:49.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.604+0000 7f6db37fe700 1 -- 172.21.15.131:0/4174924331 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f6dbc084e30 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.604+0000 7f6dcaffd700 1 -- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6db80670e0 msgr2=0x7f6db8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:49.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.604+0000 7f6dcaffd700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6db80670e0 0x7f6db8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:49.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.608+0000 7f6db37fe700 1 -- 172.21.15.131:0/4174924331 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dbc051a40 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.704 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:49.703+0000 7f178a0751c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:49:49.796 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:49.796+0000 7f178a0751c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:49:49.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.805+0000 7f6dcaffd700 1 -- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6db80670e0 msgr2=0x7f6db8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:49.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.805+0000 7f6dcaffd700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6db80670e0 0x7f6db8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:49.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.822+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dcc15a270 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.823+0000 7f6db37fe700 1 -- 172.21.15.131:0/4174924331 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16677 (secure 0 0 0) 0x7f6dbc055060 con 0x7f6dcc06e2b0 2023-12-26T21:49:49.823 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:49.823 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:49:46.168964+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:37:47.268950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-26T21:37:48.501427+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.23","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1288038854":"2023-12-27T21:38:05.306953+0000","172.21.15.131:0/518003331":"2023-12-27T21:38:05.306953+0000","172.21.15.131:6833/39288":"2023-12-27T21:38:05.306953+0000","172.21.15.131:6832/39288":"2023-12-27T21:38:05.306953+0000","172.21.15.131:0/2098522634":"2023-12-27T21:38:05.306953+0000","172.21.15.131:0/509990189":"2023-12-27T21:38:05.306953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":2},{"begin":177,"length":44}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":70,"length":1},{"begin":81,"length":1},{"begin":91,"length":1},{"begin":101,"length":1},{"begin":217,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6db80670e0 msgr2=0x7f6db8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6db80670e0 0x7f6db8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc06e2b0 msgr2=0x7f6dcc0c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc06e2b0 0x7f6dcc0c3550 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00ecb0 tx=0x7f6dbc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 shutdown_connections 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc159820 0x7f6dcc0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc06e2b0 0x7f6dcc0c3550 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6db80670e0 0x7f6db8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 --2- 172.21.15.131:0/4174924331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc0aa480 0x7f6dcc0c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 >> 172.21.15.131:0/4174924331 conn(0x7f6dcc06c290 msgr2=0x7f6dcc0adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 shutdown_connections 2023-12-26T21:49:49.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:49.825+0000 7f6dd1dff700 1 -- 172.21.15.131:0/4174924331 wait complete. 2023-12-26T21:49:49.844 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T21:49:49.844 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:49:49.913 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:49.913+0000 7ff21e09c1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:49:49.949 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:49.949+0000 7f178a0751c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:49:50.011 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:50.011+0000 7ff21e09c1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:49:50.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.041+0000 7f6bc7fff700 1 Processor -- start 2023-12-26T21:49:50.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.042+0000 7f6bc7fff700 1 -- start start 2023-12-26T21:49:50.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.043+0000 7f6bc7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6bc00c8820 0x7f6bc00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.043+0000 7f6bc7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8bf0 0x7f6bc00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.043+0000 7f6bc7fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc0157a80 con 0x7f6bc0069d10 2023-12-26T21:49:50.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.043+0000 7f6bc7fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc00cac50 con 0x7f6bc00c8820 2023-12-26T21:49:50.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.043+0000 7f6bc7fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00caf30 con 0x7f6bc00c8bf0 2023-12-26T21:49:50.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.043+0000 7f6bc5d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6bc00c8820 0x7f6bc00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:50.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.043+0000 7f6bc5d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6bc00c8820 0x7f6bc00ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60640/0 (socket says 172.21.15.131:60640) 2023-12-26T21:49:50.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.043+0000 7f6bc5d9b700 1 -- 172.21.15.131:0/3409252699 learned_addr learned my addr 172.21.15.131:0/3409252699 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:50.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.043+0000 7f6bc559a700 1 --2- 172.21.15.131:0/3409252699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8bf0 0x7f6bc00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:50.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.044+0000 7f6bc5d9b700 1 -- 172.21.15.131:0/3409252699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8bf0 msgr2=0x7f6bc00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:50.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.044+0000 7f6bc5d9b700 1 --2- 172.21.15.131:0/3409252699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8bf0 0x7f6bc00b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.044+0000 7f6bc5d9b700 1 -- 172.21.15.131:0/3409252699 >> v1:172.21.15.131:6789/0 conn(0x7f6bc0069d10 legacy=0x7f6bc0157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.044+0000 7f6bc5d9b700 1 -- 172.21.15.131:0/3409252699 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00cb210 con 0x7f6bc00c8820 2023-12-26T21:49:50.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.044+0000 7f6bc5d9b700 1 --2- 172.21.15.131:0/3409252699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc00c8820 0x7f6bc00ca300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0099b0 tx=0x7f6bbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6558e812ca763d0c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.044+0000 7f6bc4d99700 1 -- 172.21.15.131:0/3409252699 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc009ea0 con 0x7f6bc00c8820 2023-12-26T21:49:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.044+0000 7f6bc4d99700 1 -- 172.21.15.131:0/3409252699 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bbc00b860 con 0x7f6bc00c8820 2023-12-26T21:49:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.044+0000 7f6bc4d99700 1 -- 172.21.15.131:0/3409252699 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00bc00 con 0x7f6bc00c8820 2023-12-26T21:49:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.045+0000 7f6bc7fff700 1 -- 172.21.15.131:0/3409252699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc00c8820 msgr2=0x7f6bc00ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.045+0000 7f6bc7fff700 1 --2- 172.21.15.131:0/3409252699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc00c8820 0x7f6bc00ca300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0099b0 tx=0x7f6bbc009250 comp rx=0 tx=0).stop 2023-12-26T21:49:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.045+0000 7f6bc7fff700 1 -- 172.21.15.131:0/3409252699 shutdown_connections 2023-12-26T21:49:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.045+0000 7f6bc7fff700 1 --2- 172.21.15.131:0/3409252699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc00c8820 0x7f6bc00ca300 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.045+0000 7f6bc7fff700 1 --2- 172.21.15.131:0/3409252699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8bf0 0x7f6bc00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.045+0000 7f6bc7fff700 1 -- 172.21.15.131:0/3409252699 >> 172.21.15.131:0/3409252699 conn(0x7f6bc0068040 msgr2=0x7f6bc0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.045+0000 7f6bc7fff700 1 -- 172.21.15.131:0/3409252699 shutdown_connections 2023-12-26T21:49:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.045+0000 7f6bc7fff700 1 -- 172.21.15.131:0/3409252699 wait complete. 2023-12-26T21:49:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.046+0000 7f6bc7fff700 1 Processor -- start 2023-12-26T21:49:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.047+0000 7f6bc7fff700 1 -- start start 2023-12-26T21:49:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.047+0000 7f6bc7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc0069d10 0x7f6bc015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.047+0000 7f6bc7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8820 0x7f6bc00c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.047+0000 7f6bc7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bc00c8bf0 0x7f6bc016aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.047+0000 7f6bc7fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc015c700 con 0x7f6bc0069d10 2023-12-26T21:49:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.047+0000 7f6bc7fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0151880 con 0x7f6bc00c8bf0 2023-12-26T21:49:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.047+0000 7f6bc7fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00c5760 con 0x7f6bc00c8820 2023-12-26T21:49:50.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc0069d10 0x7f6bc015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:50.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc0069d10 0x7f6bc015c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60642/0 (socket says 172.21.15.131:60642) 2023-12-26T21:49:50.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc5d9b700 1 -- 172.21.15.131:0/1729509612 learned_addr learned my addr 172.21.15.131:0/1729509612 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc659c700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bc00c8bf0 0x7f6bc016aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc559a700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8820 0x7f6bc00c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc5d9b700 1 -- 172.21.15.131:0/1729509612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8820 msgr2=0x7f6bc00c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc5d9b700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8820 0x7f6bc00c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc5d9b700 1 -- 172.21.15.131:0/1729509612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bc00c8bf0 msgr2=0x7f6bc016aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc5d9b700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bc00c8bf0 0x7f6bc016aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.048+0000 7f6bc5d9b700 1 -- 172.21.15.131:0/1729509612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0094f0 con 0x7f6bc0069d10 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.049+0000 7f6bc5d9b700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc0069d10 0x7f6bc015c1c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00688f0 tx=0x7f6bbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.049+0000 7f6bb5ffb700 1 -- 172.21.15.131:0/1729509612 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc000d10 con 0x7f6bc0069d10 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.049+0000 7f6bb5ffb700 1 -- 172.21.15.131:0/1729509612 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bbc00bbe0 con 0x7f6bc0069d10 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.049+0000 7f6bb5ffb700 1 -- 172.21.15.131:0/1729509612 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc011670 con 0x7f6bc0069d10 2023-12-26T21:49:50.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.049+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc00c5a40 con 0x7f6bc0069d10 2023-12-26T21:49:50.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.049+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc0151510 con 0x7f6bc0069d10 2023-12-26T21:49:50.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.050+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc00ca950 con 0x7f6bc0069d10 2023-12-26T21:49:50.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.051+0000 7f6bb5ffb700 1 -- 172.21.15.131:0/1729509612 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f6bbc011a60 con 0x7f6bc0069d10 2023-12-26T21:49:50.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.052+0000 7f6bb5ffb700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6ba40670e0 0x7f6ba4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.052+0000 7f6bb5ffb700 1 -- 172.21.15.131:0/1729509612 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f6bbc086350 con 0x7f6bc0069d10 2023-12-26T21:49:50.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.052+0000 7f6bc559a700 1 -- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6ba40670e0 msgr2=0x7f6ba4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:50.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.052+0000 7f6bc559a700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6ba40670e0 0x7f6ba4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.056+0000 7f6bb5ffb700 1 -- 172.21.15.131:0/1729509612 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bbc053000 con 0x7f6bc0069d10 2023-12-26T21:49:50.173 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:50.173+0000 7ff21e09c1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:49:50.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.253+0000 7f6bc559a700 1 -- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6ba40670e0 msgr2=0x7f6ba4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:50.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.253+0000 7f6bc559a700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6ba40670e0 0x7f6ba4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:50.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.309+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bc00657e0 con 0x7f6bc0069d10 2023-12-26T21:49:50.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.309+0000 7f6bb5ffb700 1 -- 172.21.15.131:0/1729509612 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f6bbc056620 con 0x7f6bc0069d10 2023-12-26T21:49:50.310 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:50.310 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1410949426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1410949426},{"type":"v1","addr":"172.21.15.190:6835","nonce":1410949426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2210886653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2210886653},{"type":"v1","addr":"172.21.15.131:6836","nonce":2210886653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4878,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/241955638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":241955638},{"type":"v1","addr":"172.21.15.131:6837","nonce":241955638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:37:49.516027+0000","modified":"2023-12-26T21:37:53.065572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6833/3890149779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3890149779},{"type":"v1","addr":"172.21.15.190:6833","nonce":3890149779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-12-26T21:49:50.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6ba40670e0 msgr2=0x7f6ba4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:50.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6ba40670e0 0x7f6ba4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc0069d10 msgr2=0x7f6bc015c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc0069d10 0x7f6bc015c1c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00688f0 tx=0x7f6bbc017000 comp rx=0 tx=0).stop 2023-12-26T21:49:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 shutdown_connections 2023-12-26T21:49:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bc00c8bf0 0x7f6bc016aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bc0069d10 0x7f6bc015c1c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6ba40670e0 0x7f6ba4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 --2- 172.21.15.131:0/1729509612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bc00c8820 0x7f6bc00c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 >> 172.21.15.131:0/1729509612 conn(0x7f6bc0068040 msgr2=0x7f6bc014f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 shutdown_connections 2023-12-26T21:49:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.312+0000 7f6bc7fff700 1 -- 172.21.15.131:0/1729509612 wait complete. 2023-12-26T21:49:50.314 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2023-12-26T21:49:50.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:49:50.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.524+0000 7f079b7ff700 1 Processor -- start 2023-12-26T21:49:50.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.524+0000 7f079b7ff700 1 -- start start 2023-12-26T21:49:50.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.525+0000 7f079b7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0794159920 0x7f07940b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f079b7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07940c8bf0 con 0x7f0794155be0 2023-12-26T21:49:50.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f079b7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07940c8e60 con 0x7f0794159cf0 2023-12-26T21:49:50.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f079b7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07940c9140 con 0x7f0794159920 2023-12-26T21:49:50.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f079959b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0794159920 0x7f07940b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:50.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f079959b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0794159920 0x7f07940b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60656/0 (socket says 172.21.15.131:60656) 2023-12-26T21:49:50.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f079959b700 1 -- 172.21.15.131:0/3062135558 learned_addr learned my addr 172.21.15.131:0/3062135558 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:50.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f079959b700 1 -- 172.21.15.131:0/3062135558 >> v1:172.21.15.190:6789/0 conn(0x7f0794159cf0 legacy=0x7f07941554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f079959b700 1 -- 172.21.15.131:0/3062135558 >> v1:172.21.15.131:6789/0 conn(0x7f0794155be0 legacy=0x7f07940c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f079959b700 1 -- 172.21.15.131:0/3062135558 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07940c9420 con 0x7f0794159920 2023-12-26T21:49:50.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.526+0000 7f078bfff700 1 -- 172.21.15.131:0/3062135558 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125060682 0 0) 0x7f07940c8e60 con 0x7f0794159cf0 2023-12-26T21:49:50.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.527+0000 7f079959b700 1 --2- 172.21.15.131:0/3062135558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794159920 0x7f07940b00e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f07900099b0 tx=0x7f0790009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1574ef828a5d5ae2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:50.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.527+0000 7f078bfff700 1 -- 172.21.15.131:0/3062135558 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000b860 con 0x7f0794159920 2023-12-26T21:49:50.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.527+0000 7f078bfff700 1 -- 172.21.15.131:0/3062135558 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079000b9c0 con 0x7f0794159920 2023-12-26T21:49:50.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.527+0000 7f078bfff700 1 -- 172.21.15.131:0/3062135558 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000bd60 con 0x7f0794159920 2023-12-26T21:49:50.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.528+0000 7f079b7ff700 1 -- 172.21.15.131:0/3062135558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794159920 msgr2=0x7f07940b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.528+0000 7f079b7ff700 1 --2- 172.21.15.131:0/3062135558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794159920 0x7f07940b00e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f07900099b0 tx=0x7f0790009250 comp rx=0 tx=0).stop 2023-12-26T21:49:50.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.528+0000 7f079b7ff700 1 -- 172.21.15.131:0/3062135558 shutdown_connections 2023-12-26T21:49:50.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.528+0000 7f079b7ff700 1 --2- 172.21.15.131:0/3062135558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794159920 0x7f07940b00e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.528+0000 7f079b7ff700 1 -- 172.21.15.131:0/3062135558 >> 172.21.15.131:0/3062135558 conn(0x7f0794068040 msgr2=0x7f0794069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:50.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.528+0000 7f079b7ff700 1 -- 172.21.15.131:0/3062135558 shutdown_connections 2023-12-26T21:49:50.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.528+0000 7f079b7ff700 1 -- 172.21.15.131:0/3062135558 wait complete. 2023-12-26T21:49:50.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.529+0000 7f079b7ff700 1 Processor -- start 2023-12-26T21:49:50.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.530+0000 7f079b7ff700 1 -- start start 2023-12-26T21:49:50.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.530+0000 7f079b7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794155be0 0x7f0794169c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.530+0000 7f079b7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0794159920 0x7f0794141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.530+0000 7f079b7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0794159cf0 0x7f0794158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.530+0000 7f079b7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0794150df0 con 0x7f0794155be0 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.530+0000 7f079b7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0794151040 con 0x7f0794159cf0 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f079b7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0794151320 con 0x7f0794159920 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f079959b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794155be0 0x7f0794169c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f079959b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794155be0 0x7f0794169c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60672/0 (socket says 172.21.15.131:60672) 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f079959b700 1 -- 172.21.15.131:0/1332788168 learned_addr learned my addr 172.21.15.131:0/1332788168 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f0799d9c700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0794159cf0 0x7f0794158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f0798d9a700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0794159920 0x7f0794141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f079959b700 1 -- 172.21.15.131:0/1332788168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0794159920 msgr2=0x7f0794141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f079959b700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0794159920 0x7f0794141b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f079959b700 1 -- 172.21.15.131:0/1332788168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0794159cf0 msgr2=0x7f0794158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f079959b700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0794159cf0 0x7f0794158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.531+0000 7f079959b700 1 -- 172.21.15.131:0/1332788168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07900094f0 con 0x7f0794155be0 2023-12-26T21:49:50.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.532+0000 7f079959b700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794155be0 0x7f0794169c80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0790006f50 tx=0x7f0790017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:50.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.532+0000 7f07897fa700 1 -- 172.21.15.131:0/1332788168 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079001f070 con 0x7f0794155be0 2023-12-26T21:49:50.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.532+0000 7f07897fa700 1 -- 172.21.15.131:0/1332788168 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790011bb0 con 0x7f0794155be0 2023-12-26T21:49:50.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.532+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0794151600 con 0x7f0794155be0 2023-12-26T21:49:50.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.532+0000 7f07897fa700 1 -- 172.21.15.131:0/1332788168 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790025420 con 0x7f0794155be0 2023-12-26T21:49:50.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.532+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079416a1c0 con 0x7f0794155be0 2023-12-26T21:49:50.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.533+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07940657e0 con 0x7f0794155be0 2023-12-26T21:49:50.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.534+0000 7f07897fa700 1 -- 172.21.15.131:0/1332788168 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f0790023070 con 0x7f0794155be0 2023-12-26T21:49:50.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.535+0000 7f07897fa700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f07780670e0 0x7f0778069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:50.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.535+0000 7f07897fa700 1 -- 172.21.15.131:0/1332788168 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f0790086180 con 0x7f0794155be0 2023-12-26T21:49:50.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.535+0000 7f0798d9a700 1 -- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f07780670e0 msgr2=0x7f0778069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:50.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.535+0000 7f0798d9a700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f07780670e0 0x7f0778069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:50.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.539+0000 7f07897fa700 1 -- 172.21.15.131:0/1332788168 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0790052dc0 con 0x7f0794155be0 2023-12-26T21:49:50.583 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:50.583+0000 7f178a0751c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:49:50.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.736+0000 7f0798d9a700 1 -- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f07780670e0 msgr2=0x7f0778069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:50.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.736+0000 7f0798d9a700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f07780670e0 0x7f0778069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:50.775 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:50.775+0000 7f178a0751c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:49:50.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.786+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0794151b20 con 0x7f0794155be0 2023-12-26T21:49:50.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.787+0000 7f07897fa700 1 -- 172.21.15.131:0/1332788168 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f0794151b20 con 0x7f0794155be0 2023-12-26T21:49:50.787 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:50.787 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1410949426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1410949426},{"type":"v1","addr":"172.21.15.190:6835","nonce":1410949426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2210886653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2210886653},{"type":"v1","addr":"172.21.15.131:6836","nonce":2210886653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4878,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/241955638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":241955638},{"type":"v1","addr":"172.21.15.131:6837","nonce":241955638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:37:49.516027+0000","modified":"2023-12-26T21:37:53.065572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6833/3890149779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3890149779},{"type":"v1","addr":"172.21.15.190:6833","nonce":3890149779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f07780670e0 msgr2=0x7f0778069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f07780670e0 0x7f0778069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794155be0 msgr2=0x7f0794169c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794155be0 0x7f0794169c80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0790006f50 tx=0x7f0790017000 comp rx=0 tx=0).stop 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 shutdown_connections 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0794159cf0 0x7f0794158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0794155be0 0x7f0794169c80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f07780670e0 0x7f0778069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 --2- 172.21.15.131:0/1332788168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0794159920 0x7f0794141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 >> 172.21.15.131:0/1332788168 conn(0x7f0794068040 msgr2=0x7f079406d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:50.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 shutdown_connections 2023-12-26T21:49:50.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:50.789+0000 7f079b7ff700 1 -- 172.21.15.131:0/1332788168 wait complete. 2023-12-26T21:49:50.791 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2023-12-26T21:49:50.808 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:49:50.838 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:50.837+0000 7ff21e09c1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:49:50.962 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:50.962+0000 7f178a0751c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:49:51.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.002+0000 7f5008bff700 1 Processor -- start 2023-12-26T21:49:51.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.002+0000 7f5008bff700 1 -- start start 2023-12-26T21:49:51.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.003+0000 7f5008bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5004159930 0x7f500415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.003+0000 7f5008bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f500406a1a0 0x7f5004068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5008bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5004068a80 con 0x7f5004159560 2023-12-26T21:49:51.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5008bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50040cac50 con 0x7f5004159930 2023-12-26T21:49:51.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5008bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500415da00 con 0x7f500406a1a0 2023-12-26T21:49:51.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5001d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5004159930 0x7f500415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5001d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5004159930 0x7f500415d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60682/0 (socket says 172.21.15.131:60682) 2023-12-26T21:49:51.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5001d9b700 1 -- 172.21.15.131:0/1775314091 learned_addr learned my addr 172.21.15.131:0/1775314091 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:51.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f500259c700 1 --1- 172.21.15.131:0/1775314091 >> v1:172.21.15.190:6789/0 conn(0x7f5004159560 0x7f50040b11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:57904/0 (socket says 172.21.15.131:57904) 2023-12-26T21:49:51.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5002d9d700 1 --2- 172.21.15.131:0/1775314091 >> v2:172.21.15.190:3300/0 conn(0x7f500406a1a0 0x7f5004068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5001d9b700 1 -- 172.21.15.131:0/1775314091 >> v2:172.21.15.190:3300/0 conn(0x7f500406a1a0 msgr2=0x7f5004068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:51.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5001d9b700 1 --2- 172.21.15.131:0/1775314091 >> v2:172.21.15.190:3300/0 conn(0x7f500406a1a0 0x7f5004068f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5001d9b700 1 -- 172.21.15.131:0/1775314091 >> v1:172.21.15.190:6789/0 conn(0x7f5004159560 legacy=0x7f50040b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f500159a700 1 -- 172.21.15.131:0/1775314091 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700796702 0 0) 0x7f5004068a80 con 0x7f5004159560 2023-12-26T21:49:51.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.004+0000 7f5001d9b700 1 -- 172.21.15.131:0/1775314091 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500415dce0 con 0x7f5004159930 2023-12-26T21:49:51.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.005+0000 7f5001d9b700 1 --2- 172.21.15.131:0/1775314091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 0x7f500415d300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f50040c1dd0 tx=0x7f4ff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd63717121cfb55c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:51.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.005+0000 7f500159a700 1 -- 172.21.15.131:0/1775314091 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8009e70 con 0x7f5004159930 2023-12-26T21:49:51.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.005+0000 7f500159a700 1 -- 172.21.15.131:0/1775314091 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff800b860 con 0x7f5004159930 2023-12-26T21:49:51.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.005+0000 7f500159a700 1 -- 172.21.15.131:0/1775314091 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800bc00 con 0x7f5004159930 2023-12-26T21:49:51.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.005+0000 7f5008bff700 1 -- 172.21.15.131:0/1775314091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 msgr2=0x7f500415d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.005+0000 7f5008bff700 1 --2- 172.21.15.131:0/1775314091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 0x7f500415d300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f50040c1dd0 tx=0x7f4ff8009250 comp rx=0 tx=0).stop 2023-12-26T21:49:51.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.006+0000 7f5008bff700 1 -- 172.21.15.131:0/1775314091 shutdown_connections 2023-12-26T21:49:51.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.006+0000 7f5008bff700 1 --2- 172.21.15.131:0/1775314091 >> v2:172.21.15.190:3300/0 conn(0x7f500406a1a0 0x7f5004068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.006+0000 7f5008bff700 1 --2- 172.21.15.131:0/1775314091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 0x7f500415d300 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.006+0000 7f5008bff700 1 -- 172.21.15.131:0/1775314091 >> 172.21.15.131:0/1775314091 conn(0x7f50040ce3d0 msgr2=0x7f50040ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:51.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.006+0000 7f5008bff700 1 -- 172.21.15.131:0/1775314091 shutdown_connections 2023-12-26T21:49:51.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.006+0000 7f5008bff700 1 -- 172.21.15.131:0/1775314091 wait complete. 2023-12-26T21:49:51.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.007+0000 7f5008bff700 1 Processor -- start 2023-12-26T21:49:51.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.007+0000 7f5008bff700 1 -- start start 2023-12-26T21:49:51.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.008+0000 7f5008bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500406a1a0 0x7f5004169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.008+0000 7f5008bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5004159560 0x7f50040c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.008+0000 7f5008bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 0x7f50040c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.008+0000 7f5008bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040c8340 con 0x7f5004159930 2023-12-26T21:49:51.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.008+0000 7f5008bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040c6b00 con 0x7f500406a1a0 2023-12-26T21:49:51.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.008+0000 7f5008bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50040c5ef0 con 0x7f5004159560 2023-12-26T21:49:51.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 0x7f50040c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5001d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5004159560 0x7f50040c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f500259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500406a1a0 0x7f5004169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 0x7f50040c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60684/0 (socket says 172.21.15.131:60684) 2023-12-26T21:49:51.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5002d9d700 1 -- 172.21.15.131:0/4234477442 learned_addr learned my addr 172.21.15.131:0/4234477442 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:51.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5001d9b700 1 -- 172.21.15.131:0/4234477442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500406a1a0 msgr2=0x7f5004169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5001d9b700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500406a1a0 0x7f5004169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5001d9b700 1 -- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 msgr2=0x7f50040c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5001d9b700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 0x7f50040c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5001d9b700 1 -- 172.21.15.131:0/4234477442 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff80094f0 con 0x7f5004159560 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f500259c700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500406a1a0 0x7f5004169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5002d9d700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 0x7f50040c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.009+0000 7f5001d9b700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5004159560 0x7f50040c7e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f500406dca0 tx=0x7f4ff8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.010+0000 7f4ff27fc700 1 -- 172.21.15.131:0/4234477442 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800bbe0 con 0x7f5004159560 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.010+0000 7f4ff27fc700 1 -- 172.21.15.131:0/4234477442 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8000c90 con 0x7f5004159560 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.010+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50040c61d0 con 0x7f5004159560 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.010+0000 7f4ff27fc700 1 -- 172.21.15.131:0/4234477442 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff801a640 con 0x7f5004159560 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.010+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50040c6790 con 0x7f5004159560 2023-12-26T21:49:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.011+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50040657e0 con 0x7f5004159560 2023-12-26T21:49:51.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.012+0000 7f4ff27fc700 1 -- 172.21.15.131:0/4234477442 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f4ff801a9f0 con 0x7f5004159560 2023-12-26T21:49:51.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.012+0000 7f4ff27fc700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fe40670e0 0x7f4fe4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.012+0000 7f4ff27fc700 1 -- 172.21.15.131:0/4234477442 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f4ff8086760 con 0x7f5004159560 2023-12-26T21:49:51.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.012+0000 7f500259c700 1 -- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fe40670e0 msgr2=0x7f4fe4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:51.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.012+0000 7f500259c700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fe40670e0 0x7f4fe4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:51.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.017+0000 7f4ff27fc700 1 -- 172.21.15.131:0/4234477442 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ff80533a0 con 0x7f5004159560 2023-12-26T21:49:51.037 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:51.036+0000 7ff21e09c1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:49:51.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.213+0000 7f500259c700 1 -- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fe40670e0 msgr2=0x7f4fe4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:51.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.213+0000 7f500259c700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fe40670e0 0x7f4fe4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:51.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.218+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50040c89e0 con 0x7f5004159560 2023-12-26T21:49:51.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.220+0000 7f4ff27fc700 1 -- 172.21.15.131:0/4234477442 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f4ff80569c0 con 0x7f5004159560 2023-12-26T21:49:51.220 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:51.220 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":51,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:49:50.175359+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:37:47.268950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-26T21:37:48.501427+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.23","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1288038854":"2023-12-27T21:38:05.306953+0000","172.21.15.131:0/518003331":"2023-12-27T21:38:05.306953+0000","172.21.15.131:6833/39288":"2023-12-27T21:38:05.306953+0000","172.21.15.131:6832/39288":"2023-12-27T21:38:05.306953+0000","172.21.15.131:0/2098522634":"2023-12-27T21:38:05.306953+0000","172.21.15.131:0/509990189":"2023-12-27T21:38:05.306953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":111,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fe40670e0 msgr2=0x7f4fe4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fe40670e0 0x7f4fe4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5004159560 msgr2=0x7f50040c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5004159560 0x7f50040c7e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f500406dca0 tx=0x7f4ff8017000 comp rx=0 tx=0).stop 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 shutdown_connections 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5004159930 0x7f50040c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f500406a1a0 0x7f5004169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4fe40670e0 0x7f4fe4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 --2- 172.21.15.131:0/4234477442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5004159560 0x7f50040c7e00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 >> 172.21.15.131:0/4234477442 conn(0x7f50040ce3d0 msgr2=0x7f5004068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 shutdown_connections 2023-12-26T21:49:51.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.222+0000 7f5008bff700 1 -- 172.21.15.131:0/4234477442 wait complete. 2023-12-26T21:49:51.235 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:51.235+0000 7ff21e09c1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:49:51.241 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T21:49:51.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.477+0000 7f9054bff700 1 Processor -- start 2023-12-26T21:49:51.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.477+0000 7f9054bff700 1 -- start start 2023-12-26T21:49:51.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.479+0000 7f9054bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f90500cb320 0x7f9050069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.479+0000 7f9054bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9050156940 con 0x7f90500b0e60 2023-12-26T21:49:51.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.479+0000 7f9054bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90500ae500 con 0x7f905006a1e0 2023-12-26T21:49:51.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.479+0000 7f9054bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90500c8cc0 con 0x7f90500cb320 2023-12-26T21:49:51.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.479+0000 7f904ed9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f90500b0e60 0x7f90500c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40174/0 (socket says 172.21.15.131:40174) 2023-12-26T21:49:51.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.479+0000 7f904ed9d700 1 -- 172.21.15.131:0/2602857322 learned_addr learned my addr 172.21.15.131:0/2602857322 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:51.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.479+0000 7f904e59c700 1 --2- 172.21.15.131:0/2602857322 >> v2:172.21.15.190:3300/0 conn(0x7f90500cb320 0x7f9050069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.480+0000 7f904d59a700 1 -- 172.21.15.131:0/2602857322 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096582578 0 0) 0x7f9050156940 con 0x7f90500b0e60 2023-12-26T21:49:51.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.480+0000 7f904d59a700 1 -- 172.21.15.131:0/2602857322 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9038003570 con 0x7f90500b0e60 2023-12-26T21:49:51.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.480+0000 7f904d59a700 1 -- 172.21.15.131:0/2602857322 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617730733 0 0) 0x7f90500ae500 con 0x7f905006a1e0 2023-12-26T21:49:51.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.480+0000 7f904d59a700 1 -- 172.21.15.131:0/2602857322 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9050156940 con 0x7f905006a1e0 2023-12-26T21:49:51.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.480+0000 7f904d59a700 1 -- 172.21.15.131:0/2602857322 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2424200792 0 0) 0x7f9038003570 con 0x7f90500b0e60 2023-12-26T21:49:51.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.480+0000 7f904e59c700 1 -- 172.21.15.131:0/2602857322 >> v1:172.21.15.190:6789/0 conn(0x7f905006a1e0 legacy=0x7f90500b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.480+0000 7f904e59c700 1 -- 172.21.15.131:0/2602857322 >> v1:172.21.15.131:6789/0 conn(0x7f90500b0e60 legacy=0x7f90500c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.480+0000 7f904e59c700 1 -- 172.21.15.131:0/2602857322 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90500b0280 con 0x7f90500cb320 2023-12-26T21:49:51.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.480+0000 7f904e59c700 1 --2- 172.21.15.131:0/2602857322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90500cb320 0x7f9050069ca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90400098e0 tx=0x7f90400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcb9f529f638cf0a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:51.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f904d59a700 1 -- 172.21.15.131:0/2602857322 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90480017f0 con 0x7f90500cb320 2023-12-26T21:49:51.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f904d59a700 1 -- 172.21.15.131:0/2602857322 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9040011420 con 0x7f90500cb320 2023-12-26T21:49:51.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f904d59a700 1 -- 172.21.15.131:0/2602857322 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400117c0 con 0x7f90500cb320 2023-12-26T21:49:51.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f9054bff700 1 -- 172.21.15.131:0/2602857322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90500cb320 msgr2=0x7f9050069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f9054bff700 1 --2- 172.21.15.131:0/2602857322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90500cb320 0x7f9050069ca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90400098e0 tx=0x7f90400091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f9054bff700 1 -- 172.21.15.131:0/2602857322 shutdown_connections 2023-12-26T21:49:51.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f9054bff700 1 --2- 172.21.15.131:0/2602857322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90500cb320 0x7f9050069ca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f9054bff700 1 -- 172.21.15.131:0/2602857322 >> 172.21.15.131:0/2602857322 conn(0x7f9050068040 msgr2=0x7f9050069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:51.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f9054bff700 1 -- 172.21.15.131:0/2602857322 shutdown_connections 2023-12-26T21:49:51.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.481+0000 7f9054bff700 1 -- 172.21.15.131:0/2602857322 wait complete. 2023-12-26T21:49:51.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.483+0000 7f9054bff700 1 Processor -- start 2023-12-26T21:49:51.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.483+0000 7f9054bff700 1 -- start start 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f9054bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f905006a1e0 0x7f905016a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f9054bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90500b0e60 0x7f905014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f9054bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90500cb320 0x7f9050159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f9054bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905014f000 con 0x7f90500cb320 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f9054bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905014f2e0 con 0x7f905006a1e0 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f9054bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905014f5c0 con 0x7f90500b0e60 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f904ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90500cb320 0x7f9050159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f904ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90500cb320 0x7f9050159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60694/0 (socket says 172.21.15.131:60694) 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f904ed9d700 1 -- 172.21.15.131:0/2422950169 learned_addr learned my addr 172.21.15.131:0/2422950169 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f904e59c700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f905006a1e0 0x7f905016a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.484+0000 7f904dd9b700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90500b0e60 0x7f905014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f904ed9d700 1 -- 172.21.15.131:0/2422950169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90500b0e60 msgr2=0x7f905014e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f904ed9d700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90500b0e60 0x7f905014e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f904ed9d700 1 -- 172.21.15.131:0/2422950169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f905006a1e0 msgr2=0x7f905016a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f904ed9d700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f905006a1e0 0x7f905016a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f904ed9d700 1 -- 172.21.15.131:0/2422950169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040009420 con 0x7f90500cb320 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f904ed9d700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90500cb320 0x7f9050159340 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f904800cc40 tx=0x7f90480018e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f903e7fc700 1 -- 172.21.15.131:0/2422950169 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048004dc0 con 0x7f90500cb320 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f903e7fc700 1 -- 172.21.15.131:0/2422950169 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904800a660 con 0x7f90500cb320 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f903e7fc700 1 -- 172.21.15.131:0/2422950169 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904800aa20 con 0x7f90500cb320 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905014f8a0 con 0x7f90500cb320 2023-12-26T21:49:51.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.485+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905016aec0 con 0x7f90500cb320 2023-12-26T21:49:51.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.487+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90500cba10 con 0x7f90500cb320 2023-12-26T21:49:51.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.487+0000 7f903e7fc700 1 -- 172.21.15.131:0/2422950169 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7f9048007150 con 0x7f90500cb320 2023-12-26T21:49:51.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.488+0000 7f903e7fc700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90300670e0 0x7f9030069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.488+0000 7f903e7fc700 1 -- 172.21.15.131:0/2422950169 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f90480841b0 con 0x7f90500cb320 2023-12-26T21:49:51.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.488+0000 7f904e59c700 1 -- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90300670e0 msgr2=0x7f9030069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:51.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.488+0000 7f904e59c700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90300670e0 0x7f9030069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:51.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.492+0000 7f903e7fc700 1 -- 172.21.15.131:0/2422950169 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9048050e70 con 0x7f90500cb320 2023-12-26T21:49:51.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.689+0000 7f904e59c700 1 -- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90300670e0 msgr2=0x7f9030069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:51.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.689+0000 7f904e59c700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90300670e0 0x7f9030069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:51.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.739+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f90500657e0 con 0x7f90500cb320 2023-12-26T21:49:51.740 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T21:49:51.740+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T21:49:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.745+0000 7f903e7fc700 1 -- 172.21.15.131:0/2422950169 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9048054490 con 0x7f90500cb320 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90300670e0 msgr2=0x7f9030069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90300670e0 0x7f9030069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90500cb320 msgr2=0x7f9050159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90500cb320 0x7f9050159340 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f904800cc40 tx=0x7f90480018e0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 shutdown_connections 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90500cb320 0x7f9050159340 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f905006a1e0 0x7f905016a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90300670e0 0x7f9030069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 --2- 172.21.15.131:0/2422950169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90500b0e60 0x7f905014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 >> 172.21.15.131:0/2422950169 conn(0x7f9050068040 msgr2=0x7f9050152180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 shutdown_connections 2023-12-26T21:49:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.748+0000 7f9054bff700 1 -- 172.21.15.131:0/2422950169 wait complete. 2023-12-26T21:49:51.750 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T21:49:51.767 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T21:49:51.805 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:49:51.814 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:51.814+0000 7f178a0751c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:49:51.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.960+0000 7fb371fff700 1 Processor -- start 2023-12-26T21:49:51.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.960+0000 7fb371fff700 1 -- start start 2023-12-26T21:49:51.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb371fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb36c15d240 0x7fb36c0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb371fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb36c0ad390 0x7fb36c0aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb371fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36c0a98b0 con 0x7fb36c1595e0 2023-12-26T21:49:51.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb371fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c0add40 con 0x7fb36c0ad390 2023-12-26T21:49:51.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb371fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c0ad790 con 0x7fb36c15d240 2023-12-26T21:49:51.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb36bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb36c0ad390 0x7fb36c0aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb36bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb36c0ad390 0x7fb36c0aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60702/0 (socket says 172.21.15.131:60702) 2023-12-26T21:49:51.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb36bfff700 1 -- 172.21.15.131:0/3562910806 learned_addr learned my addr 172.21.15.131:0/3562910806 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:51.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb36affd700 1 --2- 172.21.15.131:0/3562910806 >> v2:172.21.15.190:3300/0 conn(0x7fb36c15d240 0x7fb36c0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb36bfff700 1 -- 172.21.15.131:0/3562910806 >> v2:172.21.15.190:3300/0 conn(0x7fb36c15d240 msgr2=0x7fb36c0ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb36bfff700 1 --2- 172.21.15.131:0/3562910806 >> v2:172.21.15.190:3300/0 conn(0x7fb36c15d240 0x7fb36c0ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb36bfff700 1 -- 172.21.15.131:0/3562910806 >> v1:172.21.15.190:6789/0 conn(0x7fb36c1595e0 legacy=0x7fb36c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb36bfff700 1 -- 172.21.15.131:0/3562910806 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c15a500 con 0x7fb36c0ad390 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.962+0000 7fb36a7fc700 1 -- 172.21.15.131:0/3562910806 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199589909 0 0) 0x7fb36c0a98b0 con 0x7fb36c1595e0 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.963+0000 7fb36bfff700 1 --2- 172.21.15.131:0/3562910806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 0x7fb36c0aa020 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb36c06d470 tx=0x7fb364009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8e9c8d2557207 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.963+0000 7fb36a7fc700 1 -- 172.21.15.131:0/3562910806 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36400e070 con 0x7fb36c0ad390 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.963+0000 7fb36a7fc700 1 -- 172.21.15.131:0/3562910806 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36400ba20 con 0x7fb36c0ad390 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.963+0000 7fb36a7fc700 1 -- 172.21.15.131:0/3562910806 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3640092e0 con 0x7fb36c0ad390 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.964+0000 7fb371fff700 1 -- 172.21.15.131:0/3562910806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 msgr2=0x7fb36c0aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.964+0000 7fb371fff700 1 --2- 172.21.15.131:0/3562910806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 0x7fb36c0aa020 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb36c06d470 tx=0x7fb364009840 comp rx=0 tx=0).stop 2023-12-26T21:49:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.964+0000 7fb371fff700 1 -- 172.21.15.131:0/3562910806 shutdown_connections 2023-12-26T21:49:51.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.964+0000 7fb371fff700 1 --2- 172.21.15.131:0/3562910806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 0x7fb36c0aa020 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.964+0000 7fb371fff700 1 --2- 172.21.15.131:0/3562910806 >> v2:172.21.15.190:3300/0 conn(0x7fb36c15d240 0x7fb36c0ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.964+0000 7fb371fff700 1 -- 172.21.15.131:0/3562910806 >> 172.21.15.131:0/3562910806 conn(0x7fb36c068040 msgr2=0x7fb36c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:51.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.964+0000 7fb371fff700 1 -- 172.21.15.131:0/3562910806 shutdown_connections 2023-12-26T21:49:51.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.964+0000 7fb371fff700 1 -- 172.21.15.131:0/3562910806 wait complete. 2023-12-26T21:49:51.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.965+0000 7fb371fff700 1 Processor -- start 2023-12-26T21:49:51.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.965+0000 7fb371fff700 1 -- start start 2023-12-26T21:49:51.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.966+0000 7fb371fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 0x7fb36c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.966+0000 7fb371fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb36c1595e0 0x7fb36c15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.966+0000 7fb371fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb36c15d240 0x7fb36c0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.966+0000 7fb371fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c15e4d0 con 0x7fb36c0ad390 2023-12-26T21:49:51.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.966+0000 7fb371fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c13f7d0 con 0x7fb36c1595e0 2023-12-26T21:49:51.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.966+0000 7fb371fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c15f320 con 0x7fb36c15d240 2023-12-26T21:49:51.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 0x7fb36c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb36c1595e0 0x7fb36c15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb36c15d240 0x7fb36c0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:51.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb36c1595e0 0x7fb36c15f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:48276/0 (socket says 172.21.15.131:48276) 2023-12-26T21:49:51.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36affd700 1 -- 172.21.15.131:0/2690753127 learned_addr learned my addr 172.21.15.131:0/2690753127 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:51.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36b7fe700 1 -- 172.21.15.131:0/2690753127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb36c15d240 msgr2=0x7fb36c0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36b7fe700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb36c15d240 0x7fb36c0b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36b7fe700 1 -- 172.21.15.131:0/2690753127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb36c1595e0 msgr2=0x7fb36c15f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:51.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36b7fe700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb36c1595e0 0x7fb36c15f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:51.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36b7fe700 1 -- 172.21.15.131:0/2690753127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364009ae0 con 0x7fb36c0ad390 2023-12-26T21:49:51.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36affd700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb36c1595e0 0x7fb36c15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:51.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.967+0000 7fb36b7fe700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 0x7fb36c15df90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c5b0 tx=0x7fb35c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:51.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.968+0000 7fb3537fe700 1 -- 172.21.15.131:0/2690753127 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c011590 con 0x7fb36c0ad390 2023-12-26T21:49:51.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.968+0000 7fb3537fe700 1 -- 172.21.15.131:0/2690753127 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c011710 con 0x7fb36c0ad390 2023-12-26T21:49:51.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.968+0000 7fb3537fe700 1 -- 172.21.15.131:0/2690753127 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c011ad0 con 0x7fb36c0ad390 2023-12-26T21:49:51.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.968+0000 7fb371fff700 1 -- 172.21.15.131:0/2690753127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c15e710 con 0x7fb36c0ad390 2023-12-26T21:49:51.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.968+0000 7fb371fff700 1 -- 172.21.15.131:0/2690753127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c15ecd0 con 0x7fb36c0ad390 2023-12-26T21:49:51.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.969+0000 7fb371fff700 1 -- 172.21.15.131:0/2690753127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c0657e0 con 0x7fb36c0ad390 2023-12-26T21:49:51.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.970+0000 7fb3537fe700 1 -- 172.21.15.131:0/2690753127 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 84913+0+0 (secure 0 0 0) 0x7fb35c01b3b0 con 0x7fb36c0ad390 2023-12-26T21:49:51.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.971+0000 7fb3537fe700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 0x7fb358069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:51.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.971+0000 7fb3537fe700 1 -- 172.21.15.131:0/2690753127 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fb35c083f60 con 0x7fb36c0ad390 2023-12-26T21:49:51.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.971+0000 7fb36affd700 1 -- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 msgr2=0x7fb358069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:51.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.971+0000 7fb36affd700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 0x7fb358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:49:51.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:51.975+0000 7fb3537fe700 1 -- 172.21.15.131:0/2690753127 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb35c050c10 con 0x7fb36c0ad390 2023-12-26T21:49:52.089 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:52.089+0000 7f178a0751c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:49:52.119 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:52.118+0000 7ff21e09c1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:49:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.172+0000 7fb36affd700 1 -- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 msgr2=0x7fb358069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.172+0000 7fb36affd700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 0x7fb358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:49:52.181 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:52.181+0000 7f178a0751c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:49:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.219+0000 7fb371fff700 1 -- 172.21.15.131:0/2690753127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb36c15a7e0 con 0x7fb36c0ad390 2023-12-26T21:49:52.375 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:49:52.375+0000 7f178a0751c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:49:52.400 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:52.400+0000 7ff21e09c1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:49:52.492 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:52.492+0000 7ff21e09c1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:49:52.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.573+0000 7fb36affd700 1 -- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 msgr2=0x7fb358069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:49:52.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.573+0000 7fb36affd700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 0x7fb358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:49:52.688 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:49:52.688+0000 7ff21e09c1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:49:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.699+0000 7fb3537fe700 1 -- 172.21.15.131:0/2690753127 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 15) v1 ==== 84453+0+0 (secure 0 0 0) 0x7fb35c050fe0 con 0x7fb36c0ad390 2023-12-26T21:49:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.700+0000 7fb3537fe700 1 -- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 msgr2=0x7fb358069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:49:52.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.700+0000 7fb3537fe700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 0x7fb358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:52.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.746+0000 7fb3537fe700 1 -- 172.21.15.131:0/2690753127 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb35c06ea20 con 0x7fb36c0ad390 2023-12-26T21:49:52.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.748+0000 7fb371fff700 1 -- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 msgr2=0x7fb36c15df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:52.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.748+0000 7fb371fff700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 0x7fb36c15df90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c5b0 tx=0x7fb35c007d30 comp rx=0 tx=0).stop 2023-12-26T21:49:52.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.749+0000 7fb371fff700 1 -- 172.21.15.131:0/2690753127 shutdown_connections 2023-12-26T21:49:52.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.749+0000 7fb371fff700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb3580670e0 0x7fb358069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:52.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.749+0000 7fb371fff700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb36c15d240 0x7fb36c0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:52.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.749+0000 7fb371fff700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c0ad390 0x7fb36c15df90 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c5b0 tx=0x7fb35c007d30 comp rx=0 tx=0).stop 2023-12-26T21:49:52.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.749+0000 7fb371fff700 1 --2- 172.21.15.131:0/2690753127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb36c1595e0 0x7fb36c15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:52.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.749+0000 7fb371fff700 1 -- 172.21.15.131:0/2690753127 >> 172.21.15.131:0/2690753127 conn(0x7fb36c068040 msgr2=0x7fb36c154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:52.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.749+0000 7fb371fff700 1 -- 172.21.15.131:0/2690753127 shutdown_connections 2023-12-26T21:49:52.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.749+0000 7fb371fff700 1 -- 172.21.15.131:0/2690753127 wait complete. 2023-12-26T21:49:52.773 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T21:49:52.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.966+0000 7fc049bff700 1 Processor -- start 2023-12-26T21:49:52.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.966+0000 7fc049bff700 1 -- start start 2023-12-26T21:49:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.968+0000 7fc049bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc0440cb510 0x7fc044069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.968+0000 7fc049bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cc590 0x7fc0440c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.968+0000 7fc049bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc044069640 con 0x7fc044142f00 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.968+0000 7fc049bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc044068750 con 0x7fc0440cb510 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.968+0000 7fc049bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc044068a30 con 0x7fc0440cc590 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.968+0000 7fc03affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc0440cb510 0x7fc044069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.969+0000 7fc03affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc0440cb510 0x7fc044069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60816/0 (socket says 172.21.15.131:60816) 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.969+0000 7fc03affd700 1 -- 172.21.15.131:0/618643903 learned_addr learned my addr 172.21.15.131:0/618643903 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.969+0000 7fc043fff700 1 --2- 172.21.15.131:0/618643903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cc590 0x7fc0440c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.969+0000 7fc03affd700 1 -- 172.21.15.131:0/618643903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cc590 msgr2=0x7fc0440c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.969+0000 7fc03affd700 1 --2- 172.21.15.131:0/618643903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cc590 0x7fc0440c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.969+0000 7fc03affd700 1 -- 172.21.15.131:0/618643903 >> v1:172.21.15.190:6789/0 conn(0x7fc044142f00 legacy=0x7fc0440cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.969+0000 7fc03affd700 1 -- 172.21.15.131:0/618643903 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc044068d10 con 0x7fc0440cb510 2023-12-26T21:49:52.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.969+0000 7fc03affd700 1 --2- 172.21.15.131:0/618643903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0440cb510 0x7fc044069ad0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0099b0 tx=0x7fc02c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4e53b3197b87a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.969+0000 7fc042ffd700 1 -- 172.21.15.131:0/618643903 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c009ea0 con 0x7fc0440cb510 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc042ffd700 1 -- 172.21.15.131:0/618643903 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c00b860 con 0x7fc0440cb510 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc042ffd700 1 -- 172.21.15.131:0/618643903 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00bc00 con 0x7fc0440cb510 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc049bff700 1 -- 172.21.15.131:0/618643903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0440cb510 msgr2=0x7fc044069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc049bff700 1 --2- 172.21.15.131:0/618643903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0440cb510 0x7fc044069ad0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0099b0 tx=0x7fc02c009250 comp rx=0 tx=0).stop 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc049bff700 1 -- 172.21.15.131:0/618643903 shutdown_connections 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc049bff700 1 --2- 172.21.15.131:0/618643903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cc590 0x7fc0440c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc049bff700 1 --2- 172.21.15.131:0/618643903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0440cb510 0x7fc044069ad0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc049bff700 1 -- 172.21.15.131:0/618643903 >> 172.21.15.131:0/618643903 conn(0x7fc044063cd0 msgr2=0x7fc04406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc049bff700 1 -- 172.21.15.131:0/618643903 shutdown_connections 2023-12-26T21:49:52.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.970+0000 7fc049bff700 1 -- 172.21.15.131:0/618643903 wait complete. 2023-12-26T21:49:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.972+0000 7fc049bff700 1 Processor -- start 2023-12-26T21:49:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.972+0000 7fc049bff700 1 -- start start 2023-12-26T21:49:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc049bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cb510 0x7fc0440a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc049bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0440cc590 0x7fc0440b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc049bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc044142f00 0x7fc044152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc049bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b4260 con 0x7fc044142f00 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc049bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc044152590 con 0x7fc0440cc590 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc049bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440a9fb0 con 0x7fc0440cb510 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc043fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc044142f00 0x7fc044152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc043fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc044142f00 0x7fc044152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60820/0 (socket says 172.21.15.131:60820) 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc043fff700 1 -- 172.21.15.131:0/823542625 learned_addr learned my addr 172.21.15.131:0/823542625 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc03affd700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0440cc590 0x7fc0440b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc0437fe700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cb510 0x7fc0440a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc043fff700 1 -- 172.21.15.131:0/823542625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cb510 msgr2=0x7fc0440a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc043fff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cb510 0x7fc0440a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc043fff700 1 -- 172.21.15.131:0/823542625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0440cc590 msgr2=0x7fc0440b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc043fff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0440cc590 0x7fc0440b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc043fff700 1 -- 172.21.15.131:0/823542625 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0094f0 con 0x7fc044142f00 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc0437fe700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cb510 0x7fc0440a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.973+0000 7fc043fff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc044142f00 0x7fc044152050 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00ea80 tx=0x7fc03c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.974+0000 7fc03bfff700 1 -- 172.21.15.131:0/823542625 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00c850 con 0x7fc044142f00 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.974+0000 7fc03bfff700 1 -- 172.21.15.131:0/823542625 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c00c9d0 con 0x7fc044142f00 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.974+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0440aa290 con 0x7fc044142f00 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.974+0000 7fc03bfff700 1 -- 172.21.15.131:0/823542625 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00cd90 con 0x7fc044142f00 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.974+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0440aa850 con 0x7fc044142f00 2023-12-26T21:49:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.975+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0440aaea0 con 0x7fc044142f00 2023-12-26T21:49:52.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.976+0000 7fc03bfff700 1 -- 172.21.15.131:0/823542625 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 15) v1 ==== 84453+0+0 (secure 0 0 0) 0x7fc03c013680 con 0x7fc044142f00 2023-12-26T21:49:52.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.977+0000 7fc03bfff700 1 -- 172.21.15.131:0/823542625 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6830+0+0 (secure 0 0 0) 0x7fc03c084740 con 0x7fc044142f00 2023-12-26T21:49:52.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:52.981+0000 7fc03bfff700 1 -- 172.21.15.131:0/823542625 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc03c051910 con 0x7fc044142f00 2023-12-26T21:49:53.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.175+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0440717f0 con 0x7fc044142f00 2023-12-26T21:49:53.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.703+0000 7fc03bfff700 1 -- 172.21.15.131:0/823542625 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 16) v1 ==== 84580+0+0 (secure 0 0 0) 0x7fc03c0513c0 con 0x7fc044142f00 2023-12-26T21:49:53.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.703+0000 7fc03bfff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc02406cb20 0x7fc02406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:53.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.705+0000 7fc0437fe700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc02406cb20 0x7fc02406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:53.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.707+0000 7fc0437fe700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc02406cb20 0x7fc02406ef00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc034007fd0 tx=0x7fc034009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.754+0000 7fc03bfff700 1 -- 172.21.15.131:0/823542625 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc03c06fe60 con 0x7fc044142f00 2023-12-26T21:49:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc02406cb20 msgr2=0x7fc02406ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc02406cb20 0x7fc02406ef00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc034007fd0 tx=0x7fc034009040 comp rx=0 tx=0).stop 2023-12-26T21:49:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc044142f00 msgr2=0x7fc044152050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc044142f00 0x7fc044152050 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00ea80 tx=0x7fc03c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 shutdown_connections 2023-12-26T21:49:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc044142f00 0x7fc044152050 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc02406cb20 0x7fc02406ef00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440cb510 0x7fc0440a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 --2- 172.21.15.131:0/823542625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0440cc590 0x7fc0440b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 >> 172.21.15.131:0/823542625 conn(0x7fc044063cd0 msgr2=0x7fc0440c40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 shutdown_connections 2023-12-26T21:49:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.757+0000 7fc049bff700 1 -- 172.21.15.131:0/823542625 wait complete. 2023-12-26T21:49:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T21:49:53.777 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T21:49:53.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.976+0000 7efd229ff700 1 Processor -- start 2023-12-26T21:49:53.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.976+0000 7efd229ff700 1 -- start start 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.977+0000 7efd229ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efd1c159810 0x7efd1c0c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.977+0000 7efd229ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c0cd7b0 0x7efd1c0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.977+0000 7efd229ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c0cbb00 con 0x7efd1c0cec80 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.977+0000 7efd229ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1c0cb2a0 con 0x7efd1c159810 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.977+0000 7efd229ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0cf640 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.978+0000 7efd1bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efd1c159810 0x7efd1c0c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.978+0000 7efd1bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efd1c159810 0x7efd1c0c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60824/0 (socket says 172.21.15.131:60824) 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.978+0000 7efd1bfff700 1 -- 172.21.15.131:0/73008037 learned_addr learned my addr 172.21.15.131:0/73008037 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.978+0000 7efd20f9c700 1 --2- 172.21.15.131:0/73008037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c0cd7b0 0x7efd1c0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.978+0000 7efd1bfff700 1 -- 172.21.15.131:0/73008037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c0cd7b0 msgr2=0x7efd1c0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.978+0000 7efd1bfff700 1 --2- 172.21.15.131:0/73008037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c0cd7b0 0x7efd1c0cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.978+0000 7efd1bfff700 1 -- 172.21.15.131:0/73008037 >> v1:172.21.15.190:6789/0 conn(0x7efd1c0cec80 legacy=0x7efd1c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:53.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.978+0000 7efd1bfff700 1 -- 172.21.15.131:0/73008037 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c0ce040 con 0x7efd1c159810 2023-12-26T21:49:53.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd1bfff700 1 --2- 172.21.15.131:0/73008037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c159810 0x7efd1c0c5440 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009990 tx=0x7efd0c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103aad35b798c443 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd1b7fe700 1 -- 172.21.15.131:0/73008037 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c009e80 con 0x7efd1c159810 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd1b7fe700 1 -- 172.21.15.131:0/73008037 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c005c10 con 0x7efd1c159810 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd1b7fe700 1 -- 172.21.15.131:0/73008037 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c00b650 con 0x7efd1c159810 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd229ff700 1 -- 172.21.15.131:0/73008037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c159810 msgr2=0x7efd1c0c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd229ff700 1 --2- 172.21.15.131:0/73008037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c159810 0x7efd1c0c5440 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009990 tx=0x7efd0c009220 comp rx=0 tx=0).stop 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd229ff700 1 -- 172.21.15.131:0/73008037 shutdown_connections 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd229ff700 1 --2- 172.21.15.131:0/73008037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c0cd7b0 0x7efd1c0cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd229ff700 1 --2- 172.21.15.131:0/73008037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c159810 0x7efd1c0c5440 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.979+0000 7efd229ff700 1 -- 172.21.15.131:0/73008037 >> 172.21.15.131:0/73008037 conn(0x7efd1c099950 msgr2=0x7efd1c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.980+0000 7efd229ff700 1 -- 172.21.15.131:0/73008037 shutdown_connections 2023-12-26T21:49:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.980+0000 7efd229ff700 1 -- 172.21.15.131:0/73008037 wait complete. 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.981+0000 7efd229ff700 1 Processor -- start 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.981+0000 7efd229ff700 1 -- start start 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd229ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c0cd7b0 0x7efd1c14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd229ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c0cec80 0x7efd1c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd229ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c159810 0x7efd1c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd229ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0b3ab0 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd229ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0c7910 con 0x7efd1c0cec80 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd229ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c157610 con 0x7efd1c159810 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c0cd7b0 0x7efd1c14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c0cd7b0 0x7efd1c14d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60838/0 (socket says 172.21.15.131:60838) 2023-12-26T21:49:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd1bfff700 1 -- 172.21.15.131:0/1592868940 learned_addr learned my addr 172.21.15.131:0/1592868940 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd13fff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c0cec80 0x7efd1c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.982+0000 7efd20f9c700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c159810 0x7efd1c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd1bfff700 1 -- 172.21.15.131:0/1592868940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c159810 msgr2=0x7efd1c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd1bfff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c159810 0x7efd1c1587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd1bfff700 1 -- 172.21.15.131:0/1592868940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c0cec80 msgr2=0x7efd1c1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd1bfff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c0cec80 0x7efd1c1582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd1bfff700 1 -- 172.21.15.131:0/1592868940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c009480 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd1bfff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c0cd7b0 0x7efd1c14d3b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efd0c011750 tx=0x7efd0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd18ff9700 1 -- 172.21.15.131:0/1592868940 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c011970 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd18ff9700 1 -- 172.21.15.131:0/1592868940 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c020070 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd18ff9700 1 -- 172.21.15.131:0/1592868940 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c011d00 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c1578f0 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.983+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c0b3740 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.984+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c0717f0 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.985+0000 7efd18ff9700 1 -- 172.21.15.131:0/1592868940 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) v1 ==== 84580+0+0 (secure 0 0 0) 0x7efd0c022030 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.986+0000 7efd18ff9700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7efcfc066d80 0x7efcfc069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:53.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.987+0000 7efd18ff9700 1 -- 172.21.15.131:0/1592868940 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6392+0+0 (secure 0 0 0) 0x7efd0c085f80 con 0x7efd1c0cd7b0 2023-12-26T21:49:53.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.987+0000 7efd13fff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7efcfc066d80 0x7efcfc069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:53.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.987+0000 7efd13fff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7efcfc066d80 0x7efcfc069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0ca0f0 tx=0x7efd04007ec0 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:53.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:53.991+0000 7efd18ff9700 1 -- 172.21.15.131:0/1592868940 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd0c0b75a0 con 0x7efd1c0cd7b0 2023-12-26T21:49:54.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:54.185+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd1c08a650 con 0x7efd1c0cd7b0 2023-12-26T21:49:55.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.174+0000 7efd18ff9700 1 -- 172.21.15.131:0/1592868940 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) v1 ==== 169+0+0 (secure 0 0 0) 0x7efd0c0531f0 con 0x7efd1c0cd7b0 2023-12-26T21:49:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.178+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7efcfc066d80 msgr2=0x7efcfc069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.178+0000 7efd229ff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7efcfc066d80 0x7efcfc069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0ca0f0 tx=0x7efd04007ec0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.178+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c0cd7b0 msgr2=0x7efd1c14d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.178+0000 7efd229ff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c0cd7b0 0x7efd1c14d3b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efd0c011750 tx=0x7efd0c016000 comp rx=0 tx=0).stop 2023-12-26T21:49:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.179+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 shutdown_connections 2023-12-26T21:49:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.179+0000 7efd229ff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7efcfc066d80 0x7efcfc069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.179+0000 7efd229ff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c159810 0x7efd1c1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.179+0000 7efd229ff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd1c0cd7b0 0x7efd1c14d3b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.179+0000 7efd229ff700 1 --2- 172.21.15.131:0/1592868940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c0cec80 0x7efd1c1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.179+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 >> 172.21.15.131:0/1592868940 conn(0x7efd1c099950 msgr2=0x7efd1c15a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.179+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 shutdown_connections 2023-12-26T21:49:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.179+0000 7efd229ff700 1 -- 172.21.15.131:0/1592868940 wait complete. 2023-12-26T21:49:55.181 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T21:49:55.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T21:49:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.401+0000 7f52fbdff700 1 Processor -- start 2023-12-26T21:49:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.401+0000 7f52fbdff700 1 -- start start 2023-12-26T21:49:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52fbdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52f40c8820 0x7f52f40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52fbdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f52f40b19f0 0x7f52f4161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52fbdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f4157a80 con 0x7f52f40c8bf0 2023-12-26T21:49:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52fbdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f40cac50 con 0x7f52f40c8820 2023-12-26T21:49:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52fbdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f40caf30 con 0x7f52f40b19f0 2023-12-26T21:49:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52f9b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52f40c8820 0x7f52f40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52f9b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52f40c8820 0x7f52f40ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33356/0 (socket says 172.21.15.131:33356) 2023-12-26T21:49:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52f9b9b700 1 -- 172.21.15.131:0/2549902918 learned_addr learned my addr 172.21.15.131:0/2549902918 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52fa39c700 1 --2- 172.21.15.131:0/2549902918 >> v2:172.21.15.190:3300/0 conn(0x7f52f40b19f0 0x7f52f4161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52f9b9b700 1 -- 172.21.15.131:0/2549902918 >> v2:172.21.15.190:3300/0 conn(0x7f52f40b19f0 msgr2=0x7f52f4161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52f9b9b700 1 --2- 172.21.15.131:0/2549902918 >> v2:172.21.15.190:3300/0 conn(0x7f52f40b19f0 0x7f52f4161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52f9b9b700 1 -- 172.21.15.131:0/2549902918 >> v1:172.21.15.190:6789/0 conn(0x7f52f40c8bf0 legacy=0x7f52f40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52f9b9b700 1 -- 172.21.15.131:0/2549902918 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f40cb210 con 0x7f52f40c8820 2023-12-26T21:49:55.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.403+0000 7f52fa39c700 1 --2- 172.21.15.131:0/2549902918 >> v2:172.21.15.190:3300/0 conn(0x7f52f40b19f0 0x7f52f4161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:49:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.404+0000 7f52f9b9b700 1 --2- 172.21.15.131:0/2549902918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40c8820 0x7f52f40ca300 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f52f4070f80 tx=0x7f52f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be091c159828175 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.404+0000 7f52f8b99700 1 -- 172.21.15.131:0/2549902918 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f000b860 con 0x7f52f40c8820 2023-12-26T21:49:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.404+0000 7f52f8b99700 1 -- 172.21.15.131:0/2549902918 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f000b9c0 con 0x7f52f40c8820 2023-12-26T21:49:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.404+0000 7f52f8b99700 1 -- 172.21.15.131:0/2549902918 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f000bd60 con 0x7f52f40c8820 2023-12-26T21:49:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.404+0000 7f52fbdff700 1 -- 172.21.15.131:0/2549902918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40c8820 msgr2=0x7f52f40ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.404+0000 7f52fbdff700 1 --2- 172.21.15.131:0/2549902918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40c8820 0x7f52f40ca300 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f52f4070f80 tx=0x7f52f0009250 comp rx=0 tx=0).stop 2023-12-26T21:49:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.405+0000 7f52fbdff700 1 -- 172.21.15.131:0/2549902918 shutdown_connections 2023-12-26T21:49:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.405+0000 7f52fbdff700 1 --2- 172.21.15.131:0/2549902918 >> v2:172.21.15.190:3300/0 conn(0x7f52f40b19f0 0x7f52f4161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.405+0000 7f52fbdff700 1 --2- 172.21.15.131:0/2549902918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40c8820 0x7f52f40ca300 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.405+0000 7f52fbdff700 1 -- 172.21.15.131:0/2549902918 >> 172.21.15.131:0/2549902918 conn(0x7f52f4068040 msgr2=0x7f52f4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.405+0000 7f52fbdff700 1 -- 172.21.15.131:0/2549902918 shutdown_connections 2023-12-26T21:49:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.405+0000 7f52fbdff700 1 -- 172.21.15.131:0/2549902918 wait complete. 2023-12-26T21:49:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.406+0000 7f52fbdff700 1 Processor -- start 2023-12-26T21:49:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.406+0000 7f52fbdff700 1 -- start start 2023-12-26T21:49:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52fbdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40b19f0 0x7f52f406e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52fbdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52f40c8820 0x7f52f4150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52fbdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f40c8bf0 0x7f52f4158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52fbdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f406eba0 con 0x7f52f40b19f0 2023-12-26T21:49:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52fbdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f41511c0 con 0x7f52f40c8bf0 2023-12-26T21:49:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52fbdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f415c550 con 0x7f52f40c8820 2023-12-26T21:49:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52f9b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40b19f0 0x7f52f406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52f9b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40b19f0 0x7f52f406e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33366/0 (socket says 172.21.15.131:33366) 2023-12-26T21:49:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52f9b9b700 1 -- 172.21.15.131:0/3355912221 learned_addr learned my addr 172.21.15.131:0/3355912221 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:55.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52f939a700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52f40c8820 0x7f52f4150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.407+0000 7f52fa39c700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f40c8bf0 0x7f52f4158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52f9b9b700 1 -- 172.21.15.131:0/3355912221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52f40c8820 msgr2=0x7f52f4150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52f9b9b700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52f40c8820 0x7f52f4150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52f9b9b700 1 -- 172.21.15.131:0/3355912221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f40c8bf0 msgr2=0x7f52f4158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52f9b9b700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f40c8bf0 0x7f52f4158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52f9b9b700 1 -- 172.21.15.131:0/3355912221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f00094f0 con 0x7f52f40b19f0 2023-12-26T21:49:55.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52fa39c700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f40c8bf0 0x7f52f4158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:55.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52f9b9b700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40b19f0 0x7f52f406e660 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f52f40688f0 tx=0x7f52f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:55.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52e9ffb700 1 -- 172.21.15.131:0/3355912221 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0024440 con 0x7f52f40b19f0 2023-12-26T21:49:55.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52e9ffb700 1 -- 172.21.15.131:0/3355912221 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f00245a0 con 0x7f52f40b19f0 2023-12-26T21:49:55.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f415c830 con 0x7f52f40b19f0 2023-12-26T21:49:55.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52e9ffb700 1 -- 172.21.15.131:0/3355912221 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0024960 con 0x7f52f40b19f0 2023-12-26T21:49:55.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.408+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f415c220 con 0x7f52f40b19f0 2023-12-26T21:49:55.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.409+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f40657e0 con 0x7f52f40b19f0 2023-12-26T21:49:55.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.410+0000 7f52e9ffb700 1 -- 172.21.15.131:0/3355912221 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f52f0024d50 con 0x7f52f40b19f0 2023-12-26T21:49:55.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.411+0000 7f52e9ffb700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f52d8066f30 0x7f52d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.411+0000 7f52e9ffb700 1 -- 172.21.15.131:0/3355912221 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f52f0086190 con 0x7f52f40b19f0 2023-12-26T21:49:55.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.411+0000 7f52f939a700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f52d8066f30 0x7f52d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.412+0000 7f52f939a700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f52d8066f30 0x7f52d80693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52f415d030 tx=0x7f52e4007ec0 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:55.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.416+0000 7f52e9ffb700 1 -- 172.21.15.131:0/3355912221 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52f00532c0 con 0x7f52f40b19f0 2023-12-26T21:49:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.608+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f52f415cd80 con 0x7f52f40b19f0 2023-12-26T21:49:55.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.612+0000 7f52e9ffb700 1 -- 172.21.15.131:0/3355912221 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f52f002b340 con 0x7f52f40b19f0 2023-12-26T21:49:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f52d8066f30 msgr2=0x7f52d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f52d8066f30 0x7f52d80693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52f415d030 tx=0x7f52e4007ec0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40b19f0 msgr2=0x7f52f406e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40b19f0 0x7f52f406e660 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f52f40688f0 tx=0x7f52f0017000 comp rx=0 tx=0).stop 2023-12-26T21:49:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 shutdown_connections 2023-12-26T21:49:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52f40c8bf0 0x7f52f4158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52f40b19f0 0x7f52f406e660 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f52d8066f30 0x7f52d80693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 --2- 172.21.15.131:0/3355912221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52f40c8820 0x7f52f4150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 >> 172.21.15.131:0/3355912221 conn(0x7f52f4068040 msgr2=0x7f52f4152ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 shutdown_connections 2023-12-26T21:49:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.615+0000 7f52fbdff700 1 -- 172.21.15.131:0/3355912221 wait complete. 2023-12-26T21:49:55.637 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T21:49:55.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.828+0000 7f85773ff700 1 Processor -- start 2023-12-26T21:49:55.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.828+0000 7f85773ff700 1 -- start start 2023-12-26T21:49:55.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.830+0000 7f85773ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8570159920 0x7f85700b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.830+0000 7f85773ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 0x7f85700b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.830+0000 7f85773ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85700c8bf0 con 0x7f8570159cf0 2023-12-26T21:49:55.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.830+0000 7f85773ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85700c8e60 con 0x7f8570159920 2023-12-26T21:49:55.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.830+0000 7f85773ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85700c9140 con 0x7f8570155be0 2023-12-26T21:49:55.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.830+0000 7f857499a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8570159cf0 0x7f85701554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58992/0 (socket says 172.21.15.131:58992) 2023-12-26T21:49:55.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f857499a700 1 -- 172.21.15.131:0/351405318 learned_addr learned my addr 172.21.15.131:0/351405318 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:55.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f857599c700 1 --2- 172.21.15.131:0/351405318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 0x7f85700b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f857519b700 1 --2- 172.21.15.131:0/351405318 >> v2:172.21.15.190:3300/0 conn(0x7f8570159920 0x7f85700b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f8567fff700 1 -- 172.21.15.131:0/351405318 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104329342 0 0) 0x7f85700c8bf0 con 0x7f8570159cf0 2023-12-26T21:49:55.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f8567fff700 1 -- 172.21.15.131:0/351405318 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f855c003570 con 0x7f8570159cf0 2023-12-26T21:49:55.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f857599c700 1 -- 172.21.15.131:0/351405318 >> v2:172.21.15.190:3300/0 conn(0x7f8570159920 msgr2=0x7f85700b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:55.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f857599c700 1 --2- 172.21.15.131:0/351405318 >> v2:172.21.15.190:3300/0 conn(0x7f8570159920 0x7f85700b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f857599c700 1 -- 172.21.15.131:0/351405318 >> v1:172.21.15.131:6789/0 conn(0x7f8570159cf0 legacy=0x7f85701554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f857599c700 1 -- 172.21.15.131:0/351405318 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85700c9420 con 0x7f8570155be0 2023-12-26T21:49:55.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.831+0000 7f857519b700 1 --2- 172.21.15.131:0/351405318 >> v2:172.21.15.190:3300/0 conn(0x7f8570159920 0x7f85700b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f857599c700 1 --2- 172.21.15.131:0/351405318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 0x7f85700b0620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8568009e60 tx=0x7f8568009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bae18b856fb55f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f8567fff700 1 -- 172.21.15.131:0/351405318 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800e070 con 0x7f8570155be0 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f8567fff700 1 -- 172.21.15.131:0/351405318 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856800bab0 con 0x7f8570155be0 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f8567fff700 1 -- 172.21.15.131:0/351405318 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800be70 con 0x7f8570155be0 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f85773ff700 1 -- 172.21.15.131:0/351405318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 msgr2=0x7f85700b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f85773ff700 1 --2- 172.21.15.131:0/351405318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 0x7f85700b0620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8568009e60 tx=0x7f8568009780 comp rx=0 tx=0).stop 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f85773ff700 1 -- 172.21.15.131:0/351405318 shutdown_connections 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f85773ff700 1 --2- 172.21.15.131:0/351405318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 0x7f85700b0620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f85773ff700 1 --2- 172.21.15.131:0/351405318 >> v2:172.21.15.190:3300/0 conn(0x7f8570159920 0x7f85700b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f85773ff700 1 -- 172.21.15.131:0/351405318 >> 172.21.15.131:0/351405318 conn(0x7f8570068040 msgr2=0x7f8570069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f85773ff700 1 -- 172.21.15.131:0/351405318 shutdown_connections 2023-12-26T21:49:55.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.832+0000 7f85773ff700 1 -- 172.21.15.131:0/351405318 wait complete. 2023-12-26T21:49:55.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.834+0000 7f85773ff700 1 Processor -- start 2023-12-26T21:49:55.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.835+0000 7f85773ff700 1 -- start start 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f85773ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 0x7f8570167e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f85773ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8570159920 0x7f85700c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f85773ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8570159cf0 0x7f85700c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f85773ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85701409a0 con 0x7f8570159cf0 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f85773ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85700c7970 con 0x7f8570159920 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f85773ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85700c7c50 con 0x7f8570155be0 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f857599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8570159cf0 0x7f85700c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f857599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8570159cf0 0x7f85700c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33368/0 (socket says 172.21.15.131:33368) 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f857599c700 1 -- 172.21.15.131:0/2564864877 learned_addr learned my addr 172.21.15.131:0/2564864877 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f857519b700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 0x7f8570167e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.836+0000 7f857499a700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8570159920 0x7f85700c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f857599c700 1 -- 172.21.15.131:0/2564864877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 msgr2=0x7f8570167e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f857599c700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 0x7f8570167e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f857599c700 1 -- 172.21.15.131:0/2564864877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8570159920 msgr2=0x7f85700c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f857599c700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8570159920 0x7f85700c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f857599c700 1 -- 172.21.15.131:0/2564864877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c00a5d0 con 0x7f8570159cf0 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f857499a700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8570159920 0x7f85700c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f857599c700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8570159cf0 0x7f85700c4cf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f85700c9280 tx=0x7f8568019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f85657fa700 1 -- 172.21.15.131:0/2564864877 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568027470 con 0x7f8570159cf0 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f85657fa700 1 -- 172.21.15.131:0/2564864877 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85680275f0 con 0x7f8570159cf0 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8568009a20 con 0x7f8570159cf0 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f85657fa700 1 -- 172.21.15.131:0/2564864877 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85680279b0 con 0x7f8570159cf0 2023-12-26T21:49:55.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.837+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857015c600 con 0x7f8570159cf0 2023-12-26T21:49:55.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.838+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85700657e0 con 0x7f8570159cf0 2023-12-26T21:49:55.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.839+0000 7f85657fa700 1 -- 172.21.15.131:0/2564864877 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8568024070 con 0x7f8570159cf0 2023-12-26T21:49:55.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.840+0000 7f85657fa700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8554066f30 0x7f85540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:55.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.840+0000 7f85657fa700 1 -- 172.21.15.131:0/2564864877 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f8568086fc0 con 0x7f8570159cf0 2023-12-26T21:49:55.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.840+0000 7f857519b700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8554066f30 0x7f85540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:55.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.841+0000 7f857519b700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8554066f30 0x7f85540693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f856c00a5a0 tx=0x7f856c00a330 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:55.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:55.844+0000 7f85657fa700 1 -- 172.21.15.131:0/2564864877 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85680540f0 con 0x7f8570159cf0 2023-12-26T21:49:56.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.038+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f85700c9700 con 0x7f8570159cf0 2023-12-26T21:49:56.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.042+0000 7f85657fa700 1 -- 172.21.15.131:0/2564864877 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) v1 ==== 115+0+0 (secure 0 0 0) 0x7f8568057710 con 0x7f8570159cf0 2023-12-26T21:49:56.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8554066f30 msgr2=0x7f85540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8554066f30 0x7f85540693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f856c00a5a0 tx=0x7f856c00a330 comp rx=0 tx=0).stop 2023-12-26T21:49:56.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8570159cf0 msgr2=0x7f85700c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8570159cf0 0x7f85700c4cf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f85700c9280 tx=0x7f8568019000 comp rx=0 tx=0).stop 2023-12-26T21:49:56.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 shutdown_connections 2023-12-26T21:49:56.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8570159cf0 0x7f85700c4cf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8554066f30 0x7f85540693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8570155be0 0x7f8570167e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 --2- 172.21.15.131:0/2564864877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8570159920 0x7f85700c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.044+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 >> 172.21.15.131:0/2564864877 conn(0x7f8570068040 msgr2=0x7f85700aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:56.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.045+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 shutdown_connections 2023-12-26T21:49:56.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.045+0000 7f85773ff700 1 -- 172.21.15.131:0/2564864877 wait complete. 2023-12-26T21:49:56.063 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T21:49:56.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.258+0000 7fe1687ff700 1 Processor -- start 2023-12-26T21:49:56.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.258+0000 7fe1687ff700 1 -- start start 2023-12-26T21:49:56.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.259+0000 7fe1687ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013c510 0x7fe1600c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.259+0000 7fe1687ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe16006b8c0 0x7fe16006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.259+0000 7fe1687ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16006c270 con 0x7fe1600cc100 2023-12-26T21:49:56.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.259+0000 7fe1687ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1600686c0 con 0x7fe16006b8c0 2023-12-26T21:49:56.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.259+0000 7fe1687ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe160069520 con 0x7fe16013c510 2023-12-26T21:49:56.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe166d9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe1600cc100 0x7fe1600cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58998/0 (socket says 172.21.15.131:58998) 2023-12-26T21:49:56.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe166d9c700 1 -- 172.21.15.131:0/4043054973 learned_addr learned my addr 172.21.15.131:0/4043054973 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:56.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165d9a700 1 --2- 172.21.15.131:0/4043054973 >> v2:172.21.15.190:3300/0 conn(0x7fe16006b8c0 0x7fe16006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe16659b700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013c510 0x7fe1600c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628131689 0 0) 0x7fe16006c270 con 0x7fe1600cc100 2023-12-26T21:49:56.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe14c003570 con 0x7fe1600cc100 2023-12-26T21:49:56.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287555260 0 0) 0x7fe14c003570 con 0x7fe1600cc100 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013c510 msgr2=0x7fe1600c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013c510 0x7fe1600c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 >> v2:172.21.15.190:3300/0 conn(0x7fe16006b8c0 msgr2=0x7fe16006bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 --2- 172.21.15.131:0/4043054973 >> v2:172.21.15.190:3300/0 conn(0x7fe16006b8c0 0x7fe16006bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160068910 con 0x7fe1600cc100 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fe1580017f0 con 0x7fe1600cc100 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 >> v1:172.21.15.131:6789/0 conn(0x7fe1600cc100 legacy=0x7fe1600cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013c510 0x7fe14c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe14c003fe0 0x7fe14c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c004d50 0x7fe14c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1600686c0 con 0x7fe14c004d50 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe160069520 con 0x7fe14c003fe0 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.260+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c00ba90 con 0x7fe16013c510 2023-12-26T21:49:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe166d9c700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c004d50 0x7fe14c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe165d9a700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe14c003fe0 0x7fe14c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe16659b700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013c510 0x7fe14c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe165d9a700 1 -- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013c510 msgr2=0x7fe14c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe165d9a700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013c510 0x7fe14c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe165d9a700 1 -- 172.21.15.131:0/4043054973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c004d50 msgr2=0x7fe14c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe165d9a700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c004d50 0x7fe14c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe165d9a700 1 -- 172.21.15.131:0/4043054973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c003570 con 0x7fe14c003fe0 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe166d9c700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c004d50 0x7fe14c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.261+0000 7fe165d9a700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe14c003fe0 0x7fe14c004810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe150009c00 tx=0x7fe1500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ef56b90890001d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15000e710 con 0x7fe14c003fe0 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15000e870 con 0x7fe14c003fe0 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe165599700 1 -- 172.21.15.131:0/4043054973 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15000ec30 con 0x7fe14c003fe0 2023-12-26T21:49:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe1687ff700 1 -- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe14c003fe0 msgr2=0x7fe14c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe1687ff700 1 --2- 172.21.15.131:0/4043054973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe14c003fe0 0x7fe14c004810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe150009c00 tx=0x7fe1500091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe166d9c700 1 -- 172.21.15.131:0/4043054973 reap_dead start 2023-12-26T21:49:56.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe1687ff700 1 -- 172.21.15.131:0/4043054973 shutdown_connections 2023-12-26T21:49:56.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe1687ff700 1 -- 172.21.15.131:0/4043054973 >> 172.21.15.131:0/4043054973 conn(0x7fe1600ce3e0 msgr2=0x7fe1600ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:56.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe1687ff700 1 -- 172.21.15.131:0/4043054973 shutdown_connections 2023-12-26T21:49:56.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.262+0000 7fe1687ff700 1 -- 172.21.15.131:0/4043054973 wait complete. 2023-12-26T21:49:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.264+0000 7fe1687ff700 1 Processor -- start 2023-12-26T21:49:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.264+0000 7fe1687ff700 1 -- start start 2023-12-26T21:49:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe1687ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c003fe0 0x7fe1601572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe1687ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013bf50 0x7fe16016d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe1687ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe16016d650 0x7fe1601554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe1687ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16013c3c0 con 0x7fe14c003fe0 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe1687ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe160156e30 con 0x7fe16016d650 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe1687ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16016daa0 con 0x7fe16013bf50 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe16659b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c003fe0 0x7fe1601572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe16659b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c003fe0 0x7fe1601572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33382/0 (socket says 172.21.15.131:33382) 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe16659b700 1 -- 172.21.15.131:0/3180538800 learned_addr learned my addr 172.21.15.131:0/3180538800 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.265+0000 7fe16659b700 1 -- 172.21.15.131:0/3180538800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013bf50 msgr2=0x7fe16016d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe16659b700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013bf50 0x7fe16016d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe16659b700 1 -- 172.21.15.131:0/3180538800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe16016d650 msgr2=0x7fe1601554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe16659b700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe16016d650 0x7fe1601554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe16659b700 1 -- 172.21.15.131:0/3180538800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160156220 con 0x7fe14c003fe0 2023-12-26T21:49:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe16659b700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c003fe0 0x7fe1601572c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe15c011b90 tx=0x7fe15c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe1567fc700 1 -- 172.21.15.131:0/3180538800 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c005c10 con 0x7fe14c003fe0 2023-12-26T21:49:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe1567fc700 1 -- 172.21.15.131:0/3180538800 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c005d70 con 0x7fe14c003fe0 2023-12-26T21:49:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe160156500 con 0x7fe14c003fe0 2023-12-26T21:49:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe1567fc700 1 -- 172.21.15.131:0/3180538800 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00fa90 con 0x7fe14c003fe0 2023-12-26T21:49:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.266+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe160156ac0 con 0x7fe14c003fe0 2023-12-26T21:49:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.267+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1600657e0 con 0x7fe14c003fe0 2023-12-26T21:49:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.268+0000 7fe1567fc700 1 -- 172.21.15.131:0/3180538800 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe15c006050 con 0x7fe14c003fe0 2023-12-26T21:49:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.269+0000 7fe1567fc700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe144066f30 0x7fe1440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.269+0000 7fe1567fc700 1 -- 172.21.15.131:0/3180538800 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fe15c084d90 con 0x7fe14c003fe0 2023-12-26T21:49:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.270+0000 7fe165d9a700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe144066f30 0x7fe1440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.270+0000 7fe165d9a700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe144066f30 0x7fe1440693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe16015d0a0 tx=0x7fe15001a000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:56.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.273+0000 7fe1567fc700 1 -- 172.21.15.131:0/3180538800 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe15c051f60 con 0x7fe14c003fe0 2023-12-26T21:49:56.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.468+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fe160068bf0 con 0x7fe14c003fe0 2023-12-26T21:49:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.472+0000 7fe1567fc700 1 -- 172.21.15.131:0/3180538800 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v15) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe15c055580 con 0x7fe14c003fe0 2023-12-26T21:49:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe144066f30 msgr2=0x7fe1440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe144066f30 0x7fe1440693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe16015d0a0 tx=0x7fe15001a000 comp rx=0 tx=0).stop 2023-12-26T21:49:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c003fe0 msgr2=0x7fe1601572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c003fe0 0x7fe1601572c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe15c011b90 tx=0x7fe15c009120 comp rx=0 tx=0).stop 2023-12-26T21:49:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 shutdown_connections 2023-12-26T21:49:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe16016d650 0x7fe1601554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe14c003fe0 0x7fe1601572c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe144066f30 0x7fe1440693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 --2- 172.21.15.131:0/3180538800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe16013bf50 0x7fe16016d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 >> 172.21.15.131:0/3180538800 conn(0x7fe1600ce3e0 msgr2=0x7fe160159300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 shutdown_connections 2023-12-26T21:49:56.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.475+0000 7fe1687ff700 1 -- 172.21.15.131:0/3180538800 wait complete. 2023-12-26T21:49:56.503 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-12-26T21:49:56.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.693+0000 7f91b0fff700 1 Processor -- start 2023-12-26T21:49:56.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.693+0000 7f91b0fff700 1 -- start start 2023-12-26T21:49:56.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91b0fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f91ac0b5310 0x7f91ac0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91b0fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ac0afcd0 con 0x7f91ac1596a0 2023-12-26T21:49:56.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91b0fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ac0b3e70 con 0x7f91ac159a70 2023-12-26T21:49:56.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91b0fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ac0b0f20 con 0x7f91ac0b5310 2023-12-26T21:49:56.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91aad9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f91ac0b5310 0x7f91ac0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91aa59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f91ac1596a0 0x7f91ac0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59000/0 (socket says 172.21.15.131:59000) 2023-12-26T21:49:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91aa59c700 1 -- 172.21.15.131:0/2079302940 learned_addr learned my addr 172.21.15.131:0/2079302940 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91aad9d700 1 -- 172.21.15.131:0/2079302940 >> v1:172.21.15.190:6789/0 conn(0x7f91ac159a70 legacy=0x7f91ac0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91aad9d700 1 -- 172.21.15.131:0/2079302940 >> v1:172.21.15.131:6789/0 conn(0x7f91ac1596a0 legacy=0x7f91ac0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91aad9d700 1 -- 172.21.15.131:0/2079302940 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac0b3540 con 0x7f91ac0b5310 2023-12-26T21:49:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.695+0000 7f91a9d9b700 1 -- 172.21.15.131:0/2079302940 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033774894 0 0) 0x7f91ac0afcd0 con 0x7f91ac1596a0 2023-12-26T21:49:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.696+0000 7f91aad9d700 1 --2- 172.21.15.131:0/2079302940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac0b5310 0x7f91ac0b0870 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009e60 tx=0x7f91a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951e614826925b50 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.696+0000 7f91a9d9b700 1 -- 172.21.15.131:0/2079302940 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a400e070 con 0x7f91ac0b5310 2023-12-26T21:49:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.696+0000 7f91a9d9b700 1 -- 172.21.15.131:0/2079302940 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a400bc30 con 0x7f91ac0b5310 2023-12-26T21:49:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.696+0000 7f91a9d9b700 1 -- 172.21.15.131:0/2079302940 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4012430 con 0x7f91ac0b5310 2023-12-26T21:49:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.696+0000 7f91b0fff700 1 -- 172.21.15.131:0/2079302940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac0b5310 msgr2=0x7f91ac0b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.696+0000 7f91b0fff700 1 --2- 172.21.15.131:0/2079302940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac0b5310 0x7f91ac0b0870 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009e60 tx=0x7f91a4009780 comp rx=0 tx=0).stop 2023-12-26T21:49:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.696+0000 7f91b0fff700 1 -- 172.21.15.131:0/2079302940 shutdown_connections 2023-12-26T21:49:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.696+0000 7f91b0fff700 1 --2- 172.21.15.131:0/2079302940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac0b5310 0x7f91ac0b0870 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.696+0000 7f91b0fff700 1 -- 172.21.15.131:0/2079302940 >> 172.21.15.131:0/2079302940 conn(0x7f91ac06ece0 msgr2=0x7f91ac06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.697+0000 7f91b0fff700 1 -- 172.21.15.131:0/2079302940 shutdown_connections 2023-12-26T21:49:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.697+0000 7f91b0fff700 1 -- 172.21.15.131:0/2079302940 wait complete. 2023-12-26T21:49:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.698+0000 7f91b0fff700 1 Processor -- start 2023-12-26T21:49:56.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.698+0000 7f91b0fff700 1 -- start start 2023-12-26T21:49:56.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91b0fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91ac0b5310 0x7f91ac155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91b0fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1596a0 0x7f91ac0cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91b0fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ac159a70 0x7f91ac0cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91b0fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac0cc2d0 con 0x7f91ac1596a0 2023-12-26T21:49:56.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91b0fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac156070 con 0x7f91ac159a70 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91b0fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac0c8160 con 0x7f91ac0b5310 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91a3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1596a0 0x7f91ac0cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91a3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1596a0 0x7f91ac0cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33400/0 (socket says 172.21.15.131:33400) 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91a3fff700 1 -- 172.21.15.131:0/732947632 learned_addr learned my addr 172.21.15.131:0/732947632 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91aad9d700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ac159a70 0x7f91ac0cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.699+0000 7f91aa59c700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91ac0b5310 0x7f91ac155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91a3fff700 1 -- 172.21.15.131:0/732947632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91ac0b5310 msgr2=0x7f91ac155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91a3fff700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91ac0b5310 0x7f91ac155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91a3fff700 1 -- 172.21.15.131:0/732947632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ac159a70 msgr2=0x7f91ac0cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91a3fff700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ac159a70 0x7f91ac0cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91a3fff700 1 -- 172.21.15.131:0/732947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a4009a20 con 0x7f91ac1596a0 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91a3fff700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1596a0 0x7f91ac0cb200 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f919400c4e0 tx=0x7f919400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91a27fc700 1 -- 172.21.15.131:0/732947632 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9194011590 con 0x7f91ac1596a0 2023-12-26T21:49:56.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91a27fc700 1 -- 172.21.15.131:0/732947632 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9194011710 con 0x7f91ac1596a0 2023-12-26T21:49:56.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ac0cba30 con 0x7f91ac1596a0 2023-12-26T21:49:56.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91a27fc700 1 -- 172.21.15.131:0/732947632 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9194011ad0 con 0x7f91ac1596a0 2023-12-26T21:49:56.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.700+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac0cbff0 con 0x7f91ac1596a0 2023-12-26T21:49:56.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.701+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ac0657e0 con 0x7f91ac1596a0 2023-12-26T21:49:56.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.702+0000 7f91a27fc700 1 -- 172.21.15.131:0/732947632 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9194000f90 con 0x7f91ac1596a0 2023-12-26T21:49:56.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.703+0000 7f91a27fc700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f918c066f30 0x7f918c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:56.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.703+0000 7f91a27fc700 1 -- 172.21.15.131:0/732947632 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f91940839d0 con 0x7f91ac1596a0 2023-12-26T21:49:56.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.703+0000 7f91aa59c700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f918c066f30 0x7f918c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:56.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.704+0000 7f91aa59c700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f918c066f30 0x7f918c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f919c0011e0 tx=0x7f919c009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.707+0000 7f91a27fc700 1 -- 172.21.15.131:0/732947632 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9194050b00 con 0x7f91ac1596a0 2023-12-26T21:49:56.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:56.975+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f91ac0b3820 con 0x7f91ac1596a0 2023-12-26T21:49:57.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.176+0000 7f91a27fc700 1 -- 172.21.15.131:0/732947632 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v471) v1 ==== 158+0+0 (secure 0 0 0) 0x7f9194017070 con 0x7f91ac1596a0 2023-12-26T21:49:57.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f918c066f30 msgr2=0x7f918c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:57.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f918c066f30 0x7f918c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f919c0011e0 tx=0x7f919c009040 comp rx=0 tx=0).stop 2023-12-26T21:49:57.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1596a0 msgr2=0x7f91ac0cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:57.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1596a0 0x7f91ac0cb200 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f919400c4e0 tx=0x7f919400c7f0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 shutdown_connections 2023-12-26T21:49:57.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ac159a70 0x7f91ac0cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f918c066f30 0x7f918c0693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91ac0b5310 0x7f91ac155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 --2- 172.21.15.131:0/732947632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ac1596a0 0x7f91ac0cb200 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 >> 172.21.15.131:0/732947632 conn(0x7f91ac06ece0 msgr2=0x7f91ac158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:57.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 shutdown_connections 2023-12-26T21:49:57.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.178+0000 7f91b0fff700 1 -- 172.21.15.131:0/732947632 wait complete. 2023-12-26T21:49:57.197 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-12-26T21:49:57.197 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-12-26T21:49:57.197 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T21:49:57.197 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-12-26T21:49:57.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.387+0000 7f38013ff700 1 Processor -- start 2023-12-26T21:49:57.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.387+0000 7f38013ff700 1 -- start start 2023-12-26T21:49:57.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.388+0000 7f38013ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37fc06bfd0 0x7f37fc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:57.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.388+0000 7f38013ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc0cc100 0x7f37fc0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.388+0000 7f38013ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc0686c0 con 0x7f37fc13c510 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.388+0000 7f38013ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc069520 con 0x7f37fc06bfd0 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.388+0000 7f38013ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc068910 con 0x7f37fc0cc100 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.388+0000 7f37fa7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37fc06bfd0 0x7f37fc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37fa7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37fc06bfd0 0x7f37fc0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33414/0 (socket says 172.21.15.131:33414) 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37fa7fc700 1 -- 172.21.15.131:0/2624756841 learned_addr learned my addr 172.21.15.131:0/2624756841 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37fb7fe700 1 --2- 172.21.15.131:0/2624756841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc0cc100 0x7f37fc0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37fa7fc700 1 -- 172.21.15.131:0/2624756841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc0cc100 msgr2=0x7f37fc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37fa7fc700 1 --2- 172.21.15.131:0/2624756841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc0cc100 0x7f37fc0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37fa7fc700 1 -- 172.21.15.131:0/2624756841 >> v1:172.21.15.190:6789/0 conn(0x7f37fc13c510 legacy=0x7f37fc06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37fa7fc700 1 -- 172.21.15.131:0/2624756841 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc068bf0 con 0x7f37fc06bfd0 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37fb7fe700 1 --2- 172.21.15.131:0/2624756841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc0cc100 0x7f37fc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:49:57.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37fa7fc700 1 --2- 172.21.15.131:0/2624756841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 0x7f37fc0c8a20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f37f00099b0 tx=0x7f37f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df16860d031edc14 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.389+0000 7f37f9ffb700 1 -- 172.21.15.131:0/2624756841 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000b860 con 0x7f37fc06bfd0 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f37f9ffb700 1 -- 172.21.15.131:0/2624756841 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f000b9c0 con 0x7f37fc06bfd0 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f37f9ffb700 1 -- 172.21.15.131:0/2624756841 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000bd60 con 0x7f37fc06bfd0 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f38013ff700 1 -- 172.21.15.131:0/2624756841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 msgr2=0x7f37fc0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f38013ff700 1 --2- 172.21.15.131:0/2624756841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 0x7f37fc0c8a20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f37f00099b0 tx=0x7f37f0009250 comp rx=0 tx=0).stop 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f38013ff700 1 -- 172.21.15.131:0/2624756841 shutdown_connections 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f38013ff700 1 --2- 172.21.15.131:0/2624756841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc0cc100 0x7f37fc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f38013ff700 1 --2- 172.21.15.131:0/2624756841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 0x7f37fc0c8a20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f38013ff700 1 -- 172.21.15.131:0/2624756841 >> 172.21.15.131:0/2624756841 conn(0x7f37fc0ce3e0 msgr2=0x7f37fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f38013ff700 1 -- 172.21.15.131:0/2624756841 shutdown_connections 2023-12-26T21:49:57.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.390+0000 7f38013ff700 1 -- 172.21.15.131:0/2624756841 wait complete. 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.392+0000 7f38013ff700 1 Processor -- start 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.392+0000 7f38013ff700 1 -- start start 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f38013ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 0x7f37fc13bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f38013ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37fc0cc100 0x7f37fc0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f38013ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc13c510 0x7f37fc0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f38013ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc154ba0 con 0x7f37fc06bfd0 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f38013ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc154e80 con 0x7f37fc0cc100 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f38013ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc155160 con 0x7f37fc13c510 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 0x7f37fc13bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 0x7f37fc13bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33420/0 (socket says 172.21.15.131:33420) 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37faffd700 1 -- 172.21.15.131:0/4279350418 learned_addr learned my addr 172.21.15.131:0/4279350418 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37fa7fc700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37fc0cc100 0x7f37fc0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37fb7fe700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc13c510 0x7f37fc0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37faffd700 1 -- 172.21.15.131:0/4279350418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc13c510 msgr2=0x7f37fc0c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:57.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37faffd700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc13c510 0x7f37fc0c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37faffd700 1 -- 172.21.15.131:0/4279350418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37fc0cc100 msgr2=0x7f37fc0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:57.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37faffd700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37fc0cc100 0x7f37fc0b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:57.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.393+0000 7f37faffd700 1 -- 172.21.15.131:0/4279350418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00094f0 con 0x7f37fc06bfd0 2023-12-26T21:49:57.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.394+0000 7f37faffd700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 0x7f37fc13bce0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c7c0 tx=0x7f37ec007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:57.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.394+0000 7f37e2ffd700 1 -- 172.21.15.131:0/4279350418 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011590 con 0x7f37fc06bfd0 2023-12-26T21:49:57.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.394+0000 7f37e2ffd700 1 -- 172.21.15.131:0/4279350418 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec011710 con 0x7f37fc06bfd0 2023-12-26T21:49:57.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.394+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc155440 con 0x7f37fc06bfd0 2023-12-26T21:49:57.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.394+0000 7f37e2ffd700 1 -- 172.21.15.131:0/4279350418 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011ad0 con 0x7f37fc06bfd0 2023-12-26T21:49:57.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.394+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc13c220 con 0x7f37fc06bfd0 2023-12-26T21:49:57.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.395+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc1558e0 con 0x7f37fc06bfd0 2023-12-26T21:49:57.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.396+0000 7f37e2ffd700 1 -- 172.21.15.131:0/4279350418 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f37ec01b3b0 con 0x7f37fc06bfd0 2023-12-26T21:49:57.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.397+0000 7f37e2ffd700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f37e8066f30 0x7f37e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:57.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.397+0000 7f37e2ffd700 1 -- 172.21.15.131:0/4279350418 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f37ec083b70 con 0x7f37fc06bfd0 2023-12-26T21:49:57.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.397+0000 7f37fa7fc700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f37e8066f30 0x7f37e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:57.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.398+0000 7f37fa7fc700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f37e8066f30 0x7f37e80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f37f00099b0 tx=0x7f37f0017000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:57.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.401+0000 7f37e2ffd700 1 -- 172.21.15.131:0/4279350418 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37ec050d50 con 0x7f37fc06bfd0 2023-12-26T21:49:57.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:57.668+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f37fc155a70 con 0x7f37fc06bfd0 2023-12-26T21:49:58.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.179+0000 7f37e2ffd700 1 -- 172.21.15.131:0/4279350418 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v472) v1 ==== 157+0+0 (secure 0 0 0) 0x7f37ec017070 con 0x7f37fc06bfd0 2023-12-26T21:49:58.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f37e8066f30 msgr2=0x7f37e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f37e8066f30 0x7f37e80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f37f00099b0 tx=0x7f37f0017000 comp rx=0 tx=0).stop 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 msgr2=0x7f37fc13bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 0x7f37fc13bce0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c7c0 tx=0x7f37ec007f70 comp rx=0 tx=0).stop 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 shutdown_connections 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f37e8066f30 0x7f37e80693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37fc13c510 0x7f37fc0c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37fc06bfd0 0x7f37fc13bce0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 --2- 172.21.15.131:0/4279350418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37fc0cc100 0x7f37fc0b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 >> 172.21.15.131:0/4279350418 conn(0x7f37fc0ce3e0 msgr2=0x7f37fc06d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 shutdown_connections 2023-12-26T21:49:58.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.182+0000 7f38013ff700 1 -- 172.21.15.131:0/4279350418 wait complete. 2023-12-26T21:49:58.200 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T21:49:58.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.393+0000 7f90cafff700 1 Processor -- start 2023-12-26T21:49:58.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.393+0000 7f90cafff700 1 -- start start 2023-12-26T21:49:58.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90cafff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f90c40cc950 0x7f90c406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90cafff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c406eb30 0x7f90c415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90cafff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c40b07f0 con 0x7f90c40cc580 2023-12-26T21:49:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90cafff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c40680f0 con 0x7f90c40cc950 2023-12-26T21:49:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90cafff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40b0d70 con 0x7f90c406eb30 2023-12-26T21:49:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90c959c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c406eb30 0x7f90c415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90c3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f90c40cc950 0x7f90c406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90c959c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c406eb30 0x7f90c415d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:42686/0 (socket says 172.21.15.131:42686) 2023-12-26T21:49:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90c8d9b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f90c40cc580 0x7f90c40b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:38380/0 (socket says 172.21.15.131:38380) 2023-12-26T21:49:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.395+0000 7f90c959c700 1 -- 172.21.15.131:0/1655289365 learned_addr learned my addr 172.21.15.131:0/1655289365 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.396+0000 7f90c3fff700 1 -- 172.21.15.131:0/1655289365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c406eb30 msgr2=0x7f90c415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.396+0000 7f90c3fff700 1 --2- 172.21.15.131:0/1655289365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c406eb30 0x7f90c415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.396+0000 7f90c3fff700 1 -- 172.21.15.131:0/1655289365 >> v1:172.21.15.190:6789/0 conn(0x7f90c40cc580 legacy=0x7f90c40b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.396+0000 7f90c3fff700 1 -- 172.21.15.131:0/1655289365 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c40675b0 con 0x7f90c40cc950 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.396+0000 7f90c959c700 1 --2- 172.21.15.131:0/1655289365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c406eb30 0x7f90c415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.396+0000 7f90c3fff700 1 --2- 172.21.15.131:0/1655289365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40cc950 0x7f90c406e5f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90b00098e0 tx=0x7f90b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdce540cf2a9c2a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.396+0000 7f90c37fe700 1 -- 172.21.15.131:0/1655289365 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b000b640 con 0x7f90c40cc950 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.396+0000 7f90c37fe700 1 -- 172.21.15.131:0/1655289365 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b000b7a0 con 0x7f90c40cc950 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.396+0000 7f90c37fe700 1 -- 172.21.15.131:0/1655289365 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b000bb40 con 0x7f90c40cc950 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.397+0000 7f90cafff700 1 -- 172.21.15.131:0/1655289365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40cc950 msgr2=0x7f90c406e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.397+0000 7f90cafff700 1 --2- 172.21.15.131:0/1655289365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40cc950 0x7f90c406e5f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90b00098e0 tx=0x7f90b00091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.397+0000 7f90cafff700 1 -- 172.21.15.131:0/1655289365 shutdown_connections 2023-12-26T21:49:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.397+0000 7f90cafff700 1 --2- 172.21.15.131:0/1655289365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c406eb30 0x7f90c415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.397+0000 7f90cafff700 1 --2- 172.21.15.131:0/1655289365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c40cc950 0x7f90c406e5f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.397+0000 7f90cafff700 1 -- 172.21.15.131:0/1655289365 >> 172.21.15.131:0/1655289365 conn(0x7f90c40686c0 msgr2=0x7f90c4068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.397+0000 7f90cafff700 1 -- 172.21.15.131:0/1655289365 shutdown_connections 2023-12-26T21:49:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.397+0000 7f90cafff700 1 -- 172.21.15.131:0/1655289365 wait complete. 2023-12-26T21:49:58.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.398+0000 7f90cafff700 1 Processor -- start 2023-12-26T21:49:58.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.399+0000 7f90cafff700 1 -- start start 2023-12-26T21:49:58.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.399+0000 7f90cafff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c406eb30 0x7f90c4154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.399+0000 7f90cafff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c40cc580 0x7f90c4156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.399+0000 7f90cafff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40cc950 0x7f90c4156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.399+0000 7f90cafff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c415e2a0 con 0x7f90c40cc950 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.399+0000 7f90cafff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c4155470 con 0x7f90c406eb30 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.399+0000 7f90cafff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c415ac90 con 0x7f90c40cc580 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c959c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40cc950 0x7f90c4156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c959c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40cc950 0x7f90c4156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33432/0 (socket says 172.21.15.131:33432) 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c959c700 1 -- 172.21.15.131:0/1503283636 learned_addr learned my addr 172.21.15.131:0/1503283636 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c8d9b700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c406eb30 0x7f90c4154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c3fff700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c40cc580 0x7f90c4156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c959c700 1 -- 172.21.15.131:0/1503283636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c40cc580 msgr2=0x7f90c4156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c959c700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c40cc580 0x7f90c4156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c959c700 1 -- 172.21.15.131:0/1503283636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c406eb30 msgr2=0x7f90c4154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:58.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c959c700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c406eb30 0x7f90c4154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c959c700 1 -- 172.21.15.131:0/1503283636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc009ae0 con 0x7f90c40cc950 2023-12-26T21:49:58.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.400+0000 7f90c959c700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40cc950 0x7f90c4156ed0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009f30 tx=0x7f90bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.401+0000 7f90c0ff9700 1 -- 172.21.15.131:0/1503283636 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc018070 con 0x7f90c40cc950 2023-12-26T21:49:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.401+0000 7f90c0ff9700 1 -- 172.21.15.131:0/1503283636 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc00f040 con 0x7f90c40cc950 2023-12-26T21:49:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.401+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b0009420 con 0x7f90c40cc950 2023-12-26T21:49:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.401+0000 7f90c0ff9700 1 -- 172.21.15.131:0/1503283636 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00cb70 con 0x7f90c40cc950 2023-12-26T21:49:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.401+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c415dc50 con 0x7f90c40cc950 2023-12-26T21:49:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.402+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c415e580 con 0x7f90c40cc950 2023-12-26T21:49:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.403+0000 7f90c0ff9700 1 -- 172.21.15.131:0/1503283636 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f90bc003020 con 0x7f90c40cc950 2023-12-26T21:49:58.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.404+0000 7f90c0ff9700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90a8066f30 0x7f90a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:58.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.404+0000 7f90c0ff9700 1 -- 172.21.15.131:0/1503283636 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f90bc084a10 con 0x7f90c40cc950 2023-12-26T21:49:58.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.404+0000 7f90c8d9b700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90a8066f30 0x7f90a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:58.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.404+0000 7f90c8d9b700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90a8066f30 0x7f90a80693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f90b8004fd0 tx=0x7f90b8007eb0 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.407+0000 7f90c0ff9700 1 -- 172.21.15.131:0/1503283636 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90bc051b40 con 0x7f90c40cc950 2023-12-26T21:49:58.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.599+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f90c40657e0 con 0x7f90c40cc950 2023-12-26T21:49:58.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.605+0000 7f90c0ff9700 1 -- 172.21.15.131:0/1503283636 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f90bc055160 con 0x7f90c40cc950 2023-12-26T21:49:58.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.607+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90a8066f30 msgr2=0x7f90a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:58.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.607+0000 7f90cafff700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90a8066f30 0x7f90a80693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f90b8004fd0 tx=0x7f90b8007eb0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.607+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40cc950 msgr2=0x7f90c4156ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:58.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.607+0000 7f90cafff700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40cc950 0x7f90c4156ed0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009f30 tx=0x7f90bc00b040 comp rx=0 tx=0).stop 2023-12-26T21:49:58.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.608+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 shutdown_connections 2023-12-26T21:49:58.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.608+0000 7f90cafff700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90c40cc950 0x7f90c4156ed0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.608+0000 7f90cafff700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90c406eb30 0x7f90c4154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.608+0000 7f90cafff700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f90a8066f30 0x7f90a80693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.608+0000 7f90cafff700 1 --2- 172.21.15.131:0/1503283636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90c40cc580 0x7f90c4156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.608+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 >> 172.21.15.131:0/1503283636 conn(0x7f90c40686c0 msgr2=0x7f90c40b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.608+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 shutdown_connections 2023-12-26T21:49:58.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:58.608+0000 7f90cafff700 1 -- 172.21.15.131:0/1503283636 wait complete. 2023-12-26T21:49:58.626 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:49:58.626 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:49:58.724 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:49:58.815 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:49:58.815 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T21:49:58.815 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:49:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.003+0000 7f6597dff700 1 Processor -- start 2023-12-26T21:49:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.003+0000 7f6597dff700 1 -- start start 2023-12-26T21:49:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.005+0000 7f6597dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f65901596a0 0x7f65900b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.005+0000 7f6597dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65900afcd0 con 0x7f6590159a70 2023-12-26T21:49:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.005+0000 7f6597dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65900b3e70 con 0x7f65900b0870 2023-12-26T21:49:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.005+0000 7f6597dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65900b0cb0 con 0x7f65901596a0 2023-12-26T21:49:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.005+0000 7f659539a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6590159a70 0x7f65900b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59008/0 (socket says 172.21.15.131:59008) 2023-12-26T21:49:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.005+0000 7f659539a700 1 -- 172.21.15.131:0/396690977 learned_addr learned my addr 172.21.15.131:0/396690977 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.005+0000 7f6595b9b700 1 --2- 172.21.15.131:0/396690977 >> v2:172.21.15.190:3300/0 conn(0x7f65901596a0 0x7f65900b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.005+0000 7f6594b99700 1 -- 172.21.15.131:0/396690977 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469249921 0 0) 0x7f65900afcd0 con 0x7f6590159a70 2023-12-26T21:49:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.005+0000 7f6594b99700 1 -- 172.21.15.131:0/396690977 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f657c003570 con 0x7f6590159a70 2023-12-26T21:49:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6594b99700 1 -- 172.21.15.131:0/396690977 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855910945 0 0) 0x7f65900b3e70 con 0x7f65900b0870 2023-12-26T21:49:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6594b99700 1 -- 172.21.15.131:0/396690977 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65900afcd0 con 0x7f65900b0870 2023-12-26T21:49:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6594b99700 1 -- 172.21.15.131:0/396690977 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3542332081 0 0) 0x7f657c003570 con 0x7f6590159a70 2023-12-26T21:49:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6595b9b700 1 -- 172.21.15.131:0/396690977 >> v1:172.21.15.190:6789/0 conn(0x7f65900b0870 legacy=0x7f65900b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6595b9b700 1 -- 172.21.15.131:0/396690977 >> v1:172.21.15.131:6789/0 conn(0x7f6590159a70 legacy=0x7f65900b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6595b9b700 1 -- 172.21.15.131:0/396690977 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65900b0f20 con 0x7f65901596a0 2023-12-26T21:49:59.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6595b9b700 1 --2- 172.21.15.131:0/396690977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65901596a0 0x7f65900b2380 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f658c0098e0 tx=0x7f658c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37d7c29b8cf49007 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:59.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6594b99700 1 -- 172.21.15.131:0/396690977 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580001470 con 0x7f65901596a0 2023-12-26T21:49:59.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6594b99700 1 -- 172.21.15.131:0/396690977 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c011420 con 0x7f65901596a0 2023-12-26T21:49:59.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.006+0000 7f6594b99700 1 -- 172.21.15.131:0/396690977 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c0117c0 con 0x7f65901596a0 2023-12-26T21:49:59.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.007+0000 7f6597dff700 1 -- 172.21.15.131:0/396690977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65901596a0 msgr2=0x7f65900b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.007+0000 7f6597dff700 1 --2- 172.21.15.131:0/396690977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65901596a0 0x7f65900b2380 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f658c0098e0 tx=0x7f658c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.007+0000 7f6597dff700 1 -- 172.21.15.131:0/396690977 shutdown_connections 2023-12-26T21:49:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.007+0000 7f6597dff700 1 --2- 172.21.15.131:0/396690977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65901596a0 0x7f65900b2380 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.007+0000 7f6597dff700 1 -- 172.21.15.131:0/396690977 >> 172.21.15.131:0/396690977 conn(0x7f659006ece0 msgr2=0x7f659006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.007+0000 7f6597dff700 1 -- 172.21.15.131:0/396690977 shutdown_connections 2023-12-26T21:49:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.007+0000 7f6597dff700 1 -- 172.21.15.131:0/396690977 wait complete. 2023-12-26T21:49:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.008+0000 7f6597dff700 1 Processor -- start 2023-12-26T21:49:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.008+0000 7f6597dff700 1 -- start start 2023-12-26T21:49:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.009+0000 7f6597dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65900b0870 0x7f659015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.009+0000 7f6597dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65901596a0 0x7f65900c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.009+0000 7f6597dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6590159a70 0x7f6590156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.009+0000 7f6597dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659015b2d0 con 0x7f65901596a0 2023-12-26T21:49:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.009+0000 7f6597dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65900c85e0 con 0x7f6590159a70 2023-12-26T21:49:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.009+0000 7f6597dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659015a6c0 con 0x7f65900b0870 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.009+0000 7f659539a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65901596a0 0x7f65900c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659539a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65901596a0 0x7f65900c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33444/0 (socket says 172.21.15.131:33444) 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659539a700 1 -- 172.21.15.131:0/1953078419 learned_addr learned my addr 172.21.15.131:0/1953078419 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f6595b9b700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65900b0870 0x7f659015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659639c700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6590159a70 0x7f6590156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659539a700 1 -- 172.21.15.131:0/1953078419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65900b0870 msgr2=0x7f659015a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659539a700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65900b0870 0x7f659015a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659539a700 1 -- 172.21.15.131:0/1953078419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6590159a70 msgr2=0x7f6590156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659539a700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6590159a70 0x7f6590156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659539a700 1 -- 172.21.15.131:0/1953078419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c009420 con 0x7f65901596a0 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659639c700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6590159a70 0x7f6590156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f659539a700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65901596a0 0x7f65900c80a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f658000cba0 tx=0x7f6580001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f6585ffb700 1 -- 172.21.15.131:0/1953078419 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580001d20 con 0x7f65901596a0 2023-12-26T21:49:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f6585ffb700 1 -- 172.21.15.131:0/1953078419 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6580007360 con 0x7f65901596a0 2023-12-26T21:49:59.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659015a9a0 con 0x7f65901596a0 2023-12-26T21:49:59.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.010+0000 7f6585ffb700 1 -- 172.21.15.131:0/1953078419 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580004830 con 0x7f65901596a0 2023-12-26T21:49:59.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.011+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659015af60 con 0x7f65901596a0 2023-12-26T21:49:59.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.012+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659015b480 con 0x7f65901596a0 2023-12-26T21:49:59.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.012+0000 7f6585ffb700 1 -- 172.21.15.131:0/1953078419 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6580004ba0 con 0x7f65901596a0 2023-12-26T21:49:59.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.014+0000 7f6585ffb700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6574066f30 0x7f65740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.014+0000 7f6585ffb700 1 -- 172.21.15.131:0/1953078419 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f6580083930 con 0x7f65901596a0 2023-12-26T21:49:59.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.014+0000 7f6595b9b700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6574066f30 0x7f65740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.015+0000 7f6595b9b700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6574066f30 0x7f65740693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f658c0093f0 tx=0x7f658c016000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:59.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.018+0000 7f6585ffb700 1 -- 172.21.15.131:0/1953078419 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f659015b480 con 0x7f65901596a0 2023-12-26T21:49:59.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.257+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f659015b480 con 0x7f65901596a0 2023-12-26T21:49:59.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.258+0000 7f6585ffb700 1 -- 172.21.15.131:0/1953078419 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f659015b480 con 0x7f65901596a0 2023-12-26T21:49:59.259 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:49:59.259 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1410949426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1410949426},{"type":"v1","addr":"172.21.15.190:6835","nonce":1410949426}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2210886653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2210886653},{"type":"v1","addr":"172.21.15.131:6836","nonce":2210886653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4878,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/241955638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":241955638},{"type":"v1","addr":"172.21.15.131:6837","nonce":241955638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/4173131799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4173131799},{"type":"v1","addr":"172.21.15.190:6833","nonce":4173131799}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-12-26T21:49:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6574066f30 msgr2=0x7f65740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6574066f30 0x7f65740693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f658c0093f0 tx=0x7f658c016000 comp rx=0 tx=0).stop 2023-12-26T21:49:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65901596a0 msgr2=0x7f65900c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65901596a0 0x7f65900c80a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f658000cba0 tx=0x7f6580001ab0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 shutdown_connections 2023-12-26T21:49:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6590159a70 0x7f6590156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6574066f30 0x7f65740693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65900b0870 0x7f659015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 --2- 172.21.15.131:0/1953078419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65901596a0 0x7f65900c80a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 >> 172.21.15.131:0/1953078419 conn(0x7f659006ece0 msgr2=0x7f65901576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 shutdown_connections 2023-12-26T21:49:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.260+0000 7f6597dff700 1 -- 172.21.15.131:0/1953078419 wait complete. 2023-12-26T21:49:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 19 2023-12-26T21:49:59.279 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T21:49:59.279 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T21:49:59.280 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T21:49:59.280 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:49:59.280 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T21:49:59.280 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T21:49:59.280 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T21:49:59.280 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:49:59.280 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T21:49:59.281 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T21:49:59.281 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T21:49:59.281 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:49:59.281 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T21:49:59.281 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T21:49:59.282 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T21:49:59.282 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:49:59.296 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T21:49:59.296 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T21:49:59.297 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T21:49:59.297 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T21:49:59.298 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T21:49:59.299 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T21:49:59.299 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T21:49:59.299 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T21:49:59.342 INFO:tasks.ceph.mds.a:Started 2023-12-26T21:49:59.342 DEBUG:teuthology.parallel:result is None 2023-12-26T21:49:59.343 INFO:tasks.ceph.mds.d:Started 2023-12-26T21:49:59.344 DEBUG:teuthology.parallel:result is None 2023-12-26T21:49:59.344 INFO:tasks.ceph.mds.c:Started 2023-12-26T21:49:59.344 DEBUG:teuthology.parallel:result is None 2023-12-26T21:49:59.346 INFO:tasks.ceph.mds.b:Started 2023-12-26T21:49:59.347 DEBUG:teuthology.parallel:result is None 2023-12-26T21:49:59.347 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T21:49:59.488 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:49:59.488 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T21:49:59.489 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:49:59.489 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T21:49:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.554+0000 7f9c9f5ff700 1 Processor -- start 2023-12-26T21:49:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.554+0000 7f9c9f5ff700 1 -- start start 2023-12-26T21:49:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9f5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c98159820 0x7f9c980abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9f5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9c9806e480 0x7f9c980675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9f5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9815a8c0 con 0x7f9c980aa480 2023-12-26T21:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9f5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9815ad50 con 0x7f9c9806e480 2023-12-26T21:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9f5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98159f90 con 0x7f9c98159820 2023-12-26T21:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9db9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9c9806e480 0x7f9c980675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9cb9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9c980aa480 0x7f9c9806dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59034/0 (socket says 172.21.15.131:59034) 2023-12-26T21:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9cb9a700 1 -- 172.21.15.131:0/3818290946 learned_addr learned my addr 172.21.15.131:0/3818290946 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9d39b700 1 --2- 172.21.15.131:0/3818290946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c98159820 0x7f9c980abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9db9c700 1 -- 172.21.15.131:0/3818290946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c98159820 msgr2=0x7f9c980abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9db9c700 1 --2- 172.21.15.131:0/3818290946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c98159820 0x7f9c980abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.556+0000 7f9c9db9c700 1 -- 172.21.15.131:0/3818290946 >> v1:172.21.15.131:6789/0 conn(0x7f9c980aa480 legacy=0x7f9c9806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.557+0000 7f9c9db9c700 1 -- 172.21.15.131:0/3818290946 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9815a270 con 0x7f9c9806e480 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.557+0000 7f9c9db9c700 1 --2- 172.21.15.131:0/3818290946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9806e480 0x7f9c980675b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009e60 tx=0x7f9c90009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1415ce6cf7224d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.557+0000 7f9c8ffff700 1 -- 172.21.15.131:0/3818290946 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000e070 con 0x7f9c9806e480 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.557+0000 7f9c8ffff700 1 -- 172.21.15.131:0/3818290946 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9000bb60 con 0x7f9c9806e480 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.557+0000 7f9c8ffff700 1 -- 172.21.15.131:0/3818290946 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90012430 con 0x7f9c9806e480 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.557+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3818290946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9806e480 msgr2=0x7f9c980675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.557+0000 7f9c9f5ff700 1 --2- 172.21.15.131:0/3818290946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9806e480 0x7f9c980675b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009e60 tx=0x7f9c90009780 comp rx=0 tx=0).stop 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.558+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3818290946 shutdown_connections 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.558+0000 7f9c9f5ff700 1 --2- 172.21.15.131:0/3818290946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9806e480 0x7f9c980675b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.558+0000 7f9c9f5ff700 1 --2- 172.21.15.131:0/3818290946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c98159820 0x7f9c980abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.558+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3818290946 >> 172.21.15.131:0/3818290946 conn(0x7f9c9806c290 msgr2=0x7f9c980ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.558+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3818290946 shutdown_connections 2023-12-26T21:49:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.558+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3818290946 wait complete. 2023-12-26T21:49:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.559+0000 7f9c9f5ff700 1 Processor -- start 2023-12-26T21:49:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.559+0000 7f9c9f5ff700 1 -- start start 2023-12-26T21:49:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.560+0000 7f9c9f5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9806e480 0x7f9c980c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.560+0000 7f9c9f5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c980aa480 0x7f9c981498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.560+0000 7f9c9f5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c98159820 0x7f9c9815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.560+0000 7f9c9f5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9815c7a0 con 0x7f9c98159820 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.560+0000 7f9c9f5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9815e490 con 0x7f9c9806e480 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.560+0000 7f9c9f5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980c28b0 con 0x7f9c980aa480 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.560+0000 7f9c9db9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c98159820 0x7f9c9815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.560+0000 7f9c9db9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c98159820 0x7f9c9815df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33498/0 (socket says 172.21.15.131:33498) 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.560+0000 7f9c9db9c700 1 -- 172.21.15.131:0/3000084197 learned_addr learned my addr 172.21.15.131:0/3000084197 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9d39b700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9806e480 0x7f9c980c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9cb9a700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c980aa480 0x7f9c981498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9db9c700 1 -- 172.21.15.131:0/3000084197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c980aa480 msgr2=0x7f9c981498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9db9c700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c980aa480 0x7f9c981498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9db9c700 1 -- 172.21.15.131:0/3000084197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9806e480 msgr2=0x7f9c980c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9db9c700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9806e480 0x7f9c980c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9db9c700 1 -- 172.21.15.131:0/3000084197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c90009a20 con 0x7f9c98159820 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9d39b700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9806e480 0x7f9c980c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:49:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9db9c700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c98159820 0x7f9c9815df50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9c90013000 tx=0x7f9c90019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:59.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c8d7fa700 1 -- 172.21.15.131:0/3000084197 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90027470 con 0x7f9c98159820 2023-12-26T21:49:59.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c8d7fa700 1 -- 172.21.15.131:0/3000084197 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c900275f0 con 0x7f9c98159820 2023-12-26T21:49:59.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9815be70 con 0x7f9c98159820 2023-12-26T21:49:59.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.561+0000 7f9c8d7fa700 1 -- 172.21.15.131:0/3000084197 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c900279b0 con 0x7f9c98159820 2023-12-26T21:49:59.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.562+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9815c430 con 0x7f9c98159820 2023-12-26T21:49:59.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.563+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c980657e0 con 0x7f9c98159820 2023-12-26T21:49:59.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.563+0000 7f9c8d7fa700 1 -- 172.21.15.131:0/3000084197 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9c90024070 con 0x7f9c98159820 2023-12-26T21:49:59.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.565+0000 7f9c8d7fa700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9c7c066f30 0x7f9c7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:49:59.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.565+0000 7f9c8d7fa700 1 -- 172.21.15.131:0/3000084197 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f9c900870a0 con 0x7f9c98159820 2023-12-26T21:49:59.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.565+0000 7f9c9d39b700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9c7c066f30 0x7f9c7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:49:59.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.566+0000 7f9c9d39b700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9c7c066f30 0x7f9c7c0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c980cca40 tx=0x7f9c94008040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:49:59.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.569+0000 7f9c8d7fa700 1 -- 172.21.15.131:0/3000084197 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c900541d0 con 0x7f9c98159820 2023-12-26T21:49:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:49:59.780+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9c9815a550 con 0x7f9c98159820 2023-12-26T21:50:00.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.634+0000 7f9c8d7fa700 1 -- 172.21.15.131:0/3000084197 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9c900577f0 con 0x7f9c98159820 2023-12-26T21:50:00.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9c7c066f30 msgr2=0x7f9c7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:00.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9c7c066f30 0x7f9c7c0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c980cca40 tx=0x7f9c94008040 comp rx=0 tx=0).stop 2023-12-26T21:50:00.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c98159820 msgr2=0x7f9c9815df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:00.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c98159820 0x7f9c9815df50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9c90013000 tx=0x7f9c90019000 comp rx=0 tx=0).stop 2023-12-26T21:50:00.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 shutdown_connections 2023-12-26T21:50:00.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c98159820 0x7f9c9815df50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:00.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9806e480 0x7f9c980c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:00.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9c7c066f30 0x7f9c7c0693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:00.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 --2- 172.21.15.131:0/3000084197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c980aa480 0x7f9c981498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:00.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 >> 172.21.15.131:0/3000084197 conn(0x7f9c9806c290 msgr2=0x7f9c98157380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:00.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 shutdown_connections 2023-12-26T21:50:00.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.638+0000 7f9c9f5ff700 1 -- 172.21.15.131:0/3000084197 wait complete. 2023-12-26T21:50:00.640 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T21:50:00.659 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T21:50:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.854+0000 7f88377ff700 1 Processor -- start 2023-12-26T21:50:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.854+0000 7f88377ff700 1 -- start start 2023-12-26T21:50:00.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.855+0000 7f88377ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f88300b0750 0x7f88300b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:00.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.855+0000 7f88377ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8830156940 con 0x7f88300cb320 2023-12-26T21:50:00.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.855+0000 7f88377ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88300ae500 con 0x7f88300c9400 2023-12-26T21:50:00.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.855+0000 7f88377ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88300c8cc0 con 0x7f88300b0750 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.855+0000 7f8834d9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f88300b0750 0x7f88300b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.856+0000 7f883559b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f88300cb320 0x7f8830069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59050/0 (socket says 172.21.15.131:59050) 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.856+0000 7f883559b700 1 -- 172.21.15.131:0/2710719239 learned_addr learned my addr 172.21.15.131:0/2710719239 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.856+0000 7f8834d9a700 1 -- 172.21.15.131:0/2710719239 >> v1:172.21.15.190:6789/0 conn(0x7f88300c9400 legacy=0x7f88300c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.856+0000 7f8834d9a700 1 -- 172.21.15.131:0/2710719239 >> v1:172.21.15.131:6789/0 conn(0x7f88300cb320 legacy=0x7f8830069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.856+0000 7f8834d9a700 1 -- 172.21.15.131:0/2710719239 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88300b0280 con 0x7f88300b0750 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.856+0000 7f8834d9a700 1 --2- 172.21.15.131:0/2710719239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300b0750 0x7f88300b0bf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f88200098e0 tx=0x7f88200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5439857bc7f066d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.856+0000 7f8827fff700 1 -- 172.21.15.131:0/2710719239 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820009e80 con 0x7f88300b0750 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.857+0000 7f8827fff700 1 -- 172.21.15.131:0/2710719239 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882000b860 con 0x7f88300b0750 2023-12-26T21:50:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.857+0000 7f8827fff700 1 -- 172.21.15.131:0/2710719239 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882000bbe0 con 0x7f88300b0750 2023-12-26T21:50:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.857+0000 7f88377ff700 1 -- 172.21.15.131:0/2710719239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300b0750 msgr2=0x7f88300b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.857+0000 7f88377ff700 1 --2- 172.21.15.131:0/2710719239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300b0750 0x7f88300b0bf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f88200098e0 tx=0x7f88200091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.857+0000 7f88377ff700 1 -- 172.21.15.131:0/2710719239 shutdown_connections 2023-12-26T21:50:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.857+0000 7f88377ff700 1 --2- 172.21.15.131:0/2710719239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300b0750 0x7f88300b0bf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.857+0000 7f88377ff700 1 -- 172.21.15.131:0/2710719239 >> 172.21.15.131:0/2710719239 conn(0x7f8830068040 msgr2=0x7f8830069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.857+0000 7f88377ff700 1 -- 172.21.15.131:0/2710719239 shutdown_connections 2023-12-26T21:50:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.857+0000 7f88377ff700 1 -- 172.21.15.131:0/2710719239 wait complete. 2023-12-26T21:50:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.859+0000 7f88377ff700 1 Processor -- start 2023-12-26T21:50:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.859+0000 7f88377ff700 1 -- start start 2023-12-26T21:50:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f88377ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88300b0750 0x7f883016a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f88377ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88300c9400 0x7f88301585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f88377ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300cb320 0x7f88301423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f88377ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8830158af0 con 0x7f88300cb320 2023-12-26T21:50:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f88377ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883016aed0 con 0x7f88300c9400 2023-12-26T21:50:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f88377ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88301595c0 con 0x7f88300b0750 2023-12-26T21:50:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f8835d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300cb320 0x7f88301423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f8835d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300cb320 0x7f88301423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33510/0 (socket says 172.21.15.131:33510) 2023-12-26T21:50:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f8835d9c700 1 -- 172.21.15.131:0/1402761532 learned_addr learned my addr 172.21.15.131:0/1402761532 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f883559b700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88300b0750 0x7f883016a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f8834d9a700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88300c9400 0x7f88301585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f8835d9c700 1 -- 172.21.15.131:0/1402761532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88300b0750 msgr2=0x7f883016a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f8835d9c700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88300b0750 0x7f883016a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f8835d9c700 1 -- 172.21.15.131:0/1402761532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88300c9400 msgr2=0x7f88301585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f8835d9c700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88300c9400 0x7f88301585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f8835d9c700 1 -- 172.21.15.131:0/1402761532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8820009420 con 0x7f88300cb320 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.860+0000 7f883559b700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88300b0750 0x7f883016a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.861+0000 7f8835d9c700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300cb320 0x7f88301423a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f882800c8e0 tx=0x7f8828001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.861+0000 7f88257fa700 1 -- 172.21.15.131:0/1402761532 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828012630 con 0x7f88300cb320 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.861+0000 7f88257fa700 1 -- 172.21.15.131:0/1402761532 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8828012790 con 0x7f88300cb320 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.861+0000 7f88257fa700 1 -- 172.21.15.131:0/1402761532 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828012b50 con 0x7f88300cb320 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.861+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8830158d20 con 0x7f88300cb320 2023-12-26T21:50:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.861+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88301592e0 con 0x7f88300cb320 2023-12-26T21:50:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.862+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88300657e0 con 0x7f88300cb320 2023-12-26T21:50:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.863+0000 7f88257fa700 1 -- 172.21.15.131:0/1402761532 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f882800a3b0 con 0x7f88300cb320 2023-12-26T21:50:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.864+0000 7f88257fa700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8814066f30 0x7f88140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.864+0000 7f88257fa700 1 -- 172.21.15.131:0/1402761532 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5217+0+0 (secure 0 0 0) 0x7f8828084650 con 0x7f88300cb320 2023-12-26T21:50:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.864+0000 7f883559b700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8814066f30 0x7f88140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.865+0000 7f883559b700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8814066f30 0x7f88140693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f88300c1160 tx=0x7f882c009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:00.868+0000 7f88257fa700 1 -- 172.21.15.131:0/1402761532 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8828055490 con 0x7f88300cb320 2023-12-26T21:50:01.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.132+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f88300b1550 con 0x7f88300cb320 2023-12-26T21:50:01.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.133+0000 7f88257fa700 1 -- 172.21.15.131:0/1402761532 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8828055020 con 0x7f88300cb320 2023-12-26T21:50:01.133 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:01.133 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:50:01.133 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.134 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.136 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.137 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:50:01.138 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T21:50:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T21:50:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:50:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:50:01.139 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.140 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:50:01.141 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T21:50:01.142 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:50:01.143 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.144 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:50:01.145 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T21:50:01.146 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:50:01.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8814066f30 msgr2=0x7f88140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8814066f30 0x7f88140693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f88300c1160 tx=0x7f882c009040 comp rx=0 tx=0).stop 2023-12-26T21:50:01.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300cb320 msgr2=0x7f88301423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300cb320 0x7f88301423a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f882800c8e0 tx=0x7f8828001710 comp rx=0 tx=0).stop 2023-12-26T21:50:01.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 shutdown_connections 2023-12-26T21:50:01.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88300cb320 0x7f88301423a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f8814066f30 0x7f88140693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88300b0750 0x7f883016a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 --2- 172.21.15.131:0/1402761532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88300c9400 0x7f88301585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 >> 172.21.15.131:0/1402761532 conn(0x7f8830068040 msgr2=0x7f8830155b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:01.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 shutdown_connections 2023-12-26T21:50:01.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.135+0000 7f88377ff700 1 -- 172.21.15.131:0/1402761532 wait complete. 2023-12-26T21:50:01.154 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:50:01.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.343+0000 7fcb121ff700 1 Processor -- start 2023-12-26T21:50:01.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.343+0000 7fcb121ff700 1 -- start start 2023-12-26T21:50:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.344+0000 7fcb121ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcb0c0c8820 0x7fcb0c0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.344+0000 7fcb121ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcb0c0b19f0 0x7fcb0c161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.344+0000 7fcb121ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0c157a80 con 0x7fcb0c0c8bf0 2023-12-26T21:50:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb121ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0c0cac50 con 0x7fcb0c0b19f0 2023-12-26T21:50:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb121ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0c0caf30 con 0x7fcb0c0c8820 2023-12-26T21:50:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcb0c0b19f0 0x7fcb0c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0affd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcb0c0c8bf0 0x7fcb0c0b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59062/0 (socket says 172.21.15.131:59062) 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0affd700 1 -- 172.21.15.131:0/870122637 learned_addr learned my addr 172.21.15.131:0/870122637 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0b7fe700 1 --2- 172.21.15.131:0/870122637 >> v2:172.21.15.190:3300/0 conn(0x7fcb0c0c8820 0x7fcb0c0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0bfff700 1 -- 172.21.15.131:0/870122637 >> v2:172.21.15.190:3300/0 conn(0x7fcb0c0c8820 msgr2=0x7fcb0c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0bfff700 1 --2- 172.21.15.131:0/870122637 >> v2:172.21.15.190:3300/0 conn(0x7fcb0c0c8820 0x7fcb0c0ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0bfff700 1 -- 172.21.15.131:0/870122637 >> v1:172.21.15.131:6789/0 conn(0x7fcb0c0c8bf0 legacy=0x7fcb0c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0bfff700 1 -- 172.21.15.131:0/870122637 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c0cb210 con 0x7fcb0c0b19f0 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0a7fc700 1 -- 172.21.15.131:0/870122637 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574041621 0 0) 0x7fcb0c157a80 con 0x7fcb0c0c8bf0 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.345+0000 7fcb0bfff700 1 --2- 172.21.15.131:0/870122637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0b19f0 0x7fcb0c161090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c064ea0 tx=0x7fcb04009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1fb2897786bc4b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb0a7fc700 1 -- 172.21.15.131:0/870122637 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400e070 con 0x7fcb0c0b19f0 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb0a7fc700 1 -- 172.21.15.131:0/870122637 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0400bc30 con 0x7fcb0c0b19f0 2023-12-26T21:50:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb0a7fc700 1 -- 172.21.15.131:0/870122637 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04012430 con 0x7fcb0c0b19f0 2023-12-26T21:50:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb121ff700 1 -- 172.21.15.131:0/870122637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0b19f0 msgr2=0x7fcb0c161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb121ff700 1 --2- 172.21.15.131:0/870122637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0b19f0 0x7fcb0c161090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c064ea0 tx=0x7fcb04009780 comp rx=0 tx=0).stop 2023-12-26T21:50:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb121ff700 1 -- 172.21.15.131:0/870122637 shutdown_connections 2023-12-26T21:50:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb121ff700 1 --2- 172.21.15.131:0/870122637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0b19f0 0x7fcb0c161090 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb121ff700 1 --2- 172.21.15.131:0/870122637 >> v2:172.21.15.190:3300/0 conn(0x7fcb0c0c8820 0x7fcb0c0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb121ff700 1 -- 172.21.15.131:0/870122637 >> 172.21.15.131:0/870122637 conn(0x7fcb0c068040 msgr2=0x7fcb0c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:01.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb121ff700 1 -- 172.21.15.131:0/870122637 shutdown_connections 2023-12-26T21:50:01.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.346+0000 7fcb121ff700 1 -- 172.21.15.131:0/870122637 wait complete. 2023-12-26T21:50:01.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.348+0000 7fcb121ff700 1 Processor -- start 2023-12-26T21:50:01.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.348+0000 7fcb121ff700 1 -- start start 2023-12-26T21:50:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb121ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0c0b19f0 0x7fcb0c155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb121ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0c8820 0x7fcb0c15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb121ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb0c0c8bf0 0x7fcb0c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb121ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1556a0 con 0x7fcb0c0c8820 2023-12-26T21:50:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb121ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c15c9f0 con 0x7fcb0c0b19f0 2023-12-26T21:50:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb121ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c15bde0 con 0x7fcb0c0c8bf0 2023-12-26T21:50:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb0affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0c8820 0x7fcb0c15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb0affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0c8820 0x7fcb0c15ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33532/0 (socket says 172.21.15.131:33532) 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb0affd700 1 -- 172.21.15.131:0/2556235022 learned_addr learned my addr 172.21.15.131:0/2556235022 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb0b7fe700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0c0b19f0 0x7fcb0c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb0bfff700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb0c0c8bf0 0x7fcb0c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb0affd700 1 -- 172.21.15.131:0/2556235022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb0c0c8bf0 msgr2=0x7fcb0c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.349+0000 7fcb0affd700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb0c0c8bf0 0x7fcb0c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.350+0000 7fcb0affd700 1 -- 172.21.15.131:0/2556235022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0c0b19f0 msgr2=0x7fcb0c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.350+0000 7fcb0affd700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0c0b19f0 0x7fcb0c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.350+0000 7fcb0affd700 1 -- 172.21.15.131:0/2556235022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb04009a20 con 0x7fcb0c0c8820 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.350+0000 7fcb0affd700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0c8820 0x7fcb0c15ce80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000c810 tx=0x7fcb00007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.350+0000 7fcaf37fe700 1 -- 172.21.15.131:0/2556235022 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00005c00 con 0x7fcb0c0c8820 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.350+0000 7fcaf37fe700 1 -- 172.21.15.131:0/2556235022 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb00005d60 con 0x7fcb0c0c8820 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.350+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c15c0c0 con 0x7fcb0c0c8820 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.350+0000 7fcaf37fe700 1 -- 172.21.15.131:0/2556235022 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00007320 con 0x7fcb0c0c8820 2023-12-26T21:50:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.350+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c15c680 con 0x7fcb0c0c8820 2023-12-26T21:50:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.351+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c15cb60 con 0x7fcb0c0c8820 2023-12-26T21:50:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.352+0000 7fcaf37fe700 1 -- 172.21.15.131:0/2556235022 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcb0001b400 con 0x7fcb0c0c8820 2023-12-26T21:50:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.353+0000 7fcaf37fe700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcaf8066f30 0x7fcaf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.353+0000 7fcaf37fe700 1 -- 172.21.15.131:0/2556235022 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5217+0+0 (secure 0 0 0) 0x7fcb00083ed0 con 0x7fcb0c0c8820 2023-12-26T21:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.353+0000 7fcb0b7fe700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcaf8066f30 0x7fcaf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.354+0000 7fcb0b7fe700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcaf8066f30 0x7fcaf80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcafc006fd0 tx=0x7fcafc008040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:01.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.357+0000 7fcaf37fe700 1 -- 172.21.15.131:0/2556235022 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb00051300 con 0x7fcb0c0c8820 2023-12-26T21:50:01.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.551+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb0c0657e0 con 0x7fcb0c0c8820 2023-12-26T21:50:01.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.551+0000 7fcaf37fe700 1 -- 172.21.15.131:0/2556235022 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+12734 (secure 0 0 0) 0x7fcb00054920 con 0x7fcb0c0c8820 2023-12-26T21:50:01.552 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:01.552 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":60,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:50:00.631871+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcaf8066f30 msgr2=0x7fcaf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcaf8066f30 0x7fcaf80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcafc006fd0 tx=0x7fcafc008040 comp rx=0 tx=0).stop 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0c8820 msgr2=0x7fcb0c15ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0c8820 0x7fcb0c15ce80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000c810 tx=0x7fcb00007db0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 shutdown_connections 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcaf8066f30 0x7fcaf80693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb0c0c8bf0 0x7fcb0c15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0c0b19f0 0x7fcb0c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 --2- 172.21.15.131:0/2556235022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c0c8820 0x7fcb0c15ce80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 >> 172.21.15.131:0/2556235022 conn(0x7fcb0c068040 msgr2=0x7fcb0c155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 shutdown_connections 2023-12-26T21:50:01.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.554+0000 7fcb121ff700 1 -- 172.21.15.131:0/2556235022 wait complete. 2023-12-26T21:50:01.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T21:50:01.573 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:50:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.765+0000 7f451b5ff700 1 Processor -- start 2023-12-26T21:50:01.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.765+0000 7f451b5ff700 1 -- start start 2023-12-26T21:50:01.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.766+0000 7f451b5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45141596a0 0x7f45140b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.766+0000 7f451b5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f45140b0870 0x7f45140b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.766+0000 7f451b5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45140afcd0 con 0x7f4514159a70 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.766+0000 7f451b5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45140b3e70 con 0x7f45140b0870 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.766+0000 7f451b5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45140b0cd0 con 0x7f45141596a0 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.766+0000 7f4518b9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4514159a70 0x7f45140b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59064/0 (socket says 172.21.15.131:59064) 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.766+0000 7f4518b9a700 1 -- 172.21.15.131:0/2771885357 learned_addr learned my addr 172.21.15.131:0/2771885357 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f451939b700 1 --2- 172.21.15.131:0/2771885357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45141596a0 0x7f45140b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f4519b9c700 1 --2- 172.21.15.131:0/2771885357 >> v2:172.21.15.190:3300/0 conn(0x7f45140b0870 0x7f45140b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f450bfff700 1 -- 172.21.15.131:0/2771885357 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233823694 0 0) 0x7f45140afcd0 con 0x7f4514159a70 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f450bfff700 1 -- 172.21.15.131:0/2771885357 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4500003570 con 0x7f4514159a70 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f4519b9c700 1 -- 172.21.15.131:0/2771885357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45141596a0 msgr2=0x7f45140b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f4519b9c700 1 --2- 172.21.15.131:0/2771885357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45141596a0 0x7f45140b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f4519b9c700 1 -- 172.21.15.131:0/2771885357 >> v1:172.21.15.131:6789/0 conn(0x7f4514159a70 legacy=0x7f45140b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f4519b9c700 1 -- 172.21.15.131:0/2771885357 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140b2e00 con 0x7f45140b0870 2023-12-26T21:50:01.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f451939b700 1 --2- 172.21.15.131:0/2771885357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45141596a0 0x7f45140b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.767+0000 7f4519b9c700 1 --2- 172.21.15.131:0/2771885357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45140b0870 0x7f45140b28c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f450c009e60 tx=0x7f450c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f33d47e2ab9a0bb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f450bfff700 1 -- 172.21.15.131:0/2771885357 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00e070 con 0x7f45140b0870 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f450bfff700 1 -- 172.21.15.131:0/2771885357 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c00bab0 con 0x7f45140b0870 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f450bfff700 1 -- 172.21.15.131:0/2771885357 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00be70 con 0x7f45140b0870 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f451b5ff700 1 -- 172.21.15.131:0/2771885357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45140b0870 msgr2=0x7f45140b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f451b5ff700 1 --2- 172.21.15.131:0/2771885357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45140b0870 0x7f45140b28c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f450c009e60 tx=0x7f450c009780 comp rx=0 tx=0).stop 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f451b5ff700 1 -- 172.21.15.131:0/2771885357 shutdown_connections 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f451b5ff700 1 --2- 172.21.15.131:0/2771885357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45140b0870 0x7f45140b28c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f451b5ff700 1 --2- 172.21.15.131:0/2771885357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45141596a0 0x7f45140b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f451b5ff700 1 -- 172.21.15.131:0/2771885357 >> 172.21.15.131:0/2771885357 conn(0x7f451406ece0 msgr2=0x7f451406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f451b5ff700 1 -- 172.21.15.131:0/2771885357 shutdown_connections 2023-12-26T21:50:01.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.768+0000 7f451b5ff700 1 -- 172.21.15.131:0/2771885357 wait complete. 2023-12-26T21:50:01.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.770+0000 7f451b5ff700 1 Processor -- start 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.770+0000 7f451b5ff700 1 -- start start 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f451b5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140b0870 0x7f451415c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f451b5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45141596a0 0x7f451414b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f451b5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4514159a70 0x7f45140c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f451b5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451415e6c0 con 0x7f4514159a70 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f451b5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451415e230 con 0x7f45141596a0 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f451b5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451415ec10 con 0x7f45140b0870 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f4519b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4514159a70 0x7f45140c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f4519b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4514159a70 0x7f45140c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33540/0 (socket says 172.21.15.131:33540) 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f4519b9c700 1 -- 172.21.15.131:0/3618950595 learned_addr learned my addr 172.21.15.131:0/3618950595 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f4518b9a700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45141596a0 0x7f451414b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.771+0000 7f451939b700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140b0870 0x7f451415c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f4519b9c700 1 -- 172.21.15.131:0/3618950595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140b0870 msgr2=0x7f451415c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f4519b9c700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140b0870 0x7f451415c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f4519b9c700 1 -- 172.21.15.131:0/3618950595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45141596a0 msgr2=0x7f451414b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f4519b9c700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45141596a0 0x7f451414b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f4519b9c700 1 -- 172.21.15.131:0/3618950595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4510009420 con 0x7f4514159a70 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f4518b9a700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45141596a0 0x7f451414b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f4519b9c700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4514159a70 0x7f45140c99b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4514089710 tx=0x7f450c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f45097fa700 1 -- 172.21.15.131:0/3618950595 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c027470 con 0x7f4514159a70 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f45097fa700 1 -- 172.21.15.131:0/3618950595 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c0275f0 con 0x7f4514159a70 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c009a20 con 0x7f4514159a70 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f45097fa700 1 -- 172.21.15.131:0/3618950595 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c0279b0 con 0x7f4514159a70 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.772+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451415dbe0 con 0x7f4514159a70 2023-12-26T21:50:01.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.773+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451415e3e0 con 0x7f4514159a70 2023-12-26T21:50:01.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.774+0000 7f45097fa700 1 -- 172.21.15.131:0/3618950595 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f450c024070 con 0x7f4514159a70 2023-12-26T21:50:01.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.775+0000 7f45097fa700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44f8066f30 0x7f44f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:01.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.775+0000 7f45097fa700 1 -- 172.21.15.131:0/3618950595 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5217+0+0 (secure 0 0 0) 0x7f450c086dc0 con 0x7f4514159a70 2023-12-26T21:50:01.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.775+0000 7f451939b700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44f8066f30 0x7f44f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:01.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.776+0000 7f451939b700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44f8066f30 0x7f44f80693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4510009f40 tx=0x7f45100091a0 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:01.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.779+0000 7f45097fa700 1 -- 172.21.15.131:0/3618950595 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f451415e3e0 con 0x7f4514159a70 2023-12-26T21:50:01.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.971+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f451415e3a0 con 0x7f4514159a70 2023-12-26T21:50:01.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.971+0000 7f45097fa700 1 -- 172.21.15.131:0/3618950595 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f450c02c0a0 con 0x7f4514159a70 2023-12-26T21:50:01.972 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:50:01.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.973+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44f8066f30 msgr2=0x7f44f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.973+0000 7f451b5ff700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44f8066f30 0x7f44f80693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4510009f40 tx=0x7f45100091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.973+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4514159a70 msgr2=0x7f45140c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:01.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.973+0000 7f451b5ff700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4514159a70 0x7f45140c99b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4514089710 tx=0x7f450c019000 comp rx=0 tx=0).stop 2023-12-26T21:50:01.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.974+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 shutdown_connections 2023-12-26T21:50:01.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.974+0000 7f451b5ff700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4514159a70 0x7f45140c99b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.974+0000 7f451b5ff700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f44f8066f30 0x7f44f80693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.974+0000 7f451b5ff700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140b0870 0x7f451415c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.974+0000 7f451b5ff700 1 --2- 172.21.15.131:0/3618950595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45141596a0 0x7f451414b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:01.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.974+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 >> 172.21.15.131:0/3618950595 conn(0x7f451406ece0 msgr2=0x7f45140cd210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:01.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.974+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 shutdown_connections 2023-12-26T21:50:01.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:01.974+0000 7f451b5ff700 1 -- 172.21.15.131:0/3618950595 wait complete. 2023-12-26T21:50:01.992 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T21:50:02.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.182+0000 7f0d363ff700 1 Processor -- start 2023-12-26T21:50:02.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.182+0000 7f0d363ff700 1 -- start start 2023-12-26T21:50:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.183+0000 7f0d363ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d300b5310 0x7f0d300b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.183+0000 7f0d363ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d300afcd0 con 0x7f0d30159a70 2023-12-26T21:50:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.184+0000 7f0d363ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d300b3e70 con 0x7f0d301596a0 2023-12-26T21:50:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.184+0000 7f0d363ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300b0f20 con 0x7f0d300b5310 2023-12-26T21:50:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.184+0000 7f0d27fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0d30159a70 0x7f0d300b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59066/0 (socket says 172.21.15.131:59066) 2023-12-26T21:50:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.184+0000 7f0d27fff700 1 -- 172.21.15.131:0/2623368028 learned_addr learned my addr 172.21.15.131:0/2623368028 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:02.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.184+0000 7f0d3499c700 1 --2- 172.21.15.131:0/2623368028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d300b5310 0x7f0d300b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.184+0000 7f0d2f7fe700 1 -- 172.21.15.131:0/2623368028 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903606046 0 0) 0x7f0d300afcd0 con 0x7f0d30159a70 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.184+0000 7f0d2f7fe700 1 -- 172.21.15.131:0/2623368028 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d1c003570 con 0x7f0d30159a70 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.184+0000 7f0d2f7fe700 1 -- 172.21.15.131:0/2623368028 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316773034 0 0) 0x7f0d300b3e70 con 0x7f0d301596a0 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.184+0000 7f0d2f7fe700 1 -- 172.21.15.131:0/2623368028 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d300afcd0 con 0x7f0d301596a0 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.185+0000 7f0d3499c700 1 -- 172.21.15.131:0/2623368028 >> v1:172.21.15.190:6789/0 conn(0x7f0d301596a0 legacy=0x7f0d300b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.185+0000 7f0d3499c700 1 -- 172.21.15.131:0/2623368028 >> v1:172.21.15.131:6789/0 conn(0x7f0d30159a70 legacy=0x7f0d300b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.185+0000 7f0d2f7fe700 1 -- 172.21.15.131:0/2623368028 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1153902565 0 0) 0x7f0d1c003570 con 0x7f0d30159a70 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.185+0000 7f0d3499c700 1 -- 172.21.15.131:0/2623368028 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d300b3540 con 0x7f0d300b5310 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.185+0000 7f0d3499c700 1 --2- 172.21.15.131:0/2623368028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d300b5310 0x7f0d300b0870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009e90 tx=0x7f0d28009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450f474647e4b9ab server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.185+0000 7f0d2f7fe700 1 -- 172.21.15.131:0/2623368028 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800e070 con 0x7f0d300b5310 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.185+0000 7f0d2f7fe700 1 -- 172.21.15.131:0/2623368028 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2800b870 con 0x7f0d300b5310 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.185+0000 7f0d2f7fe700 1 -- 172.21.15.131:0/2623368028 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18001470 con 0x7f0d300b5310 2023-12-26T21:50:02.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.186+0000 7f0d363ff700 1 -- 172.21.15.131:0/2623368028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d300b5310 msgr2=0x7f0d300b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.186+0000 7f0d363ff700 1 --2- 172.21.15.131:0/2623368028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d300b5310 0x7f0d300b0870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009e90 tx=0x7f0d28009780 comp rx=0 tx=0).stop 2023-12-26T21:50:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.186+0000 7f0d363ff700 1 -- 172.21.15.131:0/2623368028 shutdown_connections 2023-12-26T21:50:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.186+0000 7f0d363ff700 1 --2- 172.21.15.131:0/2623368028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d300b5310 0x7f0d300b0870 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.186+0000 7f0d363ff700 1 -- 172.21.15.131:0/2623368028 >> 172.21.15.131:0/2623368028 conn(0x7f0d3006ece0 msgr2=0x7f0d3006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.186+0000 7f0d363ff700 1 -- 172.21.15.131:0/2623368028 shutdown_connections 2023-12-26T21:50:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.186+0000 7f0d363ff700 1 -- 172.21.15.131:0/2623368028 wait complete. 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.187+0000 7f0d363ff700 1 Processor -- start 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.188+0000 7f0d363ff700 1 -- start start 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.188+0000 7f0d363ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300b5310 0x7f0d3016a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.188+0000 7f0d363ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d301596a0 0x7f0d301572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.188+0000 7f0d363ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d30159a70 0x7f0d300cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.188+0000 7f0d363ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3016aa20 con 0x7f0d301596a0 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.188+0000 7f0d363ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30157810 con 0x7f0d300b5310 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.188+0000 7f0d363ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300c70b0 con 0x7f0d30159a70 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d27fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d301596a0 0x7f0d301572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d27fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d301596a0 0x7f0d301572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33556/0 (socket says 172.21.15.131:33556) 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d2ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300b5310 0x7f0d3016a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d27fff700 1 -- 172.21.15.131:0/3793042356 learned_addr learned my addr 172.21.15.131:0/3793042356 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d3499c700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d30159a70 0x7f0d300cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d27fff700 1 -- 172.21.15.131:0/3793042356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d30159a70 msgr2=0x7f0d300cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d27fff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d30159a70 0x7f0d300cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d27fff700 1 -- 172.21.15.131:0/3793042356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300b5310 msgr2=0x7f0d3016a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d27fff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300b5310 0x7f0d3016a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d27fff700 1 -- 172.21.15.131:0/3793042356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28009a20 con 0x7f0d301596a0 2023-12-26T21:50:02.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.189+0000 7f0d27fff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d301596a0 0x7f0d301572d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800cbc0 tx=0x7f0d18001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:02.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.191+0000 7f0d2cff9700 1 -- 172.21.15.131:0/3793042356 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18007150 con 0x7f0d301596a0 2023-12-26T21:50:02.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.191+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d300c57e0 con 0x7f0d301596a0 2023-12-26T21:50:02.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.191+0000 7f0d2cff9700 1 -- 172.21.15.131:0/3793042356 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d180072d0 con 0x7f0d301596a0 2023-12-26T21:50:02.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.191+0000 7f0d2cff9700 1 -- 172.21.15.131:0/3793042356 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18007690 con 0x7f0d301596a0 2023-12-26T21:50:02.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.191+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d300c4f40 con 0x7f0d301596a0 2023-12-26T21:50:02.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.192+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d300657e0 con 0x7f0d301596a0 2023-12-26T21:50:02.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.193+0000 7f0d2cff9700 1 -- 172.21.15.131:0/3793042356 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0d18004a40 con 0x7f0d301596a0 2023-12-26T21:50:02.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.194+0000 7f0d2cff9700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d10066f30 0x7f0d100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.194+0000 7f0d2cff9700 1 -- 172.21.15.131:0/3793042356 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5217+0+0 (secure 0 0 0) 0x7f0d180836d0 con 0x7f0d301596a0 2023-12-26T21:50:02.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.194+0000 7f0d2ffff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d10066f30 0x7f0d100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.195+0000 7f0d2ffff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d10066f30 0x7f0d100693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d300cd0d0 tx=0x7f0d20007d50 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:02.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.198+0000 7f0d2cff9700 1 -- 172.21.15.131:0/3793042356 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d18050a80 con 0x7f0d301596a0 2023-12-26T21:50:02.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.395+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0d3015b2f0 con 0x7f0d301596a0 2023-12-26T21:50:02.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.395+0000 7f0d2cff9700 1 -- 172.21.15.131:0/3793042356 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0d18018090 con 0x7f0d301596a0 2023-12-26T21:50:02.396 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:02.396 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:50:02.396 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:02.396 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T21:50:02.396 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T21:50:02.397 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:02.397 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d10066f30 msgr2=0x7f0d100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d10066f30 0x7f0d100693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d300cd0d0 tx=0x7f0d20007d50 comp rx=0 tx=0).stop 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d301596a0 msgr2=0x7f0d301572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d301596a0 0x7f0d301572d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800cbc0 tx=0x7f0d18001ab0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 shutdown_connections 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d10066f30 0x7f0d100693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d30159a70 0x7f0d300cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300b5310 0x7f0d3016a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 --2- 172.21.15.131:0/3793042356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d301596a0 0x7f0d301572d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 >> 172.21.15.131:0/3793042356 conn(0x7f0d3006ece0 msgr2=0x7f0d300ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 shutdown_connections 2023-12-26T21:50:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.398+0000 7f0d363ff700 1 -- 172.21.15.131:0/3793042356 wait complete. 2023-12-26T21:50:02.416 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T21:50:02.416 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T21:50:02.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.609+0000 7f4a227ff700 1 Processor -- start 2023-12-26T21:50:02.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.609+0000 7f4a227ff700 1 -- start start 2023-12-26T21:50:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.610+0000 7f4a227ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0cb510 0x7f4a1c069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.610+0000 7f4a227ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c069640 con 0x7f4a1c142f00 2023-12-26T21:50:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.610+0000 7f4a227ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c068750 con 0x7f4a1c0cc590 2023-12-26T21:50:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.610+0000 7f4a227ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c068a30 con 0x7f4a1c0cb510 2023-12-26T21:50:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1bfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4a1c142f00 0x7f4a1c0cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59078/0 (socket says 172.21.15.131:59078) 2023-12-26T21:50:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1bfff700 1 -- 172.21.15.131:0/1859969377 learned_addr learned my addr 172.21.15.131:0/1859969377 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:02.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1b7fe700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0cb510 0x7f4a1c069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782874269 0 0) 0x7f4a1c069640 con 0x7f4a1c142f00 2023-12-26T21:50:02.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a04003570 con 0x7f4a1c142f00 2023-12-26T21:50:02.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398107527 0 0) 0x7f4a1c068750 con 0x7f4a1c0cc590 2023-12-26T21:50:02.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a1c069640 con 0x7f4a1c0cc590 2023-12-26T21:50:02.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1458516043 0 0) 0x7f4a04003570 con 0x7f4a1c142f00 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0cb510 msgr2=0x7f4a1c069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0cb510 0x7f4a1c069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 >> v1:172.21.15.190:6789/0 conn(0x7f4a1c0cc590 legacy=0x7f4a1c0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c068d10 con 0x7f4a1c142f00 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.611+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f4a0c001470 con 0x7f4a1c142f00 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 >> v1:172.21.15.131:6789/0 conn(0x7f4a1c142f00 legacy=0x7f4a1c0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a1affd700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0cb510 0x7f4a04003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a1affd700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a04004180 0x7f4a04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a1affd700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a04004d50 0x7f4a040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c068a30 con 0x7f4a04004d50 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0400bba0 con 0x7f4a04004180 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0400bd60 con 0x7f4a1c0cb510 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a20d9c700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a04004d50 0x7f4a040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a1b7fe700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a04004180 0x7f4a04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a1bfff700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0cb510 0x7f4a04003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a20d9c700 1 -- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0cb510 msgr2=0x7f4a04003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a20d9c700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0cb510 0x7f4a04003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a20d9c700 1 -- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a04004180 msgr2=0x7f4a04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a20d9c700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a04004180 0x7f4a04004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.612+0000 7f4a20d9c700 1 -- 172.21.15.131:0/1859969377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04003570 con 0x7f4a04004d50 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a20d9c700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a04004d50 0x7f4a040051c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4a14002120 tx=0x7f4a140017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254321b01fa57665 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14001ab0 con 0x7f4a04004d50 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1400c980 con 0x7f4a04004d50 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a1affd700 1 -- 172.21.15.131:0/1859969377 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400cd20 con 0x7f4a04004d50 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a227ff700 1 -- 172.21.15.131:0/1859969377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a04004d50 msgr2=0x7f4a040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a227ff700 1 --2- 172.21.15.131:0/1859969377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a04004d50 0x7f4a040051c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4a14002120 tx=0x7f4a140017a0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a20d9c700 1 -- 172.21.15.131:0/1859969377 reap_dead start 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a227ff700 1 -- 172.21.15.131:0/1859969377 shutdown_connections 2023-12-26T21:50:02.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a227ff700 1 -- 172.21.15.131:0/1859969377 >> 172.21.15.131:0/1859969377 conn(0x7f4a1c063cd0 msgr2=0x7f4a1c06b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:02.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a227ff700 1 -- 172.21.15.131:0/1859969377 shutdown_connections 2023-12-26T21:50:02.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.613+0000 7f4a227ff700 1 -- 172.21.15.131:0/1859969377 wait complete. 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.615+0000 7f4a227ff700 1 Processor -- start 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.615+0000 7f4a227ff700 1 -- start start 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a227ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a04004d50 0x7f4a1c0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a227ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0b3520 0x7f4a1c16a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a227ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a1c0b25f0 0x7f4a1c141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a227ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c159cc0 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a227ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0b3990 con 0x7f4a04004d50 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a227ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c16a7c0 con 0x7f4a1c0b3520 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a20d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a1c0b25f0 0x7f4a1c141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a20d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a1c0b25f0 0x7f4a1c141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33578/0 (socket says 172.21.15.131:33578) 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a20d9c700 1 -- 172.21.15.131:0/2392649529 learned_addr learned my addr 172.21.15.131:0/2392649529 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a1b7fe700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0b3520 0x7f4a1c16a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.616+0000 7f4a1bfff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a04004d50 0x7f4a1c0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a20d9c700 1 -- 172.21.15.131:0/2392649529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0b3520 msgr2=0x7f4a1c16a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a20d9c700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0b3520 0x7f4a1c16a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a20d9c700 1 -- 172.21.15.131:0/2392649529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a04004d50 msgr2=0x7f4a1c0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a20d9c700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a04004d50 0x7f4a1c0b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a20d9c700 1 -- 172.21.15.131:0/2392649529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0b1780 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a20d9c700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a1c0b25f0 0x7f4a1c141870 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4a14003000 tx=0x7f4a14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a03fff700 1 -- 172.21.15.131:0/2392649529 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400c6d0 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a03fff700 1 -- 172.21.15.131:0/2392649529 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a14012e50 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a03fff700 1 -- 172.21.15.131:0/2392649529 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14006dd0 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c0b1a60 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.617+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c0b2020 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.618+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c0717f0 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.619+0000 7f4a03fff700 1 -- 172.21.15.131:0/2392649529 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4a14003030 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.620+0000 7f4a03fff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4a08066f30 0x7f4a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:02.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.620+0000 7f4a03fff700 1 -- 172.21.15.131:0/2392649529 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5217+0+0 (secure 0 0 0) 0x7f4a14086150 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.620+0000 7f4a1bfff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4a08066f30 0x7f4a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:02.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.620+0000 7f4a1bfff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4a08066f30 0x7f4a080693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c001200 tx=0x7f4a0c009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:02.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.624+0000 7f4a03fff700 1 -- 172.21.15.131:0/2392649529 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a14053500 con 0x7f4a1c0b25f0 2023-12-26T21:50:02.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:02.820+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4a1c068ff0 con 0x7f4a1c0b25f0 2023-12-26T21:50:03.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.201+0000 7f4a03fff700 1 -- 172.21.15.131:0/2392649529 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4a14056b20 con 0x7f4a1c0b25f0 2023-12-26T21:50:03.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4a08066f30 msgr2=0x7f4a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:03.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4a08066f30 0x7f4a080693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c001200 tx=0x7f4a0c009040 comp rx=0 tx=0).stop 2023-12-26T21:50:03.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a1c0b25f0 msgr2=0x7f4a1c141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:03.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a1c0b25f0 0x7f4a1c141870 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4a14003000 tx=0x7f4a14016000 comp rx=0 tx=0).stop 2023-12-26T21:50:03.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 shutdown_connections 2023-12-26T21:50:03.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a1c0b25f0 0x7f4a1c141870 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a04004d50 0x7f4a1c0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4a08066f30 0x7f4a080693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 --2- 172.21.15.131:0/2392649529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c0b3520 0x7f4a1c16a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 >> 172.21.15.131:0/2392649529 conn(0x7f4a1c063cd0 msgr2=0x7f4a1c0c9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:03.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.205+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 shutdown_connections 2023-12-26T21:50:03.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.206+0000 7f4a227ff700 1 -- 172.21.15.131:0/2392649529 wait complete. 2023-12-26T21:50:03.207 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T21:50:03.225 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T21:50:03.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.420+0000 7f740d3ff700 1 Processor -- start 2023-12-26T21:50:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.420+0000 7f740d3ff700 1 -- start start 2023-12-26T21:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.421+0000 7f740d3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7408069d30 0x7f740806e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.421+0000 7f740d3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74080c6b30 con 0x7f740806e2c0 2023-12-26T21:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.421+0000 7f740d3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74080c6e10 con 0x7f74080adf60 2023-12-26T21:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.421+0000 7f740d3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74081561e0 con 0x7f7408069d30 2023-12-26T21:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.421+0000 7f74067fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f740806e2c0 0x7f7408069620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59082/0 (socket says 172.21.15.131:59082) 2023-12-26T21:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.421+0000 7f74067fc700 1 -- 172.21.15.131:0/1245496723 learned_addr learned my addr 172.21.15.131:0/1245496723 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f74077fe700 1 --2- 172.21.15.131:0/1245496723 >> v2:172.21.15.190:3300/0 conn(0x7f7408069d30 0x7f740806e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418153730 0 0) 0x7f74080c6b30 con 0x7f740806e2c0 2023-12-26T21:50:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f0003570 con 0x7f740806e2c0 2023-12-26T21:50:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4037815447 0 0) 0x7f73f0003570 con 0x7f740806e2c0 2023-12-26T21:50:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 >> v2:172.21.15.190:3300/0 conn(0x7f7408069d30 msgr2=0x7f740806e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 --2- 172.21.15.131:0/1245496723 >> v2:172.21.15.190:3300/0 conn(0x7f7408069d30 0x7f740806e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 >> v1:172.21.15.190:6789/0 conn(0x7f74080adf60 legacy=0x7f74080ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080c70f0 con 0x7f740806e2c0 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f73fc001470 con 0x7f740806e2c0 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 >> v1:172.21.15.131:6789/0 conn(0x7f740806e2c0 legacy=0x7f7408069620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73f0003a60 0x7f73f0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0004810 0x7f73f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.422+0000 7f7405ffb700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f00051c0 0x7f73f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74081561e0 con 0x7f73f0004810 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f000c0d0 con 0x7f73f00051c0 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f000c290 con 0x7f73f0003a60 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f74067fc700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0004810 0x7f73f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f7406ffd700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73f0003a60 0x7f73f0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f74077fe700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f00051c0 0x7f73f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f74067fc700 1 -- 172.21.15.131:0/1245496723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73f0003a60 msgr2=0x7f73f0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f74067fc700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73f0003a60 0x7f73f0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f74067fc700 1 -- 172.21.15.131:0/1245496723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f00051c0 msgr2=0x7f73f000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f74067fc700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f00051c0 0x7f73f000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f74067fc700 1 -- 172.21.15.131:0/1245496723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0003570 con 0x7f73f0004810 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f74077fe700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f00051c0 0x7f73f000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.423+0000 7f74077fe700 1 -- 172.21.15.131:0/1245496723 reap_dead start 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f74067fc700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0004810 0x7f73f0004c80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00d4d0 tx=0x7f73fc007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845ecfeec7d179c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc00ddc0 con 0x7f73f0004810 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc005920 con 0x7f73f0004810 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f7405ffb700 1 -- 172.21.15.131:0/1245496723 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc005ce0 con 0x7f73f0004810 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f740d3ff700 1 -- 172.21.15.131:0/1245496723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0004810 msgr2=0x7f73f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f740d3ff700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0004810 0x7f73f0004c80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00d4d0 tx=0x7f73fc007940 comp rx=0 tx=0).stop 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f740d3ff700 1 -- 172.21.15.131:0/1245496723 shutdown_connections 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f740d3ff700 1 --2- 172.21.15.131:0/1245496723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0004810 0x7f73f0004c80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f740d3ff700 1 -- 172.21.15.131:0/1245496723 >> 172.21.15.131:0/1245496723 conn(0x7f7408068040 msgr2=0x7f74080cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f740d3ff700 1 -- 172.21.15.131:0/1245496723 shutdown_connections 2023-12-26T21:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.424+0000 7f740d3ff700 1 -- 172.21.15.131:0/1245496723 wait complete. 2023-12-26T21:50:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.426+0000 7f740d3ff700 1 Processor -- start 2023-12-26T21:50:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.426+0000 7f740d3ff700 1 -- start start 2023-12-26T21:50:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f740d3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73f0004810 0x7f740814c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f740d3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080c09a0 0x7f7408158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f740d3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7408159160 0x7f74080c1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f740d3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74081595e0 con 0x7f7408159160 2023-12-26T21:50:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f740d3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080c0510 con 0x7f74080c09a0 2023-12-26T21:50:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f740d3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740814cb90 con 0x7f73f0004810 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f74077fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7408159160 0x7f74080c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f74077fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7408159160 0x7f74080c1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33594/0 (socket says 172.21.15.131:33594) 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f74077fe700 1 -- 172.21.15.131:0/1878672721 learned_addr learned my addr 172.21.15.131:0/1878672721 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f7406ffd700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73f0004810 0x7f740814c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.427+0000 7f74067fc700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080c09a0 0x7f7408158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f74077fe700 1 -- 172.21.15.131:0/1878672721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73f0004810 msgr2=0x7f740814c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f74077fe700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73f0004810 0x7f740814c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f74077fe700 1 -- 172.21.15.131:0/1878672721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080c09a0 msgr2=0x7f7408158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f74077fe700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080c09a0 0x7f7408158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f74077fe700 1 -- 172.21.15.131:0/1878672721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080bfbe0 con 0x7f7408159160 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f74077fe700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7408159160 0x7f74080c1690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f74080b2290 tx=0x7f740000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f73eeffd700 1 -- 172.21.15.131:0/1878672721 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740000e040 con 0x7f7408159160 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f73eeffd700 1 -- 172.21.15.131:0/1878672721 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740000f040 con 0x7f7408159160 2023-12-26T21:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74080bfec0 con 0x7f7408159160 2023-12-26T21:50:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f73eeffd700 1 -- 172.21.15.131:0/1878672721 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400015430 con 0x7f7408159160 2023-12-26T21:50:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.428+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74080c1bd0 con 0x7f7408159160 2023-12-26T21:50:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.429+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74080657e0 con 0x7f7408159160 2023-12-26T21:50:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.431+0000 7f73eeffd700 1 -- 172.21.15.131:0/1878672721 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f74000157a0 con 0x7f7408159160 2023-12-26T21:50:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.432+0000 7f73eeffd700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73f4066f30 0x7f73f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.432+0000 7f73eeffd700 1 -- 172.21.15.131:0/1878672721 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5587+0+0 (secure 0 0 0) 0x7f7400085bc0 con 0x7f7408159160 2023-12-26T21:50:03.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.432+0000 7f7406ffd700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73f4066f30 0x7f73f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:03.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.433+0000 7f7406ffd700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73f4066f30 0x7f73f40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74081557c0 tx=0x7f73f8007d30 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:03.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.435+0000 7f73eeffd700 1 -- 172.21.15.131:0/1878672721 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7400052f20 con 0x7f7408159160 2023-12-26T21:50:03.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:03.626+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f74080c0650 con 0x7f7408159160 2023-12-26T21:50:04.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.204+0000 7f73eeffd700 1 -- 172.21.15.131:0/1878672721 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7400056540 con 0x7f7408159160 2023-12-26T21:50:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73f4066f30 msgr2=0x7f73f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73f4066f30 0x7f73f40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74081557c0 tx=0x7f73f8007d30 comp rx=0 tx=0).stop 2023-12-26T21:50:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7408159160 msgr2=0x7f74080c1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7408159160 0x7f74080c1690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f74080b2290 tx=0x7f740000b040 comp rx=0 tx=0).stop 2023-12-26T21:50:04.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 shutdown_connections 2023-12-26T21:50:04.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7408159160 0x7f74080c1690 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:04.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f73f4066f30 0x7f73f40693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:04.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73f0004810 0x7f740814c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:04.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 --2- 172.21.15.131:0/1878672721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74080c09a0 0x7f7408158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:04.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.208+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 >> 172.21.15.131:0/1878672721 conn(0x7f7408068040 msgr2=0x7f7408069620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:04.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.212+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 shutdown_connections 2023-12-26T21:50:04.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.213+0000 7f740d3ff700 1 -- 172.21.15.131:0/1878672721 wait complete. 2023-12-26T21:50:04.226 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T21:50:04.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T21:50:04.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.445+0000 7fc6151ff700 1 Processor -- start 2023-12-26T21:50:04.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.445+0000 7fc6151ff700 1 -- start start 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc6151ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc61013c510 0x7fc6100c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc6151ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6100cc810 0x7fc61006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc6151ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6100686c0 con 0x7fc61006b8c0 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc6151ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61006c280 con 0x7fc61013c510 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc6151ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc610069520 con 0x7fc6100cc810 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc60ed9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc61013c510 0x7fc6100c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc60ed9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc61013c510 0x7fc6100c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33600/0 (socket says 172.21.15.131:33600) 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc60ed9d700 1 -- 172.21.15.131:0/1448497707 learned_addr learned my addr 172.21.15.131:0/1448497707 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc60f59e700 1 --2- 172.21.15.131:0/1448497707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6100cc810 0x7fc61006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc60ed9d700 1 -- 172.21.15.131:0/1448497707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6100cc810 msgr2=0x7fc61006bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:04.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc60ed9d700 1 --2- 172.21.15.131:0/1448497707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6100cc810 0x7fc61006bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:04.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc60ed9d700 1 -- 172.21.15.131:0/1448497707 >> v1:172.21.15.190:6789/0 conn(0x7fc61006b8c0 legacy=0x7fc6100cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:04.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.447+0000 7fc60ed9d700 1 -- 172.21.15.131:0/1448497707 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610068910 con 0x7fc61013c510 2023-12-26T21:50:04.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.448+0000 7fc60ed9d700 1 --2- 172.21.15.131:0/1448497707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc61013c510 0x7fc6100c8a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc600009990 tx=0x7fc600009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60fc5ddb2d39ab5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.448+0000 7fc60e59c700 1 -- 172.21.15.131:0/1448497707 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600005c10 con 0x7fc61013c510 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.448+0000 7fc60e59c700 1 -- 172.21.15.131:0/1448497707 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600005d70 con 0x7fc61013c510 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.448+0000 7fc60e59c700 1 -- 172.21.15.131:0/1448497707 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60000b880 con 0x7fc61013c510 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.449+0000 7fc6151ff700 1 -- 172.21.15.131:0/1448497707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc61013c510 msgr2=0x7fc6100c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.449+0000 7fc6151ff700 1 --2- 172.21.15.131:0/1448497707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc61013c510 0x7fc6100c8a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc600009990 tx=0x7fc600009220 comp rx=0 tx=0).stop 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.449+0000 7fc6151ff700 1 -- 172.21.15.131:0/1448497707 shutdown_connections 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.449+0000 7fc6151ff700 1 --2- 172.21.15.131:0/1448497707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6100cc810 0x7fc61006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.449+0000 7fc6151ff700 1 --2- 172.21.15.131:0/1448497707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc61013c510 0x7fc6100c8a20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.449+0000 7fc6151ff700 1 -- 172.21.15.131:0/1448497707 >> 172.21.15.131:0/1448497707 conn(0x7fc6100ce3e0 msgr2=0x7fc6100ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:04.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.449+0000 7fc6151ff700 1 -- 172.21.15.131:0/1448497707 shutdown_connections 2023-12-26T21:50:04.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.449+0000 7fc6151ff700 1 -- 172.21.15.131:0/1448497707 wait complete. 2023-12-26T21:50:04.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.450+0000 7fc6151ff700 1 Processor -- start 2023-12-26T21:50:04.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.450+0000 7fc6151ff700 1 -- start start 2023-12-26T21:50:04.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.451+0000 7fc6151ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc61006b8c0 0x7fc6100b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.451+0000 7fc6151ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cc810 0x7fc6101588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.451+0000 7fc6151ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc61013c510 0x7fc610158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.451+0000 7fc6151ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610159790 con 0x7fc6100cc810 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.451+0000 7fc6151ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100c7ea0 con 0x7fc61013c510 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.451+0000 7fc6151ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61016a0b0 con 0x7fc61006b8c0 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc607fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cc810 0x7fc6101588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc60ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc61006b8c0 0x7fc6100b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc607fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cc810 0x7fc6101588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33616/0 (socket says 172.21.15.131:33616) 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc607fff700 1 -- 172.21.15.131:0/86862744 learned_addr learned my addr 172.21.15.131:0/86862744 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc60f59e700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc61013c510 0x7fc610158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc607fff700 1 -- 172.21.15.131:0/86862744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc61006b8c0 msgr2=0x7fc6100b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc607fff700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc61006b8c0 0x7fc6100b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc607fff700 1 -- 172.21.15.131:0/86862744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc61013c510 msgr2=0x7fc610158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc607fff700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc61013c510 0x7fc610158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc607fff700 1 -- 172.21.15.131:0/86862744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600009480 con 0x7fc6100cc810 2023-12-26T21:50:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.452+0000 7fc607fff700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cc810 0x7fc6101588a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c4e0 tx=0x7fc5f800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.453+0000 7fc606ffd700 1 -- 172.21.15.131:0/86862744 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8011590 con 0x7fc6100cc810 2023-12-26T21:50:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.453+0000 7fc606ffd700 1 -- 172.21.15.131:0/86862744 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f8011710 con 0x7fc6100cc810 2023-12-26T21:50:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.453+0000 7fc606ffd700 1 -- 172.21.15.131:0/86862744 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8011ad0 con 0x7fc6100cc810 2023-12-26T21:50:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.453+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6101594b0 con 0x7fc6100cc810 2023-12-26T21:50:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.453+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61016a880 con 0x7fc6100cc810 2023-12-26T21:50:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.454+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6100657e0 con 0x7fc6100cc810 2023-12-26T21:50:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.455+0000 7fc606ffd700 1 -- 172.21.15.131:0/86862744 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc5f801b3b0 con 0x7fc6100cc810 2023-12-26T21:50:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.456+0000 7fc606ffd700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc5f0066f30 0x7fc5f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.456+0000 7fc606ffd700 1 -- 172.21.15.131:0/86862744 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5969+0+0 (secure 0 0 0) 0x7fc5f8083a70 con 0x7fc6100cc810 2023-12-26T21:50:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.456+0000 7fc60ed9d700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc5f0066f30 0x7fc5f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.457+0000 7fc60ed9d700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc5f0066f30 0x7fc5f00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc600009260 tx=0x7fc600016000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:04.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.461+0000 7fc606ffd700 1 -- 172.21.15.131:0/86862744 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5f8050b30 con 0x7fc6100cc810 2023-12-26T21:50:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:04.697+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc610068bf0 con 0x7fc6100cc810 2023-12-26T21:50:05.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.219+0000 7fc606ffd700 1 -- 172.21.15.131:0/86862744 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc5f8017070 con 0x7fc6100cc810 2023-12-26T21:50:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.230+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc5f0066f30 msgr2=0x7fc5f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.230+0000 7fc6151ff700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc5f0066f30 0x7fc5f00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc600009260 tx=0x7fc600016000 comp rx=0 tx=0).stop 2023-12-26T21:50:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.230+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cc810 msgr2=0x7fc6101588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.230+0000 7fc6151ff700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cc810 0x7fc6101588a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c4e0 tx=0x7fc5f800c7f0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.234+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 shutdown_connections 2023-12-26T21:50:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.234+0000 7fc6151ff700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc61013c510 0x7fc610158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.234+0000 7fc6151ff700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc5f0066f30 0x7fc5f00693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.234+0000 7fc6151ff700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc61006b8c0 0x7fc6100b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.234+0000 7fc6151ff700 1 --2- 172.21.15.131:0/86862744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cc810 0x7fc6101588a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.234+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 >> 172.21.15.131:0/86862744 conn(0x7fc6100ce3e0 msgr2=0x7fc6100c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.235+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 shutdown_connections 2023-12-26T21:50:05.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.236+0000 7fc6151ff700 1 -- 172.21.15.131:0/86862744 wait complete. 2023-12-26T21:50:05.240 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 6 and data pool 7 2023-12-26T21:50:05.261 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:50:05.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.450+0000 7f6d241ff700 1 Processor -- start 2023-12-26T21:50:05.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.450+0000 7f6d241ff700 1 -- start start 2023-12-26T21:50:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.451+0000 7f6d241ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6d1c159920 0x7f6d1c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.451+0000 7f6d241ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c159cf0 0x7f6d1c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.451+0000 7f6d241ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c0c8bf0 con 0x7f6d1c1554d0 2023-12-26T21:50:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.451+0000 7f6d241ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c8e60 con 0x7f6d1c159920 2023-12-26T21:50:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.451+0000 7f6d241ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c9140 con 0x7f6d1c159cf0 2023-12-26T21:50:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.451+0000 7f6d21f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6d1c159920 0x7f6d1c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.451+0000 7f6d2279c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6d1c1554d0 0x7f6d1c155930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59484/0 (socket says 172.21.15.131:59484) 2023-12-26T21:50:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.451+0000 7f6d2279c700 1 -- 172.21.15.131:0/3739445765 learned_addr learned my addr 172.21.15.131:0/3739445765 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.451+0000 7f6d2179a700 1 --2- 172.21.15.131:0/3739445765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c159cf0 0x7f6d1c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.452+0000 7f6d21f9b700 1 -- 172.21.15.131:0/3739445765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c159cf0 msgr2=0x7f6d1c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.452+0000 7f6d21f9b700 1 --2- 172.21.15.131:0/3739445765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c159cf0 0x7f6d1c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.452+0000 7f6d21f9b700 1 -- 172.21.15.131:0/3739445765 >> v1:172.21.15.131:6789/0 conn(0x7f6d1c1554d0 legacy=0x7f6d1c155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.452+0000 7f6d21f9b700 1 -- 172.21.15.131:0/3739445765 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c0c9420 con 0x7f6d1c159920 2023-12-26T21:50:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.452+0000 7f6d20f99700 1 -- 172.21.15.131:0/3739445765 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844881425 0 0) 0x7f6d1c0c8bf0 con 0x7f6d1c1554d0 2023-12-26T21:50:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.452+0000 7f6d21f9b700 1 --2- 172.21.15.131:0/3739445765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 0x7f6d1c0b00e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6d180098e0 tx=0x7f6d180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4495b1706dd8e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.452+0000 7f6d20f99700 1 -- 172.21.15.131:0/3739445765 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800b640 con 0x7f6d1c159920 2023-12-26T21:50:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.452+0000 7f6d20f99700 1 -- 172.21.15.131:0/3739445765 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1800b7a0 con 0x7f6d1c159920 2023-12-26T21:50:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.452+0000 7f6d20f99700 1 -- 172.21.15.131:0/3739445765 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800bb40 con 0x7f6d1c159920 2023-12-26T21:50:05.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.453+0000 7f6d241ff700 1 -- 172.21.15.131:0/3739445765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 msgr2=0x7f6d1c0b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.453+0000 7f6d241ff700 1 --2- 172.21.15.131:0/3739445765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 0x7f6d1c0b00e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6d180098e0 tx=0x7f6d180091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.453+0000 7f6d241ff700 1 -- 172.21.15.131:0/3739445765 shutdown_connections 2023-12-26T21:50:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.453+0000 7f6d241ff700 1 --2- 172.21.15.131:0/3739445765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 0x7f6d1c0b00e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.453+0000 7f6d241ff700 1 --2- 172.21.15.131:0/3739445765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c159cf0 0x7f6d1c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.453+0000 7f6d241ff700 1 -- 172.21.15.131:0/3739445765 >> 172.21.15.131:0/3739445765 conn(0x7f6d1c068040 msgr2=0x7f6d1c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.453+0000 7f6d241ff700 1 -- 172.21.15.131:0/3739445765 shutdown_connections 2023-12-26T21:50:05.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.453+0000 7f6d241ff700 1 -- 172.21.15.131:0/3739445765 wait complete. 2023-12-26T21:50:05.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.454+0000 7f6d241ff700 1 Processor -- start 2023-12-26T21:50:05.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.455+0000 7f6d241ff700 1 -- start start 2023-12-26T21:50:05.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.455+0000 7f6d241ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c1554d0 0x7f6d1c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.455+0000 7f6d241ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 0x7f6d1c15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.455+0000 7f6d241ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c159cf0 0x7f6d1c142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.455+0000 7f6d241ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c15cc60 con 0x7f6d1c159920 2023-12-26T21:50:05.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.455+0000 7f6d241ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c15cf40 con 0x7f6d1c159cf0 2023-12-26T21:50:05.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.455+0000 7f6d241ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c15d220 con 0x7f6d1c1554d0 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2179a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 0x7f6d1c15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2179a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 0x7f6d1c15c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49720/0 (socket says 172.21.15.131:49720) 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2179a700 1 -- 172.21.15.131:0/3826856594 learned_addr learned my addr 172.21.15.131:0/3826856594 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d21f9b700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c1554d0 0x7f6d1c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2279c700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c159cf0 0x7f6d1c142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2179a700 1 -- 172.21.15.131:0/3826856594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c1554d0 msgr2=0x7f6d1c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2179a700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c1554d0 0x7f6d1c06ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2179a700 1 -- 172.21.15.131:0/3826856594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c159cf0 msgr2=0x7f6d1c142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2179a700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c159cf0 0x7f6d1c142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2179a700 1 -- 172.21.15.131:0/3826856594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d18009420 con 0x7f6d1c159920 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2279c700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c159cf0 0x7f6d1c142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.456+0000 7f6d2179a700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 0x7f6d1c15c380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00ec80 tx=0x7f6d0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.457+0000 7f6d11ffb700 1 -- 172.21.15.131:0/3826856594 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00c850 con 0x7f6d1c159920 2023-12-26T21:50:05.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.457+0000 7f6d11ffb700 1 -- 172.21.15.131:0/3826856594 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c00c9b0 con 0x7f6d1c159920 2023-12-26T21:50:05.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.457+0000 7f6d11ffb700 1 -- 172.21.15.131:0/3826856594 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00cd50 con 0x7f6d1c159920 2023-12-26T21:50:05.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.457+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c15d500 con 0x7f6d1c159920 2023-12-26T21:50:05.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.457+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c15c8c0 con 0x7f6d1c159920 2023-12-26T21:50:05.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.458+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c0657e0 con 0x7f6d1c159920 2023-12-26T21:50:05.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.459+0000 7f6d11ffb700 1 -- 172.21.15.131:0/3826856594 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6d0c013680 con 0x7f6d1c159920 2023-12-26T21:50:05.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.460+0000 7f6d11ffb700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6d00066f30 0x7f6d000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.460+0000 7f6d11ffb700 1 -- 172.21.15.131:0/3826856594 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6d0c0849e0 con 0x7f6d1c159920 2023-12-26T21:50:05.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.460+0000 7f6d21f9b700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6d00066f30 0x7f6d000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.460+0000 7f6d21f9b700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6d00066f30 0x7f6d000693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6d180091e0 tx=0x7f6d18017000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:05.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.464+0000 7f6d11ffb700 1 -- 172.21.15.131:0/3826856594 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d0c051a40 con 0x7f6d1c159920 2023-12-26T21:50:05.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.657+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d1c0c9700 con 0x7f6d1c159920 2023-12-26T21:50:05.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.658+0000 7f6d11ffb700 1 -- 172.21.15.131:0/3826856594 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+15963 (secure 0 0 0) 0x7f6d0c055060 con 0x7f6d1c159920 2023-12-26T21:50:05.659 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:05.659 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":63,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:50:04.698789+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:50:02.860261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-26T21:50:03.665608+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:50:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.660+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6d00066f30 msgr2=0x7f6d000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.660+0000 7f6d241ff700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6d00066f30 0x7f6d000693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6d180091e0 tx=0x7f6d18017000 comp rx=0 tx=0).stop 2023-12-26T21:50:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.660+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 msgr2=0x7f6d1c15c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.660+0000 7f6d241ff700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 0x7f6d1c15c380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00ec80 tx=0x7f6d0c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.661+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 shutdown_connections 2023-12-26T21:50:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.661+0000 7f6d241ff700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c159cf0 0x7f6d1c142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.661+0000 7f6d241ff700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f6d00066f30 0x7f6d000693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.661+0000 7f6d241ff700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c1554d0 0x7f6d1c06ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.661+0000 7f6d241ff700 1 --2- 172.21.15.131:0/3826856594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d1c159920 0x7f6d1c15c380 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.661+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 >> 172.21.15.131:0/3826856594 conn(0x7f6d1c068040 msgr2=0x7f6d1c0aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.661+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 shutdown_connections 2023-12-26T21:50:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.661+0000 7f6d241ff700 1 -- 172.21.15.131:0/3826856594 wait complete. 2023-12-26T21:50:05.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:50:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.868+0000 7f9835bff700 1 Processor -- start 2023-12-26T21:50:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.868+0000 7f9835bff700 1 -- start start 2023-12-26T21:50:05.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.869+0000 7f9835bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f98300b0750 0x7f98300b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.869+0000 7f9835bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300c9400 0x7f98300c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.869+0000 7f9835bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9830156940 con 0x7f98300cb320 2023-12-26T21:50:05.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.869+0000 7f9835bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98300c8cc0 con 0x7f98300b0750 2023-12-26T21:50:05.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.869+0000 7f9835bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98300ad700 con 0x7f98300c9400 2023-12-26T21:50:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.870+0000 7f982ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300c9400 0x7f98300c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.870+0000 7f982ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300c9400 0x7f98300c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:47490/0 (socket says 172.21.15.131:47490) 2023-12-26T21:50:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.870+0000 7f982f7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f98300cb320 0x7f9830069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:40114/0 (socket says 172.21.15.131:40114) 2023-12-26T21:50:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.870+0000 7f982ffff700 1 -- 172.21.15.131:0/153401021 learned_addr learned my addr 172.21.15.131:0/153401021 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.870+0000 7f982effd700 1 --2- 172.21.15.131:0/153401021 >> v2:172.21.15.190:3300/0 conn(0x7f98300b0750 0x7f98300b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.870+0000 7f982ffff700 1 -- 172.21.15.131:0/153401021 >> v2:172.21.15.190:3300/0 conn(0x7f98300b0750 msgr2=0x7f98300b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:05.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.870+0000 7f982ffff700 1 --2- 172.21.15.131:0/153401021 >> v2:172.21.15.190:3300/0 conn(0x7f98300b0750 0x7f98300b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.870+0000 7f982ffff700 1 -- 172.21.15.131:0/153401021 >> v1:172.21.15.190:6789/0 conn(0x7f98300cb320 legacy=0x7f9830069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.870+0000 7f982ffff700 1 -- 172.21.15.131:0/153401021 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98300ae500 con 0x7f98300c9400 2023-12-26T21:50:05.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f982ffff700 1 --2- 172.21.15.131:0/153401021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300c9400 0x7f98300c9880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9828009f20 tx=0x7f9828009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac6997da5f3d72b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:05.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f982e7fc700 1 -- 172.21.15.131:0/153401021 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280092e0 con 0x7f98300c9400 2023-12-26T21:50:05.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f982e7fc700 1 -- 172.21.15.131:0/153401021 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982800b9d0 con 0x7f98300c9400 2023-12-26T21:50:05.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f982e7fc700 1 -- 172.21.15.131:0/153401021 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982800bd90 con 0x7f98300c9400 2023-12-26T21:50:05.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f9835bff700 1 -- 172.21.15.131:0/153401021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300c9400 msgr2=0x7f98300c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f9835bff700 1 --2- 172.21.15.131:0/153401021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300c9400 0x7f98300c9880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9828009f20 tx=0x7f9828009840 comp rx=0 tx=0).stop 2023-12-26T21:50:05.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f9835bff700 1 -- 172.21.15.131:0/153401021 shutdown_connections 2023-12-26T21:50:05.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f9835bff700 1 --2- 172.21.15.131:0/153401021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300c9400 0x7f98300c9880 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f9835bff700 1 --2- 172.21.15.131:0/153401021 >> v2:172.21.15.190:3300/0 conn(0x7f98300b0750 0x7f98300b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f9835bff700 1 -- 172.21.15.131:0/153401021 >> 172.21.15.131:0/153401021 conn(0x7f9830068040 msgr2=0x7f9830069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:05.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f9835bff700 1 -- 172.21.15.131:0/153401021 shutdown_connections 2023-12-26T21:50:05.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.871+0000 7f9835bff700 1 -- 172.21.15.131:0/153401021 wait complete. 2023-12-26T21:50:05.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.873+0000 7f9835bff700 1 Processor -- start 2023-12-26T21:50:05.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.873+0000 7f9835bff700 1 -- start start 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.874+0000 7f9835bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300b0750 0x7f9830144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.874+0000 7f9835bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98300c9400 0x7f9830156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.874+0000 7f9835bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98300cb320 0x7f98301578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.874+0000 7f9835bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983016ac40 con 0x7f98300cb320 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.874+0000 7f9835bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983016a030 con 0x7f98300c9400 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.874+0000 7f9835bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983016a310 con 0x7f98300b0750 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.874+0000 7f982ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98300cb320 0x7f98301578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.874+0000 7f982ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98300cb320 0x7f98301578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49736/0 (socket says 172.21.15.131:49736) 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.874+0000 7f982ffff700 1 -- 172.21.15.131:0/3406109900 learned_addr learned my addr 172.21.15.131:0/3406109900 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f982f7fe700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300b0750 0x7f9830144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f982effd700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98300c9400 0x7f9830156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f982ffff700 1 -- 172.21.15.131:0/3406109900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300b0750 msgr2=0x7f9830144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f982ffff700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300b0750 0x7f9830144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f982ffff700 1 -- 172.21.15.131:0/3406109900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98300c9400 msgr2=0x7f9830156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f982ffff700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98300c9400 0x7f9830156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f982ffff700 1 -- 172.21.15.131:0/3406109900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9828009ae0 con 0x7f98300cb320 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f982effd700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98300c9400 0x7f9830156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f982ffff700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98300cb320 0x7f98301578a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f982800b610 tx=0x7f982801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f98177fe700 1 -- 172.21.15.131:0/3406109900 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280032f0 con 0x7f98300cb320 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f98177fe700 1 -- 172.21.15.131:0/3406109900 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982801a660 con 0x7f98300cb320 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f98177fe700 1 -- 172.21.15.131:0/3406109900 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982801aa00 con 0x7f98300cb320 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983016a5f0 con 0x7f98300cb320 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.875+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9830169ce0 con 0x7f98300cb320 2023-12-26T21:50:05.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.876+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98300657e0 con 0x7f98300cb320 2023-12-26T21:50:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.877+0000 7f98177fe700 1 -- 172.21.15.131:0/3406109900 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9828003040 con 0x7f98300cb320 2023-12-26T21:50:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.878+0000 7f98177fe700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f981c066f30 0x7f981c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.878+0000 7f98177fe700 1 -- 172.21.15.131:0/3406109900 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9828086490 con 0x7f98300cb320 2023-12-26T21:50:05.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.878+0000 7f982f7fe700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f981c066f30 0x7f981c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:05.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.879+0000 7f982f7fe700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f981c066f30 0x7f981c0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9820007fd0 tx=0x7f9820007d70 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:05.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:05.882+0000 7f98177fe700 1 -- 172.21.15.131:0/3406109900 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98280534f0 con 0x7f98300cb320 2023-12-26T21:50:06.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.076+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9830159cf0 con 0x7f98300cb320 2023-12-26T21:50:06.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.077+0000 7f98177fe700 1 -- 172.21.15.131:0/3406109900 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+15963 (secure 0 0 0) 0x7f9828056b10 con 0x7f98300cb320 2023-12-26T21:50:06.078 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:06.078 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":63,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:50:04.698789+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:50:02.860261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-26T21:50:03.665608+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:50:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f981c066f30 msgr2=0x7f981c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f981c066f30 0x7f981c0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9820007fd0 tx=0x7f9820007d70 comp rx=0 tx=0).stop 2023-12-26T21:50:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98300cb320 msgr2=0x7f98301578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98300cb320 0x7f98301578a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f982800b610 tx=0x7f982801a000 comp rx=0 tx=0).stop 2023-12-26T21:50:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 shutdown_connections 2023-12-26T21:50:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98300cb320 0x7f98301578a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f981c066f30 0x7f981c0693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98300b0750 0x7f9830144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 --2- 172.21.15.131:0/3406109900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98300c9400 0x7f9830156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.079+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 >> 172.21.15.131:0/3406109900 conn(0x7f9830068040 msgr2=0x7f983006d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:06.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.080+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 shutdown_connections 2023-12-26T21:50:06.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.080+0000 7f9835bff700 1 -- 172.21.15.131:0/3406109900 wait complete. 2023-12-26T21:50:06.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T21:50:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.295+0000 7f5656bff700 1 Processor -- start 2023-12-26T21:50:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.295+0000 7f5656bff700 1 -- start start 2023-12-26T21:50:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.297+0000 7f5656bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5650159a70 0x7f56500b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.297+0000 7f5656bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56500afcd0 con 0x7f56501596a0 2023-12-26T21:50:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.297+0000 7f5656bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56500b3e70 con 0x7f56500b2a90 2023-12-26T21:50:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.297+0000 7f5656bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b0f20 con 0x7f5650159a70 2023-12-26T21:50:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.297+0000 7f565499b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f56501596a0 0x7f56500b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59494/0 (socket says 172.21.15.131:59494) 2023-12-26T21:50:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.297+0000 7f565499b700 1 -- 172.21.15.131:0/1226358820 learned_addr learned my addr 172.21.15.131:0/1226358820 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.297+0000 7f564ffff700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5650159a70 0x7f56500b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.297+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004918115 0 0) 0x7f56500afcd0 con 0x7f56501596a0 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.297+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5640003570 con 0x7f56501596a0 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594357313 0 0) 0x7f56500b3e70 con 0x7f56500b2a90 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56500afcd0 con 0x7f56500b2a90 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 624261109 0 0) 0x7f5640003570 con 0x7f56501596a0 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5650159a70 msgr2=0x7f56500b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5650159a70 0x7f56500b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 >> v1:172.21.15.190:6789/0 conn(0x7f56500b2a90 legacy=0x7f56500b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500b3540 con 0x7f56501596a0 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f5644001470 con 0x7f56501596a0 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 >> v1:172.21.15.131:6789/0 conn(0x7f56501596a0 legacy=0x7f56500b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5640003c20 0x7f5640004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5650159a70 0x7f5640004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5640004d50 0x7f56400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b0f20 con 0x7f5640004d50 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564000bbf0 con 0x7f5640003c20 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564000bd60 con 0x7f5650159a70 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f565519c700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5640004d50 0x7f56400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f564ffff700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5650159a70 0x7f5640004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.298+0000 7f565499b700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5640003c20 0x7f5640004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.299+0000 7f565519c700 1 -- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5650159a70 msgr2=0x7f5640004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.299+0000 7f565519c700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5650159a70 0x7f5640004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.299+0000 7f565519c700 1 -- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5640003c20 msgr2=0x7f5640004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.299+0000 7f565519c700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5640003c20 0x7f5640004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.299+0000 7f565519c700 1 -- 172.21.15.131:0/1226358820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5640003570 con 0x7f5640004d50 2023-12-26T21:50:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.299+0000 7f565519c700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5640004d50 0x7f56400051c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f56480020f0 tx=0x7f56480017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad883d60e0cfb0c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:06.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.299+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648001ab0 con 0x7f5640004d50 2023-12-26T21:50:06.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.299+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564800c8b0 con 0x7f5640004d50 2023-12-26T21:50:06.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.299+0000 7f564f7fe700 1 -- 172.21.15.131:0/1226358820 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800cc70 con 0x7f5640004d50 2023-12-26T21:50:06.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.300+0000 7f5656bff700 1 -- 172.21.15.131:0/1226358820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5640004d50 msgr2=0x7f56400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:06.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.300+0000 7f5656bff700 1 --2- 172.21.15.131:0/1226358820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5640004d50 0x7f56400051c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f56480020f0 tx=0x7f56480017a0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.300+0000 7f565519c700 1 -- 172.21.15.131:0/1226358820 reap_dead start 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.300+0000 7f5656bff700 1 -- 172.21.15.131:0/1226358820 shutdown_connections 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.300+0000 7f5656bff700 1 -- 172.21.15.131:0/1226358820 >> 172.21.15.131:0/1226358820 conn(0x7f565006ece0 msgr2=0x7f565006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.300+0000 7f5656bff700 1 -- 172.21.15.131:0/1226358820 shutdown_connections 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.300+0000 7f5656bff700 1 -- 172.21.15.131:0/1226358820 wait complete. 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.301+0000 7f5656bff700 1 Processor -- start 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.301+0000 7f5656bff700 1 -- start start 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.302+0000 7f5656bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5640004d50 0x7f5650154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.302+0000 7f5656bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56500c6870 0x7f565015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.302+0000 7f5656bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565015acb0 0x7f56500c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.302+0000 7f5656bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650154960 con 0x7f565015acb0 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.302+0000 7f5656bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565015b120 con 0x7f5640004d50 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.302+0000 7f5656bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b5d00 con 0x7f56500c6870 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565519c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565015acb0 0x7f56500c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:06.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565519c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565015acb0 0x7f56500c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49748/0 (socket says 172.21.15.131:49748) 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565519c700 1 -- 172.21.15.131:0/1196611047 learned_addr learned my addr 172.21.15.131:0/1196611047 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f564ffff700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56500c6870 0x7f565015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565499b700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5640004d50 0x7f5650154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565519c700 1 -- 172.21.15.131:0/1196611047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56500c6870 msgr2=0x7f565015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565519c700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56500c6870 0x7f565015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565519c700 1 -- 172.21.15.131:0/1196611047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5640004d50 msgr2=0x7f5650154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565519c700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5640004d50 0x7f5650154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565519c700 1 -- 172.21.15.131:0/1196611047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500b4e80 con 0x7f565015acb0 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f565519c700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565015acb0 0x7f56500c5080 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f564800c6a0 tx=0x7f5648016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f564cff9700 1 -- 172.21.15.131:0/1196611047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800cc30 con 0x7f565015acb0 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f564cff9700 1 -- 172.21.15.131:0/1196611047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5648001080 con 0x7f565015acb0 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.303+0000 7f564cff9700 1 -- 172.21.15.131:0/1196611047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648006e10 con 0x7f565015acb0 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.304+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56500b50f0 con 0x7f565015acb0 2023-12-26T21:50:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.304+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56500b56b0 con 0x7f565015acb0 2023-12-26T21:50:06.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.305+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56500657e0 con 0x7f565015acb0 2023-12-26T21:50:06.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.305+0000 7f564cff9700 1 -- 172.21.15.131:0/1196611047 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5648003030 con 0x7f565015acb0 2023-12-26T21:50:06.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.306+0000 7f564cff9700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5634066f30 0x7f56340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:06.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.306+0000 7f564cff9700 1 -- 172.21.15.131:0/1196611047 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f56480868e0 con 0x7f565015acb0 2023-12-26T21:50:06.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.306+0000 7f565499b700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5634066f30 0x7f56340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:06.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.307+0000 7f565499b700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5634066f30 0x7f56340693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5644001400 tx=0x7f5644009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:06.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.311+0000 7f564cff9700 1 -- 172.21.15.131:0/1196611047 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5648053940 con 0x7f565015acb0 2023-12-26T21:50:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:06.546+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f56500b3820 con 0x7f565015acb0 2023-12-26T21:50:07.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.233+0000 7f564cff9700 1 -- 172.21.15.131:0/1196611047 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5648056f60 con 0x7f565015acb0 2023-12-26T21:50:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5634066f30 msgr2=0x7f56340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5634066f30 0x7f56340693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5644001400 tx=0x7f5644009040 comp rx=0 tx=0).stop 2023-12-26T21:50:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565015acb0 msgr2=0x7f56500c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565015acb0 0x7f56500c5080 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f564800c6a0 tx=0x7f5648016000 comp rx=0 tx=0).stop 2023-12-26T21:50:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 shutdown_connections 2023-12-26T21:50:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565015acb0 0x7f56500c5080 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5640004d50 0x7f5650154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5634066f30 0x7f56340693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:07.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 --2- 172.21.15.131:0/1196611047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56500c6870 0x7f565015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 >> 172.21.15.131:0/1196611047 conn(0x7f565006ece0 msgr2=0x7f5650068d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 shutdown_connections 2023-12-26T21:50:07.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.236+0000 7f5656bff700 1 -- 172.21.15.131:0/1196611047 wait complete. 2023-12-26T21:50:07.258 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T21:50:07.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.452+0000 7f0a6edff700 1 Processor -- start 2023-12-26T21:50:07.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.452+0000 7f0a6edff700 1 -- start start 2023-12-26T21:50:07.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a6edff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680b0750 0x7f0a680b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:07.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a6edff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a680c9400 0x7f0a680c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:07.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a6edff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a68156940 con 0x7f0a680cb320 2023-12-26T21:50:07.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a6edff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a680c8cc0 con 0x7f0a680c9400 2023-12-26T21:50:07.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a6edff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a680ad700 con 0x7f0a680b0750 2023-12-26T21:50:07.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a6cb9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0a680cb320 0x7f0a68069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59502/0 (socket says 172.21.15.131:59502) 2023-12-26T21:50:07.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a6cb9b700 1 -- 172.21.15.131:0/442734117 learned_addr learned my addr 172.21.15.131:0/442734117 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:07.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a6d39c700 1 --2- 172.21.15.131:0/442734117 >> v2:172.21.15.131:3300/0 conn(0x7f0a680c9400 0x7f0a680c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:07.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a67fff700 1 --2- 172.21.15.131:0/442734117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680b0750 0x7f0a680b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a677fe700 1 -- 172.21.15.131:0/442734117 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368471248 0 0) 0x7f0a68156940 con 0x7f0a680cb320 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.454+0000 7f0a677fe700 1 -- 172.21.15.131:0/442734117 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a58003570 con 0x7f0a680cb320 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.455+0000 7f0a6d39c700 1 -- 172.21.15.131:0/442734117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680b0750 msgr2=0x7f0a680b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.455+0000 7f0a6d39c700 1 --2- 172.21.15.131:0/442734117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680b0750 0x7f0a680b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.455+0000 7f0a6d39c700 1 -- 172.21.15.131:0/442734117 >> v1:172.21.15.131:6789/0 conn(0x7f0a680cb320 legacy=0x7f0a68069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.455+0000 7f0a6d39c700 1 -- 172.21.15.131:0/442734117 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a680b1100 con 0x7f0a680c9400 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.455+0000 7f0a6d39c700 1 --2- 172.21.15.131:0/442734117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680c9400 0x7f0a680c9880 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009e60 tx=0x7f0a60009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acc8f6a2c85b3e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.455+0000 7f0a677fe700 1 -- 172.21.15.131:0/442734117 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000e070 con 0x7f0a680c9400 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.455+0000 7f0a677fe700 1 -- 172.21.15.131:0/442734117 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6000bab0 con 0x7f0a680c9400 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.455+0000 7f0a677fe700 1 -- 172.21.15.131:0/442734117 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000be30 con 0x7f0a680c9400 2023-12-26T21:50:07.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.456+0000 7f0a6edff700 1 -- 172.21.15.131:0/442734117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680c9400 msgr2=0x7f0a680c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:07.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.456+0000 7f0a6edff700 1 --2- 172.21.15.131:0/442734117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680c9400 0x7f0a680c9880 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009e60 tx=0x7f0a60009780 comp rx=0 tx=0).stop 2023-12-26T21:50:07.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.456+0000 7f0a6edff700 1 -- 172.21.15.131:0/442734117 shutdown_connections 2023-12-26T21:50:07.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.456+0000 7f0a6edff700 1 --2- 172.21.15.131:0/442734117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680c9400 0x7f0a680c9880 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:07.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.456+0000 7f0a6edff700 1 --2- 172.21.15.131:0/442734117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680b0750 0x7f0a680b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:07.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.456+0000 7f0a6edff700 1 -- 172.21.15.131:0/442734117 >> 172.21.15.131:0/442734117 conn(0x7f0a68068040 msgr2=0x7f0a68069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:07.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.456+0000 7f0a6edff700 1 -- 172.21.15.131:0/442734117 shutdown_connections 2023-12-26T21:50:07.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.456+0000 7f0a6edff700 1 -- 172.21.15.131:0/442734117 wait complete. 2023-12-26T21:50:07.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.457+0000 7f0a6edff700 1 Processor -- start 2023-12-26T21:50:07.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.457+0000 7f0a6edff700 1 -- start start 2023-12-26T21:50:07.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.458+0000 7f0a6edff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680b0750 0x7f0a6815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:07.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.458+0000 7f0a6edff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a680c9400 0x7f0a6815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:07.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.458+0000 7f0a6edff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680cb320 0x7f0a680ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:07.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.458+0000 7f0a6edff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6815d0a0 con 0x7f0a680b0750 2023-12-26T21:50:07.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.458+0000 7f0a6edff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681696e0 con 0x7f0a680c9400 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.458+0000 7f0a6edff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6815c520 con 0x7f0a680cb320 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6cb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680b0750 0x7f0a6815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6cb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680b0750 0x7f0a6815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49760/0 (socket says 172.21.15.131:49760) 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6cb9b700 1 -- 172.21.15.131:0/1065650159 learned_addr learned my addr 172.21.15.131:0/1065650159 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a67fff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a680c9400 0x7f0a6815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6d39c700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680cb320 0x7f0a680ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6cb9b700 1 -- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680cb320 msgr2=0x7f0a680ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6cb9b700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680cb320 0x7f0a680ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6cb9b700 1 -- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a680c9400 msgr2=0x7f0a6815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6cb9b700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a680c9400 0x7f0a6815e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6cb9b700 1 -- 172.21.15.131:0/1065650159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a60009a20 con 0x7f0a680b0750 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6d39c700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680cb320 0x7f0a680ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a67fff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a680c9400 0x7f0a6815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:07.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6cb9b700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680b0750 0x7f0a6815bc20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00c580 tx=0x7f0a5c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:07.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a64ff9700 1 -- 172.21.15.131:0/1065650159 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c005b30 con 0x7f0a680b0750 2023-12-26T21:50:07.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a64ff9700 1 -- 172.21.15.131:0/1065650159 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5c005cb0 con 0x7f0a680b0750 2023-12-26T21:50:07.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6815c800 con 0x7f0a680b0750 2023-12-26T21:50:07.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.459+0000 7f0a64ff9700 1 -- 172.21.15.131:0/1065650159 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c007320 con 0x7f0a680b0750 2023-12-26T21:50:07.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.460+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6815c160 con 0x7f0a680b0750 2023-12-26T21:50:07.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.461+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a680657e0 con 0x7f0a680b0750 2023-12-26T21:50:07.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.461+0000 7f0a64ff9700 1 -- 172.21.15.131:0/1065650159 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0a5c01b3b0 con 0x7f0a680b0750 2023-12-26T21:50:07.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.463+0000 7f0a64ff9700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a4c066f30 0x7f0a4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:07.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.463+0000 7f0a64ff9700 1 -- 172.21.15.131:0/1065650159 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0a5c083b00 con 0x7f0a680b0750 2023-12-26T21:50:07.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.463+0000 7f0a67fff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a4c066f30 0x7f0a4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:07.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.464+0000 7f0a67fff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a4c066f30 0x7f0a4c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a6815b220 tx=0x7f0a54008040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:07.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.467+0000 7f0a64ff9700 1 -- 172.21.15.131:0/1065650159 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a5c050c00 con 0x7f0a680b0750 2023-12-26T21:50:07.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:07.702+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0a68169820 con 0x7f0a680b0750 2023-12-26T21:50:08.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.234+0000 7f0a64ff9700 1 -- 172.21.15.131:0/1065650159 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0a68169820 con 0x7f0a680b0750 2023-12-26T21:50:08.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.236+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a4c066f30 msgr2=0x7f0a4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.236+0000 7f0a6edff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a4c066f30 0x7f0a4c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a6815b220 tx=0x7f0a54008040 comp rx=0 tx=0).stop 2023-12-26T21:50:08.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.236+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680b0750 msgr2=0x7f0a6815bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.236+0000 7f0a6edff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680b0750 0x7f0a6815bc20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00c580 tx=0x7f0a5c007df0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.237+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 shutdown_connections 2023-12-26T21:50:08.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.237+0000 7f0a6edff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a4c066f30 0x7f0a4c0693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.237+0000 7f0a6edff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a680cb320 0x7f0a680ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.237+0000 7f0a6edff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a680b0750 0x7f0a6815bc20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.237+0000 7f0a6edff700 1 --2- 172.21.15.131:0/1065650159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a680c9400 0x7f0a6815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.237+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 >> 172.21.15.131:0/1065650159 conn(0x7f0a68068040 msgr2=0x7f0a6806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:08.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.237+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 shutdown_connections 2023-12-26T21:50:08.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.237+0000 7f0a6edff700 1 -- 172.21.15.131:0/1065650159 wait complete. 2023-12-26T21:50:08.256 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:50:08.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.446+0000 7fedf33ff700 1 Processor -- start 2023-12-26T21:50:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.446+0000 7fedf33ff700 1 -- start start 2023-12-26T21:50:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.447+0000 7fedf33ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fedec1596a0 0x7fedec0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.447+0000 7fedf33ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedec0afcd0 con 0x7fedec159a70 2023-12-26T21:50:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.447+0000 7fedf33ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedec0b3e70 con 0x7fedec0b0870 2023-12-26T21:50:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.447+0000 7fedf33ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedec0b0cb0 con 0x7fedec1596a0 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.447+0000 7fedf099a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fedec159a70 0x7fedec0b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59510/0 (socket says 172.21.15.131:59510) 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.448+0000 7fedf099a700 1 -- 172.21.15.131:0/3345925877 learned_addr learned my addr 172.21.15.131:0/3345925877 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.448+0000 7fedf119b700 1 --2- 172.21.15.131:0/3345925877 >> v2:172.21.15.131:3300/0 conn(0x7fedec1596a0 0x7fedec0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.448+0000 7fedf119b700 1 -- 172.21.15.131:0/3345925877 >> v1:172.21.15.190:6789/0 conn(0x7fedec0b0870 legacy=0x7fedec0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.448+0000 7fedf119b700 1 -- 172.21.15.131:0/3345925877 >> v1:172.21.15.131:6789/0 conn(0x7fedec159a70 legacy=0x7fedec0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.448+0000 7fedf119b700 1 -- 172.21.15.131:0/3345925877 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec0b0f20 con 0x7fedec1596a0 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.448+0000 7fedf119b700 1 --2- 172.21.15.131:0/3345925877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 0x7fedec0b2380 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fede80098e0 tx=0x7fede80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58297064b0ac2c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.448+0000 7fede3fff700 1 -- 172.21.15.131:0/3345925877 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8009e80 con 0x7fedec1596a0 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.449+0000 7fede3fff700 1 -- 172.21.15.131:0/3345925877 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede800b860 con 0x7fedec1596a0 2023-12-26T21:50:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.449+0000 7fede3fff700 1 -- 172.21.15.131:0/3345925877 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede800bbe0 con 0x7fedec1596a0 2023-12-26T21:50:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.449+0000 7fedf33ff700 1 -- 172.21.15.131:0/3345925877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 msgr2=0x7fedec0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.449+0000 7fedf33ff700 1 --2- 172.21.15.131:0/3345925877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 0x7fedec0b2380 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fede80098e0 tx=0x7fede80091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.449+0000 7fedf33ff700 1 -- 172.21.15.131:0/3345925877 shutdown_connections 2023-12-26T21:50:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.449+0000 7fedf33ff700 1 --2- 172.21.15.131:0/3345925877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 0x7fedec0b2380 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.449+0000 7fedf33ff700 1 -- 172.21.15.131:0/3345925877 >> 172.21.15.131:0/3345925877 conn(0x7fedec06ece0 msgr2=0x7fedec06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.449+0000 7fedf33ff700 1 -- 172.21.15.131:0/3345925877 shutdown_connections 2023-12-26T21:50:08.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.449+0000 7fedf33ff700 1 -- 172.21.15.131:0/3345925877 wait complete. 2023-12-26T21:50:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.451+0000 7fedf33ff700 1 Processor -- start 2023-12-26T21:50:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.451+0000 7fedf33ff700 1 -- start start 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.451+0000 7fedf33ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedec0b0870 0x7fedec150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.451+0000 7fedf33ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 0x7fedec15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.451+0000 7fedf33ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedec159a70 0x7fedec15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf33ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedec155cd0 con 0x7fedec1596a0 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf33ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedec1511e0 con 0x7fedec0b0870 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf33ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedec0cbc50 con 0x7fedec159a70 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 0x7fedec15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 0x7fedec15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49776/0 (socket says 172.21.15.131:49776) 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf099a700 1 -- 172.21.15.131:0/566979173 learned_addr learned my addr 172.21.15.131:0/566979173 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf119b700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedec0b0870 0x7fedec150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf199c700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedec159a70 0x7fedec15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf099a700 1 -- 172.21.15.131:0/566979173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedec159a70 msgr2=0x7fedec15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf099a700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedec159a70 0x7fedec15ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf099a700 1 -- 172.21.15.131:0/566979173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedec0b0870 msgr2=0x7fedec150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf099a700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedec0b0870 0x7fedec150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.452+0000 7fedf099a700 1 -- 172.21.15.131:0/566979173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede8009420 con 0x7fedec1596a0 2023-12-26T21:50:08.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.453+0000 7fedf099a700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 0x7fedec15c550 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7feddc00c560 tx=0x7feddc001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.453+0000 7fede17fa700 1 -- 172.21.15.131:0/566979173 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc005b30 con 0x7fedec1596a0 2023-12-26T21:50:08.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.453+0000 7fede17fa700 1 -- 172.21.15.131:0/566979173 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc005c90 con 0x7fedec1596a0 2023-12-26T21:50:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.453+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedec155430 con 0x7fedec1596a0 2023-12-26T21:50:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.453+0000 7fede17fa700 1 -- 172.21.15.131:0/566979173 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc007320 con 0x7fedec1596a0 2023-12-26T21:50:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.453+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedec1559f0 con 0x7fedec1596a0 2023-12-26T21:50:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.454+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedec0cba20 con 0x7fedec1596a0 2023-12-26T21:50:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.455+0000 7fede17fa700 1 -- 172.21.15.131:0/566979173 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feddc007690 con 0x7fedec1596a0 2023-12-26T21:50:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.456+0000 7fede17fa700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fedd0066f30 0x7fedd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.456+0000 7fede17fa700 1 -- 172.21.15.131:0/566979173 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7feddc083a80 con 0x7fedec1596a0 2023-12-26T21:50:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.456+0000 7fedf119b700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fedd0066f30 0x7fedd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.456+0000 7fedf119b700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fedd0066f30 0x7fedd00693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fede80093f0 tx=0x7fede8017000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.460+0000 7fede17fa700 1 -- 172.21.15.131:0/566979173 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedec0cba20 con 0x7fedec1596a0 2023-12-26T21:50:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.693+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedec0cba20 con 0x7fedec1596a0 2023-12-26T21:50:08.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.693+0000 7fede17fa700 1 -- 172.21.15.131:0/566979173 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fedec0cba20 con 0x7fedec1596a0 2023-12-26T21:50:08.694 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:08.694 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3316959664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3316959664},{"type":"v1","addr":"172.21.15.190:6834","nonce":3316959664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/320470110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":320470110},{"type":"v1","addr":"172.21.15.190:6835","nonce":320470110}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/1337439654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1337439654},{"type":"v1","addr":"172.21.15.131:6836","nonce":1337439654}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:50:04.699122+0000","modified":"2023-12-26T21:50:08.232778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6837/607340038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":607340038},{"type":"v1","addr":"172.21.15.131:6837","nonce":607340038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-12-26T21:50:08.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fedd0066f30 msgr2=0x7fedd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fedd0066f30 0x7fedd00693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fede80093f0 tx=0x7fede8017000 comp rx=0 tx=0).stop 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 msgr2=0x7fedec15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 0x7fedec15c550 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7feddc00c560 tx=0x7feddc001020 comp rx=0 tx=0).stop 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 shutdown_connections 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fedd0066f30 0x7fedd00693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedec159a70 0x7fedec15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedec0b0870 0x7fedec150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 --2- 172.21.15.131:0/566979173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedec1596a0 0x7fedec15c550 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 >> 172.21.15.131:0/566979173 conn(0x7fedec06ece0 msgr2=0x7fedec158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 shutdown_connections 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.696+0000 7fedf33ff700 1 -- 172.21.15.131:0/566979173 wait complete. 2023-12-26T21:50:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 25 2023-12-26T21:50:08.715 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:50:08.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.904+0000 7fc2017ff700 1 Processor -- start 2023-12-26T21:50:08.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.904+0000 7fc2017ff700 1 -- start start 2023-12-26T21:50:08.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.905+0000 7fc2017ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc1fc06bfd0 0x7fc1fc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.905+0000 7fc2017ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc1fc0cc100 0x7fc1fc0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.905+0000 7fc2017ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0686c0 con 0x7fc1fc13c510 2023-12-26T21:50:08.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.905+0000 7fc2017ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc069520 con 0x7fc1fc06bfd0 2023-12-26T21:50:08.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.905+0000 7fc2017ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc068910 con 0x7fc1fc0cc100 2023-12-26T21:50:08.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.905+0000 7fc1fa7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc1fc06bfd0 0x7fc1fc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.905+0000 7fc1fa7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc1fc06bfd0 0x7fc1fc0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49788/0 (socket says 172.21.15.131:49788) 2023-12-26T21:50:08.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.905+0000 7fc1fa7fc700 1 -- 172.21.15.131:0/1149887574 learned_addr learned my addr 172.21.15.131:0/1149887574 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:08.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.905+0000 7fc1fb7fe700 1 --2- 172.21.15.131:0/1149887574 >> v2:172.21.15.190:3300/0 conn(0x7fc1fc0cc100 0x7fc1fc0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.906+0000 7fc1fa7fc700 1 -- 172.21.15.131:0/1149887574 >> v2:172.21.15.190:3300/0 conn(0x7fc1fc0cc100 msgr2=0x7fc1fc0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:08.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.906+0000 7fc1fa7fc700 1 --2- 172.21.15.131:0/1149887574 >> v2:172.21.15.190:3300/0 conn(0x7fc1fc0cc100 0x7fc1fc0cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.906+0000 7fc1fa7fc700 1 -- 172.21.15.131:0/1149887574 >> v1:172.21.15.190:6789/0 conn(0x7fc1fc13c510 legacy=0x7fc1fc06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.906+0000 7fc1fa7fc700 1 -- 172.21.15.131:0/1149887574 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc068bf0 con 0x7fc1fc06bfd0 2023-12-26T21:50:08.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.906+0000 7fc1fa7fc700 1 --2- 172.21.15.131:0/1149887574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc06bfd0 0x7fc1fc0c8a20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0cf0a0 tx=0x7fc1f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce09f130a54eed5a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:08.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.906+0000 7fc1f9ffb700 1 -- 172.21.15.131:0/1149887574 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f000b860 con 0x7fc1fc06bfd0 2023-12-26T21:50:08.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.906+0000 7fc1f9ffb700 1 -- 172.21.15.131:0/1149887574 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f000b9c0 con 0x7fc1fc06bfd0 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.906+0000 7fc1f9ffb700 1 -- 172.21.15.131:0/1149887574 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f000bd60 con 0x7fc1fc06bfd0 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.907+0000 7fc2017ff700 1 -- 172.21.15.131:0/1149887574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc06bfd0 msgr2=0x7fc1fc0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.907+0000 7fc2017ff700 1 --2- 172.21.15.131:0/1149887574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc06bfd0 0x7fc1fc0c8a20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0cf0a0 tx=0x7fc1f0009250 comp rx=0 tx=0).stop 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.907+0000 7fc2017ff700 1 -- 172.21.15.131:0/1149887574 shutdown_connections 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.907+0000 7fc2017ff700 1 --2- 172.21.15.131:0/1149887574 >> v2:172.21.15.190:3300/0 conn(0x7fc1fc0cc100 0x7fc1fc0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.907+0000 7fc2017ff700 1 --2- 172.21.15.131:0/1149887574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc06bfd0 0x7fc1fc0c8a20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.907+0000 7fc2017ff700 1 -- 172.21.15.131:0/1149887574 >> 172.21.15.131:0/1149887574 conn(0x7fc1fc0ce3e0 msgr2=0x7fc1fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.907+0000 7fc2017ff700 1 -- 172.21.15.131:0/1149887574 shutdown_connections 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.907+0000 7fc2017ff700 1 -- 172.21.15.131:0/1149887574 wait complete. 2023-12-26T21:50:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.908+0000 7fc2017ff700 1 Processor -- start 2023-12-26T21:50:08.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.909+0000 7fc2017ff700 1 -- start start 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.909+0000 7fc2017ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1fc06bfd0 0x7fc1fc13bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.909+0000 7fc2017ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc0cc100 0x7fc1fc154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.909+0000 7fc2017ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1fc13c510 0x7fc1fc13c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.909+0000 7fc2017ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b2060 con 0x7fc1fc0cc100 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.909+0000 7fc2017ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b2340 con 0x7fc1fc13c510 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.909+0000 7fc2017ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc13c0e0 con 0x7fc1fc06bfd0 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fa7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc0cc100 0x7fc1fc154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fa7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc0cc100 0x7fc1fc154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49790/0 (socket says 172.21.15.131:49790) 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fa7fc700 1 -- 172.21.15.131:0/3330957294 learned_addr learned my addr 172.21.15.131:0/3330957294 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fb7fe700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1fc13c510 0x7fc1fc13c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1faffd700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1fc06bfd0 0x7fc1fc13bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fa7fc700 1 -- 172.21.15.131:0/3330957294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1fc06bfd0 msgr2=0x7fc1fc13bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fa7fc700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1fc06bfd0 0x7fc1fc13bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fa7fc700 1 -- 172.21.15.131:0/3330957294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1fc13c510 msgr2=0x7fc1fc13c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:08.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fa7fc700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1fc13c510 0x7fc1fc13c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:08.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fa7fc700 1 -- 172.21.15.131:0/3330957294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00094f0 con 0x7fc1fc0cc100 2023-12-26T21:50:08.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.910+0000 7fc1fa7fc700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc0cc100 0x7fc1fc154980 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0006fd0 tx=0x7fc1f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:08.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.911+0000 7fc1e2ffd700 1 -- 172.21.15.131:0/3330957294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0024440 con 0x7fc1fc0cc100 2023-12-26T21:50:08.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.911+0000 7fc1e2ffd700 1 -- 172.21.15.131:0/3330957294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f00245a0 con 0x7fc1fc0cc100 2023-12-26T21:50:08.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.911+0000 7fc1e2ffd700 1 -- 172.21.15.131:0/3330957294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0024960 con 0x7fc1fc0cc100 2023-12-26T21:50:08.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.911+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc1551b0 con 0x7fc1fc0cc100 2023-12-26T21:50:08.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.911+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc155770 con 0x7fc1fc0cc100 2023-12-26T21:50:08.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.912+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc0657e0 con 0x7fc1fc0cc100 2023-12-26T21:50:08.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.913+0000 7fc1e2ffd700 1 -- 172.21.15.131:0/3330957294 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc1f001a420 con 0x7fc1fc0cc100 2023-12-26T21:50:08.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.914+0000 7fc1e2ffd700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc1e8066f30 0x7fc1e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:08.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.914+0000 7fc1e2ffd700 1 -- 172.21.15.131:0/3330957294 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc1f00863f0 con 0x7fc1fc0cc100 2023-12-26T21:50:08.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.914+0000 7fc1faffd700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc1e8066f30 0x7fc1e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:08.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.914+0000 7fc1faffd700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc1e8066f30 0x7fc1e80693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc06d100 tx=0x7fc1ec009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:08.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:08.918+0000 7fc1e2ffd700 1 -- 172.21.15.131:0/3330957294 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1f0053450 con 0x7fc1fc0cc100 2023-12-26T21:50:09.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.112+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1fc068ed0 con 0x7fc1fc0cc100 2023-12-26T21:50:09.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.113+0000 7fc1e2ffd700 1 -- 172.21.15.131:0/3330957294 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+15950 (secure 0 0 0) 0x7fc1f0056a70 con 0x7fc1fc0cc100 2023-12-26T21:50:09.114 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:09.114 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":64,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:50:06.212498+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:50:02.860261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-26T21:50:03.665608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:50:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc1e8066f30 msgr2=0x7fc1e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc1e8066f30 0x7fc1e80693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc06d100 tx=0x7fc1ec009040 comp rx=0 tx=0).stop 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc0cc100 msgr2=0x7fc1fc154980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc0cc100 0x7fc1fc154980 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0006fd0 tx=0x7fc1f0017000 comp rx=0 tx=0).stop 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 shutdown_connections 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1fc13c510 0x7fc1fc13c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fc1e8066f30 0x7fc1e80693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1fc06bfd0 0x7fc1fc13bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 --2- 172.21.15.131:0/3330957294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1fc0cc100 0x7fc1fc154980 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 >> 172.21.15.131:0/3330957294 conn(0x7fc1fc0ce3e0 msgr2=0x7fc1fc0cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 shutdown_connections 2023-12-26T21:50:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.116+0000 7fc2017ff700 1 -- 172.21.15.131:0/3330957294 wait complete. 2023-12-26T21:50:09.134 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T21:50:09.135 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:50:09.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.328+0000 7f474a3ff700 1 Processor -- start 2023-12-26T21:50:09.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.328+0000 7f474a3ff700 1 -- start start 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.329+0000 7f474a3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47440c8820 0x7f47440ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.329+0000 7f474a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8bf0 0x7f47440b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.329+0000 7f474a3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4744157a80 con 0x7f4744069d10 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.329+0000 7f474a3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47440cac50 con 0x7f47440c8820 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.329+0000 7f474a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47440caf30 con 0x7f47440c8bf0 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.329+0000 7f4743fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47440c8820 0x7f47440ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.329+0000 7f474899c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4744069d10 0x7f4744157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59520/0 (socket says 172.21.15.131:59520) 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f474899c700 1 -- 172.21.15.131:0/469577875 learned_addr learned my addr 172.21.15.131:0/469577875 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f47437fe700 1 --2- 172.21.15.131:0/469577875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8bf0 0x7f47440b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f4743fff700 1 -- 172.21.15.131:0/469577875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8bf0 msgr2=0x7f47440b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f4743fff700 1 --2- 172.21.15.131:0/469577875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8bf0 0x7f47440b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f4743fff700 1 -- 172.21.15.131:0/469577875 >> v1:172.21.15.131:6789/0 conn(0x7f4744069d10 legacy=0x7f4744157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f4743fff700 1 -- 172.21.15.131:0/469577875 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47440cb210 con 0x7f47440c8820 2023-12-26T21:50:09.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f4742ffd700 1 -- 172.21.15.131:0/469577875 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83818790 0 0) 0x7f4744157a80 con 0x7f4744069d10 2023-12-26T21:50:09.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f4743fff700 1 --2- 172.21.15.131:0/469577875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8820 0x7f47440ca300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f47340098e0 tx=0x7f47340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b465f80d536b62a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:09.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f4742ffd700 1 -- 172.21.15.131:0/469577875 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400b640 con 0x7f47440c8820 2023-12-26T21:50:09.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f4742ffd700 1 -- 172.21.15.131:0/469577875 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473400b7a0 con 0x7f47440c8820 2023-12-26T21:50:09.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.330+0000 7f4742ffd700 1 -- 172.21.15.131:0/469577875 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400bb40 con 0x7f47440c8820 2023-12-26T21:50:09.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.331+0000 7f474a3ff700 1 -- 172.21.15.131:0/469577875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8820 msgr2=0x7f47440ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.331+0000 7f474a3ff700 1 --2- 172.21.15.131:0/469577875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8820 0x7f47440ca300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f47340098e0 tx=0x7f47340091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.331+0000 7f474a3ff700 1 -- 172.21.15.131:0/469577875 shutdown_connections 2023-12-26T21:50:09.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.331+0000 7f474a3ff700 1 --2- 172.21.15.131:0/469577875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8820 0x7f47440ca300 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.331+0000 7f474a3ff700 1 --2- 172.21.15.131:0/469577875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8bf0 0x7f47440b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.331+0000 7f474a3ff700 1 -- 172.21.15.131:0/469577875 >> 172.21.15.131:0/469577875 conn(0x7f4744068040 msgr2=0x7f4744069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:09.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.331+0000 7f474a3ff700 1 -- 172.21.15.131:0/469577875 shutdown_connections 2023-12-26T21:50:09.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.331+0000 7f474a3ff700 1 -- 172.21.15.131:0/469577875 wait complete. 2023-12-26T21:50:09.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.333+0000 7f474a3ff700 1 Processor -- start 2023-12-26T21:50:09.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.333+0000 7f474a3ff700 1 -- start start 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474a3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744069d10 0x7f4744147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8820 0x7f47441485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474a3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8bf0 0x7f474416ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474a3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4744154bc0 con 0x7f47440c8bf0 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474a3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4744155bd0 con 0x7f4744069d10 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4744154e10 con 0x7f47440c8820 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474899c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8bf0 0x7f474416ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474899c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8bf0 0x7f474416ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49818/0 (socket says 172.21.15.131:49818) 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474899c700 1 -- 172.21.15.131:0/4201679594 learned_addr learned my addr 172.21.15.131:0/4201679594 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f4743fff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744069d10 0x7f4744147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f47437fe700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8820 0x7f47441485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474899c700 1 -- 172.21.15.131:0/4201679594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8820 msgr2=0x7f47441485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474899c700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8820 0x7f47441485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474899c700 1 -- 172.21.15.131:0/4201679594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744069d10 msgr2=0x7f4744147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474899c700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744069d10 0x7f4744147d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.335+0000 7f474899c700 1 -- 172.21.15.131:0/4201679594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4734009420 con 0x7f47440c8bf0 2023-12-26T21:50:09.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.336+0000 7f474899c700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8bf0 0x7f474416ad20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f473c00cbf0 tx=0x7f473c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:09.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.336+0000 7f472bfff700 1 -- 172.21.15.131:0/4201679594 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c00a9c0 con 0x7f47440c8bf0 2023-12-26T21:50:09.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.336+0000 7f472bfff700 1 -- 172.21.15.131:0/4201679594 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c00ab20 con 0x7f47440c8bf0 2023-12-26T21:50:09.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.336+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47441550f0 con 0x7f47440c8bf0 2023-12-26T21:50:09.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.336+0000 7f472bfff700 1 -- 172.21.15.131:0/4201679594 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0070f0 con 0x7f47440c8bf0 2023-12-26T21:50:09.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.336+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47441556b0 con 0x7f47440c8bf0 2023-12-26T21:50:09.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.337+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4744156aa0 con 0x7f47440c8bf0 2023-12-26T21:50:09.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.338+0000 7f472bfff700 1 -- 172.21.15.131:0/4201679594 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f473c007520 con 0x7f47440c8bf0 2023-12-26T21:50:09.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.339+0000 7f472bfff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.339+0000 7f472bfff700 1 -- 172.21.15.131:0/4201679594 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f473c084aa0 con 0x7f47440c8bf0 2023-12-26T21:50:09.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.339+0000 7f4743fff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.340+0000 7f4743fff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f47340093f0 tx=0x7f4734017000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:09.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.343+0000 7f472bfff700 1 -- 172.21.15.131:0/4201679594 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f473c051c20 con 0x7f47440c8bf0 2023-12-26T21:50:09.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.533+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47441558d0 con 0x7f4730066f30 2023-12-26T21:50:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.543+0000 7f472bfff700 1 -- 172.21.15.131:0/4201679594 <== mgr.5437 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282183 (secure 0 0 0) 0x7f47441558d0 con 0x7f4730066f30 2023-12-26T21:50:09.548 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 msgr2=0x7f47300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f47340093f0 tx=0x7f4734017000 comp rx=0 tx=0).stop 2023-12-26T21:50:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8bf0 msgr2=0x7f474416ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8bf0 0x7f474416ad20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f473c00cbf0 tx=0x7f473c0019b0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 shutdown_connections 2023-12-26T21:50:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47440c8bf0 0x7f474416ad20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4744069d10 0x7f4744147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f4730066f30 0x7f47300693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 --2- 172.21.15.131:0/4201679594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47440c8820 0x7f47441485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 >> 172.21.15.131:0/4201679594 conn(0x7f4744068040 msgr2=0x7f474414f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 shutdown_connections 2023-12-26T21:50:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.548+0000 7f474a3ff700 1 -- 172.21.15.131:0/4201679594 wait complete. 2023-12-26T21:50:09.551 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:50:09.566 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:50:08.713348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591938,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":227,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422048,"kb_used_data":6880,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752552672,"statfs":{"total":773094113280,"available":770613936128,"internally_reserved":0,"allocated":7045120,"data_stored":3882822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1570,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.514395"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239194+0000","last_change":"2023-12-26T21:50:05.233540+0000","last_active":"2023-12-26T21:50:05.239194+0000","last_peered":"2023-12-26T21:50:05.239194+0000","last_clean":"2023-12-26T21:50:05.239194+0000","last_became_active":"2023-12-26T21:50:05.233189+0000","last_became_peered":"2023-12-26T21:50:05.233189+0000","last_unstale":"2023-12-26T21:50:05.239194+0000","last_undegraded":"2023-12-26T21:50:05.239194+0000","last_fullsized":"2023-12-26T21:50:05.239194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:38.099699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.231885+0000","last_change":"2023-12-26T21:50:05.226840+0000","last_active":"2023-12-26T21:50:05.231885+0000","last_peered":"2023-12-26T21:50:05.231885+0000","last_clean":"2023-12-26T21:50:05.231885+0000","last_became_active":"2023-12-26T21:50:05.226580+0000","last_became_peered":"2023-12-26T21:50:05.226580+0000","last_unstale":"2023-12-26T21:50:05.231885+0000","last_undegraded":"2023-12-26T21:50:05.231885+0000","last_fullsized":"2023-12-26T21:50:05.231885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:15:04.988046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225805+0000","last_change":"2023-12-26T21:50:05.221852+0000","last_active":"2023-12-26T21:50:05.225805+0000","last_peered":"2023-12-26T21:50:05.225805+0000","last_clean":"2023-12-26T21:50:05.225805+0000","last_became_active":"2023-12-26T21:50:05.221574+0000","last_became_peered":"2023-12-26T21:50:05.221574+0000","last_unstale":"2023-12-26T21:50:05.225805+0000","last_undegraded":"2023-12-26T21:50:05.225805+0000","last_fullsized":"2023-12-26T21:50:05.225805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:45:09.667824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216704+0000","last_change":"2023-12-26T21:50:05.213544+0000","last_active":"2023-12-26T21:50:05.216704+0000","last_peered":"2023-12-26T21:50:05.216704+0000","last_clean":"2023-12-26T21:50:05.216704+0000","last_became_active":"2023-12-26T21:50:05.212622+0000","last_became_peered":"2023-12-26T21:50:05.212622+0000","last_unstale":"2023-12-26T21:50:05.216704+0000","last_undegraded":"2023-12-26T21:50:05.216704+0000","last_fullsized":"2023-12-26T21:50:05.216704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:45.329023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238621+0000","last_change":"2023-12-26T21:50:05.231337+0000","last_active":"2023-12-26T21:50:05.238621+0000","last_peered":"2023-12-26T21:50:05.238621+0000","last_clean":"2023-12-26T21:50:05.238621+0000","last_became_active":"2023-12-26T21:50:05.231031+0000","last_became_peered":"2023-12-26T21:50:05.231031+0000","last_unstale":"2023-12-26T21:50:05.238621+0000","last_undegraded":"2023-12-26T21:50:05.238621+0000","last_fullsized":"2023-12-26T21:50:05.238621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:51:34.322061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239316+0000","last_change":"2023-12-26T21:50:05.233798+0000","last_active":"2023-12-26T21:50:05.239316+0000","last_peered":"2023-12-26T21:50:05.239316+0000","last_clean":"2023-12-26T21:50:05.239316+0000","last_became_active":"2023-12-26T21:50:05.229125+0000","last_became_peered":"2023-12-26T21:50:05.229125+0000","last_unstale":"2023-12-26T21:50:05.239316+0000","last_undegraded":"2023-12-26T21:50:05.239316+0000","last_fullsized":"2023-12-26T21:50:05.239316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:53.114423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234135+0000","last_change":"2023-12-26T21:50:05.224391+0000","last_active":"2023-12-26T21:50:05.234135+0000","last_peered":"2023-12-26T21:50:05.234135+0000","last_clean":"2023-12-26T21:50:05.234135+0000","last_became_active":"2023-12-26T21:50:05.224146+0000","last_became_peered":"2023-12-26T21:50:05.224146+0000","last_unstale":"2023-12-26T21:50:05.234135+0000","last_undegraded":"2023-12-26T21:50:05.234135+0000","last_fullsized":"2023-12-26T21:50:05.234135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:37.819740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227740+0000","last_change":"2023-12-26T21:50:05.225269+0000","last_active":"2023-12-26T21:50:05.227740+0000","last_peered":"2023-12-26T21:50:05.227740+0000","last_clean":"2023-12-26T21:50:05.227740+0000","last_became_active":"2023-12-26T21:50:05.224997+0000","last_became_peered":"2023-12-26T21:50:05.224997+0000","last_unstale":"2023-12-26T21:50:05.227740+0000","last_undegraded":"2023-12-26T21:50:05.227740+0000","last_fullsized":"2023-12-26T21:50:05.227740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:03.075265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202405+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:18:07.070074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244190+0000","last_change":"2023-12-26T21:50:04.223739+0000","last_active":"2023-12-26T21:50:05.244190+0000","last_peered":"2023-12-26T21:50:05.244190+0000","last_clean":"2023-12-26T21:50:05.244190+0000","last_became_active":"2023-12-26T21:50:04.222755+0000","last_became_peered":"2023-12-26T21:50:04.222755+0000","last_unstale":"2023-12-26T21:50:05.244190+0000","last_undegraded":"2023-12-26T21:50:05.244190+0000","last_fullsized":"2023-12-26T21:50:05.244190+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:21.699131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.204232+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:11:47.600963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244999+0000","last_change":"2023-12-26T21:50:04.223118+0000","last_active":"2023-12-26T21:50:05.244999+0000","last_peered":"2023-12-26T21:50:05.244999+0000","last_clean":"2023-12-26T21:50:05.244999+0000","last_became_active":"2023-12-26T21:50:04.222720+0000","last_became_peered":"2023-12-26T21:50:04.222720+0000","last_unstale":"2023-12-26T21:50:05.244999+0000","last_undegraded":"2023-12-26T21:50:05.244999+0000","last_fullsized":"2023-12-26T21:50:05.244999+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:38:24.121296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.222331+0000","last_change":"2023-12-26T21:50:05.220983+0000","last_active":"2023-12-26T21:50:05.222331+0000","last_peered":"2023-12-26T21:50:05.222331+0000","last_clean":"2023-12-26T21:50:05.222331+0000","last_became_active":"2023-12-26T21:50:05.220694+0000","last_became_peered":"2023-12-26T21:50:05.220694+0000","last_unstale":"2023-12-26T21:50:05.222331+0000","last_undegraded":"2023-12-26T21:50:05.222331+0000","last_fullsized":"2023-12-26T21:50:05.222331+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:29:18.312367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.203565+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:29.468613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238883+0000","last_change":"2023-12-26T21:50:05.235706+0000","last_active":"2023-12-26T21:50:05.238883+0000","last_peered":"2023-12-26T21:50:05.238883+0000","last_clean":"2023-12-26T21:50:05.238883+0000","last_became_active":"2023-12-26T21:50:05.235416+0000","last_became_peered":"2023-12-26T21:50:05.235416+0000","last_unstale":"2023-12-26T21:50:05.238883+0000","last_undegraded":"2023-12-26T21:50:05.238883+0000","last_fullsized":"2023-12-26T21:50:05.238883+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:20.767760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248231+0000","last_change":"2023-12-26T21:50:04.225964+0000","last_active":"2023-12-26T21:50:05.248231+0000","last_peered":"2023-12-26T21:50:05.248231+0000","last_clean":"2023-12-26T21:50:05.248231+0000","last_became_active":"2023-12-26T21:50:04.225653+0000","last_became_peered":"2023-12-26T21:50:04.225653+0000","last_unstale":"2023-12-26T21:50:05.248231+0000","last_undegraded":"2023-12-26T21:50:05.248231+0000","last_fullsized":"2023-12-26T21:50:05.248231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:49.834751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.226468+0000","last_change":"2023-12-26T21:50:05.223489+0000","last_active":"2023-12-26T21:50:05.226468+0000","last_peered":"2023-12-26T21:50:05.226468+0000","last_clean":"2023-12-26T21:50:05.226468+0000","last_became_active":"2023-12-26T21:50:05.222774+0000","last_became_peered":"2023-12-26T21:50:05.222774+0000","last_unstale":"2023-12-26T21:50:05.226468+0000","last_undegraded":"2023-12-26T21:50:05.226468+0000","last_fullsized":"2023-12-26T21:50:05.226468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:50.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202039+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:38:21.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239174+0000","last_change":"2023-12-26T21:50:05.233433+0000","last_active":"2023-12-26T21:50:05.239174+0000","last_peered":"2023-12-26T21:50:05.239174+0000","last_clean":"2023-12-26T21:50:05.239174+0000","last_became_active":"2023-12-26T21:50:05.228341+0000","last_became_peered":"2023-12-26T21:50:05.228341+0000","last_unstale":"2023-12-26T21:50:05.239174+0000","last_undegraded":"2023-12-26T21:50:05.239174+0000","last_fullsized":"2023-12-26T21:50:05.239174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:58:54.385865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214676+0000","last_change":"2023-12-26T21:50:04.221595+0000","last_active":"2023-12-26T21:50:05.214676+0000","last_peered":"2023-12-26T21:50:05.214676+0000","last_clean":"2023-12-26T21:50:05.214676+0000","last_became_active":"2023-12-26T21:50:04.221337+0000","last_became_peered":"2023-12-26T21:50:04.221337+0000","last_unstale":"2023-12-26T21:50:05.214676+0000","last_undegraded":"2023-12-26T21:50:05.214676+0000","last_fullsized":"2023-12-26T21:50:05.214676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:38:20.838693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.205068+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:24.364623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214009+0000","last_change":"2023-12-26T21:50:04.229086+0000","last_active":"2023-12-26T21:50:05.214009+0000","last_peered":"2023-12-26T21:50:05.214009+0000","last_clean":"2023-12-26T21:50:05.214009+0000","last_became_active":"2023-12-26T21:50:04.228808+0000","last_became_peered":"2023-12-26T21:50:04.228808+0000","last_unstale":"2023-12-26T21:50:05.214009+0000","last_undegraded":"2023-12-26T21:50:05.214009+0000","last_fullsized":"2023-12-26T21:50:05.214009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:10:52.365673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.199650+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:03:30.049843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238522+0000","last_change":"2023-12-26T21:50:05.233319+0000","last_active":"2023-12-26T21:50:05.238522+0000","last_peered":"2023-12-26T21:50:05.238522+0000","last_clean":"2023-12-26T21:50:05.238522+0000","last_became_active":"2023-12-26T21:50:05.232699+0000","last_became_peered":"2023-12-26T21:50:05.232699+0000","last_unstale":"2023-12-26T21:50:05.238522+0000","last_undegraded":"2023-12-26T21:50:05.238522+0000","last_fullsized":"2023-12-26T21:50:05.238522+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:02:49.364640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250467+0000","last_change":"2023-12-26T21:50:04.227027+0000","last_active":"2023-12-26T21:50:05.250467+0000","last_peered":"2023-12-26T21:50:05.250467+0000","last_clean":"2023-12-26T21:50:05.250467+0000","last_became_active":"2023-12-26T21:50:04.226702+0000","last_became_peered":"2023-12-26T21:50:04.226702+0000","last_unstale":"2023-12-26T21:50:05.250467+0000","last_undegraded":"2023-12-26T21:50:05.250467+0000","last_fullsized":"2023-12-26T21:50:05.250467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:43:44.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245546+0000","last_change":"2023-12-26T21:50:04.222842+0000","last_active":"2023-12-26T21:50:05.245546+0000","last_peered":"2023-12-26T21:50:05.245546+0000","last_clean":"2023-12-26T21:50:05.245546+0000","last_became_active":"2023-12-26T21:50:04.222585+0000","last_became_peered":"2023-12-26T21:50:04.222585+0000","last_unstale":"2023-12-26T21:50:05.245546+0000","last_undegraded":"2023-12-26T21:50:05.245546+0000","last_fullsized":"2023-12-26T21:50:05.245546+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:33:01.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227858+0000","last_change":"2023-12-26T21:50:05.225230+0000","last_active":"2023-12-26T21:50:05.227858+0000","last_peered":"2023-12-26T21:50:05.227858+0000","last_clean":"2023-12-26T21:50:05.227858+0000","last_became_active":"2023-12-26T21:50:05.224946+0000","last_became_peered":"2023-12-26T21:50:05.224946+0000","last_unstale":"2023-12-26T21:50:05.227858+0000","last_undegraded":"2023-12-26T21:50:05.227858+0000","last_fullsized":"2023-12-26T21:50:05.227858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:03:31.057770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202664+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:58.818709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.219854+0000","last_change":"2023-12-26T21:50:05.217771+0000","last_active":"2023-12-26T21:50:05.219854+0000","last_peered":"2023-12-26T21:50:05.219854+0000","last_clean":"2023-12-26T21:50:05.219854+0000","last_became_active":"2023-12-26T21:50:05.217485+0000","last_became_peered":"2023-12-26T21:50:05.217485+0000","last_unstale":"2023-12-26T21:50:05.219854+0000","last_undegraded":"2023-12-26T21:50:05.219854+0000","last_fullsized":"2023-12-26T21:50:05.219854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:54.647443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.207201+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:59:15.144868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250174+0000","last_change":"2023-12-26T21:50:04.224690+0000","last_active":"2023-12-26T21:50:05.250174+0000","last_peered":"2023-12-26T21:50:05.250174+0000","last_clean":"2023-12-26T21:50:05.250174+0000","last_became_active":"2023-12-26T21:50:04.223460+0000","last_became_peered":"2023-12-26T21:50:04.223460+0000","last_unstale":"2023-12-26T21:50:05.250174+0000","last_undegraded":"2023-12-26T21:50:05.250174+0000","last_fullsized":"2023-12-26T21:50:05.250174+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:01.981864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216840+0000","last_change":"2023-12-26T21:50:05.213363+0000","last_active":"2023-12-26T21:50:05.216840+0000","last_peered":"2023-12-26T21:50:05.216840+0000","last_clean":"2023-12-26T21:50:05.216840+0000","last_became_active":"2023-12-26T21:50:05.212965+0000","last_became_peered":"2023-12-26T21:50:05.212965+0000","last_unstale":"2023-12-26T21:50:05.216840+0000","last_undegraded":"2023-12-26T21:50:05.216840+0000","last_fullsized":"2023-12-26T21:50:05.216840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:01:27.770799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202373+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:23.901091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.223902+0000","last_change":"2023-12-26T21:50:05.222959+0000","last_active":"2023-12-26T21:50:05.223902+0000","last_peered":"2023-12-26T21:50:05.223902+0000","last_clean":"2023-12-26T21:50:05.223902+0000","last_became_active":"2023-12-26T21:50:05.222618+0000","last_became_peered":"2023-12-26T21:50:05.222618+0000","last_unstale":"2023-12-26T21:50:05.223902+0000","last_undegraded":"2023-12-26T21:50:05.223902+0000","last_fullsized":"2023-12-26T21:50:05.223902+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:13.568346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.205291+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:37.290110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.204851+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:58:34.233954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.203022+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:37:47.067537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225878+0000","last_change":"2023-12-26T21:50:05.224746+0000","last_active":"2023-12-26T21:50:05.225878+0000","last_peered":"2023-12-26T21:50:05.225878+0000","last_clean":"2023-12-26T21:50:05.225878+0000","last_became_active":"2023-12-26T21:50:05.224302+0000","last_became_peered":"2023-12-26T21:50:05.224302+0000","last_unstale":"2023-12-26T21:50:05.225878+0000","last_undegraded":"2023-12-26T21:50:05.225878+0000","last_fullsized":"2023-12-26T21:50:05.225878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:44.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248205+0000","last_change":"2023-12-26T21:50:04.226030+0000","last_active":"2023-12-26T21:50:05.248205+0000","last_peered":"2023-12-26T21:50:05.248205+0000","last_clean":"2023-12-26T21:50:05.248205+0000","last_became_active":"2023-12-26T21:50:04.225766+0000","last_became_peered":"2023-12-26T21:50:04.225766+0000","last_unstale":"2023-12-26T21:50:05.248205+0000","last_undegraded":"2023-12-26T21:50:05.248205+0000","last_fullsized":"2023-12-26T21:50:05.248205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:58.136823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202075+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:55:08.870277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216148+0000","last_change":"2023-12-26T21:50:04.225794+0000","last_active":"2023-12-26T21:50:05.216148+0000","last_peered":"2023-12-26T21:50:05.216148+0000","last_clean":"2023-12-26T21:50:05.216148+0000","last_became_active":"2023-12-26T21:50:04.225535+0000","last_became_peered":"2023-12-26T21:50:04.225535+0000","last_unstale":"2023-12-26T21:50:05.216148+0000","last_undegraded":"2023-12-26T21:50:05.216148+0000","last_fullsized":"2023-12-26T21:50:05.216148+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:01:54.466108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.213208+0000","last_change":"2023-12-26T21:50:04.224397+0000","last_active":"2023-12-26T21:50:05.213208+0000","last_peered":"2023-12-26T21:50:05.213208+0000","last_clean":"2023-12-26T21:50:05.213208+0000","last_became_active":"2023-12-26T21:50:04.224078+0000","last_became_peered":"2023-12-26T21:50:04.224078+0000","last_unstale":"2023-12-26T21:50:05.213208+0000","last_undegraded":"2023-12-26T21:50:05.213208+0000","last_fullsized":"2023-12-26T21:50:05.213208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:28.511389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234103+0000","last_change":"2023-12-26T21:50:05.227159+0000","last_active":"2023-12-26T21:50:05.234103+0000","last_peered":"2023-12-26T21:50:05.234103+0000","last_clean":"2023-12-26T21:50:05.234103+0000","last_became_active":"2023-12-26T21:50:05.226887+0000","last_became_peered":"2023-12-26T21:50:05.226887+0000","last_unstale":"2023-12-26T21:50:05.234103+0000","last_undegraded":"2023-12-26T21:50:05.234103+0000","last_fullsized":"2023-12-26T21:50:05.234103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:01:34.210345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202292+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:49:20.718326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225824+0000","last_change":"2023-12-26T21:50:05.223267+0000","last_active":"2023-12-26T21:50:05.225824+0000","last_peered":"2023-12-26T21:50:05.225824+0000","last_clean":"2023-12-26T21:50:05.225824+0000","last_became_active":"2023-12-26T21:50:05.222964+0000","last_became_peered":"2023-12-26T21:50:05.222964+0000","last_unstale":"2023-12-26T21:50:05.225824+0000","last_undegraded":"2023-12-26T21:50:05.225824+0000","last_fullsized":"2023-12-26T21:50:05.225824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:42:16.214790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202129+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:39:05.981705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239297+0000","last_change":"2023-12-26T21:50:05.233495+0000","last_active":"2023-12-26T21:50:05.239297+0000","last_peered":"2023-12-26T21:50:05.239297+0000","last_clean":"2023-12-26T21:50:05.239297+0000","last_became_active":"2023-12-26T21:50:05.233051+0000","last_became_peered":"2023-12-26T21:50:05.233051+0000","last_unstale":"2023-12-26T21:50:05.239297+0000","last_undegraded":"2023-12-26T21:50:05.239297+0000","last_fullsized":"2023-12-26T21:50:05.239297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:49:53.896795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208240+0000","last_change":"2023-12-26T21:50:04.218089+0000","last_active":"2023-12-26T21:50:05.208240+0000","last_peered":"2023-12-26T21:50:05.208240+0000","last_clean":"2023-12-26T21:50:05.208240+0000","last_became_active":"2023-12-26T21:50:04.215530+0000","last_became_peered":"2023-12-26T21:50:04.215530+0000","last_unstale":"2023-12-26T21:50:05.208240+0000","last_undegraded":"2023-12-26T21:50:05.208240+0000","last_fullsized":"2023-12-26T21:50:05.208240+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:21.909145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225858+0000","last_change":"2023-12-26T21:50:05.224578+0000","last_active":"2023-12-26T21:50:05.225858+0000","last_peered":"2023-12-26T21:50:05.225858+0000","last_clean":"2023-12-26T21:50:05.225858+0000","last_became_active":"2023-12-26T21:50:05.223985+0000","last_became_peered":"2023-12-26T21:50:05.223985+0000","last_unstale":"2023-12-26T21:50:05.225858+0000","last_undegraded":"2023-12-26T21:50:05.225858+0000","last_fullsized":"2023-12-26T21:50:05.225858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:47:51.264287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.201997+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:05.549244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212760+0000","last_change":"2023-12-26T21:50:04.222797+0000","last_active":"2023-12-26T21:50:05.212760+0000","last_peered":"2023-12-26T21:50:05.212760+0000","last_clean":"2023-12-26T21:50:05.212760+0000","last_became_active":"2023-12-26T21:50:04.222490+0000","last_became_peered":"2023-12-26T21:50:04.222490+0000","last_unstale":"2023-12-26T21:50:05.212760+0000","last_undegraded":"2023-12-26T21:50:05.212760+0000","last_fullsized":"2023-12-26T21:50:05.212760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:17.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202374+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:24:45.381424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208086+0000","last_change":"2023-12-26T21:50:04.220953+0000","last_active":"2023-12-26T21:50:05.208086+0000","last_peered":"2023-12-26T21:50:05.208086+0000","last_clean":"2023-12-26T21:50:05.208086+0000","last_became_active":"2023-12-26T21:50:04.220694+0000","last_became_peered":"2023-12-26T21:50:04.220694+0000","last_unstale":"2023-12-26T21:50:05.208086+0000","last_undegraded":"2023-12-26T21:50:05.208086+0000","last_fullsized":"2023-12-26T21:50:05.208086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:05.018373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245342+0000","last_change":"2023-12-26T21:50:04.226718+0000","last_active":"2023-12-26T21:50:05.245342+0000","last_peered":"2023-12-26T21:50:05.245342+0000","last_clean":"2023-12-26T21:50:05.245342+0000","last_became_active":"2023-12-26T21:50:04.226415+0000","last_became_peered":"2023-12-26T21:50:04.226415+0000","last_unstale":"2023-12-26T21:50:05.245342+0000","last_undegraded":"2023-12-26T21:50:05.245342+0000","last_fullsized":"2023-12-26T21:50:05.245342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:29.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"53'139","reported_seq":275,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-26T21:50:00.637985+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:50:00.637985+0000","last_peered":"2023-12-26T21:50:00.637985+0000","last_clean":"2023-12-26T21:50:00.637985+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:50:00.637985+0000","last_undegraded":"2023-12-26T21:50:00.637985+0000","last_fullsized":"2023-12-26T21:50:00.637985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1570,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542308,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":351691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.89100000000000001,"5min":0.94099999999999995,"15min":1.647},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":1,"5min":1,"15min":2.0150000000000001},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72599999999999998,"15min":0.70699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.016,"5min":1.048,"15min":1.1359999999999999},"last":0.436}]},{"osd":2,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.8640000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":0.95599999999999996,"15min":1.419},"last":0.63200000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":0.77200000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.026,"15min":1.327},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91900000000000004,"5min":1.008,"15min":1.393},"last":0.60699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":0.89400000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.371},"max":{"1min":0.84399999999999997,"5min":0.85199999999999998,"15min":0.86799999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":0.97599999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75700000000000001,"15min":0.745},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.499},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79100000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":0.92600000000000005,"15min":0.999},"last":0.68799999999999994}]}]},{"osd":6,"up_from":15,"seq":64424509603,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302072,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069768,"statfs":{"total":96636764160,"available":96327442432,"internally_reserved":0,"allocated":704512,"data_stored":325082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":1.0009999999999999,"15min":1.085},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.748,"15min":0.72199999999999998},"min":{"1min":0.56399999999999995,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":1.3380000000000001},"last":0.57299999999999995}]},{"osd":1,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.60299999999999998,"5min":0.44,"15min":0.41499999999999998},"max":{"1min":0.94699999999999995,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.71999999999999997},"min":{"1min":0.52300000000000002,"5min":0.41499999999999998,"15min":0.40799999999999997},"max":{"1min":0.86599999999999999,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.5}]},{"osd":2,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73599999999999999,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.46000000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.97099999999999997,"15min":1.288},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72699999999999998,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.497,"15min":0.45600000000000002},"max":{"1min":0.92000000000000004,"5min":0.93200000000000005,"15min":1.113},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.745,"15min":0.73399999999999999},"min":{"1min":0.54700000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":0.93600000000000005,"15min":1.569},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.77000000000000002,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.50600000000000001},"max":{"1min":0.96999999999999997,"5min":1.0129999999999999,"15min":1.3839999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.63300000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.377},"max":{"1min":0.81299999999999994,"5min":0.92700000000000005,"15min":1.137},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.65700000000000003},"min":{"1min":0.503,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93200000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.64500000000000002},"min":{"1min":0.42099999999999999,"5min":0.41899999999999998,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.875,"15min":1.1200000000000001},"last":0.754},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67000000000000004,"15min":0.70099999999999996},"min":{"1min":0.54800000000000004,"5min":0.46200000000000002,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":0.93600000000000005,"15min":1.296},"last":0.76700000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69399999999999995,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.437},"max":{"1min":0.89500000000000002,"5min":1.0920000000000001,"15min":1.51},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.998,"15min":1.2010000000000001},"last":0.79900000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509603,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306760,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065080,"statfs":{"total":96636764160,"available":96322641920,"internally_reserved":0,"allocated":1310720,"data_stored":921619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.309,"5min":0.307,"15min":0.255},"max":{"1min":0.65900000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.501,"15min":0.52900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.754,"15min":4.6600000000000001},"last":0.51600000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.49199999999999999,"15min":0.497},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.59699999999999998,"5min":1.0189999999999999,"15min":2.2869999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55400000000000005,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.29699999999999999},"max":{"1min":0.78800000000000003,"5min":0.93200000000000005,"15min":4.3639999999999999},"last":0.501}]},{"osd":3,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.38500000000000001},"max":{"1min":0.871,"5min":0.998,"15min":4.3259999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.33300000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":4.548},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92100000000000004,"5min":1.252,"15min":1.252},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.754,"15min":0.747},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.441},"max":{"1min":0.90200000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.51300000000000001},"max":{"1min":0.93700000000000006,"5min":1.2949999999999999,"15min":1.345},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.81799999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.82199999999999995,"15min":0.80100000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.51400000000000001},"max":{"1min":0.89500000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.83399999999999996,"15min":0.81100000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":0.90700000000000003,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0149999999999999,"5min":1.0980000000000001,"15min":1.1539999999999999},"last":0.746},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.94399999999999995,"5min":1.3620000000000001,"15min":1.3919999999999999},"last":0.55700000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509603,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302072,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069768,"statfs":{"total":96636764160,"available":96327442432,"internally_reserved":0,"allocated":704512,"data_stored":325082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42799999999999999,"15min":0.442},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":1.5640000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.48499999999999999,"15min":0.51200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60599999999999998,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":0.68300000000000005,"15min":2.012},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.51400000000000001,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56499999999999995},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.84499999999999997,"15min":4.2610000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.746,"15min":0.78000000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.85399999999999998,"5min":1.1539999999999999,"15min":4.423},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.93500000000000005,"5min":1.1000000000000001,"15min":1.6439999999999999},"last":0.622}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.78600000000000003,"15min":0.82099999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":1.119,"15min":4.3650000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.94599999999999995,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":1.103,"15min":1.611},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79300000000000004,"15min":0.81999999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.5},"max":{"1min":1.002,"5min":1.111,"15min":4.407},"last":0.83499999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.82099999999999995},"min":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":0.89600000000000002,"5min":1.145,"15min":4.5069999999999997},"last":0.69199999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302084,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069756,"statfs":{"total":96636764160,"available":96327430144,"internally_reserved":0,"allocated":716800,"data_stored":331251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.32900000000000001,"5min":0.27000000000000002,"15min":0.248},"max":{"1min":0.67000000000000004,"5min":0.85999999999999999,"15min":1.915},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":1.7729999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48499999999999999,"15min":0.496},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70899999999999996,"15min":1.952},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.30199999999999999},"max":{"1min":0.63100000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.44800000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.438,"5min":0.437,"15min":0.375},"max":{"1min":0.78700000000000003,"5min":0.83999999999999997,"15min":0.97099999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.98299999999999998,"5min":1.1180000000000001,"15min":1.284},"last":0.753},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.98399999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.92200000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.75800000000000001},"min":{"1min":0.53900000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.5029999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78000000000000003,"15min":0.77200000000000002},"min":{"1min":0.621,"5min":0.54200000000000004,"15min":0.503},"max":{"1min":0.89900000000000002,"5min":1.028,"15min":1.355},"last":0.88100000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.91200000000000003,"5min":1.032,"15min":1.7350000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80200000000000005,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.90200000000000002,"5min":1.0640000000000001,"15min":1.349},"last":0.873}]},{"osd":7,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.47799999999999998},"max":{"1min":0.95999999999999996,"5min":1.0680000000000001,"15min":1.841},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":1.004,"5min":1.091,"15min":1.294},"last":0.81999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509605,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306228,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065612,"statfs":{"total":96636764160,"available":96323186688,"internally_reserved":0,"allocated":765952,"data_stored":353193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.438,"15min":0.442},"min":{"1min":0.312,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.627,"5min":0.68100000000000005,"15min":0.91400000000000003},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35699999999999998,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.30399999999999999},"max":{"1min":0.65900000000000003,"5min":0.73499999999999999,"15min":0.77200000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.32400000000000001},"max":{"1min":0.66800000000000004,"5min":0.72499999999999998,"15min":0.99399999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38900000000000001},"max":{"1min":0.72099999999999997,"5min":0.83699999999999997,"15min":1.0820000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.82599999999999996,"15min":1.3580000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.499,"15min":0.45100000000000001},"max":{"1min":1.018,"5min":1.105,"15min":1.3779999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":1.1870000000000001,"15min":1.2949999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.47699999999999998},"max":{"1min":1,"5min":1.27,"15min":1.4610000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.78400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.45900000000000002},"max":{"1min":0.96399999999999997,"5min":0.997,"15min":1.9239999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.1950000000000001,"15min":1.5049999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.54100000000000004},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.579},"last":0.87}]},{"osd":7,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78600000000000003,"15min":0.80200000000000005},"min":{"1min":0.63800000000000001,"5min":0.51100000000000001,"15min":0.49299999999999999},"max":{"1min":1.036,"5min":1.0640000000000001,"15min":1.964},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.1499999999999999,"15min":1.6359999999999999},"last":0.92400000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069700,"statfs":{"total":96636764160,"available":96327372800,"internally_reserved":0,"allocated":774144,"data_stored":353285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":0.96499999999999997,"15min":2.1389999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.008,"15min":1.407},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.878,"5min":0.90500000000000003,"15min":1.879},"last":0.443},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.70799999999999996,"15min":0.72199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":2.097},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.72399999999999998},"min":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":0.94399999999999995,"15min":1.7809999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":1.6870000000000001},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70799999999999996,"15min":0.72599999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.83899999999999997,"5min":0.95999999999999996,"15min":1.956},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72899999999999998,"15min":0.74399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.83199999999999996,"5min":0.97999999999999998,"15min":2.0609999999999999},"last":0.69399999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58799999999999997,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.89800000000000002,"15min":2.2429999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.64200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77000000000000002,"5min":0.92600000000000005,"15min":2.194},"last":0.81799999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60199999999999998,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.878,"15min":2.1589999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64700000000000002,"15min":0.70299999999999996},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.874,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":0.98099999999999998,"15min":2.23},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.57199999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90100000000000002,"5min":0.92700000000000005,"15min":2.5169999999999999},"last":0.79900000000000004}]}]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1310720,"data_stored":921619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":0.995,"15min":2.3900000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70799999999999996,"15min":0.73599999999999999},"min":{"1min":0.499,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.874,"5min":1.0369999999999999,"15min":1.71},"last":0.81200000000000006}]},{"osd":1,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.51500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.61499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.4219999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72399999999999998,"15min":0.746},"min":{"1min":0.47899999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.622,"5min":0.435,"15min":0.435},"max":{"1min":0.81999999999999995,"5min":0.95699999999999996,"15min":1.214},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.74299999999999999},"min":{"1min":0.56399999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.4890000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.874,"5min":0.93899999999999995,"15min":1.1439999999999999},"last":0.82399999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":2.5070000000000001},"last":0.755},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59099999999999997,"15min":0.65600000000000003},"min":{"1min":0.55400000000000005,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90100000000000002,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.73099999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64300000000000002,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"max":{"1min":0.84299999999999997,"5min":0.90700000000000003,"15min":1.637},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.76500000000000001},"min":{"1min":0.65200000000000002,"5min":0.49099999999999999,"15min":0.438},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":2.5579999999999998},"last":0.85299999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67400000000000004,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.83299999999999996,"5min":0.96299999999999997,"15min":1.248},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68300000000000005,"15min":0.73299999999999998},"min":{"1min":0.56899999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83499999999999996,"5min":0.97199999999999998,"15min":1.5980000000000001},"last":0.88}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:50:09.572 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:50:09.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.795+0000 7ffa0c1ff700 1 Processor -- start 2023-12-26T21:50:09.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.795+0000 7ffa0c1ff700 1 -- start start 2023-12-26T21:50:09.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.796+0000 7ffa0c1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa0406bfd0 0x7ffa040c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.797+0000 7ffa0c1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa040cc100 0x7ffa040cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.797+0000 7ffa0c1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa040686c0 con 0x7ffa0413c510 2023-12-26T21:50:09.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.797+0000 7ffa0c1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa04069520 con 0x7ffa0406bfd0 2023-12-26T21:50:09.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.797+0000 7ffa0c1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa04068910 con 0x7ffa040cc100 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.797+0000 7ffa0a79c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa040cc100 0x7ffa040cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.797+0000 7ffa0979a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa0406bfd0 0x7ffa040c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.797+0000 7ffa09f9b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ffa0413c510 0x7ffa0406b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:40148/0 (socket says 172.21.15.131:40148) 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.797+0000 7ffa09f9b700 1 -- 172.21.15.131:0/3587487870 learned_addr learned my addr 172.21.15.131:0/3587487870 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.797+0000 7ffa0a79c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa040cc100 0x7ffa040cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:47608/0 (socket says 172.21.15.131:47608) 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa08f99700 1 -- 172.21.15.131:0/3587487870 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317676318 0 0) 0x7ffa040686c0 con 0x7ffa0413c510 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa08f99700 1 -- 172.21.15.131:0/3587487870 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9f0003570 con 0x7ffa0413c510 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa0979a700 1 -- 172.21.15.131:0/3587487870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa040cc100 msgr2=0x7ffa040cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa0979a700 1 --2- 172.21.15.131:0/3587487870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa040cc100 0x7ffa040cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa0979a700 1 -- 172.21.15.131:0/3587487870 >> v1:172.21.15.190:6789/0 conn(0x7ffa0413c510 legacy=0x7ffa0406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa0979a700 1 -- 172.21.15.131:0/3587487870 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04068bf0 con 0x7ffa0406bfd0 2023-12-26T21:50:09.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa0a79c700 1 --2- 172.21.15.131:0/3587487870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa040cc100 0x7ffa040cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:50:09.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa0979a700 1 --2- 172.21.15.131:0/3587487870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0406bfd0 0x7ffa040c8a20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40098e0 tx=0x7ff9f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6528efd930c69b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:09.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa08f99700 1 -- 172.21.15.131:0/3587487870 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4011630 con 0x7ffa0406bfd0 2023-12-26T21:50:09.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa08f99700 1 -- 172.21.15.131:0/3587487870 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f4011790 con 0x7ffa0406bfd0 2023-12-26T21:50:09.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.798+0000 7ffa08f99700 1 -- 172.21.15.131:0/3587487870 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4011b10 con 0x7ffa0406bfd0 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.799+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/3587487870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0406bfd0 msgr2=0x7ffa040c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.799+0000 7ffa0c1ff700 1 --2- 172.21.15.131:0/3587487870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0406bfd0 0x7ffa040c8a20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40098e0 tx=0x7ff9f40091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.799+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/3587487870 shutdown_connections 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.799+0000 7ffa0c1ff700 1 --2- 172.21.15.131:0/3587487870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa040cc100 0x7ffa040cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.799+0000 7ffa0c1ff700 1 --2- 172.21.15.131:0/3587487870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0406bfd0 0x7ffa040c8a20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.799+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/3587487870 >> 172.21.15.131:0/3587487870 conn(0x7ffa040ce3e0 msgr2=0x7ffa040ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.799+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/3587487870 shutdown_connections 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.799+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/3587487870 wait complete. 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.800+0000 7ffa0c1ff700 1 Processor -- start 2023-12-26T21:50:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.801+0000 7ffa0c1ff700 1 -- start start 2023-12-26T21:50:09.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.801+0000 7ffa0c1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0406bfd0 0x7ffa04155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0c1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa040cc100 0x7ffa04156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0c1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0413c510 0x7ffa040b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0c1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa04154570 con 0x7ffa040cc100 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0c1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa04154850 con 0x7ffa0413c510 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0c1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa04154b30 con 0x7ffa0406bfd0 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0979a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa040cc100 0x7ffa04156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0979a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa040cc100 0x7ffa04156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49832/0 (socket says 172.21.15.131:49832) 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0979a700 1 -- 172.21.15.131:0/1321362825 learned_addr learned my addr 172.21.15.131:0/1321362825 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0a79c700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0413c510 0x7ffa040b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa09f9b700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0406bfd0 0x7ffa04155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0979a700 1 -- 172.21.15.131:0/1321362825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0406bfd0 msgr2=0x7ffa04155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0979a700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0406bfd0 0x7ffa04155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0979a700 1 -- 172.21.15.131:0/1321362825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0413c510 msgr2=0x7ffa040b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0979a700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0413c510 0x7ffa040b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:09.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.802+0000 7ffa0979a700 1 -- 172.21.15.131:0/1321362825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc009a20 con 0x7ffa040cc100 2023-12-26T21:50:09.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.803+0000 7ffa0979a700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa040cc100 0x7ffa04156700 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4013000 tx=0x7ff9f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:09.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.803+0000 7ff9f9ffb700 1 -- 172.21.15.131:0/1321362825 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4011b10 con 0x7ffa040cc100 2023-12-26T21:50:09.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.803+0000 7ff9f9ffb700 1 -- 172.21.15.131:0/1321362825 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f4000c90 con 0x7ffa040cc100 2023-12-26T21:50:09.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.803+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f4009420 con 0x7ffa040cc100 2023-12-26T21:50:09.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.803+0000 7ff9f9ffb700 1 -- 172.21.15.131:0/1321362825 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f401a660 con 0x7ffa040cc100 2023-12-26T21:50:09.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.803+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa041550f0 con 0x7ffa040cc100 2023-12-26T21:50:09.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.804+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa04155310 con 0x7ffa040cc100 2023-12-26T21:50:09.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.805+0000 7ff9f9ffb700 1 -- 172.21.15.131:0/1321362825 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff9f401aa50 con 0x7ffa040cc100 2023-12-26T21:50:09.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.806+0000 7ff9f9ffb700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff9e8066f30 0x7ff9e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:09.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.806+0000 7ff9f9ffb700 1 -- 172.21.15.131:0/1321362825 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff9f4086350 con 0x7ffa040cc100 2023-12-26T21:50:09.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.806+0000 7ffa09f9b700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff9e8066f30 0x7ff9e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:09.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.807+0000 7ffa09f9b700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff9e8066f30 0x7ff9e80693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa00000c00 tx=0x7ffa00009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:09.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.810+0000 7ff9f9ffb700 1 -- 172.21.15.131:0/1321362825 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9f4053470 con 0x7ffa040cc100 2023-12-26T21:50:10.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:09.999+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa040657e0 con 0x7ff9e8066f30 2023-12-26T21:50:10.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.007+0000 7ff9f9ffb700 1 -- 172.21.15.131:0/1321362825 <== mgr.5437 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282183 (secure 0 0 0) 0x7ffa040657e0 con 0x7ff9e8066f30 2023-12-26T21:50:10.011 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.011+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff9e8066f30 msgr2=0x7ff9e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.011+0000 7ffa0c1ff700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff9e8066f30 0x7ff9e80693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa00000c00 tx=0x7ffa00009040 comp rx=0 tx=0).stop 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.011+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa040cc100 msgr2=0x7ffa04156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.011+0000 7ffa0c1ff700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa040cc100 0x7ffa04156700 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4013000 tx=0x7ff9f4017000 comp rx=0 tx=0).stop 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.012+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 shutdown_connections 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.012+0000 7ffa0c1ff700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0413c510 0x7ffa040b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.012+0000 7ffa0c1ff700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff9e8066f30 0x7ff9e80693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.012+0000 7ffa0c1ff700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0406bfd0 0x7ffa04155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.012+0000 7ffa0c1ff700 1 --2- 172.21.15.131:0/1321362825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa040cc100 0x7ffa04156700 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.012+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 >> 172.21.15.131:0/1321362825 conn(0x7ffa040ce3e0 msgr2=0x7ffa04153720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.012+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 shutdown_connections 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.012+0000 7ffa0c1ff700 1 -- 172.21.15.131:0/1321362825 wait complete. 2023-12-26T21:50:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:50:10.030 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:50:08.713348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591938,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":227,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422048,"kb_used_data":6880,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752552672,"statfs":{"total":773094113280,"available":770613936128,"internally_reserved":0,"allocated":7045120,"data_stored":3882822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1570,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.514395"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239194+0000","last_change":"2023-12-26T21:50:05.233540+0000","last_active":"2023-12-26T21:50:05.239194+0000","last_peered":"2023-12-26T21:50:05.239194+0000","last_clean":"2023-12-26T21:50:05.239194+0000","last_became_active":"2023-12-26T21:50:05.233189+0000","last_became_peered":"2023-12-26T21:50:05.233189+0000","last_unstale":"2023-12-26T21:50:05.239194+0000","last_undegraded":"2023-12-26T21:50:05.239194+0000","last_fullsized":"2023-12-26T21:50:05.239194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:38.099699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.231885+0000","last_change":"2023-12-26T21:50:05.226840+0000","last_active":"2023-12-26T21:50:05.231885+0000","last_peered":"2023-12-26T21:50:05.231885+0000","last_clean":"2023-12-26T21:50:05.231885+0000","last_became_active":"2023-12-26T21:50:05.226580+0000","last_became_peered":"2023-12-26T21:50:05.226580+0000","last_unstale":"2023-12-26T21:50:05.231885+0000","last_undegraded":"2023-12-26T21:50:05.231885+0000","last_fullsized":"2023-12-26T21:50:05.231885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:15:04.988046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225805+0000","last_change":"2023-12-26T21:50:05.221852+0000","last_active":"2023-12-26T21:50:05.225805+0000","last_peered":"2023-12-26T21:50:05.225805+0000","last_clean":"2023-12-26T21:50:05.225805+0000","last_became_active":"2023-12-26T21:50:05.221574+0000","last_became_peered":"2023-12-26T21:50:05.221574+0000","last_unstale":"2023-12-26T21:50:05.225805+0000","last_undegraded":"2023-12-26T21:50:05.225805+0000","last_fullsized":"2023-12-26T21:50:05.225805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:45:09.667824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216704+0000","last_change":"2023-12-26T21:50:05.213544+0000","last_active":"2023-12-26T21:50:05.216704+0000","last_peered":"2023-12-26T21:50:05.216704+0000","last_clean":"2023-12-26T21:50:05.216704+0000","last_became_active":"2023-12-26T21:50:05.212622+0000","last_became_peered":"2023-12-26T21:50:05.212622+0000","last_unstale":"2023-12-26T21:50:05.216704+0000","last_undegraded":"2023-12-26T21:50:05.216704+0000","last_fullsized":"2023-12-26T21:50:05.216704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:45.329023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238621+0000","last_change":"2023-12-26T21:50:05.231337+0000","last_active":"2023-12-26T21:50:05.238621+0000","last_peered":"2023-12-26T21:50:05.238621+0000","last_clean":"2023-12-26T21:50:05.238621+0000","last_became_active":"2023-12-26T21:50:05.231031+0000","last_became_peered":"2023-12-26T21:50:05.231031+0000","last_unstale":"2023-12-26T21:50:05.238621+0000","last_undegraded":"2023-12-26T21:50:05.238621+0000","last_fullsized":"2023-12-26T21:50:05.238621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:51:34.322061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239316+0000","last_change":"2023-12-26T21:50:05.233798+0000","last_active":"2023-12-26T21:50:05.239316+0000","last_peered":"2023-12-26T21:50:05.239316+0000","last_clean":"2023-12-26T21:50:05.239316+0000","last_became_active":"2023-12-26T21:50:05.229125+0000","last_became_peered":"2023-12-26T21:50:05.229125+0000","last_unstale":"2023-12-26T21:50:05.239316+0000","last_undegraded":"2023-12-26T21:50:05.239316+0000","last_fullsized":"2023-12-26T21:50:05.239316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:53.114423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234135+0000","last_change":"2023-12-26T21:50:05.224391+0000","last_active":"2023-12-26T21:50:05.234135+0000","last_peered":"2023-12-26T21:50:05.234135+0000","last_clean":"2023-12-26T21:50:05.234135+0000","last_became_active":"2023-12-26T21:50:05.224146+0000","last_became_peered":"2023-12-26T21:50:05.224146+0000","last_unstale":"2023-12-26T21:50:05.234135+0000","last_undegraded":"2023-12-26T21:50:05.234135+0000","last_fullsized":"2023-12-26T21:50:05.234135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:37.819740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227740+0000","last_change":"2023-12-26T21:50:05.225269+0000","last_active":"2023-12-26T21:50:05.227740+0000","last_peered":"2023-12-26T21:50:05.227740+0000","last_clean":"2023-12-26T21:50:05.227740+0000","last_became_active":"2023-12-26T21:50:05.224997+0000","last_became_peered":"2023-12-26T21:50:05.224997+0000","last_unstale":"2023-12-26T21:50:05.227740+0000","last_undegraded":"2023-12-26T21:50:05.227740+0000","last_fullsized":"2023-12-26T21:50:05.227740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:03.075265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202405+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:18:07.070074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244190+0000","last_change":"2023-12-26T21:50:04.223739+0000","last_active":"2023-12-26T21:50:05.244190+0000","last_peered":"2023-12-26T21:50:05.244190+0000","last_clean":"2023-12-26T21:50:05.244190+0000","last_became_active":"2023-12-26T21:50:04.222755+0000","last_became_peered":"2023-12-26T21:50:04.222755+0000","last_unstale":"2023-12-26T21:50:05.244190+0000","last_undegraded":"2023-12-26T21:50:05.244190+0000","last_fullsized":"2023-12-26T21:50:05.244190+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:21.699131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.204232+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:11:47.600963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244999+0000","last_change":"2023-12-26T21:50:04.223118+0000","last_active":"2023-12-26T21:50:05.244999+0000","last_peered":"2023-12-26T21:50:05.244999+0000","last_clean":"2023-12-26T21:50:05.244999+0000","last_became_active":"2023-12-26T21:50:04.222720+0000","last_became_peered":"2023-12-26T21:50:04.222720+0000","last_unstale":"2023-12-26T21:50:05.244999+0000","last_undegraded":"2023-12-26T21:50:05.244999+0000","last_fullsized":"2023-12-26T21:50:05.244999+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:38:24.121296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.222331+0000","last_change":"2023-12-26T21:50:05.220983+0000","last_active":"2023-12-26T21:50:05.222331+0000","last_peered":"2023-12-26T21:50:05.222331+0000","last_clean":"2023-12-26T21:50:05.222331+0000","last_became_active":"2023-12-26T21:50:05.220694+0000","last_became_peered":"2023-12-26T21:50:05.220694+0000","last_unstale":"2023-12-26T21:50:05.222331+0000","last_undegraded":"2023-12-26T21:50:05.222331+0000","last_fullsized":"2023-12-26T21:50:05.222331+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:29:18.312367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.203565+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:29.468613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238883+0000","last_change":"2023-12-26T21:50:05.235706+0000","last_active":"2023-12-26T21:50:05.238883+0000","last_peered":"2023-12-26T21:50:05.238883+0000","last_clean":"2023-12-26T21:50:05.238883+0000","last_became_active":"2023-12-26T21:50:05.235416+0000","last_became_peered":"2023-12-26T21:50:05.235416+0000","last_unstale":"2023-12-26T21:50:05.238883+0000","last_undegraded":"2023-12-26T21:50:05.238883+0000","last_fullsized":"2023-12-26T21:50:05.238883+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:20.767760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248231+0000","last_change":"2023-12-26T21:50:04.225964+0000","last_active":"2023-12-26T21:50:05.248231+0000","last_peered":"2023-12-26T21:50:05.248231+0000","last_clean":"2023-12-26T21:50:05.248231+0000","last_became_active":"2023-12-26T21:50:04.225653+0000","last_became_peered":"2023-12-26T21:50:04.225653+0000","last_unstale":"2023-12-26T21:50:05.248231+0000","last_undegraded":"2023-12-26T21:50:05.248231+0000","last_fullsized":"2023-12-26T21:50:05.248231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:49.834751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.226468+0000","last_change":"2023-12-26T21:50:05.223489+0000","last_active":"2023-12-26T21:50:05.226468+0000","last_peered":"2023-12-26T21:50:05.226468+0000","last_clean":"2023-12-26T21:50:05.226468+0000","last_became_active":"2023-12-26T21:50:05.222774+0000","last_became_peered":"2023-12-26T21:50:05.222774+0000","last_unstale":"2023-12-26T21:50:05.226468+0000","last_undegraded":"2023-12-26T21:50:05.226468+0000","last_fullsized":"2023-12-26T21:50:05.226468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:50.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202039+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:38:21.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239174+0000","last_change":"2023-12-26T21:50:05.233433+0000","last_active":"2023-12-26T21:50:05.239174+0000","last_peered":"2023-12-26T21:50:05.239174+0000","last_clean":"2023-12-26T21:50:05.239174+0000","last_became_active":"2023-12-26T21:50:05.228341+0000","last_became_peered":"2023-12-26T21:50:05.228341+0000","last_unstale":"2023-12-26T21:50:05.239174+0000","last_undegraded":"2023-12-26T21:50:05.239174+0000","last_fullsized":"2023-12-26T21:50:05.239174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:58:54.385865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214676+0000","last_change":"2023-12-26T21:50:04.221595+0000","last_active":"2023-12-26T21:50:05.214676+0000","last_peered":"2023-12-26T21:50:05.214676+0000","last_clean":"2023-12-26T21:50:05.214676+0000","last_became_active":"2023-12-26T21:50:04.221337+0000","last_became_peered":"2023-12-26T21:50:04.221337+0000","last_unstale":"2023-12-26T21:50:05.214676+0000","last_undegraded":"2023-12-26T21:50:05.214676+0000","last_fullsized":"2023-12-26T21:50:05.214676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:38:20.838693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.205068+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:24.364623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214009+0000","last_change":"2023-12-26T21:50:04.229086+0000","last_active":"2023-12-26T21:50:05.214009+0000","last_peered":"2023-12-26T21:50:05.214009+0000","last_clean":"2023-12-26T21:50:05.214009+0000","last_became_active":"2023-12-26T21:50:04.228808+0000","last_became_peered":"2023-12-26T21:50:04.228808+0000","last_unstale":"2023-12-26T21:50:05.214009+0000","last_undegraded":"2023-12-26T21:50:05.214009+0000","last_fullsized":"2023-12-26T21:50:05.214009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:10:52.365673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.199650+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:03:30.049843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238522+0000","last_change":"2023-12-26T21:50:05.233319+0000","last_active":"2023-12-26T21:50:05.238522+0000","last_peered":"2023-12-26T21:50:05.238522+0000","last_clean":"2023-12-26T21:50:05.238522+0000","last_became_active":"2023-12-26T21:50:05.232699+0000","last_became_peered":"2023-12-26T21:50:05.232699+0000","last_unstale":"2023-12-26T21:50:05.238522+0000","last_undegraded":"2023-12-26T21:50:05.238522+0000","last_fullsized":"2023-12-26T21:50:05.238522+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:02:49.364640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250467+0000","last_change":"2023-12-26T21:50:04.227027+0000","last_active":"2023-12-26T21:50:05.250467+0000","last_peered":"2023-12-26T21:50:05.250467+0000","last_clean":"2023-12-26T21:50:05.250467+0000","last_became_active":"2023-12-26T21:50:04.226702+0000","last_became_peered":"2023-12-26T21:50:04.226702+0000","last_unstale":"2023-12-26T21:50:05.250467+0000","last_undegraded":"2023-12-26T21:50:05.250467+0000","last_fullsized":"2023-12-26T21:50:05.250467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:43:44.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245546+0000","last_change":"2023-12-26T21:50:04.222842+0000","last_active":"2023-12-26T21:50:05.245546+0000","last_peered":"2023-12-26T21:50:05.245546+0000","last_clean":"2023-12-26T21:50:05.245546+0000","last_became_active":"2023-12-26T21:50:04.222585+0000","last_became_peered":"2023-12-26T21:50:04.222585+0000","last_unstale":"2023-12-26T21:50:05.245546+0000","last_undegraded":"2023-12-26T21:50:05.245546+0000","last_fullsized":"2023-12-26T21:50:05.245546+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:33:01.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227858+0000","last_change":"2023-12-26T21:50:05.225230+0000","last_active":"2023-12-26T21:50:05.227858+0000","last_peered":"2023-12-26T21:50:05.227858+0000","last_clean":"2023-12-26T21:50:05.227858+0000","last_became_active":"2023-12-26T21:50:05.224946+0000","last_became_peered":"2023-12-26T21:50:05.224946+0000","last_unstale":"2023-12-26T21:50:05.227858+0000","last_undegraded":"2023-12-26T21:50:05.227858+0000","last_fullsized":"2023-12-26T21:50:05.227858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:03:31.057770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202664+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:58.818709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.219854+0000","last_change":"2023-12-26T21:50:05.217771+0000","last_active":"2023-12-26T21:50:05.219854+0000","last_peered":"2023-12-26T21:50:05.219854+0000","last_clean":"2023-12-26T21:50:05.219854+0000","last_became_active":"2023-12-26T21:50:05.217485+0000","last_became_peered":"2023-12-26T21:50:05.217485+0000","last_unstale":"2023-12-26T21:50:05.219854+0000","last_undegraded":"2023-12-26T21:50:05.219854+0000","last_fullsized":"2023-12-26T21:50:05.219854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:54.647443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.207201+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:59:15.144868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250174+0000","last_change":"2023-12-26T21:50:04.224690+0000","last_active":"2023-12-26T21:50:05.250174+0000","last_peered":"2023-12-26T21:50:05.250174+0000","last_clean":"2023-12-26T21:50:05.250174+0000","last_became_active":"2023-12-26T21:50:04.223460+0000","last_became_peered":"2023-12-26T21:50:04.223460+0000","last_unstale":"2023-12-26T21:50:05.250174+0000","last_undegraded":"2023-12-26T21:50:05.250174+0000","last_fullsized":"2023-12-26T21:50:05.250174+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:01.981864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216840+0000","last_change":"2023-12-26T21:50:05.213363+0000","last_active":"2023-12-26T21:50:05.216840+0000","last_peered":"2023-12-26T21:50:05.216840+0000","last_clean":"2023-12-26T21:50:05.216840+0000","last_became_active":"2023-12-26T21:50:05.212965+0000","last_became_peered":"2023-12-26T21:50:05.212965+0000","last_unstale":"2023-12-26T21:50:05.216840+0000","last_undegraded":"2023-12-26T21:50:05.216840+0000","last_fullsized":"2023-12-26T21:50:05.216840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:01:27.770799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202373+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:23.901091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.223902+0000","last_change":"2023-12-26T21:50:05.222959+0000","last_active":"2023-12-26T21:50:05.223902+0000","last_peered":"2023-12-26T21:50:05.223902+0000","last_clean":"2023-12-26T21:50:05.223902+0000","last_became_active":"2023-12-26T21:50:05.222618+0000","last_became_peered":"2023-12-26T21:50:05.222618+0000","last_unstale":"2023-12-26T21:50:05.223902+0000","last_undegraded":"2023-12-26T21:50:05.223902+0000","last_fullsized":"2023-12-26T21:50:05.223902+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:13.568346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.205291+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:37.290110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.204851+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:58:34.233954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.203022+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:37:47.067537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225878+0000","last_change":"2023-12-26T21:50:05.224746+0000","last_active":"2023-12-26T21:50:05.225878+0000","last_peered":"2023-12-26T21:50:05.225878+0000","last_clean":"2023-12-26T21:50:05.225878+0000","last_became_active":"2023-12-26T21:50:05.224302+0000","last_became_peered":"2023-12-26T21:50:05.224302+0000","last_unstale":"2023-12-26T21:50:05.225878+0000","last_undegraded":"2023-12-26T21:50:05.225878+0000","last_fullsized":"2023-12-26T21:50:05.225878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:44.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248205+0000","last_change":"2023-12-26T21:50:04.226030+0000","last_active":"2023-12-26T21:50:05.248205+0000","last_peered":"2023-12-26T21:50:05.248205+0000","last_clean":"2023-12-26T21:50:05.248205+0000","last_became_active":"2023-12-26T21:50:04.225766+0000","last_became_peered":"2023-12-26T21:50:04.225766+0000","last_unstale":"2023-12-26T21:50:05.248205+0000","last_undegraded":"2023-12-26T21:50:05.248205+0000","last_fullsized":"2023-12-26T21:50:05.248205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:58.136823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202075+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:55:08.870277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216148+0000","last_change":"2023-12-26T21:50:04.225794+0000","last_active":"2023-12-26T21:50:05.216148+0000","last_peered":"2023-12-26T21:50:05.216148+0000","last_clean":"2023-12-26T21:50:05.216148+0000","last_became_active":"2023-12-26T21:50:04.225535+0000","last_became_peered":"2023-12-26T21:50:04.225535+0000","last_unstale":"2023-12-26T21:50:05.216148+0000","last_undegraded":"2023-12-26T21:50:05.216148+0000","last_fullsized":"2023-12-26T21:50:05.216148+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:01:54.466108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.213208+0000","last_change":"2023-12-26T21:50:04.224397+0000","last_active":"2023-12-26T21:50:05.213208+0000","last_peered":"2023-12-26T21:50:05.213208+0000","last_clean":"2023-12-26T21:50:05.213208+0000","last_became_active":"2023-12-26T21:50:04.224078+0000","last_became_peered":"2023-12-26T21:50:04.224078+0000","last_unstale":"2023-12-26T21:50:05.213208+0000","last_undegraded":"2023-12-26T21:50:05.213208+0000","last_fullsized":"2023-12-26T21:50:05.213208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:28.511389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234103+0000","last_change":"2023-12-26T21:50:05.227159+0000","last_active":"2023-12-26T21:50:05.234103+0000","last_peered":"2023-12-26T21:50:05.234103+0000","last_clean":"2023-12-26T21:50:05.234103+0000","last_became_active":"2023-12-26T21:50:05.226887+0000","last_became_peered":"2023-12-26T21:50:05.226887+0000","last_unstale":"2023-12-26T21:50:05.234103+0000","last_undegraded":"2023-12-26T21:50:05.234103+0000","last_fullsized":"2023-12-26T21:50:05.234103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:01:34.210345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202292+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:49:20.718326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225824+0000","last_change":"2023-12-26T21:50:05.223267+0000","last_active":"2023-12-26T21:50:05.225824+0000","last_peered":"2023-12-26T21:50:05.225824+0000","last_clean":"2023-12-26T21:50:05.225824+0000","last_became_active":"2023-12-26T21:50:05.222964+0000","last_became_peered":"2023-12-26T21:50:05.222964+0000","last_unstale":"2023-12-26T21:50:05.225824+0000","last_undegraded":"2023-12-26T21:50:05.225824+0000","last_fullsized":"2023-12-26T21:50:05.225824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:42:16.214790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202129+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:39:05.981705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239297+0000","last_change":"2023-12-26T21:50:05.233495+0000","last_active":"2023-12-26T21:50:05.239297+0000","last_peered":"2023-12-26T21:50:05.239297+0000","last_clean":"2023-12-26T21:50:05.239297+0000","last_became_active":"2023-12-26T21:50:05.233051+0000","last_became_peered":"2023-12-26T21:50:05.233051+0000","last_unstale":"2023-12-26T21:50:05.239297+0000","last_undegraded":"2023-12-26T21:50:05.239297+0000","last_fullsized":"2023-12-26T21:50:05.239297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:49:53.896795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208240+0000","last_change":"2023-12-26T21:50:04.218089+0000","last_active":"2023-12-26T21:50:05.208240+0000","last_peered":"2023-12-26T21:50:05.208240+0000","last_clean":"2023-12-26T21:50:05.208240+0000","last_became_active":"2023-12-26T21:50:04.215530+0000","last_became_peered":"2023-12-26T21:50:04.215530+0000","last_unstale":"2023-12-26T21:50:05.208240+0000","last_undegraded":"2023-12-26T21:50:05.208240+0000","last_fullsized":"2023-12-26T21:50:05.208240+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:21.909145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225858+0000","last_change":"2023-12-26T21:50:05.224578+0000","last_active":"2023-12-26T21:50:05.225858+0000","last_peered":"2023-12-26T21:50:05.225858+0000","last_clean":"2023-12-26T21:50:05.225858+0000","last_became_active":"2023-12-26T21:50:05.223985+0000","last_became_peered":"2023-12-26T21:50:05.223985+0000","last_unstale":"2023-12-26T21:50:05.225858+0000","last_undegraded":"2023-12-26T21:50:05.225858+0000","last_fullsized":"2023-12-26T21:50:05.225858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:47:51.264287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.201997+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:05.549244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212760+0000","last_change":"2023-12-26T21:50:04.222797+0000","last_active":"2023-12-26T21:50:05.212760+0000","last_peered":"2023-12-26T21:50:05.212760+0000","last_clean":"2023-12-26T21:50:05.212760+0000","last_became_active":"2023-12-26T21:50:04.222490+0000","last_became_peered":"2023-12-26T21:50:04.222490+0000","last_unstale":"2023-12-26T21:50:05.212760+0000","last_undegraded":"2023-12-26T21:50:05.212760+0000","last_fullsized":"2023-12-26T21:50:05.212760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:17.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202374+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:24:45.381424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208086+0000","last_change":"2023-12-26T21:50:04.220953+0000","last_active":"2023-12-26T21:50:05.208086+0000","last_peered":"2023-12-26T21:50:05.208086+0000","last_clean":"2023-12-26T21:50:05.208086+0000","last_became_active":"2023-12-26T21:50:04.220694+0000","last_became_peered":"2023-12-26T21:50:04.220694+0000","last_unstale":"2023-12-26T21:50:05.208086+0000","last_undegraded":"2023-12-26T21:50:05.208086+0000","last_fullsized":"2023-12-26T21:50:05.208086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:05.018373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245342+0000","last_change":"2023-12-26T21:50:04.226718+0000","last_active":"2023-12-26T21:50:05.245342+0000","last_peered":"2023-12-26T21:50:05.245342+0000","last_clean":"2023-12-26T21:50:05.245342+0000","last_became_active":"2023-12-26T21:50:04.226415+0000","last_became_peered":"2023-12-26T21:50:04.226415+0000","last_unstale":"2023-12-26T21:50:05.245342+0000","last_undegraded":"2023-12-26T21:50:05.245342+0000","last_fullsized":"2023-12-26T21:50:05.245342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:29.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"53'139","reported_seq":275,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-26T21:50:00.637985+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:50:00.637985+0000","last_peered":"2023-12-26T21:50:00.637985+0000","last_clean":"2023-12-26T21:50:00.637985+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:50:00.637985+0000","last_undegraded":"2023-12-26T21:50:00.637985+0000","last_fullsized":"2023-12-26T21:50:00.637985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1570,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542308,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":351691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.89100000000000001,"5min":0.94099999999999995,"15min":1.647},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":1,"5min":1,"15min":2.0150000000000001},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72599999999999998,"15min":0.70699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.016,"5min":1.048,"15min":1.1359999999999999},"last":0.436}]},{"osd":2,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.8640000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":0.95599999999999996,"15min":1.419},"last":0.63200000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":0.77200000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.026,"15min":1.327},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91900000000000004,"5min":1.008,"15min":1.393},"last":0.60699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":0.89400000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.371},"max":{"1min":0.84399999999999997,"5min":0.85199999999999998,"15min":0.86799999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":0.97599999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75700000000000001,"15min":0.745},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.499},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79100000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":0.92600000000000005,"15min":0.999},"last":0.68799999999999994}]}]},{"osd":6,"up_from":15,"seq":64424509603,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302072,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069768,"statfs":{"total":96636764160,"available":96327442432,"internally_reserved":0,"allocated":704512,"data_stored":325082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":1.0009999999999999,"15min":1.085},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.748,"15min":0.72199999999999998},"min":{"1min":0.56399999999999995,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":1.3380000000000001},"last":0.57299999999999995}]},{"osd":1,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.60299999999999998,"5min":0.44,"15min":0.41499999999999998},"max":{"1min":0.94699999999999995,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.71999999999999997},"min":{"1min":0.52300000000000002,"5min":0.41499999999999998,"15min":0.40799999999999997},"max":{"1min":0.86599999999999999,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.5}]},{"osd":2,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73599999999999999,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.46000000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.97099999999999997,"15min":1.288},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72699999999999998,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.497,"15min":0.45600000000000002},"max":{"1min":0.92000000000000004,"5min":0.93200000000000005,"15min":1.113},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.745,"15min":0.73399999999999999},"min":{"1min":0.54700000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":0.93600000000000005,"15min":1.569},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.77000000000000002,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.50600000000000001},"max":{"1min":0.96999999999999997,"5min":1.0129999999999999,"15min":1.3839999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.63300000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.377},"max":{"1min":0.81299999999999994,"5min":0.92700000000000005,"15min":1.137},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.65700000000000003},"min":{"1min":0.503,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93200000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.64500000000000002},"min":{"1min":0.42099999999999999,"5min":0.41899999999999998,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.875,"15min":1.1200000000000001},"last":0.754},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67000000000000004,"15min":0.70099999999999996},"min":{"1min":0.54800000000000004,"5min":0.46200000000000002,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":0.93600000000000005,"15min":1.296},"last":0.76700000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69399999999999995,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.437},"max":{"1min":0.89500000000000002,"5min":1.0920000000000001,"15min":1.51},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.998,"15min":1.2010000000000001},"last":0.79900000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509603,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306760,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065080,"statfs":{"total":96636764160,"available":96322641920,"internally_reserved":0,"allocated":1310720,"data_stored":921619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.309,"5min":0.307,"15min":0.255},"max":{"1min":0.65900000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.501,"15min":0.52900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.754,"15min":4.6600000000000001},"last":0.51600000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.49199999999999999,"15min":0.497},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.59699999999999998,"5min":1.0189999999999999,"15min":2.2869999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55400000000000005,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.29699999999999999},"max":{"1min":0.78800000000000003,"5min":0.93200000000000005,"15min":4.3639999999999999},"last":0.501}]},{"osd":3,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.38500000000000001},"max":{"1min":0.871,"5min":0.998,"15min":4.3259999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.33300000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":4.548},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92100000000000004,"5min":1.252,"15min":1.252},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.754,"15min":0.747},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.441},"max":{"1min":0.90200000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.51300000000000001},"max":{"1min":0.93700000000000006,"5min":1.2949999999999999,"15min":1.345},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.81799999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.82199999999999995,"15min":0.80100000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.51400000000000001},"max":{"1min":0.89500000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.83399999999999996,"15min":0.81100000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":0.90700000000000003,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0149999999999999,"5min":1.0980000000000001,"15min":1.1539999999999999},"last":0.746},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.94399999999999995,"5min":1.3620000000000001,"15min":1.3919999999999999},"last":0.55700000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509603,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302072,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069768,"statfs":{"total":96636764160,"available":96327442432,"internally_reserved":0,"allocated":704512,"data_stored":325082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42799999999999999,"15min":0.442},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":1.5640000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.48499999999999999,"15min":0.51200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60599999999999998,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":0.68300000000000005,"15min":2.012},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.51400000000000001,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56499999999999995},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.84499999999999997,"15min":4.2610000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.746,"15min":0.78000000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.85399999999999998,"5min":1.1539999999999999,"15min":4.423},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.93500000000000005,"5min":1.1000000000000001,"15min":1.6439999999999999},"last":0.622}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.78600000000000003,"15min":0.82099999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":1.119,"15min":4.3650000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.94599999999999995,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":1.103,"15min":1.611},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79300000000000004,"15min":0.81999999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.5},"max":{"1min":1.002,"5min":1.111,"15min":4.407},"last":0.83499999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.82099999999999995},"min":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":0.89600000000000002,"5min":1.145,"15min":4.5069999999999997},"last":0.69199999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302084,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069756,"statfs":{"total":96636764160,"available":96327430144,"internally_reserved":0,"allocated":716800,"data_stored":331251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.32900000000000001,"5min":0.27000000000000002,"15min":0.248},"max":{"1min":0.67000000000000004,"5min":0.85999999999999999,"15min":1.915},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":1.7729999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48499999999999999,"15min":0.496},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70899999999999996,"15min":1.952},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.30199999999999999},"max":{"1min":0.63100000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.44800000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.438,"5min":0.437,"15min":0.375},"max":{"1min":0.78700000000000003,"5min":0.83999999999999997,"15min":0.97099999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.98299999999999998,"5min":1.1180000000000001,"15min":1.284},"last":0.753},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.98399999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.92200000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.75800000000000001},"min":{"1min":0.53900000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.5029999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78000000000000003,"15min":0.77200000000000002},"min":{"1min":0.621,"5min":0.54200000000000004,"15min":0.503},"max":{"1min":0.89900000000000002,"5min":1.028,"15min":1.355},"last":0.88100000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.91200000000000003,"5min":1.032,"15min":1.7350000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80200000000000005,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.90200000000000002,"5min":1.0640000000000001,"15min":1.349},"last":0.873}]},{"osd":7,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.47799999999999998},"max":{"1min":0.95999999999999996,"5min":1.0680000000000001,"15min":1.841},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":1.004,"5min":1.091,"15min":1.294},"last":0.81999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509605,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306228,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065612,"statfs":{"total":96636764160,"available":96323186688,"internally_reserved":0,"allocated":765952,"data_stored":353193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.438,"15min":0.442},"min":{"1min":0.312,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.627,"5min":0.68100000000000005,"15min":0.91400000000000003},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35699999999999998,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.30399999999999999},"max":{"1min":0.65900000000000003,"5min":0.73499999999999999,"15min":0.77200000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.32400000000000001},"max":{"1min":0.66800000000000004,"5min":0.72499999999999998,"15min":0.99399999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38900000000000001},"max":{"1min":0.72099999999999997,"5min":0.83699999999999997,"15min":1.0820000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.82599999999999996,"15min":1.3580000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.499,"15min":0.45100000000000001},"max":{"1min":1.018,"5min":1.105,"15min":1.3779999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":1.1870000000000001,"15min":1.2949999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.47699999999999998},"max":{"1min":1,"5min":1.27,"15min":1.4610000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.78400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.45900000000000002},"max":{"1min":0.96399999999999997,"5min":0.997,"15min":1.9239999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.1950000000000001,"15min":1.5049999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.54100000000000004},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.579},"last":0.87}]},{"osd":7,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78600000000000003,"15min":0.80200000000000005},"min":{"1min":0.63800000000000001,"5min":0.51100000000000001,"15min":0.49299999999999999},"max":{"1min":1.036,"5min":1.0640000000000001,"15min":1.964},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.1499999999999999,"15min":1.6359999999999999},"last":0.92400000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069700,"statfs":{"total":96636764160,"available":96327372800,"internally_reserved":0,"allocated":774144,"data_stored":353285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":0.96499999999999997,"15min":2.1389999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.008,"15min":1.407},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.878,"5min":0.90500000000000003,"15min":1.879},"last":0.443},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.70799999999999996,"15min":0.72199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":2.097},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.72399999999999998},"min":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":0.94399999999999995,"15min":1.7809999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":1.6870000000000001},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70799999999999996,"15min":0.72599999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.83899999999999997,"5min":0.95999999999999996,"15min":1.956},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72899999999999998,"15min":0.74399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.83199999999999996,"5min":0.97999999999999998,"15min":2.0609999999999999},"last":0.69399999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58799999999999997,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.89800000000000002,"15min":2.2429999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.64200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77000000000000002,"5min":0.92600000000000005,"15min":2.194},"last":0.81799999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60199999999999998,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.878,"15min":2.1589999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64700000000000002,"15min":0.70299999999999996},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.874,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":0.98099999999999998,"15min":2.23},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.57199999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90100000000000002,"5min":0.92700000000000005,"15min":2.5169999999999999},"last":0.79900000000000004}]}]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1310720,"data_stored":921619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":0.995,"15min":2.3900000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70799999999999996,"15min":0.73599999999999999},"min":{"1min":0.499,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.874,"5min":1.0369999999999999,"15min":1.71},"last":0.81200000000000006}]},{"osd":1,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.51500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.61499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.4219999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72399999999999998,"15min":0.746},"min":{"1min":0.47899999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.622,"5min":0.435,"15min":0.435},"max":{"1min":0.81999999999999995,"5min":0.95699999999999996,"15min":1.214},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.74299999999999999},"min":{"1min":0.56399999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.4890000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.874,"5min":0.93899999999999995,"15min":1.1439999999999999},"last":0.82399999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":2.5070000000000001},"last":0.755},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59099999999999997,"15min":0.65600000000000003},"min":{"1min":0.55400000000000005,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90100000000000002,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.73099999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64300000000000002,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"max":{"1min":0.84299999999999997,"5min":0.90700000000000003,"15min":1.637},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.76500000000000001},"min":{"1min":0.65200000000000002,"5min":0.49099999999999999,"15min":0.438},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":2.5579999999999998},"last":0.85299999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67400000000000004,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.83299999999999996,"5min":0.96299999999999997,"15min":1.248},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68300000000000005,"15min":0.73299999999999998},"min":{"1min":0.56899999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83499999999999996,"5min":0.97199999999999998,"15min":1.5980000000000001},"last":0.88}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:50:10.035 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:50:10.192 INFO:tasks.ceph.mds.c.smithi131.stderr:2023-12-26T21:50:10.192+0000 7fe5bf3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:50:10.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.260+0000 7ff7d29ff700 1 Processor -- start 2023-12-26T21:50:10.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.260+0000 7ff7d29ff700 1 -- start start 2023-12-26T21:50:10.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.261+0000 7ff7d29ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc1596a0 0x7ff7cc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.261+0000 7ff7d29ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7cc0b0870 0x7ff7cc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.262+0000 7ff7d29ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc0afcd0 con 0x7ff7cc159a70 2023-12-26T21:50:10.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.262+0000 7ff7d29ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7cc0b3e70 con 0x7ff7cc0b0870 2023-12-26T21:50:10.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.262+0000 7ff7d29ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0b0cd0 con 0x7ff7cc1596a0 2023-12-26T21:50:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.262+0000 7ff7cbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc1596a0 0x7ff7cc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.262+0000 7ff7cbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc1596a0 0x7ff7cc0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:47628/0 (socket says 172.21.15.131:47628) 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.262+0000 7ff7cbfff700 1 -- 172.21.15.131:0/1311190055 learned_addr learned my addr 172.21.15.131:0/1311190055 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.262+0000 7ff7d0f9c700 1 --2- 172.21.15.131:0/1311190055 >> v2:172.21.15.190:3300/0 conn(0x7ff7cc0b0870 0x7ff7cc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.262+0000 7ff7c3fff700 1 --1- 172.21.15.131:0/1311190055 >> v1:172.21.15.190:6789/0 conn(0x7ff7cc159a70 0x7ff7cc0b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:40152/0 (socket says 172.21.15.131:40152) 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.263+0000 7ff7cbfff700 1 -- 172.21.15.131:0/1311190055 >> v2:172.21.15.190:3300/0 conn(0x7ff7cc0b0870 msgr2=0x7ff7cc0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.263+0000 7ff7cbfff700 1 --2- 172.21.15.131:0/1311190055 >> v2:172.21.15.190:3300/0 conn(0x7ff7cc0b0870 0x7ff7cc0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.263+0000 7ff7cbfff700 1 -- 172.21.15.131:0/1311190055 >> v1:172.21.15.190:6789/0 conn(0x7ff7cc159a70 legacy=0x7ff7cc0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.263+0000 7ff7cbfff700 1 -- 172.21.15.131:0/1311190055 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc0b2e00 con 0x7ff7cc1596a0 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.263+0000 7ff7cbfff700 1 --2- 172.21.15.131:0/1311190055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc1596a0 0x7ff7cc0b2380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0099b0 tx=0x7ff7bc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca87681c556dcc53 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.263+0000 7ff7cb7fe700 1 -- 172.21.15.131:0/1311190055 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc009ea0 con 0x7ff7cc1596a0 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.263+0000 7ff7cb7fe700 1 -- 172.21.15.131:0/1311190055 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc00b860 con 0x7ff7cc1596a0 2023-12-26T21:50:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.263+0000 7ff7cb7fe700 1 -- 172.21.15.131:0/1311190055 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc00bc00 con 0x7ff7cc1596a0 2023-12-26T21:50:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.264+0000 7ff7d29ff700 1 -- 172.21.15.131:0/1311190055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc1596a0 msgr2=0x7ff7cc0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.264+0000 7ff7d29ff700 1 --2- 172.21.15.131:0/1311190055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc1596a0 0x7ff7cc0b2380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0099b0 tx=0x7ff7bc009250 comp rx=0 tx=0).stop 2023-12-26T21:50:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.264+0000 7ff7d29ff700 1 -- 172.21.15.131:0/1311190055 shutdown_connections 2023-12-26T21:50:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.264+0000 7ff7d29ff700 1 --2- 172.21.15.131:0/1311190055 >> v2:172.21.15.190:3300/0 conn(0x7ff7cc0b0870 0x7ff7cc0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.264+0000 7ff7d29ff700 1 --2- 172.21.15.131:0/1311190055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc1596a0 0x7ff7cc0b2380 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.264+0000 7ff7d29ff700 1 -- 172.21.15.131:0/1311190055 >> 172.21.15.131:0/1311190055 conn(0x7ff7cc06ece0 msgr2=0x7ff7cc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.264+0000 7ff7d29ff700 1 -- 172.21.15.131:0/1311190055 shutdown_connections 2023-12-26T21:50:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.264+0000 7ff7d29ff700 1 -- 172.21.15.131:0/1311190055 wait complete. 2023-12-26T21:50:10.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.265+0000 7ff7d29ff700 1 Processor -- start 2023-12-26T21:50:10.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.265+0000 7ff7d29ff700 1 -- start start 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.266+0000 7ff7d29ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc0b0870 0x7ff7cc1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.266+0000 7ff7d29ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7cc1596a0 0x7ff7cc154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.266+0000 7ff7d29ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7cc159a70 0x7ff7cc15c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.266+0000 7ff7d29ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0cc950 con 0x7ff7cc159a70 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.266+0000 7ff7d29ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0cbd40 con 0x7ff7cc1596a0 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.266+0000 7ff7d29ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0cc020 con 0x7ff7cc0b0870 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d0f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7cc159a70 0x7ff7cc15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d0f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7cc159a70 0x7ff7cc15c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49836/0 (socket says 172.21.15.131:49836) 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d0f9c700 1 -- 172.21.15.131:0/2982261720 learned_addr learned my addr 172.21.15.131:0/2982261720 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7cbfff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc0b0870 0x7ff7cc1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7c3fff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7cc1596a0 0x7ff7cc154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d0f9c700 1 -- 172.21.15.131:0/2982261720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc0b0870 msgr2=0x7ff7cc1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d0f9c700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc0b0870 0x7ff7cc1558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d0f9c700 1 -- 172.21.15.131:0/2982261720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7cc1596a0 msgr2=0x7ff7cc154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d0f9c700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7cc1596a0 0x7ff7cc154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d0f9c700 1 -- 172.21.15.131:0/2982261720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0094f0 con 0x7ff7cc159a70 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d0f9c700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7cc159a70 0x7ff7cc15c440 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009e90 tx=0x7ff7c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7c8ff9700 1 -- 172.21.15.131:0/2982261720 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400cb70 con 0x7ff7cc159a70 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7c8ff9700 1 -- 172.21.15.131:0/2982261720 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c400f040 con 0x7ff7cc159a70 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.267+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc0cc300 con 0x7ff7cc159a70 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.268+0000 7ff7c8ff9700 1 -- 172.21.15.131:0/2982261720 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4014480 con 0x7ff7cc159a70 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.268+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc156360 con 0x7ff7cc159a70 2023-12-26T21:50:10.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.269+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc0ccb00 con 0x7ff7cc159a70 2023-12-26T21:50:10.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.270+0000 7ff7c8ff9700 1 -- 172.21.15.131:0/2982261720 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff7c4014830 con 0x7ff7cc159a70 2023-12-26T21:50:10.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.270+0000 7ff7c8ff9700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7ac066f30 0x7ff7ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.270+0000 7ff7c8ff9700 1 -- 172.21.15.131:0/2982261720 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff7c40853f0 con 0x7ff7cc159a70 2023-12-26T21:50:10.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.271+0000 7ff7cbfff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7ac066f30 0x7ff7ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.271+0000 7ff7cbfff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7ac066f30 0x7ff7ac0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0099b0 tx=0x7ff7bc017000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:10.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.274+0000 7ff7c8ff9700 1 -- 172.21.15.131:0/2982261720 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7cc0ccb00 con 0x7ff7cc159a70 2023-12-26T21:50:10.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.466+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7cc0ccb00 con 0x7ff7ac066f30 2023-12-26T21:50:10.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.474+0000 7ff7c8ff9700 1 -- 172.21.15.131:0/2982261720 <== mgr.5437 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282183 (secure 0 0 0) 0x7ff7cc0ccb00 con 0x7ff7ac066f30 2023-12-26T21:50:10.478 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.477+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7ac066f30 msgr2=0x7ff7ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.477+0000 7ff7d29ff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7ac066f30 0x7ff7ac0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0099b0 tx=0x7ff7bc017000 comp rx=0 tx=0).stop 2023-12-26T21:50:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.477+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7cc159a70 msgr2=0x7ff7cc15c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.477+0000 7ff7d29ff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7cc159a70 0x7ff7cc15c440 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009e90 tx=0x7ff7c400b040 comp rx=0 tx=0).stop 2023-12-26T21:50:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.478+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 shutdown_connections 2023-12-26T21:50:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.478+0000 7ff7d29ff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7cc159a70 0x7ff7cc15c440 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.478+0000 7ff7d29ff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff7ac066f30 0x7ff7ac0693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.478+0000 7ff7d29ff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7cc0b0870 0x7ff7cc1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.478+0000 7ff7d29ff700 1 --2- 172.21.15.131:0/2982261720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7cc1596a0 0x7ff7cc154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.478+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 >> 172.21.15.131:0/2982261720 conn(0x7ff7cc06ece0 msgr2=0x7ff7cc0cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:10.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.478+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 shutdown_connections 2023-12-26T21:50:10.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.478+0000 7ff7d29ff700 1 -- 172.21.15.131:0/2982261720 wait complete. 2023-12-26T21:50:10.482 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:50:10.496 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:50:08.713348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591938,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":227,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422048,"kb_used_data":6880,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752552672,"statfs":{"total":773094113280,"available":770613936128,"internally_reserved":0,"allocated":7045120,"data_stored":3882822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1570,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.514395"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239194+0000","last_change":"2023-12-26T21:50:05.233540+0000","last_active":"2023-12-26T21:50:05.239194+0000","last_peered":"2023-12-26T21:50:05.239194+0000","last_clean":"2023-12-26T21:50:05.239194+0000","last_became_active":"2023-12-26T21:50:05.233189+0000","last_became_peered":"2023-12-26T21:50:05.233189+0000","last_unstale":"2023-12-26T21:50:05.239194+0000","last_undegraded":"2023-12-26T21:50:05.239194+0000","last_fullsized":"2023-12-26T21:50:05.239194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:38.099699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.231885+0000","last_change":"2023-12-26T21:50:05.226840+0000","last_active":"2023-12-26T21:50:05.231885+0000","last_peered":"2023-12-26T21:50:05.231885+0000","last_clean":"2023-12-26T21:50:05.231885+0000","last_became_active":"2023-12-26T21:50:05.226580+0000","last_became_peered":"2023-12-26T21:50:05.226580+0000","last_unstale":"2023-12-26T21:50:05.231885+0000","last_undegraded":"2023-12-26T21:50:05.231885+0000","last_fullsized":"2023-12-26T21:50:05.231885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:15:04.988046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225805+0000","last_change":"2023-12-26T21:50:05.221852+0000","last_active":"2023-12-26T21:50:05.225805+0000","last_peered":"2023-12-26T21:50:05.225805+0000","last_clean":"2023-12-26T21:50:05.225805+0000","last_became_active":"2023-12-26T21:50:05.221574+0000","last_became_peered":"2023-12-26T21:50:05.221574+0000","last_unstale":"2023-12-26T21:50:05.225805+0000","last_undegraded":"2023-12-26T21:50:05.225805+0000","last_fullsized":"2023-12-26T21:50:05.225805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:45:09.667824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216704+0000","last_change":"2023-12-26T21:50:05.213544+0000","last_active":"2023-12-26T21:50:05.216704+0000","last_peered":"2023-12-26T21:50:05.216704+0000","last_clean":"2023-12-26T21:50:05.216704+0000","last_became_active":"2023-12-26T21:50:05.212622+0000","last_became_peered":"2023-12-26T21:50:05.212622+0000","last_unstale":"2023-12-26T21:50:05.216704+0000","last_undegraded":"2023-12-26T21:50:05.216704+0000","last_fullsized":"2023-12-26T21:50:05.216704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:45.329023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238621+0000","last_change":"2023-12-26T21:50:05.231337+0000","last_active":"2023-12-26T21:50:05.238621+0000","last_peered":"2023-12-26T21:50:05.238621+0000","last_clean":"2023-12-26T21:50:05.238621+0000","last_became_active":"2023-12-26T21:50:05.231031+0000","last_became_peered":"2023-12-26T21:50:05.231031+0000","last_unstale":"2023-12-26T21:50:05.238621+0000","last_undegraded":"2023-12-26T21:50:05.238621+0000","last_fullsized":"2023-12-26T21:50:05.238621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:51:34.322061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239316+0000","last_change":"2023-12-26T21:50:05.233798+0000","last_active":"2023-12-26T21:50:05.239316+0000","last_peered":"2023-12-26T21:50:05.239316+0000","last_clean":"2023-12-26T21:50:05.239316+0000","last_became_active":"2023-12-26T21:50:05.229125+0000","last_became_peered":"2023-12-26T21:50:05.229125+0000","last_unstale":"2023-12-26T21:50:05.239316+0000","last_undegraded":"2023-12-26T21:50:05.239316+0000","last_fullsized":"2023-12-26T21:50:05.239316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:53.114423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234135+0000","last_change":"2023-12-26T21:50:05.224391+0000","last_active":"2023-12-26T21:50:05.234135+0000","last_peered":"2023-12-26T21:50:05.234135+0000","last_clean":"2023-12-26T21:50:05.234135+0000","last_became_active":"2023-12-26T21:50:05.224146+0000","last_became_peered":"2023-12-26T21:50:05.224146+0000","last_unstale":"2023-12-26T21:50:05.234135+0000","last_undegraded":"2023-12-26T21:50:05.234135+0000","last_fullsized":"2023-12-26T21:50:05.234135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:37.819740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227740+0000","last_change":"2023-12-26T21:50:05.225269+0000","last_active":"2023-12-26T21:50:05.227740+0000","last_peered":"2023-12-26T21:50:05.227740+0000","last_clean":"2023-12-26T21:50:05.227740+0000","last_became_active":"2023-12-26T21:50:05.224997+0000","last_became_peered":"2023-12-26T21:50:05.224997+0000","last_unstale":"2023-12-26T21:50:05.227740+0000","last_undegraded":"2023-12-26T21:50:05.227740+0000","last_fullsized":"2023-12-26T21:50:05.227740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:03.075265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202405+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:18:07.070074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244190+0000","last_change":"2023-12-26T21:50:04.223739+0000","last_active":"2023-12-26T21:50:05.244190+0000","last_peered":"2023-12-26T21:50:05.244190+0000","last_clean":"2023-12-26T21:50:05.244190+0000","last_became_active":"2023-12-26T21:50:04.222755+0000","last_became_peered":"2023-12-26T21:50:04.222755+0000","last_unstale":"2023-12-26T21:50:05.244190+0000","last_undegraded":"2023-12-26T21:50:05.244190+0000","last_fullsized":"2023-12-26T21:50:05.244190+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:21.699131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.204232+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:11:47.600963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244999+0000","last_change":"2023-12-26T21:50:04.223118+0000","last_active":"2023-12-26T21:50:05.244999+0000","last_peered":"2023-12-26T21:50:05.244999+0000","last_clean":"2023-12-26T21:50:05.244999+0000","last_became_active":"2023-12-26T21:50:04.222720+0000","last_became_peered":"2023-12-26T21:50:04.222720+0000","last_unstale":"2023-12-26T21:50:05.244999+0000","last_undegraded":"2023-12-26T21:50:05.244999+0000","last_fullsized":"2023-12-26T21:50:05.244999+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:38:24.121296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.222331+0000","last_change":"2023-12-26T21:50:05.220983+0000","last_active":"2023-12-26T21:50:05.222331+0000","last_peered":"2023-12-26T21:50:05.222331+0000","last_clean":"2023-12-26T21:50:05.222331+0000","last_became_active":"2023-12-26T21:50:05.220694+0000","last_became_peered":"2023-12-26T21:50:05.220694+0000","last_unstale":"2023-12-26T21:50:05.222331+0000","last_undegraded":"2023-12-26T21:50:05.222331+0000","last_fullsized":"2023-12-26T21:50:05.222331+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:29:18.312367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.203565+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:29.468613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238883+0000","last_change":"2023-12-26T21:50:05.235706+0000","last_active":"2023-12-26T21:50:05.238883+0000","last_peered":"2023-12-26T21:50:05.238883+0000","last_clean":"2023-12-26T21:50:05.238883+0000","last_became_active":"2023-12-26T21:50:05.235416+0000","last_became_peered":"2023-12-26T21:50:05.235416+0000","last_unstale":"2023-12-26T21:50:05.238883+0000","last_undegraded":"2023-12-26T21:50:05.238883+0000","last_fullsized":"2023-12-26T21:50:05.238883+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:20.767760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248231+0000","last_change":"2023-12-26T21:50:04.225964+0000","last_active":"2023-12-26T21:50:05.248231+0000","last_peered":"2023-12-26T21:50:05.248231+0000","last_clean":"2023-12-26T21:50:05.248231+0000","last_became_active":"2023-12-26T21:50:04.225653+0000","last_became_peered":"2023-12-26T21:50:04.225653+0000","last_unstale":"2023-12-26T21:50:05.248231+0000","last_undegraded":"2023-12-26T21:50:05.248231+0000","last_fullsized":"2023-12-26T21:50:05.248231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:49.834751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.226468+0000","last_change":"2023-12-26T21:50:05.223489+0000","last_active":"2023-12-26T21:50:05.226468+0000","last_peered":"2023-12-26T21:50:05.226468+0000","last_clean":"2023-12-26T21:50:05.226468+0000","last_became_active":"2023-12-26T21:50:05.222774+0000","last_became_peered":"2023-12-26T21:50:05.222774+0000","last_unstale":"2023-12-26T21:50:05.226468+0000","last_undegraded":"2023-12-26T21:50:05.226468+0000","last_fullsized":"2023-12-26T21:50:05.226468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:50.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202039+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:38:21.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239174+0000","last_change":"2023-12-26T21:50:05.233433+0000","last_active":"2023-12-26T21:50:05.239174+0000","last_peered":"2023-12-26T21:50:05.239174+0000","last_clean":"2023-12-26T21:50:05.239174+0000","last_became_active":"2023-12-26T21:50:05.228341+0000","last_became_peered":"2023-12-26T21:50:05.228341+0000","last_unstale":"2023-12-26T21:50:05.239174+0000","last_undegraded":"2023-12-26T21:50:05.239174+0000","last_fullsized":"2023-12-26T21:50:05.239174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:58:54.385865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214676+0000","last_change":"2023-12-26T21:50:04.221595+0000","last_active":"2023-12-26T21:50:05.214676+0000","last_peered":"2023-12-26T21:50:05.214676+0000","last_clean":"2023-12-26T21:50:05.214676+0000","last_became_active":"2023-12-26T21:50:04.221337+0000","last_became_peered":"2023-12-26T21:50:04.221337+0000","last_unstale":"2023-12-26T21:50:05.214676+0000","last_undegraded":"2023-12-26T21:50:05.214676+0000","last_fullsized":"2023-12-26T21:50:05.214676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:38:20.838693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.205068+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:24.364623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214009+0000","last_change":"2023-12-26T21:50:04.229086+0000","last_active":"2023-12-26T21:50:05.214009+0000","last_peered":"2023-12-26T21:50:05.214009+0000","last_clean":"2023-12-26T21:50:05.214009+0000","last_became_active":"2023-12-26T21:50:04.228808+0000","last_became_peered":"2023-12-26T21:50:04.228808+0000","last_unstale":"2023-12-26T21:50:05.214009+0000","last_undegraded":"2023-12-26T21:50:05.214009+0000","last_fullsized":"2023-12-26T21:50:05.214009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:10:52.365673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.199650+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:03:30.049843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238522+0000","last_change":"2023-12-26T21:50:05.233319+0000","last_active":"2023-12-26T21:50:05.238522+0000","last_peered":"2023-12-26T21:50:05.238522+0000","last_clean":"2023-12-26T21:50:05.238522+0000","last_became_active":"2023-12-26T21:50:05.232699+0000","last_became_peered":"2023-12-26T21:50:05.232699+0000","last_unstale":"2023-12-26T21:50:05.238522+0000","last_undegraded":"2023-12-26T21:50:05.238522+0000","last_fullsized":"2023-12-26T21:50:05.238522+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:02:49.364640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250467+0000","last_change":"2023-12-26T21:50:04.227027+0000","last_active":"2023-12-26T21:50:05.250467+0000","last_peered":"2023-12-26T21:50:05.250467+0000","last_clean":"2023-12-26T21:50:05.250467+0000","last_became_active":"2023-12-26T21:50:04.226702+0000","last_became_peered":"2023-12-26T21:50:04.226702+0000","last_unstale":"2023-12-26T21:50:05.250467+0000","last_undegraded":"2023-12-26T21:50:05.250467+0000","last_fullsized":"2023-12-26T21:50:05.250467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:43:44.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245546+0000","last_change":"2023-12-26T21:50:04.222842+0000","last_active":"2023-12-26T21:50:05.245546+0000","last_peered":"2023-12-26T21:50:05.245546+0000","last_clean":"2023-12-26T21:50:05.245546+0000","last_became_active":"2023-12-26T21:50:04.222585+0000","last_became_peered":"2023-12-26T21:50:04.222585+0000","last_unstale":"2023-12-26T21:50:05.245546+0000","last_undegraded":"2023-12-26T21:50:05.245546+0000","last_fullsized":"2023-12-26T21:50:05.245546+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:33:01.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227858+0000","last_change":"2023-12-26T21:50:05.225230+0000","last_active":"2023-12-26T21:50:05.227858+0000","last_peered":"2023-12-26T21:50:05.227858+0000","last_clean":"2023-12-26T21:50:05.227858+0000","last_became_active":"2023-12-26T21:50:05.224946+0000","last_became_peered":"2023-12-26T21:50:05.224946+0000","last_unstale":"2023-12-26T21:50:05.227858+0000","last_undegraded":"2023-12-26T21:50:05.227858+0000","last_fullsized":"2023-12-26T21:50:05.227858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:03:31.057770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202664+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:58.818709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.219854+0000","last_change":"2023-12-26T21:50:05.217771+0000","last_active":"2023-12-26T21:50:05.219854+0000","last_peered":"2023-12-26T21:50:05.219854+0000","last_clean":"2023-12-26T21:50:05.219854+0000","last_became_active":"2023-12-26T21:50:05.217485+0000","last_became_peered":"2023-12-26T21:50:05.217485+0000","last_unstale":"2023-12-26T21:50:05.219854+0000","last_undegraded":"2023-12-26T21:50:05.219854+0000","last_fullsized":"2023-12-26T21:50:05.219854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:54.647443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.207201+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:59:15.144868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250174+0000","last_change":"2023-12-26T21:50:04.224690+0000","last_active":"2023-12-26T21:50:05.250174+0000","last_peered":"2023-12-26T21:50:05.250174+0000","last_clean":"2023-12-26T21:50:05.250174+0000","last_became_active":"2023-12-26T21:50:04.223460+0000","last_became_peered":"2023-12-26T21:50:04.223460+0000","last_unstale":"2023-12-26T21:50:05.250174+0000","last_undegraded":"2023-12-26T21:50:05.250174+0000","last_fullsized":"2023-12-26T21:50:05.250174+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:01.981864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216840+0000","last_change":"2023-12-26T21:50:05.213363+0000","last_active":"2023-12-26T21:50:05.216840+0000","last_peered":"2023-12-26T21:50:05.216840+0000","last_clean":"2023-12-26T21:50:05.216840+0000","last_became_active":"2023-12-26T21:50:05.212965+0000","last_became_peered":"2023-12-26T21:50:05.212965+0000","last_unstale":"2023-12-26T21:50:05.216840+0000","last_undegraded":"2023-12-26T21:50:05.216840+0000","last_fullsized":"2023-12-26T21:50:05.216840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:01:27.770799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202373+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:23.901091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.223902+0000","last_change":"2023-12-26T21:50:05.222959+0000","last_active":"2023-12-26T21:50:05.223902+0000","last_peered":"2023-12-26T21:50:05.223902+0000","last_clean":"2023-12-26T21:50:05.223902+0000","last_became_active":"2023-12-26T21:50:05.222618+0000","last_became_peered":"2023-12-26T21:50:05.222618+0000","last_unstale":"2023-12-26T21:50:05.223902+0000","last_undegraded":"2023-12-26T21:50:05.223902+0000","last_fullsized":"2023-12-26T21:50:05.223902+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:13.568346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.205291+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:37.290110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.204851+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:58:34.233954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.203022+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:37:47.067537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225878+0000","last_change":"2023-12-26T21:50:05.224746+0000","last_active":"2023-12-26T21:50:05.225878+0000","last_peered":"2023-12-26T21:50:05.225878+0000","last_clean":"2023-12-26T21:50:05.225878+0000","last_became_active":"2023-12-26T21:50:05.224302+0000","last_became_peered":"2023-12-26T21:50:05.224302+0000","last_unstale":"2023-12-26T21:50:05.225878+0000","last_undegraded":"2023-12-26T21:50:05.225878+0000","last_fullsized":"2023-12-26T21:50:05.225878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:44.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248205+0000","last_change":"2023-12-26T21:50:04.226030+0000","last_active":"2023-12-26T21:50:05.248205+0000","last_peered":"2023-12-26T21:50:05.248205+0000","last_clean":"2023-12-26T21:50:05.248205+0000","last_became_active":"2023-12-26T21:50:04.225766+0000","last_became_peered":"2023-12-26T21:50:04.225766+0000","last_unstale":"2023-12-26T21:50:05.248205+0000","last_undegraded":"2023-12-26T21:50:05.248205+0000","last_fullsized":"2023-12-26T21:50:05.248205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:58.136823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202075+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:55:08.870277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216148+0000","last_change":"2023-12-26T21:50:04.225794+0000","last_active":"2023-12-26T21:50:05.216148+0000","last_peered":"2023-12-26T21:50:05.216148+0000","last_clean":"2023-12-26T21:50:05.216148+0000","last_became_active":"2023-12-26T21:50:04.225535+0000","last_became_peered":"2023-12-26T21:50:04.225535+0000","last_unstale":"2023-12-26T21:50:05.216148+0000","last_undegraded":"2023-12-26T21:50:05.216148+0000","last_fullsized":"2023-12-26T21:50:05.216148+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:01:54.466108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.213208+0000","last_change":"2023-12-26T21:50:04.224397+0000","last_active":"2023-12-26T21:50:05.213208+0000","last_peered":"2023-12-26T21:50:05.213208+0000","last_clean":"2023-12-26T21:50:05.213208+0000","last_became_active":"2023-12-26T21:50:04.224078+0000","last_became_peered":"2023-12-26T21:50:04.224078+0000","last_unstale":"2023-12-26T21:50:05.213208+0000","last_undegraded":"2023-12-26T21:50:05.213208+0000","last_fullsized":"2023-12-26T21:50:05.213208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:28.511389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234103+0000","last_change":"2023-12-26T21:50:05.227159+0000","last_active":"2023-12-26T21:50:05.234103+0000","last_peered":"2023-12-26T21:50:05.234103+0000","last_clean":"2023-12-26T21:50:05.234103+0000","last_became_active":"2023-12-26T21:50:05.226887+0000","last_became_peered":"2023-12-26T21:50:05.226887+0000","last_unstale":"2023-12-26T21:50:05.234103+0000","last_undegraded":"2023-12-26T21:50:05.234103+0000","last_fullsized":"2023-12-26T21:50:05.234103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:01:34.210345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202292+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:49:20.718326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225824+0000","last_change":"2023-12-26T21:50:05.223267+0000","last_active":"2023-12-26T21:50:05.225824+0000","last_peered":"2023-12-26T21:50:05.225824+0000","last_clean":"2023-12-26T21:50:05.225824+0000","last_became_active":"2023-12-26T21:50:05.222964+0000","last_became_peered":"2023-12-26T21:50:05.222964+0000","last_unstale":"2023-12-26T21:50:05.225824+0000","last_undegraded":"2023-12-26T21:50:05.225824+0000","last_fullsized":"2023-12-26T21:50:05.225824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:42:16.214790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202129+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:39:05.981705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239297+0000","last_change":"2023-12-26T21:50:05.233495+0000","last_active":"2023-12-26T21:50:05.239297+0000","last_peered":"2023-12-26T21:50:05.239297+0000","last_clean":"2023-12-26T21:50:05.239297+0000","last_became_active":"2023-12-26T21:50:05.233051+0000","last_became_peered":"2023-12-26T21:50:05.233051+0000","last_unstale":"2023-12-26T21:50:05.239297+0000","last_undegraded":"2023-12-26T21:50:05.239297+0000","last_fullsized":"2023-12-26T21:50:05.239297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:49:53.896795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208240+0000","last_change":"2023-12-26T21:50:04.218089+0000","last_active":"2023-12-26T21:50:05.208240+0000","last_peered":"2023-12-26T21:50:05.208240+0000","last_clean":"2023-12-26T21:50:05.208240+0000","last_became_active":"2023-12-26T21:50:04.215530+0000","last_became_peered":"2023-12-26T21:50:04.215530+0000","last_unstale":"2023-12-26T21:50:05.208240+0000","last_undegraded":"2023-12-26T21:50:05.208240+0000","last_fullsized":"2023-12-26T21:50:05.208240+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:21.909145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225858+0000","last_change":"2023-12-26T21:50:05.224578+0000","last_active":"2023-12-26T21:50:05.225858+0000","last_peered":"2023-12-26T21:50:05.225858+0000","last_clean":"2023-12-26T21:50:05.225858+0000","last_became_active":"2023-12-26T21:50:05.223985+0000","last_became_peered":"2023-12-26T21:50:05.223985+0000","last_unstale":"2023-12-26T21:50:05.225858+0000","last_undegraded":"2023-12-26T21:50:05.225858+0000","last_fullsized":"2023-12-26T21:50:05.225858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:47:51.264287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.201997+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:05.549244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212760+0000","last_change":"2023-12-26T21:50:04.222797+0000","last_active":"2023-12-26T21:50:05.212760+0000","last_peered":"2023-12-26T21:50:05.212760+0000","last_clean":"2023-12-26T21:50:05.212760+0000","last_became_active":"2023-12-26T21:50:04.222490+0000","last_became_peered":"2023-12-26T21:50:04.222490+0000","last_unstale":"2023-12-26T21:50:05.212760+0000","last_undegraded":"2023-12-26T21:50:05.212760+0000","last_fullsized":"2023-12-26T21:50:05.212760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:17.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:50:03.188927+0000","last_change":"2023-12-26T21:50:03.202374+0000","last_active":"2023-12-26T21:50:03.188927+0000","last_peered":"2023-12-26T21:50:03.188927+0000","last_clean":"2023-12-26T21:50:03.188927+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:03.188927+0000","last_undegraded":"2023-12-26T21:50:03.188927+0000","last_fullsized":"2023-12-26T21:50:03.188927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:24:45.381424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208086+0000","last_change":"2023-12-26T21:50:04.220953+0000","last_active":"2023-12-26T21:50:05.208086+0000","last_peered":"2023-12-26T21:50:05.208086+0000","last_clean":"2023-12-26T21:50:05.208086+0000","last_became_active":"2023-12-26T21:50:04.220694+0000","last_became_peered":"2023-12-26T21:50:04.220694+0000","last_unstale":"2023-12-26T21:50:05.208086+0000","last_undegraded":"2023-12-26T21:50:05.208086+0000","last_fullsized":"2023-12-26T21:50:05.208086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:05.018373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:50:04.196855+0000","last_change":"2023-12-26T21:50:04.196855+0000","last_active":"2023-12-26T21:50:04.196855+0000","last_peered":"2023-12-26T21:50:04.196855+0000","last_clean":"2023-12-26T21:50:04.196855+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:50:04.196855+0000","last_undegraded":"2023-12-26T21:50:04.196855+0000","last_fullsized":"2023-12-26T21:50:04.196855+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245342+0000","last_change":"2023-12-26T21:50:04.226718+0000","last_active":"2023-12-26T21:50:05.245342+0000","last_peered":"2023-12-26T21:50:05.245342+0000","last_clean":"2023-12-26T21:50:05.245342+0000","last_became_active":"2023-12-26T21:50:04.226415+0000","last_became_peered":"2023-12-26T21:50:04.226415+0000","last_unstale":"2023-12-26T21:50:05.245342+0000","last_undegraded":"2023-12-26T21:50:05.245342+0000","last_fullsized":"2023-12-26T21:50:05.245342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:29.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"53'139","reported_seq":275,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-26T21:50:00.637985+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:50:00.637985+0000","last_peered":"2023-12-26T21:50:00.637985+0000","last_clean":"2023-12-26T21:50:00.637985+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:50:00.637985+0000","last_undegraded":"2023-12-26T21:50:00.637985+0000","last_fullsized":"2023-12-26T21:50:00.637985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1570,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542308,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":351691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.89100000000000001,"5min":0.94099999999999995,"15min":1.647},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":1,"5min":1,"15min":2.0150000000000001},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72599999999999998,"15min":0.70699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.016,"5min":1.048,"15min":1.1359999999999999},"last":0.436}]},{"osd":2,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.8640000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":0.95599999999999996,"15min":1.419},"last":0.63200000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":0.77200000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.026,"15min":1.327},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91900000000000004,"5min":1.008,"15min":1.393},"last":0.60699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":0.89400000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.371},"max":{"1min":0.84399999999999997,"5min":0.85199999999999998,"15min":0.86799999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":0.97599999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75700000000000001,"15min":0.745},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.499},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79100000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":0.92600000000000005,"15min":0.999},"last":0.68799999999999994}]}]},{"osd":6,"up_from":15,"seq":64424509603,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302072,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069768,"statfs":{"total":96636764160,"available":96327442432,"internally_reserved":0,"allocated":704512,"data_stored":325082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":1.0009999999999999,"15min":1.085},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.748,"15min":0.72199999999999998},"min":{"1min":0.56399999999999995,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":1.3380000000000001},"last":0.57299999999999995}]},{"osd":1,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.60299999999999998,"5min":0.44,"15min":0.41499999999999998},"max":{"1min":0.94699999999999995,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.71999999999999997},"min":{"1min":0.52300000000000002,"5min":0.41499999999999998,"15min":0.40799999999999997},"max":{"1min":0.86599999999999999,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.5}]},{"osd":2,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73599999999999999,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.46000000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.97099999999999997,"15min":1.288},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72699999999999998,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.497,"15min":0.45600000000000002},"max":{"1min":0.92000000000000004,"5min":0.93200000000000005,"15min":1.113},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.745,"15min":0.73399999999999999},"min":{"1min":0.54700000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":0.93600000000000005,"15min":1.569},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.77000000000000002,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.50600000000000001},"max":{"1min":0.96999999999999997,"5min":1.0129999999999999,"15min":1.3839999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.63300000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.377},"max":{"1min":0.81299999999999994,"5min":0.92700000000000005,"15min":1.137},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.65700000000000003},"min":{"1min":0.503,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93200000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.64500000000000002},"min":{"1min":0.42099999999999999,"5min":0.41899999999999998,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.875,"15min":1.1200000000000001},"last":0.754},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67000000000000004,"15min":0.70099999999999996},"min":{"1min":0.54800000000000004,"5min":0.46200000000000002,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":0.93600000000000005,"15min":1.296},"last":0.76700000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69399999999999995,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.437},"max":{"1min":0.89500000000000002,"5min":1.0920000000000001,"15min":1.51},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.998,"15min":1.2010000000000001},"last":0.79900000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509603,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306760,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065080,"statfs":{"total":96636764160,"available":96322641920,"internally_reserved":0,"allocated":1310720,"data_stored":921619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.309,"5min":0.307,"15min":0.255},"max":{"1min":0.65900000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.501,"15min":0.52900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.754,"15min":4.6600000000000001},"last":0.51600000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.49199999999999999,"15min":0.497},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.59699999999999998,"5min":1.0189999999999999,"15min":2.2869999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55400000000000005,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.29699999999999999},"max":{"1min":0.78800000000000003,"5min":0.93200000000000005,"15min":4.3639999999999999},"last":0.501}]},{"osd":3,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.38500000000000001},"max":{"1min":0.871,"5min":0.998,"15min":4.3259999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.33300000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":4.548},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92100000000000004,"5min":1.252,"15min":1.252},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.754,"15min":0.747},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.441},"max":{"1min":0.90200000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.51300000000000001},"max":{"1min":0.93700000000000006,"5min":1.2949999999999999,"15min":1.345},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.81799999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.82199999999999995,"15min":0.80100000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.51400000000000001},"max":{"1min":0.89500000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.83399999999999996,"15min":0.81100000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":0.90700000000000003,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0149999999999999,"5min":1.0980000000000001,"15min":1.1539999999999999},"last":0.746},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.94399999999999995,"5min":1.3620000000000001,"15min":1.3919999999999999},"last":0.55700000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509603,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302072,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069768,"statfs":{"total":96636764160,"available":96327442432,"internally_reserved":0,"allocated":704512,"data_stored":325082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42799999999999999,"15min":0.442},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":1.5640000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.48499999999999999,"15min":0.51200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60599999999999998,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":0.68300000000000005,"15min":2.012},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.51400000000000001,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56499999999999995},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.84499999999999997,"15min":4.2610000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.746,"15min":0.78000000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.85399999999999998,"5min":1.1539999999999999,"15min":4.423},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.93500000000000005,"5min":1.1000000000000001,"15min":1.6439999999999999},"last":0.622}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.78600000000000003,"15min":0.82099999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":1.119,"15min":4.3650000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.94599999999999995,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":1.103,"15min":1.611},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79300000000000004,"15min":0.81999999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.5},"max":{"1min":1.002,"5min":1.111,"15min":4.407},"last":0.83499999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.82099999999999995},"min":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":0.89600000000000002,"5min":1.145,"15min":4.5069999999999997},"last":0.69199999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302084,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069756,"statfs":{"total":96636764160,"available":96327430144,"internally_reserved":0,"allocated":716800,"data_stored":331251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.32900000000000001,"5min":0.27000000000000002,"15min":0.248},"max":{"1min":0.67000000000000004,"5min":0.85999999999999999,"15min":1.915},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":1.7729999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48499999999999999,"15min":0.496},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70899999999999996,"15min":1.952},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.30199999999999999},"max":{"1min":0.63100000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.44800000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.438,"5min":0.437,"15min":0.375},"max":{"1min":0.78700000000000003,"5min":0.83999999999999997,"15min":0.97099999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.98299999999999998,"5min":1.1180000000000001,"15min":1.284},"last":0.753},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.98399999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.92200000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.75800000000000001},"min":{"1min":0.53900000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.5029999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78000000000000003,"15min":0.77200000000000002},"min":{"1min":0.621,"5min":0.54200000000000004,"15min":0.503},"max":{"1min":0.89900000000000002,"5min":1.028,"15min":1.355},"last":0.88100000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.91200000000000003,"5min":1.032,"15min":1.7350000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80200000000000005,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.90200000000000002,"5min":1.0640000000000001,"15min":1.349},"last":0.873}]},{"osd":7,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.47799999999999998},"max":{"1min":0.95999999999999996,"5min":1.0680000000000001,"15min":1.841},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":1.004,"5min":1.091,"15min":1.294},"last":0.81999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509605,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306228,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065612,"statfs":{"total":96636764160,"available":96323186688,"internally_reserved":0,"allocated":765952,"data_stored":353193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.438,"15min":0.442},"min":{"1min":0.312,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.627,"5min":0.68100000000000005,"15min":0.91400000000000003},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35699999999999998,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.30399999999999999},"max":{"1min":0.65900000000000003,"5min":0.73499999999999999,"15min":0.77200000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.32400000000000001},"max":{"1min":0.66800000000000004,"5min":0.72499999999999998,"15min":0.99399999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38900000000000001},"max":{"1min":0.72099999999999997,"5min":0.83699999999999997,"15min":1.0820000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.82599999999999996,"15min":1.3580000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.499,"15min":0.45100000000000001},"max":{"1min":1.018,"5min":1.105,"15min":1.3779999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":1.1870000000000001,"15min":1.2949999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.47699999999999998},"max":{"1min":1,"5min":1.27,"15min":1.4610000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.78400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.45900000000000002},"max":{"1min":0.96399999999999997,"5min":0.997,"15min":1.9239999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.1950000000000001,"15min":1.5049999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.54100000000000004},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.579},"last":0.87}]},{"osd":7,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78600000000000003,"15min":0.80200000000000005},"min":{"1min":0.63800000000000001,"5min":0.51100000000000001,"15min":0.49299999999999999},"max":{"1min":1.036,"5min":1.0640000000000001,"15min":1.964},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.1499999999999999,"15min":1.6359999999999999},"last":0.92400000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069700,"statfs":{"total":96636764160,"available":96327372800,"internally_reserved":0,"allocated":774144,"data_stored":353285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":0.96499999999999997,"15min":2.1389999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.008,"15min":1.407},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.878,"5min":0.90500000000000003,"15min":1.879},"last":0.443},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.70799999999999996,"15min":0.72199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":2.097},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.72399999999999998},"min":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":0.94399999999999995,"15min":1.7809999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":1.6870000000000001},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70799999999999996,"15min":0.72599999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.83899999999999997,"5min":0.95999999999999996,"15min":1.956},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72899999999999998,"15min":0.74399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.83199999999999996,"5min":0.97999999999999998,"15min":2.0609999999999999},"last":0.69399999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58799999999999997,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.89800000000000002,"15min":2.2429999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.64200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77000000000000002,"5min":0.92600000000000005,"15min":2.194},"last":0.81799999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60199999999999998,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.878,"15min":2.1589999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64700000000000002,"15min":0.70299999999999996},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.874,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":0.98099999999999998,"15min":2.23},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.57199999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90100000000000002,"5min":0.92700000000000005,"15min":2.5169999999999999},"last":0.79900000000000004}]}]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1310720,"data_stored":921619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":0.995,"15min":2.3900000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70799999999999996,"15min":0.73599999999999999},"min":{"1min":0.499,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.874,"5min":1.0369999999999999,"15min":1.71},"last":0.81200000000000006}]},{"osd":1,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.51500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.61499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.4219999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72399999999999998,"15min":0.746},"min":{"1min":0.47899999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.622,"5min":0.435,"15min":0.435},"max":{"1min":0.81999999999999995,"5min":0.95699999999999996,"15min":1.214},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.74299999999999999},"min":{"1min":0.56399999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.4890000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.874,"5min":0.93899999999999995,"15min":1.1439999999999999},"last":0.82399999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":2.5070000000000001},"last":0.755},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59099999999999997,"15min":0.65600000000000003},"min":{"1min":0.55400000000000005,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90100000000000002,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.73099999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64300000000000002,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"max":{"1min":0.84299999999999997,"5min":0.90700000000000003,"15min":1.637},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.76500000000000001},"min":{"1min":0.65200000000000002,"5min":0.49099999999999999,"15min":0.438},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":2.5579999999999998},"last":0.85299999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67400000000000004,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.83299999999999996,"5min":0.96299999999999997,"15min":1.248},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68300000000000005,"15min":0.73299999999999998},"min":{"1min":0.56899999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83499999999999996,"5min":0.97199999999999998,"15min":1.5980000000000001},"last":0.88}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:50:10.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2023-12-26T21:50:10.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-12-26T21:50:10.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-12-26T21:50:10.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2023-12-26T21:50:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2023-12-26T21:50:10.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-12-26T21:50:10.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2023-12-26T21:50:10.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-12-26T21:50:10.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.202405+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:18:07.070074+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:50:10.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-12-26T21:50:10.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-12-26T21:50:10.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.204232+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T22:11:47.600963+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.203565+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:06:29.468613+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.202039+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T22:38:21.377473+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-12-26T21:50:10.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-12-26T21:50:10.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.205068+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:03:24.364623+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:50:10.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-12-26T21:50:10.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-12-26T21:50:10.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-12-26T21:50:10.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.199650+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:03:30.049843+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:50:10.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-12-26T21:50:10.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-12-26T21:50:10.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.202664+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:12:58.818709+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:50:10.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-12-26T21:50:10.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.207201+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:59:15.144868+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:50:10.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-12-26T21:50:10.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.202373+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:49:23.901091+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.205291+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:33:37.290110+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.204851+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:58:34.233954+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-12-26T21:50:10.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.203022+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:37:47.067537+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:50:10.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-12-26T21:50:10.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-12-26T21:50:10.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.202075+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:55:08.870277+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:50:10.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.202292+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:49:20.718326+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.202129+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:39:05.981705+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.201997+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:33:05.549244+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-12-26T21:50:10.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-12-26T21:50:10.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-12-26T21:50:10.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:50:03.188927+0000', 'last_change': '2023-12-26T21:50:03.202374+0000', 'last_active': '2023-12-26T21:50:03.188927+0000', 'last_peered': '2023-12-26T21:50:03.188927+0000', 'last_clean': '2023-12-26T21:50:03.188927+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:03.188927+0000', 'last_undegraded': '2023-12-26T21:50:03.188927+0000', 'last_fullsized': '2023-12-26T21:50:03.188927+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:03.188927+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:24:45.381424+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:50:10.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-12-26T21:50:10.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-12-26T21:50:10.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:50:04.196855+0000', 'last_change': '2023-12-26T21:50:04.196855+0000', 'last_active': '2023-12-26T21:50:04.196855+0000', 'last_peered': '2023-12-26T21:50:04.196855+0000', 'last_clean': '2023-12-26T21:50:04.196855+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:50:04.196855+0000', 'last_undegraded': '2023-12-26T21:50:04.196855+0000', 'last_fullsized': '2023-12-26T21:50:04.196855+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'last_clean_scrub_stamp': '2023-12-26T21:50:04.196855+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:50:10.518 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-26T21:50:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.726+0000 7fe8bbfff700 1 Processor -- start 2023-12-26T21:50:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.726+0000 7fe8bbfff700 1 -- start start 2023-12-26T21:50:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.728+0000 7fe8bbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8b40b19f0 0x7fe8b41620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.728+0000 7fe8bbfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b40caa10 con 0x7fe8b40c8820 2023-12-26T21:50:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.728+0000 7fe8bbfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b4157a80 con 0x7fe8b40c8bf0 2023-12-26T21:50:10.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.728+0000 7fe8bbfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b40cac50 con 0x7fe8b40b19f0 2023-12-26T21:50:10.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.728+0000 7fe8b9d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe8b40c8820 0x7fe8b40ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59528/0 (socket says 172.21.15.131:59528) 2023-12-26T21:50:10.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.728+0000 7fe8b9d9b700 1 -- 172.21.15.131:0/2249645510 learned_addr learned my addr 172.21.15.131:0/2249645510 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:10.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.728+0000 7fe8ba59c700 1 --2- 172.21.15.131:0/2249645510 >> v2:172.21.15.190:3300/0 conn(0x7fe8b40b19f0 0x7fe8b41620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.728+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689992993 0 0) 0x7fe8b0001110 con 0x7fe8b40c8820 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.728+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a0003570 con 0x7fe8b40c8820 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515624485 0 0) 0x7fe8b4157a80 con 0x7fe8b40c8bf0 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b0001110 con 0x7fe8b40c8bf0 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4062768267 0 0) 0x7fe8a0003570 con 0x7fe8b40c8820 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 >> v2:172.21.15.190:3300/0 conn(0x7fe8b40b19f0 msgr2=0x7fe8b41620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 --2- 172.21.15.131:0/2249645510 >> v2:172.21.15.190:3300/0 conn(0x7fe8b40b19f0 0x7fe8b41620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 >> v1:172.21.15.190:6789/0 conn(0x7fe8b40c8bf0 legacy=0x7fe8b40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40caf30 con 0x7fe8b40c8820 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fe8b0001600 con 0x7fe8b40c8820 2023-12-26T21:50:10.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 >> v1:172.21.15.131:6789/0 conn(0x7fe8b40c8820 legacy=0x7fe8b40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8a0003bf0 0x7fe8a0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0004810 0x7fe8a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a00051c0 0x7fe8a000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40cac50 con 0x7fe8a0003bf0 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a000c0b0 con 0x7fe8a0004810 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a000c270 con 0x7fe8a00051c0 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b9d9b700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8a0003bf0 0x7fe8a0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8b959a700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0004810 0x7fe8a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.729+0000 7fe8ba59c700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a00051c0 0x7fe8a000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b9d9b700 1 -- 172.21.15.131:0/2249645510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a00051c0 msgr2=0x7fe8a000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b9d9b700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a00051c0 0x7fe8a000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b9d9b700 1 -- 172.21.15.131:0/2249645510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0004810 msgr2=0x7fe8a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b9d9b700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0004810 0x7fe8a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b9d9b700 1 -- 172.21.15.131:0/2249645510 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a0003570 con 0x7fe8a0003bf0 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8ba59c700 1 -- 172.21.15.131:0/2249645510 reap_dead start 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b959a700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8a0004810 0x7fe8a0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b9d9b700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8a0003bf0 0x7fe8a0003fe0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000c440 tx=0x7fe8b0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c3226265021903 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000cd70 con 0x7fe8a0003bf0 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b0005920 con 0x7fe8a0003bf0 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8b8d99700 1 -- 172.21.15.131:0/2249645510 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0005ce0 con 0x7fe8a0003bf0 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8bbfff700 1 -- 172.21.15.131:0/2249645510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8a0003bf0 msgr2=0x7fe8a0003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.730+0000 7fe8bbfff700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8a0003bf0 0x7fe8a0003fe0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000c440 tx=0x7fe8b0007a00 comp rx=0 tx=0).stop 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.731+0000 7fe8bbfff700 1 -- 172.21.15.131:0/2249645510 shutdown_connections 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.731+0000 7fe8bbfff700 1 --2- 172.21.15.131:0/2249645510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8a0003bf0 0x7fe8a0003fe0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.731+0000 7fe8bbfff700 1 -- 172.21.15.131:0/2249645510 >> 172.21.15.131:0/2249645510 conn(0x7fe8b4068040 msgr2=0x7fe8b4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.731+0000 7fe8bbfff700 1 -- 172.21.15.131:0/2249645510 shutdown_connections 2023-12-26T21:50:10.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.731+0000 7fe8bbfff700 1 -- 172.21.15.131:0/2249645510 wait complete. 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.732+0000 7fe8bbfff700 1 Processor -- start 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.732+0000 7fe8bbfff700 1 -- start start 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.733+0000 7fe8bbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0003bf0 0x7fe8b414c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.733+0000 7fe8bbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b415c190 0x7fe8b415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.733+0000 7fe8bbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b4155ae0 0x7fe8b4155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.733+0000 7fe8bbfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b415cb40 con 0x7fe8b415c190 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.733+0000 7fe8bbfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4155210 con 0x7fe8b4155ae0 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.733+0000 7fe8bbfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4154670 con 0x7fe8a0003bf0 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.733+0000 7fe8b959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b415c190 0x7fe8b415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.733+0000 7fe8b9d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0003bf0 0x7fe8b414c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.733+0000 7fe8b959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b415c190 0x7fe8b415c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49854/0 (socket says 172.21.15.131:49854) 2023-12-26T21:50:10.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8b959a700 1 -- 172.21.15.131:0/303533679 learned_addr learned my addr 172.21.15.131:0/303533679 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8ba59c700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b4155ae0 0x7fe8b4155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8b959a700 1 -- 172.21.15.131:0/303533679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0003bf0 msgr2=0x7fe8b414c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8b959a700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0003bf0 0x7fe8b414c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8b959a700 1 -- 172.21.15.131:0/303533679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b4155ae0 msgr2=0x7fe8b4155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8b959a700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b4155ae0 0x7fe8b4155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8b959a700 1 -- 172.21.15.131:0/303533679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b41548e0 con 0x7fe8b415c190 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8b959a700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b415c190 0x7fe8b415c600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e870 tx=0x7fe8a4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8a9ffb700 1 -- 172.21.15.131:0/303533679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4017070 con 0x7fe8b415c190 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8a9ffb700 1 -- 172.21.15.131:0/303533679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a4006820 con 0x7fe8b415c190 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8a9ffb700 1 -- 172.21.15.131:0/303533679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4006bc0 con 0x7fe8b415c190 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.734+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b4154bc0 con 0x7fe8b415c190 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.735+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b414ca50 con 0x7fe8b415c190 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.736+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b4155380 con 0x7fe8b415c190 2023-12-26T21:50:10.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.736+0000 7fe8a9ffb700 1 -- 172.21.15.131:0/303533679 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe8a4007020 con 0x7fe8b415c190 2023-12-26T21:50:10.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.737+0000 7fe8a9ffb700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe898066f30 0x7fe8980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:10.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.737+0000 7fe8a9ffb700 1 -- 172.21.15.131:0/303533679 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe8a4084450 con 0x7fe8b415c190 2023-12-26T21:50:10.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.738+0000 7fe8b9d9b700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe898066f30 0x7fe8980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:10.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.738+0000 7fe8b9d9b700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe898066f30 0x7fe8980693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0007e20 tx=0x7fe8b0015000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:10.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:10.741+0000 7fe8a9ffb700 1 -- 172.21.15.131:0/303533679 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8a40514b0 con 0x7fe8b415c190 2023-12-26T21:50:11.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.007+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe8b40657e0 con 0x7fe8b415c190 2023-12-26T21:50:11.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.008+0000 7fe8a9ffb700 1 -- 172.21.15.131:0/303533679 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1176 (secure 0 0 0) 0x7fe8a4054ad0 con 0x7fe8b415c190 2023-12-26T21:50:11.008 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:11.008 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"ab603302-d88b-429a-be64-92b3c298adad","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":826,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":64,"num_osds":8,"num_up_osds":8,"osd_up_since":1703626596,"num_in_osds":8,"osd_in_since":1703626587,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":42},{"state_name":"active+clean","count":38},{"state_name":"creating+peering","count":17}],"num_pgs":97,"num_pools":3,"num_objects":15,"data_bytes":591938,"bytes_used":2480177152,"bytes_avail":770613936128,"bytes_total":773094113280,"unknown_pgs_ratio":0.4329896867275238,"inactive_pgs_ratio":0.17525772750377655,"write_bytes_sec":1671,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5739}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-12-26T21:39:07.341184+0000","services":{}},"progress_events":{}} 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe898066f30 msgr2=0x7fe8980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe898066f30 0x7fe8980693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0007e20 tx=0x7fe8b0015000 comp rx=0 tx=0).stop 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b415c190 msgr2=0x7fe8b415c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b415c190 0x7fe8b415c600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e870 tx=0x7fe8a4009040 comp rx=0 tx=0).stop 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 shutdown_connections 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b4155ae0 0x7fe8b4155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe898066f30 0x7fe8980693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8a0003bf0 0x7fe8b414c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 --2- 172.21.15.131:0/303533679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b415c190 0x7fe8b415c600 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 >> 172.21.15.131:0/303533679 conn(0x7fe8b4068040 msgr2=0x7fe8b406d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 shutdown_connections 2023-12-26T21:50:11.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.010+0000 7fe8bbfff700 1 -- 172.21.15.131:0/303533679 wait complete. 2023-12-26T21:50:11.029 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-26T21:50:11.029 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:50:11.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.218+0000 7fd988fff700 1 Processor -- start 2023-12-26T21:50:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.218+0000 7fd988fff700 1 -- start start 2023-12-26T21:50:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.219+0000 7fd988fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd9840cb320 0x7fd984069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.219+0000 7fd988fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840b0e60 0x7fd9840c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.219+0000 7fd988fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd984156940 con 0x7fd98406a1e0 2023-12-26T21:50:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.219+0000 7fd988fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9840c8cc0 con 0x7fd9840cb320 2023-12-26T21:50:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.219+0000 7fd988fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840ad700 con 0x7fd9840b0e60 2023-12-26T21:50:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.220+0000 7fd982d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840b0e60 0x7fd9840c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.220+0000 7fd982d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840b0e60 0x7fd9840c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:47658/0 (socket says 172.21.15.131:47658) 2023-12-26T21:50:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.220+0000 7fd982d9d700 1 -- 172.21.15.131:0/3545242810 learned_addr learned my addr 172.21.15.131:0/3545242810 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.220+0000 7fd98259c700 1 --2- 172.21.15.131:0/3545242810 >> v2:172.21.15.190:3300/0 conn(0x7fd9840cb320 0x7fd984069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.220+0000 7fd982d9d700 1 -- 172.21.15.131:0/3545242810 >> v2:172.21.15.190:3300/0 conn(0x7fd9840cb320 msgr2=0x7fd984069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.220+0000 7fd982d9d700 1 --2- 172.21.15.131:0/3545242810 >> v2:172.21.15.190:3300/0 conn(0x7fd9840cb320 0x7fd984069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.220+0000 7fd98159a700 1 -- 172.21.15.131:0/3545242810 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118276044 0 0) 0x7fd984156940 con 0x7fd98406a1e0 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.220+0000 7fd982d9d700 1 -- 172.21.15.131:0/3545242810 >> v1:172.21.15.190:6789/0 conn(0x7fd98406a1e0 legacy=0x7fd9840b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.220+0000 7fd982d9d700 1 -- 172.21.15.131:0/3545242810 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9840ae500 con 0x7fd9840b0e60 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.221+0000 7fd982d9d700 1 --2- 172.21.15.131:0/3545242810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840b0e60 0x7fd9840c9400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009f20 tx=0x7fd97c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a083f91beeb7927 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.221+0000 7fd98159a700 1 -- 172.21.15.131:0/3545242810 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00e070 con 0x7fd9840b0e60 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.221+0000 7fd98159a700 1 -- 172.21.15.131:0/3545242810 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c00b9d0 con 0x7fd9840b0e60 2023-12-26T21:50:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.221+0000 7fd98159a700 1 -- 172.21.15.131:0/3545242810 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0092e0 con 0x7fd9840b0e60 2023-12-26T21:50:11.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.221+0000 7fd988fff700 1 -- 172.21.15.131:0/3545242810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840b0e60 msgr2=0x7fd9840c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:11.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.221+0000 7fd988fff700 1 --2- 172.21.15.131:0/3545242810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840b0e60 0x7fd9840c9400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009f20 tx=0x7fd97c009840 comp rx=0 tx=0).stop 2023-12-26T21:50:11.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.222+0000 7fd988fff700 1 -- 172.21.15.131:0/3545242810 shutdown_connections 2023-12-26T21:50:11.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.222+0000 7fd988fff700 1 --2- 172.21.15.131:0/3545242810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840b0e60 0x7fd9840c9400 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.222+0000 7fd988fff700 1 --2- 172.21.15.131:0/3545242810 >> v2:172.21.15.190:3300/0 conn(0x7fd9840cb320 0x7fd984069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.222+0000 7fd988fff700 1 -- 172.21.15.131:0/3545242810 >> 172.21.15.131:0/3545242810 conn(0x7fd984068040 msgr2=0x7fd984069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:11.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.222+0000 7fd988fff700 1 -- 172.21.15.131:0/3545242810 shutdown_connections 2023-12-26T21:50:11.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.222+0000 7fd988fff700 1 -- 172.21.15.131:0/3545242810 wait complete. 2023-12-26T21:50:11.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.223+0000 7fd988fff700 1 Processor -- start 2023-12-26T21:50:11.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.223+0000 7fd988fff700 1 -- start start 2023-12-26T21:50:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.224+0000 7fd988fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd98406a1e0 0x7fd984153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.224+0000 7fd988fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9840b0e60 0x7fd9840c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.224+0000 7fd988fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840cb320 0x7fd9840c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.224+0000 7fd988fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840c0f90 con 0x7fd98406a1e0 2023-12-26T21:50:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.224+0000 7fd988fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840c8940 con 0x7fd9840b0e60 2023-12-26T21:50:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.224+0000 7fd988fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840c4430 con 0x7fd9840cb320 2023-12-26T21:50:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.224+0000 7fd98259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd98406a1e0 0x7fd984153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd98259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd98406a1e0 0x7fd984153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49862/0 (socket says 172.21.15.131:49862) 2023-12-26T21:50:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd98259c700 1 -- 172.21.15.131:0/3575200341 learned_addr learned my addr 172.21.15.131:0/3575200341 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd981d9b700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9840b0e60 0x7fd9840c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd982d9d700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840cb320 0x7fd9840c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd98259c700 1 -- 172.21.15.131:0/3575200341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840cb320 msgr2=0x7fd9840c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd98259c700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840cb320 0x7fd9840c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd98259c700 1 -- 172.21.15.131:0/3575200341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9840b0e60 msgr2=0x7fd9840c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd98259c700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9840b0e60 0x7fd9840c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd98259c700 1 -- 172.21.15.131:0/3575200341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c009ae0 con 0x7fd98406a1e0 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd98259c700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd98406a1e0 0x7fd984153270 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd97400e930 tx=0x7fd9740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd9727fc700 1 -- 172.21.15.131:0/3575200341 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974013680 con 0x7fd98406a1e0 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd9727fc700 1 -- 172.21.15.131:0/3575200341 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9740137e0 con 0x7fd98406a1e0 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9840c46f0 con 0x7fd98406a1e0 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.225+0000 7fd9727fc700 1 -- 172.21.15.131:0/3575200341 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974013b80 con 0x7fd98406a1e0 2023-12-26T21:50:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.226+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9841537b0 con 0x7fd98406a1e0 2023-12-26T21:50:11.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.226+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9840657e0 con 0x7fd98406a1e0 2023-12-26T21:50:11.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.228+0000 7fd9727fc700 1 -- 172.21.15.131:0/3575200341 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd97400c3e0 con 0x7fd98406a1e0 2023-12-26T21:50:11.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.228+0000 7fd9727fc700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd964066f30 0x7fd9640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:11.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.228+0000 7fd9727fc700 1 -- 172.21.15.131:0/3575200341 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd974084750 con 0x7fd98406a1e0 2023-12-26T21:50:11.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.229+0000 7fd981d9b700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd964066f30 0x7fd9640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:11.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.229+0000 7fd981d9b700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd964066f30 0x7fd9640693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9840ae930 tx=0x7fd978007d30 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:11.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.232+0000 7fd9727fc700 1 -- 172.21.15.131:0/3575200341 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd974051850 con 0x7fd98406a1e0 2023-12-26T21:50:11.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.424+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9840c4b40 con 0x7fd964066f30 2023-12-26T21:50:11.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.433+0000 7fd9727fc700 1 -- 172.21.15.131:0/3575200341 <== mgr.5437 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288898 (secure 0 0 0) 0x7fd9840c4b40 con 0x7fd964066f30 2023-12-26T21:50:11.437 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.437+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd964066f30 msgr2=0x7fd9640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.437+0000 7fd988fff700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd964066f30 0x7fd9640693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9840ae930 tx=0x7fd978007d30 comp rx=0 tx=0).stop 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.437+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd98406a1e0 msgr2=0x7fd984153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.437+0000 7fd988fff700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd98406a1e0 0x7fd984153270 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd97400e930 tx=0x7fd9740091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.437+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 shutdown_connections 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.437+0000 7fd988fff700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fd964066f30 0x7fd9640693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.437+0000 7fd988fff700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9840cb320 0x7fd9840c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.437+0000 7fd988fff700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd98406a1e0 0x7fd984153270 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.437+0000 7fd988fff700 1 --2- 172.21.15.131:0/3575200341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9840b0e60 0x7fd9840c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.438+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 >> 172.21.15.131:0/3575200341 conn(0x7fd984068040 msgr2=0x7fd98406e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:11.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.438+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 shutdown_connections 2023-12-26T21:50:11.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:11.438+0000 7fd988fff700 1 -- 172.21.15.131:0/3575200341 wait complete. 2023-12-26T21:50:11.441 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:50:11.455 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-26T21:50:10.713814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422256,"kb_used_data":7088,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752552464,"statfs":{"total":773094113280,"available":770613723136,"internally_reserved":0,"allocated":7258112,"data_stored":3985300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.509955"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.226991+0000","last_change":"2023-12-26T21:50:05.226991+0000","last_active":"2023-12-26T21:50:05.226991+0000","last_peered":"2023-12-26T21:50:05.226991+0000","last_clean":"2023-12-26T21:50:05.226991+0000","last_became_active":"2023-12-26T21:50:05.226766+0000","last_became_peered":"2023-12-26T21:50:05.226766+0000","last_unstale":"2023-12-26T21:50:05.226991+0000","last_undegraded":"2023-12-26T21:50:05.226991+0000","last_fullsized":"2023-12-26T21:50:05.226991+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:30:01.379377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239587+0000","last_change":"2023-12-26T21:50:05.230491+0000","last_active":"2023-12-26T21:50:05.239587+0000","last_peered":"2023-12-26T21:50:05.239587+0000","last_clean":"2023-12-26T21:50:05.239587+0000","last_became_active":"2023-12-26T21:50:05.230277+0000","last_became_peered":"2023-12-26T21:50:05.230277+0000","last_unstale":"2023-12-26T21:50:05.239587+0000","last_undegraded":"2023-12-26T21:50:05.239587+0000","last_fullsized":"2023-12-26T21:50:05.239587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:40:13.844989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233235+0000","last_change":"2023-12-26T21:50:05.226988+0000","last_active":"2023-12-26T21:50:05.233235+0000","last_peered":"2023-12-26T21:50:05.233235+0000","last_clean":"2023-12-26T21:50:05.233235+0000","last_became_active":"2023-12-26T21:50:05.226680+0000","last_became_peered":"2023-12-26T21:50:05.226680+0000","last_unstale":"2023-12-26T21:50:05.233235+0000","last_undegraded":"2023-12-26T21:50:05.233235+0000","last_fullsized":"2023-12-26T21:50:05.233235+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:57:23.669558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238873+0000","last_change":"2023-12-26T21:50:05.236391+0000","last_active":"2023-12-26T21:50:05.238873+0000","last_peered":"2023-12-26T21:50:05.238873+0000","last_clean":"2023-12-26T21:50:05.238873+0000","last_became_active":"2023-12-26T21:50:05.235021+0000","last_became_peered":"2023-12-26T21:50:05.235021+0000","last_unstale":"2023-12-26T21:50:05.238873+0000","last_undegraded":"2023-12-26T21:50:05.238873+0000","last_fullsized":"2023-12-26T21:50:05.238873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:29.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225390+0000","last_change":"2023-12-26T21:50:05.225390+0000","last_active":"2023-12-26T21:50:05.225390+0000","last_peered":"2023-12-26T21:50:05.225390+0000","last_clean":"2023-12-26T21:50:05.225390+0000","last_became_active":"2023-12-26T21:50:05.225134+0000","last_became_peered":"2023-12-26T21:50:05.225134+0000","last_unstale":"2023-12-26T21:50:05.225390+0000","last_undegraded":"2023-12-26T21:50:05.225390+0000","last_fullsized":"2023-12-26T21:50:05.225390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:30.275494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233316+0000","last_change":"2023-12-26T21:50:05.230103+0000","last_active":"2023-12-26T21:50:05.233316+0000","last_peered":"2023-12-26T21:50:05.233316+0000","last_clean":"2023-12-26T21:50:05.233316+0000","last_became_active":"2023-12-26T21:50:05.229812+0000","last_became_peered":"2023-12-26T21:50:05.229812+0000","last_unstale":"2023-12-26T21:50:05.233316+0000","last_undegraded":"2023-12-26T21:50:05.233316+0000","last_fullsized":"2023-12-26T21:50:05.233316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:49:21.726254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239194+0000","last_change":"2023-12-26T21:50:05.233540+0000","last_active":"2023-12-26T21:50:05.239194+0000","last_peered":"2023-12-26T21:50:05.239194+0000","last_clean":"2023-12-26T21:50:05.239194+0000","last_became_active":"2023-12-26T21:50:05.233189+0000","last_became_peered":"2023-12-26T21:50:05.233189+0000","last_unstale":"2023-12-26T21:50:05.239194+0000","last_undegraded":"2023-12-26T21:50:05.239194+0000","last_fullsized":"2023-12-26T21:50:05.239194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:38.099699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233097+0000","last_change":"2023-12-26T21:50:05.227196+0000","last_active":"2023-12-26T21:50:05.233097+0000","last_peered":"2023-12-26T21:50:05.233097+0000","last_clean":"2023-12-26T21:50:05.233097+0000","last_became_active":"2023-12-26T21:50:05.226763+0000","last_became_peered":"2023-12-26T21:50:05.226763+0000","last_unstale":"2023-12-26T21:50:05.233097+0000","last_undegraded":"2023-12-26T21:50:05.233097+0000","last_fullsized":"2023-12-26T21:50:05.233097+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:27:06.557823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233389+0000","last_change":"2023-12-26T21:50:05.230175+0000","last_active":"2023-12-26T21:50:05.233389+0000","last_peered":"2023-12-26T21:50:05.233389+0000","last_clean":"2023-12-26T21:50:05.233389+0000","last_became_active":"2023-12-26T21:50:05.228975+0000","last_became_peered":"2023-12-26T21:50:05.228975+0000","last_unstale":"2023-12-26T21:50:05.233389+0000","last_undegraded":"2023-12-26T21:50:05.233389+0000","last_fullsized":"2023-12-26T21:50:05.233389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:53:36.056936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.231885+0000","last_change":"2023-12-26T21:50:05.226840+0000","last_active":"2023-12-26T21:50:05.231885+0000","last_peered":"2023-12-26T21:50:05.231885+0000","last_clean":"2023-12-26T21:50:05.231885+0000","last_became_active":"2023-12-26T21:50:05.226580+0000","last_became_peered":"2023-12-26T21:50:05.226580+0000","last_unstale":"2023-12-26T21:50:05.231885+0000","last_undegraded":"2023-12-26T21:50:05.231885+0000","last_fullsized":"2023-12-26T21:50:05.231885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:15:04.988046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233197+0000","last_change":"2023-12-26T21:50:05.225387+0000","last_active":"2023-12-26T21:50:05.233197+0000","last_peered":"2023-12-26T21:50:05.233197+0000","last_clean":"2023-12-26T21:50:05.233197+0000","last_became_active":"2023-12-26T21:50:05.225093+0000","last_became_peered":"2023-12-26T21:50:05.225093+0000","last_unstale":"2023-12-26T21:50:05.233197+0000","last_undegraded":"2023-12-26T21:50:05.233197+0000","last_fullsized":"2023-12-26T21:50:05.233197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:13.703711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.232530+0000","last_change":"2023-12-26T21:50:05.227503+0000","last_active":"2023-12-26T21:50:05.232530+0000","last_peered":"2023-12-26T21:50:05.232530+0000","last_clean":"2023-12-26T21:50:05.232530+0000","last_became_active":"2023-12-26T21:50:05.227241+0000","last_became_peered":"2023-12-26T21:50:05.227241+0000","last_unstale":"2023-12-26T21:50:05.232530+0000","last_undegraded":"2023-12-26T21:50:05.232530+0000","last_fullsized":"2023-12-26T21:50:05.232530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:21.980776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.236944+0000","last_change":"2023-12-26T21:50:05.236267+0000","last_active":"2023-12-26T21:50:05.236944+0000","last_peered":"2023-12-26T21:50:05.236944+0000","last_clean":"2023-12-26T21:50:05.236944+0000","last_became_active":"2023-12-26T21:50:05.235023+0000","last_became_peered":"2023-12-26T21:50:05.235023+0000","last_unstale":"2023-12-26T21:50:05.236944+0000","last_undegraded":"2023-12-26T21:50:05.236944+0000","last_fullsized":"2023-12-26T21:50:05.236944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:24.909019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234063+0000","last_change":"2023-12-26T21:50:05.228775+0000","last_active":"2023-12-26T21:50:05.234063+0000","last_peered":"2023-12-26T21:50:05.234063+0000","last_clean":"2023-12-26T21:50:05.234063+0000","last_became_active":"2023-12-26T21:50:05.228476+0000","last_became_peered":"2023-12-26T21:50:05.228476+0000","last_unstale":"2023-12-26T21:50:05.234063+0000","last_undegraded":"2023-12-26T21:50:05.234063+0000","last_fullsized":"2023-12-26T21:50:05.234063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:41:03.707784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.236826+0000","last_change":"2023-12-26T21:50:05.233900+0000","last_active":"2023-12-26T21:50:05.236826+0000","last_peered":"2023-12-26T21:50:05.236826+0000","last_clean":"2023-12-26T21:50:05.236826+0000","last_became_active":"2023-12-26T21:50:05.229572+0000","last_became_peered":"2023-12-26T21:50:05.229572+0000","last_unstale":"2023-12-26T21:50:05.236826+0000","last_undegraded":"2023-12-26T21:50:05.236826+0000","last_fullsized":"2023-12-26T21:50:05.236826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:50.842679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234891+0000","last_change":"2023-12-26T21:50:05.234891+0000","last_active":"2023-12-26T21:50:05.234891+0000","last_peered":"2023-12-26T21:50:05.234891+0000","last_clean":"2023-12-26T21:50:05.234891+0000","last_became_active":"2023-12-26T21:50:05.234633+0000","last_became_peered":"2023-12-26T21:50:05.234633+0000","last_unstale":"2023-12-26T21:50:05.234891+0000","last_undegraded":"2023-12-26T21:50:05.234891+0000","last_fullsized":"2023-12-26T21:50:05.234891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:27:06.557823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.235582+0000","last_change":"2023-12-26T21:50:05.228752+0000","last_active":"2023-12-26T21:50:05.235582+0000","last_peered":"2023-12-26T21:50:05.235582+0000","last_clean":"2023-12-26T21:50:05.235582+0000","last_became_active":"2023-12-26T21:50:05.227870+0000","last_became_peered":"2023-12-26T21:50:05.227870+0000","last_unstale":"2023-12-26T21:50:05.235582+0000","last_undegraded":"2023-12-26T21:50:05.235582+0000","last_fullsized":"2023-12-26T21:50:05.235582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:20.901688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238765+0000","last_change":"2023-12-26T21:50:05.232522+0000","last_active":"2023-12-26T21:50:05.238765+0000","last_peered":"2023-12-26T21:50:05.238765+0000","last_clean":"2023-12-26T21:50:05.238765+0000","last_became_active":"2023-12-26T21:50:05.232087+0000","last_became_peered":"2023-12-26T21:50:05.232087+0000","last_unstale":"2023-12-26T21:50:05.238765+0000","last_undegraded":"2023-12-26T21:50:05.238765+0000","last_fullsized":"2023-12-26T21:50:05.238765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:59:16.152796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225163+0000","last_change":"2023-12-26T21:50:05.225163+0000","last_active":"2023-12-26T21:50:05.225163+0000","last_peered":"2023-12-26T21:50:05.225163+0000","last_clean":"2023-12-26T21:50:05.225163+0000","last_became_active":"2023-12-26T21:50:05.224418+0000","last_became_peered":"2023-12-26T21:50:05.224418+0000","last_unstale":"2023-12-26T21:50:05.225163+0000","last_undegraded":"2023-12-26T21:50:05.225163+0000","last_fullsized":"2023-12-26T21:50:05.225163+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:29.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225805+0000","last_change":"2023-12-26T21:50:05.221852+0000","last_active":"2023-12-26T21:50:05.225805+0000","last_peered":"2023-12-26T21:50:05.225805+0000","last_clean":"2023-12-26T21:50:05.225805+0000","last_became_active":"2023-12-26T21:50:05.221574+0000","last_became_peered":"2023-12-26T21:50:05.221574+0000","last_unstale":"2023-12-26T21:50:05.225805+0000","last_undegraded":"2023-12-26T21:50:05.225805+0000","last_fullsized":"2023-12-26T21:50:05.225805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:45:09.667824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233949+0000","last_change":"2023-12-26T21:50:05.229424+0000","last_active":"2023-12-26T21:50:05.233949+0000","last_peered":"2023-12-26T21:50:05.233949+0000","last_clean":"2023-12-26T21:50:05.233949+0000","last_became_active":"2023-12-26T21:50:05.228038+0000","last_became_peered":"2023-12-26T21:50:05.228038+0000","last_unstale":"2023-12-26T21:50:05.233949+0000","last_undegraded":"2023-12-26T21:50:05.233949+0000","last_fullsized":"2023-12-26T21:50:05.233949+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:31:28.704242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216704+0000","last_change":"2023-12-26T21:50:05.213544+0000","last_active":"2023-12-26T21:50:05.216704+0000","last_peered":"2023-12-26T21:50:05.216704+0000","last_clean":"2023-12-26T21:50:05.216704+0000","last_became_active":"2023-12-26T21:50:05.212622+0000","last_became_peered":"2023-12-26T21:50:05.212622+0000","last_unstale":"2023-12-26T21:50:05.216704+0000","last_undegraded":"2023-12-26T21:50:05.216704+0000","last_fullsized":"2023-12-26T21:50:05.216704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:45.329023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238621+0000","last_change":"2023-12-26T21:50:05.231337+0000","last_active":"2023-12-26T21:50:05.238621+0000","last_peered":"2023-12-26T21:50:05.238621+0000","last_clean":"2023-12-26T21:50:05.238621+0000","last_became_active":"2023-12-26T21:50:05.231031+0000","last_became_peered":"2023-12-26T21:50:05.231031+0000","last_unstale":"2023-12-26T21:50:05.238621+0000","last_undegraded":"2023-12-26T21:50:05.238621+0000","last_fullsized":"2023-12-26T21:50:05.238621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:51:34.322061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239773+0000","last_change":"2023-12-26T21:50:05.235997+0000","last_active":"2023-12-26T21:50:05.239773+0000","last_peered":"2023-12-26T21:50:05.239773+0000","last_clean":"2023-12-26T21:50:05.239773+0000","last_became_active":"2023-12-26T21:50:05.235732+0000","last_became_peered":"2023-12-26T21:50:05.235732+0000","last_unstale":"2023-12-26T21:50:05.239773+0000","last_undegraded":"2023-12-26T21:50:05.239773+0000","last_fullsized":"2023-12-26T21:50:05.239773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:44.898807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.223936+0000","last_change":"2023-12-26T21:50:05.223936+0000","last_active":"2023-12-26T21:50:05.223936+0000","last_peered":"2023-12-26T21:50:05.223936+0000","last_clean":"2023-12-26T21:50:05.223936+0000","last_became_active":"2023-12-26T21:50:05.223684+0000","last_became_peered":"2023-12-26T21:50:05.223684+0000","last_unstale":"2023-12-26T21:50:05.223936+0000","last_undegraded":"2023-12-26T21:50:05.223936+0000","last_fullsized":"2023-12-26T21:50:05.223936+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:13.703711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234525+0000","last_change":"2023-12-26T21:50:05.232194+0000","last_active":"2023-12-26T21:50:05.234525+0000","last_peered":"2023-12-26T21:50:05.234525+0000","last_clean":"2023-12-26T21:50:05.234525+0000","last_became_active":"2023-12-26T21:50:05.231928+0000","last_became_peered":"2023-12-26T21:50:05.231928+0000","last_unstale":"2023-12-26T21:50:05.234525+0000","last_undegraded":"2023-12-26T21:50:05.234525+0000","last_fullsized":"2023-12-26T21:50:05.234525+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:18.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239316+0000","last_change":"2023-12-26T21:50:05.233798+0000","last_active":"2023-12-26T21:50:05.239316+0000","last_peered":"2023-12-26T21:50:05.239316+0000","last_clean":"2023-12-26T21:50:05.239316+0000","last_became_active":"2023-12-26T21:50:05.229125+0000","last_became_peered":"2023-12-26T21:50:05.229125+0000","last_unstale":"2023-12-26T21:50:05.239316+0000","last_undegraded":"2023-12-26T21:50:05.239316+0000","last_fullsized":"2023-12-26T21:50:05.239316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:53.114423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234135+0000","last_change":"2023-12-26T21:50:05.224391+0000","last_active":"2023-12-26T21:50:05.234135+0000","last_peered":"2023-12-26T21:50:05.234135+0000","last_clean":"2023-12-26T21:50:05.234135+0000","last_became_active":"2023-12-26T21:50:05.224146+0000","last_became_peered":"2023-12-26T21:50:05.224146+0000","last_unstale":"2023-12-26T21:50:05.234135+0000","last_undegraded":"2023-12-26T21:50:05.234135+0000","last_fullsized":"2023-12-26T21:50:05.234135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:37.819740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.230803+0000","last_change":"2023-12-26T21:50:05.230803+0000","last_active":"2023-12-26T21:50:05.230803+0000","last_peered":"2023-12-26T21:50:05.230803+0000","last_clean":"2023-12-26T21:50:05.230803+0000","last_became_active":"2023-12-26T21:50:05.228633+0000","last_became_peered":"2023-12-26T21:50:05.228633+0000","last_unstale":"2023-12-26T21:50:05.230803+0000","last_undegraded":"2023-12-26T21:50:05.230803+0000","last_fullsized":"2023-12-26T21:50:05.230803+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:50.842679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.224439+0000","last_change":"2023-12-26T21:50:05.221854+0000","last_active":"2023-12-26T21:50:05.224439+0000","last_peered":"2023-12-26T21:50:05.224439+0000","last_clean":"2023-12-26T21:50:05.224439+0000","last_became_active":"2023-12-26T21:50:05.221566+0000","last_became_peered":"2023-12-26T21:50:05.221566+0000","last_unstale":"2023-12-26T21:50:05.224439+0000","last_undegraded":"2023-12-26T21:50:05.224439+0000","last_fullsized":"2023-12-26T21:50:05.224439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:59.144750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.232861+0000","last_change":"2023-12-26T21:50:05.228319+0000","last_active":"2023-12-26T21:50:05.232861+0000","last_peered":"2023-12-26T21:50:05.232861+0000","last_clean":"2023-12-26T21:50:05.232861+0000","last_became_active":"2023-12-26T21:50:05.228033+0000","last_became_peered":"2023-12-26T21:50:05.228033+0000","last_unstale":"2023-12-26T21:50:05.232861+0000","last_undegraded":"2023-12-26T21:50:05.232861+0000","last_fullsized":"2023-12-26T21:50:05.232861+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:53:52.642285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233073+0000","last_change":"2023-12-26T21:50:05.227074+0000","last_active":"2023-12-26T21:50:05.233073+0000","last_peered":"2023-12-26T21:50:05.233073+0000","last_clean":"2023-12-26T21:50:05.233073+0000","last_became_active":"2023-12-26T21:50:05.226556+0000","last_became_peered":"2023-12-26T21:50:05.226556+0000","last_unstale":"2023-12-26T21:50:05.233073+0000","last_undegraded":"2023-12-26T21:50:05.233073+0000","last_fullsized":"2023-12-26T21:50:05.233073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:53:31.233090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227740+0000","last_change":"2023-12-26T21:50:05.225269+0000","last_active":"2023-12-26T21:50:05.227740+0000","last_peered":"2023-12-26T21:50:05.227740+0000","last_clean":"2023-12-26T21:50:05.227740+0000","last_became_active":"2023-12-26T21:50:05.224997+0000","last_became_peered":"2023-12-26T21:50:05.224997+0000","last_unstale":"2023-12-26T21:50:05.227740+0000","last_undegraded":"2023-12-26T21:50:05.227740+0000","last_fullsized":"2023-12-26T21:50:05.227740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:03.075265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"63'4","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248580+0000","last_change":"2023-12-26T21:50:04.227245+0000","last_active":"2023-12-26T21:50:05.248580+0000","last_peered":"2023-12-26T21:50:05.248580+0000","last_clean":"2023-12-26T21:50:05.248580+0000","last_became_active":"2023-12-26T21:50:04.226971+0000","last_became_peered":"2023-12-26T21:50:04.226971+0000","last_unstale":"2023-12-26T21:50:05.248580+0000","last_undegraded":"2023-12-26T21:50:05.248580+0000","last_fullsized":"2023-12-26T21:50:05.248580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:03:39.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.224999+0000","last_change":"2023-12-26T21:50:05.224999+0000","last_active":"2023-12-26T21:50:05.224999+0000","last_peered":"2023-12-26T21:50:05.224999+0000","last_clean":"2023-12-26T21:50:05.224999+0000","last_became_active":"2023-12-26T21:50:05.224758+0000","last_became_peered":"2023-12-26T21:50:05.224758+0000","last_unstale":"2023-12-26T21:50:05.224999+0000","last_undegraded":"2023-12-26T21:50:05.224999+0000","last_fullsized":"2023-12-26T21:50:05.224999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:18.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244190+0000","last_change":"2023-12-26T21:50:04.223739+0000","last_active":"2023-12-26T21:50:05.244190+0000","last_peered":"2023-12-26T21:50:05.244190+0000","last_clean":"2023-12-26T21:50:05.244190+0000","last_became_active":"2023-12-26T21:50:04.222755+0000","last_became_peered":"2023-12-26T21:50:04.222755+0000","last_unstale":"2023-12-26T21:50:05.244190+0000","last_undegraded":"2023-12-26T21:50:05.244190+0000","last_fullsized":"2023-12-26T21:50:05.244190+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:21.699131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239633+0000","last_change":"2023-12-26T21:50:05.229901+0000","last_active":"2023-12-26T21:50:05.239633+0000","last_peered":"2023-12-26T21:50:05.239633+0000","last_clean":"2023-12-26T21:50:05.239633+0000","last_became_active":"2023-12-26T21:50:05.229440+0000","last_became_peered":"2023-12-26T21:50:05.229440+0000","last_unstale":"2023-12-26T21:50:05.239633+0000","last_undegraded":"2023-12-26T21:50:05.239633+0000","last_fullsized":"2023-12-26T21:50:05.239633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:19.072262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248189+0000","last_change":"2023-12-26T21:50:04.226558+0000","last_active":"2023-12-26T21:50:05.248189+0000","last_peered":"2023-12-26T21:50:05.248189+0000","last_clean":"2023-12-26T21:50:05.248189+0000","last_became_active":"2023-12-26T21:50:04.226307+0000","last_became_peered":"2023-12-26T21:50:04.226307+0000","last_unstale":"2023-12-26T21:50:05.248189+0000","last_undegraded":"2023-12-26T21:50:05.248189+0000","last_fullsized":"2023-12-26T21:50:05.248189+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:59:27.515630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.240655+0000","last_change":"2023-12-26T21:50:05.237965+0000","last_active":"2023-12-26T21:50:05.240655+0000","last_peered":"2023-12-26T21:50:05.240655+0000","last_clean":"2023-12-26T21:50:05.240655+0000","last_became_active":"2023-12-26T21:50:05.237619+0000","last_became_peered":"2023-12-26T21:50:05.237619+0000","last_unstale":"2023-12-26T21:50:05.240655+0000","last_undegraded":"2023-12-26T21:50:05.240655+0000","last_fullsized":"2023-12-26T21:50:05.240655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:30.275494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244999+0000","last_change":"2023-12-26T21:50:04.223118+0000","last_active":"2023-12-26T21:50:05.244999+0000","last_peered":"2023-12-26T21:50:05.244999+0000","last_clean":"2023-12-26T21:50:05.244999+0000","last_became_active":"2023-12-26T21:50:04.222720+0000","last_became_peered":"2023-12-26T21:50:04.222720+0000","last_unstale":"2023-12-26T21:50:05.244999+0000","last_undegraded":"2023-12-26T21:50:05.244999+0000","last_fullsized":"2023-12-26T21:50:05.244999+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:38:24.121296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.222331+0000","last_change":"2023-12-26T21:50:05.220983+0000","last_active":"2023-12-26T21:50:05.222331+0000","last_peered":"2023-12-26T21:50:05.222331+0000","last_clean":"2023-12-26T21:50:05.222331+0000","last_became_active":"2023-12-26T21:50:05.220694+0000","last_became_peered":"2023-12-26T21:50:05.220694+0000","last_unstale":"2023-12-26T21:50:05.222331+0000","last_undegraded":"2023-12-26T21:50:05.222331+0000","last_fullsized":"2023-12-26T21:50:05.222331+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:29:18.312367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.241850+0000","last_change":"2023-12-26T21:50:04.225617+0000","last_active":"2023-12-26T21:50:05.241850+0000","last_peered":"2023-12-26T21:50:05.241850+0000","last_clean":"2023-12-26T21:50:05.241850+0000","last_became_active":"2023-12-26T21:50:04.224621+0000","last_became_peered":"2023-12-26T21:50:04.224621+0000","last_unstale":"2023-12-26T21:50:05.241850+0000","last_undegraded":"2023-12-26T21:50:05.241850+0000","last_fullsized":"2023-12-26T21:50:05.241850+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:09:43.053987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238883+0000","last_change":"2023-12-26T21:50:05.235706+0000","last_active":"2023-12-26T21:50:05.238883+0000","last_peered":"2023-12-26T21:50:05.238883+0000","last_clean":"2023-12-26T21:50:05.238883+0000","last_became_active":"2023-12-26T21:50:05.235416+0000","last_became_peered":"2023-12-26T21:50:05.235416+0000","last_unstale":"2023-12-26T21:50:05.238883+0000","last_undegraded":"2023-12-26T21:50:05.238883+0000","last_fullsized":"2023-12-26T21:50:05.238883+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:20.767760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248231+0000","last_change":"2023-12-26T21:50:04.225964+0000","last_active":"2023-12-26T21:50:05.248231+0000","last_peered":"2023-12-26T21:50:05.248231+0000","last_clean":"2023-12-26T21:50:05.248231+0000","last_became_active":"2023-12-26T21:50:04.225653+0000","last_became_peered":"2023-12-26T21:50:04.225653+0000","last_unstale":"2023-12-26T21:50:05.248231+0000","last_undegraded":"2023-12-26T21:50:05.248231+0000","last_fullsized":"2023-12-26T21:50:05.248231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:49.834751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.226468+0000","last_change":"2023-12-26T21:50:05.223489+0000","last_active":"2023-12-26T21:50:05.226468+0000","last_peered":"2023-12-26T21:50:05.226468+0000","last_clean":"2023-12-26T21:50:05.226468+0000","last_became_active":"2023-12-26T21:50:05.222774+0000","last_became_peered":"2023-12-26T21:50:05.222774+0000","last_unstale":"2023-12-26T21:50:05.226468+0000","last_undegraded":"2023-12-26T21:50:05.226468+0000","last_fullsized":"2023-12-26T21:50:05.226468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:50.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-12-26T21:50:04.236599+0000","last_change":"2023-12-26T21:50:04.232558+0000","last_active":"2023-12-26T21:50:04.236599+0000","last_peered":"2023-12-26T21:50:04.236599+0000","last_clean":"2023-12-26T21:50:04.236599+0000","last_became_active":"2023-12-26T21:50:04.232303+0000","last_became_peered":"2023-12-26T21:50:04.232303+0000","last_unstale":"2023-12-26T21:50:04.236599+0000","last_undegraded":"2023-12-26T21:50:04.236599+0000","last_fullsized":"2023-12-26T21:50:04.236599+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:13:48.174203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239174+0000","last_change":"2023-12-26T21:50:05.233433+0000","last_active":"2023-12-26T21:50:05.239174+0000","last_peered":"2023-12-26T21:50:05.239174+0000","last_clean":"2023-12-26T21:50:05.239174+0000","last_became_active":"2023-12-26T21:50:05.228341+0000","last_became_peered":"2023-12-26T21:50:05.228341+0000","last_unstale":"2023-12-26T21:50:05.239174+0000","last_undegraded":"2023-12-26T21:50:05.239174+0000","last_fullsized":"2023-12-26T21:50:05.239174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:58:54.385865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214676+0000","last_change":"2023-12-26T21:50:04.221595+0000","last_active":"2023-12-26T21:50:05.214676+0000","last_peered":"2023-12-26T21:50:05.214676+0000","last_clean":"2023-12-26T21:50:05.214676+0000","last_became_active":"2023-12-26T21:50:04.221337+0000","last_became_peered":"2023-12-26T21:50:04.221337+0000","last_unstale":"2023-12-26T21:50:05.214676+0000","last_undegraded":"2023-12-26T21:50:05.214676+0000","last_fullsized":"2023-12-26T21:50:05.214676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:38:20.838693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.224182+0000","last_change":"2023-12-26T21:50:05.217647+0000","last_active":"2023-12-26T21:50:05.224182+0000","last_peered":"2023-12-26T21:50:05.224182+0000","last_clean":"2023-12-26T21:50:05.224182+0000","last_became_active":"2023-12-26T21:50:05.217273+0000","last_became_peered":"2023-12-26T21:50:05.217273+0000","last_unstale":"2023-12-26T21:50:05.224182+0000","last_undegraded":"2023-12-26T21:50:05.224182+0000","last_fullsized":"2023-12-26T21:50:05.224182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:30.476540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.209456+0000","last_change":"2023-12-26T21:50:04.224494+0000","last_active":"2023-12-26T21:50:05.209456+0000","last_peered":"2023-12-26T21:50:05.209456+0000","last_clean":"2023-12-26T21:50:05.209456+0000","last_became_active":"2023-12-26T21:50:04.221693+0000","last_became_peered":"2023-12-26T21:50:04.221693+0000","last_unstale":"2023-12-26T21:50:05.209456+0000","last_undegraded":"2023-12-26T21:50:05.209456+0000","last_fullsized":"2023-12-26T21:50:05.209456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:45:52.041010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233265+0000","last_change":"2023-12-26T21:50:05.225928+0000","last_active":"2023-12-26T21:50:05.233265+0000","last_peered":"2023-12-26T21:50:05.233265+0000","last_clean":"2023-12-26T21:50:05.233265+0000","last_became_active":"2023-12-26T21:50:05.225635+0000","last_became_peered":"2023-12-26T21:50:05.225635+0000","last_unstale":"2023-12-26T21:50:05.233265+0000","last_undegraded":"2023-12-26T21:50:05.233265+0000","last_fullsized":"2023-12-26T21:50:05.233265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:11:48.608891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214009+0000","last_change":"2023-12-26T21:50:04.229086+0000","last_active":"2023-12-26T21:50:05.214009+0000","last_peered":"2023-12-26T21:50:05.214009+0000","last_clean":"2023-12-26T21:50:05.214009+0000","last_became_active":"2023-12-26T21:50:04.228808+0000","last_became_peered":"2023-12-26T21:50:04.228808+0000","last_unstale":"2023-12-26T21:50:05.214009+0000","last_undegraded":"2023-12-26T21:50:05.214009+0000","last_fullsized":"2023-12-26T21:50:05.214009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:10:52.365673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238852+0000","last_change":"2023-12-26T21:50:05.234519+0000","last_active":"2023-12-26T21:50:05.238852+0000","last_peered":"2023-12-26T21:50:05.238852+0000","last_clean":"2023-12-26T21:50:05.238852+0000","last_became_active":"2023-12-26T21:50:05.230388+0000","last_became_peered":"2023-12-26T21:50:05.230388+0000","last_unstale":"2023-12-26T21:50:05.238852+0000","last_undegraded":"2023-12-26T21:50:05.238852+0000","last_fullsized":"2023-12-26T21:50:05.238852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:20:59.601587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"63'3","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250647+0000","last_change":"2023-12-26T21:50:04.227264+0000","last_active":"2023-12-26T21:50:05.250647+0000","last_peered":"2023-12-26T21:50:05.250647+0000","last_clean":"2023-12-26T21:50:05.250647+0000","last_became_active":"2023-12-26T21:50:04.226632+0000","last_became_peered":"2023-12-26T21:50:04.226632+0000","last_unstale":"2023-12-26T21:50:05.250647+0000","last_undegraded":"2023-12-26T21:50:05.250647+0000","last_fullsized":"2023-12-26T21:50:05.250647+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:53.639515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238522+0000","last_change":"2023-12-26T21:50:05.233319+0000","last_active":"2023-12-26T21:50:05.238522+0000","last_peered":"2023-12-26T21:50:05.238522+0000","last_clean":"2023-12-26T21:50:05.238522+0000","last_became_active":"2023-12-26T21:50:05.232699+0000","last_became_peered":"2023-12-26T21:50:05.232699+0000","last_unstale":"2023-12-26T21:50:05.238522+0000","last_undegraded":"2023-12-26T21:50:05.238522+0000","last_fullsized":"2023-12-26T21:50:05.238522+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:02:49.364640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250467+0000","last_change":"2023-12-26T21:50:04.227027+0000","last_active":"2023-12-26T21:50:05.250467+0000","last_peered":"2023-12-26T21:50:05.250467+0000","last_clean":"2023-12-26T21:50:05.250467+0000","last_became_active":"2023-12-26T21:50:04.226702+0000","last_became_peered":"2023-12-26T21:50:04.226702+0000","last_unstale":"2023-12-26T21:50:05.250467+0000","last_undegraded":"2023-12-26T21:50:05.250467+0000","last_fullsized":"2023-12-26T21:50:05.250467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:43:44.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.237691+0000","last_change":"2023-12-26T21:50:05.232476+0000","last_active":"2023-12-26T21:50:05.237691+0000","last_peered":"2023-12-26T21:50:05.237691+0000","last_clean":"2023-12-26T21:50:05.237691+0000","last_became_active":"2023-12-26T21:50:05.232146+0000","last_became_peered":"2023-12-26T21:50:05.232146+0000","last_unstale":"2023-12-26T21:50:05.237691+0000","last_undegraded":"2023-12-26T21:50:05.237691+0000","last_fullsized":"2023-12-26T21:50:05.237691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:59:28.523558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245546+0000","last_change":"2023-12-26T21:50:04.222842+0000","last_active":"2023-12-26T21:50:05.245546+0000","last_peered":"2023-12-26T21:50:05.245546+0000","last_clean":"2023-12-26T21:50:05.245546+0000","last_became_active":"2023-12-26T21:50:04.222585+0000","last_became_peered":"2023-12-26T21:50:04.222585+0000","last_unstale":"2023-12-26T21:50:05.245546+0000","last_undegraded":"2023-12-26T21:50:05.245546+0000","last_fullsized":"2023-12-26T21:50:05.245546+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:33:01.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227858+0000","last_change":"2023-12-26T21:50:05.225230+0000","last_active":"2023-12-26T21:50:05.227858+0000","last_peered":"2023-12-26T21:50:05.227858+0000","last_clean":"2023-12-26T21:50:05.227858+0000","last_became_active":"2023-12-26T21:50:05.224946+0000","last_became_peered":"2023-12-26T21:50:05.224946+0000","last_unstale":"2023-12-26T21:50:05.227858+0000","last_undegraded":"2023-12-26T21:50:05.227858+0000","last_fullsized":"2023-12-26T21:50:05.227858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:03:31.057770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-12-26T21:50:04.230453+0000","last_change":"2023-12-26T21:50:04.227438+0000","last_active":"2023-12-26T21:50:04.230453+0000","last_peered":"2023-12-26T21:50:04.230453+0000","last_clean":"2023-12-26T21:50:04.230453+0000","last_became_active":"2023-12-26T21:50:04.226130+0000","last_became_peered":"2023-12-26T21:50:04.226130+0000","last_unstale":"2023-12-26T21:50:04.230453+0000","last_undegraded":"2023-12-26T21:50:04.230453+0000","last_fullsized":"2023-12-26T21:50:04.230453+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:00:03.077231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.219854+0000","last_change":"2023-12-26T21:50:05.217771+0000","last_active":"2023-12-26T21:50:05.219854+0000","last_peered":"2023-12-26T21:50:05.219854+0000","last_clean":"2023-12-26T21:50:05.219854+0000","last_became_active":"2023-12-26T21:50:05.217485+0000","last_became_peered":"2023-12-26T21:50:05.217485+0000","last_unstale":"2023-12-26T21:50:05.219854+0000","last_undegraded":"2023-12-26T21:50:05.219854+0000","last_fullsized":"2023-12-26T21:50:05.219854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:54.647443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212183+0000","last_change":"2023-12-26T21:50:04.228778+0000","last_active":"2023-12-26T21:50:05.212183+0000","last_peered":"2023-12-26T21:50:05.212183+0000","last_clean":"2023-12-26T21:50:05.212183+0000","last_became_active":"2023-12-26T21:50:04.228516+0000","last_became_peered":"2023-12-26T21:50:04.228516+0000","last_unstale":"2023-12-26T21:50:05.212183+0000","last_undegraded":"2023-12-26T21:50:05.212183+0000","last_fullsized":"2023-12-26T21:50:05.212183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:26.294398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.236011+0000","last_change":"2023-12-26T21:50:05.228184+0000","last_active":"2023-12-26T21:50:05.236011+0000","last_peered":"2023-12-26T21:50:05.236011+0000","last_clean":"2023-12-26T21:50:05.236011+0000","last_became_active":"2023-12-26T21:50:05.227940+0000","last_became_peered":"2023-12-26T21:50:05.227940+0000","last_unstale":"2023-12-26T21:50:05.236011+0000","last_undegraded":"2023-12-26T21:50:05.236011+0000","last_fullsized":"2023-12-26T21:50:05.236011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:21:03.619005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250174+0000","last_change":"2023-12-26T21:50:04.224690+0000","last_active":"2023-12-26T21:50:05.250174+0000","last_peered":"2023-12-26T21:50:05.250174+0000","last_clean":"2023-12-26T21:50:05.250174+0000","last_became_active":"2023-12-26T21:50:04.223460+0000","last_became_peered":"2023-12-26T21:50:04.223460+0000","last_unstale":"2023-12-26T21:50:05.250174+0000","last_undegraded":"2023-12-26T21:50:05.250174+0000","last_fullsized":"2023-12-26T21:50:05.250174+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:01.981864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216840+0000","last_change":"2023-12-26T21:50:05.213363+0000","last_active":"2023-12-26T21:50:05.216840+0000","last_peered":"2023-12-26T21:50:05.216840+0000","last_clean":"2023-12-26T21:50:05.216840+0000","last_became_active":"2023-12-26T21:50:05.212965+0000","last_became_peered":"2023-12-26T21:50:05.212965+0000","last_unstale":"2023-12-26T21:50:05.216840+0000","last_undegraded":"2023-12-26T21:50:05.216840+0000","last_fullsized":"2023-12-26T21:50:05.216840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:01:27.770799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.241206+0000","last_change":"2023-12-26T21:50:04.224381+0000","last_active":"2023-12-26T21:50:05.241206+0000","last_peered":"2023-12-26T21:50:05.241206+0000","last_clean":"2023-12-26T21:50:05.241206+0000","last_became_active":"2023-12-26T21:50:04.224082+0000","last_became_peered":"2023-12-26T21:50:04.224082+0000","last_unstale":"2023-12-26T21:50:05.241206+0000","last_undegraded":"2023-12-26T21:50:05.241206+0000","last_fullsized":"2023-12-26T21:50:05.241206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:49:28.351623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.223902+0000","last_change":"2023-12-26T21:50:05.222959+0000","last_active":"2023-12-26T21:50:05.223902+0000","last_peered":"2023-12-26T21:50:05.223902+0000","last_clean":"2023-12-26T21:50:05.223902+0000","last_became_active":"2023-12-26T21:50:05.222618+0000","last_became_peered":"2023-12-26T21:50:05.222618+0000","last_unstale":"2023-12-26T21:50:05.223902+0000","last_undegraded":"2023-12-26T21:50:05.223902+0000","last_fullsized":"2023-12-26T21:50:05.223902+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:13.568346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245765+0000","last_change":"2023-12-26T21:50:04.219680+0000","last_active":"2023-12-26T21:50:05.245765+0000","last_peered":"2023-12-26T21:50:05.245765+0000","last_clean":"2023-12-26T21:50:05.245765+0000","last_became_active":"2023-12-26T21:50:04.219425+0000","last_became_peered":"2023-12-26T21:50:04.219425+0000","last_unstale":"2023-12-26T21:50:05.245765+0000","last_undegraded":"2023-12-26T21:50:05.245765+0000","last_fullsized":"2023-12-26T21:50:05.245765+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:50:10.822244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239646+0000","last_change":"2023-12-26T21:50:05.236257+0000","last_active":"2023-12-26T21:50:05.239646+0000","last_peered":"2023-12-26T21:50:05.239646+0000","last_clean":"2023-12-26T21:50:05.239646+0000","last_became_active":"2023-12-26T21:50:05.235988+0000","last_became_peered":"2023-12-26T21:50:05.235988+0000","last_unstale":"2023-12-26T21:50:05.239646+0000","last_undegraded":"2023-12-26T21:50:05.239646+0000","last_fullsized":"2023-12-26T21:50:05.239646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:43:47.213090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.247822+0000","last_change":"2023-12-26T21:50:04.224911+0000","last_active":"2023-12-26T21:50:05.247822+0000","last_peered":"2023-12-26T21:50:05.247822+0000","last_clean":"2023-12-26T21:50:05.247822+0000","last_became_active":"2023-12-26T21:50:04.223726+0000","last_became_peered":"2023-12-26T21:50:04.223726+0000","last_unstale":"2023-12-26T21:50:05.247822+0000","last_undegraded":"2023-12-26T21:50:05.247822+0000","last_fullsized":"2023-12-26T21:50:05.247822+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:40:40.318772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.230157+0000","last_change":"2023-12-26T21:50:05.223489+0000","last_active":"2023-12-26T21:50:05.230157+0000","last_peered":"2023-12-26T21:50:05.230157+0000","last_clean":"2023-12-26T21:50:05.230157+0000","last_became_active":"2023-12-26T21:50:05.223200+0000","last_became_peered":"2023-12-26T21:50:05.223200+0000","last_unstale":"2023-12-26T21:50:05.230157+0000","last_undegraded":"2023-12-26T21:50:05.230157+0000","last_fullsized":"2023-12-26T21:50:05.230157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:37:48.075465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212178+0000","last_change":"2023-12-26T21:50:04.229701+0000","last_active":"2023-12-26T21:50:05.212178+0000","last_peered":"2023-12-26T21:50:05.212178+0000","last_clean":"2023-12-26T21:50:05.212178+0000","last_became_active":"2023-12-26T21:50:04.229390+0000","last_became_peered":"2023-12-26T21:50:04.229390+0000","last_unstale":"2023-12-26T21:50:05.212178+0000","last_undegraded":"2023-12-26T21:50:05.212178+0000","last_fullsized":"2023-12-26T21:50:05.212178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:16:28.242388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225878+0000","last_change":"2023-12-26T21:50:05.224746+0000","last_active":"2023-12-26T21:50:05.225878+0000","last_peered":"2023-12-26T21:50:05.225878+0000","last_clean":"2023-12-26T21:50:05.225878+0000","last_became_active":"2023-12-26T21:50:05.224302+0000","last_became_peered":"2023-12-26T21:50:05.224302+0000","last_unstale":"2023-12-26T21:50:05.225878+0000","last_undegraded":"2023-12-26T21:50:05.225878+0000","last_fullsized":"2023-12-26T21:50:05.225878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:44.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248205+0000","last_change":"2023-12-26T21:50:04.226030+0000","last_active":"2023-12-26T21:50:05.248205+0000","last_peered":"2023-12-26T21:50:05.248205+0000","last_clean":"2023-12-26T21:50:05.248205+0000","last_became_active":"2023-12-26T21:50:04.225766+0000","last_became_peered":"2023-12-26T21:50:04.225766+0000","last_unstale":"2023-12-26T21:50:05.248205+0000","last_undegraded":"2023-12-26T21:50:05.248205+0000","last_fullsized":"2023-12-26T21:50:05.248205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:58.136823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.229691+0000","last_change":"2023-12-26T21:50:05.224442+0000","last_active":"2023-12-26T21:50:05.229691+0000","last_peered":"2023-12-26T21:50:05.229691+0000","last_clean":"2023-12-26T21:50:05.229691+0000","last_became_active":"2023-12-26T21:50:05.224152+0000","last_became_peered":"2023-12-26T21:50:05.224152+0000","last_unstale":"2023-12-26T21:50:05.229691+0000","last_undegraded":"2023-12-26T21:50:05.229691+0000","last_fullsized":"2023-12-26T21:50:05.229691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:44:29.772129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-12-26T21:50:04.236562+0000","last_change":"2023-12-26T21:50:04.231701+0000","last_active":"2023-12-26T21:50:04.236562+0000","last_peered":"2023-12-26T21:50:04.236562+0000","last_clean":"2023-12-26T21:50:04.236562+0000","last_became_active":"2023-12-26T21:50:04.231270+0000","last_became_peered":"2023-12-26T21:50:04.231270+0000","last_unstale":"2023-12-26T21:50:04.236562+0000","last_undegraded":"2023-12-26T21:50:04.236562+0000","last_fullsized":"2023-12-26T21:50:04.236562+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:41:02.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233022+0000","last_change":"2023-12-26T21:50:05.226864+0000","last_active":"2023-12-26T21:50:05.233022+0000","last_peered":"2023-12-26T21:50:05.233022+0000","last_clean":"2023-12-26T21:50:05.233022+0000","last_became_active":"2023-12-26T21:50:05.226485+0000","last_became_peered":"2023-12-26T21:50:05.226485+0000","last_unstale":"2023-12-26T21:50:05.233022+0000","last_undegraded":"2023-12-26T21:50:05.233022+0000","last_fullsized":"2023-12-26T21:50:05.233022+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:29:22.091541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216148+0000","last_change":"2023-12-26T21:50:04.225794+0000","last_active":"2023-12-26T21:50:05.216148+0000","last_peered":"2023-12-26T21:50:05.216148+0000","last_clean":"2023-12-26T21:50:05.216148+0000","last_became_active":"2023-12-26T21:50:04.225535+0000","last_became_peered":"2023-12-26T21:50:04.225535+0000","last_unstale":"2023-12-26T21:50:05.216148+0000","last_undegraded":"2023-12-26T21:50:05.216148+0000","last_fullsized":"2023-12-26T21:50:05.216148+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:01:54.466108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234604+0000","last_change":"2023-12-26T21:50:05.231979+0000","last_active":"2023-12-26T21:50:05.234604+0000","last_peered":"2023-12-26T21:50:05.234604+0000","last_clean":"2023-12-26T21:50:05.234604+0000","last_became_active":"2023-12-26T21:50:05.228858+0000","last_became_peered":"2023-12-26T21:50:05.228858+0000","last_unstale":"2023-12-26T21:50:05.234604+0000","last_undegraded":"2023-12-26T21:50:05.234604+0000","last_fullsized":"2023-12-26T21:50:05.234604+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:00.075762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.213208+0000","last_change":"2023-12-26T21:50:04.224397+0000","last_active":"2023-12-26T21:50:05.213208+0000","last_peered":"2023-12-26T21:50:05.213208+0000","last_clean":"2023-12-26T21:50:05.213208+0000","last_became_active":"2023-12-26T21:50:04.224078+0000","last_became_peered":"2023-12-26T21:50:04.224078+0000","last_unstale":"2023-12-26T21:50:05.213208+0000","last_undegraded":"2023-12-26T21:50:05.213208+0000","last_fullsized":"2023-12-26T21:50:05.213208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:28.511389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234103+0000","last_change":"2023-12-26T21:50:05.227159+0000","last_active":"2023-12-26T21:50:05.234103+0000","last_peered":"2023-12-26T21:50:05.234103+0000","last_clean":"2023-12-26T21:50:05.234103+0000","last_became_active":"2023-12-26T21:50:05.226887+0000","last_became_peered":"2023-12-26T21:50:05.226887+0000","last_unstale":"2023-12-26T21:50:05.234103+0000","last_undegraded":"2023-12-26T21:50:05.234103+0000","last_fullsized":"2023-12-26T21:50:05.234103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:01:34.210345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.221410+0000","last_change":"2023-12-26T21:50:04.230019+0000","last_active":"2023-12-26T21:50:05.221410+0000","last_peered":"2023-12-26T21:50:05.221410+0000","last_clean":"2023-12-26T21:50:05.221410+0000","last_became_active":"2023-12-26T21:50:04.229584+0000","last_became_peered":"2023-12-26T21:50:04.229584+0000","last_unstale":"2023-12-26T21:50:05.221410+0000","last_undegraded":"2023-12-26T21:50:05.221410+0000","last_fullsized":"2023-12-26T21:50:05.221410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:44:28.764202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225824+0000","last_change":"2023-12-26T21:50:05.223267+0000","last_active":"2023-12-26T21:50:05.225824+0000","last_peered":"2023-12-26T21:50:05.225824+0000","last_clean":"2023-12-26T21:50:05.225824+0000","last_became_active":"2023-12-26T21:50:05.222964+0000","last_became_peered":"2023-12-26T21:50:05.222964+0000","last_unstale":"2023-12-26T21:50:05.225824+0000","last_undegraded":"2023-12-26T21:50:05.225824+0000","last_fullsized":"2023-12-26T21:50:05.225824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:42:16.214790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-12-26T21:50:04.234006+0000","last_change":"2023-12-26T21:50:04.231179+0000","last_active":"2023-12-26T21:50:04.234006+0000","last_peered":"2023-12-26T21:50:04.234006+0000","last_clean":"2023-12-26T21:50:04.234006+0000","last_became_active":"2023-12-26T21:50:04.230888+0000","last_became_peered":"2023-12-26T21:50:04.230888+0000","last_unstale":"2023-12-26T21:50:04.234006+0000","last_undegraded":"2023-12-26T21:50:04.234006+0000","last_fullsized":"2023-12-26T21:50:04.234006+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:30:14.453888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239297+0000","last_change":"2023-12-26T21:50:05.233495+0000","last_active":"2023-12-26T21:50:05.239297+0000","last_peered":"2023-12-26T21:50:05.239297+0000","last_clean":"2023-12-26T21:50:05.239297+0000","last_became_active":"2023-12-26T21:50:05.233051+0000","last_became_peered":"2023-12-26T21:50:05.233051+0000","last_unstale":"2023-12-26T21:50:05.239297+0000","last_undegraded":"2023-12-26T21:50:05.239297+0000","last_fullsized":"2023-12-26T21:50:05.239297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:49:53.896795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208240+0000","last_change":"2023-12-26T21:50:04.218089+0000","last_active":"2023-12-26T21:50:05.208240+0000","last_peered":"2023-12-26T21:50:05.208240+0000","last_clean":"2023-12-26T21:50:05.208240+0000","last_became_active":"2023-12-26T21:50:04.215530+0000","last_became_peered":"2023-12-26T21:50:04.215530+0000","last_unstale":"2023-12-26T21:50:05.208240+0000","last_undegraded":"2023-12-26T21:50:05.208240+0000","last_fullsized":"2023-12-26T21:50:05.208240+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:21.909145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225858+0000","last_change":"2023-12-26T21:50:05.224578+0000","last_active":"2023-12-26T21:50:05.225858+0000","last_peered":"2023-12-26T21:50:05.225858+0000","last_clean":"2023-12-26T21:50:05.225858+0000","last_became_active":"2023-12-26T21:50:05.223985+0000","last_became_peered":"2023-12-26T21:50:05.223985+0000","last_unstale":"2023-12-26T21:50:05.225858+0000","last_undegraded":"2023-12-26T21:50:05.225858+0000","last_fullsized":"2023-12-26T21:50:05.225858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:47:51.264287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-12-26T21:50:04.236552+0000","last_change":"2023-12-26T21:50:04.231436+0000","last_active":"2023-12-26T21:50:04.236552+0000","last_peered":"2023-12-26T21:50:04.236552+0000","last_clean":"2023-12-26T21:50:04.236552+0000","last_became_active":"2023-12-26T21:50:04.231168+0000","last_became_peered":"2023-12-26T21:50:04.231168+0000","last_unstale":"2023-12-26T21:50:04.236552+0000","last_undegraded":"2023-12-26T21:50:04.236552+0000","last_fullsized":"2023-12-26T21:50:04.236552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:12:19.198601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233969+0000","last_change":"2023-12-26T21:50:05.230590+0000","last_active":"2023-12-26T21:50:05.233969+0000","last_peered":"2023-12-26T21:50:05.233969+0000","last_clean":"2023-12-26T21:50:05.233969+0000","last_became_active":"2023-12-26T21:50:05.229413+0000","last_became_peered":"2023-12-26T21:50:05.229413+0000","last_unstale":"2023-12-26T21:50:05.233969+0000","last_undegraded":"2023-12-26T21:50:05.233969+0000","last_fullsized":"2023-12-26T21:50:05.233969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:22.707058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212760+0000","last_change":"2023-12-26T21:50:04.222797+0000","last_active":"2023-12-26T21:50:05.212760+0000","last_peered":"2023-12-26T21:50:05.212760+0000","last_clean":"2023-12-26T21:50:05.212760+0000","last_became_active":"2023-12-26T21:50:04.222490+0000","last_became_peered":"2023-12-26T21:50:04.222490+0000","last_unstale":"2023-12-26T21:50:05.212760+0000","last_undegraded":"2023-12-26T21:50:05.212760+0000","last_fullsized":"2023-12-26T21:50:05.212760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:17.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239591+0000","last_change":"2023-12-26T21:50:05.233040+0000","last_active":"2023-12-26T21:50:05.239591+0000","last_peered":"2023-12-26T21:50:05.239591+0000","last_clean":"2023-12-26T21:50:05.239591+0000","last_became_active":"2023-12-26T21:50:05.232801+0000","last_became_peered":"2023-12-26T21:50:05.232801+0000","last_unstale":"2023-12-26T21:50:05.239591+0000","last_undegraded":"2023-12-26T21:50:05.239591+0000","last_fullsized":"2023-12-26T21:50:05.239591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:03:40.240620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212142+0000","last_change":"2023-12-26T21:50:04.225898+0000","last_active":"2023-12-26T21:50:05.212142+0000","last_peered":"2023-12-26T21:50:05.212142+0000","last_clean":"2023-12-26T21:50:05.212142+0000","last_became_active":"2023-12-26T21:50:04.225376+0000","last_became_peered":"2023-12-26T21:50:04.225376+0000","last_unstale":"2023-12-26T21:50:05.212142+0000","last_undegraded":"2023-12-26T21:50:05.212142+0000","last_fullsized":"2023-12-26T21:50:05.212142+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:57:22.661631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239829+0000","last_change":"2023-12-26T21:50:05.229746+0000","last_active":"2023-12-26T21:50:05.239829+0000","last_peered":"2023-12-26T21:50:05.239829+0000","last_clean":"2023-12-26T21:50:05.239829+0000","last_became_active":"2023-12-26T21:50:05.229483+0000","last_became_peered":"2023-12-26T21:50:05.229483+0000","last_unstale":"2023-12-26T21:50:05.239829+0000","last_undegraded":"2023-12-26T21:50:05.239829+0000","last_fullsized":"2023-12-26T21:50:05.239829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:01:55.474035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208086+0000","last_change":"2023-12-26T21:50:04.220953+0000","last_active":"2023-12-26T21:50:05.208086+0000","last_peered":"2023-12-26T21:50:05.208086+0000","last_clean":"2023-12-26T21:50:05.208086+0000","last_became_active":"2023-12-26T21:50:04.220694+0000","last_became_peered":"2023-12-26T21:50:04.220694+0000","last_unstale":"2023-12-26T21:50:05.208086+0000","last_undegraded":"2023-12-26T21:50:05.208086+0000","last_fullsized":"2023-12-26T21:50:05.208086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:05.018373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238819+0000","last_change":"2023-12-26T21:50:05.235997+0000","last_active":"2023-12-26T21:50:05.238819+0000","last_peered":"2023-12-26T21:50:05.238819+0000","last_clean":"2023-12-26T21:50:05.238819+0000","last_became_active":"2023-12-26T21:50:05.234830+0000","last_became_peered":"2023-12-26T21:50:05.234830+0000","last_unstale":"2023-12-26T21:50:05.238819+0000","last_undegraded":"2023-12-26T21:50:05.238819+0000","last_fullsized":"2023-12-26T21:50:05.238819+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:30:01.379377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245342+0000","last_change":"2023-12-26T21:50:04.226718+0000","last_active":"2023-12-26T21:50:05.245342+0000","last_peered":"2023-12-26T21:50:05.245342+0000","last_clean":"2023-12-26T21:50:05.245342+0000","last_became_active":"2023-12-26T21:50:04.226415+0000","last_became_peered":"2023-12-26T21:50:04.226415+0000","last_unstale":"2023-12-26T21:50:05.245342+0000","last_undegraded":"2023-12-26T21:50:05.245342+0000","last_fullsized":"2023-12-26T21:50:05.245342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:29.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"53'139","reported_seq":278,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208912+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:50:05.208912+0000","last_peered":"2023-12-26T21:50:05.208912+0000","last_clean":"2023-12-26T21:50:05.208912+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:50:05.208912+0000","last_undegraded":"2023-12-26T21:50:05.208912+0000","last_fullsized":"2023-12-26T21:50:05.208912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542308,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":351691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.89100000000000001,"5min":0.94099999999999995,"15min":1.647},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":1,"5min":1,"15min":2.0150000000000001},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72599999999999998,"15min":0.70699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.016,"5min":1.048,"15min":1.1359999999999999},"last":0.436}]},{"osd":2,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.8640000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":0.95599999999999996,"15min":1.419},"last":0.63200000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":0.77200000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.026,"15min":1.327},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91900000000000004,"5min":1.008,"15min":1.393},"last":0.60699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":0.89400000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.371},"max":{"1min":0.84399999999999997,"5min":0.85199999999999998,"15min":0.86799999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":0.97599999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75700000000000001,"15min":0.745},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.499},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79100000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":0.92600000000000005,"15min":0.999},"last":0.68799999999999994}]}]},{"osd":6,"up_from":15,"seq":64424509604,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302116,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069724,"statfs":{"total":96636764160,"available":96327397376,"internally_reserved":0,"allocated":749568,"data_stored":345593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.70799999999999996},"min":{"1min":0.60299999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":1.0009999999999999,"15min":1.085},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.748,"15min":0.72199999999999998},"min":{"1min":0.56399999999999995,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":1.3380000000000001},"last":0.59299999999999997}]},{"osd":1,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.60299999999999998,"5min":0.44,"15min":0.41499999999999998},"max":{"1min":0.94699999999999995,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73299999999999998,"15min":0.71999999999999997},"min":{"1min":0.52300000000000002,"5min":0.41499999999999998,"15min":0.40799999999999997},"max":{"1min":0.86599999999999999,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73599999999999999,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.46000000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.97099999999999997,"15min":1.288},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.72699999999999998,"15min":0.71699999999999997},"min":{"1min":0.56299999999999994,"5min":0.497,"15min":0.45600000000000002},"max":{"1min":0.92000000000000004,"5min":0.93200000000000005,"15min":1.113},"last":0.55500000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.745,"15min":0.73399999999999999},"min":{"1min":0.54700000000000004,"5min":0.47399999999999998,"15min":0.46300000000000002},"max":{"1min":0.92700000000000005,"5min":0.93600000000000005,"15min":1.569},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.77000000000000002,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.50600000000000001},"max":{"1min":0.96999999999999997,"5min":1.0129999999999999,"15min":1.3839999999999999},"last":0.76800000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.63300000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.377},"max":{"1min":0.81299999999999994,"5min":0.92700000000000005,"15min":1.137},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.65700000000000003},"min":{"1min":0.503,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93200000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.879}]},{"osd":5,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.64500000000000002},"min":{"1min":0.42099999999999999,"5min":0.41899999999999998,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.875,"15min":1.1200000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67000000000000004,"15min":0.70099999999999996},"min":{"1min":0.54800000000000004,"5min":0.46200000000000002,"15min":0.42599999999999999},"max":{"1min":0.85699999999999998,"5min":0.93600000000000005,"15min":1.296},"last":0.81799999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:49:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69399999999999995,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.437},"max":{"1min":0.89500000000000002,"5min":1.0920000000000001,"15min":1.51},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.998,"15min":1.2010000000000001},"last":0.86399999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509604,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306800,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065040,"statfs":{"total":96636764160,"available":96322600960,"internally_reserved":0,"allocated":1351680,"data_stored":942095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.309,"5min":0.307,"15min":0.255},"max":{"1min":0.65900000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.501,"15min":0.52900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.754,"15min":4.6600000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.49199999999999999,"15min":0.497},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.59699999999999998,"5min":1.0189999999999999,"15min":2.2869999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55400000000000005,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.29699999999999999},"max":{"1min":0.78800000000000003,"5min":0.93200000000000005,"15min":4.3639999999999999},"last":0.441}]},{"osd":3,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.38500000000000001},"max":{"1min":0.871,"5min":0.998,"15min":4.3259999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.33300000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":4.548},"last":0.69499999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92100000000000004,"5min":1.252,"15min":1.252},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.754,"15min":0.747},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.441},"max":{"1min":0.90200000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.51300000000000001},"max":{"1min":0.93700000000000006,"5min":1.2949999999999999,"15min":1.345},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.81799999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.624}]},{"osd":6,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.82199999999999995,"15min":0.80100000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.51400000000000001},"max":{"1min":0.89500000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.83399999999999996,"15min":0.81100000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":0.90700000000000003,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0149999999999999,"5min":1.0980000000000001,"15min":1.1539999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.94399999999999995,"5min":1.3620000000000001,"15min":1.3919999999999999},"last":0.75900000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509604,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302116,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069724,"statfs":{"total":96636764160,"available":96327397376,"internally_reserved":0,"allocated":749568,"data_stored":345649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42799999999999999,"15min":0.442},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":1.5640000000000001},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.48499999999999999,"15min":0.51200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60599999999999998,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.44400000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":0.68300000000000005,"15min":2.012},"last":0.313},{"interface":"front","average":{"1min":0.443,"5min":0.51400000000000001,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.40000000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56499999999999995},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.84499999999999997,"15min":4.2610000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.41799999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.746,"15min":0.78000000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.85399999999999998,"5min":1.1539999999999999,"15min":4.423},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.93500000000000005,"5min":1.1000000000000001,"15min":1.6439999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.78600000000000003,"15min":0.82099999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":1.119,"15min":4.3650000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.94599999999999995,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":1.103,"15min":1.611},"last":0.623},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79300000000000004,"15min":0.81999999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.5},"max":{"1min":1.002,"5min":1.111,"15min":4.407},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.82099999999999995},"min":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":0.89600000000000002,"5min":1.145,"15min":4.5069999999999997},"last":0.745}]}]},{"osd":2,"up_from":14,"seq":60129542309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":757760,"data_stored":351747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.32900000000000001,"5min":0.27000000000000002,"15min":0.248},"max":{"1min":0.67000000000000004,"5min":0.85999999999999999,"15min":1.915},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":1.7729999999999999},"last":0.79500000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48499999999999999,"15min":0.496},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70899999999999996,"15min":1.952},"last":0.626},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.30199999999999999},"max":{"1min":0.63100000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.438,"5min":0.437,"15min":0.375},"max":{"1min":0.78700000000000003,"5min":0.83999999999999997,"15min":0.97099999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.98299999999999998,"5min":1.1180000000000001,"15min":1.284},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.98399999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.75800000000000001},"min":{"1min":0.53900000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.5029999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78000000000000003,"15min":0.77200000000000002},"min":{"1min":0.621,"5min":0.54200000000000004,"15min":0.503},"max":{"1min":0.89900000000000002,"5min":1.028,"15min":1.355},"last":0.81100000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.91200000000000003,"5min":1.032,"15min":1.7350000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80200000000000005,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.90200000000000002,"5min":1.0640000000000001,"15min":1.349},"last":0.92400000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.47799999999999998},"max":{"1min":0.95999999999999996,"5min":1.0680000000000001,"15min":1.841},"last":0.872},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":1.004,"5min":1.091,"15min":1.294},"last":0.88400000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509605,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306228,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065612,"statfs":{"total":96636764160,"available":96323186688,"internally_reserved":0,"allocated":765952,"data_stored":353193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.438,"15min":0.442},"min":{"1min":0.312,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.627,"5min":0.68100000000000005,"15min":0.91400000000000003},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35699999999999998,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.30399999999999999},"max":{"1min":0.65900000000000003,"5min":0.73499999999999999,"15min":0.77200000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.32400000000000001},"max":{"1min":0.66800000000000004,"5min":0.72499999999999998,"15min":0.99399999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38900000000000001},"max":{"1min":0.72099999999999997,"5min":0.83699999999999997,"15min":1.0820000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.82599999999999996,"15min":1.3580000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.499,"15min":0.45100000000000001},"max":{"1min":1.018,"5min":1.105,"15min":1.3779999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":1.1870000000000001,"15min":1.2949999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.47699999999999998},"max":{"1min":1,"5min":1.27,"15min":1.4610000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.78400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.45900000000000002},"max":{"1min":0.96399999999999997,"5min":0.997,"15min":1.9239999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.1950000000000001,"15min":1.5049999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.54100000000000004},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.579},"last":0.87}]},{"osd":7,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78600000000000003,"15min":0.80200000000000005},"min":{"1min":0.63800000000000001,"5min":0.51100000000000001,"15min":0.49299999999999999},"max":{"1min":1.036,"5min":1.0640000000000001,"15min":1.964},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.1499999999999999,"15min":1.6359999999999999},"last":0.92400000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069700,"statfs":{"total":96636764160,"available":96327372800,"internally_reserved":0,"allocated":774144,"data_stored":353285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":0.96499999999999997,"15min":2.1389999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.008,"15min":1.407},"last":0.64100000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.878,"5min":0.90500000000000003,"15min":1.879},"last":0.443},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.70799999999999996,"15min":0.72199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":2.097},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.72399999999999998},"min":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":0.94399999999999995,"15min":1.7809999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":1.6870000000000001},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70799999999999996,"15min":0.72599999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.83899999999999997,"5min":0.95999999999999996,"15min":1.956},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72899999999999998,"15min":0.74399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.83199999999999996,"5min":0.97999999999999998,"15min":2.0609999999999999},"last":0.69399999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58799999999999997,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.89800000000000002,"15min":2.2429999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.64200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77000000000000002,"5min":0.92600000000000005,"15min":2.194},"last":0.81799999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60199999999999998,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.878,"15min":2.1589999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64700000000000002,"15min":0.70299999999999996},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.874,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":0.98099999999999998,"15min":2.23},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.57199999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90100000000000002,"5min":0.92700000000000005,"15min":2.5169999999999999},"last":0.79900000000000004}]}]},{"osd":5,"up_from":15,"seq":64424509603,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302704,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069136,"statfs":{"total":96636764160,"available":96326795264,"internally_reserved":0,"allocated":1351680,"data_stored":942047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":0.995,"15min":2.3900000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70799999999999996,"15min":0.73599999999999999},"min":{"1min":0.499,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.874,"5min":1.0369999999999999,"15min":1.71},"last":0.748}]},{"osd":1,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.51500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.61499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.4219999999999999},"last":0.63800000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72399999999999998,"15min":0.746},"min":{"1min":0.47899999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.622,"5min":0.435,"15min":0.435},"max":{"1min":0.81999999999999995,"5min":0.95699999999999996,"15min":1.214},"last":0.61699999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.74299999999999999},"min":{"1min":0.56399999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92500000000000004,"5min":1.0640000000000001,"15min":1.4890000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.874,"5min":0.93899999999999995,"15min":1.1439999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":2.5070000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59099999999999997,"15min":0.65600000000000003},"min":{"1min":0.55400000000000005,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90100000000000002,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.84799999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64300000000000002,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"max":{"1min":0.84299999999999997,"5min":0.90700000000000003,"15min":1.637},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.76500000000000001},"min":{"1min":0.65200000000000002,"5min":0.49099999999999999,"15min":0.438},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":2.5579999999999998},"last":0.87}]},{"osd":7,"last update":"Tue Dec 26 21:49:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67400000000000004,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.83299999999999996,"5min":0.96299999999999997,"15min":1.248},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68300000000000005,"15min":0.73299999999999998},"min":{"1min":0.56899999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83499999999999996,"5min":0.97199999999999998,"15min":1.5980000000000001},"last":0.88600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:50:14.462 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:50:14.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.657+0000 7f89025ff700 1 Processor -- start 2023-12-26T21:50:14.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.657+0000 7f89025ff700 1 -- start start 2023-12-26T21:50:14.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f89025ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f88fc159a80 0x7f88fc0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:14.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f89025ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 0x7f88fc06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:14.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f89025ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88fc15ac30 con 0x7f88fc1596b0 2023-12-26T21:50:14.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f89025ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88fc0c8270 con 0x7f88fc159a80 2023-12-26T21:50:14.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f89025ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc15a020 con 0x7f88fc0c7df0 2023-12-26T21:50:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f88fb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f88fc159a80 0x7f88fc0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f88fb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f88fc159a80 0x7f88fc0c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49864/0 (socket says 172.21.15.131:49864) 2023-12-26T21:50:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f88fb7fe700 1 -- 172.21.15.131:0/1574968162 learned_addr learned my addr 172.21.15.131:0/1574968162 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:14.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f8900b9c700 1 --2- 172.21.15.131:0/1574968162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 0x7f88fc06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:14.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f88fb7fe700 1 -- 172.21.15.131:0/1574968162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 msgr2=0x7f88fc06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:14.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f88fb7fe700 1 --2- 172.21.15.131:0/1574968162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 0x7f88fc06e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f88fb7fe700 1 -- 172.21.15.131:0/1574968162 >> v1:172.21.15.190:6789/0 conn(0x7f88fc1596b0 legacy=0x7f88fc067f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:14.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f88fb7fe700 1 -- 172.21.15.131:0/1574968162 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc15a300 con 0x7f88fc159a80 2023-12-26T21:50:14.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.659+0000 7f8900b9c700 1 --2- 172.21.15.131:0/1574968162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 0x7f88fc06e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f88fb7fe700 1 --2- 172.21.15.131:0/1574968162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 0x7f88fc0c78b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f88f00099b0 tx=0x7f88f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acd4918cbbf495f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f88faffd700 1 -- 172.21.15.131:0/1574968162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f000b860 con 0x7f88fc159a80 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f88faffd700 1 -- 172.21.15.131:0/1574968162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f000b9c0 con 0x7f88fc159a80 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f88faffd700 1 -- 172.21.15.131:0/1574968162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f000bd60 con 0x7f88fc159a80 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f89025ff700 1 -- 172.21.15.131:0/1574968162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 msgr2=0x7f88fc0c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f89025ff700 1 --2- 172.21.15.131:0/1574968162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 0x7f88fc0c78b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f88f00099b0 tx=0x7f88f0009250 comp rx=0 tx=0).stop 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f89025ff700 1 -- 172.21.15.131:0/1574968162 shutdown_connections 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f89025ff700 1 --2- 172.21.15.131:0/1574968162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 0x7f88fc06e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f89025ff700 1 --2- 172.21.15.131:0/1574968162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 0x7f88fc0c78b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f89025ff700 1 -- 172.21.15.131:0/1574968162 >> 172.21.15.131:0/1574968162 conn(0x7f88fc0686c0 msgr2=0x7f88fc068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.660+0000 7f89025ff700 1 -- 172.21.15.131:0/1574968162 shutdown_connections 2023-12-26T21:50:14.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.661+0000 7f89025ff700 1 -- 172.21.15.131:0/1574968162 wait complete. 2023-12-26T21:50:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.662+0000 7f89025ff700 1 Processor -- start 2023-12-26T21:50:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.662+0000 7f89025ff700 1 -- start start 2023-12-26T21:50:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f89025ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 0x7f88fc15e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f89025ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc1596b0 0x7f88fc1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f89025ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 0x7f88fc0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f89025ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0cdce0 con 0x7f88fc159a80 2023-12-26T21:50:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f89025ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc15ed20 con 0x7f88fc1596b0 2023-12-26T21:50:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f89025ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0cd0d0 con 0x7f88fc0c7df0 2023-12-26T21:50:14.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f8900b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 0x7f88fc0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f8900b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 0x7f88fc0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49868/0 (socket says 172.21.15.131:49868) 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f8900b9c700 1 -- 172.21.15.131:0/1491387425 learned_addr learned my addr 172.21.15.131:0/1491387425 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.663+0000 7f88fbfff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 0x7f88fc15e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f88fb7fe700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc1596b0 0x7f88fc1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f8900b9c700 1 -- 172.21.15.131:0/1491387425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 msgr2=0x7f88fc15e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f8900b9c700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 0x7f88fc15e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f8900b9c700 1 -- 172.21.15.131:0/1491387425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc1596b0 msgr2=0x7f88fc1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f8900b9c700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc1596b0 0x7f88fc1577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f8900b9c700 1 -- 172.21.15.131:0/1491387425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f00094f0 con 0x7f88fc159a80 2023-12-26T21:50:14.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f8900b9c700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 0x7f88fc0c6470 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009fc0 tx=0x7f88f40099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f88e3fff700 1 -- 172.21.15.131:0/1491387425 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f400dd40 con 0x7f88fc159a80 2023-12-26T21:50:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f88e3fff700 1 -- 172.21.15.131:0/1491387425 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f400f040 con 0x7f88fc159a80 2023-12-26T21:50:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc0cd3b0 con 0x7f88fc159a80 2023-12-26T21:50:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f88e3fff700 1 -- 172.21.15.131:0/1491387425 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4015690 con 0x7f88fc159a80 2023-12-26T21:50:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.664+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc0cd970 con 0x7f88fc159a80 2023-12-26T21:50:14.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.665+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88fc169b90 con 0x7f88fc159a80 2023-12-26T21:50:14.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.666+0000 7f88e3fff700 1 -- 172.21.15.131:0/1491387425 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f88f401e3b0 con 0x7f88fc159a80 2023-12-26T21:50:14.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.667+0000 7f88e3fff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f88e8066f30 0x7f88e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:14.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.668+0000 7f88e3fff700 1 -- 172.21.15.131:0/1491387425 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f88f4084ca0 con 0x7f88fc159a80 2023-12-26T21:50:14.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.668+0000 7f88fbfff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f88e8066f30 0x7f88e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:14.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.668+0000 7f88fbfff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f88e8066f30 0x7f88e80693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88ec003860 tx=0x7f88ec009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:14.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.672+0000 7f88e3fff700 1 -- 172.21.15.131:0/1491387425 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88fc169b90 con 0x7f88fc159a80 2023-12-26T21:50:14.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.861+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88fc169b90 con 0x7f88e8066f30 2023-12-26T21:50:14.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.869+0000 7f88e3fff700 1 -- 172.21.15.131:0/1491387425 <== mgr.5437 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288869 (secure 0 0 0) 0x7f88fc169b90 con 0x7f88e8066f30 2023-12-26T21:50:14.873 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.873+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f88e8066f30 msgr2=0x7f88e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.873+0000 7f89025ff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f88e8066f30 0x7f88e80693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88ec003860 tx=0x7f88ec009040 comp rx=0 tx=0).stop 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.873+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 msgr2=0x7f88fc0c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.873+0000 7f89025ff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 0x7f88fc0c6470 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009fc0 tx=0x7f88f40099b0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.874+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 shutdown_connections 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.874+0000 7f89025ff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88fc159a80 0x7f88fc0c6470 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.874+0000 7f89025ff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f88e8066f30 0x7f88e80693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.874+0000 7f89025ff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc0c7df0 0x7f88fc15e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.874+0000 7f89025ff700 1 --2- 172.21.15.131:0/1491387425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc1596b0 0x7f88fc1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.874+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 >> 172.21.15.131:0/1491387425 conn(0x7f88fc0686c0 msgr2=0x7f88fc0b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:14.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.874+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 shutdown_connections 2023-12-26T21:50:14.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:14.874+0000 7f89025ff700 1 -- 172.21.15.131:0/1491387425 wait complete. 2023-12-26T21:50:14.877 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:50:14.892 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-26T21:50:14.714848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":244,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422280,"kb_used_data":7112,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752552440,"statfs":{"total":773094113280,"available":770613698560,"internally_reserved":0,"allocated":7282688,"data_stored":3998604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.509447"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.229352+0000","last_change":"2023-12-26T21:50:05.226991+0000","last_active":"2023-12-26T21:50:05.229352+0000","last_peered":"2023-12-26T21:50:05.229352+0000","last_clean":"2023-12-26T21:50:05.229352+0000","last_became_active":"2023-12-26T21:50:05.226766+0000","last_became_peered":"2023-12-26T21:50:05.226766+0000","last_unstale":"2023-12-26T21:50:05.229352+0000","last_undegraded":"2023-12-26T21:50:05.229352+0000","last_fullsized":"2023-12-26T21:50:05.229352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:30:01.379377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239587+0000","last_change":"2023-12-26T21:50:05.230491+0000","last_active":"2023-12-26T21:50:05.239587+0000","last_peered":"2023-12-26T21:50:05.239587+0000","last_clean":"2023-12-26T21:50:05.239587+0000","last_became_active":"2023-12-26T21:50:05.230277+0000","last_became_peered":"2023-12-26T21:50:05.230277+0000","last_unstale":"2023-12-26T21:50:05.239587+0000","last_undegraded":"2023-12-26T21:50:05.239587+0000","last_fullsized":"2023-12-26T21:50:05.239587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:40:13.844989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233235+0000","last_change":"2023-12-26T21:50:05.226988+0000","last_active":"2023-12-26T21:50:05.233235+0000","last_peered":"2023-12-26T21:50:05.233235+0000","last_clean":"2023-12-26T21:50:05.233235+0000","last_became_active":"2023-12-26T21:50:05.226680+0000","last_became_peered":"2023-12-26T21:50:05.226680+0000","last_unstale":"2023-12-26T21:50:05.233235+0000","last_undegraded":"2023-12-26T21:50:05.233235+0000","last_fullsized":"2023-12-26T21:50:05.233235+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:57:23.669558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238873+0000","last_change":"2023-12-26T21:50:05.236391+0000","last_active":"2023-12-26T21:50:05.238873+0000","last_peered":"2023-12-26T21:50:05.238873+0000","last_clean":"2023-12-26T21:50:05.238873+0000","last_became_active":"2023-12-26T21:50:05.235021+0000","last_became_peered":"2023-12-26T21:50:05.235021+0000","last_unstale":"2023-12-26T21:50:05.238873+0000","last_undegraded":"2023-12-26T21:50:05.238873+0000","last_fullsized":"2023-12-26T21:50:05.238873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:29.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.232660+0000","last_change":"2023-12-26T21:50:05.225390+0000","last_active":"2023-12-26T21:50:05.232660+0000","last_peered":"2023-12-26T21:50:05.232660+0000","last_clean":"2023-12-26T21:50:05.232660+0000","last_became_active":"2023-12-26T21:50:05.225134+0000","last_became_peered":"2023-12-26T21:50:05.225134+0000","last_unstale":"2023-12-26T21:50:05.232660+0000","last_undegraded":"2023-12-26T21:50:05.232660+0000","last_fullsized":"2023-12-26T21:50:05.232660+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:30.275494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233316+0000","last_change":"2023-12-26T21:50:05.230103+0000","last_active":"2023-12-26T21:50:05.233316+0000","last_peered":"2023-12-26T21:50:05.233316+0000","last_clean":"2023-12-26T21:50:05.233316+0000","last_became_active":"2023-12-26T21:50:05.229812+0000","last_became_peered":"2023-12-26T21:50:05.229812+0000","last_unstale":"2023-12-26T21:50:05.233316+0000","last_undegraded":"2023-12-26T21:50:05.233316+0000","last_fullsized":"2023-12-26T21:50:05.233316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:49:21.726254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239194+0000","last_change":"2023-12-26T21:50:05.233540+0000","last_active":"2023-12-26T21:50:05.239194+0000","last_peered":"2023-12-26T21:50:05.239194+0000","last_clean":"2023-12-26T21:50:05.239194+0000","last_became_active":"2023-12-26T21:50:05.233189+0000","last_became_peered":"2023-12-26T21:50:05.233189+0000","last_unstale":"2023-12-26T21:50:05.239194+0000","last_undegraded":"2023-12-26T21:50:05.239194+0000","last_fullsized":"2023-12-26T21:50:05.239194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:38.099699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233097+0000","last_change":"2023-12-26T21:50:05.227196+0000","last_active":"2023-12-26T21:50:05.233097+0000","last_peered":"2023-12-26T21:50:05.233097+0000","last_clean":"2023-12-26T21:50:05.233097+0000","last_became_active":"2023-12-26T21:50:05.226763+0000","last_became_peered":"2023-12-26T21:50:05.226763+0000","last_unstale":"2023-12-26T21:50:05.233097+0000","last_undegraded":"2023-12-26T21:50:05.233097+0000","last_fullsized":"2023-12-26T21:50:05.233097+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:27:06.557823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233389+0000","last_change":"2023-12-26T21:50:05.230175+0000","last_active":"2023-12-26T21:50:05.233389+0000","last_peered":"2023-12-26T21:50:05.233389+0000","last_clean":"2023-12-26T21:50:05.233389+0000","last_became_active":"2023-12-26T21:50:05.228975+0000","last_became_peered":"2023-12-26T21:50:05.228975+0000","last_unstale":"2023-12-26T21:50:05.233389+0000","last_undegraded":"2023-12-26T21:50:05.233389+0000","last_fullsized":"2023-12-26T21:50:05.233389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:53:36.056936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.231885+0000","last_change":"2023-12-26T21:50:05.226840+0000","last_active":"2023-12-26T21:50:05.231885+0000","last_peered":"2023-12-26T21:50:05.231885+0000","last_clean":"2023-12-26T21:50:05.231885+0000","last_became_active":"2023-12-26T21:50:05.226580+0000","last_became_peered":"2023-12-26T21:50:05.226580+0000","last_unstale":"2023-12-26T21:50:05.231885+0000","last_undegraded":"2023-12-26T21:50:05.231885+0000","last_fullsized":"2023-12-26T21:50:05.231885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:15:04.988046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233197+0000","last_change":"2023-12-26T21:50:05.225387+0000","last_active":"2023-12-26T21:50:05.233197+0000","last_peered":"2023-12-26T21:50:05.233197+0000","last_clean":"2023-12-26T21:50:05.233197+0000","last_became_active":"2023-12-26T21:50:05.225093+0000","last_became_peered":"2023-12-26T21:50:05.225093+0000","last_unstale":"2023-12-26T21:50:05.233197+0000","last_undegraded":"2023-12-26T21:50:05.233197+0000","last_fullsized":"2023-12-26T21:50:05.233197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:13.703711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.232530+0000","last_change":"2023-12-26T21:50:05.227503+0000","last_active":"2023-12-26T21:50:05.232530+0000","last_peered":"2023-12-26T21:50:05.232530+0000","last_clean":"2023-12-26T21:50:05.232530+0000","last_became_active":"2023-12-26T21:50:05.227241+0000","last_became_peered":"2023-12-26T21:50:05.227241+0000","last_unstale":"2023-12-26T21:50:05.232530+0000","last_undegraded":"2023-12-26T21:50:05.232530+0000","last_fullsized":"2023-12-26T21:50:05.232530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:21.980776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.236944+0000","last_change":"2023-12-26T21:50:05.236267+0000","last_active":"2023-12-26T21:50:05.236944+0000","last_peered":"2023-12-26T21:50:05.236944+0000","last_clean":"2023-12-26T21:50:05.236944+0000","last_became_active":"2023-12-26T21:50:05.235023+0000","last_became_peered":"2023-12-26T21:50:05.235023+0000","last_unstale":"2023-12-26T21:50:05.236944+0000","last_undegraded":"2023-12-26T21:50:05.236944+0000","last_fullsized":"2023-12-26T21:50:05.236944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:24.909019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234063+0000","last_change":"2023-12-26T21:50:05.228775+0000","last_active":"2023-12-26T21:50:05.234063+0000","last_peered":"2023-12-26T21:50:05.234063+0000","last_clean":"2023-12-26T21:50:05.234063+0000","last_became_active":"2023-12-26T21:50:05.228476+0000","last_became_peered":"2023-12-26T21:50:05.228476+0000","last_unstale":"2023-12-26T21:50:05.234063+0000","last_undegraded":"2023-12-26T21:50:05.234063+0000","last_fullsized":"2023-12-26T21:50:05.234063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:41:03.707784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.236826+0000","last_change":"2023-12-26T21:50:05.233900+0000","last_active":"2023-12-26T21:50:05.236826+0000","last_peered":"2023-12-26T21:50:05.236826+0000","last_clean":"2023-12-26T21:50:05.236826+0000","last_became_active":"2023-12-26T21:50:05.229572+0000","last_became_peered":"2023-12-26T21:50:05.229572+0000","last_unstale":"2023-12-26T21:50:05.236826+0000","last_undegraded":"2023-12-26T21:50:05.236826+0000","last_fullsized":"2023-12-26T21:50:05.236826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:50.842679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.237995+0000","last_change":"2023-12-26T21:50:05.234891+0000","last_active":"2023-12-26T21:50:05.237995+0000","last_peered":"2023-12-26T21:50:05.237995+0000","last_clean":"2023-12-26T21:50:05.237995+0000","last_became_active":"2023-12-26T21:50:05.234633+0000","last_became_peered":"2023-12-26T21:50:05.234633+0000","last_unstale":"2023-12-26T21:50:05.237995+0000","last_undegraded":"2023-12-26T21:50:05.237995+0000","last_fullsized":"2023-12-26T21:50:05.237995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:27:06.557823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.235582+0000","last_change":"2023-12-26T21:50:05.228752+0000","last_active":"2023-12-26T21:50:05.235582+0000","last_peered":"2023-12-26T21:50:05.235582+0000","last_clean":"2023-12-26T21:50:05.235582+0000","last_became_active":"2023-12-26T21:50:05.227870+0000","last_became_peered":"2023-12-26T21:50:05.227870+0000","last_unstale":"2023-12-26T21:50:05.235582+0000","last_undegraded":"2023-12-26T21:50:05.235582+0000","last_fullsized":"2023-12-26T21:50:05.235582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:20.901688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238765+0000","last_change":"2023-12-26T21:50:05.232522+0000","last_active":"2023-12-26T21:50:05.238765+0000","last_peered":"2023-12-26T21:50:05.238765+0000","last_clean":"2023-12-26T21:50:05.238765+0000","last_became_active":"2023-12-26T21:50:05.232087+0000","last_became_peered":"2023-12-26T21:50:05.232087+0000","last_unstale":"2023-12-26T21:50:05.238765+0000","last_undegraded":"2023-12-26T21:50:05.238765+0000","last_fullsized":"2023-12-26T21:50:05.238765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:59:16.152796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.228470+0000","last_change":"2023-12-26T21:50:05.225163+0000","last_active":"2023-12-26T21:50:05.228470+0000","last_peered":"2023-12-26T21:50:05.228470+0000","last_clean":"2023-12-26T21:50:05.228470+0000","last_became_active":"2023-12-26T21:50:05.224418+0000","last_became_peered":"2023-12-26T21:50:05.224418+0000","last_unstale":"2023-12-26T21:50:05.228470+0000","last_undegraded":"2023-12-26T21:50:05.228470+0000","last_fullsized":"2023-12-26T21:50:05.228470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:29.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225805+0000","last_change":"2023-12-26T21:50:05.221852+0000","last_active":"2023-12-26T21:50:05.225805+0000","last_peered":"2023-12-26T21:50:05.225805+0000","last_clean":"2023-12-26T21:50:05.225805+0000","last_became_active":"2023-12-26T21:50:05.221574+0000","last_became_peered":"2023-12-26T21:50:05.221574+0000","last_unstale":"2023-12-26T21:50:05.225805+0000","last_undegraded":"2023-12-26T21:50:05.225805+0000","last_fullsized":"2023-12-26T21:50:05.225805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:45:09.667824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233949+0000","last_change":"2023-12-26T21:50:05.229424+0000","last_active":"2023-12-26T21:50:05.233949+0000","last_peered":"2023-12-26T21:50:05.233949+0000","last_clean":"2023-12-26T21:50:05.233949+0000","last_became_active":"2023-12-26T21:50:05.228038+0000","last_became_peered":"2023-12-26T21:50:05.228038+0000","last_unstale":"2023-12-26T21:50:05.233949+0000","last_undegraded":"2023-12-26T21:50:05.233949+0000","last_fullsized":"2023-12-26T21:50:05.233949+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:31:28.704242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216704+0000","last_change":"2023-12-26T21:50:05.213544+0000","last_active":"2023-12-26T21:50:05.216704+0000","last_peered":"2023-12-26T21:50:05.216704+0000","last_clean":"2023-12-26T21:50:05.216704+0000","last_became_active":"2023-12-26T21:50:05.212622+0000","last_became_peered":"2023-12-26T21:50:05.212622+0000","last_unstale":"2023-12-26T21:50:05.216704+0000","last_undegraded":"2023-12-26T21:50:05.216704+0000","last_fullsized":"2023-12-26T21:50:05.216704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:45.329023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238621+0000","last_change":"2023-12-26T21:50:05.231337+0000","last_active":"2023-12-26T21:50:05.238621+0000","last_peered":"2023-12-26T21:50:05.238621+0000","last_clean":"2023-12-26T21:50:05.238621+0000","last_became_active":"2023-12-26T21:50:05.231031+0000","last_became_peered":"2023-12-26T21:50:05.231031+0000","last_unstale":"2023-12-26T21:50:05.238621+0000","last_undegraded":"2023-12-26T21:50:05.238621+0000","last_fullsized":"2023-12-26T21:50:05.238621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:51:34.322061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239773+0000","last_change":"2023-12-26T21:50:05.235997+0000","last_active":"2023-12-26T21:50:05.239773+0000","last_peered":"2023-12-26T21:50:05.239773+0000","last_clean":"2023-12-26T21:50:05.239773+0000","last_became_active":"2023-12-26T21:50:05.235732+0000","last_became_peered":"2023-12-26T21:50:05.235732+0000","last_unstale":"2023-12-26T21:50:05.239773+0000","last_undegraded":"2023-12-26T21:50:05.239773+0000","last_fullsized":"2023-12-26T21:50:05.239773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:54:44.898807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.228275+0000","last_change":"2023-12-26T21:50:05.223936+0000","last_active":"2023-12-26T21:50:05.228275+0000","last_peered":"2023-12-26T21:50:05.228275+0000","last_clean":"2023-12-26T21:50:05.228275+0000","last_became_active":"2023-12-26T21:50:05.223684+0000","last_became_peered":"2023-12-26T21:50:05.223684+0000","last_unstale":"2023-12-26T21:50:05.228275+0000","last_undegraded":"2023-12-26T21:50:05.228275+0000","last_fullsized":"2023-12-26T21:50:05.228275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:13.703711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234525+0000","last_change":"2023-12-26T21:50:05.232194+0000","last_active":"2023-12-26T21:50:05.234525+0000","last_peered":"2023-12-26T21:50:05.234525+0000","last_clean":"2023-12-26T21:50:05.234525+0000","last_became_active":"2023-12-26T21:50:05.231928+0000","last_became_peered":"2023-12-26T21:50:05.231928+0000","last_unstale":"2023-12-26T21:50:05.234525+0000","last_undegraded":"2023-12-26T21:50:05.234525+0000","last_fullsized":"2023-12-26T21:50:05.234525+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:18.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239316+0000","last_change":"2023-12-26T21:50:05.233798+0000","last_active":"2023-12-26T21:50:05.239316+0000","last_peered":"2023-12-26T21:50:05.239316+0000","last_clean":"2023-12-26T21:50:05.239316+0000","last_became_active":"2023-12-26T21:50:05.229125+0000","last_became_peered":"2023-12-26T21:50:05.229125+0000","last_unstale":"2023-12-26T21:50:05.239316+0000","last_undegraded":"2023-12-26T21:50:05.239316+0000","last_fullsized":"2023-12-26T21:50:05.239316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:53.114423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234135+0000","last_change":"2023-12-26T21:50:05.224391+0000","last_active":"2023-12-26T21:50:05.234135+0000","last_peered":"2023-12-26T21:50:05.234135+0000","last_clean":"2023-12-26T21:50:05.234135+0000","last_became_active":"2023-12-26T21:50:05.224146+0000","last_became_peered":"2023-12-26T21:50:05.224146+0000","last_unstale":"2023-12-26T21:50:05.234135+0000","last_undegraded":"2023-12-26T21:50:05.234135+0000","last_fullsized":"2023-12-26T21:50:05.234135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:37.819740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234888+0000","last_change":"2023-12-26T21:50:05.230803+0000","last_active":"2023-12-26T21:50:05.234888+0000","last_peered":"2023-12-26T21:50:05.234888+0000","last_clean":"2023-12-26T21:50:05.234888+0000","last_became_active":"2023-12-26T21:50:05.228633+0000","last_became_peered":"2023-12-26T21:50:05.228633+0000","last_unstale":"2023-12-26T21:50:05.234888+0000","last_undegraded":"2023-12-26T21:50:05.234888+0000","last_fullsized":"2023-12-26T21:50:05.234888+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:50.842679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.224439+0000","last_change":"2023-12-26T21:50:05.221854+0000","last_active":"2023-12-26T21:50:05.224439+0000","last_peered":"2023-12-26T21:50:05.224439+0000","last_clean":"2023-12-26T21:50:05.224439+0000","last_became_active":"2023-12-26T21:50:05.221566+0000","last_became_peered":"2023-12-26T21:50:05.221566+0000","last_unstale":"2023-12-26T21:50:05.224439+0000","last_undegraded":"2023-12-26T21:50:05.224439+0000","last_fullsized":"2023-12-26T21:50:05.224439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:59.144750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.232861+0000","last_change":"2023-12-26T21:50:05.228319+0000","last_active":"2023-12-26T21:50:05.232861+0000","last_peered":"2023-12-26T21:50:05.232861+0000","last_clean":"2023-12-26T21:50:05.232861+0000","last_became_active":"2023-12-26T21:50:05.228033+0000","last_became_peered":"2023-12-26T21:50:05.228033+0000","last_unstale":"2023-12-26T21:50:05.232861+0000","last_undegraded":"2023-12-26T21:50:05.232861+0000","last_fullsized":"2023-12-26T21:50:05.232861+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:53:52.642285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233073+0000","last_change":"2023-12-26T21:50:05.227074+0000","last_active":"2023-12-26T21:50:05.233073+0000","last_peered":"2023-12-26T21:50:05.233073+0000","last_clean":"2023-12-26T21:50:05.233073+0000","last_became_active":"2023-12-26T21:50:05.226556+0000","last_became_peered":"2023-12-26T21:50:05.226556+0000","last_unstale":"2023-12-26T21:50:05.233073+0000","last_undegraded":"2023-12-26T21:50:05.233073+0000","last_fullsized":"2023-12-26T21:50:05.233073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:53:31.233090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227740+0000","last_change":"2023-12-26T21:50:05.225269+0000","last_active":"2023-12-26T21:50:05.227740+0000","last_peered":"2023-12-26T21:50:05.227740+0000","last_clean":"2023-12-26T21:50:05.227740+0000","last_became_active":"2023-12-26T21:50:05.224997+0000","last_became_peered":"2023-12-26T21:50:05.224997+0000","last_unstale":"2023-12-26T21:50:05.227740+0000","last_undegraded":"2023-12-26T21:50:05.227740+0000","last_fullsized":"2023-12-26T21:50:05.227740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:03.075265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"63'4","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248580+0000","last_change":"2023-12-26T21:50:04.227245+0000","last_active":"2023-12-26T21:50:05.248580+0000","last_peered":"2023-12-26T21:50:05.248580+0000","last_clean":"2023-12-26T21:50:05.248580+0000","last_became_active":"2023-12-26T21:50:04.226971+0000","last_became_peered":"2023-12-26T21:50:04.226971+0000","last_unstale":"2023-12-26T21:50:05.248580+0000","last_undegraded":"2023-12-26T21:50:05.248580+0000","last_fullsized":"2023-12-26T21:50:05.248580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:03:39.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.228295+0000","last_change":"2023-12-26T21:50:05.224999+0000","last_active":"2023-12-26T21:50:05.228295+0000","last_peered":"2023-12-26T21:50:05.228295+0000","last_clean":"2023-12-26T21:50:05.228295+0000","last_became_active":"2023-12-26T21:50:05.224758+0000","last_became_peered":"2023-12-26T21:50:05.224758+0000","last_unstale":"2023-12-26T21:50:05.228295+0000","last_undegraded":"2023-12-26T21:50:05.228295+0000","last_fullsized":"2023-12-26T21:50:05.228295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:18.112393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244190+0000","last_change":"2023-12-26T21:50:04.223739+0000","last_active":"2023-12-26T21:50:05.244190+0000","last_peered":"2023-12-26T21:50:05.244190+0000","last_clean":"2023-12-26T21:50:05.244190+0000","last_became_active":"2023-12-26T21:50:04.222755+0000","last_became_peered":"2023-12-26T21:50:04.222755+0000","last_unstale":"2023-12-26T21:50:05.244190+0000","last_undegraded":"2023-12-26T21:50:05.244190+0000","last_fullsized":"2023-12-26T21:50:05.244190+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:21.699131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239633+0000","last_change":"2023-12-26T21:50:05.229901+0000","last_active":"2023-12-26T21:50:05.239633+0000","last_peered":"2023-12-26T21:50:05.239633+0000","last_clean":"2023-12-26T21:50:05.239633+0000","last_became_active":"2023-12-26T21:50:05.229440+0000","last_became_peered":"2023-12-26T21:50:05.229440+0000","last_unstale":"2023-12-26T21:50:05.239633+0000","last_undegraded":"2023-12-26T21:50:05.239633+0000","last_fullsized":"2023-12-26T21:50:05.239633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:19.072262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248189+0000","last_change":"2023-12-26T21:50:04.226558+0000","last_active":"2023-12-26T21:50:05.248189+0000","last_peered":"2023-12-26T21:50:05.248189+0000","last_clean":"2023-12-26T21:50:05.248189+0000","last_became_active":"2023-12-26T21:50:04.226307+0000","last_became_peered":"2023-12-26T21:50:04.226307+0000","last_unstale":"2023-12-26T21:50:05.248189+0000","last_undegraded":"2023-12-26T21:50:05.248189+0000","last_fullsized":"2023-12-26T21:50:05.248189+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:59:27.515630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.240655+0000","last_change":"2023-12-26T21:50:05.237965+0000","last_active":"2023-12-26T21:50:05.240655+0000","last_peered":"2023-12-26T21:50:05.240655+0000","last_clean":"2023-12-26T21:50:05.240655+0000","last_became_active":"2023-12-26T21:50:05.237619+0000","last_became_peered":"2023-12-26T21:50:05.237619+0000","last_unstale":"2023-12-26T21:50:05.240655+0000","last_undegraded":"2023-12-26T21:50:05.240655+0000","last_fullsized":"2023-12-26T21:50:05.240655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:30.275494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.244999+0000","last_change":"2023-12-26T21:50:04.223118+0000","last_active":"2023-12-26T21:50:05.244999+0000","last_peered":"2023-12-26T21:50:05.244999+0000","last_clean":"2023-12-26T21:50:05.244999+0000","last_became_active":"2023-12-26T21:50:04.222720+0000","last_became_peered":"2023-12-26T21:50:04.222720+0000","last_unstale":"2023-12-26T21:50:05.244999+0000","last_undegraded":"2023-12-26T21:50:05.244999+0000","last_fullsized":"2023-12-26T21:50:05.244999+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:38:24.121296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.222331+0000","last_change":"2023-12-26T21:50:05.220983+0000","last_active":"2023-12-26T21:50:05.222331+0000","last_peered":"2023-12-26T21:50:05.222331+0000","last_clean":"2023-12-26T21:50:05.222331+0000","last_became_active":"2023-12-26T21:50:05.220694+0000","last_became_peered":"2023-12-26T21:50:05.220694+0000","last_unstale":"2023-12-26T21:50:05.222331+0000","last_undegraded":"2023-12-26T21:50:05.222331+0000","last_fullsized":"2023-12-26T21:50:05.222331+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:29:18.312367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.241850+0000","last_change":"2023-12-26T21:50:04.225617+0000","last_active":"2023-12-26T21:50:05.241850+0000","last_peered":"2023-12-26T21:50:05.241850+0000","last_clean":"2023-12-26T21:50:05.241850+0000","last_became_active":"2023-12-26T21:50:04.224621+0000","last_became_peered":"2023-12-26T21:50:04.224621+0000","last_unstale":"2023-12-26T21:50:05.241850+0000","last_undegraded":"2023-12-26T21:50:05.241850+0000","last_fullsized":"2023-12-26T21:50:05.241850+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:09:43.053987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238883+0000","last_change":"2023-12-26T21:50:05.235706+0000","last_active":"2023-12-26T21:50:05.238883+0000","last_peered":"2023-12-26T21:50:05.238883+0000","last_clean":"2023-12-26T21:50:05.238883+0000","last_became_active":"2023-12-26T21:50:05.235416+0000","last_became_peered":"2023-12-26T21:50:05.235416+0000","last_unstale":"2023-12-26T21:50:05.238883+0000","last_undegraded":"2023-12-26T21:50:05.238883+0000","last_fullsized":"2023-12-26T21:50:05.238883+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:20.767760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248231+0000","last_change":"2023-12-26T21:50:04.225964+0000","last_active":"2023-12-26T21:50:05.248231+0000","last_peered":"2023-12-26T21:50:05.248231+0000","last_clean":"2023-12-26T21:50:05.248231+0000","last_became_active":"2023-12-26T21:50:04.225653+0000","last_became_peered":"2023-12-26T21:50:04.225653+0000","last_unstale":"2023-12-26T21:50:05.248231+0000","last_undegraded":"2023-12-26T21:50:05.248231+0000","last_fullsized":"2023-12-26T21:50:05.248231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:49.834751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.226468+0000","last_change":"2023-12-26T21:50:05.223489+0000","last_active":"2023-12-26T21:50:05.226468+0000","last_peered":"2023-12-26T21:50:05.226468+0000","last_clean":"2023-12-26T21:50:05.226468+0000","last_became_active":"2023-12-26T21:50:05.222774+0000","last_became_peered":"2023-12-26T21:50:05.222774+0000","last_unstale":"2023-12-26T21:50:05.226468+0000","last_undegraded":"2023-12-26T21:50:05.226468+0000","last_fullsized":"2023-12-26T21:50:05.226468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:50.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214454+0000","last_change":"2023-12-26T21:50:04.232558+0000","last_active":"2023-12-26T21:50:05.214454+0000","last_peered":"2023-12-26T21:50:05.214454+0000","last_clean":"2023-12-26T21:50:05.214454+0000","last_became_active":"2023-12-26T21:50:04.232303+0000","last_became_peered":"2023-12-26T21:50:04.232303+0000","last_unstale":"2023-12-26T21:50:05.214454+0000","last_undegraded":"2023-12-26T21:50:05.214454+0000","last_fullsized":"2023-12-26T21:50:05.214454+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:13:48.174203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239174+0000","last_change":"2023-12-26T21:50:05.233433+0000","last_active":"2023-12-26T21:50:05.239174+0000","last_peered":"2023-12-26T21:50:05.239174+0000","last_clean":"2023-12-26T21:50:05.239174+0000","last_became_active":"2023-12-26T21:50:05.228341+0000","last_became_peered":"2023-12-26T21:50:05.228341+0000","last_unstale":"2023-12-26T21:50:05.239174+0000","last_undegraded":"2023-12-26T21:50:05.239174+0000","last_fullsized":"2023-12-26T21:50:05.239174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:58:54.385865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214676+0000","last_change":"2023-12-26T21:50:04.221595+0000","last_active":"2023-12-26T21:50:05.214676+0000","last_peered":"2023-12-26T21:50:05.214676+0000","last_clean":"2023-12-26T21:50:05.214676+0000","last_became_active":"2023-12-26T21:50:04.221337+0000","last_became_peered":"2023-12-26T21:50:04.221337+0000","last_unstale":"2023-12-26T21:50:05.214676+0000","last_undegraded":"2023-12-26T21:50:05.214676+0000","last_fullsized":"2023-12-26T21:50:05.214676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:38:20.838693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.224182+0000","last_change":"2023-12-26T21:50:05.217647+0000","last_active":"2023-12-26T21:50:05.224182+0000","last_peered":"2023-12-26T21:50:05.224182+0000","last_clean":"2023-12-26T21:50:05.224182+0000","last_became_active":"2023-12-26T21:50:05.217273+0000","last_became_peered":"2023-12-26T21:50:05.217273+0000","last_unstale":"2023-12-26T21:50:05.224182+0000","last_undegraded":"2023-12-26T21:50:05.224182+0000","last_fullsized":"2023-12-26T21:50:05.224182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:30.476540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.209456+0000","last_change":"2023-12-26T21:50:04.224494+0000","last_active":"2023-12-26T21:50:05.209456+0000","last_peered":"2023-12-26T21:50:05.209456+0000","last_clean":"2023-12-26T21:50:05.209456+0000","last_became_active":"2023-12-26T21:50:04.221693+0000","last_became_peered":"2023-12-26T21:50:04.221693+0000","last_unstale":"2023-12-26T21:50:05.209456+0000","last_undegraded":"2023-12-26T21:50:05.209456+0000","last_fullsized":"2023-12-26T21:50:05.209456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:45:52.041010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233265+0000","last_change":"2023-12-26T21:50:05.225928+0000","last_active":"2023-12-26T21:50:05.233265+0000","last_peered":"2023-12-26T21:50:05.233265+0000","last_clean":"2023-12-26T21:50:05.233265+0000","last_became_active":"2023-12-26T21:50:05.225635+0000","last_became_peered":"2023-12-26T21:50:05.225635+0000","last_unstale":"2023-12-26T21:50:05.233265+0000","last_undegraded":"2023-12-26T21:50:05.233265+0000","last_fullsized":"2023-12-26T21:50:05.233265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:11:48.608891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.214009+0000","last_change":"2023-12-26T21:50:04.229086+0000","last_active":"2023-12-26T21:50:05.214009+0000","last_peered":"2023-12-26T21:50:05.214009+0000","last_clean":"2023-12-26T21:50:05.214009+0000","last_became_active":"2023-12-26T21:50:04.228808+0000","last_became_peered":"2023-12-26T21:50:04.228808+0000","last_unstale":"2023-12-26T21:50:05.214009+0000","last_undegraded":"2023-12-26T21:50:05.214009+0000","last_fullsized":"2023-12-26T21:50:05.214009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:10:52.365673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238852+0000","last_change":"2023-12-26T21:50:05.234519+0000","last_active":"2023-12-26T21:50:05.238852+0000","last_peered":"2023-12-26T21:50:05.238852+0000","last_clean":"2023-12-26T21:50:05.238852+0000","last_became_active":"2023-12-26T21:50:05.230388+0000","last_became_peered":"2023-12-26T21:50:05.230388+0000","last_unstale":"2023-12-26T21:50:05.238852+0000","last_undegraded":"2023-12-26T21:50:05.238852+0000","last_fullsized":"2023-12-26T21:50:05.238852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:20:59.601587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"63'3","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250647+0000","last_change":"2023-12-26T21:50:04.227264+0000","last_active":"2023-12-26T21:50:05.250647+0000","last_peered":"2023-12-26T21:50:05.250647+0000","last_clean":"2023-12-26T21:50:05.250647+0000","last_became_active":"2023-12-26T21:50:04.226632+0000","last_became_peered":"2023-12-26T21:50:04.226632+0000","last_unstale":"2023-12-26T21:50:05.250647+0000","last_undegraded":"2023-12-26T21:50:05.250647+0000","last_fullsized":"2023-12-26T21:50:05.250647+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:53.639515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238522+0000","last_change":"2023-12-26T21:50:05.233319+0000","last_active":"2023-12-26T21:50:05.238522+0000","last_peered":"2023-12-26T21:50:05.238522+0000","last_clean":"2023-12-26T21:50:05.238522+0000","last_became_active":"2023-12-26T21:50:05.232699+0000","last_became_peered":"2023-12-26T21:50:05.232699+0000","last_unstale":"2023-12-26T21:50:05.238522+0000","last_undegraded":"2023-12-26T21:50:05.238522+0000","last_fullsized":"2023-12-26T21:50:05.238522+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:02:49.364640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250467+0000","last_change":"2023-12-26T21:50:04.227027+0000","last_active":"2023-12-26T21:50:05.250467+0000","last_peered":"2023-12-26T21:50:05.250467+0000","last_clean":"2023-12-26T21:50:05.250467+0000","last_became_active":"2023-12-26T21:50:04.226702+0000","last_became_peered":"2023-12-26T21:50:04.226702+0000","last_unstale":"2023-12-26T21:50:05.250467+0000","last_undegraded":"2023-12-26T21:50:05.250467+0000","last_fullsized":"2023-12-26T21:50:05.250467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:43:44.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.237691+0000","last_change":"2023-12-26T21:50:05.232476+0000","last_active":"2023-12-26T21:50:05.237691+0000","last_peered":"2023-12-26T21:50:05.237691+0000","last_clean":"2023-12-26T21:50:05.237691+0000","last_became_active":"2023-12-26T21:50:05.232146+0000","last_became_peered":"2023-12-26T21:50:05.232146+0000","last_unstale":"2023-12-26T21:50:05.237691+0000","last_undegraded":"2023-12-26T21:50:05.237691+0000","last_fullsized":"2023-12-26T21:50:05.237691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:59:28.523558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245546+0000","last_change":"2023-12-26T21:50:04.222842+0000","last_active":"2023-12-26T21:50:05.245546+0000","last_peered":"2023-12-26T21:50:05.245546+0000","last_clean":"2023-12-26T21:50:05.245546+0000","last_became_active":"2023-12-26T21:50:04.222585+0000","last_became_peered":"2023-12-26T21:50:04.222585+0000","last_unstale":"2023-12-26T21:50:05.245546+0000","last_undegraded":"2023-12-26T21:50:05.245546+0000","last_fullsized":"2023-12-26T21:50:05.245546+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:33:01.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.227858+0000","last_change":"2023-12-26T21:50:05.225230+0000","last_active":"2023-12-26T21:50:05.227858+0000","last_peered":"2023-12-26T21:50:05.227858+0000","last_clean":"2023-12-26T21:50:05.227858+0000","last_became_active":"2023-12-26T21:50:05.224946+0000","last_became_peered":"2023-12-26T21:50:05.224946+0000","last_unstale":"2023-12-26T21:50:05.227858+0000","last_undegraded":"2023-12-26T21:50:05.227858+0000","last_fullsized":"2023-12-26T21:50:05.227858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:03:31.057770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.218171+0000","last_change":"2023-12-26T21:50:04.227438+0000","last_active":"2023-12-26T21:50:05.218171+0000","last_peered":"2023-12-26T21:50:05.218171+0000","last_clean":"2023-12-26T21:50:05.218171+0000","last_became_active":"2023-12-26T21:50:04.226130+0000","last_became_peered":"2023-12-26T21:50:04.226130+0000","last_unstale":"2023-12-26T21:50:05.218171+0000","last_undegraded":"2023-12-26T21:50:05.218171+0000","last_fullsized":"2023-12-26T21:50:05.218171+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:00:03.077231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.219854+0000","last_change":"2023-12-26T21:50:05.217771+0000","last_active":"2023-12-26T21:50:05.219854+0000","last_peered":"2023-12-26T21:50:05.219854+0000","last_clean":"2023-12-26T21:50:05.219854+0000","last_became_active":"2023-12-26T21:50:05.217485+0000","last_became_peered":"2023-12-26T21:50:05.217485+0000","last_unstale":"2023-12-26T21:50:05.219854+0000","last_undegraded":"2023-12-26T21:50:05.219854+0000","last_fullsized":"2023-12-26T21:50:05.219854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:54.647443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212183+0000","last_change":"2023-12-26T21:50:04.228778+0000","last_active":"2023-12-26T21:50:05.212183+0000","last_peered":"2023-12-26T21:50:05.212183+0000","last_clean":"2023-12-26T21:50:05.212183+0000","last_became_active":"2023-12-26T21:50:04.228516+0000","last_became_peered":"2023-12-26T21:50:04.228516+0000","last_unstale":"2023-12-26T21:50:05.212183+0000","last_undegraded":"2023-12-26T21:50:05.212183+0000","last_fullsized":"2023-12-26T21:50:05.212183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:26.294398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.236011+0000","last_change":"2023-12-26T21:50:05.228184+0000","last_active":"2023-12-26T21:50:05.236011+0000","last_peered":"2023-12-26T21:50:05.236011+0000","last_clean":"2023-12-26T21:50:05.236011+0000","last_became_active":"2023-12-26T21:50:05.227940+0000","last_became_peered":"2023-12-26T21:50:05.227940+0000","last_unstale":"2023-12-26T21:50:05.236011+0000","last_undegraded":"2023-12-26T21:50:05.236011+0000","last_fullsized":"2023-12-26T21:50:05.236011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:21:03.619005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.250174+0000","last_change":"2023-12-26T21:50:04.224690+0000","last_active":"2023-12-26T21:50:05.250174+0000","last_peered":"2023-12-26T21:50:05.250174+0000","last_clean":"2023-12-26T21:50:05.250174+0000","last_became_active":"2023-12-26T21:50:04.223460+0000","last_became_peered":"2023-12-26T21:50:04.223460+0000","last_unstale":"2023-12-26T21:50:05.250174+0000","last_undegraded":"2023-12-26T21:50:05.250174+0000","last_fullsized":"2023-12-26T21:50:05.250174+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:01.981864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216840+0000","last_change":"2023-12-26T21:50:05.213363+0000","last_active":"2023-12-26T21:50:05.216840+0000","last_peered":"2023-12-26T21:50:05.216840+0000","last_clean":"2023-12-26T21:50:05.216840+0000","last_became_active":"2023-12-26T21:50:05.212965+0000","last_became_peered":"2023-12-26T21:50:05.212965+0000","last_unstale":"2023-12-26T21:50:05.216840+0000","last_undegraded":"2023-12-26T21:50:05.216840+0000","last_fullsized":"2023-12-26T21:50:05.216840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:01:27.770799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.241206+0000","last_change":"2023-12-26T21:50:04.224381+0000","last_active":"2023-12-26T21:50:05.241206+0000","last_peered":"2023-12-26T21:50:05.241206+0000","last_clean":"2023-12-26T21:50:05.241206+0000","last_became_active":"2023-12-26T21:50:04.224082+0000","last_became_peered":"2023-12-26T21:50:04.224082+0000","last_unstale":"2023-12-26T21:50:05.241206+0000","last_undegraded":"2023-12-26T21:50:05.241206+0000","last_fullsized":"2023-12-26T21:50:05.241206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:49:28.351623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.223902+0000","last_change":"2023-12-26T21:50:05.222959+0000","last_active":"2023-12-26T21:50:05.223902+0000","last_peered":"2023-12-26T21:50:05.223902+0000","last_clean":"2023-12-26T21:50:05.223902+0000","last_became_active":"2023-12-26T21:50:05.222618+0000","last_became_peered":"2023-12-26T21:50:05.222618+0000","last_unstale":"2023-12-26T21:50:05.223902+0000","last_undegraded":"2023-12-26T21:50:05.223902+0000","last_fullsized":"2023-12-26T21:50:05.223902+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:13.568346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245765+0000","last_change":"2023-12-26T21:50:04.219680+0000","last_active":"2023-12-26T21:50:05.245765+0000","last_peered":"2023-12-26T21:50:05.245765+0000","last_clean":"2023-12-26T21:50:05.245765+0000","last_became_active":"2023-12-26T21:50:04.219425+0000","last_became_peered":"2023-12-26T21:50:04.219425+0000","last_unstale":"2023-12-26T21:50:05.245765+0000","last_undegraded":"2023-12-26T21:50:05.245765+0000","last_fullsized":"2023-12-26T21:50:05.245765+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:50:10.822244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239646+0000","last_change":"2023-12-26T21:50:05.236257+0000","last_active":"2023-12-26T21:50:05.239646+0000","last_peered":"2023-12-26T21:50:05.239646+0000","last_clean":"2023-12-26T21:50:05.239646+0000","last_became_active":"2023-12-26T21:50:05.235988+0000","last_became_peered":"2023-12-26T21:50:05.235988+0000","last_unstale":"2023-12-26T21:50:05.239646+0000","last_undegraded":"2023-12-26T21:50:05.239646+0000","last_fullsized":"2023-12-26T21:50:05.239646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:43:47.213090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.247822+0000","last_change":"2023-12-26T21:50:04.224911+0000","last_active":"2023-12-26T21:50:05.247822+0000","last_peered":"2023-12-26T21:50:05.247822+0000","last_clean":"2023-12-26T21:50:05.247822+0000","last_became_active":"2023-12-26T21:50:04.223726+0000","last_became_peered":"2023-12-26T21:50:04.223726+0000","last_unstale":"2023-12-26T21:50:05.247822+0000","last_undegraded":"2023-12-26T21:50:05.247822+0000","last_fullsized":"2023-12-26T21:50:05.247822+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:40:40.318772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.230157+0000","last_change":"2023-12-26T21:50:05.223489+0000","last_active":"2023-12-26T21:50:05.230157+0000","last_peered":"2023-12-26T21:50:05.230157+0000","last_clean":"2023-12-26T21:50:05.230157+0000","last_became_active":"2023-12-26T21:50:05.223200+0000","last_became_peered":"2023-12-26T21:50:05.223200+0000","last_unstale":"2023-12-26T21:50:05.230157+0000","last_undegraded":"2023-12-26T21:50:05.230157+0000","last_fullsized":"2023-12-26T21:50:05.230157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:37:48.075465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212178+0000","last_change":"2023-12-26T21:50:04.229701+0000","last_active":"2023-12-26T21:50:05.212178+0000","last_peered":"2023-12-26T21:50:05.212178+0000","last_clean":"2023-12-26T21:50:05.212178+0000","last_became_active":"2023-12-26T21:50:04.229390+0000","last_became_peered":"2023-12-26T21:50:04.229390+0000","last_unstale":"2023-12-26T21:50:05.212178+0000","last_undegraded":"2023-12-26T21:50:05.212178+0000","last_fullsized":"2023-12-26T21:50:05.212178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:16:28.242388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225878+0000","last_change":"2023-12-26T21:50:05.224746+0000","last_active":"2023-12-26T21:50:05.225878+0000","last_peered":"2023-12-26T21:50:05.225878+0000","last_clean":"2023-12-26T21:50:05.225878+0000","last_became_active":"2023-12-26T21:50:05.224302+0000","last_became_peered":"2023-12-26T21:50:05.224302+0000","last_unstale":"2023-12-26T21:50:05.225878+0000","last_undegraded":"2023-12-26T21:50:05.225878+0000","last_fullsized":"2023-12-26T21:50:05.225878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:44.028736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.248205+0000","last_change":"2023-12-26T21:50:04.226030+0000","last_active":"2023-12-26T21:50:05.248205+0000","last_peered":"2023-12-26T21:50:05.248205+0000","last_clean":"2023-12-26T21:50:05.248205+0000","last_became_active":"2023-12-26T21:50:04.225766+0000","last_became_peered":"2023-12-26T21:50:04.225766+0000","last_unstale":"2023-12-26T21:50:05.248205+0000","last_undegraded":"2023-12-26T21:50:05.248205+0000","last_fullsized":"2023-12-26T21:50:05.248205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:58.136823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.229691+0000","last_change":"2023-12-26T21:50:05.224442+0000","last_active":"2023-12-26T21:50:05.229691+0000","last_peered":"2023-12-26T21:50:05.229691+0000","last_clean":"2023-12-26T21:50:05.229691+0000","last_became_active":"2023-12-26T21:50:05.224152+0000","last_became_peered":"2023-12-26T21:50:05.224152+0000","last_unstale":"2023-12-26T21:50:05.229691+0000","last_undegraded":"2023-12-26T21:50:05.229691+0000","last_fullsized":"2023-12-26T21:50:05.229691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:44:29.772129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.218282+0000","last_change":"2023-12-26T21:50:04.231701+0000","last_active":"2023-12-26T21:50:05.218282+0000","last_peered":"2023-12-26T21:50:05.218282+0000","last_clean":"2023-12-26T21:50:05.218282+0000","last_became_active":"2023-12-26T21:50:04.231270+0000","last_became_peered":"2023-12-26T21:50:04.231270+0000","last_unstale":"2023-12-26T21:50:05.218282+0000","last_undegraded":"2023-12-26T21:50:05.218282+0000","last_fullsized":"2023-12-26T21:50:05.218282+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:41:02.699856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233022+0000","last_change":"2023-12-26T21:50:05.226864+0000","last_active":"2023-12-26T21:50:05.233022+0000","last_peered":"2023-12-26T21:50:05.233022+0000","last_clean":"2023-12-26T21:50:05.233022+0000","last_became_active":"2023-12-26T21:50:05.226485+0000","last_became_peered":"2023-12-26T21:50:05.226485+0000","last_unstale":"2023-12-26T21:50:05.233022+0000","last_undegraded":"2023-12-26T21:50:05.233022+0000","last_fullsized":"2023-12-26T21:50:05.233022+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:29:22.091541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216148+0000","last_change":"2023-12-26T21:50:04.225794+0000","last_active":"2023-12-26T21:50:05.216148+0000","last_peered":"2023-12-26T21:50:05.216148+0000","last_clean":"2023-12-26T21:50:05.216148+0000","last_became_active":"2023-12-26T21:50:04.225535+0000","last_became_peered":"2023-12-26T21:50:04.225535+0000","last_unstale":"2023-12-26T21:50:05.216148+0000","last_undegraded":"2023-12-26T21:50:05.216148+0000","last_fullsized":"2023-12-26T21:50:05.216148+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:01:54.466108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234604+0000","last_change":"2023-12-26T21:50:05.231979+0000","last_active":"2023-12-26T21:50:05.234604+0000","last_peered":"2023-12-26T21:50:05.234604+0000","last_clean":"2023-12-26T21:50:05.234604+0000","last_became_active":"2023-12-26T21:50:05.228858+0000","last_became_peered":"2023-12-26T21:50:05.228858+0000","last_unstale":"2023-12-26T21:50:05.234604+0000","last_undegraded":"2023-12-26T21:50:05.234604+0000","last_fullsized":"2023-12-26T21:50:05.234604+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:00.075762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.213208+0000","last_change":"2023-12-26T21:50:04.224397+0000","last_active":"2023-12-26T21:50:05.213208+0000","last_peered":"2023-12-26T21:50:05.213208+0000","last_clean":"2023-12-26T21:50:05.213208+0000","last_became_active":"2023-12-26T21:50:04.224078+0000","last_became_peered":"2023-12-26T21:50:04.224078+0000","last_unstale":"2023-12-26T21:50:05.213208+0000","last_undegraded":"2023-12-26T21:50:05.213208+0000","last_fullsized":"2023-12-26T21:50:05.213208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:26:28.511389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.234103+0000","last_change":"2023-12-26T21:50:05.227159+0000","last_active":"2023-12-26T21:50:05.234103+0000","last_peered":"2023-12-26T21:50:05.234103+0000","last_clean":"2023-12-26T21:50:05.234103+0000","last_became_active":"2023-12-26T21:50:05.226887+0000","last_became_peered":"2023-12-26T21:50:05.226887+0000","last_unstale":"2023-12-26T21:50:05.234103+0000","last_undegraded":"2023-12-26T21:50:05.234103+0000","last_fullsized":"2023-12-26T21:50:05.234103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:01:34.210345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-26T21:50:06.223423+0000","last_change":"2023-12-26T21:50:04.230019+0000","last_active":"2023-12-26T21:50:06.223423+0000","last_peered":"2023-12-26T21:50:06.223423+0000","last_clean":"2023-12-26T21:50:06.223423+0000","last_became_active":"2023-12-26T21:50:04.229584+0000","last_became_peered":"2023-12-26T21:50:04.229584+0000","last_unstale":"2023-12-26T21:50:06.223423+0000","last_undegraded":"2023-12-26T21:50:06.223423+0000","last_fullsized":"2023-12-26T21:50:06.223423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:44:28.764202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225824+0000","last_change":"2023-12-26T21:50:05.223267+0000","last_active":"2023-12-26T21:50:05.225824+0000","last_peered":"2023-12-26T21:50:05.225824+0000","last_clean":"2023-12-26T21:50:05.225824+0000","last_became_active":"2023-12-26T21:50:05.222964+0000","last_became_peered":"2023-12-26T21:50:05.222964+0000","last_unstale":"2023-12-26T21:50:05.225824+0000","last_undegraded":"2023-12-26T21:50:05.225824+0000","last_fullsized":"2023-12-26T21:50:05.225824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:42:16.214790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.215412+0000","last_change":"2023-12-26T21:50:04.231179+0000","last_active":"2023-12-26T21:50:05.215412+0000","last_peered":"2023-12-26T21:50:05.215412+0000","last_clean":"2023-12-26T21:50:05.215412+0000","last_became_active":"2023-12-26T21:50:04.230888+0000","last_became_peered":"2023-12-26T21:50:04.230888+0000","last_unstale":"2023-12-26T21:50:05.215412+0000","last_undegraded":"2023-12-26T21:50:05.215412+0000","last_fullsized":"2023-12-26T21:50:05.215412+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:30:14.453888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239297+0000","last_change":"2023-12-26T21:50:05.233495+0000","last_active":"2023-12-26T21:50:05.239297+0000","last_peered":"2023-12-26T21:50:05.239297+0000","last_clean":"2023-12-26T21:50:05.239297+0000","last_became_active":"2023-12-26T21:50:05.233051+0000","last_became_peered":"2023-12-26T21:50:05.233051+0000","last_unstale":"2023-12-26T21:50:05.239297+0000","last_undegraded":"2023-12-26T21:50:05.239297+0000","last_fullsized":"2023-12-26T21:50:05.239297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:49:53.896795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208240+0000","last_change":"2023-12-26T21:50:04.218089+0000","last_active":"2023-12-26T21:50:05.208240+0000","last_peered":"2023-12-26T21:50:05.208240+0000","last_clean":"2023-12-26T21:50:05.208240+0000","last_became_active":"2023-12-26T21:50:04.215530+0000","last_became_peered":"2023-12-26T21:50:04.215530+0000","last_unstale":"2023-12-26T21:50:05.208240+0000","last_undegraded":"2023-12-26T21:50:05.208240+0000","last_fullsized":"2023-12-26T21:50:05.208240+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:21.909145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.225858+0000","last_change":"2023-12-26T21:50:05.224578+0000","last_active":"2023-12-26T21:50:05.225858+0000","last_peered":"2023-12-26T21:50:05.225858+0000","last_clean":"2023-12-26T21:50:05.225858+0000","last_became_active":"2023-12-26T21:50:05.223985+0000","last_became_peered":"2023-12-26T21:50:05.223985+0000","last_unstale":"2023-12-26T21:50:05.225858+0000","last_undegraded":"2023-12-26T21:50:05.225858+0000","last_fullsized":"2023-12-26T21:50:05.225858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:47:51.264287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.216243+0000","last_change":"2023-12-26T21:50:04.231436+0000","last_active":"2023-12-26T21:50:05.216243+0000","last_peered":"2023-12-26T21:50:05.216243+0000","last_clean":"2023-12-26T21:50:05.216243+0000","last_became_active":"2023-12-26T21:50:04.231168+0000","last_became_peered":"2023-12-26T21:50:04.231168+0000","last_unstale":"2023-12-26T21:50:05.216243+0000","last_undegraded":"2023-12-26T21:50:05.216243+0000","last_fullsized":"2023-12-26T21:50:05.216243+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:12:19.198601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.233969+0000","last_change":"2023-12-26T21:50:05.230590+0000","last_active":"2023-12-26T21:50:05.233969+0000","last_peered":"2023-12-26T21:50:05.233969+0000","last_clean":"2023-12-26T21:50:05.233969+0000","last_became_active":"2023-12-26T21:50:05.229413+0000","last_became_peered":"2023-12-26T21:50:05.229413+0000","last_unstale":"2023-12-26T21:50:05.233969+0000","last_undegraded":"2023-12-26T21:50:05.233969+0000","last_fullsized":"2023-12-26T21:50:05.233969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:22.707058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212760+0000","last_change":"2023-12-26T21:50:04.222797+0000","last_active":"2023-12-26T21:50:05.212760+0000","last_peered":"2023-12-26T21:50:05.212760+0000","last_clean":"2023-12-26T21:50:05.212760+0000","last_became_active":"2023-12-26T21:50:04.222490+0000","last_became_peered":"2023-12-26T21:50:04.222490+0000","last_unstale":"2023-12-26T21:50:05.212760+0000","last_undegraded":"2023-12-26T21:50:05.212760+0000","last_fullsized":"2023-12-26T21:50:05.212760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:17.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239591+0000","last_change":"2023-12-26T21:50:05.233040+0000","last_active":"2023-12-26T21:50:05.239591+0000","last_peered":"2023-12-26T21:50:05.239591+0000","last_clean":"2023-12-26T21:50:05.239591+0000","last_became_active":"2023-12-26T21:50:05.232801+0000","last_became_peered":"2023-12-26T21:50:05.232801+0000","last_unstale":"2023-12-26T21:50:05.239591+0000","last_undegraded":"2023-12-26T21:50:05.239591+0000","last_fullsized":"2023-12-26T21:50:05.239591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:03:40.240620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.212142+0000","last_change":"2023-12-26T21:50:04.225898+0000","last_active":"2023-12-26T21:50:05.212142+0000","last_peered":"2023-12-26T21:50:05.212142+0000","last_clean":"2023-12-26T21:50:05.212142+0000","last_became_active":"2023-12-26T21:50:04.225376+0000","last_became_peered":"2023-12-26T21:50:04.225376+0000","last_unstale":"2023-12-26T21:50:05.212142+0000","last_undegraded":"2023-12-26T21:50:05.212142+0000","last_fullsized":"2023-12-26T21:50:05.212142+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:57:22.661631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.239829+0000","last_change":"2023-12-26T21:50:05.229746+0000","last_active":"2023-12-26T21:50:05.239829+0000","last_peered":"2023-12-26T21:50:05.239829+0000","last_clean":"2023-12-26T21:50:05.239829+0000","last_became_active":"2023-12-26T21:50:05.229483+0000","last_became_peered":"2023-12-26T21:50:05.229483+0000","last_unstale":"2023-12-26T21:50:05.239829+0000","last_undegraded":"2023-12-26T21:50:05.239829+0000","last_fullsized":"2023-12-26T21:50:05.239829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:01:55.474035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208086+0000","last_change":"2023-12-26T21:50:04.220953+0000","last_active":"2023-12-26T21:50:05.208086+0000","last_peered":"2023-12-26T21:50:05.208086+0000","last_clean":"2023-12-26T21:50:05.208086+0000","last_became_active":"2023-12-26T21:50:04.220694+0000","last_became_peered":"2023-12-26T21:50:04.220694+0000","last_unstale":"2023-12-26T21:50:05.208086+0000","last_undegraded":"2023-12-26T21:50:05.208086+0000","last_fullsized":"2023-12-26T21:50:05.208086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:06:05.018373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.238819+0000","last_change":"2023-12-26T21:50:05.235997+0000","last_active":"2023-12-26T21:50:05.238819+0000","last_peered":"2023-12-26T21:50:05.238819+0000","last_clean":"2023-12-26T21:50:05.238819+0000","last_became_active":"2023-12-26T21:50:05.234830+0000","last_became_peered":"2023-12-26T21:50:05.234830+0000","last_unstale":"2023-12-26T21:50:05.238819+0000","last_undegraded":"2023-12-26T21:50:05.238819+0000","last_fullsized":"2023-12-26T21:50:05.238819+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:04.196855+0000","last_clean_scrub_stamp":"2023-12-26T21:50:04.196855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:30:01.379377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.245342+0000","last_change":"2023-12-26T21:50:04.226718+0000","last_active":"2023-12-26T21:50:05.245342+0000","last_peered":"2023-12-26T21:50:05.245342+0000","last_clean":"2023-12-26T21:50:05.245342+0000","last_became_active":"2023-12-26T21:50:04.226415+0000","last_became_peered":"2023-12-26T21:50:04.226415+0000","last_unstale":"2023-12-26T21:50:05.245342+0000","last_undegraded":"2023-12-26T21:50:05.245342+0000","last_fullsized":"2023-12-26T21:50:05.245342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:50:03.188927+0000","last_clean_scrub_stamp":"2023-12-26T21:50:03.188927+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:38:29.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"53'139","reported_seq":278,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-26T21:50:05.208912+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:50:05.208912+0000","last_peered":"2023-12-26T21:50:05.208912+0000","last_clean":"2023-12-26T21:50:05.208912+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:50:05.208912+0000","last_undegraded":"2023-12-26T21:50:05.208912+0000","last_fullsized":"2023-12-26T21:50:05.208912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542309,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":351691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.89100000000000001,"5min":0.94099999999999995,"15min":1.647},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":1,"5min":1,"15min":2.0150000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72599999999999998,"15min":0.70699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.016,"5min":1.048,"15min":1.1359999999999999},"last":0.443}]},{"osd":2,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.8640000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":0.95599999999999996,"15min":1.419},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":0.77200000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.026,"15min":1.327},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91900000000000004,"5min":1.008,"15min":1.393},"last":0.65100000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":0.89400000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.75},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.371},"max":{"1min":0.84399999999999997,"5min":0.85199999999999998,"15min":0.86799999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":0.97599999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75700000000000001,"15min":0.745},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.499},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.78600000000000003}]},{"osd":6,"last update":"Tue Dec 26 21:49:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67600000000000005,"15min":0.66600000000000004},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79100000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72199999999999998,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":0.86099999999999999,"5min":0.92600000000000005,"15min":0.999},"last":0.83799999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509605,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302128,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069712,"statfs":{"total":96636764160,"available":96327385088,"internally_reserved":0,"allocated":761856,"data_stored":352245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.46400000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":0.98599999999999999,"15min":1.085},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.82899999999999996,"5min":0.95099999999999996,"15min":1.3380000000000001},"last":0.60999999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:50:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.76200000000000001,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.874,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.84399999999999997}]},{"osd":2,"last update":"Tue Dec 26 21:50:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.46500000000000002,"5min":0.46000000000000002,"15min":0.434},"max":{"1min":0.90100000000000002,"5min":0.97099999999999997,"15min":1.288},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.45600000000000002},"max":{"1min":0.89500000000000002,"5min":0.93200000000000005,"15min":1.113},"last":0.85399999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:50:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73999999999999999,"15min":0.73299999999999998},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.46300000000000002},"max":{"1min":0.877,"5min":0.93600000000000005,"15min":1.569},"last":0.874},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76400000000000001,"15min":0.75900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93200000000000005,"5min":1.0129999999999999,"15min":1.3839999999999999},"last":0.78900000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:50:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.62,"15min":0.63200000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.879,"5min":0.92700000000000005,"15min":1.137},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.52300000000000002,"5min":0.46400000000000002,"15min":0.38100000000000001},"max":{"1min":0.89100000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:50:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.44800000000000001,"5min":0.42099999999999999,"15min":0.38600000000000001},"max":{"1min":0.84299999999999997,"5min":0.875,"15min":1.1200000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.69899999999999995,"15min":0.70399999999999996},"min":{"1min":0.58199999999999996,"5min":0.47899999999999998,"15min":0.42599999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.296},"last":0.76900000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:50:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71799999999999997,"15min":0.72699999999999998},"min":{"1min":0.56999999999999995,"5min":0.441,"15min":0.437},"max":{"1min":0.96899999999999997,"5min":1.0920000000000001,"15min":1.51},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.58599999999999997,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.80600000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509605,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306800,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065040,"statfs":{"total":96636764160,"available":96322600960,"internally_reserved":0,"allocated":1351680,"data_stored":942095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.309,"5min":0.307,"15min":0.255},"max":{"1min":0.65900000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.501,"15min":0.52900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.754,"15min":4.6600000000000001},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.49199999999999999,"15min":0.497},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.59699999999999998,"5min":1.0189999999999999,"15min":2.2869999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55400000000000005,"15min":0.57599999999999996},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.29699999999999999},"max":{"1min":0.78800000000000003,"5min":0.93200000000000005,"15min":4.3639999999999999},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.38500000000000001},"max":{"1min":0.871,"5min":0.998,"15min":4.3259999999999996},"last":0.442},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.33300000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":4.548},"last":0.50700000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92100000000000004,"5min":1.252,"15min":1.252},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.754,"15min":0.747},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.441},"max":{"1min":0.90200000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.71599999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.51300000000000001},"max":{"1min":0.93700000000000006,"5min":1.2949999999999999,"15min":1.345},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.81799999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.79700000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.82199999999999995,"15min":0.80100000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.51400000000000001},"max":{"1min":0.89500000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.83399999999999996,"15min":0.81100000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":0.90700000000000003,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.81299999999999994}]},{"osd":7,"last update":"Tue Dec 26 21:49:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0149999999999999,"5min":1.0980000000000001,"15min":1.1539999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.80700000000000005,"15min":0.79100000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.94399999999999995,"5min":1.3620000000000001,"15min":1.3919999999999999},"last":0.81999999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509605,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302128,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069712,"statfs":{"total":96636764160,"available":96327385088,"internally_reserved":0,"allocated":761856,"data_stored":352301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.42799999999999999,"15min":0.442},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":1.5640000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.432,"5min":0.48499999999999999,"15min":0.51200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.60599999999999998,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.376}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":0.68300000000000005,"15min":2.012},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.51400000000000001,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.438}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56499999999999995},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.84499999999999997,"15min":4.2610000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.746,"15min":0.78000000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.85399999999999998,"5min":1.1539999999999999,"15min":4.423},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.93500000000000005,"5min":1.1000000000000001,"15min":1.6439999999999999},"last":0.69299999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.78600000000000003,"15min":0.82099999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":1.119,"15min":4.3650000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47499999999999998},"max":{"1min":0.94599999999999995,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.70499999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.93000000000000005,"5min":1.103,"15min":1.611},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79300000000000004,"15min":0.81999999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.5},"max":{"1min":1.002,"5min":1.111,"15min":4.407},"last":0.79400000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.79100000000000004,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.79100000000000004,"15min":0.82099999999999995},"min":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":0.89600000000000002,"5min":1.145,"15min":4.5069999999999997},"last":0.81899999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542310,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":757760,"data_stored":351747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.32900000000000001,"5min":0.27000000000000002,"15min":0.248},"max":{"1min":0.67000000000000004,"5min":0.85999999999999999,"15min":1.915},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":1.7729999999999999},"last":0.371}]},{"osd":1,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48499999999999999,"15min":0.496},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70899999999999996,"15min":1.952},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.30199999999999999},"max":{"1min":0.63100000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.31}]},{"osd":3,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.438,"5min":0.437,"15min":0.375},"max":{"1min":0.78700000000000003,"5min":0.83999999999999997,"15min":0.97099999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.98299999999999998,"5min":1.1180000000000001,"15min":1.284},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.98399999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.75800000000000001},"min":{"1min":0.53900000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.5029999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78000000000000003,"15min":0.77200000000000002},"min":{"1min":0.621,"5min":0.54200000000000004,"15min":0.503},"max":{"1min":0.89900000000000002,"5min":1.028,"15min":1.355},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.91200000000000003,"5min":1.032,"15min":1.7350000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80200000000000005,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.90200000000000002,"5min":1.0640000000000001,"15min":1.349},"last":0.76900000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.47799999999999998},"max":{"1min":0.95999999999999996,"5min":1.0680000000000001,"15min":1.841},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.80800000000000005},"min":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":1.004,"5min":1.091,"15min":1.294},"last":0.626}]}]},{"osd":3,"up_from":15,"seq":64424509606,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306228,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065612,"statfs":{"total":96636764160,"available":96323186688,"internally_reserved":0,"allocated":765952,"data_stored":353193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.438,"15min":0.442},"min":{"1min":0.312,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.627,"5min":0.68100000000000005,"15min":0.91400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.49399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35699999999999998,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58499999999999996}]},{"osd":1,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.30399999999999999},"max":{"1min":0.65900000000000003,"5min":0.73499999999999999,"15min":0.77200000000000002},"last":0.63},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.32400000000000001},"max":{"1min":0.66800000000000004,"5min":0.72499999999999998,"15min":0.99399999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38900000000000001},"max":{"1min":0.72099999999999997,"5min":0.83699999999999997,"15min":1.0820000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.40300000000000002},"max":{"1min":0.78400000000000003,"5min":0.82599999999999996,"15min":1.3580000000000001},"last":0.93600000000000005}]},{"osd":4,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.499,"15min":0.45100000000000001},"max":{"1min":1.018,"5min":1.105,"15min":1.3779999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":1.1870000000000001,"15min":1.2949999999999999},"last":0.73399999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.47699999999999998},"max":{"1min":1,"5min":1.27,"15min":1.4610000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.78400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.45900000000000002},"max":{"1min":0.96399999999999997,"5min":0.997,"15min":1.9239999999999999},"last":0.80800000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.1950000000000001,"15min":1.5049999999999999},"last":0.873},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81299999999999994,"15min":0.81799999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.54100000000000004},"max":{"1min":0.97599999999999998,"5min":1.0760000000000001,"15min":1.579},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78600000000000003,"15min":0.80200000000000005},"min":{"1min":0.63800000000000001,"5min":0.51100000000000001,"15min":0.49299999999999999},"max":{"1min":1.036,"5min":1.0640000000000001,"15min":1.964},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":1.1499999999999999,"15min":1.6359999999999999},"last":0.83799999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069700,"statfs":{"total":96636764160,"available":96327372800,"internally_reserved":0,"allocated":774144,"data_stored":353285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":0.96499999999999997,"15min":2.1389999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":1.008,"15min":1.407},"last":0.77300000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.878,"5min":0.90500000000000003,"15min":1.879},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.70799999999999996,"15min":0.72199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.95399999999999996,"15min":2.097},"last":0.751}]},{"osd":2,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.72399999999999998},"min":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":0.94399999999999995,"15min":1.7809999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":1.6870000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70799999999999996,"15min":0.72599999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.83899999999999997,"5min":0.95999999999999996,"15min":1.956},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72899999999999998,"15min":0.74399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.83199999999999996,"5min":0.97999999999999998,"15min":2.0609999999999999},"last":0.72299999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58799999999999997,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.89800000000000002,"15min":2.2429999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.64200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77000000000000002,"5min":0.92600000000000005,"15min":2.194},"last":0.67800000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60199999999999998,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.878,"15min":2.1589999999999998},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64700000000000002,"15min":0.70299999999999996},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.874,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":0.98099999999999998,"15min":2.23},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.57199999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90100000000000002,"5min":0.92700000000000005,"15min":2.5169999999999999},"last":0.78800000000000003}]}]},{"osd":5,"up_from":15,"seq":64424509604,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302704,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069136,"statfs":{"total":96636764160,"available":96326795264,"internally_reserved":0,"allocated":1351680,"data_stored":942047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:50:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.67900000000000005,"15min":0.70799999999999996},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.91200000000000003,"5min":0.995,"15min":2.3900000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.68799999999999994,"15min":0.72399999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81200000000000006,"5min":1.0369999999999999,"15min":1.71},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Dec 26 21:50:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.68799999999999994,"15min":0.71599999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96499999999999997,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.69999999999999996,"15min":0.72399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84299999999999997,"5min":1.01,"15min":1.4219999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Tue Dec 26 21:50:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69999999999999996,"15min":0.73899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70199999999999996,"15min":0.71499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":0.95699999999999996,"15min":1.214},"last":0.61799999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:50:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71899999999999997,"15min":0.73799999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":1.052,"5min":1.0640000000000001,"15min":1.4890000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.48199999999999998,"15min":0.47799999999999998},"max":{"1min":0.89200000000000002,"5min":0.93899999999999995,"15min":1.1439999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Tue Dec 26 21:50:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57899999999999996,"15min":0.60799999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":2.5070000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.64900000000000002},"min":{"1min":0.46000000000000002,"5min":0.41599999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.435}]},{"osd":6,"last update":"Tue Dec 26 21:50:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.68999999999999995},"min":{"1min":0.498,"5min":0.36899999999999999,"15min":0.35499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":1.637},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72899999999999998,"15min":0.75900000000000001},"min":{"1min":0.55600000000000005,"5min":0.496,"15min":0.438},"max":{"1min":1.042,"5min":1.042,"15min":2.5579999999999998},"last":0.65300000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:50:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.441,"15min":0.436},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":1.248},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.72799999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92900000000000005,"5min":0.97199999999999998,"15min":1.5980000000000001},"last":0.58299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:50:14.898 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T21:50:14.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T21:50:15.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.129+0000 7f7c095ff700 1 Processor -- start 2023-12-26T21:50:15.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.129+0000 7f7c095ff700 1 -- start start 2023-12-26T21:50:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.130+0000 7f7c095ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c04159650 0x7f7c040cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.130+0000 7f7c095ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c040cf430 0x7f7c040cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.130+0000 7f7c095ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c040ccbc0 con 0x7f7c04159a20 2023-12-26T21:50:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.130+0000 7f7c095ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c040cbfb0 con 0x7f7c04159650 2023-12-26T21:50:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.130+0000 7f7c095ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040cc290 con 0x7f7c040cf430 2023-12-26T21:50:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c027fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7c04159a20 0x7f7c040ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56280/0 (socket says 172.21.15.131:56280) 2023-12-26T21:50:15.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c027fc700 1 -- 172.21.15.131:0/3773980572 learned_addr learned my addr 172.21.15.131:0/3773980572 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:15.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c037fe700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c040cf430 0x7f7c040cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c02ffd700 1 --2- 172.21.15.131:0/3773980572 >> v2:172.21.15.190:3300/0 conn(0x7f7c04159650 0x7f7c040cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472610738 0 0) 0x7f7c040ccbc0 con 0x7f7c04159a20 2023-12-26T21:50:15.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bec003570 con 0x7f7c04159a20 2023-12-26T21:50:15.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 425356790 0 0) 0x7f7bec003570 con 0x7f7c04159a20 2023-12-26T21:50:15.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c040cf430 msgr2=0x7f7c040cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:15.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c01ffb700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c040cf430 0x7f7c040cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 >> v2:172.21.15.190:3300/0 conn(0x7f7c04159650 msgr2=0x7f7c040cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c01ffb700 1 --2- 172.21.15.131:0/3773980572 >> v2:172.21.15.190:3300/0 conn(0x7f7c04159650 0x7f7c040cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040cc570 con 0x7f7c04159a20 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.131+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f7bf8001470 con 0x7f7c04159a20 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 >> v1:172.21.15.131:6789/0 conn(0x7f7c04159a20 legacy=0x7f7c040ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c01ffb700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bec003a60 0x7f7bec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c01ffb700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bec004810 0x7f7bec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c01ffb700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c040cf430 0x7f7bec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040cbfb0 con 0x7f7bec004810 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040cc290 con 0x7f7bec003a60 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec00ba60 con 0x7f7c040cf430 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c027fc700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bec004810 0x7f7bec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c02ffd700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bec003a60 0x7f7bec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c037fe700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c040cf430 0x7f7bec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c027fc700 1 -- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c040cf430 msgr2=0x7f7bec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c027fc700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c040cf430 0x7f7bec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c027fc700 1 -- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bec003a60 msgr2=0x7f7bec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c027fc700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bec003a60 0x7f7bec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c027fc700 1 -- 172.21.15.131:0/3773980572 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec003570 con 0x7f7bec004810 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.132+0000 7f7c027fc700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bec004810 0x7f7bec004c80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7c04088f70 tx=0x7f7bf8007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6284c23c6aac0a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800cbc0 con 0x7f7bec004810 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf800cd20 con 0x7f7bec004810 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c01ffb700 1 -- 172.21.15.131:0/3773980572 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8005b50 con 0x7f7bec004810 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c095ff700 1 -- 172.21.15.131:0/3773980572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bec004810 msgr2=0x7f7bec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c095ff700 1 --2- 172.21.15.131:0/3773980572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bec004810 0x7f7bec004c80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7c04088f70 tx=0x7f7bf8007940 comp rx=0 tx=0).stop 2023-12-26T21:50:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c037fe700 1 -- 172.21.15.131:0/3773980572 reap_dead start 2023-12-26T21:50:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c095ff700 1 -- 172.21.15.131:0/3773980572 shutdown_connections 2023-12-26T21:50:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c095ff700 1 -- 172.21.15.131:0/3773980572 >> 172.21.15.131:0/3773980572 conn(0x7f7c04099980 msgr2=0x7f7c0406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c095ff700 1 -- 172.21.15.131:0/3773980572 shutdown_connections 2023-12-26T21:50:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.133+0000 7f7c095ff700 1 -- 172.21.15.131:0/3773980572 wait complete. 2023-12-26T21:50:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.135+0000 7f7c095ff700 1 Processor -- start 2023-12-26T21:50:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.135+0000 7f7c095ff700 1 -- start start 2023-12-26T21:50:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c095ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bec004810 0x7f7c0415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c095ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c0416d9e0 0x7f7c0414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c095ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c040b59a0 0x7f7c040b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c095ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040b6380 con 0x7f7c0416d9e0 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c095ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04157ae0 con 0x7f7c040b59a0 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c095ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040c9ac0 con 0x7f7bec004810 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c027fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c0416d9e0 0x7f7c0414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c027fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c0416d9e0 0x7f7c0414a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51804/0 (socket says 172.21.15.131:51804) 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c027fc700 1 -- 172.21.15.131:0/1711630009 learned_addr learned my addr 172.21.15.131:0/1711630009 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c02ffd700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bec004810 0x7f7c0415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c037fe700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c040b59a0 0x7f7c040b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c027fc700 1 -- 172.21.15.131:0/1711630009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bec004810 msgr2=0x7f7c0415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c027fc700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bec004810 0x7f7c0415d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c027fc700 1 -- 172.21.15.131:0/1711630009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c040b59a0 msgr2=0x7f7c040b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c027fc700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c040b59a0 0x7f7c040b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.136+0000 7f7c027fc700 1 -- 172.21.15.131:0/1711630009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0414a670 con 0x7f7c0416d9e0 2023-12-26T21:50:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.137+0000 7f7c027fc700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c0416d9e0 0x7f7c0414a130 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7c040b31c0 tx=0x7f7bf8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.137+0000 7f7beaffd700 1 -- 172.21.15.131:0/1711630009 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80072e0 con 0x7f7c0416d9e0 2023-12-26T21:50:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.137+0000 7f7beaffd700 1 -- 172.21.15.131:0/1711630009 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf8007460 con 0x7f7c0416d9e0 2023-12-26T21:50:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.137+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0416de50 con 0x7f7c0416d9e0 2023-12-26T21:50:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.137+0000 7f7beaffd700 1 -- 172.21.15.131:0/1711630009 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8007820 con 0x7f7c0416d9e0 2023-12-26T21:50:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.137+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c040ca010 con 0x7f7c0416d9e0 2023-12-26T21:50:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.138+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c040657e0 con 0x7f7c0416d9e0 2023-12-26T21:50:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.139+0000 7f7beaffd700 1 -- 172.21.15.131:0/1711630009 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7bf8019070 con 0x7f7c0416d9e0 2023-12-26T21:50:15.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.140+0000 7f7beaffd700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7bf0066f30 0x7f7bf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.140+0000 7f7beaffd700 1 -- 172.21.15.131:0/1711630009 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7bf80855c0 con 0x7f7c0416d9e0 2023-12-26T21:50:15.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.140+0000 7f7c02ffd700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7bf0066f30 0x7f7bf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.140+0000 7f7c02ffd700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7bf0066f30 0x7f7bf00693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c040c87d0 tx=0x7f7bf4009120 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.144+0000 7f7beaffd700 1 -- 172.21.15.131:0/1711630009 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bf8052620 con 0x7f7c0416d9e0 2023-12-26T21:50:15.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.415+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7c040cc850 con 0x7f7c0416d9e0 2023-12-26T21:50:15.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.419+0000 7f7beaffd700 1 -- 172.21.15.131:0/1711630009 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7bf8055c40 con 0x7f7c0416d9e0 2023-12-26T21:50:15.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7bf0066f30 msgr2=0x7f7bf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7bf0066f30 0x7f7bf00693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c040c87d0 tx=0x7f7bf4009120 comp rx=0 tx=0).stop 2023-12-26T21:50:15.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c0416d9e0 msgr2=0x7f7c0414a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c0416d9e0 0x7f7c0414a130 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7c040b31c0 tx=0x7f7bf8015000 comp rx=0 tx=0).stop 2023-12-26T21:50:15.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 shutdown_connections 2023-12-26T21:50:15.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c040b59a0 0x7f7c040b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f7bf0066f30 0x7f7bf00693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bec004810 0x7f7c0415d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 --2- 172.21.15.131:0/1711630009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c0416d9e0 0x7f7c0414a130 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 >> 172.21.15.131:0/1711630009 conn(0x7f7c04099980 msgr2=0x7f7c04068670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:15.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 shutdown_connections 2023-12-26T21:50:15.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.421+0000 7f7c095ff700 1 -- 172.21.15.131:0/1711630009 wait complete. 2023-12-26T21:50:15.423 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T21:50:15.442 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:50:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.630+0000 7f31bbfff700 1 Processor -- start 2023-12-26T21:50:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.631+0000 7f31bbfff700 1 -- start start 2023-12-26T21:50:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.632+0000 7f31bbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f31b40c8bf0 0x7f31b40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.632+0000 7f31bbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31b4069d10 0x7f31b40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.632+0000 7f31bbfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b40caa10 con 0x7f31b40c8820 2023-12-26T21:50:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.632+0000 7f31bbfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b4157a80 con 0x7f31b40c8bf0 2023-12-26T21:50:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.632+0000 7f31bbfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b40cac50 con 0x7f31b4069d10 2023-12-26T21:50:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.632+0000 7f31b959a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f31b40c8bf0 0x7f31b40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.632+0000 7f31b9d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f31b40c8820 0x7f31b40ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56282/0 (socket says 172.21.15.131:56282) 2023-12-26T21:50:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.632+0000 7f31b9d9b700 1 -- 172.21.15.131:0/676016509 learned_addr learned my addr 172.21.15.131:0/676016509 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.632+0000 7f31ba59c700 1 --2- 172.21.15.131:0/676016509 >> v2:172.21.15.190:3300/0 conn(0x7f31b4069d10 0x7f31b40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.633+0000 7f31b959a700 1 -- 172.21.15.131:0/676016509 >> v2:172.21.15.190:3300/0 conn(0x7f31b4069d10 msgr2=0x7f31b40b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.633+0000 7f31b959a700 1 --2- 172.21.15.131:0/676016509 >> v2:172.21.15.190:3300/0 conn(0x7f31b4069d10 0x7f31b40b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.633+0000 7f31b959a700 1 -- 172.21.15.131:0/676016509 >> v1:172.21.15.131:6789/0 conn(0x7f31b40c8820 legacy=0x7f31b40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.633+0000 7f31b959a700 1 -- 172.21.15.131:0/676016509 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40caf30 con 0x7f31b40c8bf0 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.633+0000 7f31b8d99700 1 -- 172.21.15.131:0/676016509 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108409992 0 0) 0x7f31b0001110 con 0x7f31b40c8820 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.633+0000 7f31b959a700 1 --2- 172.21.15.131:0/676016509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8bf0 0x7f31b40b12e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f31b4070f80 tx=0x7f31a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81d39128caab713 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.633+0000 7f31b8d99700 1 -- 172.21.15.131:0/676016509 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a400b6b0 con 0x7f31b40c8bf0 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.633+0000 7f31b8d99700 1 -- 172.21.15.131:0/676016509 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a400b810 con 0x7f31b40c8bf0 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.633+0000 7f31b8d99700 1 -- 172.21.15.131:0/676016509 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a400bbb0 con 0x7f31b40c8bf0 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.634+0000 7f31bbfff700 1 -- 172.21.15.131:0/676016509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8bf0 msgr2=0x7f31b40b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.634+0000 7f31bbfff700 1 --2- 172.21.15.131:0/676016509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8bf0 0x7f31b40b12e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f31b4070f80 tx=0x7f31a40091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.634+0000 7f31bbfff700 1 -- 172.21.15.131:0/676016509 shutdown_connections 2023-12-26T21:50:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.634+0000 7f31bbfff700 1 --2- 172.21.15.131:0/676016509 >> v2:172.21.15.190:3300/0 conn(0x7f31b4069d10 0x7f31b40b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.634+0000 7f31bbfff700 1 --2- 172.21.15.131:0/676016509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8bf0 0x7f31b40b12e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.634+0000 7f31bbfff700 1 -- 172.21.15.131:0/676016509 >> 172.21.15.131:0/676016509 conn(0x7f31b4068040 msgr2=0x7f31b4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.634+0000 7f31bbfff700 1 -- 172.21.15.131:0/676016509 shutdown_connections 2023-12-26T21:50:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.634+0000 7f31bbfff700 1 -- 172.21.15.131:0/676016509 wait complete. 2023-12-26T21:50:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.635+0000 7f31bbfff700 1 Processor -- start 2023-12-26T21:50:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.635+0000 7f31bbfff700 1 -- start start 2023-12-26T21:50:15.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.636+0000 7f31bbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4069d10 0x7f31b416a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.636+0000 7f31bbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8820 0x7f31b4158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.636+0000 7f31bbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b40c8bf0 0x7f31b4154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.636+0000 7f31bbfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b4159340 con 0x7f31b40c8820 2023-12-26T21:50:15.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.636+0000 7f31bbfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b4154e80 con 0x7f31b4069d10 2023-12-26T21:50:15.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.636+0000 7f31bbfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b415c8a0 con 0x7f31b40c8bf0 2023-12-26T21:50:15.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8820 0x7f31b4158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8820 0x7f31b4158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51822/0 (socket says 172.21.15.131:51822) 2023-12-26T21:50:15.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b959a700 1 -- 172.21.15.131:0/3823483047 learned_addr learned my addr 172.21.15.131:0/3823483047 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b9d9b700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4069d10 0x7f31b416a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31ba59c700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b40c8bf0 0x7f31b4154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b959a700 1 -- 172.21.15.131:0/3823483047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b40c8bf0 msgr2=0x7f31b4154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b959a700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b40c8bf0 0x7f31b4154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b959a700 1 -- 172.21.15.131:0/3823483047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4069d10 msgr2=0x7f31b416a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b959a700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4069d10 0x7f31b416a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b959a700 1 -- 172.21.15.131:0/3823483047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a4009420 con 0x7f31b40c8820 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b9d9b700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4069d10 0x7f31b416a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31b959a700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8820 0x7f31b4158e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f31a4013000 tx=0x7f31a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31a9ffb700 1 -- 172.21.15.131:0/3823483047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a400bb90 con 0x7f31b40c8820 2023-12-26T21:50:15.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31a9ffb700 1 -- 172.21.15.131:0/3823483047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a4011420 con 0x7f31b40c8820 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31a9ffb700 1 -- 172.21.15.131:0/3823483047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40117e0 con 0x7f31b40c8820 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.637+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b4159df0 con 0x7f31b40c8820 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.638+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b4159830 con 0x7f31b40c8820 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.639+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b415ca30 con 0x7f31b40c8820 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.640+0000 7f31a9ffb700 1 -- 172.21.15.131:0/3823483047 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f31a4011c10 con 0x7f31b40c8820 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.641+0000 7f31a9ffb700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3198066f30 0x7f31980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.641+0000 7f31a9ffb700 1 -- 172.21.15.131:0/3823483047 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f31a4086210 con 0x7f31b40c8820 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.641+0000 7f31b9d9b700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3198066f30 0x7f31980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.641+0000 7f31b9d9b700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3198066f30 0x7f31980693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31b0001940 tx=0x7f31b0009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:15.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.645+0000 7f31a9ffb700 1 -- 172.21.15.131:0/3823483047 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31a4053270 con 0x7f31b40c8820 2023-12-26T21:50:15.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.882+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31b40657e0 con 0x7f31b40c8820 2023-12-26T21:50:15.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.883+0000 7f31a9ffb700 1 -- 172.21.15.131:0/3823483047 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f31a4056890 con 0x7f31b40c8820 2023-12-26T21:50:15.884 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:15.884 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3316959664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3316959664},{"type":"v1","addr":"172.21.15.190:6834","nonce":3316959664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/320470110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":320470110},{"type":"v1","addr":"172.21.15.190:6835","nonce":320470110}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/1337439654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1337439654},{"type":"v1","addr":"172.21.15.131:6836","nonce":1337439654}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:50:04.699122+0000","modified":"2023-12-26T21:50:08.232778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6837/607340038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":607340038},{"type":"v1","addr":"172.21.15.131:6837","nonce":607340038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-12-26T21:50:15.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.885+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3198066f30 msgr2=0x7f31980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.885+0000 7f31bbfff700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3198066f30 0x7f31980693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31b0001940 tx=0x7f31b0009040 comp rx=0 tx=0).stop 2023-12-26T21:50:15.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.885+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8820 msgr2=0x7f31b4158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:15.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.885+0000 7f31bbfff700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8820 0x7f31b4158e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f31a4013000 tx=0x7f31a4017000 comp rx=0 tx=0).stop 2023-12-26T21:50:15.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.886+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 shutdown_connections 2023-12-26T21:50:15.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.886+0000 7f31bbfff700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3198066f30 0x7f31980693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.886+0000 7f31bbfff700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b40c8bf0 0x7f31b4154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.886+0000 7f31bbfff700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b4069d10 0x7f31b416a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.886+0000 7f31bbfff700 1 --2- 172.21.15.131:0/3823483047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b40c8820 0x7f31b4158e00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:15.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.886+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 >> 172.21.15.131:0/3823483047 conn(0x7f31b4068040 msgr2=0x7f31b40b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:15.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.886+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 shutdown_connections 2023-12-26T21:50:15.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:15.886+0000 7f31bbfff700 1 -- 172.21.15.131:0/3823483047 wait complete. 2023-12-26T21:50:15.887 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 25 2023-12-26T21:50:16.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T21:50:04.699122+0000', 'modified': '2023-12-26T21:50:08.232778+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5739': {'gid': 5739, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6837/607340038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 607340038}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 607340038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T21:50:16.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T21:50:16.307 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-26T21:50:16.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.498+0000 7fdba45ff700 1 Processor -- start 2023-12-26T21:50:16.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.498+0000 7fdba45ff700 1 -- start start 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.499+0000 7fdba45ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdb9c13c510 0x7fdb9c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba45ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 0x7fdb9c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba45ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c0686c0 con 0x7fdb9c06b8c0 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba45ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9c06c280 con 0x7fdb9c13c510 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba45ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c069520 con 0x7fdb9c0cc810 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba239b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdb9c13c510 0x7fdb9c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba239b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdb9c13c510 0x7fdb9c0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51834/0 (socket says 172.21.15.131:51834) 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba239b700 1 -- 172.21.15.131:0/2318129550 learned_addr learned my addr 172.21.15.131:0/2318129550 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba2b9c700 1 --2- 172.21.15.131:0/2318129550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 0x7fdb9c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba239b700 1 -- 172.21.15.131:0/2318129550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 msgr2=0x7fdb9c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba239b700 1 --2- 172.21.15.131:0/2318129550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 0x7fdb9c06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba239b700 1 -- 172.21.15.131:0/2318129550 >> v1:172.21.15.190:6789/0 conn(0x7fdb9c06b8c0 legacy=0x7fdb9c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.500+0000 7fdba239b700 1 -- 172.21.15.131:0/2318129550 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c068910 con 0x7fdb9c13c510 2023-12-26T21:50:16.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba239b700 1 --2- 172.21.15.131:0/2318129550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 0x7fdb9c0c8a20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009990 tx=0x7fdb98009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c6f49832af3f7b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba1399700 1 -- 172.21.15.131:0/2318129550 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98009e80 con 0x7fdb9c13c510 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba1399700 1 -- 172.21.15.131:0/2318129550 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98005c10 con 0x7fdb9c13c510 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba1399700 1 -- 172.21.15.131:0/2318129550 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800b650 con 0x7fdb9c13c510 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba45ff700 1 -- 172.21.15.131:0/2318129550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 msgr2=0x7fdb9c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba45ff700 1 --2- 172.21.15.131:0/2318129550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 0x7fdb9c0c8a20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009990 tx=0x7fdb98009220 comp rx=0 tx=0).stop 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba45ff700 1 -- 172.21.15.131:0/2318129550 shutdown_connections 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba45ff700 1 --2- 172.21.15.131:0/2318129550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 0x7fdb9c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba45ff700 1 --2- 172.21.15.131:0/2318129550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 0x7fdb9c0c8a20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.501+0000 7fdba45ff700 1 -- 172.21.15.131:0/2318129550 >> 172.21.15.131:0/2318129550 conn(0x7fdb9c0ce3e0 msgr2=0x7fdb9c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.502+0000 7fdba45ff700 1 -- 172.21.15.131:0/2318129550 shutdown_connections 2023-12-26T21:50:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.502+0000 7fdba45ff700 1 -- 172.21.15.131:0/2318129550 wait complete. 2023-12-26T21:50:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.503+0000 7fdba45ff700 1 Processor -- start 2023-12-26T21:50:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.503+0000 7fdba45ff700 1 -- start start 2023-12-26T21:50:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba45ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb9c06b8c0 0x7fdb9c0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba45ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 0x7fdb9c154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba45ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 0x7fdb9c1548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba45ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c15d240 con 0x7fdb9c13c510 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba45ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c15e060 con 0x7fdb9c06b8c0 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba45ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c15d450 con 0x7fdb9c0cc810 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba2b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 0x7fdb9c1548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba2b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 0x7fdb9c1548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51844/0 (socket says 172.21.15.131:51844) 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba2b9c700 1 -- 172.21.15.131:0/2589691005 learned_addr learned my addr 172.21.15.131:0/2589691005 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 0x7fdb9c154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.504+0000 7fdba239b700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb9c06b8c0 0x7fdb9c0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdba2b9c700 1 -- 172.21.15.131:0/2589691005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 msgr2=0x7fdb9c154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdba2b9c700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 0x7fdb9c154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdba2b9c700 1 -- 172.21.15.131:0/2589691005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb9c06b8c0 msgr2=0x7fdb9c0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdba2b9c700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb9c06b8c0 0x7fdb9c0b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdba2b9c700 1 -- 172.21.15.131:0/2589691005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb98009480 con 0x7fdb9c13c510 2023-12-26T21:50:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdba2b9c700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 0x7fdb9c1548a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009e90 tx=0x7fdb9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdb927fc700 1 -- 172.21.15.131:0/2589691005 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400cb70 con 0x7fdb9c13c510 2023-12-26T21:50:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdb927fc700 1 -- 172.21.15.131:0/2589691005 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9400f040 con 0x7fdb9c13c510 2023-12-26T21:50:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdb927fc700 1 -- 172.21.15.131:0/2589691005 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94014480 con 0x7fdb9c13c510 2023-12-26T21:50:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdba45ff700 1 -- 172.21.15.131:0/2589691005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c15d730 con 0x7fdb9c13c510 2023-12-26T21:50:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.505+0000 7fdba45ff700 1 -- 172.21.15.131:0/2589691005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c15dcf0 con 0x7fdb9c13c510 2023-12-26T21:50:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.508+0000 7fdb927fc700 1 -- 172.21.15.131:0/2589691005 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdb94014830 con 0x7fdb9c13c510 2023-12-26T21:50:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.509+0000 7fdb927fc700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdb80066f30 0x7fdb800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.509+0000 7fdb927fc700 1 -- 172.21.15.131:0/2589691005 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdb94085440 con 0x7fdb9c13c510 2023-12-26T21:50:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.509+0000 7fdba239b700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdb80066f30 0x7fdb800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.510+0000 7fdba239b700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdb80066f30 0x7fdb800693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009260 tx=0x7fdb98016000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.510+0000 7fdbb2d7db80 1 Processor -- start 2023-12-26T21:50:16.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.511+0000 7fdbb2d7db80 1 -- start start 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.511+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc420fa0 0x5579fc421370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.511+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc4218b0 0x5579fc41cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.511+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 0x5579fc43d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.511+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579fc34f3e0 con 0x5579fc4218b0 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.511+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579fc34cbe0 con 0x5579fc41d170 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.511+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579fc3489d0 con 0x5579fc420fa0 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba1b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc4218b0 0x5579fc41cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba1b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc4218b0 0x5579fc41cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51846/0 (socket says 172.21.15.131:51846) 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba1b9a700 1 -- 172.21.15.131:0/909806660 learned_addr learned my addr 172.21.15.131:0/909806660 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba239b700 1 --2- 172.21.15.131:0/909806660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc420fa0 0x5579fc421370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba2b9c700 1 --2- 172.21.15.131:0/909806660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 0x5579fc43d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba1b9a700 1 -- 172.21.15.131:0/909806660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc420fa0 msgr2=0x5579fc421370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/909806660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc420fa0 0x5579fc421370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba1b9a700 1 -- 172.21.15.131:0/909806660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 msgr2=0x5579fc43d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/909806660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 0x5579fc43d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba1b9a700 1 -- 172.21.15.131:0/909806660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579fc1dc360 con 0x5579fc4218b0 2023-12-26T21:50:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.512+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/909806660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc4218b0 0x5579fc41cc30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0037b0 tx=0x7fdb8c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49db418a8c20329 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdb7f7fe700 1 -- 172.21.15.131:0/909806660 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0033b0 con 0x5579fc4218b0 2023-12-26T21:50:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdb7f7fe700 1 -- 172.21.15.131:0/909806660 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c0025d0 con 0x5579fc4218b0 2023-12-26T21:50:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdb7f7fe700 1 -- 172.21.15.131:0/909806660 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c002950 con 0x5579fc4218b0 2023-12-26T21:50:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/909806660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc4218b0 msgr2=0x5579fc41cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/909806660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc4218b0 0x5579fc41cc30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0037b0 tx=0x7fdb8c007ec0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/909806660 shutdown_connections 2023-12-26T21:50:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/909806660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 0x5579fc43d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/909806660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc420fa0 0x5579fc421370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/909806660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc4218b0 0x5579fc41cc30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/909806660 >> 172.21.15.131:0/909806660 conn(0x5579fc3f7440 msgr2=0x5579fc442dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/909806660 shutdown_connections 2023-12-26T21:50:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.513+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/909806660 wait complete. 2023-12-26T21:50:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.514+0000 7fdbb2d7db80 1 -- start start 2023-12-26T21:50:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.515+0000 7fdbb2d7db80 1 Processor -- start 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.516+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 0x5579fc443220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.516+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc41fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.516+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 0x5579fc41ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.516+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579fc2f51b0 con 0x5579fc420fa0 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.516+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579fc2f4e50 con 0x5579fc41d170 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.516+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579fc241e40 con 0x5579fc4218b0 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.516+0000 7fdbb2d7db80 10 client.0 _ll_register_callbacks cb 0x5579fc2447b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.516+0000 7fdba1b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc41fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba1b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc41fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51848/0 (socket says 172.21.15.131:51848) 2023-12-26T21:50:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba1b9a700 1 -- 172.21.15.131:0/4098330446 learned_addr learned my addr 172.21.15.131:0/4098330446 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba239b700 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 0x5579fc443220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba2b9c700 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 0x5579fc41ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba1b9a700 1 -- 172.21.15.131:0/4098330446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 msgr2=0x5579fc41ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 0x5579fc41ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba1b9a700 1 -- 172.21.15.131:0/4098330446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 msgr2=0x5579fc443220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 0x5579fc443220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba1b9a700 1 -- 172.21.15.131:0/4098330446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c0686c0 con 0x5579fc420fa0 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc41fa20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c014000 tx=0x7fdb8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9787a682376d5033 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdb7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdb7cff9700 1 -- 172.21.15.131:0/4098330446 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c002e50 con 0x5579fc420fa0 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdb7cff9700 1 -- 172.21.15.131:0/4098330446 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdb8c01e410 con 0x5579fc420fa0 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.517+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/4098330446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579fc3c9eb0 con 0x5579fc420fa0 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdb7cff9700 1 -- 172.21.15.131:0/4098330446 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb8c01e7e0 con 0x5579fc420fa0 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdbb2d7db80 10 client.5895 fetch_fsmap learned FSMap version 25 2023-12-26T21:50:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/4098330446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5579fc188b10 con 0x5579fc420fa0 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdb7cff9700 1 -- 172.21.15.131:0/4098330446 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5579fc3c9eb0 con 0x5579fc420fa0 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdb7cff9700 1 -- 172.21.15.131:0/4098330446 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdb8c012070 con 0x5579fc420fa0 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdbb2d7db80 10 client.5895 fetch_fsmap finished waiting for FSMap version 25 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdbb2d7db80 10 client.5895 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdbb2d7db80 20 client.5895 populate_metadata read hostname 'smithi131' 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] conn(0x5579fc438c60 0x5579fc439030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdbb2d7db80 4 client.5895 mds_command: new command op to 5739 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/4098330446 --> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579fc31f880 con 0x5579fc438c60 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.518+0000 7fdba239b700 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] conn(0x5579fc438c60 0x5579fc439030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.519+0000 7fdba239b700 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] conn(0x5579fc438c60 0x5579fc439030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784aea721ba1de96 server_cookie=d77281b25f7b4295 in_seq=0 out_seq=0 2023-12-26T21:50:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.519+0000 7fdb7cff9700 10 client.5895 ms_handle_connect on v2:172.21.15.131:6835/607340038 2023-12-26T21:50:16.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.519+0000 7fdb7cff9700 1 -- 172.21.15.131:0/4098330446 <== mds.0 v2:172.21.15.131:6835/607340038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5579fc31f880 con 0x5579fc438c60 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.519+0000 7fdb7cff9700 10 client.5895 handle_command_reply: tid=0 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 client.5895 shutdown 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/4098330446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 msgr2=0x5579fc41fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc41fa20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c014000 tx=0x7fdb8c015000 comp rx=0 tx=0).stop 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/4098330446 shutdown_connections 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] conn(0x5579fc438c60 0x5579fc439030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 0x5579fc41ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc41d170 0x5579fc443220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/4098330446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc41fa20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdb7cff9700 0 client.5895 ms_handle_reset on v2:172.21.15.131:6835/607340038 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/4098330446 >> 172.21.15.131:0/4098330446 conn(0x5579fc3f7440 msgr2=0x5579fc3fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/4098330446 shutdown_connections 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/4098330446 wait complete. 2023-12-26T21:50:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.520+0000 7fdbb2d7db80 20 client.5895 trim_cache size 0 max 16384 2023-12-26T21:50:16.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.537+0000 7fdbb2d7db80 1 Processor -- start 2023-12-26T21:50:16.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.538+0000 7fdbb2d7db80 1 -- start start 2023-12-26T21:50:16.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.538+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc41d170 0x5579fc41b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.538+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc420fa0 0x5579fc3fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.538+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 0x5579fc41f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.538+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579fc41ccb0 con 0x5579fc41d170 2023-12-26T21:50:16.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.538+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579fc43afa0 con 0x5579fc420fa0 2023-12-26T21:50:16.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.538+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579fc4391d0 con 0x5579fc4218b0 2023-12-26T21:50:16.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba239b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc41d170 0x5579fc41b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba239b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc41d170 0x5579fc41b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51864/0 (socket says 172.21.15.131:51864) 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba239b700 1 -- 172.21.15.131:0/1840745160 learned_addr learned my addr 172.21.15.131:0/1840745160 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba2b9c700 1 --2- 172.21.15.131:0/1840745160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 0x5579fc41f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/1840745160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc420fa0 0x5579fc3fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba239b700 1 -- 172.21.15.131:0/1840745160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 msgr2=0x5579fc41f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba239b700 1 --2- 172.21.15.131:0/1840745160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 0x5579fc41f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba239b700 1 -- 172.21.15.131:0/1840745160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc420fa0 msgr2=0x5579fc3fa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba239b700 1 --2- 172.21.15.131:0/1840745160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc420fa0 0x5579fc3fa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba239b700 1 -- 172.21.15.131:0/1840745160 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579fc3f6990 con 0x5579fc41d170 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdba239b700 1 --2- 172.21.15.131:0/1840745160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc41d170 0x5579fc41b940 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c06fe40 tx=0x7fdb9802b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab72cd044e1b0c35 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdb7ffff700 1 -- 172.21.15.131:0/1840745160 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98021070 con 0x5579fc41d170 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdb7ffff700 1 -- 172.21.15.131:0/1840745160 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98031040 con 0x5579fc41d170 2023-12-26T21:50:16.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.539+0000 7fdb7ffff700 1 -- 172.21.15.131:0/1840745160 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800bee0 con 0x5579fc41d170 2023-12-26T21:50:16.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.540+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/1840745160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc41d170 msgr2=0x5579fc41b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.540+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/1840745160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc41d170 0x5579fc41b940 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c06fe40 tx=0x7fdb9802b040 comp rx=0 tx=0).stop 2023-12-26T21:50:16.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.540+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/1840745160 shutdown_connections 2023-12-26T21:50:16.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.540+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/1840745160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc4218b0 0x5579fc41f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.540+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/1840745160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc41d170 0x5579fc41b940 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.540+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/1840745160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc420fa0 0x5579fc3fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.540+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/1840745160 >> 172.21.15.131:0/1840745160 conn(0x5579fc3f7440 msgr2=0x5579fc3fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:16.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.540+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/1840745160 shutdown_connections 2023-12-26T21:50:16.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.540+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/1840745160 wait complete. 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.541+0000 7fdbb2d7db80 1 -- start start 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.541+0000 7fdbb2d7db80 1 Processor -- start 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc41d170 0x5579fc437e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc3eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdbb2d7db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc4218b0 0x5579fc3ef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579fc4a54b0 con 0x5579fc420fa0 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579fc450eb0 con 0x5579fc4218b0 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdbb2d7db80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579fc451170 con 0x5579fc41d170 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdbb2d7db80 10 client.0 _ll_register_callbacks cb 0x7fdb8c0078b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba1b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc3eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba1b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc3eeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51878/0 (socket says 172.21.15.131:51878) 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba1b9a700 1 -- 172.21.15.131:0/835563225 learned_addr learned my addr 172.21.15.131:0/835563225 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba239b700 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc41d170 0x5579fc437e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba2b9c700 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc4218b0 0x5579fc3ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba1b9a700 1 -- 172.21.15.131:0/835563225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc41d170 msgr2=0x5579fc437e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc41d170 0x5579fc437e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba1b9a700 1 -- 172.21.15.131:0/835563225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc4218b0 msgr2=0x5579fc3ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc4218b0 0x5579fc3ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba1b9a700 1 -- 172.21.15.131:0/835563225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c068910 con 0x5579fc420fa0 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.542+0000 7fdba2b9c700 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc4218b0 0x5579fc3ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdba1b9a700 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc3eeee0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c007d00 tx=0x7fdb8c021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2e37591faf8f74 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdb7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdb7cff9700 1 -- 172.21.15.131:0/835563225 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c002b70 con 0x5579fc420fa0 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdb7cff9700 1 -- 172.21.15.131:0/835563225 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdb8c002cd0 con 0x5579fc420fa0 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/835563225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb9c16d820 con 0x5579fc420fa0 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdb7cff9700 1 -- 172.21.15.131:0/835563225 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb8c002e90 con 0x5579fc420fa0 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdbb2d7db80 10 client.5901 fetch_fsmap learned FSMap version 25 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/835563225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5579fc317290 con 0x5579fc420fa0 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdb7cff9700 1 -- 172.21.15.131:0/835563225 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9c16d820 con 0x5579fc420fa0 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.543+0000 7fdb7cff9700 1 -- 172.21.15.131:0/835563225 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdb8c01ece0 con 0x5579fc420fa0 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdbb2d7db80 10 client.5901 fetch_fsmap finished waiting for FSMap version 25 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdbb2d7db80 10 client.5901 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdbb2d7db80 20 client.5901 populate_metadata read hostname 'smithi131' 2023-12-26T21:50:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] conn(0x5579fc43d780 0x5579fc43db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdbb2d7db80 4 client.5901 mds_command: new command op to 5739 tid=0[{"prefix": "status"}] 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/835563225 --> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5579fc3ed490 con 0x5579fc43d780 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdba239b700 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] conn(0x5579fc43d780 0x5579fc43db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdba239b700 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] conn(0x5579fc43d780 0x5579fc43db50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880349827f935e48 server_cookie=3c2ec879561ce0a7 in_seq=0 out_seq=0 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdb7cff9700 10 client.5901 ms_handle_connect on v2:172.21.15.131:6835/607340038 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdb7cff9700 1 -- 172.21.15.131:0/835563225 <== mds.0 v2:172.21.15.131:6835/607340038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5579fc3ed490 con 0x5579fc43d780 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.544+0000 7fdb7cff9700 10 client.5901 handle_command_reply: tid=0 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 client.5901 shutdown 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/835563225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 msgr2=0x5579fc3eeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc3eeee0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c007d00 tx=0x7fdb8c021040 comp rx=0 tx=0).stop 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/835563225 shutdown_connections 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.131:6835/607340038,v1:172.21.15.131:6837/607340038] conn(0x5579fc43d780 0x5579fc43db50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5579fc4218b0 0x5579fc3ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5579fc41d170 0x5579fc437e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdb7cff9700 0 client.5901 ms_handle_reset on v2:172.21.15.131:6835/607340038 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 --2- 172.21.15.131:0/835563225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579fc420fa0 0x5579fc3eeee0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/835563225 >> 172.21.15.131:0/835563225 conn(0x5579fc3f7440 msgr2=0x5579fc3fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/835563225 shutdown_connections 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 1 -- 172.21.15.131:0/835563225 wait complete. 2023-12-26T21:50:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.545+0000 7fdbb2d7db80 20 client.5901 trim_cache size 0 max 16384 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5739, 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 11.331024791000001, 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 25, 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 64, 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 64, 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 17.049441111 2023-12-26T21:50:16.553 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:50:16.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 -- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdb80066f30 msgr2=0x7fdb800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdb80066f30 0x7fdb800693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009260 tx=0x7fdb98016000 comp rx=0 tx=0).stop 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 -- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 msgr2=0x7fdb9c1548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 0x7fdb9c1548a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009e90 tx=0x7fdb9400b040 comp rx=0 tx=0).stop 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 -- 172.21.15.131:0/2589691005 shutdown_connections 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb9c13c510 0x7fdb9c1548a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb9c06b8c0 0x7fdb9c0b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fdb80066f30 0x7fdb800693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 --2- 172.21.15.131:0/2589691005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb9c0cc810 0x7fdb9c154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 -- 172.21.15.131:0/2589691005 >> 172.21.15.131:0/2589691005 conn(0x7fdb9c0ce3e0 msgr2=0x7fdb9c152460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 -- 172.21.15.131:0/2589691005 shutdown_connections 2023-12-26T21:50:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.546+0000 7fdba45ff700 1 -- 172.21.15.131:0/2589691005 wait complete. 2023-12-26T21:50:16.562 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:50:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.752+0000 7ff716dff700 1 Processor -- start 2023-12-26T21:50:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.752+0000 7ff716dff700 1 -- start start 2023-12-26T21:50:16.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.753+0000 7ff716dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100b5310 0x7ff7100b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff716dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7100afcd0 con 0x7ff7101596a0 2023-12-26T21:50:16.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff716dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7100b3e70 con 0x7ff710159a70 2023-12-26T21:50:16.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff716dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100b0f20 con 0x7ff7100b5310 2023-12-26T21:50:16.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff714b9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff7101596a0 0x7ff7100b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56296/0 (socket says 172.21.15.131:56296) 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff714b9b700 1 -- 172.21.15.131:0/3325392791 learned_addr learned my addr 172.21.15.131:0/3325392791 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff71539c700 1 --2- 172.21.15.131:0/3325392791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100b5310 0x7ff7100b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3325392791 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284749783 0 0) 0x7ff7100afcd0 con 0x7ff7101596a0 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3325392791 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff700003570 con 0x7ff7101596a0 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3325392791 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184420384 0 0) 0x7ff7100b3e70 con 0x7ff710159a70 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3325392791 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7100afcd0 con 0x7ff710159a70 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff71539c700 1 -- 172.21.15.131:0/3325392791 >> v1:172.21.15.190:6789/0 conn(0x7ff710159a70 legacy=0x7ff7100b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff71539c700 1 -- 172.21.15.131:0/3325392791 >> v1:172.21.15.131:6789/0 conn(0x7ff7101596a0 legacy=0x7ff7100b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3325392791 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1097412732 0 0) 0x7ff700003570 con 0x7ff7101596a0 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.754+0000 7ff71539c700 1 -- 172.21.15.131:0/3325392791 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100b3540 con 0x7ff7100b5310 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.755+0000 7ff71539c700 1 --2- 172.21.15.131:0/3325392791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100b5310 0x7ff7100b0870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff708009e90 tx=0x7ff708009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c261ef8f4de8c436 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.755+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3325392791 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70800e070 con 0x7ff7100b5310 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.755+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3325392791 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70800b870 con 0x7ff7100b5310 2023-12-26T21:50:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.755+0000 7ff70f7fe700 1 -- 172.21.15.131:0/3325392791 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704001470 con 0x7ff7100b5310 2023-12-26T21:50:16.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.755+0000 7ff716dff700 1 -- 172.21.15.131:0/3325392791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100b5310 msgr2=0x7ff7100b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.756+0000 7ff716dff700 1 --2- 172.21.15.131:0/3325392791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100b5310 0x7ff7100b0870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff708009e90 tx=0x7ff708009780 comp rx=0 tx=0).stop 2023-12-26T21:50:16.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.756+0000 7ff716dff700 1 -- 172.21.15.131:0/3325392791 shutdown_connections 2023-12-26T21:50:16.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.756+0000 7ff716dff700 1 --2- 172.21.15.131:0/3325392791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7100b5310 0x7ff7100b0870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.756+0000 7ff716dff700 1 -- 172.21.15.131:0/3325392791 >> 172.21.15.131:0/3325392791 conn(0x7ff71006ece0 msgr2=0x7ff71006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:16.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.756+0000 7ff716dff700 1 -- 172.21.15.131:0/3325392791 shutdown_connections 2023-12-26T21:50:16.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.756+0000 7ff716dff700 1 -- 172.21.15.131:0/3325392791 wait complete. 2023-12-26T21:50:16.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.757+0000 7ff716dff700 1 Processor -- start 2023-12-26T21:50:16.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.757+0000 7ff716dff700 1 -- start start 2023-12-26T21:50:16.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.758+0000 7ff716dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7100b5310 0x7ff7100c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.758+0000 7ff716dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7101596a0 0x7ff71015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.758+0000 7ff716dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710159a70 0x7ff710155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.758+0000 7ff716dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100cb200 con 0x7ff7100b5310 2023-12-26T21:50:16.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.758+0000 7ff716dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100cbd80 con 0x7ff7101596a0 2023-12-26T21:50:16.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.758+0000 7ff716dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100cb450 con 0x7ff710159a70 2023-12-26T21:50:16.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.758+0000 7ff714b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7100b5310 0x7ff7100c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.758+0000 7ff714b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7100b5310 0x7ff7100c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51888/0 (socket says 172.21.15.131:51888) 2023-12-26T21:50:16.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.758+0000 7ff714b9b700 1 -- 172.21.15.131:0/4202490749 learned_addr learned my addr 172.21.15.131:0/4202490749 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:16.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff71539c700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710159a70 0x7ff710155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff70ffff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7101596a0 0x7ff71015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff714b9b700 1 -- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710159a70 msgr2=0x7ff710155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff714b9b700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710159a70 0x7ff710155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff714b9b700 1 -- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7101596a0 msgr2=0x7ff71015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:16.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff714b9b700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7101596a0 0x7ff71015c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff714b9b700 1 -- 172.21.15.131:0/4202490749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff708009a20 con 0x7ff7100b5310 2023-12-26T21:50:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff71539c700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710159a70 0x7ff710155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:50:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff70ffff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7101596a0 0x7ff71015c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:50:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff714b9b700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7100b5310 0x7ff7100c8620 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff70400cbc0 tx=0x7ff704001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff70cff9700 1 -- 172.21.15.131:0/4202490749 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704007360 con 0x7ff7100b5310 2023-12-26T21:50:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff70cff9700 1 -- 172.21.15.131:0/4202490749 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff704007500 con 0x7ff7100b5310 2023-12-26T21:50:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7100cb730 con 0x7ff7100b5310 2023-12-26T21:50:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.759+0000 7ff70cff9700 1 -- 172.21.15.131:0/4202490749 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704004a80 con 0x7ff7100b5310 2023-12-26T21:50:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.760+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7100c8b60 con 0x7ff7100b5310 2023-12-26T21:50:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.761+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7100cc060 con 0x7ff7100b5310 2023-12-26T21:50:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.761+0000 7ff70cff9700 1 -- 172.21.15.131:0/4202490749 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff70400a510 con 0x7ff7100b5310 2023-12-26T21:50:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.762+0000 7ff70cff9700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff6f4066f30 0x7ff6f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.762+0000 7ff70cff9700 1 -- 172.21.15.131:0/4202490749 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff704083e50 con 0x7ff7100b5310 2023-12-26T21:50:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.762+0000 7ff70ffff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff6f4066f30 0x7ff6f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.763+0000 7ff70ffff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff6f4066f30 0x7ff6f40693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc007fd0 tx=0x7ff6fc009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:16.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:16.766+0000 7ff70cff9700 1 -- 172.21.15.131:0/4202490749 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff70404c5a0 con 0x7ff7100b5310 2023-12-26T21:50:17.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.004+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff7100657e0 con 0x7ff7100b5310 2023-12-26T21:50:17.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.004+0000 7ff70cff9700 1 -- 172.21.15.131:0/4202490749 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff704018070 con 0x7ff7100b5310 2023-12-26T21:50:17.004 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:50:17.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff6f4066f30 msgr2=0x7ff6f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff6f4066f30 0x7ff6f40693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc007fd0 tx=0x7ff6fc009040 comp rx=0 tx=0).stop 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7100b5310 msgr2=0x7ff7100c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7100b5310 0x7ff7100c8620 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff70400cbc0 tx=0x7ff704001ab0 comp rx=0 tx=0).stop 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 shutdown_connections 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff6f4066f30 0x7ff6f40693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff710159a70 0x7ff710155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7100b5310 0x7ff7100c8620 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 --2- 172.21.15.131:0/4202490749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7101596a0 0x7ff71015c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 >> 172.21.15.131:0/4202490749 conn(0x7ff71006ece0 msgr2=0x7ff71015db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 shutdown_connections 2023-12-26T21:50:17.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:17.007+0000 7ff716dff700 1 -- 172.21.15.131:0/4202490749 wait complete. 2023-12-26T21:50:17.024 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T21:50:17.024 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:50:17.024 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T21:50:17.024 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T21:50:17.024 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T21:50:17.024 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T21:50:17.024 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T21:50:17.025 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:50:17.025 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:50:17.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:50:17.025 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T21:50:17.069 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T21:50:17.070 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:50:17.071 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:50:17.071 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:50:17.073 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:50:17.073 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:50:17.073 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41639sec preferred_lft 41639sec 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:50:17.074 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T21:50:17.075 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T21:50:17.075 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:50:17.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:50:17.075 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:50:17.175 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T21:50:17.175 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:50:17.180 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:50:17.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:50:17.181 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T21:50:17.270 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T21:50:17.270 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:50:17.270 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:50:17.270 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:50:17.270 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:50:17.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:50:17.271 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:50:17.271 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:50:17.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:50:17.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:50:17.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:50:17.272 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:50:17.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:50:17.442 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:50:17.486 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T21:50:17.486 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T21:50:17.486 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:50:17.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:50:17.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T21:50:17.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:50:17.596 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T21:50:17.639 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:50:17.639 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:50:17.639 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:50:17.721 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T21:50:17.722 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:50:17.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:50:17.860 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:50:17.860 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:50:17.889 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:50:17.889 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:50:17.889 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:50:17.890 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:50:17.890 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:50:17.893 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:50:17.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:50:17.894 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:50:17.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T21:50:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.153+0000 7f332e7ff700 1 Processor -- start 2023-12-26T21:50:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.153+0000 7f332e7ff700 1 -- start start 2023-12-26T21:50:18.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.155+0000 7f332e7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f33280cc810 0x7f33280c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.155+0000 7f332e7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33280686c0 con 0x7f332806bfd0 2023-12-26T21:50:18.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.155+0000 7f332e7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3328069520 con 0x7f332813c510 2023-12-26T21:50:18.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.155+0000 7f332e7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3328068910 con 0x7f33280cc810 2023-12-26T21:50:18.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.155+0000 7f33277fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f332806bfd0 0x7f33280cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56300/0 (socket says 172.21.15.131:56300) 2023-12-26T21:50:18.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.155+0000 7f33277fe700 1 -- 172.21.15.131:0/4123624857 learned_addr learned my addr 172.21.15.131:0/4123624857 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:18.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.155+0000 7f332cd9c700 1 --2- 172.21.15.131:0/4123624857 >> v2:172.21.15.131:3300/0 conn(0x7f33280cc810 0x7f33280c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.155+0000 7f3326ffd700 1 -- 172.21.15.131:0/4123624857 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524719578 0 0) 0x7f33280686c0 con 0x7f332806bfd0 2023-12-26T21:50:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.155+0000 7f3326ffd700 1 -- 172.21.15.131:0/4123624857 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3310003570 con 0x7f332806bfd0 2023-12-26T21:50:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.156+0000 7f332cd9c700 1 -- 172.21.15.131:0/4123624857 >> v1:172.21.15.190:6789/0 conn(0x7f332813c510 legacy=0x7f332806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.156+0000 7f332cd9c700 1 -- 172.21.15.131:0/4123624857 >> v1:172.21.15.131:6789/0 conn(0x7f332806bfd0 legacy=0x7f33280cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.156+0000 7f332cd9c700 1 -- 172.21.15.131:0/4123624857 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3328068bf0 con 0x7f33280cc810 2023-12-26T21:50:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.156+0000 7f332cd9c700 1 --2- 172.21.15.131:0/4123624857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33280cc810 0x7f33280c8a20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3320009e60 tx=0x7f3320009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37f7a6e7e83aaef server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.156+0000 7f3326ffd700 1 -- 172.21.15.131:0/4123624857 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332000e070 con 0x7f33280cc810 2023-12-26T21:50:18.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.156+0000 7f3326ffd700 1 -- 172.21.15.131:0/4123624857 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332000bb60 con 0x7f33280cc810 2023-12-26T21:50:18.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.156+0000 7f3326ffd700 1 -- 172.21.15.131:0/4123624857 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320012430 con 0x7f33280cc810 2023-12-26T21:50:18.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.157+0000 7f332e7ff700 1 -- 172.21.15.131:0/4123624857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33280cc810 msgr2=0x7f33280c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.157+0000 7f332e7ff700 1 --2- 172.21.15.131:0/4123624857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33280cc810 0x7f33280c8a20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3320009e60 tx=0x7f3320009780 comp rx=0 tx=0).stop 2023-12-26T21:50:18.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.157+0000 7f332e7ff700 1 -- 172.21.15.131:0/4123624857 shutdown_connections 2023-12-26T21:50:18.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.157+0000 7f332e7ff700 1 --2- 172.21.15.131:0/4123624857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33280cc810 0x7f33280c8a20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.157+0000 7f332e7ff700 1 -- 172.21.15.131:0/4123624857 >> 172.21.15.131:0/4123624857 conn(0x7f33280ce3e0 msgr2=0x7f33280ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:18.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.157+0000 7f332e7ff700 1 -- 172.21.15.131:0/4123624857 shutdown_connections 2023-12-26T21:50:18.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.157+0000 7f332e7ff700 1 -- 172.21.15.131:0/4123624857 wait complete. 2023-12-26T21:50:18.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.158+0000 7f332e7ff700 1 Processor -- start 2023-12-26T21:50:18.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.158+0000 7f332e7ff700 1 -- start start 2023-12-26T21:50:18.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.159+0000 7f332e7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f332806bfd0 0x7f33281518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.159+0000 7f332e7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33280cc810 0x7f3328154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.159+0000 7f332e7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f332813c510 0x7f3328158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.159+0000 7f332e7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33281584e0 con 0x7f332806bfd0 2023-12-26T21:50:18.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.159+0000 7f332e7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33281578d0 con 0x7f33280cc810 2023-12-26T21:50:18.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.159+0000 7f332e7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3328157bb0 con 0x7f332813c510 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.159+0000 7f3327fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f332806bfd0 0x7f33281518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f3327fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f332806bfd0 0x7f33281518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51898/0 (socket says 172.21.15.131:51898) 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f3327fff700 1 -- 172.21.15.131:0/849749762 learned_addr learned my addr 172.21.15.131:0/849749762 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f33277fe700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33280cc810 0x7f3328154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f332cd9c700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f332813c510 0x7f3328158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f3327fff700 1 -- 172.21.15.131:0/849749762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f332813c510 msgr2=0x7f3328158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f3327fff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f332813c510 0x7f3328158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f3327fff700 1 -- 172.21.15.131:0/849749762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33280cc810 msgr2=0x7f3328154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f3327fff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33280cc810 0x7f3328154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f3327fff700 1 -- 172.21.15.131:0/849749762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3320009a20 con 0x7f332806bfd0 2023-12-26T21:50:18.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f3327fff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f332806bfd0 0x7f33281518d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f331800c790 tx=0x7f3318007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:18.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f330ffff700 1 -- 172.21.15.131:0/849749762 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318011590 con 0x7f332806bfd0 2023-12-26T21:50:18.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f330ffff700 1 -- 172.21.15.131:0/849749762 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33180116f0 con 0x7f332806bfd0 2023-12-26T21:50:18.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3328157e90 con 0x7f332806bfd0 2023-12-26T21:50:18.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.160+0000 7f330ffff700 1 -- 172.21.15.131:0/849749762 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318011ab0 con 0x7f332806bfd0 2023-12-26T21:50:18.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.161+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3328151e10 con 0x7f332806bfd0 2023-12-26T21:50:18.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.162+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33280657e0 con 0x7f332806bfd0 2023-12-26T21:50:18.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.162+0000 7f330ffff700 1 -- 172.21.15.131:0/849749762 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f331801b3b0 con 0x7f332806bfd0 2023-12-26T21:50:18.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.163+0000 7f330ffff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3314066f30 0x7f33140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.163+0000 7f330ffff700 1 -- 172.21.15.131:0/849749762 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f33180839d0 con 0x7f332806bfd0 2023-12-26T21:50:18.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.164+0000 7f33277fe700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3314066f30 0x7f33140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.164+0000 7f33277fe700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3314066f30 0x7f33140693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3328153640 tx=0x7f331c009040 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:18.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.167+0000 7f330ffff700 1 -- 172.21.15.131:0/849749762 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3318050b80 con 0x7f332806bfd0 2023-12-26T21:50:18.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.386+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3328158620 con 0x7f3314066f30 2023-12-26T21:50:18.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.388+0000 7f330ffff700 1 -- 172.21.15.131:0/849749762 <== mgr.5437 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f331c00f620 con 0x7f3314066f30 2023-12-26T21:50:18.387 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:50:18.387 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:50:18.387 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T21:50:18.387 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:50:18.387 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3314066f30 msgr2=0x7f33140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3314066f30 0x7f33140693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3328153640 tx=0x7f331c009040 comp rx=0 tx=0).stop 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f332806bfd0 msgr2=0x7f33281518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f332806bfd0 0x7f33281518d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f331800c790 tx=0x7f3318007f20 comp rx=0 tx=0).stop 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 shutdown_connections 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f3314066f30 0x7f33140693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f332813c510 0x7f3328158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f332806bfd0 0x7f33281518d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 --2- 172.21.15.131:0/849749762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33280cc810 0x7f3328154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 >> 172.21.15.131:0/849749762 conn(0x7f33280ce3e0 msgr2=0x7f33280c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 shutdown_connections 2023-12-26T21:50:18.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.390+0000 7f332e7ff700 1 -- 172.21.15.131:0/849749762 wait complete. 2023-12-26T21:50:18.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T21:50:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.599+0000 7f912abff700 1 Processor -- start 2023-12-26T21:50:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.600+0000 7f912abff700 1 -- start start 2023-12-26T21:50:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f912abff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f91240cb320 0x7f9124069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f912abff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f912406a1e0 0x7f91240b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f912abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240b0c90 0x7f91240aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f912abff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91240c8cc0 con 0x7f912406a1e0 2023-12-26T21:50:18.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f912abff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91240ad700 con 0x7f91240cb320 2023-12-26T21:50:18.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f912abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91240b1110 con 0x7f91240b0c90 2023-12-26T21:50:18.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f9123fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f912406a1e0 0x7f91240b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f9123fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f912406a1e0 0x7f91240b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51908/0 (socket says 172.21.15.131:51908) 2023-12-26T21:50:18.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f9123fff700 1 -- 172.21.15.131:0/2832717086 learned_addr learned my addr 172.21.15.131:0/2832717086 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.601+0000 7f912919c700 1 --2- 172.21.15.131:0/2832717086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240b0c90 0x7f91240aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f912899b700 1 --2- 172.21.15.131:0/2832717086 >> v2:172.21.15.190:3300/0 conn(0x7f91240cb320 0x7f9124069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f9123fff700 1 -- 172.21.15.131:0/2832717086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240b0c90 msgr2=0x7f91240aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f9123fff700 1 --2- 172.21.15.131:0/2832717086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240b0c90 0x7f91240aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f9123fff700 1 -- 172.21.15.131:0/2832717086 >> v2:172.21.15.190:3300/0 conn(0x7f91240cb320 msgr2=0x7f9124069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f9123fff700 1 --2- 172.21.15.131:0/2832717086 >> v2:172.21.15.190:3300/0 conn(0x7f91240cb320 0x7f9124069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f9123fff700 1 -- 172.21.15.131:0/2832717086 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124156940 con 0x7f912406a1e0 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f9123fff700 1 --2- 172.21.15.131:0/2832717086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 0x7f91240b0750 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9124064410 tx=0x7f9110009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981a160c50d52cad server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f91237fe700 1 -- 172.21.15.131:0/2832717086 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000b820 con 0x7f912406a1e0 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f91237fe700 1 -- 172.21.15.131:0/2832717086 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911000b980 con 0x7f912406a1e0 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.602+0000 7f91237fe700 1 -- 172.21.15.131:0/2832717086 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000bd20 con 0x7f912406a1e0 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.603+0000 7f912abff700 1 -- 172.21.15.131:0/2832717086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 msgr2=0x7f91240b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.603+0000 7f912abff700 1 --2- 172.21.15.131:0/2832717086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 0x7f91240b0750 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9124064410 tx=0x7f9110009250 comp rx=0 tx=0).stop 2023-12-26T21:50:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.603+0000 7f912abff700 1 -- 172.21.15.131:0/2832717086 shutdown_connections 2023-12-26T21:50:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.603+0000 7f912abff700 1 --2- 172.21.15.131:0/2832717086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240b0c90 0x7f91240aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.603+0000 7f912abff700 1 --2- 172.21.15.131:0/2832717086 >> v2:172.21.15.190:3300/0 conn(0x7f91240cb320 0x7f9124069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.603+0000 7f912abff700 1 --2- 172.21.15.131:0/2832717086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 0x7f91240b0750 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.603+0000 7f912abff700 1 -- 172.21.15.131:0/2832717086 >> 172.21.15.131:0/2832717086 conn(0x7f9124068040 msgr2=0x7f9124069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.603+0000 7f912abff700 1 -- 172.21.15.131:0/2832717086 shutdown_connections 2023-12-26T21:50:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.603+0000 7f912abff700 1 -- 172.21.15.131:0/2832717086 wait complete. 2023-12-26T21:50:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.604+0000 7f912abff700 1 Processor -- start 2023-12-26T21:50:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.605+0000 7f912abff700 1 -- start start 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.605+0000 7f912abff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 0x7f9124154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.605+0000 7f912abff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91240b0c90 0x7f91240aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.605+0000 7f912abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240cb320 0x7f9124169500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.605+0000 7f912abff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9124155480 con 0x7f912406a1e0 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.605+0000 7f912abff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912415b0d0 con 0x7f91240b0c90 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.605+0000 7f912abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912415a4c0 con 0x7f91240cb320 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912899b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 0x7f9124154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912899b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 0x7f9124154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51912/0 (socket says 172.21.15.131:51912) 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912899b700 1 -- 172.21.15.131:0/258397719 learned_addr learned my addr 172.21.15.131:0/258397719 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f9123fff700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91240b0c90 0x7f91240aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912919c700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240cb320 0x7f9124169500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912899b700 1 -- 172.21.15.131:0/258397719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240cb320 msgr2=0x7f9124169500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912899b700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240cb320 0x7f9124169500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912899b700 1 -- 172.21.15.131:0/258397719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91240b0c90 msgr2=0x7f91240aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912899b700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91240b0c90 0x7f91240aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912899b700 1 -- 172.21.15.131:0/258397719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100094f0 con 0x7f912406a1e0 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f912899b700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 0x7f9124154f40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f911800d860 tx=0x7f9118008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.606+0000 7f9120ff9700 1 -- 172.21.15.131:0/258397719 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118009610 con 0x7f912406a1e0 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.607+0000 7f9120ff9700 1 -- 172.21.15.131:0/258397719 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9118009770 con 0x7f912406a1e0 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.607+0000 7f9120ff9700 1 -- 172.21.15.131:0/258397719 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118009b30 con 0x7f912406a1e0 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.607+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912415a7a0 con 0x7f912406a1e0 2023-12-26T21:50:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.607+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912415ad60 con 0x7f912406a1e0 2023-12-26T21:50:18.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.608+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912415b3b0 con 0x7f912406a1e0 2023-12-26T21:50:18.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.608+0000 7f9120ff9700 1 -- 172.21.15.131:0/258397719 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f911801e460 con 0x7f912406a1e0 2023-12-26T21:50:18.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.610+0000 7f9120ff9700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9108066f30 0x7f91080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:18.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.610+0000 7f9120ff9700 1 -- 172.21.15.131:0/258397719 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9118084db0 con 0x7f912406a1e0 2023-12-26T21:50:18.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.610+0000 7f9123fff700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9108066f30 0x7f91080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:18.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.611+0000 7f9123fff700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9108066f30 0x7f91080693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9124064410 tx=0x7f9110017000 comp rx=0 tx=0).ready entity=mgr.5437 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.614+0000 7f9120ff9700 1 -- 172.21.15.131:0/258397719 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9118089070 con 0x7f912406a1e0 2023-12-26T21:50:18.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:18.859+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f91240657e0 con 0x7f912406a1e0 2023-12-26T21:50:19.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.279+0000 7f9120ff9700 1 -- 172.21.15.131:0/258397719 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 18) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f91180521e0 con 0x7f912406a1e0 2023-12-26T21:50:19.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.280+0000 7f9120ff9700 1 -- 172.21.15.131:0/258397719 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7f91180558a0 con 0x7f912406a1e0 2023-12-26T21:50:19.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9108066f30 msgr2=0x7f91080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9108066f30 0x7f91080693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9124064410 tx=0x7f9110017000 comp rx=0 tx=0).stop 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 msgr2=0x7f9124154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 0x7f9124154f40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f911800d860 tx=0x7f9118008040 comp rx=0 tx=0).stop 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 shutdown_connections 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f9108066f30 0x7f91080693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91240cb320 0x7f9124169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912406a1e0 0x7f9124154f40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 --2- 172.21.15.131:0/258397719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91240b0c90 0x7f91240aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 >> 172.21.15.131:0/258397719 conn(0x7f9124068040 msgr2=0x7f9124069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 shutdown_connections 2023-12-26T21:50:19.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.284+0000 7f912abff700 1 -- 172.21.15.131:0/258397719 wait complete. 2023-12-26T21:50:19.306 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T21:50:19.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.506+0000 7f17339ff700 1 Processor -- start 2023-12-26T21:50:19.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.507+0000 7f17339ff700 1 -- start start 2023-12-26T21:50:19.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f17339ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f17339ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f172c0cd4d0 0x7f172c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f17339ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f172c1591f0 0x7f172c159640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f17339ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172c0cbe50 con 0x7f172c0cd4d0 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f17339ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172c15d330 con 0x7f172c1591f0 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f17339ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0d0470 con 0x7f172c0cd100 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f1730f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f172c0cd4d0 0x7f172c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f1731f9c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f172c1591f0 0x7f172c159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f173179b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f1730f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f172c0cd4d0 0x7f172c0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51934/0 (socket says 172.21.15.131:51934) 2023-12-26T21:50:19.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.508+0000 7f1730f9a700 1 -- 172.21.15.131:0/495802067 learned_addr learned my addr 172.21.15.131:0/495802067 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f173179b700 1 -- 172.21.15.131:0/495802067 >> v2:172.21.15.190:3300/0 conn(0x7f172c1591f0 msgr2=0x7f172c159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f173179b700 1 --2- 172.21.15.131:0/495802067 >> v2:172.21.15.190:3300/0 conn(0x7f172c1591f0 0x7f172c159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f173179b700 1 -- 172.21.15.131:0/495802067 >> v2:172.21.15.131:3300/0 conn(0x7f172c0cd4d0 msgr2=0x7f172c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f173179b700 1 --2- 172.21.15.131:0/495802067 >> v2:172.21.15.131:3300/0 conn(0x7f172c0cd4d0 0x7f172c0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f173179b700 1 -- 172.21.15.131:0/495802067 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c15db90 con 0x7f172c0cd100 2023-12-26T21:50:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f1731f9c700 1 --2- 172.21.15.131:0/495802067 >> v2:172.21.15.190:3300/0 conn(0x7f172c1591f0 0x7f172c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:50:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f173179b700 1 --2- 172.21.15.131:0/495802067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c0cb590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f172c0cf8d0 tx=0x7f17280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c8f24bcc13a663 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:19.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f1723fff700 1 -- 172.21.15.131:0/495802067 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728011630 con 0x7f172c0cd100 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f1723fff700 1 -- 172.21.15.131:0/495802067 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17280117b0 con 0x7f172c0cd100 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.509+0000 7f1723fff700 1 -- 172.21.15.131:0/495802067 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728011b70 con 0x7f172c0cd100 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.510+0000 7f17339ff700 1 -- 172.21.15.131:0/495802067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 msgr2=0x7f172c0cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.510+0000 7f17339ff700 1 --2- 172.21.15.131:0/495802067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c0cb590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f172c0cf8d0 tx=0x7f17280091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.510+0000 7f17339ff700 1 -- 172.21.15.131:0/495802067 shutdown_connections 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.510+0000 7f17339ff700 1 --2- 172.21.15.131:0/495802067 >> v2:172.21.15.190:3300/0 conn(0x7f172c1591f0 0x7f172c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.510+0000 7f17339ff700 1 --2- 172.21.15.131:0/495802067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c0cb590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.510+0000 7f17339ff700 1 --2- 172.21.15.131:0/495802067 >> v2:172.21.15.131:3300/0 conn(0x7f172c0cd4d0 0x7f172c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.510+0000 7f17339ff700 1 -- 172.21.15.131:0/495802067 >> 172.21.15.131:0/495802067 conn(0x7f172c06ece0 msgr2=0x7f172c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.510+0000 7f17339ff700 1 -- 172.21.15.131:0/495802067 shutdown_connections 2023-12-26T21:50:19.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.510+0000 7f17339ff700 1 -- 172.21.15.131:0/495802067 wait complete. 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.512+0000 7f17339ff700 1 Processor -- start 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.512+0000 7f17339ff700 1 -- start start 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f17339ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c14cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f17339ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f172c0cd4d0 0x7f172c0c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f17339ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f172c1591f0 0x7f172c0c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f17339ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0b4120 con 0x7f172c1591f0 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f17339ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0b3510 con 0x7f172c0cd4d0 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f17339ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0b37f0 con 0x7f172c0cd100 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f173179b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c14cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f1731f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f172c1591f0 0x7f172c0c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f173179b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c14cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:40434/0 (socket says 172.21.15.131:40434) 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f173179b700 1 -- 172.21.15.131:0/1675145805 learned_addr learned my addr 172.21.15.131:0/1675145805 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:19.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f1730f9a700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f172c0cd4d0 0x7f172c0c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f173179b700 1 -- 172.21.15.131:0/1675145805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f172c0cd4d0 msgr2=0x7f172c0c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f173179b700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f172c0cd4d0 0x7f172c0c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f173179b700 1 -- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f172c1591f0 msgr2=0x7f172c0c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f173179b700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f172c1591f0 0x7f172c0c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.513+0000 7f173179b700 1 -- 172.21.15.131:0/1675145805 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1724009a20 con 0x7f172c0cd100 2023-12-26T21:50:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.514+0000 7f173179b700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c14cc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1728013000 tx=0x7f1728017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.514+0000 7f17217fa700 1 -- 172.21.15.131:0/1675145805 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172801f070 con 0x7f172c0cd100 2023-12-26T21:50:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.514+0000 7f17217fa700 1 -- 172.21.15.131:0/1675145805 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1728011b30 con 0x7f172c0cd100 2023-12-26T21:50:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.514+0000 7f17217fa700 1 -- 172.21.15.131:0/1675145805 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728023660 con 0x7f172c0cd100 2023-12-26T21:50:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.514+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1728009420 con 0x7f172c0cd100 2023-12-26T21:50:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.514+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c0b3db0 con 0x7f172c0cd100 2023-12-26T21:50:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.515+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172c0657e0 con 0x7f172c0cd100 2023-12-26T21:50:19.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.516+0000 7f17217fa700 1 -- 172.21.15.131:0/1675145805 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 18) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f172802c3b0 con 0x7f172c0cd100 2023-12-26T21:50:19.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.517+0000 7f17217fa700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1710066f80 0x7f1710069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.517+0000 7f17217fa700 1 -- 172.21.15.131:0/1675145805 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f17280858f0 con 0x7f172c0cd100 2023-12-26T21:50:19.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.517+0000 7f1730f9a700 1 -- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1710066f80 msgr2=0x7f1710069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:50:19.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.517+0000 7f1730f9a700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1710066f80 0x7f1710069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:50:19.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.526+0000 7f17217fa700 1 -- 172.21.15.131:0/1675145805 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1728052a10 con 0x7f172c0cd100 2023-12-26T21:50:19.715 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:19.715+0000 7f34a82891c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:50:19.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.718+0000 7f1730f9a700 1 -- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1710066f80 msgr2=0x7f1710069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:50:19.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.718+0000 7f1730f9a700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1710066f80 0x7f1710069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:50:19.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.747+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f172c0ce420 con 0x7f172c0cd100 2023-12-26T21:50:19.751 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.752+0000 7f17217fa700 1 -- 172.21.15.131:0/1675145805 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f17280150d0 con 0x7f172c0cd100 2023-12-26T21:50:19.752 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:19.753+0000 7f1711f191c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:50:19.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1710066f80 msgr2=0x7f1710069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:50:19.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1710066f80 0x7f1710069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 msgr2=0x7f172c14cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c14cc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1728013000 tx=0x7f1728017000 comp rx=0 tx=0).stop 2023-12-26T21:50:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 shutdown_connections 2023-12-26T21:50:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f172c1591f0 0x7f172c0c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1710066f80 0x7f1710069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172c0cd100 0x7f172c14cc90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 --2- 172.21.15.131:0/1675145805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f172c0cd4d0 0x7f172c0c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 >> 172.21.15.131:0/1675145805 conn(0x7f172c06ece0 msgr2=0x7f172c0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 shutdown_connections 2023-12-26T21:50:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.755+0000 7f17339ff700 1 -- 172.21.15.131:0/1675145805 wait complete. 2023-12-26T21:50:19.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T21:50:19.819 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:19.819+0000 7f34a82891c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:50:19.870 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:19.870+0000 7f1711f191c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:50:19.906 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:19.906+0000 7f34a82891c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:50:19.961 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:19.962+0000 7f1711f191c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:50:19.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.975+0000 7f20795ff700 1 Processor -- start 2023-12-26T21:50:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.975+0000 7f20795ff700 1 -- start start 2023-12-26T21:50:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.976+0000 7f20795ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f20741599d0 0x7f20740ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.976+0000 7f20795ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740d01e0 0x7f20740ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.976+0000 7f20795ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20740cfd50 con 0x7f2074159600 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.976+0000 7f20795ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20740b1280 con 0x7f20741599d0 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.976+0000 7f20795ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20740cf140 con 0x7f20740d01e0 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.977+0000 7f20727fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f20741599d0 0x7f20740ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.977+0000 7f20727fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f20741599d0 0x7f20740ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51944/0 (socket says 172.21.15.131:51944) 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.977+0000 7f20727fc700 1 -- 172.21.15.131:0/3558838630 learned_addr learned my addr 172.21.15.131:0/3558838630 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.977+0000 7f20737fe700 1 --2- 172.21.15.131:0/3558838630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740d01e0 0x7f20740ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.977+0000 7f20727fc700 1 -- 172.21.15.131:0/3558838630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740d01e0 msgr2=0x7f20740ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.977+0000 7f20727fc700 1 --2- 172.21.15.131:0/3558838630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740d01e0 0x7f20740ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.977+0000 7f20727fc700 1 -- 172.21.15.131:0/3558838630 >> v1:172.21.15.131:6789/0 conn(0x7f2074159600 legacy=0x7f207415dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.977+0000 7f20727fc700 1 -- 172.21.15.131:0/3558838630 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20740cf420 con 0x7f20741599d0 2023-12-26T21:50:19.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.977+0000 7f20727fc700 1 --2- 172.21.15.131:0/3558838630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20741599d0 0x7f20740ceb60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f20680099b0 tx=0x7f2068009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16822b731ddcc0fb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:19.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f2071ffb700 1 -- 172.21.15.131:0/3558838630 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068009ea0 con 0x7f20741599d0 2023-12-26T21:50:19.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f2071ffb700 1 -- 172.21.15.131:0/3558838630 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206800b860 con 0x7f20741599d0 2023-12-26T21:50:19.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f2071ffb700 1 -- 172.21.15.131:0/3558838630 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206800bc00 con 0x7f20741599d0 2023-12-26T21:50:19.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f20795ff700 1 -- 172.21.15.131:0/3558838630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20741599d0 msgr2=0x7f20740ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f20795ff700 1 --2- 172.21.15.131:0/3558838630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20741599d0 0x7f20740ceb60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f20680099b0 tx=0x7f2068009250 comp rx=0 tx=0).stop 2023-12-26T21:50:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f20795ff700 1 -- 172.21.15.131:0/3558838630 shutdown_connections 2023-12-26T21:50:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f20795ff700 1 --2- 172.21.15.131:0/3558838630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740d01e0 0x7f20740ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f20795ff700 1 --2- 172.21.15.131:0/3558838630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20741599d0 0x7f20740ceb60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f20795ff700 1 -- 172.21.15.131:0/3558838630 >> 172.21.15.131:0/3558838630 conn(0x7f207406ece0 msgr2=0x7f207406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.978+0000 7f20795ff700 1 -- 172.21.15.131:0/3558838630 shutdown_connections 2023-12-26T21:50:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.979+0000 7f20795ff700 1 -- 172.21.15.131:0/3558838630 wait complete. 2023-12-26T21:50:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.980+0000 7f20795ff700 1 Processor -- start 2023-12-26T21:50:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.980+0000 7f20795ff700 1 -- start start 2023-12-26T21:50:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f20795ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20740d01e0 0x7f207414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f20795ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074159600 0x7f2074159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f20795ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207415a2f0 0x7f20740c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f20795ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207414da50 con 0x7f20740d01e0 2023-12-26T21:50:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f20795ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740c8d20 con 0x7f2074159600 2023-12-26T21:50:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f20795ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20740b61b0 con 0x7f207415a2f0 2023-12-26T21:50:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f2072ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20740d01e0 0x7f207414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f2072ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20740d01e0 0x7f207414d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51952/0 (socket says 172.21.15.131:51952) 2023-12-26T21:50:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f2072ffd700 1 -- 172.21.15.131:0/3519423666 learned_addr learned my addr 172.21.15.131:0/3519423666 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f20737fe700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207415a2f0 0x7f20740c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.981+0000 7f20727fc700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074159600 0x7f2074159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f2072ffd700 1 -- 172.21.15.131:0/3519423666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207415a2f0 msgr2=0x7f20740c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f2072ffd700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207415a2f0 0x7f20740c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f2072ffd700 1 -- 172.21.15.131:0/3519423666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074159600 msgr2=0x7f2074159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f2072ffd700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074159600 0x7f2074159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f2072ffd700 1 -- 172.21.15.131:0/3519423666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20680094f0 con 0x7f20740d01e0 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f20727fc700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074159600 0x7f2074159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f2072ffd700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20740d01e0 0x7f207414d510 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f206400c7c0 tx=0x7f2064007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f205affd700 1 -- 172.21.15.131:0/3519423666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206400f470 con 0x7f20740d01e0 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f205affd700 1 -- 172.21.15.131:0/3519423666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206400f5d0 con 0x7f20740d01e0 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f205affd700 1 -- 172.21.15.131:0/3519423666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206400f990 con 0x7f20740d01e0 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.982+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20740c95a0 con 0x7f20740d01e0 2023-12-26T21:50:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.983+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20740c8f50 con 0x7f20740d01e0 2023-12-26T21:50:19.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.984+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20740657e0 con 0x7f20740d01e0 2023-12-26T21:50:19.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.985+0000 7f205affd700 1 -- 172.21.15.131:0/3519423666 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f206401a360 con 0x7f20740d01e0 2023-12-26T21:50:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.985+0000 7f205affd700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2060066f80 0x7f2060069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.985+0000 7f205affd700 1 -- 172.21.15.131:0/3519423666 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2064083740 con 0x7f20740d01e0 2023-12-26T21:50:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.985+0000 7f20727fc700 1 -- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2060066f80 msgr2=0x7f2060069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:50:19.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.986+0000 7f20727fc700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2060066f80 0x7f2060069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:50:19.987 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:19.987+0000 7f34a82891c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:50:19.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:19.990+0000 7f205affd700 1 -- 172.21.15.131:0/3519423666 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2064050840 con 0x7f20740d01e0 2023-12-26T21:50:20.048 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:20.048+0000 7f1711f191c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:50:20.175 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:20.175+0000 7f34a82891c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.187+0000 7f20727fc700 1 -- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2060066f80 msgr2=0x7f2060069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.187+0000 7f20727fc700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2060066f80 0x7f2060069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.188+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f20740cf700 con 0x7f20740d01e0 2023-12-26T21:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.192+0000 7f205affd700 1 -- 172.21.15.131:0/3519423666 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2064053e60 con 0x7f20740d01e0 2023-12-26T21:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.194+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2060066f80 msgr2=0x7f2060069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.194+0000 7f20795ff700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2060066f80 0x7f2060069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.194+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20740d01e0 msgr2=0x7f207414d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.194+0000 7f20795ff700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20740d01e0 0x7f207414d510 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f206400c7c0 tx=0x7f2064007f00 comp rx=0 tx=0).stop 2023-12-26T21:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.194+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 shutdown_connections 2023-12-26T21:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.194+0000 7f20795ff700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f2060066f80 0x7f2060069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.194+0000 7f20795ff700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207415a2f0 0x7f20740c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.194+0000 7f20795ff700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20740d01e0 0x7f207414d510 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.194+0000 7f20795ff700 1 --2- 172.21.15.131:0/3519423666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2074159600 0x7f2074159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.195+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 >> 172.21.15.131:0/3519423666 conn(0x7f207406ece0 msgr2=0x7f20740d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.195+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 shutdown_connections 2023-12-26T21:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.195+0000 7f20795ff700 1 -- 172.21.15.131:0/3519423666 wait complete. 2023-12-26T21:50:20.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:20.212 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-26T21:50:20.246 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:20.247+0000 7f1711f191c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:50:20.262 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-12-26T21:50:20.348 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:20.348+0000 7f34a82891c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:50:20.430 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:20.430+0000 7f1711f191c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:50:20.430 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:20.430+0000 7f34a82891c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:50:20.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.501+0000 7f1d1d9ff700 1 Processor -- start 2023-12-26T21:50:20.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.502+0000 7f1d1d9ff700 1 -- start start 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.503+0000 7f1d1d9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d18156360 0x7f1d1806e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.503+0000 7f1d1d9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d180c8ba0 0x7f1d1806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.503+0000 7f1d1d9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d180acfc0 con 0x7f1d18156730 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.503+0000 7f1d1d9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d180c9000 con 0x7f1d180c8ba0 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.503+0000 7f1d1d9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1806f1f0 con 0x7f1d18156360 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.503+0000 7f1d177fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d180c8ba0 0x7f1d1806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.503+0000 7f1d177fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d180c8ba0 0x7f1d1806ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51968/0 (socket says 172.21.15.131:51968) 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.503+0000 7f1d177fe700 1 -- 172.21.15.131:0/3299286227 learned_addr learned my addr 172.21.15.131:0/3299286227 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.503+0000 7f1d16ffd700 1 --2- 172.21.15.131:0/3299286227 >> v2:172.21.15.190:3300/0 conn(0x7f1d18156360 0x7f1d1806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.504+0000 7f1d177fe700 1 -- 172.21.15.131:0/3299286227 >> v2:172.21.15.190:3300/0 conn(0x7f1d18156360 msgr2=0x7f1d1806e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.504+0000 7f1d177fe700 1 --2- 172.21.15.131:0/3299286227 >> v2:172.21.15.190:3300/0 conn(0x7f1d18156360 0x7f1d1806e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.504+0000 7f1d177fe700 1 -- 172.21.15.131:0/3299286227 >> v1:172.21.15.131:6789/0 conn(0x7f1d18156730 legacy=0x7f1d180c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:20.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.504+0000 7f1d177fe700 1 -- 172.21.15.131:0/3299286227 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180c91f0 con 0x7f1d180c8ba0 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.504+0000 7f1d177fe700 1 --2- 172.21.15.131:0/3299286227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d180c8ba0 0x7f1d1806ecb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1d18069410 tx=0x7f1d10009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45564a29331a8c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.504+0000 7f1d15ffb700 1 -- 172.21.15.131:0/3299286227 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d100092e0 con 0x7f1d180c8ba0 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.504+0000 7f1d15ffb700 1 -- 172.21.15.131:0/3299286227 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1000ba20 con 0x7f1d180c8ba0 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.504+0000 7f1d15ffb700 1 -- 172.21.15.131:0/3299286227 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000bde0 con 0x7f1d180c8ba0 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.505+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/3299286227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d180c8ba0 msgr2=0x7f1d1806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.505+0000 7f1d1d9ff700 1 --2- 172.21.15.131:0/3299286227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d180c8ba0 0x7f1d1806ecb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1d18069410 tx=0x7f1d10009840 comp rx=0 tx=0).stop 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.505+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/3299286227 shutdown_connections 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.505+0000 7f1d1d9ff700 1 --2- 172.21.15.131:0/3299286227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d180c8ba0 0x7f1d1806ecb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.505+0000 7f1d1d9ff700 1 --2- 172.21.15.131:0/3299286227 >> v2:172.21.15.190:3300/0 conn(0x7f1d18156360 0x7f1d1806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.505+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/3299286227 >> 172.21.15.131:0/3299286227 conn(0x7f1d180675b0 msgr2=0x7f1d180679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.505+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/3299286227 shutdown_connections 2023-12-26T21:50:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.505+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/3299286227 wait complete. 2023-12-26T21:50:20.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.506+0000 7f1d1d9ff700 1 Processor -- start 2023-12-26T21:50:20.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.507+0000 7f1d1d9ff700 1 -- start start 2023-12-26T21:50:20.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.507+0000 7f1d1d9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d180c8ba0 0x7f1d18152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:20.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.507+0000 7f1d1d9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d18156360 0x7f1d1816cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:20.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.507+0000 7f1d1d9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d18156730 0x7f1d180c0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:20.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.507+0000 7f1d1d9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1816dd10 con 0x7f1d18156730 2023-12-26T21:50:20.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.507+0000 7f1d1d9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1816d750 con 0x7f1d180c8ba0 2023-12-26T21:50:20.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.507+0000 7f1d1d9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1816da30 con 0x7f1d18156360 2023-12-26T21:50:20.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d18156730 0x7f1d180c0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:20.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d18156730 0x7f1d180c0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51970/0 (socket says 172.21.15.131:51970) 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d177fe700 1 -- 172.21.15.131:0/642429548 learned_addr learned my addr 172.21.15.131:0/642429548 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d16ffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d180c8ba0 0x7f1d18152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d167fc700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d18156360 0x7f1d1816cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d177fe700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d18156360 msgr2=0x7f1d1816cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d177fe700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d18156360 0x7f1d1816cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d177fe700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d180c8ba0 msgr2=0x7f1d18152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d177fe700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d180c8ba0 0x7f1d18152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d177fe700 1 -- 172.21.15.131:0/642429548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d10009ae0 con 0x7f1d18156730 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d167fc700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d18156360 0x7f1d1816cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:20.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.508+0000 7f1d177fe700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d18156730 0x7f1d180c0530 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1d180690d0 tx=0x7f1d1001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.509+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d100032f0 con 0x7f1d18156730 2023-12-26T21:50:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.509+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1001a660 con 0x7f1d18156730 2023-12-26T21:50:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.509+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d180bfd30 con 0x7f1d18156730 2023-12-26T21:50:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.509+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1001aa00 con 0x7f1d18156730 2023-12-26T21:50:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.509+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d18153510 con 0x7f1d18156730 2023-12-26T21:50:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.510+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d180717f0 con 0x7f1d18156730 2023-12-26T21:50:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.511+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f1d10003040 con 0x7f1d18156730 2023-12-26T21:50:20.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.512+0000 7f1cfeffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d04069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:20.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.512+0000 7f1d16ffd700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 msgr2=0x7f1d04069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:50:20.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.512+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1d10086400 con 0x7f1d18156730 2023-12-26T21:50:20.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.512+0000 7f1d16ffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d04069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:50:20.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.516+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d10053460 con 0x7f1d18156730 2023-12-26T21:50:20.520 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:20.521+0000 7f1711f191c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:50:20.576 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:20.575+0000 7f34a82891c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:50:20.664 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:20.664+0000 7f34a82891c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:50:20.672 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:20.672+0000 7f1711f191c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:50:20.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.713+0000 7f1d16ffd700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 msgr2=0x7f1d04069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:50:20.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.713+0000 7f1d16ffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d04069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:50:20.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:20.743+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d180c94d0 con 0x7f1d04066f80 2023-12-26T21:50:20.766 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:20.766+0000 7f1711f191c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:50:20.845 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:20.845+0000 7f34a82891c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:50:20.948 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:20.948+0000 7f1711f191c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:50:20.993 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:20.993+0000 7f34a82891c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:50:21.097 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:21.097+0000 7f1711f191c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:50:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:21.114+0000 7f1d16ffd700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 msgr2=0x7f1d04069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:50:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:21.114+0000 7f1d16ffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d04069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:50:21.377 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:21.376+0000 7f34a82891c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:50:21.479 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:21.480+0000 7f1711f191c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:50:21.642 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:21.642+0000 7f34a82891c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:50:21.742 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:21.743+0000 7f1711f191c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:50:21.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:21.915+0000 7f1d16ffd700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 msgr2=0x7f1d04069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:50:21.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:21.915+0000 7f1d16ffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d04069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T21:50:21.919 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:21.919+0000 7f34a82891c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:50:22.007 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:22.007+0000 7f34a82891c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:50:22.063 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:22.020+0000 7f1711f191c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:50:22.110 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:22.109+0000 7f1711f191c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:50:22.741 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:22.741+0000 7f34a82891c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:50:22.834 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:22.834+0000 7f34a82891c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:50:22.842 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:22.843+0000 7f1711f191c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:50:22.936 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:22.937+0000 7f1711f191c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:50:22.984 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:22.984+0000 7f34a82891c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:50:23.088 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:23.088+0000 7f1711f191c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:50:23.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:23.517+0000 7f1d16ffd700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 msgr2=0x7f1d04069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:50:23.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:23.517+0000 7f1d16ffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d04069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T21:50:23.605 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:23.604+0000 7f34a82891c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:50:23.710 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:23.710+0000 7f1711f191c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:50:23.790 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:23.790+0000 7f34a82891c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:50:23.900 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:23.900+0000 7f1711f191c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:50:23.978 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:23.978+0000 7f34a82891c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:50:24.089 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:24.090+0000 7f1711f191c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:50:24.818 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:24.818+0000 7f34a82891c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:50:24.930 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:24.931+0000 7f1711f191c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:50:25.091 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:25.091+0000 7f34a82891c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:50:25.183 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:25.182+0000 7f34a82891c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:50:25.203 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.204+0000 7f1711f191c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:50:25.296 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.297+0000 7f1711f191c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:50:25.374 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:50:25.374+0000 7f34a82891c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:50:25.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:25.434+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 19) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f1d10053830 con 0x7f1d18156730 2023-12-26T21:50:25.494 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.495+0000 7f1711f191c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:50:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:25.510+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 20) v1 ==== 84470+0+0 (secure 0 0 0) 0x7f1d10056610 con 0x7f1d18156730 2023-12-26T21:50:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:25.510+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 msgr2=0x7f1d04069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:50:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:25.510+0000 7f1cfeffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d04069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:25.626 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.627+0000 7f1683871700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:25.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.627+0000 7f1683871700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:25.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.627+0000 7f1683871700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:25.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.627+0000 7f1683871700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:25.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.627+0000 7f1683871700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:25.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.627+0000 7f1686877700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:25.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.628+0000 7f1686877700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:25.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.628+0000 7f1686877700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:25.628 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.628+0000 7f1686877700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:25.628 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:50:25.628+0000 7f1686877700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:50:26.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.510+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mon.0 v2:172.21.15.131:3300/0 9 ==== mgrmap(e 21) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f1d10054100 con 0x7f1d18156730 2023-12-26T21:50:26.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.511+0000 7f1cfeffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d040693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:26.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.511+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d180c94d0 con 0x7f1d04066f80 2023-12-26T21:50:26.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.512+0000 7f1d16ffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d040693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:26.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.517+0000 7f1d16ffd700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d040693b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009810 tx=0x7f1d080091a0 comp rx=0 tx=0).ready entity=mgr.5375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:26.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.590+0000 7f1cfeffd700 1 -- 172.21.15.131:0/642429548 <== mgr.5375 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1d180c94d0 con 0x7f1d04066f80 2023-12-26T21:50:26.591 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path snap_test_dir1 2023-12-26T21:50:26.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 msgr2=0x7f1d040693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:26.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d040693b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009810 tx=0x7f1d080091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d18156730 msgr2=0x7f1d180c0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:26.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d18156730 0x7f1d180c0530 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1d180690d0 tx=0x7f1d1001a000 comp rx=0 tx=0).stop 2023-12-26T21:50:26.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 shutdown_connections 2023-12-26T21:50:26.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d18156730 0x7f1d180c0530 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d180c8ba0 0x7f1d18152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f1d04066f80 0x7f1d040693b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 --2- 172.21.15.131:0/642429548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d18156360 0x7f1d1816cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.592+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 >> 172.21.15.131:0/642429548 conn(0x7f1d180675b0 msgr2=0x7f1d1806e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.593+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 shutdown_connections 2023-12-26T21:50:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.593+0000 7f1d1d9ff700 1 -- 172.21.15.131:0/642429548 wait complete. 2023-12-26T21:50:26.613 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-26T21:50:26.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-12-26T21:50:26.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.807+0000 7f0a3e7ff700 1 Processor -- start 2023-12-26T21:50:26.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.807+0000 7f0a3e7ff700 1 -- start start 2023-12-26T21:50:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.808+0000 7f0a3e7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a381597e0 0x7f0a380675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.808+0000 7f0a3e7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a38067af0 0x7f0a380a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a3e7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a380aa480 con 0x7f0a380aa020 2023-12-26T21:50:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a3e7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3815a750 con 0x7f0a381597e0 2023-12-26T21:50:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a3e7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38159bb0 con 0x7f0a38067af0 2023-12-26T21:50:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a37fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a381597e0 0x7f0a380675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a37fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a381597e0 0x7f0a380675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56200/0 (socket says 172.21.15.131:56200) 2023-12-26T21:50:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a37fff700 1 -- 172.21.15.131:0/3355319481 learned_addr learned my addr 172.21.15.131:0/3355319481 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:26.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a3cd9c700 1 --1- 172.21.15.131:0/3355319481 >> v1:172.21.15.131:6789/0 conn(0x7f0a380aa020 0x7f0a3806e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35964/0 (socket says 172.21.15.131:35964) 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a377fe700 1 --2- 172.21.15.131:0/3355319481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a38067af0 0x7f0a380a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a37fff700 1 -- 172.21.15.131:0/3355319481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a38067af0 msgr2=0x7f0a380a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a37fff700 1 --2- 172.21.15.131:0/3355319481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a38067af0 0x7f0a380a9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a37fff700 1 -- 172.21.15.131:0/3355319481 >> v1:172.21.15.131:6789/0 conn(0x7f0a380aa020 legacy=0x7f0a3806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a37fff700 1 -- 172.21.15.131:0/3355319481 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38068070 con 0x7f0a381597e0 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.809+0000 7f0a377fe700 1 --2- 172.21.15.131:0/3355319481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a38067af0 0x7f0a380a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a37fff700 1 --2- 172.21.15.131:0/3355319481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a381597e0 0x7f0a380675b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a280099b0 tx=0x7f0a28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b26974bb8a684a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a36ffd700 1 -- 172.21.15.131:0/3355319481 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800b860 con 0x7f0a381597e0 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a36ffd700 1 -- 172.21.15.131:0/3355319481 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2800b9c0 con 0x7f0a381597e0 2023-12-26T21:50:26.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a36ffd700 1 -- 172.21.15.131:0/3355319481 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800bd60 con 0x7f0a381597e0 2023-12-26T21:50:26.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3355319481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a381597e0 msgr2=0x7f0a380675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:26.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a3e7ff700 1 --2- 172.21.15.131:0/3355319481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a381597e0 0x7f0a380675b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a280099b0 tx=0x7f0a28009250 comp rx=0 tx=0).stop 2023-12-26T21:50:26.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3355319481 shutdown_connections 2023-12-26T21:50:26.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a3e7ff700 1 --2- 172.21.15.131:0/3355319481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a381597e0 0x7f0a380675b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a3e7ff700 1 --2- 172.21.15.131:0/3355319481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a38067af0 0x7f0a380a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3355319481 >> 172.21.15.131:0/3355319481 conn(0x7f0a3806b1c0 msgr2=0x7f0a380aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:26.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3355319481 shutdown_connections 2023-12-26T21:50:26.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.810+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3355319481 wait complete. 2023-12-26T21:50:26.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.812+0000 7f0a3e7ff700 1 Processor -- start 2023-12-26T21:50:26.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.812+0000 7f0a3e7ff700 1 -- start start 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.813+0000 7f0a3e7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38067af0 0x7f0a38158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.813+0000 7f0a3e7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a380aa020 0x7f0a38160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.813+0000 7f0a3e7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a381597e0 0x7f0a38153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.813+0000 7f0a3e7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3816d210 con 0x7f0a380aa020 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.813+0000 7f0a3e7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3816dab0 con 0x7f0a38067af0 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.813+0000 7f0a3e7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38158e80 con 0x7f0a381597e0 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.813+0000 7f0a377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a380aa020 0x7f0a38160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.813+0000 7f0a377fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a380aa020 0x7f0a38160480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56202/0 (socket says 172.21.15.131:56202) 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.813+0000 7f0a377fe700 1 -- 172.21.15.131:0/3142700789 learned_addr learned my addr 172.21.15.131:0/3142700789 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a3cd9c700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a381597e0 0x7f0a38153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a37fff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38067af0 0x7f0a38158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a377fe700 1 -- 172.21.15.131:0/3142700789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a381597e0 msgr2=0x7f0a38153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:26.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a377fe700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a381597e0 0x7f0a38153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a377fe700 1 -- 172.21.15.131:0/3142700789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38067af0 msgr2=0x7f0a38158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a377fe700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38067af0 0x7f0a38158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a377fe700 1 -- 172.21.15.131:0/3142700789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280094f0 con 0x7f0a380aa020 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a37fff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38067af0 0x7f0a38158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a377fe700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a380aa020 0x7f0a38160480 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c880 tx=0x7f0a2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a1ffff700 1 -- 172.21.15.131:0/3142700789 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00a2e0 con 0x7f0a380aa020 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a1ffff700 1 -- 172.21.15.131:0/3142700789 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c011600 con 0x7f0a380aa020 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a1ffff700 1 -- 172.21.15.131:0/3142700789 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0119a0 con 0x7f0a380aa020 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3816d460 con 0x7f0a380aa020 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.814+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381609c0 con 0x7f0a380aa020 2023-12-26T21:50:26.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.816+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a380717f0 con 0x7f0a380aa020 2023-12-26T21:50:26.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.817+0000 7f0a1ffff700 1 -- 172.21.15.131:0/3142700789 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 21) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f0a2c0223a0 con 0x7f0a380aa020 2023-12-26T21:50:26.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.818+0000 7f0a1ffff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a24066dd0 0x7f0a24069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:26.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.818+0000 7f0a1ffff700 1 -- 172.21.15.131:0/3142700789 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6327+0+0 (secure 0 0 0) 0x7f0a2c0847a0 con 0x7f0a380aa020 2023-12-26T21:50:26.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.818+0000 7f0a37fff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a24066dd0 0x7f0a24069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:26.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.819+0000 7f0a37fff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a24066dd0 0x7f0a24069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0a280099b0 tx=0x7f0a28017000 comp rx=0 tx=0).ready entity=mgr.5375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:26.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:26.823+0000 7f0a1ffff700 1 -- 172.21.15.131:0/3142700789 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a2c051980 con 0x7f0a380aa020 2023-12-26T21:50:27.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.043+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a3816dbf0 con 0x7f0a24066dd0 2023-12-26T21:50:27.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.100+0000 7f0a1ffff700 1 -- 172.21.15.131:0/3142700789 <== mgr.5375 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0a2800bbc0 con 0x7f0a24066dd0 2023-12-26T21:50:27.101 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path snap_test_dir1 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a24066dd0 msgr2=0x7f0a24069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a24066dd0 0x7f0a24069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0a280099b0 tx=0x7f0a28017000 comp rx=0 tx=0).stop 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a380aa020 msgr2=0x7f0a38160480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a380aa020 0x7f0a38160480 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c880 tx=0x7f0a2c008040 comp rx=0 tx=0).stop 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 shutdown_connections 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0a24066dd0 0x7f0a24069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a381597e0 0x7f0a38153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38067af0 0x7f0a38158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 --2- 172.21.15.131:0/3142700789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a380aa020 0x7f0a38160480 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 >> 172.21.15.131:0/3142700789 conn(0x7f0a3806b1c0 msgr2=0x7f0a380ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:27.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 shutdown_connections 2023-12-26T21:50:27.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.103+0000 7f0a3e7ff700 1 -- 172.21.15.131:0/3142700789 wait complete. 2023-12-26T21:50:27.121 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-26T21:50:27.122 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-26-21_51_00 in ~33s... 2023-12-26T21:50:27.122 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-26-21_52_00 in ~93s... 2023-12-26T21:50:27.122 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-12-26T21:50:27.122 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-12-26T21:50:27.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.315+0000 7f0d279ff700 1 Processor -- start 2023-12-26T21:50:27.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.315+0000 7f0d279ff700 1 -- start start 2023-12-26T21:50:27.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.317+0000 7f0d279ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d20069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:27.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.317+0000 7f0d279ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d200cc590 0x7f0d200cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.317+0000 7f0d279ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d20069640 con 0x7f0d200cb340 2023-12-26T21:50:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.317+0000 7f0d279ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d200cb7c0 con 0x7f0d200cc590 2023-12-26T21:50:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.317+0000 7f0d279ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20068750 con 0x7f0d20142f00 2023-12-26T21:50:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.317+0000 7f0d25f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0d200cb340 0x7f0d200c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35972/0 (socket says 172.21.15.131:35972) 2023-12-26T21:50:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.317+0000 7f0d25f9c700 1 -- 172.21.15.131:0/1476525135 learned_addr learned my addr 172.21.15.131:0/1476525135 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:50:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.317+0000 7f0d24f9a700 1 --2- 172.21.15.131:0/1476525135 >> v2:172.21.15.190:3300/0 conn(0x7f0d200cc590 0x7f0d200cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.317+0000 7f0d2579b700 1 --2- 172.21.15.131:0/1476525135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d20069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d17fff700 1 -- 172.21.15.131:0/1476525135 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757239342 0 0) 0x7f0d20069640 con 0x7f0d200cb340 2023-12-26T21:50:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d17fff700 1 -- 172.21.15.131:0/1476525135 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d0c003570 con 0x7f0d200cb340 2023-12-26T21:50:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d24f9a700 1 -- 172.21.15.131:0/1476525135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 msgr2=0x7f0d20069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:50:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d24f9a700 1 --2- 172.21.15.131:0/1476525135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d20069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d24f9a700 1 -- 172.21.15.131:0/1476525135 >> v1:172.21.15.131:6789/0 conn(0x7f0d200cb340 legacy=0x7f0d200c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d24f9a700 1 -- 172.21.15.131:0/1476525135 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20068a30 con 0x7f0d200cc590 2023-12-26T21:50:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d2579b700 1 --2- 172.21.15.131:0/1476525135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d20069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d24f9a700 1 --2- 172.21.15.131:0/1476525135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 0x7f0d200cae00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d100098e0 tx=0x7f0d100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82cac64392142bb7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d17fff700 1 -- 172.21.15.131:0/1476525135 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10011630 con 0x7f0d200cc590 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d17fff700 1 -- 172.21.15.131:0/1476525135 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10011790 con 0x7f0d200cc590 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.318+0000 7f0d17fff700 1 -- 172.21.15.131:0/1476525135 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10011b10 con 0x7f0d200cc590 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.319+0000 7f0d279ff700 1 -- 172.21.15.131:0/1476525135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 msgr2=0x7f0d200cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.319+0000 7f0d279ff700 1 --2- 172.21.15.131:0/1476525135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 0x7f0d200cae00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d100098e0 tx=0x7f0d100091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.319+0000 7f0d279ff700 1 -- 172.21.15.131:0/1476525135 shutdown_connections 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.319+0000 7f0d279ff700 1 --2- 172.21.15.131:0/1476525135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d20069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.319+0000 7f0d279ff700 1 --2- 172.21.15.131:0/1476525135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 0x7f0d200cae00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.319+0000 7f0d279ff700 1 -- 172.21.15.131:0/1476525135 >> 172.21.15.131:0/1476525135 conn(0x7f0d20063cd0 msgr2=0x7f0d2006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.319+0000 7f0d279ff700 1 -- 172.21.15.131:0/1476525135 shutdown_connections 2023-12-26T21:50:27.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.319+0000 7f0d279ff700 1 -- 172.21.15.131:0/1476525135 wait complete. 2023-12-26T21:50:27.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.321+0000 7f0d279ff700 1 Processor -- start 2023-12-26T21:50:27.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.321+0000 7f0d279ff700 1 -- start start 2023-12-26T21:50:27.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.322+0000 7f0d279ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d200cb340 0x7f0d2015b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:27.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d279ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 0x7f0d200caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:27.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d279ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d200c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:27.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d279ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2015bd90 con 0x7f0d200cb340 2023-12-26T21:50:27.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d279ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20140050 con 0x7f0d200cc590 2023-12-26T21:50:27.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d279ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2015e940 con 0x7f0d20142f00 2023-12-26T21:50:27.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d2579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d200cb340 0x7f0d2015b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d2579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d200cb340 0x7f0d2015b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56206/0 (socket says 172.21.15.131:56206) 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d2579b700 1 -- 172.21.15.131:0/2474055031 learned_addr learned my addr 172.21.15.131:0/2474055031 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d24f9a700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 0x7f0d200caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d25f9c700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d200c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d2579b700 1 -- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 msgr2=0x7f0d200c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d2579b700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d200c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d2579b700 1 -- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 msgr2=0x7f0d200caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d2579b700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 0x7f0d200caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d2579b700 1 -- 172.21.15.131:0/2474055031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c009420 con 0x7f0d200cb340 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d24f9a700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 0x7f0d200caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d25f9c700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d200c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:50:27.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.323+0000 7f0d2579b700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d200cb340 0x7f0d2015b850 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00ec80 tx=0x7f0d1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:27.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.324+0000 7f0d157fa700 1 -- 172.21.15.131:0/2474055031 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00c9d0 con 0x7f0d200cb340 2023-12-26T21:50:27.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.324+0000 7f0d157fa700 1 -- 172.21.15.131:0/2474055031 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c00cb30 con 0x7f0d200cb340 2023-12-26T21:50:27.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.324+0000 7f0d157fa700 1 -- 172.21.15.131:0/2474055031 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00ceb0 con 0x7f0d200cb340 2023-12-26T21:50:27.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.324+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d10009420 con 0x7f0d200cb340 2023-12-26T21:50:27.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.324+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d200c7670 con 0x7f0d200cb340 2023-12-26T21:50:27.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.325+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d200699d0 con 0x7f0d200cb340 2023-12-26T21:50:27.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.326+0000 7f0d157fa700 1 -- 172.21.15.131:0/2474055031 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 21) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f0d1c013680 con 0x7f0d200cb340 2023-12-26T21:50:27.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.327+0000 7f0d157fa700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d04066dd0 0x7f0d04069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:50:27.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.327+0000 7f0d157fa700 1 -- 172.21.15.131:0/2474055031 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6327+0+0 (secure 0 0 0) 0x7f0d1c084810 con 0x7f0d200cb340 2023-12-26T21:50:27.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.327+0000 7f0d24f9a700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d04066dd0 0x7f0d04069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:50:27.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.327+0000 7f0d24f9a700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d04066dd0 0x7f0d04069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0d201604f0 tx=0x7f0d10017000 comp rx=0 tx=0).ready entity=mgr.5375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:50:27.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.331+0000 7f0d157fa700 1 -- 172.21.15.131:0/2474055031 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d1c051ac0 con 0x7f0d200cb340 2023-12-26T21:50:27.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.520+0000 7f0d157fa700 1 -- 172.21.15.131:0/2474055031 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 22) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f0d1c051e90 con 0x7f0d200cb340 2023-12-26T21:50:27.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.551+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d200717f0 con 0x7f0d04066dd0 2023-12-26T21:50:27.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.560+0000 7f0d157fa700 1 -- 172.21.15.131:0/2474055031 <== mgr.5375 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7f0d200717f0 con 0x7f0d04066dd0 2023-12-26T21:50:27.561 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:50:27.561 INFO:teuthology.orchestra.run.smithi131.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d04066dd0 msgr2=0x7f0d04069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d04066dd0 0x7f0d04069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0d201604f0 tx=0x7f0d10017000 comp rx=0 tx=0).stop 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d200cb340 msgr2=0x7f0d2015b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d200cb340 0x7f0d2015b850 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00ec80 tx=0x7f0d1c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 shutdown_connections 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f0d04066dd0 0x7f0d04069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d20142f00 0x7f0d200c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d200cb340 0x7f0d2015b850 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 --2- 172.21.15.131:0/2474055031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d200cc590 0x7f0d200caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:50:27.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.562+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 >> 172.21.15.131:0/2474055031 conn(0x7f0d20063cd0 msgr2=0x7f0d201408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:50:27.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.563+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 shutdown_connections 2023-12-26T21:50:27.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:50:27.563+0000 7f0d279ff700 1 -- 172.21.15.131:0/2474055031 wait complete. 2023-12-26T21:50:27.581 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-12-26T21:50:27.581 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-12-26T21:50:27.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:27.582 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:27.627 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:27.627 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:27.627 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:28.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:28.628 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:28.677 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:28.677 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:28.677 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:29.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:29.678 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:29.728 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:29.729 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:29.729 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:30.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:30.729 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:30.779 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:30.779 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:30.780 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:31.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:31.780 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:31.830 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:31.830 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:31.830 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:32.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:32.832 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:32.882 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:32.882 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:32.882 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:33.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:33.884 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:33.934 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:33.934 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:33.934 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:34.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:34.935 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:34.985 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:34.985 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:34.985 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:35.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:35.986 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:36.036 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:36.037 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:36.037 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:37.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:37.038 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:37.088 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:37.089 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:37.089 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:38.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:38.089 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:38.140 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:38.140 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:38.140 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:39.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:39.141 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:39.191 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:39.192 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:39.192 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:40.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:40.193 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:40.243 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:40.244 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:40.244 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:41.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:41.245 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:41.295 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:41.295 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:41.295 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:42.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:42.296 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:42.346 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:42.347 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:42.347 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:43.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:43.348 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:43.398 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:43.398 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:43.398 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:44.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:44.400 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:44.450 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:44.450 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:44.450 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:45.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:45.451 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:45.501 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:45.501 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:45.501 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:46.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:46.502 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:46.554 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:46.554 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:46.554 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:47.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:50:47.304 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:50:47.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:47.555 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:47.603 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:47.613 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:47.614 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:48.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:48.615 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:48.665 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:48.666 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:48.666 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:49.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:49.667 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:49.717 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:49.717 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:49.717 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:50.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:50.718 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:50.769 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:50.769 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:50.769 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:51.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:51.770 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:51.820 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:51.820 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:51.820 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:52.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:52.821 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:52.871 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:52.871 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:52.871 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:53.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:53.872 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:53.923 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:53.923 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:53.923 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:54.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:54.924 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:54.974 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:54.974 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:54.974 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:55.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:55.975 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:56.026 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:56.026 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:56.026 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:57.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:57.027 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:57.077 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:57.077 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:57.078 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:58.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:58.078 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:58.129 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:58.129 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:58.129 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:50:59.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:50:59.129 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:50:59.179 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:50:59.180 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:50:59.180 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:00.114 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:51:00.114+0000 7f1675855700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:51:00.115 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:51:00.114+0000 7f1675855700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:51:00.115 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:51:00.114+0000 7f1675855700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:51:00.115 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:51:00.114+0000 7f1675855700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:51:00.115 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:51:00.114+0000 7f1675855700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:51:00.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:00.181 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:00.227 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:00.229 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:00.229 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:00.229 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:00.229 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-26-21_51_00_UTC'] 2023-12-26T21:51:00.229 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703627427.1220043, now = 1703627460.2293987, timo = 33 2023-12-26T21:51:01.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:01.230 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:01.278 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:01.280 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:01.280 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:01.280 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:02.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:02.281 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:02.328 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:02.329 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:02.329 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:02.329 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:03.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:03.330 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:03.379 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:03.380 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:03.380 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:03.381 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:04.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:04.381 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:04.433 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:04.434 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:04.434 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:04.434 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:05.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:05.436 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:05.485 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:05.486 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:05.486 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:05.486 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:06.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:06.487 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:06.536 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:06.537 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:06.537 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:06.537 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:07.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:07.538 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:07.588 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:07.589 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:07.589 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:07.589 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:08.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:08.591 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:08.639 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:08.641 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:08.641 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:08.641 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:09.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:09.642 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:09.689 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:09.690 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:09.690 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:09.691 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:10.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:10.692 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:10.741 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:10.742 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:10.742 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:10.742 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:11.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:11.743 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:11.791 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:11.792 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:11.793 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:11.793 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:12.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:12.794 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:12.844 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:12.844 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:12.844 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:13.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:13.845 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:13.894 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:13.895 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:13.895 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:13.895 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:14.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:14.897 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:14.946 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:14.947 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:14.948 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:14.948 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:15.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:15.948 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:15.998 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:15.998 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:15.998 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:17.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:17.000 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:17.049 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:17.050 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:17.050 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:17.050 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:17.445 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:51:17.450 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:51:18.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:18.052 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:18.101 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:18.102 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:18.102 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:18.103 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:19.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:19.104 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:19.153 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:19.154 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:19.154 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:19.154 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:20.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:20.155 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:20.204 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:20.205 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:20.206 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:20.206 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:21.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:21.206 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:21.255 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:21.256 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:21.256 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:21.257 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:22.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:22.258 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:22.307 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:22.309 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:22.309 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:22.309 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:23.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:23.309 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:23.358 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:23.359 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:23.359 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:23.359 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:24.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:24.360 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:24.409 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:24.410 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:24.410 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:24.410 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:25.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:25.411 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:25.460 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:25.462 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:25.462 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:25.462 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:26.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:26.462 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:26.511 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:26.512 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:26.512 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:26.512 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:27.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:27.514 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:27.563 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:27.564 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:27.565 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:27.565 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:28.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:28.572 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:28.620 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:28.621 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:28.622 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:28.622 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:29.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:29.623 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:29.672 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:29.673 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:29.673 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:29.674 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:30.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:30.675 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:30.723 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:30.724 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:30.724 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:30.724 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:31.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:31.726 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:31.775 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:31.776 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:31.776 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:31.776 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:32.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:32.777 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:32.826 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:32.827 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:32.827 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:32.827 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:33.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:33.828 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:33.878 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:33.879 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:33.879 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:33.879 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:34.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:34.880 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:34.929 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:34.931 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:34.931 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:34.931 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:35.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:35.931 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:35.980 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:35.981 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:35.981 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:35.981 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:36.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:36.982 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:37.031 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:37.032 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:37.032 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:37.033 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:38.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:38.034 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:38.083 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:38.084 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:38.084 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:38.084 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:39.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:39.085 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:39.134 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:39.135 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:39.135 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:39.135 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:40.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:40.136 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:40.185 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:40.186 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:40.186 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:40.186 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:41.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:41.187 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:41.238 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:41.239 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:41.240 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:41.240 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:42.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:42.241 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:42.290 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:42.291 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:42.291 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:42.291 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:43.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:43.292 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:43.341 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:43.342 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:43.342 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:43.342 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:44.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:44.344 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:44.393 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:44.394 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:44.394 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:44.394 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:45.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:45.395 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:45.444 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:45.445 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:45.445 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:45.445 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:46.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:46.446 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:46.495 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:46.496 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:46.496 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:46.496 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:47.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:47.497 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:47.546 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:47.547 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:47.548 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:47.548 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:47.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:51:47.706 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:51:48.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:48.549 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:48.598 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:48.599 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:48.599 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:48.599 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:49.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:49.601 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:49.649 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:49.650 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:49.650 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:49.651 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:50.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:50.652 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:50.701 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:50.702 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:50.702 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:50.702 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:51.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:51.703 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:51.753 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:51.753 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:51.753 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:52.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:52.754 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:52.803 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:52.804 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:52.805 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:52.805 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:53.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:53.806 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:53.854 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:53.855 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:53.856 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:53.856 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:54.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:54.857 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:54.906 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:54.907 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:54.907 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:54.907 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:55.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:55.908 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:55.957 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:55.958 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:55.958 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:55.958 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:56.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:56.959 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:57.008 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:57.009 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:57.009 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:57.009 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:58.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:58.010 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:58.059 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:58.060 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:58.060 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:58.060 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:51:59.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:51:59.062 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:51:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:51:59.112 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:51:59.112 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:51:59.112 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:52:00.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:00.114 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:52:00.162 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:52:00.163 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC'} 2023-12-26T21:52:00.163 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:52:00.164 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:52:01.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:01.165 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:52:01.214 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:52:01.214 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_52_00_UTC 2023-12-26T21:52:01.215 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_51_00_UTC', 'scheduled-2023-12-26-21_52_00_UTC'} 2023-12-26T21:52:01.215 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-26-21_52_00_UTC'} 2023-12-26T21:52:01.215 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:52:01.215 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-26-21_52_00_UTC'] 2023-12-26T21:52:01.215 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703627427.1220043, now = 1703627521.2157218, timo = 93 2023-12-26T21:52:02.217 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-12-26T21:52:02.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.420+0000 7fb4df7ff700 1 Processor -- start 2023-12-26T21:52:02.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.420+0000 7fb4df7ff700 1 -- start start 2023-12-26T21:52:02.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.421+0000 7fb4df7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156360 0x7fb4d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:02.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.421+0000 7fb4df7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb4d80c8ba0 0x7fb4d806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:02.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.421+0000 7fb4df7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d80acfc0 con 0x7fb4d8156730 2023-12-26T21:52:02.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.421+0000 7fb4df7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d806f1f0 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.421+0000 7fb4df7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d80c91f0 con 0x7fb4d8156360 2023-12-26T21:52:02.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.421+0000 7fb4ddd9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb4d80c8ba0 0x7fb4d806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:02.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.421+0000 7fb4dcd9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb4d8156730 0x7fb4d80c8490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42652/0 (socket says 172.21.15.131:42652) 2023-12-26T21:52:02.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.421+0000 7fb4dcd9a700 1 -- 172.21.15.131:0/1107209962 learned_addr learned my addr 172.21.15.131:0/1107209962 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:02.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.421+0000 7fb4dd59b700 1 --2- 172.21.15.131:0/1107209962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156360 0x7fb4d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:02.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4ddd9c700 1 -- 172.21.15.131:0/1107209962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156360 msgr2=0x7fb4d806e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:02.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4ddd9c700 1 --2- 172.21.15.131:0/1107209962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156360 0x7fb4d806e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:02.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4ddd9c700 1 -- 172.21.15.131:0/1107209962 >> v1:172.21.15.131:6789/0 conn(0x7fb4d8156730 legacy=0x7fb4d80c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:02.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4ddd9c700 1 -- 172.21.15.131:0/1107209962 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d80c94d0 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4cffff700 1 -- 172.21.15.131:0/1107209962 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058205659 0 0) 0x7fb4d80acfc0 con 0x7fb4d8156730 2023-12-26T21:52:02.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4dd59b700 1 --2- 172.21.15.131:0/1107209962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156360 0x7fb4d806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:02.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4ddd9c700 1 --2- 172.21.15.131:0/1107209962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 0x7fb4d806ecb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009e60 tx=0x7fb4d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42620ef9de034b4f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:02.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4cffff700 1 -- 172.21.15.131:0/1107209962 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000e070 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4cffff700 1 -- 172.21.15.131:0/1107209962 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d000bc30 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.422+0000 7fb4cffff700 1 -- 172.21.15.131:0/1107209962 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0012430 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.423+0000 7fb4df7ff700 1 -- 172.21.15.131:0/1107209962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 msgr2=0x7fb4d806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.423+0000 7fb4df7ff700 1 --2- 172.21.15.131:0/1107209962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 0x7fb4d806ecb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009e60 tx=0x7fb4d0009780 comp rx=0 tx=0).stop 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.423+0000 7fb4df7ff700 1 -- 172.21.15.131:0/1107209962 shutdown_connections 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.423+0000 7fb4df7ff700 1 --2- 172.21.15.131:0/1107209962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 0x7fb4d806ecb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.423+0000 7fb4df7ff700 1 --2- 172.21.15.131:0/1107209962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156360 0x7fb4d806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.423+0000 7fb4df7ff700 1 -- 172.21.15.131:0/1107209962 >> 172.21.15.131:0/1107209962 conn(0x7fb4d80675b0 msgr2=0x7fb4d80679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.423+0000 7fb4df7ff700 1 -- 172.21.15.131:0/1107209962 shutdown_connections 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.423+0000 7fb4df7ff700 1 -- 172.21.15.131:0/1107209962 wait complete. 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.424+0000 7fb4df7ff700 1 Processor -- start 2023-12-26T21:52:02.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.424+0000 7fb4df7ff700 1 -- start start 2023-12-26T21:52:02.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.425+0000 7fb4df7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 0x7fb4d80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:02.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.425+0000 7fb4df7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d8156360 0x7fb4d81558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:02.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.425+0000 7fb4df7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156730 0x7fb4d80b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:02.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.425+0000 7fb4df7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d81578a0 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.425+0000 7fb4df7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d80aff80 con 0x7fb4d8156360 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.425+0000 7fb4df7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d80b0260 con 0x7fb4d8156730 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.425+0000 7fb4dd59b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 0x7fb4d80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.425+0000 7fb4dd59b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 0x7fb4d80af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57352/0 (socket says 172.21.15.131:57352) 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.425+0000 7fb4dd59b700 1 -- 172.21.15.131:0/636801679 learned_addr learned my addr 172.21.15.131:0/636801679 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4ddd9c700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156730 0x7fb4d80b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4dd59b700 1 -- 172.21.15.131:0/636801679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156730 msgr2=0x7fb4d80b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4dd59b700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156730 0x7fb4d80b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4dcd9a700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d8156360 0x7fb4d81558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4dd59b700 1 -- 172.21.15.131:0/636801679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d8156360 msgr2=0x7fb4d81558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4dd59b700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d8156360 0x7fb4d81558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4dd59b700 1 -- 172.21.15.131:0/636801679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d0009a20 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4dcd9a700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d8156360 0x7fb4d81558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4dd59b700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 0x7fb4d80af950 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c830 tx=0x7fb4d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:02.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4cd7fa700 1 -- 172.21.15.131:0/636801679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4011600 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4cd7fa700 1 -- 172.21.15.131:0/636801679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d4011760 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d80b0540 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.426+0000 7fb4cd7fa700 1 -- 172.21.15.131:0/636801679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4011b20 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.427+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d80b1bd0 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.428+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d8157670 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.428+0000 7fb4cd7fa700 1 -- 172.21.15.131:0/636801679 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 24) v1 ==== 84873+0+0 (secure 0 0 0) 0x7fb4d400a3e0 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.429+0000 7fb4cd7fa700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4bc0670a0 0x7fb4bc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.429+0000 7fb4cd7fa700 1 -- 172.21.15.131:0/636801679 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6472+0+0 (secure 0 0 0) 0x7fb4d4084ff0 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.429+0000 7fb4dcd9a700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4bc0670a0 0x7fb4bc069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.430+0000 7fb4dcd9a700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4bc0670a0 0x7fb4bc069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8160400 tx=0x7fb4c8009040 comp rx=0 tx=0).ready entity=mgr.5375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:02.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.434+0000 7fb4cd7fa700 1 -- 172.21.15.131:0/636801679 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4d8157670 con 0x7fb4d80c8ba0 2023-12-26T21:52:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.655+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 --> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4d81579e0 con 0x7fb4bc0670a0 2023-12-26T21:52:02.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.726+0000 7fb4cd7fa700 1 -- 172.21.15.131:0/636801679 <== mgr.5375 v2:172.21.15.131:6832/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb4c8001c50 con 0x7fb4bc0670a0 2023-12-26T21:52:02.727 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path snap_test_dir1 2023-12-26T21:52:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4bc0670a0 msgr2=0x7fb4bc069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4bc0670a0 0x7fb4bc069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8160400 tx=0x7fb4c8009040 comp rx=0 tx=0).stop 2023-12-26T21:52:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 msgr2=0x7fb4d80af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 0x7fb4d80af950 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c830 tx=0x7fb4d4008040 comp rx=0 tx=0).stop 2023-12-26T21:52:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 shutdown_connections 2023-12-26T21:52:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fb4bc0670a0 0x7fb4bc069550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d8156730 0x7fb4d80b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d80c8ba0 0x7fb4d80af950 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 --2- 172.21.15.131:0/636801679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d8156360 0x7fb4d81558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.728+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 >> 172.21.15.131:0/636801679 conn(0x7fb4d80675b0 msgr2=0x7fb4d80c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.729+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 shutdown_connections 2023-12-26T21:52:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:02.729+0000 7fb4df7ff700 1 -- 172.21.15.131:0/636801679 wait complete. 2023-12-26T21:52:02.747 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-26T21:52:02.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:02.747 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:52:02.791 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:52:02.791 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_52_00_UTC 2023-12-26T21:52:02.792 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-21_51_00_UTC 2023-12-26T21:52:02.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:02.792 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-21_51_00_UTC) 2023-12-26T21:52:02.881 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-21_52_00_UTC 2023-12-26T21:52:02.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:02.881 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-21_52_00_UTC) 2023-12-26T21:52:02.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:02.930 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-26T21:52:02.976 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T21:52:03.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.207+0000 7ff2663ff700 1 Processor -- start 2023-12-26T21:52:03.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.207+0000 7ff2663ff700 1 -- start start 2023-12-26T21:52:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.208+0000 7ff2663ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff260159920 0x7ff2600b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.208+0000 7ff2663ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2600c8bf0 con 0x7ff260159cf0 2023-12-26T21:52:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.208+0000 7ff2663ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2600c8e60 con 0x7ff260155be0 2023-12-26T21:52:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.208+0000 7ff2663ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2600c9140 con 0x7ff260159920 2023-12-26T21:52:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.209+0000 7ff25ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff260159920 0x7ff2600b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.209+0000 7ff25f7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff260159cf0 0x7ff2601554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42654/0 (socket says 172.21.15.131:42654) 2023-12-26T21:52:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.209+0000 7ff25f7fe700 1 -- 172.21.15.131:0/1240450959 learned_addr learned my addr 172.21.15.131:0/1240450959 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:03.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.209+0000 7ff25effd700 1 -- 172.21.15.131:0/1240450959 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132126910 0 0) 0x7ff2600c8bf0 con 0x7ff260159cf0 2023-12-26T21:52:03.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.209+0000 7ff25effd700 1 -- 172.21.15.131:0/1240450959 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff248004680 con 0x7ff260159cf0 2023-12-26T21:52:03.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.209+0000 7ff25ffff700 1 -- 172.21.15.131:0/1240450959 >> v1:172.21.15.190:6789/0 conn(0x7ff260155be0 legacy=0x7ff2600c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:03.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.209+0000 7ff25ffff700 1 -- 172.21.15.131:0/1240450959 >> v1:172.21.15.131:6789/0 conn(0x7ff260159cf0 legacy=0x7ff2601554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:03.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.209+0000 7ff25ffff700 1 -- 172.21.15.131:0/1240450959 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600c9420 con 0x7ff260159920 2023-12-26T21:52:03.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.209+0000 7ff25ffff700 1 --2- 172.21.15.131:0/1240450959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 0x7ff2600b00e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff2500098e0 tx=0x7ff2500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56c473859c1471e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:03.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff25effd700 1 -- 172.21.15.131:0/1240450959 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000f6b0 con 0x7ff260159920 2023-12-26T21:52:03.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff25effd700 1 -- 172.21.15.131:0/1240450959 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25000f810 con 0x7ff260159920 2023-12-26T21:52:03.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff25effd700 1 -- 172.21.15.131:0/1240450959 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000fbb0 con 0x7ff260159920 2023-12-26T21:52:03.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff2663ff700 1 -- 172.21.15.131:0/1240450959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 msgr2=0x7ff2600b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:03.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff2663ff700 1 --2- 172.21.15.131:0/1240450959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 0x7ff2600b00e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff2500098e0 tx=0x7ff2500091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:03.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff2663ff700 1 -- 172.21.15.131:0/1240450959 shutdown_connections 2023-12-26T21:52:03.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff2663ff700 1 --2- 172.21.15.131:0/1240450959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 0x7ff2600b00e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:03.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff2663ff700 1 -- 172.21.15.131:0/1240450959 >> 172.21.15.131:0/1240450959 conn(0x7ff260068040 msgr2=0x7ff260069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:03.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff2663ff700 1 -- 172.21.15.131:0/1240450959 shutdown_connections 2023-12-26T21:52:03.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.210+0000 7ff2663ff700 1 -- 172.21.15.131:0/1240450959 wait complete. 2023-12-26T21:52:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.212+0000 7ff2663ff700 1 Processor -- start 2023-12-26T21:52:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.212+0000 7ff2663ff700 1 -- start start 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff2663ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff260155be0 0x7ff2600c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff2663ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 0x7ff2600c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff2663ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff260159cf0 0x7ff260154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff2663ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600c6d80 con 0x7ff260159920 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff2663ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600c6170 con 0x7ff260159cf0 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff2663ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600c6450 con 0x7ff260155be0 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff25f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 0x7ff2600c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff25f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 0x7ff2600c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57360/0 (socket says 172.21.15.131:57360) 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff25f7fe700 1 -- 172.21.15.131:0/4217353891 learned_addr learned my addr 172.21.15.131:0/4217353891 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff25ffff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff260155be0 0x7ff2600c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff26499c700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff260159cf0 0x7ff260154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff25f7fe700 1 -- 172.21.15.131:0/4217353891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff260155be0 msgr2=0x7ff2600c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff25f7fe700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff260155be0 0x7ff2600c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff25f7fe700 1 -- 172.21.15.131:0/4217353891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff260159cf0 msgr2=0x7ff260154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff25f7fe700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff260159cf0 0x7ff260154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff25f7fe700 1 -- 172.21.15.131:0/4217353891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff250009420 con 0x7ff260159920 2023-12-26T21:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.213+0000 7ff26499c700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff260159cf0 0x7ff260154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.214+0000 7ff25f7fe700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 0x7ff2600c5820 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c7c0 tx=0x7ff254007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.214+0000 7ff247fff700 1 -- 172.21.15.131:0/4217353891 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254005b30 con 0x7ff260159920 2023-12-26T21:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.214+0000 7ff247fff700 1 -- 172.21.15.131:0/4217353891 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254005c90 con 0x7ff260159920 2023-12-26T21:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.214+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2600c6730 con 0x7ff260159920 2023-12-26T21:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.214+0000 7ff247fff700 1 -- 172.21.15.131:0/4217353891 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254007340 con 0x7ff260159920 2023-12-26T21:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.214+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2600c7750 con 0x7ff260159920 2023-12-26T21:52:03.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.215+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2600657e0 con 0x7ff260159920 2023-12-26T21:52:03.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.216+0000 7ff247fff700 1 -- 172.21.15.131:0/4217353891 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 24) v1 ==== 84873+0+0 (secure 0 0 0) 0x7ff25401b400 con 0x7ff260159920 2023-12-26T21:52:03.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.217+0000 7ff247fff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff24c0670a0 0x7ff24c069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:03.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.217+0000 7ff247fff700 1 -- 172.21.15.131:0/4217353891 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6472+0+0 (secure 0 0 0) 0x7ff2540845c0 con 0x7ff260159920 2023-12-26T21:52:03.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.217+0000 7ff25ffff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff24c0670a0 0x7ff24c069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:03.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.217+0000 7ff25ffff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff24c0670a0 0x7ff24c069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2500093f0 tx=0x7ff250019000 comp rx=0 tx=0).ready entity=mgr.5375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:03.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.221+0000 7ff247fff700 1 -- 172.21.15.131:0/4217353891 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff254051400 con 0x7ff260159920 2023-12-26T21:52:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:03.471+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7ff2600c6ec0 con 0x7ff260159920 2023-12-26T21:52:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.247+0000 7ff247fff700 1 -- 172.21.15.131:0/4217353891 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7ff2540517d0 con 0x7ff260159920 2023-12-26T21:52:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.249+0000 7ff247fff700 1 -- 172.21.15.131:0/4217353891 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v25) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff254054e90 con 0x7ff260159920 2023-12-26T21:52:04.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff24c0670a0 msgr2=0x7ff24c069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff24c0670a0 0x7ff24c069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2500093f0 tx=0x7ff250019000 comp rx=0 tx=0).stop 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 msgr2=0x7ff2600c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 0x7ff2600c5820 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c7c0 tx=0x7ff254007e00 comp rx=0 tx=0).stop 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 shutdown_connections 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff260159cf0 0x7ff260154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff24c0670a0 0x7ff24c069550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff260155be0 0x7ff2600c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 --2- 172.21.15.131:0/4217353891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff260159920 0x7ff2600c5820 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 >> 172.21.15.131:0/4217353891 conn(0x7ff260068040 msgr2=0x7ff2600cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 shutdown_connections 2023-12-26T21:52:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.252+0000 7ff2663ff700 1 -- 172.21.15.131:0/4217353891 wait complete. 2023-12-26T21:52:04.272 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:52:04.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:52:04.365 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:52:04.452 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:52:04.452 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T21:52:04.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:04.452 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T21:52:04.537 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:52:04.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:52:04.537 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T21:52:04.588 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:04.685 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:04.685+0000 7f66f89581c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:52:04.718 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:04.717+0000 7f1e100561c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:52:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.782+0000 7f520ebff700 1 Processor -- start 2023-12-26T21:52:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.783+0000 7f520ebff700 1 -- start start 2023-12-26T21:52:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.784+0000 7f520ebff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5208159920 0x7f52080b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.784+0000 7f520ebff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5208159cf0 0x7f52080b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.784+0000 7f520ebff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52080c8bf0 con 0x7f52081554d0 2023-12-26T21:52:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.784+0000 7f520ebff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52080c8e60 con 0x7f5208159920 2023-12-26T21:52:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.784+0000 7f520ebff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52080c9140 con 0x7f5208159cf0 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f520c99b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5208159920 0x7f52080b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f5207fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5208159cf0 0x7f52080b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f520d19c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f52081554d0 0x7f5208155930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:48006/0 (socket says 172.21.15.131:48006) 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f520d19c700 1 -- 172.21.15.131:0/3083433007 learned_addr learned my addr 172.21.15.131:0/3083433007 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f5207fff700 1 -- 172.21.15.131:0/3083433007 >> v2:172.21.15.190:3300/0 conn(0x7f5208159920 msgr2=0x7f52080b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f5207fff700 1 --2- 172.21.15.131:0/3083433007 >> v2:172.21.15.190:3300/0 conn(0x7f5208159920 0x7f52080b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f5207fff700 1 -- 172.21.15.131:0/3083433007 >> v1:172.21.15.190:6789/0 conn(0x7f52081554d0 legacy=0x7f5208155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f52077fe700 1 -- 172.21.15.131:0/3083433007 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794480885 0 0) 0x7f52080c8bf0 con 0x7f52081554d0 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f5207fff700 1 -- 172.21.15.131:0/3083433007 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080c9420 con 0x7f5208159cf0 2023-12-26T21:52:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.785+0000 7f5207fff700 1 --2- 172.21.15.131:0/3083433007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5208159cf0 0x7f52080b0620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51f40098e0 tx=0x7f51f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c427816ceb153f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.786+0000 7f52077fe700 1 -- 172.21.15.131:0/3083433007 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f400b640 con 0x7f5208159cf0 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.786+0000 7f52077fe700 1 -- 172.21.15.131:0/3083433007 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f400b7a0 con 0x7f5208159cf0 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.786+0000 7f52077fe700 1 -- 172.21.15.131:0/3083433007 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f400bb40 con 0x7f5208159cf0 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.787+0000 7f520ebff700 1 -- 172.21.15.131:0/3083433007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5208159cf0 msgr2=0x7f52080b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.787+0000 7f520ebff700 1 --2- 172.21.15.131:0/3083433007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5208159cf0 0x7f52080b0620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f51f40098e0 tx=0x7f51f40091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.787+0000 7f520ebff700 1 -- 172.21.15.131:0/3083433007 shutdown_connections 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.787+0000 7f520ebff700 1 --2- 172.21.15.131:0/3083433007 >> v2:172.21.15.190:3300/0 conn(0x7f5208159920 0x7f52080b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.787+0000 7f520ebff700 1 --2- 172.21.15.131:0/3083433007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5208159cf0 0x7f52080b0620 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.787+0000 7f520ebff700 1 -- 172.21.15.131:0/3083433007 >> 172.21.15.131:0/3083433007 conn(0x7f5208068040 msgr2=0x7f5208069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.787+0000 7f520ebff700 1 -- 172.21.15.131:0/3083433007 shutdown_connections 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.787+0000 7f520ebff700 1 -- 172.21.15.131:0/3083433007 wait complete. 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.789+0000 7f520ebff700 1 Processor -- start 2023-12-26T21:52:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.789+0000 7f520ebff700 1 -- start start 2023-12-26T21:52:04.792 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:04.788+0000 7f66f89581c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f520ebff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52081554d0 0x7f52081407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f520ebff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208159920 0x7f520815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f520ebff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5208159cf0 0x7f52080c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f520ebff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080c6430 con 0x7f5208159920 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f520ebff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080c6710 con 0x7f5208159cf0 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f520ebff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52080c69f0 con 0x7f52081554d0 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f5207fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208159920 0x7f520815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f520c99b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52081554d0 0x7f52081407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f5207fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208159920 0x7f520815a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57378/0 (socket says 172.21.15.131:57378) 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f5207fff700 1 -- 172.21.15.131:0/1403963142 learned_addr learned my addr 172.21.15.131:0/1403963142 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.791+0000 7f520d19c700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5208159cf0 0x7f52080c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f5207fff700 1 -- 172.21.15.131:0/1403963142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52081554d0 msgr2=0x7f52081407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f5207fff700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52081554d0 0x7f52081407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f5207fff700 1 -- 172.21.15.131:0/1403963142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5208159cf0 msgr2=0x7f52080c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f5207fff700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5208159cf0 0x7f52080c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f5207fff700 1 -- 172.21.15.131:0/1403963142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f4009420 con 0x7f5208159920 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f520c99b700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52081554d0 0x7f52081407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f5207fff700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208159920 0x7f520815a700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f51f40093f0 tx=0x7f51f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f5204ff9700 1 -- 172.21.15.131:0/1403963142 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f400bb20 con 0x7f5208159920 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f5204ff9700 1 -- 172.21.15.131:0/1403963142 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f4000c90 con 0x7f5208159920 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52080c6cd0 con 0x7f5208159920 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f5204ff9700 1 -- 172.21.15.131:0/1403963142 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4011650 con 0x7f5208159920 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.792+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5208140d20 con 0x7f5208159920 2023-12-26T21:52:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.793+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52080657e0 con 0x7f5208159920 2023-12-26T21:52:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.795+0000 7f5204ff9700 1 -- 172.21.15.131:0/1403963142 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f51f4011a40 con 0x7f5208159920 2023-12-26T21:52:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.796+0000 7f5204ff9700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51ec067050 0x7f51ec069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.796+0000 7f5204ff9700 1 -- 172.21.15.131:0/1403963142 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6472+0+0 (secure 0 0 0) 0x7f51f4086b50 con 0x7f5208159920 2023-12-26T21:52:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.796+0000 7f520c99b700 1 -- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51ec067050 msgr2=0x7f51ec069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.796+0000 7f520c99b700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51ec067050 0x7f51ec069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:04.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.800+0000 7f5204ff9700 1 -- 172.21.15.131:0/1403963142 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51f40539d0 con 0x7f5208159920 2023-12-26T21:52:04.833 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:04.833+0000 7f1e100561c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:52:04.878 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:04.878+0000 7f66f89581c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:52:04.927 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:04.926+0000 7f1e100561c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:52:04.961 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:04.961+0000 7f66f89581c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:52:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.997+0000 7f520c99b700 1 -- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51ec067050 msgr2=0x7f51ec069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:04.997+0000 7f520c99b700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51ec067050 0x7f51ec069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:05.013 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:05.011+0000 7f1e100561c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:52:05.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.045+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52080c9700 con 0x7f5208159920 2023-12-26T21:52:05.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.045+0000 7f5204ff9700 1 -- 172.21.15.131:0/1403963142 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f51f4015090 con 0x7f5208159920 2023-12-26T21:52:05.046 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:05.046 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3316959664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3316959664},{"type":"v1","addr":"172.21.15.190:6834","nonce":3316959664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/320470110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":320470110},{"type":"v1","addr":"172.21.15.190:6835","nonce":320470110}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/1337439654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1337439654},{"type":"v1","addr":"172.21.15.131:6836","nonce":1337439654}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:50:04.699122+0000","modified":"2023-12-26T21:50:08.232778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6837/607340038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":607340038},{"type":"v1","addr":"172.21.15.131:6837","nonce":607340038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-12-26T21:52:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51ec067050 msgr2=0x7f51ec069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51ec067050 0x7f51ec069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208159920 msgr2=0x7f520815a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208159920 0x7f520815a700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f51f40093f0 tx=0x7f51f4017000 comp rx=0 tx=0).stop 2023-12-26T21:52:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 shutdown_connections 2023-12-26T21:52:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5208159cf0 0x7f52080c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f51ec067050 0x7f51ec069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52081554d0 0x7f52081407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 --2- 172.21.15.131:0/1403963142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208159920 0x7f520815a700 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 >> 172.21.15.131:0/1403963142 conn(0x7f5208068040 msgr2=0x7f5208069360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:05.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 shutdown_connections 2023-12-26T21:52:05.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.048+0000 7f520ebff700 1 -- 172.21.15.131:0/1403963142 wait complete. 2023-12-26T21:52:05.050 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 25 2023-12-26T21:52:05.068 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:05.149 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:05.148+0000 7f66f89581c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:05.212 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:05.211+0000 7f1e100561c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:05.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.272+0000 7f48f3fff700 1 Processor -- start 2023-12-26T21:52:05.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.272+0000 7f48f3fff700 1 -- start start 2023-12-26T21:52:05.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.273+0000 7f48f3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48f415b550 0x7f48f415a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.273+0000 7f48f3fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40aedd0 con 0x7f48f4158950 2023-12-26T21:52:05.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.273+0000 7f48f3fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40af0b0 con 0x7f48f415a3c0 2023-12-26T21:52:05.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.273+0000 7f48f3fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f40675b0 con 0x7f48f415b550 2023-12-26T21:52:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.274+0000 7f48f37fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f48f4158950 0x7f48f40ae070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44374/0 (socket says 172.21.15.131:44374) 2023-12-26T21:52:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.274+0000 7f48f37fe700 1 -- 172.21.15.131:0/2300091640 learned_addr learned my addr 172.21.15.131:0/2300091640 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.274+0000 7f48f27fc700 1 --2- 172.21.15.131:0/2300091640 >> v2:172.21.15.190:3300/0 conn(0x7f48f415b550 0x7f48f415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.274+0000 7f48f27fc700 1 -- 172.21.15.131:0/2300091640 >> v1:172.21.15.190:6789/0 conn(0x7f48f415a3c0 legacy=0x7f48f415ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.274+0000 7f48f27fc700 1 -- 172.21.15.131:0/2300091640 >> v1:172.21.15.131:6789/0 conn(0x7f48f4158950 legacy=0x7f48f40ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.274+0000 7f48f27fc700 1 -- 172.21.15.131:0/2300091640 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f4067890 con 0x7f48f415b550 2023-12-26T21:52:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.274+0000 7f48f1ffb700 1 -- 172.21.15.131:0/2300091640 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011256131 0 0) 0x7f48f40aedd0 con 0x7f48f4158950 2023-12-26T21:52:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.275+0000 7f48f27fc700 1 --2- 172.21.15.131:0/2300091640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f415b550 0x7f48f415a790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009910 tx=0x7f48e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efaa32a9df236b1d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.275+0000 7f48f1ffb700 1 -- 172.21.15.131:0/2300091640 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e800b820 con 0x7f48f415b550 2023-12-26T21:52:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.275+0000 7f48f1ffb700 1 -- 172.21.15.131:0/2300091640 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e800b980 con 0x7f48f415b550 2023-12-26T21:52:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.275+0000 7f48f1ffb700 1 -- 172.21.15.131:0/2300091640 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e800bd20 con 0x7f48f415b550 2023-12-26T21:52:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.275+0000 7f48f3fff700 1 -- 172.21.15.131:0/2300091640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f415b550 msgr2=0x7f48f415a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.275+0000 7f48f3fff700 1 --2- 172.21.15.131:0/2300091640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f415b550 0x7f48f415a790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009910 tx=0x7f48e80091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.275+0000 7f48f3fff700 1 -- 172.21.15.131:0/2300091640 shutdown_connections 2023-12-26T21:52:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.275+0000 7f48f3fff700 1 --2- 172.21.15.131:0/2300091640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f415b550 0x7f48f415a790 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.275+0000 7f48f3fff700 1 -- 172.21.15.131:0/2300091640 >> 172.21.15.131:0/2300091640 conn(0x7f48f40cd330 msgr2=0x7f48f40c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.276+0000 7f48f3fff700 1 -- 172.21.15.131:0/2300091640 shutdown_connections 2023-12-26T21:52:05.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.276+0000 7f48f3fff700 1 -- 172.21.15.131:0/2300091640 wait complete. 2023-12-26T21:52:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.277+0000 7f48f3fff700 1 Processor -- start 2023-12-26T21:52:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.277+0000 7f48f3fff700 1 -- start start 2023-12-26T21:52:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f4158950 0x7f48f40af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48f415a3c0 0x7f48f4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f415b550 0x7f48f406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f3fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c0480 con 0x7f48f415a3c0 2023-12-26T21:52:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f3fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40b0a20 con 0x7f48f4158950 2023-12-26T21:52:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f3fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40afe10 con 0x7f48f415b550 2023-12-26T21:52:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f27fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48f415a3c0 0x7f48f4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f27fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48f415a3c0 0x7f48f4156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60862/0 (socket says 172.21.15.131:60862) 2023-12-26T21:52:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f27fc700 1 -- 172.21.15.131:0/2500206021 learned_addr learned my addr 172.21.15.131:0/2500206021 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f2ffd700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f4158950 0x7f48f40af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.278+0000 7f48f37fe700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f415b550 0x7f48f406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48f27fc700 1 -- 172.21.15.131:0/2500206021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f415b550 msgr2=0x7f48f406db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48f27fc700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f415b550 0x7f48f406db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48f27fc700 1 -- 172.21.15.131:0/2500206021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f4158950 msgr2=0x7f48f40af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48f27fc700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f4158950 0x7f48f40af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48f27fc700 1 -- 172.21.15.131:0/2500206021 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e8009420 con 0x7f48f415a3c0 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48f2ffd700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f4158950 0x7f48f40af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48f27fc700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48f415a3c0 0x7f48f4156d10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f48e80093f0 tx=0x7f48e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48daffd700 1 -- 172.21.15.131:0/2500206021 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e801f070 con 0x7f48f415a3c0 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48daffd700 1 -- 172.21.15.131:0/2500206021 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e8002f70 con 0x7f48f415a3c0 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48daffd700 1 -- 172.21.15.131:0/2500206021 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8025420 con 0x7f48f415a3c0 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.279+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f40b00f0 con 0x7f48f415a3c0 2023-12-26T21:52:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.280+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f40b06b0 con 0x7f48f415a3c0 2023-12-26T21:52:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.281+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f40657e0 con 0x7f48f415a3c0 2023-12-26T21:52:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.281+0000 7f48daffd700 1 -- 172.21.15.131:0/2500206021 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f48e8023070 con 0x7f48f415a3c0 2023-12-26T21:52:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.282+0000 7f48daffd700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48e0067050 0x7f48e0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.282+0000 7f48daffd700 1 -- 172.21.15.131:0/2500206021 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6560+0+0 (secure 0 0 0) 0x7f48e8085b80 con 0x7f48f415a3c0 2023-12-26T21:52:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.282+0000 7f48f2ffd700 1 -- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48e0067050 msgr2=0x7f48e0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.282+0000 7f48f2ffd700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48e0067050 0x7f48e0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:05.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.287+0000 7f48daffd700 1 -- 172.21.15.131:0/2500206021 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48e8052950 con 0x7f48f415a3c0 2023-12-26T21:52:05.326 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:05.325+0000 7f66f89581c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:52:05.409 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:05.408+0000 7f66f89581c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:52:05.417 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:05.417+0000 7f1e100561c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:52:05.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.483+0000 7f48f2ffd700 1 -- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48e0067050 msgr2=0x7f48e0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:05.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.483+0000 7f48f2ffd700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48e0067050 0x7f48e0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:05.504 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:05.504+0000 7f1e100561c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:52:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.509+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48f4067b70 con 0x7f48f415a3c0 2023-12-26T21:52:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.510+0000 7f48daffd700 1 -- 172.21.15.131:0/2500206021 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16713 (secure 0 0 0) 0x7f48e802a090 con 0x7f48f415a3c0 2023-12-26T21:52:05.511 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:05.511 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":68,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:05.243339+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:50:02.860261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-26T21:50:03.665608+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":68,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":0}]},{"pgid":"7.15","mappings":[{"from":5,"to":7}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2945752061":"2023-12-27T21:50:25.498209+0000","172.21.15.131:6833/39288":"2023-12-27T21:50:25.498209+0000","172.21.15.131:6832/39288":"2023-12-27T21:50:25.498209+0000","172.21.15.131:0/2402413168":"2023-12-27T21:50:25.498209+0000","172.21.15.131:0/471355450":"2023-12-27T21:50:25.498209+0000","172.21.15.131:0/3935646470":"2023-12-27T21:50:25.498209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48e0067050 msgr2=0x7f48e0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48e0067050 0x7f48e0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48f415a3c0 msgr2=0x7f48f4156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48f415a3c0 0x7f48f4156d10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f48e80093f0 tx=0x7f48e8017000 comp rx=0 tx=0).stop 2023-12-26T21:52:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 shutdown_connections 2023-12-26T21:52:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f48e0067050 0x7f48e0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f415b550 0x7f48f406db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f4158950 0x7f48f40af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 --2- 172.21.15.131:0/2500206021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48f415a3c0 0x7f48f4156d10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.512+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 >> 172.21.15.131:0/2500206021 conn(0x7f48f40cd330 msgr2=0x7f48f40a9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:05.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.513+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 shutdown_connections 2023-12-26T21:52:05.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.513+0000 7f48f3fff700 1 -- 172.21.15.131:0/2500206021 wait complete. 2023-12-26T21:52:05.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:50:02.860261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:50:03.665608+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 68, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:52:05.534 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:52:05.562 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:05.562+0000 7f66f89581c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:05.653 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:05.653+0000 7f66f89581c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:52:05.663 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:05.663+0000 7f1e100561c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:05.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.733+0000 7fcd9c1ff700 1 Processor -- start 2023-12-26T21:52:05.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.733+0000 7fcd9c1ff700 1 -- start start 2023-12-26T21:52:05.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.734+0000 7fcd9c1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd941596a0 0x7fcd940b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd9c1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd940afcd0 con 0x7fcd94159a70 2023-12-26T21:52:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd9c1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd940b3e70 con 0x7fcd940b0870 2023-12-26T21:52:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd9c1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940b0cb0 con 0x7fcd941596a0 2023-12-26T21:52:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd99f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd941596a0 0x7fcd940b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd99f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd941596a0 0x7fcd940b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60866/0 (socket says 172.21.15.131:60866) 2023-12-26T21:52:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd99f9b700 1 -- 172.21.15.131:0/2858637755 learned_addr learned my addr 172.21.15.131:0/2858637755 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd9979a700 1 --1- 172.21.15.131:0/2858637755 >> v1:172.21.15.131:6789/0 conn(0x7fcd94159a70 0x7fcd940b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44382/0 (socket says 172.21.15.131:44382) 2023-12-26T21:52:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd99f9b700 1 -- 172.21.15.131:0/2858637755 >> v1:172.21.15.190:6789/0 conn(0x7fcd940b0870 legacy=0x7fcd940b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd99f9b700 1 -- 172.21.15.131:0/2858637755 >> v1:172.21.15.131:6789/0 conn(0x7fcd94159a70 legacy=0x7fcd940b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.735+0000 7fcd99f9b700 1 -- 172.21.15.131:0/2858637755 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd940b0f20 con 0x7fcd941596a0 2023-12-26T21:52:05.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.736+0000 7fcd99f9b700 1 --2- 172.21.15.131:0/2858637755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 0x7fcd940b2380 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd900099b0 tx=0x7fcd90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a33a56f00cd1e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:05.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.736+0000 7fcd98f99700 1 -- 172.21.15.131:0/2858637755 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000b860 con 0x7fcd941596a0 2023-12-26T21:52:05.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.736+0000 7fcd98f99700 1 -- 172.21.15.131:0/2858637755 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9000b9c0 con 0x7fcd941596a0 2023-12-26T21:52:05.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.736+0000 7fcd98f99700 1 -- 172.21.15.131:0/2858637755 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000bd60 con 0x7fcd941596a0 2023-12-26T21:52:05.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.736+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/2858637755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 msgr2=0x7fcd940b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.736+0000 7fcd9c1ff700 1 --2- 172.21.15.131:0/2858637755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 0x7fcd940b2380 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd900099b0 tx=0x7fcd90009250 comp rx=0 tx=0).stop 2023-12-26T21:52:05.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.736+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/2858637755 shutdown_connections 2023-12-26T21:52:05.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.736+0000 7fcd9c1ff700 1 --2- 172.21.15.131:0/2858637755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 0x7fcd940b2380 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.736+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/2858637755 >> 172.21.15.131:0/2858637755 conn(0x7fcd9406ece0 msgr2=0x7fcd9406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:05.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.737+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/2858637755 shutdown_connections 2023-12-26T21:52:05.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.737+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/2858637755 wait complete. 2023-12-26T21:52:05.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.738+0000 7fcd9c1ff700 1 Processor -- start 2023-12-26T21:52:05.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.738+0000 7fcd9c1ff700 1 -- start start 2023-12-26T21:52:05.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9c1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd940b0870 0x7fcd940c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9c1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 0x7fcd940cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9c1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd94159a70 0x7fcd940cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9c1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd94156330 con 0x7fcd941596a0 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9c1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940c4700 con 0x7fcd940b0870 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9c1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940c8dc0 con 0x7fcd94159a70 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9979a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 0x7fcd940cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd99f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd940b0870 0x7fcd940c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9979a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 0x7fcd940cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60880/0 (socket says 172.21.15.131:60880) 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd99f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd940b0870 0x7fcd940c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:34918/0 (socket says 172.21.15.131:34918) 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9979a700 1 -- 172.21.15.131:0/1227462200 learned_addr learned my addr 172.21.15.131:0/1227462200 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.739+0000 7fcd9a79c700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd94159a70 0x7fcd940cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.740+0000 7fcd99f9b700 1 -- 172.21.15.131:0/1227462200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd94159a70 msgr2=0x7fcd940cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.740+0000 7fcd99f9b700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd94159a70 0x7fcd940cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.740+0000 7fcd99f9b700 1 -- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 msgr2=0x7fcd940cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.740+0000 7fcd99f9b700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 0x7fcd940cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.740+0000 7fcd99f9b700 1 -- 172.21.15.131:0/1227462200 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd900094f0 con 0x7fcd940b0870 2023-12-26T21:52:05.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.740+0000 7fcd9979a700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 0x7fcd940cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:05.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.740+0000 7fcd99f9b700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd940b0870 0x7fcd940c8090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcd940642f0 tx=0x7fcd90017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.740+0000 7fcd89ffb700 1 -- 172.21.15.131:0/1227462200 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90024440 con 0x7fcd940b0870 2023-12-26T21:52:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.741+0000 7fcd89ffb700 1 -- 172.21.15.131:0/1227462200 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd900245a0 con 0x7fcd940b0870 2023-12-26T21:52:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.741+0000 7fcd89ffb700 1 -- 172.21.15.131:0/1227462200 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90024920 con 0x7fcd940b0870 2023-12-26T21:52:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.741+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd940c8ac0 con 0x7fcd940b0870 2023-12-26T21:52:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.741+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd940cd0b0 con 0x7fcd940b0870 2023-12-26T21:52:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.742+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd940c8910 con 0x7fcd940b0870 2023-12-26T21:52:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.743+0000 7fcd89ffb700 1 -- 172.21.15.131:0/1227462200 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fcd9001a420 con 0x7fcd940b0870 2023-12-26T21:52:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.744+0000 7fcd89ffb700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcd78067050 0x7fcd78069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.744+0000 7fcd89ffb700 1 -- 172.21.15.131:0/1227462200 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6560+0+0 (secure 0 0 0) 0x7fcd90086b70 con 0x7fcd940b0870 2023-12-26T21:52:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.744+0000 7fcd9979a700 1 -- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcd78067050 msgr2=0x7fcd78069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.744+0000 7fcd9979a700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcd78067050 0x7fcd78069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:05.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.747+0000 7fcd89ffb700 1 -- 172.21.15.131:0/1227462200 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd90053a00 con 0x7fcd940b0870 2023-12-26T21:52:05.762 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:05.761+0000 7f1e100561c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:52:05.836 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:05.836+0000 7f66f89581c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:52:05.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.945+0000 7fcd9979a700 1 -- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcd78067050 msgr2=0x7fcd78069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:05.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.945+0000 7fcd9979a700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcd78067050 0x7fcd78069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:05.948 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:05.948+0000 7f1e100561c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:52:05.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.982+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd940657e0 con 0x7fcd940b0870 2023-12-26T21:52:05.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.983+0000 7fcd89ffb700 1 -- 172.21.15.131:0/1227462200 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd90057020 con 0x7fcd940b0870 2023-12-26T21:52:05.983 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:52:05.984 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:05.983+0000 7f66f89581c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:52:05.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcd78067050 msgr2=0x7fcd78069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:05.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcd78067050 0x7fcd78069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd940b0870 msgr2=0x7fcd940c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:05.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd940b0870 0x7fcd940c8090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcd940642f0 tx=0x7fcd90017000 comp rx=0 tx=0).stop 2023-12-26T21:52:05.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 shutdown_connections 2023-12-26T21:52:05.986 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcd78067050 0x7fcd78069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd94159a70 0x7fcd940cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd940b0870 0x7fcd940c8090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 --2- 172.21.15.131:0/1227462200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd941596a0 0x7fcd940cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:05.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 >> 172.21.15.131:0/1227462200 conn(0x7fcd9406ece0 msgr2=0x7fcd94158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:05.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 shutdown_connections 2023-12-26T21:52:05.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:05.985+0000 7fcd9c1ff700 1 -- 172.21.15.131:0/1227462200 wait complete. 2023-12-26T21:52:06.006 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:52:06.110 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:06.109+0000 7f1e100561c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:52:06.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.202+0000 7f9a01dff700 1 Processor -- start 2023-12-26T21:52:06.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.202+0000 7f9a01dff700 1 -- start start 2023-12-26T21:52:06.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f9a01dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f99fc1596a0 0x7f99fc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f9a01dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f99fc159a70 0x7f99fc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f9a01dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0b0160 0x7f99fc0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f9a01dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99fc0afcd0 con 0x7f99fc1596a0 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f9a01dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99fc0b3e70 con 0x7f99fc159a70 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f9a01dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0b2e00 con 0x7f99fc0b0160 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99fb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f99fc1596a0 0x7f99fc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99fb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f99fc1596a0 0x7f99fc0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60886/0 (socket says 172.21.15.131:60886) 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99fb7fe700 1 -- 172.21.15.131:0/1369578903 learned_addr learned my addr 172.21.15.131:0/1369578903 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99fbfff700 1 --2- 172.21.15.131:0/1369578903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0b0160 0x7f99fc0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99faffd700 1 --2- 172.21.15.131:0/1369578903 >> v2:172.21.15.190:3300/0 conn(0x7f99fc159a70 0x7f99fc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99fb7fe700 1 -- 172.21.15.131:0/1369578903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0b0160 msgr2=0x7f99fc0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99fb7fe700 1 --2- 172.21.15.131:0/1369578903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0b0160 0x7f99fc0b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99fb7fe700 1 -- 172.21.15.131:0/1369578903 >> v2:172.21.15.190:3300/0 conn(0x7f99fc159a70 msgr2=0x7f99fc0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99fb7fe700 1 --2- 172.21.15.131:0/1369578903 >> v2:172.21.15.190:3300/0 conn(0x7f99fc159a70 0x7f99fc0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.204+0000 7f99fb7fe700 1 -- 172.21.15.131:0/1369578903 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0b0f20 con 0x7f99fc1596a0 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.205+0000 7f99fb7fe700 1 --2- 172.21.15.131:0/1369578903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc1596a0 0x7f99fc0b2380 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0642f0 tx=0x7f99ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf45d00163df733 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:06.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.205+0000 7f99fa7fc700 1 -- 172.21.15.131:0/1369578903 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00b640 con 0x7f99fc1596a0 2023-12-26T21:52:06.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.205+0000 7f99fa7fc700 1 -- 172.21.15.131:0/1369578903 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec00b7a0 con 0x7f99fc1596a0 2023-12-26T21:52:06.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.205+0000 7f99fa7fc700 1 -- 172.21.15.131:0/1369578903 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00bb60 con 0x7f99fc1596a0 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.205+0000 7f9a01dff700 1 -- 172.21.15.131:0/1369578903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc1596a0 msgr2=0x7f99fc0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.206+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1369578903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc1596a0 0x7f99fc0b2380 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0642f0 tx=0x7f99ec009250 comp rx=0 tx=0).stop 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.206+0000 7f9a01dff700 1 -- 172.21.15.131:0/1369578903 shutdown_connections 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.206+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1369578903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc1596a0 0x7f99fc0b2380 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.206+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1369578903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0b0160 0x7f99fc0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.206+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1369578903 >> v2:172.21.15.190:3300/0 conn(0x7f99fc159a70 0x7f99fc0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.206+0000 7f9a01dff700 1 -- 172.21.15.131:0/1369578903 >> 172.21.15.131:0/1369578903 conn(0x7f99fc06ece0 msgr2=0x7f99fc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.206+0000 7f9a01dff700 1 -- 172.21.15.131:0/1369578903 shutdown_connections 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.206+0000 7f9a01dff700 1 -- 172.21.15.131:0/1369578903 wait complete. 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.207+0000 7f9a01dff700 1 Processor -- start 2023-12-26T21:52:06.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.207+0000 7f9a01dff700 1 -- start start 2023-12-26T21:52:06.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.208+0000 7f9a01dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc0b0160 0x7f99fc156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.208+0000 7f9a01dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc1596a0 0x7f99fc0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.208+0000 7f9a01dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc159a70 0x7f99fc0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.208+0000 7f9a01dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0c9b50 con 0x7f99fc0b0160 2023-12-26T21:52:06.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.208+0000 7f9a01dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0c7db0 con 0x7f99fc1596a0 2023-12-26T21:52:06.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.208+0000 7f9a01dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc16ac70 con 0x7f99fc159a70 2023-12-26T21:52:06.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.208+0000 7f99fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc0b0160 0x7f99fc156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.208+0000 7f99fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc0b0160 0x7f99fc156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60900/0 (socket says 172.21.15.131:60900) 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99fb7fe700 1 -- 172.21.15.131:0/1014443920 learned_addr learned my addr 172.21.15.131:0/1014443920 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99faffd700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc1596a0 0x7f99fc0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99fbfff700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc159a70 0x7f99fc0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99fb7fe700 1 -- 172.21.15.131:0/1014443920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc159a70 msgr2=0x7f99fc0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99fb7fe700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc159a70 0x7f99fc0ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99fb7fe700 1 -- 172.21.15.131:0/1014443920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc1596a0 msgr2=0x7f99fc0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99fb7fe700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc1596a0 0x7f99fc0cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99fb7fe700 1 -- 172.21.15.131:0/1014443920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0094f0 con 0x7f99fc0b0160 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99faffd700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc1596a0 0x7f99fc0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99fb7fe700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc0b0160 0x7f99fc156bd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f99fc089710 tx=0x7f99ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.209+0000 7f99e37fe700 1 -- 172.21.15.131:0/1014443920 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00b910 con 0x7f99fc0b0160 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.210+0000 7f99e37fe700 1 -- 172.21.15.131:0/1014443920 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec00ba70 con 0x7f99fc0b0160 2023-12-26T21:52:06.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.210+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc0c7760 con 0x7f99fc0b0160 2023-12-26T21:52:06.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.210+0000 7f99e37fe700 1 -- 172.21.15.131:0/1014443920 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec011630 con 0x7f99fc0b0160 2023-12-26T21:52:06.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.210+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc157110 con 0x7f99fc0b0160 2023-12-26T21:52:06.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.211+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc0657e0 con 0x7f99fc0b0160 2023-12-26T21:52:06.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.212+0000 7f99e37fe700 1 -- 172.21.15.131:0/1014443920 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f99ec0119a0 con 0x7f99fc0b0160 2023-12-26T21:52:06.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.212+0000 7f99e37fe700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f99e8067050 0x7f99e8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.213+0000 7f99e37fe700 1 -- 172.21.15.131:0/1014443920 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6560+0+0 (secure 0 0 0) 0x7f99ec0860e0 con 0x7f99fc0b0160 2023-12-26T21:52:06.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.213+0000 7f99faffd700 1 -- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f99e8067050 msgr2=0x7f99e8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:06.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.213+0000 7f99faffd700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f99e8067050 0x7f99e8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:06.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.217+0000 7f99e37fe700 1 -- 172.21.15.131:0/1014443920 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99ec052ef0 con 0x7f99fc0b0160 2023-12-26T21:52:06.370 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:06.370+0000 7f66f89581c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:52:06.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.414+0000 7f99faffd700 1 -- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f99e8067050 msgr2=0x7f99e8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:06.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.414+0000 7f99faffd700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f99e8067050 0x7f99e8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:06.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.422+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99fc0b3540 con 0x7f99fc0b0160 2023-12-26T21:52:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.422+0000 7f99e37fe700 1 -- 172.21.15.131:0/1014443920 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99ec056510 con 0x7f99fc0b0160 2023-12-26T21:52:06.423 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:52:06.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f99e8067050 msgr2=0x7f99e8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f99e8067050 0x7f99e8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc0b0160 msgr2=0x7f99fc156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc0b0160 0x7f99fc156bd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f99fc089710 tx=0x7f99ec017000 comp rx=0 tx=0).stop 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 shutdown_connections 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f99e8067050 0x7f99e8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc159a70 0x7f99fc0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc0b0160 0x7f99fc156bd0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 --2- 172.21.15.131:0/1014443920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc1596a0 0x7f99fc0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 >> 172.21.15.131:0/1014443920 conn(0x7f99fc06ece0 msgr2=0x7f99fc0b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 shutdown_connections 2023-12-26T21:52:06.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.425+0000 7f9a01dff700 1 -- 172.21.15.131:0/1014443920 wait complete. 2023-12-26T21:52:06.444 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:52:06.525 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:06.524+0000 7f1e100561c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:52:06.640 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:06.639+0000 7f66f89581c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:52:06.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.640+0000 7fe4f4fff700 1 Processor -- start 2023-12-26T21:52:06.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.640+0000 7fe4f4fff700 1 -- start start 2023-12-26T21:52:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.641+0000 7fe4f4fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe4f01596b0 0x7fe4f0067f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.642+0000 7fe4f4fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 0x7fe4f00c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.642+0000 7fe4f4fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f015ac30 con 0x7fe4f00c7df0 2023-12-26T21:52:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.642+0000 7fe4f4fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00c8270 con 0x7fe4f01596b0 2023-12-26T21:52:06.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.642+0000 7fe4f4fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0068480 con 0x7fe4f0159a80 2023-12-26T21:52:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.642+0000 7fe4ee59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe4f01596b0 0x7fe4f0067f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.642+0000 7fe4ee59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe4f01596b0 0x7fe4f0067f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:34940/0 (socket says 172.21.15.131:34940) 2023-12-26T21:52:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.642+0000 7fe4ee59c700 1 -- 172.21.15.131:0/4114260791 learned_addr learned my addr 172.21.15.131:0/4114260791 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.643+0000 7fe4edd9b700 1 --2- 172.21.15.131:0/4114260791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 0x7fe4f00c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.643+0000 7fe4ee59c700 1 -- 172.21.15.131:0/4114260791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 msgr2=0x7fe4f00c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.643+0000 7fe4ee59c700 1 --2- 172.21.15.131:0/4114260791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 0x7fe4f00c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.643+0000 7fe4ee59c700 1 -- 172.21.15.131:0/4114260791 >> v1:172.21.15.190:6789/0 conn(0x7fe4f00c7df0 legacy=0x7fe4f006a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.643+0000 7fe4ee59c700 1 -- 172.21.15.131:0/4114260791 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f015a020 con 0x7fe4f01596b0 2023-12-26T21:52:06.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.644+0000 7fe4ee59c700 1 --2- 172.21.15.131:0/4114260791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 0x7fe4f0067f40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00099b0 tx=0x7fe4e0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1fdd2ae5631fc4d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:06.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.645+0000 7fe4ed59a700 1 -- 172.21.15.131:0/4114260791 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0009ea0 con 0x7fe4f01596b0 2023-12-26T21:52:06.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.645+0000 7fe4ed59a700 1 -- 172.21.15.131:0/4114260791 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e000b760 con 0x7fe4f01596b0 2023-12-26T21:52:06.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.645+0000 7fe4ed59a700 1 -- 172.21.15.131:0/4114260791 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e000bb20 con 0x7fe4f01596b0 2023-12-26T21:52:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.647+0000 7fe4f4fff700 1 -- 172.21.15.131:0/4114260791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 msgr2=0x7fe4f0067f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.647+0000 7fe4f4fff700 1 --2- 172.21.15.131:0/4114260791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 0x7fe4f0067f40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00099b0 tx=0x7fe4e0009250 comp rx=0 tx=0).stop 2023-12-26T21:52:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.648+0000 7fe4f4fff700 1 -- 172.21.15.131:0/4114260791 shutdown_connections 2023-12-26T21:52:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.648+0000 7fe4f4fff700 1 --2- 172.21.15.131:0/4114260791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 0x7fe4f0067f40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.648+0000 7fe4f4fff700 1 --2- 172.21.15.131:0/4114260791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 0x7fe4f00c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.648+0000 7fe4f4fff700 1 -- 172.21.15.131:0/4114260791 >> 172.21.15.131:0/4114260791 conn(0x7fe4f00686c0 msgr2=0x7fe4f0068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:06.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.648+0000 7fe4f4fff700 1 -- 172.21.15.131:0/4114260791 shutdown_connections 2023-12-26T21:52:06.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.648+0000 7fe4f4fff700 1 -- 172.21.15.131:0/4114260791 wait complete. 2023-12-26T21:52:06.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.651+0000 7fe4f4fff700 1 Processor -- start 2023-12-26T21:52:06.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.651+0000 7fe4f4fff700 1 -- start start 2023-12-26T21:52:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4f4fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f00c7df0 0x7fe4f015de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4f4fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 0x7fe4f014a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4f4fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 0x7fe4f014a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4f4fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f015b5b0 con 0x7fe4f00c7df0 2023-12-26T21:52:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4f4fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f015c080 con 0x7fe4f01596b0 2023-12-26T21:52:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4f4fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00c6a90 con 0x7fe4f0159a80 2023-12-26T21:52:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4ee59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f00c7df0 0x7fe4f015de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4edd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 0x7fe4f014a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4eed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 0x7fe4f014a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:06.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4edd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 0x7fe4f014a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:34948/0 (socket says 172.21.15.131:34948) 2023-12-26T21:52:06.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4ee59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f00c7df0 0x7fe4f015de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60914/0 (socket says 172.21.15.131:60914) 2023-12-26T21:52:06.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.652+0000 7fe4edd9b700 1 -- 172.21.15.131:0/1096684250 learned_addr learned my addr 172.21.15.131:0/1096684250 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4eed9d700 1 -- 172.21.15.131:0/1096684250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 msgr2=0x7fe4f014a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4eed9d700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 0x7fe4f014a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4eed9d700 1 -- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f00c7df0 msgr2=0x7fe4f015de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4eed9d700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f00c7df0 0x7fe4f015de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4eed9d700 1 -- 172.21.15.131:0/1096684250 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00094f0 con 0x7fe4f0159a80 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4edd9b700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 0x7fe4f014a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4eed9d700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 0x7fe4f014a8b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c8c0 tx=0x7fe4e8001690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4de7fc700 1 -- 172.21.15.131:0/1096684250 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8012630 con 0x7fe4f0159a80 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4de7fc700 1 -- 172.21.15.131:0/1096684250 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e8012790 con 0x7fe4f0159a80 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f00c6d70 con 0x7fe4f0159a80 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.653+0000 7fe4de7fc700 1 -- 172.21.15.131:0/1096684250 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8012b50 con 0x7fe4f0159a80 2023-12-26T21:52:06.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.654+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f015bd10 con 0x7fe4f0159a80 2023-12-26T21:52:06.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.655+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f015c230 con 0x7fe4f0159a80 2023-12-26T21:52:06.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.655+0000 7fe4de7fc700 1 -- 172.21.15.131:0/1096684250 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fe4e8003030 con 0x7fe4f0159a80 2023-12-26T21:52:06.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.656+0000 7fe4de7fc700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe4d0067050 0x7fe4d0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:06.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.656+0000 7fe4de7fc700 1 -- 172.21.15.131:0/1096684250 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6544+0+0 (secure 0 0 0) 0x7fe4e8084920 con 0x7fe4f0159a80 2023-12-26T21:52:06.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.656+0000 7fe4ee59c700 1 -- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe4d0067050 msgr2=0x7fe4d0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:06.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.656+0000 7fe4ee59c700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe4d0067050 0x7fe4d0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:06.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.661+0000 7fe4de7fc700 1 -- 172.21.15.131:0/1096684250 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4f015c230 con 0x7fe4f0159a80 2023-12-26T21:52:06.819 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:06.819+0000 7f1e100561c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:52:06.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.857+0000 7fe4ee59c700 1 -- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe4d0067050 msgr2=0x7fe4d0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:06.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.857+0000 7fe4ee59c700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe4d0067050 0x7fe4d0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:06.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.880+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4f015c1f0 con 0x7fe4f0159a80 2023-12-26T21:52:06.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.881+0000 7fe4de7fc700 1 -- 172.21.15.131:0/1096684250 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4e80516c0 con 0x7fe4f0159a80 2023-12-26T21:52:06.881 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:52:06.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe4d0067050 msgr2=0x7fe4d0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:06.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe4d0067050 0x7fe4d0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 msgr2=0x7fe4f014a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:06.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 0x7fe4f014a8b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c8c0 tx=0x7fe4e8001690 comp rx=0 tx=0).stop 2023-12-26T21:52:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 shutdown_connections 2023-12-26T21:52:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe4d0067050 0x7fe4d0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f0159a80 0x7fe4f014a8b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4f00c7df0 0x7fe4f015de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 --2- 172.21.15.131:0/1096684250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f01596b0 0x7fe4f014a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.883+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 >> 172.21.15.131:0/1096684250 conn(0x7fe4f00686c0 msgr2=0x7fe4f00cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.884+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 shutdown_connections 2023-12-26T21:52:06.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:06.884+0000 7fe4f4fff700 1 -- 172.21.15.131:0/1096684250 wait complete. 2023-12-26T21:52:06.903 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:06.921 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:06.921+0000 7f66f89581c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:52:07.011 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:07.010+0000 7f66f89581c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:52:07.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.105+0000 7fcddcfff700 1 Processor -- start 2023-12-26T21:52:07.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.106+0000 7fcddcfff700 1 -- start start 2023-12-26T21:52:07.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.108+0000 7fcddcfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd8159820 0x7fcdd80abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.108+0000 7fcddcfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcdd80aa480 0x7fcdd806dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.108+0000 7fcddcfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd815a8c0 con 0x7fcdd806e2b0 2023-12-26T21:52:07.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.108+0000 7fcddcfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd815ad50 con 0x7fcdd80aa480 2023-12-26T21:52:07.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.108+0000 7fcddcfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd806e730 con 0x7fcdd8159820 2023-12-26T21:52:07.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.108+0000 7fcdd659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd8159820 0x7fcdd80abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.108+0000 7fcdd5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcdd80aa480 0x7fcdd806dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.108+0000 7fcdd6d9d700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fcdd806e2b0 0x7fcdd815d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:42366/0 (socket says 172.21.15.131:42366) 2023-12-26T21:52:07.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.108+0000 7fcdd6d9d700 1 -- 172.21.15.131:0/1205704850 learned_addr learned my addr 172.21.15.131:0/1205704850 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.109+0000 7fcdd5d9b700 1 -- 172.21.15.131:0/1205704850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd8159820 msgr2=0x7fcdd80abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.109+0000 7fcdd5d9b700 1 --2- 172.21.15.131:0/1205704850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd8159820 0x7fcdd80abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.109+0000 7fcdd5d9b700 1 -- 172.21.15.131:0/1205704850 >> v1:172.21.15.190:6789/0 conn(0x7fcdd806e2b0 legacy=0x7fcdd815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.109+0000 7fcdd5d9b700 1 -- 172.21.15.131:0/1205704850 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8159f90 con 0x7fcdd80aa480 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.109+0000 7fcdd559a700 1 -- 172.21.15.131:0/1205704850 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396217332 0 0) 0x7fcdd815a8c0 con 0x7fcdd806e2b0 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.109+0000 7fcdd5d9b700 1 --2- 172.21.15.131:0/1205704850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd80aa480 0x7fcdd806dd70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009910 tx=0x7fcdcc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf495f767735c2c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.109+0000 7fcdd559a700 1 -- 172.21.15.131:0/1205704850 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00b640 con 0x7fcdd80aa480 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.109+0000 7fcdd559a700 1 -- 172.21.15.131:0/1205704850 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc00b7a0 con 0x7fcdd80aa480 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.109+0000 7fcdd559a700 1 -- 172.21.15.131:0/1205704850 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00bb60 con 0x7fcdd80aa480 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.110+0000 7fcddcfff700 1 -- 172.21.15.131:0/1205704850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd80aa480 msgr2=0x7fcdd806dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.110+0000 7fcddcfff700 1 --2- 172.21.15.131:0/1205704850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd80aa480 0x7fcdd806dd70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009910 tx=0x7fcdcc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.110+0000 7fcddcfff700 1 -- 172.21.15.131:0/1205704850 shutdown_connections 2023-12-26T21:52:07.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.110+0000 7fcddcfff700 1 --2- 172.21.15.131:0/1205704850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd8159820 0x7fcdd80abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.110+0000 7fcddcfff700 1 --2- 172.21.15.131:0/1205704850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd80aa480 0x7fcdd806dd70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.110+0000 7fcddcfff700 1 -- 172.21.15.131:0/1205704850 >> 172.21.15.131:0/1205704850 conn(0x7fcdd806c290 msgr2=0x7fcdd80ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:07.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.110+0000 7fcddcfff700 1 -- 172.21.15.131:0/1205704850 shutdown_connections 2023-12-26T21:52:07.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.110+0000 7fcddcfff700 1 -- 172.21.15.131:0/1205704850 wait complete. 2023-12-26T21:52:07.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.112+0000 7fcddcfff700 1 Processor -- start 2023-12-26T21:52:07.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.112+0000 7fcddcfff700 1 -- start start 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcddcfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd806e2b0 0x7fcdd816afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcddcfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd80aa480 0x7fcdd816dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcddcfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd8159820 0x7fcdd80cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcddcfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd816b5f0 con 0x7fcdd806e2b0 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcddcfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd816e020 con 0x7fcdd8159820 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcddcfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd816b8b0 con 0x7fcdd80aa480 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcdd659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd806e2b0 0x7fcdd816afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcdd5d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd80aa480 0x7fcdd816dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcdd659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd806e2b0 0x7fcdd816afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60926/0 (socket says 172.21.15.131:60926) 2023-12-26T21:52:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcdd659c700 1 -- 172.21.15.131:0/2595264631 learned_addr learned my addr 172.21.15.131:0/2595264631 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.113+0000 7fcdd6d9d700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd8159820 0x7fcdd80cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdd659c700 1 -- 172.21.15.131:0/2595264631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd80aa480 msgr2=0x7fcdd816dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdd659c700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd80aa480 0x7fcdd816dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdd659c700 1 -- 172.21.15.131:0/2595264631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd8159820 msgr2=0x7fcdd80cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdd659c700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd8159820 0x7fcdd80cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdd659c700 1 -- 172.21.15.131:0/2595264631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc009420 con 0x7fcdd806e2b0 2023-12-26T21:52:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdd5d9b700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd80aa480 0x7fcdd816dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdd6d9d700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd8159820 0x7fcdd80cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:07.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdd659c700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd806e2b0 0x7fcdd816afa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800ec80 tx=0x7fcdc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:07.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdc67fc700 1 -- 172.21.15.131:0/2595264631 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc800c850 con 0x7fcdd806e2b0 2023-12-26T21:52:07.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdc67fc700 1 -- 172.21.15.131:0/2595264631 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc800c9b0 con 0x7fcdd806e2b0 2023-12-26T21:52:07.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd816e250 con 0x7fcdd806e2b0 2023-12-26T21:52:07.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcdc67fc700 1 -- 172.21.15.131:0/2595264631 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc800cd50 con 0x7fcdd806e2b0 2023-12-26T21:52:07.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.114+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd816e810 con 0x7fcdd806e2b0 2023-12-26T21:52:07.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.115+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd816bb90 con 0x7fcdd806e2b0 2023-12-26T21:52:07.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.116+0000 7fcdc67fc700 1 -- 172.21.15.131:0/2595264631 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fcdc8013680 con 0x7fcdd806e2b0 2023-12-26T21:52:07.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.118+0000 7fcdc67fc700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdb8067050 0x7fcdb8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.118+0000 7fcdc67fc700 1 -- 172.21.15.131:0/2595264631 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6544+0+0 (secure 0 0 0) 0x7fcdc8084cd0 con 0x7fcdd806e2b0 2023-12-26T21:52:07.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.118+0000 7fcdd5d9b700 1 -- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdb8067050 msgr2=0x7fcdb8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:07.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.118+0000 7fcdd5d9b700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdb8067050 0x7fcdb8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:07.121 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:07.121+0000 7f1e100561c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:52:07.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.122+0000 7fcdc67fc700 1 -- 172.21.15.131:0/2595264631 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdc8051a70 con 0x7fcdd806e2b0 2023-12-26T21:52:07.214 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:07.213+0000 7f1e100561c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:52:07.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.319+0000 7fcdd5d9b700 1 -- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdb8067050 msgr2=0x7fcdb8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:07.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.319+0000 7fcdd5d9b700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdb8067050 0x7fcdb8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:07.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.368+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdd80657e0 con 0x7fcdd806e2b0 2023-12-26T21:52:07.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.369+0000 7fcdc67fc700 1 -- 172.21.15.131:0/2595264631 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fcdc8055090 con 0x7fcdd806e2b0 2023-12-26T21:52:07.369 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:07.369 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3316959664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3316959664},{"type":"v1","addr":"172.21.15.190:6834","nonce":3316959664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/320470110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":320470110},{"type":"v1","addr":"172.21.15.190:6835","nonce":320470110}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/1337439654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1337439654},{"type":"v1","addr":"172.21.15.131:6836","nonce":1337439654}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:50:04.699122+0000","modified":"2023-12-26T21:50:08.232778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6837/607340038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":607340038},{"type":"v1","addr":"172.21.15.131:6837","nonce":607340038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-12-26T21:52:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdb8067050 msgr2=0x7fcdb8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdb8067050 0x7fcdb8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd806e2b0 msgr2=0x7fcdd816afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd806e2b0 0x7fcdd816afa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800ec80 tx=0x7fcdc80091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 shutdown_connections 2023-12-26T21:52:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd8159820 0x7fcdd80cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdd806e2b0 0x7fcdd816afa0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fcdb8067050 0x7fcdb8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 --2- 172.21.15.131:0/2595264631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd80aa480 0x7fcdd816dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 >> 172.21.15.131:0/2595264631 conn(0x7fcdd806c290 msgr2=0x7fcdd8158730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:07.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 shutdown_connections 2023-12-26T21:52:07.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.372+0000 7fcddcfff700 1 -- 172.21.15.131:0/2595264631 wait complete. 2023-12-26T21:52:07.374 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 25 2023-12-26T21:52:07.396 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.588+0000 7fe2febff700 1 Processor -- start 2023-12-26T21:52:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.588+0000 7fe2febff700 1 -- start start 2023-12-26T21:52:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.589+0000 7fe2febff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2f8158030 0x7fe2f8158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.589+0000 7fe2febff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f806b8c0 0x7fe2f80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.589+0000 7fe2febff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f815ba70 con 0x7fe2f806bd40 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2febff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f815bf00 con 0x7fe2f8158030 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2febff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f806c2a0 con 0x7fe2f806b8c0 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2fc99b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2f8158030 0x7fe2f8158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2fd19c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe2f806bd40 0x7fe2f806cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44384/0 (socket says 172.21.15.131:44384) 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2fd19c700 1 -- 172.21.15.131:0/3103746368 learned_addr learned my addr 172.21.15.131:0/3103746368 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2f7fff700 1 --2- 172.21.15.131:0/3103746368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f806b8c0 0x7fe2f80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2fc99b700 1 -- 172.21.15.131:0/3103746368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f806b8c0 msgr2=0x7fe2f80c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2fc99b700 1 --2- 172.21.15.131:0/3103746368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f806b8c0 0x7fe2f80c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2fc99b700 1 -- 172.21.15.131:0/3103746368 >> v1:172.21.15.131:6789/0 conn(0x7fe2f806bd40 legacy=0x7fe2f806cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.590+0000 7fe2fc99b700 1 -- 172.21.15.131:0/3103746368 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f815b140 con 0x7fe2f8158030 2023-12-26T21:52:07.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.591+0000 7fe2fc99b700 1 --2- 172.21.15.131:0/3103746368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f8158030 0x7fe2f8158400 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0098d0 tx=0x7fe2ec009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75f8eee47c6da43 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:07.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.591+0000 7fe2f77fe700 1 -- 172.21.15.131:0/3103746368 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec005c10 con 0x7fe2f8158030 2023-12-26T21:52:07.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.591+0000 7fe2f77fe700 1 -- 172.21.15.131:0/3103746368 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec005d70 con 0x7fe2f8158030 2023-12-26T21:52:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.591+0000 7fe2f77fe700 1 -- 172.21.15.131:0/3103746368 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec006280 con 0x7fe2f8158030 2023-12-26T21:52:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.591+0000 7fe2febff700 1 -- 172.21.15.131:0/3103746368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f8158030 msgr2=0x7fe2f8158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.591+0000 7fe2febff700 1 --2- 172.21.15.131:0/3103746368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f8158030 0x7fe2f8158400 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0098d0 tx=0x7fe2ec009120 comp rx=0 tx=0).stop 2023-12-26T21:52:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.591+0000 7fe2febff700 1 -- 172.21.15.131:0/3103746368 shutdown_connections 2023-12-26T21:52:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.591+0000 7fe2febff700 1 --2- 172.21.15.131:0/3103746368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f8158030 0x7fe2f8158400 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.591+0000 7fe2febff700 1 --2- 172.21.15.131:0/3103746368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f806b8c0 0x7fe2f80c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.592+0000 7fe2febff700 1 -- 172.21.15.131:0/3103746368 >> 172.21.15.131:0/3103746368 conn(0x7fe2f80ce3e0 msgr2=0x7fe2f80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.592+0000 7fe2febff700 1 -- 172.21.15.131:0/3103746368 shutdown_connections 2023-12-26T21:52:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.592+0000 7fe2febff700 1 -- 172.21.15.131:0/3103746368 wait complete. 2023-12-26T21:52:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.593+0000 7fe2febff700 1 Processor -- start 2023-12-26T21:52:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.593+0000 7fe2febff700 1 -- start start 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2febff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f806b8c0 0x7fe2f816a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2febff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806bd40 0x7fe2f814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2febff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f8158030 0x7fe2f81500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2febff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f816a9c0 con 0x7fe2f806b8c0 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2febff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f814fc40 con 0x7fe2f806bd40 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2febff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f814cd10 con 0x7fe2f8158030 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2fc99b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f806b8c0 0x7fe2f816a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2fc99b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f806b8c0 0x7fe2f816a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60938/0 (socket says 172.21.15.131:60938) 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2fc99b700 1 -- 172.21.15.131:0/675639191 learned_addr learned my addr 172.21.15.131:0/675639191 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2fd19c700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f8158030 0x7fe2f81500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.594+0000 7fe2f7fff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806bd40 0x7fe2f814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2fc99b700 1 -- 172.21.15.131:0/675639191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f8158030 msgr2=0x7fe2f81500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2fc99b700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f8158030 0x7fe2f81500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2fc99b700 1 -- 172.21.15.131:0/675639191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806bd40 msgr2=0x7fe2f814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2fc99b700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806bd40 0x7fe2f814c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2fc99b700 1 -- 172.21.15.131:0/675639191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec0093c0 con 0x7fe2f806b8c0 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2f7fff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806bd40 0x7fe2f814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2fc99b700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f806b8c0 0x7fe2f816a480 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009f70 tx=0x7fe2ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2f4ff9700 1 -- 172.21.15.131:0/675639191 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec01d780 con 0x7fe2f806b8c0 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2f4ff9700 1 -- 172.21.15.131:0/675639191 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec01d8e0 con 0x7fe2f806b8c0 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2f4ff9700 1 -- 172.21.15.131:0/675639191 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec01dc80 con 0x7fe2f806b8c0 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f814f030 con 0x7fe2f806b8c0 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.595+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f814f5f0 con 0x7fe2f806b8c0 2023-12-26T21:52:07.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.596+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f80657e0 con 0x7fe2f806b8c0 2023-12-26T21:52:07.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.598+0000 7fe2f4ff9700 1 -- 172.21.15.131:0/675639191 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fe2ec01b690 con 0x7fe2f806b8c0 2023-12-26T21:52:07.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.598+0000 7fe2f4ff9700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2dc067050 0x7fe2dc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:07.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.599+0000 7fe2f4ff9700 1 -- 172.21.15.131:0/675639191 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6544+0+0 (secure 0 0 0) 0x7fe2ec086ee0 con 0x7fe2f806b8c0 2023-12-26T21:52:07.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.599+0000 7fe2f7fff700 1 -- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2dc067050 msgr2=0x7fe2dc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:07.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.599+0000 7fe2f7fff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2dc067050 0x7fe2dc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:07.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.606+0000 7fe2f4ff9700 1 -- 172.21.15.131:0/675639191 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2ec053d80 con 0x7fe2f806b8c0 2023-12-26T21:52:07.745 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:07.745+0000 7f66f89581c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:52:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.800+0000 7fe2f7fff700 1 -- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2dc067050 msgr2=0x7fe2dc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.800+0000 7fe2f7fff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2dc067050 0x7fe2dc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.807+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2f806c640 con 0x7fe2f806b8c0 2023-12-26T21:52:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.808+0000 7fe2f4ff9700 1 -- 172.21.15.131:0/675639191 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16690 (secure 0 0 0) 0x7fe2ec033070 con 0x7fe2f806b8c0 2023-12-26T21:52:07.809 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:07.809 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":69,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:06.245076+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:50:02.860261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-26T21:50:03.665608+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":0}]},{"pgid":"7.15","mappings":[{"from":5,"to":7}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2945752061":"2023-12-27T21:50:25.498209+0000","172.21.15.131:6833/39288":"2023-12-27T21:50:25.498209+0000","172.21.15.131:6832/39288":"2023-12-27T21:50:25.498209+0000","172.21.15.131:0/2402413168":"2023-12-27T21:50:25.498209+0000","172.21.15.131:0/471355450":"2023-12-27T21:50:25.498209+0000","172.21.15.131:0/3935646470":"2023-12-27T21:50:25.498209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:07.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2dc067050 msgr2=0x7fe2dc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2dc067050 0x7fe2dc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f806b8c0 msgr2=0x7fe2f816a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f806b8c0 0x7fe2f816a480 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009f70 tx=0x7fe2ec016000 comp rx=0 tx=0).stop 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 shutdown_connections 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7fe2dc067050 0x7fe2dc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f8158030 0x7fe2f81500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f806b8c0 0x7fe2f816a480 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 --2- 172.21.15.131:0/675639191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806bd40 0x7fe2f814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 >> 172.21.15.131:0/675639191 conn(0x7fe2f80ce3e0 msgr2=0x7fe2f815a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 shutdown_connections 2023-12-26T21:52:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:07.811+0000 7fe2febff700 1 -- 172.21.15.131:0/675639191 wait complete. 2023-12-26T21:52:07.830 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T21:52:07.830 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:07.838 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:07.838+0000 7f66f89581c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:52:07.990 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:07.990+0000 7f66f89581c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:52:07.999 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:07.999+0000 7f1e100561c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:52:08.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.024+0000 7f5629bff700 1 Processor -- start 2023-12-26T21:52:08.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.024+0000 7f5629bff700 1 -- start start 2023-12-26T21:52:08.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.025+0000 7f5629bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f562415d240 0x7f56240ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.025+0000 7f5629bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56240ad390 0x7f56240aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.025+0000 7f5629bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56240a98b0 con 0x7f56241595e0 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.025+0000 7f5629bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56240add40 con 0x7f562415d240 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.025+0000 7f5629bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56240ad790 con 0x7f56240ad390 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.025+0000 7f5622ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f562415d240 0x7f56240ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f5622ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f562415d240 0x7f56240ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60954/0 (socket says 172.21.15.131:60954) 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f5622ffd700 1 -- 172.21.15.131:0/3007381852 learned_addr learned my addr 172.21.15.131:0/3007381852 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f5623fff700 1 --2- 172.21.15.131:0/3007381852 >> v2:172.21.15.190:3300/0 conn(0x7f56240ad390 0x7f56240aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f5622ffd700 1 -- 172.21.15.131:0/3007381852 >> v2:172.21.15.190:3300/0 conn(0x7f56240ad390 msgr2=0x7f56240aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f5622ffd700 1 --2- 172.21.15.131:0/3007381852 >> v2:172.21.15.190:3300/0 conn(0x7f56240ad390 0x7f56240aa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f5622ffd700 1 -- 172.21.15.131:0/3007381852 >> v1:172.21.15.190:6789/0 conn(0x7f56241595e0 legacy=0x7f56241599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f5622ffd700 1 -- 172.21.15.131:0/3007381852 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562415a500 con 0x7f562415d240 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f5623fff700 1 --2- 172.21.15.131:0/3007381852 >> v2:172.21.15.190:3300/0 conn(0x7f56240ad390 0x7f56240aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:08.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f5622ffd700 1 --2- 172.21.15.131:0/3007381852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f562415d240 0x7f56240ace50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f562406b450 tx=0x7f5618009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc59fb517da12f0b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.026+0000 7f56227fc700 1 -- 172.21.15.131:0/3007381852 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618009e70 con 0x7f562415d240 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f56227fc700 1 -- 172.21.15.131:0/3007381852 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561800b860 con 0x7f562415d240 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f56227fc700 1 -- 172.21.15.131:0/3007381852 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561800bc00 con 0x7f562415d240 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f5629bff700 1 -- 172.21.15.131:0/3007381852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f562415d240 msgr2=0x7f56240ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f5629bff700 1 --2- 172.21.15.131:0/3007381852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f562415d240 0x7f56240ace50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f562406b450 tx=0x7f5618009250 comp rx=0 tx=0).stop 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f5629bff700 1 -- 172.21.15.131:0/3007381852 shutdown_connections 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f5629bff700 1 --2- 172.21.15.131:0/3007381852 >> v2:172.21.15.190:3300/0 conn(0x7f56240ad390 0x7f56240aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f5629bff700 1 --2- 172.21.15.131:0/3007381852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f562415d240 0x7f56240ace50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f5629bff700 1 -- 172.21.15.131:0/3007381852 >> 172.21.15.131:0/3007381852 conn(0x7f5624068040 msgr2=0x7f5624069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f5629bff700 1 -- 172.21.15.131:0/3007381852 shutdown_connections 2023-12-26T21:52:08.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.027+0000 7f5629bff700 1 -- 172.21.15.131:0/3007381852 wait complete. 2023-12-26T21:52:08.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.029+0000 7f5629bff700 1 Processor -- start 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.029+0000 7f5629bff700 1 -- start start 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f5629bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56240ad390 0x7f5624140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f5629bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56241595e0 0x7f562415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f5629bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f562415d240 0x7f56241555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f5629bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562415df90 con 0x7f56240ad390 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f5629bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56241411a0 con 0x7f562415d240 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f5629bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5624156190 con 0x7f56241595e0 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f56237fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56240ad390 0x7f5624140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f56237fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56240ad390 0x7f5624140c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60962/0 (socket says 172.21.15.131:60962) 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f56237fe700 1 -- 172.21.15.131:0/2050242326 learned_addr learned my addr 172.21.15.131:0/2050242326 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f5623fff700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f562415d240 0x7f56241555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f5622ffd700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56241595e0 0x7f562415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f56237fe700 1 -- 172.21.15.131:0/2050242326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56241595e0 msgr2=0x7f562415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f56237fe700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56241595e0 0x7f562415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f56237fe700 1 -- 172.21.15.131:0/2050242326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f562415d240 msgr2=0x7f56241555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f56237fe700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f562415d240 0x7f56241555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.030+0000 7f56237fe700 1 -- 172.21.15.131:0/2050242326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c009a20 con 0x7f56240ad390 2023-12-26T21:52:08.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.031+0000 7f56237fe700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56240ad390 0x7f5624140c60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f561400c590 tx=0x7f561400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:08.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.031+0000 7f560b7fe700 1 -- 172.21.15.131:0/2050242326 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614011590 con 0x7f56240ad390 2023-12-26T21:52:08.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.031+0000 7f560b7fe700 1 -- 172.21.15.131:0/2050242326 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614011710 con 0x7f56240ad390 2023-12-26T21:52:08.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.031+0000 7f560b7fe700 1 -- 172.21.15.131:0/2050242326 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614011ad0 con 0x7f56240ad390 2023-12-26T21:52:08.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.031+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56180094f0 con 0x7f56240ad390 2023-12-26T21:52:08.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.031+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5624156750 con 0x7f56240ad390 2023-12-26T21:52:08.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.032+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56240ae1b0 con 0x7f56240ad390 2023-12-26T21:52:08.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.033+0000 7f560b7fe700 1 -- 172.21.15.131:0/2050242326 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f561401b3b0 con 0x7f56240ad390 2023-12-26T21:52:08.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.034+0000 7f560b7fe700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5610067050 0x7f5610069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.034+0000 7f560b7fe700 1 -- 172.21.15.131:0/2050242326 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6544+0+0 (secure 0 0 0) 0x7f5614083e60 con 0x7f56240ad390 2023-12-26T21:52:08.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.034+0000 7f5622ffd700 1 -- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5610067050 msgr2=0x7f5610069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:08.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.034+0000 7f5622ffd700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5610067050 0x7f5610069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:08.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.038+0000 7f560b7fe700 1 -- 172.21.15.131:0/2050242326 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5614050d40 con 0x7f56240ad390 2023-12-26T21:52:08.098 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:08.098+0000 7f1e100561c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:52:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.235+0000 7f5622ffd700 1 -- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5610067050 msgr2=0x7f5610069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.235+0000 7f5622ffd700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5610067050 0x7f5610069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:08.255 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:08.255+0000 7f1e100561c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:52:08.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.285+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56240657e0 con 0x7f56240ad390 2023-12-26T21:52:08.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.286+0000 7f560b7fe700 1 -- 172.21.15.131:0/2050242326 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f5614017070 con 0x7f56240ad390 2023-12-26T21:52:08.286 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:08.286 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3316959664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3316959664},{"type":"v1","addr":"172.21.15.190:6834","nonce":3316959664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/320470110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":320470110},{"type":"v1","addr":"172.21.15.190:6835","nonce":320470110}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/1337439654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1337439654},{"type":"v1","addr":"172.21.15.131:6836","nonce":1337439654}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:50:04.699122+0000","modified":"2023-12-26T21:50:08.232778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6837/607340038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":607340038},{"type":"v1","addr":"172.21.15.131:6837","nonce":607340038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-12-26T21:52:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5610067050 msgr2=0x7f5610069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5610067050 0x7f5610069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56240ad390 msgr2=0x7f5624140c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56240ad390 0x7f5624140c60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f561400c590 tx=0x7f561400c8a0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 shutdown_connections 2023-12-26T21:52:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f562415d240 0x7f56241555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56240ad390 0x7f5624140c60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f5610067050 0x7f5610069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 --2- 172.21.15.131:0/2050242326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56241595e0 0x7f562415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 >> 172.21.15.131:0/2050242326 conn(0x7f5624068040 msgr2=0x7f56240aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:08.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.288+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 shutdown_connections 2023-12-26T21:52:08.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.289+0000 7f5629bff700 1 -- 172.21.15.131:0/2050242326 wait complete. 2023-12-26T21:52:08.290 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 25 2023-12-26T21:52:08.308 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:08.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.501+0000 7f76461ff700 1 Processor -- start 2023-12-26T21:52:08.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.501+0000 7f76461ff700 1 -- start start 2023-12-26T21:52:08.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.502+0000 7f76461ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f764006e480 0x7f76400abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.502+0000 7f76461ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f76400aa480 0x7f76400675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.502+0000 7f76461ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764015a8c0 con 0x7f7640159820 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.502+0000 7f76461ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764015ad50 con 0x7f76400aa480 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.502+0000 7f76461ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7640159f90 con 0x7f764006e480 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f76400aa480 0x7f76400675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763f7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7640159820 0x7f764006dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44396/0 (socket says 172.21.15.131:44396) 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763f7fe700 1 -- 172.21.15.131:0/1499233248 learned_addr learned my addr 172.21.15.131:0/1499233248 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763effd700 1 --2- 172.21.15.131:0/1499233248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f764006e480 0x7f76400abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763ffff700 1 -- 172.21.15.131:0/1499233248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f764006e480 msgr2=0x7f76400abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763ffff700 1 --2- 172.21.15.131:0/1499233248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f764006e480 0x7f76400abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763ffff700 1 -- 172.21.15.131:0/1499233248 >> v1:172.21.15.131:6789/0 conn(0x7f7640159820 legacy=0x7f764006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763ffff700 1 -- 172.21.15.131:0/1499233248 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764015a270 con 0x7f76400aa480 2023-12-26T21:52:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763e7fc700 1 -- 172.21.15.131:0/1499233248 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271591686 0 0) 0x7f764015a8c0 con 0x7f7640159820 2023-12-26T21:52:08.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.503+0000 7f763ffff700 1 --2- 172.21.15.131:0/1499233248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76400aa480 0x7f76400675b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7638009e60 tx=0x7f7638009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0e1e54899dc83f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.504+0000 7f763e7fc700 1 -- 172.21.15.131:0/1499233248 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800e070 con 0x7f76400aa480 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.504+0000 7f763e7fc700 1 -- 172.21.15.131:0/1499233248 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763800bb60 con 0x7f76400aa480 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.504+0000 7f763e7fc700 1 -- 172.21.15.131:0/1499233248 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638012430 con 0x7f76400aa480 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.504+0000 7f76461ff700 1 -- 172.21.15.131:0/1499233248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76400aa480 msgr2=0x7f76400675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.504+0000 7f76461ff700 1 --2- 172.21.15.131:0/1499233248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76400aa480 0x7f76400675b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7638009e60 tx=0x7f7638009780 comp rx=0 tx=0).stop 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.504+0000 7f76461ff700 1 -- 172.21.15.131:0/1499233248 shutdown_connections 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.504+0000 7f76461ff700 1 --2- 172.21.15.131:0/1499233248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76400aa480 0x7f76400675b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.504+0000 7f76461ff700 1 --2- 172.21.15.131:0/1499233248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f764006e480 0x7f76400abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.504+0000 7f76461ff700 1 -- 172.21.15.131:0/1499233248 >> 172.21.15.131:0/1499233248 conn(0x7f764006c290 msgr2=0x7f76400ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.505+0000 7f76461ff700 1 -- 172.21.15.131:0/1499233248 shutdown_connections 2023-12-26T21:52:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.505+0000 7f76461ff700 1 -- 172.21.15.131:0/1499233248 wait complete. 2023-12-26T21:52:08.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.506+0000 7f76461ff700 1 Processor -- start 2023-12-26T21:52:08.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.506+0000 7f76461ff700 1 -- start start 2023-12-26T21:52:08.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f76461ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f764006e480 0x7f7640146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f76461ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76400aa480 0x7f76400ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f76461ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7640159820 0x7f76400adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f76461ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764016ddd0 con 0x7f764006e480 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f76461ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764016d1c0 con 0x7f7640159820 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f76461ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764016d4a0 con 0x7f76400aa480 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f763f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f764006e480 0x7f7640146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f763f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f764006e480 0x7f7640146390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60966/0 (socket says 172.21.15.131:60966) 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f763f7fe700 1 -- 172.21.15.131:0/4009167768 learned_addr learned my addr 172.21.15.131:0/4009167768 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f763effd700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76400aa480 0x7f76400ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.507+0000 7f763ffff700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7640159820 0x7f76400adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f763f7fe700 1 -- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76400aa480 msgr2=0x7f76400ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f763f7fe700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76400aa480 0x7f76400ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f763f7fe700 1 -- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7640159820 msgr2=0x7f76400adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f763f7fe700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7640159820 0x7f76400adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f763f7fe700 1 -- 172.21.15.131:0/4009167768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7638009a20 con 0x7f764006e480 2023-12-26T21:52:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f763effd700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76400aa480 0x7f76400ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f763ffff700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7640159820 0x7f76400adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f763f7fe700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f764006e480 0x7f7640146390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f763000c7e0 tx=0x7f7630007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f76277fe700 1 -- 172.21.15.131:0/4009167768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630005b30 con 0x7f764006e480 2023-12-26T21:52:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f76277fe700 1 -- 172.21.15.131:0/4009167768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7630005c90 con 0x7f764006e480 2023-12-26T21:52:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764016d780 con 0x7f764006e480 2023-12-26T21:52:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.508+0000 7f76277fe700 1 -- 172.21.15.131:0/4009167768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630007300 con 0x7f764006e480 2023-12-26T21:52:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.509+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76401468d0 con 0x7f764006e480 2023-12-26T21:52:08.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.510+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76400657e0 con 0x7f764006e480 2023-12-26T21:52:08.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.510+0000 7f76277fe700 1 -- 172.21.15.131:0/4009167768 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f763001b3b0 con 0x7f764006e480 2023-12-26T21:52:08.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.512+0000 7f76277fe700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f762c067050 0x7f762c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.512+0000 7f763effd700 1 -- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f762c067050 msgr2=0x7f762c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:08.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.512+0000 7f76277fe700 1 -- 172.21.15.131:0/4009167768 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6544+0+0 (secure 0 0 0) 0x7f7630083f90 con 0x7f764006e480 2023-12-26T21:52:08.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.512+0000 7f763effd700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f762c067050 0x7f762c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.516+0000 7f76277fe700 1 -- 172.21.15.131:0/4009167768 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7630050d30 con 0x7f764006e480 2023-12-26T21:52:08.609 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:08.608+0000 7f66f89581c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:52:08.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.713+0000 7f763effd700 1 -- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f762c067050 msgr2=0x7f762c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:08.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.713+0000 7f763effd700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f762c067050 0x7f762c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:08.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.762+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f764015a550 con 0x7f764006e480 2023-12-26T21:52:08.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.762+0000 7f76277fe700 1 -- 172.21.15.131:0/4009167768 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f7630017020 con 0x7f764006e480 2023-12-26T21:52:08.763 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:08.763 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3316959664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3316959664},{"type":"v1","addr":"172.21.15.190:6834","nonce":3316959664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/320470110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":320470110},{"type":"v1","addr":"172.21.15.190:6835","nonce":320470110}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/1337439654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1337439654},{"type":"v1","addr":"172.21.15.131:6836","nonce":1337439654}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:50:04.699122+0000","modified":"2023-12-26T21:50:08.232778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6837/607340038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":607340038},{"type":"v1","addr":"172.21.15.131:6837","nonce":607340038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-12-26T21:52:08.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.764+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f762c067050 msgr2=0x7f762c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:08.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.764+0000 7f76461ff700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f762c067050 0x7f762c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.764+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f764006e480 msgr2=0x7f7640146390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.764+0000 7f76461ff700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f764006e480 0x7f7640146390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f763000c7e0 tx=0x7f7630007df0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.765+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 shutdown_connections 2023-12-26T21:52:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.765+0000 7f76461ff700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7640159820 0x7f76400adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.765+0000 7f76461ff700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f764006e480 0x7f7640146390 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.765+0000 7f76461ff700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f762c067050 0x7f762c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.765+0000 7f76461ff700 1 --2- 172.21.15.131:0/4009167768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76400aa480 0x7f76400ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.765+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 >> 172.21.15.131:0/4009167768 conn(0x7f764006c290 msgr2=0x7f764015b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.765+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 shutdown_connections 2023-12-26T21:52:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.765+0000 7f76461ff700 1 -- 172.21.15.131:0/4009167768 wait complete. 2023-12-26T21:52:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 25 2023-12-26T21:52:08.784 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:08.793 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:08.793+0000 7f66f89581c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:52:08.921 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:08.921+0000 7f1e100561c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:52:08.980 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:08.980+0000 7f66f89581c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:52:08.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.987+0000 7ff83d5ff700 1 Processor -- start 2023-12-26T21:52:08.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.988+0000 7ff83d5ff700 1 -- start start 2023-12-26T21:52:08.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.989+0000 7ff83d5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff83815d220 0x7ff83815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.989+0000 7ff83d5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff8380c6b70 0x7ff8380c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.989+0000 7ff83d5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8380caeb0 con 0x7ff83806dd10 2023-12-26T21:52:08.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.989+0000 7ff83d5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83815dbb0 con 0x7ff8380c6b70 2023-12-26T21:52:08.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.989+0000 7ff83d5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380ca2a0 con 0x7ff83815d220 2023-12-26T21:52:08.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.989+0000 7ff8377fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff8380c6b70 0x7ff8380c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.989+0000 7ff836ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff83806dd10 0x7ff8380cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44408/0 (socket says 172.21.15.131:44408) 2023-12-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.989+0000 7ff836ffd700 1 -- 172.21.15.131:0/3152460598 learned_addr learned my addr 172.21.15.131:0/3152460598 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.989+0000 7ff8367fc700 1 --2- 172.21.15.131:0/3152460598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff83815d220 0x7ff83815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.990+0000 7ff8377fe700 1 -- 172.21.15.131:0/3152460598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff83815d220 msgr2=0x7ff83815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.990+0000 7ff8377fe700 1 --2- 172.21.15.131:0/3152460598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff83815d220 0x7ff83815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.990+0000 7ff8377fe700 1 -- 172.21.15.131:0/3152460598 >> v1:172.21.15.131:6789/0 conn(0x7ff83806dd10 legacy=0x7ff8380cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.990+0000 7ff8377fe700 1 -- 172.21.15.131:0/3152460598 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380ca580 con 0x7ff8380c6b70 2023-12-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.990+0000 7ff835ffb700 1 -- 172.21.15.131:0/3152460598 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214996388 0 0) 0x7ff8380caeb0 con 0x7ff83806dd10 2023-12-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.990+0000 7ff8377fe700 1 --2- 172.21.15.131:0/3152460598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8380c6b70 0x7ff8380c6f90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff830009e60 tx=0x7ff830009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3027888a264cca95 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:08.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.990+0000 7ff835ffb700 1 -- 172.21.15.131:0/3152460598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83000e070 con 0x7ff8380c6b70 2023-12-26T21:52:08.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.990+0000 7ff835ffb700 1 -- 172.21.15.131:0/3152460598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83000bb60 con 0x7ff8380c6b70 2023-12-26T21:52:08.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.990+0000 7ff835ffb700 1 -- 172.21.15.131:0/3152460598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830012430 con 0x7ff8380c6b70 2023-12-26T21:52:08.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.991+0000 7ff83d5ff700 1 -- 172.21.15.131:0/3152460598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8380c6b70 msgr2=0x7ff8380c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.991+0000 7ff83d5ff700 1 --2- 172.21.15.131:0/3152460598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8380c6b70 0x7ff8380c6f90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff830009e60 tx=0x7ff830009780 comp rx=0 tx=0).stop 2023-12-26T21:52:08.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.991+0000 7ff83d5ff700 1 -- 172.21.15.131:0/3152460598 shutdown_connections 2023-12-26T21:52:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.991+0000 7ff83d5ff700 1 --2- 172.21.15.131:0/3152460598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8380c6b70 0x7ff8380c6f90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.991+0000 7ff83d5ff700 1 --2- 172.21.15.131:0/3152460598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff83815d220 0x7ff83815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.991+0000 7ff83d5ff700 1 -- 172.21.15.131:0/3152460598 >> 172.21.15.131:0/3152460598 conn(0x7ff8380cc5d0 msgr2=0x7ff8380cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.991+0000 7ff83d5ff700 1 -- 172.21.15.131:0/3152460598 shutdown_connections 2023-12-26T21:52:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.991+0000 7ff83d5ff700 1 -- 172.21.15.131:0/3152460598 wait complete. 2023-12-26T21:52:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.993+0000 7ff83d5ff700 1 Processor -- start 2023-12-26T21:52:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.993+0000 7ff83d5ff700 1 -- start start 2023-12-26T21:52:08.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.994+0000 7ff83d5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff83806dd10 0x7ff8380c0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff83d5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8380c6b70 0x7ff838150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff83d5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff83815d220 0x7ff8381697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff83d5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83815afd0 con 0x7ff83806dd10 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff83d5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8381510f0 con 0x7ff83815d220 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff83d5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff838169ff0 con 0x7ff8380c6b70 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff836ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff83806dd10 0x7ff8380c0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff836ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff83806dd10 0x7ff8380c0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60982/0 (socket says 172.21.15.131:60982) 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff836ffd700 1 -- 172.21.15.131:0/746736926 learned_addr learned my addr 172.21.15.131:0/746736926 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff8367fc700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8380c6b70 0x7ff838150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff8377fe700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff83815d220 0x7ff8381697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff836ffd700 1 -- 172.21.15.131:0/746736926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8380c6b70 msgr2=0x7ff838150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff836ffd700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8380c6b70 0x7ff838150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff836ffd700 1 -- 172.21.15.131:0/746736926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff83815d220 msgr2=0x7ff8381697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff836ffd700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff83815d220 0x7ff8381697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff836ffd700 1 -- 172.21.15.131:0/746736926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff830009a20 con 0x7ff83806dd10 2023-12-26T21:52:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.995+0000 7ff8377fe700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff83815d220 0x7ff8381697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:08.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.996+0000 7ff836ffd700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff83806dd10 0x7ff8380c0bb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c5b0 tx=0x7ff828001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:08.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.996+0000 7ff81effd700 1 -- 172.21.15.131:0/746736926 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828005b30 con 0x7ff83806dd10 2023-12-26T21:52:08.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.996+0000 7ff81effd700 1 -- 172.21.15.131:0/746736926 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828005c90 con 0x7ff83806dd10 2023-12-26T21:52:08.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.996+0000 7ff81effd700 1 -- 172.21.15.131:0/746736926 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828007300 con 0x7ff83806dd10 2023-12-26T21:52:08.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.996+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83816a2d0 con 0x7ff83806dd10 2023-12-26T21:52:08.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.996+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff838169cf0 con 0x7ff83806dd10 2023-12-26T21:52:08.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.997+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83815b2b0 con 0x7ff83806dd10 2023-12-26T21:52:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.999+0000 7ff81effd700 1 -- 172.21.15.131:0/746736926 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7ff82801b3b0 con 0x7ff83806dd10 2023-12-26T21:52:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.999+0000 7ff81effd700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff824067050 0x7ff824069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:08.999+0000 7ff81effd700 1 -- 172.21.15.131:0/746736926 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6544+0+0 (secure 0 0 0) 0x7ff828083f20 con 0x7ff83806dd10 2023-12-26T21:52:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.000+0000 7ff8367fc700 1 -- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff824067050 msgr2=0x7ff824069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.000+0000 7ff8367fc700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff824067050 0x7ff824069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:09.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.003+0000 7ff81effd700 1 -- 172.21.15.131:0/746736926 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff828050d40 con 0x7ff83806dd10 2023-12-26T21:52:09.119 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:09.119+0000 7f1e100561c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:52:09.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.201+0000 7ff8367fc700 1 -- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff824067050 msgr2=0x7ff824069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:09.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.201+0000 7ff8367fc700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff824067050 0x7ff824069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.208+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8380657e0 con 0x7ff83806dd10 2023-12-26T21:52:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.208+0000 7ff81effd700 1 -- 172.21.15.131:0/746736926 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16690 (secure 0 0 0) 0x7ff828017020 con 0x7ff83806dd10 2023-12-26T21:52:09.209 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:09.209 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":69,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:06.245076+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:50:02.860261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-26T21:50:03.665608+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":0}]},{"pgid":"7.15","mappings":[{"from":5,"to":7}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2945752061":"2023-12-27T21:50:25.498209+0000","172.21.15.131:6833/39288":"2023-12-27T21:50:25.498209+0000","172.21.15.131:6832/39288":"2023-12-27T21:50:25.498209+0000","172.21.15.131:0/2402413168":"2023-12-27T21:50:25.498209+0000","172.21.15.131:0/471355450":"2023-12-27T21:50:25.498209+0000","172.21.15.131:0/3935646470":"2023-12-27T21:50:25.498209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff824067050 msgr2=0x7ff824069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff824067050 0x7ff824069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff83806dd10 msgr2=0x7ff8380c0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff83806dd10 0x7ff8380c0bb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c5b0 tx=0x7ff828001070 comp rx=0 tx=0).stop 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 shutdown_connections 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff83815d220 0x7ff8381697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff83806dd10 0x7ff8380c0bb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7ff824067050 0x7ff824069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 --2- 172.21.15.131:0/746736926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8380c6b70 0x7ff838150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 >> 172.21.15.131:0/746736926 conn(0x7ff8380cc5d0 msgr2=0x7ff83806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:09.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 shutdown_connections 2023-12-26T21:52:09.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.211+0000 7ff83d5ff700 1 -- 172.21.15.131:0/746736926 wait complete. 2023-12-26T21:52:09.231 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T21:52:09.326 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:09.325+0000 7f1e100561c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:52:09.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.430+0000 7f95f09ff700 1 Processor -- start 2023-12-26T21:52:09.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.430+0000 7f95f09ff700 1 -- start start 2023-12-26T21:52:09.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.432+0000 7f95f09ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec0b3d40 0x7f95ec0b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.432+0000 7f95f09ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec15d250 con 0x7f95ec0b1a40 2023-12-26T21:52:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.432+0000 7f95f09ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec15d4a0 con 0x7f95ec153bd0 2023-12-26T21:52:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.432+0000 7f95f09ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec15e260 con 0x7f95ec0b3d40 2023-12-26T21:52:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.432+0000 7f95ead9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f95ec0b1a40 0x7f95ec068920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44418/0 (socket says 172.21.15.131:44418) 2023-12-26T21:52:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.432+0000 7f95e3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec0b3d40 0x7f95ec0b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.432+0000 7f95ead9d700 1 -- 172.21.15.131:0/2363540556 learned_addr learned my addr 172.21.15.131:0/2363540556 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.432+0000 7f95ea59c700 1 --1- 172.21.15.131:0/2363540556 >> v1:172.21.15.190:6789/0 conn(0x7f95ec153bd0 0x7f95ec0b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:42376/0 (socket says 172.21.15.131:42376) 2023-12-26T21:52:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e9d9b700 1 -- 172.21.15.131:0/2363540556 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467239277 0 0) 0x7f95ec15d250 con 0x7f95ec0b1a40 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e9d9b700 1 -- 172.21.15.131:0/2363540556 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d8003570 con 0x7f95ec0b1a40 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e9d9b700 1 -- 172.21.15.131:0/2363540556 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646951434 0 0) 0x7f95ec15d4a0 con 0x7f95ec153bd0 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e3fff700 1 -- 172.21.15.131:0/2363540556 >> v1:172.21.15.190:6789/0 conn(0x7f95ec153bd0 legacy=0x7f95ec0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e3fff700 1 -- 172.21.15.131:0/2363540556 >> v1:172.21.15.131:6789/0 conn(0x7f95ec0b1a40 legacy=0x7f95ec068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e3fff700 1 -- 172.21.15.131:0/2363540556 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec15d780 con 0x7f95ec0b3d40 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e3fff700 1 --2- 172.21.15.131:0/2363540556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec0b3d40 0x7f95ec0b1500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009910 tx=0x7f95d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ad01b777e62b09 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e9d9b700 1 -- 172.21.15.131:0/2363540556 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d400b680 con 0x7f95ec0b3d40 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e9d9b700 1 -- 172.21.15.131:0/2363540556 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d400b7e0 con 0x7f95ec0b3d40 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.433+0000 7f95e9d9b700 1 -- 172.21.15.131:0/2363540556 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d400bba0 con 0x7f95ec0b3d40 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.434+0000 7f95f09ff700 1 -- 172.21.15.131:0/2363540556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec0b3d40 msgr2=0x7f95ec0b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.434+0000 7f95f09ff700 1 --2- 172.21.15.131:0/2363540556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec0b3d40 0x7f95ec0b1500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009910 tx=0x7f95d40091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.434+0000 7f95f09ff700 1 -- 172.21.15.131:0/2363540556 shutdown_connections 2023-12-26T21:52:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.434+0000 7f95f09ff700 1 --2- 172.21.15.131:0/2363540556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec0b3d40 0x7f95ec0b1500 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.434+0000 7f95f09ff700 1 -- 172.21.15.131:0/2363540556 >> 172.21.15.131:0/2363540556 conn(0x7f95ec063d00 msgr2=0x7f95ec06b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.434+0000 7f95f09ff700 1 -- 172.21.15.131:0/2363540556 shutdown_connections 2023-12-26T21:52:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.434+0000 7f95f09ff700 1 -- 172.21.15.131:0/2363540556 wait complete. 2023-12-26T21:52:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.436+0000 7f95f09ff700 1 Processor -- start 2023-12-26T21:52:09.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.436+0000 7f95f09ff700 1 -- start start 2023-12-26T21:52:09.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.437+0000 7f95f09ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ec0b1a40 0x7f95ec148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95f09ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95ec0b3d40 0x7f95ec14db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95f09ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec153bd0 0x7f95ec159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95f09ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec14caf0 con 0x7f95ec0b3d40 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95f09ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec14d670 con 0x7f95ec0b1a40 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95f09ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec14cd40 con 0x7f95ec153bd0 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95e3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95ec0b3d40 0x7f95ec14db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95e3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95ec0b3d40 0x7f95ec14db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60984/0 (socket says 172.21.15.131:60984) 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95e3fff700 1 -- 172.21.15.131:0/3873579559 learned_addr learned my addr 172.21.15.131:0/3873579559 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95ead9d700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec153bd0 0x7f95ec159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95ea59c700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ec0b1a40 0x7f95ec148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95e3fff700 1 -- 172.21.15.131:0/3873579559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec153bd0 msgr2=0x7f95ec159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95e3fff700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec153bd0 0x7f95ec159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95e3fff700 1 -- 172.21.15.131:0/3873579559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ec0b1a40 msgr2=0x7f95ec148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95e3fff700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ec0b1a40 0x7f95ec148b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.438+0000 7f95e3fff700 1 -- 172.21.15.131:0/3873579559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d4009420 con 0x7f95ec0b3d40 2023-12-26T21:52:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.439+0000 7f95e3fff700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95ec0b3d40 0x7f95ec14db00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009fb0 tx=0x7f95d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.439+0000 7f95e27fc700 1 -- 172.21.15.131:0/3873579559 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d400bb60 con 0x7f95ec0b3d40 2023-12-26T21:52:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.439+0000 7f95e27fc700 1 -- 172.21.15.131:0/3873579559 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4000c90 con 0x7f95ec0b3d40 2023-12-26T21:52:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.439+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec14d020 con 0x7f95ec0b3d40 2023-12-26T21:52:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.439+0000 7f95e27fc700 1 -- 172.21.15.131:0/3873579559 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4027680 con 0x7f95ec0b3d40 2023-12-26T21:52:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.439+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec14e040 con 0x7f95ec0b3d40 2023-12-26T21:52:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.440+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ec0657e0 con 0x7f95ec0b3d40 2023-12-26T21:52:09.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.442+0000 7f95e27fc700 1 -- 172.21.15.131:0/3873579559 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f95d4011420 con 0x7f95ec0b3d40 2023-12-26T21:52:09.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.443+0000 7f95e27fc700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f95cc067050 0x7f95cc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.443+0000 7f95ea59c700 1 -- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f95cc067050 msgr2=0x7f95cc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:09.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.443+0000 7f95e27fc700 1 -- 172.21.15.131:0/3873579559 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6544+0+0 (secure 0 0 0) 0x7f95d4086a10 con 0x7f95ec0b3d40 2023-12-26T21:52:09.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.443+0000 7f95ea59c700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f95cc067050 0x7f95cc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:09.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.446+0000 7f95e27fc700 1 -- 172.21.15.131:0/3873579559 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95d4053830 con 0x7f95ec0b3d40 2023-12-26T21:52:09.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.644+0000 7f95ea59c700 1 -- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f95cc067050 msgr2=0x7f95cc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6832/39288 2023-12-26T21:52:09.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.644+0000 7f95ea59c700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f95cc067050 0x7f95cc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:09.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.688+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f95ec0684d0 con 0x7f95ec0b3d40 2023-12-26T21:52:09.689 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T21:52:09.688+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T21:52:09.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.693+0000 7f95e27fc700 1 -- 172.21.15.131:0/3873579559 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f95d4056e50 con 0x7f95ec0b3d40 2023-12-26T21:52:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f95cc067050 msgr2=0x7f95cc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f95cc067050 0x7f95cc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95ec0b3d40 msgr2=0x7f95ec14db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95ec0b3d40 0x7f95ec14db00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009fb0 tx=0x7f95d4017000 comp rx=0 tx=0).stop 2023-12-26T21:52:09.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 shutdown_connections 2023-12-26T21:52:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7f95cc067050 0x7f95cc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95ec153bd0 0x7f95ec159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95ec0b1a40 0x7f95ec148b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 --2- 172.21.15.131:0/3873579559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95ec0b3d40 0x7f95ec14db00 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.696+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 >> 172.21.15.131:0/3873579559 conn(0x7f95ec063d00 msgr2=0x7f95ec149f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.697+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 shutdown_connections 2023-12-26T21:52:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.697+0000 7f95f09ff700 1 -- 172.21.15.131:0/3873579559 wait complete. 2023-12-26T21:52:09.698 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T21:52:09.718 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T21:52:09.755 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:52:09.823 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:09.823+0000 7f66f89581c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:52:09.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.920+0000 7fef02fff700 1 Processor -- start 2023-12-26T21:52:09.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.920+0000 7fef02fff700 1 -- start start 2023-12-26T21:52:09.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7fef02fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feefc13c510 0x7feefc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7fef02fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc0cc810 0x7feefc06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7fef02fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc0686c0 con 0x7feefc06b8c0 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7fef02fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc06c280 con 0x7feefc13c510 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7fef02fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc069520 con 0x7feefc0cc810 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7feefbfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7feefc06b8c0 0x7feefc0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44438/0 (socket says 172.21.15.131:44438) 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7feefbfff700 1 -- 172.21.15.131:0/2225329298 learned_addr learned my addr 172.21.15.131:0/2225329298 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7fef0159c700 1 --2- 172.21.15.131:0/2225329298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc0cc810 0x7feefc06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7fef00d9b700 1 --2- 172.21.15.131:0/2225329298 >> v2:172.21.15.190:3300/0 conn(0x7feefc13c510 0x7feefc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.922+0000 7feefb7fe700 1 -- 172.21.15.131:0/2225329298 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276781242 0 0) 0x7feefc0686c0 con 0x7feefc06b8c0 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7feefb7fe700 1 -- 172.21.15.131:0/2225329298 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeec003570 con 0x7feefc06b8c0 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7fef0159c700 1 -- 172.21.15.131:0/2225329298 >> v2:172.21.15.190:3300/0 conn(0x7feefc13c510 msgr2=0x7feefc0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7fef0159c700 1 --2- 172.21.15.131:0/2225329298 >> v2:172.21.15.190:3300/0 conn(0x7feefc13c510 0x7feefc0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7fef0159c700 1 -- 172.21.15.131:0/2225329298 >> v1:172.21.15.131:6789/0 conn(0x7feefc06b8c0 legacy=0x7feefc0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7fef0159c700 1 -- 172.21.15.131:0/2225329298 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc068910 con 0x7feefc0cc810 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7fef00d9b700 1 --2- 172.21.15.131:0/2225329298 >> v2:172.21.15.190:3300/0 conn(0x7feefc13c510 0x7feefc0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:09.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7fef0159c700 1 --2- 172.21.15.131:0/2225329298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc0cc810 0x7feefc06bd40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feef4009e60 tx=0x7feef4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a268512ad3b2e990 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7feefb7fe700 1 -- 172.21.15.131:0/2225329298 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400e070 con 0x7feefc0cc810 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7feefb7fe700 1 -- 172.21.15.131:0/2225329298 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef400bab0 con 0x7feefc0cc810 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.923+0000 7feefb7fe700 1 -- 172.21.15.131:0/2225329298 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400be70 con 0x7feefc0cc810 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.924+0000 7fef02fff700 1 -- 172.21.15.131:0/2225329298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc0cc810 msgr2=0x7feefc06bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.924+0000 7fef02fff700 1 --2- 172.21.15.131:0/2225329298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc0cc810 0x7feefc06bd40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feef4009e60 tx=0x7feef4009780 comp rx=0 tx=0).stop 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.924+0000 7fef02fff700 1 -- 172.21.15.131:0/2225329298 shutdown_connections 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.924+0000 7fef02fff700 1 --2- 172.21.15.131:0/2225329298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc0cc810 0x7feefc06bd40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.924+0000 7fef02fff700 1 --2- 172.21.15.131:0/2225329298 >> v2:172.21.15.190:3300/0 conn(0x7feefc13c510 0x7feefc0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.924+0000 7fef02fff700 1 -- 172.21.15.131:0/2225329298 >> 172.21.15.131:0/2225329298 conn(0x7feefc0ce3e0 msgr2=0x7feefc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:09.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.924+0000 7fef02fff700 1 -- 172.21.15.131:0/2225329298 shutdown_connections 2023-12-26T21:52:09.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.924+0000 7fef02fff700 1 -- 172.21.15.131:0/2225329298 wait complete. 2023-12-26T21:52:09.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.926+0000 7fef02fff700 1 Processor -- start 2023-12-26T21:52:09.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.926+0000 7fef02fff700 1 -- start start 2023-12-26T21:52:09.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef02fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc06b8c0 0x7feefc13bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef02fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feefc0cc810 0x7feefc0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef02fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc13c510 0x7feefc151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef02fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b1ec0 con 0x7feefc0cc810 2023-12-26T21:52:09.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef02fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b12b0 con 0x7feefc06b8c0 2023-12-26T21:52:09.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef02fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b1590 con 0x7feefc13c510 2023-12-26T21:52:09.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7feefbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feefc0cc810 0x7feefc0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef00d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc06b8c0 0x7feefc13bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef0159c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc13c510 0x7feefc151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef00d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc06b8c0 0x7feefc13bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:35080/0 (socket says 172.21.15.131:35080) 2023-12-26T21:52:09.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef00d9b700 1 -- 172.21.15.131:0/3541810191 learned_addr learned my addr 172.21.15.131:0/3541810191 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:09.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef00d9b700 1 -- 172.21.15.131:0/3541810191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc13c510 msgr2=0x7feefc151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef00d9b700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc13c510 0x7feefc151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef00d9b700 1 -- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feefc0cc810 msgr2=0x7feefc0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef00d9b700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feefc0cc810 0x7feefc0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.928+0000 7fef00d9b700 1 -- 172.21.15.131:0/3541810191 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef00093c0 con 0x7feefc06b8c0 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.929+0000 7fef00d9b700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc06b8c0 0x7feefc13bcb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feef000ec80 tx=0x7feef0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.929+0000 7feef8ff9700 1 -- 172.21.15.131:0/3541810191 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0005a00 con 0x7feefc06b8c0 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.929+0000 7feef8ff9700 1 -- 172.21.15.131:0/3541810191 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0005b60 con 0x7feefc06b8c0 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.929+0000 7fef02fff700 1 -- 172.21.15.131:0/3541810191 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef4009a20 con 0x7feefc06b8c0 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.929+0000 7feef8ff9700 1 -- 172.21.15.131:0/3541810191 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef000c850 con 0x7feefc06b8c0 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.929+0000 7fef02fff700 1 -- 172.21.15.131:0/3541810191 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc0b1b50 con 0x7feefc06b8c0 2023-12-26T21:52:09.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.930+0000 7fef02fff700 1 -- 172.21.15.131:0/3541810191 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc0657e0 con 0x7feefc06b8c0 2023-12-26T21:52:09.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.931+0000 7feef8ff9700 1 -- 172.21.15.131:0/3541810191 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 25) v1 ==== 84856+0+0 (secure 0 0 0) 0x7feef000cbc0 con 0x7feefc06b8c0 2023-12-26T21:52:09.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.932+0000 7feef8ff9700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feee0067050 0x7feee0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:09.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.932+0000 7feef8ff9700 1 -- 172.21.15.131:0/3541810191 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6586+0+0 (secure 0 0 0) 0x7feef0084f30 con 0x7feefc06b8c0 2023-12-26T21:52:09.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.932+0000 7feefbfff700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feee0067050 0x7feee0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:09.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.932+0000 7feefbfff700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feee0067050 0x7feee0069500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello connection peer type does not match what peer advertises 16 != 2 2023-12-26T21:52:09.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.932+0000 7feefbfff700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feee0067050 0x7feee0069500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.932+0000 7feef8ff9700 1 -- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feee0067050 msgr2=0x7feee0069500 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-12-26T21:52:09.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.932+0000 7feef8ff9700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feee0067050 0x7feee0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:09.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:09.936+0000 7feef8ff9700 1 -- 172.21.15.131:0/3541810191 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feef0055710 con 0x7feefc06b8c0 2023-12-26T21:52:10.096 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:10.096+0000 7f66f89581c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:10.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.182+0000 7fef02fff700 1 -- 172.21.15.131:0/3541810191 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feefc068bf0 con 0x7feefc06b8c0 2023-12-26T21:52:10.187 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:10.187+0000 7f66f89581c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:10.211 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:10.211+0000 7f1e100561c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:52:10.384 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:10.384+0000 7f66f89581c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:52:10.488 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:10.488+0000 7f1e100561c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:10.584 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:10.583+0000 7f1e100561c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:10.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.695+0000 7feef8ff9700 1 -- 172.21.15.131:0/3541810191 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7feef00552a0 con 0x7feefc06b8c0 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 -- 172.21.15.131:0/3541810191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc06b8c0 msgr2=0x7feefc13bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc06b8c0 0x7feefc13bcb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feef000ec80 tx=0x7feef0009120 comp rx=0 tx=0).stop 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 -- 172.21.15.131:0/3541810191 shutdown_connections 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:6832/39288,v1:172.21.15.131:6833/39288] conn(0x7feee0067050 0x7feee0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feefc13c510 0x7feefc151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc06b8c0 0x7feefc13bcb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 --2- 172.21.15.131:0/3541810191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feefc0cc810 0x7feefc0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 -- 172.21.15.131:0/3541810191 >> 172.21.15.131:0/3541810191 conn(0x7feefc0ce3e0 msgr2=0x7feefc0c4160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 -- 172.21.15.131:0/3541810191 shutdown_connections 2023-12-26T21:52:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.698+0000 7fef02fff700 1 -- 172.21.15.131:0/3541810191 wait complete. 2023-12-26T21:52:10.718 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T21:52:10.785 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:10.784+0000 7f1e100561c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:52:10.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.973+0000 7f4b1dfff700 1 Processor -- start 2023-12-26T21:52:10.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.973+0000 7f4b1dfff700 1 -- start start 2023-12-26T21:52:10.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b1dfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b18142f00 0x7f4b18069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b1dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b1dfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b18069640 con 0x7f4b180cc590 2023-12-26T21:52:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b1dfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b18068750 con 0x7f4b18142f00 2023-12-26T21:52:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b1dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18068a30 con 0x7f4b180cb510 2023-12-26T21:52:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b16ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4b180cc590 0x7f4b180cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44444/0 (socket says 172.21.15.131:44444) 2023-12-26T21:52:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b16ffd700 1 -- 172.21.15.131:0/3898911944 learned_addr learned my addr 172.21.15.131:0/3898911944 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b177fe700 1 --2- 172.21.15.131:0/3898911944 >> v2:172.21.15.190:3300/0 conn(0x7f4b18142f00 0x7f4b18069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:10.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b17fff700 1 --2- 172.21.15.131:0/3898911944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.975+0000 7f4b167fc700 1 -- 172.21.15.131:0/3898911944 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608504748 0 0) 0x7f4b18069640 con 0x7f4b180cc590 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b167fc700 1 -- 172.21.15.131:0/3898911944 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b00003570 con 0x7f4b180cc590 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b177fe700 1 -- 172.21.15.131:0/3898911944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 msgr2=0x7f4b180c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b177fe700 1 --2- 172.21.15.131:0/3898911944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b177fe700 1 -- 172.21.15.131:0/3898911944 >> v1:172.21.15.131:6789/0 conn(0x7f4b180cc590 legacy=0x7f4b180cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b177fe700 1 -- 172.21.15.131:0/3898911944 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18068d10 con 0x7f4b18142f00 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b17fff700 1 --2- 172.21.15.131:0/3898911944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b177fe700 1 --2- 172.21.15.131:0/3898911944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b18142f00 0x7f4b18069ad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4b080098e0 tx=0x7f4b080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0aa0f46b6fcd3fe server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b167fc700 1 -- 172.21.15.131:0/3898911944 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08011630 con 0x7f4b18142f00 2023-12-26T21:52:10.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b167fc700 1 -- 172.21.15.131:0/3898911944 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b08011790 con 0x7f4b18142f00 2023-12-26T21:52:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.976+0000 7f4b167fc700 1 -- 172.21.15.131:0/3898911944 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08011b30 con 0x7f4b18142f00 2023-12-26T21:52:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.977+0000 7f4b1dfff700 1 -- 172.21.15.131:0/3898911944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b18142f00 msgr2=0x7f4b18069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.977+0000 7f4b1dfff700 1 --2- 172.21.15.131:0/3898911944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b18142f00 0x7f4b18069ad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4b080098e0 tx=0x7f4b080091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.977+0000 7f4b1dfff700 1 -- 172.21.15.131:0/3898911944 shutdown_connections 2023-12-26T21:52:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.977+0000 7f4b1dfff700 1 --2- 172.21.15.131:0/3898911944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.977+0000 7f4b1dfff700 1 --2- 172.21.15.131:0/3898911944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b18142f00 0x7f4b18069ad0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.977+0000 7f4b1dfff700 1 -- 172.21.15.131:0/3898911944 >> 172.21.15.131:0/3898911944 conn(0x7f4b18063cd0 msgr2=0x7f4b1806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.977+0000 7f4b1dfff700 1 -- 172.21.15.131:0/3898911944 shutdown_connections 2023-12-26T21:52:10.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.977+0000 7f4b1dfff700 1 -- 172.21.15.131:0/3898911944 wait complete. 2023-12-26T21:52:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.978+0000 7f4b1dfff700 1 Processor -- start 2023-12-26T21:52:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.978+0000 7f4b1dfff700 1 -- start start 2023-12-26T21:52:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.979+0000 7f4b1dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.979+0000 7f4b1dfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180cc590 0x7f4b1813fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.979+0000 7f4b1dfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b18142f00 0x7f4b1815f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.979+0000 7f4b1dfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1815fff0 con 0x7f4b18142f00 2023-12-26T21:52:10.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.979+0000 7f4b1dfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1815e490 con 0x7f4b180cc590 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.979+0000 7f4b1dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b180b1b70 con 0x7f4b180cb510 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b18142f00 0x7f4b1815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b177fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b18142f00 0x7f4b1815f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32840/0 (socket says 172.21.15.131:32840) 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b17fff700 1 -- 172.21.15.131:0/788010745 learned_addr learned my addr 172.21.15.131:0/788010745 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b16ffd700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180cc590 0x7f4b1813fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b17fff700 1 -- 172.21.15.131:0/788010745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 msgr2=0x7f4b180c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b17fff700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b17fff700 1 -- 172.21.15.131:0/788010745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180cc590 msgr2=0x7f4b1813fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b17fff700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180cc590 0x7f4b1813fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b17fff700 1 -- 172.21.15.131:0/788010745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10009a20 con 0x7f4b18142f00 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b177fe700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4b17fff700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b18142f00 0x7f4b1815f030 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000ef90 tx=0x7f4b1000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.980+0000 7f4aff7fe700 1 -- 172.21.15.131:0/788010745 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10009d00 con 0x7f4b18142f00 2023-12-26T21:52:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.981+0000 7f4aff7fe700 1 -- 172.21.15.131:0/788010745 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1000f040 con 0x7f4b18142f00 2023-12-26T21:52:10.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.981+0000 7f4aff7fe700 1 -- 172.21.15.131:0/788010745 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1000ce20 con 0x7f4b18142f00 2023-12-26T21:52:10.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.981+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b08009420 con 0x7f4b18142f00 2023-12-26T21:52:10.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.981+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b180b1520 con 0x7f4b18142f00 2023-12-26T21:52:10.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.982+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b180c6530 con 0x7f4b18142f00 2023-12-26T21:52:10.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.983+0000 7f4aff7fe700 1 -- 172.21.15.131:0/788010745 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) v1 ==== 84453+0+0 (secure 0 0 0) 0x7f4b10014690 con 0x7f4b18142f00 2023-12-26T21:52:10.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.983+0000 7f4aff7fe700 1 -- 172.21.15.131:0/788010745 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6844+0+0 (secure 0 0 0) 0x7f4b10085200 con 0x7f4b18142f00 2023-12-26T21:52:10.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:10.988+0000 7f4aff7fe700 1 -- 172.21.15.131:0/788010745 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b100522d0 con 0x7f4b18142f00 2023-12-26T21:52:11.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.178+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4b180717f0 con 0x7f4b18142f00 2023-12-26T21:52:11.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.800+0000 7f4aff7fe700 1 -- 172.21.15.131:0/788010745 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 27) v1 ==== 84580+0+0 (secure 0 0 0) 0x7f4b10051d80 con 0x7f4b18142f00 2023-12-26T21:52:11.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.800+0000 7f4aff7fe700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4b0406cbd0 0x7f4b0406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:11.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.803+0000 7f4b177fe700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4b0406cbd0 0x7f4b0406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:11.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.807+0000 7f4b177fe700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4b0406cbd0 0x7f4b0406efb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4b080091e0 tx=0x7f4b08017000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:11.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.810+0000 7f4aff7fe700 1 -- 172.21.15.131:0/788010745 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4b1006fd30 con 0x7f4b18142f00 2023-12-26T21:52:11.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.816+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4b0406cbd0 msgr2=0x7f4b0406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:11.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.816+0000 7f4b1dfff700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4b0406cbd0 0x7f4b0406efb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4b080091e0 tx=0x7f4b08017000 comp rx=0 tx=0).stop 2023-12-26T21:52:11.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.816+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b18142f00 msgr2=0x7f4b1815f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:11.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.816+0000 7f4b1dfff700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b18142f00 0x7f4b1815f030 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000ef90 tx=0x7f4b1000b040 comp rx=0 tx=0).stop 2023-12-26T21:52:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.817+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 shutdown_connections 2023-12-26T21:52:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.817+0000 7f4b1dfff700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b18142f00 0x7f4b1815f030 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.817+0000 7f4b1dfff700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b180cb510 0x7f4b180c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.817+0000 7f4b1dfff700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4b0406cbd0 0x7f4b0406efb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.817+0000 7f4b1dfff700 1 --2- 172.21.15.131:0/788010745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b180cc590 0x7f4b1813fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.817+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 >> 172.21.15.131:0/788010745 conn(0x7f4b18063cd0 msgr2=0x7f4b180c4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.817+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 shutdown_connections 2023-12-26T21:52:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:11.817+0000 7f4b1dfff700 1 -- 172.21.15.131:0/788010745 wait complete. 2023-12-26T21:52:11.819 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T21:52:11.838 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T21:52:12.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.073+0000 7f2ce73ff700 1 Processor -- start 2023-12-26T21:52:12.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.073+0000 7f2ce73ff700 1 -- start start 2023-12-26T21:52:12.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.074+0000 7f2ce73ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2ce00adcb0 0x7f2ce00abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:12.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.074+0000 7f2ce73ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00ae080 0x7f2ce0069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:12.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.074+0000 7f2ce73ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce00c7430 con 0x7f2ce0069a60 2023-12-26T21:52:12.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.074+0000 7f2ce73ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce00ac460 con 0x7f2ce00adcb0 2023-12-26T21:52:12.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.074+0000 7f2ce73ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0069ee0 con 0x7f2ce00ae080 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce499a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00ae080 0x7f2ce0069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce519b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2ce00adcb0 0x7f2ce00abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce599c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f2ce0069a60 0x7f2ce006df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:42388/0 (socket says 172.21.15.131:42388) 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce499a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00ae080 0x7f2ce0069520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:60046/0 (socket says 172.21.15.131:60046) 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce599c700 1 -- 172.21.15.131:0/820047092 learned_addr learned my addr 172.21.15.131:0/820047092 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce519b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2ce00adcb0 0x7f2ce00abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:35220/0 (socket says 172.21.15.131:35220) 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce499a700 1 -- 172.21.15.131:0/820047092 >> v2:172.21.15.190:3300/0 conn(0x7f2ce00adcb0 msgr2=0x7f2ce00abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce499a700 1 --2- 172.21.15.131:0/820047092 >> v2:172.21.15.190:3300/0 conn(0x7f2ce00adcb0 0x7f2ce00abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce499a700 1 -- 172.21.15.131:0/820047092 >> v1:172.21.15.190:6789/0 conn(0x7f2ce0069a60 legacy=0x7f2ce006df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:12.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce499a700 1 -- 172.21.15.131:0/820047092 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce00c6820 con 0x7f2ce00ae080 2023-12-26T21:52:12.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.075+0000 7f2ce519b700 1 --2- 172.21.15.131:0/820047092 >> v2:172.21.15.190:3300/0 conn(0x7f2ce00adcb0 0x7f2ce00abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:12.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.076+0000 7f2ce499a700 1 --2- 172.21.15.131:0/820047092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00ae080 0x7f2ce0069520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000aa90 tx=0x7f2cd000a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee0259480062b56 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.076+0000 7f2cd7fff700 1 -- 172.21.15.131:0/820047092 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000c970 con 0x7f2ce00ae080 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.076+0000 7f2cd7fff700 1 -- 172.21.15.131:0/820047092 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd000cad0 con 0x7f2ce00ae080 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.076+0000 7f2cd7fff700 1 -- 172.21.15.131:0/820047092 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000ce70 con 0x7f2ce00ae080 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.077+0000 7f2ce73ff700 1 -- 172.21.15.131:0/820047092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00ae080 msgr2=0x7f2ce0069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.077+0000 7f2ce73ff700 1 --2- 172.21.15.131:0/820047092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00ae080 0x7f2ce0069520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000aa90 tx=0x7f2cd000a330 comp rx=0 tx=0).stop 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.077+0000 7f2ce73ff700 1 -- 172.21.15.131:0/820047092 shutdown_connections 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.077+0000 7f2ce73ff700 1 --2- 172.21.15.131:0/820047092 >> v2:172.21.15.190:3300/0 conn(0x7f2ce00adcb0 0x7f2ce00abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.077+0000 7f2ce73ff700 1 --2- 172.21.15.131:0/820047092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00ae080 0x7f2ce0069520 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.077+0000 7f2ce73ff700 1 -- 172.21.15.131:0/820047092 >> 172.21.15.131:0/820047092 conn(0x7f2ce00675b0 msgr2=0x7f2ce00679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.077+0000 7f2ce73ff700 1 -- 172.21.15.131:0/820047092 shutdown_connections 2023-12-26T21:52:12.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.077+0000 7f2ce73ff700 1 -- 172.21.15.131:0/820047092 wait complete. 2023-12-26T21:52:12.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.078+0000 7f2ce73ff700 1 Processor -- start 2023-12-26T21:52:12.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.078+0000 7f2ce73ff700 1 -- start start 2023-12-26T21:52:12.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.079+0000 7f2ce73ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce0069a60 0x7f2ce0151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.079+0000 7f2ce73ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ce00adcb0 0x7f2ce00c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.079+0000 7f2ce73ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce00ae080 0x7f2ce00ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.079+0000 7f2ce73ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0169c40 con 0x7f2ce00ae080 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.079+0000 7f2ce73ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce006e2a0 con 0x7f2ce00adcb0 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.079+0000 7f2ce73ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00cab60 con 0x7f2ce0069a60 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce00ae080 0x7f2ce00ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce519b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce0069a60 0x7f2ce0151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce519b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce0069a60 0x7f2ce0151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:60048/0 (socket says 172.21.15.131:60048) 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce519b700 1 -- 172.21.15.131:0/2980134884 learned_addr learned my addr 172.21.15.131:0/2980134884 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce499a700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ce00adcb0 0x7f2ce00c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce519b700 1 -- 172.21.15.131:0/2980134884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ce00adcb0 msgr2=0x7f2ce00c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce519b700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ce00adcb0 0x7f2ce00c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce519b700 1 -- 172.21.15.131:0/2980134884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce00ae080 msgr2=0x7f2ce00ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:12.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce519b700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce00ae080 0x7f2ce00ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce519b700 1 -- 172.21.15.131:0/2980134884 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc009420 con 0x7f2ce0069a60 2023-12-26T21:52:12.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.080+0000 7f2ce519b700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce0069a60 0x7f2ce0151870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00ec80 tx=0x7f2cdc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:12.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.081+0000 7f2cd57fa700 1 -- 172.21.15.131:0/2980134884 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc00c920 con 0x7f2ce0069a60 2023-12-26T21:52:12.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.081+0000 7f2cd57fa700 1 -- 172.21.15.131:0/2980134884 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc00ca80 con 0x7f2ce0069a60 2023-12-26T21:52:12.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.081+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd000a5d0 con 0x7f2ce0069a60 2023-12-26T21:52:12.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.081+0000 7f2cd57fa700 1 -- 172.21.15.131:0/2980134884 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc00ce00 con 0x7f2ce0069a60 2023-12-26T21:52:12.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.081+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce0169e30 con 0x7f2ce0069a60 2023-12-26T21:52:12.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.082+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce00717f0 con 0x7f2ce0069a60 2023-12-26T21:52:12.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.083+0000 7f2cd57fa700 1 -- 172.21.15.131:0/2980134884 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 27) v1 ==== 84580+0+0 (secure 0 0 0) 0x7f2cdc013680 con 0x7f2ce0069a60 2023-12-26T21:52:12.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.083+0000 7f2cd57fa700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2cc4066e50 0x7f2cc4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:12.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.083+0000 7f2cd57fa700 1 -- 172.21.15.131:0/2980134884 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6406+0+0 (secure 0 0 0) 0x7f2cdc084b90 con 0x7f2ce0069a60 2023-12-26T21:52:12.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.084+0000 7f2ce499a700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2cc4066e50 0x7f2cc4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:12.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.084+0000 7f2ce499a700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2cc4066e50 0x7f2cc4069300 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00ae4d0 tx=0x7f2cd001a000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:12.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.088+0000 7f2cd57fa700 1 -- 172.21.15.131:0/2980134884 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cdc051da0 con 0x7f2ce0069a60 2023-12-26T21:52:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.283+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ce00c6b00 con 0x7f2ce0069a60 2023-12-26T21:52:12.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.819+0000 7f2cd57fa700 1 -- 172.21.15.131:0/2980134884 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2cdc0170d0 con 0x7f2ce0069a60 2023-12-26T21:52:12.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.825+0000 7f2cd57fa700 1 -- 172.21.15.131:0/2980134884 <== mon.2 v2:172.21.15.190:3301/0 8 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2cdc052170 con 0x7f2ce0069a60 2023-12-26T21:52:12.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.826+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2cc4066e50 msgr2=0x7f2cc4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:12.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.826+0000 7f2ce73ff700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2cc4066e50 0x7f2cc4069300 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00ae4d0 tx=0x7f2cd001a000 comp rx=0 tx=0).stop 2023-12-26T21:52:12.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.826+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce0069a60 msgr2=0x7f2ce0151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:12.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.826+0000 7f2ce73ff700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce0069a60 0x7f2ce0151870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00ec80 tx=0x7f2cdc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.827+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 shutdown_connections 2023-12-26T21:52:12.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.827+0000 7f2ce73ff700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce00ae080 0x7f2ce00ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.827+0000 7f2ce73ff700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce0069a60 0x7f2ce0151870 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00ec80 tx=0x7f2cdc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.827+0000 7f2ce73ff700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2cc4066e50 0x7f2cc4069300 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.827+0000 7f2ce73ff700 1 --2- 172.21.15.131:0/2980134884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ce00adcb0 0x7f2ce00c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:12.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.827+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 >> 172.21.15.131:0/2980134884 conn(0x7f2ce00675b0 msgr2=0x7f2ce00bff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:12.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.827+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 shutdown_connections 2023-12-26T21:52:12.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:12.828+0000 7f2ce73ff700 1 -- 172.21.15.131:0/2980134884 wait complete. 2023-12-26T21:52:12.830 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T21:52:12.847 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T21:52:13.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.037+0000 7f7d8e9ff700 1 Processor -- start 2023-12-26T21:52:13.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.037+0000 7f7d8e9ff700 1 -- start start 2023-12-26T21:52:13.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.038+0000 7f7d8e9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d8813c510 0x7f7d880c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.038+0000 7f7d8e9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d880686c0 con 0x7f7d880cc810 2023-12-26T21:52:13.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.038+0000 7f7d8e9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d88069520 con 0x7f7d8806b8c0 2023-12-26T21:52:13.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.038+0000 7f7d8e9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d88068910 con 0x7f7d8813c510 2023-12-26T21:52:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.039+0000 7f7d87fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d8813c510 0x7f7d880c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.039+0000 7f7d8cf9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7d880cc810 0x7f7d880699b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44456/0 (socket says 172.21.15.131:44456) 2023-12-26T21:52:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.039+0000 7f7d8cf9c700 1 -- 172.21.15.131:0/671269651 learned_addr learned my addr 172.21.15.131:0/671269651 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.039+0000 7f7d86ffd700 1 -- 172.21.15.131:0/671269651 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543952322 0 0) 0x7f7d880686c0 con 0x7f7d880cc810 2023-12-26T21:52:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.039+0000 7f7d86ffd700 1 -- 172.21.15.131:0/671269651 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d70003570 con 0x7f7d880cc810 2023-12-26T21:52:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.039+0000 7f7d87fff700 1 -- 172.21.15.131:0/671269651 >> v1:172.21.15.190:6789/0 conn(0x7f7d8806b8c0 legacy=0x7f7d880cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.039+0000 7f7d87fff700 1 -- 172.21.15.131:0/671269651 >> v1:172.21.15.131:6789/0 conn(0x7f7d880cc810 legacy=0x7f7d880699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.039+0000 7f7d87fff700 1 -- 172.21.15.131:0/671269651 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d88068bf0 con 0x7f7d8813c510 2023-12-26T21:52:13.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.039+0000 7f7d87fff700 1 --2- 172.21.15.131:0/671269651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 0x7f7d880c8a20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7d780098d0 tx=0x7f7d78009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7912d8a0244e8455 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:13.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d86ffd700 1 -- 172.21.15.131:0/671269651 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78005c10 con 0x7f7d8813c510 2023-12-26T21:52:13.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d86ffd700 1 -- 172.21.15.131:0/671269651 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d78005d70 con 0x7f7d8813c510 2023-12-26T21:52:13.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d86ffd700 1 -- 172.21.15.131:0/671269651 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78006280 con 0x7f7d8813c510 2023-12-26T21:52:13.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/671269651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 msgr2=0x7f7d880c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d8e9ff700 1 --2- 172.21.15.131:0/671269651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 0x7f7d880c8a20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7d780098d0 tx=0x7f7d78009120 comp rx=0 tx=0).stop 2023-12-26T21:52:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/671269651 shutdown_connections 2023-12-26T21:52:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d8e9ff700 1 --2- 172.21.15.131:0/671269651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 0x7f7d880c8a20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/671269651 >> 172.21.15.131:0/671269651 conn(0x7f7d880ce3e0 msgr2=0x7f7d880ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/671269651 shutdown_connections 2023-12-26T21:52:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.040+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/671269651 wait complete. 2023-12-26T21:52:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.042+0000 7f7d8e9ff700 1 Processor -- start 2023-12-26T21:52:13.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.042+0000 7f7d8e9ff700 1 -- start start 2023-12-26T21:52:13.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8e9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d8806b8c0 0x7f7d8816add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8e9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d880cc810 0x7f7d880b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8e9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 0x7f7d88154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8e9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d88157250 con 0x7f7d8813c510 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8e9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8816b310 con 0x7f7d880cc810 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8e9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d88156f70 con 0x7f7d8806b8c0 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8cf9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 0x7f7d88154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8cf9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 0x7f7d88154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32862/0 (socket says 172.21.15.131:32862) 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8cf9c700 1 -- 172.21.15.131:0/3033088443 learned_addr learned my addr 172.21.15.131:0/3033088443 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d87fff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d8806b8c0 0x7f7d8816add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d877fe700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d880cc810 0x7f7d880b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8cf9c700 1 -- 172.21.15.131:0/3033088443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d8806b8c0 msgr2=0x7f7d8816add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8cf9c700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d8806b8c0 0x7f7d8816add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8cf9c700 1 -- 172.21.15.131:0/3033088443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d880cc810 msgr2=0x7f7d880b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.043+0000 7f7d8cf9c700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d880cc810 0x7f7d880b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.044+0000 7f7d8cf9c700 1 -- 172.21.15.131:0/3033088443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780093c0 con 0x7f7d8813c510 2023-12-26T21:52:13.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.044+0000 7f7d877fe700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d880cc810 0x7f7d880b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.044+0000 7f7d8cf9c700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 0x7f7d88154360 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7d80001460 tx=0x7f7d80001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.044+0000 7f7d6ffff700 1 -- 172.21.15.131:0/3033088443 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000a8e0 con 0x7f7d8813c510 2023-12-26T21:52:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.044+0000 7f7d6ffff700 1 -- 172.21.15.131:0/3033088443 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000aa40 con 0x7f7d8813c510 2023-12-26T21:52:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.044+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d880c6600 con 0x7f7d8813c510 2023-12-26T21:52:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.044+0000 7f7d6ffff700 1 -- 172.21.15.131:0/3033088443 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000adc0 con 0x7f7d8813c510 2023-12-26T21:52:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.044+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d880c6c40 con 0x7f7d8813c510 2023-12-26T21:52:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.045+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d880c6460 con 0x7f7d8813c510 2023-12-26T21:52:13.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.046+0000 7f7d6ffff700 1 -- 172.21.15.131:0/3033088443 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7d80007300 con 0x7f7d8813c510 2023-12-26T21:52:13.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.047+0000 7f7d6ffff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7d74066f30 0x7f7d740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.047+0000 7f7d6ffff700 1 -- 172.21.15.131:0/3033088443 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f7d800849f0 con 0x7f7d8813c510 2023-12-26T21:52:13.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.047+0000 7f7d87fff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7d74066f30 0x7f7d740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.048+0000 7f7d87fff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7d74066f30 0x7f7d740693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009390 tx=0x7f7d78016000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:13.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.051+0000 7f7d6ffff700 1 -- 172.21.15.131:0/3033088443 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d880c6460 con 0x7f7d8813c510 2023-12-26T21:52:13.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.247+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7d880c6460 con 0x7f7d8813c510 2023-12-26T21:52:13.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.252+0000 7f7d6ffff700 1 -- 172.21.15.131:0/3033088443 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7d880c6460 con 0x7f7d8813c510 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7d74066f30 msgr2=0x7f7d740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7d74066f30 0x7f7d740693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009390 tx=0x7f7d78016000 comp rx=0 tx=0).stop 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 msgr2=0x7f7d88154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 0x7f7d88154360 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7d80001460 tx=0x7f7d80001ae0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 shutdown_connections 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d8813c510 0x7f7d88154360 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d8806b8c0 0x7f7d8816add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7d74066f30 0x7f7d740693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 --2- 172.21.15.131:0/3033088443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d880cc810 0x7f7d880b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 >> 172.21.15.131:0/3033088443 conn(0x7f7d880ce3e0 msgr2=0x7f7d88152610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:13.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 shutdown_connections 2023-12-26T21:52:13.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.254+0000 7f7d8e9ff700 1 -- 172.21.15.131:0/3033088443 wait complete. 2023-12-26T21:52:13.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T21:52:13.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.462+0000 7f63c1dff700 1 Processor -- start 2023-12-26T21:52:13.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.462+0000 7f63c1dff700 1 -- start start 2023-12-26T21:52:13.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.463+0000 7f63c1dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc159a70 0x7f63bc0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.463+0000 7f63c1dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f63bc0b2a90 0x7f63bc0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.463+0000 7f63c1dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63bc0afcd0 con 0x7f63bc1596a0 2023-12-26T21:52:13.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.463+0000 7f63c1dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63bc0b3e70 con 0x7f63bc0b2a90 2023-12-26T21:52:13.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.463+0000 7f63c1dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0b0f20 con 0x7f63bc159a70 2023-12-26T21:52:13.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63baffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc159a70 0x7f63bc0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63bbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f63bc0b2a90 0x7f63bc0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63bbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f63bc0b2a90 0x7f63bc0b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:35248/0 (socket says 172.21.15.131:35248) 2023-12-26T21:52:13.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63baffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc159a70 0x7f63bc0b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:60066/0 (socket says 172.21.15.131:60066) 2023-12-26T21:52:13.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63bbfff700 1 -- 172.21.15.131:0/3096211325 learned_addr learned my addr 172.21.15.131:0/3096211325 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:13.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63bb7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f63bc1596a0 0x7f63bc0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:42408/0 (socket says 172.21.15.131:42408) 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63bbfff700 1 -- 172.21.15.131:0/3096211325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc159a70 msgr2=0x7f63bc0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63bbfff700 1 --2- 172.21.15.131:0/3096211325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc159a70 0x7f63bc0b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63bbfff700 1 -- 172.21.15.131:0/3096211325 >> v1:172.21.15.190:6789/0 conn(0x7f63bc1596a0 legacy=0x7f63bc0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63bbfff700 1 -- 172.21.15.131:0/3096211325 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0b3540 con 0x7f63bc0b2a90 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63ba7fc700 1 -- 172.21.15.131:0/3096211325 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312323999 0 0) 0x7f63bc0afcd0 con 0x7f63bc1596a0 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.464+0000 7f63bbfff700 1 --2- 172.21.15.131:0/3096211325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0b2a90 0x7f63bc0b06a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009f20 tx=0x7f63b4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feb9f660cad8b276 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63ba7fc700 1 -- 172.21.15.131:0/3096211325 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b400e070 con 0x7f63bc0b2a90 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63ba7fc700 1 -- 172.21.15.131:0/3096211325 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b400b9d0 con 0x7f63bc0b2a90 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63ba7fc700 1 -- 172.21.15.131:0/3096211325 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b40092e0 con 0x7f63bc0b2a90 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63c1dff700 1 -- 172.21.15.131:0/3096211325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0b2a90 msgr2=0x7f63bc0b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63c1dff700 1 --2- 172.21.15.131:0/3096211325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0b2a90 0x7f63bc0b06a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009f20 tx=0x7f63b4009840 comp rx=0 tx=0).stop 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63c1dff700 1 -- 172.21.15.131:0/3096211325 shutdown_connections 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63c1dff700 1 --2- 172.21.15.131:0/3096211325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0b2a90 0x7f63bc0b06a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63c1dff700 1 --2- 172.21.15.131:0/3096211325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc159a70 0x7f63bc0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63c1dff700 1 -- 172.21.15.131:0/3096211325 >> 172.21.15.131:0/3096211325 conn(0x7f63bc06ece0 msgr2=0x7f63bc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:13.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63c1dff700 1 -- 172.21.15.131:0/3096211325 shutdown_connections 2023-12-26T21:52:13.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.465+0000 7f63c1dff700 1 -- 172.21.15.131:0/3096211325 wait complete. 2023-12-26T21:52:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.467+0000 7f63c1dff700 1 Processor -- start 2023-12-26T21:52:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.467+0000 7f63c1dff700 1 -- start start 2023-12-26T21:52:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.468+0000 7f63c1dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b2a90 0x7f63bc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.468+0000 7f63c1dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1596a0 0x7f63bc169d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.468+0000 7f63c1dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63bc159a70 0x7f63bc168bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.468+0000 7f63c1dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0cc630 con 0x7f63bc159a70 2023-12-26T21:52:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.468+0000 7f63c1dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0cba20 con 0x7f63bc1596a0 2023-12-26T21:52:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.468+0000 7f63c1dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0cbd00 con 0x7f63bc0b2a90 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.468+0000 7f63bbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63bc159a70 0x7f63bc168bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.468+0000 7f63bbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63bc159a70 0x7f63bc168bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32874/0 (socket says 172.21.15.131:32874) 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.468+0000 7f63bbfff700 1 -- 172.21.15.131:0/2645831350 learned_addr learned my addr 172.21.15.131:0/2645831350 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63bb7fe700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b2a90 0x7f63bc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63baffd700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1596a0 0x7f63bc169d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63bbfff700 1 -- 172.21.15.131:0/2645831350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b2a90 msgr2=0x7f63bc0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63bbfff700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b2a90 0x7f63bc0ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63bbfff700 1 -- 172.21.15.131:0/2645831350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1596a0 msgr2=0x7f63bc169d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63bbfff700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1596a0 0x7f63bc169d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63bbfff700 1 -- 172.21.15.131:0/2645831350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4009ae0 con 0x7f63bc159a70 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63baffd700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1596a0 0x7f63bc169d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63bbfff700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63bc159a70 0x7f63bc168bc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f63bc089710 tx=0x7f63b4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63a37fe700 1 -- 172.21.15.131:0/2645831350 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4008c70 con 0x7f63bc159a70 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63a37fe700 1 -- 172.21.15.131:0/2645831350 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b4008df0 con 0x7f63bc159a70 2023-12-26T21:52:13.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc0cbfe0 con 0x7f63bc159a70 2023-12-26T21:52:13.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63a37fe700 1 -- 172.21.15.131:0/2645831350 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b40266b0 con 0x7f63bc159a70 2023-12-26T21:52:13.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.469+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc16a290 con 0x7f63bc159a70 2023-12-26T21:52:13.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.470+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc0cc7e0 con 0x7f63bc159a70 2023-12-26T21:52:13.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.471+0000 7f63a37fe700 1 -- 172.21.15.131:0/2645831350 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f63b4003040 con 0x7f63bc159a70 2023-12-26T21:52:13.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.473+0000 7f63a37fe700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f63a8066f30 0x7f63a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.473+0000 7f63a37fe700 1 -- 172.21.15.131:0/2645831350 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f63b4086a50 con 0x7f63bc159a70 2023-12-26T21:52:13.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.473+0000 7f63bb7fe700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f63a8066f30 0x7f63a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.474+0000 7f63bb7fe700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f63a8066f30 0x7f63a80693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0b48f0 tx=0x7f63ac007bf0 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:13.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.477+0000 7f63a37fe700 1 -- 172.21.15.131:0/2645831350 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63bc0cc7e0 con 0x7f63bc159a70 2023-12-26T21:52:13.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.667+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f63bc0cdb10 con 0x7f63bc159a70 2023-12-26T21:52:13.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.673+0000 7f63a37fe700 1 -- 172.21.15.131:0/2645831350 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f63b4053b90 con 0x7f63bc159a70 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f63a8066f30 msgr2=0x7f63a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f63a8066f30 0x7f63a80693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0b48f0 tx=0x7f63ac007bf0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63bc159a70 msgr2=0x7f63bc168bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63bc159a70 0x7f63bc168bc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f63bc089710 tx=0x7f63b4018000 comp rx=0 tx=0).stop 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 shutdown_connections 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63bc159a70 0x7f63bc168bc0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b2a90 0x7f63bc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f63a8066f30 0x7f63a80693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 --2- 172.21.15.131:0/2645831350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc1596a0 0x7f63bc169d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 >> 172.21.15.131:0/2645831350 conn(0x7f63bc06ece0 msgr2=0x7f63bc158160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:13.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 shutdown_connections 2023-12-26T21:52:13.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.676+0000 7f63c1dff700 1 -- 172.21.15.131:0/2645831350 wait complete. 2023-12-26T21:52:13.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T21:52:13.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.887+0000 7f00581ff700 1 Processor -- start 2023-12-26T21:52:13.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.888+0000 7f00581ff700 1 -- start start 2023-12-26T21:52:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.889+0000 7f00581ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f005013c510 0x7f00500c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.889+0000 7f00581ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00500cc810 0x7f005006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.889+0000 7f00581ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00500686c0 con 0x7f005006b8c0 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.889+0000 7f00581ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005006c280 con 0x7f005013c510 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.889+0000 7f00581ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0050069520 con 0x7f00500cc810 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.889+0000 7f0055f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f005013c510 0x7f00500c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.889+0000 7f0055f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f005013c510 0x7f00500c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32876/0 (socket says 172.21.15.131:32876) 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.889+0000 7f0055f9b700 1 -- 172.21.15.131:0/3054100329 learned_addr learned my addr 172.21.15.131:0/3054100329 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f005679c700 1 --2- 172.21.15.131:0/3054100329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00500cc810 0x7f005006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f0055f9b700 1 -- 172.21.15.131:0/3054100329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00500cc810 msgr2=0x7f005006bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f0055f9b700 1 --2- 172.21.15.131:0/3054100329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00500cc810 0x7f005006bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f0055f9b700 1 -- 172.21.15.131:0/3054100329 >> v1:172.21.15.190:6789/0 conn(0x7f005006b8c0 legacy=0x7f00500cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f0055f9b700 1 -- 172.21.15.131:0/3054100329 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0050068910 con 0x7f005013c510 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f005679c700 1 --2- 172.21.15.131:0/3054100329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00500cc810 0x7f005006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f0055f9b700 1 --2- 172.21.15.131:0/3054100329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f005013c510 0x7f00500c8a20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f004c009990 tx=0x7f004c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d2f63efb02cfc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f0054f99700 1 -- 172.21.15.131:0/3054100329 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c005c10 con 0x7f005013c510 2023-12-26T21:52:13.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f0054f99700 1 -- 172.21.15.131:0/3054100329 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c005d70 con 0x7f005013c510 2023-12-26T21:52:13.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.890+0000 7f0054f99700 1 -- 172.21.15.131:0/3054100329 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00b880 con 0x7f005013c510 2023-12-26T21:52:13.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.891+0000 7f00581ff700 1 -- 172.21.15.131:0/3054100329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f005013c510 msgr2=0x7f00500c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.891+0000 7f00581ff700 1 --2- 172.21.15.131:0/3054100329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f005013c510 0x7f00500c8a20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f004c009990 tx=0x7f004c009220 comp rx=0 tx=0).stop 2023-12-26T21:52:13.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.891+0000 7f00581ff700 1 -- 172.21.15.131:0/3054100329 shutdown_connections 2023-12-26T21:52:13.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.891+0000 7f00581ff700 1 --2- 172.21.15.131:0/3054100329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00500cc810 0x7f005006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.891+0000 7f00581ff700 1 --2- 172.21.15.131:0/3054100329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f005013c510 0x7f00500c8a20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.891+0000 7f00581ff700 1 -- 172.21.15.131:0/3054100329 >> 172.21.15.131:0/3054100329 conn(0x7f00500ce3e0 msgr2=0x7f00500ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:13.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.891+0000 7f00581ff700 1 -- 172.21.15.131:0/3054100329 shutdown_connections 2023-12-26T21:52:13.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.891+0000 7f00581ff700 1 -- 172.21.15.131:0/3054100329 wait complete. 2023-12-26T21:52:13.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.892+0000 7f00581ff700 1 Processor -- start 2023-12-26T21:52:13.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.893+0000 7f00581ff700 1 -- start start 2023-12-26T21:52:13.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.893+0000 7f00581ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f005006b8c0 0x7f0050154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.893+0000 7f00581ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00500cc810 0x7f0050156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f00581ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f005013c510 0x7f0050156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f00581ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0050154690 con 0x7f00500cc810 2023-12-26T21:52:13.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f00581ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0050150c50 con 0x7f005013c510 2023-12-26T21:52:13.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f00581ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0050156fb0 con 0x7f005006b8c0 2023-12-26T21:52:13.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005579a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00500cc810 0x7f0050156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005579a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00500cc810 0x7f0050156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32892/0 (socket says 172.21.15.131:32892) 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005579a700 1 -- 172.21.15.131:0/1715925171 learned_addr learned my addr 172.21.15.131:0/1715925171 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f0055f9b700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f005006b8c0 0x7f0050154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005679c700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f005013c510 0x7f0050156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005579a700 1 -- 172.21.15.131:0/1715925171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f005006b8c0 msgr2=0x7f0050154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005579a700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f005006b8c0 0x7f0050154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005579a700 1 -- 172.21.15.131:0/1715925171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f005013c510 msgr2=0x7f0050156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005579a700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f005013c510 0x7f0050156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005579a700 1 -- 172.21.15.131:0/1715925171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c009480 con 0x7f00500cc810 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.894+0000 7f005579a700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00500cc810 0x7f0050156530 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f004000c4e0 tx=0x7f004000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.895+0000 7f0045ffb700 1 -- 172.21.15.131:0/1715925171 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040011590 con 0x7f00500cc810 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.895+0000 7f0045ffb700 1 -- 172.21.15.131:0/1715925171 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0040011710 con 0x7f00500cc810 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.895+0000 7f0045ffb700 1 -- 172.21.15.131:0/1715925171 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040011ad0 con 0x7f00500cc810 2023-12-26T21:52:13.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.895+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00500b1ec0 con 0x7f00500cc810 2023-12-26T21:52:13.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.895+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00500b2480 con 0x7f00500cc810 2023-12-26T21:52:13.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.896+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00500657e0 con 0x7f00500cc810 2023-12-26T21:52:13.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.897+0000 7f0045ffb700 1 -- 172.21.15.131:0/1715925171 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f004001b3b0 con 0x7f00500cc810 2023-12-26T21:52:13.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.898+0000 7f0045ffb700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0034066f30 0x7f00340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:13.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.898+0000 7f0045ffb700 1 -- 172.21.15.131:0/1715925171 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f00400839f0 con 0x7f00500cc810 2023-12-26T21:52:13.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.898+0000 7f0055f9b700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0034066f30 0x7f00340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:13.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.899+0000 7f0055f9b700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0034066f30 0x7f00340693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f004c009260 tx=0x7f004c016000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:13.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:13.902+0000 7f0045ffb700 1 -- 172.21.15.131:0/1715925171 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0040050b30 con 0x7f00500cc810 2023-12-26T21:52:14.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.098+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f0050068bf0 con 0x7f00500cc810 2023-12-26T21:52:14.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.102+0000 7f0045ffb700 1 -- 172.21.15.131:0/1715925171 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0040017020 con 0x7f00500cc810 2023-12-26T21:52:14.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0034066f30 msgr2=0x7f00340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0034066f30 0x7f00340693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f004c009260 tx=0x7f004c016000 comp rx=0 tx=0).stop 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00500cc810 msgr2=0x7f0050156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00500cc810 0x7f0050156530 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f004000c4e0 tx=0x7f004000c7f0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 shutdown_connections 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0034066f30 0x7f00340693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f005013c510 0x7f0050156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f005006b8c0 0x7f0050154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 --2- 172.21.15.131:0/1715925171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00500cc810 0x7f0050156530 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 >> 172.21.15.131:0/1715925171 conn(0x7f00500ce3e0 msgr2=0x7f0050152460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 shutdown_connections 2023-12-26T21:52:14.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.105+0000 7f00581ff700 1 -- 172.21.15.131:0/1715925171 wait complete. 2023-12-26T21:52:14.123 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-12-26T21:52:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.317+0000 7fe8d6fff700 1 Processor -- start 2023-12-26T21:52:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.317+0000 7fe8d6fff700 1 -- start start 2023-12-26T21:52:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.318+0000 7fe8d6fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8d0159600 0x7fe8d00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.318+0000 7fe8d6fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:14.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.318+0000 7fe8d6fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d00cfd50 con 0x7fe8d01599d0 2023-12-26T21:52:14.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.318+0000 7fe8d6fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d00b1280 con 0x7fe8d0159600 2023-12-26T21:52:14.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.318+0000 7fe8d6fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00cf140 con 0x7fe8d00d01e0 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.318+0000 7fe8d4d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8d0159600 0x7fe8d00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.318+0000 7fe8d559c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8d559c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:60094/0 (socket says 172.21.15.131:60094) 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8d559c700 1 -- 172.21.15.131:0/1601346581 learned_addr learned my addr 172.21.15.131:0/1601346581 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8d4d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8d0159600 0x7fe8d00ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:35274/0 (socket says 172.21.15.131:35274) 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8cffff700 1 --1- 172.21.15.131:0/1601346581 >> v1:172.21.15.190:6789/0 conn(0x7fe8d01599d0 0x7fe8d015dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:42426/0 (socket says 172.21.15.131:42426) 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8d4d9b700 1 -- 172.21.15.131:0/1601346581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 msgr2=0x7fe8d00ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8d4d9b700 1 --2- 172.21.15.131:0/1601346581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8d4d9b700 1 -- 172.21.15.131:0/1601346581 >> v1:172.21.15.190:6789/0 conn(0x7fe8d01599d0 legacy=0x7fe8d015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8d4d9b700 1 -- 172.21.15.131:0/1601346581 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d00cf420 con 0x7fe8d0159600 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8d559c700 1 --2- 172.21.15.131:0/1601346581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.319+0000 7fe8d4d9b700 1 --2- 172.21.15.131:0/1601346581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159600 0x7fe8d00ceb60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098d0 tx=0x7fe8c4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b6cb6c0eef3f159 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:14.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8cf7fe700 1 -- 172.21.15.131:0/1601346581 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4005c10 con 0x7fe8d0159600 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8cf7fe700 1 -- 172.21.15.131:0/1601346581 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4005d70 con 0x7fe8d0159600 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8cf7fe700 1 -- 172.21.15.131:0/1601346581 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4006280 con 0x7fe8d0159600 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8d6fff700 1 -- 172.21.15.131:0/1601346581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159600 msgr2=0x7fe8d00ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8d6fff700 1 --2- 172.21.15.131:0/1601346581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159600 0x7fe8d00ceb60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098d0 tx=0x7fe8c4009120 comp rx=0 tx=0).stop 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8d6fff700 1 -- 172.21.15.131:0/1601346581 shutdown_connections 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8d6fff700 1 --2- 172.21.15.131:0/1601346581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8d6fff700 1 --2- 172.21.15.131:0/1601346581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159600 0x7fe8d00ceb60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8d6fff700 1 -- 172.21.15.131:0/1601346581 >> 172.21.15.131:0/1601346581 conn(0x7fe8d006ece0 msgr2=0x7fe8d006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8d6fff700 1 -- 172.21.15.131:0/1601346581 shutdown_connections 2023-12-26T21:52:14.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.320+0000 7fe8d6fff700 1 -- 172.21.15.131:0/1601346581 wait complete. 2023-12-26T21:52:14.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.322+0000 7fe8d6fff700 1 Processor -- start 2023-12-26T21:52:14.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.322+0000 7fe8d6fff700 1 -- start start 2023-12-26T21:52:14.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d6fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:14.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d6fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159600 0x7fe8d0169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:14.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d6fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0155650 0x7fe8d00b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:14.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d6fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d0159e50 con 0x7fe8d0155650 2023-12-26T21:52:14.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d6fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00c7be0 con 0x7fe8d0159600 2023-12-26T21:52:14.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d6fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00b5b90 con 0x7fe8d00d01e0 2023-12-26T21:52:14.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d559c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0155650 0x7fe8d00b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:14.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d559c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0155650 0x7fe8d00b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32894/0 (socket says 172.21.15.131:32894) 2023-12-26T21:52:14.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d559c700 1 -- 172.21.15.131:0/2132843446 learned_addr learned my addr 172.21.15.131:0/2132843446 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:14.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d4d9b700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:14.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8cffff700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159600 0x7fe8d0169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:14.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d559c700 1 -- 172.21.15.131:0/2132843446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 msgr2=0x7fe8d00c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d559c700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d559c700 1 -- 172.21.15.131:0/2132843446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159600 msgr2=0x7fe8d0169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d559c700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159600 0x7fe8d0169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.323+0000 7fe8d559c700 1 -- 172.21.15.131:0/2132843446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c8009ae0 con 0x7fe8d0155650 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.324+0000 7fe8d559c700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0155650 0x7fe8d00b3390 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009fc0 tx=0x7fe8c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.324+0000 7fe8ccff9700 1 -- 172.21.15.131:0/2132843446 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800cac0 con 0x7fe8d0155650 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.324+0000 7fe8ccff9700 1 -- 172.21.15.131:0/2132843446 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c800f040 con 0x7fe8d0155650 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.324+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c40093c0 con 0x7fe8d0155650 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.324+0000 7fe8ccff9700 1 -- 172.21.15.131:0/2132843446 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800ce90 con 0x7fe8d0155650 2023-12-26T21:52:14.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.324+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d015a0c0 con 0x7fe8d0155650 2023-12-26T21:52:14.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.325+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d00657e0 con 0x7fe8d0155650 2023-12-26T21:52:14.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.326+0000 7fe8ccff9700 1 -- 172.21.15.131:0/2132843446 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe8c8014650 con 0x7fe8d0155650 2023-12-26T21:52:14.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.327+0000 7fe8ccff9700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8b4066f30 0x7fe8b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:14.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.327+0000 7fe8ccff9700 1 -- 172.21.15.131:0/2132843446 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5847+0+0 (secure 0 0 0) 0x7fe8c8084f10 con 0x7fe8d0155650 2023-12-26T21:52:14.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.327+0000 7fe8d4d9b700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8b4066f30 0x7fe8b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:14.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.327+0000 7fe8d4d9b700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8b4066f30 0x7fe8b40693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009160 tx=0x7fe8c4016000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:14.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.331+0000 7fe8ccff9700 1 -- 172.21.15.131:0/2132843446 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8c8052050 con 0x7fe8d0155650 2023-12-26T21:52:14.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.602+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fe8d015a2e0 con 0x7fe8d0155650 2023-12-26T21:52:14.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.827+0000 7fe8ccff9700 1 -- 172.21.15.131:0/2132843446 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v558) v1 ==== 154+0+0 (secure 0 0 0) 0x7fe8c8055670 con 0x7fe8d0155650 2023-12-26T21:52:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8b4066f30 msgr2=0x7fe8b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8b4066f30 0x7fe8b40693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009160 tx=0x7fe8c4016000 comp rx=0 tx=0).stop 2023-12-26T21:52:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0155650 msgr2=0x7fe8d00b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0155650 0x7fe8d00b3390 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009fc0 tx=0x7fe8c800b040 comp rx=0 tx=0).stop 2023-12-26T21:52:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 shutdown_connections 2023-12-26T21:52:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0155650 0x7fe8d00b3390 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d00d01e0 0x7fe8d00c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8b4066f30 0x7fe8b40693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 --2- 172.21.15.131:0/2132843446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159600 0x7fe8d0169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:14.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 >> 172.21.15.131:0/2132843446 conn(0x7fe8d006ece0 msgr2=0x7fe8d01572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:14.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 shutdown_connections 2023-12-26T21:52:14.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:14.829+0000 7fe8d6fff700 1 -- 172.21.15.131:0/2132843446 wait complete. 2023-12-26T21:52:14.848 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-12-26T21:52:14.848 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-12-26T21:52:14.848 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T21:52:14.848 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-12-26T21:52:15.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.037+0000 7f4bf15ff700 1 Processor -- start 2023-12-26T21:52:15.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.037+0000 7f4bf15ff700 1 -- start start 2023-12-26T21:52:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.038+0000 7f4bf15ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4bec06b8c0 0x7f4bec0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.038+0000 7f4bf15ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4bec06bd40 0x7f4bec0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.038+0000 7f4bf15ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec15ba70 con 0x7f4bec158030 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.038+0000 7f4bf15ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec15bf00 con 0x7f4bec06b8c0 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.038+0000 7f4bf15ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec06c2a0 con 0x7f4bec06bd40 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.038+0000 7f4bea7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4bec06b8c0 0x7f4bec0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.039+0000 7f4bea7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4bec06b8c0 0x7f4bec0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49160/0 (socket says 172.21.15.131:49160) 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.039+0000 7f4bea7fc700 1 -- 172.21.15.131:0/1968544310 learned_addr learned my addr 172.21.15.131:0/1968544310 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.039+0000 7f4beb7fe700 1 --2- 172.21.15.131:0/1968544310 >> v2:172.21.15.190:3300/0 conn(0x7f4bec06bd40 0x7f4bec0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.039+0000 7f4bea7fc700 1 -- 172.21.15.131:0/1968544310 >> v2:172.21.15.190:3300/0 conn(0x7f4bec06bd40 msgr2=0x7f4bec0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.039+0000 7f4bea7fc700 1 --2- 172.21.15.131:0/1968544310 >> v2:172.21.15.190:3300/0 conn(0x7f4bec06bd40 0x7f4bec0c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.039+0000 7f4bea7fc700 1 -- 172.21.15.131:0/1968544310 >> v1:172.21.15.190:6789/0 conn(0x7f4bec158030 legacy=0x7f4bec158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.039+0000 7f4bea7fc700 1 -- 172.21.15.131:0/1968544310 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec15b140 con 0x7f4bec06b8c0 2023-12-26T21:52:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.039+0000 7f4bea7fc700 1 --2- 172.21.15.131:0/1968544310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec06b8c0 0x7f4bec0c8a20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4bec152080 tx=0x7f4be0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb6dcdc1de1c3f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4be9ffb700 1 -- 172.21.15.131:0/1968544310 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000b860 con 0x7f4bec06b8c0 2023-12-26T21:52:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4be9ffb700 1 -- 172.21.15.131:0/1968544310 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be000b9c0 con 0x7f4bec06b8c0 2023-12-26T21:52:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4be9ffb700 1 -- 172.21.15.131:0/1968544310 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000bd60 con 0x7f4bec06b8c0 2023-12-26T21:52:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4bf15ff700 1 -- 172.21.15.131:0/1968544310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec06b8c0 msgr2=0x7f4bec0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4bf15ff700 1 --2- 172.21.15.131:0/1968544310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec06b8c0 0x7f4bec0c8a20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4bec152080 tx=0x7f4be0009250 comp rx=0 tx=0).stop 2023-12-26T21:52:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4bf15ff700 1 -- 172.21.15.131:0/1968544310 shutdown_connections 2023-12-26T21:52:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4bf15ff700 1 --2- 172.21.15.131:0/1968544310 >> v2:172.21.15.190:3300/0 conn(0x7f4bec06bd40 0x7f4bec0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4bf15ff700 1 --2- 172.21.15.131:0/1968544310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec06b8c0 0x7f4bec0c8a20 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4bf15ff700 1 -- 172.21.15.131:0/1968544310 >> 172.21.15.131:0/1968544310 conn(0x7f4bec0ce3e0 msgr2=0x7f4bec0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4bf15ff700 1 -- 172.21.15.131:0/1968544310 shutdown_connections 2023-12-26T21:52:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.040+0000 7f4bf15ff700 1 -- 172.21.15.131:0/1968544310 wait complete. 2023-12-26T21:52:15.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.042+0000 7f4bf15ff700 1 Processor -- start 2023-12-26T21:52:15.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.042+0000 7f4bf15ff700 1 -- start start 2023-12-26T21:52:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4bf15ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bec06b8c0 0x7f4bec1516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4bf15ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bec06bd40 0x7f4bec14c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:15.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4bf15ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec158030 0x7f4bec0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4bf15ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec151220 con 0x7f4bec158030 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4bf15ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec150610 con 0x7f4bec06b8c0 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4bf15ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec1508f0 con 0x7f4bec06bd40 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4beb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec158030 0x7f4bec0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4beb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec158030 0x7f4bec0b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49162/0 (socket says 172.21.15.131:49162) 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4beb7fe700 1 -- 172.21.15.131:0/598438287 learned_addr learned my addr 172.21.15.131:0/598438287 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4beaffd700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bec06b8c0 0x7f4bec1516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.043+0000 7f4bea7fc700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bec06bd40 0x7f4bec14c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4beb7fe700 1 -- 172.21.15.131:0/598438287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bec06bd40 msgr2=0x7f4bec14c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4beb7fe700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bec06bd40 0x7f4bec14c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4beb7fe700 1 -- 172.21.15.131:0/598438287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bec06b8c0 msgr2=0x7f4bec1516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4beb7fe700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bec06b8c0 0x7f4bec1516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4beb7fe700 1 -- 172.21.15.131:0/598438287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00094f0 con 0x7f4bec158030 2023-12-26T21:52:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4beb7fe700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec158030 0x7f4bec0b1b50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009fa0 tx=0x7f4be4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4bd2ffd700 1 -- 172.21.15.131:0/598438287 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be400ebc0 con 0x7f4bec158030 2023-12-26T21:52:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4bd2ffd700 1 -- 172.21.15.131:0/598438287 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be400ed20 con 0x7f4bec158030 2023-12-26T21:52:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4bd2ffd700 1 -- 172.21.15.131:0/598438287 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4014670 con 0x7f4bec158030 2023-12-26T21:52:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec150bd0 con 0x7f4bec158030 2023-12-26T21:52:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.044+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec14ca80 con 0x7f4bec158030 2023-12-26T21:52:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.045+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec0657e0 con 0x7f4bec158030 2023-12-26T21:52:15.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.046+0000 7f4bd2ffd700 1 -- 172.21.15.131:0/598438287 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4be40149e0 con 0x7f4bec158030 2023-12-26T21:52:15.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.047+0000 7f4bd2ffd700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bd8066f30 0x7f4bd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:15.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.047+0000 7f4bd2ffd700 1 -- 172.21.15.131:0/598438287 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f4be4084d00 con 0x7f4bec158030 2023-12-26T21:52:15.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.047+0000 7f4beaffd700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bd8066f30 0x7f4bd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:15.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.048+0000 7f4beaffd700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bd8066f30 0x7f4bd80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bec16d2c0 tx=0x7f4bdc009040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:15.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.051+0000 7f4bd2ffd700 1 -- 172.21.15.131:0/598438287 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4be4051e40 con 0x7f4bec158030 2023-12-26T21:52:15.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.316+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f4bec15b420 con 0x7f4bec158030 2023-12-26T21:52:15.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.830+0000 7f4bd2ffd700 1 -- 172.21.15.131:0/598438287 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v559) v1 ==== 159+0+0 (secure 0 0 0) 0x7f4be4055460 con 0x7f4bec158030 2023-12-26T21:52:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.833+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bd8066f30 msgr2=0x7f4bd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.833+0000 7f4bf15ff700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bd8066f30 0x7f4bd80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bec16d2c0 tx=0x7f4bdc009040 comp rx=0 tx=0).stop 2023-12-26T21:52:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.833+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec158030 msgr2=0x7f4bec0b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.833+0000 7f4bf15ff700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec158030 0x7f4bec0b1b50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009fa0 tx=0x7f4be4009900 comp rx=0 tx=0).stop 2023-12-26T21:52:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.833+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 shutdown_connections 2023-12-26T21:52:15.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.833+0000 7f4bf15ff700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bec158030 0x7f4bec0b1b50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.834+0000 7f4bf15ff700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bd8066f30 0x7f4bd80693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.834+0000 7f4bf15ff700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bec06b8c0 0x7f4bec1516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.834+0000 7f4bf15ff700 1 --2- 172.21.15.131:0/598438287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bec06bd40 0x7f4bec14c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.834+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 >> 172.21.15.131:0/598438287 conn(0x7f4bec0ce3e0 msgr2=0x7f4bec1571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.834+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 shutdown_connections 2023-12-26T21:52:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:15.834+0000 7f4bf15ff700 1 -- 172.21.15.131:0/598438287 wait complete. 2023-12-26T21:52:15.853 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T21:52:16.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.044+0000 7fd4185ff700 1 Processor -- start 2023-12-26T21:52:16.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.044+0000 7fd4185ff700 1 -- start start 2023-12-26T21:52:16.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd4185ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4100cb320 0x7fd410069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd4185ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd41006a1e0 0x7fd4100b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd4185ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd410156940 con 0x7fd4100b0c90 2023-12-26T21:52:16.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd4185ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4100c8cc0 con 0x7fd41006a1e0 2023-12-26T21:52:16.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd4185ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4100ad700 con 0x7fd4100cb320 2023-12-26T21:52:16.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd415b9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd41006a1e0 0x7fd4100b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd415b9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd41006a1e0 0x7fd4100b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49176/0 (socket says 172.21.15.131:49176) 2023-12-26T21:52:16.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd415b9a700 1 -- 172.21.15.131:0/172775551 learned_addr learned my addr 172.21.15.131:0/172775551 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:16.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd41639b700 1 --2- 172.21.15.131:0/172775551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4100cb320 0x7fd410069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd415b9a700 1 -- 172.21.15.131:0/172775551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4100cb320 msgr2=0x7fd410069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:16.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.046+0000 7fd415b9a700 1 --2- 172.21.15.131:0/172775551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4100cb320 0x7fd410069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.047+0000 7fd415b9a700 1 -- 172.21.15.131:0/172775551 >> v1:172.21.15.190:6789/0 conn(0x7fd4100b0c90 legacy=0x7fd4100aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.047+0000 7fd415b9a700 1 -- 172.21.15.131:0/172775551 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4100ae500 con 0x7fd41006a1e0 2023-12-26T21:52:16.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.047+0000 7fd415b9a700 1 --2- 172.21.15.131:0/172775551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41006a1e0 0x7fd4100b0750 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd4000099b0 tx=0x7fd400009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6c834d088ebb17 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:16.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.047+0000 7fd415399700 1 -- 172.21.15.131:0/172775551 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40000b860 con 0x7fd41006a1e0 2023-12-26T21:52:16.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.047+0000 7fd415399700 1 -- 172.21.15.131:0/172775551 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40000b9c0 con 0x7fd41006a1e0 2023-12-26T21:52:16.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.047+0000 7fd415399700 1 -- 172.21.15.131:0/172775551 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40000bd60 con 0x7fd41006a1e0 2023-12-26T21:52:16.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.048+0000 7fd4185ff700 1 -- 172.21.15.131:0/172775551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41006a1e0 msgr2=0x7fd4100b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.048+0000 7fd4185ff700 1 --2- 172.21.15.131:0/172775551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41006a1e0 0x7fd4100b0750 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd4000099b0 tx=0x7fd400009250 comp rx=0 tx=0).stop 2023-12-26T21:52:16.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.048+0000 7fd4185ff700 1 -- 172.21.15.131:0/172775551 shutdown_connections 2023-12-26T21:52:16.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.048+0000 7fd4185ff700 1 --2- 172.21.15.131:0/172775551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4100cb320 0x7fd410069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.048+0000 7fd4185ff700 1 --2- 172.21.15.131:0/172775551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41006a1e0 0x7fd4100b0750 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.048+0000 7fd4185ff700 1 -- 172.21.15.131:0/172775551 >> 172.21.15.131:0/172775551 conn(0x7fd410068040 msgr2=0x7fd410069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:16.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.048+0000 7fd4185ff700 1 -- 172.21.15.131:0/172775551 shutdown_connections 2023-12-26T21:52:16.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.048+0000 7fd4185ff700 1 -- 172.21.15.131:0/172775551 wait complete. 2023-12-26T21:52:16.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.049+0000 7fd4185ff700 1 Processor -- start 2023-12-26T21:52:16.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.049+0000 7fd4185ff700 1 -- start start 2023-12-26T21:52:16.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.050+0000 7fd4185ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006a1e0 0x7fd4101535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.050+0000 7fd4185ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4100b0c90 0x7fd41015bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.050+0000 7fd4185ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4100cb320 0x7fd410169ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.050+0000 7fd4185ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41015a480 con 0x7fd4100b0c90 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.050+0000 7fd4185ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41015cc20 con 0x7fd4100cb320 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.050+0000 7fd4185ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41015c660 con 0x7fd41006a1e0 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd415b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4100b0c90 0x7fd41015bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd41639b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006a1e0 0x7fd4101535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd415b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4100b0c90 0x7fd41015bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49184/0 (socket says 172.21.15.131:49184) 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd415b9a700 1 -- 172.21.15.131:0/2003818888 learned_addr learned my addr 172.21.15.131:0/2003818888 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd41639b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006a1e0 0x7fd4101535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:43812/0 (socket says 172.21.15.131:43812) 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd416b9c700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4100cb320 0x7fd410169ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd415b9a700 1 -- 172.21.15.131:0/2003818888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006a1e0 msgr2=0x7fd4101535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd415b9a700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006a1e0 0x7fd4101535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd415b9a700 1 -- 172.21.15.131:0/2003818888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4100cb320 msgr2=0x7fd410169ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd415b9a700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4100cb320 0x7fd410169ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd415b9a700 1 -- 172.21.15.131:0/2003818888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4000094f0 con 0x7fd4100b0c90 2023-12-26T21:52:16.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.051+0000 7fd415b9a700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4100b0c90 0x7fd41015bca0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd400006fd0 tx=0x7fd400017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:16.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.052+0000 7fd4067fc700 1 -- 172.21.15.131:0/2003818888 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400011e00 con 0x7fd4100b0c90 2023-12-26T21:52:16.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.052+0000 7fd4067fc700 1 -- 172.21.15.131:0/2003818888 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40001a420 con 0x7fd4100b0c90 2023-12-26T21:52:16.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.052+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41015c940 con 0x7fd4100b0c90 2023-12-26T21:52:16.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.052+0000 7fd4067fc700 1 -- 172.21.15.131:0/2003818888 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40001a7e0 con 0x7fd4100b0c90 2023-12-26T21:52:16.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.052+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd410153ae0 con 0x7fd4100b0c90 2023-12-26T21:52:16.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.053+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd410159f70 con 0x7fd4100b0c90 2023-12-26T21:52:16.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.054+0000 7fd4067fc700 1 -- 172.21.15.131:0/2003818888 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd40001ab90 con 0x7fd4100b0c90 2023-12-26T21:52:16.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.054+0000 7fd4067fc700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd3f4066f30 0x7fd3f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.054+0000 7fd4067fc700 1 -- 172.21.15.131:0/2003818888 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5847+0+0 (secure 0 0 0) 0x7fd400085fe0 con 0x7fd4100b0c90 2023-12-26T21:52:16.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.055+0000 7fd41639b700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd3f4066f30 0x7fd3f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.055+0000 7fd41639b700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd3f4066f30 0x7fd3f40693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd40c006fd0 tx=0x7fd40c008040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:16.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.059+0000 7fd4067fc700 1 -- 172.21.15.131:0/2003818888 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd410159f70 con 0x7fd4100b0c90 2023-12-26T21:52:16.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.252+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd41015a630 con 0x7fd4100b0c90 2023-12-26T21:52:16.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.257+0000 7fd4067fc700 1 -- 172.21.15.131:0/2003818888 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd41015a630 con 0x7fd4100b0c90 2023-12-26T21:52:16.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd3f4066f30 msgr2=0x7fd3f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd3f4066f30 0x7fd3f40693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd40c006fd0 tx=0x7fd40c008040 comp rx=0 tx=0).stop 2023-12-26T21:52:16.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4100b0c90 msgr2=0x7fd41015bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4100b0c90 0x7fd41015bca0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd400006fd0 tx=0x7fd400017000 comp rx=0 tx=0).stop 2023-12-26T21:52:16.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 shutdown_connections 2023-12-26T21:52:16.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd3f4066f30 0x7fd3f40693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4100cb320 0x7fd410169ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006a1e0 0x7fd4101535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 --2- 172.21.15.131:0/2003818888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4100b0c90 0x7fd41015bca0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 >> 172.21.15.131:0/2003818888 conn(0x7fd410068040 msgr2=0x7fd41006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:16.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 shutdown_connections 2023-12-26T21:52:16.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.259+0000 7fd4185ff700 1 -- 172.21.15.131:0/2003818888 wait complete. 2023-12-26T21:52:16.277 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:52:16.278 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:52:16.378 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:52:16.471 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:52:16.471 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T21:52:16.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:16.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.659+0000 7ff99afff700 1 Processor -- start 2023-12-26T21:52:16.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.659+0000 7ff99afff700 1 -- start start 2023-12-26T21:52:16.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.660+0000 7ff99afff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff9940ae320 0x7ff9940b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff99afff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff99415ab40 0x7ff99406b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff99afff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9940afba0 con 0x7ff99415a770 2023-12-26T21:52:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff99afff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99406c920 con 0x7ff9940ae320 2023-12-26T21:52:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff99afff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9940b0420 con 0x7ff99415ab40 2023-12-26T21:52:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff993fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff9940ae320 0x7ff9940b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff993fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff9940ae320 0x7ff9940b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49192/0 (socket says 172.21.15.131:49192) 2023-12-26T21:52:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff993fff700 1 -- 172.21.15.131:0/2278754549 learned_addr learned my addr 172.21.15.131:0/2278754549 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:16.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff99959c700 1 --2- 172.21.15.131:0/2278754549 >> v2:172.21.15.190:3300/0 conn(0x7ff99415ab40 0x7ff99406b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff993fff700 1 -- 172.21.15.131:0/2278754549 >> v2:172.21.15.190:3300/0 conn(0x7ff99415ab40 msgr2=0x7ff99406b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff993fff700 1 --2- 172.21.15.131:0/2278754549 >> v2:172.21.15.190:3300/0 conn(0x7ff99415ab40 0x7ff99406b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff993fff700 1 -- 172.21.15.131:0/2278754549 >> v1:172.21.15.190:6789/0 conn(0x7ff99415a770 legacy=0x7ff9940adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.661+0000 7ff993fff700 1 -- 172.21.15.131:0/2278754549 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99406c640 con 0x7ff9940ae320 2023-12-26T21:52:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff993fff700 1 --2- 172.21.15.131:0/2278754549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9940ae320 0x7ff9940b08b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff9800099b0 tx=0x7ff980009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e8b4bcc7c4649e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff9937fe700 1 -- 172.21.15.131:0/2278754549 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980009ea0 con 0x7ff9940ae320 2023-12-26T21:52:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff9937fe700 1 -- 172.21.15.131:0/2278754549 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98000b820 con 0x7ff9940ae320 2023-12-26T21:52:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff9937fe700 1 -- 172.21.15.131:0/2278754549 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98000bbc0 con 0x7ff9940ae320 2023-12-26T21:52:16.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff99afff700 1 -- 172.21.15.131:0/2278754549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9940ae320 msgr2=0x7ff9940b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff99afff700 1 --2- 172.21.15.131:0/2278754549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9940ae320 0x7ff9940b08b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff9800099b0 tx=0x7ff980009250 comp rx=0 tx=0).stop 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff99afff700 1 -- 172.21.15.131:0/2278754549 shutdown_connections 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff99afff700 1 --2- 172.21.15.131:0/2278754549 >> v2:172.21.15.190:3300/0 conn(0x7ff99415ab40 0x7ff99406b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff99afff700 1 --2- 172.21.15.131:0/2278754549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9940ae320 0x7ff9940b08b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.662+0000 7ff99afff700 1 -- 172.21.15.131:0/2278754549 >> 172.21.15.131:0/2278754549 conn(0x7ff9940ce3d0 msgr2=0x7ff9940ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.663+0000 7ff99afff700 1 -- 172.21.15.131:0/2278754549 shutdown_connections 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.663+0000 7ff99afff700 1 -- 172.21.15.131:0/2278754549 wait complete. 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.664+0000 7ff99afff700 1 Processor -- start 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.664+0000 7ff99afff700 1 -- start start 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff99afff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9940ae320 0x7ff9940c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff99afff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff99415a770 0x7ff9940c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff99afff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff99415ab40 0x7ff9940a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff99afff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940a6df0 con 0x7ff99415ab40 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff99afff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940a61e0 con 0x7ff9940ae320 2023-12-26T21:52:16.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff99afff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940a64c0 con 0x7ff99415a770 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff99959c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff99415ab40 0x7ff9940a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff998d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9940ae320 0x7ff9940c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff99959c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff99415ab40 0x7ff9940a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49200/0 (socket says 172.21.15.131:49200) 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.665+0000 7ff99959c700 1 -- 172.21.15.131:0/1860714698 learned_addr learned my addr 172.21.15.131:0/1860714698 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff993fff700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff99415a770 0x7ff9940c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff99959c700 1 -- 172.21.15.131:0/1860714698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff99415a770 msgr2=0x7ff9940c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff99959c700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff99415a770 0x7ff9940c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff99959c700 1 -- 172.21.15.131:0/1860714698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9940ae320 msgr2=0x7ff9940c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff99959c700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9940ae320 0x7ff9940c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff99959c700 1 -- 172.21.15.131:0/1860714698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800094f0 con 0x7ff99415ab40 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff99959c700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff99415ab40 0x7ff9940a54b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009d70 tx=0x7ff98c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff990ff9700 1 -- 172.21.15.131:0/1860714698 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00caa0 con 0x7ff99415ab40 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff990ff9700 1 -- 172.21.15.131:0/1860714698 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c00f040 con 0x7ff99415ab40 2023-12-26T21:52:16.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9940a67a0 con 0x7ff99415ab40 2023-12-26T21:52:16.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff990ff9700 1 -- 172.21.15.131:0/1860714698 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00ce50 con 0x7ff99415ab40 2023-12-26T21:52:16.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.666+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99414eaa0 con 0x7ff99415ab40 2023-12-26T21:52:16.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.667+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9940ae160 con 0x7ff99415ab40 2023-12-26T21:52:16.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.668+0000 7ff990ff9700 1 -- 172.21.15.131:0/1860714698 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff98c014690 con 0x7ff99415ab40 2023-12-26T21:52:16.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.669+0000 7ff990ff9700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff978066f30 0x7ff9780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:16.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.669+0000 7ff990ff9700 1 -- 172.21.15.131:0/1860714698 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5847+0+0 (secure 0 0 0) 0x7ff98c0851d0 con 0x7ff99415ab40 2023-12-26T21:52:16.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.669+0000 7ff998d9b700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff978066f30 0x7ff9780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:16.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.670+0000 7ff998d9b700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff978066f30 0x7ff9780693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff988003a80 tx=0x7ff988007ec0 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:16.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.673+0000 7ff990ff9700 1 -- 172.21.15.131:0/1860714698 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff98c052310 con 0x7ff99415ab40 2023-12-26T21:52:16.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.911+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9940657e0 con 0x7ff99415ab40 2023-12-26T21:52:16.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.911+0000 7ff990ff9700 1 -- 172.21.15.131:0/1860714698 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3535 (secure 0 0 0) 0x7ff98c018de0 con 0x7ff99415ab40 2023-12-26T21:52:16.912 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:16.912 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3316959664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3316959664},{"type":"v1","addr":"172.21.15.190:6834","nonce":3316959664}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/320470110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":320470110},{"type":"v1","addr":"172.21.15.190:6835","nonce":320470110}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/1337439654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1337439654},{"type":"v1","addr":"172.21.15.131:6836","nonce":1337439654}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/379049017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":379049017},{"type":"v1","addr":"172.21.15.131:6833","nonce":379049017}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2023-12-26T21:52:16.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.913+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff978066f30 msgr2=0x7ff9780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.913+0000 7ff99afff700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff978066f30 0x7ff9780693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff988003a80 tx=0x7ff988007ec0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.913+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff99415ab40 msgr2=0x7ff9940a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:16.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.913+0000 7ff99afff700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff99415ab40 0x7ff9940a54b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009d70 tx=0x7ff98c00b040 comp rx=0 tx=0).stop 2023-12-26T21:52:16.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.914+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 shutdown_connections 2023-12-26T21:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.914+0000 7ff99afff700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff99415ab40 0x7ff9940a54b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.914+0000 7ff99afff700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff978066f30 0x7ff9780693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.914+0000 7ff99afff700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9940ae320 0x7ff9940c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.914+0000 7ff99afff700 1 --2- 172.21.15.131:0/1860714698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff99415a770 0x7ff9940c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.914+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 >> 172.21.15.131:0/1860714698 conn(0x7ff9940ce3d0 msgr2=0x7ff9940cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.914+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 shutdown_connections 2023-12-26T21:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:16.914+0000 7ff99afff700 1 -- 172.21.15.131:0/1860714698 wait complete. 2023-12-26T21:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2023-12-26T21:52:16.932 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T21:52:16.932 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T21:52:16.933 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T21:52:16.933 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:52:16.933 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T21:52:16.933 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T21:52:16.933 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T21:52:16.933 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:52:16.933 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T21:52:16.934 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T21:52:16.934 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T21:52:16.934 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:52:16.934 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T21:52:16.934 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T21:52:16.934 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T21:52:16.934 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:52:16.948 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T21:52:16.948 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T21:52:16.949 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T21:52:16.949 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T21:52:16.952 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T21:52:16.952 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T21:52:16.993 INFO:tasks.ceph.mds.c:Started 2023-12-26T21:52:16.993 DEBUG:teuthology.parallel:result is None 2023-12-26T21:52:16.996 INFO:tasks.ceph.mds.a:Started 2023-12-26T21:52:16.996 DEBUG:teuthology.parallel:result is None 2023-12-26T21:52:16.997 INFO:tasks.ceph.mds.b:Started 2023-12-26T21:52:16.998 DEBUG:teuthology.parallel:result is None 2023-12-26T21:52:17.132 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T21:52:17.132 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:52:17.137 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T21:52:17.931 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:52:17.936 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:52:21.634 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-12-26T21:52:22.953 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T21:52:22.953 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T21:52:22.958 INFO:tasks.ceph.mds.d:Started 2023-12-26T21:52:22.958 DEBUG:teuthology.parallel:result is None 2023-12-26T21:52:22.958 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T21:52:23.103 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:52:23.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.159+0000 7f02b31ff700 1 Processor -- start 2023-12-26T21:52:23.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.159+0000 7f02b31ff700 1 -- start start 2023-12-26T21:52:23.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.160+0000 7f02b31ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8bf0 0x7f02ac0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:23.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.160+0000 7f02b31ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ac0caa10 con 0x7f02ac069d10 2023-12-26T21:52:23.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.160+0000 7f02b31ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ac157a80 con 0x7f02ac0c8820 2023-12-26T21:52:23.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.160+0000 7f02b31ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0cac50 con 0x7f02ac0c8bf0 2023-12-26T21:52:23.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02b179c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f02ac069d10 0x7f02ac161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60458/0 (socket says 172.21.15.131:60458) 2023-12-26T21:52:23.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02b179c700 1 -- 172.21.15.131:0/2316139742 learned_addr learned my addr 172.21.15.131:0/2316139742 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:23.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02abfff700 1 --2- 172.21.15.131:0/2316139742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8bf0 0x7f02ac0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02ab7fe700 1 -- 172.21.15.131:0/2316139742 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903502130 0 0) 0x7f02a4001490 con 0x7f02ac069d10 2023-12-26T21:52:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02ab7fe700 1 -- 172.21.15.131:0/2316139742 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f029c003570 con 0x7f02ac069d10 2023-12-26T21:52:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02ab7fe700 1 -- 172.21.15.131:0/2316139742 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595329585 0 0) 0x7f02ac157a80 con 0x7f02ac0c8820 2023-12-26T21:52:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02abfff700 1 -- 172.21.15.131:0/2316139742 >> v1:172.21.15.190:6789/0 conn(0x7f02ac0c8820 legacy=0x7f02ac0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02abfff700 1 -- 172.21.15.131:0/2316139742 >> v1:172.21.15.131:6789/0 conn(0x7f02ac069d10 legacy=0x7f02ac161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02abfff700 1 -- 172.21.15.131:0/2316139742 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac0caf30 con 0x7f02ac0c8bf0 2023-12-26T21:52:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.161+0000 7f02abfff700 1 --2- 172.21.15.131:0/2316139742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8bf0 0x7f02ac0b12e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0298009910 tx=0x7f02980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa42ccf686353598 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:23.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02ab7fe700 1 -- 172.21.15.131:0/2316139742 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029800b680 con 0x7f02ac0c8bf0 2023-12-26T21:52:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02ab7fe700 1 -- 172.21.15.131:0/2316139742 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029800b7e0 con 0x7f02ac0c8bf0 2023-12-26T21:52:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02ab7fe700 1 -- 172.21.15.131:0/2316139742 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029800bba0 con 0x7f02ac0c8bf0 2023-12-26T21:52:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02b31ff700 1 -- 172.21.15.131:0/2316139742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8bf0 msgr2=0x7f02ac0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02b31ff700 1 --2- 172.21.15.131:0/2316139742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8bf0 0x7f02ac0b12e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0298009910 tx=0x7f02980091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02b31ff700 1 -- 172.21.15.131:0/2316139742 shutdown_connections 2023-12-26T21:52:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02b31ff700 1 --2- 172.21.15.131:0/2316139742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8bf0 0x7f02ac0b12e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02b31ff700 1 -- 172.21.15.131:0/2316139742 >> 172.21.15.131:0/2316139742 conn(0x7f02ac068040 msgr2=0x7f02ac069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02b31ff700 1 -- 172.21.15.131:0/2316139742 shutdown_connections 2023-12-26T21:52:23.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.162+0000 7f02b31ff700 1 -- 172.21.15.131:0/2316139742 wait complete. 2023-12-26T21:52:23.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.164+0000 7f02b31ff700 1 Processor -- start 2023-12-26T21:52:23.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.164+0000 7f02b31ff700 1 -- start start 2023-12-26T21:52:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b31ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02ac069d10 0x7f02ac0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b31ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8820 0x7f02ac0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b31ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ac0c8bf0 0x7f02ac1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b31ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0c47a0 con 0x7f02ac069d10 2023-12-26T21:52:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b31ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0c6e10 con 0x7f02ac0c8bf0 2023-12-26T21:52:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b31ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0c70f0 con 0x7f02ac0c8820 2023-12-26T21:52:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b0f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02ac069d10 0x7f02ac0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b0f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02ac069d10 0x7f02ac0c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49236/0 (socket says 172.21.15.131:49236) 2023-12-26T21:52:23.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b0f9b700 1 -- 172.21.15.131:0/3679485339 learned_addr learned my addr 172.21.15.131:0/3679485339 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b179c700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ac0c8bf0 0x7f02ac1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02abfff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8820 0x7f02ac0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b0f9b700 1 -- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8820 msgr2=0x7f02ac0c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.165+0000 7f02b0f9b700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8820 0x7f02ac0c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02b0f9b700 1 -- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ac0c8bf0 msgr2=0x7f02ac1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02b0f9b700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ac0c8bf0 0x7f02ac1558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02b0f9b700 1 -- 172.21.15.131:0/3679485339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0298009420 con 0x7f02ac069d10 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02abfff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8820 0x7f02ac0c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02b179c700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ac0c8bf0 0x7f02ac1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02b0f9b700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02ac069d10 0x7f02ac0c6730 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f02a000c530 tx=0x7f02a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:23.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02a8ff9700 1 -- 172.21.15.131:0/3679485339 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0011590 con 0x7f02ac069d10 2023-12-26T21:52:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02a8ff9700 1 -- 172.21.15.131:0/3679485339 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a00116f0 con 0x7f02ac069d10 2023-12-26T21:52:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac0c73d0 con 0x7f02ac069d10 2023-12-26T21:52:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02a8ff9700 1 -- 172.21.15.131:0/3679485339 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0011ab0 con 0x7f02ac069d10 2023-12-26T21:52:23.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.166+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac155e30 con 0x7f02ac069d10 2023-12-26T21:52:23.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.167+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac1678b0 con 0x7f02ac069d10 2023-12-26T21:52:23.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.168+0000 7f02a8ff9700 1 -- 172.21.15.131:0/3679485339 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f02a001b3b0 con 0x7f02ac069d10 2023-12-26T21:52:23.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.169+0000 7f02a8ff9700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0290066f30 0x7f02900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:23.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.169+0000 7f02a8ff9700 1 -- 172.21.15.131:0/3679485339 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f02a0083b90 con 0x7f02ac069d10 2023-12-26T21:52:23.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.169+0000 7f02abfff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0290066f30 0x7f02900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:23.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.170+0000 7f02abfff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0290066f30 0x7f02900693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0298009860 tx=0x7f0298017000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:23.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.174+0000 7f02a8ff9700 1 -- 172.21.15.131:0/3679485339 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02a0050d70 con 0x7f02ac069d10 2023-12-26T21:52:23.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:23.383+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f02ac06f450 con 0x7f02ac069d10 2023-12-26T21:52:24.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.221+0000 7f02a8ff9700 1 -- 172.21.15.131:0/3679485339 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v78) v1 ==== 97+0+0 (secure 0 0 0) 0x7f02a0017070 con 0x7f02ac069d10 2023-12-26T21:52:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0290066f30 msgr2=0x7f02900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0290066f30 0x7f02900693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0298009860 tx=0x7f0298017000 comp rx=0 tx=0).stop 2023-12-26T21:52:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02ac069d10 msgr2=0x7f02ac0c6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02ac069d10 0x7f02ac0c6730 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f02a000c530 tx=0x7f02a000c8f0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 shutdown_connections 2023-12-26T21:52:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f0290066f30 0x7f02900693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02ac0c8bf0 0x7f02ac1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02ac069d10 0x7f02ac0c6730 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 --2- 172.21.15.131:0/3679485339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02ac0c8820 0x7f02ac0c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 >> 172.21.15.131:0/3679485339 conn(0x7f02ac068040 msgr2=0x7f02ac158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 shutdown_connections 2023-12-26T21:52:24.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.224+0000 7f02b31ff700 1 -- 172.21.15.131:0/3679485339 wait complete. 2023-12-26T21:52:24.227 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T21:52:24.246 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T21:52:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.438+0000 7f23bbfff700 1 Processor -- start 2023-12-26T21:52:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.438+0000 7f23bbfff700 1 -- start start 2023-12-26T21:52:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.439+0000 7f23bbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 0x7f23b40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.439+0000 7f23bbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23b4069d10 0x7f23b40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.439+0000 7f23bbfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b40caa10 con 0x7f23b40c8820 2023-12-26T21:52:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.439+0000 7f23bbfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b4157a80 con 0x7f23b4069d10 2023-12-26T21:52:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.439+0000 7f23bbfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40cac50 con 0x7f23b40c8bf0 2023-12-26T21:52:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.439+0000 7f23ba59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23b4069d10 0x7f23b40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.439+0000 7f23b9d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f23b40c8820 0x7f23b40ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60474/0 (socket says 172.21.15.131:60474) 2023-12-26T21:52:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.439+0000 7f23b9d9b700 1 -- 172.21.15.131:0/301043661 learned_addr learned my addr 172.21.15.131:0/301043661 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23b959a700 1 --2- 172.21.15.131:0/301043661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 0x7f23b40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23ba59c700 1 -- 172.21.15.131:0/301043661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 msgr2=0x7f23b40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:24.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23ba59c700 1 --2- 172.21.15.131:0/301043661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 0x7f23b40b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23ba59c700 1 -- 172.21.15.131:0/301043661 >> v1:172.21.15.131:6789/0 conn(0x7f23b40c8820 legacy=0x7f23b40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23ba59c700 1 -- 172.21.15.131:0/301043661 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40caf30 con 0x7f23b4069d10 2023-12-26T21:52:24.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23b959a700 1 --2- 172.21.15.131:0/301043661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 0x7f23b40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:24.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23b8d99700 1 -- 172.21.15.131:0/301043661 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251398490 0 0) 0x7f23b0001110 con 0x7f23b40c8820 2023-12-26T21:52:24.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23ba59c700 1 --2- 172.21.15.131:0/301043661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 0x7f23b40b1820 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009e60 tx=0x7f23ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458c3074d752ed1c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:24.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23b8d99700 1 -- 172.21.15.131:0/301043661 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac00e070 con 0x7f23b4069d10 2023-12-26T21:52:24.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23b8d99700 1 -- 172.21.15.131:0/301043661 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac00bc30 con 0x7f23b4069d10 2023-12-26T21:52:24.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.440+0000 7f23b8d99700 1 -- 172.21.15.131:0/301043661 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac012430 con 0x7f23b4069d10 2023-12-26T21:52:24.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.441+0000 7f23bbfff700 1 -- 172.21.15.131:0/301043661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 msgr2=0x7f23b40b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.441+0000 7f23bbfff700 1 --2- 172.21.15.131:0/301043661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 0x7f23b40b1820 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009e60 tx=0x7f23ac009780 comp rx=0 tx=0).stop 2023-12-26T21:52:24.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.441+0000 7f23bbfff700 1 -- 172.21.15.131:0/301043661 shutdown_connections 2023-12-26T21:52:24.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.441+0000 7f23bbfff700 1 --2- 172.21.15.131:0/301043661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 0x7f23b40b1820 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.441+0000 7f23bbfff700 1 --2- 172.21.15.131:0/301043661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 0x7f23b40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.441+0000 7f23bbfff700 1 -- 172.21.15.131:0/301043661 >> 172.21.15.131:0/301043661 conn(0x7f23b4068040 msgr2=0x7f23b4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:24.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.441+0000 7f23bbfff700 1 -- 172.21.15.131:0/301043661 shutdown_connections 2023-12-26T21:52:24.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.441+0000 7f23bbfff700 1 -- 172.21.15.131:0/301043661 wait complete. 2023-12-26T21:52:24.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.442+0000 7f23bbfff700 1 Processor -- start 2023-12-26T21:52:24.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.442+0000 7f23bbfff700 1 -- start start 2023-12-26T21:52:24.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.443+0000 7f23bbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 0x7f23b40c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.443+0000 7f23bbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b40c8820 0x7f23b40af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.443+0000 7f23bbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 0x7f23b4152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.443+0000 7f23bbfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40c5f00 con 0x7f23b4069d10 2023-12-26T21:52:24.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.443+0000 7f23bbfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40af610 con 0x7f23b40c8820 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.443+0000 7f23bbfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40af840 con 0x7f23b40c8bf0 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 0x7f23b40c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 0x7f23b40c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49254/0 (socket says 172.21.15.131:49254) 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b9d9b700 1 -- 172.21.15.131:0/376754759 learned_addr learned my addr 172.21.15.131:0/376754759 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23ba59c700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 0x7f23b4152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b959a700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b40c8820 0x7f23b40af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b9d9b700 1 -- 172.21.15.131:0/376754759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 msgr2=0x7f23b4152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b9d9b700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 0x7f23b4152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b9d9b700 1 -- 172.21.15.131:0/376754759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b40c8820 msgr2=0x7f23b40af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b9d9b700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b40c8820 0x7f23b40af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b9d9b700 1 -- 172.21.15.131:0/376754759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac009a20 con 0x7f23b4069d10 2023-12-26T21:52:24.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23b9d9b700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 0x7f23b40c59c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f23b000c810 tx=0x7f23b0007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:24.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.444+0000 7f23a9ffb700 1 -- 172.21.15.131:0/376754759 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0005b30 con 0x7f23b4069d10 2023-12-26T21:52:24.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.445+0000 7f23a9ffb700 1 -- 172.21.15.131:0/376754759 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b0005c90 con 0x7f23b4069d10 2023-12-26T21:52:24.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.445+0000 7f23a9ffb700 1 -- 172.21.15.131:0/376754759 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0007300 con 0x7f23b4069d10 2023-12-26T21:52:24.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.445+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b40afb20 con 0x7f23b4069d10 2023-12-26T21:52:24.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.445+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b40b00e0 con 0x7f23b4069d10 2023-12-26T21:52:24.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.446+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b40b1b70 con 0x7f23b4069d10 2023-12-26T21:52:24.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.447+0000 7f23a9ffb700 1 -- 172.21.15.131:0/376754759 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f23b00076b0 con 0x7f23b4069d10 2023-12-26T21:52:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.447+0000 7f23a9ffb700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2398066f30 0x7f23980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.447+0000 7f23a9ffb700 1 -- 172.21.15.131:0/376754759 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f23b0083da0 con 0x7f23b4069d10 2023-12-26T21:52:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.448+0000 7f23b959a700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2398066f30 0x7f23980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.448+0000 7f23b959a700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2398066f30 0x7f23980693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f23a4006f00 tx=0x7f23a4008040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:24.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.452+0000 7f23a9ffb700 1 -- 172.21.15.131:0/376754759 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23b0054bd0 con 0x7f23b4069d10 2023-12-26T21:52:24.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.720+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f23b40caa10 con 0x7f23b4069d10 2023-12-26T21:52:24.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.720+0000 7f23a9ffb700 1 -- 172.21.15.131:0/376754759 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f23b40caa10 con 0x7f23b4069d10 2023-12-26T21:52:24.722 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:24.722 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:52:24.722 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T21:52:24.722 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.722 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T21:52:24.723 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T21:52:24.723 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.723 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:52:24.723 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:52:24.723 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:52:24.723 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.724 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.725 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:52:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T21:52:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.729 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.730 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.731 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.732 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T21:52:24.733 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:52:24.734 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T21:52:24.735 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T21:52:24.736 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:52:24.736 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:52:24.736 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:52:24.736 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:52:24.736 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.736 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:24.736 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T21:52:24.736 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:52:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2398066f30 msgr2=0x7f23980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2398066f30 0x7f23980693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f23a4006f00 tx=0x7f23a4008040 comp rx=0 tx=0).stop 2023-12-26T21:52:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 msgr2=0x7f23b40c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 0x7f23b40c59c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f23b000c810 tx=0x7f23b0007c50 comp rx=0 tx=0).stop 2023-12-26T21:52:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 shutdown_connections 2023-12-26T21:52:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23b40c8bf0 0x7f23b4152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23b4069d10 0x7f23b40c59c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2398066f30 0x7f23980693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 --2- 172.21.15.131:0/376754759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23b40c8820 0x7f23b40af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 >> 172.21.15.131:0/376754759 conn(0x7f23b4068040 msgr2=0x7f23b406e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.723+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 shutdown_connections 2023-12-26T21:52:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.724+0000 7f23bbfff700 1 -- 172.21.15.131:0/376754759 wait complete. 2023-12-26T21:52:24.742 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:24.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.933+0000 7fe9e35ff700 1 Processor -- start 2023-12-26T21:52:24.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.933+0000 7fe9e35ff700 1 -- start start 2023-12-26T21:52:24.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.934+0000 7fe9e35ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc0b07f0 0x7fe9dc1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.934+0000 7fe9e35ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9dc159cf0 0x7fe9dc155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.934+0000 7fe9e35ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9dc0c8bf0 con 0x7fe9dc159920 2023-12-26T21:52:24.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.934+0000 7fe9e35ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc155f50 con 0x7fe9dc159cf0 2023-12-26T21:52:24.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.934+0000 7fe9e35ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0c8e60 con 0x7fe9dc0b07f0 2023-12-26T21:52:24.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.934+0000 7fe9e1b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9dc159cf0 0x7fe9dc155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.934+0000 7fe9e1b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9dc159cf0 0x7fe9dc155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33596/0 (socket says 172.21.15.131:33596) 2023-12-26T21:52:24.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.934+0000 7fe9e1b9c700 1 -- 172.21.15.131:0/2753836194 learned_addr learned my addr 172.21.15.131:0/2753836194 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:24.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.934+0000 7fe9e0b9a700 1 --2- 172.21.15.131:0/2753836194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc0b07f0 0x7fe9dc1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.935+0000 7fe9e1b9c700 1 -- 172.21.15.131:0/2753836194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc0b07f0 msgr2=0x7fe9dc1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:24.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.935+0000 7fe9e1b9c700 1 --2- 172.21.15.131:0/2753836194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc0b07f0 0x7fe9dc1554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.935+0000 7fe9e1b9c700 1 -- 172.21.15.131:0/2753836194 >> v1:172.21.15.190:6789/0 conn(0x7fe9dc159920 legacy=0x7fe9dc0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.935+0000 7fe9e1b9c700 1 -- 172.21.15.131:0/2753836194 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc0c9140 con 0x7fe9dc159cf0 2023-12-26T21:52:24.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.935+0000 7fe9e1b9c700 1 --2- 172.21.15.131:0/2753836194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 0x7fe9dc155a10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009f20 tx=0x7fe9d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf269bb8030e463 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:24.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.935+0000 7fe9d3fff700 1 -- 172.21.15.131:0/2753836194 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400e070 con 0x7fe9dc159cf0 2023-12-26T21:52:24.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.935+0000 7fe9d3fff700 1 -- 172.21.15.131:0/2753836194 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d400ba80 con 0x7fe9dc159cf0 2023-12-26T21:52:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.935+0000 7fe9d3fff700 1 -- 172.21.15.131:0/2753836194 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40092e0 con 0x7fe9dc159cf0 2023-12-26T21:52:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.936+0000 7fe9e35ff700 1 -- 172.21.15.131:0/2753836194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 msgr2=0x7fe9dc155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.936+0000 7fe9e35ff700 1 --2- 172.21.15.131:0/2753836194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 0x7fe9dc155a10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009f20 tx=0x7fe9d4009840 comp rx=0 tx=0).stop 2023-12-26T21:52:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.936+0000 7fe9e35ff700 1 -- 172.21.15.131:0/2753836194 shutdown_connections 2023-12-26T21:52:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.936+0000 7fe9e35ff700 1 --2- 172.21.15.131:0/2753836194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 0x7fe9dc155a10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.936+0000 7fe9e35ff700 1 --2- 172.21.15.131:0/2753836194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc0b07f0 0x7fe9dc1554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.936+0000 7fe9e35ff700 1 -- 172.21.15.131:0/2753836194 >> 172.21.15.131:0/2753836194 conn(0x7fe9dc068040 msgr2=0x7fe9dc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.936+0000 7fe9e35ff700 1 -- 172.21.15.131:0/2753836194 shutdown_connections 2023-12-26T21:52:24.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.936+0000 7fe9e35ff700 1 -- 172.21.15.131:0/2753836194 wait complete. 2023-12-26T21:52:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.938+0000 7fe9e35ff700 1 Processor -- start 2023-12-26T21:52:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.938+0000 7fe9e35ff700 1 -- start start 2023-12-26T21:52:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e35ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9dc159920 0x7fe9dc154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e35ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 0x7fe9dc1694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e35ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc1580a0 0x7fe9dc15db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e35ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc1699f0 con 0x7fe9dc159cf0 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e35ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc15b750 con 0x7fe9dc159920 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e35ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc159170 con 0x7fe9dc1580a0 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e0b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 0x7fe9dc1694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e0b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 0x7fe9dc1694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33602/0 (socket says 172.21.15.131:33602) 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e0b9a700 1 -- 172.21.15.131:0/275958925 learned_addr learned my addr 172.21.15.131:0/275958925 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e1b9c700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc1580a0 0x7fe9dc15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e139b700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9dc159920 0x7fe9dc154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e0b9a700 1 -- 172.21.15.131:0/275958925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc1580a0 msgr2=0x7fe9dc15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e0b9a700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc1580a0 0x7fe9dc15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e0b9a700 1 -- 172.21.15.131:0/275958925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9dc159920 msgr2=0x7fe9dc154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e0b9a700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9dc159920 0x7fe9dc154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.939+0000 7fe9e0b9a700 1 -- 172.21.15.131:0/275958925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d4009ae0 con 0x7fe9dc159cf0 2023-12-26T21:52:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.940+0000 7fe9e0b9a700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 0x7fe9dc1694b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00d8c0 tx=0x7fe9cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.940+0000 7fe9d17fa700 1 -- 172.21.15.131:0/275958925 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc0125f0 con 0x7fe9dc159cf0 2023-12-26T21:52:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.940+0000 7fe9d17fa700 1 -- 172.21.15.131:0/275958925 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc012770 con 0x7fe9dc159cf0 2023-12-26T21:52:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.940+0000 7fe9d17fa700 1 -- 172.21.15.131:0/275958925 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc012b30 con 0x7fe9dc159cf0 2023-12-26T21:52:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.940+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc1588d0 con 0x7fe9dc159cf0 2023-12-26T21:52:24.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.940+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc158e90 con 0x7fe9dc159cf0 2023-12-26T21:52:24.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.941+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc0657e0 con 0x7fe9dc159cf0 2023-12-26T21:52:24.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.942+0000 7fe9d17fa700 1 -- 172.21.15.131:0/275958925 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe9cc01e460 con 0x7fe9dc159cf0 2023-12-26T21:52:24.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.943+0000 7fe9d17fa700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe9c0066f30 0x7fe9c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:24.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.943+0000 7fe9d17fa700 1 -- 172.21.15.131:0/275958925 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fe9cc084950 con 0x7fe9dc159cf0 2023-12-26T21:52:24.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.943+0000 7fe9e139b700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe9c0066f30 0x7fe9c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:24.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.944+0000 7fe9e139b700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe9c0066f30 0x7fe9c00693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8003a80 tx=0x7fe9d8007ec0 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:24.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:24.947+0000 7fe9d17fa700 1 -- 172.21.15.131:0/275958925 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9cc051cf0 con 0x7fe9dc159cf0 2023-12-26T21:52:25.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.141+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9dc0c9420 con 0x7fe9dc159cf0 2023-12-26T21:52:25.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.142+0000 7fe9d17fa700 1 -- 172.21.15.131:0/275958925 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+12778 (secure 0 0 0) 0x7fe9cc055310 con 0x7fe9dc159cf0 2023-12-26T21:52:25.143 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:25.143 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":78,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:24.219947+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe9c0066f30 msgr2=0x7fe9c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe9c0066f30 0x7fe9c00693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8003a80 tx=0x7fe9d8007ec0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 msgr2=0x7fe9dc1694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 0x7fe9dc1694b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00d8c0 tx=0x7fe9cc008040 comp rx=0 tx=0).stop 2023-12-26T21:52:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 shutdown_connections 2023-12-26T21:52:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9dc1580a0 0x7fe9dc15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe9c0066f30 0x7fe9c00693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9dc159920 0x7fe9dc154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 --2- 172.21.15.131:0/275958925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9dc159cf0 0x7fe9dc1694b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 >> 172.21.15.131:0/275958925 conn(0x7fe9dc068040 msgr2=0x7fe9dc0aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:25.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 shutdown_connections 2023-12-26T21:52:25.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.144+0000 7fe9e35ff700 1 -- 172.21.15.131:0/275958925 wait complete. 2023-12-26T21:52:25.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T21:52:25.164 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:52:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.357+0000 7f49767ff700 1 Processor -- start 2023-12-26T21:52:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.358+0000 7f49767ff700 1 -- start start 2023-12-26T21:52:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.359+0000 7f49767ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f497006bfd0 0x7f49700c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.359+0000 7f49767ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49700686c0 con 0x7f49700cc100 2023-12-26T21:52:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.359+0000 7f49767ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4970069520 con 0x7f497013c510 2023-12-26T21:52:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.359+0000 7f49767ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4970068910 con 0x7f497006bfd0 2023-12-26T21:52:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.359+0000 7f496f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f497006bfd0 0x7f49700c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.359+0000 7f4974d9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f49700cc100 0x7f49700cc520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46580/0 (socket says 172.21.15.131:46580) 2023-12-26T21:52:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.359+0000 7f496f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f497006bfd0 0x7f49700c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33612/0 (socket says 172.21.15.131:33612) 2023-12-26T21:52:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.359+0000 7f4974d9c700 1 -- 172.21.15.131:0/1579930958 learned_addr learned my addr 172.21.15.131:0/1579930958 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.360+0000 7f496f7fe700 1 -- 172.21.15.131:0/1579930958 >> v1:172.21.15.190:6789/0 conn(0x7f497013c510 legacy=0x7f497006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.360+0000 7f496f7fe700 1 -- 172.21.15.131:0/1579930958 >> v1:172.21.15.131:6789/0 conn(0x7f49700cc100 legacy=0x7f49700cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.360+0000 7f496f7fe700 1 -- 172.21.15.131:0/1579930958 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970068bf0 con 0x7f497006bfd0 2023-12-26T21:52:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.360+0000 7f496effd700 1 -- 172.21.15.131:0/1579930958 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113199244 0 0) 0x7f49700686c0 con 0x7f49700cc100 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.360+0000 7f496f7fe700 1 --2- 172.21.15.131:0/1579930958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 0x7f49700c8a20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f49640098e0 tx=0x7f49640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a134a2728b179c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.360+0000 7f496effd700 1 -- 172.21.15.131:0/1579930958 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964009e80 con 0x7f497006bfd0 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.360+0000 7f496effd700 1 -- 172.21.15.131:0/1579930958 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496400b860 con 0x7f497006bfd0 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.360+0000 7f496effd700 1 -- 172.21.15.131:0/1579930958 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496400bbe0 con 0x7f497006bfd0 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.361+0000 7f49767ff700 1 -- 172.21.15.131:0/1579930958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 msgr2=0x7f49700c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.361+0000 7f49767ff700 1 --2- 172.21.15.131:0/1579930958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 0x7f49700c8a20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f49640098e0 tx=0x7f49640091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.361+0000 7f49767ff700 1 -- 172.21.15.131:0/1579930958 shutdown_connections 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.361+0000 7f49767ff700 1 --2- 172.21.15.131:0/1579930958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 0x7f49700c8a20 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.361+0000 7f49767ff700 1 -- 172.21.15.131:0/1579930958 >> 172.21.15.131:0/1579930958 conn(0x7f49700ce3e0 msgr2=0x7f49700ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.361+0000 7f49767ff700 1 -- 172.21.15.131:0/1579930958 shutdown_connections 2023-12-26T21:52:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.361+0000 7f49767ff700 1 -- 172.21.15.131:0/1579930958 wait complete. 2023-12-26T21:52:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.362+0000 7f49767ff700 1 Processor -- start 2023-12-26T21:52:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.362+0000 7f49767ff700 1 -- start start 2023-12-26T21:52:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.363+0000 7f49767ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 0x7f49700b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.363+0000 7f49767ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49700cc100 0x7f4970156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.363+0000 7f49767ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f497013c510 0x7f49701588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.363+0000 7f49767ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497016a870 con 0x7f497006bfd0 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.363+0000 7f49767ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970169c60 con 0x7f497013c510 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.363+0000 7f49767ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4970169f40 con 0x7f49700cc100 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.363+0000 7f496ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 0x7f49700b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.363+0000 7f496ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 0x7f49700b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33620/0 (socket says 172.21.15.131:33620) 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.363+0000 7f496ffff700 1 -- 172.21.15.131:0/545115028 learned_addr learned my addr 172.21.15.131:0/545115028 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f4974d9c700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f497013c510 0x7f49701588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f496f7fe700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49700cc100 0x7f4970156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f496ffff700 1 -- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49700cc100 msgr2=0x7f4970156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f496ffff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49700cc100 0x7f4970156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f496ffff700 1 -- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f497013c510 msgr2=0x7f49701588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f496ffff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f497013c510 0x7f49701588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f496ffff700 1 -- 172.21.15.131:0/545115028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4964009420 con 0x7f497006bfd0 2023-12-26T21:52:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f4974d9c700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f497013c510 0x7f49701588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f496f7fe700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49700cc100 0x7f4970156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f496ffff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 0x7f49700b20b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f496000c7c0 tx=0x7f4960007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f4957fff700 1 -- 172.21.15.131:0/545115028 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960011590 con 0x7f497006bfd0 2023-12-26T21:52:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f4957fff700 1 -- 172.21.15.131:0/545115028 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4960011710 con 0x7f497006bfd0 2023-12-26T21:52:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f4957fff700 1 -- 172.21.15.131:0/545115028 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960011ad0 con 0x7f497006bfd0 2023-12-26T21:52:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.364+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497016a220 con 0x7f497006bfd0 2023-12-26T21:52:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.365+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4970169950 con 0x7f497006bfd0 2023-12-26T21:52:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.365+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497016ab50 con 0x7f497006bfd0 2023-12-26T21:52:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.366+0000 7f4957fff700 1 -- 172.21.15.131:0/545115028 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f496001b3b0 con 0x7f497006bfd0 2023-12-26T21:52:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.367+0000 7f4957fff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f495c066f30 0x7f495c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.367+0000 7f4957fff700 1 -- 172.21.15.131:0/545115028 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f4960083870 con 0x7f497006bfd0 2023-12-26T21:52:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.368+0000 7f496f7fe700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f495c066f30 0x7f495c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.368+0000 7f496f7fe700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f495c066f30 0x7f495c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f497006c3d0 tx=0x7f4964017000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.371+0000 7f4957fff700 1 -- 172.21.15.131:0/545115028 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4960050d60 con 0x7f497006bfd0 2023-12-26T21:52:25.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.567+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49700657e0 con 0x7f497006bfd0 2023-12-26T21:52:25.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.568+0000 7f4957fff700 1 -- 172.21.15.131:0/545115028 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4960017070 con 0x7f497006bfd0 2023-12-26T21:52:25.569 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:52:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f495c066f30 msgr2=0x7f495c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f495c066f30 0x7f495c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f497006c3d0 tx=0x7f4964017000 comp rx=0 tx=0).stop 2023-12-26T21:52:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 msgr2=0x7f49700b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 0x7f49700b20b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f496000c7c0 tx=0x7f4960007f70 comp rx=0 tx=0).stop 2023-12-26T21:52:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 shutdown_connections 2023-12-26T21:52:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f495c066f30 0x7f495c0693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f497013c510 0x7f49701588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f497006bfd0 0x7f49700b20b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 --2- 172.21.15.131:0/545115028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49700cc100 0x7f4970156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 >> 172.21.15.131:0/545115028 conn(0x7f49700ce3e0 msgr2=0x7f4970153500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 shutdown_connections 2023-12-26T21:52:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.570+0000 7f49767ff700 1 -- 172.21.15.131:0/545115028 wait complete. 2023-12-26T21:52:25.588 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T21:52:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.777+0000 7f86595ff700 1 Processor -- start 2023-12-26T21:52:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.778+0000 7f86595ff700 1 -- start start 2023-12-26T21:52:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f86595ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86541596a0 0x7f86540b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f86595ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8654159a70 0x7f86540b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f86595ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86540afcd0 con 0x7f86540b0160 2023-12-26T21:52:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f86595ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86540b3e70 con 0x7f8654159a70 2023-12-26T21:52:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f86595ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86540b0cd0 con 0x7f86541596a0 2023-12-26T21:52:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f86537fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f86540b0160 0x7f86540b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46592/0 (socket says 172.21.15.131:46592) 2023-12-26T21:52:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f86537fe700 1 -- 172.21.15.131:0/3186490162 learned_addr learned my addr 172.21.15.131:0/3186490162 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f8652ffd700 1 --2- 172.21.15.131:0/3186490162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86541596a0 0x7f86540b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f86527fc700 1 --2- 172.21.15.131:0/3186490162 >> v2:172.21.15.190:3300/0 conn(0x7f8654159a70 0x7f86540b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.779+0000 7f8651ffb700 1 -- 172.21.15.131:0/3186490162 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191658863 0 0) 0x7f86540afcd0 con 0x7f86540b0160 2023-12-26T21:52:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.780+0000 7f8651ffb700 1 -- 172.21.15.131:0/3186490162 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f863c003570 con 0x7f86540b0160 2023-12-26T21:52:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.780+0000 7f86527fc700 1 -- 172.21.15.131:0/3186490162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86541596a0 msgr2=0x7f86540b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.780+0000 7f86527fc700 1 --2- 172.21.15.131:0/3186490162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86541596a0 0x7f86540b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.780+0000 7f86527fc700 1 -- 172.21.15.131:0/3186490162 >> v1:172.21.15.131:6789/0 conn(0x7f86540b0160 legacy=0x7f86540b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.780+0000 7f86527fc700 1 -- 172.21.15.131:0/3186490162 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540b2e00 con 0x7f8654159a70 2023-12-26T21:52:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.780+0000 7f86527fc700 1 --2- 172.21.15.131:0/3186490162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8654159a70 0x7f86540b28c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86480098e0 tx=0x7f86480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e0369198c04ba server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.780+0000 7f8651ffb700 1 -- 172.21.15.131:0/3186490162 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648011630 con 0x7f8654159a70 2023-12-26T21:52:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.780+0000 7f8651ffb700 1 -- 172.21.15.131:0/3186490162 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8648011790 con 0x7f8654159a70 2023-12-26T21:52:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.780+0000 7f8651ffb700 1 -- 172.21.15.131:0/3186490162 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648011b10 con 0x7f8654159a70 2023-12-26T21:52:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.781+0000 7f86595ff700 1 -- 172.21.15.131:0/3186490162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8654159a70 msgr2=0x7f86540b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.781+0000 7f86595ff700 1 --2- 172.21.15.131:0/3186490162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8654159a70 0x7f86540b28c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86480098e0 tx=0x7f86480091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.781+0000 7f86595ff700 1 -- 172.21.15.131:0/3186490162 shutdown_connections 2023-12-26T21:52:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.781+0000 7f86595ff700 1 --2- 172.21.15.131:0/3186490162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86541596a0 0x7f86540b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.781+0000 7f86595ff700 1 --2- 172.21.15.131:0/3186490162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8654159a70 0x7f86540b28c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.781+0000 7f86595ff700 1 -- 172.21.15.131:0/3186490162 >> 172.21.15.131:0/3186490162 conn(0x7f865406ece0 msgr2=0x7f865406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.781+0000 7f86595ff700 1 -- 172.21.15.131:0/3186490162 shutdown_connections 2023-12-26T21:52:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.781+0000 7f86595ff700 1 -- 172.21.15.131:0/3186490162 wait complete. 2023-12-26T21:52:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.782+0000 7f86595ff700 1 Processor -- start 2023-12-26T21:52:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.783+0000 7f86595ff700 1 -- start start 2023-12-26T21:52:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.783+0000 7f86595ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86540b0160 0x7f865416c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.783+0000 7f86595ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86541596a0 0x7f86540c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.783+0000 7f86595ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8654159a70 0x7f86540cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.783+0000 7f86595ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c9680 con 0x7f86540b0160 2023-12-26T21:52:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.783+0000 7f86595ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c9960 con 0x7f86541596a0 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.783+0000 7f86595ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c9c40 con 0x7f8654159a70 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f8652ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86540b0160 0x7f865416c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f8652ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86540b0160 0x7f865416c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33622/0 (socket says 172.21.15.131:33622) 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f8652ffd700 1 -- 172.21.15.131:0/467410738 learned_addr learned my addr 172.21.15.131:0/467410738 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f86527fc700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86541596a0 0x7f86540c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f86537fe700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8654159a70 0x7f86540cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f8652ffd700 1 -- 172.21.15.131:0/467410738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8654159a70 msgr2=0x7f86540cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f8652ffd700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8654159a70 0x7f86540cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f8652ffd700 1 -- 172.21.15.131:0/467410738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86541596a0 msgr2=0x7f86540c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f8652ffd700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86541596a0 0x7f86540c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f8652ffd700 1 -- 172.21.15.131:0/467410738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8648009420 con 0x7f86540b0160 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f86537fe700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8654159a70 0x7f86540cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f8652ffd700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86540b0160 0x7f865416c7c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f864400ec80 tx=0x7f86440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.784+0000 7f863affd700 1 -- 172.21.15.131:0/467410738 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864400c850 con 0x7f86540b0160 2023-12-26T21:52:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.785+0000 7f863affd700 1 -- 172.21.15.131:0/467410738 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864400c9b0 con 0x7f86540b0160 2023-12-26T21:52:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.785+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86540c9f20 con 0x7f86540b0160 2023-12-26T21:52:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.785+0000 7f863affd700 1 -- 172.21.15.131:0/467410738 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864400cd50 con 0x7f86540b0160 2023-12-26T21:52:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.785+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865416cd00 con 0x7f86540b0160 2023-12-26T21:52:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.786+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86540c93d0 con 0x7f86540b0160 2023-12-26T21:52:25.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.787+0000 7f863affd700 1 -- 172.21.15.131:0/467410738 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8644013680 con 0x7f86540b0160 2023-12-26T21:52:25.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.787+0000 7f863affd700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f8640066f30 0x7f86400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:25.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.787+0000 7f863affd700 1 -- 172.21.15.131:0/467410738 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f8644084790 con 0x7f86540b0160 2023-12-26T21:52:25.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.788+0000 7f86527fc700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f8640066f30 0x7f86400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:25.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.788+0000 7f86527fc700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f8640066f30 0x7f86400693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f86480091e0 tx=0x7f8648017000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:25.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.792+0000 7f863affd700 1 -- 172.21.15.131:0/467410738 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8644051b30 con 0x7f86540b0160 2023-12-26T21:52:25.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.989+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f86540657e0 con 0x7f86540b0160 2023-12-26T21:52:25.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.990+0000 7f863affd700 1 -- 172.21.15.131:0/467410738 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8644055150 con 0x7f86540b0160 2023-12-26T21:52:25.990 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:25.991 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:52:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T21:52:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T21:52:25.991 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:25.991 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:52:25.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f8640066f30 msgr2=0x7f86400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f8640066f30 0x7f86400693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f86480091e0 tx=0x7f8648017000 comp rx=0 tx=0).stop 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86540b0160 msgr2=0x7f865416c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86540b0160 0x7f865416c7c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f864400ec80 tx=0x7f86440091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 shutdown_connections 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8654159a70 0x7f86540cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86540b0160 0x7f865416c7c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f8640066f30 0x7f86400693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 --2- 172.21.15.131:0/467410738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86541596a0 0x7f86540c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 >> 172.21.15.131:0/467410738 conn(0x7f865406ece0 msgr2=0x7f86541572a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 shutdown_connections 2023-12-26T21:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:25.992+0000 7f86595ff700 1 -- 172.21.15.131:0/467410738 wait complete. 2023-12-26T21:52:26.010 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T21:52:26.010 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T21:52:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.202+0000 7fda02bff700 1 Processor -- start 2023-12-26T21:52:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.202+0000 7fda02bff700 1 -- start start 2023-12-26T21:52:26.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.203+0000 7fda02bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd9fc159810 0x7fd9fc0c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:26.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.203+0000 7fda02bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc0cbb00 con 0x7fd9fc0cd7b0 2023-12-26T21:52:26.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.203+0000 7fda02bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc0ce040 con 0x7fd9fc0cec80 2023-12-26T21:52:26.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.203+0000 7fda02bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc0cb2a0 con 0x7fd9fc159810 2023-12-26T21:52:26.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.203+0000 7fda0099b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd9fc159810 0x7fd9fc0c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:26.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.203+0000 7fda0119c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd9fc0cd7b0 0x7fd9fc0cbf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46604/0 (socket says 172.21.15.131:46604) 2023-12-26T21:52:26.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.204+0000 7fda0119c700 1 -- 172.21.15.131:0/1857751848 learned_addr learned my addr 172.21.15.131:0/1857751848 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:26.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.204+0000 7fd9fb7fe700 1 -- 172.21.15.131:0/1857751848 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773320469 0 0) 0x7fd9fc0cbb00 con 0x7fd9fc0cd7b0 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.204+0000 7fda0099b700 1 -- 172.21.15.131:0/1857751848 >> v1:172.21.15.190:6789/0 conn(0x7fd9fc0cec80 legacy=0x7fd9fc0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.204+0000 7fda0099b700 1 -- 172.21.15.131:0/1857751848 >> v1:172.21.15.131:6789/0 conn(0x7fd9fc0cd7b0 legacy=0x7fd9fc0cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.204+0000 7fda0099b700 1 -- 172.21.15.131:0/1857751848 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc159be0 con 0x7fd9fc159810 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.204+0000 7fda0099b700 1 --2- 172.21.15.131:0/1857751848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 0x7fd9fc0c5440 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00098d0 tx=0x7fd9f0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fe5bf76c96fad4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.204+0000 7fd9fb7fe700 1 -- 172.21.15.131:0/1857751848 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0009e70 con 0x7fd9fc159810 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.204+0000 7fd9fb7fe700 1 -- 172.21.15.131:0/1857751848 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f0005c10 con 0x7fd9fc159810 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.205+0000 7fd9fb7fe700 1 -- 172.21.15.131:0/1857751848 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000b650 con 0x7fd9fc159810 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.205+0000 7fda02bff700 1 -- 172.21.15.131:0/1857751848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 msgr2=0x7fd9fc0c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.205+0000 7fda02bff700 1 --2- 172.21.15.131:0/1857751848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 0x7fd9fc0c5440 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00098d0 tx=0x7fd9f0009120 comp rx=0 tx=0).stop 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.205+0000 7fda02bff700 1 -- 172.21.15.131:0/1857751848 shutdown_connections 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.205+0000 7fda02bff700 1 --2- 172.21.15.131:0/1857751848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 0x7fd9fc0c5440 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.205+0000 7fda02bff700 1 -- 172.21.15.131:0/1857751848 >> 172.21.15.131:0/1857751848 conn(0x7fd9fc099950 msgr2=0x7fd9fc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.205+0000 7fda02bff700 1 -- 172.21.15.131:0/1857751848 shutdown_connections 2023-12-26T21:52:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.205+0000 7fda02bff700 1 -- 172.21.15.131:0/1857751848 wait complete. 2023-12-26T21:52:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.207+0000 7fda02bff700 1 Processor -- start 2023-12-26T21:52:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.207+0000 7fda02bff700 1 -- start start 2023-12-26T21:52:26.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.207+0000 7fda02bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9fc0cd7b0 0x7fd9fc0b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:26.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda02bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9fc0cec80 0x7fd9fc069a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:26.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda02bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 0x7fd9fc15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:26.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda02bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc16a9d0 con 0x7fd9fc159810 2023-12-26T21:52:26.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda02bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc169e50 con 0x7fd9fc0cec80 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda02bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc16a130 con 0x7fd9fc0cd7b0 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0119c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 0x7fd9fc15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0119c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 0x7fd9fc15a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33638/0 (socket says 172.21.15.131:33638) 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0119c700 1 -- 172.21.15.131:0/285242662 learned_addr learned my addr 172.21.15.131:0/285242662 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0099b700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9fc0cd7b0 0x7fd9fc0b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0119c700 1 -- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9fc0cd7b0 msgr2=0x7fd9fc0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fd9fbfff700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9fc0cec80 0x7fd9fc069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0119c700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9fc0cd7b0 0x7fd9fc0b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0119c700 1 -- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9fc0cec80 msgr2=0x7fd9fc069a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0119c700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9fc0cec80 0x7fd9fc069a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0119c700 1 -- 172.21.15.131:0/285242662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f00093c0 con 0x7fd9fc159810 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0099b700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9fc0cd7b0 0x7fd9fc0b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fd9fbfff700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9fc0cec80 0x7fd9fc069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.208+0000 7fda0119c700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 0x7fd9fc15a2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400cbf0 tx=0x7fd9f40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.209+0000 7fd9f8ff9700 1 -- 172.21.15.131:0/285242662 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400a9c0 con 0x7fd9fc159810 2023-12-26T21:52:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.209+0000 7fd9f8ff9700 1 -- 172.21.15.131:0/285242662 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f400ab40 con 0x7fd9fc159810 2023-12-26T21:52:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.209+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc16a410 con 0x7fd9fc159810 2023-12-26T21:52:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.209+0000 7fd9f8ff9700 1 -- 172.21.15.131:0/285242662 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40070f0 con 0x7fd9fc159810 2023-12-26T21:52:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.209+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc0b39a0 con 0x7fd9fc159810 2023-12-26T21:52:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.210+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc0717f0 con 0x7fd9fc159810 2023-12-26T21:52:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.211+0000 7fd9f8ff9700 1 -- 172.21.15.131:0/285242662 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd9f4007520 con 0x7fd9fc159810 2023-12-26T21:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.211+0000 7fd9f8ff9700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd9e0066f30 0x7fd9e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.212+0000 7fd9f8ff9700 1 -- 172.21.15.131:0/285242662 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fd9f4084730 con 0x7fd9fc159810 2023-12-26T21:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.212+0000 7fda0099b700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd9e0066f30 0x7fd9e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.212+0000 7fda0099b700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd9e0066f30 0x7fd9e00693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc156fe0 tx=0x7fd9f0016000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:26.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.216+0000 7fd9f8ff9700 1 -- 172.21.15.131:0/285242662 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9f4051b50 con 0x7fd9fc159810 2023-12-26T21:52:26.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:26.410+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd9fc0cae20 con 0x7fd9fc159810 2023-12-26T21:52:27.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.236+0000 7fd9f8ff9700 1 -- 172.21.15.131:0/285242662 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd9f4055170 con 0x7fd9fc159810 2023-12-26T21:52:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.242+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd9e0066f30 msgr2=0x7fd9e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.242+0000 7fda02bff700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd9e0066f30 0x7fd9e00693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc156fe0 tx=0x7fd9f0016000 comp rx=0 tx=0).stop 2023-12-26T21:52:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.242+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 msgr2=0x7fd9fc15a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.242+0000 7fda02bff700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 0x7fd9fc15a2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400cbf0 tx=0x7fd9f40019b0 comp rx=0 tx=0).stop 2023-12-26T21:52:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.243+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 shutdown_connections 2023-12-26T21:52:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.243+0000 7fda02bff700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9fc159810 0x7fd9fc15a2f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.243+0000 7fda02bff700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9fc0cd7b0 0x7fd9fc0b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.243+0000 7fda02bff700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fd9e0066f30 0x7fd9e00693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.243+0000 7fda02bff700 1 --2- 172.21.15.131:0/285242662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9fc0cec80 0x7fd9fc069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.243+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 >> 172.21.15.131:0/285242662 conn(0x7fd9fc099950 msgr2=0x7fd9fc068650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.244+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 shutdown_connections 2023-12-26T21:52:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.244+0000 7fda02bff700 1 -- 172.21.15.131:0/285242662 wait complete. 2023-12-26T21:52:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T21:52:27.266 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T21:52:27.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.465+0000 7f5b0ddff700 1 Processor -- start 2023-12-26T21:52:27.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.465+0000 7f5b0ddff700 1 -- start start 2023-12-26T21:52:27.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.466+0000 7f5b0ddff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5b080adf60 0x7f5b080ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:27.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.466+0000 7f5b0ddff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080c6b30 con 0x7f5b080ac580 2023-12-26T21:52:27.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.466+0000 7f5b0ddff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080c6e10 con 0x7f5b08069620 2023-12-26T21:52:27.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.466+0000 7f5b0ddff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b081561e0 con 0x7f5b080adf60 2023-12-26T21:52:27.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.466+0000 7f5b077fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5b080adf60 0x7f5b080ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:27.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.467+0000 7f5b06ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5b080ac580 0x7f5b0806e2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46608/0 (socket says 172.21.15.131:46608) 2023-12-26T21:52:27.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.467+0000 7f5b06ffd700 1 -- 172.21.15.131:0/72327479 learned_addr learned my addr 172.21.15.131:0/72327479 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:27.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.467+0000 7f5b067fc700 1 -- 172.21.15.131:0/72327479 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462976027 0 0) 0x7f5b080c6b30 con 0x7f5b080ac580 2023-12-26T21:52:27.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.467+0000 7f5b067fc700 1 -- 172.21.15.131:0/72327479 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af0004680 con 0x7f5b080ac580 2023-12-26T21:52:27.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.467+0000 7f5b077fe700 1 -- 172.21.15.131:0/72327479 >> v1:172.21.15.190:6789/0 conn(0x7f5b08069620 legacy=0x7f5b080c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:27.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.467+0000 7f5b077fe700 1 -- 172.21.15.131:0/72327479 >> v1:172.21.15.131:6789/0 conn(0x7f5b080ac580 legacy=0x7f5b0806e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:27.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.467+0000 7f5b077fe700 1 -- 172.21.15.131:0/72327479 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b080c70f0 con 0x7f5b080adf60 2023-12-26T21:52:27.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.467+0000 7f5b077fe700 1 --2- 172.21.15.131:0/72327479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080adf60 0x7f5b080ac040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009910 tx=0x7f5af80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38c242d7d097932 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:27.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b067fc700 1 -- 172.21.15.131:0/72327479 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af800b640 con 0x7f5b080adf60 2023-12-26T21:52:27.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b067fc700 1 -- 172.21.15.131:0/72327479 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af800b7a0 con 0x7f5b080adf60 2023-12-26T21:52:27.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b067fc700 1 -- 172.21.15.131:0/72327479 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af800bb60 con 0x7f5b080adf60 2023-12-26T21:52:27.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b0ddff700 1 -- 172.21.15.131:0/72327479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080adf60 msgr2=0x7f5b080ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:27.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b0ddff700 1 --2- 172.21.15.131:0/72327479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080adf60 0x7f5b080ac040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009910 tx=0x7f5af80091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:27.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b0ddff700 1 -- 172.21.15.131:0/72327479 shutdown_connections 2023-12-26T21:52:27.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b0ddff700 1 --2- 172.21.15.131:0/72327479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080adf60 0x7f5b080ac040 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:27.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b0ddff700 1 -- 172.21.15.131:0/72327479 >> 172.21.15.131:0/72327479 conn(0x7f5b08068040 msgr2=0x7f5b080cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:27.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b0ddff700 1 -- 172.21.15.131:0/72327479 shutdown_connections 2023-12-26T21:52:27.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.468+0000 7f5b0ddff700 1 -- 172.21.15.131:0/72327479 wait complete. 2023-12-26T21:52:27.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.470+0000 7f5b0ddff700 1 Processor -- start 2023-12-26T21:52:27.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.470+0000 7f5b0ddff700 1 -- start start 2023-12-26T21:52:27.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b0ddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b08069620 0x7f5b081557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b0ddff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080ac580 0x7f5b080af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b0ddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b080adf60 0x7f5b0815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b0ddff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b08155d20 con 0x7f5b080ac580 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b0ddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080b16d0 con 0x7f5b08069620 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b0ddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080b07e0 con 0x7f5b080adf60 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b06ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080ac580 0x7f5b080af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b06ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080ac580 0x7f5b080af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33646/0 (socket says 172.21.15.131:33646) 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b06ffd700 1 -- 172.21.15.131:0/3354667649 learned_addr learned my addr 172.21.15.131:0/3354667649 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b06ffd700 1 -- 172.21.15.131:0/3354667649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b080adf60 msgr2=0x7f5b0815bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b06ffd700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b080adf60 0x7f5b0815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:27.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b06ffd700 1 -- 172.21.15.131:0/3354667649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b08069620 msgr2=0x7f5b081557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:27.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b06ffd700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b08069620 0x7f5b081557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:27.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.471+0000 7f5b06ffd700 1 -- 172.21.15.131:0/3354667649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af8009420 con 0x7f5b080ac580 2023-12-26T21:52:27.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.472+0000 7f5b06ffd700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080ac580 0x7f5b080af920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c7c0 tx=0x7f5afc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:27.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.472+0000 7f5aef7fe700 1 -- 172.21.15.131:0/3354667649 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc005b30 con 0x7f5b080ac580 2023-12-26T21:52:27.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.472+0000 7f5aef7fe700 1 -- 172.21.15.131:0/3354667649 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc005c90 con 0x7f5b080ac580 2023-12-26T21:52:27.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.472+0000 7f5aef7fe700 1 -- 172.21.15.131:0/3354667649 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc007340 con 0x7f5b080ac580 2023-12-26T21:52:27.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.472+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b080b0ac0 con 0x7f5b080ac580 2023-12-26T21:52:27.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.472+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b080b1080 con 0x7f5b080ac580 2023-12-26T21:52:27.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.473+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b080657e0 con 0x7f5b080ac580 2023-12-26T21:52:27.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.474+0000 7f5aef7fe700 1 -- 172.21.15.131:0/3354667649 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5afc01b400 con 0x7f5b080ac580 2023-12-26T21:52:27.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.475+0000 7f5aef7fe700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f5af4066f30 0x7f5af40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:27.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.475+0000 7f5aef7fe700 1 -- 172.21.15.131:0/3354667649 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5615+0+0 (secure 0 0 0) 0x7f5afc083f60 con 0x7f5b080ac580 2023-12-26T21:52:27.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.475+0000 7f5b077fe700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f5af4066f30 0x7f5af40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:27.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.476+0000 7f5b077fe700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f5af4066f30 0x7f5af40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5af80091e0 tx=0x7f5af8017000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:27.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.479+0000 7f5aef7fe700 1 -- 172.21.15.131:0/3354667649 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5afc0512b0 con 0x7f5b080ac580 2023-12-26T21:52:27.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:27.670+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5b080b1840 con 0x7f5b080ac580 2023-12-26T21:52:28.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.243+0000 7f5aef7fe700 1 -- 172.21.15.131:0/3354667649 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5afc0548d0 con 0x7f5b080ac580 2023-12-26T21:52:28.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.247+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f5af4066f30 msgr2=0x7f5af40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:28.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.247+0000 7f5b0ddff700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f5af4066f30 0x7f5af40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5af80091e0 tx=0x7f5af8017000 comp rx=0 tx=0).stop 2023-12-26T21:52:28.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.247+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080ac580 msgr2=0x7f5b080af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:28.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.247+0000 7f5b0ddff700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080ac580 0x7f5b080af920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c7c0 tx=0x7f5afc007e00 comp rx=0 tx=0).stop 2023-12-26T21:52:28.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.247+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 shutdown_connections 2023-12-26T21:52:28.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.247+0000 7f5b0ddff700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b080adf60 0x7f5b0815bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:28.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.247+0000 7f5b0ddff700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f5af4066f30 0x7f5af40693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:28.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.247+0000 7f5b0ddff700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b08069620 0x7f5b081557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:28.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.247+0000 7f5b0ddff700 1 --2- 172.21.15.131:0/3354667649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b080ac580 0x7f5b080af920 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:28.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.248+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 >> 172.21.15.131:0/3354667649 conn(0x7f5b08068040 msgr2=0x7f5b080cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:28.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.248+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 shutdown_connections 2023-12-26T21:52:28.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.248+0000 7f5b0ddff700 1 -- 172.21.15.131:0/3354667649 wait complete. 2023-12-26T21:52:28.249 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T21:52:28.279 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T21:52:28.346 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-12-26T21:52:28.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.471+0000 7fb7387ff700 1 Processor -- start 2023-12-26T21:52:28.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.471+0000 7fb7387ff700 1 -- start start 2023-12-26T21:52:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.472+0000 7fb7387ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb7300cb320 0x7fb730069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.472+0000 7fb7387ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb73006a1e0 0x7fb7300b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.472+0000 7fb7387ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb730156940 con 0x7fb7300b0c90 2023-12-26T21:52:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb7387ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300c8cc0 con 0x7fb7300cb320 2023-12-26T21:52:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb7387ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300ad700 con 0x7fb73006a1e0 2023-12-26T21:52:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb73659b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb7300cb320 0x7fb730069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb73659b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb7300cb320 0x7fb730069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33662/0 (socket says 172.21.15.131:33662) 2023-12-26T21:52:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb73659b700 1 -- 172.21.15.131:0/4036313397 learned_addr learned my addr 172.21.15.131:0/4036313397 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb735d9a700 1 --2- 172.21.15.131:0/4036313397 >> v2:172.21.15.190:3300/0 conn(0x7fb73006a1e0 0x7fb7300b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb73659b700 1 -- 172.21.15.131:0/4036313397 >> v2:172.21.15.190:3300/0 conn(0x7fb73006a1e0 msgr2=0x7fb7300b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb73659b700 1 --2- 172.21.15.131:0/4036313397 >> v2:172.21.15.190:3300/0 conn(0x7fb73006a1e0 0x7fb7300b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb73659b700 1 -- 172.21.15.131:0/4036313397 >> v1:172.21.15.190:6789/0 conn(0x7fb7300b0c90 legacy=0x7fb7300aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.473+0000 7fb73659b700 1 -- 172.21.15.131:0/4036313397 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300ae500 con 0x7fb7300cb320 2023-12-26T21:52:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb73659b700 1 --2- 172.21.15.131:0/4036313397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300cb320 0x7fb730069ca0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb730064700 tx=0x7fb72c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5386abe21255d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb735599700 1 -- 172.21.15.131:0/4036313397 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00b7e0 con 0x7fb7300cb320 2023-12-26T21:52:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb735599700 1 -- 172.21.15.131:0/4036313397 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c00b940 con 0x7fb7300cb320 2023-12-26T21:52:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb735599700 1 -- 172.21.15.131:0/4036313397 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00bce0 con 0x7fb7300cb320 2023-12-26T21:52:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb7387ff700 1 -- 172.21.15.131:0/4036313397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300cb320 msgr2=0x7fb730069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb7387ff700 1 --2- 172.21.15.131:0/4036313397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300cb320 0x7fb730069ca0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb730064700 tx=0x7fb72c009250 comp rx=0 tx=0).stop 2023-12-26T21:52:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb7387ff700 1 -- 172.21.15.131:0/4036313397 shutdown_connections 2023-12-26T21:52:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb7387ff700 1 --2- 172.21.15.131:0/4036313397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300cb320 0x7fb730069ca0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb7387ff700 1 --2- 172.21.15.131:0/4036313397 >> v2:172.21.15.190:3300/0 conn(0x7fb73006a1e0 0x7fb7300b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb7387ff700 1 -- 172.21.15.131:0/4036313397 >> 172.21.15.131:0/4036313397 conn(0x7fb730068040 msgr2=0x7fb730069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb7387ff700 1 -- 172.21.15.131:0/4036313397 shutdown_connections 2023-12-26T21:52:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.474+0000 7fb7387ff700 1 -- 172.21.15.131:0/4036313397 wait complete. 2023-12-26T21:52:28.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.476+0000 7fb7387ff700 1 Processor -- start 2023-12-26T21:52:28.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.476+0000 7fb7387ff700 1 -- start start 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb7387ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb73006a1e0 0x7fb7301533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb7387ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300b0c90 0x7fb73014a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb7387ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7300cb320 0x7fb73016aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb7387ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb730150660 con 0x7fb7300b0c90 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb7387ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7301507c0 con 0x7fb7300cb320 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb7387ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73015b340 con 0x7fb73006a1e0 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb735d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300b0c90 0x7fb73014a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb735d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300b0c90 0x7fb73014a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33676/0 (socket says 172.21.15.131:33676) 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb735d9a700 1 -- 172.21.15.131:0/567495841 learned_addr learned my addr 172.21.15.131:0/567495841 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb736d9c700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7300cb320 0x7fb73016aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb735d9a700 1 -- 172.21.15.131:0/567495841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb73006a1e0 msgr2=0x7fb7301533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb735d9a700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb73006a1e0 0x7fb7301533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:28.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb735d9a700 1 -- 172.21.15.131:0/567495841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7300cb320 msgr2=0x7fb73016aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.477+0000 7fb735d9a700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7300cb320 0x7fb73016aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.478+0000 7fb735d9a700 1 -- 172.21.15.131:0/567495841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c0094f0 con 0x7fb7300b0c90 2023-12-26T21:52:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.478+0000 7fb736d9c700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7300cb320 0x7fb73016aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.478+0000 7fb735d9a700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300b0c90 0x7fb73014a2e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb72000d890 tx=0x7fb720008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.478+0000 7fb7267fc700 1 -- 172.21.15.131:0/567495841 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72000b600 con 0x7fb7300b0c90 2023-12-26T21:52:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.478+0000 7fb7267fc700 1 -- 172.21.15.131:0/567495841 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72000b760 con 0x7fb7300b0c90 2023-12-26T21:52:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.478+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7301500a0 con 0x7fb7300b0c90 2023-12-26T21:52:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.478+0000 7fb7267fc700 1 -- 172.21.15.131:0/567495841 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72000bb00 con 0x7fb7300b0c90 2023-12-26T21:52:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.478+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb730153930 con 0x7fb7300b0c90 2023-12-26T21:52:28.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.479+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73015b110 con 0x7fb7300b0c90 2023-12-26T21:52:28.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.480+0000 7fb7267fc700 1 -- 172.21.15.131:0/567495841 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb72001e460 con 0x7fb7300b0c90 2023-12-26T21:52:28.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.481+0000 7fb7267fc700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fb714066f30 0x7fb7140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:28.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.482+0000 7fb7267fc700 1 -- 172.21.15.131:0/567495841 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fb720084d80 con 0x7fb7300b0c90 2023-12-26T21:52:28.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.482+0000 7fb73659b700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fb714066f30 0x7fb7140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:28.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.482+0000 7fb73659b700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fb714066f30 0x7fb7140693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb730064700 tx=0x7fb72c016000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:28.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.486+0000 7fb7267fc700 1 -- 172.21.15.131:0/567495841 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb73015b110 con 0x7fb7300b0c90 2023-12-26T21:52:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:28.724+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb73015b480 con 0x7fb7300b0c90 2023-12-26T21:52:29.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.255+0000 7fb7267fc700 1 -- 172.21.15.131:0/567495841 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb73015b480 con 0x7fb7300b0c90 2023-12-26T21:52:29.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fb714066f30 msgr2=0x7fb7140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fb714066f30 0x7fb7140693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb730064700 tx=0x7fb72c016000 comp rx=0 tx=0).stop 2023-12-26T21:52:29.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300b0c90 msgr2=0x7fb73014a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300b0c90 0x7fb73014a2e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb72000d890 tx=0x7fb720008040 comp rx=0 tx=0).stop 2023-12-26T21:52:29.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 shutdown_connections 2023-12-26T21:52:29.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fb714066f30 0x7fb7140693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7300cb320 0x7fb73016aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb73006a1e0 0x7fb7301533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 --2- 172.21.15.131:0/567495841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7300b0c90 0x7fb73014a2e0 secure :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb72000d890 tx=0x7fb720008040 comp rx=0 tx=0).stop 2023-12-26T21:52:29.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.260+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 >> 172.21.15.131:0/567495841 conn(0x7fb730068040 msgr2=0x7fb73014c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:29.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.261+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 shutdown_connections 2023-12-26T21:52:29.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.262+0000 7fb7387ff700 1 -- 172.21.15.131:0/567495841 wait complete. 2023-12-26T21:52:29.265 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 8 and data pool 9 2023-12-26T21:52:29.282 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:29.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.473+0000 7f53e89ff700 1 Processor -- start 2023-12-26T21:52:29.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.473+0000 7f53e89ff700 1 -- start start 2023-12-26T21:52:29.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.474+0000 7f53e89ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e41596a0 0x7f53e40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.474+0000 7f53e89ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e40afcd0 con 0x7f53e40b0870 2023-12-26T21:52:29.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.474+0000 7f53e89ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e40b3e70 con 0x7f53e4159a70 2023-12-26T21:52:29.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.474+0000 7f53e89ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b0cd0 con 0x7f53e41596a0 2023-12-26T21:52:29.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e2d9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f53e40b0870 0x7f53e40b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46624/0 (socket says 172.21.15.131:46624) 2023-12-26T21:52:29.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e2d9d700 1 -- 172.21.15.131:0/2571744131 learned_addr learned my addr 172.21.15.131:0/2571744131 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e259c700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e41596a0 0x7f53e40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443584052 0 0) 0x7f53e40afcd0 con 0x7f53e40b0870 2023-12-26T21:52:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d0003570 con 0x7f53e40b0870 2023-12-26T21:52:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1019902984 0 0) 0x7f53d0003570 con 0x7f53e40b0870 2023-12-26T21:52:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e41596a0 msgr2=0x7f53e40b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e1d9b700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e41596a0 0x7f53e40b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 >> v1:172.21.15.190:6789/0 conn(0x7f53e4159a70 legacy=0x7f53e40b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40b0f20 con 0x7f53e40b0870 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.475+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f53dc0017f0 con 0x7f53e40b0870 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 >> v1:172.21.15.131:6789/0 conn(0x7f53e40b0870 legacy=0x7f53e40b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e1d9b700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e41596a0 0x7f53d0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e1d9b700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d0003fd0 0x7f53d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e1d9b700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53d0004d50 0x7f53d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b0cd0 con 0x7f53d0004d50 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d000bb40 con 0x7f53d0003fd0 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d000bd00 con 0x7f53e41596a0 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e2d9d700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53d0004d50 0x7f53d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e259c700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e41596a0 0x7f53d0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53dbfff700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d0003fd0 0x7f53d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e2d9d700 1 -- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e41596a0 msgr2=0x7f53d0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e2d9d700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e41596a0 0x7f53d0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e2d9d700 1 -- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d0003fd0 msgr2=0x7f53d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e2d9d700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d0003fd0 0x7f53d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.476+0000 7f53e2d9d700 1 -- 172.21.15.131:0/2571744131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0003570 con 0x7f53d0004d50 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e2d9d700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53d0004d50 0x7f53d00051c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00a9a0 tx=0x7f53e40b0e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae53b59b3a3aa43 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0070f0 con 0x7f53d0004d50 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc007250 con 0x7f53d0004d50 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e1d9b700 1 -- 172.21.15.131:0/2571744131 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc007610 con 0x7f53d0004d50 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e89ff700 1 -- 172.21.15.131:0/2571744131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53d0004d50 msgr2=0x7f53d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e89ff700 1 --2- 172.21.15.131:0/2571744131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53d0004d50 0x7f53d00051c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00a9a0 tx=0x7f53e40b0e10 comp rx=0 tx=0).stop 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e2d9d700 1 -- 172.21.15.131:0/2571744131 reap_dead start 2023-12-26T21:52:29.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e89ff700 1 -- 172.21.15.131:0/2571744131 shutdown_connections 2023-12-26T21:52:29.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e89ff700 1 -- 172.21.15.131:0/2571744131 >> 172.21.15.131:0/2571744131 conn(0x7f53e406ece0 msgr2=0x7f53e406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:29.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e89ff700 1 -- 172.21.15.131:0/2571744131 shutdown_connections 2023-12-26T21:52:29.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.477+0000 7f53e89ff700 1 -- 172.21.15.131:0/2571744131 wait complete. 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.479+0000 7f53e89ff700 1 Processor -- start 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.479+0000 7f53e89ff700 1 -- start start 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e89ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d0004d50 0x7f53e4152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e89ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4158150 0x7f53e41585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e89ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e415b3f0 0x7f53e415b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e89ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e415c6e0 con 0x7f53e415b3f0 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e89ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4158b00 con 0x7f53e4158150 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e89ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40c4520 con 0x7f53d0004d50 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e2d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e415b3f0 0x7f53e415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e2d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e415b3f0 0x7f53e415b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33686/0 (socket says 172.21.15.131:33686) 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e2d9d700 1 -- 172.21.15.131:0/3880989144 learned_addr learned my addr 172.21.15.131:0/3880989144 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53dbfff700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4158150 0x7f53e41585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e259c700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d0004d50 0x7f53e4152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.480+0000 7f53e2d9d700 1 -- 172.21.15.131:0/3880989144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d0004d50 msgr2=0x7f53e4152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53e2d9d700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d0004d50 0x7f53e4152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53e2d9d700 1 -- 172.21.15.131:0/3880989144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4158150 msgr2=0x7f53e41585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53e2d9d700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4158150 0x7f53e41585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53e2d9d700 1 -- 172.21.15.131:0/3880989144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e415c120 con 0x7f53e415b3f0 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53dbfff700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4158150 0x7f53e41585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53e2d9d700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e415b3f0 0x7f53e415b860 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f53dc012000 tx=0x7f53dc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53da7fc700 1 -- 172.21.15.131:0/3880989144 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc01f440 con 0x7f53e415b3f0 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53da7fc700 1 -- 172.21.15.131:0/3880989144 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc01f5a0 con 0x7f53e415b3f0 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e415c400 con 0x7f53e415b3f0 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53da7fc700 1 -- 172.21.15.131:0/3880989144 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc01f920 con 0x7f53e415b3f0 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.481+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e4153020 con 0x7f53e415b3f0 2023-12-26T21:52:29.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.482+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e40c4010 con 0x7f53e415b3f0 2023-12-26T21:52:29.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.484+0000 7f53da7fc700 1 -- 172.21.15.131:0/3880989144 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f53dc01fc90 con 0x7f53e415b3f0 2023-12-26T21:52:29.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.484+0000 7f53da7fc700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c4066f30 0x7f53c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.484+0000 7f53da7fc700 1 -- 172.21.15.131:0/3880989144 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f53dc087160 con 0x7f53e415b3f0 2023-12-26T21:52:29.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.485+0000 7f53e259c700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c4066f30 0x7f53c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.485+0000 7f53e259c700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c4066f30 0x7f53c40693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53d4006f00 tx=0x7f53d4008040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:29.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.489+0000 7f53da7fc700 1 -- 172.21.15.131:0/3880989144 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53e40c4010 con 0x7f53e415b3f0 2023-12-26T21:52:29.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.685+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53e40c4010 con 0x7f53e415b3f0 2023-12-26T21:52:29.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.686+0000 7f53da7fc700 1 -- 172.21.15.131:0/3880989144 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16007 (secure 0 0 0) 0x7f53e40c4010 con 0x7f53e415b3f0 2023-12-26T21:52:29.686 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:29.686 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:28.726357+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:52:26.449943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-26T21:52:27.710394+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c4066f30 msgr2=0x7f53c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c4066f30 0x7f53c40693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53d4006f00 tx=0x7f53d4008040 comp rx=0 tx=0).stop 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e415b3f0 msgr2=0x7f53e415b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e415b3f0 0x7f53e415b860 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f53dc012000 tx=0x7f53dc013000 comp rx=0 tx=0).stop 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 shutdown_connections 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e415b3f0 0x7f53e415b860 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d0004d50 0x7f53e4152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c4066f30 0x7f53c40693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 --2- 172.21.15.131:0/3880989144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4158150 0x7f53e41585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 >> 172.21.15.131:0/3880989144 conn(0x7f53e406ece0 msgr2=0x7f53e40b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 shutdown_connections 2023-12-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.688+0000 7f53e89ff700 1 -- 172.21.15.131:0/3880989144 wait complete. 2023-12-26T21:52:29.707 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:29.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.896+0000 7f08d41ff700 1 Processor -- start 2023-12-26T21:52:29.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.896+0000 7f08d41ff700 1 -- start start 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.897+0000 7f08d41ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08cc13c510 0x7f08cc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.897+0000 7f08d41ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f08cc06b8c0 0x7f08cc06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.897+0000 7f08d41ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08cc0686c0 con 0x7f08cc0cc100 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.897+0000 7f08d41ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08cc06c280 con 0x7f08cc06b8c0 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.897+0000 7f08d41ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08cc069520 con 0x7f08cc13c510 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.897+0000 7f08d179a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f08cc06b8c0 0x7f08cc06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.897+0000 7f08d279c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f08cc0cc100 0x7f08cc0cc520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46638/0 (socket says 172.21.15.131:46638) 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.897+0000 7f08d279c700 1 -- 172.21.15.131:0/3342295094 learned_addr learned my addr 172.21.15.131:0/3342295094 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d1f9b700 1 --2- 172.21.15.131:0/3342295094 >> v2:172.21.15.190:3300/0 conn(0x7f08cc13c510 0x7f08cc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d179a700 1 -- 172.21.15.131:0/3342295094 >> v2:172.21.15.190:3300/0 conn(0x7f08cc13c510 msgr2=0x7f08cc0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:29.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d179a700 1 --2- 172.21.15.131:0/3342295094 >> v2:172.21.15.190:3300/0 conn(0x7f08cc13c510 0x7f08cc0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d179a700 1 -- 172.21.15.131:0/3342295094 >> v1:172.21.15.131:6789/0 conn(0x7f08cc0cc100 legacy=0x7f08cc0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d179a700 1 -- 172.21.15.131:0/3342295094 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc068910 con 0x7f08cc06b8c0 2023-12-26T21:52:29.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d0f99700 1 -- 172.21.15.131:0/3342295094 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034214659 0 0) 0x7f08cc0686c0 con 0x7f08cc0cc100 2023-12-26T21:52:29.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d179a700 1 --2- 172.21.15.131:0/3342295094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc06b8c0 0x7f08cc06bd40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f08cc06fcc0 tx=0x7f08bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedaf10ad4a5b2da server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:29.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d0f99700 1 -- 172.21.15.131:0/3342295094 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc011630 con 0x7f08cc06b8c0 2023-12-26T21:52:29.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d0f99700 1 -- 172.21.15.131:0/3342295094 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08bc011790 con 0x7f08cc06b8c0 2023-12-26T21:52:29.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.898+0000 7f08d0f99700 1 -- 172.21.15.131:0/3342295094 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc011b30 con 0x7f08cc06b8c0 2023-12-26T21:52:29.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.899+0000 7f08d41ff700 1 -- 172.21.15.131:0/3342295094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc06b8c0 msgr2=0x7f08cc06bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.899+0000 7f08d41ff700 1 --2- 172.21.15.131:0/3342295094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc06b8c0 0x7f08cc06bd40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f08cc06fcc0 tx=0x7f08bc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.899+0000 7f08d41ff700 1 -- 172.21.15.131:0/3342295094 shutdown_connections 2023-12-26T21:52:29.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.899+0000 7f08d41ff700 1 --2- 172.21.15.131:0/3342295094 >> v2:172.21.15.190:3300/0 conn(0x7f08cc13c510 0x7f08cc0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.899+0000 7f08d41ff700 1 --2- 172.21.15.131:0/3342295094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc06b8c0 0x7f08cc06bd40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.899+0000 7f08d41ff700 1 -- 172.21.15.131:0/3342295094 >> 172.21.15.131:0/3342295094 conn(0x7f08cc0ce3e0 msgr2=0x7f08cc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.899+0000 7f08d41ff700 1 -- 172.21.15.131:0/3342295094 shutdown_connections 2023-12-26T21:52:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.899+0000 7f08d41ff700 1 -- 172.21.15.131:0/3342295094 wait complete. 2023-12-26T21:52:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.900+0000 7f08d41ff700 1 Processor -- start 2023-12-26T21:52:29.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.901+0000 7f08d41ff700 1 -- start start 2023-12-26T21:52:29.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.901+0000 7f08d41ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08cc06b8c0 0x7f08cc0b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.901+0000 7f08d41ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08cc0cc100 0x7f08cc1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.901+0000 7f08d41ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc13c510 0x7f08cc158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.901+0000 7f08d41ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc16b1c0 con 0x7f08cc13c510 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.901+0000 7f08d41ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc16df60 con 0x7f08cc06b8c0 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d41ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc16aee0 con 0x7f08cc0cc100 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d279c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc13c510 0x7f08cc158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d279c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc13c510 0x7f08cc158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33700/0 (socket says 172.21.15.131:33700) 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d279c700 1 -- 172.21.15.131:0/1850139245 learned_addr learned my addr 172.21.15.131:0/1850139245 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d1f9b700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08cc06b8c0 0x7f08cc0b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d179a700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08cc0cc100 0x7f08cc1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d279c700 1 -- 172.21.15.131:0/1850139245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08cc0cc100 msgr2=0x7f08cc1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d279c700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08cc0cc100 0x7f08cc1581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d279c700 1 -- 172.21.15.131:0/1850139245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08cc06b8c0 msgr2=0x7f08cc0b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d279c700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08cc06b8c0 0x7f08cc0b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:29.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d279c700 1 -- 172.21.15.131:0/1850139245 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc009420 con 0x7f08cc13c510 2023-12-26T21:52:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d179a700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08cc0cc100 0x7f08cc1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.902+0000 7f08d279c700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc13c510 0x7f08cc158710 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f08c400cc90 tx=0x7f08c4001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.903+0000 7f08c1ffb700 1 -- 172.21.15.131:0/1850139245 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400a960 con 0x7f08cc13c510 2023-12-26T21:52:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.903+0000 7f08c1ffb700 1 -- 172.21.15.131:0/1850139245 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c400aac0 con 0x7f08cc13c510 2023-12-26T21:52:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.903+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc16d910 con 0x7f08cc13c510 2023-12-26T21:52:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.903+0000 7f08c1ffb700 1 -- 172.21.15.131:0/1850139245 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400ae80 con 0x7f08cc13c510 2023-12-26T21:52:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.903+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc0b2a20 con 0x7f08cc13c510 2023-12-26T21:52:29.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.904+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08cc16d770 con 0x7f08cc13c510 2023-12-26T21:52:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.905+0000 7f08c1ffb700 1 -- 172.21.15.131:0/1850139245 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f08c4007300 con 0x7f08cc13c510 2023-12-26T21:52:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.906+0000 7f08c1ffb700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f08b0066f30 0x7f08b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.906+0000 7f08c1ffb700 1 -- 172.21.15.131:0/1850139245 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f08c4084b40 con 0x7f08cc13c510 2023-12-26T21:52:29.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.906+0000 7f08d1f9b700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f08b0066f30 0x7f08b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:29.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.907+0000 7f08d1f9b700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f08b0066f30 0x7f08b00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009da0 tx=0x7f08c8009120 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:29.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:29.910+0000 7f08c1ffb700 1 -- 172.21.15.131:0/1850139245 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08cc16d770 con 0x7f08cc13c510 2023-12-26T21:52:30.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.105+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08cc16d770 con 0x7f08cc13c510 2023-12-26T21:52:30.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.106+0000 7f08c1ffb700 1 -- 172.21.15.131:0/1850139245 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16007 (secure 0 0 0) 0x7f08cc16d770 con 0x7f08cc13c510 2023-12-26T21:52:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:30.107 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:28.726357+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:52:26.449943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-26T21:52:27.710394+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:30.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f08b0066f30 msgr2=0x7f08b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:30.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f08b0066f30 0x7f08b00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009da0 tx=0x7f08c8009120 comp rx=0 tx=0).stop 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc13c510 msgr2=0x7f08cc158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc13c510 0x7f08cc158710 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f08c400cc90 tx=0x7f08c4001a80 comp rx=0 tx=0).stop 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 shutdown_connections 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08cc13c510 0x7f08cc158710 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f08b0066f30 0x7f08b00693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08cc06b8c0 0x7f08cc0b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 --2- 172.21.15.131:0/1850139245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08cc0cc100 0x7f08cc1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.108+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 >> 172.21.15.131:0/1850139245 conn(0x7f08cc0ce3e0 msgr2=0x7f08cc155b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.109+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 shutdown_connections 2023-12-26T21:52:30.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.109+0000 7f08d41ff700 1 -- 172.21.15.131:0/1850139245 wait complete. 2023-12-26T21:52:30.127 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T21:52:30.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.322+0000 7feff7bff700 1 Processor -- start 2023-12-26T21:52:30.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.322+0000 7feff7bff700 1 -- start start 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.323+0000 7feff7bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feff006e480 0x7feff00abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.323+0000 7feff7bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feff00aa480 0x7feff00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.323+0000 7feff7bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff015a8c0 con 0x7feff0159820 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.323+0000 7feff7bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff015ad50 con 0x7feff006e480 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.323+0000 7feff7bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff0159f90 con 0x7feff00aa480 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.323+0000 7feff599b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7feff0159820 0x7feff006dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46648/0 (socket says 172.21.15.131:46648) 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.323+0000 7feff599b700 1 -- 172.21.15.131:0/121145436 learned_addr learned my addr 172.21.15.131:0/121145436 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.323+0000 7feff519a700 1 --2- 172.21.15.131:0/121145436 >> v2:172.21.15.131:3300/0 conn(0x7feff006e480 0x7feff00abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff619c700 1 --2- 172.21.15.131:0/121145436 >> v2:172.21.15.190:3300/0 conn(0x7feff00aa480 0x7feff00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff4999700 1 -- 172.21.15.131:0/121145436 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666077990 0 0) 0x7feff015a8c0 con 0x7feff0159820 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff4999700 1 -- 172.21.15.131:0/121145436 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefdc003570 con 0x7feff0159820 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff519a700 1 -- 172.21.15.131:0/121145436 >> v2:172.21.15.190:3300/0 conn(0x7feff00aa480 msgr2=0x7feff00675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff519a700 1 --2- 172.21.15.131:0/121145436 >> v2:172.21.15.190:3300/0 conn(0x7feff00aa480 0x7feff00675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff519a700 1 -- 172.21.15.131:0/121145436 >> v1:172.21.15.131:6789/0 conn(0x7feff0159820 legacy=0x7feff006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff519a700 1 -- 172.21.15.131:0/121145436 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff015a270 con 0x7feff006e480 2023-12-26T21:52:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff619c700 1 --2- 172.21.15.131:0/121145436 >> v2:172.21.15.190:3300/0 conn(0x7feff00aa480 0x7feff00675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff519a700 1 --2- 172.21.15.131:0/121145436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff006e480 0x7feff00abdb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7feff006cbd0 tx=0x7fefe00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9390f0f0d3335b93 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff4999700 1 -- 172.21.15.131:0/121145436 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0011630 con 0x7feff006e480 2023-12-26T21:52:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff4999700 1 -- 172.21.15.131:0/121145436 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0011790 con 0x7feff006e480 2023-12-26T21:52:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.324+0000 7feff4999700 1 -- 172.21.15.131:0/121145436 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0011b30 con 0x7feff006e480 2023-12-26T21:52:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.325+0000 7feff7bff700 1 -- 172.21.15.131:0/121145436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff006e480 msgr2=0x7feff00abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.325+0000 7feff7bff700 1 --2- 172.21.15.131:0/121145436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff006e480 0x7feff00abdb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7feff006cbd0 tx=0x7fefe00091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.325+0000 7feff7bff700 1 -- 172.21.15.131:0/121145436 shutdown_connections 2023-12-26T21:52:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.325+0000 7feff7bff700 1 --2- 172.21.15.131:0/121145436 >> v2:172.21.15.190:3300/0 conn(0x7feff00aa480 0x7feff00675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.325+0000 7feff7bff700 1 --2- 172.21.15.131:0/121145436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff006e480 0x7feff00abdb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.325+0000 7feff7bff700 1 -- 172.21.15.131:0/121145436 >> 172.21.15.131:0/121145436 conn(0x7feff006c290 msgr2=0x7feff00ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:30.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.325+0000 7feff7bff700 1 -- 172.21.15.131:0/121145436 shutdown_connections 2023-12-26T21:52:30.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.325+0000 7feff7bff700 1 -- 172.21.15.131:0/121145436 wait complete. 2023-12-26T21:52:30.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.326+0000 7feff7bff700 1 Processor -- start 2023-12-26T21:52:30.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.327+0000 7feff7bff700 1 -- start start 2023-12-26T21:52:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.327+0000 7feff7bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feff006e480 0x7feff0156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.327+0000 7feff7bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feff00aa480 0x7feff00ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.327+0000 7feff7bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff0159820 0x7feff0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff7bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff016e280 con 0x7feff0159820 2023-12-26T21:52:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff7bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff016d670 con 0x7feff006e480 2023-12-26T21:52:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff7bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff016d950 con 0x7feff00aa480 2023-12-26T21:52:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff0159820 0x7feff0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff0159820 0x7feff0069d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33718/0 (socket says 172.21.15.131:33718) 2023-12-26T21:52:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff619c700 1 -- 172.21.15.131:0/3152307601 learned_addr learned my addr 172.21.15.131:0/3152307601 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff599b700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feff006e480 0x7feff0156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff519a700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feff00aa480 0x7feff00ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff619c700 1 -- 172.21.15.131:0/3152307601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feff00aa480 msgr2=0x7feff00ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff619c700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feff00aa480 0x7feff00ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff619c700 1 -- 172.21.15.131:0/3152307601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feff006e480 msgr2=0x7feff0156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff619c700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feff006e480 0x7feff0156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff619c700 1 -- 172.21.15.131:0/3152307601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0009420 con 0x7feff0159820 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.328+0000 7feff619c700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff0159820 0x7feff0069d80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fefe800e870 tx=0x7fefe8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.329+0000 7fefe5ffb700 1 -- 172.21.15.131:0/3152307601 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8013410 con 0x7feff0159820 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.329+0000 7fefe5ffb700 1 -- 172.21.15.131:0/3152307601 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8013570 con 0x7feff0159820 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.329+0000 7fefe5ffb700 1 -- 172.21.15.131:0/3152307601 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8013910 con 0x7feff0159820 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.329+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff016dc30 con 0x7feff0159820 2023-12-26T21:52:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.329+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff016d340 con 0x7feff0159820 2023-12-26T21:52:30.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.330+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff00657e0 con 0x7feff0159820 2023-12-26T21:52:30.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.331+0000 7fefe5ffb700 1 -- 172.21.15.131:0/3152307601 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fefe8013cc0 con 0x7feff0159820 2023-12-26T21:52:30.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.332+0000 7fefe5ffb700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fefd4066f30 0x7fefd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:30.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.332+0000 7fefe5ffb700 1 -- 172.21.15.131:0/3152307601 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fefe8084290 con 0x7feff0159820 2023-12-26T21:52:30.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.332+0000 7feff599b700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fefd4066f30 0x7fefd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:30.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.333+0000 7feff599b700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fefd4066f30 0x7fefd40693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fefec005c50 tx=0x7fefec009040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:30.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.336+0000 7fefe5ffb700 1 -- 172.21.15.131:0/3152307601 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefe8051370 con 0x7feff0159820 2023-12-26T21:52:30.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:30.572+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feff015a550 con 0x7feff0159820 2023-12-26T21:52:31.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.269+0000 7fefe5ffb700 1 -- 172.21.15.131:0/3152307601 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7fefe8054990 con 0x7feff0159820 2023-12-26T21:52:31.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.272+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fefd4066f30 msgr2=0x7fefd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:31.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.272+0000 7feff7bff700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fefd4066f30 0x7fefd40693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fefec005c50 tx=0x7fefec009040 comp rx=0 tx=0).stop 2023-12-26T21:52:31.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.272+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff0159820 msgr2=0x7feff0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:31.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.272+0000 7feff7bff700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff0159820 0x7feff0069d80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fefe800e870 tx=0x7fefe8008040 comp rx=0 tx=0).stop 2023-12-26T21:52:31.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.272+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 shutdown_connections 2023-12-26T21:52:31.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.273+0000 7feff7bff700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feff0159820 0x7feff0069d80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:31.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.273+0000 7feff7bff700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fefd4066f30 0x7fefd40693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:31.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.273+0000 7feff7bff700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feff006e480 0x7feff0156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:31.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.273+0000 7feff7bff700 1 --2- 172.21.15.131:0/3152307601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feff00aa480 0x7feff00ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:31.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.273+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 >> 172.21.15.131:0/3152307601 conn(0x7feff006c290 msgr2=0x7feff01576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:31.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.273+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 shutdown_connections 2023-12-26T21:52:31.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.273+0000 7feff7bff700 1 -- 172.21.15.131:0/3152307601 wait complete. 2023-12-26T21:52:31.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T21:52:31.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.488+0000 7f7b189ff700 1 Processor -- start 2023-12-26T21:52:31.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.489+0000 7f7b189ff700 1 -- start start 2023-12-26T21:52:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.490+0000 7f7b189ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b140cc580 0x7f7b140cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.490+0000 7f7b189ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7b140b00e0 0x7f7b1406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.490+0000 7f7b189ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b140b0d70 con 0x7f7b140b0560 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.490+0000 7f7b189ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b140680f0 con 0x7f7b140cc580 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.490+0000 7f7b189ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b140675b0 con 0x7f7b140b00e0 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.490+0000 7f7b1259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b140cc580 0x7f7b140cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.490+0000 7f7b1259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b140cc580 0x7f7b140cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33728/0 (socket says 172.21.15.131:33728) 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.490+0000 7f7b1259c700 1 -- 172.21.15.131:0/1396236706 learned_addr learned my addr 172.21.15.131:0/1396236706 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.490+0000 7f7b11d9b700 1 --2- 172.21.15.131:0/1396236706 >> v2:172.21.15.190:3300/0 conn(0x7f7b140b00e0 0x7f7b1406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.491+0000 7f7b1259c700 1 -- 172.21.15.131:0/1396236706 >> v2:172.21.15.190:3300/0 conn(0x7f7b140b00e0 msgr2=0x7f7b1406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.491+0000 7f7b1259c700 1 --2- 172.21.15.131:0/1396236706 >> v2:172.21.15.190:3300/0 conn(0x7f7b140b00e0 0x7f7b1406e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.491+0000 7f7b1259c700 1 -- 172.21.15.131:0/1396236706 >> v1:172.21.15.190:6789/0 conn(0x7f7b140b0560 legacy=0x7f7b1415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.491+0000 7f7b1259c700 1 -- 172.21.15.131:0/1396236706 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b14159c30 con 0x7f7b140cc580 2023-12-26T21:52:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.491+0000 7f7b1259c700 1 --2- 172.21.15.131:0/1396236706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140cc580 0x7f7b140cc950 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7b1406a140 tx=0x7f7b04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe24d8345178487 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:31.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.491+0000 7f7b1159a700 1 -- 172.21.15.131:0/1396236706 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04009e70 con 0x7f7b140cc580 2023-12-26T21:52:31.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.491+0000 7f7b1159a700 1 -- 172.21.15.131:0/1396236706 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0400b860 con 0x7f7b140cc580 2023-12-26T21:52:31.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.491+0000 7f7b1159a700 1 -- 172.21.15.131:0/1396236706 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400bc00 con 0x7f7b140cc580 2023-12-26T21:52:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.492+0000 7f7b189ff700 1 -- 172.21.15.131:0/1396236706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140cc580 msgr2=0x7f7b140cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.492+0000 7f7b189ff700 1 --2- 172.21.15.131:0/1396236706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140cc580 0x7f7b140cc950 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7b1406a140 tx=0x7f7b04009250 comp rx=0 tx=0).stop 2023-12-26T21:52:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.492+0000 7f7b189ff700 1 -- 172.21.15.131:0/1396236706 shutdown_connections 2023-12-26T21:52:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.492+0000 7f7b189ff700 1 --2- 172.21.15.131:0/1396236706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140cc580 0x7f7b140cc950 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.492+0000 7f7b189ff700 1 --2- 172.21.15.131:0/1396236706 >> v2:172.21.15.190:3300/0 conn(0x7f7b140b00e0 0x7f7b1406e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.492+0000 7f7b189ff700 1 -- 172.21.15.131:0/1396236706 >> 172.21.15.131:0/1396236706 conn(0x7f7b140686c0 msgr2=0x7f7b14068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.492+0000 7f7b189ff700 1 -- 172.21.15.131:0/1396236706 shutdown_connections 2023-12-26T21:52:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.492+0000 7f7b189ff700 1 -- 172.21.15.131:0/1396236706 wait complete. 2023-12-26T21:52:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.493+0000 7f7b189ff700 1 Processor -- start 2023-12-26T21:52:31.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.493+0000 7f7b189ff700 1 -- start start 2023-12-26T21:52:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.494+0000 7f7b189ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140b00e0 0x7f7b14149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.494+0000 7f7b189ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b140b0560 0x7f7b140c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.494+0000 7f7b189ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b140cc580 0x7f7b140c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:31.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.494+0000 7f7b189ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1415cd40 con 0x7f7b140b00e0 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.494+0000 7f7b189ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1414a3a0 con 0x7f7b140cc580 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.494+0000 7f7b189ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1415ca60 con 0x7f7b140b0560 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.494+0000 7f7b1259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140b00e0 0x7f7b14149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b1259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140b00e0 0x7f7b14149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33744/0 (socket says 172.21.15.131:33744) 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b1259c700 1 -- 172.21.15.131:0/3883562325 learned_addr learned my addr 172.21.15.131:0/3883562325 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b11d9b700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b140b0560 0x7f7b140c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b12d9d700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b140cc580 0x7f7b140c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b1259c700 1 -- 172.21.15.131:0/3883562325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b140b0560 msgr2=0x7f7b140c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b1259c700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b140b0560 0x7f7b140c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b1259c700 1 -- 172.21.15.131:0/3883562325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b140cc580 msgr2=0x7f7b140c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b1259c700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b140cc580 0x7f7b140c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b1259c700 1 -- 172.21.15.131:0/3883562325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040094f0 con 0x7f7b140b00e0 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b12d9d700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b140cc580 0x7f7b140c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b1259c700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140b00e0 0x7f7b14149e60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7b1406b770 tx=0x7f7b04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b027fc700 1 -- 172.21.15.131:0/3883562325 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011420 con 0x7f7b140b00e0 2023-12-26T21:52:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.495+0000 7f7b027fc700 1 -- 172.21.15.131:0/3883562325 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04011580 con 0x7f7b140b00e0 2023-12-26T21:52:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.496+0000 7f7b027fc700 1 -- 172.21.15.131:0/3883562325 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011940 con 0x7f7b140b00e0 2023-12-26T21:52:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.496+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b140adf70 con 0x7f7b140b00e0 2023-12-26T21:52:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.496+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b140ae530 con 0x7f7b140b00e0 2023-12-26T21:52:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.497+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b140ae8a0 con 0x7f7b140b00e0 2023-12-26T21:52:31.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.498+0000 7f7b027fc700 1 -- 172.21.15.131:0/3883562325 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7b0402c3b0 con 0x7f7b140b00e0 2023-12-26T21:52:31.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.498+0000 7f7b027fc700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7af4066f30 0x7f7af40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:31.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.498+0000 7f7b027fc700 1 -- 172.21.15.131:0/3883562325 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7b04085c90 con 0x7f7b140b00e0 2023-12-26T21:52:31.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.499+0000 7f7b11d9b700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7af4066f30 0x7f7af40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:31.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.499+0000 7f7b11d9b700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7af4066f30 0x7f7af40693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009d40 tx=0x7f7b08008040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:31.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.502+0000 7f7b027fc700 1 -- 172.21.15.131:0/3883562325 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b04052cd0 con 0x7f7b140b00e0 2023-12-26T21:52:31.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:31.739+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7b140ae6c0 con 0x7f7b140b00e0 2023-12-26T21:52:32.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.276+0000 7f7b027fc700 1 -- 172.21.15.131:0/3883562325 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7b040150c0 con 0x7f7b140b00e0 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7af4066f30 msgr2=0x7f7af40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7af4066f30 0x7f7af40693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009d40 tx=0x7f7b08008040 comp rx=0 tx=0).stop 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140b00e0 msgr2=0x7f7b14149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140b00e0 0x7f7b14149e60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7b1406b770 tx=0x7f7b04017000 comp rx=0 tx=0).stop 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 shutdown_connections 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f7af4066f30 0x7f7af40693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b140cc580 0x7f7b140c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b140b00e0 0x7f7b14149e60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 --2- 172.21.15.131:0/3883562325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b140b0560 0x7f7b140c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 >> 172.21.15.131:0/3883562325 conn(0x7f7b140686c0 msgr2=0x7f7b14157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:32.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 shutdown_connections 2023-12-26T21:52:32.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.279+0000 7f7b189ff700 1 -- 172.21.15.131:0/3883562325 wait complete. 2023-12-26T21:52:32.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:32.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.490+0000 7ff346bff700 1 Processor -- start 2023-12-26T21:52:32.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.491+0000 7ff346bff700 1 -- start start 2023-12-26T21:52:32.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.492+0000 7ff346bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8820 0x7ff3400ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.492+0000 7ff346bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3400c8bf0 0x7ff3400b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.492+0000 7ff346bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff340157a80 con 0x7ff340069d10 2023-12-26T21:52:32.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.492+0000 7ff346bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3400cac50 con 0x7ff3400c8bf0 2023-12-26T21:52:32.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.492+0000 7ff346bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400caf30 con 0x7ff3400c8820 2023-12-26T21:52:32.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.492+0000 7ff33ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3400c8bf0 0x7ff3400b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.492+0000 7ff33ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3400c8bf0 0x7ff3400b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33748/0 (socket says 172.21.15.131:33748) 2023-12-26T21:52:32.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.492+0000 7ff33ffff700 1 -- 172.21.15.131:0/942460398 learned_addr learned my addr 172.21.15.131:0/942460398 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff33ffff700 1 -- 172.21.15.131:0/942460398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8820 msgr2=0x7ff3400ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff34499b700 1 --2- 172.21.15.131:0/942460398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8820 0x7ff3400ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff33ffff700 1 --2- 172.21.15.131:0/942460398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8820 0x7ff3400ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff33ffff700 1 -- 172.21.15.131:0/942460398 >> v1:172.21.15.190:6789/0 conn(0x7ff340069d10 legacy=0x7ff340157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff33ffff700 1 -- 172.21.15.131:0/942460398 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400cb210 con 0x7ff3400c8bf0 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff34499b700 1 --2- 172.21.15.131:0/942460398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8820 0x7ff3400ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff33ffff700 1 --2- 172.21.15.131:0/942460398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8bf0 0x7ff3400b12e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0099b0 tx=0x7ff32c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74e8b7a2e16ca21 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff33f7fe700 1 -- 172.21.15.131:0/942460398 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00b860 con 0x7ff3400c8bf0 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff33f7fe700 1 -- 172.21.15.131:0/942460398 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c00b9c0 con 0x7ff3400c8bf0 2023-12-26T21:52:32.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.493+0000 7ff33f7fe700 1 -- 172.21.15.131:0/942460398 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00bd60 con 0x7ff3400c8bf0 2023-12-26T21:52:32.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.494+0000 7ff346bff700 1 -- 172.21.15.131:0/942460398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8bf0 msgr2=0x7ff3400b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.494+0000 7ff346bff700 1 --2- 172.21.15.131:0/942460398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8bf0 0x7ff3400b12e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0099b0 tx=0x7ff32c009250 comp rx=0 tx=0).stop 2023-12-26T21:52:32.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.494+0000 7ff346bff700 1 -- 172.21.15.131:0/942460398 shutdown_connections 2023-12-26T21:52:32.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.494+0000 7ff346bff700 1 --2- 172.21.15.131:0/942460398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8820 0x7ff3400ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.494+0000 7ff346bff700 1 --2- 172.21.15.131:0/942460398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8bf0 0x7ff3400b12e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.494+0000 7ff346bff700 1 -- 172.21.15.131:0/942460398 >> 172.21.15.131:0/942460398 conn(0x7ff340068040 msgr2=0x7ff340069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:32.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.494+0000 7ff346bff700 1 -- 172.21.15.131:0/942460398 shutdown_connections 2023-12-26T21:52:32.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.494+0000 7ff346bff700 1 -- 172.21.15.131:0/942460398 wait complete. 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.496+0000 7ff346bff700 1 Processor -- start 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.496+0000 7ff346bff700 1 -- start start 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff346bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff340069d10 0x7ff34013d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff346bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8820 0x7ff34015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff346bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8bf0 0x7ff34015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff346bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400c2fd0 con 0x7ff3400c8820 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff346bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400c2b40 con 0x7ff340069d10 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff346bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34013da80 con 0x7ff3400c8bf0 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff33ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8820 0x7ff34015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff33ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8820 0x7ff34015bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33762/0 (socket says 172.21.15.131:33762) 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff33ffff700 1 -- 172.21.15.131:0/2524664693 learned_addr learned my addr 172.21.15.131:0/2524664693 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff34499b700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff340069d10 0x7ff34013d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff34519c700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8bf0 0x7ff34015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff33ffff700 1 -- 172.21.15.131:0/2524664693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8bf0 msgr2=0x7ff34015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff33ffff700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8bf0 0x7ff34015c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff33ffff700 1 -- 172.21.15.131:0/2524664693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff340069d10 msgr2=0x7ff34013d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff33ffff700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff340069d10 0x7ff34013d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.497+0000 7ff33ffff700 1 -- 172.21.15.131:0/2524664693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0094f0 con 0x7ff3400c8820 2023-12-26T21:52:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.498+0000 7ff33ffff700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8820 0x7ff34015bfc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff3400688f0 tx=0x7ff32c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.498+0000 7ff33cff9700 1 -- 172.21.15.131:0/2524664693 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c024440 con 0x7ff3400c8820 2023-12-26T21:52:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.498+0000 7ff33cff9700 1 -- 172.21.15.131:0/2524664693 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c0245a0 con 0x7ff3400c8820 2023-12-26T21:52:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.498+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3400c1f30 con 0x7ff3400c8820 2023-12-26T21:52:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.498+0000 7ff33cff9700 1 -- 172.21.15.131:0/2524664693 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c024960 con 0x7ff3400c8820 2023-12-26T21:52:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.498+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3400c24f0 con 0x7ff3400c8820 2023-12-26T21:52:32.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.499+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3400657e0 con 0x7ff3400c8820 2023-12-26T21:52:32.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.500+0000 7ff33cff9700 1 -- 172.21.15.131:0/2524664693 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff32c01a420 con 0x7ff3400c8820 2023-12-26T21:52:32.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.501+0000 7ff33cff9700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff324066f30 0x7ff3240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.501+0000 7ff33cff9700 1 -- 172.21.15.131:0/2524664693 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff32c086620 con 0x7ff3400c8820 2023-12-26T21:52:32.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.502+0000 7ff34499b700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff324066f30 0x7ff3240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.502+0000 7ff34499b700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff324066f30 0x7ff3240693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff334006fd0 tx=0x7ff334008040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:32.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.506+0000 7ff33cff9700 1 -- 172.21.15.131:0/2524664693 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff32c053710 con 0x7ff3400c8820 2023-12-26T21:52:32.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.739+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3400cb4f0 con 0x7ff3400c8820 2023-12-26T21:52:32.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.739+0000 7ff33cff9700 1 -- 172.21.15.131:0/2524664693 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4603 (secure 0 0 0) 0x7ff32c056d30 con 0x7ff3400c8820 2023-12-26T21:52:32.740 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:32.740 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/625115906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":625115906},{"type":"v1","addr":"172.21.15.190:6833","nonce":625115906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2773111229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2773111229},{"type":"v1","addr":"172.21.15.190:6835","nonce":2773111229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/899613179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":899613179},{"type":"v1","addr":"172.21.15.131:6834","nonce":899613179}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:52:28.726694+0000","modified":"2023-12-26T21:52:32.268974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6836/4173752384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":4173752384},{"type":"v1","addr":"172.21.15.131:6836","nonce":4173752384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-12-26T21:52:32.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff324066f30 msgr2=0x7ff3240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff324066f30 0x7ff3240693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff334006fd0 tx=0x7ff334008040 comp rx=0 tx=0).stop 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8820 msgr2=0x7ff34015bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8820 0x7ff34015bfc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff3400688f0 tx=0x7ff32c017000 comp rx=0 tx=0).stop 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 shutdown_connections 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400c8bf0 0x7ff34015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff324066f30 0x7ff3240693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff340069d10 0x7ff34013d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 --2- 172.21.15.131:0/2524664693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3400c8820 0x7ff34015bfc0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 >> 172.21.15.131:0/2524664693 conn(0x7ff340068040 msgr2=0x7ff340156130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 shutdown_connections 2023-12-26T21:52:32.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.742+0000 7ff346bff700 1 -- 172.21.15.131:0/2524664693 wait complete. 2023-12-26T21:52:32.744 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 34 2023-12-26T21:52:32.760 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:32.838 INFO:tasks.ceph.mds.c.smithi131.stderr:2023-12-26T21:52:32.838+0000 7f6b2e9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:52:32.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.949+0000 7fc3a7bff700 1 Processor -- start 2023-12-26T21:52:32.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.950+0000 7fc3a7bff700 1 -- start start 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a7bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc3a013c510 0x7fc3a00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a7bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a006b8c0 0x7fc3a006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a7bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc3a00cc100 0x7fc3a00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a7bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a00686c0 con 0x7fc3a00cc100 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a7bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a0069520 con 0x7fc3a013c510 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a7bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a006c250 con 0x7fc3a006b8c0 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a619c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc3a00cc100 0x7fc3a00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a519a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a006b8c0 0x7fc3a006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a619c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc3a00cc100 0x7fc3a00cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33768/0 (socket says 172.21.15.131:33768) 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a619c700 1 -- 172.21.15.131:0/3555990932 learned_addr learned my addr 172.21.15.131:0/3555990932 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:32.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.951+0000 7fc3a599b700 1 --2- 172.21.15.131:0/3555990932 >> v2:172.21.15.190:3300/0 conn(0x7fc3a013c510 0x7fc3a00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a619c700 1 -- 172.21.15.131:0/3555990932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a006b8c0 msgr2=0x7fc3a006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a619c700 1 --2- 172.21.15.131:0/3555990932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a006b8c0 0x7fc3a006bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a619c700 1 -- 172.21.15.131:0/3555990932 >> v2:172.21.15.190:3300/0 conn(0x7fc3a013c510 msgr2=0x7fc3a00c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a619c700 1 --2- 172.21.15.131:0/3555990932 >> v2:172.21.15.190:3300/0 conn(0x7fc3a013c510 0x7fc3a00c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a619c700 1 -- 172.21.15.131:0/3555990932 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a0068910 con 0x7fc3a00cc100 2023-12-26T21:52:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a519a700 1 --2- 172.21.15.131:0/3555990932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a006b8c0 0x7fc3a006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a619c700 1 --2- 172.21.15.131:0/3555990932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a00cc100 0x7fc3a00cc560 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc3a006fcc0 tx=0x7fc398009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7e3d1a33c4a84e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a4999700 1 -- 172.21.15.131:0/3555990932 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39800e070 con 0x7fc3a00cc100 2023-12-26T21:52:32.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a4999700 1 -- 172.21.15.131:0/3555990932 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39800b970 con 0x7fc3a00cc100 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.952+0000 7fc3a4999700 1 -- 172.21.15.131:0/3555990932 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3980092e0 con 0x7fc3a00cc100 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.953+0000 7fc3a7bff700 1 -- 172.21.15.131:0/3555990932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a00cc100 msgr2=0x7fc3a00cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.953+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/3555990932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a00cc100 0x7fc3a00cc560 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc3a006fcc0 tx=0x7fc398009840 comp rx=0 tx=0).stop 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.953+0000 7fc3a7bff700 1 -- 172.21.15.131:0/3555990932 shutdown_connections 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.953+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/3555990932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a00cc100 0x7fc3a00cc560 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.953+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/3555990932 >> v2:172.21.15.190:3300/0 conn(0x7fc3a013c510 0x7fc3a00c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.953+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/3555990932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a006b8c0 0x7fc3a006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.953+0000 7fc3a7bff700 1 -- 172.21.15.131:0/3555990932 >> 172.21.15.131:0/3555990932 conn(0x7fc3a00ce3e0 msgr2=0x7fc3a00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.953+0000 7fc3a7bff700 1 -- 172.21.15.131:0/3555990932 shutdown_connections 2023-12-26T21:52:32.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.953+0000 7fc3a7bff700 1 -- 172.21.15.131:0/3555990932 wait complete. 2023-12-26T21:52:32.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.955+0000 7fc3a7bff700 1 Processor -- start 2023-12-26T21:52:32.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.955+0000 7fc3a7bff700 1 -- start start 2023-12-26T21:52:32.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.956+0000 7fc3a7bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a006b8c0 0x7fc3a00b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.956+0000 7fc3a7bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a00cc100 0x7fc3a00b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.956+0000 7fc3a7bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a013c510 0x7fc3a013bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.956+0000 7fc3a7bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00b2170 con 0x7fc3a013c510 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.956+0000 7fc3a7bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00b1560 con 0x7fc3a006b8c0 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.956+0000 7fc3a7bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00b1840 con 0x7fc3a00cc100 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a013c510 0x7fc3a013bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a519a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a00cc100 0x7fc3a00b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a599b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a006b8c0 0x7fc3a00b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a519a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a00cc100 0x7fc3a00b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:60044/0 (socket says 172.21.15.131:60044) 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a519a700 1 -- 172.21.15.131:0/1086762411 learned_addr learned my addr 172.21.15.131:0/1086762411 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a619c700 1 -- 172.21.15.131:0/1086762411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a00cc100 msgr2=0x7fc3a00b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a619c700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a00cc100 0x7fc3a00b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a619c700 1 -- 172.21.15.131:0/1086762411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a006b8c0 msgr2=0x7fc3a00b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a619c700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a006b8c0 0x7fc3a00b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:32.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a619c700 1 -- 172.21.15.131:0/1086762411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390009420 con 0x7fc3a013c510 2023-12-26T21:52:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a519a700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a00cc100 0x7fc3a00b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a599b700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a006b8c0 0x7fc3a00b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.957+0000 7fc3a619c700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a013c510 0x7fc3a013bc40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc398015000 tx=0x7fc398018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.958+0000 7fc395ffb700 1 -- 172.21.15.131:0/1086762411 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398008ad0 con 0x7fc3a013c510 2023-12-26T21:52:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.958+0000 7fc395ffb700 1 -- 172.21.15.131:0/1086762411 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc398008c50 con 0x7fc3a013c510 2023-12-26T21:52:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.958+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc398009ae0 con 0x7fc3a013c510 2023-12-26T21:52:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.958+0000 7fc395ffb700 1 -- 172.21.15.131:0/1086762411 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398026460 con 0x7fc3a013c510 2023-12-26T21:52:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.958+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a00b1e00 con 0x7fc3a013c510 2023-12-26T21:52:32.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.959+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a00b22e0 con 0x7fc3a013c510 2023-12-26T21:52:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.960+0000 7fc395ffb700 1 -- 172.21.15.131:0/1086762411 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc398003040 con 0x7fc3a013c510 2023-12-26T21:52:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.961+0000 7fc395ffb700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fc384066f30 0x7fc3840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.961+0000 7fc395ffb700 1 -- 172.21.15.131:0/1086762411 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fc398085f30 con 0x7fc3a013c510 2023-12-26T21:52:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.961+0000 7fc3a599b700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fc384066f30 0x7fc3840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.962+0000 7fc3a599b700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fc384066f30 0x7fc3840693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc39c011460 tx=0x7fc39c009120 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:32.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:32.965+0000 7fc395ffb700 1 -- 172.21.15.131:0/1086762411 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc398053020 con 0x7fc3a013c510 2023-12-26T21:52:33.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.158+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3a00657e0 con 0x7fc3a013c510 2023-12-26T21:52:33.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.159+0000 7fc395ffb700 1 -- 172.21.15.131:0/1086762411 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+15994 (secure 0 0 0) 0x7fc398056640 con 0x7fc3a013c510 2023-12-26T21:52:33.160 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:33.160 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:30.218820+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:52:26.449943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-26T21:52:27.710394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fc384066f30 msgr2=0x7fc3840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fc384066f30 0x7fc3840693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc39c011460 tx=0x7fc39c009120 comp rx=0 tx=0).stop 2023-12-26T21:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a013c510 msgr2=0x7fc3a013bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a013c510 0x7fc3a013bc40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc398015000 tx=0x7fc398018000 comp rx=0 tx=0).stop 2023-12-26T21:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 shutdown_connections 2023-12-26T21:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3a013c510 0x7fc3a013bc40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fc384066f30 0x7fc3840693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a006b8c0 0x7fc3a00b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 --2- 172.21.15.131:0/1086762411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a00cc100 0x7fc3a00b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.161+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 >> 172.21.15.131:0/1086762411 conn(0x7fc3a00ce3e0 msgr2=0x7fc3a00c3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:33.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.162+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 shutdown_connections 2023-12-26T21:52:33.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.162+0000 7fc3a7bff700 1 -- 172.21.15.131:0/1086762411 wait complete. 2023-12-26T21:52:33.180 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T21:52:33.180 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:52:33.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.370+0000 7ff0f91ff700 1 Processor -- start 2023-12-26T21:52:33.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.370+0000 7ff0f91ff700 1 -- start start 2023-12-26T21:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.371+0000 7ff0f91ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0f41596a0 0x7ff0f40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.371+0000 7ff0f91ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff0f40b0870 0x7ff0f40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.371+0000 7ff0f91ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f40afcd0 con 0x7ff0f4159a70 2023-12-26T21:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.371+0000 7ff0f91ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f40b3e70 con 0x7ff0f40b0870 2023-12-26T21:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.371+0000 7ff0f91ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f40b2e00 con 0x7ff0f41596a0 2023-12-26T21:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.371+0000 7ff0f259c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff0f4159a70 0x7ff0f40b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46652/0 (socket says 172.21.15.131:46652) 2023-12-26T21:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.371+0000 7ff0f259c700 1 -- 172.21.15.131:0/39747891 learned_addr learned my addr 172.21.15.131:0/39747891 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.371+0000 7ff0f359e700 1 --2- 172.21.15.131:0/39747891 >> v2:172.21.15.131:3300/0 conn(0x7ff0f40b0870 0x7ff0f40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f2d9d700 1 --2- 172.21.15.131:0/39747891 >> v2:172.21.15.190:3300/0 conn(0x7ff0f41596a0 0x7ff0f40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f1d9b700 1 -- 172.21.15.131:0/39747891 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104767008 0 0) 0x7ff0f40afcd0 con 0x7ff0f4159a70 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f359e700 1 -- 172.21.15.131:0/39747891 >> v2:172.21.15.190:3300/0 conn(0x7ff0f41596a0 msgr2=0x7ff0f40b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f359e700 1 --2- 172.21.15.131:0/39747891 >> v2:172.21.15.190:3300/0 conn(0x7ff0f41596a0 0x7ff0f40b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f359e700 1 -- 172.21.15.131:0/39747891 >> v1:172.21.15.131:6789/0 conn(0x7ff0f4159a70 legacy=0x7ff0f40b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f359e700 1 -- 172.21.15.131:0/39747891 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f40b0cb0 con 0x7ff0f40b0870 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f359e700 1 --2- 172.21.15.131:0/39747891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 0x7ff0f40b28c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40892a0 tx=0x7ff0ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c87e0bb8775d2d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f1d9b700 1 -- 172.21.15.131:0/39747891 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00e070 con 0x7ff0f40b0870 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f1d9b700 1 -- 172.21.15.131:0/39747891 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec00bb60 con 0x7ff0f40b0870 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.372+0000 7ff0f1d9b700 1 -- 172.21.15.131:0/39747891 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec012430 con 0x7ff0f40b0870 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.373+0000 7ff0f91ff700 1 -- 172.21.15.131:0/39747891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 msgr2=0x7ff0f40b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.373+0000 7ff0f91ff700 1 --2- 172.21.15.131:0/39747891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 0x7ff0f40b28c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40892a0 tx=0x7ff0ec009780 comp rx=0 tx=0).stop 2023-12-26T21:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.373+0000 7ff0f91ff700 1 -- 172.21.15.131:0/39747891 shutdown_connections 2023-12-26T21:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.373+0000 7ff0f91ff700 1 --2- 172.21.15.131:0/39747891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 0x7ff0f40b28c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.373+0000 7ff0f91ff700 1 --2- 172.21.15.131:0/39747891 >> v2:172.21.15.190:3300/0 conn(0x7ff0f41596a0 0x7ff0f40b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.373+0000 7ff0f91ff700 1 -- 172.21.15.131:0/39747891 >> 172.21.15.131:0/39747891 conn(0x7ff0f406ece0 msgr2=0x7ff0f406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.373+0000 7ff0f91ff700 1 -- 172.21.15.131:0/39747891 shutdown_connections 2023-12-26T21:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.373+0000 7ff0f91ff700 1 -- 172.21.15.131:0/39747891 wait complete. 2023-12-26T21:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.375+0000 7ff0f91ff700 1 Processor -- start 2023-12-26T21:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.375+0000 7ff0f91ff700 1 -- start start 2023-12-26T21:52:33.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f91ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 0x7ff0f40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f91ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f41596a0 0x7ff0f40cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f91ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0f4159a70 0x7ff0f4151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f91ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40c88f0 con 0x7ff0f40b0870 2023-12-26T21:52:33.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f91ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40c7ce0 con 0x7ff0f4159a70 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f91ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40c7fc0 con 0x7ff0f41596a0 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f2d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 0x7ff0f40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f2d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 0x7ff0f40c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33808/0 (socket says 172.21.15.131:33808) 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f2d9d700 1 -- 172.21.15.131:0/3118137943 learned_addr learned my addr 172.21.15.131:0/3118137943 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f359e700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0f4159a70 0x7ff0f4151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f259c700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f41596a0 0x7ff0f40cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f2d9d700 1 -- 172.21.15.131:0/3118137943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f41596a0 msgr2=0x7ff0f40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f2d9d700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f41596a0 0x7ff0f40cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f2d9d700 1 -- 172.21.15.131:0/3118137943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0f4159a70 msgr2=0x7ff0f4151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f2d9d700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0f4159a70 0x7ff0f4151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f2d9d700 1 -- 172.21.15.131:0/3118137943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec009a20 con 0x7ff0f40b0870 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f359e700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0f4159a70 0x7ff0f4151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.376+0000 7ff0f2d9d700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 0x7ff0f40c75e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400ec80 tx=0x7ff0e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:33.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.377+0000 7ff0daffd700 1 -- 172.21.15.131:0/3118137943 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400c9d0 con 0x7ff0f40b0870 2023-12-26T21:52:33.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.377+0000 7ff0daffd700 1 -- 172.21.15.131:0/3118137943 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e400cb30 con 0x7ff0f40b0870 2023-12-26T21:52:33.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.377+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f40c82a0 con 0x7ff0f40b0870 2023-12-26T21:52:33.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.377+0000 7ff0daffd700 1 -- 172.21.15.131:0/3118137943 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4013470 con 0x7ff0f40b0870 2023-12-26T21:52:33.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.377+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f4152160 con 0x7ff0f40b0870 2023-12-26T21:52:33.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.378+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f40657e0 con 0x7ff0f40b0870 2023-12-26T21:52:33.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.379+0000 7ff0daffd700 1 -- 172.21.15.131:0/3118137943 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff0e4013820 con 0x7ff0f40b0870 2023-12-26T21:52:33.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.380+0000 7ff0daffd700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff0e0066f30 0x7ff0e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.380+0000 7ff0daffd700 1 -- 172.21.15.131:0/3118137943 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff0e4084a30 con 0x7ff0f40b0870 2023-12-26T21:52:33.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.380+0000 7ff0f259c700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff0e0066f30 0x7ff0e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.381+0000 7ff0f259c700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff0e0066f30 0x7ff0e00693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0f416cdb0 tx=0x7ff0e8009040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:33.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.384+0000 7ff0daffd700 1 -- 172.21.15.131:0/3118137943 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0e4051b10 con 0x7ff0f40b0870 2023-12-26T21:52:33.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.574+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 --> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff0f40c8a30 con 0x7ff0e0066f30 2023-12-26T21:52:33.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.585+0000 7ff0daffd700 1 -- 172.21.15.131:0/3118137943 <== mgr.6045 v2:172.21.15.131:6835/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288873 (secure 0 0 0) 0x7ff0e80014f0 con 0x7ff0e0066f30 2023-12-26T21:52:33.589 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff0e0066f30 msgr2=0x7ff0e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff0e0066f30 0x7ff0e00693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0f416cdb0 tx=0x7ff0e8009040 comp rx=0 tx=0).stop 2023-12-26T21:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 msgr2=0x7ff0f40c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 0x7ff0f40c75e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400ec80 tx=0x7ff0e40091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 shutdown_connections 2023-12-26T21:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff0e0066f30 0x7ff0e00693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0f4159a70 0x7ff0f4151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0f40b0870 0x7ff0f40c75e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 --2- 172.21.15.131:0/3118137943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f41596a0 0x7ff0f40cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 >> 172.21.15.131:0/3118137943 conn(0x7ff0f406ece0 msgr2=0x7ff0f40b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:33.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 shutdown_connections 2023-12-26T21:52:33.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.589+0000 7ff0f91ff700 1 -- 172.21.15.131:0/3118137943 wait complete. 2023-12-26T21:52:33.593 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:52:33.608 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-26T21:52:32.812828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424060,"kb_used_data":8892,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752550660,"statfs":{"total":773094113280,"available":770611875840,"internally_reserved":0,"allocated":9105408,"data_stored":4945876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.576822"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246618+0000","last_change":"2023-12-26T21:52:29.240921+0000","last_active":"2023-12-26T21:52:29.246618+0000","last_peered":"2023-12-26T21:52:29.246618+0000","last_clean":"2023-12-26T21:52:29.246618+0000","last_became_active":"2023-12-26T21:52:29.240384+0000","last_became_peered":"2023-12-26T21:52:29.240384+0000","last_unstale":"2023-12-26T21:52:29.246618+0000","last_undegraded":"2023-12-26T21:52:29.246618+0000","last_fullsized":"2023-12-26T21:52:29.246618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:01:40.250956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.241258+0000","last_change":"2023-12-26T21:52:29.237095+0000","last_active":"2023-12-26T21:52:29.241258+0000","last_peered":"2023-12-26T21:52:29.241258+0000","last_clean":"2023-12-26T21:52:29.241258+0000","last_became_active":"2023-12-26T21:52:29.236784+0000","last_became_peered":"2023-12-26T21:52:29.236784+0000","last_unstale":"2023-12-26T21:52:29.241258+0000","last_undegraded":"2023-12-26T21:52:29.241258+0000","last_fullsized":"2023-12-26T21:52:29.241258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:29:40.151871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236144+0000","last_change":"2023-12-26T21:52:29.231652+0000","last_active":"2023-12-26T21:52:29.236144+0000","last_peered":"2023-12-26T21:52:29.236144+0000","last_clean":"2023-12-26T21:52:29.236144+0000","last_became_active":"2023-12-26T21:52:29.230905+0000","last_became_peered":"2023-12-26T21:52:29.230905+0000","last_unstale":"2023-12-26T21:52:29.236144+0000","last_undegraded":"2023-12-26T21:52:29.236144+0000","last_fullsized":"2023-12-26T21:52:29.236144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:35:56.220697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.241041+0000","last_change":"2023-12-26T21:52:29.240126+0000","last_active":"2023-12-26T21:52:29.241041+0000","last_peered":"2023-12-26T21:52:29.241041+0000","last_clean":"2023-12-26T21:52:29.241041+0000","last_became_active":"2023-12-26T21:52:29.239884+0000","last_became_peered":"2023-12-26T21:52:29.239884+0000","last_unstale":"2023-12-26T21:52:29.241041+0000","last_undegraded":"2023-12-26T21:52:29.241041+0000","last_fullsized":"2023-12-26T21:52:29.241041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:17:52.678820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243707+0000","last_change":"2023-12-26T21:52:29.238194+0000","last_active":"2023-12-26T21:52:29.243707+0000","last_peered":"2023-12-26T21:52:29.243707+0000","last_clean":"2023-12-26T21:52:29.243707+0000","last_became_active":"2023-12-26T21:52:29.237767+0000","last_became_peered":"2023-12-26T21:52:29.237767+0000","last_unstale":"2023-12-26T21:52:29.243707+0000","last_undegraded":"2023-12-26T21:52:29.243707+0000","last_fullsized":"2023-12-26T21:52:29.243707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:31.371297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243964+0000","last_change":"2023-12-26T21:52:29.239551+0000","last_active":"2023-12-26T21:52:29.243964+0000","last_peered":"2023-12-26T21:52:29.243964+0000","last_clean":"2023-12-26T21:52:29.243964+0000","last_became_active":"2023-12-26T21:52:29.239254+0000","last_became_peered":"2023-12-26T21:52:29.239254+0000","last_unstale":"2023-12-26T21:52:29.243964+0000","last_undegraded":"2023-12-26T21:52:29.243964+0000","last_fullsized":"2023-12-26T21:52:29.243964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:13:51.069814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234379+0000","last_change":"2023-12-26T21:52:29.229562+0000","last_active":"2023-12-26T21:52:29.234379+0000","last_peered":"2023-12-26T21:52:29.234379+0000","last_clean":"2023-12-26T21:52:29.234379+0000","last_became_active":"2023-12-26T21:52:29.229306+0000","last_became_peered":"2023-12-26T21:52:29.229306+0000","last_unstale":"2023-12-26T21:52:29.234379+0000","last_undegraded":"2023-12-26T21:52:29.234379+0000","last_fullsized":"2023-12-26T21:52:29.234379+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:44:47.982985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234410+0000","last_change":"2023-12-26T21:52:29.230927+0000","last_active":"2023-12-26T21:52:29.234410+0000","last_peered":"2023-12-26T21:52:29.234410+0000","last_clean":"2023-12-26T21:52:29.234410+0000","last_became_active":"2023-12-26T21:52:29.230645+0000","last_became_peered":"2023-12-26T21:52:29.230645+0000","last_unstale":"2023-12-26T21:52:29.234410+0000","last_undegraded":"2023-12-26T21:52:29.234410+0000","last_fullsized":"2023-12-26T21:52:29.234410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:45:47.041678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243955+0000","last_change":"2023-12-26T21:52:29.243401+0000","last_active":"2023-12-26T21:52:29.243955+0000","last_peered":"2023-12-26T21:52:29.243955+0000","last_clean":"2023-12-26T21:52:29.243955+0000","last_became_active":"2023-12-26T21:52:29.243193+0000","last_became_peered":"2023-12-26T21:52:29.243193+0000","last_unstale":"2023-12-26T21:52:29.243955+0000","last_undegraded":"2023-12-26T21:52:29.243955+0000","last_fullsized":"2023-12-26T21:52:29.243955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:59:56.264143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.241147+0000","last_change":"2023-12-26T21:52:29.236872+0000","last_active":"2023-12-26T21:52:29.241147+0000","last_peered":"2023-12-26T21:52:29.241147+0000","last_clean":"2023-12-26T21:52:29.241147+0000","last_became_active":"2023-12-26T21:52:29.236469+0000","last_became_peered":"2023-12-26T21:52:29.236469+0000","last_unstale":"2023-12-26T21:52:29.241147+0000","last_undegraded":"2023-12-26T21:52:29.241147+0000","last_fullsized":"2023-12-26T21:52:29.241147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:03:02.542674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.242414+0000","last_change":"2023-12-26T21:52:29.236719+0000","last_active":"2023-12-26T21:52:29.242414+0000","last_peered":"2023-12-26T21:52:29.242414+0000","last_clean":"2023-12-26T21:52:29.242414+0000","last_became_active":"2023-12-26T21:52:29.236462+0000","last_became_peered":"2023-12-26T21:52:29.236462+0000","last_unstale":"2023-12-26T21:52:29.242414+0000","last_undegraded":"2023-12-26T21:52:29.242414+0000","last_fullsized":"2023-12-26T21:52:29.242414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:27:29.980110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234590+0000","last_change":"2023-12-26T21:52:29.227571+0000","last_active":"2023-12-26T21:52:29.234590+0000","last_peered":"2023-12-26T21:52:29.234590+0000","last_clean":"2023-12-26T21:52:29.234590+0000","last_became_active":"2023-12-26T21:52:29.227147+0000","last_became_peered":"2023-12-26T21:52:29.227147+0000","last_unstale":"2023-12-26T21:52:29.234590+0000","last_undegraded":"2023-12-26T21:52:29.234590+0000","last_fullsized":"2023-12-26T21:52:29.234590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:17.417194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.231197+0000","last_change":"2023-12-26T21:52:29.226381+0000","last_active":"2023-12-26T21:52:29.231197+0000","last_peered":"2023-12-26T21:52:29.231197+0000","last_clean":"2023-12-26T21:52:29.231197+0000","last_became_active":"2023-12-26T21:52:29.226080+0000","last_became_peered":"2023-12-26T21:52:29.226080+0000","last_unstale":"2023-12-26T21:52:29.231197+0000","last_undegraded":"2023-12-26T21:52:29.231197+0000","last_fullsized":"2023-12-26T21:52:29.231197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:20:48.762488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237142+0000","last_change":"2023-12-26T21:52:29.233635+0000","last_active":"2023-12-26T21:52:29.237142+0000","last_peered":"2023-12-26T21:52:29.237142+0000","last_clean":"2023-12-26T21:52:29.237142+0000","last_became_active":"2023-12-26T21:52:29.233336+0000","last_became_peered":"2023-12-26T21:52:29.233336+0000","last_unstale":"2023-12-26T21:52:29.237142+0000","last_undegraded":"2023-12-26T21:52:29.237142+0000","last_fullsized":"2023-12-26T21:52:29.237142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:22:55.692487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.238180+0000","last_change":"2023-12-26T21:52:29.236940+0000","last_active":"2023-12-26T21:52:29.238180+0000","last_peered":"2023-12-26T21:52:29.238180+0000","last_clean":"2023-12-26T21:52:29.238180+0000","last_became_active":"2023-12-26T21:52:29.236599+0000","last_became_peered":"2023-12-26T21:52:29.236599+0000","last_unstale":"2023-12-26T21:52:29.238180+0000","last_undegraded":"2023-12-26T21:52:29.238180+0000","last_fullsized":"2023-12-26T21:52:29.238180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:29.826697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243822+0000","last_change":"2023-12-26T21:52:29.238566+0000","last_active":"2023-12-26T21:52:29.243822+0000","last_peered":"2023-12-26T21:52:29.243822+0000","last_clean":"2023-12-26T21:52:29.243822+0000","last_became_active":"2023-12-26T21:52:29.236840+0000","last_became_peered":"2023-12-26T21:52:29.236840+0000","last_unstale":"2023-12-26T21:52:29.243822+0000","last_undegraded":"2023-12-26T21:52:29.243822+0000","last_fullsized":"2023-12-26T21:52:29.243822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:30:58.136825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243634+0000","last_change":"2023-12-26T21:52:29.238113+0000","last_active":"2023-12-26T21:52:29.243634+0000","last_peered":"2023-12-26T21:52:29.243634+0000","last_clean":"2023-12-26T21:52:29.243634+0000","last_became_active":"2023-12-26T21:52:29.237730+0000","last_became_peered":"2023-12-26T21:52:29.237730+0000","last_unstale":"2023-12-26T21:52:29.243634+0000","last_undegraded":"2023-12-26T21:52:29.243634+0000","last_fullsized":"2023-12-26T21:52:29.243634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:43:48.845469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236295+0000","last_change":"2023-12-26T21:52:29.232603+0000","last_active":"2023-12-26T21:52:29.236295+0000","last_peered":"2023-12-26T21:52:29.236295+0000","last_clean":"2023-12-26T21:52:29.236295+0000","last_became_active":"2023-12-26T21:52:29.232268+0000","last_became_peered":"2023-12-26T21:52:29.232268+0000","last_unstale":"2023-12-26T21:52:29.236295+0000","last_undegraded":"2023-12-26T21:52:29.236295+0000","last_fullsized":"2023-12-26T21:52:29.236295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:16:12.163141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232591+0000","last_change":"2023-12-26T21:52:29.229265+0000","last_active":"2023-12-26T21:52:29.232591+0000","last_peered":"2023-12-26T21:52:29.232591+0000","last_clean":"2023-12-26T21:52:29.232591+0000","last_became_active":"2023-12-26T21:52:29.228926+0000","last_became_peered":"2023-12-26T21:52:29.228926+0000","last_unstale":"2023-12-26T21:52:29.232591+0000","last_undegraded":"2023-12-26T21:52:29.232591+0000","last_fullsized":"2023-12-26T21:52:29.232591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:48:27.756939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246523+0000","last_change":"2023-12-26T21:52:29.241195+0000","last_active":"2023-12-26T21:52:29.246523+0000","last_peered":"2023-12-26T21:52:29.246523+0000","last_clean":"2023-12-26T21:52:29.246523+0000","last_became_active":"2023-12-26T21:52:29.239771+0000","last_became_peered":"2023-12-26T21:52:29.239771+0000","last_unstale":"2023-12-26T21:52:29.246523+0000","last_undegraded":"2023-12-26T21:52:29.246523+0000","last_fullsized":"2023-12-26T21:52:29.246523+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:35:56.220697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234669+0000","last_change":"2023-12-26T21:52:29.229535+0000","last_active":"2023-12-26T21:52:29.234669+0000","last_peered":"2023-12-26T21:52:29.234669+0000","last_clean":"2023-12-26T21:52:29.234669+0000","last_became_active":"2023-12-26T21:52:29.229262+0000","last_became_peered":"2023-12-26T21:52:29.229262+0000","last_unstale":"2023-12-26T21:52:29.234669+0000","last_undegraded":"2023-12-26T21:52:29.234669+0000","last_fullsized":"2023-12-26T21:52:29.234669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:13:03.737604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.231063+0000","last_change":"2023-12-26T21:52:29.228634+0000","last_active":"2023-12-26T21:52:29.231063+0000","last_peered":"2023-12-26T21:52:29.231063+0000","last_clean":"2023-12-26T21:52:29.231063+0000","last_became_active":"2023-12-26T21:52:29.228317+0000","last_became_peered":"2023-12-26T21:52:29.228317+0000","last_unstale":"2023-12-26T21:52:29.231063+0000","last_undegraded":"2023-12-26T21:52:29.231063+0000","last_fullsized":"2023-12-26T21:52:29.231063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:08:03.577052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.229637+0000","last_change":"2023-12-26T21:52:29.224988+0000","last_active":"2023-12-26T21:52:29.229637+0000","last_peered":"2023-12-26T21:52:29.229637+0000","last_clean":"2023-12-26T21:52:29.229637+0000","last_became_active":"2023-12-26T21:52:29.224692+0000","last_became_peered":"2023-12-26T21:52:29.224692+0000","last_unstale":"2023-12-26T21:52:29.229637+0000","last_undegraded":"2023-12-26T21:52:29.229637+0000","last_fullsized":"2023-12-26T21:52:29.229637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:37:05.801449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.242713+0000","last_change":"2023-12-26T21:52:29.242152+0000","last_active":"2023-12-26T21:52:29.242713+0000","last_peered":"2023-12-26T21:52:29.242713+0000","last_clean":"2023-12-26T21:52:29.242713+0000","last_became_active":"2023-12-26T21:52:29.241902+0000","last_became_peered":"2023-12-26T21:52:29.241902+0000","last_unstale":"2023-12-26T21:52:29.242713+0000","last_undegraded":"2023-12-26T21:52:29.242713+0000","last_fullsized":"2023-12-26T21:52:29.242713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:22.606449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246163+0000","last_change":"2023-12-26T21:52:29.241969+0000","last_active":"2023-12-26T21:52:29.246163+0000","last_peered":"2023-12-26T21:52:29.246163+0000","last_clean":"2023-12-26T21:52:29.246163+0000","last_became_active":"2023-12-26T21:52:29.241580+0000","last_became_peered":"2023-12-26T21:52:29.241580+0000","last_unstale":"2023-12-26T21:52:29.246163+0000","last_undegraded":"2023-12-26T21:52:29.246163+0000","last_fullsized":"2023-12-26T21:52:29.246163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:40:31.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.233931+0000","last_change":"2023-12-26T21:52:29.228211+0000","last_active":"2023-12-26T21:52:29.233931+0000","last_peered":"2023-12-26T21:52:29.233931+0000","last_clean":"2023-12-26T21:52:29.233931+0000","last_became_active":"2023-12-26T21:52:29.227673+0000","last_became_peered":"2023-12-26T21:52:29.227673+0000","last_unstale":"2023-12-26T21:52:29.233931+0000","last_undegraded":"2023-12-26T21:52:29.233931+0000","last_fullsized":"2023-12-26T21:52:29.233931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:09:11.466566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232612+0000","last_change":"2023-12-26T21:52:29.229577+0000","last_active":"2023-12-26T21:52:29.232612+0000","last_peered":"2023-12-26T21:52:29.232612+0000","last_clean":"2023-12-26T21:52:29.232612+0000","last_became_active":"2023-12-26T21:52:29.229303+0000","last_became_peered":"2023-12-26T21:52:29.229303+0000","last_unstale":"2023-12-26T21:52:29.232612+0000","last_undegraded":"2023-12-26T21:52:29.232612+0000","last_fullsized":"2023-12-26T21:52:29.232612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:54:23.004697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.231591+0000","last_change":"2023-12-26T21:52:29.227033+0000","last_active":"2023-12-26T21:52:29.231591+0000","last_peered":"2023-12-26T21:52:29.231591+0000","last_clean":"2023-12-26T21:52:29.231591+0000","last_became_active":"2023-12-26T21:52:29.226751+0000","last_became_peered":"2023-12-26T21:52:29.226751+0000","last_unstale":"2023-12-26T21:52:29.231591+0000","last_undegraded":"2023-12-26T21:52:29.231591+0000","last_fullsized":"2023-12-26T21:52:29.231591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:44:47.982985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232084+0000","last_change":"2023-12-26T21:52:29.227704+0000","last_active":"2023-12-26T21:52:29.232084+0000","last_peered":"2023-12-26T21:52:29.232084+0000","last_clean":"2023-12-26T21:52:29.232084+0000","last_became_active":"2023-12-26T21:52:29.225182+0000","last_became_peered":"2023-12-26T21:52:29.225182+0000","last_unstale":"2023-12-26T21:52:29.232084+0000","last_undegraded":"2023-12-26T21:52:29.232084+0000","last_fullsized":"2023-12-26T21:52:29.232084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:24:12.574316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.233047+0000","last_change":"2023-12-26T21:52:29.227388+0000","last_active":"2023-12-26T21:52:29.233047+0000","last_peered":"2023-12-26T21:52:29.233047+0000","last_clean":"2023-12-26T21:52:29.233047+0000","last_became_active":"2023-12-26T21:52:29.226885+0000","last_became_peered":"2023-12-26T21:52:29.226885+0000","last_unstale":"2023-12-26T21:52:29.233047+0000","last_undegraded":"2023-12-26T21:52:29.233047+0000","last_fullsized":"2023-12-26T21:52:29.233047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:40.287102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.224129+0000","last_change":"2023-12-26T21:52:29.219629+0000","last_active":"2023-12-26T21:52:29.224129+0000","last_peered":"2023-12-26T21:52:29.224129+0000","last_clean":"2023-12-26T21:52:29.224129+0000","last_became_active":"2023-12-26T21:52:29.218008+0000","last_became_peered":"2023-12-26T21:52:29.218008+0000","last_unstale":"2023-12-26T21:52:29.224129+0000","last_undegraded":"2023-12-26T21:52:29.224129+0000","last_fullsized":"2023-12-26T21:52:29.224129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:11.518915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230615+0000","last_change":"2023-12-26T21:52:29.224053+0000","last_active":"2023-12-26T21:52:29.230615+0000","last_peered":"2023-12-26T21:52:29.230615+0000","last_clean":"2023-12-26T21:52:29.230615+0000","last_became_active":"2023-12-26T21:52:29.223769+0000","last_became_peered":"2023-12-26T21:52:29.223769+0000","last_unstale":"2023-12-26T21:52:29.230615+0000","last_undegraded":"2023-12-26T21:52:29.230615+0000","last_fullsized":"2023-12-26T21:52:29.230615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:23:25.880180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243876+0000","last_change":"2023-12-26T21:52:29.238559+0000","last_active":"2023-12-26T21:52:29.243876+0000","last_peered":"2023-12-26T21:52:29.243876+0000","last_clean":"2023-12-26T21:52:29.243876+0000","last_became_active":"2023-12-26T21:52:29.236886+0000","last_became_peered":"2023-12-26T21:52:29.236886+0000","last_unstale":"2023-12-26T21:52:29.243876+0000","last_undegraded":"2023-12-26T21:52:29.243876+0000","last_fullsized":"2023-12-26T21:52:29.243876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:53:55.325321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"81'4","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.264802+0000","last_change":"2023-12-26T21:52:28.259566+0000","last_active":"2023-12-26T21:52:29.264802+0000","last_peered":"2023-12-26T21:52:29.264802+0000","last_clean":"2023-12-26T21:52:29.264802+0000","last_became_active":"2023-12-26T21:52:28.259103+0000","last_became_peered":"2023-12-26T21:52:28.259103+0000","last_unstale":"2023-12-26T21:52:29.264802+0000","last_undegraded":"2023-12-26T21:52:29.264802+0000","last_fullsized":"2023-12-26T21:52:29.264802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:54:46.763590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246598+0000","last_change":"2023-12-26T21:52:29.240686+0000","last_active":"2023-12-26T21:52:29.246598+0000","last_peered":"2023-12-26T21:52:29.246598+0000","last_clean":"2023-12-26T21:52:29.246598+0000","last_became_active":"2023-12-26T21:52:29.239023+0000","last_became_peered":"2023-12-26T21:52:29.239023+0000","last_unstale":"2023-12-26T21:52:29.246598+0000","last_undegraded":"2023-12-26T21:52:29.246598+0000","last_fullsized":"2023-12-26T21:52:29.246598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:39:56.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.265464+0000","last_change":"2023-12-26T21:52:28.269154+0000","last_active":"2023-12-26T21:52:29.265464+0000","last_peered":"2023-12-26T21:52:29.265464+0000","last_clean":"2023-12-26T21:52:29.265464+0000","last_became_active":"2023-12-26T21:52:28.268862+0000","last_became_peered":"2023-12-26T21:52:28.268862+0000","last_unstale":"2023-12-26T21:52:29.265464+0000","last_undegraded":"2023-12-26T21:52:29.265464+0000","last_fullsized":"2023-12-26T21:52:29.265464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:04:57.569802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246179+0000","last_change":"2023-12-26T21:52:29.243355+0000","last_active":"2023-12-26T21:52:29.246179+0000","last_peered":"2023-12-26T21:52:29.246179+0000","last_clean":"2023-12-26T21:52:29.246179+0000","last_became_active":"2023-12-26T21:52:29.242576+0000","last_became_peered":"2023-12-26T21:52:29.242576+0000","last_unstale":"2023-12-26T21:52:29.246179+0000","last_undegraded":"2023-12-26T21:52:29.246179+0000","last_fullsized":"2023-12-26T21:52:29.246179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:33:24.589239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263442+0000","last_change":"2023-12-26T21:52:28.258649+0000","last_active":"2023-12-26T21:52:29.263442+0000","last_peered":"2023-12-26T21:52:29.263442+0000","last_clean":"2023-12-26T21:52:29.263442+0000","last_became_active":"2023-12-26T21:52:28.258367+0000","last_became_peered":"2023-12-26T21:52:28.258367+0000","last_unstale":"2023-12-26T21:52:29.263442+0000","last_undegraded":"2023-12-26T21:52:29.263442+0000","last_fullsized":"2023-12-26T21:52:29.263442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:43:08.391704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243589+0000","last_change":"2023-12-26T21:52:29.237552+0000","last_active":"2023-12-26T21:52:29.243589+0000","last_peered":"2023-12-26T21:52:29.243589+0000","last_clean":"2023-12-26T21:52:29.243589+0000","last_became_active":"2023-12-26T21:52:29.236550+0000","last_became_peered":"2023-12-26T21:52:29.236550+0000","last_unstale":"2023-12-26T21:52:29.243589+0000","last_undegraded":"2023-12-26T21:52:29.243589+0000","last_fullsized":"2023-12-26T21:52:29.243589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:36:36.367110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.265627+0000","last_change":"2023-12-26T21:52:28.268778+0000","last_active":"2023-12-26T21:52:29.265627+0000","last_peered":"2023-12-26T21:52:29.265627+0000","last_clean":"2023-12-26T21:52:29.265627+0000","last_became_active":"2023-12-26T21:52:28.268447+0000","last_became_peered":"2023-12-26T21:52:28.268447+0000","last_unstale":"2023-12-26T21:52:29.265627+0000","last_undegraded":"2023-12-26T21:52:29.265627+0000","last_fullsized":"2023-12-26T21:52:29.265627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:03.429070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234552+0000","last_change":"2023-12-26T21:52:29.229890+0000","last_active":"2023-12-26T21:52:29.234552+0000","last_peered":"2023-12-26T21:52:29.234552+0000","last_clean":"2023-12-26T21:52:29.234552+0000","last_became_active":"2023-12-26T21:52:29.224974+0000","last_became_peered":"2023-12-26T21:52:29.224974+0000","last_unstale":"2023-12-26T21:52:29.234552+0000","last_undegraded":"2023-12-26T21:52:29.234552+0000","last_fullsized":"2023-12-26T21:52:29.234552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:13.676363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263619+0000","last_change":"2023-12-26T21:52:28.271068+0000","last_active":"2023-12-26T21:52:29.263619+0000","last_peered":"2023-12-26T21:52:29.263619+0000","last_clean":"2023-12-26T21:52:29.263619+0000","last_became_active":"2023-12-26T21:52:28.270808+0000","last_became_peered":"2023-12-26T21:52:28.270808+0000","last_unstale":"2023-12-26T21:52:29.263619+0000","last_undegraded":"2023-12-26T21:52:29.263619+0000","last_fullsized":"2023-12-26T21:52:29.263619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:54:21.997949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230449+0000","last_change":"2023-12-26T21:52:29.223071+0000","last_active":"2023-12-26T21:52:29.230449+0000","last_peered":"2023-12-26T21:52:29.230449+0000","last_clean":"2023-12-26T21:52:29.230449+0000","last_became_active":"2023-12-26T21:52:29.222766+0000","last_became_peered":"2023-12-26T21:52:29.222766+0000","last_unstale":"2023-12-26T21:52:29.230449+0000","last_undegraded":"2023-12-26T21:52:29.230449+0000","last_fullsized":"2023-12-26T21:52:29.230449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:36.979969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.262265+0000","last_change":"2023-12-26T21:52:28.267697+0000","last_active":"2023-12-26T21:52:29.262265+0000","last_peered":"2023-12-26T21:52:29.262265+0000","last_clean":"2023-12-26T21:52:29.262265+0000","last_became_active":"2023-12-26T21:52:28.266475+0000","last_became_peered":"2023-12-26T21:52:28.266475+0000","last_unstale":"2023-12-26T21:52:29.262265+0000","last_undegraded":"2023-12-26T21:52:29.262265+0000","last_fullsized":"2023-12-26T21:52:29.262265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:33:18.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236112+0000","last_change":"2023-12-26T21:52:29.231360+0000","last_active":"2023-12-26T21:52:29.236112+0000","last_peered":"2023-12-26T21:52:29.236112+0000","last_clean":"2023-12-26T21:52:29.236112+0000","last_became_active":"2023-12-26T21:52:29.230291+0000","last_became_peered":"2023-12-26T21:52:29.230291+0000","last_unstale":"2023-12-26T21:52:29.236112+0000","last_undegraded":"2023-12-26T21:52:29.236112+0000","last_fullsized":"2023-12-26T21:52:29.236112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:23:25.880180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214472+0000","last_change":"2023-12-26T21:52:28.268002+0000","last_active":"2023-12-26T21:52:29.214472+0000","last_peered":"2023-12-26T21:52:29.214472+0000","last_clean":"2023-12-26T21:52:29.214472+0000","last_became_active":"2023-12-26T21:52:28.267679+0000","last_became_peered":"2023-12-26T21:52:28.267679+0000","last_unstale":"2023-12-26T21:52:29.214472+0000","last_undegraded":"2023-12-26T21:52:29.214472+0000","last_fullsized":"2023-12-26T21:52:29.214472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:25:34.060689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.238834+0000","last_change":"2023-12-26T21:52:29.233862+0000","last_active":"2023-12-26T21:52:29.238834+0000","last_peered":"2023-12-26T21:52:29.238834+0000","last_clean":"2023-12-26T21:52:29.238834+0000","last_became_active":"2023-12-26T21:52:29.233574+0000","last_became_peered":"2023-12-26T21:52:29.233574+0000","last_unstale":"2023-12-26T21:52:29.238834+0000","last_undegraded":"2023-12-26T21:52:29.238834+0000","last_fullsized":"2023-12-26T21:52:29.238834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:41:18.260567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219884+0000","last_change":"2023-12-26T21:52:28.269477+0000","last_active":"2023-12-26T21:52:29.219884+0000","last_peered":"2023-12-26T21:52:29.219884+0000","last_clean":"2023-12-26T21:52:29.219884+0000","last_became_active":"2023-12-26T21:52:28.267057+0000","last_became_peered":"2023-12-26T21:52:28.267057+0000","last_unstale":"2023-12-26T21:52:29.219884+0000","last_undegraded":"2023-12-26T21:52:29.219884+0000","last_fullsized":"2023-12-26T21:52:29.219884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:43:47.838722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.235919+0000","last_change":"2023-12-26T21:52:29.231041+0000","last_active":"2023-12-26T21:52:29.235919+0000","last_peered":"2023-12-26T21:52:29.235919+0000","last_clean":"2023-12-26T21:52:29.235919+0000","last_became_active":"2023-12-26T21:52:29.230770+0000","last_became_peered":"2023-12-26T21:52:29.230770+0000","last_unstale":"2023-12-26T21:52:29.235919+0000","last_undegraded":"2023-12-26T21:52:29.235919+0000","last_fullsized":"2023-12-26T21:52:29.235919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:57:54.793636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214500+0000","last_change":"2023-12-26T21:52:28.259979+0000","last_active":"2023-12-26T21:52:29.214500+0000","last_peered":"2023-12-26T21:52:29.214500+0000","last_clean":"2023-12-26T21:52:29.214500+0000","last_became_active":"2023-12-26T21:52:28.259478+0000","last_became_peered":"2023-12-26T21:52:28.259478+0000","last_unstale":"2023-12-26T21:52:29.214500+0000","last_undegraded":"2023-12-26T21:52:29.214500+0000","last_fullsized":"2023-12-26T21:52:29.214500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:56.693294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236087+0000","last_change":"2023-12-26T21:52:29.231187+0000","last_active":"2023-12-26T21:52:29.236087+0000","last_peered":"2023-12-26T21:52:29.236087+0000","last_clean":"2023-12-26T21:52:29.236087+0000","last_became_active":"2023-12-26T21:52:29.230725+0000","last_became_peered":"2023-12-26T21:52:29.230725+0000","last_unstale":"2023-12-26T21:52:29.236087+0000","last_undegraded":"2023-12-26T21:52:29.236087+0000","last_fullsized":"2023-12-26T21:52:29.236087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:14.766932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219825+0000","last_change":"2023-12-26T21:52:28.268948+0000","last_active":"2023-12-26T21:52:29.219825+0000","last_peered":"2023-12-26T21:52:29.219825+0000","last_clean":"2023-12-26T21:52:29.219825+0000","last_became_active":"2023-12-26T21:52:28.268669+0000","last_became_peered":"2023-12-26T21:52:28.268669+0000","last_unstale":"2023-12-26T21:52:29.219825+0000","last_undegraded":"2023-12-26T21:52:29.219825+0000","last_fullsized":"2023-12-26T21:52:29.219825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:20:47.755741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243905+0000","last_change":"2023-12-26T21:52:29.240498+0000","last_active":"2023-12-26T21:52:29.243905+0000","last_peered":"2023-12-26T21:52:29.243905+0000","last_clean":"2023-12-26T21:52:29.243905+0000","last_became_active":"2023-12-26T21:52:29.240230+0000","last_became_peered":"2023-12-26T21:52:29.240230+0000","last_unstale":"2023-12-26T21:52:29.243905+0000","last_undegraded":"2023-12-26T21:52:29.243905+0000","last_fullsized":"2023-12-26T21:52:29.243905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:55:10.541118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"81'3","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.266704+0000","last_change":"2023-12-26T21:52:28.266312+0000","last_active":"2023-12-26T21:52:29.266704+0000","last_peered":"2023-12-26T21:52:29.266704+0000","last_clean":"2023-12-26T21:52:29.266704+0000","last_became_active":"2023-12-26T21:52:28.266052+0000","last_became_peered":"2023-12-26T21:52:28.266052+0000","last_unstale":"2023-12-26T21:52:29.266704+0000","last_undegraded":"2023-12-26T21:52:29.266704+0000","last_fullsized":"2023-12-26T21:52:29.266704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:15:20.270422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.229666+0000","last_change":"2023-12-26T21:52:29.225981+0000","last_active":"2023-12-26T21:52:29.229666+0000","last_peered":"2023-12-26T21:52:29.229666+0000","last_clean":"2023-12-26T21:52:29.229666+0000","last_became_active":"2023-12-26T21:52:29.225707+0000","last_became_peered":"2023-12-26T21:52:29.225707+0000","last_unstale":"2023-12-26T21:52:29.229666+0000","last_undegraded":"2023-12-26T21:52:29.229666+0000","last_fullsized":"2023-12-26T21:52:29.229666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:23:19.236424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"81'6","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.268152+0000","last_change":"2023-12-26T21:52:28.268096+0000","last_active":"2023-12-26T21:52:29.268152+0000","last_peered":"2023-12-26T21:52:29.268152+0000","last_clean":"2023-12-26T21:52:29.268152+0000","last_became_active":"2023-12-26T21:52:28.267796+0000","last_became_peered":"2023-12-26T21:52:28.267796+0000","last_unstale":"2023-12-26T21:52:29.268152+0000","last_undegraded":"2023-12-26T21:52:29.268152+0000","last_fullsized":"2023-12-26T21:52:29.268152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:13:50.063067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.222861+0000","last_change":"2023-12-26T21:52:29.218090+0000","last_active":"2023-12-26T21:52:29.222861+0000","last_peered":"2023-12-26T21:52:29.222861+0000","last_clean":"2023-12-26T21:52:29.222861+0000","last_became_active":"2023-12-26T21:52:29.217682+0000","last_became_peered":"2023-12-26T21:52:29.217682+0000","last_unstale":"2023-12-26T21:52:29.222861+0000","last_undegraded":"2023-12-26T21:52:29.222861+0000","last_fullsized":"2023-12-26T21:52:29.222861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:47:30.792844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.264160+0000","last_change":"2023-12-26T21:52:28.267126+0000","last_active":"2023-12-26T21:52:29.264160+0000","last_peered":"2023-12-26T21:52:29.264160+0000","last_clean":"2023-12-26T21:52:29.264160+0000","last_became_active":"2023-12-26T21:52:28.266873+0000","last_became_peered":"2023-12-26T21:52:28.266873+0000","last_unstale":"2023-12-26T21:52:29.264160+0000","last_undegraded":"2023-12-26T21:52:29.264160+0000","last_fullsized":"2023-12-26T21:52:29.264160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:13:02.730857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237188+0000","last_change":"2023-12-26T21:52:29.235471+0000","last_active":"2023-12-26T21:52:29.237188+0000","last_peered":"2023-12-26T21:52:29.237188+0000","last_clean":"2023-12-26T21:52:29.237188+0000","last_became_active":"2023-12-26T21:52:29.235109+0000","last_became_peered":"2023-12-26T21:52:29.235109+0000","last_unstale":"2023-12-26T21:52:29.237188+0000","last_undegraded":"2023-12-26T21:52:29.237188+0000","last_fullsized":"2023-12-26T21:52:29.237188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:36.979969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.218419+0000","last_change":"2023-12-26T21:52:28.271668+0000","last_active":"2023-12-26T21:52:29.218419+0000","last_peered":"2023-12-26T21:52:29.218419+0000","last_clean":"2023-12-26T21:52:29.218419+0000","last_became_active":"2023-12-26T21:52:28.271367+0000","last_became_peered":"2023-12-26T21:52:28.271367+0000","last_unstale":"2023-12-26T21:52:29.218419+0000","last_undegraded":"2023-12-26T21:52:29.218419+0000","last_fullsized":"2023-12-26T21:52:29.218419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:31.502500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.226304+0000","last_change":"2023-12-26T21:52:29.222412+0000","last_active":"2023-12-26T21:52:29.226304+0000","last_peered":"2023-12-26T21:52:29.226304+0000","last_clean":"2023-12-26T21:52:29.226304+0000","last_became_active":"2023-12-26T21:52:29.222015+0000","last_became_peered":"2023-12-26T21:52:29.222015+0000","last_unstale":"2023-12-26T21:52:29.226304+0000","last_undegraded":"2023-12-26T21:52:29.226304+0000","last_fullsized":"2023-12-26T21:52:29.226304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:59.407409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219359+0000","last_change":"2023-12-26T21:52:28.259455+0000","last_active":"2023-12-26T21:52:29.219359+0000","last_peered":"2023-12-26T21:52:29.219359+0000","last_clean":"2023-12-26T21:52:29.219359+0000","last_became_active":"2023-12-26T21:52:28.258970+0000","last_became_peered":"2023-12-26T21:52:28.258970+0000","last_unstale":"2023-12-26T21:52:29.219359+0000","last_undegraded":"2023-12-26T21:52:29.219359+0000","last_fullsized":"2023-12-26T21:52:29.219359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:16:39.633558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234355+0000","last_change":"2023-12-26T21:52:29.229327+0000","last_active":"2023-12-26T21:52:29.234355+0000","last_peered":"2023-12-26T21:52:29.234355+0000","last_clean":"2023-12-26T21:52:29.234355+0000","last_became_active":"2023-12-26T21:52:29.229049+0000","last_became_peered":"2023-12-26T21:52:29.229049+0000","last_unstale":"2023-12-26T21:52:29.234355+0000","last_undegraded":"2023-12-26T21:52:29.234355+0000","last_fullsized":"2023-12-26T21:52:29.234355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:29:36.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"81'8","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.266860+0000","last_change":"2023-12-26T21:52:28.271790+0000","last_active":"2023-12-26T21:52:29.266860+0000","last_peered":"2023-12-26T21:52:29.266860+0000","last_clean":"2023-12-26T21:52:29.266860+0000","last_became_active":"2023-12-26T21:52:28.271361+0000","last_became_peered":"2023-12-26T21:52:28.271361+0000","last_unstale":"2023-12-26T21:52:29.266860+0000","last_undegraded":"2023-12-26T21:52:29.266860+0000","last_fullsized":"2023-12-26T21:52:29.266860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:13:53.181133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236173+0000","last_change":"2023-12-26T21:52:29.232038+0000","last_active":"2023-12-26T21:52:29.236173+0000","last_peered":"2023-12-26T21:52:29.236173+0000","last_clean":"2023-12-26T21:52:29.236173+0000","last_became_active":"2023-12-26T21:52:29.231778+0000","last_became_peered":"2023-12-26T21:52:29.231778+0000","last_unstale":"2023-12-26T21:52:29.236173+0000","last_undegraded":"2023-12-26T21:52:29.236173+0000","last_fullsized":"2023-12-26T21:52:29.236173+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:08:03.577052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263172+0000","last_change":"2023-12-26T21:52:28.261756+0000","last_active":"2023-12-26T21:52:29.263172+0000","last_peered":"2023-12-26T21:52:29.263172+0000","last_clean":"2023-12-26T21:52:29.263172+0000","last_became_active":"2023-12-26T21:52:28.260555+0000","last_became_peered":"2023-12-26T21:52:28.260555+0000","last_unstale":"2023-12-26T21:52:29.263172+0000","last_undegraded":"2023-12-26T21:52:29.263172+0000","last_fullsized":"2023-12-26T21:52:29.263172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:48.534207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237099+0000","last_change":"2023-12-26T21:52:29.233600+0000","last_active":"2023-12-26T21:52:29.237099+0000","last_peered":"2023-12-26T21:52:29.237099+0000","last_clean":"2023-12-26T21:52:29.237099+0000","last_became_active":"2023-12-26T21:52:29.233320+0000","last_became_peered":"2023-12-26T21:52:29.233320+0000","last_unstale":"2023-12-26T21:52:29.237099+0000","last_undegraded":"2023-12-26T21:52:29.237099+0000","last_fullsized":"2023-12-26T21:52:29.237099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:48:58.284911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263975+0000","last_change":"2023-12-26T21:52:28.268837+0000","last_active":"2023-12-26T21:52:29.263975+0000","last_peered":"2023-12-26T21:52:29.263975+0000","last_clean":"2023-12-26T21:52:29.263975+0000","last_became_active":"2023-12-26T21:52:28.268568+0000","last_became_peered":"2023-12-26T21:52:28.268568+0000","last_unstale":"2023-12-26T21:52:29.263975+0000","last_undegraded":"2023-12-26T21:52:29.263975+0000","last_fullsized":"2023-12-26T21:52:29.263975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:57:53.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230852+0000","last_change":"2023-12-26T21:52:29.227912+0000","last_active":"2023-12-26T21:52:29.230852+0000","last_peered":"2023-12-26T21:52:29.230852+0000","last_clean":"2023-12-26T21:52:29.230852+0000","last_became_active":"2023-12-26T21:52:29.227624+0000","last_became_peered":"2023-12-26T21:52:29.227624+0000","last_unstale":"2023-12-26T21:52:29.230852+0000","last_undegraded":"2023-12-26T21:52:29.230852+0000","last_fullsized":"2023-12-26T21:52:29.230852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:02:53.252553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263685+0000","last_change":"2023-12-26T21:52:28.264029+0000","last_active":"2023-12-26T21:52:29.263685+0000","last_peered":"2023-12-26T21:52:29.263685+0000","last_clean":"2023-12-26T21:52:29.263685+0000","last_became_active":"2023-12-26T21:52:28.263753+0000","last_became_peered":"2023-12-26T21:52:28.263753+0000","last_unstale":"2023-12-26T21:52:29.263685+0000","last_undegraded":"2023-12-26T21:52:29.263685+0000","last_fullsized":"2023-12-26T21:52:29.263685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:10.512168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237888+0000","last_change":"2023-12-26T21:52:29.236983+0000","last_active":"2023-12-26T21:52:29.237888+0000","last_peered":"2023-12-26T21:52:29.237888+0000","last_clean":"2023-12-26T21:52:29.237888+0000","last_became_active":"2023-12-26T21:52:29.236707+0000","last_became_peered":"2023-12-26T21:52:29.236707+0000","last_unstale":"2023-12-26T21:52:29.237888+0000","last_undegraded":"2023-12-26T21:52:29.237888+0000","last_fullsized":"2023-12-26T21:52:29.237888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:27:45.114181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219399+0000","last_change":"2023-12-26T21:52:28.263164+0000","last_active":"2023-12-26T21:52:29.219399+0000","last_peered":"2023-12-26T21:52:29.219399+0000","last_clean":"2023-12-26T21:52:29.219399+0000","last_became_active":"2023-12-26T21:52:28.261982+0000","last_became_peered":"2023-12-26T21:52:28.261982+0000","last_unstale":"2023-12-26T21:52:29.219399+0000","last_undegraded":"2023-12-26T21:52:29.219399+0000","last_fullsized":"2023-12-26T21:52:29.219399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:28.819950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.242695+0000","last_change":"2023-12-26T21:52:29.242162+0000","last_active":"2023-12-26T21:52:29.242695+0000","last_peered":"2023-12-26T21:52:29.242695+0000","last_clean":"2023-12-26T21:52:29.242695+0000","last_became_active":"2023-12-26T21:52:29.241931+0000","last_became_peered":"2023-12-26T21:52:29.241931+0000","last_unstale":"2023-12-26T21:52:29.242695+0000","last_undegraded":"2023-12-26T21:52:29.242695+0000","last_fullsized":"2023-12-26T21:52:29.242695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:53:01.470505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.265802+0000","last_change":"2023-12-26T21:52:28.267744+0000","last_active":"2023-12-26T21:52:29.265802+0000","last_peered":"2023-12-26T21:52:29.265802+0000","last_clean":"2023-12-26T21:52:29.265802+0000","last_became_active":"2023-12-26T21:52:28.267302+0000","last_became_peered":"2023-12-26T21:52:28.267302+0000","last_unstale":"2023-12-26T21:52:29.265802+0000","last_undegraded":"2023-12-26T21:52:29.265802+0000","last_fullsized":"2023-12-26T21:52:29.265802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:02:52.245806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232060+0000","last_change":"2023-12-26T21:52:29.223883+0000","last_active":"2023-12-26T21:52:29.232060+0000","last_peered":"2023-12-26T21:52:29.232060+0000","last_clean":"2023-12-26T21:52:29.232060+0000","last_became_active":"2023-12-26T21:52:29.223617+0000","last_became_peered":"2023-12-26T21:52:29.223617+0000","last_unstale":"2023-12-26T21:52:29.232060+0000","last_undegraded":"2023-12-26T21:52:29.232060+0000","last_fullsized":"2023-12-26T21:52:29.232060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:29:36.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.220211+0000","last_change":"2023-12-26T21:52:28.263761+0000","last_active":"2023-12-26T21:52:29.220211+0000","last_peered":"2023-12-26T21:52:29.220211+0000","last_clean":"2023-12-26T21:52:29.220211+0000","last_became_active":"2023-12-26T21:52:28.263516+0000","last_became_peered":"2023-12-26T21:52:28.263516+0000","last_unstale":"2023-12-26T21:52:29.220211+0000","last_undegraded":"2023-12-26T21:52:29.220211+0000","last_fullsized":"2023-12-26T21:52:29.220211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:29:30.763837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232911+0000","last_change":"2023-12-26T21:52:29.227887+0000","last_active":"2023-12-26T21:52:29.232911+0000","last_peered":"2023-12-26T21:52:29.232911+0000","last_clean":"2023-12-26T21:52:29.232911+0000","last_became_active":"2023-12-26T21:52:29.226890+0000","last_became_peered":"2023-12-26T21:52:29.226890+0000","last_unstale":"2023-12-26T21:52:29.232911+0000","last_undegraded":"2023-12-26T21:52:29.232911+0000","last_fullsized":"2023-12-26T21:52:29.232911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:49.740534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214344+0000","last_change":"2023-12-26T21:52:28.263846+0000","last_active":"2023-12-26T21:52:29.214344+0000","last_peered":"2023-12-26T21:52:29.214344+0000","last_clean":"2023-12-26T21:52:29.214344+0000","last_became_active":"2023-12-26T21:52:28.263365+0000","last_became_peered":"2023-12-26T21:52:28.263365+0000","last_unstale":"2023-12-26T21:52:29.214344+0000","last_undegraded":"2023-12-26T21:52:29.214344+0000","last_fullsized":"2023-12-26T21:52:29.214344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:39.729932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236252+0000","last_change":"2023-12-26T21:52:29.232453+0000","last_active":"2023-12-26T21:52:29.236252+0000","last_peered":"2023-12-26T21:52:29.236252+0000","last_clean":"2023-12-26T21:52:29.236252+0000","last_became_active":"2023-12-26T21:52:29.232176+0000","last_became_peered":"2023-12-26T21:52:29.232176+0000","last_unstale":"2023-12-26T21:52:29.236252+0000","last_undegraded":"2023-12-26T21:52:29.236252+0000","last_fullsized":"2023-12-26T21:52:29.236252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:27:29.980110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263179+0000","last_change":"2023-12-26T21:52:28.264224+0000","last_active":"2023-12-26T21:52:29.263179+0000","last_peered":"2023-12-26T21:52:29.263179+0000","last_clean":"2023-12-26T21:52:29.263179+0000","last_became_active":"2023-12-26T21:52:28.263911+0000","last_became_peered":"2023-12-26T21:52:28.263911+0000","last_unstale":"2023-12-26T21:52:29.263179+0000","last_undegraded":"2023-12-26T21:52:29.263179+0000","last_fullsized":"2023-12-26T21:52:29.263179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:52:28.566410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236338+0000","last_change":"2023-12-26T21:52:29.230971+0000","last_active":"2023-12-26T21:52:29.236338+0000","last_peered":"2023-12-26T21:52:29.236338+0000","last_clean":"2023-12-26T21:52:29.236338+0000","last_became_active":"2023-12-26T21:52:29.230403+0000","last_became_peered":"2023-12-26T21:52:29.230403+0000","last_unstale":"2023-12-26T21:52:29.236338+0000","last_undegraded":"2023-12-26T21:52:29.236338+0000","last_fullsized":"2023-12-26T21:52:29.236338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:16:35.642844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.239044+0000","last_change":"2023-12-26T21:52:28.266467+0000","last_active":"2023-12-26T21:52:29.239044+0000","last_peered":"2023-12-26T21:52:29.239044+0000","last_clean":"2023-12-26T21:52:29.239044+0000","last_became_active":"2023-12-26T21:52:28.266175+0000","last_became_peered":"2023-12-26T21:52:28.266175+0000","last_unstale":"2023-12-26T21:52:29.239044+0000","last_undegraded":"2023-12-26T21:52:29.239044+0000","last_fullsized":"2023-12-26T21:52:29.239044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:24:49.091032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230763+0000","last_change":"2023-12-26T21:52:29.224964+0000","last_active":"2023-12-26T21:52:29.230763+0000","last_peered":"2023-12-26T21:52:29.230763+0000","last_clean":"2023-12-26T21:52:29.230763+0000","last_became_active":"2023-12-26T21:52:29.224655+0000","last_became_peered":"2023-12-26T21:52:29.224655+0000","last_unstale":"2023-12-26T21:52:29.230763+0000","last_undegraded":"2023-12-26T21:52:29.230763+0000","last_fullsized":"2023-12-26T21:52:29.230763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:28.557813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214435+0000","last_change":"2023-12-26T21:52:28.265591+0000","last_active":"2023-12-26T21:52:29.214435+0000","last_peered":"2023-12-26T21:52:29.214435+0000","last_clean":"2023-12-26T21:52:29.214435+0000","last_became_active":"2023-12-26T21:52:28.264635+0000","last_became_peered":"2023-12-26T21:52:28.264635+0000","last_unstale":"2023-12-26T21:52:29.214435+0000","last_undegraded":"2023-12-26T21:52:29.214435+0000","last_fullsized":"2023-12-26T21:52:29.214435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:39:00.418215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234459+0000","last_change":"2023-12-26T21:52:29.230344+0000","last_active":"2023-12-26T21:52:29.234459+0000","last_peered":"2023-12-26T21:52:29.234459+0000","last_clean":"2023-12-26T21:52:29.234459+0000","last_became_active":"2023-12-26T21:52:29.229837+0000","last_became_peered":"2023-12-26T21:52:29.229837+0000","last_unstale":"2023-12-26T21:52:29.234459+0000","last_undegraded":"2023-12-26T21:52:29.234459+0000","last_fullsized":"2023-12-26T21:52:29.234459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:37:07.651115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.223505+0000","last_change":"2023-12-26T21:52:28.261351+0000","last_active":"2023-12-26T21:52:29.223505+0000","last_peered":"2023-12-26T21:52:29.223505+0000","last_clean":"2023-12-26T21:52:29.223505+0000","last_became_active":"2023-12-26T21:52:28.261022+0000","last_became_peered":"2023-12-26T21:52:28.261022+0000","last_unstale":"2023-12-26T21:52:29.223505+0000","last_undegraded":"2023-12-26T21:52:29.223505+0000","last_fullsized":"2023-12-26T21:52:29.223505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:49:59.721353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243985+0000","last_change":"2023-12-26T21:52:29.239607+0000","last_active":"2023-12-26T21:52:29.243985+0000","last_peered":"2023-12-26T21:52:29.243985+0000","last_clean":"2023-12-26T21:52:29.243985+0000","last_became_active":"2023-12-26T21:52:29.237151+0000","last_became_peered":"2023-12-26T21:52:29.237151+0000","last_unstale":"2023-12-26T21:52:29.243985+0000","last_undegraded":"2023-12-26T21:52:29.243985+0000","last_fullsized":"2023-12-26T21:52:29.243985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:36:20.817379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.212054+0000","last_change":"2023-12-26T21:52:28.257384+0000","last_active":"2023-12-26T21:52:29.212054+0000","last_peered":"2023-12-26T21:52:29.212054+0000","last_clean":"2023-12-26T21:52:29.212054+0000","last_became_active":"2023-12-26T21:52:28.257082+0000","last_became_peered":"2023-12-26T21:52:28.257082+0000","last_unstale":"2023-12-26T21:52:29.212054+0000","last_undegraded":"2023-12-26T21:52:29.212054+0000","last_fullsized":"2023-12-26T21:52:29.212054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:53.632343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236280+0000","last_change":"2023-12-26T21:52:29.233498+0000","last_active":"2023-12-26T21:52:29.236280+0000","last_peered":"2023-12-26T21:52:29.236280+0000","last_clean":"2023-12-26T21:52:29.236280+0000","last_became_active":"2023-12-26T21:52:29.233226+0000","last_became_peered":"2023-12-26T21:52:29.233226+0000","last_unstale":"2023-12-26T21:52:29.236280+0000","last_undegraded":"2023-12-26T21:52:29.236280+0000","last_fullsized":"2023-12-26T21:52:29.236280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:26:56.203843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.224169+0000","last_change":"2023-12-26T21:52:28.264927+0000","last_active":"2023-12-26T21:52:29.224169+0000","last_peered":"2023-12-26T21:52:29.224169+0000","last_clean":"2023-12-26T21:52:29.224169+0000","last_became_active":"2023-12-26T21:52:28.264616+0000","last_became_peered":"2023-12-26T21:52:28.264616+0000","last_unstale":"2023-12-26T21:52:29.224169+0000","last_undegraded":"2023-12-26T21:52:29.224169+0000","last_fullsized":"2023-12-26T21:52:29.224169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:48:26.750191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234392+0000","last_change":"2023-12-26T21:52:29.229506+0000","last_active":"2023-12-26T21:52:29.234392+0000","last_peered":"2023-12-26T21:52:29.234392+0000","last_clean":"2023-12-26T21:52:29.234392+0000","last_became_active":"2023-12-26T21:52:29.229233+0000","last_became_peered":"2023-12-26T21:52:29.229233+0000","last_unstale":"2023-12-26T21:52:29.234392+0000","last_undegraded":"2023-12-26T21:52:29.234392+0000","last_fullsized":"2023-12-26T21:52:29.234392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:37:05.801449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237302+0000","last_change":"2023-12-26T21:52:28.268070+0000","last_active":"2023-12-26T21:52:29.237302+0000","last_peered":"2023-12-26T21:52:29.237302+0000","last_clean":"2023-12-26T21:52:29.237302+0000","last_became_active":"2023-12-26T21:52:28.265991+0000","last_became_peered":"2023-12-26T21:52:28.265991+0000","last_unstale":"2023-12-26T21:52:29.237302+0000","last_undegraded":"2023-12-26T21:52:29.237302+0000","last_fullsized":"2023-12-26T21:52:29.237302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:30:08.442988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.228656+0000","last_change":"2023-12-26T21:52:29.222923+0000","last_active":"2023-12-26T21:52:29.228656+0000","last_peered":"2023-12-26T21:52:29.228656+0000","last_clean":"2023-12-26T21:52:29.228656+0000","last_became_active":"2023-12-26T21:52:29.222437+0000","last_became_peered":"2023-12-26T21:52:29.222437+0000","last_unstale":"2023-12-26T21:52:29.228656+0000","last_undegraded":"2023-12-26T21:52:29.228656+0000","last_fullsized":"2023-12-26T21:52:29.228656+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:00.728100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219391+0000","last_change":"2023-12-26T21:52:28.266016+0000","last_active":"2023-12-26T21:52:29.219391+0000","last_peered":"2023-12-26T21:52:29.219391+0000","last_clean":"2023-12-26T21:52:29.219391+0000","last_became_active":"2023-12-26T21:52:28.264632+0000","last_became_peered":"2023-12-26T21:52:28.264632+0000","last_unstale":"2023-12-26T21:52:29.219391+0000","last_undegraded":"2023-12-26T21:52:29.219391+0000","last_fullsized":"2023-12-26T21:52:29.219391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:43:06.542037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230809+0000","last_change":"2023-12-26T21:52:29.226505+0000","last_active":"2023-12-26T21:52:29.230809+0000","last_peered":"2023-12-26T21:52:29.230809+0000","last_clean":"2023-12-26T21:52:29.230809+0000","last_became_active":"2023-12-26T21:52:29.226215+0000","last_became_peered":"2023-12-26T21:52:29.226215+0000","last_unstale":"2023-12-26T21:52:29.230809+0000","last_undegraded":"2023-12-26T21:52:29.230809+0000","last_fullsized":"2023-12-26T21:52:29.230809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:01:54.460352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.220332+0000","last_change":"2023-12-26T21:52:28.266993+0000","last_active":"2023-12-26T21:52:29.220332+0000","last_peered":"2023-12-26T21:52:29.220332+0000","last_clean":"2023-12-26T21:52:29.220332+0000","last_became_active":"2023-12-26T21:52:28.266697+0000","last_became_peered":"2023-12-26T21:52:28.266697+0000","last_unstale":"2023-12-26T21:52:29.220332+0000","last_undegraded":"2023-12-26T21:52:29.220332+0000","last_fullsized":"2023-12-26T21:52:29.220332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:30.364550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"71'161","reported_seq":374,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214361+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:52:29.214361+0000","last_peered":"2023-12-26T21:52:29.214361+0000","last_clean":"2023-12-26T21:52:29.214361+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:52:29.214361+0000","last_undegraded":"2023-12-26T21:52:29.214361+0000","last_fullsized":"2023-12-26T21:52:29.214361+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542335,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298256,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073584,"statfs":{"total":96636764160,"available":96331350016,"internally_reserved":0,"allocated":991232,"data_stored":470993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.647},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.61199999999999999,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.92300000000000004,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.74199999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":1,"15min":2.0150000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.58199999999999996,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.014,"5min":1.048,"15min":1.1359999999999999},"last":0.72199999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.59699999999999998,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":1.038,"15min":1.8640000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.498,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.419},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":0.93799999999999994,"5min":1.0189999999999999,"15min":1.327},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.65000000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.393},"last":0.68400000000000005}]},{"osd":4,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.374,"15min":0.371},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.53000000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70399999999999996}]}]},{"osd":6,"up_from":15,"seq":64424509631,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302348,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069492,"statfs":{"total":96636764160,"available":96327159808,"internally_reserved":0,"allocated":987136,"data_stored":470923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":1.0029999999999999,"15min":1.085},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.3380000000000001},"last":0.75900000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.88700000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":1.288},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.876,"5min":0.92000000000000004,"15min":1.113},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.92000000000000004,"5min":1.026,"15min":1.569},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.97799999999999998,"15min":1.3839999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.62,"15min":0.62},"min":{"1min":0.42699999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.137},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.56000000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.69599999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95099999999999996,"5min":1.0049999999999999,"15min":1.296},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.437},"max":{"1min":0.89200000000000002,"5min":0.96899999999999997,"15min":1.51},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.91700000000000004,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.57799999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509631,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307024,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064816,"statfs":{"total":96636764160,"available":96322371584,"internally_reserved":0,"allocated":1581056,"data_stored":1061823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.45600000000000002,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":0.68700000000000006,"5min":0.70299999999999996,"15min":0.71799999999999997},"last":1.6020000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.39000000000000001,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.68000000000000005,"5min":0.754,"15min":4.6600000000000001},"last":1.353}]},{"osd":2,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32100000000000001},"max":{"1min":0.625,"5min":0.625,"15min":2.2869999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":4.3639999999999999},"last":1.488}]},{"osd":3,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":0.871,"15min":4.3259999999999996},"last":1.4379999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.57099999999999995},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.33300000000000002},"max":{"1min":0.72799999999999998,"5min":0.83799999999999997,"15min":4.548},"last":1.506}]},{"osd":4,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.749,"15min":0.752},"min":{"1min":0.57899999999999996,"5min":0.505,"15min":0.505},"max":{"1min":0.88500000000000001,"5min":1.252,"15min":1.252},"last":1.4670000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72599999999999998,"15min":0.74399999999999999},"min":{"1min":0.56699999999999995,"5min":0.47799999999999998,"15min":0.441},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":5.5919999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.53800000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.014,"5min":1.2949999999999999,"15min":1.345},"last":1.5760000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.03,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.5269999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.63800000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91800000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.80400000000000005,"15min":0.80900000000000005},"min":{"1min":0.66900000000000004,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.615}]},{"osd":7,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.78300000000000003},"min":{"1min":0.56699999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.0980000000000001,"15min":1.1539999999999999},"last":2.3460000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77700000000000002,"15min":0.79200000000000004},"min":{"1min":0.67000000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.99399999999999999,"5min":1.3620000000000001,"15min":1.3919999999999999},"last":4.3319999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302356,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069484,"statfs":{"total":96636764160,"available":96327151616,"internally_reserved":0,"allocated":995328,"data_stored":472437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.439},"min":{"1min":0.316,"5min":0.245,"15min":0.245},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":1.5640000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69999999999999996,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.371,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.58599999999999997,"5min":0.66800000000000004,"15min":2.012},"last":0.496},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.47699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.42299999999999999,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":4.2610000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.41799999999999998,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.64000000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.94699999999999995,"5min":1.1539999999999999,"15min":4.423},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.95799999999999996,"5min":1.069,"15min":1.6439999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78200000000000003,"15min":0.80900000000000005},"min":{"1min":0.57799999999999996,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":4.3650000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.008,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.62}]},{"osd":6,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76700000000000002,"15min":0.78500000000000003},"min":{"1min":0.623,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.89000000000000001,"5min":1.103,"15min":1.611},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.55200000000000005,"5min":0.504,"15min":0.5},"max":{"1min":1.012,"5min":1.111,"15min":4.407},"last":0.80800000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.81000000000000005},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":1.145,"15min":4.5069999999999997},"last":0.53000000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542336,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302364,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069476,"statfs":{"total":96636764160,"available":96327143424,"internally_reserved":0,"allocated":1003520,"data_stored":472597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45800000000000002,"15min":0.45900000000000002},"min":{"1min":0.33200000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":1.915},"last":0.44},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.30399999999999999},"max":{"1min":0.63200000000000001,"5min":0.83899999999999997,"15min":1.7729999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.39600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":1.952},"last":0.498},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47999999999999998},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.30199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.375},"max":{"1min":1,"5min":1,"15min":1},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73899999999999999,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.34599999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.54600000000000004,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.96999999999999997,"5min":0.98299999999999998,"15min":1.284},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.60099999999999998,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.0600000000000001,"15min":1.5029999999999999},"last":1.024},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":0.94599999999999995,"5min":1.014,"15min":1.355},"last":0.749}]},{"osd":6,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.93000000000000005,"5min":1.038,"15min":1.7350000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.95999999999999996,"5min":1.0609999999999999,"15min":1.349},"last":0.88200000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.47799999999999998},"max":{"1min":0.92700000000000005,"5min":1.073,"15min":1.841},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0189999999999999,"15min":1.294},"last":0.999}]}]},{"osd":3,"up_from":15,"seq":64424509632,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306432,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065408,"statfs":{"total":96636764160,"available":96322977792,"internally_reserved":0,"allocated":974848,"data_stored":464311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61599999999999999,"5min":0.70699999999999996,"15min":0.91400000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.47999999999999998},"min":{"1min":0.38500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71499999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.35199999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46899999999999997,"15min":0.46500000000000002},"min":{"1min":0.35999999999999999,"5min":0.31900000000000001,"15min":0.30399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.77200000000000002},"last":0.372},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":0.69599999999999995,"5min":0.72499999999999998,"15min":0.99399999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.81799999999999995,"5min":0.85599999999999998,"15min":1.0820000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.621,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.40300000000000002},"max":{"1min":0.85599999999999998,"5min":0.94799999999999995,"15min":1.3580000000000001},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.879,"5min":1.105,"15min":1.3779999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.751,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":0.94299999999999995,"5min":1.1870000000000001,"15min":1.2949999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77500000000000002,"15min":0.76800000000000002},"min":{"1min":0.624,"5min":0.56399999999999995,"15min":0.47699999999999998},"max":{"1min":0.98499999999999999,"5min":1.27,"15min":1.4610000000000001},"last":0.745},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.45900000000000002},"max":{"1min":0.88100000000000001,"5min":0.997,"15min":1.9239999999999999},"last":0.67500000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.56799999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96399999999999997,"5min":1.1950000000000001,"15min":1.5049999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.81200000000000006},"min":{"1min":0.65600000000000003,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.02,"5min":1.0760000000000001,"15min":1.579},"last":0.71299999999999997}]},{"osd":7,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.49299999999999999},"max":{"1min":0.96999999999999997,"5min":1.0640000000000001,"15min":1.964},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.48499999999999999},"max":{"1min":0.97399999999999998,"5min":1.1499999999999999,"15min":1.6359999999999999},"last":0.75700000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542335,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302352,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069488,"statfs":{"total":96636764160,"available":96327155712,"internally_reserved":0,"allocated":991232,"data_stored":471489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66000000000000003,"15min":0.68400000000000005},"min":{"1min":0.439,"5min":0.307,"15min":0.307},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":2.1389999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.96399999999999997,"5min":1.008,"15min":1.407},"last":0.81100000000000005}]},{"osd":1,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.52700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.879},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69799999999999995,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.89200000000000002,"5min":0.98699999999999999,"15min":2.097},"last":0.83499999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70999999999999996,"15min":0.72199999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91000000000000003,"5min":0.94999999999999996,"15min":1.7809999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88800000000000001,"5min":0.97199999999999998,"15min":1.6870000000000001},"last":0.88900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69999999999999996,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95999999999999996,"15min":1.956},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.746},"min":{"1min":0.626,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.0609999999999999},"last":0.72999999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58899999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82099999999999995,"5min":0.86399999999999999,"15min":2.2429999999999999},"last":0.746},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84899999999999998,"5min":0.92600000000000005,"15min":2.194},"last":0.77800000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.621,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.379},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":2.1589999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":0.79200000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.432},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":2.23},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.745},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":2.5169999999999999},"last":0.877}]}]},{"osd":5,"up_from":15,"seq":64424509630,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302928,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068912,"statfs":{"total":96636764160,"available":96326565888,"internally_reserved":0,"allocated":1581056,"data_stored":1061303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.878,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.71299999999999997},"min":{"1min":0.48899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.71},"last":0.72699999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.57399999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96399999999999997,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68200000000000005,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.002,"15min":1.4219999999999999},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.871,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68799999999999994,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":1.214},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":1.052,"15min":1.4890000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":0.92200000000000004,"15min":1.1439999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84399999999999997,"5min":1.0089999999999999,"15min":2.5070000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.73899999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.92400000000000004,"15min":1.637},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.042,"15min":2.5579999999999998},"last":0.754}]},{"osd":7,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":1.248},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":0.95199999999999996,"15min":1.5980000000000001},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:52:33.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:52:33.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.836+0000 7ff3c91ff700 1 Processor -- start 2023-12-26T21:52:33.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.836+0000 7ff3c91ff700 1 -- start start 2023-12-26T21:52:33.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.837+0000 7ff3c91ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 0x7ff3c40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c91ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3c40b19f0 0x7ff3c4161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c91ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c4157a80 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c91ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c40cac50 con 0x7ff3c40b19f0 2023-12-26T21:52:33.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c91ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40caf30 con 0x7ff3c40c8820 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c359e700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3c40b19f0 0x7ff3c4161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c359e700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3c40b19f0 0x7ff3c4161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33812/0 (socket says 172.21.15.131:33812) 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c359e700 1 -- 172.21.15.131:0/3848834389 learned_addr learned my addr 172.21.15.131:0/3848834389 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c2d9d700 1 --2- 172.21.15.131:0/3848834389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 0x7ff3c40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c359e700 1 -- 172.21.15.131:0/3848834389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 msgr2=0x7ff3c40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c359e700 1 --2- 172.21.15.131:0/3848834389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 0x7ff3c40ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c359e700 1 -- 172.21.15.131:0/3848834389 >> v1:172.21.15.190:6789/0 conn(0x7ff3c40c8bf0 legacy=0x7ff3c40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c359e700 1 -- 172.21.15.131:0/3848834389 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40cb210 con 0x7ff3c40b19f0 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.838+0000 7ff3c2d9d700 1 --2- 172.21.15.131:0/3848834389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 0x7ff3c40ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c359e700 1 --2- 172.21.15.131:0/3848834389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40b19f0 0x7ff3c4161090 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009f20 tx=0x7ff3bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd1c57322affbbc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c1d9b700 1 -- 172.21.15.131:0/3848834389 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00e070 con 0x7ff3c40b19f0 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c1d9b700 1 -- 172.21.15.131:0/3848834389 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc00ba80 con 0x7ff3c40b19f0 2023-12-26T21:52:33.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c1d9b700 1 -- 172.21.15.131:0/3848834389 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0092e0 con 0x7ff3c40b19f0 2023-12-26T21:52:33.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c91ff700 1 -- 172.21.15.131:0/3848834389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40b19f0 msgr2=0x7ff3c4161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c91ff700 1 --2- 172.21.15.131:0/3848834389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40b19f0 0x7ff3c4161090 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009f20 tx=0x7ff3bc009840 comp rx=0 tx=0).stop 2023-12-26T21:52:33.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c91ff700 1 -- 172.21.15.131:0/3848834389 shutdown_connections 2023-12-26T21:52:33.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c91ff700 1 --2- 172.21.15.131:0/3848834389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40b19f0 0x7ff3c4161090 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c91ff700 1 --2- 172.21.15.131:0/3848834389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 0x7ff3c40ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.839+0000 7ff3c91ff700 1 -- 172.21.15.131:0/3848834389 >> 172.21.15.131:0/3848834389 conn(0x7ff3c4068040 msgr2=0x7ff3c4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:33.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.840+0000 7ff3c91ff700 1 -- 172.21.15.131:0/3848834389 shutdown_connections 2023-12-26T21:52:33.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.840+0000 7ff3c91ff700 1 -- 172.21.15.131:0/3848834389 wait complete. 2023-12-26T21:52:33.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.842+0000 7ff3c91ff700 1 Processor -- start 2023-12-26T21:52:33.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.842+0000 7ff3c91ff700 1 -- start start 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.842+0000 7ff3c91ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3c40b19f0 0x7ff3c4144b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c91ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 0x7ff3c40c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c91ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40c8bf0 0x7ff3c40c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c91ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40c2ea0 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c91ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40c2bc0 con 0x7ff3c40b19f0 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c91ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40c43a0 con 0x7ff3c40c8820 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40c8bf0 0x7ff3c40c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40c8bf0 0x7ff3c40c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33824/0 (socket says 172.21.15.131:33824) 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c359e700 1 -- 172.21.15.131:0/2913731350 learned_addr learned my addr 172.21.15.131:0/2913731350 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c2d9d700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3c40b19f0 0x7ff3c4144b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c259c700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 0x7ff3c40c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c359e700 1 -- 172.21.15.131:0/2913731350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 msgr2=0x7ff3c40c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c359e700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 0x7ff3c40c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c359e700 1 -- 172.21.15.131:0/2913731350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3c40b19f0 msgr2=0x7ff3c4144b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:33.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c359e700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3c40b19f0 0x7ff3c4144b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:33.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.843+0000 7ff3c359e700 1 -- 172.21.15.131:0/2913731350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc009ae0 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.844+0000 7ff3c359e700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40c8bf0 0x7ff3c40c7050 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40688f0 tx=0x7ff3bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:33.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.844+0000 7ff3aaffd700 1 -- 172.21.15.131:0/2913731350 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0032f0 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.844+0000 7ff3aaffd700 1 -- 172.21.15.131:0/2913731350 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc008d20 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.844+0000 7ff3aaffd700 1 -- 172.21.15.131:0/2913731350 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc01c640 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.844+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c40c4680 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.844+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c41450b0 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.845+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c40c4200 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.846+0000 7ff3aaffd700 1 -- 172.21.15.131:0/2913731350 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff3bc003040 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.847+0000 7ff3aaffd700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff3b0066f30 0x7ff3b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:33.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.847+0000 7ff3aaffd700 1 -- 172.21.15.131:0/2913731350 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff3bc085df0 con 0x7ff3c40c8bf0 2023-12-26T21:52:33.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.847+0000 7ff3c2d9d700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff3b0066f30 0x7ff3b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:33.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.848+0000 7ff3c2d9d700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff3b0066f30 0x7ff3b00693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4006f00 tx=0x7ff3b4008040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:33.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:33.851+0000 7ff3aaffd700 1 -- 172.21.15.131:0/2913731350 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3c40c4200 con 0x7ff3c40c8bf0 2023-12-26T21:52:34.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.040+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 --> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3c40c4200 con 0x7ff3b0066f30 2023-12-26T21:52:34.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.048+0000 7ff3aaffd700 1 -- 172.21.15.131:0/2913731350 <== mgr.6045 v2:172.21.15.131:6835/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288873 (secure 0 0 0) 0x7ff3c40c4200 con 0x7ff3b0066f30 2023-12-26T21:52:34.053 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:34.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff3b0066f30 msgr2=0x7ff3b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff3b0066f30 0x7ff3b00693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4006f00 tx=0x7ff3b4008040 comp rx=0 tx=0).stop 2023-12-26T21:52:34.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40c8bf0 msgr2=0x7ff3c40c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40c8bf0 0x7ff3c40c7050 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40688f0 tx=0x7ff3bc018000 comp rx=0 tx=0).stop 2023-12-26T21:52:34.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 shutdown_connections 2023-12-26T21:52:34.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c40c8bf0 0x7ff3c40c7050 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff3b0066f30 0x7ff3b00693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3c40b19f0 0x7ff3c4144b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 --2- 172.21.15.131:0/2913731350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3c40c8820 0x7ff3c40c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 >> 172.21.15.131:0/2913731350 conn(0x7ff3c4068040 msgr2=0x7ff3c4159090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:34.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 shutdown_connections 2023-12-26T21:52:34.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.053+0000 7ff3c91ff700 1 -- 172.21.15.131:0/2913731350 wait complete. 2023-12-26T21:52:34.056 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:52:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-26T21:52:32.812828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424060,"kb_used_data":8892,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752550660,"statfs":{"total":773094113280,"available":770611875840,"internally_reserved":0,"allocated":9105408,"data_stored":4945876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.576822"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246618+0000","last_change":"2023-12-26T21:52:29.240921+0000","last_active":"2023-12-26T21:52:29.246618+0000","last_peered":"2023-12-26T21:52:29.246618+0000","last_clean":"2023-12-26T21:52:29.246618+0000","last_became_active":"2023-12-26T21:52:29.240384+0000","last_became_peered":"2023-12-26T21:52:29.240384+0000","last_unstale":"2023-12-26T21:52:29.246618+0000","last_undegraded":"2023-12-26T21:52:29.246618+0000","last_fullsized":"2023-12-26T21:52:29.246618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:01:40.250956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.241258+0000","last_change":"2023-12-26T21:52:29.237095+0000","last_active":"2023-12-26T21:52:29.241258+0000","last_peered":"2023-12-26T21:52:29.241258+0000","last_clean":"2023-12-26T21:52:29.241258+0000","last_became_active":"2023-12-26T21:52:29.236784+0000","last_became_peered":"2023-12-26T21:52:29.236784+0000","last_unstale":"2023-12-26T21:52:29.241258+0000","last_undegraded":"2023-12-26T21:52:29.241258+0000","last_fullsized":"2023-12-26T21:52:29.241258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:29:40.151871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236144+0000","last_change":"2023-12-26T21:52:29.231652+0000","last_active":"2023-12-26T21:52:29.236144+0000","last_peered":"2023-12-26T21:52:29.236144+0000","last_clean":"2023-12-26T21:52:29.236144+0000","last_became_active":"2023-12-26T21:52:29.230905+0000","last_became_peered":"2023-12-26T21:52:29.230905+0000","last_unstale":"2023-12-26T21:52:29.236144+0000","last_undegraded":"2023-12-26T21:52:29.236144+0000","last_fullsized":"2023-12-26T21:52:29.236144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:35:56.220697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.241041+0000","last_change":"2023-12-26T21:52:29.240126+0000","last_active":"2023-12-26T21:52:29.241041+0000","last_peered":"2023-12-26T21:52:29.241041+0000","last_clean":"2023-12-26T21:52:29.241041+0000","last_became_active":"2023-12-26T21:52:29.239884+0000","last_became_peered":"2023-12-26T21:52:29.239884+0000","last_unstale":"2023-12-26T21:52:29.241041+0000","last_undegraded":"2023-12-26T21:52:29.241041+0000","last_fullsized":"2023-12-26T21:52:29.241041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:17:52.678820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243707+0000","last_change":"2023-12-26T21:52:29.238194+0000","last_active":"2023-12-26T21:52:29.243707+0000","last_peered":"2023-12-26T21:52:29.243707+0000","last_clean":"2023-12-26T21:52:29.243707+0000","last_became_active":"2023-12-26T21:52:29.237767+0000","last_became_peered":"2023-12-26T21:52:29.237767+0000","last_unstale":"2023-12-26T21:52:29.243707+0000","last_undegraded":"2023-12-26T21:52:29.243707+0000","last_fullsized":"2023-12-26T21:52:29.243707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:31.371297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243964+0000","last_change":"2023-12-26T21:52:29.239551+0000","last_active":"2023-12-26T21:52:29.243964+0000","last_peered":"2023-12-26T21:52:29.243964+0000","last_clean":"2023-12-26T21:52:29.243964+0000","last_became_active":"2023-12-26T21:52:29.239254+0000","last_became_peered":"2023-12-26T21:52:29.239254+0000","last_unstale":"2023-12-26T21:52:29.243964+0000","last_undegraded":"2023-12-26T21:52:29.243964+0000","last_fullsized":"2023-12-26T21:52:29.243964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:13:51.069814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234379+0000","last_change":"2023-12-26T21:52:29.229562+0000","last_active":"2023-12-26T21:52:29.234379+0000","last_peered":"2023-12-26T21:52:29.234379+0000","last_clean":"2023-12-26T21:52:29.234379+0000","last_became_active":"2023-12-26T21:52:29.229306+0000","last_became_peered":"2023-12-26T21:52:29.229306+0000","last_unstale":"2023-12-26T21:52:29.234379+0000","last_undegraded":"2023-12-26T21:52:29.234379+0000","last_fullsized":"2023-12-26T21:52:29.234379+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:44:47.982985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234410+0000","last_change":"2023-12-26T21:52:29.230927+0000","last_active":"2023-12-26T21:52:29.234410+0000","last_peered":"2023-12-26T21:52:29.234410+0000","last_clean":"2023-12-26T21:52:29.234410+0000","last_became_active":"2023-12-26T21:52:29.230645+0000","last_became_peered":"2023-12-26T21:52:29.230645+0000","last_unstale":"2023-12-26T21:52:29.234410+0000","last_undegraded":"2023-12-26T21:52:29.234410+0000","last_fullsized":"2023-12-26T21:52:29.234410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:45:47.041678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243955+0000","last_change":"2023-12-26T21:52:29.243401+0000","last_active":"2023-12-26T21:52:29.243955+0000","last_peered":"2023-12-26T21:52:29.243955+0000","last_clean":"2023-12-26T21:52:29.243955+0000","last_became_active":"2023-12-26T21:52:29.243193+0000","last_became_peered":"2023-12-26T21:52:29.243193+0000","last_unstale":"2023-12-26T21:52:29.243955+0000","last_undegraded":"2023-12-26T21:52:29.243955+0000","last_fullsized":"2023-12-26T21:52:29.243955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:59:56.264143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.241147+0000","last_change":"2023-12-26T21:52:29.236872+0000","last_active":"2023-12-26T21:52:29.241147+0000","last_peered":"2023-12-26T21:52:29.241147+0000","last_clean":"2023-12-26T21:52:29.241147+0000","last_became_active":"2023-12-26T21:52:29.236469+0000","last_became_peered":"2023-12-26T21:52:29.236469+0000","last_unstale":"2023-12-26T21:52:29.241147+0000","last_undegraded":"2023-12-26T21:52:29.241147+0000","last_fullsized":"2023-12-26T21:52:29.241147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:03:02.542674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.242414+0000","last_change":"2023-12-26T21:52:29.236719+0000","last_active":"2023-12-26T21:52:29.242414+0000","last_peered":"2023-12-26T21:52:29.242414+0000","last_clean":"2023-12-26T21:52:29.242414+0000","last_became_active":"2023-12-26T21:52:29.236462+0000","last_became_peered":"2023-12-26T21:52:29.236462+0000","last_unstale":"2023-12-26T21:52:29.242414+0000","last_undegraded":"2023-12-26T21:52:29.242414+0000","last_fullsized":"2023-12-26T21:52:29.242414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:27:29.980110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234590+0000","last_change":"2023-12-26T21:52:29.227571+0000","last_active":"2023-12-26T21:52:29.234590+0000","last_peered":"2023-12-26T21:52:29.234590+0000","last_clean":"2023-12-26T21:52:29.234590+0000","last_became_active":"2023-12-26T21:52:29.227147+0000","last_became_peered":"2023-12-26T21:52:29.227147+0000","last_unstale":"2023-12-26T21:52:29.234590+0000","last_undegraded":"2023-12-26T21:52:29.234590+0000","last_fullsized":"2023-12-26T21:52:29.234590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:17.417194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.231197+0000","last_change":"2023-12-26T21:52:29.226381+0000","last_active":"2023-12-26T21:52:29.231197+0000","last_peered":"2023-12-26T21:52:29.231197+0000","last_clean":"2023-12-26T21:52:29.231197+0000","last_became_active":"2023-12-26T21:52:29.226080+0000","last_became_peered":"2023-12-26T21:52:29.226080+0000","last_unstale":"2023-12-26T21:52:29.231197+0000","last_undegraded":"2023-12-26T21:52:29.231197+0000","last_fullsized":"2023-12-26T21:52:29.231197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:20:48.762488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237142+0000","last_change":"2023-12-26T21:52:29.233635+0000","last_active":"2023-12-26T21:52:29.237142+0000","last_peered":"2023-12-26T21:52:29.237142+0000","last_clean":"2023-12-26T21:52:29.237142+0000","last_became_active":"2023-12-26T21:52:29.233336+0000","last_became_peered":"2023-12-26T21:52:29.233336+0000","last_unstale":"2023-12-26T21:52:29.237142+0000","last_undegraded":"2023-12-26T21:52:29.237142+0000","last_fullsized":"2023-12-26T21:52:29.237142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:22:55.692487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.238180+0000","last_change":"2023-12-26T21:52:29.236940+0000","last_active":"2023-12-26T21:52:29.238180+0000","last_peered":"2023-12-26T21:52:29.238180+0000","last_clean":"2023-12-26T21:52:29.238180+0000","last_became_active":"2023-12-26T21:52:29.236599+0000","last_became_peered":"2023-12-26T21:52:29.236599+0000","last_unstale":"2023-12-26T21:52:29.238180+0000","last_undegraded":"2023-12-26T21:52:29.238180+0000","last_fullsized":"2023-12-26T21:52:29.238180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:29.826697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243822+0000","last_change":"2023-12-26T21:52:29.238566+0000","last_active":"2023-12-26T21:52:29.243822+0000","last_peered":"2023-12-26T21:52:29.243822+0000","last_clean":"2023-12-26T21:52:29.243822+0000","last_became_active":"2023-12-26T21:52:29.236840+0000","last_became_peered":"2023-12-26T21:52:29.236840+0000","last_unstale":"2023-12-26T21:52:29.243822+0000","last_undegraded":"2023-12-26T21:52:29.243822+0000","last_fullsized":"2023-12-26T21:52:29.243822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:30:58.136825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243634+0000","last_change":"2023-12-26T21:52:29.238113+0000","last_active":"2023-12-26T21:52:29.243634+0000","last_peered":"2023-12-26T21:52:29.243634+0000","last_clean":"2023-12-26T21:52:29.243634+0000","last_became_active":"2023-12-26T21:52:29.237730+0000","last_became_peered":"2023-12-26T21:52:29.237730+0000","last_unstale":"2023-12-26T21:52:29.243634+0000","last_undegraded":"2023-12-26T21:52:29.243634+0000","last_fullsized":"2023-12-26T21:52:29.243634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:43:48.845469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236295+0000","last_change":"2023-12-26T21:52:29.232603+0000","last_active":"2023-12-26T21:52:29.236295+0000","last_peered":"2023-12-26T21:52:29.236295+0000","last_clean":"2023-12-26T21:52:29.236295+0000","last_became_active":"2023-12-26T21:52:29.232268+0000","last_became_peered":"2023-12-26T21:52:29.232268+0000","last_unstale":"2023-12-26T21:52:29.236295+0000","last_undegraded":"2023-12-26T21:52:29.236295+0000","last_fullsized":"2023-12-26T21:52:29.236295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:16:12.163141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232591+0000","last_change":"2023-12-26T21:52:29.229265+0000","last_active":"2023-12-26T21:52:29.232591+0000","last_peered":"2023-12-26T21:52:29.232591+0000","last_clean":"2023-12-26T21:52:29.232591+0000","last_became_active":"2023-12-26T21:52:29.228926+0000","last_became_peered":"2023-12-26T21:52:29.228926+0000","last_unstale":"2023-12-26T21:52:29.232591+0000","last_undegraded":"2023-12-26T21:52:29.232591+0000","last_fullsized":"2023-12-26T21:52:29.232591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:48:27.756939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246523+0000","last_change":"2023-12-26T21:52:29.241195+0000","last_active":"2023-12-26T21:52:29.246523+0000","last_peered":"2023-12-26T21:52:29.246523+0000","last_clean":"2023-12-26T21:52:29.246523+0000","last_became_active":"2023-12-26T21:52:29.239771+0000","last_became_peered":"2023-12-26T21:52:29.239771+0000","last_unstale":"2023-12-26T21:52:29.246523+0000","last_undegraded":"2023-12-26T21:52:29.246523+0000","last_fullsized":"2023-12-26T21:52:29.246523+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:35:56.220697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234669+0000","last_change":"2023-12-26T21:52:29.229535+0000","last_active":"2023-12-26T21:52:29.234669+0000","last_peered":"2023-12-26T21:52:29.234669+0000","last_clean":"2023-12-26T21:52:29.234669+0000","last_became_active":"2023-12-26T21:52:29.229262+0000","last_became_peered":"2023-12-26T21:52:29.229262+0000","last_unstale":"2023-12-26T21:52:29.234669+0000","last_undegraded":"2023-12-26T21:52:29.234669+0000","last_fullsized":"2023-12-26T21:52:29.234669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:13:03.737604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.231063+0000","last_change":"2023-12-26T21:52:29.228634+0000","last_active":"2023-12-26T21:52:29.231063+0000","last_peered":"2023-12-26T21:52:29.231063+0000","last_clean":"2023-12-26T21:52:29.231063+0000","last_became_active":"2023-12-26T21:52:29.228317+0000","last_became_peered":"2023-12-26T21:52:29.228317+0000","last_unstale":"2023-12-26T21:52:29.231063+0000","last_undegraded":"2023-12-26T21:52:29.231063+0000","last_fullsized":"2023-12-26T21:52:29.231063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:08:03.577052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.229637+0000","last_change":"2023-12-26T21:52:29.224988+0000","last_active":"2023-12-26T21:52:29.229637+0000","last_peered":"2023-12-26T21:52:29.229637+0000","last_clean":"2023-12-26T21:52:29.229637+0000","last_became_active":"2023-12-26T21:52:29.224692+0000","last_became_peered":"2023-12-26T21:52:29.224692+0000","last_unstale":"2023-12-26T21:52:29.229637+0000","last_undegraded":"2023-12-26T21:52:29.229637+0000","last_fullsized":"2023-12-26T21:52:29.229637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:37:05.801449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.242713+0000","last_change":"2023-12-26T21:52:29.242152+0000","last_active":"2023-12-26T21:52:29.242713+0000","last_peered":"2023-12-26T21:52:29.242713+0000","last_clean":"2023-12-26T21:52:29.242713+0000","last_became_active":"2023-12-26T21:52:29.241902+0000","last_became_peered":"2023-12-26T21:52:29.241902+0000","last_unstale":"2023-12-26T21:52:29.242713+0000","last_undegraded":"2023-12-26T21:52:29.242713+0000","last_fullsized":"2023-12-26T21:52:29.242713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:22.606449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246163+0000","last_change":"2023-12-26T21:52:29.241969+0000","last_active":"2023-12-26T21:52:29.246163+0000","last_peered":"2023-12-26T21:52:29.246163+0000","last_clean":"2023-12-26T21:52:29.246163+0000","last_became_active":"2023-12-26T21:52:29.241580+0000","last_became_peered":"2023-12-26T21:52:29.241580+0000","last_unstale":"2023-12-26T21:52:29.246163+0000","last_undegraded":"2023-12-26T21:52:29.246163+0000","last_fullsized":"2023-12-26T21:52:29.246163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:40:31.111452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.233931+0000","last_change":"2023-12-26T21:52:29.228211+0000","last_active":"2023-12-26T21:52:29.233931+0000","last_peered":"2023-12-26T21:52:29.233931+0000","last_clean":"2023-12-26T21:52:29.233931+0000","last_became_active":"2023-12-26T21:52:29.227673+0000","last_became_peered":"2023-12-26T21:52:29.227673+0000","last_unstale":"2023-12-26T21:52:29.233931+0000","last_undegraded":"2023-12-26T21:52:29.233931+0000","last_fullsized":"2023-12-26T21:52:29.233931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:09:11.466566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232612+0000","last_change":"2023-12-26T21:52:29.229577+0000","last_active":"2023-12-26T21:52:29.232612+0000","last_peered":"2023-12-26T21:52:29.232612+0000","last_clean":"2023-12-26T21:52:29.232612+0000","last_became_active":"2023-12-26T21:52:29.229303+0000","last_became_peered":"2023-12-26T21:52:29.229303+0000","last_unstale":"2023-12-26T21:52:29.232612+0000","last_undegraded":"2023-12-26T21:52:29.232612+0000","last_fullsized":"2023-12-26T21:52:29.232612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:54:23.004697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.231591+0000","last_change":"2023-12-26T21:52:29.227033+0000","last_active":"2023-12-26T21:52:29.231591+0000","last_peered":"2023-12-26T21:52:29.231591+0000","last_clean":"2023-12-26T21:52:29.231591+0000","last_became_active":"2023-12-26T21:52:29.226751+0000","last_became_peered":"2023-12-26T21:52:29.226751+0000","last_unstale":"2023-12-26T21:52:29.231591+0000","last_undegraded":"2023-12-26T21:52:29.231591+0000","last_fullsized":"2023-12-26T21:52:29.231591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:44:47.982985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232084+0000","last_change":"2023-12-26T21:52:29.227704+0000","last_active":"2023-12-26T21:52:29.232084+0000","last_peered":"2023-12-26T21:52:29.232084+0000","last_clean":"2023-12-26T21:52:29.232084+0000","last_became_active":"2023-12-26T21:52:29.225182+0000","last_became_peered":"2023-12-26T21:52:29.225182+0000","last_unstale":"2023-12-26T21:52:29.232084+0000","last_undegraded":"2023-12-26T21:52:29.232084+0000","last_fullsized":"2023-12-26T21:52:29.232084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:24:12.574316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.233047+0000","last_change":"2023-12-26T21:52:29.227388+0000","last_active":"2023-12-26T21:52:29.233047+0000","last_peered":"2023-12-26T21:52:29.233047+0000","last_clean":"2023-12-26T21:52:29.233047+0000","last_became_active":"2023-12-26T21:52:29.226885+0000","last_became_peered":"2023-12-26T21:52:29.226885+0000","last_unstale":"2023-12-26T21:52:29.233047+0000","last_undegraded":"2023-12-26T21:52:29.233047+0000","last_fullsized":"2023-12-26T21:52:29.233047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:40.287102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.224129+0000","last_change":"2023-12-26T21:52:29.219629+0000","last_active":"2023-12-26T21:52:29.224129+0000","last_peered":"2023-12-26T21:52:29.224129+0000","last_clean":"2023-12-26T21:52:29.224129+0000","last_became_active":"2023-12-26T21:52:29.218008+0000","last_became_peered":"2023-12-26T21:52:29.218008+0000","last_unstale":"2023-12-26T21:52:29.224129+0000","last_undegraded":"2023-12-26T21:52:29.224129+0000","last_fullsized":"2023-12-26T21:52:29.224129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:11.518915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230615+0000","last_change":"2023-12-26T21:52:29.224053+0000","last_active":"2023-12-26T21:52:29.230615+0000","last_peered":"2023-12-26T21:52:29.230615+0000","last_clean":"2023-12-26T21:52:29.230615+0000","last_became_active":"2023-12-26T21:52:29.223769+0000","last_became_peered":"2023-12-26T21:52:29.223769+0000","last_unstale":"2023-12-26T21:52:29.230615+0000","last_undegraded":"2023-12-26T21:52:29.230615+0000","last_fullsized":"2023-12-26T21:52:29.230615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:23:25.880180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243876+0000","last_change":"2023-12-26T21:52:29.238559+0000","last_active":"2023-12-26T21:52:29.243876+0000","last_peered":"2023-12-26T21:52:29.243876+0000","last_clean":"2023-12-26T21:52:29.243876+0000","last_became_active":"2023-12-26T21:52:29.236886+0000","last_became_peered":"2023-12-26T21:52:29.236886+0000","last_unstale":"2023-12-26T21:52:29.243876+0000","last_undegraded":"2023-12-26T21:52:29.243876+0000","last_fullsized":"2023-12-26T21:52:29.243876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:53:55.325321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"81'4","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.264802+0000","last_change":"2023-12-26T21:52:28.259566+0000","last_active":"2023-12-26T21:52:29.264802+0000","last_peered":"2023-12-26T21:52:29.264802+0000","last_clean":"2023-12-26T21:52:29.264802+0000","last_became_active":"2023-12-26T21:52:28.259103+0000","last_became_peered":"2023-12-26T21:52:28.259103+0000","last_unstale":"2023-12-26T21:52:29.264802+0000","last_undegraded":"2023-12-26T21:52:29.264802+0000","last_fullsized":"2023-12-26T21:52:29.264802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:54:46.763590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246598+0000","last_change":"2023-12-26T21:52:29.240686+0000","last_active":"2023-12-26T21:52:29.246598+0000","last_peered":"2023-12-26T21:52:29.246598+0000","last_clean":"2023-12-26T21:52:29.246598+0000","last_became_active":"2023-12-26T21:52:29.239023+0000","last_became_peered":"2023-12-26T21:52:29.239023+0000","last_unstale":"2023-12-26T21:52:29.246598+0000","last_undegraded":"2023-12-26T21:52:29.246598+0000","last_fullsized":"2023-12-26T21:52:29.246598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:39:56.699397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.265464+0000","last_change":"2023-12-26T21:52:28.269154+0000","last_active":"2023-12-26T21:52:29.265464+0000","last_peered":"2023-12-26T21:52:29.265464+0000","last_clean":"2023-12-26T21:52:29.265464+0000","last_became_active":"2023-12-26T21:52:28.268862+0000","last_became_peered":"2023-12-26T21:52:28.268862+0000","last_unstale":"2023-12-26T21:52:29.265464+0000","last_undegraded":"2023-12-26T21:52:29.265464+0000","last_fullsized":"2023-12-26T21:52:29.265464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:04:57.569802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.246179+0000","last_change":"2023-12-26T21:52:29.243355+0000","last_active":"2023-12-26T21:52:29.246179+0000","last_peered":"2023-12-26T21:52:29.246179+0000","last_clean":"2023-12-26T21:52:29.246179+0000","last_became_active":"2023-12-26T21:52:29.242576+0000","last_became_peered":"2023-12-26T21:52:29.242576+0000","last_unstale":"2023-12-26T21:52:29.246179+0000","last_undegraded":"2023-12-26T21:52:29.246179+0000","last_fullsized":"2023-12-26T21:52:29.246179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:33:24.589239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263442+0000","last_change":"2023-12-26T21:52:28.258649+0000","last_active":"2023-12-26T21:52:29.263442+0000","last_peered":"2023-12-26T21:52:29.263442+0000","last_clean":"2023-12-26T21:52:29.263442+0000","last_became_active":"2023-12-26T21:52:28.258367+0000","last_became_peered":"2023-12-26T21:52:28.258367+0000","last_unstale":"2023-12-26T21:52:29.263442+0000","last_undegraded":"2023-12-26T21:52:29.263442+0000","last_fullsized":"2023-12-26T21:52:29.263442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:43:08.391704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243589+0000","last_change":"2023-12-26T21:52:29.237552+0000","last_active":"2023-12-26T21:52:29.243589+0000","last_peered":"2023-12-26T21:52:29.243589+0000","last_clean":"2023-12-26T21:52:29.243589+0000","last_became_active":"2023-12-26T21:52:29.236550+0000","last_became_peered":"2023-12-26T21:52:29.236550+0000","last_unstale":"2023-12-26T21:52:29.243589+0000","last_undegraded":"2023-12-26T21:52:29.243589+0000","last_fullsized":"2023-12-26T21:52:29.243589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:36:36.367110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.265627+0000","last_change":"2023-12-26T21:52:28.268778+0000","last_active":"2023-12-26T21:52:29.265627+0000","last_peered":"2023-12-26T21:52:29.265627+0000","last_clean":"2023-12-26T21:52:29.265627+0000","last_became_active":"2023-12-26T21:52:28.268447+0000","last_became_peered":"2023-12-26T21:52:28.268447+0000","last_unstale":"2023-12-26T21:52:29.265627+0000","last_undegraded":"2023-12-26T21:52:29.265627+0000","last_fullsized":"2023-12-26T21:52:29.265627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:03.429070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234552+0000","last_change":"2023-12-26T21:52:29.229890+0000","last_active":"2023-12-26T21:52:29.234552+0000","last_peered":"2023-12-26T21:52:29.234552+0000","last_clean":"2023-12-26T21:52:29.234552+0000","last_became_active":"2023-12-26T21:52:29.224974+0000","last_became_peered":"2023-12-26T21:52:29.224974+0000","last_unstale":"2023-12-26T21:52:29.234552+0000","last_undegraded":"2023-12-26T21:52:29.234552+0000","last_fullsized":"2023-12-26T21:52:29.234552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:13.676363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263619+0000","last_change":"2023-12-26T21:52:28.271068+0000","last_active":"2023-12-26T21:52:29.263619+0000","last_peered":"2023-12-26T21:52:29.263619+0000","last_clean":"2023-12-26T21:52:29.263619+0000","last_became_active":"2023-12-26T21:52:28.270808+0000","last_became_peered":"2023-12-26T21:52:28.270808+0000","last_unstale":"2023-12-26T21:52:29.263619+0000","last_undegraded":"2023-12-26T21:52:29.263619+0000","last_fullsized":"2023-12-26T21:52:29.263619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:54:21.997949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230449+0000","last_change":"2023-12-26T21:52:29.223071+0000","last_active":"2023-12-26T21:52:29.230449+0000","last_peered":"2023-12-26T21:52:29.230449+0000","last_clean":"2023-12-26T21:52:29.230449+0000","last_became_active":"2023-12-26T21:52:29.222766+0000","last_became_peered":"2023-12-26T21:52:29.222766+0000","last_unstale":"2023-12-26T21:52:29.230449+0000","last_undegraded":"2023-12-26T21:52:29.230449+0000","last_fullsized":"2023-12-26T21:52:29.230449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:36.979969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.262265+0000","last_change":"2023-12-26T21:52:28.267697+0000","last_active":"2023-12-26T21:52:29.262265+0000","last_peered":"2023-12-26T21:52:29.262265+0000","last_clean":"2023-12-26T21:52:29.262265+0000","last_became_active":"2023-12-26T21:52:28.266475+0000","last_became_peered":"2023-12-26T21:52:28.266475+0000","last_unstale":"2023-12-26T21:52:29.262265+0000","last_undegraded":"2023-12-26T21:52:29.262265+0000","last_fullsized":"2023-12-26T21:52:29.262265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:33:18.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236112+0000","last_change":"2023-12-26T21:52:29.231360+0000","last_active":"2023-12-26T21:52:29.236112+0000","last_peered":"2023-12-26T21:52:29.236112+0000","last_clean":"2023-12-26T21:52:29.236112+0000","last_became_active":"2023-12-26T21:52:29.230291+0000","last_became_peered":"2023-12-26T21:52:29.230291+0000","last_unstale":"2023-12-26T21:52:29.236112+0000","last_undegraded":"2023-12-26T21:52:29.236112+0000","last_fullsized":"2023-12-26T21:52:29.236112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:23:25.880180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214472+0000","last_change":"2023-12-26T21:52:28.268002+0000","last_active":"2023-12-26T21:52:29.214472+0000","last_peered":"2023-12-26T21:52:29.214472+0000","last_clean":"2023-12-26T21:52:29.214472+0000","last_became_active":"2023-12-26T21:52:28.267679+0000","last_became_peered":"2023-12-26T21:52:28.267679+0000","last_unstale":"2023-12-26T21:52:29.214472+0000","last_undegraded":"2023-12-26T21:52:29.214472+0000","last_fullsized":"2023-12-26T21:52:29.214472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:25:34.060689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.238834+0000","last_change":"2023-12-26T21:52:29.233862+0000","last_active":"2023-12-26T21:52:29.238834+0000","last_peered":"2023-12-26T21:52:29.238834+0000","last_clean":"2023-12-26T21:52:29.238834+0000","last_became_active":"2023-12-26T21:52:29.233574+0000","last_became_peered":"2023-12-26T21:52:29.233574+0000","last_unstale":"2023-12-26T21:52:29.238834+0000","last_undegraded":"2023-12-26T21:52:29.238834+0000","last_fullsized":"2023-12-26T21:52:29.238834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:41:18.260567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219884+0000","last_change":"2023-12-26T21:52:28.269477+0000","last_active":"2023-12-26T21:52:29.219884+0000","last_peered":"2023-12-26T21:52:29.219884+0000","last_clean":"2023-12-26T21:52:29.219884+0000","last_became_active":"2023-12-26T21:52:28.267057+0000","last_became_peered":"2023-12-26T21:52:28.267057+0000","last_unstale":"2023-12-26T21:52:29.219884+0000","last_undegraded":"2023-12-26T21:52:29.219884+0000","last_fullsized":"2023-12-26T21:52:29.219884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:43:47.838722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.235919+0000","last_change":"2023-12-26T21:52:29.231041+0000","last_active":"2023-12-26T21:52:29.235919+0000","last_peered":"2023-12-26T21:52:29.235919+0000","last_clean":"2023-12-26T21:52:29.235919+0000","last_became_active":"2023-12-26T21:52:29.230770+0000","last_became_peered":"2023-12-26T21:52:29.230770+0000","last_unstale":"2023-12-26T21:52:29.235919+0000","last_undegraded":"2023-12-26T21:52:29.235919+0000","last_fullsized":"2023-12-26T21:52:29.235919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:57:54.793636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214500+0000","last_change":"2023-12-26T21:52:28.259979+0000","last_active":"2023-12-26T21:52:29.214500+0000","last_peered":"2023-12-26T21:52:29.214500+0000","last_clean":"2023-12-26T21:52:29.214500+0000","last_became_active":"2023-12-26T21:52:28.259478+0000","last_became_peered":"2023-12-26T21:52:28.259478+0000","last_unstale":"2023-12-26T21:52:29.214500+0000","last_undegraded":"2023-12-26T21:52:29.214500+0000","last_fullsized":"2023-12-26T21:52:29.214500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:56.693294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236087+0000","last_change":"2023-12-26T21:52:29.231187+0000","last_active":"2023-12-26T21:52:29.236087+0000","last_peered":"2023-12-26T21:52:29.236087+0000","last_clean":"2023-12-26T21:52:29.236087+0000","last_became_active":"2023-12-26T21:52:29.230725+0000","last_became_peered":"2023-12-26T21:52:29.230725+0000","last_unstale":"2023-12-26T21:52:29.236087+0000","last_undegraded":"2023-12-26T21:52:29.236087+0000","last_fullsized":"2023-12-26T21:52:29.236087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:14.766932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219825+0000","last_change":"2023-12-26T21:52:28.268948+0000","last_active":"2023-12-26T21:52:29.219825+0000","last_peered":"2023-12-26T21:52:29.219825+0000","last_clean":"2023-12-26T21:52:29.219825+0000","last_became_active":"2023-12-26T21:52:28.268669+0000","last_became_peered":"2023-12-26T21:52:28.268669+0000","last_unstale":"2023-12-26T21:52:29.219825+0000","last_undegraded":"2023-12-26T21:52:29.219825+0000","last_fullsized":"2023-12-26T21:52:29.219825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:20:47.755741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243905+0000","last_change":"2023-12-26T21:52:29.240498+0000","last_active":"2023-12-26T21:52:29.243905+0000","last_peered":"2023-12-26T21:52:29.243905+0000","last_clean":"2023-12-26T21:52:29.243905+0000","last_became_active":"2023-12-26T21:52:29.240230+0000","last_became_peered":"2023-12-26T21:52:29.240230+0000","last_unstale":"2023-12-26T21:52:29.243905+0000","last_undegraded":"2023-12-26T21:52:29.243905+0000","last_fullsized":"2023-12-26T21:52:29.243905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:55:10.541118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"81'3","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.266704+0000","last_change":"2023-12-26T21:52:28.266312+0000","last_active":"2023-12-26T21:52:29.266704+0000","last_peered":"2023-12-26T21:52:29.266704+0000","last_clean":"2023-12-26T21:52:29.266704+0000","last_became_active":"2023-12-26T21:52:28.266052+0000","last_became_peered":"2023-12-26T21:52:28.266052+0000","last_unstale":"2023-12-26T21:52:29.266704+0000","last_undegraded":"2023-12-26T21:52:29.266704+0000","last_fullsized":"2023-12-26T21:52:29.266704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:15:20.270422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.229666+0000","last_change":"2023-12-26T21:52:29.225981+0000","last_active":"2023-12-26T21:52:29.229666+0000","last_peered":"2023-12-26T21:52:29.229666+0000","last_clean":"2023-12-26T21:52:29.229666+0000","last_became_active":"2023-12-26T21:52:29.225707+0000","last_became_peered":"2023-12-26T21:52:29.225707+0000","last_unstale":"2023-12-26T21:52:29.229666+0000","last_undegraded":"2023-12-26T21:52:29.229666+0000","last_fullsized":"2023-12-26T21:52:29.229666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:23:19.236424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"81'6","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.268152+0000","last_change":"2023-12-26T21:52:28.268096+0000","last_active":"2023-12-26T21:52:29.268152+0000","last_peered":"2023-12-26T21:52:29.268152+0000","last_clean":"2023-12-26T21:52:29.268152+0000","last_became_active":"2023-12-26T21:52:28.267796+0000","last_became_peered":"2023-12-26T21:52:28.267796+0000","last_unstale":"2023-12-26T21:52:29.268152+0000","last_undegraded":"2023-12-26T21:52:29.268152+0000","last_fullsized":"2023-12-26T21:52:29.268152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:13:50.063067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.222861+0000","last_change":"2023-12-26T21:52:29.218090+0000","last_active":"2023-12-26T21:52:29.222861+0000","last_peered":"2023-12-26T21:52:29.222861+0000","last_clean":"2023-12-26T21:52:29.222861+0000","last_became_active":"2023-12-26T21:52:29.217682+0000","last_became_peered":"2023-12-26T21:52:29.217682+0000","last_unstale":"2023-12-26T21:52:29.222861+0000","last_undegraded":"2023-12-26T21:52:29.222861+0000","last_fullsized":"2023-12-26T21:52:29.222861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:47:30.792844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.264160+0000","last_change":"2023-12-26T21:52:28.267126+0000","last_active":"2023-12-26T21:52:29.264160+0000","last_peered":"2023-12-26T21:52:29.264160+0000","last_clean":"2023-12-26T21:52:29.264160+0000","last_became_active":"2023-12-26T21:52:28.266873+0000","last_became_peered":"2023-12-26T21:52:28.266873+0000","last_unstale":"2023-12-26T21:52:29.264160+0000","last_undegraded":"2023-12-26T21:52:29.264160+0000","last_fullsized":"2023-12-26T21:52:29.264160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:13:02.730857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237188+0000","last_change":"2023-12-26T21:52:29.235471+0000","last_active":"2023-12-26T21:52:29.237188+0000","last_peered":"2023-12-26T21:52:29.237188+0000","last_clean":"2023-12-26T21:52:29.237188+0000","last_became_active":"2023-12-26T21:52:29.235109+0000","last_became_peered":"2023-12-26T21:52:29.235109+0000","last_unstale":"2023-12-26T21:52:29.237188+0000","last_undegraded":"2023-12-26T21:52:29.237188+0000","last_fullsized":"2023-12-26T21:52:29.237188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:36.979969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.218419+0000","last_change":"2023-12-26T21:52:28.271668+0000","last_active":"2023-12-26T21:52:29.218419+0000","last_peered":"2023-12-26T21:52:29.218419+0000","last_clean":"2023-12-26T21:52:29.218419+0000","last_became_active":"2023-12-26T21:52:28.271367+0000","last_became_peered":"2023-12-26T21:52:28.271367+0000","last_unstale":"2023-12-26T21:52:29.218419+0000","last_undegraded":"2023-12-26T21:52:29.218419+0000","last_fullsized":"2023-12-26T21:52:29.218419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:31.502500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.226304+0000","last_change":"2023-12-26T21:52:29.222412+0000","last_active":"2023-12-26T21:52:29.226304+0000","last_peered":"2023-12-26T21:52:29.226304+0000","last_clean":"2023-12-26T21:52:29.226304+0000","last_became_active":"2023-12-26T21:52:29.222015+0000","last_became_peered":"2023-12-26T21:52:29.222015+0000","last_unstale":"2023-12-26T21:52:29.226304+0000","last_undegraded":"2023-12-26T21:52:29.226304+0000","last_fullsized":"2023-12-26T21:52:29.226304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:59.407409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219359+0000","last_change":"2023-12-26T21:52:28.259455+0000","last_active":"2023-12-26T21:52:29.219359+0000","last_peered":"2023-12-26T21:52:29.219359+0000","last_clean":"2023-12-26T21:52:29.219359+0000","last_became_active":"2023-12-26T21:52:28.258970+0000","last_became_peered":"2023-12-26T21:52:28.258970+0000","last_unstale":"2023-12-26T21:52:29.219359+0000","last_undegraded":"2023-12-26T21:52:29.219359+0000","last_fullsized":"2023-12-26T21:52:29.219359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:16:39.633558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234355+0000","last_change":"2023-12-26T21:52:29.229327+0000","last_active":"2023-12-26T21:52:29.234355+0000","last_peered":"2023-12-26T21:52:29.234355+0000","last_clean":"2023-12-26T21:52:29.234355+0000","last_became_active":"2023-12-26T21:52:29.229049+0000","last_became_peered":"2023-12-26T21:52:29.229049+0000","last_unstale":"2023-12-26T21:52:29.234355+0000","last_undegraded":"2023-12-26T21:52:29.234355+0000","last_fullsized":"2023-12-26T21:52:29.234355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:29:36.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"81'8","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.266860+0000","last_change":"2023-12-26T21:52:28.271790+0000","last_active":"2023-12-26T21:52:29.266860+0000","last_peered":"2023-12-26T21:52:29.266860+0000","last_clean":"2023-12-26T21:52:29.266860+0000","last_became_active":"2023-12-26T21:52:28.271361+0000","last_became_peered":"2023-12-26T21:52:28.271361+0000","last_unstale":"2023-12-26T21:52:29.266860+0000","last_undegraded":"2023-12-26T21:52:29.266860+0000","last_fullsized":"2023-12-26T21:52:29.266860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:13:53.181133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236173+0000","last_change":"2023-12-26T21:52:29.232038+0000","last_active":"2023-12-26T21:52:29.236173+0000","last_peered":"2023-12-26T21:52:29.236173+0000","last_clean":"2023-12-26T21:52:29.236173+0000","last_became_active":"2023-12-26T21:52:29.231778+0000","last_became_peered":"2023-12-26T21:52:29.231778+0000","last_unstale":"2023-12-26T21:52:29.236173+0000","last_undegraded":"2023-12-26T21:52:29.236173+0000","last_fullsized":"2023-12-26T21:52:29.236173+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:08:03.577052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263172+0000","last_change":"2023-12-26T21:52:28.261756+0000","last_active":"2023-12-26T21:52:29.263172+0000","last_peered":"2023-12-26T21:52:29.263172+0000","last_clean":"2023-12-26T21:52:29.263172+0000","last_became_active":"2023-12-26T21:52:28.260555+0000","last_became_peered":"2023-12-26T21:52:28.260555+0000","last_unstale":"2023-12-26T21:52:29.263172+0000","last_undegraded":"2023-12-26T21:52:29.263172+0000","last_fullsized":"2023-12-26T21:52:29.263172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:48.534207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237099+0000","last_change":"2023-12-26T21:52:29.233600+0000","last_active":"2023-12-26T21:52:29.237099+0000","last_peered":"2023-12-26T21:52:29.237099+0000","last_clean":"2023-12-26T21:52:29.237099+0000","last_became_active":"2023-12-26T21:52:29.233320+0000","last_became_peered":"2023-12-26T21:52:29.233320+0000","last_unstale":"2023-12-26T21:52:29.237099+0000","last_undegraded":"2023-12-26T21:52:29.237099+0000","last_fullsized":"2023-12-26T21:52:29.237099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:48:58.284911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263975+0000","last_change":"2023-12-26T21:52:28.268837+0000","last_active":"2023-12-26T21:52:29.263975+0000","last_peered":"2023-12-26T21:52:29.263975+0000","last_clean":"2023-12-26T21:52:29.263975+0000","last_became_active":"2023-12-26T21:52:28.268568+0000","last_became_peered":"2023-12-26T21:52:28.268568+0000","last_unstale":"2023-12-26T21:52:29.263975+0000","last_undegraded":"2023-12-26T21:52:29.263975+0000","last_fullsized":"2023-12-26T21:52:29.263975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:57:53.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230852+0000","last_change":"2023-12-26T21:52:29.227912+0000","last_active":"2023-12-26T21:52:29.230852+0000","last_peered":"2023-12-26T21:52:29.230852+0000","last_clean":"2023-12-26T21:52:29.230852+0000","last_became_active":"2023-12-26T21:52:29.227624+0000","last_became_peered":"2023-12-26T21:52:29.227624+0000","last_unstale":"2023-12-26T21:52:29.230852+0000","last_undegraded":"2023-12-26T21:52:29.230852+0000","last_fullsized":"2023-12-26T21:52:29.230852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:02:53.252553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263685+0000","last_change":"2023-12-26T21:52:28.264029+0000","last_active":"2023-12-26T21:52:29.263685+0000","last_peered":"2023-12-26T21:52:29.263685+0000","last_clean":"2023-12-26T21:52:29.263685+0000","last_became_active":"2023-12-26T21:52:28.263753+0000","last_became_peered":"2023-12-26T21:52:28.263753+0000","last_unstale":"2023-12-26T21:52:29.263685+0000","last_undegraded":"2023-12-26T21:52:29.263685+0000","last_fullsized":"2023-12-26T21:52:29.263685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:10.512168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237888+0000","last_change":"2023-12-26T21:52:29.236983+0000","last_active":"2023-12-26T21:52:29.237888+0000","last_peered":"2023-12-26T21:52:29.237888+0000","last_clean":"2023-12-26T21:52:29.237888+0000","last_became_active":"2023-12-26T21:52:29.236707+0000","last_became_peered":"2023-12-26T21:52:29.236707+0000","last_unstale":"2023-12-26T21:52:29.237888+0000","last_undegraded":"2023-12-26T21:52:29.237888+0000","last_fullsized":"2023-12-26T21:52:29.237888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:27:45.114181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219399+0000","last_change":"2023-12-26T21:52:28.263164+0000","last_active":"2023-12-26T21:52:29.219399+0000","last_peered":"2023-12-26T21:52:29.219399+0000","last_clean":"2023-12-26T21:52:29.219399+0000","last_became_active":"2023-12-26T21:52:28.261982+0000","last_became_peered":"2023-12-26T21:52:28.261982+0000","last_unstale":"2023-12-26T21:52:29.219399+0000","last_undegraded":"2023-12-26T21:52:29.219399+0000","last_fullsized":"2023-12-26T21:52:29.219399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:28.819950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.242695+0000","last_change":"2023-12-26T21:52:29.242162+0000","last_active":"2023-12-26T21:52:29.242695+0000","last_peered":"2023-12-26T21:52:29.242695+0000","last_clean":"2023-12-26T21:52:29.242695+0000","last_became_active":"2023-12-26T21:52:29.241931+0000","last_became_peered":"2023-12-26T21:52:29.241931+0000","last_unstale":"2023-12-26T21:52:29.242695+0000","last_undegraded":"2023-12-26T21:52:29.242695+0000","last_fullsized":"2023-12-26T21:52:29.242695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:53:01.470505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.265802+0000","last_change":"2023-12-26T21:52:28.267744+0000","last_active":"2023-12-26T21:52:29.265802+0000","last_peered":"2023-12-26T21:52:29.265802+0000","last_clean":"2023-12-26T21:52:29.265802+0000","last_became_active":"2023-12-26T21:52:28.267302+0000","last_became_peered":"2023-12-26T21:52:28.267302+0000","last_unstale":"2023-12-26T21:52:29.265802+0000","last_undegraded":"2023-12-26T21:52:29.265802+0000","last_fullsized":"2023-12-26T21:52:29.265802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:02:52.245806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232060+0000","last_change":"2023-12-26T21:52:29.223883+0000","last_active":"2023-12-26T21:52:29.232060+0000","last_peered":"2023-12-26T21:52:29.232060+0000","last_clean":"2023-12-26T21:52:29.232060+0000","last_became_active":"2023-12-26T21:52:29.223617+0000","last_became_peered":"2023-12-26T21:52:29.223617+0000","last_unstale":"2023-12-26T21:52:29.232060+0000","last_undegraded":"2023-12-26T21:52:29.232060+0000","last_fullsized":"2023-12-26T21:52:29.232060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:29:36.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.220211+0000","last_change":"2023-12-26T21:52:28.263761+0000","last_active":"2023-12-26T21:52:29.220211+0000","last_peered":"2023-12-26T21:52:29.220211+0000","last_clean":"2023-12-26T21:52:29.220211+0000","last_became_active":"2023-12-26T21:52:28.263516+0000","last_became_peered":"2023-12-26T21:52:28.263516+0000","last_unstale":"2023-12-26T21:52:29.220211+0000","last_undegraded":"2023-12-26T21:52:29.220211+0000","last_fullsized":"2023-12-26T21:52:29.220211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:29:30.763837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.232911+0000","last_change":"2023-12-26T21:52:29.227887+0000","last_active":"2023-12-26T21:52:29.232911+0000","last_peered":"2023-12-26T21:52:29.232911+0000","last_clean":"2023-12-26T21:52:29.232911+0000","last_became_active":"2023-12-26T21:52:29.226890+0000","last_became_peered":"2023-12-26T21:52:29.226890+0000","last_unstale":"2023-12-26T21:52:29.232911+0000","last_undegraded":"2023-12-26T21:52:29.232911+0000","last_fullsized":"2023-12-26T21:52:29.232911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:02:49.740534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214344+0000","last_change":"2023-12-26T21:52:28.263846+0000","last_active":"2023-12-26T21:52:29.214344+0000","last_peered":"2023-12-26T21:52:29.214344+0000","last_clean":"2023-12-26T21:52:29.214344+0000","last_became_active":"2023-12-26T21:52:28.263365+0000","last_became_peered":"2023-12-26T21:52:28.263365+0000","last_unstale":"2023-12-26T21:52:29.214344+0000","last_undegraded":"2023-12-26T21:52:29.214344+0000","last_fullsized":"2023-12-26T21:52:29.214344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:39.729932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236252+0000","last_change":"2023-12-26T21:52:29.232453+0000","last_active":"2023-12-26T21:52:29.236252+0000","last_peered":"2023-12-26T21:52:29.236252+0000","last_clean":"2023-12-26T21:52:29.236252+0000","last_became_active":"2023-12-26T21:52:29.232176+0000","last_became_peered":"2023-12-26T21:52:29.232176+0000","last_unstale":"2023-12-26T21:52:29.236252+0000","last_undegraded":"2023-12-26T21:52:29.236252+0000","last_fullsized":"2023-12-26T21:52:29.236252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:27:29.980110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.263179+0000","last_change":"2023-12-26T21:52:28.264224+0000","last_active":"2023-12-26T21:52:29.263179+0000","last_peered":"2023-12-26T21:52:29.263179+0000","last_clean":"2023-12-26T21:52:29.263179+0000","last_became_active":"2023-12-26T21:52:28.263911+0000","last_became_peered":"2023-12-26T21:52:28.263911+0000","last_unstale":"2023-12-26T21:52:29.263179+0000","last_undegraded":"2023-12-26T21:52:29.263179+0000","last_fullsized":"2023-12-26T21:52:29.263179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:52:28.566410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236338+0000","last_change":"2023-12-26T21:52:29.230971+0000","last_active":"2023-12-26T21:52:29.236338+0000","last_peered":"2023-12-26T21:52:29.236338+0000","last_clean":"2023-12-26T21:52:29.236338+0000","last_became_active":"2023-12-26T21:52:29.230403+0000","last_became_peered":"2023-12-26T21:52:29.230403+0000","last_unstale":"2023-12-26T21:52:29.236338+0000","last_undegraded":"2023-12-26T21:52:29.236338+0000","last_fullsized":"2023-12-26T21:52:29.236338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:16:35.642844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.239044+0000","last_change":"2023-12-26T21:52:28.266467+0000","last_active":"2023-12-26T21:52:29.239044+0000","last_peered":"2023-12-26T21:52:29.239044+0000","last_clean":"2023-12-26T21:52:29.239044+0000","last_became_active":"2023-12-26T21:52:28.266175+0000","last_became_peered":"2023-12-26T21:52:28.266175+0000","last_unstale":"2023-12-26T21:52:29.239044+0000","last_undegraded":"2023-12-26T21:52:29.239044+0000","last_fullsized":"2023-12-26T21:52:29.239044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:24:49.091032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230763+0000","last_change":"2023-12-26T21:52:29.224964+0000","last_active":"2023-12-26T21:52:29.230763+0000","last_peered":"2023-12-26T21:52:29.230763+0000","last_clean":"2023-12-26T21:52:29.230763+0000","last_became_active":"2023-12-26T21:52:29.224655+0000","last_became_peered":"2023-12-26T21:52:29.224655+0000","last_unstale":"2023-12-26T21:52:29.230763+0000","last_undegraded":"2023-12-26T21:52:29.230763+0000","last_fullsized":"2023-12-26T21:52:29.230763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:28.557813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214435+0000","last_change":"2023-12-26T21:52:28.265591+0000","last_active":"2023-12-26T21:52:29.214435+0000","last_peered":"2023-12-26T21:52:29.214435+0000","last_clean":"2023-12-26T21:52:29.214435+0000","last_became_active":"2023-12-26T21:52:28.264635+0000","last_became_peered":"2023-12-26T21:52:28.264635+0000","last_unstale":"2023-12-26T21:52:29.214435+0000","last_undegraded":"2023-12-26T21:52:29.214435+0000","last_fullsized":"2023-12-26T21:52:29.214435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:39:00.418215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234459+0000","last_change":"2023-12-26T21:52:29.230344+0000","last_active":"2023-12-26T21:52:29.234459+0000","last_peered":"2023-12-26T21:52:29.234459+0000","last_clean":"2023-12-26T21:52:29.234459+0000","last_became_active":"2023-12-26T21:52:29.229837+0000","last_became_peered":"2023-12-26T21:52:29.229837+0000","last_unstale":"2023-12-26T21:52:29.234459+0000","last_undegraded":"2023-12-26T21:52:29.234459+0000","last_fullsized":"2023-12-26T21:52:29.234459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:37:07.651115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.223505+0000","last_change":"2023-12-26T21:52:28.261351+0000","last_active":"2023-12-26T21:52:29.223505+0000","last_peered":"2023-12-26T21:52:29.223505+0000","last_clean":"2023-12-26T21:52:29.223505+0000","last_became_active":"2023-12-26T21:52:28.261022+0000","last_became_peered":"2023-12-26T21:52:28.261022+0000","last_unstale":"2023-12-26T21:52:29.223505+0000","last_undegraded":"2023-12-26T21:52:29.223505+0000","last_fullsized":"2023-12-26T21:52:29.223505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:49:59.721353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.243985+0000","last_change":"2023-12-26T21:52:29.239607+0000","last_active":"2023-12-26T21:52:29.243985+0000","last_peered":"2023-12-26T21:52:29.243985+0000","last_clean":"2023-12-26T21:52:29.243985+0000","last_became_active":"2023-12-26T21:52:29.237151+0000","last_became_peered":"2023-12-26T21:52:29.237151+0000","last_unstale":"2023-12-26T21:52:29.243985+0000","last_undegraded":"2023-12-26T21:52:29.243985+0000","last_fullsized":"2023-12-26T21:52:29.243985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:36:20.817379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.212054+0000","last_change":"2023-12-26T21:52:28.257384+0000","last_active":"2023-12-26T21:52:29.212054+0000","last_peered":"2023-12-26T21:52:29.212054+0000","last_clean":"2023-12-26T21:52:29.212054+0000","last_became_active":"2023-12-26T21:52:28.257082+0000","last_became_peered":"2023-12-26T21:52:28.257082+0000","last_unstale":"2023-12-26T21:52:29.212054+0000","last_undegraded":"2023-12-26T21:52:29.212054+0000","last_fullsized":"2023-12-26T21:52:29.212054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:53.632343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.236280+0000","last_change":"2023-12-26T21:52:29.233498+0000","last_active":"2023-12-26T21:52:29.236280+0000","last_peered":"2023-12-26T21:52:29.236280+0000","last_clean":"2023-12-26T21:52:29.236280+0000","last_became_active":"2023-12-26T21:52:29.233226+0000","last_became_peered":"2023-12-26T21:52:29.233226+0000","last_unstale":"2023-12-26T21:52:29.236280+0000","last_undegraded":"2023-12-26T21:52:29.236280+0000","last_fullsized":"2023-12-26T21:52:29.236280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:26:56.203843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.224169+0000","last_change":"2023-12-26T21:52:28.264927+0000","last_active":"2023-12-26T21:52:29.224169+0000","last_peered":"2023-12-26T21:52:29.224169+0000","last_clean":"2023-12-26T21:52:29.224169+0000","last_became_active":"2023-12-26T21:52:28.264616+0000","last_became_peered":"2023-12-26T21:52:28.264616+0000","last_unstale":"2023-12-26T21:52:29.224169+0000","last_undegraded":"2023-12-26T21:52:29.224169+0000","last_fullsized":"2023-12-26T21:52:29.224169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:48:26.750191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.234392+0000","last_change":"2023-12-26T21:52:29.229506+0000","last_active":"2023-12-26T21:52:29.234392+0000","last_peered":"2023-12-26T21:52:29.234392+0000","last_clean":"2023-12-26T21:52:29.234392+0000","last_became_active":"2023-12-26T21:52:29.229233+0000","last_became_peered":"2023-12-26T21:52:29.229233+0000","last_unstale":"2023-12-26T21:52:29.234392+0000","last_undegraded":"2023-12-26T21:52:29.234392+0000","last_fullsized":"2023-12-26T21:52:29.234392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:37:05.801449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.237302+0000","last_change":"2023-12-26T21:52:28.268070+0000","last_active":"2023-12-26T21:52:29.237302+0000","last_peered":"2023-12-26T21:52:29.237302+0000","last_clean":"2023-12-26T21:52:29.237302+0000","last_became_active":"2023-12-26T21:52:28.265991+0000","last_became_peered":"2023-12-26T21:52:28.265991+0000","last_unstale":"2023-12-26T21:52:29.237302+0000","last_undegraded":"2023-12-26T21:52:29.237302+0000","last_fullsized":"2023-12-26T21:52:29.237302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:30:08.442988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.228656+0000","last_change":"2023-12-26T21:52:29.222923+0000","last_active":"2023-12-26T21:52:29.228656+0000","last_peered":"2023-12-26T21:52:29.228656+0000","last_clean":"2023-12-26T21:52:29.228656+0000","last_became_active":"2023-12-26T21:52:29.222437+0000","last_became_peered":"2023-12-26T21:52:29.222437+0000","last_unstale":"2023-12-26T21:52:29.228656+0000","last_undegraded":"2023-12-26T21:52:29.228656+0000","last_fullsized":"2023-12-26T21:52:29.228656+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:00.728100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.219391+0000","last_change":"2023-12-26T21:52:28.266016+0000","last_active":"2023-12-26T21:52:29.219391+0000","last_peered":"2023-12-26T21:52:29.219391+0000","last_clean":"2023-12-26T21:52:29.219391+0000","last_became_active":"2023-12-26T21:52:28.264632+0000","last_became_peered":"2023-12-26T21:52:28.264632+0000","last_unstale":"2023-12-26T21:52:29.219391+0000","last_undegraded":"2023-12-26T21:52:29.219391+0000","last_fullsized":"2023-12-26T21:52:29.219391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:43:06.542037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.230809+0000","last_change":"2023-12-26T21:52:29.226505+0000","last_active":"2023-12-26T21:52:29.230809+0000","last_peered":"2023-12-26T21:52:29.230809+0000","last_clean":"2023-12-26T21:52:29.230809+0000","last_became_active":"2023-12-26T21:52:29.226215+0000","last_became_peered":"2023-12-26T21:52:29.226215+0000","last_unstale":"2023-12-26T21:52:29.230809+0000","last_undegraded":"2023-12-26T21:52:29.230809+0000","last_fullsized":"2023-12-26T21:52:29.230809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:28.236032+0000","last_clean_scrub_stamp":"2023-12-26T21:52:28.236032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:01:54.460352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.220332+0000","last_change":"2023-12-26T21:52:28.266993+0000","last_active":"2023-12-26T21:52:29.220332+0000","last_peered":"2023-12-26T21:52:29.220332+0000","last_clean":"2023-12-26T21:52:29.220332+0000","last_became_active":"2023-12-26T21:52:28.266697+0000","last_became_peered":"2023-12-26T21:52:28.266697+0000","last_unstale":"2023-12-26T21:52:29.220332+0000","last_undegraded":"2023-12-26T21:52:29.220332+0000","last_fullsized":"2023-12-26T21:52:29.220332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:52:27.229285+0000","last_clean_scrub_stamp":"2023-12-26T21:52:27.229285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:31:30.364550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"71'161","reported_seq":374,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-26T21:52:29.214361+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:52:29.214361+0000","last_peered":"2023-12-26T21:52:29.214361+0000","last_clean":"2023-12-26T21:52:29.214361+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:52:29.214361+0000","last_undegraded":"2023-12-26T21:52:29.214361+0000","last_fullsized":"2023-12-26T21:52:29.214361+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542335,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298256,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073584,"statfs":{"total":96636764160,"available":96331350016,"internally_reserved":0,"allocated":991232,"data_stored":470993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.647},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.61199999999999999,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.92300000000000004,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.74199999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":1,"15min":2.0150000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.58199999999999996,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.014,"5min":1.048,"15min":1.1359999999999999},"last":0.72199999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.59699999999999998,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":1.038,"15min":1.8640000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.498,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.419},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":0.93799999999999994,"5min":1.0189999999999999,"15min":1.327},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.65000000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.393},"last":0.68400000000000005}]},{"osd":4,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.374,"15min":0.371},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.53000000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70399999999999996}]}]},{"osd":6,"up_from":15,"seq":64424509631,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302348,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069492,"statfs":{"total":96636764160,"available":96327159808,"internally_reserved":0,"allocated":987136,"data_stored":470923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":1.0029999999999999,"15min":1.085},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.3380000000000001},"last":0.75900000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.88700000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":1.288},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.876,"5min":0.92000000000000004,"15min":1.113},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.92000000000000004,"5min":1.026,"15min":1.569},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.97799999999999998,"15min":1.3839999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.62,"15min":0.62},"min":{"1min":0.42699999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.137},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.56000000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.69599999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95099999999999996,"5min":1.0049999999999999,"15min":1.296},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.437},"max":{"1min":0.89200000000000002,"5min":0.96899999999999997,"15min":1.51},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.91700000000000004,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.57799999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509631,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307024,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064816,"statfs":{"total":96636764160,"available":96322371584,"internally_reserved":0,"allocated":1581056,"data_stored":1061823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.45600000000000002,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":0.68700000000000006,"5min":0.70299999999999996,"15min":0.71799999999999997},"last":1.6020000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.39000000000000001,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.68000000000000005,"5min":0.754,"15min":4.6600000000000001},"last":1.353}]},{"osd":2,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32100000000000001},"max":{"1min":0.625,"5min":0.625,"15min":2.2869999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":4.3639999999999999},"last":1.488}]},{"osd":3,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":0.871,"15min":4.3259999999999996},"last":1.4379999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.57099999999999995},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.33300000000000002},"max":{"1min":0.72799999999999998,"5min":0.83799999999999997,"15min":4.548},"last":1.506}]},{"osd":4,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.749,"15min":0.752},"min":{"1min":0.57899999999999996,"5min":0.505,"15min":0.505},"max":{"1min":0.88500000000000001,"5min":1.252,"15min":1.252},"last":1.4670000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72599999999999998,"15min":0.74399999999999999},"min":{"1min":0.56699999999999995,"5min":0.47799999999999998,"15min":0.441},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":5.5919999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.53800000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.014,"5min":1.2949999999999999,"15min":1.345},"last":1.5760000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.03,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.5269999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.63800000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91800000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.80400000000000005,"15min":0.80900000000000005},"min":{"1min":0.66900000000000004,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.615}]},{"osd":7,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.78300000000000003},"min":{"1min":0.56699999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.0980000000000001,"15min":1.1539999999999999},"last":2.3460000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77700000000000002,"15min":0.79200000000000004},"min":{"1min":0.67000000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.99399999999999999,"5min":1.3620000000000001,"15min":1.3919999999999999},"last":4.3319999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302356,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069484,"statfs":{"total":96636764160,"available":96327151616,"internally_reserved":0,"allocated":995328,"data_stored":472437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.439},"min":{"1min":0.316,"5min":0.245,"15min":0.245},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":1.5640000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69999999999999996,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.371,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.58599999999999997,"5min":0.66800000000000004,"15min":2.012},"last":0.496},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.47699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.42299999999999999,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":4.2610000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.41799999999999998,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.64000000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.94699999999999995,"5min":1.1539999999999999,"15min":4.423},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.95799999999999996,"5min":1.069,"15min":1.6439999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78200000000000003,"15min":0.80900000000000005},"min":{"1min":0.57799999999999996,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":4.3650000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.008,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.62}]},{"osd":6,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76700000000000002,"15min":0.78500000000000003},"min":{"1min":0.623,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.89000000000000001,"5min":1.103,"15min":1.611},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.55200000000000005,"5min":0.504,"15min":0.5},"max":{"1min":1.012,"5min":1.111,"15min":4.407},"last":0.80800000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.81000000000000005},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":1.145,"15min":4.5069999999999997},"last":0.53000000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542336,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302364,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069476,"statfs":{"total":96636764160,"available":96327143424,"internally_reserved":0,"allocated":1003520,"data_stored":472597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45800000000000002,"15min":0.45900000000000002},"min":{"1min":0.33200000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":1.915},"last":0.44},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.30399999999999999},"max":{"1min":0.63200000000000001,"5min":0.83899999999999997,"15min":1.7729999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.39600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":1.952},"last":0.498},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47999999999999998},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.30199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.375},"max":{"1min":1,"5min":1,"15min":1},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73899999999999999,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.34599999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.54600000000000004,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.96999999999999997,"5min":0.98299999999999998,"15min":1.284},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.60099999999999998,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.0600000000000001,"15min":1.5029999999999999},"last":1.024},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":0.94599999999999995,"5min":1.014,"15min":1.355},"last":0.749}]},{"osd":6,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.93000000000000005,"5min":1.038,"15min":1.7350000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.95999999999999996,"5min":1.0609999999999999,"15min":1.349},"last":0.88200000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.47799999999999998},"max":{"1min":0.92700000000000005,"5min":1.073,"15min":1.841},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0189999999999999,"15min":1.294},"last":0.999}]}]},{"osd":3,"up_from":15,"seq":64424509632,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306432,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065408,"statfs":{"total":96636764160,"available":96322977792,"internally_reserved":0,"allocated":974848,"data_stored":464311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61599999999999999,"5min":0.70699999999999996,"15min":0.91400000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.47999999999999998},"min":{"1min":0.38500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71499999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.35199999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46899999999999997,"15min":0.46500000000000002},"min":{"1min":0.35999999999999999,"5min":0.31900000000000001,"15min":0.30399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.77200000000000002},"last":0.372},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":0.69599999999999995,"5min":0.72499999999999998,"15min":0.99399999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.81799999999999995,"5min":0.85599999999999998,"15min":1.0820000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.621,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.40300000000000002},"max":{"1min":0.85599999999999998,"5min":0.94799999999999995,"15min":1.3580000000000001},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.879,"5min":1.105,"15min":1.3779999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.751,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":0.94299999999999995,"5min":1.1870000000000001,"15min":1.2949999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77500000000000002,"15min":0.76800000000000002},"min":{"1min":0.624,"5min":0.56399999999999995,"15min":0.47699999999999998},"max":{"1min":0.98499999999999999,"5min":1.27,"15min":1.4610000000000001},"last":0.745},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.45900000000000002},"max":{"1min":0.88100000000000001,"5min":0.997,"15min":1.9239999999999999},"last":0.67500000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.56799999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96399999999999997,"5min":1.1950000000000001,"15min":1.5049999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.81200000000000006},"min":{"1min":0.65600000000000003,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.02,"5min":1.0760000000000001,"15min":1.579},"last":0.71299999999999997}]},{"osd":7,"last update":"Tue Dec 26 21:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.49299999999999999},"max":{"1min":0.96999999999999997,"5min":1.0640000000000001,"15min":1.964},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.48499999999999999},"max":{"1min":0.97399999999999998,"5min":1.1499999999999999,"15min":1.6359999999999999},"last":0.75700000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542335,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302352,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069488,"statfs":{"total":96636764160,"available":96327155712,"internally_reserved":0,"allocated":991232,"data_stored":471489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66000000000000003,"15min":0.68400000000000005},"min":{"1min":0.439,"5min":0.307,"15min":0.307},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":2.1389999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.96399999999999997,"5min":1.008,"15min":1.407},"last":0.81100000000000005}]},{"osd":1,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.52700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.879},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69799999999999995,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.89200000000000002,"5min":0.98699999999999999,"15min":2.097},"last":0.83499999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70999999999999996,"15min":0.72199999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91000000000000003,"5min":0.94999999999999996,"15min":1.7809999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88800000000000001,"5min":0.97199999999999998,"15min":1.6870000000000001},"last":0.88900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69999999999999996,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95999999999999996,"15min":1.956},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.746},"min":{"1min":0.626,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.0609999999999999},"last":0.72999999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58899999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82099999999999995,"5min":0.86399999999999999,"15min":2.2429999999999999},"last":0.746},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84899999999999998,"5min":0.92600000000000005,"15min":2.194},"last":0.77800000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.621,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.379},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":2.1589999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":0.79200000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.432},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":2.23},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.745},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":2.5169999999999999},"last":0.877}]}]},{"osd":5,"up_from":15,"seq":64424509630,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302928,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068912,"statfs":{"total":96636764160,"available":96326565888,"internally_reserved":0,"allocated":1581056,"data_stored":1061303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.878,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.71299999999999997},"min":{"1min":0.48899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.71},"last":0.72699999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.57399999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96399999999999997,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68200000000000005,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.002,"15min":1.4219999999999999},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.871,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68799999999999994,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":1.214},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":1.052,"15min":1.4890000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":0.92200000000000004,"15min":1.1439999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84399999999999997,"5min":1.0089999999999999,"15min":2.5070000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.73899999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.92400000000000004,"15min":1.637},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.042,"15min":2.5579999999999998},"last":0.754}]},{"osd":7,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":1.248},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":0.95199999999999996,"15min":1.5980000000000001},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:52:34.076 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T21:52:34.076 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T21:52:34.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.309+0000 7ff7a93ff700 1 Processor -- start 2023-12-26T21:52:34.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.309+0000 7ff7a93ff700 1 -- start start 2023-12-26T21:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.310+0000 7ff7a93ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7a4159a20 0x7ff7a40ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.310+0000 7ff7a93ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.310+0000 7ff7a93ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a40ccbc0 con 0x7ff7a4159650 2023-12-26T21:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.310+0000 7ff7a93ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a40cf6a0 con 0x7ff7a4159a20 2023-12-26T21:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.310+0000 7ff7a93ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40cbfb0 con 0x7ff7a40cf260 2023-12-26T21:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a37fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a27fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7a4159a20 0x7ff7a40ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a37fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:60080/0 (socket says 172.21.15.131:60080) 2023-12-26T21:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a37fe700 1 -- 172.21.15.131:0/1116684548 learned_addr learned my addr 172.21.15.131:0/1116684548 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a1ffb700 1 -- 172.21.15.131:0/1116684548 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075555279 0 0) 0x7ff7a40ccbc0 con 0x7ff7a4159650 2023-12-26T21:52:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a1ffb700 1 -- 172.21.15.131:0/1116684548 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff78c003570 con 0x7ff7a4159650 2023-12-26T21:52:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a37fe700 1 -- 172.21.15.131:0/1116684548 >> v2:172.21.15.190:3300/0 conn(0x7ff7a4159a20 msgr2=0x7ff7a40ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a37fe700 1 --2- 172.21.15.131:0/1116684548 >> v2:172.21.15.190:3300/0 conn(0x7ff7a4159a20 0x7ff7a40ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a37fe700 1 -- 172.21.15.131:0/1116684548 >> v1:172.21.15.190:6789/0 conn(0x7ff7a4159650 legacy=0x7ff7a40cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a37fe700 1 -- 172.21.15.131:0/1116684548 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a40cc290 con 0x7ff7a40cf260 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a27fc700 1 --2- 172.21.15.131:0/1116684548 >> v2:172.21.15.190:3300/0 conn(0x7ff7a4159a20 0x7ff7a40ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.311+0000 7ff7a37fe700 1 --2- 172.21.15.131:0/1116684548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40c55d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009f20 tx=0x7ff79c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54bba46b0361a571 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a1ffb700 1 -- 172.21.15.131:0/1116684548 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00e070 con 0x7ff7a40cf260 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a1ffb700 1 -- 172.21.15.131:0/1116684548 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c00b970 con 0x7ff7a40cf260 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a1ffb700 1 -- 172.21.15.131:0/1116684548 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0092e0 con 0x7ff7a40cf260 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a93ff700 1 -- 172.21.15.131:0/1116684548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 msgr2=0x7ff7a40c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a93ff700 1 --2- 172.21.15.131:0/1116684548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40c55d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009f20 tx=0x7ff79c009840 comp rx=0 tx=0).stop 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a93ff700 1 -- 172.21.15.131:0/1116684548 shutdown_connections 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a93ff700 1 --2- 172.21.15.131:0/1116684548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40c55d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a93ff700 1 --2- 172.21.15.131:0/1116684548 >> v2:172.21.15.190:3300/0 conn(0x7ff7a4159a20 0x7ff7a40ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a93ff700 1 -- 172.21.15.131:0/1116684548 >> 172.21.15.131:0/1116684548 conn(0x7ff7a4099980 msgr2=0x7ff7a406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a93ff700 1 -- 172.21.15.131:0/1116684548 shutdown_connections 2023-12-26T21:52:34.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.312+0000 7ff7a93ff700 1 -- 172.21.15.131:0/1116684548 wait complete. 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.314+0000 7ff7a93ff700 1 Processor -- start 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.314+0000 7ff7a93ff700 1 -- start start 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a93ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a93ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a4159650 0x7ff7a4143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a93ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7a4159a20 0x7ff7a4155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a93ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a4143670 con 0x7ff7a4159650 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a93ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40c9fb0 con 0x7ff7a4159a20 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a93ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40c93a0 con 0x7ff7a40cf260 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a27fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a4159650 0x7ff7a4143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a27fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a4159650 0x7ff7a4143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33828/0 (socket says 172.21.15.131:33828) 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a27fc700 1 -- 172.21.15.131:0/3507447258 learned_addr learned my addr 172.21.15.131:0/3507447258 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a2ffd700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.315+0000 7ff7a37fe700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7a4159a20 0x7ff7a4155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff7a27fc700 1 -- 172.21.15.131:0/3507447258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 msgr2=0x7ff7a40ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff7a27fc700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff7a27fc700 1 -- 172.21.15.131:0/3507447258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7a4159a20 msgr2=0x7ff7a4155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff7a27fc700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7a4159a20 0x7ff7a4155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff7a27fc700 1 -- 172.21.15.131:0/3507447258 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798009420 con 0x7ff7a4159650 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff7a27fc700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a4159650 0x7ff7a4143130 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff79800ecb0 tx=0x7ff7980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff78affd700 1 -- 172.21.15.131:0/3507447258 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79800c850 con 0x7ff7a4159650 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff78affd700 1 -- 172.21.15.131:0/3507447258 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79800c9b0 con 0x7ff7a4159650 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c009ae0 con 0x7ff7a4159650 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff78affd700 1 -- 172.21.15.131:0/3507447258 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79800cd70 con 0x7ff7a4159650 2023-12-26T21:52:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.316+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a40c9960 con 0x7ff7a4159650 2023-12-26T21:52:34.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.317+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a40657e0 con 0x7ff7a4159650 2023-12-26T21:52:34.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.318+0000 7ff78affd700 1 -- 172.21.15.131:0/3507447258 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff798013680 con 0x7ff7a4159650 2023-12-26T21:52:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.319+0000 7ff78affd700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff790066f30 0x7ff7900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.319+0000 7ff78affd700 1 -- 172.21.15.131:0/3507447258 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff7980848d0 con 0x7ff7a4159650 2023-12-26T21:52:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.320+0000 7ff7a2ffd700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff790066f30 0x7ff7900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.320+0000 7ff7a2ffd700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff790066f30 0x7ff7900693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40c8c40 tx=0x7ff794007d40 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:34.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.323+0000 7ff78affd700 1 -- 172.21.15.131:0/3507447258 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7980519b0 con 0x7ff7a4159650 2023-12-26T21:52:34.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.592+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff7a40ca0f0 con 0x7ff7a4159650 2023-12-26T21:52:34.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.596+0000 7ff78affd700 1 -- 172.21.15.131:0/3507447258 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff798054fd0 con 0x7ff7a4159650 2023-12-26T21:52:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.598+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff790066f30 msgr2=0x7ff7900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.598+0000 7ff7a93ff700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff790066f30 0x7ff7900693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40c8c40 tx=0x7ff794007d40 comp rx=0 tx=0).stop 2023-12-26T21:52:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.598+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a4159650 msgr2=0x7ff7a4143130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.598+0000 7ff7a93ff700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a4159650 0x7ff7a4143130 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff79800ecb0 tx=0x7ff7980091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.598+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 shutdown_connections 2023-12-26T21:52:34.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.598+0000 7ff7a93ff700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7ff790066f30 0x7ff7900693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.598+0000 7ff7a93ff700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7a4159a20 0x7ff7a4155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.598+0000 7ff7a93ff700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7a40cf260 0x7ff7a40ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.598+0000 7ff7a93ff700 1 --2- 172.21.15.131:0/3507447258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a4159650 0x7ff7a4143130 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.599+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 >> 172.21.15.131:0/3507447258 conn(0x7ff7a4099980 msgr2=0x7ff7a40ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:34.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.599+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 shutdown_connections 2023-12-26T21:52:34.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.599+0000 7ff7a93ff700 1 -- 172.21.15.131:0/3507447258 wait complete. 2023-12-26T21:52:34.600 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T21:52:34.617 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:34.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.806+0000 7f4393fff700 1 Processor -- start 2023-12-26T21:52:34.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.806+0000 7f4393fff700 1 -- start start 2023-12-26T21:52:34.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f4393fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c1599d0 0x7f438c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f4393fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f438c0d01e0 0x7f438c0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f4393fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438c0cfd50 con 0x7f438c159600 2023-12-26T21:52:34.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f4393fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438c0b1280 con 0x7f438c0d01e0 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f4393fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0cf140 con 0x7f438c1599d0 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f439259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f438c0d01e0 0x7f438c0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f439259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f438c0d01e0 0x7f438c0ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33836/0 (socket says 172.21.15.131:33836) 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f439259c700 1 -- 172.21.15.131:0/477909179 learned_addr learned my addr 172.21.15.131:0/477909179 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f439159a700 1 --2- 172.21.15.131:0/477909179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c1599d0 0x7f438c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f439259c700 1 -- 172.21.15.131:0/477909179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c1599d0 msgr2=0x7f438c0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f439259c700 1 --2- 172.21.15.131:0/477909179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c1599d0 0x7f438c0ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f439259c700 1 -- 172.21.15.131:0/477909179 >> v1:172.21.15.190:6789/0 conn(0x7f438c159600 legacy=0x7f438c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.808+0000 7f439259c700 1 -- 172.21.15.131:0/477909179 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c0cf420 con 0x7f438c0d01e0 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f439259c700 1 --2- 172.21.15.131:0/477909179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c0d01e0 0x7f438c0ccf00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4384009f20 tx=0x7f4384009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5926c824a1a67e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f4390d99700 1 -- 172.21.15.131:0/477909179 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438400e070 con 0x7f438c0d01e0 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f4390d99700 1 -- 172.21.15.131:0/477909179 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438400ba40 con 0x7f438c0d01e0 2023-12-26T21:52:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f4390d99700 1 -- 172.21.15.131:0/477909179 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840092e0 con 0x7f438c0d01e0 2023-12-26T21:52:34.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f4393fff700 1 -- 172.21.15.131:0/477909179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c0d01e0 msgr2=0x7f438c0ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f4393fff700 1 --2- 172.21.15.131:0/477909179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c0d01e0 0x7f438c0ccf00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4384009f20 tx=0x7f4384009840 comp rx=0 tx=0).stop 2023-12-26T21:52:34.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f4393fff700 1 -- 172.21.15.131:0/477909179 shutdown_connections 2023-12-26T21:52:34.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f4393fff700 1 --2- 172.21.15.131:0/477909179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c0d01e0 0x7f438c0ccf00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f4393fff700 1 --2- 172.21.15.131:0/477909179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c1599d0 0x7f438c0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.809+0000 7f4393fff700 1 -- 172.21.15.131:0/477909179 >> 172.21.15.131:0/477909179 conn(0x7f438c06ece0 msgr2=0x7f438c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:34.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.810+0000 7f4393fff700 1 -- 172.21.15.131:0/477909179 shutdown_connections 2023-12-26T21:52:34.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.810+0000 7f4393fff700 1 -- 172.21.15.131:0/477909179 wait complete. 2023-12-26T21:52:34.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.811+0000 7f4393fff700 1 Processor -- start 2023-12-26T21:52:34.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.811+0000 7f4393fff700 1 -- start start 2023-12-26T21:52:34.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.812+0000 7f4393fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f438c0d01e0 0x7f438c16a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.812+0000 7f4393fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c159600 0x7f438c0b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.812+0000 7f4393fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c16af30 0x7f438c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.812+0000 7f4393fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0b3e10 con 0x7f438c16af30 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.812+0000 7f4393fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0b4050 con 0x7f438c0d01e0 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.812+0000 7f4393fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c14bcc0 con 0x7f438c159600 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c16af30 0x7f438c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f4391d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f438c0d01e0 0x7f438c16a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c16af30 0x7f438c0b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33840/0 (socket says 172.21.15.131:33840) 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f4391d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f438c0d01e0 0x7f438c16a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:59716/0 (socket says 172.21.15.131:59716) 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439259c700 1 -- 172.21.15.131:0/785789383 learned_addr learned my addr 172.21.15.131:0/785789383 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439159a700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c159600 0x7f438c0b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439259c700 1 -- 172.21.15.131:0/785789383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c159600 msgr2=0x7f438c0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439259c700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c159600 0x7f438c0b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439259c700 1 -- 172.21.15.131:0/785789383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f438c0d01e0 msgr2=0x7f438c16a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:34.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439259c700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f438c0d01e0 0x7f438c16a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:34.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439259c700 1 -- 172.21.15.131:0/785789383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4384009ae0 con 0x7f438c16af30 2023-12-26T21:52:34.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.813+0000 7f439259c700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c16af30 0x7f438c0b38d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4384003000 tx=0x7f4384017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:34.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.814+0000 7f4381ffb700 1 -- 172.21.15.131:0/785789383 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840032f0 con 0x7f438c16af30 2023-12-26T21:52:34.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.814+0000 7f4381ffb700 1 -- 172.21.15.131:0/785789383 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4384012d00 con 0x7f438c16af30 2023-12-26T21:52:34.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.814+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c14bfa0 con 0x7f438c16af30 2023-12-26T21:52:34.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.814+0000 7f4381ffb700 1 -- 172.21.15.131:0/785789383 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438401d650 con 0x7f438c16af30 2023-12-26T21:52:34.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.814+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c14c560 con 0x7f438c16af30 2023-12-26T21:52:34.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.815+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c0657e0 con 0x7f438c16af30 2023-12-26T21:52:34.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.816+0000 7f4381ffb700 1 -- 172.21.15.131:0/785789383 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f438401da40 con 0x7f438c16af30 2023-12-26T21:52:34.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.817+0000 7f4381ffb700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4370066f30 0x7f43700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:34.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.817+0000 7f4381ffb700 1 -- 172.21.15.131:0/785789383 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f4384085fa0 con 0x7f438c16af30 2023-12-26T21:52:34.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.817+0000 7f4391d9b700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4370066f30 0x7f43700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:34.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.817+0000 7f4391d9b700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4370066f30 0x7f43700693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f438c1580d0 tx=0x7f4388007ef0 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:34.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:34.821+0000 7f4381ffb700 1 -- 172.21.15.131:0/785789383 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4384052fe0 con 0x7f438c16af30 2023-12-26T21:52:35.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.053+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f438c14c750 con 0x7f438c16af30 2023-12-26T21:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.054+0000 7f4381ffb700 1 -- 172.21.15.131:0/785789383 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f4384056600 con 0x7f438c16af30 2023-12-26T21:52:35.054 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:35.054 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/625115906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":625115906},{"type":"v1","addr":"172.21.15.190:6833","nonce":625115906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2773111229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2773111229},{"type":"v1","addr":"172.21.15.190:6835","nonce":2773111229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/899613179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":899613179},{"type":"v1","addr":"172.21.15.131:6834","nonce":899613179}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:52:28.726694+0000","modified":"2023-12-26T21:52:32.268974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6836/4173752384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":4173752384},{"type":"v1","addr":"172.21.15.131:6836","nonce":4173752384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-12-26T21:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4370066f30 msgr2=0x7f43700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4370066f30 0x7f43700693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f438c1580d0 tx=0x7f4388007ef0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c16af30 msgr2=0x7f438c0b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c16af30 0x7f438c0b38d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4384003000 tx=0x7f4384017000 comp rx=0 tx=0).stop 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 shutdown_connections 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f438c16af30 0x7f438c0b38d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4370066f30 0x7f43700693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f438c0d01e0 0x7f438c16a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 --2- 172.21.15.131:0/785789383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f438c159600 0x7f438c0b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 >> 172.21.15.131:0/785789383 conn(0x7f438c06ece0 msgr2=0x7f438c157610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 shutdown_connections 2023-12-26T21:52:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.056+0000 7f4393fff700 1 -- 172.21.15.131:0/785789383 wait complete. 2023-12-26T21:52:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 34 2023-12-26T21:52:35.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T21:52:28.726694+0000', 'modified': '2023-12-26T21:52:32.268974+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6219}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6219': {'gid': 6219, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.131:6836/4173752384', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 4173752384}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 4173752384}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T21:52:35.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T21:52:35.477 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-26T21:52:35.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.670+0000 7fe8915ff700 1 Processor -- start 2023-12-26T21:52:35.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.670+0000 7fe8915ff700 1 -- start start 2023-12-26T21:52:35.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.671+0000 7fe8915ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe88c0cc810 0x7fe88c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.671+0000 7fe8915ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c0686c0 con 0x7fe88c06bfd0 2023-12-26T21:52:35.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.671+0000 7fe8915ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c069520 con 0x7fe88c13c510 2023-12-26T21:52:35.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.671+0000 7fe8915ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88c068910 con 0x7fe88c0cc810 2023-12-26T21:52:35.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.671+0000 7fe88a7fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe88c06bfd0 0x7fe88c0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45754/0 (socket says 172.21.15.131:45754) 2023-12-26T21:52:35.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.671+0000 7fe88a7fc700 1 -- 172.21.15.131:0/1312308757 learned_addr learned my addr 172.21.15.131:0/1312308757 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:35.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.672+0000 7fe88b7fe700 1 --2- 172.21.15.131:0/1312308757 >> v2:172.21.15.190:3300/0 conn(0x7fe88c0cc810 0x7fe88c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.672+0000 7fe889ffb700 1 -- 172.21.15.131:0/1312308757 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536127374 0 0) 0x7fe88c0686c0 con 0x7fe88c06bfd0 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.672+0000 7fe889ffb700 1 -- 172.21.15.131:0/1312308757 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe874003570 con 0x7fe88c06bfd0 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.672+0000 7fe889ffb700 1 -- 172.21.15.131:0/1312308757 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803921023 0 0) 0x7fe88c069520 con 0x7fe88c13c510 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.672+0000 7fe889ffb700 1 -- 172.21.15.131:0/1312308757 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe88c0686c0 con 0x7fe88c13c510 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.672+0000 7fe88b7fe700 1 -- 172.21.15.131:0/1312308757 >> v1:172.21.15.190:6789/0 conn(0x7fe88c13c510 legacy=0x7fe88c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.672+0000 7fe88b7fe700 1 -- 172.21.15.131:0/1312308757 >> v1:172.21.15.131:6789/0 conn(0x7fe88c06bfd0 legacy=0x7fe88c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.672+0000 7fe88b7fe700 1 -- 172.21.15.131:0/1312308757 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c068bf0 con 0x7fe88c0cc810 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.672+0000 7fe88b7fe700 1 --2- 172.21.15.131:0/1312308757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 0x7fe88c0c8a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe884009e60 tx=0x7fe884009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99980d5734ca0ae3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe889ffb700 1 -- 172.21.15.131:0/1312308757 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88400e070 con 0x7fe88c0cc810 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe889ffb700 1 -- 172.21.15.131:0/1312308757 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88400bab0 con 0x7fe88c0cc810 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe889ffb700 1 -- 172.21.15.131:0/1312308757 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88400be70 con 0x7fe88c0cc810 2023-12-26T21:52:35.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe8915ff700 1 -- 172.21.15.131:0/1312308757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 msgr2=0x7fe88c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe8915ff700 1 --2- 172.21.15.131:0/1312308757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 0x7fe88c0c8a20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe884009e60 tx=0x7fe884009780 comp rx=0 tx=0).stop 2023-12-26T21:52:35.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe8915ff700 1 -- 172.21.15.131:0/1312308757 shutdown_connections 2023-12-26T21:52:35.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe8915ff700 1 --2- 172.21.15.131:0/1312308757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 0x7fe88c0c8a20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe8915ff700 1 -- 172.21.15.131:0/1312308757 >> 172.21.15.131:0/1312308757 conn(0x7fe88c0ce3e0 msgr2=0x7fe88c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:35.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe8915ff700 1 -- 172.21.15.131:0/1312308757 shutdown_connections 2023-12-26T21:52:35.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.673+0000 7fe8915ff700 1 -- 172.21.15.131:0/1312308757 wait complete. 2023-12-26T21:52:35.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.675+0000 7fe8915ff700 1 Processor -- start 2023-12-26T21:52:35.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.675+0000 7fe8915ff700 1 -- start start 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe8915ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88c06bfd0 0x7fe88c15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe8915ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 0x7fe88c15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe8915ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe88c13c510 0x7fe88c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe8915ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c15c900 con 0x7fe88c13c510 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe8915ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c15cb40 con 0x7fe88c0cc810 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe8915ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c154330 con 0x7fe88c06bfd0 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe88b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe88c13c510 0x7fe88c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe88b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe88c13c510 0x7fe88c1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35880/0 (socket says 172.21.15.131:35880) 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe88b7fe700 1 -- 172.21.15.131:0/1805368076 learned_addr learned my addr 172.21.15.131:0/1805368076 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 0x7fe88c15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.676+0000 7fe88affd700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88c06bfd0 0x7fe88c15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe88b7fe700 1 -- 172.21.15.131:0/1805368076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88c06bfd0 msgr2=0x7fe88c15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe88b7fe700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88c06bfd0 0x7fe88c15b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe88b7fe700 1 -- 172.21.15.131:0/1805368076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 msgr2=0x7fe88c15c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe88b7fe700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 0x7fe88c15c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe88b7fe700 1 -- 172.21.15.131:0/1805368076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe884009a20 con 0x7fe88c13c510 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 0x7fe88c15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe88b7fe700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe88c13c510 0x7fe88c1539f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe88c06fe40 tx=0x7fe884019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe872ffd700 1 -- 172.21.15.131:0/1805368076 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884026480 con 0x7fe88c13c510 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe872ffd700 1 -- 172.21.15.131:0/1805368076 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe884026600 con 0x7fe88c13c510 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe8915ff700 1 -- 172.21.15.131:0/1805368076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c154610 con 0x7fe88c13c510 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe872ffd700 1 -- 172.21.15.131:0/1805368076 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8840269c0 con 0x7fe88c13c510 2023-12-26T21:52:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.677+0000 7fe8915ff700 1 -- 172.21.15.131:0/1805368076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c154bd0 con 0x7fe88c13c510 2023-12-26T21:52:35.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.680+0000 7fe872ffd700 1 -- 172.21.15.131:0/1805368076 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe88401c400 con 0x7fe88c13c510 2023-12-26T21:52:35.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.681+0000 7fe872ffd700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe878066f30 0x7fe8780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.681+0000 7fe872ffd700 1 -- 172.21.15.131:0/1805368076 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe884086a00 con 0x7fe88c13c510 2023-12-26T21:52:35.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.681+0000 7fe88affd700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe878066f30 0x7fe8780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.682+0000 7fe88affd700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe878066f30 0x7fe8780693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe87c003950 tx=0x7fe87c009040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.683+0000 7fe89fd1fb80 1 Processor -- start 2023-12-26T21:52:35.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.683+0000 7fe89fd1fb80 1 -- start start 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.684+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.684+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c28b0 0x55c1a83bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.684+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.684+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a82f03e0 con 0x55c1a83c28b0 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.684+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a82edbe0 con 0x55c1a83c1fa0 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.684+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1a82e99d0 con 0x55c1a83be170 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.684+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c28b0 0x55c1a83bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.684+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c28b0 0x55c1a83bdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35884/0 (socket says 172.21.15.131:35884) 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.684+0000 7fe88a7fc700 1 -- 172.21.15.131:0/3048303850 learned_addr learned my addr 172.21.15.131:0/3048303850 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe88affd700 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe88b7fe700 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe88a7fc700 1 -- 172.21.15.131:0/3048303850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 msgr2=0x55c1a83de360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83de360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe88a7fc700 1 -- 172.21.15.131:0/3048303850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 msgr2=0x55c1a83c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe88a7fc700 1 -- 172.21.15.131:0/3048303850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1a817d360 con 0x55c1a83c28b0 2023-12-26T21:52:35.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe88affd700 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:35.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c28b0 0x55c1a83bdc30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe880001700 tx=0x7fe880009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2051c34dae93c15a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe86bfff700 1 -- 172.21.15.131:0/3048303850 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880005b10 con 0x55c1a83c28b0 2023-12-26T21:52:35.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe86bfff700 1 -- 172.21.15.131:0/3048303850 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe880005c70 con 0x55c1a83c28b0 2023-12-26T21:52:35.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.685+0000 7fe86bfff700 1 -- 172.21.15.131:0/3048303850 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800070d0 con 0x55c1a83c28b0 2023-12-26T21:52:35.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.686+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3048303850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c28b0 msgr2=0x55c1a83bdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.686+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c28b0 0x55c1a83bdc30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe880001700 tx=0x7fe880009040 comp rx=0 tx=0).stop 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.686+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3048303850 shutdown_connections 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.686+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83de360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.686+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.686+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3048303850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c28b0 0x55c1a83bdc30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.686+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3048303850 >> 172.21.15.131:0/3048303850 conn(0x55c1a8398440 msgr2=0x55c1a83e3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.686+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3048303850 shutdown_connections 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.686+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3048303850 wait complete. 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.687+0000 7fe89fd1fb80 1 -- start start 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.687+0000 7fe89fd1fb80 1 Processor -- start 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 0x55c1a83c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a82961b0 con 0x55c1a83c1fa0 2023-12-26T21:52:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a8295e50 con 0x55c1a83c28b0 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1a81e2e40 con 0x55c1a83be170 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe89fd1fb80 10 client.0 _ll_register_callbacks cb 0x55c1a81e57b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35896/0 (socket says 172.21.15.131:35896) 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe88a7fc700 1 -- 172.21.15.131:0/3137031815 learned_addr learned my addr 172.21.15.131:0/3137031815 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe88affd700 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.688+0000 7fe88b7fe700 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 0x55c1a83c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe88a7fc700 1 -- 172.21.15.131:0/3137031815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 msgr2=0x55c1a83c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe88a7fc700 1 -- 172.21.15.131:0/3137031815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 msgr2=0x55c1a83c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 0x55c1a83c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe88a7fc700 1 -- 172.21.15.131:0/3137031815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88000f040 con 0x55c1a83c1fa0 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c9460 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe880013000 tx=0x7fe880019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11c0c194c4ff2db server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe8697fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe8697fa700 1 -- 172.21.15.131:0/3137031815 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880005d20 con 0x55c1a83c1fa0 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe8697fa700 1 -- 172.21.15.131:0/3137031815 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe880009890 con 0x55c1a83c1fa0 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3137031815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1a836aeb0 con 0x55c1a83c1fa0 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe8697fa700 1 -- 172.21.15.131:0/3137031815 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe880005e80 con 0x55c1a83c1fa0 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe89fd1fb80 10 client.6366 fetch_fsmap learned FSMap version 34 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3137031815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c1a8129b10 con 0x55c1a83c1fa0 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.689+0000 7fe8697fa700 1 -- 172.21.15.131:0/3137031815 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1a836aeb0 con 0x55c1a83c1fa0 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe8697fa700 1 -- 172.21.15.131:0/3137031815 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe880027070 con 0x55c1a83c1fa0 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe89fd1fb80 10 client.6366 fetch_fsmap finished waiting for FSMap version 34 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe89fd1fb80 10 client.6366 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe89fd1fb80 20 client.6366 populate_metadata read hostname 'smithi131' 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] conn(0x55c1a83f0f70 0x55c1a83f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe89fd1fb80 4 client.6366 mds_command: new command op to 6219 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T21:52:35.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3137031815 --> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1a82c0880 con 0x55c1a83f0f70 2023-12-26T21:52:35.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe88affd700 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] conn(0x55c1a83f0f70 0x55c1a83f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe8697fa700 10 client.6366 ms_handle_connect on v2:172.21.15.131:6833/4173752384 2023-12-26T21:52:35.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.690+0000 7fe88affd700 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] conn(0x55c1a83f0f70 0x55c1a83f1340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee8dd49008c8e875 server_cookie=b3f98998378d9562 in_seq=0 out_seq=0 2023-12-26T21:52:35.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe8697fa700 1 -- 172.21.15.131:0/3137031815 <== mds.0 v2:172.21.15.131:6833/4173752384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c1a82c0880 con 0x55c1a83f0f70 2023-12-26T21:52:35.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe8697fa700 10 client.6366 handle_command_reply: tid=0 2023-12-26T21:52:35.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe89fd1fb80 1 client.6366 shutdown 2023-12-26T21:52:35.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3137031815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 msgr2=0x55c1a83c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c9460 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe880013000 tx=0x7fe880019000 comp rx=0 tx=0).stop 2023-12-26T21:52:35.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3137031815 shutdown_connections 2023-12-26T21:52:35.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 0x55c1a83c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] conn(0x55c1a83f0f70 0x55c1a83f1340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/3137031815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a83c9460 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe8697fa700 0 client.6366 ms_handle_reset on v2:172.21.15.131:6833/4173752384 2023-12-26T21:52:35.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.691+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3137031815 >> 172.21.15.131:0/3137031815 conn(0x55c1a8398440 msgr2=0x55c1a83966c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:35.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.692+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3137031815 shutdown_connections 2023-12-26T21:52:35.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.692+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/3137031815 wait complete. 2023-12-26T21:52:35.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.692+0000 7fe89fd1fb80 20 client.6366 trim_cache size 0 max 16384 2023-12-26T21:52:35.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.709+0000 7fe89fd1fb80 1 Processor -- start 2023-12-26T21:52:35.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.709+0000 7fe89fd1fb80 1 -- start start 2023-12-26T21:52:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83be170 0x55c1a838fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 0x55c1a8390420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83c28b0 0x55c1a83eff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a83e4860 con 0x55c1a83be170 2023-12-26T21:52:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a8391fc0 con 0x55c1a83c1fa0 2023-12-26T21:52:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1a83e3ba0 con 0x55c1a83c28b0 2023-12-26T21:52:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83be170 0x55c1a838fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83be170 0x55c1a838fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35900/0 (socket says 172.21.15.131:35900) 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88affd700 1 -- 172.21.15.131:0/2190234924 learned_addr learned my addr 172.21.15.131:0/2190234924 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88b7fe700 1 --2- 172.21.15.131:0/2190234924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83c28b0 0x55c1a83eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/2190234924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 0x55c1a8390420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88affd700 1 -- 172.21.15.131:0/2190234924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83c28b0 msgr2=0x55c1a83eff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88affd700 1 --2- 172.21.15.131:0/2190234924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83c28b0 0x55c1a83eff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88affd700 1 -- 172.21.15.131:0/2190234924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 msgr2=0x55c1a8390420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88affd700 1 --2- 172.21.15.131:0/2190234924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 0x55c1a8390420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.710+0000 7fe88affd700 1 -- 172.21.15.131:0/2190234924 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c0b3030 con 0x55c1a83be170 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe88affd700 1 --2- 172.21.15.131:0/2190234924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83be170 0x55c1a838fee0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00e9b0 tx=0x7fe87c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9230aa0f2cd704 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe870ff9700 1 -- 172.21.15.131:0/2190234924 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c00ee20 con 0x55c1a83be170 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe870ff9700 1 -- 172.21.15.131:0/2190234924 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c026af0 con 0x55c1a83be170 2023-12-26T21:52:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe870ff9700 1 -- 172.21.15.131:0/2190234924 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c026e90 con 0x55c1a83be170 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2190234924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83be170 msgr2=0x55c1a838fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2190234924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83be170 0x55c1a838fee0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00e9b0 tx=0x7fe87c013000 comp rx=0 tx=0).stop 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2190234924 shutdown_connections 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2190234924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83c28b0 0x55c1a83eff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2190234924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83be170 0x55c1a838fee0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2190234924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c1fa0 0x55c1a8390420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2190234924 >> 172.21.15.131:0/2190234924 conn(0x55c1a8398440 msgr2=0x55c1a8395620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.711+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2190234924 shutdown_connections 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.712+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2190234924 wait complete. 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.712+0000 7fe89fd1fb80 1 -- start start 2023-12-26T21:52:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.712+0000 7fe89fd1fb80 1 Processor -- start 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.713+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.713+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a8445b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.713+0000 7fe89fd1fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 0x55c1a83f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.713+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a839b3a0 con 0x55c1a83c1fa0 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.713+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a8445160 con 0x55c1a83c28b0 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.713+0000 7fe89fd1fb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1a83f28b0 con 0x55c1a83be170 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.713+0000 7fe89fd1fb80 10 client.0 _ll_register_callbacks cb 0x7fe880001730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a8445b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a8445b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35904/0 (socket says 172.21.15.131:35904) 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88a7fc700 1 -- 172.21.15.131:0/2501857421 learned_addr learned my addr 172.21.15.131:0/2501857421 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88b7fe700 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 0x55c1a83f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88a7fc700 1 -- 172.21.15.131:0/2501857421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 msgr2=0x55c1a83c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88a7fc700 1 -- 172.21.15.131:0/2501857421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 msgr2=0x55c1a83f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 0x55c1a83f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88a7fc700 1 -- 172.21.15.131:0/2501857421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c00f040 con 0x55c1a83c1fa0 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe88a7fc700 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a8445b30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x55c1a83c8500 tx=0x7fe88001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbbc633c4b4fab1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe8697fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe8697fa700 1 -- 172.21.15.131:0/2501857421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800011f0 con 0x55c1a83c1fa0 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe8697fa700 1 -- 172.21.15.131:0/2501857421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe880009040 con 0x55c1a83c1fa0 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.714+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2501857421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe88c155880 con 0x55c1a83c1fa0 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe8697fa700 1 -- 172.21.15.131:0/2501857421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe880001350 con 0x55c1a83c1fa0 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe89fd1fb80 10 client.6372 fetch_fsmap learned FSMap version 34 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2501857421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c1a82b8290 con 0x55c1a83c1fa0 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe8697fa700 1 -- 172.21.15.131:0/2501857421 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c155880 con 0x55c1a83c1fa0 2023-12-26T21:52:35.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe8697fa700 1 -- 172.21.15.131:0/2501857421 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe880005d20 con 0x55c1a83c1fa0 2023-12-26T21:52:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe89fd1fb80 10 client.6372 fetch_fsmap finished waiting for FSMap version 34 2023-12-26T21:52:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe89fd1fb80 10 client.6372 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:52:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe89fd1fb80 20 client.6372 populate_metadata read hostname 'smithi131' 2023-12-26T21:52:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] conn(0x55c1a838fee0 0x55c1a83902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe89fd1fb80 4 client.6372 mds_command: new command op to 6219 tid=0[{"prefix": "status"}] 2023-12-26T21:52:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2501857421 --> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c1a838e490 con 0x55c1a838fee0 2023-12-26T21:52:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.715+0000 7fe88affd700 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] conn(0x55c1a838fee0 0x55c1a83902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe8697fa700 10 client.6372 ms_handle_connect on v2:172.21.15.131:6833/4173752384 2023-12-26T21:52:35.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe88affd700 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] conn(0x55c1a838fee0 0x55c1a83902b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723d4b14e325307f server_cookie=8dd7b4a561cd4561 in_seq=0 out_seq=0 2023-12-26T21:52:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe8697fa700 1 -- 172.21.15.131:0/2501857421 <== mds.0 v2:172.21.15.131:6833/4173752384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c1a838e490 con 0x55c1a838fee0 2023-12-26T21:52:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe8697fa700 10 client.6372 handle_command_reply: tid=0 2023-12-26T21:52:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 client.6372 shutdown 2023-12-26T21:52:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2501857421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 msgr2=0x55c1a8445b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a8445b30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x55c1a83c8500 tx=0x7fe88001f000 comp rx=0 tx=0).stop 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2501857421 shutdown_connections 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c1a83c28b0 0x55c1a83f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.131:6833/4173752384,v1:172.21.15.131:6836/4173752384] conn(0x55c1a838fee0 0x55c1a83902b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c1a83be170 0x55c1a83c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 --2- 172.21.15.131:0/2501857421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c1a83c1fa0 0x55c1a8445b30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2501857421 >> 172.21.15.131:0/2501857421 conn(0x55c1a8398440 msgr2=0x55c1a8395620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe8697fa700 0 client.6372 ms_handle_reset on v2:172.21.15.131:6833/4173752384 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2501857421 shutdown_connections 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 1 -- 172.21.15.131:0/2501857421 wait complete. 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.716+0000 7fe89fd1fb80 20 client.6372 trim_cache size 0 max 16384 2023-12-26T21:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6219, 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.4638228910000004, 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 34, 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 82, 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 18.579263682000001 2023-12-26T21:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:52:35.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 -- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe878066f30 msgr2=0x7fe8780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe878066f30 0x7fe8780693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe87c003950 tx=0x7fe87c009040 comp rx=0 tx=0).stop 2023-12-26T21:52:35.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 -- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe88c13c510 msgr2=0x7fe88c1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe88c13c510 0x7fe88c1539f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe88c06fe40 tx=0x7fe884019000 comp rx=0 tx=0).stop 2023-12-26T21:52:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 -- 172.21.15.131:0/1805368076 shutdown_connections 2023-12-26T21:52:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe88c13c510 0x7fe88c1539f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88c06bfd0 0x7fe88c15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe878066f30 0x7fe8780693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 --2- 172.21.15.131:0/1805368076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe88c0cc810 0x7fe88c15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 -- 172.21.15.131:0/1805368076 >> 172.21.15.131:0/1805368076 conn(0x7fe88c0ce3e0 msgr2=0x7fe88c0ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 -- 172.21.15.131:0/1805368076 shutdown_connections 2023-12-26T21:52:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.717+0000 7fe8915ff700 1 -- 172.21.15.131:0/1805368076 wait complete. 2023-12-26T21:52:35.734 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:52:35.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.925+0000 7f1b3bdff700 1 Processor -- start 2023-12-26T21:52:35.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.925+0000 7f1b3bdff700 1 -- start start 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.926+0000 7f1b3bdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b340b0750 0x7f1b340b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.926+0000 7f1b3bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340c9400 0x7f1b340c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.926+0000 7f1b3bdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b34156940 con 0x7f1b340cb320 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.926+0000 7f1b3bdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b340c8cc0 con 0x7f1b340b0750 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.926+0000 7f1b3bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b340ad700 con 0x7f1b340c9400 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.926+0000 7f1b3939a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b340b0750 0x7f1b340b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.926+0000 7f1b3939a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b340b0750 0x7f1b340b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35920/0 (socket says 172.21.15.131:35920) 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.926+0000 7f1b3939a700 1 -- 172.21.15.131:0/2874994767 learned_addr learned my addr 172.21.15.131:0/2874994767 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b3a39c700 1 --2- 172.21.15.131:0/2874994767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340c9400 0x7f1b340c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b3939a700 1 -- 172.21.15.131:0/2874994767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340c9400 msgr2=0x7f1b340c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b3939a700 1 --2- 172.21.15.131:0/2874994767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340c9400 0x7f1b340c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b3939a700 1 -- 172.21.15.131:0/2874994767 >> v1:172.21.15.190:6789/0 conn(0x7f1b340cb320 legacy=0x7f1b34069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b3939a700 1 -- 172.21.15.131:0/2874994767 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b340ae500 con 0x7f1b340b0750 2023-12-26T21:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b3a39c700 1 --2- 172.21.15.131:0/2874994767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340c9400 0x7f1b340c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b3939a700 1 --2- 172.21.15.131:0/2874994767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b0750 0x7f1b340b0bf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1b240099b0 tx=0x7f1b24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7281594f4ae457 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b38b99700 1 -- 172.21.15.131:0/2874994767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400b820 con 0x7f1b340b0750 2023-12-26T21:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b38b99700 1 -- 172.21.15.131:0/2874994767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2400b980 con 0x7f1b340b0750 2023-12-26T21:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.927+0000 7f1b38b99700 1 -- 172.21.15.131:0/2874994767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400bd20 con 0x7f1b340b0750 2023-12-26T21:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.928+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2874994767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b0750 msgr2=0x7f1b340b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.928+0000 7f1b3bdff700 1 --2- 172.21.15.131:0/2874994767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b0750 0x7f1b340b0bf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1b240099b0 tx=0x7f1b24009250 comp rx=0 tx=0).stop 2023-12-26T21:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.928+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2874994767 shutdown_connections 2023-12-26T21:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.928+0000 7f1b3bdff700 1 --2- 172.21.15.131:0/2874994767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340c9400 0x7f1b340c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.928+0000 7f1b3bdff700 1 --2- 172.21.15.131:0/2874994767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b0750 0x7f1b340b0bf0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.928+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2874994767 >> 172.21.15.131:0/2874994767 conn(0x7f1b34068040 msgr2=0x7f1b34069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.928+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2874994767 shutdown_connections 2023-12-26T21:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.928+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2874994767 wait complete. 2023-12-26T21:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.929+0000 7f1b3bdff700 1 Processor -- start 2023-12-26T21:52:35.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.929+0000 7f1b3bdff700 1 -- start start 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.930+0000 7f1b3bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340b0750 0x7f1b340c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.930+0000 7f1b3bdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340c9400 0x7f1b34148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.930+0000 7f1b3bdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340cb320 0x7f1b340c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.930+0000 7f1b3bdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b340c6a80 con 0x7f1b340cb320 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.930+0000 7f1b3bdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b340c39a0 con 0x7f1b340c9400 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.930+0000 7f1b3bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b340c3c80 con 0x7f1b340b0750 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3a39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340cb320 0x7f1b340c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b39b9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340b0750 0x7f1b340c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3a39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340cb320 0x7f1b340c4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35922/0 (socket says 172.21.15.131:35922) 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3a39c700 1 -- 172.21.15.131:0/2127579524 learned_addr learned my addr 172.21.15.131:0/2127579524 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b39b9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340b0750 0x7f1b340c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:59750/0 (socket says 172.21.15.131:59750) 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3939a700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340c9400 0x7f1b34148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3a39c700 1 -- 172.21.15.131:0/2127579524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340b0750 msgr2=0x7f1b340c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3a39c700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340b0750 0x7f1b340c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3a39c700 1 -- 172.21.15.131:0/2127579524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340c9400 msgr2=0x7f1b34148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3a39c700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340c9400 0x7f1b34148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3a39c700 1 -- 172.21.15.131:0/2127579524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b240094f0 con 0x7f1b340cb320 2023-12-26T21:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.931+0000 7f1b3a39c700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340cb320 0x7f1b340c4a40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00a380 tx=0x7f1b2c007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.932+0000 7f1b29ffb700 1 -- 172.21.15.131:0/2127579524 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c015440 con 0x7f1b340cb320 2023-12-26T21:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.932+0000 7f1b29ffb700 1 -- 172.21.15.131:0/2127579524 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c0155a0 con 0x7f1b340cb320 2023-12-26T21:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.932+0000 7f1b29ffb700 1 -- 172.21.15.131:0/2127579524 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c015960 con 0x7f1b340cb320 2023-12-26T21:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.932+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b340c3f60 con 0x7f1b340cb320 2023-12-26T21:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.932+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b340c4520 con 0x7f1b340cb320 2023-12-26T21:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.933+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b340657e0 con 0x7f1b340cb320 2023-12-26T21:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.934+0000 7f1b29ffb700 1 -- 172.21.15.131:0/2127579524 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1b2c00d440 con 0x7f1b340cb320 2023-12-26T21:52:35.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.935+0000 7f1b29ffb700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b18066f30 0x7f1b180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:35.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.935+0000 7f1b29ffb700 1 -- 172.21.15.131:0/2127579524 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f1b2c085260 con 0x7f1b340cb320 2023-12-26T21:52:35.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.935+0000 7f1b39b9b700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b18066f30 0x7f1b180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:35.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.935+0000 7f1b39b9b700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b18066f30 0x7f1b180693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009af0 tx=0x7f1b30009060 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:35.939+0000 7f1b29ffb700 1 -- 172.21.15.131:0/2127579524 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b2c052340 con 0x7f1b340cb320 2023-12-26T21:52:36.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.174+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b340b1550 con 0x7f1b340cb320 2023-12-26T21:52:36.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.175+0000 7f1b29ffb700 1 -- 172.21.15.131:0/2127579524 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1b2c055960 con 0x7f1b340cb320 2023-12-26T21:52:36.175 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b18066f30 msgr2=0x7f1b180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b18066f30 0x7f1b180693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009af0 tx=0x7f1b30009060 comp rx=0 tx=0).stop 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340cb320 msgr2=0x7f1b340c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340cb320 0x7f1b340c4a40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00a380 tx=0x7f1b2c007900 comp rx=0 tx=0).stop 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 shutdown_connections 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340cb320 0x7f1b340c4a40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b340b0750 0x7f1b340c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b18066f30 0x7f1b180693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 --2- 172.21.15.131:0/2127579524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340c9400 0x7f1b34148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 >> 172.21.15.131:0/2127579524 conn(0x7f1b34068040 msgr2=0x7f1b3415bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:36.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 shutdown_connections 2023-12-26T21:52:36.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:36.178+0000 7f1b3bdff700 1 -- 172.21.15.131:0/2127579524 wait complete. 2023-12-26T21:52:36.195 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T21:52:36.195 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:52:36.195 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T21:52:36.195 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T21:52:36.195 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T21:52:36.195 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T21:52:36.195 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T21:52:36.195 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:52:36.195 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:52:36.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:52:36.196 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T21:52:36.239 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T21:52:36.240 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41500sec preferred_lft 41500sec 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:52:36.241 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T21:52:36.242 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T21:52:36.242 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:52:36.242 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:52:36.242 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T21:52:36.242 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T21:52:36.242 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:52:36.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:52:36.243 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:52:36.349 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T21:52:36.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:52:36.353 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T21:52:36.398 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T21:52:36.399 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:52:36.399 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:52:36.399 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:52:36.399 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:52:36.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:52:36.400 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:52:36.400 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:52:36.400 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:52:36.400 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:52:36.400 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:52:36.400 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:52:36.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:52:36.570 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:52:36.613 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T21:52:36.613 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T21:52:36.614 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:52:36.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:52:36.614 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T21:52:36.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:52:36.723 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T21:52:36.765 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:52:36.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:52:36.766 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:52:36.849 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T21:52:36.850 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:52:36.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:52:36.988 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:52:36.988 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:52:37.015 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:52:37.015 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:52:37.015 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:52:37.015 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:52:37.015 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:52:37.020 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:52:37.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:52:37.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:52:37.089 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T21:52:37.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.284+0000 7f6aa15ff700 1 Processor -- start 2023-12-26T21:52:37.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.284+0000 7f6aa15ff700 1 -- start start 2023-12-26T21:52:37.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.285+0000 7f6aa15ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6a9c0cb320 0x7f6a9c069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6aa15ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 0x7f6a9c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6aa15ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9c156940 con 0x7f6a9c0b0c90 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6aa15ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9c0c8cc0 con 0x7f6a9c0cb320 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6aa15ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0ad700 con 0x7f6a9c06a1e0 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6a9affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6a9c0cb320 0x7f6a9c069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6a9b7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6a9c0b0c90 0x7f6a9c0aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45766/0 (socket says 172.21.15.131:45766) 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6a9affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6a9c0cb320 0x7f6a9c069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35928/0 (socket says 172.21.15.131:35928) 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6a9b7fe700 1 -- 172.21.15.131:0/266670446 learned_addr learned my addr 172.21.15.131:0/266670446 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6a9a7fc700 1 --2- 172.21.15.131:0/266670446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 0x7f6a9c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6a9affd700 1 -- 172.21.15.131:0/266670446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 msgr2=0x7f6a9c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6a9affd700 1 --2- 172.21.15.131:0/266670446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 0x7f6a9c0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6a9affd700 1 -- 172.21.15.131:0/266670446 >> v1:172.21.15.131:6789/0 conn(0x7f6a9c0b0c90 legacy=0x7f6a9c0aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.286+0000 7f6a9affd700 1 -- 172.21.15.131:0/266670446 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0ae500 con 0x7f6a9c0cb320 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.287+0000 7f6a9affd700 1 --2- 172.21.15.131:0/266670446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 0x7f6a9c069ca0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0098e0 tx=0x7f6a8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3591b2018a34f32e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.287+0000 7f6a99ffb700 1 -- 172.21.15.131:0/266670446 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c009e80 con 0x7f6a9c0cb320 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.287+0000 7f6a99ffb700 1 -- 172.21.15.131:0/266670446 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c00b860 con 0x7f6a9c0cb320 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.287+0000 7f6a99ffb700 1 -- 172.21.15.131:0/266670446 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00bbe0 con 0x7f6a9c0cb320 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.287+0000 7f6aa15ff700 1 -- 172.21.15.131:0/266670446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 msgr2=0x7f6a9c069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.287+0000 7f6aa15ff700 1 --2- 172.21.15.131:0/266670446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 0x7f6a9c069ca0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0098e0 tx=0x7f6a8c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.287+0000 7f6aa15ff700 1 -- 172.21.15.131:0/266670446 shutdown_connections 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.287+0000 7f6aa15ff700 1 --2- 172.21.15.131:0/266670446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 0x7f6a9c069ca0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.288+0000 7f6aa15ff700 1 --2- 172.21.15.131:0/266670446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 0x7f6a9c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.288+0000 7f6aa15ff700 1 -- 172.21.15.131:0/266670446 >> 172.21.15.131:0/266670446 conn(0x7f6a9c068040 msgr2=0x7f6a9c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.288+0000 7f6aa15ff700 1 -- 172.21.15.131:0/266670446 shutdown_connections 2023-12-26T21:52:37.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.288+0000 7f6aa15ff700 1 -- 172.21.15.131:0/266670446 wait complete. 2023-12-26T21:52:37.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.290+0000 7f6aa15ff700 1 Processor -- start 2023-12-26T21:52:37.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.290+0000 7f6aa15ff700 1 -- start start 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.291+0000 7f6aa15ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 0x7f6a9c0c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.291+0000 7f6aa15ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9c0b0c90 0x7f6a9c158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.291+0000 7f6aa15ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 0x7f6a9c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.291+0000 7f6aa15ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0c6650 con 0x7f6a9c0cb320 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.291+0000 7f6aa15ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0c5a40 con 0x7f6a9c0b0c90 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.291+0000 7f6aa15ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0c5d20 con 0x7f6a9c06a1e0 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.291+0000 7f6a9b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 0x7f6a9c149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.291+0000 7f6a9b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 0x7f6a9c149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35936/0 (socket says 172.21.15.131:35936) 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.291+0000 7f6a9b7fe700 1 -- 172.21.15.131:0/3251543406 learned_addr learned my addr 172.21.15.131:0/3251543406 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a9affd700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 0x7f6a9c0c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a9a7fc700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9c0b0c90 0x7f6a9c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a9b7fe700 1 -- 172.21.15.131:0/3251543406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 msgr2=0x7f6a9c0c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a9b7fe700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 0x7f6a9c0c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a9b7fe700 1 -- 172.21.15.131:0/3251543406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9c0b0c90 msgr2=0x7f6a9c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a9b7fe700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9c0b0c90 0x7f6a9c158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a9b7fe700 1 -- 172.21.15.131:0/3251543406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c009420 con 0x7f6a9c0cb320 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a9b7fe700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 0x7f6a9c149bc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400cc40 tx=0x7f6a94001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a82ffd700 1 -- 172.21.15.131:0/3251543406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400a8b0 con 0x7f6a9c0cb320 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a82ffd700 1 -- 172.21.15.131:0/3251543406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9400aa10 con 0x7f6a9c0cb320 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c0c6000 con 0x7f6a9c0cb320 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6a82ffd700 1 -- 172.21.15.131:0/3251543406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400add0 con 0x7f6a9c0cb320 2023-12-26T21:52:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.292+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c14a100 con 0x7f6a9c0cb320 2023-12-26T21:52:37.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.294+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9c156710 con 0x7f6a9c0cb320 2023-12-26T21:52:37.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.294+0000 7f6a82ffd700 1 -- 172.21.15.131:0/3251543406 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6a94007300 con 0x7f6a9c0cb320 2023-12-26T21:52:37.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.295+0000 7f6a82ffd700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f6a88066f30 0x7f6a880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.295+0000 7f6a82ffd700 1 -- 172.21.15.131:0/3251543406 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f6a940849c0 con 0x7f6a9c0cb320 2023-12-26T21:52:37.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.296+0000 7f6a9affd700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f6a88066f30 0x7f6a880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.296+0000 7f6a9affd700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f6a88066f30 0x7f6a880693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0091e0 tx=0x7f6a8c019000 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:37.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.300+0000 7f6a82ffd700 1 -- 172.21.15.131:0/3251543406 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a94051b20 con 0x7f6a9c0cb320 2023-12-26T21:52:37.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.523+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 --> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a9c0657e0 con 0x7f6a88066f30 2023-12-26T21:52:37.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.525+0000 7f6a82ffd700 1 -- 172.21.15.131:0/3251543406 <== mgr.6045 v2:172.21.15.131:6835/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6a9c0657e0 con 0x7f6a88066f30 2023-12-26T21:52:37.524 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:52:37.524 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:52:37.524 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T21:52:37.524 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:52:37.524 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:52:37.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f6a88066f30 msgr2=0x7f6a880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f6a88066f30 0x7f6a880693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0091e0 tx=0x7f6a8c019000 comp rx=0 tx=0).stop 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 msgr2=0x7f6a9c149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 0x7f6a9c149bc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400cc40 tx=0x7f6a94001a50 comp rx=0 tx=0).stop 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 shutdown_connections 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a9c0cb320 0x7f6a9c149bc0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a9c06a1e0 0x7f6a9c0c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f6a88066f30 0x7f6a880693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 --2- 172.21.15.131:0/3251543406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9c0b0c90 0x7f6a9c158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 >> 172.21.15.131:0/3251543406 conn(0x7f6a9c068040 msgr2=0x7f6a9c0cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 shutdown_connections 2023-12-26T21:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.527+0000 7f6aa15ff700 1 -- 172.21.15.131:0/3251543406 wait complete. 2023-12-26T21:52:37.545 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T21:52:37.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.735+0000 7f2a699ff700 1 Processor -- start 2023-12-26T21:52:37.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.735+0000 7f2a699ff700 1 -- start start 2023-12-26T21:52:37.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a699ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640b0750 0x7f2a640b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a699ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a640c9400 0x7f2a640c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a699ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a64156940 con 0x7f2a640cb320 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a699ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a640c8cc0 con 0x7f2a640c9400 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a699ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a640ad700 con 0x7f2a640b0750 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a637fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a640c9400 0x7f2a640c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a62ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2a640cb320 0x7f2a64069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45768/0 (socket says 172.21.15.131:45768) 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a62ffd700 1 -- 172.21.15.131:0/1262314322 learned_addr learned my addr 172.21.15.131:0/1262314322 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a627fc700 1 --2- 172.21.15.131:0/1262314322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640b0750 0x7f2a640b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a637fe700 1 -- 172.21.15.131:0/1262314322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640b0750 msgr2=0x7f2a640b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a637fe700 1 --2- 172.21.15.131:0/1262314322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640b0750 0x7f2a640b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a637fe700 1 -- 172.21.15.131:0/1262314322 >> v1:172.21.15.131:6789/0 conn(0x7f2a640cb320 legacy=0x7f2a64069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a637fe700 1 -- 172.21.15.131:0/1262314322 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a640b1100 con 0x7f2a640c9400 2023-12-26T21:52:37.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.737+0000 7f2a61ffb700 1 -- 172.21.15.131:0/1262314322 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120007075 0 0) 0x7f2a64156940 con 0x7f2a640cb320 2023-12-26T21:52:37.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.738+0000 7f2a637fe700 1 --2- 172.21.15.131:0/1262314322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640c9400 0x7f2a640c9880 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009e60 tx=0x7f2a5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1694768c7f960ce server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:37.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.738+0000 7f2a61ffb700 1 -- 172.21.15.131:0/1262314322 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00e070 con 0x7f2a640c9400 2023-12-26T21:52:37.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.738+0000 7f2a61ffb700 1 -- 172.21.15.131:0/1262314322 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c00bb60 con 0x7f2a640c9400 2023-12-26T21:52:37.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.738+0000 7f2a61ffb700 1 -- 172.21.15.131:0/1262314322 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c012430 con 0x7f2a640c9400 2023-12-26T21:52:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.738+0000 7f2a699ff700 1 -- 172.21.15.131:0/1262314322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640c9400 msgr2=0x7f2a640c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.738+0000 7f2a699ff700 1 --2- 172.21.15.131:0/1262314322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640c9400 0x7f2a640c9880 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009e60 tx=0x7f2a5c009780 comp rx=0 tx=0).stop 2023-12-26T21:52:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.739+0000 7f2a699ff700 1 -- 172.21.15.131:0/1262314322 shutdown_connections 2023-12-26T21:52:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.739+0000 7f2a699ff700 1 --2- 172.21.15.131:0/1262314322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640c9400 0x7f2a640c9880 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.739+0000 7f2a699ff700 1 --2- 172.21.15.131:0/1262314322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640b0750 0x7f2a640b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.739+0000 7f2a699ff700 1 -- 172.21.15.131:0/1262314322 >> 172.21.15.131:0/1262314322 conn(0x7f2a64068040 msgr2=0x7f2a64069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.739+0000 7f2a699ff700 1 -- 172.21.15.131:0/1262314322 shutdown_connections 2023-12-26T21:52:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.739+0000 7f2a699ff700 1 -- 172.21.15.131:0/1262314322 wait complete. 2023-12-26T21:52:37.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.741+0000 7f2a699ff700 1 Processor -- start 2023-12-26T21:52:37.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.741+0000 7f2a699ff700 1 -- start start 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.742+0000 7f2a699ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a640b0750 0x7f2a64147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.742+0000 7f2a699ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640c9400 0x7f2a640c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.742+0000 7f2a699ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640cb320 0x7f2a64169270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.742+0000 7f2a699ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a640aa000 con 0x7f2a640cb320 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.742+0000 7f2a699ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a640a9430 con 0x7f2a640b0750 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.742+0000 7f2a699ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a640a96d0 con 0x7f2a640c9400 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.742+0000 7f2a637fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640cb320 0x7f2a64169270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.742+0000 7f2a637fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640cb320 0x7f2a64169270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35950/0 (socket says 172.21.15.131:35950) 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.742+0000 7f2a637fe700 1 -- 172.21.15.131:0/3119398217 learned_addr learned my addr 172.21.15.131:0/3119398217 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a62ffd700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a640b0750 0x7f2a64147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a627fc700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640c9400 0x7f2a640c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a637fe700 1 -- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640c9400 msgr2=0x7f2a640c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a637fe700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640c9400 0x7f2a640c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a637fe700 1 -- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a640b0750 msgr2=0x7f2a64147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a637fe700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a640b0750 0x7f2a64147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a637fe700 1 -- 172.21.15.131:0/3119398217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c009a20 con 0x7f2a640cb320 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a627fc700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640c9400 0x7f2a640c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a62ffd700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a640b0750 0x7f2a64147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a637fe700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640cb320 0x7f2a64169270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00b6c0 tx=0x7f2a5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a4affd700 1 -- 172.21.15.131:0/3119398217 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c022070 con 0x7f2a640cb320 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a4affd700 1 -- 172.21.15.131:0/3119398217 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c012e20 con 0x7f2a640cb320 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a640a99b0 con 0x7f2a640cb320 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a4affd700 1 -- 172.21.15.131:0/3119398217 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c01b980 con 0x7f2a640cb320 2023-12-26T21:52:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.743+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a64148180 con 0x7f2a640cb320 2023-12-26T21:52:37.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.744+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a640aa170 con 0x7f2a640cb320 2023-12-26T21:52:37.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.745+0000 7f2a4affd700 1 -- 172.21.15.131:0/3119398217 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 28) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2a5c003040 con 0x7f2a640cb320 2023-12-26T21:52:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.746+0000 7f2a4affd700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2a50066f30 0x7f2a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.747+0000 7f2a4affd700 1 -- 172.21.15.131:0/3119398217 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f2a5c087e20 con 0x7f2a640cb320 2023-12-26T21:52:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.747+0000 7f2a62ffd700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2a50066f30 0x7f2a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.747+0000 7f2a62ffd700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2a50066f30 0x7f2a500693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2a54001890 tx=0x7f2a54009040 comp rx=0 tx=0).ready entity=mgr.6045 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.751+0000 7f2a4affd700 1 -- 172.21.15.131:0/3119398217 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a5c054f00 con 0x7f2a640cb320 2023-12-26T21:52:37.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:37.999+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2a640657e0 con 0x7f2a640cb320 2023-12-26T21:52:38.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.296+0000 7f2a4affd700 1 -- 172.21.15.131:0/3119398217 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 29) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f2a5c0552d0 con 0x7f2a640cb320 2023-12-26T21:52:38.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.298+0000 7f2a4affd700 1 -- 172.21.15.131:0/3119398217 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v29) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2a5c058990 con 0x7f2a640cb320 2023-12-26T21:52:38.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2a50066f30 msgr2=0x7f2a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:38.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2a50066f30 0x7f2a500693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2a54001890 tx=0x7f2a54009040 comp rx=0 tx=0).stop 2023-12-26T21:52:38.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640cb320 msgr2=0x7f2a64169270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:38.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640cb320 0x7f2a64169270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00b6c0 tx=0x7f2a5c019000 comp rx=0 tx=0).stop 2023-12-26T21:52:38.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 shutdown_connections 2023-12-26T21:52:38.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a640cb320 0x7f2a64169270 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2a50066f30 0x7f2a500693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a640b0750 0x7f2a64147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 --2- 172.21.15.131:0/3119398217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a640c9400 0x7f2a640c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 >> 172.21.15.131:0/3119398217 conn(0x7f2a64068040 msgr2=0x7f2a64158ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.300+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 shutdown_connections 2023-12-26T21:52:38.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.301+0000 7f2a699ff700 1 -- 172.21.15.131:0/3119398217 wait complete. 2023-12-26T21:52:38.319 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T21:52:38.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.526+0000 7fe9317ff700 1 Processor -- start 2023-12-26T21:52:38.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.527+0000 7fe9317ff700 1 -- start start 2023-12-26T21:52:38.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe9317ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe92c0cc580 0x7fe92c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:38.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe9317ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe92c0b00e0 0x7fe92c06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:38.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe9317ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c0b0d70 con 0x7fe92c0b0560 2023-12-26T21:52:38.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe9317ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0680f0 con 0x7fe92c0b00e0 2023-12-26T21:52:38.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe9317ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0675b0 con 0x7fe92c0cc580 2023-12-26T21:52:38.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe92a7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe92c0b00e0 0x7fe92c06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe92b7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe92c0b0560 0x7fe92c15d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45774/0 (socket says 172.21.15.131:45774) 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe92a7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe92c0b00e0 0x7fe92c06e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35980/0 (socket says 172.21.15.131:35980) 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe92b7fe700 1 -- 172.21.15.131:0/1468198611 learned_addr learned my addr 172.21.15.131:0/1468198611 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.528+0000 7fe92affd700 1 --2- 172.21.15.131:0/1468198611 >> v2:172.21.15.190:3300/0 conn(0x7fe92c0cc580 0x7fe92c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.529+0000 7fe92a7fc700 1 -- 172.21.15.131:0/1468198611 >> v2:172.21.15.190:3300/0 conn(0x7fe92c0cc580 msgr2=0x7fe92c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.529+0000 7fe92a7fc700 1 --2- 172.21.15.131:0/1468198611 >> v2:172.21.15.190:3300/0 conn(0x7fe92c0cc580 0x7fe92c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.529+0000 7fe92a7fc700 1 -- 172.21.15.131:0/1468198611 >> v1:172.21.15.131:6789/0 conn(0x7fe92c0b0560 legacy=0x7fe92c15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.529+0000 7fe929ffb700 1 -- 172.21.15.131:0/1468198611 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059231519 0 0) 0x7fe92c0b0d70 con 0x7fe92c0b0560 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.529+0000 7fe92a7fc700 1 -- 172.21.15.131:0/1468198611 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c159c30 con 0x7fe92c0b00e0 2023-12-26T21:52:38.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.529+0000 7fe92a7fc700 1 --2- 172.21.15.131:0/1468198611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 0x7fe92c06e5f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe92c06b740 tx=0x7fe9200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e7549ec12c3971 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:38.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.529+0000 7fe929ffb700 1 -- 172.21.15.131:0/1468198611 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920011630 con 0x7fe92c0b00e0 2023-12-26T21:52:38.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.529+0000 7fe929ffb700 1 -- 172.21.15.131:0/1468198611 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe920011790 con 0x7fe92c0b00e0 2023-12-26T21:52:38.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.529+0000 7fe929ffb700 1 -- 172.21.15.131:0/1468198611 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920011b30 con 0x7fe92c0b00e0 2023-12-26T21:52:38.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.530+0000 7fe9317ff700 1 -- 172.21.15.131:0/1468198611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 msgr2=0x7fe92c06e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:38.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.530+0000 7fe9317ff700 1 --2- 172.21.15.131:0/1468198611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 0x7fe92c06e5f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe92c06b740 tx=0x7fe9200091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.530+0000 7fe9317ff700 1 -- 172.21.15.131:0/1468198611 shutdown_connections 2023-12-26T21:52:38.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.530+0000 7fe9317ff700 1 --2- 172.21.15.131:0/1468198611 >> v2:172.21.15.190:3300/0 conn(0x7fe92c0cc580 0x7fe92c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.530+0000 7fe9317ff700 1 --2- 172.21.15.131:0/1468198611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 0x7fe92c06e5f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.530+0000 7fe9317ff700 1 -- 172.21.15.131:0/1468198611 >> 172.21.15.131:0/1468198611 conn(0x7fe92c0686c0 msgr2=0x7fe92c068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:38.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.530+0000 7fe9317ff700 1 -- 172.21.15.131:0/1468198611 shutdown_connections 2023-12-26T21:52:38.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.530+0000 7fe9317ff700 1 -- 172.21.15.131:0/1468198611 wait complete. 2023-12-26T21:52:38.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.531+0000 7fe9317ff700 1 Processor -- start 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.532+0000 7fe9317ff700 1 -- start start 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.532+0000 7fe9317ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 0x7fe92c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.532+0000 7fe9317ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe92c0b0560 0x7fe92c0aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.532+0000 7fe9317ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe92c0cc580 0x7fe92c0af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.532+0000 7fe9317ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0c3ae0 con 0x7fe92c0b00e0 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.532+0000 7fe9317ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0c2ed0 con 0x7fe92c0b0560 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.532+0000 7fe9317ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0c31b0 con 0x7fe92c0cc580 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 0x7fe92c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 0x7fe92c156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35986/0 (socket says 172.21.15.131:35986) 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92affd700 1 -- 172.21.15.131:0/2631705615 learned_addr learned my addr 172.21.15.131:0/2631705615 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92a7fc700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe92c0b0560 0x7fe92c0aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:38.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92b7fe700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe92c0cc580 0x7fe92c0af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:38.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92affd700 1 -- 172.21.15.131:0/2631705615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe92c0cc580 msgr2=0x7fe92c0af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:38.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92affd700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe92c0cc580 0x7fe92c0af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92affd700 1 -- 172.21.15.131:0/2631705615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe92c0b0560 msgr2=0x7fe92c0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:38.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92affd700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe92c0b0560 0x7fe92c0aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92affd700 1 -- 172.21.15.131:0/2631705615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe920009420 con 0x7fe92c0b00e0 2023-12-26T21:52:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92b7fe700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe92c0cc580 0x7fe92c0af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.533+0000 7fe92affd700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 0x7fe92c156b90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00e930 tx=0x7fe91c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.534+0000 7fe912ffd700 1 -- 172.21.15.131:0/2631705615 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c013680 con 0x7fe92c0b00e0 2023-12-26T21:52:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.534+0000 7fe912ffd700 1 -- 172.21.15.131:0/2631705615 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c0137e0 con 0x7fe92c0b00e0 2023-12-26T21:52:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.534+0000 7fe912ffd700 1 -- 172.21.15.131:0/2631705615 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c013b80 con 0x7fe92c0b00e0 2023-12-26T21:52:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.534+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c0c3490 con 0x7fe92c0b00e0 2023-12-26T21:52:38.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.534+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c1570d0 con 0x7fe92c0b00e0 2023-12-26T21:52:38.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.535+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c0657e0 con 0x7fe92c0b00e0 2023-12-26T21:52:38.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.536+0000 7fe912ffd700 1 -- 172.21.15.131:0/2631705615 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 29) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fe91c00c460 con 0x7fe92c0b00e0 2023-12-26T21:52:38.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.537+0000 7fe912ffd700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe918066f80 0x7fe918069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:38.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.537+0000 7fe912ffd700 1 -- 172.21.15.131:0/2631705615 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe91c084890 con 0x7fe92c0b00e0 2023-12-26T21:52:38.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.537+0000 7fe92a7fc700 1 -- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe918066f80 msgr2=0x7fe918069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:38.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.537+0000 7fe92a7fc700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe918066f80 0x7fe918069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:38.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.541+0000 7fe912ffd700 1 -- 172.21.15.131:0/2631705615 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe91c051970 con 0x7fe92c0b00e0 2023-12-26T21:52:38.732 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:38.732+0000 7f9d480531c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:52:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.738+0000 7fe92a7fc700 1 -- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe918066f80 msgr2=0x7fe918069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.738+0000 7fe92a7fc700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe918066f80 0x7fe918069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:38.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.783+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe92c067850 con 0x7fe92c0b00e0 2023-12-26T21:52:38.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.787+0000 7fe912ffd700 1 -- 172.21.15.131:0/2631705615 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe91c054f90 con 0x7fe92c0b00e0 2023-12-26T21:52:38.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe918066f80 msgr2=0x7fe918069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe918066f80 0x7fe918069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 msgr2=0x7fe92c156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 0x7fe92c156b90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00e930 tx=0x7fe91c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 shutdown_connections 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe92c0cc580 0x7fe92c0af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe92c0b00e0 0x7fe92c156b90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fe918066f80 0x7fe918069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 --2- 172.21.15.131:0/2631705615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe92c0b0560 0x7fe92c0aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 >> 172.21.15.131:0/2631705615 conn(0x7fe92c0686c0 msgr2=0x7fe92c0aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 shutdown_connections 2023-12-26T21:52:38.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:38.790+0000 7fe9317ff700 1 -- 172.21.15.131:0/2631705615 wait complete. 2023-12-26T21:52:38.794 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:38.795+0000 7fa8a5c371c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:52:38.809 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T21:52:38.839 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:38.839+0000 7f9d480531c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:52:38.912 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:38.912+0000 7fa8a5c371c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:52:38.929 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:38.929+0000 7f9d480531c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:52:39.007 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:39.007+0000 7fa8a5c371c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:52:39.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.013+0000 7f29b81ff700 1 Processor -- start 2023-12-26T21:52:39.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.014+0000 7f29b81ff700 1 -- start start 2023-12-26T21:52:39.014 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:39.013+0000 7f9d480531c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:52:39.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b81ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00cc580 0x7f29b00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b81ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f29b00b00e0 0x7f29b006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b81ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b00b0d70 con 0x7f29b00b0550 2023-12-26T21:52:39.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b81ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b00680f0 con 0x7f29b00b00e0 2023-12-26T21:52:39.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b81ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00675b0 con 0x7f29b00cc580 2023-12-26T21:52:39.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b579a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f29b00b00e0 0x7f29b006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:39.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b579a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f29b00b00e0 0x7f29b006e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35988/0 (socket says 172.21.15.131:35988) 2023-12-26T21:52:39.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b679c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f29b00b0550 0x7f29b015d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45790/0 (socket says 172.21.15.131:45790) 2023-12-26T21:52:39.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b579a700 1 -- 172.21.15.131:0/23878920 learned_addr learned my addr 172.21.15.131:0/23878920 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:39.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.015+0000 7f29b5f9b700 1 --2- 172.21.15.131:0/23878920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00cc580 0x7f29b00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:39.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.016+0000 7f29b579a700 1 -- 172.21.15.131:0/23878920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00cc580 msgr2=0x7f29b00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:39.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.016+0000 7f29b579a700 1 --2- 172.21.15.131:0/23878920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00cc580 0x7f29b00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.016+0000 7f29b579a700 1 -- 172.21.15.131:0/23878920 >> v1:172.21.15.131:6789/0 conn(0x7f29b00b0550 legacy=0x7f29b015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.016+0000 7f29b579a700 1 -- 172.21.15.131:0/23878920 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0159c30 con 0x7f29b00b00e0 2023-12-26T21:52:39.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.016+0000 7f29b579a700 1 --2- 172.21.15.131:0/23878920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 0x7f29b006e5f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f29a00099b0 tx=0x7f29a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be63e02ba4982134 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:39.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.016+0000 7f29b4f99700 1 -- 172.21.15.131:0/23878920 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0009ea0 con 0x7f29b00b00e0 2023-12-26T21:52:39.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.016+0000 7f29b4f99700 1 -- 172.21.15.131:0/23878920 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a000b860 con 0x7f29b00b00e0 2023-12-26T21:52:39.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.016+0000 7f29b4f99700 1 -- 172.21.15.131:0/23878920 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000bc00 con 0x7f29b00b00e0 2023-12-26T21:52:39.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.017+0000 7f29b81ff700 1 -- 172.21.15.131:0/23878920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 msgr2=0x7f29b006e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.017+0000 7f29b81ff700 1 --2- 172.21.15.131:0/23878920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 0x7f29b006e5f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f29a00099b0 tx=0x7f29a0009250 comp rx=0 tx=0).stop 2023-12-26T21:52:39.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.017+0000 7f29b81ff700 1 -- 172.21.15.131:0/23878920 shutdown_connections 2023-12-26T21:52:39.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.017+0000 7f29b81ff700 1 --2- 172.21.15.131:0/23878920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00cc580 0x7f29b00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.017+0000 7f29b81ff700 1 --2- 172.21.15.131:0/23878920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 0x7f29b006e5f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.017+0000 7f29b81ff700 1 -- 172.21.15.131:0/23878920 >> 172.21.15.131:0/23878920 conn(0x7f29b00686c0 msgr2=0x7f29b0068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:39.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.017+0000 7f29b81ff700 1 -- 172.21.15.131:0/23878920 shutdown_connections 2023-12-26T21:52:39.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.017+0000 7f29b81ff700 1 -- 172.21.15.131:0/23878920 wait complete. 2023-12-26T21:52:39.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.019+0000 7f29b81ff700 1 Processor -- start 2023-12-26T21:52:39.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.019+0000 7f29b81ff700 1 -- start start 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.020+0000 7f29b81ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 0x7f29b0142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.020+0000 7f29b81ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00b0550 0x7f29b00c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.020+0000 7f29b81ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b00cc580 0x7f29b00c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.020+0000 7f29b81ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b016a980 con 0x7f29b00b00e0 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.020+0000 7f29b81ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b015bf00 con 0x7f29b00cc580 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.020+0000 7f29b81ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b015c1e0 con 0x7f29b00b0550 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b5f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 0x7f29b0142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b5f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 0x7f29b0142be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35990/0 (socket says 172.21.15.131:35990) 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b5f9b700 1 -- 172.21.15.131:0/991697027 learned_addr learned my addr 172.21.15.131:0/991697027 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:39.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b579a700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00b0550 0x7f29b00c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:39.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b679c700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b00cc580 0x7f29b00c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:39.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b5f9b700 1 -- 172.21.15.131:0/991697027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00b0550 msgr2=0x7f29b00c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b5f9b700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00b0550 0x7f29b00c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b5f9b700 1 -- 172.21.15.131:0/991697027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b00cc580 msgr2=0x7f29b00c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b5f9b700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b00cc580 0x7f29b00c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b5f9b700 1 -- 172.21.15.131:0/991697027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00094f0 con 0x7f29b00b00e0 2023-12-26T21:52:39.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.021+0000 7f29b5f9b700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 0x7f29b0142be0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00e930 tx=0x7f29ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:39.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.022+0000 7f29a5ffb700 1 -- 172.21.15.131:0/991697027 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac013680 con 0x7f29b00b00e0 2023-12-26T21:52:39.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.022+0000 7f29a5ffb700 1 -- 172.21.15.131:0/991697027 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac0137e0 con 0x7f29b00b00e0 2023-12-26T21:52:39.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.022+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b015c4c0 con 0x7f29b00b00e0 2023-12-26T21:52:39.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.022+0000 7f29a5ffb700 1 -- 172.21.15.131:0/991697027 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac013b80 con 0x7f29b00b00e0 2023-12-26T21:52:39.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.022+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b0143120 con 0x7f29b00b00e0 2023-12-26T21:52:39.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.023+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b00657e0 con 0x7f29b00b00e0 2023-12-26T21:52:39.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.025+0000 7f29a5ffb700 1 -- 172.21.15.131:0/991697027 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 29) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f29ac00c3e0 con 0x7f29b00b00e0 2023-12-26T21:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.026+0000 7f29a5ffb700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2994066f80 0x7f2994069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.026+0000 7f29a5ffb700 1 -- 172.21.15.131:0/991697027 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f29ac0847d0 con 0x7f29b00b00e0 2023-12-26T21:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.026+0000 7f29b579a700 1 -- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2994066f80 msgr2=0x7f2994069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.026+0000 7f29b579a700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2994066f80 0x7f2994069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.033+0000 7f29a5ffb700 1 -- 172.21.15.131:0/991697027 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29ac0518b0 con 0x7f29b00b00e0 2023-12-26T21:52:39.098 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:39.098+0000 7fa8a5c371c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:52:39.204 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:39.204+0000 7f9d480531c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:39.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.227+0000 7f29b579a700 1 -- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2994066f80 msgr2=0x7f2994069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:39.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.227+0000 7f29b579a700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2994066f80 0x7f2994069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:39.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.236+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f29b0067850 con 0x7f29b00b00e0 2023-12-26T21:52:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.241+0000 7f29a5ffb700 1 -- 172.21.15.131:0/991697027 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f29ac054ed0 con 0x7f29b00b00e0 2023-12-26T21:52:39.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2994066f80 msgr2=0x7f2994069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:39.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2994066f80 0x7f2994069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 msgr2=0x7f29b0142be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 0x7f29b0142be0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00e930 tx=0x7f29ac0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 shutdown_connections 2023-12-26T21:52:39.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2994066f80 0x7f2994069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b00cc580 0x7f29b00c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b00b00e0 0x7f29b0142be0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 --2- 172.21.15.131:0/991697027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b00b0550 0x7f29b00c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 >> 172.21.15.131:0/991697027 conn(0x7f29b00686c0 msgr2=0x7f29b00aaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:39.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 shutdown_connections 2023-12-26T21:52:39.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.243+0000 7f29b81ff700 1 -- 172.21.15.131:0/991697027 wait complete. 2023-12-26T21:52:39.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:39.261 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-26T21:52:39.298 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:39.299+0000 7fa8a5c371c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:39.309 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-12-26T21:52:39.383 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:39.382+0000 7f9d480531c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:52:39.467 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:39.467+0000 7f9d480531c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:52:39.477 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:39.478+0000 7fa8a5c371c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:52:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.548+0000 7fa0c3bff700 1 Processor -- start 2023-12-26T21:52:39.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.549+0000 7fa0c3bff700 1 -- start start 2023-12-26T21:52:39.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.550+0000 7fa0c3bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc142f00 0x7fa0bc069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.550+0000 7fa0c3bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0bc069640 con 0x7fa0bc0cb510 2023-12-26T21:52:39.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.550+0000 7fa0c3bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0bc068750 con 0x7fa0bc0cc590 2023-12-26T21:52:39.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.550+0000 7fa0c3bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc068a30 con 0x7fa0bc142f00 2023-12-26T21:52:39.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.550+0000 7fa0c219c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa0bc0cb510 0x7fa0bc0c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45794/0 (socket says 172.21.15.131:45794) 2023-12-26T21:52:39.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.550+0000 7fa0c219c700 1 -- 172.21.15.131:0/2863779543 learned_addr learned my addr 172.21.15.131:0/2863779543 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:39.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.550+0000 7fa0c199b700 1 --2- 172.21.15.131:0/2863779543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc142f00 0x7fa0bc069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:39.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c0999700 1 -- 172.21.15.131:0/2863779543 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394011304 0 0) 0x7fa0bc069640 con 0x7fa0bc0cb510 2023-12-26T21:52:39.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c0999700 1 -- 172.21.15.131:0/2863779543 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a8003570 con 0x7fa0bc0cb510 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c0999700 1 -- 172.21.15.131:0/2863779543 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204590675 0 0) 0x7fa0bc068750 con 0x7fa0bc0cc590 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c0999700 1 -- 172.21.15.131:0/2863779543 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0bc069640 con 0x7fa0bc0cc590 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c199b700 1 -- 172.21.15.131:0/2863779543 >> v1:172.21.15.190:6789/0 conn(0x7fa0bc0cc590 legacy=0x7fa0bc0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c199b700 1 -- 172.21.15.131:0/2863779543 >> v1:172.21.15.131:6789/0 conn(0x7fa0bc0cb510 legacy=0x7fa0bc0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c199b700 1 -- 172.21.15.131:0/2863779543 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc068d10 con 0x7fa0bc142f00 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c199b700 1 --2- 172.21.15.131:0/2863779543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc142f00 0x7fa0bc069ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009910 tx=0x7fa0b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7766e004286154e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c0999700 1 -- 172.21.15.131:0/2863779543 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8011630 con 0x7fa0bc142f00 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c0999700 1 -- 172.21.15.131:0/2863779543 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8011790 con 0x7fa0bc142f00 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.551+0000 7fa0c0999700 1 -- 172.21.15.131:0/2863779543 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8011b30 con 0x7fa0bc142f00 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.552+0000 7fa0c3bff700 1 -- 172.21.15.131:0/2863779543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc142f00 msgr2=0x7fa0bc069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.552+0000 7fa0c3bff700 1 --2- 172.21.15.131:0/2863779543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc142f00 0x7fa0bc069ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009910 tx=0x7fa0b80091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.552+0000 7fa0c3bff700 1 -- 172.21.15.131:0/2863779543 shutdown_connections 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.552+0000 7fa0c3bff700 1 --2- 172.21.15.131:0/2863779543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc142f00 0x7fa0bc069ad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.552+0000 7fa0c3bff700 1 -- 172.21.15.131:0/2863779543 >> 172.21.15.131:0/2863779543 conn(0x7fa0bc063cd0 msgr2=0x7fa0bc06b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:39.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.552+0000 7fa0c3bff700 1 -- 172.21.15.131:0/2863779543 shutdown_connections 2023-12-26T21:52:39.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.552+0000 7fa0c3bff700 1 -- 172.21.15.131:0/2863779543 wait complete. 2023-12-26T21:52:39.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.553+0000 7fa0c3bff700 1 Processor -- start 2023-12-26T21:52:39.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.554+0000 7fa0c3bff700 1 -- start start 2023-12-26T21:52:39.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.554+0000 7fa0c3bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc0cb510 0x7fa0bc15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.554+0000 7fa0c3bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0bc0cc590 0x7fa0bc0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.554+0000 7fa0c3bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc142f00 0x7fa0bc0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.554+0000 7fa0c3bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0c7560 con 0x7fa0bc142f00 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.554+0000 7fa0c3bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc16ade0 con 0x7fa0bc0cc590 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.554+0000 7fa0c3bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15dca0 con 0x7fa0bc0cb510 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c199b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc0cb510 0x7fa0bc15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c219c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc142f00 0x7fa0bc0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c199b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc0cb510 0x7fa0bc15e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:59872/0 (socket says 172.21.15.131:59872) 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c199b700 1 -- 172.21.15.131:0/2626057145 learned_addr learned my addr 172.21.15.131:0/2626057145 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c119a700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0bc0cc590 0x7fa0bc0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c199b700 1 -- 172.21.15.131:0/2626057145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0bc0cc590 msgr2=0x7fa0bc0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c199b700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0bc0cc590 0x7fa0bc0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c199b700 1 -- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc142f00 msgr2=0x7fa0bc0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c199b700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc142f00 0x7fa0bc0ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:39.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c199b700 1 -- 172.21.15.131:0/2626057145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8009420 con 0x7fa0bc0cb510 2023-12-26T21:52:39.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.555+0000 7fa0c199b700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0bc0cb510 0x7fa0bc15e130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc06f3d0 tx=0x7fa0b8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:39.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.556+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8020070 con 0x7fa0bc0cb510 2023-12-26T21:52:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.556+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8002f70 con 0x7fa0bc0cb510 2023-12-26T21:52:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.556+0000 7fa0c3bff700 1 -- 172.21.15.131:0/2626057145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc16ab00 con 0x7fa0bc0cb510 2023-12-26T21:52:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.556+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8011900 con 0x7fa0bc0cb510 2023-12-26T21:52:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.556+0000 7fa0c3bff700 1 -- 172.21.15.131:0/2626057145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc15d930 con 0x7fa0bc0cb510 2023-12-26T21:52:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.557+0000 7fa0c3bff700 1 -- 172.21.15.131:0/2626057145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc15df80 con 0x7fa0bc0cb510 2023-12-26T21:52:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.558+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 29) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fa0b8024070 con 0x7fa0bc0cb510 2023-12-26T21:52:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.558+0000 7fa0b1ffb700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.558+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fa0b8085a40 con 0x7fa0bc0cb510 2023-12-26T21:52:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.559+0000 7fa0c119a700 1 -- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 msgr2=0x7fa0a0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.559+0000 7fa0c119a700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:39.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.563+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b8052a80 con 0x7fa0bc0cb510 2023-12-26T21:52:39.565 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:39.566+0000 7fa8a5c371c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:52:39.615 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:39.615+0000 7f9d480531c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:39.706 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:39.705+0000 7f9d480531c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:52:39.714 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:39.715+0000 7fa8a5c371c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:39.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.760+0000 7fa0c119a700 1 -- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 msgr2=0x7fa0a0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:39.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.760+0000 7fa0c119a700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:39.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:39.787+0000 7fa0c3bff700 1 -- 172.21.15.131:0/2626057145 --> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0bc15dde0 con 0x7fa0a0066f80 2023-12-26T21:52:39.806 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:39.806+0000 7fa8a5c371c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:52:39.888 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:39.888+0000 7f9d480531c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:52:39.990 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:39.991+0000 7fa8a5c371c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:52:40.038 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:40.038+0000 7f9d480531c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:52:40.139 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:40.140+0000 7fa8a5c371c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:52:40.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:40.161+0000 7fa0c119a700 1 -- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 msgr2=0x7fa0a0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:40.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:40.161+0000 7fa0c119a700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:52:40.422 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:40.422+0000 7f9d480531c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:52:40.521 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:40.522+0000 7fa8a5c371c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:52:40.688 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:40.688+0000 7f9d480531c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:52:40.785 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:40.785+0000 7fa8a5c371c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:52:40.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:40.962+0000 7fa0c119a700 1 -- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 msgr2=0x7fa0a0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:40.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:40.962+0000 7fa0c119a700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T21:52:40.966 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:40.966+0000 7f9d480531c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:52:41.055 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:41.055+0000 7f9d480531c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:52:41.062 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:41.063+0000 7fa8a5c371c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:52:41.151 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:41.151+0000 7fa8a5c371c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:52:41.791 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:41.791+0000 7f9d480531c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:52:41.883 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:41.883+0000 7fa8a5c371c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:52:41.885 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:41.885+0000 7f9d480531c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:52:41.976 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:41.977+0000 7fa8a5c371c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:52:42.037 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:42.037+0000 7f9d480531c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:52:42.127 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:42.128+0000 7fa8a5c371c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:52:42.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:42.564+0000 7fa0c119a700 1 -- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 msgr2=0x7fa0a0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:42.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:42.564+0000 7fa0c119a700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T21:52:42.660 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:42.660+0000 7f9d480531c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:52:42.753 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:42.754+0000 7fa8a5c371c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:52:42.846 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:42.846+0000 7f9d480531c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:52:42.941 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:42.942+0000 7fa8a5c371c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:52:43.034 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:43.034+0000 7f9d480531c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:52:43.131 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:43.131+0000 7fa8a5c371c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:52:43.874 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:43.874+0000 7f9d480531c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:52:43.976 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:43.976+0000 7fa8a5c371c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:52:44.145 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:44.145+0000 7f9d480531c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:44.239 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:44.239+0000 7f9d480531c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:44.253 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.254+0000 7fa8a5c371c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:44.349 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.349+0000 7fa8a5c371c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:44.438 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:44.438+0000 7f9d480531c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:52:44.546 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.547+0000 7fa8a5c371c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:52:44.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:44.565+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mgrmap(e 30) v1 ==== 84470+0+0 (secure 0 0 0) 0x7fa0b8052e50 con 0x7fa0bc0cb510 2023-12-26T21:52:44.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:44.566+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 msgr2=0x7fa0a0069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:44.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:44.566+0000 7fa0b1ffb700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:44.686 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8175b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:44.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8175b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:44.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8175b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:44.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8175b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:44.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8175b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:44.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8195b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:44.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8195b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:44.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8195b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:44.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8195b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:44.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:44.687+0000 7fa8195b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:52:45.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.562+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 <== mon.2 v2:172.21.15.190:3301/0 8 ==== mgrmap(e 31) v1 ==== 84597+0+0 (secure 0 0 0) 0x7fa0b80877e0 con 0x7fa0bc0cb510 2023-12-26T21:52:45.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.563+0000 7fa0b1ffb700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a00693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:45.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.563+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 --> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0a000e8c0 con 0x7fa0a0066f80 2023-12-26T21:52:45.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.565+0000 7fa0c119a700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a00693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:45.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.567+0000 7fa0c119a700 1 --2- 172.21.15.131:0/2626057145 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0a0066f80 0x7fa0a00693b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc15bb40 tx=0x7fa0ac007d50 comp rx=0 tx=0).ready entity=mgr.5717 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:45.576 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:45.575+0000 7fa8215c5700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-26T21:52:45.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.575+0000 7fa0b1ffb700 1 -- 172.21.15.131:0/2626057145 <== mgr.5717 v2:172.21.15.131:6835/39288 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fa0a000e8c0 con 0x7fa0a0066f80 2023-12-26T21:52:45.578 INFO:teuthology.orchestra.run.smithi131.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-26T21:52:45.587 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-26T21:52:45.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:45.588 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-26T21:52:45.635 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T21:52:45.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.826+0000 7fadec1ff700 1 Processor -- start 2023-12-26T21:52:45.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.826+0000 7fadec1ff700 1 -- start start 2023-12-26T21:52:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fadec1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fade40b0750 0x7fade40b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fadec1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 0x7fade40c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fadec1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade4156940 con 0x7fade40cb320 2023-12-26T21:52:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fadec1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade40c8cc0 con 0x7fade40b0750 2023-12-26T21:52:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fadec1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40ad700 con 0x7fade40c9400 2023-12-26T21:52:45.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fade979a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fade40b0750 0x7fade40b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fade9f9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fade40cb320 0x7fade4069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43594/0 (socket says 172.21.15.131:43594) 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fade9f9b700 1 -- 172.21.15.131:0/2218235270 learned_addr learned my addr 172.21.15.131:0/2218235270 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fade979a700 1 -- 172.21.15.131:0/2218235270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 msgr2=0x7fade40c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fadea79c700 1 --2- 172.21.15.131:0/2218235270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 0x7fade40c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fade8f99700 1 -- 172.21.15.131:0/2218235270 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896298813 0 0) 0x7fade4156940 con 0x7fade40cb320 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fade979a700 1 --2- 172.21.15.131:0/2218235270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 0x7fade40c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fade979a700 1 -- 172.21.15.131:0/2218235270 >> v1:172.21.15.131:6789/0 conn(0x7fade40cb320 legacy=0x7fade4069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fade979a700 1 -- 172.21.15.131:0/2218235270 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40ae500 con 0x7fade40b0750 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.828+0000 7fadea79c700 1 --2- 172.21.15.131:0/2218235270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 0x7fade40c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.829+0000 7fade979a700 1 --2- 172.21.15.131:0/2218235270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40b0750 0x7fade40b0bf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fadd400bb90 tx=0x7fadd400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6289a3fd6ab58b73 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.829+0000 7fade8f99700 1 -- 172.21.15.131:0/2218235270 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400dba0 con 0x7fade40b0750 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.829+0000 7fade8f99700 1 -- 172.21.15.131:0/2218235270 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd400dd00 con 0x7fade40b0750 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.829+0000 7fade8f99700 1 -- 172.21.15.131:0/2218235270 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40196b0 con 0x7fade40b0750 2023-12-26T21:52:45.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.829+0000 7fadec1ff700 1 -- 172.21.15.131:0/2218235270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40b0750 msgr2=0x7fade40b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:45.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.829+0000 7fadec1ff700 1 --2- 172.21.15.131:0/2218235270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40b0750 0x7fade40b0bf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fadd400bb90 tx=0x7fadd400b430 comp rx=0 tx=0).stop 2023-12-26T21:52:45.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.829+0000 7fadec1ff700 1 -- 172.21.15.131:0/2218235270 shutdown_connections 2023-12-26T21:52:45.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.829+0000 7fadec1ff700 1 --2- 172.21.15.131:0/2218235270 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 0x7fade40c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:45.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.829+0000 7fadec1ff700 1 --2- 172.21.15.131:0/2218235270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40b0750 0x7fade40b0bf0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:45.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.830+0000 7fadec1ff700 1 -- 172.21.15.131:0/2218235270 >> 172.21.15.131:0/2218235270 conn(0x7fade4068040 msgr2=0x7fade4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:45.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.830+0000 7fadec1ff700 1 -- 172.21.15.131:0/2218235270 shutdown_connections 2023-12-26T21:52:45.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.830+0000 7fadec1ff700 1 -- 172.21.15.131:0/2218235270 wait complete. 2023-12-26T21:52:45.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.831+0000 7fadec1ff700 1 Processor -- start 2023-12-26T21:52:45.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.831+0000 7fadec1ff700 1 -- start start 2023-12-26T21:52:45.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fadec1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fade40b0750 0x7fade4159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:45.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fadec1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 0x7fade4157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:45.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fadec1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40cb320 0x7fade4142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fadec1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade415a4b0 con 0x7fade40cb320 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fadec1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade4156a70 con 0x7fade40b0750 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fadec1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade4156d50 con 0x7fade40c9400 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fadea79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40cb320 0x7fade4142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fadea79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40cb320 0x7fade4142b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54040/0 (socket says 172.21.15.131:54040) 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fadea79c700 1 -- 172.21.15.131:0/2966804247 learned_addr learned my addr 172.21.15.131:0/2966804247 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fade979a700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 0x7fade4157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.832+0000 7fade9f9b700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fade40b0750 0x7fade4159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadea79c700 1 -- 172.21.15.131:0/2966804247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 msgr2=0x7fade4157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadea79c700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 0x7fade4157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadea79c700 1 -- 172.21.15.131:0/2966804247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fade40b0750 msgr2=0x7fade4159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadea79c700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fade40b0750 0x7fade4159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadea79c700 1 -- 172.21.15.131:0/2966804247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd400b6d0 con 0x7fade40cb320 2023-12-26T21:52:45.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadea79c700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40cb320 0x7fade4142b50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faddc00a380 tx=0x7faddc007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadd9ffb700 1 -- 172.21.15.131:0/2966804247 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc015440 con 0x7fade40cb320 2023-12-26T21:52:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadd9ffb700 1 -- 172.21.15.131:0/2966804247 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc0155a0 con 0x7fade40cb320 2023-12-26T21:52:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade4157030 con 0x7fade40cb320 2023-12-26T21:52:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadd9ffb700 1 -- 172.21.15.131:0/2966804247 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc015960 con 0x7fade40cb320 2023-12-26T21:52:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.833+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade4157e70 con 0x7fade40cb320 2023-12-26T21:52:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.835+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade40657e0 con 0x7fade40cb320 2023-12-26T21:52:45.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.836+0000 7fadd9ffb700 1 -- 172.21.15.131:0/2966804247 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 31) v1 ==== 84597+0+0 (secure 0 0 0) 0x7faddc00d440 con 0x7fade40cb320 2023-12-26T21:52:45.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.836+0000 7fadd9ffb700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fadc8066dd0 0x7fadc8069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:45.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.837+0000 7fadd9ffb700 1 -- 172.21.15.131:0/2966804247 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7faddc084f00 con 0x7fade40cb320 2023-12-26T21:52:45.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.837+0000 7fade9f9b700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fadc8066dd0 0x7fadc8069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:45.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.837+0000 7fade9f9b700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fadc8066dd0 0x7fadc8069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fade0007160 tx=0x7fade0007f50 comp rx=0 tx=0).ready entity=mgr.5717 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:45.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:45.840+0000 7fadd9ffb700 1 -- 172.21.15.131:0/2966804247 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faddc052190 con 0x7fade40cb320 2023-12-26T21:52:46.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.086+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fade40c5620 con 0x7fade40cb320 2023-12-26T21:52:46.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.569+0000 7fadd9ffb700 1 -- 172.21.15.131:0/2966804247 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7faddc052560 con 0x7fade40cb320 2023-12-26T21:52:46.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.570+0000 7fadd9ffb700 1 -- 172.21.15.131:0/2966804247 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v32) v1 ==== 93+0+0 (secure 0 0 0) 0x7faddc055c20 con 0x7fade40cb320 2023-12-26T21:52:46.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fadc8066dd0 msgr2=0x7fadc8069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:46.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fadc8066dd0 0x7fadc8069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fade0007160 tx=0x7fade0007f50 comp rx=0 tx=0).stop 2023-12-26T21:52:46.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40cb320 msgr2=0x7fade4142b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:46.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40cb320 0x7fade4142b50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faddc00a380 tx=0x7faddc007900 comp rx=0 tx=0).stop 2023-12-26T21:52:46.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 shutdown_connections 2023-12-26T21:52:46.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade40cb320 0x7fade4142b50 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:46.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fadc8066dd0 0x7fadc8069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:46.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fade40b0750 0x7fade4159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:46.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 --2- 172.21.15.131:0/2966804247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fade40c9400 0x7fade4157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:46.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.572+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 >> 172.21.15.131:0/2966804247 conn(0x7fade4068040 msgr2=0x7fade406d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:46.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.573+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 shutdown_connections 2023-12-26T21:52:46.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:46.573+0000 7fadec1ff700 1 -- 172.21.15.131:0/2966804247 wait complete. 2023-12-26T21:52:46.592 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:52:46.593 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:52:46.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:52:46.764 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:52:46.764 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T21:52:46.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:52:46.764 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T21:52:46.849 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:52:46.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:52:46.850 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T21:52:46.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:47.002 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:47.002+0000 7f86d929b1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:52:47.031 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:47.031+0000 7efe30cac1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:52:47.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.090+0000 7f5746fff700 1 Processor -- start 2023-12-26T21:52:47.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.090+0000 7f5746fff700 1 -- start start 2023-12-26T21:52:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f5746fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f57400cac30 0x7f5740155be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f5746fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57400c8bf0 con 0x7f5740159920 2023-12-26T21:52:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f5746fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57400c8e60 con 0x7f57400b07f0 2023-12-26T21:52:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f5746fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57400c9140 con 0x7f57400cac30 2023-12-26T21:52:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f5744d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5740159920 0x7f57400b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43610/0 (socket says 172.21.15.131:43610) 2023-12-26T21:52:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f5744d9b700 1 -- 172.21.15.131:0/3135987905 learned_addr learned my addr 172.21.15.131:0/3135987905 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f574559c700 1 --2- 172.21.15.131:0/3135987905 >> v2:172.21.15.131:3300/0 conn(0x7f57400cac30 0x7f5740155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f574559c700 1 -- 172.21.15.131:0/3135987905 >> v1:172.21.15.190:6789/0 conn(0x7f57400b07f0 legacy=0x7f57401554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f574559c700 1 -- 172.21.15.131:0/3135987905 >> v1:172.21.15.131:6789/0 conn(0x7f5740159920 legacy=0x7f57400b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f574559c700 1 -- 172.21.15.131:0/3135987905 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57400c9420 con 0x7f57400cac30 2023-12-26T21:52:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.093+0000 7f573f7fe700 1 -- 172.21.15.131:0/3135987905 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735180666 0 0) 0x7f57400c8bf0 con 0x7f5740159920 2023-12-26T21:52:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.094+0000 7f574559c700 1 --2- 172.21.15.131:0/3135987905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 0x7f5740155be0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f573800af70 tx=0x7f573800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8554c1c2e9e0422 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.094+0000 7f573f7fe700 1 -- 172.21.15.131:0/3135987905 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800f070 con 0x7f57400cac30 2023-12-26T21:52:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.094+0000 7f573f7fe700 1 -- 172.21.15.131:0/3135987905 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573800a2e0 con 0x7f57400cac30 2023-12-26T21:52:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.094+0000 7f573f7fe700 1 -- 172.21.15.131:0/3135987905 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800cd20 con 0x7f57400cac30 2023-12-26T21:52:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.094+0000 7f5746fff700 1 -- 172.21.15.131:0/3135987905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 msgr2=0x7f5740155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.094+0000 7f5746fff700 1 --2- 172.21.15.131:0/3135987905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 0x7f5740155be0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f573800af70 tx=0x7f573800a890 comp rx=0 tx=0).stop 2023-12-26T21:52:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.095+0000 7f5746fff700 1 -- 172.21.15.131:0/3135987905 shutdown_connections 2023-12-26T21:52:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.095+0000 7f5746fff700 1 --2- 172.21.15.131:0/3135987905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 0x7f5740155be0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.095+0000 7f5746fff700 1 -- 172.21.15.131:0/3135987905 >> 172.21.15.131:0/3135987905 conn(0x7f5740068040 msgr2=0x7f5740069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.095+0000 7f5746fff700 1 -- 172.21.15.131:0/3135987905 shutdown_connections 2023-12-26T21:52:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.095+0000 7f5746fff700 1 -- 172.21.15.131:0/3135987905 wait complete. 2023-12-26T21:52:47.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.096+0000 7f5746fff700 1 Processor -- start 2023-12-26T21:52:47.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.096+0000 7f5746fff700 1 -- start start 2023-12-26T21:52:47.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.097+0000 7f5746fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 0x7f57400acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.097+0000 7f5746fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5740159920 0x7f574006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.097+0000 7f5746fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57400c6d70 0x7f5740158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.097+0000 7f5746fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574006e440 con 0x7f57400cac30 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.097+0000 7f5746fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57400c5760 con 0x7f57400c6d70 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.097+0000 7f5746fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5740169490 con 0x7f5740159920 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f5744d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 0x7f57400acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f5744d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 0x7f57400acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54068/0 (socket says 172.21.15.131:54068) 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f5744d9b700 1 -- 172.21.15.131:0/670894668 learned_addr learned my addr 172.21.15.131:0/670894668 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f574559c700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57400c6d70 0x7f5740158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f573ffff700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5740159920 0x7f574006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f5744d9b700 1 -- 172.21.15.131:0/670894668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5740159920 msgr2=0x7f574006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f5744d9b700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5740159920 0x7f574006ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f5744d9b700 1 -- 172.21.15.131:0/670894668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57400c6d70 msgr2=0x7f5740158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f5744d9b700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57400c6d70 0x7f5740158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f5744d9b700 1 -- 172.21.15.131:0/670894668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573800ab30 con 0x7f57400cac30 2023-12-26T21:52:47.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f574559c700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57400c6d70 0x7f5740158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:47.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.098+0000 7f5744d9b700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 0x7f57400acd60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f573400c560 tx=0x7f5734000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:47.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.099+0000 7f573cff9700 1 -- 172.21.15.131:0/670894668 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734005b30 con 0x7f57400cac30 2023-12-26T21:52:47.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.099+0000 7f573cff9700 1 -- 172.21.15.131:0/670894668 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734005c90 con 0x7f57400cac30 2023-12-26T21:52:47.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.099+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574006d8c0 con 0x7f57400cac30 2023-12-26T21:52:47.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.099+0000 7f573cff9700 1 -- 172.21.15.131:0/670894668 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734007320 con 0x7f57400cac30 2023-12-26T21:52:47.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.099+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574006de80 con 0x7f57400cac30 2023-12-26T21:52:47.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.100+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57400657e0 con 0x7f57400cac30 2023-12-26T21:52:47.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.101+0000 7f573cff9700 1 -- 172.21.15.131:0/670894668 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f573401b3b0 con 0x7f57400cac30 2023-12-26T21:52:47.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.102+0000 7f573cff9700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f57240670e0 0x7f5724069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.102+0000 7f573cff9700 1 -- 172.21.15.131:0/670894668 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f5734083fa0 con 0x7f57400cac30 2023-12-26T21:52:47.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.102+0000 7f573ffff700 1 -- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f57240670e0 msgr2=0x7f5724069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:47.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.102+0000 7f573ffff700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f57240670e0 0x7f5724069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:47.106 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:47.106+0000 7f86d929b1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:52:47.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.106+0000 7f573cff9700 1 -- 172.21.15.131:0/670894668 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5734050c60 con 0x7f57400cac30 2023-12-26T21:52:47.145 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:47.144+0000 7efe30cac1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:52:47.195 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:47.195+0000 7f86d929b1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:52:47.235 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:47.235+0000 7efe30cac1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:52:47.278 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:47.278+0000 7f86d929b1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:52:47.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.303+0000 7f573ffff700 1 -- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f57240670e0 msgr2=0x7f5724069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:47.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.303+0000 7f573ffff700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f57240670e0 0x7f5724069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:47.320 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:47.320+0000 7efe30cac1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:52:47.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.353+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5740169620 con 0x7f57400cac30 2023-12-26T21:52:47.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.353+0000 7f573cff9700 1 -- 172.21.15.131:0/670894668 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f5734017090 con 0x7f57400cac30 2023-12-26T21:52:47.354 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:47.354 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/625115906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":625115906},{"type":"v1","addr":"172.21.15.190:6833","nonce":625115906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2773111229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2773111229},{"type":"v1","addr":"172.21.15.190:6835","nonce":2773111229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/899613179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":899613179},{"type":"v1","addr":"172.21.15.131:6834","nonce":899613179}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:52:28.726694+0000","modified":"2023-12-26T21:52:32.268974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6836/4173752384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":4173752384},{"type":"v1","addr":"172.21.15.131:6836","nonce":4173752384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f57240670e0 msgr2=0x7f5724069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f57240670e0 0x7f5724069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 msgr2=0x7f57400acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 0x7f57400acd60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f573400c560 tx=0x7f5734000fe0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 shutdown_connections 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f57240670e0 0x7f5724069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57400c6d70 0x7f5740158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57400cac30 0x7f57400acd60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 --2- 172.21.15.131:0/670894668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5740159920 0x7f574006ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 >> 172.21.15.131:0/670894668 conn(0x7f5740068040 msgr2=0x7f57400ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 shutdown_connections 2023-12-26T21:52:47.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.356+0000 7f5746fff700 1 -- 172.21.15.131:0/670894668 wait complete. 2023-12-26T21:52:47.358 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 34 2023-12-26T21:52:47.376 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:47.467 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:47.467+0000 7f86d929b1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:47.520 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:47.519+0000 7efe30cac1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:47.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.570+0000 7fdb1d9ff700 1 Processor -- start 2023-12-26T21:52:47.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.570+0000 7fdb1d9ff700 1 -- start start 2023-12-26T21:52:47.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.572+0000 7fdb1d9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806bfd0 0x7fdb180c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.572+0000 7fdb1d9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb180686c0 con 0x7fdb1813c510 2023-12-26T21:52:47.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.572+0000 7fdb1d9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb18069520 con 0x7fdb180cc100 2023-12-26T21:52:47.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.572+0000 7fdb1d9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18068910 con 0x7fdb1806bfd0 2023-12-26T21:52:47.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.572+0000 7fdb16ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdb1813c510 0x7fdb1806b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43614/0 (socket says 172.21.15.131:43614) 2023-12-26T21:52:47.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.572+0000 7fdb16ffd700 1 -- 172.21.15.131:0/2004194765 learned_addr learned my addr 172.21.15.131:0/2004194765 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:47.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.572+0000 7fdb167fc700 1 --2- 172.21.15.131:0/2004194765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806bfd0 0x7fdb180c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.572+0000 7fdb15ffb700 1 -- 172.21.15.131:0/2004194765 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72609459 0 0) 0x7fdb180686c0 con 0x7fdb1813c510 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb15ffb700 1 -- 172.21.15.131:0/2004194765 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb00003570 con 0x7fdb1813c510 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb15ffb700 1 -- 172.21.15.131:0/2004194765 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029721807 0 0) 0x7fdb18069520 con 0x7fdb180cc100 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb15ffb700 1 -- 172.21.15.131:0/2004194765 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb180686c0 con 0x7fdb180cc100 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb167fc700 1 -- 172.21.15.131:0/2004194765 >> v1:172.21.15.190:6789/0 conn(0x7fdb180cc100 legacy=0x7fdb180cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb167fc700 1 -- 172.21.15.131:0/2004194765 >> v1:172.21.15.131:6789/0 conn(0x7fdb1813c510 legacy=0x7fdb1806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb167fc700 1 -- 172.21.15.131:0/2004194765 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18068bf0 con 0x7fdb1806bfd0 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb167fc700 1 --2- 172.21.15.131:0/2004194765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806bfd0 0x7fdb180c8a20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009910 tx=0x7fdb0c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cacdef7d8117d34b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb15ffb700 1 -- 172.21.15.131:0/2004194765 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c011630 con 0x7fdb1806bfd0 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb15ffb700 1 -- 172.21.15.131:0/2004194765 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c011790 con 0x7fdb1806bfd0 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.573+0000 7fdb15ffb700 1 -- 172.21.15.131:0/2004194765 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c011b30 con 0x7fdb1806bfd0 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.574+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/2004194765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806bfd0 msgr2=0x7fdb180c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.574+0000 7fdb1d9ff700 1 --2- 172.21.15.131:0/2004194765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806bfd0 0x7fdb180c8a20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009910 tx=0x7fdb0c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.574+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/2004194765 shutdown_connections 2023-12-26T21:52:47.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.574+0000 7fdb1d9ff700 1 --2- 172.21.15.131:0/2004194765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806bfd0 0x7fdb180c8a20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.574+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/2004194765 >> 172.21.15.131:0/2004194765 conn(0x7fdb180ce3e0 msgr2=0x7fdb180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:47.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.574+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/2004194765 shutdown_connections 2023-12-26T21:52:47.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.574+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/2004194765 wait complete. 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.575+0000 7fdb1d9ff700 1 Processor -- start 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.576+0000 7fdb1d9ff700 1 -- start start 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.576+0000 7fdb1d9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb1806bfd0 0x7fdb18158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.576+0000 7fdb1d9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb180cc100 0x7fdb181591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.576+0000 7fdb1d9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1813c510 0x7fdb181394e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.576+0000 7fdb1d9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181596f0 con 0x7fdb1806bfd0 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.576+0000 7fdb1d9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1816dac0 con 0x7fdb180cc100 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.576+0000 7fdb1d9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1813d920 con 0x7fdb1813c510 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb1806bfd0 0x7fdb18158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb167fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb180cc100 0x7fdb181591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb1806bfd0 0x7fdb18158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54072/0 (socket says 172.21.15.131:54072) 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb16ffd700 1 -- 172.21.15.131:0/3469886836 learned_addr learned my addr 172.21.15.131:0/3469886836 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:47.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb177fe700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1813c510 0x7fdb181394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:47.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb167fc700 1 -- 172.21.15.131:0/3469886836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1813c510 msgr2=0x7fdb181394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb167fc700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1813c510 0x7fdb181394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb167fc700 1 -- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb1806bfd0 msgr2=0x7fdb18158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb167fc700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb1806bfd0 0x7fdb18158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb167fc700 1 -- 172.21.15.131:0/3469886836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c009420 con 0x7fdb180cc100 2023-12-26T21:52:47.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb16ffd700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb1806bfd0 0x7fdb18158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:47.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.577+0000 7fdb167fc700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb180cc100 0x7fdb181591b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb1806fcc0 tx=0x7fdb0c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:47.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.578+0000 7fdafeffd700 1 -- 172.21.15.131:0/3469886836 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c01f070 con 0x7fdb180cc100 2023-12-26T21:52:47.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.578+0000 7fdafeffd700 1 -- 172.21.15.131:0/3469886836 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c002f70 con 0x7fdb180cc100 2023-12-26T21:52:47.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.578+0000 7fdafeffd700 1 -- 172.21.15.131:0/3469886836 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c011b10 con 0x7fdb180cc100 2023-12-26T21:52:47.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.578+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb181598e0 con 0x7fdb180cc100 2023-12-26T21:52:47.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.578+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1816d470 con 0x7fdb180cc100 2023-12-26T21:52:47.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.579+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1816dc70 con 0x7fdb180cc100 2023-12-26T21:52:47.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.580+0000 7fdafeffd700 1 -- 172.21.15.131:0/3469886836 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fdb0c02c450 con 0x7fdb180cc100 2023-12-26T21:52:47.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.580+0000 7fdafeffd700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdb040670e0 0x7fdb04069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:47.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.580+0000 7fdafeffd700 1 -- 172.21.15.131:0/3469886836 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fdb0c085a30 con 0x7fdb180cc100 2023-12-26T21:52:47.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.580+0000 7fdb16ffd700 1 -- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdb040670e0 msgr2=0x7fdb04069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:47.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.580+0000 7fdb16ffd700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdb040670e0 0x7fdb04069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:47.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.585+0000 7fdafeffd700 1 -- 172.21.15.131:0/3469886836 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb1816dc70 con 0x7fdb180cc100 2023-12-26T21:52:47.640 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:47.640+0000 7f86d929b1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:52:47.712 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:47.711+0000 7efe30cac1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:52:47.724 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:47.724+0000 7f86d929b1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:52:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.781+0000 7fdb16ffd700 1 -- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdb040670e0 msgr2=0x7fdb04069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:47.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.781+0000 7fdb16ffd700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdb040670e0 0x7fdb04069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:47.797 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:47.797+0000 7efe30cac1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:52:47.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.804+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb1816dc70 con 0x7fdb180cc100 2023-12-26T21:52:47.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.805+0000 7fdafeffd700 1 -- 172.21.15.131:0/3469886836 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16367 (secure 0 0 0) 0x7fdb1816dc70 con 0x7fdb180cc100 2023-12-26T21:52:47.806 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:47.806 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:44.550240+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:52:26.449943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-26T21:52:27.710394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3995334343":"2023-12-27T21:52:44.550195+0000","172.21.15.131:6837/39288":"2023-12-27T21:52:44.550195+0000","172.21.15.131:6835/39288":"2023-12-27T21:52:44.550195+0000","172.21.15.131:0/3569898592":"2023-12-27T21:52:44.550195+0000","172.21.15.131:0/1131448406":"2023-12-27T21:52:44.550195+0000","172.21.15.131:0/3103975191":"2023-12-27T21:52:44.550195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdb040670e0 msgr2=0x7fdb04069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdb040670e0 0x7fdb04069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb180cc100 msgr2=0x7fdb181591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb180cc100 0x7fdb181591b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb1806fcc0 tx=0x7fdb0c017000 comp rx=0 tx=0).stop 2023-12-26T21:52:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 shutdown_connections 2023-12-26T21:52:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1813c510 0x7fdb181394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb1806bfd0 0x7fdb18158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdb040670e0 0x7fdb04069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 --2- 172.21.15.131:0/3469886836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb180cc100 0x7fdb181591b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 >> 172.21.15.131:0/3469886836 conn(0x7fdb180ce3e0 msgr2=0x7fdb181538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 shutdown_connections 2023-12-26T21:52:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:47.808+0000 7fdb1d9ff700 1 -- 172.21.15.131:0/3469886836 wait complete. 2023-12-26T21:52:47.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:52:26.449943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:52:27.710394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:52:47.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:52:47.873 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:47.872+0000 7f86d929b1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:47.962 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:47.962+0000 7efe30cac1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:47.963 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:47.963+0000 7f86d929b1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:52:48.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.035+0000 7f9530fff700 1 Processor -- start 2023-12-26T21:52:48.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.036+0000 7f9530fff700 1 -- start start 2023-12-26T21:52:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.037+0000 7f9530fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f952c15b260 0x7f952c0caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.037+0000 7f9530fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f952c0aecc0 0x7f952c15b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.037+0000 7f9530fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c067c30 con 0x7f952c0c8e70 2023-12-26T21:52:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.037+0000 7f9530fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952c067650 con 0x7f952c15b260 2023-12-26T21:52:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.037+0000 7f9530fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952c15bcd0 con 0x7f952c0aecc0 2023-12-26T21:52:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.037+0000 7f952a59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f952c15b260 0x7f952c0caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.037+0000 7f952ad9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f952c0c8e70 0x7f952c0b1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43630/0 (socket says 172.21.15.131:43630) 2023-12-26T21:52:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.037+0000 7f952ad9d700 1 -- 172.21.15.131:0/3782513406 learned_addr learned my addr 172.21.15.131:0/3782513406 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.037+0000 7f9529d9b700 1 --2- 172.21.15.131:0/3782513406 >> v2:172.21.15.190:3300/0 conn(0x7f952c0aecc0 0x7f952c15b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.038+0000 7f952a59c700 1 -- 172.21.15.131:0/3782513406 >> v2:172.21.15.190:3300/0 conn(0x7f952c0aecc0 msgr2=0x7f952c15b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.038+0000 7f952a59c700 1 --2- 172.21.15.131:0/3782513406 >> v2:172.21.15.190:3300/0 conn(0x7f952c0aecc0 0x7f952c15b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.038+0000 7f952a59c700 1 -- 172.21.15.131:0/3782513406 >> v1:172.21.15.131:6789/0 conn(0x7f952c0c8e70 legacy=0x7f952c0b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.038+0000 7f952a59c700 1 -- 172.21.15.131:0/3782513406 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0637b0 con 0x7f952c15b260 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.038+0000 7f952959a700 1 -- 172.21.15.131:0/3782513406 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331557269 0 0) 0x7f952c067c30 con 0x7f952c0c8e70 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.038+0000 7f952a59c700 1 --2- 172.21.15.131:0/3782513406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 0x7f952c0caae0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f952c0692f0 tx=0x7f951c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a34e20d62f6ff68 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.038+0000 7f952959a700 1 -- 172.21.15.131:0/3782513406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00b690 con 0x7f952c15b260 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.038+0000 7f952959a700 1 -- 172.21.15.131:0/3782513406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c00b7f0 con 0x7f952c15b260 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.038+0000 7f952959a700 1 -- 172.21.15.131:0/3782513406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00bb90 con 0x7f952c15b260 2023-12-26T21:52:48.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.039+0000 7f9530fff700 1 -- 172.21.15.131:0/3782513406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 msgr2=0x7f952c0caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.039+0000 7f9530fff700 1 --2- 172.21.15.131:0/3782513406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 0x7f952c0caae0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f952c0692f0 tx=0x7f951c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.039+0000 7f9530fff700 1 -- 172.21.15.131:0/3782513406 shutdown_connections 2023-12-26T21:52:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.039+0000 7f9530fff700 1 --2- 172.21.15.131:0/3782513406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 0x7f952c0caae0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.039+0000 7f9530fff700 1 --2- 172.21.15.131:0/3782513406 >> v2:172.21.15.190:3300/0 conn(0x7f952c0aecc0 0x7f952c15b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.039+0000 7f9530fff700 1 -- 172.21.15.131:0/3782513406 >> 172.21.15.131:0/3782513406 conn(0x7f952c06c290 msgr2=0x7f952c0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.039+0000 7f9530fff700 1 -- 172.21.15.131:0/3782513406 shutdown_connections 2023-12-26T21:52:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.039+0000 7f9530fff700 1 -- 172.21.15.131:0/3782513406 wait complete. 2023-12-26T21:52:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.041+0000 7f9530fff700 1 Processor -- start 2023-12-26T21:52:48.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.041+0000 7f9530fff700 1 -- start start 2023-12-26T21:52:48.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9530fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f952c0aecc0 0x7f952c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9530fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f952c0c8e70 0x7f952c16d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9530fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 0x7f952c0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9530fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0b09e0 con 0x7f952c15b260 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9530fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0b0140 con 0x7f952c0c8e70 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9530fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0b0420 con 0x7f952c0aecc0 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f952ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 0x7f952c0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f952a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f952c0aecc0 0x7f952c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9529d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f952c0c8e70 0x7f952c16d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f952ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 0x7f952c0cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54086/0 (socket says 172.21.15.131:54086) 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f952ad9d700 1 -- 172.21.15.131:0/3250394095 learned_addr learned my addr 172.21.15.131:0/3250394095 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9529d9b700 1 -- 172.21.15.131:0/3250394095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f952c0aecc0 msgr2=0x7f952c0afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9529d9b700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f952c0aecc0 0x7f952c0afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9529d9b700 1 -- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 msgr2=0x7f952c0cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9529d9b700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 0x7f952c0cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.042+0000 7f9529d9b700 1 -- 172.21.15.131:0/3250394095 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c009420 con 0x7f952c0c8e70 2023-12-26T21:52:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.043+0000 7f9529d9b700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f952c0c8e70 0x7f952c16d9a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f952000ea50 tx=0x7f95200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.043+0000 7f951a7fc700 1 -- 172.21.15.131:0/3250394095 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000ee90 con 0x7f952c0c8e70 2023-12-26T21:52:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.043+0000 7f951a7fc700 1 -- 172.21.15.131:0/3250394095 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952000c7f0 con 0x7f952c0c8e70 2023-12-26T21:52:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.043+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c0b0700 con 0x7f952c0c8e70 2023-12-26T21:52:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.043+0000 7f951a7fc700 1 -- 172.21.15.131:0/3250394095 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000cb90 con 0x7f952c0c8e70 2023-12-26T21:52:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.043+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c14d160 con 0x7f952c0c8e70 2023-12-26T21:52:48.045 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:52:48.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.044+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c0c7150 con 0x7f952c0c8e70 2023-12-26T21:52:48.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.045+0000 7f951a7fc700 1 -- 172.21.15.131:0/3250394095 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f9520013470 con 0x7f952c0c8e70 2023-12-26T21:52:48.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.046+0000 7f951a7fc700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f950c0670e0 0x7f950c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.046+0000 7f951a7fc700 1 -- 172.21.15.131:0/3250394095 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f9520084cc0 con 0x7f952c0c8e70 2023-12-26T21:52:48.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.046+0000 7f952a59c700 1 -- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f950c0670e0 msgr2=0x7f950c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:48.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.046+0000 7f952a59c700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f950c0670e0 0x7f950c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:48.051 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:52:48.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.050+0000 7f951a7fc700 1 -- 172.21.15.131:0/3250394095 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f952008b080 con 0x7f952c0c8e70 2023-12-26T21:52:48.058 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:48.058+0000 7efe30cac1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:52:48.146 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:48.146+0000 7f86d929b1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:52:48.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.247+0000 7f952a59c700 1 -- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f950c0670e0 msgr2=0x7f950c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:48.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.247+0000 7f952a59c700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f950c0670e0 0x7f950c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:48.249 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:48.249+0000 7efe30cac1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:52:48.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.291+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f952c0657e0 con 0x7f952c0c8e70 2023-12-26T21:52:48.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.291+0000 7f951a7fc700 1 -- 172.21.15.131:0/3250394095 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9520051980 con 0x7f952c0c8e70 2023-12-26T21:52:48.292 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:52:48.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f950c0670e0 msgr2=0x7f950c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:48.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f950c0670e0 0x7f950c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f952c0c8e70 msgr2=0x7f952c16d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f952c0c8e70 0x7f952c16d9a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f952000ea50 tx=0x7f95200091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 shutdown_connections 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952c15b260 0x7f952c0cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f952c0aecc0 0x7f952c0afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f950c0670e0 0x7f950c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 --2- 172.21.15.131:0/3250394095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f952c0c8e70 0x7f952c16d9a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 >> 172.21.15.131:0/3250394095 conn(0x7f952c06c290 msgr2=0x7f952c069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 shutdown_connections 2023-12-26T21:52:48.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.294+0000 7f9530fff700 1 -- 172.21.15.131:0/3250394095 wait complete. 2023-12-26T21:52:48.296 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:48.296+0000 7f86d929b1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:52:48.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:52:48.406 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:48.406+0000 7efe30cac1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:52:48.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.510+0000 7f437c3ff700 1 Processor -- start 2023-12-26T21:52:48.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.511+0000 7f437c3ff700 1 -- start start 2023-12-26T21:52:48.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.512+0000 7f437c3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 0x7f43741599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.512+0000 7f437c3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43740a98b0 con 0x7f43740ad560 2023-12-26T21:52:48.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.512+0000 7f437c3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437415a500 con 0x7f437415d240 2023-12-26T21:52:48.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.512+0000 7f437c3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43740add40 con 0x7f43741595e0 2023-12-26T21:52:48.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.512+0000 7f437a99c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f43740ad560 0x7f43740aa020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43634/0 (socket says 172.21.15.131:43634) 2023-12-26T21:52:48.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.512+0000 7f437a99c700 1 -- 172.21.15.131:0/3218847209 learned_addr learned my addr 172.21.15.131:0/3218847209 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:48.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.512+0000 7f437a19b700 1 --2- 172.21.15.131:0/3218847209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 0x7f43741599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f4379199700 1 -- 172.21.15.131:0/3218847209 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541887806 0 0) 0x7f43740a98b0 con 0x7f43740ad560 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f4379199700 1 -- 172.21.15.131:0/3218847209 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4360003570 con 0x7f43740ad560 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f4379199700 1 -- 172.21.15.131:0/3218847209 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878511584 0 0) 0x7f437415a500 con 0x7f437415d240 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f4379199700 1 -- 172.21.15.131:0/3218847209 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43740a98b0 con 0x7f437415d240 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f437a19b700 1 -- 172.21.15.131:0/3218847209 >> v1:172.21.15.190:6789/0 conn(0x7f437415d240 legacy=0x7f43740ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f437a19b700 1 -- 172.21.15.131:0/3218847209 >> v1:172.21.15.131:6789/0 conn(0x7f43740ad560 legacy=0x7f43740aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f437a19b700 1 -- 172.21.15.131:0/3218847209 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437415a7e0 con 0x7f43741595e0 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f437a19b700 1 --2- 172.21.15.131:0/3218847209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 0x7f43741599b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4370009910 tx=0x7f43700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d8e17ffa161258 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f4379199700 1 -- 172.21.15.131:0/3218847209 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370011630 con 0x7f43741595e0 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f4379199700 1 -- 172.21.15.131:0/3218847209 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370011790 con 0x7f43741595e0 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.513+0000 7f4379199700 1 -- 172.21.15.131:0/3218847209 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370011b30 con 0x7f43741595e0 2023-12-26T21:52:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.514+0000 7f437c3ff700 1 -- 172.21.15.131:0/3218847209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 msgr2=0x7f43741599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.514+0000 7f437c3ff700 1 --2- 172.21.15.131:0/3218847209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 0x7f43741599b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4370009910 tx=0x7f43700091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.514+0000 7f437c3ff700 1 -- 172.21.15.131:0/3218847209 shutdown_connections 2023-12-26T21:52:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.514+0000 7f437c3ff700 1 --2- 172.21.15.131:0/3218847209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 0x7f43741599b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.514+0000 7f437c3ff700 1 -- 172.21.15.131:0/3218847209 >> 172.21.15.131:0/3218847209 conn(0x7f4374068040 msgr2=0x7f4374069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.514+0000 7f437c3ff700 1 -- 172.21.15.131:0/3218847209 shutdown_connections 2023-12-26T21:52:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.514+0000 7f437c3ff700 1 -- 172.21.15.131:0/3218847209 wait complete. 2023-12-26T21:52:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.516+0000 7f437c3ff700 1 Processor -- start 2023-12-26T21:52:48.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.516+0000 7f437c3ff700 1 -- start start 2023-12-26T21:52:48.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437c3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43740ad560 0x7f43740c1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437c3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 0x7f437406e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437c3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f437415d240 0x7f4374148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437c3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43740c2b90 con 0x7f437415d240 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437c3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437406ebe0 con 0x7f43740ad560 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437c3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43740c1f80 con 0x7f43741595e0 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437a99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f437415d240 0x7f4374148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437a99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f437415d240 0x7f4374148bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54090/0 (socket says 172.21.15.131:54090) 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437a99c700 1 -- 172.21.15.131:0/427977857 learned_addr learned my addr 172.21.15.131:0/427977857 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437a19b700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43740ad560 0x7f43740c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437999a700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 0x7f437406e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437a99c700 1 -- 172.21.15.131:0/427977857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 msgr2=0x7f437406e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437a99c700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 0x7f437406e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437a99c700 1 -- 172.21.15.131:0/427977857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43740ad560 msgr2=0x7f43740c1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437a99c700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43740ad560 0x7f43740c1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.517+0000 7f437a99c700 1 -- 172.21.15.131:0/427977857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370009420 con 0x7f437415d240 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.518+0000 7f437a99c700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f437415d240 0x7f4374148bb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f436c001460 tx=0x7f436c001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.518+0000 7f436a7fc700 1 -- 172.21.15.131:0/427977857 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c00a8b0 con 0x7f437415d240 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.518+0000 7f436a7fc700 1 -- 172.21.15.131:0/427977857 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436c00aa10 con 0x7f437415d240 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.518+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43740c2260 con 0x7f437415d240 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.518+0000 7f436a7fc700 1 -- 172.21.15.131:0/427977857 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c00add0 con 0x7f437415d240 2023-12-26T21:52:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.518+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43740c2820 con 0x7f437415d240 2023-12-26T21:52:48.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.519+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43740657e0 con 0x7f437415d240 2023-12-26T21:52:48.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.521+0000 7f436a7fc700 1 -- 172.21.15.131:0/427977857 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f436c007300 con 0x7f437415d240 2023-12-26T21:52:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.522+0000 7f436a7fc700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f43580670e0 0x7f4358069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.522+0000 7f436a7fc700 1 -- 172.21.15.131:0/427977857 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f436c085990 con 0x7f437415d240 2023-12-26T21:52:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.522+0000 7f437a19b700 1 -- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f43580670e0 msgr2=0x7f4358069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.522+0000 7f437a19b700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f43580670e0 0x7f4358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:48.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.524+0000 7f436a7fc700 1 -- 172.21.15.131:0/427977857 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f436c052750 con 0x7f437415d240 2023-12-26T21:52:48.679 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:48.679+0000 7f86d929b1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:52:48.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.723+0000 7f437a19b700 1 -- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f43580670e0 msgr2=0x7f4358069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:48.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.723+0000 7f437a19b700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f43580670e0 0x7f4358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:48.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.729+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f437415aac0 con 0x7f437415d240 2023-12-26T21:52:48.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.729+0000 7f436a7fc700 1 -- 172.21.15.131:0/427977857 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f436c055d70 con 0x7f437415d240 2023-12-26T21:52:48.729 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:52:48.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.731+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f43580670e0 msgr2=0x7f4358069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:48.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.731+0000 7f437c3ff700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f43580670e0 0x7f4358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.731+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f437415d240 msgr2=0x7f4374148bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.731+0000 7f437c3ff700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f437415d240 0x7f4374148bb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f436c001460 tx=0x7f436c001aa0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.732+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 shutdown_connections 2023-12-26T21:52:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.732+0000 7f437c3ff700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f437415d240 0x7f4374148bb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.732+0000 7f437c3ff700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f43580670e0 0x7f4358069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.732+0000 7f437c3ff700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43740ad560 0x7f43740c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.732+0000 7f437c3ff700 1 --2- 172.21.15.131:0/427977857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43741595e0 0x7f437406e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.732+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 >> 172.21.15.131:0/427977857 conn(0x7f4374068040 msgr2=0x7f4374153f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.732+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 shutdown_connections 2023-12-26T21:52:48.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.732+0000 7f437c3ff700 1 -- 172.21.15.131:0/427977857 wait complete. 2023-12-26T21:52:48.751 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:52:48.811 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:48.810+0000 7efe30cac1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:52:48.942 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:48.942+0000 7f86d929b1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:52:48.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.946+0000 7f47879ff700 1 Processor -- start 2023-12-26T21:52:48.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.946+0000 7f47879ff700 1 -- start start 2023-12-26T21:52:48.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.947+0000 7f47879ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47800c8820 0x7f47800ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.947+0000 7f47879ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8bf0 0x7f47800b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.947+0000 7f47879ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4780069d10 0x7f47800b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.947+0000 7f47879ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4780157a80 con 0x7f47800c8820 2023-12-26T21:52:48.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.947+0000 7f47879ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47800cac50 con 0x7f4780069d10 2023-12-26T21:52:48.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.947+0000 7f47879ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800caf30 con 0x7f47800c8bf0 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f478579b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47800c8820 0x7f47800ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f478579b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47800c8820 0x7f47800ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54106/0 (socket says 172.21.15.131:54106) 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f478579b700 1 -- 172.21.15.131:0/1973732642 learned_addr learned my addr 172.21.15.131:0/1973732642 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f4784f9a700 1 --2- 172.21.15.131:0/1973732642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8bf0 0x7f47800b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f4785f9c700 1 --2- 172.21.15.131:0/1973732642 >> v2:172.21.15.190:3300/0 conn(0x7f4780069d10 0x7f47800b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f478579b700 1 -- 172.21.15.131:0/1973732642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8bf0 msgr2=0x7f47800b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f478579b700 1 --2- 172.21.15.131:0/1973732642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8bf0 0x7f47800b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f478579b700 1 -- 172.21.15.131:0/1973732642 >> v2:172.21.15.190:3300/0 conn(0x7f4780069d10 msgr2=0x7f47800b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f478579b700 1 --2- 172.21.15.131:0/1973732642 >> v2:172.21.15.190:3300/0 conn(0x7f4780069d10 0x7f47800b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f478579b700 1 -- 172.21.15.131:0/1973732642 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800cb210 con 0x7f47800c8820 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.948+0000 7f478579b700 1 --2- 172.21.15.131:0/1973732642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8820 0x7f47800ca300 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f478006b560 tx=0x7f477c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd442a469783ee8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f4777fff700 1 -- 172.21.15.131:0/1973732642 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c00b640 con 0x7f47800c8820 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f4777fff700 1 -- 172.21.15.131:0/1973732642 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477c00b7a0 con 0x7f47800c8820 2023-12-26T21:52:48.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f4777fff700 1 -- 172.21.15.131:0/1973732642 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c00bb60 con 0x7f47800c8820 2023-12-26T21:52:48.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f47879ff700 1 -- 172.21.15.131:0/1973732642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8820 msgr2=0x7f47800ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f47879ff700 1 --2- 172.21.15.131:0/1973732642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8820 0x7f47800ca300 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f478006b560 tx=0x7f477c009250 comp rx=0 tx=0).stop 2023-12-26T21:52:48.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f47879ff700 1 -- 172.21.15.131:0/1973732642 shutdown_connections 2023-12-26T21:52:48.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f47879ff700 1 --2- 172.21.15.131:0/1973732642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8820 0x7f47800ca300 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f47879ff700 1 --2- 172.21.15.131:0/1973732642 >> v2:172.21.15.190:3300/0 conn(0x7f4780069d10 0x7f47800b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f47879ff700 1 --2- 172.21.15.131:0/1973732642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8bf0 0x7f47800b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f47879ff700 1 -- 172.21.15.131:0/1973732642 >> 172.21.15.131:0/1973732642 conn(0x7f4780068040 msgr2=0x7f4780069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:48.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.949+0000 7f47879ff700 1 -- 172.21.15.131:0/1973732642 shutdown_connections 2023-12-26T21:52:48.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.950+0000 7f47879ff700 1 -- 172.21.15.131:0/1973732642 wait complete. 2023-12-26T21:52:48.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.952+0000 7f47879ff700 1 Processor -- start 2023-12-26T21:52:48.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.952+0000 7f47879ff700 1 -- start start 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f47879ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4780069d10 0x7f47800b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f47879ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8820 0x7f4780154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f47879ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8bf0 0x7f47800af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f47879ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800b0820 con 0x7f47800c8bf0 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f47879ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800afc10 con 0x7f4780069d10 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f47879ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800afef0 con 0x7f47800c8820 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f4785f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8bf0 0x7f47800af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f4785f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8bf0 0x7f47800af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54114/0 (socket says 172.21.15.131:54114) 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f478579b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4780069d10 0x7f47800b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f4785f9c700 1 -- 172.21.15.131:0/3475591056 learned_addr learned my addr 172.21.15.131:0/3475591056 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:48.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.953+0000 7f4784f9a700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8820 0x7f4780154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f4784f9a700 1 -- 172.21.15.131:0/3475591056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4780069d10 msgr2=0x7f47800b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f4784f9a700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4780069d10 0x7f47800b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f4784f9a700 1 -- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8bf0 msgr2=0x7f47800af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f4784f9a700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8bf0 0x7f47800af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f4784f9a700 1 -- 172.21.15.131:0/3475591056 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0094f0 con 0x7f47800c8820 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f4785f9c700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8bf0 0x7f47800af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f4784f9a700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8820 0x7f4780154970 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f477000e9d0 tx=0x7f47700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f47757fa700 1 -- 172.21.15.131:0/3475591056 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770013680 con 0x7f47800c8820 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f47757fa700 1 -- 172.21.15.131:0/3475591056 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770013800 con 0x7f47800c8820 2023-12-26T21:52:48.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.954+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47800b01d0 con 0x7f47800c8820 2023-12-26T21:52:48.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.955+0000 7f47757fa700 1 -- 172.21.15.131:0/3475591056 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770013bc0 con 0x7f47800c8820 2023-12-26T21:52:48.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.955+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4780154eb0 con 0x7f47800c8820 2023-12-26T21:52:48.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.956+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47800657e0 con 0x7f47800c8820 2023-12-26T21:52:48.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.956+0000 7f47757fa700 1 -- 172.21.15.131:0/3475591056 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f477000c490 con 0x7f47800c8820 2023-12-26T21:52:48.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.957+0000 7f47757fa700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f47640670e0 0x7f4764069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:48.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.957+0000 7f478579b700 1 -- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f47640670e0 msgr2=0x7f4764069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:48.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.957+0000 7f47757fa700 1 -- 172.21.15.131:0/3475591056 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f4770084b20 con 0x7f47800c8820 2023-12-26T21:52:48.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.957+0000 7f478579b700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f47640670e0 0x7f4764069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:48.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:48.961+0000 7f47757fa700 1 -- 172.21.15.131:0/3475591056 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47700517e0 con 0x7f47800c8820 2023-12-26T21:52:49.090 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:49.090+0000 7efe30cac1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:52:49.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.158+0000 7f478579b700 1 -- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f47640670e0 msgr2=0x7f4764069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:49.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.158+0000 7f478579b700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f47640670e0 0x7f4764069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:49.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.163+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47800b0960 con 0x7f47800c8820 2023-12-26T21:52:49.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.163+0000 7f47757fa700 1 -- 172.21.15.131:0/3475591056 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47700170a0 con 0x7f47800c8820 2023-12-26T21:52:49.164 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:52:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f47640670e0 msgr2=0x7f4764069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f47640670e0 0x7f4764069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8820 msgr2=0x7f4780154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8820 0x7f4780154970 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f477000e9d0 tx=0x7f47700091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 shutdown_connections 2023-12-26T21:52:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47800c8bf0 0x7f47800af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f47640670e0 0x7f4764069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4780069d10 0x7f47800b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 --2- 172.21.15.131:0/3475591056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800c8820 0x7f4780154970 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 >> 172.21.15.131:0/3475591056 conn(0x7f4780068040 msgr2=0x7f47800c80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 shutdown_connections 2023-12-26T21:52:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.166+0000 7f47879ff700 1 -- 172.21.15.131:0/3475591056 wait complete. 2023-12-26T21:52:49.185 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:49.222 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:49.222+0000 7f86d929b1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:52:49.311 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:49.311+0000 7f86d929b1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:52:49.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.381+0000 7fdce09ff700 1 Processor -- start 2023-12-26T21:52:49.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.381+0000 7fdce09ff700 1 -- start start 2023-12-26T21:52:49.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.382+0000 7fdce09ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdcdc15d240 0x7fdcdc0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.382+0000 7fdce09ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 0x7fdcdc0aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.382+0000 7fdce09ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc0a98b0 con 0x7fdcdc1595e0 2023-12-26T21:52:49.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.382+0000 7fdce09ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc0add40 con 0x7fdcdc15d240 2023-12-26T21:52:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.382+0000 7fdce09ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0ad7b0 con 0x7fdcdc0ad390 2023-12-26T21:52:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.382+0000 7fdcd9d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdcdc15d240 0x7fdcdc0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.383+0000 7fdcda59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdcdc1595e0 0x7fdcdc1599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43642/0 (socket says 172.21.15.131:43642) 2023-12-26T21:52:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.383+0000 7fdcda59c700 1 -- 172.21.15.131:0/3575239651 learned_addr learned my addr 172.21.15.131:0/3575239651 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.383+0000 7fdcdad9d700 1 --2- 172.21.15.131:0/3575239651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 0x7fdcdc0aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.383+0000 7fdcd9d9b700 1 -- 172.21.15.131:0/3575239651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 msgr2=0x7fdcdc0aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.383+0000 7fdcd9d9b700 1 --2- 172.21.15.131:0/3575239651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 0x7fdcdc0aa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.383+0000 7fdcd9d9b700 1 -- 172.21.15.131:0/3575239651 >> v1:172.21.15.131:6789/0 conn(0x7fdcdc1595e0 legacy=0x7fdcdc1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.383+0000 7fdcd9d9b700 1 -- 172.21.15.131:0/3575239651 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc15a500 con 0x7fdcdc15d240 2023-12-26T21:52:49.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.383+0000 7fdcd9d9b700 1 --2- 172.21.15.131:0/3575239651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc15d240 0x7fdcdc0ace50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00098e0 tx=0x7fdcd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dede451429e601 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:49.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.383+0000 7fdcd959a700 1 -- 172.21.15.131:0/3575239651 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0011630 con 0x7fdcdc15d240 2023-12-26T21:52:49.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdcd959a700 1 -- 172.21.15.131:0/3575239651 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0011790 con 0x7fdcdc15d240 2023-12-26T21:52:49.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdcd959a700 1 -- 172.21.15.131:0/3575239651 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0011b30 con 0x7fdcdc15d240 2023-12-26T21:52:49.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdce09ff700 1 -- 172.21.15.131:0/3575239651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc15d240 msgr2=0x7fdcdc0ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdce09ff700 1 --2- 172.21.15.131:0/3575239651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc15d240 0x7fdcdc0ace50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00098e0 tx=0x7fdcd00091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdce09ff700 1 -- 172.21.15.131:0/3575239651 shutdown_connections 2023-12-26T21:52:49.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdce09ff700 1 --2- 172.21.15.131:0/3575239651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 0x7fdcdc0aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdce09ff700 1 --2- 172.21.15.131:0/3575239651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc15d240 0x7fdcdc0ace50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdce09ff700 1 -- 172.21.15.131:0/3575239651 >> 172.21.15.131:0/3575239651 conn(0x7fdcdc068040 msgr2=0x7fdcdc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:49.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdce09ff700 1 -- 172.21.15.131:0/3575239651 shutdown_connections 2023-12-26T21:52:49.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.384+0000 7fdce09ff700 1 -- 172.21.15.131:0/3575239651 wait complete. 2023-12-26T21:52:49.386 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:49.384+0000 7efe30cac1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:52:49.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.386+0000 7fdce09ff700 1 Processor -- start 2023-12-26T21:52:49.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.386+0000 7fdce09ff700 1 -- start start 2023-12-26T21:52:49.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdce09ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 0x7fdcdc14b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdce09ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc1595e0 0x7fdcdc14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdce09ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcdc15d240 0x7fdcdc06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdce09ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc16a120 con 0x7fdcdc1595e0 2023-12-26T21:52:49.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdce09ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc153b50 con 0x7fdcdc15d240 2023-12-26T21:52:49.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdce09ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0b0ce0 con 0x7fdcdc0ad390 2023-12-26T21:52:49.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdcd9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc1595e0 0x7fdcdc14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdcd9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc1595e0 0x7fdcdc14f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54130/0 (socket says 172.21.15.131:54130) 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdcd9d9b700 1 -- 172.21.15.131:0/277188372 learned_addr learned my addr 172.21.15.131:0/277188372 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdcda59c700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 0x7fdcdc14b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.387+0000 7fdcdad9d700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcdc15d240 0x7fdcdc06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcd9d9b700 1 -- 172.21.15.131:0/277188372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 msgr2=0x7fdcdc14b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcd9d9b700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 0x7fdcdc14b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcd9d9b700 1 -- 172.21.15.131:0/277188372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcdc15d240 msgr2=0x7fdcdc06e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcd9d9b700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcdc15d240 0x7fdcdc06e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcd9d9b700 1 -- 172.21.15.131:0/277188372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0009420 con 0x7fdcdc1595e0 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcda59c700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 0x7fdcdc14b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcd9d9b700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc1595e0 0x7fdcdc14f850 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00030c0 tx=0x7fdcd0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcca7fc700 1 -- 172.21.15.131:0/277188372 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd001cd50 con 0x7fdcdc1595e0 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcca7fc700 1 -- 172.21.15.131:0/277188372 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd001a450 con 0x7fdcdc1595e0 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdcca7fc700 1 -- 172.21.15.131:0/277188372 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd001a810 con 0x7fdcdc1595e0 2023-12-26T21:52:49.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc0b0fc0 con 0x7fdcdc1595e0 2023-12-26T21:52:49.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.388+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc1551b0 con 0x7fdcdc1595e0 2023-12-26T21:52:49.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.389+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc155430 con 0x7fdcdc1595e0 2023-12-26T21:52:49.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.391+0000 7fdcca7fc700 1 -- 172.21.15.131:0/277188372 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fdcd001ac00 con 0x7fdcdc1595e0 2023-12-26T21:52:49.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.392+0000 7fdcca7fc700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdcbc0670e0 0x7fdcbc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.392+0000 7fdcca7fc700 1 -- 172.21.15.131:0/277188372 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fdcd0086860 con 0x7fdcdc1595e0 2023-12-26T21:52:49.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.392+0000 7fdcda59c700 1 -- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdcbc0670e0 msgr2=0x7fdcbc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:49.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.392+0000 7fdcda59c700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdcbc0670e0 0x7fdcbc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:49.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.397+0000 7fdcca7fc700 1 -- 172.21.15.131:0/277188372 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcd00535c0 con 0x7fdcdc1595e0 2023-12-26T21:52:49.479 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:49.478+0000 7efe30cac1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:52:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.593+0000 7fdcda59c700 1 -- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdcbc0670e0 msgr2=0x7fdcbc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.593+0000 7fdcda59c700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdcbc0670e0 0x7fdcbc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:49.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.637+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcdc155430 con 0x7fdcdc1595e0 2023-12-26T21:52:49.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.637+0000 7fdcca7fc700 1 -- 172.21.15.131:0/277188372 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fdcd0056be0 con 0x7fdcdc1595e0 2023-12-26T21:52:49.638 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:49.638 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/625115906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":625115906},{"type":"v1","addr":"172.21.15.190:6833","nonce":625115906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2773111229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2773111229},{"type":"v1","addr":"172.21.15.190:6835","nonce":2773111229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/899613179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":899613179},{"type":"v1","addr":"172.21.15.131:6834","nonce":899613179}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:52:28.726694+0000","modified":"2023-12-26T21:52:32.268974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6836/4173752384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":4173752384},{"type":"v1","addr":"172.21.15.131:6836","nonce":4173752384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-12-26T21:52:49.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdcbc0670e0 msgr2=0x7fdcbc069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdcbc0670e0 0x7fdcbc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc1595e0 msgr2=0x7fdcdc14f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc1595e0 0x7fdcdc14f850 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00030c0 tx=0x7fdcd0016000 comp rx=0 tx=0).stop 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 shutdown_connections 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fdcbc0670e0 0x7fdcbc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcdc15d240 0x7fdcdc06e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcdc0ad390 0x7fdcdc14b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 --2- 172.21.15.131:0/277188372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc1595e0 0x7fdcdc14f850 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 >> 172.21.15.131:0/277188372 conn(0x7fdcdc068040 msgr2=0x7fdcdc0a9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 shutdown_connections 2023-12-26T21:52:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.640+0000 7fdce09ff700 1 -- 172.21.15.131:0/277188372 wait complete. 2023-12-26T21:52:49.642 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 34 2023-12-26T21:52:49.659 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:49.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.850+0000 7fdeaa7ff700 1 Processor -- start 2023-12-26T21:52:49.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.850+0000 7fdeaa7ff700 1 -- start start 2023-12-26T21:52:49.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.851+0000 7fdeaa7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdea41595e0 0x7fdea41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdeaa7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdea40ad560 0x7fdea4159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdeaa7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea40a98b0 con 0x7fdea415d240 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdeaa7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea40add40 con 0x7fdea41595e0 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdeaa7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea415a500 con 0x7fdea40ad560 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdea3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdea41595e0 0x7fdea41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdea3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdea41595e0 0x7fdea41599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54142/0 (socket says 172.21.15.131:54142) 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdea3fff700 1 -- 172.21.15.131:0/1188659468 learned_addr learned my addr 172.21.15.131:0/1188659468 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdea8d9c700 1 --2- 172.21.15.131:0/1188659468 >> v2:172.21.15.190:3300/0 conn(0x7fdea40ad560 0x7fdea4159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdea3fff700 1 -- 172.21.15.131:0/1188659468 >> v2:172.21.15.190:3300/0 conn(0x7fdea40ad560 msgr2=0x7fdea4159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdea3fff700 1 --2- 172.21.15.131:0/1188659468 >> v2:172.21.15.190:3300/0 conn(0x7fdea40ad560 0x7fdea4159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdea3fff700 1 -- 172.21.15.131:0/1188659468 >> v1:172.21.15.190:6789/0 conn(0x7fdea415d240 legacy=0x7fdea40ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.852+0000 7fdea3fff700 1 -- 172.21.15.131:0/1188659468 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea415a7e0 con 0x7fdea41595e0 2023-12-26T21:52:49.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.853+0000 7fdea3fff700 1 --2- 172.21.15.131:0/1188659468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea41595e0 0x7fdea41599b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdea406b450 tx=0x7fde94009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af47f8787ef63d51 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:49.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.853+0000 7fdea2ffd700 1 -- 172.21.15.131:0/1188659468 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400e070 con 0x7fdea41595e0 2023-12-26T21:52:49.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.853+0000 7fdea2ffd700 1 -- 172.21.15.131:0/1188659468 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9400ba50 con 0x7fdea41595e0 2023-12-26T21:52:49.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.853+0000 7fdea2ffd700 1 -- 172.21.15.131:0/1188659468 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde940092e0 con 0x7fdea41595e0 2023-12-26T21:52:49.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.853+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/1188659468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea41595e0 msgr2=0x7fdea41599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.853+0000 7fdeaa7ff700 1 --2- 172.21.15.131:0/1188659468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea41595e0 0x7fdea41599b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdea406b450 tx=0x7fde94009840 comp rx=0 tx=0).stop 2023-12-26T21:52:49.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.854+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/1188659468 shutdown_connections 2023-12-26T21:52:49.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.854+0000 7fdeaa7ff700 1 --2- 172.21.15.131:0/1188659468 >> v2:172.21.15.190:3300/0 conn(0x7fdea40ad560 0x7fdea4159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.854+0000 7fdeaa7ff700 1 --2- 172.21.15.131:0/1188659468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea41595e0 0x7fdea41599b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.854+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/1188659468 >> 172.21.15.131:0/1188659468 conn(0x7fdea4068040 msgr2=0x7fdea4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:49.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.854+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/1188659468 shutdown_connections 2023-12-26T21:52:49.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.854+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/1188659468 wait complete. 2023-12-26T21:52:49.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.856+0000 7fdeaa7ff700 1 Processor -- start 2023-12-26T21:52:49.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.856+0000 7fdeaa7ff700 1 -- start start 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdeaa7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea40ad560 0x7fdea40c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdeaa7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdea41595e0 0x7fdea415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdeaa7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdea415d240 0x7fdea4152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdeaa7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40c17e0 con 0x7fdea40ad560 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdeaa7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40af6b0 con 0x7fdea41595e0 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdeaa7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40af990 con 0x7fdea415d240 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdea3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea40ad560 0x7fdea40c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdea3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea40ad560 0x7fdea40c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54144/0 (socket says 172.21.15.131:54144) 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdea3fff700 1 -- 172.21.15.131:0/4292111576 learned_addr learned my addr 172.21.15.131:0/4292111576 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdea37fe700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdea41595e0 0x7fdea415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.858+0000 7fdea8d9c700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdea415d240 0x7fdea4152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fdea3fff700 1 -- 172.21.15.131:0/4292111576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdea415d240 msgr2=0x7fdea4152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fdea3fff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdea415d240 0x7fdea4152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fdea3fff700 1 -- 172.21.15.131:0/4292111576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdea41595e0 msgr2=0x7fdea415bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fdea3fff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdea41595e0 0x7fdea415bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fdea3fff700 1 -- 172.21.15.131:0/4292111576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94009ae0 con 0x7fdea40ad560 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fdea8d9c700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdea415d240 0x7fdea4152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fdea3fff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea40ad560 0x7fdea40c0f90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fde94009880 tx=0x7fde94018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fde9bfff700 1 -- 172.21.15.131:0/4292111576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94008ac0 con 0x7fdea40ad560 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fde9bfff700 1 -- 172.21.15.131:0/4292111576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde94008c40 con 0x7fdea40ad560 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fde9bfff700 1 -- 172.21.15.131:0/4292111576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94026460 con 0x7fdea40ad560 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea40afc70 con 0x7fdea40ad560 2023-12-26T21:52:49.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.859+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea40c14d0 con 0x7fdea40ad560 2023-12-26T21:52:49.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.860+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea40657e0 con 0x7fdea40ad560 2023-12-26T21:52:49.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.861+0000 7fde9bfff700 1 -- 172.21.15.131:0/4292111576 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fde94003040 con 0x7fdea40ad560 2023-12-26T21:52:49.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.862+0000 7fde9bfff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fde840670e0 0x7fde84069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:49.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.862+0000 7fde9bfff700 1 -- 172.21.15.131:0/4292111576 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fde94086a30 con 0x7fdea40ad560 2023-12-26T21:52:49.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.862+0000 7fdea37fe700 1 -- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fde840670e0 msgr2=0x7fde84069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:49.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.862+0000 7fdea37fe700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fde840670e0 0x7fde84069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:49.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:49.866+0000 7fde9bfff700 1 -- 172.21.15.131:0/4292111576 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde94053680 con 0x7fdea40ad560 2023-12-26T21:52:50.047 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:50.047+0000 7f86d929b1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:52:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.063+0000 7fdea37fe700 1 -- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fde840670e0 msgr2=0x7fde84069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.063+0000 7fdea37fe700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fde840670e0 0x7fde84069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.069+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdea40b0190 con 0x7fdea40ad560 2023-12-26T21:52:50.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.070+0000 7fde9bfff700 1 -- 172.21.15.131:0/4292111576 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16367 (secure 0 0 0) 0x7fde94056ca0 con 0x7fdea40ad560 2023-12-26T21:52:50.071 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:50.071 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:44.550240+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:52:26.449943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-26T21:52:27.710394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3995334343":"2023-12-27T21:52:44.550195+0000","172.21.15.131:6837/39288":"2023-12-27T21:52:44.550195+0000","172.21.15.131:6835/39288":"2023-12-27T21:52:44.550195+0000","172.21.15.131:0/3569898592":"2023-12-27T21:52:44.550195+0000","172.21.15.131:0/1131448406":"2023-12-27T21:52:44.550195+0000","172.21.15.131:0/3103975191":"2023-12-27T21:52:44.550195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.072+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fde840670e0 msgr2=0x7fde84069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.072+0000 7fdeaa7ff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fde840670e0 0x7fde84069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.072+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea40ad560 msgr2=0x7fdea40c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.072+0000 7fdeaa7ff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea40ad560 0x7fdea40c0f90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fde94009880 tx=0x7fde94018000 comp rx=0 tx=0).stop 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.073+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 shutdown_connections 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.073+0000 7fdeaa7ff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdea415d240 0x7fdea4152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.073+0000 7fdeaa7ff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdea40ad560 0x7fdea40c0f90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.073+0000 7fdeaa7ff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7fde840670e0 0x7fde84069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.073+0000 7fdeaa7ff700 1 --2- 172.21.15.131:0/4292111576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdea41595e0 0x7fdea415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.073+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 >> 172.21.15.131:0/4292111576 conn(0x7fdea4068040 msgr2=0x7fdea40abb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.073+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 shutdown_connections 2023-12-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.073+0000 7fdeaa7ff700 1 -- 172.21.15.131:0/4292111576 wait complete. 2023-12-26T21:52:50.092 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T21:52:50.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:50.140 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:50.140+0000 7f86d929b1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:52:50.249 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:50.249+0000 7efe30cac1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:52:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.284+0000 7f4e401ff700 1 Processor -- start 2023-12-26T21:52:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.284+0000 7f4e401ff700 1 -- start start 2023-12-26T21:52:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e401ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e401ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e380ad390 0x7f4e380aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e401ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e380a98b0 con 0x7f4e381595e0 2023-12-26T21:52:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e401ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e380add40 con 0x7f4e380ad390 2023-12-26T21:52:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e401ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380ad7b0 con 0x7f4e3815d240 2023-12-26T21:52:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e3df9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4e381595e0 0x7f4e381599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43654/0 (socket says 172.21.15.131:43654) 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e3df9b700 1 -- 172.21.15.131:0/3177000512 learned_addr learned my addr 172.21.15.131:0/3177000512 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e3e79c700 1 --2- 172.21.15.131:0/3177000512 >> v2:172.21.15.131:3300/0 conn(0x7f4e380ad390 0x7f4e380aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e3d79a700 1 --2- 172.21.15.131:0/3177000512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e3cf99700 1 -- 172.21.15.131:0/3177000512 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772208361 0 0) 0x7f4e380a98b0 con 0x7f4e381595e0 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.286+0000 7f4e3cf99700 1 -- 172.21.15.131:0/3177000512 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e24003570 con 0x7f4e381595e0 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.287+0000 7f4e3e79c700 1 -- 172.21.15.131:0/3177000512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 msgr2=0x7f4e380ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.287+0000 7f4e3e79c700 1 --2- 172.21.15.131:0/3177000512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.287+0000 7f4e3e79c700 1 -- 172.21.15.131:0/3177000512 >> v1:172.21.15.131:6789/0 conn(0x7f4e381595e0 legacy=0x7f4e381599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.287+0000 7f4e3e79c700 1 -- 172.21.15.131:0/3177000512 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3815a500 con 0x7f4e380ad390 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.287+0000 7f4e3d79a700 1 --2- 172.21.15.131:0/3177000512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.287+0000 7f4e3e79c700 1 --2- 172.21.15.131:0/3177000512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e380ad390 0x7f4e380aa020 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4e30009e60 tx=0x7f4e30009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f70f1c102711e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:50.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.287+0000 7f4e3cf99700 1 -- 172.21.15.131:0/3177000512 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000e070 con 0x7f4e380ad390 2023-12-26T21:52:50.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.287+0000 7f4e3cf99700 1 -- 172.21.15.131:0/3177000512 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3000bab0 con 0x7f4e380ad390 2023-12-26T21:52:50.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.287+0000 7f4e3cf99700 1 -- 172.21.15.131:0/3177000512 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000be70 con 0x7f4e380ad390 2023-12-26T21:52:50.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.288+0000 7f4e401ff700 1 -- 172.21.15.131:0/3177000512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e380ad390 msgr2=0x7f4e380aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.288+0000 7f4e401ff700 1 --2- 172.21.15.131:0/3177000512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e380ad390 0x7f4e380aa020 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4e30009e60 tx=0x7f4e30009780 comp rx=0 tx=0).stop 2023-12-26T21:52:50.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.288+0000 7f4e401ff700 1 -- 172.21.15.131:0/3177000512 shutdown_connections 2023-12-26T21:52:50.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.288+0000 7f4e401ff700 1 --2- 172.21.15.131:0/3177000512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e380ad390 0x7f4e380aa020 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.288+0000 7f4e401ff700 1 --2- 172.21.15.131:0/3177000512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.288+0000 7f4e401ff700 1 -- 172.21.15.131:0/3177000512 >> 172.21.15.131:0/3177000512 conn(0x7f4e38068040 msgr2=0x7f4e38069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:50.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.288+0000 7f4e401ff700 1 -- 172.21.15.131:0/3177000512 shutdown_connections 2023-12-26T21:52:50.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.288+0000 7f4e401ff700 1 -- 172.21.15.131:0/3177000512 wait complete. 2023-12-26T21:52:50.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.289+0000 7f4e401ff700 1 Processor -- start 2023-12-26T21:52:50.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.290+0000 7f4e401ff700 1 -- start start 2023-12-26T21:52:50.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.290+0000 7f4e401ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e380ad390 0x7f4e381524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.290+0000 7f4e401ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e381595e0 0x7f4e38150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.290+0000 7f4e401ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.290+0000 7f4e401ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380affd0 con 0x7f4e381595e0 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.290+0000 7f4e401ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380b0590 con 0x7f4e380ad390 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.290+0000 7f4e401ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e381529f0 con 0x7f4e3815d240 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3d79a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e381595e0 0x7f4e38150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3e79c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3df9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e380ad390 0x7f4e381524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3d79a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e381595e0 0x7f4e38150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54158/0 (socket says 172.21.15.131:54158) 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3d79a700 1 -- 172.21.15.131:0/2612906840 learned_addr learned my addr 172.21.15.131:0/2612906840 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3d79a700 1 -- 172.21.15.131:0/2612906840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 msgr2=0x7f4e380abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3d79a700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3d79a700 1 -- 172.21.15.131:0/2612906840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e380ad390 msgr2=0x7f4e381524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3d79a700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e380ad390 0x7f4e381524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3d79a700 1 -- 172.21.15.131:0/2612906840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e28009420 con 0x7f4e381595e0 2023-12-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3e79c700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.291+0000 7f4e3d79a700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e381595e0 0x7f4e38150010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800ecb0 tx=0x7f4e280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.292+0000 7f4e2dffb700 1 -- 172.21.15.131:0/2612906840 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800c850 con 0x7f4e381595e0 2023-12-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.292+0000 7f4e2dffb700 1 -- 172.21.15.131:0/2612906840 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2800c9b0 con 0x7f4e381595e0 2023-12-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.292+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e30009a20 con 0x7f4e381595e0 2023-12-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.292+0000 7f4e2dffb700 1 -- 172.21.15.131:0/2612906840 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800cd50 con 0x7f4e381595e0 2023-12-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.292+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e380af9d0 con 0x7f4e381595e0 2023-12-26T21:52:50.295 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:50.293+0000 7f86d929b1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:52:50.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.293+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e380b0740 con 0x7f4e381595e0 2023-12-26T21:52:50.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.294+0000 7f4e2dffb700 1 -- 172.21.15.131:0/2612906840 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f4e28013680 con 0x7f4e381595e0 2023-12-26T21:52:50.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.295+0000 7f4e2dffb700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e1c0670e0 0x7f4e1c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.295+0000 7f4e2dffb700 1 -- 172.21.15.131:0/2612906840 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f4e28084d30 con 0x7f4e381595e0 2023-12-26T21:52:50.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.295+0000 7f4e3df9b700 1 -- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e1c0670e0 msgr2=0x7f4e1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:50.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.295+0000 7f4e3df9b700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e1c0670e0 0x7f4e1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.299+0000 7f4e2dffb700 1 -- 172.21.15.131:0/2612906840 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e380b0740 con 0x7f4e381595e0 2023-12-26T21:52:50.349 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:50.348+0000 7efe30cac1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:52:50.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.496+0000 7f4e3df9b700 1 -- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e1c0670e0 msgr2=0x7f4e1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:50.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.496+0000 7f4e3df9b700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e1c0670e0 0x7f4e1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:50.507 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:50.507+0000 7efe30cac1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:52:50.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.542+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e380b0700 con 0x7f4e381595e0 2023-12-26T21:52:50.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.543+0000 7f4e2dffb700 1 -- 172.21.15.131:0/2612906840 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f4e280519f0 con 0x7f4e381595e0 2023-12-26T21:52:50.543 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:50.543 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/625115906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":625115906},{"type":"v1","addr":"172.21.15.190:6833","nonce":625115906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2773111229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2773111229},{"type":"v1","addr":"172.21.15.190:6835","nonce":2773111229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/899613179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":899613179},{"type":"v1","addr":"172.21.15.131:6834","nonce":899613179}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:52:28.726694+0000","modified":"2023-12-26T21:52:32.268974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6836/4173752384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":4173752384},{"type":"v1","addr":"172.21.15.131:6836","nonce":4173752384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e1c0670e0 msgr2=0x7f4e1c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e1c0670e0 0x7f4e1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e381595e0 msgr2=0x7f4e38150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e381595e0 0x7f4e38150010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800ecb0 tx=0x7f4e280091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 shutdown_connections 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e3815d240 0x7f4e380abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e1c0670e0 0x7f4e1c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e380ad390 0x7f4e381524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 --2- 172.21.15.131:0/2612906840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e381595e0 0x7f4e38150010 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 >> 172.21.15.131:0/2612906840 conn(0x7f4e38068040 msgr2=0x7f4e38157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:50.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 shutdown_connections 2023-12-26T21:52:50.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.545+0000 7f4e401ff700 1 -- 172.21.15.131:0/2612906840 wait complete. 2023-12-26T21:52:50.547 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 34 2023-12-26T21:52:50.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:50.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.758+0000 7f67637ff700 1 Processor -- start 2023-12-26T21:52:50.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.758+0000 7f67637ff700 1 -- start start 2023-12-26T21:52:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.759+0000 7f67637ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f675c159820 0x7f675c0abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.759+0000 7f67637ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c0675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.759+0000 7f67637ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675c15a8c0 con 0x7f675c0aa480 2023-12-26T21:52:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.759+0000 7f67637ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675c15ad50 con 0x7f675c159820 2023-12-26T21:52:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.759+0000 7f67637ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c159f90 con 0x7f675c06e480 2023-12-26T21:52:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f6760d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f675c0aa480 0x7f675c06dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43662/0 (socket says 172.21.15.131:43662) 2023-12-26T21:52:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f6760d9a700 1 -- 172.21.15.131:0/2242927907 learned_addr learned my addr 172.21.15.131:0/2242927907 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f6761d9c700 1 --2- 172.21.15.131:0/2242927907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c0675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f676159b700 1 --2- 172.21.15.131:0/2242927907 >> v2:172.21.15.190:3300/0 conn(0x7f675c159820 0x7f675c0abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f6753fff700 1 -- 172.21.15.131:0/2242927907 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232838719 0 0) 0x7f675c15a8c0 con 0x7f675c0aa480 2023-12-26T21:52:50.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f6753fff700 1 -- 172.21.15.131:0/2242927907 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6748003570 con 0x7f675c0aa480 2023-12-26T21:52:50.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f6761d9c700 1 -- 172.21.15.131:0/2242927907 >> v2:172.21.15.190:3300/0 conn(0x7f675c159820 msgr2=0x7f675c0abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f6761d9c700 1 --2- 172.21.15.131:0/2242927907 >> v2:172.21.15.190:3300/0 conn(0x7f675c159820 0x7f675c0abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f6761d9c700 1 -- 172.21.15.131:0/2242927907 >> v1:172.21.15.131:6789/0 conn(0x7f675c0aa480 legacy=0x7f675c06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f6761d9c700 1 -- 172.21.15.131:0/2242927907 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c15a270 con 0x7f675c06e480 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.760+0000 7f676159b700 1 --2- 172.21.15.131:0/2242927907 >> v2:172.21.15.190:3300/0 conn(0x7f675c159820 0x7f675c0abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.761+0000 7f6761d9c700 1 --2- 172.21.15.131:0/2242927907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c0675b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6754009e60 tx=0x7f6754009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2df5fc6a2945e715 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.761+0000 7f6753fff700 1 -- 172.21.15.131:0/2242927907 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675400e070 con 0x7f675c06e480 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.761+0000 7f6753fff700 1 -- 172.21.15.131:0/2242927907 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675400bab0 con 0x7f675c06e480 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.761+0000 7f6753fff700 1 -- 172.21.15.131:0/2242927907 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675400be70 con 0x7f675c06e480 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.761+0000 7f67637ff700 1 -- 172.21.15.131:0/2242927907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 msgr2=0x7f675c0675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.761+0000 7f67637ff700 1 --2- 172.21.15.131:0/2242927907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c0675b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6754009e60 tx=0x7f6754009780 comp rx=0 tx=0).stop 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.762+0000 7f67637ff700 1 -- 172.21.15.131:0/2242927907 shutdown_connections 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.762+0000 7f67637ff700 1 --2- 172.21.15.131:0/2242927907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c0675b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.762+0000 7f67637ff700 1 --2- 172.21.15.131:0/2242927907 >> v2:172.21.15.190:3300/0 conn(0x7f675c159820 0x7f675c0abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.762+0000 7f67637ff700 1 -- 172.21.15.131:0/2242927907 >> 172.21.15.131:0/2242927907 conn(0x7f675c06c290 msgr2=0x7f675c0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:50.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.762+0000 7f67637ff700 1 -- 172.21.15.131:0/2242927907 shutdown_connections 2023-12-26T21:52:50.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.762+0000 7f67637ff700 1 -- 172.21.15.131:0/2242927907 wait complete. 2023-12-26T21:52:50.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.763+0000 7f67637ff700 1 Processor -- start 2023-12-26T21:52:50.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.763+0000 7f67637ff700 1 -- start start 2023-12-26T21:52:50.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f67637ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c14a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f67637ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0aa480 0x7f675c0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f67637ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f675c159820 0x7f675c0c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f67637ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c14aac0 con 0x7f675c159820 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f67637ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c0af9e0 con 0x7f675c0aa480 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f67637ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c0afcc0 con 0x7f675c06e480 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f676159b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c14a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f6761d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f675c159820 0x7f675c0c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f676159b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c14a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57876/0 (socket says 172.21.15.131:57876) 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f676159b700 1 -- 172.21.15.131:0/634226213 learned_addr learned my addr 172.21.15.131:0/634226213 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.764+0000 7f6760d9a700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0aa480 0x7f675c0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f676159b700 1 -- 172.21.15.131:0/634226213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0aa480 msgr2=0x7f675c0b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f676159b700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0aa480 0x7f675c0b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f676159b700 1 -- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f675c159820 msgr2=0x7f675c0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f676159b700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f675c159820 0x7f675c0c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f676159b700 1 -- 172.21.15.131:0/634226213 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758009420 con 0x7f675c06e480 2023-12-26T21:52:50.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f676159b700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c14a580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f675800ec80 tx=0x7f67580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:50.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f67517fa700 1 -- 172.21.15.131:0/634226213 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675800c850 con 0x7f675c06e480 2023-12-26T21:52:50.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f67517fa700 1 -- 172.21.15.131:0/634226213 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675800c9b0 con 0x7f675c06e480 2023-12-26T21:52:50.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6754009a20 con 0x7f675c06e480 2023-12-26T21:52:50.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.765+0000 7f67517fa700 1 -- 172.21.15.131:0/634226213 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675800cd30 con 0x7f675c06e480 2023-12-26T21:52:50.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.766+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c0b0280 con 0x7f675c06e480 2023-12-26T21:52:50.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.766+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675c0657e0 con 0x7f675c06e480 2023-12-26T21:52:50.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.767+0000 7f67517fa700 1 -- 172.21.15.131:0/634226213 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f6758013680 con 0x7f675c06e480 2023-12-26T21:52:50.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.768+0000 7f67517fa700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f67400670e0 0x7f6740069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:50.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.768+0000 7f67517fa700 1 -- 172.21.15.131:0/634226213 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f6758084dc0 con 0x7f675c06e480 2023-12-26T21:52:50.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.768+0000 7f6760d9a700 1 -- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f67400670e0 msgr2=0x7f6740069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:50.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.768+0000 7f6760d9a700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f67400670e0 0x7f6740069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:50.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.773+0000 7f67517fa700 1 -- 172.21.15.131:0/634226213 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6758055510 con 0x7f675c06e480 2023-12-26T21:52:50.913 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:50.913+0000 7f86d929b1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:52:50.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.969+0000 7f6760d9a700 1 -- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f67400670e0 msgr2=0x7f6740069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:50.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:50.969+0000 7f6760d9a700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f67400670e0 0x7f6740069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.016+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f675c15a550 con 0x7f675c06e480 2023-12-26T21:52:51.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.017+0000 7f67517fa700 1 -- 172.21.15.131:0/634226213 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f67580550a0 con 0x7f675c06e480 2023-12-26T21:52:51.018 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:51.018 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/625115906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":625115906},{"type":"v1","addr":"172.21.15.190:6833","nonce":625115906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2773111229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2773111229},{"type":"v1","addr":"172.21.15.190:6835","nonce":2773111229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/899613179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":899613179},{"type":"v1","addr":"172.21.15.131:6834","nonce":899613179}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:52:28.726694+0000","modified":"2023-12-26T21:52:32.268974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6219},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6219":{"gid":6219,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6836/4173752384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":4173752384},{"type":"v1","addr":"172.21.15.131:6836","nonce":4173752384}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-12-26T21:52:51.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.019+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f67400670e0 msgr2=0x7f6740069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:51.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.019+0000 7f67637ff700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f67400670e0 0x7f6740069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.019+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 msgr2=0x7f675c14a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.019+0000 7f67637ff700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c14a580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f675800ec80 tx=0x7f67580091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.020+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 shutdown_connections 2023-12-26T21:52:51.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.020+0000 7f67637ff700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f675c159820 0x7f675c0c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.020+0000 7f67637ff700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c06e480 0x7f675c14a580 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.020+0000 7f67637ff700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f67400670e0 0x7f6740069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.020+0000 7f67637ff700 1 --2- 172.21.15.131:0/634226213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0aa480 0x7f675c0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.020+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 >> 172.21.15.131:0/634226213 conn(0x7f675c06c290 msgr2=0x7f675c15aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:51.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.020+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 shutdown_connections 2023-12-26T21:52:51.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.020+0000 7f67637ff700 1 -- 172.21.15.131:0/634226213 wait complete. 2023-12-26T21:52:51.022 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 34 2023-12-26T21:52:51.038 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:52:51.098 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:51.098+0000 7f86d929b1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:52:51.168 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:51.167+0000 7efe30cac1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:52:51.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.232+0000 7f40a11ff700 1 Processor -- start 2023-12-26T21:52:51.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.232+0000 7f40a11ff700 1 -- start start 2023-12-26T21:52:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.233+0000 7f40a11ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c158030 0x7f409c158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.233+0000 7f40a11ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f409c06b8c0 0x7f409c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.233+0000 7f40a11ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c15ba70 con 0x7f409c06bd60 2023-12-26T21:52:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.233+0000 7f40a11ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c15bf00 con 0x7f409c06b8c0 2023-12-26T21:52:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.233+0000 7f40a11ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c06c2a0 con 0x7f409c158030 2023-12-26T21:52:51.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.233+0000 7f409a59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f409c06b8c0 0x7f409c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:51.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.234+0000 7f409a59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f409c06b8c0 0x7f409c0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54174/0 (socket says 172.21.15.131:54174) 2023-12-26T21:52:51.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.234+0000 7f409a59c700 1 -- 172.21.15.131:0/1125480478 learned_addr learned my addr 172.21.15.131:0/1125480478 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:51.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.234+0000 7f409ad9d700 1 --2- 172.21.15.131:0/1125480478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c158030 0x7f409c158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:51.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.234+0000 7f409a59c700 1 -- 172.21.15.131:0/1125480478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c158030 msgr2=0x7f409c158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:51.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.234+0000 7f409a59c700 1 --2- 172.21.15.131:0/1125480478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c158030 0x7f409c158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.234+0000 7f409a59c700 1 -- 172.21.15.131:0/1125480478 >> v1:172.21.15.190:6789/0 conn(0x7f409c06bd60 legacy=0x7f409c06cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.234+0000 7f409a59c700 1 -- 172.21.15.131:0/1125480478 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c15b140 con 0x7f409c06b8c0 2023-12-26T21:52:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.234+0000 7f409a59c700 1 --2- 172.21.15.131:0/1125480478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c06b8c0 0x7f409c0c8a20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f40900099b0 tx=0x7f4090009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf564df078e99f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f4099d9b700 1 -- 172.21.15.131:0/1125480478 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090009ea0 con 0x7f409c06b8c0 2023-12-26T21:52:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f4099d9b700 1 -- 172.21.15.131:0/1125480478 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409000b860 con 0x7f409c06b8c0 2023-12-26T21:52:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f4099d9b700 1 -- 172.21.15.131:0/1125480478 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409000bc00 con 0x7f409c06b8c0 2023-12-26T21:52:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f40a11ff700 1 -- 172.21.15.131:0/1125480478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c06b8c0 msgr2=0x7f409c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f40a11ff700 1 --2- 172.21.15.131:0/1125480478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c06b8c0 0x7f409c0c8a20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f40900099b0 tx=0x7f4090009250 comp rx=0 tx=0).stop 2023-12-26T21:52:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f40a11ff700 1 -- 172.21.15.131:0/1125480478 shutdown_connections 2023-12-26T21:52:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f40a11ff700 1 --2- 172.21.15.131:0/1125480478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c158030 0x7f409c158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f40a11ff700 1 --2- 172.21.15.131:0/1125480478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c06b8c0 0x7f409c0c8a20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f40a11ff700 1 -- 172.21.15.131:0/1125480478 >> 172.21.15.131:0/1125480478 conn(0x7f409c0ce3e0 msgr2=0x7f409c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.235+0000 7f40a11ff700 1 -- 172.21.15.131:0/1125480478 shutdown_connections 2023-12-26T21:52:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.236+0000 7f40a11ff700 1 -- 172.21.15.131:0/1125480478 wait complete. 2023-12-26T21:52:51.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.237+0000 7f40a11ff700 1 Processor -- start 2023-12-26T21:52:51.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.237+0000 7f40a11ff700 1 -- start start 2023-12-26T21:52:51.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f40a11ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c06b8c0 0x7f409c13c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f40a11ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c06bd60 0x7f409c13b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f40a11ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c158030 0x7f409c0cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f40a11ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c13c6d0 con 0x7f409c158030 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f40a11ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c13d520 con 0x7f409c06b8c0 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f40a11ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c13bec0 con 0x7f409c06bd60 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f409b59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c158030 0x7f409c0cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f409ad9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c06b8c0 0x7f409c13c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f409ad9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c06b8c0 0x7f409c13c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:58104/0 (socket says 172.21.15.131:58104) 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f409b59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c158030 0x7f409c0cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54188/0 (socket says 172.21.15.131:54188) 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f409ad9d700 1 -- 172.21.15.131:0/1486486811 learned_addr learned my addr 172.21.15.131:0/1486486811 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.238+0000 7f409a59c700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c06bd60 0x7f409c13b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.239+0000 7f409a59c700 1 -- 172.21.15.131:0/1486486811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c06b8c0 msgr2=0x7f409c13c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.239+0000 7f409a59c700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c06b8c0 0x7f409c13c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.239+0000 7f409a59c700 1 -- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c158030 msgr2=0x7f409c0cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.239+0000 7f409a59c700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c158030 0x7f409c0cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.239+0000 7f409a59c700 1 -- 172.21.15.131:0/1486486811 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900094f0 con 0x7f409c06bd60 2023-12-26T21:52:51.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.239+0000 7f409a59c700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c06bd60 0x7f409c13b980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f409c0cae00 tx=0x7f4090017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:51.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.239+0000 7f4082ffd700 1 -- 172.21.15.131:0/1486486811 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090000d10 con 0x7f409c06bd60 2023-12-26T21:52:51.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.239+0000 7f4082ffd700 1 -- 172.21.15.131:0/1486486811 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409000bbe0 con 0x7f409c06bd60 2023-12-26T21:52:51.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.239+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c13c910 con 0x7f409c06bd60 2023-12-26T21:52:51.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.240+0000 7f4082ffd700 1 -- 172.21.15.131:0/1486486811 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090011670 con 0x7f409c06bd60 2023-12-26T21:52:51.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.240+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c13ced0 con 0x7f409c06bd60 2023-12-26T21:52:51.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.241+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c0657e0 con 0x7f409c06bd60 2023-12-26T21:52:51.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.241+0000 7f4082ffd700 1 -- 172.21.15.131:0/1486486811 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f4090011aa0 con 0x7f409c06bd60 2023-12-26T21:52:51.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.242+0000 7f4082ffd700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f40880670e0 0x7f4088069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.242+0000 7f4082ffd700 1 -- 172.21.15.131:0/1486486811 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f40900862c0 con 0x7f409c06bd60 2023-12-26T21:52:51.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.242+0000 7f409ad9d700 1 -- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f40880670e0 msgr2=0x7f4088069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:51.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.242+0000 7f409ad9d700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f40880670e0 0x7f4088069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.246+0000 7f4082ffd700 1 -- 172.21.15.131:0/1486486811 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4090052f80 con 0x7f409c06bd60 2023-12-26T21:52:51.289 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:51.287+0000 7f86d929b1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:52:51.366 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:51.365+0000 7efe30cac1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:52:51.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.443+0000 7f409ad9d700 1 -- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f40880670e0 msgr2=0x7f4088069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:51.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.443+0000 7f409ad9d700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f40880670e0 0x7f4088069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:51.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.449+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f409c13d660 con 0x7f409c06bd60 2023-12-26T21:52:51.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.451+0000 7f4082ffd700 1 -- 172.21.15.131:0/1486486811 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f40900565a0 con 0x7f409c06bd60 2023-12-26T21:52:51.451 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:51.451 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:52:44.550240+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:52:26.449943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-26T21:52:27.710394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3995334343":"2023-12-27T21:52:44.550195+0000","172.21.15.131:6837/39288":"2023-12-27T21:52:44.550195+0000","172.21.15.131:6835/39288":"2023-12-27T21:52:44.550195+0000","172.21.15.131:0/3569898592":"2023-12-27T21:52:44.550195+0000","172.21.15.131:0/1131448406":"2023-12-27T21:52:44.550195+0000","172.21.15.131:0/3103975191":"2023-12-27T21:52:44.550195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:52:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f40880670e0 msgr2=0x7f4088069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f40880670e0 0x7f4088069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c06bd60 msgr2=0x7f409c13b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c06bd60 0x7f409c13b980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f409c0cae00 tx=0x7f4090017000 comp rx=0 tx=0).stop 2023-12-26T21:52:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 shutdown_connections 2023-12-26T21:52:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f409c158030 0x7f409c0cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f40880670e0 0x7f4088069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409c06b8c0 0x7f409c13c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 --2- 172.21.15.131:0/1486486811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409c06bd60 0x7f409c13b980 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 >> 172.21.15.131:0/1486486811 conn(0x7f409c0ce3e0 msgr2=0x7f409c14dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:51.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 shutdown_connections 2023-12-26T21:52:51.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.453+0000 7f40a11ff700 1 -- 172.21.15.131:0/1486486811 wait complete. 2023-12-26T21:52:51.473 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T21:52:51.572 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:51.571+0000 7efe30cac1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:52:51.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.667+0000 7f3bb11ff700 1 Processor -- start 2023-12-26T21:52:51.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.667+0000 7f3bb11ff700 1 -- start start 2023-12-26T21:52:51.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.669+0000 7f3bb11ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3bac153bd0 0x7f3bac0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.669+0000 7f3bb11ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bac15d250 con 0x7f3bac0b1c10 2023-12-26T21:52:51.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.669+0000 7f3bb11ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bac15d4a0 con 0x7f3bac0b3b70 2023-12-26T21:52:51.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.669+0000 7f3bb11ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bac15e260 con 0x7f3bac153bd0 2023-12-26T21:52:51.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.669+0000 7f3bab59e700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3bac0b1c10 0x7f3bac068920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43674/0 (socket says 172.21.15.131:43674) 2023-12-26T21:52:51.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.669+0000 7f3bab59e700 1 -- 172.21.15.131:0/3526822975 learned_addr learned my addr 172.21.15.131:0/3526822975 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:51.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.669+0000 7f3baad9d700 1 --2- 172.21.15.131:0/3526822975 >> v2:172.21.15.190:3300/0 conn(0x7f3bac153bd0 0x7f3bac0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:51.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.669+0000 7f3ba9d9b700 1 -- 172.21.15.131:0/3526822975 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173685648 0 0) 0x7f3bac15d250 con 0x7f3bac0b1c10 2023-12-26T21:52:51.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.669+0000 7f3ba9d9b700 1 -- 172.21.15.131:0/3526822975 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b94003570 con 0x7f3bac0b1c10 2023-12-26T21:52:51.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.670+0000 7f3baad9d700 1 -- 172.21.15.131:0/3526822975 >> v1:172.21.15.190:6789/0 conn(0x7f3bac0b3b70 legacy=0x7f3bac0b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.670+0000 7f3baad9d700 1 -- 172.21.15.131:0/3526822975 >> v1:172.21.15.131:6789/0 conn(0x7f3bac0b1c10 legacy=0x7f3bac068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.670+0000 7f3baad9d700 1 -- 172.21.15.131:0/3526822975 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac15d780 con 0x7f3bac153bd0 2023-12-26T21:52:51.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.670+0000 7f3baad9d700 1 --2- 172.21.15.131:0/3526822975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bac153bd0 0x7f3bac0b3630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0098e0 tx=0x7f3b9c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce1d27aeffbf697f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:51.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.670+0000 7f3ba9d9b700 1 -- 172.21.15.131:0/3526822975 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00b640 con 0x7f3bac153bd0 2023-12-26T21:52:51.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.670+0000 7f3ba9d9b700 1 -- 172.21.15.131:0/3526822975 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c00b7a0 con 0x7f3bac153bd0 2023-12-26T21:52:51.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.670+0000 7f3ba9d9b700 1 -- 172.21.15.131:0/3526822975 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00bb40 con 0x7f3bac153bd0 2023-12-26T21:52:51.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.671+0000 7f3bb11ff700 1 -- 172.21.15.131:0/3526822975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bac153bd0 msgr2=0x7f3bac0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.671+0000 7f3bb11ff700 1 --2- 172.21.15.131:0/3526822975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bac153bd0 0x7f3bac0b3630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0098e0 tx=0x7f3b9c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.671+0000 7f3bb11ff700 1 -- 172.21.15.131:0/3526822975 shutdown_connections 2023-12-26T21:52:51.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.671+0000 7f3bb11ff700 1 --2- 172.21.15.131:0/3526822975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bac153bd0 0x7f3bac0b3630 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.671+0000 7f3bb11ff700 1 -- 172.21.15.131:0/3526822975 >> 172.21.15.131:0/3526822975 conn(0x7f3bac063d00 msgr2=0x7f3bac06b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:51.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.671+0000 7f3bb11ff700 1 -- 172.21.15.131:0/3526822975 shutdown_connections 2023-12-26T21:52:51.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.671+0000 7f3bb11ff700 1 -- 172.21.15.131:0/3526822975 wait complete. 2023-12-26T21:52:51.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.672+0000 7f3bb11ff700 1 Processor -- start 2023-12-26T21:52:51.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.672+0000 7f3bb11ff700 1 -- start start 2023-12-26T21:52:51.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.673+0000 7f3bb11ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bac0b1c10 0x7f3bac149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.673+0000 7f3bb11ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bac0b3b70 0x7f3bac0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.673+0000 7f3bb11ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bac153bd0 0x7f3bac154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.673+0000 7f3bb11ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac14a410 con 0x7f3bac0b1c10 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.673+0000 7f3bb11ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0cb880 con 0x7f3bac153bd0 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.673+0000 7f3bb11ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0cac70 con 0x7f3bac0b3b70 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bac0b1c10 0x7f3bac149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bac0b1c10 0x7f3bac149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54204/0 (socket says 172.21.15.131:54204) 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baad9d700 1 -- 172.21.15.131:0/129887034 learned_addr learned my addr 172.21.15.131:0/129887034 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3bab59e700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bac153bd0 0x7f3bac154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baa59c700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bac0b3b70 0x7f3bac0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baad9d700 1 -- 172.21.15.131:0/129887034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bac0b3b70 msgr2=0x7f3bac0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baad9d700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bac0b3b70 0x7f3bac0cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baad9d700 1 -- 172.21.15.131:0/129887034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bac153bd0 msgr2=0x7f3bac154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baad9d700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bac153bd0 0x7f3bac154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baad9d700 1 -- 172.21.15.131:0/129887034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009420 con 0x7f3bac0b1c10 2023-12-26T21:52:51.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.674+0000 7f3baad9d700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bac0b1c10 0x7f3bac149a90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009f80 tx=0x7f3b9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:51.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.675+0000 7f3b92ffd700 1 -- 172.21.15.131:0/129887034 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c000d10 con 0x7f3bac0b1c10 2023-12-26T21:52:51.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.675+0000 7f3b92ffd700 1 -- 172.21.15.131:0/129887034 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c00bb20 con 0x7f3bac0b1c10 2023-12-26T21:52:51.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.675+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bac0caf50 con 0x7f3bac0b1c10 2023-12-26T21:52:51.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.675+0000 7f3b92ffd700 1 -- 172.21.15.131:0/129887034 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c011650 con 0x7f3bac0b1c10 2023-12-26T21:52:51.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.675+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bac0cb510 con 0x7f3bac0b1c10 2023-12-26T21:52:51.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.676+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bac0657e0 con 0x7f3bac0b1c10 2023-12-26T21:52:51.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.677+0000 7f3b92ffd700 1 -- 172.21.15.131:0/129887034 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f3b9c0119c0 con 0x7f3bac0b1c10 2023-12-26T21:52:51.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.678+0000 7f3b92ffd700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b980670e0 0x7f3b98069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:51.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.678+0000 7f3b92ffd700 1 -- 172.21.15.131:0/129887034 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f3b9c085e60 con 0x7f3bac0b1c10 2023-12-26T21:52:51.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.678+0000 7f3baa59c700 1 -- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b980670e0 msgr2=0x7f3b98069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:51.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.678+0000 7f3baa59c700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b980670e0 0x7f3b98069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:51.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.682+0000 7f3b92ffd700 1 -- 172.21.15.131:0/129887034 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b9c052ba0 con 0x7f3bac0b1c10 2023-12-26T21:52:51.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.879+0000 7f3baa59c700 1 -- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b980670e0 msgr2=0x7f3b98069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6835/39288 2023-12-26T21:52:51.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.879+0000 7f3baa59c700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b980670e0 0x7f3b98069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:51.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.929+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3bac0b0c60 con 0x7f3bac0b1c10 2023-12-26T21:52:51.930 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T21:52:51.930+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T21:52:51.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.934+0000 7f3b92ffd700 1 -- 172.21.15.131:0/129887034 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3b9c0561c0 con 0x7f3bac0b1c10 2023-12-26T21:52:51.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.936+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b980670e0 msgr2=0x7f3b98069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:51.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.936+0000 7f3bb11ff700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b980670e0 0x7f3b98069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.936+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bac0b1c10 msgr2=0x7f3bac149a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.936+0000 7f3bb11ff700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bac0b1c10 0x7f3bac149a90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009f80 tx=0x7f3b9c017000 comp rx=0 tx=0).stop 2023-12-26T21:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.937+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 shutdown_connections 2023-12-26T21:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.937+0000 7f3bb11ff700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b980670e0 0x7f3b98069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.937+0000 7f3bb11ff700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3bac153bd0 0x7f3bac154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.937+0000 7f3bb11ff700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bac0b1c10 0x7f3bac149a90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.937+0000 7f3bb11ff700 1 --2- 172.21.15.131:0/129887034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bac0b3b70 0x7f3bac0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.937+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 >> 172.21.15.131:0/129887034 conn(0x7f3bac063d00 msgr2=0x7f3bac15c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.937+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 shutdown_connections 2023-12-26T21:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:51.937+0000 7f3bb11ff700 1 -- 172.21.15.131:0/129887034 wait complete. 2023-12-26T21:52:51.939 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T21:52:51.944 INFO:tasks.ceph.mds.c.smithi131.stderr: -2121> 2023-12-26T21:52:32.838+0000 7f6b2e9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:52:51.958 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T21:52:51.990 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:52:52.125 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:52.125+0000 7f86d929b1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:52:52.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.160+0000 7f2d855ff700 1 Processor -- start 2023-12-26T21:52:52.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.160+0000 7f2d855ff700 1 -- start start 2023-12-26T21:52:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.161+0000 7f2d855ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2d80158030 0x7f2d80158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.161+0000 7f2d855ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2d800c9130 0x7f2d800cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.161+0000 7f2d855ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8015ba70 con 0x7f2d8006b8c0 2023-12-26T21:52:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.161+0000 7f2d855ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8015bf00 con 0x7f2d80158030 2023-12-26T21:52:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.161+0000 7f2d855ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8006c2a0 con 0x7f2d800c9130 2023-12-26T21:52:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.162+0000 7f2d7effd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2d80158030 0x7f2d80158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:52.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.162+0000 7f2d7effd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2d80158030 0x7f2d80158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54246/0 (socket says 172.21.15.131:54246) 2023-12-26T21:52:52.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.162+0000 7f2d7effd700 1 -- 172.21.15.131:0/3883466515 learned_addr learned my addr 172.21.15.131:0/3883466515 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:52.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.162+0000 7f2d7f7fe700 1 --2- 172.21.15.131:0/3883466515 >> v2:172.21.15.190:3300/0 conn(0x7f2d800c9130 0x7f2d800cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:52.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.162+0000 7f2d7effd700 1 -- 172.21.15.131:0/3883466515 >> v2:172.21.15.190:3300/0 conn(0x7f2d800c9130 msgr2=0x7f2d800cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:52.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.162+0000 7f2d7effd700 1 --2- 172.21.15.131:0/3883466515 >> v2:172.21.15.190:3300/0 conn(0x7f2d800c9130 0x7f2d800cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.162+0000 7f2d7effd700 1 -- 172.21.15.131:0/3883466515 >> v1:172.21.15.190:6789/0 conn(0x7f2d8006b8c0 legacy=0x7f2d800c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:52.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.162+0000 7f2d7effd700 1 -- 172.21.15.131:0/3883466515 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8015b140 con 0x7f2d80158030 2023-12-26T21:52:52.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.162+0000 7f2d7effd700 1 --2- 172.21.15.131:0/3883466515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d80158030 0x7f2d80158400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2d80152080 tx=0x7f2d70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf0694a996342c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:52.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.163+0000 7f2d7dffb700 1 -- 172.21.15.131:0/3883466515 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70009e60 con 0x7f2d80158030 2023-12-26T21:52:52.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.163+0000 7f2d7dffb700 1 -- 172.21.15.131:0/3883466515 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000b860 con 0x7f2d80158030 2023-12-26T21:52:52.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.163+0000 7f2d7dffb700 1 -- 172.21.15.131:0/3883466515 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000bc00 con 0x7f2d80158030 2023-12-26T21:52:52.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.163+0000 7f2d855ff700 1 -- 172.21.15.131:0/3883466515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d80158030 msgr2=0x7f2d80158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:52.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.163+0000 7f2d855ff700 1 --2- 172.21.15.131:0/3883466515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d80158030 0x7f2d80158400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2d80152080 tx=0x7f2d70009250 comp rx=0 tx=0).stop 2023-12-26T21:52:52.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.163+0000 7f2d855ff700 1 -- 172.21.15.131:0/3883466515 shutdown_connections 2023-12-26T21:52:52.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.163+0000 7f2d855ff700 1 --2- 172.21.15.131:0/3883466515 >> v2:172.21.15.190:3300/0 conn(0x7f2d800c9130 0x7f2d800cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.163+0000 7f2d855ff700 1 --2- 172.21.15.131:0/3883466515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d80158030 0x7f2d80158400 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.163+0000 7f2d855ff700 1 -- 172.21.15.131:0/3883466515 >> 172.21.15.131:0/3883466515 conn(0x7f2d800ce3e0 msgr2=0x7f2d800ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:52.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.164+0000 7f2d855ff700 1 -- 172.21.15.131:0/3883466515 shutdown_connections 2023-12-26T21:52:52.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.164+0000 7f2d855ff700 1 -- 172.21.15.131:0/3883466515 wait complete. 2023-12-26T21:52:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.165+0000 7f2d855ff700 1 Processor -- start 2023-12-26T21:52:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.165+0000 7f2d855ff700 1 -- start start 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d855ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8006b8c0 0x7f2d80149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d855ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d800c9130 0x7f2d800b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d855ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d80158030 0x7f2d80159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d855ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80149590 con 0x7f2d800c9130 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d855ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800c6630 con 0x7f2d8006b8c0 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d855ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800b1e50 con 0x7f2d80158030 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d7e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d800c9130 0x7f2d800b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d7effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8006b8c0 0x7f2d80149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d7e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d800c9130 0x7f2d800b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54258/0 (socket says 172.21.15.131:54258) 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d7e7fc700 1 -- 172.21.15.131:0/2647705543 learned_addr learned my addr 172.21.15.131:0/2647705543 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.166+0000 7f2d7f7fe700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d80158030 0x7f2d80159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d7effd700 1 -- 172.21.15.131:0/2647705543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d80158030 msgr2=0x7f2d80159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d7effd700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d80158030 0x7f2d80159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d7effd700 1 -- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d800c9130 msgr2=0x7f2d800b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d7effd700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d800c9130 0x7f2d800b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d7effd700 1 -- 172.21.15.131:0/2647705543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70009490 con 0x7f2d8006b8c0 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d7e7fc700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d800c9130 0x7f2d800b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d7effd700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8006b8c0 0x7f2d80149050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2d800c89a0 tx=0x7f2d70017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d66ffd700 1 -- 172.21.15.131:0/2647705543 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000bbe0 con 0x7f2d8006b8c0 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d66ffd700 1 -- 172.21.15.131:0/2647705543 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d70000c90 con 0x7f2d8006b8c0 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d66ffd700 1 -- 172.21.15.131:0/2647705543 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70011650 con 0x7f2d8006b8c0 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.167+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d800b2130 con 0x7f2d8006b8c0 2023-12-26T21:52:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.168+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d800b26f0 con 0x7f2d8006b8c0 2023-12-26T21:52:52.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.168+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d800657e0 con 0x7f2d8006b8c0 2023-12-26T21:52:52.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.169+0000 7f2d66ffd700 1 -- 172.21.15.131:0/2647705543 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 32) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f2d70011a40 con 0x7f2d8006b8c0 2023-12-26T21:52:52.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.170+0000 7f2d66ffd700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 0x7f2d6c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:52.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.170+0000 7f2d66ffd700 1 -- 172.21.15.131:0/2647705543 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f2d70086340 con 0x7f2d8006b8c0 2023-12-26T21:52:52.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.170+0000 7f2d7e7fc700 -1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 0x7f2d6c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.170+0000 7f2d7e7fc700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 0x7f2d6c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:52:52.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.175+0000 7f2d66ffd700 1 -- 172.21.15.131:0/2647705543 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d70053030 con 0x7f2d8006b8c0 2023-12-26T21:52:52.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.372+0000 7f2d7e7fc700 -1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 0x7f2d6c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.372+0000 7f2d7e7fc700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 0x7f2d6c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:52:52.400 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:52.399+0000 7f86d929b1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:52.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.422+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2d800c67c0 con 0x7f2d8006b8c0 2023-12-26T21:52:52.471 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:52.471+0000 7efe30cac1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:52:52.492 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:52.492+0000 7f86d929b1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:52.684 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:52.684+0000 7f86d929b1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:52:52.687 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:52.688+0000 7f86cebfb700 -1 --2- 172.21.15.190:0/39347 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x55ead76cc000 0x55ead76d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.748 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:52.747+0000 7efe30cac1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:52:52.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.773+0000 7f2d7e7fc700 -1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 0x7f2d6c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.773+0000 7f2d7e7fc700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 0x7f2d6c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:52:52.835 INFO:tasks.ceph.osd.2.smithi131.stderr:2023-12-26T21:52:52.834+0000 7ff3f99fd700 -1 --2- [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x5588ae99c800 0x5588a529d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.835 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-12-26T21:52:52.834+0000 7f157c1fd700 -1 --2- [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x5643a6b1c400 0x5643a626c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.835 INFO:tasks.ceph.osd.0.smithi131.stderr:2023-12-26T21:52:52.834+0000 7f980dbfc700 -1 --2- [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x55749e850800 0x55748caca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.836 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-12-26T21:52:52.834+0000 7f9044dfd700 -1 --2- [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x55b442c58400 0x55b434e72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.836 INFO:tasks.ceph.osd.1.smithi131.stderr:2023-12-26T21:52:52.834+0000 7f3cce5fc700 -1 --2- [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x5570f830b000 0x5570f5423700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.836 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-12-26T21:52:52.834+0000 7f5fff3fc700 -1 --2- [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x55dd42e47800 0x55dd3f822580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.836 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T21:52:52.834+0000 7fd02a5ee700 -1 --2- 172.21.15.131:0/39287 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x5555801b1c00 0x5555808f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.837 INFO:tasks.ceph.mon.c.smithi190.stderr:2023-12-26T21:52:52.834+0000 7fd115df9700 -1 --2- 172.21.15.190:0/39346 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x55b4d4030000 0x55b4d3fbb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.837 INFO:tasks.ceph.osd.3.smithi131.stderr:2023-12-26T21:52:52.834+0000 7f835a7fb700 -1 --2- [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x55dd864c6000 0x55dd83f32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.837 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-12-26T21:52:52.834+0000 7f86c6ffd700 -1 --2- [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x556b5b003800 0x556b605c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.838 INFO:tasks.ceph.mds.a.smithi131.stderr:2023-12-26T21:52:52.836+0000 7f8b31dfc700 -1 --2- [v2:172.21.15.131:6832/899613179,v1:172.21.15.131:6834/899613179] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x56215402d800 0x5621533ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.838 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-12-26T21:52:52.834+0000 7f3b4f1fc700 -1 --2- [v2:172.21.15.190:6832/625115906,v1:172.21.15.190:6833/625115906] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x556ee6e5f800 0x556ee6221080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.838 INFO:tasks.ceph.mon.b.smithi190.stderr:2023-12-26T21:52:52.834+0000 7f7f4c9f9700 -1 --2- 172.21.15.190:0/39345 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x557242711800 0x5572427a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.838 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-12-26T21:52:52.836+0000 7fe6daffc700 -1 --2- [v2:172.21.15.190:6834/2773111229,v1:172.21.15.190:6835/2773111229] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x5585f7e03800 0x5585f71c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.841 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:52.841+0000 7efe30cac1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:52:52.889 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:52:52.889+0000 7f86cebfb700 -1 --2- 172.21.15.190:0/39347 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x55ead76cc000 0x55ead76d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.938+0000 7f2d66ffd700 1 -- 172.21.15.131:0/2647705543 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d800c67c0 con 0x7f2d8006b8c0 2023-12-26T21:52:52.939 INFO:tasks.ceph.mds.c.smithi131.stderr:2023-12-26T21:52:52.938+0000 7fab291fc700 -1 --2- [v2:172.21.15.131:6833/2948404662,v1:172.21.15.131:6835/2948404662] >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x55dedd9a5c00 0x55dedcd6a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:52.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 msgr2=0x7f2d6c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 0x7f2d6c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8006b8c0 msgr2=0x7f2d80149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8006b8c0 0x7f2d80149050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2d800c89a0 tx=0x7f2d70017000 comp rx=0 tx=0).stop 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 shutdown_connections 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d80158030 0x7f2d80159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d6c0670e0 0x7f2d6c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8006b8c0 0x7f2d80149050 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 --2- 172.21.15.131:0/2647705543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d800c9130 0x7f2d800b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 >> 172.21.15.131:0/2647705543 conn(0x7f2d800ce3e0 msgr2=0x7f2d8006cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 shutdown_connections 2023-12-26T21:52:52.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:52.941+0000 7f2d855ff700 1 -- 172.21.15.131:0/2647705543 wait complete. 2023-12-26T21:52:52.962 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T21:52:53.048 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:53.047+0000 7efe30cac1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:52:53.049 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:52:53.049+0000 7efe26dfc700 -1 --2- 172.21.15.131:0/39288 >> [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] conn(0x564f11a24000 0x564f11a2a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.131:6835/39288,v1:172.21.15.131:6837/39288] is using msgr V1 protocol 2023-12-26T21:52:53.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.217+0000 7fc1f79ff700 1 Processor -- start 2023-12-26T21:52:53.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.217+0000 7fc1f79ff700 1 -- start start 2023-12-26T21:52:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.219+0000 7fc1f79ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc1f0142f00 0x7fc1f0069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.219+0000 7fc1f79ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f00c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.219+0000 7fc1f79ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f0069640 con 0x7fc1f00cc590 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.219+0000 7fc1f79ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f0068750 con 0x7fc1f0142f00 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.219+0000 7fc1f79ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0068a30 con 0x7fc1f00cb510 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.219+0000 7fc1f579b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc1f0142f00 0x7fc1f0069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.219+0000 7fc1f579b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc1f0142f00 0x7fc1f0069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54358/0 (socket says 172.21.15.131:54358) 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.219+0000 7fc1f579b700 1 -- 172.21.15.131:0/210397114 learned_addr learned my addr 172.21.15.131:0/210397114 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.219+0000 7fc1f5f9c700 1 --2- 172.21.15.131:0/210397114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f00c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.220+0000 7fc1f579b700 1 -- 172.21.15.131:0/210397114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 msgr2=0x7fc1f00c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.220+0000 7fc1f579b700 1 --2- 172.21.15.131:0/210397114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f00c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.220+0000 7fc1f579b700 1 -- 172.21.15.131:0/210397114 >> v1:172.21.15.190:6789/0 conn(0x7fc1f00cc590 legacy=0x7fc1f00cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.220+0000 7fc1f579b700 1 -- 172.21.15.131:0/210397114 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f0068d10 con 0x7fc1f0142f00 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.220+0000 7fc1f5f9c700 1 --2- 172.21.15.131:0/210397114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f00c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.220+0000 7fc1f579b700 1 --2- 172.21.15.131:0/210397114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1f0142f00 0x7fc1f0069ad0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0099b0 tx=0x7fc1ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c849265753c9055 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:53.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.220+0000 7fc1e7fff700 1 -- 172.21.15.131:0/210397114 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec00b860 con 0x7fc1f0142f00 2023-12-26T21:52:53.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.220+0000 7fc1e7fff700 1 -- 172.21.15.131:0/210397114 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ec00b9c0 con 0x7fc1f0142f00 2023-12-26T21:52:53.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.220+0000 7fc1e7fff700 1 -- 172.21.15.131:0/210397114 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec00bd60 con 0x7fc1f0142f00 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.221+0000 7fc1f79ff700 1 -- 172.21.15.131:0/210397114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1f0142f00 msgr2=0x7fc1f0069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.221+0000 7fc1f79ff700 1 --2- 172.21.15.131:0/210397114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1f0142f00 0x7fc1f0069ad0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0099b0 tx=0x7fc1ec009250 comp rx=0 tx=0).stop 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.221+0000 7fc1f79ff700 1 -- 172.21.15.131:0/210397114 shutdown_connections 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.221+0000 7fc1f79ff700 1 --2- 172.21.15.131:0/210397114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f00c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.221+0000 7fc1f79ff700 1 --2- 172.21.15.131:0/210397114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1f0142f00 0x7fc1f0069ad0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.221+0000 7fc1f79ff700 1 -- 172.21.15.131:0/210397114 >> 172.21.15.131:0/210397114 conn(0x7fc1f0063cd0 msgr2=0x7fc1f006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.221+0000 7fc1f79ff700 1 -- 172.21.15.131:0/210397114 shutdown_connections 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.221+0000 7fc1f79ff700 1 -- 172.21.15.131:0/210397114 wait complete. 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.222+0000 7fc1f79ff700 1 Processor -- start 2023-12-26T21:52:53.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.223+0000 7fc1f79ff700 1 -- start start 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.223+0000 7fc1f79ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f0132bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.223+0000 7fc1f79ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f00cc590 0x7fc1f00c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f79ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1f0142f00 0x7fc1f00c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f79ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00b1d70 con 0x7fc1f0142f00 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f79ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00b11a0 con 0x7fc1f00cc590 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f79ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00b1440 con 0x7fc1f00cb510 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f579b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f0132bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f5f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1f0142f00 0x7fc1f00c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f579b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f0132bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:58038/0 (socket says 172.21.15.131:58038) 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f579b700 1 -- 172.21.15.131:0/3295100254 learned_addr learned my addr 172.21.15.131:0/3295100254 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f4f9a700 1 --2- 172.21.15.131:0/3295100254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f00cc590 0x7fc1f00c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:53.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f579b700 1 -- 172.21.15.131:0/3295100254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f00cc590 msgr2=0x7fc1f00c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:53.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f579b700 1 --2- 172.21.15.131:0/3295100254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f00cc590 0x7fc1f00c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:53.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f579b700 1 -- 172.21.15.131:0/3295100254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1f0142f00 msgr2=0x7fc1f00c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:53.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f579b700 1 --2- 172.21.15.131:0/3295100254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1f0142f00 0x7fc1f00c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:53.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.224+0000 7fc1f579b700 1 -- 172.21.15.131:0/3295100254 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec0094f0 con 0x7fc1f00cb510 2023-12-26T21:52:53.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.225+0000 7fc1f579b700 1 --2- 172.21.15.131:0/3295100254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f0132bb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009290 tx=0x7fc1ec017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:53.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.225+0000 7fc1e57fa700 1 -- 172.21.15.131:0/3295100254 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec011e00 con 0x7fc1f00cb510 2023-12-26T21:52:53.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.225+0000 7fc1e57fa700 1 -- 172.21.15.131:0/3295100254 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ec01f070 con 0x7fc1f00cb510 2023-12-26T21:52:53.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.225+0000 7fc1f79ff700 1 -- 172.21.15.131:0/3295100254 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f00b1720 con 0x7fc1f00cb510 2023-12-26T21:52:53.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.225+0000 7fc1e57fa700 1 -- 172.21.15.131:0/3295100254 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec0266d0 con 0x7fc1f00cb510 2023-12-26T21:52:53.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.225+0000 7fc1f79ff700 1 -- 172.21.15.131:0/3295100254 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f01330f0 con 0x7fc1f00cb510 2023-12-26T21:52:53.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.226+0000 7fc1f79ff700 1 -- 172.21.15.131:0/3295100254 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f00717f0 con 0x7fc1f00cb510 2023-12-26T21:52:53.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.227+0000 7fc1e57fa700 1 -- 172.21.15.131:0/3295100254 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 33) v1 ==== 84453+0+0 (secure 0 0 0) 0x7fc1ec026a40 con 0x7fc1f00cb510 2023-12-26T21:52:53.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.229+0000 7fc1e57fa700 1 -- 172.21.15.131:0/3295100254 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6742+0+0 (secure 0 0 0) 0x7fc1ec021070 con 0x7fc1f00cb510 2023-12-26T21:52:53.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.233+0000 7fc1e57fa700 1 -- 172.21.15.131:0/3295100254 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1ec053320 con 0x7fc1f00cb510 2023-12-26T21:52:53.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:53.428+0000 7fc1f79ff700 1 -- 172.21.15.131:0/3295100254 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc1f0068ff0 con 0x7fc1f00cb510 2023-12-26T21:52:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.062+0000 7fc1e57fa700 1 -- 172.21.15.131:0/3295100254 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc1ec056940 con 0x7fc1f00cb510 2023-12-26T21:52:54.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.065+0000 7fc1f79ff700 1 -- 172.21.15.131:0/3295100254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 msgr2=0x7fc1f0132bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:54.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.065+0000 7fc1f79ff700 1 --2- 172.21.15.131:0/3295100254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f0132bb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009290 tx=0x7fc1ec017000 comp rx=0 tx=0).stop 2023-12-26T21:52:54.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.065+0000 7fc1f79ff700 1 -- 172.21.15.131:0/3295100254 shutdown_connections 2023-12-26T21:52:54.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.065+0000 7fc1f79ff700 1 --2- 172.21.15.131:0/3295100254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1f0142f00 0x7fc1f00c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:54.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.065+0000 7fc1f79ff700 1 --2- 172.21.15.131:0/3295100254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1f00cb510 0x7fc1f0132bb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:54.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.065+0000 7fc1f79ff700 1 --2- 172.21.15.131:0/3295100254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1f00cc590 0x7fc1f00c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.065+0000 7fc1f79ff700 1 -- 172.21.15.131:0/3295100254 >> 172.21.15.131:0/3295100254 conn(0x7fc1f0063cd0 msgr2=0x7fc1f00c9350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.065+0000 7fc1f79ff700 1 -- 172.21.15.131:0/3295100254 shutdown_connections 2023-12-26T21:52:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.065+0000 7fc1f79ff700 1 -- 172.21.15.131:0/3295100254 wait complete. 2023-12-26T21:52:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T21:52:54.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T21:52:54.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.326+0000 7fa9aedff700 1 Processor -- start 2023-12-26T21:52:54.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.326+0000 7fa9aedff700 1 -- start start 2023-12-26T21:52:54.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.327+0000 7fa9aedff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa9a80cb510 0x7fa9a8069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:54.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.327+0000 7fa9aedff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa9a80cc590 0x7fa9a80c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:54.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.327+0000 7fa9aedff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a8069640 con 0x7fa9a8142f00 2023-12-26T21:52:54.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.327+0000 7fa9aedff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a8068750 con 0x7fa9a80cc590 2023-12-26T21:52:54.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.327+0000 7fa9aedff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a8068a30 con 0x7fa9a80cb510 2023-12-26T21:52:54.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.327+0000 7fa9ad39c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa9a80cc590 0x7fa9a80c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:54.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9acb9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa9a8142f00 0x7fa9a80cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43680/0 (socket says 172.21.15.131:43680) 2023-12-26T21:52:54.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9acb9b700 1 -- 172.21.15.131:0/1511154569 learned_addr learned my addr 172.21.15.131:0/1511154569 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9ad39c700 1 --2- 172.21.15.131:0/1511154569 >> v2:172.21.15.131:3300/0 conn(0x7fa9a80cc590 0x7fa9a80c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54366/0 (socket says 172.21.15.131:54366) 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9a7fff700 1 --2- 172.21.15.131:0/1511154569 >> v2:172.21.15.190:3300/0 conn(0x7fa9a80cb510 0x7fa9a8069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9ad39c700 1 -- 172.21.15.131:0/1511154569 >> v2:172.21.15.190:3300/0 conn(0x7fa9a80cb510 msgr2=0x7fa9a8069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9ad39c700 1 --2- 172.21.15.131:0/1511154569 >> v2:172.21.15.190:3300/0 conn(0x7fa9a80cb510 0x7fa9a8069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9ad39c700 1 -- 172.21.15.131:0/1511154569 >> v1:172.21.15.131:6789/0 conn(0x7fa9a8142f00 legacy=0x7fa9a80cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9ad39c700 1 -- 172.21.15.131:0/1511154569 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8068d10 con 0x7fa9a80cc590 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9a77fe700 1 -- 172.21.15.131:0/1511154569 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191854502 0 0) 0x7fa9a8069640 con 0x7fa9a8142f00 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9ad39c700 1 --2- 172.21.15.131:0/1511154569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a80cc590 0x7fa9a80c8a40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa9a806eeb0 tx=0x7fa9a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5dcaf8fbfb7d58 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.328+0000 7fa9a77fe700 1 -- 172.21.15.131:0/1511154569 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a000e070 con 0x7fa9a80cc590 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9a77fe700 1 -- 172.21.15.131:0/1511154569 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a000bc30 con 0x7fa9a80cc590 2023-12-26T21:52:54.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9a77fe700 1 -- 172.21.15.131:0/1511154569 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0012430 con 0x7fa9a80cc590 2023-12-26T21:52:54.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9aedff700 1 -- 172.21.15.131:0/1511154569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a80cc590 msgr2=0x7fa9a80c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:54.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9aedff700 1 --2- 172.21.15.131:0/1511154569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a80cc590 0x7fa9a80c8a40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa9a806eeb0 tx=0x7fa9a0009780 comp rx=0 tx=0).stop 2023-12-26T21:52:54.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9aedff700 1 -- 172.21.15.131:0/1511154569 shutdown_connections 2023-12-26T21:52:54.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9aedff700 1 --2- 172.21.15.131:0/1511154569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a80cc590 0x7fa9a80c8a40 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:54.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9aedff700 1 --2- 172.21.15.131:0/1511154569 >> v2:172.21.15.190:3300/0 conn(0x7fa9a80cb510 0x7fa9a8069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:54.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9aedff700 1 -- 172.21.15.131:0/1511154569 >> 172.21.15.131:0/1511154569 conn(0x7fa9a8063cd0 msgr2=0x7fa9a806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:54.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9aedff700 1 -- 172.21.15.131:0/1511154569 shutdown_connections 2023-12-26T21:52:54.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.329+0000 7fa9aedff700 1 -- 172.21.15.131:0/1511154569 wait complete. 2023-12-26T21:52:54.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.331+0000 7fa9aedff700 1 Processor -- start 2023-12-26T21:52:54.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.331+0000 7fa9aedff700 1 -- start start 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9aedff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9a80cb510 0x7fa9a813ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9aedff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9a80cc590 0x7fa9a815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9aedff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a8142f00 0x7fa9a80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9aedff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a815ead0 con 0x7fa9a8142f00 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9aedff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80c5650 con 0x7fa9a80cc590 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9aedff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80c5930 con 0x7fa9a80cb510 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9ad39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a8142f00 0x7fa9a80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9ad39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a8142f00 0x7fa9a80b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54374/0 (socket says 172.21.15.131:54374) 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9ad39c700 1 -- 172.21.15.131:0/3139856538 learned_addr learned my addr 172.21.15.131:0/3139856538 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9acb9b700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9a80cb510 0x7fa9a813ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9a7fff700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9a80cc590 0x7fa9a815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:54.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9ad39c700 1 -- 172.21.15.131:0/3139856538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9a80cb510 msgr2=0x7fa9a813ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9ad39c700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9a80cb510 0x7fa9a813ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9ad39c700 1 -- 172.21.15.131:0/3139856538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9a80cc590 msgr2=0x7fa9a815ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9ad39c700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9a80cc590 0x7fa9a815ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9ad39c700 1 -- 172.21.15.131:0/3139856538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a0009a20 con 0x7fa9a8142f00 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.332+0000 7fa9a7fff700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9a80cc590 0x7fa9a815ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.333+0000 7fa9ad39c700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a8142f00 0x7fa9a80b2170 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa9a806f3d0 tx=0x7fa9a0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.333+0000 7fa9a4ff9700 1 -- 172.21.15.131:0/3139856538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0012ea0 con 0x7fa9a8142f00 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.333+0000 7fa9a4ff9700 1 -- 172.21.15.131:0/3139856538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a001d630 con 0x7fa9a8142f00 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.333+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a815e480 con 0x7fa9a8142f00 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.333+0000 7fa9a4ff9700 1 -- 172.21.15.131:0/3139856538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a001d9f0 con 0x7fa9a8142f00 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.333+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a813f470 con 0x7fa9a8142f00 2023-12-26T21:52:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.334+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a815ec80 con 0x7fa9a8142f00 2023-12-26T21:52:54.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.335+0000 7fa9a4ff9700 1 -- 172.21.15.131:0/3139856538 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 34) v1 ==== 84580+0+0 (secure 0 0 0) 0x7fa9a0024070 con 0x7fa9a8142f00 2023-12-26T21:52:54.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.336+0000 7fa9a4ff9700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa98c066d80 0x7fa98c069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:54.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.336+0000 7fa9a4ff9700 1 -- 172.21.15.131:0/3139856538 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fa9a0086d80 con 0x7fa9a8142f00 2023-12-26T21:52:54.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.336+0000 7fa9acb9b700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa98c066d80 0x7fa98c069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:54.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.336+0000 7fa9acb9b700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa98c066d80 0x7fa98c069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00c8f0 tx=0x7fa99c009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:54.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.340+0000 7fa9a4ff9700 1 -- 172.21.15.131:0/3139856538 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9a815ec80 con 0x7fa9a8142f00 2023-12-26T21:52:54.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:54.542+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa9a80717f0 con 0x7fa9a8142f00 2023-12-26T21:52:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.216+0000 7fa9a4ff9700 1 -- 172.21.15.131:0/3139856538 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa9a0057a30 con 0x7fa9a8142f00 2023-12-26T21:52:55.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa98c066d80 msgr2=0x7fa98c069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa98c066d80 0x7fa98c069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00c8f0 tx=0x7fa99c009040 comp rx=0 tx=0).stop 2023-12-26T21:52:55.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a8142f00 msgr2=0x7fa9a80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a8142f00 0x7fa9a80b2170 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa9a806f3d0 tx=0x7fa9a0019000 comp rx=0 tx=0).stop 2023-12-26T21:52:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 shutdown_connections 2023-12-26T21:52:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa98c066d80 0x7fa98c069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9a8142f00 0x7fa9a80b2170 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9a80cb510 0x7fa9a813ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 --2- 172.21.15.131:0/3139856538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9a80cc590 0x7fa9a815ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 >> 172.21.15.131:0/3139856538 conn(0x7fa9a8063cd0 msgr2=0x7fa9a80c31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 shutdown_connections 2023-12-26T21:52:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.220+0000 7fa9aedff700 1 -- 172.21.15.131:0/3139856538 wait complete. 2023-12-26T21:52:55.222 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T21:52:55.246 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T21:52:55.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.439+0000 7f11393ff700 1 Processor -- start 2023-12-26T21:52:55.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.439+0000 7f11393ff700 1 -- start start 2023-12-26T21:52:55.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.441+0000 7f11393ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.441+0000 7f11393ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11340686c0 con 0x7f113406bfd0 2023-12-26T21:52:55.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.441+0000 7f11393ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1134069520 con 0x7f113413c510 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.441+0000 7f11393ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1134068910 con 0x7f11340cc810 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.441+0000 7f11327fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f113406bfd0 0x7f11340cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48892/0 (socket says 172.21.15.131:48892) 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.441+0000 7f11327fc700 1 -- 172.21.15.131:0/3804902132 learned_addr learned my addr 172.21.15.131:0/3804902132 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.441+0000 7f11337fe700 1 --2- 172.21.15.131:0/3804902132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.441+0000 7f1131ffb700 1 -- 172.21.15.131:0/3804902132 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814403812 0 0) 0x7f11340686c0 con 0x7f113406bfd0 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f1131ffb700 1 -- 172.21.15.131:0/3804902132 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f111c003570 con 0x7f113406bfd0 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f1131ffb700 1 -- 172.21.15.131:0/3804902132 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098675961 0 0) 0x7f1134069520 con 0x7f113413c510 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f1131ffb700 1 -- 172.21.15.131:0/3804902132 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11340686c0 con 0x7f113413c510 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f11337fe700 1 -- 172.21.15.131:0/3804902132 >> v1:172.21.15.190:6789/0 conn(0x7f113413c510 legacy=0x7f113406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f11337fe700 1 -- 172.21.15.131:0/3804902132 >> v1:172.21.15.131:6789/0 conn(0x7f113406bfd0 legacy=0x7f11340cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f11337fe700 1 -- 172.21.15.131:0/3804902132 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134068bf0 con 0x7f11340cc810 2023-12-26T21:52:55.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f11337fe700 1 --2- 172.21.15.131:0/3804902132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c8a20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f112c009e90 tx=0x7f112c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e7af4525aaa6a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f1131ffb700 1 -- 172.21.15.131:0/3804902132 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00e070 con 0x7f11340cc810 2023-12-26T21:52:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f1131ffb700 1 -- 172.21.15.131:0/3804902132 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c00bb80 con 0x7f11340cc810 2023-12-26T21:52:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.442+0000 7f1131ffb700 1 -- 172.21.15.131:0/3804902132 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c012430 con 0x7f11340cc810 2023-12-26T21:52:55.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.443+0000 7f11393ff700 1 -- 172.21.15.131:0/3804902132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 msgr2=0x7f11340c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.443+0000 7f11393ff700 1 --2- 172.21.15.131:0/3804902132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c8a20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f112c009e90 tx=0x7f112c009780 comp rx=0 tx=0).stop 2023-12-26T21:52:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.443+0000 7f11393ff700 1 -- 172.21.15.131:0/3804902132 shutdown_connections 2023-12-26T21:52:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.443+0000 7f11393ff700 1 --2- 172.21.15.131:0/3804902132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c8a20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.443+0000 7f11393ff700 1 -- 172.21.15.131:0/3804902132 >> 172.21.15.131:0/3804902132 conn(0x7f11340ce3e0 msgr2=0x7f11340ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.443+0000 7f11393ff700 1 -- 172.21.15.131:0/3804902132 shutdown_connections 2023-12-26T21:52:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.443+0000 7f11393ff700 1 -- 172.21.15.131:0/3804902132 wait complete. 2023-12-26T21:52:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.444+0000 7f11393ff700 1 Processor -- start 2023-12-26T21:52:55.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.445+0000 7f11393ff700 1 -- start start 2023-12-26T21:52:55.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.445+0000 7f11393ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f113406bfd0 0x7f11340c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f11393ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f11393ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f113413c510 0x7f11340c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f11393ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341540b0 con 0x7f113406bfd0 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f11393ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340b14b0 con 0x7f113413c510 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f11393ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11340b29d0 con 0x7f11340cc810 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f113406bfd0 0x7f11340c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f113406bfd0 0x7f11340c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41410/0 (socket says 172.21.15.131:41410) 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f1132ffd700 1 -- 172.21.15.131:0/2321841271 learned_addr learned my addr 172.21.15.131:0/2321841271 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f11327fc700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f11337fe700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f113413c510 0x7f11340c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f1132ffd700 1 -- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 msgr2=0x7f11340c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f1132ffd700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f1132ffd700 1 -- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f113413c510 msgr2=0x7f11340c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f1132ffd700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f113413c510 0x7f11340c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f1132ffd700 1 -- 172.21.15.131:0/2321841271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c009a20 con 0x7f113406bfd0 2023-12-26T21:52:55.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f11327fc700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:55.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f11337fe700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f113413c510 0x7f11340c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:55.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.446+0000 7f1132ffd700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f113406bfd0 0x7f11340c7d60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f112400c7c0 tx=0x7f1124007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:55.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.447+0000 7f111affd700 1 -- 172.21.15.131:0/2321841271 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124011680 con 0x7f113406bfd0 2023-12-26T21:52:55.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.447+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11340c82a0 con 0x7f113406bfd0 2023-12-26T21:52:55.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.447+0000 7f111affd700 1 -- 172.21.15.131:0/2321841271 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11240117e0 con 0x7f113406bfd0 2023-12-26T21:52:55.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.447+0000 7f111affd700 1 -- 172.21.15.131:0/2321841271 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124011ba0 con 0x7f113406bfd0 2023-12-26T21:52:55.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.447+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113413a380 con 0x7f113406bfd0 2023-12-26T21:52:55.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.448+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11340cba40 con 0x7f113406bfd0 2023-12-26T21:52:55.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.449+0000 7f111affd700 1 -- 172.21.15.131:0/2321841271 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1124012440 con 0x7f113406bfd0 2023-12-26T21:52:55.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.449+0000 7f111affd700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1120066f30 0x7f11200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.450+0000 7f111affd700 1 -- 172.21.15.131:0/2321841271 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f11240845d0 con 0x7f113406bfd0 2023-12-26T21:52:55.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.450+0000 7f11327fc700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1120066f30 0x7f11200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.450+0000 7f11327fc700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1120066f30 0x7f11200693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f11340c8dd0 tx=0x7f1128009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:55.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.453+0000 7f111affd700 1 -- 172.21.15.131:0/2321841271 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1124051780 con 0x7f113406bfd0 2023-12-26T21:52:55.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.647+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f11340657e0 con 0x7f113406bfd0 2023-12-26T21:52:55.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.652+0000 7f111affd700 1 -- 172.21.15.131:0/2321841271 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1124054da0 con 0x7f113406bfd0 2023-12-26T21:52:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1120066f30 msgr2=0x7f11200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1120066f30 0x7f11200693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f11340c8dd0 tx=0x7f1128009040 comp rx=0 tx=0).stop 2023-12-26T21:52:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f113406bfd0 msgr2=0x7f11340c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f113406bfd0 0x7f11340c7d60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f112400c7c0 tx=0x7f1124007dc0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 shutdown_connections 2023-12-26T21:52:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f113413c510 0x7f11340c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1120066f30 0x7f11200693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f113406bfd0 0x7f11340c7d60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 --2- 172.21.15.131:0/2321841271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11340cc810 0x7f11340c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 >> 172.21.15.131:0/2321841271 conn(0x7f11340ce3e0 msgr2=0x7f113413b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:55.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 shutdown_connections 2023-12-26T21:52:55.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.654+0000 7f11393ff700 1 -- 172.21.15.131:0/2321841271 wait complete. 2023-12-26T21:52:55.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T21:52:55.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.862+0000 7f0b203ff700 1 Processor -- start 2023-12-26T21:52:55.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.862+0000 7f0b203ff700 1 -- start start 2023-12-26T21:52:55.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.863+0000 7f0b203ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0b18159820 0x7f0b180abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.863+0000 7f0b203ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1815a8c0 con 0x7f0b1806e480 2023-12-26T21:52:55.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.863+0000 7f0b203ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b18159f90 con 0x7f0b180aa480 2023-12-26T21:52:55.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.863+0000 7f0b203ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1815ad50 con 0x7f0b18159820 2023-12-26T21:52:55.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.863+0000 7f0b1e19b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0b18159820 0x7f0b180abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.863+0000 7f0b1e99c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0b1806e480 0x7f0b180675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48904/0 (socket says 172.21.15.131:48904) 2023-12-26T21:52:55.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.863+0000 7f0b1e99c700 1 -- 172.21.15.131:0/637137739 learned_addr learned my addr 172.21.15.131:0/637137739 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.864+0000 7f0b1e19b700 1 -- 172.21.15.131:0/637137739 >> v1:172.21.15.190:6789/0 conn(0x7f0b180aa480 legacy=0x7f0b1806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.864+0000 7f0b1e19b700 1 -- 172.21.15.131:0/637137739 >> v1:172.21.15.131:6789/0 conn(0x7f0b1806e480 legacy=0x7f0b180675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.864+0000 7f0b1e19b700 1 -- 172.21.15.131:0/637137739 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1815a270 con 0x7f0b18159820 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.864+0000 7f0b1d199700 1 -- 172.21.15.131:0/637137739 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358741554 0 0) 0x7f0b1815a8c0 con 0x7f0b1806e480 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.864+0000 7f0b1e19b700 1 --2- 172.21.15.131:0/637137739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b18159820 0x7f0b180abdb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0b140098e0 tx=0x7f0b140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa824636939a0259 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.864+0000 7f0b1d199700 1 -- 172.21.15.131:0/637137739 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14009e80 con 0x7f0b18159820 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.864+0000 7f0b1d199700 1 -- 172.21.15.131:0/637137739 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1400b860 con 0x7f0b18159820 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.864+0000 7f0b1d199700 1 -- 172.21.15.131:0/637137739 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400bbe0 con 0x7f0b18159820 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.865+0000 7f0b203ff700 1 -- 172.21.15.131:0/637137739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b18159820 msgr2=0x7f0b180abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.865+0000 7f0b203ff700 1 --2- 172.21.15.131:0/637137739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b18159820 0x7f0b180abdb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0b140098e0 tx=0x7f0b140091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.865+0000 7f0b203ff700 1 -- 172.21.15.131:0/637137739 shutdown_connections 2023-12-26T21:52:55.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.865+0000 7f0b203ff700 1 --2- 172.21.15.131:0/637137739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b18159820 0x7f0b180abdb0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.865+0000 7f0b203ff700 1 -- 172.21.15.131:0/637137739 >> 172.21.15.131:0/637137739 conn(0x7f0b1806c290 msgr2=0x7f0b180ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:55.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.865+0000 7f0b203ff700 1 -- 172.21.15.131:0/637137739 shutdown_connections 2023-12-26T21:52:55.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.865+0000 7f0b203ff700 1 -- 172.21.15.131:0/637137739 wait complete. 2023-12-26T21:52:55.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.867+0000 7f0b203ff700 1 Processor -- start 2023-12-26T21:52:55.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.867+0000 7f0b203ff700 1 -- start start 2023-12-26T21:52:55.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b203ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b1806e480 0x7f0b180ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b203ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b180aa480 0x7f0b180ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b203ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b18159820 0x7f0b1816a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b203ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b18144a20 con 0x7f0b1806e480 2023-12-26T21:52:55.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b203ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1816b470 con 0x7f0b18159820 2023-12-26T21:52:55.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b203ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1816b750 con 0x7f0b180aa480 2023-12-26T21:52:55.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1e19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b1806e480 0x7f0b180ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1e19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b1806e480 0x7f0b180ad370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41428/0 (socket says 172.21.15.131:41428) 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1e19b700 1 -- 172.21.15.131:0/2251635196 learned_addr learned my addr 172.21.15.131:0/2251635196 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1d99a700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b180aa480 0x7f0b180ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1e99c700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b18159820 0x7f0b1816a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1e19b700 1 -- 172.21.15.131:0/2251635196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b180aa480 msgr2=0x7f0b180ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1e19b700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b180aa480 0x7f0b180ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1e19b700 1 -- 172.21.15.131:0/2251635196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b18159820 msgr2=0x7f0b1816a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1e19b700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b18159820 0x7f0b1816a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1e19b700 1 -- 172.21.15.131:0/2251635196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b14009420 con 0x7f0b1806e480 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.868+0000 7f0b1d99a700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b180aa480 0x7f0b180ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:55.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.869+0000 7f0b1e19b700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b1806e480 0x7f0b180ad370 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0b180af700 tx=0x7f0b14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:55.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.869+0000 7f0b0e7fc700 1 -- 172.21.15.131:0/2251635196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1401f070 con 0x7f0b1806e480 2023-12-26T21:52:55.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.869+0000 7f0b0e7fc700 1 -- 172.21.15.131:0/2251635196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1400b9d0 con 0x7f0b1806e480 2023-12-26T21:52:55.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.869+0000 7f0b0e7fc700 1 -- 172.21.15.131:0/2251635196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14025670 con 0x7f0b1806e480 2023-12-26T21:52:55.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.869+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1816ba30 con 0x7f0b1806e480 2023-12-26T21:52:55.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.869+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b180addf0 con 0x7f0b1806e480 2023-12-26T21:52:55.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.870+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b180657e0 con 0x7f0b1806e480 2023-12-26T21:52:55.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.871+0000 7f0b0e7fc700 1 -- 172.21.15.131:0/2251635196 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0b14023070 con 0x7f0b1806e480 2023-12-26T21:52:55.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.872+0000 7f0b0e7fc700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0afc066f30 0x7f0afc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:55.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.872+0000 7f0b0e7fc700 1 -- 172.21.15.131:0/2251635196 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f0b14085c10 con 0x7f0b1806e480 2023-12-26T21:52:55.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.872+0000 7f0b1d99a700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0afc066f30 0x7f0afc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:55.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.872+0000 7f0b1d99a700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0afc066f30 0x7f0afc0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0b08007fd0 tx=0x7f0b08009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:55.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:55.876+0000 7f0b0e7fc700 1 -- 172.21.15.131:0/2251635196 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b14052d20 con 0x7f0b1806e480 2023-12-26T21:52:56.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.070+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f0b1816bf30 con 0x7f0b1806e480 2023-12-26T21:52:56.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.076+0000 7f0b0e7fc700 1 -- 172.21.15.131:0/2251635196 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v26) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0b140150b0 con 0x7f0b1806e480 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0afc066f30 msgr2=0x7f0afc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0afc066f30 0x7f0afc0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0b08007fd0 tx=0x7f0b08009040 comp rx=0 tx=0).stop 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b1806e480 msgr2=0x7f0b180ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b1806e480 0x7f0b180ad370 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0b180af700 tx=0x7f0b14017000 comp rx=0 tx=0).stop 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 shutdown_connections 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b18159820 0x7f0b1816a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0afc066f30 0x7f0afc0693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b1806e480 0x7f0b180ad370 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 --2- 172.21.15.131:0/2251635196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b180aa480 0x7f0b180ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 >> 172.21.15.131:0/2251635196 conn(0x7f0b1806c290 msgr2=0x7f0b18158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 shutdown_connections 2023-12-26T21:52:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.079+0000 7f0b203ff700 1 -- 172.21.15.131:0/2251635196 wait complete. 2023-12-26T21:52:56.099 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T21:52:56.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.290+0000 7ff93a9ff700 1 Processor -- start 2023-12-26T21:52:56.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.290+0000 7ff93a9ff700 1 -- start start 2023-12-26T21:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.291+0000 7ff93a9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9340b0750 0x7ff9340b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.291+0000 7ff93a9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff934156940 con 0x7ff9340c9400 2023-12-26T21:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.291+0000 7ff93a9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340ae500 con 0x7ff9340cb320 2023-12-26T21:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.291+0000 7ff93a9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9340c8cc0 con 0x7ff9340b0750 2023-12-26T21:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.291+0000 7ff938f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff9340c9400 0x7ff9340c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48920/0 (socket says 172.21.15.131:48920) 2023-12-26T21:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.291+0000 7ff938f9c700 1 -- 172.21.15.131:0/3996544733 learned_addr learned my addr 172.21.15.131:0/3996544733 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.291+0000 7ff9337fe700 1 --2- 172.21.15.131:0/3996544733 >> v2:172.21.15.190:3300/0 conn(0x7ff9340b0750 0x7ff9340b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.292+0000 7ff932ffd700 1 -- 172.21.15.131:0/3996544733 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336211241 0 0) 0x7ff934156940 con 0x7ff9340c9400 2023-12-26T21:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.292+0000 7ff932ffd700 1 -- 172.21.15.131:0/3996544733 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff91c003570 con 0x7ff9340c9400 2023-12-26T21:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.292+0000 7ff932ffd700 1 -- 172.21.15.131:0/3996544733 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217042929 0 0) 0x7ff9340ae500 con 0x7ff9340cb320 2023-12-26T21:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.292+0000 7ff932ffd700 1 -- 172.21.15.131:0/3996544733 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff934156940 con 0x7ff9340cb320 2023-12-26T21:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.292+0000 7ff9337fe700 1 -- 172.21.15.131:0/3996544733 >> v1:172.21.15.190:6789/0 conn(0x7ff9340cb320 legacy=0x7ff934069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.292+0000 7ff9337fe700 1 -- 172.21.15.131:0/3996544733 >> v1:172.21.15.131:6789/0 conn(0x7ff9340c9400 legacy=0x7ff9340c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.292+0000 7ff9337fe700 1 -- 172.21.15.131:0/3996544733 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9340b0280 con 0x7ff9340b0750 2023-12-26T21:52:56.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.292+0000 7ff9337fe700 1 --2- 172.21.15.131:0/3996544733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340b0750 0x7ff9340b0bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff9280098e0 tx=0x7ff9280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c436a874b003a8ee server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:56.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.292+0000 7ff932ffd700 1 -- 172.21.15.131:0/3996544733 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011630 con 0x7ff9340b0750 2023-12-26T21:52:56.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.293+0000 7ff932ffd700 1 -- 172.21.15.131:0/3996544733 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff928011790 con 0x7ff9340b0750 2023-12-26T21:52:56.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.293+0000 7ff932ffd700 1 -- 172.21.15.131:0/3996544733 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011b30 con 0x7ff9340b0750 2023-12-26T21:52:56.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.293+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3996544733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340b0750 msgr2=0x7ff9340b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.293+0000 7ff93a9ff700 1 --2- 172.21.15.131:0/3996544733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340b0750 0x7ff9340b0bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff9280098e0 tx=0x7ff9280091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.293+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3996544733 shutdown_connections 2023-12-26T21:52:56.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.293+0000 7ff93a9ff700 1 --2- 172.21.15.131:0/3996544733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340b0750 0x7ff9340b0bf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.293+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3996544733 >> 172.21.15.131:0/3996544733 conn(0x7ff934068040 msgr2=0x7ff934069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:56.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.293+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3996544733 shutdown_connections 2023-12-26T21:52:56.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.293+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3996544733 wait complete. 2023-12-26T21:52:56.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.295+0000 7ff93a9ff700 1 Processor -- start 2023-12-26T21:52:56.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.295+0000 7ff93a9ff700 1 -- start start 2023-12-26T21:52:56.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff93a9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9340b0750 0x7ff934142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff93a9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340c9400 0x7ff934157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff93a9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9340cb320 0x7ff93415f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff93a9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93415d4a0 con 0x7ff9340b0750 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff93a9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93415d170 con 0x7ff9340c9400 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff93a9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341580b0 con 0x7ff9340cb320 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff933fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9340b0750 0x7ff934142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff933fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9340b0750 0x7ff934142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41440/0 (socket says 172.21.15.131:41440) 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff933fff700 1 -- 172.21.15.131:0/3485165725 learned_addr learned my addr 172.21.15.131:0/3485165725 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff9337fe700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340c9400 0x7ff934157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff938f9c700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9340cb320 0x7ff93415f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff933fff700 1 -- 172.21.15.131:0/3485165725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9340cb320 msgr2=0x7ff93415f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff933fff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9340cb320 0x7ff93415f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff933fff700 1 -- 172.21.15.131:0/3485165725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340c9400 msgr2=0x7ff934157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff933fff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340c9400 0x7ff934157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff933fff700 1 -- 172.21.15.131:0/3485165725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff928009420 con 0x7ff9340b0750 2023-12-26T21:52:56.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.296+0000 7ff9337fe700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340c9400 0x7ff934157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.297+0000 7ff933fff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9340b0750 0x7ff934142c10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c580 tx=0x7ff924007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.297+0000 7ff91bfff700 1 -- 172.21.15.131:0/3485165725 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924010510 con 0x7ff9340b0750 2023-12-26T21:52:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.297+0000 7ff91bfff700 1 -- 172.21.15.131:0/3485165725 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff924010670 con 0x7ff9340b0750 2023-12-26T21:52:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.297+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff934158390 con 0x7ff9340b0750 2023-12-26T21:52:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.297+0000 7ff91bfff700 1 -- 172.21.15.131:0/3485165725 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924010a30 con 0x7ff9340b0750 2023-12-26T21:52:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.297+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff934143150 con 0x7ff9340b0750 2023-12-26T21:52:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.298+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9340b1250 con 0x7ff9340b0750 2023-12-26T21:52:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.299+0000 7ff91bfff700 1 -- 172.21.15.131:0/3485165725 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff92401b400 con 0x7ff9340b0750 2023-12-26T21:52:56.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.300+0000 7ff91bfff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff920066f30 0x7ff9200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.300+0000 7ff91bfff700 1 -- 172.21.15.131:0/3485165725 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5890+0+0 (secure 0 0 0) 0x7ff924084390 con 0x7ff9340b0750 2023-12-26T21:52:56.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.300+0000 7ff9337fe700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff920066f30 0x7ff9200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.300+0000 7ff9337fe700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff920066f30 0x7ff9200693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff9280093f0 tx=0x7ff928017000 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:56.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.304+0000 7ff91bfff700 1 -- 172.21.15.131:0/3485165725 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9240514a0 con 0x7ff9340b0750 2023-12-26T21:52:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.496+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7ff9340657e0 con 0x7ff9340b0750 2023-12-26T21:52:56.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.501+0000 7ff91bfff700 1 -- 172.21.15.131:0/3485165725 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v27) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff924054ac0 con 0x7ff9340b0750 2023-12-26T21:52:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff920066f30 msgr2=0x7ff9200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff920066f30 0x7ff9200693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff9280093f0 tx=0x7ff928017000 comp rx=0 tx=0).stop 2023-12-26T21:52:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9340b0750 msgr2=0x7ff934142c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9340b0750 0x7ff934142c10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c580 tx=0x7ff924007e10 comp rx=0 tx=0).stop 2023-12-26T21:52:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 shutdown_connections 2023-12-26T21:52:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9340cb320 0x7ff93415f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff920066f30 0x7ff9200693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9340b0750 0x7ff934142c10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 --2- 172.21.15.131:0/3485165725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9340c9400 0x7ff934157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.503+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 >> 172.21.15.131:0/3485165725 conn(0x7ff934068040 msgr2=0x7ff93406e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:56.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.504+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 shutdown_connections 2023-12-26T21:52:56.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.504+0000 7ff93a9ff700 1 -- 172.21.15.131:0/3485165725 wait complete. 2023-12-26T21:52:56.522 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-12-26T21:52:56.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.710+0000 7faaa83ff700 1 Processor -- start 2023-12-26T21:52:56.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.710+0000 7faaa83ff700 1 -- start start 2023-12-26T21:52:56.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.711+0000 7faaa83ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faaa006bfd0 0x7faaa00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.711+0000 7faaa83ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faaa00cc100 0x7faaa00cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.711+0000 7faaa83ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa00686c0 con 0x7faaa013c510 2023-12-26T21:52:56.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.711+0000 7faaa83ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa0069520 con 0x7faaa006bfd0 2023-12-26T21:52:56.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.711+0000 7faaa83ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa0068910 con 0x7faaa00cc100 2023-12-26T21:52:56.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa599a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faaa006bfd0 0x7faaa00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa619b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7faaa013c510 0x7faaa006b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48934/0 (socket says 172.21.15.131:48934) 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa619b700 1 -- 172.21.15.131:0/566989245 learned_addr learned my addr 172.21.15.131:0/566989245 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa599a700 1 -- 172.21.15.131:0/566989245 >> v2:172.21.15.190:3300/0 conn(0x7faaa00cc100 msgr2=0x7faaa00cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa699c700 1 --2- 172.21.15.131:0/566989245 >> v2:172.21.15.190:3300/0 conn(0x7faaa00cc100 0x7faaa00cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa599a700 1 --2- 172.21.15.131:0/566989245 >> v2:172.21.15.190:3300/0 conn(0x7faaa00cc100 0x7faaa00cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa599a700 1 -- 172.21.15.131:0/566989245 >> v1:172.21.15.131:6789/0 conn(0x7faaa013c510 legacy=0x7faaa006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa5199700 1 -- 172.21.15.131:0/566989245 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470159709 0 0) 0x7faaa00686c0 con 0x7faaa013c510 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa599a700 1 -- 172.21.15.131:0/566989245 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0068bf0 con 0x7faaa006bfd0 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa699c700 1 --2- 172.21.15.131:0/566989245 >> v2:172.21.15.190:3300/0 conn(0x7faaa00cc100 0x7faaa00cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.712+0000 7faaa599a700 1 --2- 172.21.15.131:0/566989245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa006bfd0 0x7faaa00c8a20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faaa00cf0a0 tx=0x7faa900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af772720b1920eb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa5199700 1 -- 172.21.15.131:0/566989245 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9000b6b0 con 0x7faaa006bfd0 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa5199700 1 -- 172.21.15.131:0/566989245 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9000b810 con 0x7faaa006bfd0 2023-12-26T21:52:56.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa5199700 1 -- 172.21.15.131:0/566989245 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9000bbb0 con 0x7faaa006bfd0 2023-12-26T21:52:56.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa83ff700 1 -- 172.21.15.131:0/566989245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa006bfd0 msgr2=0x7faaa00c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa83ff700 1 --2- 172.21.15.131:0/566989245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa006bfd0 0x7faaa00c8a20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faaa00cf0a0 tx=0x7faa900091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa83ff700 1 -- 172.21.15.131:0/566989245 shutdown_connections 2023-12-26T21:52:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa83ff700 1 --2- 172.21.15.131:0/566989245 >> v2:172.21.15.190:3300/0 conn(0x7faaa00cc100 0x7faaa00cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa83ff700 1 --2- 172.21.15.131:0/566989245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa006bfd0 0x7faaa00c8a20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa83ff700 1 -- 172.21.15.131:0/566989245 >> 172.21.15.131:0/566989245 conn(0x7faaa00ce3e0 msgr2=0x7faaa00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa83ff700 1 -- 172.21.15.131:0/566989245 shutdown_connections 2023-12-26T21:52:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.713+0000 7faaa83ff700 1 -- 172.21.15.131:0/566989245 wait complete. 2023-12-26T21:52:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.715+0000 7faaa83ff700 1 Processor -- start 2023-12-26T21:52:56.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.715+0000 7faaa83ff700 1 -- start start 2023-12-26T21:52:56.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa83ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaa006bfd0 0x7faaa0156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa83ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa00cc100 0x7faaa016d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa83ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaa013c510 0x7faaa016ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa83ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa015edb0 con 0x7faaa00cc100 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa83ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa015ead0 con 0x7faaa006bfd0 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa83ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa00b1320 con 0x7faaa013c510 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa599a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa00cc100 0x7faaa016d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa599a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa00cc100 0x7faaa016d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41454/0 (socket says 172.21.15.131:41454) 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa599a700 1 -- 172.21.15.131:0/2944880889 learned_addr learned my addr 172.21.15.131:0/2944880889 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa699c700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaa013c510 0x7faaa016ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa619b700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaa006bfd0 0x7faaa0156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa599a700 1 -- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaa013c510 msgr2=0x7faaa016ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa599a700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaa013c510 0x7faaa016ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa599a700 1 -- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaa006bfd0 msgr2=0x7faaa0156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa599a700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaa006bfd0 0x7faaa0156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa599a700 1 -- 172.21.15.131:0/2944880889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa90009420 con 0x7faaa00cc100 2023-12-26T21:52:56.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa699c700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaa013c510 0x7faaa016ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.716+0000 7faaa619b700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaa006bfd0 0x7faaa0156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.717+0000 7faaa599a700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa00cc100 0x7faaa016d8b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faa90013000 tx=0x7faa90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.717+0000 7faa967fc700 1 -- 172.21.15.131:0/2944880889 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa90011420 con 0x7faaa00cc100 2023-12-26T21:52:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.717+0000 7faa967fc700 1 -- 172.21.15.131:0/2944880889 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa90011580 con 0x7faaa00cc100 2023-12-26T21:52:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.717+0000 7faa967fc700 1 -- 172.21.15.131:0/2944880889 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa90011920 con 0x7faaa00cc100 2023-12-26T21:52:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.717+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa00b1600 con 0x7faaa00cc100 2023-12-26T21:52:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.717+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa015dda0 con 0x7faaa00cc100 2023-12-26T21:52:56.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.718+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa00b1180 con 0x7faaa00cc100 2023-12-26T21:52:56.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.719+0000 7faa967fc700 1 -- 172.21.15.131:0/2944880889 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faa90011d10 con 0x7faaa00cc100 2023-12-26T21:52:56.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.720+0000 7faa967fc700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa84066f30 0x7faa840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:56.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.720+0000 7faa967fc700 1 -- 172.21.15.131:0/2944880889 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5890+0+0 (secure 0 0 0) 0x7faa90086070 con 0x7faaa00cc100 2023-12-26T21:52:56.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.720+0000 7faaa619b700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa84066f30 0x7faa840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:56.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.721+0000 7faaa619b700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa84066f30 0x7faa840693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa9c001360 tx=0x7faa9c009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:56.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.724+0000 7faa967fc700 1 -- 172.21.15.131:0/2944880889 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa900b75a0 con 0x7faaa00cc100 2023-12-26T21:52:56.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:56.998+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7faaa00b1b20 con 0x7faaa00cc100 2023-12-26T21:52:57.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.220+0000 7faa967fc700 1 -- 172.21.15.131:0/2944880889 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v592) v1 ==== 156+0+0 (secure 0 0 0) 0x7faaa00b1b20 con 0x7faaa00cc100 2023-12-26T21:52:57.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.222+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa84066f30 msgr2=0x7faa840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:57.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.222+0000 7faaa83ff700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa84066f30 0x7faa840693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa9c001360 tx=0x7faa9c009040 comp rx=0 tx=0).stop 2023-12-26T21:52:57.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.222+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa00cc100 msgr2=0x7faaa016d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.222+0000 7faaa83ff700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa00cc100 0x7faaa016d8b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faa90013000 tx=0x7faa90017000 comp rx=0 tx=0).stop 2023-12-26T21:52:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.222+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 shutdown_connections 2023-12-26T21:52:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.222+0000 7faaa83ff700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faaa013c510 0x7faaa016ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.222+0000 7faaa83ff700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faaa006bfd0 0x7faaa0156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.222+0000 7faaa83ff700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa84066f30 0x7faa840693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.222+0000 7faaa83ff700 1 --2- 172.21.15.131:0/2944880889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa00cc100 0x7faaa016d8b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.223+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 >> 172.21.15.131:0/2944880889 conn(0x7faaa00ce3e0 msgr2=0x7faaa0153910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.223+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 shutdown_connections 2023-12-26T21:52:57.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.223+0000 7faaa83ff700 1 -- 172.21.15.131:0/2944880889 wait complete. 2023-12-26T21:52:57.241 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-12-26T21:52:57.241 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-12-26T21:52:57.242 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T21:52:57.242 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-12-26T21:52:57.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.431+0000 7f22641ff700 1 Processor -- start 2023-12-26T21:52:57.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.431+0000 7f22641ff700 1 -- start start 2023-12-26T21:52:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.432+0000 7f22641ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c0b07f0 0x7f225c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.432+0000 7f22641ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225c0c8bf0 con 0x7f225c06a070 2023-12-26T21:52:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.432+0000 7f22641ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225c15a030 con 0x7f225c069ca0 2023-12-26T21:52:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.432+0000 7f22641ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c0c8e60 con 0x7f225c0b07f0 2023-12-26T21:52:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.432+0000 7f226179a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f225c06a070 0x7f225c0b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48940/0 (socket says 172.21.15.131:48940) 2023-12-26T21:52:57.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.432+0000 7f226179a700 1 -- 172.21.15.131:0/3577603008 learned_addr learned my addr 172.21.15.131:0/3577603008 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:57.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f226279c700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c0b07f0 0x7f225c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:57.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631271283 0 0) 0x7f225c0c8bf0 con 0x7f225c06a070 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2248003570 con 0x7f225c06a070 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489782962 0 0) 0x7f225c15a030 con 0x7f225c069ca0 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f225c0c8bf0 con 0x7f225c069ca0 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3434285371 0 0) 0x7f2248003570 con 0x7f225c06a070 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c0b07f0 msgr2=0x7f225c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c0b07f0 0x7f225c1554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 >> v1:172.21.15.190:6789/0 conn(0x7f225c069ca0 legacy=0x7f225c159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c0c9140 con 0x7f225c06a070 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f224c001470 con 0x7f225c06a070 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 >> v1:172.21.15.131:6789/0 conn(0x7f225c06a070 legacy=0x7f225c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2248003c20 0x7f2248003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248004810 0x7f2248004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:57.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22480051c0 0x7f224800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c0c8e60 con 0x7f22480051c0 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224800bfd0 con 0x7f2248004810 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.433+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224800c190 con 0x7f2248003c20 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f2261f9b700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2248003c20 0x7f2248003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f226279c700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22480051c0 0x7f224800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f226179a700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248004810 0x7f2248004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f226279c700 1 -- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2248003c20 msgr2=0x7f2248003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f226279c700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2248003c20 0x7f2248003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f226279c700 1 -- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248004810 msgr2=0x7f2248004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f226279c700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2248004810 0x7f2248004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f226279c700 1 -- 172.21.15.131:0/3577603008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2248003570 con 0x7f22480051c0 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f2261f9b700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2248003c20 0x7f2248003ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.434+0000 7f226279c700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22480051c0 0x7f224800ba60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f225c0b1720 tx=0x7f225400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd6821e8498f13a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:57.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225400ea90 con 0x7f22480051c0 2023-12-26T21:52:57.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225400f040 con 0x7f22480051c0 2023-12-26T21:52:57.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f2260f99700 1 -- 172.21.15.131:0/3577603008 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225400ee40 con 0x7f22480051c0 2023-12-26T21:52:57.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f22641ff700 1 -- 172.21.15.131:0/3577603008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22480051c0 msgr2=0x7f224800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:57.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f22641ff700 1 --2- 172.21.15.131:0/3577603008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22480051c0 0x7f224800ba60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f225c0b1720 tx=0x7f225400b040 comp rx=0 tx=0).stop 2023-12-26T21:52:57.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f226279c700 1 -- 172.21.15.131:0/3577603008 reap_dead start 2023-12-26T21:52:57.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f22641ff700 1 -- 172.21.15.131:0/3577603008 shutdown_connections 2023-12-26T21:52:57.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f22641ff700 1 -- 172.21.15.131:0/3577603008 >> 172.21.15.131:0/3577603008 conn(0x7f225c068040 msgr2=0x7f225c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:57.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f22641ff700 1 -- 172.21.15.131:0/3577603008 shutdown_connections 2023-12-26T21:52:57.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.435+0000 7f22641ff700 1 -- 172.21.15.131:0/3577603008 wait complete. 2023-12-26T21:52:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.437+0000 7f22641ff700 1 Processor -- start 2023-12-26T21:52:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.437+0000 7f22641ff700 1 -- start start 2023-12-26T21:52:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f22641ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f225c15f490 0x7f225c14a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f22641ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c15f860 0x7f225c14ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:57.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f22641ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f225c15cf60 0x7f225c15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f22641ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c14b370 con 0x7f225c15cf60 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f22641ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c158880 con 0x7f225c15f490 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f22641ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c0a5b50 con 0x7f225c15f860 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226279c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f225c15cf60 0x7f225c15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226279c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f225c15cf60 0x7f225c15d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41480/0 (socket says 172.21.15.131:41480) 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226279c700 1 -- 172.21.15.131:0/1027562701 learned_addr learned my addr 172.21.15.131:0/1027562701 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f2261f9b700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f225c15f490 0x7f225c14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226179a700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c15f860 0x7f225c14ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226279c700 1 -- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c15f860 msgr2=0x7f225c14ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226279c700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c15f860 0x7f225c14ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226279c700 1 -- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f225c15f490 msgr2=0x7f225c14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226279c700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f225c15f490 0x7f225c14a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226279c700 1 -- 172.21.15.131:0/1027562701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c0a5540 con 0x7f225c15cf60 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f226179a700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c15f860 0x7f225c14ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:57.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.438+0000 7f2261f9b700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f225c15f490 0x7f225c14a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:57.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.439+0000 7f226279c700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f225c15cf60 0x7f225c15d3c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f225400afd0 tx=0x7f225401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:57.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.439+0000 7f2251ffb700 1 -- 172.21.15.131:0/1027562701 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225401e640 con 0x7f225c15cf60 2023-12-26T21:52:57.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.439+0000 7f2251ffb700 1 -- 172.21.15.131:0/1027562701 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2254022070 con 0x7f225c15cf60 2023-12-26T21:52:57.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.439+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c15dbe0 con 0x7f225c15cf60 2023-12-26T21:52:57.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.439+0000 7f2251ffb700 1 -- 172.21.15.131:0/1027562701 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225401ea30 con 0x7f225c15cf60 2023-12-26T21:52:57.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.439+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225c15d900 con 0x7f225c15cf60 2023-12-26T21:52:57.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.440+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225c0a5ce0 con 0x7f225c15cf60 2023-12-26T21:52:57.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.442+0000 7f2251ffb700 1 -- 172.21.15.131:0/1027562701 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2254003040 con 0x7f225c15cf60 2023-12-26T21:52:57.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.442+0000 7f2251ffb700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:57.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.442+0000 7f2251ffb700 1 -- 172.21.15.131:0/1027562701 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f2254024070 con 0x7f225c15cf60 2023-12-26T21:52:57.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.443+0000 7f2261f9b700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:57.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.443+0000 7f2261f9b700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2240066f30 0x7f22400693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2258007a80 tx=0x7f2258009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:57.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.446+0000 7f2251ffb700 1 -- 172.21.15.131:0/1027562701 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2254053cd0 con 0x7f225c15cf60 2023-12-26T21:52:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:57.714+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f225c0657e0 con 0x7f225c15cf60 2023-12-26T21:52:58.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.222+0000 7f2251ffb700 1 -- 172.21.15.131:0/1027562701 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v593) v1 ==== 169+0+0 (secure 0 0 0) 0x7f225401ee60 con 0x7f225c15cf60 2023-12-26T21:52:58.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2240066f30 msgr2=0x7f22400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:58.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2240066f30 0x7f22400693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2258007a80 tx=0x7f2258009040 comp rx=0 tx=0).stop 2023-12-26T21:52:58.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f225c15cf60 msgr2=0x7f225c15d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:58.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f225c15cf60 0x7f225c15d3c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f225400afd0 tx=0x7f225401a000 comp rx=0 tx=0).stop 2023-12-26T21:52:58.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 shutdown_connections 2023-12-26T21:52:58.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f225c15cf60 0x7f225c15d3c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f225c15f490 0x7f225c14a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 --2- 172.21.15.131:0/1027562701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f225c15f860 0x7f225c14ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.224+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 >> 172.21.15.131:0/1027562701 conn(0x7f225c068040 msgr2=0x7f225c0aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:58.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.225+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 shutdown_connections 2023-12-26T21:52:58.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.225+0000 7f22641ff700 1 -- 172.21.15.131:0/1027562701 wait complete. 2023-12-26T21:52:58.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T21:52:58.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.437+0000 7f865f3ff700 1 Processor -- start 2023-12-26T21:52:58.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.437+0000 7f865f3ff700 1 -- start start 2023-12-26T21:52:58.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.438+0000 7f865f3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86581560e0 0x7f86581564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:58.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.438+0000 7f865f3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86580b0a10 con 0x7f865806a690 2023-12-26T21:52:58.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.438+0000 7f865f3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86580ce030 con 0x7f865806efa0 2023-12-26T21:52:58.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.438+0000 7f865f3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86580b00e0 con 0x7f86581560e0 2023-12-26T21:52:58.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f865d99c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f865806a690 0x7f86580b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48950/0 (socket says 172.21.15.131:48950) 2023-12-26T21:52:58.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f865d99c700 1 -- 172.21.15.131:0/2827938496 learned_addr learned my addr 172.21.15.131:0/2827938496 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f865d19b700 1 --2- 172.21.15.131:0/2827938496 >> v2:172.21.15.190:3300/0 conn(0x7f86581560e0 0x7f86581564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f864ffff700 1 -- 172.21.15.131:0/2827938496 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262582744 0 0) 0x7f86580b0a10 con 0x7f865806a690 2023-12-26T21:52:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f864ffff700 1 -- 172.21.15.131:0/2827938496 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8644003570 con 0x7f865806a690 2023-12-26T21:52:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f864ffff700 1 -- 172.21.15.131:0/2827938496 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790011838 0 0) 0x7f86580ce030 con 0x7f865806efa0 2023-12-26T21:52:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f864ffff700 1 -- 172.21.15.131:0/2827938496 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86580b0a10 con 0x7f865806efa0 2023-12-26T21:52:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f865d19b700 1 -- 172.21.15.131:0/2827938496 >> v1:172.21.15.190:6789/0 conn(0x7f865806efa0 legacy=0x7f86580cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f864ffff700 1 -- 172.21.15.131:0/2827938496 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3396273169 0 0) 0x7f8644003570 con 0x7f865806a690 2023-12-26T21:52:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f865d19b700 1 -- 172.21.15.131:0/2827938496 >> v1:172.21.15.131:6789/0 conn(0x7f865806a690 legacy=0x7f86580b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:58.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.439+0000 7f865d19b700 1 -- 172.21.15.131:0/2827938496 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86580b03c0 con 0x7f86581560e0 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.440+0000 7f865d19b700 1 --2- 172.21.15.131:0/2827938496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86581560e0 0x7f86581564b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f86540098e0 tx=0x7f86540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b65732a7542b999 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.440+0000 7f864ffff700 1 -- 172.21.15.131:0/2827938496 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86500017f0 con 0x7f86581560e0 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.440+0000 7f864ffff700 1 -- 172.21.15.131:0/2827938496 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654011420 con 0x7f86581560e0 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.440+0000 7f864ffff700 1 -- 172.21.15.131:0/2827938496 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540117c0 con 0x7f86581560e0 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.440+0000 7f865f3ff700 1 -- 172.21.15.131:0/2827938496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86581560e0 msgr2=0x7f86581564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.440+0000 7f865f3ff700 1 --2- 172.21.15.131:0/2827938496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86581560e0 0x7f86581564b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f86540098e0 tx=0x7f86540091a0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.440+0000 7f865f3ff700 1 -- 172.21.15.131:0/2827938496 shutdown_connections 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.440+0000 7f865f3ff700 1 --2- 172.21.15.131:0/2827938496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86581560e0 0x7f86581564b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.440+0000 7f865f3ff700 1 -- 172.21.15.131:0/2827938496 >> 172.21.15.131:0/2827938496 conn(0x7f86580686c0 msgr2=0x7f8658068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.441+0000 7f865f3ff700 1 -- 172.21.15.131:0/2827938496 shutdown_connections 2023-12-26T21:52:58.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.441+0000 7f865f3ff700 1 -- 172.21.15.131:0/2827938496 wait complete. 2023-12-26T21:52:58.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.442+0000 7f865f3ff700 1 Processor -- start 2023-12-26T21:52:58.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.442+0000 7f865f3ff700 1 -- start start 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.443+0000 7f865f3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865806a690 0x7f865814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.443+0000 7f865f3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f865806efa0 0x7f865814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.443+0000 7f865f3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581560e0 0x7f86581549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.443+0000 7f865f3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86580ab350 con 0x7f865806a690 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.443+0000 7f865f3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865814e2b0 con 0x7f865806efa0 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.443+0000 7f865f3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86580aaec0 con 0x7f86581560e0 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.443+0000 7f865d19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865806a690 0x7f865814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.443+0000 7f865d19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865806a690 0x7f865814d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41484/0 (socket says 172.21.15.131:41484) 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.443+0000 7f865d19b700 1 -- 172.21.15.131:0/2996590949 learned_addr learned my addr 172.21.15.131:0/2996590949 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865d99c700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581560e0 0x7f86581549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865c99a700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f865806efa0 0x7f865814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865d19b700 1 -- 172.21.15.131:0/2996590949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581560e0 msgr2=0x7f86581549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865d19b700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581560e0 0x7f86581549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865d19b700 1 -- 172.21.15.131:0/2996590949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f865806efa0 msgr2=0x7f865814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865d19b700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f865806efa0 0x7f865814dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865d19b700 1 -- 172.21.15.131:0/2996590949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654009420 con 0x7f865806a690 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865c99a700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f865806efa0 0x7f865814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865d19b700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865806a690 0x7f865814d830 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f86580642e0 tx=0x7f8654016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f864d7fa700 1 -- 172.21.15.131:0/2996590949 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865401cc90 con 0x7f865806a690 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f864d7fa700 1 -- 172.21.15.131:0/2996590949 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865401cdf0 con 0x7f865806a690 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86580aa2f0 con 0x7f865806a690 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f864d7fa700 1 -- 172.21.15.131:0/2996590949 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865401a6a0 con 0x7f865806a690 2023-12-26T21:52:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.444+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86580aa870 con 0x7f865806a690 2023-12-26T21:52:58.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.445+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86580657e0 con 0x7f865806a690 2023-12-26T21:52:58.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.446+0000 7f864d7fa700 1 -- 172.21.15.131:0/2996590949 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f865401aa50 con 0x7f865806a690 2023-12-26T21:52:58.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.447+0000 7f864d7fa700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f863c066f30 0x7f863c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:58.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.448+0000 7f864d7fa700 1 -- 172.21.15.131:0/2996590949 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f8654085ff0 con 0x7f865806a690 2023-12-26T21:52:58.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.448+0000 7f865c99a700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f863c066f30 0x7f863c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:58.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.448+0000 7f865c99a700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f863c066f30 0x7f863c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86580abed0 tx=0x7f8648009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:58.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.452+0000 7f864d7fa700 1 -- 172.21.15.131:0/2996590949 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86540b75a0 con 0x7f865806a690 2023-12-26T21:52:58.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.644+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f865806de40 con 0x7f865806a690 2023-12-26T21:52:58.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.648+0000 7f864d7fa700 1 -- 172.21.15.131:0/2996590949 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8654053120 con 0x7f865806a690 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f863c066f30 msgr2=0x7f863c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f863c066f30 0x7f863c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86580abed0 tx=0x7f8648009040 comp rx=0 tx=0).stop 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865806a690 msgr2=0x7f865814d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865806a690 0x7f865814d830 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f86580642e0 tx=0x7f8654016000 comp rx=0 tx=0).stop 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 shutdown_connections 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86581560e0 0x7f86581549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f863c066f30 0x7f863c0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865806a690 0x7f865814d830 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 --2- 172.21.15.131:0/2996590949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f865806efa0 0x7f865814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 >> 172.21.15.131:0/2996590949 conn(0x7f86580686c0 msgr2=0x7f86580ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 shutdown_connections 2023-12-26T21:52:58.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:58.651+0000 7f865f3ff700 1 -- 172.21.15.131:0/2996590949 wait complete. 2023-12-26T21:52:58.670 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:52:58.670 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:52:58.753 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:52:58.845 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:52:58.845 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T21:52:58.845 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:52:59.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.034+0000 7f371c1ff700 1 Processor -- start 2023-12-26T21:52:59.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.035+0000 7f371c1ff700 1 -- start start 2023-12-26T21:52:59.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f371c1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37141596a0 0x7f37140b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f371c1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f37140b0870 0x7f37140b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f371c1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37140afcd0 con 0x7f3714159a70 2023-12-26T21:52:59.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f371c1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37140b3e70 con 0x7f37141596a0 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f371c1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37140b2e00 con 0x7f37140b0870 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f3719f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37141596a0 0x7f37140b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f3719f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37141596a0 0x7f37140b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41490/0 (socket says 172.21.15.131:41490) 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f3719f9b700 1 -- 172.21.15.131:0/2215249022 learned_addr learned my addr 172.21.15.131:0/2215249022 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f3719f9b700 1 -- 172.21.15.131:0/2215249022 >> v2:172.21.15.190:3300/0 conn(0x7f37140b0870 msgr2=0x7f37140b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f371a79c700 1 --2- 172.21.15.131:0/2215249022 >> v2:172.21.15.190:3300/0 conn(0x7f37140b0870 0x7f37140b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f3719f9b700 1 --2- 172.21.15.131:0/2215249022 >> v2:172.21.15.190:3300/0 conn(0x7f37140b0870 0x7f37140b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.036+0000 7f3719f9b700 1 -- 172.21.15.131:0/2215249022 >> v1:172.21.15.190:6789/0 conn(0x7f3714159a70 legacy=0x7f37140b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.037+0000 7f3719f9b700 1 -- 172.21.15.131:0/2215249022 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140b0cb0 con 0x7f37141596a0 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.037+0000 7f371a79c700 1 --2- 172.21.15.131:0/2215249022 >> v2:172.21.15.190:3300/0 conn(0x7f37140b0870 0x7f37140b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.037+0000 7f3719f9b700 1 --2- 172.21.15.131:0/2215249022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37141596a0 0x7f37140b2380 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f37140b66b0 tx=0x7f3710009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cda11efce6369c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.037+0000 7f3718f99700 1 -- 172.21.15.131:0/2215249022 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371000b820 con 0x7f37141596a0 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.037+0000 7f3718f99700 1 -- 172.21.15.131:0/2215249022 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371000b980 con 0x7f37141596a0 2023-12-26T21:52:59.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.037+0000 7f3718f99700 1 -- 172.21.15.131:0/2215249022 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371000bd20 con 0x7f37141596a0 2023-12-26T21:52:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.038+0000 7f371c1ff700 1 -- 172.21.15.131:0/2215249022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37141596a0 msgr2=0x7f37140b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.038+0000 7f371c1ff700 1 --2- 172.21.15.131:0/2215249022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37141596a0 0x7f37140b2380 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f37140b66b0 tx=0x7f3710009250 comp rx=0 tx=0).stop 2023-12-26T21:52:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.038+0000 7f371c1ff700 1 -- 172.21.15.131:0/2215249022 shutdown_connections 2023-12-26T21:52:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.038+0000 7f371c1ff700 1 --2- 172.21.15.131:0/2215249022 >> v2:172.21.15.190:3300/0 conn(0x7f37140b0870 0x7f37140b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.038+0000 7f371c1ff700 1 --2- 172.21.15.131:0/2215249022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37141596a0 0x7f37140b2380 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.038+0000 7f371c1ff700 1 -- 172.21.15.131:0/2215249022 >> 172.21.15.131:0/2215249022 conn(0x7f371406ece0 msgr2=0x7f371406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.038+0000 7f371c1ff700 1 -- 172.21.15.131:0/2215249022 shutdown_connections 2023-12-26T21:52:59.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.038+0000 7f371c1ff700 1 -- 172.21.15.131:0/2215249022 wait complete. 2023-12-26T21:52:59.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.039+0000 7f371c1ff700 1 Processor -- start 2023-12-26T21:52:59.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.039+0000 7f371c1ff700 1 -- start start 2023-12-26T21:52:59.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.040+0000 7f371c1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37140b0870 0x7f3714169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.040+0000 7f371c1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37141596a0 0x7f371415a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.040+0000 7f371c1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3714159a70 0x7f37140c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.040+0000 7f371c1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37140c5c60 con 0x7f3714159a70 2023-12-26T21:52:59.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.040+0000 7f371c1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371416a170 con 0x7f37140b0870 2023-12-26T21:52:59.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.040+0000 7f371c1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3714159f90 con 0x7f37141596a0 2023-12-26T21:52:59.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371a79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3714159a70 0x7f37140c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f3719f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37140b0870 0x7f3714169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371a79c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3714159a70 0x7f37140c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41496/0 (socket says 172.21.15.131:41496) 2023-12-26T21:52:59.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371a79c700 1 -- 172.21.15.131:0/932751488 learned_addr learned my addr 172.21.15.131:0/932751488 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:59.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371979a700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37141596a0 0x7f371415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371a79c700 1 -- 172.21.15.131:0/932751488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37141596a0 msgr2=0x7f371415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:59.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371a79c700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37141596a0 0x7f371415a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371a79c700 1 -- 172.21.15.131:0/932751488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37140b0870 msgr2=0x7f3714169c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:59.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371a79c700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37140b0870 0x7f3714169c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371a79c700 1 -- 172.21.15.131:0/932751488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37100094f0 con 0x7f3714159a70 2023-12-26T21:52:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.041+0000 7f371a79c700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3714159a70 0x7f37140c4010 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f370c00cef0 tx=0x7f370c007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.042+0000 7f3709ffb700 1 -- 172.21.15.131:0/932751488 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c00ad20 con 0x7f3714159a70 2023-12-26T21:52:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.042+0000 7f3709ffb700 1 -- 172.21.15.131:0/932751488 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c00ae80 con 0x7f3714159a70 2023-12-26T21:52:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.042+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371415a270 con 0x7f3714159a70 2023-12-26T21:52:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.042+0000 7f3709ffb700 1 -- 172.21.15.131:0/932751488 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c016670 con 0x7f3714159a70 2023-12-26T21:52:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.042+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37140c5880 con 0x7f3714159a70 2023-12-26T21:52:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.043+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37140c5ed0 con 0x7f3714159a70 2023-12-26T21:52:59.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.044+0000 7f3709ffb700 1 -- 172.21.15.131:0/932751488 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f370c016aa0 con 0x7f3714159a70 2023-12-26T21:52:59.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.045+0000 7f3709ffb700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36f8066f30 0x7f36f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.045+0000 7f3709ffb700 1 -- 172.21.15.131:0/932751488 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f370c012070 con 0x7f3714159a70 2023-12-26T21:52:59.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.045+0000 7f3719f9b700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36f8066f30 0x7f36f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.045+0000 7f3719f9b700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36f8066f30 0x7f36f80693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f37140b66b0 tx=0x7f3710016000 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:59.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.049+0000 7f3709ffb700 1 -- 172.21.15.131:0/932751488 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f370c04e5a0 con 0x7f3714159a70 2023-12-26T21:52:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.288+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37140c56d0 con 0x7f3714159a70 2023-12-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.288+0000 7f3709ffb700 1 -- 172.21.15.131:0/932751488 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f370c051f50 con 0x7f3714159a70 2023-12-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/625115906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":625115906},{"type":"v1","addr":"172.21.15.190:6833","nonce":625115906}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2773111229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2773111229},{"type":"v1","addr":"172.21.15.190:6835","nonce":2773111229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/899613179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":899613179},{"type":"v1","addr":"172.21.15.131:6834","nonce":899613179}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6582,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2948404662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2948404662},{"type":"v1","addr":"172.21.15.131:6835","nonce":2948404662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[]} 2023-12-26T21:52:59.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36f8066f30 msgr2=0x7f36f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:59.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36f8066f30 0x7f36f80693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f37140b66b0 tx=0x7f3710016000 comp rx=0 tx=0).stop 2023-12-26T21:52:59.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3714159a70 msgr2=0x7f37140c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3714159a70 0x7f37140c4010 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f370c00cef0 tx=0x7f370c007860 comp rx=0 tx=0).stop 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 shutdown_connections 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36f8066f30 0x7f36f80693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3714159a70 0x7f37140c4010 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37140b0870 0x7f3714169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 --2- 172.21.15.131:0/932751488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37141596a0 0x7f371415a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 >> 172.21.15.131:0/932751488 conn(0x7f371406ece0 msgr2=0x7f37140ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 shutdown_connections 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.291+0000 7f371c1ff700 1 -- 172.21.15.131:0/932751488 wait complete. 2023-12-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 36 2023-12-26T21:52:59.309 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T21:52:59.309 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T21:52:59.309 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T21:52:59.310 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:52:59.310 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T21:52:59.310 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T21:52:59.310 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T21:52:59.310 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:52:59.310 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T21:52:59.310 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T21:52:59.311 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T21:52:59.311 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:52:59.311 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T21:52:59.311 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T21:52:59.311 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T21:52:59.311 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:52:59.323 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T21:52:59.323 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T21:52:59.324 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T21:52:59.324 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T21:52:59.326 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T21:52:59.327 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T21:52:59.327 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T21:52:59.327 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T21:52:59.368 INFO:tasks.ceph.mds.a:Started 2023-12-26T21:52:59.368 DEBUG:teuthology.parallel:result is None 2023-12-26T21:52:59.370 INFO:tasks.ceph.mds.c:Started 2023-12-26T21:52:59.370 DEBUG:teuthology.parallel:result is None 2023-12-26T21:52:59.372 INFO:tasks.ceph.mds.d:Started 2023-12-26T21:52:59.372 DEBUG:teuthology.parallel:result is None 2023-12-26T21:52:59.375 INFO:tasks.ceph.mds.b:Started 2023-12-26T21:52:59.375 DEBUG:teuthology.parallel:result is None 2023-12-26T21:52:59.377 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T21:52:59.512 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T21:52:59.514 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:52:59.517 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T21:52:59.517 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:52:59.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.582+0000 7f68643ff700 1 Processor -- start 2023-12-26T21:52:59.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.582+0000 7f68643ff700 1 -- start start 2023-12-26T21:52:59.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.583+0000 7f68643ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f685c0cc950 0x7f685c06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.583+0000 7f68643ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c06eb30 0x7f685c15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f68643ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c0b07f0 con 0x7f685c0cc580 2023-12-26T21:52:59.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f68643ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0680f0 con 0x7f685c0cc950 2023-12-26T21:52:59.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f68643ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0b0d70 con 0x7f685c06eb30 2023-12-26T21:52:59.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f686219b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f685c0cc580 0x7f685c0b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48968/0 (socket says 172.21.15.131:48968) 2023-12-26T21:52:59.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f686219b700 1 -- 172.21.15.131:0/2355206988 learned_addr learned my addr 172.21.15.131:0/2355206988 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:52:59.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f686199a700 1 --2- 172.21.15.131:0/2355206988 >> v2:172.21.15.190:3300/0 conn(0x7f685c0cc950 0x7f685c06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f686299c700 1 --2- 172.21.15.131:0/2355206988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c06eb30 0x7f685c15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f6861199700 1 -- 172.21.15.131:0/2355206988 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467708197 0 0) 0x7f6858001110 con 0x7f685c0cc580 2023-12-26T21:52:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f6861199700 1 -- 172.21.15.131:0/2355206988 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6848003570 con 0x7f685c0cc580 2023-12-26T21:52:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f686299c700 1 -- 172.21.15.131:0/2355206988 >> v2:172.21.15.190:3300/0 conn(0x7f685c0cc950 msgr2=0x7f685c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:52:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f686299c700 1 --2- 172.21.15.131:0/2355206988 >> v2:172.21.15.190:3300/0 conn(0x7f685c0cc950 0x7f685c06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.584+0000 7f686299c700 1 -- 172.21.15.131:0/2355206988 >> v1:172.21.15.131:6789/0 conn(0x7f685c0cc580 legacy=0x7f685c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.585+0000 7f686299c700 1 -- 172.21.15.131:0/2355206988 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0675b0 con 0x7f685c06eb30 2023-12-26T21:52:59.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.585+0000 7f686199a700 1 --2- 172.21.15.131:0/2355206988 >> v2:172.21.15.190:3300/0 conn(0x7f685c0cc950 0x7f685c06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.585+0000 7f686299c700 1 --2- 172.21.15.131:0/2355206988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c06eb30 0x7f685c15d9b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6854009e60 tx=0x7f6854009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d544e9d3ecf1a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.585+0000 7f6861199700 1 -- 172.21.15.131:0/2355206988 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400e070 con 0x7f685c06eb30 2023-12-26T21:52:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.585+0000 7f6861199700 1 -- 172.21.15.131:0/2355206988 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685400bab0 con 0x7f685c06eb30 2023-12-26T21:52:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.585+0000 7f6861199700 1 -- 172.21.15.131:0/2355206988 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400be70 con 0x7f685c06eb30 2023-12-26T21:52:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.586+0000 7f68643ff700 1 -- 172.21.15.131:0/2355206988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c06eb30 msgr2=0x7f685c15d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.586+0000 7f68643ff700 1 --2- 172.21.15.131:0/2355206988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c06eb30 0x7f685c15d9b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6854009e60 tx=0x7f6854009780 comp rx=0 tx=0).stop 2023-12-26T21:52:59.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.586+0000 7f68643ff700 1 -- 172.21.15.131:0/2355206988 shutdown_connections 2023-12-26T21:52:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.586+0000 7f68643ff700 1 --2- 172.21.15.131:0/2355206988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c06eb30 0x7f685c15d9b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.586+0000 7f68643ff700 1 --2- 172.21.15.131:0/2355206988 >> v2:172.21.15.190:3300/0 conn(0x7f685c0cc950 0x7f685c06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.586+0000 7f68643ff700 1 -- 172.21.15.131:0/2355206988 >> 172.21.15.131:0/2355206988 conn(0x7f685c0686c0 msgr2=0x7f685c068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:52:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.586+0000 7f68643ff700 1 -- 172.21.15.131:0/2355206988 shutdown_connections 2023-12-26T21:52:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.586+0000 7f68643ff700 1 -- 172.21.15.131:0/2355206988 wait complete. 2023-12-26T21:52:59.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.587+0000 7f68643ff700 1 Processor -- start 2023-12-26T21:52:59.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.587+0000 7f68643ff700 1 -- start start 2023-12-26T21:52:59.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.588+0000 7f68643ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c06eb30 0x7f685c145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.588+0000 7f68643ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c0cc580 0x7f685c0ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.588+0000 7f68643ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0cc950 0x7f685c0c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.588+0000 7f68643ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c1460a0 con 0x7f685c0cc950 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.588+0000 7f68643ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c2480 con 0x7f685c06eb30 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.588+0000 7f68643ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c2760 con 0x7f685c0cc580 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.588+0000 7f686219b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c06eb30 0x7f685c145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.588+0000 7f686199a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c0cc580 0x7f685c0ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.588+0000 7f686299c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0cc950 0x7f685c0c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686199a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c0cc580 0x7f685c0ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:48620/0 (socket says 172.21.15.131:48620) 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686199a700 1 -- 172.21.15.131:0/1486220206 learned_addr learned my addr 172.21.15.131:0/1486220206 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686199a700 1 -- 172.21.15.131:0/1486220206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c06eb30 msgr2=0x7f685c145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686199a700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c06eb30 0x7f685c145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686199a700 1 -- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0cc950 msgr2=0x7f685c0c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686199a700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0cc950 0x7f685c0c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:52:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686199a700 1 -- 172.21.15.131:0/1486220206 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c009420 con 0x7f685c0cc580 2023-12-26T21:52:59.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686219b700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c06eb30 0x7f685c145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:59.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686299c700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0cc950 0x7f685c0c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:52:59.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f686199a700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c0cc580 0x7f685c0ae0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f684c00ec80 tx=0x7f684c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:59.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.589+0000 7f68527fc700 1 -- 172.21.15.131:0/1486220206 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c00c920 con 0x7f685c0cc580 2023-12-26T21:52:59.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.590+0000 7f68527fc700 1 -- 172.21.15.131:0/1486220206 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c00ca80 con 0x7f685c0cc580 2023-12-26T21:52:59.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.590+0000 7f68527fc700 1 -- 172.21.15.131:0/1486220206 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c00ce00 con 0x7f685c0cc580 2023-12-26T21:52:59.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.590+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6854009a20 con 0x7f685c0cc580 2023-12-26T21:52:59.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.590+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0c2d20 con 0x7f685c0cc580 2023-12-26T21:52:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.591+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c0657e0 con 0x7f685c0cc580 2023-12-26T21:52:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.591+0000 7f68527fc700 1 -- 172.21.15.131:0/1486220206 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f684c013680 con 0x7f685c0cc580 2023-12-26T21:52:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.592+0000 7f68527fc700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6840066f30 0x7f68400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:52:59.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.592+0000 7f68527fc700 1 -- 172.21.15.131:0/1486220206 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f684c084a20 con 0x7f685c0cc580 2023-12-26T21:52:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.593+0000 7f686219b700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6840066f30 0x7f68400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:52:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.593+0000 7f686219b700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6840066f30 0x7f68400693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6858001270 tx=0x7f6858007bf0 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:52:59.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.597+0000 7f68527fc700 1 -- 172.21.15.131:0/1486220206 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f684c051b30 con 0x7f685c0cc580 2023-12-26T21:52:59.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:52:59.815+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f685c0c28a0 con 0x7f685c0cc580 2023-12-26T21:53:00.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.678+0000 7f68527fc700 1 -- 172.21.15.131:0/1486220206 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7f685c0c28a0 con 0x7f685c0cc580 2023-12-26T21:53:00.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6840066f30 msgr2=0x7f68400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6840066f30 0x7f68400693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6858001270 tx=0x7f6858007bf0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c0cc580 msgr2=0x7f685c0ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c0cc580 0x7f685c0ae0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f684c00ec80 tx=0x7f684c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 shutdown_connections 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6840066f30 0x7f68400693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0cc950 0x7f685c0c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c06eb30 0x7f685c145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 --2- 172.21.15.131:0/1486220206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c0cc580 0x7f685c0ae0c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 >> 172.21.15.131:0/1486220206 conn(0x7f685c0686c0 msgr2=0x7f685c0b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 shutdown_connections 2023-12-26T21:53:00.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.681+0000 7f68643ff700 1 -- 172.21.15.131:0/1486220206 wait complete. 2023-12-26T21:53:00.683 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T21:53:00.703 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T21:53:00.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.957+0000 7f98581ff700 1 Processor -- start 2023-12-26T21:53:00.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.957+0000 7f98581ff700 1 -- start start 2023-12-26T21:53:00.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.958+0000 7f98581ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 0x7f9850158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:00.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.958+0000 7f98581ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f98500c9130 0x7f98500cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:00.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f98581ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985015ba70 con 0x7f985006b8c0 2023-12-26T21:53:00.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f98581ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985015bf00 con 0x7f98500c9130 2023-12-26T21:53:00.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f98581ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985006c2a0 con 0x7f9850158030 2023-12-26T21:53:00.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f985679c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f98500c9130 0x7f98500cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:00.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f9855f9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 0x7f9850158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:00.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f985679c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f98500c9130 0x7f98500cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:55664/0 (socket says 172.21.15.131:55664) 2023-12-26T21:53:00.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f985679c700 1 -- 172.21.15.131:0/111040744 learned_addr learned my addr 172.21.15.131:0/111040744 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:00.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f985679c700 1 -- 172.21.15.131:0/111040744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 msgr2=0x7f9850158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:00.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f985679c700 1 --2- 172.21.15.131:0/111040744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 0x7f9850158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.959+0000 7f985679c700 1 -- 172.21.15.131:0/111040744 >> v1:172.21.15.190:6789/0 conn(0x7f985006b8c0 legacy=0x7f98500c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.960+0000 7f985679c700 1 -- 172.21.15.131:0/111040744 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985015b140 con 0x7f98500c9130 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.960+0000 7f985679c700 1 --2- 172.21.15.131:0/111040744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98500c9130 0x7f98500cbfb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9848009f20 tx=0x7f9848009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9f5e32e948b519 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.960+0000 7f9854f99700 1 -- 172.21.15.131:0/111040744 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984800e070 con 0x7f98500c9130 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.960+0000 7f9854f99700 1 -- 172.21.15.131:0/111040744 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984800b9d0 con 0x7f98500c9130 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.960+0000 7f9854f99700 1 -- 172.21.15.131:0/111040744 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480092e0 con 0x7f98500c9130 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.961+0000 7f98581ff700 1 -- 172.21.15.131:0/111040744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98500c9130 msgr2=0x7f98500cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.961+0000 7f98581ff700 1 --2- 172.21.15.131:0/111040744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98500c9130 0x7f98500cbfb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9848009f20 tx=0x7f9848009840 comp rx=0 tx=0).stop 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.961+0000 7f98581ff700 1 -- 172.21.15.131:0/111040744 shutdown_connections 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.961+0000 7f98581ff700 1 --2- 172.21.15.131:0/111040744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98500c9130 0x7f98500cbfb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.961+0000 7f98581ff700 1 --2- 172.21.15.131:0/111040744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 0x7f9850158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.961+0000 7f98581ff700 1 -- 172.21.15.131:0/111040744 >> 172.21.15.131:0/111040744 conn(0x7f98500ce3e0 msgr2=0x7f98500ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.961+0000 7f98581ff700 1 -- 172.21.15.131:0/111040744 shutdown_connections 2023-12-26T21:53:00.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.961+0000 7f98581ff700 1 -- 172.21.15.131:0/111040744 wait complete. 2023-12-26T21:53:00.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.962+0000 7f98581ff700 1 Processor -- start 2023-12-26T21:53:00.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.962+0000 7f98581ff700 1 -- start start 2023-12-26T21:53:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.963+0000 7f98581ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f985006b8c0 0x7f98500b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.963+0000 7f98581ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98500c9130 0x7f98500c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.963+0000 7f98581ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 0x7f9850168470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.963+0000 7f98581ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501689b0 con 0x7f985006b8c0 2023-12-26T21:53:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.963+0000 7f98581ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98500c7130 con 0x7f98500c9130 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.963+0000 7f98581ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98500c5b50 con 0x7f9850158030 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9855f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f985006b8c0 0x7f98500b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9855f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f985006b8c0 0x7f98500b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41564/0 (socket says 172.21.15.131:41564) 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9855f9b700 1 -- 172.21.15.131:0/763510818 learned_addr learned my addr 172.21.15.131:0/763510818 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f985579a700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98500c9130 0x7f98500c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f985679c700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 0x7f9850168470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9855f9b700 1 -- 172.21.15.131:0/763510818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 msgr2=0x7f9850168470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9855f9b700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 0x7f9850168470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9855f9b700 1 -- 172.21.15.131:0/763510818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98500c9130 msgr2=0x7f98500c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9855f9b700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98500c9130 0x7f98500c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9855f9b700 1 -- 172.21.15.131:0/763510818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9848009ae0 con 0x7f985006b8c0 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9855f9b700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f985006b8c0 0x7f98500b1180 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f984c00ea00 tx=0x7f984c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.964+0000 7f9845ffb700 1 -- 172.21.15.131:0/763510818 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c00c6a0 con 0x7f985006b8c0 2023-12-26T21:53:00.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.965+0000 7f9845ffb700 1 -- 172.21.15.131:0/763510818 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c00c800 con 0x7f985006b8c0 2023-12-26T21:53:00.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.965+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98500c5e30 con 0x7f985006b8c0 2023-12-26T21:53:00.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.965+0000 7f9845ffb700 1 -- 172.21.15.131:0/763510818 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c00cb80 con 0x7f985006b8c0 2023-12-26T21:53:00.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.965+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98500c63f0 con 0x7f985006b8c0 2023-12-26T21:53:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.966+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98500657e0 con 0x7f985006b8c0 2023-12-26T21:53:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.967+0000 7f9845ffb700 1 -- 172.21.15.131:0/763510818 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f984c013470 con 0x7f985006b8c0 2023-12-26T21:53:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.967+0000 7f9845ffb700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9834066f30 0x7f98340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.967+0000 7f9845ffb700 1 -- 172.21.15.131:0/763510818 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f984c084840 con 0x7f985006b8c0 2023-12-26T21:53:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.968+0000 7f985579a700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9834066f30 0x7f98340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.968+0000 7f985579a700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9834066f30 0x7f98340693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f98500c47f0 tx=0x7f9840007ec0 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:00.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:00.972+0000 7f9845ffb700 1 -- 172.21.15.131:0/763510818 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f984c051c80 con 0x7f985006b8c0 2023-12-26T21:53:01.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.242+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f985015b420 con 0x7f985006b8c0 2023-12-26T21:53:01.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.243+0000 7f9845ffb700 1 -- 172.21.15.131:0/763510818 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f984c021090 con 0x7f985006b8c0 2023-12-26T21:53:01.243 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.244 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.245 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.246 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:01.247 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.248 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:01.249 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T21:53:01.250 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:53:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:53:01.254 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T21:53:01.255 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T21:53:01.256 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:53:01.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9834066f30 msgr2=0x7f98340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9834066f30 0x7f98340693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f98500c47f0 tx=0x7f9840007ec0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f985006b8c0 msgr2=0x7f98500b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f985006b8c0 0x7f98500b1180 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f984c00ea00 tx=0x7f984c009120 comp rx=0 tx=0).stop 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 shutdown_connections 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9850158030 0x7f9850168470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9834066f30 0x7f98340693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f985006b8c0 0x7f98500b1180 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 --2- 172.21.15.131:0/763510818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98500c9130 0x7f98500c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 >> 172.21.15.131:0/763510818 conn(0x7f98500ce3e0 msgr2=0x7f98500c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 shutdown_connections 2023-12-26T21:53:01.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.245+0000 7f98581ff700 1 -- 172.21.15.131:0/763510818 wait complete. 2023-12-26T21:53:01.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.454+0000 7f7f577ff700 1 Processor -- start 2023-12-26T21:53:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.454+0000 7f7f577ff700 1 -- start start 2023-12-26T21:53:01.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f577ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7f500cb320 0x7f7f50069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f577ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5006a1e0 0x7f7f500b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f577ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7f500b0c90 0x7f7f500aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f577ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f500c8cc0 con 0x7f7f500cb320 2023-12-26T21:53:01.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f577ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f500ad700 con 0x7f7f500b0c90 2023-12-26T21:53:01.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f577ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f500b1110 con 0x7f7f5006a1e0 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f5559b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7f500cb320 0x7f7f50069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f5559b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7f500cb320 0x7f7f50069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41568/0 (socket says 172.21.15.131:41568) 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f5559b700 1 -- 172.21.15.131:0/3961386548 learned_addr learned my addr 172.21.15.131:0/3961386548 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.455+0000 7f7f54d9a700 1 --2- 172.21.15.131:0/3961386548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5006a1e0 0x7f7f500b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f55d9c700 1 --2- 172.21.15.131:0/3961386548 >> v2:172.21.15.190:3300/0 conn(0x7f7f500b0c90 0x7f7f500aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f5559b700 1 -- 172.21.15.131:0/3961386548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5006a1e0 msgr2=0x7f7f500b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f5559b700 1 --2- 172.21.15.131:0/3961386548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5006a1e0 0x7f7f500b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f5559b700 1 -- 172.21.15.131:0/3961386548 >> v2:172.21.15.190:3300/0 conn(0x7f7f500b0c90 msgr2=0x7f7f500aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f5559b700 1 --2- 172.21.15.131:0/3961386548 >> v2:172.21.15.190:3300/0 conn(0x7f7f500b0c90 0x7f7f500aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f5559b700 1 -- 172.21.15.131:0/3961386548 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f50156940 con 0x7f7f500cb320 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f54d9a700 1 --2- 172.21.15.131:0/3961386548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5006a1e0 0x7f7f500b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f5559b700 1 --2- 172.21.15.131:0/3961386548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f500cb320 0x7f7f50069ca0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7f5006cf60 tx=0x7f7f4c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c86a83246ffec04 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f47fff700 1 -- 172.21.15.131:0/3961386548 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00dc50 con 0x7f7f500cb320 2023-12-26T21:53:01.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f47fff700 1 -- 172.21.15.131:0/3961386548 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c00ddb0 con 0x7f7f500cb320 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.456+0000 7f7f47fff700 1 -- 172.21.15.131:0/3961386548 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0196d0 con 0x7f7f500cb320 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.457+0000 7f7f577ff700 1 -- 172.21.15.131:0/3961386548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f500cb320 msgr2=0x7f7f50069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.457+0000 7f7f577ff700 1 --2- 172.21.15.131:0/3961386548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f500cb320 0x7f7f50069ca0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7f5006cf60 tx=0x7f7f4c00b4f0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.457+0000 7f7f577ff700 1 -- 172.21.15.131:0/3961386548 shutdown_connections 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.457+0000 7f7f577ff700 1 --2- 172.21.15.131:0/3961386548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f500cb320 0x7f7f50069ca0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.457+0000 7f7f577ff700 1 --2- 172.21.15.131:0/3961386548 >> v2:172.21.15.190:3300/0 conn(0x7f7f500b0c90 0x7f7f500aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.457+0000 7f7f577ff700 1 --2- 172.21.15.131:0/3961386548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5006a1e0 0x7f7f500b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.457+0000 7f7f577ff700 1 -- 172.21.15.131:0/3961386548 >> 172.21.15.131:0/3961386548 conn(0x7f7f50068040 msgr2=0x7f7f50069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.457+0000 7f7f577ff700 1 -- 172.21.15.131:0/3961386548 shutdown_connections 2023-12-26T21:53:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.457+0000 7f7f577ff700 1 -- 172.21.15.131:0/3961386548 wait complete. 2023-12-26T21:53:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.458+0000 7f7f577ff700 1 Processor -- start 2023-12-26T21:53:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.459+0000 7f7f577ff700 1 -- start start 2023-12-26T21:53:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.459+0000 7f7f577ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f5006a1e0 0x7f7f5014a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.459+0000 7f7f577ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f500b0c90 0x7f7f5016a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f577ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f500cb320 0x7f7f50169540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f577ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5016a570 con 0x7f7f5006a1e0 2023-12-26T21:53:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f577ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f500c74c0 con 0x7f7f500b0c90 2023-12-26T21:53:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f577ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f50169fb0 con 0x7f7f500cb320 2023-12-26T21:53:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f5559b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f5006a1e0 0x7f7f5014a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f5559b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f5006a1e0 0x7f7f5014a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41578/0 (socket says 172.21.15.131:41578) 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f5559b700 1 -- 172.21.15.131:0/1446962664 learned_addr learned my addr 172.21.15.131:0/1446962664 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f54d9a700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f500b0c90 0x7f7f5016a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f55d9c700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f500cb320 0x7f7f50169540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f5559b700 1 -- 172.21.15.131:0/1446962664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f500cb320 msgr2=0x7f7f50169540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f5559b700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f500cb320 0x7f7f50169540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f5559b700 1 -- 172.21.15.131:0/1446962664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f500b0c90 msgr2=0x7f7f5016a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f5559b700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f500b0c90 0x7f7f5016a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f5559b700 1 -- 172.21.15.131:0/1446962664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c00b790 con 0x7f7f5006a1e0 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f55d9c700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f500cb320 0x7f7f50169540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.460+0000 7f7f5559b700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f5006a1e0 0x7f7f5014a810 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7f500688f0 tx=0x7f7f4c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.461+0000 7f7f457fa700 1 -- 172.21.15.131:0/1446962664 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c010680 con 0x7f7f5006a1e0 2023-12-26T21:53:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.461+0000 7f7f457fa700 1 -- 172.21.15.131:0/1446962664 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c010820 con 0x7f7f5006a1e0 2023-12-26T21:53:01.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.461+0000 7f7f457fa700 1 -- 172.21.15.131:0/1446962664 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c010be0 con 0x7f7f5006a1e0 2023-12-26T21:53:01.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.461+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5016a290 con 0x7f7f5006a1e0 2023-12-26T21:53:01.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.461+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f500c57b0 con 0x7f7f5006a1e0 2023-12-26T21:53:01.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.462+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f500657e0 con 0x7f7f5006a1e0 2023-12-26T21:53:01.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.463+0000 7f7f457fa700 1 -- 172.21.15.131:0/1446962664 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7f4c022410 con 0x7f7f5006a1e0 2023-12-26T21:53:01.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.464+0000 7f7f457fa700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7f34066f30 0x7f7f340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.464+0000 7f7f457fa700 1 -- 172.21.15.131:0/1446962664 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f7f4c014070 con 0x7f7f5006a1e0 2023-12-26T21:53:01.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.464+0000 7f7f54d9a700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7f34066f30 0x7f7f340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.465+0000 7f7f54d9a700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7f34066f30 0x7f7f340693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f40006f00 tx=0x7f7f40008040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:01.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.468+0000 7f7f457fa700 1 -- 172.21.15.131:0/1446962664 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f4c0537e0 con 0x7f7f5006a1e0 2023-12-26T21:53:01.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.666+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f500c59a0 con 0x7f7f5006a1e0 2023-12-26T21:53:01.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.667+0000 7f7f457fa700 1 -- 172.21.15.131:0/1446962664 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+12778 (secure 0 0 0) 0x7f7f4c00b5f0 con 0x7f7f5006a1e0 2023-12-26T21:53:01.667 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:01.667 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":92,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:00.672893+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7f34066f30 msgr2=0x7f7f340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7f34066f30 0x7f7f340693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f40006f00 tx=0x7f7f40008040 comp rx=0 tx=0).stop 2023-12-26T21:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f5006a1e0 msgr2=0x7f7f5014a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f5006a1e0 0x7f7f5014a810 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7f500688f0 tx=0x7f7f4c01b000 comp rx=0 tx=0).stop 2023-12-26T21:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 shutdown_connections 2023-12-26T21:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f500cb320 0x7f7f50169540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7f34066f30 0x7f7f340693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f5006a1e0 0x7f7f5014a810 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 --2- 172.21.15.131:0/1446962664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f500b0c90 0x7f7f5016a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 >> 172.21.15.131:0/1446962664 conn(0x7f7f50068040 msgr2=0x7f7f5014d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 shutdown_connections 2023-12-26T21:53:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.669+0000 7f7f577ff700 1 -- 172.21.15.131:0/1446962664 wait complete. 2023-12-26T21:53:01.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T21:53:01.689 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:53:01.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.880+0000 7fd0631ff700 1 Processor -- start 2023-12-26T21:53:01.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.880+0000 7fd0631ff700 1 -- start start 2023-12-26T21:53:01.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.881+0000 7fd0631ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c13c510 0x7fd05c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.881+0000 7fd0631ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd05c0cc810 0x7fd05c06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.881+0000 7fd0631ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05c06c270 con 0x7fd05c06b8c0 2023-12-26T21:53:01.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.881+0000 7fd0631ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05c0686c0 con 0x7fd05c0cc810 2023-12-26T21:53:01.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.881+0000 7fd0631ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c069520 con 0x7fd05c13c510 2023-12-26T21:53:01.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd060f9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c13c510 0x7fd05c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd06179c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd05c0cc810 0x7fd05c06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd060f9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c13c510 0x7fd05c0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:48656/0 (socket says 172.21.15.131:48656) 2023-12-26T21:53:01.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd06179c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd05c0cc810 0x7fd05c06bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:55706/0 (socket says 172.21.15.131:55706) 2023-12-26T21:53:01.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd060f9b700 1 -- 172.21.15.131:0/1090200205 learned_addr learned my addr 172.21.15.131:0/1090200205 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:01.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd060f9b700 1 -- 172.21.15.131:0/1090200205 >> v2:172.21.15.190:3300/0 conn(0x7fd05c0cc810 msgr2=0x7fd05c06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:01.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd060f9b700 1 --2- 172.21.15.131:0/1090200205 >> v2:172.21.15.190:3300/0 conn(0x7fd05c0cc810 0x7fd05c06bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd060f9b700 1 -- 172.21.15.131:0/1090200205 >> v1:172.21.15.190:6789/0 conn(0x7fd05c06b8c0 legacy=0x7fd05c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd060f9b700 1 -- 172.21.15.131:0/1090200205 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c068910 con 0x7fd05c13c510 2023-12-26T21:53:01.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.882+0000 7fd06179c700 1 --2- 172.21.15.131:0/1090200205 >> v2:172.21.15.190:3300/0 conn(0x7fd05c0cc810 0x7fd05c06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.883+0000 7fd060f9b700 1 --2- 172.21.15.131:0/1090200205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c13c510 0x7fd05c0c8a20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd050009990 tx=0x7fd050009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64f10f10888f072 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.883+0000 7fd05b7fe700 1 -- 172.21.15.131:0/1090200205 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050009e80 con 0x7fd05c13c510 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.883+0000 7fd05b7fe700 1 -- 172.21.15.131:0/1090200205 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd050005c10 con 0x7fd05c13c510 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.883+0000 7fd05b7fe700 1 -- 172.21.15.131:0/1090200205 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05000b650 con 0x7fd05c13c510 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.883+0000 7fd0631ff700 1 -- 172.21.15.131:0/1090200205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c13c510 msgr2=0x7fd05c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.883+0000 7fd0631ff700 1 --2- 172.21.15.131:0/1090200205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c13c510 0x7fd05c0c8a20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd050009990 tx=0x7fd050009220 comp rx=0 tx=0).stop 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.884+0000 7fd0631ff700 1 -- 172.21.15.131:0/1090200205 shutdown_connections 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.884+0000 7fd0631ff700 1 --2- 172.21.15.131:0/1090200205 >> v2:172.21.15.190:3300/0 conn(0x7fd05c0cc810 0x7fd05c06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.884+0000 7fd0631ff700 1 --2- 172.21.15.131:0/1090200205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c13c510 0x7fd05c0c8a20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.884+0000 7fd0631ff700 1 -- 172.21.15.131:0/1090200205 >> 172.21.15.131:0/1090200205 conn(0x7fd05c0ce3e0 msgr2=0x7fd05c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.884+0000 7fd0631ff700 1 -- 172.21.15.131:0/1090200205 shutdown_connections 2023-12-26T21:53:01.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.884+0000 7fd0631ff700 1 -- 172.21.15.131:0/1090200205 wait complete. 2023-12-26T21:53:01.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.886+0000 7fd0631ff700 1 Processor -- start 2023-12-26T21:53:01.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.886+0000 7fd0631ff700 1 -- start start 2023-12-26T21:53:01.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.887+0000 7fd0631ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05c06b8c0 0x7fd05c13b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.887+0000 7fd0631ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c0cc810 0x7fd05c0c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.887+0000 7fd0631ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05c13c510 0x7fd05c0c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.887+0000 7fd0631ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c13c320 con 0x7fd05c06b8c0 2023-12-26T21:53:01.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.887+0000 7fd0631ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c16a7d0 con 0x7fd05c13c510 2023-12-26T21:53:01.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.887+0000 7fd0631ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c169f10 con 0x7fd05c0cc810 2023-12-26T21:53:01.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.887+0000 7fd060f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05c06b8c0 0x7fd05c13b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.887+0000 7fd060f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05c06b8c0 0x7fd05c13b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41588/0 (socket says 172.21.15.131:41588) 2023-12-26T21:53:01.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.887+0000 7fd060f9b700 1 -- 172.21.15.131:0/4263376172 learned_addr learned my addr 172.21.15.131:0/4263376172 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:01.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd06179c700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05c13c510 0x7fd05c0c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd05bfff700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c0cc810 0x7fd05c0c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd060f9b700 1 -- 172.21.15.131:0/4263376172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c0cc810 msgr2=0x7fd05c0c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd060f9b700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c0cc810 0x7fd05c0c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd060f9b700 1 -- 172.21.15.131:0/4263376172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05c13c510 msgr2=0x7fd05c0c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:01.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd060f9b700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05c13c510 0x7fd05c0c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:01.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd060f9b700 1 -- 172.21.15.131:0/4263376172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054009a20 con 0x7fd05c06b8c0 2023-12-26T21:53:01.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd060f9b700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05c06b8c0 0x7fd05c13b500 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd050005fb0 tx=0x7fd050016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:01.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd058ff9700 1 -- 172.21.15.131:0/4263376172 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500062d0 con 0x7fd05c06b8c0 2023-12-26T21:53:01.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd058ff9700 1 -- 172.21.15.131:0/4263376172 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd050020070 con 0x7fd05c06b8c0 2023-12-26T21:53:01.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd050009480 con 0x7fd05c06b8c0 2023-12-26T21:53:01.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.888+0000 7fd058ff9700 1 -- 172.21.15.131:0/4263376172 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050006640 con 0x7fd05c06b8c0 2023-12-26T21:53:01.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.889+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c16a180 con 0x7fd05c06b8c0 2023-12-26T21:53:01.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.890+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c0657e0 con 0x7fd05c06b8c0 2023-12-26T21:53:01.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.890+0000 7fd058ff9700 1 -- 172.21.15.131:0/4263376172 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd050022030 con 0x7fd05c06b8c0 2023-12-26T21:53:01.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.891+0000 7fd058ff9700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd040066f30 0x7fd0400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:01.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.891+0000 7fd058ff9700 1 -- 172.21.15.131:0/4263376172 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fd050085df0 con 0x7fd05c06b8c0 2023-12-26T21:53:01.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.892+0000 7fd05bfff700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd040066f30 0x7fd0400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:01.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.892+0000 7fd05bfff700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd040066f30 0x7fd0400693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd05c16d360 tx=0x7fd048007ec0 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:01.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:01.896+0000 7fd058ff9700 1 -- 172.21.15.131:0/4263376172 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0500532f0 con 0x7fd05c06b8c0 2023-12-26T21:53:02.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.087+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd05c068bf0 con 0x7fd05c06b8c0 2023-12-26T21:53:02.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.087+0000 7fd058ff9700 1 -- 172.21.15.131:0/4263376172 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd050056910 con 0x7fd05c06b8c0 2023-12-26T21:53:02.088 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd040066f30 msgr2=0x7fd0400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd040066f30 0x7fd0400693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd05c16d360 tx=0x7fd048007ec0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05c06b8c0 msgr2=0x7fd05c13b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05c06b8c0 0x7fd05c13b500 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd050005fb0 tx=0x7fd050016000 comp rx=0 tx=0).stop 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 shutdown_connections 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05c13c510 0x7fd05c0c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd040066f30 0x7fd0400693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05c06b8c0 0x7fd05c13b500 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 --2- 172.21.15.131:0/4263376172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c0cc810 0x7fd05c0c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 >> 172.21.15.131:0/4263376172 conn(0x7fd05c0ce3e0 msgr2=0x7fd05c0c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 shutdown_connections 2023-12-26T21:53:02.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.090+0000 7fd0631ff700 1 -- 172.21.15.131:0/4263376172 wait complete. 2023-12-26T21:53:02.109 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T21:53:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.301+0000 7fd728bff700 1 Processor -- start 2023-12-26T21:53:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.301+0000 7fd728bff700 1 -- start start 2023-12-26T21:53:02.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.302+0000 7fd728bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd72413c510 0x7fd7240c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.302+0000 7fd728bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd7240cc810 0x7fd72406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.302+0000 7fd728bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72406c270 con 0x7fd72406b8c0 2023-12-26T21:53:02.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.302+0000 7fd728bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7240686c0 con 0x7fd7240cc810 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.302+0000 7fd728bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd724069520 con 0x7fd72413c510 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.302+0000 7fd722d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd7240cc810 0x7fd72406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.302+0000 7fd722d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd7240cc810 0x7fd72406bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41598/0 (socket says 172.21.15.131:41598) 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.302+0000 7fd722d9d700 1 -- 172.21.15.131:0/3050667735 learned_addr learned my addr 172.21.15.131:0/3050667735 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.303+0000 7fd72259c700 1 --2- 172.21.15.131:0/3050667735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd72413c510 0x7fd7240c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.303+0000 7fd722d9d700 1 -- 172.21.15.131:0/3050667735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd72413c510 msgr2=0x7fd7240c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.303+0000 7fd722d9d700 1 --2- 172.21.15.131:0/3050667735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd72413c510 0x7fd7240c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.303+0000 7fd722d9d700 1 -- 172.21.15.131:0/3050667735 >> v1:172.21.15.190:6789/0 conn(0x7fd72406b8c0 legacy=0x7fd7240cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.303+0000 7fd722d9d700 1 -- 172.21.15.131:0/3050667735 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd724068910 con 0x7fd7240cc810 2023-12-26T21:53:02.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.303+0000 7fd722d9d700 1 --2- 172.21.15.131:0/3050667735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 0x7fd72406bd30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009f20 tx=0x7fd71c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04aebda4c5bbeaf server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:02.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.303+0000 7fd72159a700 1 -- 172.21.15.131:0/3050667735 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c00e070 con 0x7fd7240cc810 2023-12-26T21:53:02.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd72159a700 1 -- 172.21.15.131:0/3050667735 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c00b9d0 con 0x7fd7240cc810 2023-12-26T21:53:02.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd72159a700 1 -- 172.21.15.131:0/3050667735 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0092e0 con 0x7fd7240cc810 2023-12-26T21:53:02.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd728bff700 1 -- 172.21.15.131:0/3050667735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 msgr2=0x7fd72406bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd728bff700 1 --2- 172.21.15.131:0/3050667735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 0x7fd72406bd30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009f20 tx=0x7fd71c009840 comp rx=0 tx=0).stop 2023-12-26T21:53:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd728bff700 1 -- 172.21.15.131:0/3050667735 shutdown_connections 2023-12-26T21:53:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd728bff700 1 --2- 172.21.15.131:0/3050667735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 0x7fd72406bd30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd728bff700 1 --2- 172.21.15.131:0/3050667735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd72413c510 0x7fd7240c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd728bff700 1 -- 172.21.15.131:0/3050667735 >> 172.21.15.131:0/3050667735 conn(0x7fd7240ce3e0 msgr2=0x7fd7240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd728bff700 1 -- 172.21.15.131:0/3050667735 shutdown_connections 2023-12-26T21:53:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.304+0000 7fd728bff700 1 -- 172.21.15.131:0/3050667735 wait complete. 2023-12-26T21:53:02.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.306+0000 7fd728bff700 1 Processor -- start 2023-12-26T21:53:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.306+0000 7fd728bff700 1 -- start start 2023-12-26T21:53:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd728bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd72406b8c0 0x7fd72415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd728bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 0x7fd72413be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd728bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd72413c510 0x7fd7241506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd728bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72413c390 con 0x7fd7240cc810 2023-12-26T21:53:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd728bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b1210 con 0x7fd72413c510 2023-12-26T21:53:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd728bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd724150bf0 con 0x7fd72406b8c0 2023-12-26T21:53:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd721d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 0x7fd72413be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd721d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 0x7fd72413be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41600/0 (socket says 172.21.15.131:41600) 2023-12-26T21:53:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd721d9b700 1 -- 172.21.15.131:0/3275835964 learned_addr learned my addr 172.21.15.131:0/3275835964 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd721d9b700 1 -- 172.21.15.131:0/3275835964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd72406b8c0 msgr2=0x7fd72415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd721d9b700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd72406b8c0 0x7fd72415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd721d9b700 1 -- 172.21.15.131:0/3275835964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd72413c510 msgr2=0x7fd7241506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd721d9b700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd72413c510 0x7fd7241506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.308+0000 7fd721d9b700 1 -- 172.21.15.131:0/3275835964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c009ae0 con 0x7fd7240cc810 2023-12-26T21:53:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.309+0000 7fd721d9b700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 0x7fd72413be50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd71800c510 tx=0x7fd71800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.309+0000 7fd7127fc700 1 -- 172.21.15.131:0/3275835964 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718011590 con 0x7fd7240cc810 2023-12-26T21:53:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.309+0000 7fd7127fc700 1 -- 172.21.15.131:0/3275835964 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd718011710 con 0x7fd7240cc810 2023-12-26T21:53:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.309+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72415e210 con 0x7fd7240cc810 2023-12-26T21:53:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.309+0000 7fd7127fc700 1 -- 172.21.15.131:0/3275835964 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718011ad0 con 0x7fd7240cc810 2023-12-26T21:53:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.309+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72415e7d0 con 0x7fd7240cc810 2023-12-26T21:53:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.310+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7240657e0 con 0x7fd7240cc810 2023-12-26T21:53:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.311+0000 7fd7127fc700 1 -- 172.21.15.131:0/3275835964 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd71801b3b0 con 0x7fd7240cc810 2023-12-26T21:53:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.312+0000 7fd7127fc700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd704066f60 0x7fd704069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.312+0000 7fd7127fc700 1 -- 172.21.15.131:0/3275835964 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fd718083880 con 0x7fd7240cc810 2023-12-26T21:53:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.312+0000 7fd72259c700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd704066f60 0x7fd704069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.312+0000 7fd72259c700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd704066f60 0x7fd704069410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd714009ba0 tx=0x7fd714009120 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.316+0000 7fd7127fc700 1 -- 172.21.15.131:0/3275835964 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd718050c20 con 0x7fd7240cc810 2023-12-26T21:53:02.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.510+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd724068bf0 con 0x7fd7240cc810 2023-12-26T21:53:02.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.510+0000 7fd7127fc700 1 -- 172.21.15.131:0/3275835964 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd718017020 con 0x7fd7240cc810 2023-12-26T21:53:02.511 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:02.511 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:53:02.511 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:02.511 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T21:53:02.511 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T21:53:02.511 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:02.512 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:53:02.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.512+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd704066f60 msgr2=0x7fd704069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.512+0000 7fd728bff700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd704066f60 0x7fd704069410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd714009ba0 tx=0x7fd714009120 comp rx=0 tx=0).stop 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.512+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 msgr2=0x7fd72413be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.512+0000 7fd728bff700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 0x7fd72413be50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd71800c510 tx=0x7fd71800c8d0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.513+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 shutdown_connections 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.513+0000 7fd728bff700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd72413c510 0x7fd7241506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.513+0000 7fd728bff700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd72406b8c0 0x7fd72415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.513+0000 7fd728bff700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd704066f60 0x7fd704069410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.513+0000 7fd728bff700 1 --2- 172.21.15.131:0/3275835964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7240cc810 0x7fd72413be50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.513+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 >> 172.21.15.131:0/3275835964 conn(0x7fd7240ce3e0 msgr2=0x7fd7240c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.513+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 shutdown_connections 2023-12-26T21:53:02.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.513+0000 7fd728bff700 1 -- 172.21.15.131:0/3275835964 wait complete. 2023-12-26T21:53:02.531 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T21:53:02.531 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T21:53:02.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.724+0000 7f44ef5ff700 1 Processor -- start 2023-12-26T21:53:02.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.724+0000 7f44ef5ff700 1 -- start start 2023-12-26T21:53:02.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.725+0000 7f44ef5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e8156360 0x7f44e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.725+0000 7f44ef5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f44e80c8ba0 0x7f44e806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.725+0000 7f44ef5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e80acfc0 con 0x7f44e8156730 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.725+0000 7f44ef5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e806f1f0 con 0x7f44e80c8ba0 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.725+0000 7f44ef5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80c91f0 con 0x7f44e8156360 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.725+0000 7f44edb9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f44e80c8ba0 0x7f44e806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.725+0000 7f44edb9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f44e80c8ba0 0x7f44e806ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41604/0 (socket says 172.21.15.131:41604) 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.725+0000 7f44edb9c700 1 -- 172.21.15.131:0/1788129353 learned_addr learned my addr 172.21.15.131:0/1788129353 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.726+0000 7f44edb9c700 1 -- 172.21.15.131:0/1788129353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e8156360 msgr2=0x7f44e806e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.726+0000 7f44ed39b700 1 --2- 172.21.15.131:0/1788129353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e8156360 0x7f44e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.726+0000 7f44edb9c700 1 --2- 172.21.15.131:0/1788129353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e8156360 0x7f44e806e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.726+0000 7f44edb9c700 1 -- 172.21.15.131:0/1788129353 >> v1:172.21.15.190:6789/0 conn(0x7f44e8156730 legacy=0x7f44e80c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.726+0000 7f44edb9c700 1 -- 172.21.15.131:0/1788129353 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80c94d0 con 0x7f44e80c8ba0 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.726+0000 7f44ed39b700 1 --2- 172.21.15.131:0/1788129353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e8156360 0x7f44e806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:02.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.726+0000 7f44edb9c700 1 --2- 172.21.15.131:0/1788129353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e80c8ba0 0x7f44e806ecb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009f20 tx=0x7f44e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9b446202e150f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:02.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.726+0000 7f44dffff700 1 -- 172.21.15.131:0/1788129353 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e000e070 con 0x7f44e80c8ba0 2023-12-26T21:53:02.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44dffff700 1 -- 172.21.15.131:0/1788129353 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e000ba80 con 0x7f44e80c8ba0 2023-12-26T21:53:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44dffff700 1 -- 172.21.15.131:0/1788129353 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e00092e0 con 0x7f44e80c8ba0 2023-12-26T21:53:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1788129353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e80c8ba0 msgr2=0x7f44e806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44ef5ff700 1 --2- 172.21.15.131:0/1788129353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e80c8ba0 0x7f44e806ecb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009f20 tx=0x7f44e0009840 comp rx=0 tx=0).stop 2023-12-26T21:53:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1788129353 shutdown_connections 2023-12-26T21:53:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44ef5ff700 1 --2- 172.21.15.131:0/1788129353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e80c8ba0 0x7f44e806ecb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44ef5ff700 1 --2- 172.21.15.131:0/1788129353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e8156360 0x7f44e806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1788129353 >> 172.21.15.131:0/1788129353 conn(0x7f44e80675b0 msgr2=0x7f44e80679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1788129353 shutdown_connections 2023-12-26T21:53:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.727+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1788129353 wait complete. 2023-12-26T21:53:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.729+0000 7f44ef5ff700 1 Processor -- start 2023-12-26T21:53:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.729+0000 7f44ef5ff700 1 -- start start 2023-12-26T21:53:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44ef5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e80c8ba0 0x7f44e814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44ef5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44e8156360 0x7f44e816d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44ef5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e8156730 0x7f44e80c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44ef5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e814c910 con 0x7f44e8156730 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44ef5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e816db60 con 0x7f44e8156360 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44ef5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e814e3d0 con 0x7f44e80c8ba0 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44edb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e8156730 0x7f44e80c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44edb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e8156730 0x7f44e80c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41606/0 (socket says 172.21.15.131:41606) 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44edb9c700 1 -- 172.21.15.131:0/1215221215 learned_addr learned my addr 172.21.15.131:0/1215221215 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44ed39b700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e80c8ba0 0x7f44e814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44ecb9a700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44e8156360 0x7f44e816d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44edb9c700 1 -- 172.21.15.131:0/1215221215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e80c8ba0 msgr2=0x7f44e814ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44edb9c700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e80c8ba0 0x7f44e814ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44edb9c700 1 -- 172.21.15.131:0/1215221215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44e8156360 msgr2=0x7f44e816d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44edb9c700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44e8156360 0x7f44e816d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:02.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.730+0000 7f44edb9c700 1 -- 172.21.15.131:0/1215221215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e0009ae0 con 0x7f44e8156730 2023-12-26T21:53:02.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.731+0000 7f44edb9c700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e8156730 0x7f44e80c6b30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f44e80690d0 tx=0x7f44e0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:02.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.731+0000 7f44dd7fa700 1 -- 172.21.15.131:0/1215221215 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0008d20 con 0x7f44e8156730 2023-12-26T21:53:02.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.731+0000 7f44dd7fa700 1 -- 172.21.15.131:0/1215221215 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e0008e80 con 0x7f44e8156730 2023-12-26T21:53:02.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.731+0000 7f44dd7fa700 1 -- 172.21.15.131:0/1215221215 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e00266b0 con 0x7f44e8156730 2023-12-26T21:53:02.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.731+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e814e6b0 con 0x7f44e8156730 2023-12-26T21:53:02.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.731+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e814e990 con 0x7f44e8156730 2023-12-26T21:53:02.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.732+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e814cac0 con 0x7f44e8156730 2023-12-26T21:53:02.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.733+0000 7f44dd7fa700 1 -- 172.21.15.131:0/1215221215 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f44e0003040 con 0x7f44e8156730 2023-12-26T21:53:02.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.734+0000 7f44dd7fa700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44cc066f30 0x7f44cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:02.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.734+0000 7f44dd7fa700 1 -- 172.21.15.131:0/1215221215 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f44e0086380 con 0x7f44e8156730 2023-12-26T21:53:02.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.734+0000 7f44ed39b700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44cc066f30 0x7f44cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:02.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.734+0000 7f44ed39b700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44cc066f30 0x7f44cc0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44e400adc0 tx=0x7f44e4006f90 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:02.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.738+0000 7f44dd7fa700 1 -- 172.21.15.131:0/1215221215 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44e814cac0 con 0x7f44e8156730 2023-12-26T21:53:02.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:02.932+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f44e80717f0 con 0x7f44e8156730 2023-12-26T21:53:03.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.239+0000 7f44dd7fa700 1 -- 172.21.15.131:0/1215221215 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 138+0+0 (secure 0 0 0) 0x7f44e0053870 con 0x7f44e8156730 2023-12-26T21:53:03.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44cc066f30 msgr2=0x7f44cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:03.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44cc066f30 0x7f44cc0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44e400adc0 tx=0x7f44e4006f90 comp rx=0 tx=0).stop 2023-12-26T21:53:03.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e8156730 msgr2=0x7f44e80c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:03.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e8156730 0x7f44e80c6b30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f44e80690d0 tx=0x7f44e0018000 comp rx=0 tx=0).stop 2023-12-26T21:53:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 shutdown_connections 2023-12-26T21:53:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44cc066f30 0x7f44cc0693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44e8156730 0x7f44e80c6b30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44e80c8ba0 0x7f44e814ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 --2- 172.21.15.131:0/1215221215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44e8156360 0x7f44e816d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.244+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 >> 172.21.15.131:0/1215221215 conn(0x7f44e80675b0 msgr2=0x7f44e80ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.246+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 shutdown_connections 2023-12-26T21:53:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.246+0000 7f44ef5ff700 1 -- 172.21.15.131:0/1215221215 wait complete. 2023-12-26T21:53:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T21:53:03.266 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T21:53:03.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.463+0000 7fd5dc3ff700 1 Processor -- start 2023-12-26T21:53:03.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.463+0000 7fd5dc3ff700 1 -- start start 2023-12-26T21:53:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.464+0000 7fd5dc3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5d4157050 0x7fd5d40686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.464+0000 7fd5dc3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d40caed0 con 0x7fd5d4157420 2023-12-26T21:53:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.464+0000 7fd5dc3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d40ca8e0 con 0x7fd5d4068c00 2023-12-26T21:53:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.464+0000 7fd5dc3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d40c9520 con 0x7fd5d4157050 2023-12-26T21:53:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.464+0000 7fd5da19b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5d4157050 0x7fd5d40686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.464+0000 7fd5da19b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5d4157050 0x7fd5d40686c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41610/0 (socket says 172.21.15.131:41610) 2023-12-26T21:53:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.464+0000 7fd5da19b700 1 -- 172.21.15.131:0/75322801 learned_addr learned my addr 172.21.15.131:0/75322801 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.464+0000 7fd5d999a700 1 --1- 172.21.15.131:0/75322801 >> v1:172.21.15.131:6789/0 conn(0x7fd5d4157420 0x7fd5d4159c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48972/0 (socket says 172.21.15.131:48972) 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.465+0000 7fd5da19b700 1 -- 172.21.15.131:0/75322801 >> v1:172.21.15.190:6789/0 conn(0x7fd5d4068c00 legacy=0x7fd5d40cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.465+0000 7fd5da19b700 1 -- 172.21.15.131:0/75322801 >> v1:172.21.15.131:6789/0 conn(0x7fd5d4157420 legacy=0x7fd5d4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.465+0000 7fd5da19b700 1 -- 172.21.15.131:0/75322801 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d415a6c0 con 0x7fd5d4157050 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.465+0000 7fd5d9199700 1 -- 172.21.15.131:0/75322801 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223812137 0 0) 0x7fd5d40caed0 con 0x7fd5d4157420 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.465+0000 7fd5da19b700 1 --2- 172.21.15.131:0/75322801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4157050 0x7fd5d40686c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00099b0 tx=0x7fd5d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132c573ebb76ca88 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.465+0000 7fd5d9199700 1 -- 172.21.15.131:0/75322801 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000b820 con 0x7fd5d4157050 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.465+0000 7fd5d9199700 1 -- 172.21.15.131:0/75322801 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d000b980 con 0x7fd5d4157050 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.465+0000 7fd5d9199700 1 -- 172.21.15.131:0/75322801 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000bd20 con 0x7fd5d4157050 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.466+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/75322801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4157050 msgr2=0x7fd5d40686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.466+0000 7fd5dc3ff700 1 --2- 172.21.15.131:0/75322801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4157050 0x7fd5d40686c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00099b0 tx=0x7fd5d0009250 comp rx=0 tx=0).stop 2023-12-26T21:53:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.466+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/75322801 shutdown_connections 2023-12-26T21:53:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.466+0000 7fd5dc3ff700 1 --2- 172.21.15.131:0/75322801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4157050 0x7fd5d40686c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.466+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/75322801 >> 172.21.15.131:0/75322801 conn(0x7fd5d406dc80 msgr2=0x7fd5d40675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.466+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/75322801 shutdown_connections 2023-12-26T21:53:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.466+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/75322801 wait complete. 2023-12-26T21:53:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.467+0000 7fd5dc3ff700 1 Processor -- start 2023-12-26T21:53:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.468+0000 7fd5dc3ff700 1 -- start start 2023-12-26T21:53:03.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.468+0000 7fd5dc3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4068c00 0x7fd5d4145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:03.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5dc3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d4157050 0x7fd5d40b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:03.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5dc3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d4157420 0x7fd5d40c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5dc3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c3b50 con 0x7fd5d4068c00 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5dc3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c2f40 con 0x7fd5d4157420 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5dc3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c3220 con 0x7fd5d4157050 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4068c00 0x7fd5d4145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4068c00 0x7fd5d4145670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41620/0 (socket says 172.21.15.131:41620) 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da19b700 1 -- 172.21.15.131:0/150644566 learned_addr learned my addr 172.21.15.131:0/150644566 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5d999a700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d4157050 0x7fd5d40b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da99c700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d4157420 0x7fd5d40c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da19b700 1 -- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d4157050 msgr2=0x7fd5d40b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da19b700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d4157050 0x7fd5d40b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da19b700 1 -- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d4157420 msgr2=0x7fd5d40c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da19b700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d4157420 0x7fd5d40c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da19b700 1 -- 172.21.15.131:0/150644566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d00094f0 con 0x7fd5d4068c00 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5d999a700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d4157050 0x7fd5d40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:03.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da99c700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d4157420 0x7fd5d40c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.469+0000 7fd5da19b700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4068c00 0x7fd5d4145670 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0003080 tx=0x7fd5d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.470+0000 7fd5ca7fc700 1 -- 172.21.15.131:0/150644566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d001b470 con 0x7fd5d4068c00 2023-12-26T21:53:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.470+0000 7fd5ca7fc700 1 -- 172.21.15.131:0/150644566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d001b5f0 con 0x7fd5d4068c00 2023-12-26T21:53:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.470+0000 7fd5ca7fc700 1 -- 172.21.15.131:0/150644566 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d001b9b0 con 0x7fd5d4068c00 2023-12-26T21:53:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.470+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d40c3500 con 0x7fd5d4068c00 2023-12-26T21:53:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.470+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d40c84a0 con 0x7fd5d4068c00 2023-12-26T21:53:03.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.471+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d40657e0 con 0x7fd5d4068c00 2023-12-26T21:53:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.472+0000 7fd5ca7fc700 1 -- 172.21.15.131:0/150644566 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd5d001bd20 con 0x7fd5d4068c00 2023-12-26T21:53:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.473+0000 7fd5ca7fc700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5b8066f30 0x7fd5b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:03.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.473+0000 7fd5ca7fc700 1 -- 172.21.15.131:0/150644566 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5615+0+0 (secure 0 0 0) 0x7fd5d0086650 con 0x7fd5d4068c00 2023-12-26T21:53:03.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.473+0000 7fd5d999a700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5b8066f30 0x7fd5b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:03.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.474+0000 7fd5d999a700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5b8066f30 0x7fd5b80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40b15f0 tx=0x7fd5c4009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:03.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.477+0000 7fd5ca7fc700 1 -- 172.21.15.131:0/150644566 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5d0053900 con 0x7fd5d4068c00 2023-12-26T21:53:03.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:03.669+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd5d415a9a0 con 0x7fd5d4068c00 2023-12-26T21:53:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.242+0000 7fd5ca7fc700 1 -- 172.21.15.131:0/150644566 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd5d002a0d0 con 0x7fd5d4068c00 2023-12-26T21:53:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5b8066f30 msgr2=0x7fd5b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5b8066f30 0x7fd5b80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40b15f0 tx=0x7fd5c4009040 comp rx=0 tx=0).stop 2023-12-26T21:53:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4068c00 msgr2=0x7fd5d4145670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4068c00 0x7fd5d4145670 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0003080 tx=0x7fd5d0016000 comp rx=0 tx=0).stop 2023-12-26T21:53:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 shutdown_connections 2023-12-26T21:53:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d4157420 0x7fd5d40c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5b8066f30 0x7fd5b80693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d4068c00 0x7fd5d4145670 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 --2- 172.21.15.131:0/150644566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d4157050 0x7fd5d40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.244+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 >> 172.21.15.131:0/150644566 conn(0x7fd5d406dc80 msgr2=0x7fd5d41568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.245+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 shutdown_connections 2023-12-26T21:53:04.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.245+0000 7fd5dc3ff700 1 -- 172.21.15.131:0/150644566 wait complete. 2023-12-26T21:53:04.247 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T21:53:04.279 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T21:53:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.472+0000 7f80345ff700 1 Processor -- start 2023-12-26T21:53:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.472+0000 7f80345ff700 1 -- start start 2023-12-26T21:53:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.473+0000 7f80345ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f802c0cc810 0x7f802c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.473+0000 7f80345ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c0686c0 con 0x7f802c13c510 2023-12-26T21:53:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.473+0000 7f80345ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c069520 con 0x7f802c06bfd0 2023-12-26T21:53:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.473+0000 7f80345ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c068910 con 0x7f802c0cc810 2023-12-26T21:53:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.473+0000 7f803239b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f802c13c510 0x7f802c06b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48988/0 (socket says 172.21.15.131:48988) 2023-12-26T21:53:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.473+0000 7f803239b700 1 -- 172.21.15.131:0/1774977887 learned_addr learned my addr 172.21.15.131:0/1774977887 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:04.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.473+0000 7f8032b9c700 1 --2- 172.21.15.131:0/1774977887 >> v2:172.21.15.131:3300/0 conn(0x7f802c0cc810 0x7f802c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.474+0000 7f8031399700 1 -- 172.21.15.131:0/1774977887 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702555874 0 0) 0x7f802c0686c0 con 0x7f802c13c510 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.474+0000 7f8031399700 1 -- 172.21.15.131:0/1774977887 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8018003570 con 0x7f802c13c510 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.474+0000 7f8032b9c700 1 -- 172.21.15.131:0/1774977887 >> v1:172.21.15.190:6789/0 conn(0x7f802c06bfd0 legacy=0x7f802c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.474+0000 7f8032b9c700 1 -- 172.21.15.131:0/1774977887 >> v1:172.21.15.131:6789/0 conn(0x7f802c13c510 legacy=0x7f802c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.474+0000 7f8032b9c700 1 -- 172.21.15.131:0/1774977887 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c068bf0 con 0x7f802c0cc810 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.474+0000 7f8032b9c700 1 --2- 172.21.15.131:0/1774977887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 0x7f802c0c8a20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8024009e60 tx=0x7f8024009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ad455eec975e14 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.474+0000 7f8031399700 1 -- 172.21.15.131:0/1774977887 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802400e070 con 0x7f802c0cc810 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.474+0000 7f8031399700 1 -- 172.21.15.131:0/1774977887 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802400bb60 con 0x7f802c0cc810 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.474+0000 7f8031399700 1 -- 172.21.15.131:0/1774977887 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024012430 con 0x7f802c0cc810 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.475+0000 7f80345ff700 1 -- 172.21.15.131:0/1774977887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 msgr2=0x7f802c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.475+0000 7f80345ff700 1 --2- 172.21.15.131:0/1774977887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 0x7f802c0c8a20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8024009e60 tx=0x7f8024009780 comp rx=0 tx=0).stop 2023-12-26T21:53:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.475+0000 7f80345ff700 1 -- 172.21.15.131:0/1774977887 shutdown_connections 2023-12-26T21:53:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.475+0000 7f80345ff700 1 --2- 172.21.15.131:0/1774977887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 0x7f802c0c8a20 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.475+0000 7f80345ff700 1 -- 172.21.15.131:0/1774977887 >> 172.21.15.131:0/1774977887 conn(0x7f802c0ce3e0 msgr2=0x7f802c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.475+0000 7f80345ff700 1 -- 172.21.15.131:0/1774977887 shutdown_connections 2023-12-26T21:53:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.475+0000 7f80345ff700 1 -- 172.21.15.131:0/1774977887 wait complete. 2023-12-26T21:53:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.476+0000 7f80345ff700 1 Processor -- start 2023-12-26T21:53:04.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.477+0000 7f80345ff700 1 -- start start 2023-12-26T21:53:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.477+0000 7f80345ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f802c06bfd0 0x7f802c153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.477+0000 7f80345ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 0x7f802c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f80345ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f802c13c510 0x7f802c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f80345ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c13ac70 con 0x7f802c0cc810 2023-12-26T21:53:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f80345ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0b3380 con 0x7f802c06bfd0 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f80345ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0b2770 con 0x7f802c13c510 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f8031b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 0x7f802c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f803239b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f802c06bfd0 0x7f802c153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f8031b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 0x7f802c1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41640/0 (socket says 172.21.15.131:41640) 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f8031b9a700 1 -- 172.21.15.131:0/1994751001 learned_addr learned my addr 172.21.15.131:0/1994751001 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f8032b9c700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f802c13c510 0x7f802c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f8031b9a700 1 -- 172.21.15.131:0/1994751001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f802c13c510 msgr2=0x7f802c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f8031b9a700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f802c13c510 0x7f802c158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f8031b9a700 1 -- 172.21.15.131:0/1994751001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f802c06bfd0 msgr2=0x7f802c153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f8031b9a700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f802c06bfd0 0x7f802c153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.478+0000 7f8031b9a700 1 -- 172.21.15.131:0/1994751001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024009a20 con 0x7f802c0cc810 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.479+0000 7f8031b9a700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 0x7f802c1588a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c590 tx=0x7f801c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.479+0000 7f80227fc700 1 -- 172.21.15.131:0/1994751001 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c011590 con 0x7f802c0cc810 2023-12-26T21:53:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.479+0000 7f80227fc700 1 -- 172.21.15.131:0/1994751001 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c011710 con 0x7f802c0cc810 2023-12-26T21:53:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.479+0000 7f80227fc700 1 -- 172.21.15.131:0/1994751001 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c011ad0 con 0x7f802c0cc810 2023-12-26T21:53:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.479+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c0b2a50 con 0x7f802c0cc810 2023-12-26T21:53:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.479+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c0b3010 con 0x7f802c0cc810 2023-12-26T21:53:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.480+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c0b34f0 con 0x7f802c0cc810 2023-12-26T21:53:04.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.481+0000 7f80227fc700 1 -- 172.21.15.131:0/1994751001 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f801c006820 con 0x7f802c0cc810 2023-12-26T21:53:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.482+0000 7f80227fc700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8010066f30 0x7f80100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.482+0000 7f80227fc700 1 -- 172.21.15.131:0/1994751001 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f801c083810 con 0x7f802c0cc810 2023-12-26T21:53:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.482+0000 7f803239b700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8010066f30 0x7f80100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.483+0000 7f803239b700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8010066f30 0x7f80100693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8028001360 tx=0x7f8028009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:04.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.486+0000 7f80227fc700 1 -- 172.21.15.131:0/1994751001 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f801c050940 con 0x7f802c0cc810 2023-12-26T21:53:04.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:04.722+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f802c0657e0 con 0x7f802c0cc810 2023-12-26T21:53:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.278+0000 7f80227fc700 1 -- 172.21.15.131:0/1994751001 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7f801c0170c0 con 0x7f802c0cc810 2023-12-26T21:53:05.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.283+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8010066f30 msgr2=0x7f80100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.283+0000 7f80345ff700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8010066f30 0x7f80100693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8028001360 tx=0x7f8028009040 comp rx=0 tx=0).stop 2023-12-26T21:53:05.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.283+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 msgr2=0x7f802c1588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.283+0000 7f80345ff700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 0x7f802c1588a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c590 tx=0x7f801c007f50 comp rx=0 tx=0).stop 2023-12-26T21:53:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.286+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 shutdown_connections 2023-12-26T21:53:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.286+0000 7f80345ff700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f802c13c510 0x7f802c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.286+0000 7f80345ff700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f802c06bfd0 0x7f802c153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.286+0000 7f80345ff700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8010066f30 0x7f80100693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.286+0000 7f80345ff700 1 --2- 172.21.15.131:0/1994751001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f802c0cc810 0x7f802c1588a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.286+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 >> 172.21.15.131:0/1994751001 conn(0x7f802c0ce3e0 msgr2=0x7f802c0ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.289+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 shutdown_connections 2023-12-26T21:53:05.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.291+0000 7f80345ff700 1 -- 172.21.15.131:0/1994751001 wait complete. 2023-12-26T21:53:05.297 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 10 and data pool 11 2023-12-26T21:53:05.316 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:05.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.505+0000 7fb6e03ff700 1 Processor -- start 2023-12-26T21:53:05.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.505+0000 7fb6e03ff700 1 -- start start 2023-12-26T21:53:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.506+0000 7fb6e03ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb6d81596a0 0x7fb6d80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.506+0000 7fb6e03ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d80b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.506+0000 7fb6e03ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d80afcd0 con 0x7fb6d8159a70 2023-12-26T21:53:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.506+0000 7fb6e03ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d80b3e70 con 0x7fb6d81596a0 2023-12-26T21:53:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.506+0000 7fb6e03ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80b2e00 con 0x7fb6d80b0870 2023-12-26T21:53:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.507+0000 7fb6de99c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d80b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.507+0000 7fb6de19b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb6d81596a0 0x7fb6d80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.507+0000 7fb6de19b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb6d81596a0 0x7fb6d80b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:60488/0 (socket says 172.21.15.131:60488) 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.507+0000 7fb6de19b700 1 -- 172.21.15.131:0/3328923442 learned_addr learned my addr 172.21.15.131:0/3328923442 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.507+0000 7fb6de99c700 1 --2- 172.21.15.131:0/3328923442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d80b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:50742/0 (socket says 172.21.15.131:50742) 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.507+0000 7fb6de19b700 1 -- 172.21.15.131:0/3328923442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 msgr2=0x7fb6d80b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.507+0000 7fb6de19b700 1 --2- 172.21.15.131:0/3328923442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d80b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.507+0000 7fb6de19b700 1 -- 172.21.15.131:0/3328923442 >> v1:172.21.15.190:6789/0 conn(0x7fb6d8159a70 legacy=0x7fb6d80b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.507+0000 7fb6de19b700 1 -- 172.21.15.131:0/3328923442 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80b0cb0 con 0x7fb6d81596a0 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.508+0000 7fb6de19b700 1 --2- 172.21.15.131:0/3328923442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d81596a0 0x7fb6d80b2380 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40099b0 tx=0x7fb6d4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef733ce92f95f42 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.508+0000 7fb6dd199700 1 -- 172.21.15.131:0/3328923442 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4009ea0 con 0x7fb6d81596a0 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.508+0000 7fb6dd199700 1 -- 172.21.15.131:0/3328923442 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d400b860 con 0x7fb6d81596a0 2023-12-26T21:53:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.508+0000 7fb6dd199700 1 -- 172.21.15.131:0/3328923442 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400bc00 con 0x7fb6d81596a0 2023-12-26T21:53:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.508+0000 7fb6e03ff700 1 -- 172.21.15.131:0/3328923442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d81596a0 msgr2=0x7fb6d80b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.508+0000 7fb6e03ff700 1 --2- 172.21.15.131:0/3328923442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d81596a0 0x7fb6d80b2380 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40099b0 tx=0x7fb6d4009250 comp rx=0 tx=0).stop 2023-12-26T21:53:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.509+0000 7fb6e03ff700 1 -- 172.21.15.131:0/3328923442 shutdown_connections 2023-12-26T21:53:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.509+0000 7fb6e03ff700 1 --2- 172.21.15.131:0/3328923442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d80b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.509+0000 7fb6e03ff700 1 --2- 172.21.15.131:0/3328923442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d81596a0 0x7fb6d80b2380 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.509+0000 7fb6e03ff700 1 -- 172.21.15.131:0/3328923442 >> 172.21.15.131:0/3328923442 conn(0x7fb6d806ece0 msgr2=0x7fb6d806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.509+0000 7fb6e03ff700 1 -- 172.21.15.131:0/3328923442 shutdown_connections 2023-12-26T21:53:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.509+0000 7fb6e03ff700 1 -- 172.21.15.131:0/3328923442 wait complete. 2023-12-26T21:53:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.510+0000 7fb6e03ff700 1 Processor -- start 2023-12-26T21:53:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.510+0000 7fb6e03ff700 1 -- start start 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.511+0000 7fb6e03ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d8155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.511+0000 7fb6e03ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d81596a0 0x7fb6d8169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.511+0000 7fb6e03ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8159a70 0x7fb6d80c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.511+0000 7fb6e03ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d815a1d0 con 0x7fb6d81596a0 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.511+0000 7fb6e03ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d816a690 con 0x7fb6d8159a70 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.511+0000 7fb6e03ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d816a970 con 0x7fb6d80b0870 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.511+0000 7fb6dd99a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d81596a0 0x7fb6d8169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.511+0000 7fb6dd99a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d81596a0 0x7fb6d8169e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55674/0 (socket says 172.21.15.131:55674) 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.511+0000 7fb6dd99a700 1 -- 172.21.15.131:0/1858268834 learned_addr learned my addr 172.21.15.131:0/1858268834 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6de99c700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8159a70 0x7fb6d80c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6de19b700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d8155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6dd99a700 1 -- 172.21.15.131:0/1858268834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 msgr2=0x7fb6d8155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6dd99a700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d8155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6dd99a700 1 -- 172.21.15.131:0/1858268834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8159a70 msgr2=0x7fb6d80c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6dd99a700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8159a70 0x7fb6d80c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6dd99a700 1 -- 172.21.15.131:0/1858268834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40094f0 con 0x7fb6d81596a0 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6de19b700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6dd99a700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d81596a0 0x7fb6d8169e40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800c590 tx=0x7fb6c800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6ce7fc700 1 -- 172.21.15.131:0/1858268834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8011590 con 0x7fb6d81596a0 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6ce7fc700 1 -- 172.21.15.131:0/1858268834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c8011710 con 0x7fb6d81596a0 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d816ac50 con 0x7fb6d81596a0 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6ce7fc700 1 -- 172.21.15.131:0/1858268834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8011ad0 con 0x7fb6d81596a0 2023-12-26T21:53:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.512+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d816a380 con 0x7fb6d81596a0 2023-12-26T21:53:05.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.513+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d80657e0 con 0x7fb6d81596a0 2023-12-26T21:53:05.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.514+0000 7fb6ce7fc700 1 -- 172.21.15.131:0/1858268834 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb6c801b3b0 con 0x7fb6d81596a0 2023-12-26T21:53:05.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.516+0000 7fb6ce7fc700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6bc066f30 0x7fb6bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.516+0000 7fb6ce7fc700 1 -- 172.21.15.131:0/1858268834 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fb6c8083ca0 con 0x7fb6d81596a0 2023-12-26T21:53:05.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.516+0000 7fb6de19b700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6bc066f30 0x7fb6bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.516+0000 7fb6de19b700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6bc066f30 0x7fb6bc0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40099b0 tx=0x7fb6d4017000 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:05.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.520+0000 7fb6ce7fc700 1 -- 172.21.15.131:0/1858268834 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6c8050ce0 con 0x7fb6d81596a0 2023-12-26T21:53:05.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.713+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6d80ca8f0 con 0x7fb6d81596a0 2023-12-26T21:53:05.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.714+0000 7fb6ce7fc700 1 -- 172.21.15.131:0/1858268834 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16010 (secure 0 0 0) 0x7fb6c8017070 con 0x7fb6d81596a0 2023-12-26T21:53:05.715 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:05.715 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:04.724513+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:02.972636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:03.709325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:05.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.716+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6bc066f30 msgr2=0x7fb6bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.716+0000 7fb6e03ff700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6bc066f30 0x7fb6bc0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40099b0 tx=0x7fb6d4017000 comp rx=0 tx=0).stop 2023-12-26T21:53:05.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.716+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d81596a0 msgr2=0x7fb6d8169e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.716+0000 7fb6e03ff700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d81596a0 0x7fb6d8169e40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800c590 tx=0x7fb6c800c8a0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.717+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 shutdown_connections 2023-12-26T21:53:05.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.717+0000 7fb6e03ff700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8159a70 0x7fb6d80c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.717+0000 7fb6e03ff700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6d80b0870 0x7fb6d8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.717+0000 7fb6e03ff700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6bc066f30 0x7fb6bc0693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.717+0000 7fb6e03ff700 1 --2- 172.21.15.131:0/1858268834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d81596a0 0x7fb6d8169e40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.717+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 >> 172.21.15.131:0/1858268834 conn(0x7fb6d806ece0 msgr2=0x7fb6d81568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:05.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.717+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 shutdown_connections 2023-12-26T21:53:05.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.717+0000 7fb6e03ff700 1 -- 172.21.15.131:0/1858268834 wait complete. 2023-12-26T21:53:05.735 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:05.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.924+0000 7f239abff700 1 Processor -- start 2023-12-26T21:53:05.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.924+0000 7f239abff700 1 -- start start 2023-12-26T21:53:05.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239abff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23940cb320 0x7f2394069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f239406a1e0 0x7f23940b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239abff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23940b0c90 0x7f23940aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239abff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23940c8cc0 con 0x7f23940cb320 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239abff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23940ad700 con 0x7f23940b0c90 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23940b1110 con 0x7f239406a1e0 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239899b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23940cb320 0x7f2394069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239899b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23940cb320 0x7f2394069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55680/0 (socket says 172.21.15.131:55680) 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239899b700 1 -- 172.21.15.131:0/2421755500 learned_addr learned my addr 172.21.15.131:0/2421755500 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239919c700 1 --2- 172.21.15.131:0/2421755500 >> v2:172.21.15.190:3300/0 conn(0x7f23940b0c90 0x7f23940aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f2393fff700 1 --2- 172.21.15.131:0/2421755500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f239406a1e0 0x7f23940b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239899b700 1 -- 172.21.15.131:0/2421755500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f239406a1e0 msgr2=0x7f23940b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239899b700 1 --2- 172.21.15.131:0/2421755500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f239406a1e0 0x7f23940b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239899b700 1 -- 172.21.15.131:0/2421755500 >> v2:172.21.15.190:3300/0 conn(0x7f23940b0c90 msgr2=0x7f23940aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239899b700 1 --2- 172.21.15.131:0/2421755500 >> v2:172.21.15.190:3300/0 conn(0x7f23940b0c90 0x7f23940aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239899b700 1 -- 172.21.15.131:0/2421755500 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2394156940 con 0x7f23940cb320 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.926+0000 7f239919c700 1 --2- 172.21.15.131:0/2421755500 >> v2:172.21.15.190:3300/0 conn(0x7f23940b0c90 0x7f23940aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.927+0000 7f239899b700 1 --2- 172.21.15.131:0/2421755500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23940cb320 0x7f2394069ca0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f239406cf60 tx=0x7f2388009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df69883715393c33 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.927+0000 7f23937fe700 1 -- 172.21.15.131:0/2421755500 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238800b860 con 0x7f23940cb320 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.927+0000 7f23937fe700 1 -- 172.21.15.131:0/2421755500 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238800b9c0 con 0x7f23940cb320 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.927+0000 7f23937fe700 1 -- 172.21.15.131:0/2421755500 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238800bd60 con 0x7f23940cb320 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.927+0000 7f239abff700 1 -- 172.21.15.131:0/2421755500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23940cb320 msgr2=0x7f2394069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.927+0000 7f239abff700 1 --2- 172.21.15.131:0/2421755500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23940cb320 0x7f2394069ca0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f239406cf60 tx=0x7f2388009250 comp rx=0 tx=0).stop 2023-12-26T21:53:05.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.928+0000 7f239abff700 1 -- 172.21.15.131:0/2421755500 shutdown_connections 2023-12-26T21:53:05.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.928+0000 7f239abff700 1 --2- 172.21.15.131:0/2421755500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23940cb320 0x7f2394069ca0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.928+0000 7f239abff700 1 --2- 172.21.15.131:0/2421755500 >> v2:172.21.15.190:3300/0 conn(0x7f23940b0c90 0x7f23940aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.928+0000 7f239abff700 1 --2- 172.21.15.131:0/2421755500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f239406a1e0 0x7f23940b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.928+0000 7f239abff700 1 -- 172.21.15.131:0/2421755500 >> 172.21.15.131:0/2421755500 conn(0x7f2394068040 msgr2=0x7f2394069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:05.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.928+0000 7f239abff700 1 -- 172.21.15.131:0/2421755500 shutdown_connections 2023-12-26T21:53:05.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.928+0000 7f239abff700 1 -- 172.21.15.131:0/2421755500 wait complete. 2023-12-26T21:53:05.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.929+0000 7f239abff700 1 Processor -- start 2023-12-26T21:53:05.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.929+0000 7f239abff700 1 -- start start 2023-12-26T21:53:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f239abff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f239406a1e0 0x7f23940a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f239abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23940b0c90 0x7f23940abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f239abff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23940cb320 0x7f2394144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f239abff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23940ac6b0 con 0x7f239406a1e0 2023-12-26T21:53:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f239abff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23940ac990 con 0x7f23940cb320 2023-12-26T21:53:05.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f239abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23940acc70 con 0x7f23940b0c90 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f239899b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f239406a1e0 0x7f23940a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f239899b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f239406a1e0 0x7f23940a9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55686/0 (socket says 172.21.15.131:55686) 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f239899b700 1 -- 172.21.15.131:0/2885967083 learned_addr learned my addr 172.21.15.131:0/2885967083 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.930+0000 7f2393fff700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23940b0c90 0x7f23940abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f239919c700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23940cb320 0x7f2394144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f239899b700 1 -- 172.21.15.131:0/2885967083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23940b0c90 msgr2=0x7f23940abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f239899b700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23940b0c90 0x7f23940abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f239899b700 1 -- 172.21.15.131:0/2885967083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23940cb320 msgr2=0x7f2394144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f239899b700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23940cb320 0x7f2394144c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f239899b700 1 -- 172.21.15.131:0/2885967083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880094f0 con 0x7f239406a1e0 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f239899b700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f239406a1e0 0x7f23940a9870 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2388009290 tx=0x7f2388017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f2390ff9700 1 -- 172.21.15.131:0/2885967083 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388011e40 con 0x7f239406a1e0 2023-12-26T21:53:05.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f2390ff9700 1 -- 172.21.15.131:0/2885967083 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238801f070 con 0x7f239406a1e0 2023-12-26T21:53:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23940acf50 con 0x7f239406a1e0 2023-12-26T21:53:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.931+0000 7f2390ff9700 1 -- 172.21.15.131:0/2885967083 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23880266d0 con 0x7f239406a1e0 2023-12-26T21:53:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.932+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23940ac3c0 con 0x7f239406a1e0 2023-12-26T21:53:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.932+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23940a5650 con 0x7f239406a1e0 2023-12-26T21:53:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.934+0000 7f2390ff9700 1 -- 172.21.15.131:0/2885967083 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2388026a40 con 0x7f239406a1e0 2023-12-26T21:53:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.934+0000 7f2390ff9700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378066f30 0x7f23780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.935+0000 7f2390ff9700 1 -- 172.21.15.131:0/2885967083 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f2388021070 con 0x7f239406a1e0 2023-12-26T21:53:05.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.935+0000 7f2393fff700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378066f30 0x7f23780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.935+0000 7f2393fff700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378066f30 0x7f23780693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f239406be90 tx=0x7f2380008040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:05.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:05.938+0000 7f2390ff9700 1 -- 172.21.15.131:0/2885967083 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2388053a80 con 0x7f239406a1e0 2023-12-26T21:53:06.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.130+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23940657e0 con 0x7f239406a1e0 2023-12-26T21:53:06.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.131+0000 7f2390ff9700 1 -- 172.21.15.131:0/2885967083 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16010 (secure 0 0 0) 0x7f23880570a0 con 0x7f239406a1e0 2023-12-26T21:53:06.131 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:06.131 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:04.724513+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:02.972636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:03.709325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:06.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378066f30 msgr2=0x7f23780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:06.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378066f30 0x7f23780693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f239406be90 tx=0x7f2380008040 comp rx=0 tx=0).stop 2023-12-26T21:53:06.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f239406a1e0 msgr2=0x7f23940a9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:06.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f239406a1e0 0x7f23940a9870 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2388009290 tx=0x7f2388017000 comp rx=0 tx=0).stop 2023-12-26T21:53:06.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 shutdown_connections 2023-12-26T21:53:06.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23940cb320 0x7f2394144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378066f30 0x7f23780693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f239406a1e0 0x7f23940a9870 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 --2- 172.21.15.131:0/2885967083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23940b0c90 0x7f23940abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.133+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 >> 172.21.15.131:0/2885967083 conn(0x7f2394068040 msgr2=0x7f239406e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:06.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.134+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 shutdown_connections 2023-12-26T21:53:06.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.134+0000 7f239abff700 1 -- 172.21.15.131:0/2885967083 wait complete. 2023-12-26T21:53:06.152 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T21:53:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.350+0000 7f88417ff700 1 Processor -- start 2023-12-26T21:53:06.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.350+0000 7f88417ff700 1 -- start start 2023-12-26T21:53:06.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.351+0000 7f88417ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f883c13c510 0x7f883c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:06.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.351+0000 7f88417ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0cc810 0x7f883c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:06.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.351+0000 7f88417ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883c0686c0 con 0x7f883c06b8c0 2023-12-26T21:53:06.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.351+0000 7f88417ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883c06c280 con 0x7f883c13c510 2023-12-26T21:53:06.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.351+0000 7f88417ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c069520 con 0x7f883c0cc810 2023-12-26T21:53:06.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f883a7fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f883c06b8c0 0x7f883c0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49268/0 (socket says 172.21.15.131:49268) 2023-12-26T21:53:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f883a7fc700 1 -- 172.21.15.131:0/3176505679 learned_addr learned my addr 172.21.15.131:0/3176505679 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f883affd700 1 --2- 172.21.15.131:0/3176505679 >> v2:172.21.15.190:3300/0 conn(0x7f883c13c510 0x7f883c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f883b7fe700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0cc810 0x7f883c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977785447 0 0) 0x7f883c0686c0 con 0x7f883c06b8c0 2023-12-26T21:53:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8824003570 con 0x7f883c06b8c0 2023-12-26T21:53:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255358930 0 0) 0x7f8824003570 con 0x7f883c06b8c0 2023-12-26T21:53:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0cc810 msgr2=0x7f883c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0cc810 0x7f883c06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 >> v2:172.21.15.190:3300/0 conn(0x7f883c13c510 msgr2=0x7f883c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 --2- 172.21.15.131:0/3176505679 >> v2:172.21.15.190:3300/0 conn(0x7f883c13c510 0x7f883c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c068910 con 0x7f883c06b8c0 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f8830001470 con 0x7f883c06b8c0 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.352+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 >> v1:172.21.15.131:6789/0 conn(0x7f883c06b8c0 legacy=0x7f883c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f8839ffb700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0cc810 0x7f8824003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f8839ffb700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f8824004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f8839ffb700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8824004d50 0x7f88240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c06c280 con 0x7f8824003fe0 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c069520 con 0x7f8824004d50 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882400ba90 con 0x7f883c0cc810 2023-12-26T21:53:06.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f883a7fc700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f8824004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f883affd700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0cc810 0x7f8824003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f883b7fe700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8824004d50 0x7f88240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f883a7fc700 1 -- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0cc810 msgr2=0x7f8824003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f883a7fc700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0cc810 0x7f8824003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f883a7fc700 1 -- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8824004d50 msgr2=0x7f88240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f883a7fc700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8824004d50 0x7f88240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.353+0000 7f883a7fc700 1 -- 172.21.15.131:0/3176505679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824003570 con 0x7f8824003fe0 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f883a7fc700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f8824004810 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f883000c3c0 tx=0x7f8830001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17341ca4dfb91026 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000cc30 con 0x7f8824003fe0 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883000cdb0 con 0x7f8824003fe0 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f8839ffb700 1 -- 172.21.15.131:0/3176505679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830005b70 con 0x7f8824003fe0 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f88417ff700 1 -- 172.21.15.131:0/3176505679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 msgr2=0x7f8824004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f88417ff700 1 --2- 172.21.15.131:0/3176505679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f8824004810 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f883000c3c0 tx=0x7f8830001e90 comp rx=0 tx=0).stop 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f883b7fe700 1 -- 172.21.15.131:0/3176505679 reap_dead start 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f88417ff700 1 -- 172.21.15.131:0/3176505679 shutdown_connections 2023-12-26T21:53:06.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f88417ff700 1 -- 172.21.15.131:0/3176505679 >> 172.21.15.131:0/3176505679 conn(0x7f883c0ce3e0 msgr2=0x7f883c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:06.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f88417ff700 1 -- 172.21.15.131:0/3176505679 shutdown_connections 2023-12-26T21:53:06.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.354+0000 7f88417ff700 1 -- 172.21.15.131:0/3176505679 wait complete. 2023-12-26T21:53:06.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.356+0000 7f88417ff700 1 Processor -- start 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.356+0000 7f88417ff700 1 -- start start 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f88417ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f883c157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f88417ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0c7760 0x7f883c0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f88417ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f883c0b3640 0x7f883c0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f88417ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c0b25e0 con 0x7f8824003fe0 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f88417ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c0b3ff0 con 0x7f883c0b3640 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f88417ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c157540 con 0x7f883c0c7760 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f883c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f883c157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55692/0 (socket says 172.21.15.131:55692) 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883affd700 1 -- 172.21.15.131:0/4177849206 learned_addr learned my addr 172.21.15.131:0/4177849206 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883a7fc700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0c7760 0x7f883c0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:06.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883b7fe700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f883c0b3640 0x7f883c0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883affd700 1 -- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0c7760 msgr2=0x7f883c0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883affd700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0c7760 0x7f883c0b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883affd700 1 -- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f883c0b3640 msgr2=0x7f883c0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883affd700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f883c0b3640 0x7f883c0b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883affd700 1 -- 172.21.15.131:0/4177849206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c0b19d0 con 0x7f8824003fe0 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883a7fc700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0c7760 0x7f883c0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.357+0000 7f883b7fe700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f883c0b3640 0x7f883c0b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.358+0000 7f883affd700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f883c157000 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f882c011b60 tx=0x7f882c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.358+0000 7f8822ffd700 1 -- 172.21.15.131:0/4177849206 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c005c10 con 0x7f8824003fe0 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.358+0000 7f8822ffd700 1 -- 172.21.15.131:0/4177849206 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c005d70 con 0x7f8824003fe0 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.358+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883c0b1cb0 con 0x7f8824003fe0 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.358+0000 7f8822ffd700 1 -- 172.21.15.131:0/4177849206 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00fa60 con 0x7f8824003fe0 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.358+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c0b2270 con 0x7f8824003fe0 2023-12-26T21:53:06.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.359+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883c0b28c0 con 0x7f8824003fe0 2023-12-26T21:53:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.361+0000 7f8822ffd700 1 -- 172.21.15.131:0/4177849206 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f882c006050 con 0x7f8824003fe0 2023-12-26T21:53:06.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.362+0000 7f8822ffd700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8828066f30 0x7f88280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:06.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.362+0000 7f8822ffd700 1 -- 172.21.15.131:0/4177849206 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f882c085380 con 0x7f8824003fe0 2023-12-26T21:53:06.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.362+0000 7f883a7fc700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8828066f30 0x7f88280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:06.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.363+0000 7f883a7fc700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8828066f30 0x7f88280693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f883c1432d0 tx=0x7f8830015000 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:06.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.365+0000 7f8822ffd700 1 -- 172.21.15.131:0/4177849206 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f882c0524e0 con 0x7f8824003fe0 2023-12-26T21:53:06.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:06.604+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f883c06db70 con 0x7f8824003fe0 2023-12-26T21:53:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.284+0000 7f8822ffd700 1 -- 172.21.15.131:0/4177849206 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v41) v1 ==== 118+0+0 (secure 0 0 0) 0x7f882c0210b0 con 0x7f8824003fe0 2023-12-26T21:53:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8828066f30 msgr2=0x7f88280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8828066f30 0x7f88280693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f883c1432d0 tx=0x7f8830015000 comp rx=0 tx=0).stop 2023-12-26T21:53:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 msgr2=0x7f883c157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f883c157000 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f882c011b60 tx=0x7f882c009120 comp rx=0 tx=0).stop 2023-12-26T21:53:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 shutdown_connections 2023-12-26T21:53:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f883c0b3640 0x7f883c0b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8828066f30 0x7f88280693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8824003fe0 0x7f883c157000 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 --2- 172.21.15.131:0/4177849206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f883c0c7760 0x7f883c0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 >> 172.21.15.131:0/4177849206 conn(0x7f883c0ce3e0 msgr2=0x7f883c13a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 shutdown_connections 2023-12-26T21:53:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.287+0000 7f88417ff700 1 -- 172.21.15.131:0/4177849206 wait complete. 2023-12-26T21:53:07.306 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T21:53:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.499+0000 7f6f543ff700 1 Processor -- start 2023-12-26T21:53:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.500+0000 7f6f543ff700 1 -- start start 2023-12-26T21:53:07.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.501+0000 7f6f543ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6f4c1596a0 0x7f6f4c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:07.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.501+0000 7f6f543ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c159a70 0x7f6f4c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:07.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.501+0000 7f6f543ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4c0afcd0 con 0x7f6f4c0b0160 2023-12-26T21:53:07.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.501+0000 7f6f543ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4c0b3e70 con 0x7f6f4c1596a0 2023-12-26T21:53:07.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.501+0000 7f6f543ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0b0cd0 con 0x7f6f4c159a70 2023-12-26T21:53:07.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.501+0000 7f6f5299c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6f4c0b0160 0x7f6f4c0b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49280/0 (socket says 172.21.15.131:49280) 2023-12-26T21:53:07.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.501+0000 7f6f5299c700 1 -- 172.21.15.131:0/3885233191 learned_addr learned my addr 172.21.15.131:0/3885233191 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:07.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.501+0000 7f6f5219b700 1 --2- 172.21.15.131:0/3885233191 >> v2:172.21.15.190:3300/0 conn(0x7f6f4c1596a0 0x7f6f4c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:07.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.501+0000 7f6f5199a700 1 --2- 172.21.15.131:0/3885233191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c159a70 0x7f6f4c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f51199700 1 -- 172.21.15.131:0/3885233191 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806857219 0 0) 0x7f6f4c0afcd0 con 0x7f6f4c0b0160 2023-12-26T21:53:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f51199700 1 -- 172.21.15.131:0/3885233191 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f38003570 con 0x7f6f4c0b0160 2023-12-26T21:53:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f5219b700 1 -- 172.21.15.131:0/3885233191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c159a70 msgr2=0x7f6f4c0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f5219b700 1 --2- 172.21.15.131:0/3885233191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c159a70 0x7f6f4c0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f5219b700 1 -- 172.21.15.131:0/3885233191 >> v1:172.21.15.131:6789/0 conn(0x7f6f4c0b0160 legacy=0x7f6f4c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f5219b700 1 -- 172.21.15.131:0/3885233191 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c0b2e00 con 0x7f6f4c1596a0 2023-12-26T21:53:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f5199a700 1 --2- 172.21.15.131:0/3885233191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c159a70 0x7f6f4c0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f5219b700 1 --2- 172.21.15.131:0/3885233191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c1596a0 0x7f6f4c0b2380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6f480098e0 tx=0x7f6f480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51030e3bfb5e6e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f51199700 1 -- 172.21.15.131:0/3885233191 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48011630 con 0x7f6f4c1596a0 2023-12-26T21:53:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f51199700 1 -- 172.21.15.131:0/3885233191 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f48011790 con 0x7f6f4c1596a0 2023-12-26T21:53:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.502+0000 7f6f51199700 1 -- 172.21.15.131:0/3885233191 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48011b10 con 0x7f6f4c1596a0 2023-12-26T21:53:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.503+0000 7f6f543ff700 1 -- 172.21.15.131:0/3885233191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c1596a0 msgr2=0x7f6f4c0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.503+0000 7f6f543ff700 1 --2- 172.21.15.131:0/3885233191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c1596a0 0x7f6f4c0b2380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6f480098e0 tx=0x7f6f480091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.503+0000 7f6f543ff700 1 -- 172.21.15.131:0/3885233191 shutdown_connections 2023-12-26T21:53:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.503+0000 7f6f543ff700 1 --2- 172.21.15.131:0/3885233191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c1596a0 0x7f6f4c0b2380 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.503+0000 7f6f543ff700 1 --2- 172.21.15.131:0/3885233191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c159a70 0x7f6f4c0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.503+0000 7f6f543ff700 1 -- 172.21.15.131:0/3885233191 >> 172.21.15.131:0/3885233191 conn(0x7f6f4c06ece0 msgr2=0x7f6f4c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.503+0000 7f6f543ff700 1 -- 172.21.15.131:0/3885233191 shutdown_connections 2023-12-26T21:53:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.503+0000 7f6f543ff700 1 -- 172.21.15.131:0/3885233191 wait complete. 2023-12-26T21:53:07.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.504+0000 7f6f543ff700 1 Processor -- start 2023-12-26T21:53:07.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.504+0000 7f6f543ff700 1 -- start start 2023-12-26T21:53:07.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.505+0000 7f6f543ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c0b0160 0x7f6f4c0c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:07.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.505+0000 7f6f543ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c1596a0 0x7f6f4c156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:07.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.505+0000 7f6f543ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f4c159a70 0x7f6f4c0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:07.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.505+0000 7f6f543ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c15b1b0 con 0x7f6f4c159a70 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.505+0000 7f6f543ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c15ae00 con 0x7f6f4c0b0160 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.505+0000 7f6f543ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c15b400 con 0x7f6f4c1596a0 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5299c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f4c159a70 0x7f6f4c0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5299c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f4c159a70 0x7f6f4c0cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55706/0 (socket says 172.21.15.131:55706) 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5299c700 1 -- 172.21.15.131:0/2289154831 learned_addr learned my addr 172.21.15.131:0/2289154831 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5219b700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c0b0160 0x7f6f4c0c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5199a700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c1596a0 0x7f6f4c156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5299c700 1 -- 172.21.15.131:0/2289154831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c1596a0 msgr2=0x7f6f4c156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5299c700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c1596a0 0x7f6f4c156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5299c700 1 -- 172.21.15.131:0/2289154831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c0b0160 msgr2=0x7f6f4c0c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:07.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5299c700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c0b0160 0x7f6f4c0c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5299c700 1 -- 172.21.15.131:0/2289154831 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c009420 con 0x7f6f4c159a70 2023-12-26T21:53:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5219b700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c0b0160 0x7f6f4c0c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f5299c700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f4c159a70 0x7f6f4c0cc380 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6f44001460 tx=0x7f6f44001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.506+0000 7f6f427fc700 1 -- 172.21.15.131:0/2289154831 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4400a940 con 0x7f6f4c159a70 2023-12-26T21:53:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.507+0000 7f6f427fc700 1 -- 172.21.15.131:0/2289154831 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4400aaa0 con 0x7f6f4c159a70 2023-12-26T21:53:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.507+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f48009420 con 0x7f6f4c159a70 2023-12-26T21:53:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.507+0000 7f6f427fc700 1 -- 172.21.15.131:0/2289154831 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4400ae20 con 0x7f6f4c159a70 2023-12-26T21:53:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.507+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c15b9c0 con 0x7f6f4c159a70 2023-12-26T21:53:07.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.508+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4c15bd30 con 0x7f6f4c159a70 2023-12-26T21:53:07.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.509+0000 7f6f427fc700 1 -- 172.21.15.131:0/2289154831 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6f44007300 con 0x7f6f4c159a70 2023-12-26T21:53:07.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.509+0000 7f6f427fc700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f30066f30 0x7f6f300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:07.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.509+0000 7f6f427fc700 1 -- 172.21.15.131:0/2289154831 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f6f44084b00 con 0x7f6f4c159a70 2023-12-26T21:53:07.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.510+0000 7f6f5219b700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f30066f30 0x7f6f300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:07.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.510+0000 7f6f5219b700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f30066f30 0x7f6f300693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f480091e0 tx=0x7f6f48017000 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:07.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.513+0000 7f6f427fc700 1 -- 172.21.15.131:0/2289154831 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f44086e60 con 0x7f6f4c159a70 2023-12-26T21:53:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:07.748+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6f4c0657e0 con 0x7f6f4c159a70 2023-12-26T21:53:08.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.291+0000 7f6f427fc700 1 -- 172.21.15.131:0/2289154831 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f44051bc0 con 0x7f6f4c159a70 2023-12-26T21:53:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.293+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f30066f30 msgr2=0x7f6f300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.293+0000 7f6f543ff700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f30066f30 0x7f6f300693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f480091e0 tx=0x7f6f48017000 comp rx=0 tx=0).stop 2023-12-26T21:53:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.293+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f4c159a70 msgr2=0x7f6f4c0cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.293+0000 7f6f543ff700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f4c159a70 0x7f6f4c0cc380 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6f44001460 tx=0x7f6f44001ae0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.294+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 shutdown_connections 2023-12-26T21:53:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.294+0000 7f6f543ff700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f30066f30 0x7f6f300693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.294+0000 7f6f543ff700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f4c159a70 0x7f6f4c0cc380 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.294+0000 7f6f543ff700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4c0b0160 0x7f6f4c0c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.294+0000 7f6f543ff700 1 --2- 172.21.15.131:0/2289154831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4c1596a0 0x7f6f4c156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.294+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 >> 172.21.15.131:0/2289154831 conn(0x7f6f4c06ece0 msgr2=0x7f6f4c158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.294+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 shutdown_connections 2023-12-26T21:53:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.294+0000 7f6f543ff700 1 -- 172.21.15.131:0/2289154831 wait complete. 2023-12-26T21:53:08.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:53:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.506+0000 7f72ecdff700 1 Processor -- start 2023-12-26T21:53:08.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.506+0000 7f72ecdff700 1 -- start start 2023-12-26T21:53:08.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.507+0000 7f72ecdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f72e813c510 0x7f72e80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.507+0000 7f72ecdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e80686c0 con 0x7f72e806b8c0 2023-12-26T21:53:08.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.507+0000 7f72ecdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e8069520 con 0x7f72e80cc810 2023-12-26T21:53:08.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.507+0000 7f72ecdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e8068910 con 0x7f72e813c510 2023-12-26T21:53:08.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.508+0000 7f72e659c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f72e813c510 0x7f72e80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.508+0000 7f72e5d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f72e806b8c0 0x7f72e80cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49292/0 (socket says 172.21.15.131:49292) 2023-12-26T21:53:08.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.508+0000 7f72e5d9b700 1 -- 172.21.15.131:0/561681635 learned_addr learned my addr 172.21.15.131:0/561681635 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.508+0000 7f72e559a700 1 -- 172.21.15.131:0/561681635 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100866180 0 0) 0x7f72e80686c0 con 0x7f72e806b8c0 2023-12-26T21:53:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.508+0000 7f72e559a700 1 -- 172.21.15.131:0/561681635 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d0003570 con 0x7f72e806b8c0 2023-12-26T21:53:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.508+0000 7f72e659c700 1 -- 172.21.15.131:0/561681635 >> v1:172.21.15.190:6789/0 conn(0x7f72e80cc810 legacy=0x7f72e80699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.508+0000 7f72e659c700 1 -- 172.21.15.131:0/561681635 >> v1:172.21.15.131:6789/0 conn(0x7f72e806b8c0 legacy=0x7f72e80cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.508+0000 7f72e659c700 1 -- 172.21.15.131:0/561681635 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8068bf0 con 0x7f72e813c510 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72e659c700 1 --2- 172.21.15.131:0/561681635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e813c510 0x7f72e80c8a20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f72d80098d0 tx=0x7f72d8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624b8db7135d48f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72e559a700 1 -- 172.21.15.131:0/561681635 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8005c10 con 0x7f72e813c510 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72e559a700 1 -- 172.21.15.131:0/561681635 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d8005d70 con 0x7f72e813c510 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72e559a700 1 -- 172.21.15.131:0/561681635 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8006280 con 0x7f72e813c510 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72ecdff700 1 -- 172.21.15.131:0/561681635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e813c510 msgr2=0x7f72e80c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72ecdff700 1 --2- 172.21.15.131:0/561681635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e813c510 0x7f72e80c8a20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f72d80098d0 tx=0x7f72d8009120 comp rx=0 tx=0).stop 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72ecdff700 1 -- 172.21.15.131:0/561681635 shutdown_connections 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72ecdff700 1 --2- 172.21.15.131:0/561681635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e813c510 0x7f72e80c8a20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72ecdff700 1 -- 172.21.15.131:0/561681635 >> 172.21.15.131:0/561681635 conn(0x7f72e80ce3e0 msgr2=0x7f72e80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.509+0000 7f72ecdff700 1 -- 172.21.15.131:0/561681635 shutdown_connections 2023-12-26T21:53:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.510+0000 7f72ecdff700 1 -- 172.21.15.131:0/561681635 wait complete. 2023-12-26T21:53:08.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.511+0000 7f72ecdff700 1 Processor -- start 2023-12-26T21:53:08.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.511+0000 7f72ecdff700 1 -- start start 2023-12-26T21:53:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72ecdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e806b8c0 0x7f72e80b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72ecdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e80cc810 0x7f72e80b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72ecdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e813c510 0x7f72e80c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72ecdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e8151bf0 con 0x7f72e80cc810 2023-12-26T21:53:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72ecdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e816ad80 con 0x7f72e806b8c0 2023-12-26T21:53:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72ecdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e816d700 con 0x7f72e813c510 2023-12-26T21:53:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72e5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e80cc810 0x7f72e80b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72e5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e80cc810 0x7f72e80b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55722/0 (socket says 172.21.15.131:55722) 2023-12-26T21:53:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72e5d9b700 1 -- 172.21.15.131:0/2687739250 learned_addr learned my addr 172.21.15.131:0/2687739250 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72e659c700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e806b8c0 0x7f72e80b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.512+0000 7f72e6d9d700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e813c510 0x7f72e80c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72e5d9b700 1 -- 172.21.15.131:0/2687739250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e813c510 msgr2=0x7f72e80c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72e5d9b700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e813c510 0x7f72e80c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72e5d9b700 1 -- 172.21.15.131:0/2687739250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e806b8c0 msgr2=0x7f72e80b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72e5d9b700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e806b8c0 0x7f72e80b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72e5d9b700 1 -- 172.21.15.131:0/2687739250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d80093c0 con 0x7f72e80cc810 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72e5d9b700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e80cc810 0x7f72e80b17a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c5c0 tx=0x7f72dc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72d67fc700 1 -- 172.21.15.131:0/2687739250 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc005b30 con 0x7f72e80cc810 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72d67fc700 1 -- 172.21.15.131:0/2687739250 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc005cb0 con 0x7f72e80cc810 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e816d9e0 con 0x7f72e80cc810 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72d67fc700 1 -- 172.21.15.131:0/2687739250 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc007320 con 0x7f72e80cc810 2023-12-26T21:53:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.513+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e80b3ef0 con 0x7f72e80cc810 2023-12-26T21:53:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.514+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e816b080 con 0x7f72e80cc810 2023-12-26T21:53:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.515+0000 7f72d67fc700 1 -- 172.21.15.131:0/2687739250 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f72dc01b3b0 con 0x7f72e80cc810 2023-12-26T21:53:08.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.516+0000 7f72d67fc700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.516+0000 7f72d67fc700 1 -- 172.21.15.131:0/2687739250 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f72dc083ce0 con 0x7f72e80cc810 2023-12-26T21:53:08.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.516+0000 7f72e659c700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.517+0000 7f72e659c700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f72c8066f30 0x7f72c80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009390 tx=0x7f72d8016000 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.520+0000 7f72d67fc700 1 -- 172.21.15.131:0/2687739250 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72dc050d20 con 0x7f72e80cc810 2023-12-26T21:53:08.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.756+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72e80657e0 con 0x7f72e80cc810 2023-12-26T21:53:08.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.757+0000 7f72d67fc700 1 -- 172.21.15.131:0/2687739250 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f72dc017020 con 0x7f72e80cc810 2023-12-26T21:53:08.758 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:08.758 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6454,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1671363867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1671363867},{"type":"v1","addr":"172.21.15.131:6834","nonce":1671363867}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3807864961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3807864961},{"type":"v1","addr":"172.21.15.190:6834","nonce":3807864961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/18678440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":18678440},{"type":"v1","addr":"172.21.15.131:6835","nonce":18678440}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:04.724845+0000","modified":"2023-12-26T21:53:08.284069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/168268053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":168268053},{"type":"v1","addr":"172.21.15.190:6835","nonce":168268053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-12-26T21:53:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f72c8066f30 msgr2=0x7f72c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f72c8066f30 0x7f72c80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009390 tx=0x7f72d8016000 comp rx=0 tx=0).stop 2023-12-26T21:53:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e80cc810 msgr2=0x7f72e80b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e80cc810 0x7f72e80b17a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c5c0 tx=0x7f72dc007e00 comp rx=0 tx=0).stop 2023-12-26T21:53:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 shutdown_connections 2023-12-26T21:53:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72e813c510 0x7f72e80c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72e806b8c0 0x7f72e80b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 --2- 172.21.15.131:0/2687739250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72e80cc810 0x7f72e80b17a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 >> 172.21.15.131:0/2687739250 conn(0x7f72e80ce3e0 msgr2=0x7f72e81539c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.759+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 shutdown_connections 2023-12-26T21:53:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.760+0000 7f72ecdff700 1 -- 172.21.15.131:0/2687739250 wait complete. 2023-12-26T21:53:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 42 2023-12-26T21:53:08.778 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:08.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.966+0000 7fa9101ff700 1 Processor -- start 2023-12-26T21:53:08.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.967+0000 7fa9101ff700 1 -- start start 2023-12-26T21:53:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.968+0000 7fa9101ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa9080c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.968+0000 7fa9101ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa9080cc810 0x7fa90806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.968+0000 7fa9101ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90806c270 con 0x7fa90806b8c0 2023-12-26T21:53:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.968+0000 7fa9101ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9080686c0 con 0x7fa9080cc810 2023-12-26T21:53:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.968+0000 7fa9101ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa908069520 con 0x7fa90813c510 2023-12-26T21:53:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.968+0000 7fa90d79a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa90806b8c0 0x7fa9080cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49294/0 (socket says 172.21.15.131:49294) 2023-12-26T21:53:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.968+0000 7fa90d79a700 1 -- 172.21.15.131:0/1911403613 learned_addr learned my addr 172.21.15.131:0/1911403613 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.968+0000 7fa90df9b700 1 --2- 172.21.15.131:0/1911403613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa9080c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.968+0000 7fa90e79c700 1 --2- 172.21.15.131:0/1911403613 >> v2:172.21.15.190:3300/0 conn(0x7fa9080cc810 0x7fa90806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90cf99700 1 -- 172.21.15.131:0/1911403613 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112885310 0 0) 0x7fa90806c270 con 0x7fa90806b8c0 2023-12-26T21:53:08.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90cf99700 1 -- 172.21.15.131:0/1911403613 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f4003570 con 0x7fa90806b8c0 2023-12-26T21:53:08.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90df9b700 1 -- 172.21.15.131:0/1911403613 >> v2:172.21.15.190:3300/0 conn(0x7fa9080cc810 msgr2=0x7fa90806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:08.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90df9b700 1 --2- 172.21.15.131:0/1911403613 >> v2:172.21.15.190:3300/0 conn(0x7fa9080cc810 0x7fa90806bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90df9b700 1 -- 172.21.15.131:0/1911403613 >> v1:172.21.15.131:6789/0 conn(0x7fa90806b8c0 legacy=0x7fa9080cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90df9b700 1 -- 172.21.15.131:0/1911403613 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908068910 con 0x7fa90813c510 2023-12-26T21:53:08.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90e79c700 1 --2- 172.21.15.131:0/1911403613 >> v2:172.21.15.190:3300/0 conn(0x7fa9080cc810 0x7fa90806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:08.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90df9b700 1 --2- 172.21.15.131:0/1911403613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa9080c8a20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa9040098d0 tx=0x7fa904009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda3dddbabe8854f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:08.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90cf99700 1 -- 172.21.15.131:0/1911403613 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904005c10 con 0x7fa90813c510 2023-12-26T21:53:08.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90cf99700 1 -- 172.21.15.131:0/1911403613 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa904005d70 con 0x7fa90813c510 2023-12-26T21:53:08.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.969+0000 7fa90cf99700 1 -- 172.21.15.131:0/1911403613 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904006260 con 0x7fa90813c510 2023-12-26T21:53:08.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.970+0000 7fa9101ff700 1 -- 172.21.15.131:0/1911403613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 msgr2=0x7fa9080c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.970+0000 7fa9101ff700 1 --2- 172.21.15.131:0/1911403613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa9080c8a20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa9040098d0 tx=0x7fa904009120 comp rx=0 tx=0).stop 2023-12-26T21:53:08.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.970+0000 7fa9101ff700 1 -- 172.21.15.131:0/1911403613 shutdown_connections 2023-12-26T21:53:08.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.970+0000 7fa9101ff700 1 --2- 172.21.15.131:0/1911403613 >> v2:172.21.15.190:3300/0 conn(0x7fa9080cc810 0x7fa90806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.970+0000 7fa9101ff700 1 --2- 172.21.15.131:0/1911403613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa9080c8a20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.970+0000 7fa9101ff700 1 -- 172.21.15.131:0/1911403613 >> 172.21.15.131:0/1911403613 conn(0x7fa9080ce3e0 msgr2=0x7fa9080ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:08.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.970+0000 7fa9101ff700 1 -- 172.21.15.131:0/1911403613 shutdown_connections 2023-12-26T21:53:08.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.970+0000 7fa9101ff700 1 -- 172.21.15.131:0/1911403613 wait complete. 2023-12-26T21:53:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.971+0000 7fa9101ff700 1 Processor -- start 2023-12-26T21:53:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.972+0000 7fa9101ff700 1 -- start start 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.972+0000 7fa9101ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa90806b8c0 0x7fa908150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.972+0000 7fa9101ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9080cc810 0x7fa908154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.972+0000 7fa9101ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa90816d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.972+0000 7fa9101ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080b1970 con 0x7fa9080cc810 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.972+0000 7fa9101ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90816dc50 con 0x7fa90806b8c0 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.972+0000 7fa9101ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90816df30 con 0x7fa90813c510 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90d79a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9080cc810 0x7fa908154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90d79a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9080cc810 0x7fa908154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55732/0 (socket says 172.21.15.131:55732) 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90d79a700 1 -- 172.21.15.131:0/1904982042 learned_addr learned my addr 172.21.15.131:0/1904982042 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90df9b700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa90806b8c0 0x7fa908150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90e79c700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa90816d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90d79a700 1 -- 172.21.15.131:0/1904982042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 msgr2=0x7fa90816d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90d79a700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa90816d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90d79a700 1 -- 172.21.15.131:0/1904982042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa90806b8c0 msgr2=0x7fa908150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:08.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90d79a700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa90806b8c0 0x7fa908150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:08.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90d79a700 1 -- 172.21.15.131:0/1904982042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900009a20 con 0x7fa9080cc810 2023-12-26T21:53:08.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90e79c700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa90816d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:08.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.973+0000 7fa90d79a700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9080cc810 0x7fa908154960 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c590 tx=0x7fa8f8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:08.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.974+0000 7fa8fdffb700 1 -- 172.21.15.131:0/1904982042 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8005b30 con 0x7fa9080cc810 2023-12-26T21:53:08.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.974+0000 7fa8fdffb700 1 -- 172.21.15.131:0/1904982042 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f8005cb0 con 0x7fa9080cc810 2023-12-26T21:53:08.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.974+0000 7fa8fdffb700 1 -- 172.21.15.131:0/1904982042 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8007320 con 0x7fa9080cc810 2023-12-26T21:53:08.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.974+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9040093c0 con 0x7fa9080cc810 2023-12-26T21:53:08.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.974+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9080b1f10 con 0x7fa9080cc810 2023-12-26T21:53:08.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.975+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9080b20d0 con 0x7fa9080cc810 2023-12-26T21:53:08.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.976+0000 7fa8fdffb700 1 -- 172.21.15.131:0/1904982042 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa8f801b3b0 con 0x7fa9080cc810 2023-12-26T21:53:08.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.977+0000 7fa8fdffb700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa8ec066f30 0x7fa8ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:08.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.977+0000 7fa8fdffb700 1 -- 172.21.15.131:0/1904982042 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fa8f8083ce0 con 0x7fa9080cc810 2023-12-26T21:53:08.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.977+0000 7fa90df9b700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa8ec066f30 0x7fa8ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:08.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.978+0000 7fa90df9b700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa8ec066f30 0x7fa8ec0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa904009160 tx=0x7fa904016000 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:08.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:08.981+0000 7fa8fdffb700 1 -- 172.21.15.131:0/1904982042 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8f8050dc0 con 0x7fa9080cc810 2023-12-26T21:53:09.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.176+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9080657e0 con 0x7fa9080cc810 2023-12-26T21:53:09.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.177+0000 7fa8fdffb700 1 -- 172.21.15.131:0/1904982042 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+15997 (secure 0 0 0) 0x7fa8f8017020 con 0x7fa9080cc810 2023-12-26T21:53:09.178 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:09.178 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":96,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:06.269801+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:02.972636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:03.709325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:09.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa8ec066f30 msgr2=0x7fa8ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa8ec066f30 0x7fa8ec0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa904009160 tx=0x7fa904016000 comp rx=0 tx=0).stop 2023-12-26T21:53:09.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9080cc810 msgr2=0x7fa908154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9080cc810 0x7fa908154960 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c590 tx=0x7fa8f8007e00 comp rx=0 tx=0).stop 2023-12-26T21:53:09.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 shutdown_connections 2023-12-26T21:53:09.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa90813c510 0x7fa90816d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa90806b8c0 0x7fa908150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa8ec066f30 0x7fa8ec0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 --2- 172.21.15.131:0/1904982042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9080cc810 0x7fa908154960 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 >> 172.21.15.131:0/1904982042 conn(0x7fa9080ce3e0 msgr2=0x7fa9080cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:09.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 shutdown_connections 2023-12-26T21:53:09.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.180+0000 7fa9101ff700 1 -- 172.21.15.131:0/1904982042 wait complete. 2023-12-26T21:53:09.199 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T21:53:09.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.390+0000 7fa691bff700 1 Processor -- start 2023-12-26T21:53:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.390+0000 7fa691bff700 1 -- start start 2023-12-26T21:53:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.391+0000 7fa691bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa68c0c8bf0 0x7fa68c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.391+0000 7fa691bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa68c069d10 0x7fa68c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.391+0000 7fa691bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68c0caa10 con 0x7fa68c0c8820 2023-12-26T21:53:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.391+0000 7fa691bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68c157a80 con 0x7fa68c069d10 2023-12-26T21:53:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.391+0000 7fa691bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68c0cac50 con 0x7fa68c0c8bf0 2023-12-26T21:53:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.391+0000 7fa68b7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa68c0c8820 0x7fa68c0ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49302/0 (socket says 172.21.15.131:49302) 2023-12-26T21:53:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.391+0000 7fa68b7fe700 1 -- 172.21.15.131:0/1276023794 learned_addr learned my addr 172.21.15.131:0/1276023794 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.391+0000 7fa68bfff700 1 --2- 172.21.15.131:0/1276023794 >> v2:172.21.15.131:3300/0 conn(0x7fa68c069d10 0x7fa68c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.391+0000 7fa68affd700 1 --2- 172.21.15.131:0/1276023794 >> v2:172.21.15.190:3300/0 conn(0x7fa68c0c8bf0 0x7fa68c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.392+0000 7fa68bfff700 1 -- 172.21.15.131:0/1276023794 >> v2:172.21.15.190:3300/0 conn(0x7fa68c0c8bf0 msgr2=0x7fa68c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.392+0000 7fa68bfff700 1 --2- 172.21.15.131:0/1276023794 >> v2:172.21.15.190:3300/0 conn(0x7fa68c0c8bf0 0x7fa68c0b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.392+0000 7fa68bfff700 1 -- 172.21.15.131:0/1276023794 >> v1:172.21.15.131:6789/0 conn(0x7fa68c0c8820 legacy=0x7fa68c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.392+0000 7fa68bfff700 1 -- 172.21.15.131:0/1276023794 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c0caf30 con 0x7fa68c069d10 2023-12-26T21:53:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.392+0000 7fa68affd700 1 --2- 172.21.15.131:0/1276023794 >> v2:172.21.15.190:3300/0 conn(0x7fa68c0c8bf0 0x7fa68c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.392+0000 7fa68bfff700 1 --2- 172.21.15.131:0/1276023794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c069d10 0x7fa68c0b1820 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0646b0 tx=0x7fa684009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a596a483f3ce3ac server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.392+0000 7fa68a7fc700 1 -- 172.21.15.131:0/1276023794 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68400e070 con 0x7fa68c069d10 2023-12-26T21:53:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.392+0000 7fa68a7fc700 1 -- 172.21.15.131:0/1276023794 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68400bc30 con 0x7fa68c069d10 2023-12-26T21:53:09.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.392+0000 7fa68a7fc700 1 -- 172.21.15.131:0/1276023794 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684012430 con 0x7fa68c069d10 2023-12-26T21:53:09.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.393+0000 7fa691bff700 1 -- 172.21.15.131:0/1276023794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c069d10 msgr2=0x7fa68c0b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.393+0000 7fa691bff700 1 --2- 172.21.15.131:0/1276023794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c069d10 0x7fa68c0b1820 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0646b0 tx=0x7fa684009780 comp rx=0 tx=0).stop 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.393+0000 7fa691bff700 1 -- 172.21.15.131:0/1276023794 shutdown_connections 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.393+0000 7fa691bff700 1 --2- 172.21.15.131:0/1276023794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c069d10 0x7fa68c0b1820 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.393+0000 7fa691bff700 1 --2- 172.21.15.131:0/1276023794 >> v2:172.21.15.190:3300/0 conn(0x7fa68c0c8bf0 0x7fa68c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.393+0000 7fa691bff700 1 -- 172.21.15.131:0/1276023794 >> 172.21.15.131:0/1276023794 conn(0x7fa68c068040 msgr2=0x7fa68c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.393+0000 7fa691bff700 1 -- 172.21.15.131:0/1276023794 shutdown_connections 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.393+0000 7fa691bff700 1 -- 172.21.15.131:0/1276023794 wait complete. 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.394+0000 7fa691bff700 1 Processor -- start 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.394+0000 7fa691bff700 1 -- start start 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.395+0000 7fa691bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa68c069d10 0x7fa68c16d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.395+0000 7fa691bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c0c8820 0x7fa68c169560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.395+0000 7fa691bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68c0c8bf0 0x7fa68c15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.395+0000 7fa691bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c15d7e0 con 0x7fa68c0c8820 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.395+0000 7fa691bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c16d9e0 con 0x7fa68c069d10 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.395+0000 7fa691bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c15cbd0 con 0x7fa68c0c8bf0 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c0c8820 0x7fa68c169560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c0c8820 0x7fa68c169560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55746/0 (socket says 172.21.15.131:55746) 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68affd700 1 -- 172.21.15.131:0/1465099599 learned_addr learned my addr 172.21.15.131:0/1465099599 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68bfff700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68c0c8bf0 0x7fa68c15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68b7fe700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa68c069d10 0x7fa68c16d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68affd700 1 -- 172.21.15.131:0/1465099599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68c0c8bf0 msgr2=0x7fa68c15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68affd700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68c0c8bf0 0x7fa68c15a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68affd700 1 -- 172.21.15.131:0/1465099599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa68c069d10 msgr2=0x7fa68c16d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68affd700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa68c069d10 0x7fa68c16d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68affd700 1 -- 172.21.15.131:0/1465099599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa684009a20 con 0x7fa68c0c8820 2023-12-26T21:53:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.396+0000 7fa68affd700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c0c8820 0x7fa68c169560 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa68000e8a0 tx=0x7fa680008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.397+0000 7fa6737fe700 1 -- 172.21.15.131:0/1465099599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680013410 con 0x7fa68c0c8820 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.397+0000 7fa6737fe700 1 -- 172.21.15.131:0/1465099599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa680013570 con 0x7fa68c0c8820 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.397+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68c15ceb0 con 0x7fa68c0c8820 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.397+0000 7fa6737fe700 1 -- 172.21.15.131:0/1465099599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680013930 con 0x7fa68c0c8820 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.397+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68c15d470 con 0x7fa68c0c8820 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.398+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68c15d950 con 0x7fa68c0c8820 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.399+0000 7fa6737fe700 1 -- 172.21.15.131:0/1465099599 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa680013ca0 con 0x7fa68c0c8820 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.399+0000 7fa6737fe700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa678066f30 0x7fa6780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.399+0000 7fa6737fe700 1 -- 172.21.15.131:0/1465099599 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fa6800843c0 con 0x7fa68c0c8820 2023-12-26T21:53:09.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.400+0000 7fa68b7fe700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa678066f30 0x7fa6780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.400+0000 7fa68b7fe700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa678066f30 0x7fa6780693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa67c007130 tx=0x7fa67c007eb0 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:09.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.404+0000 7fa6737fe700 1 -- 172.21.15.131:0/1465099599 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa680054f30 con 0x7fa68c0c8820 2023-12-26T21:53:09.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.595+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa68c0657e0 con 0x7fa678066f30 2023-12-26T21:53:09.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.605+0000 7fa6737fe700 1 -- 172.21.15.131:0/1465099599 <== mgr.5789 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278033 (secure 0 0 0) 0x7fa68c0657e0 con 0x7fa678066f30 2023-12-26T21:53:09.610 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa678066f30 msgr2=0x7fa6780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa678066f30 0x7fa6780693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa67c007130 tx=0x7fa67c007eb0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c0c8820 msgr2=0x7fa68c169560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c0c8820 0x7fa68c169560 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa68000e8a0 tx=0x7fa680008040 comp rx=0 tx=0).stop 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 shutdown_connections 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68c0c8bf0 0x7fa68c15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa68c069d10 0x7fa68c16d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa678066f30 0x7fa6780693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 --2- 172.21.15.131:0/1465099599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa68c0c8820 0x7fa68c169560 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 >> 172.21.15.131:0/1465099599 conn(0x7fa68c068040 msgr2=0x7fa68c0c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:09.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 shutdown_connections 2023-12-26T21:53:09.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.609+0000 7fa691bff700 1 -- 172.21.15.131:0/1465099599 wait complete. 2023-12-26T21:53:09.613 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:09.628 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:53:09.075462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2433308,"kb_used_data":9948,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752541412,"statfs":{"total":773094113280,"available":770602405888,"internally_reserved":0,"allocated":10186752,"data_stored":5507971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.836671"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245654+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:01.295523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.249280+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:52:30.438143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241892+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:37:32.016433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.247029+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:40.412864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.246251+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:01.295523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245806+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:33:07.506029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248204+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:00:12.212821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245149+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:43.215374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.243193+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:01:47.175322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242805+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:50.422991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.246075+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:43.178061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248058+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:38:11.856188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248109+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:26:51.025444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245284+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:56.748993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245399+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:14:29.184662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245154+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:33:01.757024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242720+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:14:43.420525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241931+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:12.857594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241987+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:33.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.246301+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:07.445143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245832+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:47:44.760773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242453+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:07:55.195262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248812+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:44:47.409883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242202+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:36:43.945806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241940+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:58.054482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245513+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:48:01.129675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245537+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:18.916620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245618+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:23:09.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248855+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:09:46.836661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241921+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:15:35.448309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245367+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:43:17.471063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245188+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:01:23.220141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'183","reported_seq":461,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-26T21:53:02.161129+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:02.161129+0000","last_peered":"2023-12-26T21:53:02.161129+0000","last_clean":"2023-12-26T21:53:02.161129+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:02.161129+0000","last_undegraded":"2023-12-26T21:53:02.161129+0000","last_fullsized":"2023-12-26T21:53:02.161129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.647},"last":0.497},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.61199999999999999,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.92300000000000004,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":1,"15min":2.0150000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.58199999999999996,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.014,"5min":1.048,"15min":1.1359999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.59699999999999998,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":1.038,"15min":1.8640000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.498,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.419},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":0.93799999999999994,"5min":1.0189999999999999,"15min":1.327},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.65000000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.393},"last":0.67100000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.374,"15min":0.371},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72799999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.85999999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509636,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":535482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":1.0029999999999999,"15min":1.085},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.3380000000000001},"last":0.85499999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":1.288},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.876,"5min":0.92000000000000004,"15min":1.113},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.92000000000000004,"5min":1.026,"15min":1.569},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.97799999999999998,"15min":1.3839999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.62,"15min":0.62},"min":{"1min":0.42699999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.137},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.91200000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.69599999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95099999999999996,"5min":1.0049999999999999,"15min":1.296},"last":0.82099999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.437},"max":{"1min":0.89200000000000002,"5min":0.96899999999999997,"15min":1.51},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.91700000000000004,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.94799999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307160,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064680,"statfs":{"total":96636764160,"available":96322232320,"internally_reserved":0,"allocated":1720320,"data_stored":1132047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.34699999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.38},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.42899999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.745}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.56200000000000006}]}]},{"osd":0,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302484,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069356,"statfs":{"total":96636764160,"available":96327020544,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.439},"min":{"1min":0.316,"5min":0.245,"15min":0.245},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":1.5640000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69999999999999996,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.371,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.58599999999999997,"5min":0.66800000000000004,"15min":2.012},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.47699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.42299999999999999,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":4.2610000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.41799999999999998,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.498}]},{"osd":4,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.94699999999999995,"5min":1.1539999999999999,"15min":4.423},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.95799999999999996,"5min":1.069,"15min":1.6439999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78200000000000003,"15min":0.80900000000000005},"min":{"1min":0.57799999999999996,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":4.3650000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.008,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76700000000000002,"15min":0.78500000000000003},"min":{"1min":0.623,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.89000000000000001,"5min":1.103,"15min":1.611},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.55200000000000005,"5min":0.504,"15min":0.5},"max":{"1min":1.012,"5min":1.111,"15min":4.407},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.81000000000000005},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":1.145,"15min":4.5069999999999997},"last":0.57799999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306580,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065260,"statfs":{"total":96636764160,"available":96322826240,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45800000000000002,"15min":0.45900000000000002},"min":{"1min":0.33200000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":1.915},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.30399999999999999},"max":{"1min":0.63200000000000001,"5min":0.83899999999999997,"15min":1.7729999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.39600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":1.952},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47999999999999998},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.30199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.318}]},{"osd":3,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.375},"max":{"1min":1,"5min":1,"15min":1},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73899999999999999,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.495}]},{"osd":4,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.54600000000000004,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.96999999999999997,"5min":0.98299999999999998,"15min":1.284},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.60099999999999998,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.0600000000000001,"15min":1.5029999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":0.94599999999999995,"5min":1.014,"15min":1.355},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.93000000000000005,"5min":1.038,"15min":1.7350000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.95999999999999996,"5min":1.0609999999999999,"15min":1.349},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.47799999999999998},"max":{"1min":0.92700000000000005,"5min":1.073,"15min":1.841},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0189999999999999,"15min":1.294},"last":0.56100000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310676,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061164,"statfs":{"total":96636764160,"available":96318631936,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.40200000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.58299999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.498},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.75}]}]},{"osd":4,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302484,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069356,"statfs":{"total":96636764160,"available":96327020544,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66000000000000003,"15min":0.68400000000000005},"min":{"1min":0.439,"5min":0.307,"15min":0.307},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":2.1389999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.96399999999999997,"5min":1.008,"15min":1.407},"last":0.73499999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.52700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.879},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69799999999999995,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.89200000000000002,"5min":0.98699999999999999,"15min":2.097},"last":1.0209999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70999999999999996,"15min":0.72199999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91000000000000003,"5min":0.94999999999999996,"15min":1.7809999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88800000000000001,"5min":0.97199999999999998,"15min":1.6870000000000001},"last":0.72099999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69999999999999996,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95999999999999996,"15min":1.956},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.746},"min":{"1min":0.626,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.0609999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58899999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82099999999999995,"5min":0.86399999999999999,"15min":2.2429999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84899999999999998,"5min":0.92600000000000005,"15min":2.194},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.621,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.379},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":2.1589999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":1.133}]},{"osd":7,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.432},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":2.23},"last":1.002},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.745},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":2.5169999999999999},"last":1.266}]}]},{"osd":5,"up_from":15,"seq":64424509635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303064,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068776,"statfs":{"total":96636764160,"available":96326426624,"internally_reserved":0,"allocated":1720320,"data_stored":1132047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.878,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.71299999999999997},"min":{"1min":0.48899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.71},"last":0.81000000000000005}]},{"osd":1,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.57399999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96399999999999997,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68200000000000005,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.002,"15min":1.4219999999999999},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.871,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68799999999999994,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":1.214},"last":0.69199999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":1.052,"15min":1.4890000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":0.92200000000000004,"15min":1.1439999999999999},"last":0.624}]},{"osd":4,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84399999999999997,"5min":1.0089999999999999,"15min":2.5070000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.73799999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.92400000000000004,"15min":1.637},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.042,"15min":2.5579999999999998},"last":0.82999999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":1.248},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":0.95199999999999996,"15min":1.5980000000000001},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:09.634 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:09.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.856+0000 7fa0b89ff700 1 Processor -- start 2023-12-26T21:53:09.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.857+0000 7fa0b89ff700 1 -- start start 2023-12-26T21:53:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0b89ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b413c510 0x7fa0b40c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0b89ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0b406b8c0 0x7fa0b406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0b89ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b406c270 con 0x7fa0b40cc100 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0b89ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b40686c0 con 0x7fa0b406b8c0 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0b89ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4069520 con 0x7fa0b413c510 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0abfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0b406b8c0 0x7fa0b406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0abfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0b406b8c0 0x7fa0b406bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55754/0 (socket says 172.21.15.131:55754) 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0abfff700 1 -- 172.21.15.131:0/3019885696 learned_addr learned my addr 172.21.15.131:0/3019885696 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0b259c700 1 --2- 172.21.15.131:0/3019885696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b413c510 0x7fa0b40c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.858+0000 7fa0abfff700 1 -- 172.21.15.131:0/3019885696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b413c510 msgr2=0x7fa0b40c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.859+0000 7fa0abfff700 1 --2- 172.21.15.131:0/3019885696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b413c510 0x7fa0b40c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.859+0000 7fa0abfff700 1 -- 172.21.15.131:0/3019885696 >> v1:172.21.15.190:6789/0 conn(0x7fa0b40cc100 legacy=0x7fa0b40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.859+0000 7fa0abfff700 1 -- 172.21.15.131:0/3019885696 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4068910 con 0x7fa0b406b8c0 2023-12-26T21:53:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.859+0000 7fa0abfff700 1 --2- 172.21.15.131:0/3019885696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b406b8c0 0x7fa0b406bd30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0099b0 tx=0x7fa09c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9e412bab19d1c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.859+0000 7fa0b1d9b700 1 -- 172.21.15.131:0/3019885696 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c009ea0 con 0x7fa0b406b8c0 2023-12-26T21:53:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.859+0000 7fa0b1d9b700 1 -- 172.21.15.131:0/3019885696 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c00b860 con 0x7fa0b406b8c0 2023-12-26T21:53:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.859+0000 7fa0b1d9b700 1 -- 172.21.15.131:0/3019885696 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00bc00 con 0x7fa0b406b8c0 2023-12-26T21:53:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.860+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3019885696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b406b8c0 msgr2=0x7fa0b406bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.860+0000 7fa0b89ff700 1 --2- 172.21.15.131:0/3019885696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b406b8c0 0x7fa0b406bd30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0099b0 tx=0x7fa09c009250 comp rx=0 tx=0).stop 2023-12-26T21:53:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.860+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3019885696 shutdown_connections 2023-12-26T21:53:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.860+0000 7fa0b89ff700 1 --2- 172.21.15.131:0/3019885696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b413c510 0x7fa0b40c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.860+0000 7fa0b89ff700 1 --2- 172.21.15.131:0/3019885696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b406b8c0 0x7fa0b406bd30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.860+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3019885696 >> 172.21.15.131:0/3019885696 conn(0x7fa0b40ce3e0 msgr2=0x7fa0b40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.860+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3019885696 shutdown_connections 2023-12-26T21:53:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.860+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3019885696 wait complete. 2023-12-26T21:53:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.861+0000 7fa0b89ff700 1 Processor -- start 2023-12-26T21:53:09.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.861+0000 7fa0b89ff700 1 -- start start 2023-12-26T21:53:09.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.862+0000 7fa0b89ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406b8c0 0x7fa0b413aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.862+0000 7fa0b89ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40cc100 0x7fa0b40b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.862+0000 7fa0b89ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b413c510 0x7fa0b40c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.862+0000 7fa0b89ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4151fe0 con 0x7fa0b40cc100 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.862+0000 7fa0b89ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40c5490 con 0x7fa0b413c510 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.862+0000 7fa0b89ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4152230 con 0x7fa0b406b8c0 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40cc100 0x7fa0b40b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0b259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406b8c0 0x7fa0b413aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40cc100 0x7fa0b40b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55766/0 (socket says 172.21.15.131:55766) 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0abfff700 1 -- 172.21.15.131:0/3736144065 learned_addr learned my addr 172.21.15.131:0/3736144065 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0b2d9d700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b413c510 0x7fa0b40c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0abfff700 1 -- 172.21.15.131:0/3736144065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406b8c0 msgr2=0x7fa0b413aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0abfff700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406b8c0 0x7fa0b413aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0abfff700 1 -- 172.21.15.131:0/3736144065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b413c510 msgr2=0x7fa0b40c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0abfff700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b413c510 0x7fa0b40c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0abfff700 1 -- 172.21.15.131:0/3736144065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0094f0 con 0x7fa0b40cc100 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0b2d9d700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b413c510 0x7fa0b40c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.863+0000 7fa0abfff700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40cc100 0x7fa0b40b1c90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa09c006fd0 tx=0x7fa09c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.864+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3736144065 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c000d10 con 0x7fa0b40cc100 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.864+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3736144065 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c00bbe0 con 0x7fa0b40cc100 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.864+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b4152510 con 0x7fa0b40cc100 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.864+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3736144065 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c011650 con 0x7fa0b40cc100 2023-12-26T21:53:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.864+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b4152ad0 con 0x7fa0b40cc100 2023-12-26T21:53:09.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.865+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b40c5620 con 0x7fa0b40cc100 2023-12-26T21:53:09.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.866+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3736144065 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa09c011a80 con 0x7fa0b40cc100 2023-12-26T21:53:09.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.866+0000 7fa0aa7fc700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa094066f30 0x7fa0940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:09.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.866+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3736144065 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fa09c085fd0 con 0x7fa0b40cc100 2023-12-26T21:53:09.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.867+0000 7fa0b259c700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa094066f30 0x7fa0940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:09.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.867+0000 7fa0b259c700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa094066f30 0x7fa0940693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009da0 tx=0x7fa0a4009120 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:09.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:09.871+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3736144065 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa09c053010 con 0x7fa0b40cc100 2023-12-26T21:53:10.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.059+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0b40657e0 con 0x7fa094066f30 2023-12-26T21:53:10.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.067+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3736144065 <== mgr.5789 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278033 (secure 0 0 0) 0x7fa0b40657e0 con 0x7fa094066f30 2023-12-26T21:53:10.071 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:10.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa094066f30 msgr2=0x7fa0940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa094066f30 0x7fa0940693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009da0 tx=0x7fa0a4009120 comp rx=0 tx=0).stop 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40cc100 msgr2=0x7fa0b40b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40cc100 0x7fa0b40b1c90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa09c006fd0 tx=0x7fa09c017000 comp rx=0 tx=0).stop 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 shutdown_connections 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b413c510 0x7fa0b40c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406b8c0 0x7fa0b413aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa094066f30 0x7fa0940693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 --2- 172.21.15.131:0/3736144065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b40cc100 0x7fa0b40b1c90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.071+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 >> 172.21.15.131:0/3736144065 conn(0x7fa0b40ce3e0 msgr2=0x7fa0b40c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.072+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 shutdown_connections 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.072+0000 7fa0b89ff700 1 -- 172.21.15.131:0/3736144065 wait complete. 2023-12-26T21:53:10.074 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:10.090 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:53:09.075462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2433308,"kb_used_data":9948,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752541412,"statfs":{"total":773094113280,"available":770602405888,"internally_reserved":0,"allocated":10186752,"data_stored":5507971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.836671"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245654+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:01.295523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.249280+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:52:30.438143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241892+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:37:32.016433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.247029+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:40.412864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.246251+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:01.295523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245806+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:33:07.506029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248204+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:00:12.212821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245149+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:43.215374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.243193+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:01:47.175322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242805+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:50.422991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.246075+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:43.178061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248058+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:38:11.856188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248109+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:26:51.025444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245284+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:56.748993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245399+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:14:29.184662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245154+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:33:01.757024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242720+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:14:43.420525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241931+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:12.857594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241987+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:33.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.246301+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:07.445143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245832+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:47:44.760773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242453+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:07:55.195262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248812+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:44:47.409883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242202+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:36:43.945806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241940+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:58.054482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245513+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:48:01.129675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245537+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:18.916620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245618+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:23:09.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248855+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:09:46.836661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241921+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:15:35.448309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245367+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:43:17.471063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245188+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:01:23.220141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'183","reported_seq":461,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-26T21:53:02.161129+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:02.161129+0000","last_peered":"2023-12-26T21:53:02.161129+0000","last_clean":"2023-12-26T21:53:02.161129+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:02.161129+0000","last_undegraded":"2023-12-26T21:53:02.161129+0000","last_fullsized":"2023-12-26T21:53:02.161129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.647},"last":0.497},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.61199999999999999,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.92300000000000004,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":1,"15min":2.0150000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.58199999999999996,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.014,"5min":1.048,"15min":1.1359999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.59699999999999998,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":1.038,"15min":1.8640000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.498,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.419},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":0.93799999999999994,"5min":1.0189999999999999,"15min":1.327},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.65000000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.393},"last":0.67100000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.374,"15min":0.371},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72799999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.85999999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509636,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":535482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":1.0029999999999999,"15min":1.085},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.3380000000000001},"last":0.85499999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":1.288},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.876,"5min":0.92000000000000004,"15min":1.113},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.92000000000000004,"5min":1.026,"15min":1.569},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.97799999999999998,"15min":1.3839999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.62,"15min":0.62},"min":{"1min":0.42699999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.137},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.91200000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.69599999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95099999999999996,"5min":1.0049999999999999,"15min":1.296},"last":0.82099999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.437},"max":{"1min":0.89200000000000002,"5min":0.96899999999999997,"15min":1.51},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.91700000000000004,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.94799999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307160,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064680,"statfs":{"total":96636764160,"available":96322232320,"internally_reserved":0,"allocated":1720320,"data_stored":1132047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.34699999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.38},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.42899999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.745}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.56200000000000006}]}]},{"osd":0,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302484,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069356,"statfs":{"total":96636764160,"available":96327020544,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.439},"min":{"1min":0.316,"5min":0.245,"15min":0.245},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":1.5640000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69999999999999996,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.371,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.58599999999999997,"5min":0.66800000000000004,"15min":2.012},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.47699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.42299999999999999,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":4.2610000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.41799999999999998,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.498}]},{"osd":4,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.94699999999999995,"5min":1.1539999999999999,"15min":4.423},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.95799999999999996,"5min":1.069,"15min":1.6439999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78200000000000003,"15min":0.80900000000000005},"min":{"1min":0.57799999999999996,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":4.3650000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.008,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76700000000000002,"15min":0.78500000000000003},"min":{"1min":0.623,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.89000000000000001,"5min":1.103,"15min":1.611},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.55200000000000005,"5min":0.504,"15min":0.5},"max":{"1min":1.012,"5min":1.111,"15min":4.407},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.81000000000000005},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":1.145,"15min":4.5069999999999997},"last":0.57799999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306580,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065260,"statfs":{"total":96636764160,"available":96322826240,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45800000000000002,"15min":0.45900000000000002},"min":{"1min":0.33200000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":1.915},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.30399999999999999},"max":{"1min":0.63200000000000001,"5min":0.83899999999999997,"15min":1.7729999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.39600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":1.952},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47999999999999998},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.30199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.318}]},{"osd":3,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.375},"max":{"1min":1,"5min":1,"15min":1},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73899999999999999,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.495}]},{"osd":4,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.54600000000000004,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.96999999999999997,"5min":0.98299999999999998,"15min":1.284},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.60099999999999998,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.0600000000000001,"15min":1.5029999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":0.94599999999999995,"5min":1.014,"15min":1.355},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.93000000000000005,"5min":1.038,"15min":1.7350000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.95999999999999996,"5min":1.0609999999999999,"15min":1.349},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.47799999999999998},"max":{"1min":0.92700000000000005,"5min":1.073,"15min":1.841},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0189999999999999,"15min":1.294},"last":0.56100000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310676,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061164,"statfs":{"total":96636764160,"available":96318631936,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.40200000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.58299999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.498},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.75}]}]},{"osd":4,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302484,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069356,"statfs":{"total":96636764160,"available":96327020544,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66000000000000003,"15min":0.68400000000000005},"min":{"1min":0.439,"5min":0.307,"15min":0.307},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":2.1389999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.96399999999999997,"5min":1.008,"15min":1.407},"last":0.73499999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.52700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.879},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69799999999999995,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.89200000000000002,"5min":0.98699999999999999,"15min":2.097},"last":1.0209999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70999999999999996,"15min":0.72199999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91000000000000003,"5min":0.94999999999999996,"15min":1.7809999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88800000000000001,"5min":0.97199999999999998,"15min":1.6870000000000001},"last":0.72099999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69999999999999996,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95999999999999996,"15min":1.956},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.746},"min":{"1min":0.626,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.0609999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58899999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82099999999999995,"5min":0.86399999999999999,"15min":2.2429999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84899999999999998,"5min":0.92600000000000005,"15min":2.194},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.621,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.379},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":2.1589999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":1.133}]},{"osd":7,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.432},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":2.23},"last":1.002},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.745},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":2.5169999999999999},"last":1.266}]}]},{"osd":5,"up_from":15,"seq":64424509635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303064,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068776,"statfs":{"total":96636764160,"available":96326426624,"internally_reserved":0,"allocated":1720320,"data_stored":1132047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.878,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.71299999999999997},"min":{"1min":0.48899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.71},"last":0.81000000000000005}]},{"osd":1,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.57399999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96399999999999997,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68200000000000005,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.002,"15min":1.4219999999999999},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.871,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68799999999999994,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":1.214},"last":0.69199999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":1.052,"15min":1.4890000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":0.92200000000000004,"15min":1.1439999999999999},"last":0.624}]},{"osd":4,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84399999999999997,"5min":1.0089999999999999,"15min":2.5070000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.73799999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.92400000000000004,"15min":1.637},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.042,"15min":2.5579999999999998},"last":0.82999999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":1.248},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":0.95199999999999996,"15min":1.5980000000000001},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:10.095 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:10.233 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-12-26T21:53:10.232+0000 7f2fdb3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:53:10.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.320+0000 7f4ee69ff700 1 Processor -- start 2023-12-26T21:53:10.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.320+0000 7f4ee69ff700 1 -- start start 2023-12-26T21:53:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.321+0000 7f4ee69ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee0159a70 0x7f4ee00b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.321+0000 7f4ee69ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ee00b2a90 0x7f4ee00b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.321+0000 7f4ee69ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee00afcd0 con 0x7f4ee01596a0 2023-12-26T21:53:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.321+0000 7f4ee69ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee00b3e70 con 0x7f4ee00b2a90 2023-12-26T21:53:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.321+0000 7f4ee69ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00b0f20 con 0x7f4ee0159a70 2023-12-26T21:53:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4ee01596a0 0x7f4ee00b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49304/0 (socket says 172.21.15.131:49304) 2023-12-26T21:53:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edffff700 1 -- 172.21.15.131:0/898264432 learned_addr learned my addr 172.21.15.131:0/898264432 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4ee4f9c700 1 --2- 172.21.15.131:0/898264432 >> v2:172.21.15.190:3300/0 conn(0x7f4ee00b2a90 0x7f4ee00b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edf7fe700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee0159a70 0x7f4ee00b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73533942 0 0) 0x7f4ee00afcd0 con 0x7f4ee01596a0 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec8003570 con 0x7f4ee01596a0 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4042979445 0 0) 0x7f4ec8003570 con 0x7f4ee01596a0 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee0159a70 msgr2=0x7f4ee00b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee0159a70 0x7f4ee00b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 >> v2:172.21.15.190:3300/0 conn(0x7f4ee00b2a90 msgr2=0x7f4ee00b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 --2- 172.21.15.131:0/898264432 >> v2:172.21.15.190:3300/0 conn(0x7f4ee00b2a90 0x7f4ee00b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00b3540 con 0x7f4ee01596a0 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f4ed0001470 con 0x7f4ee01596a0 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4ee4f9c700 1 --2- 172.21.15.131:0/898264432 >> v2:172.21.15.190:3300/0 conn(0x7f4ee00b2a90 0x7f4ee00b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 >> v1:172.21.15.131:6789/0 conn(0x7f4ee01596a0 legacy=0x7f4ee00b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee0159a70 0x7f4ec8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.322+0000 7f4edeffd700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ec8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edeffd700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8004d50 0x7f4ec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00b3e70 con 0x7f4ec8003fe0 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00b0f20 con 0x7f4ec8004d50 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec800ba90 con 0x7f4ee0159a70 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edf7fe700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ec8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edffff700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee0159a70 0x7f4ec8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4ee4f9c700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8004d50 0x7f4ec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edf7fe700 1 -- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee0159a70 msgr2=0x7f4ec8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:10.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edf7fe700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee0159a70 0x7f4ec8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edf7fe700 1 -- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8004d50 msgr2=0x7f4ec80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edf7fe700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ec8004d50 0x7f4ec80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edf7fe700 1 -- 172.21.15.131:0/898264432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec8003570 con 0x7f4ec8003fe0 2023-12-26T21:53:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.323+0000 7f4edf7fe700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ec8004810 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009c00 tx=0x7f4ed40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0996d3ecf4437ac server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400e6b0 con 0x7f4ec8003fe0 2023-12-26T21:53:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed400e810 con 0x7f4ec8003fe0 2023-12-26T21:53:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4edeffd700 1 -- 172.21.15.131:0/898264432 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400ebd0 con 0x7f4ec8003fe0 2023-12-26T21:53:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4ee69ff700 1 -- 172.21.15.131:0/898264432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 msgr2=0x7f4ec8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4ee69ff700 1 --2- 172.21.15.131:0/898264432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ec8004810 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009c00 tx=0x7f4ed40091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4ee4f9c700 1 -- 172.21.15.131:0/898264432 reap_dead start 2023-12-26T21:53:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4ee69ff700 1 -- 172.21.15.131:0/898264432 shutdown_connections 2023-12-26T21:53:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4ee69ff700 1 -- 172.21.15.131:0/898264432 >> 172.21.15.131:0/898264432 conn(0x7f4ee006ece0 msgr2=0x7f4ee006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4ee69ff700 1 -- 172.21.15.131:0/898264432 shutdown_connections 2023-12-26T21:53:10.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.324+0000 7f4ee69ff700 1 -- 172.21.15.131:0/898264432 wait complete. 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.326+0000 7f4ee69ff700 1 Processor -- start 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.327+0000 7f4ee69ff700 1 -- start start 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4ee69ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ee00c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4ee69ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee00d0810 0x7f4ee00d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4ee69ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee016d450 0x7f4ee016d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4ee69ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee015b6a0 con 0x7f4ec8003fe0 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4ee69ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee016dde0 con 0x7f4ee00d0810 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4ee69ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee015b090 con 0x7f4ee016d450 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4edffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ee00c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4edffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ee00c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55776/0 (socket says 172.21.15.131:55776) 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4edffff700 1 -- 172.21.15.131:0/99831951 learned_addr learned my addr 172.21.15.131:0/99831951 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4edffff700 1 -- 172.21.15.131:0/99831951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee016d450 msgr2=0x7f4ee016d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4edffff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee016d450 0x7f4ee016d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4edffff700 1 -- 172.21.15.131:0/99831951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee00d0810 msgr2=0x7f4ee00d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4edffff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee00d0810 0x7f4ee00d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.328+0000 7f4edffff700 1 -- 172.21.15.131:0/99831951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00ceef0 con 0x7f4ec8003fe0 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.329+0000 7f4edffff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ee00c9fa0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00016e0 tx=0x7f4ee00b3de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.329+0000 7f4ec7fff700 1 -- 172.21.15.131:0/99831951 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0018030 con 0x7f4ec8003fe0 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.329+0000 7f4ec7fff700 1 -- 172.21.15.131:0/99831951 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed000f8d0 con 0x7f4ec8003fe0 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.329+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee00cf1d0 con 0x7f4ec8003fe0 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.329+0000 7f4ec7fff700 1 -- 172.21.15.131:0/99831951 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000fc70 con 0x7f4ec8003fe0 2023-12-26T21:53:10.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.329+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee00c8410 con 0x7f4ec8003fe0 2023-12-26T21:53:10.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.330+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee00657e0 con 0x7f4ec8003fe0 2023-12-26T21:53:10.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.331+0000 7f4ec7fff700 1 -- 172.21.15.131:0/99831951 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4ed001b070 con 0x7f4ec8003fe0 2023-12-26T21:53:10.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.332+0000 7f4ec7fff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4ecc066f30 0x7f4ecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.332+0000 7f4ec7fff700 1 -- 172.21.15.131:0/99831951 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f4ed0019070 con 0x7f4ec8003fe0 2023-12-26T21:53:10.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.332+0000 7f4edf7fe700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4ecc066f30 0x7f4ecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.332+0000 7f4edf7fe700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4ecc066f30 0x7f4ecc0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00883b0 tx=0x7f4ed401a000 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:10.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.336+0000 7f4ec7fff700 1 -- 172.21.15.131:0/99831951 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ed0051840 con 0x7f4ec8003fe0 2023-12-26T21:53:10.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.528+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ee00cf410 con 0x7f4ecc066f30 2023-12-26T21:53:10.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.536+0000 7f4ec7fff700 1 -- 172.21.15.131:0/99831951 <== mgr.5789 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278033 (secure 0 0 0) 0x7f4ed40095b0 con 0x7f4ecc066f30 2023-12-26T21:53:10.541 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.539+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4ecc066f30 msgr2=0x7f4ecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.539+0000 7f4ee69ff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4ecc066f30 0x7f4ecc0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00883b0 tx=0x7f4ed401a000 comp rx=0 tx=0).stop 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 msgr2=0x7f4ee00c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ee00c9fa0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00016e0 tx=0x7f4ee00b3de0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 shutdown_connections 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ee016d450 0x7f4ee016d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4ecc066f30 0x7f4ecc0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ec8003fe0 0x7f4ee00c9fa0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 --2- 172.21.15.131:0/99831951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee00d0810 0x7f4ee00d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 >> 172.21.15.131:0/99831951 conn(0x7f4ee006ece0 msgr2=0x7f4ee0156650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 shutdown_connections 2023-12-26T21:53:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.540+0000 7f4ee69ff700 1 -- 172.21.15.131:0/99831951 wait complete. 2023-12-26T21:53:10.545 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:10.558 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:53:09.075462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2433308,"kb_used_data":9948,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752541412,"statfs":{"total":773094113280,"available":770602405888,"internally_reserved":0,"allocated":10186752,"data_stored":5507971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.836671"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245654+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:01.295523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.249280+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:52:30.438143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241892+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:37:32.016433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.247029+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:40.412864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.246251+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:01.295523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245806+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:33:07.506029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248204+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:00:12.212821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245149+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:43.215374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.243193+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:01:47.175322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242805+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:50.422991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.246075+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:43.178061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248058+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:38:11.856188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248109+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:26:51.025444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245284+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:07:56.748993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245399+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:14:29.184662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245154+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:33:01.757024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242720+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:14:43.420525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241931+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:12.857594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241987+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:33.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.246301+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:07.445143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245832+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:47:44.760773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242453+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:07:55.195262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248812+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:44:47.409883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.242202+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:36:43.945806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241940+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:58.054482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245513+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:48:01.129675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245537+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:18.916620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245618+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:23:09.747188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.248855+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:09:46.836661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.241921+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:15:35.448309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245367+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:43:17.471063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:04.238368+0000","last_change":"2023-12-26T21:53:04.238368+0000","last_active":"2023-12-26T21:53:04.238368+0000","last_peered":"2023-12-26T21:53:04.238368+0000","last_clean":"2023-12-26T21:53:04.238368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:04.238368+0000","last_undegraded":"2023-12-26T21:53:04.238368+0000","last_fullsized":"2023-12-26T21:53:04.238368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:03.232814+0000","last_change":"2023-12-26T21:53:03.245188+0000","last_active":"2023-12-26T21:53:03.232814+0000","last_peered":"2023-12-26T21:53:03.232814+0000","last_clean":"2023-12-26T21:53:03.232814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:03.232814+0000","last_undegraded":"2023-12-26T21:53:03.232814+0000","last_fullsized":"2023-12-26T21:53:03.232814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:01:23.220141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'183","reported_seq":461,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-26T21:53:02.161129+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:02.161129+0000","last_peered":"2023-12-26T21:53:02.161129+0000","last_clean":"2023-12-26T21:53:02.161129+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:02.161129+0000","last_undegraded":"2023-12-26T21:53:02.161129+0000","last_fullsized":"2023-12-26T21:53:02.161129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.647},"last":0.497},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.61199999999999999,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.92300000000000004,"5min":0.98299999999999998,"15min":2.9750000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":1,"15min":2.0150000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.58199999999999996,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.014,"5min":1.048,"15min":1.1359999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.59699999999999998,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":1.038,"15min":1.8640000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.498,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.419},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76900000000000002,"15min":0.77600000000000002},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.46800000000000003},"max":{"1min":0.93799999999999994,"5min":1.0189999999999999,"15min":1.327},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.65000000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.393},"last":0.67100000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.36399999999999999},"max":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.374,"15min":0.371},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72799999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:51:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.437},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.85999999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509636,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":535482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":1.0029999999999999,"15min":1.085},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.3380000000000001},"last":0.85499999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":1.288},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.876,"5min":0.92000000000000004,"15min":1.113},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.92000000000000004,"5min":1.026,"15min":1.569},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.97799999999999998,"15min":1.3839999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.62,"15min":0.62},"min":{"1min":0.42699999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.137},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.91200000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.69599999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95099999999999996,"5min":1.0049999999999999,"15min":1.296},"last":0.82099999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.437},"max":{"1min":0.89200000000000002,"5min":0.96899999999999997,"15min":1.51},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.91700000000000004,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.94799999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307160,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064680,"statfs":{"total":96636764160,"available":96322232320,"internally_reserved":0,"allocated":1720320,"data_stored":1132047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.34699999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.38},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.42899999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.745}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.56200000000000006}]}]},{"osd":0,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302484,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069356,"statfs":{"total":96636764160,"available":96327020544,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.439},"min":{"1min":0.316,"5min":0.245,"15min":0.245},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":1.5640000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69999999999999996,"5min":0.80000000000000004,"15min":4.3150000000000004},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.371,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.58599999999999997,"5min":0.66800000000000004,"15min":2.012},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.47699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":0.71199999999999997,"15min":4.6479999999999997},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.55400000000000005},"min":{"1min":0.42299999999999999,"5min":0.35499999999999998,"15min":0.34300000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":4.2610000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.48299999999999998,"5min":0.41799999999999998,"15min":0.40500000000000003},"max":{"1min":0.76800000000000002,"5min":0.86699999999999999,"15min":2.23},"last":0.498}]},{"osd":4,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73299999999999998,"15min":0.76300000000000001},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.94699999999999995,"5min":1.1539999999999999,"15min":4.423},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":0.95799999999999996,"5min":1.069,"15min":1.6439999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78200000000000003,"15min":0.80900000000000005},"min":{"1min":0.57799999999999996,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":4.3650000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.008,"5min":1.1240000000000001,"15min":4.5519999999999996},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76700000000000002,"15min":0.78500000000000003},"min":{"1min":0.623,"5min":0.55000000000000004,"15min":0.45300000000000001},"max":{"1min":0.89000000000000001,"5min":1.103,"15min":1.611},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.55200000000000005,"5min":0.504,"15min":0.5},"max":{"1min":1.012,"5min":1.111,"15min":4.407},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.81000000000000005},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.1160000000000001,"15min":4.6070000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":1.145,"15min":4.5069999999999997},"last":0.57799999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306580,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065260,"statfs":{"total":96636764160,"available":96322826240,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45800000000000002,"15min":0.45900000000000002},"min":{"1min":0.33200000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":1.915},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.30399999999999999},"max":{"1min":0.63200000000000001,"5min":0.83899999999999997,"15min":1.7729999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.39600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":1.952},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47999999999999998},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.30199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.318}]},{"osd":3,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.375},"max":{"1min":1,"5min":1,"15min":1},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73899999999999999,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.495}]},{"osd":4,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.54600000000000004,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.96999999999999997,"5min":0.98299999999999998,"15min":1.284},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.60099999999999998,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.0600000000000001,"15min":1.5029999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":0.94599999999999995,"5min":1.014,"15min":1.355},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.93000000000000005,"5min":1.038,"15min":1.7350000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.95999999999999996,"5min":1.0609999999999999,"15min":1.349},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.47799999999999998},"max":{"1min":0.92700000000000005,"5min":1.073,"15min":1.841},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0189999999999999,"15min":1.294},"last":0.56100000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310676,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061164,"statfs":{"total":96636764160,"available":96318631936,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.40200000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.58299999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.498},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.75}]}]},{"osd":4,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302484,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069356,"statfs":{"total":96636764160,"available":96327020544,"internally_reserved":0,"allocated":1126400,"data_stored":541679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66000000000000003,"15min":0.68400000000000005},"min":{"1min":0.439,"5min":0.307,"15min":0.307},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":2.1389999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.96399999999999997,"5min":1.008,"15min":1.407},"last":0.73499999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.52700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.879},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69799999999999995,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.89200000000000002,"5min":0.98699999999999999,"15min":2.097},"last":1.0209999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70999999999999996,"15min":0.72199999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91000000000000003,"5min":0.94999999999999996,"15min":1.7809999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88800000000000001,"5min":0.97199999999999998,"15min":1.6870000000000001},"last":0.72099999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69999999999999996,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.91000000000000003,"5min":0.95999999999999996,"15min":1.956},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.746},"min":{"1min":0.626,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.0609999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58899999999999997,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82099999999999995,"5min":0.86399999999999999,"15min":2.2429999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84899999999999998,"5min":0.92600000000000005,"15min":2.194},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.621,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.379},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":2.1589999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":2.3450000000000002},"last":1.133}]},{"osd":7,"last update":"Tue Dec 26 21:52:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.432},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":2.23},"last":1.002},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.745},"min":{"1min":0.57999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":2.5169999999999999},"last":1.266}]}]},{"osd":5,"up_from":15,"seq":64424509635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303064,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068776,"statfs":{"total":96636764160,"available":96326426624,"internally_reserved":0,"allocated":1720320,"data_stored":1132047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.878,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.71299999999999997},"min":{"1min":0.48899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.71},"last":0.81000000000000005}]},{"osd":1,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.57399999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96399999999999997,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68200000000000005,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.002,"15min":1.4219999999999999},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.871,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68799999999999994,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":1.214},"last":0.69199999999999995}]},{"osd":3,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":1.052,"15min":1.4890000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":0.92200000000000004,"15min":1.1439999999999999},"last":0.624}]},{"osd":4,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84399999999999997,"5min":1.0089999999999999,"15min":2.5070000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.73799999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.92400000000000004,"15min":1.637},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.042,"15min":2.5579999999999998},"last":0.82999999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":1.248},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":0.95199999999999996,"15min":1.5980000000000001},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:10.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-12-26T21:53:10.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-12-26T21:53:10.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-12-26T21:53:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-12-26T21:53:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-12-26T21:53:10.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-12-26T21:53:10.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-12-26T21:53:10.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-12-26T21:53:10.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-12-26T21:53:10.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245654+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:19:01.295523+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:10.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-12-26T21:53:10.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-12-26T21:53:10.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.249280+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:52:30.438143+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:53:10.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-12-26T21:53:10.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-12-26T21:53:10.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.241892+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:37:32.016433+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.247029+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T04:11:40.412864+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.246251+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:19:01.295523+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245806+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T07:33:07.506029+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.248204+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:00:12.212821+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245149+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:37:43.215374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:53:10.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.243193+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:01:47.175322+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.242805+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:52:50.422991+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-12-26T21:53:10.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.246075+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:16:43.178061+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:53:10.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-12-26T21:53:10.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-12-26T21:53:10.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.248058+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:38:11.856188+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:53:10.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-12-26T21:53:10.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-12-26T21:53:10.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.248109+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:26:51.025444+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245284+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:07:56.748993+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245399+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:14:29.184662+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-12-26T21:53:10.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245154+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:33:01.757024+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.242720+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:14:43.420525+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.241931+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:08:12.857594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-12-26T21:53:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.241987+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:45:33.045124+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.246301+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:19:07.445143+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245832+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:47:44.760773+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-12-26T21:53:10.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.242453+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T07:07:55.195262+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.248812+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T22:44:47.409883+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-12-26T21:53:10.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.242202+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:36:43.945806+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.241940+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:21:58.054482+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245513+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:48:01.129675+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-12-26T21:53:10.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245537+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T04:40:18.916620+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245618+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:23:09.747188+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.248855+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:09:46.836661+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-12-26T21:53:10.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.241921+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:15:35.448309+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245367+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T22:43:17.471063+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:04.238368+0000', 'last_change': '2023-12-26T21:53:04.238368+0000', 'last_active': '2023-12-26T21:53:04.238368+0000', 'last_peered': '2023-12-26T21:53:04.238368+0000', 'last_clean': '2023-12-26T21:53:04.238368+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:04.238368+0000', 'last_undegraded': '2023-12-26T21:53:04.238368+0000', 'last_fullsized': '2023-12-26T21:53:04.238368+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:04.238368+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-12-26T21:53:10.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:03.232814+0000', 'last_change': '2023-12-26T21:53:03.245188+0000', 'last_active': '2023-12-26T21:53:03.232814+0000', 'last_peered': '2023-12-26T21:53:03.232814+0000', 'last_clean': '2023-12-26T21:53:03.232814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:03.232814+0000', 'last_undegraded': '2023-12-26T21:53:03.232814+0000', 'last_fullsized': '2023-12-26T21:53:03.232814+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:03.232814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:01:23.220141+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:53:10.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-26T21:53:10.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.787+0000 7fbd617ff700 1 Processor -- start 2023-12-26T21:53:10.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.787+0000 7fbd617ff700 1 -- start start 2023-12-26T21:53:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.788+0000 7fbd617ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbd5c0c8bf0 0x7fbd5c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.788+0000 7fbd617ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c069d10 0x7fbd5c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.788+0000 7fbd617ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5c0caa10 con 0x7fbd5c0c8820 2023-12-26T21:53:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.788+0000 7fbd617ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c157a80 con 0x7fbd5c0c8bf0 2023-12-26T21:53:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.788+0000 7fbd617ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0cac50 con 0x7fbd5c069d10 2023-12-26T21:53:10.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c069d10 0x7fbd5c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5a7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbd5c0c8bf0 0x7fbd5c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c069d10 0x7fbd5c0b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:50906/0 (socket says 172.21.15.131:50906) 2023-12-26T21:53:10.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5b7fe700 1 -- 172.21.15.131:0/3073972848 learned_addr learned my addr 172.21.15.131:0/3073972848 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:10.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5a7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbd5c0c8bf0 0x7fbd5c0b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:60642/0 (socket says 172.21.15.131:60642) 2023-12-26T21:53:10.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5affd700 1 --1- 172.21.15.131:0/3073972848 >> v1:172.21.15.190:6789/0 conn(0x7fbd5c0c8820 0x7fbd5c0ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:36292/0 (socket says 172.21.15.131:36292) 2023-12-26T21:53:10.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5b7fe700 1 -- 172.21.15.131:0/3073972848 >> v2:172.21.15.190:3300/0 conn(0x7fbd5c0c8bf0 msgr2=0x7fbd5c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:10.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5b7fe700 1 --2- 172.21.15.131:0/3073972848 >> v2:172.21.15.190:3300/0 conn(0x7fbd5c0c8bf0 0x7fbd5c0b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5b7fe700 1 -- 172.21.15.131:0/3073972848 >> v1:172.21.15.190:6789/0 conn(0x7fbd5c0c8820 legacy=0x7fbd5c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.789+0000 7fbd5b7fe700 1 -- 172.21.15.131:0/3073972848 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c0caf30 con 0x7fbd5c069d10 2023-12-26T21:53:10.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.790+0000 7fbd5a7fc700 1 --2- 172.21.15.131:0/3073972848 >> v2:172.21.15.190:3300/0 conn(0x7fbd5c0c8bf0 0x7fbd5c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:10.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.790+0000 7fbd5b7fe700 1 --2- 172.21.15.131:0/3073972848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c069d10 0x7fbd5c0b1820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009f20 tx=0x7fbd54009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c33399fdd4ba947e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:10.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.790+0000 7fbd59ffb700 1 -- 172.21.15.131:0/3073972848 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5400e070 con 0x7fbd5c069d10 2023-12-26T21:53:10.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.790+0000 7fbd59ffb700 1 -- 172.21.15.131:0/3073972848 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5400b9d0 con 0x7fbd5c069d10 2023-12-26T21:53:10.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.790+0000 7fbd59ffb700 1 -- 172.21.15.131:0/3073972848 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd540092e0 con 0x7fbd5c069d10 2023-12-26T21:53:10.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.791+0000 7fbd617ff700 1 -- 172.21.15.131:0/3073972848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c069d10 msgr2=0x7fbd5c0b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.791+0000 7fbd617ff700 1 --2- 172.21.15.131:0/3073972848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c069d10 0x7fbd5c0b1820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009f20 tx=0x7fbd54009840 comp rx=0 tx=0).stop 2023-12-26T21:53:10.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.791+0000 7fbd617ff700 1 -- 172.21.15.131:0/3073972848 shutdown_connections 2023-12-26T21:53:10.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.791+0000 7fbd617ff700 1 --2- 172.21.15.131:0/3073972848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c069d10 0x7fbd5c0b1820 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.791+0000 7fbd617ff700 1 --2- 172.21.15.131:0/3073972848 >> v2:172.21.15.190:3300/0 conn(0x7fbd5c0c8bf0 0x7fbd5c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.791+0000 7fbd617ff700 1 -- 172.21.15.131:0/3073972848 >> 172.21.15.131:0/3073972848 conn(0x7fbd5c068040 msgr2=0x7fbd5c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:10.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.791+0000 7fbd617ff700 1 -- 172.21.15.131:0/3073972848 shutdown_connections 2023-12-26T21:53:10.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.791+0000 7fbd617ff700 1 -- 172.21.15.131:0/3073972848 wait complete. 2023-12-26T21:53:10.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.792+0000 7fbd617ff700 1 Processor -- start 2023-12-26T21:53:10.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.792+0000 7fbd617ff700 1 -- start start 2023-12-26T21:53:10.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.793+0000 7fbd617ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd5c069d10 0x7fbd5c1487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.793+0000 7fbd617ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c0c8820 0x7fbd5c143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.793+0000 7fbd617ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd5c0c8bf0 0x7fbd5c143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.793+0000 7fbd617ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1452e0 con 0x7fbd5c069d10 2023-12-26T21:53:10.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.793+0000 7fbd617ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1446d0 con 0x7fbd5c0c8bf0 2023-12-26T21:53:10.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.793+0000 7fbd617ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1449b0 con 0x7fbd5c0c8820 2023-12-26T21:53:10.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd5c069d10 0x7fbd5c1487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd5c069d10 0x7fbd5c1487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55788/0 (socket says 172.21.15.131:55788) 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5affd700 1 -- 172.21.15.131:0/4219822413 learned_addr learned my addr 172.21.15.131:0/4219822413 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5a7fc700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c0c8820 0x7fbd5c143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5b7fe700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd5c0c8bf0 0x7fbd5c143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5affd700 1 -- 172.21.15.131:0/4219822413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c0c8820 msgr2=0x7fbd5c143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5affd700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c0c8820 0x7fbd5c143960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5affd700 1 -- 172.21.15.131:0/4219822413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd5c0c8bf0 msgr2=0x7fbd5c143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5affd700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd5c0c8bf0 0x7fbd5c143ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5affd700 1 -- 172.21.15.131:0/4219822413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd50009420 con 0x7fbd5c069d10 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5b7fe700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd5c0c8bf0 0x7fbd5c143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd5affd700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd5c069d10 0x7fbd5c1487c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c7c0 tx=0x7fbd4c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.794+0000 7fbd42ffd700 1 -- 172.21.15.131:0/4219822413 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c005a70 con 0x7fbd5c069d10 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.795+0000 7fbd42ffd700 1 -- 172.21.15.131:0/4219822413 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c005bd0 con 0x7fbd5c069d10 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.795+0000 7fbd42ffd700 1 -- 172.21.15.131:0/4219822413 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0070d0 con 0x7fbd5c069d10 2023-12-26T21:53:10.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.795+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd54009ae0 con 0x7fbd5c069d10 2023-12-26T21:53:10.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.795+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c144f70 con 0x7fbd5c069d10 2023-12-26T21:53:10.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.796+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5c0657e0 con 0x7fbd5c069d10 2023-12-26T21:53:10.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.797+0000 7fbd42ffd700 1 -- 172.21.15.131:0/4219822413 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbd4c007480 con 0x7fbd5c069d10 2023-12-26T21:53:10.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.797+0000 7fbd42ffd700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbd48066f30 0x7fbd480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:10.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.798+0000 7fbd42ffd700 1 -- 172.21.15.131:0/4219822413 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fbd4c083a40 con 0x7fbd5c069d10 2023-12-26T21:53:10.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.798+0000 7fbd5a7fc700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbd48066f30 0x7fbd480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:10.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.798+0000 7fbd5a7fc700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbd48066f30 0x7fbd480693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0c3450 tx=0x7fbd500091a0 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:10.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:10.802+0000 7fbd42ffd700 1 -- 172.21.15.131:0/4219822413 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd4c085e60 con 0x7fbd5c069d10 2023-12-26T21:53:11.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.071+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbd5c0caa10 con 0x7fbd5c069d10 2023-12-26T21:53:11.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.071+0000 7fbd42ffd700 1 -- 172.21.15.131:0/4219822413 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1112 (secure 0 0 0) 0x7fbd5c0caa10 con 0x7fbd5c069d10 2023-12-26T21:53:11.072 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:11.073 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"ab603302-d88b-429a-be64-92b3c298adad","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1006,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1703626596,"num_in_osds":8,"osd_in_since":1703626587,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2491707392,"bytes_avail":770602405888,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":42,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":6708}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-12-26T21:51:27.533973+0000","services":{}},"progress_events":{}} 2023-12-26T21:53:11.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbd48066f30 msgr2=0x7fbd480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:11.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbd48066f30 0x7fbd480693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0c3450 tx=0x7fbd500091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd5c069d10 msgr2=0x7fbd5c1487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:11.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd5c069d10 0x7fbd5c1487c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c7c0 tx=0x7fbd4c007f70 comp rx=0 tx=0).stop 2023-12-26T21:53:11.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 shutdown_connections 2023-12-26T21:53:11.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd5c0c8bf0 0x7fbd5c143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbd48066f30 0x7fbd480693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd5c069d10 0x7fbd5c1487c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 --2- 172.21.15.131:0/4219822413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd5c0c8820 0x7fbd5c143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 >> 172.21.15.131:0/4219822413 conn(0x7fbd5c068040 msgr2=0x7fbd5c0b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:11.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 shutdown_connections 2023-12-26T21:53:11.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.074+0000 7fbd617ff700 1 -- 172.21.15.131:0/4219822413 wait complete. 2023-12-26T21:53:11.092 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-26T21:53:11.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.281+0000 7fb1acdff700 1 Processor -- start 2023-12-26T21:53:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.281+0000 7fb1acdff700 1 -- start start 2023-12-26T21:53:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.282+0000 7fb1acdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1a8159560 0x7fb1a815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.282+0000 7fb1acdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a806a1a0 0x7fb1a8068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1acdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a8068a80 con 0x7fb1a8159930 2023-12-26T21:53:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1acdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a80cac50 con 0x7fb1a8159560 2023-12-26T21:53:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1acdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a815da00 con 0x7fb1a806a1a0 2023-12-26T21:53:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1a659c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1a8159560 0x7fb1a815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1a659c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1a8159560 0x7fb1a815d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55800/0 (socket says 172.21.15.131:55800) 2023-12-26T21:53:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1a659c700 1 -- 172.21.15.131:0/2201607398 learned_addr learned my addr 172.21.15.131:0/2201607398 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1a6d9d700 1 --2- 172.21.15.131:0/2201607398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a806a1a0 0x7fb1a8068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:11.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1a659c700 1 -- 172.21.15.131:0/2201607398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a806a1a0 msgr2=0x7fb1a8068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:11.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1a659c700 1 --2- 172.21.15.131:0/2201607398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a806a1a0 0x7fb1a8068f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1a659c700 1 -- 172.21.15.131:0/2201607398 >> v1:172.21.15.190:6789/0 conn(0x7fb1a8159930 legacy=0x7fb1a80b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:11.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.283+0000 7fb1a659c700 1 -- 172.21.15.131:0/2201607398 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a815dce0 con 0x7fb1a8159560 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1a659c700 1 --2- 172.21.15.131:0/2201607398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8159560 0x7fb1a815d300 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb1980099b0 tx=0x7fb198009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4491803646df49 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1a559a700 1 -- 172.21.15.131:0/2201607398 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198009ea0 con 0x7fb1a8159560 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1a559a700 1 -- 172.21.15.131:0/2201607398 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19800b860 con 0x7fb1a8159560 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1a559a700 1 -- 172.21.15.131:0/2201607398 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800bc00 con 0x7fb1a8159560 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1acdff700 1 -- 172.21.15.131:0/2201607398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8159560 msgr2=0x7fb1a815d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1acdff700 1 --2- 172.21.15.131:0/2201607398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8159560 0x7fb1a815d300 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb1980099b0 tx=0x7fb198009250 comp rx=0 tx=0).stop 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1acdff700 1 -- 172.21.15.131:0/2201607398 shutdown_connections 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1acdff700 1 --2- 172.21.15.131:0/2201607398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a806a1a0 0x7fb1a8068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1acdff700 1 --2- 172.21.15.131:0/2201607398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a8159560 0x7fb1a815d300 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1acdff700 1 -- 172.21.15.131:0/2201607398 >> 172.21.15.131:0/2201607398 conn(0x7fb1a80ce3d0 msgr2=0x7fb1a80ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.284+0000 7fb1acdff700 1 -- 172.21.15.131:0/2201607398 shutdown_connections 2023-12-26T21:53:11.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.285+0000 7fb1acdff700 1 -- 172.21.15.131:0/2201607398 wait complete. 2023-12-26T21:53:11.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.286+0000 7fb1acdff700 1 Processor -- start 2023-12-26T21:53:11.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.286+0000 7fb1acdff700 1 -- start start 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1acdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a806a1a0 0x7fb1a8169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1acdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a8159560 0x7fb1a815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1acdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a8159930 0x7fb1a815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1acdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a816af00 con 0x7fb1a806a1a0 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1acdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a816a480 con 0x7fb1a8159560 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1acdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a816d7b0 con 0x7fb1a8159930 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a806a1a0 0x7fb1a8169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1a659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a806a1a0 0x7fb1a8169f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55816/0 (socket says 172.21.15.131:55816) 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1a659c700 1 -- 172.21.15.131:0/2362679 learned_addr learned my addr 172.21.15.131:0/2362679 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1a6d9d700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a8159930 0x7fb1a815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.287+0000 7fb1a5d9b700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a8159560 0x7fb1a815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1a659c700 1 -- 172.21.15.131:0/2362679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a8159930 msgr2=0x7fb1a815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1a659c700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a8159930 0x7fb1a815ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1a659c700 1 -- 172.21.15.131:0/2362679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a8159560 msgr2=0x7fb1a815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1a659c700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a8159560 0x7fb1a815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1a659c700 1 -- 172.21.15.131:0/2362679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1980094f0 con 0x7fb1a806a1a0 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1a5d9b700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a8159560 0x7fb1a815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1a659c700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a806a1a0 0x7fb1a8169f40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb1a806efe0 tx=0x7fb198017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1967fc700 1 -- 172.21.15.131:0/2362679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800bbe0 con 0x7fb1a806a1a0 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1967fc700 1 -- 172.21.15.131:0/2362679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198011420 con 0x7fb1a806a1a0 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1967fc700 1 -- 172.21.15.131:0/2362679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980117c0 con 0x7fb1a806a1a0 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a815a720 con 0x7fb1a806a1a0 2023-12-26T21:53:11.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.288+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a816a940 con 0x7fb1a806a1a0 2023-12-26T21:53:11.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.290+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a816da90 con 0x7fb1a806a1a0 2023-12-26T21:53:11.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.290+0000 7fb1967fc700 1 -- 172.21.15.131:0/2362679 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb198011bf0 con 0x7fb1a806a1a0 2023-12-26T21:53:11.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.291+0000 7fb1967fc700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb188066f30 0x7fb1880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:11.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.291+0000 7fb1967fc700 1 -- 172.21.15.131:0/2362679 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fb198085f10 con 0x7fb1a806a1a0 2023-12-26T21:53:11.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.291+0000 7fb1a5d9b700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb188066f30 0x7fb1880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:11.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.292+0000 7fb1a5d9b700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb188066f30 0x7fb1880693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb19c007fd0 tx=0x7fb19c009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:11.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.295+0000 7fb1967fc700 1 -- 172.21.15.131:0/2362679 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb198052ff0 con 0x7fb1a806a1a0 2023-12-26T21:53:11.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.484+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1a816d8f0 con 0x7fb188066f30 2023-12-26T21:53:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.492+0000 7fb1967fc700 1 -- 172.21.15.131:0/2362679 <== mgr.5789 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288701 (secure 0 0 0) 0x7fb19c001d20 con 0x7fb188066f30 2023-12-26T21:53:11.496 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:11.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.496+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb188066f30 msgr2=0x7fb1880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:11.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.496+0000 7fb1acdff700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb188066f30 0x7fb1880693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb19c007fd0 tx=0x7fb19c009040 comp rx=0 tx=0).stop 2023-12-26T21:53:11.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a806a1a0 msgr2=0x7fb1a8169f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:11.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a806a1a0 0x7fb1a8169f40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb1a806efe0 tx=0x7fb198017000 comp rx=0 tx=0).stop 2023-12-26T21:53:11.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 shutdown_connections 2023-12-26T21:53:11.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a8159930 0x7fb1a815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb188066f30 0x7fb1880693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1a806a1a0 0x7fb1a8169f40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 --2- 172.21.15.131:0/2362679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a8159560 0x7fb1a815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:11.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 >> 172.21.15.131:0/2362679 conn(0x7fb1a80ce3d0 msgr2=0x7fb1a80b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:11.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 shutdown_connections 2023-12-26T21:53:11.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:11.497+0000 7fb1acdff700 1 -- 172.21.15.131:0/2362679 wait complete. 2023-12-26T21:53:11.500 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:11.515 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-26T21:53:11.075759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2441864,"kb_used_data":10312,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752532856,"statfs":{"total":773094113280,"available":770593644544,"internally_reserved":0,"allocated":10559488,"data_stored":5682636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.834053"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.298141+0000","last_change":"2023-12-26T21:53:05.289049+0000","last_active":"2023-12-26T21:53:05.298141+0000","last_peered":"2023-12-26T21:53:05.298141+0000","last_clean":"2023-12-26T21:53:05.298141+0000","last_became_active":"2023-12-26T21:53:05.288687+0000","last_became_peered":"2023-12-26T21:53:05.288687+0000","last_unstale":"2023-12-26T21:53:05.298141+0000","last_undegraded":"2023-12-26T21:53:05.298141+0000","last_fullsized":"2023-12-26T21:53:05.298141+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:52.286840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.282288+0000","last_change":"2023-12-26T21:53:05.278284+0000","last_active":"2023-12-26T21:53:05.282288+0000","last_peered":"2023-12-26T21:53:05.282288+0000","last_clean":"2023-12-26T21:53:05.282288+0000","last_became_active":"2023-12-26T21:53:05.277672+0000","last_became_peered":"2023-12-26T21:53:05.277672+0000","last_unstale":"2023-12-26T21:53:05.282288+0000","last_undegraded":"2023-12-26T21:53:05.282288+0000","last_fullsized":"2023-12-26T21:53:05.282288+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:13:21.639979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.280736+0000","last_change":"2023-12-26T21:53:05.269473+0000","last_active":"2023-12-26T21:53:05.280736+0000","last_peered":"2023-12-26T21:53:05.280736+0000","last_clean":"2023-12-26T21:53:05.280736+0000","last_became_active":"2023-12-26T21:53:05.269149+0000","last_became_peered":"2023-12-26T21:53:05.269149+0000","last_unstale":"2023-12-26T21:53:05.280736+0000","last_undegraded":"2023-12-26T21:53:05.280736+0000","last_fullsized":"2023-12-26T21:53:05.280736+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:19.922173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.280769+0000","last_change":"2023-12-26T21:53:05.268989+0000","last_active":"2023-12-26T21:53:05.280769+0000","last_peered":"2023-12-26T21:53:05.280769+0000","last_clean":"2023-12-26T21:53:05.280769+0000","last_became_active":"2023-12-26T21:53:05.268632+0000","last_became_peered":"2023-12-26T21:53:05.268632+0000","last_unstale":"2023-12-26T21:53:05.280769+0000","last_undegraded":"2023-12-26T21:53:05.280769+0000","last_fullsized":"2023-12-26T21:53:05.280769+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:48:02.135228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.281175+0000","last_change":"2023-12-26T21:53:05.275156+0000","last_active":"2023-12-26T21:53:05.281175+0000","last_peered":"2023-12-26T21:53:05.281175+0000","last_clean":"2023-12-26T21:53:05.281175+0000","last_became_active":"2023-12-26T21:53:05.274871+0000","last_became_peered":"2023-12-26T21:53:05.274871+0000","last_unstale":"2023-12-26T21:53:05.281175+0000","last_undegraded":"2023-12-26T21:53:05.281175+0000","last_fullsized":"2023-12-26T21:53:05.281175+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:40:09.813749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287955+0000","last_change":"2023-12-26T21:53:05.281287+0000","last_active":"2023-12-26T21:53:05.287955+0000","last_peered":"2023-12-26T21:53:05.287955+0000","last_clean":"2023-12-26T21:53:05.287955+0000","last_became_active":"2023-12-26T21:53:05.281013+0000","last_became_peered":"2023-12-26T21:53:05.281013+0000","last_unstale":"2023-12-26T21:53:05.287955+0000","last_undegraded":"2023-12-26T21:53:05.287955+0000","last_fullsized":"2023-12-26T21:53:05.287955+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:38:31.950113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.275532+0000","last_change":"2023-12-26T21:53:05.271615+0000","last_active":"2023-12-26T21:53:05.275532+0000","last_peered":"2023-12-26T21:53:05.275532+0000","last_clean":"2023-12-26T21:53:05.275532+0000","last_became_active":"2023-12-26T21:53:05.270549+0000","last_became_peered":"2023-12-26T21:53:05.270549+0000","last_unstale":"2023-12-26T21:53:05.275532+0000","last_undegraded":"2023-12-26T21:53:05.275532+0000","last_fullsized":"2023-12-26T21:53:05.275532+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:19:17.880103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286534+0000","last_change":"2023-12-26T21:53:05.283059+0000","last_active":"2023-12-26T21:53:05.286534+0000","last_peered":"2023-12-26T21:53:05.286534+0000","last_clean":"2023-12-26T21:53:05.286534+0000","last_became_active":"2023-12-26T21:53:05.282417+0000","last_became_peered":"2023-12-26T21:53:05.282417+0000","last_unstale":"2023-12-26T21:53:05.286534+0000","last_undegraded":"2023-12-26T21:53:05.286534+0000","last_fullsized":"2023-12-26T21:53:05.286534+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:11:56.617393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286624+0000","last_change":"2023-12-26T21:53:05.279396+0000","last_active":"2023-12-26T21:53:05.286624+0000","last_peered":"2023-12-26T21:53:05.286624+0000","last_clean":"2023-12-26T21:53:05.286624+0000","last_became_active":"2023-12-26T21:53:05.278230+0000","last_became_peered":"2023-12-26T21:53:05.278230+0000","last_unstale":"2023-12-26T21:53:05.286624+0000","last_undegraded":"2023-12-26T21:53:05.286624+0000","last_fullsized":"2023-12-26T21:53:05.286624+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:44.183615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277521+0000","last_change":"2023-12-26T21:53:05.274464+0000","last_active":"2023-12-26T21:53:05.277521+0000","last_peered":"2023-12-26T21:53:05.277521+0000","last_clean":"2023-12-26T21:53:05.277521+0000","last_became_active":"2023-12-26T21:53:05.274140+0000","last_became_peered":"2023-12-26T21:53:05.274140+0000","last_unstale":"2023-12-26T21:53:05.277521+0000","last_undegraded":"2023-12-26T21:53:05.277521+0000","last_fullsized":"2023-12-26T21:53:05.277521+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:53:45.847934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288274+0000","last_change":"2023-12-26T21:53:05.286187+0000","last_active":"2023-12-26T21:53:05.288274+0000","last_peered":"2023-12-26T21:53:05.288274+0000","last_clean":"2023-12-26T21:53:05.288274+0000","last_became_active":"2023-12-26T21:53:05.285865+0000","last_became_peered":"2023-12-26T21:53:05.285865+0000","last_unstale":"2023-12-26T21:53:05.288274+0000","last_undegraded":"2023-12-26T21:53:05.288274+0000","last_fullsized":"2023-12-26T21:53:05.288274+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:56.825076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.292943+0000","last_change":"2023-12-26T21:53:05.290743+0000","last_active":"2023-12-26T21:53:05.292943+0000","last_peered":"2023-12-26T21:53:05.292943+0000","last_clean":"2023-12-26T21:53:05.292943+0000","last_became_active":"2023-12-26T21:53:05.290440+0000","last_became_peered":"2023-12-26T21:53:05.290440+0000","last_unstale":"2023-12-26T21:53:05.292943+0000","last_undegraded":"2023-12-26T21:53:05.292943+0000","last_fullsized":"2023-12-26T21:53:05.292943+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:26:31.991512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.284804+0000","last_change":"2023-12-26T21:53:05.271579+0000","last_active":"2023-12-26T21:53:05.284804+0000","last_peered":"2023-12-26T21:53:05.284804+0000","last_clean":"2023-12-26T21:53:05.284804+0000","last_became_active":"2023-12-26T21:53:05.271278+0000","last_became_peered":"2023-12-26T21:53:05.271278+0000","last_unstale":"2023-12-26T21:53:05.284804+0000","last_undegraded":"2023-12-26T21:53:05.284804+0000","last_fullsized":"2023-12-26T21:53:05.284804+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:23:10.752742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.296099+0000","last_change":"2023-12-26T21:53:05.293294+0000","last_active":"2023-12-26T21:53:05.296099+0000","last_peered":"2023-12-26T21:53:05.296099+0000","last_clean":"2023-12-26T21:53:05.296099+0000","last_became_active":"2023-12-26T21:53:05.292950+0000","last_became_peered":"2023-12-26T21:53:05.292950+0000","last_unstale":"2023-12-26T21:53:05.296099+0000","last_undegraded":"2023-12-26T21:53:05.296099+0000","last_fullsized":"2023-12-26T21:53:05.296099+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:36:13.991875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293166+0000","last_change":"2023-12-26T21:53:05.286014+0000","last_active":"2023-12-26T21:53:05.293166+0000","last_peered":"2023-12-26T21:53:05.293166+0000","last_clean":"2023-12-26T21:53:05.293166+0000","last_became_active":"2023-12-26T21:53:05.285657+0000","last_became_peered":"2023-12-26T21:53:05.285657+0000","last_unstale":"2023-12-26T21:53:05.293166+0000","last_undegraded":"2023-12-26T21:53:05.293166+0000","last_fullsized":"2023-12-26T21:53:05.293166+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:14:27.017262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286554+0000","last_change":"2023-12-26T21:53:05.283231+0000","last_active":"2023-12-26T21:53:05.286554+0000","last_peered":"2023-12-26T21:53:05.286554+0000","last_clean":"2023-12-26T21:53:05.286554+0000","last_became_active":"2023-12-26T21:53:05.282799+0000","last_became_peered":"2023-12-26T21:53:05.282799+0000","last_unstale":"2023-12-26T21:53:05.286554+0000","last_undegraded":"2023-12-26T21:53:05.286554+0000","last_fullsized":"2023-12-26T21:53:05.286554+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:12:37.012787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287779+0000","last_change":"2023-12-26T21:53:05.282286+0000","last_active":"2023-12-26T21:53:05.287779+0000","last_peered":"2023-12-26T21:53:05.287779+0000","last_clean":"2023-12-26T21:53:05.287779+0000","last_became_active":"2023-12-26T21:53:05.282036+0000","last_became_peered":"2023-12-26T21:53:05.282036+0000","last_unstale":"2023-12-26T21:53:05.287779+0000","last_undegraded":"2023-12-26T21:53:05.287779+0000","last_fullsized":"2023-12-26T21:53:05.287779+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:01:48.180876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.282819+0000","last_change":"2023-12-26T21:53:05.276742+0000","last_active":"2023-12-26T21:53:05.282819+0000","last_peered":"2023-12-26T21:53:05.282819+0000","last_clean":"2023-12-26T21:53:05.282819+0000","last_became_active":"2023-12-26T21:53:05.276279+0000","last_became_peered":"2023-12-26T21:53:05.276279+0000","last_unstale":"2023-12-26T21:53:05.282819+0000","last_undegraded":"2023-12-26T21:53:05.282819+0000","last_fullsized":"2023-12-26T21:53:05.282819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:59.099979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293935+0000","last_change":"2023-12-26T21:53:05.289933+0000","last_active":"2023-12-26T21:53:05.293935+0000","last_peered":"2023-12-26T21:53:05.293935+0000","last_clean":"2023-12-26T21:53:05.293935+0000","last_became_active":"2023-12-26T21:53:05.289653+0000","last_became_peered":"2023-12-26T21:53:05.289653+0000","last_unstale":"2023-12-26T21:53:05.293935+0000","last_undegraded":"2023-12-26T21:53:05.293935+0000","last_fullsized":"2023-12-26T21:53:05.293935+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:59.099979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287990+0000","last_change":"2023-12-26T21:53:05.282192+0000","last_active":"2023-12-26T21:53:05.287990+0000","last_peered":"2023-12-26T21:53:05.287990+0000","last_clean":"2023-12-26T21:53:05.287990+0000","last_became_active":"2023-12-26T21:53:05.281930+0000","last_became_peered":"2023-12-26T21:53:05.281930+0000","last_unstale":"2023-12-26T21:53:05.287990+0000","last_undegraded":"2023-12-26T21:53:05.287990+0000","last_fullsized":"2023-12-26T21:53:05.287990+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:15.519080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.283699+0000","last_change":"2023-12-26T21:53:05.282140+0000","last_active":"2023-12-26T21:53:05.283699+0000","last_peered":"2023-12-26T21:53:05.283699+0000","last_clean":"2023-12-26T21:53:05.283699+0000","last_became_active":"2023-12-26T21:53:05.281870+0000","last_became_peered":"2023-12-26T21:53:05.281870+0000","last_unstale":"2023-12-26T21:53:05.283699+0000","last_undegraded":"2023-12-26T21:53:05.283699+0000","last_fullsized":"2023-12-26T21:53:05.283699+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:37:33.021987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287165+0000","last_change":"2023-12-26T21:53:05.279440+0000","last_active":"2023-12-26T21:53:05.287165+0000","last_peered":"2023-12-26T21:53:05.287165+0000","last_clean":"2023-12-26T21:53:05.287165+0000","last_became_active":"2023-12-26T21:53:05.279154+0000","last_became_peered":"2023-12-26T21:53:05.279154+0000","last_unstale":"2023-12-26T21:53:05.287165+0000","last_undegraded":"2023-12-26T21:53:05.287165+0000","last_fullsized":"2023-12-26T21:53:05.287165+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:26:31.991512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.275551+0000","last_change":"2023-12-26T21:53:05.273034+0000","last_active":"2023-12-26T21:53:05.275551+0000","last_peered":"2023-12-26T21:53:05.275551+0000","last_clean":"2023-12-26T21:53:05.275551+0000","last_became_active":"2023-12-26T21:53:05.272727+0000","last_became_peered":"2023-12-26T21:53:05.272727+0000","last_unstale":"2023-12-26T21:53:05.275551+0000","last_undegraded":"2023-12-26T21:53:05.275551+0000","last_fullsized":"2023-12-26T21:53:05.275551+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:56:41.642952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293630+0000","last_change":"2023-12-26T21:53:05.286668+0000","last_active":"2023-12-26T21:53:05.293630+0000","last_peered":"2023-12-26T21:53:05.293630+0000","last_clean":"2023-12-26T21:53:05.293630+0000","last_became_active":"2023-12-26T21:53:05.286348+0000","last_became_peered":"2023-12-26T21:53:05.286348+0000","last_unstale":"2023-12-26T21:53:05.293630+0000","last_undegraded":"2023-12-26T21:53:05.293630+0000","last_fullsized":"2023-12-26T21:53:05.293630+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:06:28.041436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.275929+0000","last_change":"2023-12-26T21:53:05.273113+0000","last_active":"2023-12-26T21:53:05.275929+0000","last_peered":"2023-12-26T21:53:05.275929+0000","last_clean":"2023-12-26T21:53:05.275929+0000","last_became_active":"2023-12-26T21:53:05.272876+0000","last_became_peered":"2023-12-26T21:53:05.272876+0000","last_unstale":"2023-12-26T21:53:05.275929+0000","last_undegraded":"2023-12-26T21:53:05.275929+0000","last_fullsized":"2023-12-26T21:53:05.275929+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:13:24.873466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294036+0000","last_change":"2023-12-26T21:53:05.290390+0000","last_active":"2023-12-26T21:53:05.294036+0000","last_peered":"2023-12-26T21:53:05.294036+0000","last_clean":"2023-12-26T21:53:05.294036+0000","last_became_active":"2023-12-26T21:53:05.290100+0000","last_became_peered":"2023-12-26T21:53:05.290100+0000","last_unstale":"2023-12-26T21:53:05.294036+0000","last_undegraded":"2023-12-26T21:53:05.294036+0000","last_fullsized":"2023-12-26T21:53:05.294036+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:22:39.661507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294394+0000","last_change":"2023-12-26T21:53:05.290485+0000","last_active":"2023-12-26T21:53:05.294394+0000","last_peered":"2023-12-26T21:53:05.294394+0000","last_clean":"2023-12-26T21:53:05.294394+0000","last_became_active":"2023-12-26T21:53:05.290018+0000","last_became_peered":"2023-12-26T21:53:05.290018+0000","last_unstale":"2023-12-26T21:53:05.294394+0000","last_undegraded":"2023-12-26T21:53:05.294394+0000","last_fullsized":"2023-12-26T21:53:05.294394+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:13:56.991990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.290915+0000","last_change":"2023-12-26T21:53:05.288396+0000","last_active":"2023-12-26T21:53:05.290915+0000","last_peered":"2023-12-26T21:53:05.290915+0000","last_clean":"2023-12-26T21:53:05.290915+0000","last_became_active":"2023-12-26T21:53:05.284936+0000","last_became_peered":"2023-12-26T21:53:05.284936+0000","last_unstale":"2023-12-26T21:53:05.290915+0000","last_undegraded":"2023-12-26T21:53:05.290915+0000","last_fullsized":"2023-12-26T21:53:05.290915+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:01:48.180876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286211+0000","last_change":"2023-12-26T21:53:05.280350+0000","last_active":"2023-12-26T21:53:05.286211+0000","last_peered":"2023-12-26T21:53:05.286211+0000","last_clean":"2023-12-26T21:53:05.286211+0000","last_became_active":"2023-12-26T21:53:05.280012+0000","last_became_peered":"2023-12-26T21:53:05.280012+0000","last_unstale":"2023-12-26T21:53:05.286211+0000","last_undegraded":"2023-12-26T21:53:05.286211+0000","last_fullsized":"2023-12-26T21:53:05.286211+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:08.450696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294476+0000","last_change":"2023-12-26T21:53:05.290633+0000","last_active":"2023-12-26T21:53:05.294476+0000","last_peered":"2023-12-26T21:53:05.294476+0000","last_clean":"2023-12-26T21:53:05.294476+0000","last_became_active":"2023-12-26T21:53:05.289344+0000","last_became_peered":"2023-12-26T21:53:05.289344+0000","last_unstale":"2023-12-26T21:53:05.294476+0000","last_undegraded":"2023-12-26T21:53:05.294476+0000","last_fullsized":"2023-12-26T21:53:05.294476+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:15.904401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293831+0000","last_change":"2023-12-26T21:53:05.291158+0000","last_active":"2023-12-26T21:53:05.293831+0000","last_peered":"2023-12-26T21:53:05.293831+0000","last_clean":"2023-12-26T21:53:05.293831+0000","last_became_active":"2023-12-26T21:53:05.290863+0000","last_became_peered":"2023-12-26T21:53:05.290863+0000","last_unstale":"2023-12-26T21:53:05.293831+0000","last_undegraded":"2023-12-26T21:53:05.293831+0000","last_fullsized":"2023-12-26T21:53:05.293831+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:59.060036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.289040+0000","last_change":"2023-12-26T21:53:05.286142+0000","last_active":"2023-12-26T21:53:05.289040+0000","last_peered":"2023-12-26T21:53:05.289040+0000","last_clean":"2023-12-26T21:53:05.289040+0000","last_became_active":"2023-12-26T21:53:05.285761+0000","last_became_peered":"2023-12-26T21:53:05.285761+0000","last_unstale":"2023-12-26T21:53:05.289040+0000","last_undegraded":"2023-12-26T21:53:05.289040+0000","last_fullsized":"2023-12-26T21:53:05.289040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:36:44.951360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286999+0000","last_change":"2023-12-26T21:53:05.282844+0000","last_active":"2023-12-26T21:53:05.286999+0000","last_peered":"2023-12-26T21:53:05.286999+0000","last_clean":"2023-12-26T21:53:05.286999+0000","last_became_active":"2023-12-26T21:53:05.282596+0000","last_became_peered":"2023-12-26T21:53:05.282596+0000","last_unstale":"2023-12-26T21:53:05.286999+0000","last_undegraded":"2023-12-26T21:53:05.286999+0000","last_fullsized":"2023-12-26T21:53:05.286999+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:49:30.259823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"95'4","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.303732+0000","last_change":"2023-12-26T21:53:04.270399+0000","last_active":"2023-12-26T21:53:05.303732+0000","last_peered":"2023-12-26T21:53:05.303732+0000","last_clean":"2023-12-26T21:53:05.303732+0000","last_became_active":"2023-12-26T21:53:04.270117+0000","last_became_peered":"2023-12-26T21:53:04.270117+0000","last_unstale":"2023-12-26T21:53:05.303732+0000","last_undegraded":"2023-12-26T21:53:05.303732+0000","last_fullsized":"2023-12-26T21:53:05.303732+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:13:31.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.295742+0000","last_change":"2023-12-26T21:53:05.291846+0000","last_active":"2023-12-26T21:53:05.295742+0000","last_peered":"2023-12-26T21:53:05.295742+0000","last_clean":"2023-12-26T21:53:05.295742+0000","last_became_active":"2023-12-26T21:53:05.291574+0000","last_became_peered":"2023-12-26T21:53:05.291574+0000","last_unstale":"2023-12-26T21:53:05.295742+0000","last_undegraded":"2023-12-26T21:53:05.295742+0000","last_fullsized":"2023-12-26T21:53:05.295742+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:02.301077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.296485+0000","last_change":"2023-12-26T21:53:04.262700+0000","last_active":"2023-12-26T21:53:05.296485+0000","last_peered":"2023-12-26T21:53:05.296485+0000","last_clean":"2023-12-26T21:53:05.296485+0000","last_became_active":"2023-12-26T21:53:04.262378+0000","last_became_peered":"2023-12-26T21:53:04.262378+0000","last_unstale":"2023-12-26T21:53:05.296485+0000","last_undegraded":"2023-12-26T21:53:05.296485+0000","last_fullsized":"2023-12-26T21:53:05.296485+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:58.054482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288056+0000","last_change":"2023-12-26T21:53:05.281239+0000","last_active":"2023-12-26T21:53:05.288056+0000","last_peered":"2023-12-26T21:53:05.288056+0000","last_clean":"2023-12-26T21:53:05.288056+0000","last_became_active":"2023-12-26T21:53:05.280899+0000","last_became_peered":"2023-12-26T21:53:05.280899+0000","last_unstale":"2023-12-26T21:53:05.288056+0000","last_undegraded":"2023-12-26T21:53:05.288056+0000","last_fullsized":"2023-12-26T21:53:05.288056+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:46:33.933439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.299915+0000","last_change":"2023-12-26T21:53:04.262003+0000","last_active":"2023-12-26T21:53:05.299915+0000","last_peered":"2023-12-26T21:53:05.299915+0000","last_clean":"2023-12-26T21:53:05.299915+0000","last_became_active":"2023-12-26T21:53:04.261800+0000","last_became_peered":"2023-12-26T21:53:04.261800+0000","last_unstale":"2023-12-26T21:53:05.299915+0000","last_undegraded":"2023-12-26T21:53:05.299915+0000","last_fullsized":"2023-12-26T21:53:05.299915+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:00:23.255023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287438+0000","last_change":"2023-12-26T21:53:05.285641+0000","last_active":"2023-12-26T21:53:05.287438+0000","last_peered":"2023-12-26T21:53:05.287438+0000","last_clean":"2023-12-26T21:53:05.287438+0000","last_became_active":"2023-12-26T21:53:05.285318+0000","last_became_peered":"2023-12-26T21:53:05.285318+0000","last_unstale":"2023-12-26T21:53:05.287438+0000","last_undegraded":"2023-12-26T21:53:05.287438+0000","last_fullsized":"2023-12-26T21:53:05.287438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:28:21.116516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.301350+0000","last_change":"2023-12-26T21:53:04.262466+0000","last_active":"2023-12-26T21:53:05.301350+0000","last_peered":"2023-12-26T21:53:05.301350+0000","last_clean":"2023-12-26T21:53:05.301350+0000","last_became_active":"2023-12-26T21:53:04.262140+0000","last_became_peered":"2023-12-26T21:53:04.262140+0000","last_unstale":"2023-12-26T21:53:05.301350+0000","last_undegraded":"2023-12-26T21:53:05.301350+0000","last_fullsized":"2023-12-26T21:53:05.301350+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:46:20.997096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286063+0000","last_change":"2023-12-26T21:53:05.283242+0000","last_active":"2023-12-26T21:53:05.286063+0000","last_peered":"2023-12-26T21:53:05.286063+0000","last_clean":"2023-12-26T21:53:05.286063+0000","last_became_active":"2023-12-26T21:53:05.282980+0000","last_became_peered":"2023-12-26T21:53:05.282980+0000","last_unstale":"2023-12-26T21:53:05.286063+0000","last_undegraded":"2023-12-26T21:53:05.286063+0000","last_fullsized":"2023-12-26T21:53:05.286063+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:48:57.465172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.299674+0000","last_change":"2023-12-26T21:53:04.267058+0000","last_active":"2023-12-26T21:53:05.299674+0000","last_peered":"2023-12-26T21:53:05.299674+0000","last_clean":"2023-12-26T21:53:05.299674+0000","last_became_active":"2023-12-26T21:53:04.264901+0000","last_became_peered":"2023-12-26T21:53:04.264901+0000","last_unstale":"2023-12-26T21:53:05.299674+0000","last_undegraded":"2023-12-26T21:53:05.299674+0000","last_fullsized":"2023-12-26T21:53:05.299674+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:40.412864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287254+0000","last_change":"2023-12-26T21:53:05.280108+0000","last_active":"2023-12-26T21:53:05.287254+0000","last_peered":"2023-12-26T21:53:05.287254+0000","last_clean":"2023-12-26T21:53:05.287254+0000","last_became_active":"2023-12-26T21:53:05.279864+0000","last_became_peered":"2023-12-26T21:53:05.279864+0000","last_unstale":"2023-12-26T21:53:05.287254+0000","last_undegraded":"2023-12-26T21:53:05.287254+0000","last_fullsized":"2023-12-26T21:53:05.287254+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:52:15.234127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.298157+0000","last_change":"2023-12-26T21:53:04.260812+0000","last_active":"2023-12-26T21:53:05.298157+0000","last_peered":"2023-12-26T21:53:05.298157+0000","last_clean":"2023-12-26T21:53:05.298157+0000","last_became_active":"2023-12-26T21:53:04.260497+0000","last_became_peered":"2023-12-26T21:53:04.260497+0000","last_unstale":"2023-12-26T21:53:05.298157+0000","last_undegraded":"2023-12-26T21:53:05.298157+0000","last_fullsized":"2023-12-26T21:53:05.298157+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:15:56.273951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.298144+0000","last_change":"2023-12-26T21:53:05.291430+0000","last_active":"2023-12-26T21:53:05.298144+0000","last_peered":"2023-12-26T21:53:05.298144+0000","last_clean":"2023-12-26T21:53:05.298144+0000","last_became_active":"2023-12-26T21:53:05.291126+0000","last_became_peered":"2023-12-26T21:53:05.291126+0000","last_unstale":"2023-12-26T21:53:05.298144+0000","last_undegraded":"2023-12-26T21:53:05.298144+0000","last_fullsized":"2023-12-26T21:53:05.298144+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:15:36.453863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277179+0000","last_change":"2023-12-26T21:53:04.261961+0000","last_active":"2023-12-26T21:53:05.277179+0000","last_peered":"2023-12-26T21:53:05.277179+0000","last_clean":"2023-12-26T21:53:05.277179+0000","last_became_active":"2023-12-26T21:53:04.260903+0000","last_became_peered":"2023-12-26T21:53:04.260903+0000","last_unstale":"2023-12-26T21:53:05.277179+0000","last_undegraded":"2023-12-26T21:53:05.277179+0000","last_fullsized":"2023-12-26T21:53:05.277179+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:20:48.412586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293188+0000","last_change":"2023-12-26T21:53:05.291513+0000","last_active":"2023-12-26T21:53:05.293188+0000","last_peered":"2023-12-26T21:53:05.293188+0000","last_clean":"2023-12-26T21:53:05.293188+0000","last_became_active":"2023-12-26T21:53:05.289718+0000","last_became_peered":"2023-12-26T21:53:05.289718+0000","last_unstale":"2023-12-26T21:53:05.293188+0000","last_undegraded":"2023-12-26T21:53:05.293188+0000","last_fullsized":"2023-12-26T21:53:05.293188+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:42:40.127448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.271145+0000","last_change":"2023-12-26T21:53:04.259820+0000","last_active":"2023-12-26T21:53:05.271145+0000","last_peered":"2023-12-26T21:53:05.271145+0000","last_clean":"2023-12-26T21:53:05.271145+0000","last_became_active":"2023-12-26T21:53:04.259549+0000","last_became_peered":"2023-12-26T21:53:04.259549+0000","last_unstale":"2023-12-26T21:53:05.271145+0000","last_undegraded":"2023-12-26T21:53:05.271145+0000","last_fullsized":"2023-12-26T21:53:05.271145+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:43:17.471063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277501+0000","last_change":"2023-12-26T21:53:05.273198+0000","last_active":"2023-12-26T21:53:05.277501+0000","last_peered":"2023-12-26T21:53:05.277501+0000","last_clean":"2023-12-26T21:53:05.277501+0000","last_became_active":"2023-12-26T21:53:05.272661+0000","last_became_peered":"2023-12-26T21:53:05.272661+0000","last_unstale":"2023-12-26T21:53:05.277501+0000","last_undegraded":"2023-12-26T21:53:05.277501+0000","last_fullsized":"2023-12-26T21:53:05.277501+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:58:14.337531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.276858+0000","last_change":"2023-12-26T21:53:04.265435+0000","last_active":"2023-12-26T21:53:05.276858+0000","last_peered":"2023-12-26T21:53:05.276858+0000","last_clean":"2023-12-26T21:53:05.276858+0000","last_became_active":"2023-12-26T21:53:04.264942+0000","last_became_peered":"2023-12-26T21:53:04.264942+0000","last_unstale":"2023-12-26T21:53:05.276858+0000","last_undegraded":"2023-12-26T21:53:05.276858+0000","last_fullsized":"2023-12-26T21:53:05.276858+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:17:30.720991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.282429+0000","last_change":"2023-12-26T21:53:05.279536+0000","last_active":"2023-12-26T21:53:05.282429+0000","last_peered":"2023-12-26T21:53:05.282429+0000","last_clean":"2023-12-26T21:53:05.282429+0000","last_became_active":"2023-12-26T21:53:05.279236+0000","last_became_peered":"2023-12-26T21:53:05.279236+0000","last_unstale":"2023-12-26T21:53:05.282429+0000","last_undegraded":"2023-12-26T21:53:05.282429+0000","last_fullsized":"2023-12-26T21:53:05.282429+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:17:31.726545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.268670+0000","last_change":"2023-12-26T21:53:04.272739+0000","last_active":"2023-12-26T21:53:05.268670+0000","last_peered":"2023-12-26T21:53:05.268670+0000","last_clean":"2023-12-26T21:53:05.268670+0000","last_became_active":"2023-12-26T21:53:04.272478+0000","last_became_peered":"2023-12-26T21:53:04.272478+0000","last_unstale":"2023-12-26T21:53:05.268670+0000","last_undegraded":"2023-12-26T21:53:05.268670+0000","last_fullsized":"2023-12-26T21:53:05.268670+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:10.638441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293993+0000","last_change":"2023-12-26T21:53:05.287348+0000","last_active":"2023-12-26T21:53:05.293993+0000","last_peered":"2023-12-26T21:53:05.293993+0000","last_clean":"2023-12-26T21:53:05.293993+0000","last_became_active":"2023-12-26T21:53:05.286579+0000","last_became_peered":"2023-12-26T21:53:05.286579+0000","last_unstale":"2023-12-26T21:53:05.293993+0000","last_undegraded":"2023-12-26T21:53:05.293993+0000","last_fullsized":"2023-12-26T21:53:05.293993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:13:56.991990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"95'3","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.303439+0000","last_change":"2023-12-26T21:53:04.262478+0000","last_active":"2023-12-26T21:53:05.303439+0000","last_peered":"2023-12-26T21:53:05.303439+0000","last_clean":"2023-12-26T21:53:05.303439+0000","last_became_active":"2023-12-26T21:53:04.262209+0000","last_became_peered":"2023-12-26T21:53:04.262209+0000","last_unstale":"2023-12-26T21:53:05.303439+0000","last_undegraded":"2023-12-26T21:53:05.303439+0000","last_fullsized":"2023-12-26T21:53:05.303439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:03:39.647487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294376+0000","last_change":"2023-12-26T21:53:05.290318+0000","last_active":"2023-12-26T21:53:05.294376+0000","last_peered":"2023-12-26T21:53:05.294376+0000","last_clean":"2023-12-26T21:53:05.294376+0000","last_became_active":"2023-12-26T21:53:05.289587+0000","last_became_peered":"2023-12-26T21:53:05.289587+0000","last_unstale":"2023-12-26T21:53:05.294376+0000","last_undegraded":"2023-12-26T21:53:05.294376+0000","last_fullsized":"2023-12-26T21:53:05.294376+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:46:19.715350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"95'6","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.303735+0000","last_change":"2023-12-26T21:53:04.260381+0000","last_active":"2023-12-26T21:53:05.303735+0000","last_peered":"2023-12-26T21:53:05.303735+0000","last_clean":"2023-12-26T21:53:05.303735+0000","last_became_active":"2023-12-26T21:53:04.260099+0000","last_became_peered":"2023-12-26T21:53:04.260099+0000","last_unstale":"2023-12-26T21:53:05.303735+0000","last_undegraded":"2023-12-26T21:53:05.303735+0000","last_fullsized":"2023-12-26T21:53:05.303735+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:59:37.808016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288136+0000","last_change":"2023-12-26T21:53:05.283574+0000","last_active":"2023-12-26T21:53:05.288136+0000","last_peered":"2023-12-26T21:53:05.288136+0000","last_clean":"2023-12-26T21:53:05.288136+0000","last_became_active":"2023-12-26T21:53:05.283328+0000","last_became_peered":"2023-12-26T21:53:05.283328+0000","last_unstale":"2023-12-26T21:53:05.288136+0000","last_undegraded":"2023-12-26T21:53:05.288136+0000","last_fullsized":"2023-12-26T21:53:05.288136+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:11.021924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.302090+0000","last_change":"2023-12-26T21:53:04.265036+0000","last_active":"2023-12-26T21:53:05.302090+0000","last_peered":"2023-12-26T21:53:05.302090+0000","last_clean":"2023-12-26T21:53:05.302090+0000","last_became_active":"2023-12-26T21:53:04.264765+0000","last_became_peered":"2023-12-26T21:53:04.264765+0000","last_unstale":"2023-12-26T21:53:05.302090+0000","last_undegraded":"2023-12-26T21:53:05.302090+0000","last_fullsized":"2023-12-26T21:53:05.302090+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:55.298639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287789+0000","last_change":"2023-12-26T21:53:05.283134+0000","last_active":"2023-12-26T21:53:05.287789+0000","last_peered":"2023-12-26T21:53:05.287789+0000","last_clean":"2023-12-26T21:53:05.287789+0000","last_became_active":"2023-12-26T21:53:05.280388+0000","last_became_peered":"2023-12-26T21:53:05.280388+0000","last_unstale":"2023-12-26T21:53:05.287789+0000","last_undegraded":"2023-12-26T21:53:05.287789+0000","last_fullsized":"2023-12-26T21:53:05.287789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:23:55.238759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.264081+0000","last_change":"2023-12-26T21:53:04.265447+0000","last_active":"2023-12-26T21:53:05.264081+0000","last_peered":"2023-12-26T21:53:05.264081+0000","last_clean":"2023-12-26T21:53:05.264081+0000","last_became_active":"2023-12-26T21:53:04.265176+0000","last_became_peered":"2023-12-26T21:53:04.265176+0000","last_unstale":"2023-12-26T21:53:05.264081+0000","last_undegraded":"2023-12-26T21:53:05.264081+0000","last_fullsized":"2023-12-26T21:53:05.264081+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:24.737315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.276642+0000","last_change":"2023-12-26T21:53:05.273244+0000","last_active":"2023-12-26T21:53:05.276642+0000","last_peered":"2023-12-26T21:53:05.276642+0000","last_clean":"2023-12-26T21:53:05.276642+0000","last_became_active":"2023-12-26T21:53:05.272924+0000","last_became_peered":"2023-12-26T21:53:05.272924+0000","last_unstale":"2023-12-26T21:53:05.276642+0000","last_undegraded":"2023-12-26T21:53:05.276642+0000","last_fullsized":"2023-12-26T21:53:05.276642+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:19:20.160976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.264417+0000","last_change":"2023-12-26T21:53:04.264321+0000","last_active":"2023-12-26T21:53:05.264417+0000","last_peered":"2023-12-26T21:53:05.264417+0000","last_clean":"2023-12-26T21:53:05.264417+0000","last_became_active":"2023-12-26T21:53:04.264050+0000","last_became_peered":"2023-12-26T21:53:04.264050+0000","last_unstale":"2023-12-26T21:53:05.264417+0000","last_undegraded":"2023-12-26T21:53:05.264417+0000","last_fullsized":"2023-12-26T21:53:05.264417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:02:29.457133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.295815+0000","last_change":"2023-12-26T21:53:05.289902+0000","last_active":"2023-12-26T21:53:05.295815+0000","last_peered":"2023-12-26T21:53:05.295815+0000","last_clean":"2023-12-26T21:53:05.295815+0000","last_became_active":"2023-12-26T21:53:05.289623+0000","last_became_peered":"2023-12-26T21:53:05.289623+0000","last_unstale":"2023-12-26T21:53:05.295815+0000","last_undegraded":"2023-12-26T21:53:05.295815+0000","last_fullsized":"2023-12-26T21:53:05.295815+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:47:45.766327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"95'8","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.302755+0000","last_change":"2023-12-26T21:53:04.268072+0000","last_active":"2023-12-26T21:53:05.302755+0000","last_peered":"2023-12-26T21:53:05.302755+0000","last_clean":"2023-12-26T21:53:05.302755+0000","last_became_active":"2023-12-26T21:53:04.267818+0000","last_became_peered":"2023-12-26T21:53:04.267818+0000","last_unstale":"2023-12-26T21:53:05.302755+0000","last_undegraded":"2023-12-26T21:53:05.302755+0000","last_fullsized":"2023-12-26T21:53:05.302755+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:39.432599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.282308+0000","last_change":"2023-12-26T21:53:05.278470+0000","last_active":"2023-12-26T21:53:05.282308+0000","last_peered":"2023-12-26T21:53:05.282308+0000","last_clean":"2023-12-26T21:53:05.282308+0000","last_became_active":"2023-12-26T21:53:05.278014+0000","last_became_peered":"2023-12-26T21:53:05.278014+0000","last_unstale":"2023-12-26T21:53:05.282308+0000","last_undegraded":"2023-12-26T21:53:05.282308+0000","last_fullsized":"2023-12-26T21:53:05.282308+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:14.635797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.303685+0000","last_change":"2023-12-26T21:53:04.265951+0000","last_active":"2023-12-26T21:53:05.303685+0000","last_peered":"2023-12-26T21:53:05.303685+0000","last_clean":"2023-12-26T21:53:05.303685+0000","last_became_active":"2023-12-26T21:53:04.265704+0000","last_became_peered":"2023-12-26T21:53:04.265704+0000","last_unstale":"2023-12-26T21:53:05.303685+0000","last_undegraded":"2023-12-26T21:53:05.303685+0000","last_fullsized":"2023-12-26T21:53:05.303685+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:13:20.634426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.285593+0000","last_change":"2023-12-26T21:53:05.284308+0000","last_active":"2023-12-26T21:53:05.285593+0000","last_peered":"2023-12-26T21:53:05.285593+0000","last_clean":"2023-12-26T21:53:05.285593+0000","last_became_active":"2023-12-26T21:53:05.283912+0000","last_became_peered":"2023-12-26T21:53:05.283912+0000","last_unstale":"2023-12-26T21:53:05.285593+0000","last_undegraded":"2023-12-26T21:53:05.285593+0000","last_fullsized":"2023-12-26T21:53:05.285593+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:52:31.443697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.298760+0000","last_change":"2023-12-26T21:53:04.270594+0000","last_active":"2023-12-26T21:53:05.298760+0000","last_peered":"2023-12-26T21:53:05.298760+0000","last_clean":"2023-12-26T21:53:05.298760+0000","last_became_active":"2023-12-26T21:53:04.270353+0000","last_became_peered":"2023-12-26T21:53:04.270353+0000","last_unstale":"2023-12-26T21:53:05.298760+0000","last_undegraded":"2023-12-26T21:53:05.298760+0000","last_fullsized":"2023-12-26T21:53:05.298760+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:28:07.301140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286680+0000","last_change":"2023-12-26T21:53:05.283823+0000","last_active":"2023-12-26T21:53:05.286680+0000","last_peered":"2023-12-26T21:53:05.286680+0000","last_clean":"2023-12-26T21:53:05.286680+0000","last_became_active":"2023-12-26T21:53:05.283463+0000","last_became_peered":"2023-12-26T21:53:05.283463+0000","last_unstale":"2023-12-26T21:53:05.286680+0000","last_undegraded":"2023-12-26T21:53:05.286680+0000","last_fullsized":"2023-12-26T21:53:05.286680+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:43:18.476616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.299474+0000","last_change":"2023-12-26T21:53:04.270403+0000","last_active":"2023-12-26T21:53:05.299474+0000","last_peered":"2023-12-26T21:53:05.299474+0000","last_clean":"2023-12-26T21:53:05.299474+0000","last_became_active":"2023-12-26T21:53:04.270117+0000","last_became_peered":"2023-12-26T21:53:04.270117+0000","last_unstale":"2023-12-26T21:53:05.299474+0000","last_undegraded":"2023-12-26T21:53:05.299474+0000","last_fullsized":"2023-12-26T21:53:05.299474+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:22:41.561642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286264+0000","last_change":"2023-12-26T21:53:05.280232+0000","last_active":"2023-12-26T21:53:05.286264+0000","last_peered":"2023-12-26T21:53:05.286264+0000","last_clean":"2023-12-26T21:53:05.286264+0000","last_became_active":"2023-12-26T21:53:05.279934+0000","last_became_peered":"2023-12-26T21:53:05.279934+0000","last_unstale":"2023-12-26T21:53:05.286264+0000","last_undegraded":"2023-12-26T21:53:05.286264+0000","last_fullsized":"2023-12-26T21:53:05.286264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:41.418418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277177+0000","last_change":"2023-12-26T21:53:04.269097+0000","last_active":"2023-12-26T21:53:05.277177+0000","last_peered":"2023-12-26T21:53:05.277177+0000","last_clean":"2023-12-26T21:53:05.277177+0000","last_became_active":"2023-12-26T21:53:04.268508+0000","last_became_peered":"2023-12-26T21:53:04.268508+0000","last_unstale":"2023-12-26T21:53:05.277177+0000","last_undegraded":"2023-12-26T21:53:05.277177+0000","last_fullsized":"2023-12-26T21:53:05.277177+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:03:39.647487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.296192+0000","last_change":"2023-12-26T21:53:05.288594+0000","last_active":"2023-12-26T21:53:05.296192+0000","last_peered":"2023-12-26T21:53:05.296192+0000","last_clean":"2023-12-26T21:53:05.296192+0000","last_became_active":"2023-12-26T21:53:05.288312+0000","last_became_peered":"2023-12-26T21:53:05.288312+0000","last_unstale":"2023-12-26T21:53:05.296192+0000","last_undegraded":"2023-12-26T21:53:05.296192+0000","last_fullsized":"2023-12-26T21:53:05.296192+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:34.050678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277502+0000","last_change":"2023-12-26T21:53:04.272377+0000","last_active":"2023-12-26T21:53:05.277502+0000","last_peered":"2023-12-26T21:53:05.277502+0000","last_clean":"2023-12-26T21:53:05.277502+0000","last_became_active":"2023-12-26T21:53:04.270643+0000","last_became_peered":"2023-12-26T21:53:04.270643+0000","last_unstale":"2023-12-26T21:53:05.277502+0000","last_undegraded":"2023-12-26T21:53:05.277502+0000","last_fullsized":"2023-12-26T21:53:05.277502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:31:33.643977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288252+0000","last_change":"2023-12-26T21:53:05.285016+0000","last_active":"2023-12-26T21:53:05.288252+0000","last_peered":"2023-12-26T21:53:05.288252+0000","last_clean":"2023-12-26T21:53:05.288252+0000","last_became_active":"2023-12-26T21:53:05.283467+0000","last_became_peered":"2023-12-26T21:53:05.283467+0000","last_unstale":"2023-12-26T21:53:05.288252+0000","last_undegraded":"2023-12-26T21:53:05.288252+0000","last_fullsized":"2023-12-26T21:53:05.288252+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:13.863148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.269265+0000","last_change":"2023-12-26T21:53:04.269596+0000","last_active":"2023-12-26T21:53:05.269265+0000","last_peered":"2023-12-26T21:53:05.269265+0000","last_clean":"2023-12-26T21:53:05.269265+0000","last_became_active":"2023-12-26T21:53:04.269314+0000","last_became_peered":"2023-12-26T21:53:04.269314+0000","last_unstale":"2023-12-26T21:53:05.269265+0000","last_undegraded":"2023-12-26T21:53:05.269265+0000","last_fullsized":"2023-12-26T21:53:05.269265+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:33:19.638070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294572+0000","last_change":"2023-12-26T21:53:05.292994+0000","last_active":"2023-12-26T21:53:05.294572+0000","last_peered":"2023-12-26T21:53:05.294572+0000","last_clean":"2023-12-26T21:53:05.294572+0000","last_became_active":"2023-12-26T21:53:05.289294+0000","last_became_peered":"2023-12-26T21:53:05.289294+0000","last_unstale":"2023-12-26T21:53:05.294572+0000","last_undegraded":"2023-12-26T21:53:05.294572+0000","last_fullsized":"2023-12-26T21:53:05.294572+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:02:39.101474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.268459+0000","last_change":"2023-12-26T21:53:04.270655+0000","last_active":"2023-12-26T21:53:05.268459+0000","last_peered":"2023-12-26T21:53:05.268459+0000","last_clean":"2023-12-26T21:53:05.268459+0000","last_became_active":"2023-12-26T21:53:04.270381+0000","last_became_peered":"2023-12-26T21:53:04.270381+0000","last_unstale":"2023-12-26T21:53:05.268459+0000","last_undegraded":"2023-12-26T21:53:05.268459+0000","last_fullsized":"2023-12-26T21:53:05.268459+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:39.475708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294233+0000","last_change":"2023-12-26T21:53:05.288484+0000","last_active":"2023-12-26T21:53:05.294233+0000","last_peered":"2023-12-26T21:53:05.294233+0000","last_clean":"2023-12-26T21:53:05.294233+0000","last_became_active":"2023-12-26T21:53:05.288199+0000","last_became_peered":"2023-12-26T21:53:05.288199+0000","last_unstale":"2023-12-26T21:53:05.294233+0000","last_undegraded":"2023-12-26T21:53:05.294233+0000","last_fullsized":"2023-12-26T21:53:05.294233+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:53.622961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.300999+0000","last_change":"2023-12-26T21:53:04.265832+0000","last_active":"2023-12-26T21:53:05.300999+0000","last_peered":"2023-12-26T21:53:05.300999+0000","last_clean":"2023-12-26T21:53:05.300999+0000","last_became_active":"2023-12-26T21:53:04.265580+0000","last_became_peered":"2023-12-26T21:53:04.265580+0000","last_unstale":"2023-12-26T21:53:05.300999+0000","last_undegraded":"2023-12-26T21:53:05.300999+0000","last_fullsized":"2023-12-26T21:53:05.300999+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:13.630243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294323+0000","last_change":"2023-12-26T21:53:05.289951+0000","last_active":"2023-12-26T21:53:05.294323+0000","last_peered":"2023-12-26T21:53:05.294323+0000","last_clean":"2023-12-26T21:53:05.294323+0000","last_became_active":"2023-12-26T21:53:05.289699+0000","last_became_peered":"2023-12-26T21:53:05.289699+0000","last_unstale":"2023-12-26T21:53:05.294323+0000","last_undegraded":"2023-12-26T21:53:05.294323+0000","last_fullsized":"2023-12-26T21:53:05.294323+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:07:16.289438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.275437+0000","last_change":"2023-12-26T21:53:04.263420+0000","last_active":"2023-12-26T21:53:05.275437+0000","last_peered":"2023-12-26T21:53:05.275437+0000","last_clean":"2023-12-26T21:53:05.275437+0000","last_became_active":"2023-12-26T21:53:04.263154+0000","last_became_peered":"2023-12-26T21:53:04.263154+0000","last_unstale":"2023-12-26T21:53:05.275437+0000","last_undegraded":"2023-12-26T21:53:05.275437+0000","last_fullsized":"2023-12-26T21:53:05.275437+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:40:02.936486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288100+0000","last_change":"2023-12-26T21:53:05.284585+0000","last_active":"2023-12-26T21:53:05.288100+0000","last_peered":"2023-12-26T21:53:05.288100+0000","last_clean":"2023-12-26T21:53:05.288100+0000","last_became_active":"2023-12-26T21:53:05.282315+0000","last_became_peered":"2023-12-26T21:53:05.282315+0000","last_unstale":"2023-12-26T21:53:05.288100+0000","last_undegraded":"2023-12-26T21:53:05.288100+0000","last_fullsized":"2023-12-26T21:53:05.288100+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:40.481262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.272793+0000","last_change":"2023-12-26T21:53:04.265574+0000","last_active":"2023-12-26T21:53:05.272793+0000","last_peered":"2023-12-26T21:53:05.272793+0000","last_clean":"2023-12-26T21:53:05.272793+0000","last_became_active":"2023-12-26T21:53:04.265312+0000","last_became_peered":"2023-12-26T21:53:04.265312+0000","last_unstale":"2023-12-26T21:53:05.272793+0000","last_undegraded":"2023-12-26T21:53:05.272793+0000","last_fullsized":"2023-12-26T21:53:05.272793+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:47:44.760773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287420+0000","last_change":"2023-12-26T21:53:05.280022+0000","last_active":"2023-12-26T21:53:05.287420+0000","last_peered":"2023-12-26T21:53:05.287420+0000","last_clean":"2023-12-26T21:53:05.287420+0000","last_became_active":"2023-12-26T21:53:05.279769+0000","last_became_peered":"2023-12-26T21:53:05.279769+0000","last_unstale":"2023-12-26T21:53:05.287420+0000","last_undegraded":"2023-12-26T21:53:05.287420+0000","last_fullsized":"2023-12-26T21:53:05.287420+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:44:30.740351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.274000+0000","last_change":"2023-12-26T21:53:04.267879+0000","last_active":"2023-12-26T21:53:05.274000+0000","last_peered":"2023-12-26T21:53:05.274000+0000","last_clean":"2023-12-26T21:53:05.274000+0000","last_became_active":"2023-12-26T21:53:04.267583+0000","last_became_peered":"2023-12-26T21:53:04.267583+0000","last_unstale":"2023-12-26T21:53:05.274000+0000","last_undegraded":"2023-12-26T21:53:05.274000+0000","last_fullsized":"2023-12-26T21:53:05.274000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:02:20.185873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.292438+0000","last_change":"2023-12-26T21:53:05.283335+0000","last_active":"2023-12-26T21:53:05.292438+0000","last_peered":"2023-12-26T21:53:05.292438+0000","last_clean":"2023-12-26T21:53:05.292438+0000","last_became_active":"2023-12-26T21:53:05.283025+0000","last_became_peered":"2023-12-26T21:53:05.283025+0000","last_unstale":"2023-12-26T21:53:05.292438+0000","last_undegraded":"2023-12-26T21:53:05.292438+0000","last_fullsized":"2023-12-26T21:53:05.292438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:12.309723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.271990+0000","last_change":"2023-12-26T21:53:04.268290+0000","last_active":"2023-12-26T21:53:05.271990+0000","last_peered":"2023-12-26T21:53:05.271990+0000","last_clean":"2023-12-26T21:53:05.271990+0000","last_became_active":"2023-12-26T21:53:04.267992+0000","last_became_peered":"2023-12-26T21:53:04.267992+0000","last_unstale":"2023-12-26T21:53:05.271990+0000","last_undegraded":"2023-12-26T21:53:05.271990+0000","last_fullsized":"2023-12-26T21:53:05.271990+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:07.445143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294020+0000","last_change":"2023-12-26T21:53:05.288258+0000","last_active":"2023-12-26T21:53:05.294020+0000","last_peered":"2023-12-26T21:53:05.294020+0000","last_clean":"2023-12-26T21:53:05.294020+0000","last_became_active":"2023-12-26T21:53:05.286668+0000","last_became_peered":"2023-12-26T21:53:05.286668+0000","last_unstale":"2023-12-26T21:53:05.294020+0000","last_undegraded":"2023-12-26T21:53:05.294020+0000","last_fullsized":"2023-12-26T21:53:05.294020+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:30:55.484438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.268491+0000","last_change":"2023-12-26T21:53:04.265372+0000","last_active":"2023-12-26T21:53:05.268491+0000","last_peered":"2023-12-26T21:53:05.268491+0000","last_clean":"2023-12-26T21:53:05.268491+0000","last_became_active":"2023-12-26T21:53:04.264345+0000","last_became_peered":"2023-12-26T21:53:04.264345+0000","last_unstale":"2023-12-26T21:53:05.268491+0000","last_undegraded":"2023-12-26T21:53:05.268491+0000","last_fullsized":"2023-12-26T21:53:05.268491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:37:32.016433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288429+0000","last_change":"2023-12-26T21:53:05.285193+0000","last_active":"2023-12-26T21:53:05.288429+0000","last_peered":"2023-12-26T21:53:05.288429+0000","last_clean":"2023-12-26T21:53:05.288429+0000","last_became_active":"2023-12-26T21:53:05.284623+0000","last_became_peered":"2023-12-26T21:53:05.284623+0000","last_unstale":"2023-12-26T21:53:05.288429+0000","last_undegraded":"2023-12-26T21:53:05.288429+0000","last_fullsized":"2023-12-26T21:53:05.288429+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:07:56.200815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.269358+0000","last_change":"2023-12-26T21:53:04.267467+0000","last_active":"2023-12-26T21:53:05.269358+0000","last_peered":"2023-12-26T21:53:05.269358+0000","last_clean":"2023-12-26T21:53:05.269358+0000","last_became_active":"2023-12-26T21:53:04.267199+0000","last_became_peered":"2023-12-26T21:53:04.267199+0000","last_unstale":"2023-12-26T21:53:05.269358+0000","last_undegraded":"2023-12-26T21:53:05.269358+0000","last_fullsized":"2023-12-26T21:53:05.269358+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:53.612712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.273847+0000","last_change":"2023-12-26T21:53:05.271778+0000","last_active":"2023-12-26T21:53:05.273847+0000","last_peered":"2023-12-26T21:53:05.273847+0000","last_clean":"2023-12-26T21:53:05.273847+0000","last_became_active":"2023-12-26T21:53:05.271352+0000","last_became_peered":"2023-12-26T21:53:05.271352+0000","last_unstale":"2023-12-26T21:53:05.273847+0000","last_undegraded":"2023-12-26T21:53:05.273847+0000","last_fullsized":"2023-12-26T21:53:05.273847+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:14:44.426079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.272518+0000","last_change":"2023-12-26T21:53:04.271557+0000","last_active":"2023-12-26T21:53:05.272518+0000","last_peered":"2023-12-26T21:53:05.272518+0000","last_clean":"2023-12-26T21:53:05.272518+0000","last_became_active":"2023-12-26T21:53:04.271300+0000","last_became_peered":"2023-12-26T21:53:04.271300+0000","last_unstale":"2023-12-26T21:53:05.272518+0000","last_undegraded":"2023-12-26T21:53:05.272518+0000","last_fullsized":"2023-12-26T21:53:05.272518+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:19.256644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287808+0000","last_change":"2023-12-26T21:53:05.282558+0000","last_active":"2023-12-26T21:53:05.287808+0000","last_peered":"2023-12-26T21:53:05.287808+0000","last_clean":"2023-12-26T21:53:05.287808+0000","last_became_active":"2023-12-26T21:53:05.281302+0000","last_became_peered":"2023-12-26T21:53:05.281302+0000","last_unstale":"2023-12-26T21:53:05.287808+0000","last_undegraded":"2023-12-26T21:53:05.287808+0000","last_fullsized":"2023-12-26T21:53:05.287808+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:29:24.182590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.297316+0000","last_change":"2023-12-26T21:53:04.264566+0000","last_active":"2023-12-26T21:53:05.297316+0000","last_peered":"2023-12-26T21:53:05.297316+0000","last_clean":"2023-12-26T21:53:05.297316+0000","last_became_active":"2023-12-26T21:53:04.264140+0000","last_became_peered":"2023-12-26T21:53:04.264140+0000","last_unstale":"2023-12-26T21:53:05.297316+0000","last_undegraded":"2023-12-26T21:53:05.297316+0000","last_fullsized":"2023-12-26T21:53:05.297316+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:17:15.637087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'183","reported_seq":464,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.263934+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:05.263934+0000","last_peered":"2023-12-26T21:53:05.263934+0000","last_clean":"2023-12-26T21:53:05.263934+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:05.263934+0000","last_undegraded":"2023-12-26T21:53:05.263934+0000","last_fullsized":"2023-12-26T21:53:05.263934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542341,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":563697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.47199999999999998,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.76900000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.69799999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74399999999999999,"5min":1.016,"15min":1.048},"last":0.61099999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.038,"15min":1.0509999999999999},"last":0.752},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.85699999999999998,"5min":0.99199999999999999,"15min":1.02},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.62,"5min":0.504,"15min":0.5},"max":{"1min":0.94299999999999995,"5min":1.0189999999999999,"15min":1.052},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.63400000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83499999999999996,"5min":1.036,"15min":1.036},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.747,"15min":0.746},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0129999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.81499999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.117,"15min":1.117},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.437},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.006},"last":0.66400000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509637,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306632,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065208,"statfs":{"total":96636764160,"available":96322772992,"internally_reserved":0,"allocated":1179648,"data_stored":563731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":1.0029999999999999,"15min":1.085},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.3380000000000001},"last":0.63}]},{"osd":1,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.69399999999999995}]},{"osd":2,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":1.288},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.876,"5min":0.92000000000000004,"15min":1.113},"last":0.64600000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.92000000000000004,"5min":1.026,"15min":1.569},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.97799999999999998,"15min":1.3839999999999999},"last":0.71799999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.62,"15min":0.62},"min":{"1min":0.42699999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.137},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.69599999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95099999999999996,"5min":1.0049999999999999,"15min":1.296},"last":0.74299999999999999}]},{"osd":7,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.437},"max":{"1min":0.89200000000000002,"5min":0.96899999999999997,"15min":1.51},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.91700000000000004,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.70499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509637,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307196,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064644,"statfs":{"total":96636764160,"available":96322195456,"internally_reserved":0,"allocated":1757184,"data_stored":1152529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.442}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.73099999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.625},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.69399999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.79200000000000004}]}]},{"osd":0,"up_from":15,"seq":64424509637,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302524,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069316,"statfs":{"total":96636764160,"available":96326979584,"internally_reserved":0,"allocated":1167360,"data_stored":562231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":1.056},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.33800000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47799999999999998,"15min":0.49399999999999999},"min":{"1min":0.315,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":1.006},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.84699999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80000000000000004,"5min":0.86699999999999999,"15min":1.1870000000000001},"last":0.47899999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.752},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.042,"5min":1.042,"15min":1.411},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.76800000000000002},"min":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.2070000000000001},"last":0.70699999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.754,"15min":0.79800000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.2569999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77100000000000002,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.452},"last":0.80600000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.214},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.012,"15min":1.3779999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46400000000000002},"max":{"1min":1.018,"5min":1.042,"15min":1.4359999999999999},"last":0.749},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.79200000000000004},"min":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":1.0029999999999999,"5min":1.0069999999999999,"15min":1.3240000000000001},"last":0.874}]}]},{"osd":2,"up_from":14,"seq":60129542342,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306620,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065220,"statfs":{"total":96636764160,"available":96322785280,"internally_reserved":0,"allocated":1167360,"data_stored":562715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45800000000000002,"15min":0.45900000000000002},"min":{"1min":0.33200000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":1.915},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.30399999999999999},"max":{"1min":0.63200000000000001,"5min":0.83899999999999997,"15min":1.7729999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.39600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":1.952},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47999999999999998},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.30199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.39100000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.375},"max":{"1min":1,"5min":1,"15min":1},"last":0.496},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73899999999999999,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.54600000000000004,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.96999999999999997,"5min":0.98299999999999998,"15min":1.284},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.60099999999999998,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.0600000000000001,"15min":1.5029999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":0.94599999999999995,"5min":1.014,"15min":1.355},"last":0.85099999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.93000000000000005,"5min":1.038,"15min":1.7350000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.95999999999999996,"5min":1.0609999999999999,"15min":1.349},"last":0.67000000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.47799999999999998},"max":{"1min":0.92700000000000005,"5min":1.073,"15min":1.841},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0189999999999999,"15min":1.294},"last":0.751}]}]},{"osd":3,"up_from":15,"seq":64424509638,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310724,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061116,"statfs":{"total":96636764160,"available":96318582784,"internally_reserved":0,"allocated":1175552,"data_stored":562287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.317},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.42099999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.39900000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.498}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63100000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.61599999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542341,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306628,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065212,"statfs":{"total":96636764160,"available":96322777088,"internally_reserved":0,"allocated":1175552,"data_stored":562827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97899999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.48499999999999999,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":1.008},"last":0.88}]},{"osd":1,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.95199999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.84799999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88900000000000001,"5min":0.97199999999999998,"15min":0.97299999999999998},"last":0.86499999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.71499999999999997},"min":{"1min":0.504,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.93000000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.438,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.747},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.91100000000000003}]}]},{"osd":5,"up_from":15,"seq":64424509636,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303104,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068736,"statfs":{"total":96636764160,"available":96326385664,"internally_reserved":0,"allocated":1761280,"data_stored":1152619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.878,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.71299999999999997},"min":{"1min":0.48899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.71},"last":0.55300000000000005}]},{"osd":1,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.57399999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96399999999999997,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68200000000000005,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.002,"15min":1.4219999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.871,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68799999999999994,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":1.214},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":1.052,"15min":1.4890000000000001},"last":0.752},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":0.92200000000000004,"15min":1.1439999999999999},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84399999999999997,"5min":1.0089999999999999,"15min":2.5070000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.71299999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.92400000000000004,"15min":1.637},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.042,"15min":2.5579999999999998},"last":0.85499999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":1.248},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":0.95199999999999996,"15min":1.5980000000000001},"last":0.72799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:14.521 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:14.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.720+0000 7f76837ff700 1 Processor -- start 2023-12-26T21:53:14.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.720+0000 7f76837ff700 1 -- start start 2023-12-26T21:53:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.721+0000 7f76837ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767c0b07f0 0x7f767c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.721+0000 7f76837ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c0c8bf0 con 0x7f767c159cf0 2023-12-26T21:53:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.721+0000 7f76837ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c0c8e60 con 0x7f767c159920 2023-12-26T21:53:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.721+0000 7f76837ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0c9140 con 0x7f767c0b07f0 2023-12-26T21:53:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.721+0000 7f7681d9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f767c159cf0 0x7f767c155a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49306/0 (socket says 172.21.15.131:49306) 2023-12-26T21:53:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.721+0000 7f7681d9c700 1 -- 172.21.15.131:0/3463420665 learned_addr learned my addr 172.21.15.131:0/3463420665 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7680d9a700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767c0b07f0 0x7f767c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457151747 0 0) 0x7f767c0c8bf0 con 0x7f767c159cf0 2023-12-26T21:53:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7668003570 con 0x7f767c159cf0 2023-12-26T21:53:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821785205 0 0) 0x7f767c0c8e60 con 0x7f767c159920 2023-12-26T21:53:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f767c0c8bf0 con 0x7f767c159920 2023-12-26T21:53:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3952660886 0 0) 0x7f7668003570 con 0x7f767c159cf0 2023-12-26T21:53:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767c0b07f0 msgr2=0x7f767c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767c0b07f0 0x7f767c1554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 >> v1:172.21.15.190:6789/0 conn(0x7f767c159920 legacy=0x7f767c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0c9420 con 0x7f767c159cf0 2023-12-26T21:53:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f76740017f0 con 0x7f767c159cf0 2023-12-26T21:53:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 >> v1:172.21.15.131:6789/0 conn(0x7f767c159cf0 legacy=0x7f767c155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7668003c20 0x7f7668004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7668004810 0x7f7668004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680051c0 0x7f766800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0c9140 con 0x7f7668004810 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766800bfa0 con 0x7f7668003c20 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.722+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766800c0e0 con 0x7f76680051c0 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7680d9a700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7668004810 0x7f7668004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f768159b700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7668003c20 0x7f7668004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7681d9c700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680051c0 0x7f766800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7680d9a700 1 -- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680051c0 msgr2=0x7f766800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7680d9a700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680051c0 0x7f766800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7680d9a700 1 -- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7668003c20 msgr2=0x7f7668004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7680d9a700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7668003c20 0x7f7668004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7680d9a700 1 -- 172.21.15.131:0/3463420665 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7668003570 con 0x7f7668004810 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7680d9a700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7668004810 0x7f7668004c80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f766c00ade0 tx=0x7f766c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e318fdfcef355b91 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c00fa30 con 0x7f7668004810 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.723+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c00fb90 con 0x7f7668004810 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.724+0000 7f7673fff700 1 -- 172.21.15.131:0/3463420665 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c00fd20 con 0x7f7668004810 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.724+0000 7f76837ff700 1 -- 172.21.15.131:0/3463420665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7668004810 msgr2=0x7f7668004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.724+0000 7f76837ff700 1 --2- 172.21.15.131:0/3463420665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7668004810 0x7f7668004c80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f766c00ade0 tx=0x7f766c00a330 comp rx=0 tx=0).stop 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.724+0000 7f7681d9c700 1 -- 172.21.15.131:0/3463420665 reap_dead start 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.724+0000 7f76837ff700 1 -- 172.21.15.131:0/3463420665 shutdown_connections 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.724+0000 7f76837ff700 1 -- 172.21.15.131:0/3463420665 >> 172.21.15.131:0/3463420665 conn(0x7f767c068040 msgr2=0x7f767c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.724+0000 7f76837ff700 1 -- 172.21.15.131:0/3463420665 shutdown_connections 2023-12-26T21:53:14.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.724+0000 7f76837ff700 1 -- 172.21.15.131:0/3463420665 wait complete. 2023-12-26T21:53:14.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.726+0000 7f76837ff700 1 Processor -- start 2023-12-26T21:53:14.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.726+0000 7f76837ff700 1 -- start start 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f76837ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7668004810 0x7f767c0a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f76837ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f767c0c7600 0x7f767c0c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f76837ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f767c169760 0x7f767c169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f76837ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0c7fb0 con 0x7f767c0c7600 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f76837ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c06d4e0 con 0x7f767c169760 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f76837ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c06e060 con 0x7f7668004810 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f7680d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f767c0c7600 0x7f767c0c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f7680d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f767c0c7600 0x7f767c0c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55838/0 (socket says 172.21.15.131:55838) 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f7680d9a700 1 -- 172.21.15.131:0/2921856874 learned_addr learned my addr 172.21.15.131:0/2921856874 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f7680d9a700 1 -- 172.21.15.131:0/2921856874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7668004810 msgr2=0x7f767c0a7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f7680d9a700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7668004810 0x7f767c0a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f7680d9a700 1 -- 172.21.15.131:0/2921856874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f767c169760 msgr2=0x7f767c169bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f7680d9a700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f767c169760 0x7f767c169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:14.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.727+0000 7f7680d9a700 1 -- 172.21.15.131:0/2921856874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c06d730 con 0x7f767c0c7600 2023-12-26T21:53:14.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.728+0000 7f7680d9a700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f767c0c7600 0x7f767c0c7a70 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f766c006ed0 tx=0x7f766c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:14.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.728+0000 7f76717fa700 1 -- 172.21.15.131:0/2921856874 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c00fe50 con 0x7f767c0c7600 2023-12-26T21:53:14.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.728+0000 7f76717fa700 1 -- 172.21.15.131:0/2921856874 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c012670 con 0x7f767c0c7600 2023-12-26T21:53:14.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.728+0000 7f76717fa700 1 -- 172.21.15.131:0/2921856874 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c012a30 con 0x7f767c0c7600 2023-12-26T21:53:14.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.728+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c06da10 con 0x7f767c0c7600 2023-12-26T21:53:14.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.728+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c0a8380 con 0x7f767c0c7600 2023-12-26T21:53:14.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.729+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767c0657e0 con 0x7f767c0c7600 2023-12-26T21:53:14.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.730+0000 7f76717fa700 1 -- 172.21.15.131:0/2921856874 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f766c025410 con 0x7f767c0c7600 2023-12-26T21:53:14.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.731+0000 7f76717fa700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7660066f30 0x7f76600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:14.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.731+0000 7f76717fa700 1 -- 172.21.15.131:0/2921856874 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f766c016070 con 0x7f767c0c7600 2023-12-26T21:53:14.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.731+0000 7f768159b700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7660066f30 0x7f76600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:14.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.731+0000 7f768159b700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7660066f30 0x7f76600693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76780029d0 tx=0x7f7678007d30 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:14.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.735+0000 7f76717fa700 1 -- 172.21.15.131:0/2921856874 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f766c053750 con 0x7f767c0c7600 2023-12-26T21:53:14.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.920+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f767c06e1a0 con 0x7f7660066f30 2023-12-26T21:53:14.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.929+0000 7f76717fa700 1 -- 172.21.15.131:0/2921856874 <== mgr.5789 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288701 (secure 0 0 0) 0x7f767c06e1a0 con 0x7f7660066f30 2023-12-26T21:53:14.933 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7660066f30 msgr2=0x7f76600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7660066f30 0x7f76600693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76780029d0 tx=0x7f7678007d30 comp rx=0 tx=0).stop 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f767c0c7600 msgr2=0x7f767c0c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f767c0c7600 0x7f767c0c7a70 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f766c006ed0 tx=0x7f766c01d000 comp rx=0 tx=0).stop 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 shutdown_connections 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f767c169760 0x7f767c169bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7668004810 0x7f767c0a7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7660066f30 0x7f76600693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 --2- 172.21.15.131:0/2921856874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f767c0c7600 0x7f767c0c7a70 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 >> 172.21.15.131:0/2921856874 conn(0x7f767c068040 msgr2=0x7f767c156140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 shutdown_connections 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:14.934+0000 7f76837ff700 1 -- 172.21.15.131:0/2921856874 wait complete. 2023-12-26T21:53:14.936 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:14.952 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-26T21:53:13.076315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2441864,"kb_used_data":10312,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752532856,"statfs":{"total":773094113280,"available":770593644544,"internally_reserved":0,"allocated":10559488,"data_stored":5682636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001532"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.298141+0000","last_change":"2023-12-26T21:53:05.289049+0000","last_active":"2023-12-26T21:53:05.298141+0000","last_peered":"2023-12-26T21:53:05.298141+0000","last_clean":"2023-12-26T21:53:05.298141+0000","last_became_active":"2023-12-26T21:53:05.288687+0000","last_became_peered":"2023-12-26T21:53:05.288687+0000","last_unstale":"2023-12-26T21:53:05.298141+0000","last_undegraded":"2023-12-26T21:53:05.298141+0000","last_fullsized":"2023-12-26T21:53:05.298141+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:52.286840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.282288+0000","last_change":"2023-12-26T21:53:05.278284+0000","last_active":"2023-12-26T21:53:05.282288+0000","last_peered":"2023-12-26T21:53:05.282288+0000","last_clean":"2023-12-26T21:53:05.282288+0000","last_became_active":"2023-12-26T21:53:05.277672+0000","last_became_peered":"2023-12-26T21:53:05.277672+0000","last_unstale":"2023-12-26T21:53:05.282288+0000","last_undegraded":"2023-12-26T21:53:05.282288+0000","last_fullsized":"2023-12-26T21:53:05.282288+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:13:21.639979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.280736+0000","last_change":"2023-12-26T21:53:05.269473+0000","last_active":"2023-12-26T21:53:05.280736+0000","last_peered":"2023-12-26T21:53:05.280736+0000","last_clean":"2023-12-26T21:53:05.280736+0000","last_became_active":"2023-12-26T21:53:05.269149+0000","last_became_peered":"2023-12-26T21:53:05.269149+0000","last_unstale":"2023-12-26T21:53:05.280736+0000","last_undegraded":"2023-12-26T21:53:05.280736+0000","last_fullsized":"2023-12-26T21:53:05.280736+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:19.922173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.280769+0000","last_change":"2023-12-26T21:53:05.268989+0000","last_active":"2023-12-26T21:53:05.280769+0000","last_peered":"2023-12-26T21:53:05.280769+0000","last_clean":"2023-12-26T21:53:05.280769+0000","last_became_active":"2023-12-26T21:53:05.268632+0000","last_became_peered":"2023-12-26T21:53:05.268632+0000","last_unstale":"2023-12-26T21:53:05.280769+0000","last_undegraded":"2023-12-26T21:53:05.280769+0000","last_fullsized":"2023-12-26T21:53:05.280769+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:48:02.135228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.281175+0000","last_change":"2023-12-26T21:53:05.275156+0000","last_active":"2023-12-26T21:53:05.281175+0000","last_peered":"2023-12-26T21:53:05.281175+0000","last_clean":"2023-12-26T21:53:05.281175+0000","last_became_active":"2023-12-26T21:53:05.274871+0000","last_became_peered":"2023-12-26T21:53:05.274871+0000","last_unstale":"2023-12-26T21:53:05.281175+0000","last_undegraded":"2023-12-26T21:53:05.281175+0000","last_fullsized":"2023-12-26T21:53:05.281175+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:40:09.813749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287955+0000","last_change":"2023-12-26T21:53:05.281287+0000","last_active":"2023-12-26T21:53:05.287955+0000","last_peered":"2023-12-26T21:53:05.287955+0000","last_clean":"2023-12-26T21:53:05.287955+0000","last_became_active":"2023-12-26T21:53:05.281013+0000","last_became_peered":"2023-12-26T21:53:05.281013+0000","last_unstale":"2023-12-26T21:53:05.287955+0000","last_undegraded":"2023-12-26T21:53:05.287955+0000","last_fullsized":"2023-12-26T21:53:05.287955+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:38:31.950113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.275532+0000","last_change":"2023-12-26T21:53:05.271615+0000","last_active":"2023-12-26T21:53:05.275532+0000","last_peered":"2023-12-26T21:53:05.275532+0000","last_clean":"2023-12-26T21:53:05.275532+0000","last_became_active":"2023-12-26T21:53:05.270549+0000","last_became_peered":"2023-12-26T21:53:05.270549+0000","last_unstale":"2023-12-26T21:53:05.275532+0000","last_undegraded":"2023-12-26T21:53:05.275532+0000","last_fullsized":"2023-12-26T21:53:05.275532+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:19:17.880103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286534+0000","last_change":"2023-12-26T21:53:05.283059+0000","last_active":"2023-12-26T21:53:05.286534+0000","last_peered":"2023-12-26T21:53:05.286534+0000","last_clean":"2023-12-26T21:53:05.286534+0000","last_became_active":"2023-12-26T21:53:05.282417+0000","last_became_peered":"2023-12-26T21:53:05.282417+0000","last_unstale":"2023-12-26T21:53:05.286534+0000","last_undegraded":"2023-12-26T21:53:05.286534+0000","last_fullsized":"2023-12-26T21:53:05.286534+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:11:56.617393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286624+0000","last_change":"2023-12-26T21:53:05.279396+0000","last_active":"2023-12-26T21:53:05.286624+0000","last_peered":"2023-12-26T21:53:05.286624+0000","last_clean":"2023-12-26T21:53:05.286624+0000","last_became_active":"2023-12-26T21:53:05.278230+0000","last_became_peered":"2023-12-26T21:53:05.278230+0000","last_unstale":"2023-12-26T21:53:05.286624+0000","last_undegraded":"2023-12-26T21:53:05.286624+0000","last_fullsized":"2023-12-26T21:53:05.286624+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:44.183615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277521+0000","last_change":"2023-12-26T21:53:05.274464+0000","last_active":"2023-12-26T21:53:05.277521+0000","last_peered":"2023-12-26T21:53:05.277521+0000","last_clean":"2023-12-26T21:53:05.277521+0000","last_became_active":"2023-12-26T21:53:05.274140+0000","last_became_peered":"2023-12-26T21:53:05.274140+0000","last_unstale":"2023-12-26T21:53:05.277521+0000","last_undegraded":"2023-12-26T21:53:05.277521+0000","last_fullsized":"2023-12-26T21:53:05.277521+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:53:45.847934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288274+0000","last_change":"2023-12-26T21:53:05.286187+0000","last_active":"2023-12-26T21:53:05.288274+0000","last_peered":"2023-12-26T21:53:05.288274+0000","last_clean":"2023-12-26T21:53:05.288274+0000","last_became_active":"2023-12-26T21:53:05.285865+0000","last_became_peered":"2023-12-26T21:53:05.285865+0000","last_unstale":"2023-12-26T21:53:05.288274+0000","last_undegraded":"2023-12-26T21:53:05.288274+0000","last_fullsized":"2023-12-26T21:53:05.288274+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:56.825076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.292943+0000","last_change":"2023-12-26T21:53:05.290743+0000","last_active":"2023-12-26T21:53:05.292943+0000","last_peered":"2023-12-26T21:53:05.292943+0000","last_clean":"2023-12-26T21:53:05.292943+0000","last_became_active":"2023-12-26T21:53:05.290440+0000","last_became_peered":"2023-12-26T21:53:05.290440+0000","last_unstale":"2023-12-26T21:53:05.292943+0000","last_undegraded":"2023-12-26T21:53:05.292943+0000","last_fullsized":"2023-12-26T21:53:05.292943+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:26:31.991512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.284804+0000","last_change":"2023-12-26T21:53:05.271579+0000","last_active":"2023-12-26T21:53:05.284804+0000","last_peered":"2023-12-26T21:53:05.284804+0000","last_clean":"2023-12-26T21:53:05.284804+0000","last_became_active":"2023-12-26T21:53:05.271278+0000","last_became_peered":"2023-12-26T21:53:05.271278+0000","last_unstale":"2023-12-26T21:53:05.284804+0000","last_undegraded":"2023-12-26T21:53:05.284804+0000","last_fullsized":"2023-12-26T21:53:05.284804+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:23:10.752742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.296099+0000","last_change":"2023-12-26T21:53:05.293294+0000","last_active":"2023-12-26T21:53:05.296099+0000","last_peered":"2023-12-26T21:53:05.296099+0000","last_clean":"2023-12-26T21:53:05.296099+0000","last_became_active":"2023-12-26T21:53:05.292950+0000","last_became_peered":"2023-12-26T21:53:05.292950+0000","last_unstale":"2023-12-26T21:53:05.296099+0000","last_undegraded":"2023-12-26T21:53:05.296099+0000","last_fullsized":"2023-12-26T21:53:05.296099+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:36:13.991875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293166+0000","last_change":"2023-12-26T21:53:05.286014+0000","last_active":"2023-12-26T21:53:05.293166+0000","last_peered":"2023-12-26T21:53:05.293166+0000","last_clean":"2023-12-26T21:53:05.293166+0000","last_became_active":"2023-12-26T21:53:05.285657+0000","last_became_peered":"2023-12-26T21:53:05.285657+0000","last_unstale":"2023-12-26T21:53:05.293166+0000","last_undegraded":"2023-12-26T21:53:05.293166+0000","last_fullsized":"2023-12-26T21:53:05.293166+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:14:27.017262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286554+0000","last_change":"2023-12-26T21:53:05.283231+0000","last_active":"2023-12-26T21:53:05.286554+0000","last_peered":"2023-12-26T21:53:05.286554+0000","last_clean":"2023-12-26T21:53:05.286554+0000","last_became_active":"2023-12-26T21:53:05.282799+0000","last_became_peered":"2023-12-26T21:53:05.282799+0000","last_unstale":"2023-12-26T21:53:05.286554+0000","last_undegraded":"2023-12-26T21:53:05.286554+0000","last_fullsized":"2023-12-26T21:53:05.286554+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:12:37.012787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287779+0000","last_change":"2023-12-26T21:53:05.282286+0000","last_active":"2023-12-26T21:53:05.287779+0000","last_peered":"2023-12-26T21:53:05.287779+0000","last_clean":"2023-12-26T21:53:05.287779+0000","last_became_active":"2023-12-26T21:53:05.282036+0000","last_became_peered":"2023-12-26T21:53:05.282036+0000","last_unstale":"2023-12-26T21:53:05.287779+0000","last_undegraded":"2023-12-26T21:53:05.287779+0000","last_fullsized":"2023-12-26T21:53:05.287779+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:01:48.180876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.282819+0000","last_change":"2023-12-26T21:53:05.276742+0000","last_active":"2023-12-26T21:53:05.282819+0000","last_peered":"2023-12-26T21:53:05.282819+0000","last_clean":"2023-12-26T21:53:05.282819+0000","last_became_active":"2023-12-26T21:53:05.276279+0000","last_became_peered":"2023-12-26T21:53:05.276279+0000","last_unstale":"2023-12-26T21:53:05.282819+0000","last_undegraded":"2023-12-26T21:53:05.282819+0000","last_fullsized":"2023-12-26T21:53:05.282819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:59.099979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293935+0000","last_change":"2023-12-26T21:53:05.289933+0000","last_active":"2023-12-26T21:53:05.293935+0000","last_peered":"2023-12-26T21:53:05.293935+0000","last_clean":"2023-12-26T21:53:05.293935+0000","last_became_active":"2023-12-26T21:53:05.289653+0000","last_became_peered":"2023-12-26T21:53:05.289653+0000","last_unstale":"2023-12-26T21:53:05.293935+0000","last_undegraded":"2023-12-26T21:53:05.293935+0000","last_fullsized":"2023-12-26T21:53:05.293935+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:59.099979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287990+0000","last_change":"2023-12-26T21:53:05.282192+0000","last_active":"2023-12-26T21:53:05.287990+0000","last_peered":"2023-12-26T21:53:05.287990+0000","last_clean":"2023-12-26T21:53:05.287990+0000","last_became_active":"2023-12-26T21:53:05.281930+0000","last_became_peered":"2023-12-26T21:53:05.281930+0000","last_unstale":"2023-12-26T21:53:05.287990+0000","last_undegraded":"2023-12-26T21:53:05.287990+0000","last_fullsized":"2023-12-26T21:53:05.287990+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:15.519080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.283699+0000","last_change":"2023-12-26T21:53:05.282140+0000","last_active":"2023-12-26T21:53:05.283699+0000","last_peered":"2023-12-26T21:53:05.283699+0000","last_clean":"2023-12-26T21:53:05.283699+0000","last_became_active":"2023-12-26T21:53:05.281870+0000","last_became_peered":"2023-12-26T21:53:05.281870+0000","last_unstale":"2023-12-26T21:53:05.283699+0000","last_undegraded":"2023-12-26T21:53:05.283699+0000","last_fullsized":"2023-12-26T21:53:05.283699+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:37:33.021987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287165+0000","last_change":"2023-12-26T21:53:05.279440+0000","last_active":"2023-12-26T21:53:05.287165+0000","last_peered":"2023-12-26T21:53:05.287165+0000","last_clean":"2023-12-26T21:53:05.287165+0000","last_became_active":"2023-12-26T21:53:05.279154+0000","last_became_peered":"2023-12-26T21:53:05.279154+0000","last_unstale":"2023-12-26T21:53:05.287165+0000","last_undegraded":"2023-12-26T21:53:05.287165+0000","last_fullsized":"2023-12-26T21:53:05.287165+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:26:31.991512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.275551+0000","last_change":"2023-12-26T21:53:05.273034+0000","last_active":"2023-12-26T21:53:05.275551+0000","last_peered":"2023-12-26T21:53:05.275551+0000","last_clean":"2023-12-26T21:53:05.275551+0000","last_became_active":"2023-12-26T21:53:05.272727+0000","last_became_peered":"2023-12-26T21:53:05.272727+0000","last_unstale":"2023-12-26T21:53:05.275551+0000","last_undegraded":"2023-12-26T21:53:05.275551+0000","last_fullsized":"2023-12-26T21:53:05.275551+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:56:41.642952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293630+0000","last_change":"2023-12-26T21:53:05.286668+0000","last_active":"2023-12-26T21:53:05.293630+0000","last_peered":"2023-12-26T21:53:05.293630+0000","last_clean":"2023-12-26T21:53:05.293630+0000","last_became_active":"2023-12-26T21:53:05.286348+0000","last_became_peered":"2023-12-26T21:53:05.286348+0000","last_unstale":"2023-12-26T21:53:05.293630+0000","last_undegraded":"2023-12-26T21:53:05.293630+0000","last_fullsized":"2023-12-26T21:53:05.293630+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:06:28.041436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.275929+0000","last_change":"2023-12-26T21:53:05.273113+0000","last_active":"2023-12-26T21:53:05.275929+0000","last_peered":"2023-12-26T21:53:05.275929+0000","last_clean":"2023-12-26T21:53:05.275929+0000","last_became_active":"2023-12-26T21:53:05.272876+0000","last_became_peered":"2023-12-26T21:53:05.272876+0000","last_unstale":"2023-12-26T21:53:05.275929+0000","last_undegraded":"2023-12-26T21:53:05.275929+0000","last_fullsized":"2023-12-26T21:53:05.275929+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:13:24.873466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294036+0000","last_change":"2023-12-26T21:53:05.290390+0000","last_active":"2023-12-26T21:53:05.294036+0000","last_peered":"2023-12-26T21:53:05.294036+0000","last_clean":"2023-12-26T21:53:05.294036+0000","last_became_active":"2023-12-26T21:53:05.290100+0000","last_became_peered":"2023-12-26T21:53:05.290100+0000","last_unstale":"2023-12-26T21:53:05.294036+0000","last_undegraded":"2023-12-26T21:53:05.294036+0000","last_fullsized":"2023-12-26T21:53:05.294036+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:22:39.661507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294394+0000","last_change":"2023-12-26T21:53:05.290485+0000","last_active":"2023-12-26T21:53:05.294394+0000","last_peered":"2023-12-26T21:53:05.294394+0000","last_clean":"2023-12-26T21:53:05.294394+0000","last_became_active":"2023-12-26T21:53:05.290018+0000","last_became_peered":"2023-12-26T21:53:05.290018+0000","last_unstale":"2023-12-26T21:53:05.294394+0000","last_undegraded":"2023-12-26T21:53:05.294394+0000","last_fullsized":"2023-12-26T21:53:05.294394+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:13:56.991990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.290915+0000","last_change":"2023-12-26T21:53:05.288396+0000","last_active":"2023-12-26T21:53:05.290915+0000","last_peered":"2023-12-26T21:53:05.290915+0000","last_clean":"2023-12-26T21:53:05.290915+0000","last_became_active":"2023-12-26T21:53:05.284936+0000","last_became_peered":"2023-12-26T21:53:05.284936+0000","last_unstale":"2023-12-26T21:53:05.290915+0000","last_undegraded":"2023-12-26T21:53:05.290915+0000","last_fullsized":"2023-12-26T21:53:05.290915+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:01:48.180876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286211+0000","last_change":"2023-12-26T21:53:05.280350+0000","last_active":"2023-12-26T21:53:05.286211+0000","last_peered":"2023-12-26T21:53:05.286211+0000","last_clean":"2023-12-26T21:53:05.286211+0000","last_became_active":"2023-12-26T21:53:05.280012+0000","last_became_peered":"2023-12-26T21:53:05.280012+0000","last_unstale":"2023-12-26T21:53:05.286211+0000","last_undegraded":"2023-12-26T21:53:05.286211+0000","last_fullsized":"2023-12-26T21:53:05.286211+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:08.450696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294476+0000","last_change":"2023-12-26T21:53:05.290633+0000","last_active":"2023-12-26T21:53:05.294476+0000","last_peered":"2023-12-26T21:53:05.294476+0000","last_clean":"2023-12-26T21:53:05.294476+0000","last_became_active":"2023-12-26T21:53:05.289344+0000","last_became_peered":"2023-12-26T21:53:05.289344+0000","last_unstale":"2023-12-26T21:53:05.294476+0000","last_undegraded":"2023-12-26T21:53:05.294476+0000","last_fullsized":"2023-12-26T21:53:05.294476+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:15.904401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293831+0000","last_change":"2023-12-26T21:53:05.291158+0000","last_active":"2023-12-26T21:53:05.293831+0000","last_peered":"2023-12-26T21:53:05.293831+0000","last_clean":"2023-12-26T21:53:05.293831+0000","last_became_active":"2023-12-26T21:53:05.290863+0000","last_became_peered":"2023-12-26T21:53:05.290863+0000","last_unstale":"2023-12-26T21:53:05.293831+0000","last_undegraded":"2023-12-26T21:53:05.293831+0000","last_fullsized":"2023-12-26T21:53:05.293831+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:59.060036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.289040+0000","last_change":"2023-12-26T21:53:05.286142+0000","last_active":"2023-12-26T21:53:05.289040+0000","last_peered":"2023-12-26T21:53:05.289040+0000","last_clean":"2023-12-26T21:53:05.289040+0000","last_became_active":"2023-12-26T21:53:05.285761+0000","last_became_peered":"2023-12-26T21:53:05.285761+0000","last_unstale":"2023-12-26T21:53:05.289040+0000","last_undegraded":"2023-12-26T21:53:05.289040+0000","last_fullsized":"2023-12-26T21:53:05.289040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:36:44.951360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286999+0000","last_change":"2023-12-26T21:53:05.282844+0000","last_active":"2023-12-26T21:53:05.286999+0000","last_peered":"2023-12-26T21:53:05.286999+0000","last_clean":"2023-12-26T21:53:05.286999+0000","last_became_active":"2023-12-26T21:53:05.282596+0000","last_became_peered":"2023-12-26T21:53:05.282596+0000","last_unstale":"2023-12-26T21:53:05.286999+0000","last_undegraded":"2023-12-26T21:53:05.286999+0000","last_fullsized":"2023-12-26T21:53:05.286999+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:49:30.259823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"95'4","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.303732+0000","last_change":"2023-12-26T21:53:04.270399+0000","last_active":"2023-12-26T21:53:05.303732+0000","last_peered":"2023-12-26T21:53:05.303732+0000","last_clean":"2023-12-26T21:53:05.303732+0000","last_became_active":"2023-12-26T21:53:04.270117+0000","last_became_peered":"2023-12-26T21:53:04.270117+0000","last_unstale":"2023-12-26T21:53:05.303732+0000","last_undegraded":"2023-12-26T21:53:05.303732+0000","last_fullsized":"2023-12-26T21:53:05.303732+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:13:31.882896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.295742+0000","last_change":"2023-12-26T21:53:05.291846+0000","last_active":"2023-12-26T21:53:05.295742+0000","last_peered":"2023-12-26T21:53:05.295742+0000","last_clean":"2023-12-26T21:53:05.295742+0000","last_became_active":"2023-12-26T21:53:05.291574+0000","last_became_peered":"2023-12-26T21:53:05.291574+0000","last_unstale":"2023-12-26T21:53:05.295742+0000","last_undegraded":"2023-12-26T21:53:05.295742+0000","last_fullsized":"2023-12-26T21:53:05.295742+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:02.301077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.296485+0000","last_change":"2023-12-26T21:53:04.262700+0000","last_active":"2023-12-26T21:53:05.296485+0000","last_peered":"2023-12-26T21:53:05.296485+0000","last_clean":"2023-12-26T21:53:05.296485+0000","last_became_active":"2023-12-26T21:53:04.262378+0000","last_became_peered":"2023-12-26T21:53:04.262378+0000","last_unstale":"2023-12-26T21:53:05.296485+0000","last_undegraded":"2023-12-26T21:53:05.296485+0000","last_fullsized":"2023-12-26T21:53:05.296485+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:58.054482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288056+0000","last_change":"2023-12-26T21:53:05.281239+0000","last_active":"2023-12-26T21:53:05.288056+0000","last_peered":"2023-12-26T21:53:05.288056+0000","last_clean":"2023-12-26T21:53:05.288056+0000","last_became_active":"2023-12-26T21:53:05.280899+0000","last_became_peered":"2023-12-26T21:53:05.280899+0000","last_unstale":"2023-12-26T21:53:05.288056+0000","last_undegraded":"2023-12-26T21:53:05.288056+0000","last_fullsized":"2023-12-26T21:53:05.288056+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:46:33.933439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.299915+0000","last_change":"2023-12-26T21:53:04.262003+0000","last_active":"2023-12-26T21:53:05.299915+0000","last_peered":"2023-12-26T21:53:05.299915+0000","last_clean":"2023-12-26T21:53:05.299915+0000","last_became_active":"2023-12-26T21:53:04.261800+0000","last_became_peered":"2023-12-26T21:53:04.261800+0000","last_unstale":"2023-12-26T21:53:05.299915+0000","last_undegraded":"2023-12-26T21:53:05.299915+0000","last_fullsized":"2023-12-26T21:53:05.299915+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:00:23.255023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287438+0000","last_change":"2023-12-26T21:53:05.285641+0000","last_active":"2023-12-26T21:53:05.287438+0000","last_peered":"2023-12-26T21:53:05.287438+0000","last_clean":"2023-12-26T21:53:05.287438+0000","last_became_active":"2023-12-26T21:53:05.285318+0000","last_became_peered":"2023-12-26T21:53:05.285318+0000","last_unstale":"2023-12-26T21:53:05.287438+0000","last_undegraded":"2023-12-26T21:53:05.287438+0000","last_fullsized":"2023-12-26T21:53:05.287438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:28:21.116516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.301350+0000","last_change":"2023-12-26T21:53:04.262466+0000","last_active":"2023-12-26T21:53:05.301350+0000","last_peered":"2023-12-26T21:53:05.301350+0000","last_clean":"2023-12-26T21:53:05.301350+0000","last_became_active":"2023-12-26T21:53:04.262140+0000","last_became_peered":"2023-12-26T21:53:04.262140+0000","last_unstale":"2023-12-26T21:53:05.301350+0000","last_undegraded":"2023-12-26T21:53:05.301350+0000","last_fullsized":"2023-12-26T21:53:05.301350+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:46:20.997096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286063+0000","last_change":"2023-12-26T21:53:05.283242+0000","last_active":"2023-12-26T21:53:05.286063+0000","last_peered":"2023-12-26T21:53:05.286063+0000","last_clean":"2023-12-26T21:53:05.286063+0000","last_became_active":"2023-12-26T21:53:05.282980+0000","last_became_peered":"2023-12-26T21:53:05.282980+0000","last_unstale":"2023-12-26T21:53:05.286063+0000","last_undegraded":"2023-12-26T21:53:05.286063+0000","last_fullsized":"2023-12-26T21:53:05.286063+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:48:57.465172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.299674+0000","last_change":"2023-12-26T21:53:04.267058+0000","last_active":"2023-12-26T21:53:05.299674+0000","last_peered":"2023-12-26T21:53:05.299674+0000","last_clean":"2023-12-26T21:53:05.299674+0000","last_became_active":"2023-12-26T21:53:04.264901+0000","last_became_peered":"2023-12-26T21:53:04.264901+0000","last_unstale":"2023-12-26T21:53:05.299674+0000","last_undegraded":"2023-12-26T21:53:05.299674+0000","last_fullsized":"2023-12-26T21:53:05.299674+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:40.412864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287254+0000","last_change":"2023-12-26T21:53:05.280108+0000","last_active":"2023-12-26T21:53:05.287254+0000","last_peered":"2023-12-26T21:53:05.287254+0000","last_clean":"2023-12-26T21:53:05.287254+0000","last_became_active":"2023-12-26T21:53:05.279864+0000","last_became_peered":"2023-12-26T21:53:05.279864+0000","last_unstale":"2023-12-26T21:53:05.287254+0000","last_undegraded":"2023-12-26T21:53:05.287254+0000","last_fullsized":"2023-12-26T21:53:05.287254+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:52:15.234127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.298157+0000","last_change":"2023-12-26T21:53:04.260812+0000","last_active":"2023-12-26T21:53:05.298157+0000","last_peered":"2023-12-26T21:53:05.298157+0000","last_clean":"2023-12-26T21:53:05.298157+0000","last_became_active":"2023-12-26T21:53:04.260497+0000","last_became_peered":"2023-12-26T21:53:04.260497+0000","last_unstale":"2023-12-26T21:53:05.298157+0000","last_undegraded":"2023-12-26T21:53:05.298157+0000","last_fullsized":"2023-12-26T21:53:05.298157+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:15:56.273951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.298144+0000","last_change":"2023-12-26T21:53:05.291430+0000","last_active":"2023-12-26T21:53:05.298144+0000","last_peered":"2023-12-26T21:53:05.298144+0000","last_clean":"2023-12-26T21:53:05.298144+0000","last_became_active":"2023-12-26T21:53:05.291126+0000","last_became_peered":"2023-12-26T21:53:05.291126+0000","last_unstale":"2023-12-26T21:53:05.298144+0000","last_undegraded":"2023-12-26T21:53:05.298144+0000","last_fullsized":"2023-12-26T21:53:05.298144+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:15:36.453863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277179+0000","last_change":"2023-12-26T21:53:04.261961+0000","last_active":"2023-12-26T21:53:05.277179+0000","last_peered":"2023-12-26T21:53:05.277179+0000","last_clean":"2023-12-26T21:53:05.277179+0000","last_became_active":"2023-12-26T21:53:04.260903+0000","last_became_peered":"2023-12-26T21:53:04.260903+0000","last_unstale":"2023-12-26T21:53:05.277179+0000","last_undegraded":"2023-12-26T21:53:05.277179+0000","last_fullsized":"2023-12-26T21:53:05.277179+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:20:48.412586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293188+0000","last_change":"2023-12-26T21:53:05.291513+0000","last_active":"2023-12-26T21:53:05.293188+0000","last_peered":"2023-12-26T21:53:05.293188+0000","last_clean":"2023-12-26T21:53:05.293188+0000","last_became_active":"2023-12-26T21:53:05.289718+0000","last_became_peered":"2023-12-26T21:53:05.289718+0000","last_unstale":"2023-12-26T21:53:05.293188+0000","last_undegraded":"2023-12-26T21:53:05.293188+0000","last_fullsized":"2023-12-26T21:53:05.293188+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:42:40.127448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.271145+0000","last_change":"2023-12-26T21:53:04.259820+0000","last_active":"2023-12-26T21:53:05.271145+0000","last_peered":"2023-12-26T21:53:05.271145+0000","last_clean":"2023-12-26T21:53:05.271145+0000","last_became_active":"2023-12-26T21:53:04.259549+0000","last_became_peered":"2023-12-26T21:53:04.259549+0000","last_unstale":"2023-12-26T21:53:05.271145+0000","last_undegraded":"2023-12-26T21:53:05.271145+0000","last_fullsized":"2023-12-26T21:53:05.271145+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:43:17.471063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277501+0000","last_change":"2023-12-26T21:53:05.273198+0000","last_active":"2023-12-26T21:53:05.277501+0000","last_peered":"2023-12-26T21:53:05.277501+0000","last_clean":"2023-12-26T21:53:05.277501+0000","last_became_active":"2023-12-26T21:53:05.272661+0000","last_became_peered":"2023-12-26T21:53:05.272661+0000","last_unstale":"2023-12-26T21:53:05.277501+0000","last_undegraded":"2023-12-26T21:53:05.277501+0000","last_fullsized":"2023-12-26T21:53:05.277501+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:58:14.337531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.276858+0000","last_change":"2023-12-26T21:53:04.265435+0000","last_active":"2023-12-26T21:53:05.276858+0000","last_peered":"2023-12-26T21:53:05.276858+0000","last_clean":"2023-12-26T21:53:05.276858+0000","last_became_active":"2023-12-26T21:53:04.264942+0000","last_became_peered":"2023-12-26T21:53:04.264942+0000","last_unstale":"2023-12-26T21:53:05.276858+0000","last_undegraded":"2023-12-26T21:53:05.276858+0000","last_fullsized":"2023-12-26T21:53:05.276858+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:17:30.720991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.282429+0000","last_change":"2023-12-26T21:53:05.279536+0000","last_active":"2023-12-26T21:53:05.282429+0000","last_peered":"2023-12-26T21:53:05.282429+0000","last_clean":"2023-12-26T21:53:05.282429+0000","last_became_active":"2023-12-26T21:53:05.279236+0000","last_became_peered":"2023-12-26T21:53:05.279236+0000","last_unstale":"2023-12-26T21:53:05.282429+0000","last_undegraded":"2023-12-26T21:53:05.282429+0000","last_fullsized":"2023-12-26T21:53:05.282429+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:17:31.726545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.268670+0000","last_change":"2023-12-26T21:53:04.272739+0000","last_active":"2023-12-26T21:53:05.268670+0000","last_peered":"2023-12-26T21:53:05.268670+0000","last_clean":"2023-12-26T21:53:05.268670+0000","last_became_active":"2023-12-26T21:53:04.272478+0000","last_became_peered":"2023-12-26T21:53:04.272478+0000","last_unstale":"2023-12-26T21:53:05.268670+0000","last_undegraded":"2023-12-26T21:53:05.268670+0000","last_fullsized":"2023-12-26T21:53:05.268670+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:10.638441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.293993+0000","last_change":"2023-12-26T21:53:05.287348+0000","last_active":"2023-12-26T21:53:05.293993+0000","last_peered":"2023-12-26T21:53:05.293993+0000","last_clean":"2023-12-26T21:53:05.293993+0000","last_became_active":"2023-12-26T21:53:05.286579+0000","last_became_peered":"2023-12-26T21:53:05.286579+0000","last_unstale":"2023-12-26T21:53:05.293993+0000","last_undegraded":"2023-12-26T21:53:05.293993+0000","last_fullsized":"2023-12-26T21:53:05.293993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:13:56.991990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"95'3","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.303439+0000","last_change":"2023-12-26T21:53:04.262478+0000","last_active":"2023-12-26T21:53:05.303439+0000","last_peered":"2023-12-26T21:53:05.303439+0000","last_clean":"2023-12-26T21:53:05.303439+0000","last_became_active":"2023-12-26T21:53:04.262209+0000","last_became_peered":"2023-12-26T21:53:04.262209+0000","last_unstale":"2023-12-26T21:53:05.303439+0000","last_undegraded":"2023-12-26T21:53:05.303439+0000","last_fullsized":"2023-12-26T21:53:05.303439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:03:39.647487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294376+0000","last_change":"2023-12-26T21:53:05.290318+0000","last_active":"2023-12-26T21:53:05.294376+0000","last_peered":"2023-12-26T21:53:05.294376+0000","last_clean":"2023-12-26T21:53:05.294376+0000","last_became_active":"2023-12-26T21:53:05.289587+0000","last_became_peered":"2023-12-26T21:53:05.289587+0000","last_unstale":"2023-12-26T21:53:05.294376+0000","last_undegraded":"2023-12-26T21:53:05.294376+0000","last_fullsized":"2023-12-26T21:53:05.294376+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:46:19.715350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"95'6","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.303735+0000","last_change":"2023-12-26T21:53:04.260381+0000","last_active":"2023-12-26T21:53:05.303735+0000","last_peered":"2023-12-26T21:53:05.303735+0000","last_clean":"2023-12-26T21:53:05.303735+0000","last_became_active":"2023-12-26T21:53:04.260099+0000","last_became_peered":"2023-12-26T21:53:04.260099+0000","last_unstale":"2023-12-26T21:53:05.303735+0000","last_undegraded":"2023-12-26T21:53:05.303735+0000","last_fullsized":"2023-12-26T21:53:05.303735+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:59:37.808016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288136+0000","last_change":"2023-12-26T21:53:05.283574+0000","last_active":"2023-12-26T21:53:05.288136+0000","last_peered":"2023-12-26T21:53:05.288136+0000","last_clean":"2023-12-26T21:53:05.288136+0000","last_became_active":"2023-12-26T21:53:05.283328+0000","last_became_peered":"2023-12-26T21:53:05.283328+0000","last_unstale":"2023-12-26T21:53:05.288136+0000","last_undegraded":"2023-12-26T21:53:05.288136+0000","last_fullsized":"2023-12-26T21:53:05.288136+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:11.021924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.302090+0000","last_change":"2023-12-26T21:53:04.265036+0000","last_active":"2023-12-26T21:53:05.302090+0000","last_peered":"2023-12-26T21:53:05.302090+0000","last_clean":"2023-12-26T21:53:05.302090+0000","last_became_active":"2023-12-26T21:53:04.264765+0000","last_became_peered":"2023-12-26T21:53:04.264765+0000","last_unstale":"2023-12-26T21:53:05.302090+0000","last_undegraded":"2023-12-26T21:53:05.302090+0000","last_fullsized":"2023-12-26T21:53:05.302090+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:55.298639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287789+0000","last_change":"2023-12-26T21:53:05.283134+0000","last_active":"2023-12-26T21:53:05.287789+0000","last_peered":"2023-12-26T21:53:05.287789+0000","last_clean":"2023-12-26T21:53:05.287789+0000","last_became_active":"2023-12-26T21:53:05.280388+0000","last_became_peered":"2023-12-26T21:53:05.280388+0000","last_unstale":"2023-12-26T21:53:05.287789+0000","last_undegraded":"2023-12-26T21:53:05.287789+0000","last_fullsized":"2023-12-26T21:53:05.287789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:23:55.238759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.264081+0000","last_change":"2023-12-26T21:53:04.265447+0000","last_active":"2023-12-26T21:53:05.264081+0000","last_peered":"2023-12-26T21:53:05.264081+0000","last_clean":"2023-12-26T21:53:05.264081+0000","last_became_active":"2023-12-26T21:53:04.265176+0000","last_became_peered":"2023-12-26T21:53:04.265176+0000","last_unstale":"2023-12-26T21:53:05.264081+0000","last_undegraded":"2023-12-26T21:53:05.264081+0000","last_fullsized":"2023-12-26T21:53:05.264081+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:24.737315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.276642+0000","last_change":"2023-12-26T21:53:05.273244+0000","last_active":"2023-12-26T21:53:05.276642+0000","last_peered":"2023-12-26T21:53:05.276642+0000","last_clean":"2023-12-26T21:53:05.276642+0000","last_became_active":"2023-12-26T21:53:05.272924+0000","last_became_peered":"2023-12-26T21:53:05.272924+0000","last_unstale":"2023-12-26T21:53:05.276642+0000","last_undegraded":"2023-12-26T21:53:05.276642+0000","last_fullsized":"2023-12-26T21:53:05.276642+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:19:20.160976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.264417+0000","last_change":"2023-12-26T21:53:04.264321+0000","last_active":"2023-12-26T21:53:05.264417+0000","last_peered":"2023-12-26T21:53:05.264417+0000","last_clean":"2023-12-26T21:53:05.264417+0000","last_became_active":"2023-12-26T21:53:04.264050+0000","last_became_peered":"2023-12-26T21:53:04.264050+0000","last_unstale":"2023-12-26T21:53:05.264417+0000","last_undegraded":"2023-12-26T21:53:05.264417+0000","last_fullsized":"2023-12-26T21:53:05.264417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:02:29.457133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.295815+0000","last_change":"2023-12-26T21:53:05.289902+0000","last_active":"2023-12-26T21:53:05.295815+0000","last_peered":"2023-12-26T21:53:05.295815+0000","last_clean":"2023-12-26T21:53:05.295815+0000","last_became_active":"2023-12-26T21:53:05.289623+0000","last_became_peered":"2023-12-26T21:53:05.289623+0000","last_unstale":"2023-12-26T21:53:05.295815+0000","last_undegraded":"2023-12-26T21:53:05.295815+0000","last_fullsized":"2023-12-26T21:53:05.295815+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:47:45.766327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"95'8","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.302755+0000","last_change":"2023-12-26T21:53:04.268072+0000","last_active":"2023-12-26T21:53:05.302755+0000","last_peered":"2023-12-26T21:53:05.302755+0000","last_clean":"2023-12-26T21:53:05.302755+0000","last_became_active":"2023-12-26T21:53:04.267818+0000","last_became_peered":"2023-12-26T21:53:04.267818+0000","last_unstale":"2023-12-26T21:53:05.302755+0000","last_undegraded":"2023-12-26T21:53:05.302755+0000","last_fullsized":"2023-12-26T21:53:05.302755+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:39.432599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.282308+0000","last_change":"2023-12-26T21:53:05.278470+0000","last_active":"2023-12-26T21:53:05.282308+0000","last_peered":"2023-12-26T21:53:05.282308+0000","last_clean":"2023-12-26T21:53:05.282308+0000","last_became_active":"2023-12-26T21:53:05.278014+0000","last_became_peered":"2023-12-26T21:53:05.278014+0000","last_unstale":"2023-12-26T21:53:05.282308+0000","last_undegraded":"2023-12-26T21:53:05.282308+0000","last_fullsized":"2023-12-26T21:53:05.282308+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:14.635797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.303685+0000","last_change":"2023-12-26T21:53:04.265951+0000","last_active":"2023-12-26T21:53:05.303685+0000","last_peered":"2023-12-26T21:53:05.303685+0000","last_clean":"2023-12-26T21:53:05.303685+0000","last_became_active":"2023-12-26T21:53:04.265704+0000","last_became_peered":"2023-12-26T21:53:04.265704+0000","last_unstale":"2023-12-26T21:53:05.303685+0000","last_undegraded":"2023-12-26T21:53:05.303685+0000","last_fullsized":"2023-12-26T21:53:05.303685+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:13:20.634426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.285593+0000","last_change":"2023-12-26T21:53:05.284308+0000","last_active":"2023-12-26T21:53:05.285593+0000","last_peered":"2023-12-26T21:53:05.285593+0000","last_clean":"2023-12-26T21:53:05.285593+0000","last_became_active":"2023-12-26T21:53:05.283912+0000","last_became_peered":"2023-12-26T21:53:05.283912+0000","last_unstale":"2023-12-26T21:53:05.285593+0000","last_undegraded":"2023-12-26T21:53:05.285593+0000","last_fullsized":"2023-12-26T21:53:05.285593+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:52:31.443697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.298760+0000","last_change":"2023-12-26T21:53:04.270594+0000","last_active":"2023-12-26T21:53:05.298760+0000","last_peered":"2023-12-26T21:53:05.298760+0000","last_clean":"2023-12-26T21:53:05.298760+0000","last_became_active":"2023-12-26T21:53:04.270353+0000","last_became_peered":"2023-12-26T21:53:04.270353+0000","last_unstale":"2023-12-26T21:53:05.298760+0000","last_undegraded":"2023-12-26T21:53:05.298760+0000","last_fullsized":"2023-12-26T21:53:05.298760+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:28:07.301140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286680+0000","last_change":"2023-12-26T21:53:05.283823+0000","last_active":"2023-12-26T21:53:05.286680+0000","last_peered":"2023-12-26T21:53:05.286680+0000","last_clean":"2023-12-26T21:53:05.286680+0000","last_became_active":"2023-12-26T21:53:05.283463+0000","last_became_peered":"2023-12-26T21:53:05.283463+0000","last_unstale":"2023-12-26T21:53:05.286680+0000","last_undegraded":"2023-12-26T21:53:05.286680+0000","last_fullsized":"2023-12-26T21:53:05.286680+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:43:18.476616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"95'1","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.299474+0000","last_change":"2023-12-26T21:53:04.270403+0000","last_active":"2023-12-26T21:53:05.299474+0000","last_peered":"2023-12-26T21:53:05.299474+0000","last_clean":"2023-12-26T21:53:05.299474+0000","last_became_active":"2023-12-26T21:53:04.270117+0000","last_became_peered":"2023-12-26T21:53:04.270117+0000","last_unstale":"2023-12-26T21:53:05.299474+0000","last_undegraded":"2023-12-26T21:53:05.299474+0000","last_fullsized":"2023-12-26T21:53:05.299474+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:22:41.561642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.286264+0000","last_change":"2023-12-26T21:53:05.280232+0000","last_active":"2023-12-26T21:53:05.286264+0000","last_peered":"2023-12-26T21:53:05.286264+0000","last_clean":"2023-12-26T21:53:05.286264+0000","last_became_active":"2023-12-26T21:53:05.279934+0000","last_became_peered":"2023-12-26T21:53:05.279934+0000","last_unstale":"2023-12-26T21:53:05.286264+0000","last_undegraded":"2023-12-26T21:53:05.286264+0000","last_fullsized":"2023-12-26T21:53:05.286264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:41.418418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277177+0000","last_change":"2023-12-26T21:53:04.269097+0000","last_active":"2023-12-26T21:53:05.277177+0000","last_peered":"2023-12-26T21:53:05.277177+0000","last_clean":"2023-12-26T21:53:05.277177+0000","last_became_active":"2023-12-26T21:53:04.268508+0000","last_became_peered":"2023-12-26T21:53:04.268508+0000","last_unstale":"2023-12-26T21:53:05.277177+0000","last_undegraded":"2023-12-26T21:53:05.277177+0000","last_fullsized":"2023-12-26T21:53:05.277177+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:03:39.647487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.296192+0000","last_change":"2023-12-26T21:53:05.288594+0000","last_active":"2023-12-26T21:53:05.296192+0000","last_peered":"2023-12-26T21:53:05.296192+0000","last_clean":"2023-12-26T21:53:05.296192+0000","last_became_active":"2023-12-26T21:53:05.288312+0000","last_became_peered":"2023-12-26T21:53:05.288312+0000","last_unstale":"2023-12-26T21:53:05.296192+0000","last_undegraded":"2023-12-26T21:53:05.296192+0000","last_fullsized":"2023-12-26T21:53:05.296192+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:34.050678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.277502+0000","last_change":"2023-12-26T21:53:04.272377+0000","last_active":"2023-12-26T21:53:05.277502+0000","last_peered":"2023-12-26T21:53:05.277502+0000","last_clean":"2023-12-26T21:53:05.277502+0000","last_became_active":"2023-12-26T21:53:04.270643+0000","last_became_peered":"2023-12-26T21:53:04.270643+0000","last_unstale":"2023-12-26T21:53:05.277502+0000","last_undegraded":"2023-12-26T21:53:05.277502+0000","last_fullsized":"2023-12-26T21:53:05.277502+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:31:33.643977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288252+0000","last_change":"2023-12-26T21:53:05.285016+0000","last_active":"2023-12-26T21:53:05.288252+0000","last_peered":"2023-12-26T21:53:05.288252+0000","last_clean":"2023-12-26T21:53:05.288252+0000","last_became_active":"2023-12-26T21:53:05.283467+0000","last_became_peered":"2023-12-26T21:53:05.283467+0000","last_unstale":"2023-12-26T21:53:05.288252+0000","last_undegraded":"2023-12-26T21:53:05.288252+0000","last_fullsized":"2023-12-26T21:53:05.288252+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:13.863148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.269265+0000","last_change":"2023-12-26T21:53:04.269596+0000","last_active":"2023-12-26T21:53:05.269265+0000","last_peered":"2023-12-26T21:53:05.269265+0000","last_clean":"2023-12-26T21:53:05.269265+0000","last_became_active":"2023-12-26T21:53:04.269314+0000","last_became_peered":"2023-12-26T21:53:04.269314+0000","last_unstale":"2023-12-26T21:53:05.269265+0000","last_undegraded":"2023-12-26T21:53:05.269265+0000","last_fullsized":"2023-12-26T21:53:05.269265+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:33:19.638070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294572+0000","last_change":"2023-12-26T21:53:05.292994+0000","last_active":"2023-12-26T21:53:05.294572+0000","last_peered":"2023-12-26T21:53:05.294572+0000","last_clean":"2023-12-26T21:53:05.294572+0000","last_became_active":"2023-12-26T21:53:05.289294+0000","last_became_peered":"2023-12-26T21:53:05.289294+0000","last_unstale":"2023-12-26T21:53:05.294572+0000","last_undegraded":"2023-12-26T21:53:05.294572+0000","last_fullsized":"2023-12-26T21:53:05.294572+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:02:39.101474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.268459+0000","last_change":"2023-12-26T21:53:04.270655+0000","last_active":"2023-12-26T21:53:05.268459+0000","last_peered":"2023-12-26T21:53:05.268459+0000","last_clean":"2023-12-26T21:53:05.268459+0000","last_became_active":"2023-12-26T21:53:04.270381+0000","last_became_peered":"2023-12-26T21:53:04.270381+0000","last_unstale":"2023-12-26T21:53:05.268459+0000","last_undegraded":"2023-12-26T21:53:05.268459+0000","last_fullsized":"2023-12-26T21:53:05.268459+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:39.475708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294233+0000","last_change":"2023-12-26T21:53:05.288484+0000","last_active":"2023-12-26T21:53:05.294233+0000","last_peered":"2023-12-26T21:53:05.294233+0000","last_clean":"2023-12-26T21:53:05.294233+0000","last_became_active":"2023-12-26T21:53:05.288199+0000","last_became_peered":"2023-12-26T21:53:05.288199+0000","last_unstale":"2023-12-26T21:53:05.294233+0000","last_undegraded":"2023-12-26T21:53:05.294233+0000","last_fullsized":"2023-12-26T21:53:05.294233+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:53.622961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.300999+0000","last_change":"2023-12-26T21:53:04.265832+0000","last_active":"2023-12-26T21:53:05.300999+0000","last_peered":"2023-12-26T21:53:05.300999+0000","last_clean":"2023-12-26T21:53:05.300999+0000","last_became_active":"2023-12-26T21:53:04.265580+0000","last_became_peered":"2023-12-26T21:53:04.265580+0000","last_unstale":"2023-12-26T21:53:05.300999+0000","last_undegraded":"2023-12-26T21:53:05.300999+0000","last_fullsized":"2023-12-26T21:53:05.300999+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:45:13.630243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294323+0000","last_change":"2023-12-26T21:53:05.289951+0000","last_active":"2023-12-26T21:53:05.294323+0000","last_peered":"2023-12-26T21:53:05.294323+0000","last_clean":"2023-12-26T21:53:05.294323+0000","last_became_active":"2023-12-26T21:53:05.289699+0000","last_became_peered":"2023-12-26T21:53:05.289699+0000","last_unstale":"2023-12-26T21:53:05.294323+0000","last_undegraded":"2023-12-26T21:53:05.294323+0000","last_fullsized":"2023-12-26T21:53:05.294323+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:07:16.289438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.275437+0000","last_change":"2023-12-26T21:53:04.263420+0000","last_active":"2023-12-26T21:53:05.275437+0000","last_peered":"2023-12-26T21:53:05.275437+0000","last_clean":"2023-12-26T21:53:05.275437+0000","last_became_active":"2023-12-26T21:53:04.263154+0000","last_became_peered":"2023-12-26T21:53:04.263154+0000","last_unstale":"2023-12-26T21:53:05.275437+0000","last_undegraded":"2023-12-26T21:53:05.275437+0000","last_fullsized":"2023-12-26T21:53:05.275437+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:40:02.936486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288100+0000","last_change":"2023-12-26T21:53:05.284585+0000","last_active":"2023-12-26T21:53:05.288100+0000","last_peered":"2023-12-26T21:53:05.288100+0000","last_clean":"2023-12-26T21:53:05.288100+0000","last_became_active":"2023-12-26T21:53:05.282315+0000","last_became_peered":"2023-12-26T21:53:05.282315+0000","last_unstale":"2023-12-26T21:53:05.288100+0000","last_undegraded":"2023-12-26T21:53:05.288100+0000","last_fullsized":"2023-12-26T21:53:05.288100+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:40.481262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.272793+0000","last_change":"2023-12-26T21:53:04.265574+0000","last_active":"2023-12-26T21:53:05.272793+0000","last_peered":"2023-12-26T21:53:05.272793+0000","last_clean":"2023-12-26T21:53:05.272793+0000","last_became_active":"2023-12-26T21:53:04.265312+0000","last_became_peered":"2023-12-26T21:53:04.265312+0000","last_unstale":"2023-12-26T21:53:05.272793+0000","last_undegraded":"2023-12-26T21:53:05.272793+0000","last_fullsized":"2023-12-26T21:53:05.272793+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:47:44.760773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287420+0000","last_change":"2023-12-26T21:53:05.280022+0000","last_active":"2023-12-26T21:53:05.287420+0000","last_peered":"2023-12-26T21:53:05.287420+0000","last_clean":"2023-12-26T21:53:05.287420+0000","last_became_active":"2023-12-26T21:53:05.279769+0000","last_became_peered":"2023-12-26T21:53:05.279769+0000","last_unstale":"2023-12-26T21:53:05.287420+0000","last_undegraded":"2023-12-26T21:53:05.287420+0000","last_fullsized":"2023-12-26T21:53:05.287420+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:44:30.740351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.274000+0000","last_change":"2023-12-26T21:53:04.267879+0000","last_active":"2023-12-26T21:53:05.274000+0000","last_peered":"2023-12-26T21:53:05.274000+0000","last_clean":"2023-12-26T21:53:05.274000+0000","last_became_active":"2023-12-26T21:53:04.267583+0000","last_became_peered":"2023-12-26T21:53:04.267583+0000","last_unstale":"2023-12-26T21:53:05.274000+0000","last_undegraded":"2023-12-26T21:53:05.274000+0000","last_fullsized":"2023-12-26T21:53:05.274000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:02:20.185873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.292438+0000","last_change":"2023-12-26T21:53:05.283335+0000","last_active":"2023-12-26T21:53:05.292438+0000","last_peered":"2023-12-26T21:53:05.292438+0000","last_clean":"2023-12-26T21:53:05.292438+0000","last_became_active":"2023-12-26T21:53:05.283025+0000","last_became_peered":"2023-12-26T21:53:05.283025+0000","last_unstale":"2023-12-26T21:53:05.292438+0000","last_undegraded":"2023-12-26T21:53:05.292438+0000","last_fullsized":"2023-12-26T21:53:05.292438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:12.309723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.271990+0000","last_change":"2023-12-26T21:53:04.268290+0000","last_active":"2023-12-26T21:53:05.271990+0000","last_peered":"2023-12-26T21:53:05.271990+0000","last_clean":"2023-12-26T21:53:05.271990+0000","last_became_active":"2023-12-26T21:53:04.267992+0000","last_became_peered":"2023-12-26T21:53:04.267992+0000","last_unstale":"2023-12-26T21:53:05.271990+0000","last_undegraded":"2023-12-26T21:53:05.271990+0000","last_fullsized":"2023-12-26T21:53:05.271990+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:07.445143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.294020+0000","last_change":"2023-12-26T21:53:05.288258+0000","last_active":"2023-12-26T21:53:05.294020+0000","last_peered":"2023-12-26T21:53:05.294020+0000","last_clean":"2023-12-26T21:53:05.294020+0000","last_became_active":"2023-12-26T21:53:05.286668+0000","last_became_peered":"2023-12-26T21:53:05.286668+0000","last_unstale":"2023-12-26T21:53:05.294020+0000","last_undegraded":"2023-12-26T21:53:05.294020+0000","last_fullsized":"2023-12-26T21:53:05.294020+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:30:55.484438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.268491+0000","last_change":"2023-12-26T21:53:04.265372+0000","last_active":"2023-12-26T21:53:05.268491+0000","last_peered":"2023-12-26T21:53:05.268491+0000","last_clean":"2023-12-26T21:53:05.268491+0000","last_became_active":"2023-12-26T21:53:04.264345+0000","last_became_peered":"2023-12-26T21:53:04.264345+0000","last_unstale":"2023-12-26T21:53:05.268491+0000","last_undegraded":"2023-12-26T21:53:05.268491+0000","last_fullsized":"2023-12-26T21:53:05.268491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:37:32.016433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.288429+0000","last_change":"2023-12-26T21:53:05.285193+0000","last_active":"2023-12-26T21:53:05.288429+0000","last_peered":"2023-12-26T21:53:05.288429+0000","last_clean":"2023-12-26T21:53:05.288429+0000","last_became_active":"2023-12-26T21:53:05.284623+0000","last_became_peered":"2023-12-26T21:53:05.284623+0000","last_unstale":"2023-12-26T21:53:05.288429+0000","last_undegraded":"2023-12-26T21:53:05.288429+0000","last_fullsized":"2023-12-26T21:53:05.288429+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:07:56.200815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.269358+0000","last_change":"2023-12-26T21:53:04.267467+0000","last_active":"2023-12-26T21:53:05.269358+0000","last_peered":"2023-12-26T21:53:05.269358+0000","last_clean":"2023-12-26T21:53:05.269358+0000","last_became_active":"2023-12-26T21:53:04.267199+0000","last_became_peered":"2023-12-26T21:53:04.267199+0000","last_unstale":"2023-12-26T21:53:05.269358+0000","last_undegraded":"2023-12-26T21:53:05.269358+0000","last_fullsized":"2023-12-26T21:53:05.269358+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:53.612712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.273847+0000","last_change":"2023-12-26T21:53:05.271778+0000","last_active":"2023-12-26T21:53:05.273847+0000","last_peered":"2023-12-26T21:53:05.273847+0000","last_clean":"2023-12-26T21:53:05.273847+0000","last_became_active":"2023-12-26T21:53:05.271352+0000","last_became_peered":"2023-12-26T21:53:05.271352+0000","last_unstale":"2023-12-26T21:53:05.273847+0000","last_undegraded":"2023-12-26T21:53:05.273847+0000","last_fullsized":"2023-12-26T21:53:05.273847+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:14:44.426079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.272518+0000","last_change":"2023-12-26T21:53:04.271557+0000","last_active":"2023-12-26T21:53:05.272518+0000","last_peered":"2023-12-26T21:53:05.272518+0000","last_clean":"2023-12-26T21:53:05.272518+0000","last_became_active":"2023-12-26T21:53:04.271300+0000","last_became_peered":"2023-12-26T21:53:04.271300+0000","last_unstale":"2023-12-26T21:53:05.272518+0000","last_undegraded":"2023-12-26T21:53:05.272518+0000","last_fullsized":"2023-12-26T21:53:05.272518+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:19.256644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.287808+0000","last_change":"2023-12-26T21:53:05.282558+0000","last_active":"2023-12-26T21:53:05.287808+0000","last_peered":"2023-12-26T21:53:05.287808+0000","last_clean":"2023-12-26T21:53:05.287808+0000","last_became_active":"2023-12-26T21:53:05.281302+0000","last_became_peered":"2023-12-26T21:53:05.281302+0000","last_unstale":"2023-12-26T21:53:05.287808+0000","last_undegraded":"2023-12-26T21:53:05.287808+0000","last_fullsized":"2023-12-26T21:53:05.287808+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:04.238368+0000","last_clean_scrub_stamp":"2023-12-26T21:53:04.238368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:29:24.182590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"95'2","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.297316+0000","last_change":"2023-12-26T21:53:04.264566+0000","last_active":"2023-12-26T21:53:05.297316+0000","last_peered":"2023-12-26T21:53:05.297316+0000","last_clean":"2023-12-26T21:53:05.297316+0000","last_became_active":"2023-12-26T21:53:04.264140+0000","last_became_peered":"2023-12-26T21:53:04.264140+0000","last_unstale":"2023-12-26T21:53:05.297316+0000","last_undegraded":"2023-12-26T21:53:05.297316+0000","last_fullsized":"2023-12-26T21:53:05.297316+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:03.232814+0000","last_clean_scrub_stamp":"2023-12-26T21:53:03.232814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:17:15.637087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'183","reported_seq":464,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-26T21:53:05.263934+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:05.263934+0000","last_peered":"2023-12-26T21:53:05.263934+0000","last_clean":"2023-12-26T21:53:05.263934+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:05.263934+0000","last_undegraded":"2023-12-26T21:53:05.263934+0000","last_fullsized":"2023-12-26T21:53:05.263934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542341,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298436,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073404,"statfs":{"total":96636764160,"available":96331165696,"internally_reserved":0,"allocated":1175552,"data_stored":563697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.47199999999999998,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.76900000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.69799999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74399999999999999,"5min":1.016,"15min":1.048},"last":0.61099999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.038,"15min":1.0509999999999999},"last":0.752},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.85699999999999998,"5min":0.99199999999999999,"15min":1.02},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.62,"5min":0.504,"15min":0.5},"max":{"1min":0.94299999999999995,"5min":1.0189999999999999,"15min":1.052},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.63400000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83499999999999996,"5min":1.036,"15min":1.036},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.747,"15min":0.746},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0129999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.81499999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.117,"15min":1.117},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.437},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.006},"last":0.66400000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509637,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306632,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065208,"statfs":{"total":96636764160,"available":96322772992,"internally_reserved":0,"allocated":1179648,"data_stored":563731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":1.0029999999999999,"15min":1.085},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.34799999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.3380000000000001},"last":0.63}]},{"osd":1,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0489999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":0.97599999999999998,"15min":1.0920000000000001},"last":0.69399999999999995}]},{"osd":2,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":1.288},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.876,"5min":0.92000000000000004,"15min":1.113},"last":0.64600000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.92000000000000004,"5min":1.026,"15min":1.569},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.97799999999999998,"15min":1.3839999999999999},"last":0.71799999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.62,"15min":0.62},"min":{"1min":0.42699999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.137},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.69599999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95099999999999996,"5min":1.0049999999999999,"15min":1.296},"last":0.74299999999999999}]},{"osd":7,"last update":"Tue Dec 26 21:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.437},"max":{"1min":0.89200000000000002,"5min":0.96899999999999997,"15min":1.51},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.91700000000000004,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.70499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509637,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307196,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064644,"statfs":{"total":96636764160,"available":96322195456,"internally_reserved":0,"allocated":1757184,"data_stored":1152529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.442}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.73099999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.625},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.69399999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.79200000000000004}]}]},{"osd":0,"up_from":15,"seq":64424509637,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302524,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069316,"statfs":{"total":96636764160,"available":96326979584,"internally_reserved":0,"allocated":1167360,"data_stored":562231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":1.056},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.33800000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47799999999999998,"15min":0.49399999999999999},"min":{"1min":0.315,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":1.006},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.84699999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80000000000000004,"5min":0.86699999999999999,"15min":1.1870000000000001},"last":0.47899999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.752},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.042,"5min":1.042,"15min":1.411},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.76800000000000002},"min":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.2070000000000001},"last":0.70699999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.754,"15min":0.79800000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.2569999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77100000000000002,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.452},"last":0.80600000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.214},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.012,"15min":1.3779999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46400000000000002},"max":{"1min":1.018,"5min":1.042,"15min":1.4359999999999999},"last":0.749},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.79200000000000004},"min":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":1.0029999999999999,"5min":1.0069999999999999,"15min":1.3240000000000001},"last":0.874}]}]},{"osd":2,"up_from":14,"seq":60129542342,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306620,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065220,"statfs":{"total":96636764160,"available":96322785280,"internally_reserved":0,"allocated":1167360,"data_stored":562715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45800000000000002,"15min":0.45900000000000002},"min":{"1min":0.33200000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":1.915},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.30399999999999999},"max":{"1min":0.63200000000000001,"5min":0.83899999999999997,"15min":1.7729999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.39600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":1.952},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47999999999999998},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.30199999999999999},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.85499999999999998},"last":0.39100000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.375},"max":{"1min":1,"5min":1,"15min":1},"last":0.496},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73899999999999999,"5min":0.85699999999999998,"15min":1.6899999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.54600000000000004,"5min":0.38300000000000001,"15min":0.34799999999999998},"max":{"1min":0.96999999999999997,"5min":0.98299999999999998,"15min":1.284},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.60099999999999998,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":1.0349999999999999,"15min":1.1759999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.502,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":1.0600000000000001,"15min":1.5029999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":0.94599999999999995,"5min":1.014,"15min":1.355},"last":0.85099999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.58799999999999997,"5min":0.52400000000000002,"15min":0.48399999999999999},"max":{"1min":0.93000000000000005,"5min":1.038,"15min":1.7350000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.40300000000000002},"max":{"1min":0.95999999999999996,"5min":1.0609999999999999,"15min":1.349},"last":0.67000000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.47799999999999998},"max":{"1min":0.92700000000000005,"5min":1.073,"15min":1.841},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.49099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0189999999999999,"15min":1.294},"last":0.751}]}]},{"osd":3,"up_from":15,"seq":64424509638,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310724,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061116,"statfs":{"total":96636764160,"available":96318582784,"internally_reserved":0,"allocated":1175552,"data_stored":562287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.317},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.42099999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.39900000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.498}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63100000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.61599999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542341,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306628,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065212,"statfs":{"total":96636764160,"available":96322777088,"internally_reserved":0,"allocated":1175552,"data_stored":562827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97899999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.48499999999999999,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":1.008},"last":0.88}]},{"osd":1,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.95199999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.84799999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88900000000000001,"5min":0.97199999999999998,"15min":0.97299999999999998},"last":0.86499999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.71499999999999997},"min":{"1min":0.504,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.93000000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.438,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.747},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.91100000000000003}]}]},{"osd":5,"up_from":15,"seq":64424509636,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303104,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068736,"statfs":{"total":96636764160,"available":96326385664,"internally_reserved":0,"allocated":1761280,"data_stored":1152619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.878,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.71299999999999997},"min":{"1min":0.48899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.71},"last":0.55300000000000005}]},{"osd":1,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.57399999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96399999999999997,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68200000000000005,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90800000000000003,"5min":1.002,"15min":1.4219999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.871,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68799999999999994,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":1.214},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.88300000000000001,"5min":1.052,"15min":1.4890000000000001},"last":0.752},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":0.92200000000000004,"15min":1.1439999999999999},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84399999999999997,"5min":1.0089999999999999,"15min":2.5070000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.91300000000000003,"15min":2.8039999999999998},"last":0.71299999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.92400000000000004,"15min":1.637},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.746},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.042,"15min":2.5579999999999998},"last":0.85499999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":1.248},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":0.95199999999999996,"15min":1.5980000000000001},"last":0.72799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:14.957 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T21:53:14.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T21:53:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.188+0000 7fc9831ff700 1 Processor -- start 2023-12-26T21:53:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.188+0000 7fc9831ff700 1 -- start start 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc9831ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc97c157050 0x7fc97c0686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc9831ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc97c068c00 0x7fc97c0cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc9831ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c0caed0 con 0x7fc97c157420 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc9831ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0c9520 con 0x7fc97c157050 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc9831ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0ca8e0 con 0x7fc97c068c00 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc980f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc97c157050 0x7fc97c0686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc980f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc97c157050 0x7fc97c0686c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53828/0 (socket says 172.21.15.131:53828) 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc980f9b700 1 -- 172.21.15.131:0/4042004860 learned_addr learned my addr 172.21.15.131:0/4042004860 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc97bfff700 1 --1- 172.21.15.131:0/4042004860 >> v1:172.21.15.131:6789/0 conn(0x7fc97c157420 0x7fc97c159c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:57128/0 (socket says 172.21.15.131:57128) 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.189+0000 7fc98179c700 1 --2- 172.21.15.131:0/4042004860 >> v2:172.21.15.190:3300/0 conn(0x7fc97c068c00 0x7fc97c0cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc980f9b700 1 -- 172.21.15.131:0/4042004860 >> v2:172.21.15.190:3300/0 conn(0x7fc97c068c00 msgr2=0x7fc97c0cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc980f9b700 1 --2- 172.21.15.131:0/4042004860 >> v2:172.21.15.190:3300/0 conn(0x7fc97c068c00 0x7fc97c0cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc980f9b700 1 -- 172.21.15.131:0/4042004860 >> v1:172.21.15.131:6789/0 conn(0x7fc97c157420 legacy=0x7fc97c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc980f9b700 1 -- 172.21.15.131:0/4042004860 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c15a6c0 con 0x7fc97c157050 2023-12-26T21:53:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc98179c700 1 --2- 172.21.15.131:0/4042004860 >> v2:172.21.15.190:3300/0 conn(0x7fc97c068c00 0x7fc97c0cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc97b7fe700 1 -- 172.21.15.131:0/4042004860 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307853915 0 0) 0x7fc97c0caed0 con 0x7fc97c157420 2023-12-26T21:53:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc980f9b700 1 --2- 172.21.15.131:0/4042004860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c157050 0x7fc97c0686c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc97c069bb0 tx=0x7fc970009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4a9ca245b8de95 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc97b7fe700 1 -- 172.21.15.131:0/4042004860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97000b860 con 0x7fc97c157050 2023-12-26T21:53:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc97b7fe700 1 -- 172.21.15.131:0/4042004860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97000b9c0 con 0x7fc97c157050 2023-12-26T21:53:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.190+0000 7fc97b7fe700 1 -- 172.21.15.131:0/4042004860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97000bd60 con 0x7fc97c157050 2023-12-26T21:53:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.191+0000 7fc9831ff700 1 -- 172.21.15.131:0/4042004860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c157050 msgr2=0x7fc97c0686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.191+0000 7fc9831ff700 1 --2- 172.21.15.131:0/4042004860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c157050 0x7fc97c0686c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc97c069bb0 tx=0x7fc970009250 comp rx=0 tx=0).stop 2023-12-26T21:53:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.191+0000 7fc9831ff700 1 -- 172.21.15.131:0/4042004860 shutdown_connections 2023-12-26T21:53:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.191+0000 7fc9831ff700 1 --2- 172.21.15.131:0/4042004860 >> v2:172.21.15.190:3300/0 conn(0x7fc97c068c00 0x7fc97c0cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.191+0000 7fc9831ff700 1 --2- 172.21.15.131:0/4042004860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c157050 0x7fc97c0686c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.191+0000 7fc9831ff700 1 -- 172.21.15.131:0/4042004860 >> 172.21.15.131:0/4042004860 conn(0x7fc97c06dc80 msgr2=0x7fc97c0675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.191+0000 7fc9831ff700 1 -- 172.21.15.131:0/4042004860 shutdown_connections 2023-12-26T21:53:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.191+0000 7fc9831ff700 1 -- 172.21.15.131:0/4042004860 wait complete. 2023-12-26T21:53:15.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.193+0000 7fc9831ff700 1 Processor -- start 2023-12-26T21:53:15.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.193+0000 7fc9831ff700 1 -- start start 2023-12-26T21:53:15.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.194+0000 7fc9831ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c068c00 0x7fc97c0c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.194+0000 7fc9831ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c157050 0x7fc97c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.194+0000 7fc9831ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c157420 0x7fc97c158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc9831ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0b29d0 con 0x7fc97c068c00 2023-12-26T21:53:15.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc9831ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0c4c50 con 0x7fc97c157050 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc9831ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0b1dc0 con 0x7fc97c157420 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc980f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c068c00 0x7fc97c0c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc980f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c068c00 0x7fc97c0c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53838/0 (socket says 172.21.15.131:53838) 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc980f9b700 1 -- 172.21.15.131:0/2716465706 learned_addr learned my addr 172.21.15.131:0/2716465706 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc97bfff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c157050 0x7fc97c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc98179c700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c157420 0x7fc97c158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc980f9b700 1 -- 172.21.15.131:0/2716465706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c157420 msgr2=0x7fc97c158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc980f9b700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c157420 0x7fc97c158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc980f9b700 1 -- 172.21.15.131:0/2716465706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c157050 msgr2=0x7fc97c155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc980f9b700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c157050 0x7fc97c155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc980f9b700 1 -- 172.21.15.131:0/2716465706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9700094f0 con 0x7fc97c068c00 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc97bfff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c157050 0x7fc97c155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.195+0000 7fc980f9b700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c068c00 0x7fc97c0c4710 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc970009290 tx=0x7fc970017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.196+0000 7fc978ff9700 1 -- 172.21.15.131:0/2716465706 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97000bcd0 con 0x7fc97c068c00 2023-12-26T21:53:15.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.196+0000 7fc978ff9700 1 -- 172.21.15.131:0/2716465706 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97001a660 con 0x7fc97c068c00 2023-12-26T21:53:15.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.196+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c0b20a0 con 0x7fc97c068c00 2023-12-26T21:53:15.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.196+0000 7fc978ff9700 1 -- 172.21.15.131:0/2716465706 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97001aa20 con 0x7fc97c068c00 2023-12-26T21:53:15.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.196+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c0b2660 con 0x7fc97c068c00 2023-12-26T21:53:15.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.197+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c0657e0 con 0x7fc97c068c00 2023-12-26T21:53:15.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.198+0000 7fc978ff9700 1 -- 172.21.15.131:0/2716465706 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc97001b430 con 0x7fc97c068c00 2023-12-26T21:53:15.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.198+0000 7fc978ff9700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc960066f30 0x7fc9600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.198+0000 7fc978ff9700 1 -- 172.21.15.131:0/2716465706 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fc970085ea0 con 0x7fc97c068c00 2023-12-26T21:53:15.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.199+0000 7fc97bfff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc960066f30 0x7fc9600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.199+0000 7fc97bfff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc960066f30 0x7fc9600693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc97c06c040 tx=0x7fc968007c90 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:15.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.202+0000 7fc978ff9700 1 -- 172.21.15.131:0/2716465706 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc970052ee0 con 0x7fc97c068c00 2023-12-26T21:53:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.472+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc97c15a9a0 con 0x7fc97c068c00 2023-12-26T21:53:15.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.477+0000 7fc978ff9700 1 -- 172.21.15.131:0/2716465706 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc9700200e0 con 0x7fc97c068c00 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc960066f30 msgr2=0x7fc9600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc960066f30 0x7fc9600693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc97c06c040 tx=0x7fc968007c90 comp rx=0 tx=0).stop 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c068c00 msgr2=0x7fc97c0c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c068c00 0x7fc97c0c4710 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc970009290 tx=0x7fc970017000 comp rx=0 tx=0).stop 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 shutdown_connections 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c157420 0x7fc97c158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc960066f30 0x7fc9600693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97c068c00 0x7fc97c0c4710 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 --2- 172.21.15.131:0/2716465706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c157050 0x7fc97c155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 >> 172.21.15.131:0/2716465706 conn(0x7fc97c06dc80 msgr2=0x7fc97c15ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 shutdown_connections 2023-12-26T21:53:15.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.479+0000 7fc9831ff700 1 -- 172.21.15.131:0/2716465706 wait complete. 2023-12-26T21:53:15.481 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T21:53:15.498 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:53:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.687+0000 7f7dd19ff700 1 Processor -- start 2023-12-26T21:53:15.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.687+0000 7f7dd19ff700 1 -- start start 2023-12-26T21:53:15.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.688+0000 7f7dd19ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7dcc13c510 0x7f7dcc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.688+0000 7f7dd19ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dcc0686c0 con 0x7f7dcc0cc810 2023-12-26T21:53:15.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.688+0000 7f7dd19ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dcc069520 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.688+0000 7f7dd19ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc068910 con 0x7f7dcc13c510 2023-12-26T21:53:15.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.689+0000 7f7dcaffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7dcc13c510 0x7f7dcc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.689+0000 7f7dcb7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7dcc0cc810 0x7f7dcc0699b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:57142/0 (socket says 172.21.15.131:57142) 2023-12-26T21:53:15.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.689+0000 7f7dcb7fe700 1 -- 172.21.15.131:0/224746316 learned_addr learned my addr 172.21.15.131:0/224746316 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.689+0000 7f7dc9ffb700 1 -- 172.21.15.131:0/224746316 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818101723 0 0) 0x7f7dcc0686c0 con 0x7f7dcc0cc810 2023-12-26T21:53:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.689+0000 7f7dc9ffb700 1 -- 172.21.15.131:0/224746316 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db4003570 con 0x7f7dcc0cc810 2023-12-26T21:53:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.689+0000 7f7dcaffd700 1 -- 172.21.15.131:0/224746316 >> v1:172.21.15.190:6789/0 conn(0x7f7dcc06b8c0 legacy=0x7f7dcc0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.689+0000 7f7dcaffd700 1 -- 172.21.15.131:0/224746316 >> v1:172.21.15.131:6789/0 conn(0x7f7dcc0cc810 legacy=0x7f7dcc0699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.689+0000 7f7dcaffd700 1 -- 172.21.15.131:0/224746316 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc068bf0 con 0x7f7dcc13c510 2023-12-26T21:53:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.689+0000 7f7dcaffd700 1 --2- 172.21.15.131:0/224746316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc13c510 0x7f7dcc0c8a20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0098d0 tx=0x7f7dbc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73f6d404d400d3e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dc9ffb700 1 -- 172.21.15.131:0/224746316 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc005c10 con 0x7f7dcc13c510 2023-12-26T21:53:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dc9ffb700 1 -- 172.21.15.131:0/224746316 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc005d70 con 0x7f7dcc13c510 2023-12-26T21:53:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dc9ffb700 1 -- 172.21.15.131:0/224746316 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc006280 con 0x7f7dcc13c510 2023-12-26T21:53:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dd19ff700 1 -- 172.21.15.131:0/224746316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc13c510 msgr2=0x7f7dcc0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dd19ff700 1 --2- 172.21.15.131:0/224746316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc13c510 0x7f7dcc0c8a20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0098d0 tx=0x7f7dbc009120 comp rx=0 tx=0).stop 2023-12-26T21:53:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dd19ff700 1 -- 172.21.15.131:0/224746316 shutdown_connections 2023-12-26T21:53:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dd19ff700 1 --2- 172.21.15.131:0/224746316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc13c510 0x7f7dcc0c8a20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dd19ff700 1 -- 172.21.15.131:0/224746316 >> 172.21.15.131:0/224746316 conn(0x7f7dcc0ce3e0 msgr2=0x7f7dcc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dd19ff700 1 -- 172.21.15.131:0/224746316 shutdown_connections 2023-12-26T21:53:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.690+0000 7f7dd19ff700 1 -- 172.21.15.131:0/224746316 wait complete. 2023-12-26T21:53:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.692+0000 7f7dd19ff700 1 Processor -- start 2023-12-26T21:53:15.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.692+0000 7f7dd19ff700 1 -- start start 2023-12-26T21:53:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dd19ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc06b8c0 0x7f7dcc13bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dd19ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc0cc810 0x7f7dcc0b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dd19ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc13c510 0x7f7dcc15be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dd19ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc0b1180 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dd19ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc0b1740 con 0x7f7dcc0cc810 2023-12-26T21:53:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dd19ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc13c320 con 0x7f7dcc13c510 2023-12-26T21:53:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dcaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc06b8c0 0x7f7dcc13bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dcaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc06b8c0 0x7f7dcc13bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53860/0 (socket says 172.21.15.131:53860) 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dcaffd700 1 -- 172.21.15.131:0/1530895686 learned_addr learned my addr 172.21.15.131:0/1530895686 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dca7fc700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc0cc810 0x7f7dcc0b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.693+0000 7f7dcb7fe700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc13c510 0x7f7dcc15be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7dcaffd700 1 -- 172.21.15.131:0/1530895686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc13c510 msgr2=0x7f7dcc15be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7dcaffd700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc13c510 0x7f7dcc15be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7dcaffd700 1 -- 172.21.15.131:0/1530895686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc0cc810 msgr2=0x7f7dcc0b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7dcaffd700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc0cc810 0x7f7dcc0b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7dcaffd700 1 -- 172.21.15.131:0/1530895686 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0093c0 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7dcaffd700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc06b8c0 0x7f7dcc13bde0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc06fe40 tx=0x7f7dbc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:15.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7db2ffd700 1 -- 172.21.15.131:0/1530895686 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc01d780 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7db2ffd700 1 -- 172.21.15.131:0/1530895686 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc01d900 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7db2ffd700 1 -- 172.21.15.131:0/1530895686 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc01dcc0 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc15c840 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.694+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc0b13d0 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.696 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.695+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc0657e0 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.696+0000 7f7db2ffd700 1 -- 172.21.15.131:0/1530895686 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7dbc01b690 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.697+0000 7f7db2ffd700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7db8066f30 0x7f7db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:15.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.697+0000 7f7db2ffd700 1 -- 172.21.15.131:0/1530895686 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7dbc086270 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.697+0000 7f7dca7fc700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7db8066f30 0x7f7db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:15.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.698+0000 7f7dca7fc700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7db8066f30 0x7f7db80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc15ea20 tx=0x7f7dc0007ec0 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:15.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.701+0000 7f7db2ffd700 1 -- 172.21.15.131:0/1530895686 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7dbc053350 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.937+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dcc068ed0 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.938+0000 7f7db2ffd700 1 -- 172.21.15.131:0/1530895686 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f7dbc056970 con 0x7f7dcc06b8c0 2023-12-26T21:53:15.939 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:15.939 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6454,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1671363867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1671363867},{"type":"v1","addr":"172.21.15.131:6834","nonce":1671363867}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3807864961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3807864961},{"type":"v1","addr":"172.21.15.190:6834","nonce":3807864961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/18678440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":18678440},{"type":"v1","addr":"172.21.15.131:6835","nonce":18678440}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:04.724845+0000","modified":"2023-12-26T21:53:08.284069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/168268053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":168268053},{"type":"v1","addr":"172.21.15.190:6835","nonce":168268053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7db8066f30 msgr2=0x7f7db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7db8066f30 0x7f7db80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc15ea20 tx=0x7f7dc0007ec0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc06b8c0 msgr2=0x7f7dcc13bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc06b8c0 0x7f7dcc13bde0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc06fe40 tx=0x7f7dbc016000 comp rx=0 tx=0).stop 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 shutdown_connections 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc13c510 0x7f7dcc15be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7db8066f30 0x7f7db80693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dcc06b8c0 0x7f7dcc13bde0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 --2- 172.21.15.131:0/1530895686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc0cc810 0x7f7dcc0b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 >> 172.21.15.131:0/1530895686 conn(0x7f7dcc0ce3e0 msgr2=0x7f7dcc1539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 shutdown_connections 2023-12-26T21:53:15.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:15.940+0000 7f7dd19ff700 1 -- 172.21.15.131:0/1530895686 wait complete. 2023-12-26T21:53:15.942 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 42 2023-12-26T21:53:16.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T21:53:04.724845+0000', 'modified': '2023-12-26T21:53:08.284069+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6708}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6708': {'gid': 6708, 'name': 'd', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/168268053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 168268053}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 168268053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T21:53:16.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T21:53:16.362 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-26T21:53:16.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.553+0000 7fc0313ff700 1 Processor -- start 2023-12-26T21:53:16.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.553+0000 7fc0313ff700 1 -- start start 2023-12-26T21:53:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.554+0000 7fc0313ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc02c1596b0 0x7fc02c067f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.554+0000 7fc0313ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c159a80 0x7fc02c0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.554+0000 7fc0313ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c15ac30 con 0x7fc02c0c7df0 2023-12-26T21:53:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.554+0000 7fc0313ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0c8270 con 0x7fc02c1596b0 2023-12-26T21:53:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.554+0000 7fc0313ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c068480 con 0x7fc02c159a80 2023-12-26T21:53:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc02b7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc02c0c7df0 0x7fc02c06a6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:57148/0 (socket says 172.21.15.131:57148) 2023-12-26T21:53:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc02b7fe700 1 -- 172.21.15.131:0/2052889366 learned_addr learned my addr 172.21.15.131:0/2052889366 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc02affd700 1 --2- 172.21.15.131:0/2052889366 >> v2:172.21.15.190:3300/0 conn(0x7fc02c1596b0 0x7fc02c067f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c159a80 0x7fc02c0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405163978 0 0) 0x7fc02c15ac30 con 0x7fc02c0c7df0 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc014003570 con 0x7fc02c0c7df0 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3722864748 0 0) 0x7fc014003570 con 0x7fc02c0c7df0 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c159a80 msgr2=0x7fc02c0c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c159a80 0x7fc02c0c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 >> v2:172.21.15.190:3300/0 conn(0x7fc02c1596b0 msgr2=0x7fc02c067f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 --2- 172.21.15.131:0/2052889366 >> v2:172.21.15.190:3300/0 conn(0x7fc02c1596b0 0x7fc02c067f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c15a020 con 0x7fc02c0c7df0 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fc0240017f0 con 0x7fc02c0c7df0 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 >> v1:172.21.15.131:6789/0 conn(0x7fc02c0c7df0 legacy=0x7fc02c06a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc014003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.555+0000 7fc029ffb700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc014004810 0x7fc014004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc029ffb700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c159a80 0x7fc0140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0c8270 con 0x7fc014003ae0 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c068480 con 0x7fc014004810 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01400bb10 con 0x7fc02c159a80 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02affd700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc014003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c159a80 0x7fc0140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc014004810 0x7fc014004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02affd700 1 -- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c159a80 msgr2=0x7fc0140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02affd700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c159a80 0x7fc0140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02affd700 1 -- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc014004810 msgr2=0x7fc014004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02affd700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc014004810 0x7fc014004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02affd700 1 -- 172.21.15.131:0/2052889366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014003570 con 0x7fc014003ae0 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc014004810 0x7fc014004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.556+0000 7fc02affd700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc014003ef0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009c00 tx=0x7fc01c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0f9b7bed15b2c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00e710 con 0x7fc014003ae0 2023-12-26T21:53:16.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c00e870 con 0x7fc014003ae0 2023-12-26T21:53:16.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc029ffb700 1 -- 172.21.15.131:0/2052889366 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00ec30 con 0x7fc014003ae0 2023-12-26T21:53:16.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc0313ff700 1 -- 172.21.15.131:0/2052889366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 msgr2=0x7fc014003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc0313ff700 1 --2- 172.21.15.131:0/2052889366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc014003ef0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009c00 tx=0x7fc01c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc02b7fe700 1 -- 172.21.15.131:0/2052889366 reap_dead start 2023-12-26T21:53:16.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc0313ff700 1 -- 172.21.15.131:0/2052889366 shutdown_connections 2023-12-26T21:53:16.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc0313ff700 1 -- 172.21.15.131:0/2052889366 >> 172.21.15.131:0/2052889366 conn(0x7fc02c0686c0 msgr2=0x7fc02c068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:16.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc0313ff700 1 -- 172.21.15.131:0/2052889366 shutdown_connections 2023-12-26T21:53:16.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.557+0000 7fc0313ff700 1 -- 172.21.15.131:0/2052889366 wait complete. 2023-12-26T21:53:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.559+0000 7fc0313ff700 1 Processor -- start 2023-12-26T21:53:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.559+0000 7fc0313ff700 1 -- start start 2023-12-26T21:53:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc0313ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc02c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc0313ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc02c0c66c0 0x7fc02c157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc0313ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c0c6b30 0x7fc02c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc0313ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c15e810 con 0x7fc014003ae0 2023-12-26T21:53:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc0313ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c15eaf0 con 0x7fc02c0c66c0 2023-12-26T21:53:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc0313ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c15edd0 con 0x7fc02c0c6b30 2023-12-26T21:53:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc02c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc02c15f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53876/0 (socket says 172.21.15.131:53876) 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02affd700 1 -- 172.21.15.131:0/3302722986 learned_addr learned my addr 172.21.15.131:0/3302722986 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc02c0c66c0 0x7fc02c157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c0c6b30 0x7fc02c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02affd700 1 -- 172.21.15.131:0/3302722986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c0c6b30 msgr2=0x7fc02c157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02affd700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c0c6b30 0x7fc02c157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02affd700 1 -- 172.21.15.131:0/3302722986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc02c0c66c0 msgr2=0x7fc02c157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02affd700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc02c0c66c0 0x7fc02c157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.560+0000 7fc02affd700 1 -- 172.21.15.131:0/3302722986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c15f0b0 con 0x7fc014003ae0 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.561+0000 7fc02affd700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc02c15f8b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc01400ba60 tx=0x7fc01c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.561+0000 7fc012ffd700 1 -- 172.21.15.131:0/3302722986 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c022070 con 0x7fc014003ae0 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.561+0000 7fc012ffd700 1 -- 172.21.15.131:0/3302722986 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c00e9e0 con 0x7fc014003ae0 2023-12-26T21:53:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.561+0000 7fc0313ff700 1 -- 172.21.15.131:0/3302722986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c15f390 con 0x7fc014003ae0 2023-12-26T21:53:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.561+0000 7fc012ffd700 1 -- 172.21.15.131:0/3302722986 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00e710 con 0x7fc014003ae0 2023-12-26T21:53:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.561+0000 7fc0313ff700 1 -- 172.21.15.131:0/3302722986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c15dae0 con 0x7fc014003ae0 2023-12-26T21:53:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.563+0000 7fc012ffd700 1 -- 172.21.15.131:0/3302722986 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc01c01c660 con 0x7fc014003ae0 2023-12-26T21:53:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.564+0000 7fc012ffd700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc018066f30 0x7fc0180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.564+0000 7fc012ffd700 1 -- 172.21.15.131:0/3302722986 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fc01c085da0 con 0x7fc014003ae0 2023-12-26T21:53:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.564+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc018066f30 0x7fc0180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.565+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc018066f30 0x7fc0180693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc02c15fdf0 tx=0x7fc020008040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.566+0000 7fc03fafcb80 1 Processor -- start 2023-12-26T21:53:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.566+0000 7fc03fafcb80 1 -- start start 2023-12-26T21:53:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 0x56304e8bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8e0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304e7f23e0 con 0x56304e8c0170 2023-12-26T21:53:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304e7efbe0 con 0x56304e8c3fa0 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56304e7eb9d0 con 0x56304e8c48b0 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8e0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8e0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53882/0 (socket says 172.21.15.131:53882) 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02b7fe700 1 -- 172.21.15.131:0/510714585 learned_addr learned my addr 172.21.15.131:0/510714585 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02affd700 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 0x56304e8bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02b7fe700 1 -- 172.21.15.131:0/510714585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 msgr2=0x56304e8bfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 0x56304e8bfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02b7fe700 1 -- 172.21.15.131:0/510714585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 msgr2=0x56304e8c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02b7fe700 1 -- 172.21.15.131:0/510714585 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56304e67f360 con 0x56304e8c0170 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.567+0000 7fc02affd700 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.568+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8e0360 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc024005c80 tx=0x7fc024001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c69aa3068a2b460 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.568+0000 7fc00bfff700 1 -- 172.21.15.131:0/510714585 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024002110 con 0x56304e8c0170 2023-12-26T21:53:16.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.568+0000 7fc00bfff700 1 -- 172.21.15.131:0/510714585 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024005e90 con 0x56304e8c0170 2023-12-26T21:53:16.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.568+0000 7fc00bfff700 1 -- 172.21.15.131:0/510714585 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024015680 con 0x56304e8c0170 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.568+0000 7fc03fafcb80 1 -- 172.21.15.131:0/510714585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 msgr2=0x56304e8e0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.568+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8e0360 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc024005c80 tx=0x7fc024001800 comp rx=0 tx=0).stop 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.568+0000 7fc03fafcb80 1 -- 172.21.15.131:0/510714585 shutdown_connections 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.568+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8e0360 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.568+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.569+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/510714585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 0x56304e8bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.569+0000 7fc03fafcb80 1 -- 172.21.15.131:0/510714585 >> 172.21.15.131:0/510714585 conn(0x56304e89a440 msgr2=0x56304e8e5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.569+0000 7fc03fafcb80 1 -- 172.21.15.131:0/510714585 shutdown_connections 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.569+0000 7fc03fafcb80 1 -- 172.21.15.131:0/510714585 wait complete. 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.569+0000 7fc03fafcb80 1 -- start start 2023-12-26T21:53:16.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.570+0000 7fc03fafcb80 1 Processor -- start 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.570+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.570+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.570+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 0x56304e8dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.570+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304e7981b0 con 0x56304e8c0170 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.570+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304e797e50 con 0x56304e8c3fa0 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.570+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56304e6e4e40 con 0x56304e8c48b0 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc03fafcb80 10 client.0 _ll_register_callbacks cb 0x56304e6e77b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53894/0 (socket says 172.21.15.131:53894) 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02affd700 1 -- 172.21.15.131:0/80352490 learned_addr learned my addr 172.21.15.131:0/80352490 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 0x56304e8dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02affd700 1 -- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 msgr2=0x56304e8dc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02affd700 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 0x56304e8dc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02affd700 1 -- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 msgr2=0x56304e8c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02affd700 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02affd700 1 -- 172.21.15.131:0/80352490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02400f040 con 0x56304e8c0170 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc02affd700 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8cb510 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009650 tx=0x7fc01c083ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba8a16ac95bdbbc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc0097fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.571+0000 7fc0097fa700 1 -- 172.21.15.131:0/80352490 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c059cf0 con 0x56304e8c0170 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc0097fa700 1 -- 172.21.15.131:0/80352490 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fc01c05da40 con 0x56304e8c0170 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc03fafcb80 1 -- 172.21.15.131:0/80352490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56304e86ceb0 con 0x56304e8c0170 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc0097fa700 1 -- 172.21.15.131:0/80352490 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc01c055f90 con 0x56304e8c0170 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc03fafcb80 10 client.6858 fetch_fsmap learned FSMap version 42 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc03fafcb80 1 -- 172.21.15.131:0/80352490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x56304e62bb10 con 0x56304e8c0170 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc0097fa700 1 -- 172.21.15.131:0/80352490 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56304e86ceb0 con 0x56304e8c0170 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc0097fa700 1 -- 172.21.15.131:0/80352490 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc01c05d5b0 con 0x56304e8c0170 2023-12-26T21:53:16.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc03fafcb80 10 client.6858 fetch_fsmap finished waiting for FSMap version 42 2023-12-26T21:53:16.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc03fafcb80 10 client.6858 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-26T21:53:16.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc03fafcb80 20 client.6858 populate_metadata read hostname 'smithi131' 2023-12-26T21:53:16.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] conn(0x56304e8d9f70 0x56304e8c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc03fafcb80 4 client.6858 mds_command: new command op to 6708 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T21:53:16.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.572+0000 7fc03fafcb80 1 -- 172.21.15.131:0/80352490 --> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56304e7c2880 con 0x56304e8d9f70 2023-12-26T21:53:16.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.573+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] conn(0x56304e8d9f70 0x56304e8c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.573+0000 7fc0097fa700 10 client.6858 ms_handle_connect on v2:172.21.15.190:6833/168268053 2023-12-26T21:53:16.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.573+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] conn(0x56304e8d9f70 0x56304e8c8fa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d54a86b2ca9cc71 server_cookie=cdc35e50f3c019f7 in_seq=0 out_seq=0 2023-12-26T21:53:16.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc0097fa700 1 -- 172.21.15.131:0/80352490 <== mds.0 v2:172.21.15.190:6833/168268053 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56304e7c2880 con 0x56304e8d9f70 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc0097fa700 10 client.6858 handle_command_reply: tid=0 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 client.6858 shutdown 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 -- 172.21.15.131:0/80352490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 msgr2=0x56304e8cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8cb510 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009650 tx=0x7fc01c083ff0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 -- 172.21.15.131:0/80352490 shutdown_connections 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] conn(0x56304e8d9f70 0x56304e8c8fa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c48b0 0x56304e8dc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c0170 0x56304e8cb510 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/80352490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc0097fa700 0 client.6858 ms_handle_reset on v2:172.21.15.190:6833/168268053 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 -- 172.21.15.131:0/80352490 >> 172.21.15.131:0/80352490 conn(0x56304e89a440 msgr2=0x56304e89a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.574+0000 7fc03fafcb80 1 -- 172.21.15.131:0/80352490 shutdown_connections 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.575+0000 7fc03fafcb80 1 -- 172.21.15.131:0/80352490 wait complete. 2023-12-26T21:53:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.575+0000 7fc03fafcb80 20 client.6858 trim_cache size 0 max 16384 2023-12-26T21:53:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.592+0000 7fc03fafcb80 1 Processor -- start 2023-12-26T21:53:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.592+0000 7fc03fafcb80 1 -- start start 2023-12-26T21:53:16.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.593+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 0x56304e89a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.593+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8bf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.593+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c48b0 0x56304e898510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.593+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304e89b910 con 0x56304e8c48b0 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.593+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304e8c39a0 con 0x56304e8c3fa0 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.593+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56304e8e6a30 con 0x56304e8c0170 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c48b0 0x56304e898510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c48b0 0x56304e898510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53898/0 (socket says 172.21.15.131:53898) 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02b7fe700 1 -- 172.21.15.131:0/1693970711 learned_addr learned my addr 172.21.15.131:0/1693970711 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02affd700 1 --2- 172.21.15.131:0/1693970711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 0x56304e89a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/1693970711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02b7fe700 1 -- 172.21.15.131:0/1693970711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 msgr2=0x56304e89a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/1693970711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 0x56304e89a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02b7fe700 1 -- 172.21.15.131:0/1693970711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 msgr2=0x56304e8bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/1693970711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02b7fe700 1 -- 172.21.15.131:0/1693970711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56304e7ba290 con 0x56304e8c48b0 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.594+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/1693970711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c48b0 0x56304e898510 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x56304e8e5d90 tx=0x7fc024019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5d9bbd2dc471dc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc010ff9700 1 -- 172.21.15.131:0/1693970711 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024005e90 con 0x56304e8c48b0 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc010ff9700 1 -- 172.21.15.131:0/1693970711 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024007ae0 con 0x56304e8c48b0 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc010ff9700 1 -- 172.21.15.131:0/1693970711 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240012d0 con 0x56304e8c48b0 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1693970711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c48b0 msgr2=0x56304e898510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1693970711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c48b0 0x56304e898510 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x56304e8e5d90 tx=0x7fc024019000 comp rx=0 tx=0).stop 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1693970711 shutdown_connections 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1693970711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c48b0 0x56304e898510 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1693970711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 0x56304e89a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1693970711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c3fa0 0x56304e8bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1693970711 >> 172.21.15.131:0/1693970711 conn(0x56304e89a440 msgr2=0x56304e891ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1693970711 shutdown_connections 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.595+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1693970711 wait complete. 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.596+0000 7fc03fafcb80 1 -- start start 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.596+0000 7fc03fafcb80 1 Processor -- start 2023-12-26T21:53:16.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 0x56304e8dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c3fa0 0x56304e89dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc03fafcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c48b0 0x56304e9471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304e8c5350 con 0x56304e8c3fa0 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304e947c40 con 0x56304e8c48b0 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc03fafcb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56304e8dc140 con 0x56304e8c0170 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc03fafcb80 10 client.0 _ll_register_callbacks cb 0x7fc01c056400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc02a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c3fa0 0x56304e89dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc02a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c3fa0 0x56304e89dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53908/0 (socket says 172.21.15.131:53908) 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.597+0000 7fc02a7fc700 1 -- 172.21.15.131:0/1255992882 learned_addr learned my addr 172.21.15.131:0/1255992882 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc02affd700 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 0x56304e8dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c48b0 0x56304e9471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc02a7fc700 1 -- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 msgr2=0x56304e8dbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 0x56304e8dbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc02a7fc700 1 -- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c48b0 msgr2=0x56304e9471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c48b0 0x56304e9471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc02a7fc700 1 -- 172.21.15.131:0/1255992882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56304e67f360 con 0x56304e8c3fa0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc02b7fe700 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c48b0 0x56304e9471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc02a7fc700 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c3fa0 0x56304e89dab0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc020009840 tx=0x7fc02001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f942781b13582d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc0097fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc0097fa700 1 -- 172.21.15.131:0/1255992882 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020025e50 con 0x56304e8c3fa0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc0097fa700 1 -- 172.21.15.131:0/1255992882 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fc02002a070 con 0x56304e8c3fa0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.598+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1255992882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc02c15bea0 con 0x56304e8c3fa0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc0097fa700 1 -- 172.21.15.131:0/1255992882 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc020016640 con 0x56304e8c3fa0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc03fafcb80 10 client.6864 fetch_fsmap learned FSMap version 42 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1255992882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x56304e683c80 con 0x56304e8c3fa0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc0097fa700 1 -- 172.21.15.131:0/1255992882 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c15bea0 con 0x56304e8c3fa0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc0097fa700 1 -- 172.21.15.131:0/1255992882 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 42) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc0200167a0 con 0x56304e8c3fa0 2023-12-26T21:53:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc03fafcb80 10 client.6864 fetch_fsmap finished waiting for FSMap version 42 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc03fafcb80 10 client.6864 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc03fafcb80 20 client.6864 populate_metadata read hostname 'smithi131' 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] conn(0x56304e93d820 0x56304e93dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc03fafcb80 4 client.6864 mds_command: new command op to 6708 tid=0[{"prefix": "status"}] 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1255992882 --> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56304e890490 con 0x56304e93d820 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.599+0000 7fc02affd700 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] conn(0x56304e93d820 0x56304e93dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc0097fa700 10 client.6864 ms_handle_connect on v2:172.21.15.190:6833/168268053 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc02affd700 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] conn(0x56304e93d820 0x56304e93dbf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d67eb1f6d5cde7c server_cookie=78f088d08b8c0211 in_seq=0 out_seq=0 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc0097fa700 1 -- 172.21.15.131:0/1255992882 <== mds.0 v2:172.21.15.190:6833/168268053 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x56304e890490 con 0x56304e93d820 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc0097fa700 10 client.6864 handle_command_reply: tid=0 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc03fafcb80 1 client.6864 shutdown 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1255992882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c3fa0 msgr2=0x56304e89dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c3fa0 0x56304e89dab0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc020009840 tx=0x7fc02001b000 comp rx=0 tx=0).stop 2023-12-26T21:53:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1255992882 shutdown_connections 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:6833/168268053,v1:172.21.15.190:6835/168268053] conn(0x56304e93d820 0x56304e93dbf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56304e8c48b0 0x56304e9471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56304e8c0170 0x56304e8dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc0097fa700 0 client.6864 ms_handle_reset on v2:172.21.15.190:6833/168268053 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc03fafcb80 1 --2- 172.21.15.131:0/1255992882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56304e8c3fa0 0x56304e89dab0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.600+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1255992882 >> 172.21.15.131:0/1255992882 conn(0x56304e89a440 msgr2=0x56304e89be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1255992882 shutdown_connections 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc03fafcb80 1 -- 172.21.15.131:0/1255992882 wait complete. 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc03fafcb80 20 client.6864 trim_cache size 0 max 16384 2023-12-26T21:53:16.607 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6708, 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 11.330702192, 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 42, 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 96, 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 96, 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 17.078943014 2023-12-26T21:53:16.608 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:53:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 -- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc018066f30 msgr2=0x7fc0180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc018066f30 0x7fc0180693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc02c15fdf0 tx=0x7fc020008040 comp rx=0 tx=0).stop 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 -- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 msgr2=0x7fc02c15f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc02c15f8b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc01400ba60 tx=0x7fc01c01a000 comp rx=0 tx=0).stop 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 -- 172.21.15.131:0/3302722986 shutdown_connections 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc02c0c6b30 0x7fc02c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc018066f30 0x7fc0180693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc014003ae0 0x7fc02c15f8b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 --2- 172.21.15.131:0/3302722986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc02c0c66c0 0x7fc02c157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.601+0000 7fc0313ff700 1 -- 172.21.15.131:0/3302722986 >> 172.21.15.131:0/3302722986 conn(0x7fc02c0686c0 msgr2=0x7fc02c06f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.602+0000 7fc0313ff700 1 -- 172.21.15.131:0/3302722986 shutdown_connections 2023-12-26T21:53:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.602+0000 7fc0313ff700 1 -- 172.21.15.131:0/3302722986 wait complete. 2023-12-26T21:53:16.619 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:53:16.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.809+0000 7f27df9ff700 1 Processor -- start 2023-12-26T21:53:16.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.809+0000 7f27df9ff700 1 -- start start 2023-12-26T21:53:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27df9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d80cc810 0x7f27d80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27df9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d80686c0 con 0x7f27d813c510 2023-12-26T21:53:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27df9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d8069520 con 0x7f27d806bfd0 2023-12-26T21:53:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27df9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d8068910 con 0x7f27d80cc810 2023-12-26T21:53:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27dd79b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f27d813c510 0x7f27d806b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:57160/0 (socket says 172.21.15.131:57160) 2023-12-26T21:53:16.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27dd79b700 1 -- 172.21.15.131:0/3039967041 learned_addr learned my addr 172.21.15.131:0/3039967041 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:16.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27ddf9c700 1 --2- 172.21.15.131:0/3039967041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d80cc810 0x7f27d80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27cffff700 1 -- 172.21.15.131:0/3039967041 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319092248 0 0) 0x7f27d80686c0 con 0x7f27d813c510 2023-12-26T21:53:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27cffff700 1 -- 172.21.15.131:0/3039967041 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c4003570 con 0x7f27d813c510 2023-12-26T21:53:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27ddf9c700 1 -- 172.21.15.131:0/3039967041 >> v1:172.21.15.190:6789/0 conn(0x7f27d806bfd0 legacy=0x7f27d80cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27cffff700 1 -- 172.21.15.131:0/3039967041 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918579581 0 0) 0x7f27d8069520 con 0x7f27d806bfd0 2023-12-26T21:53:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27ddf9c700 1 -- 172.21.15.131:0/3039967041 >> v1:172.21.15.131:6789/0 conn(0x7f27d813c510 legacy=0x7f27d806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.811+0000 7f27ddf9c700 1 -- 172.21.15.131:0/3039967041 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8068bf0 con 0x7f27d80cc810 2023-12-26T21:53:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.812+0000 7f27ddf9c700 1 --2- 172.21.15.131:0/3039967041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d80cc810 0x7f27d80c8a20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009e90 tx=0x7f27d0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c684190efb0754b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.812+0000 7f27cffff700 1 -- 172.21.15.131:0/3039967041 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d000e070 con 0x7f27d80cc810 2023-12-26T21:53:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.812+0000 7f27cffff700 1 -- 172.21.15.131:0/3039967041 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d000bc30 con 0x7f27d80cc810 2023-12-26T21:53:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.812+0000 7f27cffff700 1 -- 172.21.15.131:0/3039967041 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0012430 con 0x7f27d80cc810 2023-12-26T21:53:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.813+0000 7f27df9ff700 1 -- 172.21.15.131:0/3039967041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d80cc810 msgr2=0x7f27d80c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.813+0000 7f27df9ff700 1 --2- 172.21.15.131:0/3039967041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d80cc810 0x7f27d80c8a20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009e90 tx=0x7f27d0009780 comp rx=0 tx=0).stop 2023-12-26T21:53:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.813+0000 7f27df9ff700 1 -- 172.21.15.131:0/3039967041 shutdown_connections 2023-12-26T21:53:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.813+0000 7f27df9ff700 1 --2- 172.21.15.131:0/3039967041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d80cc810 0x7f27d80c8a20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.813+0000 7f27df9ff700 1 -- 172.21.15.131:0/3039967041 >> 172.21.15.131:0/3039967041 conn(0x7f27d80ce3e0 msgr2=0x7f27d80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.813+0000 7f27df9ff700 1 -- 172.21.15.131:0/3039967041 shutdown_connections 2023-12-26T21:53:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.813+0000 7f27df9ff700 1 -- 172.21.15.131:0/3039967041 wait complete. 2023-12-26T21:53:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.814+0000 7f27df9ff700 1 Processor -- start 2023-12-26T21:53:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.814+0000 7f27df9ff700 1 -- start start 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.815+0000 7f27df9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d806bfd0 0x7f27d80b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.815+0000 7f27df9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27d80cc810 0x7f27d80c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.815+0000 7f27df9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d813c510 0x7f27d813a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.815+0000 7f27df9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d80b2b00 con 0x7f27d80cc810 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.815+0000 7f27df9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d80c5650 con 0x7f27d806bfd0 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.815+0000 7f27df9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d80c82d0 con 0x7f27d813c510 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.815+0000 7f27dcf9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27d80cc810 0x7f27d80c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27dcf9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27d80cc810 0x7f27d80c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53924/0 (socket says 172.21.15.131:53924) 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27dcf9a700 1 -- 172.21.15.131:0/2550887445 learned_addr learned my addr 172.21.15.131:0/2550887445 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27dd79b700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d806bfd0 0x7f27d80b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27ddf9c700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d813c510 0x7f27d813a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27dcf9a700 1 -- 172.21.15.131:0/2550887445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d813c510 msgr2=0x7f27d813a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27dcf9a700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d813c510 0x7f27d813a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27dcf9a700 1 -- 172.21.15.131:0/2550887445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d806bfd0 msgr2=0x7f27d80b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27dcf9a700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d806bfd0 0x7f27d80b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27dcf9a700 1 -- 172.21.15.131:0/2550887445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d0009a20 con 0x7f27d80cc810 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27dcf9a700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27d80cc810 0x7f27d80c7250 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f27c800c510 tx=0x7f27c8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27cd7fa700 1 -- 172.21.15.131:0/2550887445 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8011590 con 0x7f27d80cc810 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27cd7fa700 1 -- 172.21.15.131:0/2550887445 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c8011710 con 0x7f27d80cc810 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d80c7790 con 0x7f27d80cc810 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.816+0000 7f27cd7fa700 1 -- 172.21.15.131:0/2550887445 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8011ad0 con 0x7f27d80cc810 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.817+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d80c7d10 con 0x7f27d80cc810 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.818+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d80c57c0 con 0x7f27d80cc810 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.818+0000 7f27cd7fa700 1 -- 172.21.15.131:0/2550887445 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f27c801b3b0 con 0x7f27d80cc810 2023-12-26T21:53:16.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.819+0000 7f27cd7fa700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f27bc066f30 0x7f27bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:16.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.819+0000 7f27cd7fa700 1 -- 172.21.15.131:0/2550887445 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f27c8083cb0 con 0x7f27d80cc810 2023-12-26T21:53:16.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.820+0000 7f27dd79b700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f27bc066f30 0x7f27bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:16.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.820+0000 7f27dd79b700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f27bc066f30 0x7f27bc0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27d4000c00 tx=0x7f27d4009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:16.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:16.823+0000 7f27cd7fa700 1 -- 172.21.15.131:0/2550887445 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27d80c57c0 con 0x7f27d80cc810 2023-12-26T21:53:17.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.059+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f27d80c57c0 con 0x7f27d80cc810 2023-12-26T21:53:17.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.059+0000 7f27cd7fa700 1 -- 172.21.15.131:0/2550887445 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7f27d80c57c0 con 0x7f27d80cc810 2023-12-26T21:53:17.060 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:53:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f27bc066f30 msgr2=0x7f27bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f27bc066f30 0x7f27bc0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27d4000c00 tx=0x7f27d4009040 comp rx=0 tx=0).stop 2023-12-26T21:53:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27d80cc810 msgr2=0x7f27d80c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27d80cc810 0x7f27d80c7250 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f27c800c510 tx=0x7f27c8007f50 comp rx=0 tx=0).stop 2023-12-26T21:53:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 shutdown_connections 2023-12-26T21:53:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27d813c510 0x7f27d813a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:17.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27d806bfd0 0x7f27d80b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f27bc066f30 0x7f27bc0693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 --2- 172.21.15.131:0/2550887445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27d80cc810 0x7f27d80c7250 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 >> 172.21.15.131:0/2550887445 conn(0x7f27d80ce3e0 msgr2=0x7f27d813afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 shutdown_connections 2023-12-26T21:53:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:17.061+0000 7f27df9ff700 1 -- 172.21.15.131:0/2550887445 wait complete. 2023-12-26T21:53:17.080 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T21:53:17.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:53:17.081 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T21:53:17.081 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T21:53:17.081 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T21:53:17.081 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T21:53:17.081 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T21:53:17.081 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:53:17.082 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:53:17.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:17.082 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:17.127 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41459sec preferred_lft 41459sec 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:17.128 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:53:17.129 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T21:53:17.129 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T21:53:17.129 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:17.129 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:53:17.129 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T21:53:17.129 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T21:53:17.129 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:17.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:17.129 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:53:17.233 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T21:53:17.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:17.237 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T21:53:17.282 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T21:53:17.282 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:53:17.282 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:53:17.282 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:53:17.282 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:53:17.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:17.284 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:53:17.284 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:53:17.284 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:53:17.284 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:53:17.284 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:53:17.285 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:53:17.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:17.494 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:53:17.537 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T21:53:17.537 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T21:53:17.538 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:53:17.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:17.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T21:53:17.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:53:17.648 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T21:53:17.691 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:53:17.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:53:17.692 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:53:17.775 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T21:53:17.775 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:53:17.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:53:17.914 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:53:17.914 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:53:17.942 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:53:17.942 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:53:17.942 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:53:17.942 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:53:17.942 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:53:17.947 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:53:17.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:17.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:53:18.016 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T21:53:18.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.207+0000 7f79345ff700 1 Processor -- start 2023-12-26T21:53:18.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.207+0000 7f79345ff700 1 -- start start 2023-12-26T21:53:18.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.208+0000 7f79345ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c159900 0x7f792c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.208+0000 7f79345ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f792c0cd4d0 0x7f792c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.208+0000 7f79345ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792c15db90 con 0x7f792c0cd100 2023-12-26T21:53:18.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.208+0000 7f79345ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792c0cbe50 con 0x7f792c0cd4d0 2023-12-26T21:53:18.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.208+0000 7f79345ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c15d330 con 0x7f792c159900 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.208+0000 7f793239b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f792c0cd100 0x7f792c1591f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:57172/0 (socket says 172.21.15.131:57172) 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.208+0000 7f793239b700 1 -- 172.21.15.131:0/2486146844 learned_addr learned my addr 172.21.15.131:0/2486146844 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931b9a700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c159900 0x7f792c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7932b9c700 1 --2- 172.21.15.131:0/2486146844 >> v2:172.21.15.190:3300/0 conn(0x7f792c0cd4d0 0x7f792c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104022893 0 0) 0x7f792c15db90 con 0x7f792c0cd100 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7918003570 con 0x7f792c0cd100 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1796785316 0 0) 0x7f7918003570 con 0x7f792c0cd100 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c159900 msgr2=0x7f792c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c159900 0x7f792c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 >> v2:172.21.15.190:3300/0 conn(0x7f792c0cd4d0 msgr2=0x7f792c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 --2- 172.21.15.131:0/2486146844 >> v2:172.21.15.190:3300/0 conn(0x7f792c0cd4d0 0x7f792c0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c0cbb70 con 0x7f792c0cd100 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f7928001470 con 0x7f792c0cd100 2023-12-26T21:53:18.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 >> v1:172.21.15.131:6789/0 conn(0x7f792c0cd100 legacy=0x7f792c1591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c159900 0x7f7918003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7918003fe0 0x7f7918004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f79180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0cbe50 con 0x7f7918004d50 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c15d330 con 0x7f7918003fe0 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.209+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791800ba90 con 0x7f792c159900 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7932b9c700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f79180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f793239b700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c159900 0x7f7918003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7931b9a700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7918003fe0 0x7f7918004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7932b9c700 1 -- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c159900 msgr2=0x7f7918003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7932b9c700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c159900 0x7f7918003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7932b9c700 1 -- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7918003fe0 msgr2=0x7f7918004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:18.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7932b9c700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7918003fe0 0x7f7918004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7932b9c700 1 -- 172.21.15.131:0/2486146844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7918003570 con 0x7f7918004d50 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7932b9c700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f79180051c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f792400e210 tx=0x7f792400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dc63ee20d8ff28 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792400ea60 con 0x7f7918004d50 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792400f040 con 0x7f7918004d50 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.210+0000 7f7931399700 1 -- 172.21.15.131:0/2486146844 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792400ee10 con 0x7f7918004d50 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.211+0000 7f79345ff700 1 -- 172.21.15.131:0/2486146844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 msgr2=0x7f79180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.211+0000 7f79345ff700 1 --2- 172.21.15.131:0/2486146844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f79180051c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f792400e210 tx=0x7f792400b040 comp rx=0 tx=0).stop 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.211+0000 7f7932b9c700 1 -- 172.21.15.131:0/2486146844 reap_dead start 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.211+0000 7f79345ff700 1 -- 172.21.15.131:0/2486146844 shutdown_connections 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.211+0000 7f79345ff700 1 -- 172.21.15.131:0/2486146844 >> 172.21.15.131:0/2486146844 conn(0x7f792c06ece0 msgr2=0x7f792c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.211+0000 7f79345ff700 1 -- 172.21.15.131:0/2486146844 shutdown_connections 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.211+0000 7f79345ff700 1 -- 172.21.15.131:0/2486146844 wait complete. 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.213+0000 7f79345ff700 1 Processor -- start 2023-12-26T21:53:18.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.213+0000 7f79345ff700 1 -- start start 2023-12-26T21:53:18.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f79345ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f792c0c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f79345ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f792c0c77f0 0x7f792c0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f79345ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c0ca380 0x7f792c157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f79345ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0b4040 con 0x7f7918004d50 2023-12-26T21:53:18.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f79345ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0b3430 con 0x7f792c0c77f0 2023-12-26T21:53:18.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f79345ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0b3710 con 0x7f792c0ca380 2023-12-26T21:53:18.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f793239b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f792c0c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f793239b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f792c0c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53946/0 (socket says 172.21.15.131:53946) 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f793239b700 1 -- 172.21.15.131:0/1231741576 learned_addr learned my addr 172.21.15.131:0/1231741576 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f7931b9a700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f792c0c77f0 0x7f792c0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f7932b9c700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c0ca380 0x7f792c157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f793239b700 1 -- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c0ca380 msgr2=0x7f792c157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f793239b700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c0ca380 0x7f792c157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f793239b700 1 -- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f792c0c77f0 msgr2=0x7f792c0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f793239b700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f792c0c77f0 0x7f792c0c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f793239b700 1 -- 172.21.15.131:0/1231741576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c0b39f0 con 0x7f7918004d50 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f7931b9a700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f792c0c77f0 0x7f792c0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f7932b9c700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c0ca380 0x7f792c157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.214+0000 7f793239b700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f792c0c5890 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7928007390 tx=0x7f7928001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.215+0000 7f79227fc700 1 -- 172.21.15.131:0/1231741576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928018030 con 0x7f7918004d50 2023-12-26T21:53:18.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.215+0000 7f79227fc700 1 -- 172.21.15.131:0/1231741576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792800f8d0 con 0x7f7918004d50 2023-12-26T21:53:18.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.215+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792c0b3cd0 con 0x7f7918004d50 2023-12-26T21:53:18.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.215+0000 7f79227fc700 1 -- 172.21.15.131:0/1231741576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800fc70 con 0x7f7918004d50 2023-12-26T21:53:18.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.215+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792c1584a0 con 0x7f7918004d50 2023-12-26T21:53:18.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.216+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792c0b4320 con 0x7f7918004d50 2023-12-26T21:53:18.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.217+0000 7f79227fc700 1 -- 172.21.15.131:0/1231741576 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f792801b070 con 0x7f7918004d50 2023-12-26T21:53:18.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.218+0000 7f79227fc700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7910066f30 0x7f79100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.218+0000 7f79227fc700 1 -- 172.21.15.131:0/1231741576 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7928019070 con 0x7f7918004d50 2023-12-26T21:53:18.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.218+0000 7f7931b9a700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7910066f30 0x7f79100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.218+0000 7f7931b9a700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7910066f30 0x7f79100693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f791c0093f0 tx=0x7f791c008040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:18.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.222+0000 7f79227fc700 1 -- 172.21.15.131:0/1231741576 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7928051760 con 0x7f7918004d50 2023-12-26T21:53:18.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:53:18.364 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:53:18.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.449+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f792c0c6010 con 0x7f7910066f30 2023-12-26T21:53:18.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.451+0000 7f79227fc700 1 -- 172.21.15.131:0/1231741576 <== mgr.5789 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f792c0c6010 con 0x7f7910066f30 2023-12-26T21:53:18.452 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:53:18.452 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:18.452 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T21:53:18.452 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:18.452 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:53:18.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7910066f30 msgr2=0x7f79100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7910066f30 0x7f79100693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f791c0093f0 tx=0x7f791c008040 comp rx=0 tx=0).stop 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 msgr2=0x7f792c0c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f792c0c5890 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7928007390 tx=0x7f7928001e90 comp rx=0 tx=0).stop 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 shutdown_connections 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c0ca380 0x7f792c157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7910066f30 0x7f79100693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7918004d50 0x7f792c0c5890 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 --2- 172.21.15.131:0/1231741576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f792c0c77f0 0x7f792c0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 >> 172.21.15.131:0/1231741576 conn(0x7f792c06ece0 msgr2=0x7f792c152e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 shutdown_connections 2023-12-26T21:53:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.454+0000 7f79345ff700 1 -- 172.21.15.131:0/1231741576 wait complete. 2023-12-26T21:53:18.472 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T21:53:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.704+0000 7f97941ff700 1 Processor -- start 2023-12-26T21:53:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.704+0000 7f97941ff700 1 -- start start 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f97941ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f978c0b0750 0x7f978c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f97941ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f97941ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978c156940 con 0x7f978c0cb320 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f97941ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c0c8cc0 con 0x7f978c0b0750 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f97941ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0ad700 con 0x7f978c0c9400 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f979279c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f979179a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f978c0b0750 0x7f978c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f979279c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:42828/0 (socket says 172.21.15.131:42828) 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f979179a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f978c0b0750 0x7f978c0b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:53460/0 (socket says 172.21.15.131:53460) 2023-12-26T21:53:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.705+0000 7f979279c700 1 -- 172.21.15.131:0/4150100951 learned_addr learned my addr 172.21.15.131:0/4150100951 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:18.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.706+0000 7f979279c700 1 -- 172.21.15.131:0/4150100951 >> v2:172.21.15.190:3300/0 conn(0x7f978c0b0750 msgr2=0x7f978c0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:18.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.706+0000 7f979279c700 1 --2- 172.21.15.131:0/4150100951 >> v2:172.21.15.190:3300/0 conn(0x7f978c0b0750 0x7f978c0b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.706+0000 7f979279c700 1 -- 172.21.15.131:0/4150100951 >> v1:172.21.15.190:6789/0 conn(0x7f978c0cb320 legacy=0x7f978c069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.706+0000 7f979279c700 1 -- 172.21.15.131:0/4150100951 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c0ae500 con 0x7f978c0c9400 2023-12-26T21:53:18.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.706+0000 7f979179a700 1 --2- 172.21.15.131:0/4150100951 >> v2:172.21.15.190:3300/0 conn(0x7f978c0b0750 0x7f978c0b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:18.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.706+0000 7f979279c700 1 --2- 172.21.15.131:0/4150100951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c9880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9784009f20 tx=0x7f9784009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70d4ac5f71b06e16 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:18.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.706+0000 7f9790f99700 1 -- 172.21.15.131:0/4150100951 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978400e070 con 0x7f978c0c9400 2023-12-26T21:53:18.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.706+0000 7f9790f99700 1 -- 172.21.15.131:0/4150100951 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978400b9d0 con 0x7f978c0c9400 2023-12-26T21:53:18.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.706+0000 7f9790f99700 1 -- 172.21.15.131:0/4150100951 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97840092e0 con 0x7f978c0c9400 2023-12-26T21:53:18.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.707+0000 7f97941ff700 1 -- 172.21.15.131:0/4150100951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 msgr2=0x7f978c0c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.707+0000 7f97941ff700 1 --2- 172.21.15.131:0/4150100951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c9880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9784009f20 tx=0x7f9784009840 comp rx=0 tx=0).stop 2023-12-26T21:53:18.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.707+0000 7f97941ff700 1 -- 172.21.15.131:0/4150100951 shutdown_connections 2023-12-26T21:53:18.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.707+0000 7f97941ff700 1 --2- 172.21.15.131:0/4150100951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c9880 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.707+0000 7f97941ff700 1 --2- 172.21.15.131:0/4150100951 >> v2:172.21.15.190:3300/0 conn(0x7f978c0b0750 0x7f978c0b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.707+0000 7f97941ff700 1 -- 172.21.15.131:0/4150100951 >> 172.21.15.131:0/4150100951 conn(0x7f978c068040 msgr2=0x7f978c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:18.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.707+0000 7f97941ff700 1 -- 172.21.15.131:0/4150100951 shutdown_connections 2023-12-26T21:53:18.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.707+0000 7f97941ff700 1 -- 172.21.15.131:0/4150100951 wait complete. 2023-12-26T21:53:18.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.708+0000 7f97941ff700 1 Processor -- start 2023-12-26T21:53:18.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.709+0000 7f97941ff700 1 -- start start 2023-12-26T21:53:18.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.709+0000 7f97941ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f978c0b0750 0x7f978c14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f97941ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f97941ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f978c0cb320 0x7f978c0c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f97941ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0c76a0 con 0x7f978c0cb320 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f97941ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0c5ff0 con 0x7f978c0b0750 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f97941ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0c62d0 con 0x7f978c0c9400 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f979279c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f978c0cb320 0x7f978c0c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f9791f9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f978c0b0750 0x7f978c14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f979279c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f978c0cb320 0x7f978c0c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53956/0 (socket says 172.21.15.131:53956) 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f979179a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f979279c700 1 -- 172.21.15.131:0/2482964651 learned_addr learned my addr 172.21.15.131:0/2482964651 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f979279c700 1 -- 172.21.15.131:0/2482964651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 msgr2=0x7f978c0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f979279c700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f979279c700 1 -- 172.21.15.131:0/2482964651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f978c0b0750 msgr2=0x7f978c14fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f979279c700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f978c0b0750 0x7f978c14fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f979279c700 1 -- 172.21.15.131:0/2482964651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c009420 con 0x7f978c0cb320 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.710+0000 7f9791f9b700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f978c0b0750 0x7f978c14fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.711+0000 7f979279c700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f978c0cb320 0x7f978c0c5a00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f978400b610 tx=0x7f9784018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.711+0000 7f9781ffb700 1 -- 172.21.15.131:0/2482964651 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784008ad0 con 0x7f978c0cb320 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.711+0000 7f9781ffb700 1 -- 172.21.15.131:0/2482964651 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9784008c30 con 0x7f978c0cb320 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.711+0000 7f9781ffb700 1 -- 172.21.15.131:0/2482964651 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784026460 con 0x7f978c0cb320 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.711+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9784009ae0 con 0x7f978c0cb320 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.711+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c0c6890 con 0x7f978c0cb320 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.712+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c158180 con 0x7f978c0cb320 2023-12-26T21:53:18.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.713+0000 7f9781ffb700 1 -- 172.21.15.131:0/2482964651 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 35) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9784003040 con 0x7f978c0cb320 2023-12-26T21:53:18.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.714+0000 7f9781ffb700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9770066f30 0x7f97700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:18.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.714+0000 7f9781ffb700 1 -- 172.21.15.131:0/2482964651 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f9784085fe0 con 0x7f978c0cb320 2023-12-26T21:53:18.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.714+0000 7f9791f9b700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9770066f30 0x7f97700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:18.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.715+0000 7f9791f9b700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9770066f30 0x7f97700693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9788001d20 tx=0x7f9788009040 comp rx=0 tx=0).ready entity=mgr.5789 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:18.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.718+0000 7f9781ffb700 1 -- 172.21.15.131:0/2482964651 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f978c158180 con 0x7f978c0cb320 2023-12-26T21:53:18.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:18.962+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f978c158180 con 0x7f978c0cb320 2023-12-26T21:53:19.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.333+0000 7f9781ffb700 1 -- 172.21.15.131:0/2482964651 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 36) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f9784053410 con 0x7f978c0cb320 2023-12-26T21:53:19.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.335+0000 7f9781ffb700 1 -- 172.21.15.131:0/2482964651 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v36) v1 ==== 92+0+0 (secure 0 0 0) 0x7f9784056660 con 0x7f978c0cb320 2023-12-26T21:53:19.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9770066f30 msgr2=0x7f97700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:19.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9770066f30 0x7f97700693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9788001d20 tx=0x7f9788009040 comp rx=0 tx=0).stop 2023-12-26T21:53:19.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f978c0cb320 msgr2=0x7f978c0c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:19.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f978c0cb320 0x7f978c0c5a00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f978400b610 tx=0x7f9784018000 comp rx=0 tx=0).stop 2023-12-26T21:53:19.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 shutdown_connections 2023-12-26T21:53:19.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9770066f30 0x7f97700693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f978c0cb320 0x7f978c0c5a00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f978c0b0750 0x7f978c14fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 --2- 172.21.15.131:0/2482964651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f978c0c9400 0x7f978c0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 >> 172.21.15.131:0/2482964651 conn(0x7f978c068040 msgr2=0x7f978c06d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:19.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 shutdown_connections 2023-12-26T21:53:19.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.339+0000 7f97941ff700 1 -- 172.21.15.131:0/2482964651 wait complete. 2023-12-26T21:53:19.366 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T21:53:19.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.565+0000 7f83f39ff700 1 Processor -- start 2023-12-26T21:53:19.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.565+0000 7f83f39ff700 1 -- start start 2023-12-26T21:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.566+0000 7f83f39ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f83ec13c510 0x7f83ec0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.566+0000 7f83f39ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f83ec06b8c0 0x7f83ec06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.566+0000 7f83f39ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 0x7f83ec0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.566+0000 7f83f39ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec06c280 con 0x7f83ec13c510 2023-12-26T21:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.566+0000 7f83f39ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0686c0 con 0x7f83ec06b8c0 2023-12-26T21:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.566+0000 7f83f39ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec069520 con 0x7f83ec0cc100 2023-12-26T21:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f179b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f83ec13c510 0x7f83ec0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f179b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f83ec13c510 0x7f83ec0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53972/0 (socket says 172.21.15.131:53972) 2023-12-26T21:53:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f179b700 1 -- 172.21.15.131:0/1779445739 learned_addr learned my addr 172.21.15.131:0/1779445739 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f1f9c700 1 --2- 172.21.15.131:0/1779445739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 0x7f83ec0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f0f9a700 1 --2- 172.21.15.131:0/1779445739 >> v2:172.21.15.190:3300/0 conn(0x7f83ec06b8c0 0x7f83ec06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f179b700 1 -- 172.21.15.131:0/1779445739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 msgr2=0x7f83ec0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f179b700 1 --2- 172.21.15.131:0/1779445739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 0x7f83ec0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f179b700 1 -- 172.21.15.131:0/1779445739 >> v2:172.21.15.190:3300/0 conn(0x7f83ec06b8c0 msgr2=0x7f83ec06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f179b700 1 --2- 172.21.15.131:0/1779445739 >> v2:172.21.15.190:3300/0 conn(0x7f83ec06b8c0 0x7f83ec06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f179b700 1 -- 172.21.15.131:0/1779445739 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec068910 con 0x7f83ec13c510 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.567+0000 7f83f179b700 1 --2- 172.21.15.131:0/1779445739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec13c510 0x7f83ec0c8a20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0b43e0 tx=0x7f83e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36879ff72a109f47 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83e3fff700 1 -- 172.21.15.131:0/1779445739 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e800b640 con 0x7f83ec13c510 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83e3fff700 1 -- 172.21.15.131:0/1779445739 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e800b7a0 con 0x7f83ec13c510 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83e3fff700 1 -- 172.21.15.131:0/1779445739 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e800bb60 con 0x7f83ec13c510 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83f39ff700 1 -- 172.21.15.131:0/1779445739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec13c510 msgr2=0x7f83ec0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1779445739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec13c510 0x7f83ec0c8a20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0b43e0 tx=0x7f83e8009250 comp rx=0 tx=0).stop 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83f39ff700 1 -- 172.21.15.131:0/1779445739 shutdown_connections 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1779445739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec13c510 0x7f83ec0c8a20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1779445739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 0x7f83ec0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1779445739 >> v2:172.21.15.190:3300/0 conn(0x7f83ec06b8c0 0x7f83ec06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.568+0000 7f83f39ff700 1 -- 172.21.15.131:0/1779445739 >> 172.21.15.131:0/1779445739 conn(0x7f83ec0ce3e0 msgr2=0x7f83ec0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.569+0000 7f83f39ff700 1 -- 172.21.15.131:0/1779445739 shutdown_connections 2023-12-26T21:53:19.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.569+0000 7f83f39ff700 1 -- 172.21.15.131:0/1779445739 wait complete. 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.570+0000 7f83f39ff700 1 Processor -- start 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.570+0000 7f83f39ff700 1 -- start start 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f39ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec06b8c0 0x7f83ec13bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f39ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 0x7f83ec154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f39ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83ec13c510 0x7f83ec1548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f39ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0b2090 con 0x7f83ec06b8c0 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f39ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec15ea30 con 0x7f83ec13c510 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f39ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec15ed10 con 0x7f83ec0cc100 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f179b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec06b8c0 0x7f83ec13bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f179b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec06b8c0 0x7f83ec13bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53984/0 (socket says 172.21.15.131:53984) 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f179b700 1 -- 172.21.15.131:0/1457836534 learned_addr learned my addr 172.21.15.131:0/1457836534 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f0f9a700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 0x7f83ec154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.571+0000 7f83f1f9c700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83ec13c510 0x7f83ec1548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83f179b700 1 -- 172.21.15.131:0/1457836534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 msgr2=0x7f83ec154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83f179b700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 0x7f83ec154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83f179b700 1 -- 172.21.15.131:0/1457836534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83ec13c510 msgr2=0x7f83ec1548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83f179b700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83ec13c510 0x7f83ec1548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83f179b700 1 -- 172.21.15.131:0/1457836534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e8009490 con 0x7f83ec06b8c0 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83f179b700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec06b8c0 0x7f83ec13bd50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009fd0 tx=0x7f83e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83e17fa700 1 -- 172.21.15.131:0/1457836534 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e801f070 con 0x7f83ec06b8c0 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83e17fa700 1 -- 172.21.15.131:0/1457836534 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8002f30 con 0x7f83ec06b8c0 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83e17fa700 1 -- 172.21.15.131:0/1457836534 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e800b910 con 0x7f83ec06b8c0 2023-12-26T21:53:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.572+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec0b2330 con 0x7f83ec06b8c0 2023-12-26T21:53:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.573+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec13c290 con 0x7f83ec06b8c0 2023-12-26T21:53:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.574+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec0657e0 con 0x7f83ec06b8c0 2023-12-26T21:53:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.575+0000 7f83e17fa700 1 -- 172.21.15.131:0/1457836534 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 36) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f83e8023070 con 0x7f83ec06b8c0 2023-12-26T21:53:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.576+0000 7f83e17fa700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f83d0066f80 0x7f83d0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.576+0000 7f83e17fa700 1 -- 172.21.15.131:0/1457836534 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f83e8085570 con 0x7f83ec06b8c0 2023-12-26T21:53:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.576+0000 7f83f0f9a700 1 -- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f83d0066f80 msgr2=0x7f83d0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.576+0000 7f83f0f9a700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f83d0066f80 0x7f83d0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.581+0000 7f83e17fa700 1 -- 172.21.15.131:0/1457836534 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83e8052650 con 0x7f83ec06b8c0 2023-12-26T21:53:19.765 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:19.764+0000 7fafb20731c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:53:19.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.777+0000 7f83f0f9a700 1 -- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f83d0066f80 msgr2=0x7f83d0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:19.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.777+0000 7f83f0f9a700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f83d0066f80 0x7f83d0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:19.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.782+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f83ec0b2850 con 0x7f83ec06b8c0 2023-12-26T21:53:19.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.786+0000 7f83e17fa700 1 -- 172.21.15.131:0/1457836534 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f83e8055c70 con 0x7f83ec06b8c0 2023-12-26T21:53:19.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f83d0066f80 msgr2=0x7f83d0069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:19.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f83d0066f80 0x7f83d0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec06b8c0 msgr2=0x7f83ec13bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:19.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec06b8c0 0x7f83ec13bd50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009fd0 tx=0x7f83e8017000 comp rx=0 tx=0).stop 2023-12-26T21:53:19.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 shutdown_connections 2023-12-26T21:53:19.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83ec13c510 0x7f83ec1548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f83d0066f80 0x7f83d0069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83ec06b8c0 0x7f83ec13bd50 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 --2- 172.21.15.131:0/1457836534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83ec0cc100 0x7f83ec154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:19.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.788+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 >> 172.21.15.131:0/1457836534 conn(0x7f83ec0ce3e0 msgr2=0x7f83ec1524f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:19.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.789+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 shutdown_connections 2023-12-26T21:53:19.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:19.789+0000 7f83f39ff700 1 -- 172.21.15.131:0/1457836534 wait complete. 2023-12-26T21:53:19.799 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:19.799+0000 7f19fb2bc1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:53:19.808 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T21:53:19.870 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:19.870+0000 7fafb20731c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:53:19.910 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:19.910+0000 7f19fb2bc1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:53:19.958 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:19.958+0000 7fafb20731c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:53:20.004 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:20.003+0000 7f19fb2bc1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:53:20.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.005+0000 7fa21d7ff700 1 Processor -- start 2023-12-26T21:53:20.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.005+0000 7fa21d7ff700 1 -- start start 2023-12-26T21:53:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.006+0000 7fa21d7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 0x7fa218069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.006+0000 7fa21d7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa2180b0e60 0x7fa2180c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.006+0000 7fa21d7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa218156940 con 0x7fa21806a1e0 2023-12-26T21:53:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.006+0000 7fa21d7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2180c8cc0 con 0x7fa2180b0e60 2023-12-26T21:53:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.006+0000 7fa21d7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180ad700 con 0x7fa2180cb320 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa2177fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa2180b0e60 0x7fa2180c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa2177fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa2180b0e60 0x7fa2180c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53998/0 (socket says 172.21.15.131:53998) 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa2177fe700 1 -- 172.21.15.131:0/1457227843 learned_addr learned my addr 172.21.15.131:0/1457227843 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa216ffd700 1 --2- 172.21.15.131:0/1457227843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 0x7fa218069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa2177fe700 1 -- 172.21.15.131:0/1457227843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 msgr2=0x7fa218069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa2177fe700 1 --2- 172.21.15.131:0/1457227843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 0x7fa218069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa2177fe700 1 -- 172.21.15.131:0/1457227843 >> v1:172.21.15.131:6789/0 conn(0x7fa21806a1e0 legacy=0x7fa2180b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa2177fe700 1 -- 172.21.15.131:0/1457227843 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2180ae500 con 0x7fa2180b0e60 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa216ffd700 1 --2- 172.21.15.131:0/1457227843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 0x7fa218069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.007+0000 7fa2177fe700 1 --2- 172.21.15.131:0/1457227843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 0x7fa2180c9400 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa210009f20 tx=0x7fa210009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab069e05d2ed633 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa215ffb700 1 -- 172.21.15.131:0/1457227843 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21000e070 con 0x7fa2180b0e60 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa215ffb700 1 -- 172.21.15.131:0/1457227843 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21000ba80 con 0x7fa2180b0e60 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa215ffb700 1 -- 172.21.15.131:0/1457227843 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2100092e0 con 0x7fa2180b0e60 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1457227843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 msgr2=0x7fa2180c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa21d7ff700 1 --2- 172.21.15.131:0/1457227843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 0x7fa2180c9400 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa210009f20 tx=0x7fa210009840 comp rx=0 tx=0).stop 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1457227843 shutdown_connections 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa21d7ff700 1 --2- 172.21.15.131:0/1457227843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 0x7fa2180c9400 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa21d7ff700 1 --2- 172.21.15.131:0/1457227843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 0x7fa218069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1457227843 >> 172.21.15.131:0/1457227843 conn(0x7fa218068040 msgr2=0x7fa218069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.008+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1457227843 shutdown_connections 2023-12-26T21:53:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.009+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1457227843 wait complete. 2023-12-26T21:53:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.010+0000 7fa21d7ff700 1 Processor -- start 2023-12-26T21:53:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.010+0000 7fa21d7ff700 1 -- start start 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa21d7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa21806a1e0 0x7fa2180c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa21d7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 0x7fa218144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa21d7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 0x7fa2180c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa21d7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180c7de0 con 0x7fa2180b0e60 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa21d7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21815e000 con 0x7fa21806a1e0 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa21d7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180c7410 con 0x7fa2180cb320 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa2167fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 0x7fa218144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa2167fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 0x7fa218144ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54002/0 (socket says 172.21.15.131:54002) 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa2167fc700 1 -- 172.21.15.131:0/1095065811 learned_addr learned my addr 172.21.15.131:0/1095065811 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa216ffd700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa21806a1e0 0x7fa2180c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.011+0000 7fa2177fe700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 0x7fa2180c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa2167fc700 1 -- 172.21.15.131:0/1095065811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 msgr2=0x7fa2180c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa2167fc700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 0x7fa2180c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa2167fc700 1 -- 172.21.15.131:0/1095065811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa21806a1e0 msgr2=0x7fa2180c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa2167fc700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa21806a1e0 0x7fa2180c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa2167fc700 1 -- 172.21.15.131:0/1095065811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa210009ae0 con 0x7fa2180b0e60 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa2167fc700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 0x7fa218144ba0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00c590 tx=0x7fa20c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa1feffd700 1 -- 172.21.15.131:0/1095065811 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c011590 con 0x7fa2180b0e60 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa1feffd700 1 -- 172.21.15.131:0/1095065811 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c011710 con 0x7fa2180b0e60 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa1feffd700 1 -- 172.21.15.131:0/1095065811 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c011ad0 con 0x7fa2180b0e60 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.012+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2180c6800 con 0x7fa2180b0e60 2023-12-26T21:53:20.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.013+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2180c6dc0 con 0x7fa2180b0e60 2023-12-26T21:53:20.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.014+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2180c76f0 con 0x7fa2180b0e60 2023-12-26T21:53:20.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.014+0000 7fa1feffd700 1 -- 172.21.15.131:0/1095065811 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 36) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fa20c01b3b0 con 0x7fa2180b0e60 2023-12-26T21:53:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.015+0000 7fa1feffd700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa204066f80 0x7fa204069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.015+0000 7fa1feffd700 1 -- 172.21.15.131:0/1095065811 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fa20c083b30 con 0x7fa2180b0e60 2023-12-26T21:53:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.015+0000 7fa216ffd700 1 -- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa204066f80 msgr2=0x7fa204069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.015+0000 7fa216ffd700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa204066f80 0x7fa204069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.019+0000 7fa1feffd700 1 -- 172.21.15.131:0/1095065811 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa20c050c10 con 0x7fa2180b0e60 2023-12-26T21:53:20.040 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:20.040+0000 7fafb20731c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:53:20.091 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:20.090+0000 7f19fb2bc1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:53:20.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.216+0000 7fa216ffd700 1 -- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa204066f80 msgr2=0x7fa204069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:20.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.216+0000 7fa216ffd700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa204066f80 0x7fa204069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:20.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.221+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fa2180c7550 con 0x7fa2180b0e60 2023-12-26T21:53:20.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.225+0000 7fa1feffd700 1 -- 172.21.15.131:0/1095065811 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa2180c7550 con 0x7fa2180b0e60 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa204066f80 msgr2=0x7fa204069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa204066f80 0x7fa204069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 msgr2=0x7fa218144ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 0x7fa218144ba0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00c590 tx=0x7fa20c00c8a0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 shutdown_connections 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2180cb320 0x7fa2180c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa21806a1e0 0x7fa2180c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa204066f80 0x7fa204069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 --2- 172.21.15.131:0/1095065811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2180b0e60 0x7fa218144ba0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 >> 172.21.15.131:0/1095065811 conn(0x7fa218068040 msgr2=0x7fa2180c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:20.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 shutdown_connections 2023-12-26T21:53:20.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.228+0000 7fa21d7ff700 1 -- 172.21.15.131:0/1095065811 wait complete. 2023-12-26T21:53:20.230 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:20.230+0000 7fafb20731c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:53:20.248 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-12-26T21:53:20.291 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:20.291+0000 7f19fb2bc1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:53:20.403 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:20.402+0000 7fafb20731c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:53:20.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.447+0000 7ff6e55ff700 1 Processor -- start 2023-12-26T21:53:20.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.447+0000 7ff6e55ff700 1 -- start start 2023-12-26T21:53:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.449+0000 7ff6e55ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e01593f0 0x7ff6e00abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.449+0000 7ff6e55ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff6e01597c0 0x7ff6e00aa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.449+0000 7ff6e55ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6e0069e60 0x7ff6e006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.449+0000 7ff6e55ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e0068480 con 0x7ff6e01597c0 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.449+0000 7ff6e55ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e006a820 con 0x7ff6e0069e60 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.449+0000 7ff6e55ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0067ff0 con 0x7ff6e01593f0 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.449+0000 7ff6de7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff6e01597c0 0x7ff6e00aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6deffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e01593f0 0x7ff6e00abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6de7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff6e01597c0 0x7ff6e00aa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54018/0 (socket says 172.21.15.131:54018) 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6de7fc700 1 -- 172.21.15.131:0/4203405189 learned_addr learned my addr 172.21.15.131:0/4203405189 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6df7fe700 1 --2- 172.21.15.131:0/4203405189 >> v2:172.21.15.190:3300/0 conn(0x7ff6e0069e60 0x7ff6e006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6de7fc700 1 -- 172.21.15.131:0/4203405189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e01593f0 msgr2=0x7ff6e00abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6de7fc700 1 --2- 172.21.15.131:0/4203405189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e01593f0 0x7ff6e00abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6de7fc700 1 -- 172.21.15.131:0/4203405189 >> v2:172.21.15.190:3300/0 conn(0x7ff6e0069e60 msgr2=0x7ff6e006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6de7fc700 1 --2- 172.21.15.131:0/4203405189 >> v2:172.21.15.190:3300/0 conn(0x7ff6e0069e60 0x7ff6e006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6de7fc700 1 -- 172.21.15.131:0/4203405189 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00ac350 con 0x7ff6e01597c0 2023-12-26T21:53:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6deffd700 1 --2- 172.21.15.131:0/4203405189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e01593f0 0x7ff6e00abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6df7fe700 1 --2- 172.21.15.131:0/4203405189 >> v2:172.21.15.190:3300/0 conn(0x7ff6e0069e60 0x7ff6e006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6de7fc700 1 --2- 172.21.15.131:0/4203405189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6e01597c0 0x7ff6e00aa270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0063a60 tx=0x7ff6d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c741a4c9b0501b9f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:20.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.450+0000 7ff6ddffb700 1 -- 172.21.15.131:0/4203405189 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400b640 con 0x7ff6e01597c0 2023-12-26T21:53:20.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.451+0000 7ff6ddffb700 1 -- 172.21.15.131:0/4203405189 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d400b7a0 con 0x7ff6e01597c0 2023-12-26T21:53:20.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.451+0000 7ff6ddffb700 1 -- 172.21.15.131:0/4203405189 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400bb60 con 0x7ff6e01597c0 2023-12-26T21:53:20.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.451+0000 7ff6e55ff700 1 -- 172.21.15.131:0/4203405189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6e01597c0 msgr2=0x7ff6e00aa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:20.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.451+0000 7ff6e55ff700 1 --2- 172.21.15.131:0/4203405189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6e01597c0 0x7ff6e00aa270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0063a60 tx=0x7ff6d4009250 comp rx=0 tx=0).stop 2023-12-26T21:53:20.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.451+0000 7ff6e55ff700 1 -- 172.21.15.131:0/4203405189 shutdown_connections 2023-12-26T21:53:20.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.451+0000 7ff6e55ff700 1 --2- 172.21.15.131:0/4203405189 >> v2:172.21.15.190:3300/0 conn(0x7ff6e0069e60 0x7ff6e006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.451+0000 7ff6e55ff700 1 --2- 172.21.15.131:0/4203405189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e01593f0 0x7ff6e00abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.451+0000 7ff6e55ff700 1 --2- 172.21.15.131:0/4203405189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6e01597c0 0x7ff6e00aa270 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.451+0000 7ff6e55ff700 1 -- 172.21.15.131:0/4203405189 >> 172.21.15.131:0/4203405189 conn(0x7ff6e006b710 msgr2=0x7ff6e006bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:20.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.452+0000 7ff6e55ff700 1 -- 172.21.15.131:0/4203405189 shutdown_connections 2023-12-26T21:53:20.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.452+0000 7ff6e55ff700 1 -- 172.21.15.131:0/4203405189 wait complete. 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.453+0000 7ff6e55ff700 1 Processor -- start 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.453+0000 7ff6e55ff700 1 -- start start 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6e55ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e0069e60 0x7ff6e014d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6e55ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6e01593f0 0x7ff6e015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6e55ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6e01597c0 0x7ff6e015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6e55ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0151d00 con 0x7ff6e01597c0 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6e55ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0155480 con 0x7ff6e01593f0 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6e55ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0156000 con 0x7ff6e0069e60 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6e01597c0 0x7ff6e015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6de7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6e01593f0 0x7ff6e015bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6deffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e0069e60 0x7ff6e014d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:20.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6df7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6e01597c0 0x7ff6e015c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54030/0 (socket says 172.21.15.131:54030) 2023-12-26T21:53:20.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.454+0000 7ff6df7fe700 1 -- 172.21.15.131:0/1849804369 learned_addr learned my addr 172.21.15.131:0/1849804369 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6de7fc700 1 -- 172.21.15.131:0/1849804369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e0069e60 msgr2=0x7ff6e014d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6de7fc700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e0069e60 0x7ff6e014d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6de7fc700 1 -- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6e01597c0 msgr2=0x7ff6e015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6de7fc700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6e01597c0 0x7ff6e015c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6de7fc700 1 -- 172.21.15.131:0/1849804369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d8009a20 con 0x7ff6e01593f0 2023-12-26T21:53:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6de7fc700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6e01593f0 0x7ff6e015bed0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff6e006f8b0 tx=0x7ff6d4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400b910 con 0x7ff6e01593f0 2023-12-26T21:53:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d400ba70 con 0x7ff6e01593f0 2023-12-26T21:53:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4011650 con 0x7ff6e01593f0 2023-12-26T21:53:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.455+0000 7ff6e55ff700 1 -- 172.21.15.131:0/1849804369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d0009420 con 0x7ff6e01593f0 2023-12-26T21:53:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.456+0000 7ff6e55ff700 1 -- 172.21.15.131:0/1849804369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d40094f0 con 0x7ff6e01593f0 2023-12-26T21:53:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.456+0000 7ff6e55ff700 1 -- 172.21.15.131:0/1849804369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e0156170 con 0x7ff6e01593f0 2023-12-26T21:53:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.458+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 36) v1 ==== 84759+0+0 (secure 0 0 0) 0x7ff6d4044be0 con 0x7ff6e01593f0 2023-12-26T21:53:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.458+0000 7ff6c6ffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.458+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff6d4085e80 con 0x7ff6e01593f0 2023-12-26T21:53:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.458+0000 7ff6deffd700 1 -- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 msgr2=0x7ff6cc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.459+0000 7ff6deffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:20.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.462+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6d4052fc0 con 0x7ff6e01593f0 2023-12-26T21:53:20.475 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:20.475+0000 7f19fb2bc1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:53:20.487 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:20.487+0000 7fafb20731c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:53:20.563 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:20.563+0000 7f19fb2bc1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:53:20.632 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:20.632+0000 7fafb20731c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:53:20.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.660+0000 7ff6deffd700 1 -- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 msgr2=0x7ff6cc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:20.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.660+0000 7ff6deffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:20.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:20.689+0000 7ff6e55ff700 1 -- 172.21.15.131:0/1849804369 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6e00af960 con 0x7ff6cc066f80 2023-12-26T21:53:20.714 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:20.714+0000 7f19fb2bc1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:53:20.722 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:20.722+0000 7fafb20731c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:53:20.806 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:20.805+0000 7f19fb2bc1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:53:20.904 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:20.904+0000 7fafb20731c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:53:20.989 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:20.988+0000 7f19fb2bc1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:53:21.052 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:21.052+0000 7fafb20731c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:53:21.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:21.061+0000 7ff6deffd700 1 -- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 msgr2=0x7ff6cc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:21.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:21.061+0000 7ff6deffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:53:21.138 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:21.138+0000 7f19fb2bc1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:53:21.439 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:21.439+0000 7fafb20731c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:53:21.526 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:21.526+0000 7f19fb2bc1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:53:21.705 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:21.705+0000 7fafb20731c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:53:21.792 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:21.792+0000 7f19fb2bc1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:53:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:21.863+0000 7ff6deffd700 1 -- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 msgr2=0x7ff6cc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:21.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:21.863+0000 7ff6deffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T21:53:21.984 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:21.982+0000 7fafb20731c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:53:22.071 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:22.071+0000 7fafb20731c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:53:22.072 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:22.072+0000 7f19fb2bc1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:53:22.160 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:22.161+0000 7f19fb2bc1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:53:22.802 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:22.802+0000 7fafb20731c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:53:22.897 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:22.897+0000 7fafb20731c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:53:22.897 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:22.897+0000 7f19fb2bc1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:53:22.991 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:22.992+0000 7f19fb2bc1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:53:23.048 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:23.048+0000 7fafb20731c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:53:23.145 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:23.145+0000 7f19fb2bc1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:53:23.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:23.465+0000 7ff6deffd700 1 -- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 msgr2=0x7ff6cc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:23.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:23.465+0000 7ff6deffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T21:53:23.667 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:23.667+0000 7fafb20731c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:53:23.772 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:23.772+0000 7f19fb2bc1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:53:23.855 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:23.855+0000 7fafb20731c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:53:23.963 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:23.964+0000 7f19fb2bc1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:53:24.043 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:24.043+0000 7fafb20731c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:53:24.154 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:24.154+0000 7f19fb2bc1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:53:24.883 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:24.882+0000 7fafb20731c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:53:24.994 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:24.995+0000 7f19fb2bc1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:53:25.155 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:25.155+0000 7fafb20731c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:53:25.248 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:25.247+0000 7fafb20731c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:53:25.271 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.271+0000 7f19fb2bc1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:53:25.364 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.364+0000 7f19fb2bc1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:53:25.442 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:25.441+0000 7fafb20731c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:53:25.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:25.505+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mgrmap(e 37) v1 ==== 84759+0+0 (secure 0 0 0) 0x7ff6d4053390 con 0x7ff6e01593f0 2023-12-26T21:53:25.564 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.565+0000 7f19fb2bc1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:53:25.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:25.576+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mgrmap(e 38) v1 ==== 84470+0+0 (secure 0 0 0) 0x7ff6d4054a30 con 0x7ff6e01593f0 2023-12-26T21:53:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:25.576+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 msgr2=0x7ff6cc069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:25.576+0000 7ff6c6ffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:25.696 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.696+0000 7f196cbf1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:25.696 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.696+0000 7f196cbf1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:25.696 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.696+0000 7f196cbf1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:25.696 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.696+0000 7f196cbf1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:25.696 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.696+0000 7f196cbf1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:25.697 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.697+0000 7f196ebf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:25.697 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.697+0000 7f196ebf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:25.697 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.697+0000 7f196ebf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:25.697 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.697+0000 7f196ebf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:25.697 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:25.697+0000 7f196ebf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:53:26.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.582+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mon.1 v2:172.21.15.190:3300/0 9 ==== mgrmap(e 39) v1 ==== 84597+0+0 (secure 0 0 0) 0x7ff6d405f030 con 0x7ff6e01593f0 2023-12-26T21:53:26.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.583+0000 7ff6c6ffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:26.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.583+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6cc033140 con 0x7ff6cc066f80 2023-12-26T21:53:26.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.586+0000 7ff6deffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:26.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.590+0000 7ff6deffd700 1 --2- 172.21.15.131:0/1849804369 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff6cc066f80 0x7ff6cc0693b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00098c0 tx=0x7ff6d00091a0 comp rx=0 tx=0).ready entity=mgr.6894 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:26.597 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:26.598+0000 7f1976c05700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-26T21:53:26.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.598+0000 7ff6c6ffd700 1 -- 172.21.15.131:0/1849804369 <== mgr.6894 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7ff6cc033140 con 0x7ff6cc066f80 2023-12-26T21:53:26.599 INFO:teuthology.orchestra.run.smithi131.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-26T21:53:26.608 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-26T21:53:26.609 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T21:53:26.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.843+0000 7f8aca9ff700 1 Processor -- start 2023-12-26T21:53:26.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.843+0000 7f8aca9ff700 1 -- start start 2023-12-26T21:53:26.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.844+0000 7f8aca9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 0x7f8ac40c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:26.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.844+0000 7f8aca9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ac406b8c0 0x7f8ac406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:26.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.844+0000 7f8aca9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8ac40cc100 0x7f8ac40cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:26.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.844+0000 7f8aca9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac406c270 con 0x7f8ac40cc100 2023-12-26T21:53:26.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.844+0000 7f8aca9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac40686c0 con 0x7f8ac406b8c0 2023-12-26T21:53:26.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.844+0000 7f8aca9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac4069520 con 0x7f8ac413c510 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac8f9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8ac40cc100 0x7f8ac40cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac37fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ac406b8c0 0x7f8ac406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 0x7f8ac40c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac8f9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8ac40cc100 0x7f8ac40cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48960/0 (socket says 172.21.15.131:48960) 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac37fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ac406b8c0 0x7f8ac406bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:36946/0 (socket says 172.21.15.131:36946) 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 0x7f8ac40c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:40012/0 (socket says 172.21.15.131:40012) 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac8f9c700 1 -- 172.21.15.131:0/3467054598 learned_addr learned my addr 172.21.15.131:0/3467054598 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac8f9c700 1 -- 172.21.15.131:0/3467054598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 msgr2=0x7f8ac40c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac8f9c700 1 --2- 172.21.15.131:0/3467054598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 0x7f8ac40c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac8f9c700 1 -- 172.21.15.131:0/3467054598 >> v2:172.21.15.190:3300/0 conn(0x7f8ac406b8c0 msgr2=0x7f8ac406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac8f9c700 1 --2- 172.21.15.131:0/3467054598 >> v2:172.21.15.190:3300/0 conn(0x7f8ac406b8c0 0x7f8ac406bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac8f9c700 1 -- 172.21.15.131:0/3467054598 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4068910 con 0x7f8ac40cc100 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.845+0000 7f8ac8f9c700 1 --2- 172.21.15.131:0/3467054598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac40cc100 0x7f8ac40cc560 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8ac406fd20 tx=0x7f8abc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4feb492c80a8467 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:26.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8ac2ffd700 1 -- 172.21.15.131:0/3467054598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc00e070 con 0x7f8ac40cc100 2023-12-26T21:53:26.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8ac2ffd700 1 -- 172.21.15.131:0/3467054598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc00b970 con 0x7f8ac40cc100 2023-12-26T21:53:26.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8ac2ffd700 1 -- 172.21.15.131:0/3467054598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0092e0 con 0x7f8ac40cc100 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8aca9ff700 1 -- 172.21.15.131:0/3467054598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac40cc100 msgr2=0x7f8ac40cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/3467054598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac40cc100 0x7f8ac40cc560 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8ac406fd20 tx=0x7f8abc009840 comp rx=0 tx=0).stop 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8aca9ff700 1 -- 172.21.15.131:0/3467054598 shutdown_connections 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/3467054598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac40cc100 0x7f8ac40cc560 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/3467054598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 0x7f8ac40c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/3467054598 >> v2:172.21.15.190:3300/0 conn(0x7f8ac406b8c0 0x7f8ac406bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8aca9ff700 1 -- 172.21.15.131:0/3467054598 >> 172.21.15.131:0/3467054598 conn(0x7f8ac40ce3e0 msgr2=0x7f8ac40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8aca9ff700 1 -- 172.21.15.131:0/3467054598 shutdown_connections 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.846+0000 7f8aca9ff700 1 -- 172.21.15.131:0/3467054598 wait complete. 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.848+0000 7f8aca9ff700 1 Processor -- start 2023-12-26T21:53:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.848+0000 7f8aca9ff700 1 -- start start 2023-12-26T21:53:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8aca9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac406b8c0 0x7f8ac4168a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8aca9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40cc100 0x7f8ac41566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8aca9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 0x7f8ac4156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8aca9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac415a030 con 0x7f8ac406b8c0 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8aca9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac415bb00 con 0x7f8ac40cc100 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8aca9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac415b820 con 0x7f8ac413c510 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8ac3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac406b8c0 0x7f8ac4168a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8ac3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac406b8c0 0x7f8ac4168a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48974/0 (socket says 172.21.15.131:48974) 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8ac3fff700 1 -- 172.21.15.131:0/2267591284 learned_addr learned my addr 172.21.15.131:0/2267591284 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8ac37fe700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40cc100 0x7f8ac41566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8ac8f9c700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 0x7f8ac4156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8ac3fff700 1 -- 172.21.15.131:0/2267591284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 msgr2=0x7f8ac4156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.849+0000 7f8ac3fff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 0x7f8ac4156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.850+0000 7f8ac3fff700 1 -- 172.21.15.131:0/2267591284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40cc100 msgr2=0x7f8ac41566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.850+0000 7f8ac3fff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40cc100 0x7f8ac41566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.850+0000 7f8ac3fff700 1 -- 172.21.15.131:0/2267591284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc009ae0 con 0x7f8ac406b8c0 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.850+0000 7f8ac3fff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac406b8c0 0x7f8ac4168a30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400ec80 tx=0x7f8ab4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.850+0000 7f8aabfff700 1 -- 172.21.15.131:0/2267591284 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400c850 con 0x7f8ac406b8c0 2023-12-26T21:53:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.850+0000 7f8aabfff700 1 -- 172.21.15.131:0/2267591284 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab400c9b0 con 0x7f8ac406b8c0 2023-12-26T21:53:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.850+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac41543e0 con 0x7f8ac406b8c0 2023-12-26T21:53:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.850+0000 7f8aabfff700 1 -- 172.21.15.131:0/2267591284 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400cd30 con 0x7f8ac406b8c0 2023-12-26T21:53:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.850+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac41549a0 con 0x7f8ac406b8c0 2023-12-26T21:53:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.851+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac40657e0 con 0x7f8ac406b8c0 2023-12-26T21:53:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.852+0000 7f8aabfff700 1 -- 172.21.15.131:0/2267591284 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 39) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f8ab4013680 con 0x7f8ac406b8c0 2023-12-26T21:53:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.853+0000 7f8aabfff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8ab0066dd0 0x7f8ab0069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.853+0000 7f8aabfff700 1 -- 172.21.15.131:0/2267591284 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f8ab4084b00 con 0x7f8ac406b8c0 2023-12-26T21:53:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.853+0000 7f8ac37fe700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8ab0066dd0 0x7f8ab0069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.854+0000 7f8ac37fe700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8ab0066dd0 0x7f8ab0069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40c4ab0 tx=0x7f8ab80091a0 comp rx=0 tx=0).ready entity=mgr.6894 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:26.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:26.857+0000 7f8aabfff700 1 -- 172.21.15.131:0/2267591284 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ab4086e60 con 0x7f8ac406b8c0 2023-12-26T21:53:27.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.103+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8ac4068bf0 con 0x7f8ac406b8c0 2023-12-26T21:53:27.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.588+0000 7f8aabfff700 1 -- 172.21.15.131:0/2267591284 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f8ab4052090 con 0x7f8ac406b8c0 2023-12-26T21:53:27.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.589+0000 7f8aabfff700 1 -- 172.21.15.131:0/2267591284 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v40) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8ab4070760 con 0x7f8ac406b8c0 2023-12-26T21:53:27.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8ab0066dd0 msgr2=0x7f8ab0069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:27.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8ab0066dd0 0x7f8ab0069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40c4ab0 tx=0x7f8ab80091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:27.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac406b8c0 msgr2=0x7f8ac4168a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:27.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac406b8c0 0x7f8ac4168a30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400ec80 tx=0x7f8ab4009120 comp rx=0 tx=0).stop 2023-12-26T21:53:27.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 shutdown_connections 2023-12-26T21:53:27.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac413c510 0x7f8ac4156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:27.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8ab0066dd0 0x7f8ab0069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:27.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac406b8c0 0x7f8ac4168a30 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:27.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 --2- 172.21.15.131:0/2267591284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40cc100 0x7f8ac41566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:27.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 >> 172.21.15.131:0/2267591284 conn(0x7f8ac40ce3e0 msgr2=0x7f8ac406cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:27.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 shutdown_connections 2023-12-26T21:53:27.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:27.592+0000 7f8aca9ff700 1 -- 172.21.15.131:0/2267591284 wait complete. 2023-12-26T21:53:27.611 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:53:27.611 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:53:27.709 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:53:27.812 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:53:27.812 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T21:53:27.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:53:27.812 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T21:53:27.887 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:53:27.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:27.887 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T21:53:27.931 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:53:28.036 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:28.036+0000 7f5bf1b951c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:53:28.045 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:28.046+0000 7f9a5bac81c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:53:28.140 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:28.140+0000 7f5bf1b951c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:53:28.154 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:28.155+0000 7f9a5bac81c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:53:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.164+0000 7f2ca3fff700 1 Processor -- start 2023-12-26T21:53:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.165+0000 7f2ca3fff700 1 -- start start 2023-12-26T21:53:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.166+0000 7f2ca3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2c9c0b0750 0x7f2c9c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.166+0000 7f2ca3fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c156940 con 0x7f2c9c0c9400 2023-12-26T21:53:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.166+0000 7f2ca3fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c0ae500 con 0x7f2c9c0cb320 2023-12-26T21:53:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.166+0000 7f2ca3fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c0c8cc0 con 0x7f2c9c0b0750 2023-12-26T21:53:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.166+0000 7f2ca259c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2c9c0c9400 0x7f2c9c0c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44652/0 (socket says 172.21.15.131:44652) 2023-12-26T21:53:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.166+0000 7f2ca259c700 1 -- 172.21.15.131:0/2896079939 learned_addr learned my addr 172.21.15.131:0/2896079939 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.166+0000 7f2ca159a700 1 --2- 172.21.15.131:0/2896079939 >> v2:172.21.15.190:3300/0 conn(0x7f2c9c0b0750 0x7f2c9c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca0d99700 1 -- 172.21.15.131:0/2896079939 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967739235 0 0) 0x7f2c9c156940 con 0x7f2c9c0c9400 2023-12-26T21:53:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca0d99700 1 -- 172.21.15.131:0/2896079939 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c88003570 con 0x7f2c9c0c9400 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca159a700 1 -- 172.21.15.131:0/2896079939 >> v1:172.21.15.190:6789/0 conn(0x7f2c9c0cb320 legacy=0x7f2c9c069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca0d99700 1 -- 172.21.15.131:0/2896079939 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484160353 0 0) 0x7f2c9c0ae500 con 0x7f2c9c0cb320 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca159a700 1 -- 172.21.15.131:0/2896079939 >> v1:172.21.15.131:6789/0 conn(0x7f2c9c0c9400 legacy=0x7f2c9c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca159a700 1 -- 172.21.15.131:0/2896079939 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0b0280 con 0x7f2c9c0b0750 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca159a700 1 --2- 172.21.15.131:0/2896079939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0b0750 0x7f2c9c0b0bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0098e0 tx=0x7f2c8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fa41668bce6a7df server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca0d99700 1 -- 172.21.15.131:0/2896079939 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00b640 con 0x7f2c9c0b0750 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca0d99700 1 -- 172.21.15.131:0/2896079939 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c00b7a0 con 0x7f2c9c0b0750 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.167+0000 7f2ca0d99700 1 -- 172.21.15.131:0/2896079939 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00bb40 con 0x7f2c9c0b0750 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.168+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2896079939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0b0750 msgr2=0x7f2c9c0b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.168+0000 7f2ca3fff700 1 --2- 172.21.15.131:0/2896079939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0b0750 0x7f2c9c0b0bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0098e0 tx=0x7f2c8c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.168+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2896079939 shutdown_connections 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.168+0000 7f2ca3fff700 1 --2- 172.21.15.131:0/2896079939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0b0750 0x7f2c9c0b0bf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.168+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2896079939 >> 172.21.15.131:0/2896079939 conn(0x7f2c9c068040 msgr2=0x7f2c9c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:28.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.168+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2896079939 shutdown_connections 2023-12-26T21:53:28.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.168+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2896079939 wait complete. 2023-12-26T21:53:28.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.170+0000 7f2ca3fff700 1 Processor -- start 2023-12-26T21:53:28.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.170+0000 7f2ca3fff700 1 -- start start 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.170+0000 7f2ca3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9c0b0750 0x7f2c9c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.170+0000 7f2ca3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0c9400 0x7f2c9c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c9c0cb320 0x7f2c9c156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca3fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c15b0d0 con 0x7f2c9c0cb320 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca3fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c15e0f0 con 0x7f2c9c0c9400 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca3fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c15d800 con 0x7f2c9c0b0750 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c9c0cb320 0x7f2c9c156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca159a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0c9400 0x7f2c9c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca159a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0c9400 0x7f2c9c15e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:36992/0 (socket says 172.21.15.131:36992) 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca159a700 1 -- 172.21.15.131:0/2408252424 learned_addr learned my addr 172.21.15.131:0/2408252424 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca1d9b700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9c0b0750 0x7f2c9c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca159a700 1 -- 172.21.15.131:0/2408252424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9c0b0750 msgr2=0x7f2c9c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca159a700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9c0b0750 0x7f2c9c15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca159a700 1 -- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c9c0cb320 msgr2=0x7f2c9c156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca159a700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c9c0cb320 0x7f2c9c156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca159a700 1 -- 172.21.15.131:0/2408252424 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c009420 con 0x7f2c9c0c9400 2023-12-26T21:53:28.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.171+0000 7f2ca259c700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c9c0cb320 0x7f2c9c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:28.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.172+0000 7f2ca159a700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0c9400 0x7f2c9c15e580 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0688f0 tx=0x7f2c8c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:28.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.172+0000 7f2c91ffb700 1 -- 172.21.15.131:0/2408252424 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c01f070 con 0x7f2c9c0c9400 2023-12-26T21:53:28.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.172+0000 7f2c91ffb700 1 -- 172.21.15.131:0/2408252424 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c002f70 con 0x7f2c9c0c9400 2023-12-26T21:53:28.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.172+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c15adf0 con 0x7f2c9c0c9400 2023-12-26T21:53:28.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.172+0000 7f2c91ffb700 1 -- 172.21.15.131:0/2408252424 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00b910 con 0x7f2c9c0c9400 2023-12-26T21:53:28.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.172+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c15bf70 con 0x7f2c9c0c9400 2023-12-26T21:53:28.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.173+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c15e280 con 0x7f2c9c0c9400 2023-12-26T21:53:28.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.174+0000 7f2c91ffb700 1 -- 172.21.15.131:0/2408252424 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f2c8c023070 con 0x7f2c9c0c9400 2023-12-26T21:53:28.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.175+0000 7f2c91ffb700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2c800670e0 0x7f2c80069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.175+0000 7f2c91ffb700 1 -- 172.21.15.131:0/2408252424 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f2c8c0861e0 con 0x7f2c9c0c9400 2023-12-26T21:53:28.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.175+0000 7f2ca1d9b700 1 -- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2c800670e0 msgr2=0x7f2c80069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:28.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.175+0000 7f2ca1d9b700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2c800670e0 0x7f2c80069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:28.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.179+0000 7f2c91ffb700 1 -- 172.21.15.131:0/2408252424 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c8c052f20 con 0x7f2c9c0c9400 2023-12-26T21:53:28.232 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:28.231+0000 7f5bf1b951c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:53:28.250 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:28.251+0000 7f9a5bac81c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:53:28.316 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:28.316+0000 7f5bf1b951c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:53:28.336 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:28.337+0000 7f9a5bac81c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:53:28.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.376+0000 7f2ca1d9b700 1 -- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2c800670e0 msgr2=0x7f2c80069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:28.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.376+0000 7f2ca1d9b700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2c800670e0 0x7f2c80069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:28.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.426+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c9c15d9c0 con 0x7f2c9c0c9400 2023-12-26T21:53:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.427+0000 7f2c91ffb700 1 -- 172.21.15.131:0/2408252424 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f2c9c15d9c0 con 0x7f2c9c0c9400 2023-12-26T21:53:28.426 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:28.426 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6454,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1671363867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1671363867},{"type":"v1","addr":"172.21.15.131:6834","nonce":1671363867}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3807864961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3807864961},{"type":"v1","addr":"172.21.15.190:6834","nonce":3807864961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/18678440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":18678440},{"type":"v1","addr":"172.21.15.131:6835","nonce":18678440}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:04.724845+0000","modified":"2023-12-26T21:53:08.284069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/168268053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":168268053},{"type":"v1","addr":"172.21.15.190:6835","nonce":168268053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-12-26T21:53:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2c800670e0 msgr2=0x7f2c80069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2c800670e0 0x7f2c80069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0c9400 msgr2=0x7f2c9c15e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0c9400 0x7f2c9c15e580 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0688f0 tx=0x7f2c8c017000 comp rx=0 tx=0).stop 2023-12-26T21:53:28.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 shutdown_connections 2023-12-26T21:53:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2c800670e0 0x7f2c80069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c9c0cb320 0x7f2c9c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9c0b0750 0x7f2c9c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 --2- 172.21.15.131:0/2408252424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c9c0c9400 0x7f2c9c15e580 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 >> 172.21.15.131:0/2408252424 conn(0x7f2c9c068040 msgr2=0x7f2c9c155ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 shutdown_connections 2023-12-26T21:53:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.429+0000 7f2ca3fff700 1 -- 172.21.15.131:0/2408252424 wait complete. 2023-12-26T21:53:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 42 2023-12-26T21:53:28.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:28.507 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:28.507+0000 7f5bf1b951c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:53:28.540 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:28.540+0000 7f9a5bac81c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:53:28.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.645+0000 7f914f5ff700 1 Processor -- start 2023-12-26T21:53:28.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.645+0000 7f914f5ff700 1 -- start start 2023-12-26T21:53:28.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.646+0000 7f914f5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9148159920 0x7f91480b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.646+0000 7f914f5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9148159cf0 0x7f91480b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.646+0000 7f914f5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91480c8bf0 con 0x7f91481554d0 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.646+0000 7f914f5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91480c8e60 con 0x7f9148159cf0 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.646+0000 7f914f5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91480c9140 con 0x7f9148159920 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914cb9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9148159cf0 0x7f91480b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914db9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f91481554d0 0x7f9148155930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44654/0 (socket says 172.21.15.131:44654) 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914db9c700 1 -- 172.21.15.131:0/2784714746 learned_addr learned my addr 172.21.15.131:0/2784714746 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914d39b700 1 --2- 172.21.15.131:0/2784714746 >> v2:172.21.15.190:3300/0 conn(0x7f9148159920 0x7f91480b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914cb9a700 1 -- 172.21.15.131:0/2784714746 >> v2:172.21.15.190:3300/0 conn(0x7f9148159920 msgr2=0x7f91480b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914cb9a700 1 --2- 172.21.15.131:0/2784714746 >> v2:172.21.15.190:3300/0 conn(0x7f9148159920 0x7f91480b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914cb9a700 1 -- 172.21.15.131:0/2784714746 >> v1:172.21.15.131:6789/0 conn(0x7f91481554d0 legacy=0x7f9148155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914cb9a700 1 -- 172.21.15.131:0/2784714746 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91480c9420 con 0x7f9148159cf0 2023-12-26T21:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914d39b700 1 --2- 172.21.15.131:0/2784714746 >> v2:172.21.15.190:3300/0 conn(0x7f9148159920 0x7f91480b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.647+0000 7f914cb9a700 1 --2- 172.21.15.131:0/2784714746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159cf0 0x7f91480b0620 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f91480642a0 tx=0x7f913800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e916095427286f81 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.648+0000 7f913ffff700 1 -- 172.21.15.131:0/2784714746 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800c970 con 0x7f9148159cf0 2023-12-26T21:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.648+0000 7f913ffff700 1 -- 172.21.15.131:0/2784714746 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913800cad0 con 0x7f9148159cf0 2023-12-26T21:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.648+0000 7f913ffff700 1 -- 172.21.15.131:0/2784714746 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800ce70 con 0x7f9148159cf0 2023-12-26T21:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.648+0000 7f914f5ff700 1 -- 172.21.15.131:0/2784714746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159cf0 msgr2=0x7f91480b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.648+0000 7f914f5ff700 1 --2- 172.21.15.131:0/2784714746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159cf0 0x7f91480b0620 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f91480642a0 tx=0x7f913800a330 comp rx=0 tx=0).stop 2023-12-26T21:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.648+0000 7f914f5ff700 1 -- 172.21.15.131:0/2784714746 shutdown_connections 2023-12-26T21:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.648+0000 7f914f5ff700 1 --2- 172.21.15.131:0/2784714746 >> v2:172.21.15.190:3300/0 conn(0x7f9148159920 0x7f91480b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.648+0000 7f914f5ff700 1 --2- 172.21.15.131:0/2784714746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159cf0 0x7f91480b0620 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.648+0000 7f914f5ff700 1 -- 172.21.15.131:0/2784714746 >> 172.21.15.131:0/2784714746 conn(0x7f9148068040 msgr2=0x7f9148069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.649+0000 7f914f5ff700 1 -- 172.21.15.131:0/2784714746 shutdown_connections 2023-12-26T21:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.649+0000 7f914f5ff700 1 -- 172.21.15.131:0/2784714746 wait complete. 2023-12-26T21:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.651+0000 7f914f5ff700 1 Processor -- start 2023-12-26T21:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.651+0000 7f914f5ff700 1 -- start start 2023-12-26T21:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914f5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91481554d0 0x7f91481306e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914f5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159920 0x7f9148158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914f5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9148159cf0 0x7f91481473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914f5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9148159120 con 0x7f9148159920 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914f5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481560d0 con 0x7f91481554d0 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914f5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9148156f30 con 0x7f9148159cf0 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914cb9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159920 0x7f9148158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914cb9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159920 0x7f9148158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49002/0 (socket says 172.21.15.131:49002) 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914cb9a700 1 -- 172.21.15.131:0/776056507 learned_addr learned my addr 172.21.15.131:0/776056507 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914d39b700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91481554d0 0x7f91481306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914db9c700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9148159cf0 0x7f91481473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914cb9a700 1 -- 172.21.15.131:0/776056507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9148159cf0 msgr2=0x7f91481473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914cb9a700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9148159cf0 0x7f91481473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914cb9a700 1 -- 172.21.15.131:0/776056507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91481554d0 msgr2=0x7f91481306e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914cb9a700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91481554d0 0x7f91481306e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914cb9a700 1 -- 172.21.15.131:0/776056507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9144009420 con 0x7f9148159920 2023-12-26T21:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.653+0000 7f914db9c700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9148159cf0 0x7f91481473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.654+0000 7f914cb9a700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159920 0x7f9148158be0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f91480688f0 tx=0x7f913801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.654+0000 7f913d7fa700 1 -- 172.21.15.131:0/776056507 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138019e90 con 0x7f9148159920 2023-12-26T21:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.654+0000 7f913d7fa700 1 -- 172.21.15.131:0/776056507 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9138010600 con 0x7f9148159920 2023-12-26T21:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.654+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913800a5d0 con 0x7f9148159920 2023-12-26T21:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.654+0000 7f913d7fa700 1 -- 172.21.15.131:0/776056507 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380109e0 con 0x7f9148159920 2023-12-26T21:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.654+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9148156600 con 0x7f9148159920 2023-12-26T21:53:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.655+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91480ae290 con 0x7f9148159920 2023-12-26T21:53:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.656+0000 7f913d7fa700 1 -- 172.21.15.131:0/776056507 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f9138010d50 con 0x7f9148159920 2023-12-26T21:53:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.657+0000 7f913d7fa700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f912c0670e0 0x7f912c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.658+0000 7f913d7fa700 1 -- 172.21.15.131:0/776056507 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f9138086240 con 0x7f9148159920 2023-12-26T21:53:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.658+0000 7f914d39b700 1 -- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f912c0670e0 msgr2=0x7f912c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.658+0000 7f914d39b700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f912c0670e0 0x7f912c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.662+0000 7f913d7fa700 1 -- 172.21.15.131:0/776056507 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9138052f00 con 0x7f9148159920 2023-12-26T21:53:28.678 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:28.678+0000 7f5bf1b951c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:53:28.738 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:28.738+0000 7f9a5bac81c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:53:28.762 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:28.762+0000 7f5bf1b951c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:53:28.822 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:28.823+0000 7f9a5bac81c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:53:28.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.859+0000 7f914d39b700 1 -- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f912c0670e0 msgr2=0x7f912c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:28.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.859+0000 7f914d39b700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f912c0670e0 0x7f912c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:28.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.882+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91480657e0 con 0x7f9148159920 2023-12-26T21:53:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.883+0000 7f913d7fa700 1 -- 172.21.15.131:0/776056507 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16370 (secure 0 0 0) 0x7f9138056520 con 0x7f9148159920 2023-12-26T21:53:28.883 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:28.883 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:25.566998+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:02.972636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:03.709325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6837/39288":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/3724250355":"2023-12-27T21:53:25.566963+0000","172.21.15.131:6836/39288":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/4100737385":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/1623991397":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/4146017606":"2023-12-27T21:53:25.566963+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f912c0670e0 msgr2=0x7f912c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f912c0670e0 0x7f912c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159920 msgr2=0x7f9148158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159920 0x7f9148158be0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f91480688f0 tx=0x7f913801a000 comp rx=0 tx=0).stop 2023-12-26T21:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 shutdown_connections 2023-12-26T21:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9148159cf0 0x7f91481473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91481554d0 0x7f91481306e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f912c0670e0 0x7f912c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 --2- 172.21.15.131:0/776056507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9148159920 0x7f9148158be0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:28.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 >> 172.21.15.131:0/776056507 conn(0x7f9148068040 msgr2=0x7f914806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:28.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 shutdown_connections 2023-12-26T21:53:28.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:28.886+0000 7f914f5ff700 1 -- 172.21.15.131:0/776056507 wait complete. 2023-12-26T21:53:28.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:53:02.972636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:53:03.709325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:53:28.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:53:28.910 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:28.910+0000 7f5bf1b951c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:53:28.977 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:28.977+0000 7f9a5bac81c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:53:28.999 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:28.999+0000 7f5bf1b951c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:53:29.071 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:29.072+0000 7f9a5bac81c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:53:29.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.102+0000 7f8be2dff700 1 Processor -- start 2023-12-26T21:53:29.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.102+0000 7f8be2dff700 1 -- start start 2023-12-26T21:53:29.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.103+0000 7f8be2dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8bdc06f4b0 0x7f8bdc157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.103+0000 7f8be2dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc157f90 0x7f8bdc0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.103+0000 7f8be2dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bdc06f900 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.103+0000 7f8be2dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bdc158af0 con 0x7f8bdc06f4b0 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.104+0000 7f8be2dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc15b5c0 con 0x7f8bdc157f90 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.104+0000 7f8bdbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8bdc06f4b0 0x7f8bdc157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.104+0000 7f8bdbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8bdc06f4b0 0x7f8bdc157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49006/0 (socket says 172.21.15.131:49006) 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.104+0000 7f8bdbfff700 1 -- 172.21.15.131:0/3692492182 learned_addr learned my addr 172.21.15.131:0/3692492182 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.104+0000 7f8be139c700 1 --2- 172.21.15.131:0/3692492182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc157f90 0x7f8bdc0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.104+0000 7f8bdbfff700 1 -- 172.21.15.131:0/3692492182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc157f90 msgr2=0x7f8bdc0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.104+0000 7f8bdbfff700 1 --2- 172.21.15.131:0/3692492182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc157f90 0x7f8bdc0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.104+0000 7f8bdbfff700 1 -- 172.21.15.131:0/3692492182 >> v1:172.21.15.131:6789/0 conn(0x7f8bdc06f0e0 legacy=0x7f8bdc15ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.104+0000 7f8bdbfff700 1 -- 172.21.15.131:0/3692492182 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc158530 con 0x7f8bdc06f4b0 2023-12-26T21:53:29.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.105+0000 7f8bdbfff700 1 --2- 172.21.15.131:0/3692492182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f4b0 0x7f8bdc157a50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80099b0 tx=0x7f8bc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12728f81f4b722e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:29.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.105+0000 7f8bdb7fe700 1 -- 172.21.15.131:0/3692492182 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800b860 con 0x7f8bdc06f4b0 2023-12-26T21:53:29.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.105+0000 7f8bdb7fe700 1 -- 172.21.15.131:0/3692492182 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc800b9c0 con 0x7f8bdc06f4b0 2023-12-26T21:53:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.105+0000 7f8bdb7fe700 1 -- 172.21.15.131:0/3692492182 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800bd60 con 0x7f8bdc06f4b0 2023-12-26T21:53:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.105+0000 7f8be2dff700 1 -- 172.21.15.131:0/3692492182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f4b0 msgr2=0x7f8bdc157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.105+0000 7f8be2dff700 1 --2- 172.21.15.131:0/3692492182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f4b0 0x7f8bdc157a50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80099b0 tx=0x7f8bc8009250 comp rx=0 tx=0).stop 2023-12-26T21:53:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.106+0000 7f8be2dff700 1 -- 172.21.15.131:0/3692492182 shutdown_connections 2023-12-26T21:53:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.106+0000 7f8be2dff700 1 --2- 172.21.15.131:0/3692492182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc157f90 0x7f8bdc0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.106+0000 7f8be2dff700 1 --2- 172.21.15.131:0/3692492182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f4b0 0x7f8bdc157a50 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.106+0000 7f8be2dff700 1 -- 172.21.15.131:0/3692492182 >> 172.21.15.131:0/3692492182 conn(0x7f8bdc0686c0 msgr2=0x7f8bdc068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.106+0000 7f8be2dff700 1 -- 172.21.15.131:0/3692492182 shutdown_connections 2023-12-26T21:53:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.106+0000 7f8be2dff700 1 -- 172.21.15.131:0/3692492182 wait complete. 2023-12-26T21:53:29.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.108+0000 7f8be2dff700 1 Processor -- start 2023-12-26T21:53:29.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.108+0000 7f8be2dff700 1 -- start start 2023-12-26T21:53:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.109+0000 7f8be2dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f0e0 0x7f8bdc0af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.109+0000 7f8be2dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc06f4b0 0x7f8bdc156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.109+0000 7f8be2dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bdc157f90 0x7f8bdc156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.109+0000 7f8be2dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc16a220 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.109+0000 7f8be2dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc16a500 con 0x7f8bdc157f90 2023-12-26T21:53:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.109+0000 7f8be2dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc16a7e0 con 0x7f8bdc06f4b0 2023-12-26T21:53:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.109+0000 7f8be0b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f0e0 0x7f8bdc0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.109+0000 7f8be0b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f0e0 0x7f8bdc0af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49010/0 (socket says 172.21.15.131:49010) 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.109+0000 7f8be0b9b700 1 -- 172.21.15.131:0/2800913027 learned_addr learned my addr 172.21.15.131:0/2800913027 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8be139c700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bdc157f90 0x7f8bdc156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8bdbfff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc06f4b0 0x7f8bdc156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8be0b9b700 1 -- 172.21.15.131:0/2800913027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc06f4b0 msgr2=0x7f8bdc156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8be0b9b700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc06f4b0 0x7f8bdc156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8be0b9b700 1 -- 172.21.15.131:0/2800913027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bdc157f90 msgr2=0x7f8bdc156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8be0b9b700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bdc157f90 0x7f8bdc156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8be0b9b700 1 -- 172.21.15.131:0/2800913027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc80094f0 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8bdbfff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc06f4b0 0x7f8bdc156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8be0b9b700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f0e0 0x7f8bdc0af540 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c590 tx=0x7f8bd000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8bd8ff9700 1 -- 172.21.15.131:0/2800913027 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011590 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8bd8ff9700 1 -- 172.21.15.131:0/2800913027 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd00116f0 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8bd8ff9700 1 -- 172.21.15.131:0/2800913027 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011ab0 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.110+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc16aac0 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.111+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc0afa80 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.112+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc0afc70 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.112+0000 7f8bd8ff9700 1 -- 172.21.15.131:0/2800913027 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f8bd001b3b0 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.113+0000 7f8bd8ff9700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8bc00670e0 0x7f8bc0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.113+0000 7f8bd8ff9700 1 -- 172.21.15.131:0/2800913027 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f8bd0084090 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.113+0000 7f8bdbfff700 1 -- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8bc00670e0 msgr2=0x7f8bc0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.113+0000 7f8bdbfff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8bc00670e0 0x7f8bc0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:29.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.117+0000 7f8bd8ff9700 1 -- 172.21.15.131:0/2800913027 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bdc0afc70 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.182 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:29.181+0000 7f5bf1b951c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:53:29.261 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:29.261+0000 7f9a5bac81c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:53:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.314+0000 7f8bdbfff700 1 -- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8bc00670e0 msgr2=0x7f8bc0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:29.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.314+0000 7f8bdbfff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8bc00670e0 0x7f8bc0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:29.337 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:29.336+0000 7f5bf1b951c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:53:29.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.356+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bdc0afc70 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.356+0000 7f8bd8ff9700 1 -- 172.21.15.131:0/2800913027 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bdc0afc70 con 0x7f8bdc06f0e0 2023-12-26T21:53:29.356 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8bc00670e0 msgr2=0x7f8bc0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8bc00670e0 0x7f8bc0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f0e0 msgr2=0x7f8bdc0af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f0e0 0x7f8bdc0af540 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c590 tx=0x7f8bd000c950 comp rx=0 tx=0).stop 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 shutdown_connections 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bdc157f90 0x7f8bdc156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8bc00670e0 0x7f8bc0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc06f0e0 0x7f8bdc0af540 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 --2- 172.21.15.131:0/2800913027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bdc06f4b0 0x7f8bdc156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 >> 172.21.15.131:0/2800913027 conn(0x7f8bdc0686c0 msgr2=0x7f8bdc15b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 shutdown_connections 2023-12-26T21:53:29.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.359+0000 7f8be2dff700 1 -- 172.21.15.131:0/2800913027 wait complete. 2023-12-26T21:53:29.377 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:53:29.419 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:29.419+0000 7f9a5bac81c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:53:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.575+0000 7f72405ff700 1 Processor -- start 2023-12-26T21:53:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.575+0000 7f72405ff700 1 -- start start 2023-12-26T21:53:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.576+0000 7f72405ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7238158030 0x7f7238158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.576+0000 7f72405ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f72380c9130 0x7f72380cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.576+0000 7f72405ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723815ba70 con 0x7f723806b8c0 2023-12-26T21:53:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.576+0000 7f72405ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723815bf00 con 0x7f7238158030 2023-12-26T21:53:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.576+0000 7f72405ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723806c2a0 con 0x7f72380c9130 2023-12-26T21:53:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.576+0000 7f723e39b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7238158030 0x7f7238158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.576+0000 7f723e39b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7238158030 0x7f7238158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49024/0 (socket says 172.21.15.131:49024) 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.576+0000 7f723e39b700 1 -- 172.21.15.131:0/2108518750 learned_addr learned my addr 172.21.15.131:0/2108518750 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.577+0000 7f723eb9c700 1 --2- 172.21.15.131:0/2108518750 >> v2:172.21.15.190:3300/0 conn(0x7f72380c9130 0x7f72380cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.577+0000 7f723e39b700 1 -- 172.21.15.131:0/2108518750 >> v2:172.21.15.190:3300/0 conn(0x7f72380c9130 msgr2=0x7f72380cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.577+0000 7f723e39b700 1 --2- 172.21.15.131:0/2108518750 >> v2:172.21.15.190:3300/0 conn(0x7f72380c9130 0x7f72380cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.577+0000 7f723e39b700 1 -- 172.21.15.131:0/2108518750 >> v1:172.21.15.131:6789/0 conn(0x7f723806b8c0 legacy=0x7f72380c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.577+0000 7f723e39b700 1 -- 172.21.15.131:0/2108518750 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723815b140 con 0x7f7238158030 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.577+0000 7f723e39b700 1 --2- 172.21.15.131:0/2108518750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 0x7f7238158400 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7238152080 tx=0x7f7234009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752ded1a5f9cd99a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.577+0000 7f723d399700 1 -- 172.21.15.131:0/2108518750 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234005c10 con 0x7f7238158030 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f723d399700 1 -- 172.21.15.131:0/2108518750 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234005d70 con 0x7f7238158030 2023-12-26T21:53:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f723d399700 1 -- 172.21.15.131:0/2108518750 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234006260 con 0x7f7238158030 2023-12-26T21:53:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f72405ff700 1 -- 172.21.15.131:0/2108518750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 msgr2=0x7f7238158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f72405ff700 1 --2- 172.21.15.131:0/2108518750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 0x7f7238158400 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7238152080 tx=0x7f7234009220 comp rx=0 tx=0).stop 2023-12-26T21:53:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f72405ff700 1 -- 172.21.15.131:0/2108518750 shutdown_connections 2023-12-26T21:53:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f72405ff700 1 --2- 172.21.15.131:0/2108518750 >> v2:172.21.15.190:3300/0 conn(0x7f72380c9130 0x7f72380cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f72405ff700 1 --2- 172.21.15.131:0/2108518750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 0x7f7238158400 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f72405ff700 1 -- 172.21.15.131:0/2108518750 >> 172.21.15.131:0/2108518750 conn(0x7f72380ce3e0 msgr2=0x7f72380ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f72405ff700 1 -- 172.21.15.131:0/2108518750 shutdown_connections 2023-12-26T21:53:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.578+0000 7f72405ff700 1 -- 172.21.15.131:0/2108518750 wait complete. 2023-12-26T21:53:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.580+0000 7f72405ff700 1 Processor -- start 2023-12-26T21:53:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.580+0000 7f72405ff700 1 -- start start 2023-12-26T21:53:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f72405ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723806b8c0 0x7f72380c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f72405ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72380c9130 0x7f72380c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f72405ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 0x7f72380c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f72405ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c7d40 con 0x7f7238158030 2023-12-26T21:53:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f72405ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c7130 con 0x7f723806b8c0 2023-12-26T21:53:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f72405ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c7410 con 0x7f72380c9130 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f723eb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 0x7f72380c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f723eb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 0x7f72380c3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49026/0 (socket says 172.21.15.131:49026) 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f723eb9c700 1 -- 172.21.15.131:0/941301733 learned_addr learned my addr 172.21.15.131:0/941301733 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f723e39b700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723806b8c0 0x7f72380c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.581+0000 7f723db9a700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72380c9130 0x7f72380c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f723e39b700 1 -- 172.21.15.131:0/941301733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72380c9130 msgr2=0x7f72380c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f723e39b700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72380c9130 0x7f72380c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f723e39b700 1 -- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 msgr2=0x7f72380c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f723e39b700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 0x7f72380c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f723e39b700 1 -- 172.21.15.131:0/941301733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7234009480 con 0x7f723806b8c0 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f723eb9c700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 0x7f72380c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f723e39b700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723806b8c0 0x7f72380c6a70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7234015000 tx=0x7f7234016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f722e7fc700 1 -- 172.21.15.131:0/941301733 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723400b6c0 con 0x7f723806b8c0 2023-12-26T21:53:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f722e7fc700 1 -- 172.21.15.131:0/941301733 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234020070 con 0x7f723806b8c0 2023-12-26T21:53:29.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72380c76f0 con 0x7f723806b8c0 2023-12-26T21:53:29.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f722e7fc700 1 -- 172.21.15.131:0/941301733 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723400ba70 con 0x7f723806b8c0 2023-12-26T21:53:29.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.582+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72380c36c0 con 0x7f723806b8c0 2023-12-26T21:53:29.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.583+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72380657e0 con 0x7f723806b8c0 2023-12-26T21:53:29.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.584+0000 7f722e7fc700 1 -- 172.21.15.131:0/941301733 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f7234022030 con 0x7f723806b8c0 2023-12-26T21:53:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.585+0000 7f722e7fc700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f721c0670e0 0x7f721c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.585+0000 7f722e7fc700 1 -- 172.21.15.131:0/941301733 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f7234086600 con 0x7f723806b8c0 2023-12-26T21:53:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.585+0000 7f723db9a700 1 -- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f721c0670e0 msgr2=0x7f721c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.585+0000 7f723db9a700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f721c0670e0 0x7f721c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:29.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.590+0000 7f722e7fc700 1 -- 172.21.15.131:0/941301733 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72340532c0 con 0x7f723806b8c0 2023-12-26T21:53:29.719 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:29.719+0000 7f5bf1b951c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:53:29.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.786+0000 7f723db9a700 1 -- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f721c0670e0 msgr2=0x7f721c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:29.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.786+0000 7f723db9a700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f721c0670e0 0x7f721c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:29.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.790+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f723815b420 con 0x7f723806b8c0 2023-12-26T21:53:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.791+0000 7f722e7fc700 1 -- 172.21.15.131:0/941301733 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f723400bea0 con 0x7f723806b8c0 2023-12-26T21:53:29.790 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f721c0670e0 msgr2=0x7f721c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f721c0670e0 0x7f721c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723806b8c0 msgr2=0x7f72380c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723806b8c0 0x7f72380c6a70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7234015000 tx=0x7f7234016000 comp rx=0 tx=0).stop 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 shutdown_connections 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f721c0670e0 0x7f721c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7238158030 0x7f72380c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723806b8c0 0x7f72380c6a70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 --2- 172.21.15.131:0/941301733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72380c9130 0x7f72380c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 >> 172.21.15.131:0/941301733 conn(0x7f72380ce3e0 msgr2=0x7f72380cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:29.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 shutdown_connections 2023-12-26T21:53:29.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:29.793+0000 7f72405ff700 1 -- 172.21.15.131:0/941301733 wait complete. 2023-12-26T21:53:29.812 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:53:29.826 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:29.826+0000 7f9a5bac81c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:53:29.984 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:29.984+0000 7f5bf1b951c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:53:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.023+0000 7fae0e9ff700 1 Processor -- start 2023-12-26T21:53:30.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.024+0000 7fae0e9ff700 1 -- start start 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae0e9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159560 0x7fae0815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae0e9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fae08159930 0x7fae080b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae0e9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fae0806a1a0 0x7fae08068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae0e9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae08068a80 con 0x7fae0806a1a0 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae0e9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae080cac50 con 0x7fae08159930 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae0e9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0815da00 con 0x7fae08159560 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae0cf9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fae0806a1a0 0x7fae08068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae07fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159560 0x7fae0815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fadfffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fae08159930 0x7fae080b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae0cf9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fae0806a1a0 0x7fae08068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49040/0 (socket says 172.21.15.131:49040) 2023-12-26T21:53:30.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.025+0000 7fae0cf9c700 1 -- 172.21.15.131:0/2954704079 learned_addr learned my addr 172.21.15.131:0/2954704079 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fae0cf9c700 1 -- 172.21.15.131:0/2954704079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159560 msgr2=0x7fae0815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fae0cf9c700 1 --2- 172.21.15.131:0/2954704079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159560 0x7fae0815d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fae0cf9c700 1 -- 172.21.15.131:0/2954704079 >> v2:172.21.15.190:3300/0 conn(0x7fae08159930 msgr2=0x7fae080b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fae0cf9c700 1 --2- 172.21.15.131:0/2954704079 >> v2:172.21.15.190:3300/0 conn(0x7fae08159930 0x7fae080b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fae0cf9c700 1 -- 172.21.15.131:0/2954704079 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0815dce0 con 0x7fae0806a1a0 2023-12-26T21:53:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fadfffff700 1 --2- 172.21.15.131:0/2954704079 >> v2:172.21.15.190:3300/0 conn(0x7fae08159930 0x7fae080b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fae0cf9c700 1 --2- 172.21.15.131:0/2954704079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae0806a1a0 0x7fae08068f10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fae0806e810 tx=0x7fae00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b01625e03c30e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fae077fe700 1 -- 172.21.15.131:0/2954704079 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0000e070 con 0x7fae0806a1a0 2023-12-26T21:53:30.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fae077fe700 1 -- 172.21.15.131:0/2954704079 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0000b970 con 0x7fae0806a1a0 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.026+0000 7fae077fe700 1 -- 172.21.15.131:0/2954704079 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000092e0 con 0x7fae0806a1a0 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.027+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2954704079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae0806a1a0 msgr2=0x7fae08068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.027+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2954704079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae0806a1a0 0x7fae08068f10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fae0806e810 tx=0x7fae00009840 comp rx=0 tx=0).stop 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.027+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2954704079 shutdown_connections 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.027+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2954704079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae0806a1a0 0x7fae08068f10 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.027+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2954704079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159560 0x7fae0815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.027+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2954704079 >> v2:172.21.15.190:3300/0 conn(0x7fae08159930 0x7fae080b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.027+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2954704079 >> 172.21.15.131:0/2954704079 conn(0x7fae080ce3d0 msgr2=0x7fae080ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.027+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2954704079 shutdown_connections 2023-12-26T21:53:30.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.027+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2954704079 wait complete. 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.028+0000 7fae0e9ff700 1 Processor -- start 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.029+0000 7fae0e9ff700 1 -- start start 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.029+0000 7fae0e9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae0806a1a0 0x7fae08156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.029+0000 7fae0e9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae08159560 0x7fae080cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.029+0000 7fae0e9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159930 0x7fae080ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.029+0000 7fae0e9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae080c2520 con 0x7fae08159560 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.029+0000 7fae0e9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae080c4ff0 con 0x7fae0806a1a0 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.029+0000 7fae0e9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae080c4d10 con 0x7fae08159930 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fadfffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae08159560 0x7fae080cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fadfffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae08159560 0x7fae080cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49056/0 (socket says 172.21.15.131:49056) 2023-12-26T21:53:30.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fadfffff700 1 -- 172.21.15.131:0/2871095331 learned_addr learned my addr 172.21.15.131:0/2871095331 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:30.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fae07fff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae0806a1a0 0x7fae08156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fae0cf9c700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159930 0x7fae080ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fadfffff700 1 -- 172.21.15.131:0/2871095331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159930 msgr2=0x7fae080ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fadfffff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159930 0x7fae080ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fadfffff700 1 -- 172.21.15.131:0/2871095331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae0806a1a0 msgr2=0x7fae08156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fadfffff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae0806a1a0 0x7fae08156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fadfffff700 1 -- 172.21.15.131:0/2871095331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf0009420 con 0x7fae08159560 2023-12-26T21:53:30.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.030+0000 7fadfffff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae08159560 0x7fae080cc890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fadf000ec80 tx=0x7fadf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:30.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.031+0000 7fae04ff9700 1 -- 172.21.15.131:0/2871095331 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000c920 con 0x7fae08159560 2023-12-26T21:53:30.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.031+0000 7fae04ff9700 1 -- 172.21.15.131:0/2871095331 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf000ca80 con 0x7fae08159560 2023-12-26T21:53:30.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.031+0000 7fae04ff9700 1 -- 172.21.15.131:0/2871095331 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000ce00 con 0x7fae08159560 2023-12-26T21:53:30.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.031+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae00009ae0 con 0x7fae08159560 2023-12-26T21:53:30.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.031+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae080c21b0 con 0x7fae08159560 2023-12-26T21:53:30.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.032+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae080c2690 con 0x7fae08159560 2023-12-26T21:53:30.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.033+0000 7fae04ff9700 1 -- 172.21.15.131:0/2871095331 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fadf0013680 con 0x7fae08159560 2023-12-26T21:53:30.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.034+0000 7fae04ff9700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fade80670e0 0x7fade8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.034+0000 7fae04ff9700 1 -- 172.21.15.131:0/2871095331 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fadf0084ec0 con 0x7fae08159560 2023-12-26T21:53:30.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.034+0000 7fae07fff700 1 -- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fade80670e0 msgr2=0x7fade8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:30.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.034+0000 7fae07fff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fade80670e0 0x7fade8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:30.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.038+0000 7fae04ff9700 1 -- 172.21.15.131:0/2871095331 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadf0051c20 con 0x7fae08159560 2023-12-26T21:53:30.110 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:30.111+0000 7f9a5bac81c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:53:30.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.235+0000 7fae07fff700 1 -- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fade80670e0 msgr2=0x7fade8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:30.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.235+0000 7fae07fff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fade80670e0 0x7fade8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:30.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.239+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae080657e0 con 0x7fae08159560 2023-12-26T21:53:30.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.240+0000 7fae04ff9700 1 -- 172.21.15.131:0/2871095331 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadf0055240 con 0x7fae08159560 2023-12-26T21:53:30.239 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fade80670e0 msgr2=0x7fade8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fade80670e0 0x7fade8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae08159560 msgr2=0x7fae080cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae08159560 0x7fae080cc890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fadf000ec80 tx=0x7fadf00091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 shutdown_connections 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae08159930 0x7fae080ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae0806a1a0 0x7fae08156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fade80670e0 0x7fade8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 --2- 172.21.15.131:0/2871095331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae08159560 0x7fae080cc890 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 >> 172.21.15.131:0/2871095331 conn(0x7fae080ce3d0 msgr2=0x7fae08157950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:30.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 shutdown_connections 2023-12-26T21:53:30.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.242+0000 7fae0e9ff700 1 -- 172.21.15.131:0/2871095331 wait complete. 2023-12-26T21:53:30.262 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:53:30.263 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:30.263+0000 7f5bf1b951c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:53:30.352 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:30.352+0000 7f5bf1b951c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:53:30.401 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:30.402+0000 7f9a5bac81c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:53:30.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.458+0000 7fd565fff700 1 Processor -- start 2023-12-26T21:53:30.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.458+0000 7fd565fff700 1 -- start start 2023-12-26T21:53:30.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.459+0000 7fd565fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5600b0750 0x7fd5600b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.459+0000 7fd565fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5600c9400 0x7fd5600c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.459+0000 7fd565fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd560156940 con 0x7fd5600cb320 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.459+0000 7fd565fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600c8cc0 con 0x7fd5600c9400 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.459+0000 7fd565fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600ad700 con 0x7fd5600b0750 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.460+0000 7fd55ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5600c9400 0x7fd5600c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.460+0000 7fd55ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5600c9400 0x7fd5600c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49068/0 (socket says 172.21.15.131:49068) 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.460+0000 7fd55ffff700 1 -- 172.21.15.131:0/3376376023 learned_addr learned my addr 172.21.15.131:0/3376376023 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.460+0000 7fd55effd700 1 --2- 172.21.15.131:0/3376376023 >> v2:172.21.15.190:3300/0 conn(0x7fd5600b0750 0x7fd5600b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.460+0000 7fd55ffff700 1 -- 172.21.15.131:0/3376376023 >> v2:172.21.15.190:3300/0 conn(0x7fd5600b0750 msgr2=0x7fd5600b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.460+0000 7fd55ffff700 1 --2- 172.21.15.131:0/3376376023 >> v2:172.21.15.190:3300/0 conn(0x7fd5600b0750 0x7fd5600b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.460+0000 7fd55ffff700 1 -- 172.21.15.131:0/3376376023 >> v1:172.21.15.131:6789/0 conn(0x7fd5600cb320 legacy=0x7fd560069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.460+0000 7fd55ffff700 1 -- 172.21.15.131:0/3376376023 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600ae500 con 0x7fd5600c9400 2023-12-26T21:53:30.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.460+0000 7fd55effd700 1 --2- 172.21.15.131:0/3376376023 >> v2:172.21.15.190:3300/0 conn(0x7fd5600b0750 0x7fd5600b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.461+0000 7fd55ffff700 1 --2- 172.21.15.131:0/3376376023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600c9400 0x7fd5600c9840 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd560064700 tx=0x7fd558009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffa48d2d19f75a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.461+0000 7fd55e7fc700 1 -- 172.21.15.131:0/3376376023 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800e070 con 0x7fd5600c9400 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.461+0000 7fd55e7fc700 1 -- 172.21.15.131:0/3376376023 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55800b990 con 0x7fd5600c9400 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.461+0000 7fd55e7fc700 1 -- 172.21.15.131:0/3376376023 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580092e0 con 0x7fd5600c9400 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.461+0000 7fd565fff700 1 -- 172.21.15.131:0/3376376023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600c9400 msgr2=0x7fd5600c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.461+0000 7fd565fff700 1 --2- 172.21.15.131:0/3376376023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600c9400 0x7fd5600c9840 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd560064700 tx=0x7fd558009840 comp rx=0 tx=0).stop 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.462+0000 7fd565fff700 1 -- 172.21.15.131:0/3376376023 shutdown_connections 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.462+0000 7fd565fff700 1 --2- 172.21.15.131:0/3376376023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600c9400 0x7fd5600c9840 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.462+0000 7fd565fff700 1 --2- 172.21.15.131:0/3376376023 >> v2:172.21.15.190:3300/0 conn(0x7fd5600b0750 0x7fd5600b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.462+0000 7fd565fff700 1 -- 172.21.15.131:0/3376376023 >> 172.21.15.131:0/3376376023 conn(0x7fd560068040 msgr2=0x7fd560069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:30.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.462+0000 7fd565fff700 1 -- 172.21.15.131:0/3376376023 shutdown_connections 2023-12-26T21:53:30.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.462+0000 7fd565fff700 1 -- 172.21.15.131:0/3376376023 wait complete. 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.463+0000 7fd565fff700 1 Processor -- start 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.463+0000 7fd565fff700 1 -- start start 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd565fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600b0750 0x7fd56015e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd565fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600c9400 0x7fd560147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd565fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600cb320 0x7fd560157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd565fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd560158b90 con 0x7fd5600b0750 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd565fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5601582f0 con 0x7fd5600c9400 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd565fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5601585d0 con 0x7fd5600cb320 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600b0750 0x7fd56015e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600b0750 0x7fd56015e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49078/0 (socket says 172.21.15.131:49078) 2023-12-26T21:53:30.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd55f7fe700 1 -- 172.21.15.131:0/1429488259 learned_addr learned my addr 172.21.15.131:0/1429488259 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:30.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.464+0000 7fd55effd700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600c9400 0x7fd560147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd55ffff700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600cb320 0x7fd560157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd55f7fe700 1 -- 172.21.15.131:0/1429488259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600cb320 msgr2=0x7fd560157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd55f7fe700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600cb320 0x7fd560157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd55f7fe700 1 -- 172.21.15.131:0/1429488259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600c9400 msgr2=0x7fd560147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd55f7fe700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600c9400 0x7fd560147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd55f7fe700 1 -- 172.21.15.131:0/1429488259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd554009420 con 0x7fd5600b0750 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd55effd700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600c9400 0x7fd560147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd55f7fe700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600b0750 0x7fd56015e0a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c4e0 tx=0x7fd55000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd5477fe700 1 -- 172.21.15.131:0/1429488259 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550011590 con 0x7fd5600b0750 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd5477fe700 1 -- 172.21.15.131:0/1429488259 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd550011710 con 0x7fd5600b0750 2023-12-26T21:53:30.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd558009ae0 con 0x7fd5600b0750 2023-12-26T21:53:30.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.465+0000 7fd5477fe700 1 -- 172.21.15.131:0/1429488259 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550011ad0 con 0x7fd5600b0750 2023-12-26T21:53:30.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.466+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5601597a0 con 0x7fd5600b0750 2023-12-26T21:53:30.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.467+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5600657e0 con 0x7fd5600b0750 2023-12-26T21:53:30.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.468+0000 7fd5477fe700 1 -- 172.21.15.131:0/1429488259 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fd55001b3b0 con 0x7fd5600b0750 2023-12-26T21:53:30.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.468+0000 7fd5477fe700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd54c0670e0 0x7fd54c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.469+0000 7fd5477fe700 1 -- 172.21.15.131:0/1429488259 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fd550083ee0 con 0x7fd5600b0750 2023-12-26T21:53:30.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.469+0000 7fd55effd700 1 -- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd54c0670e0 msgr2=0x7fd54c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:30.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.469+0000 7fd55effd700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd54c0670e0 0x7fd54c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:30.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.472+0000 7fd5477fe700 1 -- 172.21.15.131:0/1429488259 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd550050c40 con 0x7fd5600b0750 2023-12-26T21:53:30.498 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:30.499+0000 7f9a5bac81c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:53:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.670+0000 7fd55effd700 1 -- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd54c0670e0 msgr2=0x7fd54c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.670+0000 7fd55effd700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd54c0670e0 0x7fd54c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:30.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.718+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd560159990 con 0x7fd5600b0750 2023-12-26T21:53:30.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.718+0000 7fd5477fe700 1 -- 172.21.15.131:0/1429488259 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fd550017070 con 0x7fd5600b0750 2023-12-26T21:53:30.718 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:30.718 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6454,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1671363867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1671363867},{"type":"v1","addr":"172.21.15.131:6834","nonce":1671363867}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3807864961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3807864961},{"type":"v1","addr":"172.21.15.190:6834","nonce":3807864961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/18678440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":18678440},{"type":"v1","addr":"172.21.15.131:6835","nonce":18678440}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:04.724845+0000","modified":"2023-12-26T21:53:08.284069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/168268053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":168268053},{"type":"v1","addr":"172.21.15.190:6835","nonce":168268053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd54c0670e0 msgr2=0x7fd54c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd54c0670e0 0x7fd54c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600b0750 msgr2=0x7fd56015e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600b0750 0x7fd56015e0a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c4e0 tx=0x7fd55000c7f0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 shutdown_connections 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600cb320 0x7fd560157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd54c0670e0 0x7fd54c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5600b0750 0x7fd56015e0a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 --2- 172.21.15.131:0/1429488259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600c9400 0x7fd560147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 >> 172.21.15.131:0/1429488259 conn(0x7fd560068040 msgr2=0x7fd5600c53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 shutdown_connections 2023-12-26T21:53:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.721+0000 7fd565fff700 1 -- 172.21.15.131:0/1429488259 wait complete. 2023-12-26T21:53:30.722 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 42 2023-12-26T21:53:30.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:30.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.931+0000 7f6f22bff700 1 Processor -- start 2023-12-26T21:53:30.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.931+0000 7f6f22bff700 1 -- start start 2023-12-26T21:53:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.932+0000 7f6f22bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6f1c159a70 0x7f6f1c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.932+0000 7f6f22bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6f1c0b2a90 0x7f6f1c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.932+0000 7f6f22bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0afcd0 con 0x7f6f1c1596a0 2023-12-26T21:53:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.932+0000 7f6f22bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1c0b3e70 con 0x7f6f1c159a70 2023-12-26T21:53:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.932+0000 7f6f22bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1c0b0f20 con 0x7f6f1c0b2a90 2023-12-26T21:53:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.932+0000 7f6f1bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6f1c159a70 0x7f6f1c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.932+0000 7f6f1bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6f1c159a70 0x7f6f1c0b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49088/0 (socket says 172.21.15.131:49088) 2023-12-26T21:53:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.932+0000 7f6f2119c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6f1c0b2a90 0x7f6f1c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.932+0000 7f6f1bfff700 1 -- 172.21.15.131:0/2421296441 learned_addr learned my addr 172.21.15.131:0/2421296441 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.933+0000 7f6f1bfff700 1 -- 172.21.15.131:0/2421296441 >> v2:172.21.15.190:3300/0 conn(0x7f6f1c0b2a90 msgr2=0x7f6f1c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.933+0000 7f6f1bfff700 1 --2- 172.21.15.131:0/2421296441 >> v2:172.21.15.190:3300/0 conn(0x7f6f1c0b2a90 0x7f6f1c0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.933+0000 7f6f1bfff700 1 -- 172.21.15.131:0/2421296441 >> v1:172.21.15.190:6789/0 conn(0x7f6f1c1596a0 legacy=0x7f6f1c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.933+0000 7f6f1bfff700 1 -- 172.21.15.131:0/2421296441 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0b3540 con 0x7f6f1c159a70 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.933+0000 7f6f1bfff700 1 --2- 172.21.15.131:0/2421296441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 0x7f6f1c0b0160 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0b66b0 tx=0x7f6f08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6efa5e8032e75e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.933+0000 7f6f1b7fe700 1 -- 172.21.15.131:0/2421296441 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08009e70 con 0x7f6f1c159a70 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.933+0000 7f6f1b7fe700 1 -- 172.21.15.131:0/2421296441 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0800b860 con 0x7f6f1c159a70 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.933+0000 7f6f1b7fe700 1 -- 172.21.15.131:0/2421296441 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800bc00 con 0x7f6f1c159a70 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.934+0000 7f6f22bff700 1 -- 172.21.15.131:0/2421296441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 msgr2=0x7f6f1c0b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.934+0000 7f6f22bff700 1 --2- 172.21.15.131:0/2421296441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 0x7f6f1c0b0160 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0b66b0 tx=0x7f6f08009250 comp rx=0 tx=0).stop 2023-12-26T21:53:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.934+0000 7f6f22bff700 1 -- 172.21.15.131:0/2421296441 shutdown_connections 2023-12-26T21:53:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.934+0000 7f6f22bff700 1 --2- 172.21.15.131:0/2421296441 >> v2:172.21.15.190:3300/0 conn(0x7f6f1c0b2a90 0x7f6f1c0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.934+0000 7f6f22bff700 1 --2- 172.21.15.131:0/2421296441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 0x7f6f1c0b0160 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.934+0000 7f6f22bff700 1 -- 172.21.15.131:0/2421296441 >> 172.21.15.131:0/2421296441 conn(0x7f6f1c06ece0 msgr2=0x7f6f1c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.934+0000 7f6f22bff700 1 -- 172.21.15.131:0/2421296441 shutdown_connections 2023-12-26T21:53:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.934+0000 7f6f22bff700 1 -- 172.21.15.131:0/2421296441 wait complete. 2023-12-26T21:53:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.936+0000 7f6f22bff700 1 Processor -- start 2023-12-26T21:53:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.936+0000 7f6f22bff700 1 -- start start 2023-12-26T21:53:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f22bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c0b2a90 0x7f6f1c1478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f22bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f1c1596a0 0x7f6f1c0c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f22bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 0x7f6f1c0c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f22bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c159ef0 con 0x7f6f1c159a70 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f22bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0c9a90 con 0x7f6f1c0b2a90 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f22bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0c9d70 con 0x7f6f1c1596a0 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f2119c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 0x7f6f1c0c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f2119c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 0x7f6f1c0c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49092/0 (socket says 172.21.15.131:49092) 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f2119c700 1 -- 172.21.15.131:0/1879991022 learned_addr learned my addr 172.21.15.131:0/1879991022 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f2099b700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c0b2a90 0x7f6f1c1478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f1bfff700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f1c1596a0 0x7f6f1c0c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f2119c700 1 -- 172.21.15.131:0/1879991022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f1c1596a0 msgr2=0x7f6f1c0c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f2119c700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f1c1596a0 0x7f6f1c0c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f2119c700 1 -- 172.21.15.131:0/1879991022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c0b2a90 msgr2=0x7f6f1c1478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f2119c700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c0b2a90 0x7f6f1c1478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.937+0000 7f6f2119c700 1 -- 172.21.15.131:0/1879991022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f080094f0 con 0x7f6f1c159a70 2023-12-26T21:53:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.938+0000 7f6f2119c700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 0x7f6f1c0c9430 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009e90 tx=0x7f6f1400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.938+0000 7f6f18ff9700 1 -- 172.21.15.131:0/1879991022 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400cb70 con 0x7f6f1c159a70 2023-12-26T21:53:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.938+0000 7f6f18ff9700 1 -- 172.21.15.131:0/1879991022 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1400f040 con 0x7f6f1c159a70 2023-12-26T21:53:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.938+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c0ca050 con 0x7f6f1c159a70 2023-12-26T21:53:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.938+0000 7f6f18ff9700 1 -- 172.21.15.131:0/1879991022 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14014480 con 0x7f6f1c159a70 2023-12-26T21:53:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.938+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c157150 con 0x7f6f1c159a70 2023-12-26T21:53:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.939+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1c0657e0 con 0x7f6f1c159a70 2023-12-26T21:53:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.940+0000 7f6f18ff9700 1 -- 172.21.15.131:0/1879991022 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f6f14014830 con 0x7f6f1c159a70 2023-12-26T21:53:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.941+0000 7f6f18ff9700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f000670e0 0x7f6f00069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.941+0000 7f6f18ff9700 1 -- 172.21.15.131:0/1879991022 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f6f140855c0 con 0x7f6f1c159a70 2023-12-26T21:53:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.941+0000 7f6f2099b700 1 -- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f000670e0 msgr2=0x7f6f00069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.941+0000 7f6f2099b700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f000670e0 0x7f6f00069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:30.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:30.945+0000 7f6f18ff9700 1 -- 172.21.15.131:0/1879991022 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f14052210 con 0x7f6f1c159a70 2023-12-26T21:53:31.088 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:31.088+0000 7f5bf1b951c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:53:31.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.142+0000 7f6f2099b700 1 -- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f000670e0 msgr2=0x7f6f00069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:31.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.142+0000 7f6f2099b700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f000670e0 0x7f6f00069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:31.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.149+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f1c0b3820 con 0x7f6f1c159a70 2023-12-26T21:53:31.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.150+0000 7f6f18ff9700 1 -- 172.21.15.131:0/1879991022 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16370 (secure 0 0 0) 0x7f6f14059e60 con 0x7f6f1c159a70 2023-12-26T21:53:31.149 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:31.149 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:25.566998+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:02.972636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:03.709325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6837/39288":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/3724250355":"2023-12-27T21:53:25.566963+0000","172.21.15.131:6836/39288":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/4100737385":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/1623991397":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/4146017606":"2023-12-27T21:53:25.566963+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f000670e0 msgr2=0x7f6f00069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f000670e0 0x7f6f00069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 msgr2=0x7f6f1c0c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 0x7f6f1c0c9430 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009e90 tx=0x7f6f1400b040 comp rx=0 tx=0).stop 2023-12-26T21:53:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 shutdown_connections 2023-12-26T21:53:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6f000670e0 0x7f6f00069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f1c159a70 0x7f6f1c0c9430 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c0b2a90 0x7f6f1c1478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 --2- 172.21.15.131:0/1879991022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f1c1596a0 0x7f6f1c0c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 >> 172.21.15.131:0/1879991022 conn(0x7f6f1c06ece0 msgr2=0x7f6f1c157760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:31.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 shutdown_connections 2023-12-26T21:53:31.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.152+0000 7f6f22bff700 1 -- 172.21.15.131:0/1879991022 wait complete. 2023-12-26T21:53:31.171 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T21:53:31.171 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:53:31.182 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:31.182+0000 7f5bf1b951c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:53:31.275 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:31.275+0000 7f9a5bac81c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:53:31.334 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:31.334+0000 7f5bf1b951c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:53:31.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.364+0000 7f0dc51ff700 1 Processor -- start 2023-12-26T21:53:31.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.365+0000 7f0dc51ff700 1 -- start start 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.366+0000 7f0dc51ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0dc00c8bf0 0x7f0dc00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.366+0000 7f0dc51ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc0069d10 0x7f0dc00b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.366+0000 7f0dc51ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc00caa10 con 0x7f0dc00c8820 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.366+0000 7f0dc51ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc0157a80 con 0x7f0dc00c8bf0 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.366+0000 7f0dc51ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00cac50 con 0x7f0dc0069d10 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.366+0000 7f0dbed9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0dc00c8820 0x7f0dc00ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44682/0 (socket says 172.21.15.131:44682) 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.366+0000 7f0dbed9d700 1 -- 172.21.15.131:0/1555758416 learned_addr learned my addr 172.21.15.131:0/1555758416 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.366+0000 7f0dbe59c700 1 --2- 172.21.15.131:0/1555758416 >> v2:172.21.15.190:3300/0 conn(0x7f0dc00c8bf0 0x7f0dc00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.366+0000 7f0dbf59e700 1 --2- 172.21.15.131:0/1555758416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc0069d10 0x7f0dc00b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbdd9b700 1 -- 172.21.15.131:0/1555758416 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319596371 0 0) 0x7f0db0001110 con 0x7f0dc00c8820 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbdd9b700 1 -- 172.21.15.131:0/1555758416 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0da8003570 con 0x7f0dc00c8820 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbe59c700 1 -- 172.21.15.131:0/1555758416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc0069d10 msgr2=0x7f0dc00b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:31.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbe59c700 1 --2- 172.21.15.131:0/1555758416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc0069d10 0x7f0dc00b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbe59c700 1 -- 172.21.15.131:0/1555758416 >> v1:172.21.15.131:6789/0 conn(0x7f0dc00c8820 legacy=0x7f0dc00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbe59c700 1 -- 172.21.15.131:0/1555758416 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc00caf30 con 0x7f0dc00c8bf0 2023-12-26T21:53:31.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbe59c700 1 --2- 172.21.15.131:0/1555758416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8bf0 0x7f0dc00b12e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098e0 tx=0x7f0db40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9acb74485b5320c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:31.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbdd9b700 1 -- 172.21.15.131:0/1555758416 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4011630 con 0x7f0dc00c8bf0 2023-12-26T21:53:31.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbdd9b700 1 -- 172.21.15.131:0/1555758416 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db4011790 con 0x7f0dc00c8bf0 2023-12-26T21:53:31.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.367+0000 7f0dbdd9b700 1 -- 172.21.15.131:0/1555758416 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4011b30 con 0x7f0dc00c8bf0 2023-12-26T21:53:31.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.368+0000 7f0dc51ff700 1 -- 172.21.15.131:0/1555758416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8bf0 msgr2=0x7f0dc00b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.368+0000 7f0dc51ff700 1 --2- 172.21.15.131:0/1555758416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8bf0 0x7f0dc00b12e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098e0 tx=0x7f0db40091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.368+0000 7f0dc51ff700 1 -- 172.21.15.131:0/1555758416 shutdown_connections 2023-12-26T21:53:31.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.368+0000 7f0dc51ff700 1 --2- 172.21.15.131:0/1555758416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc0069d10 0x7f0dc00b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.368+0000 7f0dc51ff700 1 --2- 172.21.15.131:0/1555758416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8bf0 0x7f0dc00b12e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.368+0000 7f0dc51ff700 1 -- 172.21.15.131:0/1555758416 >> 172.21.15.131:0/1555758416 conn(0x7f0dc0068040 msgr2=0x7f0dc0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:31.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.368+0000 7f0dc51ff700 1 -- 172.21.15.131:0/1555758416 shutdown_connections 2023-12-26T21:53:31.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.368+0000 7f0dc51ff700 1 -- 172.21.15.131:0/1555758416 wait complete. 2023-12-26T21:53:31.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.370+0000 7f0dc51ff700 1 Processor -- start 2023-12-26T21:53:31.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.370+0000 7f0dc51ff700 1 -- start start 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dc51ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dc0069d10 0x7f0dc016d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dc51ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8820 0x7f0dc0143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dc51ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc00c8bf0 0x7f0dc015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dc51ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc0151ee0 con 0x7f0dc0069d10 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dc51ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc016d7c0 con 0x7f0dc00c8820 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dc51ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc016daa0 con 0x7f0dc00c8bf0 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dc0069d10 0x7f0dc016d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dc0069d10 0x7f0dc016d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49104/0 (socket says 172.21.15.131:49104) 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbed9d700 1 -- 172.21.15.131:0/97452706 learned_addr learned my addr 172.21.15.131:0/97452706 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbe59c700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8820 0x7f0dc0143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbf59e700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc00c8bf0 0x7f0dc015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbed9d700 1 -- 172.21.15.131:0/97452706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc00c8bf0 msgr2=0x7f0dc015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbed9d700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc00c8bf0 0x7f0dc015c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbed9d700 1 -- 172.21.15.131:0/97452706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8820 msgr2=0x7f0dc0143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbed9d700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8820 0x7f0dc0143db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbed9d700 1 -- 172.21.15.131:0/97452706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db4009420 con 0x7f0dc0069d10 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.371+0000 7f0dbe59c700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8820 0x7f0dc0143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.372+0000 7f0dbed9d700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dc0069d10 0x7f0dc016d1e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c7c0 tx=0x7f0db0007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.372+0000 7f0da6ffd700 1 -- 172.21.15.131:0/97452706 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0005c00 con 0x7f0dc0069d10 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.372+0000 7f0da6ffd700 1 -- 172.21.15.131:0/97452706 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db0005d60 con 0x7f0dc0069d10 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.372+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc016dd80 con 0x7f0dc0069d10 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.372+0000 7f0da6ffd700 1 -- 172.21.15.131:0/97452706 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0007320 con 0x7f0dc0069d10 2023-12-26T21:53:31.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.372+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc01519d0 con 0x7f0dc0069d10 2023-12-26T21:53:31.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.373+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc006b590 con 0x7f0dc0069d10 2023-12-26T21:53:31.374 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:31.374+0000 7f9a5bac81c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:53:31.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.376+0000 7f0da6ffd700 1 -- 172.21.15.131:0/97452706 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f0db001b3b0 con 0x7f0dc0069d10 2023-12-26T21:53:31.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.377+0000 7f0da6ffd700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0dac0670e0 0x7f0dac069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.377+0000 7f0da6ffd700 1 -- 172.21.15.131:0/97452706 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f0db0083ee0 con 0x7f0dc0069d10 2023-12-26T21:53:31.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.377+0000 7f0dbe59c700 1 -- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0dac0670e0 msgr2=0x7f0dac069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:31.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.377+0000 7f0dbe59c700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0dac0670e0 0x7f0dac069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:31.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.381+0000 7f0da6ffd700 1 -- 172.21.15.131:0/97452706 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0db0050c40 con 0x7f0dc0069d10 2023-12-26T21:53:31.535 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:31.534+0000 7f9a5bac81c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:53:31.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.577+0000 7f0dbe59c700 1 -- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0dac0670e0 msgr2=0x7f0dac069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:31.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.577+0000 7f0dbe59c700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0dac0670e0 0x7f0dac069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:31.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.627+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dc00caa10 con 0x7f0dc0069d10 2023-12-26T21:53:31.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.628+0000 7f0da6ffd700 1 -- 172.21.15.131:0/97452706 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f0dc00caa10 con 0x7f0dc0069d10 2023-12-26T21:53:31.629 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:31.630 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6454,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1671363867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1671363867},{"type":"v1","addr":"172.21.15.131:6834","nonce":1671363867}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3807864961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3807864961},{"type":"v1","addr":"172.21.15.190:6834","nonce":3807864961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/18678440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":18678440},{"type":"v1","addr":"172.21.15.131:6835","nonce":18678440}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:04.724845+0000","modified":"2023-12-26T21:53:08.284069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/168268053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":168268053},{"type":"v1","addr":"172.21.15.190:6835","nonce":168268053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0dac0670e0 msgr2=0x7f0dac069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0dac0670e0 0x7f0dac069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dc0069d10 msgr2=0x7f0dc016d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dc0069d10 0x7f0dc016d1e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c7c0 tx=0x7f0db0007c70 comp rx=0 tx=0).stop 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 shutdown_connections 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dc00c8bf0 0x7f0dc015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0dac0670e0 0x7f0dac069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dc0069d10 0x7f0dc016d1e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 --2- 172.21.15.131:0/97452706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dc00c8820 0x7f0dc0143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 >> 172.21.15.131:0/97452706 conn(0x7f0dc0068040 msgr2=0x7f0dc006ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:31.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 shutdown_connections 2023-12-26T21:53:31.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.630+0000 7f0dc51ff700 1 -- 172.21.15.131:0/97452706 wait complete. 2023-12-26T21:53:31.633 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 42 2023-12-26T21:53:31.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.843+0000 7f8d7d9ff700 1 Processor -- start 2023-12-26T21:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.844+0000 7f8d7d9ff700 1 -- start start 2023-12-26T21:53:31.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.845+0000 7f8d7d9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8d780c8bf0 0x7f8d780b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.845+0000 7f8d7d9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d78069d10 0x7f8d780b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.845+0000 7f8d7d9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d780caa10 con 0x7f8d780c8820 2023-12-26T21:53:31.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.845+0000 7f8d7d9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d78157a80 con 0x7f8d780c8bf0 2023-12-26T21:53:31.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.845+0000 7f8d7d9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780cac50 con 0x7f8d78069d10 2023-12-26T21:53:31.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.845+0000 7f8d767fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8d780c8bf0 0x7f8d780b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.845+0000 7f8d767fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8d780c8bf0 0x7f8d780b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49116/0 (socket says 172.21.15.131:49116) 2023-12-26T21:53:31.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.845+0000 7f8d767fc700 1 -- 172.21.15.131:0/851503980 learned_addr learned my addr 172.21.15.131:0/851503980 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:31.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.845+0000 7f8d777fe700 1 --2- 172.21.15.131:0/851503980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d78069d10 0x7f8d780b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d75ffb700 1 -- 172.21.15.131:0/851503980 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331911519 0 0) 0x7f8d68001110 con 0x7f8d780c8820 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d75ffb700 1 -- 172.21.15.131:0/851503980 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d60003570 con 0x7f8d780c8820 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d767fc700 1 -- 172.21.15.131:0/851503980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d78069d10 msgr2=0x7f8d780b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d767fc700 1 --2- 172.21.15.131:0/851503980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d78069d10 0x7f8d780b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d767fc700 1 -- 172.21.15.131:0/851503980 >> v1:172.21.15.190:6789/0 conn(0x7f8d780c8820 legacy=0x7f8d780ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d767fc700 1 -- 172.21.15.131:0/851503980 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d780caf30 con 0x7f8d780c8bf0 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d767fc700 1 --2- 172.21.15.131:0/851503980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d780c8bf0 0x7f8d780b12e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0099b0 tx=0x7f8d6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30734a69009c42b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d75ffb700 1 -- 172.21.15.131:0/851503980 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00b680 con 0x7f8d780c8bf0 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d75ffb700 1 -- 172.21.15.131:0/851503980 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c00b7e0 con 0x7f8d780c8bf0 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.846+0000 7f8d75ffb700 1 -- 172.21.15.131:0/851503980 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00bb80 con 0x7f8d780c8bf0 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.847+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/851503980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d780c8bf0 msgr2=0x7f8d780b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.847+0000 7f8d7d9ff700 1 --2- 172.21.15.131:0/851503980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d780c8bf0 0x7f8d780b12e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0099b0 tx=0x7f8d6c009250 comp rx=0 tx=0).stop 2023-12-26T21:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.847+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/851503980 shutdown_connections 2023-12-26T21:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.847+0000 7f8d7d9ff700 1 --2- 172.21.15.131:0/851503980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d78069d10 0x7f8d780b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.847+0000 7f8d7d9ff700 1 --2- 172.21.15.131:0/851503980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d780c8bf0 0x7f8d780b12e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.847+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/851503980 >> 172.21.15.131:0/851503980 conn(0x7f8d78068040 msgr2=0x7f8d78069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.847+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/851503980 shutdown_connections 2023-12-26T21:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.847+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/851503980 wait complete. 2023-12-26T21:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.849+0000 7f8d7d9ff700 1 Processor -- start 2023-12-26T21:53:31.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.850+0000 7f8d7d9ff700 1 -- start start 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d7d9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d78069d10 0x7f8d78146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d7d9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d780c8820 0x7f8d780c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d7d9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d780c8bf0 0x7f8d780c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d7d9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780a57d0 con 0x7f8d78069d10 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d7d9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7815acd0 con 0x7f8d780c8bf0 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d7d9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780afd00 con 0x7f8d780c8820 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d76ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d78069d10 0x7f8d78146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d767fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d780c8820 0x7f8d780c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d76ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d78069d10 0x7f8d78146ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49124/0 (socket says 172.21.15.131:49124) 2023-12-26T21:53:31.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d777fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d780c8bf0 0x7f8d780c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.851+0000 7f8d76ffd700 1 -- 172.21.15.131:0/2157217159 learned_addr learned my addr 172.21.15.131:0/2157217159 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d76ffd700 1 -- 172.21.15.131:0/2157217159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d780c8820 msgr2=0x7f8d780c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d76ffd700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d780c8820 0x7f8d780c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d76ffd700 1 -- 172.21.15.131:0/2157217159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d780c8bf0 msgr2=0x7f8d780c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d76ffd700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d780c8bf0 0x7f8d780c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d76ffd700 1 -- 172.21.15.131:0/2157217159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c0094f0 con 0x7f8d78069d10 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d76ffd700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d78069d10 0x7f8d78146ed0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c7c0 tx=0x7f8d68007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d5effd700 1 -- 172.21.15.131:0/2157217159 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68011590 con 0x7f8d78069d10 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d5effd700 1 -- 172.21.15.131:0/2157217159 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d680116f0 con 0x7f8d78069d10 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d780affe0 con 0x7f8d78069d10 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d5effd700 1 -- 172.21.15.131:0/2157217159 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68011a90 con 0x7f8d78069d10 2023-12-26T21:53:31.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.852+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7815a960 con 0x7f8d78069d10 2023-12-26T21:53:31.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.854+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d780657e0 con 0x7f8d78069d10 2023-12-26T21:53:31.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.854+0000 7f8d5effd700 1 -- 172.21.15.131:0/2157217159 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f8d6801c460 con 0x7f8d78069d10 2023-12-26T21:53:31.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.855+0000 7f8d5effd700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8d640670e0 0x7f8d64069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:31.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.855+0000 7f8d5effd700 1 -- 172.21.15.131:0/2157217159 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f8d68084670 con 0x7f8d78069d10 2023-12-26T21:53:31.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.855+0000 7f8d767fc700 1 -- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8d640670e0 msgr2=0x7f8d64069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:31.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.855+0000 7f8d767fc700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8d640670e0 0x7f8d64069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:31.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:31.859+0000 7f8d5effd700 1 -- 172.21.15.131:0/2157217159 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d68051330 con 0x7f8d78069d10 2023-12-26T21:53:31.960 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:31.960+0000 7f5bf1b951c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:53:32.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.056+0000 7f8d767fc700 1 -- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8d640670e0 msgr2=0x7f8d64069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:32.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.056+0000 7f8d767fc700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8d640670e0 0x7f8d64069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:32.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.105+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d780caa10 con 0x7f8d78069d10 2023-12-26T21:53:32.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.106+0000 7f8d5effd700 1 -- 172.21.15.131:0/2157217159 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f8d780caa10 con 0x7f8d78069d10 2023-12-26T21:53:32.106 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:32.106 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6454,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1671363867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1671363867},{"type":"v1","addr":"172.21.15.131:6834","nonce":1671363867}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3807864961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3807864961},{"type":"v1","addr":"172.21.15.190:6834","nonce":3807864961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/18678440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":18678440},{"type":"v1","addr":"172.21.15.131:6835","nonce":18678440}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:04.724845+0000","modified":"2023-12-26T21:53:08.284069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/168268053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":168268053},{"type":"v1","addr":"172.21.15.190:6835","nonce":168268053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-12-26T21:53:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8d640670e0 msgr2=0x7f8d64069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8d640670e0 0x7f8d64069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d78069d10 msgr2=0x7f8d78146ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d78069d10 0x7f8d78146ed0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c7c0 tx=0x7f8d68007c80 comp rx=0 tx=0).stop 2023-12-26T21:53:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 shutdown_connections 2023-12-26T21:53:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d780c8bf0 0x7f8d780c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8d640670e0 0x7f8d64069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d78069d10 0x7f8d78146ed0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 --2- 172.21.15.131:0/2157217159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d780c8820 0x7f8d780c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 >> 172.21.15.131:0/2157217159 conn(0x7f8d78068040 msgr2=0x7f8d780ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:32.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 shutdown_connections 2023-12-26T21:53:32.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.108+0000 7f8d7d9ff700 1 -- 172.21.15.131:0/2157217159 wait complete. 2023-12-26T21:53:32.110 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 42 2023-12-26T21:53:32.127 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:32.146 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:32.146+0000 7f5bf1b951c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:53:32.192 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:32.191+0000 7f9a5bac81c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:53:32.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.319+0000 7f565f3ff700 1 Processor -- start 2023-12-26T21:53:32.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.319+0000 7f565f3ff700 1 -- start start 2023-12-26T21:53:32.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.321+0000 7f565f3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5658159920 0x7f56580b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.321+0000 7f565f3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 0x7f56580b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.321+0000 7f565f3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56580c8bf0 con 0x7f56581554d0 2023-12-26T21:53:32.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.321+0000 7f565f3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56580c8e60 con 0x7f5658159920 2023-12-26T21:53:32.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.321+0000 7f565f3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56580c9140 con 0x7f5658159cf0 2023-12-26T21:53:32.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.321+0000 7f565d99c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f56581554d0 0x7f5658155930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44696/0 (socket says 172.21.15.131:44696) 2023-12-26T21:53:32.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.321+0000 7f565d99c700 1 -- 172.21.15.131:0/2942680346 learned_addr learned my addr 172.21.15.131:0/2942680346 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:32.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.321+0000 7f565d19b700 1 --2- 172.21.15.131:0/2942680346 >> v2:172.21.15.190:3300/0 conn(0x7f5658159920 0x7f56580b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.321+0000 7f565c99a700 1 --2- 172.21.15.131:0/2942680346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 0x7f56580b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f564ffff700 1 -- 172.21.15.131:0/2942680346 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527614343 0 0) 0x7f56580c8bf0 con 0x7f56581554d0 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f564ffff700 1 -- 172.21.15.131:0/2942680346 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5644003570 con 0x7f56581554d0 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f565d19b700 1 -- 172.21.15.131:0/2942680346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 msgr2=0x7f56580b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f565d19b700 1 --2- 172.21.15.131:0/2942680346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 0x7f56580b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f565d19b700 1 -- 172.21.15.131:0/2942680346 >> v1:172.21.15.131:6789/0 conn(0x7f56581554d0 legacy=0x7f5658155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f565d19b700 1 -- 172.21.15.131:0/2942680346 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56580c9420 con 0x7f5658159920 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f565d19b700 1 --2- 172.21.15.131:0/2942680346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5658159920 0x7f56580b00e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f565400aa90 tx=0x7f565400a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9659cdb4ea3b7b16 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f564ffff700 1 -- 172.21.15.131:0/2942680346 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565400c970 con 0x7f5658159920 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f564ffff700 1 -- 172.21.15.131:0/2942680346 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565400cad0 con 0x7f5658159920 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.322+0000 7f564ffff700 1 -- 172.21.15.131:0/2942680346 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565400ce70 con 0x7f5658159920 2023-12-26T21:53:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.323+0000 7f565f3ff700 1 -- 172.21.15.131:0/2942680346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5658159920 msgr2=0x7f56580b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.323+0000 7f565f3ff700 1 --2- 172.21.15.131:0/2942680346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5658159920 0x7f56580b00e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f565400aa90 tx=0x7f565400a330 comp rx=0 tx=0).stop 2023-12-26T21:53:32.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.323+0000 7f565f3ff700 1 -- 172.21.15.131:0/2942680346 shutdown_connections 2023-12-26T21:53:32.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.323+0000 7f565f3ff700 1 --2- 172.21.15.131:0/2942680346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5658159920 0x7f56580b00e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.323+0000 7f565f3ff700 1 --2- 172.21.15.131:0/2942680346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 0x7f56580b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.323+0000 7f565f3ff700 1 -- 172.21.15.131:0/2942680346 >> 172.21.15.131:0/2942680346 conn(0x7f5658068040 msgr2=0x7f5658069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:32.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.323+0000 7f565f3ff700 1 -- 172.21.15.131:0/2942680346 shutdown_connections 2023-12-26T21:53:32.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.323+0000 7f565f3ff700 1 -- 172.21.15.131:0/2942680346 wait complete. 2023-12-26T21:53:32.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.325+0000 7f565f3ff700 1 Processor -- start 2023-12-26T21:53:32.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.325+0000 7f565f3ff700 1 -- start start 2023-12-26T21:53:32.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.325+0000 7f565f3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56581554d0 0x7f56581517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565f3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5658159920 0x7f5658151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565f3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 0x7f5658157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565f3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565815dac0 con 0x7f56581554d0 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565f3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565816b600 con 0x7f5658159920 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565f3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565815d7e0 con 0x7f5658159cf0 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56581554d0 0x7f56581517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56581554d0 0x7f56581517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49132/0 (socket says 172.21.15.131:49132) 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d19b700 1 -- 172.21.15.131:0/1336254571 learned_addr learned my addr 172.21.15.131:0/1336254571 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d99c700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 0x7f5658157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565c99a700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5658159920 0x7f5658151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d19b700 1 -- 172.21.15.131:0/1336254571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 msgr2=0x7f5658157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d19b700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 0x7f5658157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d19b700 1 -- 172.21.15.131:0/1336254571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5658159920 msgr2=0x7f5658151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d19b700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5658159920 0x7f5658151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d19b700 1 -- 172.21.15.131:0/1336254571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565400a5d0 con 0x7f56581554d0 2023-12-26T21:53:32.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d99c700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 0x7f5658157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:32.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.326+0000 7f565d19b700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56581554d0 0x7f56581517b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f565400c4d0 tx=0x7f565401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:32.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.327+0000 7f564d7fa700 1 -- 172.21.15.131:0/1336254571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654021430 con 0x7f56581554d0 2023-12-26T21:53:32.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.327+0000 7f564d7fa700 1 -- 172.21.15.131:0/1336254571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654021590 con 0x7f56581554d0 2023-12-26T21:53:32.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.327+0000 7f564d7fa700 1 -- 172.21.15.131:0/1336254571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654021930 con 0x7f56581554d0 2023-12-26T21:53:32.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.327+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565816afb0 con 0x7f56581554d0 2023-12-26T21:53:32.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.327+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565815d400 con 0x7f56581554d0 2023-12-26T21:53:32.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.328+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565816b450 con 0x7f56581554d0 2023-12-26T21:53:32.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.329+0000 7f564d7fa700 1 -- 172.21.15.131:0/1336254571 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f56540103f0 con 0x7f56581554d0 2023-12-26T21:53:32.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.330+0000 7f564d7fa700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f563c0670e0 0x7f563c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.330+0000 7f565c99a700 1 -- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f563c0670e0 msgr2=0x7f563c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:32.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.330+0000 7f564d7fa700 1 -- 172.21.15.131:0/1336254571 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f5654086380 con 0x7f56581554d0 2023-12-26T21:53:32.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.330+0000 7f565c99a700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f563c0670e0 0x7f563c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:32.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.334+0000 7f564d7fa700 1 -- 172.21.15.131:0/1336254571 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5654053040 con 0x7f56581554d0 2023-12-26T21:53:32.335 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:32.335+0000 7f5bf1b951c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:53:32.395 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:32.394+0000 7f9a5bac81c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:53:32.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.531+0000 7f565c99a700 1 -- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f563c0670e0 msgr2=0x7f563c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:32.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.531+0000 7f565c99a700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f563c0670e0 0x7f563c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:32.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.535+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56580657e0 con 0x7f56581554d0 2023-12-26T21:53:32.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.536+0000 7f564d7fa700 1 -- 172.21.15.131:0/1336254571 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16370 (secure 0 0 0) 0x7f5654056660 con 0x7f56581554d0 2023-12-26T21:53:32.537 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:32.537 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":97,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:25.566998+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:02.972636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:03.709325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6837/39288":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/3724250355":"2023-12-27T21:53:25.566963+0000","172.21.15.131:6836/39288":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/4100737385":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/1623991397":"2023-12-27T21:53:25.566963+0000","172.21.15.131:0/4146017606":"2023-12-27T21:53:25.566963+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:32.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.538+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f563c0670e0 msgr2=0x7f563c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:32.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.538+0000 7f565f3ff700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f563c0670e0 0x7f563c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.538+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56581554d0 msgr2=0x7f56581517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.538+0000 7f565f3ff700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56581554d0 0x7f56581517b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f565400c4d0 tx=0x7f565401a000 comp rx=0 tx=0).stop 2023-12-26T21:53:32.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.539+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 shutdown_connections 2023-12-26T21:53:32.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.539+0000 7f565f3ff700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5658159cf0 0x7f5658157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.539+0000 7f565f3ff700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f563c0670e0 0x7f563c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.539+0000 7f565f3ff700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56581554d0 0x7f56581517b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.539+0000 7f565f3ff700 1 --2- 172.21.15.131:0/1336254571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5658159920 0x7f5658151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.539+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 >> 172.21.15.131:0/1336254571 conn(0x7f5658068040 msgr2=0x7f56580aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:32.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.539+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 shutdown_connections 2023-12-26T21:53:32.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.539+0000 7f565f3ff700 1 -- 172.21.15.131:0/1336254571 wait complete. 2023-12-26T21:53:32.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T21:53:32.594 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:32.593+0000 7f9a5bac81c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:53:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.756+0000 7f44987ff700 1 Processor -- start 2023-12-26T21:53:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.756+0000 7f44987ff700 1 -- start start 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.757+0000 7f44987ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44900cc580 0x7f44900cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f44987ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f44900b0550 0x7f449015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f44987ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44900b0d70 con 0x7f44900b00e0 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f44987ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900680f0 con 0x7f44900b0550 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f44987ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44900675b0 con 0x7f44900cc580 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f4495d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f44900b00e0 0x7f449006e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44700/0 (socket says 172.21.15.131:44700) 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f4495d9a700 1 -- 172.21.15.131:0/122970948 learned_addr learned my addr 172.21.15.131:0/122970948 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f449659b700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44900cc580 0x7f44900cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f4496d9c700 1 --2- 172.21.15.131:0/122970948 >> v2:172.21.15.190:3300/0 conn(0x7f44900b0550 0x7f449015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426223378 0 0) 0x7f44900b0d70 con 0x7f44900b00e0 2023-12-26T21:53:32.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.758+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f447c003570 con 0x7f44900b00e0 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1716716727 0 0) 0x7f447c003570 con 0x7f44900b00e0 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44900cc580 msgr2=0x7f44900cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44900cc580 0x7f44900cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 >> v2:172.21.15.190:3300/0 conn(0x7f44900b0550 msgr2=0x7f449015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 --2- 172.21.15.131:0/122970948 >> v2:172.21.15.190:3300/0 conn(0x7f44900b0550 0x7f449015d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4490159c30 con 0x7f44900b00e0 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f4480001470 con 0x7f44900b00e0 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 >> v1:172.21.15.131:6789/0 conn(0x7f44900b00e0 legacy=0x7f449006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44900cc580 0x7f447c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447c003fe0 0x7f447c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f447c004d50 0x7f447c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900680f0 con 0x7f447c004d50 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900675b0 con 0x7f447c003fe0 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c00ba90 con 0x7f44900cc580 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4496d9c700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f447c004d50 0x7f447c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f449659b700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44900cc580 0x7f447c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4495d9a700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447c003fe0 0x7f447c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4496d9c700 1 -- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44900cc580 msgr2=0x7f447c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4496d9c700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44900cc580 0x7f447c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4496d9c700 1 -- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447c003fe0 msgr2=0x7f447c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4496d9c700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447c003fe0 0x7f447c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.759+0000 7f4496d9c700 1 -- 172.21.15.131:0/122970948 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c003570 con 0x7f447c004d50 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.760+0000 7f4496d9c700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f447c004d50 0x7f447c0051c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f448800e210 tx=0x7f448800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732ac0912ae79b4a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.760+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800ea60 con 0x7f447c004d50 2023-12-26T21:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.760+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448800f040 con 0x7f447c004d50 2023-12-26T21:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.760+0000 7f4495599700 1 -- 172.21.15.131:0/122970948 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800ee10 con 0x7f447c004d50 2023-12-26T21:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.761+0000 7f44987ff700 1 -- 172.21.15.131:0/122970948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f447c004d50 msgr2=0x7f447c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.761+0000 7f44987ff700 1 --2- 172.21.15.131:0/122970948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f447c004d50 0x7f447c0051c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f448800e210 tx=0x7f448800b040 comp rx=0 tx=0).stop 2023-12-26T21:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.761+0000 7f4496d9c700 1 -- 172.21.15.131:0/122970948 reap_dead start 2023-12-26T21:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.761+0000 7f44987ff700 1 -- 172.21.15.131:0/122970948 shutdown_connections 2023-12-26T21:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.761+0000 7f44987ff700 1 -- 172.21.15.131:0/122970948 >> 172.21.15.131:0/122970948 conn(0x7f44900686c0 msgr2=0x7f4490068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.761+0000 7f44987ff700 1 -- 172.21.15.131:0/122970948 shutdown_connections 2023-12-26T21:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.761+0000 7f44987ff700 1 -- 172.21.15.131:0/122970948 wait complete. 2023-12-26T21:53:32.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.763+0000 7f44987ff700 1 Processor -- start 2023-12-26T21:53:32.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.763+0000 7f44987ff700 1 -- start start 2023-12-26T21:53:32.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.764+0000 7f44987ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c004d50 0x7f449015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.764+0000 7f44987ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449015ae80 0x7f4490160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.764+0000 7f44987ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f449016a170 0x7f44900ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.764+0000 7f44987ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449014e700 con 0x7f449016a170 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.764+0000 7f44987ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449015b380 con 0x7f449015ae80 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.764+0000 7f44987ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449015b660 con 0x7f447c004d50 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.764+0000 7f4496d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f449016a170 0x7f44900ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f4496d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f449016a170 0x7f44900ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49142/0 (socket says 172.21.15.131:49142) 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f4496d9c700 1 -- 172.21.15.131:0/441724860 learned_addr learned my addr 172.21.15.131:0/441724860 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f4495d9a700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449015ae80 0x7f4490160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f449659b700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c004d50 0x7f449015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f4496d9c700 1 -- 172.21.15.131:0/441724860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c004d50 msgr2=0x7f449015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f4496d9c700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c004d50 0x7f449015c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f4496d9c700 1 -- 172.21.15.131:0/441724860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449015ae80 msgr2=0x7f4490160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f4496d9c700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449015ae80 0x7f4490160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f4496d9c700 1 -- 172.21.15.131:0/441724860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449015b940 con 0x7f449016a170 2023-12-26T21:53:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f4496d9c700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f449016a170 0x7f44900ad520 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4488016000 tx=0x7f448801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f44867fc700 1 -- 172.21.15.131:0/441724860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800ee60 con 0x7f449016a170 2023-12-26T21:53:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f44867fc700 1 -- 172.21.15.131:0/441724860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4488022070 con 0x7f449016a170 2023-12-26T21:53:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449015bc20 con 0x7f449016a170 2023-12-26T21:53:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.765+0000 7f44867fc700 1 -- 172.21.15.131:0/441724860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448801e890 con 0x7f449016a170 2023-12-26T21:53:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.766+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449016a610 con 0x7f449016a170 2023-12-26T21:53:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.767+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44900657e0 con 0x7f449016a170 2023-12-26T21:53:32.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.768+0000 7f44867fc700 1 -- 172.21.15.131:0/441724860 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f448801ec80 con 0x7f449016a170 2023-12-26T21:53:32.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.769+0000 7f44867fc700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44740670e0 0x7f4474069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:32.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.769+0000 7f449659b700 1 -- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44740670e0 msgr2=0x7f4474069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:32.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.769+0000 7f44867fc700 1 -- 172.21.15.131:0/441724860 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f4488024070 con 0x7f449016a170 2023-12-26T21:53:32.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.769+0000 7f449659b700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44740670e0 0x7f4474069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:32.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.776+0000 7f44867fc700 1 -- 172.21.15.131:0/441724860 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4488053c80 con 0x7f449016a170 2023-12-26T21:53:32.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.970+0000 7f449659b700 1 -- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44740670e0 msgr2=0x7f4474069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:32.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:32.970+0000 7f449659b700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44740670e0 0x7f4474069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.023+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f449015c120 con 0x7f449016a170 2023-12-26T21:53:33.024 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T21:53:33.024+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T21:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.028+0000 7f44867fc700 1 -- 172.21.15.131:0/441724860 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f44880572a0 con 0x7f449016a170 2023-12-26T21:53:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44740670e0 msgr2=0x7f4474069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44740670e0 0x7f4474069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f449016a170 msgr2=0x7f44900ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f449016a170 0x7f44900ad520 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4488016000 tx=0x7f448801a000 comp rx=0 tx=0).stop 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 shutdown_connections 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f44740670e0 0x7f4474069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f449016a170 0x7f44900ad520 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c004d50 0x7f449015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 --2- 172.21.15.131:0/441724860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f449015ae80 0x7f4490160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 >> 172.21.15.131:0/441724860 conn(0x7f44900686c0 msgr2=0x7f44900b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 shutdown_connections 2023-12-26T21:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.031+0000 7f44987ff700 1 -- 172.21.15.131:0/441724860 wait complete. 2023-12-26T21:53:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T21:53:33.034 INFO:tasks.ceph.mds.d.smithi190.stderr: -1274> 2023-12-26T21:53:10.232+0000 7f2fdb3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:53:33.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T21:53:33.070 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:53:33.185 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:33.185+0000 7f5bf1b951c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:53:33.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.295+0000 7f85461ff700 1 Processor -- start 2023-12-26T21:53:33.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.295+0000 7f85461ff700 1 -- start start 2023-12-26T21:53:33.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.297+0000 7f85461ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f85400c9130 0x7f8540162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:33.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.297+0000 7f85461ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854015ba70 con 0x7f854006b8c0 2023-12-26T21:53:33.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.297+0000 7f85461ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854015bf00 con 0x7f8540158030 2023-12-26T21:53:33.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.297+0000 7f85461ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854006c2a0 con 0x7f85400c9130 2023-12-26T21:53:33.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.297+0000 7f8537fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f854006b8c0 0x7f85400c8a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44714/0 (socket says 172.21.15.131:44714) 2023-12-26T21:53:33.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.297+0000 7f8537fff700 1 -- 172.21.15.131:0/969170808 learned_addr learned my addr 172.21.15.131:0/969170808 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:33.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.297+0000 7f853ffff700 1 --2- 172.21.15.131:0/969170808 >> v2:172.21.15.190:3300/0 conn(0x7f85400c9130 0x7f8540162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:33.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.297+0000 7f853effd700 1 -- 172.21.15.131:0/969170808 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018194052 0 0) 0x7f854015ba70 con 0x7f854006b8c0 2023-12-26T21:53:33.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.297+0000 7f853effd700 1 -- 172.21.15.131:0/969170808 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f852c003570 con 0x7f854006b8c0 2023-12-26T21:53:33.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.298+0000 7f853ffff700 1 -- 172.21.15.131:0/969170808 >> v1:172.21.15.190:6789/0 conn(0x7f8540158030 legacy=0x7f8540158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:33.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.298+0000 7f853effd700 1 -- 172.21.15.131:0/969170808 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544431802 0 0) 0x7f854015bf00 con 0x7f8540158030 2023-12-26T21:53:33.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.298+0000 7f853ffff700 1 -- 172.21.15.131:0/969170808 >> v1:172.21.15.131:6789/0 conn(0x7f854006b8c0 legacy=0x7f85400c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:33.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.298+0000 7f853ffff700 1 -- 172.21.15.131:0/969170808 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854015b140 con 0x7f85400c9130 2023-12-26T21:53:33.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.298+0000 7f853ffff700 1 --2- 172.21.15.131:0/969170808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85400c9130 0x7f8540162440 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8538009e60 tx=0x7f8538009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=842fbd97a3d3a76d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:33.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.298+0000 7f853effd700 1 -- 172.21.15.131:0/969170808 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853800e070 con 0x7f85400c9130 2023-12-26T21:53:33.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.298+0000 7f853effd700 1 -- 172.21.15.131:0/969170808 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853800bb60 con 0x7f85400c9130 2023-12-26T21:53:33.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.298+0000 7f853effd700 1 -- 172.21.15.131:0/969170808 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538012430 con 0x7f85400c9130 2023-12-26T21:53:33.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.299+0000 7f85461ff700 1 -- 172.21.15.131:0/969170808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85400c9130 msgr2=0x7f8540162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:33.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.299+0000 7f85461ff700 1 --2- 172.21.15.131:0/969170808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85400c9130 0x7f8540162440 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8538009e60 tx=0x7f8538009780 comp rx=0 tx=0).stop 2023-12-26T21:53:33.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.299+0000 7f85461ff700 1 -- 172.21.15.131:0/969170808 shutdown_connections 2023-12-26T21:53:33.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.299+0000 7f85461ff700 1 --2- 172.21.15.131:0/969170808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85400c9130 0x7f8540162440 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:33.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.299+0000 7f85461ff700 1 -- 172.21.15.131:0/969170808 >> 172.21.15.131:0/969170808 conn(0x7f85400ce3e0 msgr2=0x7f85400ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:33.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.299+0000 7f85461ff700 1 -- 172.21.15.131:0/969170808 shutdown_connections 2023-12-26T21:53:33.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.299+0000 7f85461ff700 1 -- 172.21.15.131:0/969170808 wait complete. 2023-12-26T21:53:33.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.301+0000 7f85461ff700 1 Processor -- start 2023-12-26T21:53:33.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.301+0000 7f85461ff700 1 -- start start 2023-12-26T21:53:33.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.302+0000 7f85461ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f854006b8c0 0x7f85400c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:33.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.302+0000 7f85461ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85400c9130 0x7f85400c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:33.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.302+0000 7f85461ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8540158030 0x7f85400c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.302+0000 7f85461ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854014ca20 con 0x7f85400c9130 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.302+0000 7f85461ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854014be10 con 0x7f8540158030 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.302+0000 7f85461ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854014c0f0 con 0x7f854006b8c0 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f8537fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85400c9130 0x7f85400c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f853ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8540158030 0x7f85400c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f853f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f854006b8c0 0x7f85400c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f8537fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85400c9130 0x7f85400c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49146/0 (socket says 172.21.15.131:49146) 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f8537fff700 1 -- 172.21.15.131:0/230272004 learned_addr learned my addr 172.21.15.131:0/230272004 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f8537fff700 1 -- 172.21.15.131:0/230272004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f854006b8c0 msgr2=0x7f85400c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f8537fff700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f854006b8c0 0x7f85400c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f8537fff700 1 -- 172.21.15.131:0/230272004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8540158030 msgr2=0x7f85400c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f8537fff700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8540158030 0x7f85400c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:33.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f8537fff700 1 -- 172.21.15.131:0/230272004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8538009a20 con 0x7f85400c9130 2023-12-26T21:53:33.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f853f7fe700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f854006b8c0 0x7f85400c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:33.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.303+0000 7f8537fff700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85400c9130 0x7f85400c5bf0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f85280011e0 tx=0x7f8528007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:33.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.304+0000 7f85377fe700 1 -- 172.21.15.131:0/230272004 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528005b30 con 0x7f85400c9130 2023-12-26T21:53:33.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.304+0000 7f85377fe700 1 -- 172.21.15.131:0/230272004 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8528005c90 con 0x7f85400c9130 2023-12-26T21:53:33.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.304+0000 7f85377fe700 1 -- 172.21.15.131:0/230272004 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528007300 con 0x7f85400c9130 2023-12-26T21:53:33.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.304+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854014c3d0 con 0x7f85400c9130 2023-12-26T21:53:33.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.304+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85401670b0 con 0x7f85400c9130 2023-12-26T21:53:33.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.305+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85400657e0 con 0x7f85400c9130 2023-12-26T21:53:33.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.306+0000 7f85377fe700 1 -- 172.21.15.131:0/230272004 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 40) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f8528007670 con 0x7f85400c9130 2023-12-26T21:53:33.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.307+0000 7f85377fe700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 0x7f8520069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:33.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.307+0000 7f85377fe700 1 -- 172.21.15.131:0/230272004 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8528083fe0 con 0x7f85400c9130 2023-12-26T21:53:33.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.307+0000 7f853f7fe700 1 -- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 msgr2=0x7f8520069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:33.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.307+0000 7f853f7fe700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 0x7f8520069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:53:33.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.311+0000 7f85377fe700 1 -- 172.21.15.131:0/230272004 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8528050c50 con 0x7f85400c9130 2023-12-26T21:53:33.462 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:33.461+0000 7f5bf1b951c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:53:33.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.508+0000 7f853f7fe700 1 -- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 msgr2=0x7f8520069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:33.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.508+0000 7f853f7fe700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 0x7f8520069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:53:33.510 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:33.509+0000 7f9a5bac81c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:53:33.553 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:33.553+0000 7f5bf1b951c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:53:33.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.570+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f854015b420 con 0x7f85400c9130 2023-12-26T21:53:33.751 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:53:33.751+0000 7f5bf1b951c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:53:33.790 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:33.790+0000 7f9a5bac81c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:53:33.885 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:33.885+0000 7f9a5bac81c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:53:33.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.909+0000 7f853f7fe700 1 -- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 msgr2=0x7f8520069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:53:33.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:33.909+0000 7f853f7fe700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 0x7f8520069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:53:34.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.029+0000 7f85377fe700 1 -- 172.21.15.131:0/230272004 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85280170a0 con 0x7f85400c9130 2023-12-26T21:53:34.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.031+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 msgr2=0x7f8520069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:53:34.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.031+0000 7f85461ff700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 0x7f8520069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.031+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85400c9130 msgr2=0x7f85400c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.031+0000 7f85461ff700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85400c9130 0x7f85400c5bf0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f85280011e0 tx=0x7f8528007df0 comp rx=0 tx=0).stop 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.032+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 shutdown_connections 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.032+0000 7f85461ff700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8540158030 0x7f85400c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.032+0000 7f85461ff700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f854006b8c0 0x7f85400c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.032+0000 7f85461ff700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f85200670e0 0x7f8520069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.032+0000 7f85461ff700 1 --2- 172.21.15.131:0/230272004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85400c9130 0x7f85400c5bf0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.032+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 >> 172.21.15.131:0/230272004 conn(0x7f85400ce3e0 msgr2=0x7f85400c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.032+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 shutdown_connections 2023-12-26T21:53:34.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.032+0000 7f85461ff700 1 -- 172.21.15.131:0/230272004 wait complete. 2023-12-26T21:53:34.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T21:53:34.086 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:53:34.086+0000 7f9a5bac81c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:53:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.272+0000 7fd0733ff700 1 Processor -- start 2023-12-26T21:53:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.272+0000 7fd0733ff700 1 -- start start 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.273+0000 7fd0733ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c069c30 0x7fd06c0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.273+0000 7fd0733ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c0c7430 con 0x7fd06c0ae080 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.273+0000 7fd0733ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c0ac460 con 0x7fd06c0adcb0 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.273+0000 7fd0733ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c6820 con 0x7fd06c069c30 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd07119b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd06c0adcb0 0x7fd06c069520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:60568/0 (socket says 172.21.15.131:60568) 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd07119b700 1 -- 172.21.15.131:0/1856877689 learned_addr learned my addr 172.21.15.131:0/1856877689 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd07099a700 1 --2- 172.21.15.131:0/1856877689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c069c30 0x7fd06c0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd07199c700 1 --1- 172.21.15.131:0/1856877689 >> v1:172.21.15.131:6789/0 conn(0x7fd06c0ae080 0x7fd06c06df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44718/0 (socket says 172.21.15.131:44718) 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd063fff700 1 -- 172.21.15.131:0/1856877689 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960669547 0 0) 0x7fd06c0ac460 con 0x7fd06c0adcb0 2023-12-26T21:53:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd063fff700 1 -- 172.21.15.131:0/1856877689 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd058003570 con 0x7fd06c0adcb0 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd063fff700 1 -- 172.21.15.131:0/1856877689 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432711788 0 0) 0x7fd06c0c7430 con 0x7fd06c0ae080 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd07099a700 1 -- 172.21.15.131:0/1856877689 >> v1:172.21.15.190:6789/0 conn(0x7fd06c0adcb0 legacy=0x7fd06c069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd07099a700 1 -- 172.21.15.131:0/1856877689 >> v1:172.21.15.131:6789/0 conn(0x7fd06c0ae080 legacy=0x7fd06c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd07099a700 1 -- 172.21.15.131:0/1856877689 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0c6b00 con 0x7fd06c069c30 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.274+0000 7fd07099a700 1 --2- 172.21.15.131:0/1856877689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c069c30 0x7fd06c0abf20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00aac0 tx=0x7fd05c00a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b341e9168b7a4e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd063fff700 1 -- 172.21.15.131:0/1856877689 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c00c970 con 0x7fd06c069c30 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd063fff700 1 -- 172.21.15.131:0/1856877689 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c00cad0 con 0x7fd06c069c30 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd063fff700 1 -- 172.21.15.131:0/1856877689 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c00ce70 con 0x7fd06c069c30 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd0733ff700 1 -- 172.21.15.131:0/1856877689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c069c30 msgr2=0x7fd06c0abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd0733ff700 1 --2- 172.21.15.131:0/1856877689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c069c30 0x7fd06c0abf20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00aac0 tx=0x7fd05c00a330 comp rx=0 tx=0).stop 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd0733ff700 1 -- 172.21.15.131:0/1856877689 shutdown_connections 2023-12-26T21:53:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd0733ff700 1 --2- 172.21.15.131:0/1856877689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c069c30 0x7fd06c0abf20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd0733ff700 1 -- 172.21.15.131:0/1856877689 >> 172.21.15.131:0/1856877689 conn(0x7fd06c0675b0 msgr2=0x7fd06c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd0733ff700 1 -- 172.21.15.131:0/1856877689 shutdown_connections 2023-12-26T21:53:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.275+0000 7fd0733ff700 1 -- 172.21.15.131:0/1856877689 wait complete. 2023-12-26T21:53:34.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.277+0000 7fd0733ff700 1 Processor -- start 2023-12-26T21:53:34.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.277+0000 7fd0733ff700 1 -- start start 2023-12-26T21:53:34.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd0733ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd06c069c30 0x7fd06c0bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:34.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd0733ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c0adcb0 0x7fd06c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd0733ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd06c0ae080 0x7fd06c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd0733ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c159070 con 0x7fd06c0ae080 2023-12-26T21:53:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd0733ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c0e10 con 0x7fd06c069c30 2023-12-26T21:53:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd0733ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c0200 con 0x7fd06c0adcb0 2023-12-26T21:53:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd07199c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd06c0ae080 0x7fd06c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd07199c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd06c0ae080 0x7fd06c1536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49226/0 (socket says 172.21.15.131:49226) 2023-12-26T21:53:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd07199c700 1 -- 172.21.15.131:0/2543429070 learned_addr learned my addr 172.21.15.131:0/2543429070 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd07099a700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c0adcb0 0x7fd06c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd07119b700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd06c069c30 0x7fd06c0bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd07199c700 1 -- 172.21.15.131:0/2543429070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c0adcb0 msgr2=0x7fd06c158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.278+0000 7fd07199c700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c0adcb0 0x7fd06c158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd07199c700 1 -- 172.21.15.131:0/2543429070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd06c069c30 msgr2=0x7fd06c0bfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd07199c700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd06c069c30 0x7fd06c0bfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd07199c700 1 -- 172.21.15.131:0/2543429070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c00a5d0 con 0x7fd06c0ae080 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd07099a700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c0adcb0 0x7fd06c158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd07199c700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd06c0ae080 0x7fd06c1536d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd06400cd50 tx=0x7fd064001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd0617fa700 1 -- 172.21.15.131:0/2543429070 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064005df0 con 0x7fd06c0ae080 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd0617fa700 1 -- 172.21.15.131:0/2543429070 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06400a960 con 0x7fd06c0ae080 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c0c04e0 con 0x7fd06c0ae080 2023-12-26T21:53:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd0617fa700 1 -- 172.21.15.131:0/2543429070 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400ad20 con 0x7fd06c0ae080 2023-12-26T21:53:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.279+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c0c0aa0 con 0x7fd06c0ae080 2023-12-26T21:53:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.280+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c0717f0 con 0x7fd06c0ae080 2023-12-26T21:53:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.281+0000 7fd0617fa700 1 -- 172.21.15.131:0/2543429070 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 41) v1 ==== 84453+0+0 (secure 0 0 0) 0x7fd064007300 con 0x7fd06c0ae080 2023-12-26T21:53:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.282+0000 7fd0617fa700 1 -- 172.21.15.131:0/2543429070 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6742+0+0 (secure 0 0 0) 0x7fd0640847d0 con 0x7fd06c0ae080 2023-12-26T21:53:34.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.286+0000 7fd0617fa700 1 -- 172.21.15.131:0/2543429070 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd064051980 con 0x7fd06c0ae080 2023-12-26T21:53:34.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:34.479+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd06c0c6de0 con 0x7fd06c0ae080 2023-12-26T21:53:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.101+0000 7fd0617fa700 1 -- 172.21.15.131:0/2543429070 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 42) v1 ==== 84580+0+0 (secure 0 0 0) 0x7fd064051430 con 0x7fd06c0ae080 2023-12-26T21:53:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.102+0000 7fd0617fa700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd05006c9e0 0x7fd05006edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:35.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.103+0000 7fd07119b700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd05006c9e0 0x7fd05006edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.106+0000 7fd07119b700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd05006c9e0 0x7fd05006edc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0680058d0 tx=0x7fd068009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:35.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.229+0000 7fd0617fa700 1 -- 172.21.15.131:0/2543429070 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd064080f80 con 0x7fd06c0ae080 2023-12-26T21:53:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.233+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd05006c9e0 msgr2=0x7fd05006edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.233+0000 7fd0733ff700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd05006c9e0 0x7fd05006edc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0680058d0 tx=0x7fd068009040 comp rx=0 tx=0).stop 2023-12-26T21:53:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.233+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd06c0ae080 msgr2=0x7fd06c1536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.233+0000 7fd0733ff700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd06c0ae080 0x7fd06c1536d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd06400cd50 tx=0x7fd064001b10 comp rx=0 tx=0).stop 2023-12-26T21:53:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.234+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 shutdown_connections 2023-12-26T21:53:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.234+0000 7fd0733ff700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd05006c9e0 0x7fd05006edc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.234+0000 7fd0733ff700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd06c0ae080 0x7fd06c1536d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.234+0000 7fd0733ff700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd06c069c30 0x7fd06c0bfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.234+0000 7fd0733ff700 1 --2- 172.21.15.131:0/2543429070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06c0adcb0 0x7fd06c158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.234+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 >> 172.21.15.131:0/2543429070 conn(0x7fd06c0675b0 msgr2=0x7fd06c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.234+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 shutdown_connections 2023-12-26T21:53:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.234+0000 7fd0733ff700 1 -- 172.21.15.131:0/2543429070 wait complete. 2023-12-26T21:53:35.236 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T21:53:35.255 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T21:53:35.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.488+0000 7fa27f9ff700 1 Processor -- start 2023-12-26T21:53:35.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.488+0000 7fa27f9ff700 1 -- start start 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.489+0000 7fa27f9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa278156360 0x7fa27806e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.489+0000 7fa27f9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa278156730 0x7fa27806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.489+0000 7fa27f9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa2780c8490 0x7fa2780c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.489+0000 7fa27f9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27806f1f0 con 0x7fa278156730 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.489+0000 7fa27f9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2780acfc0 con 0x7fa2780c8490 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.489+0000 7fa27f9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2780c91f0 con 0x7fa278156360 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.489+0000 7fa27cf9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa278156730 0x7fa27806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27cf9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa278156730 0x7fa27806ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43162/0 (socket says 172.21.15.131:43162) 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27cf9a700 1 -- 172.21.15.131:0/3195330266 learned_addr learned my addr 172.21.15.131:0/3195330266 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27df9c700 1 --2- 172.21.15.131:0/3195330266 >> v2:172.21.15.190:3300/0 conn(0x7fa2780c8490 0x7fa2780c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:35.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27d79b700 1 --2- 172.21.15.131:0/3195330266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa278156360 0x7fa27806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27cf9a700 1 -- 172.21.15.131:0/3195330266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa278156360 msgr2=0x7fa27806e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27cf9a700 1 --2- 172.21.15.131:0/3195330266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa278156360 0x7fa27806e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27cf9a700 1 -- 172.21.15.131:0/3195330266 >> v2:172.21.15.190:3300/0 conn(0x7fa2780c8490 msgr2=0x7fa2780c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27cf9a700 1 --2- 172.21.15.131:0/3195330266 >> v2:172.21.15.190:3300/0 conn(0x7fa2780c8490 0x7fa2780c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27cf9a700 1 -- 172.21.15.131:0/3195330266 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2780c94d0 con 0x7fa278156730 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa27cf9a700 1 --2- 172.21.15.131:0/3195330266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 0x7fa27806ecb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa2780bfc10 tx=0x7fa268009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c35085bcab374e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.490+0000 7fa26ffff700 1 -- 172.21.15.131:0/3195330266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800b860 con 0x7fa278156730 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa26ffff700 1 -- 172.21.15.131:0/3195330266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26800b9c0 con 0x7fa278156730 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa26ffff700 1 -- 172.21.15.131:0/3195330266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800bd60 con 0x7fa278156730 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa27f9ff700 1 -- 172.21.15.131:0/3195330266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 msgr2=0x7fa27806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/3195330266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 0x7fa27806ecb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa2780bfc10 tx=0x7fa268009250 comp rx=0 tx=0).stop 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa27f9ff700 1 -- 172.21.15.131:0/3195330266 shutdown_connections 2023-12-26T21:53:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/3195330266 >> v2:172.21.15.190:3300/0 conn(0x7fa2780c8490 0x7fa2780c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/3195330266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa278156360 0x7fa27806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/3195330266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 0x7fa27806ecb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa27f9ff700 1 -- 172.21.15.131:0/3195330266 >> 172.21.15.131:0/3195330266 conn(0x7fa2780675b0 msgr2=0x7fa2780679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:35.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa27f9ff700 1 -- 172.21.15.131:0/3195330266 shutdown_connections 2023-12-26T21:53:35.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.491+0000 7fa27f9ff700 1 -- 172.21.15.131:0/3195330266 wait complete. 2023-12-26T21:53:35.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.493+0000 7fa27f9ff700 1 Processor -- start 2023-12-26T21:53:35.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.493+0000 7fa27f9ff700 1 -- start start 2023-12-26T21:53:35.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27f9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2780c8490 0x7fa2780c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:35.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27f9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278156360 0x7fa2780c1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:35.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27f9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 0x7fa278154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:35.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27f9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2780c4750 con 0x7fa278156730 2023-12-26T21:53:35.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27f9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa278154cd0 con 0x7fa278156360 2023-12-26T21:53:35.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27f9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2780c15a0 con 0x7fa2780c8490 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27df9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 0x7fa278154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27df9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 0x7fa278154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43174/0 (socket says 172.21.15.131:43174) 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27df9c700 1 -- 172.21.15.131:0/2347644929 learned_addr learned my addr 172.21.15.131:0/2347644929 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27cf9a700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278156360 0x7fa2780c1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27d79b700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2780c8490 0x7fa2780c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27df9c700 1 -- 172.21.15.131:0/2347644929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2780c8490 msgr2=0x7fa2780c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27df9c700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2780c8490 0x7fa2780c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27df9c700 1 -- 172.21.15.131:0/2347644929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278156360 msgr2=0x7fa2780c1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27df9c700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278156360 0x7fa2780c1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27df9c700 1 -- 172.21.15.131:0/2347644929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2680094f0 con 0x7fa278156730 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.494+0000 7fa27d79b700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2780c8490 0x7fa2780c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.495+0000 7fa27df9c700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 0x7fa278154070 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa270009fd0 tx=0x7fa270009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.495+0000 7fa26d7fa700 1 -- 172.21.15.131:0/2347644929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000eca0 con 0x7fa278156730 2023-12-26T21:53:35.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.495+0000 7fa26d7fa700 1 -- 172.21.15.131:0/2347644929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27000ee00 con 0x7fa278156730 2023-12-26T21:53:35.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.495+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2780c71e0 con 0x7fa278156730 2023-12-26T21:53:35.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.495+0000 7fa26d7fa700 1 -- 172.21.15.131:0/2347644929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270014670 con 0x7fa278156730 2023-12-26T21:53:35.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.495+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2780c4100 con 0x7fa278156730 2023-12-26T21:53:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.496+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2780717f0 con 0x7fa278156730 2023-12-26T21:53:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.497+0000 7fa26d7fa700 1 -- 172.21.15.131:0/2347644929 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 42) v1 ==== 84580+0+0 (secure 0 0 0) 0x7fa2700149e0 con 0x7fa278156730 2023-12-26T21:53:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.497+0000 7fa26d7fa700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa25c066d80 0x7fa25c069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.498+0000 7fa26d7fa700 1 -- 172.21.15.131:0/2347644929 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fa270084b60 con 0x7fa278156730 2023-12-26T21:53:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.498+0000 7fa27d79b700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa25c066d80 0x7fa25c069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.498+0000 7fa27d79b700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa25c066d80 0x7fa25c069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa274006fd0 tx=0x7fa274008040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:35.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.501+0000 7fa26d7fa700 1 -- 172.21.15.131:0/2347644929 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa270051e30 con 0x7fa278156730 2023-12-26T21:53:35.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:35.700+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa2780acc50 con 0x7fa278156730 2023-12-26T21:53:36.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.236+0000 7fa26d7fa700 1 -- 172.21.15.131:0/2347644929 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa270055450 con 0x7fa278156730 2023-12-26T21:53:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.243+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa25c066d80 msgr2=0x7fa25c069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.243+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa25c066d80 0x7fa25c069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa274006fd0 tx=0x7fa274008040 comp rx=0 tx=0).stop 2023-12-26T21:53:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.243+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 msgr2=0x7fa278154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.243+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 0x7fa278154070 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa270009fd0 tx=0x7fa270009900 comp rx=0 tx=0).stop 2023-12-26T21:53:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.245+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 shutdown_connections 2023-12-26T21:53:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.245+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa25c066d80 0x7fa25c069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.245+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278156730 0x7fa278154070 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.245+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2780c8490 0x7fa2780c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.245+0000 7fa27f9ff700 1 --2- 172.21.15.131:0/2347644929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278156360 0x7fa2780c1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.245+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 >> 172.21.15.131:0/2347644929 conn(0x7fa2780675b0 msgr2=0x7fa278155320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.246+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 shutdown_connections 2023-12-26T21:53:36.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.246+0000 7fa27f9ff700 1 -- 172.21.15.131:0/2347644929 wait complete. 2023-12-26T21:53:36.249 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T21:53:36.267 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T21:53:36.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.460+0000 7f520bbff700 1 Processor -- start 2023-12-26T21:53:36.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.460+0000 7f520bbff700 1 -- start start 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.461+0000 7f520bbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52040cb320 0x7f5204069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.461+0000 7f520bbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f520406a1e0 0x7f52040b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.461+0000 7f520bbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5204156940 con 0x7f52040b0c90 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.461+0000 7f520bbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52040c8cc0 con 0x7f52040cb320 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.461+0000 7f520bbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52040ad700 con 0x7f520406a1e0 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.461+0000 7f520999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52040cb320 0x7f5204069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.461+0000 7f520a19c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f52040b0c90 0x7f52040aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58508/0 (socket says 172.21.15.131:58508) 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.461+0000 7f520a19c700 1 -- 172.21.15.131:0/2380668194 learned_addr learned my addr 172.21.15.131:0/2380668194 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f520919a700 1 --2- 172.21.15.131:0/2380668194 >> v2:172.21.15.190:3300/0 conn(0x7f520406a1e0 0x7f52040b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f5208999700 1 -- 172.21.15.131:0/2380668194 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969145989 0 0) 0x7f5204156940 con 0x7f52040b0c90 2023-12-26T21:53:36.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f5208999700 1 -- 172.21.15.131:0/2380668194 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f0003570 con 0x7f52040b0c90 2023-12-26T21:53:36.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f520999b700 1 -- 172.21.15.131:0/2380668194 >> v2:172.21.15.190:3300/0 conn(0x7f520406a1e0 msgr2=0x7f52040b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:36.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f520999b700 1 --2- 172.21.15.131:0/2380668194 >> v2:172.21.15.190:3300/0 conn(0x7f520406a1e0 0x7f52040b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f520999b700 1 -- 172.21.15.131:0/2380668194 >> v1:172.21.15.131:6789/0 conn(0x7f52040b0c90 legacy=0x7f52040aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f520999b700 1 -- 172.21.15.131:0/2380668194 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040ae500 con 0x7f52040cb320 2023-12-26T21:53:36.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f520999b700 1 --2- 172.21.15.131:0/2380668194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52040cb320 0x7f5204069ca0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5204064700 tx=0x7f52000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5bf1f31d63ecb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:36.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f5208999700 1 -- 172.21.15.131:0/2380668194 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200011630 con 0x7f52040cb320 2023-12-26T21:53:36.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f5208999700 1 -- 172.21.15.131:0/2380668194 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200011790 con 0x7f52040cb320 2023-12-26T21:53:36.464 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.462+0000 7f5208999700 1 -- 172.21.15.131:0/2380668194 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200011b30 con 0x7f52040cb320 2023-12-26T21:53:36.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.463+0000 7f520bbff700 1 -- 172.21.15.131:0/2380668194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52040cb320 msgr2=0x7f5204069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.463+0000 7f520bbff700 1 --2- 172.21.15.131:0/2380668194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52040cb320 0x7f5204069ca0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5204064700 tx=0x7f52000091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.463+0000 7f520bbff700 1 -- 172.21.15.131:0/2380668194 shutdown_connections 2023-12-26T21:53:36.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.463+0000 7f520bbff700 1 --2- 172.21.15.131:0/2380668194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52040cb320 0x7f5204069ca0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.463+0000 7f520bbff700 1 --2- 172.21.15.131:0/2380668194 >> v2:172.21.15.190:3300/0 conn(0x7f520406a1e0 0x7f52040b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.463+0000 7f520bbff700 1 -- 172.21.15.131:0/2380668194 >> 172.21.15.131:0/2380668194 conn(0x7f5204068040 msgr2=0x7f5204069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:36.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.463+0000 7f520bbff700 1 -- 172.21.15.131:0/2380668194 shutdown_connections 2023-12-26T21:53:36.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.463+0000 7f520bbff700 1 -- 172.21.15.131:0/2380668194 wait complete. 2023-12-26T21:53:36.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.465+0000 7f520bbff700 1 Processor -- start 2023-12-26T21:53:36.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.465+0000 7f520bbff700 1 -- start start 2023-12-26T21:53:36.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520bbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520406a1e0 0x7f5204150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52040b0c90 0x7f52040c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520bbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52040cb320 0x7f5204153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520bbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5204169ee0 con 0x7f520406a1e0 2023-12-26T21:53:36.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520bbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040c70a0 con 0x7f52040cb320 2023-12-26T21:53:36.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52040c6800 con 0x7f52040b0c90 2023-12-26T21:53:36.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520999b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520406a1e0 0x7f5204150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520999b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520406a1e0 0x7f5204150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43194/0 (socket says 172.21.15.131:43194) 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520999b700 1 -- 172.21.15.131:0/977490162 learned_addr learned my addr 172.21.15.131:0/977490162 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520a19c700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52040cb320 0x7f5204153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520919a700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52040b0c90 0x7f52040c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520999b700 1 -- 172.21.15.131:0/977490162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52040b0c90 msgr2=0x7f52040c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520999b700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52040b0c90 0x7f52040c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520999b700 1 -- 172.21.15.131:0/977490162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52040cb320 msgr2=0x7f5204153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520999b700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52040cb320 0x7f5204153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.466+0000 7f520999b700 1 -- 172.21.15.131:0/977490162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5200009420 con 0x7f520406a1e0 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.467+0000 7f520999b700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520406a1e0 0x7f5204150ba0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5200011f70 tx=0x7f5200017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.467+0000 7f51f9ffb700 1 -- 172.21.15.131:0/977490162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200011900 con 0x7f520406a1e0 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.467+0000 7f51f9ffb700 1 -- 172.21.15.131:0/977490162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200011a60 con 0x7f520406a1e0 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.467+0000 7f51f9ffb700 1 -- 172.21.15.131:0/977490162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200026640 con 0x7f520406a1e0 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.467+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52040c6ae0 con 0x7f520406a1e0 2023-12-26T21:53:36.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.467+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52041510e0 con 0x7f520406a1e0 2023-12-26T21:53:36.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.468+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5204169c90 con 0x7f520406a1e0 2023-12-26T21:53:36.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.469+0000 7f51f9ffb700 1 -- 172.21.15.131:0/977490162 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5200023030 con 0x7f520406a1e0 2023-12-26T21:53:36.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.470+0000 7f51f9ffb700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51e8066f30 0x7f51e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.470+0000 7f51f9ffb700 1 -- 172.21.15.131:0/977490162 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f5200085cc0 con 0x7f520406a1e0 2023-12-26T21:53:36.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.470+0000 7f520919a700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51e8066f30 0x7f51e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.471+0000 7f520919a700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51e8066f30 0x7f51e80693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52040cae10 tx=0x7f51f4008040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:36.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.474+0000 7f51f9ffb700 1 -- 172.21.15.131:0/977490162 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f520008c080 con 0x7f520406a1e0 2023-12-26T21:53:36.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.667+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f52040657e0 con 0x7f520406a1e0 2023-12-26T21:53:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.671+0000 7f51f9ffb700 1 -- 172.21.15.131:0/977490162 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5200052e50 con 0x7f520406a1e0 2023-12-26T21:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51e8066f30 msgr2=0x7f51e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51e8066f30 0x7f51e80693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52040cae10 tx=0x7f51f4008040 comp rx=0 tx=0).stop 2023-12-26T21:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520406a1e0 msgr2=0x7f5204150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520406a1e0 0x7f5204150ba0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5200011f70 tx=0x7f5200017000 comp rx=0 tx=0).stop 2023-12-26T21:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 shutdown_connections 2023-12-26T21:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52040cb320 0x7f5204153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51e8066f30 0x7f51e80693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520406a1e0 0x7f5204150ba0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 --2- 172.21.15.131:0/977490162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52040b0c90 0x7f52040c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 >> 172.21.15.131:0/977490162 conn(0x7f5204068040 msgr2=0x7f52041549c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 shutdown_connections 2023-12-26T21:53:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.673+0000 7f520bbff700 1 -- 172.21.15.131:0/977490162 wait complete. 2023-12-26T21:53:36.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T21:53:36.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.881+0000 7fccb09ff700 1 Processor -- start 2023-12-26T21:53:36.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.882+0000 7fccb09ff700 1 -- start start 2023-12-26T21:53:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccb09ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac158030 0x7fccac158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccb09ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fccac0c9130 0x7fccac0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccb09ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac15ba70 con 0x7fccac06b8c0 2023-12-26T21:53:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccb09ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac15bf00 con 0x7fccac0c9130 2023-12-26T21:53:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccb09ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac06c2a0 con 0x7fccac158030 2023-12-26T21:53:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccaad9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fccac0c9130 0x7fccac0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccaad9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fccac0c9130 0x7fccac0cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43204/0 (socket says 172.21.15.131:43204) 2023-12-26T21:53:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccaad9d700 1 -- 172.21.15.131:0/431464925 learned_addr learned my addr 172.21.15.131:0/431464925 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccaa59c700 1 --2- 172.21.15.131:0/431464925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac158030 0x7fccac158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccaad9d700 1 -- 172.21.15.131:0/431464925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac158030 msgr2=0x7fccac158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.883+0000 7fccaad9d700 1 --2- 172.21.15.131:0/431464925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac158030 0x7fccac158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.884+0000 7fccaad9d700 1 -- 172.21.15.131:0/431464925 >> v1:172.21.15.190:6789/0 conn(0x7fccac06b8c0 legacy=0x7fccac0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.884+0000 7fccaad9d700 1 -- 172.21.15.131:0/431464925 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac15b140 con 0x7fccac0c9130 2023-12-26T21:53:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.884+0000 7fccaad9d700 1 --2- 172.21.15.131:0/431464925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac0c9130 0x7fccac0cbfb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009f20 tx=0x7fcca4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15108b0d8e22b4d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.884+0000 7fcca959a700 1 -- 172.21.15.131:0/431464925 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca400e070 con 0x7fccac0c9130 2023-12-26T21:53:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.884+0000 7fcca959a700 1 -- 172.21.15.131:0/431464925 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca400ba80 con 0x7fccac0c9130 2023-12-26T21:53:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.884+0000 7fcca959a700 1 -- 172.21.15.131:0/431464925 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca40092e0 con 0x7fccac0c9130 2023-12-26T21:53:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.884+0000 7fccb09ff700 1 -- 172.21.15.131:0/431464925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac0c9130 msgr2=0x7fccac0cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.884+0000 7fccb09ff700 1 --2- 172.21.15.131:0/431464925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac0c9130 0x7fccac0cbfb0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009f20 tx=0x7fcca4009840 comp rx=0 tx=0).stop 2023-12-26T21:53:36.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.885+0000 7fccb09ff700 1 -- 172.21.15.131:0/431464925 shutdown_connections 2023-12-26T21:53:36.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.885+0000 7fccb09ff700 1 --2- 172.21.15.131:0/431464925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac0c9130 0x7fccac0cbfb0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.885+0000 7fccb09ff700 1 --2- 172.21.15.131:0/431464925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac158030 0x7fccac158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.885+0000 7fccb09ff700 1 -- 172.21.15.131:0/431464925 >> 172.21.15.131:0/431464925 conn(0x7fccac0ce3e0 msgr2=0x7fccac0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:36.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.885+0000 7fccb09ff700 1 -- 172.21.15.131:0/431464925 shutdown_connections 2023-12-26T21:53:36.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.885+0000 7fccb09ff700 1 -- 172.21.15.131:0/431464925 wait complete. 2023-12-26T21:53:36.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.886+0000 7fccb09ff700 1 Processor -- start 2023-12-26T21:53:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.886+0000 7fccb09ff700 1 -- start start 2023-12-26T21:53:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.887+0000 7fccb09ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac06b8c0 0x7fccac0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.887+0000 7fccb09ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac0c9130 0x7fccac0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.887+0000 7fccb09ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac158030 0x7fccac0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.887+0000 7fccb09ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac151b70 con 0x7fccac158030 2023-12-26T21:53:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.887+0000 7fccb09ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac145ad0 con 0x7fccac06b8c0 2023-12-26T21:53:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.887+0000 7fccb09ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac144ec0 con 0x7fccac0c9130 2023-12-26T21:53:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.887+0000 7fccaad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac158030 0x7fccac0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.887+0000 7fccaad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac158030 0x7fccac0b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43208/0 (socket says 172.21.15.131:43208) 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.887+0000 7fccaad9d700 1 -- 172.21.15.131:0/2489513813 learned_addr learned my addr 172.21.15.131:0/2489513813 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fccaa59c700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac06b8c0 0x7fccac0c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fcca9d9b700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac0c9130 0x7fccac0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fccaad9d700 1 -- 172.21.15.131:0/2489513813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac0c9130 msgr2=0x7fccac0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fccaad9d700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac0c9130 0x7fccac0b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fccaad9d700 1 -- 172.21.15.131:0/2489513813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac06b8c0 msgr2=0x7fccac0c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fccaad9d700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac06b8c0 0x7fccac0c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fccaad9d700 1 -- 172.21.15.131:0/2489513813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca4009ae0 con 0x7fccac158030 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fccaad9d700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac158030 0x7fccac0b16c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fccac06fe40 tx=0x7fcca4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fcc9a7fc700 1 -- 172.21.15.131:0/2489513813 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca40032f0 con 0x7fccac158030 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fcc9a7fc700 1 -- 172.21.15.131:0/2489513813 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca4008dd0 con 0x7fccac158030 2023-12-26T21:53:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac1451a0 con 0x7fccac158030 2023-12-26T21:53:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.888+0000 7fcc9a7fc700 1 -- 172.21.15.131:0/2489513813 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4026690 con 0x7fccac158030 2023-12-26T21:53:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.889+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac145760 con 0x7fccac158030 2023-12-26T21:53:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.889+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccac145c60 con 0x7fccac158030 2023-12-26T21:53:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.890+0000 7fcc9a7fc700 1 -- 172.21.15.131:0/2489513813 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcca4003040 con 0x7fccac158030 2023-12-26T21:53:36.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.891+0000 7fcc9a7fc700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcc8c066f30 0x7fcc8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:36.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.891+0000 7fcc9a7fc700 1 -- 172.21.15.131:0/2489513813 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fcca4086580 con 0x7fccac158030 2023-12-26T21:53:36.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.891+0000 7fccaa59c700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcc8c066f30 0x7fcc8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:36.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.892+0000 7fccaa59c700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcc8c066f30 0x7fcc8c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0061f0 tx=0x7fcc9c008040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:36.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:36.895+0000 7fcc9a7fc700 1 -- 172.21.15.131:0/2489513813 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcca4053730 con 0x7fccac158030 2023-12-26T21:53:37.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.087+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fccac0657e0 con 0x7fccac158030 2023-12-26T21:53:37.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.091+0000 7fcc9a7fc700 1 -- 172.21.15.131:0/2489513813 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v32) v1 ==== 115+0+0 (secure 0 0 0) 0x7fcca4056d50 con 0x7fccac158030 2023-12-26T21:53:37.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.094+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcc8c066f30 msgr2=0x7fcc8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.094+0000 7fccb09ff700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcc8c066f30 0x7fcc8c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0061f0 tx=0x7fcc9c008040 comp rx=0 tx=0).stop 2023-12-26T21:53:37.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.094+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac158030 msgr2=0x7fccac0b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.094+0000 7fccb09ff700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac158030 0x7fccac0b16c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fccac06fe40 tx=0x7fcca4018000 comp rx=0 tx=0).stop 2023-12-26T21:53:37.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.095+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 shutdown_connections 2023-12-26T21:53:37.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.095+0000 7fccb09ff700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcc8c066f30 0x7fcc8c0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.095+0000 7fccb09ff700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccac158030 0x7fccac0b16c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.095+0000 7fccb09ff700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac06b8c0 0x7fccac0c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.095+0000 7fccb09ff700 1 --2- 172.21.15.131:0/2489513813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac0c9130 0x7fccac0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.095+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 >> 172.21.15.131:0/2489513813 conn(0x7fccac0ce3e0 msgr2=0x7fccac0c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:37.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.095+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 shutdown_connections 2023-12-26T21:53:37.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.095+0000 7fccb09ff700 1 -- 172.21.15.131:0/2489513813 wait complete. 2023-12-26T21:53:37.117 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T21:53:37.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.311+0000 7fb241fff700 1 Processor -- start 2023-12-26T21:53:37.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.311+0000 7fb241fff700 1 -- start start 2023-12-26T21:53:37.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.312+0000 7fb241fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.312+0000 7fb241fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb23c069d10 0x7fb23c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.312+0000 7fb241fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c0caa10 con 0x7fb23c0c8820 2023-12-26T21:53:37.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.312+0000 7fb241fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c157a80 con 0x7fb23c069d10 2023-12-26T21:53:37.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.312+0000 7fb241fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0cac50 con 0x7fb23c0c8bf0 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.313+0000 7fb23bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb23c069d10 0x7fb23c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.313+0000 7fb23affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.313+0000 7fb23affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:46314/0 (socket says 172.21.15.131:46314) 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.313+0000 7fb23bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb23c069d10 0x7fb23c0b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:45328/0 (socket says 172.21.15.131:45328) 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.313+0000 7fb23affd700 1 -- 172.21.15.131:0/1606593392 learned_addr learned my addr 172.21.15.131:0/1606593392 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.313+0000 7fb23b7fe700 1 --1- 172.21.15.131:0/1606593392 >> v1:172.21.15.190:6789/0 conn(0x7fb23c0c8820 0x7fb23c0ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:37572/0 (socket says 172.21.15.131:37572) 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.313+0000 7fb23affd700 1 -- 172.21.15.131:0/1606593392 >> v2:172.21.15.190:3300/0 conn(0x7fb23c069d10 msgr2=0x7fb23c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.313+0000 7fb23affd700 1 --2- 172.21.15.131:0/1606593392 >> v2:172.21.15.190:3300/0 conn(0x7fb23c069d10 0x7fb23c0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.313+0000 7fb23affd700 1 -- 172.21.15.131:0/1606593392 >> v1:172.21.15.190:6789/0 conn(0x7fb23c0c8820 legacy=0x7fb23c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.314+0000 7fb23affd700 1 -- 172.21.15.131:0/1606593392 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c0caf30 con 0x7fb23c0c8bf0 2023-12-26T21:53:37.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.314+0000 7fb23bfff700 1 --2- 172.21.15.131:0/1606593392 >> v2:172.21.15.190:3300/0 conn(0x7fb23c069d10 0x7fb23c0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:37.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.314+0000 7fb23affd700 1 --2- 172.21.15.131:0/1606593392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b12e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2300099b0 tx=0x7fb230009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6e6796a0ec3c390 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:37.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.314+0000 7fb23a7fc700 1 -- 172.21.15.131:0/1606593392 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230009ea0 con 0x7fb23c0c8bf0 2023-12-26T21:53:37.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.314+0000 7fb23a7fc700 1 -- 172.21.15.131:0/1606593392 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23000b860 con 0x7fb23c0c8bf0 2023-12-26T21:53:37.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.314+0000 7fb23a7fc700 1 -- 172.21.15.131:0/1606593392 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000bc00 con 0x7fb23c0c8bf0 2023-12-26T21:53:37.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.315+0000 7fb241fff700 1 -- 172.21.15.131:0/1606593392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 msgr2=0x7fb23c0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.315+0000 7fb241fff700 1 --2- 172.21.15.131:0/1606593392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b12e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2300099b0 tx=0x7fb230009250 comp rx=0 tx=0).stop 2023-12-26T21:53:37.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.315+0000 7fb241fff700 1 -- 172.21.15.131:0/1606593392 shutdown_connections 2023-12-26T21:53:37.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.315+0000 7fb241fff700 1 --2- 172.21.15.131:0/1606593392 >> v2:172.21.15.190:3300/0 conn(0x7fb23c069d10 0x7fb23c0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.315+0000 7fb241fff700 1 --2- 172.21.15.131:0/1606593392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b12e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.315+0000 7fb241fff700 1 -- 172.21.15.131:0/1606593392 >> 172.21.15.131:0/1606593392 conn(0x7fb23c068040 msgr2=0x7fb23c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:37.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.315+0000 7fb241fff700 1 -- 172.21.15.131:0/1606593392 shutdown_connections 2023-12-26T21:53:37.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.315+0000 7fb241fff700 1 -- 172.21.15.131:0/1606593392 wait complete. 2023-12-26T21:53:37.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.316+0000 7fb241fff700 1 Processor -- start 2023-12-26T21:53:37.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.316+0000 7fb241fff700 1 -- start start 2023-12-26T21:53:37.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.317+0000 7fb241fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c069d10 0x7fb23c0c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.317+0000 7fb241fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb23c0c8820 0x7fb23c0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.317+0000 7fb241fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.317+0000 7fb241fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c1507f0 con 0x7fb23c0c8820 2023-12-26T21:53:37.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.317+0000 7fb241fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c150ad0 con 0x7fb23c069d10 2023-12-26T21:53:37.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.317+0000 7fb241fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c150db0 con 0x7fb23c0c8bf0 2023-12-26T21:53:37.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb23c0c8820 0x7fb23c0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb23c0c8820 0x7fb23c0b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43212/0 (socket says 172.21.15.131:43212) 2023-12-26T21:53:37.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23affd700 1 -- 172.21.15.131:0/2859922841 learned_addr learned my addr 172.21.15.131:0/2859922841 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:37.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23b7fe700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c069d10 0x7fb23c0c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23bfff700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23affd700 1 -- 172.21.15.131:0/2859922841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 msgr2=0x7fb23c0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23affd700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23affd700 1 -- 172.21.15.131:0/2859922841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c069d10 msgr2=0x7fb23c0c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23affd700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c069d10 0x7fb23c0c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23affd700 1 -- 172.21.15.131:0/2859922841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb234009a20 con 0x7fb23c0c8820 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb23affd700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb23c0c8820 0x7fb23c0b0420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0ae030 tx=0x7fb230017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.318+0000 7fb2237fe700 1 -- 172.21.15.131:0/2859922841 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000bbe0 con 0x7fb23c0c8820 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.319+0000 7fb2237fe700 1 -- 172.21.15.131:0/2859922841 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb230011420 con 0x7fb23c0c8820 2023-12-26T21:53:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.319+0000 7fb2237fe700 1 -- 172.21.15.131:0/2859922841 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2300117c0 con 0x7fb23c0c8820 2023-12-26T21:53:37.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.319+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300094f0 con 0x7fb23c0c8820 2023-12-26T21:53:37.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.319+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c151370 con 0x7fb23c0c8820 2023-12-26T21:53:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.320+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23c06b560 con 0x7fb23c0c8820 2023-12-26T21:53:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.321+0000 7fb2237fe700 1 -- 172.21.15.131:0/2859922841 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb230011bf0 con 0x7fb23c0c8820 2023-12-26T21:53:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.321+0000 7fb2237fe700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb228066f30 0x7fb2280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.321+0000 7fb2237fe700 1 -- 172.21.15.131:0/2859922841 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fb230085f90 con 0x7fb23c0c8820 2023-12-26T21:53:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.322+0000 7fb23b7fe700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb228066f30 0x7fb2280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.322+0000 7fb23b7fe700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb228066f30 0x7fb2280693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb22c004fd0 tx=0x7fb22c007eb0 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:37.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.325+0000 7fb2237fe700 1 -- 172.21.15.131:0/2859922841 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2300530a0 con 0x7fb23c0c8820 2023-12-26T21:53:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.527+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fb23c151590 con 0x7fb23c0c8820 2023-12-26T21:53:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.531+0000 7fb2237fe700 1 -- 172.21.15.131:0/2859922841 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7fb2300566c0 con 0x7fb23c0c8820 2023-12-26T21:53:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.533+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb228066f30 msgr2=0x7fb2280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.533+0000 7fb241fff700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb228066f30 0x7fb2280693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb22c004fd0 tx=0x7fb22c007eb0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.533+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb23c0c8820 msgr2=0x7fb23c0b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.533+0000 7fb241fff700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb23c0c8820 0x7fb23c0b0420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0ae030 tx=0x7fb230017000 comp rx=0 tx=0).stop 2023-12-26T21:53:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.534+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 shutdown_connections 2023-12-26T21:53:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.534+0000 7fb241fff700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c0c8bf0 0x7fb23c0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.534+0000 7fb241fff700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c069d10 0x7fb23c0c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.534+0000 7fb241fff700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb228066f30 0x7fb2280693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.534+0000 7fb241fff700 1 --2- 172.21.15.131:0/2859922841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb23c0c8820 0x7fb23c0b0420 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.534+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 >> 172.21.15.131:0/2859922841 conn(0x7fb23c068040 msgr2=0x7fb23c156720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.534+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 shutdown_connections 2023-12-26T21:53:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.534+0000 7fb241fff700 1 -- 172.21.15.131:0/2859922841 wait complete. 2023-12-26T21:53:37.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-12-26T21:53:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.740+0000 7fe8d63ff700 1 Processor -- start 2023-12-26T21:53:37.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.740+0000 7fe8d63ff700 1 -- start start 2023-12-26T21:53:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.741+0000 7fe8d63ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d0159920 0x7fe8d00b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.741+0000 7fe8d63ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8d0159cf0 0x7fe8d00b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.741+0000 7fe8d63ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe8d01554d0 0x7fe8d0155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.741+0000 7fe8d63ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d00c8bf0 con 0x7fe8d01554d0 2023-12-26T21:53:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.741+0000 7fe8d63ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d00c8e60 con 0x7fe8d0159cf0 2023-12-26T21:53:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.741+0000 7fe8d63ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00c9140 con 0x7fe8d0159920 2023-12-26T21:53:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.741+0000 7fe8d499c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe8d01554d0 0x7fe8d0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8d499c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe8d01554d0 0x7fe8d0155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43214/0 (socket says 172.21.15.131:43214) 2023-12-26T21:53:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8d499c700 1 -- 172.21.15.131:0/2127369086 learned_addr learned my addr 172.21.15.131:0/2127369086 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8cffff700 1 --2- 172.21.15.131:0/2127369086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d0159920 0x7fe8d00b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8cf7fe700 1 --2- 172.21.15.131:0/2127369086 >> v2:172.21.15.190:3300/0 conn(0x7fe8d0159cf0 0x7fe8d00b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8d499c700 1 -- 172.21.15.131:0/2127369086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d0159920 msgr2=0x7fe8d00b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8d499c700 1 --2- 172.21.15.131:0/2127369086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d0159920 0x7fe8d00b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8d499c700 1 -- 172.21.15.131:0/2127369086 >> v2:172.21.15.190:3300/0 conn(0x7fe8d0159cf0 msgr2=0x7fe8d00b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8d499c700 1 --2- 172.21.15.131:0/2127369086 >> v2:172.21.15.190:3300/0 conn(0x7fe8d0159cf0 0x7fe8d00b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8d499c700 1 -- 172.21.15.131:0/2127369086 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d00c9420 con 0x7fe8d01554d0 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8d499c700 1 --2- 172.21.15.131:0/2127369086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d01554d0 0x7fe8d0155930 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0068920 tx=0x7fe8c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c444c6bb6d967cdd server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.742+0000 7fe8ceffd700 1 -- 172.21.15.131:0/2127369086 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800e070 con 0x7fe8d01554d0 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8ceffd700 1 -- 172.21.15.131:0/2127369086 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c800b930 con 0x7fe8d01554d0 2023-12-26T21:53:37.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8ceffd700 1 -- 172.21.15.131:0/2127369086 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80092e0 con 0x7fe8d01554d0 2023-12-26T21:53:37.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8d63ff700 1 -- 172.21.15.131:0/2127369086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d01554d0 msgr2=0x7fe8d0155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/2127369086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d01554d0 0x7fe8d0155930 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0068920 tx=0x7fe8c8009840 comp rx=0 tx=0).stop 2023-12-26T21:53:37.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8d63ff700 1 -- 172.21.15.131:0/2127369086 shutdown_connections 2023-12-26T21:53:37.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/2127369086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d01554d0 0x7fe8d0155930 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/2127369086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d0159920 0x7fe8d00b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/2127369086 >> v2:172.21.15.190:3300/0 conn(0x7fe8d0159cf0 0x7fe8d00b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8d63ff700 1 -- 172.21.15.131:0/2127369086 >> 172.21.15.131:0/2127369086 conn(0x7fe8d0068040 msgr2=0x7fe8d0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8d63ff700 1 -- 172.21.15.131:0/2127369086 shutdown_connections 2023-12-26T21:53:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.743+0000 7fe8d63ff700 1 -- 172.21.15.131:0/2127369086 wait complete. 2023-12-26T21:53:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.745+0000 7fe8d63ff700 1 Processor -- start 2023-12-26T21:53:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.745+0000 7fe8d63ff700 1 -- start start 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d63ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d01554d0 0x7fe8d0158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d63ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159920 0x7fe8d01522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d63ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0159cf0 0x7fe8d0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d63ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d015a7e0 con 0x7fe8d0159cf0 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d63ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d01570f0 con 0x7fe8d0159920 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d63ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d015b2a0 con 0x7fe8d01554d0 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d499c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0159cf0 0x7fe8d0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d499c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0159cf0 0x7fe8d0157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43224/0 (socket says 172.21.15.131:43224) 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d499c700 1 -- 172.21.15.131:0/3667189141 learned_addr learned my addr 172.21.15.131:0/3667189141 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8cf7fe700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159920 0x7fe8d01522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8cffff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d01554d0 0x7fe8d0158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d499c700 1 -- 172.21.15.131:0/3667189141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d01554d0 msgr2=0x7fe8d0158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d499c700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d01554d0 0x7fe8d0158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d499c700 1 -- 172.21.15.131:0/3667189141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159920 msgr2=0x7fe8d01522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d499c700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159920 0x7fe8d01522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.746+0000 7fe8d499c700 1 -- 172.21.15.131:0/3667189141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c8009ae0 con 0x7fe8d0159cf0 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.747+0000 7fe8d499c700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0159cf0 0x7fe8d0157580 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009ab0 tx=0x7fe8c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.747+0000 7fe8b7fff700 1 -- 172.21.15.131:0/3667189141 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80032f0 con 0x7fe8d0159cf0 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.747+0000 7fe8b7fff700 1 -- 172.21.15.131:0/3667189141 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c8008b80 con 0x7fe8d0159cf0 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.747+0000 7fe8b7fff700 1 -- 172.21.15.131:0/3667189141 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8026460 con 0x7fe8d0159cf0 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.747+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d015b580 con 0x7fe8d0159cf0 2023-12-26T21:53:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.747+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d0156d80 con 0x7fe8d0159cf0 2023-12-26T21:53:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.748+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d0156900 con 0x7fe8d0159cf0 2023-12-26T21:53:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.749+0000 7fe8b7fff700 1 -- 172.21.15.131:0/3667189141 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe8c8003040 con 0x7fe8d0159cf0 2023-12-26T21:53:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.750+0000 7fe8b7fff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8bc066f30 0x7fe8bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.750+0000 7fe8b7fff700 1 -- 172.21.15.131:0/3667189141 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fe8c80863b0 con 0x7fe8d0159cf0 2023-12-26T21:53:37.751 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.750+0000 7fe8cffff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8bc066f30 0x7fe8bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:37.751 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.751+0000 7fe8cffff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8bc066f30 0x7fe8bc0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009d40 tx=0x7fe8c0008040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:37.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:37.754+0000 7fe8b7fff700 1 -- 172.21.15.131:0/3667189141 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8d0156900 con 0x7fe8d0159cf0 2023-12-26T21:53:38.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.022+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7fe8d0156900 con 0x7fe8d0159cf0 2023-12-26T21:53:38.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.235+0000 7fe8b7fff700 1 -- 172.21.15.131:0/3667189141 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v626) v1 ==== 166+0+0 (secure 0 0 0) 0x7fe8d0156900 con 0x7fe8d0159cf0 2023-12-26T21:53:38.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8bc066f30 msgr2=0x7fe8bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:38.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8bc066f30 0x7fe8bc0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009d40 tx=0x7fe8c0008040 comp rx=0 tx=0).stop 2023-12-26T21:53:38.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0159cf0 msgr2=0x7fe8d0157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0159cf0 0x7fe8d0157580 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009ab0 tx=0x7fe8c8018000 comp rx=0 tx=0).stop 2023-12-26T21:53:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 shutdown_connections 2023-12-26T21:53:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe8bc066f30 0x7fe8bc0693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8d0159cf0 0x7fe8d0157580 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d01554d0 0x7fe8d0158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 --2- 172.21.15.131:0/3667189141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d0159920 0x7fe8d01522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.237+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 >> 172.21.15.131:0/3667189141 conn(0x7fe8d0068040 msgr2=0x7fe8d00cc050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.238+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 shutdown_connections 2023-12-26T21:53:38.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.238+0000 7fe8d63ff700 1 -- 172.21.15.131:0/3667189141 wait complete. 2023-12-26T21:53:38.256 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-12-26T21:53:38.257 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-12-26T21:53:38.257 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T21:53:38.257 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-12-26T21:53:38.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.446+0000 7f9a1e1ff700 1 Processor -- start 2023-12-26T21:53:38.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.446+0000 7f9a1e1ff700 1 -- start start 2023-12-26T21:53:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.447+0000 7f9a1e1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a180cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.447+0000 7f9a1e1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a180b00e0 0x7f9a1806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.447+0000 7f9a1e1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a180b0550 0x7f9a1815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.447+0000 7f9a1e1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180680f0 con 0x7f9a180b00e0 2023-12-26T21:53:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.447+0000 7f9a1e1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180b0d70 con 0x7f9a180b0550 2023-12-26T21:53:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.447+0000 7f9a1e1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180675b0 con 0x7f9a180cc580 2023-12-26T21:53:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a16ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a180b00e0 0x7f9a1806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a16ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a180b00e0 0x7f9a1806e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43234/0 (socket says 172.21.15.131:43234) 2023-12-26T21:53:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a16ffd700 1 -- 172.21.15.131:0/2326779523 learned_addr learned my addr 172.21.15.131:0/2326779523 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a177fe700 1 --2- 172.21.15.131:0/2326779523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a180cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a17fff700 1 --2- 172.21.15.131:0/2326779523 >> v2:172.21.15.190:3300/0 conn(0x7f9a180b0550 0x7f9a1815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a16ffd700 1 -- 172.21.15.131:0/2326779523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 msgr2=0x7f9a180cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a16ffd700 1 --2- 172.21.15.131:0/2326779523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a180cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a16ffd700 1 -- 172.21.15.131:0/2326779523 >> v2:172.21.15.190:3300/0 conn(0x7f9a180b0550 msgr2=0x7f9a1815d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a16ffd700 1 --2- 172.21.15.131:0/2326779523 >> v2:172.21.15.190:3300/0 conn(0x7f9a180b0550 0x7f9a1815d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a16ffd700 1 -- 172.21.15.131:0/2326779523 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a18159c30 con 0x7f9a180b00e0 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a17fff700 1 --2- 172.21.15.131:0/2326779523 >> v2:172.21.15.190:3300/0 conn(0x7f9a180b0550 0x7f9a1815d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a177fe700 1 --2- 172.21.15.131:0/2326779523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a180cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a16ffd700 1 --2- 172.21.15.131:0/2326779523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 0x7f9a1806e5f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9a180b21c0 tx=0x7f9a0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31e04e6a33536e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.448+0000 7f9a167fc700 1 -- 172.21.15.131:0/2326779523 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00b860 con 0x7f9a180b00e0 2023-12-26T21:53:38.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a167fc700 1 -- 172.21.15.131:0/2326779523 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00b9c0 con 0x7f9a180b00e0 2023-12-26T21:53:38.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a167fc700 1 -- 172.21.15.131:0/2326779523 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00bd60 con 0x7f9a180b00e0 2023-12-26T21:53:38.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/2326779523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 msgr2=0x7f9a1806e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:38.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/2326779523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 0x7f9a1806e5f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9a180b21c0 tx=0x7f9a0c009250 comp rx=0 tx=0).stop 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/2326779523 shutdown_connections 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/2326779523 >> v2:172.21.15.190:3300/0 conn(0x7f9a180b0550 0x7f9a1815d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/2326779523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a180cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/2326779523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 0x7f9a1806e5f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/2326779523 >> 172.21.15.131:0/2326779523 conn(0x7f9a180686c0 msgr2=0x7f9a18068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/2326779523 shutdown_connections 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.449+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/2326779523 wait complete. 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.451+0000 7f9a1e1ff700 1 Processor -- start 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.451+0000 7f9a1e1ff700 1 -- start start 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a1e1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 0x7f9a181445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a1e1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a180b0550 0x7f9a18144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a1e1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a18145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a1e1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a181435b0 con 0x7f9a180b00e0 2023-12-26T21:53:38.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a1e1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18144130 con 0x7f9a180b0550 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a1e1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18143800 con 0x7f9a180cc580 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 0x7f9a181445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 0x7f9a181445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43242/0 (socket says 172.21.15.131:43242) 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a177fe700 1 -- 172.21.15.131:0/1330556805 learned_addr learned my addr 172.21.15.131:0/1330556805 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a16ffd700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a180b0550 0x7f9a18144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a17fff700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a18145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a177fe700 1 -- 172.21.15.131:0/1330556805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 msgr2=0x7f9a18145040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a177fe700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a18145040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a177fe700 1 -- 172.21.15.131:0/1330556805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a180b0550 msgr2=0x7f9a18144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a177fe700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a180b0550 0x7f9a18144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a177fe700 1 -- 172.21.15.131:0/1330556805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0094f0 con 0x7f9a180b00e0 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.452+0000 7f9a17fff700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a18145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:38.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.453+0000 7f9a177fe700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 0x7f9a181445c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800d860 tx=0x7f9a08008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:38.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.453+0000 7f99ff7fe700 1 -- 172.21.15.131:0/1330556805 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080125f0 con 0x7f9a180b00e0 2023-12-26T21:53:38.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.453+0000 7f99ff7fe700 1 -- 172.21.15.131:0/1330556805 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a08012750 con 0x7f9a180b00e0 2023-12-26T21:53:38.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.453+0000 7f99ff7fe700 1 -- 172.21.15.131:0/1330556805 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08012af0 con 0x7f9a180b00e0 2023-12-26T21:53:38.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.453+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a18143ae0 con 0x7f9a180b00e0 2023-12-26T21:53:38.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.453+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a18156e80 con 0x7f9a180b00e0 2023-12-26T21:53:38.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.454+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a18144410 con 0x7f9a180b00e0 2023-12-26T21:53:38.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.455+0000 7f99ff7fe700 1 -- 172.21.15.131:0/1330556805 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9a08014430 con 0x7f9a180b00e0 2023-12-26T21:53:38.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.456+0000 7f99ff7fe700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9a04066f30 0x7f9a040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:38.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.456+0000 7f99ff7fe700 1 -- 172.21.15.131:0/1330556805 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f9a080849d0 con 0x7f9a180b00e0 2023-12-26T21:53:38.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.456+0000 7f9a16ffd700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9a04066f30 0x7f9a040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:38.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.457+0000 7f9a16ffd700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9a04066f30 0x7f9a040693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9a180b21c0 tx=0x7f9a0c017000 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:38.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.460+0000 7f99ff7fe700 1 -- 172.21.15.131:0/1330556805 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a08051b80 con 0x7f9a180b00e0 2023-12-26T21:53:38.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:38.727+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f9a180657e0 con 0x7f9a180b00e0 2023-12-26T21:53:39.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.236+0000 7f99ff7fe700 1 -- 172.21.15.131:0/1330556805 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v627) v1 ==== 186+0+0 (secure 0 0 0) 0x7f9a080551a0 con 0x7f9a180b00e0 2023-12-26T21:53:39.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9a04066f30 msgr2=0x7f9a040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:39.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9a04066f30 0x7f9a040693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9a180b21c0 tx=0x7f9a0c017000 comp rx=0 tx=0).stop 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 msgr2=0x7f9a181445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 0x7f9a181445c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800d860 tx=0x7f9a08008040 comp rx=0 tx=0).stop 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 shutdown_connections 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a180cc580 0x7f9a18145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9a04066f30 0x7f9a040693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a180b00e0 0x7f9a181445c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 --2- 172.21.15.131:0/1330556805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a180b0550 0x7f9a18144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 >> 172.21.15.131:0/1330556805 conn(0x7f9a180686c0 msgr2=0x7f9a18068a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 shutdown_connections 2023-12-26T21:53:39.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.239+0000 7f9a1e1ff700 1 -- 172.21.15.131:0/1330556805 wait complete. 2023-12-26T21:53:39.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T21:53:39.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.450+0000 7f9ddf7ff700 1 Processor -- start 2023-12-26T21:53:39.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.451+0000 7f9ddf7ff700 1 -- start start 2023-12-26T21:53:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddf7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9dd80cd230 0x7f9dd80cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddf7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9dd80cd600 0x7f9dd815db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddf7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd80cb4d0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddf7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd8159b30 con 0x7f9dd80cd600 2023-12-26T21:53:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddf7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd815d700 con 0x7f9dd80cd230 2023-12-26T21:53:39.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddcd9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9dd80cd600 0x7f9dd815db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:39.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddcd9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9dd80cd600 0x7f9dd815db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43248/0 (socket says 172.21.15.131:43248) 2023-12-26T21:53:39.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddcd9a700 1 -- 172.21.15.131:0/255126015 learned_addr learned my addr 172.21.15.131:0/255126015 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:39.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddd59b700 1 --2- 172.21.15.131:0/255126015 >> v2:172.21.15.190:3300/0 conn(0x7f9dd80cd230 0x7f9dd80cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:39.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddcd9a700 1 -- 172.21.15.131:0/255126015 >> v2:172.21.15.190:3300/0 conn(0x7f9dd80cd230 msgr2=0x7f9dd80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:39.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddcd9a700 1 --2- 172.21.15.131:0/255126015 >> v2:172.21.15.190:3300/0 conn(0x7f9dd80cd230 0x7f9dd80cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddcd9a700 1 -- 172.21.15.131:0/255126015 >> v1:172.21.15.190:6789/0 conn(0x7f9dd8158fe0 legacy=0x7f9dd8159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:39.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.452+0000 7f9ddcd9a700 1 -- 172.21.15.131:0/255126015 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd80cbce0 con 0x7f9dd80cd600 2023-12-26T21:53:39.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.453+0000 7f9ddcd9a700 1 --2- 172.21.15.131:0/255126015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd80cd600 0x7f9dd815db90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80d16f0 tx=0x7f9dc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e23c09481e4162e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:39.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.453+0000 7f9dcffff700 1 -- 172.21.15.131:0/255126015 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc800b820 con 0x7f9dd80cd600 2023-12-26T21:53:39.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.453+0000 7f9dcffff700 1 -- 172.21.15.131:0/255126015 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc800b980 con 0x7f9dd80cd600 2023-12-26T21:53:39.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.453+0000 7f9dcffff700 1 -- 172.21.15.131:0/255126015 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc800bd20 con 0x7f9dd80cd600 2023-12-26T21:53:39.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.453+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/255126015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd80cd600 msgr2=0x7f9dd815db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:39.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.453+0000 7f9ddf7ff700 1 --2- 172.21.15.131:0/255126015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd80cd600 0x7f9dd815db90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80d16f0 tx=0x7f9dc8009250 comp rx=0 tx=0).stop 2023-12-26T21:53:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.454+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/255126015 shutdown_connections 2023-12-26T21:53:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.454+0000 7f9ddf7ff700 1 --2- 172.21.15.131:0/255126015 >> v2:172.21.15.190:3300/0 conn(0x7f9dd80cd230 0x7f9dd80cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.454+0000 7f9ddf7ff700 1 --2- 172.21.15.131:0/255126015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd80cd600 0x7f9dd815db90 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.454+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/255126015 >> 172.21.15.131:0/255126015 conn(0x7f9dd806ece0 msgr2=0x7f9dd806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.454+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/255126015 shutdown_connections 2023-12-26T21:53:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.454+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/255126015 wait complete. 2023-12-26T21:53:39.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.455+0000 7f9ddf7ff700 1 Processor -- start 2023-12-26T21:53:39.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.455+0000 7f9ddf7ff700 1 -- start start 2023-12-26T21:53:39.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.456+0000 7f9ddf7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dd80cd230 0x7f9dd816ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:39.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.456+0000 7f9ddf7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dd80cd600 0x7f9dd815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.456+0000 7f9ddf7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd8158fe0 0x7f9dd816c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.456+0000 7f9ddf7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd816b0b0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.456+0000 7f9ddf7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd816d060 con 0x7f9dd80cd600 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.456+0000 7f9ddf7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd816d340 con 0x7f9dd80cd230 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dddd9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd8158fe0 0x7f9dd816c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dddd9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd8158fe0 0x7f9dd816c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43262/0 (socket says 172.21.15.131:43262) 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dddd9c700 1 -- 172.21.15.131:0/2166391294 learned_addr learned my addr 172.21.15.131:0/2166391294 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9ddcd9a700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dd80cd600 0x7f9dd815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9ddd59b700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dd80cd230 0x7f9dd816ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dddd9c700 1 -- 172.21.15.131:0/2166391294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dd80cd230 msgr2=0x7f9dd816ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dddd9c700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dd80cd230 0x7f9dd816ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dddd9c700 1 -- 172.21.15.131:0/2166391294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dd80cd600 msgr2=0x7f9dd815b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dddd9c700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dd80cd600 0x7f9dd815b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dddd9c700 1 -- 172.21.15.131:0/2166391294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80094f0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dddd9c700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd8158fe0 0x7f9dd816c7b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000bc30 tx=0x7f9dd0009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.457+0000 7f9dcd7fa700 1 -- 172.21.15.131:0/2166391294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000c840 con 0x7f9dd8158fe0 2023-12-26T21:53:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.458+0000 7f9dcd7fa700 1 -- 172.21.15.131:0/2166391294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd000c9e0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.458+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd816d620 con 0x7f9dd8158fe0 2023-12-26T21:53:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.458+0000 7f9dcd7fa700 1 -- 172.21.15.131:0/2166391294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000cda0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.458+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd816dbe0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.459+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd80657e0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.460+0000 7f9dcd7fa700 1 -- 172.21.15.131:0/2166391294 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9dd0014620 con 0x7f9dd8158fe0 2023-12-26T21:53:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.460+0000 7f9dcd7fa700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9dbc066f30 0x7f9dbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.460+0000 7f9dcd7fa700 1 -- 172.21.15.131:0/2166391294 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f9dd00845d0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.461+0000 7f9ddd59b700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9dbc066f30 0x7f9dbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.461+0000 7f9ddd59b700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9dbc066f30 0x7f9dbc0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4006fd0 tx=0x7f9dd4008040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:39.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.465+0000 7f9dcd7fa700 1 -- 172.21.15.131:0/2166391294 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dd004c5a0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.659+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9dd80cb720 con 0x7f9dd8158fe0 2023-12-26T21:53:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.667+0000 7f9dcd7fa700 1 -- 172.21.15.131:0/2166391294 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9dd00516e0 con 0x7f9dd8158fe0 2023-12-26T21:53:39.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9dbc066f30 msgr2=0x7f9dbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9dbc066f30 0x7f9dbc0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4006fd0 tx=0x7f9dd4008040 comp rx=0 tx=0).stop 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd8158fe0 msgr2=0x7f9dd816c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd8158fe0 0x7f9dd816c7b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000bc30 tx=0x7f9dd0009820 comp rx=0 tx=0).stop 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 shutdown_connections 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9dbc066f30 0x7f9dbc0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dd8158fe0 0x7f9dd816c7b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dd80cd230 0x7f9dd816ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 --2- 172.21.15.131:0/2166391294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dd80cd600 0x7f9dd815b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 >> 172.21.15.131:0/2166391294 conn(0x7f9dd806ece0 msgr2=0x7f9dd80cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.669+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 shutdown_connections 2023-12-26T21:53:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:39.670+0000 7f9ddf7ff700 1 -- 172.21.15.131:0/2166391294 wait complete. 2023-12-26T21:53:39.687 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:53:39.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:53:39.784 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:53:39.890 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:53:39.890 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T21:53:39.890 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:53:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.080+0000 7f410c5ff700 1 Processor -- start 2023-12-26T21:53:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.080+0000 7f410c5ff700 1 -- start start 2023-12-26T21:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.081+0000 7f410c5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f41040c8820 0x7f41040ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.081+0000 7f410c5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f41040b19f0 0x7f4104161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.081+0000 7f410c5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4104157a80 con 0x7f41040c8bf0 2023-12-26T21:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.081+0000 7f410c5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41040cac50 con 0x7f41040c8820 2023-12-26T21:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.081+0000 7f410c5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41040caf30 con 0x7f41040b19f0 2023-12-26T21:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.081+0000 7f410a39b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f41040c8820 0x7f41040ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.081+0000 7f4109b9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f41040c8bf0 0x7f41040b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58524/0 (socket says 172.21.15.131:58524) 2023-12-26T21:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.081+0000 7f4109b9a700 1 -- 172.21.15.131:0/1193235337 learned_addr learned my addr 172.21.15.131:0/1193235337 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f410ab9c700 1 --2- 172.21.15.131:0/1193235337 >> v2:172.21.15.190:3300/0 conn(0x7f41040b19f0 0x7f4104161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f410a39b700 1 -- 172.21.15.131:0/1193235337 >> v2:172.21.15.190:3300/0 conn(0x7f41040b19f0 msgr2=0x7f4104161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f410a39b700 1 --2- 172.21.15.131:0/1193235337 >> v2:172.21.15.190:3300/0 conn(0x7f41040b19f0 0x7f4104161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f410a39b700 1 -- 172.21.15.131:0/1193235337 >> v1:172.21.15.131:6789/0 conn(0x7f41040c8bf0 legacy=0x7f41040b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f410a39b700 1 -- 172.21.15.131:0/1193235337 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41040cb210 con 0x7f41040c8820 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f4109399700 1 -- 172.21.15.131:0/1193235337 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466987485 0 0) 0x7f4104157a80 con 0x7f41040c8bf0 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f410ab9c700 1 --2- 172.21.15.131:0/1193235337 >> v2:172.21.15.190:3300/0 conn(0x7f41040b19f0 0x7f4104161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f410a39b700 1 --2- 172.21.15.131:0/1193235337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 0x7f41040ca300 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4104070f80 tx=0x7f41000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bc869f59654c1c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f4109399700 1 -- 172.21.15.131:0/1193235337 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410000b6b0 con 0x7f41040c8820 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f4109399700 1 -- 172.21.15.131:0/1193235337 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410000b810 con 0x7f41040c8820 2023-12-26T21:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.082+0000 7f4109399700 1 -- 172.21.15.131:0/1193235337 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410000bbb0 con 0x7f41040c8820 2023-12-26T21:53:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.083+0000 7f410c5ff700 1 -- 172.21.15.131:0/1193235337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 msgr2=0x7f41040ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.083+0000 7f410c5ff700 1 --2- 172.21.15.131:0/1193235337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 0x7f41040ca300 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4104070f80 tx=0x7f41000091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.083+0000 7f410c5ff700 1 -- 172.21.15.131:0/1193235337 shutdown_connections 2023-12-26T21:53:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.083+0000 7f410c5ff700 1 --2- 172.21.15.131:0/1193235337 >> v2:172.21.15.190:3300/0 conn(0x7f41040b19f0 0x7f4104161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.083+0000 7f410c5ff700 1 --2- 172.21.15.131:0/1193235337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 0x7f41040ca300 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.083+0000 7f410c5ff700 1 -- 172.21.15.131:0/1193235337 >> 172.21.15.131:0/1193235337 conn(0x7f4104068040 msgr2=0x7f4104069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.083+0000 7f410c5ff700 1 -- 172.21.15.131:0/1193235337 shutdown_connections 2023-12-26T21:53:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.083+0000 7f410c5ff700 1 -- 172.21.15.131:0/1193235337 wait complete. 2023-12-26T21:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.085+0000 7f410c5ff700 1 Processor -- start 2023-12-26T21:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.085+0000 7f410c5ff700 1 -- start start 2023-12-26T21:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f410c5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41040b19f0 0x7f4104147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f410c5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 0x7f410415b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f410c5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41040c8bf0 0x7f410415ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f410c5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41040c76f0 con 0x7f41040c8820 2023-12-26T21:53:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f410c5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4104158d00 con 0x7f41040b19f0 2023-12-26T21:53:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f410c5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4104158fe0 con 0x7f41040c8bf0 2023-12-26T21:53:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f4109b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 0x7f410415b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f4109b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 0x7f410415b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43274/0 (socket says 172.21.15.131:43274) 2023-12-26T21:53:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f4109b9a700 1 -- 172.21.15.131:0/1004885242 learned_addr learned my addr 172.21.15.131:0/1004885242 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f410a39b700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41040b19f0 0x7f4104147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f4109b9a700 1 -- 172.21.15.131:0/1004885242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41040c8bf0 msgr2=0x7f410415ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f4109b9a700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41040c8bf0 0x7f410415ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f4109b9a700 1 -- 172.21.15.131:0/1004885242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41040b19f0 msgr2=0x7f4104147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f4109b9a700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41040b19f0 0x7f4104147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f4109b9a700 1 -- 172.21.15.131:0/1004885242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100009420 con 0x7f41040c8820 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.086+0000 7f410a39b700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41040b19f0 0x7f4104147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.087+0000 7f4109b9a700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 0x7f410415b4f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c810 tx=0x7f40f4007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.087+0000 7f40fa7fc700 1 -- 172.21.15.131:0/1004885242 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4005c00 con 0x7f41040c8820 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.087+0000 7f40fa7fc700 1 -- 172.21.15.131:0/1004885242 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f4005d60 con 0x7f41040c8820 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.087+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41041592c0 con 0x7f41040c8820 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.087+0000 7f40fa7fc700 1 -- 172.21.15.131:0/1004885242 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4007320 con 0x7f41040c8820 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.087+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41040c6930 con 0x7f41040c8820 2023-12-26T21:53:40.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.088+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4104158b60 con 0x7f41040c8820 2023-12-26T21:53:40.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.089+0000 7f40fa7fc700 1 -- 172.21.15.131:0/1004885242 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f40f401b400 con 0x7f41040c8820 2023-12-26T21:53:40.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.090+0000 7f40fa7fc700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f40e8066f30 0x7f40e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.090+0000 7f40fa7fc700 1 -- 172.21.15.131:0/1004885242 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f40f40840d0 con 0x7f41040c8820 2023-12-26T21:53:40.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.090+0000 7f410a39b700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f40e8066f30 0x7f40e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.090+0000 7f410a39b700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f40e8066f30 0x7f40e80693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f410000c010 tx=0x7f4100017000 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:40.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.094+0000 7f40fa7fc700 1 -- 172.21.15.131:0/1004885242 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4104158b60 con 0x7f41040c8820 2023-12-26T21:53:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.330+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4104158b60 con 0x7f41040c8820 2023-12-26T21:53:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.331+0000 7f40fa7fc700 1 -- 172.21.15.131:0/1004885242 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f4104158b60 con 0x7f41040c8820 2023-12-26T21:53:40.331 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:40.331 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2955210822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2955210822},{"type":"v1","addr":"172.21.15.190:6835","nonce":2955210822}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6454,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1671363867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1671363867},{"type":"v1","addr":"172.21.15.131:6834","nonce":1671363867}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3807864961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3807864961},{"type":"v1","addr":"172.21.15.190:6834","nonce":3807864961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/18678440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":18678440},{"type":"v1","addr":"172.21.15.131:6835","nonce":18678440}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[]} 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f40e8066f30 msgr2=0x7f40e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f40e8066f30 0x7f40e80693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f410000c010 tx=0x7f4100017000 comp rx=0 tx=0).stop 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 msgr2=0x7f410415b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 0x7f410415b4f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c810 tx=0x7f40f4007db0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 shutdown_connections 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41040c8bf0 0x7f410415ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41040b19f0 0x7f4104147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f40e8066f30 0x7f40e80693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 --2- 172.21.15.131:0/1004885242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41040c8820 0x7f410415b4f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 >> 172.21.15.131:0/1004885242 conn(0x7f4104068040 msgr2=0x7f41040c2860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 shutdown_connections 2023-12-26T21:53:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.333+0000 7f410c5ff700 1 -- 172.21.15.131:0/1004885242 wait complete. 2023-12-26T21:53:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 44 2023-12-26T21:53:40.352 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T21:53:40.352 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T21:53:40.352 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T21:53:40.352 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:53:40.352 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T21:53:40.352 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T21:53:40.353 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T21:53:40.353 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:53:40.353 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T21:53:40.353 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T21:53:40.353 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T21:53:40.353 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:53:40.354 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T21:53:40.354 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T21:53:40.354 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T21:53:40.354 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:53:40.366 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T21:53:40.366 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T21:53:40.367 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T21:53:40.367 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T21:53:40.369 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T21:53:40.370 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T21:53:40.370 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T21:53:40.371 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T21:53:40.411 INFO:tasks.ceph.mds.a:Started 2023-12-26T21:53:40.411 DEBUG:teuthology.parallel:result is None 2023-12-26T21:53:40.413 INFO:tasks.ceph.mds.c:Started 2023-12-26T21:53:40.414 DEBUG:teuthology.parallel:result is None 2023-12-26T21:53:40.415 INFO:tasks.ceph.mds.d:Started 2023-12-26T21:53:40.416 DEBUG:teuthology.parallel:result is None 2023-12-26T21:53:40.418 INFO:tasks.ceph.mds.b:Started 2023-12-26T21:53:40.419 DEBUG:teuthology.parallel:result is None 2023-12-26T21:53:40.419 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T21:53:40.553 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T21:53:40.553 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:53:40.563 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T21:53:40.564 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:53:40.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.619+0000 7f2e21dff700 1 Processor -- start 2023-12-26T21:53:40.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.619+0000 7f2e21dff700 1 -- start start 2023-12-26T21:53:40.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e21dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 0x7f2e1c0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e21dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2e1c0b19f0 0x7f2e1c161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e21dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c157a80 con 0x7f2e1c0c8bf0 2023-12-26T21:53:40.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e21dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c0cac50 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e21dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0caf30 con 0x7f2e1c0c8820 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e1bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2e1c0b19f0 0x7f2e1c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e1affd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2e1c0c8bf0 0x7f2e1c0b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58536/0 (socket says 172.21.15.131:58536) 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e1affd700 1 -- 172.21.15.131:0/2539424064 learned_addr learned my addr 172.21.15.131:0/2539424064 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e1b7fe700 1 --2- 172.21.15.131:0/2539424064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 0x7f2e1c0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e1bfff700 1 -- 172.21.15.131:0/2539424064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 msgr2=0x7f2e1c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e1bfff700 1 --2- 172.21.15.131:0/2539424064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 0x7f2e1c0ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e1bfff700 1 -- 172.21.15.131:0/2539424064 >> v1:172.21.15.131:6789/0 conn(0x7f2e1c0c8bf0 legacy=0x7f2e1c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e1bfff700 1 -- 172.21.15.131:0/2539424064 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c0cb210 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.621+0000 7f2e1a7fc700 1 -- 172.21.15.131:0/2539424064 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386076885 0 0) 0x7f2e1c157a80 con 0x7f2e1c0c8bf0 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.622+0000 7f2e1bfff700 1 --2- 172.21.15.131:0/2539424064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 0x7f2e1c161090 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009e60 tx=0x7f2e14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b476100a96b3ad server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.622+0000 7f2e1a7fc700 1 -- 172.21.15.131:0/2539424064 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400e070 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.622+0000 7f2e1a7fc700 1 -- 172.21.15.131:0/2539424064 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1400bc30 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.622+0000 7f2e1a7fc700 1 -- 172.21.15.131:0/2539424064 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14012430 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.622+0000 7f2e21dff700 1 -- 172.21.15.131:0/2539424064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 msgr2=0x7f2e1c161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.622+0000 7f2e21dff700 1 --2- 172.21.15.131:0/2539424064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 0x7f2e1c161090 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009e60 tx=0x7f2e14009780 comp rx=0 tx=0).stop 2023-12-26T21:53:40.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.623+0000 7f2e21dff700 1 -- 172.21.15.131:0/2539424064 shutdown_connections 2023-12-26T21:53:40.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.623+0000 7f2e21dff700 1 --2- 172.21.15.131:0/2539424064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 0x7f2e1c161090 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.623+0000 7f2e21dff700 1 --2- 172.21.15.131:0/2539424064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 0x7f2e1c0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.623+0000 7f2e21dff700 1 -- 172.21.15.131:0/2539424064 >> 172.21.15.131:0/2539424064 conn(0x7f2e1c068040 msgr2=0x7f2e1c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:40.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.623+0000 7f2e21dff700 1 -- 172.21.15.131:0/2539424064 shutdown_connections 2023-12-26T21:53:40.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.623+0000 7f2e21dff700 1 -- 172.21.15.131:0/2539424064 wait complete. 2023-12-26T21:53:40.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.624+0000 7f2e21dff700 1 Processor -- start 2023-12-26T21:53:40.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.624+0000 7f2e21dff700 1 -- start start 2023-12-26T21:53:40.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e21dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 0x7f2e1c156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e21dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 0x7f2e1c15d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e21dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c0c8bf0 0x7f2e1c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e21dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c1694a0 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e21dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c15dca0 con 0x7f2e1c0c8bf0 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e21dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c15df80 con 0x7f2e1c0c8820 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e1b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 0x7f2e1c156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e1b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 0x7f2e1c156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43316/0 (socket says 172.21.15.131:43316) 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e1b7fe700 1 -- 172.21.15.131:0/1673541368 learned_addr learned my addr 172.21.15.131:0/1673541368 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e1bfff700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c0c8bf0 0x7f2e1c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.625+0000 7f2e1affd700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 0x7f2e1c15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e1b7fe700 1 -- 172.21.15.131:0/1673541368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 msgr2=0x7f2e1c15d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e1b7fe700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 0x7f2e1c15d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e1b7fe700 1 -- 172.21.15.131:0/1673541368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c0c8bf0 msgr2=0x7f2e1c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e1b7fe700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c0c8bf0 0x7f2e1c159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:40.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e1b7fe700 1 -- 172.21.15.131:0/1673541368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e14009a20 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e1b7fe700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 0x7f2e1c156150 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00d890 tx=0x7f2e0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:40.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e037fe700 1 -- 172.21.15.131:0/1673541368 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0125f0 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e037fe700 1 -- 172.21.15.131:0/1673541368 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c012750 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e037fe700 1 -- 172.21.15.131:0/1673541368 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c012af0 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c15e260 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.626+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c156690 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.628+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c0657e0 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.629+0000 7f2e037fe700 1 -- 172.21.15.131:0/1673541368 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2e0c01e460 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.630+0000 7f2e037fe700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2e08066f30 0x7f2e080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:40.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.630+0000 7f2e037fe700 1 -- 172.21.15.131:0/1673541368 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f2e0c084b60 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.630+0000 7f2e1affd700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2e08066f30 0x7f2e080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:40.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.631+0000 7f2e1affd700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2e08066f30 0x7f2e080693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c15ac20 tx=0x7f2e10007d30 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:40.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.634+0000 7f2e037fe700 1 -- 172.21.15.131:0/1673541368 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e0c051d10 con 0x7f2e1c0b19f0 2023-12-26T21:53:40.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:40.847+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2e1c0cb4f0 con 0x7f2e1c0b19f0 2023-12-26T21:53:41.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.698+0000 7f2e037fe700 1 -- 172.21.15.131:0/1673541368 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2e0c055330 con 0x7f2e1c0b19f0 2023-12-26T21:53:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2e08066f30 msgr2=0x7f2e080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2e08066f30 0x7f2e080693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c15ac20 tx=0x7f2e10007d30 comp rx=0 tx=0).stop 2023-12-26T21:53:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 msgr2=0x7f2e1c156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 0x7f2e1c156150 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00d890 tx=0x7f2e0c008040 comp rx=0 tx=0).stop 2023-12-26T21:53:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 shutdown_connections 2023-12-26T21:53:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1c0c8bf0 0x7f2e1c159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:41.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2e08066f30 0x7f2e080693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:41.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e1c0b19f0 0x7f2e1c156150 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:41.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 --2- 172.21.15.131:0/1673541368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e1c0c8820 0x7f2e1c15d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:41.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 >> 172.21.15.131:0/1673541368 conn(0x7f2e1c068040 msgr2=0x7f2e1c0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:41.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 shutdown_connections 2023-12-26T21:53:41.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.701+0000 7f2e21dff700 1 -- 172.21.15.131:0/1673541368 wait complete. 2023-12-26T21:53:41.703 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T21:53:41.723 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T21:53:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.917+0000 7f1a43fff700 1 Processor -- start 2023-12-26T21:53:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.917+0000 7f1a43fff700 1 -- start start 2023-12-26T21:53:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.918+0000 7f1a43fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 0x7f1a3c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.918+0000 7f1a43fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c0caa10 con 0x7f1a3c0c8820 2023-12-26T21:53:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.918+0000 7f1a43fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c157a80 con 0x7f1a3c069d10 2023-12-26T21:53:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.918+0000 7f1a43fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0cac50 con 0x7f1a3c0c8bf0 2023-12-26T21:53:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.918+0000 7f1a41d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1a3c0c8820 0x7f1a3c0ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58542/0 (socket says 172.21.15.131:58542) 2023-12-26T21:53:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.918+0000 7f1a41d9b700 1 -- 172.21.15.131:0/3799388442 learned_addr learned my addr 172.21.15.131:0/3799388442 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a4159a700 1 --2- 172.21.15.131:0/3799388442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 0x7f1a3c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a40d99700 1 -- 172.21.15.131:0/3799388442 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276814716 0 0) 0x7f1a38001110 con 0x7f1a3c0c8820 2023-12-26T21:53:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a40d99700 1 -- 172.21.15.131:0/3799388442 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a28003570 con 0x7f1a3c0c8820 2023-12-26T21:53:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a40d99700 1 -- 172.21.15.131:0/3799388442 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230621485 0 0) 0x7f1a3c157a80 con 0x7f1a3c069d10 2023-12-26T21:53:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a40d99700 1 -- 172.21.15.131:0/3799388442 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a38001110 con 0x7f1a3c069d10 2023-12-26T21:53:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a40d99700 1 -- 172.21.15.131:0/3799388442 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2487683997 0 0) 0x7f1a28003570 con 0x7f1a3c0c8820 2023-12-26T21:53:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a4159a700 1 -- 172.21.15.131:0/3799388442 >> v1:172.21.15.190:6789/0 conn(0x7f1a3c069d10 legacy=0x7f1a3c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a4159a700 1 -- 172.21.15.131:0/3799388442 >> v1:172.21.15.131:6789/0 conn(0x7f1a3c0c8820 legacy=0x7f1a3c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a4159a700 1 -- 172.21.15.131:0/3799388442 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0caf30 con 0x7f1a3c0c8bf0 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.919+0000 7f1a4159a700 1 --2- 172.21.15.131:0/3799388442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 0x7f1a3c0b12e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009910 tx=0x7f1a2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b49b1a79936b035 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a40d99700 1 -- 172.21.15.131:0/3799388442 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38001600 con 0x7f1a3c0c8bf0 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a40d99700 1 -- 172.21.15.131:0/3799388442 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c011420 con 0x7f1a3c0c8bf0 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a40d99700 1 -- 172.21.15.131:0/3799388442 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0117a0 con 0x7f1a3c0c8bf0 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a43fff700 1 -- 172.21.15.131:0/3799388442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 msgr2=0x7f1a3c0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a43fff700 1 --2- 172.21.15.131:0/3799388442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 0x7f1a3c0b12e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009910 tx=0x7f1a2c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a43fff700 1 -- 172.21.15.131:0/3799388442 shutdown_connections 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a43fff700 1 --2- 172.21.15.131:0/3799388442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 0x7f1a3c0b12e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a43fff700 1 -- 172.21.15.131:0/3799388442 >> 172.21.15.131:0/3799388442 conn(0x7f1a3c068040 msgr2=0x7f1a3c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a43fff700 1 -- 172.21.15.131:0/3799388442 shutdown_connections 2023-12-26T21:53:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.920+0000 7f1a43fff700 1 -- 172.21.15.131:0/3799388442 wait complete. 2023-12-26T21:53:41.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.922+0000 7f1a43fff700 1 Processor -- start 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.922+0000 7f1a43fff700 1 -- start start 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a43fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3c069d10 0x7f1a3c0c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a43fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a3c0c8820 0x7f1a3c0c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a43fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 0x7f1a3c1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a43fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0c48b0 con 0x7f1a3c0c8820 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a43fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0c4b50 con 0x7f1a3c069d10 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a43fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0c17e0 con 0x7f1a3c0c8bf0 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a4159a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a3c0c8820 0x7f1a3c0c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a4159a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a3c0c8820 0x7f1a3c0c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43326/0 (socket says 172.21.15.131:43326) 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a4159a700 1 -- 172.21.15.131:0/2851564901 learned_addr learned my addr 172.21.15.131:0/2851564901 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a41d9b700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3c069d10 0x7f1a3c0c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.923+0000 7f1a4259c700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 0x7f1a3c1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a4159a700 1 -- 172.21.15.131:0/2851564901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 msgr2=0x7f1a3c1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a4159a700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 0x7f1a3c1534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a4159a700 1 -- 172.21.15.131:0/2851564901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3c069d10 msgr2=0x7f1a3c0c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a4159a700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3c069d10 0x7f1a3c0c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a4159a700 1 -- 172.21.15.131:0/2851564901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c009420 con 0x7f1a3c0c8820 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a4159a700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a3c0c8820 0x7f1a3c0c0f90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0688f0 tx=0x7f1a2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a31ffb700 1 -- 172.21.15.131:0/2851564901 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c01caf0 con 0x7f1a3c0c8820 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a31ffb700 1 -- 172.21.15.131:0/2851564901 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c01cc70 con 0x7f1a3c0c8820 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a31ffb700 1 -- 172.21.15.131:0/2851564901 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c01a450 con 0x7f1a3c0c8820 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c0c1ac0 con 0x7f1a3c0c8820 2023-12-26T21:53:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.924+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c0c14d0 con 0x7f1a3c0c8820 2023-12-26T21:53:41.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.925+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c0657e0 con 0x7f1a3c0c8820 2023-12-26T21:53:41.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.926+0000 7f1a31ffb700 1 -- 172.21.15.131:0/2851564901 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1a2c01a7c0 con 0x7f1a3c0c8820 2023-12-26T21:53:41.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.927+0000 7f1a31ffb700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1a20066f30 0x7f1a200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:41.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.927+0000 7f1a31ffb700 1 -- 172.21.15.131:0/2851564901 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f1a2c085cc0 con 0x7f1a3c0c8820 2023-12-26T21:53:41.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.927+0000 7f1a41d9b700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1a20066f30 0x7f1a200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:41.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.928+0000 7f1a41d9b700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1a20066f30 0x7f1a200693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a380015d0 tx=0x7f1a38000bc0 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:41.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:41.931+0000 7f1a31ffb700 1 -- 172.21.15.131:0/2851564901 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a2c08c080 con 0x7f1a3c0c8820 2023-12-26T21:53:42.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.200+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1a3c0cb210 con 0x7f1a3c0c8820 2023-12-26T21:53:42.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.201+0000 7f1a31ffb700 1 -- 172.21.15.131:0/2851564901 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1a2c053060 con 0x7f1a3c0c8820 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.202 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.203 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.204 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.205 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T21:53:42.207 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:42.208 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:42.209 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.209 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.209 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.209 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.209 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T21:53:42.209 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T21:53:42.209 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.210 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:42.210 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:42.210 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.210 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.210 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.210 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T21:53:42.210 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T21:53:42.210 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.211 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:42.211 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:42.211 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.211 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.211 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.211 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.211 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T21:53:42.212 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T21:53:42.212 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.212 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:53:42.212 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:53:42.212 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.212 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.213 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.213 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.213 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T21:53:42.213 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T21:53:42.213 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.213 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:42.213 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T21:53:42.213 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T21:53:42.214 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.214 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.214 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.214 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T21:53:42.214 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T21:53:42.214 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.214 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:42.215 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:53:42.215 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:53:42.215 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:53:42.215 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.215 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.215 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.215 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T21:53:42.215 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T21:53:42.215 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.216 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:42.216 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:53:42.216 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:53:42.216 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:53:42.216 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.216 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.216 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.217 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T21:53:42.217 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T21:53:42.217 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.217 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:53:42.217 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:53:42.217 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:53:42.217 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:53:42.218 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.218 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.218 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.218 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T21:53:42.218 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T21:53:42.218 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.219 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:42.220 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T21:53:42.221 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1a20066f30 msgr2=0x7f1a200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1a20066f30 0x7f1a200693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a380015d0 tx=0x7f1a38000bc0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a3c0c8820 msgr2=0x7f1a3c0c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a3c0c8820 0x7f1a3c0c0f90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0688f0 tx=0x7f1a2c016000 comp rx=0 tx=0).stop 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 shutdown_connections 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3c0c8bf0 0x7f1a3c1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3c069d10 0x7f1a3c0c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1a20066f30 0x7f1a200693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 --2- 172.21.15.131:0/2851564901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a3c0c8820 0x7f1a3c0c0f90 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 >> 172.21.15.131:0/2851564901 conn(0x7f1a3c068040 msgr2=0x7f1a3c06ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:42.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 shutdown_connections 2023-12-26T21:53:42.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.203+0000 7f1a43fff700 1 -- 172.21.15.131:0/2851564901 wait complete. 2023-12-26T21:53:42.226 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.416+0000 7f35ccbff700 1 Processor -- start 2023-12-26T21:53:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.416+0000 7f35ccbff700 1 -- start start 2023-12-26T21:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.417+0000 7f35ccbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35c80c8bf0 0x7f35c80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.417+0000 7f35ccbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c80caa10 con 0x7f35c8069d10 2023-12-26T21:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.417+0000 7f35ccbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c8157a80 con 0x7f35c80c8820 2023-12-26T21:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.417+0000 7f35ccbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80cac50 con 0x7f35c80c8bf0 2023-12-26T21:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.417+0000 7f35c6d9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f35c8069d10 0x7f35c8161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58558/0 (socket says 172.21.15.131:58558) 2023-12-26T21:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.417+0000 7f35c6d9d700 1 -- 172.21.15.131:0/4143901199 learned_addr learned my addr 172.21.15.131:0/4143901199 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c5d9b700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35c80c8bf0 0x7f35c80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696578144 0 0) 0x7f35c0001490 con 0x7f35c8069d10 2023-12-26T21:53:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35b0003570 con 0x7f35c8069d10 2023-12-26T21:53:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245709950 0 0) 0x7f35c8157a80 con 0x7f35c80c8820 2023-12-26T21:53:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c0001490 con 0x7f35c80c8820 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3713519766 0 0) 0x7f35b0003570 con 0x7f35c8069d10 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35c80c8bf0 msgr2=0x7f35c80b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35c80c8bf0 0x7f35c80b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 >> v1:172.21.15.190:6789/0 conn(0x7f35c80c8820 legacy=0x7f35c80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c80caf30 con 0x7f35c8069d10 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f35c0001980 con 0x7f35c8069d10 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 >> v1:172.21.15.131:6789/0 conn(0x7f35c8069d10 legacy=0x7f35c8161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35c80c8bf0 0x7f35b0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b0004180 0x7f35b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b0004d50 0x7f35b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80cac50 con 0x7f35b0004180 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b000bba0 con 0x7f35b0004d50 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.418+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b000bd60 con 0x7f35c80c8bf0 2023-12-26T21:53:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.419+0000 7f35c5d9b700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b0004180 0x7f35b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.419+0000 7f35c659c700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35c80c8bf0 0x7f35b0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.419+0000 7f35c6d9d700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b0004d50 0x7f35b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.419+0000 7f35c5d9b700 1 -- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35c80c8bf0 msgr2=0x7f35b0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.419+0000 7f35c5d9b700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35c80c8bf0 0x7f35b0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.419+0000 7f35c5d9b700 1 -- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b0004d50 msgr2=0x7f35b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.419+0000 7f35c5d9b700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35b0004d50 0x7f35b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.419+0000 7f35c5d9b700 1 -- 172.21.15.131:0/4143901199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b0003570 con 0x7f35b0004180 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.419+0000 7f35c5d9b700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b0004180 0x7f35b0004810 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009c00 tx=0x7f35bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e831545a5ef7b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00e710 con 0x7f35b0004180 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc00e870 con 0x7f35b0004180 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35c559a700 1 -- 172.21.15.131:0/4143901199 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00ec30 con 0x7f35b0004180 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35ccbff700 1 -- 172.21.15.131:0/4143901199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b0004180 msgr2=0x7f35b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35ccbff700 1 --2- 172.21.15.131:0/4143901199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35b0004180 0x7f35b0004810 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009c00 tx=0x7f35bc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35c6d9d700 1 -- 172.21.15.131:0/4143901199 reap_dead start 2023-12-26T21:53:42.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35ccbff700 1 -- 172.21.15.131:0/4143901199 shutdown_connections 2023-12-26T21:53:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35ccbff700 1 -- 172.21.15.131:0/4143901199 >> 172.21.15.131:0/4143901199 conn(0x7f35c8068040 msgr2=0x7f35c8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35ccbff700 1 -- 172.21.15.131:0/4143901199 shutdown_connections 2023-12-26T21:53:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.420+0000 7f35ccbff700 1 -- 172.21.15.131:0/4143901199 wait complete. 2023-12-26T21:53:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.422+0000 7f35ccbff700 1 Processor -- start 2023-12-26T21:53:42.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.422+0000 7f35ccbff700 1 -- start start 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35ccbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35b0004180 0x7f35c81506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35ccbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35c81675d0 0x7f35c815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35ccbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35c8167a40 0x7f35c81584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35ccbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c8151c20 con 0x7f35c81675d0 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35ccbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c81512f0 con 0x7f35c8167a40 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35ccbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c81515d0 con 0x7f35b0004180 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35c5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35c81675d0 0x7f35c815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35c5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35c81675d0 0x7f35c815a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43352/0 (socket says 172.21.15.131:43352) 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35c5d9b700 1 -- 172.21.15.131:0/3371413887 learned_addr learned my addr 172.21.15.131:0/3371413887 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35c5d9b700 1 -- 172.21.15.131:0/3371413887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35b0004180 msgr2=0x7f35c81506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35c5d9b700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35b0004180 0x7f35c81506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35c5d9b700 1 -- 172.21.15.131:0/3371413887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35c8167a40 msgr2=0x7f35c81584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35c5d9b700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35c8167a40 0x7f35c81584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.423+0000 7f35c5d9b700 1 -- 172.21.15.131:0/3371413887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c81518b0 con 0x7f35c81675d0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.424+0000 7f35c5d9b700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35c81675d0 0x7f35c815a650 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f35b000bb70 tx=0x7f35bc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.424+0000 7f35b67fc700 1 -- 172.21.15.131:0/3371413887 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc022070 con 0x7f35c81675d0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.424+0000 7f35b67fc700 1 -- 172.21.15.131:0/3371413887 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc00e9e0 con 0x7f35c81675d0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.424+0000 7f35b67fc700 1 -- 172.21.15.131:0/3371413887 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc027470 con 0x7f35c81675d0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.424+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c8159200 con 0x7f35c81675d0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.424+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c815ad50 con 0x7f35c81675d0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.425+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c80657e0 con 0x7f35c81675d0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.426+0000 7f35b67fc700 1 -- 172.21.15.131:0/3371413887 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f35bc01c450 con 0x7f35c81675d0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.427+0000 7f35b67fc700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f35a8066f30 0x7f35a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.427+0000 7f35b67fc700 1 -- 172.21.15.131:0/3371413887 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f35bc085ff0 con 0x7f35c81675d0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.428+0000 7f35c659c700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f35a8066f30 0x7f35a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.428+0000 7f35c659c700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f35a8066f30 0x7f35a80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f35b80010e0 tx=0x7f35b8007d30 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:42.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.431+0000 7f35b67fc700 1 -- 172.21.15.131:0/3371413887 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35bc053390 con 0x7f35c81675d0 2023-12-26T21:53:42.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.626+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35c80cb210 con 0x7f35c81675d0 2023-12-26T21:53:42.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.627+0000 7f35b67fc700 1 -- 172.21.15.131:0/3371413887 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+12780 (secure 0 0 0) 0x7f35bc0569b0 con 0x7f35c81675d0 2023-12-26T21:53:42.628 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:42.629 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":106,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:41.691267+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:42.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f35a8066f30 msgr2=0x7f35a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f35a8066f30 0x7f35a80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f35b80010e0 tx=0x7f35b8007d30 comp rx=0 tx=0).stop 2023-12-26T21:53:42.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35c81675d0 msgr2=0x7f35c815a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35c81675d0 0x7f35c815a650 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f35b000bb70 tx=0x7f35bc01a000 comp rx=0 tx=0).stop 2023-12-26T21:53:42.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 shutdown_connections 2023-12-26T21:53:42.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35c8167a40 0x7f35c81584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35b0004180 0x7f35c81506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f35a8066f30 0x7f35a80693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 --2- 172.21.15.131:0/3371413887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35c81675d0 0x7f35c815a650 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 >> 172.21.15.131:0/3371413887 conn(0x7f35c8068040 msgr2=0x7f35c80c1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:42.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 shutdown_connections 2023-12-26T21:53:42.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.630+0000 7f35ccbff700 1 -- 172.21.15.131:0/3371413887 wait complete. 2023-12-26T21:53:42.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T21:53:42.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:53:42.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.841+0000 7f1593dff700 1 Processor -- start 2023-12-26T21:53:42.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.841+0000 7f1593dff700 1 -- start start 2023-12-26T21:53:42.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.842+0000 7f1593dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f158c0b0bf0 0x7f158c0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.842+0000 7f1593dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c156940 con 0x7f158c0b0750 2023-12-26T21:53:42.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.842+0000 7f1593dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0ae500 con 0x7f158c0cb320 2023-12-26T21:53:42.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.842+0000 7f1593dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0c8cc0 con 0x7f158c0b0bf0 2023-12-26T21:53:42.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.842+0000 7f159239c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f158c0b0bf0 0x7f158c0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.843+0000 7f159139a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f158c0b0750 0x7f158c0c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58570/0 (socket says 172.21.15.131:58570) 2023-12-26T21:53:42.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.843+0000 7f159139a700 1 -- 172.21.15.131:0/3689082992 learned_addr learned my addr 172.21.15.131:0/3689082992 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.843+0000 7f159239c700 1 -- 172.21.15.131:0/3689082992 >> v1:172.21.15.190:6789/0 conn(0x7f158c0cb320 legacy=0x7f158c069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.843+0000 7f159239c700 1 -- 172.21.15.131:0/3689082992 >> v1:172.21.15.131:6789/0 conn(0x7f158c0b0750 legacy=0x7f158c0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.843+0000 7f159239c700 1 -- 172.21.15.131:0/3689082992 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0b0280 con 0x7f158c0b0bf0 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.843+0000 7f159239c700 1 --2- 172.21.15.131:0/3689082992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0b0bf0 0x7f158c0c9b10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1584009e60 tx=0x7f1584009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e065c5d0f2cae3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.843+0000 7f1590b99700 1 -- 172.21.15.131:0/3689082992 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400e070 con 0x7f158c0b0bf0 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.844+0000 7f1590b99700 1 -- 172.21.15.131:0/3689082992 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158400bc30 con 0x7f158c0b0bf0 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.844+0000 7f1590b99700 1 -- 172.21.15.131:0/3689082992 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584012430 con 0x7f158c0b0bf0 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.844+0000 7f1593dff700 1 -- 172.21.15.131:0/3689082992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0b0bf0 msgr2=0x7f158c0c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.844+0000 7f1593dff700 1 --2- 172.21.15.131:0/3689082992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0b0bf0 0x7f158c0c9b10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1584009e60 tx=0x7f1584009780 comp rx=0 tx=0).stop 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.844+0000 7f1593dff700 1 -- 172.21.15.131:0/3689082992 shutdown_connections 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.844+0000 7f1593dff700 1 --2- 172.21.15.131:0/3689082992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0b0bf0 0x7f158c0c9b10 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.844+0000 7f1593dff700 1 -- 172.21.15.131:0/3689082992 >> 172.21.15.131:0/3689082992 conn(0x7f158c068040 msgr2=0x7f158c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:42.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.844+0000 7f1593dff700 1 -- 172.21.15.131:0/3689082992 shutdown_connections 2023-12-26T21:53:42.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.844+0000 7f1593dff700 1 -- 172.21.15.131:0/3689082992 wait complete. 2023-12-26T21:53:42.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.846+0000 7f1593dff700 1 Processor -- start 2023-12-26T21:53:42.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.846+0000 7f1593dff700 1 -- start start 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f1593dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f158c0b0750 0x7f158c13eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f1593dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f158c0b0bf0 0x7f158c06d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f1593dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0cb320 0x7f158c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f1593dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c150ac0 con 0x7f158c0cb320 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f1593dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c150da0 con 0x7f158c0b0bf0 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f1593dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c151080 con 0x7f158c0b0750 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f159239c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0cb320 0x7f158c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f159239c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0cb320 0x7f158c06def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43358/0 (socket says 172.21.15.131:43358) 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f159239c700 1 -- 172.21.15.131:0/3762296642 learned_addr learned my addr 172.21.15.131:0/3762296642 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f1591b9b700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f158c0b0750 0x7f158c13eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f159139a700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f158c0b0bf0 0x7f158c06d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f159239c700 1 -- 172.21.15.131:0/3762296642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f158c0b0750 msgr2=0x7f158c13eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f159239c700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f158c0b0750 0x7f158c13eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f159239c700 1 -- 172.21.15.131:0/3762296642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f158c0b0bf0 msgr2=0x7f158c06d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:42.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f159239c700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f158c0b0bf0 0x7f158c06d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:42.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.847+0000 7f159239c700 1 -- 172.21.15.131:0/3762296642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584009a20 con 0x7f158c0cb320 2023-12-26T21:53:42.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.848+0000 7f159239c700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0cb320 0x7f158c06def0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1584013000 tx=0x7f1584019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:42.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.848+0000 7f1581ffb700 1 -- 172.21.15.131:0/3762296642 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158401e700 con 0x7f158c0cb320 2023-12-26T21:53:42.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.848+0000 7f1581ffb700 1 -- 172.21.15.131:0/3762296642 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158401e880 con 0x7f158c0cb320 2023-12-26T21:53:42.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.848+0000 7f1581ffb700 1 -- 172.21.15.131:0/3762296642 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158401ec40 con 0x7f158c0cb320 2023-12-26T21:53:42.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.848+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c151360 con 0x7f158c0cb320 2023-12-26T21:53:42.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.848+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c13f050 con 0x7f158c0cb320 2023-12-26T21:53:42.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.849+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c0657e0 con 0x7f158c0cb320 2023-12-26T21:53:42.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.850+0000 7f1581ffb700 1 -- 172.21.15.131:0/3762296642 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1584024070 con 0x7f158c0cb320 2023-12-26T21:53:42.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.851+0000 7f1581ffb700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1570066f30 0x7f15700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:42.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.851+0000 7f1581ffb700 1 -- 172.21.15.131:0/3762296642 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f1584086e10 con 0x7f158c0cb320 2023-12-26T21:53:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.851+0000 7f1591b9b700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1570066f30 0x7f15700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.852+0000 7f1591b9b700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1570066f30 0x7f15700693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1588000c00 tx=0x7f1588009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:42.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:42.855+0000 7f1581ffb700 1 -- 172.21.15.131:0/3762296642 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f158408a030 con 0x7f158c0cb320 2023-12-26T21:53:43.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.046+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f158c151880 con 0x7f158c0cb320 2023-12-26T21:53:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.046+0000 7f1581ffb700 1 -- 172.21.15.131:0/3762296642 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f158c151880 con 0x7f158c0cb320 2023-12-26T21:53:43.047 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:53:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1570066f30 msgr2=0x7f15700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1570066f30 0x7f15700693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1588000c00 tx=0x7f1588009040 comp rx=0 tx=0).stop 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0cb320 msgr2=0x7f158c06def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0cb320 0x7f158c06def0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1584013000 tx=0x7f1584019000 comp rx=0 tx=0).stop 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 shutdown_connections 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1570066f30 0x7f15700693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c0cb320 0x7f158c06def0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f158c0b0750 0x7f158c13eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 --2- 172.21.15.131:0/3762296642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f158c0b0bf0 0x7f158c06d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 >> 172.21.15.131:0/3762296642 conn(0x7f158c068040 msgr2=0x7f158c0c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 shutdown_connections 2023-12-26T21:53:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.049+0000 7f1593dff700 1 -- 172.21.15.131:0/3762296642 wait complete. 2023-12-26T21:53:43.067 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T21:53:43.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.255+0000 7f5615fff700 1 Processor -- start 2023-12-26T21:53:43.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.255+0000 7f5615fff700 1 -- start start 2023-12-26T21:53:43.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.256+0000 7f5615fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5610159820 0x7f56100abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.256+0000 7f5615fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 0x7f561006dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.256+0000 7f5615fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561015a8c0 con 0x7f561006e2b0 2023-12-26T21:53:43.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.256+0000 7f5615fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561015ad50 con 0x7f5610159820 2023-12-26T21:53:43.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.256+0000 7f5615fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561006e730 con 0x7f56100aa480 2023-12-26T21:53:43.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5610159820 0x7f56100abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560ffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f561006e2b0 0x7f561015d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58578/0 (socket says 172.21.15.131:58578) 2023-12-26T21:53:43.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560ffff700 1 -- 172.21.15.131:0/467434518 learned_addr learned my addr 172.21.15.131:0/467434518 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:43.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560effd700 1 --2- 172.21.15.131:0/467434518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 0x7f561006dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560f7fe700 1 -- 172.21.15.131:0/467434518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 msgr2=0x7f561006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:43.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560f7fe700 1 --2- 172.21.15.131:0/467434518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 0x7f561006dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560f7fe700 1 -- 172.21.15.131:0/467434518 >> v1:172.21.15.131:6789/0 conn(0x7f561006e2b0 legacy=0x7f561015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560f7fe700 1 -- 172.21.15.131:0/467434518 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5610159f90 con 0x7f5610159820 2023-12-26T21:53:43.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560e7fc700 1 -- 172.21.15.131:0/467434518 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768881964 0 0) 0x7f561015a8c0 con 0x7f561006e2b0 2023-12-26T21:53:43.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.257+0000 7f560f7fe700 1 --2- 172.21.15.131:0/467434518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5610159820 0x7f56100abdb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5600009910 tx=0x7f56000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be04752c35427477 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:43.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f560e7fc700 1 -- 172.21.15.131:0/467434518 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560000b640 con 0x7f5610159820 2023-12-26T21:53:43.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f560e7fc700 1 -- 172.21.15.131:0/467434518 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560000b7a0 con 0x7f5610159820 2023-12-26T21:53:43.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f560e7fc700 1 -- 172.21.15.131:0/467434518 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560000bb60 con 0x7f5610159820 2023-12-26T21:53:43.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f5615fff700 1 -- 172.21.15.131:0/467434518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5610159820 msgr2=0x7f56100abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f5615fff700 1 --2- 172.21.15.131:0/467434518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5610159820 0x7f56100abdb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5600009910 tx=0x7f56000091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f5615fff700 1 -- 172.21.15.131:0/467434518 shutdown_connections 2023-12-26T21:53:43.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f5615fff700 1 --2- 172.21.15.131:0/467434518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5610159820 0x7f56100abdb0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f5615fff700 1 --2- 172.21.15.131:0/467434518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 0x7f561006dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f5615fff700 1 -- 172.21.15.131:0/467434518 >> 172.21.15.131:0/467434518 conn(0x7f561006c290 msgr2=0x7f56100ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:43.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f5615fff700 1 -- 172.21.15.131:0/467434518 shutdown_connections 2023-12-26T21:53:43.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.258+0000 7f5615fff700 1 -- 172.21.15.131:0/467434518 wait complete. 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.260+0000 7f5615fff700 1 Processor -- start 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.260+0000 7f5615fff700 1 -- start start 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f5615fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561006e2b0 0x7f561016c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f5615fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 0x7f5610155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f5615fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5610159820 0x7f561016ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f5615fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561016cdf0 con 0x7f561006e2b0 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f5615fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561016c420 con 0x7f5610159820 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f5615fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5610169e20 con 0x7f56100aa480 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f560f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561006e2b0 0x7f561016c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f560f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561006e2b0 0x7f561016c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43378/0 (socket says 172.21.15.131:43378) 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f560f7fe700 1 -- 172.21.15.131:0/2936055891 learned_addr learned my addr 172.21.15.131:0/2936055891 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f560effd700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 0x7f5610155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f560ffff700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5610159820 0x7f561016ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f560f7fe700 1 -- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 msgr2=0x7f5610155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f560f7fe700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 0x7f5610155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.261+0000 7f560f7fe700 1 -- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5610159820 msgr2=0x7f561016ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f560f7fe700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5610159820 0x7f561016ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f560f7fe700 1 -- 172.21.15.131:0/2936055891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5600009420 con 0x7f561006e2b0 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f560effd700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 0x7f5610155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:43.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f560ffff700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5610159820 0x7f561016ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f560f7fe700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561006e2b0 0x7f561016c8b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f560000b6e0 tx=0x7f5600017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f55f77fe700 1 -- 172.21.15.131:0/2936055891 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560001f070 con 0x7f561006e2b0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f55f77fe700 1 -- 172.21.15.131:0/2936055891 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5600000c90 con 0x7f561006e2b0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f55f77fe700 1 -- 172.21.15.131:0/2936055891 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560000bb40 con 0x7f561006e2b0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561016b810 con 0x7f561006e2b0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.262+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561016bdd0 con 0x7f561006e2b0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.263+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56100657e0 con 0x7f561006e2b0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.264+0000 7f55f77fe700 1 -- 172.21.15.131:0/2936055891 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5600023070 con 0x7f561006e2b0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.265+0000 7f55f77fe700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55fc066f30 0x7f55fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.266+0000 7f55f77fe700 1 -- 172.21.15.131:0/2936055891 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f56000857b0 con 0x7f561006e2b0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.266+0000 7f560effd700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55fc066f30 0x7f55fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.266+0000 7f560effd700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55fc066f30 0x7f55fc0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5604006f00 tx=0x7f5604008040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:43.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.270+0000 7f55f77fe700 1 -- 172.21.15.131:0/2936055891 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5600052bf0 con 0x7f561006e2b0 2023-12-26T21:53:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.461+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f561015a270 con 0x7f561006e2b0 2023-12-26T21:53:43.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.462+0000 7f55f77fe700 1 -- 172.21.15.131:0/2936055891 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v106) v1 ==== 84+0+65 (secure 0 0 0) 0x7f560002a090 con 0x7f561006e2b0 2023-12-26T21:53:43.462 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:43.463 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:53:43.463 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:43.463 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T21:53:43.463 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T21:53:43.463 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:43.463 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55fc066f30 msgr2=0x7f55fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55fc066f30 0x7f55fc0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5604006f00 tx=0x7f5604008040 comp rx=0 tx=0).stop 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561006e2b0 msgr2=0x7f561016c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561006e2b0 0x7f561016c8b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f560000b6e0 tx=0x7f5600017000 comp rx=0 tx=0).stop 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 shutdown_connections 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5610159820 0x7f561016ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55fc066f30 0x7f55fc0693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561006e2b0 0x7f561016c8b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 --2- 172.21.15.131:0/2936055891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56100aa480 0x7f5610155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 >> 172.21.15.131:0/2936055891 conn(0x7f561006c290 msgr2=0x7f5610156500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:43.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 shutdown_connections 2023-12-26T21:53:43.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.464+0000 7f5615fff700 1 -- 172.21.15.131:0/2936055891 wait complete. 2023-12-26T21:53:43.483 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T21:53:43.483 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T21:53:43.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.676+0000 7f43c13ff700 1 Processor -- start 2023-12-26T21:53:43.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.676+0000 7f43c13ff700 1 -- start start 2023-12-26T21:53:43.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.677+0000 7f43c13ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 0x7f43bc069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.677+0000 7f43c13ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc069640 con 0x7f43bc142f00 2023-12-26T21:53:43.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.677+0000 7f43c13ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc068750 con 0x7f43bc0cc590 2023-12-26T21:53:43.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.677+0000 7f43c13ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc068a30 con 0x7f43bc0cb510 2023-12-26T21:53:43.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43baffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f43bc142f00 0x7f43bc0cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58590/0 (socket says 172.21.15.131:58590) 2023-12-26T21:53:43.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43baffd700 1 -- 172.21.15.131:0/1700574936 learned_addr learned my addr 172.21.15.131:0/1700574936 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:43.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43ba7fc700 1 --2- 172.21.15.131:0/1700574936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 0x7f43bc069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43b9ffb700 1 -- 172.21.15.131:0/1700574936 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214781858 0 0) 0x7f43bc069640 con 0x7f43bc142f00 2023-12-26T21:53:43.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43b9ffb700 1 -- 172.21.15.131:0/1700574936 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a4003570 con 0x7f43bc142f00 2023-12-26T21:53:43.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43b9ffb700 1 -- 172.21.15.131:0/1700574936 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242317788 0 0) 0x7f43bc068750 con 0x7f43bc0cc590 2023-12-26T21:53:43.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43b9ffb700 1 -- 172.21.15.131:0/1700574936 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43bc069640 con 0x7f43bc0cc590 2023-12-26T21:53:43.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43ba7fc700 1 -- 172.21.15.131:0/1700574936 >> v1:172.21.15.190:6789/0 conn(0x7f43bc0cc590 legacy=0x7f43bc0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43ba7fc700 1 -- 172.21.15.131:0/1700574936 >> v1:172.21.15.131:6789/0 conn(0x7f43bc142f00 legacy=0x7f43bc0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43ba7fc700 1 -- 172.21.15.131:0/1700574936 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc068d10 con 0x7f43bc0cb510 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.678+0000 7f43ba7fc700 1 --2- 172.21.15.131:0/1700574936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 0x7f43bc069ad0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009910 tx=0x7f43b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b40eeb5dafe113 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.679+0000 7f43b9ffb700 1 -- 172.21.15.131:0/1700574936 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0011630 con 0x7f43bc0cb510 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.679+0000 7f43b9ffb700 1 -- 172.21.15.131:0/1700574936 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b0011790 con 0x7f43bc0cb510 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.679+0000 7f43b9ffb700 1 -- 172.21.15.131:0/1700574936 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0011b30 con 0x7f43bc0cb510 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.679+0000 7f43c13ff700 1 -- 172.21.15.131:0/1700574936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 msgr2=0x7f43bc069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.679+0000 7f43c13ff700 1 --2- 172.21.15.131:0/1700574936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 0x7f43bc069ad0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009910 tx=0x7f43b00091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.679+0000 7f43c13ff700 1 -- 172.21.15.131:0/1700574936 shutdown_connections 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.679+0000 7f43c13ff700 1 --2- 172.21.15.131:0/1700574936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 0x7f43bc069ad0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.679+0000 7f43c13ff700 1 -- 172.21.15.131:0/1700574936 >> 172.21.15.131:0/1700574936 conn(0x7f43bc063cd0 msgr2=0x7f43bc06b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.679+0000 7f43c13ff700 1 -- 172.21.15.131:0/1700574936 shutdown_connections 2023-12-26T21:53:43.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.680+0000 7f43c13ff700 1 -- 172.21.15.131:0/1700574936 wait complete. 2023-12-26T21:53:43.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.681+0000 7f43c13ff700 1 Processor -- start 2023-12-26T21:53:43.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.681+0000 7f43c13ff700 1 -- start start 2023-12-26T21:53:43.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43c13ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 0x7f43bc13ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43c13ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cc590 0x7f43bc0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43c13ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43bc142f00 0x7f43bc06ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43c13ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc169a60 con 0x7f43bc0cc590 2023-12-26T21:53:43.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43c13ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0c6d70 con 0x7f43bc142f00 2023-12-26T21:53:43.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43c13ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0c7050 con 0x7f43bc0cb510 2023-12-26T21:53:43.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43ba7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cc590 0x7f43bc0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43ba7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cc590 0x7f43bc0c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43394/0 (socket says 172.21.15.131:43394) 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43ba7fc700 1 -- 172.21.15.131:0/3457897119 learned_addr learned my addr 172.21.15.131:0/3457897119 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43baffd700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 0x7f43bc13ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.682+0000 7f43bb7fe700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43bc142f00 0x7f43bc06ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43ba7fc700 1 -- 172.21.15.131:0/3457897119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 msgr2=0x7f43bc13ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43ba7fc700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 0x7f43bc13ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43ba7fc700 1 -- 172.21.15.131:0/3457897119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43bc142f00 msgr2=0x7f43bc06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43ba7fc700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43bc142f00 0x7f43bc06ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43ba7fc700 1 -- 172.21.15.131:0/3457897119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0009420 con 0x7f43bc0cc590 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43ba7fc700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cc590 0x7f43bc0c6360 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f43b00091e0 tx=0x7f43b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43a2ffd700 1 -- 172.21.15.131:0/3457897119 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b001f070 con 0x7f43bc0cc590 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43a2ffd700 1 -- 172.21.15.131:0/3457897119 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b0000bc0 con 0x7f43bc0cc590 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43a2ffd700 1 -- 172.21.15.131:0/3457897119 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0025420 con 0x7f43bc0cc590 2023-12-26T21:53:43.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc0c7330 con 0x7f43bc0cc590 2023-12-26T21:53:43.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.683+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc13f480 con 0x7f43bc0cc590 2023-12-26T21:53:43.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.684+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc169bd0 con 0x7f43bc0cc590 2023-12-26T21:53:43.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.685+0000 7f43a2ffd700 1 -- 172.21.15.131:0/3457897119 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f43b0023070 con 0x7f43bc0cc590 2023-12-26T21:53:43.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.686+0000 7f43a2ffd700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43a8066f30 0x7f43a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:43.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.687+0000 7f43a2ffd700 1 -- 172.21.15.131:0/3457897119 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f43b0085760 con 0x7f43bc0cc590 2023-12-26T21:53:43.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.687+0000 7f43baffd700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43a8066f30 0x7f43a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:43.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.687+0000 7f43baffd700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43a8066f30 0x7f43a80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0058d0 tx=0x7f43ac009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:43.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.691+0000 7f43a2ffd700 1 -- 172.21.15.131:0/3457897119 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43bc169bd0 con 0x7f43bc0cc590 2023-12-26T21:53:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:43.882+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f43bc169ba0 con 0x7f43bc0cc590 2023-12-26T21:53:44.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.257+0000 7f43a2ffd700 1 -- 172.21.15.131:0/3457897119 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v107) v1 ==== 138+0+0 (secure 0 0 0) 0x7f43b002a0b0 con 0x7f43bc0cc590 2023-12-26T21:53:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43a8066f30 msgr2=0x7f43a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43a8066f30 0x7f43a80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0058d0 tx=0x7f43ac009040 comp rx=0 tx=0).stop 2023-12-26T21:53:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cc590 msgr2=0x7f43bc0c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cc590 0x7f43bc0c6360 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f43b00091e0 tx=0x7f43b0017000 comp rx=0 tx=0).stop 2023-12-26T21:53:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 shutdown_connections 2023-12-26T21:53:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43bc142f00 0x7f43bc06ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43bc0cb510 0x7f43bc13ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43a8066f30 0x7f43a80693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 --2- 172.21.15.131:0/3457897119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43bc0cc590 0x7f43bc0c6360 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.263+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 >> 172.21.15.131:0/3457897119 conn(0x7f43bc063cd0 msgr2=0x7f43bc13fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.264+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 shutdown_connections 2023-12-26T21:53:44.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.264+0000 7f43c13ff700 1 -- 172.21.15.131:0/3457897119 wait complete. 2023-12-26T21:53:44.268 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T21:53:44.283 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T21:53:44.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.480+0000 7f53ec5ff700 1 Processor -- start 2023-12-26T21:53:44.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.480+0000 7f53ec5ff700 1 -- start start 2023-12-26T21:53:44.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.481+0000 7f53ec5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 0x7f53e40686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.481+0000 7f53ec5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f53e4157420 0x7f53e4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.481+0000 7f53ec5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53e4068c00 0x7f53e40cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.481+0000 7f53ec5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40c9520 con 0x7f53e4157420 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.481+0000 7f53ec5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40caed0 con 0x7f53e4068c00 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.481+0000 7f53ec5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40ca8e0 con 0x7f53e4157050 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9b9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f53e4157420 0x7f53e4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9b9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f53e4157420 0x7f53e4159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43398/0 (socket says 172.21.15.131:43398) 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9b9a700 1 -- 172.21.15.131:0/2516917883 learned_addr learned my addr 172.21.15.131:0/2516917883 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53eab9c700 1 --2- 172.21.15.131:0/2516917883 >> v2:172.21.15.190:3300/0 conn(0x7f53e4068c00 0x7f53e40cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53ea39b700 1 --2- 172.21.15.131:0/2516917883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 0x7f53e40686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:44.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9b9a700 1 -- 172.21.15.131:0/2516917883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 msgr2=0x7f53e40686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9b9a700 1 --2- 172.21.15.131:0/2516917883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 0x7f53e40686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9b9a700 1 -- 172.21.15.131:0/2516917883 >> v2:172.21.15.190:3300/0 conn(0x7f53e4068c00 msgr2=0x7f53e40cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9b9a700 1 --2- 172.21.15.131:0/2516917883 >> v2:172.21.15.190:3300/0 conn(0x7f53e4068c00 0x7f53e40cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9b9a700 1 -- 172.21.15.131:0/2516917883 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e415a6c0 con 0x7f53e4157420 2023-12-26T21:53:44.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9b9a700 1 --2- 172.21.15.131:0/2516917883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 0x7f53e4159c70 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f53e40ce000 tx=0x7f53d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72086c5145dec146 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:44.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.482+0000 7f53e9399700 1 -- 172.21.15.131:0/2516917883 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400b640 con 0x7f53e4157420 2023-12-26T21:53:44.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53e9399700 1 -- 172.21.15.131:0/2516917883 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d400b7a0 con 0x7f53e4157420 2023-12-26T21:53:44.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53e9399700 1 -- 172.21.15.131:0/2516917883 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400bb60 con 0x7f53e4157420 2023-12-26T21:53:44.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53ec5ff700 1 -- 172.21.15.131:0/2516917883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 msgr2=0x7f53e4159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:44.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/2516917883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 0x7f53e4159c70 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f53e40ce000 tx=0x7f53d4009250 comp rx=0 tx=0).stop 2023-12-26T21:53:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53ec5ff700 1 -- 172.21.15.131:0/2516917883 shutdown_connections 2023-12-26T21:53:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/2516917883 >> v2:172.21.15.190:3300/0 conn(0x7f53e4068c00 0x7f53e40cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/2516917883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 0x7f53e40686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/2516917883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 0x7f53e4159c70 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53ec5ff700 1 -- 172.21.15.131:0/2516917883 >> 172.21.15.131:0/2516917883 conn(0x7f53e406dc80 msgr2=0x7f53e40675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53ec5ff700 1 -- 172.21.15.131:0/2516917883 shutdown_connections 2023-12-26T21:53:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.483+0000 7f53ec5ff700 1 -- 172.21.15.131:0/2516917883 wait complete. 2023-12-26T21:53:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.485+0000 7f53ec5ff700 1 Processor -- start 2023-12-26T21:53:44.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.485+0000 7f53ec5ff700 1 -- start start 2023-12-26T21:53:44.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53ec5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4068c00 0x7f53e40b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:44.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53ec5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 0x7f53e4155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:44.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53ec5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 0x7f53e40c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53ec5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b2960 con 0x7f53e4157420 2023-12-26T21:53:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53ec5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b2c40 con 0x7f53e4068c00 2023-12-26T21:53:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53ec5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b2f20 con 0x7f53e4157050 2023-12-26T21:53:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53eab9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 0x7f53e40c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:44.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53eab9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 0x7f53e40c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43400/0 (socket says 172.21.15.131:43400) 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53eab9c700 1 -- 172.21.15.131:0/1263957579 learned_addr learned my addr 172.21.15.131:0/1263957579 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53ea39b700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4068c00 0x7f53e40b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.486+0000 7f53e9b9a700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 0x7f53e4155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53eab9c700 1 -- 172.21.15.131:0/1263957579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 msgr2=0x7f53e4155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53eab9c700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 0x7f53e4155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53eab9c700 1 -- 172.21.15.131:0/1263957579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4068c00 msgr2=0x7f53e40b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53eab9c700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4068c00 0x7f53e40b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53eab9c700 1 -- 172.21.15.131:0/1263957579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40094f0 con 0x7f53e4157420 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53eab9c700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 0x7f53e40c6520 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d70 tx=0x7f53dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53da7fc700 1 -- 172.21.15.131:0/1263957579 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00caa0 con 0x7f53e4157420 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53da7fc700 1 -- 172.21.15.131:0/1263957579 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc00f040 con 0x7f53e4157420 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e40b3200 con 0x7f53e4157420 2023-12-26T21:53:44.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53da7fc700 1 -- 172.21.15.131:0/1263957579 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00ce50 con 0x7f53e4157420 2023-12-26T21:53:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.487+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e40c8440 con 0x7f53e4157420 2023-12-26T21:53:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.489+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e40b36a0 con 0x7f53e4157420 2023-12-26T21:53:44.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.490+0000 7f53da7fc700 1 -- 172.21.15.131:0/1263957579 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f53dc014690 con 0x7f53e4157420 2023-12-26T21:53:44.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.492+0000 7f53da7fc700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c8066f30 0x7f53c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:44.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.492+0000 7f53da7fc700 1 -- 172.21.15.131:0/1263957579 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5615+0+0 (secure 0 0 0) 0x7f53dc0855e0 con 0x7f53e4157420 2023-12-26T21:53:44.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.492+0000 7f53ea39b700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c8066f30 0x7f53c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:44.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.492+0000 7f53ea39b700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c8066f30 0x7f53c80693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009db0 tx=0x7f53e00091a0 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:44.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.495+0000 7f53da7fc700 1 -- 172.21.15.131:0/1263957579 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53dc052890 con 0x7f53e4157420 2023-12-26T21:53:44.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:44.691+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f53e40657e0 con 0x7f53e4157420 2023-12-26T21:53:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.262+0000 7f53da7fc700 1 -- 172.21.15.131:0/1263957579 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v108) v1 ==== 170+0+0 (secure 0 0 0) 0x7f53dc055eb0 con 0x7f53e4157420 2023-12-26T21:53:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.269+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c8066f30 msgr2=0x7f53c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.269+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c8066f30 0x7f53c80693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009db0 tx=0x7f53e00091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.269+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 msgr2=0x7f53e40c6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.269+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 0x7f53e40c6520 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d70 tx=0x7f53dc00b040 comp rx=0 tx=0).stop 2023-12-26T21:53:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.270+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 shutdown_connections 2023-12-26T21:53:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.270+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f53c8066f30 0x7f53c80693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.270+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53e4157420 0x7f53e40c6520 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.270+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53e4068c00 0x7f53e40b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.270+0000 7f53ec5ff700 1 --2- 172.21.15.131:0/1263957579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53e4157050 0x7f53e4155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.270+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 >> 172.21.15.131:0/1263957579 conn(0x7f53e406dc80 msgr2=0x7f53e4156000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:45.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.273+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 shutdown_connections 2023-12-26T21:53:45.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.273+0000 7f53ec5ff700 1 -- 172.21.15.131:0/1263957579 wait complete. 2023-12-26T21:53:45.280 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T21:53:45.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T21:53:45.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.491+0000 7fa2947ff700 1 Processor -- start 2023-12-26T21:53:45.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.491+0000 7fa2947ff700 1 -- start start 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.492+0000 7fa2947ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c06bfd0 0x7fa28c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.492+0000 7fa2947ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa28c0cc100 0x7fa28c0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.492+0000 7fa2947ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c0686c0 con 0x7fa28c13c510 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.492+0000 7fa2947ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28c069520 con 0x7fa28c0cc100 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa2947ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c068910 con 0x7fa28c06bfd0 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa29259b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa28c13c510 0x7fa28c06b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45072/0 (socket says 172.21.15.131:45072) 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa29259b700 1 -- 172.21.15.131:0/3356284036 learned_addr learned my addr 172.21.15.131:0/3356284036 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa292d9c700 1 --2- 172.21.15.131:0/3356284036 >> v2:172.21.15.190:3300/0 conn(0x7fa28c0cc100 0x7fa28c0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa291d9a700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c06bfd0 0x7fa28c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512471654 0 0) 0x7fa28c0686c0 con 0x7fa28c13c510 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa278003570 con 0x7fa28c13c510 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 129517945 0 0) 0x7fa278003570 con 0x7fa28c13c510 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c06bfd0 msgr2=0x7fa28c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.493+0000 7fa291599700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c06bfd0 0x7fa28c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 >> v2:172.21.15.190:3300/0 conn(0x7fa28c0cc100 msgr2=0x7fa28c0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 --2- 172.21.15.131:0/3356284036 >> v2:172.21.15.190:3300/0 conn(0x7fa28c0cc100 0x7fa28c0cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c068bf0 con 0x7fa28c13c510 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fa2880015f0 con 0x7fa28c13c510 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa292d9c700 1 --2- 172.21.15.131:0/3356284036 >> v2:172.21.15.190:3300/0 conn(0x7fa28c0cc100 0x7fa28c0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 >> v1:172.21.15.131:6789/0 conn(0x7fa28c13c510 legacy=0x7fa28c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278003a60 0x7fa278003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c06bfd0 0x7fa278004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:45.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278004d50 0x7fa2780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c069520 con 0x7fa278004d50 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c068910 con 0x7fa278003a60 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27800ba60 con 0x7fa28c06bfd0 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa29259b700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278003a60 0x7fa278003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa292d9c700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278004d50 0x7fa2780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa291d9a700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c06bfd0 0x7fa278004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa292d9c700 1 -- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c06bfd0 msgr2=0x7fa278004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa292d9c700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c06bfd0 0x7fa278004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa292d9c700 1 -- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278003a60 msgr2=0x7fa278003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa292d9c700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278003a60 0x7fa278003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.494+0000 7fa292d9c700 1 -- 172.21.15.131:0/3356284036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa278003570 con 0x7fa278004d50 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.495+0000 7fa292d9c700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278004d50 0x7fa2780051c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e3c0 tx=0x7fa28400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e599b18b044b30d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.495+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28400ec10 con 0x7fa278004d50 2023-12-26T21:53:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.495+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28400f040 con 0x7fa278004d50 2023-12-26T21:53:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.495+0000 7fa291599700 1 -- 172.21.15.131:0/3356284036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284014480 con 0x7fa278004d50 2023-12-26T21:53:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.495+0000 7fa2947ff700 1 -- 172.21.15.131:0/3356284036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278004d50 msgr2=0x7fa2780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.495+0000 7fa2947ff700 1 --2- 172.21.15.131:0/3356284036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa278004d50 0x7fa2780051c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e3c0 tx=0x7fa28400b040 comp rx=0 tx=0).stop 2023-12-26T21:53:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.495+0000 7fa292d9c700 1 -- 172.21.15.131:0/3356284036 reap_dead start 2023-12-26T21:53:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.495+0000 7fa2947ff700 1 -- 172.21.15.131:0/3356284036 shutdown_connections 2023-12-26T21:53:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.496+0000 7fa2947ff700 1 -- 172.21.15.131:0/3356284036 >> 172.21.15.131:0/3356284036 conn(0x7fa28c0ce3e0 msgr2=0x7fa28c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.496+0000 7fa2947ff700 1 -- 172.21.15.131:0/3356284036 shutdown_connections 2023-12-26T21:53:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.496+0000 7fa2947ff700 1 -- 172.21.15.131:0/3356284036 wait complete. 2023-12-26T21:53:45.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.497+0000 7fa2947ff700 1 Processor -- start 2023-12-26T21:53:45.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.497+0000 7fa2947ff700 1 -- start start 2023-12-26T21:53:45.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa2947ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278004d50 0x7fa28c153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:45.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa2947ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c0b1f40 0x7fa28c0b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa2947ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa28c0cade0 0x7fa28c0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa2947ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0cb790 con 0x7fa28c0cade0 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa2947ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c153dc0 con 0x7fa278004d50 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa2947ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0b36f0 con 0x7fa28c0b1f40 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa292d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa28c0cade0 0x7fa28c0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa292d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa28c0cade0 0x7fa28c0cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58694/0 (socket says 172.21.15.131:58694) 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa292d9c700 1 -- 172.21.15.131:0/1801259789 learned_addr learned my addr 172.21.15.131:0/1801259789 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa29259b700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278004d50 0x7fa28c153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.498+0000 7fa291d9a700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c0b1f40 0x7fa28c0b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa292d9c700 1 -- 172.21.15.131:0/1801259789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c0b1f40 msgr2=0x7fa28c0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa292d9c700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c0b1f40 0x7fa28c0b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa292d9c700 1 -- 172.21.15.131:0/1801259789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278004d50 msgr2=0x7fa28c153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa292d9c700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278004d50 0x7fa28c153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa292d9c700 1 -- 172.21.15.131:0/1801259789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c0b39d0 con 0x7fa28c0cade0 2023-12-26T21:53:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa292d9c700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa28c0cade0 0x7fa28c0cb250 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa278004730 tx=0x7fa28401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa2827fc700 1 -- 172.21.15.131:0/1801259789 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284023070 con 0x7fa28c0cade0 2023-12-26T21:53:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa2827fc700 1 -- 172.21.15.131:0/1801259789 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa284014480 con 0x7fa28c0cade0 2023-12-26T21:53:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa2827fc700 1 -- 172.21.15.131:0/1801259789 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28401d8a0 con 0x7fa28c0cade0 2023-12-26T21:53:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c0b3cb0 con 0x7fa28c0cade0 2023-12-26T21:53:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.499+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c0c6950 con 0x7fa28c0cade0 2023-12-26T21:53:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.500+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c0c6cc0 con 0x7fa28c0cade0 2023-12-26T21:53:45.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.502+0000 7fa2827fc700 1 -- 172.21.15.131:0/1801259789 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa28401dcd0 con 0x7fa28c0cade0 2023-12-26T21:53:45.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.503+0000 7fa2827fc700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa270066f30 0x7fa2700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:45.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.503+0000 7fa2827fc700 1 -- 172.21.15.131:0/1801259789 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fa284086ad0 con 0x7fa28c0cade0 2023-12-26T21:53:45.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.503+0000 7fa29259b700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa270066f30 0x7fa2700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:45.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.504+0000 7fa29259b700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa270066f30 0x7fa2700693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa2880011d0 tx=0x7fa288009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:45.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.506+0000 7fa2827fc700 1 -- 172.21.15.131:0/1801259789 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa284053d60 con 0x7fa28c0cade0 2023-12-26T21:53:45.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:45.742+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa28c0c67b0 con 0x7fa28c0cade0 2023-12-26T21:53:46.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.294+0000 7fa2827fc700 1 -- 172.21.15.131:0/1801259789 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa28c0c67b0 con 0x7fa28c0cade0 2023-12-26T21:53:46.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.309+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa270066f30 msgr2=0x7fa2700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.309+0000 7fa2947ff700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa270066f30 0x7fa2700693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa2880011d0 tx=0x7fa288009040 comp rx=0 tx=0).stop 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.309+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa28c0cade0 msgr2=0x7fa28c0cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.309+0000 7fa2947ff700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa28c0cade0 0x7fa28c0cb250 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa278004730 tx=0x7fa28401a000 comp rx=0 tx=0).stop 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.312+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 shutdown_connections 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.312+0000 7fa2947ff700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa270066f30 0x7fa2700693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.312+0000 7fa2947ff700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa28c0cade0 0x7fa28c0cb250 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.312+0000 7fa2947ff700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278004d50 0x7fa28c153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.312+0000 7fa2947ff700 1 --2- 172.21.15.131:0/1801259789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa28c0b1f40 0x7fa28c0b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.312+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 >> 172.21.15.131:0/1801259789 conn(0x7fa28c0ce3e0 msgr2=0x7fa28c155660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.313+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 shutdown_connections 2023-12-26T21:53:46.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.313+0000 7fa2947ff700 1 -- 172.21.15.131:0/1801259789 wait complete. 2023-12-26T21:53:46.317 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 12 and data pool 13 2023-12-26T21:53:46.336 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:46.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.527+0000 7ff37e5ff700 1 Processor -- start 2023-12-26T21:53:46.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.527+0000 7ff37e5ff700 1 -- start start 2023-12-26T21:53:46.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.528+0000 7ff37e5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 0x7ff3780b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.528+0000 7ff37e5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff3780b0870 0x7ff3780b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.528+0000 7ff37e5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3780afcd0 con 0x7ff378159a70 2023-12-26T21:53:46.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.528+0000 7ff37e5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3780b3e70 con 0x7ff3780b0870 2023-12-26T21:53:46.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.528+0000 7ff37e5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780b0cd0 con 0x7ff3781596a0 2023-12-26T21:53:46.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.528+0000 7ff3777fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff378159a70 0x7ff3780b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45076/0 (socket says 172.21.15.131:45076) 2023-12-26T21:53:46.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff3777fe700 1 -- 172.21.15.131:0/2993688078 learned_addr learned my addr 172.21.15.131:0/2993688078 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:46.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff37cb9c700 1 --2- 172.21.15.131:0/2993688078 >> v2:172.21.15.190:3300/0 conn(0x7ff3780b0870 0x7ff3780b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff377fff700 1 --2- 172.21.15.131:0/2993688078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 0x7ff3780b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff376ffd700 1 -- 172.21.15.131:0/2993688078 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989089412 0 0) 0x7ff3780afcd0 con 0x7ff378159a70 2023-12-26T21:53:46.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff376ffd700 1 -- 172.21.15.131:0/2993688078 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff360003570 con 0x7ff378159a70 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff377fff700 1 -- 172.21.15.131:0/2993688078 >> v2:172.21.15.190:3300/0 conn(0x7ff3780b0870 msgr2=0x7ff3780b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff377fff700 1 --2- 172.21.15.131:0/2993688078 >> v2:172.21.15.190:3300/0 conn(0x7ff3780b0870 0x7ff3780b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff377fff700 1 -- 172.21.15.131:0/2993688078 >> v1:172.21.15.131:6789/0 conn(0x7ff378159a70 legacy=0x7ff3780b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff377fff700 1 -- 172.21.15.131:0/2993688078 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3780b2e00 con 0x7ff3781596a0 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.529+0000 7ff377fff700 1 --2- 172.21.15.131:0/2993688078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 0x7ff3780b2380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff3680098e0 tx=0x7ff3680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b3ad364392f4f33 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff376ffd700 1 -- 172.21.15.131:0/2993688078 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368011630 con 0x7ff3781596a0 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff376ffd700 1 -- 172.21.15.131:0/2993688078 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff368011790 con 0x7ff3781596a0 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff376ffd700 1 -- 172.21.15.131:0/2993688078 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368011b30 con 0x7ff3781596a0 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff37e5ff700 1 -- 172.21.15.131:0/2993688078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 msgr2=0x7ff3780b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff37e5ff700 1 --2- 172.21.15.131:0/2993688078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 0x7ff3780b2380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff3680098e0 tx=0x7ff3680091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff37e5ff700 1 -- 172.21.15.131:0/2993688078 shutdown_connections 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff37e5ff700 1 --2- 172.21.15.131:0/2993688078 >> v2:172.21.15.190:3300/0 conn(0x7ff3780b0870 0x7ff3780b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff37e5ff700 1 --2- 172.21.15.131:0/2993688078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 0x7ff3780b2380 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff37e5ff700 1 -- 172.21.15.131:0/2993688078 >> 172.21.15.131:0/2993688078 conn(0x7ff37806ece0 msgr2=0x7ff37806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff37e5ff700 1 -- 172.21.15.131:0/2993688078 shutdown_connections 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.530+0000 7ff37e5ff700 1 -- 172.21.15.131:0/2993688078 wait complete. 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.532+0000 7ff37e5ff700 1 Processor -- start 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.532+0000 7ff37e5ff700 1 -- start start 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff37e5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b0870 0x7ff37816db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff37e5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 0x7ff3780c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff37e5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff378159a70 0x7ff3780c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff37e5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37816e0d0 con 0x7ff378159a70 2023-12-26T21:53:46.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff37e5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c43f0 con 0x7ff3780b0870 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff37e5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c46d0 con 0x7ff3781596a0 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff37cb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff378159a70 0x7ff3780c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff37cb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff378159a70 0x7ff3780c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58710/0 (socket says 172.21.15.131:58710) 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff37cb9c700 1 -- 172.21.15.131:0/943566759 learned_addr learned my addr 172.21.15.131:0/943566759 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff377fff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b0870 0x7ff37816db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.533+0000 7ff3777fe700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 0x7ff3780c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff37cb9c700 1 -- 172.21.15.131:0/943566759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 msgr2=0x7ff3780c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff37cb9c700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 0x7ff3780c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff37cb9c700 1 -- 172.21.15.131:0/943566759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b0870 msgr2=0x7ff37816db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff37cb9c700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b0870 0x7ff37816db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff37cb9c700 1 -- 172.21.15.131:0/943566759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff368009420 con 0x7ff378159a70 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff37cb9c700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff378159a70 0x7ff3780c59d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff370009df0 tx=0x7ff37000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff35ffff700 1 -- 172.21.15.131:0/943566759 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37000caa0 con 0x7ff378159a70 2023-12-26T21:53:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff35ffff700 1 -- 172.21.15.131:0/943566759 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37000f040 con 0x7ff378159a70 2023-12-26T21:53:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff35ffff700 1 -- 172.21.15.131:0/943566759 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37000ce70 con 0x7ff378159a70 2023-12-26T21:53:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3780c49b0 con 0x7ff378159a70 2023-12-26T21:53:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.534+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3780c4f70 con 0x7ff378159a70 2023-12-26T21:53:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.536+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3780657e0 con 0x7ff378159a70 2023-12-26T21:53:46.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.536+0000 7ff35ffff700 1 -- 172.21.15.131:0/943566759 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff370014690 con 0x7ff378159a70 2023-12-26T21:53:46.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.538+0000 7ff35ffff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff364066f30 0x7ff3640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.538+0000 7ff35ffff700 1 -- 172.21.15.131:0/943566759 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff370084f70 con 0x7ff378159a70 2023-12-26T21:53:46.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.538+0000 7ff377fff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff364066f30 0x7ff3640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.539+0000 7ff377fff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff364066f30 0x7ff3640693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff3680091e0 tx=0x7ff368017000 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:46.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.542+0000 7ff35ffff700 1 -- 172.21.15.131:0/943566759 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff370055a40 con 0x7ff378159a70 2023-12-26T21:53:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.734+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3780c5190 con 0x7ff378159a70 2023-12-26T21:53:46.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.735+0000 7ff35ffff700 1 -- 172.21.15.131:0/943566759 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16021 (secure 0 0 0) 0x7ff3700555d0 con 0x7ff378159a70 2023-12-26T21:53:46.736 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:46.736 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":109,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:45.744465+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:43.922429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:44.731388+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff364066f30 msgr2=0x7ff3640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff364066f30 0x7ff3640693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff3680091e0 tx=0x7ff368017000 comp rx=0 tx=0).stop 2023-12-26T21:53:46.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff378159a70 msgr2=0x7ff3780c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff378159a70 0x7ff3780c59d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff370009df0 tx=0x7ff37000b040 comp rx=0 tx=0).stop 2023-12-26T21:53:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 shutdown_connections 2023-12-26T21:53:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff364066f30 0x7ff3640693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff378159a70 0x7ff3780c59d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b0870 0x7ff37816db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 --2- 172.21.15.131:0/943566759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3781596a0 0x7ff3780c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 >> 172.21.15.131:0/943566759 conn(0x7ff37806ece0 msgr2=0x7ff3780b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 shutdown_connections 2023-12-26T21:53:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.738+0000 7ff37e5ff700 1 -- 172.21.15.131:0/943566759 wait complete. 2023-12-26T21:53:46.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:46.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.945+0000 7fb6373ff700 1 Processor -- start 2023-12-26T21:53:46.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.945+0000 7fb6373ff700 1 -- start start 2023-12-26T21:53:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.946+0000 7fb6373ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb63006e480 0x7fb6300abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.947+0000 7fb6373ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63015a8c0 con 0x7fb6300aa480 2023-12-26T21:53:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.947+0000 7fb6373ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb630159f90 con 0x7fb630159820 2023-12-26T21:53:46.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.947+0000 7fb6373ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63015ad50 con 0x7fb63006e480 2023-12-26T21:53:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.947+0000 7fb63499a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb63006e480 0x7fb6300abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.947+0000 7fb63599c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb6300aa480 0x7fb6300675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45088/0 (socket says 172.21.15.131:45088) 2023-12-26T21:53:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.947+0000 7fb63599c700 1 -- 172.21.15.131:0/1876524579 learned_addr learned my addr 172.21.15.131:0/1876524579 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.947+0000 7fb63499a700 1 -- 172.21.15.131:0/1876524579 >> v1:172.21.15.190:6789/0 conn(0x7fb630159820 legacy=0x7fb63006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.947+0000 7fb63499a700 1 -- 172.21.15.131:0/1876524579 >> v1:172.21.15.131:6789/0 conn(0x7fb6300aa480 legacy=0x7fb6300675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.947+0000 7fb63499a700 1 -- 172.21.15.131:0/1876524579 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63015a270 con 0x7fb63006e480 2023-12-26T21:53:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.948+0000 7fb63499a700 1 --2- 172.21.15.131:0/1876524579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb63006e480 0x7fb6300abdb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb6200098e0 tx=0x7fb6200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c872943e12fdf205 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.948+0000 7fb627fff700 1 -- 172.21.15.131:0/1876524579 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620009e80 con 0x7fb63006e480 2023-12-26T21:53:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.948+0000 7fb627fff700 1 -- 172.21.15.131:0/1876524579 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62000b860 con 0x7fb63006e480 2023-12-26T21:53:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.948+0000 7fb627fff700 1 -- 172.21.15.131:0/1876524579 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62000bbe0 con 0x7fb63006e480 2023-12-26T21:53:46.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.948+0000 7fb6373ff700 1 -- 172.21.15.131:0/1876524579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb63006e480 msgr2=0x7fb6300abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.948+0000 7fb6373ff700 1 --2- 172.21.15.131:0/1876524579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb63006e480 0x7fb6300abdb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb6200098e0 tx=0x7fb6200091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.948+0000 7fb6373ff700 1 -- 172.21.15.131:0/1876524579 shutdown_connections 2023-12-26T21:53:46.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.948+0000 7fb6373ff700 1 --2- 172.21.15.131:0/1876524579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb63006e480 0x7fb6300abdb0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.948+0000 7fb6373ff700 1 -- 172.21.15.131:0/1876524579 >> 172.21.15.131:0/1876524579 conn(0x7fb63006c290 msgr2=0x7fb6300ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:46.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.949+0000 7fb6373ff700 1 -- 172.21.15.131:0/1876524579 shutdown_connections 2023-12-26T21:53:46.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.949+0000 7fb6373ff700 1 -- 172.21.15.131:0/1876524579 wait complete. 2023-12-26T21:53:46.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.950+0000 7fb6373ff700 1 Processor -- start 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.950+0000 7fb6373ff700 1 -- start start 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb6373ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb63006e480 0x7fb6300c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb6373ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6300aa480 0x7fb6301552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb6373ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb630159820 0x7fb6300ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb6373ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300c2c40 con 0x7fb630159820 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb6373ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300c2f20 con 0x7fb63006e480 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb6373ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300c3200 con 0x7fb6300aa480 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb63599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb630159820 0x7fb6300ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb63599c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb630159820 0x7fb6300ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58726/0 (socket says 172.21.15.131:58726) 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb63599c700 1 -- 172.21.15.131:0/3411411249 learned_addr learned my addr 172.21.15.131:0/3411411249 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:46.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.951+0000 7fb63499a700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6300aa480 0x7fb6301552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb63519b700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb63006e480 0x7fb6300c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb63599c700 1 -- 172.21.15.131:0/3411411249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6300aa480 msgr2=0x7fb6301552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb63599c700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6300aa480 0x7fb6301552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb63599c700 1 -- 172.21.15.131:0/3411411249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb63006e480 msgr2=0x7fb6300c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb63599c700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb63006e480 0x7fb6300c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb63599c700 1 -- 172.21.15.131:0/3411411249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb620009420 con 0x7fb630159820 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb63519b700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb63006e480 0x7fb6300c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb63599c700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb630159820 0x7fb6300ac580 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb62800cc40 tx=0x7fb628001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb6257fa700 1 -- 172.21.15.131:0/3411411249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62800a9c0 con 0x7fb630159820 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb6257fa700 1 -- 172.21.15.131:0/3411411249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62800ab20 con 0x7fb630159820 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb6257fa700 1 -- 172.21.15.131:0/3411411249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6280070f0 con 0x7fb630159820 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6300c34e0 con 0x7fb630159820 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.952+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6300c2810 con 0x7fb630159820 2023-12-26T21:53:46.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.953+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6300657e0 con 0x7fb630159820 2023-12-26T21:53:46.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.954+0000 7fb6257fa700 1 -- 172.21.15.131:0/3411411249 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb6280074e0 con 0x7fb630159820 2023-12-26T21:53:46.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.955+0000 7fb6257fa700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb614066f30 0x7fb6140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:46.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.956+0000 7fb6257fa700 1 -- 172.21.15.131:0/3411411249 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fb628084b80 con 0x7fb630159820 2023-12-26T21:53:46.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.956+0000 7fb63519b700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb614066f30 0x7fb6140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:46.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.956+0000 7fb63519b700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb614066f30 0x7fb6140693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0017a0 tx=0x7fb62c009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:46.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:46.959+0000 7fb6257fa700 1 -- 172.21.15.131:0/3411411249 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb628051c40 con 0x7fb630159820 2023-12-26T21:53:47.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.155+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb63015a550 con 0x7fb630159820 2023-12-26T21:53:47.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.156+0000 7fb6257fa700 1 -- 172.21.15.131:0/3411411249 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16021 (secure 0 0 0) 0x7fb628055260 con 0x7fb630159820 2023-12-26T21:53:47.156 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:47.156 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":109,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:45.744465+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:43.922429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:44.731388+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb614066f30 msgr2=0x7fb6140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb614066f30 0x7fb6140693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0017a0 tx=0x7fb62c009040 comp rx=0 tx=0).stop 2023-12-26T21:53:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb630159820 msgr2=0x7fb6300ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb630159820 0x7fb6300ac580 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb62800cc40 tx=0x7fb628001a50 comp rx=0 tx=0).stop 2023-12-26T21:53:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 shutdown_connections 2023-12-26T21:53:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb614066f30 0x7fb6140693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb630159820 0x7fb6300ac580 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb63006e480 0x7fb6300c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 --2- 172.21.15.131:0/3411411249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6300aa480 0x7fb6301552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 >> 172.21.15.131:0/3411411249 conn(0x7fb63006c290 msgr2=0x7fb630069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 shutdown_connections 2023-12-26T21:53:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.158+0000 7fb6373ff700 1 -- 172.21.15.131:0/3411411249 wait complete. 2023-12-26T21:53:47.177 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T21:53:47.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.372+0000 7f8a023ff700 1 Processor -- start 2023-12-26T21:53:47.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.373+0000 7f8a023ff700 1 -- start start 2023-12-26T21:53:47.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a023ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f89fc06bfd0 0x7f89fc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:47.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a023ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f89fc0cc100 0x7f89fc0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:47.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a023ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc0686c0 con 0x7f89fc13c510 2023-12-26T21:53:47.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a023ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc069520 con 0x7f89fc0cc100 2023-12-26T21:53:47.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a023ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc068910 con 0x7f89fc06bfd0 2023-12-26T21:53:47.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a0099c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f89fc0cc100 0x7f89fc0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:47.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a0099c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f89fc0cc100 0x7f89fc0cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58732/0 (socket says 172.21.15.131:58732) 2023-12-26T21:53:47.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a0099c700 1 -- 172.21.15.131:0/2068971289 learned_addr learned my addr 172.21.15.131:0/2068971289 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:47.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f89fb7fe700 1 --2- 172.21.15.131:0/2068971289 >> v2:172.21.15.190:3300/0 conn(0x7f89fc06bfd0 0x7f89fc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a0099c700 1 -- 172.21.15.131:0/2068971289 >> v2:172.21.15.190:3300/0 conn(0x7f89fc06bfd0 msgr2=0x7f89fc0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a0099c700 1 --2- 172.21.15.131:0/2068971289 >> v2:172.21.15.190:3300/0 conn(0x7f89fc06bfd0 0x7f89fc0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a0099c700 1 -- 172.21.15.131:0/2068971289 >> v1:172.21.15.190:6789/0 conn(0x7f89fc13c510 legacy=0x7f89fc06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.374+0000 7f8a0099c700 1 -- 172.21.15.131:0/2068971289 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc068bf0 con 0x7f89fc0cc100 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.375+0000 7f8a0099c700 1 --2- 172.21.15.131:0/2068971289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc0cc100 0x7f89fc0cc520 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0b43e0 tx=0x7f89f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0d5ba331c1e8a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.375+0000 7f89faffd700 1 -- 172.21.15.131:0/2068971289 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f400e070 con 0x7f89fc0cc100 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.375+0000 7f89faffd700 1 -- 172.21.15.131:0/2068971289 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f400ba10 con 0x7f89fc0cc100 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.375+0000 7f89faffd700 1 -- 172.21.15.131:0/2068971289 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f40092e0 con 0x7f89fc0cc100 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.375+0000 7f8a023ff700 1 -- 172.21.15.131:0/2068971289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc0cc100 msgr2=0x7f89fc0cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.375+0000 7f8a023ff700 1 --2- 172.21.15.131:0/2068971289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc0cc100 0x7f89fc0cc520 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0b43e0 tx=0x7f89f4009840 comp rx=0 tx=0).stop 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.376+0000 7f8a023ff700 1 -- 172.21.15.131:0/2068971289 shutdown_connections 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.376+0000 7f8a023ff700 1 --2- 172.21.15.131:0/2068971289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc0cc100 0x7f89fc0cc520 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.376+0000 7f8a023ff700 1 --2- 172.21.15.131:0/2068971289 >> v2:172.21.15.190:3300/0 conn(0x7f89fc06bfd0 0x7f89fc0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:47.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.376+0000 7f8a023ff700 1 -- 172.21.15.131:0/2068971289 >> 172.21.15.131:0/2068971289 conn(0x7f89fc0ce3e0 msgr2=0x7f89fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:47.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.376+0000 7f8a023ff700 1 -- 172.21.15.131:0/2068971289 shutdown_connections 2023-12-26T21:53:47.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.376+0000 7f8a023ff700 1 -- 172.21.15.131:0/2068971289 wait complete. 2023-12-26T21:53:47.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.377+0000 7f8a023ff700 1 Processor -- start 2023-12-26T21:53:47.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.377+0000 7f8a023ff700 1 -- start start 2023-12-26T21:53:47.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.378+0000 7f8a023ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89fc06bfd0 0x7f89fc0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:47.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.378+0000 7f8a023ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89fc0cc100 0x7f89fc15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:47.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.378+0000 7f8a023ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc13c510 0x7f89fc153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:47.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.378+0000 7f8a023ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc13dad0 con 0x7f89fc13c510 2023-12-26T21:53:47.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.378+0000 7f8a023ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc13ddb0 con 0x7f89fc0cc100 2023-12-26T21:53:47.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.378+0000 7f8a023ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc13e090 con 0x7f89fc06bfd0 2023-12-26T21:53:47.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.378+0000 7f8a0099c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc13c510 0x7f89fc153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:47.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f8a0099c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc13c510 0x7f89fc153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58734/0 (socket says 172.21.15.131:58734) 2023-12-26T21:53:47.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f8a0099c700 1 -- 172.21.15.131:0/317571931 learned_addr learned my addr 172.21.15.131:0/317571931 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f89fb7fe700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89fc0cc100 0x7f89fc15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f89fbfff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89fc06bfd0 0x7f89fc0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f8a0099c700 1 -- 172.21.15.131:0/317571931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89fc06bfd0 msgr2=0x7f89fc0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f8a0099c700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89fc06bfd0 0x7f89fc0b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f8a0099c700 1 -- 172.21.15.131:0/317571931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89fc0cc100 msgr2=0x7f89fc15b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f8a0099c700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89fc0cc100 0x7f89fc15b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f8a0099c700 1 -- 172.21.15.131:0/317571931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f4009ae0 con 0x7f89fc13c510 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f8a0099c700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc13c510 0x7f89fc153a80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f89fc06fe40 tx=0x7f89f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f89e3fff700 1 -- 172.21.15.131:0/317571931 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f40092e0 con 0x7f89fc13c510 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f89e3fff700 1 -- 172.21.15.131:0/317571931 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f4012d00 con 0x7f89fc13c510 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f89e3fff700 1 -- 172.21.15.131:0/317571931 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f401d670 con 0x7f89fc13c510 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.379+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc13e370 con 0x7f89fc13c510 2023-12-26T21:53:47.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.380+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc15b620 con 0x7f89fc13c510 2023-12-26T21:53:47.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.381+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc0657e0 con 0x7f89fc13c510 2023-12-26T21:53:47.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.381+0000 7f89e3fff700 1 -- 172.21.15.131:0/317571931 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f89f401d9e0 con 0x7f89fc13c510 2023-12-26T21:53:47.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.382+0000 7f89e3fff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:47.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.382+0000 7f89e3fff700 1 -- 172.21.15.131:0/317571931 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f89f4085d70 con 0x7f89fc13c510 2023-12-26T21:53:47.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.382+0000 7f89fbfff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:47.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.383+0000 7f89fbfff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f89e8066f30 0x7f89e80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89ec003a70 tx=0x7f89ec007ef0 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:47.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.386+0000 7f89e3fff700 1 -- 172.21.15.131:0/317571931 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89f4052e50 con 0x7f89fc13c510 2023-12-26T21:53:47.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:47.624+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f89fc13e890 con 0x7f89fc13c510 2023-12-26T21:53:48.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.297+0000 7f89e3fff700 1 -- 172.21.15.131:0/317571931 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7f89fc13e890 con 0x7f89fc13c510 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f89e8066f30 msgr2=0x7f89e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f89e8066f30 0x7f89e80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89ec003a70 tx=0x7f89ec007ef0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc13c510 msgr2=0x7f89fc153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc13c510 0x7f89fc153a80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f89fc06fe40 tx=0x7f89f4017000 comp rx=0 tx=0).stop 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 shutdown_connections 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f89e8066f30 0x7f89e80693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89fc13c510 0x7f89fc153a80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89fc06bfd0 0x7f89fc0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 --2- 172.21.15.131:0/317571931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89fc0cc100 0x7f89fc15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 >> 172.21.15.131:0/317571931 conn(0x7f89fc0ce3e0 msgr2=0x7f89fc15e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 shutdown_connections 2023-12-26T21:53:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.300+0000 7f8a023ff700 1 -- 172.21.15.131:0/317571931 wait complete. 2023-12-26T21:53:48.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T21:53:48.473 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:53:48.478 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:53:48.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.521+0000 7f151b7ff700 1 Processor -- start 2023-12-26T21:53:48.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.521+0000 7f151b7ff700 1 -- start start 2023-12-26T21:53:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.522+0000 7f151b7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f15141596a0 0x7f15140b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.522+0000 7f151b7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1514159a70 0x7f15140b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.522+0000 7f151b7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15140afcd0 con 0x7f15140b0160 2023-12-26T21:53:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.522+0000 7f151b7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15140b3e70 con 0x7f15141596a0 2023-12-26T21:53:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.522+0000 7f151b7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15140b2e00 con 0x7f1514159a70 2023-12-26T21:53:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f151959b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f15141596a0 0x7f15140b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f1519d9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f15140b0160 0x7f15140b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45092/0 (socket says 172.21.15.131:45092) 2023-12-26T21:53:48.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f1519d9c700 1 -- 172.21.15.131:0/634676219 learned_addr learned my addr 172.21.15.131:0/634676219 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:48.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f1518d9a700 1 --2- 172.21.15.131:0/634676219 >> v2:172.21.15.190:3300/0 conn(0x7f1514159a70 0x7f15140b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f151959b700 1 -- 172.21.15.131:0/634676219 >> v2:172.21.15.190:3300/0 conn(0x7f1514159a70 msgr2=0x7f15140b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f151959b700 1 --2- 172.21.15.131:0/634676219 >> v2:172.21.15.190:3300/0 conn(0x7f1514159a70 0x7f15140b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f151959b700 1 -- 172.21.15.131:0/634676219 >> v1:172.21.15.131:6789/0 conn(0x7f15140b0160 legacy=0x7f15140b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f151959b700 1 -- 172.21.15.131:0/634676219 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15140b0cb0 con 0x7f15141596a0 2023-12-26T21:53:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f150bfff700 1 -- 172.21.15.131:0/634676219 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294992626 0 0) 0x7f15140afcd0 con 0x7f15140b0160 2023-12-26T21:53:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.523+0000 7f151959b700 1 --2- 172.21.15.131:0/634676219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15141596a0 0x7f15140b2380 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f15140642f0 tx=0x7f15100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5689b8c58098de server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f150bfff700 1 -- 172.21.15.131:0/634676219 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510011630 con 0x7f15141596a0 2023-12-26T21:53:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f150bfff700 1 -- 172.21.15.131:0/634676219 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510011790 con 0x7f15141596a0 2023-12-26T21:53:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f150bfff700 1 -- 172.21.15.131:0/634676219 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510011b30 con 0x7f15141596a0 2023-12-26T21:53:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f151b7ff700 1 -- 172.21.15.131:0/634676219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15141596a0 msgr2=0x7f15140b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f151b7ff700 1 --2- 172.21.15.131:0/634676219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15141596a0 0x7f15140b2380 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f15140642f0 tx=0x7f15100091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f151b7ff700 1 -- 172.21.15.131:0/634676219 shutdown_connections 2023-12-26T21:53:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f151b7ff700 1 --2- 172.21.15.131:0/634676219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15141596a0 0x7f15140b2380 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f151b7ff700 1 --2- 172.21.15.131:0/634676219 >> v2:172.21.15.190:3300/0 conn(0x7f1514159a70 0x7f15140b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f151b7ff700 1 -- 172.21.15.131:0/634676219 >> 172.21.15.131:0/634676219 conn(0x7f151406ece0 msgr2=0x7f151406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f151b7ff700 1 -- 172.21.15.131:0/634676219 shutdown_connections 2023-12-26T21:53:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.524+0000 7f151b7ff700 1 -- 172.21.15.131:0/634676219 wait complete. 2023-12-26T21:53:48.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.526+0000 7f151b7ff700 1 Processor -- start 2023-12-26T21:53:48.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.526+0000 7f151b7ff700 1 -- start start 2023-12-26T21:53:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f151b7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15140b0160 0x7f151415c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f151b7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15141596a0 0x7f15140c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f151b7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1514159a70 0x7f15140c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f151b7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15140c7040 con 0x7f15140b0160 2023-12-26T21:53:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f151b7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15140c4800 con 0x7f15141596a0 2023-12-26T21:53:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f151b7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15140c86a0 con 0x7f1514159a70 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f151959b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15140b0160 0x7f151415c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f151959b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15140b0160 0x7f151415c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58754/0 (socket says 172.21.15.131:58754) 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f151959b700 1 -- 172.21.15.131:0/411060907 learned_addr learned my addr 172.21.15.131:0/411060907 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f1518d9a700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15141596a0 0x7f15140c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.527+0000 7f1519d9c700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1514159a70 0x7f15140c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f151959b700 1 -- 172.21.15.131:0/411060907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1514159a70 msgr2=0x7f15140c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f151959b700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1514159a70 0x7f15140c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f151959b700 1 -- 172.21.15.131:0/411060907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15141596a0 msgr2=0x7f15140c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f151959b700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15141596a0 0x7f15140c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f151959b700 1 -- 172.21.15.131:0/411060907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510009420 con 0x7f15140b0160 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f1519d9c700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1514159a70 0x7f15140c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f151959b700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15140b0160 0x7f151415c2b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f15100097b0 tx=0x7f1510017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f15097fa700 1 -- 172.21.15.131:0/411060907 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510011b10 con 0x7f15140b0160 2023-12-26T21:53:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f15097fa700 1 -- 172.21.15.131:0/411060907 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15100233f0 con 0x7f15140b0160 2023-12-26T21:53:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f15097fa700 1 -- 172.21.15.131:0/411060907 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510023790 con 0x7f15140b0160 2023-12-26T21:53:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15140c6a90 con 0x7f15140b0160 2023-12-26T21:53:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.528+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15140c41b0 con 0x7f15140b0160 2023-12-26T21:53:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.530+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15140657e0 con 0x7f15140b0160 2023-12-26T21:53:48.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.531+0000 7f15097fa700 1 -- 172.21.15.131:0/411060907 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1510023b80 con 0x7f15140b0160 2023-12-26T21:53:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.532+0000 7f15097fa700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f14f8066f30 0x7f14f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.532+0000 7f15097fa700 1 -- 172.21.15.131:0/411060907 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f1510085ac0 con 0x7f15140b0160 2023-12-26T21:53:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.532+0000 7f1518d9a700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f14f8066f30 0x7f14f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.533+0000 7f1518d9a700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f14f8066f30 0x7f14f80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1514159220 tx=0x7f15040091a0 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:48.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.536+0000 7f15097fa700 1 -- 172.21.15.131:0/411060907 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1510052b00 con 0x7f15140b0160 2023-12-26T21:53:48.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:48.779+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1514069560 con 0x7f15140b0160 2023-12-26T21:53:49.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.299+0000 7f15097fa700 1 -- 172.21.15.131:0/411060907 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7f15100150c0 con 0x7f15140b0160 2023-12-26T21:53:49.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f14f8066f30 msgr2=0x7f14f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f14f8066f30 0x7f14f80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1514159220 tx=0x7f15040091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15140b0160 msgr2=0x7f151415c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15140b0160 0x7f151415c2b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f15100097b0 tx=0x7f1510017000 comp rx=0 tx=0).stop 2023-12-26T21:53:49.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 shutdown_connections 2023-12-26T21:53:49.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1514159a70 0x7f15140c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f14f8066f30 0x7f14f80693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15140b0160 0x7f151415c2b0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 --2- 172.21.15.131:0/411060907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15141596a0 0x7f15140c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.301+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 >> 172.21.15.131:0/411060907 conn(0x7f151406ece0 msgr2=0x7f15140cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:49.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.302+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 shutdown_connections 2023-12-26T21:53:49.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.302+0000 7f151b7ff700 1 -- 172.21.15.131:0/411060907 wait complete. 2023-12-26T21:53:49.320 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:53:49.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.509+0000 7fbcad1ff700 1 Processor -- start 2023-12-26T21:53:49.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.509+0000 7fbcad1ff700 1 -- start start 2023-12-26T21:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.510+0000 7fbcad1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca813c510 0x7fbca80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.511+0000 7fbcad1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbca806b8c0 0x7fbca806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.511+0000 7fbcad1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca806c270 con 0x7fbca80cc100 2023-12-26T21:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.511+0000 7fbcad1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca80686c0 con 0x7fbca806b8c0 2023-12-26T21:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.511+0000 7fbcad1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8069520 con 0x7fbca813c510 2023-12-26T21:53:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.511+0000 7fbca6d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca813c510 0x7fbca80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.511+0000 7fbc9ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbca806b8c0 0x7fbca806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.511+0000 7fbca759e700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbca80cc100 0x7fbca80cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:42056/0 (socket says 172.21.15.131:42056) 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.511+0000 7fbca759e700 1 -- 172.21.15.131:0/647213320 learned_addr learned my addr 172.21.15.131:0/647213320 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.512+0000 7fbca6d9d700 1 -- 172.21.15.131:0/647213320 >> v2:172.21.15.190:3300/0 conn(0x7fbca806b8c0 msgr2=0x7fbca806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.512+0000 7fbca6d9d700 1 --2- 172.21.15.131:0/647213320 >> v2:172.21.15.190:3300/0 conn(0x7fbca806b8c0 0x7fbca806bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.512+0000 7fbca6d9d700 1 -- 172.21.15.131:0/647213320 >> v1:172.21.15.190:6789/0 conn(0x7fbca80cc100 legacy=0x7fbca80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.512+0000 7fbca6d9d700 1 -- 172.21.15.131:0/647213320 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca8068910 con 0x7fbca813c510 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.512+0000 7fbc9ffff700 1 --2- 172.21.15.131:0/647213320 >> v2:172.21.15.190:3300/0 conn(0x7fbca806b8c0 0x7fbca806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.512+0000 7fbca6d9d700 1 --2- 172.21.15.131:0/647213320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca813c510 0x7fbca80c8a20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbc980098d0 tx=0x7fbc98009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e50ccaa9c1fc34 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.512+0000 7fbca659c700 1 -- 172.21.15.131:0/647213320 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98005c10 con 0x7fbca813c510 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.512+0000 7fbca659c700 1 -- 172.21.15.131:0/647213320 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc98005d70 con 0x7fbca813c510 2023-12-26T21:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.512+0000 7fbca659c700 1 -- 172.21.15.131:0/647213320 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98006280 con 0x7fbca813c510 2023-12-26T21:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.513+0000 7fbcad1ff700 1 -- 172.21.15.131:0/647213320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca813c510 msgr2=0x7fbca80c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.513+0000 7fbcad1ff700 1 --2- 172.21.15.131:0/647213320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca813c510 0x7fbca80c8a20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbc980098d0 tx=0x7fbc98009120 comp rx=0 tx=0).stop 2023-12-26T21:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.513+0000 7fbcad1ff700 1 -- 172.21.15.131:0/647213320 shutdown_connections 2023-12-26T21:53:49.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.513+0000 7fbcad1ff700 1 --2- 172.21.15.131:0/647213320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca813c510 0x7fbca80c8a20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.513+0000 7fbcad1ff700 1 --2- 172.21.15.131:0/647213320 >> v2:172.21.15.190:3300/0 conn(0x7fbca806b8c0 0x7fbca806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.513+0000 7fbcad1ff700 1 -- 172.21.15.131:0/647213320 >> 172.21.15.131:0/647213320 conn(0x7fbca80ce3e0 msgr2=0x7fbca80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:49.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.513+0000 7fbcad1ff700 1 -- 172.21.15.131:0/647213320 shutdown_connections 2023-12-26T21:53:49.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.513+0000 7fbcad1ff700 1 -- 172.21.15.131:0/647213320 wait complete. 2023-12-26T21:53:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.514+0000 7fbcad1ff700 1 Processor -- start 2023-12-26T21:53:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.515+0000 7fbcad1ff700 1 -- start start 2023-12-26T21:53:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.515+0000 7fbcad1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca806b8c0 0x7fbca80b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.515+0000 7fbcad1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca80cc100 0x7fbca8159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.515+0000 7fbcad1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca813c510 0x7fbca815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.515+0000 7fbcad1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca815dd30 con 0x7fbca813c510 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.515+0000 7fbcad1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80b3560 con 0x7fbca806b8c0 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.515+0000 7fbcad1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca815d6e0 con 0x7fbca80cc100 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca759e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca813c510 0x7fbca815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca759e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca813c510 0x7fbca815a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58762/0 (socket says 172.21.15.131:58762) 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca759e700 1 -- 172.21.15.131:0/3551691456 learned_addr learned my addr 172.21.15.131:0/3551691456 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbc9ffff700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca80cc100 0x7fbca8159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca6d9d700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca806b8c0 0x7fbca80b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca759e700 1 -- 172.21.15.131:0/3551691456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca80cc100 msgr2=0x7fbca8159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca759e700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca80cc100 0x7fbca8159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca759e700 1 -- 172.21.15.131:0/3551691456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca806b8c0 msgr2=0x7fbca80b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca759e700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca806b8c0 0x7fbca80b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca759e700 1 -- 172.21.15.131:0/3551691456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90009420 con 0x7fbca813c510 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.516+0000 7fbca759e700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca813c510 0x7fbca815a260 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbca000cc40 tx=0x7fbca0001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.517+0000 7fbc9effd700 1 -- 172.21.15.131:0/3551691456 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000a8b0 con 0x7fbca813c510 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.517+0000 7fbc9effd700 1 -- 172.21.15.131:0/3551691456 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca000aa10 con 0x7fbca813c510 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.517+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc980093c0 con 0x7fbca813c510 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.517+0000 7fbc9effd700 1 -- 172.21.15.131:0/3551691456 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000add0 con 0x7fbca813c510 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.517+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca80b3840 con 0x7fbca813c510 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.518+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca80657e0 con 0x7fbca813c510 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.519+0000 7fbc9effd700 1 -- 172.21.15.131:0/3551691456 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbca0007300 con 0x7fbca813c510 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.519+0000 7fbc9effd700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc88066f30 0x7fbc880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.520+0000 7fbc9effd700 1 -- 172.21.15.131:0/3551691456 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fbca0084980 con 0x7fbca813c510 2023-12-26T21:53:49.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.520+0000 7fbca6d9d700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc88066f30 0x7fbc880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.520+0000 7fbca6d9d700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc88066f30 0x7fbc880693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009160 tx=0x7fbc98016000 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:49.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.524+0000 7fbc9effd700 1 -- 172.21.15.131:0/3551691456 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbca0051ae0 con 0x7fbca813c510 2023-12-26T21:53:49.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.758+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbca8068bf0 con 0x7fbca813c510 2023-12-26T21:53:49.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.758+0000 7fbc9effd700 1 -- 172.21.15.131:0/3551691456 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fbca0055100 con 0x7fbca813c510 2023-12-26T21:53:49.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:49.760 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2416144644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2416144644},{"type":"v1","addr":"172.21.15.131:6835","nonce":2416144644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/559791617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":559791617},{"type":"v1","addr":"172.21.15.190:6834","nonce":559791617}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6901,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3954759429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3954759429},{"type":"v1","addr":"172.21.15.190:6835","nonce":3954759429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:45.744809+0000","modified":"2023-12-26T21:53:49.296563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6834/3631709496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3631709496},{"type":"v1","addr":"172.21.15.131:6834","nonce":3631709496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-12-26T21:53:49.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc88066f30 msgr2=0x7fbc880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc88066f30 0x7fbc880693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009160 tx=0x7fbc98016000 comp rx=0 tx=0).stop 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca813c510 msgr2=0x7fbca815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca813c510 0x7fbca815a260 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbca000cc40 tx=0x7fbca0001a50 comp rx=0 tx=0).stop 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 shutdown_connections 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc88066f30 0x7fbc880693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca813c510 0x7fbca815a260 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca806b8c0 0x7fbca80b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 --2- 172.21.15.131:0/3551691456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca80cc100 0x7fbca8159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 >> 172.21.15.131:0/3551691456 conn(0x7fbca80ce3e0 msgr2=0x7fbca80c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 shutdown_connections 2023-12-26T21:53:49.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.761+0000 7fbcad1ff700 1 -- 172.21.15.131:0/3551691456 wait complete. 2023-12-26T21:53:49.763 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 50 2023-12-26T21:53:49.779 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:53:49.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.968+0000 7f09701ff700 1 Processor -- start 2023-12-26T21:53:49.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.968+0000 7f09701ff700 1 -- start start 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.969+0000 7f09701ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096806bfd0 0x7f09680c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.969+0000 7f09701ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09680686c0 con 0x7f096813c510 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.969+0000 7f09701ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0968069520 con 0x7f09680cc100 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.969+0000 7f09701ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0968068910 con 0x7f096806bfd0 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096df9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f096813c510 0x7f096806b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45096/0 (socket says 172.21.15.131:45096) 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096df9b700 1 -- 172.21.15.131:0/1688498271 learned_addr learned my addr 172.21.15.131:0/1688498271 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096d79a700 1 --2- 172.21.15.131:0/1688498271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096806bfd0 0x7f09680c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096cf99700 1 -- 172.21.15.131:0/1688498271 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055374120 0 0) 0x7f09680686c0 con 0x7f096813c510 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096cf99700 1 -- 172.21.15.131:0/1688498271 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0954003570 con 0x7f096813c510 2023-12-26T21:53:49.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096cf99700 1 -- 172.21.15.131:0/1688498271 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146561565 0 0) 0x7f0968069520 con 0x7f09680cc100 2023-12-26T21:53:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096cf99700 1 -- 172.21.15.131:0/1688498271 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09680686c0 con 0x7f09680cc100 2023-12-26T21:53:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096d79a700 1 -- 172.21.15.131:0/1688498271 >> v1:172.21.15.190:6789/0 conn(0x7f09680cc100 legacy=0x7f09680cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096d79a700 1 -- 172.21.15.131:0/1688498271 >> v1:172.21.15.131:6789/0 conn(0x7f096813c510 legacy=0x7f096806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.970+0000 7f096d79a700 1 -- 172.21.15.131:0/1688498271 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968068bf0 con 0x7f096806bfd0 2023-12-26T21:53:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.971+0000 7f096d79a700 1 --2- 172.21.15.131:0/1688498271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096806bfd0 0x7f09680c8a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0958009910 tx=0x7f09580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daa57d3e62f0a055 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.971+0000 7f096cf99700 1 -- 172.21.15.131:0/1688498271 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958011630 con 0x7f096806bfd0 2023-12-26T21:53:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.971+0000 7f096cf99700 1 -- 172.21.15.131:0/1688498271 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0958011790 con 0x7f096806bfd0 2023-12-26T21:53:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.971+0000 7f096cf99700 1 -- 172.21.15.131:0/1688498271 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958011b30 con 0x7f096806bfd0 2023-12-26T21:53:49.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.971+0000 7f09701ff700 1 -- 172.21.15.131:0/1688498271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096806bfd0 msgr2=0x7f09680c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.971+0000 7f09701ff700 1 --2- 172.21.15.131:0/1688498271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096806bfd0 0x7f09680c8a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0958009910 tx=0x7f09580091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.972+0000 7f09701ff700 1 -- 172.21.15.131:0/1688498271 shutdown_connections 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.972+0000 7f09701ff700 1 --2- 172.21.15.131:0/1688498271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096806bfd0 0x7f09680c8a20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.972+0000 7f09701ff700 1 -- 172.21.15.131:0/1688498271 >> 172.21.15.131:0/1688498271 conn(0x7f09680ce3e0 msgr2=0x7f09680ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.972+0000 7f09701ff700 1 -- 172.21.15.131:0/1688498271 shutdown_connections 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.972+0000 7f09701ff700 1 -- 172.21.15.131:0/1688498271 wait complete. 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.973+0000 7f09701ff700 1 Processor -- start 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.973+0000 7f09701ff700 1 -- start start 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.974+0000 7f09701ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096806bfd0 0x7f0968158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.974+0000 7f09701ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680cc100 0x7f09681591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.974+0000 7f09701ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096813c510 0x7f09681394e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.974+0000 7f09701ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09681596f0 con 0x7f096806bfd0 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.974+0000 7f09701ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096816dac0 con 0x7f09680cc100 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.974+0000 7f09701ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096813d920 con 0x7f096813c510 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.974+0000 7f096df9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096806bfd0 0x7f0968158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.974+0000 7f096df9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096806bfd0 0x7f0968158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58770/0 (socket says 172.21.15.131:58770) 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.974+0000 7f096df9b700 1 -- 172.21.15.131:0/2757935153 learned_addr learned my addr 172.21.15.131:0/2757935153 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f096d79a700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680cc100 0x7f09681591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f096e79c700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096813c510 0x7f09681394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f096df9b700 1 -- 172.21.15.131:0/2757935153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096813c510 msgr2=0x7f09681394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f096df9b700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096813c510 0x7f09681394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f096df9b700 1 -- 172.21.15.131:0/2757935153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680cc100 msgr2=0x7f09681591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:49.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f096df9b700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680cc100 0x7f09681591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f096df9b700 1 -- 172.21.15.131:0/2757935153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958009420 con 0x7f096806bfd0 2023-12-26T21:53:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f096df9b700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096806bfd0 0x7f0968158320 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f096400c7c0 tx=0x7f0964007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f095dffb700 1 -- 172.21.15.131:0/2757935153 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09640072e0 con 0x7f096806bfd0 2023-12-26T21:53:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f095dffb700 1 -- 172.21.15.131:0/2757935153 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0964007440 con 0x7f096806bfd0 2023-12-26T21:53:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09681598e0 con 0x7f096806bfd0 2023-12-26T21:53:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f095dffb700 1 -- 172.21.15.131:0/2757935153 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09640057f0 con 0x7f096806bfd0 2023-12-26T21:53:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.975+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096816d470 con 0x7f096806bfd0 2023-12-26T21:53:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.976+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096816dda0 con 0x7f096806bfd0 2023-12-26T21:53:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.977+0000 7f095dffb700 1 -- 172.21.15.131:0/2757935153 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0964005ba0 con 0x7f096806bfd0 2023-12-26T21:53:49.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.978+0000 7f095dffb700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f094c066f30 0x7f094c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:49.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.978+0000 7f095dffb700 1 -- 172.21.15.131:0/2757935153 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f0964083ca0 con 0x7f096806bfd0 2023-12-26T21:53:49.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.978+0000 7f096d79a700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f094c066f30 0x7f094c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:49.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.979+0000 7f096d79a700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f094c066f30 0x7f094c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f096813f240 tx=0x7f0958017000 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:49.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:49.982+0000 7f095dffb700 1 -- 172.21.15.131:0/2757935153 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0964050ce0 con 0x7f096806bfd0 2023-12-26T21:53:50.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.180+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09680657e0 con 0x7f096806bfd0 2023-12-26T21:53:50.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.181+0000 7f095dffb700 1 -- 172.21.15.131:0/2757935153 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16008 (secure 0 0 0) 0x7f0964017020 con 0x7f096806bfd0 2023-12-26T21:53:50.182 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:50.182 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":110,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:53:47.273606+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:43.922429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:44.731388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:53:50.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.183+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f094c066f30 msgr2=0x7f094c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.183+0000 7f09701ff700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f094c066f30 0x7f094c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f096813f240 tx=0x7f0958017000 comp rx=0 tx=0).stop 2023-12-26T21:53:50.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.183+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096806bfd0 msgr2=0x7f0968158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.183+0000 7f09701ff700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096806bfd0 0x7f0968158320 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f096400c7c0 tx=0x7f0964007ce0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.184+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 shutdown_connections 2023-12-26T21:53:50.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.184+0000 7f09701ff700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f096813c510 0x7f09681394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.184+0000 7f09701ff700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f094c066f30 0x7f094c0693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.184+0000 7f09701ff700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f096806bfd0 0x7f0968158320 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.184+0000 7f09701ff700 1 --2- 172.21.15.131:0/2757935153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09680cc100 0x7f09681591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.184+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 >> 172.21.15.131:0/2757935153 conn(0x7f09680ce3e0 msgr2=0x7f09681538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:50.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.184+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 shutdown_connections 2023-12-26T21:53:50.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.184+0000 7f09701ff700 1 -- 172.21.15.131:0/2757935153 wait complete. 2023-12-26T21:53:50.203 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T21:53:50.203 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:50.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.394+0000 7f87e8bff700 1 Processor -- start 2023-12-26T21:53:50.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.395+0000 7f87e8bff700 1 -- start start 2023-12-26T21:53:50.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e8bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f87e40cb320 0x7f87e4069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e8bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87e40b0e60 0x7f87e40c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e8bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e4156940 con 0x7f87e406a1e0 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e8bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e40c8cc0 con 0x7f87e40b0e60 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e8bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e40ad700 con 0x7f87e40cb320 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e2d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87e40b0e60 0x7f87e40c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e2d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87e40b0e60 0x7f87e40c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58782/0 (socket says 172.21.15.131:58782) 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e2d9d700 1 -- 172.21.15.131:0/1964013413 learned_addr learned my addr 172.21.15.131:0/1964013413 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e2d9d700 1 -- 172.21.15.131:0/1964013413 >> v2:172.21.15.190:3300/0 conn(0x7f87e40cb320 msgr2=0x7f87e4069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.396+0000 7f87e259c700 1 --2- 172.21.15.131:0/1964013413 >> v2:172.21.15.190:3300/0 conn(0x7f87e40cb320 0x7f87e4069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.397+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/1964013413 >> v2:172.21.15.190:3300/0 conn(0x7f87e40cb320 0x7f87e4069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.397+0000 7f87e2d9d700 1 -- 172.21.15.131:0/1964013413 >> v1:172.21.15.190:6789/0 conn(0x7f87e406a1e0 legacy=0x7f87e40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.397+0000 7f87e2d9d700 1 -- 172.21.15.131:0/1964013413 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40ae500 con 0x7f87e40b0e60 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.397+0000 7f87e259c700 1 --2- 172.21.15.131:0/1964013413 >> v2:172.21.15.190:3300/0 conn(0x7f87e40cb320 0x7f87e4069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.397+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/1964013413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b0e60 0x7f87e40c9400 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f87e4066d80 tx=0x7f87dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6b2aa5adcfc263 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:50.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.397+0000 7f87e159a700 1 -- 172.21.15.131:0/1964013413 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00e070 con 0x7f87e40b0e60 2023-12-26T21:53:50.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.397+0000 7f87e159a700 1 -- 172.21.15.131:0/1964013413 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc00ba10 con 0x7f87e40b0e60 2023-12-26T21:53:50.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.397+0000 7f87e159a700 1 -- 172.21.15.131:0/1964013413 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc0092e0 con 0x7f87e40b0e60 2023-12-26T21:53:50.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.398+0000 7f87e8bff700 1 -- 172.21.15.131:0/1964013413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b0e60 msgr2=0x7f87e40c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.398+0000 7f87e8bff700 1 --2- 172.21.15.131:0/1964013413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b0e60 0x7f87e40c9400 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f87e4066d80 tx=0x7f87dc009840 comp rx=0 tx=0).stop 2023-12-26T21:53:50.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.398+0000 7f87e8bff700 1 -- 172.21.15.131:0/1964013413 shutdown_connections 2023-12-26T21:53:50.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.398+0000 7f87e8bff700 1 --2- 172.21.15.131:0/1964013413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40b0e60 0x7f87e40c9400 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.398+0000 7f87e8bff700 1 --2- 172.21.15.131:0/1964013413 >> v2:172.21.15.190:3300/0 conn(0x7f87e40cb320 0x7f87e4069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.398+0000 7f87e8bff700 1 -- 172.21.15.131:0/1964013413 >> 172.21.15.131:0/1964013413 conn(0x7f87e4068040 msgr2=0x7f87e4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:50.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.398+0000 7f87e8bff700 1 -- 172.21.15.131:0/1964013413 shutdown_connections 2023-12-26T21:53:50.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.398+0000 7f87e8bff700 1 -- 172.21.15.131:0/1964013413 wait complete. 2023-12-26T21:53:50.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.400+0000 7f87e8bff700 1 Processor -- start 2023-12-26T21:53:50.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.400+0000 7f87e8bff700 1 -- start start 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.401+0000 7f87e8bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406a1e0 0x7f87e40c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.401+0000 7f87e8bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e40b0e60 0x7f87e40c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.401+0000 7f87e8bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40cb320 0x7f87e4154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.401+0000 7f87e8bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40c7610 con 0x7f87e40cb320 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.401+0000 7f87e8bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40c6a00 con 0x7f87e40b0e60 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.401+0000 7f87e8bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40c6ce0 con 0x7f87e406a1e0 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.401+0000 7f87e2d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40cb320 0x7f87e4154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e2d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40cb320 0x7f87e4154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58794/0 (socket says 172.21.15.131:58794) 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406a1e0 0x7f87e40c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e2d9d700 1 -- 172.21.15.131:0/3035625594 learned_addr learned my addr 172.21.15.131:0/3035625594 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:50.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e1d9b700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e40b0e60 0x7f87e40c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e2d9d700 1 -- 172.21.15.131:0/3035625594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406a1e0 msgr2=0x7f87e40c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406a1e0 0x7f87e40c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e2d9d700 1 -- 172.21.15.131:0/3035625594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e40b0e60 msgr2=0x7f87e40c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e40b0e60 0x7f87e40c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e2d9d700 1 -- 172.21.15.131:0/3035625594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc009ae0 con 0x7f87e40cb320 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e2d9d700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40cb320 0x7f87e4154bf0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009880 tx=0x7f87dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87d27fc700 1 -- 172.21.15.131:0/3035625594 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc012d00 con 0x7f87e40cb320 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87d27fc700 1 -- 172.21.15.131:0/3035625594 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc012e80 con 0x7f87e40cb320 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87d27fc700 1 -- 172.21.15.131:0/3035625594 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc01d670 con 0x7f87e40cb320 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.402+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e40c6fc0 con 0x7f87e40cb320 2023-12-26T21:53:50.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.403+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e4155130 con 0x7f87e40cb320 2023-12-26T21:53:50.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.404+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e40ad510 con 0x7f87e40cb320 2023-12-26T21:53:50.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.405+0000 7f87d27fc700 1 -- 172.21.15.131:0/3035625594 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f87dc01d9e0 con 0x7f87e40cb320 2023-12-26T21:53:50.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.405+0000 7f87d27fc700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87c4066f30 0x7f87c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.406+0000 7f87d27fc700 1 -- 172.21.15.131:0/3035625594 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f87dc085ed0 con 0x7f87e40cb320 2023-12-26T21:53:50.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.406+0000 7f87e259c700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87c4066f30 0x7f87c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.406+0000 7f87e259c700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87c4066f30 0x7f87c40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87d4006fd0 tx=0x7f87d4008040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:50.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.410+0000 7f87d27fc700 1 -- 172.21.15.131:0/3035625594 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87dc052fb0 con 0x7f87e40cb320 2023-12-26T21:53:50.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.600+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87e40657e0 con 0x7f87c4066f30 2023-12-26T21:53:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.610+0000 7f87d27fc700 1 -- 172.21.15.131:0/3035625594 <== mgr.6975 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283598 (secure 0 0 0) 0x7f87e40657e0 con 0x7f87c4066f30 2023-12-26T21:53:50.614 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.614+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87c4066f30 msgr2=0x7f87c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.614+0000 7f87e8bff700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87c4066f30 0x7f87c40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87d4006fd0 tx=0x7f87d4008040 comp rx=0 tx=0).stop 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.614+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40cb320 msgr2=0x7f87e4154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.614+0000 7f87e8bff700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40cb320 0x7f87e4154bf0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009880 tx=0x7f87dc017000 comp rx=0 tx=0).stop 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.614+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 shutdown_connections 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.615+0000 7f87e8bff700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87c4066f30 0x7f87c40693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.615+0000 7f87e8bff700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e40cb320 0x7f87e4154bf0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.615+0000 7f87e8bff700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e406a1e0 0x7f87e40c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.615+0000 7f87e8bff700 1 --2- 172.21.15.131:0/3035625594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e40b0e60 0x7f87e40c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.615+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 >> 172.21.15.131:0/3035625594 conn(0x7f87e4068040 msgr2=0x7f87e4155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:50.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.615+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 shutdown_connections 2023-12-26T21:53:50.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.615+0000 7f87e8bff700 1 -- 172.21.15.131:0/3035625594 wait complete. 2023-12-26T21:53:50.618 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:50.633 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:53:50.112262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591104,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":336,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2451256,"kb_used_data":11512,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752523464,"statfs":{"total":773094113280,"available":770584027136,"internally_reserved":0,"allocated":11788288,"data_stored":6314654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.856251"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310309+0000","last_change":"2023-12-26T21:53:46.307184+0000","last_active":"2023-12-26T21:53:46.310309+0000","last_peered":"2023-12-26T21:53:46.310309+0000","last_clean":"2023-12-26T21:53:46.310309+0000","last_became_active":"2023-12-26T21:53:46.306640+0000","last_became_peered":"2023-12-26T21:53:46.306640+0000","last_unstale":"2023-12-26T21:53:46.310309+0000","last_undegraded":"2023-12-26T21:53:46.310309+0000","last_fullsized":"2023-12-26T21:53:46.310309+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:18.030160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306053+0000","last_change":"2023-12-26T21:53:46.299743+0000","last_active":"2023-12-26T21:53:46.306053+0000","last_peered":"2023-12-26T21:53:46.306053+0000","last_clean":"2023-12-26T21:53:46.306053+0000","last_became_active":"2023-12-26T21:53:46.299485+0000","last_became_peered":"2023-12-26T21:53:46.299485+0000","last_unstale":"2023-12-26T21:53:46.306053+0000","last_undegraded":"2023-12-26T21:53:46.306053+0000","last_fullsized":"2023-12-26T21:53:46.306053+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299767+0000","last_change":"2023-12-26T21:53:46.289973+0000","last_active":"2023-12-26T21:53:46.299767+0000","last_peered":"2023-12-26T21:53:46.299767+0000","last_clean":"2023-12-26T21:53:46.299767+0000","last_became_active":"2023-12-26T21:53:46.289695+0000","last_became_peered":"2023-12-26T21:53:46.289695+0000","last_unstale":"2023-12-26T21:53:46.299767+0000","last_undegraded":"2023-12-26T21:53:46.299767+0000","last_fullsized":"2023-12-26T21:53:46.299767+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:43:34.451389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307583+0000","last_change":"2023-12-26T21:53:46.296559+0000","last_active":"2023-12-26T21:53:46.307583+0000","last_peered":"2023-12-26T21:53:46.307583+0000","last_clean":"2023-12-26T21:53:46.307583+0000","last_became_active":"2023-12-26T21:53:46.296237+0000","last_became_peered":"2023-12-26T21:53:46.296237+0000","last_unstale":"2023-12-26T21:53:46.307583+0000","last_undegraded":"2023-12-26T21:53:46.307583+0000","last_fullsized":"2023-12-26T21:53:46.307583+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308991+0000","last_change":"2023-12-26T21:53:46.303931+0000","last_active":"2023-12-26T21:53:46.308991+0000","last_peered":"2023-12-26T21:53:46.308991+0000","last_clean":"2023-12-26T21:53:46.308991+0000","last_became_active":"2023-12-26T21:53:46.303627+0000","last_became_peered":"2023-12-26T21:53:46.303627+0000","last_unstale":"2023-12-26T21:53:46.308991+0000","last_undegraded":"2023-12-26T21:53:46.308991+0000","last_fullsized":"2023-12-26T21:53:46.308991+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:28.427569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314549+0000","last_change":"2023-12-26T21:53:46.308209+0000","last_active":"2023-12-26T21:53:46.314549+0000","last_peered":"2023-12-26T21:53:46.314549+0000","last_clean":"2023-12-26T21:53:46.314549+0000","last_became_active":"2023-12-26T21:53:46.308025+0000","last_became_peered":"2023-12-26T21:53:46.308025+0000","last_unstale":"2023-12-26T21:53:46.314549+0000","last_undegraded":"2023-12-26T21:53:46.314549+0000","last_fullsized":"2023-12-26T21:53:46.314549+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:53:11.039774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311289+0000","last_change":"2023-12-26T21:53:46.303517+0000","last_active":"2023-12-26T21:53:46.311289+0000","last_peered":"2023-12-26T21:53:46.311289+0000","last_clean":"2023-12-26T21:53:46.311289+0000","last_became_active":"2023-12-26T21:53:46.303137+0000","last_became_peered":"2023-12-26T21:53:46.303137+0000","last_unstale":"2023-12-26T21:53:46.311289+0000","last_undegraded":"2023-12-26T21:53:46.311289+0000","last_fullsized":"2023-12-26T21:53:46.311289+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314209+0000","last_change":"2023-12-26T21:53:46.308456+0000","last_active":"2023-12-26T21:53:46.314209+0000","last_peered":"2023-12-26T21:53:46.314209+0000","last_clean":"2023-12-26T21:53:46.314209+0000","last_became_active":"2023-12-26T21:53:46.308203+0000","last_became_peered":"2023-12-26T21:53:46.308203+0000","last_unstale":"2023-12-26T21:53:46.314209+0000","last_undegraded":"2023-12-26T21:53:46.314209+0000","last_fullsized":"2023-12-26T21:53:46.314209+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311951+0000","last_change":"2023-12-26T21:53:46.305482+0000","last_active":"2023-12-26T21:53:46.311951+0000","last_peered":"2023-12-26T21:53:46.311951+0000","last_clean":"2023-12-26T21:53:46.311951+0000","last_became_active":"2023-12-26T21:53:46.305210+0000","last_became_peered":"2023-12-26T21:53:46.305210+0000","last_unstale":"2023-12-26T21:53:46.311951+0000","last_undegraded":"2023-12-26T21:53:46.311951+0000","last_fullsized":"2023-12-26T21:53:46.311951+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306262+0000","last_change":"2023-12-26T21:53:46.301077+0000","last_active":"2023-12-26T21:53:46.306262+0000","last_peered":"2023-12-26T21:53:46.306262+0000","last_clean":"2023-12-26T21:53:46.306262+0000","last_became_active":"2023-12-26T21:53:46.300762+0000","last_became_peered":"2023-12-26T21:53:46.300762+0000","last_unstale":"2023-12-26T21:53:46.306262+0000","last_undegraded":"2023-12-26T21:53:46.306262+0000","last_fullsized":"2023-12-26T21:53:46.306262+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289642+0000","last_change":"2023-12-26T21:53:46.287813+0000","last_active":"2023-12-26T21:53:46.289642+0000","last_peered":"2023-12-26T21:53:46.289642+0000","last_clean":"2023-12-26T21:53:46.289642+0000","last_became_active":"2023-12-26T21:53:46.287544+0000","last_became_peered":"2023-12-26T21:53:46.287544+0000","last_unstale":"2023-12-26T21:53:46.289642+0000","last_undegraded":"2023-12-26T21:53:46.289642+0000","last_fullsized":"2023-12-26T21:53:46.289642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:27.394417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.297090+0000","last_change":"2023-12-26T21:53:46.287979+0000","last_active":"2023-12-26T21:53:46.297090+0000","last_peered":"2023-12-26T21:53:46.297090+0000","last_clean":"2023-12-26T21:53:46.297090+0000","last_became_active":"2023-12-26T21:53:46.287682+0000","last_became_peered":"2023-12-26T21:53:46.287682+0000","last_unstale":"2023-12-26T21:53:46.297090+0000","last_undegraded":"2023-12-26T21:53:46.297090+0000","last_fullsized":"2023-12-26T21:53:46.297090+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:38:29.898590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311484+0000","last_change":"2023-12-26T21:53:46.306111+0000","last_active":"2023-12-26T21:53:46.311484+0000","last_peered":"2023-12-26T21:53:46.311484+0000","last_clean":"2023-12-26T21:53:46.311484+0000","last_became_active":"2023-12-26T21:53:46.305852+0000","last_became_peered":"2023-12-26T21:53:46.305852+0000","last_unstale":"2023-12-26T21:53:46.311484+0000","last_undegraded":"2023-12-26T21:53:46.311484+0000","last_fullsized":"2023-12-26T21:53:46.311484+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:03:27.957388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311050+0000","last_change":"2023-12-26T21:53:46.305148+0000","last_active":"2023-12-26T21:53:46.311050+0000","last_peered":"2023-12-26T21:53:46.311050+0000","last_clean":"2023-12-26T21:53:46.311050+0000","last_became_active":"2023-12-26T21:53:46.304893+0000","last_became_peered":"2023-12-26T21:53:46.304893+0000","last_unstale":"2023-12-26T21:53:46.311050+0000","last_undegraded":"2023-12-26T21:53:46.311050+0000","last_fullsized":"2023-12-26T21:53:46.311050+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:05.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307642+0000","last_change":"2023-12-26T21:53:46.301677+0000","last_active":"2023-12-26T21:53:46.307642+0000","last_peered":"2023-12-26T21:53:46.307642+0000","last_clean":"2023-12-26T21:53:46.307642+0000","last_became_active":"2023-12-26T21:53:46.301432+0000","last_became_peered":"2023-12-26T21:53:46.301432+0000","last_unstale":"2023-12-26T21:53:46.307642+0000","last_undegraded":"2023-12-26T21:53:46.307642+0000","last_fullsized":"2023-12-26T21:53:46.307642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"109'4","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321799+0000","last_change":"2023-12-26T21:53:45.285708+0000","last_active":"2023-12-26T21:53:46.321799+0000","last_peered":"2023-12-26T21:53:46.321799+0000","last_clean":"2023-12-26T21:53:46.321799+0000","last_became_active":"2023-12-26T21:53:45.285466+0000","last_became_peered":"2023-12-26T21:53:45.285466+0000","last_unstale":"2023-12-26T21:53:46.321799+0000","last_undegraded":"2023-12-26T21:53:46.321799+0000","last_fullsized":"2023-12-26T21:53:46.321799+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:11.158488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310328+0000","last_change":"2023-12-26T21:53:46.307223+0000","last_active":"2023-12-26T21:53:46.310328+0000","last_peered":"2023-12-26T21:53:46.310328+0000","last_clean":"2023-12-26T21:53:46.310328+0000","last_became_active":"2023-12-26T21:53:46.306963+0000","last_became_peered":"2023-12-26T21:53:46.306963+0000","last_unstale":"2023-12-26T21:53:46.310328+0000","last_undegraded":"2023-12-26T21:53:46.310328+0000","last_fullsized":"2023-12-26T21:53:46.310328+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:12:37.634766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.264186+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:17.475634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262584+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:17.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.305631+0000","last_change":"2023-12-26T21:53:46.299487+0000","last_active":"2023-12-26T21:53:46.305631+0000","last_peered":"2023-12-26T21:53:46.305631+0000","last_clean":"2023-12-26T21:53:46.305631+0000","last_became_active":"2023-12-26T21:53:46.299063+0000","last_became_peered":"2023-12-26T21:53:46.299063+0000","last_unstale":"2023-12-26T21:53:46.305631+0000","last_undegraded":"2023-12-26T21:53:46.305631+0000","last_fullsized":"2023-12-26T21:53:46.305631+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:48.449758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262237+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:56:53.977018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258326+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:40:24.430368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.263523+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:08:23.502904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299738+0000","last_change":"2023-12-26T21:53:46.297785+0000","last_active":"2023-12-26T21:53:46.299738+0000","last_peered":"2023-12-26T21:53:46.299738+0000","last_clean":"2023-12-26T21:53:46.299738+0000","last_became_active":"2023-12-26T21:53:46.297505+0000","last_became_peered":"2023-12-26T21:53:46.297505+0000","last_unstale":"2023-12-26T21:53:46.299738+0000","last_undegraded":"2023-12-26T21:53:46.299738+0000","last_fullsized":"2023-12-26T21:53:46.299738+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:33.304213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281100+0000","last_change":"2023-12-26T21:53:45.279536+0000","last_active":"2023-12-26T21:53:46.281100+0000","last_peered":"2023-12-26T21:53:46.281100+0000","last_clean":"2023-12-26T21:53:46.281100+0000","last_became_active":"2023-12-26T21:53:45.279282+0000","last_became_peered":"2023-12-26T21:53:45.279282+0000","last_unstale":"2023-12-26T21:53:46.281100+0000","last_undegraded":"2023-12-26T21:53:46.281100+0000","last_fullsized":"2023-12-26T21:53:46.281100+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:58.794519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.288617+0000","last_change":"2023-12-26T21:53:46.285357+0000","last_active":"2023-12-26T21:53:46.288617+0000","last_peered":"2023-12-26T21:53:46.288617+0000","last_clean":"2023-12-26T21:53:46.288617+0000","last_became_active":"2023-12-26T21:53:46.285051+0000","last_became_peered":"2023-12-26T21:53:46.285051+0000","last_unstale":"2023-12-26T21:53:46.288617+0000","last_undegraded":"2023-12-26T21:53:46.288617+0000","last_fullsized":"2023-12-26T21:53:46.288617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:31:16.372604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281226+0000","last_change":"2023-12-26T21:53:45.268538+0000","last_active":"2023-12-26T21:53:46.281226+0000","last_peered":"2023-12-26T21:53:46.281226+0000","last_clean":"2023-12-26T21:53:46.281226+0000","last_became_active":"2023-12-26T21:53:45.268069+0000","last_became_peered":"2023-12-26T21:53:45.268069+0000","last_unstale":"2023-12-26T21:53:46.281226+0000","last_undegraded":"2023-12-26T21:53:46.281226+0000","last_fullsized":"2023-12-26T21:53:46.281226+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:25.546517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300878+0000","last_change":"2023-12-26T21:53:46.297819+0000","last_active":"2023-12-26T21:53:46.300878+0000","last_peered":"2023-12-26T21:53:46.300878+0000","last_clean":"2023-12-26T21:53:46.300878+0000","last_became_active":"2023-12-26T21:53:46.297541+0000","last_became_peered":"2023-12-26T21:53:46.297541+0000","last_unstale":"2023-12-26T21:53:46.300878+0000","last_undegraded":"2023-12-26T21:53:46.300878+0000","last_fullsized":"2023-12-26T21:53:46.300878+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:10:09.104721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289148+0000","last_change":"2023-12-26T21:53:45.281269+0000","last_active":"2023-12-26T21:53:46.289148+0000","last_peered":"2023-12-26T21:53:46.289148+0000","last_clean":"2023-12-26T21:53:46.289148+0000","last_became_active":"2023-12-26T21:53:45.280967+0000","last_became_peered":"2023-12-26T21:53:45.280967+0000","last_unstale":"2023-12-26T21:53:46.289148+0000","last_undegraded":"2023-12-26T21:53:46.289148+0000","last_fullsized":"2023-12-26T21:53:46.289148+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:20:00.172583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299929+0000","last_change":"2023-12-26T21:53:46.298319+0000","last_active":"2023-12-26T21:53:46.299929+0000","last_peered":"2023-12-26T21:53:46.299929+0000","last_clean":"2023-12-26T21:53:46.299929+0000","last_became_active":"2023-12-26T21:53:46.298050+0000","last_became_peered":"2023-12-26T21:53:46.298050+0000","last_unstale":"2023-12-26T21:53:46.299929+0000","last_undegraded":"2023-12-26T21:53:46.299929+0000","last_fullsized":"2023-12-26T21:53:46.299929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:34.648293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262776+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:43:16.270159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"109'3","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321640+0000","last_change":"2023-12-26T21:53:45.285950+0000","last_active":"2023-12-26T21:53:46.321640+0000","last_peered":"2023-12-26T21:53:46.321640+0000","last_clean":"2023-12-26T21:53:46.321640+0000","last_became_active":"2023-12-26T21:53:45.285657+0000","last_became_peered":"2023-12-26T21:53:45.285657+0000","last_unstale":"2023-12-26T21:53:46.321640+0000","last_undegraded":"2023-12-26T21:53:46.321640+0000","last_fullsized":"2023-12-26T21:53:46.321640+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:55.530687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303089+0000","last_change":"2023-12-26T21:53:46.301426+0000","last_active":"2023-12-26T21:53:46.303089+0000","last_peered":"2023-12-26T21:53:46.303089+0000","last_clean":"2023-12-26T21:53:46.303089+0000","last_became_active":"2023-12-26T21:53:46.301116+0000","last_became_peered":"2023-12-26T21:53:46.301116+0000","last_unstale":"2023-12-26T21:53:46.303089+0000","last_undegraded":"2023-12-26T21:53:46.303089+0000","last_fullsized":"2023-12-26T21:53:46.303089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:49.402467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258535+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:44:24.266594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262178+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:30:14.941645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304318+0000","last_change":"2023-12-26T21:53:46.298424+0000","last_active":"2023-12-26T21:53:46.304318+0000","last_peered":"2023-12-26T21:53:46.304318+0000","last_clean":"2023-12-26T21:53:46.304318+0000","last_became_active":"2023-12-26T21:53:46.298145+0000","last_became_peered":"2023-12-26T21:53:46.298145+0000","last_unstale":"2023-12-26T21:53:46.304318+0000","last_undegraded":"2023-12-26T21:53:46.304318+0000","last_fullsized":"2023-12-26T21:53:46.304318+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:24:04.826615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285162+0000","last_change":"2023-12-26T21:53:45.281444+0000","last_active":"2023-12-26T21:53:46.285162+0000","last_peered":"2023-12-26T21:53:46.285162+0000","last_clean":"2023-12-26T21:53:46.285162+0000","last_became_active":"2023-12-26T21:53:45.280420+0000","last_became_peered":"2023-12-26T21:53:45.280420+0000","last_unstale":"2023-12-26T21:53:46.285162+0000","last_undegraded":"2023-12-26T21:53:46.285162+0000","last_fullsized":"2023-12-26T21:53:46.285162+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:17.475634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300883+0000","last_change":"2023-12-26T21:53:46.297808+0000","last_active":"2023-12-26T21:53:46.300883+0000","last_peered":"2023-12-26T21:53:46.300883+0000","last_clean":"2023-12-26T21:53:46.300883+0000","last_became_active":"2023-12-26T21:53:46.296873+0000","last_became_peered":"2023-12-26T21:53:46.296873+0000","last_unstale":"2023-12-26T21:53:46.300883+0000","last_undegraded":"2023-12-26T21:53:46.300883+0000","last_fullsized":"2023-12-26T21:53:46.300883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:29.573501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262772+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:25:10.062797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262093+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:12:36.629000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.320094+0000","last_change":"2023-12-26T21:53:45.280067+0000","last_active":"2023-12-26T21:53:46.320094+0000","last_peered":"2023-12-26T21:53:46.320094+0000","last_clean":"2023-12-26T21:53:46.320094+0000","last_became_active":"2023-12-26T21:53:45.279713+0000","last_became_peered":"2023-12-26T21:53:45.279713+0000","last_unstale":"2023-12-26T21:53:46.320094+0000","last_undegraded":"2023-12-26T21:53:46.320094+0000","last_fullsized":"2023-12-26T21:53:46.320094+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:15.927146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.316477+0000","last_change":"2023-12-26T21:53:45.277896+0000","last_active":"2023-12-26T21:53:46.316477+0000","last_peered":"2023-12-26T21:53:46.316477+0000","last_clean":"2023-12-26T21:53:46.316477+0000","last_became_active":"2023-12-26T21:53:45.277584+0000","last_became_peered":"2023-12-26T21:53:45.277584+0000","last_unstale":"2023-12-26T21:53:46.316477+0000","last_undegraded":"2023-12-26T21:53:46.316477+0000","last_fullsized":"2023-12-26T21:53:46.316477+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:44:17.169190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.282526+0000","last_change":"2023-12-26T21:53:45.280486+0000","last_active":"2023-12-26T21:53:46.282526+0000","last_peered":"2023-12-26T21:53:46.282526+0000","last_clean":"2023-12-26T21:53:46.282526+0000","last_became_active":"2023-12-26T21:53:45.280178+0000","last_became_peered":"2023-12-26T21:53:45.280178+0000","last_unstale":"2023-12-26T21:53:46.282526+0000","last_undegraded":"2023-12-26T21:53:46.282526+0000","last_fullsized":"2023-12-26T21:53:46.282526+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:13:56.557753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304197+0000","last_change":"2023-12-26T21:53:46.297254+0000","last_active":"2023-12-26T21:53:46.304197+0000","last_peered":"2023-12-26T21:53:46.304197+0000","last_clean":"2023-12-26T21:53:46.304197+0000","last_became_active":"2023-12-26T21:53:46.296358+0000","last_became_peered":"2023-12-26T21:53:46.296358+0000","last_unstale":"2023-12-26T21:53:46.304197+0000","last_undegraded":"2023-12-26T21:53:46.304197+0000","last_fullsized":"2023-12-26T21:53:46.304197+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.317134+0000","last_change":"2023-12-26T21:53:45.271721+0000","last_active":"2023-12-26T21:53:46.317134+0000","last_peered":"2023-12-26T21:53:46.317134+0000","last_clean":"2023-12-26T21:53:46.317134+0000","last_became_active":"2023-12-26T21:53:45.271453+0000","last_became_peered":"2023-12-26T21:53:45.271453+0000","last_unstale":"2023-12-26T21:53:46.317134+0000","last_undegraded":"2023-12-26T21:53:46.317134+0000","last_fullsized":"2023-12-26T21:53:46.317134+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:49:24.818172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303057+0000","last_change":"2023-12-26T21:53:46.301435+0000","last_active":"2023-12-26T21:53:46.303057+0000","last_peered":"2023-12-26T21:53:46.303057+0000","last_clean":"2023-12-26T21:53:46.303057+0000","last_became_active":"2023-12-26T21:53:46.301117+0000","last_became_peered":"2023-12-26T21:53:46.301117+0000","last_unstale":"2023-12-26T21:53:46.303057+0000","last_undegraded":"2023-12-26T21:53:46.303057+0000","last_fullsized":"2023-12-26T21:53:46.303057+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:39.625634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285208+0000","last_change":"2023-12-26T21:53:45.279979+0000","last_active":"2023-12-26T21:53:46.285208+0000","last_peered":"2023-12-26T21:53:46.285208+0000","last_clean":"2023-12-26T21:53:46.285208+0000","last_became_active":"2023-12-26T21:53:45.279563+0000","last_became_peered":"2023-12-26T21:53:45.279563+0000","last_unstale":"2023-12-26T21:53:46.285208+0000","last_undegraded":"2023-12-26T21:53:46.285208+0000","last_fullsized":"2023-12-26T21:53:46.285208+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:00:28.640861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262063+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:32.298447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281331+0000","last_change":"2023-12-26T21:53:45.282408+0000","last_active":"2023-12-26T21:53:46.281331+0000","last_peered":"2023-12-26T21:53:46.281331+0000","last_clean":"2023-12-26T21:53:46.281331+0000","last_became_active":"2023-12-26T21:53:45.282074+0000","last_became_peered":"2023-12-26T21:53:45.282074+0000","last_unstale":"2023-12-26T21:53:46.281331+0000","last_undegraded":"2023-12-26T21:53:46.281331+0000","last_fullsized":"2023-12-26T21:53:46.281331+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:18:52.576472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.302717+0000","last_change":"2023-12-26T21:53:46.300574+0000","last_active":"2023-12-26T21:53:46.302717+0000","last_peered":"2023-12-26T21:53:46.302717+0000","last_clean":"2023-12-26T21:53:46.302717+0000","last_became_active":"2023-12-26T21:53:46.300296+0000","last_became_peered":"2023-12-26T21:53:46.300296+0000","last_unstale":"2023-12-26T21:53:46.302717+0000","last_undegraded":"2023-12-26T21:53:46.302717+0000","last_fullsized":"2023-12-26T21:53:46.302717+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:51.026196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.318605+0000","last_change":"2023-12-26T21:53:45.280489+0000","last_active":"2023-12-26T21:53:46.318605+0000","last_peered":"2023-12-26T21:53:46.318605+0000","last_clean":"2023-12-26T21:53:46.318605+0000","last_became_active":"2023-12-26T21:53:45.279071+0000","last_became_peered":"2023-12-26T21:53:45.279071+0000","last_unstale":"2023-12-26T21:53:46.318605+0000","last_undegraded":"2023-12-26T21:53:46.318605+0000","last_fullsized":"2023-12-26T21:53:46.318605+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:28.851567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308510+0000","last_change":"2023-12-26T21:53:46.304923+0000","last_active":"2023-12-26T21:53:46.308510+0000","last_peered":"2023-12-26T21:53:46.308510+0000","last_clean":"2023-12-26T21:53:46.308510+0000","last_became_active":"2023-12-26T21:53:46.304621+0000","last_became_peered":"2023-12-26T21:53:46.304621+0000","last_unstale":"2023-12-26T21:53:46.308510+0000","last_undegraded":"2023-12-26T21:53:46.308510+0000","last_fullsized":"2023-12-26T21:53:46.308510+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:15:24.617194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262644+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:16.240935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299826+0000","last_change":"2023-12-26T21:53:46.288775+0000","last_active":"2023-12-26T21:53:46.299826+0000","last_peered":"2023-12-26T21:53:46.299826+0000","last_clean":"2023-12-26T21:53:46.299826+0000","last_became_active":"2023-12-26T21:53:46.288489+0000","last_became_peered":"2023-12-26T21:53:46.288489+0000","last_unstale":"2023-12-26T21:53:46.299826+0000","last_undegraded":"2023-12-26T21:53:46.299826+0000","last_fullsized":"2023-12-26T21:53:46.299826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:48:03.925563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258576+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:59:57.204768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262786+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:00:28.640861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310423+0000","last_change":"2023-12-26T21:53:46.306701+0000","last_active":"2023-12-26T21:53:46.310423+0000","last_peered":"2023-12-26T21:53:46.310423+0000","last_clean":"2023-12-26T21:53:46.310423+0000","last_became_active":"2023-12-26T21:53:46.306450+0000","last_became_peered":"2023-12-26T21:53:46.306450+0000","last_unstale":"2023-12-26T21:53:46.310423+0000","last_undegraded":"2023-12-26T21:53:46.310423+0000","last_fullsized":"2023-12-26T21:53:46.310423+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:56.921774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.284986+0000","last_change":"2023-12-26T21:53:45.279374+0000","last_active":"2023-12-26T21:53:46.284986+0000","last_peered":"2023-12-26T21:53:46.284986+0000","last_clean":"2023-12-26T21:53:46.284986+0000","last_became_active":"2023-12-26T21:53:45.278479+0000","last_became_peered":"2023-12-26T21:53:45.278479+0000","last_unstale":"2023-12-26T21:53:46.284986+0000","last_undegraded":"2023-12-26T21:53:46.284986+0000","last_fullsized":"2023-12-26T21:53:46.284986+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:29.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262986+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:29.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308046+0000","last_change":"2023-12-26T21:53:46.306183+0000","last_active":"2023-12-26T21:53:46.308046+0000","last_peered":"2023-12-26T21:53:46.308046+0000","last_clean":"2023-12-26T21:53:46.308046+0000","last_became_active":"2023-12-26T21:53:46.305918+0000","last_became_peered":"2023-12-26T21:53:46.305918+0000","last_unstale":"2023-12-26T21:53:46.308046+0000","last_undegraded":"2023-12-26T21:53:46.308046+0000","last_fullsized":"2023-12-26T21:53:46.308046+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:49:38.482545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286935+0000","last_change":"2023-12-26T21:53:45.284416+0000","last_active":"2023-12-26T21:53:46.286935+0000","last_peered":"2023-12-26T21:53:46.286935+0000","last_clean":"2023-12-26T21:53:46.286935+0000","last_became_active":"2023-12-26T21:53:45.284072+0000","last_became_peered":"2023-12-26T21:53:45.284072+0000","last_unstale":"2023-12-26T21:53:46.286935+0000","last_undegraded":"2023-12-26T21:53:46.286935+0000","last_fullsized":"2023-12-26T21:53:46.286935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:46:59.726957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307933+0000","last_change":"2023-12-26T21:53:46.303791+0000","last_active":"2023-12-26T21:53:46.307933+0000","last_peered":"2023-12-26T21:53:46.307933+0000","last_clean":"2023-12-26T21:53:46.307933+0000","last_became_active":"2023-12-26T21:53:46.303518+0000","last_became_peered":"2023-12-26T21:53:46.303518+0000","last_unstale":"2023-12-26T21:53:46.307933+0000","last_undegraded":"2023-12-26T21:53:46.307933+0000","last_fullsized":"2023-12-26T21:53:46.307933+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:17.246701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258326+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:17:49.107900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.319541+0000","last_change":"2023-12-26T21:53:45.277923+0000","last_active":"2023-12-26T21:53:46.319541+0000","last_peered":"2023-12-26T21:53:46.319541+0000","last_clean":"2023-12-26T21:53:46.319541+0000","last_became_active":"2023-12-26T21:53:45.277681+0000","last_became_peered":"2023-12-26T21:53:45.277681+0000","last_unstale":"2023-12-26T21:53:46.319541+0000","last_undegraded":"2023-12-26T21:53:46.319541+0000","last_fullsized":"2023-12-26T21:53:46.319541+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:40.310455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'205","reported_seq":555,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286580+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:46.286580+0000","last_peered":"2023-12-26T21:53:46.286580+0000","last_clean":"2023-12-26T21:53:46.286580+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:46.286580+0000","last_undegraded":"2023-12-26T21:53:46.286580+0000","last_fullsized":"2023-12-26T21:53:46.286580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":4},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542348,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302700,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069140,"statfs":{"total":96636764160,"available":96326799360,"internally_reserved":0,"allocated":1347584,"data_stored":653499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.47199999999999998,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.72499999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.69799999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74399999999999999,"5min":1.016,"15min":1.048},"last":0.74199999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.038,"15min":1.0509999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.85699999999999998,"5min":0.99199999999999999,"15min":1.02},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.62,"5min":0.504,"15min":0.5},"max":{"1min":0.94299999999999995,"5min":1.0189999999999999,"15min":1.052},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83499999999999996,"5min":1.036,"15min":1.036},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.747,"15min":0.746},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0129999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86399999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.117,"15min":1.117},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.437},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.006},"last":0.88}]}]},{"osd":6,"up_from":15,"seq":64424509643,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306744,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065096,"statfs":{"total":96636764160,"available":96322658304,"internally_reserved":0,"allocated":1294336,"data_stored":626742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.085},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69199999999999995,"15min":0.71799999999999997},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.34799999999999998},"max":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67800000000000005,"15min":0.70199999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.95299999999999996,"15min":0.98799999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.54200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.97599999999999998,"15min":1.054},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.93200000000000005,"15min":1.105},"last":0.753},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.79200000000000004,"5min":0.92000000000000004,"15min":1.048},"last":0.83999999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.90200000000000002,"5min":1.026,"15min":1.1799999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86899999999999999,"5min":0.96999999999999997,"15min":1.069},"last":0.85599999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.91100000000000003,"15min":1.137},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93100000000000005,"5min":1.0049999999999999,"15min":1.296},"last":0.78200000000000003}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.437},"max":{"1min":1.008,"5min":1.008,"15min":1.3380000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.56799999999999995,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.95599999999999996,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.87}]}]},{"osd":1,"up_from":15,"seq":64424509644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311472,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060368,"statfs":{"total":96636764160,"available":96317816832,"internally_reserved":0,"allocated":1941504,"data_stored":1243887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.436},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.55700000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.84699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.70399999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.87}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.88400000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.90500000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509643,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302660,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069180,"statfs":{"total":96636764160,"available":96326840320,"internally_reserved":0,"allocated":1306624,"data_stored":632939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":1.056},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47799999999999998,"15min":0.49399999999999999},"min":{"1min":0.315,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":1.006},"last":0.311},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.84699999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80000000000000004,"5min":0.86699999999999999,"15min":1.1870000000000001},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.752},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.042,"5min":1.042,"15min":1.411},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.76800000000000002},"min":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.2070000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.754,"15min":0.79800000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.2569999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77100000000000002,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.452},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.214},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.012,"15min":1.3779999999999999},"last":0.50600000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46400000000000002},"max":{"1min":1.018,"5min":1.042,"15min":1.4359999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.79200000000000004},"min":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":1.0029999999999999,"5min":1.0069999999999999,"15min":1.3240000000000001},"last":0.49299999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306744,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065096,"statfs":{"total":96636764160,"available":96322658304,"internally_reserved":0,"allocated":1294336,"data_stored":626742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.877},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68600000000000005,"5min":0.83899999999999997,"15min":0.84399999999999997},"last":0.433}]},{"osd":1,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.65300000000000002,"5min":0.70799999999999996,"15min":0.92000000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.31,"15min":0.309},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.78600000000000003},"last":0.38900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":1,"15min":1},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.82299999999999995,"15min":0.85699999999999998},"last":0.34599999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.46800000000000003,"15min":0.34799999999999998},"max":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":1.284},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.76300000000000001},"min":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.95699999999999996,"5min":0.98399999999999999,"15min":1.1759999999999999},"last":0.83699999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.024,"5min":1.0600000000000001,"15min":1.2170000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":0.95499999999999996,"5min":1.014,"15min":1.1319999999999999},"last":0.86499999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.038,"15min":1.242},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.40300000000000002},"max":{"1min":1.018,"5min":1.0609999999999999,"15min":1.272},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":1.073,"15min":1.1220000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.50800000000000001},"max":{"1min":0.999,"5min":1.0189999999999999,"15min":1.1990000000000001},"last":0.84899999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509645,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310900,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060940,"statfs":{"total":96636764160,"available":96318402560,"internally_reserved":0,"allocated":1355776,"data_stored":654075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.378}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82499999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.753}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.81299999999999994}]}]},{"osd":4,"up_from":14,"seq":60129542348,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306796,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065044,"statfs":{"total":96636764160,"available":96322605056,"internally_reserved":0,"allocated":1347584,"data_stored":653463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97899999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.48499999999999999,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":1.008},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.95199999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88900000000000001,"5min":0.97199999999999998,"15min":0.97299999999999998},"last":0.624}]},{"osd":3,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.71499999999999997},"min":{"1min":0.504,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.78100000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.438,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.84399999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509642,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":1223307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.69999999999999996},"min":{"1min":0.44500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84199999999999997,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.749},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.4570000000000001},"last":0.73399999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68700000000000006,"15min":0.71499999999999997},"min":{"1min":0.499,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.625},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67800000000000005,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.4219999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.72799999999999998},"min":{"1min":0.54900000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.88500000000000001,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68600000000000005,"15min":0.71099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.92400000000000004,"15min":1.0569999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":1.052,"15min":1.4890000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.73599999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47799999999999998},"max":{"1min":0.88300000000000001,"5min":0.92200000000000004,"15min":1.081},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":2.5070000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.54500000000000004,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":2.8039999999999998},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.65700000000000003,"15min":0.66400000000000003},"min":{"1min":0.505,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":1.637},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.623,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":1.111,"5min":1.111,"15min":2.5579999999999998},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.61399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.97499999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.71299999999999997,"15min":0.70999999999999996},"min":{"1min":0.52200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.5980000000000001},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:50.639 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:50.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.863+0000 7f96097ff700 1 Processor -- start 2023-12-26T21:53:50.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.863+0000 7f96097ff700 1 -- start start 2023-12-26T21:53:50.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.864+0000 7f96097ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f96040b0750 0x7f96040b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.864+0000 7f96097ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96040c9400 0x7f96040c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.864+0000 7f96097ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9604156940 con 0x7f96040cb320 2023-12-26T21:53:50.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.864+0000 7f96097ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040c8cc0 con 0x7f96040b0750 2023-12-26T21:53:50.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.864+0000 7f96097ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040ad700 con 0x7f96040c9400 2023-12-26T21:53:50.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f9602ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f96040cb320 0x7f9604069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45102/0 (socket says 172.21.15.131:45102) 2023-12-26T21:53:50.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f9602ffd700 1 -- 172.21.15.131:0/1046712961 learned_addr learned my addr 172.21.15.131:0/1046712961 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:50.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f96027fc700 1 --2- 172.21.15.131:0/1046712961 >> v2:172.21.15.131:3300/0 conn(0x7f96040b0750 0x7f96040b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f96037fe700 1 --2- 172.21.15.131:0/1046712961 >> v2:172.21.15.190:3300/0 conn(0x7f96040c9400 0x7f96040c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f9601ffb700 1 -- 172.21.15.131:0/1046712961 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907577521 0 0) 0x7f9604156940 con 0x7f96040cb320 2023-12-26T21:53:50.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f9601ffb700 1 -- 172.21.15.131:0/1046712961 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ec003570 con 0x7f96040cb320 2023-12-26T21:53:50.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f96027fc700 1 -- 172.21.15.131:0/1046712961 >> v2:172.21.15.190:3300/0 conn(0x7f96040c9400 msgr2=0x7f96040c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f96027fc700 1 --2- 172.21.15.131:0/1046712961 >> v2:172.21.15.190:3300/0 conn(0x7f96040c9400 0x7f96040c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f96027fc700 1 -- 172.21.15.131:0/1046712961 >> v1:172.21.15.131:6789/0 conn(0x7f96040cb320 legacy=0x7f9604069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.865+0000 7f96027fc700 1 -- 172.21.15.131:0/1046712961 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040ae500 con 0x7f96040b0750 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f96027fc700 1 --2- 172.21.15.131:0/1046712961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 0x7f96040b0bf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f960406fb50 tx=0x7f95f800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8982021a46aa421b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f9601ffb700 1 -- 172.21.15.131:0/1046712961 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800da30 con 0x7f96040b0750 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f9601ffb700 1 -- 172.21.15.131:0/1046712961 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f800db90 con 0x7f96040b0750 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f9601ffb700 1 -- 172.21.15.131:0/1046712961 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8019480 con 0x7f96040b0750 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f96097ff700 1 -- 172.21.15.131:0/1046712961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 msgr2=0x7f96040b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f96097ff700 1 --2- 172.21.15.131:0/1046712961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 0x7f96040b0bf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f960406fb50 tx=0x7f95f800b430 comp rx=0 tx=0).stop 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f96097ff700 1 -- 172.21.15.131:0/1046712961 shutdown_connections 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f96097ff700 1 --2- 172.21.15.131:0/1046712961 >> v2:172.21.15.190:3300/0 conn(0x7f96040c9400 0x7f96040c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f96097ff700 1 --2- 172.21.15.131:0/1046712961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 0x7f96040b0bf0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f96097ff700 1 -- 172.21.15.131:0/1046712961 >> 172.21.15.131:0/1046712961 conn(0x7f9604068040 msgr2=0x7f9604069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:50.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f96097ff700 1 -- 172.21.15.131:0/1046712961 shutdown_connections 2023-12-26T21:53:50.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.866+0000 7f96097ff700 1 -- 172.21.15.131:0/1046712961 wait complete. 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.868+0000 7f96097ff700 1 Processor -- start 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.868+0000 7f96097ff700 1 -- start start 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f96097ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 0x7f96040a7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f96097ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96040c9400 0x7f96040aa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f96097ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96040cb320 0x7f9604148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f96097ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040a8340 con 0x7f96040b0750 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f96097ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040a8620 con 0x7f96040cb320 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f96097ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96040a8900 con 0x7f96040c9400 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f9602ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 0x7f96040a7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f9602ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 0x7f96040a7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58810/0 (socket says 172.21.15.131:58810) 2023-12-26T21:53:50.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f9602ffd700 1 -- 172.21.15.131:0/3357907954 learned_addr learned my addr 172.21.15.131:0/3357907954 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f96027fc700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96040c9400 0x7f96040aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.869+0000 7f96037fe700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96040cb320 0x7f9604148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f9602ffd700 1 -- 172.21.15.131:0/3357907954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96040c9400 msgr2=0x7f96040aa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f9602ffd700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96040c9400 0x7f96040aa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f9602ffd700 1 -- 172.21.15.131:0/3357907954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96040cb320 msgr2=0x7f9604148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f9602ffd700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96040cb320 0x7f9604148bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f9602ffd700 1 -- 172.21.15.131:0/3357907954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f800b6d0 con 0x7f96040b0750 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f9602ffd700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 0x7f96040a7d10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f95f400c590 tx=0x7f95f4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f95eaffd700 1 -- 172.21.15.131:0/3357907954 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4005b30 con 0x7f96040b0750 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f95eaffd700 1 -- 172.21.15.131:0/3357907954 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f4005cb0 con 0x7f96040b0750 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f95eaffd700 1 -- 172.21.15.131:0/3357907954 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4007320 con 0x7f96040b0750 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96040a8be0 con 0x7f96040b0750 2023-12-26T21:53:50.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.870+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96040aa7c0 con 0x7f96040b0750 2023-12-26T21:53:50.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.871+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96040657e0 con 0x7f96040b0750 2023-12-26T21:53:50.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.872+0000 7f95eaffd700 1 -- 172.21.15.131:0/3357907954 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f95f401b3b0 con 0x7f96040b0750 2023-12-26T21:53:50.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.873+0000 7f95eaffd700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f95f0066f30 0x7f95f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:50.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.873+0000 7f95eaffd700 1 -- 172.21.15.131:0/3357907954 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f95f4083c80 con 0x7f96040b0750 2023-12-26T21:53:50.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.873+0000 7f96027fc700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f95f0066f30 0x7f95f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:50.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.874+0000 7f96027fc700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f95f0066f30 0x7f95f00693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95f800bb10 tx=0x7f95f801b000 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:50.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:50.877+0000 7f95eaffd700 1 -- 172.21.15.131:0/3357907954 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95f4050d60 con 0x7f96040b0750 2023-12-26T21:53:51.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.066+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96040a8790 con 0x7f95f0066f30 2023-12-26T21:53:51.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.075+0000 7f95eaffd700 1 -- 172.21.15.131:0/3357907954 <== mgr.6975 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283598 (secure 0 0 0) 0x7f96040a8790 con 0x7f95f0066f30 2023-12-26T21:53:51.079 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f95f0066f30 msgr2=0x7f95f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f95f0066f30 0x7f95f00693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95f800bb10 tx=0x7f95f801b000 comp rx=0 tx=0).stop 2023-12-26T21:53:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 msgr2=0x7f96040a7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 0x7f96040a7d10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f95f400c590 tx=0x7f95f4007e00 comp rx=0 tx=0).stop 2023-12-26T21:53:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 shutdown_connections 2023-12-26T21:53:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96040cb320 0x7f9604148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f95f0066f30 0x7f95f00693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96040b0750 0x7f96040a7d10 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 --2- 172.21.15.131:0/3357907954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96040c9400 0x7f96040aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 >> 172.21.15.131:0/3357907954 conn(0x7f9604068040 msgr2=0x7f96040acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 shutdown_connections 2023-12-26T21:53:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.079+0000 7f96097ff700 1 -- 172.21.15.131:0/3357907954 wait complete. 2023-12-26T21:53:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:51.097 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:53:50.112262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591104,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":336,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2451256,"kb_used_data":11512,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752523464,"statfs":{"total":773094113280,"available":770584027136,"internally_reserved":0,"allocated":11788288,"data_stored":6314654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.856251"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310309+0000","last_change":"2023-12-26T21:53:46.307184+0000","last_active":"2023-12-26T21:53:46.310309+0000","last_peered":"2023-12-26T21:53:46.310309+0000","last_clean":"2023-12-26T21:53:46.310309+0000","last_became_active":"2023-12-26T21:53:46.306640+0000","last_became_peered":"2023-12-26T21:53:46.306640+0000","last_unstale":"2023-12-26T21:53:46.310309+0000","last_undegraded":"2023-12-26T21:53:46.310309+0000","last_fullsized":"2023-12-26T21:53:46.310309+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:18.030160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306053+0000","last_change":"2023-12-26T21:53:46.299743+0000","last_active":"2023-12-26T21:53:46.306053+0000","last_peered":"2023-12-26T21:53:46.306053+0000","last_clean":"2023-12-26T21:53:46.306053+0000","last_became_active":"2023-12-26T21:53:46.299485+0000","last_became_peered":"2023-12-26T21:53:46.299485+0000","last_unstale":"2023-12-26T21:53:46.306053+0000","last_undegraded":"2023-12-26T21:53:46.306053+0000","last_fullsized":"2023-12-26T21:53:46.306053+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299767+0000","last_change":"2023-12-26T21:53:46.289973+0000","last_active":"2023-12-26T21:53:46.299767+0000","last_peered":"2023-12-26T21:53:46.299767+0000","last_clean":"2023-12-26T21:53:46.299767+0000","last_became_active":"2023-12-26T21:53:46.289695+0000","last_became_peered":"2023-12-26T21:53:46.289695+0000","last_unstale":"2023-12-26T21:53:46.299767+0000","last_undegraded":"2023-12-26T21:53:46.299767+0000","last_fullsized":"2023-12-26T21:53:46.299767+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:43:34.451389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307583+0000","last_change":"2023-12-26T21:53:46.296559+0000","last_active":"2023-12-26T21:53:46.307583+0000","last_peered":"2023-12-26T21:53:46.307583+0000","last_clean":"2023-12-26T21:53:46.307583+0000","last_became_active":"2023-12-26T21:53:46.296237+0000","last_became_peered":"2023-12-26T21:53:46.296237+0000","last_unstale":"2023-12-26T21:53:46.307583+0000","last_undegraded":"2023-12-26T21:53:46.307583+0000","last_fullsized":"2023-12-26T21:53:46.307583+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308991+0000","last_change":"2023-12-26T21:53:46.303931+0000","last_active":"2023-12-26T21:53:46.308991+0000","last_peered":"2023-12-26T21:53:46.308991+0000","last_clean":"2023-12-26T21:53:46.308991+0000","last_became_active":"2023-12-26T21:53:46.303627+0000","last_became_peered":"2023-12-26T21:53:46.303627+0000","last_unstale":"2023-12-26T21:53:46.308991+0000","last_undegraded":"2023-12-26T21:53:46.308991+0000","last_fullsized":"2023-12-26T21:53:46.308991+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:28.427569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314549+0000","last_change":"2023-12-26T21:53:46.308209+0000","last_active":"2023-12-26T21:53:46.314549+0000","last_peered":"2023-12-26T21:53:46.314549+0000","last_clean":"2023-12-26T21:53:46.314549+0000","last_became_active":"2023-12-26T21:53:46.308025+0000","last_became_peered":"2023-12-26T21:53:46.308025+0000","last_unstale":"2023-12-26T21:53:46.314549+0000","last_undegraded":"2023-12-26T21:53:46.314549+0000","last_fullsized":"2023-12-26T21:53:46.314549+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:53:11.039774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311289+0000","last_change":"2023-12-26T21:53:46.303517+0000","last_active":"2023-12-26T21:53:46.311289+0000","last_peered":"2023-12-26T21:53:46.311289+0000","last_clean":"2023-12-26T21:53:46.311289+0000","last_became_active":"2023-12-26T21:53:46.303137+0000","last_became_peered":"2023-12-26T21:53:46.303137+0000","last_unstale":"2023-12-26T21:53:46.311289+0000","last_undegraded":"2023-12-26T21:53:46.311289+0000","last_fullsized":"2023-12-26T21:53:46.311289+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314209+0000","last_change":"2023-12-26T21:53:46.308456+0000","last_active":"2023-12-26T21:53:46.314209+0000","last_peered":"2023-12-26T21:53:46.314209+0000","last_clean":"2023-12-26T21:53:46.314209+0000","last_became_active":"2023-12-26T21:53:46.308203+0000","last_became_peered":"2023-12-26T21:53:46.308203+0000","last_unstale":"2023-12-26T21:53:46.314209+0000","last_undegraded":"2023-12-26T21:53:46.314209+0000","last_fullsized":"2023-12-26T21:53:46.314209+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311951+0000","last_change":"2023-12-26T21:53:46.305482+0000","last_active":"2023-12-26T21:53:46.311951+0000","last_peered":"2023-12-26T21:53:46.311951+0000","last_clean":"2023-12-26T21:53:46.311951+0000","last_became_active":"2023-12-26T21:53:46.305210+0000","last_became_peered":"2023-12-26T21:53:46.305210+0000","last_unstale":"2023-12-26T21:53:46.311951+0000","last_undegraded":"2023-12-26T21:53:46.311951+0000","last_fullsized":"2023-12-26T21:53:46.311951+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306262+0000","last_change":"2023-12-26T21:53:46.301077+0000","last_active":"2023-12-26T21:53:46.306262+0000","last_peered":"2023-12-26T21:53:46.306262+0000","last_clean":"2023-12-26T21:53:46.306262+0000","last_became_active":"2023-12-26T21:53:46.300762+0000","last_became_peered":"2023-12-26T21:53:46.300762+0000","last_unstale":"2023-12-26T21:53:46.306262+0000","last_undegraded":"2023-12-26T21:53:46.306262+0000","last_fullsized":"2023-12-26T21:53:46.306262+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289642+0000","last_change":"2023-12-26T21:53:46.287813+0000","last_active":"2023-12-26T21:53:46.289642+0000","last_peered":"2023-12-26T21:53:46.289642+0000","last_clean":"2023-12-26T21:53:46.289642+0000","last_became_active":"2023-12-26T21:53:46.287544+0000","last_became_peered":"2023-12-26T21:53:46.287544+0000","last_unstale":"2023-12-26T21:53:46.289642+0000","last_undegraded":"2023-12-26T21:53:46.289642+0000","last_fullsized":"2023-12-26T21:53:46.289642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:27.394417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.297090+0000","last_change":"2023-12-26T21:53:46.287979+0000","last_active":"2023-12-26T21:53:46.297090+0000","last_peered":"2023-12-26T21:53:46.297090+0000","last_clean":"2023-12-26T21:53:46.297090+0000","last_became_active":"2023-12-26T21:53:46.287682+0000","last_became_peered":"2023-12-26T21:53:46.287682+0000","last_unstale":"2023-12-26T21:53:46.297090+0000","last_undegraded":"2023-12-26T21:53:46.297090+0000","last_fullsized":"2023-12-26T21:53:46.297090+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:38:29.898590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311484+0000","last_change":"2023-12-26T21:53:46.306111+0000","last_active":"2023-12-26T21:53:46.311484+0000","last_peered":"2023-12-26T21:53:46.311484+0000","last_clean":"2023-12-26T21:53:46.311484+0000","last_became_active":"2023-12-26T21:53:46.305852+0000","last_became_peered":"2023-12-26T21:53:46.305852+0000","last_unstale":"2023-12-26T21:53:46.311484+0000","last_undegraded":"2023-12-26T21:53:46.311484+0000","last_fullsized":"2023-12-26T21:53:46.311484+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:03:27.957388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311050+0000","last_change":"2023-12-26T21:53:46.305148+0000","last_active":"2023-12-26T21:53:46.311050+0000","last_peered":"2023-12-26T21:53:46.311050+0000","last_clean":"2023-12-26T21:53:46.311050+0000","last_became_active":"2023-12-26T21:53:46.304893+0000","last_became_peered":"2023-12-26T21:53:46.304893+0000","last_unstale":"2023-12-26T21:53:46.311050+0000","last_undegraded":"2023-12-26T21:53:46.311050+0000","last_fullsized":"2023-12-26T21:53:46.311050+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:05.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307642+0000","last_change":"2023-12-26T21:53:46.301677+0000","last_active":"2023-12-26T21:53:46.307642+0000","last_peered":"2023-12-26T21:53:46.307642+0000","last_clean":"2023-12-26T21:53:46.307642+0000","last_became_active":"2023-12-26T21:53:46.301432+0000","last_became_peered":"2023-12-26T21:53:46.301432+0000","last_unstale":"2023-12-26T21:53:46.307642+0000","last_undegraded":"2023-12-26T21:53:46.307642+0000","last_fullsized":"2023-12-26T21:53:46.307642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"109'4","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321799+0000","last_change":"2023-12-26T21:53:45.285708+0000","last_active":"2023-12-26T21:53:46.321799+0000","last_peered":"2023-12-26T21:53:46.321799+0000","last_clean":"2023-12-26T21:53:46.321799+0000","last_became_active":"2023-12-26T21:53:45.285466+0000","last_became_peered":"2023-12-26T21:53:45.285466+0000","last_unstale":"2023-12-26T21:53:46.321799+0000","last_undegraded":"2023-12-26T21:53:46.321799+0000","last_fullsized":"2023-12-26T21:53:46.321799+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:11.158488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310328+0000","last_change":"2023-12-26T21:53:46.307223+0000","last_active":"2023-12-26T21:53:46.310328+0000","last_peered":"2023-12-26T21:53:46.310328+0000","last_clean":"2023-12-26T21:53:46.310328+0000","last_became_active":"2023-12-26T21:53:46.306963+0000","last_became_peered":"2023-12-26T21:53:46.306963+0000","last_unstale":"2023-12-26T21:53:46.310328+0000","last_undegraded":"2023-12-26T21:53:46.310328+0000","last_fullsized":"2023-12-26T21:53:46.310328+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:12:37.634766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.264186+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:17.475634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262584+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:17.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.305631+0000","last_change":"2023-12-26T21:53:46.299487+0000","last_active":"2023-12-26T21:53:46.305631+0000","last_peered":"2023-12-26T21:53:46.305631+0000","last_clean":"2023-12-26T21:53:46.305631+0000","last_became_active":"2023-12-26T21:53:46.299063+0000","last_became_peered":"2023-12-26T21:53:46.299063+0000","last_unstale":"2023-12-26T21:53:46.305631+0000","last_undegraded":"2023-12-26T21:53:46.305631+0000","last_fullsized":"2023-12-26T21:53:46.305631+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:48.449758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262237+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:56:53.977018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258326+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:40:24.430368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.263523+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:08:23.502904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299738+0000","last_change":"2023-12-26T21:53:46.297785+0000","last_active":"2023-12-26T21:53:46.299738+0000","last_peered":"2023-12-26T21:53:46.299738+0000","last_clean":"2023-12-26T21:53:46.299738+0000","last_became_active":"2023-12-26T21:53:46.297505+0000","last_became_peered":"2023-12-26T21:53:46.297505+0000","last_unstale":"2023-12-26T21:53:46.299738+0000","last_undegraded":"2023-12-26T21:53:46.299738+0000","last_fullsized":"2023-12-26T21:53:46.299738+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:33.304213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281100+0000","last_change":"2023-12-26T21:53:45.279536+0000","last_active":"2023-12-26T21:53:46.281100+0000","last_peered":"2023-12-26T21:53:46.281100+0000","last_clean":"2023-12-26T21:53:46.281100+0000","last_became_active":"2023-12-26T21:53:45.279282+0000","last_became_peered":"2023-12-26T21:53:45.279282+0000","last_unstale":"2023-12-26T21:53:46.281100+0000","last_undegraded":"2023-12-26T21:53:46.281100+0000","last_fullsized":"2023-12-26T21:53:46.281100+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:58.794519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.288617+0000","last_change":"2023-12-26T21:53:46.285357+0000","last_active":"2023-12-26T21:53:46.288617+0000","last_peered":"2023-12-26T21:53:46.288617+0000","last_clean":"2023-12-26T21:53:46.288617+0000","last_became_active":"2023-12-26T21:53:46.285051+0000","last_became_peered":"2023-12-26T21:53:46.285051+0000","last_unstale":"2023-12-26T21:53:46.288617+0000","last_undegraded":"2023-12-26T21:53:46.288617+0000","last_fullsized":"2023-12-26T21:53:46.288617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:31:16.372604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281226+0000","last_change":"2023-12-26T21:53:45.268538+0000","last_active":"2023-12-26T21:53:46.281226+0000","last_peered":"2023-12-26T21:53:46.281226+0000","last_clean":"2023-12-26T21:53:46.281226+0000","last_became_active":"2023-12-26T21:53:45.268069+0000","last_became_peered":"2023-12-26T21:53:45.268069+0000","last_unstale":"2023-12-26T21:53:46.281226+0000","last_undegraded":"2023-12-26T21:53:46.281226+0000","last_fullsized":"2023-12-26T21:53:46.281226+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:25.546517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300878+0000","last_change":"2023-12-26T21:53:46.297819+0000","last_active":"2023-12-26T21:53:46.300878+0000","last_peered":"2023-12-26T21:53:46.300878+0000","last_clean":"2023-12-26T21:53:46.300878+0000","last_became_active":"2023-12-26T21:53:46.297541+0000","last_became_peered":"2023-12-26T21:53:46.297541+0000","last_unstale":"2023-12-26T21:53:46.300878+0000","last_undegraded":"2023-12-26T21:53:46.300878+0000","last_fullsized":"2023-12-26T21:53:46.300878+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:10:09.104721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289148+0000","last_change":"2023-12-26T21:53:45.281269+0000","last_active":"2023-12-26T21:53:46.289148+0000","last_peered":"2023-12-26T21:53:46.289148+0000","last_clean":"2023-12-26T21:53:46.289148+0000","last_became_active":"2023-12-26T21:53:45.280967+0000","last_became_peered":"2023-12-26T21:53:45.280967+0000","last_unstale":"2023-12-26T21:53:46.289148+0000","last_undegraded":"2023-12-26T21:53:46.289148+0000","last_fullsized":"2023-12-26T21:53:46.289148+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:20:00.172583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299929+0000","last_change":"2023-12-26T21:53:46.298319+0000","last_active":"2023-12-26T21:53:46.299929+0000","last_peered":"2023-12-26T21:53:46.299929+0000","last_clean":"2023-12-26T21:53:46.299929+0000","last_became_active":"2023-12-26T21:53:46.298050+0000","last_became_peered":"2023-12-26T21:53:46.298050+0000","last_unstale":"2023-12-26T21:53:46.299929+0000","last_undegraded":"2023-12-26T21:53:46.299929+0000","last_fullsized":"2023-12-26T21:53:46.299929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:34.648293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262776+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:43:16.270159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"109'3","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321640+0000","last_change":"2023-12-26T21:53:45.285950+0000","last_active":"2023-12-26T21:53:46.321640+0000","last_peered":"2023-12-26T21:53:46.321640+0000","last_clean":"2023-12-26T21:53:46.321640+0000","last_became_active":"2023-12-26T21:53:45.285657+0000","last_became_peered":"2023-12-26T21:53:45.285657+0000","last_unstale":"2023-12-26T21:53:46.321640+0000","last_undegraded":"2023-12-26T21:53:46.321640+0000","last_fullsized":"2023-12-26T21:53:46.321640+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:55.530687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303089+0000","last_change":"2023-12-26T21:53:46.301426+0000","last_active":"2023-12-26T21:53:46.303089+0000","last_peered":"2023-12-26T21:53:46.303089+0000","last_clean":"2023-12-26T21:53:46.303089+0000","last_became_active":"2023-12-26T21:53:46.301116+0000","last_became_peered":"2023-12-26T21:53:46.301116+0000","last_unstale":"2023-12-26T21:53:46.303089+0000","last_undegraded":"2023-12-26T21:53:46.303089+0000","last_fullsized":"2023-12-26T21:53:46.303089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:49.402467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258535+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:44:24.266594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262178+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:30:14.941645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304318+0000","last_change":"2023-12-26T21:53:46.298424+0000","last_active":"2023-12-26T21:53:46.304318+0000","last_peered":"2023-12-26T21:53:46.304318+0000","last_clean":"2023-12-26T21:53:46.304318+0000","last_became_active":"2023-12-26T21:53:46.298145+0000","last_became_peered":"2023-12-26T21:53:46.298145+0000","last_unstale":"2023-12-26T21:53:46.304318+0000","last_undegraded":"2023-12-26T21:53:46.304318+0000","last_fullsized":"2023-12-26T21:53:46.304318+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:24:04.826615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285162+0000","last_change":"2023-12-26T21:53:45.281444+0000","last_active":"2023-12-26T21:53:46.285162+0000","last_peered":"2023-12-26T21:53:46.285162+0000","last_clean":"2023-12-26T21:53:46.285162+0000","last_became_active":"2023-12-26T21:53:45.280420+0000","last_became_peered":"2023-12-26T21:53:45.280420+0000","last_unstale":"2023-12-26T21:53:46.285162+0000","last_undegraded":"2023-12-26T21:53:46.285162+0000","last_fullsized":"2023-12-26T21:53:46.285162+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:17.475634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300883+0000","last_change":"2023-12-26T21:53:46.297808+0000","last_active":"2023-12-26T21:53:46.300883+0000","last_peered":"2023-12-26T21:53:46.300883+0000","last_clean":"2023-12-26T21:53:46.300883+0000","last_became_active":"2023-12-26T21:53:46.296873+0000","last_became_peered":"2023-12-26T21:53:46.296873+0000","last_unstale":"2023-12-26T21:53:46.300883+0000","last_undegraded":"2023-12-26T21:53:46.300883+0000","last_fullsized":"2023-12-26T21:53:46.300883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:29.573501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262772+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:25:10.062797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262093+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:12:36.629000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.320094+0000","last_change":"2023-12-26T21:53:45.280067+0000","last_active":"2023-12-26T21:53:46.320094+0000","last_peered":"2023-12-26T21:53:46.320094+0000","last_clean":"2023-12-26T21:53:46.320094+0000","last_became_active":"2023-12-26T21:53:45.279713+0000","last_became_peered":"2023-12-26T21:53:45.279713+0000","last_unstale":"2023-12-26T21:53:46.320094+0000","last_undegraded":"2023-12-26T21:53:46.320094+0000","last_fullsized":"2023-12-26T21:53:46.320094+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:15.927146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.316477+0000","last_change":"2023-12-26T21:53:45.277896+0000","last_active":"2023-12-26T21:53:46.316477+0000","last_peered":"2023-12-26T21:53:46.316477+0000","last_clean":"2023-12-26T21:53:46.316477+0000","last_became_active":"2023-12-26T21:53:45.277584+0000","last_became_peered":"2023-12-26T21:53:45.277584+0000","last_unstale":"2023-12-26T21:53:46.316477+0000","last_undegraded":"2023-12-26T21:53:46.316477+0000","last_fullsized":"2023-12-26T21:53:46.316477+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:44:17.169190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.282526+0000","last_change":"2023-12-26T21:53:45.280486+0000","last_active":"2023-12-26T21:53:46.282526+0000","last_peered":"2023-12-26T21:53:46.282526+0000","last_clean":"2023-12-26T21:53:46.282526+0000","last_became_active":"2023-12-26T21:53:45.280178+0000","last_became_peered":"2023-12-26T21:53:45.280178+0000","last_unstale":"2023-12-26T21:53:46.282526+0000","last_undegraded":"2023-12-26T21:53:46.282526+0000","last_fullsized":"2023-12-26T21:53:46.282526+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:13:56.557753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304197+0000","last_change":"2023-12-26T21:53:46.297254+0000","last_active":"2023-12-26T21:53:46.304197+0000","last_peered":"2023-12-26T21:53:46.304197+0000","last_clean":"2023-12-26T21:53:46.304197+0000","last_became_active":"2023-12-26T21:53:46.296358+0000","last_became_peered":"2023-12-26T21:53:46.296358+0000","last_unstale":"2023-12-26T21:53:46.304197+0000","last_undegraded":"2023-12-26T21:53:46.304197+0000","last_fullsized":"2023-12-26T21:53:46.304197+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.317134+0000","last_change":"2023-12-26T21:53:45.271721+0000","last_active":"2023-12-26T21:53:46.317134+0000","last_peered":"2023-12-26T21:53:46.317134+0000","last_clean":"2023-12-26T21:53:46.317134+0000","last_became_active":"2023-12-26T21:53:45.271453+0000","last_became_peered":"2023-12-26T21:53:45.271453+0000","last_unstale":"2023-12-26T21:53:46.317134+0000","last_undegraded":"2023-12-26T21:53:46.317134+0000","last_fullsized":"2023-12-26T21:53:46.317134+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:49:24.818172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303057+0000","last_change":"2023-12-26T21:53:46.301435+0000","last_active":"2023-12-26T21:53:46.303057+0000","last_peered":"2023-12-26T21:53:46.303057+0000","last_clean":"2023-12-26T21:53:46.303057+0000","last_became_active":"2023-12-26T21:53:46.301117+0000","last_became_peered":"2023-12-26T21:53:46.301117+0000","last_unstale":"2023-12-26T21:53:46.303057+0000","last_undegraded":"2023-12-26T21:53:46.303057+0000","last_fullsized":"2023-12-26T21:53:46.303057+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:39.625634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285208+0000","last_change":"2023-12-26T21:53:45.279979+0000","last_active":"2023-12-26T21:53:46.285208+0000","last_peered":"2023-12-26T21:53:46.285208+0000","last_clean":"2023-12-26T21:53:46.285208+0000","last_became_active":"2023-12-26T21:53:45.279563+0000","last_became_peered":"2023-12-26T21:53:45.279563+0000","last_unstale":"2023-12-26T21:53:46.285208+0000","last_undegraded":"2023-12-26T21:53:46.285208+0000","last_fullsized":"2023-12-26T21:53:46.285208+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:00:28.640861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262063+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:32.298447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281331+0000","last_change":"2023-12-26T21:53:45.282408+0000","last_active":"2023-12-26T21:53:46.281331+0000","last_peered":"2023-12-26T21:53:46.281331+0000","last_clean":"2023-12-26T21:53:46.281331+0000","last_became_active":"2023-12-26T21:53:45.282074+0000","last_became_peered":"2023-12-26T21:53:45.282074+0000","last_unstale":"2023-12-26T21:53:46.281331+0000","last_undegraded":"2023-12-26T21:53:46.281331+0000","last_fullsized":"2023-12-26T21:53:46.281331+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:18:52.576472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.302717+0000","last_change":"2023-12-26T21:53:46.300574+0000","last_active":"2023-12-26T21:53:46.302717+0000","last_peered":"2023-12-26T21:53:46.302717+0000","last_clean":"2023-12-26T21:53:46.302717+0000","last_became_active":"2023-12-26T21:53:46.300296+0000","last_became_peered":"2023-12-26T21:53:46.300296+0000","last_unstale":"2023-12-26T21:53:46.302717+0000","last_undegraded":"2023-12-26T21:53:46.302717+0000","last_fullsized":"2023-12-26T21:53:46.302717+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:51.026196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.318605+0000","last_change":"2023-12-26T21:53:45.280489+0000","last_active":"2023-12-26T21:53:46.318605+0000","last_peered":"2023-12-26T21:53:46.318605+0000","last_clean":"2023-12-26T21:53:46.318605+0000","last_became_active":"2023-12-26T21:53:45.279071+0000","last_became_peered":"2023-12-26T21:53:45.279071+0000","last_unstale":"2023-12-26T21:53:46.318605+0000","last_undegraded":"2023-12-26T21:53:46.318605+0000","last_fullsized":"2023-12-26T21:53:46.318605+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:28.851567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308510+0000","last_change":"2023-12-26T21:53:46.304923+0000","last_active":"2023-12-26T21:53:46.308510+0000","last_peered":"2023-12-26T21:53:46.308510+0000","last_clean":"2023-12-26T21:53:46.308510+0000","last_became_active":"2023-12-26T21:53:46.304621+0000","last_became_peered":"2023-12-26T21:53:46.304621+0000","last_unstale":"2023-12-26T21:53:46.308510+0000","last_undegraded":"2023-12-26T21:53:46.308510+0000","last_fullsized":"2023-12-26T21:53:46.308510+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:15:24.617194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262644+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:16.240935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299826+0000","last_change":"2023-12-26T21:53:46.288775+0000","last_active":"2023-12-26T21:53:46.299826+0000","last_peered":"2023-12-26T21:53:46.299826+0000","last_clean":"2023-12-26T21:53:46.299826+0000","last_became_active":"2023-12-26T21:53:46.288489+0000","last_became_peered":"2023-12-26T21:53:46.288489+0000","last_unstale":"2023-12-26T21:53:46.299826+0000","last_undegraded":"2023-12-26T21:53:46.299826+0000","last_fullsized":"2023-12-26T21:53:46.299826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:48:03.925563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258576+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:59:57.204768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262786+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:00:28.640861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310423+0000","last_change":"2023-12-26T21:53:46.306701+0000","last_active":"2023-12-26T21:53:46.310423+0000","last_peered":"2023-12-26T21:53:46.310423+0000","last_clean":"2023-12-26T21:53:46.310423+0000","last_became_active":"2023-12-26T21:53:46.306450+0000","last_became_peered":"2023-12-26T21:53:46.306450+0000","last_unstale":"2023-12-26T21:53:46.310423+0000","last_undegraded":"2023-12-26T21:53:46.310423+0000","last_fullsized":"2023-12-26T21:53:46.310423+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:56.921774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.284986+0000","last_change":"2023-12-26T21:53:45.279374+0000","last_active":"2023-12-26T21:53:46.284986+0000","last_peered":"2023-12-26T21:53:46.284986+0000","last_clean":"2023-12-26T21:53:46.284986+0000","last_became_active":"2023-12-26T21:53:45.278479+0000","last_became_peered":"2023-12-26T21:53:45.278479+0000","last_unstale":"2023-12-26T21:53:46.284986+0000","last_undegraded":"2023-12-26T21:53:46.284986+0000","last_fullsized":"2023-12-26T21:53:46.284986+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:29.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262986+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:29.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308046+0000","last_change":"2023-12-26T21:53:46.306183+0000","last_active":"2023-12-26T21:53:46.308046+0000","last_peered":"2023-12-26T21:53:46.308046+0000","last_clean":"2023-12-26T21:53:46.308046+0000","last_became_active":"2023-12-26T21:53:46.305918+0000","last_became_peered":"2023-12-26T21:53:46.305918+0000","last_unstale":"2023-12-26T21:53:46.308046+0000","last_undegraded":"2023-12-26T21:53:46.308046+0000","last_fullsized":"2023-12-26T21:53:46.308046+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:49:38.482545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286935+0000","last_change":"2023-12-26T21:53:45.284416+0000","last_active":"2023-12-26T21:53:46.286935+0000","last_peered":"2023-12-26T21:53:46.286935+0000","last_clean":"2023-12-26T21:53:46.286935+0000","last_became_active":"2023-12-26T21:53:45.284072+0000","last_became_peered":"2023-12-26T21:53:45.284072+0000","last_unstale":"2023-12-26T21:53:46.286935+0000","last_undegraded":"2023-12-26T21:53:46.286935+0000","last_fullsized":"2023-12-26T21:53:46.286935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:46:59.726957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307933+0000","last_change":"2023-12-26T21:53:46.303791+0000","last_active":"2023-12-26T21:53:46.307933+0000","last_peered":"2023-12-26T21:53:46.307933+0000","last_clean":"2023-12-26T21:53:46.307933+0000","last_became_active":"2023-12-26T21:53:46.303518+0000","last_became_peered":"2023-12-26T21:53:46.303518+0000","last_unstale":"2023-12-26T21:53:46.307933+0000","last_undegraded":"2023-12-26T21:53:46.307933+0000","last_fullsized":"2023-12-26T21:53:46.307933+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:17.246701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258326+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:17:49.107900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.319541+0000","last_change":"2023-12-26T21:53:45.277923+0000","last_active":"2023-12-26T21:53:46.319541+0000","last_peered":"2023-12-26T21:53:46.319541+0000","last_clean":"2023-12-26T21:53:46.319541+0000","last_became_active":"2023-12-26T21:53:45.277681+0000","last_became_peered":"2023-12-26T21:53:45.277681+0000","last_unstale":"2023-12-26T21:53:46.319541+0000","last_undegraded":"2023-12-26T21:53:46.319541+0000","last_fullsized":"2023-12-26T21:53:46.319541+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:40.310455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'205","reported_seq":555,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286580+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:46.286580+0000","last_peered":"2023-12-26T21:53:46.286580+0000","last_clean":"2023-12-26T21:53:46.286580+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:46.286580+0000","last_undegraded":"2023-12-26T21:53:46.286580+0000","last_fullsized":"2023-12-26T21:53:46.286580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":4},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542348,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302700,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069140,"statfs":{"total":96636764160,"available":96326799360,"internally_reserved":0,"allocated":1347584,"data_stored":653499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.47199999999999998,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.72499999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.69799999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74399999999999999,"5min":1.016,"15min":1.048},"last":0.74199999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.038,"15min":1.0509999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.85699999999999998,"5min":0.99199999999999999,"15min":1.02},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.62,"5min":0.504,"15min":0.5},"max":{"1min":0.94299999999999995,"5min":1.0189999999999999,"15min":1.052},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83499999999999996,"5min":1.036,"15min":1.036},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.747,"15min":0.746},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0129999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86399999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.117,"15min":1.117},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.437},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.006},"last":0.88}]}]},{"osd":6,"up_from":15,"seq":64424509643,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306744,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065096,"statfs":{"total":96636764160,"available":96322658304,"internally_reserved":0,"allocated":1294336,"data_stored":626742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.085},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69199999999999995,"15min":0.71799999999999997},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.34799999999999998},"max":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67800000000000005,"15min":0.70199999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.95299999999999996,"15min":0.98799999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.54200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.97599999999999998,"15min":1.054},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.93200000000000005,"15min":1.105},"last":0.753},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.79200000000000004,"5min":0.92000000000000004,"15min":1.048},"last":0.83999999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.90200000000000002,"5min":1.026,"15min":1.1799999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86899999999999999,"5min":0.96999999999999997,"15min":1.069},"last":0.85599999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.91100000000000003,"15min":1.137},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93100000000000005,"5min":1.0049999999999999,"15min":1.296},"last":0.78200000000000003}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.437},"max":{"1min":1.008,"5min":1.008,"15min":1.3380000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.56799999999999995,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.95599999999999996,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.87}]}]},{"osd":1,"up_from":15,"seq":64424509644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311472,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060368,"statfs":{"total":96636764160,"available":96317816832,"internally_reserved":0,"allocated":1941504,"data_stored":1243887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.436},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.55700000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.84699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.70399999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.87}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.88400000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.90500000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509643,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302660,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069180,"statfs":{"total":96636764160,"available":96326840320,"internally_reserved":0,"allocated":1306624,"data_stored":632939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":1.056},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47799999999999998,"15min":0.49399999999999999},"min":{"1min":0.315,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":1.006},"last":0.311},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.84699999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80000000000000004,"5min":0.86699999999999999,"15min":1.1870000000000001},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.752},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.042,"5min":1.042,"15min":1.411},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.76800000000000002},"min":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.2070000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.754,"15min":0.79800000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.2569999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77100000000000002,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.452},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.214},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.012,"15min":1.3779999999999999},"last":0.50600000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46400000000000002},"max":{"1min":1.018,"5min":1.042,"15min":1.4359999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.79200000000000004},"min":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":1.0029999999999999,"5min":1.0069999999999999,"15min":1.3240000000000001},"last":0.49299999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306744,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065096,"statfs":{"total":96636764160,"available":96322658304,"internally_reserved":0,"allocated":1294336,"data_stored":626742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.877},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68600000000000005,"5min":0.83899999999999997,"15min":0.84399999999999997},"last":0.433}]},{"osd":1,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.65300000000000002,"5min":0.70799999999999996,"15min":0.92000000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.31,"15min":0.309},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.78600000000000003},"last":0.38900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":1,"15min":1},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.82299999999999995,"15min":0.85699999999999998},"last":0.34599999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.46800000000000003,"15min":0.34799999999999998},"max":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":1.284},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.76300000000000001},"min":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.95699999999999996,"5min":0.98399999999999999,"15min":1.1759999999999999},"last":0.83699999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.024,"5min":1.0600000000000001,"15min":1.2170000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":0.95499999999999996,"5min":1.014,"15min":1.1319999999999999},"last":0.86499999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.038,"15min":1.242},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.40300000000000002},"max":{"1min":1.018,"5min":1.0609999999999999,"15min":1.272},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":1.073,"15min":1.1220000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.50800000000000001},"max":{"1min":0.999,"5min":1.0189999999999999,"15min":1.1990000000000001},"last":0.84899999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509645,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310900,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060940,"statfs":{"total":96636764160,"available":96318402560,"internally_reserved":0,"allocated":1355776,"data_stored":654075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.378}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82499999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.753}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.81299999999999994}]}]},{"osd":4,"up_from":14,"seq":60129542348,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306796,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065044,"statfs":{"total":96636764160,"available":96322605056,"internally_reserved":0,"allocated":1347584,"data_stored":653463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97899999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.48499999999999999,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":1.008},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.95199999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88900000000000001,"5min":0.97199999999999998,"15min":0.97299999999999998},"last":0.624}]},{"osd":3,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.71499999999999997},"min":{"1min":0.504,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.78100000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.438,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.84399999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509642,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":1223307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.69999999999999996},"min":{"1min":0.44500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84199999999999997,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.749},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.4570000000000001},"last":0.73399999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68700000000000006,"15min":0.71499999999999997},"min":{"1min":0.499,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.625},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67800000000000005,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.4219999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.72799999999999998},"min":{"1min":0.54900000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.88500000000000001,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68600000000000005,"15min":0.71099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.92400000000000004,"15min":1.0569999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":1.052,"15min":1.4890000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.73599999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47799999999999998},"max":{"1min":0.88300000000000001,"5min":0.92200000000000004,"15min":1.081},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":2.5070000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.54500000000000004,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":2.8039999999999998},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.65700000000000003,"15min":0.66400000000000003},"min":{"1min":0.505,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":1.637},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.623,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":1.111,"5min":1.111,"15min":2.5579999999999998},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.61399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.97499999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.71299999999999997,"15min":0.70999999999999996},"min":{"1min":0.52200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.5980000000000001},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:51.102 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:51.246 INFO:tasks.ceph.mds.c.smithi131.stderr:2023-12-26T21:53:51.246+0000 7f8d6d1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:53:51.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.326+0000 7f9bc37ff700 1 Processor -- start 2023-12-26T21:53:51.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.327+0000 7f9bc37ff700 1 -- start start 2023-12-26T21:53:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bc37ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bc37ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9bbc0b0870 0x7f9bbc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bc37ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc0afcd0 con 0x7f9bbc159a70 2023-12-26T21:53:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bc37ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc0b3e70 con 0x7f9bbc0b0870 2023-12-26T21:53:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bc37ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0b0cd0 con 0x7f9bbc1596a0 2023-12-26T21:53:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bc0d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9bbc159a70 0x7f9bbc0b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45108/0 (socket says 172.21.15.131:45108) 2023-12-26T21:53:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bc0d9a700 1 -- 172.21.15.131:0/361221466 learned_addr learned my addr 172.21.15.131:0/361221466 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bc159b700 1 --2- 172.21.15.131:0/361221466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bc1d9c700 1 --2- 172.21.15.131:0/361221466 >> v2:172.21.15.190:3300/0 conn(0x7f9bbc0b0870 0x7f9bbc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.328+0000 7f9bb3fff700 1 -- 172.21.15.131:0/361221466 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004699495 0 0) 0x7f9bbc0afcd0 con 0x7f9bbc159a70 2023-12-26T21:53:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.329+0000 7f9bb3fff700 1 -- 172.21.15.131:0/361221466 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba8003570 con 0x7f9bbc159a70 2023-12-26T21:53:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.329+0000 7f9bc159b700 1 -- 172.21.15.131:0/361221466 >> v2:172.21.15.190:3300/0 conn(0x7f9bbc0b0870 msgr2=0x7f9bbc0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.329+0000 7f9bc159b700 1 --2- 172.21.15.131:0/361221466 >> v2:172.21.15.190:3300/0 conn(0x7f9bbc0b0870 0x7f9bbc0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.329+0000 7f9bc159b700 1 -- 172.21.15.131:0/361221466 >> v1:172.21.15.131:6789/0 conn(0x7f9bbc159a70 legacy=0x7f9bbc0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.329+0000 7f9bc159b700 1 -- 172.21.15.131:0/361221466 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0b2e00 con 0x7f9bbc1596a0 2023-12-26T21:53:51.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.329+0000 7f9bc159b700 1 --2- 172.21.15.131:0/361221466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0b2380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80098e0 tx=0x7f9bb80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b51ee6bd2b29a15 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:51.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.329+0000 7f9bb3fff700 1 -- 172.21.15.131:0/361221466 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8011630 con 0x7f9bbc1596a0 2023-12-26T21:53:51.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.329+0000 7f9bb3fff700 1 -- 172.21.15.131:0/361221466 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb8011790 con 0x7f9bbc1596a0 2023-12-26T21:53:51.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.329+0000 7f9bb3fff700 1 -- 172.21.15.131:0/361221466 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8011b10 con 0x7f9bbc1596a0 2023-12-26T21:53:51.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.330+0000 7f9bc37ff700 1 -- 172.21.15.131:0/361221466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 msgr2=0x7f9bbc0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.330+0000 7f9bc37ff700 1 --2- 172.21.15.131:0/361221466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0b2380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80098e0 tx=0x7f9bb80091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.330+0000 7f9bc37ff700 1 -- 172.21.15.131:0/361221466 shutdown_connections 2023-12-26T21:53:51.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.330+0000 7f9bc37ff700 1 --2- 172.21.15.131:0/361221466 >> v2:172.21.15.190:3300/0 conn(0x7f9bbc0b0870 0x7f9bbc0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.330+0000 7f9bc37ff700 1 --2- 172.21.15.131:0/361221466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0b2380 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.330+0000 7f9bc37ff700 1 -- 172.21.15.131:0/361221466 >> 172.21.15.131:0/361221466 conn(0x7f9bbc06ece0 msgr2=0x7f9bbc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:51.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.330+0000 7f9bc37ff700 1 -- 172.21.15.131:0/361221466 shutdown_connections 2023-12-26T21:53:51.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.330+0000 7f9bc37ff700 1 -- 172.21.15.131:0/361221466 wait complete. 2023-12-26T21:53:51.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.331+0000 7f9bc37ff700 1 Processor -- start 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.331+0000 7f9bc37ff700 1 -- start start 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.332+0000 7f9bc37ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bbc0b0870 0x7f9bbc0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.332+0000 7f9bc37ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.332+0000 7f9bc37ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bbc159a70 0x7f9bbc0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.332+0000 7f9bc37ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc159100 con 0x7f9bbc0b0870 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.332+0000 7f9bc37ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0cd2c0 con 0x7f9bbc159a70 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.332+0000 7f9bc37ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1584f0 con 0x7f9bbc1596a0 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc159b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bbc0b0870 0x7f9bbc0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc159b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bbc0b0870 0x7f9bbc0cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58822/0 (socket says 172.21.15.131:58822) 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc159b700 1 -- 172.21.15.131:0/1432501104 learned_addr learned my addr 172.21.15.131:0/1432501104 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc1d9c700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bbc159a70 0x7f9bbc0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc0d9a700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc159b700 1 -- 172.21.15.131:0/1432501104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 msgr2=0x7f9bbc0c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc159b700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc159b700 1 -- 172.21.15.131:0/1432501104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bbc159a70 msgr2=0x7f9bbc0c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc159b700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bbc159a70 0x7f9bbc0c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc159b700 1 -- 172.21.15.131:0/1432501104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8009420 con 0x7f9bbc0b0870 2023-12-26T21:53:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc0d9a700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.333+0000 7f9bc159b700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bbc0b0870 0x7f9bbc0cc420 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc15e440 tx=0x7f9bb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:51.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.334+0000 7f9bb17fa700 1 -- 172.21.15.131:0/1432501104 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8011900 con 0x7f9bbc0b0870 2023-12-26T21:53:51.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.334+0000 7f9bb17fa700 1 -- 172.21.15.131:0/1432501104 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb8011a80 con 0x7f9bbc0b0870 2023-12-26T21:53:51.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.334+0000 7f9bb17fa700 1 -- 172.21.15.131:0/1432501104 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb801a680 con 0x7f9bbc0b0870 2023-12-26T21:53:51.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.334+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc1587d0 con 0x7f9bbc0b0870 2023-12-26T21:53:51.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.334+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc158d90 con 0x7f9bbc0b0870 2023-12-26T21:53:51.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.335+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc0657e0 con 0x7f9bbc0b0870 2023-12-26T21:53:51.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.335+0000 7f9bb17fa700 1 -- 172.21.15.131:0/1432501104 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9bb801aa70 con 0x7f9bbc0b0870 2023-12-26T21:53:51.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.336+0000 7f9bb17fa700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9ba0066f30 0x7f9ba00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.336+0000 7f9bb17fa700 1 -- 172.21.15.131:0/1432501104 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f9bb8086290 con 0x7f9bbc0b0870 2023-12-26T21:53:51.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.337+0000 7f9bc0d9a700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9ba0066f30 0x7f9ba00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.337+0000 7f9bc0d9a700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9ba0066f30 0x7f9ba00693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0cef70 tx=0x7f9bac009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:51.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.341+0000 7f9bb17fa700 1 -- 172.21.15.131:0/1432501104 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bb8053370 con 0x7f9bbc0b0870 2023-12-26T21:53:51.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.530+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bbc15a320 con 0x7f9ba0066f30 2023-12-26T21:53:51.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.538+0000 7f9bb17fa700 1 -- 172.21.15.131:0/1432501104 <== mgr.6975 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283598 (secure 0 0 0) 0x7f9bbc15a320 con 0x7f9ba0066f30 2023-12-26T21:53:51.543 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.542+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9ba0066f30 msgr2=0x7f9ba00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.542+0000 7f9bc37ff700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9ba0066f30 0x7f9ba00693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0cef70 tx=0x7f9bac009040 comp rx=0 tx=0).stop 2023-12-26T21:53:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.542+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bbc0b0870 msgr2=0x7f9bbc0cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.542+0000 7f9bc37ff700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bbc0b0870 0x7f9bbc0cc420 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc15e440 tx=0x7f9bb8017000 comp rx=0 tx=0).stop 2023-12-26T21:53:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.543+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 shutdown_connections 2023-12-26T21:53:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.543+0000 7f9bc37ff700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bbc159a70 0x7f9bbc0c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.543+0000 7f9bc37ff700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9ba0066f30 0x7f9ba00693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.543+0000 7f9bc37ff700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bbc0b0870 0x7f9bbc0cc420 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.543+0000 7f9bc37ff700 1 --2- 172.21.15.131:0/1432501104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bbc1596a0 0x7f9bbc0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.543+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 >> 172.21.15.131:0/1432501104 conn(0x7f9bbc06ece0 msgr2=0x7f9bbc0ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.543+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 shutdown_connections 2023-12-26T21:53:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.543+0000 7f9bc37ff700 1 -- 172.21.15.131:0/1432501104 wait complete. 2023-12-26T21:53:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:51.561 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:53:50.112262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591104,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":336,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2451256,"kb_used_data":11512,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752523464,"statfs":{"total":773094113280,"available":770584027136,"internally_reserved":0,"allocated":11788288,"data_stored":6314654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.856251"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310309+0000","last_change":"2023-12-26T21:53:46.307184+0000","last_active":"2023-12-26T21:53:46.310309+0000","last_peered":"2023-12-26T21:53:46.310309+0000","last_clean":"2023-12-26T21:53:46.310309+0000","last_became_active":"2023-12-26T21:53:46.306640+0000","last_became_peered":"2023-12-26T21:53:46.306640+0000","last_unstale":"2023-12-26T21:53:46.310309+0000","last_undegraded":"2023-12-26T21:53:46.310309+0000","last_fullsized":"2023-12-26T21:53:46.310309+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:18.030160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306053+0000","last_change":"2023-12-26T21:53:46.299743+0000","last_active":"2023-12-26T21:53:46.306053+0000","last_peered":"2023-12-26T21:53:46.306053+0000","last_clean":"2023-12-26T21:53:46.306053+0000","last_became_active":"2023-12-26T21:53:46.299485+0000","last_became_peered":"2023-12-26T21:53:46.299485+0000","last_unstale":"2023-12-26T21:53:46.306053+0000","last_undegraded":"2023-12-26T21:53:46.306053+0000","last_fullsized":"2023-12-26T21:53:46.306053+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299767+0000","last_change":"2023-12-26T21:53:46.289973+0000","last_active":"2023-12-26T21:53:46.299767+0000","last_peered":"2023-12-26T21:53:46.299767+0000","last_clean":"2023-12-26T21:53:46.299767+0000","last_became_active":"2023-12-26T21:53:46.289695+0000","last_became_peered":"2023-12-26T21:53:46.289695+0000","last_unstale":"2023-12-26T21:53:46.299767+0000","last_undegraded":"2023-12-26T21:53:46.299767+0000","last_fullsized":"2023-12-26T21:53:46.299767+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:43:34.451389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307583+0000","last_change":"2023-12-26T21:53:46.296559+0000","last_active":"2023-12-26T21:53:46.307583+0000","last_peered":"2023-12-26T21:53:46.307583+0000","last_clean":"2023-12-26T21:53:46.307583+0000","last_became_active":"2023-12-26T21:53:46.296237+0000","last_became_peered":"2023-12-26T21:53:46.296237+0000","last_unstale":"2023-12-26T21:53:46.307583+0000","last_undegraded":"2023-12-26T21:53:46.307583+0000","last_fullsized":"2023-12-26T21:53:46.307583+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308991+0000","last_change":"2023-12-26T21:53:46.303931+0000","last_active":"2023-12-26T21:53:46.308991+0000","last_peered":"2023-12-26T21:53:46.308991+0000","last_clean":"2023-12-26T21:53:46.308991+0000","last_became_active":"2023-12-26T21:53:46.303627+0000","last_became_peered":"2023-12-26T21:53:46.303627+0000","last_unstale":"2023-12-26T21:53:46.308991+0000","last_undegraded":"2023-12-26T21:53:46.308991+0000","last_fullsized":"2023-12-26T21:53:46.308991+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:28.427569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314549+0000","last_change":"2023-12-26T21:53:46.308209+0000","last_active":"2023-12-26T21:53:46.314549+0000","last_peered":"2023-12-26T21:53:46.314549+0000","last_clean":"2023-12-26T21:53:46.314549+0000","last_became_active":"2023-12-26T21:53:46.308025+0000","last_became_peered":"2023-12-26T21:53:46.308025+0000","last_unstale":"2023-12-26T21:53:46.314549+0000","last_undegraded":"2023-12-26T21:53:46.314549+0000","last_fullsized":"2023-12-26T21:53:46.314549+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:53:11.039774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311289+0000","last_change":"2023-12-26T21:53:46.303517+0000","last_active":"2023-12-26T21:53:46.311289+0000","last_peered":"2023-12-26T21:53:46.311289+0000","last_clean":"2023-12-26T21:53:46.311289+0000","last_became_active":"2023-12-26T21:53:46.303137+0000","last_became_peered":"2023-12-26T21:53:46.303137+0000","last_unstale":"2023-12-26T21:53:46.311289+0000","last_undegraded":"2023-12-26T21:53:46.311289+0000","last_fullsized":"2023-12-26T21:53:46.311289+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314209+0000","last_change":"2023-12-26T21:53:46.308456+0000","last_active":"2023-12-26T21:53:46.314209+0000","last_peered":"2023-12-26T21:53:46.314209+0000","last_clean":"2023-12-26T21:53:46.314209+0000","last_became_active":"2023-12-26T21:53:46.308203+0000","last_became_peered":"2023-12-26T21:53:46.308203+0000","last_unstale":"2023-12-26T21:53:46.314209+0000","last_undegraded":"2023-12-26T21:53:46.314209+0000","last_fullsized":"2023-12-26T21:53:46.314209+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311951+0000","last_change":"2023-12-26T21:53:46.305482+0000","last_active":"2023-12-26T21:53:46.311951+0000","last_peered":"2023-12-26T21:53:46.311951+0000","last_clean":"2023-12-26T21:53:46.311951+0000","last_became_active":"2023-12-26T21:53:46.305210+0000","last_became_peered":"2023-12-26T21:53:46.305210+0000","last_unstale":"2023-12-26T21:53:46.311951+0000","last_undegraded":"2023-12-26T21:53:46.311951+0000","last_fullsized":"2023-12-26T21:53:46.311951+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306262+0000","last_change":"2023-12-26T21:53:46.301077+0000","last_active":"2023-12-26T21:53:46.306262+0000","last_peered":"2023-12-26T21:53:46.306262+0000","last_clean":"2023-12-26T21:53:46.306262+0000","last_became_active":"2023-12-26T21:53:46.300762+0000","last_became_peered":"2023-12-26T21:53:46.300762+0000","last_unstale":"2023-12-26T21:53:46.306262+0000","last_undegraded":"2023-12-26T21:53:46.306262+0000","last_fullsized":"2023-12-26T21:53:46.306262+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289642+0000","last_change":"2023-12-26T21:53:46.287813+0000","last_active":"2023-12-26T21:53:46.289642+0000","last_peered":"2023-12-26T21:53:46.289642+0000","last_clean":"2023-12-26T21:53:46.289642+0000","last_became_active":"2023-12-26T21:53:46.287544+0000","last_became_peered":"2023-12-26T21:53:46.287544+0000","last_unstale":"2023-12-26T21:53:46.289642+0000","last_undegraded":"2023-12-26T21:53:46.289642+0000","last_fullsized":"2023-12-26T21:53:46.289642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:27.394417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.297090+0000","last_change":"2023-12-26T21:53:46.287979+0000","last_active":"2023-12-26T21:53:46.297090+0000","last_peered":"2023-12-26T21:53:46.297090+0000","last_clean":"2023-12-26T21:53:46.297090+0000","last_became_active":"2023-12-26T21:53:46.287682+0000","last_became_peered":"2023-12-26T21:53:46.287682+0000","last_unstale":"2023-12-26T21:53:46.297090+0000","last_undegraded":"2023-12-26T21:53:46.297090+0000","last_fullsized":"2023-12-26T21:53:46.297090+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:38:29.898590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311484+0000","last_change":"2023-12-26T21:53:46.306111+0000","last_active":"2023-12-26T21:53:46.311484+0000","last_peered":"2023-12-26T21:53:46.311484+0000","last_clean":"2023-12-26T21:53:46.311484+0000","last_became_active":"2023-12-26T21:53:46.305852+0000","last_became_peered":"2023-12-26T21:53:46.305852+0000","last_unstale":"2023-12-26T21:53:46.311484+0000","last_undegraded":"2023-12-26T21:53:46.311484+0000","last_fullsized":"2023-12-26T21:53:46.311484+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:03:27.957388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311050+0000","last_change":"2023-12-26T21:53:46.305148+0000","last_active":"2023-12-26T21:53:46.311050+0000","last_peered":"2023-12-26T21:53:46.311050+0000","last_clean":"2023-12-26T21:53:46.311050+0000","last_became_active":"2023-12-26T21:53:46.304893+0000","last_became_peered":"2023-12-26T21:53:46.304893+0000","last_unstale":"2023-12-26T21:53:46.311050+0000","last_undegraded":"2023-12-26T21:53:46.311050+0000","last_fullsized":"2023-12-26T21:53:46.311050+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:05.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307642+0000","last_change":"2023-12-26T21:53:46.301677+0000","last_active":"2023-12-26T21:53:46.307642+0000","last_peered":"2023-12-26T21:53:46.307642+0000","last_clean":"2023-12-26T21:53:46.307642+0000","last_became_active":"2023-12-26T21:53:46.301432+0000","last_became_peered":"2023-12-26T21:53:46.301432+0000","last_unstale":"2023-12-26T21:53:46.307642+0000","last_undegraded":"2023-12-26T21:53:46.307642+0000","last_fullsized":"2023-12-26T21:53:46.307642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"109'4","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321799+0000","last_change":"2023-12-26T21:53:45.285708+0000","last_active":"2023-12-26T21:53:46.321799+0000","last_peered":"2023-12-26T21:53:46.321799+0000","last_clean":"2023-12-26T21:53:46.321799+0000","last_became_active":"2023-12-26T21:53:45.285466+0000","last_became_peered":"2023-12-26T21:53:45.285466+0000","last_unstale":"2023-12-26T21:53:46.321799+0000","last_undegraded":"2023-12-26T21:53:46.321799+0000","last_fullsized":"2023-12-26T21:53:46.321799+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:11.158488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310328+0000","last_change":"2023-12-26T21:53:46.307223+0000","last_active":"2023-12-26T21:53:46.310328+0000","last_peered":"2023-12-26T21:53:46.310328+0000","last_clean":"2023-12-26T21:53:46.310328+0000","last_became_active":"2023-12-26T21:53:46.306963+0000","last_became_peered":"2023-12-26T21:53:46.306963+0000","last_unstale":"2023-12-26T21:53:46.310328+0000","last_undegraded":"2023-12-26T21:53:46.310328+0000","last_fullsized":"2023-12-26T21:53:46.310328+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:12:37.634766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.264186+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:17.475634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262584+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:17.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.305631+0000","last_change":"2023-12-26T21:53:46.299487+0000","last_active":"2023-12-26T21:53:46.305631+0000","last_peered":"2023-12-26T21:53:46.305631+0000","last_clean":"2023-12-26T21:53:46.305631+0000","last_became_active":"2023-12-26T21:53:46.299063+0000","last_became_peered":"2023-12-26T21:53:46.299063+0000","last_unstale":"2023-12-26T21:53:46.305631+0000","last_undegraded":"2023-12-26T21:53:46.305631+0000","last_fullsized":"2023-12-26T21:53:46.305631+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:48.449758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262237+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:56:53.977018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258326+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:40:24.430368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.263523+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:08:23.502904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299738+0000","last_change":"2023-12-26T21:53:46.297785+0000","last_active":"2023-12-26T21:53:46.299738+0000","last_peered":"2023-12-26T21:53:46.299738+0000","last_clean":"2023-12-26T21:53:46.299738+0000","last_became_active":"2023-12-26T21:53:46.297505+0000","last_became_peered":"2023-12-26T21:53:46.297505+0000","last_unstale":"2023-12-26T21:53:46.299738+0000","last_undegraded":"2023-12-26T21:53:46.299738+0000","last_fullsized":"2023-12-26T21:53:46.299738+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:33.304213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281100+0000","last_change":"2023-12-26T21:53:45.279536+0000","last_active":"2023-12-26T21:53:46.281100+0000","last_peered":"2023-12-26T21:53:46.281100+0000","last_clean":"2023-12-26T21:53:46.281100+0000","last_became_active":"2023-12-26T21:53:45.279282+0000","last_became_peered":"2023-12-26T21:53:45.279282+0000","last_unstale":"2023-12-26T21:53:46.281100+0000","last_undegraded":"2023-12-26T21:53:46.281100+0000","last_fullsized":"2023-12-26T21:53:46.281100+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:58.794519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.288617+0000","last_change":"2023-12-26T21:53:46.285357+0000","last_active":"2023-12-26T21:53:46.288617+0000","last_peered":"2023-12-26T21:53:46.288617+0000","last_clean":"2023-12-26T21:53:46.288617+0000","last_became_active":"2023-12-26T21:53:46.285051+0000","last_became_peered":"2023-12-26T21:53:46.285051+0000","last_unstale":"2023-12-26T21:53:46.288617+0000","last_undegraded":"2023-12-26T21:53:46.288617+0000","last_fullsized":"2023-12-26T21:53:46.288617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:31:16.372604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281226+0000","last_change":"2023-12-26T21:53:45.268538+0000","last_active":"2023-12-26T21:53:46.281226+0000","last_peered":"2023-12-26T21:53:46.281226+0000","last_clean":"2023-12-26T21:53:46.281226+0000","last_became_active":"2023-12-26T21:53:45.268069+0000","last_became_peered":"2023-12-26T21:53:45.268069+0000","last_unstale":"2023-12-26T21:53:46.281226+0000","last_undegraded":"2023-12-26T21:53:46.281226+0000","last_fullsized":"2023-12-26T21:53:46.281226+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:25.546517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300878+0000","last_change":"2023-12-26T21:53:46.297819+0000","last_active":"2023-12-26T21:53:46.300878+0000","last_peered":"2023-12-26T21:53:46.300878+0000","last_clean":"2023-12-26T21:53:46.300878+0000","last_became_active":"2023-12-26T21:53:46.297541+0000","last_became_peered":"2023-12-26T21:53:46.297541+0000","last_unstale":"2023-12-26T21:53:46.300878+0000","last_undegraded":"2023-12-26T21:53:46.300878+0000","last_fullsized":"2023-12-26T21:53:46.300878+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:10:09.104721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289148+0000","last_change":"2023-12-26T21:53:45.281269+0000","last_active":"2023-12-26T21:53:46.289148+0000","last_peered":"2023-12-26T21:53:46.289148+0000","last_clean":"2023-12-26T21:53:46.289148+0000","last_became_active":"2023-12-26T21:53:45.280967+0000","last_became_peered":"2023-12-26T21:53:45.280967+0000","last_unstale":"2023-12-26T21:53:46.289148+0000","last_undegraded":"2023-12-26T21:53:46.289148+0000","last_fullsized":"2023-12-26T21:53:46.289148+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:20:00.172583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299929+0000","last_change":"2023-12-26T21:53:46.298319+0000","last_active":"2023-12-26T21:53:46.299929+0000","last_peered":"2023-12-26T21:53:46.299929+0000","last_clean":"2023-12-26T21:53:46.299929+0000","last_became_active":"2023-12-26T21:53:46.298050+0000","last_became_peered":"2023-12-26T21:53:46.298050+0000","last_unstale":"2023-12-26T21:53:46.299929+0000","last_undegraded":"2023-12-26T21:53:46.299929+0000","last_fullsized":"2023-12-26T21:53:46.299929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:34.648293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262776+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:43:16.270159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"109'3","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321640+0000","last_change":"2023-12-26T21:53:45.285950+0000","last_active":"2023-12-26T21:53:46.321640+0000","last_peered":"2023-12-26T21:53:46.321640+0000","last_clean":"2023-12-26T21:53:46.321640+0000","last_became_active":"2023-12-26T21:53:45.285657+0000","last_became_peered":"2023-12-26T21:53:45.285657+0000","last_unstale":"2023-12-26T21:53:46.321640+0000","last_undegraded":"2023-12-26T21:53:46.321640+0000","last_fullsized":"2023-12-26T21:53:46.321640+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:55.530687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303089+0000","last_change":"2023-12-26T21:53:46.301426+0000","last_active":"2023-12-26T21:53:46.303089+0000","last_peered":"2023-12-26T21:53:46.303089+0000","last_clean":"2023-12-26T21:53:46.303089+0000","last_became_active":"2023-12-26T21:53:46.301116+0000","last_became_peered":"2023-12-26T21:53:46.301116+0000","last_unstale":"2023-12-26T21:53:46.303089+0000","last_undegraded":"2023-12-26T21:53:46.303089+0000","last_fullsized":"2023-12-26T21:53:46.303089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:49.402467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258535+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:44:24.266594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262178+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:30:14.941645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304318+0000","last_change":"2023-12-26T21:53:46.298424+0000","last_active":"2023-12-26T21:53:46.304318+0000","last_peered":"2023-12-26T21:53:46.304318+0000","last_clean":"2023-12-26T21:53:46.304318+0000","last_became_active":"2023-12-26T21:53:46.298145+0000","last_became_peered":"2023-12-26T21:53:46.298145+0000","last_unstale":"2023-12-26T21:53:46.304318+0000","last_undegraded":"2023-12-26T21:53:46.304318+0000","last_fullsized":"2023-12-26T21:53:46.304318+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:24:04.826615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285162+0000","last_change":"2023-12-26T21:53:45.281444+0000","last_active":"2023-12-26T21:53:46.285162+0000","last_peered":"2023-12-26T21:53:46.285162+0000","last_clean":"2023-12-26T21:53:46.285162+0000","last_became_active":"2023-12-26T21:53:45.280420+0000","last_became_peered":"2023-12-26T21:53:45.280420+0000","last_unstale":"2023-12-26T21:53:46.285162+0000","last_undegraded":"2023-12-26T21:53:46.285162+0000","last_fullsized":"2023-12-26T21:53:46.285162+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:17.475634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300883+0000","last_change":"2023-12-26T21:53:46.297808+0000","last_active":"2023-12-26T21:53:46.300883+0000","last_peered":"2023-12-26T21:53:46.300883+0000","last_clean":"2023-12-26T21:53:46.300883+0000","last_became_active":"2023-12-26T21:53:46.296873+0000","last_became_peered":"2023-12-26T21:53:46.296873+0000","last_unstale":"2023-12-26T21:53:46.300883+0000","last_undegraded":"2023-12-26T21:53:46.300883+0000","last_fullsized":"2023-12-26T21:53:46.300883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:29.573501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262772+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:25:10.062797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262093+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:12:36.629000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.320094+0000","last_change":"2023-12-26T21:53:45.280067+0000","last_active":"2023-12-26T21:53:46.320094+0000","last_peered":"2023-12-26T21:53:46.320094+0000","last_clean":"2023-12-26T21:53:46.320094+0000","last_became_active":"2023-12-26T21:53:45.279713+0000","last_became_peered":"2023-12-26T21:53:45.279713+0000","last_unstale":"2023-12-26T21:53:46.320094+0000","last_undegraded":"2023-12-26T21:53:46.320094+0000","last_fullsized":"2023-12-26T21:53:46.320094+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:15.927146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.316477+0000","last_change":"2023-12-26T21:53:45.277896+0000","last_active":"2023-12-26T21:53:46.316477+0000","last_peered":"2023-12-26T21:53:46.316477+0000","last_clean":"2023-12-26T21:53:46.316477+0000","last_became_active":"2023-12-26T21:53:45.277584+0000","last_became_peered":"2023-12-26T21:53:45.277584+0000","last_unstale":"2023-12-26T21:53:46.316477+0000","last_undegraded":"2023-12-26T21:53:46.316477+0000","last_fullsized":"2023-12-26T21:53:46.316477+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:44:17.169190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.282526+0000","last_change":"2023-12-26T21:53:45.280486+0000","last_active":"2023-12-26T21:53:46.282526+0000","last_peered":"2023-12-26T21:53:46.282526+0000","last_clean":"2023-12-26T21:53:46.282526+0000","last_became_active":"2023-12-26T21:53:45.280178+0000","last_became_peered":"2023-12-26T21:53:45.280178+0000","last_unstale":"2023-12-26T21:53:46.282526+0000","last_undegraded":"2023-12-26T21:53:46.282526+0000","last_fullsized":"2023-12-26T21:53:46.282526+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:13:56.557753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304197+0000","last_change":"2023-12-26T21:53:46.297254+0000","last_active":"2023-12-26T21:53:46.304197+0000","last_peered":"2023-12-26T21:53:46.304197+0000","last_clean":"2023-12-26T21:53:46.304197+0000","last_became_active":"2023-12-26T21:53:46.296358+0000","last_became_peered":"2023-12-26T21:53:46.296358+0000","last_unstale":"2023-12-26T21:53:46.304197+0000","last_undegraded":"2023-12-26T21:53:46.304197+0000","last_fullsized":"2023-12-26T21:53:46.304197+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.317134+0000","last_change":"2023-12-26T21:53:45.271721+0000","last_active":"2023-12-26T21:53:46.317134+0000","last_peered":"2023-12-26T21:53:46.317134+0000","last_clean":"2023-12-26T21:53:46.317134+0000","last_became_active":"2023-12-26T21:53:45.271453+0000","last_became_peered":"2023-12-26T21:53:45.271453+0000","last_unstale":"2023-12-26T21:53:46.317134+0000","last_undegraded":"2023-12-26T21:53:46.317134+0000","last_fullsized":"2023-12-26T21:53:46.317134+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:49:24.818172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303057+0000","last_change":"2023-12-26T21:53:46.301435+0000","last_active":"2023-12-26T21:53:46.303057+0000","last_peered":"2023-12-26T21:53:46.303057+0000","last_clean":"2023-12-26T21:53:46.303057+0000","last_became_active":"2023-12-26T21:53:46.301117+0000","last_became_peered":"2023-12-26T21:53:46.301117+0000","last_unstale":"2023-12-26T21:53:46.303057+0000","last_undegraded":"2023-12-26T21:53:46.303057+0000","last_fullsized":"2023-12-26T21:53:46.303057+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:39.625634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285208+0000","last_change":"2023-12-26T21:53:45.279979+0000","last_active":"2023-12-26T21:53:46.285208+0000","last_peered":"2023-12-26T21:53:46.285208+0000","last_clean":"2023-12-26T21:53:46.285208+0000","last_became_active":"2023-12-26T21:53:45.279563+0000","last_became_peered":"2023-12-26T21:53:45.279563+0000","last_unstale":"2023-12-26T21:53:46.285208+0000","last_undegraded":"2023-12-26T21:53:46.285208+0000","last_fullsized":"2023-12-26T21:53:46.285208+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:00:28.640861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262063+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:32.298447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281331+0000","last_change":"2023-12-26T21:53:45.282408+0000","last_active":"2023-12-26T21:53:46.281331+0000","last_peered":"2023-12-26T21:53:46.281331+0000","last_clean":"2023-12-26T21:53:46.281331+0000","last_became_active":"2023-12-26T21:53:45.282074+0000","last_became_peered":"2023-12-26T21:53:45.282074+0000","last_unstale":"2023-12-26T21:53:46.281331+0000","last_undegraded":"2023-12-26T21:53:46.281331+0000","last_fullsized":"2023-12-26T21:53:46.281331+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:18:52.576472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.302717+0000","last_change":"2023-12-26T21:53:46.300574+0000","last_active":"2023-12-26T21:53:46.302717+0000","last_peered":"2023-12-26T21:53:46.302717+0000","last_clean":"2023-12-26T21:53:46.302717+0000","last_became_active":"2023-12-26T21:53:46.300296+0000","last_became_peered":"2023-12-26T21:53:46.300296+0000","last_unstale":"2023-12-26T21:53:46.302717+0000","last_undegraded":"2023-12-26T21:53:46.302717+0000","last_fullsized":"2023-12-26T21:53:46.302717+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:51.026196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.318605+0000","last_change":"2023-12-26T21:53:45.280489+0000","last_active":"2023-12-26T21:53:46.318605+0000","last_peered":"2023-12-26T21:53:46.318605+0000","last_clean":"2023-12-26T21:53:46.318605+0000","last_became_active":"2023-12-26T21:53:45.279071+0000","last_became_peered":"2023-12-26T21:53:45.279071+0000","last_unstale":"2023-12-26T21:53:46.318605+0000","last_undegraded":"2023-12-26T21:53:46.318605+0000","last_fullsized":"2023-12-26T21:53:46.318605+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:28.851567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308510+0000","last_change":"2023-12-26T21:53:46.304923+0000","last_active":"2023-12-26T21:53:46.308510+0000","last_peered":"2023-12-26T21:53:46.308510+0000","last_clean":"2023-12-26T21:53:46.308510+0000","last_became_active":"2023-12-26T21:53:46.304621+0000","last_became_peered":"2023-12-26T21:53:46.304621+0000","last_unstale":"2023-12-26T21:53:46.308510+0000","last_undegraded":"2023-12-26T21:53:46.308510+0000","last_fullsized":"2023-12-26T21:53:46.308510+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:15:24.617194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262644+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:16.240935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299826+0000","last_change":"2023-12-26T21:53:46.288775+0000","last_active":"2023-12-26T21:53:46.299826+0000","last_peered":"2023-12-26T21:53:46.299826+0000","last_clean":"2023-12-26T21:53:46.299826+0000","last_became_active":"2023-12-26T21:53:46.288489+0000","last_became_peered":"2023-12-26T21:53:46.288489+0000","last_unstale":"2023-12-26T21:53:46.299826+0000","last_undegraded":"2023-12-26T21:53:46.299826+0000","last_fullsized":"2023-12-26T21:53:46.299826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:48:03.925563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258576+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:59:57.204768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262786+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:00:28.640861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310423+0000","last_change":"2023-12-26T21:53:46.306701+0000","last_active":"2023-12-26T21:53:46.310423+0000","last_peered":"2023-12-26T21:53:46.310423+0000","last_clean":"2023-12-26T21:53:46.310423+0000","last_became_active":"2023-12-26T21:53:46.306450+0000","last_became_peered":"2023-12-26T21:53:46.306450+0000","last_unstale":"2023-12-26T21:53:46.310423+0000","last_undegraded":"2023-12-26T21:53:46.310423+0000","last_fullsized":"2023-12-26T21:53:46.310423+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:56.921774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.284986+0000","last_change":"2023-12-26T21:53:45.279374+0000","last_active":"2023-12-26T21:53:46.284986+0000","last_peered":"2023-12-26T21:53:46.284986+0000","last_clean":"2023-12-26T21:53:46.284986+0000","last_became_active":"2023-12-26T21:53:45.278479+0000","last_became_peered":"2023-12-26T21:53:45.278479+0000","last_unstale":"2023-12-26T21:53:46.284986+0000","last_undegraded":"2023-12-26T21:53:46.284986+0000","last_fullsized":"2023-12-26T21:53:46.284986+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:29.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.262986+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:29.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308046+0000","last_change":"2023-12-26T21:53:46.306183+0000","last_active":"2023-12-26T21:53:46.308046+0000","last_peered":"2023-12-26T21:53:46.308046+0000","last_clean":"2023-12-26T21:53:46.308046+0000","last_became_active":"2023-12-26T21:53:46.305918+0000","last_became_peered":"2023-12-26T21:53:46.305918+0000","last_unstale":"2023-12-26T21:53:46.308046+0000","last_undegraded":"2023-12-26T21:53:46.308046+0000","last_fullsized":"2023-12-26T21:53:46.308046+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:49:38.482545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286935+0000","last_change":"2023-12-26T21:53:45.284416+0000","last_active":"2023-12-26T21:53:46.286935+0000","last_peered":"2023-12-26T21:53:46.286935+0000","last_clean":"2023-12-26T21:53:46.286935+0000","last_became_active":"2023-12-26T21:53:45.284072+0000","last_became_peered":"2023-12-26T21:53:45.284072+0000","last_unstale":"2023-12-26T21:53:46.286935+0000","last_undegraded":"2023-12-26T21:53:46.286935+0000","last_fullsized":"2023-12-26T21:53:46.286935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:46:59.726957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307933+0000","last_change":"2023-12-26T21:53:46.303791+0000","last_active":"2023-12-26T21:53:46.307933+0000","last_peered":"2023-12-26T21:53:46.307933+0000","last_clean":"2023-12-26T21:53:46.307933+0000","last_became_active":"2023-12-26T21:53:46.303518+0000","last_became_peered":"2023-12-26T21:53:46.303518+0000","last_unstale":"2023-12-26T21:53:46.307933+0000","last_undegraded":"2023-12-26T21:53:46.307933+0000","last_fullsized":"2023-12-26T21:53:46.307933+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:17.246701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:53:44.249974+0000","last_change":"2023-12-26T21:53:44.258326+0000","last_active":"2023-12-26T21:53:44.249974+0000","last_peered":"2023-12-26T21:53:44.249974+0000","last_clean":"2023-12-26T21:53:44.249974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:44.249974+0000","last_undegraded":"2023-12-26T21:53:44.249974+0000","last_fullsized":"2023-12-26T21:53:44.249974+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:17:49.107900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:53:45.255741+0000","last_change":"2023-12-26T21:53:45.255741+0000","last_active":"2023-12-26T21:53:45.255741+0000","last_peered":"2023-12-26T21:53:45.255741+0000","last_clean":"2023-12-26T21:53:45.255741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:53:45.255741+0000","last_undegraded":"2023-12-26T21:53:45.255741+0000","last_fullsized":"2023-12-26T21:53:45.255741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.319541+0000","last_change":"2023-12-26T21:53:45.277923+0000","last_active":"2023-12-26T21:53:46.319541+0000","last_peered":"2023-12-26T21:53:46.319541+0000","last_clean":"2023-12-26T21:53:46.319541+0000","last_became_active":"2023-12-26T21:53:45.277681+0000","last_became_peered":"2023-12-26T21:53:45.277681+0000","last_unstale":"2023-12-26T21:53:46.319541+0000","last_undegraded":"2023-12-26T21:53:46.319541+0000","last_fullsized":"2023-12-26T21:53:46.319541+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:40.310455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'205","reported_seq":555,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286580+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:46.286580+0000","last_peered":"2023-12-26T21:53:46.286580+0000","last_clean":"2023-12-26T21:53:46.286580+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:46.286580+0000","last_undegraded":"2023-12-26T21:53:46.286580+0000","last_fullsized":"2023-12-26T21:53:46.286580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":4},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542348,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302700,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069140,"statfs":{"total":96636764160,"available":96326799360,"internally_reserved":0,"allocated":1347584,"data_stored":653499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.47199999999999998,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.72499999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.69799999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74399999999999999,"5min":1.016,"15min":1.048},"last":0.74199999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.038,"15min":1.0509999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.85699999999999998,"5min":0.99199999999999999,"15min":1.02},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.62,"5min":0.504,"15min":0.5},"max":{"1min":0.94299999999999995,"5min":1.0189999999999999,"15min":1.052},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83499999999999996,"5min":1.036,"15min":1.036},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.747,"15min":0.746},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0129999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86399999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.117,"15min":1.117},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.437},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.006},"last":0.88}]}]},{"osd":6,"up_from":15,"seq":64424509643,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306744,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065096,"statfs":{"total":96636764160,"available":96322658304,"internally_reserved":0,"allocated":1294336,"data_stored":626742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.085},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69199999999999995,"15min":0.71799999999999997},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.34799999999999998},"max":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67800000000000005,"15min":0.70199999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.95299999999999996,"15min":0.98799999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.54200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.97599999999999998,"15min":1.054},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.93200000000000005,"15min":1.105},"last":0.753},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.79200000000000004,"5min":0.92000000000000004,"15min":1.048},"last":0.83999999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.90200000000000002,"5min":1.026,"15min":1.1799999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86899999999999999,"5min":0.96999999999999997,"15min":1.069},"last":0.85599999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.91100000000000003,"15min":1.137},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93100000000000005,"5min":1.0049999999999999,"15min":1.296},"last":0.78200000000000003}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.437},"max":{"1min":1.008,"5min":1.008,"15min":1.3380000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.56799999999999995,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.95599999999999996,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.87}]}]},{"osd":1,"up_from":15,"seq":64424509644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311472,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060368,"statfs":{"total":96636764160,"available":96317816832,"internally_reserved":0,"allocated":1941504,"data_stored":1243887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.436},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.55700000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.84699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.70399999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.87}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.88400000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.90500000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509643,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302660,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069180,"statfs":{"total":96636764160,"available":96326840320,"internally_reserved":0,"allocated":1306624,"data_stored":632939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":1.056},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47799999999999998,"15min":0.49399999999999999},"min":{"1min":0.315,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":1.006},"last":0.311},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.84699999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80000000000000004,"5min":0.86699999999999999,"15min":1.1870000000000001},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.752},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.042,"5min":1.042,"15min":1.411},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.76800000000000002},"min":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.2070000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.754,"15min":0.79800000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.2569999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77100000000000002,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.452},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.214},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.012,"15min":1.3779999999999999},"last":0.50600000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46400000000000002},"max":{"1min":1.018,"5min":1.042,"15min":1.4359999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.79200000000000004},"min":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":1.0029999999999999,"5min":1.0069999999999999,"15min":1.3240000000000001},"last":0.49299999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306744,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065096,"statfs":{"total":96636764160,"available":96322658304,"internally_reserved":0,"allocated":1294336,"data_stored":626742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.877},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68600000000000005,"5min":0.83899999999999997,"15min":0.84399999999999997},"last":0.433}]},{"osd":1,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.65300000000000002,"5min":0.70799999999999996,"15min":0.92000000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.31,"15min":0.309},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.78600000000000003},"last":0.38900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":1,"15min":1},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.82299999999999995,"15min":0.85699999999999998},"last":0.34599999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.46800000000000003,"15min":0.34799999999999998},"max":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":1.284},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.76300000000000001},"min":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.95699999999999996,"5min":0.98399999999999999,"15min":1.1759999999999999},"last":0.83699999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.024,"5min":1.0600000000000001,"15min":1.2170000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":0.95499999999999996,"5min":1.014,"15min":1.1319999999999999},"last":0.86499999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.038,"15min":1.242},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.40300000000000002},"max":{"1min":1.018,"5min":1.0609999999999999,"15min":1.272},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":1.073,"15min":1.1220000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.50800000000000001},"max":{"1min":0.999,"5min":1.0189999999999999,"15min":1.1990000000000001},"last":0.84899999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509645,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310900,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060940,"statfs":{"total":96636764160,"available":96318402560,"internally_reserved":0,"allocated":1355776,"data_stored":654075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.378}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82499999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.753}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.81299999999999994}]}]},{"osd":4,"up_from":14,"seq":60129542348,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306796,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065044,"statfs":{"total":96636764160,"available":96322605056,"internally_reserved":0,"allocated":1347584,"data_stored":653463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97899999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.48499999999999999,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":1.008},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.95199999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88900000000000001,"5min":0.97199999999999998,"15min":0.97299999999999998},"last":0.624}]},{"osd":3,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.71499999999999997},"min":{"1min":0.504,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.78100000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.438,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.84399999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509642,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":1223307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.69999999999999996},"min":{"1min":0.44500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84199999999999997,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.749},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.4570000000000001},"last":0.73399999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68700000000000006,"15min":0.71499999999999997},"min":{"1min":0.499,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.625},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67800000000000005,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.4219999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.72799999999999998},"min":{"1min":0.54900000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.88500000000000001,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68600000000000005,"15min":0.71099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.92400000000000004,"15min":1.0569999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":1.052,"15min":1.4890000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.73599999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47799999999999998},"max":{"1min":0.88300000000000001,"5min":0.92200000000000004,"15min":1.081},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":2.5070000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.54500000000000004,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":2.8039999999999998},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.65700000000000003,"15min":0.66400000000000003},"min":{"1min":0.505,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":1.637},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.623,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":1.111,"5min":1.111,"15min":2.5579999999999998},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.61399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.97499999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.71299999999999997,"15min":0.70999999999999996},"min":{"1min":0.52200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.5980000000000001},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:51.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2023-12-26T21:53:51.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2023-12-26T21:53:51.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2023-12-26T21:53:51.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2023-12-26T21:53:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2023-12-26T21:53:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2023-12-26T21:53:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2023-12-26T21:53:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2023-12-26T21:53:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2023-12-26T21:53:51.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.264186+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:20:17.475634+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262584+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:13:17.024394+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262237+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:56:53.977018+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2023-12-26T21:53:51.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-12-26T21:53:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.258326+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:40:24.430368+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:53:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2023-12-26T21:53:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-12-26T21:53:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.263523+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:08:23.502904+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2023-12-26T21:53:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262776+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:43:16.270159+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:51.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2023-12-26T21:53:51.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2023-12-26T21:53:51.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.258535+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T07:44:24.266594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:53:51.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2023-12-26T21:53:51.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2023-12-26T21:53:51.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262178+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:30:14.941645+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:53:51.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2023-12-26T21:53:51.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262772+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:25:10.062797+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262093+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:12:36.629000+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2023-12-26T21:53:51.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262063+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T07:18:32.298447+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262644+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:27:16.240935+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2023-12-26T21:53:51.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.258576+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:59:57.204768+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262786+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T22:00:28.640861+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.262986+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T22:23:29.442183+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-12-26T21:53:51.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:53:44.249974+0000', 'last_change': '2023-12-26T21:53:44.258326+0000', 'last_active': '2023-12-26T21:53:44.249974+0000', 'last_peered': '2023-12-26T21:53:44.249974+0000', 'last_clean': '2023-12-26T21:53:44.249974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:44.249974+0000', 'last_undegraded': '2023-12-26T21:53:44.249974+0000', 'last_fullsized': '2023-12-26T21:53:44.249974+0000', 'mapping_epoch': 107, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 107, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:44.249974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T22:17:49.107900+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:53:51.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2023-12-26T21:53:51.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:53:45.255741+0000', 'last_change': '2023-12-26T21:53:45.255741+0000', 'last_active': '2023-12-26T21:53:45.255741+0000', 'last_peered': '2023-12-26T21:53:45.255741+0000', 'last_clean': '2023-12-26T21:53:45.255741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:53:45.255741+0000', 'last_undegraded': '2023-12-26T21:53:45.255741+0000', 'last_fullsized': '2023-12-26T21:53:45.255741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'last_clean_scrub_stamp': '2023-12-26T21:53:45.255741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:53:51.577 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-26T21:53:51.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.790+0000 7fc0a6dff700 1 Processor -- start 2023-12-26T21:53:51.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.790+0000 7fc0a6dff700 1 -- start start 2023-12-26T21:53:51.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.791+0000 7fc0a6dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc0a0159a70 0x7fc0a00b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.791+0000 7fc0a6dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc0a00b2a90 0x7fc0a00b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.791+0000 7fc0a6dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a00afcd0 con 0x7fc0a01596a0 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.791+0000 7fc0a6dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a00b3e70 con 0x7fc0a00b2a90 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.791+0000 7fc0a6dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a00b0f20 con 0x7fc0a0159a70 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.791+0000 7fc0a4b9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc0a01596a0 0x7fc0a00b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45114/0 (socket says 172.21.15.131:45114) 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.791+0000 7fc0a4b9b700 1 -- 172.21.15.131:0/2047882396 learned_addr learned my addr 172.21.15.131:0/2047882396 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.792+0000 7fc0a539c700 1 --2- 172.21.15.131:0/2047882396 >> v2:172.21.15.131:3300/0 conn(0x7fc0a00b2a90 0x7fc0a00b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.792+0000 7fc09ffff700 1 --2- 172.21.15.131:0/2047882396 >> v2:172.21.15.190:3300/0 conn(0x7fc0a0159a70 0x7fc0a00b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.792+0000 7fc09f7fe700 1 -- 172.21.15.131:0/2047882396 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787953399 0 0) 0x7fc0a00afcd0 con 0x7fc0a01596a0 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.792+0000 7fc09f7fe700 1 -- 172.21.15.131:0/2047882396 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc090003570 con 0x7fc0a01596a0 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.792+0000 7fc0a539c700 1 -- 172.21.15.131:0/2047882396 >> v2:172.21.15.190:3300/0 conn(0x7fc0a0159a70 msgr2=0x7fc0a00b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.792+0000 7fc0a539c700 1 --2- 172.21.15.131:0/2047882396 >> v2:172.21.15.190:3300/0 conn(0x7fc0a0159a70 0x7fc0a00b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.792+0000 7fc0a539c700 1 -- 172.21.15.131:0/2047882396 >> v1:172.21.15.131:6789/0 conn(0x7fc0a01596a0 legacy=0x7fc0a00b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.792+0000 7fc0a539c700 1 -- 172.21.15.131:0/2047882396 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a00b3540 con 0x7fc0a00b2a90 2023-12-26T21:53:51.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.792+0000 7fc0a539c700 1 --2- 172.21.15.131:0/2047882396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a00b2a90 0x7fc0a00b06a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00642f0 tx=0x7fc098009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961f7ec902fde6a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc09f7fe700 1 -- 172.21.15.131:0/2047882396 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800e070 con 0x7fc0a00b2a90 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc09f7fe700 1 -- 172.21.15.131:0/2047882396 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09800bab0 con 0x7fc0a00b2a90 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc09f7fe700 1 -- 172.21.15.131:0/2047882396 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800be30 con 0x7fc0a00b2a90 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2047882396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a00b2a90 msgr2=0x7fc0a00b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc0a6dff700 1 --2- 172.21.15.131:0/2047882396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a00b2a90 0x7fc0a00b06a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00642f0 tx=0x7fc098009780 comp rx=0 tx=0).stop 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2047882396 shutdown_connections 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc0a6dff700 1 --2- 172.21.15.131:0/2047882396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a00b2a90 0x7fc0a00b06a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc0a6dff700 1 --2- 172.21.15.131:0/2047882396 >> v2:172.21.15.190:3300/0 conn(0x7fc0a0159a70 0x7fc0a00b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2047882396 >> 172.21.15.131:0/2047882396 conn(0x7fc0a006ece0 msgr2=0x7fc0a006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2047882396 shutdown_connections 2023-12-26T21:53:51.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.793+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2047882396 wait complete. 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.795+0000 7fc0a6dff700 1 Processor -- start 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.795+0000 7fc0a6dff700 1 -- start start 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc0a6dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0a00b2a90 0x7fc0a00c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc0a6dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a01596a0 0x7fc0a00ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc0a6dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0a0159a70 0x7fc0a0152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc0a6dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00c9030 con 0x7fc0a01596a0 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc0a6dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00c9a00 con 0x7fc0a0159a70 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc0a6dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00cd9b0 con 0x7fc0a00b2a90 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a01596a0 0x7fc0a00ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a01596a0 0x7fc0a00ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58834/0 (socket says 172.21.15.131:58834) 2023-12-26T21:53:51.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc09ffff700 1 -- 172.21.15.131:0/2867874400 learned_addr learned my addr 172.21.15.131:0/2867874400 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc0a4b9b700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0a00b2a90 0x7fc0a00c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc0a539c700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0a0159a70 0x7fc0a0152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc09ffff700 1 -- 172.21.15.131:0/2867874400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0a00b2a90 msgr2=0x7fc0a00c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc09ffff700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0a00b2a90 0x7fc0a00c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc09ffff700 1 -- 172.21.15.131:0/2867874400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0a0159a70 msgr2=0x7fc0a0152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc09ffff700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0a0159a70 0x7fc0a0152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc09ffff700 1 -- 172.21.15.131:0/2867874400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098009a20 con 0x7fc0a01596a0 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.796+0000 7fc0a539c700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0a0159a70 0x7fc0a0152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.797+0000 7fc09ffff700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a01596a0 0x7fc0a00ca050 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00e930 tx=0x7fc08c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.797+0000 7fc09cff9700 1 -- 172.21.15.131:0/2867874400 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00eed0 con 0x7fc0a01596a0 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.797+0000 7fc09cff9700 1 -- 172.21.15.131:0/2867874400 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c00c850 con 0x7fc0a01596a0 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.797+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a00c89e0 con 0x7fc0a01596a0 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.797+0000 7fc09cff9700 1 -- 172.21.15.131:0/2867874400 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00cbf0 con 0x7fc0a01596a0 2023-12-26T21:53:51.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.797+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a00c86a0 con 0x7fc0a01596a0 2023-12-26T21:53:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.798+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a00c91e0 con 0x7fc0a01596a0 2023-12-26T21:53:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.799+0000 7fc09cff9700 1 -- 172.21.15.131:0/2867874400 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc08c013470 con 0x7fc0a01596a0 2023-12-26T21:53:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.800+0000 7fc09cff9700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc084066f30 0x7fc0840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.800+0000 7fc09cff9700 1 -- 172.21.15.131:0/2867874400 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fc08c084950 con 0x7fc0a01596a0 2023-12-26T21:53:51.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.800+0000 7fc0a4b9b700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc084066f30 0x7fc0840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:51.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.801+0000 7fc0a4b9b700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc084066f30 0x7fc0840693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0940058d0 tx=0x7fc094009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:51.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:51.804+0000 7fc09cff9700 1 -- 172.21.15.131:0/2867874400 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0a00c91e0 con 0x7fc0a01596a0 2023-12-26T21:53:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.075+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc0a00657e0 con 0x7fc0a01596a0 2023-12-26T21:53:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.076+0000 7fc09cff9700 1 -- 172.21.15.131:0/2867874400 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1178 (secure 0 0 0) 0x7fc08c051990 con 0x7fc0a01596a0 2023-12-26T21:53:52.076 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:52.077 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"ab603302-d88b-429a-be64-92b3c298adad","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1047,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1703626596,"num_in_osds":8,"osd_in_since":1703626587,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":49},{"state_name":"unknown","count":32},{"state_name":"creating+peering","count":16}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":591104,"bytes_used":2510086144,"bytes_avail":770584027136,"bytes_total":773094113280,"unknown_pgs_ratio":0.32989689707756042,"inactive_pgs_ratio":0.16494844853878021,"write_bytes_sec":699,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":50,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7146}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-12-26T21:51:27.533973+0000","services":{}},"progress_events":{}} 2023-12-26T21:53:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc084066f30 msgr2=0x7fc0840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc084066f30 0x7fc0840693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0940058d0 tx=0x7fc094009040 comp rx=0 tx=0).stop 2023-12-26T21:53:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a01596a0 msgr2=0x7fc0a00ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a01596a0 0x7fc0a00ca050 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00e930 tx=0x7fc08c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 shutdown_connections 2023-12-26T21:53:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0a0159a70 0x7fc0a0152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0a00b2a90 0x7fc0a00c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc084066f30 0x7fc0840693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 --2- 172.21.15.131:0/2867874400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0a01596a0 0x7fc0a00ca050 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 >> 172.21.15.131:0/2867874400 conn(0x7fc0a006ece0 msgr2=0x7fc0a00b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:52.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 shutdown_connections 2023-12-26T21:53:52.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.078+0000 7fc0a6dff700 1 -- 172.21.15.131:0/2867874400 wait complete. 2023-12-26T21:53:52.096 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-26T21:53:52.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.284+0000 7fba043ff700 1 Processor -- start 2023-12-26T21:53:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.284+0000 7fba043ff700 1 -- start start 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.285+0000 7fba043ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb9fc1596a0 0x7fb9fc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba043ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc0b0870 0x7fb9fc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba043ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc0afcd0 con 0x7fb9fc159a70 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba043ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9fc0b3e70 con 0x7fb9fc1596a0 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba043ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0b2e00 con 0x7fb9fc0b0870 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba0219b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb9fc1596a0 0x7fb9fc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba0219b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb9fc1596a0 0x7fb9fc0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58842/0 (socket says 172.21.15.131:58842) 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba0219b700 1 -- 172.21.15.131:0/915664978 learned_addr learned my addr 172.21.15.131:0/915664978 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba0299c700 1 --2- 172.21.15.131:0/915664978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc0b0870 0x7fb9fc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba0219b700 1 -- 172.21.15.131:0/915664978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc0b0870 msgr2=0x7fb9fc0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba0219b700 1 --2- 172.21.15.131:0/915664978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc0b0870 0x7fb9fc0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba0219b700 1 -- 172.21.15.131:0/915664978 >> v1:172.21.15.190:6789/0 conn(0x7fb9fc159a70 legacy=0x7fb9fc0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.286+0000 7fba0219b700 1 -- 172.21.15.131:0/915664978 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0b0cb0 con 0x7fb9fc1596a0 2023-12-26T21:53:52.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba0219b700 1 --2- 172.21.15.131:0/915664978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 0x7fb9fc0b2380 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80099b0 tx=0x7fb9f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612e8f13eaae0210 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:52.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba01199700 1 -- 172.21.15.131:0/915664978 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800b820 con 0x7fb9fc1596a0 2023-12-26T21:53:52.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba01199700 1 -- 172.21.15.131:0/915664978 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f800b980 con 0x7fb9fc1596a0 2023-12-26T21:53:52.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba01199700 1 -- 172.21.15.131:0/915664978 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800bd20 con 0x7fb9fc1596a0 2023-12-26T21:53:52.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba043ff700 1 -- 172.21.15.131:0/915664978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 msgr2=0x7fb9fc0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:52.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba043ff700 1 --2- 172.21.15.131:0/915664978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 0x7fb9fc0b2380 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80099b0 tx=0x7fb9f8009250 comp rx=0 tx=0).stop 2023-12-26T21:53:52.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba043ff700 1 -- 172.21.15.131:0/915664978 shutdown_connections 2023-12-26T21:53:52.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba043ff700 1 --2- 172.21.15.131:0/915664978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc0b0870 0x7fb9fc0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba043ff700 1 --2- 172.21.15.131:0/915664978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 0x7fb9fc0b2380 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba043ff700 1 -- 172.21.15.131:0/915664978 >> 172.21.15.131:0/915664978 conn(0x7fb9fc06ece0 msgr2=0x7fb9fc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba043ff700 1 -- 172.21.15.131:0/915664978 shutdown_connections 2023-12-26T21:53:52.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.287+0000 7fba043ff700 1 -- 172.21.15.131:0/915664978 wait complete. 2023-12-26T21:53:52.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.289+0000 7fba043ff700 1 Processor -- start 2023-12-26T21:53:52.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.289+0000 7fba043ff700 1 -- start start 2023-12-26T21:53:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba043ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9fc0b0870 0x7fb9fc155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba043ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 0x7fb9fc0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba043ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc159a70 0x7fb9fc0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba043ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc16aae0 con 0x7fb9fc1596a0 2023-12-26T21:53:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba043ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc169ca0 con 0x7fb9fc0b0870 2023-12-26T21:53:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba043ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc169f10 con 0x7fb9fc159a70 2023-12-26T21:53:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba0199a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 0x7fb9fc0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:52.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba0199a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 0x7fb9fc0c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58852/0 (socket says 172.21.15.131:58852) 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba0219b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9fc0b0870 0x7fb9fc155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba0199a700 1 -- 172.21.15.131:0/2417707950 learned_addr learned my addr 172.21.15.131:0/2417707950 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.290+0000 7fba0299c700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc159a70 0x7fb9fc0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fba0199a700 1 -- 172.21.15.131:0/2417707950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc159a70 msgr2=0x7fb9fc0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fba0199a700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc159a70 0x7fb9fc0d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fba0199a700 1 -- 172.21.15.131:0/2417707950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9fc0b0870 msgr2=0x7fb9fc155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fba0199a700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9fc0b0870 0x7fb9fc155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fba0199a700 1 -- 172.21.15.131:0/2417707950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f80094f0 con 0x7fb9fc1596a0 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fba0199a700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 0x7fb9fc0c6770 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c5d0 tx=0x7fb9ec007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fb9f27fc700 1 -- 172.21.15.131:0/2417707950 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec011590 con 0x7fb9fc1596a0 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fb9f27fc700 1 -- 172.21.15.131:0/2417707950 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec011710 con 0x7fb9fc1596a0 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fb9f27fc700 1 -- 172.21.15.131:0/2417707950 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec011ad0 con 0x7fb9fc1596a0 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc16a1b0 con 0x7fb9fc1596a0 2023-12-26T21:53:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.291+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc16a770 con 0x7fb9fc1596a0 2023-12-26T21:53:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.292+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc16ac90 con 0x7fb9fc1596a0 2023-12-26T21:53:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.293+0000 7fb9f27fc700 1 -- 172.21.15.131:0/2417707950 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb9ec01c460 con 0x7fb9fc1596a0 2023-12-26T21:53:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.294+0000 7fb9f27fc700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb9e0066f30 0x7fb9e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.294+0000 7fb9f27fc700 1 -- 172.21.15.131:0/2417707950 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fb9ec0843e0 con 0x7fb9fc1596a0 2023-12-26T21:53:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.294+0000 7fba0219b700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb9e0066f30 0x7fb9e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.295+0000 7fba0219b700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb9e0066f30 0x7fb9e00693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80099b0 tx=0x7fb9f8016000 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:52.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.298+0000 7fb9f27fc700 1 -- 172.21.15.131:0/2417707950 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9ec051420 con 0x7fb9fc1596a0 2023-12-26T21:53:52.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.486+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9fc0657e0 con 0x7fb9e0066f30 2023-12-26T21:53:52.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.494+0000 7fb9f27fc700 1 -- 172.21.15.131:0/2417707950 <== mgr.6975 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289062 (secure 0 0 0) 0x7fb9fc0657e0 con 0x7fb9e0066f30 2023-12-26T21:53:52.499 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb9e0066f30 msgr2=0x7fb9e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb9e0066f30 0x7fb9e00693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80099b0 tx=0x7fb9f8016000 comp rx=0 tx=0).stop 2023-12-26T21:53:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 msgr2=0x7fb9fc0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 0x7fb9fc0c6770 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c5d0 tx=0x7fb9ec007f60 comp rx=0 tx=0).stop 2023-12-26T21:53:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 shutdown_connections 2023-12-26T21:53:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9fc159a70 0x7fb9fc0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9fc0b0870 0x7fb9fc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb9e0066f30 0x7fb9e00693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 --2- 172.21.15.131:0/2417707950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9fc1596a0 0x7fb9fc0c6770 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.498+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 >> 172.21.15.131:0/2417707950 conn(0x7fb9fc06ece0 msgr2=0x7fb9fc1567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.499+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 shutdown_connections 2023-12-26T21:53:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:52.499+0000 7fba043ff700 1 -- 172.21.15.131:0/2417707950 wait complete. 2023-12-26T21:53:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:52.517 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-26T21:53:52.112792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2451464,"kb_used_data":11720,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752523256,"statfs":{"total":773094113280,"available":770583814144,"internally_reserved":0,"allocated":12001280,"data_stored":6412780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.853406"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310309+0000","last_change":"2023-12-26T21:53:46.307184+0000","last_active":"2023-12-26T21:53:46.310309+0000","last_peered":"2023-12-26T21:53:46.310309+0000","last_clean":"2023-12-26T21:53:46.310309+0000","last_became_active":"2023-12-26T21:53:46.306640+0000","last_became_peered":"2023-12-26T21:53:46.306640+0000","last_unstale":"2023-12-26T21:53:46.310309+0000","last_undegraded":"2023-12-26T21:53:46.310309+0000","last_fullsized":"2023-12-26T21:53:46.310309+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:18.030160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306053+0000","last_change":"2023-12-26T21:53:46.299743+0000","last_active":"2023-12-26T21:53:46.306053+0000","last_peered":"2023-12-26T21:53:46.306053+0000","last_clean":"2023-12-26T21:53:46.306053+0000","last_became_active":"2023-12-26T21:53:46.299485+0000","last_became_peered":"2023-12-26T21:53:46.299485+0000","last_unstale":"2023-12-26T21:53:46.306053+0000","last_undegraded":"2023-12-26T21:53:46.306053+0000","last_fullsized":"2023-12-26T21:53:46.306053+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.316105+0000","last_change":"2023-12-26T21:53:46.308139+0000","last_active":"2023-12-26T21:53:46.316105+0000","last_peered":"2023-12-26T21:53:46.316105+0000","last_clean":"2023-12-26T21:53:46.316105+0000","last_became_active":"2023-12-26T21:53:46.302969+0000","last_became_peered":"2023-12-26T21:53:46.302969+0000","last_unstale":"2023-12-26T21:53:46.316105+0000","last_undegraded":"2023-12-26T21:53:46.316105+0000","last_fullsized":"2023-12-26T21:53:46.316105+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:40:03.180960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.309784+0000","last_change":"2023-12-26T21:53:46.305271+0000","last_active":"2023-12-26T21:53:46.309784+0000","last_peered":"2023-12-26T21:53:46.309784+0000","last_clean":"2023-12-26T21:53:46.309784+0000","last_became_active":"2023-12-26T21:53:46.304996+0000","last_became_peered":"2023-12-26T21:53:46.304996+0000","last_unstale":"2023-12-26T21:53:46.309784+0000","last_undegraded":"2023-12-26T21:53:46.309784+0000","last_fullsized":"2023-12-26T21:53:46.309784+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:55.354802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299767+0000","last_change":"2023-12-26T21:53:46.289973+0000","last_active":"2023-12-26T21:53:46.299767+0000","last_peered":"2023-12-26T21:53:46.299767+0000","last_clean":"2023-12-26T21:53:46.299767+0000","last_became_active":"2023-12-26T21:53:46.289695+0000","last_became_peered":"2023-12-26T21:53:46.289695+0000","last_unstale":"2023-12-26T21:53:46.299767+0000","last_undegraded":"2023-12-26T21:53:46.299767+0000","last_fullsized":"2023-12-26T21:53:46.299767+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:43:34.451389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311686+0000","last_change":"2023-12-26T21:53:46.304859+0000","last_active":"2023-12-26T21:53:46.311686+0000","last_peered":"2023-12-26T21:53:46.311686+0000","last_clean":"2023-12-26T21:53:46.311686+0000","last_became_active":"2023-12-26T21:53:46.304606+0000","last_became_peered":"2023-12-26T21:53:46.304606+0000","last_unstale":"2023-12-26T21:53:46.311686+0000","last_undegraded":"2023-12-26T21:53:46.311686+0000","last_fullsized":"2023-12-26T21:53:46.311686+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:19:50.873182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307583+0000","last_change":"2023-12-26T21:53:46.296559+0000","last_active":"2023-12-26T21:53:46.307583+0000","last_peered":"2023-12-26T21:53:46.307583+0000","last_clean":"2023-12-26T21:53:46.307583+0000","last_became_active":"2023-12-26T21:53:46.296237+0000","last_became_peered":"2023-12-26T21:53:46.296237+0000","last_unstale":"2023-12-26T21:53:46.307583+0000","last_undegraded":"2023-12-26T21:53:46.307583+0000","last_fullsized":"2023-12-26T21:53:46.307583+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.312831+0000","last_change":"2023-12-26T21:53:46.299618+0000","last_active":"2023-12-26T21:53:46.312831+0000","last_peered":"2023-12-26T21:53:46.312831+0000","last_clean":"2023-12-26T21:53:46.312831+0000","last_became_active":"2023-12-26T21:53:46.299320+0000","last_became_peered":"2023-12-26T21:53:46.299320+0000","last_unstale":"2023-12-26T21:53:46.312831+0000","last_undegraded":"2023-12-26T21:53:46.312831+0000","last_fullsized":"2023-12-26T21:53:46.312831+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:16.932912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308991+0000","last_change":"2023-12-26T21:53:46.303931+0000","last_active":"2023-12-26T21:53:46.308991+0000","last_peered":"2023-12-26T21:53:46.308991+0000","last_clean":"2023-12-26T21:53:46.308991+0000","last_became_active":"2023-12-26T21:53:46.303627+0000","last_became_peered":"2023-12-26T21:53:46.303627+0000","last_unstale":"2023-12-26T21:53:46.308991+0000","last_undegraded":"2023-12-26T21:53:46.308991+0000","last_fullsized":"2023-12-26T21:53:46.308991+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:28.427569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315982+0000","last_change":"2023-12-26T21:53:46.306473+0000","last_active":"2023-12-26T21:53:46.315982+0000","last_peered":"2023-12-26T21:53:46.315982+0000","last_clean":"2023-12-26T21:53:46.315982+0000","last_became_active":"2023-12-26T21:53:46.306137+0000","last_became_peered":"2023-12-26T21:53:46.306137+0000","last_unstale":"2023-12-26T21:53:46.315982+0000","last_undegraded":"2023-12-26T21:53:46.315982+0000","last_fullsized":"2023-12-26T21:53:46.315982+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:54:06.271609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.313313+0000","last_change":"2023-12-26T21:53:46.309492+0000","last_active":"2023-12-26T21:53:46.313313+0000","last_peered":"2023-12-26T21:53:46.313313+0000","last_clean":"2023-12-26T21:53:46.313313+0000","last_became_active":"2023-12-26T21:53:46.309200+0000","last_became_peered":"2023-12-26T21:53:46.309200+0000","last_unstale":"2023-12-26T21:53:46.313313+0000","last_undegraded":"2023-12-26T21:53:46.313313+0000","last_fullsized":"2023-12-26T21:53:46.313313+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:28.847137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314884+0000","last_change":"2023-12-26T21:53:46.301794+0000","last_active":"2023-12-26T21:53:46.314884+0000","last_peered":"2023-12-26T21:53:46.314884+0000","last_clean":"2023-12-26T21:53:46.314884+0000","last_became_active":"2023-12-26T21:53:46.298746+0000","last_became_peered":"2023-12-26T21:53:46.298746+0000","last_unstale":"2023-12-26T21:53:46.314884+0000","last_undegraded":"2023-12-26T21:53:46.314884+0000","last_fullsized":"2023-12-26T21:53:46.314884+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:26:29.931919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314549+0000","last_change":"2023-12-26T21:53:46.308209+0000","last_active":"2023-12-26T21:53:46.314549+0000","last_peered":"2023-12-26T21:53:46.314549+0000","last_clean":"2023-12-26T21:53:46.314549+0000","last_became_active":"2023-12-26T21:53:46.308025+0000","last_became_peered":"2023-12-26T21:53:46.308025+0000","last_unstale":"2023-12-26T21:53:46.314549+0000","last_undegraded":"2023-12-26T21:53:46.314549+0000","last_fullsized":"2023-12-26T21:53:46.314549+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:53:11.039774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311289+0000","last_change":"2023-12-26T21:53:46.303517+0000","last_active":"2023-12-26T21:53:46.311289+0000","last_peered":"2023-12-26T21:53:46.311289+0000","last_clean":"2023-12-26T21:53:46.311289+0000","last_became_active":"2023-12-26T21:53:46.303137+0000","last_became_peered":"2023-12-26T21:53:46.303137+0000","last_unstale":"2023-12-26T21:53:46.311289+0000","last_undegraded":"2023-12-26T21:53:46.311289+0000","last_fullsized":"2023-12-26T21:53:46.311289+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314209+0000","last_change":"2023-12-26T21:53:46.308456+0000","last_active":"2023-12-26T21:53:46.314209+0000","last_peered":"2023-12-26T21:53:46.314209+0000","last_clean":"2023-12-26T21:53:46.314209+0000","last_became_active":"2023-12-26T21:53:46.308203+0000","last_became_peered":"2023-12-26T21:53:46.308203+0000","last_unstale":"2023-12-26T21:53:46.314209+0000","last_undegraded":"2023-12-26T21:53:46.314209+0000","last_fullsized":"2023-12-26T21:53:46.314209+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315918+0000","last_change":"2023-12-26T21:53:46.306283+0000","last_active":"2023-12-26T21:53:46.315918+0000","last_peered":"2023-12-26T21:53:46.315918+0000","last_clean":"2023-12-26T21:53:46.315918+0000","last_became_active":"2023-12-26T21:53:46.305800+0000","last_became_peered":"2023-12-26T21:53:46.305800+0000","last_unstale":"2023-12-26T21:53:46.315918+0000","last_undegraded":"2023-12-26T21:53:46.315918+0000","last_fullsized":"2023-12-26T21:53:46.315918+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:09:24.176979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314753+0000","last_change":"2023-12-26T21:53:46.300055+0000","last_active":"2023-12-26T21:53:46.314753+0000","last_peered":"2023-12-26T21:53:46.314753+0000","last_clean":"2023-12-26T21:53:46.314753+0000","last_became_active":"2023-12-26T21:53:46.298259+0000","last_became_peered":"2023-12-26T21:53:46.298259+0000","last_unstale":"2023-12-26T21:53:46.314753+0000","last_undegraded":"2023-12-26T21:53:46.314753+0000","last_fullsized":"2023-12-26T21:53:46.314753+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:55:34.210900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308204+0000","last_change":"2023-12-26T21:53:46.302662+0000","last_active":"2023-12-26T21:53:46.308204+0000","last_peered":"2023-12-26T21:53:46.308204+0000","last_clean":"2023-12-26T21:53:46.308204+0000","last_became_active":"2023-12-26T21:53:46.302401+0000","last_became_peered":"2023-12-26T21:53:46.302401+0000","last_unstale":"2023-12-26T21:53:46.308204+0000","last_undegraded":"2023-12-26T21:53:46.308204+0000","last_fullsized":"2023-12-26T21:53:46.308204+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:20:45.804162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311951+0000","last_change":"2023-12-26T21:53:46.305482+0000","last_active":"2023-12-26T21:53:46.311951+0000","last_peered":"2023-12-26T21:53:46.311951+0000","last_clean":"2023-12-26T21:53:46.311951+0000","last_became_active":"2023-12-26T21:53:46.305210+0000","last_became_peered":"2023-12-26T21:53:46.305210+0000","last_unstale":"2023-12-26T21:53:46.311951+0000","last_undegraded":"2023-12-26T21:53:46.311951+0000","last_fullsized":"2023-12-26T21:53:46.311951+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306262+0000","last_change":"2023-12-26T21:53:46.301077+0000","last_active":"2023-12-26T21:53:46.306262+0000","last_peered":"2023-12-26T21:53:46.306262+0000","last_clean":"2023-12-26T21:53:46.306262+0000","last_became_active":"2023-12-26T21:53:46.300762+0000","last_became_peered":"2023-12-26T21:53:46.300762+0000","last_unstale":"2023-12-26T21:53:46.306262+0000","last_undegraded":"2023-12-26T21:53:46.306262+0000","last_fullsized":"2023-12-26T21:53:46.306262+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311637+0000","last_change":"2023-12-26T21:53:46.306696+0000","last_active":"2023-12-26T21:53:46.311637+0000","last_peered":"2023-12-26T21:53:46.311637+0000","last_clean":"2023-12-26T21:53:46.311637+0000","last_became_active":"2023-12-26T21:53:46.306420+0000","last_became_peered":"2023-12-26T21:53:46.306420+0000","last_unstale":"2023-12-26T21:53:46.311637+0000","last_undegraded":"2023-12-26T21:53:46.311637+0000","last_fullsized":"2023-12-26T21:53:46.311637+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:31.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289642+0000","last_change":"2023-12-26T21:53:46.287813+0000","last_active":"2023-12-26T21:53:46.289642+0000","last_peered":"2023-12-26T21:53:46.289642+0000","last_clean":"2023-12-26T21:53:46.289642+0000","last_became_active":"2023-12-26T21:53:46.287544+0000","last_became_peered":"2023-12-26T21:53:46.287544+0000","last_unstale":"2023-12-26T21:53:46.289642+0000","last_undegraded":"2023-12-26T21:53:46.289642+0000","last_fullsized":"2023-12-26T21:53:46.289642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:27.394417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314855+0000","last_change":"2023-12-26T21:53:46.303413+0000","last_active":"2023-12-26T21:53:46.314855+0000","last_peered":"2023-12-26T21:53:46.314855+0000","last_clean":"2023-12-26T21:53:46.314855+0000","last_became_active":"2023-12-26T21:53:46.299017+0000","last_became_peered":"2023-12-26T21:53:46.299017+0000","last_unstale":"2023-12-26T21:53:46.314855+0000","last_undegraded":"2023-12-26T21:53:46.314855+0000","last_fullsized":"2023-12-26T21:53:46.314855+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:23:40.412951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.309404+0000","last_change":"2023-12-26T21:53:46.305600+0000","last_active":"2023-12-26T21:53:46.309404+0000","last_peered":"2023-12-26T21:53:46.309404+0000","last_clean":"2023-12-26T21:53:46.309404+0000","last_became_active":"2023-12-26T21:53:46.305279+0000","last_became_peered":"2023-12-26T21:53:46.305279+0000","last_unstale":"2023-12-26T21:53:46.309404+0000","last_undegraded":"2023-12-26T21:53:46.309404+0000","last_fullsized":"2023-12-26T21:53:46.309404+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:00:42.816954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308838+0000","last_change":"2023-12-26T21:53:46.306183+0000","last_active":"2023-12-26T21:53:46.308838+0000","last_peered":"2023-12-26T21:53:46.308838+0000","last_clean":"2023-12-26T21:53:46.308838+0000","last_became_active":"2023-12-26T21:53:46.304494+0000","last_became_peered":"2023-12-26T21:53:46.304494+0000","last_unstale":"2023-12-26T21:53:46.308838+0000","last_undegraded":"2023-12-26T21:53:46.308838+0000","last_fullsized":"2023-12-26T21:53:46.308838+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:50:21.227281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.297090+0000","last_change":"2023-12-26T21:53:46.287979+0000","last_active":"2023-12-26T21:53:46.297090+0000","last_peered":"2023-12-26T21:53:46.297090+0000","last_clean":"2023-12-26T21:53:46.297090+0000","last_became_active":"2023-12-26T21:53:46.287682+0000","last_became_peered":"2023-12-26T21:53:46.287682+0000","last_unstale":"2023-12-26T21:53:46.297090+0000","last_undegraded":"2023-12-26T21:53:46.297090+0000","last_fullsized":"2023-12-26T21:53:46.297090+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:38:29.898590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311484+0000","last_change":"2023-12-26T21:53:46.306111+0000","last_active":"2023-12-26T21:53:46.311484+0000","last_peered":"2023-12-26T21:53:46.311484+0000","last_clean":"2023-12-26T21:53:46.311484+0000","last_became_active":"2023-12-26T21:53:46.305852+0000","last_became_peered":"2023-12-26T21:53:46.305852+0000","last_unstale":"2023-12-26T21:53:46.311484+0000","last_undegraded":"2023-12-26T21:53:46.311484+0000","last_fullsized":"2023-12-26T21:53:46.311484+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:03:27.957388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311050+0000","last_change":"2023-12-26T21:53:46.305148+0000","last_active":"2023-12-26T21:53:46.311050+0000","last_peered":"2023-12-26T21:53:46.311050+0000","last_clean":"2023-12-26T21:53:46.311050+0000","last_became_active":"2023-12-26T21:53:46.304893+0000","last_became_peered":"2023-12-26T21:53:46.304893+0000","last_unstale":"2023-12-26T21:53:46.311050+0000","last_undegraded":"2023-12-26T21:53:46.311050+0000","last_fullsized":"2023-12-26T21:53:46.311050+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:05.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311656+0000","last_change":"2023-12-26T21:53:46.306908+0000","last_active":"2023-12-26T21:53:46.311656+0000","last_peered":"2023-12-26T21:53:46.311656+0000","last_clean":"2023-12-26T21:53:46.311656+0000","last_became_active":"2023-12-26T21:53:46.306331+0000","last_became_peered":"2023-12-26T21:53:46.306331+0000","last_unstale":"2023-12-26T21:53:46.311656+0000","last_undegraded":"2023-12-26T21:53:46.311656+0000","last_fullsized":"2023-12-26T21:53:46.311656+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:50:40.961352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307642+0000","last_change":"2023-12-26T21:53:46.301677+0000","last_active":"2023-12-26T21:53:46.307642+0000","last_peered":"2023-12-26T21:53:46.307642+0000","last_clean":"2023-12-26T21:53:46.307642+0000","last_became_active":"2023-12-26T21:53:46.301432+0000","last_became_peered":"2023-12-26T21:53:46.301432+0000","last_unstale":"2023-12-26T21:53:46.307642+0000","last_undegraded":"2023-12-26T21:53:46.307642+0000","last_fullsized":"2023-12-26T21:53:46.307642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.317281+0000","last_change":"2023-12-26T21:53:46.313093+0000","last_active":"2023-12-26T21:53:46.317281+0000","last_peered":"2023-12-26T21:53:46.317281+0000","last_clean":"2023-12-26T21:53:46.317281+0000","last_became_active":"2023-12-26T21:53:46.312678+0000","last_became_peered":"2023-12-26T21:53:46.312678+0000","last_unstale":"2023-12-26T21:53:46.317281+0000","last_undegraded":"2023-12-26T21:53:46.317281+0000","last_fullsized":"2023-12-26T21:53:46.317281+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:42.880417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315827+0000","last_change":"2023-12-26T21:53:46.306101+0000","last_active":"2023-12-26T21:53:46.315827+0000","last_peered":"2023-12-26T21:53:46.315827+0000","last_clean":"2023-12-26T21:53:46.315827+0000","last_became_active":"2023-12-26T21:53:46.305481+0000","last_became_peered":"2023-12-26T21:53:46.305481+0000","last_unstale":"2023-12-26T21:53:46.315827+0000","last_undegraded":"2023-12-26T21:53:46.315827+0000","last_fullsized":"2023-12-26T21:53:46.315827+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:39:06.014521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314881+0000","last_change":"2023-12-26T21:53:46.302506+0000","last_active":"2023-12-26T21:53:46.314881+0000","last_peered":"2023-12-26T21:53:46.314881+0000","last_clean":"2023-12-26T21:53:46.314881+0000","last_became_active":"2023-12-26T21:53:46.301746+0000","last_became_peered":"2023-12-26T21:53:46.301746+0000","last_unstale":"2023-12-26T21:53:46.314881+0000","last_undegraded":"2023-12-26T21:53:46.314881+0000","last_fullsized":"2023-12-26T21:53:46.314881+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:55.354802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"109'4","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321799+0000","last_change":"2023-12-26T21:53:45.285708+0000","last_active":"2023-12-26T21:53:46.321799+0000","last_peered":"2023-12-26T21:53:46.321799+0000","last_clean":"2023-12-26T21:53:46.321799+0000","last_became_active":"2023-12-26T21:53:45.285466+0000","last_became_peered":"2023-12-26T21:53:45.285466+0000","last_unstale":"2023-12-26T21:53:46.321799+0000","last_undegraded":"2023-12-26T21:53:46.321799+0000","last_fullsized":"2023-12-26T21:53:46.321799+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:11.158488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310328+0000","last_change":"2023-12-26T21:53:46.307223+0000","last_active":"2023-12-26T21:53:46.310328+0000","last_peered":"2023-12-26T21:53:46.310328+0000","last_clean":"2023-12-26T21:53:46.310328+0000","last_became_active":"2023-12-26T21:53:46.306963+0000","last_became_peered":"2023-12-26T21:53:46.306963+0000","last_unstale":"2023-12-26T21:53:46.310328+0000","last_undegraded":"2023-12-26T21:53:46.310328+0000","last_fullsized":"2023-12-26T21:53:46.310328+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:12:37.634766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321262+0000","last_change":"2023-12-26T21:53:45.273356+0000","last_active":"2023-12-26T21:53:46.321262+0000","last_peered":"2023-12-26T21:53:46.321262+0000","last_clean":"2023-12-26T21:53:46.321262+0000","last_became_active":"2023-12-26T21:53:45.273096+0000","last_became_peered":"2023-12-26T21:53:45.273096+0000","last_unstale":"2023-12-26T21:53:46.321262+0000","last_undegraded":"2023-12-26T21:53:46.321262+0000","last_fullsized":"2023-12-26T21:53:46.321262+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:26.388651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314910+0000","last_change":"2023-12-26T21:53:46.303618+0000","last_active":"2023-12-26T21:53:46.314910+0000","last_peered":"2023-12-26T21:53:46.314910+0000","last_clean":"2023-12-26T21:53:46.314910+0000","last_became_active":"2023-12-26T21:53:46.301521+0000","last_became_peered":"2023-12-26T21:53:46.301521+0000","last_unstale":"2023-12-26T21:53:46.314910+0000","last_undegraded":"2023-12-26T21:53:46.314910+0000","last_fullsized":"2023-12-26T21:53:46.314910+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:01:53.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.318512+0000","last_change":"2023-12-26T21:53:45.287570+0000","last_active":"2023-12-26T21:53:46.318512+0000","last_peered":"2023-12-26T21:53:46.318512+0000","last_clean":"2023-12-26T21:53:46.318512+0000","last_became_active":"2023-12-26T21:53:45.287333+0000","last_became_peered":"2023-12-26T21:53:45.287333+0000","last_unstale":"2023-12-26T21:53:46.318512+0000","last_undegraded":"2023-12-26T21:53:46.318512+0000","last_fullsized":"2023-12-26T21:53:46.318512+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:33.642526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.305631+0000","last_change":"2023-12-26T21:53:46.299487+0000","last_active":"2023-12-26T21:53:46.305631+0000","last_peered":"2023-12-26T21:53:46.305631+0000","last_clean":"2023-12-26T21:53:46.305631+0000","last_became_active":"2023-12-26T21:53:46.299063+0000","last_became_peered":"2023-12-26T21:53:46.299063+0000","last_unstale":"2023-12-26T21:53:46.305631+0000","last_undegraded":"2023-12-26T21:53:46.305631+0000","last_fullsized":"2023-12-26T21:53:46.305631+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:48.449758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.318647+0000","last_change":"2023-12-26T21:53:45.283310+0000","last_active":"2023-12-26T21:53:46.318647+0000","last_peered":"2023-12-26T21:53:46.318647+0000","last_clean":"2023-12-26T21:53:46.318647+0000","last_became_active":"2023-12-26T21:53:45.282931+0000","last_became_peered":"2023-12-26T21:53:45.282931+0000","last_unstale":"2023-12-26T21:53:46.318647+0000","last_undegraded":"2023-12-26T21:53:46.318647+0000","last_fullsized":"2023-12-26T21:53:46.318647+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:44:34.249955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306117+0000","last_change":"2023-12-26T21:53:46.302615+0000","last_active":"2023-12-26T21:53:46.306117+0000","last_peered":"2023-12-26T21:53:46.306117+0000","last_clean":"2023-12-26T21:53:46.306117+0000","last_became_active":"2023-12-26T21:53:46.302098+0000","last_became_peered":"2023-12-26T21:53:46.302098+0000","last_unstale":"2023-12-26T21:53:46.306117+0000","last_undegraded":"2023-12-26T21:53:46.306117+0000","last_fullsized":"2023-12-26T21:53:46.306117+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:01:53.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315670+0000","last_change":"2023-12-26T21:53:45.283590+0000","last_active":"2023-12-26T21:53:46.315670+0000","last_peered":"2023-12-26T21:53:46.315670+0000","last_clean":"2023-12-26T21:53:46.315670+0000","last_became_active":"2023-12-26T21:53:45.282825+0000","last_became_peered":"2023-12-26T21:53:45.282825+0000","last_unstale":"2023-12-26T21:53:46.315670+0000","last_undegraded":"2023-12-26T21:53:46.315670+0000","last_fullsized":"2023-12-26T21:53:46.315670+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:27:44.133796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306503+0000","last_change":"2023-12-26T21:53:46.297813+0000","last_active":"2023-12-26T21:53:46.306503+0000","last_peered":"2023-12-26T21:53:46.306503+0000","last_clean":"2023-12-26T21:53:46.306503+0000","last_became_active":"2023-12-26T21:53:46.297448+0000","last_became_peered":"2023-12-26T21:53:46.297448+0000","last_unstale":"2023-12-26T21:53:46.306503+0000","last_undegraded":"2023-12-26T21:53:46.306503+0000","last_fullsized":"2023-12-26T21:53:46.306503+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:13:57.688004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.319450+0000","last_change":"2023-12-26T21:53:45.275507+0000","last_active":"2023-12-26T21:53:46.319450+0000","last_peered":"2023-12-26T21:53:46.319450+0000","last_clean":"2023-12-26T21:53:46.319450+0000","last_became_active":"2023-12-26T21:53:45.275041+0000","last_became_peered":"2023-12-26T21:53:45.275041+0000","last_unstale":"2023-12-26T21:53:46.319450+0000","last_undegraded":"2023-12-26T21:53:46.319450+0000","last_fullsized":"2023-12-26T21:53:46.319450+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:52:49.674188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299738+0000","last_change":"2023-12-26T21:53:46.297785+0000","last_active":"2023-12-26T21:53:46.299738+0000","last_peered":"2023-12-26T21:53:46.299738+0000","last_clean":"2023-12-26T21:53:46.299738+0000","last_became_active":"2023-12-26T21:53:46.297505+0000","last_became_peered":"2023-12-26T21:53:46.297505+0000","last_unstale":"2023-12-26T21:53:46.299738+0000","last_undegraded":"2023-12-26T21:53:46.299738+0000","last_fullsized":"2023-12-26T21:53:46.299738+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:33.304213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281100+0000","last_change":"2023-12-26T21:53:45.279536+0000","last_active":"2023-12-26T21:53:46.281100+0000","last_peered":"2023-12-26T21:53:46.281100+0000","last_clean":"2023-12-26T21:53:46.281100+0000","last_became_active":"2023-12-26T21:53:45.279282+0000","last_became_peered":"2023-12-26T21:53:45.279282+0000","last_unstale":"2023-12-26T21:53:46.281100+0000","last_undegraded":"2023-12-26T21:53:46.281100+0000","last_fullsized":"2023-12-26T21:53:46.281100+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:58.794519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.288617+0000","last_change":"2023-12-26T21:53:46.285357+0000","last_active":"2023-12-26T21:53:46.288617+0000","last_peered":"2023-12-26T21:53:46.288617+0000","last_clean":"2023-12-26T21:53:46.288617+0000","last_became_active":"2023-12-26T21:53:46.285051+0000","last_became_peered":"2023-12-26T21:53:46.285051+0000","last_unstale":"2023-12-26T21:53:46.288617+0000","last_undegraded":"2023-12-26T21:53:46.288617+0000","last_fullsized":"2023-12-26T21:53:46.288617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:31:16.372604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281226+0000","last_change":"2023-12-26T21:53:45.268538+0000","last_active":"2023-12-26T21:53:46.281226+0000","last_peered":"2023-12-26T21:53:46.281226+0000","last_clean":"2023-12-26T21:53:46.281226+0000","last_became_active":"2023-12-26T21:53:45.268069+0000","last_became_peered":"2023-12-26T21:53:45.268069+0000","last_unstale":"2023-12-26T21:53:46.281226+0000","last_undegraded":"2023-12-26T21:53:46.281226+0000","last_fullsized":"2023-12-26T21:53:46.281226+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:25.546517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300878+0000","last_change":"2023-12-26T21:53:46.297819+0000","last_active":"2023-12-26T21:53:46.300878+0000","last_peered":"2023-12-26T21:53:46.300878+0000","last_clean":"2023-12-26T21:53:46.300878+0000","last_became_active":"2023-12-26T21:53:46.297541+0000","last_became_peered":"2023-12-26T21:53:46.297541+0000","last_unstale":"2023-12-26T21:53:46.300878+0000","last_undegraded":"2023-12-26T21:53:46.300878+0000","last_fullsized":"2023-12-26T21:53:46.300878+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:10:09.104721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289148+0000","last_change":"2023-12-26T21:53:45.281269+0000","last_active":"2023-12-26T21:53:46.289148+0000","last_peered":"2023-12-26T21:53:46.289148+0000","last_clean":"2023-12-26T21:53:46.289148+0000","last_became_active":"2023-12-26T21:53:45.280967+0000","last_became_peered":"2023-12-26T21:53:45.280967+0000","last_unstale":"2023-12-26T21:53:46.289148+0000","last_undegraded":"2023-12-26T21:53:46.289148+0000","last_fullsized":"2023-12-26T21:53:46.289148+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:20:00.172583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299929+0000","last_change":"2023-12-26T21:53:46.298319+0000","last_active":"2023-12-26T21:53:46.299929+0000","last_peered":"2023-12-26T21:53:46.299929+0000","last_clean":"2023-12-26T21:53:46.299929+0000","last_became_active":"2023-12-26T21:53:46.298050+0000","last_became_peered":"2023-12-26T21:53:46.298050+0000","last_unstale":"2023-12-26T21:53:46.299929+0000","last_undegraded":"2023-12-26T21:53:46.299929+0000","last_fullsized":"2023-12-26T21:53:46.299929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:34.648293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289970+0000","last_change":"2023-12-26T21:53:45.279444+0000","last_active":"2023-12-26T21:53:46.289970+0000","last_peered":"2023-12-26T21:53:46.289970+0000","last_clean":"2023-12-26T21:53:46.289970+0000","last_became_active":"2023-12-26T21:53:45.279155+0000","last_became_peered":"2023-12-26T21:53:45.279155+0000","last_unstale":"2023-12-26T21:53:46.289970+0000","last_undegraded":"2023-12-26T21:53:46.289970+0000","last_fullsized":"2023-12-26T21:53:46.289970+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:30:46.434273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.313332+0000","last_change":"2023-12-26T21:53:46.310040+0000","last_active":"2023-12-26T21:53:46.313332+0000","last_peered":"2023-12-26T21:53:46.313332+0000","last_clean":"2023-12-26T21:53:46.313332+0000","last_became_active":"2023-12-26T21:53:46.309454+0000","last_became_peered":"2023-12-26T21:53:46.309454+0000","last_unstale":"2023-12-26T21:53:46.313332+0000","last_undegraded":"2023-12-26T21:53:46.313332+0000","last_fullsized":"2023-12-26T21:53:46.313332+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:57:08.838429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"109'3","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321640+0000","last_change":"2023-12-26T21:53:45.285950+0000","last_active":"2023-12-26T21:53:46.321640+0000","last_peered":"2023-12-26T21:53:46.321640+0000","last_clean":"2023-12-26T21:53:46.321640+0000","last_became_active":"2023-12-26T21:53:45.285657+0000","last_became_peered":"2023-12-26T21:53:45.285657+0000","last_unstale":"2023-12-26T21:53:46.321640+0000","last_undegraded":"2023-12-26T21:53:46.321640+0000","last_fullsized":"2023-12-26T21:53:46.321640+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:55.530687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303089+0000","last_change":"2023-12-26T21:53:46.301426+0000","last_active":"2023-12-26T21:53:46.303089+0000","last_peered":"2023-12-26T21:53:46.303089+0000","last_clean":"2023-12-26T21:53:46.303089+0000","last_became_active":"2023-12-26T21:53:46.301116+0000","last_became_peered":"2023-12-26T21:53:46.301116+0000","last_unstale":"2023-12-26T21:53:46.303089+0000","last_undegraded":"2023-12-26T21:53:46.303089+0000","last_fullsized":"2023-12-26T21:53:46.303089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:49.402467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"109'6","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321852+0000","last_change":"2023-12-26T21:53:45.281131+0000","last_active":"2023-12-26T21:53:46.321852+0000","last_peered":"2023-12-26T21:53:46.321852+0000","last_clean":"2023-12-26T21:53:46.321852+0000","last_became_active":"2023-12-26T21:53:45.280843+0000","last_became_peered":"2023-12-26T21:53:45.280843+0000","last_unstale":"2023-12-26T21:53:46.321852+0000","last_undegraded":"2023-12-26T21:53:46.321852+0000","last_fullsized":"2023-12-26T21:53:46.321852+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:45:10.491412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307228+0000","last_change":"2023-12-26T21:53:46.302146+0000","last_active":"2023-12-26T21:53:46.307228+0000","last_peered":"2023-12-26T21:53:46.307228+0000","last_clean":"2023-12-26T21:53:46.307228+0000","last_became_active":"2023-12-26T21:53:46.301846+0000","last_became_peered":"2023-12-26T21:53:46.301846+0000","last_unstale":"2023-12-26T21:53:46.307228+0000","last_undegraded":"2023-12-26T21:53:46.307228+0000","last_fullsized":"2023-12-26T21:53:46.307228+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:57:09.425627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.319761+0000","last_change":"2023-12-26T21:53:45.277993+0000","last_active":"2023-12-26T21:53:46.319761+0000","last_peered":"2023-12-26T21:53:46.319761+0000","last_clean":"2023-12-26T21:53:46.319761+0000","last_became_active":"2023-12-26T21:53:45.276119+0000","last_became_peered":"2023-12-26T21:53:45.276119+0000","last_unstale":"2023-12-26T21:53:46.319761+0000","last_undegraded":"2023-12-26T21:53:46.319761+0000","last_fullsized":"2023-12-26T21:53:46.319761+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:49.247010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304318+0000","last_change":"2023-12-26T21:53:46.298424+0000","last_active":"2023-12-26T21:53:46.304318+0000","last_peered":"2023-12-26T21:53:46.304318+0000","last_clean":"2023-12-26T21:53:46.304318+0000","last_became_active":"2023-12-26T21:53:46.298145+0000","last_became_peered":"2023-12-26T21:53:46.298145+0000","last_unstale":"2023-12-26T21:53:46.304318+0000","last_undegraded":"2023-12-26T21:53:46.304318+0000","last_fullsized":"2023-12-26T21:53:46.304318+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:24:04.826615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285162+0000","last_change":"2023-12-26T21:53:45.281444+0000","last_active":"2023-12-26T21:53:46.285162+0000","last_peered":"2023-12-26T21:53:46.285162+0000","last_clean":"2023-12-26T21:53:46.285162+0000","last_became_active":"2023-12-26T21:53:45.280420+0000","last_became_peered":"2023-12-26T21:53:45.280420+0000","last_unstale":"2023-12-26T21:53:46.285162+0000","last_undegraded":"2023-12-26T21:53:46.285162+0000","last_fullsized":"2023-12-26T21:53:46.285162+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:17.475634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300883+0000","last_change":"2023-12-26T21:53:46.297808+0000","last_active":"2023-12-26T21:53:46.300883+0000","last_peered":"2023-12-26T21:53:46.300883+0000","last_clean":"2023-12-26T21:53:46.300883+0000","last_became_active":"2023-12-26T21:53:46.296873+0000","last_became_peered":"2023-12-26T21:53:46.296873+0000","last_unstale":"2023-12-26T21:53:46.300883+0000","last_undegraded":"2023-12-26T21:53:46.300883+0000","last_fullsized":"2023-12-26T21:53:46.300883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:29.573501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286405+0000","last_change":"2023-12-26T21:53:45.283894+0000","last_active":"2023-12-26T21:53:46.286405+0000","last_peered":"2023-12-26T21:53:46.286405+0000","last_clean":"2023-12-26T21:53:46.286405+0000","last_became_active":"2023-12-26T21:53:45.283609+0000","last_became_peered":"2023-12-26T21:53:45.283609+0000","last_unstale":"2023-12-26T21:53:46.286405+0000","last_undegraded":"2023-12-26T21:53:46.286405+0000","last_fullsized":"2023-12-26T21:53:46.286405+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:15.433996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.313371+0000","last_change":"2023-12-26T21:53:46.309163+0000","last_active":"2023-12-26T21:53:46.313371+0000","last_peered":"2023-12-26T21:53:46.313371+0000","last_clean":"2023-12-26T21:53:46.313371+0000","last_became_active":"2023-12-26T21:53:46.308834+0000","last_became_peered":"2023-12-26T21:53:46.308834+0000","last_unstale":"2023-12-26T21:53:46.313371+0000","last_undegraded":"2023-12-26T21:53:46.313371+0000","last_fullsized":"2023-12-26T21:53:46.313371+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:23.271735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"109'8","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321108+0000","last_change":"2023-12-26T21:53:45.282550+0000","last_active":"2023-12-26T21:53:46.321108+0000","last_peered":"2023-12-26T21:53:46.321108+0000","last_clean":"2023-12-26T21:53:46.321108+0000","last_became_active":"2023-12-26T21:53:45.282289+0000","last_became_peered":"2023-12-26T21:53:45.282289+0000","last_unstale":"2023-12-26T21:53:46.321108+0000","last_undegraded":"2023-12-26T21:53:46.321108+0000","last_fullsized":"2023-12-26T21:53:46.321108+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:04.117197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314843+0000","last_change":"2023-12-26T21:53:46.299765+0000","last_active":"2023-12-26T21:53:46.314843+0000","last_peered":"2023-12-26T21:53:46.314843+0000","last_clean":"2023-12-26T21:53:46.314843+0000","last_became_active":"2023-12-26T21:53:46.299465+0000","last_became_peered":"2023-12-26T21:53:46.299465+0000","last_unstale":"2023-12-26T21:53:46.314843+0000","last_undegraded":"2023-12-26T21:53:46.314843+0000","last_fullsized":"2023-12-26T21:53:46.314843+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:50:40.961352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.320094+0000","last_change":"2023-12-26T21:53:45.280067+0000","last_active":"2023-12-26T21:53:46.320094+0000","last_peered":"2023-12-26T21:53:46.320094+0000","last_clean":"2023-12-26T21:53:46.320094+0000","last_became_active":"2023-12-26T21:53:45.279713+0000","last_became_peered":"2023-12-26T21:53:45.279713+0000","last_unstale":"2023-12-26T21:53:46.320094+0000","last_undegraded":"2023-12-26T21:53:46.320094+0000","last_fullsized":"2023-12-26T21:53:46.320094+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:15.927146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.316148+0000","last_change":"2023-12-26T21:53:46.307450+0000","last_active":"2023-12-26T21:53:46.316148+0000","last_peered":"2023-12-26T21:53:46.316148+0000","last_clean":"2023-12-26T21:53:46.316148+0000","last_became_active":"2023-12-26T21:53:46.307168+0000","last_became_peered":"2023-12-26T21:53:46.307168+0000","last_unstale":"2023-12-26T21:53:46.316148+0000","last_undegraded":"2023-12-26T21:53:46.316148+0000","last_fullsized":"2023-12-26T21:53:46.316148+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:46.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.316477+0000","last_change":"2023-12-26T21:53:45.277896+0000","last_active":"2023-12-26T21:53:46.316477+0000","last_peered":"2023-12-26T21:53:46.316477+0000","last_clean":"2023-12-26T21:53:46.316477+0000","last_became_active":"2023-12-26T21:53:45.277584+0000","last_became_peered":"2023-12-26T21:53:45.277584+0000","last_unstale":"2023-12-26T21:53:46.316477+0000","last_undegraded":"2023-12-26T21:53:46.316477+0000","last_fullsized":"2023-12-26T21:53:46.316477+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:44:17.169190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303450+0000","last_change":"2023-12-26T21:53:46.298211+0000","last_active":"2023-12-26T21:53:46.303450+0000","last_peered":"2023-12-26T21:53:46.303450+0000","last_clean":"2023-12-26T21:53:46.303450+0000","last_became_active":"2023-12-26T21:53:46.297939+0000","last_became_peered":"2023-12-26T21:53:46.297939+0000","last_unstale":"2023-12-26T21:53:46.303450+0000","last_undegraded":"2023-12-26T21:53:46.303450+0000","last_fullsized":"2023-12-26T21:53:46.303450+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:31.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.282526+0000","last_change":"2023-12-26T21:53:45.280486+0000","last_active":"2023-12-26T21:53:46.282526+0000","last_peered":"2023-12-26T21:53:46.282526+0000","last_clean":"2023-12-26T21:53:46.282526+0000","last_became_active":"2023-12-26T21:53:45.280178+0000","last_became_peered":"2023-12-26T21:53:45.280178+0000","last_unstale":"2023-12-26T21:53:46.282526+0000","last_undegraded":"2023-12-26T21:53:46.282526+0000","last_fullsized":"2023-12-26T21:53:46.282526+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:13:56.557753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304197+0000","last_change":"2023-12-26T21:53:46.297254+0000","last_active":"2023-12-26T21:53:46.304197+0000","last_peered":"2023-12-26T21:53:46.304197+0000","last_clean":"2023-12-26T21:53:46.304197+0000","last_became_active":"2023-12-26T21:53:46.296358+0000","last_became_peered":"2023-12-26T21:53:46.296358+0000","last_unstale":"2023-12-26T21:53:46.304197+0000","last_undegraded":"2023-12-26T21:53:46.304197+0000","last_fullsized":"2023-12-26T21:53:46.304197+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.317134+0000","last_change":"2023-12-26T21:53:45.271721+0000","last_active":"2023-12-26T21:53:46.317134+0000","last_peered":"2023-12-26T21:53:46.317134+0000","last_clean":"2023-12-26T21:53:46.317134+0000","last_became_active":"2023-12-26T21:53:45.271453+0000","last_became_peered":"2023-12-26T21:53:45.271453+0000","last_unstale":"2023-12-26T21:53:46.317134+0000","last_undegraded":"2023-12-26T21:53:46.317134+0000","last_fullsized":"2023-12-26T21:53:46.317134+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:49:24.818172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303057+0000","last_change":"2023-12-26T21:53:46.301435+0000","last_active":"2023-12-26T21:53:46.303057+0000","last_peered":"2023-12-26T21:53:46.303057+0000","last_clean":"2023-12-26T21:53:46.303057+0000","last_became_active":"2023-12-26T21:53:46.301117+0000","last_became_peered":"2023-12-26T21:53:46.301117+0000","last_unstale":"2023-12-26T21:53:46.303057+0000","last_undegraded":"2023-12-26T21:53:46.303057+0000","last_fullsized":"2023-12-26T21:53:46.303057+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:39.625634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285208+0000","last_change":"2023-12-26T21:53:45.279979+0000","last_active":"2023-12-26T21:53:46.285208+0000","last_peered":"2023-12-26T21:53:46.285208+0000","last_clean":"2023-12-26T21:53:46.285208+0000","last_became_active":"2023-12-26T21:53:45.279563+0000","last_became_peered":"2023-12-26T21:53:45.279563+0000","last_unstale":"2023-12-26T21:53:46.285208+0000","last_undegraded":"2023-12-26T21:53:46.285208+0000","last_fullsized":"2023-12-26T21:53:46.285208+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:00:28.640861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308603+0000","last_change":"2023-12-26T21:53:46.302870+0000","last_active":"2023-12-26T21:53:46.308603+0000","last_peered":"2023-12-26T21:53:46.308603+0000","last_clean":"2023-12-26T21:53:46.308603+0000","last_became_active":"2023-12-26T21:53:46.302516+0000","last_became_peered":"2023-12-26T21:53:46.302516+0000","last_unstale":"2023-12-26T21:53:46.308603+0000","last_undegraded":"2023-12-26T21:53:46.308603+0000","last_fullsized":"2023-12-26T21:53:46.308603+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:26:29.931919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.294226+0000","last_change":"2023-12-26T21:53:45.284362+0000","last_active":"2023-12-26T21:53:46.294226+0000","last_peered":"2023-12-26T21:53:46.294226+0000","last_clean":"2023-12-26T21:53:46.294226+0000","last_became_active":"2023-12-26T21:53:45.284069+0000","last_became_peered":"2023-12-26T21:53:45.284069+0000","last_unstale":"2023-12-26T21:53:46.294226+0000","last_undegraded":"2023-12-26T21:53:46.294226+0000","last_fullsized":"2023-12-26T21:53:46.294226+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:24:03.820849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315503+0000","last_change":"2023-12-26T21:53:46.306165+0000","last_active":"2023-12-26T21:53:46.315503+0000","last_peered":"2023-12-26T21:53:46.315503+0000","last_clean":"2023-12-26T21:53:46.315503+0000","last_became_active":"2023-12-26T21:53:46.305894+0000","last_became_peered":"2023-12-26T21:53:46.305894+0000","last_unstale":"2023-12-26T21:53:46.315503+0000","last_undegraded":"2023-12-26T21:53:46.315503+0000","last_fullsized":"2023-12-26T21:53:46.315503+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:47:46.340543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281331+0000","last_change":"2023-12-26T21:53:45.282408+0000","last_active":"2023-12-26T21:53:46.281331+0000","last_peered":"2023-12-26T21:53:46.281331+0000","last_clean":"2023-12-26T21:53:46.281331+0000","last_became_active":"2023-12-26T21:53:45.282074+0000","last_became_peered":"2023-12-26T21:53:45.282074+0000","last_unstale":"2023-12-26T21:53:46.281331+0000","last_undegraded":"2023-12-26T21:53:46.281331+0000","last_fullsized":"2023-12-26T21:53:46.281331+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:18:52.576472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.302717+0000","last_change":"2023-12-26T21:53:46.300574+0000","last_active":"2023-12-26T21:53:46.302717+0000","last_peered":"2023-12-26T21:53:46.302717+0000","last_clean":"2023-12-26T21:53:46.302717+0000","last_became_active":"2023-12-26T21:53:46.300296+0000","last_became_peered":"2023-12-26T21:53:46.300296+0000","last_unstale":"2023-12-26T21:53:46.302717+0000","last_undegraded":"2023-12-26T21:53:46.302717+0000","last_fullsized":"2023-12-26T21:53:46.302717+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:51.026196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.318605+0000","last_change":"2023-12-26T21:53:45.280489+0000","last_active":"2023-12-26T21:53:46.318605+0000","last_peered":"2023-12-26T21:53:46.318605+0000","last_clean":"2023-12-26T21:53:46.318605+0000","last_became_active":"2023-12-26T21:53:45.279071+0000","last_became_peered":"2023-12-26T21:53:45.279071+0000","last_unstale":"2023-12-26T21:53:46.318605+0000","last_undegraded":"2023-12-26T21:53:46.318605+0000","last_fullsized":"2023-12-26T21:53:46.318605+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:28.851567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308510+0000","last_change":"2023-12-26T21:53:46.304923+0000","last_active":"2023-12-26T21:53:46.308510+0000","last_peered":"2023-12-26T21:53:46.308510+0000","last_clean":"2023-12-26T21:53:46.308510+0000","last_became_active":"2023-12-26T21:53:46.304621+0000","last_became_peered":"2023-12-26T21:53:46.304621+0000","last_unstale":"2023-12-26T21:53:46.308510+0000","last_undegraded":"2023-12-26T21:53:46.308510+0000","last_fullsized":"2023-12-26T21:53:46.308510+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:15:24.617194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.294726+0000","last_change":"2023-12-26T21:53:45.284941+0000","last_active":"2023-12-26T21:53:46.294726+0000","last_peered":"2023-12-26T21:53:46.294726+0000","last_clean":"2023-12-26T21:53:46.294726+0000","last_became_active":"2023-12-26T21:53:45.282717+0000","last_became_peered":"2023-12-26T21:53:45.282717+0000","last_unstale":"2023-12-26T21:53:46.294726+0000","last_undegraded":"2023-12-26T21:53:46.294726+0000","last_fullsized":"2023-12-26T21:53:46.294726+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:27.421803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299826+0000","last_change":"2023-12-26T21:53:46.288775+0000","last_active":"2023-12-26T21:53:46.299826+0000","last_peered":"2023-12-26T21:53:46.299826+0000","last_clean":"2023-12-26T21:53:46.299826+0000","last_became_active":"2023-12-26T21:53:46.288489+0000","last_became_peered":"2023-12-26T21:53:46.288489+0000","last_unstale":"2023-12-26T21:53:46.299826+0000","last_undegraded":"2023-12-26T21:53:46.299826+0000","last_fullsized":"2023-12-26T21:53:46.299826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:48:03.925563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.291033+0000","last_change":"2023-12-26T21:53:45.285839+0000","last_active":"2023-12-26T21:53:46.291033+0000","last_peered":"2023-12-26T21:53:46.291033+0000","last_clean":"2023-12-26T21:53:46.291033+0000","last_became_active":"2023-12-26T21:53:45.285409+0000","last_became_peered":"2023-12-26T21:53:45.285409+0000","last_unstale":"2023-12-26T21:53:46.291033+0000","last_undegraded":"2023-12-26T21:53:46.291033+0000","last_fullsized":"2023-12-26T21:53:46.291033+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:40.376576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310802+0000","last_change":"2023-12-26T21:53:46.302758+0000","last_active":"2023-12-26T21:53:46.310802+0000","last_peered":"2023-12-26T21:53:46.310802+0000","last_clean":"2023-12-26T21:53:46.310802+0000","last_became_active":"2023-12-26T21:53:46.299461+0000","last_became_peered":"2023-12-26T21:53:46.299461+0000","last_unstale":"2023-12-26T21:53:46.310802+0000","last_undegraded":"2023-12-26T21:53:46.310802+0000","last_fullsized":"2023-12-26T21:53:46.310802+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:39:28.335893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285877+0000","last_change":"2023-12-26T21:53:45.272282+0000","last_active":"2023-12-26T21:53:46.285877+0000","last_peered":"2023-12-26T21:53:46.285877+0000","last_clean":"2023-12-26T21:53:46.285877+0000","last_became_active":"2023-12-26T21:53:45.272019+0000","last_became_peered":"2023-12-26T21:53:45.272019+0000","last_unstale":"2023-12-26T21:53:46.285877+0000","last_undegraded":"2023-12-26T21:53:46.285877+0000","last_fullsized":"2023-12-26T21:53:46.285877+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:11:24.710650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310423+0000","last_change":"2023-12-26T21:53:46.306701+0000","last_active":"2023-12-26T21:53:46.310423+0000","last_peered":"2023-12-26T21:53:46.310423+0000","last_clean":"2023-12-26T21:53:46.310423+0000","last_became_active":"2023-12-26T21:53:46.306450+0000","last_became_peered":"2023-12-26T21:53:46.306450+0000","last_unstale":"2023-12-26T21:53:46.310423+0000","last_undegraded":"2023-12-26T21:53:46.310423+0000","last_fullsized":"2023-12-26T21:53:46.310423+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:56.921774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.284986+0000","last_change":"2023-12-26T21:53:45.279374+0000","last_active":"2023-12-26T21:53:46.284986+0000","last_peered":"2023-12-26T21:53:46.284986+0000","last_clean":"2023-12-26T21:53:46.284986+0000","last_became_active":"2023-12-26T21:53:45.278479+0000","last_became_peered":"2023-12-26T21:53:45.278479+0000","last_unstale":"2023-12-26T21:53:46.284986+0000","last_undegraded":"2023-12-26T21:53:46.284986+0000","last_fullsized":"2023-12-26T21:53:46.284986+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:29.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.313358+0000","last_change":"2023-12-26T21:53:46.308504+0000","last_active":"2023-12-26T21:53:46.313358+0000","last_peered":"2023-12-26T21:53:46.313358+0000","last_clean":"2023-12-26T21:53:46.313358+0000","last_became_active":"2023-12-26T21:53:46.308197+0000","last_became_peered":"2023-12-26T21:53:46.308197+0000","last_unstale":"2023-12-26T21:53:46.313358+0000","last_undegraded":"2023-12-26T21:53:46.313358+0000","last_fullsized":"2023-12-26T21:53:46.313358+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:42:54.558071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286559+0000","last_change":"2023-12-26T21:53:45.275306+0000","last_active":"2023-12-26T21:53:46.286559+0000","last_peered":"2023-12-26T21:53:46.286559+0000","last_clean":"2023-12-26T21:53:46.286559+0000","last_became_active":"2023-12-26T21:53:45.274732+0000","last_became_peered":"2023-12-26T21:53:45.274732+0000","last_unstale":"2023-12-26T21:53:46.286559+0000","last_undegraded":"2023-12-26T21:53:46.286559+0000","last_fullsized":"2023-12-26T21:53:46.286559+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:01:46.076671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308046+0000","last_change":"2023-12-26T21:53:46.306183+0000","last_active":"2023-12-26T21:53:46.308046+0000","last_peered":"2023-12-26T21:53:46.308046+0000","last_clean":"2023-12-26T21:53:46.308046+0000","last_became_active":"2023-12-26T21:53:46.305918+0000","last_became_peered":"2023-12-26T21:53:46.305918+0000","last_unstale":"2023-12-26T21:53:46.308046+0000","last_undegraded":"2023-12-26T21:53:46.308046+0000","last_fullsized":"2023-12-26T21:53:46.308046+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:49:38.482545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286935+0000","last_change":"2023-12-26T21:53:45.284416+0000","last_active":"2023-12-26T21:53:46.286935+0000","last_peered":"2023-12-26T21:53:46.286935+0000","last_clean":"2023-12-26T21:53:46.286935+0000","last_became_active":"2023-12-26T21:53:45.284072+0000","last_became_peered":"2023-12-26T21:53:45.284072+0000","last_unstale":"2023-12-26T21:53:46.286935+0000","last_undegraded":"2023-12-26T21:53:46.286935+0000","last_fullsized":"2023-12-26T21:53:46.286935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:46:59.726957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307933+0000","last_change":"2023-12-26T21:53:46.303791+0000","last_active":"2023-12-26T21:53:46.307933+0000","last_peered":"2023-12-26T21:53:46.307933+0000","last_clean":"2023-12-26T21:53:46.307933+0000","last_became_active":"2023-12-26T21:53:46.303518+0000","last_became_peered":"2023-12-26T21:53:46.303518+0000","last_unstale":"2023-12-26T21:53:46.307933+0000","last_undegraded":"2023-12-26T21:53:46.307933+0000","last_fullsized":"2023-12-26T21:53:46.307933+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:17.246701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.292408+0000","last_change":"2023-12-26T21:53:45.277961+0000","last_active":"2023-12-26T21:53:46.292408+0000","last_peered":"2023-12-26T21:53:46.292408+0000","last_clean":"2023-12-26T21:53:46.292408+0000","last_became_active":"2023-12-26T21:53:45.277682+0000","last_became_peered":"2023-12-26T21:53:45.277682+0000","last_unstale":"2023-12-26T21:53:46.292408+0000","last_undegraded":"2023-12-26T21:53:46.292408+0000","last_fullsized":"2023-12-26T21:53:46.292408+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:51:21.098860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308052+0000","last_change":"2023-12-26T21:53:46.305187+0000","last_active":"2023-12-26T21:53:46.308052+0000","last_peered":"2023-12-26T21:53:46.308052+0000","last_clean":"2023-12-26T21:53:46.308052+0000","last_became_active":"2023-12-26T21:53:46.304710+0000","last_became_peered":"2023-12-26T21:53:46.304710+0000","last_unstale":"2023-12-26T21:53:46.308052+0000","last_undegraded":"2023-12-26T21:53:46.308052+0000","last_fullsized":"2023-12-26T21:53:46.308052+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:29:09.046358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.319541+0000","last_change":"2023-12-26T21:53:45.277923+0000","last_active":"2023-12-26T21:53:46.319541+0000","last_peered":"2023-12-26T21:53:46.319541+0000","last_clean":"2023-12-26T21:53:46.319541+0000","last_became_active":"2023-12-26T21:53:45.277681+0000","last_became_peered":"2023-12-26T21:53:45.277681+0000","last_unstale":"2023-12-26T21:53:46.319541+0000","last_undegraded":"2023-12-26T21:53:46.319541+0000","last_fullsized":"2023-12-26T21:53:46.319541+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:40.310455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'205","reported_seq":555,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286580+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:46.286580+0000","last_peered":"2023-12-26T21:53:46.286580+0000","last_clean":"2023-12-26T21:53:46.286580+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:46.286580+0000","last_undegraded":"2023-12-26T21:53:46.286580+0000","last_fullsized":"2023-12-26T21:53:46.286580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542348,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302700,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069140,"statfs":{"total":96636764160,"available":96326799360,"internally_reserved":0,"allocated":1347584,"data_stored":653499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.47199999999999998,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.72499999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.69799999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74399999999999999,"5min":1.016,"15min":1.048},"last":0.74199999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.038,"15min":1.0509999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.85699999999999998,"5min":0.99199999999999999,"15min":1.02},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.62,"5min":0.504,"15min":0.5},"max":{"1min":0.94299999999999995,"5min":1.0189999999999999,"15min":1.052},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83499999999999996,"5min":1.036,"15min":1.036},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.747,"15min":0.746},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0129999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86399999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.117,"15min":1.117},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.437},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.006},"last":0.88}]}]},{"osd":6,"up_from":15,"seq":64424509644,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306800,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065040,"statfs":{"total":96636764160,"available":96322600960,"internally_reserved":0,"allocated":1351680,"data_stored":654073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.085},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69199999999999995,"15min":0.71799999999999997},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.34799999999999998},"max":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.624}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67800000000000005,"15min":0.70199999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.95299999999999996,"15min":0.98799999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.54200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.97599999999999998,"15min":1.054},"last":0.60299999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.93200000000000005,"15min":1.105},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.79200000000000004,"5min":0.92000000000000004,"15min":1.048},"last":0.48799999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.90200000000000002,"5min":1.026,"15min":1.1799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86899999999999999,"5min":0.96999999999999997,"15min":1.069},"last":0.56799999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.91100000000000003,"15min":1.137},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.47499999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93100000000000005,"5min":1.0049999999999999,"15min":1.296},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.437},"max":{"1min":1.008,"5min":1.008,"15min":1.3380000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.56799999999999995,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.95599999999999996,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.59099999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311472,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060368,"statfs":{"total":96636764160,"available":96317816832,"internally_reserved":0,"allocated":1941504,"data_stored":1243887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.436},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.55700000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.84699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.70399999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.87}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.88400000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.90500000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302712,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069128,"statfs":{"total":96636764160,"available":96326787072,"internally_reserved":0,"allocated":1359872,"data_stored":654987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":1.056},"last":0.377},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.41099999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47799999999999998,"15min":0.49399999999999999},"min":{"1min":0.315,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":1.006},"last":0.443},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.501}]},{"osd":3,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.84699999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80000000000000004,"5min":0.86699999999999999,"15min":1.1870000000000001},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.752},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.042,"5min":1.042,"15min":1.411},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.76800000000000002},"min":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.2070000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.754,"15min":0.79800000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.2569999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77100000000000002,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.452},"last":0.84399999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.214},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.012,"15min":1.3779999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46400000000000002},"max":{"1min":1.018,"5min":1.042,"15min":1.4359999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.79200000000000004},"min":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":1.0029999999999999,"5min":1.0069999999999999,"15min":1.3240000000000001},"last":0.74199999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542349,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306792,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065048,"statfs":{"total":96636764160,"available":96322609152,"internally_reserved":0,"allocated":1343488,"data_stored":653429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.877},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68600000000000005,"5min":0.83899999999999997,"15min":0.84399999999999997},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.65300000000000002,"5min":0.70799999999999996,"15min":0.92000000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.31,"15min":0.309},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.78600000000000003},"last":0.47899999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":1,"15min":1},"last":0.5},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.82299999999999995,"15min":0.85699999999999998},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.46800000000000003,"15min":0.34799999999999998},"max":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":1.284},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.76300000000000001},"min":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.95699999999999996,"5min":0.98399999999999999,"15min":1.1759999999999999},"last":1.018}]},{"osd":5,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.024,"5min":1.0600000000000001,"15min":1.2170000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":0.95499999999999996,"5min":1.014,"15min":1.1319999999999999},"last":0.91300000000000003}]},{"osd":6,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.038,"15min":1.242},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.40300000000000002},"max":{"1min":1.018,"5min":1.0609999999999999,"15min":1.272},"last":0.88200000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":1.073,"15min":1.1220000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.50800000000000001},"max":{"1min":0.999,"5min":1.0189999999999999,"15min":1.1990000000000001},"last":0.92600000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509645,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310900,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060940,"statfs":{"total":96636764160,"available":96318402560,"internally_reserved":0,"allocated":1355776,"data_stored":654075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.378}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82499999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.753}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.81299999999999994}]}]},{"osd":4,"up_from":14,"seq":60129542348,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306796,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065044,"statfs":{"total":96636764160,"available":96322605056,"internally_reserved":0,"allocated":1347584,"data_stored":653463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97899999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.48499999999999999,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":1.008},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.95199999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88900000000000001,"5min":0.97199999999999998,"15min":0.97299999999999998},"last":0.624}]},{"osd":3,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.71499999999999997},"min":{"1min":0.504,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.78100000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.438,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.84399999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509643,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1245367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.69999999999999996},"min":{"1min":0.44500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84199999999999997,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.749},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.4570000000000001},"last":0.73399999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68700000000000006,"15min":0.71499999999999997},"min":{"1min":0.499,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.625},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67800000000000005,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.4219999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.72799999999999998},"min":{"1min":0.54900000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.88500000000000001,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68600000000000005,"15min":0.71099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.92400000000000004,"15min":1.0569999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":1.052,"15min":1.4890000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.73599999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47799999999999998},"max":{"1min":0.88300000000000001,"5min":0.92200000000000004,"15min":1.081},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":2.5070000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.54500000000000004,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":2.8039999999999998},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.65700000000000003,"15min":0.66400000000000003},"min":{"1min":0.505,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":1.637},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.623,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":1.111,"5min":1.111,"15min":2.5579999999999998},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.61399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.97499999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.71299999999999997,"15min":0.70999999999999996},"min":{"1min":0.52200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.5980000000000001},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:55.523 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:53:55.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.722+0000 7f7365fff700 1 Processor -- start 2023-12-26T21:53:55.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.722+0000 7f7365fff700 1 -- start start 2023-12-26T21:53:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.723+0000 7f7365fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7360159600 0x7f73600ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.723+0000 7f7365fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73601599d0 0x7f736015dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.723+0000 7f7365fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73600cfd50 con 0x7f73600d01e0 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.723+0000 7f7365fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736015e210 con 0x7f7360159600 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.723+0000 7f7365fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600b1280 con 0x7f73601599d0 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.723+0000 7f735f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7360159600 0x7f73600ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.724+0000 7f735f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7360159600 0x7f73600ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46638/0 (socket says 172.21.15.131:46638) 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.724+0000 7f735f7fe700 1 -- 172.21.15.131:0/3743092943 learned_addr learned my addr 172.21.15.131:0/3743092943 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.724+0000 7f735f7fe700 1 -- 172.21.15.131:0/3743092943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73601599d0 msgr2=0x7f736015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.724+0000 7f735f7fe700 1 --2- 172.21.15.131:0/3743092943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73601599d0 0x7f736015dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.724+0000 7f735f7fe700 1 -- 172.21.15.131:0/3743092943 >> v1:172.21.15.190:6789/0 conn(0x7f73600d01e0 legacy=0x7f73600ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.724+0000 7f735f7fe700 1 -- 172.21.15.131:0/3743092943 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600cf140 con 0x7f7360159600 2023-12-26T21:53:55.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.724+0000 7f735f7fe700 1 --2- 172.21.15.131:0/3743092943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360159600 0x7f73600ceb60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7350009990 tx=0x7f7350009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfceb5b0d8ea9ff server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:55.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.724+0000 7f735e7fc700 1 -- 172.21.15.131:0/3743092943 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350005c10 con 0x7f7360159600 2023-12-26T21:53:55.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f735e7fc700 1 -- 172.21.15.131:0/3743092943 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7350005d70 con 0x7f7360159600 2023-12-26T21:53:55.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f735e7fc700 1 -- 172.21.15.131:0/3743092943 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735000b840 con 0x7f7360159600 2023-12-26T21:53:55.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f7365fff700 1 -- 172.21.15.131:0/3743092943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360159600 msgr2=0x7f73600ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f7365fff700 1 --2- 172.21.15.131:0/3743092943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360159600 0x7f73600ceb60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7350009990 tx=0x7f7350009220 comp rx=0 tx=0).stop 2023-12-26T21:53:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f7365fff700 1 -- 172.21.15.131:0/3743092943 shutdown_connections 2023-12-26T21:53:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f7365fff700 1 --2- 172.21.15.131:0/3743092943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360159600 0x7f73600ceb60 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f7365fff700 1 --2- 172.21.15.131:0/3743092943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73601599d0 0x7f736015dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f7365fff700 1 -- 172.21.15.131:0/3743092943 >> 172.21.15.131:0/3743092943 conn(0x7f736006ece0 msgr2=0x7f736006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f7365fff700 1 -- 172.21.15.131:0/3743092943 shutdown_connections 2023-12-26T21:53:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.725+0000 7f7365fff700 1 -- 172.21.15.131:0/3743092943 wait complete. 2023-12-26T21:53:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.727+0000 7f7365fff700 1 Processor -- start 2023-12-26T21:53:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.727+0000 7f7365fff700 1 -- start start 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f7365fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73600d01e0 0x7f736015c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f7365fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7360159600 0x7f73600b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f7365fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73600b42e0 0x7f7360156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f7365fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600c76a0 con 0x7f73600d01e0 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f7365fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736016ba70 con 0x7f7360159600 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f7365fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736016ae60 con 0x7f73600b42e0 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f735f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73600d01e0 0x7f736015c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f735f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73600d01e0 0x7f736015c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46654/0 (socket says 172.21.15.131:46654) 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f735f7fe700 1 -- 172.21.15.131:0/2483838789 learned_addr learned my addr 172.21.15.131:0/2483838789 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:55.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f735f7fe700 1 -- 172.21.15.131:0/2483838789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73600b42e0 msgr2=0x7f7360156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:55.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f735f7fe700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73600b42e0 0x7f7360156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:55.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f735f7fe700 1 -- 172.21.15.131:0/2483838789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7360159600 msgr2=0x7f73600b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:55.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f735f7fe700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7360159600 0x7f73600b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:55.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.728+0000 7f735f7fe700 1 -- 172.21.15.131:0/2483838789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350009480 con 0x7f73600d01e0 2023-12-26T21:53:55.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.729+0000 7f735f7fe700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73600d01e0 0x7f736015c480 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7350009260 tx=0x7f7350015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:55.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.729+0000 7f73477fe700 1 -- 172.21.15.131:0/2483838789 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350000d10 con 0x7f73600d01e0 2023-12-26T21:53:55.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.729+0000 7f73477fe700 1 -- 172.21.15.131:0/2483838789 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7350011c80 con 0x7f73600d01e0 2023-12-26T21:53:55.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.729+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736016b140 con 0x7f73600d01e0 2023-12-26T21:53:55.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.729+0000 7f73477fe700 1 -- 172.21.15.131:0/2483838789 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735001d430 con 0x7f73600d01e0 2023-12-26T21:53:55.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.729+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736016b700 con 0x7f73600d01e0 2023-12-26T21:53:55.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.730+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73600657e0 con 0x7f73600d01e0 2023-12-26T21:53:55.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.731+0000 7f73477fe700 1 -- 172.21.15.131:0/2483838789 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f735001d7e0 con 0x7f73600d01e0 2023-12-26T21:53:55.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.732+0000 7f73477fe700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f734c066f30 0x7f734c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:55.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.732+0000 7f73477fe700 1 -- 172.21.15.131:0/2483838789 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7350086310 con 0x7f73600d01e0 2023-12-26T21:53:55.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.732+0000 7f735effd700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f734c066f30 0x7f734c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:55.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.733+0000 7f735effd700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f734c066f30 0x7f734c0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f73600ca310 tx=0x7f7354006d10 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:55.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.737+0000 7f73477fe700 1 -- 172.21.15.131:0/2483838789 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7350053350 con 0x7f73600d01e0 2023-12-26T21:53:55.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.926+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f736016bbb0 con 0x7f734c066f30 2023-12-26T21:53:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.935+0000 7f73477fe700 1 -- 172.21.15.131:0/2483838789 <== mgr.6975 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289062 (secure 0 0 0) 0x7f73540113f0 con 0x7f734c066f30 2023-12-26T21:53:55.936 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f734c066f30 msgr2=0x7f734c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f734c066f30 0x7f734c0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f73600ca310 tx=0x7f7354006d10 comp rx=0 tx=0).stop 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73600d01e0 msgr2=0x7f736015c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73600d01e0 0x7f736015c480 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7350009260 tx=0x7f7350015000 comp rx=0 tx=0).stop 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 shutdown_connections 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73600b42e0 0x7f7360156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f734c066f30 0x7f734c0693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73600d01e0 0x7f736015c480 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 --2- 172.21.15.131:0/2483838789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7360159600 0x7f73600b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 >> 172.21.15.131:0/2483838789 conn(0x7f736006ece0 msgr2=0x7f7360068580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 shutdown_connections 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:55.939+0000 7f7365fff700 1 -- 172.21.15.131:0/2483838789 wait complete. 2023-12-26T21:53:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:53:55.958 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-26T21:53:54.113314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2451464,"kb_used_data":11720,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752523256,"statfs":{"total":773094113280,"available":770583814144,"internally_reserved":0,"allocated":12001280,"data_stored":6412780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001701"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310309+0000","last_change":"2023-12-26T21:53:46.307184+0000","last_active":"2023-12-26T21:53:46.310309+0000","last_peered":"2023-12-26T21:53:46.310309+0000","last_clean":"2023-12-26T21:53:46.310309+0000","last_became_active":"2023-12-26T21:53:46.306640+0000","last_became_peered":"2023-12-26T21:53:46.306640+0000","last_unstale":"2023-12-26T21:53:46.310309+0000","last_undegraded":"2023-12-26T21:53:46.310309+0000","last_fullsized":"2023-12-26T21:53:46.310309+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:18.030160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306053+0000","last_change":"2023-12-26T21:53:46.299743+0000","last_active":"2023-12-26T21:53:46.306053+0000","last_peered":"2023-12-26T21:53:46.306053+0000","last_clean":"2023-12-26T21:53:46.306053+0000","last_became_active":"2023-12-26T21:53:46.299485+0000","last_became_peered":"2023-12-26T21:53:46.299485+0000","last_unstale":"2023-12-26T21:53:46.306053+0000","last_undegraded":"2023-12-26T21:53:46.306053+0000","last_fullsized":"2023-12-26T21:53:46.306053+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.316105+0000","last_change":"2023-12-26T21:53:46.308139+0000","last_active":"2023-12-26T21:53:46.316105+0000","last_peered":"2023-12-26T21:53:46.316105+0000","last_clean":"2023-12-26T21:53:46.316105+0000","last_became_active":"2023-12-26T21:53:46.302969+0000","last_became_peered":"2023-12-26T21:53:46.302969+0000","last_unstale":"2023-12-26T21:53:46.316105+0000","last_undegraded":"2023-12-26T21:53:46.316105+0000","last_fullsized":"2023-12-26T21:53:46.316105+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:40:03.180960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.309784+0000","last_change":"2023-12-26T21:53:46.305271+0000","last_active":"2023-12-26T21:53:46.309784+0000","last_peered":"2023-12-26T21:53:46.309784+0000","last_clean":"2023-12-26T21:53:46.309784+0000","last_became_active":"2023-12-26T21:53:46.304996+0000","last_became_peered":"2023-12-26T21:53:46.304996+0000","last_unstale":"2023-12-26T21:53:46.309784+0000","last_undegraded":"2023-12-26T21:53:46.309784+0000","last_fullsized":"2023-12-26T21:53:46.309784+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:55.354802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299767+0000","last_change":"2023-12-26T21:53:46.289973+0000","last_active":"2023-12-26T21:53:46.299767+0000","last_peered":"2023-12-26T21:53:46.299767+0000","last_clean":"2023-12-26T21:53:46.299767+0000","last_became_active":"2023-12-26T21:53:46.289695+0000","last_became_peered":"2023-12-26T21:53:46.289695+0000","last_unstale":"2023-12-26T21:53:46.299767+0000","last_undegraded":"2023-12-26T21:53:46.299767+0000","last_fullsized":"2023-12-26T21:53:46.299767+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:43:34.451389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311686+0000","last_change":"2023-12-26T21:53:46.304859+0000","last_active":"2023-12-26T21:53:46.311686+0000","last_peered":"2023-12-26T21:53:46.311686+0000","last_clean":"2023-12-26T21:53:46.311686+0000","last_became_active":"2023-12-26T21:53:46.304606+0000","last_became_peered":"2023-12-26T21:53:46.304606+0000","last_unstale":"2023-12-26T21:53:46.311686+0000","last_undegraded":"2023-12-26T21:53:46.311686+0000","last_fullsized":"2023-12-26T21:53:46.311686+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:19:50.873182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307583+0000","last_change":"2023-12-26T21:53:46.296559+0000","last_active":"2023-12-26T21:53:46.307583+0000","last_peered":"2023-12-26T21:53:46.307583+0000","last_clean":"2023-12-26T21:53:46.307583+0000","last_became_active":"2023-12-26T21:53:46.296237+0000","last_became_peered":"2023-12-26T21:53:46.296237+0000","last_unstale":"2023-12-26T21:53:46.307583+0000","last_undegraded":"2023-12-26T21:53:46.307583+0000","last_fullsized":"2023-12-26T21:53:46.307583+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.312831+0000","last_change":"2023-12-26T21:53:46.299618+0000","last_active":"2023-12-26T21:53:46.312831+0000","last_peered":"2023-12-26T21:53:46.312831+0000","last_clean":"2023-12-26T21:53:46.312831+0000","last_became_active":"2023-12-26T21:53:46.299320+0000","last_became_peered":"2023-12-26T21:53:46.299320+0000","last_unstale":"2023-12-26T21:53:46.312831+0000","last_undegraded":"2023-12-26T21:53:46.312831+0000","last_fullsized":"2023-12-26T21:53:46.312831+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:16.932912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308991+0000","last_change":"2023-12-26T21:53:46.303931+0000","last_active":"2023-12-26T21:53:46.308991+0000","last_peered":"2023-12-26T21:53:46.308991+0000","last_clean":"2023-12-26T21:53:46.308991+0000","last_became_active":"2023-12-26T21:53:46.303627+0000","last_became_peered":"2023-12-26T21:53:46.303627+0000","last_unstale":"2023-12-26T21:53:46.308991+0000","last_undegraded":"2023-12-26T21:53:46.308991+0000","last_fullsized":"2023-12-26T21:53:46.308991+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:28.427569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315982+0000","last_change":"2023-12-26T21:53:46.306473+0000","last_active":"2023-12-26T21:53:46.315982+0000","last_peered":"2023-12-26T21:53:46.315982+0000","last_clean":"2023-12-26T21:53:46.315982+0000","last_became_active":"2023-12-26T21:53:46.306137+0000","last_became_peered":"2023-12-26T21:53:46.306137+0000","last_unstale":"2023-12-26T21:53:46.315982+0000","last_undegraded":"2023-12-26T21:53:46.315982+0000","last_fullsized":"2023-12-26T21:53:46.315982+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:54:06.271609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.313313+0000","last_change":"2023-12-26T21:53:46.309492+0000","last_active":"2023-12-26T21:53:46.313313+0000","last_peered":"2023-12-26T21:53:46.313313+0000","last_clean":"2023-12-26T21:53:46.313313+0000","last_became_active":"2023-12-26T21:53:46.309200+0000","last_became_peered":"2023-12-26T21:53:46.309200+0000","last_unstale":"2023-12-26T21:53:46.313313+0000","last_undegraded":"2023-12-26T21:53:46.313313+0000","last_fullsized":"2023-12-26T21:53:46.313313+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:28.847137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314884+0000","last_change":"2023-12-26T21:53:46.301794+0000","last_active":"2023-12-26T21:53:46.314884+0000","last_peered":"2023-12-26T21:53:46.314884+0000","last_clean":"2023-12-26T21:53:46.314884+0000","last_became_active":"2023-12-26T21:53:46.298746+0000","last_became_peered":"2023-12-26T21:53:46.298746+0000","last_unstale":"2023-12-26T21:53:46.314884+0000","last_undegraded":"2023-12-26T21:53:46.314884+0000","last_fullsized":"2023-12-26T21:53:46.314884+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:26:29.931919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314549+0000","last_change":"2023-12-26T21:53:46.308209+0000","last_active":"2023-12-26T21:53:46.314549+0000","last_peered":"2023-12-26T21:53:46.314549+0000","last_clean":"2023-12-26T21:53:46.314549+0000","last_became_active":"2023-12-26T21:53:46.308025+0000","last_became_peered":"2023-12-26T21:53:46.308025+0000","last_unstale":"2023-12-26T21:53:46.314549+0000","last_undegraded":"2023-12-26T21:53:46.314549+0000","last_fullsized":"2023-12-26T21:53:46.314549+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:53:11.039774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311289+0000","last_change":"2023-12-26T21:53:46.303517+0000","last_active":"2023-12-26T21:53:46.311289+0000","last_peered":"2023-12-26T21:53:46.311289+0000","last_clean":"2023-12-26T21:53:46.311289+0000","last_became_active":"2023-12-26T21:53:46.303137+0000","last_became_peered":"2023-12-26T21:53:46.303137+0000","last_unstale":"2023-12-26T21:53:46.311289+0000","last_undegraded":"2023-12-26T21:53:46.311289+0000","last_fullsized":"2023-12-26T21:53:46.311289+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:25:43.711079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314209+0000","last_change":"2023-12-26T21:53:46.308456+0000","last_active":"2023-12-26T21:53:46.314209+0000","last_peered":"2023-12-26T21:53:46.314209+0000","last_clean":"2023-12-26T21:53:46.314209+0000","last_became_active":"2023-12-26T21:53:46.308203+0000","last_became_peered":"2023-12-26T21:53:46.308203+0000","last_unstale":"2023-12-26T21:53:46.314209+0000","last_undegraded":"2023-12-26T21:53:46.314209+0000","last_fullsized":"2023-12-26T21:53:46.314209+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315918+0000","last_change":"2023-12-26T21:53:46.306283+0000","last_active":"2023-12-26T21:53:46.315918+0000","last_peered":"2023-12-26T21:53:46.315918+0000","last_clean":"2023-12-26T21:53:46.315918+0000","last_became_active":"2023-12-26T21:53:46.305800+0000","last_became_peered":"2023-12-26T21:53:46.305800+0000","last_unstale":"2023-12-26T21:53:46.315918+0000","last_undegraded":"2023-12-26T21:53:46.315918+0000","last_fullsized":"2023-12-26T21:53:46.315918+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:09:24.176979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314753+0000","last_change":"2023-12-26T21:53:46.300055+0000","last_active":"2023-12-26T21:53:46.314753+0000","last_peered":"2023-12-26T21:53:46.314753+0000","last_clean":"2023-12-26T21:53:46.314753+0000","last_became_active":"2023-12-26T21:53:46.298259+0000","last_became_peered":"2023-12-26T21:53:46.298259+0000","last_unstale":"2023-12-26T21:53:46.314753+0000","last_undegraded":"2023-12-26T21:53:46.314753+0000","last_fullsized":"2023-12-26T21:53:46.314753+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:55:34.210900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308204+0000","last_change":"2023-12-26T21:53:46.302662+0000","last_active":"2023-12-26T21:53:46.308204+0000","last_peered":"2023-12-26T21:53:46.308204+0000","last_clean":"2023-12-26T21:53:46.308204+0000","last_became_active":"2023-12-26T21:53:46.302401+0000","last_became_peered":"2023-12-26T21:53:46.302401+0000","last_unstale":"2023-12-26T21:53:46.308204+0000","last_undegraded":"2023-12-26T21:53:46.308204+0000","last_fullsized":"2023-12-26T21:53:46.308204+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:20:45.804162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311951+0000","last_change":"2023-12-26T21:53:46.305482+0000","last_active":"2023-12-26T21:53:46.311951+0000","last_peered":"2023-12-26T21:53:46.311951+0000","last_clean":"2023-12-26T21:53:46.311951+0000","last_became_active":"2023-12-26T21:53:46.305210+0000","last_became_peered":"2023-12-26T21:53:46.305210+0000","last_unstale":"2023-12-26T21:53:46.311951+0000","last_undegraded":"2023-12-26T21:53:46.311951+0000","last_fullsized":"2023-12-26T21:53:46.311951+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.439762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306262+0000","last_change":"2023-12-26T21:53:46.301077+0000","last_active":"2023-12-26T21:53:46.306262+0000","last_peered":"2023-12-26T21:53:46.306262+0000","last_clean":"2023-12-26T21:53:46.306262+0000","last_became_active":"2023-12-26T21:53:46.300762+0000","last_became_peered":"2023-12-26T21:53:46.300762+0000","last_unstale":"2023-12-26T21:53:46.306262+0000","last_undegraded":"2023-12-26T21:53:46.306262+0000","last_fullsized":"2023-12-26T21:53:46.306262+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311637+0000","last_change":"2023-12-26T21:53:46.306696+0000","last_active":"2023-12-26T21:53:46.311637+0000","last_peered":"2023-12-26T21:53:46.311637+0000","last_clean":"2023-12-26T21:53:46.311637+0000","last_became_active":"2023-12-26T21:53:46.306420+0000","last_became_peered":"2023-12-26T21:53:46.306420+0000","last_unstale":"2023-12-26T21:53:46.311637+0000","last_undegraded":"2023-12-26T21:53:46.311637+0000","last_fullsized":"2023-12-26T21:53:46.311637+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:31.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289642+0000","last_change":"2023-12-26T21:53:46.287813+0000","last_active":"2023-12-26T21:53:46.289642+0000","last_peered":"2023-12-26T21:53:46.289642+0000","last_clean":"2023-12-26T21:53:46.289642+0000","last_became_active":"2023-12-26T21:53:46.287544+0000","last_became_peered":"2023-12-26T21:53:46.287544+0000","last_unstale":"2023-12-26T21:53:46.289642+0000","last_undegraded":"2023-12-26T21:53:46.289642+0000","last_fullsized":"2023-12-26T21:53:46.289642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:27.394417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314855+0000","last_change":"2023-12-26T21:53:46.303413+0000","last_active":"2023-12-26T21:53:46.314855+0000","last_peered":"2023-12-26T21:53:46.314855+0000","last_clean":"2023-12-26T21:53:46.314855+0000","last_became_active":"2023-12-26T21:53:46.299017+0000","last_became_peered":"2023-12-26T21:53:46.299017+0000","last_unstale":"2023-12-26T21:53:46.314855+0000","last_undegraded":"2023-12-26T21:53:46.314855+0000","last_fullsized":"2023-12-26T21:53:46.314855+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:23:40.412951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.309404+0000","last_change":"2023-12-26T21:53:46.305600+0000","last_active":"2023-12-26T21:53:46.309404+0000","last_peered":"2023-12-26T21:53:46.309404+0000","last_clean":"2023-12-26T21:53:46.309404+0000","last_became_active":"2023-12-26T21:53:46.305279+0000","last_became_peered":"2023-12-26T21:53:46.305279+0000","last_unstale":"2023-12-26T21:53:46.309404+0000","last_undegraded":"2023-12-26T21:53:46.309404+0000","last_fullsized":"2023-12-26T21:53:46.309404+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:00:42.816954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308838+0000","last_change":"2023-12-26T21:53:46.306183+0000","last_active":"2023-12-26T21:53:46.308838+0000","last_peered":"2023-12-26T21:53:46.308838+0000","last_clean":"2023-12-26T21:53:46.308838+0000","last_became_active":"2023-12-26T21:53:46.304494+0000","last_became_peered":"2023-12-26T21:53:46.304494+0000","last_unstale":"2023-12-26T21:53:46.308838+0000","last_undegraded":"2023-12-26T21:53:46.308838+0000","last_fullsized":"2023-12-26T21:53:46.308838+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:50:21.227281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.297090+0000","last_change":"2023-12-26T21:53:46.287979+0000","last_active":"2023-12-26T21:53:46.297090+0000","last_peered":"2023-12-26T21:53:46.297090+0000","last_clean":"2023-12-26T21:53:46.297090+0000","last_became_active":"2023-12-26T21:53:46.287682+0000","last_became_peered":"2023-12-26T21:53:46.287682+0000","last_unstale":"2023-12-26T21:53:46.297090+0000","last_undegraded":"2023-12-26T21:53:46.297090+0000","last_fullsized":"2023-12-26T21:53:46.297090+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:38:29.898590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311484+0000","last_change":"2023-12-26T21:53:46.306111+0000","last_active":"2023-12-26T21:53:46.311484+0000","last_peered":"2023-12-26T21:53:46.311484+0000","last_clean":"2023-12-26T21:53:46.311484+0000","last_became_active":"2023-12-26T21:53:46.305852+0000","last_became_peered":"2023-12-26T21:53:46.305852+0000","last_unstale":"2023-12-26T21:53:46.311484+0000","last_undegraded":"2023-12-26T21:53:46.311484+0000","last_fullsized":"2023-12-26T21:53:46.311484+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:03:27.957388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311050+0000","last_change":"2023-12-26T21:53:46.305148+0000","last_active":"2023-12-26T21:53:46.311050+0000","last_peered":"2023-12-26T21:53:46.311050+0000","last_clean":"2023-12-26T21:53:46.311050+0000","last_became_active":"2023-12-26T21:53:46.304893+0000","last_became_peered":"2023-12-26T21:53:46.304893+0000","last_unstale":"2023-12-26T21:53:46.311050+0000","last_undegraded":"2023-12-26T21:53:46.311050+0000","last_fullsized":"2023-12-26T21:53:46.311050+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:05.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.311656+0000","last_change":"2023-12-26T21:53:46.306908+0000","last_active":"2023-12-26T21:53:46.311656+0000","last_peered":"2023-12-26T21:53:46.311656+0000","last_clean":"2023-12-26T21:53:46.311656+0000","last_became_active":"2023-12-26T21:53:46.306331+0000","last_became_peered":"2023-12-26T21:53:46.306331+0000","last_unstale":"2023-12-26T21:53:46.311656+0000","last_undegraded":"2023-12-26T21:53:46.311656+0000","last_fullsized":"2023-12-26T21:53:46.311656+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:50:40.961352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307642+0000","last_change":"2023-12-26T21:53:46.301677+0000","last_active":"2023-12-26T21:53:46.307642+0000","last_peered":"2023-12-26T21:53:46.307642+0000","last_clean":"2023-12-26T21:53:46.307642+0000","last_became_active":"2023-12-26T21:53:46.301432+0000","last_became_peered":"2023-12-26T21:53:46.301432+0000","last_unstale":"2023-12-26T21:53:46.307642+0000","last_undegraded":"2023-12-26T21:53:46.307642+0000","last_fullsized":"2023-12-26T21:53:46.307642+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:50:59.325688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.317281+0000","last_change":"2023-12-26T21:53:46.313093+0000","last_active":"2023-12-26T21:53:46.317281+0000","last_peered":"2023-12-26T21:53:46.317281+0000","last_clean":"2023-12-26T21:53:46.317281+0000","last_became_active":"2023-12-26T21:53:46.312678+0000","last_became_peered":"2023-12-26T21:53:46.312678+0000","last_unstale":"2023-12-26T21:53:46.317281+0000","last_undegraded":"2023-12-26T21:53:46.317281+0000","last_fullsized":"2023-12-26T21:53:46.317281+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:42.880417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315827+0000","last_change":"2023-12-26T21:53:46.306101+0000","last_active":"2023-12-26T21:53:46.315827+0000","last_peered":"2023-12-26T21:53:46.315827+0000","last_clean":"2023-12-26T21:53:46.315827+0000","last_became_active":"2023-12-26T21:53:46.305481+0000","last_became_peered":"2023-12-26T21:53:46.305481+0000","last_unstale":"2023-12-26T21:53:46.315827+0000","last_undegraded":"2023-12-26T21:53:46.315827+0000","last_fullsized":"2023-12-26T21:53:46.315827+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:39:06.014521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314881+0000","last_change":"2023-12-26T21:53:46.302506+0000","last_active":"2023-12-26T21:53:46.314881+0000","last_peered":"2023-12-26T21:53:46.314881+0000","last_clean":"2023-12-26T21:53:46.314881+0000","last_became_active":"2023-12-26T21:53:46.301746+0000","last_became_peered":"2023-12-26T21:53:46.301746+0000","last_unstale":"2023-12-26T21:53:46.314881+0000","last_undegraded":"2023-12-26T21:53:46.314881+0000","last_fullsized":"2023-12-26T21:53:46.314881+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:55.354802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"109'4","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321799+0000","last_change":"2023-12-26T21:53:45.285708+0000","last_active":"2023-12-26T21:53:46.321799+0000","last_peered":"2023-12-26T21:53:46.321799+0000","last_clean":"2023-12-26T21:53:46.321799+0000","last_became_active":"2023-12-26T21:53:45.285466+0000","last_became_peered":"2023-12-26T21:53:45.285466+0000","last_unstale":"2023-12-26T21:53:46.321799+0000","last_undegraded":"2023-12-26T21:53:46.321799+0000","last_fullsized":"2023-12-26T21:53:46.321799+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:11.158488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310328+0000","last_change":"2023-12-26T21:53:46.307223+0000","last_active":"2023-12-26T21:53:46.310328+0000","last_peered":"2023-12-26T21:53:46.310328+0000","last_clean":"2023-12-26T21:53:46.310328+0000","last_became_active":"2023-12-26T21:53:46.306963+0000","last_became_peered":"2023-12-26T21:53:46.306963+0000","last_unstale":"2023-12-26T21:53:46.310328+0000","last_undegraded":"2023-12-26T21:53:46.310328+0000","last_fullsized":"2023-12-26T21:53:46.310328+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:12:37.634766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321262+0000","last_change":"2023-12-26T21:53:45.273356+0000","last_active":"2023-12-26T21:53:46.321262+0000","last_peered":"2023-12-26T21:53:46.321262+0000","last_clean":"2023-12-26T21:53:46.321262+0000","last_became_active":"2023-12-26T21:53:45.273096+0000","last_became_peered":"2023-12-26T21:53:45.273096+0000","last_unstale":"2023-12-26T21:53:46.321262+0000","last_undegraded":"2023-12-26T21:53:46.321262+0000","last_fullsized":"2023-12-26T21:53:46.321262+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:26.388651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314910+0000","last_change":"2023-12-26T21:53:46.303618+0000","last_active":"2023-12-26T21:53:46.314910+0000","last_peered":"2023-12-26T21:53:46.314910+0000","last_clean":"2023-12-26T21:53:46.314910+0000","last_became_active":"2023-12-26T21:53:46.301521+0000","last_became_peered":"2023-12-26T21:53:46.301521+0000","last_unstale":"2023-12-26T21:53:46.314910+0000","last_undegraded":"2023-12-26T21:53:46.314910+0000","last_fullsized":"2023-12-26T21:53:46.314910+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:01:53.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.318512+0000","last_change":"2023-12-26T21:53:45.287570+0000","last_active":"2023-12-26T21:53:46.318512+0000","last_peered":"2023-12-26T21:53:46.318512+0000","last_clean":"2023-12-26T21:53:46.318512+0000","last_became_active":"2023-12-26T21:53:45.287333+0000","last_became_peered":"2023-12-26T21:53:45.287333+0000","last_unstale":"2023-12-26T21:53:46.318512+0000","last_undegraded":"2023-12-26T21:53:46.318512+0000","last_fullsized":"2023-12-26T21:53:46.318512+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:33.642526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.305631+0000","last_change":"2023-12-26T21:53:46.299487+0000","last_active":"2023-12-26T21:53:46.305631+0000","last_peered":"2023-12-26T21:53:46.305631+0000","last_clean":"2023-12-26T21:53:46.305631+0000","last_became_active":"2023-12-26T21:53:46.299063+0000","last_became_peered":"2023-12-26T21:53:46.299063+0000","last_unstale":"2023-12-26T21:53:46.305631+0000","last_undegraded":"2023-12-26T21:53:46.305631+0000","last_fullsized":"2023-12-26T21:53:46.305631+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:48.449758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.318647+0000","last_change":"2023-12-26T21:53:45.283310+0000","last_active":"2023-12-26T21:53:46.318647+0000","last_peered":"2023-12-26T21:53:46.318647+0000","last_clean":"2023-12-26T21:53:46.318647+0000","last_became_active":"2023-12-26T21:53:45.282931+0000","last_became_peered":"2023-12-26T21:53:45.282931+0000","last_unstale":"2023-12-26T21:53:46.318647+0000","last_undegraded":"2023-12-26T21:53:46.318647+0000","last_fullsized":"2023-12-26T21:53:46.318647+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:44:34.249955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306117+0000","last_change":"2023-12-26T21:53:46.302615+0000","last_active":"2023-12-26T21:53:46.306117+0000","last_peered":"2023-12-26T21:53:46.306117+0000","last_clean":"2023-12-26T21:53:46.306117+0000","last_became_active":"2023-12-26T21:53:46.302098+0000","last_became_peered":"2023-12-26T21:53:46.302098+0000","last_unstale":"2023-12-26T21:53:46.306117+0000","last_undegraded":"2023-12-26T21:53:46.306117+0000","last_fullsized":"2023-12-26T21:53:46.306117+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:01:53.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315670+0000","last_change":"2023-12-26T21:53:45.283590+0000","last_active":"2023-12-26T21:53:46.315670+0000","last_peered":"2023-12-26T21:53:46.315670+0000","last_clean":"2023-12-26T21:53:46.315670+0000","last_became_active":"2023-12-26T21:53:45.282825+0000","last_became_peered":"2023-12-26T21:53:45.282825+0000","last_unstale":"2023-12-26T21:53:46.315670+0000","last_undegraded":"2023-12-26T21:53:46.315670+0000","last_fullsized":"2023-12-26T21:53:46.315670+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:27:44.133796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.306503+0000","last_change":"2023-12-26T21:53:46.297813+0000","last_active":"2023-12-26T21:53:46.306503+0000","last_peered":"2023-12-26T21:53:46.306503+0000","last_clean":"2023-12-26T21:53:46.306503+0000","last_became_active":"2023-12-26T21:53:46.297448+0000","last_became_peered":"2023-12-26T21:53:46.297448+0000","last_unstale":"2023-12-26T21:53:46.306503+0000","last_undegraded":"2023-12-26T21:53:46.306503+0000","last_fullsized":"2023-12-26T21:53:46.306503+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:13:57.688004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.319450+0000","last_change":"2023-12-26T21:53:45.275507+0000","last_active":"2023-12-26T21:53:46.319450+0000","last_peered":"2023-12-26T21:53:46.319450+0000","last_clean":"2023-12-26T21:53:46.319450+0000","last_became_active":"2023-12-26T21:53:45.275041+0000","last_became_peered":"2023-12-26T21:53:45.275041+0000","last_unstale":"2023-12-26T21:53:46.319450+0000","last_undegraded":"2023-12-26T21:53:46.319450+0000","last_fullsized":"2023-12-26T21:53:46.319450+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:52:49.674188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299738+0000","last_change":"2023-12-26T21:53:46.297785+0000","last_active":"2023-12-26T21:53:46.299738+0000","last_peered":"2023-12-26T21:53:46.299738+0000","last_clean":"2023-12-26T21:53:46.299738+0000","last_became_active":"2023-12-26T21:53:46.297505+0000","last_became_peered":"2023-12-26T21:53:46.297505+0000","last_unstale":"2023-12-26T21:53:46.299738+0000","last_undegraded":"2023-12-26T21:53:46.299738+0000","last_fullsized":"2023-12-26T21:53:46.299738+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:33.304213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281100+0000","last_change":"2023-12-26T21:53:45.279536+0000","last_active":"2023-12-26T21:53:46.281100+0000","last_peered":"2023-12-26T21:53:46.281100+0000","last_clean":"2023-12-26T21:53:46.281100+0000","last_became_active":"2023-12-26T21:53:45.279282+0000","last_became_peered":"2023-12-26T21:53:45.279282+0000","last_unstale":"2023-12-26T21:53:46.281100+0000","last_undegraded":"2023-12-26T21:53:46.281100+0000","last_fullsized":"2023-12-26T21:53:46.281100+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:58.794519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.288617+0000","last_change":"2023-12-26T21:53:46.285357+0000","last_active":"2023-12-26T21:53:46.288617+0000","last_peered":"2023-12-26T21:53:46.288617+0000","last_clean":"2023-12-26T21:53:46.288617+0000","last_became_active":"2023-12-26T21:53:46.285051+0000","last_became_peered":"2023-12-26T21:53:46.285051+0000","last_unstale":"2023-12-26T21:53:46.288617+0000","last_undegraded":"2023-12-26T21:53:46.288617+0000","last_fullsized":"2023-12-26T21:53:46.288617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:31:16.372604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281226+0000","last_change":"2023-12-26T21:53:45.268538+0000","last_active":"2023-12-26T21:53:46.281226+0000","last_peered":"2023-12-26T21:53:46.281226+0000","last_clean":"2023-12-26T21:53:46.281226+0000","last_became_active":"2023-12-26T21:53:45.268069+0000","last_became_peered":"2023-12-26T21:53:45.268069+0000","last_unstale":"2023-12-26T21:53:46.281226+0000","last_undegraded":"2023-12-26T21:53:46.281226+0000","last_fullsized":"2023-12-26T21:53:46.281226+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:36:25.546517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300878+0000","last_change":"2023-12-26T21:53:46.297819+0000","last_active":"2023-12-26T21:53:46.300878+0000","last_peered":"2023-12-26T21:53:46.300878+0000","last_clean":"2023-12-26T21:53:46.300878+0000","last_became_active":"2023-12-26T21:53:46.297541+0000","last_became_peered":"2023-12-26T21:53:46.297541+0000","last_unstale":"2023-12-26T21:53:46.300878+0000","last_undegraded":"2023-12-26T21:53:46.300878+0000","last_fullsized":"2023-12-26T21:53:46.300878+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:10:09.104721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289148+0000","last_change":"2023-12-26T21:53:45.281269+0000","last_active":"2023-12-26T21:53:46.289148+0000","last_peered":"2023-12-26T21:53:46.289148+0000","last_clean":"2023-12-26T21:53:46.289148+0000","last_became_active":"2023-12-26T21:53:45.280967+0000","last_became_peered":"2023-12-26T21:53:45.280967+0000","last_unstale":"2023-12-26T21:53:46.289148+0000","last_undegraded":"2023-12-26T21:53:46.289148+0000","last_fullsized":"2023-12-26T21:53:46.289148+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:20:00.172583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299929+0000","last_change":"2023-12-26T21:53:46.298319+0000","last_active":"2023-12-26T21:53:46.299929+0000","last_peered":"2023-12-26T21:53:46.299929+0000","last_clean":"2023-12-26T21:53:46.299929+0000","last_became_active":"2023-12-26T21:53:46.298050+0000","last_became_peered":"2023-12-26T21:53:46.298050+0000","last_unstale":"2023-12-26T21:53:46.299929+0000","last_undegraded":"2023-12-26T21:53:46.299929+0000","last_fullsized":"2023-12-26T21:53:46.299929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:34.648293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.289970+0000","last_change":"2023-12-26T21:53:45.279444+0000","last_active":"2023-12-26T21:53:46.289970+0000","last_peered":"2023-12-26T21:53:46.289970+0000","last_clean":"2023-12-26T21:53:46.289970+0000","last_became_active":"2023-12-26T21:53:45.279155+0000","last_became_peered":"2023-12-26T21:53:45.279155+0000","last_unstale":"2023-12-26T21:53:46.289970+0000","last_undegraded":"2023-12-26T21:53:46.289970+0000","last_fullsized":"2023-12-26T21:53:46.289970+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:30:46.434273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.313332+0000","last_change":"2023-12-26T21:53:46.310040+0000","last_active":"2023-12-26T21:53:46.313332+0000","last_peered":"2023-12-26T21:53:46.313332+0000","last_clean":"2023-12-26T21:53:46.313332+0000","last_became_active":"2023-12-26T21:53:46.309454+0000","last_became_peered":"2023-12-26T21:53:46.309454+0000","last_unstale":"2023-12-26T21:53:46.313332+0000","last_undegraded":"2023-12-26T21:53:46.313332+0000","last_fullsized":"2023-12-26T21:53:46.313332+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:57:08.838429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"109'3","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321640+0000","last_change":"2023-12-26T21:53:45.285950+0000","last_active":"2023-12-26T21:53:46.321640+0000","last_peered":"2023-12-26T21:53:46.321640+0000","last_clean":"2023-12-26T21:53:46.321640+0000","last_became_active":"2023-12-26T21:53:45.285657+0000","last_became_peered":"2023-12-26T21:53:45.285657+0000","last_unstale":"2023-12-26T21:53:46.321640+0000","last_undegraded":"2023-12-26T21:53:46.321640+0000","last_fullsized":"2023-12-26T21:53:46.321640+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:55.530687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303089+0000","last_change":"2023-12-26T21:53:46.301426+0000","last_active":"2023-12-26T21:53:46.303089+0000","last_peered":"2023-12-26T21:53:46.303089+0000","last_clean":"2023-12-26T21:53:46.303089+0000","last_became_active":"2023-12-26T21:53:46.301116+0000","last_became_peered":"2023-12-26T21:53:46.301116+0000","last_unstale":"2023-12-26T21:53:46.303089+0000","last_undegraded":"2023-12-26T21:53:46.303089+0000","last_fullsized":"2023-12-26T21:53:46.303089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:49.402467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"109'6","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321852+0000","last_change":"2023-12-26T21:53:45.281131+0000","last_active":"2023-12-26T21:53:46.321852+0000","last_peered":"2023-12-26T21:53:46.321852+0000","last_clean":"2023-12-26T21:53:46.321852+0000","last_became_active":"2023-12-26T21:53:45.280843+0000","last_became_peered":"2023-12-26T21:53:45.280843+0000","last_unstale":"2023-12-26T21:53:46.321852+0000","last_undegraded":"2023-12-26T21:53:46.321852+0000","last_fullsized":"2023-12-26T21:53:46.321852+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:45:10.491412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307228+0000","last_change":"2023-12-26T21:53:46.302146+0000","last_active":"2023-12-26T21:53:46.307228+0000","last_peered":"2023-12-26T21:53:46.307228+0000","last_clean":"2023-12-26T21:53:46.307228+0000","last_became_active":"2023-12-26T21:53:46.301846+0000","last_became_peered":"2023-12-26T21:53:46.301846+0000","last_unstale":"2023-12-26T21:53:46.307228+0000","last_undegraded":"2023-12-26T21:53:46.307228+0000","last_fullsized":"2023-12-26T21:53:46.307228+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:57:09.425627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.319761+0000","last_change":"2023-12-26T21:53:45.277993+0000","last_active":"2023-12-26T21:53:46.319761+0000","last_peered":"2023-12-26T21:53:46.319761+0000","last_clean":"2023-12-26T21:53:46.319761+0000","last_became_active":"2023-12-26T21:53:45.276119+0000","last_became_peered":"2023-12-26T21:53:45.276119+0000","last_unstale":"2023-12-26T21:53:46.319761+0000","last_undegraded":"2023-12-26T21:53:46.319761+0000","last_fullsized":"2023-12-26T21:53:46.319761+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:49.247010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304318+0000","last_change":"2023-12-26T21:53:46.298424+0000","last_active":"2023-12-26T21:53:46.304318+0000","last_peered":"2023-12-26T21:53:46.304318+0000","last_clean":"2023-12-26T21:53:46.304318+0000","last_became_active":"2023-12-26T21:53:46.298145+0000","last_became_peered":"2023-12-26T21:53:46.298145+0000","last_unstale":"2023-12-26T21:53:46.304318+0000","last_undegraded":"2023-12-26T21:53:46.304318+0000","last_fullsized":"2023-12-26T21:53:46.304318+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:24:04.826615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285162+0000","last_change":"2023-12-26T21:53:45.281444+0000","last_active":"2023-12-26T21:53:46.285162+0000","last_peered":"2023-12-26T21:53:46.285162+0000","last_clean":"2023-12-26T21:53:46.285162+0000","last_became_active":"2023-12-26T21:53:45.280420+0000","last_became_peered":"2023-12-26T21:53:45.280420+0000","last_unstale":"2023-12-26T21:53:46.285162+0000","last_undegraded":"2023-12-26T21:53:46.285162+0000","last_fullsized":"2023-12-26T21:53:46.285162+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:17.475634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.300883+0000","last_change":"2023-12-26T21:53:46.297808+0000","last_active":"2023-12-26T21:53:46.300883+0000","last_peered":"2023-12-26T21:53:46.300883+0000","last_clean":"2023-12-26T21:53:46.300883+0000","last_became_active":"2023-12-26T21:53:46.296873+0000","last_became_peered":"2023-12-26T21:53:46.296873+0000","last_unstale":"2023-12-26T21:53:46.300883+0000","last_undegraded":"2023-12-26T21:53:46.300883+0000","last_fullsized":"2023-12-26T21:53:46.300883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:29.573501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286405+0000","last_change":"2023-12-26T21:53:45.283894+0000","last_active":"2023-12-26T21:53:46.286405+0000","last_peered":"2023-12-26T21:53:46.286405+0000","last_clean":"2023-12-26T21:53:46.286405+0000","last_became_active":"2023-12-26T21:53:45.283609+0000","last_became_peered":"2023-12-26T21:53:45.283609+0000","last_unstale":"2023-12-26T21:53:46.286405+0000","last_undegraded":"2023-12-26T21:53:46.286405+0000","last_fullsized":"2023-12-26T21:53:46.286405+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:15.433996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.313371+0000","last_change":"2023-12-26T21:53:46.309163+0000","last_active":"2023-12-26T21:53:46.313371+0000","last_peered":"2023-12-26T21:53:46.313371+0000","last_clean":"2023-12-26T21:53:46.313371+0000","last_became_active":"2023-12-26T21:53:46.308834+0000","last_became_peered":"2023-12-26T21:53:46.308834+0000","last_unstale":"2023-12-26T21:53:46.313371+0000","last_undegraded":"2023-12-26T21:53:46.313371+0000","last_fullsized":"2023-12-26T21:53:46.313371+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:23.271735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"109'8","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.321108+0000","last_change":"2023-12-26T21:53:45.282550+0000","last_active":"2023-12-26T21:53:46.321108+0000","last_peered":"2023-12-26T21:53:46.321108+0000","last_clean":"2023-12-26T21:53:46.321108+0000","last_became_active":"2023-12-26T21:53:45.282289+0000","last_became_peered":"2023-12-26T21:53:45.282289+0000","last_unstale":"2023-12-26T21:53:46.321108+0000","last_undegraded":"2023-12-26T21:53:46.321108+0000","last_fullsized":"2023-12-26T21:53:46.321108+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:37:04.117197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.314843+0000","last_change":"2023-12-26T21:53:46.299765+0000","last_active":"2023-12-26T21:53:46.314843+0000","last_peered":"2023-12-26T21:53:46.314843+0000","last_clean":"2023-12-26T21:53:46.314843+0000","last_became_active":"2023-12-26T21:53:46.299465+0000","last_became_peered":"2023-12-26T21:53:46.299465+0000","last_unstale":"2023-12-26T21:53:46.314843+0000","last_undegraded":"2023-12-26T21:53:46.314843+0000","last_fullsized":"2023-12-26T21:53:46.314843+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:50:40.961352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.320094+0000","last_change":"2023-12-26T21:53:45.280067+0000","last_active":"2023-12-26T21:53:46.320094+0000","last_peered":"2023-12-26T21:53:46.320094+0000","last_clean":"2023-12-26T21:53:46.320094+0000","last_became_active":"2023-12-26T21:53:45.279713+0000","last_became_peered":"2023-12-26T21:53:45.279713+0000","last_unstale":"2023-12-26T21:53:46.320094+0000","last_undegraded":"2023-12-26T21:53:46.320094+0000","last_fullsized":"2023-12-26T21:53:46.320094+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:15.927146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.316148+0000","last_change":"2023-12-26T21:53:46.307450+0000","last_active":"2023-12-26T21:53:46.316148+0000","last_peered":"2023-12-26T21:53:46.316148+0000","last_clean":"2023-12-26T21:53:46.316148+0000","last_became_active":"2023-12-26T21:53:46.307168+0000","last_became_peered":"2023-12-26T21:53:46.307168+0000","last_unstale":"2023-12-26T21:53:46.316148+0000","last_undegraded":"2023-12-26T21:53:46.316148+0000","last_fullsized":"2023-12-26T21:53:46.316148+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:46.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.316477+0000","last_change":"2023-12-26T21:53:45.277896+0000","last_active":"2023-12-26T21:53:46.316477+0000","last_peered":"2023-12-26T21:53:46.316477+0000","last_clean":"2023-12-26T21:53:46.316477+0000","last_became_active":"2023-12-26T21:53:45.277584+0000","last_became_peered":"2023-12-26T21:53:45.277584+0000","last_unstale":"2023-12-26T21:53:46.316477+0000","last_undegraded":"2023-12-26T21:53:46.316477+0000","last_fullsized":"2023-12-26T21:53:46.316477+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:44:17.169190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303450+0000","last_change":"2023-12-26T21:53:46.298211+0000","last_active":"2023-12-26T21:53:46.303450+0000","last_peered":"2023-12-26T21:53:46.303450+0000","last_clean":"2023-12-26T21:53:46.303450+0000","last_became_active":"2023-12-26T21:53:46.297939+0000","last_became_peered":"2023-12-26T21:53:46.297939+0000","last_unstale":"2023-12-26T21:53:46.303450+0000","last_undegraded":"2023-12-26T21:53:46.303450+0000","last_fullsized":"2023-12-26T21:53:46.303450+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:31.772113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.282526+0000","last_change":"2023-12-26T21:53:45.280486+0000","last_active":"2023-12-26T21:53:46.282526+0000","last_peered":"2023-12-26T21:53:46.282526+0000","last_clean":"2023-12-26T21:53:46.282526+0000","last_became_active":"2023-12-26T21:53:45.280178+0000","last_became_peered":"2023-12-26T21:53:45.280178+0000","last_unstale":"2023-12-26T21:53:46.282526+0000","last_undegraded":"2023-12-26T21:53:46.282526+0000","last_fullsized":"2023-12-26T21:53:46.282526+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:13:56.557753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.304197+0000","last_change":"2023-12-26T21:53:46.297254+0000","last_active":"2023-12-26T21:53:46.304197+0000","last_peered":"2023-12-26T21:53:46.304197+0000","last_clean":"2023-12-26T21:53:46.304197+0000","last_became_active":"2023-12-26T21:53:46.296358+0000","last_became_peered":"2023-12-26T21:53:46.296358+0000","last_unstale":"2023-12-26T21:53:46.304197+0000","last_undegraded":"2023-12-26T21:53:46.304197+0000","last_fullsized":"2023-12-26T21:53:46.304197+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:48:11.304516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.317134+0000","last_change":"2023-12-26T21:53:45.271721+0000","last_active":"2023-12-26T21:53:46.317134+0000","last_peered":"2023-12-26T21:53:46.317134+0000","last_clean":"2023-12-26T21:53:46.317134+0000","last_became_active":"2023-12-26T21:53:45.271453+0000","last_became_peered":"2023-12-26T21:53:45.271453+0000","last_unstale":"2023-12-26T21:53:46.317134+0000","last_undegraded":"2023-12-26T21:53:46.317134+0000","last_fullsized":"2023-12-26T21:53:46.317134+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:49:24.818172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.303057+0000","last_change":"2023-12-26T21:53:46.301435+0000","last_active":"2023-12-26T21:53:46.303057+0000","last_peered":"2023-12-26T21:53:46.303057+0000","last_clean":"2023-12-26T21:53:46.303057+0000","last_became_active":"2023-12-26T21:53:46.301117+0000","last_became_peered":"2023-12-26T21:53:46.301117+0000","last_unstale":"2023-12-26T21:53:46.303057+0000","last_undegraded":"2023-12-26T21:53:46.303057+0000","last_fullsized":"2023-12-26T21:53:46.303057+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:39.625634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285208+0000","last_change":"2023-12-26T21:53:45.279979+0000","last_active":"2023-12-26T21:53:46.285208+0000","last_peered":"2023-12-26T21:53:46.285208+0000","last_clean":"2023-12-26T21:53:46.285208+0000","last_became_active":"2023-12-26T21:53:45.279563+0000","last_became_peered":"2023-12-26T21:53:45.279563+0000","last_unstale":"2023-12-26T21:53:46.285208+0000","last_undegraded":"2023-12-26T21:53:46.285208+0000","last_fullsized":"2023-12-26T21:53:46.285208+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:00:28.640861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308603+0000","last_change":"2023-12-26T21:53:46.302870+0000","last_active":"2023-12-26T21:53:46.308603+0000","last_peered":"2023-12-26T21:53:46.308603+0000","last_clean":"2023-12-26T21:53:46.308603+0000","last_became_active":"2023-12-26T21:53:46.302516+0000","last_became_peered":"2023-12-26T21:53:46.302516+0000","last_unstale":"2023-12-26T21:53:46.308603+0000","last_undegraded":"2023-12-26T21:53:46.308603+0000","last_fullsized":"2023-12-26T21:53:46.308603+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:26:29.931919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.294226+0000","last_change":"2023-12-26T21:53:45.284362+0000","last_active":"2023-12-26T21:53:46.294226+0000","last_peered":"2023-12-26T21:53:46.294226+0000","last_clean":"2023-12-26T21:53:46.294226+0000","last_became_active":"2023-12-26T21:53:45.284069+0000","last_became_peered":"2023-12-26T21:53:45.284069+0000","last_unstale":"2023-12-26T21:53:46.294226+0000","last_undegraded":"2023-12-26T21:53:46.294226+0000","last_fullsized":"2023-12-26T21:53:46.294226+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:24:03.820849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.315503+0000","last_change":"2023-12-26T21:53:46.306165+0000","last_active":"2023-12-26T21:53:46.315503+0000","last_peered":"2023-12-26T21:53:46.315503+0000","last_clean":"2023-12-26T21:53:46.315503+0000","last_became_active":"2023-12-26T21:53:46.305894+0000","last_became_peered":"2023-12-26T21:53:46.305894+0000","last_unstale":"2023-12-26T21:53:46.315503+0000","last_undegraded":"2023-12-26T21:53:46.315503+0000","last_fullsized":"2023-12-26T21:53:46.315503+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:47:46.340543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.281331+0000","last_change":"2023-12-26T21:53:45.282408+0000","last_active":"2023-12-26T21:53:46.281331+0000","last_peered":"2023-12-26T21:53:46.281331+0000","last_clean":"2023-12-26T21:53:46.281331+0000","last_became_active":"2023-12-26T21:53:45.282074+0000","last_became_peered":"2023-12-26T21:53:45.282074+0000","last_unstale":"2023-12-26T21:53:46.281331+0000","last_undegraded":"2023-12-26T21:53:46.281331+0000","last_fullsized":"2023-12-26T21:53:46.281331+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:18:52.576472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.302717+0000","last_change":"2023-12-26T21:53:46.300574+0000","last_active":"2023-12-26T21:53:46.302717+0000","last_peered":"2023-12-26T21:53:46.302717+0000","last_clean":"2023-12-26T21:53:46.302717+0000","last_became_active":"2023-12-26T21:53:46.300296+0000","last_became_peered":"2023-12-26T21:53:46.300296+0000","last_unstale":"2023-12-26T21:53:46.302717+0000","last_undegraded":"2023-12-26T21:53:46.302717+0000","last_fullsized":"2023-12-26T21:53:46.302717+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:05:51.026196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"109'2","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.318605+0000","last_change":"2023-12-26T21:53:45.280489+0000","last_active":"2023-12-26T21:53:46.318605+0000","last_peered":"2023-12-26T21:53:46.318605+0000","last_clean":"2023-12-26T21:53:46.318605+0000","last_became_active":"2023-12-26T21:53:45.279071+0000","last_became_peered":"2023-12-26T21:53:45.279071+0000","last_unstale":"2023-12-26T21:53:46.318605+0000","last_undegraded":"2023-12-26T21:53:46.318605+0000","last_fullsized":"2023-12-26T21:53:46.318605+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:19:28.851567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308510+0000","last_change":"2023-12-26T21:53:46.304923+0000","last_active":"2023-12-26T21:53:46.308510+0000","last_peered":"2023-12-26T21:53:46.308510+0000","last_clean":"2023-12-26T21:53:46.308510+0000","last_became_active":"2023-12-26T21:53:46.304621+0000","last_became_peered":"2023-12-26T21:53:46.304621+0000","last_unstale":"2023-12-26T21:53:46.308510+0000","last_undegraded":"2023-12-26T21:53:46.308510+0000","last_fullsized":"2023-12-26T21:53:46.308510+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:15:24.617194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.294726+0000","last_change":"2023-12-26T21:53:45.284941+0000","last_active":"2023-12-26T21:53:46.294726+0000","last_peered":"2023-12-26T21:53:46.294726+0000","last_clean":"2023-12-26T21:53:46.294726+0000","last_became_active":"2023-12-26T21:53:45.282717+0000","last_became_peered":"2023-12-26T21:53:45.282717+0000","last_unstale":"2023-12-26T21:53:46.294726+0000","last_undegraded":"2023-12-26T21:53:46.294726+0000","last_fullsized":"2023-12-26T21:53:46.294726+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:27.421803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.299826+0000","last_change":"2023-12-26T21:53:46.288775+0000","last_active":"2023-12-26T21:53:46.299826+0000","last_peered":"2023-12-26T21:53:46.299826+0000","last_clean":"2023-12-26T21:53:46.299826+0000","last_became_active":"2023-12-26T21:53:46.288489+0000","last_became_peered":"2023-12-26T21:53:46.288489+0000","last_unstale":"2023-12-26T21:53:46.299826+0000","last_undegraded":"2023-12-26T21:53:46.299826+0000","last_fullsized":"2023-12-26T21:53:46.299826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:48:03.925563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.291033+0000","last_change":"2023-12-26T21:53:45.285839+0000","last_active":"2023-12-26T21:53:46.291033+0000","last_peered":"2023-12-26T21:53:46.291033+0000","last_clean":"2023-12-26T21:53:46.291033+0000","last_became_active":"2023-12-26T21:53:45.285409+0000","last_became_peered":"2023-12-26T21:53:45.285409+0000","last_unstale":"2023-12-26T21:53:46.291033+0000","last_undegraded":"2023-12-26T21:53:46.291033+0000","last_fullsized":"2023-12-26T21:53:46.291033+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:11:40.376576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310802+0000","last_change":"2023-12-26T21:53:46.302758+0000","last_active":"2023-12-26T21:53:46.310802+0000","last_peered":"2023-12-26T21:53:46.310802+0000","last_clean":"2023-12-26T21:53:46.310802+0000","last_became_active":"2023-12-26T21:53:46.299461+0000","last_became_peered":"2023-12-26T21:53:46.299461+0000","last_unstale":"2023-12-26T21:53:46.310802+0000","last_undegraded":"2023-12-26T21:53:46.310802+0000","last_fullsized":"2023-12-26T21:53:46.310802+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:39:28.335893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.285877+0000","last_change":"2023-12-26T21:53:45.272282+0000","last_active":"2023-12-26T21:53:46.285877+0000","last_peered":"2023-12-26T21:53:46.285877+0000","last_clean":"2023-12-26T21:53:46.285877+0000","last_became_active":"2023-12-26T21:53:45.272019+0000","last_became_peered":"2023-12-26T21:53:45.272019+0000","last_unstale":"2023-12-26T21:53:46.285877+0000","last_undegraded":"2023-12-26T21:53:46.285877+0000","last_fullsized":"2023-12-26T21:53:46.285877+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:11:24.710650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.310423+0000","last_change":"2023-12-26T21:53:46.306701+0000","last_active":"2023-12-26T21:53:46.310423+0000","last_peered":"2023-12-26T21:53:46.310423+0000","last_clean":"2023-12-26T21:53:46.310423+0000","last_became_active":"2023-12-26T21:53:46.306450+0000","last_became_peered":"2023-12-26T21:53:46.306450+0000","last_unstale":"2023-12-26T21:53:46.310423+0000","last_undegraded":"2023-12-26T21:53:46.310423+0000","last_fullsized":"2023-12-26T21:53:46.310423+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:28:56.921774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.284986+0000","last_change":"2023-12-26T21:53:45.279374+0000","last_active":"2023-12-26T21:53:46.284986+0000","last_peered":"2023-12-26T21:53:46.284986+0000","last_clean":"2023-12-26T21:53:46.284986+0000","last_became_active":"2023-12-26T21:53:45.278479+0000","last_became_peered":"2023-12-26T21:53:45.278479+0000","last_unstale":"2023-12-26T21:53:46.284986+0000","last_undegraded":"2023-12-26T21:53:46.284986+0000","last_fullsized":"2023-12-26T21:53:46.284986+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:29.442183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.313358+0000","last_change":"2023-12-26T21:53:46.308504+0000","last_active":"2023-12-26T21:53:46.313358+0000","last_peered":"2023-12-26T21:53:46.313358+0000","last_clean":"2023-12-26T21:53:46.313358+0000","last_became_active":"2023-12-26T21:53:46.308197+0000","last_became_peered":"2023-12-26T21:53:46.308197+0000","last_unstale":"2023-12-26T21:53:46.313358+0000","last_undegraded":"2023-12-26T21:53:46.313358+0000","last_fullsized":"2023-12-26T21:53:46.313358+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:42:54.558071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286559+0000","last_change":"2023-12-26T21:53:45.275306+0000","last_active":"2023-12-26T21:53:46.286559+0000","last_peered":"2023-12-26T21:53:46.286559+0000","last_clean":"2023-12-26T21:53:46.286559+0000","last_became_active":"2023-12-26T21:53:45.274732+0000","last_became_peered":"2023-12-26T21:53:45.274732+0000","last_unstale":"2023-12-26T21:53:46.286559+0000","last_undegraded":"2023-12-26T21:53:46.286559+0000","last_fullsized":"2023-12-26T21:53:46.286559+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:01:46.076671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308046+0000","last_change":"2023-12-26T21:53:46.306183+0000","last_active":"2023-12-26T21:53:46.308046+0000","last_peered":"2023-12-26T21:53:46.308046+0000","last_clean":"2023-12-26T21:53:46.308046+0000","last_became_active":"2023-12-26T21:53:46.305918+0000","last_became_peered":"2023-12-26T21:53:46.305918+0000","last_unstale":"2023-12-26T21:53:46.308046+0000","last_undegraded":"2023-12-26T21:53:46.308046+0000","last_fullsized":"2023-12-26T21:53:46.308046+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:49:38.482545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286935+0000","last_change":"2023-12-26T21:53:45.284416+0000","last_active":"2023-12-26T21:53:46.286935+0000","last_peered":"2023-12-26T21:53:46.286935+0000","last_clean":"2023-12-26T21:53:46.286935+0000","last_became_active":"2023-12-26T21:53:45.284072+0000","last_became_peered":"2023-12-26T21:53:45.284072+0000","last_unstale":"2023-12-26T21:53:46.286935+0000","last_undegraded":"2023-12-26T21:53:46.286935+0000","last_fullsized":"2023-12-26T21:53:46.286935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:46:59.726957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.307933+0000","last_change":"2023-12-26T21:53:46.303791+0000","last_active":"2023-12-26T21:53:46.307933+0000","last_peered":"2023-12-26T21:53:46.307933+0000","last_clean":"2023-12-26T21:53:46.307933+0000","last_became_active":"2023-12-26T21:53:46.303518+0000","last_became_peered":"2023-12-26T21:53:46.303518+0000","last_unstale":"2023-12-26T21:53:46.307933+0000","last_undegraded":"2023-12-26T21:53:46.307933+0000","last_fullsized":"2023-12-26T21:53:46.307933+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:17.246701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.292408+0000","last_change":"2023-12-26T21:53:45.277961+0000","last_active":"2023-12-26T21:53:46.292408+0000","last_peered":"2023-12-26T21:53:46.292408+0000","last_clean":"2023-12-26T21:53:46.292408+0000","last_became_active":"2023-12-26T21:53:45.277682+0000","last_became_peered":"2023-12-26T21:53:45.277682+0000","last_unstale":"2023-12-26T21:53:46.292408+0000","last_undegraded":"2023-12-26T21:53:46.292408+0000","last_fullsized":"2023-12-26T21:53:46.292408+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:51:21.098860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.308052+0000","last_change":"2023-12-26T21:53:46.305187+0000","last_active":"2023-12-26T21:53:46.308052+0000","last_peered":"2023-12-26T21:53:46.308052+0000","last_clean":"2023-12-26T21:53:46.308052+0000","last_became_active":"2023-12-26T21:53:46.304710+0000","last_became_peered":"2023-12-26T21:53:46.304710+0000","last_unstale":"2023-12-26T21:53:46.308052+0000","last_undegraded":"2023-12-26T21:53:46.308052+0000","last_fullsized":"2023-12-26T21:53:46.308052+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:45.255741+0000","last_clean_scrub_stamp":"2023-12-26T21:53:45.255741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:29:09.046358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"109'1","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.319541+0000","last_change":"2023-12-26T21:53:45.277923+0000","last_active":"2023-12-26T21:53:46.319541+0000","last_peered":"2023-12-26T21:53:46.319541+0000","last_clean":"2023-12-26T21:53:46.319541+0000","last_became_active":"2023-12-26T21:53:45.277681+0000","last_became_peered":"2023-12-26T21:53:45.277681+0000","last_unstale":"2023-12-26T21:53:46.319541+0000","last_undegraded":"2023-12-26T21:53:46.319541+0000","last_fullsized":"2023-12-26T21:53:46.319541+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:53:44.249974+0000","last_clean_scrub_stamp":"2023-12-26T21:53:44.249974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:40.310455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'205","reported_seq":555,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-26T21:53:46.286580+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:53:46.286580+0000","last_peered":"2023-12-26T21:53:46.286580+0000","last_clean":"2023-12-26T21:53:46.286580+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:53:46.286580+0000","last_undegraded":"2023-12-26T21:53:46.286580+0000","last_fullsized":"2023-12-26T21:53:46.286580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542348,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302700,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069140,"statfs":{"total":96636764160,"available":96326799360,"internally_reserved":0,"allocated":1347584,"data_stored":653499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.47199999999999998,"5min":0.39600000000000002,"15min":0.38300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.39300000000000002},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.72499999999999998}]},{"osd":1,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.41899999999999998},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.69799999999999995,"15min":0.71099999999999997},"min":{"1min":0.44900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74399999999999999,"5min":1.016,"15min":1.048},"last":0.74199999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.038,"15min":1.0509999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.39100000000000001},"max":{"1min":0.85699999999999998,"5min":0.99199999999999999,"15min":1.02},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.62,"5min":0.504,"15min":0.5},"max":{"1min":0.94299999999999995,"5min":1.0189999999999999,"15min":1.052},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83499999999999996,"5min":1.036,"15min":1.036},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.747,"15min":0.746},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0129999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86399999999999999}]},{"osd":6,"last update":"Tue Dec 26 21:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.117,"15min":1.117},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.437},"max":{"1min":0.96599999999999997,"5min":1.006,"15min":1.006},"last":0.88}]}]},{"osd":6,"up_from":15,"seq":64424509644,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306800,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065040,"statfs":{"total":96636764160,"available":96322600960,"internally_reserved":0,"allocated":1351680,"data_stored":654073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.35399999999999998},"max":{"1min":0.874,"5min":1.0029999999999999,"15min":1.085},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69199999999999995,"15min":0.71799999999999997},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.34799999999999998},"max":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.624}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67800000000000005,"15min":0.70199999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.95299999999999996,"15min":0.98799999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.54200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.97599999999999998,"15min":1.054},"last":0.60299999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.93200000000000005,"15min":1.105},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.46400000000000002,"15min":0.45600000000000002},"max":{"1min":0.79200000000000004,"5min":0.92000000000000004,"15min":1.048},"last":0.48799999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.90200000000000002,"5min":1.026,"15min":1.1799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86899999999999999,"5min":0.96999999999999997,"15min":1.069},"last":0.56799999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.91100000000000003,"15min":1.137},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65100000000000002,"15min":0.64400000000000002},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.93200000000000005,"15min":1.0529999999999999},"last":0.47499999999999998}]},{"osd":5,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.95199999999999996,"15min":1.1200000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93100000000000005,"5min":1.0049999999999999,"15min":1.296},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.437},"max":{"1min":1.008,"5min":1.008,"15min":1.3380000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.56799999999999995,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.95599999999999996,"5min":1.0920000000000001,"15min":1.2010000000000001},"last":0.59099999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311472,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060368,"statfs":{"total":96636764160,"available":96317816832,"internally_reserved":0,"allocated":1941504,"data_stored":1243887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29199999999999998,"15min":0.255},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.436},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.5,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.29699999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.55700000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.84699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72499999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.70399999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.52900000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.87}]},{"osd":6,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.53700000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.88400000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76400000000000001,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.90500000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302712,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069128,"statfs":{"total":96636764160,"available":96326787072,"internally_reserved":0,"allocated":1359872,"data_stored":654987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":1.056},"last":0.377},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28299999999999997},"max":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.41099999999999998}]},{"osd":2,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47799999999999998,"15min":0.49399999999999999},"min":{"1min":0.315,"5min":0.313,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":1.006},"last":0.443},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.501}]},{"osd":3,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77900000000000003,"5min":0.81299999999999994,"15min":0.84699999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80000000000000004,"5min":0.86699999999999999,"15min":1.1870000000000001},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.752},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.042,"5min":1.042,"15min":1.411},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.76800000000000002},"min":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.45000000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.2070000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.754,"15min":0.79800000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.41999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.2569999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77100000000000002,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.452},"last":0.84399999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.214},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.79800000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98999999999999999,"5min":1.012,"15min":1.3779999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.79800000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.46400000000000002},"max":{"1min":1.018,"5min":1.042,"15min":1.4359999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.79200000000000004},"min":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.504},"max":{"1min":1.0029999999999999,"5min":1.0069999999999999,"15min":1.3240000000000001},"last":0.74199999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542349,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306792,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065048,"statfs":{"total":96636764160,"available":96322609152,"internally_reserved":0,"allocated":1343488,"data_stored":653429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.877},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68600000000000005,"5min":0.83899999999999997,"15min":0.84399999999999997},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47499999999999998,"15min":0.48499999999999999},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.65300000000000002,"5min":0.70799999999999996,"15min":0.92000000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46700000000000003,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.31,"15min":0.309},"max":{"1min":0.60299999999999998,"5min":0.75700000000000001,"15min":0.78600000000000003},"last":0.47899999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":1,"15min":1},"last":0.5},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.755,"5min":0.82299999999999995,"15min":0.85699999999999998},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.46800000000000003,"15min":0.34799999999999998},"max":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":1.284},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.76300000000000001},"min":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48499999999999999},"max":{"1min":0.95699999999999996,"5min":0.98399999999999999,"15min":1.1759999999999999},"last":1.018}]},{"osd":5,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.024,"5min":1.0600000000000001,"15min":1.2170000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":0.95499999999999996,"5min":1.014,"15min":1.1319999999999999},"last":0.91300000000000003}]},{"osd":6,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.038,"15min":1.242},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.40300000000000002},"max":{"1min":1.018,"5min":1.0609999999999999,"15min":1.272},"last":0.88200000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":1.073,"15min":1.1220000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.50800000000000001},"max":{"1min":0.999,"5min":1.0189999999999999,"15min":1.1990000000000001},"last":0.92600000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509645,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310900,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060940,"statfs":{"total":96636764160,"available":96318402560,"internally_reserved":0,"allocated":1355776,"data_stored":654075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.378}]},{"osd":1,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.32400000000000001},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.82499999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":0.92500000000000004,"5min":0.98999999999999999,"15min":1.1399999999999999},"last":0.753}]},{"osd":6,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.59099999999999997,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.038,"15min":1.1659999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.81299999999999994}]}]},{"osd":4,"up_from":14,"seq":60129542348,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306796,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065044,"statfs":{"total":96636764160,"available":96322605056,"internally_reserved":0,"allocated":1347584,"data_stored":653463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.34699999999999998,"15min":0.307},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97899999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.48499999999999999,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":1.008},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.95199999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.71599999999999997},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.40000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88900000000000001,"5min":0.97199999999999998,"15min":0.97299999999999998},"last":0.624}]},{"osd":3,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.71499999999999997},"min":{"1min":0.504,"5min":0.45700000000000002,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.78100000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.438,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:53:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.84399999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509643,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1245367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.69999999999999996},"min":{"1min":0.44500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84199999999999997,"5min":0.91200000000000003,"15min":2.3900000000000001},"last":0.749},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.4570000000000001},"last":0.73399999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68700000000000006,"15min":0.71499999999999997},"min":{"1min":0.499,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":2.6000000000000001},"last":0.625},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67800000000000005,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.4219999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.72799999999999998},"min":{"1min":0.54900000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.88500000000000001,"5min":0.94299999999999995,"15min":2.4609999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68600000000000005,"15min":0.71099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80800000000000005,"5min":0.92400000000000004,"15min":1.0569999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.47999999999999998,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":1.052,"15min":1.4890000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.73599999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.47799999999999998},"max":{"1min":0.88300000000000001,"5min":0.92200000000000004,"15min":1.081},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":2.5070000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63,"15min":0.623},"min":{"1min":0.54500000000000004,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":2.8039999999999998},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.65700000000000003,"15min":0.66400000000000003},"min":{"1min":0.505,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":1.637},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.623,"5min":0.54200000000000004,"15min":0.438},"max":{"1min":1.111,"5min":1.111,"15min":2.5579999999999998},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.61399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.97499999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.71299999999999997,"15min":0.70999999999999996},"min":{"1min":0.52200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.5980000000000001},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:53:55.963 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T21:53:55.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T21:53:56.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.195+0000 7f54479ff700 1 Processor -- start 2023-12-26T21:53:56.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.195+0000 7f54479ff700 1 -- start start 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.196+0000 7f54479ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5440159650 0x7f54400cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.196+0000 7f54479ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 0x7f54400cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.196+0000 7f54479ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f54400ced20 0x7f54400cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.196+0000 7f54479ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54400ccbc0 con 0x7f54400ced20 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.196+0000 7f54479ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54400cf6a0 con 0x7f5440159650 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.196+0000 7f54479ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54400cbfb0 con 0x7f5440159a20 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.196+0000 7f5445f9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f54400ced20 0x7f54400cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5445f9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f54400ced20 0x7f54400cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46664/0 (socket says 172.21.15.131:46664) 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5445f9c700 1 -- 172.21.15.131:0/2830419037 learned_addr learned my addr 172.21.15.131:0/2830419037 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:56.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f544579b700 1 --2- 172.21.15.131:0/2830419037 >> v2:172.21.15.190:3300/0 conn(0x7f5440159650 0x7f54400cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5444f9a700 1 --2- 172.21.15.131:0/2830419037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 0x7f54400cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5445f9c700 1 -- 172.21.15.131:0/2830419037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 msgr2=0x7f54400cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5445f9c700 1 --2- 172.21.15.131:0/2830419037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 0x7f54400cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5445f9c700 1 -- 172.21.15.131:0/2830419037 >> v2:172.21.15.190:3300/0 conn(0x7f5440159650 msgr2=0x7f54400cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5445f9c700 1 --2- 172.21.15.131:0/2830419037 >> v2:172.21.15.190:3300/0 conn(0x7f5440159650 0x7f54400cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5445f9c700 1 -- 172.21.15.131:0/2830419037 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400cc290 con 0x7f54400ced20 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5445f9c700 1 --2- 172.21.15.131:0/2830419037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 0x7f54400cf160 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5440088ef0 tx=0x7f5438009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95451eff84da1903 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.197+0000 7f5437fff700 1 -- 172.21.15.131:0/2830419037 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543800e070 con 0x7f54400ced20 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f5437fff700 1 -- 172.21.15.131:0/2830419037 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543800b970 con 0x7f54400ced20 2023-12-26T21:53:56.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f5437fff700 1 -- 172.21.15.131:0/2830419037 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380092e0 con 0x7f54400ced20 2023-12-26T21:53:56.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f54479ff700 1 -- 172.21.15.131:0/2830419037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 msgr2=0x7f54400cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f54479ff700 1 --2- 172.21.15.131:0/2830419037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 0x7f54400cf160 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5440088ef0 tx=0x7f5438009840 comp rx=0 tx=0).stop 2023-12-26T21:53:56.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f54479ff700 1 -- 172.21.15.131:0/2830419037 shutdown_connections 2023-12-26T21:53:56.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f54479ff700 1 --2- 172.21.15.131:0/2830419037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 0x7f54400cf160 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f54479ff700 1 --2- 172.21.15.131:0/2830419037 >> v2:172.21.15.190:3300/0 conn(0x7f5440159650 0x7f54400cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f54479ff700 1 --2- 172.21.15.131:0/2830419037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 0x7f54400cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f54479ff700 1 -- 172.21.15.131:0/2830419037 >> 172.21.15.131:0/2830419037 conn(0x7f5440099980 msgr2=0x7f544006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:56.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f54479ff700 1 -- 172.21.15.131:0/2830419037 shutdown_connections 2023-12-26T21:53:56.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.198+0000 7f54479ff700 1 -- 172.21.15.131:0/2830419037 wait complete. 2023-12-26T21:53:56.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.200+0000 7f54479ff700 1 Processor -- start 2023-12-26T21:53:56.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.200+0000 7f54479ff700 1 -- start start 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f54479ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 0x7f5440158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f54479ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5440159650 0x7f54400c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f54479ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 0x7f54400c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f54479ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54400ca3c0 con 0x7f54400ced20 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f54479ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544016a650 con 0x7f5440159650 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f54479ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54400c97b0 con 0x7f5440159a20 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f544579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 0x7f5440158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f544579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 0x7f5440158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46668/0 (socket says 172.21.15.131:46668) 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f544579b700 1 -- 172.21.15.131:0/236445291 learned_addr learned my addr 172.21.15.131:0/236445291 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f5445f9c700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 0x7f54400c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f5444f9a700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5440159650 0x7f54400c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f544579b700 1 -- 172.21.15.131:0/236445291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 msgr2=0x7f54400c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f544579b700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 0x7f54400c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f544579b700 1 -- 172.21.15.131:0/236445291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5440159650 msgr2=0x7f54400c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f544579b700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5440159650 0x7f54400c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.201+0000 7f544579b700 1 -- 172.21.15.131:0/236445291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5438009ae0 con 0x7f54400ced20 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.202+0000 7f544579b700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 0x7f5440158a40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f543c00ec80 tx=0x7f543c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.202+0000 7f54357fa700 1 -- 172.21.15.131:0/236445291 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c00c850 con 0x7f54400ced20 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.202+0000 7f54357fa700 1 -- 172.21.15.131:0/236445291 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c00c9b0 con 0x7f54400ced20 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.202+0000 7f54357fa700 1 -- 172.21.15.131:0/236445291 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c00cd30 con 0x7f54400ced20 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.202+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54400c9a90 con 0x7f54400ced20 2023-12-26T21:53:56.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.202+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54400ca050 con 0x7f54400ced20 2023-12-26T21:53:56.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.203+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544015bbd0 con 0x7f54400ced20 2023-12-26T21:53:56.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.204+0000 7f54357fa700 1 -- 172.21.15.131:0/236445291 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f543c013680 con 0x7f54400ced20 2023-12-26T21:53:56.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.205+0000 7f54357fa700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5424066f30 0x7f54240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.205+0000 7f54357fa700 1 -- 172.21.15.131:0/236445291 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f543c084de0 con 0x7f54400ced20 2023-12-26T21:53:56.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.205+0000 7f5444f9a700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5424066f30 0x7f54240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.206+0000 7f5444f9a700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5424066f30 0x7f54240693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5440148d10 tx=0x7f54300091a0 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:56.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.209+0000 7f54357fa700 1 -- 172.21.15.131:0/236445291 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f543c051e20 con 0x7f54400ced20 2023-12-26T21:53:56.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.482+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f54400657e0 con 0x7f54400ced20 2023-12-26T21:53:56.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.486+0000 7f54357fa700 1 -- 172.21.15.131:0/236445291 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f543c0210e0 con 0x7f54400ced20 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5424066f30 msgr2=0x7f54240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5424066f30 0x7f54240693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5440148d10 tx=0x7f54300091a0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 msgr2=0x7f5440158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 0x7f5440158a40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f543c00ec80 tx=0x7f543c009120 comp rx=0 tx=0).stop 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 shutdown_connections 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5440159a20 0x7f54400c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5424066f30 0x7f54240693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54400ced20 0x7f5440158a40 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 --2- 172.21.15.131:0/236445291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5440159650 0x7f54400c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 >> 172.21.15.131:0/236445291 conn(0x7f5440099980 msgr2=0x7f54400b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 shutdown_connections 2023-12-26T21:53:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.489+0000 7f54479ff700 1 -- 172.21.15.131:0/236445291 wait complete. 2023-12-26T21:53:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T21:53:56.507 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.697+0000 7f09c57ff700 1 Processor -- start 2023-12-26T21:53:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.697+0000 7f09c57ff700 1 -- start start 2023-12-26T21:53:56.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.698+0000 7f09c57ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09c006e480 0x7f09c00abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.698+0000 7f09c57ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09c00aa480 0x7f09c00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.698+0000 7f09c57ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c015a8c0 con 0x7f09c0159820 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.698+0000 7f09c57ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c015ad50 con 0x7f09c00aa480 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.698+0000 7f09c57ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c0159f90 con 0x7f09c006e480 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.699+0000 7f09bf7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09c00aa480 0x7f09c00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.699+0000 7f09bf7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09c00aa480 0x7f09c00675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46676/0 (socket says 172.21.15.131:46676) 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.699+0000 7f09bf7fe700 1 -- 172.21.15.131:0/3560060103 learned_addr learned my addr 172.21.15.131:0/3560060103 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.699+0000 7f09be7fc700 1 --2- 172.21.15.131:0/3560060103 >> v2:172.21.15.190:3300/0 conn(0x7f09c006e480 0x7f09c00abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.699+0000 7f09bf7fe700 1 -- 172.21.15.131:0/3560060103 >> v2:172.21.15.190:3300/0 conn(0x7f09c006e480 msgr2=0x7f09c00abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.699+0000 7f09bf7fe700 1 --2- 172.21.15.131:0/3560060103 >> v2:172.21.15.190:3300/0 conn(0x7f09c006e480 0x7f09c00abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.699+0000 7f09bf7fe700 1 -- 172.21.15.131:0/3560060103 >> v1:172.21.15.190:6789/0 conn(0x7f09c0159820 legacy=0x7f09c006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.699+0000 7f09bf7fe700 1 -- 172.21.15.131:0/3560060103 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c015a270 con 0x7f09c00aa480 2023-12-26T21:53:56.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.699+0000 7f09bf7fe700 1 --2- 172.21.15.131:0/3560060103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c00aa480 0x7f09c00675b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f09c00af3f0 tx=0x7f09b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43f7cbbd731ac8c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:56.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09bdffb700 1 -- 172.21.15.131:0/3560060103 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b800e070 con 0x7f09c00aa480 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09bdffb700 1 -- 172.21.15.131:0/3560060103 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b800ba50 con 0x7f09c00aa480 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09bdffb700 1 -- 172.21.15.131:0/3560060103 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b80092e0 con 0x7f09c00aa480 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09c57ff700 1 -- 172.21.15.131:0/3560060103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c00aa480 msgr2=0x7f09c00675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09c57ff700 1 --2- 172.21.15.131:0/3560060103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c00aa480 0x7f09c00675b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f09c00af3f0 tx=0x7f09b8009840 comp rx=0 tx=0).stop 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09c57ff700 1 -- 172.21.15.131:0/3560060103 shutdown_connections 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09c57ff700 1 --2- 172.21.15.131:0/3560060103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c00aa480 0x7f09c00675b0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09c57ff700 1 --2- 172.21.15.131:0/3560060103 >> v2:172.21.15.190:3300/0 conn(0x7f09c006e480 0x7f09c00abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09c57ff700 1 -- 172.21.15.131:0/3560060103 >> 172.21.15.131:0/3560060103 conn(0x7f09c006c290 msgr2=0x7f09c00ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09c57ff700 1 -- 172.21.15.131:0/3560060103 shutdown_connections 2023-12-26T21:53:56.702 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.700+0000 7f09c57ff700 1 -- 172.21.15.131:0/3560060103 wait complete. 2023-12-26T21:53:56.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.702+0000 7f09c57ff700 1 Processor -- start 2023-12-26T21:53:56.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.703+0000 7f09c57ff700 1 -- start start 2023-12-26T21:53:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09c57ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c006e480 0x7f09c0157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09c57ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c00aa480 0x7f09c014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09c57ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159820 0x7f09c014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09c57ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00c18c0 con 0x7f09c0159820 2023-12-26T21:53:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09c57ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c014aa90 con 0x7f09c006e480 2023-12-26T21:53:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09c57ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c016d330 con 0x7f09c00aa480 2023-12-26T21:53:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09bf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159820 0x7f09c014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09bf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159820 0x7f09c014a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46690/0 (socket says 172.21.15.131:46690) 2023-12-26T21:53:56.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09bf7fe700 1 -- 172.21.15.131:0/3971426630 learned_addr learned my addr 172.21.15.131:0/3971426630 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09be7fc700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c00aa480 0x7f09c014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09beffd700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c006e480 0x7f09c0157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09bf7fe700 1 -- 172.21.15.131:0/3971426630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c00aa480 msgr2=0x7f09c014a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09bf7fe700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c00aa480 0x7f09c014a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09bf7fe700 1 -- 172.21.15.131:0/3971426630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c006e480 msgr2=0x7f09c0157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09bf7fe700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c006e480 0x7f09c0157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09bf7fe700 1 -- 172.21.15.131:0/3971426630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8009ae0 con 0x7f09c0159820 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.704+0000 7f09beffd700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c006e480 0x7f09c0157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.705+0000 7f09bf7fe700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159820 0x7f09c014a550 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f09c006d0a0 tx=0x7f09b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.705+0000 7f09a6ffd700 1 -- 172.21.15.131:0/3971426630 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8008e80 con 0x7f09c0159820 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.705+0000 7f09a6ffd700 1 -- 172.21.15.131:0/3971426630 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b800be60 con 0x7f09c0159820 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.705+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c016d610 con 0x7f09c0159820 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.705+0000 7f09a6ffd700 1 -- 172.21.15.131:0/3971426630 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8026690 con 0x7f09c0159820 2023-12-26T21:53:56.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.705+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c016dbd0 con 0x7f09c0159820 2023-12-26T21:53:56.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.706+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c0159bf0 con 0x7f09c0159820 2023-12-26T21:53:56.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.707+0000 7f09a6ffd700 1 -- 172.21.15.131:0/3971426630 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f09b8003040 con 0x7f09c0159820 2023-12-26T21:53:56.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.708+0000 7f09a6ffd700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09ac066f30 0x7f09ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:56.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.708+0000 7f09a6ffd700 1 -- 172.21.15.131:0/3971426630 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f09b80866e0 con 0x7f09c0159820 2023-12-26T21:53:56.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.708+0000 7f09beffd700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09ac066f30 0x7f09ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:56.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.709+0000 7f09beffd700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09ac066f30 0x7f09ac0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09b00017a0 tx=0x7f09b0009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:56.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.712+0000 7f09a6ffd700 1 -- 172.21.15.131:0/3971426630 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09b80537c0 con 0x7f09c0159820 2023-12-26T21:53:56.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.949+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09c00657e0 con 0x7f09c0159820 2023-12-26T21:53:56.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.949+0000 7f09a6ffd700 1 -- 172.21.15.131:0/3971426630 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f09b8009940 con 0x7f09c0159820 2023-12-26T21:53:56.950 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:53:56.950 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2416144644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2416144644},{"type":"v1","addr":"172.21.15.131:6835","nonce":2416144644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/559791617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":559791617},{"type":"v1","addr":"172.21.15.190:6834","nonce":559791617}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6901,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3954759429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3954759429},{"type":"v1","addr":"172.21.15.190:6835","nonce":3954759429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:45.744809+0000","modified":"2023-12-26T21:53:49.296563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6834/3631709496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3631709496},{"type":"v1","addr":"172.21.15.131:6834","nonce":3631709496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09ac066f30 msgr2=0x7f09ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09ac066f30 0x7f09ac0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09b00017a0 tx=0x7f09b0009040 comp rx=0 tx=0).stop 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159820 msgr2=0x7f09c014a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159820 0x7f09c014a550 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f09c006d0a0 tx=0x7f09b8018000 comp rx=0 tx=0).stop 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 shutdown_connections 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09ac066f30 0x7f09ac0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159820 0x7f09c014a550 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c006e480 0x7f09c0157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 --2- 172.21.15.131:0/3971426630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c00aa480 0x7f09c014a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:56.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 >> 172.21.15.131:0/3971426630 conn(0x7f09c006c290 msgr2=0x7f09c015b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:56.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 shutdown_connections 2023-12-26T21:53:56.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:56.952+0000 7f09c57ff700 1 -- 172.21.15.131:0/3971426630 wait complete. 2023-12-26T21:53:56.954 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 50 2023-12-26T21:53:57.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T21:53:45.744809+0000', 'modified': '2023-12-26T21:53:49.296563+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7146}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7146': {'gid': 7146, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6834/3631709496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 3631709496}, {'type': 'v1', 'addr': '172.21.15.131:6834', 'nonce': 3631709496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T21:53:57.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T21:53:57.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-26T21:53:57.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.565+0000 7f6175dff700 1 Processor -- start 2023-12-26T21:53:57.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.565+0000 7f6175dff700 1 -- start start 2023-12-26T21:53:57.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.566+0000 7f6175dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f617015d9b0 0x7f617015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.566+0000 7f6175dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700b07f0 con 0x7f61700cc580 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.566+0000 7f6175dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700b0d70 con 0x7f61700cc950 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.566+0000 7f6175dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61700680f0 con 0x7f617015d9b0 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.566+0000 7f616f7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f61700cc580 0x7f61700b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37862/0 (socket says 172.21.15.131:37862) 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.566+0000 7f616f7fe700 1 -- 172.21.15.131:0/4087874356 learned_addr learned my addr 172.21.15.131:0/4087874356 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.566+0000 7f616ffff700 1 --2- 172.21.15.131:0/4087874356 >> v2:172.21.15.131:3300/0 conn(0x7f617015d9b0 0x7f617015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616e7fc700 1 -- 172.21.15.131:0/4087874356 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502374763 0 0) 0x7f6160001110 con 0x7f61700cc580 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616e7fc700 1 -- 172.21.15.131:0/4087874356 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6158003570 con 0x7f61700cc580 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616ffff700 1 -- 172.21.15.131:0/4087874356 >> v1:172.21.15.190:6789/0 conn(0x7f61700cc950 legacy=0x7f617006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616e7fc700 1 -- 172.21.15.131:0/4087874356 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958898589 0 0) 0x7f61700b0d70 con 0x7f61700cc950 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616ffff700 1 -- 172.21.15.131:0/4087874356 >> v1:172.21.15.131:6789/0 conn(0x7f61700cc580 legacy=0x7f61700b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616ffff700 1 -- 172.21.15.131:0/4087874356 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700675b0 con 0x7f617015d9b0 2023-12-26T21:53:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616ffff700 1 --2- 172.21.15.131:0/4087874356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 0x7f617015de30 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6168009e60 tx=0x7f6168009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc294c03c5d9777a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616e7fc700 1 -- 172.21.15.131:0/4087874356 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800e070 con 0x7f617015d9b0 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616e7fc700 1 -- 172.21.15.131:0/4087874356 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800bb60 con 0x7f617015d9b0 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.567+0000 7f616e7fc700 1 -- 172.21.15.131:0/4087874356 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168012430 con 0x7f617015d9b0 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.568+0000 7f6175dff700 1 -- 172.21.15.131:0/4087874356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 msgr2=0x7f617015de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.568+0000 7f6175dff700 1 --2- 172.21.15.131:0/4087874356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 0x7f617015de30 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6168009e60 tx=0x7f6168009780 comp rx=0 tx=0).stop 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.568+0000 7f6175dff700 1 -- 172.21.15.131:0/4087874356 shutdown_connections 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.568+0000 7f6175dff700 1 --2- 172.21.15.131:0/4087874356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 0x7f617015de30 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.568+0000 7f6175dff700 1 -- 172.21.15.131:0/4087874356 >> 172.21.15.131:0/4087874356 conn(0x7f61700686c0 msgr2=0x7f6170068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.568+0000 7f6175dff700 1 -- 172.21.15.131:0/4087874356 shutdown_connections 2023-12-26T21:53:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.568+0000 7f6175dff700 1 -- 172.21.15.131:0/4087874356 wait complete. 2023-12-26T21:53:57.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.570+0000 7f6175dff700 1 Processor -- start 2023-12-26T21:53:57.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.570+0000 7f6175dff700 1 -- start start 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f6175dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700cc580 0x7f61700aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f6175dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700cc950 0x7f61701580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f6175dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 0x7f6170158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f6175dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700ae590 con 0x7f617015d9b0 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f6175dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700ad980 con 0x7f61700cc580 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f6175dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61700adc60 con 0x7f61700cc950 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 0x7f6170158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 0x7f6170158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46718/0 (socket says 172.21.15.131:46718) 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616ffff700 1 -- 172.21.15.131:0/1698509452 learned_addr learned my addr 172.21.15.131:0/1698509452 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616effd700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700cc950 0x7f61701580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616f7fe700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700cc580 0x7f61700aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616ffff700 1 -- 172.21.15.131:0/1698509452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700cc950 msgr2=0x7f61701580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616ffff700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700cc950 0x7f61701580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616ffff700 1 -- 172.21.15.131:0/1698509452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700cc580 msgr2=0x7f61700aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616ffff700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700cc580 0x7f61700aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.571+0000 7f616ffff700 1 -- 172.21.15.131:0/1698509452 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168009a20 con 0x7f617015d9b0 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.572+0000 7f616ffff700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 0x7f6170158630 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f617006b770 tx=0x7f6168019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.572+0000 7f61577fe700 1 -- 172.21.15.131:0/1698509452 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168026480 con 0x7f617015d9b0 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.572+0000 7f61577fe700 1 -- 172.21.15.131:0/1698509452 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6168026640 con 0x7f617015d9b0 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.572+0000 7f6175dff700 1 -- 172.21.15.131:0/1698509452 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61700adf40 con 0x7f617015d9b0 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.572+0000 7f61577fe700 1 -- 172.21.15.131:0/1698509452 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168026a00 con 0x7f617015d9b0 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.572+0000 7f6175dff700 1 -- 172.21.15.131:0/1698509452 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61701481a0 con 0x7f617015d9b0 2023-12-26T21:53:57.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.574+0000 7f61577fe700 1 -- 172.21.15.131:0/1698509452 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f616801c400 con 0x7f617015d9b0 2023-12-26T21:53:57.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.575+0000 7f61577fe700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f615c066f30 0x7f615c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.575+0000 7f61577fe700 1 -- 172.21.15.131:0/1698509452 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f61680866b0 con 0x7f617015d9b0 2023-12-26T21:53:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.575+0000 7f616f7fe700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f615c066f30 0x7f615c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.576+0000 7f616f7fe700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f615c066f30 0x7f615c0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6160001270 tx=0x7f6160009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.577+0000 7f61844e9b80 1 Processor -- start 2023-12-26T21:53:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.577+0000 7f61844e9b80 1 -- start start 2023-12-26T21:53:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6783370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 0x55d2c677ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c679f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f61844e9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c66b13e0 con 0x55d2c677f170 2023-12-26T21:53:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f61844e9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c66aebe0 con 0x55d2c67838b0 2023-12-26T21:53:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f61844e9b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2c66aa9d0 con 0x55d2c6782fa0 2023-12-26T21:53:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c679f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c679f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46734/0 (socket says 172.21.15.131:46734) 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616ffff700 1 -- 172.21.15.131:0/1587240650 learned_addr learned my addr 172.21.15.131:0/1587240650 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616effd700 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 0x55d2c677ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616ffff700 1 -- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 msgr2=0x55d2c6783370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616f7fe700 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6783370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616ffff700 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6783370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616ffff700 1 -- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 msgr2=0x55d2c677ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616ffff700 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 0x55d2c677ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616ffff700 1 -- 172.21.15.131:0/1587240650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2c653e360 con 0x55d2c677f170 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616effd700 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 0x55d2c677ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.578+0000 7f616f7fe700 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6783370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f616ffff700 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c679f360 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f61680538b0 tx=0x7f6168084900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53df2a00f10203eb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f6154ff9700 1 -- 172.21.15.131:0/1587240650 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800e050 con 0x55d2c677f170 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f6154ff9700 1 -- 172.21.15.131:0/1587240650 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61680536f0 con 0x55d2c677f170 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f6154ff9700 1 -- 172.21.15.131:0/1587240650 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680573b0 con 0x55d2c677f170 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f61844e9b80 1 -- 172.21.15.131:0/1587240650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 msgr2=0x55d2c679f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f61844e9b80 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c679f360 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f61680538b0 tx=0x7f6168084900 comp rx=0 tx=0).stop 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f61844e9b80 1 -- 172.21.15.131:0/1587240650 shutdown_connections 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f61844e9b80 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c679f360 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f61844e9b80 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6783370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.579+0000 7f61844e9b80 1 --2- 172.21.15.131:0/1587240650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 0x55d2c677ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.580+0000 7f61844e9b80 1 -- 172.21.15.131:0/1587240650 >> 172.21.15.131:0/1587240650 conn(0x55d2c6759440 msgr2=0x55d2c67a4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.580+0000 7f61844e9b80 1 -- 172.21.15.131:0/1587240650 shutdown_connections 2023-12-26T21:53:57.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.580+0000 7f61844e9b80 1 -- 172.21.15.131:0/1587240650 wait complete. 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.580+0000 7f61844e9b80 1 -- start start 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.581+0000 7f61844e9b80 1 Processor -- start 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.581+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c6785540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.581+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6798820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.581+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 0x55d2c6798d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.581+0000 7f61844e9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c66571b0 con 0x55d2c677f170 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.581+0000 7f61844e9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c6656e50 con 0x55d2c67838b0 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.581+0000 7f61844e9b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2c65a3e40 con 0x55d2c6782fa0 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f61844e9b80 10 client.0 _ll_register_callbacks cb 0x55d2c65a67b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:53:57.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c6785540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c6785540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46738/0 (socket says 172.21.15.131:46738) 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616f7fe700 1 -- 172.21.15.131:0/404265409 learned_addr learned my addr 172.21.15.131:0/404265409 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616ffff700 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 0x55d2c6798d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616effd700 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6798820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616f7fe700 1 -- 172.21.15.131:0/404265409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 msgr2=0x55d2c6798820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616f7fe700 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6798820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616f7fe700 1 -- 172.21.15.131:0/404265409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 msgr2=0x55d2c6798d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616f7fe700 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 0x55d2c6798d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616f7fe700 1 -- 172.21.15.131:0/404265409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61701481a0 con 0x55d2c677f170 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f616f7fe700 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c6785540 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6160007250 tx=0x7f6160017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb279ecd3fba919a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.582+0000 7f614dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f614dffb700 1 -- 172.21.15.131:0/404265409 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000e940 con 0x55d2c677f170 2023-12-26T21:53:57.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f614dffb700 1 -- 172.21.15.131:0/404265409 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f6160022c70 con 0x55d2c677f170 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f61844e9b80 1 -- 172.21.15.131:0/404265409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2c672beb0 con 0x55d2c677f170 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f614dffb700 1 -- 172.21.15.131:0/404265409 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6160022e50 con 0x55d2c677f170 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f61844e9b80 10 client.7326 fetch_fsmap learned FSMap version 50 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f61844e9b80 1 -- 172.21.15.131:0/404265409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55d2c64eab10 con 0x55d2c677f170 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f614dffb700 1 -- 172.21.15.131:0/404265409 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2c672beb0 con 0x55d2c677f170 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f614dffb700 1 -- 172.21.15.131:0/404265409 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f616000ecb0 con 0x55d2c677f170 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f61844e9b80 10 client.7326 fetch_fsmap finished waiting for FSMap version 50 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f61844e9b80 10 client.7326 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f61844e9b80 20 client.7326 populate_metadata read hostname 'smithi131' 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f61844e9b80 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] conn(0x55d2c67885e0 0x55d2c67889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f61844e9b80 4 client.7326 mds_command: new command op to 7146 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.583+0000 7f61844e9b80 1 -- 172.21.15.131:0/404265409 --> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2c6681880 con 0x55d2c67885e0 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.584+0000 7f616effd700 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] conn(0x55d2c67885e0 0x55d2c67889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.584+0000 7f614dffb700 10 client.7326 ms_handle_connect on v2:172.21.15.131:6832/3631709496 2023-12-26T21:53:57.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.584+0000 7f616effd700 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] conn(0x55d2c67885e0 0x55d2c67889b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50740ec2a539aa server_cookie=e07819c957ed077c in_seq=0 out_seq=0 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f614dffb700 1 -- 172.21.15.131:0/404265409 <== mds.0 v2:172.21.15.131:6832/3631709496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d2c6681880 con 0x55d2c67885e0 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f614dffb700 10 client.7326 handle_command_reply: tid=0 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 client.7326 shutdown 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 -- 172.21.15.131:0/404265409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 msgr2=0x55d2c6785540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c6785540 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6160007250 tx=0x7f6160017040 comp rx=0 tx=0).stop 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 -- 172.21.15.131:0/404265409 shutdown_connections 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] conn(0x55d2c67885e0 0x55d2c67889b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c67838b0 0x55d2c6798d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c677f170 0x55d2c6785540 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f614dffb700 0 client.7326 ms_handle_reset on v2:172.21.15.131:6832/3631709496 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 --2- 172.21.15.131:0/404265409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6798820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 -- 172.21.15.131:0/404265409 >> 172.21.15.131:0/404265409 conn(0x55d2c6759440 msgr2=0x55d2c6759810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 -- 172.21.15.131:0/404265409 shutdown_connections 2023-12-26T21:53:57.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.585+0000 7f61844e9b80 1 -- 172.21.15.131:0/404265409 wait complete. 2023-12-26T21:53:57.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.586+0000 7f61844e9b80 20 client.7326 trim_cache size 0 max 16384 2023-12-26T21:53:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.603+0000 7f61844e9b80 1 Processor -- start 2023-12-26T21:53:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.603+0000 7f61844e9b80 1 -- start start 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.603+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c677f170 0x55d2c67510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6781590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f61844e9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c67580b0 con 0x55d2c67838b0 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f61844e9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c6782c20 con 0x55d2c677f170 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f61844e9b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2c6787200 con 0x55d2c6782fa0 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f616ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f616ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46750/0 (socket says 172.21.15.131:46750) 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f616ffff700 1 -- 172.21.15.131:0/2236126145 learned_addr learned my addr 172.21.15.131:0/2236126145 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f616ffff700 1 -- 172.21.15.131:0/2236126145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 msgr2=0x55d2c6781590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f616ffff700 1 --2- 172.21.15.131:0/2236126145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6781590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f616ffff700 1 -- 172.21.15.131:0/2236126145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c677f170 msgr2=0x55d2c67510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f616ffff700 1 --2- 172.21.15.131:0/2236126145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c677f170 0x55d2c67510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f616ffff700 1 -- 172.21.15.131:0/2236126145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2c6679290 con 0x55d2c67838b0 2023-12-26T21:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f616ffff700 1 --2- 172.21.15.131:0/2236126145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f61680570b0 tx=0x7f616806a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f0d4a0886deee1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.604+0000 7f61557fa700 1 -- 172.21.15.131:0/2236126145 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616805a600 con 0x55d2c67838b0 2023-12-26T21:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61557fa700 1 -- 172.21.15.131:0/2236126145 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616805e350 con 0x55d2c67838b0 2023-12-26T21:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61557fa700 1 -- 172.21.15.131:0/2236126145 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680573b0 con 0x55d2c67838b0 2023-12-26T21:53:57.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61844e9b80 1 -- 172.21.15.131:0/2236126145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 msgr2=0x55d2c6781ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61844e9b80 1 --2- 172.21.15.131:0/2236126145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f61680570b0 tx=0x7f616806a000 comp rx=0 tx=0).stop 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61844e9b80 1 -- 172.21.15.131:0/2236126145 shutdown_connections 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61844e9b80 1 --2- 172.21.15.131:0/2236126145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61844e9b80 1 --2- 172.21.15.131:0/2236126145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c677f170 0x55d2c67510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61844e9b80 1 --2- 172.21.15.131:0/2236126145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6781590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61844e9b80 1 -- 172.21.15.131:0/2236126145 >> 172.21.15.131:0/2236126145 conn(0x55d2c6759440 msgr2=0x55d2c67aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61844e9b80 1 -- 172.21.15.131:0/2236126145 shutdown_connections 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.605+0000 7f61844e9b80 1 -- 172.21.15.131:0/2236126145 wait complete. 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.606+0000 7f61844e9b80 1 -- start start 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.606+0000 7f61844e9b80 1 Processor -- start 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c677f170 0x55d2c6805da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6781590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f61844e9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f61844e9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c679c720 con 0x55d2c67838b0 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f61844e9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c679fe80 con 0x55d2c677f170 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f61844e9b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2c6751830 con 0x55d2c6782fa0 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f61844e9b80 10 client.0 _ll_register_callbacks cb 0x7f61600059c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f616ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f616ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46764/0 (socket says 172.21.15.131:46764) 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f616ffff700 1 -- 172.21.15.131:0/4187172315 learned_addr learned my addr 172.21.15.131:0/4187172315 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f616effd700 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6781590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.607+0000 7f616f7fe700 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c677f170 0x55d2c6805da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f616ffff700 1 -- 172.21.15.131:0/4187172315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 msgr2=0x55d2c6781590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f616ffff700 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6781590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f616ffff700 1 -- 172.21.15.131:0/4187172315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c677f170 msgr2=0x55d2c6805da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f616ffff700 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c677f170 0x55d2c6805da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f616ffff700 1 -- 172.21.15.131:0/4187172315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2c653e360 con 0x55d2c67838b0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f616ffff700 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6168009490 tx=0x7f6168080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4369cd0da0847a26 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f614dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f614dffb700 1 -- 172.21.15.131:0/4187172315 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800e040 con 0x55d2c67838b0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f614dffb700 1 -- 172.21.15.131:0/4187172315 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f6168083050 con 0x55d2c67838b0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f61844e9b80 1 -- 172.21.15.131:0/4187172315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f617015a9e0 con 0x55d2c67838b0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f614dffb700 1 -- 172.21.15.131:0/4187172315 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f616805a600 con 0x55d2c67838b0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f61844e9b80 10 client.7332 fetch_fsmap learned FSMap version 50 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f61844e9b80 1 -- 172.21.15.131:0/4187172315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55d2c6542c80 con 0x55d2c67838b0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.608+0000 7f614dffb700 1 -- 172.21.15.131:0/4187172315 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617015a9e0 con 0x55d2c67838b0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f614dffb700 1 -- 172.21.15.131:0/4187172315 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f61680568a0 con 0x55d2c67838b0 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f61844e9b80 10 client.7332 fetch_fsmap finished waiting for FSMap version 50 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f61844e9b80 10 client.7332 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f61844e9b80 20 client.7332 populate_metadata read hostname 'smithi131' 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f61844e9b80 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] conn(0x55d2c67842c0 0x55d2c6784690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f61844e9b80 4 client.7332 mds_command: new command op to 7146 tid=0[{"prefix": "status"}] 2023-12-26T21:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f61844e9b80 1 -- 172.21.15.131:0/4187172315 --> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d2c674f490 con 0x55d2c67842c0 2023-12-26T21:53:57.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f616f7fe700 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] conn(0x55d2c67842c0 0x55d2c6784690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f614dffb700 10 client.7332 ms_handle_connect on v2:172.21.15.131:6832/3631709496 2023-12-26T21:53:57.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.609+0000 7f616f7fe700 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] conn(0x55d2c67842c0 0x55d2c6784690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269f97fbd4e84c1d server_cookie=835e4c353a43789d in_seq=0 out_seq=0 2023-12-26T21:53:57.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f614dffb700 1 -- 172.21.15.131:0/4187172315 <== mds.0 v2:172.21.15.131:6832/3631709496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x55d2c674f490 con 0x55d2c67842c0 2023-12-26T21:53:57.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f614dffb700 10 client.7332 handle_command_reply: tid=0 2023-12-26T21:53:57.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 client.7332 shutdown 2023-12-26T21:53:57.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 -- 172.21.15.131:0/4187172315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 msgr2=0x55d2c6781ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6168009490 tx=0x7f6168080000 comp rx=0 tx=0).stop 2023-12-26T21:53:57.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 -- 172.21.15.131:0/4187172315 shutdown_connections 2023-12-26T21:53:57.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.131:6832/3631709496,v1:172.21.15.131:6834/3631709496] conn(0x55d2c67842c0 0x55d2c6784690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2c67838b0 0x55d2c6781ad0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2c677f170 0x55d2c6805da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f614dffb700 0 client.7332 ms_handle_reset on v2:172.21.15.131:6832/3631709496 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 --2- 172.21.15.131:0/4187172315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d2c6782fa0 0x55d2c6781590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 -- 172.21.15.131:0/4187172315 >> 172.21.15.131:0/4187172315 conn(0x55d2c6759440 msgr2=0x55d2c6759940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 -- 172.21.15.131:0/4187172315 shutdown_connections 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 1 -- 172.21.15.131:0/4187172315 wait complete. 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.610+0000 7f61844e9b80 20 client.7332 trim_cache size 0 max 16384 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7146, 2023-12-26T21:53:57.615 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T21:53:57.616 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T21:53:57.616 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T21:53:57.616 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 11.32182268, 2023-12-26T21:53:57.616 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 50, 2023-12-26T21:53:57.616 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 110, 2023-12-26T21:53:57.616 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 110, 2023-12-26T21:53:57.616 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 17.053274111 2023-12-26T21:53:57.616 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:53:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 -- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f615c066f30 msgr2=0x7f615c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f615c066f30 0x7f615c0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6160001270 tx=0x7f6160009040 comp rx=0 tx=0).stop 2023-12-26T21:53:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 -- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 msgr2=0x7f6170158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 0x7f6170158630 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f617006b770 tx=0x7f6168019000 comp rx=0 tx=0).stop 2023-12-26T21:53:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 -- 172.21.15.131:0/1698509452 shutdown_connections 2023-12-26T21:53:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f615c066f30 0x7f615c0693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f617015d9b0 0x7f6170158630 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700cc580 0x7f61700aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 --2- 172.21.15.131:0/1698509452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700cc950 0x7f61701580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 -- 172.21.15.131:0/1698509452 >> 172.21.15.131:0/1698509452 conn(0x7f61700686c0 msgr2=0x7f61700ab190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:57.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 -- 172.21.15.131:0/1698509452 shutdown_connections 2023-12-26T21:53:57.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.611+0000 7f6175dff700 1 -- 172.21.15.131:0/1698509452 wait complete. 2023-12-26T21:53:57.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:53:57.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.818+0000 7f9539fff700 1 Processor -- start 2023-12-26T21:53:57.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.819+0000 7f9539fff700 1 -- start start 2023-12-26T21:53:57.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.820+0000 7f9539fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f953413c510 0x7f95340c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.820+0000 7f9539fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95340686c0 con 0x7f95340cc810 2023-12-26T21:53:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.820+0000 7f9539fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9534069520 con 0x7f953406b8c0 2023-12-26T21:53:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.820+0000 7f9539fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9534068910 con 0x7f953413c510 2023-12-26T21:53:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.820+0000 7f9533fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f95340cc810 0x7f95340699b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37874/0 (socket says 172.21.15.131:37874) 2023-12-26T21:53:57.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.820+0000 7f9533fff700 1 -- 172.21.15.131:0/3076109350 learned_addr learned my addr 172.21.15.131:0/3076109350 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95337fe700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f953413c510 0x7f95340c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579169778 0 0) 0x7f95340686c0 con 0x7f95340cc810 2023-12-26T21:53:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f951c003570 con 0x7f95340cc810 2023-12-26T21:53:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010983466 0 0) 0x7f9534069520 con 0x7f953406b8c0 2023-12-26T21:53:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95340686c0 con 0x7f953406b8c0 2023-12-26T21:53:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1720223781 0 0) 0x7f951c003570 con 0x7f95340cc810 2023-12-26T21:53:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f953413c510 msgr2=0x7f95340c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f953413c510 0x7f95340c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 >> v1:172.21.15.190:6789/0 conn(0x7f953406b8c0 legacy=0x7f95340cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534068bf0 con 0x7f95340cc810 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f952c0017f0 con 0x7f95340cc810 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 >> v1:172.21.15.131:6789/0 conn(0x7f95340cc810 legacy=0x7f95340699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f953413c510 0x7f951c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f951c004180 0x7f951c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f951c004d50 0x7f951c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9534068910 con 0x7f951c004180 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c00bba0 con 0x7f951c004d50 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c00bd60 con 0x7f953413c510 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.821+0000 7f9532ffd700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f951c004180 0x7f951c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f95337fe700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f953413c510 0x7f951c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f9533fff700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f951c004d50 0x7f951c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f9532ffd700 1 -- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f953413c510 msgr2=0x7f951c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f9532ffd700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f953413c510 0x7f951c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f9532ffd700 1 -- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f951c004d50 msgr2=0x7f951c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f9532ffd700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f951c004d50 0x7f951c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f9532ffd700 1 -- 172.21.15.131:0/3076109350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c003570 con 0x7f951c004180 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f9532ffd700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f951c004180 0x7f951c004810 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f9528001a10 tx=0x7f9528007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf9f8b64a89b4d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952800c5d0 con 0x7f951c004180 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952800c730 con 0x7f951c004180 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.822+0000 7f95327fc700 1 -- 172.21.15.131:0/3076109350 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952800cad0 con 0x7f951c004180 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.823+0000 7f9539fff700 1 -- 172.21.15.131:0/3076109350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f951c004180 msgr2=0x7f951c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.823+0000 7f9539fff700 1 --2- 172.21.15.131:0/3076109350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f951c004180 0x7f951c004810 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f9528001a10 tx=0x7f9528007d30 comp rx=0 tx=0).stop 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.823+0000 7f9533fff700 1 -- 172.21.15.131:0/3076109350 reap_dead start 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.823+0000 7f9539fff700 1 -- 172.21.15.131:0/3076109350 shutdown_connections 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.823+0000 7f9539fff700 1 -- 172.21.15.131:0/3076109350 >> 172.21.15.131:0/3076109350 conn(0x7f95340ce3e0 msgr2=0x7f95340ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.823+0000 7f9539fff700 1 -- 172.21.15.131:0/3076109350 shutdown_connections 2023-12-26T21:53:57.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.823+0000 7f9539fff700 1 -- 172.21.15.131:0/3076109350 wait complete. 2023-12-26T21:53:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.824+0000 7f9539fff700 1 Processor -- start 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.825+0000 7f9539fff700 1 -- start start 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.825+0000 7f9539fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f951c004180 0x7f9534154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9539fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9534151610 0x7f9534151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9539fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95340c7240 0x7f95340c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9539fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953415d240 con 0x7f95340c7240 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9539fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95340c6210 con 0x7f9534151610 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9539fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9534151fc0 con 0x7f951c004180 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9533fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95340c7240 0x7f95340c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9533fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95340c7240 0x7f95340c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46794/0 (socket says 172.21.15.131:46794) 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9533fff700 1 -- 172.21.15.131:0/4204453038 learned_addr learned my addr 172.21.15.131:0/4204453038 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f95337fe700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f951c004180 0x7f9534154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9532ffd700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9534151610 0x7f9534151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9533fff700 1 -- 172.21.15.131:0/4204453038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f951c004180 msgr2=0x7f9534154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9533fff700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f951c004180 0x7f9534154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9533fff700 1 -- 172.21.15.131:0/4204453038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9534151610 msgr2=0x7f9534151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9533fff700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9534151610 0x7f9534151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9533fff700 1 -- 172.21.15.131:0/4204453038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534152470 con 0x7f95340c7240 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.826+0000 7f9533fff700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95340c7240 0x7f95340c76b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f953406f1c0 tx=0x7f952c002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.827+0000 7f951b7fe700 1 -- 172.21.15.131:0/4204453038 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c00ad40 con 0x7f95340c7240 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.827+0000 7f951b7fe700 1 -- 172.21.15.131:0/4204453038 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c0133f0 con 0x7f95340c7240 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.827+0000 7f951b7fe700 1 -- 172.21.15.131:0/4204453038 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c0137b0 con 0x7f95340c7240 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.827+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95340c6480 con 0x7f95340c7240 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.827+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95340c6a40 con 0x7f95340c7240 2023-12-26T21:53:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.828+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95340657e0 con 0x7f95340c7240 2023-12-26T21:53:57.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.829+0000 7f951b7fe700 1 -- 172.21.15.131:0/4204453038 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f952c013be0 con 0x7f95340c7240 2023-12-26T21:53:57.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.829+0000 7f951b7fe700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9520066f30 0x7f95200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:57.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.830+0000 7f951b7fe700 1 -- 172.21.15.131:0/4204453038 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f952c085250 con 0x7f95340c7240 2023-12-26T21:53:57.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.830+0000 7f95337fe700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9520066f30 0x7f95200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:57.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.830+0000 7f95337fe700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9520066f30 0x7f95200693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9524006d00 tx=0x7f9524008040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:57.833+0000 7f951b7fe700 1 -- 172.21.15.131:0/4204453038 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f952c052330 con 0x7f95340c7240 2023-12-26T21:53:58.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.070+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9534068ed0 con 0x7f95340c7240 2023-12-26T21:53:58.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.071+0000 7f951b7fe700 1 -- 172.21.15.131:0/4204453038 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f952c055950 con 0x7f95340c7240 2023-12-26T21:53:58.072 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:53:58.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.073+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9520066f30 msgr2=0x7f95200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.073+0000 7f9539fff700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9520066f30 0x7f95200693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9524006d00 tx=0x7f9524008040 comp rx=0 tx=0).stop 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95340c7240 msgr2=0x7f95340c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95340c7240 0x7f95340c76b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f953406f1c0 tx=0x7f952c002170 comp rx=0 tx=0).stop 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 shutdown_connections 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9520066f30 0x7f95200693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95340c7240 0x7f95340c76b0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f951c004180 0x7f9534154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 --2- 172.21.15.131:0/4204453038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9534151610 0x7f9534151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 >> 172.21.15.131:0/4204453038 conn(0x7f95340ce3e0 msgr2=0x7f95340b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 shutdown_connections 2023-12-26T21:53:58.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:58.074+0000 7f9539fff700 1 -- 172.21.15.131:0/4204453038 wait complete. 2023-12-26T21:53:58.092 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T21:53:58.092 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:53:58.092 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T21:53:58.092 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T21:53:58.092 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T21:53:58.092 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T21:53:58.092 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T21:53:58.092 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:53:58.092 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:53:58.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:58.093 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T21:53:58.137 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:53:58.137 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:53:58.137 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:53:58.137 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:58.137 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:53:58.137 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:58.137 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41418sec preferred_lft 41418sec 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:58.138 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T21:53:58.139 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:53:58.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:58.140 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:53:58.243 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T21:53:58.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:58.246 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T21:53:58.292 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T21:53:58.292 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:53:58.292 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:53:58.292 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:53:58.292 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:53:58.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:58.293 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:53:58.293 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:53:58.293 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:53:58.293 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:53:58.293 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:53:58.293 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:53:58.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:58.502 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:53:58.546 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T21:53:58.546 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T21:53:58.546 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:53:58.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:58.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T21:53:58.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:53:58.655 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T21:53:58.698 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:53:58.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:53:58.699 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:53:58.782 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T21:53:58.782 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:53:58.782 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:53:58.920 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:53:58.921 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:53:58.950 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:53:58.950 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:53:58.950 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:53:58.950 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:53:58.950 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:53:58.956 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:53:58.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:53:58.956 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:53:59.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T21:53:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.215+0000 7f4f457ff700 1 Processor -- start 2023-12-26T21:53:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.216+0000 7f4f457ff700 1 -- start start 2023-12-26T21:53:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.217+0000 7f4f457ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f400ced90 0x7f4f400cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.217+0000 7f4f457ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f400c6810 con 0x7f4f40159920 2023-12-26T21:53:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.217+0000 7f4f457ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f400cf6f0 con 0x7f4f40159cf0 2023-12-26T21:53:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.217+0000 7f4f457ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f400c5c00 con 0x7f4f400ced90 2023-12-26T21:53:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.217+0000 7f4f3effd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4f40159920 0x7f4f400c5420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37884/0 (socket says 172.21.15.131:37884) 2023-12-26T21:53:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.217+0000 7f4f3effd700 1 -- 172.21.15.131:0/1836849380 learned_addr learned my addr 172.21.15.131:0/1836849380 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:53:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.217+0000 7f4f3f7fe700 1 --2- 172.21.15.131:0/1836849380 >> v2:172.21.15.190:3300/0 conn(0x7f4f400ced90 0x7f4f400cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.218+0000 7f4f3dffb700 1 -- 172.21.15.131:0/1836849380 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647676740 0 0) 0x7f4f400c6810 con 0x7f4f40159920 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.218+0000 7f4f3dffb700 1 -- 172.21.15.131:0/1836849380 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f28003570 con 0x7f4f40159920 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.218+0000 7f4f3f7fe700 1 -- 172.21.15.131:0/1836849380 >> v1:172.21.15.190:6789/0 conn(0x7f4f40159cf0 legacy=0x7f4f400c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.218+0000 7f4f3f7fe700 1 -- 172.21.15.131:0/1836849380 >> v1:172.21.15.131:6789/0 conn(0x7f4f40159920 legacy=0x7f4f400c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.218+0000 7f4f3f7fe700 1 -- 172.21.15.131:0/1836849380 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400c5ee0 con 0x7f4f400ced90 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.218+0000 7f4f3f7fe700 1 --2- 172.21.15.131:0/1836849380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f400ced90 0x7f4f400cf1b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009e60 tx=0x7f4f38009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b62ca561824fa8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f3dffb700 1 -- 172.21.15.131:0/1836849380 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800e070 con 0x7f4f400ced90 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f3dffb700 1 -- 172.21.15.131:0/1836849380 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3800bb60 con 0x7f4f400ced90 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f3dffb700 1 -- 172.21.15.131:0/1836849380 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38012430 con 0x7f4f400ced90 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f457ff700 1 -- 172.21.15.131:0/1836849380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f400ced90 msgr2=0x7f4f400cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f457ff700 1 --2- 172.21.15.131:0/1836849380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f400ced90 0x7f4f400cf1b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009e60 tx=0x7f4f38009780 comp rx=0 tx=0).stop 2023-12-26T21:53:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f457ff700 1 -- 172.21.15.131:0/1836849380 shutdown_connections 2023-12-26T21:53:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f457ff700 1 --2- 172.21.15.131:0/1836849380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f400ced90 0x7f4f400cf1b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f457ff700 1 -- 172.21.15.131:0/1836849380 >> 172.21.15.131:0/1836849380 conn(0x7f4f40099980 msgr2=0x7f4f4006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f457ff700 1 -- 172.21.15.131:0/1836849380 shutdown_connections 2023-12-26T21:53:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.219+0000 7f4f457ff700 1 -- 172.21.15.131:0/1836849380 wait complete. 2023-12-26T21:53:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.221+0000 7f4f457ff700 1 Processor -- start 2023-12-26T21:53:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.221+0000 7f4f457ff700 1 -- start start 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f457ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f400ced90 0x7f4f40155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f457ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f40159920 0x7f4f400c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f457ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f40159cf0 0x7f4f400b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f457ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4015f7f0 con 0x7f4f40159920 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f457ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4015e030 con 0x7f4f400ced90 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f457ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4015d420 con 0x7f4f40159cf0 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f40159920 0x7f4f400c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f40159920 0x7f4f400c9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46802/0 (socket says 172.21.15.131:46802) 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3e7fc700 1 -- 172.21.15.131:0/2335251854 learned_addr learned my addr 172.21.15.131:0/2335251854 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3effd700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f400ced90 0x7f4f40155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3f7fe700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f40159cf0 0x7f4f400b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3e7fc700 1 -- 172.21.15.131:0/2335251854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f40159cf0 msgr2=0x7f4f400b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3e7fc700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f40159cf0 0x7f4f400b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3e7fc700 1 -- 172.21.15.131:0/2335251854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f400ced90 msgr2=0x7f4f40155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3e7fc700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f400ced90 0x7f4f40155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.222+0000 7f4f3e7fc700 1 -- 172.21.15.131:0/2335251854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38009a20 con 0x7f4f40159920 2023-12-26T21:53:59.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.223+0000 7f4f3e7fc700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f40159920 0x7f4f400c9eb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c510 tx=0x7f4f3400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:59.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.223+0000 7f4f26ffd700 1 -- 172.21.15.131:0/2335251854 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34011590 con 0x7f4f40159920 2023-12-26T21:53:59.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.223+0000 7f4f26ffd700 1 -- 172.21.15.131:0/2335251854 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f34011710 con 0x7f4f40159920 2023-12-26T21:53:59.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.223+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4015d700 con 0x7f4f40159920 2023-12-26T21:53:59.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.223+0000 7f4f26ffd700 1 -- 172.21.15.131:0/2335251854 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34011ad0 con 0x7f4f40159920 2023-12-26T21:53:59.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.223+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4015dcc0 con 0x7f4f40159920 2023-12-26T21:53:59.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.224+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f400657e0 con 0x7f4f40159920 2023-12-26T21:53:59.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.225+0000 7f4f26ffd700 1 -- 172.21.15.131:0/2335251854 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4f3401b3b0 con 0x7f4f40159920 2023-12-26T21:53:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.226+0000 7f4f26ffd700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.227+0000 7f4f26ffd700 1 -- 172.21.15.131:0/2335251854 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f4f34083bd0 con 0x7f4f40159920 2023-12-26T21:53:59.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.227+0000 7f4f3effd700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.227+0000 7f4f3effd700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f2c066f30 0x7f4f2c0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f30001360 tx=0x7f4f30009040 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:59.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.231+0000 7f4f26ffd700 1 -- 172.21.15.131:0/2335251854 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f34050cb0 con 0x7f4f40159920 2023-12-26T21:53:59.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.452+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f400cfe40 con 0x7f4f2c066f30 2023-12-26T21:53:59.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.454+0000 7f4f26ffd700 1 -- 172.21.15.131:0/2335251854 <== mgr.6975 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4f400cfe40 con 0x7f4f2c066f30 2023-12-26T21:53:59.454 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:53:59.454 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:53:59.454 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T21:53:59.455 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:53:59.455 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:53:59.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f2c066f30 msgr2=0x7f4f2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f2c066f30 0x7f4f2c0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f30001360 tx=0x7f4f30009040 comp rx=0 tx=0).stop 2023-12-26T21:53:59.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f40159920 msgr2=0x7f4f400c9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f40159920 0x7f4f400c9eb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c510 tx=0x7f4f3400c8d0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 shutdown_connections 2023-12-26T21:53:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f40159cf0 0x7f4f400b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f400ced90 0x7f4f40155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 --2- 172.21.15.131:0/2335251854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f40159920 0x7f4f400c9eb0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.456+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 >> 172.21.15.131:0/2335251854 conn(0x7f4f40099980 msgr2=0x7f4f400694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.457+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 shutdown_connections 2023-12-26T21:53:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.457+0000 7f4f457ff700 1 -- 172.21.15.131:0/2335251854 wait complete. 2023-12-26T21:53:59.475 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T21:53:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.666+0000 7f749abff700 1 Processor -- start 2023-12-26T21:53:59.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.666+0000 7f749abff700 1 -- start start 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.667+0000 7f749abff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f74940cb320 0x7f7494069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.667+0000 7f749abff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7494156940 con 0x7f74940b0e60 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.667+0000 7f749abff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74940ae500 con 0x7f749406a1e0 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.667+0000 7f749abff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74940c8cc0 con 0x7f74940cb320 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.667+0000 7f749899b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f74940cb320 0x7f7494069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f749899b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f74940cb320 0x7f7494069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46812/0 (socket says 172.21.15.131:46812) 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f749919c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f74940b0e60 0x7f74940c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37892/0 (socket says 172.21.15.131:37892) 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f749899b700 1 -- 172.21.15.131:0/3459443967 learned_addr learned my addr 172.21.15.131:0/3459443967 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f749899b700 1 -- 172.21.15.131:0/3459443967 >> v1:172.21.15.190:6789/0 conn(0x7f749406a1e0 legacy=0x7f74940b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f749899b700 1 -- 172.21.15.131:0/3459443967 >> v1:172.21.15.131:6789/0 conn(0x7f74940b0e60 legacy=0x7f74940c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f749899b700 1 -- 172.21.15.131:0/3459443967 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940b0280 con 0x7f74940cb320 2023-12-26T21:53:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f749899b700 1 --2- 172.21.15.131:0/3459443967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 0x7f7494069ca0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f748800bc50 tx=0x7f748800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a850ef7011f77b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f74937fe700 1 -- 172.21.15.131:0/3459443967 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748800be60 con 0x7f74940cb320 2023-12-26T21:53:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f74937fe700 1 -- 172.21.15.131:0/3459443967 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748800de00 con 0x7f74940cb320 2023-12-26T21:53:59.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.668+0000 7f74937fe700 1 -- 172.21.15.131:0/3459443967 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880196b0 con 0x7f74940cb320 2023-12-26T21:53:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.669+0000 7f749abff700 1 -- 172.21.15.131:0/3459443967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 msgr2=0x7f7494069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.669+0000 7f749abff700 1 --2- 172.21.15.131:0/3459443967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 0x7f7494069ca0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f748800bc50 tx=0x7f748800b4f0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.669+0000 7f749abff700 1 -- 172.21.15.131:0/3459443967 shutdown_connections 2023-12-26T21:53:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.669+0000 7f749abff700 1 --2- 172.21.15.131:0/3459443967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 0x7f7494069ca0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.669+0000 7f749abff700 1 -- 172.21.15.131:0/3459443967 >> 172.21.15.131:0/3459443967 conn(0x7f7494068040 msgr2=0x7f7494069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:53:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.669+0000 7f749abff700 1 -- 172.21.15.131:0/3459443967 shutdown_connections 2023-12-26T21:53:59.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.669+0000 7f749abff700 1 -- 172.21.15.131:0/3459443967 wait complete. 2023-12-26T21:53:59.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.671+0000 7f749abff700 1 Processor -- start 2023-12-26T21:53:59.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.671+0000 7f749abff700 1 -- start start 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749406a1e0 0x7f7494158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749abff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74940b0e60 0x7f74940a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749abff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 0x7f7494155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749abff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494150020 con 0x7f74940cb320 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749abff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74940aa0e0 con 0x7f74940b0e60 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74940aa3c0 con 0x7f749406a1e0 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 0x7f7494155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 0x7f7494155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46816/0 (socket says 172.21.15.131:46816) 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749919c700 1 -- 172.21.15.131:0/1647263759 learned_addr learned my addr 172.21.15.131:0/1647263759 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749899b700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749406a1e0 0x7f7494158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f7493fff700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74940b0e60 0x7f74940a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749919c700 1 -- 172.21.15.131:0/1647263759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749406a1e0 msgr2=0x7f7494158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749919c700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749406a1e0 0x7f7494158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749919c700 1 -- 172.21.15.131:0/1647263759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74940b0e60 msgr2=0x7f74940a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749919c700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74940b0e60 0x7f74940a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.672+0000 7f749919c700 1 -- 172.21.15.131:0/1647263759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748800b790 con 0x7f74940cb320 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.673+0000 7f749919c700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 0x7f7494155810 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f748c00cd30 tx=0x7f748c001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.673+0000 7f7490ff9700 1 -- 172.21.15.131:0/1647263759 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00ce30 con 0x7f74940cb320 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.673+0000 7f7490ff9700 1 -- 172.21.15.131:0/1647263759 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c005df0 con 0x7f74940cb320 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.673+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74940aa6a0 con 0x7f74940cb320 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.673+0000 7f7490ff9700 1 -- 172.21.15.131:0/1647263759 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00ab00 con 0x7f74940cb320 2023-12-26T21:53:59.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.673+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749414fc40 con 0x7f74940cb320 2023-12-26T21:53:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.674+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74941501d0 con 0x7f74940cb320 2023-12-26T21:53:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.675+0000 7f7490ff9700 1 -- 172.21.15.131:0/1647263759 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 43) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f748c0070f0 con 0x7f74940cb320 2023-12-26T21:53:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.676+0000 7f7490ff9700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7478066f30 0x7f74780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:53:59.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.676+0000 7f7490ff9700 1 -- 172.21.15.131:0/1647263759 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f748c084630 con 0x7f74940cb320 2023-12-26T21:53:59.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.676+0000 7f749899b700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7478066f30 0x7f74780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:53:59.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.676+0000 7f749899b700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7478066f30 0x7f74780693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f748800bc50 tx=0x7f748801b000 comp rx=0 tx=0).ready entity=mgr.6975 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:53:59.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.680+0000 7f7490ff9700 1 -- 172.21.15.131:0/1647263759 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74941501d0 con 0x7f74940cb320 2023-12-26T21:53:59.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:53:59.926+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f74940657e0 con 0x7f74940cb320 2023-12-26T21:54:00.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.337+0000 7f7490ff9700 1 -- 172.21.15.131:0/1647263759 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 44) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f748c051b60 con 0x7f74940cb320 2023-12-26T21:54:00.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.339+0000 7f7490ff9700 1 -- 172.21.15.131:0/1647263759 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v44) v1 ==== 92+0+0 (secure 0 0 0) 0x7f748c05c640 con 0x7f74940cb320 2023-12-26T21:54:00.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7478066f30 msgr2=0x7f74780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:00.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7478066f30 0x7f74780693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f748800bc50 tx=0x7f748801b000 comp rx=0 tx=0).stop 2023-12-26T21:54:00.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 msgr2=0x7f7494155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:00.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 0x7f7494155810 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f748c00cd30 tx=0x7f748c001ab0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 shutdown_connections 2023-12-26T21:54:00.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7478066f30 0x7f74780693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74940cb320 0x7f7494155810 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749406a1e0 0x7f7494158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 --2- 172.21.15.131:0/1647263759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74940b0e60 0x7f74940a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 >> 172.21.15.131:0/1647263759 conn(0x7f7494068040 msgr2=0x7f749406e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:54:00.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 shutdown_connections 2023-12-26T21:54:00.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.342+0000 7f749abff700 1 -- 172.21.15.131:0/1647263759 wait complete. 2023-12-26T21:54:00.362 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T21:54:00.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.568+0000 7fc4a03ff700 1 Processor -- start 2023-12-26T21:54:00.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.569+0000 7fc4a03ff700 1 -- start start 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.570+0000 7fc4a03ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980cc580 0x7fc4980cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.570+0000 7fc4a03ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4980b0550 0x7fc49815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.570+0000 7fc4a03ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4980b0d70 con 0x7fc4980b00e0 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.570+0000 7fc4a03ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4980680f0 con 0x7fc4980b0550 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.570+0000 7fc4a03ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980675b0 con 0x7fc4980cc580 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.570+0000 7fc49d99a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc4980b00e0 0x7fc49806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37906/0 (socket says 172.21.15.131:37906) 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.570+0000 7fc49d99a700 1 -- 172.21.15.131:0/551313826 learned_addr learned my addr 172.21.15.131:0/551313826 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.570+0000 7fc49e99c700 1 --2- 172.21.15.131:0/551313826 >> v2:172.21.15.190:3300/0 conn(0x7fc4980b0550 0x7fc49815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.570+0000 7fc49e19b700 1 --2- 172.21.15.131:0/551313826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980cc580 0x7fc4980cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.571+0000 7fc49d199700 1 -- 172.21.15.131:0/551313826 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006998693 0 0) 0x7fc4980b0d70 con 0x7fc4980b00e0 2023-12-26T21:54:00.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.571+0000 7fc49d199700 1 -- 172.21.15.131:0/551313826 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc484003570 con 0x7fc4980b00e0 2023-12-26T21:54:00.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.571+0000 7fc49e99c700 1 -- 172.21.15.131:0/551313826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980cc580 msgr2=0x7fc4980cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:54:00.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.571+0000 7fc49e99c700 1 --2- 172.21.15.131:0/551313826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980cc580 0x7fc4980cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.571+0000 7fc49e99c700 1 -- 172.21.15.131:0/551313826 >> v1:172.21.15.131:6789/0 conn(0x7fc4980b00e0 legacy=0x7fc49806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:00.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.571+0000 7fc49e99c700 1 -- 172.21.15.131:0/551313826 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc498159c30 con 0x7fc4980b0550 2023-12-26T21:54:00.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.571+0000 7fc49e19b700 1 --2- 172.21.15.131:0/551313826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980cc580 0x7fc4980cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:54:00.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.571+0000 7fc49e99c700 1 --2- 172.21.15.131:0/551313826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4980b0550 0x7fc49815d9b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc490009e60 tx=0x7fc490009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb7609f197398a07 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.572+0000 7fc49d199700 1 -- 172.21.15.131:0/551313826 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49000e070 con 0x7fc4980b0550 2023-12-26T21:54:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.572+0000 7fc49d199700 1 -- 172.21.15.131:0/551313826 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49000bab0 con 0x7fc4980b0550 2023-12-26T21:54:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.572+0000 7fc49d199700 1 -- 172.21.15.131:0/551313826 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49000be70 con 0x7fc4980b0550 2023-12-26T21:54:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.572+0000 7fc4a03ff700 1 -- 172.21.15.131:0/551313826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4980b0550 msgr2=0x7fc49815d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.572+0000 7fc4a03ff700 1 --2- 172.21.15.131:0/551313826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4980b0550 0x7fc49815d9b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc490009e60 tx=0x7fc490009780 comp rx=0 tx=0).stop 2023-12-26T21:54:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.572+0000 7fc4a03ff700 1 -- 172.21.15.131:0/551313826 shutdown_connections 2023-12-26T21:54:00.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.572+0000 7fc4a03ff700 1 --2- 172.21.15.131:0/551313826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4980b0550 0x7fc49815d9b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.572+0000 7fc4a03ff700 1 --2- 172.21.15.131:0/551313826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980cc580 0x7fc4980cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.572+0000 7fc4a03ff700 1 -- 172.21.15.131:0/551313826 >> 172.21.15.131:0/551313826 conn(0x7fc4980686c0 msgr2=0x7fc498068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:54:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.573+0000 7fc4a03ff700 1 -- 172.21.15.131:0/551313826 shutdown_connections 2023-12-26T21:54:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.573+0000 7fc4a03ff700 1 -- 172.21.15.131:0/551313826 wait complete. 2023-12-26T21:54:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.575+0000 7fc4a03ff700 1 Processor -- start 2023-12-26T21:54:00.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.575+0000 7fc4a03ff700 1 -- start start 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc4a03ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4980b00e0 0x7fc4980ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc4a03ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980b0550 0x7fc4980af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc4a03ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4980cc580 0x7fc49815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc4a03ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49815bdd0 con 0x7fc4980b00e0 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc4a03ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49815baa0 con 0x7fc4980cc580 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc4a03ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4981522a0 con 0x7fc4980b0550 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49e19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4980b00e0 0x7fc4980ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49e19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4980b00e0 0x7fc4980ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46824/0 (socket says 172.21.15.131:46824) 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49e19b700 1 -- 172.21.15.131:0/1174312740 learned_addr learned my addr 172.21.15.131:0/1174312740 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49d99a700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980b0550 0x7fc4980af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49e99c700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4980cc580 0x7fc49815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:00.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49e19b700 1 -- 172.21.15.131:0/1174312740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980b0550 msgr2=0x7fc4980af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:00.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49e19b700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980b0550 0x7fc4980af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49e19b700 1 -- 172.21.15.131:0/1174312740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4980cc580 msgr2=0x7fc49815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:00.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49e19b700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4980cc580 0x7fc49815c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.577+0000 7fc49e19b700 1 -- 172.21.15.131:0/1174312740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494009420 con 0x7fc4980b00e0 2023-12-26T21:54:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.578+0000 7fc49e19b700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4980b00e0 0x7fc4980ad970 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc49400ec80 tx=0x7fc4940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.578+0000 7fc48e7fc700 1 -- 172.21.15.131:0/1174312740 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49400c850 con 0x7fc4980b00e0 2023-12-26T21:54:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.578+0000 7fc48e7fc700 1 -- 172.21.15.131:0/1174312740 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49400c9b0 con 0x7fc4980b00e0 2023-12-26T21:54:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.578+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc490009a20 con 0x7fc4980b00e0 2023-12-26T21:54:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.578+0000 7fc48e7fc700 1 -- 172.21.15.131:0/1174312740 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49400cd30 con 0x7fc4980b00e0 2023-12-26T21:54:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.578+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4980adeb0 con 0x7fc4980b00e0 2023-12-26T21:54:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.579+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4980657e0 con 0x7fc4980b00e0 2023-12-26T21:54:00.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.580+0000 7fc48e7fc700 1 -- 172.21.15.131:0/1174312740 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 44) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fc494013680 con 0x7fc4980b00e0 2023-12-26T21:54:00.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.581+0000 7fc48e7fc700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc47c066f80 0x7fc47c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:00.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.581+0000 7fc49d99a700 1 -- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc47c066f80 msgr2=0x7fc47c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:54:00.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.581+0000 7fc48e7fc700 1 -- 172.21.15.131:0/1174312740 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fc494084c40 con 0x7fc4980b00e0 2023-12-26T21:54:00.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.581+0000 7fc49d99a700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc47c066f80 0x7fc47c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:54:00.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.589+0000 7fc48e7fc700 1 -- 172.21.15.131:0/1174312740 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4940557b0 con 0x7fc4980b00e0 2023-12-26T21:54:00.773 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:00.773+0000 7fd1f650c1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:54:00.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.782+0000 7fc49d99a700 1 -- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc47c066f80 msgr2=0x7fc47c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:54:00.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.782+0000 7fc49d99a700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc47c066f80 0x7fc47c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:54:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.789+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc498067850 con 0x7fc4980b00e0 2023-12-26T21:54:00.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.794+0000 7fc48e7fc700 1 -- 172.21.15.131:0/1174312740 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc4940170d0 con 0x7fc4980b00e0 2023-12-26T21:54:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc47c066f80 msgr2=0x7fc47c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:54:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc47c066f80 0x7fc47c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4980b00e0 msgr2=0x7fc4980ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4980b00e0 0x7fc4980ad970 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc49400ec80 tx=0x7fc4940091a0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 shutdown_connections 2023-12-26T21:54:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4980cc580 0x7fc49815c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc47c066f80 0x7fc47c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4980b00e0 0x7fc4980ad970 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 --2- 172.21.15.131:0/1174312740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4980b0550 0x7fc4980af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 >> 172.21.15.131:0/1174312740 conn(0x7fc4980686c0 msgr2=0x7fc4980a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:54:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 shutdown_connections 2023-12-26T21:54:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:00.796+0000 7fc4a03ff700 1 -- 172.21.15.131:0/1174312740 wait complete. 2023-12-26T21:54:00.802 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:00.802+0000 7f536269f1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:54:00.816 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T21:54:00.879 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:00.878+0000 7fd1f650c1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:54:00.915 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:00.914+0000 7f536269f1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:54:00.968 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:00.967+0000 7fd1f650c1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:54:01.008 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:01.008+0000 7f536269f1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:54:01.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.013+0000 7f89a47ff700 1 Processor -- start 2023-12-26T21:54:01.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.013+0000 7f89a47ff700 1 -- start start 2023-12-26T21:54:01.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.014+0000 7f89a47ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f899c0c8820 0x7f899c0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.014+0000 7f89a47ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0b19f0 0x7f899c161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.014+0000 7f89a47ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c157a80 con 0x7f899c0c8bf0 2023-12-26T21:54:01.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.014+0000 7f89a47ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899c0cac50 con 0x7f899c0c8820 2023-12-26T21:54:01.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.014+0000 7f89a47ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0caf30 con 0x7f899c0b19f0 2023-12-26T21:54:01.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.014+0000 7f89a1d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f899c0c8bf0 0x7f899c0b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37908/0 (socket says 172.21.15.131:37908) 2023-12-26T21:54:01.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.014+0000 7f89a259b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f899c0c8820 0x7f899c0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.014+0000 7f89a1d9a700 1 -- 172.21.15.131:0/67680583 learned_addr learned my addr 172.21.15.131:0/67680583 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:54:01.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.015+0000 7f89a2d9c700 1 --2- 172.21.15.131:0/67680583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0b19f0 0x7f899c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.015+0000 7f89a1599700 1 -- 172.21.15.131:0/67680583 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563011700 0 0) 0x7f899c157a80 con 0x7f899c0c8bf0 2023-12-26T21:54:01.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.015+0000 7f89a1599700 1 -- 172.21.15.131:0/67680583 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8988003570 con 0x7f899c0c8bf0 2023-12-26T21:54:01.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.015+0000 7f89a259b700 1 -- 172.21.15.131:0/67680583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0b19f0 msgr2=0x7f899c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:54:01.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.015+0000 7f89a259b700 1 --2- 172.21.15.131:0/67680583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0b19f0 0x7f899c161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.015+0000 7f89a259b700 1 -- 172.21.15.131:0/67680583 >> v1:172.21.15.131:6789/0 conn(0x7f899c0c8bf0 legacy=0x7f899c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:01.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.015+0000 7f89a259b700 1 -- 172.21.15.131:0/67680583 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c0cb210 con 0x7f899c0c8820 2023-12-26T21:54:01.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.015+0000 7f89a259b700 1 --2- 172.21.15.131:0/67680583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f899c0c8820 0x7f899c0ca300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f89980098e0 tx=0x7f89980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ae1bf15aa113b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:01.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a1599700 1 -- 172.21.15.131:0/67680583 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998011630 con 0x7f899c0c8820 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a1599700 1 -- 172.21.15.131:0/67680583 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998011790 con 0x7f899c0c8820 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a1599700 1 -- 172.21.15.131:0/67680583 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998011b30 con 0x7f899c0c8820 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a47ff700 1 -- 172.21.15.131:0/67680583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f899c0c8820 msgr2=0x7f899c0ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a47ff700 1 --2- 172.21.15.131:0/67680583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f899c0c8820 0x7f899c0ca300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f89980098e0 tx=0x7f89980091a0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a47ff700 1 -- 172.21.15.131:0/67680583 shutdown_connections 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a47ff700 1 --2- 172.21.15.131:0/67680583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0b19f0 0x7f899c161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a47ff700 1 --2- 172.21.15.131:0/67680583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f899c0c8820 0x7f899c0ca300 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a47ff700 1 -- 172.21.15.131:0/67680583 >> 172.21.15.131:0/67680583 conn(0x7f899c068040 msgr2=0x7f899c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a47ff700 1 -- 172.21.15.131:0/67680583 shutdown_connections 2023-12-26T21:54:01.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.016+0000 7f89a47ff700 1 -- 172.21.15.131:0/67680583 wait complete. 2023-12-26T21:54:01.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.018+0000 7f89a47ff700 1 Processor -- start 2023-12-26T21:54:01.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.019+0000 7f89a47ff700 1 -- start start 2023-12-26T21:54:01.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a47ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f899c0b19f0 0x7f899c167530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a47ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0c8820 0x7f899c154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a47ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f899c0c8bf0 0x7f899c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a47ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0c2c60 con 0x7f899c0b19f0 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a47ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0c64b0 con 0x7f899c0c8bf0 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a47ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0c6790 con 0x7f899c0c8820 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a259b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f899c0b19f0 0x7f899c167530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a259b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f899c0b19f0 0x7f899c167530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46830/0 (socket says 172.21.15.131:46830) 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a259b700 1 -- 172.21.15.131:0/2460131885 learned_addr learned my addr 172.21.15.131:0/2460131885 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a1d9a700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0c8820 0x7f899c154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.020+0000 7f89a2d9c700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f899c0c8bf0 0x7f899c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89a259b700 1 -- 172.21.15.131:0/2460131885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0c8820 msgr2=0x7f899c154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:01.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89a259b700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0c8820 0x7f899c154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89a259b700 1 -- 172.21.15.131:0/2460131885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f899c0c8bf0 msgr2=0x7f899c0c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:01.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89a259b700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f899c0c8bf0 0x7f899c0c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89a259b700 1 -- 172.21.15.131:0/2460131885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998009420 con 0x7f899c0b19f0 2023-12-26T21:54:01.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89a259b700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f899c0b19f0 0x7f899c167530 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f899c0ae030 tx=0x7f8998017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:01.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89927fc700 1 -- 172.21.15.131:0/2460131885 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998000d10 con 0x7f899c0b19f0 2023-12-26T21:54:01.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89927fc700 1 -- 172.21.15.131:0/2460131885 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998011b10 con 0x7f899c0b19f0 2023-12-26T21:54:01.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89927fc700 1 -- 172.21.15.131:0/2460131885 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998025640 con 0x7f899c0b19f0 2023-12-26T21:54:01.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c0c6a70 con 0x7f899c0b19f0 2023-12-26T21:54:01.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.021+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c0c6d50 con 0x7f899c0b19f0 2023-12-26T21:54:01.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.023+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c0657e0 con 0x7f899c0b19f0 2023-12-26T21:54:01.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.023+0000 7f89927fc700 1 -- 172.21.15.131:0/2460131885 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 44) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f8998025a70 con 0x7f899c0b19f0 2023-12-26T21:54:01.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.024+0000 7f89927fc700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8980066f80 0x7f8980069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.024+0000 7f89927fc700 1 -- 172.21.15.131:0/2460131885 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f8998085f10 con 0x7f899c0b19f0 2023-12-26T21:54:01.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.024+0000 7f89a1d9a700 1 -- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8980066f80 msgr2=0x7f8980069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:54:01.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.024+0000 7f89a1d9a700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8980066f80 0x7f8980069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:54:01.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.028+0000 7f89927fc700 1 -- 172.21.15.131:0/2460131885 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8998052ff0 con 0x7f899c0b19f0 2023-12-26T21:54:01.053 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:01.052+0000 7fd1f650c1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:54:01.107 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:01.107+0000 7f536269f1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:54:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.225+0000 7f89a1d9a700 1 -- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8980066f80 msgr2=0x7f8980069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:54:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.225+0000 7f89a1d9a700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8980066f80 0x7f8980069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:54:01.242 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:01.242+0000 7fd1f650c1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:54:01.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.243+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f899c0cb4f0 con 0x7f899c0b19f0 2023-12-26T21:54:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.250+0000 7f89927fc700 1 -- 172.21.15.131:0/2460131885 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8998056610 con 0x7f899c0b19f0 2023-12-26T21:54:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8980066f80 msgr2=0x7f8980069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:54:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8980066f80 0x7f8980069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f899c0b19f0 msgr2=0x7f899c167530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f899c0b19f0 0x7f899c167530 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f899c0ae030 tx=0x7f8998017000 comp rx=0 tx=0).stop 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 shutdown_connections 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f899c0c8bf0 0x7f899c0c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8980066f80 0x7f8980069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f899c0b19f0 0x7f899c167530 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 --2- 172.21.15.131:0/2460131885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f899c0c8820 0x7f899c154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 >> 172.21.15.131:0/2460131885 conn(0x7f899c068040 msgr2=0x7f899c156590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 shutdown_connections 2023-12-26T21:54:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.253+0000 7f89a47ff700 1 -- 172.21.15.131:0/2460131885 wait complete. 2023-12-26T21:54:01.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2023-12-26T21:54:01.311 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:01.311+0000 7f536269f1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:54:01.416 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:01.416+0000 7fd1f650c1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:54:01.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.473+0000 7fe301dff700 1 Processor -- start 2023-12-26T21:54:01.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.473+0000 7fe301dff700 1 -- start start 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.474+0000 7fe301dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2fc1593f0 0x7fe2fc0abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.474+0000 7fe301dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2fc069e60 0x7fe2fc06a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.474+0000 7fe301dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc067ff0 con 0x7fe2fc1597c0 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.474+0000 7fe301dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2fc068480 con 0x7fe2fc1593f0 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.474+0000 7fe301dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2fc0ac350 con 0x7fe2fc069e60 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.474+0000 7fe2fb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2fc1593f0 0x7fe2fc0abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.474+0000 7fe2fb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2fc1593f0 0x7fe2fc0abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46836/0 (socket says 172.21.15.131:46836) 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.474+0000 7fe2fb7fe700 1 -- 172.21.15.131:0/3983980825 learned_addr learned my addr 172.21.15.131:0/3983980825 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.474+0000 7fe2faffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe2fc1597c0 0x7fe2fc0aa270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37922/0 (socket says 172.21.15.131:37922) 2023-12-26T21:54:01.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fbfff700 1 --2- 172.21.15.131:0/3983980825 >> v2:172.21.15.190:3300/0 conn(0x7fe2fc069e60 0x7fe2fc06a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fb7fe700 1 -- 172.21.15.131:0/3983980825 >> v2:172.21.15.190:3300/0 conn(0x7fe2fc069e60 msgr2=0x7fe2fc06a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:54:01.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fb7fe700 1 --2- 172.21.15.131:0/3983980825 >> v2:172.21.15.190:3300/0 conn(0x7fe2fc069e60 0x7fe2fc06a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fb7fe700 1 -- 172.21.15.131:0/3983980825 >> v1:172.21.15.131:6789/0 conn(0x7fe2fc1597c0 legacy=0x7fe2fc0aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:01.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fb7fe700 1 -- 172.21.15.131:0/3983980825 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc06a7d0 con 0x7fe2fc1593f0 2023-12-26T21:54:01.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fbfff700 1 --2- 172.21.15.131:0/3983980825 >> v2:172.21.15.190:3300/0 conn(0x7fe2fc069e60 0x7fe2fc06a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:54:01.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fa7fc700 1 -- 172.21.15.131:0/3983980825 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516990298 0 0) 0x7fe2fc067ff0 con 0x7fe2fc1597c0 2023-12-26T21:54:01.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fb7fe700 1 --2- 172.21.15.131:0/3983980825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc1593f0 0x7fe2fc0abc90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc06fa10 tx=0x7fe2ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ace5ada98261a85 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:01.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fa7fc700 1 -- 172.21.15.131:0/3983980825 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00b860 con 0x7fe2fc1593f0 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fa7fc700 1 -- 172.21.15.131:0/3983980825 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec00b9c0 con 0x7fe2fc1593f0 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.475+0000 7fe2fa7fc700 1 -- 172.21.15.131:0/3983980825 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00bd60 con 0x7fe2fc1593f0 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.476+0000 7fe301dff700 1 -- 172.21.15.131:0/3983980825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc1593f0 msgr2=0x7fe2fc0abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.476+0000 7fe301dff700 1 --2- 172.21.15.131:0/3983980825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc1593f0 0x7fe2fc0abc90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc06fa10 tx=0x7fe2ec009250 comp rx=0 tx=0).stop 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.476+0000 7fe301dff700 1 -- 172.21.15.131:0/3983980825 shutdown_connections 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.476+0000 7fe301dff700 1 --2- 172.21.15.131:0/3983980825 >> v2:172.21.15.190:3300/0 conn(0x7fe2fc069e60 0x7fe2fc06a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.476+0000 7fe301dff700 1 --2- 172.21.15.131:0/3983980825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc1593f0 0x7fe2fc0abc90 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.476+0000 7fe301dff700 1 -- 172.21.15.131:0/3983980825 >> 172.21.15.131:0/3983980825 conn(0x7fe2fc06b710 msgr2=0x7fe2fc06bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.476+0000 7fe301dff700 1 -- 172.21.15.131:0/3983980825 shutdown_connections 2023-12-26T21:54:01.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.476+0000 7fe301dff700 1 -- 172.21.15.131:0/3983980825 wait complete. 2023-12-26T21:54:01.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.478+0000 7fe301dff700 1 Processor -- start 2023-12-26T21:54:01.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.478+0000 7fe301dff700 1 -- start start 2023-12-26T21:54:01.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe301dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc069e60 0x7fe2fc14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe301dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2fc1593f0 0x7fe2fc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe301dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2fc1597c0 0x7fe2fc0aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe301dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc13afa0 con 0x7fe2fc069e60 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe301dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc15de30 con 0x7fe2fc1597c0 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe301dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc15d290 con 0x7fe2fc1593f0 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc069e60 0x7fe2fc14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc069e60 0x7fe2fc14e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46838/0 (socket says 172.21.15.131:46838) 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2fb7fe700 1 -- 172.21.15.131:0/1344632545 learned_addr learned my addr 172.21.15.131:0/1344632545 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2faffd700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2fc1593f0 0x7fe2fc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2fbfff700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2fc1597c0 0x7fe2fc0aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2fb7fe700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2fc1593f0 msgr2=0x7fe2fc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2fb7fe700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2fc1593f0 0x7fe2fc0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2fb7fe700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2fc1597c0 msgr2=0x7fe2fc0aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2fb7fe700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2fc1597c0 0x7fe2fc0aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.480+0000 7fe2fb7fe700 1 -- 172.21.15.131:0/1344632545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec0094f0 con 0x7fe2fc069e60 2023-12-26T21:54:01.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.481+0000 7fe2fb7fe700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc069e60 0x7fe2fc14e090 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec006f50 tx=0x7fe2ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.481+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec01a450 con 0x7fe2fc069e60 2023-12-26T21:54:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.481+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec01a5d0 con 0x7fe2fc069e60 2023-12-26T21:54:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.481+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc15d500 con 0x7fe2fc069e60 2023-12-26T21:54:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.481+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec01a990 con 0x7fe2fc069e60 2023-12-26T21:54:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.481+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc15dac0 con 0x7fe2fc069e60 2023-12-26T21:54:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.482+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc15e110 con 0x7fe2fc069e60 2023-12-26T21:54:01.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.483+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 44) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fe2ec01ad00 con 0x7fe2fc069e60 2023-12-26T21:54:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.484+0000 7fe2e37fe700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.484+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe2ec085c90 con 0x7fe2fc069e60 2023-12-26T21:54:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.484+0000 7fe2faffd700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 msgr2=0x7fe2e8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:54:01.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.484+0000 7fe2faffd700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:54:01.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.488+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2ec052d70 con 0x7fe2fc069e60 2023-12-26T21:54:01.493 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:01.492+0000 7f536269f1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:54:01.501 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:01.501+0000 7fd1f650c1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:54:01.579 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:01.579+0000 7f536269f1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:54:01.649 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:01.649+0000 7fd1f650c1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:54:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.685+0000 7fe2faffd700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 msgr2=0x7fe2e8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:54:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.685+0000 7fe2faffd700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:54:01.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:01.714+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2fc0717f0 con 0x7fe2e8066f80 2023-12-26T21:54:01.730 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:01.729+0000 7f536269f1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:54:01.740 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:01.740+0000 7fd1f650c1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:54:01.822 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:01.821+0000 7f536269f1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:54:01.922 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:01.922+0000 7fd1f650c1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:54:02.004 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:02.003+0000 7f536269f1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:54:02.070 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:02.070+0000 7fd1f650c1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:54:02.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:02.086+0000 7fe2faffd700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 msgr2=0x7fe2e8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:54:02.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:02.086+0000 7fe2faffd700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:54:02.154 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:02.153+0000 7f536269f1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:54:02.455 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:02.455+0000 7fd1f650c1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:54:02.539 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:02.538+0000 7f536269f1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:54:02.720 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:02.719+0000 7fd1f650c1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:54:02.804 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:02.804+0000 7f536269f1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:54:02.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:02.887+0000 7fe2faffd700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 msgr2=0x7fe2e8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:54:02.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:02.887+0000 7fe2faffd700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T21:54:02.996 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:02.996+0000 7fd1f650c1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:54:03.082 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:03.081+0000 7f536269f1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:54:03.085 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:03.085+0000 7fd1f650c1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:54:03.170 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:03.170+0000 7f536269f1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:54:03.813 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:03.813+0000 7fd1f650c1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:54:03.898 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:03.898+0000 7f536269f1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:54:03.905 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:03.905+0000 7fd1f650c1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:54:03.991 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:03.991+0000 7f536269f1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:54:04.058 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:04.058+0000 7fd1f650c1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:54:04.142 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:04.141+0000 7f536269f1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:54:04.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:04.489+0000 7fe2faffd700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 msgr2=0x7fe2e8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:54:04.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:04.489+0000 7fe2faffd700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T21:54:04.681 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:04.681+0000 7fd1f650c1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:54:04.766 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:04.765+0000 7f536269f1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:54:04.867 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:04.866+0000 7fd1f650c1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:54:04.956 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:04.955+0000 7f536269f1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:54:05.054 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:05.054+0000 7fd1f650c1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:54:05.145 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:05.145+0000 7f536269f1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:54:05.898 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:05.898+0000 7fd1f650c1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:54:05.987 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:05.987+0000 7f536269f1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:54:06.172 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:06.172+0000 7fd1f650c1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:54:06.265 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.264+0000 7f536269f1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:54:06.266 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:06.266+0000 7fd1f650c1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:54:06.359 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.358+0000 7f536269f1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:54:06.461 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:54:06.461+0000 7fd1f650c1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:54:06.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:06.519+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 45) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fe2ec053140 con 0x7fe2fc069e60 2023-12-26T21:54:06.558 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.557+0000 7f536269f1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:54:06.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:06.568+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 46) v1 ==== 84470+0+0 (secure 0 0 0) 0x7fe2ec0547e0 con 0x7fe2fc069e60 2023-12-26T21:54:06.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:06.569+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 msgr2=0x7fe2e8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:54:06.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:06.569+0000 7fe2e37fe700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:06.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.686+0000 7f52d6fb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:06.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.686+0000 7f52d6fb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:06.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.686+0000 7f52d6fb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:06.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.686+0000 7f52d6fb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:06.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.686+0000 7f52d6fb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:06.688 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.687+0000 7f52d3fb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:06.688 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.687+0000 7f52d3fb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:06.688 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.687+0000 7f52d3fb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:06.688 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.687+0000 7f52d3fb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:06.688 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:54:06.687+0000 7f52d3fb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:54:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.571+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mon.0 v2:172.21.15.131:3300/0 9 ==== mgrmap(e 47) v1 ==== 84597+0+0 (secure 0 0 0) 0x7fe2ec016d10 con 0x7fe2fc069e60 2023-12-26T21:54:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.572+0000 7fe2e37fe700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e80693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.572+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2fc0717f0 con 0x7fe2e8066f80 2023-12-26T21:54:07.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.575+0000 7fe2faffd700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e80693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:07.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.577+0000 7fe2faffd700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e80693b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc155320 tx=0x7fe2f0007ef0 comp rx=0 tx=0).ready entity=mgr.7368 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.643+0000 7fe2e37fe700 1 -- 172.21.15.131:0/1344632545 <== mgr.7368 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fe2fc0717f0 con 0x7fe2e8066f80 2023-12-26T21:54:07.643 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /bad-path 2023-12-26T21:54:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.645+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 msgr2=0x7fe2e80693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.645+0000 7fe301dff700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e80693b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc155320 tx=0x7fe2f0007ef0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.645+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc069e60 msgr2=0x7fe2fc14e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.645+0000 7fe301dff700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc069e60 0x7fe2fc14e090 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec006f50 tx=0x7fe2ec017000 comp rx=0 tx=0).stop 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.646+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 shutdown_connections 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.646+0000 7fe301dff700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2fc1597c0 0x7fe2fc0aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.646+0000 7fe301dff700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2e8066f80 0x7fe2e80693b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.646+0000 7fe301dff700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2fc069e60 0x7fe2fc14e090 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.646+0000 7fe301dff700 1 --2- 172.21.15.131:0/1344632545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2fc1593f0 0x7fe2fc0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.646+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 >> 172.21.15.131:0/1344632545 conn(0x7fe2fc06b710 msgr2=0x7fe2fc0b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.646+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 shutdown_connections 2023-12-26T21:54:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.646+0000 7fe301dff700 1 -- 172.21.15.131:0/1344632545 wait complete. 2023-12-26T21:54:07.665 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-12-26T21:54:07.666 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-12-26T21:54:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.902+0000 7fcf791ff700 1 Processor -- start 2023-12-26T21:54:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.902+0000 7fcf791ff700 1 -- start start 2023-12-26T21:54:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.903+0000 7fcf791ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf74156730 0x7fcf740c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.903+0000 7fcf791ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf7406ee80 0x7fcf740c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.903+0000 7fcf791ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf740acfc0 con 0x7fcf74156360 2023-12-26T21:54:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf791ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740c91f0 con 0x7fcf74156730 2023-12-26T21:54:07.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf791ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740c94d0 con 0x7fcf7406ee80 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf7259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf74156730 0x7fcf740c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf7259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf74156730 0x7fcf740c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36080/0 (socket says 172.21.15.131:36080) 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf7259c700 1 -- 172.21.15.131:0/3909476056 learned_addr learned my addr 172.21.15.131:0/3909476056 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf7359e700 1 --2- 172.21.15.131:0/3909476056 >> v2:172.21.15.190:3300/0 conn(0x7fcf7406ee80 0x7fcf740c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf7259c700 1 -- 172.21.15.131:0/3909476056 >> v2:172.21.15.190:3300/0 conn(0x7fcf7406ee80 msgr2=0x7fcf740c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf7259c700 1 --2- 172.21.15.131:0/3909476056 >> v2:172.21.15.190:3300/0 conn(0x7fcf7406ee80 0x7fcf740c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf7259c700 1 -- 172.21.15.131:0/3909476056 >> v1:172.21.15.190:6789/0 conn(0x7fcf74156360 legacy=0x7fcf7406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf7259c700 1 -- 172.21.15.131:0/3909476056 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740acc50 con 0x7fcf74156730 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.904+0000 7fcf7259c700 1 --2- 172.21.15.131:0/3909476056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 0x7fcf740c8490 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcf740b22a0 tx=0x7fcf6800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8db2aaf765e6ab server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf71d9b700 1 -- 172.21.15.131:0/3909476056 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800dc50 con 0x7fcf74156730 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf71d9b700 1 -- 172.21.15.131:0/3909476056 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6800ddb0 con 0x7fcf74156730 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf71d9b700 1 -- 172.21.15.131:0/3909476056 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680196d0 con 0x7fcf74156730 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf791ff700 1 -- 172.21.15.131:0/3909476056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 msgr2=0x7fcf740c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf791ff700 1 --2- 172.21.15.131:0/3909476056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 0x7fcf740c8490 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcf740b22a0 tx=0x7fcf6800b4f0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf791ff700 1 -- 172.21.15.131:0/3909476056 shutdown_connections 2023-12-26T21:54:07.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf791ff700 1 --2- 172.21.15.131:0/3909476056 >> v2:172.21.15.190:3300/0 conn(0x7fcf7406ee80 0x7fcf740c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf791ff700 1 --2- 172.21.15.131:0/3909476056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 0x7fcf740c8490 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf791ff700 1 -- 172.21.15.131:0/3909476056 >> 172.21.15.131:0/3909476056 conn(0x7fcf740675b0 msgr2=0x7fcf740679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:54:07.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf791ff700 1 -- 172.21.15.131:0/3909476056 shutdown_connections 2023-12-26T21:54:07.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.905+0000 7fcf791ff700 1 -- 172.21.15.131:0/3909476056 wait complete. 2023-12-26T21:54:07.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.907+0000 7fcf791ff700 1 Processor -- start 2023-12-26T21:54:07.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.908+0000 7fcf791ff700 1 -- start start 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf791ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf7406ee80 0x7fcf740c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf791ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf74156360 0x7fcf7414e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf791ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 0x7fcf740c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf791ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7414ed80 con 0x7fcf74156730 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf791ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74165ef0 con 0x7fcf7406ee80 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf791ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c41b0 con 0x7fcf74156360 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf7359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 0x7fcf740c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf7359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 0x7fcf740c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36088/0 (socket says 172.21.15.131:36088) 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf7359e700 1 -- 172.21.15.131:0/755043542 learned_addr learned my addr 172.21.15.131:0/755043542 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf72d9d700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf7406ee80 0x7fcf740c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf7259c700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf74156360 0x7fcf7414e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf7359e700 1 -- 172.21.15.131:0/755043542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf74156360 msgr2=0x7fcf7414e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.909+0000 7fcf7359e700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf74156360 0x7fcf7414e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.910+0000 7fcf7359e700 1 -- 172.21.15.131:0/755043542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf7406ee80 msgr2=0x7fcf740c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.910+0000 7fcf7359e700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf7406ee80 0x7fcf740c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.910+0000 7fcf7359e700 1 -- 172.21.15.131:0/755043542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6800b790 con 0x7fcf74156730 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.910+0000 7fcf7359e700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 0x7fcf740c5760 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009fa0 tx=0x7fcf6c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.910+0000 7fcf5affd700 1 -- 172.21.15.131:0/755043542 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00ebc0 con 0x7fcf74156730 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.910+0000 7fcf5affd700 1 -- 172.21.15.131:0/755043542 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c00ed20 con 0x7fcf74156730 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.910+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf741655c0 con 0x7fcf74156730 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.910+0000 7fcf5affd700 1 -- 172.21.15.131:0/755043542 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c014670 con 0x7fcf74156730 2023-12-26T21:54:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.910+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf74165b80 con 0x7fcf74156730 2023-12-26T21:54:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.911+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf740717f0 con 0x7fcf74156730 2023-12-26T21:54:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.912+0000 7fcf5affd700 1 -- 172.21.15.131:0/755043542 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 47) v1 ==== 84597+0+0 (secure 0 0 0) 0x7fcf6c0149e0 con 0x7fcf74156730 2023-12-26T21:54:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.913+0000 7fcf5affd700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf60066dd0 0x7fcf60069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:54:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.913+0000 7fcf5affd700 1 -- 172.21.15.131:0/755043542 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fcf6c084c40 con 0x7fcf74156730 2023-12-26T21:54:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.913+0000 7fcf72d9d700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf60066dd0 0x7fcf60069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:54:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.914+0000 7fcf72d9d700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf60066dd0 0x7fcf60069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcf64003a80 tx=0x7fcf64007ec0 comp rx=0 tx=0).ready entity=mgr.7368 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:54:07.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:07.918+0000 7fcf5affd700 1 -- 172.21.15.131:0/755043542 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf6c051e00 con 0x7fcf74156730 2023-12-26T21:54:08.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.134+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf74166030 con 0x7fcf60066dd0 2023-12-26T21:54:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.143+0000 7fcf5affd700 1 -- 172.21.15.131:0/755043542 <== mgr.7368 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fcf64002480 con 0x7fcf60066dd0 2023-12-26T21:54:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:54:08.143 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:54:07", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-26T21:54:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf60066dd0 msgr2=0x7fcf60069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf60066dd0 0x7fcf60069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcf64003a80 tx=0x7fcf64007ec0 comp rx=0 tx=0).stop 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 msgr2=0x7fcf740c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 0x7fcf740c5760 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009fa0 tx=0x7fcf6c009900 comp rx=0 tx=0).stop 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 shutdown_connections 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf60066dd0 0x7fcf60069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf74156730 0x7fcf740c5760 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf7406ee80 0x7fcf740c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 --2- 172.21.15.131:0/755043542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf74156360 0x7fcf7414e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 >> 172.21.15.131:0/755043542 conn(0x7fcf740675b0 msgr2=0x7fcf741552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 shutdown_connections 2023-12-26T21:54:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:54:08.146+0000 7fcf791ff700 1 -- 172.21.15.131:0/755043542 wait complete. 2023-12-26T21:54:08.163 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:54:07", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-26T21:54:18.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:54:18.592 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:54:48.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:54:48.707 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:55:00.656 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:00.656+0000 7f52c6796700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:00.656 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:00.656+0000 7f52c6796700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:00.656 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:00.656+0000 7f52c6796700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:00.656 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:00.656+0000 7f52c6796700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:00.656 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:00.656+0000 7f52c6796700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:08.165 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-12-26T21:55:08.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.368+0000 7f549e5ff700 1 Processor -- start 2023-12-26T21:55:08.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.368+0000 7f549e5ff700 1 -- start start 2023-12-26T21:55:08.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.369+0000 7f549e5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54981597e0 0x7f54980675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.369+0000 7f549e5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5498067af0 0x7f54980a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.369+0000 7f549e5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549815a750 con 0x7f54980aa020 2023-12-26T21:55:08.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.369+0000 7f549e5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5498159bb0 con 0x7f5498067af0 2023-12-26T21:55:08.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.369+0000 7f549e5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54980aa440 con 0x7f54981597e0 2023-12-26T21:55:08.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f5497fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54981597e0 0x7f54980675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f5497fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54981597e0 0x7f54980675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:51334/0 (socket says 172.21.15.131:51334) 2023-12-26T21:55:08.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f5497fff700 1 -- 172.21.15.131:0/489292950 learned_addr learned my addr 172.21.15.131:0/489292950 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:08.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f54977fe700 1 --2- 172.21.15.131:0/489292950 >> v2:172.21.15.190:3300/0 conn(0x7f5498067af0 0x7f54980a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f549cb9c700 1 --1- 172.21.15.131:0/489292950 >> v1:172.21.15.190:6789/0 conn(0x7f54980aa020 0x7f549806e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:48636/0 (socket says 172.21.15.131:48636) 2023-12-26T21:55:08.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f54977fe700 1 -- 172.21.15.131:0/489292950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54981597e0 msgr2=0x7f54980675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:08.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f54977fe700 1 --2- 172.21.15.131:0/489292950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54981597e0 0x7f54980675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f54977fe700 1 -- 172.21.15.131:0/489292950 >> v1:172.21.15.190:6789/0 conn(0x7f54980aa020 legacy=0x7f549806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f54977fe700 1 -- 172.21.15.131:0/489292950 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498068070 con 0x7f5498067af0 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.370+0000 7f5496ffd700 1 -- 172.21.15.131:0/489292950 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732906594 0 0) 0x7f549815a750 con 0x7f54980aa020 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.371+0000 7f54977fe700 1 --2- 172.21.15.131:0/489292950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5498067af0 0x7f54980a9ae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f548c0098e0 tx=0x7f548c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546363034066c6ac server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.371+0000 7f5496ffd700 1 -- 172.21.15.131:0/489292950 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00b640 con 0x7f5498067af0 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.371+0000 7f5496ffd700 1 -- 172.21.15.131:0/489292950 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c00b7a0 con 0x7f5498067af0 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.371+0000 7f5496ffd700 1 -- 172.21.15.131:0/489292950 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00bb40 con 0x7f5498067af0 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.371+0000 7f549e5ff700 1 -- 172.21.15.131:0/489292950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5498067af0 msgr2=0x7f54980a9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.371+0000 7f549e5ff700 1 --2- 172.21.15.131:0/489292950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5498067af0 0x7f54980a9ae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f548c0098e0 tx=0x7f548c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.372+0000 7f549e5ff700 1 -- 172.21.15.131:0/489292950 shutdown_connections 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.372+0000 7f549e5ff700 1 --2- 172.21.15.131:0/489292950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54981597e0 0x7f54980675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.372+0000 7f549e5ff700 1 --2- 172.21.15.131:0/489292950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5498067af0 0x7f54980a9ae0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.372+0000 7f549e5ff700 1 -- 172.21.15.131:0/489292950 >> 172.21.15.131:0/489292950 conn(0x7f549806b1c0 msgr2=0x7f54980aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:08.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.372+0000 7f549e5ff700 1 -- 172.21.15.131:0/489292950 shutdown_connections 2023-12-26T21:55:08.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.372+0000 7f549e5ff700 1 -- 172.21.15.131:0/489292950 wait complete. 2023-12-26T21:55:08.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.373+0000 7f549e5ff700 1 Processor -- start 2023-12-26T21:55:08.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.373+0000 7f549e5ff700 1 -- start start 2023-12-26T21:55:08.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.374+0000 7f549e5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5498067af0 0x7f54981521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.374+0000 7f549e5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54980aa020 0x7f5498156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.374+0000 7f549e5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54981597e0 0x7f5498158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.374+0000 7f549e5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980b0890 con 0x7f54980aa020 2023-12-26T21:55:08.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.374+0000 7f549e5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498156db0 con 0x7f54981597e0 2023-12-26T21:55:08.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.374+0000 7f549e5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5498157090 con 0x7f5498067af0 2023-12-26T21:55:08.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.374+0000 7f54977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54980aa020 0x7f5498156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.374+0000 7f54977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54980aa020 0x7f5498156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38874/0 (socket says 172.21.15.131:38874) 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.374+0000 7f54977fe700 1 -- 172.21.15.131:0/1554742536 learned_addr learned my addr 172.21.15.131:0/1554742536 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f54977fe700 1 -- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5498067af0 msgr2=0x7f54981521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f5497fff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5498067af0 0x7f54981521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f549cb9c700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54981597e0 0x7f5498158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f54977fe700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5498067af0 0x7f54981521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f54977fe700 1 -- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54981597e0 msgr2=0x7f5498158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f54977fe700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54981597e0 0x7f5498158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f54977fe700 1 -- 172.21.15.131:0/1554742536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c009420 con 0x7f54980aa020 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f5497fff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5498067af0 0x7f54981521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f549cb9c700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54981597e0 0x7f5498158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f54977fe700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54980aa020 0x7f5498156440 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f549806e290 tx=0x7f548c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:08.378 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f547ffff700 1 -- 172.21.15.131:0/1554742536 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00bb20 con 0x7f54980aa020 2023-12-26T21:55:08.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f547ffff700 1 -- 172.21.15.131:0/1554742536 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c011420 con 0x7f54980aa020 2023-12-26T21:55:08.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f547ffff700 1 -- 172.21.15.131:0/1554742536 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0117a0 con 0x7f54980aa020 2023-12-26T21:55:08.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.375+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54980b0240 con 0x7f54980aa020 2023-12-26T21:55:08.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.376+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5498156980 con 0x7f54980aa020 2023-12-26T21:55:08.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.377+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5498156b70 con 0x7f54980aa020 2023-12-26T21:55:08.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.377+0000 7f547ffff700 1 -- 172.21.15.131:0/1554742536 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 49) v1 ==== 84975+0+0 (secure 0 0 0) 0x7f548c011b50 con 0x7f54980aa020 2023-12-26T21:55:08.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.378+0000 7f547ffff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f54840671c0 0x7f5484069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.378+0000 7f547ffff700 1 -- 172.21.15.131:0/1554742536 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f548c0862d0 con 0x7f54980aa020 2023-12-26T21:55:08.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.378+0000 7f5497fff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f54840671c0 0x7f5484069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.379+0000 7f5497fff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f54840671c0 0x7f5484069670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5488006fd0 tx=0x7f5488008040 comp rx=0 tx=0).ready entity=mgr.7368 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:08.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.383+0000 7f547ffff700 1 -- 172.21.15.131:0/1554742536 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5498156b70 con 0x7f54980aa020 2023-12-26T21:55:08.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.604+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54980b00a0 con 0x7f54840671c0 2023-12-26T21:55:08.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.612+0000 7f547ffff700 1 -- 172.21.15.131:0/1554742536 <== mgr.7368 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f54980b00a0 con 0x7f54840671c0 2023-12-26T21:55:08.613 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:08.613 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:54:07", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-12-26T21:55:08.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f54840671c0 msgr2=0x7f5484069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f54840671c0 0x7f5484069670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5488006fd0 tx=0x7f5488008040 comp rx=0 tx=0).stop 2023-12-26T21:55:08.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54980aa020 msgr2=0x7f5498156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54980aa020 0x7f5498156440 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f549806e290 tx=0x7f548c017000 comp rx=0 tx=0).stop 2023-12-26T21:55:08.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 shutdown_connections 2023-12-26T21:55:08.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54981597e0 0x7f5498158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5498067af0 0x7f54981521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f54840671c0 0x7f5484069670 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 --2- 172.21.15.131:0/1554742536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54980aa020 0x7f5498156440 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.614+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 >> 172.21.15.131:0/1554742536 conn(0x7f549806b1c0 msgr2=0x7f54980aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:08.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.615+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 shutdown_connections 2023-12-26T21:55:08.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.615+0000 7f549e5ff700 1 -- 172.21.15.131:0/1554742536 wait complete. 2023-12-26T21:55:08.633 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:54:07", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-12-26T21:55:08.633 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2023-12-26T21:55:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.826+0000 7f0bfb9ff700 1 Processor -- start 2023-12-26T21:55:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.826+0000 7f0bfb9ff700 1 -- start start 2023-12-26T21:55:08.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.828+0000 7f0bfb9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf4069ad0 0x7f0bf40cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.828+0000 7f0bfb9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf4069f50 con 0x7f0bf4142f00 2023-12-26T21:55:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.828+0000 7f0bfb9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf40ccad0 con 0x7f0bf40cb510 2023-12-26T21:55:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.828+0000 7f0bfb9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4069640 con 0x7f0bf4069ad0 2023-12-26T21:55:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.828+0000 7f0bf979b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0bf4142f00 0x7f0bf40cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44214/0 (socket says 172.21.15.131:44214) 2023-12-26T21:55:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.828+0000 7f0bf979b700 1 -- 172.21.15.131:0/1508046989 learned_addr learned my addr 172.21.15.131:0/1508046989 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.828+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059842826 0 0) 0x7f0bf0001110 con 0x7f0bf4142f00 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.828+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0be0003570 con 0x7f0bf4142f00 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.828+0000 7f0bf9f9c700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf4069ad0 0x7f0bf40cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677640360 0 0) 0x7f0be0003570 con 0x7f0bf4142f00 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf4069ad0 msgr2=0x7f0bf40cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf4069ad0 0x7f0bf40cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 >> v1:172.21.15.190:6789/0 conn(0x7f0bf40cb510 legacy=0x7f0bf40c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf4068750 con 0x7f0bf4142f00 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f0bf0001600 con 0x7f0bf4142f00 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 >> v1:172.21.15.131:6789/0 conn(0x7f0bf4142f00 legacy=0x7f0bf40cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be0003a90 0x7f0be0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf4069ad0 0x7f0be0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0004d50 0x7f0be00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4069640 con 0x7f0be0003a90 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be000bbf0 con 0x7f0be0004d50 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be000bd80 con 0x7f0bf4069ad0 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bf979b700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be0003a90 0x7f0be0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bf8f9a700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf4069ad0 0x7f0be0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.829+0000 7f0bf9f9c700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0004d50 0x7f0be00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bf979b700 1 -- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf4069ad0 msgr2=0x7f0be0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bf979b700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf4069ad0 0x7f0be0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bf979b700 1 -- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0004d50 msgr2=0x7f0be00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bf979b700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0004d50 0x7f0be00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bf979b700 1 -- 172.21.15.131:0/1508046989 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0003570 con 0x7f0be0003a90 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bf8f9a700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf4069ad0 0x7f0be0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bf979b700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be0003a90 0x7f0be0003ea0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c3c0 tx=0x7f0bf00079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723b7f5cf761254d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:08.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000ccf0 con 0x7f0be0003a90 2023-12-26T21:55:08.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf000ce70 con 0x7f0be0003a90 2023-12-26T21:55:08.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.830+0000 7f0bebfff700 1 -- 172.21.15.131:0/1508046989 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0005b50 con 0x7f0be0003a90 2023-12-26T21:55:08.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.831+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/1508046989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be0003a90 msgr2=0x7f0be0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.831+0000 7f0bfb9ff700 1 --2- 172.21.15.131:0/1508046989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0be0003a90 0x7f0be0003ea0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c3c0 tx=0x7f0bf00079d0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.831+0000 7f0bf9f9c700 1 -- 172.21.15.131:0/1508046989 reap_dead start 2023-12-26T21:55:08.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.831+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/1508046989 shutdown_connections 2023-12-26T21:55:08.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.831+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/1508046989 >> 172.21.15.131:0/1508046989 conn(0x7f0bf4063cd0 msgr2=0x7f0bf406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:08.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.831+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/1508046989 shutdown_connections 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.831+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/1508046989 wait complete. 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.832+0000 7f0bfb9ff700 1 Processor -- start 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.832+0000 7f0bfb9ff700 1 -- start start 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.833+0000 7f0bfb9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0003a90 0x7f0bf40c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.833+0000 7f0bfb9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf40c54b0 0x7f0bf40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.833+0000 7f0bfb9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf415d950 0x7f0bf415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.833+0000 7f0bfb9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c6430 con 0x7f0bf40c54b0 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.833+0000 7f0bfb9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4169d70 con 0x7f0be0003a90 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.833+0000 7f0bfb9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c5920 con 0x7f0bf415d950 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf8f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf40c54b0 0x7f0bf40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf979b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0003a90 0x7f0bf40c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf8f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf40c54b0 0x7f0bf40b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38882/0 (socket says 172.21.15.131:38882) 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf8f9a700 1 -- 172.21.15.131:0/3729807901 learned_addr learned my addr 172.21.15.131:0/3729807901 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf9f9c700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf415d950 0x7f0bf415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf8f9a700 1 -- 172.21.15.131:0/3729807901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf415d950 msgr2=0x7f0bf415ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf8f9a700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf415d950 0x7f0bf415ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf8f9a700 1 -- 172.21.15.131:0/3729807901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0003a90 msgr2=0x7f0bf40c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf8f9a700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0003a90 0x7f0bf40c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf8f9a700 1 -- 172.21.15.131:0/3729807901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf40c5b90 con 0x7f0bf40c54b0 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bf8f9a700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf40c54b0 0x7f0bf40b2ef0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e810 tx=0x7f0be4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0be97fa700 1 -- 172.21.15.131:0/3729807901 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4010520 con 0x7f0bf40c54b0 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0be97fa700 1 -- 172.21.15.131:0/3729807901 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be4010680 con 0x7f0bf40c54b0 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0be97fa700 1 -- 172.21.15.131:0/3729807901 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4010a00 con 0x7f0bf40c54b0 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.834+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf40c5e70 con 0x7f0bf40c54b0 2023-12-26T21:55:08.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.835+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf4169f80 con 0x7f0bf40c54b0 2023-12-26T21:55:08.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.836+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf40717f0 con 0x7f0bf40c54b0 2023-12-26T21:55:08.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.836+0000 7f0be97fa700 1 -- 172.21.15.131:0/3729807901 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 49) v1 ==== 84975+0+0 (secure 0 0 0) 0x7f0be4014480 con 0x7f0bf40c54b0 2023-12-26T21:55:08.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.837+0000 7f0be97fa700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0bd80671c0 0x7f0bd8069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:08.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.837+0000 7f0be97fa700 1 -- 172.21.15.131:0/3729807901 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f0be40844b0 con 0x7f0bf40c54b0 2023-12-26T21:55:08.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.837+0000 7f0bf979b700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0bd80671c0 0x7f0bd8069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:08.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.838+0000 7f0bf979b700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0bd80671c0 0x7f0bd8069670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0001010 tx=0x7f0bf0015000 comp rx=0 tx=0).ready entity=mgr.7368 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:08.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:08.841+0000 7f0be97fa700 1 -- 172.21.15.131:0/3729807901 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0be404d5a0 con 0x7f0bf40c54b0 2023-12-26T21:55:09.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.058+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bf40ccad0 con 0x7f0bd80671c0 2023-12-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.131+0000 7f0be97fa700 1 -- 172.21.15.131:0/3729807901 <== mgr.7368 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f0bf40ccad0 con 0x7f0bd80671c0 2023-12-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /bad-path 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0bd80671c0 msgr2=0x7f0bd8069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0bd80671c0 0x7f0bd8069670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0001010 tx=0x7f0bf0015000 comp rx=0 tx=0).stop 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf40c54b0 msgr2=0x7f0bf40b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf40c54b0 0x7f0bf40b2ef0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e810 tx=0x7f0be4009040 comp rx=0 tx=0).stop 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 shutdown_connections 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bf415d950 0x7f0bf415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0be0003a90 0x7f0bf40c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0bd80671c0 0x7f0bd8069670 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 --2- 172.21.15.131:0/3729807901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bf40c54b0 0x7f0bf40b2ef0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 >> 172.21.15.131:0/3729807901 conn(0x7f0bf4063cd0 msgr2=0x7f0bf40c8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 shutdown_connections 2023-12-26T21:55:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.134+0000 7f0bfb9ff700 1 -- 172.21.15.131:0/3729807901 wait complete. 2023-12-26T21:55:09.152 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-12-26T21:55:09.153 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T21:55:09.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.345+0000 7fcf657ff700 1 Processor -- start 2023-12-26T21:55:09.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.345+0000 7fcf657ff700 1 -- start start 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.346+0000 7fcf657ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf600cb320 0x7fcf60069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.346+0000 7fcf657ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6006a1e0 0x7fcf600b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.346+0000 7fcf657ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf60156940 con 0x7fcf600b0c90 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.346+0000 7fcf657ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf600c8cc0 con 0x7fcf600cb320 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.346+0000 7fcf657ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf600ad700 con 0x7fcf6006a1e0 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.346+0000 7fcf5effd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf600cb320 0x7fcf60069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.346+0000 7fcf5effd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf600cb320 0x7fcf60069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38884/0 (socket says 172.21.15.131:38884) 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.346+0000 7fcf5effd700 1 -- 172.21.15.131:0/3266705349 learned_addr learned my addr 172.21.15.131:0/3266705349 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5e7fc700 1 --2- 172.21.15.131:0/3266705349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6006a1e0 0x7fcf600b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5effd700 1 -- 172.21.15.131:0/3266705349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6006a1e0 msgr2=0x7fcf600b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5effd700 1 --2- 172.21.15.131:0/3266705349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6006a1e0 0x7fcf600b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5effd700 1 -- 172.21.15.131:0/3266705349 >> v1:172.21.15.190:6789/0 conn(0x7fcf600b0c90 legacy=0x7fcf600aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5effd700 1 -- 172.21.15.131:0/3266705349 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf600ae500 con 0x7fcf600cb320 2023-12-26T21:55:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5e7fc700 1 --2- 172.21.15.131:0/3266705349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6006a1e0 0x7fcf600b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5effd700 1 --2- 172.21.15.131:0/3266705349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 0x7fcf60069ca0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000bc50 tx=0x7fcf5000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9998f29dae647c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5dffb700 1 -- 172.21.15.131:0/3266705349 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000be60 con 0x7fcf600cb320 2023-12-26T21:55:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5dffb700 1 -- 172.21.15.131:0/3266705349 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5000de00 con 0x7fcf600cb320 2023-12-26T21:55:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.347+0000 7fcf5dffb700 1 -- 172.21.15.131:0/3266705349 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500196b0 con 0x7fcf600cb320 2023-12-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.348+0000 7fcf657ff700 1 -- 172.21.15.131:0/3266705349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 msgr2=0x7fcf60069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.348+0000 7fcf657ff700 1 --2- 172.21.15.131:0/3266705349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 0x7fcf60069ca0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000bc50 tx=0x7fcf5000b4f0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.348+0000 7fcf657ff700 1 -- 172.21.15.131:0/3266705349 shutdown_connections 2023-12-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.348+0000 7fcf657ff700 1 --2- 172.21.15.131:0/3266705349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 0x7fcf60069ca0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.348+0000 7fcf657ff700 1 --2- 172.21.15.131:0/3266705349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6006a1e0 0x7fcf600b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.348+0000 7fcf657ff700 1 -- 172.21.15.131:0/3266705349 >> 172.21.15.131:0/3266705349 conn(0x7fcf60068040 msgr2=0x7fcf60069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.348+0000 7fcf657ff700 1 -- 172.21.15.131:0/3266705349 shutdown_connections 2023-12-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.348+0000 7fcf657ff700 1 -- 172.21.15.131:0/3266705349 wait complete. 2023-12-26T21:55:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.349+0000 7fcf657ff700 1 Processor -- start 2023-12-26T21:55:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.350+0000 7fcf657ff700 1 -- start start 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.350+0000 7fcf657ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6006a1e0 0x7fcf60153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf657ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf600b0c90 0x7fcf60143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf657ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 0x7fcf6015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf657ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60144260 con 0x7fcf600cb320 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf657ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6015e4f0 con 0x7fcf6006a1e0 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf657ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6015e7d0 con 0x7fcf600b0c90 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 0x7fcf6015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5e7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf600b0c90 0x7fcf60143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 0x7fcf6015dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38898/0 (socket says 172.21.15.131:38898) 2023-12-26T21:55:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5f7fe700 1 -- 172.21.15.131:0/1067117802 learned_addr learned my addr 172.21.15.131:0/1067117802 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6006a1e0 0x7fcf60153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5f7fe700 1 -- 172.21.15.131:0/1067117802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf600b0c90 msgr2=0x7fcf60143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5f7fe700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf600b0c90 0x7fcf60143d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5f7fe700 1 -- 172.21.15.131:0/1067117802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6006a1e0 msgr2=0x7fcf60153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5f7fe700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6006a1e0 0x7fcf60153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.351+0000 7fcf5f7fe700 1 -- 172.21.15.131:0/1067117802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5000b790 con 0x7fcf600cb320 2023-12-26T21:55:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.352+0000 7fcf5f7fe700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 0x7fcf6015dc20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c8e0 tx=0x7fcf58001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.352+0000 7fcf46ffd700 1 -- 172.21.15.131:0/1067117802 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58012630 con 0x7fcf600cb320 2023-12-26T21:55:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.352+0000 7fcf46ffd700 1 -- 172.21.15.131:0/1067117802 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58012790 con 0x7fcf600cb320 2023-12-26T21:55:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.352+0000 7fcf46ffd700 1 -- 172.21.15.131:0/1067117802 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58012b50 con 0x7fcf600cb320 2023-12-26T21:55:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.352+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6015eab0 con 0x7fcf600cb320 2023-12-26T21:55:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.352+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6015e160 con 0x7fcf600cb320 2023-12-26T21:55:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.353+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf600657e0 con 0x7fcf600cb320 2023-12-26T21:55:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.354+0000 7fcf46ffd700 1 -- 172.21.15.131:0/1067117802 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 49) v1 ==== 84975+0+0 (secure 0 0 0) 0x7fcf5801a3a0 con 0x7fcf600cb320 2023-12-26T21:55:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.355+0000 7fcf46ffd700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf4c0671f0 0x7fcf4c0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.355+0000 7fcf46ffd700 1 -- 172.21.15.131:0/1067117802 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fcf58084d60 con 0x7fcf600cb320 2023-12-26T21:55:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.355+0000 7fcf5effd700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf4c0671f0 0x7fcf4c0696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.356+0000 7fcf5effd700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf4c0671f0 0x7fcf4c0696a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000bc50 tx=0x7fcf5001b000 comp rx=0 tx=0).ready entity=mgr.7368 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:09.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.360+0000 7fcf46ffd700 1 -- 172.21.15.131:0/1067117802 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf580554a0 con 0x7fcf600cb320 2023-12-26T21:55:09.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:09.608+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fcf60159f70 con 0x7fcf600cb320 2023-12-26T21:55:10.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.360+0000 7fcf46ffd700 1 -- 172.21.15.131:0/1067117802 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7fcf58051de0 con 0x7fcf600cb320 2023-12-26T21:55:10.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.362+0000 7fcf46ffd700 1 -- 172.21.15.131:0/1067117802 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v50) v1 ==== 93+0+0 (secure 0 0 0) 0x7fcf5801ace0 con 0x7fcf600cb320 2023-12-26T21:55:10.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf4c0671f0 msgr2=0x7fcf4c0696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf4c0671f0 0x7fcf4c0696a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000bc50 tx=0x7fcf5001b000 comp rx=0 tx=0).stop 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 msgr2=0x7fcf6015dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 0x7fcf6015dc20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c8e0 tx=0x7fcf58001710 comp rx=0 tx=0).stop 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 shutdown_connections 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf4c0671f0 0x7fcf4c0696a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf600cb320 0x7fcf6015dc20 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6006a1e0 0x7fcf60153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 --2- 172.21.15.131:0/1067117802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf600b0c90 0x7fcf60143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 >> 172.21.15.131:0/1067117802 conn(0x7fcf60068040 msgr2=0x7fcf600cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 shutdown_connections 2023-12-26T21:55:10.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.366+0000 7fcf657ff700 1 -- 172.21.15.131:0/1067117802 wait complete. 2023-12-26T21:55:10.387 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:55:10.387 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:55:10.501 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:55:10.603 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:55:10.603 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T21:55:10.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:10.603 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T21:55:10.678 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:55:10.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:55:10.678 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T21:55:10.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:55:10.801 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:10.801+0000 7f483205b1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:55:10.828 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:10.828+0000 7fb077d431c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:55:10.904 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:10.903+0000 7f483205b1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:55:10.939 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:10.938+0000 7fb077d431c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:55:10.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.955+0000 7ff1e97ff700 1 Processor -- start 2023-12-26T21:55:10.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.955+0000 7ff1e97ff700 1 -- start start 2023-12-26T21:55:10.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.956+0000 7ff1e97ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 0x7ff1e4069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:10.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.956+0000 7ff1e97ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e4156940 con 0x7ff1e406a1e0 2023-12-26T21:55:10.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.956+0000 7ff1e97ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e40ae500 con 0x7ff1e40b0e60 2023-12-26T21:55:10.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.956+0000 7ff1e97ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40c8cc0 con 0x7ff1e40cb320 2023-12-26T21:55:10.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.956+0000 7ff1e27fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff1e406a1e0 0x7ff1e40b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44234/0 (socket says 172.21.15.131:44234) 2023-12-26T21:55:10.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.956+0000 7ff1e27fc700 1 -- 172.21.15.131:0/415124983 learned_addr learned my addr 172.21.15.131:0/415124983 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:10.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.957+0000 7ff1e2ffd700 1 --2- 172.21.15.131:0/415124983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 0x7ff1e4069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:10.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.957+0000 7ff1e1ffb700 1 -- 172.21.15.131:0/415124983 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881312410 0 0) 0x7ff1e4156940 con 0x7ff1e406a1e0 2023-12-26T21:55:10.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.957+0000 7ff1e1ffb700 1 -- 172.21.15.131:0/415124983 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1cc003570 con 0x7ff1e406a1e0 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.957+0000 7ff1e1ffb700 1 -- 172.21.15.131:0/415124983 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083019789 0 0) 0x7ff1e40ae500 con 0x7ff1e40b0e60 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.957+0000 7ff1e2ffd700 1 -- 172.21.15.131:0/415124983 >> v1:172.21.15.190:6789/0 conn(0x7ff1e40b0e60 legacy=0x7ff1e40c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.957+0000 7ff1e2ffd700 1 -- 172.21.15.131:0/415124983 >> v1:172.21.15.131:6789/0 conn(0x7ff1e406a1e0 legacy=0x7ff1e40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.957+0000 7ff1e2ffd700 1 -- 172.21.15.131:0/415124983 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40b0280 con 0x7ff1e40cb320 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.957+0000 7ff1e2ffd700 1 --2- 172.21.15.131:0/415124983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 0x7ff1e4069ca0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009910 tx=0x7ff1d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32e482b7a52eaa7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.958+0000 7ff1e1ffb700 1 -- 172.21.15.131:0/415124983 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400b680 con 0x7ff1e40cb320 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.958+0000 7ff1e1ffb700 1 -- 172.21.15.131:0/415124983 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d400b7e0 con 0x7ff1e40cb320 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.958+0000 7ff1e1ffb700 1 -- 172.21.15.131:0/415124983 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400bba0 con 0x7ff1e40cb320 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.958+0000 7ff1e97ff700 1 -- 172.21.15.131:0/415124983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 msgr2=0x7ff1e4069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.958+0000 7ff1e97ff700 1 --2- 172.21.15.131:0/415124983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 0x7ff1e4069ca0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009910 tx=0x7ff1d40091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.958+0000 7ff1e97ff700 1 -- 172.21.15.131:0/415124983 shutdown_connections 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.958+0000 7ff1e97ff700 1 --2- 172.21.15.131:0/415124983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 0x7ff1e4069ca0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:10.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.958+0000 7ff1e97ff700 1 -- 172.21.15.131:0/415124983 >> 172.21.15.131:0/415124983 conn(0x7ff1e4068040 msgr2=0x7ff1e4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:10.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.959+0000 7ff1e97ff700 1 -- 172.21.15.131:0/415124983 shutdown_connections 2023-12-26T21:55:10.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.959+0000 7ff1e97ff700 1 -- 172.21.15.131:0/415124983 wait complete. 2023-12-26T21:55:10.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.960+0000 7ff1e97ff700 1 Processor -- start 2023-12-26T21:55:10.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.960+0000 7ff1e97ff700 1 -- start start 2023-12-26T21:55:10.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e97ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e406a1e0 0x7ff1e4152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:10.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e97ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e40b0e60 0x7ff1e415af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:10.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e97ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 0x7ff1e415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:10.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e97ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e406ee00 con 0x7ff1e406a1e0 2023-12-26T21:55:10.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e97ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40aa5c0 con 0x7ff1e40b0e60 2023-12-26T21:55:10.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e97ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40a9c90 con 0x7ff1e40cb320 2023-12-26T21:55:10.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e406a1e0 0x7ff1e4152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:10.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e406a1e0 0x7ff1e4152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38904/0 (socket says 172.21.15.131:38904) 2023-12-26T21:55:10.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e2ffd700 1 -- 172.21.15.131:0/4192850555 learned_addr learned my addr 172.21.15.131:0/4192850555 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:10.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e27fc700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e40b0e60 0x7ff1e415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:10.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.961+0000 7ff1e37fe700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 0x7ff1e415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:10.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.962+0000 7ff1e27fc700 1 -- 172.21.15.131:0/4192850555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 msgr2=0x7ff1e415b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.962+0000 7ff1e27fc700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 0x7ff1e415b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.962+0000 7ff1e27fc700 1 -- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e406a1e0 msgr2=0x7ff1e4152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.962+0000 7ff1e27fc700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e406a1e0 0x7ff1e4152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.962+0000 7ff1e27fc700 1 -- 172.21.15.131:0/4192850555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4009420 con 0x7ff1e40b0e60 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.962+0000 7ff1e2ffd700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e406a1e0 0x7ff1e4152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.962+0000 7ff1e27fc700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e40b0e60 0x7ff1e415af80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80058d0 tx=0x7ff1d8007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.963+0000 7ff1caffd700 1 -- 172.21.15.131:0/4192850555 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8005b30 con 0x7ff1e40b0e60 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.963+0000 7ff1caffd700 1 -- 172.21.15.131:0/4192850555 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d8005cd0 con 0x7ff1e40b0e60 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.963+0000 7ff1caffd700 1 -- 172.21.15.131:0/4192850555 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8007320 con 0x7ff1e40b0e60 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.963+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e40a9f70 con 0x7ff1e40b0e60 2023-12-26T21:55:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.963+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e4153010 con 0x7ff1e40b0e60 2023-12-26T21:55:10.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.964+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e40ae180 con 0x7ff1e40b0e60 2023-12-26T21:55:10.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.965+0000 7ff1caffd700 1 -- 172.21.15.131:0/4192850555 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7ff1d8007690 con 0x7ff1e40b0e60 2023-12-26T21:55:10.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.966+0000 7ff1caffd700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff1d0067170 0x7ff1d0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:10.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.966+0000 7ff1caffd700 1 -- 172.21.15.131:0/4192850555 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7ff1d8083de0 con 0x7ff1e40b0e60 2023-12-26T21:55:10.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.966+0000 7ff1e2ffd700 1 -- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff1d0067170 msgr2=0x7ff1d0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:10.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.966+0000 7ff1e2ffd700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff1d0067170 0x7ff1d0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:10.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:10.970+0000 7ff1caffd700 1 -- 172.21.15.131:0/4192850555 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1d804b5a0 con 0x7ff1e40b0e60 2023-12-26T21:55:10.993 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:10.993+0000 7f483205b1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:55:11.035 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:11.034+0000 7fb077d431c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:55:11.075 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:11.075+0000 7f483205b1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:55:11.119 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:11.119+0000 7fb077d431c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:55:11.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.167+0000 7ff1e2ffd700 1 -- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff1d0067170 msgr2=0x7ff1d0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:11.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.167+0000 7ff1e2ffd700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff1d0067170 0x7ff1d0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:11.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.216+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1e40c8e00 con 0x7ff1e40b0e60 2023-12-26T21:55:11.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.217+0000 7ff1caffd700 1 -- 172.21.15.131:0/4192850555 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4608 (secure 0 0 0) 0x7ff1d8017020 con 0x7ff1e40b0e60 2023-12-26T21:55:11.218 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:11.218 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2416144644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2416144644},{"type":"v1","addr":"172.21.15.131:6835","nonce":2416144644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/559791617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":559791617},{"type":"v1","addr":"172.21.15.190:6834","nonce":559791617}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6901,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3954759429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3954759429},{"type":"v1","addr":"172.21.15.190:6835","nonce":3954759429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:45.744809+0000","modified":"2023-12-26T21:53:49.296563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6834/3631709496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3631709496},{"type":"v1","addr":"172.21.15.131:6834","nonce":3631709496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-12-26T21:55:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff1d0067170 msgr2=0x7ff1d0069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff1d0067170 0x7ff1d0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e40b0e60 msgr2=0x7ff1e415af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e40b0e60 0x7ff1e415af80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80058d0 tx=0x7ff1d8007d10 comp rx=0 tx=0).stop 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 shutdown_connections 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1e40cb320 0x7ff1e415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff1d0067170 0x7ff1d0069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1e406a1e0 0x7ff1e4152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 --2- 172.21.15.131:0/4192850555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e40b0e60 0x7ff1e415af80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 >> 172.21.15.131:0/4192850555 conn(0x7ff1e4068040 msgr2=0x7ff1e40ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 shutdown_connections 2023-12-26T21:55:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.220+0000 7ff1e97ff700 1 -- 172.21.15.131:0/4192850555 wait complete. 2023-12-26T21:55:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 50 2023-12-26T21:55:11.239 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:55:11.264 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:11.263+0000 7f483205b1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:55:11.318 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:11.318+0000 7fb077d431c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:55:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.433+0000 7f2ad29ff700 1 Processor -- start 2023-12-26T21:55:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.433+0000 7f2ad29ff700 1 -- start start 2023-12-26T21:55:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.434+0000 7f2ad29ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2acc1595e0 0x7f2acc1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ad29ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2acc15d240 0x7f2acc159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ad29ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc0ace50 0x7f2acc0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ad29ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2acc0add40 con 0x7f2acc15d240 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ad29ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2acc0ad7f0 con 0x7f2acc1595e0 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ad29ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc0a98b0 con 0x7f2acc0ace50 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ac3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2acc15d240 0x7f2acc159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ac3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2acc15d240 0x7f2acc159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38910/0 (socket says 172.21.15.131:38910) 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ac3fff700 1 -- 172.21.15.131:0/4261496289 learned_addr learned my addr 172.21.15.131:0/4261496289 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ad0f9c700 1 --2- 172.21.15.131:0/4261496289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc0ace50 0x7f2acc0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2acbfff700 1 --2- 172.21.15.131:0/4261496289 >> v2:172.21.15.190:3300/0 conn(0x7f2acc1595e0 0x7f2acc1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ac3fff700 1 -- 172.21.15.131:0/4261496289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc0ace50 msgr2=0x7f2acc0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ac3fff700 1 --2- 172.21.15.131:0/4261496289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc0ace50 0x7f2acc0ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ac3fff700 1 -- 172.21.15.131:0/4261496289 >> v2:172.21.15.190:3300/0 conn(0x7f2acc1595e0 msgr2=0x7f2acc1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ac3fff700 1 --2- 172.21.15.131:0/4261496289 >> v2:172.21.15.190:3300/0 conn(0x7f2acc1595e0 0x7f2acc1599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.435+0000 7f2ac3fff700 1 -- 172.21.15.131:0/4261496289 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc15a500 con 0x7f2acc15d240 2023-12-26T21:55:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.436+0000 7f2ac3fff700 1 --2- 172.21.15.131:0/4261496289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc15d240 0x7f2acc159ef0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2acc066db0 tx=0x7f2ab4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39596f7f58257ae server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:11.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.436+0000 7f2acb7fe700 1 -- 172.21.15.131:0/4261496289 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400b640 con 0x7f2acc15d240 2023-12-26T21:55:11.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.436+0000 7f2acb7fe700 1 -- 172.21.15.131:0/4261496289 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab400b7a0 con 0x7f2acc15d240 2023-12-26T21:55:11.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.436+0000 7f2acb7fe700 1 -- 172.21.15.131:0/4261496289 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400bb60 con 0x7f2acc15d240 2023-12-26T21:55:11.438 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:11.436+0000 7f483205b1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:55:11.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.437+0000 7f2ad29ff700 1 -- 172.21.15.131:0/4261496289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc15d240 msgr2=0x7f2acc159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:11.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.437+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/4261496289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc15d240 0x7f2acc159ef0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2acc066db0 tx=0x7f2ab4009250 comp rx=0 tx=0).stop 2023-12-26T21:55:11.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.437+0000 7f2ad29ff700 1 -- 172.21.15.131:0/4261496289 shutdown_connections 2023-12-26T21:55:11.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.437+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/4261496289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc0ace50 0x7f2acc0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.437+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/4261496289 >> v2:172.21.15.190:3300/0 conn(0x7f2acc1595e0 0x7f2acc1599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.437+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/4261496289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc15d240 0x7f2acc159ef0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.437+0000 7f2ad29ff700 1 -- 172.21.15.131:0/4261496289 >> 172.21.15.131:0/4261496289 conn(0x7f2acc068040 msgr2=0x7f2acc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.437+0000 7f2ad29ff700 1 -- 172.21.15.131:0/4261496289 shutdown_connections 2023-12-26T21:55:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.437+0000 7f2ad29ff700 1 -- 172.21.15.131:0/4261496289 wait complete. 2023-12-26T21:55:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.438+0000 7f2ad29ff700 1 Processor -- start 2023-12-26T21:55:11.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.438+0000 7f2ad29ff700 1 -- start start 2023-12-26T21:55:11.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.439+0000 7f2ad29ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc0ace50 0x7f2acc15c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.439+0000 7f2ad29ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc1595e0 0x7f2acc0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.439+0000 7f2ad29ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2acc15d240 0x7f2acc0b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.439+0000 7f2ad29ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc15cc00 con 0x7f2acc0ace50 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.439+0000 7f2ad29ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc154f00 con 0x7f2acc15d240 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.439+0000 7f2ad29ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc1542f0 con 0x7f2acc1595e0 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2acbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc0ace50 0x7f2acc15c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2acbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc0ace50 0x7f2acc15c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38914/0 (socket says 172.21.15.131:38914) 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2ac3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc1595e0 0x7f2acc0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2acbfff700 1 -- 172.21.15.131:0/3590923 learned_addr learned my addr 172.21.15.131:0/3590923 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2ad0f9c700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2acc15d240 0x7f2acc0b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2acbfff700 1 -- 172.21.15.131:0/3590923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc1595e0 msgr2=0x7f2acc0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2acbfff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc1595e0 0x7f2acc0b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2acbfff700 1 -- 172.21.15.131:0/3590923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2acc15d240 msgr2=0x7f2acc0b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2acbfff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2acc15d240 0x7f2acc0b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2acbfff700 1 -- 172.21.15.131:0/3590923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab40094f0 con 0x7f2acc0ace50 2023-12-26T21:55:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2acbfff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc0ace50 0x7f2acc15c6c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00ec80 tx=0x7f2abc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.440+0000 7f2ac8ff9700 1 -- 172.21.15.131:0/3590923 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00c7f0 con 0x7f2acc0ace50 2023-12-26T21:55:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.441+0000 7f2ac8ff9700 1 -- 172.21.15.131:0/3590923 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc00c950 con 0x7f2acc0ace50 2023-12-26T21:55:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.441+0000 7f2ac8ff9700 1 -- 172.21.15.131:0/3590923 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00ccf0 con 0x7f2acc0ace50 2023-12-26T21:55:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.441+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc1545d0 con 0x7f2acc0ace50 2023-12-26T21:55:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.441+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc154b90 con 0x7f2acc0ace50 2023-12-26T21:55:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.442+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2acc0657e0 con 0x7f2acc0ace50 2023-12-26T21:55:11.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.443+0000 7f2ac8ff9700 1 -- 172.21.15.131:0/3590923 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f2abc013680 con 0x7f2acc0ace50 2023-12-26T21:55:11.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.444+0000 7f2ac8ff9700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2aac067170 0x7f2aac069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.444+0000 7f2ac8ff9700 1 -- 172.21.15.131:0/3590923 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f2abc0850b0 con 0x7f2acc0ace50 2023-12-26T21:55:11.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.444+0000 7f2ac3fff700 1 -- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2aac067170 msgr2=0x7f2aac069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:11.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.444+0000 7f2ac3fff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2aac067170 0x7f2aac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:11.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.448+0000 7f2ac8ff9700 1 -- 172.21.15.131:0/3590923 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2abc051e00 con 0x7f2acc0ace50 2023-12-26T21:55:11.498 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:11.497+0000 7fb077d431c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:55:11.519 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:11.518+0000 7f483205b1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:55:11.584 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:11.583+0000 7fb077d431c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:55:11.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.645+0000 7f2ac3fff700 1 -- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2aac067170 msgr2=0x7f2aac069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:11.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.645+0000 7f2ac3fff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2aac067170 0x7f2aac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:11.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.649+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2acc15a7e0 con 0x7f2acc0ace50 2023-12-26T21:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.650+0000 7f2ac8ff9700 1 -- 172.21.15.131:0/3590923 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16381 (secure 0 0 0) 0x7f2abc055420 con 0x7f2acc0ace50 2023-12-26T21:55:11.651 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:11.651 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":111,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:54:06.560587+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:43.922429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:44.731388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1281002455":"2023-12-27T21:54:06.560556+0000","172.21.15.131:6836/39288":"2023-12-27T21:54:06.560556+0000","172.21.15.131:0/2183813711":"2023-12-27T21:54:06.560556+0000","172.21.15.131:0/1210036286":"2023-12-27T21:54:06.560556+0000","172.21.15.131:6837/39288":"2023-12-27T21:54:06.560556+0000","172.21.15.131:0/2864499457":"2023-12-27T21:54:06.560556+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2aac067170 msgr2=0x7f2aac069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2aac067170 0x7f2aac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc0ace50 msgr2=0x7f2acc15c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc0ace50 0x7f2acc15c6c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00ec80 tx=0x7f2abc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 shutdown_connections 2023-12-26T21:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2acc15d240 0x7f2acc0b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2aac067170 0x7f2aac069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2acc0ace50 0x7f2acc15c6c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 --2- 172.21.15.131:0/3590923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2acc1595e0 0x7f2acc0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 >> 172.21.15.131:0/3590923 conn(0x7f2acc068040 msgr2=0x7f2acc151eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:11.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 shutdown_connections 2023-12-26T21:55:11.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.653+0000 7f2ad29ff700 1 -- 172.21.15.131:0/3590923 wait complete. 2023-12-26T21:55:11.663 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:11.662+0000 7f483205b1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:55:11.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:53:43.922429+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:53:44.731388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:55:11.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:55:11.739 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:11.738+0000 7fb077d431c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:55:11.752 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:11.752+0000 7f483205b1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:55:11.834 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:11.833+0000 7fb077d431c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:55:11.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.870+0000 7f9468dff700 1 Processor -- start 2023-12-26T21:55:11.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.870+0000 7f9468dff700 1 -- start start 2023-12-26T21:55:11.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f9468dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94640cb320 0x7f9464069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f9468dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f946406a1e0 0x7f94640b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f9468dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9464156940 con 0x7f94640b0c90 2023-12-26T21:55:11.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f9468dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94640c8cc0 con 0x7f946406a1e0 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f9468dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94640ad700 con 0x7f94640cb320 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f9462d9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f94640b0c90 0x7f94640aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44250/0 (socket says 172.21.15.131:44250) 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f9462d9d700 1 -- 172.21.15.131:0/1552606031 learned_addr learned my addr 172.21.15.131:0/1552606031 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f946259c700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94640cb320 0x7f9464069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191913928 0 0) 0x7f9464156940 con 0x7f94640b0c90 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.872+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f944c003570 con 0x7f94640b0c90 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f9461d9b700 1 --2- 172.21.15.131:0/1552606031 >> v2:172.21.15.190:3300/0 conn(0x7f946406a1e0 0x7f94640b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 714615347 0 0) 0x7f944c003570 con 0x7f94640b0c90 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94640cb320 msgr2=0x7f9464069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94640cb320 0x7f9464069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 >> v2:172.21.15.190:3300/0 conn(0x7f946406a1e0 msgr2=0x7f94640b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 --2- 172.21.15.131:0/1552606031 >> v2:172.21.15.190:3300/0 conn(0x7f946406a1e0 0x7f94640b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640ae500 con 0x7f94640b0c90 2023-12-26T21:55:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f945c0017f0 con 0x7f94640b0c90 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 >> v1:172.21.15.131:6789/0 conn(0x7f94640b0c90 legacy=0x7f94640aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f944c003a60 0x7f944c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c004810 0x7f944c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94640cb320 0x7f944c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640c8cc0 con 0x7f944c004810 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c00ba60 con 0x7f944c003a60 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c00bc20 con 0x7f94640cb320 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.873+0000 7f9461d9b700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c004810 0x7f944c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.874+0000 7f9461d9b700 1 -- 172.21.15.131:0/1552606031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94640cb320 msgr2=0x7f944c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.874+0000 7f9461d9b700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94640cb320 0x7f944c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.874+0000 7f9461d9b700 1 -- 172.21.15.131:0/1552606031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f944c003a60 msgr2=0x7f944c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.874+0000 7f9461d9b700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f944c003a60 0x7f944c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.874+0000 7f9461d9b700 1 -- 172.21.15.131:0/1552606031 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c003570 con 0x7f944c004810 2023-12-26T21:55:11.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.874+0000 7f9461d9b700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c004810 0x7f944c004c80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f945800bb90 tx=0x7f9458008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce84898404b1b07b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.874+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800d5d0 con 0x7f944c004810 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.874+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945800d730 con 0x7f944c004810 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.874+0000 7f946159a700 1 -- 172.21.15.131:0/1552606031 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800daf0 con 0x7f944c004810 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.875+0000 7f9468dff700 1 -- 172.21.15.131:0/1552606031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c004810 msgr2=0x7f944c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.875+0000 7f9468dff700 1 --2- 172.21.15.131:0/1552606031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f944c004810 0x7f944c004c80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f945800bb90 tx=0x7f9458008040 comp rx=0 tx=0).stop 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.875+0000 7f9462d9d700 1 -- 172.21.15.131:0/1552606031 reap_dead start 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.875+0000 7f9468dff700 1 -- 172.21.15.131:0/1552606031 shutdown_connections 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.875+0000 7f9468dff700 1 -- 172.21.15.131:0/1552606031 >> 172.21.15.131:0/1552606031 conn(0x7f9464068040 msgr2=0x7f9464069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.875+0000 7f9468dff700 1 -- 172.21.15.131:0/1552606031 shutdown_connections 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.875+0000 7f9468dff700 1 -- 172.21.15.131:0/1552606031 wait complete. 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.876+0000 7f9468dff700 1 Processor -- start 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.877+0000 7f9468dff700 1 -- start start 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.877+0000 7f9468dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f944c004810 0x7f9464147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9468dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94640c80a0 0x7f94640c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9468dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94640c5f60 0x7f94640c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9468dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640c6910 con 0x7f94640c80a0 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9468dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640c2890 con 0x7f94640c5f60 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9468dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94641557f0 con 0x7f944c004810 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9461d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94640c80a0 0x7f94640c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9461d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94640c80a0 0x7f94640c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38942/0 (socket says 172.21.15.131:38942) 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9461d9b700 1 -- 172.21.15.131:0/1684692553 learned_addr learned my addr 172.21.15.131:0/1684692553 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9462d9d700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94640c5f60 0x7f94640c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f946259c700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f944c004810 0x7f9464147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9461d9b700 1 -- 172.21.15.131:0/1684692553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f944c004810 msgr2=0x7f9464147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9461d9b700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f944c004810 0x7f9464147d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9461d9b700 1 -- 172.21.15.131:0/1684692553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94640c5f60 msgr2=0x7f94640c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9461d9b700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94640c5f60 0x7f94640c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.878+0000 7f9461d9b700 1 -- 172.21.15.131:0/1684692553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464148280 con 0x7f94640c80a0 2023-12-26T21:55:11.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.879+0000 7f9461d9b700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94640c80a0 0x7f94640c8510 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9458006fd0 tx=0x7f945801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:11.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.879+0000 7f94527fc700 1 -- 172.21.15.131:0/1684692553 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458014790 con 0x7f94640c80a0 2023-12-26T21:55:11.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.879+0000 7f94527fc700 1 -- 172.21.15.131:0/1684692553 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9458014930 con 0x7f94640c80a0 2023-12-26T21:55:11.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.879+0000 7f94527fc700 1 -- 172.21.15.131:0/1684692553 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458014cf0 con 0x7f94640c80a0 2023-12-26T21:55:11.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.879+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94640c1c80 con 0x7f94640c80a0 2023-12-26T21:55:11.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.879+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94640c2240 con 0x7f94640c80a0 2023-12-26T21:55:11.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.881+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94640c2a40 con 0x7f94640c80a0 2023-12-26T21:55:11.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.881+0000 7f94527fc700 1 -- 172.21.15.131:0/1684692553 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f945801d610 con 0x7f94640c80a0 2023-12-26T21:55:11.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.883+0000 7f94527fc700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9444067170 0x7f9444069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:11.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.883+0000 7f946259c700 1 -- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9444067170 msgr2=0x7f9444069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:11.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.883+0000 7f94527fc700 1 -- 172.21.15.131:0/1684692553 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f9458086530 con 0x7f94640c80a0 2023-12-26T21:55:11.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.883+0000 7f946259c700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9444067170 0x7f9444069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:11.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:11.886+0000 7f94527fc700 1 -- 172.21.15.131:0/1684692553 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94640c2a40 con 0x7f94640c80a0 2023-12-26T21:55:11.937 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:11.937+0000 7f483205b1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:55:12.026 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:12.025+0000 7fb077d431c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:55:12.084 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:12.084+0000 7f483205b1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:55:12.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.084+0000 7f946259c700 1 -- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9444067170 msgr2=0x7f9444069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:12.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.084+0000 7f946259c700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9444067170 0x7f9444069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.119+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94640c2a00 con 0x7f94640c80a0 2023-12-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.119+0000 7f94527fc700 1 -- 172.21.15.131:0/1684692553 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94580531e0 con 0x7f94640c80a0 2023-12-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:55:12.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9444067170 msgr2=0x7f9444069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9444067170 0x7f9444069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94640c80a0 msgr2=0x7f94640c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94640c80a0 0x7f94640c8510 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9458006fd0 tx=0x7f945801b000 comp rx=0 tx=0).stop 2023-12-26T21:55:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 shutdown_connections 2023-12-26T21:55:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94640c5f60 0x7f94640c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f944c004810 0x7f9464147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9444067170 0x7f9444069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 --2- 172.21.15.131:0/1684692553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94640c80a0 0x7f94640c8510 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 >> 172.21.15.131:0/1684692553 conn(0x7f9464068040 msgr2=0x7f94640c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 shutdown_connections 2023-12-26T21:55:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.122+0000 7f9468dff700 1 -- 172.21.15.131:0/1684692553 wait complete. 2023-12-26T21:55:12.141 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:55:12.182 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:12.181+0000 7fb077d431c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:55:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.340+0000 7fb71f9ff700 1 Processor -- start 2023-12-26T21:55:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.340+0000 7fb71f9ff700 1 -- start start 2023-12-26T21:55:12.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.342+0000 7fb71f9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7180c8bf0 0x7fb7180b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.342+0000 7fb71f9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb718069d10 0x7fb7180b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.342+0000 7fb71f9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7180caa10 con 0x7fb7180c8820 2023-12-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.342+0000 7fb71f9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb718157a80 con 0x7fb718069d10 2023-12-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.342+0000 7fb71f9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7180cac50 con 0x7fb7180c8bf0 2023-12-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.342+0000 7fb71df9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb718069d10 0x7fb7180b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.342+0000 7fb71df9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb718069d10 0x7fb7180b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38946/0 (socket says 172.21.15.131:38946) 2023-12-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.343+0000 7fb71df9c700 1 -- 172.21.15.131:0/2061955242 learned_addr learned my addr 172.21.15.131:0/2061955242 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.342+0000 7fb71d79b700 1 --1- 172.21.15.131:0/2061955242 >> v1:172.21.15.190:6789/0 conn(0x7fb7180c8820 0x7fb7180ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:48684/0 (socket says 172.21.15.131:48684) 2023-12-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.343+0000 7fb71cf9a700 1 --2- 172.21.15.131:0/2061955242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7180c8bf0 0x7fb7180b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.343+0000 7fb70ffff700 1 -- 172.21.15.131:0/2061955242 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771849785 0 0) 0x7fb714001110 con 0x7fb7180c8820 2023-12-26T21:55:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.343+0000 7fb71df9c700 1 -- 172.21.15.131:0/2061955242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7180c8bf0 msgr2=0x7fb7180b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.343+0000 7fb71df9c700 1 --2- 172.21.15.131:0/2061955242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7180c8bf0 0x7fb7180b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.343+0000 7fb71df9c700 1 -- 172.21.15.131:0/2061955242 >> v1:172.21.15.190:6789/0 conn(0x7fb7180c8820 legacy=0x7fb7180ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.343+0000 7fb71df9c700 1 -- 172.21.15.131:0/2061955242 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7180caf30 con 0x7fb718069d10 2023-12-26T21:55:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.343+0000 7fb71df9c700 1 --2- 172.21.15.131:0/2061955242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb718069d10 0x7fb7180b1820 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb710009f20 tx=0x7fb710009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251e7400d02a7304 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.343+0000 7fb70ffff700 1 -- 172.21.15.131:0/2061955242 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71000e070 con 0x7fb718069d10 2023-12-26T21:55:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.344+0000 7fb70ffff700 1 -- 172.21.15.131:0/2061955242 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71000b9d0 con 0x7fb718069d10 2023-12-26T21:55:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.344+0000 7fb70ffff700 1 -- 172.21.15.131:0/2061955242 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7100092e0 con 0x7fb718069d10 2023-12-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.344+0000 7fb71f9ff700 1 -- 172.21.15.131:0/2061955242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb718069d10 msgr2=0x7fb7180b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.344+0000 7fb71f9ff700 1 --2- 172.21.15.131:0/2061955242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb718069d10 0x7fb7180b1820 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb710009f20 tx=0x7fb710009840 comp rx=0 tx=0).stop 2023-12-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.344+0000 7fb71f9ff700 1 -- 172.21.15.131:0/2061955242 shutdown_connections 2023-12-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.344+0000 7fb71f9ff700 1 --2- 172.21.15.131:0/2061955242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb718069d10 0x7fb7180b1820 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.344+0000 7fb71f9ff700 1 --2- 172.21.15.131:0/2061955242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7180c8bf0 0x7fb7180b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.344+0000 7fb71f9ff700 1 -- 172.21.15.131:0/2061955242 >> 172.21.15.131:0/2061955242 conn(0x7fb718068040 msgr2=0x7fb718069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:12.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.345+0000 7fb71f9ff700 1 -- 172.21.15.131:0/2061955242 shutdown_connections 2023-12-26T21:55:12.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.345+0000 7fb71f9ff700 1 -- 172.21.15.131:0/2061955242 wait complete. 2023-12-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.347+0000 7fb71f9ff700 1 Processor -- start 2023-12-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.347+0000 7fb71f9ff700 1 -- start start 2023-12-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71f9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb718069d10 0x7fb71815c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71f9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7180c8820 0x7fb71815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71f9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7180c8bf0 0x7fb7181695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71f9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71815daa0 con 0x7fb7180c8bf0 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71f9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7180afef0 con 0x7fb7180c8820 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71f9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71815ce20 con 0x7fb718069d10 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71df9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7180c8bf0 0x7fb7181695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71d79b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb718069d10 0x7fb71815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71cf9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7180c8820 0x7fb71815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71df9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7180c8bf0 0x7fb7181695e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38948/0 (socket says 172.21.15.131:38948) 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.348+0000 7fb71df9c700 1 -- 172.21.15.131:0/3064662088 learned_addr learned my addr 172.21.15.131:0/3064662088 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.349+0000 7fb71cf9a700 1 -- 172.21.15.131:0/3064662088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb718069d10 msgr2=0x7fb71815c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.349+0000 7fb71cf9a700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb718069d10 0x7fb71815c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.349+0000 7fb71cf9a700 1 -- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7180c8bf0 msgr2=0x7fb7181695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.349+0000 7fb71cf9a700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7180c8bf0 0x7fb7181695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.349+0000 7fb71cf9a700 1 -- 172.21.15.131:0/3064662088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb710009ae0 con 0x7fb7180c8820 2023-12-26T21:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.349+0000 7fb71df9c700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7180c8bf0 0x7fb7181695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.349+0000 7fb71cf9a700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7180c8820 0x7fb71815c780 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb70800ea50 tx=0x7fb7080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.349+0000 7fb70d7fa700 1 -- 172.21.15.131:0/3064662088 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70800ee90 con 0x7fb7180c8820 2023-12-26T21:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.349+0000 7fb70d7fa700 1 -- 172.21.15.131:0/3064662088 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70800c7f0 con 0x7fb7180c8820 2023-12-26T21:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.350+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71815d100 con 0x7fb7180c8820 2023-12-26T21:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.350+0000 7fb70d7fa700 1 -- 172.21.15.131:0/3064662088 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70800cb90 con 0x7fb7180c8820 2023-12-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.350+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71815d6c0 con 0x7fb7180c8820 2023-12-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.351+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7180657e0 con 0x7fb7180c8820 2023-12-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.351+0000 7fb70d7fa700 1 -- 172.21.15.131:0/3064662088 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7fb708013470 con 0x7fb7180c8820 2023-12-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.352+0000 7fb70d7fa700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6fc067170 0x7fb6fc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.352+0000 7fb70d7fa700 1 -- 172.21.15.131:0/3064662088 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fb708084d50 con 0x7fb7180c8820 2023-12-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.352+0000 7fb71d79b700 1 -- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6fc067170 msgr2=0x7fb6fc069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.352+0000 7fb71d79b700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6fc067170 0x7fb6fc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.357+0000 7fb70d7fa700 1 -- 172.21.15.131:0/3064662088 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb708051a00 con 0x7fb7180c8820 2023-12-26T21:55:12.469 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:12.468+0000 7f483205b1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:55:12.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.553+0000 7fb71d79b700 1 -- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6fc067170 msgr2=0x7fb6fc069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:12.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.553+0000 7fb71d79b700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6fc067170 0x7fb6fc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:12.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.557+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7180cb210 con 0x7fb7180c8820 2023-12-26T21:55:12.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.557+0000 7fb70d7fa700 1 -- 172.21.15.131:0/3064662088 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb708055020 con 0x7fb7180c8820 2023-12-26T21:55:12.558 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:55:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6fc067170 msgr2=0x7fb6fc069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6fc067170 0x7fb6fc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7180c8820 msgr2=0x7fb71815c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7180c8820 0x7fb71815c780 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb70800ea50 tx=0x7fb7080091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 shutdown_connections 2023-12-26T21:55:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb6fc067170 0x7fb6fc069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7180c8bf0 0x7fb7181695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb718069d10 0x7fb71815c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 --2- 172.21.15.131:0/3064662088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7180c8820 0x7fb71815c780 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 >> 172.21.15.131:0/3064662088 conn(0x7fb718068040 msgr2=0x7fb7180c1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 shutdown_connections 2023-12-26T21:55:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.560+0000 7fb71f9ff700 1 -- 172.21.15.131:0/3064662088 wait complete. 2023-12-26T21:55:12.581 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:55:12.590 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:12.589+0000 7fb077d431c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:55:12.733 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:12.732+0000 7f483205b1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:55:12.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.800+0000 7f41d27ff700 1 Processor -- start 2023-12-26T21:55:12.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.800+0000 7f41d27ff700 1 -- start start 2023-12-26T21:55:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.801+0000 7f41d27ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8820 0x7f41cc0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.801+0000 7f41d27ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41cc157a80 con 0x7f41cc0c8bf0 2023-12-26T21:55:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.801+0000 7f41d27ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41cc0cac50 con 0x7f41cc0b19f0 2023-12-26T21:55:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.801+0000 7f41d27ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0caf30 con 0x7f41cc0c8820 2023-12-26T21:55:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.801+0000 7f41c3fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f41cc0c8bf0 0x7f41cc0b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44256/0 (socket says 172.21.15.131:44256) 2023-12-26T21:55:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.802+0000 7f41c3fff700 1 -- 172.21.15.131:0/3382982769 learned_addr learned my addr 172.21.15.131:0/3382982769 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:12.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.802+0000 7f41cbfff700 1 --2- 172.21.15.131:0/3382982769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8820 0x7f41cc0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:12.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.802+0000 7f41cb7fe700 1 -- 172.21.15.131:0/3382982769 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801904272 0 0) 0x7f41cc157a80 con 0x7f41cc0c8bf0 2023-12-26T21:55:12.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.802+0000 7f41cb7fe700 1 -- 172.21.15.131:0/3382982769 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b8003570 con 0x7f41cc0c8bf0 2023-12-26T21:55:12.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.802+0000 7f41cb7fe700 1 -- 172.21.15.131:0/3382982769 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932365069 0 0) 0x7f41cc0cac50 con 0x7f41cc0b19f0 2023-12-26T21:55:12.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.802+0000 7f41cb7fe700 1 -- 172.21.15.131:0/3382982769 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41cc157a80 con 0x7f41cc0b19f0 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.802+0000 7f41cbfff700 1 -- 172.21.15.131:0/3382982769 >> v1:172.21.15.190:6789/0 conn(0x7f41cc0b19f0 legacy=0x7f41cc161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.802+0000 7f41cbfff700 1 -- 172.21.15.131:0/3382982769 >> v1:172.21.15.131:6789/0 conn(0x7f41cc0c8bf0 legacy=0x7f41cc0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.802+0000 7f41cbfff700 1 -- 172.21.15.131:0/3382982769 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc0cb210 con 0x7f41cc0c8820 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.803+0000 7f41cbfff700 1 --2- 172.21.15.131:0/3382982769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8820 0x7f41cc0ca300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009910 tx=0x7f41bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99021ab84e4c989 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.803+0000 7f41cb7fe700 1 -- 172.21.15.131:0/3382982769 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc011630 con 0x7f41cc0c8820 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.803+0000 7f41cb7fe700 1 -- 172.21.15.131:0/3382982769 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc011790 con 0x7f41cc0c8820 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.803+0000 7f41cb7fe700 1 -- 172.21.15.131:0/3382982769 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc011b30 con 0x7f41cc0c8820 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.803+0000 7f41d27ff700 1 -- 172.21.15.131:0/3382982769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8820 msgr2=0x7f41cc0ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.803+0000 7f41d27ff700 1 --2- 172.21.15.131:0/3382982769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8820 0x7f41cc0ca300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009910 tx=0x7f41bc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.803+0000 7f41d27ff700 1 -- 172.21.15.131:0/3382982769 shutdown_connections 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.804+0000 7f41d27ff700 1 --2- 172.21.15.131:0/3382982769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8820 0x7f41cc0ca300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.804+0000 7f41d27ff700 1 -- 172.21.15.131:0/3382982769 >> 172.21.15.131:0/3382982769 conn(0x7f41cc068040 msgr2=0x7f41cc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.804+0000 7f41d27ff700 1 -- 172.21.15.131:0/3382982769 shutdown_connections 2023-12-26T21:55:12.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.804+0000 7f41d27ff700 1 -- 172.21.15.131:0/3382982769 wait complete. 2023-12-26T21:55:12.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.806+0000 7f41d27ff700 1 Processor -- start 2023-12-26T21:55:12.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.806+0000 7f41d27ff700 1 -- start start 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41d27ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41cc0b19f0 0x7f41cc14a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41d27ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41cc0c8820 0x7f41cc0c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41d27ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8bf0 0x7f41cc0c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41d27ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0c6be0 con 0x7f41cc0b19f0 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41d27ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0afba0 con 0x7f41cc0c8820 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41d27ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc14aae0 con 0x7f41cc0c8bf0 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41cbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41cc0b19f0 0x7f41cc14a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41cbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41cc0b19f0 0x7f41cc14a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38958/0 (socket says 172.21.15.131:38958) 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41cbfff700 1 -- 172.21.15.131:0/1228395343 learned_addr learned my addr 172.21.15.131:0/1228395343 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:12.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.807+0000 7f41c3fff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41cc0c8820 0x7f41cc0c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41d0d9c700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8bf0 0x7f41cc0c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41cbfff700 1 -- 172.21.15.131:0/1228395343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8bf0 msgr2=0x7f41cc0c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41cbfff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8bf0 0x7f41cc0c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41cbfff700 1 -- 172.21.15.131:0/1228395343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41cc0c8820 msgr2=0x7f41cc0c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41cbfff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41cc0c8820 0x7f41cc0c1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41cbfff700 1 -- 172.21.15.131:0/1228395343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc009420 con 0x7f41cc0b19f0 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41cbfff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41cc0b19f0 0x7f41cc14a5a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009f00 tx=0x7f41bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41c8ff9700 1 -- 172.21.15.131:0/1228395343 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc000d10 con 0x7f41cc0b19f0 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41c8ff9700 1 -- 172.21.15.131:0/1228395343 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc011b10 con 0x7f41cc0b19f0 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc147a80 con 0x7f41cc0b19f0 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.808+0000 7f41c8ff9700 1 -- 172.21.15.131:0/1228395343 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc025630 con 0x7f41cc0b19f0 2023-12-26T21:55:12.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.809+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc0af550 con 0x7f41cc0b19f0 2023-12-26T21:55:12.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.810+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc0657e0 con 0x7f41cc0b19f0 2023-12-26T21:55:12.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.810+0000 7f41c8ff9700 1 -- 172.21.15.131:0/1228395343 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f41bc0193b0 con 0x7f41cc0b19f0 2023-12-26T21:55:12.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.811+0000 7f41c8ff9700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f41ac067170 0x7f41ac069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:12.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.811+0000 7f41c8ff9700 1 -- 172.21.15.131:0/1228395343 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f41bc086490 con 0x7f41cc0b19f0 2023-12-26T21:55:12.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.811+0000 7f41c3fff700 1 -- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f41ac067170 msgr2=0x7f41ac069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:12.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.811+0000 7f41c3fff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f41ac067170 0x7f41ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:12.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:12.815+0000 7f41c8ff9700 1 -- 172.21.15.131:0/1228395343 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41bc0531c0 con 0x7f41cc0b19f0 2023-12-26T21:55:12.885 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:12.883+0000 7fb077d431c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:55:13.009 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:13.009+0000 7f483205b1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:55:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.012+0000 7f41c3fff700 1 -- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f41ac067170 msgr2=0x7f41ac069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.012+0000 7f41c3fff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f41ac067170 0x7f41ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:13.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.013+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41cc0cb4f0 con 0x7f41cc0b19f0 2023-12-26T21:55:13.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.014+0000 7f41c8ff9700 1 -- 172.21.15.131:0/1228395343 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7f41bc0567e0 con 0x7f41cc0b19f0 2023-12-26T21:55:13.014 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:55:13.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f41ac067170 msgr2=0x7f41ac069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:13.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f41ac067170 0x7f41ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41cc0b19f0 msgr2=0x7f41cc14a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41cc0b19f0 0x7f41cc14a5a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009f00 tx=0x7f41bc017000 comp rx=0 tx=0).stop 2023-12-26T21:55:13.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 shutdown_connections 2023-12-26T21:55:13.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41cc0c8bf0 0x7f41cc0c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f41ac067170 0x7f41ac069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41cc0b19f0 0x7f41cc14a5a0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 --2- 172.21.15.131:0/1228395343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41cc0c8820 0x7f41cc0c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 >> 172.21.15.131:0/1228395343 conn(0x7f41cc068040 msgr2=0x7f41cc1566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:13.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.016+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 shutdown_connections 2023-12-26T21:55:13.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.017+0000 7f41d27ff700 1 -- 172.21.15.131:0/1228395343 wait complete. 2023-12-26T21:55:13.038 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:55:13.097 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:13.097+0000 7f483205b1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:55:13.172 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:13.172+0000 7fb077d431c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:55:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.230+0000 7f32cc9ff700 1 Processor -- start 2023-12-26T21:55:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.230+0000 7f32cc9ff700 1 -- start start 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32cc9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f32c80aa480 0x7f32c80abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32cc9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c815a8c0 con 0x7f32c8159820 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32cc9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c8159f90 con 0x7f32c806e480 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32cc9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c815ad50 con 0x7f32c80aa480 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32c6d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f32c80aa480 0x7f32c80abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32c6d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f32c80aa480 0x7f32c80abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38974/0 (socket says 172.21.15.131:38974) 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32c6d9d700 1 -- 172.21.15.131:0/91224290 learned_addr learned my addr 172.21.15.131:0/91224290 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32c659c700 1 --1- 172.21.15.131:0/91224290 >> v1:172.21.15.131:6789/0 conn(0x7f32c8159820 0x7f32c806dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44272/0 (socket says 172.21.15.131:44272) 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32c6d9d700 1 -- 172.21.15.131:0/91224290 >> v1:172.21.15.190:6789/0 conn(0x7f32c806e480 legacy=0x7f32c80675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32c6d9d700 1 -- 172.21.15.131:0/91224290 >> v1:172.21.15.131:6789/0 conn(0x7f32c8159820 legacy=0x7f32c806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.232+0000 7f32c6d9d700 1 -- 172.21.15.131:0/91224290 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c815a270 con 0x7f32c80aa480 2023-12-26T21:55:13.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.233+0000 7f32c6d9d700 1 --2- 172.21.15.131:0/91224290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c80aa480 0x7f32c80abdb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009f20 tx=0x7f32c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9537c424abbc49b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:13.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.233+0000 7f32c5d9b700 1 -- 172.21.15.131:0/91224290 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000e070 con 0x7f32c80aa480 2023-12-26T21:55:13.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.233+0000 7f32c5d9b700 1 -- 172.21.15.131:0/91224290 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000ba80 con 0x7f32c80aa480 2023-12-26T21:55:13.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.233+0000 7f32c5d9b700 1 -- 172.21.15.131:0/91224290 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00092e0 con 0x7f32c80aa480 2023-12-26T21:55:13.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.233+0000 7f32cc9ff700 1 -- 172.21.15.131:0/91224290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c80aa480 msgr2=0x7f32c80abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.233+0000 7f32cc9ff700 1 --2- 172.21.15.131:0/91224290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c80aa480 0x7f32c80abdb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009f20 tx=0x7f32c0009840 comp rx=0 tx=0).stop 2023-12-26T21:55:13.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.234+0000 7f32cc9ff700 1 -- 172.21.15.131:0/91224290 shutdown_connections 2023-12-26T21:55:13.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.234+0000 7f32cc9ff700 1 --2- 172.21.15.131:0/91224290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c80aa480 0x7f32c80abdb0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.234+0000 7f32cc9ff700 1 -- 172.21.15.131:0/91224290 >> 172.21.15.131:0/91224290 conn(0x7f32c806c290 msgr2=0x7f32c80ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:13.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.234+0000 7f32cc9ff700 1 -- 172.21.15.131:0/91224290 shutdown_connections 2023-12-26T21:55:13.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.234+0000 7f32cc9ff700 1 -- 172.21.15.131:0/91224290 wait complete. 2023-12-26T21:55:13.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.235+0000 7f32cc9ff700 1 Processor -- start 2023-12-26T21:55:13.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.235+0000 7f32cc9ff700 1 -- start start 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32cc9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c806e480 0x7f32c80c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32cc9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c80aa480 0x7f32c80c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32cc9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8159820 0x7f32c8169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32cc9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80c3590 con 0x7f32c806e480 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32cc9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80c27a0 con 0x7f32c8159820 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32cc9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80c2a10 con 0x7f32c80aa480 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32c659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c806e480 0x7f32c80c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32c659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c806e480 0x7f32c80c1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38976/0 (socket says 172.21.15.131:38976) 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32c659c700 1 -- 172.21.15.131:0/372119056 learned_addr learned my addr 172.21.15.131:0/372119056 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32c6d9d700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8159820 0x7f32c8169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:13.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.236+0000 7f32bdd9b700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c80aa480 0x7f32c80c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32c659c700 1 -- 172.21.15.131:0/372119056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c80aa480 msgr2=0x7f32c80c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32c659c700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c80aa480 0x7f32c80c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32c659c700 1 -- 172.21.15.131:0/372119056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8159820 msgr2=0x7f32c8169f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32c659c700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8159820 0x7f32c8169f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32c659c700 1 -- 172.21.15.131:0/372119056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c0009ae0 con 0x7f32c806e480 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32bdd9b700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c80aa480 0x7f32c80c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32c659c700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c806e480 0x7f32c80c1d20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f32b800c590 tx=0x7f32b8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32beffd700 1 -- 172.21.15.131:0/372119056 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8005a70 con 0x7f32c806e480 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32beffd700 1 -- 172.21.15.131:0/372119056 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b8005bd0 con 0x7f32c806e480 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c80c2cf0 con 0x7f32c806e480 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32beffd700 1 -- 172.21.15.131:0/372119056 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b80070d0 con 0x7f32c806e480 2023-12-26T21:55:13.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.237+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c80c32b0 con 0x7f32c806e480 2023-12-26T21:55:13.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.239+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c80aa000 con 0x7f32c806e480 2023-12-26T21:55:13.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.240+0000 7f32beffd700 1 -- 172.21.15.131:0/372119056 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f32b8007480 con 0x7f32c806e480 2023-12-26T21:55:13.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.241+0000 7f32beffd700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f32a8067170 0x7f32a8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.241+0000 7f32beffd700 1 -- 172.21.15.131:0/372119056 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f32b8083f30 con 0x7f32c806e480 2023-12-26T21:55:13.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.241+0000 7f32bdd9b700 1 -- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f32a8067170 msgr2=0x7f32a8069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:13.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.241+0000 7f32bdd9b700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f32a8067170 0x7f32a8069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:13.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.245+0000 7f32beffd700 1 -- 172.21.15.131:0/372119056 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32b8050c80 con 0x7f32c806e480 2023-12-26T21:55:13.270 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:13.268+0000 7fb077d431c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:55:13.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.442+0000 7f32bdd9b700 1 -- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f32a8067170 msgr2=0x7f32a8069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:13.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.442+0000 7f32bdd9b700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f32a8067170 0x7f32a8069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:13.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.489+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32c80657e0 con 0x7f32c806e480 2023-12-26T21:55:13.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.490+0000 7f32beffd700 1 -- 172.21.15.131:0/372119056 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f32b8017020 con 0x7f32c806e480 2023-12-26T21:55:13.491 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:13.491 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2416144644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2416144644},{"type":"v1","addr":"172.21.15.131:6835","nonce":2416144644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/559791617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":559791617},{"type":"v1","addr":"172.21.15.190:6834","nonce":559791617}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6901,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3954759429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3954759429},{"type":"v1","addr":"172.21.15.190:6835","nonce":3954759429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:45.744809+0000","modified":"2023-12-26T21:53:49.296563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6834/3631709496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3631709496},{"type":"v1","addr":"172.21.15.131:6834","nonce":3631709496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-12-26T21:55:13.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f32a8067170 msgr2=0x7f32a8069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:13.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f32a8067170 0x7f32a8069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c806e480 msgr2=0x7f32c80c1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c806e480 0x7f32c80c1d20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f32b800c590 tx=0x7f32b8007f70 comp rx=0 tx=0).stop 2023-12-26T21:55:13.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 shutdown_connections 2023-12-26T21:55:13.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8159820 0x7f32c8169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f32a8067170 0x7f32a8069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32c806e480 0x7f32c80c1d20 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 --2- 172.21.15.131:0/372119056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c80aa480 0x7f32c80c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 >> 172.21.15.131:0/372119056 conn(0x7f32c806c290 msgr2=0x7f32c8156ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:13.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.492+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 shutdown_connections 2023-12-26T21:55:13.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.493+0000 7f32cc9ff700 1 -- 172.21.15.131:0/372119056 wait complete. 2023-12-26T21:55:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 50 2023-12-26T21:55:13.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.701+0000 7f9dfe5ff700 1 Processor -- start 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.701+0000 7f9dfe5ff700 1 -- start start 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.702+0000 7f9dfe5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df80abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.702+0000 7f9dfe5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9df80aa480 0x7f9df80675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.702+0000 7f9dfe5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df815a8c0 con 0x7f9df8159820 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.702+0000 7f9dfe5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df815ad50 con 0x7f9df80aa480 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.702+0000 7f9dfe5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df8159f90 con 0x7f9df806e480 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9dfcb9c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9df80aa480 0x7f9df80675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df7fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9df8159820 0x7f9df806dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:48718/0 (socket says 172.21.15.131:48718) 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df7fff700 1 -- 172.21.15.131:0/1419697056 learned_addr learned my addr 172.21.15.131:0/1419697056 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:13.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df77fe700 1 --2- 172.21.15.131:0/1419697056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df80abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df6ffd700 1 -- 172.21.15.131:0/1419697056 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236656886 0 0) 0x7f9df815a8c0 con 0x7f9df8159820 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df6ffd700 1 -- 172.21.15.131:0/1419697056 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de0003570 con 0x7f9df8159820 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df77fe700 1 -- 172.21.15.131:0/1419697056 >> v2:172.21.15.190:3300/0 conn(0x7f9df80aa480 msgr2=0x7f9df80675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df77fe700 1 --2- 172.21.15.131:0/1419697056 >> v2:172.21.15.190:3300/0 conn(0x7f9df80aa480 0x7f9df80675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df77fe700 1 -- 172.21.15.131:0/1419697056 >> v1:172.21.15.190:6789/0 conn(0x7f9df8159820 legacy=0x7f9df806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df77fe700 1 -- 172.21.15.131:0/1419697056 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df815a270 con 0x7f9df806e480 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9dfcb9c700 1 --2- 172.21.15.131:0/1419697056 >> v2:172.21.15.190:3300/0 conn(0x7f9df80aa480 0x7f9df80675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.703+0000 7f9df77fe700 1 --2- 172.21.15.131:0/1419697056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df80abdb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0098e0 tx=0x7f9dec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5aca91b5e75c296 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9df6ffd700 1 -- 172.21.15.131:0/1419697056 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec011630 con 0x7f9df806e480 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9df6ffd700 1 -- 172.21.15.131:0/1419697056 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec011790 con 0x7f9df806e480 2023-12-26T21:55:13.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9df6ffd700 1 -- 172.21.15.131:0/1419697056 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec011b30 con 0x7f9df806e480 2023-12-26T21:55:13.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/1419697056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 msgr2=0x7f9df80abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9dfe5ff700 1 --2- 172.21.15.131:0/1419697056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df80abdb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0098e0 tx=0x7f9dec0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/1419697056 shutdown_connections 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9dfe5ff700 1 --2- 172.21.15.131:0/1419697056 >> v2:172.21.15.190:3300/0 conn(0x7f9df80aa480 0x7f9df80675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9dfe5ff700 1 --2- 172.21.15.131:0/1419697056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df80abdb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/1419697056 >> 172.21.15.131:0/1419697056 conn(0x7f9df806c290 msgr2=0x7f9df80ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.704+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/1419697056 shutdown_connections 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.705+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/1419697056 wait complete. 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.706+0000 7f9dfe5ff700 1 Processor -- start 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.706+0000 7f9dfe5ff700 1 -- start start 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9dfe5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df8144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9dfe5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df80aa480 0x7f9df80c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9dfe5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9df8159820 0x7f9df80c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9dfe5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df8144840 con 0x7f9df8159820 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9dfe5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80b1280 con 0x7f9df80aa480 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9dfe5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80b0670 con 0x7f9df806e480 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9df7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df8144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9dfcb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9df8159820 0x7f9df80c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9df77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df80aa480 0x7f9df80c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9dfcb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9df8159820 0x7f9df80c4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38992/0 (socket says 172.21.15.131:38992) 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9df77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df80aa480 0x7f9df80c4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:40598/0 (socket says 172.21.15.131:40598) 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.707+0000 7f9dfcb9c700 1 -- 172.21.15.131:0/2472621616 learned_addr learned my addr 172.21.15.131:0/2472621616 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9dfcb9c700 1 -- 172.21.15.131:0/2472621616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 msgr2=0x7f9df8144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9dfcb9c700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df8144300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9dfcb9c700 1 -- 172.21.15.131:0/2472621616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df80aa480 msgr2=0x7f9df80c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9dfcb9c700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df80aa480 0x7f9df80c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9dfcb9c700 1 -- 172.21.15.131:0/2472621616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df0009a20 con 0x7f9df8159820 2023-12-26T21:55:13.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9df7fff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df8144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:13.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9dfcb9c700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9df8159820 0x7f9df80c4890 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009f90 tx=0x7f9df000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:13.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9ddffff700 1 -- 172.21.15.131:0/2472621616 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000cbd0 con 0x7f9df8159820 2023-12-26T21:55:13.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9ddffff700 1 -- 172.21.15.131:0/2472621616 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df000f040 con 0x7f9df8159820 2023-12-26T21:55:13.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.708+0000 7f9ddffff700 1 -- 172.21.15.131:0/2472621616 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0014480 con 0x7f9df8159820 2023-12-26T21:55:13.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.709+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec009420 con 0x7f9df8159820 2023-12-26T21:55:13.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.709+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df80b0c30 con 0x7f9df8159820 2023-12-26T21:55:13.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.710+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df80657e0 con 0x7f9df8159820 2023-12-26T21:55:13.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.711+0000 7f9ddffff700 1 -- 172.21.15.131:0/2472621616 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f9df00147f0 con 0x7f9df8159820 2023-12-26T21:55:13.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.711+0000 7f9ddffff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9de4067170 0x7f9de4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:13.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.711+0000 7f9ddffff700 1 -- 172.21.15.131:0/2472621616 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f9df0085970 con 0x7f9df8159820 2023-12-26T21:55:13.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.712+0000 7f9df7fff700 1 -- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9de4067170 msgr2=0x7f9de4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:13.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.712+0000 7f9df7fff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9de4067170 0x7f9de4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:13.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.715+0000 7f9ddffff700 1 -- 172.21.15.131:0/2472621616 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9df00525b0 con 0x7f9df8159820 2023-12-26T21:55:13.832 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:13.832+0000 7f483205b1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:55:13.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.913+0000 7f9df7fff700 1 -- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9de4067170 msgr2=0x7f9de4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:13.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.913+0000 7f9df7fff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9de4067170 0x7f9de4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:13.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.915+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9df815a550 con 0x7f9df8159820 2023-12-26T21:55:13.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.916+0000 7f9ddffff700 1 -- 172.21.15.131:0/2472621616 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16381 (secure 0 0 0) 0x7f9df0018020 con 0x7f9df8159820 2023-12-26T21:55:13.916 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:13.916 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":111,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:54:06.560587+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:43.922429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:44.731388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1281002455":"2023-12-27T21:54:06.560556+0000","172.21.15.131:6836/39288":"2023-12-27T21:54:06.560556+0000","172.21.15.131:0/2183813711":"2023-12-27T21:54:06.560556+0000","172.21.15.131:0/1210036286":"2023-12-27T21:54:06.560556+0000","172.21.15.131:6837/39288":"2023-12-27T21:54:06.560556+0000","172.21.15.131:0/2864499457":"2023-12-27T21:54:06.560556+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:55:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9de4067170 msgr2=0x7f9de4069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9de4067170 0x7f9de4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9df8159820 msgr2=0x7f9df80c4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9df8159820 0x7f9df80c4890 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009f90 tx=0x7f9df000b040 comp rx=0 tx=0).stop 2023-12-26T21:55:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 shutdown_connections 2023-12-26T21:55:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9de4067170 0x7f9de4069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9df8159820 0x7f9df80c4890 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df806e480 0x7f9df8144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 --2- 172.21.15.131:0/2472621616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df80aa480 0x7f9df80c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 >> 172.21.15.131:0/2472621616 conn(0x7f9df806c290 msgr2=0x7f9df815bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 shutdown_connections 2023-12-26T21:55:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:13.918+0000 7f9dfe5ff700 1 -- 172.21.15.131:0/2472621616 wait complete. 2023-12-26T21:55:13.925 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:13.924+0000 7f483205b1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:55:13.938 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T21:55:13.938 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:55:14.039 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:14.038+0000 7fb077d431c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:55:14.076 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:14.076+0000 7f483205b1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:55:14.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.131+0000 7f9f329ff700 1 Processor -- start 2023-12-26T21:55:14.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.131+0000 7f9f329ff700 1 -- start start 2023-12-26T21:55:14.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.132+0000 7f9f329ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f2c06f0e0 0x7f9f2c06f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.132+0000 7f9f329ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c15b5c0 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.132+0000 7f9f329ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c158530 con 0x7f9f2c15ba50 2023-12-26T21:55:14.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.132+0000 7f9f329ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c158af0 con 0x7f9f2c06f0e0 2023-12-26T21:55:14.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.133+0000 7f9f2bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f2c06f0e0 0x7f9f2c06f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:14.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.133+0000 7f9f2bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f2c06f0e0 0x7f9f2c06f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39002/0 (socket says 172.21.15.131:39002) 2023-12-26T21:55:14.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.133+0000 7f9f2bfff700 1 -- 172.21.15.131:0/2295712285 learned_addr learned my addr 172.21.15.131:0/2295712285 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.133+0000 7f9f2bfff700 1 -- 172.21.15.131:0/2295712285 >> v1:172.21.15.190:6789/0 conn(0x7f9f2c15ba50 legacy=0x7f9f2c157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.133+0000 7f9f2bfff700 1 -- 172.21.15.131:0/2295712285 >> v1:172.21.15.131:6789/0 conn(0x7f9f2c0b23f0 legacy=0x7f9f2c0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.133+0000 7f9f2bfff700 1 -- 172.21.15.131:0/2295712285 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c158810 con 0x7f9f2c06f0e0 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.133+0000 7f9f2bfff700 1 --2- 172.21.15.131:0/2295712285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c06f0e0 0x7f9f2c06f4b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0099b0 tx=0x7f9f1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d004a2e280b462d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.133+0000 7f9f2affd700 1 -- 172.21.15.131:0/2295712285 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00b860 con 0x7f9f2c06f0e0 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.134+0000 7f9f2affd700 1 -- 172.21.15.131:0/2295712285 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c00b9c0 con 0x7f9f2c06f0e0 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.134+0000 7f9f2affd700 1 -- 172.21.15.131:0/2295712285 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00bd60 con 0x7f9f2c06f0e0 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.134+0000 7f9f329ff700 1 -- 172.21.15.131:0/2295712285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c06f0e0 msgr2=0x7f9f2c06f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.134+0000 7f9f329ff700 1 --2- 172.21.15.131:0/2295712285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c06f0e0 0x7f9f2c06f4b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0099b0 tx=0x7f9f1c009250 comp rx=0 tx=0).stop 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.134+0000 7f9f329ff700 1 -- 172.21.15.131:0/2295712285 shutdown_connections 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.134+0000 7f9f329ff700 1 --2- 172.21.15.131:0/2295712285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c06f0e0 0x7f9f2c06f4b0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.134+0000 7f9f329ff700 1 -- 172.21.15.131:0/2295712285 >> 172.21.15.131:0/2295712285 conn(0x7f9f2c0686c0 msgr2=0x7f9f2c068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:14.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.134+0000 7f9f329ff700 1 -- 172.21.15.131:0/2295712285 shutdown_connections 2023-12-26T21:55:14.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.134+0000 7f9f329ff700 1 -- 172.21.15.131:0/2295712285 wait complete. 2023-12-26T21:55:14.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.136+0000 7f9f329ff700 1 Processor -- start 2023-12-26T21:55:14.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.136+0000 7f9f329ff700 1 -- start start 2023-12-26T21:55:14.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f329ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c06f0e0 0x7f9f2c14b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f329ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c0b23f0 0x7f9f2c0ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f329ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c15ba50 0x7f9f2c152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f329ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c0ade10 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f329ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c0ad200 con 0x7f9f2c15ba50 2023-12-26T21:55:14.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f329ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c0ad4e0 con 0x7f9f2c06f0e0 2023-12-26T21:55:14.138 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:14.137+0000 7fb077d431c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:55:14.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f2b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c0b23f0 0x7f9f2c0ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:14.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f2bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c06f0e0 0x7f9f2c14b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:14.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f2b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c0b23f0 0x7f9f2c0ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39016/0 (socket says 172.21.15.131:39016) 2023-12-26T21:55:14.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f2b7fe700 1 -- 172.21.15.131:0/1666501140 learned_addr learned my addr 172.21.15.131:0/1666501140 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:14.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.137+0000 7f9f30f9c700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c15ba50 0x7f9f2c152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:14.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f2b7fe700 1 -- 172.21.15.131:0/1666501140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c06f0e0 msgr2=0x7f9f2c14b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f2b7fe700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c06f0e0 0x7f9f2c14b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f2b7fe700 1 -- 172.21.15.131:0/1666501140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c15ba50 msgr2=0x7f9f2c152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f2b7fe700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c15ba50 0x7f9f2c152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f2b7fe700 1 -- 172.21.15.131:0/1666501140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0094f0 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f2b7fe700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c0b23f0 0x7f9f2c0ae2a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000c4e0 tx=0x7f9f2000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:14.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f13fff700 1 -- 172.21.15.131:0/1666501140 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20011590 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f13fff700 1 -- 172.21.15.131:0/1666501140 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f20011710 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c0ad7c0 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f13fff700 1 -- 172.21.15.131:0/1666501140 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20011ad0 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.138+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c14bb60 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.139+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c0adfc0 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.141+0000 7f9f13fff700 1 -- 172.21.15.131:0/1666501140 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f9f2001b3b0 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.142+0000 7f9f13fff700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f180671a0 0x7f9f18069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.142+0000 7f9f13fff700 1 -- 172.21.15.131:0/1666501140 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f9f20083eb0 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.142+0000 7f9f2bfff700 1 -- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f180671a0 msgr2=0x7f9f18069650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:14.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.142+0000 7f9f2bfff700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f180671a0 0x7f9f18069650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:14.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.146+0000 7f9f13fff700 1 -- 172.21.15.131:0/1666501140 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f20050c00 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.299 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:14.298+0000 7fb077d431c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:55:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.343+0000 7f9f2bfff700 1 -- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f180671a0 msgr2=0x7f9f18069650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.343+0000 7f9f2bfff700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f180671a0 0x7f9f18069650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:14.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.396+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f2c0657e0 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.397+0000 7f9f13fff700 1 -- 172.21.15.131:0/1666501140 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f9f20017070 con 0x7f9f2c0b23f0 2023-12-26T21:55:14.397 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:14.398 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2416144644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2416144644},{"type":"v1","addr":"172.21.15.131:6835","nonce":2416144644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/559791617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":559791617},{"type":"v1","addr":"172.21.15.190:6834","nonce":559791617}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6901,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3954759429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3954759429},{"type":"v1","addr":"172.21.15.190:6835","nonce":3954759429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:45.744809+0000","modified":"2023-12-26T21:53:49.296563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6834/3631709496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3631709496},{"type":"v1","addr":"172.21.15.131:6834","nonce":3631709496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-12-26T21:55:14.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f180671a0 msgr2=0x7f9f18069650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:14.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f180671a0 0x7f9f18069650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c0b23f0 msgr2=0x7f9f2c0ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c0b23f0 0x7f9f2c0ae2a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000c4e0 tx=0x7f9f2000c7f0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 shutdown_connections 2023-12-26T21:55:14.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c15ba50 0x7f9f2c152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c06f0e0 0x7f9f2c14b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f180671a0 0x7f9f18069650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 --2- 172.21.15.131:0/1666501140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2c0b23f0 0x7f9f2c0ae2a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 >> 172.21.15.131:0/1666501140 conn(0x7f9f2c0686c0 msgr2=0x7f9f2c15a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:14.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 shutdown_connections 2023-12-26T21:55:14.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.399+0000 7f9f329ff700 1 -- 172.21.15.131:0/1666501140 wait complete. 2023-12-26T21:55:14.401 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 50 2023-12-26T21:55:14.418 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:55:14.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.613+0000 7fcdbc3ff700 1 Processor -- start 2023-12-26T21:55:14.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.614+0000 7fcdbc3ff700 1 -- start start 2023-12-26T21:55:14.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.615+0000 7fcdbc3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb40b07f0 0x7fcdb41554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.615+0000 7fcdbc3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcdb4159cf0 0x7fcdb4155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.615+0000 7fcdbc3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb40c8bf0 con 0x7fcdb4159920 2023-12-26T21:55:14.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.615+0000 7fcdbc3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb4155f50 con 0x7fcdb4159cf0 2023-12-26T21:55:14.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdbc3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb40c8e60 con 0x7fcdb40b07f0 2023-12-26T21:55:14.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdba19b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcdb4159920 0x7fcdb40b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44298/0 (socket says 172.21.15.131:44298) 2023-12-26T21:55:14.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdba19b700 1 -- 172.21.15.131:0/3356841872 learned_addr learned my addr 172.21.15.131:0/3356841872 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:14.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdba99c700 1 --2- 172.21.15.131:0/3356841872 >> v2:172.21.15.190:3300/0 conn(0x7fcdb4159cf0 0x7fcdb4155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:14.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdb999a700 1 --2- 172.21.15.131:0/3356841872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb40b07f0 0x7fcdb41554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:14.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdb9199700 1 -- 172.21.15.131:0/3356841872 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155715276 0 0) 0x7fcdb40c8bf0 con 0x7fcdb4159920 2023-12-26T21:55:14.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdb9199700 1 -- 172.21.15.131:0/3356841872 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcda0003570 con 0x7fcdb4159920 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdb999a700 1 -- 172.21.15.131:0/3356841872 >> v2:172.21.15.190:3300/0 conn(0x7fcdb4159cf0 msgr2=0x7fcdb4155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdb999a700 1 --2- 172.21.15.131:0/3356841872 >> v2:172.21.15.190:3300/0 conn(0x7fcdb4159cf0 0x7fcdb4155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdb999a700 1 -- 172.21.15.131:0/3356841872 >> v1:172.21.15.131:6789/0 conn(0x7fcdb4159920 legacy=0x7fcdb40b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdb999a700 1 -- 172.21.15.131:0/3356841872 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb40c9140 con 0x7fcdb40b07f0 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.616+0000 7fcdba99c700 1 --2- 172.21.15.131:0/3356841872 >> v2:172.21.15.190:3300/0 conn(0x7fcdb4159cf0 0x7fcdb4155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.617+0000 7fcdb999a700 1 --2- 172.21.15.131:0/3356841872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb40b07f0 0x7fcdb41554d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcda400aa90 tx=0x7fcda400a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=818867643e585f51 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.617+0000 7fcdb9199700 1 -- 172.21.15.131:0/3356841872 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400c970 con 0x7fcdb40b07f0 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.617+0000 7fcdb9199700 1 -- 172.21.15.131:0/3356841872 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda400cad0 con 0x7fcdb40b07f0 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.617+0000 7fcdb9199700 1 -- 172.21.15.131:0/3356841872 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400ce70 con 0x7fcdb40b07f0 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.618+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/3356841872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb40b07f0 msgr2=0x7fcdb41554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.618+0000 7fcdbc3ff700 1 --2- 172.21.15.131:0/3356841872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb40b07f0 0x7fcdb41554d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcda400aa90 tx=0x7fcda400a330 comp rx=0 tx=0).stop 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.618+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/3356841872 shutdown_connections 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.618+0000 7fcdbc3ff700 1 --2- 172.21.15.131:0/3356841872 >> v2:172.21.15.190:3300/0 conn(0x7fcdb4159cf0 0x7fcdb4155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.618+0000 7fcdbc3ff700 1 --2- 172.21.15.131:0/3356841872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb40b07f0 0x7fcdb41554d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.618+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/3356841872 >> 172.21.15.131:0/3356841872 conn(0x7fcdb4068040 msgr2=0x7fcdb4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:14.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.618+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/3356841872 shutdown_connections 2023-12-26T21:55:14.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.618+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/3356841872 wait complete. 2023-12-26T21:55:14.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.619+0000 7fcdbc3ff700 1 Processor -- start 2023-12-26T21:55:14.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.620+0000 7fcdbc3ff700 1 -- start start 2023-12-26T21:55:14.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.620+0000 7fcdbc3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdb4159920 0x7fcdb415d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.620+0000 7fcdbc3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdb4159cf0 0x7fcdb41548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.620+0000 7fcdbc3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb4156e80 0x7fcdb415c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdbc3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb415e650 con 0x7fcdb4159920 2023-12-26T21:55:14.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdbc3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb415da40 con 0x7fcdb4159cf0 2023-12-26T21:55:14.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdbc3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb415dd20 con 0x7fcdb4156e80 2023-12-26T21:55:14.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdba19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdb4159920 0x7fcdb415d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:14.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdba19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdb4159920 0x7fcdb415d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39022/0 (socket says 172.21.15.131:39022) 2023-12-26T21:55:14.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdba19b700 1 -- 172.21.15.131:0/2492484065 learned_addr learned my addr 172.21.15.131:0/2492484065 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:14.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdb999a700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdb4159cf0 0x7fcdb41548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:14.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdba99c700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb4156e80 0x7fcdb415c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:14.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdb999a700 1 -- 172.21.15.131:0/2492484065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb4156e80 msgr2=0x7fcdb415c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdb999a700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb4156e80 0x7fcdb415c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdb999a700 1 -- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdb4159920 msgr2=0x7fcdb415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdb999a700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdb4159920 0x7fcdb415d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdb999a700 1 -- 172.21.15.131:0/2492484065 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac009a20 con 0x7fcdb4159cf0 2023-12-26T21:55:14.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.621+0000 7fcdba19b700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdb4159920 0x7fcdb415d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:14.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.622+0000 7fcdb999a700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdb4159cf0 0x7fcdb41548c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcda4017000 tx=0x7fcda401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:14.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.622+0000 7fcdaa7fc700 1 -- 172.21.15.131:0/2492484065 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4021430 con 0x7fcdb4159cf0 2023-12-26T21:55:14.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.622+0000 7fcdaa7fc700 1 -- 172.21.15.131:0/2492484065 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda4021590 con 0x7fcdb4159cf0 2023-12-26T21:55:14.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.622+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda400a5d0 con 0x7fcdb4159cf0 2023-12-26T21:55:14.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.622+0000 7fcdaa7fc700 1 -- 172.21.15.131:0/2492484065 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4021950 con 0x7fcdb4159cf0 2023-12-26T21:55:14.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.622+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb415e2e0 con 0x7fcdb4159cf0 2023-12-26T21:55:14.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.623+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb40657e0 con 0x7fcdb4159cf0 2023-12-26T21:55:14.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.624+0000 7fcdaa7fc700 1 -- 172.21.15.131:0/2492484065 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7fcda40103f0 con 0x7fcdb4159cf0 2023-12-26T21:55:14.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.624+0000 7fcdaa7fc700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd98067170 0x7fcd98069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:14.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.625+0000 7fcdaa7fc700 1 -- 172.21.15.131:0/2492484065 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fcda40860c0 con 0x7fcdb4159cf0 2023-12-26T21:55:14.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.625+0000 7fcdba19b700 1 -- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd98067170 msgr2=0x7fcd98069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:14.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.625+0000 7fcdba19b700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd98067170 0x7fcd98069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:14.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.629+0000 7fcdaa7fc700 1 -- 172.21.15.131:0/2492484065 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcda4052ee0 con 0x7fcdb4159cf0 2023-12-26T21:55:14.703 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:14.702+0000 7f483205b1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:55:14.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.826+0000 7fcdba19b700 1 -- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd98067170 msgr2=0x7fcd98069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:14.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.826+0000 7fcdba19b700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd98067170 0x7fcd98069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:14.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.888+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdb40c9420 con 0x7fcdb4159cf0 2023-12-26T21:55:14.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.889+0000 7fcdaa7fc700 1 -- 172.21.15.131:0/2492484065 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fcda4056500 con 0x7fcdb4159cf0 2023-12-26T21:55:14.890 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:14.890 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2416144644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2416144644},{"type":"v1","addr":"172.21.15.131:6835","nonce":2416144644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/559791617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":559791617},{"type":"v1","addr":"172.21.15.190:6834","nonce":559791617}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6901,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3954759429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3954759429},{"type":"v1","addr":"172.21.15.190:6835","nonce":3954759429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:53:45.744809+0000","modified":"2023-12-26T21:53:49.296563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6834/3631709496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3631709496},{"type":"v1","addr":"172.21.15.131:6834","nonce":3631709496}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-12-26T21:55:14.890 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:14.890+0000 7f483205b1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd98067170 msgr2=0x7fcd98069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd98067170 0x7fcd98069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdb4159cf0 msgr2=0x7fcdb41548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdb4159cf0 0x7fcdb41548c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcda4017000 tx=0x7fcda401a000 comp rx=0 tx=0).stop 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 shutdown_connections 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdb4156e80 0x7fcdb415c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd98067170 0x7fcd98069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdb4159920 0x7fcdb415d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 --2- 172.21.15.131:0/2492484065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdb4159cf0 0x7fcdb41548c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 >> 172.21.15.131:0/2492484065 conn(0x7fcdb4068040 msgr2=0x7fcdb406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:14.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 shutdown_connections 2023-12-26T21:55:14.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:14.892+0000 7fcdbc3ff700 1 -- 172.21.15.131:0/2492484065 wait complete. 2023-12-26T21:55:14.894 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 50 2023-12-26T21:55:14.911 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:55:14.972 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:14.971+0000 7fb077d431c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:55:15.079 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:15.079+0000 7f483205b1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:55:15.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.110+0000 7fd69d9ff700 1 Processor -- start 2023-12-26T21:55:15.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.110+0000 7fd69d9ff700 1 -- start start 2023-12-26T21:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd69d9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd6981599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd69d9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6980ad560 0x7fd698159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd69d9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980a98b0 con 0x7fd69815d240 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd69d9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980add40 con 0x7fd6980ad560 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd69d9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69815a500 con 0x7fd6981595e0 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd6967fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd69815d240 0x7fd6980ace50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48518/0 (socket says 172.21.15.131:48518) 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd6967fc700 1 -- 172.21.15.131:0/2935485192 learned_addr learned my addr 172.21.15.131:0/2935485192 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd6977fe700 1 --2- 172.21.15.131:0/2935485192 >> v2:172.21.15.190:3300/0 conn(0x7fd6980ad560 0x7fd698159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd696ffd700 1 --2- 172.21.15.131:0/2935485192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd6981599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd695ffb700 1 -- 172.21.15.131:0/2935485192 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581110476 0 0) 0x7fd6980a98b0 con 0x7fd69815d240 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd695ffb700 1 -- 172.21.15.131:0/2935485192 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd680003570 con 0x7fd69815d240 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.112+0000 7fd6977fe700 1 -- 172.21.15.131:0/2935485192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 msgr2=0x7fd6981599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.113+0000 7fd6977fe700 1 --2- 172.21.15.131:0/2935485192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd6981599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.113+0000 7fd6977fe700 1 -- 172.21.15.131:0/2935485192 >> v1:172.21.15.131:6789/0 conn(0x7fd69815d240 legacy=0x7fd6980ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.113+0000 7fd6977fe700 1 -- 172.21.15.131:0/2935485192 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69815a7e0 con 0x7fd6980ad560 2023-12-26T21:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.113+0000 7fd696ffd700 1 --2- 172.21.15.131:0/2935485192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd6981599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.113+0000 7fd6977fe700 1 --2- 172.21.15.131:0/2935485192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980ad560 0x7fd698159ef0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd690009e60 tx=0x7fd690009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0219dd4dd7f048d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.113+0000 7fd695ffb700 1 -- 172.21.15.131:0/2935485192 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000e070 con 0x7fd6980ad560 2023-12-26T21:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.113+0000 7fd695ffb700 1 -- 172.21.15.131:0/2935485192 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69000bab0 con 0x7fd6980ad560 2023-12-26T21:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.113+0000 7fd695ffb700 1 -- 172.21.15.131:0/2935485192 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000be70 con 0x7fd6980ad560 2023-12-26T21:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.114+0000 7fd69d9ff700 1 -- 172.21.15.131:0/2935485192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980ad560 msgr2=0x7fd698159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.114+0000 7fd69d9ff700 1 --2- 172.21.15.131:0/2935485192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980ad560 0x7fd698159ef0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd690009e60 tx=0x7fd690009780 comp rx=0 tx=0).stop 2023-12-26T21:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.114+0000 7fd69d9ff700 1 -- 172.21.15.131:0/2935485192 shutdown_connections 2023-12-26T21:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.114+0000 7fd69d9ff700 1 --2- 172.21.15.131:0/2935485192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6980ad560 0x7fd698159ef0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.114+0000 7fd69d9ff700 1 --2- 172.21.15.131:0/2935485192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd6981599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.114+0000 7fd69d9ff700 1 -- 172.21.15.131:0/2935485192 >> 172.21.15.131:0/2935485192 conn(0x7fd698068040 msgr2=0x7fd698069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.114+0000 7fd69d9ff700 1 -- 172.21.15.131:0/2935485192 shutdown_connections 2023-12-26T21:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.114+0000 7fd69d9ff700 1 -- 172.21.15.131:0/2935485192 wait complete. 2023-12-26T21:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.115+0000 7fd69d9ff700 1 Processor -- start 2023-12-26T21:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.115+0000 7fd69d9ff700 1 -- start start 2023-12-26T21:55:15.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.116+0000 7fd69d9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6980ad560 0x7fd6981524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.116+0000 7fd69d9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd69815bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.116+0000 7fd69d9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815d240 0x7fd69816b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.116+0000 7fd69d9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69816af80 con 0x7fd6980ad560 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.116+0000 7fd69d9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69815aea0 con 0x7fd69815d240 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.116+0000 7fd69d9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69815b180 con 0x7fd6981595e0 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd696ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6980ad560 0x7fd6981524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd696ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6980ad560 0x7fd6981524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54316/0 (socket says 172.21.15.131:54316) 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd6967fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd69815bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd696ffd700 1 -- 172.21.15.131:0/4221585942 learned_addr learned my addr 172.21.15.131:0/4221585942 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd6977fe700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815d240 0x7fd69816b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd6967fc700 1 -- 172.21.15.131:0/4221585942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815d240 msgr2=0x7fd69816b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd6967fc700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815d240 0x7fd69816b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd6967fc700 1 -- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6980ad560 msgr2=0x7fd6981524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd6967fc700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6980ad560 0x7fd6981524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd6967fc700 1 -- 172.21.15.131:0/4221585942 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688009420 con 0x7fd6981595e0 2023-12-26T21:55:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd696ffd700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6980ad560 0x7fd6981524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.117+0000 7fd6967fc700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd69815bc60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c590 tx=0x7fd68c007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.118+0000 7fd67effd700 1 -- 172.21.15.131:0/4221585942 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c005b30 con 0x7fd6981595e0 2023-12-26T21:55:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.118+0000 7fd67effd700 1 -- 172.21.15.131:0/4221585942 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c005cb0 con 0x7fd6981595e0 2023-12-26T21:55:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.118+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd690009a20 con 0x7fd6981595e0 2023-12-26T21:55:15.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.118+0000 7fd67effd700 1 -- 172.21.15.131:0/4221585942 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c007320 con 0x7fd6981595e0 2023-12-26T21:55:15.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.118+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69815b740 con 0x7fd6981595e0 2023-12-26T21:55:15.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.119+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69815b900 con 0x7fd6981595e0 2023-12-26T21:55:15.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.120+0000 7fd67effd700 1 -- 172.21.15.131:0/4221585942 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7fd68c01b3b0 con 0x7fd6981595e0 2023-12-26T21:55:15.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.120+0000 7fd67effd700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd684067170 0x7fd684069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.121+0000 7fd67effd700 1 -- 172.21.15.131:0/4221585942 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fd68c083f00 con 0x7fd6981595e0 2023-12-26T21:55:15.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.121+0000 7fd696ffd700 1 -- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd684067170 msgr2=0x7fd684069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:15.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.121+0000 7fd696ffd700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd684067170 0x7fd684069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:15.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.125+0000 7fd67effd700 1 -- 172.21.15.131:0/4221585942 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd68c050bb0 con 0x7fd6981595e0 2023-12-26T21:55:15.174 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:15.171+0000 7fb077d431c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:55:15.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.322+0000 7fd696ffd700 1 -- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd684067170 msgr2=0x7fd684069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:15.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.322+0000 7fd696ffd700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd684067170 0x7fd684069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:15.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.330+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd69815b2c0 con 0x7fd6981595e0 2023-12-26T21:55:15.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.331+0000 7fd67effd700 1 -- 172.21.15.131:0/4221585942 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16381 (secure 0 0 0) 0x7fd69815b2c0 con 0x7fd6981595e0 2023-12-26T21:55:15.332 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:15.333 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":111,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:54:06.560587+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:53:43.922429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-26T21:53:44.731388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1281002455":"2023-12-27T21:54:06.560556+0000","172.21.15.131:6836/39288":"2023-12-27T21:54:06.560556+0000","172.21.15.131:0/2183813711":"2023-12-27T21:54:06.560556+0000","172.21.15.131:0/1210036286":"2023-12-27T21:54:06.560556+0000","172.21.15.131:6837/39288":"2023-12-27T21:54:06.560556+0000","172.21.15.131:0/2864499457":"2023-12-27T21:54:06.560556+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:55:15.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd684067170 msgr2=0x7fd684069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:15.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd684067170 0x7fd684069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 msgr2=0x7fd69815bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd69815bc60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c590 tx=0x7fd68c007d50 comp rx=0 tx=0).stop 2023-12-26T21:55:15.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 shutdown_connections 2023-12-26T21:55:15.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815d240 0x7fd69816b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd684067170 0x7fd684069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6980ad560 0x7fd6981524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 --2- 172.21.15.131:0/4221585942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6981595e0 0x7fd69815bc60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 >> 172.21.15.131:0/4221585942 conn(0x7fd698068040 msgr2=0x7fd6980aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:15.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 shutdown_connections 2023-12-26T21:55:15.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.334+0000 7fd69d9ff700 1 -- 172.21.15.131:0/4221585942 wait complete. 2023-12-26T21:55:15.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T21:55:15.375 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:15.374+0000 7fb077d431c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:55:15.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.559+0000 7f8147bff700 1 Processor -- start 2023-12-26T21:55:15.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.559+0000 7f8147bff700 1 -- start start 2023-12-26T21:55:15.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.560+0000 7f8147bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81401595e0 0x7f81401599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.560+0000 7f8147bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81400ad560 0x7f8140159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.560+0000 7f8147bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81400a98b0 con 0x7f814015d240 2023-12-26T21:55:15.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.560+0000 7f8147bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81400add40 con 0x7f81401595e0 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.560+0000 7f8147bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814015a500 con 0x7f81400ad560 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.561+0000 7f814599b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81401595e0 0x7f81401599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.561+0000 7f814599b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81401595e0 0x7f81401599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54332/0 (socket says 172.21.15.131:54332) 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.561+0000 7f814599b700 1 -- 172.21.15.131:0/3585039445 learned_addr learned my addr 172.21.15.131:0/3585039445 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.561+0000 7f814619c700 1 --2- 172.21.15.131:0/3585039445 >> v2:172.21.15.190:3300/0 conn(0x7f81400ad560 0x7f8140159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.561+0000 7f814599b700 1 -- 172.21.15.131:0/3585039445 >> v2:172.21.15.190:3300/0 conn(0x7f81400ad560 msgr2=0x7f8140159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.561+0000 7f814599b700 1 --2- 172.21.15.131:0/3585039445 >> v2:172.21.15.190:3300/0 conn(0x7f81400ad560 0x7f8140159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.561+0000 7f814599b700 1 -- 172.21.15.131:0/3585039445 >> v1:172.21.15.131:6789/0 conn(0x7f814015d240 legacy=0x7f81400ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.561+0000 7f814599b700 1 -- 172.21.15.131:0/3585039445 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814015a7e0 con 0x7f81401595e0 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.561+0000 7f814599b700 1 --2- 172.21.15.131:0/3585039445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81401595e0 0x7f81401599b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f814006b450 tx=0x7f813c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc55501358ddb51 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:15.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8144999700 1 -- 172.21.15.131:0/3585039445 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c009e70 con 0x7f81401595e0 2023-12-26T21:55:15.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8144999700 1 -- 172.21.15.131:0/3585039445 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c00b860 con 0x7f81401595e0 2023-12-26T21:55:15.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8144999700 1 -- 172.21.15.131:0/3585039445 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c00bc00 con 0x7f81401595e0 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8147bff700 1 -- 172.21.15.131:0/3585039445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81401595e0 msgr2=0x7f81401599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8147bff700 1 --2- 172.21.15.131:0/3585039445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81401595e0 0x7f81401599b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f814006b450 tx=0x7f813c009250 comp rx=0 tx=0).stop 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8147bff700 1 -- 172.21.15.131:0/3585039445 shutdown_connections 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8147bff700 1 --2- 172.21.15.131:0/3585039445 >> v2:172.21.15.190:3300/0 conn(0x7f81400ad560 0x7f8140159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8147bff700 1 --2- 172.21.15.131:0/3585039445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81401595e0 0x7f81401599b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8147bff700 1 -- 172.21.15.131:0/3585039445 >> 172.21.15.131:0/3585039445 conn(0x7f8140068040 msgr2=0x7f8140069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8147bff700 1 -- 172.21.15.131:0/3585039445 shutdown_connections 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.562+0000 7f8147bff700 1 -- 172.21.15.131:0/3585039445 wait complete. 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.564+0000 7f8147bff700 1 Processor -- start 2023-12-26T21:55:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.564+0000 7f8147bff700 1 -- start start 2023-12-26T21:55:15.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f8147bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81400ad560 0x7f814015cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f8147bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81401595e0 0x7f8140151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f8147bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814015d240 0x7f814006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f8147bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814015ad00 con 0x7f814015d240 2023-12-26T21:55:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f8147bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814015bb20 con 0x7f81400ad560 2023-12-26T21:55:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f8147bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814015d040 con 0x7f81401595e0 2023-12-26T21:55:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f814619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814015d240 0x7f814006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f814599b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81400ad560 0x7f814015cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f814619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814015d240 0x7f814006e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54334/0 (socket says 172.21.15.131:54334) 2023-12-26T21:55:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f814619c700 1 -- 172.21.15.131:0/3552608266 learned_addr learned my addr 172.21.15.131:0/3552608266 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.565+0000 7f814519a700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81401595e0 0x7f8140151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f814619c700 1 -- 172.21.15.131:0/3552608266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81401595e0 msgr2=0x7f8140151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f814619c700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81401595e0 0x7f8140151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f814619c700 1 -- 172.21.15.131:0/3552608266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81400ad560 msgr2=0x7f814015cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f814619c700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81400ad560 0x7f814015cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f814619c700 1 -- 172.21.15.131:0/3552608266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0094f0 con 0x7f814015d240 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f814599b700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81400ad560 0x7f814015cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f814619c700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814015d240 0x7f814006e4b0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8138009ec0 tx=0x7f813800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f8135ffb700 1 -- 172.21.15.131:0/3552608266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813800cb10 con 0x7f814015d240 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f8135ffb700 1 -- 172.21.15.131:0/3552608266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813800f040 con 0x7f814015d240 2023-12-26T21:55:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814015af10 con 0x7f814015d240 2023-12-26T21:55:15.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f8135ffb700 1 -- 172.21.15.131:0/3552608266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813800cee0 con 0x7f814015d240 2023-12-26T21:55:15.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.566+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814015b4d0 con 0x7f814015d240 2023-12-26T21:55:15.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.567+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81400657e0 con 0x7f814015d240 2023-12-26T21:55:15.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.570+0000 7f8135ffb700 1 -- 172.21.15.131:0/3552608266 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f8138014690 con 0x7f814015d240 2023-12-26T21:55:15.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.571+0000 7f8135ffb700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8124067170 0x7f8124069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:15.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.571+0000 7f8135ffb700 1 -- 172.21.15.131:0/3552608266 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f8138085840 con 0x7f814015d240 2023-12-26T21:55:15.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.571+0000 7f814599b700 1 -- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8124067170 msgr2=0x7f8124069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:15.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.571+0000 7f814599b700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8124067170 0x7f8124069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:15.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.575+0000 7f8135ffb700 1 -- 172.21.15.131:0/3552608266 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81380523e0 con 0x7f814015d240 2023-12-26T21:55:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.772+0000 7f814599b700 1 -- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8124067170 msgr2=0x7f8124069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.772+0000 7f814599b700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8124067170 0x7f8124069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:15.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.820+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f814015aac0 con 0x7f814015d240 2023-12-26T21:55:15.822 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T21:55:15.821+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T21:55:15.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.825+0000 7f8135ffb700 1 -- 172.21.15.131:0/3552608266 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8138055a00 con 0x7f814015d240 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.828+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8124067170 msgr2=0x7f8124069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.828+0000 7f8147bff700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8124067170 0x7f8124069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.828+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814015d240 msgr2=0x7f814006e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.828+0000 7f8147bff700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814015d240 0x7f814006e4b0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8138009ec0 tx=0x7f813800b040 comp rx=0 tx=0).stop 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.829+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 shutdown_connections 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.829+0000 7f8147bff700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8124067170 0x7f8124069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.829+0000 7f8147bff700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814015d240 0x7f814006e4b0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.829+0000 7f8147bff700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81400ad560 0x7f814015cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.829+0000 7f8147bff700 1 --2- 172.21.15.131:0/3552608266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81401595e0 0x7f8140151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.829+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 >> 172.21.15.131:0/3552608266 conn(0x7f8140068040 msgr2=0x7f81400aadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.829+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 shutdown_connections 2023-12-26T21:55:15.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:15.829+0000 7f8147bff700 1 -- 172.21.15.131:0/3552608266 wait complete. 2023-12-26T21:55:15.831 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T21:55:15.837 INFO:tasks.ceph.mds.c.smithi131.stderr: -5457> 2023-12-26T21:53:51.246+0000 7f8d6d1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:55:15.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T21:55:15.882 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:55:15.927 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:15.926+0000 7f483205b1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:55:16.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.077+0000 7f23919ff700 1 Processor -- start 2023-12-26T21:55:16.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.077+0000 7f23919ff700 1 -- start start 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f23919ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c06f0e0 0x7f238c06f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f23919ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f238c15ba50 0x7f238c157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f23919ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f238c157f90 0x7f238c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f23919ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f238c158af0 con 0x7f238c15ba50 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f23919ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f238c15b5c0 con 0x7f238c157f90 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f23919ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238c158530 con 0x7f238c06f0e0 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f238a7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f238c15ba50 0x7f238c157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f238affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c06f0e0 0x7f238c06f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f238a7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f238c15ba50 0x7f238c157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54344/0 (socket says 172.21.15.131:54344) 2023-12-26T21:55:16.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.078+0000 7f238a7fc700 1 -- 172.21.15.131:0/3151896035 learned_addr learned my addr 172.21.15.131:0/3151896035 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:16.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f238b7fe700 1 --2- 172.21.15.131:0/3151896035 >> v2:172.21.15.190:3300/0 conn(0x7f238c157f90 0x7f238c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f238affd700 1 -- 172.21.15.131:0/3151896035 >> v2:172.21.15.190:3300/0 conn(0x7f238c157f90 msgr2=0x7f238c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f238affd700 1 --2- 172.21.15.131:0/3151896035 >> v2:172.21.15.190:3300/0 conn(0x7f238c157f90 0x7f238c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f238affd700 1 -- 172.21.15.131:0/3151896035 >> v2:172.21.15.131:3300/0 conn(0x7f238c15ba50 msgr2=0x7f238c157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f238affd700 1 --2- 172.21.15.131:0/3151896035 >> v2:172.21.15.131:3300/0 conn(0x7f238c15ba50 0x7f238c157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f238affd700 1 -- 172.21.15.131:0/3151896035 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c158810 con 0x7f238c06f0e0 2023-12-26T21:55:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f238a7fc700 1 --2- 172.21.15.131:0/3151896035 >> v2:172.21.15.131:3300/0 conn(0x7f238c15ba50 0x7f238c157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:16.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f238affd700 1 --2- 172.21.15.131:0/3151896035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c06f0e0 0x7f238c06f4b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f238c0acd10 tx=0x7f237c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cedf0e1970b4848f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f2389ffb700 1 -- 172.21.15.131:0/3151896035 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c011630 con 0x7f238c06f0e0 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f2389ffb700 1 -- 172.21.15.131:0/3151896035 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c0117b0 con 0x7f238c06f0e0 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.079+0000 7f2389ffb700 1 -- 172.21.15.131:0/3151896035 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c011b70 con 0x7f238c06f0e0 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.080+0000 7f23919ff700 1 -- 172.21.15.131:0/3151896035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c06f0e0 msgr2=0x7f238c06f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.080+0000 7f23919ff700 1 --2- 172.21.15.131:0/3151896035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c06f0e0 0x7f238c06f4b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f238c0acd10 tx=0x7f237c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.080+0000 7f23919ff700 1 -- 172.21.15.131:0/3151896035 shutdown_connections 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.080+0000 7f23919ff700 1 --2- 172.21.15.131:0/3151896035 >> v2:172.21.15.190:3300/0 conn(0x7f238c157f90 0x7f238c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.080+0000 7f23919ff700 1 --2- 172.21.15.131:0/3151896035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c06f0e0 0x7f238c06f4b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.080+0000 7f23919ff700 1 --2- 172.21.15.131:0/3151896035 >> v2:172.21.15.131:3300/0 conn(0x7f238c15ba50 0x7f238c157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.080+0000 7f23919ff700 1 -- 172.21.15.131:0/3151896035 >> 172.21.15.131:0/3151896035 conn(0x7f238c0686c0 msgr2=0x7f238c068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.080+0000 7f23919ff700 1 -- 172.21.15.131:0/3151896035 shutdown_connections 2023-12-26T21:55:16.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.080+0000 7f23919ff700 1 -- 172.21.15.131:0/3151896035 wait complete. 2023-12-26T21:55:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.082+0000 7f23919ff700 1 Processor -- start 2023-12-26T21:55:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.082+0000 7f23919ff700 1 -- start start 2023-12-26T21:55:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f23919ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f238c06f0e0 0x7f238c150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f23919ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c157f90 0x7f238c14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f23919ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f238c15ba50 0x7f238c14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f23919ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c14d2f0 con 0x7f238c06f0e0 2023-12-26T21:55:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f23919ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c151100 con 0x7f238c15ba50 2023-12-26T21:55:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f23919ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238c1513e0 con 0x7f238c157f90 2023-12-26T21:55:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f238c06f0e0 0x7f238c150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f238c06f0e0 0x7f238c150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54346/0 (socket says 172.21.15.131:54346) 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238affd700 1 -- 172.21.15.131:0/3784264495 learned_addr learned my addr 172.21.15.131:0/3784264495 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238a7fc700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c157f90 0x7f238c14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238b7fe700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f238c15ba50 0x7f238c14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238affd700 1 -- 172.21.15.131:0/3784264495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c157f90 msgr2=0x7f238c14eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238affd700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c157f90 0x7f238c14eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238affd700 1 -- 172.21.15.131:0/3784264495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f238c15ba50 msgr2=0x7f238c14f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238affd700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f238c15ba50 0x7f238c14f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.083+0000 7f238affd700 1 -- 172.21.15.131:0/3784264495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23800094f0 con 0x7f238c06f0e0 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.084+0000 7f238affd700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f238c06f0e0 0x7f238c150b00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f237c013000 tx=0x7f237c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.084+0000 7f2372ffd700 1 -- 172.21.15.131:0/3784264495 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c01f070 con 0x7f238c06f0e0 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.084+0000 7f2372ffd700 1 -- 172.21.15.131:0/3784264495 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c000bc0 con 0x7f238c06f0e0 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.084+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c009420 con 0x7f238c06f0e0 2023-12-26T21:55:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.084+0000 7f2372ffd700 1 -- 172.21.15.131:0/3784264495 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c011b70 con 0x7f238c06f0e0 2023-12-26T21:55:16.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.084+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c1519a0 con 0x7f238c06f0e0 2023-12-26T21:55:16.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.085+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238c0657e0 con 0x7f238c06f0e0 2023-12-26T21:55:16.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.086+0000 7f2372ffd700 1 -- 172.21.15.131:0/3784264495 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 50) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f237c023070 con 0x7f238c06f0e0 2023-12-26T21:55:16.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.087+0000 7f2372ffd700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 0x7f2378069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.087+0000 7f238a7fc700 1 -- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 msgr2=0x7f2378069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.087+0000 7f2372ffd700 1 -- 172.21.15.131:0/3784264495 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f237c085df0 con 0x7f238c06f0e0 2023-12-26T21:55:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.087+0000 7f238a7fc700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 0x7f2378069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.091+0000 7f2372ffd700 1 -- 172.21.15.131:0/3784264495 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f237c052a50 con 0x7f238c06f0e0 2023-12-26T21:55:16.205 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:16.205+0000 7f483205b1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:55:16.280 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:16.280+0000 7fb077d431c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:55:16.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.288+0000 7f238a7fc700 1 -- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 msgr2=0x7f2378069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:16.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.288+0000 7f238a7fc700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 0x7f2378069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:16.298 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:16.298+0000 7f483205b1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:55:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.340+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f238c15b250 con 0x7f238c06f0e0 2023-12-26T21:55:16.495 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:16.494+0000 7f483205b1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:55:16.559 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:16.558+0000 7fb077d431c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:55:16.654 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:16.653+0000 7fb077d431c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:55:16.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.689+0000 7f238a7fc700 1 -- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 msgr2=0x7f2378069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:16.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.689+0000 7f238a7fc700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 0x7f2378069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:55:16.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.826+0000 7f2372ffd700 1 -- 172.21.15.131:0/3784264495 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f237c056070 con 0x7f238c06f0e0 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.828+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 msgr2=0x7f2378069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.828+0000 7f23919ff700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 0x7f2378069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.828+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f238c06f0e0 msgr2=0x7f238c150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.828+0000 7f23919ff700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f238c06f0e0 0x7f238c150b00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f237c013000 tx=0x7f237c017000 comp rx=0 tx=0).stop 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.829+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 shutdown_connections 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.829+0000 7f23919ff700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f238c15ba50 0x7f238c14f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.829+0000 7f23919ff700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2378067170 0x7f2378069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.829+0000 7f23919ff700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f238c06f0e0 0x7f238c150b00 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.829+0000 7f23919ff700 1 --2- 172.21.15.131:0/3784264495 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f238c157f90 0x7f238c14eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:16.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.829+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 >> 172.21.15.131:0/3784264495 conn(0x7f238c0686c0 msgr2=0x7f238c0cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:16.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.829+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 shutdown_connections 2023-12-26T21:55:16.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:16.829+0000 7f23919ff700 1 -- 172.21.15.131:0/3784264495 wait complete. 2023-12-26T21:55:16.848 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T21:55:16.854 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:16.854+0000 7fb077d431c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:55:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.060+0000 7f60ebdff700 1 Processor -- start 2023-12-26T21:55:17.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.061+0000 7f60ebdff700 1 -- start start 2023-12-26T21:55:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.062+0000 7f60ebdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e4069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.062+0000 7f60ebdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e4069640 con 0x7f60e4142f00 2023-12-26T21:55:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.062+0000 7f60ebdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e4068750 con 0x7f60e40cc590 2023-12-26T21:55:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.062+0000 7f60ebdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4068a30 con 0x7f60e40cb510 2023-12-26T21:55:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.062+0000 7f60e9b9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f60e4142f00 0x7f60e40cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48538/0 (socket says 172.21.15.131:48538) 2023-12-26T21:55:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.062+0000 7f60e9b9b700 1 -- 172.21.15.131:0/2045684012 learned_addr learned my addr 172.21.15.131:0/2045684012 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:17.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.062+0000 7f60e939a700 1 --2- 172.21.15.131:0/2045684012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e4069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e8b99700 1 -- 172.21.15.131:0/2045684012 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894473386 0 0) 0x7f60e4069640 con 0x7f60e4142f00 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e8b99700 1 -- 172.21.15.131:0/2045684012 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60d0003570 con 0x7f60e4142f00 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e8b99700 1 -- 172.21.15.131:0/2045684012 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771477059 0 0) 0x7f60e4068750 con 0x7f60e40cc590 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e8b99700 1 -- 172.21.15.131:0/2045684012 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60e4069640 con 0x7f60e40cc590 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e939a700 1 -- 172.21.15.131:0/2045684012 >> v1:172.21.15.190:6789/0 conn(0x7f60e40cc590 legacy=0x7f60e40c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e939a700 1 -- 172.21.15.131:0/2045684012 >> v1:172.21.15.131:6789/0 conn(0x7f60e4142f00 legacy=0x7f60e40cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e939a700 1 -- 172.21.15.131:0/2045684012 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4068d10 con 0x7f60e40cb510 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e939a700 1 --2- 172.21.15.131:0/2045684012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e4069ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009910 tx=0x7f60d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8f64bd0c0087aa server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e8b99700 1 -- 172.21.15.131:0/2045684012 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4011630 con 0x7f60e40cb510 2023-12-26T21:55:17.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e8b99700 1 -- 172.21.15.131:0/2045684012 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d4011790 con 0x7f60e40cb510 2023-12-26T21:55:17.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.063+0000 7f60e8b99700 1 -- 172.21.15.131:0/2045684012 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4011b30 con 0x7f60e40cb510 2023-12-26T21:55:17.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.064+0000 7f60ebdff700 1 -- 172.21.15.131:0/2045684012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 msgr2=0x7f60e4069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:17.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.064+0000 7f60ebdff700 1 --2- 172.21.15.131:0/2045684012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e4069ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009910 tx=0x7f60d40091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:17.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.064+0000 7f60ebdff700 1 -- 172.21.15.131:0/2045684012 shutdown_connections 2023-12-26T21:55:17.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.064+0000 7f60ebdff700 1 --2- 172.21.15.131:0/2045684012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e4069ad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:17.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.064+0000 7f60ebdff700 1 -- 172.21.15.131:0/2045684012 >> 172.21.15.131:0/2045684012 conn(0x7f60e4063cd0 msgr2=0x7f60e406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:17.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.064+0000 7f60ebdff700 1 -- 172.21.15.131:0/2045684012 shutdown_connections 2023-12-26T21:55:17.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.064+0000 7f60ebdff700 1 -- 172.21.15.131:0/2045684012 wait complete. 2023-12-26T21:55:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.065+0000 7f60ebdff700 1 Processor -- start 2023-12-26T21:55:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.066+0000 7f60ebdff700 1 -- start start 2023-12-26T21:55:17.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.066+0000 7f60ebdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e413ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.066+0000 7f60ebdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e40cc590 0x7f60e40c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ebdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60e4142f00 0x7f60e406ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ebdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4169a60 con 0x7f60e4142f00 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ebdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e40c6d70 con 0x7f60e40cc590 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ebdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e40c7050 con 0x7f60e40cb510 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ea39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60e4142f00 0x7f60e406ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ea39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60e4142f00 0x7f60e406ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54428/0 (socket says 172.21.15.131:54428) 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ea39c700 1 -- 172.21.15.131:0/3141459961 learned_addr learned my addr 172.21.15.131:0/3141459961 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60e939a700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e40cc590 0x7f60e40c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60e9b9b700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e413ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ea39c700 1 -- 172.21.15.131:0/3141459961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 msgr2=0x7f60e413ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ea39c700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e413ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:17.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ea39c700 1 -- 172.21.15.131:0/3141459961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e40cc590 msgr2=0x7f60e40c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ea39c700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e40cc590 0x7f60e40c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ea39c700 1 -- 172.21.15.131:0/3141459961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d4009420 con 0x7f60e4142f00 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60e9b9b700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e413ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.067+0000 7f60ea39c700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60e4142f00 0x7f60e406ce60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00c910 tx=0x7f60dc0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.068+0000 7f60d9ffb700 1 -- 172.21.15.131:0/3141459961 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00a7e0 con 0x7f60e4142f00 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.068+0000 7f60d9ffb700 1 -- 172.21.15.131:0/3141459961 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc00a960 con 0x7f60e4142f00 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.068+0000 7f60ebdff700 1 -- 172.21.15.131:0/3141459961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e40c7330 con 0x7f60e4142f00 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.068+0000 7f60d9ffb700 1 -- 172.21.15.131:0/3141459961 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00ad20 con 0x7f60e4142f00 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.068+0000 7f60ebdff700 1 -- 172.21.15.131:0/3141459961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e413f480 con 0x7f60e4142f00 2023-12-26T21:55:17.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.069+0000 7f60ebdff700 1 -- 172.21.15.131:0/3141459961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e4169bd0 con 0x7f60e4142f00 2023-12-26T21:55:17.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.070+0000 7f60d9ffb700 1 -- 172.21.15.131:0/3141459961 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 51) v1 ==== 84453+0+0 (secure 0 0 0) 0x7f60dc012630 con 0x7f60e4142f00 2023-12-26T21:55:17.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.071+0000 7f60d9ffb700 1 -- 172.21.15.131:0/3141459961 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f60dc084a00 con 0x7f60e4142f00 2023-12-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.076+0000 7f60d9ffb700 1 -- 172.21.15.131:0/3141459961 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60dc051bb0 con 0x7f60e4142f00 2023-12-26T21:55:17.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.272+0000 7f60ebdff700 1 -- 172.21.15.131:0/3141459961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60e40717f0 con 0x7f60e4142f00 2023-12-26T21:55:17.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.868+0000 7f60d9ffb700 1 -- 172.21.15.131:0/3141459961 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 181+0+0 (secure 0 0 0) 0x7f60dc082e60 con 0x7f60e4142f00 2023-12-26T21:55:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.874+0000 7f60ebdff700 1 -- 172.21.15.131:0/3141459961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60e4142f00 msgr2=0x7f60e406ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.874+0000 7f60ebdff700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60e4142f00 0x7f60e406ce60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00c910 tx=0x7f60dc0027a0 comp rx=0 tx=0).stop 2023-12-26T21:55:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.875+0000 7f60ebdff700 1 -- 172.21.15.131:0/3141459961 shutdown_connections 2023-12-26T21:55:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.875+0000 7f60ebdff700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60e4142f00 0x7f60e406ce60 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.875+0000 7f60ebdff700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60e40cb510 0x7f60e413ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.875+0000 7f60ebdff700 1 --2- 172.21.15.131:0/3141459961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60e40cc590 0x7f60e40c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.875+0000 7f60ebdff700 1 -- 172.21.15.131:0/3141459961 >> 172.21.15.131:0/3141459961 conn(0x7f60e4063cd0 msgr2=0x7f60e413fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:17.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.876+0000 7f60ebdff700 1 -- 172.21.15.131:0/3141459961 shutdown_connections 2023-12-26T21:55:17.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:17.876+0000 7f60ebdff700 1 -- 172.21.15.131:0/3141459961 wait complete. 2023-12-26T21:55:17.880 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T21:55:17.904 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T21:55:18.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.138+0000 7f26b3dff700 1 Processor -- start 2023-12-26T21:55:18.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.138+0000 7f26b3dff700 1 -- start start 2023-12-26T21:55:18.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.139+0000 7f26b3dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f26ac0c8d40 0x7f26ac15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:18.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.139+0000 7f26b3dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f26ac156e50 0x7f26ac1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:18.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.139+0000 7f26b3dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac0af040 con 0x7f26ac0c9110 2023-12-26T21:55:18.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.139+0000 7f26b3dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ac0ae0e0 con 0x7f26ac0c8d40 2023-12-26T21:55:18.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.139+0000 7f26b3dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ac0ae3c0 con 0x7f26ac156e50 2023-12-26T21:55:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.139+0000 7f26b1b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f26ac0c8d40 0x7f26ac15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b1b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f26ac0c8d40 0x7f26ac15bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54440/0 (socket says 172.21.15.131:54440) 2023-12-26T21:55:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b1b9b700 1 -- 172.21.15.131:0/4173477190 learned_addr learned my addr 172.21.15.131:0/4173477190 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b239c700 1 --2- 172.21.15.131:0/4173477190 >> v2:172.21.15.190:3300/0 conn(0x7f26ac156e50 0x7f26ac1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b1b9b700 1 -- 172.21.15.131:0/4173477190 >> v2:172.21.15.190:3300/0 conn(0x7f26ac156e50 msgr2=0x7f26ac1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b1b9b700 1 --2- 172.21.15.131:0/4173477190 >> v2:172.21.15.190:3300/0 conn(0x7f26ac156e50 0x7f26ac1572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b1b9b700 1 -- 172.21.15.131:0/4173477190 >> v1:172.21.15.190:6789/0 conn(0x7f26ac0c9110 legacy=0x7f26ac0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:18.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b1b9b700 1 -- 172.21.15.131:0/4173477190 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac0ae6a0 con 0x7f26ac0c8d40 2023-12-26T21:55:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b1b9b700 1 --2- 172.21.15.131:0/4173477190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 0x7f26ac15bc30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0ac5c0 tx=0x7f26a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa42253cb030cd79 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b0b99700 1 -- 172.21.15.131:0/4173477190 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800b860 con 0x7f26ac0c8d40 2023-12-26T21:55:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b0b99700 1 -- 172.21.15.131:0/4173477190 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a800b9c0 con 0x7f26ac0c8d40 2023-12-26T21:55:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.140+0000 7f26b0b99700 1 -- 172.21.15.131:0/4173477190 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800bd60 con 0x7f26ac0c8d40 2023-12-26T21:55:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.141+0000 7f26b3dff700 1 -- 172.21.15.131:0/4173477190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 msgr2=0x7f26ac15bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.141+0000 7f26b3dff700 1 --2- 172.21.15.131:0/4173477190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 0x7f26ac15bc30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0ac5c0 tx=0x7f26a8009250 comp rx=0 tx=0).stop 2023-12-26T21:55:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.141+0000 7f26b3dff700 1 -- 172.21.15.131:0/4173477190 shutdown_connections 2023-12-26T21:55:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.141+0000 7f26b3dff700 1 --2- 172.21.15.131:0/4173477190 >> v2:172.21.15.190:3300/0 conn(0x7f26ac156e50 0x7f26ac1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.141+0000 7f26b3dff700 1 --2- 172.21.15.131:0/4173477190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 0x7f26ac15bc30 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.141+0000 7f26b3dff700 1 -- 172.21.15.131:0/4173477190 >> 172.21.15.131:0/4173477190 conn(0x7f26ac0675b0 msgr2=0x7f26ac0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.141+0000 7f26b3dff700 1 -- 172.21.15.131:0/4173477190 shutdown_connections 2023-12-26T21:55:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.141+0000 7f26b3dff700 1 -- 172.21.15.131:0/4173477190 wait complete. 2023-12-26T21:55:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.142+0000 7f26b3dff700 1 Processor -- start 2023-12-26T21:55:18.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.143+0000 7f26b3dff700 1 -- start start 2023-12-26T21:55:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.143+0000 7f26b3dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 0x7f26ac14e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.143+0000 7f26b3dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26ac0c9110 0x7f26ac0c17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.143+0000 7f26b3dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26ac156e50 0x7f26ac06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b3dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0bf800 con 0x7f26ac0c8d40 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b3dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0c0660 con 0x7f26ac0c9110 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b3dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac15a500 con 0x7f26ac156e50 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b1b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 0x7f26ac14e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b1b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 0x7f26ac14e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54452/0 (socket says 172.21.15.131:54452) 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b1b9b700 1 -- 172.21.15.131:0/1129138360 learned_addr learned my addr 172.21.15.131:0/1129138360 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b239c700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26ac156e50 0x7f26ac06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b139a700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26ac0c9110 0x7f26ac0c17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b1b9b700 1 -- 172.21.15.131:0/1129138360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26ac156e50 msgr2=0x7f26ac06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b1b9b700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26ac156e50 0x7f26ac06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b1b9b700 1 -- 172.21.15.131:0/1129138360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26ac0c9110 msgr2=0x7f26ac0c17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b1b9b700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26ac0c9110 0x7f26ac0c17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b1b9b700 1 -- 172.21.15.131:0/1129138360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80094f0 con 0x7f26ac0c8d40 2023-12-26T21:55:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.144+0000 7f26b1b9b700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 0x7f26ac14e710 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f26a8006fd0 tx=0x7f26a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.145+0000 7f26a1ffb700 1 -- 172.21.15.131:0/1129138360 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8011e20 con 0x7f26ac0c8d40 2023-12-26T21:55:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.145+0000 7f26a1ffb700 1 -- 172.21.15.131:0/1129138360 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a801a630 con 0x7f26ac0c8d40 2023-12-26T21:55:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.145+0000 7f26a1ffb700 1 -- 172.21.15.131:0/1129138360 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a801a9f0 con 0x7f26ac0c8d40 2023-12-26T21:55:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.145+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac14ec50 con 0x7f26ac0c8d40 2023-12-26T21:55:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.145+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac0bfd30 con 0x7f26ac0c8d40 2023-12-26T21:55:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.146+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac0717f0 con 0x7f26ac0c8d40 2023-12-26T21:55:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.146+0000 7f26a1ffb700 1 -- 172.21.15.131:0/1129138360 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 52) v1 ==== 84580+0+0 (secure 0 0 0) 0x7f26a802d440 con 0x7f26ac0c8d40 2023-12-26T21:55:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.147+0000 7f26a1ffb700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2690066d80 0x7f2690069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.147+0000 7f26a1ffb700 1 -- 172.21.15.131:0/1129138360 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6347+0+0 (secure 0 0 0) 0x7f26a8086160 con 0x7f26ac0c8d40 2023-12-26T21:55:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.148+0000 7f26b139a700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2690066d80 0x7f2690069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.148+0000 7f26b139a700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2690066d80 0x7f2690069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f269c003860 tx=0x7f269c009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.151+0000 7f26a1ffb700 1 -- 172.21.15.131:0/1129138360 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26a8053330 con 0x7f26ac0c8d40 2023-12-26T21:55:18.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.350+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f26ac0ae980 con 0x7f26ac0c8d40 2023-12-26T21:55:18.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.887+0000 7f26a1ffb700 1 -- 172.21.15.131:0/1129138360 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f26a8053700 con 0x7f26ac0c8d40 2023-12-26T21:55:18.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.888+0000 7f26a1ffb700 1 -- 172.21.15.131:0/1129138360 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 169+0+0 (secure 0 0 0) 0x7f26a804f010 con 0x7f26ac0c8d40 2023-12-26T21:55:18.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.899+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2690066d80 msgr2=0x7f2690069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.899+0000 7f26b3dff700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2690066d80 0x7f2690069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f269c003860 tx=0x7f269c009040 comp rx=0 tx=0).stop 2023-12-26T21:55:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.899+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 msgr2=0x7f26ac14e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.899+0000 7f26b3dff700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 0x7f26ac14e710 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f26a8006fd0 tx=0x7f26a8017000 comp rx=0 tx=0).stop 2023-12-26T21:55:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.900+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 shutdown_connections 2023-12-26T21:55:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.900+0000 7f26b3dff700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26ac156e50 0x7f26ac06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.900+0000 7f26b3dff700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2690066d80 0x7f2690069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.900+0000 7f26b3dff700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26ac0c8d40 0x7f26ac14e710 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.900+0000 7f26b3dff700 1 --2- 172.21.15.131:0/1129138360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26ac0c9110 0x7f26ac0c17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:18.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.900+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 >> 172.21.15.131:0/1129138360 conn(0x7f26ac0675b0 msgr2=0x7f26ac067980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.900+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 shutdown_connections 2023-12-26T21:55:18.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:18.900+0000 7f26b3dff700 1 -- 172.21.15.131:0/1129138360 wait complete. 2023-12-26T21:55:18.905 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T21:55:18.924 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T21:55:18.981 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:55:18.997 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:55:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.119+0000 7f3d53dff700 1 Processor -- start 2023-12-26T21:55:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.119+0000 7f3d53dff700 1 -- start start 2023-12-26T21:55:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.120+0000 7f3d53dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c159cf0 0x7f3d4c0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.120+0000 7f3d53dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c0c6810 con 0x7f3d4c0ced90 2023-12-26T21:55:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.120+0000 7f3d53dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c0c5c00 con 0x7f3d4c159920 2023-12-26T21:55:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.120+0000 7f3d53dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0c5ee0 con 0x7f3d4c159cf0 2023-12-26T21:55:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d5239c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3d4c0ced90 0x7f3d4c15b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48546/0 (socket says 172.21.15.131:48546) 2023-12-26T21:55:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d5239c700 1 -- 172.21.15.131:0/1872500364 learned_addr learned my addr 172.21.15.131:0/1872500364 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d5139a700 1 --2- 172.21.15.131:0/1872500364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c159cf0 0x7f3d4c0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d50b99700 1 -- 172.21.15.131:0/1872500364 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050365099 0 0) 0x7f3d4c0c5c00 con 0x7f3d4c159920 2023-12-26T21:55:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d50b99700 1 -- 172.21.15.131:0/1872500364 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d38003570 con 0x7f3d4c159920 2023-12-26T21:55:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d50b99700 1 -- 172.21.15.131:0/1872500364 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945486492 0 0) 0x7f3d4c0c6810 con 0x7f3d4c0ced90 2023-12-26T21:55:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d50b99700 1 -- 172.21.15.131:0/1872500364 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d4c0c5c00 con 0x7f3d4c0ced90 2023-12-26T21:55:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d5139a700 1 -- 172.21.15.131:0/1872500364 >> v1:172.21.15.190:6789/0 conn(0x7f3d4c159920 legacy=0x7f3d4c0c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d5139a700 1 -- 172.21.15.131:0/1872500364 >> v1:172.21.15.131:6789/0 conn(0x7f3d4c0ced90 legacy=0x7f3d4c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d5139a700 1 -- 172.21.15.131:0/1872500364 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0c61c0 con 0x7f3d4c159cf0 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.121+0000 7f3d5139a700 1 --2- 172.21.15.131:0/1872500364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c159cf0 0x7f3d4c0c6ca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009910 tx=0x7f3d3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c95640fd5780ff server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d50b99700 1 -- 172.21.15.131:0/1872500364 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c011630 con 0x7f3d4c159cf0 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d50b99700 1 -- 172.21.15.131:0/1872500364 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c011790 con 0x7f3d4c159cf0 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d50b99700 1 -- 172.21.15.131:0/1872500364 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c011b30 con 0x7f3d4c159cf0 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d53dff700 1 -- 172.21.15.131:0/1872500364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c159cf0 msgr2=0x7f3d4c0c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d53dff700 1 --2- 172.21.15.131:0/1872500364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c159cf0 0x7f3d4c0c6ca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009910 tx=0x7f3d3c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d53dff700 1 -- 172.21.15.131:0/1872500364 shutdown_connections 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d53dff700 1 --2- 172.21.15.131:0/1872500364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c159cf0 0x7f3d4c0c6ca0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d53dff700 1 -- 172.21.15.131:0/1872500364 >> 172.21.15.131:0/1872500364 conn(0x7f3d4c099980 msgr2=0x7f3d4c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d53dff700 1 -- 172.21.15.131:0/1872500364 shutdown_connections 2023-12-26T21:55:19.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.122+0000 7f3d53dff700 1 -- 172.21.15.131:0/1872500364 wait complete. 2023-12-26T21:55:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.124+0000 7f3d53dff700 1 Processor -- start 2023-12-26T21:55:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.124+0000 7f3d53dff700 1 -- start start 2023-12-26T21:55:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d53dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c0ced90 0x7f3d4c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d53dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d4c159920 0x7f3d4c0c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d53dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d4c159cf0 0x7f3d4c0c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d53dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0c9760 con 0x7f3d4c159920 2023-12-26T21:55:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d53dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0ca580 con 0x7f3d4c159cf0 2023-12-26T21:55:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d53dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0c7c70 con 0x7f3d4c0ced90 2023-12-26T21:55:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d5139a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d4c159920 0x7f3d4c0c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d5139a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d4c159920 0x7f3d4c0c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54468/0 (socket says 172.21.15.131:54468) 2023-12-26T21:55:19.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d5139a700 1 -- 172.21.15.131:0/2779882742 learned_addr learned my addr 172.21.15.131:0/2779882742 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d51b9b700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c0ced90 0x7f3d4c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d5239c700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d4c159cf0 0x7f3d4c0c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d5139a700 1 -- 172.21.15.131:0/2779882742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c0ced90 msgr2=0x7f3d4c1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d5139a700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c0ced90 0x7f3d4c1546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d5139a700 1 -- 172.21.15.131:0/2779882742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d4c159cf0 msgr2=0x7f3d4c0c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.125+0000 7f3d5139a700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d4c159cf0 0x7f3d4c0c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.126+0000 7f3d5139a700 1 -- 172.21.15.131:0/2779882742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c009420 con 0x7f3d4c159920 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.126+0000 7f3d5239c700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d4c159cf0 0x7f3d4c0c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.126+0000 7f3d5139a700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d4c159920 0x7f3d4c0c7730 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009fd0 tx=0x7f3d3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.126+0000 7f3d41ffb700 1 -- 172.21.15.131:0/2779882742 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c01f070 con 0x7f3d4c159920 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.126+0000 7f3d41ffb700 1 -- 172.21.15.131:0/2779882742 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c000bc0 con 0x7f3d4c159920 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.126+0000 7f3d41ffb700 1 -- 172.21.15.131:0/2779882742 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c011b30 con 0x7f3d4c159920 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.126+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c0c9970 con 0x7f3d4c159920 2023-12-26T21:55:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.126+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c0c9f30 con 0x7f3d4c159920 2023-12-26T21:55:19.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.127+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c0b34d0 con 0x7f3d4c159920 2023-12-26T21:55:19.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.128+0000 7f3d41ffb700 1 -- 172.21.15.131:0/2779882742 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3d3c023070 con 0x7f3d4c159920 2023-12-26T21:55:19.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.129+0000 7f3d41ffb700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3d30066f30 0x7f3d300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.130+0000 7f3d41ffb700 1 -- 172.21.15.131:0/2779882742 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5933+0+0 (secure 0 0 0) 0x7f3d3c0859b0 con 0x7f3d4c159920 2023-12-26T21:55:19.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.130+0000 7f3d51b9b700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3d30066f30 0x7f3d300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.130+0000 7f3d51b9b700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3d30066f30 0x7f3d300693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d48007fd0 tx=0x7f3d48007d30 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:19.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.134+0000 7f3d41ffb700 1 -- 172.21.15.131:0/2779882742 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d4c0b34d0 con 0x7f3d4c159920 2023-12-26T21:55:19.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.329+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3d4c0b34d0 con 0x7f3d4c159920 2023-12-26T21:55:19.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.335+0000 7f3d41ffb700 1 -- 172.21.15.131:0/2779882742 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3d4c0b34d0 con 0x7f3d4c159920 2023-12-26T21:55:19.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3d30066f30 msgr2=0x7f3d300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3d30066f30 0x7f3d300693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d48007fd0 tx=0x7f3d48007d30 comp rx=0 tx=0).stop 2023-12-26T21:55:19.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d4c159920 msgr2=0x7f3d4c0c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d4c159920 0x7f3d4c0c7730 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009fd0 tx=0x7f3d3c017000 comp rx=0 tx=0).stop 2023-12-26T21:55:19.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 shutdown_connections 2023-12-26T21:55:19.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d4c159cf0 0x7f3d4c0c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d4c0ced90 0x7f3d4c1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3d30066f30 0x7f3d300693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 --2- 172.21.15.131:0/2779882742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d4c159920 0x7f3d4c0c7730 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.337+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 >> 172.21.15.131:0/2779882742 conn(0x7f3d4c099980 msgr2=0x7f3d4c154f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:19.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.338+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 shutdown_connections 2023-12-26T21:55:19.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.338+0000 7f3d53dff700 1 -- 172.21.15.131:0/2779882742 wait complete. 2023-12-26T21:55:19.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T21:55:19.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.590+0000 7fa5a319d700 1 Processor -- start 2023-12-26T21:55:19.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.590+0000 7fa5a319d700 1 -- start start 2023-12-26T21:55:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.591+0000 7fa5a319d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa59c0cb980 0x7fa59c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.591+0000 7fa5a319d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c0c63f0 0x7fa59c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.591+0000 7fa5a319d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59c0cca40 con 0x7fa59c159490 2023-12-26T21:55:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.591+0000 7fa5a319d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c0cbe30 con 0x7fa59c0cb980 2023-12-26T21:55:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.591+0000 7fa5a319d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c0cc110 con 0x7fa59c0c63f0 2023-12-26T21:55:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a299c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c0c63f0 0x7fa59c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a299c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c0c63f0 0x7fa59c0ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:37012/0 (socket says 172.21.15.131:37012) 2023-12-26T21:55:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a219b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa59c159490 0x7fa59c0cb270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:41714/0 (socket says 172.21.15.131:41714) 2023-12-26T21:55:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a199a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa59c0cb980 0x7fa59c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a299c700 1 -- 172.21.15.131:0/391353218 learned_addr learned my addr 172.21.15.131:0/391353218 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a199a700 1 -- 172.21.15.131:0/391353218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c0c63f0 msgr2=0x7fa59c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a1199700 1 -- 172.21.15.131:0/391353218 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968144802 0 0) 0x7fa59c0cca40 con 0x7fa59c159490 2023-12-26T21:55:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a199a700 1 --2- 172.21.15.131:0/391353218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c0c63f0 0x7fa59c0ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a199a700 1 -- 172.21.15.131:0/391353218 >> v1:172.21.15.190:6789/0 conn(0x7fa59c159490 legacy=0x7fa59c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a199a700 1 -- 172.21.15.131:0/391353218 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c0cc3f0 con 0x7fa59c0cb980 2023-12-26T21:55:19.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.592+0000 7fa5a299c700 1 --2- 172.21.15.131:0/391353218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c0c63f0 0x7fa59c0ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:19.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.593+0000 7fa5a199a700 1 --2- 172.21.15.131:0/391353218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0cb980 0x7fa59c0ce970 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0098e0 tx=0x7fa58c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e233b015d74c14 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:19.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.593+0000 7fa5a1199700 1 -- 172.21.15.131:0/391353218 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00b640 con 0x7fa59c0cb980 2023-12-26T21:55:19.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.593+0000 7fa5a1199700 1 -- 172.21.15.131:0/391353218 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00b7a0 con 0x7fa59c0cb980 2023-12-26T21:55:19.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.593+0000 7fa5a1199700 1 -- 172.21.15.131:0/391353218 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00bb40 con 0x7fa59c0cb980 2023-12-26T21:55:19.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.593+0000 7fa5a319d700 1 -- 172.21.15.131:0/391353218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0cb980 msgr2=0x7fa59c0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.593+0000 7fa5a319d700 1 --2- 172.21.15.131:0/391353218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0cb980 0x7fa59c0ce970 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0098e0 tx=0x7fa58c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.594+0000 7fa5a319d700 1 -- 172.21.15.131:0/391353218 shutdown_connections 2023-12-26T21:55:19.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.594+0000 7fa5a319d700 1 --2- 172.21.15.131:0/391353218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c0c63f0 0x7fa59c0ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.594+0000 7fa5a319d700 1 --2- 172.21.15.131:0/391353218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0cb980 0x7fa59c0ce970 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.594+0000 7fa5a319d700 1 -- 172.21.15.131:0/391353218 >> 172.21.15.131:0/391353218 conn(0x7fa59c099750 msgr2=0x7fa59c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:19.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.594+0000 7fa5a319d700 1 -- 172.21.15.131:0/391353218 shutdown_connections 2023-12-26T21:55:19.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.594+0000 7fa5a319d700 1 -- 172.21.15.131:0/391353218 wait complete. 2023-12-26T21:55:19.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.596+0000 7fa5a319d700 1 Processor -- start 2023-12-26T21:55:19.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.596+0000 7fa5a319d700 1 -- start start 2023-12-26T21:55:19.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a319d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0c63f0 0x7fa59c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a319d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa59c0cb980 0x7fa59c15da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a319d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c159490 0x7fa59c148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a319d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c169530 con 0x7fa59c0cb980 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a319d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c15d730 con 0x7fa59c0c63f0 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a319d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c0c9260 con 0x7fa59c159490 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a199a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa59c0cb980 0x7fa59c15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a219b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0c63f0 0x7fa59c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a199a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa59c0cb980 0x7fa59c15da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54470/0 (socket says 172.21.15.131:54470) 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a199a700 1 -- 172.21.15.131:0/2291997558 learned_addr learned my addr 172.21.15.131:0/2291997558 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:19.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.597+0000 7fa5a299c700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c159490 0x7fa59c148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5a199a700 1 -- 172.21.15.131:0/2291997558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c159490 msgr2=0x7fa59c148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5a199a700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c159490 0x7fa59c148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5a199a700 1 -- 172.21.15.131:0/2291997558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0c63f0 msgr2=0x7fa59c0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5a199a700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0c63f0 0x7fa59c0c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5a199a700 1 -- 172.21.15.131:0/2291997558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594009ae0 con 0x7fa59c0cb980 2023-12-26T21:55:19.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5a219b700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0c63f0 0x7fa59c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:19.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5a199a700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa59c0cb980 0x7fa59c15da80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa59c088e10 tx=0x7fa58c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:19.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5927fc700 1 -- 172.21.15.131:0/2291997558 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00bb20 con 0x7fa59c0cb980 2023-12-26T21:55:19.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5927fc700 1 -- 172.21.15.131:0/2291997558 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c000c90 con 0x7fa59c0cb980 2023-12-26T21:55:19.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5927fc700 1 -- 172.21.15.131:0/2291997558 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c011630 con 0x7fa59c0cb980 2023-12-26T21:55:19.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58c009420 con 0x7fa59c0cb980 2023-12-26T21:55:19.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.598+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c0c8650 con 0x7fa59c0cb980 2023-12-26T21:55:19.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.599+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c0c9410 con 0x7fa59c0cb980 2023-12-26T21:55:19.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.600+0000 7fa5927fc700 1 -- 172.21.15.131:0/2291997558 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa58c0119a0 con 0x7fa59c0cb980 2023-12-26T21:55:19.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.601+0000 7fa5927fc700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:19.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.601+0000 7fa5927fc700 1 -- 172.21.15.131:0/2291997558 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5933+0+0 (secure 0 0 0) 0x7fa58c085a90 con 0x7fa59c0cb980 2023-12-26T21:55:19.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.601+0000 7fa5a219b700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:19.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.602+0000 7fa5a219b700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa598007fd0 tx=0x7fa598007f40 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:19.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.605+0000 7fa5927fc700 1 -- 172.21.15.131:0/2291997558 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa59c0c9410 con 0x7fa59c0cb980 2023-12-26T21:55:19.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.798+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fa59c0c93d0 con 0x7fa59c0cb980 2023-12-26T21:55:19.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.805+0000 7fa5927fc700 1 -- 172.21.15.131:0/2291997558 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v38) v1 ==== 115+0+0 (secure 0 0 0) 0x7fa58c052b80 con 0x7fa59c0cb980 2023-12-26T21:55:19.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa580066f30 msgr2=0x7fa5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa598007fd0 tx=0x7fa598007f40 comp rx=0 tx=0).stop 2023-12-26T21:55:19.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa59c0cb980 msgr2=0x7fa59c15da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:19.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa59c0cb980 0x7fa59c15da80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa59c088e10 tx=0x7fa58c017000 comp rx=0 tx=0).stop 2023-12-26T21:55:19.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 shutdown_connections 2023-12-26T21:55:19.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa59c159490 0x7fa59c148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa59c0c63f0 0x7fa59c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 --2- 172.21.15.131:0/2291997558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa59c0cb980 0x7fa59c15da80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:19.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 >> 172.21.15.131:0/2291997558 conn(0x7fa59c099750 msgr2=0x7fa59c158200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:19.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 shutdown_connections 2023-12-26T21:55:19.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:19.808+0000 7fa5a319d700 1 -- 172.21.15.131:0/2291997558 wait complete. 2023-12-26T21:55:19.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T21:55:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.018+0000 7f49df1ff700 1 Processor -- start 2023-12-26T21:55:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.018+0000 7f49df1ff700 1 -- start start 2023-12-26T21:55:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.019+0000 7f49df1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49d80cd100 0x7f49d80cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.019+0000 7f49df1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d815db90 con 0x7f49d80cd4d0 2023-12-26T21:55:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.019+0000 7f49df1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d80cbb70 con 0x7f49d8159900 2023-12-26T21:55:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.019+0000 7f49df1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d80cbe50 con 0x7f49d80cd100 2023-12-26T21:55:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.019+0000 7f49dcf9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49d80cd100 0x7f49d80cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49dcf9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49d80cd100 0x7f49d80cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54476/0 (socket says 172.21.15.131:54476) 2023-12-26T21:55:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49dcf9b700 1 -- 172.21.15.131:0/989047666 learned_addr learned my addr 172.21.15.131:0/989047666 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.019+0000 7f49d7fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f49d80cd4d0 0x7f49d81591f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48554/0 (socket says 172.21.15.131:48554) 2023-12-26T21:55:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49dcf9b700 1 -- 172.21.15.131:0/989047666 >> v1:172.21.15.190:6789/0 conn(0x7f49d8159900 legacy=0x7f49d80ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49dcf9b700 1 -- 172.21.15.131:0/989047666 >> v1:172.21.15.131:6789/0 conn(0x7f49d80cd4d0 legacy=0x7f49d81591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49dcf9b700 1 -- 172.21.15.131:0/989047666 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d815d540 con 0x7f49d80cd100 2023-12-26T21:55:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49d77fe700 1 -- 172.21.15.131:0/989047666 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467990720 0 0) 0x7f49d815db90 con 0x7f49d80cd4d0 2023-12-26T21:55:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49dcf9b700 1 --2- 172.21.15.131:0/989047666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd100 0x7f49d80cb590 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0099b0 tx=0x7f49cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5c9e4ab7c7986e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49d77fe700 1 -- 172.21.15.131:0/989047666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00b860 con 0x7f49d80cd100 2023-12-26T21:55:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49d77fe700 1 -- 172.21.15.131:0/989047666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc00b9c0 con 0x7f49d80cd100 2023-12-26T21:55:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.020+0000 7f49d77fe700 1 -- 172.21.15.131:0/989047666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00bd60 con 0x7f49d80cd100 2023-12-26T21:55:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.021+0000 7f49df1ff700 1 -- 172.21.15.131:0/989047666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd100 msgr2=0x7f49d80cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.021+0000 7f49df1ff700 1 --2- 172.21.15.131:0/989047666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd100 0x7f49d80cb590 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0099b0 tx=0x7f49cc009250 comp rx=0 tx=0).stop 2023-12-26T21:55:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.021+0000 7f49df1ff700 1 -- 172.21.15.131:0/989047666 shutdown_connections 2023-12-26T21:55:20.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.021+0000 7f49df1ff700 1 --2- 172.21.15.131:0/989047666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd100 0x7f49d80cb590 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.021+0000 7f49df1ff700 1 -- 172.21.15.131:0/989047666 >> 172.21.15.131:0/989047666 conn(0x7f49d806ece0 msgr2=0x7f49d806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:20.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.021+0000 7f49df1ff700 1 -- 172.21.15.131:0/989047666 shutdown_connections 2023-12-26T21:55:20.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.021+0000 7f49df1ff700 1 -- 172.21.15.131:0/989047666 wait complete. 2023-12-26T21:55:20.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.023+0000 7f49df1ff700 1 Processor -- start 2023-12-26T21:55:20.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.024+0000 7f49df1ff700 1 -- start start 2023-12-26T21:55:20.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49df1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d80cd100 0x7f49d80b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49df1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd4d0 0x7f49d80b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49df1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d8159900 0x7f49d816a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49df1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80b1760 con 0x7f49d80cd4d0 2023-12-26T21:55:20.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49df1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80b2880 con 0x7f49d80cd100 2023-12-26T21:55:20.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49df1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80b0ec0 con 0x7f49d8159900 2023-12-26T21:55:20.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd4d0 0x7f49d80b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:20.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd4d0 0x7f49d80b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54478/0 (socket says 172.21.15.131:54478) 2023-12-26T21:55:20.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49d7fff700 1 -- 172.21.15.131:0/1767303808 learned_addr learned my addr 172.21.15.131:0/1767303808 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:20.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49dcf9b700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d80cd100 0x7f49d80b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49dd79c700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d8159900 0x7f49d816a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49d7fff700 1 -- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d8159900 msgr2=0x7f49d816a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49d7fff700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d8159900 0x7f49d816a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49d7fff700 1 -- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d80cd100 msgr2=0x7f49d80b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49d7fff700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d80cd100 0x7f49d80b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49d7fff700 1 -- 172.21.15.131:0/1767303808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0094f0 con 0x7f49d80cd4d0 2023-12-26T21:55:20.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49dd79c700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d8159900 0x7f49d816a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:20.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.025+0000 7f49dcf9b700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d80cd100 0x7f49d80b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:20.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.026+0000 7f49d7fff700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd4d0 0x7f49d80b2340 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c810 tx=0x7f49c4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:20.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.026+0000 7f49d4ff9700 1 -- 172.21.15.131:0/1767303808 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c40073d0 con 0x7f49d80cd4d0 2023-12-26T21:55:20.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.026+0000 7f49d4ff9700 1 -- 172.21.15.131:0/1767303808 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4007550 con 0x7f49d80cd4d0 2023-12-26T21:55:20.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.026+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d80b11a0 con 0x7f49d80cd4d0 2023-12-26T21:55:20.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.026+0000 7f49d4ff9700 1 -- 172.21.15.131:0/1767303808 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4004a80 con 0x7f49d80cd4d0 2023-12-26T21:55:20.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.026+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d816a320 con 0x7f49d80cd4d0 2023-12-26T21:55:20.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.027+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d80657e0 con 0x7f49d80cd4d0 2023-12-26T21:55:20.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.028+0000 7f49d4ff9700 1 -- 172.21.15.131:0/1767303808 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f49c401c3b0 con 0x7f49d80cd4d0 2023-12-26T21:55:20.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.029+0000 7f49d4ff9700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f49bc066f30 0x7f49bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.029+0000 7f49d4ff9700 1 -- 172.21.15.131:0/1767303808 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5933+0+0 (secure 0 0 0) 0x7f49c4083af0 con 0x7f49d80cd4d0 2023-12-26T21:55:20.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.029+0000 7f49dcf9b700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f49bc066f30 0x7f49bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:20.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.030+0000 7f49dcf9b700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f49bc066f30 0x7f49bc0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0099b0 tx=0x7f49cc017000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:20.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.033+0000 7f49d4ff9700 1 -- 172.21.15.131:0/1767303808 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49c4050d00 con 0x7f49d80cd4d0 2023-12-26T21:55:20.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.228+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f49d816a540 con 0x7f49d80cd4d0 2023-12-26T21:55:20.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.233+0000 7f49d4ff9700 1 -- 172.21.15.131:0/1767303808 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v39) v1 ==== 120+0+0 (secure 0 0 0) 0x7f49c40180b0 con 0x7f49d80cd4d0 2023-12-26T21:55:20.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f49bc066f30 msgr2=0x7f49bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f49bc066f30 0x7f49bc0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0099b0 tx=0x7f49cc017000 comp rx=0 tx=0).stop 2023-12-26T21:55:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd4d0 msgr2=0x7f49d80b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd4d0 0x7f49d80b2340 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c810 tx=0x7f49c4007d90 comp rx=0 tx=0).stop 2023-12-26T21:55:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 shutdown_connections 2023-12-26T21:55:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d8159900 0x7f49d816a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d80cd100 0x7f49d80b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f49bc066f30 0x7f49bc0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 --2- 172.21.15.131:0/1767303808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d80cd4d0 0x7f49d80b2340 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 >> 172.21.15.131:0/1767303808 conn(0x7f49d806ece0 msgr2=0x7f49d815f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 shutdown_connections 2023-12-26T21:55:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.236+0000 7f49df1ff700 1 -- 172.21.15.131:0/1767303808 wait complete. 2023-12-26T21:55:20.255 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-12-26T21:55:20.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.446+0000 7fe179bff700 1 Processor -- start 2023-12-26T21:55:20.446 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.446+0000 7fe179bff700 1 -- start start 2023-12-26T21:55:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.447+0000 7fe179bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe17406bfd0 0x7fe1740c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.447+0000 7fe179bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe1740cc100 0x7fe1740cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.447+0000 7fe179bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1740686c0 con 0x7fe17413c510 2023-12-26T21:55:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.447+0000 7fe179bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe174069520 con 0x7fe1740cc100 2023-12-26T21:55:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.447+0000 7fe179bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe174068910 con 0x7fe17406bfd0 2023-12-26T21:55:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.447+0000 7fe173fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe1740cc100 0x7fe1740cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.447+0000 7fe173fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe1740cc100 0x7fe1740cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54484/0 (socket says 172.21.15.131:54484) 2023-12-26T21:55:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.447+0000 7fe173fff700 1 -- 172.21.15.131:0/2764722847 learned_addr learned my addr 172.21.15.131:0/2764722847 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe172ffd700 1 --2- 172.21.15.131:0/2764722847 >> v2:172.21.15.190:3300/0 conn(0x7fe17406bfd0 0x7fe1740c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe173fff700 1 -- 172.21.15.131:0/2764722847 >> v2:172.21.15.190:3300/0 conn(0x7fe17406bfd0 msgr2=0x7fe1740c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe173fff700 1 --2- 172.21.15.131:0/2764722847 >> v2:172.21.15.190:3300/0 conn(0x7fe17406bfd0 0x7fe1740c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe173fff700 1 -- 172.21.15.131:0/2764722847 >> v1:172.21.15.190:6789/0 conn(0x7fe17413c510 legacy=0x7fe17406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe173fff700 1 -- 172.21.15.131:0/2764722847 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174068bf0 con 0x7fe1740cc100 2023-12-26T21:55:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe172ffd700 1 --2- 172.21.15.131:0/2764722847 >> v2:172.21.15.190:3300/0 conn(0x7fe17406bfd0 0x7fe1740c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe173fff700 1 --2- 172.21.15.131:0/2764722847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 0x7fe1740cc520 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe1740b43e0 tx=0x7fe16c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62d9c2ee01eaf2e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe1727fc700 1 -- 172.21.15.131:0/2764722847 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00e070 con 0x7fe1740cc100 2023-12-26T21:55:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe1727fc700 1 -- 172.21.15.131:0/2764722847 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c00ba10 con 0x7fe1740cc100 2023-12-26T21:55:20.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.448+0000 7fe1727fc700 1 -- 172.21.15.131:0/2764722847 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0092e0 con 0x7fe1740cc100 2023-12-26T21:55:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.449+0000 7fe179bff700 1 -- 172.21.15.131:0/2764722847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 msgr2=0x7fe1740cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.449+0000 7fe179bff700 1 --2- 172.21.15.131:0/2764722847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 0x7fe1740cc520 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe1740b43e0 tx=0x7fe16c009840 comp rx=0 tx=0).stop 2023-12-26T21:55:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.449+0000 7fe179bff700 1 -- 172.21.15.131:0/2764722847 shutdown_connections 2023-12-26T21:55:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.449+0000 7fe179bff700 1 --2- 172.21.15.131:0/2764722847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 0x7fe1740cc520 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.449+0000 7fe179bff700 1 --2- 172.21.15.131:0/2764722847 >> v2:172.21.15.190:3300/0 conn(0x7fe17406bfd0 0x7fe1740c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.449+0000 7fe179bff700 1 -- 172.21.15.131:0/2764722847 >> 172.21.15.131:0/2764722847 conn(0x7fe1740ce3e0 msgr2=0x7fe1740ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.449+0000 7fe179bff700 1 -- 172.21.15.131:0/2764722847 shutdown_connections 2023-12-26T21:55:20.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.449+0000 7fe179bff700 1 -- 172.21.15.131:0/2764722847 wait complete. 2023-12-26T21:55:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.450+0000 7fe179bff700 1 Processor -- start 2023-12-26T21:55:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.450+0000 7fe179bff700 1 -- start start 2023-12-26T21:55:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.451+0000 7fe179bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17406bfd0 0x7fe1741531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.451+0000 7fe179bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 0x7fe174152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.451+0000 7fe179bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17413c510 0x7fe17415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.451+0000 7fe179bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17415e2c0 con 0x7fe1740cc100 2023-12-26T21:55:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.451+0000 7fe179bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1741536f0 con 0x7fe17413c510 2023-12-26T21:55:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.451+0000 7fe179bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17415d6b0 con 0x7fe17406bfd0 2023-12-26T21:55:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe172ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 0x7fe174152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe172ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 0x7fe174152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54486/0 (socket says 172.21.15.131:54486) 2023-12-26T21:55:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe172ffd700 1 -- 172.21.15.131:0/592160047 learned_addr learned my addr 172.21.15.131:0/592160047 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe172ffd700 1 -- 172.21.15.131:0/592160047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17406bfd0 msgr2=0x7fe1741531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe172ffd700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17406bfd0 0x7fe1741531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe172ffd700 1 -- 172.21.15.131:0/592160047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17413c510 msgr2=0x7fe17415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe172ffd700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17413c510 0x7fe17415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe172ffd700 1 -- 172.21.15.131:0/592160047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c009ae0 con 0x7fe1740cc100 2023-12-26T21:55:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe172ffd700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 0x7fe174152130 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c830 tx=0x7fe168008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe15b7fe700 1 -- 172.21.15.131:0/592160047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168011600 con 0x7fe1740cc100 2023-12-26T21:55:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.452+0000 7fe15b7fe700 1 -- 172.21.15.131:0/592160047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168011760 con 0x7fe1740cc100 2023-12-26T21:55:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.453+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17415d990 con 0x7fe1740cc100 2023-12-26T21:55:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.453+0000 7fe15b7fe700 1 -- 172.21.15.131:0/592160047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168011ae0 con 0x7fe1740cc100 2023-12-26T21:55:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.453+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17415df50 con 0x7fe1740cc100 2023-12-26T21:55:20.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.454+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1740657e0 con 0x7fe1740cc100 2023-12-26T21:55:20.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.454+0000 7fe15b7fe700 1 -- 172.21.15.131:0/592160047 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe1680223a0 con 0x7fe1740cc100 2023-12-26T21:55:20.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.455+0000 7fe15b7fe700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe160066f30 0x7fe1600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:20.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.455+0000 7fe15b7fe700 1 -- 172.21.15.131:0/592160047 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5933+0+0 (secure 0 0 0) 0x7fe168084980 con 0x7fe1740cc100 2023-12-26T21:55:20.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.456+0000 7fe1737fe700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe160066f30 0x7fe1600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:20.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.456+0000 7fe1737fe700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe160066f30 0x7fe1600693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1640026e0 tx=0x7fe164007b70 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:20.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.459+0000 7fe15b7fe700 1 -- 172.21.15.131:0/592160047 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe168051a70 con 0x7fe1740cc100 2023-12-26T21:55:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.732+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fe174068ed0 con 0x7fe1740cc100 2023-12-26T21:55:20.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.891+0000 7fe15b7fe700 1 -- 172.21.15.131:0/592160047 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v691) v1 ==== 183+0+0 (secure 0 0 0) 0x7fe16801e0f0 con 0x7fe1740cc100 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe160066f30 msgr2=0x7fe1600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe160066f30 0x7fe1600693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1640026e0 tx=0x7fe164007b70 comp rx=0 tx=0).stop 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 msgr2=0x7fe174152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 0x7fe174152130 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c830 tx=0x7fe168008040 comp rx=0 tx=0).stop 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 shutdown_connections 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17413c510 0x7fe17415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17406bfd0 0x7fe1741531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe160066f30 0x7fe1600693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 --2- 172.21.15.131:0/592160047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1740cc100 0x7fe174152130 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 >> 172.21.15.131:0/592160047 conn(0x7fe1740ce3e0 msgr2=0x7fe1741539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:20.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 shutdown_connections 2023-12-26T21:55:20.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:20.893+0000 7fe179bff700 1 -- 172.21.15.131:0/592160047 wait complete. 2023-12-26T21:55:20.911 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-12-26T21:55:20.912 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-12-26T21:55:20.912 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T21:55:20.912 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-12-26T21:55:21.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.101+0000 7f3978fff700 1 Processor -- start 2023-12-26T21:55:21.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.102+0000 7f3978fff700 1 -- start start 2023-12-26T21:55:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.103+0000 7f3978fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 0x7f397415de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.103+0000 7f3978fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39740b07f0 con 0x7f39740cc580 2023-12-26T21:55:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.103+0000 7f3978fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39740b0d70 con 0x7f39740cc950 2023-12-26T21:55:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.103+0000 7f3978fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39740680f0 con 0x7f397415d9b0 2023-12-26T21:55:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.103+0000 7f397259c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f39740cc580 0x7f39740b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48568/0 (socket says 172.21.15.131:48568) 2023-12-26T21:55:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.103+0000 7f397259c700 1 -- 172.21.15.131:0/3404303192 learned_addr learned my addr 172.21.15.131:0/3404303192 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:21.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.103+0000 7f3972d9d700 1 --2- 172.21.15.131:0/3404303192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 0x7f397415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f397159a700 1 -- 172.21.15.131:0/3404303192 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202598864 0 0) 0x7f3964001110 con 0x7f39740cc580 2023-12-26T21:55:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f397159a700 1 -- 172.21.15.131:0/3404303192 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f395c003570 con 0x7f39740cc580 2023-12-26T21:55:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f397159a700 1 -- 172.21.15.131:0/3404303192 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218205296 0 0) 0x7f39740b0d70 con 0x7f39740cc950 2023-12-26T21:55:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f397159a700 1 -- 172.21.15.131:0/3404303192 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3964001110 con 0x7f39740cc950 2023-12-26T21:55:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f3972d9d700 1 -- 172.21.15.131:0/3404303192 >> v1:172.21.15.190:6789/0 conn(0x7f39740cc950 legacy=0x7f397406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f3972d9d700 1 -- 172.21.15.131:0/3404303192 >> v1:172.21.15.131:6789/0 conn(0x7f39740cc580 legacy=0x7f39740b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f3972d9d700 1 -- 172.21.15.131:0/3404303192 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39740675b0 con 0x7f397415d9b0 2023-12-26T21:55:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f3972d9d700 1 --2- 172.21.15.131:0/3404303192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 0x7f397415de30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f396c009e90 tx=0x7f396c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611c8a07ace36886 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:21.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f397159a700 1 -- 172.21.15.131:0/3404303192 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c00e070 con 0x7f397415d9b0 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f397159a700 1 -- 172.21.15.131:0/3404303192 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c00bb80 con 0x7f397415d9b0 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.104+0000 7f397159a700 1 -- 172.21.15.131:0/3404303192 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c012430 con 0x7f397415d9b0 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.105+0000 7f3978fff700 1 -- 172.21.15.131:0/3404303192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 msgr2=0x7f397415de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.105+0000 7f3978fff700 1 --2- 172.21.15.131:0/3404303192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 0x7f397415de30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f396c009e90 tx=0x7f396c009780 comp rx=0 tx=0).stop 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.105+0000 7f3978fff700 1 -- 172.21.15.131:0/3404303192 shutdown_connections 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.105+0000 7f3978fff700 1 --2- 172.21.15.131:0/3404303192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 0x7f397415de30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.105+0000 7f3978fff700 1 -- 172.21.15.131:0/3404303192 >> 172.21.15.131:0/3404303192 conn(0x7f39740686c0 msgr2=0x7f3974068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.105+0000 7f3978fff700 1 -- 172.21.15.131:0/3404303192 shutdown_connections 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.105+0000 7f3978fff700 1 -- 172.21.15.131:0/3404303192 wait complete. 2023-12-26T21:55:21.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.107+0000 7f3978fff700 1 Processor -- start 2023-12-26T21:55:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.107+0000 7f3978fff700 1 -- start start 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3978fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39740cc580 0x7f3974147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3978fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39740cc950 0x7f39740acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3978fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 0x7f39740ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3978fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3974148410 con 0x7f39740cc950 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3978fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3974159e50 con 0x7f39740cc580 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3978fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397415a130 con 0x7f397415d9b0 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3971d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39740cc950 0x7f39740acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3971d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39740cc950 0x7f39740acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54500/0 (socket says 172.21.15.131:54500) 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3971d9b700 1 -- 172.21.15.131:0/3617837443 learned_addr learned my addr 172.21.15.131:0/3617837443 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f397259c700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39740cc580 0x7f3974147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3972d9d700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 0x7f39740ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3971d9b700 1 -- 172.21.15.131:0/3617837443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 msgr2=0x7f39740ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3971d9b700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 0x7f39740ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3971d9b700 1 -- 172.21.15.131:0/3617837443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39740cc580 msgr2=0x7f3974147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3971d9b700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39740cc580 0x7f3974147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f3971d9b700 1 -- 172.21.15.131:0/3617837443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c009a20 con 0x7f39740cc950 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.108+0000 7f397259c700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39740cc580 0x7f3974147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.109+0000 7f3971d9b700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39740cc950 0x7f39740acbc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f396800c7c0 tx=0x7f3968007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.109+0000 7f39627fc700 1 -- 172.21.15.131:0/3617837443 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39680015f0 con 0x7f39740cc950 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.109+0000 7f39627fc700 1 -- 172.21.15.131:0/3617837443 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3968001770 con 0x7f39740cc950 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.109+0000 7f39627fc700 1 -- 172.21.15.131:0/3617837443 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968001b30 con 0x7f39740cc950 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.109+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3974169a60 con 0x7f39740cc950 2023-12-26T21:55:21.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.109+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39740ad940 con 0x7f39740cc950 2023-12-26T21:55:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.110+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39740657e0 con 0x7f39740cc950 2023-12-26T21:55:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.111+0000 7f39627fc700 1 -- 172.21.15.131:0/3617837443 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f39680123e0 con 0x7f39740cc950 2023-12-26T21:55:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.111+0000 7f39627fc700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3954066f30 0x7f39540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.112+0000 7f39627fc700 1 -- 172.21.15.131:0/3617837443 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5933+0+0 (secure 0 0 0) 0x7f3968084370 con 0x7f39740cc950 2023-12-26T21:55:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.112+0000 7f397259c700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3954066f30 0x7f39540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:21.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.112+0000 7f397259c700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3954066f30 0x7f39540693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3964007fd0 tx=0x7f3964009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:21.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.116+0000 7f39627fc700 1 -- 172.21.15.131:0/3617837443 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3968051460 con 0x7f39740cc950 2023-12-26T21:55:21.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.384+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f397415a270 con 0x7f39740cc950 2023-12-26T21:55:21.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.892+0000 7f39627fc700 1 -- 172.21.15.131:0/3617837443 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v692) v1 ==== 151+0+0 (secure 0 0 0) 0x7f3968054a80 con 0x7f39740cc950 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3954066f30 msgr2=0x7f39540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3954066f30 0x7f39540693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3964007fd0 tx=0x7f3964009040 comp rx=0 tx=0).stop 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39740cc950 msgr2=0x7f39740acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39740cc950 0x7f39740acbc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f396800c7c0 tx=0x7f3968007d90 comp rx=0 tx=0).stop 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 shutdown_connections 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f397415d9b0 0x7f39740ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39740cc580 0x7f3974147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3954066f30 0x7f39540693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 --2- 172.21.15.131:0/3617837443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39740cc950 0x7f39740acbc0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 >> 172.21.15.131:0/3617837443 conn(0x7f39740686c0 msgr2=0x7f39740cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 shutdown_connections 2023-12-26T21:55:21.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:21.895+0000 7f3978fff700 1 -- 172.21.15.131:0/3617837443 wait complete. 2023-12-26T21:55:21.914 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T21:55:22.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.106+0000 7f28ee3ff700 1 Processor -- start 2023-12-26T21:55:22.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.106+0000 7f28ee3ff700 1 -- start start 2023-12-26T21:55:22.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.107+0000 7f28ee3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80cc580 0x7f28e80cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.107+0000 7f28ee3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f28e80b0550 0x7f28e815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.107+0000 7f28ee3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e80b0d70 con 0x7f28e80b00e0 2023-12-26T21:55:22.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.107+0000 7f28ee3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e80680f0 con 0x7f28e80b0550 2023-12-26T21:55:22.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.107+0000 7f28ee3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80675b0 con 0x7f28e80cc580 2023-12-26T21:55:22.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e77fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f28e80b00e0 0x7f28e806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48578/0 (socket says 172.21.15.131:48578) 2023-12-26T21:55:22.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e77fe700 1 -- 172.21.15.131:0/3487816911 learned_addr learned my addr 172.21.15.131:0/3487816911 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:22.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e7fff700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80cc580 0x7f28e80cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28ec99c700 1 --2- 172.21.15.131:0/3487816911 >> v2:172.21.15.190:3300/0 conn(0x7f28e80b0550 0x7f28e815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120886265 0 0) 0x7f28e80b0d70 con 0x7f28e80b00e0 2023-12-26T21:55:22.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d0003570 con 0x7f28e80b00e0 2023-12-26T21:55:22.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1675355905 0 0) 0x7f28d0003570 con 0x7f28e80b00e0 2023-12-26T21:55:22.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80cc580 msgr2=0x7f28e80cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:22.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80cc580 0x7f28e80cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 >> v2:172.21.15.190:3300/0 conn(0x7f28e80b0550 msgr2=0x7f28e815d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:22.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 --2- 172.21.15.131:0/3487816911 >> v2:172.21.15.190:3300/0 conn(0x7f28e80b0550 0x7f28e815d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e8159c30 con 0x7f28e80b00e0 2023-12-26T21:55:22.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f28dc001470 con 0x7f28e80b00e0 2023-12-26T21:55:22.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28ec99c700 1 --2- 172.21.15.131:0/3487816911 >> v2:172.21.15.190:3300/0 conn(0x7f28e80b0550 0x7f28e815d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.108+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 >> v1:172.21.15.131:6789/0 conn(0x7f28e80b00e0 legacy=0x7f28e806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28e6ffd700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d0003a60 0x7f28d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28e6ffd700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80cc580 0x7f28d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28e6ffd700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d0004d50 0x7f28d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80680f0 con 0x7f28d0004d50 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80675b0 con 0x7f28d0003a60 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d000ba60 con 0x7f28e80cc580 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28ec99c700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d0004d50 0x7f28d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28e7fff700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d0003a60 0x7f28d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28e77fe700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80cc580 0x7f28d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28ec99c700 1 -- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80cc580 msgr2=0x7f28d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28ec99c700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80cc580 0x7f28d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28ec99c700 1 -- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d0003a60 msgr2=0x7f28d0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28ec99c700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d0003a60 0x7f28d0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.109+0000 7f28ec99c700 1 -- 172.21.15.131:0/3487816911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d0003570 con 0x7f28d0004d50 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28ec99c700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d0004d50 0x7f28d00051c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f28e000e3c0 tx=0x7f28e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0c0be97997b44f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e000ec10 con 0x7f28d0004d50 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e000f040 con 0x7f28d0004d50 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28e6ffd700 1 -- 172.21.15.131:0/3487816911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0014480 con 0x7f28d0004d50 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28ee3ff700 1 -- 172.21.15.131:0/3487816911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d0004d50 msgr2=0x7f28d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28ee3ff700 1 --2- 172.21.15.131:0/3487816911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d0004d50 0x7f28d00051c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f28e000e3c0 tx=0x7f28e000b040 comp rx=0 tx=0).stop 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28ec99c700 1 -- 172.21.15.131:0/3487816911 reap_dead start 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28ee3ff700 1 -- 172.21.15.131:0/3487816911 shutdown_connections 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28ee3ff700 1 -- 172.21.15.131:0/3487816911 >> 172.21.15.131:0/3487816911 conn(0x7f28e80686c0 msgr2=0x7f28e8068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:22.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28ee3ff700 1 -- 172.21.15.131:0/3487816911 shutdown_connections 2023-12-26T21:55:22.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.110+0000 7f28ee3ff700 1 -- 172.21.15.131:0/3487816911 wait complete. 2023-12-26T21:55:22.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.112+0000 7f28ee3ff700 1 Processor -- start 2023-12-26T21:55:22.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.112+0000 7f28ee3ff700 1 -- start start 2023-12-26T21:55:22.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28ee3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d0004d50 0x7f28e8156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28ee3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80af110 0x7f28e80af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28ee3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e80ad0e0 0x7f28e80ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28ee3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80afac0 con 0x7f28e80ad0e0 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28ee3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e8169c10 con 0x7f28d0004d50 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28ee3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80ae6b0 con 0x7f28e80af110 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28ec99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e80ad0e0 0x7f28e80ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28ec99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e80ad0e0 0x7f28e80ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54526/0 (socket says 172.21.15.131:54526) 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28ec99c700 1 -- 172.21.15.131:0/1559452517 learned_addr learned my addr 172.21.15.131:0/1559452517 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28e7fff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d0004d50 0x7f28e8156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.113+0000 7f28e77fe700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80af110 0x7f28e80af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28ec99c700 1 -- 172.21.15.131:0/1559452517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80af110 msgr2=0x7f28e80af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28ec99c700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80af110 0x7f28e80af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28ec99c700 1 -- 172.21.15.131:0/1559452517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d0004d50 msgr2=0x7f28e8156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28ec99c700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d0004d50 0x7f28e8156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28ec99c700 1 -- 172.21.15.131:0/1559452517 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80ade10 con 0x7f28e80ad0e0 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28ec99c700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e80ad0e0 0x7f28e80ad530 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f28e0002230 tx=0x7f28e001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28cffff700 1 -- 172.21.15.131:0/1559452517 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0014480 con 0x7f28e80ad0e0 2023-12-26T21:55:22.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28cffff700 1 -- 172.21.15.131:0/1559452517 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e001d630 con 0x7f28e80ad0e0 2023-12-26T21:55:22.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e80ae0f0 con 0x7f28e80ad0e0 2023-12-26T21:55:22.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28cffff700 1 -- 172.21.15.131:0/1559452517 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e001d9f0 con 0x7f28e80ad0e0 2023-12-26T21:55:22.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.114+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e80ada70 con 0x7f28e80ad0e0 2023-12-26T21:55:22.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.115+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e80657e0 con 0x7f28e80ad0e0 2023-12-26T21:55:22.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.116+0000 7f28cffff700 1 -- 172.21.15.131:0/1559452517 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f28e0003040 con 0x7f28e80ad0e0 2023-12-26T21:55:22.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.117+0000 7f28cffff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f28d4066f30 0x7f28d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.117+0000 7f28cffff700 1 -- 172.21.15.131:0/1559452517 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5933+0+0 (secure 0 0 0) 0x7f28e0085c50 con 0x7f28e80ad0e0 2023-12-26T21:55:22.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.118+0000 7f28e7fff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f28d4066f30 0x7f28d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.118+0000 7f28e7fff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f28d4066f30 0x7f28d40693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f28d8006f00 tx=0x7f28d80091a0 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:22.123 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.121+0000 7f28cffff700 1 -- 172.21.15.131:0/1559452517 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28e0052e60 con 0x7f28e80ad0e0 2023-12-26T21:55:22.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.316+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f28e8067850 con 0x7f28e80ad0e0 2023-12-26T21:55:22.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.320+0000 7f28cffff700 1 -- 172.21.15.131:0/1559452517 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f28e0056480 con 0x7f28e80ad0e0 2023-12-26T21:55:22.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f28d4066f30 msgr2=0x7f28d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f28d4066f30 0x7f28d40693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f28d8006f00 tx=0x7f28d80091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e80ad0e0 msgr2=0x7f28e80ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e80ad0e0 0x7f28e80ad530 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f28e0002230 tx=0x7f28e001a000 comp rx=0 tx=0).stop 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 shutdown_connections 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f28d4066f30 0x7f28d40693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28e80ad0e0 0x7f28e80ad530 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28d0004d50 0x7f28e8156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 --2- 172.21.15.131:0/1559452517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28e80af110 0x7f28e80af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 >> 172.21.15.131:0/1559452517 conn(0x7f28e80686c0 msgr2=0x7f28e8158130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 shutdown_connections 2023-12-26T21:55:22.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.323+0000 7f28ee3ff700 1 -- 172.21.15.131:0/1559452517 wait complete. 2023-12-26T21:55:22.341 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:55:22.342 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:55:22.441 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:55:22.534 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:55:22.534 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T21:55:22.534 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:55:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.722+0000 7f1f6bbff700 1 Processor -- start 2023-12-26T21:55:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.722+0000 7f1f6bbff700 1 -- start start 2023-12-26T21:55:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.723+0000 7f1f6bbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f640c8bf0 0x7f1f640b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.723+0000 7f1f6bbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f64069d10 0x7f1f640b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.723+0000 7f1f6bbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640caa10 con 0x7f1f640c8820 2023-12-26T21:55:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.723+0000 7f1f6bbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f64157a80 con 0x7f1f640c8bf0 2023-12-26T21:55:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.723+0000 7f1f6bbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640cac50 con 0x7f1f64069d10 2023-12-26T21:55:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.723+0000 7f1f6919a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f640c8bf0 0x7f1f640b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.723+0000 7f1f6919a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f640c8bf0 0x7f1f640b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54530/0 (socket says 172.21.15.131:54530) 2023-12-26T21:55:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.723+0000 7f1f6919a700 1 -- 172.21.15.131:0/4251192777 learned_addr learned my addr 172.21.15.131:0/4251192777 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.724+0000 7f1f6a19c700 1 --2- 172.21.15.131:0/4251192777 >> v2:172.21.15.190:3300/0 conn(0x7f1f64069d10 0x7f1f640b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.724+0000 7f1f6919a700 1 -- 172.21.15.131:0/4251192777 >> v2:172.21.15.190:3300/0 conn(0x7f1f64069d10 msgr2=0x7f1f640b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.724+0000 7f1f6919a700 1 --2- 172.21.15.131:0/4251192777 >> v2:172.21.15.190:3300/0 conn(0x7f1f64069d10 0x7f1f640b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.724+0000 7f1f6919a700 1 -- 172.21.15.131:0/4251192777 >> v1:172.21.15.190:6789/0 conn(0x7f1f640c8820 legacy=0x7f1f640ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.724+0000 7f1f6919a700 1 -- 172.21.15.131:0/4251192777 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640caf30 con 0x7f1f640c8bf0 2023-12-26T21:55:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.724+0000 7f1f6919a700 1 --2- 172.21.15.131:0/4251192777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8bf0 0x7f1f640b12e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1f64070f80 tx=0x7f1f54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4795f519edc6778b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.724+0000 7f1f68999700 1 -- 172.21.15.131:0/4251192777 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400b860 con 0x7f1f640c8bf0 2023-12-26T21:55:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.724+0000 7f1f68999700 1 -- 172.21.15.131:0/4251192777 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5400b9c0 con 0x7f1f640c8bf0 2023-12-26T21:55:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.724+0000 7f1f68999700 1 -- 172.21.15.131:0/4251192777 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400bd60 con 0x7f1f640c8bf0 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.725+0000 7f1f6bbff700 1 -- 172.21.15.131:0/4251192777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8bf0 msgr2=0x7f1f640b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.725+0000 7f1f6bbff700 1 --2- 172.21.15.131:0/4251192777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8bf0 0x7f1f640b12e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1f64070f80 tx=0x7f1f54009250 comp rx=0 tx=0).stop 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.725+0000 7f1f6bbff700 1 -- 172.21.15.131:0/4251192777 shutdown_connections 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.725+0000 7f1f6bbff700 1 --2- 172.21.15.131:0/4251192777 >> v2:172.21.15.190:3300/0 conn(0x7f1f64069d10 0x7f1f640b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.725+0000 7f1f6bbff700 1 --2- 172.21.15.131:0/4251192777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8bf0 0x7f1f640b12e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.725+0000 7f1f6bbff700 1 -- 172.21.15.131:0/4251192777 >> 172.21.15.131:0/4251192777 conn(0x7f1f64068040 msgr2=0x7f1f64069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.725+0000 7f1f6bbff700 1 -- 172.21.15.131:0/4251192777 shutdown_connections 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.725+0000 7f1f6bbff700 1 -- 172.21.15.131:0/4251192777 wait complete. 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.726+0000 7f1f6bbff700 1 Processor -- start 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.727+0000 7f1f6bbff700 1 -- start start 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6bbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f64069d10 0x7f1f64150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6bbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8820 0x7f1f640c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f640c8bf0 0x7f1f64142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6bbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640c7f80 con 0x7f1f640c8820 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6bbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641696a0 con 0x7f1f64069d10 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64150820 con 0x7f1f640c8bf0 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6919a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8820 0x7f1f640c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6919a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8820 0x7f1f640c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54544/0 (socket says 172.21.15.131:54544) 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6919a700 1 -- 172.21.15.131:0/1908249431 learned_addr learned my addr 172.21.15.131:0/1908249431 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6a19c700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f640c8bf0 0x7f1f64142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6999b700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f64069d10 0x7f1f64150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6919a700 1 -- 172.21.15.131:0/1908249431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f640c8bf0 msgr2=0x7f1f64142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6919a700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f640c8bf0 0x7f1f64142ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6919a700 1 -- 172.21.15.131:0/1908249431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f64069d10 msgr2=0x7f1f64150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6919a700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f64069d10 0x7f1f64150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.728+0000 7f1f6919a700 1 -- 172.21.15.131:0/1908249431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f540094f0 con 0x7f1f640c8820 2023-12-26T21:55:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.729+0000 7f1f6919a700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8820 0x7f1f640c7a40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1f640688f0 tx=0x7f1f54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.729+0000 7f1f59ffb700 1 -- 172.21.15.131:0/1908249431 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54011e40 con 0x7f1f640c8820 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.729+0000 7f1f59ffb700 1 -- 172.21.15.131:0/1908249431 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5401a450 con 0x7f1f640c8820 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.729+0000 7f1f59ffb700 1 -- 172.21.15.131:0/1908249431 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5401a7f0 con 0x7f1f640c8820 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.729+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f64150b00 con 0x7f1f640c8820 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.729+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f641510c0 con 0x7f1f640c8820 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.730+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f640657e0 con 0x7f1f640c8820 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.731+0000 7f1f59ffb700 1 -- 172.21.15.131:0/1908249431 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f5401aba0 con 0x7f1f640c8820 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.732+0000 7f1f59ffb700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1f48066f30 0x7f1f480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.732+0000 7f1f59ffb700 1 -- 172.21.15.131:0/1908249431 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5933+0+0 (secure 0 0 0) 0x7f1f540861b0 con 0x7f1f640c8820 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.732+0000 7f1f6999b700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1f48066f30 0x7f1f480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.732+0000 7f1f6999b700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1f48066f30 0x7f1f480693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f60003a80 tx=0x7f1f60007ec0 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:22.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.736+0000 7f1f59ffb700 1 -- 172.21.15.131:0/1908249431 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f540532a0 con 0x7f1f640c8820 2023-12-26T21:55:22.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.973+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f64169810 con 0x7f1f640c8820 2023-12-26T21:55:22.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.974+0000 7f1f59ffb700 1 -- 172.21.15.131:0/1908249431 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f1f54015df0 con 0x7f1f640c8820 2023-12-26T21:55:22.975 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:22.975 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2416144644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2416144644},{"type":"v1","addr":"172.21.15.131:6835","nonce":2416144644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/559791617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":559791617},{"type":"v1","addr":"172.21.15.190:6834","nonce":559791617}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6901,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3954759429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3954759429},{"type":"v1","addr":"172.21.15.190:6835","nonce":3954759429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":7234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/2388584698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2388584698},{"type":"v1","addr":"172.21.15.131:6834","nonce":2388584698}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[]} 2023-12-26T21:55:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1f48066f30 msgr2=0x7f1f480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1f48066f30 0x7f1f480693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f60003a80 tx=0x7f1f60007ec0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8820 msgr2=0x7f1f640c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8820 0x7f1f640c7a40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1f640688f0 tx=0x7f1f54017000 comp rx=0 tx=0).stop 2023-12-26T21:55:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 shutdown_connections 2023-12-26T21:55:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f640c8bf0 0x7f1f64142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f64069d10 0x7f1f64150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1f48066f30 0x7f1f480693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 --2- 172.21.15.131:0/1908249431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f640c8820 0x7f1f640c7a40 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:22.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 >> 172.21.15.131:0/1908249431 conn(0x7f1f64068040 msgr2=0x7f1f640ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:22.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.976+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 shutdown_connections 2023-12-26T21:55:22.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:22.977+0000 7f1f6bbff700 1 -- 172.21.15.131:0/1908249431 wait complete. 2023-12-26T21:55:22.978 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 52 2023-12-26T21:55:22.996 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T21:55:22.996 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T21:55:22.996 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T21:55:22.996 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:55:22.996 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T21:55:22.996 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T21:55:22.997 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T21:55:22.997 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:55:22.997 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T21:55:22.997 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T21:55:22.997 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T21:55:22.997 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:55:22.997 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T21:55:22.997 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T21:55:22.998 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T21:55:22.998 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:55:23.010 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T21:55:23.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T21:55:23.011 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T21:55:23.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T21:55:23.018 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T21:55:23.018 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T21:55:23.019 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T21:55:23.019 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T21:55:23.056 INFO:tasks.ceph.mds.a:Started 2023-12-26T21:55:23.056 DEBUG:teuthology.parallel:result is None 2023-12-26T21:55:23.058 INFO:tasks.ceph.mds.c:Started 2023-12-26T21:55:23.058 DEBUG:teuthology.parallel:result is None 2023-12-26T21:55:23.062 INFO:tasks.ceph.mds.b:Started 2023-12-26T21:55:23.063 DEBUG:teuthology.parallel:result is None 2023-12-26T21:55:23.065 INFO:tasks.ceph.mds.d:Started 2023-12-26T21:55:23.066 DEBUG:teuthology.parallel:result is None 2023-12-26T21:55:23.066 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T21:55:23.199 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T21:55:23.200 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:55:23.209 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T21:55:23.209 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:55:23.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.266+0000 7faf127ff700 1 Processor -- start 2023-12-26T21:55:23.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.266+0000 7faf127ff700 1 -- start start 2023-12-26T21:55:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.267+0000 7faf127ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faf0c0c8820 0x7faf0c0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.267+0000 7faf127ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0b19f0 0x7faf0c161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.267+0000 7faf127ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0c157a80 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.267+0000 7faf127ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf0c0cac50 con 0x7faf0c0c8820 2023-12-26T21:55:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.267+0000 7faf127ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0caf30 con 0x7faf0c0b19f0 2023-12-26T21:55:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.267+0000 7faf0b7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7faf0c0c8bf0 0x7faf0c0b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48604/0 (socket says 172.21.15.131:48604) 2023-12-26T21:55:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.267+0000 7faf0b7fe700 1 -- 172.21.15.131:0/3414327557 learned_addr learned my addr 172.21.15.131:0/3414327557 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.267+0000 7faf0bfff700 1 --2- 172.21.15.131:0/3414327557 >> v2:172.21.15.190:3300/0 conn(0x7faf0c0c8820 0x7faf0c0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.267+0000 7faf10d9c700 1 --2- 172.21.15.131:0/3414327557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0b19f0 0x7faf0c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0affd700 1 -- 172.21.15.131:0/3414327557 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095043837 0 0) 0x7faf0c157a80 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0affd700 1 -- 172.21.15.131:0/3414327557 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faef4003570 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0bfff700 1 -- 172.21.15.131:0/3414327557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0b19f0 msgr2=0x7faf0c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0bfff700 1 --2- 172.21.15.131:0/3414327557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0b19f0 0x7faf0c161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0bfff700 1 -- 172.21.15.131:0/3414327557 >> v1:172.21.15.131:6789/0 conn(0x7faf0c0c8bf0 legacy=0x7faf0c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0bfff700 1 -- 172.21.15.131:0/3414327557 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0cb210 con 0x7faf0c0c8820 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0bfff700 1 --2- 172.21.15.131:0/3414327557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf0c0c8820 0x7faf0c0ca300 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faefc0098e0 tx=0x7faefc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32fa489c4f81ca54 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0affd700 1 -- 172.21.15.131:0/3414327557 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc011630 con 0x7faf0c0c8820 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0affd700 1 -- 172.21.15.131:0/3414327557 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc011790 con 0x7faf0c0c8820 2023-12-26T21:55:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.268+0000 7faf0affd700 1 -- 172.21.15.131:0/3414327557 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc011b30 con 0x7faf0c0c8820 2023-12-26T21:55:23.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.269+0000 7faf127ff700 1 -- 172.21.15.131:0/3414327557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf0c0c8820 msgr2=0x7faf0c0ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:23.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.269+0000 7faf127ff700 1 --2- 172.21.15.131:0/3414327557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf0c0c8820 0x7faf0c0ca300 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faefc0098e0 tx=0x7faefc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.269+0000 7faf127ff700 1 -- 172.21.15.131:0/3414327557 shutdown_connections 2023-12-26T21:55:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.269+0000 7faf127ff700 1 --2- 172.21.15.131:0/3414327557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0b19f0 0x7faf0c161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.269+0000 7faf127ff700 1 --2- 172.21.15.131:0/3414327557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf0c0c8820 0x7faf0c0ca300 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.269+0000 7faf127ff700 1 -- 172.21.15.131:0/3414327557 >> 172.21.15.131:0/3414327557 conn(0x7faf0c068040 msgr2=0x7faf0c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.269+0000 7faf127ff700 1 -- 172.21.15.131:0/3414327557 shutdown_connections 2023-12-26T21:55:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.269+0000 7faf127ff700 1 -- 172.21.15.131:0/3414327557 wait complete. 2023-12-26T21:55:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.270+0000 7faf127ff700 1 Processor -- start 2023-12-26T21:55:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.271+0000 7faf127ff700 1 -- start start 2023-12-26T21:55:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf127ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf0c0b19f0 0x7faf0c0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf127ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0c8820 0x7faf0c16c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf127ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf0c0c8bf0 0x7faf0c154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:23.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf127ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c16be60 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf127ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c16b250 con 0x7faf0c0b19f0 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf127ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c16b530 con 0x7faf0c0c8820 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf10d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf0c0c8bf0 0x7faf0c154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf10d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf0c0c8bf0 0x7faf0c154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54600/0 (socket says 172.21.15.131:54600) 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf10d9c700 1 -- 172.21.15.131:0/3215058816 learned_addr learned my addr 172.21.15.131:0/3215058816 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf0bfff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf0c0b19f0 0x7faf0c0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf0b7fe700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0c8820 0x7faf0c16c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf10d9c700 1 -- 172.21.15.131:0/3215058816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0c8820 msgr2=0x7faf0c16c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf10d9c700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0c8820 0x7faf0c16c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf10d9c700 1 -- 172.21.15.131:0/3215058816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf0c0b19f0 msgr2=0x7faf0c0c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf10d9c700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf0c0b19f0 0x7faf0c0c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf10d9c700 1 -- 172.21.15.131:0/3215058816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc009420 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.272+0000 7faf0b7fe700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0c8820 0x7faf0c16c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:23.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.273+0000 7faf10d9c700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf0c0c8bf0 0x7faf0c154590 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7faf04009d10 tx=0x7faf0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.273+0000 7faef3fff700 1 -- 172.21.15.131:0/3215058816 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04009e70 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.273+0000 7faef3fff700 1 -- 172.21.15.131:0/3215058816 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0400f040 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.273+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c16b810 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.273+0000 7faef3fff700 1 -- 172.21.15.131:0/3215058816 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400cda0 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.273+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c0c7f60 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.274+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0c16c010 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.275+0000 7faef3fff700 1 -- 172.21.15.131:0/3215058816 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faf04014690 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.276+0000 7faef3fff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faef8066f30 0x7faef80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:23.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.276+0000 7faef3fff700 1 -- 172.21.15.131:0/3215058816 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5933+0+0 (secure 0 0 0) 0x7faf04085210 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.276+0000 7faf0bfff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faef8066f30 0x7faef80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:23.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.277+0000 7faf0bfff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faef8066f30 0x7faef80693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faefc0091e0 tx=0x7faefc017000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:23.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.280+0000 7faef3fff700 1 -- 172.21.15.131:0/3215058816 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf0c16c010 con 0x7faf0c0c8bf0 2023-12-26T21:55:23.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:23.492+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7faf0c0657e0 con 0x7faf0c0c8bf0 2023-12-26T21:55:24.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.349+0000 7faef3fff700 1 -- 172.21.15.131:0/3215058816 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7faf04052300 con 0x7faf0c0c8bf0 2023-12-26T21:55:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.351+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faef8066f30 msgr2=0x7faef80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.351+0000 7faf127ff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faef8066f30 0x7faef80693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faefc0091e0 tx=0x7faefc017000 comp rx=0 tx=0).stop 2023-12-26T21:55:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.351+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf0c0c8bf0 msgr2=0x7faf0c154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.351+0000 7faf127ff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf0c0c8bf0 0x7faf0c154590 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7faf04009d10 tx=0x7faf0400b040 comp rx=0 tx=0).stop 2023-12-26T21:55:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.352+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 shutdown_connections 2023-12-26T21:55:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.352+0000 7faf127ff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faef8066f30 0x7faef80693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.352+0000 7faf127ff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf0c0c8bf0 0x7faf0c154590 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.352+0000 7faf127ff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf0c0b19f0 0x7faf0c0c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.352+0000 7faf127ff700 1 --2- 172.21.15.131:0/3215058816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf0c0c8820 0x7faf0c16c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.352+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 >> 172.21.15.131:0/3215058816 conn(0x7faf0c068040 msgr2=0x7faf0c0ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.352+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 shutdown_connections 2023-12-26T21:55:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.352+0000 7faf127ff700 1 -- 172.21.15.131:0/3215058816 wait complete. 2023-12-26T21:55:24.354 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T21:55:24.374 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T21:55:24.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.566+0000 7f819a3ff700 1 Processor -- start 2023-12-26T21:55:24.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.566+0000 7f819a3ff700 1 -- start start 2023-12-26T21:55:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.567+0000 7f819a3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f819413c510 0x7f81940c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.567+0000 7f819a3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f819406b8c0 0x7f819406bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.567+0000 7f819a3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81940686c0 con 0x7f81940cc100 2023-12-26T21:55:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.567+0000 7f819a3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819406c280 con 0x7f819406b8c0 2023-12-26T21:55:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.567+0000 7f819a3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8194069520 con 0x7f819413c510 2023-12-26T21:55:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.567+0000 7f81937fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f819406b8c0 0x7f819406bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.567+0000 7f81937fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f819406b8c0 0x7f819406bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54614/0 (socket says 172.21.15.131:54614) 2023-12-26T21:55:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.567+0000 7f81937fe700 1 -- 172.21.15.131:0/3253694675 learned_addr learned my addr 172.21.15.131:0/3253694675 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.568+0000 7f8193fff700 1 --2- 172.21.15.131:0/3253694675 >> v2:172.21.15.190:3300/0 conn(0x7f819413c510 0x7f81940c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.568+0000 7f81937fe700 1 -- 172.21.15.131:0/3253694675 >> v2:172.21.15.190:3300/0 conn(0x7f819413c510 msgr2=0x7f81940c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.568+0000 7f81937fe700 1 --2- 172.21.15.131:0/3253694675 >> v2:172.21.15.190:3300/0 conn(0x7f819413c510 0x7f81940c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.568+0000 7f81937fe700 1 -- 172.21.15.131:0/3253694675 >> v1:172.21.15.190:6789/0 conn(0x7f81940cc100 legacy=0x7f81940cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.568+0000 7f81937fe700 1 -- 172.21.15.131:0/3253694675 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8194068910 con 0x7f819406b8c0 2023-12-26T21:55:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.568+0000 7f81937fe700 1 --2- 172.21.15.131:0/3253694675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 0x7f819406bd40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f819406fcc0 tx=0x7f8188009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50db7f33b1c4254 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.568+0000 7f8192ffd700 1 -- 172.21.15.131:0/3253694675 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818800b820 con 0x7f819406b8c0 2023-12-26T21:55:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.568+0000 7f8192ffd700 1 -- 172.21.15.131:0/3253694675 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818800b980 con 0x7f819406b8c0 2023-12-26T21:55:24.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.568+0000 7f8192ffd700 1 -- 172.21.15.131:0/3253694675 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818800bd20 con 0x7f819406b8c0 2023-12-26T21:55:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.569+0000 7f819a3ff700 1 -- 172.21.15.131:0/3253694675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 msgr2=0x7f819406bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.569+0000 7f819a3ff700 1 --2- 172.21.15.131:0/3253694675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 0x7f819406bd40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f819406fcc0 tx=0x7f8188009250 comp rx=0 tx=0).stop 2023-12-26T21:55:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.569+0000 7f819a3ff700 1 -- 172.21.15.131:0/3253694675 shutdown_connections 2023-12-26T21:55:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.569+0000 7f819a3ff700 1 --2- 172.21.15.131:0/3253694675 >> v2:172.21.15.190:3300/0 conn(0x7f819413c510 0x7f81940c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.569+0000 7f819a3ff700 1 --2- 172.21.15.131:0/3253694675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 0x7f819406bd40 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.569+0000 7f819a3ff700 1 -- 172.21.15.131:0/3253694675 >> 172.21.15.131:0/3253694675 conn(0x7f81940ce3e0 msgr2=0x7f81940ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.569+0000 7f819a3ff700 1 -- 172.21.15.131:0/3253694675 shutdown_connections 2023-12-26T21:55:24.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.569+0000 7f819a3ff700 1 -- 172.21.15.131:0/3253694675 wait complete. 2023-12-26T21:55:24.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.570+0000 7f819a3ff700 1 Processor -- start 2023-12-26T21:55:24.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.571+0000 7f819a3ff700 1 -- start start 2023-12-26T21:55:24.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f819a3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 0x7f8194153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:24.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f819a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81940cc100 0x7f81940c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f819a3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f819413c510 0x7f81941578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f819a3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8194154120 con 0x7f819406b8c0 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f819a3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8194154f80 con 0x7f819413c510 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f819a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8194154370 con 0x7f81940cc100 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f8193fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 0x7f8194153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f8193fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 0x7f8194153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54618/0 (socket says 172.21.15.131:54618) 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f8193fff700 1 -- 172.21.15.131:0/4066387183 learned_addr learned my addr 172.21.15.131:0/4066387183 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f81937fe700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81940cc100 0x7f81940c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f819899c700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f819413c510 0x7f81941578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f8193fff700 1 -- 172.21.15.131:0/4066387183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81940cc100 msgr2=0x7f81940c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:24.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f8193fff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81940cc100 0x7f81940c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f8193fff700 1 -- 172.21.15.131:0/4066387183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f819413c510 msgr2=0x7f81941578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f8193fff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f819413c510 0x7f81941578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f8193fff700 1 -- 172.21.15.131:0/4066387183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880094f0 con 0x7f819406b8c0 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.572+0000 7f81937fe700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81940cc100 0x7f81940c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.573+0000 7f8193fff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 0x7f8194153900 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f818400d890 tx=0x7f8184008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.573+0000 7f817bfff700 1 -- 172.21.15.131:0/4066387183 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184018070 con 0x7f819406b8c0 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.573+0000 7f817bfff700 1 -- 172.21.15.131:0/4066387183 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8184005a00 con 0x7f819406b8c0 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.573+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8194154650 con 0x7f819406b8c0 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.573+0000 7f817bfff700 1 -- 172.21.15.131:0/4066387183 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184005dc0 con 0x7f819406b8c0 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.573+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8194154c10 con 0x7f819406b8c0 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.574+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81941576b0 con 0x7f819406b8c0 2023-12-26T21:55:24.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.575+0000 7f817bfff700 1 -- 172.21.15.131:0/4066387183 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8184012800 con 0x7f819406b8c0 2023-12-26T21:55:24.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.576+0000 7f817bfff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8180066f30 0x7f81800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:24.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.576+0000 7f817bfff700 1 -- 172.21.15.131:0/4066387183 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f81840844a0 con 0x7f819406b8c0 2023-12-26T21:55:24.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.576+0000 7f81937fe700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8180066f30 0x7f81800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:24.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.577+0000 7f81937fe700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8180066f30 0x7f81800693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f819406fcc0 tx=0x7f8188016000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:24.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.580+0000 7f817bfff700 1 -- 172.21.15.131:0/4066387183 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8184055370 con 0x7f819406b8c0 2023-12-26T21:55:24.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.847+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f81940657e0 con 0x7f819406b8c0 2023-12-26T21:55:24.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.848+0000 7f817bfff700 1 -- 172.21.15.131:0/4066387183 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8184054f00 con 0x7f819406b8c0 2023-12-26T21:55:24.848 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:24.848 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:55:24.848 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T21:55:24.849 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:55:24.850 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T21:55:24.851 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T21:55:24.852 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.852 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:55:24.852 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:55:24.852 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.852 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.852 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.852 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.852 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T21:55:24.852 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.853 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.854 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.855 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T21:55:24.856 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.857 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T21:55:24.858 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.859 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:55:24.859 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:55:24.859 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:55:24.859 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T21:55:24.860 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:55:24.861 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T21:55:24.862 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:55:24.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8180066f30 msgr2=0x7f81800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:24.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8180066f30 0x7f81800693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f819406fcc0 tx=0x7f8188016000 comp rx=0 tx=0).stop 2023-12-26T21:55:24.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 msgr2=0x7f8194153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:24.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 0x7f8194153900 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f818400d890 tx=0x7f8184008040 comp rx=0 tx=0).stop 2023-12-26T21:55:24.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 shutdown_connections 2023-12-26T21:55:24.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f819413c510 0x7f81941578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8180066f30 0x7f81800693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f819406b8c0 0x7f8194153900 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 --2- 172.21.15.131:0/4066387183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81940cc100 0x7f81940c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:24.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 >> 172.21.15.131:0/4066387183 conn(0x7f81940ce3e0 msgr2=0x7f819413bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:24.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 shutdown_connections 2023-12-26T21:55:24.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:24.850+0000 7f819a3ff700 1 -- 172.21.15.131:0/4066387183 wait complete. 2023-12-26T21:55:24.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:55:25.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.059+0000 7f5932fff700 1 Processor -- start 2023-12-26T21:55:25.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.059+0000 7f5932fff700 1 -- start start 2023-12-26T21:55:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.060+0000 7f5932fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f592c159920 0x7f592c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.060+0000 7f5932fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c0c8bf0 con 0x7f592c159cf0 2023-12-26T21:55:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.060+0000 7f5932fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c0c8e60 con 0x7f592c155be0 2023-12-26T21:55:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.060+0000 7f5932fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0c9140 con 0x7f592c159920 2023-12-26T21:55:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.060+0000 7f5930d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f592c159920 0x7f592c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.061+0000 7f5930d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f592c159920 0x7f592c0b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46544/0 (socket says 172.21.15.131:46544) 2023-12-26T21:55:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.061+0000 7f592bfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f592c159cf0 0x7f592c1554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36412/0 (socket says 172.21.15.131:36412) 2023-12-26T21:55:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.061+0000 7f5930d9b700 1 -- 172.21.15.131:0/1554412041 learned_addr learned my addr 172.21.15.131:0/1554412041 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.061+0000 7f5930d9b700 1 -- 172.21.15.131:0/1554412041 >> v1:172.21.15.190:6789/0 conn(0x7f592c155be0 legacy=0x7f592c0c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.061+0000 7f5930d9b700 1 -- 172.21.15.131:0/1554412041 >> v1:172.21.15.131:6789/0 conn(0x7f592c159cf0 legacy=0x7f592c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.061+0000 7f5930d9b700 1 -- 172.21.15.131:0/1554412041 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c0c9420 con 0x7f592c159920 2023-12-26T21:55:25.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.061+0000 7f592b7fe700 1 -- 172.21.15.131:0/1554412041 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819736411 0 0) 0x7f592c0c8bf0 con 0x7f592c159cf0 2023-12-26T21:55:25.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.061+0000 7f5930d9b700 1 --2- 172.21.15.131:0/1554412041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c159920 0x7f592c0b00e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f592000ab50 tx=0x7f592000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f8b25b712b2ad9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:25.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.061+0000 7f592b7fe700 1 -- 172.21.15.131:0/1554412041 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592000ca60 con 0x7f592c159920 2023-12-26T21:55:25.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.062+0000 7f592b7fe700 1 -- 172.21.15.131:0/1554412041 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592000cbc0 con 0x7f592c159920 2023-12-26T21:55:25.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.062+0000 7f592b7fe700 1 -- 172.21.15.131:0/1554412041 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920018400 con 0x7f592c159920 2023-12-26T21:55:25.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.062+0000 7f5932fff700 1 -- 172.21.15.131:0/1554412041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c159920 msgr2=0x7f592c0b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.062+0000 7f5932fff700 1 --2- 172.21.15.131:0/1554412041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c159920 0x7f592c0b00e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f592000ab50 tx=0x7f592000a3f0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.062+0000 7f5932fff700 1 -- 172.21.15.131:0/1554412041 shutdown_connections 2023-12-26T21:55:25.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.062+0000 7f5932fff700 1 --2- 172.21.15.131:0/1554412041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c159920 0x7f592c0b00e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.062+0000 7f5932fff700 1 -- 172.21.15.131:0/1554412041 >> 172.21.15.131:0/1554412041 conn(0x7f592c068040 msgr2=0x7f592c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:25.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.062+0000 7f5932fff700 1 -- 172.21.15.131:0/1554412041 shutdown_connections 2023-12-26T21:55:25.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.062+0000 7f5932fff700 1 -- 172.21.15.131:0/1554412041 wait complete. 2023-12-26T21:55:25.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.064+0000 7f5932fff700 1 Processor -- start 2023-12-26T21:55:25.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.064+0000 7f5932fff700 1 -- start start 2023-12-26T21:55:25.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5932fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c155be0 0x7f592c145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5932fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f592c159920 0x7f592c06e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5932fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f592c159cf0 0x7f592c06e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5932fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c14c700 con 0x7f592c155be0 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5932fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c14b660 con 0x7f592c159920 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5932fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c14b940 con 0x7f592c159cf0 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5930d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c155be0 0x7f592c145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5930d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c155be0 0x7f592c145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46550/0 (socket says 172.21.15.131:46550) 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5930d9b700 1 -- 172.21.15.131:0/3714211204 learned_addr learned my addr 172.21.15.131:0/3714211204 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5930d9b700 1 -- 172.21.15.131:0/3714211204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f592c159cf0 msgr2=0x7f592c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5930d9b700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f592c159cf0 0x7f592c06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5930d9b700 1 -- 172.21.15.131:0/3714211204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f592c159920 msgr2=0x7f592c06e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5930d9b700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f592c159920 0x7f592c06e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.065+0000 7f5930d9b700 1 -- 172.21.15.131:0/3714211204 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592000a690 con 0x7f592c155be0 2023-12-26T21:55:25.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.066+0000 7f5930d9b700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c155be0 0x7f592c145d20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f592c0688f0 tx=0x7f592001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:25.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.066+0000 7f5928ff9700 1 -- 172.21.15.131:0/3714211204 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920010600 con 0x7f592c155be0 2023-12-26T21:55:25.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.066+0000 7f5928ff9700 1 -- 172.21.15.131:0/3714211204 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200107a0 con 0x7f592c155be0 2023-12-26T21:55:25.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.066+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c14bc20 con 0x7f592c155be0 2023-12-26T21:55:25.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.066+0000 7f5928ff9700 1 -- 172.21.15.131:0/3714211204 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920010b60 con 0x7f592c155be0 2023-12-26T21:55:25.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.066+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c14c1e0 con 0x7f592c155be0 2023-12-26T21:55:25.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.067+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c0657e0 con 0x7f592c155be0 2023-12-26T21:55:25.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.068+0000 7f5928ff9700 1 -- 172.21.15.131:0/3714211204 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5920021420 con 0x7f592c155be0 2023-12-26T21:55:25.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.068+0000 7f5928ff9700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5910066f30 0x7f59100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.069+0000 7f5928ff9700 1 -- 172.21.15.131:0/3714211204 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f5920085a30 con 0x7f592c155be0 2023-12-26T21:55:25.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.069+0000 7f592bfff700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5910066f30 0x7f59100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.069+0000 7f592bfff700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5910066f30 0x7f59100693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f59180011e0 tx=0x7f5918007990 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:25.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.073+0000 7f5928ff9700 1 -- 172.21.15.131:0/3714211204 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5920052e80 con 0x7f592c155be0 2023-12-26T21:55:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.271+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f592c14c3d0 con 0x7f592c155be0 2023-12-26T21:55:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.272+0000 7f5928ff9700 1 -- 172.21.15.131:0/3714211204 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+12788 (secure 0 0 0) 0x7f59200564a0 con 0x7f592c155be0 2023-12-26T21:55:25.273 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:25.273 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":120,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:55:24.343118+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:55:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.274+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5910066f30 msgr2=0x7f59100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.274+0000 7f5932fff700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5910066f30 0x7f59100693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f59180011e0 tx=0x7f5918007990 comp rx=0 tx=0).stop 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c155be0 msgr2=0x7f592c145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c155be0 0x7f592c145d20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f592c0688f0 tx=0x7f592001a000 comp rx=0 tx=0).stop 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 shutdown_connections 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f592c159cf0 0x7f592c06e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5910066f30 0x7f59100693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f592c155be0 0x7f592c145d20 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 --2- 172.21.15.131:0/3714211204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f592c159920 0x7f592c06e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 >> 172.21.15.131:0/3714211204 conn(0x7f592c068040 msgr2=0x7f592c06f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 shutdown_connections 2023-12-26T21:55:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.275+0000 7f5932fff700 1 -- 172.21.15.131:0/3714211204 wait complete. 2023-12-26T21:55:25.294 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T21:55:25.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:55:25.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.486+0000 7f84581ff700 1 Processor -- start 2023-12-26T21:55:25.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.486+0000 7f84581ff700 1 -- start start 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.487+0000 7f84581ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f845013c510 0x7f84500c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.487+0000 7f84581ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 0x7f845006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.487+0000 7f84581ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84500686c0 con 0x7f84500cc100 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.487+0000 7f84581ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8450069520 con 0x7f845013c510 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.487+0000 7f84581ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845006c250 con 0x7f845006b8c0 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.488+0000 7f8455f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f845013c510 0x7f84500c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.488+0000 7f8455f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f845013c510 0x7f84500c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46560/0 (socket says 172.21.15.131:46560) 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.488+0000 7f8455f9b700 1 -- 172.21.15.131:0/3592069031 learned_addr learned my addr 172.21.15.131:0/3592069031 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.488+0000 7f845579a700 1 --2- 172.21.15.131:0/3592069031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 0x7f845006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.488+0000 7f8455f9b700 1 -- 172.21.15.131:0/3592069031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 msgr2=0x7f845006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.488+0000 7f8455f9b700 1 --2- 172.21.15.131:0/3592069031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 0x7f845006bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.488+0000 7f8455f9b700 1 -- 172.21.15.131:0/3592069031 >> v1:172.21.15.190:6789/0 conn(0x7f84500cc100 legacy=0x7f84500cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.488+0000 7f8455f9b700 1 -- 172.21.15.131:0/3592069031 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8450068910 con 0x7f845013c510 2023-12-26T21:55:25.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.488+0000 7f8455f9b700 1 --2- 172.21.15.131:0/3592069031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f845013c510 0x7f84500c8a20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f844c009990 tx=0x7f844c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7aecf628caa2393 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f8454f99700 1 -- 172.21.15.131:0/3592069031 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c005c10 con 0x7f845013c510 2023-12-26T21:55:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f8454f99700 1 -- 172.21.15.131:0/3592069031 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c005d70 con 0x7f845013c510 2023-12-26T21:55:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f8454f99700 1 -- 172.21.15.131:0/3592069031 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c00b880 con 0x7f845013c510 2023-12-26T21:55:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f84581ff700 1 -- 172.21.15.131:0/3592069031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f845013c510 msgr2=0x7f84500c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f84581ff700 1 --2- 172.21.15.131:0/3592069031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f845013c510 0x7f84500c8a20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f844c009990 tx=0x7f844c009220 comp rx=0 tx=0).stop 2023-12-26T21:55:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f84581ff700 1 -- 172.21.15.131:0/3592069031 shutdown_connections 2023-12-26T21:55:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f84581ff700 1 --2- 172.21.15.131:0/3592069031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f845013c510 0x7f84500c8a20 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f84581ff700 1 --2- 172.21.15.131:0/3592069031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 0x7f845006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f84581ff700 1 -- 172.21.15.131:0/3592069031 >> 172.21.15.131:0/3592069031 conn(0x7f84500ce3e0 msgr2=0x7f84500ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:25.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f84581ff700 1 -- 172.21.15.131:0/3592069031 shutdown_connections 2023-12-26T21:55:25.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.489+0000 7f84581ff700 1 -- 172.21.15.131:0/3592069031 wait complete. 2023-12-26T21:55:25.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.491+0000 7f84581ff700 1 Processor -- start 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.491+0000 7f84581ff700 1 -- start start 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f84581ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 0x7f84500b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f84581ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84500cc100 0x7f845015a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f84581ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845013c510 0x7f845015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f84581ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845015dd80 con 0x7f84500cc100 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f84581ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8450153fa0 con 0x7f845013c510 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f84581ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8450159fc0 con 0x7f845006b8c0 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845579a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84500cc100 0x7f845015a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845579a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84500cc100 0x7f845015a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46576/0 (socket says 172.21.15.131:46576) 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845579a700 1 -- 172.21.15.131:0/3038261313 learned_addr learned my addr 172.21.15.131:0/3038261313 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f8455f9b700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 0x7f84500b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845679c700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845013c510 0x7f845015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845579a700 1 -- 172.21.15.131:0/3038261313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 msgr2=0x7f84500b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845579a700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 0x7f84500b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845579a700 1 -- 172.21.15.131:0/3038261313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845013c510 msgr2=0x7f845015e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845579a700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845013c510 0x7f845015e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845579a700 1 -- 172.21.15.131:0/3038261313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c009480 con 0x7f84500cc100 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.492+0000 7f845679c700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845013c510 0x7f845015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.493+0000 7f845579a700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84500cc100 0x7f845015a820 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f844000d860 tx=0x7f8440008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.493+0000 7f8445ffb700 1 -- 172.21.15.131:0/3038261313 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440009610 con 0x7f84500cc100 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.493+0000 7f8445ffb700 1 -- 172.21.15.131:0/3038261313 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8440009770 con 0x7f84500cc100 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.493+0000 7f8445ffb700 1 -- 172.21.15.131:0/3038261313 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440009b30 con 0x7f84500cc100 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.493+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845015a2a0 con 0x7f84500cc100 2023-12-26T21:55:25.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.493+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845015a580 con 0x7f84500cc100 2023-12-26T21:55:25.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.494+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845015def0 con 0x7f84500cc100 2023-12-26T21:55:25.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.495+0000 7f8445ffb700 1 -- 172.21.15.131:0/3038261313 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f844001e460 con 0x7f84500cc100 2023-12-26T21:55:25.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.496+0000 7f8445ffb700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8434066f30 0x7f84340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.496+0000 7f8445ffb700 1 -- 172.21.15.131:0/3038261313 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f84400848c0 con 0x7f84500cc100 2023-12-26T21:55:25.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.496+0000 7f8455f9b700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8434066f30 0x7f84340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.497+0000 7f8455f9b700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8434066f30 0x7f84340693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f844c009260 tx=0x7f844c016000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:25.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.500+0000 7f8445ffb700 1 -- 172.21.15.131:0/3038261313 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f845015def0 con 0x7f84500cc100 2023-12-26T21:55:25.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.690+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f845015dec0 con 0x7f84500cc100 2023-12-26T21:55:25.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.691+0000 7f8445ffb700 1 -- 172.21.15.131:0/3038261313 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8440051d00 con 0x7f84500cc100 2023-12-26T21:55:25.691 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:55:25.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8434066f30 msgr2=0x7f84340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8434066f30 0x7f84340693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f844c009260 tx=0x7f844c016000 comp rx=0 tx=0).stop 2023-12-26T21:55:25.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84500cc100 msgr2=0x7f845015a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84500cc100 0x7f845015a820 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f844000d860 tx=0x7f8440008040 comp rx=0 tx=0).stop 2023-12-26T21:55:25.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 shutdown_connections 2023-12-26T21:55:25.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845013c510 0x7f845015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f845006b8c0 0x7f84500b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8434066f30 0x7f84340693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 --2- 172.21.15.131:0/3038261313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84500cc100 0x7f845015a820 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 >> 172.21.15.131:0/3038261313 conn(0x7f84500ce3e0 msgr2=0x7f84500c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:25.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 shutdown_connections 2023-12-26T21:55:25.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.693+0000 7f84581ff700 1 -- 172.21.15.131:0/3038261313 wait complete. 2023-12-26T21:55:25.711 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T21:55:25.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.899+0000 7efe999ff700 1 Processor -- start 2023-12-26T21:55:25.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.899+0000 7efe999ff700 1 -- start start 2023-12-26T21:55:25.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.900+0000 7efe999ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe94159600 0x7efe940ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.900+0000 7efe999ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efe941599d0 0x7efe9415dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.900+0000 7efe999ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe940cfd50 con 0x7efe940d01e0 2023-12-26T21:55:25.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.900+0000 7efe999ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9415e210 con 0x7efe941599d0 2023-12-26T21:55:25.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.900+0000 7efe999ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940b1280 con 0x7efe94159600 2023-12-26T21:55:25.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.900+0000 7efe927fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efe941599d0 0x7efe9415dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.901+0000 7efe927fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efe941599d0 0x7efe9415dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46586/0 (socket says 172.21.15.131:46586) 2023-12-26T21:55:25.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.901+0000 7efe927fc700 1 -- 172.21.15.131:0/608706849 learned_addr learned my addr 172.21.15.131:0/608706849 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:25.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.901+0000 7efe92ffd700 1 --2- 172.21.15.131:0/608706849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe94159600 0x7efe940ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.901+0000 7efe927fc700 1 -- 172.21.15.131:0/608706849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe94159600 msgr2=0x7efe940ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:25.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.901+0000 7efe927fc700 1 --2- 172.21.15.131:0/608706849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe94159600 0x7efe940ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.901+0000 7efe927fc700 1 -- 172.21.15.131:0/608706849 >> v1:172.21.15.190:6789/0 conn(0x7efe940d01e0 legacy=0x7efe940ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.901+0000 7efe927fc700 1 -- 172.21.15.131:0/608706849 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940cf140 con 0x7efe941599d0 2023-12-26T21:55:25.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.901+0000 7efe92ffd700 1 --2- 172.21.15.131:0/608706849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe94159600 0x7efe940ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:25.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.901+0000 7efe927fc700 1 --2- 172.21.15.131:0/608706849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe941599d0 0x7efe9415dcd0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7efe880099b0 tx=0x7efe88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9261cdd5dc9c638e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:25.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.903+0000 7efe91ffb700 1 -- 172.21.15.131:0/608706849 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800b860 con 0x7efe941599d0 2023-12-26T21:55:25.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.903+0000 7efe91ffb700 1 -- 172.21.15.131:0/608706849 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8800b9c0 con 0x7efe941599d0 2023-12-26T21:55:25.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.903+0000 7efe91ffb700 1 -- 172.21.15.131:0/608706849 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800bd60 con 0x7efe941599d0 2023-12-26T21:55:25.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.904+0000 7efe999ff700 1 -- 172.21.15.131:0/608706849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe941599d0 msgr2=0x7efe9415dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.904+0000 7efe999ff700 1 --2- 172.21.15.131:0/608706849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe941599d0 0x7efe9415dcd0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7efe880099b0 tx=0x7efe88009250 comp rx=0 tx=0).stop 2023-12-26T21:55:25.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.904+0000 7efe999ff700 1 -- 172.21.15.131:0/608706849 shutdown_connections 2023-12-26T21:55:25.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.904+0000 7efe999ff700 1 --2- 172.21.15.131:0/608706849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe94159600 0x7efe940ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.904+0000 7efe999ff700 1 --2- 172.21.15.131:0/608706849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe941599d0 0x7efe9415dcd0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.904+0000 7efe999ff700 1 -- 172.21.15.131:0/608706849 >> 172.21.15.131:0/608706849 conn(0x7efe9406ece0 msgr2=0x7efe9406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:25.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.904+0000 7efe999ff700 1 -- 172.21.15.131:0/608706849 shutdown_connections 2023-12-26T21:55:25.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.904+0000 7efe999ff700 1 -- 172.21.15.131:0/608706849 wait complete. 2023-12-26T21:55:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.905+0000 7efe999ff700 1 Processor -- start 2023-12-26T21:55:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.905+0000 7efe999ff700 1 -- start start 2023-12-26T21:55:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.906+0000 7efe999ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe940d01e0 0x7efe940b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.906+0000 7efe999ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe94159600 0x7efe941551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.906+0000 7efe999ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe9415a410 0x7efe9415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.906+0000 7efe999ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940b5d40 con 0x7efe9415a410 2023-12-26T21:55:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.906+0000 7efe999ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940b5130 con 0x7efe94159600 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.906+0000 7efe999ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940b5410 con 0x7efe940d01e0 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe937fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe9415a410 0x7efe9415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe937fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe9415a410 0x7efe9415a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46602/0 (socket says 172.21.15.131:46602) 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe937fe700 1 -- 172.21.15.131:0/2635685087 learned_addr learned my addr 172.21.15.131:0/2635685087 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe927fc700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe94159600 0x7efe941551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe92ffd700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe940d01e0 0x7efe940b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe937fe700 1 -- 172.21.15.131:0/2635685087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe940d01e0 msgr2=0x7efe940b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe937fe700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe940d01e0 0x7efe940b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe937fe700 1 -- 172.21.15.131:0/2635685087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe94159600 msgr2=0x7efe941551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe937fe700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe94159600 0x7efe941551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe937fe700 1 -- 172.21.15.131:0/2635685087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880094f0 con 0x7efe9415a410 2023-12-26T21:55:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.907+0000 7efe937fe700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe9415a410 0x7efe9415a9a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c910 tx=0x7efe8c0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.908+0000 7efe7affd700 1 -- 172.21.15.131:0/2635685087 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c012630 con 0x7efe9415a410 2023-12-26T21:55:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.908+0000 7efe7affd700 1 -- 172.21.15.131:0/2635685087 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c0127b0 con 0x7efe9415a410 2023-12-26T21:55:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.908+0000 7efe7affd700 1 -- 172.21.15.131:0/2635685087 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c012b70 con 0x7efe9415a410 2023-12-26T21:55:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.908+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe940b56f0 con 0x7efe9415a410 2023-12-26T21:55:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.908+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe940b38d0 con 0x7efe9415a410 2023-12-26T21:55:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.909+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe940b5eb0 con 0x7efe9415a410 2023-12-26T21:55:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.910+0000 7efe7affd700 1 -- 172.21.15.131:0/2635685087 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efe8c003030 con 0x7efe9415a410 2023-12-26T21:55:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.910+0000 7efe7affd700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efe80066f30 0x7efe800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.911+0000 7efe7affd700 1 -- 172.21.15.131:0/2635685087 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5245+0+0 (secure 0 0 0) 0x7efe8c0841b0 con 0x7efe9415a410 2023-12-26T21:55:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.911+0000 7efe92ffd700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efe80066f30 0x7efe800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.911+0000 7efe92ffd700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efe80066f30 0x7efe800693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe84006ea0 tx=0x7efe84008040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:25.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:25.915+0000 7efe7affd700 1 -- 172.21.15.131:0/2635685087 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe8c0515f0 con 0x7efe9415a410 2023-12-26T21:55:26.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.109+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efe940657e0 con 0x7efe9415a410 2023-12-26T21:55:26.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.109+0000 7efe7affd700 1 -- 172.21.15.131:0/2635685087 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v120) v1 ==== 84+0+65 (secure 0 0 0) 0x7efe8c054c10 con 0x7efe9415a410 2023-12-26T21:55:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:26.110 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:55:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T21:55:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T21:55:26.110 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:26.110 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:55:26.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.111+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efe80066f30 msgr2=0x7efe800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.111+0000 7efe999ff700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efe80066f30 0x7efe800693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe84006ea0 tx=0x7efe84008040 comp rx=0 tx=0).stop 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.111+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe9415a410 msgr2=0x7efe9415a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.111+0000 7efe999ff700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe9415a410 0x7efe9415a9a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c910 tx=0x7efe8c0016f0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.112+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 shutdown_connections 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.112+0000 7efe999ff700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efe80066f30 0x7efe800693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.112+0000 7efe999ff700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe9415a410 0x7efe9415a9a0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.112+0000 7efe999ff700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe940d01e0 0x7efe940b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.112+0000 7efe999ff700 1 --2- 172.21.15.131:0/2635685087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe94159600 0x7efe941551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.112+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 >> 172.21.15.131:0/2635685087 conn(0x7efe9406ece0 msgr2=0x7efe94156130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.112+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 shutdown_connections 2023-12-26T21:55:26.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.112+0000 7efe999ff700 1 -- 172.21.15.131:0/2635685087 wait complete. 2023-12-26T21:55:26.130 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T21:55:26.130 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T21:55:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.322+0000 7f87d6bff700 1 Processor -- start 2023-12-26T21:55:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.322+0000 7f87d6bff700 1 -- start start 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.323+0000 7f87d6bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f87d0142f00 0x7f87d0069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.323+0000 7f87d6bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87d00cc590 0x7f87d00cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.323+0000 7f87d6bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d0069640 con 0x7f87d00cb340 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.323+0000 7f87d6bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d00cb7c0 con 0x7f87d00cc590 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.323+0000 7f87d6bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d0068750 con 0x7f87d0142f00 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.324+0000 7f87cffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87d00cc590 0x7f87d00cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.324+0000 7f87cffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87d00cc590 0x7f87d00cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46608/0 (socket says 172.21.15.131:46608) 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.324+0000 7f87cffff700 1 -- 172.21.15.131:0/1327342074 learned_addr learned my addr 172.21.15.131:0/1327342074 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.324+0000 7f87d499b700 1 --2- 172.21.15.131:0/1327342074 >> v2:172.21.15.190:3300/0 conn(0x7f87d0142f00 0x7f87d0069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.324+0000 7f87cffff700 1 -- 172.21.15.131:0/1327342074 >> v2:172.21.15.190:3300/0 conn(0x7f87d0142f00 msgr2=0x7f87d0069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.324+0000 7f87cffff700 1 --2- 172.21.15.131:0/1327342074 >> v2:172.21.15.190:3300/0 conn(0x7f87d0142f00 0x7f87d0069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.324+0000 7f87cffff700 1 -- 172.21.15.131:0/1327342074 >> v1:172.21.15.190:6789/0 conn(0x7f87d00cb340 legacy=0x7f87d00c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.324+0000 7f87cffff700 1 -- 172.21.15.131:0/1327342074 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0068a30 con 0x7f87d00cc590 2023-12-26T21:55:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.324+0000 7f87cffff700 1 --2- 172.21.15.131:0/1327342074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d00cc590 0x7f87d00cae00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f87d006f5b0 tx=0x7f87bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6041b15370bc65f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87cf7fe700 1 -- 172.21.15.131:0/1327342074 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc009e70 con 0x7f87d00cc590 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87cf7fe700 1 -- 172.21.15.131:0/1327342074 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc00b860 con 0x7f87d00cc590 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87cf7fe700 1 -- 172.21.15.131:0/1327342074 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc00bc00 con 0x7f87d00cc590 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87d6bff700 1 -- 172.21.15.131:0/1327342074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d00cc590 msgr2=0x7f87d00cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87d6bff700 1 --2- 172.21.15.131:0/1327342074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d00cc590 0x7f87d00cae00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f87d006f5b0 tx=0x7f87bc009250 comp rx=0 tx=0).stop 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87d6bff700 1 -- 172.21.15.131:0/1327342074 shutdown_connections 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87d6bff700 1 --2- 172.21.15.131:0/1327342074 >> v2:172.21.15.190:3300/0 conn(0x7f87d0142f00 0x7f87d0069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87d6bff700 1 --2- 172.21.15.131:0/1327342074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d00cc590 0x7f87d00cae00 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87d6bff700 1 -- 172.21.15.131:0/1327342074 >> 172.21.15.131:0/1327342074 conn(0x7f87d0063cd0 msgr2=0x7f87d006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87d6bff700 1 -- 172.21.15.131:0/1327342074 shutdown_connections 2023-12-26T21:55:26.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.325+0000 7f87d6bff700 1 -- 172.21.15.131:0/1327342074 wait complete. 2023-12-26T21:55:26.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.327+0000 7f87d6bff700 1 Processor -- start 2023-12-26T21:55:26.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.328+0000 7f87d6bff700 1 -- start start 2023-12-26T21:55:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d6bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d00cb340 0x7f87d00c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d6bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d00cc590 0x7f87d013fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d6bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d0142f00 0x7f87d00b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d6bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00c63e0 con 0x7f87d0142f00 2023-12-26T21:55:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d6bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d0140410 con 0x7f87d00cc590 2023-12-26T21:55:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d6bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d015e570 con 0x7f87d00cb340 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d519c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d0142f00 0x7f87d00b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d519c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d0142f00 0x7f87d00b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46614/0 (socket says 172.21.15.131:46614) 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d519c700 1 -- 172.21.15.131:0/4244679663 learned_addr learned my addr 172.21.15.131:0/4244679663 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87cffff700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d00cc590 0x7f87d013fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.329+0000 7f87d499b700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d00cb340 0x7f87d00c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87d519c700 1 -- 172.21.15.131:0/4244679663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d00cb340 msgr2=0x7f87d00c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87d519c700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d00cb340 0x7f87d00c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87d519c700 1 -- 172.21.15.131:0/4244679663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d00cc590 msgr2=0x7f87d013fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87d519c700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d00cc590 0x7f87d013fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87d519c700 1 -- 172.21.15.131:0/4244679663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0094f0 con 0x7f87d0142f00 2023-12-26T21:55:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87d519c700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d0142f00 0x7f87d00b2eb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c8e0 tx=0x7f87c8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87ccff9700 1 -- 172.21.15.131:0/4244679663 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8012630 con 0x7f87d0142f00 2023-12-26T21:55:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87ccff9700 1 -- 172.21.15.131:0/4244679663 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8012790 con 0x7f87d0142f00 2023-12-26T21:55:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87ccff9700 1 -- 172.21.15.131:0/4244679663 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8012b50 con 0x7f87d0142f00 2023-12-26T21:55:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d00c5b40 con 0x7f87d0142f00 2023-12-26T21:55:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.330+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d00c6100 con 0x7f87d0142f00 2023-12-26T21:55:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.331+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d006c170 con 0x7f87d0142f00 2023-12-26T21:55:26.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.332+0000 7f87ccff9700 1 -- 172.21.15.131:0/4244679663 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f87c801a3a0 con 0x7f87d0142f00 2023-12-26T21:55:26.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.333+0000 7f87ccff9700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87b4066f30 0x7f87b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:26.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.333+0000 7f87ccff9700 1 -- 172.21.15.131:0/4244679663 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f87c8084610 con 0x7f87d0142f00 2023-12-26T21:55:26.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.333+0000 7f87d499b700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87b4066f30 0x7f87b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:26.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.334+0000 7f87d499b700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87b4066f30 0x7f87b40693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009d10 tx=0x7f87c40091a0 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:26.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.337+0000 7f87ccff9700 1 -- 172.21.15.131:0/4244679663 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87c80519b0 con 0x7f87d0142f00 2023-12-26T21:55:26.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.529+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f87d00717f0 con 0x7f87d0142f00 2023-12-26T21:55:26.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.910+0000 7f87ccff9700 1 -- 172.21.15.131:0/4244679663 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v121) v1 ==== 138+0+0 (secure 0 0 0) 0x7f87c8054fd0 con 0x7f87d0142f00 2023-12-26T21:55:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87b4066f30 msgr2=0x7f87b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87b4066f30 0x7f87b40693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009d10 tx=0x7f87c40091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d0142f00 msgr2=0x7f87d00b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d0142f00 0x7f87d00b2eb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c8e0 tx=0x7f87c8001710 comp rx=0 tx=0).stop 2023-12-26T21:55:26.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 shutdown_connections 2023-12-26T21:55:26.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f87b4066f30 0x7f87b40693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87d0142f00 0x7f87d00b2eb0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d00cb340 0x7f87d00c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 --2- 172.21.15.131:0/4244679663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d00cc590 0x7f87d013fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:26.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 >> 172.21.15.131:0/4244679663 conn(0x7f87d0063cd0 msgr2=0x7f87d015ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:26.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 shutdown_connections 2023-12-26T21:55:26.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:26.914+0000 7f87d6bff700 1 -- 172.21.15.131:0/4244679663 wait complete. 2023-12-26T21:55:26.916 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T21:55:26.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T21:55:27.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.129+0000 7efdbd5ff700 1 Processor -- start 2023-12-26T21:55:27.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.129+0000 7efdbd5ff700 1 -- start start 2023-12-26T21:55:27.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.130+0000 7efdbd5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efdb8156450 0x7efdb80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:27.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.130+0000 7efdbd5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb80c85c0 con 0x7efdb80c7110 2023-12-26T21:55:27.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.130+0000 7efdbd5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb80c79b0 con 0x7efdb806edf0 2023-12-26T21:55:27.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.130+0000 7efdbd5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb80c7c90 con 0x7efdb8156450 2023-12-26T21:55:27.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.130+0000 7efdb6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efdb8156450 0x7efdb80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:27.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.131+0000 7efdb67fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efdb80c7110 0x7efdb806e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36424/0 (socket says 172.21.15.131:36424) 2023-12-26T21:55:27.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.131+0000 7efdb6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efdb8156450 0x7efdb80cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46620/0 (socket says 172.21.15.131:46620) 2023-12-26T21:55:27.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.131+0000 7efdb67fc700 1 -- 172.21.15.131:0/3858662894 learned_addr learned my addr 172.21.15.131:0/3858662894 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:27.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.131+0000 7efdb6ffd700 1 -- 172.21.15.131:0/3858662894 >> v1:172.21.15.190:6789/0 conn(0x7efdb806edf0 legacy=0x7efdb80acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:27.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.131+0000 7efdb6ffd700 1 -- 172.21.15.131:0/3858662894 >> v1:172.21.15.131:6789/0 conn(0x7efdb80c7110 legacy=0x7efdb806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:27.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.131+0000 7efdb6ffd700 1 -- 172.21.15.131:0/3858662894 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80c7f70 con 0x7efdb8156450 2023-12-26T21:55:27.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.131+0000 7efdb5ffb700 1 -- 172.21.15.131:0/3858662894 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488405246 0 0) 0x7efdb80c85c0 con 0x7efdb80c7110 2023-12-26T21:55:27.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.131+0000 7efdb6ffd700 1 --2- 172.21.15.131:0/3858662894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 0x7efdb80cccb0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7efda800bb90 tx=0x7efda800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563253c42b6b2351 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:27.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.131+0000 7efdb5ffb700 1 -- 172.21.15.131:0/3858662894 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda800dba0 con 0x7efdb8156450 2023-12-26T21:55:27.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.132+0000 7efdb5ffb700 1 -- 172.21.15.131:0/3858662894 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda800dd00 con 0x7efdb8156450 2023-12-26T21:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.132+0000 7efdb5ffb700 1 -- 172.21.15.131:0/3858662894 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda80196b0 con 0x7efdb8156450 2023-12-26T21:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.132+0000 7efdbd5ff700 1 -- 172.21.15.131:0/3858662894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 msgr2=0x7efdb80cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.132+0000 7efdbd5ff700 1 --2- 172.21.15.131:0/3858662894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 0x7efdb80cccb0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7efda800bb90 tx=0x7efda800b430 comp rx=0 tx=0).stop 2023-12-26T21:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.132+0000 7efdbd5ff700 1 -- 172.21.15.131:0/3858662894 shutdown_connections 2023-12-26T21:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.132+0000 7efdbd5ff700 1 --2- 172.21.15.131:0/3858662894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 0x7efdb80cccb0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.132+0000 7efdbd5ff700 1 -- 172.21.15.131:0/3858662894 >> 172.21.15.131:0/3858662894 conn(0x7efdb8068040 msgr2=0x7efdb80cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.132+0000 7efdbd5ff700 1 -- 172.21.15.131:0/3858662894 shutdown_connections 2023-12-26T21:55:27.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.132+0000 7efdbd5ff700 1 -- 172.21.15.131:0/3858662894 wait complete. 2023-12-26T21:55:27.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.134+0000 7efdbd5ff700 1 Processor -- start 2023-12-26T21:55:27.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.134+0000 7efdbd5ff700 1 -- start start 2023-12-26T21:55:27.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdbd5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdb806edf0 0x7efdb80c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:27.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdbd5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb80c7110 0x7efdb8153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:27.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdbd5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 0x7efdb80caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:27.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdbd5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80c38b0 con 0x7efdb8156450 2023-12-26T21:55:27.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdbd5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb816a2a0 con 0x7efdb80c7110 2023-12-26T21:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdbd5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb816a580 con 0x7efdb806edf0 2023-12-26T21:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 0x7efdb80caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 0x7efdb80caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46624/0 (socket says 172.21.15.131:46624) 2023-12-26T21:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb77fe700 1 -- 172.21.15.131:0/4238385598 learned_addr learned my addr 172.21.15.131:0/4238385598 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb6ffd700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdb806edf0 0x7efdb80c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb67fc700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb80c7110 0x7efdb8153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb77fe700 1 -- 172.21.15.131:0/4238385598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdb806edf0 msgr2=0x7efdb80c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb77fe700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdb806edf0 0x7efdb80c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:27.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb77fe700 1 -- 172.21.15.131:0/4238385598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb80c7110 msgr2=0x7efdb8153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb77fe700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb80c7110 0x7efdb8153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.135+0000 7efdb77fe700 1 -- 172.21.15.131:0/4238385598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda800b6d0 con 0x7efdb8156450 2023-12-26T21:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.136+0000 7efdb77fe700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 0x7efdb80caee0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c910 tx=0x7efdb00016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.136+0000 7efd9effd700 1 -- 172.21.15.131:0/4238385598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb000a7e0 con 0x7efdb8156450 2023-12-26T21:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.136+0000 7efd9effd700 1 -- 172.21.15.131:0/4238385598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb000a940 con 0x7efdb8156450 2023-12-26T21:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.136+0000 7efd9effd700 1 -- 172.21.15.131:0/4238385598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb000ace0 con 0x7efdb8156450 2023-12-26T21:55:27.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.136+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb80c3260 con 0x7efdb8156450 2023-12-26T21:55:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.136+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb80c4280 con 0x7efdb8156450 2023-12-26T21:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.137+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb80c3b90 con 0x7efdb8156450 2023-12-26T21:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.138+0000 7efd9effd700 1 -- 172.21.15.131:0/4238385598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efdb0012630 con 0x7efdb8156450 2023-12-26T21:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.140+0000 7efd9effd700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efda4066f30 0x7efda40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.140+0000 7efd9effd700 1 -- 172.21.15.131:0/4238385598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5615+0+0 (secure 0 0 0) 0x7efdb0084ee0 con 0x7efdb8156450 2023-12-26T21:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.140+0000 7efdb6ffd700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efda4066f30 0x7efda40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.141+0000 7efdb6ffd700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efda4066f30 0x7efda40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efda800b2c0 tx=0x7efda801b000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:27.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.143+0000 7efd9effd700 1 -- 172.21.15.131:0/4238385598 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdb0052230 con 0x7efdb8156450 2023-12-26T21:55:27.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.336+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efdb80657e0 con 0x7efdb8156450 2023-12-26T21:55:27.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.917+0000 7efd9effd700 1 -- 172.21.15.131:0/4238385598 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v122) v1 ==== 170+0+0 (secure 0 0 0) 0x7efdb0055850 con 0x7efdb8156450 2023-12-26T21:55:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.922+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efda4066f30 msgr2=0x7efda40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.922+0000 7efdbd5ff700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efda4066f30 0x7efda40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efda800b2c0 tx=0x7efda801b000 comp rx=0 tx=0).stop 2023-12-26T21:55:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.922+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 msgr2=0x7efdb80caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:27.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.922+0000 7efdbd5ff700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 0x7efdb80caee0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c910 tx=0x7efdb00016b0 comp rx=0 tx=0).stop 2023-12-26T21:55:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.924+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 shutdown_connections 2023-12-26T21:55:27.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.924+0000 7efdbd5ff700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efda4066f30 0x7efda40693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:27.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.924+0000 7efdbd5ff700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdb8156450 0x7efdb80caee0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:27.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.924+0000 7efdbd5ff700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdb806edf0 0x7efdb80c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:27.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.924+0000 7efdbd5ff700 1 --2- 172.21.15.131:0/4238385598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdb80c7110 0x7efdb8153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:27.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.924+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 >> 172.21.15.131:0/4238385598 conn(0x7efdb8068040 msgr2=0x7efdb8155690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:27.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.924+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 shutdown_connections 2023-12-26T21:55:27.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:27.925+0000 7efdbd5ff700 1 -- 172.21.15.131:0/4238385598 wait complete. 2023-12-26T21:55:27.933 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T21:55:27.954 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T21:55:28.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.146+0000 7fd1179ff700 1 Processor -- start 2023-12-26T21:55:28.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.146+0000 7fd1179ff700 1 -- start start 2023-12-26T21:55:28.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.147+0000 7fd1179ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1101596a0 0x7fd1100b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:28.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.147+0000 7fd1179ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd110159a70 0x7fd1100b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:28.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.147+0000 7fd1179ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 0x7fd1100b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:28.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.147+0000 7fd1179ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1100afcd0 con 0x7fd110159a70 2023-12-26T21:55:28.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.147+0000 7fd1179ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1100b3e70 con 0x7fd1101596a0 2023-12-26T21:55:28.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.147+0000 7fd1179ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1100b2e00 con 0x7fd1100b0160 2023-12-26T21:55:28.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd114f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd110159a70 0x7fd1100b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:28.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd114f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd110159a70 0x7fd1100b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46626/0 (socket says 172.21.15.131:46626) 2023-12-26T21:55:28.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd114f9a700 1 -- 172.21.15.131:0/3398174249 learned_addr learned my addr 172.21.15.131:0/3398174249 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:28.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd115f9c700 1 --2- 172.21.15.131:0/3398174249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 0x7fd1100b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:28.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd11579b700 1 --2- 172.21.15.131:0/3398174249 >> v2:172.21.15.190:3300/0 conn(0x7fd1101596a0 0x7fd1100b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd114f9a700 1 -- 172.21.15.131:0/3398174249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 msgr2=0x7fd1100b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd114f9a700 1 --2- 172.21.15.131:0/3398174249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 0x7fd1100b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd114f9a700 1 -- 172.21.15.131:0/3398174249 >> v2:172.21.15.190:3300/0 conn(0x7fd1101596a0 msgr2=0x7fd1100b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd114f9a700 1 --2- 172.21.15.131:0/3398174249 >> v2:172.21.15.190:3300/0 conn(0x7fd1101596a0 0x7fd1100b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.148+0000 7fd114f9a700 1 -- 172.21.15.131:0/3398174249 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1100b0f20 con 0x7fd110159a70 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.149+0000 7fd114f9a700 1 --2- 172.21.15.131:0/3398174249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd110159a70 0x7fd1100b28c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd1100892a0 tx=0x7fd100009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6591abcee70b9cc server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.149+0000 7fd107fff700 1 -- 172.21.15.131:0/3398174249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10000b640 con 0x7fd110159a70 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.149+0000 7fd107fff700 1 -- 172.21.15.131:0/3398174249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10000b7a0 con 0x7fd110159a70 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.149+0000 7fd107fff700 1 -- 172.21.15.131:0/3398174249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10000bb60 con 0x7fd110159a70 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.149+0000 7fd1179ff700 1 -- 172.21.15.131:0/3398174249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd110159a70 msgr2=0x7fd1100b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.149+0000 7fd1179ff700 1 --2- 172.21.15.131:0/3398174249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd110159a70 0x7fd1100b28c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd1100892a0 tx=0x7fd100009250 comp rx=0 tx=0).stop 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.150+0000 7fd1179ff700 1 -- 172.21.15.131:0/3398174249 shutdown_connections 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.150+0000 7fd1179ff700 1 --2- 172.21.15.131:0/3398174249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 0x7fd1100b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.150+0000 7fd1179ff700 1 --2- 172.21.15.131:0/3398174249 >> v2:172.21.15.190:3300/0 conn(0x7fd1101596a0 0x7fd1100b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.150+0000 7fd1179ff700 1 --2- 172.21.15.131:0/3398174249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd110159a70 0x7fd1100b28c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.150+0000 7fd1179ff700 1 -- 172.21.15.131:0/3398174249 >> 172.21.15.131:0/3398174249 conn(0x7fd11006ece0 msgr2=0x7fd11006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.150+0000 7fd1179ff700 1 -- 172.21.15.131:0/3398174249 shutdown_connections 2023-12-26T21:55:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.150+0000 7fd1179ff700 1 -- 172.21.15.131:0/3398174249 wait complete. 2023-12-26T21:55:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.151+0000 7fd1179ff700 1 Processor -- start 2023-12-26T21:55:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.151+0000 7fd1179ff700 1 -- start start 2023-12-26T21:55:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.152+0000 7fd1179ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 0x7fd11015b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.152+0000 7fd1179ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1101596a0 0x7fd110156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.152+0000 7fd1179ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd110159a70 0x7fd110159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.152+0000 7fd1179ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11015b2a0 con 0x7fd1101596a0 2023-12-26T21:55:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.152+0000 7fd1179ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110157080 con 0x7fd110159a70 2023-12-26T21:55:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.152+0000 7fd1179ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11015a690 con 0x7fd1100b0160 2023-12-26T21:55:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.152+0000 7fd114f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1101596a0 0x7fd110156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:28.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.152+0000 7fd114f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1101596a0 0x7fd110156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46634/0 (socket says 172.21.15.131:46634) 2023-12-26T21:55:28.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd115f9c700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd110159a70 0x7fd110159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:28.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd114f9a700 1 -- 172.21.15.131:0/50882702 learned_addr learned my addr 172.21.15.131:0/50882702 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:28.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd11579b700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 0x7fd11015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:28.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd114f9a700 1 -- 172.21.15.131:0/50882702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 msgr2=0x7fd11015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:28.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd114f9a700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 0x7fd11015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd114f9a700 1 -- 172.21.15.131:0/50882702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd110159a70 msgr2=0x7fd110159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:28.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd114f9a700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd110159a70 0x7fd110159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd114f9a700 1 -- 172.21.15.131:0/50882702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000094f0 con 0x7fd1101596a0 2023-12-26T21:55:28.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd115f9c700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd110159a70 0x7fd110159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:28.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd114f9a700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1101596a0 0x7fd110156b40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd110089710 tx=0x7fd100017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:28.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.153+0000 7fd1057fa700 1 -- 172.21.15.131:0/50882702 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10000b910 con 0x7fd1101596a0 2023-12-26T21:55:28.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.154+0000 7fd1057fa700 1 -- 172.21.15.131:0/50882702 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10000ba70 con 0x7fd1101596a0 2023-12-26T21:55:28.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.154+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11015a970 con 0x7fd1101596a0 2023-12-26T21:55:28.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.154+0000 7fd1057fa700 1 -- 172.21.15.131:0/50882702 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100011630 con 0x7fd1101596a0 2023-12-26T21:55:28.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.154+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11015af30 con 0x7fd1101596a0 2023-12-26T21:55:28.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.155+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1100b29a0 con 0x7fd1101596a0 2023-12-26T21:55:28.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.156+0000 7fd1057fa700 1 -- 172.21.15.131:0/50882702 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd1000119a0 con 0x7fd1101596a0 2023-12-26T21:55:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.157+0000 7fd1057fa700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd0f4066f30 0x7fd0f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.157+0000 7fd1057fa700 1 -- 172.21.15.131:0/50882702 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fd100086490 con 0x7fd1101596a0 2023-12-26T21:55:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.157+0000 7fd11579b700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd0f4066f30 0x7fd0f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.158+0000 7fd11579b700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd0f4066f30 0x7fd0f40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009d90 tx=0x7fd10c0091a0 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:28.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.161+0000 7fd1057fa700 1 -- 172.21.15.131:0/50882702 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd100053660 con 0x7fd1101596a0 2023-12-26T21:55:28.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.394+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd1100657e0 con 0x7fd1101596a0 2023-12-26T21:55:28.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.920+0000 7fd1057fa700 1 -- 172.21.15.131:0/50882702 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd100056c80 con 0x7fd1101596a0 2023-12-26T21:55:28.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd0f4066f30 msgr2=0x7fd0f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd0f4066f30 0x7fd0f40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009d90 tx=0x7fd10c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1101596a0 msgr2=0x7fd110156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1101596a0 0x7fd110156b40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd110089710 tx=0x7fd100017000 comp rx=0 tx=0).stop 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 shutdown_connections 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd110159a70 0x7fd110159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1100b0160 0x7fd11015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd0f4066f30 0x7fd0f40693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 --2- 172.21.15.131:0/50882702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1101596a0 0x7fd110156b40 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 >> 172.21.15.131:0/50882702 conn(0x7fd11006ece0 msgr2=0x7fd110158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 shutdown_connections 2023-12-26T21:55:28.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:28.924+0000 7fd1179ff700 1 -- 172.21.15.131:0/50882702 wait complete. 2023-12-26T21:55:28.930 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 14 and data pool 15 2023-12-26T21:55:28.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:55:29.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.151+0000 7f87a5bff700 1 Processor -- start 2023-12-26T21:55:29.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.151+0000 7f87a5bff700 1 -- start start 2023-12-26T21:55:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f87a5bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87a00cb320 0x7f87a0069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f87a5bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f87a006a1e0 0x7f87a00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f87a5bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a0156940 con 0x7f87a00b0c90 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f87a5bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a00c8cc0 con 0x7f87a00cb320 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f87a5bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a00ad700 con 0x7f87a006a1e0 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f879f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87a00cb320 0x7f87a0069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f879f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87a00cb320 0x7f87a0069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46642/0 (socket says 172.21.15.131:46642) 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f879f7fe700 1 -- 172.21.15.131:0/4102862338 learned_addr learned my addr 172.21.15.131:0/4102862338 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f879effd700 1 --2- 172.21.15.131:0/4102862338 >> v2:172.21.15.190:3300/0 conn(0x7f87a006a1e0 0x7f87a00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f879f7fe700 1 -- 172.21.15.131:0/4102862338 >> v2:172.21.15.190:3300/0 conn(0x7f87a006a1e0 msgr2=0x7f87a00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f879f7fe700 1 --2- 172.21.15.131:0/4102862338 >> v2:172.21.15.190:3300/0 conn(0x7f87a006a1e0 0x7f87a00b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f879f7fe700 1 -- 172.21.15.131:0/4102862338 >> v1:172.21.15.190:6789/0 conn(0x7f87a00b0c90 legacy=0x7f87a00aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f879f7fe700 1 -- 172.21.15.131:0/4102862338 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00ae500 con 0x7f87a00cb320 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.153+0000 7f879effd700 1 --2- 172.21.15.131:0/4102862338 >> v2:172.21.15.190:3300/0 conn(0x7f87a006a1e0 0x7f87a00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.154+0000 7f879f7fe700 1 --2- 172.21.15.131:0/4102862338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 0x7f87a0069ca0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f87a0064700 tx=0x7f8790009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a7856071ada8b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.154+0000 7f879e7fc700 1 -- 172.21.15.131:0/4102862338 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879000b7e0 con 0x7f87a00cb320 2023-12-26T21:55:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.154+0000 7f879e7fc700 1 -- 172.21.15.131:0/4102862338 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879000b940 con 0x7f87a00cb320 2023-12-26T21:55:29.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.154+0000 7f879e7fc700 1 -- 172.21.15.131:0/4102862338 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879000bce0 con 0x7f87a00cb320 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.154+0000 7f87a5bff700 1 -- 172.21.15.131:0/4102862338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 msgr2=0x7f87a0069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.154+0000 7f87a5bff700 1 --2- 172.21.15.131:0/4102862338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 0x7f87a0069ca0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f87a0064700 tx=0x7f8790009250 comp rx=0 tx=0).stop 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.155+0000 7f87a5bff700 1 -- 172.21.15.131:0/4102862338 shutdown_connections 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.155+0000 7f87a5bff700 1 --2- 172.21.15.131:0/4102862338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 0x7f87a0069ca0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.155+0000 7f87a5bff700 1 --2- 172.21.15.131:0/4102862338 >> v2:172.21.15.190:3300/0 conn(0x7f87a006a1e0 0x7f87a00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.155+0000 7f87a5bff700 1 -- 172.21.15.131:0/4102862338 >> 172.21.15.131:0/4102862338 conn(0x7f87a0068040 msgr2=0x7f87a0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.155+0000 7f87a5bff700 1 -- 172.21.15.131:0/4102862338 shutdown_connections 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.155+0000 7f87a5bff700 1 -- 172.21.15.131:0/4102862338 wait complete. 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.156+0000 7f87a5bff700 1 Processor -- start 2023-12-26T21:55:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.156+0000 7f87a5bff700 1 -- start start 2023-12-26T21:55:29.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.157+0000 7f87a5bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a006a1e0 0x7f87a00c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.157+0000 7f87a5bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a00b0c90 0x7f87a016d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.157+0000 7f87a5bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 0x7f87a014beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.157+0000 7f87a5bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00c7970 con 0x7f87a00cb320 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.157+0000 7f87a5bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a016de60 con 0x7f87a00b0c90 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.157+0000 7f87a5bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015a900 con 0x7f87a006a1e0 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.157+0000 7f879ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 0x7f87a014beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.157+0000 7f879ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 0x7f87a014beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46650/0 (socket says 172.21.15.131:46650) 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.157+0000 7f879ffff700 1 -- 172.21.15.131:0/3204641509 learned_addr learned my addr 172.21.15.131:0/3204641509 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f879effd700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a00b0c90 0x7f87a016d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f879f7fe700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a006a1e0 0x7f87a00c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f879ffff700 1 -- 172.21.15.131:0/3204641509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a006a1e0 msgr2=0x7f87a00c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f879ffff700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a006a1e0 0x7f87a00c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f879ffff700 1 -- 172.21.15.131:0/3204641509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a00b0c90 msgr2=0x7f87a016d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f879ffff700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a00b0c90 0x7f87a016d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f879ffff700 1 -- 172.21.15.131:0/3204641509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87900094f0 con 0x7f87a00cb320 2023-12-26T21:55:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f879f7fe700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a006a1e0 0x7f87a00c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f879ffff700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 0x7f87a014beb0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f879800c960 tx=0x7f87980026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f87877fe700 1 -- 172.21.15.131:0/3204641509 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800cea0 con 0x7f87a00cb320 2023-12-26T21:55:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f87877fe700 1 -- 172.21.15.131:0/3204641509 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879800a7a0 con 0x7f87a00cb320 2023-12-26T21:55:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f87877fe700 1 -- 172.21.15.131:0/3204641509 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800ab60 con 0x7f87a00cb320 2023-12-26T21:55:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a00c7b60 con 0x7f87a00cb320 2023-12-26T21:55:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.158+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a015a2b0 con 0x7f87a00cb320 2023-12-26T21:55:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.159+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a00657e0 con 0x7f87a00cb320 2023-12-26T21:55:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.160+0000 7f87877fe700 1 -- 172.21.15.131:0/3204641509 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8798012420 con 0x7f87a00cb320 2023-12-26T21:55:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.161+0000 7f87877fe700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f878c066f30 0x7f878c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.161+0000 7f87877fe700 1 -- 172.21.15.131:0/3204641509 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f8798084ae0 con 0x7f87a00cb320 2023-12-26T21:55:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.161+0000 7f879f7fe700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f878c066f30 0x7f878c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.162+0000 7f879f7fe700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f878c066f30 0x7f878c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87a0064700 tx=0x7f8790016000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:29.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.165+0000 7f87877fe700 1 -- 172.21.15.131:0/3204641509 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8798051bc0 con 0x7f87a00cb320 2023-12-26T21:55:29.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.362+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87a015aa40 con 0x7f87a00cb320 2023-12-26T21:55:29.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.363+0000 7f87877fe700 1 -- 172.21.15.131:0/3204641509 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16021 (secure 0 0 0) 0x7f87980551e0 con 0x7f87a00cb320 2023-12-26T21:55:29.364 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:29.364 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":123,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:55:28.395893+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:55:26.570089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-26T21:55:27.376474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:55:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.365+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f878c066f30 msgr2=0x7f878c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.365+0000 7f87a5bff700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f878c066f30 0x7f878c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87a0064700 tx=0x7f8790016000 comp rx=0 tx=0).stop 2023-12-26T21:55:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.365+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 msgr2=0x7f87a014beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.365+0000 7f87a5bff700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 0x7f87a014beb0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f879800c960 tx=0x7f87980026e0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.366+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 shutdown_connections 2023-12-26T21:55:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.366+0000 7f87a5bff700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f878c066f30 0x7f878c0693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.366+0000 7f87a5bff700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87a00cb320 0x7f87a014beb0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.366+0000 7f87a5bff700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87a006a1e0 0x7f87a00c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.366+0000 7f87a5bff700 1 --2- 172.21.15.131:0/3204641509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a00b0c90 0x7f87a016d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.366+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 >> 172.21.15.131:0/3204641509 conn(0x7f87a0068040 msgr2=0x7f87a014c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.366+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 shutdown_connections 2023-12-26T21:55:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.366+0000 7f87a5bff700 1 -- 172.21.15.131:0/3204641509 wait complete. 2023-12-26T21:55:29.385 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:55:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.574+0000 7f65abfff700 1 Processor -- start 2023-12-26T21:55:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.574+0000 7f65abfff700 1 -- start start 2023-12-26T21:55:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.575+0000 7f65abfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f65a413c510 0x7f65a40c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.575+0000 7f65abfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a40686c0 con 0x7f65a406b8c0 2023-12-26T21:55:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.575+0000 7f65abfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a4069520 con 0x7f65a40cc810 2023-12-26T21:55:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.575+0000 7f65abfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a4068910 con 0x7f65a413c510 2023-12-26T21:55:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.576+0000 7f65a9d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f65a413c510 0x7f65a40c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.576+0000 7f65a959a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f65a406b8c0 0x7f65a40cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36432/0 (socket says 172.21.15.131:36432) 2023-12-26T21:55:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.576+0000 7f65a9d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f65a413c510 0x7f65a40c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46666/0 (socket says 172.21.15.131:46666) 2023-12-26T21:55:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.576+0000 7f65a959a700 1 -- 172.21.15.131:0/4288854855 learned_addr learned my addr 172.21.15.131:0/4288854855 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.576+0000 7f65a9d9b700 1 -- 172.21.15.131:0/4288854855 >> v1:172.21.15.190:6789/0 conn(0x7f65a40cc810 legacy=0x7f65a40699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.576+0000 7f65a9d9b700 1 -- 172.21.15.131:0/4288854855 >> v1:172.21.15.131:6789/0 conn(0x7f65a406b8c0 legacy=0x7f65a40cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.576+0000 7f65a9d9b700 1 -- 172.21.15.131:0/4288854855 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4068bf0 con 0x7f65a413c510 2023-12-26T21:55:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.576+0000 7f65a8d99700 1 -- 172.21.15.131:0/4288854855 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96251165 0 0) 0x7f65a40686c0 con 0x7f65a406b8c0 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65a9d9b700 1 --2- 172.21.15.131:0/4288854855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a413c510 0x7f65a40c8a20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f65a00098d0 tx=0x7f65a0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c21377021043f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65a8d99700 1 -- 172.21.15.131:0/4288854855 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0009e70 con 0x7f65a413c510 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65a8d99700 1 -- 172.21.15.131:0/4288854855 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a0005c10 con 0x7f65a413c510 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65a8d99700 1 -- 172.21.15.131:0/4288854855 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000b650 con 0x7f65a413c510 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65abfff700 1 -- 172.21.15.131:0/4288854855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a413c510 msgr2=0x7f65a40c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65abfff700 1 --2- 172.21.15.131:0/4288854855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a413c510 0x7f65a40c8a20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f65a00098d0 tx=0x7f65a0009120 comp rx=0 tx=0).stop 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65abfff700 1 -- 172.21.15.131:0/4288854855 shutdown_connections 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65abfff700 1 --2- 172.21.15.131:0/4288854855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a413c510 0x7f65a40c8a20 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65abfff700 1 -- 172.21.15.131:0/4288854855 >> 172.21.15.131:0/4288854855 conn(0x7f65a40ce3e0 msgr2=0x7f65a40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65abfff700 1 -- 172.21.15.131:0/4288854855 shutdown_connections 2023-12-26T21:55:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.577+0000 7f65abfff700 1 -- 172.21.15.131:0/4288854855 wait complete. 2023-12-26T21:55:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.579+0000 7f65abfff700 1 Processor -- start 2023-12-26T21:55:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.579+0000 7f65abfff700 1 -- start start 2023-12-26T21:55:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.580+0000 7f65abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a406b8c0 0x7f65a416c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.580+0000 7f65abfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65a40cc810 0x7f65a40b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.580+0000 7f65abfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65a413c510 0x7f65a40b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.580+0000 7f65abfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a416cee0 con 0x7f65a406b8c0 2023-12-26T21:55:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.580+0000 7f65abfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a4157fa0 con 0x7f65a40cc810 2023-12-26T21:55:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.580+0000 7f65abfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a4157970 con 0x7f65a413c510 2023-12-26T21:55:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.580+0000 7f65a9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a406b8c0 0x7f65a416c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.580+0000 7f65a9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a406b8c0 0x7f65a416c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46674/0 (socket says 172.21.15.131:46674) 2023-12-26T21:55:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.580+0000 7f65a9d9b700 1 -- 172.21.15.131:0/3157493228 learned_addr learned my addr 172.21.15.131:0/3157493228 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65a9d9b700 1 -- 172.21.15.131:0/3157493228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65a413c510 msgr2=0x7f65a40b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65a9d9b700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65a413c510 0x7f65a40b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65a9d9b700 1 -- 172.21.15.131:0/3157493228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65a40cc810 msgr2=0x7f65a40b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65a959a700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65a40cc810 0x7f65a40b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65a9d9b700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65a40cc810 0x7f65a40b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65a9d9b700 1 -- 172.21.15.131:0/3157493228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00093c0 con 0x7f65a406b8c0 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65a959a700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65a40cc810 0x7f65a40b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65a9d9b700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a406b8c0 0x7f65a416c9a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009160 tx=0x7f65a0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f6599ffb700 1 -- 172.21.15.131:0/3157493228 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0006460 con 0x7f65a406b8c0 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f6599ffb700 1 -- 172.21.15.131:0/3157493228 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a00065c0 con 0x7f65a406b8c0 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f6599ffb700 1 -- 172.21.15.131:0/3157493228 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a001b480 con 0x7f65a406b8c0 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a40b3c90 con 0x7f65a406b8c0 2023-12-26T21:55:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.581+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a40b3f70 con 0x7f65a406b8c0 2023-12-26T21:55:29.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.582+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a40657e0 con 0x7f65a406b8c0 2023-12-26T21:55:29.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.583+0000 7f6599ffb700 1 -- 172.21.15.131:0/3157493228 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f65a001b830 con 0x7f65a406b8c0 2023-12-26T21:55:29.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.584+0000 7f6599ffb700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6588066f30 0x7f65880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.584+0000 7f6599ffb700 1 -- 172.21.15.131:0/3157493228 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f65a0086970 con 0x7f65a406b8c0 2023-12-26T21:55:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.584+0000 7f65a959a700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6588066f30 0x7f65880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.585+0000 7f65a959a700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6588066f30 0x7f65880693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65a41547e0 tx=0x7f6594009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:29.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.588+0000 7f6599ffb700 1 -- 172.21.15.131:0/3157493228 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65a0053a50 con 0x7f65a406b8c0 2023-12-26T21:55:29.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.786+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65a4068ed0 con 0x7f65a406b8c0 2023-12-26T21:55:29.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.787+0000 7f6599ffb700 1 -- 172.21.15.131:0/3157493228 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16021 (secure 0 0 0) 0x7f65a0057070 con 0x7f65a406b8c0 2023-12-26T21:55:29.788 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:29.788 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":123,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:55:28.395893+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:55:26.570089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-26T21:55:27.376474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6588066f30 msgr2=0x7f65880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6588066f30 0x7f65880693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65a41547e0 tx=0x7f6594009040 comp rx=0 tx=0).stop 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a406b8c0 msgr2=0x7f65a416c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a406b8c0 0x7f65a416c9a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009160 tx=0x7f65a0016000 comp rx=0 tx=0).stop 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 shutdown_connections 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65a413c510 0x7f65a40b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6588066f30 0x7f65880693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65a406b8c0 0x7f65a416c9a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 --2- 172.21.15.131:0/3157493228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65a40cc810 0x7f65a40b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:29.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 >> 172.21.15.131:0/3157493228 conn(0x7f65a40ce3e0 msgr2=0x7f65a41527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:29.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 shutdown_connections 2023-12-26T21:55:29.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:29.789+0000 7f65abfff700 1 -- 172.21.15.131:0/3157493228 wait complete. 2023-12-26T21:55:29.808 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T21:55:30.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.004+0000 7f05353ff700 1 Processor -- start 2023-12-26T21:55:30.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.004+0000 7f05353ff700 1 -- start start 2023-12-26T21:55:30.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.005+0000 7f05353ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f05300c8820 0x7f05300ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:30.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.005+0000 7f05353ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05300b19f0 0x7f0530161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:30.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f05353ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0530157a80 con 0x7f05300c8bf0 2023-12-26T21:55:30.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f05353ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05300cac50 con 0x7f05300c8820 2023-12-26T21:55:30.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f05353ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05300caf30 con 0x7f05300b19f0 2023-12-26T21:55:30.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f052effd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f05300c8820 0x7f05300ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:30.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f0527fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f05300c8bf0 0x7f05300b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36448/0 (socket says 172.21.15.131:36448) 2023-12-26T21:55:30.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f0527fff700 1 -- 172.21.15.131:0/1691837251 learned_addr learned my addr 172.21.15.131:0/1691837251 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:30.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f052f7fe700 1 --2- 172.21.15.131:0/1691837251 >> v2:172.21.15.190:3300/0 conn(0x7f05300b19f0 0x7f0530161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:30.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f052effd700 1 -- 172.21.15.131:0/1691837251 >> v2:172.21.15.190:3300/0 conn(0x7f05300b19f0 msgr2=0x7f0530161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:30.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f052effd700 1 --2- 172.21.15.131:0/1691837251 >> v2:172.21.15.190:3300/0 conn(0x7f05300b19f0 0x7f0530161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f052effd700 1 -- 172.21.15.131:0/1691837251 >> v1:172.21.15.131:6789/0 conn(0x7f05300c8bf0 legacy=0x7f05300b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:30.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f052effd700 1 -- 172.21.15.131:0/1691837251 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300cb210 con 0x7f05300c8820 2023-12-26T21:55:30.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f052e7fc700 1 -- 172.21.15.131:0/1691837251 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987506922 0 0) 0x7f0530157a80 con 0x7f05300c8bf0 2023-12-26T21:55:30.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.006+0000 7f052f7fe700 1 --2- 172.21.15.131:0/1691837251 >> v2:172.21.15.190:3300/0 conn(0x7f05300b19f0 0x7f0530161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:30.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f052effd700 1 --2- 172.21.15.131:0/1691837251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300c8820 0x7f05300ca300 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0530070f80 tx=0x7f05200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71de0f928c2a34e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:30.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f052e7fc700 1 -- 172.21.15.131:0/1691837251 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052000b6b0 con 0x7f05300c8820 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f052e7fc700 1 -- 172.21.15.131:0/1691837251 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052000b810 con 0x7f05300c8820 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f052e7fc700 1 -- 172.21.15.131:0/1691837251 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052000bbb0 con 0x7f05300c8820 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f05353ff700 1 -- 172.21.15.131:0/1691837251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300c8820 msgr2=0x7f05300ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f05353ff700 1 --2- 172.21.15.131:0/1691837251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300c8820 0x7f05300ca300 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0530070f80 tx=0x7f05200091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f05353ff700 1 -- 172.21.15.131:0/1691837251 shutdown_connections 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f05353ff700 1 --2- 172.21.15.131:0/1691837251 >> v2:172.21.15.190:3300/0 conn(0x7f05300b19f0 0x7f0530161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f05353ff700 1 --2- 172.21.15.131:0/1691837251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300c8820 0x7f05300ca300 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f05353ff700 1 -- 172.21.15.131:0/1691837251 >> 172.21.15.131:0/1691837251 conn(0x7f0530068040 msgr2=0x7f0530069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.007+0000 7f05353ff700 1 -- 172.21.15.131:0/1691837251 shutdown_connections 2023-12-26T21:55:30.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.008+0000 7f05353ff700 1 -- 172.21.15.131:0/1691837251 wait complete. 2023-12-26T21:55:30.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.009+0000 7f05353ff700 1 Processor -- start 2023-12-26T21:55:30.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.009+0000 7f05353ff700 1 -- start start 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f05353ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300b19f0 0x7f05300b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f05353ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05300c8820 0x7f0530169010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f05353ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05300c8bf0 0x7f05300a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f05353ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300a5eb0 con 0x7f05300b19f0 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f05353ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300a6750 con 0x7f05300c8bf0 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f05353ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b0810 con 0x7f05300c8820 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f052effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300b19f0 0x7f05300b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f052effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300b19f0 0x7f05300b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46700/0 (socket says 172.21.15.131:46700) 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f052effd700 1 -- 172.21.15.131:0/3937414480 learned_addr learned my addr 172.21.15.131:0/3937414480 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f0527fff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05300c8820 0x7f0530169010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f052f7fe700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05300c8bf0 0x7f05300a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f052effd700 1 -- 172.21.15.131:0/3937414480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05300c8820 msgr2=0x7f0530169010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:30.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f052effd700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05300c8820 0x7f0530169010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.010+0000 7f052effd700 1 -- 172.21.15.131:0/3937414480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05300c8bf0 msgr2=0x7f05300a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.011+0000 7f052effd700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05300c8bf0 0x7f05300a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.011+0000 7f052effd700 1 -- 172.21.15.131:0/3937414480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0520009420 con 0x7f05300b19f0 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.011+0000 7f0527fff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05300c8820 0x7f0530169010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.011+0000 7f052effd700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300b19f0 0x7f05300b0ca0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f05300688f0 tx=0x7f0520017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.011+0000 7f0526ffd700 1 -- 172.21.15.131:0/3937414480 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052000bb90 con 0x7f05300b19f0 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.011+0000 7f0526ffd700 1 -- 172.21.15.131:0/3937414480 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520011420 con 0x7f05300b19f0 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.011+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05300a6100 con 0x7f05300b19f0 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.011+0000 7f0526ffd700 1 -- 172.21.15.131:0/3937414480 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200117c0 con 0x7f05300b19f0 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.011+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0530169550 con 0x7f05300b19f0 2023-12-26T21:55:30.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.012+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05300657e0 con 0x7f05300b19f0 2023-12-26T21:55:30.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.013+0000 7f0526ffd700 1 -- 172.21.15.131:0/3937414480 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0520011bb0 con 0x7f05300b19f0 2023-12-26T21:55:30.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.014+0000 7f0526ffd700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0510066f30 0x7f05100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:30.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.014+0000 7f0526ffd700 1 -- 172.21.15.131:0/3937414480 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f05200860c0 con 0x7f05300b19f0 2023-12-26T21:55:30.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.014+0000 7f0527fff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0510066f30 0x7f05100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:30.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.015+0000 7f0527fff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0510066f30 0x7f05100693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05300c3a50 tx=0x7f0518009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:30.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.018+0000 7f0526ffd700 1 -- 172.21.15.131:0/3937414480 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05200531a0 con 0x7f05300b19f0 2023-12-26T21:55:30.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.254+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f05300cb4f0 con 0x7f05300b19f0 2023-12-26T21:55:30.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.951+0000 7f0526ffd700 1 -- 172.21.15.131:0/3937414480 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0520011e60 con 0x7f05300b19f0 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0510066f30 msgr2=0x7f05100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0510066f30 0x7f05100693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05300c3a50 tx=0x7f0518009040 comp rx=0 tx=0).stop 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300b19f0 msgr2=0x7f05300b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300b19f0 0x7f05300b0ca0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f05300688f0 tx=0x7f0520017000 comp rx=0 tx=0).stop 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 shutdown_connections 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05300c8bf0 0x7f05300a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0510066f30 0x7f05100693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05300b19f0 0x7f05300b0ca0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 --2- 172.21.15.131:0/3937414480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05300c8820 0x7f0530169010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 >> 172.21.15.131:0/3937414480 conn(0x7f0530068040 msgr2=0x7f05300c1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 shutdown_connections 2023-12-26T21:55:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:30.954+0000 7f05353ff700 1 -- 172.21.15.131:0/3937414480 wait complete. 2023-12-26T21:55:30.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T21:55:31.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.169+0000 7fc442dff700 1 Processor -- start 2023-12-26T21:55:31.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.169+0000 7fc442dff700 1 -- start start 2023-12-26T21:55:31.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.171+0000 7fc442dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc43c159600 0x7fc43c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:31.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.171+0000 7fc442dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc43c1599d0 0x7fc43c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:31.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.171+0000 7fc442dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43c0cfd50 con 0x7fc43c0d01e0 2023-12-26T21:55:31.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.171+0000 7fc442dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c15e210 con 0x7fc43c1599d0 2023-12-26T21:55:31.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.171+0000 7fc442dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c0b1280 con 0x7fc43c159600 2023-12-26T21:55:31.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.171+0000 7fc43bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc43c1599d0 0x7fc43c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:31.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.171+0000 7fc44139c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc43c0d01e0 0x7fc43c0ccf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36458/0 (socket says 172.21.15.131:36458) 2023-12-26T21:55:31.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.171+0000 7fc44139c700 1 -- 172.21.15.131:0/1772513298 learned_addr learned my addr 172.21.15.131:0/1772513298 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:31.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.171+0000 7fc440b9b700 1 --2- 172.21.15.131:0/1772513298 >> v2:172.21.15.190:3300/0 conn(0x7fc43c159600 0x7fc43c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:31.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.172+0000 7fc43bfff700 1 -- 172.21.15.131:0/1772513298 >> v2:172.21.15.190:3300/0 conn(0x7fc43c159600 msgr2=0x7fc43c0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:31.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.172+0000 7fc43bfff700 1 --2- 172.21.15.131:0/1772513298 >> v2:172.21.15.190:3300/0 conn(0x7fc43c159600 0x7fc43c0ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.172+0000 7fc43bfff700 1 -- 172.21.15.131:0/1772513298 >> v1:172.21.15.131:6789/0 conn(0x7fc43c0d01e0 legacy=0x7fc43c0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:31.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.172+0000 7fc43bfff700 1 -- 172.21.15.131:0/1772513298 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c0cf140 con 0x7fc43c1599d0 2023-12-26T21:55:31.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.172+0000 7fc43b7fe700 1 -- 172.21.15.131:0/1772513298 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697333808 0 0) 0x7fc43c0cfd50 con 0x7fc43c0d01e0 2023-12-26T21:55:31.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.172+0000 7fc43bfff700 1 --2- 172.21.15.131:0/1772513298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c1599d0 0x7fc43c15dcd0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc43c089850 tx=0x7fc4280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c32849e00799b6f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:31.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.172+0000 7fc43b7fe700 1 -- 172.21.15.131:0/1772513298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc428011630 con 0x7fc43c1599d0 2023-12-26T21:55:31.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.172+0000 7fc43b7fe700 1 -- 172.21.15.131:0/1772513298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc428011790 con 0x7fc43c1599d0 2023-12-26T21:55:31.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.172+0000 7fc43b7fe700 1 -- 172.21.15.131:0/1772513298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc428011b30 con 0x7fc43c1599d0 2023-12-26T21:55:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.173+0000 7fc442dff700 1 -- 172.21.15.131:0/1772513298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c1599d0 msgr2=0x7fc43c15dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.173+0000 7fc442dff700 1 --2- 172.21.15.131:0/1772513298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c1599d0 0x7fc43c15dcd0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc43c089850 tx=0x7fc4280091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.173+0000 7fc442dff700 1 -- 172.21.15.131:0/1772513298 shutdown_connections 2023-12-26T21:55:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.173+0000 7fc442dff700 1 --2- 172.21.15.131:0/1772513298 >> v2:172.21.15.190:3300/0 conn(0x7fc43c159600 0x7fc43c0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.173+0000 7fc442dff700 1 --2- 172.21.15.131:0/1772513298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c1599d0 0x7fc43c15dcd0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.173+0000 7fc442dff700 1 -- 172.21.15.131:0/1772513298 >> 172.21.15.131:0/1772513298 conn(0x7fc43c06ece0 msgr2=0x7fc43c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.173+0000 7fc442dff700 1 -- 172.21.15.131:0/1772513298 shutdown_connections 2023-12-26T21:55:31.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.173+0000 7fc442dff700 1 -- 172.21.15.131:0/1772513298 wait complete. 2023-12-26T21:55:31.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.174+0000 7fc442dff700 1 Processor -- start 2023-12-26T21:55:31.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.174+0000 7fc442dff700 1 -- start start 2023-12-26T21:55:31.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.175+0000 7fc442dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c0d01e0 0x7fc43c166c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:31.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.175+0000 7fc442dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43c159600 0x7fc43c15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:31.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.175+0000 7fc442dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc43c15bd60 0x7fc43c1695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:31.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.175+0000 7fc442dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c15a180 con 0x7fc43c0d01e0 2023-12-26T21:55:31.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.175+0000 7fc442dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c159e50 con 0x7fc43c159600 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.175+0000 7fc442dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c169fb0 con 0x7fc43c15bd60 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.175+0000 7fc440b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c0d01e0 0x7fc43c166c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc440b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c0d01e0 0x7fc43c166c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46730/0 (socket says 172.21.15.131:46730) 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc440b9b700 1 -- 172.21.15.131:0/2879847932 learned_addr learned my addr 172.21.15.131:0/2879847932 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc43bfff700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43c159600 0x7fc43c15a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc44139c700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc43c15bd60 0x7fc43c1695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc440b9b700 1 -- 172.21.15.131:0/2879847932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc43c15bd60 msgr2=0x7fc43c1695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc440b9b700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc43c15bd60 0x7fc43c1695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc440b9b700 1 -- 172.21.15.131:0/2879847932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43c159600 msgr2=0x7fc43c15a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc440b9b700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43c159600 0x7fc43c15a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc440b9b700 1 -- 172.21.15.131:0/2879847932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428009420 con 0x7fc43c0d01e0 2023-12-26T21:55:31.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc440b9b700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c0d01e0 0x7fc43c166c00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc43000ec80 tx=0x7fc430009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:31.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.176+0000 7fc438ff9700 1 -- 172.21.15.131:0/2879847932 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000c850 con 0x7fc43c0d01e0 2023-12-26T21:55:31.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.177+0000 7fc438ff9700 1 -- 172.21.15.131:0/2879847932 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43000c9b0 con 0x7fc43c0d01e0 2023-12-26T21:55:31.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.177+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c16a290 con 0x7fc43c0d01e0 2023-12-26T21:55:31.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.177+0000 7fc438ff9700 1 -- 172.21.15.131:0/2879847932 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000cd30 con 0x7fc43c0d01e0 2023-12-26T21:55:31.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.177+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c15ab50 con 0x7fc43c0d01e0 2023-12-26T21:55:31.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.178+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c15ad40 con 0x7fc43c0d01e0 2023-12-26T21:55:31.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.179+0000 7fc438ff9700 1 -- 172.21.15.131:0/2879847932 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc430013680 con 0x7fc43c0d01e0 2023-12-26T21:55:31.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.180+0000 7fc438ff9700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc420066f30 0x7fc4200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:31.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.180+0000 7fc438ff9700 1 -- 172.21.15.131:0/2879847932 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fc430084de0 con 0x7fc43c0d01e0 2023-12-26T21:55:31.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.180+0000 7fc43bfff700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc420066f30 0x7fc4200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:31.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.180+0000 7fc43bfff700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc420066f30 0x7fc4200693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4280093f0 tx=0x7fc428017000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:31.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.183+0000 7fc438ff9700 1 -- 172.21.15.131:0/2879847932 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc430051e20 con 0x7fc43c0d01e0 2023-12-26T21:55:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.419+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc43c15a460 con 0x7fc43c0d01e0 2023-12-26T21:55:31.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.952+0000 7fc438ff9700 1 -- 172.21.15.131:0/2879847932 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc4300210c0 con 0x7fc43c0d01e0 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.954+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc420066f30 msgr2=0x7fc4200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.954+0000 7fc442dff700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc420066f30 0x7fc4200693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4280093f0 tx=0x7fc428017000 comp rx=0 tx=0).stop 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.954+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c0d01e0 msgr2=0x7fc43c166c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.954+0000 7fc442dff700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c0d01e0 0x7fc43c166c00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc43000ec80 tx=0x7fc430009120 comp rx=0 tx=0).stop 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.955+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 shutdown_connections 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.955+0000 7fc442dff700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc43c15bd60 0x7fc43c1695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.955+0000 7fc442dff700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc420066f30 0x7fc4200693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.955+0000 7fc442dff700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc43c0d01e0 0x7fc43c166c00 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.955+0000 7fc442dff700 1 --2- 172.21.15.131:0/2879847932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc43c159600 0x7fc43c15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.955+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 >> 172.21.15.131:0/2879847932 conn(0x7fc43c06ece0 msgr2=0x7fc43c15d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:31.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.955+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 shutdown_connections 2023-12-26T21:55:31.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:31.955+0000 7fc442dff700 1 -- 172.21.15.131:0/2879847932 wait complete. 2023-12-26T21:55:31.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:55:32.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.163+0000 7f68147ff700 1 Processor -- start 2023-12-26T21:55:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.163+0000 7f68147ff700 1 -- start start 2023-12-26T21:55:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.164+0000 7f68147ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f680c06a1a0 0x7f680c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.164+0000 7f68147ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c068a80 con 0x7f680c159930 2023-12-26T21:55:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.164+0000 7f68147ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c0cac50 con 0x7f680c159560 2023-12-26T21:55:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.164+0000 7f68147ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c15da00 con 0x7f680c06a1a0 2023-12-26T21:55:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.164+0000 7f6812d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f680c06a1a0 0x7f680c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.164+0000 7f6811d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f680c159930 0x7f680c068f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36468/0 (socket says 172.21.15.131:36468) 2023-12-26T21:55:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.164+0000 7f6812d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f680c06a1a0 0x7f680c15d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46736/0 (socket says 172.21.15.131:46736) 2023-12-26T21:55:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.164+0000 7f6811d9a700 1 -- 172.21.15.131:0/3267689626 learned_addr learned my addr 172.21.15.131:0/3267689626 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.165+0000 7f6812d9c700 1 -- 172.21.15.131:0/3267689626 >> v1:172.21.15.190:6789/0 conn(0x7f680c159560 legacy=0x7f680c0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.165+0000 7f6812d9c700 1 -- 172.21.15.131:0/3267689626 >> v1:172.21.15.131:6789/0 conn(0x7f680c159930 legacy=0x7f680c068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.165+0000 7f6812d9c700 1 -- 172.21.15.131:0/3267689626 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c15dce0 con 0x7f680c06a1a0 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.165+0000 7f6811599700 1 -- 172.21.15.131:0/3267689626 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158550345 0 0) 0x7f680c068a80 con 0x7f680c159930 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.165+0000 7f6812d9c700 1 --2- 172.21.15.131:0/3267689626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c06a1a0 0x7f680c15d300 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6804009e60 tx=0x7f6804009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50b19d1c36f57b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.165+0000 7f6811599700 1 -- 172.21.15.131:0/3267689626 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400e070 con 0x7f680c06a1a0 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.165+0000 7f6811599700 1 -- 172.21.15.131:0/3267689626 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680400bc30 con 0x7f680c06a1a0 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.165+0000 7f6811599700 1 -- 172.21.15.131:0/3267689626 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804012430 con 0x7f680c06a1a0 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.166+0000 7f68147ff700 1 -- 172.21.15.131:0/3267689626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c06a1a0 msgr2=0x7f680c15d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.166+0000 7f68147ff700 1 --2- 172.21.15.131:0/3267689626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c06a1a0 0x7f680c15d300 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6804009e60 tx=0x7f6804009780 comp rx=0 tx=0).stop 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.166+0000 7f68147ff700 1 -- 172.21.15.131:0/3267689626 shutdown_connections 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.166+0000 7f68147ff700 1 --2- 172.21.15.131:0/3267689626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c06a1a0 0x7f680c15d300 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.166+0000 7f68147ff700 1 -- 172.21.15.131:0/3267689626 >> 172.21.15.131:0/3267689626 conn(0x7f680c0ce3d0 msgr2=0x7f680c0ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.166+0000 7f68147ff700 1 -- 172.21.15.131:0/3267689626 shutdown_connections 2023-12-26T21:55:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.166+0000 7f68147ff700 1 -- 172.21.15.131:0/3267689626 wait complete. 2023-12-26T21:55:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.167+0000 7f68147ff700 1 Processor -- start 2023-12-26T21:55:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.168+0000 7f68147ff700 1 -- start start 2023-12-26T21:55:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.168+0000 7f68147ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f680c06a1a0 0x7f680c15c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.168+0000 7f68147ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c159560 0x7f680c155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.168+0000 7f68147ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f680c159930 0x7f680c068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.168+0000 7f68147ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c15bbb0 con 0x7f680c159560 2023-12-26T21:55:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.168+0000 7f68147ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c15b280 con 0x7f680c159930 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.168+0000 7f68147ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c15b560 con 0x7f680c06a1a0 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6811d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c159560 0x7f680c155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6811d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c159560 0x7f680c155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46752/0 (socket says 172.21.15.131:46752) 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6811d9a700 1 -- 172.21.15.131:0/854398644 learned_addr learned my addr 172.21.15.131:0/854398644 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f681259b700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f680c06a1a0 0x7f680c15c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6812d9c700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f680c159930 0x7f680c068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6811d9a700 1 -- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f680c06a1a0 msgr2=0x7f680c15c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6811d9a700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f680c06a1a0 0x7f680c15c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6811d9a700 1 -- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f680c159930 msgr2=0x7f680c068e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6811d9a700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f680c159930 0x7f680c068e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6811d9a700 1 -- 172.21.15.131:0/854398644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6804009a20 con 0x7f680c159560 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f681259b700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f680c06a1a0 0x7f680c15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6812d9c700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f680c159930 0x7f680c068e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.169+0000 7f6811d9a700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c159560 0x7f680c155b00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c5b0 tx=0x7f67fc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.170+0000 7f68027fc700 1 -- 172.21.15.131:0/854398644 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc005b30 con 0x7f680c159560 2023-12-26T21:55:32.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.170+0000 7f68027fc700 1 -- 172.21.15.131:0/854398644 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc005c90 con 0x7f680c159560 2023-12-26T21:55:32.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.170+0000 7f68027fc700 1 -- 172.21.15.131:0/854398644 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc007300 con 0x7f680c159560 2023-12-26T21:55:32.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.170+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c15b840 con 0x7f680c159560 2023-12-26T21:55:32.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.170+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c15af30 con 0x7f680c159560 2023-12-26T21:55:32.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.171+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c15bd20 con 0x7f680c159560 2023-12-26T21:55:32.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.172+0000 7f68027fc700 1 -- 172.21.15.131:0/854398644 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f67fc01b3b0 con 0x7f680c159560 2023-12-26T21:55:32.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.172+0000 7f68027fc700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f67f0066f30 0x7f67f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.173+0000 7f68027fc700 1 -- 172.21.15.131:0/854398644 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f67fc083ca0 con 0x7f680c159560 2023-12-26T21:55:32.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.173+0000 7f681259b700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f67f0066f30 0x7f67f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.173+0000 7f681259b700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f67f0066f30 0x7f67f00693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6808007fd0 tx=0x7f6808009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:32.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.176+0000 7f68027fc700 1 -- 172.21.15.131:0/854398644 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67fc050ce0 con 0x7f680c159560 2023-12-26T21:55:32.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.412+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f680c15b9d0 con 0x7f680c159560 2023-12-26T21:55:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.412+0000 7f68027fc700 1 -- 172.21.15.131:0/854398644 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f67fc017090 con 0x7f680c159560 2023-12-26T21:55:32.413 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:32.413 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6623,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3234851896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3234851896},{"type":"v1","addr":"172.21.15.131:6834","nonce":3234851896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2363679591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2363679591},{"type":"v1","addr":"172.21.15.190:6834","nonce":2363679591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3018492544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3018492544},{"type":"v1","addr":"172.21.15.190:6835","nonce":3018492544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:55:28.396211+0000","modified":"2023-12-26T21:55:31.950235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1932798418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1932798418},{"type":"v1","addr":"172.21.15.131:6835","nonce":1932798418}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f67f0066f30 msgr2=0x7f67f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f67f0066f30 0x7f67f00693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6808007fd0 tx=0x7f6808009040 comp rx=0 tx=0).stop 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c159560 msgr2=0x7f680c155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c159560 0x7f680c155b00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c5b0 tx=0x7f67fc001070 comp rx=0 tx=0).stop 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 shutdown_connections 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f680c159930 0x7f680c068e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f680c06a1a0 0x7f680c15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f67f0066f30 0x7f67f00693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 --2- 172.21.15.131:0/854398644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f680c159560 0x7f680c155b00 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 >> 172.21.15.131:0/854398644 conn(0x7f680c0ce3d0 msgr2=0x7f680c156470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 shutdown_connections 2023-12-26T21:55:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.415+0000 7f68147ff700 1 -- 172.21.15.131:0/854398644 wait complete. 2023-12-26T21:55:32.417 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 57 2023-12-26T21:55:32.434 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:55:32.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.623+0000 7f4d23fff700 1 Processor -- start 2023-12-26T21:55:32.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.623+0000 7f4d23fff700 1 -- start start 2023-12-26T21:55:32.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.624+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2415af90 0x7f4d240b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.624+0000 7f4d23fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d240b4100 0x7f4d2415c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.624+0000 7f4d23fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2415c0a0 con 0x7f4d240b6020 2023-12-26T21:55:32.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.624+0000 7f4d23fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2415ad40 con 0x7f4d240b4100 2023-12-26T21:55:32.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.624+0000 7f4d23fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240b63f0 con 0x7f4d2415af90 2023-12-26T21:55:32.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.625+0000 7f4d227fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2415af90 0x7f4d240b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.625+0000 7f4d237fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d240b4100 0x7f4d2415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.625+0000 7f4d227fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2415af90 0x7f4d240b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:45586/0 (socket says 172.21.15.131:45586) 2023-12-26T21:55:32.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.625+0000 7f4d237fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d240b4100 0x7f4d2415c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:54388/0 (socket says 172.21.15.131:54388) 2023-12-26T21:55:32.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.625+0000 7f4d227fc700 1 -- 172.21.15.131:0/856244342 learned_addr learned my addr 172.21.15.131:0/856244342 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:32.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.625+0000 7f4d22ffd700 1 --1- 172.21.15.131:0/856244342 >> v1:172.21.15.190:6789/0 conn(0x7f4d240b6020 0x7f4d2415a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:45142/0 (socket says 172.21.15.131:45142) 2023-12-26T21:55:32.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.626+0000 7f4d227fc700 1 -- 172.21.15.131:0/856244342 >> v2:172.21.15.190:3300/0 conn(0x7f4d240b4100 msgr2=0x7f4d2415c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:32.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.626+0000 7f4d227fc700 1 --2- 172.21.15.131:0/856244342 >> v2:172.21.15.190:3300/0 conn(0x7f4d240b4100 0x7f4d2415c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.626+0000 7f4d227fc700 1 -- 172.21.15.131:0/856244342 >> v1:172.21.15.190:6789/0 conn(0x7f4d240b6020 legacy=0x7f4d2415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.626+0000 7f4d227fc700 1 -- 172.21.15.131:0/856244342 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d241598e0 con 0x7f4d2415af90 2023-12-26T21:55:32.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.626+0000 7f4d237fe700 1 --2- 172.21.15.131:0/856244342 >> v2:172.21.15.190:3300/0 conn(0x7f4d240b4100 0x7f4d2415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:32.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.626+0000 7f4d227fc700 1 --2- 172.21.15.131:0/856244342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2415af90 0x7f4d240b3bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4d180099b0 tx=0x7f4d18009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1381a76cfdfa7c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:32.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.626+0000 7f4d21ffb700 1 -- 172.21.15.131:0/856244342 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18009ea0 con 0x7f4d2415af90 2023-12-26T21:55:32.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.626+0000 7f4d21ffb700 1 -- 172.21.15.131:0/856244342 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1800b860 con 0x7f4d2415af90 2023-12-26T21:55:32.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.626+0000 7f4d21ffb700 1 -- 172.21.15.131:0/856244342 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800bc00 con 0x7f4d2415af90 2023-12-26T21:55:32.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.627+0000 7f4d23fff700 1 -- 172.21.15.131:0/856244342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2415af90 msgr2=0x7f4d240b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.627+0000 7f4d23fff700 1 --2- 172.21.15.131:0/856244342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2415af90 0x7f4d240b3bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4d180099b0 tx=0x7f4d18009250 comp rx=0 tx=0).stop 2023-12-26T21:55:32.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.627+0000 7f4d23fff700 1 -- 172.21.15.131:0/856244342 shutdown_connections 2023-12-26T21:55:32.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.627+0000 7f4d23fff700 1 --2- 172.21.15.131:0/856244342 >> v2:172.21.15.190:3300/0 conn(0x7f4d240b4100 0x7f4d2415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.627+0000 7f4d23fff700 1 --2- 172.21.15.131:0/856244342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2415af90 0x7f4d240b3bc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.627+0000 7f4d23fff700 1 -- 172.21.15.131:0/856244342 >> 172.21.15.131:0/856244342 conn(0x7f4d2406ece0 msgr2=0x7f4d2406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:32.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.627+0000 7f4d23fff700 1 -- 172.21.15.131:0/856244342 shutdown_connections 2023-12-26T21:55:32.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.627+0000 7f4d23fff700 1 -- 172.21.15.131:0/856244342 wait complete. 2023-12-26T21:55:32.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.628+0000 7f4d23fff700 1 Processor -- start 2023-12-26T21:55:32.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.628+0000 7f4d23fff700 1 -- start start 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.629+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d240b4100 0x7f4d24153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.629+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d240b6020 0x7f4d2416e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.629+0000 7f4d23fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2415af90 0x7f4d2406c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.629+0000 7f4d23fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2416f440 con 0x7f4d2415af90 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.629+0000 7f4d23fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2416ee80 con 0x7f4d240b6020 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d23fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2416f160 con 0x7f4d240b4100 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d237fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2415af90 0x7f4d2406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d237fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2415af90 0x7f4d2406c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46758/0 (socket says 172.21.15.131:46758) 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d237fe700 1 -- 172.21.15.131:0/2812061653 learned_addr learned my addr 172.21.15.131:0/2812061653 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d22ffd700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d240b4100 0x7f4d24153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d227fc700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d240b6020 0x7f4d2416e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d237fe700 1 -- 172.21.15.131:0/2812061653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d240b4100 msgr2=0x7f4d24153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d237fe700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d240b4100 0x7f4d24153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d237fe700 1 -- 172.21.15.131:0/2812061653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d240b6020 msgr2=0x7f4d2416e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d237fe700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d240b6020 0x7f4d2416e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d237fe700 1 -- 172.21.15.131:0/2812061653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c009a20 con 0x7f4d2415af90 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d227fc700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d240b6020 0x7f4d2416e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.630+0000 7f4d237fe700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2415af90 0x7f4d2406c740 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009f20 tx=0x7f4d1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.631+0000 7f4d0affd700 1 -- 172.21.15.131:0/2812061653 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00cbd0 con 0x7f4d2415af90 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.631+0000 7f4d0affd700 1 -- 172.21.15.131:0/2812061653 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c00f040 con 0x7f4d2415af90 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.631+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d180094f0 con 0x7f4d2415af90 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.631+0000 7f4d0affd700 1 -- 172.21.15.131:0/2812061653 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c014480 con 0x7f4d2415af90 2023-12-26T21:55:32.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.631+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2406ad50 con 0x7f4d2415af90 2023-12-26T21:55:32.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.632+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2406a8d0 con 0x7f4d2415af90 2023-12-26T21:55:32.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.633+0000 7f4d0affd700 1 -- 172.21.15.131:0/2812061653 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4d1c003020 con 0x7f4d2415af90 2023-12-26T21:55:32.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.634+0000 7f4d0affd700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4d10066f30 0x7f4d100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:32.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.634+0000 7f4d0affd700 1 -- 172.21.15.131:0/2812061653 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f4d1c084fc0 con 0x7f4d2415af90 2023-12-26T21:55:32.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.634+0000 7f4d22ffd700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4d10066f30 0x7f4d100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:32.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.635+0000 7f4d22ffd700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4d10066f30 0x7f4d100693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d240b24e0 tx=0x7f4d14007eb0 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:32.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.638+0000 7f4d0affd700 1 -- 172.21.15.131:0/2812061653 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d2406a8d0 con 0x7f4d2415af90 2023-12-26T21:55:32.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.831+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d2406a8d0 con 0x7f4d2415af90 2023-12-26T21:55:32.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.832+0000 7f4d0affd700 1 -- 172.21.15.131:0/2812061653 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16008 (secure 0 0 0) 0x7f4d2406a8d0 con 0x7f4d2415af90 2023-12-26T21:55:32.833 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:32.833 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":124,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:55:29.931730+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:55:26.570089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-26T21:55:27.376474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:55:32.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.834+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4d10066f30 msgr2=0x7f4d100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.834+0000 7f4d23fff700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4d10066f30 0x7f4d100693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d240b24e0 tx=0x7f4d14007eb0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.834+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2415af90 msgr2=0x7f4d2406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.834+0000 7f4d23fff700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2415af90 0x7f4d2406c740 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009f20 tx=0x7f4d1c00b040 comp rx=0 tx=0).stop 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.835+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 shutdown_connections 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.835+0000 7f4d23fff700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4d10066f30 0x7f4d100693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.835+0000 7f4d23fff700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2415af90 0x7f4d2406c740 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.835+0000 7f4d23fff700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d240b4100 0x7f4d24153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.835+0000 7f4d23fff700 1 --2- 172.21.15.131:0/2812061653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d240b6020 0x7f4d2416e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.835+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 >> 172.21.15.131:0/2812061653 conn(0x7f4d2406ece0 msgr2=0x7f4d240cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.835+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 shutdown_connections 2023-12-26T21:55:32.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:32.835+0000 7f4d23fff700 1 -- 172.21.15.131:0/2812061653 wait complete. 2023-12-26T21:55:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T21:55:32.854 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:55:33.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.044+0000 7f21791ff700 1 Processor -- start 2023-12-26T21:55:33.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.044+0000 7f21791ff700 1 -- start start 2023-12-26T21:55:33.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f21791ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2174159a70 0x7f21740b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f21791ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740afcd0 con 0x7f21740b2a90 2023-12-26T21:55:33.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f21791ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740b3e70 con 0x7f21741596a0 2023-12-26T21:55:33.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f21791ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21740b0f20 con 0x7f2174159a70 2023-12-26T21:55:33.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f217359e700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f21740b2a90 0x7f21740b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36474/0 (socket says 172.21.15.131:36474) 2023-12-26T21:55:33.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f217359e700 1 -- 172.21.15.131:0/153504154 learned_addr learned my addr 172.21.15.131:0/153504154 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:33.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f216bfff700 1 --2- 172.21.15.131:0/153504154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2174159a70 0x7f21740b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f217259c700 1 -- 172.21.15.131:0/153504154 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499326971 0 0) 0x7f21740afcd0 con 0x7f21740b2a90 2023-12-26T21:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f217259c700 1 -- 172.21.15.131:0/153504154 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2160003570 con 0x7f21740b2a90 2023-12-26T21:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f217259c700 1 -- 172.21.15.131:0/153504154 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495046905 0 0) 0x7f21740b3e70 con 0x7f21741596a0 2023-12-26T21:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.046+0000 7f217259c700 1 -- 172.21.15.131:0/153504154 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21740afcd0 con 0x7f21741596a0 2023-12-26T21:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.047+0000 7f216bfff700 1 -- 172.21.15.131:0/153504154 >> v1:172.21.15.190:6789/0 conn(0x7f21741596a0 legacy=0x7f21740b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.047+0000 7f216bfff700 1 -- 172.21.15.131:0/153504154 >> v1:172.21.15.131:6789/0 conn(0x7f21740b2a90 legacy=0x7f21740b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.047+0000 7f216bfff700 1 -- 172.21.15.131:0/153504154 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740b3540 con 0x7f2174159a70 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.047+0000 7f216bfff700 1 --2- 172.21.15.131:0/153504154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2174159a70 0x7f21740b0160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f215c009910 tx=0x7f215c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b582607baf121f38 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.047+0000 7f217259c700 1 -- 172.21.15.131:0/153504154 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011630 con 0x7f2174159a70 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.047+0000 7f217259c700 1 -- 172.21.15.131:0/153504154 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c011790 con 0x7f2174159a70 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.047+0000 7f217259c700 1 -- 172.21.15.131:0/153504154 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011b30 con 0x7f2174159a70 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.048+0000 7f21791ff700 1 -- 172.21.15.131:0/153504154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2174159a70 msgr2=0x7f21740b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.048+0000 7f21791ff700 1 --2- 172.21.15.131:0/153504154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2174159a70 0x7f21740b0160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f215c009910 tx=0x7f215c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.048+0000 7f21791ff700 1 -- 172.21.15.131:0/153504154 shutdown_connections 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.048+0000 7f21791ff700 1 --2- 172.21.15.131:0/153504154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2174159a70 0x7f21740b0160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.048+0000 7f21791ff700 1 -- 172.21.15.131:0/153504154 >> 172.21.15.131:0/153504154 conn(0x7f217406ece0 msgr2=0x7f217406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.048+0000 7f21791ff700 1 -- 172.21.15.131:0/153504154 shutdown_connections 2023-12-26T21:55:33.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.048+0000 7f21791ff700 1 -- 172.21.15.131:0/153504154 wait complete. 2023-12-26T21:55:33.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.049+0000 7f21791ff700 1 Processor -- start 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.049+0000 7f21791ff700 1 -- start start 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.050+0000 7f21791ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21740b2a90 0x7f2174155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.050+0000 7f21791ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21741596a0 0x7f21740c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.050+0000 7f21791ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2174159a70 0x7f217416d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.050+0000 7f21791ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21740d04c0 con 0x7f2174159a70 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.050+0000 7f21791ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217415caf0 con 0x7f21740b2a90 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.050+0000 7f21791ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21740cfe70 con 0x7f21741596a0 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f217359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2174159a70 0x7f217416d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f217359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2174159a70 0x7f217416d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46762/0 (socket says 172.21.15.131:46762) 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f217359e700 1 -- 172.21.15.131:0/2020778040 learned_addr learned my addr 172.21.15.131:0/2020778040 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f216bfff700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21741596a0 0x7f21740c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f2172d9d700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21740b2a90 0x7f2174155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f217359e700 1 -- 172.21.15.131:0/2020778040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21741596a0 msgr2=0x7f21740c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f217359e700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21741596a0 0x7f21740c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f217359e700 1 -- 172.21.15.131:0/2020778040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21740b2a90 msgr2=0x7f2174155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f217359e700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21740b2a90 0x7f2174155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f217359e700 1 -- 172.21.15.131:0/2020778040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c009420 con 0x7f2174159a70 2023-12-26T21:55:33.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f217359e700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2174159a70 0x7f217416d2e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f216c00cc60 tx=0x7f216c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:33.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f216affd700 1 -- 172.21.15.131:0/2020778040 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c00a960 con 0x7f2174159a70 2023-12-26T21:55:33.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f216affd700 1 -- 172.21.15.131:0/2020778040 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c00aae0 con 0x7f2174159a70 2023-12-26T21:55:33.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.051+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21740d0150 con 0x7f2174159a70 2023-12-26T21:55:33.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.052+0000 7f216affd700 1 -- 172.21.15.131:0/2020778040 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c00aea0 con 0x7f2174159a70 2023-12-26T21:55:33.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.052+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21740ca230 con 0x7f2174159a70 2023-12-26T21:55:33.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.053+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21740d0600 con 0x7f2174159a70 2023-12-26T21:55:33.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.054+0000 7f216affd700 1 -- 172.21.15.131:0/2020778040 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f216c007300 con 0x7f2174159a70 2023-12-26T21:55:33.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.055+0000 7f216affd700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2154066f30 0x7f21540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.055+0000 7f216affd700 1 -- 172.21.15.131:0/2020778040 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f216c084b80 con 0x7f2174159a70 2023-12-26T21:55:33.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.055+0000 7f2172d9d700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2154066f30 0x7f21540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.056+0000 7f2172d9d700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2154066f30 0x7f21540693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2164003870 tx=0x7f2164009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:33.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.059+0000 7f216affd700 1 -- 172.21.15.131:0/2020778040 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21740d0600 con 0x7f2174159a70 2023-12-26T21:55:33.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.261+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21740d0600 con 0x7f2154066f30 2023-12-26T21:55:33.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.271+0000 7f216affd700 1 -- 172.21.15.131:0/2020778040 <== mgr.7159 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278507 (secure 0 0 0) 0x7f21740d0600 con 0x7f2154066f30 2023-12-26T21:55:33.275 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:33.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.274+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2154066f30 msgr2=0x7f21540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.274+0000 7f21791ff700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2154066f30 0x7f21540693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2164003870 tx=0x7f2164009040 comp rx=0 tx=0).stop 2023-12-26T21:55:33.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.274+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2174159a70 msgr2=0x7f217416d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.274+0000 7f21791ff700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2174159a70 0x7f217416d2e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f216c00cc60 tx=0x7f216c001ae0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.275+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 shutdown_connections 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.275+0000 7f21791ff700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2154066f30 0x7f21540693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.275+0000 7f21791ff700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2174159a70 0x7f217416d2e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.275+0000 7f21791ff700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21740b2a90 0x7f2174155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.275+0000 7f21791ff700 1 --2- 172.21.15.131:0/2020778040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21741596a0 0x7f21740c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.275+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 >> 172.21.15.131:0/2020778040 conn(0x7f217406ece0 msgr2=0x7f2174156390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.275+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 shutdown_connections 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.275+0000 7f21791ff700 1 -- 172.21.15.131:0/2020778040 wait complete. 2023-12-26T21:55:33.278 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:55:33.294 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:55:32.884543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452520,"kb_used_data":12776,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752522200,"statfs":{"total":773094113280,"available":770582732800,"internally_reserved":0,"allocated":13082624,"data_stored":6977424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.975447"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913253+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:42.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.918996+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:07.284268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911558+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:21:46.237715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915152+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:13:41.710446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916671+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:41:07.987957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911553+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:43:35.363935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.918930+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:59:34.209365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915447+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:33:02.690125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911486+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:05.564372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916740+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:04:26.503604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.919481+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:37:09.996669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911381+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:45.785072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916672+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:42.197805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916100+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:16:34.758852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911432+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:51:22.466513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915267+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:28:21.708669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911489+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:11.586387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.919052+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:46:40.098721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916158+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:00.846344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916082+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:18.461025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.914095+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:16:51.694872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.946132+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:08.710785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913046+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:15:03.359532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911437+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:15:32.157127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915893+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:37:16.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916812+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:34:05.377326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913601+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:25.800001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911658+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:36:49.479164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913584+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:08:48.836374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913370+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:43:35.363935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911486+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:42.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911896+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:03:02.206304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"113'227","reported_seq":642,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-12-26T21:55:25.150969+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:55:25.150969+0000","last_peered":"2023-12-26T21:55:25.150969+0000","last_clean":"2023-12-26T21:55:25.150969+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:55:25.150969+0000","last_undegraded":"2023-12-26T21:55:25.150969+0000","last_fullsized":"2023-12-26T21:55:25.150969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542366,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.51200000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90700000000000003,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"max":{"1min":0.82199999999999995,"5min":0.94099999999999995,"15min":0.98299999999999998},"last":0.77400000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.67400000000000004},"min":{"1min":0.53000000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.79500000000000004,"5min":0.85899999999999999,"15min":1},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.54000000000000004,"5min":0.436,"15min":0.34200000000000003},"max":{"1min":0.96899999999999997,"5min":1.014,"15min":1.048},"last":0.70899999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":1.0509999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66900000000000004,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.434},"max":{"1min":0.88900000000000001,"5min":0.99199999999999999,"15min":1.02},"last":0.73099999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94399999999999995,"5min":0.94499999999999995,"15min":1.052},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.59699999999999998,"5min":0.48299999999999998,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.80300000000000005}]},{"osd":4,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.753,"5min":1.036,"15min":1.036},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84299999999999997,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.51200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.95599999999999996,"5min":1.117,"15min":1.117},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.44},"max":{"1min":0.998,"5min":1.006,"15min":1.006},"last":0.68799999999999994}]}]},{"osd":6,"up_from":15,"seq":64424509662,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.55500000000000005,"5min":0.48999999999999999,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.439}]},{"osd":1,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.60699999999999998,"5min":0.46100000000000002,"15min":0.40200000000000002},"max":{"1min":0.85399999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.496}]},{"osd":2,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.70699999999999996},"min":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.89200000000000002,"5min":0.93200000000000005,"15min":0.97099999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69199999999999995,"15min":0.70699999999999996},"min":{"1min":0.54600000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.65900000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90900000000000003,"5min":1.026,"15min":1.026},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.751},"min":{"1min":0.64000000000000001,"5min":0.46200000000000002,"15min":0.42799999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.0129999999999999},"last":0.74399999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.42399999999999999,"15min":0.372},"max":{"1min":0.82299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.48299999999999998,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.82099999999999995,"5min":0.91200000000000003,"15min":1.0529999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.97299999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.874,"5min":1.0049999999999999,"15min":1.2609999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.54200000000000004,"5min":0.47099999999999997,"15min":0.438},"max":{"1min":0.98599999999999999,"5min":1.008,"15min":1.0920000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0920000000000001},"last":0.80600000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509662,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311608,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060232,"statfs":{"total":96636764160,"available":96317677568,"internally_reserved":0,"allocated":2080768,"data_stored":1314954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.30499999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.628,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.42999999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.72799999999999998,"5min":1.353,"15min":1.353},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68300000000000005,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":1.488,"15min":1.488},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.42399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79300000000000004,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":1.506,"15min":1.506},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71199999999999997,"15min":0.746},"min":{"1min":0.58199999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0509999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76500000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77300000000000002},"min":{"1min":0.58199999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.032,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76200000000000001,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.54400000000000004},"max":{"1min":1.0529999999999999,"5min":1.615,"15min":1.615},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.77900000000000003},"min":{"1min":0.57899999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.075,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.04,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.56999999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509662,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.245},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.37},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.29799999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.48999999999999999},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.30599999999999999},"max":{"1min":0.621,"5min":0.66800000000000004,"15min":0.73099999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.52200000000000002},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.38}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.317,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84699999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.44,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86699999999999999},"last":0.505}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.72099999999999997,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.2749999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47699999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77600000000000002,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.153},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.745,"15min":0.77900000000000003},"min":{"1min":0.56899999999999995,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.0700000000000001,"5min":1.093,"15min":1.214},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.749,"15min":0.79000000000000004},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.51700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.624}]}]},{"osd":2,"up_from":14,"seq":60129542367,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61099999999999999,"5min":0.70399999999999996,"15min":0.85999999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.48399999999999999},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.35399999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.60499999999999998},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78200000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.58099999999999996,"5min":0.51300000000000001,"15min":0.38300000000000001},"max":{"1min":1.155,"5min":1.3,"15min":1.3},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.49199999999999999,"15min":0.48899999999999999},"max":{"1min":1.014,"5min":1.048,"15min":1.1759999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.1000000000000001,"5min":1.25,"15min":1.25},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.002,"5min":1.014,"15min":1.1319999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.53900000000000003,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.202,"15min":1.272},"last":0.70699999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.69499999999999995,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1220000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.54300000000000004},"max":{"1min":1.1659999999999999,"5min":1.232,"15min":1.232},"last":0.67300000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509663,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311028,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060812,"statfs":{"total":96636764160,"available":96318271488,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66200000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.63400000000000001,"5min":1.238,"15min":1.238},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":1.468,"15min":1.468},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.65700000000000003,"5min":1.756,"15min":1.756},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.52700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.83299999999999996,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.51000000000000001,"5min":0.42299999999999999,"15min":0.42199999999999999},"max":{"1min":0.80800000000000005,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.69999999999999996}]},{"osd":4,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.63800000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.0029999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.022},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.61199999999999999,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.88900000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.069},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.61899999999999999,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1399999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.79900000000000004},"min":{"1min":0.65900000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.056,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.70699999999999996,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.071,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.038}]},{"osd":7,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.1659999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.101,"5min":1.272,"15min":1.272},"last":0.88}]}]},{"osd":4,"up_from":14,"seq":60129542366,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67300000000000004,"15min":0.68400000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.371},"max":{"1min":0.91700000000000004,"5min":1.006,"15min":1.008},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.311},"max":{"1min":1.0229999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.628},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40000000000000002},"max":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92900000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97199999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.55200000000000005,"5min":0.46000000000000002,"15min":0.38800000000000001},"max":{"1min":0.875,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.45300000000000001},"max":{"1min":0.90400000000000003,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.753},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.442,"15min":0.438},"max":{"1min":0.98799999999999999,"5min":1.046,"15min":1.046},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84499999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83499999999999996,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.441,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.88600000000000001,"5min":1.133,"15min":1.133},"last":0.88700000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":0.88600000000000001,"5min":1.002,"15min":1.002},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.58399999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89800000000000002,"5min":1.266,"15min":1.266},"last":0.84799999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509661,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":1314954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.68400000000000005},"min":{"1min":0.48199999999999998,"5min":0.39800000000000002,"15min":0.27300000000000002},"max":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.46800000000000003,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.91600000000000004,"15min":1.0369999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.69699999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.34000000000000002},"max":{"1min":0.88400000000000001,"5min":0.96399999999999997,"15min":1.04},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66800000000000004,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.443,"15min":0.36899999999999999},"max":{"1min":0.86899999999999999,"5min":1.002,"15min":1.01},"last":0.57599999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67800000000000005,"15min":0.71099999999999997},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.88500000000000001,"15min":1.3029999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.52400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.92400000000000004,"15min":0.95699999999999996},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.71999999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.39200000000000002},"max":{"1min":0.90500000000000003,"5min":0.93000000000000005,"15min":1.0640000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.47799999999999998},"max":{"1min":0.85499999999999998,"5min":0.92200000000000004,"15min":0.93899999999999995},"last":0.71499999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.79300000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.41899999999999998,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.01},"last":0.77300000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.65000000000000002},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35499999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46899999999999997},"max":{"1min":0.85499999999999998,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67100000000000004,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95099999999999996,"15min":0.96299999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68700000000000006,"15min":0.69799999999999995},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.95399999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:55:33.300 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:55:33.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.522+0000 7fd9969ff700 1 Processor -- start 2023-12-26T21:55:33.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.522+0000 7fd9969ff700 1 -- start start 2023-12-26T21:55:33.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd9969ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd9900b0750 0x7fd9900b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd9969ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900c9400 0x7fd9900c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd9969ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd990156940 con 0x7fd9900cb320 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd9969ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9900c8cc0 con 0x7fd9900b0750 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd9969ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900ad700 con 0x7fd9900c9400 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd98f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd9900b0750 0x7fd9900b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd994f9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900c9400 0x7fd9900c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd98f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd9900b0750 0x7fd9900b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:54418/0 (socket says 172.21.15.131:54418) 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd98f7fe700 1 -- 172.21.15.131:0/1065046924 learned_addr learned my addr 172.21.15.131:0/1065046924 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd994f9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900c9400 0x7fd9900c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:45628/0 (socket says 172.21.15.131:45628) 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.524+0000 7fd98ffff700 1 --1- 172.21.15.131:0/1065046924 >> v1:172.21.15.190:6789/0 conn(0x7fd9900cb320 0x7fd990069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:45166/0 (socket says 172.21.15.131:45166) 2023-12-26T21:55:33.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.525+0000 7fd98f7fe700 1 -- 172.21.15.131:0/1065046924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900c9400 msgr2=0x7fd9900c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.525+0000 7fd98f7fe700 1 --2- 172.21.15.131:0/1065046924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900c9400 0x7fd9900c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.525+0000 7fd98f7fe700 1 -- 172.21.15.131:0/1065046924 >> v1:172.21.15.190:6789/0 conn(0x7fd9900cb320 legacy=0x7fd990069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.525+0000 7fd98f7fe700 1 -- 172.21.15.131:0/1065046924 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9900ae500 con 0x7fd9900b0750 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.525+0000 7fd994f9c700 1 --2- 172.21.15.131:0/1065046924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900c9400 0x7fd9900c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.525+0000 7fd98f7fe700 1 --2- 172.21.15.131:0/1065046924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9900b0750 0x7fd9900b0bf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd9840099b0 tx=0x7fd984009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed5f375456ed7ee server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.525+0000 7fd98effd700 1 -- 172.21.15.131:0/1065046924 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984009ea0 con 0x7fd9900b0750 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.525+0000 7fd98effd700 1 -- 172.21.15.131:0/1065046924 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98400b860 con 0x7fd9900b0750 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.525+0000 7fd98effd700 1 -- 172.21.15.131:0/1065046924 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98400bc00 con 0x7fd9900b0750 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.526+0000 7fd9969ff700 1 -- 172.21.15.131:0/1065046924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9900b0750 msgr2=0x7fd9900b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.526+0000 7fd9969ff700 1 --2- 172.21.15.131:0/1065046924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9900b0750 0x7fd9900b0bf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd9840099b0 tx=0x7fd984009250 comp rx=0 tx=0).stop 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.526+0000 7fd9969ff700 1 -- 172.21.15.131:0/1065046924 shutdown_connections 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.526+0000 7fd9969ff700 1 --2- 172.21.15.131:0/1065046924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900c9400 0x7fd9900c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.526+0000 7fd9969ff700 1 --2- 172.21.15.131:0/1065046924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9900b0750 0x7fd9900b0bf0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.526+0000 7fd9969ff700 1 -- 172.21.15.131:0/1065046924 >> 172.21.15.131:0/1065046924 conn(0x7fd990068040 msgr2=0x7fd990069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:33.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.526+0000 7fd9969ff700 1 -- 172.21.15.131:0/1065046924 shutdown_connections 2023-12-26T21:55:33.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.526+0000 7fd9969ff700 1 -- 172.21.15.131:0/1065046924 wait complete. 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.527+0000 7fd9969ff700 1 Processor -- start 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.528+0000 7fd9969ff700 1 -- start start 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.528+0000 7fd9969ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900b0750 0x7fd9901543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.528+0000 7fd9969ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9900c9400 0x7fd9900ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.528+0000 7fd9969ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9900cb320 0x7fd99015f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.528+0000 7fd9969ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900acbe0 con 0x7fd9900cb320 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.528+0000 7fd9969ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900ab330 con 0x7fd9900c9400 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.528+0000 7fd9969ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900ab7c0 con 0x7fd9900b0750 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd994f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9900cb320 0x7fd99015f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd98ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900b0750 0x7fd9901543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd994f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9900cb320 0x7fd99015f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46772/0 (socket says 172.21.15.131:46772) 2023-12-26T21:55:33.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd994f9c700 1 -- 172.21.15.131:0/3544719030 learned_addr learned my addr 172.21.15.131:0/3544719030 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd98f7fe700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9900c9400 0x7fd9900ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd994f9c700 1 -- 172.21.15.131:0/3544719030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900b0750 msgr2=0x7fd9901543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd994f9c700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900b0750 0x7fd9901543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd994f9c700 1 -- 172.21.15.131:0/3544719030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9900c9400 msgr2=0x7fd9900ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd994f9c700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9900c9400 0x7fd9900ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd994f9c700 1 -- 172.21.15.131:0/3544719030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988009a20 con 0x7fd9900cb320 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.529+0000 7fd994f9c700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9900cb320 0x7fd99015f5b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd988009fc0 tx=0x7fd98800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.530+0000 7fd977fff700 1 -- 172.21.15.131:0/3544719030 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98800cbd0 con 0x7fd9900cb320 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.530+0000 7fd977fff700 1 -- 172.21.15.131:0/3544719030 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98800f040 con 0x7fd9900cb320 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.530+0000 7fd977fff700 1 -- 172.21.15.131:0/3544719030 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988014480 con 0x7fd9900cb320 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.530+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9840094f0 con 0x7fd9900cb320 2023-12-26T21:55:33.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.530+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9900aace0 con 0x7fd9900cb320 2023-12-26T21:55:33.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.531+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9900657e0 con 0x7fd9900cb320 2023-12-26T21:55:33.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.532+0000 7fd977fff700 1 -- 172.21.15.131:0/3544719030 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd988003020 con 0x7fd9900cb320 2023-12-26T21:55:33.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.533+0000 7fd977fff700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd97c066f30 0x7fd97c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.533+0000 7fd977fff700 1 -- 172.21.15.131:0/3544719030 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fd988084fe0 con 0x7fd9900cb320 2023-12-26T21:55:33.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.533+0000 7fd98ffff700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd97c066f30 0x7fd97c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.534+0000 7fd98ffff700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd97c066f30 0x7fd97c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd980004f40 tx=0x7fd980007eb0 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:33.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.537+0000 7fd977fff700 1 -- 172.21.15.131:0/3544719030 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9880520c0 con 0x7fd9900cb320 2023-12-26T21:55:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.727+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd990154930 con 0x7fd97c066f30 2023-12-26T21:55:33.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.735+0000 7fd977fff700 1 -- 172.21.15.131:0/3544719030 <== mgr.7159 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278507 (secure 0 0 0) 0x7fd990154930 con 0x7fd97c066f30 2023-12-26T21:55:33.740 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:33.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.739+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd97c066f30 msgr2=0x7fd97c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.739+0000 7fd9969ff700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd97c066f30 0x7fd97c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd980004f40 tx=0x7fd980007eb0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.739+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9900cb320 msgr2=0x7fd99015f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.739+0000 7fd9969ff700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9900cb320 0x7fd99015f5b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd988009fc0 tx=0x7fd98800b040 comp rx=0 tx=0).stop 2023-12-26T21:55:33.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.740+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 shutdown_connections 2023-12-26T21:55:33.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.740+0000 7fd9969ff700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd97c066f30 0x7fd97c0693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.740+0000 7fd9969ff700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9900cb320 0x7fd99015f5b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.740+0000 7fd9969ff700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9900b0750 0x7fd9901543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.740+0000 7fd9969ff700 1 --2- 172.21.15.131:0/3544719030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9900c9400 0x7fd9900ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.740+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 >> 172.21.15.131:0/3544719030 conn(0x7fd990068040 msgr2=0x7fd99006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:33.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.740+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 shutdown_connections 2023-12-26T21:55:33.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.740+0000 7fd9969ff700 1 -- 172.21.15.131:0/3544719030 wait complete. 2023-12-26T21:55:33.743 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:55:33.759 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:55:32.884543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452520,"kb_used_data":12776,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752522200,"statfs":{"total":773094113280,"available":770582732800,"internally_reserved":0,"allocated":13082624,"data_stored":6977424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.975447"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913253+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:42.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.918996+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:07.284268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911558+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:21:46.237715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915152+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:13:41.710446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916671+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:41:07.987957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911553+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:43:35.363935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.918930+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:59:34.209365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915447+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:33:02.690125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911486+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:05.564372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916740+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:04:26.503604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.919481+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:37:09.996669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911381+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:45.785072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916672+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:42.197805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916100+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:16:34.758852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911432+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:51:22.466513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915267+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:28:21.708669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911489+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:11.586387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.919052+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:46:40.098721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916158+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:00.846344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916082+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:18.461025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.914095+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:16:51.694872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.946132+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:08.710785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913046+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:15:03.359532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911437+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:15:32.157127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915893+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:37:16.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916812+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:34:05.377326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913601+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:25.800001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911658+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:36:49.479164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913584+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:08:48.836374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913370+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:43:35.363935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911486+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:42.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911896+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:03:02.206304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"113'227","reported_seq":642,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-12-26T21:55:25.150969+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:55:25.150969+0000","last_peered":"2023-12-26T21:55:25.150969+0000","last_clean":"2023-12-26T21:55:25.150969+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:55:25.150969+0000","last_undegraded":"2023-12-26T21:55:25.150969+0000","last_fullsized":"2023-12-26T21:55:25.150969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542366,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.51200000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90700000000000003,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"max":{"1min":0.82199999999999995,"5min":0.94099999999999995,"15min":0.98299999999999998},"last":0.77400000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.67400000000000004},"min":{"1min":0.53000000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.79500000000000004,"5min":0.85899999999999999,"15min":1},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.54000000000000004,"5min":0.436,"15min":0.34200000000000003},"max":{"1min":0.96899999999999997,"5min":1.014,"15min":1.048},"last":0.70899999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":1.0509999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66900000000000004,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.434},"max":{"1min":0.88900000000000001,"5min":0.99199999999999999,"15min":1.02},"last":0.73099999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94399999999999995,"5min":0.94499999999999995,"15min":1.052},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.59699999999999998,"5min":0.48299999999999998,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.80300000000000005}]},{"osd":4,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.753,"5min":1.036,"15min":1.036},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84299999999999997,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.51200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.95599999999999996,"5min":1.117,"15min":1.117},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.44},"max":{"1min":0.998,"5min":1.006,"15min":1.006},"last":0.68799999999999994}]}]},{"osd":6,"up_from":15,"seq":64424509662,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.55500000000000005,"5min":0.48999999999999999,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.439}]},{"osd":1,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.60699999999999998,"5min":0.46100000000000002,"15min":0.40200000000000002},"max":{"1min":0.85399999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.496}]},{"osd":2,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.70699999999999996},"min":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.89200000000000002,"5min":0.93200000000000005,"15min":0.97099999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69199999999999995,"15min":0.70699999999999996},"min":{"1min":0.54600000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.65900000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90900000000000003,"5min":1.026,"15min":1.026},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.751},"min":{"1min":0.64000000000000001,"5min":0.46200000000000002,"15min":0.42799999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.0129999999999999},"last":0.74399999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.42399999999999999,"15min":0.372},"max":{"1min":0.82299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.48299999999999998,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.82099999999999995,"5min":0.91200000000000003,"15min":1.0529999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.97299999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.874,"5min":1.0049999999999999,"15min":1.2609999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.54200000000000004,"5min":0.47099999999999997,"15min":0.438},"max":{"1min":0.98599999999999999,"5min":1.008,"15min":1.0920000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0920000000000001},"last":0.80600000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509662,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311608,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060232,"statfs":{"total":96636764160,"available":96317677568,"internally_reserved":0,"allocated":2080768,"data_stored":1314954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.30499999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.628,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.42999999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.72799999999999998,"5min":1.353,"15min":1.353},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68300000000000005,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":1.488,"15min":1.488},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.42399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79300000000000004,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":1.506,"15min":1.506},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71199999999999997,"15min":0.746},"min":{"1min":0.58199999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0509999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76500000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77300000000000002},"min":{"1min":0.58199999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.032,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76200000000000001,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.54400000000000004},"max":{"1min":1.0529999999999999,"5min":1.615,"15min":1.615},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.77900000000000003},"min":{"1min":0.57899999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.075,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.04,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.56999999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509662,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.245},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.37},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.29799999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.48999999999999999},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.30599999999999999},"max":{"1min":0.621,"5min":0.66800000000000004,"15min":0.73099999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.52200000000000002},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.38}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.317,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84699999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.44,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86699999999999999},"last":0.505}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.72099999999999997,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.2749999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47699999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77600000000000002,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.153},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.745,"15min":0.77900000000000003},"min":{"1min":0.56899999999999995,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.0700000000000001,"5min":1.093,"15min":1.214},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.749,"15min":0.79000000000000004},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.51700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.624}]}]},{"osd":2,"up_from":14,"seq":60129542367,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61099999999999999,"5min":0.70399999999999996,"15min":0.85999999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.48399999999999999},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.35399999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.60499999999999998},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78200000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.58099999999999996,"5min":0.51300000000000001,"15min":0.38300000000000001},"max":{"1min":1.155,"5min":1.3,"15min":1.3},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.49199999999999999,"15min":0.48899999999999999},"max":{"1min":1.014,"5min":1.048,"15min":1.1759999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.1000000000000001,"5min":1.25,"15min":1.25},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.002,"5min":1.014,"15min":1.1319999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.53900000000000003,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.202,"15min":1.272},"last":0.70699999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.69499999999999995,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1220000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.54300000000000004},"max":{"1min":1.1659999999999999,"5min":1.232,"15min":1.232},"last":0.67300000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509663,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311028,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060812,"statfs":{"total":96636764160,"available":96318271488,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66200000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.63400000000000001,"5min":1.238,"15min":1.238},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":1.468,"15min":1.468},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.65700000000000003,"5min":1.756,"15min":1.756},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.52700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.83299999999999996,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.51000000000000001,"5min":0.42299999999999999,"15min":0.42199999999999999},"max":{"1min":0.80800000000000005,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.69999999999999996}]},{"osd":4,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.63800000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.0029999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.022},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.61199999999999999,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.88900000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.069},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.61899999999999999,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1399999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.79900000000000004},"min":{"1min":0.65900000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.056,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.70699999999999996,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.071,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.038}]},{"osd":7,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.1659999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.101,"5min":1.272,"15min":1.272},"last":0.88}]}]},{"osd":4,"up_from":14,"seq":60129542366,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67300000000000004,"15min":0.68400000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.371},"max":{"1min":0.91700000000000004,"5min":1.006,"15min":1.008},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.311},"max":{"1min":1.0229999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.628},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40000000000000002},"max":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92900000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97199999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.55200000000000005,"5min":0.46000000000000002,"15min":0.38800000000000001},"max":{"1min":0.875,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.45300000000000001},"max":{"1min":0.90400000000000003,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.753},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.442,"15min":0.438},"max":{"1min":0.98799999999999999,"5min":1.046,"15min":1.046},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84499999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83499999999999996,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.441,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.88600000000000001,"5min":1.133,"15min":1.133},"last":0.88700000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":0.88600000000000001,"5min":1.002,"15min":1.002},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.58399999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89800000000000002,"5min":1.266,"15min":1.266},"last":0.84799999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509661,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":1314954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.68400000000000005},"min":{"1min":0.48199999999999998,"5min":0.39800000000000002,"15min":0.27300000000000002},"max":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.46800000000000003,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.91600000000000004,"15min":1.0369999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.69699999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.34000000000000002},"max":{"1min":0.88400000000000001,"5min":0.96399999999999997,"15min":1.04},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66800000000000004,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.443,"15min":0.36899999999999999},"max":{"1min":0.86899999999999999,"5min":1.002,"15min":1.01},"last":0.57599999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67800000000000005,"15min":0.71099999999999997},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.88500000000000001,"15min":1.3029999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.52400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.92400000000000004,"15min":0.95699999999999996},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.71999999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.39200000000000002},"max":{"1min":0.90500000000000003,"5min":0.93000000000000005,"15min":1.0640000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.47799999999999998},"max":{"1min":0.85499999999999998,"5min":0.92200000000000004,"15min":0.93899999999999995},"last":0.71499999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.79300000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.41899999999999998,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.01},"last":0.77300000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.65000000000000002},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35499999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46899999999999997},"max":{"1min":0.85499999999999998,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67100000000000004,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95099999999999996,"15min":0.96299999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68700000000000006,"15min":0.69799999999999995},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.95399999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:55:33.764 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:55:33.902 INFO:tasks.ceph.mds.c.smithi131.stderr:2023-12-26T21:55:33.901+0000 7f72bbdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:55:33.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.991+0000 7f7352bff700 1 Processor -- start 2023-12-26T21:55:33.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.991+0000 7f7352bff700 1 -- start start 2023-12-26T21:55:33.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.992+0000 7f7352bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f734c06bfd0 0x7f734c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.992+0000 7f7352bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 0x7f734c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.992+0000 7f7352bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c0686c0 con 0x7f734c13c510 2023-12-26T21:55:33.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.992+0000 7f7352bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c069520 con 0x7f734c06bfd0 2023-12-26T21:55:33.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.992+0000 7f7352bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c068910 con 0x7f734c0cc100 2023-12-26T21:55:33.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f734bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f734c06bfd0 0x7f734c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f735099b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f734c13c510 0x7f734c06b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36480/0 (socket says 172.21.15.131:36480) 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f735099b700 1 -- 172.21.15.131:0/1858758569 learned_addr learned my addr 172.21.15.131:0/1858758569 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f735119c700 1 --2- 172.21.15.131:0/1858758569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 0x7f734c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f734b7fe700 1 -- 172.21.15.131:0/1858758569 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734458370 0 0) 0x7f734c0686c0 con 0x7f734c13c510 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f734bfff700 1 -- 172.21.15.131:0/1858758569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 msgr2=0x7f734c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f734bfff700 1 --2- 172.21.15.131:0/1858758569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 0x7f734c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f734bfff700 1 -- 172.21.15.131:0/1858758569 >> v1:172.21.15.131:6789/0 conn(0x7f734c13c510 legacy=0x7f734c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f734bfff700 1 -- 172.21.15.131:0/1858758569 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c068bf0 con 0x7f734c06bfd0 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f735119c700 1 --2- 172.21.15.131:0/1858758569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 0x7f734c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.993+0000 7f734bfff700 1 --2- 172.21.15.131:0/1858758569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c06bfd0 0x7f734c0c8a20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f73380098e0 tx=0x7f73380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b36058b22a8bb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f734b7fe700 1 -- 172.21.15.131:0/1858758569 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338009e80 con 0x7f734c06bfd0 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f734b7fe700 1 -- 172.21.15.131:0/1858758569 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733800b860 con 0x7f734c06bfd0 2023-12-26T21:55:33.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f734b7fe700 1 -- 172.21.15.131:0/1858758569 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733800bbe0 con 0x7f734c06bfd0 2023-12-26T21:55:33.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f7352bff700 1 -- 172.21.15.131:0/1858758569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c06bfd0 msgr2=0x7f734c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:33.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f7352bff700 1 --2- 172.21.15.131:0/1858758569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c06bfd0 0x7f734c0c8a20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f73380098e0 tx=0x7f73380091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f7352bff700 1 -- 172.21.15.131:0/1858758569 shutdown_connections 2023-12-26T21:55:33.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f7352bff700 1 --2- 172.21.15.131:0/1858758569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 0x7f734c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f7352bff700 1 --2- 172.21.15.131:0/1858758569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c06bfd0 0x7f734c0c8a20 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:33.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f7352bff700 1 -- 172.21.15.131:0/1858758569 >> 172.21.15.131:0/1858758569 conn(0x7f734c0ce3e0 msgr2=0x7f734c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:33.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f7352bff700 1 -- 172.21.15.131:0/1858758569 shutdown_connections 2023-12-26T21:55:33.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.994+0000 7f7352bff700 1 -- 172.21.15.131:0/1858758569 wait complete. 2023-12-26T21:55:33.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.996+0000 7f7352bff700 1 Processor -- start 2023-12-26T21:55:33.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.996+0000 7f7352bff700 1 -- start start 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f7352bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c06bfd0 0x7f734c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f7352bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 0x7f734c0c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f7352bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c13c510 0x7f734c15d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f7352bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0c6f60 con 0x7f734c13c510 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f7352bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c13d340 con 0x7f734c06bfd0 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f7352bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c15c850 con 0x7f734c0cc100 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735119c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c13c510 0x7f734c15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735119c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c13c510 0x7f734c15d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46788/0 (socket says 172.21.15.131:46788) 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735119c700 1 -- 172.21.15.131:0/1886332392 learned_addr learned my addr 172.21.15.131:0/1886332392 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f734bfff700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 0x7f734c0c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735099b700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c06bfd0 0x7f734c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735119c700 1 -- 172.21.15.131:0/1886332392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 msgr2=0x7f734c0c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735119c700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 0x7f734c0c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735119c700 1 -- 172.21.15.131:0/1886332392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c06bfd0 msgr2=0x7f734c15c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735119c700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c06bfd0 0x7f734c15c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735119c700 1 -- 172.21.15.131:0/1886332392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7338009420 con 0x7f734c13c510 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.997+0000 7f735099b700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c06bfd0 0x7f734c15c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.998+0000 7f735119c700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c13c510 0x7f734c15d750 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f734400a380 tx=0x7f7344007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.998+0000 7f7348ff9700 1 -- 172.21.15.131:0/1886332392 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344015440 con 0x7f734c13c510 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.998+0000 7f7348ff9700 1 -- 172.21.15.131:0/1886332392 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73440155a0 con 0x7f734c13c510 2023-12-26T21:55:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.998+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c15a220 con 0x7f734c13c510 2023-12-26T21:55:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.998+0000 7f7348ff9700 1 -- 172.21.15.131:0/1886332392 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344015960 con 0x7f734c13c510 2023-12-26T21:55:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.998+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c15a7e0 con 0x7f734c13c510 2023-12-26T21:55:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:33.999+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c13d4b0 con 0x7f734c13c510 2023-12-26T21:55:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.000+0000 7f7348ff9700 1 -- 172.21.15.131:0/1886332392 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f734400d440 con 0x7f734c13c510 2023-12-26T21:55:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.001+0000 7f7348ff9700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.001+0000 7f7348ff9700 1 -- 172.21.15.131:0/1886332392 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7344085250 con 0x7f734c13c510 2023-12-26T21:55:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.001+0000 7f735099b700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.001+0000 7f735099b700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7330066f30 0x7f73300693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7340001360 tx=0x7f7340009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.004+0000 7f7348ff9700 1 -- 172.21.15.131:0/1886332392 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7344052330 con 0x7f734c13c510 2023-12-26T21:55:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.194+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f734c0657e0 con 0x7f7330066f30 2023-12-26T21:55:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.202+0000 7f7348ff9700 1 -- 172.21.15.131:0/1886332392 <== mgr.7159 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278507 (secure 0 0 0) 0x7f734c0657e0 con 0x7f7330066f30 2023-12-26T21:55:34.206 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.206+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7330066f30 msgr2=0x7f73300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.206+0000 7f7352bff700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7330066f30 0x7f73300693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7340001360 tx=0x7f7340009040 comp rx=0 tx=0).stop 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.206+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c13c510 msgr2=0x7f734c15d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.207+0000 7f7352bff700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c13c510 0x7f734c15d750 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f734400a380 tx=0x7f7344007900 comp rx=0 tx=0).stop 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.207+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 shutdown_connections 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.207+0000 7f7352bff700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.207+0000 7f7352bff700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f734c13c510 0x7f734c15d750 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.207+0000 7f7352bff700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c06bfd0 0x7f734c15c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.207+0000 7f7352bff700 1 --2- 172.21.15.131:0/1886332392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f734c0cc100 0x7f734c0c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.207+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 >> 172.21.15.131:0/1886332392 conn(0x7f734c0ce3e0 msgr2=0x7f734c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:34.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.207+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 shutdown_connections 2023-12-26T21:55:34.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.207+0000 7f7352bff700 1 -- 172.21.15.131:0/1886332392 wait complete. 2023-12-26T21:55:34.210 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:55:34.225 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T21:55:32.884543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452520,"kb_used_data":12776,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752522200,"statfs":{"total":773094113280,"available":770582732800,"internally_reserved":0,"allocated":13082624,"data_stored":6977424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.975447"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913253+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:42.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.918996+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:07.284268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911558+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:21:46.237715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915152+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:13:41.710446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916671+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:41:07.987957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911553+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:43:35.363935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.918930+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:59:34.209365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915447+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:33:02.690125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911486+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:05.564372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916740+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:04:26.503604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.919481+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:37:09.996669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911381+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:45.785072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916672+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:42.197805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916100+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:16:34.758852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911432+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:51:22.466513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915267+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:28:21.708669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911489+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:24:11.586387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.919052+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:46:40.098721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916158+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:00.846344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916082+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:23:18.461025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.914095+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:16:51.694872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.946132+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:08.710785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913046+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:15:03.359532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911437+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:15:32.157127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.915893+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:37:16.634357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.916812+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:34:05.377326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913601+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:25.800001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911658+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:36:49.479164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913584+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:08:48.836374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.913370+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:43:35.363935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911486+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:42.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T21:55:27.908991+0000","last_change":"2023-12-26T21:55:27.908991+0000","last_active":"2023-12-26T21:55:27.908991+0000","last_peered":"2023-12-26T21:55:27.908991+0000","last_clean":"2023-12-26T21:55:27.908991+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:27.908991+0000","last_undegraded":"2023-12-26T21:55:27.908991+0000","last_fullsized":"2023-12-26T21:55:27.908991+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T21:55:26.902820+0000","last_change":"2023-12-26T21:55:26.911896+0000","last_active":"2023-12-26T21:55:26.902820+0000","last_peered":"2023-12-26T21:55:26.902820+0000","last_clean":"2023-12-26T21:55:26.902820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T21:55:26.902820+0000","last_undegraded":"2023-12-26T21:55:26.902820+0000","last_fullsized":"2023-12-26T21:55:26.902820+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:03:02.206304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"113'227","reported_seq":642,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-12-26T21:55:25.150969+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:55:25.150969+0000","last_peered":"2023-12-26T21:55:25.150969+0000","last_clean":"2023-12-26T21:55:25.150969+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:55:25.150969+0000","last_undegraded":"2023-12-26T21:55:25.150969+0000","last_fullsized":"2023-12-26T21:55:25.150969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542366,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.51200000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90700000000000003,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"max":{"1min":0.82199999999999995,"5min":0.94099999999999995,"15min":0.98299999999999998},"last":0.77400000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.67400000000000004},"min":{"1min":0.53000000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.79500000000000004,"5min":0.85899999999999999,"15min":1},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.54000000000000004,"5min":0.436,"15min":0.34200000000000003},"max":{"1min":0.96899999999999997,"5min":1.014,"15min":1.048},"last":0.70899999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":1.0509999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66900000000000004,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.434},"max":{"1min":0.88900000000000001,"5min":0.99199999999999999,"15min":1.02},"last":0.73099999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94399999999999995,"5min":0.94499999999999995,"15min":1.052},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.59699999999999998,"5min":0.48299999999999998,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.80300000000000005}]},{"osd":4,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.753,"5min":1.036,"15min":1.036},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84299999999999997,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.51200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.95599999999999996,"5min":1.117,"15min":1.117},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.44},"max":{"1min":0.998,"5min":1.006,"15min":1.006},"last":0.68799999999999994}]}]},{"osd":6,"up_from":15,"seq":64424509662,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.55500000000000005,"5min":0.48999999999999999,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.439}]},{"osd":1,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.60699999999999998,"5min":0.46100000000000002,"15min":0.40200000000000002},"max":{"1min":0.85399999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.496}]},{"osd":2,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.70699999999999996},"min":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.89200000000000002,"5min":0.93200000000000005,"15min":0.97099999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69199999999999995,"15min":0.70699999999999996},"min":{"1min":0.54600000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":3,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.65900000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90900000000000003,"5min":1.026,"15min":1.026},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.751},"min":{"1min":0.64000000000000001,"5min":0.46200000000000002,"15min":0.42799999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.0129999999999999},"last":0.74399999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.42399999999999999,"15min":0.372},"max":{"1min":0.82299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.48299999999999998,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.82099999999999995,"5min":0.91200000000000003,"15min":1.0529999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.97299999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.874,"5min":1.0049999999999999,"15min":1.2609999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.54200000000000004,"5min":0.47099999999999997,"15min":0.438},"max":{"1min":0.98599999999999999,"5min":1.008,"15min":1.0920000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0920000000000001},"last":0.80600000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509662,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311608,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060232,"statfs":{"total":96636764160,"available":96317677568,"internally_reserved":0,"allocated":2080768,"data_stored":1314954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.30499999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.628,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.42999999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.72799999999999998,"5min":1.353,"15min":1.353},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68300000000000005,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":1.488,"15min":1.488},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.42399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79300000000000004,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":1.506,"15min":1.506},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71199999999999997,"15min":0.746},"min":{"1min":0.58199999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0509999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76500000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77300000000000002},"min":{"1min":0.58199999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.032,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76200000000000001,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.54400000000000004},"max":{"1min":1.0529999999999999,"5min":1.615,"15min":1.615},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.77900000000000003},"min":{"1min":0.57899999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.075,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.04,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.56999999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509662,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.245},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.37},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.29799999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.48999999999999999},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.30599999999999999},"max":{"1min":0.621,"5min":0.66800000000000004,"15min":0.73099999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.52200000000000002},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.38}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.317,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84699999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.44,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86699999999999999},"last":0.505}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.72099999999999997,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.2749999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47699999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77600000000000002,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.153},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.745,"15min":0.77900000000000003},"min":{"1min":0.56899999999999995,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.0700000000000001,"5min":1.093,"15min":1.214},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.749,"15min":0.79000000000000004},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.51700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.624}]}]},{"osd":2,"up_from":14,"seq":60129542367,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61099999999999999,"5min":0.70399999999999996,"15min":0.85999999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.48399999999999999},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.35399999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.60499999999999998},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78200000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.58099999999999996,"5min":0.51300000000000001,"15min":0.38300000000000001},"max":{"1min":1.155,"5min":1.3,"15min":1.3},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.49199999999999999,"15min":0.48899999999999999},"max":{"1min":1.014,"5min":1.048,"15min":1.1759999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.1000000000000001,"5min":1.25,"15min":1.25},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.002,"5min":1.014,"15min":1.1319999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.53900000000000003,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.202,"15min":1.272},"last":0.70699999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.69499999999999995,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1220000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.54300000000000004},"max":{"1min":1.1659999999999999,"5min":1.232,"15min":1.232},"last":0.67300000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509663,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311028,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060812,"statfs":{"total":96636764160,"available":96318271488,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66200000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.63400000000000001,"5min":1.238,"15min":1.238},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":1.468,"15min":1.468},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.65700000000000003,"5min":1.756,"15min":1.756},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.52700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.83299999999999996,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.51000000000000001,"5min":0.42299999999999999,"15min":0.42199999999999999},"max":{"1min":0.80800000000000005,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.69999999999999996}]},{"osd":4,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.63800000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.0029999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.022},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.61199999999999999,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.88900000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.069},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.61899999999999999,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1399999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.79900000000000004},"min":{"1min":0.65900000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.056,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.70699999999999996,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.071,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.038}]},{"osd":7,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.1659999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.101,"5min":1.272,"15min":1.272},"last":0.88}]}]},{"osd":4,"up_from":14,"seq":60129542366,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306932,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064908,"statfs":{"total":96636764160,"available":96322465792,"internally_reserved":0,"allocated":1486848,"data_stored":724586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67300000000000004,"15min":0.68400000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.371},"max":{"1min":0.91700000000000004,"5min":1.006,"15min":1.008},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.311},"max":{"1min":1.0229999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.628},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40000000000000002},"max":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92900000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97199999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.55200000000000005,"5min":0.46000000000000002,"15min":0.38800000000000001},"max":{"1min":0.875,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.45300000000000001},"max":{"1min":0.90400000000000003,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.753},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.442,"15min":0.438},"max":{"1min":0.98799999999999999,"5min":1.046,"15min":1.046},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84499999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83499999999999996,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.441,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.88600000000000001,"5min":1.133,"15min":1.133},"last":0.88700000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":0.88600000000000001,"5min":1.002,"15min":1.002},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.58399999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89800000000000002,"5min":1.266,"15min":1.266},"last":0.84799999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509661,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":1314954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.68400000000000005},"min":{"1min":0.48199999999999998,"5min":0.39800000000000002,"15min":0.27300000000000002},"max":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.46800000000000003,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.91600000000000004,"15min":1.0369999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.69699999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.34000000000000002},"max":{"1min":0.88400000000000001,"5min":0.96399999999999997,"15min":1.04},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66800000000000004,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.443,"15min":0.36899999999999999},"max":{"1min":0.86899999999999999,"5min":1.002,"15min":1.01},"last":0.57599999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67800000000000005,"15min":0.71099999999999997},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.88500000000000001,"15min":1.3029999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.52400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.92400000000000004,"15min":0.95699999999999996},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.71999999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.39200000000000002},"max":{"1min":0.90500000000000003,"5min":0.93000000000000005,"15min":1.0640000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.47799999999999998},"max":{"1min":0.85499999999999998,"5min":0.92200000000000004,"15min":0.93899999999999995},"last":0.71499999999999997}]},{"osd":4,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.79300000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.41899999999999998,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.01},"last":0.77300000000000002}]},{"osd":6,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.65000000000000002},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35499999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46899999999999997},"max":{"1min":0.85499999999999998,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67100000000000004,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95099999999999996,"15min":0.96299999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68700000000000006,"15min":0.69799999999999995},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.95399999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:55:34.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2023-12-26T21:55:34.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2023-12-26T21:55:34.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2023-12-26T21:55:34.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2023-12-26T21:55:34.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2023-12-26T21:55:34.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2023-12-26T21:55:34.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2023-12-26T21:55:34.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-12-26T21:55:34.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2023-12-26T21:55:34.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.913253+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:26:42.473474+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T21:55:34.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-12-26T21:55:34.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2023-12-26T21:55:34.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.918996+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:06:07.284268+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T21:55:34.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2023-12-26T21:55:34.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911558+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:21:46.237715+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.915152+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:13:41.710446+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.916671+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:41:07.987957+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:55:34.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911553+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T07:43:35.363935+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.918930+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T21:59:34.209365+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2023-12-26T21:55:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.915447+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:33:02.690125+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911486+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:50:05.564372+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.916740+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:04:26.503604+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-12-26T21:55:34.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.919481+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:37:09.996669+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911381+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T04:50:45.785072+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.916672+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T04:50:42.197805+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2023-12-26T21:55:34.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.916100+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:16:34.758852+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911432+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:51:22.466513+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.915267+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:28:21.708669+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:55:34.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911489+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:24:11.586387+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.919052+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T07:46:40.098721+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2023-12-26T21:55:34.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.916158+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:45:00.846344+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.916082+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T22:23:18.461025+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.914095+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:16:51.694872+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-12-26T21:55:34.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.946132+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:53:08.710785+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.913046+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:15:03.359532+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2023-12-26T21:55:34.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911437+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T04:15:32.157127+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.915893+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:37:16.634357+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.916812+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T07:34:05.377326+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.913601+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:00:25.800001+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911658+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:36:49.479164+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.913584+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:08:48.836374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T21:55:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.913370+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T07:43:35.363935+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911486+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:26:42.473474+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T21:55:27.908991+0000', 'last_change': '2023-12-26T21:55:27.908991+0000', 'last_active': '2023-12-26T21:55:27.908991+0000', 'last_peered': '2023-12-26T21:55:27.908991+0000', 'last_clean': '2023-12-26T21:55:27.908991+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:27.908991+0000', 'last_undegraded': '2023-12-26T21:55:27.908991+0000', 'last_fullsized': '2023-12-26T21:55:27.908991+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:27.908991+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T21:55:34.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2023-12-26T21:55:34.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T21:55:26.902820+0000', 'last_change': '2023-12-26T21:55:26.911896+0000', 'last_active': '2023-12-26T21:55:26.902820+0000', 'last_peered': '2023-12-26T21:55:26.902820+0000', 'last_clean': '2023-12-26T21:55:26.902820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T21:55:26.902820+0000', 'last_undegraded': '2023-12-26T21:55:26.902820+0000', 'last_fullsized': '2023-12-26T21:55:26.902820+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'last_clean_scrub_stamp': '2023-12-26T21:55:26.902820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:03:02.206304+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T21:55:34.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-26T21:55:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.453+0000 7f1c713ff700 1 Processor -- start 2023-12-26T21:55:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.453+0000 7f1c713ff700 1 -- start start 2023-12-26T21:55:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.454+0000 7f1c713ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c1596a0 0x7f1c6c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.454+0000 7f1c713ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1c6c159a70 0x7f1c6c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.454+0000 7f1c713ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c0afcd0 con 0x7f1c6c0b0160 2023-12-26T21:55:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.454+0000 7f1c713ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6c0b3e70 con 0x7f1c6c159a70 2023-12-26T21:55:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.454+0000 7f1c713ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0b0cd0 con 0x7f1c6c1596a0 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6a7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1c6c159a70 0x7f1c6c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6a7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1c6c159a70 0x7f1c6c0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:54440/0 (socket says 172.21.15.131:54440) 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6a7fc700 1 -- 172.21.15.131:0/926541031 learned_addr learned my addr 172.21.15.131:0/926541031 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c1596a0 0x7f1c6c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6affd700 1 -- 172.21.15.131:0/926541031 >> v2:172.21.15.190:3300/0 conn(0x7f1c6c159a70 msgr2=0x7f1c6c0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6affd700 1 --2- 172.21.15.131:0/926541031 >> v2:172.21.15.190:3300/0 conn(0x7f1c6c159a70 0x7f1c6c0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6affd700 1 -- 172.21.15.131:0/926541031 >> v1:172.21.15.190:6789/0 conn(0x7f1c6c0b0160 legacy=0x7f1c6c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6affd700 1 -- 172.21.15.131:0/926541031 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0b2e00 con 0x7f1c6c1596a0 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6a7fc700 1 --2- 172.21.15.131:0/926541031 >> v2:172.21.15.190:3300/0 conn(0x7f1c6c159a70 0x7f1c6c0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.455+0000 7f1c6affd700 1 --2- 172.21.15.131:0/926541031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c1596a0 0x7f1c6c0b2380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0098e0 tx=0x7f1c5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b4563df042bb1bd server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:34.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c69ffb700 1 -- 172.21.15.131:0/926541031 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00b640 con 0x7f1c6c1596a0 2023-12-26T21:55:34.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c69ffb700 1 -- 172.21.15.131:0/926541031 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c00b7a0 con 0x7f1c6c1596a0 2023-12-26T21:55:34.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c69ffb700 1 -- 172.21.15.131:0/926541031 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00bb40 con 0x7f1c6c1596a0 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c713ff700 1 -- 172.21.15.131:0/926541031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c1596a0 msgr2=0x7f1c6c0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c713ff700 1 --2- 172.21.15.131:0/926541031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c1596a0 0x7f1c6c0b2380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0098e0 tx=0x7f1c5c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c713ff700 1 -- 172.21.15.131:0/926541031 shutdown_connections 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c713ff700 1 --2- 172.21.15.131:0/926541031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c1596a0 0x7f1c6c0b2380 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c713ff700 1 --2- 172.21.15.131:0/926541031 >> v2:172.21.15.190:3300/0 conn(0x7f1c6c159a70 0x7f1c6c0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c713ff700 1 -- 172.21.15.131:0/926541031 >> 172.21.15.131:0/926541031 conn(0x7f1c6c06ece0 msgr2=0x7f1c6c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c713ff700 1 -- 172.21.15.131:0/926541031 shutdown_connections 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.456+0000 7f1c713ff700 1 -- 172.21.15.131:0/926541031 wait complete. 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.458+0000 7f1c713ff700 1 Processor -- start 2023-12-26T21:55:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.458+0000 7f1c713ff700 1 -- start start 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c713ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c0b0160 0x7f1c6c155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c713ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c6c1596a0 0x7f1c6c0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c713ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c6c159a70 0x7f1c6c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c713ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c16b080 con 0x7f1c6c1596a0 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c713ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c15a8c0 con 0x7f1c6c159a70 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c713ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c159f90 con 0x7f1c6c0b0160 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c6a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c6c1596a0 0x7f1c6c0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c6a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c6c1596a0 0x7f1c6c0d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46792/0 (socket says 172.21.15.131:46792) 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c6a7fc700 1 -- 172.21.15.131:0/3746204964 learned_addr learned my addr 172.21.15.131:0/3746204964 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c6affd700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c0b0160 0x7f1c6c155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c6b7fe700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c6c159a70 0x7f1c6c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.459+0000 7f1c6a7fc700 1 -- 172.21.15.131:0/3746204964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c0b0160 msgr2=0x7f1c6c155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c6a7fc700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c0b0160 0x7f1c6c155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c6a7fc700 1 -- 172.21.15.131:0/3746204964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c6c159a70 msgr2=0x7f1c6c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c6a7fc700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c6c159a70 0x7f1c6c0d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c6a7fc700 1 -- 172.21.15.131:0/3746204964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c600094f0 con 0x7f1c6c1596a0 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c6b7fe700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c6c159a70 0x7f1c6c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c6a7fc700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c6c1596a0 0x7f1c6c0d0560 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000ec50 tx=0x7f1c60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c52ffd700 1 -- 172.21.15.131:0/3746204964 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000c850 con 0x7f1c6c1596a0 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c52ffd700 1 -- 172.21.15.131:0/3746204964 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6000c9b0 con 0x7f1c6c1596a0 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c52ffd700 1 -- 172.21.15.131:0/3746204964 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000cd30 con 0x7f1c6c1596a0 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c009420 con 0x7f1c6c1596a0 2023-12-26T21:55:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.460+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c15a550 con 0x7f1c6c1596a0 2023-12-26T21:55:34.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.461+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6c15aa30 con 0x7f1c6c1596a0 2023-12-26T21:55:34.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.462+0000 7f1c52ffd700 1 -- 172.21.15.131:0/3746204964 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1c60013680 con 0x7f1c6c1596a0 2023-12-26T21:55:34.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.463+0000 7f1c52ffd700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1c58066f30 0x7f1c580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.463+0000 7f1c52ffd700 1 -- 172.21.15.131:0/3746204964 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f1c60084e60 con 0x7f1c6c1596a0 2023-12-26T21:55:34.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.464+0000 7f1c6affd700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1c58066f30 0x7f1c580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.464+0000 7f1c6affd700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1c58066f30 0x7f1c580693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0093f0 tx=0x7f1c5c017000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:34.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.468+0000 7f1c52ffd700 1 -- 172.21.15.131:0/3746204964 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c60051ea0 con 0x7f1c6c1596a0 2023-12-26T21:55:34.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.733+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1c6c0657e0 con 0x7f1c6c1596a0 2023-12-26T21:55:34.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.734+0000 7f1c52ffd700 1 -- 172.21.15.131:0/3746204964 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1113 (secure 0 0 0) 0x7f1c60021470 con 0x7f1c6c1596a0 2023-12-26T21:55:34.735 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:34.736 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"ab603302-d88b-429a-be64-92b3c298adad","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1150,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1703626596,"num_in_osds":8,"osd_in_since":1703626587,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2511380480,"bytes_avail":770582732800,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":57,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7644}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-12-26T21:55:08.596628+0000","services":{}},"progress_events":{}} 2023-12-26T21:55:34.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1c58066f30 msgr2=0x7f1c580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1c58066f30 0x7f1c580693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0093f0 tx=0x7f1c5c017000 comp rx=0 tx=0).stop 2023-12-26T21:55:34.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c6c1596a0 msgr2=0x7f1c6c0d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c6c1596a0 0x7f1c6c0d0560 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000ec50 tx=0x7f1c60009250 comp rx=0 tx=0).stop 2023-12-26T21:55:34.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 shutdown_connections 2023-12-26T21:55:34.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c6c159a70 0x7f1c6c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c6c0b0160 0x7f1c6c155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1c58066f30 0x7f1c580693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 --2- 172.21.15.131:0/3746204964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c6c1596a0 0x7f1c6c0d0560 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 >> 172.21.15.131:0/3746204964 conn(0x7f1c6c06ece0 msgr2=0x7f1c6c156420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:34.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 shutdown_connections 2023-12-26T21:55:34.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.736+0000 7f1c713ff700 1 -- 172.21.15.131:0/3746204964 wait complete. 2023-12-26T21:55:34.754 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-26T21:55:34.755 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:55:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.946+0000 7f795d3ff700 1 Processor -- start 2023-12-26T21:55:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.946+0000 7f795d3ff700 1 -- start start 2023-12-26T21:55:34.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f795d3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f795806bfd0 0x7f79580c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f795d3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 0x7f79580cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f795d3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580686c0 con 0x7f795813c510 2023-12-26T21:55:34.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f795d3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7958069520 con 0x7f795806bfd0 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f795d3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7958068910 con 0x7f79580cc100 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f79567fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f795806bfd0 0x7f79580c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f79567fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f795806bfd0 0x7f79580c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49426/0 (socket says 172.21.15.131:49426) 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f7956ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f795813c510 0x7f795806b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44802/0 (socket says 172.21.15.131:44802) 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f79567fc700 1 -- 172.21.15.131:0/4280306448 learned_addr learned my addr 172.21.15.131:0/4280306448 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.947+0000 7f79577fe700 1 --2- 172.21.15.131:0/4280306448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 0x7f79580cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.948+0000 7f79567fc700 1 -- 172.21.15.131:0/4280306448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 msgr2=0x7f79580cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.948+0000 7f79567fc700 1 --2- 172.21.15.131:0/4280306448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 0x7f79580cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.948+0000 7f79567fc700 1 -- 172.21.15.131:0/4280306448 >> v1:172.21.15.131:6789/0 conn(0x7f795813c510 legacy=0x7f795806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.948+0000 7f79567fc700 1 -- 172.21.15.131:0/4280306448 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958068bf0 con 0x7f795806bfd0 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.948+0000 7f79577fe700 1 --2- 172.21.15.131:0/4280306448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 0x7f79580cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.948+0000 7f79567fc700 1 --2- 172.21.15.131:0/4280306448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 0x7f79580c8a20 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f794c0099b0 tx=0x7f794c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7f993abe4eedb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.948+0000 7f7955ffb700 1 -- 172.21.15.131:0/4280306448 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c00b860 con 0x7f795806bfd0 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.948+0000 7f7955ffb700 1 -- 172.21.15.131:0/4280306448 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c00b9c0 con 0x7f795806bfd0 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.948+0000 7f7955ffb700 1 -- 172.21.15.131:0/4280306448 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c00bd60 con 0x7f795806bfd0 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.949+0000 7f795d3ff700 1 -- 172.21.15.131:0/4280306448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 msgr2=0x7f79580c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.949+0000 7f795d3ff700 1 --2- 172.21.15.131:0/4280306448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 0x7f79580c8a20 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f794c0099b0 tx=0x7f794c009250 comp rx=0 tx=0).stop 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.949+0000 7f795d3ff700 1 -- 172.21.15.131:0/4280306448 shutdown_connections 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.949+0000 7f795d3ff700 1 --2- 172.21.15.131:0/4280306448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 0x7f79580cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.949+0000 7f795d3ff700 1 --2- 172.21.15.131:0/4280306448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 0x7f79580c8a20 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.949+0000 7f795d3ff700 1 -- 172.21.15.131:0/4280306448 >> 172.21.15.131:0/4280306448 conn(0x7f79580ce3e0 msgr2=0x7f79580ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.949+0000 7f795d3ff700 1 -- 172.21.15.131:0/4280306448 shutdown_connections 2023-12-26T21:55:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.949+0000 7f795d3ff700 1 -- 172.21.15.131:0/4280306448 wait complete. 2023-12-26T21:55:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.951+0000 7f795d3ff700 1 Processor -- start 2023-12-26T21:55:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.951+0000 7f795d3ff700 1 -- start start 2023-12-26T21:55:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f795d3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 0x7f7958155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f795d3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 0x7f79581530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f795d3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f795813c510 0x7f79580c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f795d3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7958156690 con 0x7f795806bfd0 2023-12-26T21:55:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f795d3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7958153600 con 0x7f795813c510 2023-12-26T21:55:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f795d3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79580ca010 con 0x7f79580cc100 2023-12-26T21:55:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f7956ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 0x7f7958155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f7956ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 0x7f7958155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49436/0 (socket says 172.21.15.131:49436) 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f7956ffd700 1 -- 172.21.15.131:0/908084278 learned_addr learned my addr 172.21.15.131:0/908084278 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f7956ffd700 1 -- 172.21.15.131:0/908084278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 msgr2=0x7f79581530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f7956ffd700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 0x7f79581530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f7956ffd700 1 -- 172.21.15.131:0/908084278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f795813c510 msgr2=0x7f79580c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f7956ffd700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f795813c510 0x7f79580c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.952+0000 7f7956ffd700 1 -- 172.21.15.131:0/908084278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c0094f0 con 0x7f795806bfd0 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.953+0000 7f7956ffd700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 0x7f7958155cf0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f794800c7f0 tx=0x7f7948007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.953+0000 7f793effd700 1 -- 172.21.15.131:0/908084278 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948007300 con 0x7f795806bfd0 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.953+0000 7f793effd700 1 -- 172.21.15.131:0/908084278 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7948007460 con 0x7f795806bfd0 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.953+0000 7f793effd700 1 -- 172.21.15.131:0/908084278 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948005a40 con 0x7f795806bfd0 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.953+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79580b1970 con 0x7f795806bfd0 2023-12-26T21:55:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.953+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79580b1320 con 0x7f795806bfd0 2023-12-26T21:55:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.954+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79580b1180 con 0x7f795806bfd0 2023-12-26T21:55:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.955+0000 7f793effd700 1 -- 172.21.15.131:0/908084278 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f794801b400 con 0x7f795806bfd0 2023-12-26T21:55:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.956+0000 7f793effd700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7944066f30 0x7f79440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.956+0000 7f793effd700 1 -- 172.21.15.131:0/908084278 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7948084120 con 0x7f795806bfd0 2023-12-26T21:55:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.956+0000 7f79567fc700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7944066f30 0x7f79440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.957+0000 7f79567fc700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7944066f30 0x7f79440693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f794c0099b0 tx=0x7f794c017000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:34.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:34.960+0000 7f793effd700 1 -- 172.21.15.131:0/908084278 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79580b1180 con 0x7f795806bfd0 2023-12-26T21:55:35.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.147+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79580b1180 con 0x7f7944066f30 2023-12-26T21:55:35.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.155+0000 7f793effd700 1 -- 172.21.15.131:0/908084278 <== mgr.7159 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289444 (secure 0 0 0) 0x7f79580b1180 con 0x7f7944066f30 2023-12-26T21:55:35.161 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:35.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7944066f30 msgr2=0x7f79440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7944066f30 0x7f79440693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f794c0099b0 tx=0x7f794c017000 comp rx=0 tx=0).stop 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 msgr2=0x7f7958155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 0x7f7958155cf0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f794800c7f0 tx=0x7f7948007ea0 comp rx=0 tx=0).stop 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 shutdown_connections 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f795813c510 0x7f79580c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7944066f30 0x7f79440693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f795806bfd0 0x7f7958155cf0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 --2- 172.21.15.131:0/908084278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79580cc100 0x7f79581530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 >> 172.21.15.131:0/908084278 conn(0x7f79580ce3e0 msgr2=0x7f795806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 shutdown_connections 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:35.160+0000 7f795d3ff700 1 -- 172.21.15.131:0/908084278 wait complete. 2023-12-26T21:55:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:55:35.182 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-26T21:55:34.884872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452872,"kb_used_data":13128,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752521848,"statfs":{"total":773094113280,"available":770582372352,"internally_reserved":0,"allocated":13443072,"data_stored":7145956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.968274"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956465+0000","last_change":"2023-12-26T21:55:28.951885+0000","last_active":"2023-12-26T21:55:28.956465+0000","last_peered":"2023-12-26T21:55:28.956465+0000","last_clean":"2023-12-26T21:55:28.956465+0000","last_became_active":"2023-12-26T21:55:28.951617+0000","last_became_peered":"2023-12-26T21:55:28.951617+0000","last_unstale":"2023-12-26T21:55:28.956465+0000","last_undegraded":"2023-12-26T21:55:28.956465+0000","last_fullsized":"2023-12-26T21:55:28.956465+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:02:02.032351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.946675+0000","last_change":"2023-12-26T21:55:28.944219+0000","last_active":"2023-12-26T21:55:28.946675+0000","last_peered":"2023-12-26T21:55:28.946675+0000","last_clean":"2023-12-26T21:55:28.946675+0000","last_became_active":"2023-12-26T21:55:28.943949+0000","last_became_peered":"2023-12-26T21:55:28.943949+0000","last_unstale":"2023-12-26T21:55:28.946675+0000","last_undegraded":"2023-12-26T21:55:28.946675+0000","last_fullsized":"2023-12-26T21:55:28.946675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:20:16.839977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955152+0000","last_change":"2023-12-26T21:55:28.952166+0000","last_active":"2023-12-26T21:55:28.955152+0000","last_peered":"2023-12-26T21:55:28.955152+0000","last_clean":"2023-12-26T21:55:28.955152+0000","last_became_active":"2023-12-26T21:55:28.951876+0000","last_became_peered":"2023-12-26T21:55:28.951876+0000","last_unstale":"2023-12-26T21:55:28.955152+0000","last_undegraded":"2023-12-26T21:55:28.955152+0000","last_fullsized":"2023-12-26T21:55:28.955152+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:18:19.770941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956257+0000","last_change":"2023-12-26T21:55:28.948988+0000","last_active":"2023-12-26T21:55:28.956257+0000","last_peered":"2023-12-26T21:55:28.956257+0000","last_clean":"2023-12-26T21:55:28.956257+0000","last_became_active":"2023-12-26T21:55:28.945767+0000","last_became_peered":"2023-12-26T21:55:28.945767+0000","last_unstale":"2023-12-26T21:55:28.956257+0000","last_undegraded":"2023-12-26T21:55:28.956257+0000","last_fullsized":"2023-12-26T21:55:28.956257+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:33:03.696296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955260+0000","last_change":"2023-12-26T21:55:28.953010+0000","last_active":"2023-12-26T21:55:28.955260+0000","last_peered":"2023-12-26T21:55:28.955260+0000","last_clean":"2023-12-26T21:55:28.955260+0000","last_became_active":"2023-12-26T21:55:28.949970+0000","last_became_peered":"2023-12-26T21:55:28.949970+0000","last_unstale":"2023-12-26T21:55:28.955260+0000","last_undegraded":"2023-12-26T21:55:28.955260+0000","last_fullsized":"2023-12-26T21:55:28.955260+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:28:29.427026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955531+0000","last_change":"2023-12-26T21:55:28.952337+0000","last_active":"2023-12-26T21:55:28.955531+0000","last_peered":"2023-12-26T21:55:28.955531+0000","last_clean":"2023-12-26T21:55:28.955531+0000","last_became_active":"2023-12-26T21:55:28.951388+0000","last_became_peered":"2023-12-26T21:55:28.951388+0000","last_unstale":"2023-12-26T21:55:28.955531+0000","last_undegraded":"2023-12-26T21:55:28.955531+0000","last_fullsized":"2023-12-26T21:55:28.955531+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:53:42.935300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954843+0000","last_change":"2023-12-26T21:55:28.952055+0000","last_active":"2023-12-26T21:55:28.954843+0000","last_peered":"2023-12-26T21:55:28.954843+0000","last_clean":"2023-12-26T21:55:28.954843+0000","last_became_active":"2023-12-26T21:55:28.950302+0000","last_became_peered":"2023-12-26T21:55:28.950302+0000","last_unstale":"2023-12-26T21:55:28.954843+0000","last_undegraded":"2023-12-26T21:55:28.954843+0000","last_fullsized":"2023-12-26T21:55:28.954843+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:58:48.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954113+0000","last_change":"2023-12-26T21:55:28.949123+0000","last_active":"2023-12-26T21:55:28.954113+0000","last_peered":"2023-12-26T21:55:28.954113+0000","last_clean":"2023-12-26T21:55:28.954113+0000","last_became_active":"2023-12-26T21:55:28.948710+0000","last_became_peered":"2023-12-26T21:55:28.948710+0000","last_unstale":"2023-12-26T21:55:28.954113+0000","last_undegraded":"2023-12-26T21:55:28.954113+0000","last_fullsized":"2023-12-26T21:55:28.954113+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:26.806171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957051+0000","last_change":"2023-12-26T21:55:28.954784+0000","last_active":"2023-12-26T21:55:28.957051+0000","last_peered":"2023-12-26T21:55:28.957051+0000","last_clean":"2023-12-26T21:55:28.957051+0000","last_became_active":"2023-12-26T21:55:28.954379+0000","last_became_peered":"2023-12-26T21:55:28.954379+0000","last_unstale":"2023-12-26T21:55:28.957051+0000","last_undegraded":"2023-12-26T21:55:28.957051+0000","last_fullsized":"2023-12-26T21:55:28.957051+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:46:27.951865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.950325+0000","last_change":"2023-12-26T21:55:28.947592+0000","last_active":"2023-12-26T21:55:28.950325+0000","last_peered":"2023-12-26T21:55:28.950325+0000","last_clean":"2023-12-26T21:55:28.950325+0000","last_became_active":"2023-12-26T21:55:28.947338+0000","last_became_peered":"2023-12-26T21:55:28.947338+0000","last_unstale":"2023-12-26T21:55:28.950325+0000","last_undegraded":"2023-12-26T21:55:28.950325+0000","last_fullsized":"2023-12-26T21:55:28.950325+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:49:24.070796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954921+0000","last_change":"2023-12-26T21:55:28.952846+0000","last_active":"2023-12-26T21:55:28.954921+0000","last_peered":"2023-12-26T21:55:28.954921+0000","last_clean":"2023-12-26T21:55:28.954921+0000","last_became_active":"2023-12-26T21:55:28.952443+0000","last_became_peered":"2023-12-26T21:55:28.952443+0000","last_unstale":"2023-12-26T21:55:28.954921+0000","last_undegraded":"2023-12-26T21:55:28.954921+0000","last_fullsized":"2023-12-26T21:55:28.954921+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:36:19.760130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.951193+0000","last_change":"2023-12-26T21:55:28.949327+0000","last_active":"2023-12-26T21:55:28.951193+0000","last_peered":"2023-12-26T21:55:28.951193+0000","last_clean":"2023-12-26T21:55:28.951193+0000","last_became_active":"2023-12-26T21:55:28.949039+0000","last_became_peered":"2023-12-26T21:55:28.949039+0000","last_unstale":"2023-12-26T21:55:28.951193+0000","last_undegraded":"2023-12-26T21:55:28.951193+0000","last_fullsized":"2023-12-26T21:55:28.951193+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:46.647706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943403+0000","last_change":"2023-12-26T21:55:28.935945+0000","last_active":"2023-12-26T21:55:28.943403+0000","last_peered":"2023-12-26T21:55:28.943403+0000","last_clean":"2023-12-26T21:55:28.943403+0000","last_became_active":"2023-12-26T21:55:28.935453+0000","last_became_peered":"2023-12-26T21:55:28.935453+0000","last_unstale":"2023-12-26T21:55:28.943403+0000","last_undegraded":"2023-12-26T21:55:28.943403+0000","last_fullsized":"2023-12-26T21:55:28.943403+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:02.123742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.946718+0000","last_change":"2023-12-26T21:55:28.941523+0000","last_active":"2023-12-26T21:55:28.946718+0000","last_peered":"2023-12-26T21:55:28.946718+0000","last_clean":"2023-12-26T21:55:28.946718+0000","last_became_active":"2023-12-26T21:55:28.941265+0000","last_became_peered":"2023-12-26T21:55:28.941265+0000","last_unstale":"2023-12-26T21:55:28.946718+0000","last_undegraded":"2023-12-26T21:55:28.946718+0000","last_fullsized":"2023-12-26T21:55:28.946718+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:35:46.585609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955605+0000","last_change":"2023-12-26T21:55:28.951022+0000","last_active":"2023-12-26T21:55:28.955605+0000","last_peered":"2023-12-26T21:55:28.955605+0000","last_clean":"2023-12-26T21:55:28.955605+0000","last_became_active":"2023-12-26T21:55:28.950736+0000","last_became_peered":"2023-12-26T21:55:28.950736+0000","last_unstale":"2023-12-26T21:55:28.955605+0000","last_undegraded":"2023-12-26T21:55:28.955605+0000","last_fullsized":"2023-12-26T21:55:28.955605+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:24:04.095512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.961384+0000","last_change":"2023-12-26T21:55:28.958702+0000","last_active":"2023-12-26T21:55:28.961384+0000","last_peered":"2023-12-26T21:55:28.961384+0000","last_clean":"2023-12-26T21:55:28.961384+0000","last_became_active":"2023-12-26T21:55:28.958401+0000","last_became_peered":"2023-12-26T21:55:28.958401+0000","last_unstale":"2023-12-26T21:55:28.961384+0000","last_undegraded":"2023-12-26T21:55:28.961384+0000","last_fullsized":"2023-12-26T21:55:28.961384+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:04:00.547049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957155+0000","last_change":"2023-12-26T21:55:28.946413+0000","last_active":"2023-12-26T21:55:28.957155+0000","last_peered":"2023-12-26T21:55:28.957155+0000","last_clean":"2023-12-26T21:55:28.957155+0000","last_became_active":"2023-12-26T21:55:28.946073+0000","last_became_peered":"2023-12-26T21:55:28.946073+0000","last_unstale":"2023-12-26T21:55:28.957155+0000","last_undegraded":"2023-12-26T21:55:28.957155+0000","last_fullsized":"2023-12-26T21:55:28.957155+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:22.666559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954467+0000","last_change":"2023-12-26T21:55:28.950636+0000","last_active":"2023-12-26T21:55:28.954467+0000","last_peered":"2023-12-26T21:55:28.954467+0000","last_clean":"2023-12-26T21:55:28.954467+0000","last_became_active":"2023-12-26T21:55:28.949943+0000","last_became_peered":"2023-12-26T21:55:28.949943+0000","last_unstale":"2023-12-26T21:55:28.954467+0000","last_undegraded":"2023-12-26T21:55:28.954467+0000","last_fullsized":"2023-12-26T21:55:28.954467+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:24.696009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.958113+0000","last_change":"2023-12-26T21:55:28.951084+0000","last_active":"2023-12-26T21:55:28.958113+0000","last_peered":"2023-12-26T21:55:28.958113+0000","last_clean":"2023-12-26T21:55:28.958113+0000","last_became_active":"2023-12-26T21:55:28.950820+0000","last_became_peered":"2023-12-26T21:55:28.950820+0000","last_unstale":"2023-12-26T21:55:28.958113+0000","last_undegraded":"2023-12-26T21:55:28.958113+0000","last_fullsized":"2023-12-26T21:55:28.958113+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:08.290439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.947399+0000","last_change":"2023-12-26T21:55:28.944022+0000","last_active":"2023-12-26T21:55:28.947399+0000","last_peered":"2023-12-26T21:55:28.947399+0000","last_clean":"2023-12-26T21:55:28.947399+0000","last_became_active":"2023-12-26T21:55:28.943809+0000","last_became_peered":"2023-12-26T21:55:28.943809+0000","last_unstale":"2023-12-26T21:55:28.947399+0000","last_undegraded":"2023-12-26T21:55:28.947399+0000","last_fullsized":"2023-12-26T21:55:28.947399+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:20.853510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.952768+0000","last_change":"2023-12-26T21:55:28.945727+0000","last_active":"2023-12-26T21:55:28.952768+0000","last_peered":"2023-12-26T21:55:28.952768+0000","last_clean":"2023-12-26T21:55:28.952768+0000","last_became_active":"2023-12-26T21:55:28.945478+0000","last_became_peered":"2023-12-26T21:55:28.945478+0000","last_unstale":"2023-12-26T21:55:28.952768+0000","last_undegraded":"2023-12-26T21:55:28.952768+0000","last_fullsized":"2023-12-26T21:55:28.952768+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:21.089546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956895+0000","last_change":"2023-12-26T21:55:28.955777+0000","last_active":"2023-12-26T21:55:28.956895+0000","last_peered":"2023-12-26T21:55:28.956895+0000","last_clean":"2023-12-26T21:55:28.956895+0000","last_became_active":"2023-12-26T21:55:28.955463+0000","last_became_peered":"2023-12-26T21:55:28.955463+0000","last_unstale":"2023-12-26T21:55:28.956895+0000","last_undegraded":"2023-12-26T21:55:28.956895+0000","last_fullsized":"2023-12-26T21:55:28.956895+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:37:16.907722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.948409+0000","last_change":"2023-12-26T21:55:28.940305+0000","last_active":"2023-12-26T21:55:28.948409+0000","last_peered":"2023-12-26T21:55:28.948409+0000","last_clean":"2023-12-26T21:55:28.948409+0000","last_became_active":"2023-12-26T21:55:28.940058+0000","last_became_peered":"2023-12-26T21:55:28.940058+0000","last_unstale":"2023-12-26T21:55:28.948409+0000","last_undegraded":"2023-12-26T21:55:28.948409+0000","last_fullsized":"2023-12-26T21:55:28.948409+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:02:51.779760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.940323+0000","last_change":"2023-12-26T21:55:28.936544+0000","last_active":"2023-12-26T21:55:28.940323+0000","last_peered":"2023-12-26T21:55:28.940323+0000","last_clean":"2023-12-26T21:55:28.940323+0000","last_became_active":"2023-12-26T21:55:28.936305+0000","last_became_peered":"2023-12-26T21:55:28.936305+0000","last_unstale":"2023-12-26T21:55:28.940323+0000","last_undegraded":"2023-12-26T21:55:28.940323+0000","last_fullsized":"2023-12-26T21:55:28.940323+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:25:25.368816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.950366+0000","last_change":"2023-12-26T21:55:28.947504+0000","last_active":"2023-12-26T21:55:28.950366+0000","last_peered":"2023-12-26T21:55:28.950366+0000","last_clean":"2023-12-26T21:55:28.950366+0000","last_became_active":"2023-12-26T21:55:28.947222+0000","last_became_peered":"2023-12-26T21:55:28.947222+0000","last_unstale":"2023-12-26T21:55:28.950366+0000","last_undegraded":"2023-12-26T21:55:28.950366+0000","last_fullsized":"2023-12-26T21:55:28.950366+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:42:48.634872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943318+0000","last_change":"2023-12-26T21:55:28.936887+0000","last_active":"2023-12-26T21:55:28.943318+0000","last_peered":"2023-12-26T21:55:28.943318+0000","last_clean":"2023-12-26T21:55:28.943318+0000","last_became_active":"2023-12-26T21:55:28.936646+0000","last_became_peered":"2023-12-26T21:55:28.936646+0000","last_unstale":"2023-12-26T21:55:28.943318+0000","last_undegraded":"2023-12-26T21:55:28.943318+0000","last_fullsized":"2023-12-26T21:55:28.943318+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:06:44.485471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.940407+0000","last_change":"2023-12-26T21:55:28.937919+0000","last_active":"2023-12-26T21:55:28.940407+0000","last_peered":"2023-12-26T21:55:28.940407+0000","last_clean":"2023-12-26T21:55:28.940407+0000","last_became_active":"2023-12-26T21:55:28.937645+0000","last_became_peered":"2023-12-26T21:55:28.937645+0000","last_unstale":"2023-12-26T21:55:28.940407+0000","last_undegraded":"2023-12-26T21:55:28.940407+0000","last_fullsized":"2023-12-26T21:55:28.940407+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:45:01.825508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.953686+0000","last_change":"2023-12-26T21:55:28.949259+0000","last_active":"2023-12-26T21:55:28.953686+0000","last_peered":"2023-12-26T21:55:28.953686+0000","last_clean":"2023-12-26T21:55:28.953686+0000","last_became_active":"2023-12-26T21:55:28.948999+0000","last_became_peered":"2023-12-26T21:55:28.948999+0000","last_unstale":"2023-12-26T21:55:28.953686+0000","last_undegraded":"2023-12-26T21:55:28.953686+0000","last_fullsized":"2023-12-26T21:55:28.953686+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:25:02.095513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943385+0000","last_change":"2023-12-26T21:55:28.935745+0000","last_active":"2023-12-26T21:55:28.943385+0000","last_peered":"2023-12-26T21:55:28.943385+0000","last_clean":"2023-12-26T21:55:28.943385+0000","last_became_active":"2023-12-26T21:55:28.935078+0000","last_became_peered":"2023-12-26T21:55:28.935078+0000","last_unstale":"2023-12-26T21:55:28.943385+0000","last_undegraded":"2023-12-26T21:55:28.943385+0000","last_fullsized":"2023-12-26T21:55:28.943385+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:47:01.650507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943236+0000","last_change":"2023-12-26T21:55:28.940106+0000","last_active":"2023-12-26T21:55:28.943236+0000","last_peered":"2023-12-26T21:55:28.943236+0000","last_clean":"2023-12-26T21:55:28.943236+0000","last_became_active":"2023-12-26T21:55:28.939727+0000","last_became_peered":"2023-12-26T21:55:28.939727+0000","last_unstale":"2023-12-26T21:55:28.943236+0000","last_undegraded":"2023-12-26T21:55:28.943236+0000","last_fullsized":"2023-12-26T21:55:28.943236+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:08:49.842545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954788+0000","last_change":"2023-12-26T21:55:28.950126+0000","last_active":"2023-12-26T21:55:28.954788+0000","last_peered":"2023-12-26T21:55:28.954788+0000","last_clean":"2023-12-26T21:55:28.954788+0000","last_became_active":"2023-12-26T21:55:28.949338+0000","last_became_peered":"2023-12-26T21:55:28.949338+0000","last_unstale":"2023-12-26T21:55:28.954788+0000","last_undegraded":"2023-12-26T21:55:28.954788+0000","last_fullsized":"2023-12-26T21:55:28.954788+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:03:36.162383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.947347+0000","last_change":"2023-12-26T21:55:28.945216+0000","last_active":"2023-12-26T21:55:28.947347+0000","last_peered":"2023-12-26T21:55:28.947347+0000","last_clean":"2023-12-26T21:55:28.947347+0000","last_became_active":"2023-12-26T21:55:28.944954+0000","last_became_peered":"2023-12-26T21:55:28.944954+0000","last_unstale":"2023-12-26T21:55:28.947347+0000","last_undegraded":"2023-12-26T21:55:28.947347+0000","last_fullsized":"2023-12-26T21:55:28.947347+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:21:27.670725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.945496+0000","last_change":"2023-12-26T21:55:28.939360+0000","last_active":"2023-12-26T21:55:28.945496+0000","last_peered":"2023-12-26T21:55:28.945496+0000","last_clean":"2023-12-26T21:55:28.945496+0000","last_became_active":"2023-12-26T21:55:28.939089+0000","last_became_peered":"2023-12-26T21:55:28.939089+0000","last_unstale":"2023-12-26T21:55:28.945496+0000","last_undegraded":"2023-12-26T21:55:28.945496+0000","last_fullsized":"2023-12-26T21:55:28.945496+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:03:04.946777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"123'4","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.950621+0000","last_change":"2023-12-26T21:55:27.931297+0000","last_active":"2023-12-26T21:55:28.950621+0000","last_peered":"2023-12-26T21:55:28.950621+0000","last_clean":"2023-12-26T21:55:28.950621+0000","last_became_active":"2023-12-26T21:55:27.930888+0000","last_became_peered":"2023-12-26T21:55:27.930888+0000","last_unstale":"2023-12-26T21:55:28.950621+0000","last_undegraded":"2023-12-26T21:55:28.950621+0000","last_fullsized":"2023-12-26T21:55:28.950621+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:21:46.237715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943204+0000","last_change":"2023-12-26T21:55:28.936059+0000","last_active":"2023-12-26T21:55:28.943204+0000","last_peered":"2023-12-26T21:55:28.943204+0000","last_clean":"2023-12-26T21:55:28.943204+0000","last_became_active":"2023-12-26T21:55:28.935732+0000","last_became_peered":"2023-12-26T21:55:28.935732+0000","last_unstale":"2023-12-26T21:55:28.943204+0000","last_undegraded":"2023-12-26T21:55:28.943204+0000","last_fullsized":"2023-12-26T21:55:28.943204+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:31:13.763746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.958080+0000","last_change":"2023-12-26T21:55:27.942294+0000","last_active":"2023-12-26T21:55:28.958080+0000","last_peered":"2023-12-26T21:55:28.958080+0000","last_clean":"2023-12-26T21:55:28.958080+0000","last_became_active":"2023-12-26T21:55:27.942019+0000","last_became_peered":"2023-12-26T21:55:27.942019+0000","last_unstale":"2023-12-26T21:55:28.958080+0000","last_undegraded":"2023-12-26T21:55:28.958080+0000","last_fullsized":"2023-12-26T21:55:28.958080+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:36:14.042224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.949092+0000","last_change":"2023-12-26T21:55:28.940549+0000","last_active":"2023-12-26T21:55:28.949092+0000","last_peered":"2023-12-26T21:55:28.949092+0000","last_clean":"2023-12-26T21:55:28.949092+0000","last_became_active":"2023-12-26T21:55:28.937952+0000","last_became_peered":"2023-12-26T21:55:28.937952+0000","last_unstale":"2023-12-26T21:55:28.949092+0000","last_undegraded":"2023-12-26T21:55:28.949092+0000","last_fullsized":"2023-12-26T21:55:28.949092+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:56:12.537308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954988+0000","last_change":"2023-12-26T21:55:27.944761+0000","last_active":"2023-12-26T21:55:28.954988+0000","last_peered":"2023-12-26T21:55:28.954988+0000","last_clean":"2023-12-26T21:55:28.954988+0000","last_became_active":"2023-12-26T21:55:27.944454+0000","last_became_peered":"2023-12-26T21:55:27.944454+0000","last_unstale":"2023-12-26T21:55:28.954988+0000","last_undegraded":"2023-12-26T21:55:28.954988+0000","last_fullsized":"2023-12-26T21:55:28.954988+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:35:27.172805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.952873+0000","last_change":"2023-12-26T21:55:28.951410+0000","last_active":"2023-12-26T21:55:28.952873+0000","last_peered":"2023-12-26T21:55:28.952873+0000","last_clean":"2023-12-26T21:55:28.952873+0000","last_became_active":"2023-12-26T21:55:28.951185+0000","last_became_peered":"2023-12-26T21:55:28.951185+0000","last_unstale":"2023-12-26T21:55:28.952873+0000","last_undegraded":"2023-12-26T21:55:28.952873+0000","last_fullsized":"2023-12-26T21:55:28.952873+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:32:17.225200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956200+0000","last_change":"2023-12-26T21:55:27.940411+0000","last_active":"2023-12-26T21:55:28.956200+0000","last_peered":"2023-12-26T21:55:28.956200+0000","last_clean":"2023-12-26T21:55:28.956200+0000","last_became_active":"2023-12-26T21:55:27.940140+0000","last_became_peered":"2023-12-26T21:55:27.940140+0000","last_unstale":"2023-12-26T21:55:28.956200+0000","last_undegraded":"2023-12-26T21:55:28.956200+0000","last_fullsized":"2023-12-26T21:55:28.956200+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:19.847339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.960115+0000","last_change":"2023-12-26T21:55:28.955415+0000","last_active":"2023-12-26T21:55:28.960115+0000","last_peered":"2023-12-26T21:55:28.960115+0000","last_clean":"2023-12-26T21:55:28.960115+0000","last_became_active":"2023-12-26T21:55:28.955107+0000","last_became_peered":"2023-12-26T21:55:28.955107+0000","last_unstale":"2023-12-26T21:55:28.960115+0000","last_undegraded":"2023-12-26T21:55:28.960115+0000","last_fullsized":"2023-12-26T21:55:28.960115+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:07.416651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.946866+0000","last_change":"2023-12-26T21:55:27.933902+0000","last_active":"2023-12-26T21:55:28.946866+0000","last_peered":"2023-12-26T21:55:28.946866+0000","last_clean":"2023-12-26T21:55:28.946866+0000","last_became_active":"2023-12-26T21:55:27.933632+0000","last_became_peered":"2023-12-26T21:55:27.933632+0000","last_unstale":"2023-12-26T21:55:28.946866+0000","last_undegraded":"2023-12-26T21:55:28.946866+0000","last_fullsized":"2023-12-26T21:55:28.946866+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:45.641535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954282+0000","last_change":"2023-12-26T21:55:28.950844+0000","last_active":"2023-12-26T21:55:28.954282+0000","last_peered":"2023-12-26T21:55:28.954282+0000","last_clean":"2023-12-26T21:55:28.954282+0000","last_became_active":"2023-12-26T21:55:28.949928+0000","last_became_peered":"2023-12-26T21:55:28.949928+0000","last_unstale":"2023-12-26T21:55:28.954282+0000","last_undegraded":"2023-12-26T21:55:28.954282+0000","last_fullsized":"2023-12-26T21:55:28.954282+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:13.330832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957430+0000","last_change":"2023-12-26T21:55:27.940766+0000","last_active":"2023-12-26T21:55:28.957430+0000","last_peered":"2023-12-26T21:55:28.957430+0000","last_clean":"2023-12-26T21:55:28.957430+0000","last_became_active":"2023-12-26T21:55:27.940488+0000","last_became_peered":"2023-12-26T21:55:27.940488+0000","last_unstale":"2023-12-26T21:55:28.957430+0000","last_undegraded":"2023-12-26T21:55:28.957430+0000","last_fullsized":"2023-12-26T21:55:28.957430+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:55:25.001993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955907+0000","last_change":"2023-12-26T21:55:28.949084+0000","last_active":"2023-12-26T21:55:28.955907+0000","last_peered":"2023-12-26T21:55:28.955907+0000","last_clean":"2023-12-26T21:55:28.955907+0000","last_became_active":"2023-12-26T21:55:28.948739+0000","last_became_peered":"2023-12-26T21:55:28.948739+0000","last_unstale":"2023-12-26T21:55:28.955907+0000","last_undegraded":"2023-12-26T21:55:28.955907+0000","last_fullsized":"2023-12-26T21:55:28.955907+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:36:09.516332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943808+0000","last_change":"2023-12-26T21:55:27.934641+0000","last_active":"2023-12-26T21:55:28.943808+0000","last_peered":"2023-12-26T21:55:28.943808+0000","last_clean":"2023-12-26T21:55:28.943808+0000","last_became_active":"2023-12-26T21:55:27.934248+0000","last_became_peered":"2023-12-26T21:55:27.934248+0000","last_unstale":"2023-12-26T21:55:28.943808+0000","last_undegraded":"2023-12-26T21:55:28.943808+0000","last_fullsized":"2023-12-26T21:55:28.943808+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:47:57.015825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957465+0000","last_change":"2023-12-26T21:55:28.955122+0000","last_active":"2023-12-26T21:55:28.957465+0000","last_peered":"2023-12-26T21:55:28.957465+0000","last_clean":"2023-12-26T21:55:28.957465+0000","last_became_active":"2023-12-26T21:55:28.954855+0000","last_became_peered":"2023-12-26T21:55:28.954855+0000","last_unstale":"2023-12-26T21:55:28.957465+0000","last_undegraded":"2023-12-26T21:55:28.957465+0000","last_fullsized":"2023-12-26T21:55:28.957465+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:19:02.090780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.940992+0000","last_change":"2023-12-26T21:55:27.945186+0000","last_active":"2023-12-26T21:55:28.940992+0000","last_peered":"2023-12-26T21:55:28.940992+0000","last_clean":"2023-12-26T21:55:28.940992+0000","last_became_active":"2023-12-26T21:55:27.944845+0000","last_became_peered":"2023-12-26T21:55:27.944845+0000","last_unstale":"2023-12-26T21:55:28.940992+0000","last_undegraded":"2023-12-26T21:55:28.940992+0000","last_fullsized":"2023-12-26T21:55:28.940992+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:07:11.181231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.946746+0000","last_change":"2023-12-26T21:55:28.944747+0000","last_active":"2023-12-26T21:55:28.946746+0000","last_peered":"2023-12-26T21:55:28.946746+0000","last_clean":"2023-12-26T21:55:28.946746+0000","last_became_active":"2023-12-26T21:55:28.944463+0000","last_became_peered":"2023-12-26T21:55:28.944463+0000","last_unstale":"2023-12-26T21:55:28.946746+0000","last_undegraded":"2023-12-26T21:55:28.946746+0000","last_fullsized":"2023-12-26T21:55:28.946746+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:37:17.640528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.938853+0000","last_change":"2023-12-26T21:55:27.938063+0000","last_active":"2023-12-26T21:55:28.938853+0000","last_peered":"2023-12-26T21:55:28.938853+0000","last_clean":"2023-12-26T21:55:28.938853+0000","last_became_active":"2023-12-26T21:55:27.937758+0000","last_became_peered":"2023-12-26T21:55:27.937758+0000","last_unstale":"2023-12-26T21:55:28.938853+0000","last_undegraded":"2023-12-26T21:55:28.938853+0000","last_fullsized":"2023-12-26T21:55:28.938853+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:41:23.169703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.940309+0000","last_change":"2023-12-26T21:55:28.935441+0000","last_active":"2023-12-26T21:55:28.940309+0000","last_peered":"2023-12-26T21:55:28.940309+0000","last_clean":"2023-12-26T21:55:28.940309+0000","last_became_active":"2023-12-26T21:55:28.935108+0000","last_became_peered":"2023-12-26T21:55:28.935108+0000","last_unstale":"2023-12-26T21:55:28.940309+0000","last_undegraded":"2023-12-26T21:55:28.940309+0000","last_fullsized":"2023-12-26T21:55:28.940309+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:54:02.768222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.936294+0000","last_change":"2023-12-26T21:55:27.942768+0000","last_active":"2023-12-26T21:55:28.936294+0000","last_peered":"2023-12-26T21:55:28.936294+0000","last_clean":"2023-12-26T21:55:28.936294+0000","last_became_active":"2023-12-26T21:55:27.941502+0000","last_became_peered":"2023-12-26T21:55:27.941502+0000","last_unstale":"2023-12-26T21:55:28.936294+0000","last_undegraded":"2023-12-26T21:55:28.936294+0000","last_fullsized":"2023-12-26T21:55:28.936294+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:11:55.886267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955480+0000","last_change":"2023-12-26T21:55:28.954281+0000","last_active":"2023-12-26T21:55:28.955480+0000","last_peered":"2023-12-26T21:55:28.955480+0000","last_clean":"2023-12-26T21:55:28.955480+0000","last_became_active":"2023-12-26T21:55:28.953959+0000","last_became_peered":"2023-12-26T21:55:28.953959+0000","last_unstale":"2023-12-26T21:55:28.955480+0000","last_undegraded":"2023-12-26T21:55:28.955480+0000","last_fullsized":"2023-12-26T21:55:28.955480+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:14.517699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"123'3","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954257+0000","last_change":"2023-12-26T21:55:27.934182+0000","last_active":"2023-12-26T21:55:28.954257+0000","last_peered":"2023-12-26T21:55:28.954257+0000","last_clean":"2023-12-26T21:55:28.954257+0000","last_became_active":"2023-12-26T21:55:27.933905+0000","last_became_peered":"2023-12-26T21:55:27.933905+0000","last_unstale":"2023-12-26T21:55:28.954257+0000","last_undegraded":"2023-12-26T21:55:28.954257+0000","last_fullsized":"2023-12-26T21:55:28.954257+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:35.490311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941368+0000","last_change":"2023-12-26T21:55:28.938504+0000","last_active":"2023-12-26T21:55:28.941368+0000","last_peered":"2023-12-26T21:55:28.941368+0000","last_clean":"2023-12-26T21:55:28.941368+0000","last_became_active":"2023-12-26T21:55:28.938195+0000","last_became_peered":"2023-12-26T21:55:28.938195+0000","last_unstale":"2023-12-26T21:55:28.941368+0000","last_undegraded":"2023-12-26T21:55:28.941368+0000","last_fullsized":"2023-12-26T21:55:28.941368+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:24:32.258021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"123'6","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.961182+0000","last_change":"2023-12-26T21:55:27.937280+0000","last_active":"2023-12-26T21:55:28.961182+0000","last_peered":"2023-12-26T21:55:28.961182+0000","last_clean":"2023-12-26T21:55:28.961182+0000","last_became_active":"2023-12-26T21:55:27.936969+0000","last_became_peered":"2023-12-26T21:55:27.936969+0000","last_unstale":"2023-12-26T21:55:28.961182+0000","last_undegraded":"2023-12-26T21:55:28.961182+0000","last_fullsized":"2023-12-26T21:55:28.961182+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:36:44.783191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957950+0000","last_change":"2023-12-26T21:55:28.955355+0000","last_active":"2023-12-26T21:55:28.957950+0000","last_peered":"2023-12-26T21:55:28.957950+0000","last_clean":"2023-12-26T21:55:28.957950+0000","last_became_active":"2023-12-26T21:55:28.953368+0000","last_became_peered":"2023-12-26T21:55:28.953368+0000","last_unstale":"2023-12-26T21:55:28.957950+0000","last_undegraded":"2023-12-26T21:55:28.957950+0000","last_fullsized":"2023-12-26T21:55:28.957950+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:25:14.133756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957595+0000","last_change":"2023-12-26T21:55:27.941254+0000","last_active":"2023-12-26T21:55:28.957595+0000","last_peered":"2023-12-26T21:55:28.957595+0000","last_clean":"2023-12-26T21:55:28.957595+0000","last_became_active":"2023-12-26T21:55:27.940962+0000","last_became_peered":"2023-12-26T21:55:27.940962+0000","last_unstale":"2023-12-26T21:55:28.957595+0000","last_undegraded":"2023-12-26T21:55:28.957595+0000","last_fullsized":"2023-12-26T21:55:28.957595+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:08.710785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954825+0000","last_change":"2023-12-26T21:55:28.950427+0000","last_active":"2023-12-26T21:55:28.954825+0000","last_peered":"2023-12-26T21:55:28.954825+0000","last_clean":"2023-12-26T21:55:28.954825+0000","last_became_active":"2023-12-26T21:55:28.949403+0000","last_became_peered":"2023-12-26T21:55:28.949403+0000","last_unstale":"2023-12-26T21:55:28.954825+0000","last_undegraded":"2023-12-26T21:55:28.954825+0000","last_fullsized":"2023-12-26T21:55:28.954825+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:40:39.012070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.932325+0000","last_change":"2023-12-26T21:55:27.937404+0000","last_active":"2023-12-26T21:55:28.932325+0000","last_peered":"2023-12-26T21:55:28.932325+0000","last_clean":"2023-12-26T21:55:28.932325+0000","last_became_active":"2023-12-26T21:55:27.937146+0000","last_became_peered":"2023-12-26T21:55:27.937146+0000","last_unstale":"2023-12-26T21:55:28.932325+0000","last_undegraded":"2023-12-26T21:55:28.932325+0000","last_fullsized":"2023-12-26T21:55:28.932325+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:06.410480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954954+0000","last_change":"2023-12-26T21:55:28.952850+0000","last_active":"2023-12-26T21:55:28.954954+0000","last_peered":"2023-12-26T21:55:28.954954+0000","last_clean":"2023-12-26T21:55:28.954954+0000","last_became_active":"2023-12-26T21:55:28.952575+0000","last_became_peered":"2023-12-26T21:55:28.952575+0000","last_unstale":"2023-12-26T21:55:28.954954+0000","last_undegraded":"2023-12-26T21:55:28.954954+0000","last_fullsized":"2023-12-26T21:55:28.954954+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:26:07.994316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.927823+0000","last_change":"2023-12-26T21:55:27.928144+0000","last_active":"2023-12-26T21:55:28.927823+0000","last_peered":"2023-12-26T21:55:28.927823+0000","last_clean":"2023-12-26T21:55:28.927823+0000","last_became_active":"2023-12-26T21:55:27.927858+0000","last_became_peered":"2023-12-26T21:55:27.927858+0000","last_unstale":"2023-12-26T21:55:28.927823+0000","last_undegraded":"2023-12-26T21:55:28.927823+0000","last_fullsized":"2023-12-26T21:55:28.927823+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:08.014072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.952492+0000","last_change":"2023-12-26T21:55:28.948005+0000","last_active":"2023-12-26T21:55:28.952492+0000","last_peered":"2023-12-26T21:55:28.952492+0000","last_clean":"2023-12-26T21:55:28.952492+0000","last_became_active":"2023-12-26T21:55:28.947697+0000","last_became_peered":"2023-12-26T21:55:28.947697+0000","last_unstale":"2023-12-26T21:55:28.952492+0000","last_undegraded":"2023-12-26T21:55:28.952492+0000","last_fullsized":"2023-12-26T21:55:28.952492+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:47.197569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"123'8","reported_seq":18,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.951210+0000","last_change":"2023-12-26T21:55:27.931862+0000","last_active":"2023-12-26T21:55:28.951210+0000","last_peered":"2023-12-26T21:55:28.951210+0000","last_clean":"2023-12-26T21:55:28.951210+0000","last_became_active":"2023-12-26T21:55:27.931563+0000","last_became_peered":"2023-12-26T21:55:27.931563+0000","last_unstale":"2023-12-26T21:55:28.951210+0000","last_undegraded":"2023-12-26T21:55:28.951210+0000","last_fullsized":"2023-12-26T21:55:28.951210+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:19:01.084609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941071+0000","last_change":"2023-12-26T21:55:28.938214+0000","last_active":"2023-12-26T21:55:28.941071+0000","last_peered":"2023-12-26T21:55:28.941071+0000","last_clean":"2023-12-26T21:55:28.941071+0000","last_became_active":"2023-12-26T21:55:28.937922+0000","last_became_peered":"2023-12-26T21:55:28.937922+0000","last_unstale":"2023-12-26T21:55:28.941071+0000","last_undegraded":"2023-12-26T21:55:28.941071+0000","last_fullsized":"2023-12-26T21:55:28.941071+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:34.158829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.942237+0000","last_change":"2023-12-26T21:55:27.936517+0000","last_active":"2023-12-26T21:55:28.942237+0000","last_peered":"2023-12-26T21:55:28.942237+0000","last_clean":"2023-12-26T21:55:28.942237+0000","last_became_active":"2023-12-26T21:55:27.936191+0000","last_became_peered":"2023-12-26T21:55:27.936191+0000","last_unstale":"2023-12-26T21:55:28.942237+0000","last_undegraded":"2023-12-26T21:55:28.942237+0000","last_fullsized":"2023-12-26T21:55:28.942237+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:05:04.439968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941087+0000","last_change":"2023-12-26T21:55:28.938294+0000","last_active":"2023-12-26T21:55:28.941087+0000","last_peered":"2023-12-26T21:55:28.941087+0000","last_clean":"2023-12-26T21:55:28.941087+0000","last_became_active":"2023-12-26T21:55:28.937964+0000","last_became_peered":"2023-12-26T21:55:28.937964+0000","last_unstale":"2023-12-26T21:55:28.941087+0000","last_undegraded":"2023-12-26T21:55:28.941087+0000","last_fullsized":"2023-12-26T21:55:28.941087+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:35:25.344565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954053+0000","last_change":"2023-12-26T21:55:27.932580+0000","last_active":"2023-12-26T21:55:28.954053+0000","last_peered":"2023-12-26T21:55:28.954053+0000","last_clean":"2023-12-26T21:55:28.954053+0000","last_became_active":"2023-12-26T21:55:27.932245+0000","last_became_peered":"2023-12-26T21:55:27.932245+0000","last_unstale":"2023-12-26T21:55:28.954053+0000","last_undegraded":"2023-12-26T21:55:28.954053+0000","last_fullsized":"2023-12-26T21:55:28.954053+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:32:25.291634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.934174+0000","last_change":"2023-12-26T21:55:28.931166+0000","last_active":"2023-12-26T21:55:28.934174+0000","last_peered":"2023-12-26T21:55:28.934174+0000","last_clean":"2023-12-26T21:55:28.934174+0000","last_became_active":"2023-12-26T21:55:28.930910+0000","last_became_peered":"2023-12-26T21:55:28.930910+0000","last_unstale":"2023-12-26T21:55:28.934174+0000","last_undegraded":"2023-12-26T21:55:28.934174+0000","last_fullsized":"2023-12-26T21:55:28.934174+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:04:27.509774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.948934+0000","last_change":"2023-12-26T21:55:27.941897+0000","last_active":"2023-12-26T21:55:28.948934+0000","last_peered":"2023-12-26T21:55:28.948934+0000","last_clean":"2023-12-26T21:55:28.948934+0000","last_became_active":"2023-12-26T21:55:27.941561+0000","last_became_peered":"2023-12-26T21:55:27.941561+0000","last_unstale":"2023-12-26T21:55:28.948934+0000","last_undegraded":"2023-12-26T21:55:28.948934+0000","last_fullsized":"2023-12-26T21:55:28.948934+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:22:21.911001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957469+0000","last_change":"2023-12-26T21:55:28.947262+0000","last_active":"2023-12-26T21:55:28.957469+0000","last_peered":"2023-12-26T21:55:28.957469+0000","last_clean":"2023-12-26T21:55:28.957469+0000","last_became_active":"2023-12-26T21:55:28.947018+0000","last_became_peered":"2023-12-26T21:55:28.947018+0000","last_unstale":"2023-12-26T21:55:28.957469+0000","last_undegraded":"2023-12-26T21:55:28.957469+0000","last_fullsized":"2023-12-26T21:55:28.957469+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:55:22.572475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.949644+0000","last_change":"2023-12-26T21:55:27.941828+0000","last_active":"2023-12-26T21:55:28.949644+0000","last_peered":"2023-12-26T21:55:28.949644+0000","last_clean":"2023-12-26T21:55:28.949644+0000","last_became_active":"2023-12-26T21:55:27.941202+0000","last_became_peered":"2023-12-26T21:55:27.941202+0000","last_unstale":"2023-12-26T21:55:28.949644+0000","last_undegraded":"2023-12-26T21:55:28.949644+0000","last_fullsized":"2023-12-26T21:55:28.949644+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:52:15.645769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956956+0000","last_change":"2023-12-26T21:55:28.946502+0000","last_active":"2023-12-26T21:55:28.956956+0000","last_peered":"2023-12-26T21:55:28.956956+0000","last_clean":"2023-12-26T21:55:28.956956+0000","last_became_active":"2023-12-26T21:55:28.946218+0000","last_became_peered":"2023-12-26T21:55:28.946218+0000","last_unstale":"2023-12-26T21:55:28.956956+0000","last_undegraded":"2023-12-26T21:55:28.956956+0000","last_fullsized":"2023-12-26T21:55:28.956956+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:30:14.400717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.930831+0000","last_change":"2023-12-26T21:55:27.934967+0000","last_active":"2023-12-26T21:55:28.930831+0000","last_peered":"2023-12-26T21:55:28.930831+0000","last_clean":"2023-12-26T21:55:28.930831+0000","last_became_active":"2023-12-26T21:55:27.934632+0000","last_became_peered":"2023-12-26T21:55:27.934632+0000","last_unstale":"2023-12-26T21:55:28.930831+0000","last_undegraded":"2023-12-26T21:55:28.930831+0000","last_fullsized":"2023-12-26T21:55:28.930831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:40:38.005900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.947330+0000","last_change":"2023-12-26T21:55:28.944677+0000","last_active":"2023-12-26T21:55:28.947330+0000","last_peered":"2023-12-26T21:55:28.947330+0000","last_clean":"2023-12-26T21:55:28.947330+0000","last_became_active":"2023-12-26T21:55:28.943860+0000","last_became_peered":"2023-12-26T21:55:28.943860+0000","last_unstale":"2023-12-26T21:55:28.947330+0000","last_undegraded":"2023-12-26T21:55:28.947330+0000","last_fullsized":"2023-12-26T21:55:28.947330+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:02:39.593740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.931998+0000","last_change":"2023-12-26T21:55:27.924300+0000","last_active":"2023-12-26T21:55:28.931998+0000","last_peered":"2023-12-26T21:55:28.931998+0000","last_clean":"2023-12-26T21:55:28.931998+0000","last_became_active":"2023-12-26T21:55:27.924011+0000","last_became_peered":"2023-12-26T21:55:27.924011+0000","last_unstale":"2023-12-26T21:55:28.931998+0000","last_undegraded":"2023-12-26T21:55:28.931998+0000","last_fullsized":"2023-12-26T21:55:28.931998+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:22:21.911001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954458+0000","last_change":"2023-12-26T21:55:28.949282+0000","last_active":"2023-12-26T21:55:28.954458+0000","last_peered":"2023-12-26T21:55:28.954458+0000","last_clean":"2023-12-26T21:55:28.954458+0000","last_became_active":"2023-12-26T21:55:28.949019+0000","last_became_peered":"2023-12-26T21:55:28.949019+0000","last_unstale":"2023-12-26T21:55:28.954458+0000","last_undegraded":"2023-12-26T21:55:28.954458+0000","last_fullsized":"2023-12-26T21:55:28.954458+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:21:21.611427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941964+0000","last_change":"2023-12-26T21:55:27.936707+0000","last_active":"2023-12-26T21:55:28.941964+0000","last_peered":"2023-12-26T21:55:28.941964+0000","last_clean":"2023-12-26T21:55:28.941964+0000","last_became_active":"2023-12-26T21:55:27.936185+0000","last_became_peered":"2023-12-26T21:55:27.936185+0000","last_unstale":"2023-12-26T21:55:28.941964+0000","last_undegraded":"2023-12-26T21:55:28.941964+0000","last_fullsized":"2023-12-26T21:55:28.941964+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:05:08.865568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.949009+0000","last_change":"2023-12-26T21:55:28.940878+0000","last_active":"2023-12-26T21:55:28.949009+0000","last_peered":"2023-12-26T21:55:28.949009+0000","last_clean":"2023-12-26T21:55:28.949009+0000","last_became_active":"2023-12-26T21:55:28.940609+0000","last_became_peered":"2023-12-26T21:55:28.940609+0000","last_unstale":"2023-12-26T21:55:28.949009+0000","last_undegraded":"2023-12-26T21:55:28.949009+0000","last_fullsized":"2023-12-26T21:55:28.949009+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:11.503441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.936565+0000","last_change":"2023-12-26T21:55:27.929058+0000","last_active":"2023-12-26T21:55:28.936565+0000","last_peered":"2023-12-26T21:55:28.936565+0000","last_clean":"2023-12-26T21:55:28.936565+0000","last_became_active":"2023-12-26T21:55:27.928274+0000","last_became_peered":"2023-12-26T21:55:27.928274+0000","last_unstale":"2023-12-26T21:55:28.936565+0000","last_undegraded":"2023-12-26T21:55:28.936565+0000","last_fullsized":"2023-12-26T21:55:28.936565+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:38.766212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.960006+0000","last_change":"2023-12-26T21:55:28.956071+0000","last_active":"2023-12-26T21:55:28.960006+0000","last_peered":"2023-12-26T21:55:28.960006+0000","last_clean":"2023-12-26T21:55:28.960006+0000","last_became_active":"2023-12-26T21:55:28.955796+0000","last_became_peered":"2023-12-26T21:55:28.955796+0000","last_unstale":"2023-12-26T21:55:28.960006+0000","last_undegraded":"2023-12-26T21:55:28.960006+0000","last_fullsized":"2023-12-26T21:55:28.960006+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:59:35.215536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.948655+0000","last_change":"2023-12-26T21:55:27.940113+0000","last_active":"2023-12-26T21:55:28.948655+0000","last_peered":"2023-12-26T21:55:28.948655+0000","last_clean":"2023-12-26T21:55:28.948655+0000","last_became_active":"2023-12-26T21:55:27.939829+0000","last_became_peered":"2023-12-26T21:55:27.939829+0000","last_unstale":"2023-12-26T21:55:28.948655+0000","last_undegraded":"2023-12-26T21:55:28.948655+0000","last_fullsized":"2023-12-26T21:55:28.948655+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:28:49.990101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954758+0000","last_change":"2023-12-26T21:55:28.950137+0000","last_active":"2023-12-26T21:55:28.954758+0000","last_peered":"2023-12-26T21:55:28.954758+0000","last_clean":"2023-12-26T21:55:28.954758+0000","last_became_active":"2023-12-26T21:55:28.949666+0000","last_became_peered":"2023-12-26T21:55:28.949666+0000","last_unstale":"2023-12-26T21:55:28.954758+0000","last_undegraded":"2023-12-26T21:55:28.954758+0000","last_fullsized":"2023-12-26T21:55:28.954758+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:16:52.701042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.949278+0000","last_change":"2023-12-26T21:55:27.939530+0000","last_active":"2023-12-26T21:55:28.949278+0000","last_peered":"2023-12-26T21:55:28.949278+0000","last_clean":"2023-12-26T21:55:28.949278+0000","last_became_active":"2023-12-26T21:55:27.939278+0000","last_became_peered":"2023-12-26T21:55:27.939278+0000","last_unstale":"2023-12-26T21:55:28.949278+0000","last_undegraded":"2023-12-26T21:55:28.949278+0000","last_fullsized":"2023-12-26T21:55:28.949278+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:56:19.585038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.953352+0000","last_change":"2023-12-26T21:55:28.948414+0000","last_active":"2023-12-26T21:55:28.953352+0000","last_peered":"2023-12-26T21:55:28.953352+0000","last_clean":"2023-12-26T21:55:28.953352+0000","last_became_active":"2023-12-26T21:55:28.947976+0000","last_became_peered":"2023-12-26T21:55:28.947976+0000","last_unstale":"2023-12-26T21:55:28.953352+0000","last_undegraded":"2023-12-26T21:55:28.953352+0000","last_fullsized":"2023-12-26T21:55:28.953352+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:32:52.790709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.931171+0000","last_change":"2023-12-26T21:55:27.936466+0000","last_active":"2023-12-26T21:55:28.931171+0000","last_peered":"2023-12-26T21:55:28.931171+0000","last_clean":"2023-12-26T21:55:28.931171+0000","last_became_active":"2023-12-26T21:55:27.936158+0000","last_became_peered":"2023-12-26T21:55:27.936158+0000","last_unstale":"2023-12-26T21:55:28.931171+0000","last_undegraded":"2023-12-26T21:55:28.931171+0000","last_fullsized":"2023-12-26T21:55:28.931171+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:26:07.817382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.947442+0000","last_change":"2023-12-26T21:55:28.937330+0000","last_active":"2023-12-26T21:55:28.947442+0000","last_peered":"2023-12-26T21:55:28.947442+0000","last_clean":"2023-12-26T21:55:28.947442+0000","last_became_active":"2023-12-26T21:55:28.936903+0000","last_became_peered":"2023-12-26T21:55:28.936903+0000","last_unstale":"2023-12-26T21:55:28.947442+0000","last_undegraded":"2023-12-26T21:55:28.947442+0000","last_fullsized":"2023-12-26T21:55:28.947442+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:20:41.343157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.942474+0000","last_change":"2023-12-26T21:55:27.937700+0000","last_active":"2023-12-26T21:55:28.942474+0000","last_peered":"2023-12-26T21:55:28.942474+0000","last_clean":"2023-12-26T21:55:28.942474+0000","last_became_active":"2023-12-26T21:55:27.937360+0000","last_became_peered":"2023-12-26T21:55:27.937360+0000","last_unstale":"2023-12-26T21:55:28.942474+0000","last_undegraded":"2023-12-26T21:55:28.942474+0000","last_fullsized":"2023-12-26T21:55:28.942474+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:39:07.519282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955280+0000","last_change":"2023-12-26T21:55:28.953589+0000","last_active":"2023-12-26T21:55:28.955280+0000","last_peered":"2023-12-26T21:55:28.955280+0000","last_clean":"2023-12-26T21:55:28.955280+0000","last_became_active":"2023-12-26T21:55:28.953342+0000","last_became_peered":"2023-12-26T21:55:28.953342+0000","last_unstale":"2023-12-26T21:55:28.955280+0000","last_undegraded":"2023-12-26T21:55:28.955280+0000","last_fullsized":"2023-12-26T21:55:28.955280+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:51.588745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.930930+0000","last_change":"2023-12-26T21:55:27.942244+0000","last_active":"2023-12-26T21:55:28.930930+0000","last_peered":"2023-12-26T21:55:28.930930+0000","last_clean":"2023-12-26T21:55:28.930930+0000","last_became_active":"2023-12-26T21:55:27.941921+0000","last_became_peered":"2023-12-26T21:55:27.941921+0000","last_unstale":"2023-12-26T21:55:28.930930+0000","last_undegraded":"2023-12-26T21:55:28.930930+0000","last_fullsized":"2023-12-26T21:55:28.930930+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:20.083375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.960002+0000","last_change":"2023-12-26T21:55:28.957129+0000","last_active":"2023-12-26T21:55:28.960002+0000","last_peered":"2023-12-26T21:55:28.960002+0000","last_clean":"2023-12-26T21:55:28.960002+0000","last_became_active":"2023-12-26T21:55:28.956842+0000","last_became_peered":"2023-12-26T21:55:28.956842+0000","last_unstale":"2023-12-26T21:55:28.960002+0000","last_undegraded":"2023-12-26T21:55:28.960002+0000","last_fullsized":"2023-12-26T21:55:28.960002+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:30:59.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.923574+0000","last_change":"2023-12-26T21:55:27.938256+0000","last_active":"2023-12-26T21:55:28.923574+0000","last_peered":"2023-12-26T21:55:28.923574+0000","last_clean":"2023-12-26T21:55:28.923574+0000","last_became_active":"2023-12-26T21:55:27.937952+0000","last_became_peered":"2023-12-26T21:55:27.937952+0000","last_unstale":"2023-12-26T21:55:28.923574+0000","last_undegraded":"2023-12-26T21:55:28.923574+0000","last_fullsized":"2023-12-26T21:55:28.923574+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:36:49.479164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.938572+0000","last_change":"2023-12-26T21:55:28.937069+0000","last_active":"2023-12-26T21:55:28.938572+0000","last_peered":"2023-12-26T21:55:28.938572+0000","last_clean":"2023-12-26T21:55:28.938572+0000","last_became_active":"2023-12-26T21:55:28.936202+0000","last_became_peered":"2023-12-26T21:55:28.936202+0000","last_unstale":"2023-12-26T21:55:28.938572+0000","last_undegraded":"2023-12-26T21:55:28.938572+0000","last_fullsized":"2023-12-26T21:55:28.938572+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:52:16.651940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941760+0000","last_change":"2023-12-26T21:55:27.936507+0000","last_active":"2023-12-26T21:55:28.941760+0000","last_peered":"2023-12-26T21:55:28.941760+0000","last_clean":"2023-12-26T21:55:28.941760+0000","last_became_active":"2023-12-26T21:55:27.935777+0000","last_became_peered":"2023-12-26T21:55:27.935777+0000","last_unstale":"2023-12-26T21:55:28.941760+0000","last_undegraded":"2023-12-26T21:55:28.941760+0000","last_fullsized":"2023-12-26T21:55:28.941760+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:03.249024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.952276+0000","last_change":"2023-12-26T21:55:28.948343+0000","last_active":"2023-12-26T21:55:28.952276+0000","last_peered":"2023-12-26T21:55:28.952276+0000","last_clean":"2023-12-26T21:55:28.952276+0000","last_became_active":"2023-12-26T21:55:28.948090+0000","last_became_peered":"2023-12-26T21:55:28.948090+0000","last_unstale":"2023-12-26T21:55:28.952276+0000","last_undegraded":"2023-12-26T21:55:28.952276+0000","last_fullsized":"2023-12-26T21:55:28.952276+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:34:50.695410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.942106+0000","last_change":"2023-12-26T21:55:27.939439+0000","last_active":"2023-12-26T21:55:28.942106+0000","last_peered":"2023-12-26T21:55:28.942106+0000","last_clean":"2023-12-26T21:55:28.942106+0000","last_became_active":"2023-12-26T21:55:27.939199+0000","last_became_peered":"2023-12-26T21:55:27.939199+0000","last_unstale":"2023-12-26T21:55:28.942106+0000","last_undegraded":"2023-12-26T21:55:28.942106+0000","last_fullsized":"2023-12-26T21:55:28.942106+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:57:34.757217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"113'227","reported_seq":645,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.927419+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:55:28.927419+0000","last_peered":"2023-12-26T21:55:28.927419+0000","last_clean":"2023-12-26T21:55:28.927419+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:55:28.927419+0000","last_undegraded":"2023-12-26T21:55:28.927419+0000","last_fullsized":"2023-12-26T21:55:28.927419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542367,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302876,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068964,"statfs":{"total":96636764160,"available":96326619136,"internally_reserved":0,"allocated":1527808,"data_stored":745630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.51200000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90700000000000003,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"max":{"1min":0.82199999999999995,"5min":0.94099999999999995,"15min":0.98299999999999998},"last":0.89100000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.67400000000000004},"min":{"1min":0.53000000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.79500000000000004,"5min":0.85899999999999999,"15min":1},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.54000000000000004,"5min":0.436,"15min":0.34200000000000003},"max":{"1min":0.96899999999999997,"5min":1.014,"15min":1.048},"last":1.2190000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":1.0509999999999999},"last":1.173},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66900000000000004,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.434},"max":{"1min":0.88900000000000001,"5min":0.99199999999999999,"15min":1.02},"last":1.143}]},{"osd":3,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94399999999999995,"5min":0.94499999999999995,"15min":1.052},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.59699999999999998,"5min":0.48299999999999998,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79500000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.753,"5min":1.036,"15min":1.036},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84299999999999997,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.51200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.329}]},{"osd":6,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.95599999999999996,"5min":1.117,"15min":1.117},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.44},"max":{"1min":0.998,"5min":1.006,"15min":1.006},"last":1.2509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509663,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306968,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064872,"statfs":{"total":96636764160,"available":96322428928,"internally_reserved":0,"allocated":1523712,"data_stored":745076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.55500000000000005,"5min":0.48999999999999999,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.82899999999999996}]},{"osd":1,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.60699999999999998,"5min":0.46100000000000002,"15min":0.40200000000000002},"max":{"1min":0.85399999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.73399999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.70699999999999996},"min":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.89200000000000002,"5min":0.93200000000000005,"15min":0.97099999999999997},"last":0.62},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69199999999999995,"15min":0.70699999999999996},"min":{"1min":0.54600000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.94699999999999995},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.65900000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90900000000000003,"5min":1.026,"15min":1.026},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.751},"min":{"1min":0.64000000000000001,"5min":0.46200000000000002,"15min":0.42799999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.0129999999999999},"last":0.69299999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.42399999999999999,"15min":0.372},"max":{"1min":0.82299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.48299999999999998,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.82099999999999995,"5min":0.91200000000000003,"15min":1.0529999999999999},"last":0.751}]},{"osd":5,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.97299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.874,"5min":1.0049999999999999,"15min":1.2609999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.54200000000000004,"5min":0.47099999999999997,"15min":0.438},"max":{"1min":0.98599999999999999,"5min":1.008,"15min":1.0920000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0920000000000001},"last":0.77700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509663,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311664,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060176,"statfs":{"total":96636764160,"available":96317620224,"internally_reserved":0,"allocated":2138112,"data_stored":1337092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.30499999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.628,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.42999999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.72799999999999998,"5min":1.353,"15min":1.353},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68300000000000005,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":1.488,"15min":1.488},"last":0.41499999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.42399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79300000000000004,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":1.506,"15min":1.506},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71199999999999997,"15min":0.746},"min":{"1min":0.58199999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0509999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76500000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77300000000000002},"min":{"1min":0.58199999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.032,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76200000000000001,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.627},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.54400000000000004},"max":{"1min":1.0529999999999999,"5min":1.615,"15min":1.615},"last":0.70999999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.77900000000000003},"min":{"1min":0.57899999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.075,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.752},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.04,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.68200000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509663,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302872,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068968,"statfs":{"total":96636764160,"available":96326623232,"internally_reserved":0,"allocated":1523712,"data_stored":745076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.245},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.45400000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.48999999999999999},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.30599999999999999},"max":{"1min":0.621,"5min":0.66800000000000004,"15min":0.73099999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.52200000000000002},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.317,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84699999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.44,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86699999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.72099999999999997,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.2749999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47699999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.58399999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77600000000000002,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.153},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.63}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.745,"15min":0.77900000000000003},"min":{"1min":0.56899999999999995,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.0700000000000001,"5min":1.093,"15min":1.214},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.749,"15min":0.79000000000000004},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.51700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.65300000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542368,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306972,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064868,"statfs":{"total":96636764160,"available":96322424832,"internally_reserved":0,"allocated":1527808,"data_stored":745146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61099999999999999,"5min":0.70399999999999996,"15min":0.85999999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.433}]},{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.48399999999999999},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.373},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.41599999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.60499999999999998},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78200000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.47199999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.58099999999999996,"5min":0.51300000000000001,"15min":0.38300000000000001},"max":{"1min":1.155,"5min":1.3,"15min":1.3},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.49199999999999999,"15min":0.48899999999999999},"max":{"1min":1.014,"5min":1.048,"15min":1.1759999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.1000000000000001,"5min":1.25,"15min":1.25},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.002,"5min":1.014,"15min":1.1319999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.53900000000000003,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.202,"15min":1.272},"last":0.751}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.69499999999999995,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1220000000000001},"last":0.625},{"interface":"front","average":{"1min":0.871,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.54300000000000004},"max":{"1min":1.1659999999999999,"5min":1.232,"15min":1.232},"last":0.73199999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509664,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311076,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060764,"statfs":{"total":96636764160,"available":96318222336,"internally_reserved":0,"allocated":1536000,"data_stored":745270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66200000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.63400000000000001,"5min":1.238,"15min":1.238},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":1.468,"15min":1.468},"last":0.432},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.65700000000000003,"5min":1.756,"15min":1.756},"last":0.40400000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.52700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.83299999999999996,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.51000000000000001,"5min":0.42299999999999999,"15min":0.42199999999999999},"max":{"1min":0.80800000000000005,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.63800000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.0029999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.61199999999999999,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.61899999999999999,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1399999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.79900000000000004},"min":{"1min":0.65900000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.056,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.70699999999999996,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.071,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.68400000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.1659999999999999},"last":0.623},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.101,"5min":1.272,"15min":1.272},"last":0.56299999999999994}]}]},{"osd":4,"up_from":14,"seq":60129542367,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306972,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064868,"statfs":{"total":96636764160,"available":96322424832,"internally_reserved":0,"allocated":1527808,"data_stored":745630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67300000000000004,"15min":0.68400000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.371},"max":{"1min":0.91700000000000004,"5min":1.006,"15min":1.008},"last":0.70699999999999996}]},{"osd":1,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.311},"max":{"1min":1.0229999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40000000000000002},"max":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97199999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.55200000000000005,"5min":0.46000000000000002,"15min":0.38800000000000001},"max":{"1min":0.875,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.50800000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.45300000000000001},"max":{"1min":0.90400000000000003,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.442,"15min":0.438},"max":{"1min":0.98799999999999999,"5min":1.046,"15min":1.046},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84499999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83499999999999996,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.59899999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.441,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.88600000000000001,"5min":1.133,"15min":1.133},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":0.88600000000000001,"5min":1.002,"15min":1.002},"last":0.746},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.58399999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89800000000000002,"5min":1.266,"15min":1.266},"last":0.72399999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509662,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303472,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068368,"statfs":{"total":96636764160,"available":96326008832,"internally_reserved":0,"allocated":2138112,"data_stored":1337036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.68400000000000005},"min":{"1min":0.48199999999999998,"5min":0.39800000000000002,"15min":0.27300000000000002},"max":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.995},"last":0.751},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.46800000000000003,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.91600000000000004,"15min":1.0369999999999999},"last":0.89900000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.69699999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.34000000000000002},"max":{"1min":0.88400000000000001,"5min":0.96399999999999997,"15min":1.04},"last":0.88},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66800000000000004,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.443,"15min":0.36899999999999999},"max":{"1min":0.86899999999999999,"5min":1.002,"15min":1.01},"last":0.67100000000000004}]},{"osd":2,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67800000000000005,"15min":0.71099999999999997},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.88500000000000001,"15min":1.3029999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.52400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.92400000000000004,"15min":0.95699999999999996},"last":0.85099999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.71999999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.39200000000000002},"max":{"1min":0.90500000000000003,"5min":0.93000000000000005,"15min":1.0640000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.47799999999999998},"max":{"1min":0.85499999999999998,"5min":0.92200000000000004,"15min":0.93899999999999995},"last":0.72699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.79300000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.41899999999999998,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.01},"last":0.60499999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.65000000000000002},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35499999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46899999999999997},"max":{"1min":0.85499999999999998,"5min":1.111,"15min":1.111},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67100000000000004,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95099999999999996,"15min":0.96299999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68700000000000006,"15min":0.69799999999999995},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.95399999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:55:38.188 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:55:38.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.388+0000 7f1bb33ff700 1 Processor -- start 2023-12-26T21:55:38.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.388+0000 7f1bb33ff700 1 -- start start 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.389+0000 7f1bb33ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1bac1596a0 0x7f1bac0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.389+0000 7f1bb33ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac159a70 0x7f1bac0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.389+0000 7f1bb33ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bac0afcd0 con 0x7f1bac0b0160 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.389+0000 7f1bb33ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac0b3e70 con 0x7f1bac1596a0 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.389+0000 7f1bb33ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac0b0cd0 con 0x7f1bac159a70 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.389+0000 7f1bb119b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1bac1596a0 0x7f1bac0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.389+0000 7f1bb119b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1bac1596a0 0x7f1bac0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49442/0 (socket says 172.21.15.131:49442) 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.389+0000 7f1bb119b700 1 -- 172.21.15.131:0/3846597293 learned_addr learned my addr 172.21.15.131:0/3846597293 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.389+0000 7f1bb199c700 1 --1- 172.21.15.131:0/3846597293 >> v1:172.21.15.131:6789/0 conn(0x7f1bac0b0160 0x7f1bac0b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44804/0 (socket says 172.21.15.131:44804) 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.390+0000 7f1bb099a700 1 --2- 172.21.15.131:0/3846597293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac159a70 0x7f1bac0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.390+0000 7f1bb119b700 1 -- 172.21.15.131:0/3846597293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac159a70 msgr2=0x7f1bac0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:38.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.390+0000 7f1bb119b700 1 --2- 172.21.15.131:0/3846597293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac159a70 0x7f1bac0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.390+0000 7f1bb119b700 1 -- 172.21.15.131:0/3846597293 >> v1:172.21.15.131:6789/0 conn(0x7f1bac0b0160 legacy=0x7f1bac0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.390+0000 7f1bb119b700 1 -- 172.21.15.131:0/3846597293 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac0b2e00 con 0x7f1bac1596a0 2023-12-26T21:55:38.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.390+0000 7f1bb119b700 1 --2- 172.21.15.131:0/3846597293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 0x7f1bac0b2380 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80099b0 tx=0x7f1ba8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50526f89e6e934e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:38.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.390+0000 7f1ba3fff700 1 -- 172.21.15.131:0/3846597293 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800b860 con 0x7f1bac1596a0 2023-12-26T21:55:38.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.390+0000 7f1ba3fff700 1 -- 172.21.15.131:0/3846597293 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba800b9c0 con 0x7f1bac1596a0 2023-12-26T21:55:38.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.390+0000 7f1ba3fff700 1 -- 172.21.15.131:0/3846597293 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800bd60 con 0x7f1bac1596a0 2023-12-26T21:55:38.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.391+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3846597293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 msgr2=0x7f1bac0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.391+0000 7f1bb33ff700 1 --2- 172.21.15.131:0/3846597293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 0x7f1bac0b2380 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80099b0 tx=0x7f1ba8009250 comp rx=0 tx=0).stop 2023-12-26T21:55:38.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.391+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3846597293 shutdown_connections 2023-12-26T21:55:38.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.391+0000 7f1bb33ff700 1 --2- 172.21.15.131:0/3846597293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 0x7f1bac0b2380 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.391+0000 7f1bb33ff700 1 --2- 172.21.15.131:0/3846597293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac159a70 0x7f1bac0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.391+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3846597293 >> 172.21.15.131:0/3846597293 conn(0x7f1bac06ece0 msgr2=0x7f1bac06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:38.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.391+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3846597293 shutdown_connections 2023-12-26T21:55:38.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.391+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3846597293 wait complete. 2023-12-26T21:55:38.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.392+0000 7f1bb33ff700 1 Processor -- start 2023-12-26T21:55:38.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.393+0000 7f1bb33ff700 1 -- start start 2023-12-26T21:55:38.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.393+0000 7f1bb33ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac0b0160 0x7f1bac0af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.393+0000 7f1bb33ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 0x7f1bac0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.393+0000 7f1bb33ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bac159a70 0x7f1bac169ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.393+0000 7f1bb33ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac15b4a0 con 0x7f1bac1596a0 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.393+0000 7f1bb33ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac15bd40 con 0x7f1bac159a70 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.393+0000 7f1bb33ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac15b210 con 0x7f1bac0b0160 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 0x7f1bac0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 0x7f1bac0c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49454/0 (socket says 172.21.15.131:49454) 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb099a700 1 -- 172.21.15.131:0/3219867808 learned_addr learned my addr 172.21.15.131:0/3219867808 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb119b700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac0b0160 0x7f1bac0af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb199c700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bac159a70 0x7f1bac169ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb099a700 1 -- 172.21.15.131:0/3219867808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac0b0160 msgr2=0x7f1bac0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb099a700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac0b0160 0x7f1bac0af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb099a700 1 -- 172.21.15.131:0/3219867808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bac159a70 msgr2=0x7f1bac169ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb099a700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bac159a70 0x7f1bac169ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb099a700 1 -- 172.21.15.131:0/3219867808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba80094f0 con 0x7f1bac1596a0 2023-12-26T21:55:38.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.394+0000 7f1bb099a700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 0x7f1bac0c9df0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00d890 tx=0x7f1b9c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:38.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.395+0000 7f1ba17fa700 1 -- 172.21.15.131:0/3219867808 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0125f0 con 0x7f1bac1596a0 2023-12-26T21:55:38.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.395+0000 7f1ba17fa700 1 -- 172.21.15.131:0/3219867808 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c012750 con 0x7f1bac1596a0 2023-12-26T21:55:38.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.395+0000 7f1ba17fa700 1 -- 172.21.15.131:0/3219867808 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c012b10 con 0x7f1bac1596a0 2023-12-26T21:55:38.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.395+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac16a530 con 0x7f1bac1596a0 2023-12-26T21:55:38.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.395+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac15b6f0 con 0x7f1bac1596a0 2023-12-26T21:55:38.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.396+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bac0b07f0 con 0x7f1bac1596a0 2023-12-26T21:55:38.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.397+0000 7f1ba17fa700 1 -- 172.21.15.131:0/3219867808 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1b9c01e460 con 0x7f1bac1596a0 2023-12-26T21:55:38.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.397+0000 7f1ba17fa700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b90066f30 0x7f1b900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.397+0000 7f1ba17fa700 1 -- 172.21.15.131:0/3219867808 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f1b9c084c90 con 0x7f1bac1596a0 2023-12-26T21:55:38.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.398+0000 7f1bb119b700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b90066f30 0x7f1b900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.398+0000 7f1bb119b700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b90066f30 0x7f1b900693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80099b0 tx=0x7f1ba8017000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:38.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.402+0000 7f1ba17fa700 1 -- 172.21.15.131:0/3219867808 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b9c051cd0 con 0x7f1bac1596a0 2023-12-26T21:55:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.592+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bac0657e0 con 0x7f1b90066f30 2023-12-26T21:55:38.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.600+0000 7f1ba17fa700 1 -- 172.21.15.131:0/3219867808 <== mgr.7159 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289444 (secure 0 0 0) 0x7f1bac0657e0 con 0x7f1b90066f30 2023-12-26T21:55:38.605 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:38.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.604+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b90066f30 msgr2=0x7f1b900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.604+0000 7f1bb33ff700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b90066f30 0x7f1b900693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80099b0 tx=0x7f1ba8017000 comp rx=0 tx=0).stop 2023-12-26T21:55:38.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.604+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 msgr2=0x7f1bac0c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.604+0000 7f1bb33ff700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 0x7f1bac0c9df0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00d890 tx=0x7f1b9c008040 comp rx=0 tx=0).stop 2023-12-26T21:55:38.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.605+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 shutdown_connections 2023-12-26T21:55:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.605+0000 7f1bb33ff700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bac159a70 0x7f1bac169ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.605+0000 7f1bb33ff700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bac0b0160 0x7f1bac0af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.605+0000 7f1bb33ff700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b90066f30 0x7f1b900693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.605+0000 7f1bb33ff700 1 --2- 172.21.15.131:0/3219867808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bac1596a0 0x7f1bac0c9df0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.605+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 >> 172.21.15.131:0/3219867808 conn(0x7f1bac06ece0 msgr2=0x7f1bac0ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.605+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 shutdown_connections 2023-12-26T21:55:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.605+0000 7f1bb33ff700 1 -- 172.21.15.131:0/3219867808 wait complete. 2023-12-26T21:55:38.609 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:55:38.623 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-26T21:55:36.885369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452872,"kb_used_data":13128,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752521848,"statfs":{"total":773094113280,"available":770582372352,"internally_reserved":0,"allocated":13443072,"data_stored":7145956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001682"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956465+0000","last_change":"2023-12-26T21:55:28.951885+0000","last_active":"2023-12-26T21:55:28.956465+0000","last_peered":"2023-12-26T21:55:28.956465+0000","last_clean":"2023-12-26T21:55:28.956465+0000","last_became_active":"2023-12-26T21:55:28.951617+0000","last_became_peered":"2023-12-26T21:55:28.951617+0000","last_unstale":"2023-12-26T21:55:28.956465+0000","last_undegraded":"2023-12-26T21:55:28.956465+0000","last_fullsized":"2023-12-26T21:55:28.956465+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:02:02.032351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.946675+0000","last_change":"2023-12-26T21:55:28.944219+0000","last_active":"2023-12-26T21:55:28.946675+0000","last_peered":"2023-12-26T21:55:28.946675+0000","last_clean":"2023-12-26T21:55:28.946675+0000","last_became_active":"2023-12-26T21:55:28.943949+0000","last_became_peered":"2023-12-26T21:55:28.943949+0000","last_unstale":"2023-12-26T21:55:28.946675+0000","last_undegraded":"2023-12-26T21:55:28.946675+0000","last_fullsized":"2023-12-26T21:55:28.946675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:20:16.839977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955152+0000","last_change":"2023-12-26T21:55:28.952166+0000","last_active":"2023-12-26T21:55:28.955152+0000","last_peered":"2023-12-26T21:55:28.955152+0000","last_clean":"2023-12-26T21:55:28.955152+0000","last_became_active":"2023-12-26T21:55:28.951876+0000","last_became_peered":"2023-12-26T21:55:28.951876+0000","last_unstale":"2023-12-26T21:55:28.955152+0000","last_undegraded":"2023-12-26T21:55:28.955152+0000","last_fullsized":"2023-12-26T21:55:28.955152+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:18:19.770941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956257+0000","last_change":"2023-12-26T21:55:28.948988+0000","last_active":"2023-12-26T21:55:28.956257+0000","last_peered":"2023-12-26T21:55:28.956257+0000","last_clean":"2023-12-26T21:55:28.956257+0000","last_became_active":"2023-12-26T21:55:28.945767+0000","last_became_peered":"2023-12-26T21:55:28.945767+0000","last_unstale":"2023-12-26T21:55:28.956257+0000","last_undegraded":"2023-12-26T21:55:28.956257+0000","last_fullsized":"2023-12-26T21:55:28.956257+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:33:03.696296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955260+0000","last_change":"2023-12-26T21:55:28.953010+0000","last_active":"2023-12-26T21:55:28.955260+0000","last_peered":"2023-12-26T21:55:28.955260+0000","last_clean":"2023-12-26T21:55:28.955260+0000","last_became_active":"2023-12-26T21:55:28.949970+0000","last_became_peered":"2023-12-26T21:55:28.949970+0000","last_unstale":"2023-12-26T21:55:28.955260+0000","last_undegraded":"2023-12-26T21:55:28.955260+0000","last_fullsized":"2023-12-26T21:55:28.955260+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:28:29.427026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955531+0000","last_change":"2023-12-26T21:55:28.952337+0000","last_active":"2023-12-26T21:55:28.955531+0000","last_peered":"2023-12-26T21:55:28.955531+0000","last_clean":"2023-12-26T21:55:28.955531+0000","last_became_active":"2023-12-26T21:55:28.951388+0000","last_became_peered":"2023-12-26T21:55:28.951388+0000","last_unstale":"2023-12-26T21:55:28.955531+0000","last_undegraded":"2023-12-26T21:55:28.955531+0000","last_fullsized":"2023-12-26T21:55:28.955531+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:53:42.935300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954843+0000","last_change":"2023-12-26T21:55:28.952055+0000","last_active":"2023-12-26T21:55:28.954843+0000","last_peered":"2023-12-26T21:55:28.954843+0000","last_clean":"2023-12-26T21:55:28.954843+0000","last_became_active":"2023-12-26T21:55:28.950302+0000","last_became_peered":"2023-12-26T21:55:28.950302+0000","last_unstale":"2023-12-26T21:55:28.954843+0000","last_undegraded":"2023-12-26T21:55:28.954843+0000","last_fullsized":"2023-12-26T21:55:28.954843+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:58:48.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954113+0000","last_change":"2023-12-26T21:55:28.949123+0000","last_active":"2023-12-26T21:55:28.954113+0000","last_peered":"2023-12-26T21:55:28.954113+0000","last_clean":"2023-12-26T21:55:28.954113+0000","last_became_active":"2023-12-26T21:55:28.948710+0000","last_became_peered":"2023-12-26T21:55:28.948710+0000","last_unstale":"2023-12-26T21:55:28.954113+0000","last_undegraded":"2023-12-26T21:55:28.954113+0000","last_fullsized":"2023-12-26T21:55:28.954113+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:26.806171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957051+0000","last_change":"2023-12-26T21:55:28.954784+0000","last_active":"2023-12-26T21:55:28.957051+0000","last_peered":"2023-12-26T21:55:28.957051+0000","last_clean":"2023-12-26T21:55:28.957051+0000","last_became_active":"2023-12-26T21:55:28.954379+0000","last_became_peered":"2023-12-26T21:55:28.954379+0000","last_unstale":"2023-12-26T21:55:28.957051+0000","last_undegraded":"2023-12-26T21:55:28.957051+0000","last_fullsized":"2023-12-26T21:55:28.957051+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:46:27.951865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.950325+0000","last_change":"2023-12-26T21:55:28.947592+0000","last_active":"2023-12-26T21:55:28.950325+0000","last_peered":"2023-12-26T21:55:28.950325+0000","last_clean":"2023-12-26T21:55:28.950325+0000","last_became_active":"2023-12-26T21:55:28.947338+0000","last_became_peered":"2023-12-26T21:55:28.947338+0000","last_unstale":"2023-12-26T21:55:28.950325+0000","last_undegraded":"2023-12-26T21:55:28.950325+0000","last_fullsized":"2023-12-26T21:55:28.950325+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:49:24.070796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954921+0000","last_change":"2023-12-26T21:55:28.952846+0000","last_active":"2023-12-26T21:55:28.954921+0000","last_peered":"2023-12-26T21:55:28.954921+0000","last_clean":"2023-12-26T21:55:28.954921+0000","last_became_active":"2023-12-26T21:55:28.952443+0000","last_became_peered":"2023-12-26T21:55:28.952443+0000","last_unstale":"2023-12-26T21:55:28.954921+0000","last_undegraded":"2023-12-26T21:55:28.954921+0000","last_fullsized":"2023-12-26T21:55:28.954921+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:36:19.760130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.951193+0000","last_change":"2023-12-26T21:55:28.949327+0000","last_active":"2023-12-26T21:55:28.951193+0000","last_peered":"2023-12-26T21:55:28.951193+0000","last_clean":"2023-12-26T21:55:28.951193+0000","last_became_active":"2023-12-26T21:55:28.949039+0000","last_became_peered":"2023-12-26T21:55:28.949039+0000","last_unstale":"2023-12-26T21:55:28.951193+0000","last_undegraded":"2023-12-26T21:55:28.951193+0000","last_fullsized":"2023-12-26T21:55:28.951193+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:46.647706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943403+0000","last_change":"2023-12-26T21:55:28.935945+0000","last_active":"2023-12-26T21:55:28.943403+0000","last_peered":"2023-12-26T21:55:28.943403+0000","last_clean":"2023-12-26T21:55:28.943403+0000","last_became_active":"2023-12-26T21:55:28.935453+0000","last_became_peered":"2023-12-26T21:55:28.935453+0000","last_unstale":"2023-12-26T21:55:28.943403+0000","last_undegraded":"2023-12-26T21:55:28.943403+0000","last_fullsized":"2023-12-26T21:55:28.943403+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:02.123742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.946718+0000","last_change":"2023-12-26T21:55:28.941523+0000","last_active":"2023-12-26T21:55:28.946718+0000","last_peered":"2023-12-26T21:55:28.946718+0000","last_clean":"2023-12-26T21:55:28.946718+0000","last_became_active":"2023-12-26T21:55:28.941265+0000","last_became_peered":"2023-12-26T21:55:28.941265+0000","last_unstale":"2023-12-26T21:55:28.946718+0000","last_undegraded":"2023-12-26T21:55:28.946718+0000","last_fullsized":"2023-12-26T21:55:28.946718+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:35:46.585609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955605+0000","last_change":"2023-12-26T21:55:28.951022+0000","last_active":"2023-12-26T21:55:28.955605+0000","last_peered":"2023-12-26T21:55:28.955605+0000","last_clean":"2023-12-26T21:55:28.955605+0000","last_became_active":"2023-12-26T21:55:28.950736+0000","last_became_peered":"2023-12-26T21:55:28.950736+0000","last_unstale":"2023-12-26T21:55:28.955605+0000","last_undegraded":"2023-12-26T21:55:28.955605+0000","last_fullsized":"2023-12-26T21:55:28.955605+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:24:04.095512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.961384+0000","last_change":"2023-12-26T21:55:28.958702+0000","last_active":"2023-12-26T21:55:28.961384+0000","last_peered":"2023-12-26T21:55:28.961384+0000","last_clean":"2023-12-26T21:55:28.961384+0000","last_became_active":"2023-12-26T21:55:28.958401+0000","last_became_peered":"2023-12-26T21:55:28.958401+0000","last_unstale":"2023-12-26T21:55:28.961384+0000","last_undegraded":"2023-12-26T21:55:28.961384+0000","last_fullsized":"2023-12-26T21:55:28.961384+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:04:00.547049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957155+0000","last_change":"2023-12-26T21:55:28.946413+0000","last_active":"2023-12-26T21:55:28.957155+0000","last_peered":"2023-12-26T21:55:28.957155+0000","last_clean":"2023-12-26T21:55:28.957155+0000","last_became_active":"2023-12-26T21:55:28.946073+0000","last_became_peered":"2023-12-26T21:55:28.946073+0000","last_unstale":"2023-12-26T21:55:28.957155+0000","last_undegraded":"2023-12-26T21:55:28.957155+0000","last_fullsized":"2023-12-26T21:55:28.957155+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:40:22.666559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954467+0000","last_change":"2023-12-26T21:55:28.950636+0000","last_active":"2023-12-26T21:55:28.954467+0000","last_peered":"2023-12-26T21:55:28.954467+0000","last_clean":"2023-12-26T21:55:28.954467+0000","last_became_active":"2023-12-26T21:55:28.949943+0000","last_became_peered":"2023-12-26T21:55:28.949943+0000","last_unstale":"2023-12-26T21:55:28.954467+0000","last_undegraded":"2023-12-26T21:55:28.954467+0000","last_fullsized":"2023-12-26T21:55:28.954467+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:24.696009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.958113+0000","last_change":"2023-12-26T21:55:28.951084+0000","last_active":"2023-12-26T21:55:28.958113+0000","last_peered":"2023-12-26T21:55:28.958113+0000","last_clean":"2023-12-26T21:55:28.958113+0000","last_became_active":"2023-12-26T21:55:28.950820+0000","last_became_peered":"2023-12-26T21:55:28.950820+0000","last_unstale":"2023-12-26T21:55:28.958113+0000","last_undegraded":"2023-12-26T21:55:28.958113+0000","last_fullsized":"2023-12-26T21:55:28.958113+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:08.290439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.947399+0000","last_change":"2023-12-26T21:55:28.944022+0000","last_active":"2023-12-26T21:55:28.947399+0000","last_peered":"2023-12-26T21:55:28.947399+0000","last_clean":"2023-12-26T21:55:28.947399+0000","last_became_active":"2023-12-26T21:55:28.943809+0000","last_became_peered":"2023-12-26T21:55:28.943809+0000","last_unstale":"2023-12-26T21:55:28.947399+0000","last_undegraded":"2023-12-26T21:55:28.947399+0000","last_fullsized":"2023-12-26T21:55:28.947399+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:20.853510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.952768+0000","last_change":"2023-12-26T21:55:28.945727+0000","last_active":"2023-12-26T21:55:28.952768+0000","last_peered":"2023-12-26T21:55:28.952768+0000","last_clean":"2023-12-26T21:55:28.952768+0000","last_became_active":"2023-12-26T21:55:28.945478+0000","last_became_peered":"2023-12-26T21:55:28.945478+0000","last_unstale":"2023-12-26T21:55:28.952768+0000","last_undegraded":"2023-12-26T21:55:28.952768+0000","last_fullsized":"2023-12-26T21:55:28.952768+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:21.089546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956895+0000","last_change":"2023-12-26T21:55:28.955777+0000","last_active":"2023-12-26T21:55:28.956895+0000","last_peered":"2023-12-26T21:55:28.956895+0000","last_clean":"2023-12-26T21:55:28.956895+0000","last_became_active":"2023-12-26T21:55:28.955463+0000","last_became_peered":"2023-12-26T21:55:28.955463+0000","last_unstale":"2023-12-26T21:55:28.956895+0000","last_undegraded":"2023-12-26T21:55:28.956895+0000","last_fullsized":"2023-12-26T21:55:28.956895+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:37:16.907722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.948409+0000","last_change":"2023-12-26T21:55:28.940305+0000","last_active":"2023-12-26T21:55:28.948409+0000","last_peered":"2023-12-26T21:55:28.948409+0000","last_clean":"2023-12-26T21:55:28.948409+0000","last_became_active":"2023-12-26T21:55:28.940058+0000","last_became_peered":"2023-12-26T21:55:28.940058+0000","last_unstale":"2023-12-26T21:55:28.948409+0000","last_undegraded":"2023-12-26T21:55:28.948409+0000","last_fullsized":"2023-12-26T21:55:28.948409+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:02:51.779760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.940323+0000","last_change":"2023-12-26T21:55:28.936544+0000","last_active":"2023-12-26T21:55:28.940323+0000","last_peered":"2023-12-26T21:55:28.940323+0000","last_clean":"2023-12-26T21:55:28.940323+0000","last_became_active":"2023-12-26T21:55:28.936305+0000","last_became_peered":"2023-12-26T21:55:28.936305+0000","last_unstale":"2023-12-26T21:55:28.940323+0000","last_undegraded":"2023-12-26T21:55:28.940323+0000","last_fullsized":"2023-12-26T21:55:28.940323+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:25:25.368816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.950366+0000","last_change":"2023-12-26T21:55:28.947504+0000","last_active":"2023-12-26T21:55:28.950366+0000","last_peered":"2023-12-26T21:55:28.950366+0000","last_clean":"2023-12-26T21:55:28.950366+0000","last_became_active":"2023-12-26T21:55:28.947222+0000","last_became_peered":"2023-12-26T21:55:28.947222+0000","last_unstale":"2023-12-26T21:55:28.950366+0000","last_undegraded":"2023-12-26T21:55:28.950366+0000","last_fullsized":"2023-12-26T21:55:28.950366+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:42:48.634872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943318+0000","last_change":"2023-12-26T21:55:28.936887+0000","last_active":"2023-12-26T21:55:28.943318+0000","last_peered":"2023-12-26T21:55:28.943318+0000","last_clean":"2023-12-26T21:55:28.943318+0000","last_became_active":"2023-12-26T21:55:28.936646+0000","last_became_peered":"2023-12-26T21:55:28.936646+0000","last_unstale":"2023-12-26T21:55:28.943318+0000","last_undegraded":"2023-12-26T21:55:28.943318+0000","last_fullsized":"2023-12-26T21:55:28.943318+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:06:44.485471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.940407+0000","last_change":"2023-12-26T21:55:28.937919+0000","last_active":"2023-12-26T21:55:28.940407+0000","last_peered":"2023-12-26T21:55:28.940407+0000","last_clean":"2023-12-26T21:55:28.940407+0000","last_became_active":"2023-12-26T21:55:28.937645+0000","last_became_peered":"2023-12-26T21:55:28.937645+0000","last_unstale":"2023-12-26T21:55:28.940407+0000","last_undegraded":"2023-12-26T21:55:28.940407+0000","last_fullsized":"2023-12-26T21:55:28.940407+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:45:01.825508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.953686+0000","last_change":"2023-12-26T21:55:28.949259+0000","last_active":"2023-12-26T21:55:28.953686+0000","last_peered":"2023-12-26T21:55:28.953686+0000","last_clean":"2023-12-26T21:55:28.953686+0000","last_became_active":"2023-12-26T21:55:28.948999+0000","last_became_peered":"2023-12-26T21:55:28.948999+0000","last_unstale":"2023-12-26T21:55:28.953686+0000","last_undegraded":"2023-12-26T21:55:28.953686+0000","last_fullsized":"2023-12-26T21:55:28.953686+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:25:02.095513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943385+0000","last_change":"2023-12-26T21:55:28.935745+0000","last_active":"2023-12-26T21:55:28.943385+0000","last_peered":"2023-12-26T21:55:28.943385+0000","last_clean":"2023-12-26T21:55:28.943385+0000","last_became_active":"2023-12-26T21:55:28.935078+0000","last_became_peered":"2023-12-26T21:55:28.935078+0000","last_unstale":"2023-12-26T21:55:28.943385+0000","last_undegraded":"2023-12-26T21:55:28.943385+0000","last_fullsized":"2023-12-26T21:55:28.943385+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:47:01.650507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943236+0000","last_change":"2023-12-26T21:55:28.940106+0000","last_active":"2023-12-26T21:55:28.943236+0000","last_peered":"2023-12-26T21:55:28.943236+0000","last_clean":"2023-12-26T21:55:28.943236+0000","last_became_active":"2023-12-26T21:55:28.939727+0000","last_became_peered":"2023-12-26T21:55:28.939727+0000","last_unstale":"2023-12-26T21:55:28.943236+0000","last_undegraded":"2023-12-26T21:55:28.943236+0000","last_fullsized":"2023-12-26T21:55:28.943236+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:08:49.842545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954788+0000","last_change":"2023-12-26T21:55:28.950126+0000","last_active":"2023-12-26T21:55:28.954788+0000","last_peered":"2023-12-26T21:55:28.954788+0000","last_clean":"2023-12-26T21:55:28.954788+0000","last_became_active":"2023-12-26T21:55:28.949338+0000","last_became_peered":"2023-12-26T21:55:28.949338+0000","last_unstale":"2023-12-26T21:55:28.954788+0000","last_undegraded":"2023-12-26T21:55:28.954788+0000","last_fullsized":"2023-12-26T21:55:28.954788+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:03:36.162383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.947347+0000","last_change":"2023-12-26T21:55:28.945216+0000","last_active":"2023-12-26T21:55:28.947347+0000","last_peered":"2023-12-26T21:55:28.947347+0000","last_clean":"2023-12-26T21:55:28.947347+0000","last_became_active":"2023-12-26T21:55:28.944954+0000","last_became_peered":"2023-12-26T21:55:28.944954+0000","last_unstale":"2023-12-26T21:55:28.947347+0000","last_undegraded":"2023-12-26T21:55:28.947347+0000","last_fullsized":"2023-12-26T21:55:28.947347+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:21:27.670725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.945496+0000","last_change":"2023-12-26T21:55:28.939360+0000","last_active":"2023-12-26T21:55:28.945496+0000","last_peered":"2023-12-26T21:55:28.945496+0000","last_clean":"2023-12-26T21:55:28.945496+0000","last_became_active":"2023-12-26T21:55:28.939089+0000","last_became_peered":"2023-12-26T21:55:28.939089+0000","last_unstale":"2023-12-26T21:55:28.945496+0000","last_undegraded":"2023-12-26T21:55:28.945496+0000","last_fullsized":"2023-12-26T21:55:28.945496+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:03:04.946777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"123'4","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.950621+0000","last_change":"2023-12-26T21:55:27.931297+0000","last_active":"2023-12-26T21:55:28.950621+0000","last_peered":"2023-12-26T21:55:28.950621+0000","last_clean":"2023-12-26T21:55:28.950621+0000","last_became_active":"2023-12-26T21:55:27.930888+0000","last_became_peered":"2023-12-26T21:55:27.930888+0000","last_unstale":"2023-12-26T21:55:28.950621+0000","last_undegraded":"2023-12-26T21:55:28.950621+0000","last_fullsized":"2023-12-26T21:55:28.950621+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:21:46.237715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943204+0000","last_change":"2023-12-26T21:55:28.936059+0000","last_active":"2023-12-26T21:55:28.943204+0000","last_peered":"2023-12-26T21:55:28.943204+0000","last_clean":"2023-12-26T21:55:28.943204+0000","last_became_active":"2023-12-26T21:55:28.935732+0000","last_became_peered":"2023-12-26T21:55:28.935732+0000","last_unstale":"2023-12-26T21:55:28.943204+0000","last_undegraded":"2023-12-26T21:55:28.943204+0000","last_fullsized":"2023-12-26T21:55:28.943204+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:31:13.763746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.958080+0000","last_change":"2023-12-26T21:55:27.942294+0000","last_active":"2023-12-26T21:55:28.958080+0000","last_peered":"2023-12-26T21:55:28.958080+0000","last_clean":"2023-12-26T21:55:28.958080+0000","last_became_active":"2023-12-26T21:55:27.942019+0000","last_became_peered":"2023-12-26T21:55:27.942019+0000","last_unstale":"2023-12-26T21:55:28.958080+0000","last_undegraded":"2023-12-26T21:55:28.958080+0000","last_fullsized":"2023-12-26T21:55:28.958080+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:36:14.042224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.949092+0000","last_change":"2023-12-26T21:55:28.940549+0000","last_active":"2023-12-26T21:55:28.949092+0000","last_peered":"2023-12-26T21:55:28.949092+0000","last_clean":"2023-12-26T21:55:28.949092+0000","last_became_active":"2023-12-26T21:55:28.937952+0000","last_became_peered":"2023-12-26T21:55:28.937952+0000","last_unstale":"2023-12-26T21:55:28.949092+0000","last_undegraded":"2023-12-26T21:55:28.949092+0000","last_fullsized":"2023-12-26T21:55:28.949092+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:56:12.537308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954988+0000","last_change":"2023-12-26T21:55:27.944761+0000","last_active":"2023-12-26T21:55:28.954988+0000","last_peered":"2023-12-26T21:55:28.954988+0000","last_clean":"2023-12-26T21:55:28.954988+0000","last_became_active":"2023-12-26T21:55:27.944454+0000","last_became_peered":"2023-12-26T21:55:27.944454+0000","last_unstale":"2023-12-26T21:55:28.954988+0000","last_undegraded":"2023-12-26T21:55:28.954988+0000","last_fullsized":"2023-12-26T21:55:28.954988+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:35:27.172805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.952873+0000","last_change":"2023-12-26T21:55:28.951410+0000","last_active":"2023-12-26T21:55:28.952873+0000","last_peered":"2023-12-26T21:55:28.952873+0000","last_clean":"2023-12-26T21:55:28.952873+0000","last_became_active":"2023-12-26T21:55:28.951185+0000","last_became_peered":"2023-12-26T21:55:28.951185+0000","last_unstale":"2023-12-26T21:55:28.952873+0000","last_undegraded":"2023-12-26T21:55:28.952873+0000","last_fullsized":"2023-12-26T21:55:28.952873+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:32:17.225200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956200+0000","last_change":"2023-12-26T21:55:27.940411+0000","last_active":"2023-12-26T21:55:28.956200+0000","last_peered":"2023-12-26T21:55:28.956200+0000","last_clean":"2023-12-26T21:55:28.956200+0000","last_became_active":"2023-12-26T21:55:27.940140+0000","last_became_peered":"2023-12-26T21:55:27.940140+0000","last_unstale":"2023-12-26T21:55:28.956200+0000","last_undegraded":"2023-12-26T21:55:28.956200+0000","last_fullsized":"2023-12-26T21:55:28.956200+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:51:19.847339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.960115+0000","last_change":"2023-12-26T21:55:28.955415+0000","last_active":"2023-12-26T21:55:28.960115+0000","last_peered":"2023-12-26T21:55:28.960115+0000","last_clean":"2023-12-26T21:55:28.960115+0000","last_became_active":"2023-12-26T21:55:28.955107+0000","last_became_peered":"2023-12-26T21:55:28.955107+0000","last_unstale":"2023-12-26T21:55:28.960115+0000","last_undegraded":"2023-12-26T21:55:28.960115+0000","last_fullsized":"2023-12-26T21:55:28.960115+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:07.416651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.946866+0000","last_change":"2023-12-26T21:55:27.933902+0000","last_active":"2023-12-26T21:55:28.946866+0000","last_peered":"2023-12-26T21:55:28.946866+0000","last_clean":"2023-12-26T21:55:28.946866+0000","last_became_active":"2023-12-26T21:55:27.933632+0000","last_became_peered":"2023-12-26T21:55:27.933632+0000","last_unstale":"2023-12-26T21:55:28.946866+0000","last_undegraded":"2023-12-26T21:55:28.946866+0000","last_fullsized":"2023-12-26T21:55:28.946866+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:06:45.641535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954282+0000","last_change":"2023-12-26T21:55:28.950844+0000","last_active":"2023-12-26T21:55:28.954282+0000","last_peered":"2023-12-26T21:55:28.954282+0000","last_clean":"2023-12-26T21:55:28.954282+0000","last_became_active":"2023-12-26T21:55:28.949928+0000","last_became_peered":"2023-12-26T21:55:28.949928+0000","last_unstale":"2023-12-26T21:55:28.954282+0000","last_undegraded":"2023-12-26T21:55:28.954282+0000","last_fullsized":"2023-12-26T21:55:28.954282+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:26:13.330832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957430+0000","last_change":"2023-12-26T21:55:27.940766+0000","last_active":"2023-12-26T21:55:28.957430+0000","last_peered":"2023-12-26T21:55:28.957430+0000","last_clean":"2023-12-26T21:55:28.957430+0000","last_became_active":"2023-12-26T21:55:27.940488+0000","last_became_peered":"2023-12-26T21:55:27.940488+0000","last_unstale":"2023-12-26T21:55:28.957430+0000","last_undegraded":"2023-12-26T21:55:28.957430+0000","last_fullsized":"2023-12-26T21:55:28.957430+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:55:25.001993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955907+0000","last_change":"2023-12-26T21:55:28.949084+0000","last_active":"2023-12-26T21:55:28.955907+0000","last_peered":"2023-12-26T21:55:28.955907+0000","last_clean":"2023-12-26T21:55:28.955907+0000","last_became_active":"2023-12-26T21:55:28.948739+0000","last_became_peered":"2023-12-26T21:55:28.948739+0000","last_unstale":"2023-12-26T21:55:28.955907+0000","last_undegraded":"2023-12-26T21:55:28.955907+0000","last_fullsized":"2023-12-26T21:55:28.955907+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:36:09.516332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.943808+0000","last_change":"2023-12-26T21:55:27.934641+0000","last_active":"2023-12-26T21:55:28.943808+0000","last_peered":"2023-12-26T21:55:28.943808+0000","last_clean":"2023-12-26T21:55:28.943808+0000","last_became_active":"2023-12-26T21:55:27.934248+0000","last_became_peered":"2023-12-26T21:55:27.934248+0000","last_unstale":"2023-12-26T21:55:28.943808+0000","last_undegraded":"2023-12-26T21:55:28.943808+0000","last_fullsized":"2023-12-26T21:55:28.943808+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:47:57.015825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957465+0000","last_change":"2023-12-26T21:55:28.955122+0000","last_active":"2023-12-26T21:55:28.957465+0000","last_peered":"2023-12-26T21:55:28.957465+0000","last_clean":"2023-12-26T21:55:28.957465+0000","last_became_active":"2023-12-26T21:55:28.954855+0000","last_became_peered":"2023-12-26T21:55:28.954855+0000","last_unstale":"2023-12-26T21:55:28.957465+0000","last_undegraded":"2023-12-26T21:55:28.957465+0000","last_fullsized":"2023-12-26T21:55:28.957465+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:19:02.090780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.940992+0000","last_change":"2023-12-26T21:55:27.945186+0000","last_active":"2023-12-26T21:55:28.940992+0000","last_peered":"2023-12-26T21:55:28.940992+0000","last_clean":"2023-12-26T21:55:28.940992+0000","last_became_active":"2023-12-26T21:55:27.944845+0000","last_became_peered":"2023-12-26T21:55:27.944845+0000","last_unstale":"2023-12-26T21:55:28.940992+0000","last_undegraded":"2023-12-26T21:55:28.940992+0000","last_fullsized":"2023-12-26T21:55:28.940992+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:07:11.181231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.946746+0000","last_change":"2023-12-26T21:55:28.944747+0000","last_active":"2023-12-26T21:55:28.946746+0000","last_peered":"2023-12-26T21:55:28.946746+0000","last_clean":"2023-12-26T21:55:28.946746+0000","last_became_active":"2023-12-26T21:55:28.944463+0000","last_became_peered":"2023-12-26T21:55:28.944463+0000","last_unstale":"2023-12-26T21:55:28.946746+0000","last_undegraded":"2023-12-26T21:55:28.946746+0000","last_fullsized":"2023-12-26T21:55:28.946746+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:37:17.640528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.938853+0000","last_change":"2023-12-26T21:55:27.938063+0000","last_active":"2023-12-26T21:55:28.938853+0000","last_peered":"2023-12-26T21:55:28.938853+0000","last_clean":"2023-12-26T21:55:28.938853+0000","last_became_active":"2023-12-26T21:55:27.937758+0000","last_became_peered":"2023-12-26T21:55:27.937758+0000","last_unstale":"2023-12-26T21:55:28.938853+0000","last_undegraded":"2023-12-26T21:55:28.938853+0000","last_fullsized":"2023-12-26T21:55:28.938853+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:41:23.169703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.940309+0000","last_change":"2023-12-26T21:55:28.935441+0000","last_active":"2023-12-26T21:55:28.940309+0000","last_peered":"2023-12-26T21:55:28.940309+0000","last_clean":"2023-12-26T21:55:28.940309+0000","last_became_active":"2023-12-26T21:55:28.935108+0000","last_became_peered":"2023-12-26T21:55:28.935108+0000","last_unstale":"2023-12-26T21:55:28.940309+0000","last_undegraded":"2023-12-26T21:55:28.940309+0000","last_fullsized":"2023-12-26T21:55:28.940309+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:54:02.768222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.936294+0000","last_change":"2023-12-26T21:55:27.942768+0000","last_active":"2023-12-26T21:55:28.936294+0000","last_peered":"2023-12-26T21:55:28.936294+0000","last_clean":"2023-12-26T21:55:28.936294+0000","last_became_active":"2023-12-26T21:55:27.941502+0000","last_became_peered":"2023-12-26T21:55:27.941502+0000","last_unstale":"2023-12-26T21:55:28.936294+0000","last_undegraded":"2023-12-26T21:55:28.936294+0000","last_fullsized":"2023-12-26T21:55:28.936294+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:11:55.886267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955480+0000","last_change":"2023-12-26T21:55:28.954281+0000","last_active":"2023-12-26T21:55:28.955480+0000","last_peered":"2023-12-26T21:55:28.955480+0000","last_clean":"2023-12-26T21:55:28.955480+0000","last_became_active":"2023-12-26T21:55:28.953959+0000","last_became_peered":"2023-12-26T21:55:28.953959+0000","last_unstale":"2023-12-26T21:55:28.955480+0000","last_undegraded":"2023-12-26T21:55:28.955480+0000","last_fullsized":"2023-12-26T21:55:28.955480+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:14.517699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"123'3","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954257+0000","last_change":"2023-12-26T21:55:27.934182+0000","last_active":"2023-12-26T21:55:28.954257+0000","last_peered":"2023-12-26T21:55:28.954257+0000","last_clean":"2023-12-26T21:55:28.954257+0000","last_became_active":"2023-12-26T21:55:27.933905+0000","last_became_peered":"2023-12-26T21:55:27.933905+0000","last_unstale":"2023-12-26T21:55:28.954257+0000","last_undegraded":"2023-12-26T21:55:28.954257+0000","last_fullsized":"2023-12-26T21:55:28.954257+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:35.490311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941368+0000","last_change":"2023-12-26T21:55:28.938504+0000","last_active":"2023-12-26T21:55:28.941368+0000","last_peered":"2023-12-26T21:55:28.941368+0000","last_clean":"2023-12-26T21:55:28.941368+0000","last_became_active":"2023-12-26T21:55:28.938195+0000","last_became_peered":"2023-12-26T21:55:28.938195+0000","last_unstale":"2023-12-26T21:55:28.941368+0000","last_undegraded":"2023-12-26T21:55:28.941368+0000","last_fullsized":"2023-12-26T21:55:28.941368+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:24:32.258021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"123'6","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.961182+0000","last_change":"2023-12-26T21:55:27.937280+0000","last_active":"2023-12-26T21:55:28.961182+0000","last_peered":"2023-12-26T21:55:28.961182+0000","last_clean":"2023-12-26T21:55:28.961182+0000","last_became_active":"2023-12-26T21:55:27.936969+0000","last_became_peered":"2023-12-26T21:55:27.936969+0000","last_unstale":"2023-12-26T21:55:28.961182+0000","last_undegraded":"2023-12-26T21:55:28.961182+0000","last_fullsized":"2023-12-26T21:55:28.961182+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:36:44.783191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957950+0000","last_change":"2023-12-26T21:55:28.955355+0000","last_active":"2023-12-26T21:55:28.957950+0000","last_peered":"2023-12-26T21:55:28.957950+0000","last_clean":"2023-12-26T21:55:28.957950+0000","last_became_active":"2023-12-26T21:55:28.953368+0000","last_became_peered":"2023-12-26T21:55:28.953368+0000","last_unstale":"2023-12-26T21:55:28.957950+0000","last_undegraded":"2023-12-26T21:55:28.957950+0000","last_fullsized":"2023-12-26T21:55:28.957950+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:25:14.133756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957595+0000","last_change":"2023-12-26T21:55:27.941254+0000","last_active":"2023-12-26T21:55:28.957595+0000","last_peered":"2023-12-26T21:55:28.957595+0000","last_clean":"2023-12-26T21:55:28.957595+0000","last_became_active":"2023-12-26T21:55:27.940962+0000","last_became_peered":"2023-12-26T21:55:27.940962+0000","last_unstale":"2023-12-26T21:55:28.957595+0000","last_undegraded":"2023-12-26T21:55:28.957595+0000","last_fullsized":"2023-12-26T21:55:28.957595+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:08.710785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954825+0000","last_change":"2023-12-26T21:55:28.950427+0000","last_active":"2023-12-26T21:55:28.954825+0000","last_peered":"2023-12-26T21:55:28.954825+0000","last_clean":"2023-12-26T21:55:28.954825+0000","last_became_active":"2023-12-26T21:55:28.949403+0000","last_became_peered":"2023-12-26T21:55:28.949403+0000","last_unstale":"2023-12-26T21:55:28.954825+0000","last_undegraded":"2023-12-26T21:55:28.954825+0000","last_fullsized":"2023-12-26T21:55:28.954825+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:40:39.012070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.932325+0000","last_change":"2023-12-26T21:55:27.937404+0000","last_active":"2023-12-26T21:55:28.932325+0000","last_peered":"2023-12-26T21:55:28.932325+0000","last_clean":"2023-12-26T21:55:28.932325+0000","last_became_active":"2023-12-26T21:55:27.937146+0000","last_became_peered":"2023-12-26T21:55:27.937146+0000","last_unstale":"2023-12-26T21:55:28.932325+0000","last_undegraded":"2023-12-26T21:55:28.932325+0000","last_fullsized":"2023-12-26T21:55:28.932325+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:06.410480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954954+0000","last_change":"2023-12-26T21:55:28.952850+0000","last_active":"2023-12-26T21:55:28.954954+0000","last_peered":"2023-12-26T21:55:28.954954+0000","last_clean":"2023-12-26T21:55:28.954954+0000","last_became_active":"2023-12-26T21:55:28.952575+0000","last_became_peered":"2023-12-26T21:55:28.952575+0000","last_unstale":"2023-12-26T21:55:28.954954+0000","last_undegraded":"2023-12-26T21:55:28.954954+0000","last_fullsized":"2023-12-26T21:55:28.954954+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:26:07.994316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.927823+0000","last_change":"2023-12-26T21:55:27.928144+0000","last_active":"2023-12-26T21:55:28.927823+0000","last_peered":"2023-12-26T21:55:28.927823+0000","last_clean":"2023-12-26T21:55:28.927823+0000","last_became_active":"2023-12-26T21:55:27.927858+0000","last_became_peered":"2023-12-26T21:55:27.927858+0000","last_unstale":"2023-12-26T21:55:28.927823+0000","last_undegraded":"2023-12-26T21:55:28.927823+0000","last_fullsized":"2023-12-26T21:55:28.927823+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:08.014072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.952492+0000","last_change":"2023-12-26T21:55:28.948005+0000","last_active":"2023-12-26T21:55:28.952492+0000","last_peered":"2023-12-26T21:55:28.952492+0000","last_clean":"2023-12-26T21:55:28.952492+0000","last_became_active":"2023-12-26T21:55:28.947697+0000","last_became_peered":"2023-12-26T21:55:28.947697+0000","last_unstale":"2023-12-26T21:55:28.952492+0000","last_undegraded":"2023-12-26T21:55:28.952492+0000","last_fullsized":"2023-12-26T21:55:28.952492+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:47.197569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"123'8","reported_seq":18,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.951210+0000","last_change":"2023-12-26T21:55:27.931862+0000","last_active":"2023-12-26T21:55:28.951210+0000","last_peered":"2023-12-26T21:55:28.951210+0000","last_clean":"2023-12-26T21:55:28.951210+0000","last_became_active":"2023-12-26T21:55:27.931563+0000","last_became_peered":"2023-12-26T21:55:27.931563+0000","last_unstale":"2023-12-26T21:55:28.951210+0000","last_undegraded":"2023-12-26T21:55:28.951210+0000","last_fullsized":"2023-12-26T21:55:28.951210+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:19:01.084609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941071+0000","last_change":"2023-12-26T21:55:28.938214+0000","last_active":"2023-12-26T21:55:28.941071+0000","last_peered":"2023-12-26T21:55:28.941071+0000","last_clean":"2023-12-26T21:55:28.941071+0000","last_became_active":"2023-12-26T21:55:28.937922+0000","last_became_peered":"2023-12-26T21:55:28.937922+0000","last_unstale":"2023-12-26T21:55:28.941071+0000","last_undegraded":"2023-12-26T21:55:28.941071+0000","last_fullsized":"2023-12-26T21:55:28.941071+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:34.158829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.942237+0000","last_change":"2023-12-26T21:55:27.936517+0000","last_active":"2023-12-26T21:55:28.942237+0000","last_peered":"2023-12-26T21:55:28.942237+0000","last_clean":"2023-12-26T21:55:28.942237+0000","last_became_active":"2023-12-26T21:55:27.936191+0000","last_became_peered":"2023-12-26T21:55:27.936191+0000","last_unstale":"2023-12-26T21:55:28.942237+0000","last_undegraded":"2023-12-26T21:55:28.942237+0000","last_fullsized":"2023-12-26T21:55:28.942237+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:05:04.439968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941087+0000","last_change":"2023-12-26T21:55:28.938294+0000","last_active":"2023-12-26T21:55:28.941087+0000","last_peered":"2023-12-26T21:55:28.941087+0000","last_clean":"2023-12-26T21:55:28.941087+0000","last_became_active":"2023-12-26T21:55:28.937964+0000","last_became_peered":"2023-12-26T21:55:28.937964+0000","last_unstale":"2023-12-26T21:55:28.941087+0000","last_undegraded":"2023-12-26T21:55:28.941087+0000","last_fullsized":"2023-12-26T21:55:28.941087+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:35:25.344565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954053+0000","last_change":"2023-12-26T21:55:27.932580+0000","last_active":"2023-12-26T21:55:28.954053+0000","last_peered":"2023-12-26T21:55:28.954053+0000","last_clean":"2023-12-26T21:55:28.954053+0000","last_became_active":"2023-12-26T21:55:27.932245+0000","last_became_peered":"2023-12-26T21:55:27.932245+0000","last_unstale":"2023-12-26T21:55:28.954053+0000","last_undegraded":"2023-12-26T21:55:28.954053+0000","last_fullsized":"2023-12-26T21:55:28.954053+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:32:25.291634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.934174+0000","last_change":"2023-12-26T21:55:28.931166+0000","last_active":"2023-12-26T21:55:28.934174+0000","last_peered":"2023-12-26T21:55:28.934174+0000","last_clean":"2023-12-26T21:55:28.934174+0000","last_became_active":"2023-12-26T21:55:28.930910+0000","last_became_peered":"2023-12-26T21:55:28.930910+0000","last_unstale":"2023-12-26T21:55:28.934174+0000","last_undegraded":"2023-12-26T21:55:28.934174+0000","last_fullsized":"2023-12-26T21:55:28.934174+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:04:27.509774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.948934+0000","last_change":"2023-12-26T21:55:27.941897+0000","last_active":"2023-12-26T21:55:28.948934+0000","last_peered":"2023-12-26T21:55:28.948934+0000","last_clean":"2023-12-26T21:55:28.948934+0000","last_became_active":"2023-12-26T21:55:27.941561+0000","last_became_peered":"2023-12-26T21:55:27.941561+0000","last_unstale":"2023-12-26T21:55:28.948934+0000","last_undegraded":"2023-12-26T21:55:28.948934+0000","last_fullsized":"2023-12-26T21:55:28.948934+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:22:21.911001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.957469+0000","last_change":"2023-12-26T21:55:28.947262+0000","last_active":"2023-12-26T21:55:28.957469+0000","last_peered":"2023-12-26T21:55:28.957469+0000","last_clean":"2023-12-26T21:55:28.957469+0000","last_became_active":"2023-12-26T21:55:28.947018+0000","last_became_peered":"2023-12-26T21:55:28.947018+0000","last_unstale":"2023-12-26T21:55:28.957469+0000","last_undegraded":"2023-12-26T21:55:28.957469+0000","last_fullsized":"2023-12-26T21:55:28.957469+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:55:22.572475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.949644+0000","last_change":"2023-12-26T21:55:27.941828+0000","last_active":"2023-12-26T21:55:28.949644+0000","last_peered":"2023-12-26T21:55:28.949644+0000","last_clean":"2023-12-26T21:55:28.949644+0000","last_became_active":"2023-12-26T21:55:27.941202+0000","last_became_peered":"2023-12-26T21:55:27.941202+0000","last_unstale":"2023-12-26T21:55:28.949644+0000","last_undegraded":"2023-12-26T21:55:28.949644+0000","last_fullsized":"2023-12-26T21:55:28.949644+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:52:15.645769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.956956+0000","last_change":"2023-12-26T21:55:28.946502+0000","last_active":"2023-12-26T21:55:28.956956+0000","last_peered":"2023-12-26T21:55:28.956956+0000","last_clean":"2023-12-26T21:55:28.956956+0000","last_became_active":"2023-12-26T21:55:28.946218+0000","last_became_peered":"2023-12-26T21:55:28.946218+0000","last_unstale":"2023-12-26T21:55:28.956956+0000","last_undegraded":"2023-12-26T21:55:28.956956+0000","last_fullsized":"2023-12-26T21:55:28.956956+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:30:14.400717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.930831+0000","last_change":"2023-12-26T21:55:27.934967+0000","last_active":"2023-12-26T21:55:28.930831+0000","last_peered":"2023-12-26T21:55:28.930831+0000","last_clean":"2023-12-26T21:55:28.930831+0000","last_became_active":"2023-12-26T21:55:27.934632+0000","last_became_peered":"2023-12-26T21:55:27.934632+0000","last_unstale":"2023-12-26T21:55:28.930831+0000","last_undegraded":"2023-12-26T21:55:28.930831+0000","last_fullsized":"2023-12-26T21:55:28.930831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:40:38.005900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.947330+0000","last_change":"2023-12-26T21:55:28.944677+0000","last_active":"2023-12-26T21:55:28.947330+0000","last_peered":"2023-12-26T21:55:28.947330+0000","last_clean":"2023-12-26T21:55:28.947330+0000","last_became_active":"2023-12-26T21:55:28.943860+0000","last_became_peered":"2023-12-26T21:55:28.943860+0000","last_unstale":"2023-12-26T21:55:28.947330+0000","last_undegraded":"2023-12-26T21:55:28.947330+0000","last_fullsized":"2023-12-26T21:55:28.947330+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:02:39.593740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.931998+0000","last_change":"2023-12-26T21:55:27.924300+0000","last_active":"2023-12-26T21:55:28.931998+0000","last_peered":"2023-12-26T21:55:28.931998+0000","last_clean":"2023-12-26T21:55:28.931998+0000","last_became_active":"2023-12-26T21:55:27.924011+0000","last_became_peered":"2023-12-26T21:55:27.924011+0000","last_unstale":"2023-12-26T21:55:28.931998+0000","last_undegraded":"2023-12-26T21:55:28.931998+0000","last_fullsized":"2023-12-26T21:55:28.931998+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:22:21.911001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954458+0000","last_change":"2023-12-26T21:55:28.949282+0000","last_active":"2023-12-26T21:55:28.954458+0000","last_peered":"2023-12-26T21:55:28.954458+0000","last_clean":"2023-12-26T21:55:28.954458+0000","last_became_active":"2023-12-26T21:55:28.949019+0000","last_became_peered":"2023-12-26T21:55:28.949019+0000","last_unstale":"2023-12-26T21:55:28.954458+0000","last_undegraded":"2023-12-26T21:55:28.954458+0000","last_fullsized":"2023-12-26T21:55:28.954458+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:21:21.611427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941964+0000","last_change":"2023-12-26T21:55:27.936707+0000","last_active":"2023-12-26T21:55:28.941964+0000","last_peered":"2023-12-26T21:55:28.941964+0000","last_clean":"2023-12-26T21:55:28.941964+0000","last_became_active":"2023-12-26T21:55:27.936185+0000","last_became_peered":"2023-12-26T21:55:27.936185+0000","last_unstale":"2023-12-26T21:55:28.941964+0000","last_undegraded":"2023-12-26T21:55:28.941964+0000","last_fullsized":"2023-12-26T21:55:28.941964+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:05:08.865568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.949009+0000","last_change":"2023-12-26T21:55:28.940878+0000","last_active":"2023-12-26T21:55:28.949009+0000","last_peered":"2023-12-26T21:55:28.949009+0000","last_clean":"2023-12-26T21:55:28.949009+0000","last_became_active":"2023-12-26T21:55:28.940609+0000","last_became_peered":"2023-12-26T21:55:28.940609+0000","last_unstale":"2023-12-26T21:55:28.949009+0000","last_undegraded":"2023-12-26T21:55:28.949009+0000","last_fullsized":"2023-12-26T21:55:28.949009+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:11.503441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"123'2","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.936565+0000","last_change":"2023-12-26T21:55:27.929058+0000","last_active":"2023-12-26T21:55:28.936565+0000","last_peered":"2023-12-26T21:55:28.936565+0000","last_clean":"2023-12-26T21:55:28.936565+0000","last_became_active":"2023-12-26T21:55:27.928274+0000","last_became_peered":"2023-12-26T21:55:27.928274+0000","last_unstale":"2023-12-26T21:55:28.936565+0000","last_undegraded":"2023-12-26T21:55:28.936565+0000","last_fullsized":"2023-12-26T21:55:28.936565+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:38.766212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.960006+0000","last_change":"2023-12-26T21:55:28.956071+0000","last_active":"2023-12-26T21:55:28.960006+0000","last_peered":"2023-12-26T21:55:28.960006+0000","last_clean":"2023-12-26T21:55:28.960006+0000","last_became_active":"2023-12-26T21:55:28.955796+0000","last_became_peered":"2023-12-26T21:55:28.955796+0000","last_unstale":"2023-12-26T21:55:28.960006+0000","last_undegraded":"2023-12-26T21:55:28.960006+0000","last_fullsized":"2023-12-26T21:55:28.960006+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T21:59:35.215536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.948655+0000","last_change":"2023-12-26T21:55:27.940113+0000","last_active":"2023-12-26T21:55:28.948655+0000","last_peered":"2023-12-26T21:55:28.948655+0000","last_clean":"2023-12-26T21:55:28.948655+0000","last_became_active":"2023-12-26T21:55:27.939829+0000","last_became_peered":"2023-12-26T21:55:27.939829+0000","last_unstale":"2023-12-26T21:55:28.948655+0000","last_undegraded":"2023-12-26T21:55:28.948655+0000","last_fullsized":"2023-12-26T21:55:28.948655+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:28:49.990101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.954758+0000","last_change":"2023-12-26T21:55:28.950137+0000","last_active":"2023-12-26T21:55:28.954758+0000","last_peered":"2023-12-26T21:55:28.954758+0000","last_clean":"2023-12-26T21:55:28.954758+0000","last_became_active":"2023-12-26T21:55:28.949666+0000","last_became_peered":"2023-12-26T21:55:28.949666+0000","last_unstale":"2023-12-26T21:55:28.954758+0000","last_undegraded":"2023-12-26T21:55:28.954758+0000","last_fullsized":"2023-12-26T21:55:28.954758+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:16:52.701042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.949278+0000","last_change":"2023-12-26T21:55:27.939530+0000","last_active":"2023-12-26T21:55:28.949278+0000","last_peered":"2023-12-26T21:55:28.949278+0000","last_clean":"2023-12-26T21:55:28.949278+0000","last_became_active":"2023-12-26T21:55:27.939278+0000","last_became_peered":"2023-12-26T21:55:27.939278+0000","last_unstale":"2023-12-26T21:55:28.949278+0000","last_undegraded":"2023-12-26T21:55:28.949278+0000","last_fullsized":"2023-12-26T21:55:28.949278+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:56:19.585038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.953352+0000","last_change":"2023-12-26T21:55:28.948414+0000","last_active":"2023-12-26T21:55:28.953352+0000","last_peered":"2023-12-26T21:55:28.953352+0000","last_clean":"2023-12-26T21:55:28.953352+0000","last_became_active":"2023-12-26T21:55:28.947976+0000","last_became_peered":"2023-12-26T21:55:28.947976+0000","last_unstale":"2023-12-26T21:55:28.953352+0000","last_undegraded":"2023-12-26T21:55:28.953352+0000","last_fullsized":"2023-12-26T21:55:28.953352+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:32:52.790709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.931171+0000","last_change":"2023-12-26T21:55:27.936466+0000","last_active":"2023-12-26T21:55:28.931171+0000","last_peered":"2023-12-26T21:55:28.931171+0000","last_clean":"2023-12-26T21:55:28.931171+0000","last_became_active":"2023-12-26T21:55:27.936158+0000","last_became_peered":"2023-12-26T21:55:27.936158+0000","last_unstale":"2023-12-26T21:55:28.931171+0000","last_undegraded":"2023-12-26T21:55:28.931171+0000","last_fullsized":"2023-12-26T21:55:28.931171+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:26:07.817382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.947442+0000","last_change":"2023-12-26T21:55:28.937330+0000","last_active":"2023-12-26T21:55:28.947442+0000","last_peered":"2023-12-26T21:55:28.947442+0000","last_clean":"2023-12-26T21:55:28.947442+0000","last_became_active":"2023-12-26T21:55:28.936903+0000","last_became_peered":"2023-12-26T21:55:28.936903+0000","last_unstale":"2023-12-26T21:55:28.947442+0000","last_undegraded":"2023-12-26T21:55:28.947442+0000","last_fullsized":"2023-12-26T21:55:28.947442+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:20:41.343157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.942474+0000","last_change":"2023-12-26T21:55:27.937700+0000","last_active":"2023-12-26T21:55:28.942474+0000","last_peered":"2023-12-26T21:55:28.942474+0000","last_clean":"2023-12-26T21:55:28.942474+0000","last_became_active":"2023-12-26T21:55:27.937360+0000","last_became_peered":"2023-12-26T21:55:27.937360+0000","last_unstale":"2023-12-26T21:55:28.942474+0000","last_undegraded":"2023-12-26T21:55:28.942474+0000","last_fullsized":"2023-12-26T21:55:28.942474+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:39:07.519282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.955280+0000","last_change":"2023-12-26T21:55:28.953589+0000","last_active":"2023-12-26T21:55:28.955280+0000","last_peered":"2023-12-26T21:55:28.955280+0000","last_clean":"2023-12-26T21:55:28.955280+0000","last_became_active":"2023-12-26T21:55:28.953342+0000","last_became_peered":"2023-12-26T21:55:28.953342+0000","last_unstale":"2023-12-26T21:55:28.955280+0000","last_undegraded":"2023-12-26T21:55:28.955280+0000","last_fullsized":"2023-12-26T21:55:28.955280+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:50:51.588745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.930930+0000","last_change":"2023-12-26T21:55:27.942244+0000","last_active":"2023-12-26T21:55:28.930930+0000","last_peered":"2023-12-26T21:55:28.930930+0000","last_clean":"2023-12-26T21:55:28.930930+0000","last_became_active":"2023-12-26T21:55:27.941921+0000","last_became_peered":"2023-12-26T21:55:27.941921+0000","last_unstale":"2023-12-26T21:55:28.930930+0000","last_undegraded":"2023-12-26T21:55:28.930930+0000","last_fullsized":"2023-12-26T21:55:28.930930+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:20.083375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.960002+0000","last_change":"2023-12-26T21:55:28.957129+0000","last_active":"2023-12-26T21:55:28.960002+0000","last_peered":"2023-12-26T21:55:28.960002+0000","last_clean":"2023-12-26T21:55:28.960002+0000","last_became_active":"2023-12-26T21:55:28.956842+0000","last_became_peered":"2023-12-26T21:55:28.956842+0000","last_unstale":"2023-12-26T21:55:28.960002+0000","last_undegraded":"2023-12-26T21:55:28.960002+0000","last_fullsized":"2023-12-26T21:55:28.960002+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:30:59.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.923574+0000","last_change":"2023-12-26T21:55:27.938256+0000","last_active":"2023-12-26T21:55:28.923574+0000","last_peered":"2023-12-26T21:55:28.923574+0000","last_clean":"2023-12-26T21:55:28.923574+0000","last_became_active":"2023-12-26T21:55:27.937952+0000","last_became_peered":"2023-12-26T21:55:27.937952+0000","last_unstale":"2023-12-26T21:55:28.923574+0000","last_undegraded":"2023-12-26T21:55:28.923574+0000","last_fullsized":"2023-12-26T21:55:28.923574+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:36:49.479164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.938572+0000","last_change":"2023-12-26T21:55:28.937069+0000","last_active":"2023-12-26T21:55:28.938572+0000","last_peered":"2023-12-26T21:55:28.938572+0000","last_clean":"2023-12-26T21:55:28.938572+0000","last_became_active":"2023-12-26T21:55:28.936202+0000","last_became_peered":"2023-12-26T21:55:28.936202+0000","last_unstale":"2023-12-26T21:55:28.938572+0000","last_undegraded":"2023-12-26T21:55:28.938572+0000","last_fullsized":"2023-12-26T21:55:28.938572+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:52:16.651940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.941760+0000","last_change":"2023-12-26T21:55:27.936507+0000","last_active":"2023-12-26T21:55:28.941760+0000","last_peered":"2023-12-26T21:55:28.941760+0000","last_clean":"2023-12-26T21:55:28.941760+0000","last_became_active":"2023-12-26T21:55:27.935777+0000","last_became_peered":"2023-12-26T21:55:27.935777+0000","last_unstale":"2023-12-26T21:55:28.941760+0000","last_undegraded":"2023-12-26T21:55:28.941760+0000","last_fullsized":"2023-12-26T21:55:28.941760+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:03.249024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.952276+0000","last_change":"2023-12-26T21:55:28.948343+0000","last_active":"2023-12-26T21:55:28.952276+0000","last_peered":"2023-12-26T21:55:28.952276+0000","last_clean":"2023-12-26T21:55:28.952276+0000","last_became_active":"2023-12-26T21:55:28.948090+0000","last_became_peered":"2023-12-26T21:55:28.948090+0000","last_unstale":"2023-12-26T21:55:28.952276+0000","last_undegraded":"2023-12-26T21:55:28.952276+0000","last_fullsized":"2023-12-26T21:55:28.952276+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:27.908991+0000","last_clean_scrub_stamp":"2023-12-26T21:55:27.908991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:34:50.695410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"123'1","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.942106+0000","last_change":"2023-12-26T21:55:27.939439+0000","last_active":"2023-12-26T21:55:28.942106+0000","last_peered":"2023-12-26T21:55:28.942106+0000","last_clean":"2023-12-26T21:55:28.942106+0000","last_became_active":"2023-12-26T21:55:27.939199+0000","last_became_peered":"2023-12-26T21:55:27.939199+0000","last_unstale":"2023-12-26T21:55:28.942106+0000","last_undegraded":"2023-12-26T21:55:28.942106+0000","last_fullsized":"2023-12-26T21:55:28.942106+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:55:26.902820+0000","last_clean_scrub_stamp":"2023-12-26T21:55:26.902820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:57:34.757217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"113'227","reported_seq":645,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-26T21:55:28.927419+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:55:28.927419+0000","last_peered":"2023-12-26T21:55:28.927419+0000","last_clean":"2023-12-26T21:55:28.927419+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:55:28.927419+0000","last_undegraded":"2023-12-26T21:55:28.927419+0000","last_fullsized":"2023-12-26T21:55:28.927419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542367,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302876,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068964,"statfs":{"total":96636764160,"available":96326619136,"internally_reserved":0,"allocated":1527808,"data_stored":745630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.51200000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90700000000000003,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"max":{"1min":0.82199999999999995,"5min":0.94099999999999995,"15min":0.98299999999999998},"last":0.89100000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.67400000000000004},"min":{"1min":0.53000000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.79500000000000004,"5min":0.85899999999999999,"15min":1},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.70599999999999996},"min":{"1min":0.54000000000000004,"5min":0.436,"15min":0.34200000000000003},"max":{"1min":0.96899999999999997,"5min":1.014,"15min":1.048},"last":1.2190000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":1.0509999999999999},"last":1.173},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66900000000000004,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.434},"max":{"1min":0.88900000000000001,"5min":0.99199999999999999,"15min":1.02},"last":1.143}]},{"osd":3,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94399999999999995,"5min":0.94499999999999995,"15min":1.052},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.59699999999999998,"5min":0.48299999999999998,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79500000000000004}]},{"osd":4,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.753,"5min":1.036,"15min":1.036},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84299999999999997,"5min":0.95199999999999996,"15min":0.97599999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.51200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86899999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.329}]},{"osd":6,"last update":"Tue Dec 26 21:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.95599999999999996,"5min":1.117,"15min":1.117},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.44},"max":{"1min":0.998,"5min":1.006,"15min":1.006},"last":1.2509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509663,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306968,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064872,"statfs":{"total":96636764160,"available":96322428928,"internally_reserved":0,"allocated":1523712,"data_stored":745076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.55500000000000005,"5min":0.48999999999999999,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.82899999999999996}]},{"osd":1,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.60699999999999998,"5min":0.46100000000000002,"15min":0.40200000000000002},"max":{"1min":0.85399999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.73399999999999999}]},{"osd":2,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.70699999999999996},"min":{"1min":0.51600000000000001,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.89200000000000002,"5min":0.93200000000000005,"15min":0.97099999999999997},"last":0.62},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69199999999999995,"15min":0.70699999999999996},"min":{"1min":0.54600000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.94699999999999995},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.65900000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90900000000000003,"5min":1.026,"15min":1.026},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.751},"min":{"1min":0.64000000000000001,"5min":0.46200000000000002,"15min":0.42799999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.0129999999999999},"last":0.69299999999999995}]},{"osd":4,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.42399999999999999,"15min":0.372},"max":{"1min":0.82299999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.48299999999999998,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.82099999999999995,"5min":0.91200000000000003,"15min":1.0529999999999999},"last":0.751}]},{"osd":5,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.97299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.874,"5min":1.0049999999999999,"15min":1.2609999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Tue Dec 26 21:55:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.54200000000000004,"5min":0.47099999999999997,"15min":0.438},"max":{"1min":0.98599999999999999,"5min":1.008,"15min":1.0920000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.40899999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0920000000000001},"last":0.77700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509663,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311664,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060176,"statfs":{"total":96636764160,"available":96317620224,"internally_reserved":0,"allocated":2138112,"data_stored":1337092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.30499999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.628,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.42999999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.72799999999999998,"5min":1.353,"15min":1.353},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.48099999999999998},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68300000000000005,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":1.488,"15min":1.488},"last":0.41499999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.42399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79300000000000004,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":1.506,"15min":1.506},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71199999999999997,"15min":0.746},"min":{"1min":0.58199999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0509999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76500000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.77300000000000002},"min":{"1min":0.58199999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.032,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76200000000000001,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.0189999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.59399999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.627},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.54400000000000004},"max":{"1min":1.0529999999999999,"5min":1.615,"15min":1.615},"last":0.70999999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:55:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.77900000000000003},"min":{"1min":0.57899999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.075,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.752},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.04,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.68200000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509663,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302872,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068968,"statfs":{"total":96636764160,"available":96326623232,"internally_reserved":0,"allocated":1523712,"data_stored":745076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.245},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.28299999999999997},"max":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.45400000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.48999999999999999},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.30599999999999999},"max":{"1min":0.621,"5min":0.66800000000000004,"15min":0.73099999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.52200000000000002},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.317,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84699999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.44,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86699999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.72099999999999997,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.2749999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47699999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.58399999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77600000000000002,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.153},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.63}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.745,"15min":0.77900000000000003},"min":{"1min":0.56899999999999995,"5min":0.51500000000000001,"15min":0.45300000000000001},"max":{"1min":1.0700000000000001,"5min":1.093,"15min":1.214},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.749,"15min":0.79000000000000004},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.79800000000000004},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.51700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.65300000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542368,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306972,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064868,"statfs":{"total":96636764160,"available":96322424832,"internally_reserved":0,"allocated":1527808,"data_stored":745146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45500000000000002,"15min":0.45600000000000002},"min":{"1min":0.34399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61099999999999999,"5min":0.70399999999999996,"15min":0.85999999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.433}]},{"osd":1,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.48399999999999999},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.373},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.41599999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.60499999999999998},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78300000000000003,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78200000000000003,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.47199999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.58099999999999996,"5min":0.51300000000000001,"15min":0.38300000000000001},"max":{"1min":1.155,"5min":1.3,"15min":1.3},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.49199999999999999,"15min":0.48899999999999999},"max":{"1min":1.014,"5min":1.048,"15min":1.1759999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76700000000000002},"min":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.502},"max":{"1min":1.1000000000000001,"5min":1.25,"15min":1.25},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75900000000000001,"15min":0.77200000000000002},"min":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":1.002,"5min":1.014,"15min":1.1319999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.53900000000000003,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.0249999999999999,"5min":1.202,"15min":1.272},"last":0.751}]},{"osd":7,"last update":"Tue Dec 26 21:55:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.69499999999999995,"5min":0.53100000000000003,"15min":0.47799999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1220000000000001},"last":0.625},{"interface":"front","average":{"1min":0.871,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.54300000000000004},"max":{"1min":1.1659999999999999,"5min":1.232,"15min":1.232},"last":0.73199999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509664,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311076,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060764,"statfs":{"total":96636764160,"available":96318222336,"internally_reserved":0,"allocated":1536000,"data_stored":745270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.443},"min":{"1min":0.311,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66200000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.48599999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.63400000000000001,"5min":1.238,"15min":1.238},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":1.468,"15min":1.468},"last":0.432},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":0.65700000000000003,"5min":1.756,"15min":1.756},"last":0.40400000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.52700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.83299999999999996,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.51000000000000001,"5min":0.42299999999999999,"15min":0.42199999999999999},"max":{"1min":0.80800000000000005,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.63800000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.0029999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.61199999999999999,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.61899999999999999,"5min":0.52300000000000002,"15min":0.497},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1399999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.79900000000000004},"min":{"1min":0.65900000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.056,"5min":1.2070000000000001,"15min":1.2430000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.70699999999999996,"5min":0.55700000000000005,"15min":0.54100000000000004},"max":{"1min":1.071,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.68400000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.1659999999999999},"last":0.623},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.101,"5min":1.272,"15min":1.272},"last":0.56299999999999994}]}]},{"osd":4,"up_from":14,"seq":60129542367,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306972,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064868,"statfs":{"total":96636764160,"available":96322424832,"internally_reserved":0,"allocated":1527808,"data_stored":745630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67300000000000004,"15min":0.68400000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.70599999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.371},"max":{"1min":0.91700000000000004,"5min":1.006,"15min":1.008},"last":0.70699999999999996}]},{"osd":1,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.311},"max":{"1min":1.0229999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40000000000000002},"max":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97199999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.55200000000000005,"5min":0.46000000000000002,"15min":0.38800000000000001},"max":{"1min":0.875,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.50800000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.45300000000000001},"max":{"1min":0.90400000000000003,"5min":0.93700000000000006,"15min":0.97599999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.442,"15min":0.438},"max":{"1min":0.98799999999999999,"5min":1.046,"15min":1.046},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84499999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83499999999999996,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.59899999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.441,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.88600000000000001,"5min":1.133,"15min":1.133},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Dec 26 21:55:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":0.88600000000000001,"5min":1.002,"15min":1.002},"last":0.746},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.58399999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89800000000000002,"5min":1.266,"15min":1.266},"last":0.72399999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509662,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303472,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068368,"statfs":{"total":96636764160,"available":96326008832,"internally_reserved":0,"allocated":2138112,"data_stored":1337036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.68400000000000005},"min":{"1min":0.48199999999999998,"5min":0.39800000000000002,"15min":0.27300000000000002},"max":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.995},"last":0.751},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.46800000000000003,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.91600000000000004,"15min":1.0369999999999999},"last":0.89900000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.69699999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.34000000000000002},"max":{"1min":0.88400000000000001,"5min":0.96399999999999997,"15min":1.04},"last":0.88},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66800000000000004,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.443,"15min":0.36899999999999999},"max":{"1min":0.86899999999999999,"5min":1.002,"15min":1.01},"last":0.67100000000000004}]},{"osd":2,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67800000000000005,"15min":0.71099999999999997},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.88500000000000001,"15min":1.3029999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67400000000000004,"15min":0.69899999999999995},"min":{"1min":0.52400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.92400000000000004,"15min":0.95699999999999996},"last":0.85099999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.71999999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.39200000000000002},"max":{"1min":0.90500000000000003,"5min":0.93000000000000005,"15min":1.0640000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.47799999999999998},"max":{"1min":0.85499999999999998,"5min":0.92200000000000004,"15min":0.93899999999999995},"last":0.72699999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":0.79300000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.41899999999999998,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.01},"last":0.60499999999999998}]},{"osd":6,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.65000000000000002},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35499999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46899999999999997},"max":{"1min":0.85499999999999998,"5min":1.111,"15min":1.111},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67100000000000004,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":0.95099999999999996,"15min":0.96299999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68700000000000006,"15min":0.69799999999999995},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":0.95399999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:55:38.629 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T21:55:38.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T21:55:38.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.861+0000 7fbc239ff700 1 Processor -- start 2023-12-26T21:55:38.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.861+0000 7fbc239ff700 1 -- start start 2023-12-26T21:55:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc239ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbc1c069620 0x7fbc1c0699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc239ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c06e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc239ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1c157b30 con 0x7fbc1c0ad9f0 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc239ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c06f450 con 0x7fbc1c069620 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc239ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c06f8e0 con 0x7fbc1c06dd90 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc21f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbc1c0ad9f0 0x7fbc1c0ade30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44808/0 (socket says 172.21.15.131:44808) 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc21f9c700 1 -- 172.21.15.131:0/3110612063 learned_addr learned my addr 172.21.15.131:0/3110612063 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc2179b700 1 --2- 172.21.15.131:0/3110612063 >> v2:172.21.15.190:3300/0 conn(0x7fbc1c069620 0x7fbc1c0699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc20f9a700 1 --2- 172.21.15.131:0/3110612063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c06e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.863+0000 7fbc13fff700 1 -- 172.21.15.131:0/3110612063 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503826427 0 0) 0x7fbc1c157b30 con 0x7fbc1c0ad9f0 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc13fff700 1 -- 172.21.15.131:0/3110612063 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc08003570 con 0x7fbc1c0ad9f0 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc13fff700 1 -- 172.21.15.131:0/3110612063 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1296417420 0 0) 0x7fbc08003570 con 0x7fbc1c0ad9f0 2023-12-26T21:55:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc20f9a700 1 -- 172.21.15.131:0/3110612063 >> v2:172.21.15.190:3300/0 conn(0x7fbc1c069620 msgr2=0x7fbc1c0699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc20f9a700 1 --2- 172.21.15.131:0/3110612063 >> v2:172.21.15.190:3300/0 conn(0x7fbc1c069620 0x7fbc1c0699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc20f9a700 1 -- 172.21.15.131:0/3110612063 >> v1:172.21.15.131:6789/0 conn(0x7fbc1c0ad9f0 legacy=0x7fbc1c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc20f9a700 1 -- 172.21.15.131:0/3110612063 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c069f30 con 0x7fbc1c06dd90 2023-12-26T21:55:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc2179b700 1 --2- 172.21.15.131:0/3110612063 >> v2:172.21.15.190:3300/0 conn(0x7fbc1c069620 0x7fbc1c0699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc20f9a700 1 --2- 172.21.15.131:0/3110612063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c06e200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0098e0 tx=0x7fbc0c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e3bcc2b16e1c97 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc13fff700 1 -- 172.21.15.131:0/3110612063 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc140017f0 con 0x7fbc1c06dd90 2023-12-26T21:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc13fff700 1 -- 172.21.15.131:0/3110612063 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c011420 con 0x7fbc1c06dd90 2023-12-26T21:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.864+0000 7fbc13fff700 1 -- 172.21.15.131:0/3110612063 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0117c0 con 0x7fbc1c06dd90 2023-12-26T21:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.865+0000 7fbc239ff700 1 -- 172.21.15.131:0/3110612063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 msgr2=0x7fbc1c06e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.865+0000 7fbc239ff700 1 --2- 172.21.15.131:0/3110612063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c06e200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0098e0 tx=0x7fbc0c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.865+0000 7fbc239ff700 1 -- 172.21.15.131:0/3110612063 shutdown_connections 2023-12-26T21:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.865+0000 7fbc239ff700 1 --2- 172.21.15.131:0/3110612063 >> v2:172.21.15.190:3300/0 conn(0x7fbc1c069620 0x7fbc1c0699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.865+0000 7fbc239ff700 1 --2- 172.21.15.131:0/3110612063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c06e200 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.865+0000 7fbc239ff700 1 -- 172.21.15.131:0/3110612063 >> 172.21.15.131:0/3110612063 conn(0x7fbc1c068040 msgr2=0x7fbc1c0cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.865+0000 7fbc239ff700 1 -- 172.21.15.131:0/3110612063 shutdown_connections 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.865+0000 7fbc239ff700 1 -- 172.21.15.131:0/3110612063 wait complete. 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.866+0000 7fbc239ff700 1 Processor -- start 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.867+0000 7fbc239ff700 1 -- start start 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.867+0000 7fbc239ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c069620 0x7fbc1c14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.867+0000 7fbc239ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.867+0000 7fbc239ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc1c0ad9f0 0x7fbc1c15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc239ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0c98b0 con 0x7fbc1c069620 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc239ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0c0be0 con 0x7fbc1c0ad9f0 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc239ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0c1760 con 0x7fbc1c06dd90 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc2179b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c069620 0x7fbc1c14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc2179b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c069620 0x7fbc1c14b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49458/0 (socket says 172.21.15.131:49458) 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc2179b700 1 -- 172.21.15.131:0/1664758220 learned_addr learned my addr 172.21.15.131:0/1664758220 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc20f9a700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc2179b700 1 -- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 msgr2=0x7fbc1c15bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc21f9c700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc1c0ad9f0 0x7fbc1c15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc2179b700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c15bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc2179b700 1 -- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc1c0ad9f0 msgr2=0x7fbc1c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc2179b700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc1c0ad9f0 0x7fbc1c15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc2179b700 1 -- 172.21.15.131:0/1664758220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc18009420 con 0x7fbc1c069620 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc20f9a700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc21f9c700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc1c0ad9f0 0x7fbc1c15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.868+0000 7fbc2179b700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c069620 0x7fbc1c14b8e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800ecb0 tx=0x7fbc180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.869+0000 7fbc117fa700 1 -- 172.21.15.131:0/1664758220 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1800ee60 con 0x7fbc1c069620 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.869+0000 7fbc117fa700 1 -- 172.21.15.131:0/1664758220 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1800cbe0 con 0x7fbc1c069620 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.869+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0c009420 con 0x7fbc1c069620 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.869+0000 7fbc117fa700 1 -- 172.21.15.131:0/1664758220 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18013470 con 0x7fbc1c069620 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.869+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c0c1110 con 0x7fbc1c069620 2023-12-26T21:55:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.870+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c0657e0 con 0x7fbc1c069620 2023-12-26T21:55:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.871+0000 7fbc117fa700 1 -- 172.21.15.131:0/1664758220 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbc18013820 con 0x7fbc1c069620 2023-12-26T21:55:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.872+0000 7fbc117fa700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc00066f30 0x7fbc000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.872+0000 7fbc117fa700 1 -- 172.21.15.131:0/1664758220 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fbc18084aa0 con 0x7fbc1c069620 2023-12-26T21:55:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.872+0000 7fbc20f9a700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc00066f30 0x7fbc000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.873+0000 7fbc20f9a700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc00066f30 0x7fbc000693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c15cb10 tx=0x7fbc0c016000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:38.875+0000 7fbc117fa700 1 -- 172.21.15.131:0/1664758220 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc18051b80 con 0x7fbc1c069620 2023-12-26T21:55:39.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.142+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbc1c06e840 con 0x7fbc1c069620 2023-12-26T21:55:39.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.146+0000 7fbc117fa700 1 -- 172.21.15.131:0/1664758220 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7fbc180551a0 con 0x7fbc1c069620 2023-12-26T21:55:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc00066f30 msgr2=0x7fbc000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc00066f30 0x7fbc000693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c15cb10 tx=0x7fbc0c016000 comp rx=0 tx=0).stop 2023-12-26T21:55:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c069620 msgr2=0x7fbc1c14b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c069620 0x7fbc1c14b8e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800ecb0 tx=0x7fbc180091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 shutdown_connections 2023-12-26T21:55:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc1c0ad9f0 0x7fbc1c15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc00066f30 0x7fbc000693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc1c069620 0x7fbc1c14b8e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 --2- 172.21.15.131:0/1664758220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc1c06dd90 0x7fbc1c15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 >> 172.21.15.131:0/1664758220 conn(0x7fbc1c068040 msgr2=0x7fbc1c0cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:39.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 shutdown_connections 2023-12-26T21:55:39.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.148+0000 7fbc239ff700 1 -- 172.21.15.131:0/1664758220 wait complete. 2023-12-26T21:55:39.150 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T21:55:39.167 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:55:39.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.358+0000 7f3f1d9ff700 1 Processor -- start 2023-12-26T21:55:39.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.358+0000 7f3f1d9ff700 1 -- start start 2023-12-26T21:55:39.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.359+0000 7f3f1d9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3f1806bfd0 0x7f3f180c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:39.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.359+0000 7f3f1d9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f180686c0 con 0x7f3f1813c510 2023-12-26T21:55:39.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.359+0000 7f3f1d9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f18069520 con 0x7f3f180cc100 2023-12-26T21:55:39.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.359+0000 7f3f1d9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f18068910 con 0x7f3f1806bfd0 2023-12-26T21:55:39.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f16ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3f1813c510 0x7f3f1806b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44818/0 (socket says 172.21.15.131:44818) 2023-12-26T21:55:39.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f16ffd700 1 -- 172.21.15.131:0/818393196 learned_addr learned my addr 172.21.15.131:0/818393196 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f167fc700 1 --2- 172.21.15.131:0/818393196 >> v2:172.21.15.190:3300/0 conn(0x7f3f1806bfd0 0x7f3f180c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668626762 0 0) 0x7f3f180686c0 con 0x7f3f1813c510 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f00003570 con 0x7f3f1813c510 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2743109598 0 0) 0x7f3f00003570 con 0x7f3f1813c510 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 >> v2:172.21.15.190:3300/0 conn(0x7f3f1806bfd0 msgr2=0x7f3f180c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 --2- 172.21.15.131:0/818393196 >> v2:172.21.15.190:3300/0 conn(0x7f3f1806bfd0 0x7f3f180c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 >> v1:172.21.15.190:6789/0 conn(0x7f3f180cc100 legacy=0x7f3f180cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f18068bf0 con 0x7f3f1813c510 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f3f080015f0 con 0x7f3f1813c510 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 >> v1:172.21.15.131:6789/0 conn(0x7f3f1813c510 legacy=0x7f3f1806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f00003a60 0x7f3f00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f00004810 0x7f3f00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f000051c0 0x7f3f0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18068910 con 0x7f3f000051c0 2023-12-26T21:55:39.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0000c0d0 con 0x7f3f00004810 2023-12-26T21:55:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.360+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0000c290 con 0x7f3f00003a60 2023-12-26T21:55:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f177fe700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f000051c0 0x7f3f0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f167fc700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f00004810 0x7f3f00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f16ffd700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f00003a60 0x7f3f00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:39.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f177fe700 1 -- 172.21.15.131:0/818393196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f00003a60 msgr2=0x7f3f00003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f177fe700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f00003a60 0x7f3f00003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f177fe700 1 -- 172.21.15.131:0/818393196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f00004810 msgr2=0x7f3f00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f177fe700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f00004810 0x7f3f00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f177fe700 1 -- 172.21.15.131:0/818393196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00003570 con 0x7f3f000051c0 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f177fe700 1 -- 172.21.15.131:0/818393196 reap_dead start 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.361+0000 7f3f177fe700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f000051c0 0x7f3f0000ba60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3f10001fc0 tx=0x7f3f10001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b8498c20e88e68 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10001950 con 0x7f3f000051c0 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1000c8b0 con 0x7f3f000051c0 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f15ffb700 1 -- 172.21.15.131:0/818393196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1000cc90 con 0x7f3f000051c0 2023-12-26T21:55:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/818393196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f000051c0 msgr2=0x7f3f0000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f1d9ff700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f000051c0 0x7f3f0000ba60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3f10001fc0 tx=0x7f3f10001640 comp rx=0 tx=0).stop 2023-12-26T21:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/818393196 shutdown_connections 2023-12-26T21:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f1d9ff700 1 --2- 172.21.15.131:0/818393196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f000051c0 0x7f3f0000ba60 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/818393196 >> 172.21.15.131:0/818393196 conn(0x7f3f180ce3e0 msgr2=0x7f3f180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/818393196 shutdown_connections 2023-12-26T21:55:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.362+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/818393196 wait complete. 2023-12-26T21:55:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.364+0000 7f3f1d9ff700 1 Processor -- start 2023-12-26T21:55:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.364+0000 7f3f1d9ff700 1 -- start start 2023-12-26T21:55:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f1d9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000051c0 0x7f3f181536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f1d9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f1815bfb0 0x7f3f1815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f1d9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1815ad60 0x7f3f1815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f1d9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1815c960 con 0x7f3f1815bfb0 2023-12-26T21:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f1d9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1815b740 con 0x7f3f000051c0 2023-12-26T21:55:39.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f1d9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1813e330 con 0x7f3f1815ad60 2023-12-26T21:55:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f167fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f1815bfb0 0x7f3f1815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f167fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f1815bfb0 0x7f3f1815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49484/0 (socket says 172.21.15.131:49484) 2023-12-26T21:55:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f167fc700 1 -- 172.21.15.131:0/2698454352 learned_addr learned my addr 172.21.15.131:0/2698454352 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f16ffd700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000051c0 0x7f3f181536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f177fe700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1815ad60 0x7f3f1815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f167fc700 1 -- 172.21.15.131:0/2698454352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1815ad60 msgr2=0x7f3f1815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f167fc700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1815ad60 0x7f3f1815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f167fc700 1 -- 172.21.15.131:0/2698454352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000051c0 msgr2=0x7f3f181536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f167fc700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000051c0 0x7f3f181536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f167fc700 1 -- 172.21.15.131:0/2698454352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1813d4a0 con 0x7f3f1815bfb0 2023-12-26T21:55:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.365+0000 7f3f16ffd700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000051c0 0x7f3f181536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.366+0000 7f3f167fc700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f1815bfb0 0x7f3f1815c420 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009fd0 tx=0x7f3f0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.366+0000 7f3efeffd700 1 -- 172.21.15.131:0/2698454352 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00f940 con 0x7f3f1815bfb0 2023-12-26T21:55:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.366+0000 7f3efeffd700 1 -- 172.21.15.131:0/2698454352 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c00faa0 con 0x7f3f1815bfb0 2023-12-26T21:55:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.366+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1813d720 con 0x7f3f1815bfb0 2023-12-26T21:55:39.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.366+0000 7f3efeffd700 1 -- 172.21.15.131:0/2698454352 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00fe20 con 0x7f3f1815bfb0 2023-12-26T21:55:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.366+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1813dce0 con 0x7f3f1815bfb0 2023-12-26T21:55:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.367+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f180657e0 con 0x7f3f1815bfb0 2023-12-26T21:55:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.368+0000 7f3efeffd700 1 -- 172.21.15.131:0/2698454352 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3f0c014650 con 0x7f3f1815bfb0 2023-12-26T21:55:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.369+0000 7f3efeffd700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3f04066f30 0x7f3f040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.369+0000 7f3efeffd700 1 -- 172.21.15.131:0/2698454352 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f3f0c018030 con 0x7f3f1815bfb0 2023-12-26T21:55:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.369+0000 7f3f16ffd700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3f04066f30 0x7f3f040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:39.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.370+0000 7f3f16ffd700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3f04066f30 0x7f3f040693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f08001720 tx=0x7f3f08009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.373+0000 7f3efeffd700 1 -- 172.21.15.131:0/2698454352 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f0c0521a0 con 0x7f3f1815bfb0 2023-12-26T21:55:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.611+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f1813e470 con 0x7f3f1815bfb0 2023-12-26T21:55:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.612+0000 7f3efeffd700 1 -- 172.21.15.131:0/2698454352 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f3f0c022090 con 0x7f3f1815bfb0 2023-12-26T21:55:39.612 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:39.613 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6623,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3234851896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3234851896},{"type":"v1","addr":"172.21.15.131:6834","nonce":3234851896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2363679591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2363679591},{"type":"v1","addr":"172.21.15.190:6834","nonce":2363679591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3018492544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3018492544},{"type":"v1","addr":"172.21.15.190:6835","nonce":3018492544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:55:28.396211+0000","modified":"2023-12-26T21:55:31.950235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1932798418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1932798418},{"type":"v1","addr":"172.21.15.131:6835","nonce":1932798418}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3f04066f30 msgr2=0x7f3f040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3f04066f30 0x7f3f040693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f08001720 tx=0x7f3f08009040 comp rx=0 tx=0).stop 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f1815bfb0 msgr2=0x7f3f1815c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f1815bfb0 0x7f3f1815c420 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009fd0 tx=0x7f3f0c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 shutdown_connections 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1815ad60 0x7f3f1815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000051c0 0x7f3f181536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3f04066f30 0x7f3f040693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 --2- 172.21.15.131:0/2698454352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f1815bfb0 0x7f3f1815c420 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 >> 172.21.15.131:0/2698454352 conn(0x7f3f180ce3e0 msgr2=0x7f3f1806d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:39.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 shutdown_connections 2023-12-26T21:55:39.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:39.614+0000 7f3f1d9ff700 1 -- 172.21.15.131:0/2698454352 wait complete. 2023-12-26T21:55:39.616 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 57 2023-12-26T21:55:40.035 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T21:55:28.396211+0000', 'modified': '2023-12-26T21:55:31.950235+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7644}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7644': {'gid': 7644, 'name': 'c', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6835/1932798418', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 1932798418}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1932798418}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T21:55:40.035 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T21:55:40.035 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-26T21:55:40.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.228+0000 7fe1a7dff700 1 Processor -- start 2023-12-26T21:55:40.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.228+0000 7fe1a7dff700 1 -- start start 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.228+0000 7fe1a7dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00c8820 0x7fe1a00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.228+0000 7fe1a7dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe1a00b19f0 0x7fe1a0161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.228+0000 7fe1a7dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a0157a80 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.228+0000 7fe1a7dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a00cac50 con 0x7fe1a00b19f0 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.228+0000 7fe1a7dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00caf30 con 0x7fe1a00c8820 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.229+0000 7fe1a639c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe1a00b19f0 0x7fe1a0161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.229+0000 7fe1a639c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe1a00b19f0 0x7fe1a0161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49488/0 (socket says 172.21.15.131:49488) 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.229+0000 7fe1a639c700 1 -- 172.21.15.131:0/1917864786 learned_addr learned my addr 172.21.15.131:0/1917864786 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.229+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/1917864786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00c8820 0x7fe1a00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.229+0000 7fe1a639c700 1 -- 172.21.15.131:0/1917864786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00c8820 msgr2=0x7fe1a00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.229+0000 7fe1a639c700 1 --2- 172.21.15.131:0/1917864786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00c8820 0x7fe1a00ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.229+0000 7fe1a639c700 1 -- 172.21.15.131:0/1917864786 >> v1:172.21.15.190:6789/0 conn(0x7fe1a00c8bf0 legacy=0x7fe1a00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.229+0000 7fe1a639c700 1 -- 172.21.15.131:0/1917864786 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00cb210 con 0x7fe1a00b19f0 2023-12-26T21:55:40.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.229+0000 7fe1a639c700 1 --2- 172.21.15.131:0/1917864786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00b19f0 0x7fe1a0161090 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe198009f20 tx=0x7fe198009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9be0ee29297fda server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a4b99700 1 -- 172.21.15.131:0/1917864786 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800e070 con 0x7fe1a00b19f0 2023-12-26T21:55:40.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a4b99700 1 -- 172.21.15.131:0/1917864786 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19800b9d0 con 0x7fe1a00b19f0 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a4b99700 1 -- 172.21.15.131:0/1917864786 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1980092e0 con 0x7fe1a00b19f0 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a7dff700 1 -- 172.21.15.131:0/1917864786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00b19f0 msgr2=0x7fe1a0161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a7dff700 1 --2- 172.21.15.131:0/1917864786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00b19f0 0x7fe1a0161090 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe198009f20 tx=0x7fe198009840 comp rx=0 tx=0).stop 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a7dff700 1 -- 172.21.15.131:0/1917864786 shutdown_connections 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a7dff700 1 --2- 172.21.15.131:0/1917864786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00b19f0 0x7fe1a0161090 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a7dff700 1 --2- 172.21.15.131:0/1917864786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00c8820 0x7fe1a00ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a7dff700 1 -- 172.21.15.131:0/1917864786 >> 172.21.15.131:0/1917864786 conn(0x7fe1a0068040 msgr2=0x7fe1a0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a7dff700 1 -- 172.21.15.131:0/1917864786 shutdown_connections 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.230+0000 7fe1a7dff700 1 -- 172.21.15.131:0/1917864786 wait complete. 2023-12-26T21:55:40.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.232+0000 7fe1a7dff700 1 Processor -- start 2023-12-26T21:55:40.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.233+0000 7fe1a7dff700 1 -- start start 2023-12-26T21:55:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a7dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00b19f0 0x7fe1a0147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a7dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a00c8820 0x7fe1a00c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a7dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00c8bf0 0x7fe1a0167540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a7dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00c4d20 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a7dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0148210 con 0x7fe1a00c8820 2023-12-26T21:55:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a7dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00c4110 con 0x7fe1a00b19f0 2023-12-26T21:55:40.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00c8bf0 0x7fe1a0167540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00c8bf0 0x7fe1a0167540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49498/0 (socket says 172.21.15.131:49498) 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a639c700 1 -- 172.21.15.131:0/2187011349 learned_addr learned my addr 172.21.15.131:0/2187011349 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00b19f0 0x7fe1a0147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a539a700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a00c8820 0x7fe1a00c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a639c700 1 -- 172.21.15.131:0/2187011349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00b19f0 msgr2=0x7fe1a0147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a639c700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00b19f0 0x7fe1a0147cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a639c700 1 -- 172.21.15.131:0/2187011349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a00c8820 msgr2=0x7fe1a00c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a639c700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a00c8820 0x7fe1a00c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a639c700 1 -- 172.21.15.131:0/2187011349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe198009ae0 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a639c700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00c8bf0 0x7fe1a0167540 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00688f0 tx=0x7fe198018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe195ffb700 1 -- 172.21.15.131:0/2187011349 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198008ad0 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe195ffb700 1 -- 172.21.15.131:0/2187011349 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe198008c30 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a7dff700 1 -- 172.21.15.131:0/2187011349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a00c43f0 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe195ffb700 1 -- 172.21.15.131:0/2187011349 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198025460 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.235+0000 7fe1a7dff700 1 -- 172.21.15.131:0/2187011349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a00c49b0 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.237+0000 7fe195ffb700 1 -- 172.21.15.131:0/2187011349 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe198003040 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.238+0000 7fe195ffb700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe184066f30 0x7fe1840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.238+0000 7fe195ffb700 1 -- 172.21.15.131:0/2187011349 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe198085cb0 con 0x7fe1a00c8bf0 2023-12-26T21:55:40.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.238+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe184066f30 0x7fe1840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.239+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe184066f30 0x7fe1840693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009db0 tx=0x7fe19c0091a0 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.240+0000 7fe1b6421b80 1 Processor -- start 2023-12-26T21:55:40.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.240+0000 7fe1b6421b80 1 -- start start 2023-12-26T21:55:40.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a776178530 0x55a776178900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a77615fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e940 0x55a776160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77608a3e0 con 0x55a77617e4d0 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a776087be0 con 0x55a776178530 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7760839d0 con 0x55a77617e940 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a539a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a77615fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a539a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a77615fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49502/0 (socket says 172.21.15.131:49502) 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a539a700 1 -- 172.21.15.131:0/281338971 learned_addr learned my addr 172.21.15.131:0/281338971 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/281338971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a776178530 0x55a776178900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a639c700 1 --2- 172.21.15.131:0/281338971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e940 0x55a776160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a539a700 1 -- 172.21.15.131:0/281338971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e940 msgr2=0x55a776160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a539a700 1 --2- 172.21.15.131:0/281338971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e940 0x55a776160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a539a700 1 -- 172.21.15.131:0/281338971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a776178530 msgr2=0x55a776178900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a539a700 1 --2- 172.21.15.131:0/281338971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a776178530 0x55a776178900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.241+0000 7fe1a539a700 1 -- 172.21.15.131:0/281338971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a775f17360 con 0x55a77617e4d0 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.242+0000 7fe1a539a700 1 --2- 172.21.15.131:0/281338971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a77615fdc0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe190003860 tx=0x7fe190007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281cba8ecbc58522 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.242+0000 7fe18affd700 1 -- 172.21.15.131:0/281338971 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190011590 con 0x55a77617e4d0 2023-12-26T21:55:40.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.242+0000 7fe18affd700 1 -- 172.21.15.131:0/281338971 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1900116f0 con 0x55a77617e4d0 2023-12-26T21:55:40.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.242+0000 7fe18affd700 1 -- 172.21.15.131:0/281338971 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190011a90 con 0x55a77617e4d0 2023-12-26T21:55:40.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 -- 172.21.15.131:0/281338971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 msgr2=0x55a77615fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/281338971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a77615fdc0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe190003860 tx=0x7fe190007ec0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 -- 172.21.15.131:0/281338971 shutdown_connections 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/281338971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e940 0x55a776160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/281338971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a776178530 0x55a776178900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/281338971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a77615fdc0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 -- 172.21.15.131:0/281338971 >> 172.21.15.131:0/281338971 conn(0x55a776132440 msgr2=0x55a77617ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 -- 172.21.15.131:0/281338971 shutdown_connections 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 -- 172.21.15.131:0/281338971 wait complete. 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.243+0000 7fe1b6421b80 1 -- start start 2023-12-26T21:55:40.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.244+0000 7fe1b6421b80 1 Processor -- start 2023-12-26T21:55:40.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a776178530 0x55a7761721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e4d0 0x55a77615be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 0x55a77615c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7760301b0 con 0x55a776178530 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77602fe50 con 0x55a77617e940 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a775f7ce40 con 0x55a77617e4d0 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1b6421b80 10 client.0 _ll_register_callbacks cb 0x55a775f7f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a5b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a776178530 0x55a7761721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a5b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a776178530 0x55a7761721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49518/0 (socket says 172.21.15.131:49518) 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a5b9b700 1 -- 172.21.15.131:0/633168761 learned_addr learned my addr 172.21.15.131:0/633168761 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a539a700 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e4d0 0x55a77615be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a639c700 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 0x55a77615c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a5b9b700 1 -- 172.21.15.131:0/633168761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e4d0 msgr2=0x55a77615be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e4d0 0x55a77615be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.249 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a5b9b700 1 -- 172.21.15.131:0/633168761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 msgr2=0x55a77615c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 0x55a77615c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.245+0000 7fe1a5b9b700 1 -- 172.21.15.131:0/633168761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a0157a80 con 0x55a776178530 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.246+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a776178530 0x55a7761721d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009c60 tx=0x7fe19c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ad78cc32f8ab40 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.246+0000 7fe17ffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.246+0000 7fe17ffff700 1 -- 172.21.15.131:0/633168761 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c013830 con 0x55a776178530 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.246+0000 7fe17ffff700 1 -- 172.21.15.131:0/633168761 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe19c01a040 con 0x55a776178530 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.246+0000 7fe1b6421b80 1 -- 172.21.15.131:0/633168761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a776104eb0 con 0x55a776178530 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.246+0000 7fe17ffff700 1 -- 172.21.15.131:0/633168761 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe19c013ba0 con 0x55a776178530 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.246+0000 7fe1b6421b80 10 client.7815 fetch_fsmap learned FSMap version 57 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.246+0000 7fe1b6421b80 1 -- 172.21.15.131:0/633168761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55a775ec3b10 con 0x55a776178530 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.246+0000 7fe17ffff700 1 -- 172.21.15.131:0/633168761 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a776104eb0 con 0x55a776178530 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe17ffff700 1 -- 172.21.15.131:0/633168761 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 57) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe19c013d00 con 0x55a776178530 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe1b6421b80 10 client.7815 fetch_fsmap finished waiting for FSMap version 57 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe1b6421b80 10 client.7815 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:55:40.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe1b6421b80 20 client.7815 populate_metadata read hostname 'smithi131' 2023-12-26T21:55:40.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] conn(0x55a776184f80 0x55a7761737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe1b6421b80 4 client.7815 mds_command: new command op to 7644 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T21:55:40.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe1b6421b80 1 -- 172.21.15.131:0/633168761 --> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a77605a880 con 0x55a776184f80 2023-12-26T21:55:40.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe1a539a700 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] conn(0x55a776184f80 0x55a7761737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe17ffff700 10 client.7815 ms_handle_connect on v2:172.21.15.131:6833/1932798418 2023-12-26T21:55:40.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.247+0000 7fe1a539a700 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] conn(0x55a776184f80 0x55a7761737e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c9a8f60bc41fc3 server_cookie=e8666f42188eaf99 in_seq=0 out_seq=0 2023-12-26T21:55:40.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe17ffff700 1 -- 172.21.15.131:0/633168761 <== mds.0 v2:172.21.15.131:6833/1932798418 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a77605a880 con 0x55a776184f80 2023-12-26T21:55:40.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe17ffff700 10 client.7815 handle_command_reply: tid=0 2023-12-26T21:55:40.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe1b6421b80 1 client.7815 shutdown 2023-12-26T21:55:40.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe1b6421b80 1 -- 172.21.15.131:0/633168761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a776178530 msgr2=0x55a7761721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a776178530 0x55a7761721d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009c60 tx=0x7fe19c019000 comp rx=0 tx=0).stop 2023-12-26T21:55:40.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe1b6421b80 1 -- 172.21.15.131:0/633168761 shutdown_connections 2023-12-26T21:55:40.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] conn(0x55a776184f80 0x55a7761737e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 0x55a77615c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a776178530 0x55a7761721d0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe17ffff700 0 client.7815 ms_handle_reset on v2:172.21.15.131:6833/1932798418 2023-12-26T21:55:40.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/633168761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a77617e4d0 0x55a77615be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.248+0000 7fe1b6421b80 1 -- 172.21.15.131:0/633168761 >> 172.21.15.131:0/633168761 conn(0x55a776132440 msgr2=0x55a7761328c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:40.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.249+0000 7fe1b6421b80 1 -- 172.21.15.131:0/633168761 shutdown_connections 2023-12-26T21:55:40.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.249+0000 7fe1b6421b80 1 -- 172.21.15.131:0/633168761 wait complete. 2023-12-26T21:55:40.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.249+0000 7fe1b6421b80 20 client.7815 trim_cache size 0 max 16384 2023-12-26T21:55:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.266+0000 7fe1b6421b80 1 Processor -- start 2023-12-26T21:55:40.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.266+0000 7fe1b6421b80 1 -- start start 2023-12-26T21:55:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a776178530 0x55a776135fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e4d0 0x55a77615be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e940 0x55a77617d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a776172d40 con 0x55a77617e940 2023-12-26T21:55:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a776184b10 con 0x55a77617e4d0 2023-12-26T21:55:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a77612c270 con 0x55a776178530 2023-12-26T21:55:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1a639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e940 0x55a77617d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1a639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e940 0x55a77617d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49526/0 (socket says 172.21.15.131:49526) 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1a639c700 1 -- 172.21.15.131:0/3991714880 learned_addr learned my addr 172.21.15.131:0/3991714880 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1a639c700 1 -- 172.21.15.131:0/3991714880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a776178530 msgr2=0x55a776135fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1a639c700 1 --2- 172.21.15.131:0/3991714880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a776178530 0x55a776135fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1a639c700 1 -- 172.21.15.131:0/3991714880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e4d0 msgr2=0x55a77615be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1a639c700 1 --2- 172.21.15.131:0/3991714880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e4d0 0x55a77615be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1a639c700 1 -- 172.21.15.131:0/3991714880 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a776052290 con 0x55a77617e940 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1a639c700 1 --2- 172.21.15.131:0/3991714880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e940 0x55a77617d580 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe198056940 tx=0x7fe198083f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf456b106fe5393 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe18b7fe700 1 -- 172.21.15.131:0/3991714880 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198057110 con 0x55a77617e940 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe18b7fe700 1 -- 172.21.15.131:0/3991714880 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1980569b0 con 0x55a77617e940 2023-12-26T21:55:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe18b7fe700 1 -- 172.21.15.131:0/3991714880 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198059c00 con 0x55a77617e940 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3991714880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e940 msgr2=0x55a77617d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3991714880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e940 0x55a77617d580 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe198056940 tx=0x7fe198083f00 comp rx=0 tx=0).stop 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3991714880 shutdown_connections 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3991714880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e940 0x55a77617d580 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3991714880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a776178530 0x55a776135fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3991714880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e4d0 0x55a77615be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3991714880 >> 172.21.15.131:0/3991714880 conn(0x55a776132440 msgr2=0x55a776129700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3991714880 shutdown_connections 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.269+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3991714880 wait complete. 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.270+0000 7fe1b6421b80 1 -- start start 2023-12-26T21:55:40.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.270+0000 7fe1b6421b80 1 Processor -- start 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a776178530 0x55a77617d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a776130ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1b6421b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 0x55a7761736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a776131530 con 0x55a77617e4d0 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77617d840 con 0x55a77617e940 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1b6421b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a776190dc0 con 0x55a776178530 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1b6421b80 10 client.0 _ll_register_callbacks cb 0x7fe19c014500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1a539a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a776130ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1a539a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a776130ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49528/0 (socket says 172.21.15.131:49528) 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1a539a700 1 -- 172.21.15.131:0/3750134440 learned_addr learned my addr 172.21.15.131:0/3750134440 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a776178530 0x55a77617d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.271+0000 7fe1a639c700 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 0x55a7761736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe1a539a700 1 -- 172.21.15.131:0/3750134440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a776178530 msgr2=0x55a77617d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe1a539a700 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a776178530 0x55a77617d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe1a539a700 1 -- 172.21.15.131:0/3750134440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 msgr2=0x55a7761736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe1a539a700 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 0x55a7761736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe1a539a700 1 -- 172.21.15.131:0/3750134440 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a00c49b0 con 0x55a77617e4d0 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe1a539a700 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a776130ff0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x55a776184180 tx=0x7fe190014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0b0d5bc10a6fb0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe17ffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe17ffff700 1 -- 172.21.15.131:0/3750134440 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1900117a0 con 0x55a77617e4d0 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3750134440 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1a00c5460 con 0x55a77617e4d0 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe17ffff700 1 -- 172.21.15.131:0/3750134440 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe190011900 con 0x55a77617e4d0 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe17ffff700 1 -- 172.21.15.131:0/3750134440 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe190011ac0 con 0x55a77617e4d0 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe1b6421b80 10 client.7821 fetch_fsmap learned FSMap version 57 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.272+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3750134440 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55a775f1bc80 con 0x55a77617e4d0 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe17ffff700 1 -- 172.21.15.131:0/3750134440 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a00c5460 con 0x55a77617e4d0 2023-12-26T21:55:40.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe17ffff700 1 -- 172.21.15.131:0/3750134440 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 57) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe190003040 con 0x55a77617e4d0 2023-12-26T21:55:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe1b6421b80 10 client.7821 fetch_fsmap finished waiting for FSMap version 57 2023-12-26T21:55:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe1b6421b80 10 client.7821 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-26T21:55:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe1b6421b80 20 client.7821 populate_metadata read hostname 'smithi131' 2023-12-26T21:55:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] conn(0x55a77615a060 0x55a77615a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe1b6421b80 4 client.7821 mds_command: new command op to 7644 tid=0[{"prefix": "status"}] 2023-12-26T21:55:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3750134440 --> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a776128490 con 0x55a77615a060 2023-12-26T21:55:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] conn(0x55a77615a060 0x55a77615a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe1a5b9b700 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] conn(0x55a77615a060 0x55a77615a430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93247616312e58d server_cookie=9d4f2d547d17c4d4 in_seq=0 out_seq=0 2023-12-26T21:55:40.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.273+0000 7fe17ffff700 10 client.7821 ms_handle_connect on v2:172.21.15.131:6833/1932798418 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe17ffff700 1 -- 172.21.15.131:0/3750134440 <== mds.0 v2:172.21.15.131:6833/1932798418 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55a776128490 con 0x55a77615a060 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe17ffff700 10 client.7821 handle_command_reply: tid=0 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 client.7821 shutdown 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3750134440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 msgr2=0x55a776130ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a776130ff0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x55a776184180 tx=0x7fe190014000 comp rx=0 tx=0).stop 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3750134440 shutdown_connections 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.131:6833/1932798418,v1:172.21.15.131:6835/1932798418] conn(0x55a77615a060 0x55a77615a430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a77617e940 0x55a7761736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a776178530 0x55a77617d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe17ffff700 0 client.7821 ms_handle_reset on v2:172.21.15.131:6833/1932798418 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 --2- 172.21.15.131:0/3750134440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a77617e4d0 0x55a776130ff0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3750134440 >> 172.21.15.131:0/3750134440 conn(0x55a776132440 msgr2=0x55a776129700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3750134440 shutdown_connections 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 1 -- 172.21.15.131:0/3750134440 wait complete. 2023-12-26T21:55:40.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.274+0000 7fe1b6421b80 20 client.7821 trim_cache size 0 max 16384 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7644, 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 11.356811778999999, 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 57, 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 124, 2023-12-26T21:55:40.280 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 124, 2023-12-26T21:55:40.281 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 17.070386071000001 2023-12-26T21:55:40.281 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:55:40.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 -- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe184066f30 msgr2=0x7fe1840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe184066f30 0x7fe1840693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009db0 tx=0x7fe19c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 -- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00c8bf0 msgr2=0x7fe1a0167540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00c8bf0 0x7fe1a0167540 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00688f0 tx=0x7fe198018000 comp rx=0 tx=0).stop 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 -- 172.21.15.131:0/2187011349 shutdown_connections 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe184066f30 0x7fe1840693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1a00c8bf0 0x7fe1a0167540 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1a00b19f0 0x7fe1a0147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 --2- 172.21.15.131:0/2187011349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a00c8820 0x7fe1a00c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 -- 172.21.15.131:0/2187011349 >> 172.21.15.131:0/2187011349 conn(0x7fe1a0068040 msgr2=0x7fe1a0155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 -- 172.21.15.131:0/2187011349 shutdown_connections 2023-12-26T21:55:40.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.275+0000 7fe1a7dff700 1 -- 172.21.15.131:0/2187011349 wait complete. 2023-12-26T21:55:40.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:55:40.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.484+0000 7f4953dff700 1 Processor -- start 2023-12-26T21:55:40.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.484+0000 7f4953dff700 1 -- start start 2023-12-26T21:55:40.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.484+0000 7f4953dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f494c159920 0x7f494c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.484+0000 7f4953dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.484+0000 7f4953dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494c0c8bf0 con 0x7f494c159cf0 2023-12-26T21:55:40.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.484+0000 7f4953dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494c0c8e60 con 0x7f494c159920 2023-12-26T21:55:40.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.484+0000 7f4953dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c0c9140 con 0x7f494c155be0 2023-12-26T21:55:40.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f4951b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f494c159920 0x7f494c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f4951b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f494c159920 0x7f494c0b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49538/0 (socket says 172.21.15.131:49538) 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f4951b9b700 1 -- 172.21.15.131:0/259483607 learned_addr learned my addr 172.21.15.131:0/259483607 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f495239c700 1 --2- 172.21.15.131:0/259483607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f4951b9b700 1 -- 172.21.15.131:0/259483607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 msgr2=0x7f494c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f4951b9b700 1 --2- 172.21.15.131:0/259483607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f4951b9b700 1 -- 172.21.15.131:0/259483607 >> v1:172.21.15.190:6789/0 conn(0x7f494c159cf0 legacy=0x7f494c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f4951b9b700 1 -- 172.21.15.131:0/259483607 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c0c9420 con 0x7f494c159920 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f495239c700 1 --2- 172.21.15.131:0/259483607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.485+0000 7f4951b9b700 1 --2- 172.21.15.131:0/259483607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 0x7f494c0b00e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f49480099b0 tx=0x7f4948009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edece2d2feaac09 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4950b99700 1 -- 172.21.15.131:0/259483607 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800b860 con 0x7f494c159920 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4950b99700 1 -- 172.21.15.131:0/259483607 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494800b9c0 con 0x7f494c159920 2023-12-26T21:55:40.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4950b99700 1 -- 172.21.15.131:0/259483607 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800bd60 con 0x7f494c159920 2023-12-26T21:55:40.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4953dff700 1 -- 172.21.15.131:0/259483607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 msgr2=0x7f494c0b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4953dff700 1 --2- 172.21.15.131:0/259483607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 0x7f494c0b00e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f49480099b0 tx=0x7f4948009250 comp rx=0 tx=0).stop 2023-12-26T21:55:40.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4953dff700 1 -- 172.21.15.131:0/259483607 shutdown_connections 2023-12-26T21:55:40.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4953dff700 1 --2- 172.21.15.131:0/259483607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4953dff700 1 --2- 172.21.15.131:0/259483607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 0x7f494c0b00e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4953dff700 1 -- 172.21.15.131:0/259483607 >> 172.21.15.131:0/259483607 conn(0x7f494c068040 msgr2=0x7f494c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4953dff700 1 -- 172.21.15.131:0/259483607 shutdown_connections 2023-12-26T21:55:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.486+0000 7f4953dff700 1 -- 172.21.15.131:0/259483607 wait complete. 2023-12-26T21:55:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.488+0000 7f4953dff700 1 Processor -- start 2023-12-26T21:55:40.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.488+0000 7f4953dff700 1 -- start start 2023-12-26T21:55:40.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f4953dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f4953dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 0x7f494c141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f4953dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f494c159cf0 0x7f494c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f4953dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c142250 con 0x7f494c159920 2023-12-26T21:55:40.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f4953dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c150b30 con 0x7f494c159cf0 2023-12-26T21:55:40.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f4953dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c14ff20 con 0x7f494c155be0 2023-12-26T21:55:40.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f495139a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 0x7f494c141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f495139a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 0x7f494c141d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49546/0 (socket says 172.21.15.131:49546) 2023-12-26T21:55:40.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f495139a700 1 -- 172.21.15.131:0/2702676012 learned_addr learned my addr 172.21.15.131:0/2702676012 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f495239c700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f494c159cf0 0x7f494c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.489+0000 7f4951b9b700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f495139a700 1 -- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 msgr2=0x7f494c14f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f495139a700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c14f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f495139a700 1 -- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f494c159cf0 msgr2=0x7f494c158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f495139a700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f494c159cf0 0x7f494c158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f495139a700 1 -- 172.21.15.131:0/2702676012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480094f0 con 0x7f494c159920 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f495239c700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f494c159cf0 0x7f494c158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f4951b9b700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f495139a700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 0x7f494c141d10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c4e0 tx=0x7f493c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f4941ffb700 1 -- 172.21.15.131:0/2702676012 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c011590 con 0x7f494c159920 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f4941ffb700 1 -- 172.21.15.131:0/2702676012 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c011710 con 0x7f494c159920 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f4941ffb700 1 -- 172.21.15.131:0/2702676012 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c011ad0 con 0x7f494c159920 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c150200 con 0x7f494c159920 2023-12-26T21:55:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.490+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c1507c0 con 0x7f494c159920 2023-12-26T21:55:40.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.491+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c150cc0 con 0x7f494c159920 2023-12-26T21:55:40.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.492+0000 7f4941ffb700 1 -- 172.21.15.131:0/2702676012 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f493c01c440 con 0x7f494c159920 2023-12-26T21:55:40.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.493+0000 7f4941ffb700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4930066f30 0x7f49300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:40.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.493+0000 7f4941ffb700 1 -- 172.21.15.131:0/2702676012 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f493c083b80 con 0x7f494c159920 2023-12-26T21:55:40.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.493+0000 7f4951b9b700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4930066f30 0x7f49300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:40.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.494+0000 7f4951b9b700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4930066f30 0x7f49300693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49480099b0 tx=0x7f4948017000 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:40.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.498+0000 7f4941ffb700 1 -- 172.21.15.131:0/2702676012 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f493c050c60 con 0x7f494c159920 2023-12-26T21:55:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.732+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f494c0657e0 con 0x7f494c159920 2023-12-26T21:55:40.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.732+0000 7f4941ffb700 1 -- 172.21.15.131:0/2702676012 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f493c017070 con 0x7f494c159920 2023-12-26T21:55:40.733 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:55:40.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.734+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4930066f30 msgr2=0x7f49300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.734+0000 7f4953dff700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4930066f30 0x7f49300693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49480099b0 tx=0x7f4948017000 comp rx=0 tx=0).stop 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.734+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 msgr2=0x7f494c141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.734+0000 7f4953dff700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 0x7f494c141d10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c4e0 tx=0x7f493c00c8a0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.735+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 shutdown_connections 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.735+0000 7f4953dff700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f494c159cf0 0x7f494c158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.735+0000 7f4953dff700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c155be0 0x7f494c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.735+0000 7f4953dff700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4930066f30 0x7f49300693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.735+0000 7f4953dff700 1 --2- 172.21.15.131:0/2702676012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f494c159920 0x7f494c141d10 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.735+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 >> 172.21.15.131:0/2702676012 conn(0x7f494c068040 msgr2=0x7f494c0aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.735+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 shutdown_connections 2023-12-26T21:55:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:40.735+0000 7f4953dff700 1 -- 172.21.15.131:0/2702676012 wait complete. 2023-12-26T21:55:40.753 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T21:55:40.753 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:55:40.753 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T21:55:40.754 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T21:55:40.754 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T21:55:40.754 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T21:55:40.754 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T21:55:40.754 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:55:40.754 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:55:40.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:55:40.754 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T21:55:40.798 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:55:40.798 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:55:40.799 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41315sec preferred_lft 41315sec 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T21:55:40.800 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:55:40.801 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:55:40.801 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T21:55:40.801 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T21:55:40.801 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:55:40.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:55:40.801 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:55:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T21:55:40.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:55:40.907 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T21:55:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T21:55:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:55:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:55:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:55:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:55:40.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:55:40.954 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:55:40.955 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:55:40.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:55:40.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:55:40.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:55:40.955 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:55:41.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:55:41.125 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:55:41.169 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T21:55:41.169 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T21:55:41.169 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:55:41.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:55:41.169 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T21:55:41.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:55:41.280 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T21:55:41.323 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:55:41.323 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:55:41.323 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:55:41.406 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T21:55:41.406 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:55:41.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:55:41.544 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:55:41.545 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:55:41.573 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:55:41.573 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:55:41.573 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:55:41.573 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:55:41.573 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:55:41.577 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:55:41.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:55:41.578 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:55:41.646 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T21:55:41.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.838+0000 7f81b41ff700 1 Processor -- start 2023-12-26T21:55:41.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.838+0000 7f81b41ff700 1 -- start start 2023-12-26T21:55:41.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.839+0000 7f81b41ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0b0bf0 0x7f81ac0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:41.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.839+0000 7f81b41ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac156940 con 0x7f81ac0b0750 2023-12-26T21:55:41.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.839+0000 7f81b41ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac0ae500 con 0x7f81ac0cb320 2023-12-26T21:55:41.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.839+0000 7f81b41ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0c8cc0 con 0x7f81ac0b0bf0 2023-12-26T21:55:41.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b179a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f81ac0b0750 0x7f81ac0c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44820/0 (socket says 172.21.15.131:44820) 2023-12-26T21:55:41.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b179a700 1 -- 172.21.15.131:0/365980503 learned_addr learned my addr 172.21.15.131:0/365980503 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:41.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b279c700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0b0bf0 0x7f81ac0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:41.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559042015 0 0) 0x7f81ac156940 con 0x7f81ac0b0750 2023-12-26T21:55:41.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8198003570 con 0x7f81ac0b0750 2023-12-26T21:55:41.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2729882717 0 0) 0x7f8198003570 con 0x7f81ac0b0750 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0b0bf0 msgr2=0x7f81ac0c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b0f99700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0b0bf0 0x7f81ac0c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 >> v1:172.21.15.190:6789/0 conn(0x7f81ac0cb320 legacy=0x7f81ac069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.840+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac0b0280 con 0x7f81ac0b0750 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f819c001470 con 0x7f81ac0b0750 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 >> v1:172.21.15.131:6789/0 conn(0x7f81ac0b0750 legacy=0x7f81ac0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b0f99700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0b0bf0 0x7f8198003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b0f99700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8198003fd0 0x7f8198004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b0f99700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198004d50 0x7f81980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0c8cc0 con 0x7f8198004d50 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819800bb40 con 0x7f8198003fd0 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819800df10 con 0x7f81ac0b0bf0 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b279c700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198004d50 0x7f81980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:41.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b1f9b700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0b0bf0 0x7f8198003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b179a700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8198003fd0 0x7f8198004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b279c700 1 -- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0b0bf0 msgr2=0x7f8198003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b279c700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0b0bf0 0x7f8198003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b279c700 1 -- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8198003fd0 msgr2=0x7f8198004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b279c700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8198003fd0 0x7f8198004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.841+0000 7f81b279c700 1 -- 172.21.15.131:0/365980503 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198003570 con 0x7f8198004d50 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.842+0000 7f81b279c700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198004d50 0x7f81980051c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e210 tx=0x7f81a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89713244a773edd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.842+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400eac0 con 0x7f8198004d50 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.842+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a400f040 con 0x7f8198004d50 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.842+0000 7f81b0f99700 1 -- 172.21.15.131:0/365980503 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400ee70 con 0x7f8198004d50 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.842+0000 7f81b41ff700 1 -- 172.21.15.131:0/365980503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198004d50 msgr2=0x7f81980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.842+0000 7f81b41ff700 1 --2- 172.21.15.131:0/365980503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8198004d50 0x7f81980051c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e210 tx=0x7f81a400b040 comp rx=0 tx=0).stop 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.842+0000 7f81b279c700 1 -- 172.21.15.131:0/365980503 reap_dead start 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.842+0000 7f81b41ff700 1 -- 172.21.15.131:0/365980503 shutdown_connections 2023-12-26T21:55:41.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.842+0000 7f81b41ff700 1 -- 172.21.15.131:0/365980503 >> 172.21.15.131:0/365980503 conn(0x7f81ac068040 msgr2=0x7f81ac069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:41.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.843+0000 7f81b41ff700 1 -- 172.21.15.131:0/365980503 shutdown_connections 2023-12-26T21:55:41.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.843+0000 7f81b41ff700 1 -- 172.21.15.131:0/365980503 wait complete. 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.844+0000 7f81b41ff700 1 Processor -- start 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.844+0000 7f81b41ff700 1 -- start start 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b41ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8198004d50 0x7f81ac1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b41ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac14fad0 0x7f81ac150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b41ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ac1509a0 0x7f81ac150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b41ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac151330 con 0x7f81ac1509a0 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b41ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac1528f0 con 0x7f81ac14fad0 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b41ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac150090 con 0x7f8198004d50 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b279c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ac1509a0 0x7f81ac150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b279c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ac1509a0 0x7f81ac150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49556/0 (socket says 172.21.15.131:49556) 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b279c700 1 -- 172.21.15.131:0/136464294 learned_addr learned my addr 172.21.15.131:0/136464294 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.845+0000 7f81b179a700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac14fad0 0x7f81ac150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81b1f9b700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8198004d50 0x7f81ac1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81b279c700 1 -- 172.21.15.131:0/136464294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8198004d50 msgr2=0x7f81ac1523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81b279c700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8198004d50 0x7f81ac1523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81b279c700 1 -- 172.21.15.131:0/136464294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac14fad0 msgr2=0x7f81ac150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81b279c700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac14fad0 0x7f81ac150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81b279c700 1 -- 172.21.15.131:0/136464294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac0aa360 con 0x7f81ac1509a0 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81b279c700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ac1509a0 0x7f81ac150df0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f819800bb10 tx=0x7f81a401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81a1ffb700 1 -- 172.21.15.131:0/136464294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4027430 con 0x7f81ac1509a0 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81a1ffb700 1 -- 172.21.15.131:0/136464294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4022070 con 0x7f81ac1509a0 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81a1ffb700 1 -- 172.21.15.131:0/136464294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4027800 con 0x7f81ac1509a0 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac0aa640 con 0x7f81ac1509a0 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.846+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac0aac00 con 0x7f81ac1509a0 2023-12-26T21:55:41.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.847+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac0657e0 con 0x7f81ac1509a0 2023-12-26T21:55:41.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.848+0000 7f81a1ffb700 1 -- 172.21.15.131:0/136464294 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f81a4024030 con 0x7f81ac1509a0 2023-12-26T21:55:41.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.849+0000 7f81a1ffb700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8190066f30 0x7f81900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:41.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.849+0000 7f81a1ffb700 1 -- 172.21.15.131:0/136464294 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f81a4086c50 con 0x7f81ac1509a0 2023-12-26T21:55:41.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.850+0000 7f81b1f9b700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8190066f30 0x7f81900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:41.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.850+0000 7f81b1f9b700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8190066f30 0x7f81900693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81a8000f60 tx=0x7f81a8009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:41.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:41.853+0000 7f81a1ffb700 1 -- 172.21.15.131:0/136464294 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81a4053c90 con 0x7f81ac1509a0 2023-12-26T21:55:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.074+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f81ac0ca4b0 con 0x7f8190066f30 2023-12-26T21:55:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.075+0000 7f81a1ffb700 1 -- 172.21.15.131:0/136464294 <== mgr.7159 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f81ac0ca4b0 con 0x7f8190066f30 2023-12-26T21:55:42.075 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:55:42.075 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:55:42.075 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T21:55:42.075 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:55:42.075 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:55:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.077+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8190066f30 msgr2=0x7f81900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.077+0000 7f81b41ff700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8190066f30 0x7f81900693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81a8000f60 tx=0x7f81a8009040 comp rx=0 tx=0).stop 2023-12-26T21:55:42.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.077+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ac1509a0 msgr2=0x7f81ac150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.077+0000 7f81b41ff700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ac1509a0 0x7f81ac150df0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f819800bb10 tx=0x7f81a401a000 comp rx=0 tx=0).stop 2023-12-26T21:55:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.077+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 shutdown_connections 2023-12-26T21:55:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.077+0000 7f81b41ff700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8190066f30 0x7f81900693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.077+0000 7f81b41ff700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ac1509a0 0x7f81ac150df0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.078+0000 7f81b41ff700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8198004d50 0x7f81ac1523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.078+0000 7f81b41ff700 1 --2- 172.21.15.131:0/136464294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac14fad0 0x7f81ac150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.078+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 >> 172.21.15.131:0/136464294 conn(0x7f81ac068040 msgr2=0x7f81ac0acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.078+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 shutdown_connections 2023-12-26T21:55:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.078+0000 7f81b41ff700 1 -- 172.21.15.131:0/136464294 wait complete. 2023-12-26T21:55:42.095 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T21:55:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.287+0000 7fce5cdff700 1 Processor -- start 2023-12-26T21:55:42.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.287+0000 7fce5cdff700 1 -- start start 2023-12-26T21:55:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.288+0000 7fce5cdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fce580b0bf0 0x7fce580c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.288+0000 7fce5cdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce58156940 con 0x7fce580cb320 2023-12-26T21:55:42.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.288+0000 7fce5cdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce580ae500 con 0x7fce580b0750 2023-12-26T21:55:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.288+0000 7fce5cdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce580c8cc0 con 0x7fce580b0bf0 2023-12-26T21:55:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.288+0000 7fce5659c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fce580cb320 0x7fce58069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44822/0 (socket says 172.21.15.131:44822) 2023-12-26T21:55:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.289+0000 7fce5659c700 1 -- 172.21.15.131:0/3506886926 learned_addr learned my addr 172.21.15.131:0/3506886926 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.289+0000 7fce56d9d700 1 --2- 172.21.15.131:0/3506886926 >> v2:172.21.15.190:3300/0 conn(0x7fce580b0bf0 0x7fce580c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.289+0000 7fce56d9d700 1 -- 172.21.15.131:0/3506886926 >> v1:172.21.15.190:6789/0 conn(0x7fce580b0750 legacy=0x7fce580c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:42.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.289+0000 7fce56d9d700 1 -- 172.21.15.131:0/3506886926 >> v1:172.21.15.131:6789/0 conn(0x7fce580cb320 legacy=0x7fce58069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:42.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.289+0000 7fce56d9d700 1 -- 172.21.15.131:0/3506886926 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580b0280 con 0x7fce580b0bf0 2023-12-26T21:55:42.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.290+0000 7fce56d9d700 1 --2- 172.21.15.131:0/3506886926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce580b0bf0 0x7fce580c9b10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fce50009e60 tx=0x7fce50009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea746e6b5a2cb034 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.292+0000 7fce5559a700 1 -- 172.21.15.131:0/3506886926 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000e070 con 0x7fce580b0bf0 2023-12-26T21:55:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.292+0000 7fce5559a700 1 -- 172.21.15.131:0/3506886926 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5000bc30 con 0x7fce580b0bf0 2023-12-26T21:55:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.292+0000 7fce5559a700 1 -- 172.21.15.131:0/3506886926 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50012430 con 0x7fce580b0bf0 2023-12-26T21:55:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.292+0000 7fce5cdff700 1 -- 172.21.15.131:0/3506886926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce580b0bf0 msgr2=0x7fce580c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.292+0000 7fce5cdff700 1 --2- 172.21.15.131:0/3506886926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce580b0bf0 0x7fce580c9b10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fce50009e60 tx=0x7fce50009780 comp rx=0 tx=0).stop 2023-12-26T21:55:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.293+0000 7fce5cdff700 1 -- 172.21.15.131:0/3506886926 shutdown_connections 2023-12-26T21:55:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.293+0000 7fce5cdff700 1 --2- 172.21.15.131:0/3506886926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce580b0bf0 0x7fce580c9b10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.293+0000 7fce5cdff700 1 -- 172.21.15.131:0/3506886926 >> 172.21.15.131:0/3506886926 conn(0x7fce58068040 msgr2=0x7fce58069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.293+0000 7fce5cdff700 1 -- 172.21.15.131:0/3506886926 shutdown_connections 2023-12-26T21:55:42.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.293+0000 7fce5cdff700 1 -- 172.21.15.131:0/3506886926 wait complete. 2023-12-26T21:55:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.294+0000 7fce5cdff700 1 Processor -- start 2023-12-26T21:55:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.294+0000 7fce5cdff700 1 -- start start 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.295+0000 7fce5cdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce580b0750 0x7fce5816c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.295+0000 7fce5cdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce580b0bf0 0x7fce5806db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.295+0000 7fce5cdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce580cb320 0x7fce580c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.295+0000 7fce5cdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58169540 con 0x7fce580cb320 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.295+0000 7fce5cdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58169800 con 0x7fce580b0bf0 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.295+0000 7fce5cdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce58169ae0 con 0x7fce580b0750 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.295+0000 7fce56d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce580cb320 0x7fce580c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.295+0000 7fce56d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce580cb320 0x7fce580c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49572/0 (socket says 172.21.15.131:49572) 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.295+0000 7fce56d9d700 1 -- 172.21.15.131:0/1238962923 learned_addr learned my addr 172.21.15.131:0/1238962923 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:42.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce5659c700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce580b0750 0x7fce5816c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce55d9b700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce580b0bf0 0x7fce5806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce56d9d700 1 -- 172.21.15.131:0/1238962923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce580b0750 msgr2=0x7fce5816c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce56d9d700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce580b0750 0x7fce5816c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce56d9d700 1 -- 172.21.15.131:0/1238962923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce580b0bf0 msgr2=0x7fce5806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce56d9d700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce580b0bf0 0x7fce5806db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce56d9d700 1 -- 172.21.15.131:0/1238962923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50009a20 con 0x7fce580cb320 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce56d9d700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce580cb320 0x7fce580c9d10 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fce50013000 tx=0x7fce50019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce467fc700 1 -- 172.21.15.131:0/1238962923 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50012e10 con 0x7fce580cb320 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce467fc700 1 -- 172.21.15.131:0/1238962923 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5001e630 con 0x7fce580cb320 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce58169dc0 con 0x7fce580cb320 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.296+0000 7fce467fc700 1 -- 172.21.15.131:0/1238962923 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5001e9f0 con 0x7fce580cb320 2023-12-26T21:55:42.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.297+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5816a380 con 0x7fce580cb320 2023-12-26T21:55:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.298+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce580657e0 con 0x7fce580cb320 2023-12-26T21:55:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.298+0000 7fce467fc700 1 -- 172.21.15.131:0/1238962923 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 53) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fce50024070 con 0x7fce580cb320 2023-12-26T21:55:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.299+0000 7fce467fc700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fce38066f30 0x7fce380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.299+0000 7fce467fc700 1 -- 172.21.15.131:0/1238962923 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fce50086e50 con 0x7fce580cb320 2023-12-26T21:55:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.300+0000 7fce5659c700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fce38066f30 0x7fce380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:42.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.300+0000 7fce5659c700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fce38066f30 0x7fce380693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce48001950 tx=0x7fce48009040 comp rx=0 tx=0).ready entity=mgr.7159 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:42.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.304+0000 7fce467fc700 1 -- 172.21.15.131:0/1238962923 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce50053f30 con 0x7fce580cb320 2023-12-26T21:55:42.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:42.551+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fce5806e380 con 0x7fce580cb320 2023-12-26T21:55:43.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.297+0000 7fce467fc700 1 -- 172.21.15.131:0/1238962923 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 54) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fce50003040 con 0x7fce580cb320 2023-12-26T21:55:43.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.298+0000 7fce467fc700 1 -- 172.21.15.131:0/1238962923 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v54) v1 ==== 92+0+0 (secure 0 0 0) 0x7fce50053f30 con 0x7fce580cb320 2023-12-26T21:55:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.301+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fce38066f30 msgr2=0x7fce380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.301+0000 7fce5cdff700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fce38066f30 0x7fce380693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce48001950 tx=0x7fce48009040 comp rx=0 tx=0).stop 2023-12-26T21:55:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.301+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce580cb320 msgr2=0x7fce580c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.301+0000 7fce5cdff700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce580cb320 0x7fce580c9d10 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fce50013000 tx=0x7fce50019000 comp rx=0 tx=0).stop 2023-12-26T21:55:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.302+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 shutdown_connections 2023-12-26T21:55:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.302+0000 7fce5cdff700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fce38066f30 0x7fce380693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.302+0000 7fce5cdff700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce580cb320 0x7fce580c9d10 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.302+0000 7fce5cdff700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce580b0750 0x7fce5816c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.302+0000 7fce5cdff700 1 --2- 172.21.15.131:0/1238962923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce580b0bf0 0x7fce5806db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.302+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 >> 172.21.15.131:0/1238962923 conn(0x7fce58068040 msgr2=0x7fce58158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.302+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 shutdown_connections 2023-12-26T21:55:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.302+0000 7fce5cdff700 1 -- 172.21.15.131:0/1238962923 wait complete. 2023-12-26T21:55:43.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T21:55:43.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.521+0000 7f4bf5fff700 1 Processor -- start 2023-12-26T21:55:43.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.521+0000 7f4bf5fff700 1 -- start start 2023-12-26T21:55:43.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.522+0000 7f4bf5fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf0158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.522+0000 7f4bf5fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4bf006b8c0 0x7f4bf00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.522+0000 7f4bf5fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf015ba70 con 0x7f4bf006bd60 2023-12-26T21:55:43.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.522+0000 7f4bf5fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf015bf00 con 0x7f4bf006b8c0 2023-12-26T21:55:43.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.522+0000 7f4bf5fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf006c2a0 con 0x7f4bf0158030 2023-12-26T21:55:43.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.522+0000 7f4beeffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4bf006b8c0 0x7f4bf00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.522+0000 7f4bef7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf0158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4beffff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f4bf006bd60 0x7f4bf006cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:34064/0 (socket says 172.21.15.131:34064) 2023-12-26T21:55:43.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4bef7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf0158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:50124/0 (socket says 172.21.15.131:50124) 2023-12-26T21:55:43.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4beffff700 1 -- 172.21.15.131:0/785361066 learned_addr learned my addr 172.21.15.131:0/785361066 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:43.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4beeffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4bf006b8c0 0x7f4bf00c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:60844/0 (socket says 172.21.15.131:60844) 2023-12-26T21:55:43.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4bee7fc700 1 -- 172.21.15.131:0/785361066 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713595830 0 0) 0x7f4bf015ba70 con 0x7f4bf006bd60 2023-12-26T21:55:43.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4bee7fc700 1 -- 172.21.15.131:0/785361066 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd8003570 con 0x7f4bf006bd60 2023-12-26T21:55:43.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4beeffd700 1 -- 172.21.15.131:0/785361066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 msgr2=0x7f4bf0158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:43.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4beeffd700 1 --2- 172.21.15.131:0/785361066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf0158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4beeffd700 1 -- 172.21.15.131:0/785361066 >> v1:172.21.15.190:6789/0 conn(0x7f4bf006bd60 legacy=0x7f4bf006cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4beeffd700 1 -- 172.21.15.131:0/785361066 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf015b140 con 0x7f4bf006b8c0 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4bef7fe700 1 --2- 172.21.15.131:0/785361066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf0158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.523+0000 7f4beeffd700 1 --2- 172.21.15.131:0/785361066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf006b8c0 0x7f4bf00c8a20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4be40098e0 tx=0x7f4be40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42681c7ff8af07b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bee7fc700 1 -- 172.21.15.131:0/785361066 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4011630 con 0x7f4bf006b8c0 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bee7fc700 1 -- 172.21.15.131:0/785361066 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be4011790 con 0x7f4bf006b8c0 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bee7fc700 1 -- 172.21.15.131:0/785361066 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4011b30 con 0x7f4bf006b8c0 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bf5fff700 1 -- 172.21.15.131:0/785361066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf006b8c0 msgr2=0x7f4bf00c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bf5fff700 1 --2- 172.21.15.131:0/785361066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf006b8c0 0x7f4bf00c8a20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4be40098e0 tx=0x7f4be40091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bf5fff700 1 -- 172.21.15.131:0/785361066 shutdown_connections 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bf5fff700 1 --2- 172.21.15.131:0/785361066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf0158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bf5fff700 1 --2- 172.21.15.131:0/785361066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf006b8c0 0x7f4bf00c8a20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bf5fff700 1 -- 172.21.15.131:0/785361066 >> 172.21.15.131:0/785361066 conn(0x7f4bf00ce3e0 msgr2=0x7f4bf00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bf5fff700 1 -- 172.21.15.131:0/785361066 shutdown_connections 2023-12-26T21:55:43.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.524+0000 7f4bf5fff700 1 -- 172.21.15.131:0/785361066 wait complete. 2023-12-26T21:55:43.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.526+0000 7f4bf5fff700 1 Processor -- start 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.526+0000 7f4bf5fff700 1 -- start start 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bf5fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf006b8c0 0x7f4bf014edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bf5fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf006bd60 0x7f4bf015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bf5fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bf5fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf01497b0 con 0x7f4bf006b8c0 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bf5fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0148ba0 con 0x7f4bf006bd60 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bf5fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0148e80 con 0x7f4bf0158030 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bef7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf006b8c0 0x7f4bf014edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bef7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf006b8c0 0x7f4bf014edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49594/0 (socket says 172.21.15.131:49594) 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bef7fe700 1 -- 172.21.15.131:0/1628461619 learned_addr learned my addr 172.21.15.131:0/1628461619 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4beeffd700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf006bd60 0x7f4bf015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4beffff700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bef7fe700 1 -- 172.21.15.131:0/1628461619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 msgr2=0x7f4bf015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bef7fe700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bef7fe700 1 -- 172.21.15.131:0/1628461619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf006bd60 msgr2=0x7f4bf015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bef7fe700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf006bd60 0x7f4bf015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.527+0000 7f4bef7fe700 1 -- 172.21.15.131:0/1628461619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be000d860 con 0x7f4bf006b8c0 2023-12-26T21:55:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.528+0000 7f4bef7fe700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf006b8c0 0x7f4bf014edf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4be000de90 tx=0x7f4be000f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:43.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.528+0000 7f4bd77fe700 1 -- 172.21.15.131:0/1628461619 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0005a00 con 0x7f4bf006b8c0 2023-12-26T21:55:43.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.528+0000 7f4bd77fe700 1 -- 172.21.15.131:0/1628461619 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be0013040 con 0x7f4bf006b8c0 2023-12-26T21:55:43.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.528+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be4009420 con 0x7f4bf006b8c0 2023-12-26T21:55:43.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.528+0000 7f4bd77fe700 1 -- 172.21.15.131:0/1628461619 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0005db0 con 0x7f4bf006b8c0 2023-12-26T21:55:43.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.528+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf0149440 con 0x7f4bf006b8c0 2023-12-26T21:55:43.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.529+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf00657e0 con 0x7f4bf006b8c0 2023-12-26T21:55:43.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.530+0000 7f4bd77fe700 1 -- 172.21.15.131:0/1628461619 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 54) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f4be0010aa0 con 0x7f4bf006b8c0 2023-12-26T21:55:43.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.531+0000 7f4bd77fe700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bdc066f80 0x7f4bdc069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.531+0000 7f4bd77fe700 1 -- 172.21.15.131:0/1628461619 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f4be0088b50 con 0x7f4bf006b8c0 2023-12-26T21:55:43.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.531+0000 7f4beeffd700 1 -- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bdc066f80 msgr2=0x7f4bdc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:43.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.531+0000 7f4beeffd700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bdc066f80 0x7f4bdc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:43.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.535+0000 7f4bd77fe700 1 -- 172.21.15.131:0/1628461619 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4be0055b90 con 0x7f4bf006b8c0 2023-12-26T21:55:43.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.732+0000 7f4beeffd700 1 -- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bdc066f80 msgr2=0x7f4bdc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:43.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.732+0000 7f4beeffd700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bdc066f80 0x7f4bdc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:43.732 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:43.731+0000 7fa67dc4d1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:55:43.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.738+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f4bf015b420 con 0x7f4bf006b8c0 2023-12-26T21:55:43.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.742+0000 7f4bd77fe700 1 -- 172.21.15.131:0/1628461619 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4be00591b0 con 0x7f4bf006b8c0 2023-12-26T21:55:43.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bdc066f80 msgr2=0x7f4bdc069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:43.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bdc066f80 0x7f4bdc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf006b8c0 msgr2=0x7f4bf014edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf006b8c0 0x7f4bf014edf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4be000de90 tx=0x7f4be000f040 comp rx=0 tx=0).stop 2023-12-26T21:55:43.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 shutdown_connections 2023-12-26T21:55:43.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bf0158030 0x7f4bf015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4bdc066f80 0x7f4bdc069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4bf006b8c0 0x7f4bf014edf0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 --2- 172.21.15.131:0/1628461619 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bf006bd60 0x7f4bf015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 >> 172.21.15.131:0/1628461619 conn(0x7f4bf00ce3e0 msgr2=0x7f4bf00c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:43.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 shutdown_connections 2023-12-26T21:55:43.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.744+0000 7f4bf5fff700 1 -- 172.21.15.131:0/1628461619 wait complete. 2023-12-26T21:55:43.751 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:43.750+0000 7f4d891061c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:55:43.763 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T21:55:43.836 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:43.836+0000 7fa67dc4d1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:55:43.862 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:43.862+0000 7f4d891061c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:55:43.925 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:43.925+0000 7fa67dc4d1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:55:43.956 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:43.956+0000 7f4d891061c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:55:43.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.961+0000 7f55c45ff700 1 Processor -- start 2023-12-26T21:55:43.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.962+0000 7f55c45ff700 1 -- start start 2023-12-26T21:55:43.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.963+0000 7f55c45ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc1599d0 0x7f55bc0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.963+0000 7f55c45ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55bc0d01e0 0x7f55bc0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.963+0000 7f55c45ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0cfd50 con 0x7f55bc159600 2023-12-26T21:55:43.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.963+0000 7f55c45ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55bc0b1280 con 0x7f55bc0d01e0 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.963+0000 7f55c45ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0cf140 con 0x7f55bc1599d0 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.963+0000 7f55c239b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f55bc159600 0x7f55bc15dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44844/0 (socket says 172.21.15.131:44844) 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.963+0000 7f55c239b700 1 -- 172.21.15.131:0/2841690471 learned_addr learned my addr 172.21.15.131:0/2841690471 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.963+0000 7f55c1b9a700 1 --2- 172.21.15.131:0/2841690471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc1599d0 0x7f55bc0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.963+0000 7f55c2b9c700 1 --2- 172.21.15.131:0/2841690471 >> v2:172.21.15.190:3300/0 conn(0x7f55bc0d01e0 0x7f55bc0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c1b9a700 1 -- 172.21.15.131:0/2841690471 >> v2:172.21.15.190:3300/0 conn(0x7f55bc0d01e0 msgr2=0x7f55bc0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c1b9a700 1 --2- 172.21.15.131:0/2841690471 >> v2:172.21.15.190:3300/0 conn(0x7f55bc0d01e0 0x7f55bc0ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c1b9a700 1 -- 172.21.15.131:0/2841690471 >> v1:172.21.15.131:6789/0 conn(0x7f55bc159600 legacy=0x7f55bc15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c1399700 1 -- 172.21.15.131:0/2841690471 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223930708 0 0) 0x7f55bc0cfd50 con 0x7f55bc159600 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c1b9a700 1 -- 172.21.15.131:0/2841690471 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0cf420 con 0x7f55bc1599d0 2023-12-26T21:55:43.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c2b9c700 1 --2- 172.21.15.131:0/2841690471 >> v2:172.21.15.190:3300/0 conn(0x7f55bc0d01e0 0x7f55bc0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c1b9a700 1 --2- 172.21.15.131:0/2841690471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc1599d0 0x7f55bc0ceb60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0098e0 tx=0x7f55ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dab8bb3dc14e4b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c1399700 1 -- 172.21.15.131:0/2841690471 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00b640 con 0x7f55bc1599d0 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c1399700 1 -- 172.21.15.131:0/2841690471 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac00b7a0 con 0x7f55bc1599d0 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.964+0000 7f55c1399700 1 -- 172.21.15.131:0/2841690471 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00bb40 con 0x7f55bc1599d0 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.965+0000 7f55c45ff700 1 -- 172.21.15.131:0/2841690471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc1599d0 msgr2=0x7f55bc0ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.965+0000 7f55c45ff700 1 --2- 172.21.15.131:0/2841690471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc1599d0 0x7f55bc0ceb60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0098e0 tx=0x7f55ac0091a0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.965+0000 7f55c45ff700 1 -- 172.21.15.131:0/2841690471 shutdown_connections 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.965+0000 7f55c45ff700 1 --2- 172.21.15.131:0/2841690471 >> v2:172.21.15.190:3300/0 conn(0x7f55bc0d01e0 0x7f55bc0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.965+0000 7f55c45ff700 1 --2- 172.21.15.131:0/2841690471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc1599d0 0x7f55bc0ceb60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.965+0000 7f55c45ff700 1 -- 172.21.15.131:0/2841690471 >> 172.21.15.131:0/2841690471 conn(0x7f55bc06ece0 msgr2=0x7f55bc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.965+0000 7f55c45ff700 1 -- 172.21.15.131:0/2841690471 shutdown_connections 2023-12-26T21:55:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.965+0000 7f55c45ff700 1 -- 172.21.15.131:0/2841690471 wait complete. 2023-12-26T21:55:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.967+0000 7f55c45ff700 1 Processor -- start 2023-12-26T21:55:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.967+0000 7f55c45ff700 1 -- start start 2023-12-26T21:55:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c45ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc0d01e0 0x7f55bc0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c45ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55bc159600 0x7f55bc0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c45ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55bc0c68e0 0x7f55bc169aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c45ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0b4240 con 0x7f55bc0c68e0 2023-12-26T21:55:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c45ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc157820 con 0x7f55bc159600 2023-12-26T21:55:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c45ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0c6d50 con 0x7f55bc0d01e0 2023-12-26T21:55:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c2b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55bc0c68e0 0x7f55bc169aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c239b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc0d01e0 0x7f55bc0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c1b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55bc159600 0x7f55bc0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c2b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55bc0c68e0 0x7f55bc169aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49604/0 (socket says 172.21.15.131:49604) 2023-12-26T21:55:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c2b9c700 1 -- 172.21.15.131:0/1303205138 learned_addr learned my addr 172.21.15.131:0/1303205138 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c2b9c700 1 -- 172.21.15.131:0/1303205138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc0d01e0 msgr2=0x7f55bc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c2b9c700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc0d01e0 0x7f55bc0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c2b9c700 1 -- 172.21.15.131:0/1303205138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55bc159600 msgr2=0x7f55bc0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c2b9c700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55bc159600 0x7f55bc0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c2b9c700 1 -- 172.21.15.131:0/1303205138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b4009a20 con 0x7f55bc0c68e0 2023-12-26T21:55:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.968+0000 7f55c1b9a700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55bc159600 0x7f55bc0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:55:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.969+0000 7f55c2b9c700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55bc0c68e0 0x7f55bc169aa0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009f60 tx=0x7f55b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.969+0000 7f55b27fc700 1 -- 172.21.15.131:0/1303205138 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b400cb70 con 0x7f55bc0c68e0 2023-12-26T21:55:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.969+0000 7f55b27fc700 1 -- 172.21.15.131:0/1303205138 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b400f040 con 0x7f55bc0c68e0 2023-12-26T21:55:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.969+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac009420 con 0x7f55bc0c68e0 2023-12-26T21:55:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.969+0000 7f55b27fc700 1 -- 172.21.15.131:0/1303205138 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4014480 con 0x7f55bc0c68e0 2023-12-26T21:55:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.969+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc0b5300 con 0x7f55bc0c68e0 2023-12-26T21:55:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.970+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc0657e0 con 0x7f55bc0c68e0 2023-12-26T21:55:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.971+0000 7f55b27fc700 1 -- 172.21.15.131:0/1303205138 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 54) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f55b4003020 con 0x7f55bc0c68e0 2023-12-26T21:55:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.972+0000 7f55b27fc700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55a0066f80 0x7f55a0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.972+0000 7f55b27fc700 1 -- 172.21.15.131:0/1303205138 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f55b4085100 con 0x7f55bc0c68e0 2023-12-26T21:55:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.972+0000 7f55c239b700 1 -- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55a0066f80 msgr2=0x7f55a0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.972+0000 7f55c239b700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55a0066f80 0x7f55a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:43.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:43.980+0000 7f55b27fc700 1 -- 172.21.15.131:0/1303205138 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55b4052140 con 0x7f55bc0c68e0 2023-12-26T21:55:44.010 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:44.009+0000 7fa67dc4d1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:55:44.046 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:44.047+0000 7f4d891061c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:55:44.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.173+0000 7f55c239b700 1 -- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55a0066f80 msgr2=0x7f55a0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:44.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.173+0000 7f55c239b700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55a0066f80 0x7f55a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:44.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.179+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f55bc0cf700 con 0x7f55bc0c68e0 2023-12-26T21:55:44.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.184+0000 7f55b27fc700 1 -- 172.21.15.131:0/1303205138 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f55b4055760 con 0x7f55bc0c68e0 2023-12-26T21:55:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55a0066f80 msgr2=0x7f55a0069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55a0066f80 0x7f55a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55bc0c68e0 msgr2=0x7f55bc169aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55bc0c68e0 0x7f55bc169aa0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009f60 tx=0x7f55b400b040 comp rx=0 tx=0).stop 2023-12-26T21:55:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 shutdown_connections 2023-12-26T21:55:44.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f55a0066f80 0x7f55a0069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55bc0c68e0 0x7f55bc169aa0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55bc0d01e0 0x7f55bc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 --2- 172.21.15.131:0/1303205138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55bc159600 0x7f55bc0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 >> 172.21.15.131:0/1303205138 conn(0x7f55bc06ece0 msgr2=0x7f55bc14e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 shutdown_connections 2023-12-26T21:55:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.186+0000 7f55c45ff700 1 -- 172.21.15.131:0/1303205138 wait complete. 2023-12-26T21:55:44.200 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:44.200+0000 7fa67dc4d1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:55:44.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:44.204 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-26T21:55:44.248 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:44.248+0000 7f4d891061c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:55:44.257 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-12-26T21:55:44.373 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:44.373+0000 7fa67dc4d1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:55:44.426 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:44.426+0000 7f4d891061c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:55:44.457 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:44.457+0000 7fa67dc4d1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:55:44.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.496+0000 7f5b4d1ff700 1 Processor -- start 2023-12-26T21:55:44.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.496+0000 7f5b4d1ff700 1 -- start start 2023-12-26T21:55:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.497+0000 7f5b4d1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cf390 0x7f5b480c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:44.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.497+0000 7f5b4d1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5b480cd0a0 0x7f5b480cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.497+0000 7f5b4d1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b480cbb00 con 0x7f5b48159810 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.497+0000 7f5b4d1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b480cb2a0 con 0x7f5b480cd0a0 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.497+0000 7f5b4d1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480ce040 con 0x7f5b480cf390 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b46d9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5b48159810 0x7f5b480cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44860/0 (socket says 172.21.15.131:44860) 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b46d9d700 1 -- 172.21.15.131:0/3659195353 learned_addr learned my addr 172.21.15.131:0/3659195353 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b4759e700 1 --2- 172.21.15.131:0/3659195353 >> v2:172.21.15.131:3300/0 conn(0x7f5b480cd0a0 0x7f5b480cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b4659c700 1 --2- 172.21.15.131:0/3659195353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cf390 0x7f5b480c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b45d9b700 1 -- 172.21.15.131:0/3659195353 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769227524 0 0) 0x7f5b480cbb00 con 0x7f5b48159810 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b45d9b700 1 -- 172.21.15.131:0/3659195353 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b30003570 con 0x7f5b48159810 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b4759e700 1 -- 172.21.15.131:0/3659195353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cf390 msgr2=0x7f5b480c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b4759e700 1 --2- 172.21.15.131:0/3659195353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cf390 0x7f5b480c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b4759e700 1 -- 172.21.15.131:0/3659195353 >> v1:172.21.15.131:6789/0 conn(0x7f5b48159810 legacy=0x7f5b480cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:44.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.498+0000 7f5b4759e700 1 -- 172.21.15.131:0/3659195353 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b48159be0 con 0x7f5b480cd0a0 2023-12-26T21:55:44.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.499+0000 7f5b4759e700 1 --2- 172.21.15.131:0/3659195353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b480cd0a0 0x7f5b480cd4e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009e60 tx=0x7f5b40009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6951ec115da6f43 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:44.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.499+0000 7f5b45d9b700 1 -- 172.21.15.131:0/3659195353 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000e070 con 0x7f5b480cd0a0 2023-12-26T21:55:44.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.499+0000 7f5b45d9b700 1 -- 172.21.15.131:0/3659195353 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4000bab0 con 0x7f5b480cd0a0 2023-12-26T21:55:44.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.499+0000 7f5b45d9b700 1 -- 172.21.15.131:0/3659195353 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000be70 con 0x7f5b480cd0a0 2023-12-26T21:55:44.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.499+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/3659195353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b480cd0a0 msgr2=0x7f5b480cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:44.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.499+0000 7f5b4d1ff700 1 --2- 172.21.15.131:0/3659195353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b480cd0a0 0x7f5b480cd4e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009e60 tx=0x7f5b40009780 comp rx=0 tx=0).stop 2023-12-26T21:55:44.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.500+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/3659195353 shutdown_connections 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.500+0000 7f5b4d1ff700 1 --2- 172.21.15.131:0/3659195353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b480cd0a0 0x7f5b480cd4e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.500+0000 7f5b4d1ff700 1 --2- 172.21.15.131:0/3659195353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cf390 0x7f5b480c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.500+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/3659195353 >> 172.21.15.131:0/3659195353 conn(0x7f5b48099950 msgr2=0x7f5b4806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.500+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/3659195353 shutdown_connections 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.500+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/3659195353 wait complete. 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.501+0000 7f5b4d1ff700 1 Processor -- start 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.501+0000 7f5b4d1ff700 1 -- start start 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4d1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cd0a0 0x7f5b48169330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4d1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b480cf390 0x7f5b480b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4d1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b48159810 0x7f5b4816a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4d1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480b4360 con 0x7f5b48159810 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4d1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b48152a80 con 0x7f5b480cf390 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4d1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4816aa40 con 0x7f5b480cd0a0 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4759e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b48159810 0x7f5b4816a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4759e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b48159810 0x7f5b4816a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49614/0 (socket says 172.21.15.131:49614) 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4759e700 1 -- 172.21.15.131:0/1311832220 learned_addr learned my addr 172.21.15.131:0/1311832220 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b46d9d700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cd0a0 0x7f5b48169330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.502+0000 7f5b4659c700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b480cf390 0x7f5b480b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b4759e700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cd0a0 msgr2=0x7f5b48169330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b4759e700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cd0a0 0x7f5b48169330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b4759e700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b480cf390 msgr2=0x7f5b480b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b4759e700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b480cf390 0x7f5b480b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b4759e700 1 -- 172.21.15.131:0/1311832220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b40009a20 con 0x7f5b48159810 2023-12-26T21:55:44.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b4759e700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b48159810 0x7f5b4816a500 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000b610 tx=0x7f5b40019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40012dd0 con 0x7f5b48159810 2023-12-26T21:55:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b40027470 con 0x7f5b48159810 2023-12-26T21:55:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40027830 con 0x7f5b48159810 2023-12-26T21:55:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.503+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b480b5b00 con 0x7f5b48159810 2023-12-26T21:55:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.504+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b48169c60 con 0x7f5b48159810 2023-12-26T21:55:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.505+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b480717f0 con 0x7f5b48159810 2023-12-26T21:55:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.505+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 54) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f5b40024070 con 0x7f5b48159810 2023-12-26T21:55:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.506+0000 7f5b2effd700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b34069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:44.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.506+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f5b400871e0 con 0x7f5b48159810 2023-12-26T21:55:44.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.506+0000 7f5b46d9d700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 msgr2=0x7f5b34069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:44.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.506+0000 7f5b46d9d700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b34069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:55:44.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.511+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b400542d0 con 0x7f5b48159810 2023-12-26T21:55:44.516 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:44.516+0000 7f4d891061c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:55:44.604 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:44.604+0000 7fa67dc4d1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:55:44.665 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:44.665+0000 7f4d891061c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:55:44.695 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:44.694+0000 7fa67dc4d1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:55:44.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.707+0000 7f5b46d9d700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 msgr2=0x7f5b34069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:44.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.707+0000 7f5b46d9d700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b34069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:55:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:44.733+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b48152c20 con 0x7f5b34066f80 2023-12-26T21:55:44.756 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:44.756+0000 7f4d891061c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:55:44.875 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:44.875+0000 7fa67dc4d1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:55:44.939 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:44.939+0000 7f4d891061c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:55:45.028 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:45.027+0000 7fa67dc4d1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:55:45.090 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:45.090+0000 7f4d891061c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:55:45.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:45.108+0000 7f5b46d9d700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 msgr2=0x7f5b34069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:45.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:45.108+0000 7f5b46d9d700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b34069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:55:45.413 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:45.410+0000 7fa67dc4d1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:55:45.471 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:45.471+0000 7f4d891061c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:55:45.674 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:45.674+0000 7fa67dc4d1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:55:45.734 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:45.735+0000 7f4d891061c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:55:45.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:45.909+0000 7f5b46d9d700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 msgr2=0x7f5b34069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:45.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:45.909+0000 7f5b46d9d700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b34069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T21:55:45.952 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:45.952+0000 7fa67dc4d1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:55:46.014 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:46.014+0000 7f4d891061c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:55:46.040 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:46.040+0000 7fa67dc4d1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:55:46.102 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:46.103+0000 7f4d891061c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:55:46.767 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:46.766+0000 7fa67dc4d1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:55:46.832 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:46.832+0000 7f4d891061c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:55:46.861 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:46.861+0000 7fa67dc4d1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:55:46.924 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:46.924+0000 7f4d891061c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:55:47.011 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:47.011+0000 7fa67dc4d1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:55:47.074 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:47.075+0000 7f4d891061c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:55:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:47.511+0000 7f5b46d9d700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 msgr2=0x7f5b34069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:55:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:47.511+0000 7f5b46d9d700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b34069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T21:55:47.632 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:47.631+0000 7fa67dc4d1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:55:47.695 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:47.696+0000 7f4d891061c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:55:47.818 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:47.818+0000 7fa67dc4d1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:55:47.882 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:47.883+0000 7f4d891061c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:55:48.008 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:48.008+0000 7fa67dc4d1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:55:48.073 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:48.073+0000 7f4d891061c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:55:48.851 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:48.850+0000 7fa67dc4d1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:55:48.912 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:48.912+0000 7f4d891061c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:55:49.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:55:49.108 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:55:49.130 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:49.129+0000 7fa67dc4d1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:55:49.194 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.195+0000 7f4d891061c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:55:49.226 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:49.226+0000 7fa67dc4d1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:55:49.289 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.290+0000 7f4d891061c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:55:49.425 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:55:49.425+0000 7fa67dc4d1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:55:49.484 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.485+0000 7f4d891061c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:55:49.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:49.497+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 55) v1 ==== 84470+0+0 (secure 0 0 0) 0x7f5b40003040 con 0x7f5b48159810 2023-12-26T21:55:49.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:49.497+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 msgr2=0x7f5b34069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:55:49.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:49.497+0000 7f5b2effd700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b34069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:49.616 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.616+0000 7f4cfda77700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:49.616 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.616+0000 7f4cfda77700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:49.616 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.616+0000 7f4cfda77700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:49.616 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.616+0000 7f4cfda77700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:49.616 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.616+0000 7f4cfda77700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:49.617 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.617+0000 7f4cfaa71700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:49.617 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.617+0000 7f4cfaa71700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:49.617 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.617+0000 7f4cfaa71700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:49.617 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.617+0000 7f4cfaa71700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:49.617 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:55:49.617+0000 7f4cfaa71700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:55:50.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.498+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 56) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f5b400546a0 con 0x7f5b48159810 2023-12-26T21:55:50.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.498+0000 7f5b2effd700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b340693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:50.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.498+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b48152c20 con 0x7f5b34066f80 2023-12-26T21:55:50.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.499+0000 7f5b46d9d700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b340693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:50.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.501+0000 7f5b46d9d700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b340693b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5b4814e3a0 tx=0x7f5b38009040 comp rx=0 tx=0).ready entity=mgr.7848 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.575+0000 7f5b2effd700 1 -- 172.21.15.131:0/1311832220 <== mgr.7848 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5b48152c20 con 0x7f5b34066f80 2023-12-26T21:55:50.576 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path snap_test_dir1 2023-12-26T21:55:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 msgr2=0x7f5b340693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b340693b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5b4814e3a0 tx=0x7f5b38009040 comp rx=0 tx=0).stop 2023-12-26T21:55:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b48159810 msgr2=0x7f5b4816a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b48159810 0x7f5b4816a500 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000b610 tx=0x7f5b40019000 comp rx=0 tx=0).stop 2023-12-26T21:55:50.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 shutdown_connections 2023-12-26T21:55:50.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5b34066f80 0x7f5b340693b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:50.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b48159810 0x7f5b4816a500 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:50.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b480cd0a0 0x7f5b48169330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:50.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 --2- 172.21.15.131:0/1311832220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b480cf390 0x7f5b480b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:50.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 >> 172.21.15.131:0/1311832220 conn(0x7f5b48099950 msgr2=0x7f5b4815a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:50.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 shutdown_connections 2023-12-26T21:55:50.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.578+0000 7f5b4d1ff700 1 -- 172.21.15.131:0/1311832220 wait complete. 2023-12-26T21:55:50.598 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-26T21:55:50.598 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-26-21_56_00 in ~10s... 2023-12-26T21:55:50.599 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-12-26T21:55:50.599 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-12-26T21:55:50.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.794+0000 7fe2d9dff700 1 Processor -- start 2023-12-26T21:55:50.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.795+0000 7fe2d9dff700 1 -- start start 2023-12-26T21:55:50.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d9dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2d4156730 0x7fe2d40c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:50.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d9dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d406ee80 0x7fe2d40c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:50.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d9dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d40acfc0 con 0x7fe2d4156360 2023-12-26T21:55:50.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d9dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d40c91f0 con 0x7fe2d4156730 2023-12-26T21:55:50.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d9dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40c94d0 con 0x7fe2d406ee80 2023-12-26T21:55:50.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2d4156730 0x7fe2d40c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:50.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2d4156730 0x7fe2d40c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60848/0 (socket says 172.21.15.131:60848) 2023-12-26T21:55:50.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d37fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe2d4156360 0x7fe2d406e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41876/0 (socket says 172.21.15.131:41876) 2023-12-26T21:55:50.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d2ffd700 1 -- 172.21.15.131:0/2803974296 learned_addr learned my addr 172.21.15.131:0/2803974296 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d2ffd700 1 -- 172.21.15.131:0/2803974296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d406ee80 msgr2=0x7fe2d40c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d3fff700 1 --2- 172.21.15.131:0/2803974296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d406ee80 0x7fe2d40c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d2ffd700 1 --2- 172.21.15.131:0/2803974296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d406ee80 0x7fe2d40c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d2ffd700 1 -- 172.21.15.131:0/2803974296 >> v1:172.21.15.131:6789/0 conn(0x7fe2d4156360 legacy=0x7fe2d406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.797+0000 7fe2d2ffd700 1 -- 172.21.15.131:0/2803974296 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d40acc50 con 0x7fe2d4156730 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.797+0000 7fe2d3fff700 1 --2- 172.21.15.131:0/2803974296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d406ee80 0x7fe2d40c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.796+0000 7fe2d27fc700 1 -- 172.21.15.131:0/2803974296 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374707717 0 0) 0x7fe2d40acfc0 con 0x7fe2d4156360 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.797+0000 7fe2d2ffd700 1 --2- 172.21.15.131:0/2803974296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156730 0x7fe2d40c8490 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80099b0 tx=0x7fe2c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c655c8813baf86 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.797+0000 7fe2d27fc700 1 -- 172.21.15.131:0/2803974296 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800b860 con 0x7fe2d4156730 2023-12-26T21:55:50.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.797+0000 7fe2d27fc700 1 -- 172.21.15.131:0/2803974296 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c800b9c0 con 0x7fe2d4156730 2023-12-26T21:55:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.797+0000 7fe2d27fc700 1 -- 172.21.15.131:0/2803974296 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800bd60 con 0x7fe2d4156730 2023-12-26T21:55:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.798+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2803974296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156730 msgr2=0x7fe2d40c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.798+0000 7fe2d9dff700 1 --2- 172.21.15.131:0/2803974296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156730 0x7fe2d40c8490 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80099b0 tx=0x7fe2c8009250 comp rx=0 tx=0).stop 2023-12-26T21:55:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.798+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2803974296 shutdown_connections 2023-12-26T21:55:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.798+0000 7fe2d9dff700 1 --2- 172.21.15.131:0/2803974296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d406ee80 0x7fe2d40c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.798+0000 7fe2d9dff700 1 --2- 172.21.15.131:0/2803974296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156730 0x7fe2d40c8490 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.798+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2803974296 >> 172.21.15.131:0/2803974296 conn(0x7fe2d40675b0 msgr2=0x7fe2d40679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.798+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2803974296 shutdown_connections 2023-12-26T21:55:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.798+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2803974296 wait complete. 2023-12-26T21:55:50.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.800+0000 7fe2d9dff700 1 Processor -- start 2023-12-26T21:55:50.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.800+0000 7fe2d9dff700 1 -- start start 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.801+0000 7fe2d9dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d406ee80 0x7fe2d40ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.801+0000 7fe2d9dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156360 0x7fe2d414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.801+0000 7fe2d9dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4156730 0x7fe2d416d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.801+0000 7fe2d9dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40b12f0 con 0x7fe2d4156360 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.801+0000 7fe2d9dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40b15d0 con 0x7fe2d406ee80 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.801+0000 7fe2d9dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40b18b0 con 0x7fe2d4156730 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.801+0000 7fe2d2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156360 0x7fe2d414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156360 0x7fe2d414cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60858/0 (socket says 172.21.15.131:60858) 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d2ffd700 1 -- 172.21.15.131:0/2290150043 learned_addr learned my addr 172.21.15.131:0/2290150043 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d3fff700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4156730 0x7fe2d416d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d37fe700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d406ee80 0x7fe2d40ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d2ffd700 1 -- 172.21.15.131:0/2290150043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4156730 msgr2=0x7fe2d416d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d2ffd700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4156730 0x7fe2d416d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d2ffd700 1 -- 172.21.15.131:0/2290150043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d406ee80 msgr2=0x7fe2d40ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d2ffd700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d406ee80 0x7fe2d40ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d2ffd700 1 -- 172.21.15.131:0/2290150043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80094f0 con 0x7fe2d4156360 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2d2ffd700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156360 0x7fe2d414cdb0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40690d0 tx=0x7fe2c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.802+0000 7fe2bb7fe700 1 -- 172.21.15.131:0/2290150043 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8011e00 con 0x7fe2d4156360 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.803+0000 7fe2bb7fe700 1 -- 172.21.15.131:0/2290150043 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c801a450 con 0x7fe2d4156360 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.803+0000 7fe2bb7fe700 1 -- 172.21.15.131:0/2290150043 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c801a810 con 0x7fe2d4156360 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.803+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d40b1b90 con 0x7fe2d4156360 2023-12-26T21:55:50.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.803+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d40b0dd0 con 0x7fe2d4156360 2023-12-26T21:55:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.804+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d40717f0 con 0x7fe2d4156360 2023-12-26T21:55:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.805+0000 7fe2bb7fe700 1 -- 172.21.15.131:0/2290150043 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 56) v1 ==== 84597+0+0 (secure 0 0 0) 0x7fe2c801ac40 con 0x7fe2d4156360 2023-12-26T21:55:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.806+0000 7fe2bb7fe700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2c0066dd0 0x7fe2c0069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:55:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.806+0000 7fe2bb7fe700 1 -- 172.21.15.131:0/2290150043 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fe2c8085ed0 con 0x7fe2d4156360 2023-12-26T21:55:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.806+0000 7fe2d37fe700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2c0066dd0 0x7fe2c0069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:55:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.806+0000 7fe2d37fe700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2c0066dd0 0x7fe2c0069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4007fd0 tx=0x7fe2c4007d50 comp rx=0 tx=0).ready entity=mgr.7848 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:55:50.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:50.810+0000 7fe2bb7fe700 1 -- 172.21.15.131:0/2290150043 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2c8053090 con 0x7fe2d4156360 2023-12-26T21:55:51.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.027+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2d40b1740 con 0x7fe2c0066dd0 2023-12-26T21:55:51.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.036+0000 7fe2bb7fe700 1 -- 172.21.15.131:0/2290150043 <== mgr.7848 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fe2d40b1740 con 0x7fe2c0066dd0 2023-12-26T21:55:51.036 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:55:51.036 INFO:teuthology.orchestra.run.smithi131.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-12-26T21:55:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2c0066dd0 msgr2=0x7fe2c0069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2c0066dd0 0x7fe2c0069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4007fd0 tx=0x7fe2c4007d50 comp rx=0 tx=0).stop 2023-12-26T21:55:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156360 msgr2=0x7fe2d414cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:55:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156360 0x7fe2d414cdb0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40690d0 tx=0x7fe2c8017000 comp rx=0 tx=0).stop 2023-12-26T21:55:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 shutdown_connections 2023-12-26T21:55:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4156730 0x7fe2d416d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d406ee80 0x7fe2d40ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe2c0066dd0 0x7fe2c0069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 --2- 172.21.15.131:0/2290150043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2d4156360 0x7fe2d414cdb0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:55:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 >> 172.21.15.131:0/2290150043 conn(0x7fe2d40675b0 msgr2=0x7fe2d40ad450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:55:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 shutdown_connections 2023-12-26T21:55:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:55:51.039+0000 7fe2d9dff700 1 -- 172.21.15.131:0/2290150043 wait complete. 2023-12-26T21:55:51.057 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-12-26T21:55:51.057 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-12-26T21:55:51.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:51.057 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:55:51.102 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:55:51.102 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:55:51.102 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:55:52.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:52.103 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:55:52.153 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:55:52.153 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:55:52.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:55:53.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:53.155 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:55:53.205 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:55:53.205 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:55:53.205 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:55:54.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:54.206 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:55:54.256 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:55:54.256 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:55:54.256 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:55:55.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:55.257 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:55:55.308 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:55:55.308 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:55:55.308 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:55:56.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:56.309 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:55:56.360 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:55:56.360 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:55:56.360 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:55:57.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:57.361 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:55:57.411 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:55:57.411 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:55:57.412 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:55:58.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:58.412 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:55:58.462 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:55:58.462 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:55:58.462 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:55:59.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:55:59.463 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:55:59.513 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:55:59.513 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:55:59.514 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:56:00.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:56:00.514 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:56:00.563 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T21:56:00.564 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T21:56:00.564 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:56:00.587 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:00.586+0000 7f4ced256700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:00.587 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:00.586+0000 7f4ced256700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:00.587 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:00.586+0000 7f4ced256700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:00.587 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:00.586+0000 7f4ced256700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:00.587 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:00.586+0000 7f4ced256700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:01.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:56:01.565 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:56:01.617 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_56_00_UTC 2023-12-26T21:56:01.618 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-21_56_00_UTC'} 2023-12-26T21:56:01.618 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-26-21_56_00_UTC'} 2023-12-26T21:56:01.618 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T21:56:01.618 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-26-21_56_00_UTC'] 2023-12-26T21:56:01.618 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703627750.5989022, now = 1703627761.6185427, timo = 10 2023-12-26T21:56:02.619 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-12-26T21:56:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.821+0000 7f098ddff700 1 Processor -- start 2023-12-26T21:56:02.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.822+0000 7f098ddff700 1 -- start start 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f098ddff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0988142f00 0x7f0988069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f098ddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09880cc590 0x7f09880cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f098ddff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09880cb340 0x7f09880c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f098ddff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09880cb7c0 con 0x7f09880cb340 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f098ddff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0988069640 con 0x7f0988142f00 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f098ddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0988068750 con 0x7f09880cc590 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f0987fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09880cb340 0x7f09880c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f0987fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09880cb340 0x7f09880c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48688/0 (socket says 172.21.15.131:48688) 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f0987fff700 1 -- 172.21.15.131:0/1907188041 learned_addr learned my addr 172.21.15.131:0/1907188041 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f0986ffd700 1 --2- 172.21.15.131:0/1907188041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09880cc590 0x7f09880cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.823+0000 7f09877fe700 1 --2- 172.21.15.131:0/1907188041 >> v2:172.21.15.190:3300/0 conn(0x7f0988142f00 0x7f0988069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.824+0000 7f0987fff700 1 -- 172.21.15.131:0/1907188041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09880cc590 msgr2=0x7f09880cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.824+0000 7f0987fff700 1 --2- 172.21.15.131:0/1907188041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09880cc590 0x7f09880cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.824+0000 7f0987fff700 1 -- 172.21.15.131:0/1907188041 >> v2:172.21.15.190:3300/0 conn(0x7f0988142f00 msgr2=0x7f0988069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.824+0000 7f0987fff700 1 --2- 172.21.15.131:0/1907188041 >> v2:172.21.15.190:3300/0 conn(0x7f0988142f00 0x7f0988069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.824+0000 7f0987fff700 1 -- 172.21.15.131:0/1907188041 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0988068a30 con 0x7f09880cb340 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.824+0000 7f0987fff700 1 --2- 172.21.15.131:0/1907188041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 0x7f09880c8a40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f098806f5b0 tx=0x7f0980009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab72d86053eeaa73 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.824+0000 7f09867fc700 1 -- 172.21.15.131:0/1907188041 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098000e070 con 0x7f09880cb340 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.824+0000 7f09867fc700 1 -- 172.21.15.131:0/1907188041 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098000ba50 con 0x7f09880cb340 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.824+0000 7f09867fc700 1 -- 172.21.15.131:0/1907188041 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09800092e0 con 0x7f09880cb340 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.825+0000 7f098ddff700 1 -- 172.21.15.131:0/1907188041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 msgr2=0x7f09880c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.825+0000 7f098ddff700 1 --2- 172.21.15.131:0/1907188041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 0x7f09880c8a40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f098806f5b0 tx=0x7f0980009840 comp rx=0 tx=0).stop 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.825+0000 7f098ddff700 1 -- 172.21.15.131:0/1907188041 shutdown_connections 2023-12-26T21:56:02.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.825+0000 7f098ddff700 1 --2- 172.21.15.131:0/1907188041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 0x7f09880c8a40 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.825+0000 7f098ddff700 1 --2- 172.21.15.131:0/1907188041 >> v2:172.21.15.190:3300/0 conn(0x7f0988142f00 0x7f0988069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.825+0000 7f098ddff700 1 --2- 172.21.15.131:0/1907188041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09880cc590 0x7f09880cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.825+0000 7f098ddff700 1 -- 172.21.15.131:0/1907188041 >> 172.21.15.131:0/1907188041 conn(0x7f0988063cd0 msgr2=0x7f098806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.825+0000 7f098ddff700 1 -- 172.21.15.131:0/1907188041 shutdown_connections 2023-12-26T21:56:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.825+0000 7f098ddff700 1 -- 172.21.15.131:0/1907188041 wait complete. 2023-12-26T21:56:02.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.826+0000 7f098ddff700 1 Processor -- start 2023-12-26T21:56:02.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.826+0000 7f098ddff700 1 -- start start 2023-12-26T21:56:02.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.827+0000 7f098ddff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 0x7f098813fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:02.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.827+0000 7f098ddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09880cc590 0x7f09881583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:02.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.827+0000 7f098ddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0988142f00 0x7f09881588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.827+0000 7f098ddff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0988159200 con 0x7f09880cb340 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.827+0000 7f098ddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09880c5ca0 con 0x7f09880cc590 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.827+0000 7f098ddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09880ca7e0 con 0x7f0988142f00 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f09877fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 0x7f098813fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f09877fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 0x7f098813fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48690/0 (socket says 172.21.15.131:48690) 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f09877fe700 1 -- 172.21.15.131:0/3543919299 learned_addr learned my addr 172.21.15.131:0/3543919299 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f0986ffd700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09880cc590 0x7f09881583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f0987fff700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0988142f00 0x7f09881588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f09877fe700 1 -- 172.21.15.131:0/3543919299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0988142f00 msgr2=0x7f09881588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f09877fe700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0988142f00 0x7f09881588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f09877fe700 1 -- 172.21.15.131:0/3543919299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09880cc590 msgr2=0x7f09881583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f09877fe700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09880cc590 0x7f09881583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f09877fe700 1 -- 172.21.15.131:0/3543919299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0980009ae0 con 0x7f09880cb340 2023-12-26T21:56:02.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.828+0000 7f09877fe700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 0x7f098813fed0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f097800d860 tx=0x7f0978008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:02.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.829+0000 7f096f7fe700 1 -- 172.21.15.131:0/3543919299 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09780125f0 con 0x7f09880cb340 2023-12-26T21:56:02.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.829+0000 7f096f7fe700 1 -- 172.21.15.131:0/3543919299 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978012750 con 0x7f09880cb340 2023-12-26T21:56:02.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.829+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0988140410 con 0x7f09880cb340 2023-12-26T21:56:02.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.829+0000 7f096f7fe700 1 -- 172.21.15.131:0/3543919299 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978012b10 con 0x7f09880cb340 2023-12-26T21:56:02.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.829+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09880c5650 con 0x7f09880cb340 2023-12-26T21:56:02.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.830+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09880c54b0 con 0x7f09880cb340 2023-12-26T21:56:02.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.830+0000 7f096f7fe700 1 -- 172.21.15.131:0/3543919299 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 58) v1 ==== 84975+0+0 (secure 0 0 0) 0x7f097801e460 con 0x7f09880cb340 2023-12-26T21:56:02.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.831+0000 7f096f7fe700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09740671c0 0x7f0974069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:02.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.831+0000 7f096f7fe700 1 -- 172.21.15.131:0/3543919299 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f0978085150 con 0x7f09880cb340 2023-12-26T21:56:02.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.832+0000 7f0986ffd700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09740671c0 0x7f0974069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:02.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.832+0000 7f0986ffd700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09740671c0 0x7f0974069670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f09880c4ef0 tx=0x7f097c008040 comp rx=0 tx=0).ready entity=mgr.7848 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:02.836+0000 7f096f7fe700 1 -- 172.21.15.131:0/3543919299 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09880c54b0 con 0x7f09880cb340 2023-12-26T21:56:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.054+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f09880c54b0 con 0x7f09740671c0 2023-12-26T21:56:03.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.128+0000 7f096f7fe700 1 -- 172.21.15.131:0/3543919299 <== mgr.7848 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f09880c54b0 con 0x7f09740671c0 2023-12-26T21:56:03.129 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path snap_test_dir1 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09740671c0 msgr2=0x7f0974069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09740671c0 0x7f0974069670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f09880c4ef0 tx=0x7f097c008040 comp rx=0 tx=0).stop 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 msgr2=0x7f098813fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 0x7f098813fed0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f097800d860 tx=0x7f0978008040 comp rx=0 tx=0).stop 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 shutdown_connections 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0988142f00 0x7f09881588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09740671c0 0x7f0974069670 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09880cb340 0x7f098813fed0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 --2- 172.21.15.131:0/3543919299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09880cc590 0x7f09881583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 >> 172.21.15.131:0/3543919299 conn(0x7f0988063cd0 msgr2=0x7f09880b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:03.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 shutdown_connections 2023-12-26T21:56:03.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.131+0000 7f098ddff700 1 -- 172.21.15.131:0/3543919299 wait complete. 2023-12-26T21:56:03.150 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-26T21:56:03.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:56:03.150 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T21:56:03.193 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_56_00_UTC 2023-12-26T21:56:03.194 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-21_56_00_UTC 2023-12-26T21:56:03.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:56:03.194 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-21_56_00_UTC) 2023-12-26T21:56:03.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:56:03.284 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-26T21:56:03.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T21:56:03.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.564+0000 7f52751ff700 1 Processor -- start 2023-12-26T21:56:03.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.565+0000 7f52751ff700 1 -- start start 2023-12-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f52751ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52700b0750 0x7f52700b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f52751ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 0x7f52700c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f52751ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5270156940 con 0x7f52700cb320 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f52751ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52700c8cc0 con 0x7f52700b0750 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f52751ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52700ad700 con 0x7f52700c9400 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f526e59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52700b0750 0x7f52700b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f526e59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52700b0750 0x7f52700b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48694/0 (socket says 172.21.15.131:48694) 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f526e59c700 1 -- 172.21.15.131:0/2353463261 learned_addr learned my addr 172.21.15.131:0/2353463261 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f526f59e700 1 --2- 172.21.15.131:0/2353463261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 0x7f52700c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f526e59c700 1 -- 172.21.15.131:0/2353463261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 msgr2=0x7f52700c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.566+0000 7f526e59c700 1 --2- 172.21.15.131:0/2353463261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 0x7f52700c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.567+0000 7f526e59c700 1 -- 172.21.15.131:0/2353463261 >> v1:172.21.15.190:6789/0 conn(0x7f52700cb320 legacy=0x7f5270069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.567+0000 7f526e59c700 1 -- 172.21.15.131:0/2353463261 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52700ae500 con 0x7f52700b0750 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.567+0000 7f526e59c700 1 --2- 172.21.15.131:0/2353463261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 0x7f52700b0bf0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f52640099b0 tx=0x7f5264009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a35d660606f6f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.567+0000 7f526dd9b700 1 -- 172.21.15.131:0/2353463261 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264009ea0 con 0x7f52700b0750 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.567+0000 7f526dd9b700 1 -- 172.21.15.131:0/2353463261 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526400b860 con 0x7f52700b0750 2023-12-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.567+0000 7f526dd9b700 1 -- 172.21.15.131:0/2353463261 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400bc00 con 0x7f52700b0750 2023-12-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.568+0000 7f52751ff700 1 -- 172.21.15.131:0/2353463261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 msgr2=0x7f52700b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.568+0000 7f52751ff700 1 --2- 172.21.15.131:0/2353463261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 0x7f52700b0bf0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f52640099b0 tx=0x7f5264009250 comp rx=0 tx=0).stop 2023-12-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.568+0000 7f52751ff700 1 -- 172.21.15.131:0/2353463261 shutdown_connections 2023-12-26T21:56:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.568+0000 7f52751ff700 1 --2- 172.21.15.131:0/2353463261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 0x7f52700c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.568+0000 7f52751ff700 1 --2- 172.21.15.131:0/2353463261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 0x7f52700b0bf0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.568+0000 7f52751ff700 1 -- 172.21.15.131:0/2353463261 >> 172.21.15.131:0/2353463261 conn(0x7f5270068040 msgr2=0x7f5270069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.568+0000 7f52751ff700 1 -- 172.21.15.131:0/2353463261 shutdown_connections 2023-12-26T21:56:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.568+0000 7f52751ff700 1 -- 172.21.15.131:0/2353463261 wait complete. 2023-12-26T21:56:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.569+0000 7f52751ff700 1 Processor -- start 2023-12-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.569+0000 7f52751ff700 1 -- start start 2023-12-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.570+0000 7f52751ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 0x7f527006efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.570+0000 7f52751ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 0x7f5270154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.570+0000 7f52751ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52700cb320 0x7f52700c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.570+0000 7f52751ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527006f510 con 0x7f52700b0750 2023-12-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.570+0000 7f52751ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527006eb40 con 0x7f52700cb320 2023-12-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.570+0000 7f52751ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52701598a0 con 0x7f52700c9400 2023-12-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.570+0000 7f526ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 0x7f527006efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 0x7f527006efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48704/0 (socket says 172.21.15.131:48704) 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526ed9d700 1 -- 172.21.15.131:0/3247840145 learned_addr learned my addr 172.21.15.131:0/3247840145 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526e59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 0x7f5270154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526f59e700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52700cb320 0x7f52700c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526ed9d700 1 -- 172.21.15.131:0/3247840145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 msgr2=0x7f5270154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526ed9d700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 0x7f5270154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526ed9d700 1 -- 172.21.15.131:0/3247840145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52700cb320 msgr2=0x7f52700c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526ed9d700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52700cb320 0x7f52700c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526ed9d700 1 -- 172.21.15.131:0/3247840145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52640094f0 con 0x7f52700b0750 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f526ed9d700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 0x7f527006efd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f526000c4e0 tx=0x7f526000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f5256ffd700 1 -- 172.21.15.131:0/3247840145 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260011590 con 0x7f52700b0750 2023-12-26T21:56:03.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f5256ffd700 1 -- 172.21.15.131:0/3247840145 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5260011710 con 0x7f52700b0750 2023-12-26T21:56:03.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5270159b80 con 0x7f52700b0750 2023-12-26T21:56:03.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.571+0000 7f5256ffd700 1 -- 172.21.15.131:0/3247840145 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260011ad0 con 0x7f52700b0750 2023-12-26T21:56:03.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.572+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5270155480 con 0x7f52700b0750 2023-12-26T21:56:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.572+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52700657e0 con 0x7f52700b0750 2023-12-26T21:56:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.574+0000 7f5256ffd700 1 -- 172.21.15.131:0/3247840145 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 58) v1 ==== 84975+0+0 (secure 0 0 0) 0x7f526001b3b0 con 0x7f52700b0750 2023-12-26T21:56:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.575+0000 7f5256ffd700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f525c0671c0 0x7f525c069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.575+0000 7f5256ffd700 1 -- 172.21.15.131:0/3247840145 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f5260083f10 con 0x7f52700b0750 2023-12-26T21:56:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.575+0000 7f526e59c700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f525c0671c0 0x7f525c069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.576+0000 7f526e59c700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f525c0671c0 0x7f525c069670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52640099b0 tx=0x7f5264017000 comp rx=0 tx=0).ready entity=mgr.7848 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:03.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.579+0000 7f5256ffd700 1 -- 172.21.15.131:0/3247840145 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5260050c60 con 0x7f52700b0750 2023-12-26T21:56:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:03.823+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f5270158860 con 0x7f52700b0750 2023-12-26T21:56:04.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.297+0000 7f5256ffd700 1 -- 172.21.15.131:0/3247840145 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f5260051030 con 0x7f52700b0750 2023-12-26T21:56:04.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.298+0000 7f5256ffd700 1 -- 172.21.15.131:0/3247840145 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v59) v1 ==== 93+0+0 (secure 0 0 0) 0x7f52600546f0 con 0x7f52700b0750 2023-12-26T21:56:04.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f525c0671c0 msgr2=0x7f525c069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:04.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f525c0671c0 0x7f525c069670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52640099b0 tx=0x7f5264017000 comp rx=0 tx=0).stop 2023-12-26T21:56:04.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 msgr2=0x7f527006efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:04.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 0x7f527006efd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f526000c4e0 tx=0x7f526000c7f0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 shutdown_connections 2023-12-26T21:56:04.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52700cb320 0x7f52700c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f525c0671c0 0x7f525c069670 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52700b0750 0x7f527006efd0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 --2- 172.21.15.131:0/3247840145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52700c9400 0x7f5270154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 >> 172.21.15.131:0/3247840145 conn(0x7f5270068040 msgr2=0x7f5270156ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:04.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 shutdown_connections 2023-12-26T21:56:04.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.300+0000 7f52751ff700 1 -- 172.21.15.131:0/3247840145 wait complete. 2023-12-26T21:56:04.320 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:56:04.321 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:56:04.421 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:56:04.519 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:56:04.519 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T21:56:04.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:56:04.519 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T21:56:04.598 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T21:56:04.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:56:04.599 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T21:56:04.642 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:56:04.763 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:04.762+0000 7f26818441c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:56:04.818 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:04.816+0000 7f8d73e371c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:56:04.871 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:04.871+0000 7f26818441c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.873+0000 7f08333ff700 1 Processor -- start 2023-12-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.874+0000 7f08333ff700 1 -- start start 2023-12-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.875+0000 7f08333ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f082c0cb320 0x7f082c069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.875+0000 7f08333ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c06a1e0 0x7f082c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.875+0000 7f08333ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c156940 con 0x7f082c0b0c90 2023-12-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.875+0000 7f08333ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082c0c8cc0 con 0x7f082c0cb320 2023-12-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.875+0000 7f08333ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0ad700 con 0x7f082c06a1e0 2023-12-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.875+0000 7f083119b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f082c0cb320 0x7f082c069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.875+0000 7f083119b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f082c0cb320 0x7f082c069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56972/0 (socket says 172.21.15.131:56972) 2023-12-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.875+0000 7f083119b700 1 -- 172.21.15.131:0/1310196073 learned_addr learned my addr 172.21.15.131:0/1310196073 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.875+0000 7f083099a700 1 --2- 172.21.15.131:0/1310196073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c06a1e0 0x7f082c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.876+0000 7f083119b700 1 -- 172.21.15.131:0/1310196073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c06a1e0 msgr2=0x7f082c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.876+0000 7f083119b700 1 --2- 172.21.15.131:0/1310196073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c06a1e0 0x7f082c0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.876+0000 7f083119b700 1 -- 172.21.15.131:0/1310196073 >> v1:172.21.15.131:6789/0 conn(0x7f082c0b0c90 legacy=0x7f082c0aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.876+0000 7f083119b700 1 -- 172.21.15.131:0/1310196073 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c0ae500 con 0x7f082c0cb320 2023-12-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.876+0000 7f083119b700 1 --2- 172.21.15.131:0/1310196073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c0cb320 0x7f082c069ca0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f08280099b0 tx=0x7f0828009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc92e009ab23ddb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:04.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.876+0000 7f0823fff700 1 -- 172.21.15.131:0/1310196073 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828009ea0 con 0x7f082c0cb320 2023-12-26T21:56:04.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.876+0000 7f0823fff700 1 -- 172.21.15.131:0/1310196073 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082800b860 con 0x7f082c0cb320 2023-12-26T21:56:04.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.876+0000 7f0823fff700 1 -- 172.21.15.131:0/1310196073 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082800bc00 con 0x7f082c0cb320 2023-12-26T21:56:04.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.877+0000 7f08333ff700 1 -- 172.21.15.131:0/1310196073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c0cb320 msgr2=0x7f082c069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:04.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.877+0000 7f08333ff700 1 --2- 172.21.15.131:0/1310196073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c0cb320 0x7f082c069ca0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f08280099b0 tx=0x7f0828009250 comp rx=0 tx=0).stop 2023-12-26T21:56:04.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.877+0000 7f08333ff700 1 -- 172.21.15.131:0/1310196073 shutdown_connections 2023-12-26T21:56:04.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.877+0000 7f08333ff700 1 --2- 172.21.15.131:0/1310196073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c0cb320 0x7f082c069ca0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.877+0000 7f08333ff700 1 --2- 172.21.15.131:0/1310196073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c06a1e0 0x7f082c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.877+0000 7f08333ff700 1 -- 172.21.15.131:0/1310196073 >> 172.21.15.131:0/1310196073 conn(0x7f082c068040 msgr2=0x7f082c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:04.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.877+0000 7f08333ff700 1 -- 172.21.15.131:0/1310196073 shutdown_connections 2023-12-26T21:56:04.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.877+0000 7f08333ff700 1 -- 172.21.15.131:0/1310196073 wait complete. 2023-12-26T21:56:04.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.879+0000 7f08333ff700 1 Processor -- start 2023-12-26T21:56:04.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.879+0000 7f08333ff700 1 -- start start 2023-12-26T21:56:04.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.880+0000 7f08333ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c06a1e0 0x7f082c165430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:04.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.880+0000 7f08333ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f082c0b0c90 0x7f082c157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:04.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.880+0000 7f08333ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c0cb320 0x7f082c144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.880+0000 7f08333ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c165970 con 0x7f082c06a1e0 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.880+0000 7f08333ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0c2030 con 0x7f082c0b0c90 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.880+0000 7f08333ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0c1420 con 0x7f082c0cb320 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083119b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c06a1e0 0x7f082c165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083119b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c06a1e0 0x7f082c165430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56984/0 (socket says 172.21.15.131:56984) 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083119b700 1 -- 172.21.15.131:0/118032338 learned_addr learned my addr 172.21.15.131:0/118032338 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083099a700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f082c0b0c90 0x7f082c157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083199c700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c0cb320 0x7f082c144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083119b700 1 -- 172.21.15.131:0/118032338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c0cb320 msgr2=0x7f082c144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083119b700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c0cb320 0x7f082c144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083119b700 1 -- 172.21.15.131:0/118032338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f082c0b0c90 msgr2=0x7f082c157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083119b700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f082c0b0c90 0x7f082c157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083119b700 1 -- 172.21.15.131:0/118032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280094f0 con 0x7f082c06a1e0 2023-12-26T21:56:04.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.881+0000 7f083119b700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c06a1e0 0x7f082c165430 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f082c0688f0 tx=0x7f0828017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.882+0000 7f08217fa700 1 -- 172.21.15.131:0/118032338 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082800bbe0 con 0x7f082c06a1e0 2023-12-26T21:56:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.882+0000 7f08217fa700 1 -- 172.21.15.131:0/118032338 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0828011420 con 0x7f082c06a1e0 2023-12-26T21:56:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.882+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c0c1700 con 0x7f082c06a1e0 2023-12-26T21:56:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.882+0000 7f08217fa700 1 -- 172.21.15.131:0/118032338 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08280117e0 con 0x7f082c06a1e0 2023-12-26T21:56:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.882+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c0c1cc0 con 0x7f082c06a1e0 2023-12-26T21:56:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.883+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c0b1550 con 0x7f082c06a1e0 2023-12-26T21:56:04.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.884+0000 7f08217fa700 1 -- 172.21.15.131:0/118032338 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f0828011b90 con 0x7f082c06a1e0 2023-12-26T21:56:04.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.885+0000 7f08217fa700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0810067170 0x7f0810069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:04.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.885+0000 7f08217fa700 1 -- 172.21.15.131:0/118032338 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f08280861f0 con 0x7f082c06a1e0 2023-12-26T21:56:04.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.885+0000 7f083099a700 1 -- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0810067170 msgr2=0x7f0810069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:04.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.885+0000 7f083099a700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0810067170 0x7f0810069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:04.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:04.889+0000 7f08217fa700 1 -- 172.21.15.131:0/118032338 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0828052f40 con 0x7f082c06a1e0 2023-12-26T21:56:04.922 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:04.921+0000 7f8d73e371c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:56:04.989 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:04.989+0000 7f26818441c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:56:05.011 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:05.010+0000 7f8d73e371c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:56:05.074 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:05.073+0000 7f26818441c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:56:05.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.086+0000 7f083099a700 1 -- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0810067170 msgr2=0x7f0810069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:05.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.086+0000 7f083099a700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0810067170 0x7f0810069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:05.091 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:05.090+0000 7f8d73e371c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:56:05.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.163+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f082c0657e0 con 0x7f082c06a1e0 2023-12-26T21:56:05.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.164+0000 7f08217fa700 1 -- 172.21.15.131:0/118032338 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f0828056560 con 0x7f082c06a1e0 2023-12-26T21:56:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:05.165 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6623,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3234851896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3234851896},{"type":"v1","addr":"172.21.15.131:6834","nonce":3234851896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2363679591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2363679591},{"type":"v1","addr":"172.21.15.190:6834","nonce":2363679591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3018492544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3018492544},{"type":"v1","addr":"172.21.15.190:6835","nonce":3018492544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:55:28.396211+0000","modified":"2023-12-26T21:55:31.950235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1932798418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1932798418},{"type":"v1","addr":"172.21.15.131:6835","nonce":1932798418}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-12-26T21:56:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0810067170 msgr2=0x7f0810069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0810067170 0x7f0810069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c06a1e0 msgr2=0x7f082c165430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c06a1e0 0x7f082c165430 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f082c0688f0 tx=0x7f0828017000 comp rx=0 tx=0).stop 2023-12-26T21:56:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 shutdown_connections 2023-12-26T21:56:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f082c0cb320 0x7f082c144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0810067170 0x7f0810069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082c06a1e0 0x7f082c165430 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 --2- 172.21.15.131:0/118032338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f082c0b0c90 0x7f082c157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 >> 172.21.15.131:0/118032338 conn(0x7f082c068040 msgr2=0x7f082c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:05.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 shutdown_connections 2023-12-26T21:56:05.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.166+0000 7f08333ff700 1 -- 172.21.15.131:0/118032338 wait complete. 2023-12-26T21:56:05.168 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 57 2023-12-26T21:56:05.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:56:05.274 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:05.273+0000 7f26818441c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:56:05.279 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:05.279+0000 7f8d73e371c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:56:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.379+0000 7f86dfbff700 1 Processor -- start 2023-12-26T21:56:05.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.379+0000 7f86dfbff700 1 -- start start 2023-12-26T21:56:05.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.380+0000 7f86dfbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f86d806dd10 0x7f86d80cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.380+0000 7f86dfbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d80caeb0 con 0x7f86d80cc030 2023-12-26T21:56:05.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.380+0000 7f86dfbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d80ca2a0 con 0x7f86d815d930 2023-12-26T21:56:05.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.380+0000 7f86dfbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d80ca580 con 0x7f86d806dd10 2023-12-26T21:56:05.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.380+0000 7f86dd99b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f86d806dd10 0x7f86d80cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:05.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.380+0000 7f86dd99b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f86d806dd10 0x7f86d80cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56992/0 (socket says 172.21.15.131:56992) 2023-12-26T21:56:05.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.381+0000 7f86dd99b700 1 -- 172.21.15.131:0/4232594998 learned_addr learned my addr 172.21.15.131:0/4232594998 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:05.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.381+0000 7f86dd19a700 1 --1- 172.21.15.131:0/4232594998 >> v1:172.21.15.131:6789/0 conn(0x7f86d80cc030 0x7f86d815d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47142/0 (socket says 172.21.15.131:47142) 2023-12-26T21:56:05.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.381+0000 7f86dd99b700 1 -- 172.21.15.131:0/4232594998 >> v1:172.21.15.190:6789/0 conn(0x7f86d815d930 legacy=0x7f86d80c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.381+0000 7f86dd99b700 1 -- 172.21.15.131:0/4232594998 >> v1:172.21.15.131:6789/0 conn(0x7f86d80cc030 legacy=0x7f86d815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.381+0000 7f86dd99b700 1 -- 172.21.15.131:0/4232594998 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80ca860 con 0x7f86d806dd10 2023-12-26T21:56:05.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.381+0000 7f86dd99b700 1 --2- 172.21.15.131:0/4232594998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d806dd10 0x7f86d80cbaf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f86d40099b0 tx=0x7f86d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e429efece9e91c09 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:05.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.381+0000 7f86dc999700 1 -- 172.21.15.131:0/4232594998 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400b860 con 0x7f86d806dd10 2023-12-26T21:56:05.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.381+0000 7f86dc999700 1 -- 172.21.15.131:0/4232594998 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d400b9c0 con 0x7f86d806dd10 2023-12-26T21:56:05.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.382+0000 7f86dc999700 1 -- 172.21.15.131:0/4232594998 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400bd60 con 0x7f86d806dd10 2023-12-26T21:56:05.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.382+0000 7f86dfbff700 1 -- 172.21.15.131:0/4232594998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d806dd10 msgr2=0x7f86d80cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.382+0000 7f86dfbff700 1 --2- 172.21.15.131:0/4232594998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d806dd10 0x7f86d80cbaf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f86d40099b0 tx=0x7f86d4009250 comp rx=0 tx=0).stop 2023-12-26T21:56:05.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.382+0000 7f86dfbff700 1 -- 172.21.15.131:0/4232594998 shutdown_connections 2023-12-26T21:56:05.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.382+0000 7f86dfbff700 1 --2- 172.21.15.131:0/4232594998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d806dd10 0x7f86d80cbaf0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.382+0000 7f86dfbff700 1 -- 172.21.15.131:0/4232594998 >> 172.21.15.131:0/4232594998 conn(0x7f86d80cc5d0 msgr2=0x7f86d80cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:05.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.382+0000 7f86dfbff700 1 -- 172.21.15.131:0/4232594998 shutdown_connections 2023-12-26T21:56:05.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.382+0000 7f86dfbff700 1 -- 172.21.15.131:0/4232594998 wait complete. 2023-12-26T21:56:05.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.384+0000 7f86dfbff700 1 Processor -- start 2023-12-26T21:56:05.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.385+0000 7f86dfbff700 1 -- start start 2023-12-26T21:56:05.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dfbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d806dd10 0x7f86d80af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dfbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d80cc030 0x7f86d80bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dfbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d815d930 0x7f86d8141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dfbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d814f640 con 0x7f86d815d930 2023-12-26T21:56:05.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dfbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8098920 con 0x7f86d806dd10 2023-12-26T21:56:05.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dfbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8097d10 con 0x7f86d80cc030 2023-12-26T21:56:05.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86de19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d815d930 0x7f86d8141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:05.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dd99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d806dd10 0x7f86d80af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:05.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dd99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d806dd10 0x7f86d80af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:52368/0 (socket says 172.21.15.131:52368) 2023-12-26T21:56:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dd99b700 1 -- 172.21.15.131:0/4022703907 learned_addr learned my addr 172.21.15.131:0/4022703907 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86dd19a700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d80cc030 0x7f86d80bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86de19c700 1 -- 172.21.15.131:0/4022703907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d80cc030 msgr2=0x7f86d80bfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.386+0000 7f86de19c700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d80cc030 0x7f86d80bfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.387+0000 7f86de19c700 1 -- 172.21.15.131:0/4022703907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d806dd10 msgr2=0x7f86d80af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.387+0000 7f86de19c700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d806dd10 0x7f86d80af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.387+0000 7f86de19c700 1 -- 172.21.15.131:0/4022703907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d40094f0 con 0x7f86d815d930 2023-12-26T21:56:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.387+0000 7f86de19c700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d815d930 0x7f86d8141800 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f86d000c860 tx=0x7f86d0001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:05.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.387+0000 7f86cdffb700 1 -- 172.21.15.131:0/4022703907 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0012630 con 0x7f86d815d930 2023-12-26T21:56:05.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.387+0000 7f86cdffb700 1 -- 172.21.15.131:0/4022703907 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d00127b0 con 0x7f86d815d930 2023-12-26T21:56:05.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.387+0000 7f86cdffb700 1 -- 172.21.15.131:0/4022703907 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0012b70 con 0x7f86d815d930 2023-12-26T21:56:05.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.387+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d8097ff0 con 0x7f86d815d930 2023-12-26T21:56:05.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.387+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d80985b0 con 0x7f86d815d930 2023-12-26T21:56:05.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.388+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d80657e0 con 0x7f86d815d930 2023-12-26T21:56:05.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.389+0000 7f86cdffb700 1 -- 172.21.15.131:0/4022703907 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f86d0003030 con 0x7f86d815d930 2023-12-26T21:56:05.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.390+0000 7f86cdffb700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f86bc067170 0x7f86bc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.390+0000 7f86cdffb700 1 -- 172.21.15.131:0/4022703907 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f86d00848d0 con 0x7f86d815d930 2023-12-26T21:56:05.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.390+0000 7f86dd99b700 1 -- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f86bc067170 msgr2=0x7f86bc069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:05.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.391+0000 7f86dd99b700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f86bc067170 0x7f86bc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:05.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.394+0000 7f86cdffb700 1 -- 172.21.15.131:0/4022703907 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86d0051620 con 0x7f86d815d930 2023-12-26T21:56:05.451 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:05.450+0000 7f8d73e371c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:56:05.452 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:05.452+0000 7f26818441c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:56:05.534 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:05.534+0000 7f8d73e371c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:56:05.538 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:05.538+0000 7f26818441c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:56:05.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.592+0000 7f86dd99b700 1 -- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f86bc067170 msgr2=0x7f86bc069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:05.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.592+0000 7f86dd99b700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f86bc067170 0x7f86bc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:05.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.596+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86d806e0e0 con 0x7f86d815d930 2023-12-26T21:56:05.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.597+0000 7f86cdffb700 1 -- 172.21.15.131:0/4022703907 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16380 (secure 0 0 0) 0x7f86d0054c40 con 0x7f86d815d930 2023-12-26T21:56:05.598 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:05.598 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":125,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:55:49.487110+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:55:26.570089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-26T21:55:27.376474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2993830584":"2023-12-27T21:55:49.487069+0000","172.21.15.131:6837/39288":"2023-12-27T21:55:49.487069+0000","172.21.15.131:6836/39288":"2023-12-27T21:55:49.487069+0000","172.21.15.131:0/4249324124":"2023-12-27T21:55:49.487069+0000","172.21.15.131:0/324814667":"2023-12-27T21:55:49.487069+0000","172.21.15.131:0/1631075145":"2023-12-27T21:55:49.487069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:56:05.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f86bc067170 msgr2=0x7f86bc069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f86bc067170 0x7f86bc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d815d930 msgr2=0x7f86d8141800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d815d930 0x7f86d8141800 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f86d000c860 tx=0x7f86d0001690 comp rx=0 tx=0).stop 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 shutdown_connections 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f86bc067170 0x7f86bc069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86d815d930 0x7f86d8141800 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d806dd10 0x7f86d80af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 --2- 172.21.15.131:0/4022703907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d80cc030 0x7f86d80bfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 >> 172.21.15.131:0/4022703907 conn(0x7f86d80cc5d0 msgr2=0x7f86d8069340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 shutdown_connections 2023-12-26T21:56:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.600+0000 7f86dfbff700 1 -- 172.21.15.131:0/4022703907 wait complete. 2023-12-26T21:56:05.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:55:26.570089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:55:27.376474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T21:56:05.620 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:56:05.680 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:05.680+0000 7f8d73e371c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:56:05.693 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:05.693+0000 7f26818441c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:56:05.771 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:05.771+0000 7f8d73e371c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:56:05.789 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:05.788+0000 7f26818441c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:56:05.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.816+0000 7f5890dff700 1 Processor -- start 2023-12-26T21:56:05.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.817+0000 7f5890dff700 1 -- start start 2023-12-26T21:56:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.818+0000 7f5890dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f588c06a1a0 0x7f588c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.818+0000 7f5890dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c068a80 con 0x7f588c159930 2023-12-26T21:56:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.818+0000 7f5890dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c0cac50 con 0x7f588c159560 2023-12-26T21:56:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.818+0000 7f5890dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c15da00 con 0x7f588c06a1a0 2023-12-26T21:56:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.818+0000 7f5883fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f588c159930 0x7f588c068f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47146/0 (socket says 172.21.15.131:47146) 2023-12-26T21:56:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.818+0000 7f5883fff700 1 -- 172.21.15.131:0/3908428295 learned_addr learned my addr 172.21.15.131:0/3908428295 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.818+0000 7f588ad9d700 1 --2- 172.21.15.131:0/3908428295 >> v2:172.21.15.190:3300/0 conn(0x7f588c06a1a0 0x7f588c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f5889d9b700 1 -- 172.21.15.131:0/3908428295 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036734160 0 0) 0x7f588c068a80 con 0x7f588c159930 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f5889d9b700 1 -- 172.21.15.131:0/3908428295 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5878003570 con 0x7f588c159930 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f5889d9b700 1 -- 172.21.15.131:0/3908428295 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019378638 0 0) 0x7f588c0cac50 con 0x7f588c159560 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f5889d9b700 1 -- 172.21.15.131:0/3908428295 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f588c068a80 con 0x7f588c159560 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f588ad9d700 1 -- 172.21.15.131:0/3908428295 >> v1:172.21.15.190:6789/0 conn(0x7f588c159560 legacy=0x7f588c0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f588ad9d700 1 -- 172.21.15.131:0/3908428295 >> v1:172.21.15.131:6789/0 conn(0x7f588c159930 legacy=0x7f588c068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f588ad9d700 1 -- 172.21.15.131:0/3908428295 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c15dce0 con 0x7f588c06a1a0 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f588ad9d700 1 --2- 172.21.15.131:0/3908428295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f588c06a1a0 0x7f588c15d300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e60 tx=0x7f5884009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b892941c5c2c3041 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f5889d9b700 1 -- 172.21.15.131:0/3908428295 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588400e070 con 0x7f588c06a1a0 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f5889d9b700 1 -- 172.21.15.131:0/3908428295 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588400bab0 con 0x7f588c06a1a0 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.819+0000 7f5889d9b700 1 -- 172.21.15.131:0/3908428295 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588400be70 con 0x7f588c06a1a0 2023-12-26T21:56:05.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.820+0000 7f5890dff700 1 -- 172.21.15.131:0/3908428295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f588c06a1a0 msgr2=0x7f588c15d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.820+0000 7f5890dff700 1 --2- 172.21.15.131:0/3908428295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f588c06a1a0 0x7f588c15d300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e60 tx=0x7f5884009780 comp rx=0 tx=0).stop 2023-12-26T21:56:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.820+0000 7f5890dff700 1 -- 172.21.15.131:0/3908428295 shutdown_connections 2023-12-26T21:56:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.820+0000 7f5890dff700 1 --2- 172.21.15.131:0/3908428295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f588c06a1a0 0x7f588c15d300 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.820+0000 7f5890dff700 1 -- 172.21.15.131:0/3908428295 >> 172.21.15.131:0/3908428295 conn(0x7f588c0ce3d0 msgr2=0x7f588c0ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.820+0000 7f5890dff700 1 -- 172.21.15.131:0/3908428295 shutdown_connections 2023-12-26T21:56:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.820+0000 7f5890dff700 1 -- 172.21.15.131:0/3908428295 wait complete. 2023-12-26T21:56:05.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.822+0000 7f5890dff700 1 Processor -- start 2023-12-26T21:56:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.822+0000 7f5890dff700 1 -- start start 2023-12-26T21:56:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5890dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f588c06a1a0 0x7f588c069060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5890dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c159560 0x7f588c0c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5890dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f588c159930 0x7f588c0c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5890dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0c49f0 con 0x7f588c159560 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5890dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0c53e0 con 0x7f588c159930 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5890dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0c5610 con 0x7f588c06a1a0 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5883fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c159560 0x7f588c0c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5883fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c159560 0x7f588c0c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57010/0 (socket says 172.21.15.131:57010) 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5883fff700 1 -- 172.21.15.131:0/3930801578 learned_addr learned my addr 172.21.15.131:0/3930801578 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f588a59c700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f588c06a1a0 0x7f588c069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f588ad9d700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f588c159930 0x7f588c0c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5883fff700 1 -- 172.21.15.131:0/3930801578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f588c06a1a0 msgr2=0x7f588c069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5883fff700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f588c06a1a0 0x7f588c069060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5883fff700 1 -- 172.21.15.131:0/3930801578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f588c159930 msgr2=0x7f588c0c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5883fff700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f588c159930 0x7f588c0c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.824+0000 7f5883fff700 1 -- 172.21.15.131:0/3930801578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884009a20 con 0x7f588c159560 2023-12-26T21:56:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.825+0000 7f5883fff700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c159560 0x7f588c0c3920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f587400c790 tx=0x7f5874007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.825+0000 7f58827fc700 1 -- 172.21.15.131:0/3930801578 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874005b30 con 0x7f588c159560 2023-12-26T21:56:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.825+0000 7f58827fc700 1 -- 172.21.15.131:0/3930801578 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5874005c90 con 0x7f588c159560 2023-12-26T21:56:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.825+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c0c58b0 con 0x7f588c159560 2023-12-26T21:56:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.825+0000 7f58827fc700 1 -- 172.21.15.131:0/3930801578 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58740072e0 con 0x7f588c159560 2023-12-26T21:56:05.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.825+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c0c5e70 con 0x7f588c159560 2023-12-26T21:56:05.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.826+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c0657e0 con 0x7f588c159560 2023-12-26T21:56:05.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.827+0000 7f58827fc700 1 -- 172.21.15.131:0/3930801578 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f587401b3b0 con 0x7f588c159560 2023-12-26T21:56:05.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.828+0000 7f58827fc700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f586c067170 0x7f586c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:05.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.828+0000 7f58827fc700 1 -- 172.21.15.131:0/3930801578 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f5874083e10 con 0x7f588c159560 2023-12-26T21:56:05.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.828+0000 7f588a59c700 1 -- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f586c067170 msgr2=0x7f586c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:05.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.828+0000 7f588a59c700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f586c067170 0x7f586c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:05.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:05.832+0000 7f58827fc700 1 -- 172.21.15.131:0/3930801578 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f587404b5a0 con 0x7f588c159560 2023-12-26T21:56:05.954 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:05.953+0000 7f8d73e371c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:56:05.977 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:05.977+0000 7f26818441c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:56:06.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.029+0000 7f588a59c700 1 -- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f586c067170 msgr2=0x7f586c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:06.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.029+0000 7f588a59c700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f586c067170 0x7f586c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:06.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.068+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f588c0ccd70 con 0x7f588c159560 2023-12-26T21:56:06.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.069+0000 7f58827fc700 1 -- 172.21.15.131:0/3930801578 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5874017020 con 0x7f588c159560 2023-12-26T21:56:06.070 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:56:06.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.071+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f586c067170 msgr2=0x7f586c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.071+0000 7f5890dff700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f586c067170 0x7f586c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.071+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c159560 msgr2=0x7f588c0c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.071+0000 7f5890dff700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c159560 0x7f588c0c3920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f587400c790 tx=0x7f5874007ea0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.072+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 shutdown_connections 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.072+0000 7f5890dff700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f588c159930 0x7f588c0c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.072+0000 7f5890dff700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f588c06a1a0 0x7f588c069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.072+0000 7f5890dff700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f586c067170 0x7f586c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.072+0000 7f5890dff700 1 --2- 172.21.15.131:0/3930801578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c159560 0x7f588c0c3920 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.072+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 >> 172.21.15.131:0/3930801578 conn(0x7f588c0ce3d0 msgr2=0x7f588c0cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.072+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 shutdown_connections 2023-12-26T21:56:06.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.072+0000 7f5890dff700 1 -- 172.21.15.131:0/3930801578 wait complete. 2023-12-26T21:56:06.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T21:56:06.102 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:06.102+0000 7f8d73e371c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:56:06.136 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:06.135+0000 7f26818441c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:56:06.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.290+0000 7fefc3dff700 1 Processor -- start 2023-12-26T21:56:06.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.291+0000 7fefc3dff700 1 -- start start 2023-12-26T21:56:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.292+0000 7fefc3dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fefbc159820 0x7fefbc0abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.292+0000 7fefc3dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefbc15a8c0 con 0x7fefbc06e480 2023-12-26T21:56:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.292+0000 7fefc3dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefbc159f90 con 0x7fefbc0aa480 2023-12-26T21:56:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.292+0000 7fefc3dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc15ad50 con 0x7fefbc159820 2023-12-26T21:56:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.292+0000 7fefc239c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fefbc06e480 0x7fefbc0675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47154/0 (socket says 172.21.15.131:47154) 2023-12-26T21:56:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.292+0000 7fefc239c700 1 -- 172.21.15.131:0/849868322 learned_addr learned my addr 172.21.15.131:0/849868322 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:06.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.293+0000 7fefc1b9b700 1 --2- 172.21.15.131:0/849868322 >> v2:172.21.15.190:3300/0 conn(0x7fefbc159820 0x7fefbc0abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.293+0000 7fefc0b99700 1 -- 172.21.15.131:0/849868322 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542070033 0 0) 0x7fefbc15a8c0 con 0x7fefbc06e480 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.293+0000 7fefc0b99700 1 -- 172.21.15.131:0/849868322 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefa8003570 con 0x7fefbc06e480 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.293+0000 7fefc0b99700 1 -- 172.21.15.131:0/849868322 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33007751 0 0) 0x7fefbc159f90 con 0x7fefbc0aa480 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.293+0000 7fefc0b99700 1 -- 172.21.15.131:0/849868322 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefbc15a8c0 con 0x7fefbc0aa480 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.293+0000 7fefc1b9b700 1 -- 172.21.15.131:0/849868322 >> v1:172.21.15.190:6789/0 conn(0x7fefbc0aa480 legacy=0x7fefbc06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.293+0000 7fefc1b9b700 1 -- 172.21.15.131:0/849868322 >> v1:172.21.15.131:6789/0 conn(0x7fefbc06e480 legacy=0x7fefbc0675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.293+0000 7fefc1b9b700 1 -- 172.21.15.131:0/849868322 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc15a270 con 0x7fefbc159820 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.293+0000 7fefc1b9b700 1 --2- 172.21.15.131:0/849868322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefbc159820 0x7fefbc0abdb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fefb80098e0 tx=0x7fefb80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55881cfc602ef4d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.294+0000 7fefc0b99700 1 -- 172.21.15.131:0/849868322 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb8011630 con 0x7fefbc159820 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.294+0000 7fefc0b99700 1 -- 172.21.15.131:0/849868322 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb8011790 con 0x7fefbc159820 2023-12-26T21:56:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.294+0000 7fefc0b99700 1 -- 172.21.15.131:0/849868322 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb8011b30 con 0x7fefbc159820 2023-12-26T21:56:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.295+0000 7fefc3dff700 1 -- 172.21.15.131:0/849868322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefbc159820 msgr2=0x7fefbc0abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.295+0000 7fefc3dff700 1 --2- 172.21.15.131:0/849868322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefbc159820 0x7fefbc0abdb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fefb80098e0 tx=0x7fefb80091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.295+0000 7fefc3dff700 1 -- 172.21.15.131:0/849868322 shutdown_connections 2023-12-26T21:56:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.295+0000 7fefc3dff700 1 --2- 172.21.15.131:0/849868322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefbc159820 0x7fefbc0abdb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.295+0000 7fefc3dff700 1 -- 172.21.15.131:0/849868322 >> 172.21.15.131:0/849868322 conn(0x7fefbc06c290 msgr2=0x7fefbc0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.295+0000 7fefc3dff700 1 -- 172.21.15.131:0/849868322 shutdown_connections 2023-12-26T21:56:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.295+0000 7fefc3dff700 1 -- 172.21.15.131:0/849868322 wait complete. 2023-12-26T21:56:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.296+0000 7fefc3dff700 1 Processor -- start 2023-12-26T21:56:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.297+0000 7fefc3dff700 1 -- start start 2023-12-26T21:56:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc3dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefbc06e480 0x7fefbc14b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc3dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefbc0aa480 0x7fefbc0c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc3dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefbc159820 0x7fefbc0c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc3dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc16b0f0 con 0x7fefbc159820 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc3dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc16b3d0 con 0x7fefbc0aa480 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc3dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc16b6b0 con 0x7fefbc06e480 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc239c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefbc159820 0x7fefbc0c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc1b9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefbc06e480 0x7fefbc14b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc239c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefbc159820 0x7fefbc0c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57018/0 (socket says 172.21.15.131:57018) 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc239c700 1 -- 172.21.15.131:0/2005926708 learned_addr learned my addr 172.21.15.131:0/2005926708 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc139a700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefbc0aa480 0x7fefbc0c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc239c700 1 -- 172.21.15.131:0/2005926708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefbc06e480 msgr2=0x7fefbc14b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc239c700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefbc06e480 0x7fefbc14b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc239c700 1 -- 172.21.15.131:0/2005926708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefbc0aa480 msgr2=0x7fefbc0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc239c700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefbc0aa480 0x7fefbc0c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.298+0000 7fefc239c700 1 -- 172.21.15.131:0/2005926708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb8009420 con 0x7fefbc159820 2023-12-26T21:56:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.299+0000 7fefc239c700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefbc159820 0x7fefbc0c5760 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fefb400cc40 tx=0x7fefb4001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.299+0000 7fefb1ffb700 1 -- 172.21.15.131:0/2005926708 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb400a870 con 0x7fefbc159820 2023-12-26T21:56:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.299+0000 7fefb1ffb700 1 -- 172.21.15.131:0/2005926708 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb400a9d0 con 0x7fefbc159820 2023-12-26T21:56:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.299+0000 7fefb1ffb700 1 -- 172.21.15.131:0/2005926708 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb400ad90 con 0x7fefbc159820 2023-12-26T21:56:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.299+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc16b990 con 0x7fefbc159820 2023-12-26T21:56:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.299+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc14baf0 con 0x7fefbc159820 2023-12-26T21:56:06.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.300+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc16be30 con 0x7fefbc159820 2023-12-26T21:56:06.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.301+0000 7fefb1ffb700 1 -- 172.21.15.131:0/2005926708 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7fefb4007300 con 0x7fefbc159820 2023-12-26T21:56:06.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.302+0000 7fefb1ffb700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fefa0067170 0x7fefa0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.302+0000 7fefb1ffb700 1 -- 172.21.15.131:0/2005926708 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fefb4084ed0 con 0x7fefbc159820 2023-12-26T21:56:06.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.302+0000 7fefc1b9b700 1 -- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fefa0067170 msgr2=0x7fefa0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:06.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.302+0000 7fefc1b9b700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fefa0067170 0x7fefa0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.306+0000 7fefb1ffb700 1 -- 172.21.15.131:0/2005926708 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefb4051c00 con 0x7fefbc159820 2023-12-26T21:56:06.483 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:06.483+0000 7f8d73e371c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:56:06.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.503+0000 7fefc1b9b700 1 -- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fefa0067170 msgr2=0x7fefa0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:06.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.503+0000 7fefc1b9b700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fefa0067170 0x7fefa0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:06.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.504+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fefbc15a550 con 0x7fefbc159820 2023-12-26T21:56:06.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.505+0000 7fefb1ffb700 1 -- 172.21.15.131:0/2005926708 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7fefb4055220 con 0x7fefbc159820 2023-12-26T21:56:06.505 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T21:56:06.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fefa0067170 msgr2=0x7fefa0069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:06.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fefa0067170 0x7fefa0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefbc159820 msgr2=0x7fefbc0c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefbc159820 0x7fefbc0c5760 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fefb400cc40 tx=0x7fefb4001a60 comp rx=0 tx=0).stop 2023-12-26T21:56:06.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 shutdown_connections 2023-12-26T21:56:06.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fefa0067170 0x7fefa0069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefbc159820 0x7fefbc0c5760 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefbc06e480 0x7fefbc14b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 --2- 172.21.15.131:0/2005926708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefbc0aa480 0x7fefbc0c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 >> 172.21.15.131:0/2005926708 conn(0x7fefbc06c290 msgr2=0x7fefbc0adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 shutdown_connections 2023-12-26T21:56:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.507+0000 7fefc3dff700 1 -- 172.21.15.131:0/2005926708 wait complete. 2023-12-26T21:56:06.526 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T21:56:06.540 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:06.540+0000 7f26818441c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:56:06.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.735+0000 7fa240bff700 1 Processor -- start 2023-12-26T21:56:06.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.735+0000 7fa240bff700 1 -- start start 2023-12-26T21:56:06.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.736+0000 7fa240bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c0b5310 0x7fa23c0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.736+0000 7fa240bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23c0afcd0 con 0x7fa23c1596a0 2023-12-26T21:56:06.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.736+0000 7fa240bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23c0b3e70 con 0x7fa23c159a70 2023-12-26T21:56:06.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.736+0000 7fa240bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0b0f20 con 0x7fa23c0b5310 2023-12-26T21:56:06.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.736+0000 7fa23a59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa23c1596a0 0x7fa23c0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47158/0 (socket says 172.21.15.131:47158) 2023-12-26T21:56:06.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.736+0000 7fa23a59c700 1 -- 172.21.15.131:0/2680489684 learned_addr learned my addr 172.21.15.131:0/2680489684 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:06.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.736+0000 7fa23ad9d700 1 --2- 172.21.15.131:0/2680489684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c0b5310 0x7fa23c0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.737+0000 7fa23959a700 1 -- 172.21.15.131:0/2680489684 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927419613 0 0) 0x7fa23c0afcd0 con 0x7fa23c1596a0 2023-12-26T21:56:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.737+0000 7fa23959a700 1 -- 172.21.15.131:0/2680489684 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa224003570 con 0x7fa23c1596a0 2023-12-26T21:56:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.737+0000 7fa23959a700 1 -- 172.21.15.131:0/2680489684 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233574587 0 0) 0x7fa23c0b3e70 con 0x7fa23c159a70 2023-12-26T21:56:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.737+0000 7fa23959a700 1 -- 172.21.15.131:0/2680489684 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa23c0afcd0 con 0x7fa23c159a70 2023-12-26T21:56:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.737+0000 7fa23ad9d700 1 -- 172.21.15.131:0/2680489684 >> v1:172.21.15.190:6789/0 conn(0x7fa23c159a70 legacy=0x7fa23c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.737+0000 7fa23ad9d700 1 -- 172.21.15.131:0/2680489684 >> v1:172.21.15.131:6789/0 conn(0x7fa23c1596a0 legacy=0x7fa23c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.737+0000 7fa23ad9d700 1 -- 172.21.15.131:0/2680489684 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0b3540 con 0x7fa23c0b5310 2023-12-26T21:56:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.737+0000 7fa23ad9d700 1 --2- 172.21.15.131:0/2680489684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c0b5310 0x7fa23c0b0870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa234009e90 tx=0x7fa234009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6766e195092e2661 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.738+0000 7fa23959a700 1 -- 172.21.15.131:0/2680489684 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23400e070 con 0x7fa23c0b5310 2023-12-26T21:56:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.738+0000 7fa23959a700 1 -- 172.21.15.131:0/2680489684 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23400bb80 con 0x7fa23c0b5310 2023-12-26T21:56:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.738+0000 7fa23959a700 1 -- 172.21.15.131:0/2680489684 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234012430 con 0x7fa23c0b5310 2023-12-26T21:56:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.738+0000 7fa240bff700 1 -- 172.21.15.131:0/2680489684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c0b5310 msgr2=0x7fa23c0b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.738+0000 7fa240bff700 1 --2- 172.21.15.131:0/2680489684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c0b5310 0x7fa23c0b0870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa234009e90 tx=0x7fa234009780 comp rx=0 tx=0).stop 2023-12-26T21:56:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.738+0000 7fa240bff700 1 -- 172.21.15.131:0/2680489684 shutdown_connections 2023-12-26T21:56:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.738+0000 7fa240bff700 1 --2- 172.21.15.131:0/2680489684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c0b5310 0x7fa23c0b0870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.738+0000 7fa240bff700 1 -- 172.21.15.131:0/2680489684 >> 172.21.15.131:0/2680489684 conn(0x7fa23c06ece0 msgr2=0x7fa23c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.739+0000 7fa240bff700 1 -- 172.21.15.131:0/2680489684 shutdown_connections 2023-12-26T21:56:06.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.739+0000 7fa240bff700 1 -- 172.21.15.131:0/2680489684 wait complete. 2023-12-26T21:56:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.740+0000 7fa240bff700 1 Processor -- start 2023-12-26T21:56:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.740+0000 7fa240bff700 1 -- start start 2023-12-26T21:56:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa240bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa23c0b5310 0x7fa23c0c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa240bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c1596a0 0x7fa23c16df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa240bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa23c159a70 0x7fa23c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa240bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0c8890 con 0x7fa23c159a70 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa240bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c16e4d0 con 0x7fa23c0b5310 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa240bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0c7c80 con 0x7fa23c1596a0 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa23ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa23c159a70 0x7fa23c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa23ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa23c159a70 0x7fa23c155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57034/0 (socket says 172.21.15.131:57034) 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa23ad9d700 1 -- 172.21.15.131:0/3786506691 learned_addr learned my addr 172.21.15.131:0/3786506691 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa23a59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa23c0b5310 0x7fa23c0c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.741+0000 7fa239d9b700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c1596a0 0x7fa23c16df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa23ad9d700 1 -- 172.21.15.131:0/3786506691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c1596a0 msgr2=0x7fa23c16df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa23ad9d700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c1596a0 0x7fa23c16df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa23ad9d700 1 -- 172.21.15.131:0/3786506691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa23c0b5310 msgr2=0x7fa23c0c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa23ad9d700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa23c0b5310 0x7fa23c0c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa23ad9d700 1 -- 172.21.15.131:0/3786506691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa234009a20 con 0x7fa23c159a70 2023-12-26T21:56:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa239d9b700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c1596a0 0x7fa23c16df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa23ad9d700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa23c159a70 0x7fa23c155aa0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa23c089710 tx=0x7fa234019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa22a7fc700 1 -- 172.21.15.131:0/3786506691 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23401b420 con 0x7fa23c159a70 2023-12-26T21:56:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa22a7fc700 1 -- 172.21.15.131:0/3786506691 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23401b5a0 con 0x7fa23c159a70 2023-12-26T21:56:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.742+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23c0c7f60 con 0x7fa23c159a70 2023-12-26T21:56:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.743+0000 7fa22a7fc700 1 -- 172.21.15.131:0/3786506691 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23401b960 con 0x7fa23c159a70 2023-12-26T21:56:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.743+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c0c8520 con 0x7fa23c159a70 2023-12-26T21:56:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.744+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23c0657e0 con 0x7fa23c159a70 2023-12-26T21:56:06.748 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:06.747+0000 7f8d73e371c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:56:06.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.745+0000 7fa22a7fc700 1 -- 172.21.15.131:0/3786506691 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7fa23401bd50 con 0x7fa23c159a70 2023-12-26T21:56:06.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.746+0000 7fa22a7fc700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa21c067170 0x7fa21c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:06.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.746+0000 7fa22a7fc700 1 -- 172.21.15.131:0/3786506691 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fa234086900 con 0x7fa23c159a70 2023-12-26T21:56:06.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.746+0000 7fa23a59c700 1 -- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa21c067170 msgr2=0x7fa21c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:06.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.746+0000 7fa23a59c700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa21c067170 0x7fa21c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:06.751 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.750+0000 7fa22a7fc700 1 -- 172.21.15.131:0/3786506691 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2340535c0 con 0x7fa23c159a70 2023-12-26T21:56:06.827 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:06.827+0000 7f26818441c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:56:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.947+0000 7fa23a59c700 1 -- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa21c067170 msgr2=0x7fa21c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.947+0000 7fa23a59c700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa21c067170 0x7fa21c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:06.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.949+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa23c0b3820 con 0x7fa23c159a70 2023-12-26T21:56:06.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.950+0000 7fa22a7fc700 1 -- 172.21.15.131:0/3786506691 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa23402a020 con 0x7fa23c159a70 2023-12-26T21:56:06.952 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa21c067170 msgr2=0x7fa21c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa21c067170 0x7fa21c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa23c159a70 msgr2=0x7fa23c155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa23c159a70 0x7fa23c155aa0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa23c089710 tx=0x7fa234019000 comp rx=0 tx=0).stop 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 shutdown_connections 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa21c067170 0x7fa21c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa23c159a70 0x7fa23c155aa0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa23c0b5310 0x7fa23c0c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 --2- 172.21.15.131:0/3786506691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa23c1596a0 0x7fa23c16df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 >> 172.21.15.131:0/3786506691 conn(0x7fa23c06ece0 msgr2=0x7fa23c0d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:06.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 shutdown_connections 2023-12-26T21:56:06.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:06.952+0000 7fa240bff700 1 -- 172.21.15.131:0/3786506691 wait complete. 2023-12-26T21:56:06.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:56:07.024 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:07.024+0000 7f8d73e371c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:56:07.112 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:07.111+0000 7f8d73e371c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:56:07.120 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:07.119+0000 7f26818441c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:56:07.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.169+0000 7f22c85ff700 1 Processor -- start 2023-12-26T21:56:07.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.169+0000 7f22c85ff700 1 -- start start 2023-12-26T21:56:07.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.171+0000 7f22c85ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f22c0159920 0x7f22c00b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.171+0000 7f22c85ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f22c0159cf0 0x7f22c00b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.171+0000 7f22c85ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c00c8bf0 con 0x7f22c01554d0 2023-12-26T21:56:07.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.171+0000 7f22c85ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c00c8e60 con 0x7f22c0159920 2023-12-26T21:56:07.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.172+0000 7f22c85ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c00c9140 con 0x7f22c0159cf0 2023-12-26T21:56:07.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.172+0000 7f22c639b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f22c0159920 0x7f22c00b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.172+0000 7f22c639b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f22c0159920 0x7f22c00b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57040/0 (socket says 172.21.15.131:57040) 2023-12-26T21:56:07.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.172+0000 7f22c639b700 1 -- 172.21.15.131:0/3445526850 learned_addr learned my addr 172.21.15.131:0/3445526850 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:07.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.172+0000 7f22c5b9a700 1 --2- 172.21.15.131:0/3445526850 >> v2:172.21.15.190:3300/0 conn(0x7f22c0159cf0 0x7f22c00b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.172+0000 7f22c639b700 1 -- 172.21.15.131:0/3445526850 >> v2:172.21.15.190:3300/0 conn(0x7f22c0159cf0 msgr2=0x7f22c00b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:07.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.172+0000 7f22c639b700 1 --2- 172.21.15.131:0/3445526850 >> v2:172.21.15.190:3300/0 conn(0x7f22c0159cf0 0x7f22c00b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.172+0000 7f22c639b700 1 -- 172.21.15.131:0/3445526850 >> v1:172.21.15.131:6789/0 conn(0x7f22c01554d0 legacy=0x7f22c0155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.172+0000 7f22c639b700 1 -- 172.21.15.131:0/3445526850 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c00c9420 con 0x7f22c0159920 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.173+0000 7f22c639b700 1 --2- 172.21.15.131:0/3445526850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 0x7f22c00b00e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f22c00b0b60 tx=0x7f22bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae2ad18c41a3175 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.173+0000 7f22c5399700 1 -- 172.21.15.131:0/3445526850 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc009ea0 con 0x7f22c0159920 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.173+0000 7f22c5399700 1 -- 172.21.15.131:0/3445526850 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc00b820 con 0x7f22c0159920 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.173+0000 7f22c5399700 1 -- 172.21.15.131:0/3445526850 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00bbc0 con 0x7f22c0159920 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.174+0000 7f22c85ff700 1 -- 172.21.15.131:0/3445526850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 msgr2=0x7f22c00b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.174+0000 7f22c85ff700 1 --2- 172.21.15.131:0/3445526850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 0x7f22c00b00e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f22c00b0b60 tx=0x7f22bc009250 comp rx=0 tx=0).stop 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.174+0000 7f22c85ff700 1 -- 172.21.15.131:0/3445526850 shutdown_connections 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.174+0000 7f22c85ff700 1 --2- 172.21.15.131:0/3445526850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 0x7f22c00b00e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.174+0000 7f22c85ff700 1 --2- 172.21.15.131:0/3445526850 >> v2:172.21.15.190:3300/0 conn(0x7f22c0159cf0 0x7f22c00b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.174+0000 7f22c85ff700 1 -- 172.21.15.131:0/3445526850 >> 172.21.15.131:0/3445526850 conn(0x7f22c0068040 msgr2=0x7f22c0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:07.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.174+0000 7f22c85ff700 1 -- 172.21.15.131:0/3445526850 shutdown_connections 2023-12-26T21:56:07.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.174+0000 7f22c85ff700 1 -- 172.21.15.131:0/3445526850 wait complete. 2023-12-26T21:56:07.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.176+0000 7f22c85ff700 1 Processor -- start 2023-12-26T21:56:07.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.176+0000 7f22c85ff700 1 -- start start 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c85ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22c01554d0 0x7f22c013f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c85ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 0x7f22c00caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c85ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22c0159cf0 0x7f22c00cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c85ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c013fef0 con 0x7f22c0159920 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c85ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00c79a0 con 0x7f22c01554d0 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c85ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00c7c80 con 0x7f22c0159cf0 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c5b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 0x7f22c00caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c5b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 0x7f22c00caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57052/0 (socket says 172.21.15.131:57052) 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c5b9a700 1 -- 172.21.15.131:0/626653110 learned_addr learned my addr 172.21.15.131:0/626653110 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c639b700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22c01554d0 0x7f22c013f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.178+0000 7f22c6b9c700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22c0159cf0 0x7f22c00cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22c5b9a700 1 -- 172.21.15.131:0/626653110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22c0159cf0 msgr2=0x7f22c00cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22c5b9a700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22c0159cf0 0x7f22c00cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22c5b9a700 1 -- 172.21.15.131:0/626653110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22c01554d0 msgr2=0x7f22c013f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22c5b9a700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22c01554d0 0x7f22c013f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22c5b9a700 1 -- 172.21.15.131:0/626653110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0094f0 con 0x7f22c0159920 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22c5b9a700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 0x7f22c00caec0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f22b000ec80 tx=0x7f22b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22b67fc700 1 -- 172.21.15.131:0/626653110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b000c850 con 0x7f22c0159920 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22b67fc700 1 -- 172.21.15.131:0/626653110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b000c9b0 con 0x7f22c0159920 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22b67fc700 1 -- 172.21.15.131:0/626653110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b000cd50 con 0x7f22c0159920 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c0140160 con 0x7f22c0159920 2023-12-26T21:56:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.179+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c00c7f60 con 0x7f22c0159920 2023-12-26T21:56:07.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.180+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c00657e0 con 0x7f22c0159920 2023-12-26T21:56:07.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.181+0000 7f22b67fc700 1 -- 172.21.15.131:0/626653110 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f22b0013680 con 0x7f22c0159920 2023-12-26T21:56:07.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.182+0000 7f22b67fc700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f22a4067170 0x7f22a4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.183+0000 7f22c639b700 1 -- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f22a4067170 msgr2=0x7f22a4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:07.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.183+0000 7f22b67fc700 1 -- 172.21.15.131:0/626653110 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f22b00851a0 con 0x7f22c0159920 2023-12-26T21:56:07.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.183+0000 7f22c639b700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f22a4067170 0x7f22a4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:07.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.187+0000 7f22b67fc700 1 -- 172.21.15.131:0/626653110 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22b0051e50 con 0x7f22c0159920 2023-12-26T21:56:07.218 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:07.218+0000 7f26818441c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:56:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.384+0000 7f22c639b700 1 -- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f22a4067170 msgr2=0x7f22a4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.384+0000 7f22c639b700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f22a4067170 0x7f22a4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:07.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.431+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22c01402f0 con 0x7f22c0159920 2023-12-26T21:56:07.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.432+0000 7f22b67fc700 1 -- 172.21.15.131:0/626653110 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f22b0055470 con 0x7f22c0159920 2023-12-26T21:56:07.432 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:07.432 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6623,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3234851896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3234851896},{"type":"v1","addr":"172.21.15.131:6834","nonce":3234851896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2363679591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2363679591},{"type":"v1","addr":"172.21.15.190:6834","nonce":2363679591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3018492544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3018492544},{"type":"v1","addr":"172.21.15.190:6835","nonce":3018492544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:55:28.396211+0000","modified":"2023-12-26T21:55:31.950235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1932798418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1932798418},{"type":"v1","addr":"172.21.15.131:6835","nonce":1932798418}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f22a4067170 msgr2=0x7f22a4069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f22a4067170 0x7f22a4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 msgr2=0x7f22c00caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 0x7f22c00caec0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f22b000ec80 tx=0x7f22b00091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 shutdown_connections 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22c0159cf0 0x7f22c00cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22c01554d0 0x7f22c013f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f22a4067170 0x7f22a4069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 --2- 172.21.15.131:0/626653110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22c0159920 0x7f22c00caec0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 >> 172.21.15.131:0/626653110 conn(0x7f22c0068040 msgr2=0x7f22c0069360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:07.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 shutdown_connections 2023-12-26T21:56:07.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.434+0000 7f22c85ff700 1 -- 172.21.15.131:0/626653110 wait complete. 2023-12-26T21:56:07.436 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 57 2023-12-26T21:56:07.453 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:56:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.645+0000 7fb3a7fff700 1 Processor -- start 2023-12-26T21:56:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.646+0000 7fb3a7fff700 1 -- start start 2023-12-26T21:56:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.647+0000 7fb3a7fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3a006f0e0 0x7fb3a006f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.647+0000 7fb3a7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 0x7fb3a0157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.647+0000 7fb3a7fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a015b5c0 con 0x7fb3a0157f90 2023-12-26T21:56:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.647+0000 7fb3a7fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a0158af0 con 0x7fb3a006f0e0 2023-12-26T21:56:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.647+0000 7fb3a7fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a0158530 con 0x7fb3a015ba50 2023-12-26T21:56:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.647+0000 7fb3a5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3a006f0e0 0x7fb3a006f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.647+0000 7fb3a559a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 0x7fb3a0157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.647+0000 7fb3a5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3a006f0e0 0x7fb3a006f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:52434/0 (socket says 172.21.15.131:52434) 2023-12-26T21:56:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.647+0000 7fb3a5d9b700 1 -- 172.21.15.131:0/544592628 learned_addr learned my addr 172.21.15.131:0/544592628 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.648+0000 7fb3a559a700 1 -- 172.21.15.131:0/544592628 >> v2:172.21.15.190:3300/0 conn(0x7fb3a006f0e0 msgr2=0x7fb3a006f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.648+0000 7fb3a559a700 1 --2- 172.21.15.131:0/544592628 >> v2:172.21.15.190:3300/0 conn(0x7fb3a006f0e0 0x7fb3a006f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.648+0000 7fb3a559a700 1 -- 172.21.15.131:0/544592628 >> v1:172.21.15.190:6789/0 conn(0x7fb3a0157f90 legacy=0x7fb3a00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.648+0000 7fb3a559a700 1 -- 172.21.15.131:0/544592628 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a0158810 con 0x7fb3a015ba50 2023-12-26T21:56:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.648+0000 7fb3a5d9b700 1 --2- 172.21.15.131:0/544592628 >> v2:172.21.15.190:3300/0 conn(0x7fb3a006f0e0 0x7fb3a006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.648+0000 7fb3a559a700 1 --2- 172.21.15.131:0/544592628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 0x7fb3a0157a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb3900098e0 tx=0x7fb3900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ddd66196d7ab04 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.648+0000 7fb3a4d99700 1 -- 172.21.15.131:0/544592628 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39000b640 con 0x7fb3a015ba50 2023-12-26T21:56:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.648+0000 7fb3a4d99700 1 -- 172.21.15.131:0/544592628 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39000b7a0 con 0x7fb3a015ba50 2023-12-26T21:56:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.649+0000 7fb3a4d99700 1 -- 172.21.15.131:0/544592628 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39000bb40 con 0x7fb3a015ba50 2023-12-26T21:56:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.649+0000 7fb3a7fff700 1 -- 172.21.15.131:0/544592628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 msgr2=0x7fb3a0157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.649+0000 7fb3a7fff700 1 --2- 172.21.15.131:0/544592628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 0x7fb3a0157a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb3900098e0 tx=0x7fb3900091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.649+0000 7fb3a7fff700 1 -- 172.21.15.131:0/544592628 shutdown_connections 2023-12-26T21:56:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.649+0000 7fb3a7fff700 1 --2- 172.21.15.131:0/544592628 >> v2:172.21.15.190:3300/0 conn(0x7fb3a006f0e0 0x7fb3a006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.649+0000 7fb3a7fff700 1 --2- 172.21.15.131:0/544592628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 0x7fb3a0157a50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.649+0000 7fb3a7fff700 1 -- 172.21.15.131:0/544592628 >> 172.21.15.131:0/544592628 conn(0x7fb3a00686c0 msgr2=0x7fb3a0068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.649+0000 7fb3a7fff700 1 -- 172.21.15.131:0/544592628 shutdown_connections 2023-12-26T21:56:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.649+0000 7fb3a7fff700 1 -- 172.21.15.131:0/544592628 wait complete. 2023-12-26T21:56:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.651+0000 7fb3a7fff700 1 Processor -- start 2023-12-26T21:56:07.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.651+0000 7fb3a7fff700 1 -- start start 2023-12-26T21:56:07.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3a006f0e0 0x7fb3a0142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a0157f90 0x7fb3a015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 0x7fb3a00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a7fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a015cd60 con 0x7fb3a0157f90 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a7fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a014fcc0 con 0x7fb3a006f0e0 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a7fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a014c160 con 0x7fb3a015ba50 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a559a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a0157f90 0x7fb3a015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a559a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a0157f90 0x7fb3a015dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57068/0 (socket says 172.21.15.131:57068) 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a559a700 1 -- 172.21.15.131:0/3412194012 learned_addr learned my addr 172.21.15.131:0/3412194012 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a5d9b700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3a006f0e0 0x7fb3a0142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a659c700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 0x7fb3a00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:07.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a559a700 1 -- 172.21.15.131:0/3412194012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 msgr2=0x7fb3a00675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a559a700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 0x7fb3a00675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a559a700 1 -- 172.21.15.131:0/3412194012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3a006f0e0 msgr2=0x7fb3a0142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a559a700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3a006f0e0 0x7fb3a0142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.652+0000 7fb3a559a700 1 -- 172.21.15.131:0/3412194012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0094f0 con 0x7fb3a0157f90 2023-12-26T21:56:07.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.653+0000 7fb3a559a700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a0157f90 0x7fb3a015dd30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0158670 tx=0x7fb390017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:07.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.653+0000 7fb395ffb700 1 -- 172.21.15.131:0/3412194012 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39001bc40 con 0x7fb3a0157f90 2023-12-26T21:56:07.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.653+0000 7fb395ffb700 1 -- 172.21.15.131:0/3412194012 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39001bde0 con 0x7fb3a0157f90 2023-12-26T21:56:07.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.653+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb390009420 con 0x7fb3a0157f90 2023-12-26T21:56:07.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.653+0000 7fb395ffb700 1 -- 172.21.15.131:0/3412194012 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390011670 con 0x7fb3a0157f90 2023-12-26T21:56:07.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.653+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a0143330 con 0x7fb3a0157f90 2023-12-26T21:56:07.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.654+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a00657e0 con 0x7fb3a0157f90 2023-12-26T21:56:07.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.655+0000 7fb395ffb700 1 -- 172.21.15.131:0/3412194012 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7fb3900119e0 con 0x7fb3a0157f90 2023-12-26T21:56:07.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.656+0000 7fb395ffb700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb384067170 0x7fb384069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:07.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.656+0000 7fb395ffb700 1 -- 172.21.15.131:0/3412194012 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fb3900860d0 con 0x7fb3a0157f90 2023-12-26T21:56:07.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.656+0000 7fb3a5d9b700 1 -- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb384067170 msgr2=0x7fb384069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:07.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.656+0000 7fb3a5d9b700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb384067170 0x7fb384069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:07.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.660+0000 7fb395ffb700 1 -- 172.21.15.131:0/3412194012 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb390052d80 con 0x7fb3a0157f90 2023-12-26T21:56:07.839 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:07.839+0000 7f8d73e371c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:56:07.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.857+0000 7fb3a5d9b700 1 -- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb384067170 msgr2=0x7fb384069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:07.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.857+0000 7fb3a5d9b700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb384067170 0x7fb384069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:07.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.861+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3a006aaa0 con 0x7fb3a0157f90 2023-12-26T21:56:07.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.862+0000 7fb395ffb700 1 -- 172.21.15.131:0/3412194012 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16380 (secure 0 0 0) 0x7fb3900563a0 con 0x7fb3a0157f90 2023-12-26T21:56:07.863 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:07.863 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":125,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:55:49.487110+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:55:26.570089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-26T21:55:27.376474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2993830584":"2023-12-27T21:55:49.487069+0000","172.21.15.131:6837/39288":"2023-12-27T21:55:49.487069+0000","172.21.15.131:6836/39288":"2023-12-27T21:55:49.487069+0000","172.21.15.131:0/4249324124":"2023-12-27T21:55:49.487069+0000","172.21.15.131:0/324814667":"2023-12-27T21:55:49.487069+0000","172.21.15.131:0/1631075145":"2023-12-27T21:55:49.487069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:56:07.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb384067170 msgr2=0x7fb384069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb384067170 0x7fb384069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a0157f90 msgr2=0x7fb3a015dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a0157f90 0x7fb3a015dd30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0158670 tx=0x7fb390017000 comp rx=0 tx=0).stop 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 shutdown_connections 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3a015ba50 0x7fb3a00675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3a006f0e0 0x7fb3a0142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb384067170 0x7fb384069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 --2- 172.21.15.131:0/3412194012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a0157f90 0x7fb3a015dd30 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 >> 172.21.15.131:0/3412194012 conn(0x7fb3a00686c0 msgr2=0x7fb3a0158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 shutdown_connections 2023-12-26T21:56:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:07.865+0000 7fb3a7fff700 1 -- 172.21.15.131:0/3412194012 wait complete. 2023-12-26T21:56:07.885 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T21:56:07.885 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:56:07.931 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:07.931+0000 7f8d73e371c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:56:07.991 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:07.991+0000 7f26818441c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:56:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.079+0000 7efdf3fff700 1 Processor -- start 2023-12-26T21:56:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.079+0000 7efdf3fff700 1 -- start start 2023-12-26T21:56:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.080+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec0cb320 0x7efdec069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.080+0000 7efdf3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efdec0b0e60 0x7efdec0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.080+0000 7efdf3fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec156940 con 0x7efdec06a1e0 2023-12-26T21:56:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf3fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec0c8cc0 con 0x7efdec0b0e60 2023-12-26T21:56:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf3fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0ad700 con 0x7efdec0cb320 2023-12-26T21:56:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf1d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec0cb320 0x7efdec069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf259c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efdec0b0e60 0x7efdec0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf159a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7efdec06a1e0 0x7efdec0b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:53638/0 (socket says 172.21.15.131:53638) 2023-12-26T21:56:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf1d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec0cb320 0x7efdec069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:54434/0 (socket says 172.21.15.131:54434) 2023-12-26T21:56:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf159a700 1 -- 172.21.15.131:0/1337968251 learned_addr learned my addr 172.21.15.131:0/1337968251 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf259c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efdec0b0e60 0x7efdec0c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:52452/0 (socket says 172.21.15.131:52452) 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf0d99700 1 -- 172.21.15.131:0/1337968251 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603233665 0 0) 0x7efdec156940 con 0x7efdec06a1e0 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.081+0000 7efdf0d99700 1 -- 172.21.15.131:0/1337968251 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd8003570 con 0x7efdec06a1e0 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.082+0000 7efdf1d9b700 1 -- 172.21.15.131:0/1337968251 >> v2:172.21.15.190:3300/0 conn(0x7efdec0b0e60 msgr2=0x7efdec0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.082+0000 7efdf1d9b700 1 --2- 172.21.15.131:0/1337968251 >> v2:172.21.15.190:3300/0 conn(0x7efdec0b0e60 0x7efdec0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.082+0000 7efdf1d9b700 1 -- 172.21.15.131:0/1337968251 >> v1:172.21.15.190:6789/0 conn(0x7efdec06a1e0 legacy=0x7efdec0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.082+0000 7efdf1d9b700 1 -- 172.21.15.131:0/1337968251 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec0ae500 con 0x7efdec0cb320 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.082+0000 7efdf259c700 1 --2- 172.21.15.131:0/1337968251 >> v2:172.21.15.190:3300/0 conn(0x7efdec0b0e60 0x7efdec0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.082+0000 7efdf1d9b700 1 --2- 172.21.15.131:0/1337968251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec0cb320 0x7efdec069ca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efde80098e0 tx=0x7efde80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a164fa6d5358dd3f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.082+0000 7efdf0d99700 1 -- 172.21.15.131:0/1337968251 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8011630 con 0x7efdec0cb320 2023-12-26T21:56:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.082+0000 7efdf0d99700 1 -- 172.21.15.131:0/1337968251 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde8011790 con 0x7efdec0cb320 2023-12-26T21:56:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.082+0000 7efdf0d99700 1 -- 172.21.15.131:0/1337968251 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8011b30 con 0x7efdec0cb320 2023-12-26T21:56:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.083+0000 7efdf3fff700 1 -- 172.21.15.131:0/1337968251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec0cb320 msgr2=0x7efdec069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.083+0000 7efdf3fff700 1 --2- 172.21.15.131:0/1337968251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec0cb320 0x7efdec069ca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efde80098e0 tx=0x7efde80091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.083+0000 7efdf3fff700 1 -- 172.21.15.131:0/1337968251 shutdown_connections 2023-12-26T21:56:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.083+0000 7efdf3fff700 1 --2- 172.21.15.131:0/1337968251 >> v2:172.21.15.190:3300/0 conn(0x7efdec0b0e60 0x7efdec0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.083+0000 7efdf3fff700 1 --2- 172.21.15.131:0/1337968251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec0cb320 0x7efdec069ca0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.083+0000 7efdf3fff700 1 -- 172.21.15.131:0/1337968251 >> 172.21.15.131:0/1337968251 conn(0x7efdec068040 msgr2=0x7efdec069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.083+0000 7efdf3fff700 1 -- 172.21.15.131:0/1337968251 shutdown_connections 2023-12-26T21:56:08.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.083+0000 7efdf3fff700 1 -- 172.21.15.131:0/1337968251 wait complete. 2023-12-26T21:56:08.086 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:08.083+0000 7f8d73e371c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:56:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.084+0000 7efdf3fff700 1 Processor -- start 2023-12-26T21:56:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.084+0000 7efdf3fff700 1 -- start start 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.085+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec06a1e0 0x7efdec1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.085+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdec0b0e60 0x7efdec1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.085+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdec0cb320 0x7efdec15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.085+0000 7efdf3fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec15ba90 con 0x7efdec0cb320 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.085+0000 7efdf3fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec15b790 con 0x7efdec0b0e60 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.085+0000 7efdf3fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec15e850 con 0x7efdec06a1e0 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf1d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec06a1e0 0x7efdec1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf1d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec06a1e0 0x7efdec1532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:54438/0 (socket says 172.21.15.131:54438) 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf1d9b700 1 -- 172.21.15.131:0/1942372627 learned_addr learned my addr 172.21.15.131:0/1942372627 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf259c700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdec0cb320 0x7efdec15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf159a700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdec0b0e60 0x7efdec1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf1d9b700 1 -- 172.21.15.131:0/1942372627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdec0b0e60 msgr2=0x7efdec1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf1d9b700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdec0b0e60 0x7efdec1568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf1d9b700 1 -- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdec0cb320 msgr2=0x7efdec15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf1d9b700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdec0cb320 0x7efdec15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf1d9b700 1 -- 172.21.15.131:0/1942372627 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde400bc30 con 0x7efdec06a1e0 2023-12-26T21:56:08.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.086+0000 7efdf1d9b700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec06a1e0 0x7efdec1532f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efdec159f40 tx=0x7efde8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:08.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.087+0000 7efde1ffb700 1 -- 172.21.15.131:0/1942372627 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8000d10 con 0x7efdec06a1e0 2023-12-26T21:56:08.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.087+0000 7efde1ffb700 1 -- 172.21.15.131:0/1942372627 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde8011900 con 0x7efdec06a1e0 2023-12-26T21:56:08.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.087+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde8009420 con 0x7efdec06a1e0 2023-12-26T21:56:08.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.087+0000 7efde1ffb700 1 -- 172.21.15.131:0/1942372627 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8025420 con 0x7efdec06a1e0 2023-12-26T21:56:08.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.087+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec15ee10 con 0x7efdec06a1e0 2023-12-26T21:56:08.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.088+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec15bd70 con 0x7efdec06a1e0 2023-12-26T21:56:08.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.089+0000 7efde1ffb700 1 -- 172.21.15.131:0/1942372627 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7efde8023070 con 0x7efdec06a1e0 2023-12-26T21:56:08.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.089+0000 7efde1ffb700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efdd0067170 0x7efdd0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.090+0000 7efde1ffb700 1 -- 172.21.15.131:0/1942372627 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6355+0+0 (secure 0 0 0) 0x7efde8085ea0 con 0x7efdec06a1e0 2023-12-26T21:56:08.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.090+0000 7efdf159a700 1 -- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efdd0067170 msgr2=0x7efdd0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:08.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.090+0000 7efdf159a700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efdd0067170 0x7efdd0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:08.091 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:08.091+0000 7f26818441c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:56:08.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.094+0000 7efde1ffb700 1 -- 172.21.15.131:0/1942372627 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efde8052c00 con 0x7efdec06a1e0 2023-12-26T21:56:08.263 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:08.262+0000 7f26818441c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:56:08.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.291+0000 7efdf159a700 1 -- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efdd0067170 msgr2=0x7efdd0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:08.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.291+0000 7efdf159a700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efdd0067170 0x7efdd0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:08.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.358+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdec0c8e80 con 0x7efdec06a1e0 2023-12-26T21:56:08.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.359+0000 7efde1ffb700 1 -- 172.21.15.131:0/1942372627 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7efde802a090 con 0x7efdec06a1e0 2023-12-26T21:56:08.359 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:08.360 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6623,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3234851896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3234851896},{"type":"v1","addr":"172.21.15.131:6834","nonce":3234851896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2363679591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2363679591},{"type":"v1","addr":"172.21.15.190:6834","nonce":2363679591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3018492544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3018492544},{"type":"v1","addr":"172.21.15.190:6835","nonce":3018492544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:55:28.396211+0000","modified":"2023-12-26T21:55:31.950235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1932798418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1932798418},{"type":"v1","addr":"172.21.15.131:6835","nonce":1932798418}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-12-26T21:56:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efdd0067170 msgr2=0x7efdd0069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efdd0067170 0x7efdd0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec06a1e0 msgr2=0x7efdec1532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec06a1e0 0x7efdec1532f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efdec159f40 tx=0x7efde8017000 comp rx=0 tx=0).stop 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 shutdown_connections 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efdd0067170 0x7efdd0069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdec0cb320 0x7efdec15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdec06a1e0 0x7efdec1532f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 --2- 172.21.15.131:0/1942372627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdec0b0e60 0x7efdec1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 >> 172.21.15.131:0/1942372627 conn(0x7efdec068040 msgr2=0x7efdec0c9910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 shutdown_connections 2023-12-26T21:56:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.362+0000 7efdf3fff700 1 -- 172.21.15.131:0/1942372627 wait complete. 2023-12-26T21:56:08.364 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 57 2023-12-26T21:56:08.380 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:56:08.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.574+0000 7f925b5ff700 1 Processor -- start 2023-12-26T21:56:08.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.574+0000 7f925b5ff700 1 -- start start 2023-12-26T21:56:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f925b5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9254159560 0x7f925415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f925b5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159930 0x7f92540b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f925b5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9254068a80 con 0x7f925406a1a0 2023-12-26T21:56:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f925b5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92540cac50 con 0x7f9254159560 2023-12-26T21:56:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f925b5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925415da00 con 0x7f9254159930 2023-12-26T21:56:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f9258b9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159930 0x7f92540b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f925939b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9254159560 0x7f925415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f925939b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9254159560 0x7f925415d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:52480/0 (socket says 172.21.15.131:52480) 2023-12-26T21:56:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f9258b9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159930 0x7f92540b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:54450/0 (socket says 172.21.15.131:54450) 2023-12-26T21:56:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f925939b700 1 -- 172.21.15.131:0/3010640798 learned_addr learned my addr 172.21.15.131:0/3010640798 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.576+0000 7f9259b9c700 1 --1- 172.21.15.131:0/3010640798 >> v1:172.21.15.190:6789/0 conn(0x7f925406a1a0 0x7f9254068f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:53642/0 (socket says 172.21.15.131:53642) 2023-12-26T21:56:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.577+0000 7f925939b700 1 -- 172.21.15.131:0/3010640798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159930 msgr2=0x7f92540b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.577+0000 7f925939b700 1 --2- 172.21.15.131:0/3010640798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159930 0x7f92540b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.577+0000 7f925939b700 1 -- 172.21.15.131:0/3010640798 >> v1:172.21.15.190:6789/0 conn(0x7f925406a1a0 legacy=0x7f9254068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.577+0000 7f925939b700 1 -- 172.21.15.131:0/3010640798 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925415dce0 con 0x7f9254159560 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.577+0000 7f9258b9a700 1 --2- 172.21.15.131:0/3010640798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159930 0x7f92540b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.577+0000 7f925939b700 1 --2- 172.21.15.131:0/3010640798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9254159560 0x7f925415d300 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f92500099b0 tx=0x7f9250009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7119b7c2dab1496 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.578+0000 7f924bfff700 1 -- 172.21.15.131:0/3010640798 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250009ea0 con 0x7f9254159560 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.578+0000 7f924bfff700 1 -- 172.21.15.131:0/3010640798 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925000b860 con 0x7f9254159560 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.578+0000 7f924bfff700 1 -- 172.21.15.131:0/3010640798 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925000bc00 con 0x7f9254159560 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.578+0000 7f925b5ff700 1 -- 172.21.15.131:0/3010640798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9254159560 msgr2=0x7f925415d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.578+0000 7f925b5ff700 1 --2- 172.21.15.131:0/3010640798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9254159560 0x7f925415d300 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f92500099b0 tx=0x7f9250009250 comp rx=0 tx=0).stop 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.578+0000 7f925b5ff700 1 -- 172.21.15.131:0/3010640798 shutdown_connections 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.578+0000 7f925b5ff700 1 --2- 172.21.15.131:0/3010640798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9254159560 0x7f925415d300 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.578+0000 7f925b5ff700 1 --2- 172.21.15.131:0/3010640798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159930 0x7f92540b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.578+0000 7f925b5ff700 1 -- 172.21.15.131:0/3010640798 >> 172.21.15.131:0/3010640798 conn(0x7f92540ce3d0 msgr2=0x7f92540ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.579+0000 7f925b5ff700 1 -- 172.21.15.131:0/3010640798 shutdown_connections 2023-12-26T21:56:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.579+0000 7f925b5ff700 1 -- 172.21.15.131:0/3010640798 wait complete. 2023-12-26T21:56:08.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.580+0000 7f925b5ff700 1 Processor -- start 2023-12-26T21:56:08.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.580+0000 7f925b5ff700 1 -- start start 2023-12-26T21:56:08.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f925b5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f925406a1a0 0x7f92540adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f925b5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159560 0x7f92540ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f925b5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9254159930 0x7f92540af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f925b5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92540aee20 con 0x7f9254159930 2023-12-26T21:56:08.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f925b5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925416d690 con 0x7f925406a1a0 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f925b5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925416d930 con 0x7f9254159560 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f9259b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9254159930 0x7f92540af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f9258b9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159560 0x7f92540ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f925939b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f925406a1a0 0x7f92540adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f9258b9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159560 0x7f92540ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:54452/0 (socket says 172.21.15.131:54452) 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f9259b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9254159930 0x7f92540af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57098/0 (socket says 172.21.15.131:57098) 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.581+0000 7f9258b9a700 1 -- 172.21.15.131:0/4122064764 learned_addr learned my addr 172.21.15.131:0/4122064764 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f9258b9a700 1 -- 172.21.15.131:0/4122064764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f925406a1a0 msgr2=0x7f92540adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f9258b9a700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f925406a1a0 0x7f92540adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f9258b9a700 1 -- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9254159930 msgr2=0x7f92540af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f9258b9a700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9254159930 0x7f92540af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f9258b9a700 1 -- 172.21.15.131:0/4122064764 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9244009420 con 0x7f9254159560 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f9258b9a700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159560 0x7f92540ae8e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f924400ec80 tx=0x7f92440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f92497fa700 1 -- 172.21.15.131:0/4122064764 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924400c920 con 0x7f9254159560 2023-12-26T21:56:08.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f92497fa700 1 -- 172.21.15.131:0/4122064764 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924400ca80 con 0x7f9254159560 2023-12-26T21:56:08.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f92497fa700 1 -- 172.21.15.131:0/4122064764 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924400ce00 con 0x7f9254159560 2023-12-26T21:56:08.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.582+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92500094f0 con 0x7f9254159560 2023-12-26T21:56:08.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.583+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925416def0 con 0x7f9254159560 2023-12-26T21:56:08.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.584+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92540657e0 con 0x7f9254159560 2023-12-26T21:56:08.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.584+0000 7f92497fa700 1 -- 172.21.15.131:0/4122064764 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f9244013680 con 0x7f9254159560 2023-12-26T21:56:08.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.585+0000 7f92497fa700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9238067170 0x7f9238069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:08.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.585+0000 7f92497fa700 1 -- 172.21.15.131:0/4122064764 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f9244085230 con 0x7f9254159560 2023-12-26T21:56:08.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.585+0000 7f925939b700 1 -- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9238067170 msgr2=0x7f9238069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:08.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.585+0000 7f925939b700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9238067170 0x7f9238069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:08.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.590+0000 7f92497fa700 1 -- 172.21.15.131:0/4122064764 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9244051f30 con 0x7f9254159560 2023-12-26T21:56:08.708 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:08.708+0000 7f8d73e371c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:56:08.786 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.786+0000 7f925939b700 1 -- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9238067170 msgr2=0x7f9238069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:08.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.786+0000 7f925939b700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9238067170 0x7f9238069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:08.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.856+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92540c5130 con 0x7f9254159560 2023-12-26T21:56:08.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.856+0000 7f92497fa700 1 -- 172.21.15.131:0/4122064764 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f9244055550 con 0x7f9254159560 2023-12-26T21:56:08.857 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:08.857 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6623,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3234851896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3234851896},{"type":"v1","addr":"172.21.15.131:6834","nonce":3234851896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2363679591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2363679591},{"type":"v1","addr":"172.21.15.190:6834","nonce":2363679591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3018492544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3018492544},{"type":"v1","addr":"172.21.15.190:6835","nonce":3018492544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:55:28.396211+0000","modified":"2023-12-26T21:55:31.950235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7644":{"gid":7644,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1932798418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1932798418},{"type":"v1","addr":"172.21.15.131:6835","nonce":1932798418}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-12-26T21:56:08.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9238067170 msgr2=0x7f9238069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:08.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9238067170 0x7f9238069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159560 msgr2=0x7f92540ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159560 0x7f92540ae8e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f924400ec80 tx=0x7f92440091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 shutdown_connections 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9238067170 0x7f9238069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9254159930 0x7f92540af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f925406a1a0 0x7f92540adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 --2- 172.21.15.131:0/4122064764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9254159560 0x7f92540ae8e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 >> 172.21.15.131:0/4122064764 conn(0x7f92540ce3d0 msgr2=0x7f92540b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 shutdown_connections 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:08.859+0000 7f925b5ff700 1 -- 172.21.15.131:0/4122064764 wait complete. 2023-12-26T21:56:08.860 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 57 2023-12-26T21:56:08.879 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:56:08.899 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:08.899+0000 7f8d73e371c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:56:08.942 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:08.942+0000 7f26818441c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:56:09.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.075+0000 7f212b7ff700 1 Processor -- start 2023-12-26T21:56:09.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.075+0000 7f212b7ff700 1 -- start start 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.076+0000 7f212b7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21240c8820 0x7f21240ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.076+0000 7f212b7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f21240c8bf0 0x7f21240b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.076+0000 7f212b7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2124157a80 con 0x7f2124069d10 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.076+0000 7f212b7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21240cac50 con 0x7f21240c8bf0 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.076+0000 7f212b7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21240caf30 con 0x7f21240c8820 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f2128d9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f21240c8bf0 0x7f21240b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f212959b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21240c8820 0x7f21240ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f2129d9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2124069d10 0x7f2124157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47176/0 (socket says 172.21.15.131:47176) 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f2129d9c700 1 -- 172.21.15.131:0/448284215 learned_addr learned my addr 172.21.15.131:0/448284215 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f2128d9a700 1 -- 172.21.15.131:0/448284215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21240c8820 msgr2=0x7f21240ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:09.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f2128d9a700 1 --2- 172.21.15.131:0/448284215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21240c8820 0x7f21240ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f2128d9a700 1 -- 172.21.15.131:0/448284215 >> v1:172.21.15.131:6789/0 conn(0x7f2124069d10 legacy=0x7f2124157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f2128d9a700 1 -- 172.21.15.131:0/448284215 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240cb210 con 0x7f21240c8bf0 2023-12-26T21:56:09.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f212959b700 1 --2- 172.21.15.131:0/448284215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21240c8820 0x7f21240ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:09.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.077+0000 7f211bfff700 1 -- 172.21.15.131:0/448284215 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836649473 0 0) 0x7f2124157a80 con 0x7f2124069d10 2023-12-26T21:56:09.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f2128d9a700 1 --2- 172.21.15.131:0/448284215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 0x7f21240b12e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f21140098e0 tx=0x7f21140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffc6eccf3ab27afb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:09.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f211bfff700 1 -- 172.21.15.131:0/448284215 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211400b640 con 0x7f21240c8bf0 2023-12-26T21:56:09.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f211bfff700 1 -- 172.21.15.131:0/448284215 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211400b7a0 con 0x7f21240c8bf0 2023-12-26T21:56:09.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f211bfff700 1 -- 172.21.15.131:0/448284215 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211400bb40 con 0x7f21240c8bf0 2023-12-26T21:56:09.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f212b7ff700 1 -- 172.21.15.131:0/448284215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 msgr2=0x7f21240b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f212b7ff700 1 --2- 172.21.15.131:0/448284215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 0x7f21240b12e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f21140098e0 tx=0x7f21140091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f212b7ff700 1 -- 172.21.15.131:0/448284215 shutdown_connections 2023-12-26T21:56:09.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f212b7ff700 1 --2- 172.21.15.131:0/448284215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21240c8820 0x7f21240ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f212b7ff700 1 --2- 172.21.15.131:0/448284215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 0x7f21240b12e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.078+0000 7f212b7ff700 1 -- 172.21.15.131:0/448284215 >> 172.21.15.131:0/448284215 conn(0x7f2124068040 msgr2=0x7f2124069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:09.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.079+0000 7f212b7ff700 1 -- 172.21.15.131:0/448284215 shutdown_connections 2023-12-26T21:56:09.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.079+0000 7f212b7ff700 1 -- 172.21.15.131:0/448284215 wait complete. 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.080+0000 7f212b7ff700 1 Processor -- start 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.080+0000 7f212b7ff700 1 -- start start 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f212b7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2124069d10 0x7f212416d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f212b7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21240c8820 0x7f21240adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f212b7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 0x7f21240ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f212b7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21240ae590 con 0x7f21240c8820 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f212b7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124159fd0 con 0x7f21240c8bf0 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f212b7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212415a2b0 con 0x7f2124069d10 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f2128d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21240c8820 0x7f21240adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f212959b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2124069d10 0x7f212416d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f2129d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 0x7f21240ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f2128d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21240c8820 0x7f21240adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57106/0 (socket says 172.21.15.131:57106) 2023-12-26T21:56:09.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.081+0000 7f2128d9a700 1 -- 172.21.15.131:0/2574428971 learned_addr learned my addr 172.21.15.131:0/2574428971 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:09.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f2129d9c700 1 -- 172.21.15.131:0/2574428971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2124069d10 msgr2=0x7f212416d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f2129d9c700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2124069d10 0x7f212416d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f2129d9c700 1 -- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21240c8820 msgr2=0x7f21240adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f2129d9c700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21240c8820 0x7f21240adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f2129d9c700 1 -- 172.21.15.131:0/2574428971 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2120009420 con 0x7f21240c8bf0 2023-12-26T21:56:09.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f2128d9a700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21240c8820 0x7f21240adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:09.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f212959b700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2124069d10 0x7f212416d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:09.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f2129d9c700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 0x7f21240ae050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f211c00cb90 tx=0x7f211c0018f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:09.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f21197fa700 1 -- 172.21.15.131:0/2574428971 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00a780 con 0x7f21240c8bf0 2023-12-26T21:56:09.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f21197fa700 1 -- 172.21.15.131:0/2574428971 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c00a8e0 con 0x7f21240c8bf0 2023-12-26T21:56:09.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2114009420 con 0x7f21240c8bf0 2023-12-26T21:56:09.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.082+0000 7f21197fa700 1 -- 172.21.15.131:0/2574428971 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00aca0 con 0x7f21240c8bf0 2023-12-26T21:56:09.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.083+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212415a870 con 0x7f21240c8bf0 2023-12-26T21:56:09.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.084+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2124167b70 con 0x7f21240c8bf0 2023-12-26T21:56:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.084+0000 7f21197fa700 1 -- 172.21.15.131:0/2574428971 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f211c007300 con 0x7f21240c8bf0 2023-12-26T21:56:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.085+0000 7f21197fa700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2108067170 0x7f2108069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.085+0000 7f21197fa700 1 -- 172.21.15.131:0/2574428971 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f211c084dd0 con 0x7f21240c8bf0 2023-12-26T21:56:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.085+0000 7f212959b700 1 -- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2108067170 msgr2=0x7f2108069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.085+0000 7f212959b700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2108067170 0x7f2108069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:09.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.090+0000 7f21197fa700 1 -- 172.21.15.131:0/2574428971 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2124167b70 con 0x7f21240c8bf0 2023-12-26T21:56:09.091 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:09.091+0000 7f8d73e371c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:56:09.144 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:09.143+0000 7f26818441c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:56:09.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.286+0000 7f212959b700 1 -- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2108067170 msgr2=0x7f2108069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:09.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.286+0000 7f212959b700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2108067170 0x7f2108069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:09.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.292+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2124167b70 con 0x7f21240c8bf0 2023-12-26T21:56:09.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.293+0000 7f21197fa700 1 -- 172.21.15.131:0/2574428971 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16492 (secure 0 0 0) 0x7f2124167b70 con 0x7f21240c8bf0 2023-12-26T21:56:09.294 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:09.294 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":126,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:56:08.297731+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:55:26.570089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-26T21:55:27.376474+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2993830584":"2023-12-27T21:55:49.487069+0000","172.21.15.131:6837/39288":"2023-12-27T21:55:49.487069+0000","172.21.15.131:6836/39288":"2023-12-27T21:55:49.487069+0000","172.21.15.131:0/4249324124":"2023-12-27T21:55:49.487069+0000","172.21.15.131:0/324814667":"2023-12-27T21:55:49.487069+0000","172.21.15.131:0/1631075145":"2023-12-27T21:55:49.487069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:56:09.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2108067170 msgr2=0x7f2108069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2108067170 0x7f2108069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 msgr2=0x7f21240ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 0x7f21240ae050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f211c00cb90 tx=0x7f211c0018f0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 shutdown_connections 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21240c8bf0 0x7f21240ae050 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2124069d10 0x7f212416d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2108067170 0x7f2108069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 --2- 172.21.15.131:0/2574428971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21240c8820 0x7f21240adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 >> 172.21.15.131:0/2574428971 conn(0x7f2124068040 msgr2=0x7f21240af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 shutdown_connections 2023-12-26T21:56:09.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.296+0000 7f212b7ff700 1 -- 172.21.15.131:0/2574428971 wait complete. 2023-12-26T21:56:09.316 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T21:56:09.347 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:09.346+0000 7f26818441c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:56:09.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.511+0000 7f0814dff700 1 Processor -- start 2023-12-26T21:56:09.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.511+0000 7f0814dff700 1 -- start start 2023-12-26T21:56:09.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.512+0000 7f0814dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08100cc580 0x7f08100cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.512+0000 7f0814dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f08100b0560 0x7f081015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.512+0000 7f0814dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100b0d70 con 0x7f08100b00e0 2023-12-26T21:56:09.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.512+0000 7f0814dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08100680f0 con 0x7f08100b0560 2023-12-26T21:56:09.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.512+0000 7f0814dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08100675b0 con 0x7f08100cc580 2023-12-26T21:56:09.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.513+0000 7f080ed9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f08100b0560 0x7f081015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.513+0000 7f080e59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08100cc580 0x7f08100cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.513+0000 7f080ed9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f08100b0560 0x7f081015d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57110/0 (socket says 172.21.15.131:57110) 2023-12-26T21:56:09.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.513+0000 7f080ed9d700 1 -- 172.21.15.131:0/1857602866 learned_addr learned my addr 172.21.15.131:0/1857602866 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:09.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.513+0000 7f080e59c700 1 -- 172.21.15.131:0/1857602866 >> v2:172.21.15.131:3300/0 conn(0x7f08100b0560 msgr2=0x7f081015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:09.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.513+0000 7f080e59c700 1 --2- 172.21.15.131:0/1857602866 >> v2:172.21.15.131:3300/0 conn(0x7f08100b0560 0x7f081015d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.513+0000 7f080e59c700 1 -- 172.21.15.131:0/1857602866 >> v1:172.21.15.190:6789/0 conn(0x7f08100b00e0 legacy=0x7f081006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.513+0000 7f080e59c700 1 -- 172.21.15.131:0/1857602866 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0810159c30 con 0x7f08100cc580 2023-12-26T21:56:09.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.513+0000 7f080e59c700 1 --2- 172.21.15.131:0/1857602866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 0x7f08100cc950 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f081006b740 tx=0x7f08000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f243bd77b18cb5ae server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:09.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f080d59a700 1 -- 172.21.15.131:0/1857602866 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800011630 con 0x7f08100cc580 2023-12-26T21:56:09.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f080d59a700 1 -- 172.21.15.131:0/1857602866 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0800011790 con 0x7f08100cc580 2023-12-26T21:56:09.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f080d59a700 1 -- 172.21.15.131:0/1857602866 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800011b30 con 0x7f08100cc580 2023-12-26T21:56:09.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f0814dff700 1 -- 172.21.15.131:0/1857602866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 msgr2=0x7f08100cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f0814dff700 1 --2- 172.21.15.131:0/1857602866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 0x7f08100cc950 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f081006b740 tx=0x7f08000091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f0814dff700 1 -- 172.21.15.131:0/1857602866 shutdown_connections 2023-12-26T21:56:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f0814dff700 1 --2- 172.21.15.131:0/1857602866 >> v2:172.21.15.131:3300/0 conn(0x7f08100b0560 0x7f081015d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f0814dff700 1 --2- 172.21.15.131:0/1857602866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 0x7f08100cc950 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f0814dff700 1 -- 172.21.15.131:0/1857602866 >> 172.21.15.131:0/1857602866 conn(0x7f08100686c0 msgr2=0x7f0810068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f0814dff700 1 -- 172.21.15.131:0/1857602866 shutdown_connections 2023-12-26T21:56:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.514+0000 7f0814dff700 1 -- 172.21.15.131:0/1857602866 wait complete. 2023-12-26T21:56:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.516+0000 7f0814dff700 1 Processor -- start 2023-12-26T21:56:09.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.516+0000 7f0814dff700 1 -- start start 2023-12-26T21:56:09.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f0814dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08100b00e0 0x7f081015e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f0814dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08100b0560 0x7f0810143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f0814dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 0x7f0810143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f0814dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081016dd60 con 0x7f08100b00e0 2023-12-26T21:56:09.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f0814dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0810169a90 con 0x7f08100cc580 2023-12-26T21:56:09.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f0814dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0810169d70 con 0x7f08100b0560 2023-12-26T21:56:09.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08100b00e0 0x7f081015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08100b00e0 0x7f081015e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57124/0 (socket says 172.21.15.131:57124) 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080e59c700 1 -- 172.21.15.131:0/3260511354 learned_addr learned my addr 172.21.15.131:0/3260511354 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080ed9d700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 0x7f0810143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080dd9b700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08100b0560 0x7f0810143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080e59c700 1 -- 172.21.15.131:0/3260511354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08100b0560 msgr2=0x7f0810143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080e59c700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08100b0560 0x7f0810143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080e59c700 1 -- 172.21.15.131:0/3260511354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 msgr2=0x7f0810143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080e59c700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 0x7f0810143e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080e59c700 1 -- 172.21.15.131:0/3260511354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800009420 con 0x7f08100b00e0 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.517+0000 7f080ed9d700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 0x7f0810143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:09.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.518+0000 7f080e59c700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08100b00e0 0x7f081015e570 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f08100676f0 tx=0x7f0800017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:09.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.518+0000 7f07fe7fc700 1 -- 172.21.15.131:0/3260511354 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800011b10 con 0x7f08100b00e0 2023-12-26T21:56:09.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.518+0000 7f07fe7fc700 1 -- 172.21.15.131:0/3260511354 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08000253f0 con 0x7f08100b00e0 2023-12-26T21:56:09.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.518+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081016a050 con 0x7f08100b00e0 2023-12-26T21:56:09.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.518+0000 7f07fe7fc700 1 -- 172.21.15.131:0/3260511354 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800025790 con 0x7f08100b00e0 2023-12-26T21:56:09.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.518+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081016a850 con 0x7f08100b00e0 2023-12-26T21:56:09.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.519+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081016a4f0 con 0x7f08100b00e0 2023-12-26T21:56:09.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.520+0000 7f07fe7fc700 1 -- 172.21.15.131:0/3260511354 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7f0800025b80 con 0x7f08100b00e0 2023-12-26T21:56:09.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.521+0000 7f07fe7fc700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f07f0067170 0x7f07f0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:09.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.521+0000 7f07fe7fc700 1 -- 172.21.15.131:0/3260511354 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f08000863d0 con 0x7f08100b00e0 2023-12-26T21:56:09.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.521+0000 7f080dd9b700 1 -- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f07f0067170 msgr2=0x7f07f0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:09.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.521+0000 7f080dd9b700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f07f0067170 0x7f07f0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:09.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.525+0000 7f07fe7fc700 1 -- 172.21.15.131:0/3260511354 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08000530d0 con 0x7f08100b00e0 2023-12-26T21:56:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.722+0000 7f080dd9b700 1 -- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f07f0067170 msgr2=0x7f07f0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:09.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.722+0000 7f080dd9b700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f07f0067170 0x7f07f0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:09.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.769+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f081016a680 con 0x7f08100b00e0 2023-12-26T21:56:09.773 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T21:56:09.770+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T21:56:09.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.779+0000 7f07fe7fc700 1 -- 172.21.15.131:0/3260511354 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f08000566f0 con 0x7f08100b00e0 2023-12-26T21:56:09.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.781+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f07f0067170 msgr2=0x7f07f0069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.781+0000 7f0814dff700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f07f0067170 0x7f07f0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.781+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08100b00e0 msgr2=0x7f081015e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.781+0000 7f0814dff700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08100b00e0 0x7f081015e570 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f08100676f0 tx=0x7f0800017000 comp rx=0 tx=0).stop 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.782+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 shutdown_connections 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.782+0000 7f0814dff700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08100cc580 0x7f0810143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.782+0000 7f0814dff700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f07f0067170 0x7f07f0069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.782+0000 7f0814dff700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08100b00e0 0x7f081015e570 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.782+0000 7f0814dff700 1 --2- 172.21.15.131:0/3260511354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08100b0560 0x7f0810143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.782+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 >> 172.21.15.131:0/3260511354 conn(0x7f08100686c0 msgr2=0x7f08100a9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.782+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 shutdown_connections 2023-12-26T21:56:09.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:09.782+0000 7f0814dff700 1 -- 172.21.15.131:0/3260511354 wait complete. 2023-12-26T21:56:09.784 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T21:56:09.787 INFO:tasks.ceph.mds.c.smithi131.stderr: -4630> 2023-12-26T21:55:33.901+0000 7f72bbdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:56:09.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T21:56:09.842 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:56:09.933 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:09.933+0000 7f8d73e371c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:56:10.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.037+0000 7fa0c5dff700 1 Processor -- start 2023-12-26T21:56:10.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.037+0000 7fa0c5dff700 1 -- start start 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.038+0000 7fa0c5dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0c00ad560 0x7fa0c01645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.038+0000 7fa0c5dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c00a98b0 con 0x7fa0c01595e0 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.038+0000 7fa0c5dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c015a500 con 0x7fa0c015d240 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.038+0000 7fa0c5dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c00add40 con 0x7fa0c00ad560 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0bf7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa0c01595e0 0x7fa0c01599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47196/0 (socket says 172.21.15.131:47196) 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0bf7fe700 1 -- 172.21.15.131:0/911200956 learned_addr learned my addr 172.21.15.131:0/911200956 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0bffff700 1 --2- 172.21.15.131:0/911200956 >> v2:172.21.15.131:3300/0 conn(0x7fa0c00ad560 0x7fa0c01645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0be7fc700 1 -- 172.21.15.131:0/911200956 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848951272 0 0) 0x7fa0c00a98b0 con 0x7fa0c01595e0 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0be7fc700 1 -- 172.21.15.131:0/911200956 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a8003570 con 0x7fa0c01595e0 2023-12-26T21:56:10.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0be7fc700 1 -- 172.21.15.131:0/911200956 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607498485 0 0) 0x7fa0c015a500 con 0x7fa0c015d240 2023-12-26T21:56:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0be7fc700 1 -- 172.21.15.131:0/911200956 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c00a98b0 con 0x7fa0c015d240 2023-12-26T21:56:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0bffff700 1 -- 172.21.15.131:0/911200956 >> v1:172.21.15.190:6789/0 conn(0x7fa0c015d240 legacy=0x7fa0c00ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0bffff700 1 -- 172.21.15.131:0/911200956 >> v1:172.21.15.131:6789/0 conn(0x7fa0c01595e0 legacy=0x7fa0c01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.039+0000 7fa0bffff700 1 -- 172.21.15.131:0/911200956 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c015a7e0 con 0x7fa0c00ad560 2023-12-26T21:56:10.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.040+0000 7fa0bffff700 1 --2- 172.21.15.131:0/911200956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c00ad560 0x7fa0c01645e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e60 tx=0x7fa0b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcde02b0b39715a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.040+0000 7fa0be7fc700 1 -- 172.21.15.131:0/911200956 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800e070 con 0x7fa0c00ad560 2023-12-26T21:56:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.040+0000 7fa0be7fc700 1 -- 172.21.15.131:0/911200956 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b800bab0 con 0x7fa0c00ad560 2023-12-26T21:56:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.040+0000 7fa0be7fc700 1 -- 172.21.15.131:0/911200956 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800be70 con 0x7fa0c00ad560 2023-12-26T21:56:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.040+0000 7fa0c5dff700 1 -- 172.21.15.131:0/911200956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c00ad560 msgr2=0x7fa0c01645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.040+0000 7fa0c5dff700 1 --2- 172.21.15.131:0/911200956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c00ad560 0x7fa0c01645e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e60 tx=0x7fa0b8009780 comp rx=0 tx=0).stop 2023-12-26T21:56:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.041+0000 7fa0c5dff700 1 -- 172.21.15.131:0/911200956 shutdown_connections 2023-12-26T21:56:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.041+0000 7fa0c5dff700 1 --2- 172.21.15.131:0/911200956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c00ad560 0x7fa0c01645e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.041+0000 7fa0c5dff700 1 -- 172.21.15.131:0/911200956 >> 172.21.15.131:0/911200956 conn(0x7fa0c0068040 msgr2=0x7fa0c0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:10.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.041+0000 7fa0c5dff700 1 -- 172.21.15.131:0/911200956 shutdown_connections 2023-12-26T21:56:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.041+0000 7fa0c5dff700 1 -- 172.21.15.131:0/911200956 wait complete. 2023-12-26T21:56:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.043+0000 7fa0c5dff700 1 Processor -- start 2023-12-26T21:56:10.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.043+0000 7fa0c5dff700 1 -- start start 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0c5dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0c00ad560 0x7fa0c0145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0c5dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0c01595e0 0x7fa0c006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0c5dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c015d240 0x7fa0c016db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0c5dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c006eb40 con 0x7fa0c015d240 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0c5dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0146260 con 0x7fa0c01595e0 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0c5dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c006df30 con 0x7fa0c00ad560 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0bffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c015d240 0x7fa0c016db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0beffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0c01595e0 0x7fa0c006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0bf7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0c00ad560 0x7fa0c0145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:10.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0bffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c015d240 0x7fa0c016db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57150/0 (socket says 172.21.15.131:57150) 2023-12-26T21:56:10.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.044+0000 7fa0bffff700 1 -- 172.21.15.131:0/830537324 learned_addr learned my addr 172.21.15.131:0/830537324 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0bffff700 1 -- 172.21.15.131:0/830537324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0c00ad560 msgr2=0x7fa0c0145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0bffff700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0c00ad560 0x7fa0c0145d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0bffff700 1 -- 172.21.15.131:0/830537324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0c01595e0 msgr2=0x7fa0c006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0bffff700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0c01595e0 0x7fa0c006d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0bffff700 1 -- 172.21.15.131:0/830537324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8009a20 con 0x7fa0c015d240 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0bffff700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c015d240 0x7fa0c016db50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00688f0 tx=0x7fa0b8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0a77fe700 1 -- 172.21.15.131:0/830537324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8012db0 con 0x7fa0c015d240 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0a77fe700 1 -- 172.21.15.131:0/830537324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8026480 con 0x7fa0c015d240 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0a77fe700 1 -- 172.21.15.131:0/830537324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8026840 con 0x7fa0c015d240 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.045+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c006e210 con 0x7fa0c015d240 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.046+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c006e7d0 con 0x7fa0c015d240 2023-12-26T21:56:10.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.047+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c006ecb0 con 0x7fa0c015d240 2023-12-26T21:56:10.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.048+0000 7fa0a77fe700 1 -- 172.21.15.131:0/830537324 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 59) v1 ==== 84958+0+0 (secure 0 0 0) 0x7fa0b801c400 con 0x7fa0c015d240 2023-12-26T21:56:10.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.048+0000 7fa0a77fe700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 0x7fa0ac069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:10.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.048+0000 7fa0a77fe700 1 -- 172.21.15.131:0/830537324 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fa0b8086b30 con 0x7fa0c015d240 2023-12-26T21:56:10.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.048+0000 7fa0bf7fe700 1 -- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 msgr2=0x7fa0ac069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:10.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.049+0000 7fa0bf7fe700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 0x7fa0ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:10.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.052+0000 7fa0a77fe700 1 -- 172.21.15.131:0/830537324 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b8053700 con 0x7fa0c015d240 2023-12-26T21:56:10.209 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:10.209+0000 7f8d73e371c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:56:10.245 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:10.245+0000 7f26818441c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:56:10.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.250+0000 7fa0bf7fe700 1 -- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 msgr2=0x7fa0ac069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:10.250 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.250+0000 7fa0bf7fe700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 0x7fa0ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.297+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa0c006eed0 con 0x7fa0c015d240 2023-12-26T21:56:10.300 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:10.300+0000 7f8d73e371c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:56:10.499 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:10.499+0000 7f8d73e371c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:56:10.527 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:10.526+0000 7f26818441c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:56:10.623 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:10.622+0000 7f26818441c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:56:10.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.651+0000 7fa0bf7fe700 1 -- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 msgr2=0x7fa0ac069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:10.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.651+0000 7fa0bf7fe700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 0x7fa0ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:56:10.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.778+0000 7fa0a77fe700 1 -- 172.21.15.131:0/830537324 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa0b80860a0 con 0x7fa0c015d240 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.780+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 msgr2=0x7fa0ac069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.780+0000 7fa0c5dff700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 0x7fa0ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c015d240 msgr2=0x7fa0c016db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c015d240 0x7fa0c016db50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00688f0 tx=0x7fa0b8019000 comp rx=0 tx=0).stop 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 shutdown_connections 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa0ac067170 0x7fa0ac069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0c015d240 0x7fa0c016db50 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0c00ad560 0x7fa0c0145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 --2- 172.21.15.131:0/830537324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0c01595e0 0x7fa0c006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 >> 172.21.15.131:0/830537324 conn(0x7fa0c0068040 msgr2=0x7fa0c00694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 shutdown_connections 2023-12-26T21:56:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:10.781+0000 7fa0c5dff700 1 -- 172.21.15.131:0/830537324 wait complete. 2023-12-26T21:56:10.805 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T21:56:10.831 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:10.830+0000 7f26818441c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:56:11.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.017+0000 7fe676fff700 1 Processor -- start 2023-12-26T21:56:11.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.017+0000 7fe676fff700 1 -- start start 2023-12-26T21:56:11.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.018+0000 7fe676fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe670156360 0x7fe67006e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:11.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.018+0000 7fe676fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 0x7fe67006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:11.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.018+0000 7fe676fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700acfc0 con 0x7fe670156730 2023-12-26T21:56:11.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe676fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67006f1f0 con 0x7fe670156360 2023-12-26T21:56:11.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe676fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700c91f0 con 0x7fe6700c8ba0 2023-12-26T21:56:11.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe674d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe670156360 0x7fe67006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:11.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe66ffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe670156730 0x7fe6700c8490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47202/0 (socket says 172.21.15.131:47202) 2023-12-26T21:56:11.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe66ffff700 1 -- 172.21.15.131:0/1958810627 learned_addr learned my addr 172.21.15.131:0/1958810627 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:11.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe67559c700 1 --2- 172.21.15.131:0/1958810627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 0x7fe67006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:11.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe674d9b700 1 -- 172.21.15.131:0/1958810627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 msgr2=0x7fe67006ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe674d9b700 1 --2- 172.21.15.131:0/1958810627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 0x7fe67006ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe674d9b700 1 -- 172.21.15.131:0/1958810627 >> v1:172.21.15.131:6789/0 conn(0x7fe670156730 legacy=0x7fe6700c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe674d9b700 1 -- 172.21.15.131:0/1958810627 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6700c94d0 con 0x7fe670156360 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.019+0000 7fe66f7fe700 1 -- 172.21.15.131:0/1958810627 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280028111 0 0) 0x7fe6700acfc0 con 0x7fe670156730 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe674d9b700 1 --2- 172.21.15.131:0/1958810627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 0x7fe67006e770 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe6640098e0 tx=0x7fe6640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bd8cbdb5d2fd78 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe66f7fe700 1 -- 172.21.15.131:0/1958810627 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66400b640 con 0x7fe670156360 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe66f7fe700 1 -- 172.21.15.131:0/1958810627 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66400b7a0 con 0x7fe670156360 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe66f7fe700 1 -- 172.21.15.131:0/1958810627 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66400bb40 con 0x7fe670156360 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe676fff700 1 -- 172.21.15.131:0/1958810627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 msgr2=0x7fe67006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe676fff700 1 --2- 172.21.15.131:0/1958810627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 0x7fe67006e770 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe6640098e0 tx=0x7fe6640091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe676fff700 1 -- 172.21.15.131:0/1958810627 shutdown_connections 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe676fff700 1 --2- 172.21.15.131:0/1958810627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 0x7fe67006ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe676fff700 1 --2- 172.21.15.131:0/1958810627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 0x7fe67006e770 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.020+0000 7fe676fff700 1 -- 172.21.15.131:0/1958810627 >> 172.21.15.131:0/1958810627 conn(0x7fe6700675b0 msgr2=0x7fe6700679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:11.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.021+0000 7fe676fff700 1 -- 172.21.15.131:0/1958810627 shutdown_connections 2023-12-26T21:56:11.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.021+0000 7fe676fff700 1 -- 172.21.15.131:0/1958810627 wait complete. 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.022+0000 7fe676fff700 1 Processor -- start 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.022+0000 7fe676fff700 1 -- start start 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.023+0000 7fe676fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 0x7fe670169440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.023+0000 7fe676fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 0x7fe67016c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.023+0000 7fe676fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670156730 0x7fe6700b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.023+0000 7fe676fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67016c830 con 0x7fe670156360 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.023+0000 7fe676fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe670159dd0 con 0x7fe670156730 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.023+0000 7fe676fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701591c0 con 0x7fe6700c8ba0 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.023+0000 7fe66ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 0x7fe67016c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.023+0000 7fe66ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 0x7fe67016c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57234/0 (socket says 172.21.15.131:57234) 2023-12-26T21:56:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.023+0000 7fe66ffff700 1 -- 172.21.15.131:0/762175336 learned_addr learned my addr 172.21.15.131:0/762175336 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe674d9b700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 0x7fe670169440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe67559c700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670156730 0x7fe6700b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe66ffff700 1 -- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 msgr2=0x7fe670169440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe66ffff700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 0x7fe670169440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe66ffff700 1 -- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670156730 msgr2=0x7fe6700b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe66ffff700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670156730 0x7fe6700b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe66ffff700 1 -- 172.21.15.131:0/762175336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664009420 con 0x7fe670156360 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe67559c700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670156730 0x7fe6700b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe674d9b700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 0x7fe670169440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe66ffff700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 0x7fe67016c2f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00c810 tx=0x7fe65c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe66cff9700 1 -- 172.21.15.131:0/762175336 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c005c00 con 0x7fe670156360 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe66cff9700 1 -- 172.21.15.131:0/762175336 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c005d60 con 0x7fe670156360 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe66cff9700 1 -- 172.21.15.131:0/762175336 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c007320 con 0x7fe670156360 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe676fff700 1 -- 172.21.15.131:0/762175336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6701594a0 con 0x7fe670156360 2023-12-26T21:56:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.024+0000 7fe676fff700 1 -- 172.21.15.131:0/762175336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe670159a60 con 0x7fe670156360 2023-12-26T21:56:11.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.025+0000 7fe676fff700 1 -- 172.21.15.131:0/762175336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6700aef90 con 0x7fe670156360 2023-12-26T21:56:11.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.026+0000 7fe66cff9700 1 -- 172.21.15.131:0/762175336 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 60) v1 ==== 84453+0+0 (secure 0 0 0) 0x7fe65c007690 con 0x7fe670156360 2023-12-26T21:56:11.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.027+0000 7fe66cff9700 1 -- 172.21.15.131:0/762175336 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe65c084090 con 0x7fe670156360 2023-12-26T21:56:11.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.031+0000 7fe66cff9700 1 -- 172.21.15.131:0/762175336 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe65c051200 con 0x7fe670156360 2023-12-26T21:56:11.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.226+0000 7fe676fff700 1 -- 172.21.15.131:0/762175336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe6700717f0 con 0x7fe670156360 2023-12-26T21:56:11.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.840+0000 7fe66cff9700 1 -- 172.21.15.131:0/762175336 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe65c054820 con 0x7fe670156360 2023-12-26T21:56:11.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.844+0000 7fe676fff700 1 -- 172.21.15.131:0/762175336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 msgr2=0x7fe67016c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:11.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.844+0000 7fe676fff700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 0x7fe67016c2f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00c810 tx=0x7fe65c007db0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.844+0000 7fe676fff700 1 -- 172.21.15.131:0/762175336 shutdown_connections 2023-12-26T21:56:11.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.844+0000 7fe676fff700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe670156730 0x7fe6700b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.844+0000 7fe676fff700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6700c8ba0 0x7fe670169440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.844+0000 7fe676fff700 1 --2- 172.21.15.131:0/762175336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe670156360 0x7fe67016c2f0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:11.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.844+0000 7fe676fff700 1 -- 172.21.15.131:0/762175336 >> 172.21.15.131:0/762175336 conn(0x7fe6700675b0 msgr2=0x7fe670067980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:11.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.844+0000 7fe676fff700 1 -- 172.21.15.131:0/762175336 shutdown_connections 2023-12-26T21:56:11.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:11.845+0000 7fe676fff700 1 -- 172.21.15.131:0/762175336 wait complete. 2023-12-26T21:56:11.846 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T21:56:11.872 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T21:56:12.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.106+0000 7f6dd3dff700 1 Processor -- start 2023-12-26T21:56:12.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.106+0000 7f6dd3dff700 1 -- start start 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.107+0000 7f6dd3dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6dcc0c8d40 0x7f6dcc15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.107+0000 7f6dd3dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 0x7f6dcc1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.107+0000 7f6dd3dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc0af040 con 0x7f6dcc0c9110 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.107+0000 7f6dd3dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc0ae0e0 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.107+0000 7f6dd3dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0ae3c0 con 0x7f6dcc156e50 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd1b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6dcc0c8d40 0x7f6dcc15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd139a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6dcc0c9110 0x7f6dcc0ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47216/0 (socket says 172.21.15.131:47216) 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd139a700 1 -- 172.21.15.131:0/1083105582 learned_addr learned my addr 172.21.15.131:0/1083105582 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd239c700 1 --2- 172.21.15.131:0/1083105582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 0x7f6dcc1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:12.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd1b9b700 1 -- 172.21.15.131:0/1083105582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 msgr2=0x7f6dcc1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd1b9b700 1 --2- 172.21.15.131:0/1083105582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 0x7f6dcc1572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd1b9b700 1 -- 172.21.15.131:0/1083105582 >> v1:172.21.15.131:6789/0 conn(0x7f6dcc0c9110 legacy=0x7f6dcc0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd1b9b700 1 -- 172.21.15.131:0/1083105582 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0ae6a0 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd239c700 1 --2- 172.21.15.131:0/1083105582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 0x7f6dcc1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd0b99700 1 -- 172.21.15.131:0/1083105582 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494951454 0 0) 0x7f6dcc0af040 con 0x7f6dcc0c9110 2023-12-26T21:56:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.108+0000 7f6dd1b9b700 1 --2- 172.21.15.131:0/1083105582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 0x7f6dcc15bc30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80098e0 tx=0x7f6dc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a944b3d015e224f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd0b99700 1 -- 172.21.15.131:0/1083105582 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8009e80 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd0b99700 1 -- 172.21.15.131:0/1083105582 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc800b860 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd0b99700 1 -- 172.21.15.131:0/1083105582 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800bbe0 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd3dff700 1 -- 172.21.15.131:0/1083105582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 msgr2=0x7f6dcc15bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd3dff700 1 --2- 172.21.15.131:0/1083105582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 0x7f6dcc15bc30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80098e0 tx=0x7f6dc80091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd3dff700 1 -- 172.21.15.131:0/1083105582 shutdown_connections 2023-12-26T21:56:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd3dff700 1 --2- 172.21.15.131:0/1083105582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 0x7f6dcc1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd3dff700 1 --2- 172.21.15.131:0/1083105582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 0x7f6dcc15bc30 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd3dff700 1 -- 172.21.15.131:0/1083105582 >> 172.21.15.131:0/1083105582 conn(0x7f6dcc0675b0 msgr2=0x7f6dcc0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd3dff700 1 -- 172.21.15.131:0/1083105582 shutdown_connections 2023-12-26T21:56:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.109+0000 7f6dd3dff700 1 -- 172.21.15.131:0/1083105582 wait complete. 2023-12-26T21:56:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.111+0000 7f6dd3dff700 1 Processor -- start 2023-12-26T21:56:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.111+0000 7f6dd3dff700 1 -- start start 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd3dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 0x7f6dcc159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd3dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc0c9110 0x7f6dcc158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd3dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 0x7f6dcc1546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd3dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1589a0 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd3dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc15a790 con 0x7f6dcc0c9110 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd3dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0a9d00 con 0x7f6dcc156e50 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd1b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 0x7f6dcc159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd1b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 0x7f6dcc159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57256/0 (socket says 172.21.15.131:57256) 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd1b9b700 1 -- 172.21.15.131:0/935185739 learned_addr learned my addr 172.21.15.131:0/935185739 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd239c700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 0x7f6dcc1546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd139a700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc0c9110 0x7f6dcc158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd1b9b700 1 -- 172.21.15.131:0/935185739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 msgr2=0x7f6dcc1546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd1b9b700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 0x7f6dcc1546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd1b9b700 1 -- 172.21.15.131:0/935185739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc0c9110 msgr2=0x7f6dcc158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd1b9b700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc0c9110 0x7f6dcc158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.112+0000 7f6dd1b9b700 1 -- 172.21.15.131:0/935185739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8009420 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.113+0000 7f6dd1b9b700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 0x7f6dcc159e80 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800b3c0 tx=0x7f6dc8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.113+0000 7f6dc1ffb700 1 -- 172.21.15.131:0/935185739 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800bbe0 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.113+0000 7f6dc1ffb700 1 -- 172.21.15.131:0/935185739 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc8011420 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.113+0000 7f6dc1ffb700 1 -- 172.21.15.131:0/935185739 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc80117c0 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.113+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc15a4b0 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.113+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc0aa280 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.114+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc0717f0 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.115+0000 7f6dc1ffb700 1 -- 172.21.15.131:0/935185739 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 61) v1 ==== 84580+0+0 (secure 0 0 0) 0x7f6dc8011b70 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.116+0000 7f6dc1ffb700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6db0066d80 0x7f6db0069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:12.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.116+0000 7f6dc1ffb700 1 -- 172.21.15.131:0/935185739 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6375+0+0 (secure 0 0 0) 0x7f6dc8085eb0 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.116+0000 7f6dd139a700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6db0066d80 0x7f6db0069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:12.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.117+0000 7f6dd139a700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6db0066d80 0x7f6db0069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0c83b0 tx=0x7f6dbc009040 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:12.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.120+0000 7f6dc1ffb700 1 -- 172.21.15.131:0/935185739 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dc8053130 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.318+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6dcc0ae980 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.865+0000 7f6dc1ffb700 1 -- 172.21.15.131:0/935185739 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6dc8053500 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.865+0000 7f6dc1ffb700 1 -- 172.21.15.131:0/935185739 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6dc8050360 con 0x7f6dcc0c8d40 2023-12-26T21:56:12.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6db0066d80 msgr2=0x7f6db0069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:12.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6db0066d80 0x7f6db0069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0c83b0 tx=0x7f6dbc009040 comp rx=0 tx=0).stop 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 msgr2=0x7f6dcc159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 0x7f6dcc159e80 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800b3c0 tx=0x7f6dc8017000 comp rx=0 tx=0).stop 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 shutdown_connections 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dcc156e50 0x7f6dcc1546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6db0066d80 0x7f6db0069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6dcc0c8d40 0x7f6dcc159e80 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 --2- 172.21.15.131:0/935185739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dcc0c9110 0x7f6dcc158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.870+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 >> 172.21.15.131:0/935185739 conn(0x7f6dcc0675b0 msgr2=0x7f6dcc15aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.871+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 shutdown_connections 2023-12-26T21:56:12.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:12.871+0000 7f6dd3dff700 1 -- 172.21.15.131:0/935185739 wait complete. 2023-12-26T21:56:12.873 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T21:56:12.890 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T21:56:13.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.080+0000 7fed10bff700 1 Processor -- start 2023-12-26T21:56:13.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.080+0000 7fed10bff700 1 -- start start 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.081+0000 7fed10bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fed0c0b0750 0x7fed0c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.081+0000 7fed10bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fed0c0c9400 0x7fed0c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.081+0000 7fed10bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed0c156940 con 0x7fed0c0cb320 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.081+0000 7fed10bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0c0c8cc0 con 0x7fed0c0c9400 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.081+0000 7fed10bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0c0ad700 con 0x7fed0c0b0750 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.081+0000 7fed0ad9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fed0c0c9400 0x7fed0c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0a59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fed0c0cb320 0x7fed0c069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47230/0 (socket says 172.21.15.131:47230) 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0a59c700 1 -- 172.21.15.131:0/1977989298 learned_addr learned my addr 172.21.15.131:0/1977989298 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0ad9d700 1 --2- 172.21.15.131:0/1977989298 >> v2:172.21.15.131:3300/0 conn(0x7fed0c0c9400 0x7fed0c0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57272/0 (socket says 172.21.15.131:57272) 2023-12-26T21:56:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed09d9b700 1 --2- 172.21.15.131:0/1977989298 >> v2:172.21.15.190:3300/0 conn(0x7fed0c0b0750 0x7fed0c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0ad9d700 1 -- 172.21.15.131:0/1977989298 >> v2:172.21.15.190:3300/0 conn(0x7fed0c0b0750 msgr2=0x7fed0c0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0ad9d700 1 --2- 172.21.15.131:0/1977989298 >> v2:172.21.15.190:3300/0 conn(0x7fed0c0b0750 0x7fed0c0b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0ad9d700 1 -- 172.21.15.131:0/1977989298 >> v1:172.21.15.131:6789/0 conn(0x7fed0c0cb320 legacy=0x7fed0c069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0ad9d700 1 -- 172.21.15.131:0/1977989298 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c0ae500 con 0x7fed0c0c9400 2023-12-26T21:56:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0959a700 1 -- 172.21.15.131:0/1977989298 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330105723 0 0) 0x7fed0c156940 con 0x7fed0c0cb320 2023-12-26T21:56:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0ad9d700 1 --2- 172.21.15.131:0/1977989298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0c9400 0x7fed0c0c9840 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fed0c064700 tx=0x7fed0400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6b9482fa1a61d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.082+0000 7fed0959a700 1 -- 172.21.15.131:0/1977989298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04010070 con 0x7fed0c0c9400 2023-12-26T21:56:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed0959a700 1 -- 172.21.15.131:0/1977989298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0400ddb0 con 0x7fed0c0c9400 2023-12-26T21:56:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed0959a700 1 -- 172.21.15.131:0/1977989298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0400b320 con 0x7fed0c0c9400 2023-12-26T21:56:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed10bff700 1 -- 172.21.15.131:0/1977989298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0c9400 msgr2=0x7fed0c0c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed10bff700 1 --2- 172.21.15.131:0/1977989298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0c9400 0x7fed0c0c9840 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fed0c064700 tx=0x7fed0400b990 comp rx=0 tx=0).stop 2023-12-26T21:56:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed10bff700 1 -- 172.21.15.131:0/1977989298 shutdown_connections 2023-12-26T21:56:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed10bff700 1 --2- 172.21.15.131:0/1977989298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0c9400 0x7fed0c0c9840 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed10bff700 1 --2- 172.21.15.131:0/1977989298 >> v2:172.21.15.190:3300/0 conn(0x7fed0c0b0750 0x7fed0c0b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed10bff700 1 -- 172.21.15.131:0/1977989298 >> 172.21.15.131:0/1977989298 conn(0x7fed0c068040 msgr2=0x7fed0c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed10bff700 1 -- 172.21.15.131:0/1977989298 shutdown_connections 2023-12-26T21:56:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.083+0000 7fed10bff700 1 -- 172.21.15.131:0/1977989298 wait complete. 2023-12-26T21:56:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.085+0000 7fed10bff700 1 Processor -- start 2023-12-26T21:56:13.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.085+0000 7fed10bff700 1 -- start start 2023-12-26T21:56:13.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed10bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0b0750 0x7fed0c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed10bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed0c0c9400 0x7fed0c153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed10bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed0c0cb320 0x7fed0c0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed10bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c0a7820 con 0x7fed0c0b0750 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed10bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c0a7b00 con 0x7fed0c0cb320 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed10bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c0a7de0 con 0x7fed0c0c9400 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed0a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0b0750 0x7fed0c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed0a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0b0750 0x7fed0c0ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57282/0 (socket says 172.21.15.131:57282) 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed0a59c700 1 -- 172.21.15.131:0/1029799505 learned_addr learned my addr 172.21.15.131:0/1029799505 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed0a59c700 1 -- 172.21.15.131:0/1029799505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed0c0c9400 msgr2=0x7fed0c153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed0a59c700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed0c0c9400 0x7fed0c153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed0a59c700 1 -- 172.21.15.131:0/1029799505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed0c0cb320 msgr2=0x7fed0c0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed0a59c700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed0c0cb320 0x7fed0c0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed0a59c700 1 -- 172.21.15.131:0/1029799505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0400bc30 con 0x7fed0c0b0750 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.086+0000 7fed0a59c700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0b0750 0x7fed0c0ace30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c5b0 tx=0x7fecfc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.087+0000 7fecfa7fc700 1 -- 172.21.15.131:0/1029799505 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc005b30 con 0x7fed0c0b0750 2023-12-26T21:56:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.087+0000 7fecfa7fc700 1 -- 172.21.15.131:0/1029799505 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc005c90 con 0x7fed0c0b0750 2023-12-26T21:56:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.087+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0c0a80c0 con 0x7fed0c0b0750 2023-12-26T21:56:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.087+0000 7fecfa7fc700 1 -- 172.21.15.131:0/1029799505 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc007300 con 0x7fed0c0b0750 2023-12-26T21:56:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.087+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c0ad370 con 0x7fed0c0b0750 2023-12-26T21:56:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.088+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0c0a8560 con 0x7fed0c0b0750 2023-12-26T21:56:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.089+0000 7fecfa7fc700 1 -- 172.21.15.131:0/1029799505 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fecfc01b3b0 con 0x7fed0c0b0750 2023-12-26T21:56:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.089+0000 7fecfa7fc700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fecec066f30 0x7fecec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.090+0000 7fecfa7fc700 1 -- 172.21.15.131:0/1029799505 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5961+0+0 (secure 0 0 0) 0x7fecfc083c00 con 0x7fed0c0b0750 2023-12-26T21:56:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.090+0000 7fed09d9b700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fecec066f30 0x7fecec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.090+0000 7fed09d9b700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fecec066f30 0x7fecec0693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed0c06f100 tx=0x7fed00008040 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:13.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.094+0000 7fecfa7fc700 1 -- 172.21.15.131:0/1029799505 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecfc050cd0 con 0x7fed0c0b0750 2023-12-26T21:56:13.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.288+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fed0c0657e0 con 0x7fed0c0b0750 2023-12-26T21:56:13.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.292+0000 7fecfa7fc700 1 -- 172.21.15.131:0/1029799505 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fecfc0542f0 con 0x7fed0c0b0750 2023-12-26T21:56:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fecec066f30 msgr2=0x7fecec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fecec066f30 0x7fecec0693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed0c06f100 tx=0x7fed00008040 comp rx=0 tx=0).stop 2023-12-26T21:56:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0b0750 msgr2=0x7fed0c0ace30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0b0750 0x7fed0c0ace30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c5b0 tx=0x7fecfc001070 comp rx=0 tx=0).stop 2023-12-26T21:56:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 shutdown_connections 2023-12-26T21:56:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed0c0cb320 0x7fed0c0aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fecec066f30 0x7fecec0693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed0c0b0750 0x7fed0c0ace30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 --2- 172.21.15.131:0/1029799505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed0c0c9400 0x7fed0c153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 >> 172.21.15.131:0/1029799505 conn(0x7fed0c068040 msgr2=0x7fed0c155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:13.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 shutdown_connections 2023-12-26T21:56:13.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.295+0000 7fed10bff700 1 -- 172.21.15.131:0/1029799505 wait complete. 2023-12-26T21:56:13.313 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T21:56:13.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.504+0000 7f47c3dff700 1 Processor -- start 2023-12-26T21:56:13.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.504+0000 7f47c3dff700 1 -- start start 2023-12-26T21:56:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.505+0000 7f47c3dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47bc0c8820 0x7f47bc0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.505+0000 7f47c3dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.505+0000 7f47c3dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47bc157a80 con 0x7f47bc0c8bf0 2023-12-26T21:56:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.505+0000 7f47c3dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc0cac50 con 0x7f47bc0c8820 2023-12-26T21:56:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.505+0000 7f47c3dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0caf30 con 0x7f47bc0b19f0 2023-12-26T21:56:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c1b9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47bc0c8820 0x7f47bc0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c239c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c1b9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47bc0c8820 0x7f47bc0ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:52704/0 (socket says 172.21.15.131:52704) 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c139a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f47bc0c8bf0 0x7f47bc0b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:53660/0 (socket says 172.21.15.131:53660) 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c239c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:54632/0 (socket says 172.21.15.131:54632) 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c1b9b700 1 -- 172.21.15.131:0/4158131802 learned_addr learned my addr 172.21.15.131:0/4158131802 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c1b9b700 1 -- 172.21.15.131:0/4158131802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 msgr2=0x7f47bc161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c1b9b700 1 --2- 172.21.15.131:0/4158131802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c1b9b700 1 -- 172.21.15.131:0/4158131802 >> v1:172.21.15.190:6789/0 conn(0x7f47bc0c8bf0 legacy=0x7f47bc0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c1b9b700 1 -- 172.21.15.131:0/4158131802 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0cb210 con 0x7f47bc0c8820 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.506+0000 7f47c239c700 1 --2- 172.21.15.131:0/4158131802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c1b9b700 1 --2- 172.21.15.131:0/4158131802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47bc0c8820 0x7f47bc0ca300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f47b80099b0 tx=0x7f47b8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f6765b17d07823 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c0b99700 1 -- 172.21.15.131:0/4158131802 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8009ea0 con 0x7f47bc0c8820 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c0b99700 1 -- 172.21.15.131:0/4158131802 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b800b860 con 0x7f47bc0c8820 2023-12-26T21:56:13.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c0b99700 1 -- 172.21.15.131:0/4158131802 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b800bc00 con 0x7f47bc0c8820 2023-12-26T21:56:13.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c3dff700 1 -- 172.21.15.131:0/4158131802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47bc0c8820 msgr2=0x7f47bc0ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c3dff700 1 --2- 172.21.15.131:0/4158131802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47bc0c8820 0x7f47bc0ca300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f47b80099b0 tx=0x7f47b8009250 comp rx=0 tx=0).stop 2023-12-26T21:56:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c3dff700 1 -- 172.21.15.131:0/4158131802 shutdown_connections 2023-12-26T21:56:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c3dff700 1 --2- 172.21.15.131:0/4158131802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c3dff700 1 --2- 172.21.15.131:0/4158131802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47bc0c8820 0x7f47bc0ca300 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.507+0000 7f47c3dff700 1 -- 172.21.15.131:0/4158131802 >> 172.21.15.131:0/4158131802 conn(0x7f47bc068040 msgr2=0x7f47bc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.508+0000 7f47c3dff700 1 -- 172.21.15.131:0/4158131802 shutdown_connections 2023-12-26T21:56:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.508+0000 7f47c3dff700 1 -- 172.21.15.131:0/4158131802 wait complete. 2023-12-26T21:56:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.509+0000 7f47c3dff700 1 Processor -- start 2023-12-26T21:56:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.509+0000 7f47c3dff700 1 -- start start 2023-12-26T21:56:13.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c3dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc165570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c3dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47bc0c8820 0x7f47bc0c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c3dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47bc0c8bf0 0x7f47bc141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c3dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc166510 con 0x7f47bc0c8820 2023-12-26T21:56:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c3dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0c2200 con 0x7f47bc0c8bf0 2023-12-26T21:56:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c3dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc141e80 con 0x7f47bc0b19f0 2023-12-26T21:56:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c139a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47bc0c8820 0x7f47bc0c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c139a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47bc0c8820 0x7f47bc0c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57292/0 (socket says 172.21.15.131:57292) 2023-12-26T21:56:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c139a700 1 -- 172.21.15.131:0/1434445129 learned_addr learned my addr 172.21.15.131:0/1434445129 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c1b9b700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.510+0000 7f47c239c700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47bc0c8bf0 0x7f47bc141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47c139a700 1 -- 172.21.15.131:0/1434445129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 msgr2=0x7f47bc165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47c139a700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47c139a700 1 -- 172.21.15.131:0/1434445129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47bc0c8bf0 msgr2=0x7f47bc141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47c139a700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47bc0c8bf0 0x7f47bc141940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47c139a700 1 -- 172.21.15.131:0/1434445129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b400de30 con 0x7f47bc0c8820 2023-12-26T21:56:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47c139a700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47bc0c8820 0x7f47bc0c3e00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c790 tx=0x7f47ac007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47b1ffb700 1 -- 172.21.15.131:0/1434445129 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac005a70 con 0x7f47bc0c8820 2023-12-26T21:56:13.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47b1ffb700 1 -- 172.21.15.131:0/1434445129 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac005bf0 con 0x7f47bc0c8820 2023-12-26T21:56:13.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b80094f0 con 0x7f47bc0c8820 2023-12-26T21:56:13.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.511+0000 7f47b1ffb700 1 -- 172.21.15.131:0/1434445129 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0070d0 con 0x7f47bc0c8820 2023-12-26T21:56:13.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.512+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc165f50 con 0x7f47bc0c8820 2023-12-26T21:56:13.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.513+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47bc0c2340 con 0x7f47bc0c8820 2023-12-26T21:56:13.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.513+0000 7f47b1ffb700 1 -- 172.21.15.131:0/1434445129 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f47ac007480 con 0x7f47bc0c8820 2023-12-26T21:56:13.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.514+0000 7f47b1ffb700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f47a0066f30 0x7f47a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.514+0000 7f47b1ffb700 1 -- 172.21.15.131:0/1434445129 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f47ac083a60 con 0x7f47bc0c8820 2023-12-26T21:56:13.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.515+0000 7f47c1b9b700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f47a0066f30 0x7f47a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.515+0000 7f47c1b9b700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f47a0066f30 0x7f47a00693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f47b80099b0 tx=0x7f47b8017000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.518+0000 7f47b1ffb700 1 -- 172.21.15.131:0/1434445129 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47bc0c2340 con 0x7f47bc0c8820 2023-12-26T21:56:13.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.713+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f47bc0c2340 con 0x7f47bc0c8820 2023-12-26T21:56:13.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.719+0000 7f47b1ffb700 1 -- 172.21.15.131:0/1434445129 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v44) v1 ==== 115+0+0 (secure 0 0 0) 0x7f47bc0c2340 con 0x7f47bc0c8820 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f47a0066f30 msgr2=0x7f47a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f47a0066f30 0x7f47a00693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f47b80099b0 tx=0x7f47b8017000 comp rx=0 tx=0).stop 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47bc0c8820 msgr2=0x7f47bc0c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47bc0c8820 0x7f47bc0c3e00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c790 tx=0x7f47ac007f40 comp rx=0 tx=0).stop 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 shutdown_connections 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47bc0c8bf0 0x7f47bc141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47bc0b19f0 0x7f47bc165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f47a0066f30 0x7f47a00693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 --2- 172.21.15.131:0/1434445129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47bc0c8820 0x7f47bc0c3e00 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 >> 172.21.15.131:0/1434445129 conn(0x7f47bc068040 msgr2=0x7f47bc0ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 shutdown_connections 2023-12-26T21:56:13.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.722+0000 7f47c3dff700 1 -- 172.21.15.131:0/1434445129 wait complete. 2023-12-26T21:56:13.742 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T21:56:13.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.934+0000 7f46c7dff700 1 Processor -- start 2023-12-26T21:56:13.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.934+0000 7f46c7dff700 1 -- start start 2023-12-26T21:56:13.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.935+0000 7f46c7dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8bf0 0x7f46c00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.935+0000 7f46c7dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f46c0069d10 0x7f46c00b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.935+0000 7f46c7dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c00caa10 con 0x7f46c00c8820 2023-12-26T21:56:13.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.935+0000 7f46c7dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c0157a80 con 0x7f46c0069d10 2023-12-26T21:56:13.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.935+0000 7f46c7dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c00cac50 con 0x7f46c00c8bf0 2023-12-26T21:56:13.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.935+0000 7f46c639c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f46c0069d10 0x7f46c00b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.935+0000 7f46c639c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f46c0069d10 0x7f46c00b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57296/0 (socket says 172.21.15.131:57296) 2023-12-26T21:56:13.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.935+0000 7f46c639c700 1 -- 172.21.15.131:0/864547656 learned_addr learned my addr 172.21.15.131:0/864547656 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:13.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c539a700 1 --2- 172.21.15.131:0/864547656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8bf0 0x7f46c00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c639c700 1 -- 172.21.15.131:0/864547656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8bf0 msgr2=0x7f46c00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:13.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c639c700 1 --2- 172.21.15.131:0/864547656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8bf0 0x7f46c00b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c639c700 1 -- 172.21.15.131:0/864547656 >> v1:172.21.15.190:6789/0 conn(0x7f46c00c8820 legacy=0x7f46c00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c639c700 1 -- 172.21.15.131:0/864547656 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00caf30 con 0x7f46c0069d10 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c539a700 1 --2- 172.21.15.131:0/864547656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8bf0 0x7f46c00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c639c700 1 --2- 172.21.15.131:0/864547656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c0069d10 0x7f46c00b1820 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009f20 tx=0x7f46b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae758ee0961aee89 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c4b99700 1 -- 172.21.15.131:0/864547656 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800e070 con 0x7f46c0069d10 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c4b99700 1 -- 172.21.15.131:0/864547656 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b800ba80 con 0x7f46c0069d10 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.936+0000 7f46c4b99700 1 -- 172.21.15.131:0/864547656 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b80092e0 con 0x7f46c0069d10 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.937+0000 7f46c7dff700 1 -- 172.21.15.131:0/864547656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c0069d10 msgr2=0x7f46c00b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.937+0000 7f46c7dff700 1 --2- 172.21.15.131:0/864547656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c0069d10 0x7f46c00b1820 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009f20 tx=0x7f46b8009840 comp rx=0 tx=0).stop 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.937+0000 7f46c7dff700 1 -- 172.21.15.131:0/864547656 shutdown_connections 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.937+0000 7f46c7dff700 1 --2- 172.21.15.131:0/864547656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c0069d10 0x7f46c00b1820 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.937+0000 7f46c7dff700 1 --2- 172.21.15.131:0/864547656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8bf0 0x7f46c00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.937+0000 7f46c7dff700 1 -- 172.21.15.131:0/864547656 >> 172.21.15.131:0/864547656 conn(0x7f46c0068040 msgr2=0x7f46c0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:13.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.937+0000 7f46c7dff700 1 -- 172.21.15.131:0/864547656 shutdown_connections 2023-12-26T21:56:13.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.937+0000 7f46c7dff700 1 -- 172.21.15.131:0/864547656 wait complete. 2023-12-26T21:56:13.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.938+0000 7f46c7dff700 1 Processor -- start 2023-12-26T21:56:13.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.939+0000 7f46c7dff700 1 -- start start 2023-12-26T21:56:13.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.939+0000 7f46c7dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c0069d10 0x7f46c00c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.939+0000 7f46c7dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8820 0x7f46c0152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.939+0000 7f46c7dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c00c8bf0 0x7f46c00af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.939+0000 7f46c7dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c015c0f0 con 0x7f46c00c8bf0 2023-12-26T21:56:13.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c7dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c00c7f80 con 0x7f46c0069d10 2023-12-26T21:56:13.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c7dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c00c5db0 con 0x7f46c00c8820 2023-12-26T21:56:13.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c00c8bf0 0x7f46c00af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c00c8bf0 0x7f46c00af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57298/0 (socket says 172.21.15.131:57298) 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c639c700 1 -- 172.21.15.131:0/3228486705 learned_addr learned my addr 172.21.15.131:0/3228486705 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c539a700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8820 0x7f46c0152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c5b9b700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c0069d10 0x7f46c00c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c639c700 1 -- 172.21.15.131:0/3228486705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8820 msgr2=0x7f46c0152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c639c700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8820 0x7f46c0152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c639c700 1 -- 172.21.15.131:0/3228486705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c0069d10 msgr2=0x7f46c00c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c639c700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c0069d10 0x7f46c00c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c639c700 1 -- 172.21.15.131:0/3228486705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8009ae0 con 0x7f46c00c8bf0 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.940+0000 7f46c639c700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c00c8bf0 0x7f46c00af0d0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f46b800b6c0 tx=0x7f46b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.941+0000 7f46b5ffb700 1 -- 172.21.15.131:0/3228486705 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b80032f0 con 0x7f46c00c8bf0 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.941+0000 7f46b5ffb700 1 -- 172.21.15.131:0/3228486705 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b8025460 con 0x7f46c00c8bf0 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.941+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c00c6090 con 0x7f46c00c8bf0 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.941+0000 7f46b5ffb700 1 -- 172.21.15.131:0/3228486705 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8025800 con 0x7f46c00c8bf0 2023-12-26T21:56:13.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.941+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c00c6370 con 0x7f46c00c8bf0 2023-12-26T21:56:13.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.942+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c015c260 con 0x7f46c00c8bf0 2023-12-26T21:56:13.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.943+0000 7f46b5ffb700 1 -- 172.21.15.131:0/3228486705 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f46b8003040 con 0x7f46c00c8bf0 2023-12-26T21:56:13.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.944+0000 7f46b5ffb700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f46a4066f30 0x7f46a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:13.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.944+0000 7f46b5ffb700 1 -- 172.21.15.131:0/3228486705 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f46b80863a0 con 0x7f46c00c8bf0 2023-12-26T21:56:13.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.944+0000 7f46c5b9b700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f46a4066f30 0x7f46a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:13.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.945+0000 7f46c5b9b700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f46a4066f30 0x7f46a40693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f46bc003860 tx=0x7f46bc009040 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:13.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:13.948+0000 7f46b5ffb700 1 -- 172.21.15.131:0/3228486705 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46c015c260 con 0x7f46c00c8bf0 2023-12-26T21:56:14.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.139+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f46c015c230 con 0x7f46c00c8bf0 2023-12-26T21:56:14.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.144+0000 7f46b5ffb700 1 -- 172.21.15.131:0/3228486705 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v45) v1 ==== 120+0+0 (secure 0 0 0) 0x7f46b8053470 con 0x7f46c00c8bf0 2023-12-26T21:56:14.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.147+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f46a4066f30 msgr2=0x7f46a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:14.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.147+0000 7f46c7dff700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f46a4066f30 0x7f46a40693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f46bc003860 tx=0x7f46bc009040 comp rx=0 tx=0).stop 2023-12-26T21:56:14.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.147+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c00c8bf0 msgr2=0x7f46c00af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:14.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.147+0000 7f46c7dff700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c00c8bf0 0x7f46c00af0d0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f46b800b6c0 tx=0x7f46b8018000 comp rx=0 tx=0).stop 2023-12-26T21:56:14.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.148+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 shutdown_connections 2023-12-26T21:56:14.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.148+0000 7f46c7dff700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f46a4066f30 0x7f46a40693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.148+0000 7f46c7dff700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c00c8bf0 0x7f46c00af0d0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.148+0000 7f46c7dff700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c0069d10 0x7f46c00c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.148+0000 7f46c7dff700 1 --2- 172.21.15.131:0/3228486705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46c00c8820 0x7f46c0152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.148+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 >> 172.21.15.131:0/3228486705 conn(0x7f46c0068040 msgr2=0x7f46c0155cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:14.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.148+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 shutdown_connections 2023-12-26T21:56:14.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.148+0000 7f46c7dff700 1 -- 172.21.15.131:0/3228486705 wait complete. 2023-12-26T21:56:14.167 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-12-26T21:56:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.356+0000 7fd5df3ff700 1 Processor -- start 2023-12-26T21:56:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.356+0000 7fd5df3ff700 1 -- start start 2023-12-26T21:56:14.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.357+0000 7fd5df3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5d80cb320 0x7fd5d8069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:14.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5df3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5d806a1e0 0x7fd5d80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:14.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5df3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d8156940 con 0x7fd5d80b0c90 2023-12-26T21:56:14.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5df3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d80c8cc0 con 0x7fd5d80cb320 2023-12-26T21:56:14.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5df3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d80ad700 con 0x7fd5d806a1e0 2023-12-26T21:56:14.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5dd19b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5d80cb320 0x7fd5d8069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5dd19b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5d80cb320 0x7fd5d8069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57300/0 (socket says 172.21.15.131:57300) 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5dd19b700 1 -- 172.21.15.131:0/3773151497 learned_addr learned my addr 172.21.15.131:0/3773151497 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5dc99a700 1 --2- 172.21.15.131:0/3773151497 >> v2:172.21.15.190:3300/0 conn(0x7fd5d806a1e0 0x7fd5d80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5dd19b700 1 -- 172.21.15.131:0/3773151497 >> v2:172.21.15.190:3300/0 conn(0x7fd5d806a1e0 msgr2=0x7fd5d80b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5dd19b700 1 --2- 172.21.15.131:0/3773151497 >> v2:172.21.15.190:3300/0 conn(0x7fd5d806a1e0 0x7fd5d80b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5dd19b700 1 -- 172.21.15.131:0/3773151497 >> v1:172.21.15.190:6789/0 conn(0x7fd5d80b0c90 legacy=0x7fd5d80aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.358+0000 7fd5dd19b700 1 -- 172.21.15.131:0/3773151497 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80ae500 con 0x7fd5d80cb320 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.359+0000 7fd5dd19b700 1 --2- 172.21.15.131:0/3773151497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 0x7fd5d8069ca0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8064700 tx=0x7fd5d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b356e2c74d35e65 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.359+0000 7fd5cffff700 1 -- 172.21.15.131:0/3773151497 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d400e070 con 0x7fd5d80cb320 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.359+0000 7fd5cffff700 1 -- 172.21.15.131:0/3773151497 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d400ba40 con 0x7fd5d80cb320 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.359+0000 7fd5cffff700 1 -- 172.21.15.131:0/3773151497 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40092e0 con 0x7fd5d80cb320 2023-12-26T21:56:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.359+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3773151497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 msgr2=0x7fd5d8069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.359+0000 7fd5df3ff700 1 --2- 172.21.15.131:0/3773151497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 0x7fd5d8069ca0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8064700 tx=0x7fd5d4009840 comp rx=0 tx=0).stop 2023-12-26T21:56:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.360+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3773151497 shutdown_connections 2023-12-26T21:56:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.360+0000 7fd5df3ff700 1 --2- 172.21.15.131:0/3773151497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 0x7fd5d8069ca0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.360+0000 7fd5df3ff700 1 --2- 172.21.15.131:0/3773151497 >> v2:172.21.15.190:3300/0 conn(0x7fd5d806a1e0 0x7fd5d80b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.360+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3773151497 >> 172.21.15.131:0/3773151497 conn(0x7fd5d8068040 msgr2=0x7fd5d8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.360+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3773151497 shutdown_connections 2023-12-26T21:56:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.360+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3773151497 wait complete. 2023-12-26T21:56:14.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.361+0000 7fd5df3ff700 1 Processor -- start 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.361+0000 7fd5df3ff700 1 -- start start 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.362+0000 7fd5df3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d806a1e0 0x7fd5d80accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.362+0000 7fd5df3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d80b0c90 0x7fd5d8144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.362+0000 7fd5df3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 0x7fd5d815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.362+0000 7fd5df3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8146020 con 0x7fd5d80cb320 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.362+0000 7fd5df3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8145410 con 0x7fd5d80b0c90 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.362+0000 7fd5df3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81456f0 con 0x7fd5d806a1e0 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.362+0000 7fd5dd99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 0x7fd5d815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dd99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 0x7fd5d815be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57316/0 (socket says 172.21.15.131:57316) 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dd99c700 1 -- 172.21.15.131:0/3203340783 learned_addr learned my addr 172.21.15.131:0/3203340783 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dc99a700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d80b0c90 0x7fd5d8144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dd19b700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d806a1e0 0x7fd5d80accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:14.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dd99c700 1 -- 172.21.15.131:0/3203340783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d806a1e0 msgr2=0x7fd5d80accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dd99c700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d806a1e0 0x7fd5d80accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dd99c700 1 -- 172.21.15.131:0/3203340783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d80b0c90 msgr2=0x7fd5d8144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dd99c700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d80b0c90 0x7fd5d8144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dd99c700 1 -- 172.21.15.131:0/3203340783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d4009ae0 con 0x7fd5d80cb320 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5dd99c700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 0x7fd5d815be50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000c560 tx=0x7fd5d0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5cd7fa700 1 -- 172.21.15.131:0/3203340783 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0011590 con 0x7fd5d80cb320 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5cd7fa700 1 -- 172.21.15.131:0/3203340783 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d00116f0 con 0x7fd5d80cb320 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d81459d0 con 0x7fd5d80cb320 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.363+0000 7fd5cd7fa700 1 -- 172.21.15.131:0/3203340783 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0011ab0 con 0x7fd5d80cb320 2023-12-26T21:56:14.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.364+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d80ad200 con 0x7fd5d80cb320 2023-12-26T21:56:14.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.364+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d80657e0 con 0x7fd5d80cb320 2023-12-26T21:56:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.365+0000 7fd5cd7fa700 1 -- 172.21.15.131:0/3203340783 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd5d001b3b0 con 0x7fd5d80cb320 2023-12-26T21:56:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.366+0000 7fd5cd7fa700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5bc066f30 0x7fd5bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.366+0000 7fd5cd7fa700 1 -- 172.21.15.131:0/3203340783 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5961+0+0 (secure 0 0 0) 0x7fd5d0083a90 con 0x7fd5d80cb320 2023-12-26T21:56:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.366+0000 7fd5dd19b700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5bc066f30 0x7fd5bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:14.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.367+0000 7fd5dd19b700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5bc066f30 0x7fd5bc0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009880 tx=0x7fd5d4018000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:14.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.370+0000 7fd5cd7fa700 1 -- 172.21.15.131:0/3203340783 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5d0050b60 con 0x7fd5d80cb320 2023-12-26T21:56:14.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.647+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fd5d8146160 con 0x7fd5d80cb320 2023-12-26T21:56:14.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.866+0000 7fd5cd7fa700 1 -- 172.21.15.131:0/3203340783 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v732) v1 ==== 148+0+0 (secure 0 0 0) 0x7fd5d0017070 con 0x7fd5d80cb320 2023-12-26T21:56:14.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.868+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5bc066f30 msgr2=0x7fd5bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.868+0000 7fd5df3ff700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5bc066f30 0x7fd5bc0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009880 tx=0x7fd5d4018000 comp rx=0 tx=0).stop 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.868+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 msgr2=0x7fd5d815be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.868+0000 7fd5df3ff700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 0x7fd5d815be50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000c560 tx=0x7fd5d0007ef0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.869+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 shutdown_connections 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.869+0000 7fd5df3ff700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd5bc066f30 0x7fd5bc0693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.869+0000 7fd5df3ff700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5d80cb320 0x7fd5d815be50 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.869+0000 7fd5df3ff700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5d806a1e0 0x7fd5d80accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.869+0000 7fd5df3ff700 1 --2- 172.21.15.131:0/3203340783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5d80b0c90 0x7fd5d8144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.869+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 >> 172.21.15.131:0/3203340783 conn(0x7fd5d8068040 msgr2=0x7fd5d8155710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.869+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 shutdown_connections 2023-12-26T21:56:14.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:14.869+0000 7fd5df3ff700 1 -- 172.21.15.131:0/3203340783 wait complete. 2023-12-26T21:56:14.887 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-12-26T21:56:14.887 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-12-26T21:56:14.888 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T21:56:14.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-12-26T21:56:15.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.077+0000 7f0f7d1ff700 1 Processor -- start 2023-12-26T21:56:15.078 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.077+0000 7f0f7d1ff700 1 -- start start 2023-12-26T21:56:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.078+0000 7f0f7d1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f780cb320 0x7f0f78069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.078+0000 7f0f7d1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0f7806a1e0 0x7f0f780b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.078+0000 7f0f7d1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f78156940 con 0x7f0f780b0c90 2023-12-26T21:56:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.078+0000 7f0f7d1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f780c8cc0 con 0x7f0f7806a1e0 2023-12-26T21:56:15.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.078+0000 7f0f7d1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f780ad700 con 0x7f0f780cb320 2023-12-26T21:56:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.079+0000 7f0f76d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f780cb320 0x7f0f78069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.079+0000 7f0f7659c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0f7806a1e0 0x7f0f780b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:15.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.079+0000 7f0f76d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f780cb320 0x7f0f78069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57528/0 (socket says 172.21.15.131:57528) 2023-12-26T21:56:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.079+0000 7f0f76d9d700 1 -- 172.21.15.131:0/2653071183 learned_addr learned my addr 172.21.15.131:0/2653071183 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.079+0000 7f0f7659c700 1 -- 172.21.15.131:0/2653071183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f780cb320 msgr2=0x7f0f78069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.079+0000 7f0f7659c700 1 --2- 172.21.15.131:0/2653071183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f780cb320 0x7f0f78069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.079+0000 7f0f7659c700 1 -- 172.21.15.131:0/2653071183 >> v1:172.21.15.190:6789/0 conn(0x7f0f780b0c90 legacy=0x7f0f780aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.079+0000 7f0f7659c700 1 -- 172.21.15.131:0/2653071183 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f780ae500 con 0x7f0f7806a1e0 2023-12-26T21:56:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f7659c700 1 --2- 172.21.15.131:0/2653071183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f7806a1e0 0x7f0f780b0750 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0098e0 tx=0x7f0f6c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54299c671f1147b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f75d9b700 1 -- 172.21.15.131:0/2653071183 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00b640 con 0x7f0f7806a1e0 2023-12-26T21:56:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f75d9b700 1 -- 172.21.15.131:0/2653071183 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c00b7a0 con 0x7f0f7806a1e0 2023-12-26T21:56:15.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f75d9b700 1 -- 172.21.15.131:0/2653071183 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00bb40 con 0x7f0f7806a1e0 2023-12-26T21:56:15.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/2653071183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f7806a1e0 msgr2=0x7f0f780b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:15.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f7d1ff700 1 --2- 172.21.15.131:0/2653071183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f7806a1e0 0x7f0f780b0750 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0098e0 tx=0x7f0f6c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/2653071183 shutdown_connections 2023-12-26T21:56:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f7d1ff700 1 --2- 172.21.15.131:0/2653071183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f780cb320 0x7f0f78069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f7d1ff700 1 --2- 172.21.15.131:0/2653071183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f7806a1e0 0x7f0f780b0750 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.080+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/2653071183 >> 172.21.15.131:0/2653071183 conn(0x7f0f78068040 msgr2=0x7f0f78069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.081+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/2653071183 shutdown_connections 2023-12-26T21:56:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.081+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/2653071183 wait complete. 2023-12-26T21:56:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.082+0000 7f0f7d1ff700 1 Processor -- start 2023-12-26T21:56:15.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.082+0000 7f0f7d1ff700 1 -- start start 2023-12-26T21:56:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f7d1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806a1e0 0x7f0f7814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f7d1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f780b0c90 0x7f0f78158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f7d1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f780cb320 0x7f0f781593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f7d1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f780c6150 con 0x7f0f780b0c90 2023-12-26T21:56:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f7d1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f780c65e0 con 0x7f0f780cb320 2023-12-26T21:56:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f7d1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f780c5540 con 0x7f0f7806a1e0 2023-12-26T21:56:15.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f7659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f780b0c90 0x7f0f78158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f7659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f780b0c90 0x7f0f78158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58192/0 (socket says 172.21.15.131:58192) 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f7659c700 1 -- 172.21.15.131:0/984832687 learned_addr learned my addr 172.21.15.131:0/984832687 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.083+0000 7f0f76d9d700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806a1e0 0x7f0f7814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f7759e700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f780cb320 0x7f0f781593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f7659c700 1 -- 172.21.15.131:0/984832687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806a1e0 msgr2=0x7f0f7814fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f7659c700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806a1e0 0x7f0f7814fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f7659c700 1 -- 172.21.15.131:0/984832687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f780cb320 msgr2=0x7f0f781593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f7659c700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f780cb320 0x7f0f781593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f7659c700 1 -- 172.21.15.131:0/984832687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c009420 con 0x7f0f780b0c90 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f7659c700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f780b0c90 0x7f0f78158ea0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00a010 tx=0x7f0f6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f5effd700 1 -- 172.21.15.131:0/984832687 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c01f070 con 0x7f0f780b0c90 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f5effd700 1 -- 172.21.15.131:0/984832687 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c01aab0 con 0x7f0f780b0c90 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f780c5820 con 0x7f0f780b0c90 2023-12-26T21:56:15.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f5effd700 1 -- 172.21.15.131:0/984832687 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c01ae50 con 0x7f0f780b0c90 2023-12-26T21:56:15.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.084+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f780c5de0 con 0x7f0f780b0c90 2023-12-26T21:56:15.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.085+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f780657e0 con 0x7f0f780b0c90 2023-12-26T21:56:15.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.086+0000 7f0f5effd700 1 -- 172.21.15.131:0/984832687 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0f6c023070 con 0x7f0f780b0c90 2023-12-26T21:56:15.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.087+0000 7f0f5effd700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0f64066f30 0x7f0f640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:15.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.087+0000 7f0f5effd700 1 -- 172.21.15.131:0/984832687 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f0f6c085830 con 0x7f0f780b0c90 2023-12-26T21:56:15.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.087+0000 7f0f76d9d700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0f64066f30 0x7f0f640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:15.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.088+0000 7f0f76d9d700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0f64066f30 0x7f0f640693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f6800b4f0 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:15.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.091+0000 7f0f5effd700 1 -- 172.21.15.131:0/984832687 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f6c052900 con 0x7f0f780b0c90 2023-12-26T21:56:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.361+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f0f780c9ff0 con 0x7f0f780b0c90 2023-12-26T21:56:15.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.870+0000 7f0f5effd700 1 -- 172.21.15.131:0/984832687 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v733) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0f6c055f20 con 0x7f0f780b0c90 2023-12-26T21:56:15.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0f64066f30 msgr2=0x7f0f640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:15.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0f64066f30 0x7f0f640693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f6800b4f0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f780b0c90 msgr2=0x7f0f78158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:15.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f780b0c90 0x7f0f78158ea0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00a010 tx=0x7f0f6c017000 comp rx=0 tx=0).stop 2023-12-26T21:56:15.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 shutdown_connections 2023-12-26T21:56:15.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f780cb320 0x7f0f781593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806a1e0 0x7f0f7814fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0f64066f30 0x7f0f640693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 --2- 172.21.15.131:0/984832687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f780b0c90 0x7f0f78158ea0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:15.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 >> 172.21.15.131:0/984832687 conn(0x7f0f78068040 msgr2=0x7f0f7806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:15.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 shutdown_connections 2023-12-26T21:56:15.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:15.873+0000 7f0f7d1ff700 1 -- 172.21.15.131:0/984832687 wait complete. 2023-12-26T21:56:15.894 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T21:56:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.089+0000 7fdd5abff700 1 Processor -- start 2023-12-26T21:56:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.089+0000 7fdd5abff700 1 -- start start 2023-12-26T21:56:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.090+0000 7fdd5abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd5406bfd0 0x7fdd540c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.090+0000 7fdd5abff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd540686c0 con 0x7fdd540cc100 2023-12-26T21:56:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.090+0000 7fdd5abff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd54069520 con 0x7fdd5413c510 2023-12-26T21:56:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.090+0000 7fdd5abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54068910 con 0x7fdd5406bfd0 2023-12-26T21:56:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd5919c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdd540cc100 0x7fdd540cc580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43102/0 (socket says 172.21.15.131:43102) 2023-12-26T21:56:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd5919c700 1 -- 172.21.15.131:0/889667389 learned_addr learned my addr 172.21.15.131:0/889667389 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd53fff700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd5406bfd0 0x7fdd540c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989247001 0 0) 0x7fdd540686c0 con 0x7fdd540cc100 2023-12-26T21:56:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd44003570 con 0x7fdd540cc100 2023-12-26T21:56:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213870655 0 0) 0x7fdd54069520 con 0x7fdd5413c510 2023-12-26T21:56:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd540686c0 con 0x7fdd5413c510 2023-12-26T21:56:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2950804367 0 0) 0x7fdd44003570 con 0x7fdd540cc100 2023-12-26T21:56:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd5406bfd0 msgr2=0x7fdd540c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd5406bfd0 0x7fdd540c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 >> v1:172.21.15.190:6789/0 conn(0x7fdd5413c510 legacy=0x7fdd5406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd54068bf0 con 0x7fdd540cc100 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.091+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fdd4c0017f0 con 0x7fdd540cc100 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 >> v1:172.21.15.131:6789/0 conn(0x7fdd540cc100 legacy=0x7fdd540cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd537fe700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd44003c20 0x7fdd44003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd537fe700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd44004810 0x7fdd44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd537fe700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd440051c0 0x7fdd4400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54068910 con 0x7fdd44003c20 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4400c050 con 0x7fdd440051c0 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4400c210 con 0x7fdd44004810 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd53fff700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd5406bfd0 0x7fdd540c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd5899b700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd44003c20 0x7fdd44003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd5919c700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd440051c0 0x7fdd4400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd53fff700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd44004810 0x7fdd44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd5899b700 1 -- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd44004810 msgr2=0x7fdd44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd5899b700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd44004810 0x7fdd44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd5899b700 1 -- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd440051c0 msgr2=0x7fdd4400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd5899b700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd440051c0 0x7fdd4400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd5899b700 1 -- 172.21.15.131:0/889667389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44003570 con 0x7fdd44003c20 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.092+0000 7fdd5919c700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd440051c0 0x7fdd4400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd5899b700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd44003c20 0x7fdd44003ff0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fdd48007fd0 tx=0x7fdd48007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4f1c5b74647872 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48001420 con 0x7fdd44003c20 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800c5a0 con 0x7fdd44003c20 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd537fe700 1 -- 172.21.15.131:0/889667389 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800c960 con 0x7fdd44003c20 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd5abff700 1 -- 172.21.15.131:0/889667389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd44003c20 msgr2=0x7fdd44003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd5abff700 1 --2- 172.21.15.131:0/889667389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd44003c20 0x7fdd44003ff0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fdd48007fd0 tx=0x7fdd48007d60 comp rx=0 tx=0).stop 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd5919c700 1 -- 172.21.15.131:0/889667389 reap_dead start 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd5abff700 1 -- 172.21.15.131:0/889667389 shutdown_connections 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd5abff700 1 -- 172.21.15.131:0/889667389 >> 172.21.15.131:0/889667389 conn(0x7fdd540ce3e0 msgr2=0x7fdd540ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd5abff700 1 -- 172.21.15.131:0/889667389 shutdown_connections 2023-12-26T21:56:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.093+0000 7fdd5abff700 1 -- 172.21.15.131:0/889667389 wait complete. 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.095+0000 7fdd5abff700 1 Processor -- start 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.096+0000 7fdd5abff700 1 -- start start 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5abff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd44003c20 0x7fdd54154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd54139f90 0x7fdd5415dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5abff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5406cdb0 0x7fdd5406d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5abff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5415f660 con 0x7fdd5406cdb0 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5abff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5406d770 con 0x7fdd44003c20 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5415f1d0 con 0x7fdd54139f90 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5406cdb0 0x7fdd5406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5406cdb0 0x7fdd5406d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58216/0 (socket says 172.21.15.131:58216) 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5919c700 1 -- 172.21.15.131:0/1718202569 learned_addr learned my addr 172.21.15.131:0/1718202569 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd53fff700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd54139f90 0x7fdd5415dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.097+0000 7fdd5899b700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd44003c20 0x7fdd54154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd5919c700 1 -- 172.21.15.131:0/1718202569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd54139f90 msgr2=0x7fdd5415dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd5919c700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd54139f90 0x7fdd5415dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd5919c700 1 -- 172.21.15.131:0/1718202569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd44003c20 msgr2=0x7fdd54154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd5919c700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd44003c20 0x7fdd54154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd5919c700 1 -- 172.21.15.131:0/1718202569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5415e5c0 con 0x7fdd5406cdb0 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd53fff700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd54139f90 0x7fdd5415dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd5919c700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5406cdb0 0x7fdd5406d230 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c008ba0 tx=0x7fdd4c001280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd50ff9700 1 -- 172.21.15.131:0/1718202569 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00fbf0 con 0x7fdd5406cdb0 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd50ff9700 1 -- 172.21.15.131:0/1718202569 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00fd70 con 0x7fdd5406cdb0 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd50ff9700 1 -- 172.21.15.131:0/1718202569 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00fee0 con 0x7fdd5406cdb0 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5415e8a0 con 0x7fdd5406cdb0 2023-12-26T21:56:16.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.098+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5415ee60 con 0x7fdd5406cdb0 2023-12-26T21:56:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.099+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5415f360 con 0x7fdd5406cdb0 2023-12-26T21:56:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.100+0000 7fdd50ff9700 1 -- 172.21.15.131:0/1718202569 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdd4c01f440 con 0x7fdd5406cdb0 2023-12-26T21:56:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.101+0000 7fdd50ff9700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fdd38066f30 0x7fdd380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.101+0000 7fdd50ff9700 1 -- 172.21.15.131:0/1718202569 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5961+0+0 (secure 0 0 0) 0x7fdd4c084870 con 0x7fdd5406cdb0 2023-12-26T21:56:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.101+0000 7fdd5899b700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fdd38066f30 0x7fdd380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.102+0000 7fdd5899b700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fdd38066f30 0x7fdd380693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdd48007fd0 tx=0x7fdd48015000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:16.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.105+0000 7fdd50ff9700 1 -- 172.21.15.131:0/1718202569 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd4c0519e0 con 0x7fdd5406cdb0 2023-12-26T21:56:16.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.297+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdd540657e0 con 0x7fdd5406cdb0 2023-12-26T21:56:16.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.302+0000 7fdd50ff9700 1 -- 172.21.15.131:0/1718202569 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdd4c01a070 con 0x7fdd5406cdb0 2023-12-26T21:56:16.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fdd38066f30 msgr2=0x7fdd380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fdd38066f30 0x7fdd380693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdd48007fd0 tx=0x7fdd48015000 comp rx=0 tx=0).stop 2023-12-26T21:56:16.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5406cdb0 msgr2=0x7fdd5406d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5406cdb0 0x7fdd5406d230 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c008ba0 tx=0x7fdd4c001280 comp rx=0 tx=0).stop 2023-12-26T21:56:16.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 shutdown_connections 2023-12-26T21:56:16.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fdd38066f30 0x7fdd380693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5406cdb0 0x7fdd5406d230 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd44003c20 0x7fdd54154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 --2- 172.21.15.131:0/1718202569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd54139f90 0x7fdd5415dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 >> 172.21.15.131:0/1718202569 conn(0x7fdd540ce3e0 msgr2=0x7fdd54156f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:16.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 shutdown_connections 2023-12-26T21:56:16.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.304+0000 7fdd5abff700 1 -- 172.21.15.131:0/1718202569 wait complete. 2023-12-26T21:56:16.322 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T21:56:16.323 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:56:16.409 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T21:56:16.515 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T21:56:16.515 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T21:56:16.515 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:56:16.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.703+0000 7f4e657ff700 1 Processor -- start 2023-12-26T21:56:16.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.704+0000 7f4e657ff700 1 -- start start 2023-12-26T21:56:16.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e657ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e601596a0 0x7f4e600b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e657ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e60159a70 0x7f4e600b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e657ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e600afcd0 con 0x7f4e600b0160 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e657ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e600b3e70 con 0x7f4e60159a70 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e657ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600b0cd0 con 0x7f4e601596a0 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e5e7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e60159a70 0x7f4e600b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e5e7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e60159a70 0x7f4e600b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58220/0 (socket says 172.21.15.131:58220) 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e5e7fc700 1 -- 172.21.15.131:0/2852965308 learned_addr learned my addr 172.21.15.131:0/2852965308 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e5effd700 1 --2- 172.21.15.131:0/2852965308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e601596a0 0x7f4e600b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e5e7fc700 1 -- 172.21.15.131:0/2852965308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e601596a0 msgr2=0x7f4e600b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e5e7fc700 1 --2- 172.21.15.131:0/2852965308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e601596a0 0x7f4e600b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e5e7fc700 1 -- 172.21.15.131:0/2852965308 >> v1:172.21.15.190:6789/0 conn(0x7f4e600b0160 legacy=0x7f4e600b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.705+0000 7f4e5e7fc700 1 -- 172.21.15.131:0/2852965308 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600b2e00 con 0x7f4e60159a70 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.706+0000 7f4e5e7fc700 1 --2- 172.21.15.131:0/2852965308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 0x7f4e600b28c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4e540099b0 tx=0x7f4e54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9705fb4bbb426e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:16.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.706+0000 7f4e5dffb700 1 -- 172.21.15.131:0/2852965308 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400b860 con 0x7f4e60159a70 2023-12-26T21:56:16.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.706+0000 7f4e5dffb700 1 -- 172.21.15.131:0/2852965308 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5400b9c0 con 0x7f4e60159a70 2023-12-26T21:56:16.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.706+0000 7f4e5dffb700 1 -- 172.21.15.131:0/2852965308 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400bd60 con 0x7f4e60159a70 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.706+0000 7f4e657ff700 1 -- 172.21.15.131:0/2852965308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 msgr2=0x7f4e600b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.707+0000 7f4e657ff700 1 --2- 172.21.15.131:0/2852965308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 0x7f4e600b28c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4e540099b0 tx=0x7f4e54009250 comp rx=0 tx=0).stop 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.707+0000 7f4e657ff700 1 -- 172.21.15.131:0/2852965308 shutdown_connections 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.707+0000 7f4e657ff700 1 --2- 172.21.15.131:0/2852965308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e601596a0 0x7f4e600b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.707+0000 7f4e657ff700 1 --2- 172.21.15.131:0/2852965308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 0x7f4e600b28c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.707+0000 7f4e657ff700 1 -- 172.21.15.131:0/2852965308 >> 172.21.15.131:0/2852965308 conn(0x7f4e6006ece0 msgr2=0x7f4e6006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.707+0000 7f4e657ff700 1 -- 172.21.15.131:0/2852965308 shutdown_connections 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.707+0000 7f4e657ff700 1 -- 172.21.15.131:0/2852965308 wait complete. 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.708+0000 7f4e657ff700 1 Processor -- start 2023-12-26T21:56:16.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.708+0000 7f4e657ff700 1 -- start start 2023-12-26T21:56:16.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.709+0000 7f4e657ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e600b0160 0x7f4e60169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.709+0000 7f4e657ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e601596a0 0x7f4e600cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.709+0000 7f4e657ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 0x7f4e60159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.709+0000 7f4e657ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600ccc00 con 0x7f4e60159a70 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.709+0000 7f4e657ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600cbf80 con 0x7f4e601596a0 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.709+0000 7f4e657ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600cc260 con 0x7f4e600b0160 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e600b0160 0x7f4e60169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e600b0160 0x7f4e60169b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:57584/0 (socket says 172.21.15.131:57584) 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5effd700 1 -- 172.21.15.131:0/443539094 learned_addr learned my addr 172.21.15.131:0/443539094 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5f7fe700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 0x7f4e60159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5e7fc700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e601596a0 0x7f4e600cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5f7fe700 1 -- 172.21.15.131:0/443539094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e600b0160 msgr2=0x7f4e60169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5f7fe700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e600b0160 0x7f4e60169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5f7fe700 1 -- 172.21.15.131:0/443539094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e601596a0 msgr2=0x7f4e600cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5f7fe700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e601596a0 0x7f4e600cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5f7fe700 1 -- 172.21.15.131:0/443539094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e540094f0 con 0x7f4e60159a70 2023-12-26T21:56:16.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e5f7fe700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 0x7f4e60159e60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800c830 tx=0x7f4e58001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:16.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e46ffd700 1 -- 172.21.15.131:0/443539094 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58012630 con 0x7f4e60159a70 2023-12-26T21:56:16.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e46ffd700 1 -- 172.21.15.131:0/443539094 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e58012790 con 0x7f4e60159a70 2023-12-26T21:56:16.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e46ffd700 1 -- 172.21.15.131:0/443539094 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58012b50 con 0x7f4e60159a70 2023-12-26T21:56:16.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.710+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e600cc540 con 0x7f4e60159a70 2023-12-26T21:56:16.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.711+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6016a0d0 con 0x7f4e60159a70 2023-12-26T21:56:16.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.712+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e600657e0 con 0x7f4e60159a70 2023-12-26T21:56:16.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.712+0000 7f4e46ffd700 1 -- 172.21.15.131:0/443539094 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4e58011380 con 0x7f4e60159a70 2023-12-26T21:56:16.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.713+0000 7f4e46ffd700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e4c066f30 0x7f4e4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:16.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.713+0000 7f4e46ffd700 1 -- 172.21.15.131:0/443539094 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f4e580847c0 con 0x7f4e60159a70 2023-12-26T21:56:16.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.714+0000 7f4e5effd700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e4c066f30 0x7f4e4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:16.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.714+0000 7f4e5effd700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e4c066f30 0x7f4e4c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4e50006fd0 tx=0x7f4e50008040 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:16.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.717+0000 7f4e46ffd700 1 -- 172.21.15.131:0/443539094 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e58051890 con 0x7f4e60159a70 2023-12-26T21:56:16.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.956+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e600ccd40 con 0x7f4e60159a70 2023-12-26T21:56:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.956+0000 7f4e46ffd700 1 -- 172.21.15.131:0/443539094 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f4e600ccd40 con 0x7f4e60159a70 2023-12-26T21:56:16.957 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:16.957 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6623,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3234851896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3234851896},{"type":"v1","addr":"172.21.15.131:6834","nonce":3234851896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2363679591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2363679591},{"type":"v1","addr":"172.21.15.190:6834","nonce":2363679591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2089888777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2089888777},{"type":"v1","addr":"172.21.15.131:6835","nonce":2089888777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3018492544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3018492544},{"type":"v1","addr":"172.21.15.190:6835","nonce":3018492544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[]} 2023-12-26T21:56:16.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.958+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e4c066f30 msgr2=0x7f4e4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e4c066f30 0x7f4e4c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4e50006fd0 tx=0x7f4e50008040 comp rx=0 tx=0).stop 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 msgr2=0x7f4e60159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 0x7f4e60159e60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800c830 tx=0x7f4e58001690 comp rx=0 tx=0).stop 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 shutdown_connections 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4e4c066f30 0x7f4e4c0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e60159a70 0x7f4e60159e60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e600b0160 0x7f4e60169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 --2- 172.21.15.131:0/443539094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e601596a0 0x7f4e600cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 >> 172.21.15.131:0/443539094 conn(0x7f4e6006ece0 msgr2=0x7f4e600d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 shutdown_connections 2023-12-26T21:56:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:16.959+0000 7f4e657ff700 1 -- 172.21.15.131:0/443539094 wait complete. 2023-12-26T21:56:16.961 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 59 2023-12-26T21:56:16.978 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T21:56:16.978 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T21:56:16.978 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T21:56:16.978 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:56:16.978 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T21:56:16.978 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T21:56:16.979 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T21:56:16.979 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:56:16.979 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T21:56:16.979 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T21:56:16.979 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T21:56:16.979 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:56:16.979 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T21:56:16.979 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T21:56:16.980 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T21:56:16.980 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T21:56:16.995 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T21:56:16.996 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T21:56:16.996 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T21:56:16.997 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T21:56:16.998 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T21:56:16.999 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T21:56:17.040 INFO:tasks.ceph.mds.b:Started 2023-12-26T21:56:17.041 DEBUG:teuthology.parallel:result is None 2023-12-26T21:56:17.044 INFO:tasks.ceph.mds.c:Started 2023-12-26T21:56:17.044 INFO:tasks.ceph.mds.d:Started 2023-12-26T21:56:17.044 DEBUG:teuthology.parallel:result is None 2023-12-26T21:56:17.044 DEBUG:teuthology.parallel:result is None 2023-12-26T21:56:17.179 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T21:56:17.186 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T21:56:17.186 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T21:56:19.267 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:56:19.272 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:56:22.998 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T21:56:22.998 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T21:56:23.004 INFO:tasks.ceph.mds.a:Started 2023-12-26T21:56:23.004 DEBUG:teuthology.parallel:result is None 2023-12-26T21:56:23.005 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T21:56:23.153 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T21:56:23.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-12-26T21:56:23.211 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.210+0000 7fcf47fff700 1 Processor -- start 2023-12-26T21:56:23.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.211+0000 7fcf47fff700 1 -- start start 2023-12-26T21:56:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.212+0000 7fcf47fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf400c8820 0x7fcf400ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.212+0000 7fcf47fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400b19f0 0x7fcf40161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.212+0000 7fcf47fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf40157a80 con 0x7fcf400c8bf0 2023-12-26T21:56:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.212+0000 7fcf47fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf400cac50 con 0x7fcf400c8820 2023-12-26T21:56:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.212+0000 7fcf47fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400caf30 con 0x7fcf400b19f0 2023-12-26T21:56:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.212+0000 7fcf45d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf400c8820 0x7fcf400ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.212+0000 7fcf45d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf400c8820 0x7fcf400ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58274/0 (socket says 172.21.15.131:58274) 2023-12-26T21:56:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.212+0000 7fcf45d9b700 1 -- 172.21.15.131:0/3026004716 learned_addr learned my addr 172.21.15.131:0/3026004716 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.212+0000 7fcf4559a700 1 --1- 172.21.15.131:0/3026004716 >> v1:172.21.15.131:6789/0 conn(0x7fcf400c8bf0 0x7fcf400b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43116/0 (socket says 172.21.15.131:43116) 2023-12-26T21:56:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf4659c700 1 --2- 172.21.15.131:0/3026004716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400b19f0 0x7fcf40161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf45d9b700 1 -- 172.21.15.131:0/3026004716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400b19f0 msgr2=0x7fcf40161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf45d9b700 1 --2- 172.21.15.131:0/3026004716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400b19f0 0x7fcf40161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf45d9b700 1 -- 172.21.15.131:0/3026004716 >> v1:172.21.15.131:6789/0 conn(0x7fcf400c8bf0 legacy=0x7fcf400b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf45d9b700 1 -- 172.21.15.131:0/3026004716 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf400cb210 con 0x7fcf400c8820 2023-12-26T21:56:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf44d99700 1 -- 172.21.15.131:0/3026004716 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130795480 0 0) 0x7fcf40157a80 con 0x7fcf400c8bf0 2023-12-26T21:56:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf4659c700 1 --2- 172.21.15.131:0/3026004716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400b19f0 0x7fcf40161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T21:56:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf45d9b700 1 --2- 172.21.15.131:0/3026004716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8820 0x7fcf400ca300 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0099b0 tx=0x7fcf3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7724cd84c5d039e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf44d99700 1 -- 172.21.15.131:0/3026004716 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00b860 con 0x7fcf400c8820 2023-12-26T21:56:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf44d99700 1 -- 172.21.15.131:0/3026004716 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c00b9c0 con 0x7fcf400c8820 2023-12-26T21:56:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.213+0000 7fcf44d99700 1 -- 172.21.15.131:0/3026004716 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00bd60 con 0x7fcf400c8820 2023-12-26T21:56:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.214+0000 7fcf47fff700 1 -- 172.21.15.131:0/3026004716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8820 msgr2=0x7fcf400ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.214+0000 7fcf47fff700 1 --2- 172.21.15.131:0/3026004716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8820 0x7fcf400ca300 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0099b0 tx=0x7fcf3c009250 comp rx=0 tx=0).stop 2023-12-26T21:56:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.214+0000 7fcf47fff700 1 -- 172.21.15.131:0/3026004716 shutdown_connections 2023-12-26T21:56:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.214+0000 7fcf47fff700 1 --2- 172.21.15.131:0/3026004716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400b19f0 0x7fcf40161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.214+0000 7fcf47fff700 1 --2- 172.21.15.131:0/3026004716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8820 0x7fcf400ca300 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:23.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.214+0000 7fcf47fff700 1 -- 172.21.15.131:0/3026004716 >> 172.21.15.131:0/3026004716 conn(0x7fcf40068040 msgr2=0x7fcf40069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:23.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.214+0000 7fcf47fff700 1 -- 172.21.15.131:0/3026004716 shutdown_connections 2023-12-26T21:56:23.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.214+0000 7fcf47fff700 1 -- 172.21.15.131:0/3026004716 wait complete. 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.215+0000 7fcf47fff700 1 Processor -- start 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.216+0000 7fcf47fff700 1 -- start start 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.216+0000 7fcf47fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf400b19f0 0x7fcf400af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.216+0000 7fcf47fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400c8820 0x7fcf40149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.216+0000 7fcf47fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8bf0 0x7fcf400c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.216+0000 7fcf47fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40169380 con 0x7fcf400c8bf0 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.216+0000 7fcf47fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400afc90 con 0x7fcf400b19f0 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.216+0000 7fcf47fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400aff70 con 0x7fcf400c8820 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8bf0 0x7fcf400c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8bf0 0x7fcf400c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58276/0 (socket says 172.21.15.131:58276) 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4659c700 1 -- 172.21.15.131:0/2330473899 learned_addr learned my addr 172.21.15.131:0/2330473899 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4559a700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400c8820 0x7fcf40149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf45d9b700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf400b19f0 0x7fcf400af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4659c700 1 -- 172.21.15.131:0/2330473899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400c8820 msgr2=0x7fcf40149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4659c700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400c8820 0x7fcf40149c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4659c700 1 -- 172.21.15.131:0/2330473899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf400b19f0 msgr2=0x7fcf400af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4659c700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf400b19f0 0x7fcf400af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4659c700 1 -- 172.21.15.131:0/2330473899 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0094f0 con 0x7fcf400c8bf0 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf4659c700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8bf0 0x7fcf400c6fb0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800ce40 tx=0x7fcf380077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.217+0000 7fcf35ffb700 1 -- 172.21.15.131:0/2330473899 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800adf0 con 0x7fcf400c8bf0 2023-12-26T21:56:23.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.218+0000 7fcf35ffb700 1 -- 172.21.15.131:0/2330473899 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf38016420 con 0x7fcf400c8bf0 2023-12-26T21:56:23.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.218+0000 7fcf35ffb700 1 -- 172.21.15.131:0/2330473899 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf380167e0 con 0x7fcf400c8bf0 2023-12-26T21:56:23.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.218+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf400b0250 con 0x7fcf400c8bf0 2023-12-26T21:56:23.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.218+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf400af9a0 con 0x7fcf400c8bf0 2023-12-26T21:56:23.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.219+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf400657e0 con 0x7fcf400c8bf0 2023-12-26T21:56:23.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.220+0000 7fcf35ffb700 1 -- 172.21.15.131:0/2330473899 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcf38016bd0 con 0x7fcf400c8bf0 2023-12-26T21:56:23.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.221+0000 7fcf35ffb700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf24066f30 0x7fcf240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:23.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.221+0000 7fcf35ffb700 1 -- 172.21.15.131:0/2330473899 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5961+0+0 (secure 0 0 0) 0x7fcf38012070 con 0x7fcf400c8bf0 2023-12-26T21:56:23.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.222+0000 7fcf45d9b700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf24066f30 0x7fcf240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:23.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.222+0000 7fcf45d9b700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf24066f30 0x7fcf240693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0099b0 tx=0x7fcf3c017000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:23.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.226+0000 7fcf35ffb700 1 -- 172.21.15.131:0/2330473899 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf3804e5a0 con 0x7fcf400c8bf0 2023-12-26T21:56:23.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:23.438+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcf401694c0 con 0x7fcf400c8bf0 2023-12-26T21:56:24.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.286+0000 7fcf35ffb700 1 -- 172.21.15.131:0/2330473899 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v135) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcf38088030 con 0x7fcf400c8bf0 2023-12-26T21:56:24.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf24066f30 msgr2=0x7fcf240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:24.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf24066f30 0x7fcf240693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0099b0 tx=0x7fcf3c017000 comp rx=0 tx=0).stop 2023-12-26T21:56:24.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8bf0 msgr2=0x7fcf400c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:24.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8bf0 0x7fcf400c6fb0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800ce40 tx=0x7fcf380077d0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 shutdown_connections 2023-12-26T21:56:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf24066f30 0x7fcf240693e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf400c8bf0 0x7fcf400c6fb0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf400b19f0 0x7fcf400af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 --2- 172.21.15.131:0/2330473899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf400c8820 0x7fcf40149c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.289+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 >> 172.21.15.131:0/2330473899 conn(0x7fcf40068040 msgr2=0x7fcf400c1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.290+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 shutdown_connections 2023-12-26T21:56:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.290+0000 7fcf47fff700 1 -- 172.21.15.131:0/2330473899 wait complete. 2023-12-26T21:56:24.291 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T21:56:24.310 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T21:56:24.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.506+0000 7f049e3ff700 1 Processor -- start 2023-12-26T21:56:24.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.506+0000 7f049e3ff700 1 -- start start 2023-12-26T21:56:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.507+0000 7f049e3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04980c8820 0x7f04980ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.507+0000 7f049e3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04980b19f0 0x7f0498161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f049e3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0498157a80 con 0x7f04980c8bf0 2023-12-26T21:56:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f049e3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04980cac50 con 0x7f04980c8820 2023-12-26T21:56:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f049e3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04980caf30 con 0x7f04980b19f0 2023-12-26T21:56:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f04977fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f04980c8bf0 0x7f04980b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43124/0 (socket says 172.21.15.131:43124) 2023-12-26T21:56:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f04977fe700 1 -- 172.21.15.131:0/452555045 learned_addr learned my addr 172.21.15.131:0/452555045 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:24.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f0497fff700 1 --2- 172.21.15.131:0/452555045 >> v2:172.21.15.190:3300/0 conn(0x7f04980c8820 0x7f04980ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f049c99c700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04980b19f0 0x7f0498161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106145167 0 0) 0x7f0498157a80 con 0x7f04980c8bf0 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0480003570 con 0x7f04980c8bf0 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977240767 0 0) 0x7f0480003570 con 0x7f04980c8bf0 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04980b19f0 msgr2=0x7f0498161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f0496ffd700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04980b19f0 0x7f0498161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 >> v2:172.21.15.190:3300/0 conn(0x7f04980c8820 msgr2=0x7f04980ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.508+0000 7f0496ffd700 1 --2- 172.21.15.131:0/452555045 >> v2:172.21.15.190:3300/0 conn(0x7f04980c8820 0x7f04980ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04980cb210 con 0x7f04980c8bf0 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f048c001470 con 0x7f04980c8bf0 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 >> v1:172.21.15.131:6789/0 conn(0x7f04980c8bf0 legacy=0x7f04980b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0496ffd700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f0480003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0496ffd700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04980b19f0 0x7f0480004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0496ffd700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0480004d50 0x7f04800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04980cac50 con 0x7f0480003a60 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04980caf30 con 0x7f0480004d50 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048000ba60 con 0x7f04980b19f0 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0497fff700 1 --2- 172.21.15.131:0/452555045 >> v2:172.21.15.190:3300/0 conn(0x7f04980c8820 0x7f04980ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f049c99c700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0480004d50 0x7f04800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0497fff700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f0480003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f04977fe700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04980b19f0 0x7f0480004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0497fff700 1 -- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04980b19f0 msgr2=0x7f0480004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0497fff700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04980b19f0 0x7f0480004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0497fff700 1 -- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0480004d50 msgr2=0x7f04800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0497fff700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0480004d50 0x7f04800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f0497fff700 1 -- 172.21.15.131:0/452555045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480003570 con 0x7f0480003a60 2023-12-26T21:56:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.509+0000 7f049c99c700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0480004d50 0x7f04800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.510+0000 7f0497fff700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f0480003e70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f0488009db0 tx=0x7f04880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd166b13214e1ff server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.510+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488009740 con 0x7f0480003a60 2023-12-26T21:56:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.510+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048800e8c0 con 0x7f0480003a60 2023-12-26T21:56:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.510+0000 7f0496ffd700 1 -- 172.21.15.131:0/452555045 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800ec40 con 0x7f0480003a60 2023-12-26T21:56:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.510+0000 7f049e3ff700 1 -- 172.21.15.131:0/452555045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 msgr2=0x7f0480003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:24.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.510+0000 7f049e3ff700 1 --2- 172.21.15.131:0/452555045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f0480003e70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f0488009db0 tx=0x7f04880091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.510+0000 7f049c99c700 1 -- 172.21.15.131:0/452555045 reap_dead start 2023-12-26T21:56:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.510+0000 7f049e3ff700 1 -- 172.21.15.131:0/452555045 shutdown_connections 2023-12-26T21:56:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.510+0000 7f049e3ff700 1 -- 172.21.15.131:0/452555045 >> 172.21.15.131:0/452555045 conn(0x7f0498068040 msgr2=0x7f0498069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.511+0000 7f049e3ff700 1 -- 172.21.15.131:0/452555045 shutdown_connections 2023-12-26T21:56:24.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.511+0000 7f049e3ff700 1 -- 172.21.15.131:0/452555045 wait complete. 2023-12-26T21:56:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.512+0000 7f049e3ff700 1 Processor -- start 2023-12-26T21:56:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.512+0000 7f049e3ff700 1 -- start start 2023-12-26T21:56:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.513+0000 7f049e3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f049815b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.513+0000 7f049e3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0498155ea0 0x7f04981443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.513+0000 7f049e3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f049815a680 0x7f049815aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.513+0000 7f049e3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049816d8c0 con 0x7f0480003a60 2023-12-26T21:56:24.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.513+0000 7f049e3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049815b030 con 0x7f049815a680 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.513+0000 7f049e3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049815beb0 con 0x7f0498155ea0 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.513+0000 7f0497fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f049815b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.513+0000 7f0497fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f049815b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58286/0 (socket says 172.21.15.131:58286) 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.513+0000 7f0497fff700 1 -- 172.21.15.131:0/2312844448 learned_addr learned my addr 172.21.15.131:0/2312844448 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f049c99c700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f049815a680 0x7f049815aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f04977fe700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0498155ea0 0x7f04981443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f0497fff700 1 -- 172.21.15.131:0/2312844448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0498155ea0 msgr2=0x7f04981443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f0497fff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0498155ea0 0x7f04981443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f0497fff700 1 -- 172.21.15.131:0/2312844448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f049815a680 msgr2=0x7f049815aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f0497fff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f049815a680 0x7f049815aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f0497fff700 1 -- 172.21.15.131:0/2312844448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049816cc40 con 0x7f0480003a60 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f0497fff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f049815b970 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0488013f70 tx=0x7f0488019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f047ffff700 1 -- 172.21.15.131:0/2312844448 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800ec40 con 0x7f0480003a60 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f047ffff700 1 -- 172.21.15.131:0/2312844448 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488021040 con 0x7f0480003a60 2023-12-26T21:56:24.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049816cf20 con 0x7f0480003a60 2023-12-26T21:56:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.514+0000 7f047ffff700 1 -- 172.21.15.131:0/2312844448 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488025640 con 0x7f0480003a60 2023-12-26T21:56:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.515+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049816d4e0 con 0x7f0480003a60 2023-12-26T21:56:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.516+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04980657e0 con 0x7f0480003a60 2023-12-26T21:56:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.516+0000 7f047ffff700 1 -- 172.21.15.131:0/2312844448 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f04880259b0 con 0x7f0480003a60 2023-12-26T21:56:24.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.518+0000 7f047ffff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0484066f60 0x7f0484069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:24.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.518+0000 7f047ffff700 1 -- 172.21.15.131:0/2312844448 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5273+0+0 (secure 0 0 0) 0x7f0488085c10 con 0x7f0480003a60 2023-12-26T21:56:24.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.518+0000 7f04977fe700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0484066f60 0x7f0484069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:24.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.518+0000 7f04977fe700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0484066f60 0x7f0484069410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f049815b400 tx=0x7f048c001e90 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:24.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.522+0000 7f047ffff700 1 -- 172.21.15.131:0/2312844448 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0488052f90 con 0x7f0480003a60 2023-12-26T21:56:24.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.790+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f04980cb4f0 con 0x7f0480003a60 2023-12-26T21:56:24.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.790+0000 7f047ffff700 1 -- 172.21.15.131:0/2312844448 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f04880565b0 con 0x7f0480003a60 2023-12-26T21:56:24.791 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:24.791 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:56:24.791 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T21:56:24.791 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.793 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.794 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.795 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.796 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.797 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.798 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T21:56:24.799 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T21:56:24.800 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T21:56:24.801 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T21:56:24.802 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:24.804 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T21:56:24.804 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0484066f60 msgr2=0x7f0484069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0484066f60 0x7f0484069410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f049815b400 tx=0x7f048c001e90 comp rx=0 tx=0).stop 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 msgr2=0x7f049815b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f049815b970 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0488013f70 tx=0x7f0488019000 comp rx=0 tx=0).stop 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 shutdown_connections 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f049815a680 0x7f049815aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0484066f60 0x7f0484069410 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0480003a60 0x7f049815b970 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 --2- 172.21.15.131:0/2312844448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0498155ea0 0x7f04981443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 >> 172.21.15.131:0/2312844448 conn(0x7f0498068040 msgr2=0x7f04980c2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 shutdown_connections 2023-12-26T21:56:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:24.793+0000 7f049e3ff700 1 -- 172.21.15.131:0/2312844448 wait complete. 2023-12-26T21:56:24.812 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:56:25.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.003+0000 7fa808fff700 1 Processor -- start 2023-12-26T21:56:25.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.003+0000 7fa808fff700 1 -- start start 2023-12-26T21:56:25.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.004+0000 7fa808fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 0x7fa8040c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.004+0000 7fa808fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8040686c0 con 0x7fa8040cc810 2023-12-26T21:56:25.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.004+0000 7fa808fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa804069520 con 0x7fa80406b8c0 2023-12-26T21:56:25.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.004+0000 7fa808fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa804068910 con 0x7fa80413c510 2023-12-26T21:56:25.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.004+0000 7fa802d9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa8040cc810 0x7fa8040699b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48856/0 (socket says 172.21.15.131:48856) 2023-12-26T21:56:25.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.004+0000 7fa802d9d700 1 -- 172.21.15.131:0/2374976584 learned_addr learned my addr 172.21.15.131:0/2374976584 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80259c700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 0x7fa8040c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751980637 0 0) 0x7fa8040686c0 con 0x7fa8040cc810 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ec003570 con 0x7fa8040cc810 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474482658 0 0) 0x7fa804069520 con 0x7fa80406b8c0 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8040686c0 con 0x7fa80406b8c0 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3867527808 0 0) 0x7fa7ec003570 con 0x7fa8040cc810 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 msgr2=0x7fa8040c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 0x7fa8040c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 >> v1:172.21.15.190:6789/0 conn(0x7fa80406b8c0 legacy=0x7fa8040cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa804068bf0 con 0x7fa8040cc810 2023-12-26T21:56:25.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fa7fc0017f0 con 0x7fa8040cc810 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80259c700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 0x7fa8040c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 >> v1:172.21.15.131:6789/0 conn(0x7fa8040cc810 legacy=0x7fa8040699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 0x7fa7ec003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004160 0x7fa7ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec004d50 0x7fa7ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804068910 con 0x7fa7ec004d50 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.005+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec00bba0 con 0x7fa7ec004160 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec00bd60 con 0x7fa80413c510 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa80259c700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 0x7fa7ec003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa802d9d700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec004d50 0x7fa7ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa801d9b700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004160 0x7fa7ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa802d9d700 1 -- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 msgr2=0x7fa7ec003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:25.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa802d9d700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 0x7fa7ec003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa802d9d700 1 -- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004160 msgr2=0x7fa7ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa802d9d700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004160 0x7fa7ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa802d9d700 1 -- 172.21.15.131:0/2374976584 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40093c0 con 0x7fa7ec004d50 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa80259c700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa80413c510 0x7fa7ec003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa801d9b700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004160 0x7fa7ec004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.006+0000 7fa802d9d700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec004d50 0x7fa7ec0051c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00a8d0 tx=0x7fa7fc002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb824e1661c7d0f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc0070d0 con 0x7fa7ec004d50 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc007230 con 0x7fa7ec004d50 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa80159a700 1 -- 172.21.15.131:0/2374976584 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc0075f0 con 0x7fa7ec004d50 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa808fff700 1 -- 172.21.15.131:0/2374976584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec004d50 msgr2=0x7fa7ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa808fff700 1 --2- 172.21.15.131:0/2374976584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec004d50 0x7fa7ec0051c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00a8d0 tx=0x7fa7fc002190 comp rx=0 tx=0).stop 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa802d9d700 1 -- 172.21.15.131:0/2374976584 reap_dead start 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa808fff700 1 -- 172.21.15.131:0/2374976584 shutdown_connections 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa808fff700 1 -- 172.21.15.131:0/2374976584 >> 172.21.15.131:0/2374976584 conn(0x7fa8040ce3e0 msgr2=0x7fa8040ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:25.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa808fff700 1 -- 172.21.15.131:0/2374976584 shutdown_connections 2023-12-26T21:56:25.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.007+0000 7fa808fff700 1 -- 172.21.15.131:0/2374976584 wait complete. 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.009+0000 7fa808fff700 1 Processor -- start 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.009+0000 7fa808fff700 1 -- start start 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa808fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004d50 0x7fa80414fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa808fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa804156dd0 0x7fa8041578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa808fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa804154340 0x7fa8041547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa808fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804154cf0 con 0x7fa804154340 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa808fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804158be0 con 0x7fa7ec004d50 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa808fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa804157240 con 0x7fa804156dd0 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa802d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa804154340 0x7fa8041547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa802d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa804154340 0x7fa8041547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53012/0 (socket says 172.21.15.131:53012) 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa802d9d700 1 -- 172.21.15.131:0/3537594601 learned_addr learned my addr 172.21.15.131:0/3537594601 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:25.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa80259c700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004d50 0x7fa80414fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa801d9b700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa804156dd0 0x7fa8041578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa802d9d700 1 -- 172.21.15.131:0/3537594601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa804156dd0 msgr2=0x7fa8041578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa802d9d700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa804156dd0 0x7fa8041578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa802d9d700 1 -- 172.21.15.131:0/3537594601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004d50 msgr2=0x7fa80414fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa802d9d700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004d50 0x7fa80414fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.010+0000 7fa802d9d700 1 -- 172.21.15.131:0/3537594601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa804157fd0 con 0x7fa804154340 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.011+0000 7fa802d9d700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa804154340 0x7fa8041547b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc012000 tx=0x7fa7fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.011+0000 7fa7f27fc700 1 -- 172.21.15.131:0/3537594601 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc023070 con 0x7fa804154340 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.011+0000 7fa7f27fc700 1 -- 172.21.15.131:0/3537594601 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc004bb0 con 0x7fa804154340 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.011+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8041582b0 con 0x7fa804154340 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.011+0000 7fa7f27fc700 1 -- 172.21.15.131:0/3537594601 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc01e430 con 0x7fa804154340 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.011+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa804158870 con 0x7fa804154340 2023-12-26T21:56:25.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.012+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8040657e0 con 0x7fa804154340 2023-12-26T21:56:25.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.013+0000 7fa7f27fc700 1 -- 172.21.15.131:0/3537594601 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa7fc01e860 con 0x7fa804154340 2023-12-26T21:56:25.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.014+0000 7fa7f27fc700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa7e4066f30 0x7fa7e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.014+0000 7fa7f27fc700 1 -- 172.21.15.131:0/3537594601 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5273+0+0 (secure 0 0 0) 0x7fa7fc086480 con 0x7fa804154340 2023-12-26T21:56:25.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.014+0000 7fa80259c700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa7e4066f30 0x7fa7e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.015+0000 7fa80259c700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa7e4066f30 0x7fa7e40693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40098c0 tx=0x7fa7f4009120 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:25.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.018+0000 7fa7f27fc700 1 -- 172.21.15.131:0/3537594601 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7fc053800 con 0x7fa804154340 2023-12-26T21:56:25.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.213+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa804158d20 con 0x7fa804154340 2023-12-26T21:56:25.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.213+0000 7fa7f27fc700 1 -- 172.21.15.131:0/3537594601 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+12833 (secure 0 0 0) 0x7fa7fc056e20 con 0x7fa804154340 2023-12-26T21:56:25.214 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:25.214 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":135,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:56:24.284617+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:56:25.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa7e4066f30 msgr2=0x7fa7e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa7e4066f30 0x7fa7e40693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40098c0 tx=0x7fa7f4009120 comp rx=0 tx=0).stop 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa804154340 msgr2=0x7fa8041547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa804154340 0x7fa8041547b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc012000 tx=0x7fa7fc016000 comp rx=0 tx=0).stop 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 shutdown_connections 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa7e4066f30 0x7fa7e40693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa804154340 0x7fa8041547b0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7ec004d50 0x7fa80414fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 --2- 172.21.15.131:0/3537594601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa804156dd0 0x7fa8041578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 >> 172.21.15.131:0/3537594601 conn(0x7fa8040ce3e0 msgr2=0x7fa804152e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 shutdown_connections 2023-12-26T21:56:25.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.216+0000 7fa808fff700 1 -- 172.21.15.131:0/3537594601 wait complete. 2023-12-26T21:56:25.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T21:56:25.236 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T21:56:25.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.428+0000 7fcd93fff700 1 Processor -- start 2023-12-26T21:56:25.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.428+0000 7fcd93fff700 1 -- start start 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.429+0000 7fcd93fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd8c13c510 0x7fcd8c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.429+0000 7fcd93fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 0x7fcd8c06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.429+0000 7fcd93fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c0686c0 con 0x7fcd8c0cc100 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd93fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8c069520 con 0x7fcd8c13c510 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd93fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c06c250 con 0x7fcd8c06b8c0 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd91d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd8c13c510 0x7fcd8c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd91d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd8c13c510 0x7fcd8c0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53014/0 (socket says 172.21.15.131:53014) 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd91d9b700 1 -- 172.21.15.131:0/3351382170 learned_addr learned my addr 172.21.15.131:0/3351382170 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd9259c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcd8c0cc100 0x7fcd8c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48862/0 (socket says 172.21.15.131:48862) 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd9159a700 1 --2- 172.21.15.131:0/3351382170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 0x7fcd8c06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd91d9b700 1 -- 172.21.15.131:0/3351382170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 msgr2=0x7fcd8c06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd91d9b700 1 --2- 172.21.15.131:0/3351382170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 0x7fcd8c06bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd91d9b700 1 -- 172.21.15.131:0/3351382170 >> v1:172.21.15.131:6789/0 conn(0x7fcd8c0cc100 legacy=0x7fcd8c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd91d9b700 1 -- 172.21.15.131:0/3351382170 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c068910 con 0x7fcd8c13c510 2023-12-26T21:56:25.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd90d99700 1 -- 172.21.15.131:0/3351382170 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433884510 0 0) 0x7fcd8c0686c0 con 0x7fcd8c0cc100 2023-12-26T21:56:25.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.430+0000 7fcd91d9b700 1 --2- 172.21.15.131:0/3351382170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 0x7fcd8c0c8a20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009990 tx=0x7fcd88009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85de44d5ef7bbcb8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd90d99700 1 -- 172.21.15.131:0/3351382170 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88009e80 con 0x7fcd8c13c510 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd90d99700 1 -- 172.21.15.131:0/3351382170 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd88005c10 con 0x7fcd8c13c510 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd90d99700 1 -- 172.21.15.131:0/3351382170 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800b650 con 0x7fcd8c13c510 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd93fff700 1 -- 172.21.15.131:0/3351382170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 msgr2=0x7fcd8c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd93fff700 1 --2- 172.21.15.131:0/3351382170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 0x7fcd8c0c8a20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009990 tx=0x7fcd88009220 comp rx=0 tx=0).stop 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd93fff700 1 -- 172.21.15.131:0/3351382170 shutdown_connections 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd93fff700 1 --2- 172.21.15.131:0/3351382170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 0x7fcd8c0c8a20 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd93fff700 1 --2- 172.21.15.131:0/3351382170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 0x7fcd8c06bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd93fff700 1 -- 172.21.15.131:0/3351382170 >> 172.21.15.131:0/3351382170 conn(0x7fcd8c0ce3e0 msgr2=0x7fcd8c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd93fff700 1 -- 172.21.15.131:0/3351382170 shutdown_connections 2023-12-26T21:56:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.431+0000 7fcd93fff700 1 -- 172.21.15.131:0/3351382170 wait complete. 2023-12-26T21:56:25.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.433+0000 7fcd93fff700 1 Processor -- start 2023-12-26T21:56:25.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.434+0000 7fcd93fff700 1 -- start start 2023-12-26T21:56:25.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd93fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 0x7fcd8c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd93fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd8c0cc100 0x7fcd8c1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd93fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 0x7fcd8c0cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd93fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b2c80 con 0x7fcd8c13c510 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd93fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b2070 con 0x7fcd8c0cc100 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd93fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b2350 con 0x7fcd8c06b8c0 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd9259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 0x7fcd8c0cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd9259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 0x7fcd8c0cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53016/0 (socket says 172.21.15.131:53016) 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd9259c700 1 -- 172.21.15.131:0/3877252522 learned_addr learned my addr 172.21.15.131:0/3877252522 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd91d9b700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 0x7fcd8c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.435+0000 7fcd9159a700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd8c0cc100 0x7fcd8c1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd9259c700 1 -- 172.21.15.131:0/3877252522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 msgr2=0x7fcd8c1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd9259c700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 0x7fcd8c1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd9259c700 1 -- 172.21.15.131:0/3877252522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd8c0cc100 msgr2=0x7fcd8c1538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd9259c700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd8c0cc100 0x7fcd8c1538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd9259c700 1 -- 172.21.15.131:0/3877252522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd88009480 con 0x7fcd8c13c510 2023-12-26T21:56:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd91d9b700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 0x7fcd8c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd9259c700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 0x7fcd8c0cade0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400cd00 tx=0x7fcd84001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd81ffb700 1 -- 172.21.15.131:0/3877252522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8400ce60 con 0x7fcd8c13c510 2023-12-26T21:56:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd81ffb700 1 -- 172.21.15.131:0/3877252522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84005df0 con 0x7fcd8c13c510 2023-12-26T21:56:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c0b2630 con 0x7fcd8c13c510 2023-12-26T21:56:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd81ffb700 1 -- 172.21.15.131:0/3877252522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8400ab90 con 0x7fcd8c13c510 2023-12-26T21:56:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.436+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1519e0 con 0x7fcd8c13c510 2023-12-26T21:56:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.437+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c159da0 con 0x7fcd8c13c510 2023-12-26T21:56:25.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.438+0000 7fcd81ffb700 1 -- 172.21.15.131:0/3877252522 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcd840070f0 con 0x7fcd8c13c510 2023-12-26T21:56:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.439+0000 7fcd81ffb700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd70066f60 0x7fcd70069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.439+0000 7fcd81ffb700 1 -- 172.21.15.131:0/3877252522 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5273+0+0 (secure 0 0 0) 0x7fcd840843f0 con 0x7fcd8c13c510 2023-12-26T21:56:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.440+0000 7fcd91d9b700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd70066f60 0x7fcd70069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.440+0000 7fcd91d9b700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd70066f60 0x7fcd70069410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009260 tx=0x7fcd88016000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:25.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.444+0000 7fcd81ffb700 1 -- 172.21.15.131:0/3877252522 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd84051890 con 0x7fcd8c13c510 2023-12-26T21:56:25.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.638+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd8c0657e0 con 0x7fcd8c13c510 2023-12-26T21:56:25.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.638+0000 7fcd81ffb700 1 -- 172.21.15.131:0/3877252522 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd84054eb0 con 0x7fcd8c13c510 2023-12-26T21:56:25.639 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T21:56:25.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd70066f60 msgr2=0x7fcd70069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd70066f60 0x7fcd70069410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009260 tx=0x7fcd88016000 comp rx=0 tx=0).stop 2023-12-26T21:56:25.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 msgr2=0x7fcd8c0cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 0x7fcd8c0cade0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400cd00 tx=0x7fcd84001ac0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 shutdown_connections 2023-12-26T21:56:25.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcd70066f60 0x7fcd70069410 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd8c13c510 0x7fcd8c0cade0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd8c06b8c0 0x7fcd8c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 --2- 172.21.15.131:0/3877252522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd8c0cc100 0x7fcd8c1538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.640+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 >> 172.21.15.131:0/3877252522 conn(0x7fcd8c0ce3e0 msgr2=0x7fcd8c155850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:25.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.641+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 shutdown_connections 2023-12-26T21:56:25.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.641+0000 7fcd93fff700 1 -- 172.21.15.131:0/3877252522 wait complete. 2023-12-26T21:56:25.659 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T21:56:25.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.848+0000 7f7af5bff700 1 Processor -- start 2023-12-26T21:56:25.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.848+0000 7f7af5bff700 1 -- start start 2023-12-26T21:56:25.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.849+0000 7f7af5bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7af006e480 0x7f7af00abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.849+0000 7f7af5bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af00aa480 0x7f7af00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.849+0000 7f7af5bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af015a8c0 con 0x7f7af0159820 2023-12-26T21:56:25.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.849+0000 7f7af5bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af015ad50 con 0x7f7af006e480 2023-12-26T21:56:25.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.849+0000 7f7af5bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0159f90 con 0x7f7af00aa480 2023-12-26T21:56:25.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af00aa480 0x7f7af00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aeeffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7af006e480 0x7f7af00abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aeeffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7af006e480 0x7f7af00abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:56238/0 (socket says 172.21.15.131:56238) 2023-12-26T21:56:25.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aeeffd700 1 -- 172.21.15.131:0/189376282 learned_addr learned my addr 172.21.15.131:0/189376282 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:25.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af00aa480 0x7f7af00675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:51982/0 (socket says 172.21.15.131:51982) 2023-12-26T21:56:25.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aeeffd700 1 -- 172.21.15.131:0/189376282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af00aa480 msgr2=0x7f7af00675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:25.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aeeffd700 1 --2- 172.21.15.131:0/189376282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af00aa480 0x7f7af00675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aeeffd700 1 -- 172.21.15.131:0/189376282 >> v1:172.21.15.190:6789/0 conn(0x7f7af0159820 legacy=0x7f7af006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aeeffd700 1 -- 172.21.15.131:0/189376282 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af015a270 con 0x7f7af006e480 2023-12-26T21:56:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.850+0000 7f7aee7fc700 1 -- 172.21.15.131:0/189376282 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254478566 0 0) 0x7f7af015a8c0 con 0x7f7af0159820 2023-12-26T21:56:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7aeeffd700 1 --2- 172.21.15.131:0/189376282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af006e480 0x7f7af00abdb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40099b0 tx=0x7f7ae4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c1a7843cb31fe8c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7aee7fc700 1 -- 172.21.15.131:0/189376282 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4009ea0 con 0x7f7af006e480 2023-12-26T21:56:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7aee7fc700 1 -- 172.21.15.131:0/189376282 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae400b860 con 0x7f7af006e480 2023-12-26T21:56:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7aee7fc700 1 -- 172.21.15.131:0/189376282 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400bc00 con 0x7f7af006e480 2023-12-26T21:56:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7af5bff700 1 -- 172.21.15.131:0/189376282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af006e480 msgr2=0x7f7af00abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7af5bff700 1 --2- 172.21.15.131:0/189376282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af006e480 0x7f7af00abdb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40099b0 tx=0x7f7ae4009250 comp rx=0 tx=0).stop 2023-12-26T21:56:25.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7af5bff700 1 -- 172.21.15.131:0/189376282 shutdown_connections 2023-12-26T21:56:25.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7af5bff700 1 --2- 172.21.15.131:0/189376282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af00aa480 0x7f7af00675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7af5bff700 1 --2- 172.21.15.131:0/189376282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af006e480 0x7f7af00abdb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.851+0000 7f7af5bff700 1 -- 172.21.15.131:0/189376282 >> 172.21.15.131:0/189376282 conn(0x7f7af006c290 msgr2=0x7f7af00ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:25.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.852+0000 7f7af5bff700 1 -- 172.21.15.131:0/189376282 shutdown_connections 2023-12-26T21:56:25.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.852+0000 7f7af5bff700 1 -- 172.21.15.131:0/189376282 wait complete. 2023-12-26T21:56:25.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.853+0000 7f7af5bff700 1 Processor -- start 2023-12-26T21:56:25.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.853+0000 7f7af5bff700 1 -- start start 2023-12-26T21:56:25.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7af5bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af006e480 0x7f7af0155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7af5bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af00aa480 0x7f7af00c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7af5bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0159820 0x7f7af00cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7af5bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00c2d00 con 0x7f7af0159820 2023-12-26T21:56:25.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7af5bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00c20f0 con 0x7f7af00aa480 2023-12-26T21:56:25.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7af5bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00c23d0 con 0x7f7af006e480 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0159820 0x7f7af00cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0159820 0x7f7af00cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53020/0 (socket says 172.21.15.131:53020) 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7aeffff700 1 -- 172.21.15.131:0/3860365513 learned_addr learned my addr 172.21.15.131:0/3860365513 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7aeeffd700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af00aa480 0x7f7af00c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.854+0000 7f7aef7fe700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af006e480 0x7f7af0155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7aeffff700 1 -- 172.21.15.131:0/3860365513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af006e480 msgr2=0x7f7af0155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7aeffff700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af006e480 0x7f7af0155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7aeffff700 1 -- 172.21.15.131:0/3860365513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af00aa480 msgr2=0x7f7af00c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7aeffff700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af00aa480 0x7f7af00c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7aeffff700 1 -- 172.21.15.131:0/3860365513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40094f0 con 0x7f7af0159820 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7aeffff700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0159820 0x7f7af00cc1c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009e90 tx=0x7f7ae800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7ad77fe700 1 -- 172.21.15.131:0/3860365513 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800cb70 con 0x7f7af0159820 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7ad77fe700 1 -- 172.21.15.131:0/3860365513 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae800f040 con 0x7f7af0159820 2023-12-26T21:56:25.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7ad77fe700 1 -- 172.21.15.131:0/3860365513 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8014480 con 0x7f7af0159820 2023-12-26T21:56:25.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af00c26b0 con 0x7f7af0159820 2023-12-26T21:56:25.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.855+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af00c1e00 con 0x7f7af0159820 2023-12-26T21:56:25.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.856+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af00c2e70 con 0x7f7af0159820 2023-12-26T21:56:25.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.857+0000 7f7ad77fe700 1 -- 172.21.15.131:0/3860365513 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7ae8014830 con 0x7f7af0159820 2023-12-26T21:56:25.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.858+0000 7f7ad77fe700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7adc066f30 0x7f7adc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:25.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.858+0000 7f7ad77fe700 1 -- 172.21.15.131:0/3860365513 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5273+0+0 (secure 0 0 0) 0x7f7ae8084d50 con 0x7f7af0159820 2023-12-26T21:56:25.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.858+0000 7f7aef7fe700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7adc066f30 0x7f7adc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:25.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.859+0000 7f7aef7fe700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7adc066f30 0x7f7adc0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00039b0 tx=0x7f7ae0007ec0 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:25.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:25.862+0000 7f7ad77fe700 1 -- 172.21.15.131:0/3860365513 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ae8088030 con 0x7f7af0159820 2023-12-26T21:56:26.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.056+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7af00657e0 con 0x7f7af0159820 2023-12-26T21:56:26.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.056+0000 7f7ad77fe700 1 -- 172.21.15.131:0/3860365513 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v135) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7ae8052170 con 0x7f7af0159820 2023-12-26T21:56:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:26.057 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:56:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T21:56:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T21:56:26.057 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:26.057 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:56:26.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7adc066f30 msgr2=0x7f7adc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:26.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7adc066f30 0x7f7adc0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00039b0 tx=0x7f7ae0007ec0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0159820 msgr2=0x7f7af00cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:26.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0159820 0x7f7af00cc1c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009e90 tx=0x7f7ae800b040 comp rx=0 tx=0).stop 2023-12-26T21:56:26.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 shutdown_connections 2023-12-26T21:56:26.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7adc066f30 0x7f7adc0693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af0159820 0x7f7af00cc1c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7af006e480 0x7f7af0155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 --2- 172.21.15.131:0/3860365513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af00aa480 0x7f7af00c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 >> 172.21.15.131:0/3860365513 conn(0x7f7af006c290 msgr2=0x7f7af0156570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:26.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 shutdown_connections 2023-12-26T21:56:26.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.058+0000 7f7af5bff700 1 -- 172.21.15.131:0/3860365513 wait complete. 2023-12-26T21:56:26.077 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T21:56:26.077 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T21:56:26.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.270+0000 7f53bb5ff700 1 Processor -- start 2023-12-26T21:56:26.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.271+0000 7f53bb5ff700 1 -- start start 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.272+0000 7f53bb5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f53b40cf390 0x7f53b40c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.272+0000 7f53bb5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53b40cd0a0 0x7f53b40cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.272+0000 7f53bb5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40cbb00 con 0x7f53b4159810 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.272+0000 7f53bb5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b40cb2a0 con 0x7f53b40cf390 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.272+0000 7f53bb5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b40ce040 con 0x7f53b40cd0a0 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.272+0000 7f53b8b9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f53b40cf390 0x7f53b40c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.272+0000 7f53b939b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f53b4159810 0x7f53b40cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48868/0 (socket says 172.21.15.131:48868) 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.272+0000 7f53b939b700 1 -- 172.21.15.131:0/292098984 learned_addr learned my addr 172.21.15.131:0/292098984 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.272+0000 7f53b9b9c700 1 --2- 172.21.15.131:0/292098984 >> v2:172.21.15.190:3300/0 conn(0x7f53b40cd0a0 0x7f53b40cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.273+0000 7f53b8b9a700 1 -- 172.21.15.131:0/292098984 >> v2:172.21.15.190:3300/0 conn(0x7f53b40cd0a0 msgr2=0x7f53b40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.273+0000 7f53b8b9a700 1 --2- 172.21.15.131:0/292098984 >> v2:172.21.15.190:3300/0 conn(0x7f53b40cd0a0 0x7f53b40cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.273+0000 7f53b8b9a700 1 -- 172.21.15.131:0/292098984 >> v1:172.21.15.131:6789/0 conn(0x7f53b4159810 legacy=0x7f53b40cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.273+0000 7f53b8b9a700 1 -- 172.21.15.131:0/292098984 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4159be0 con 0x7f53b40cf390 2023-12-26T21:56:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.273+0000 7f53abfff700 1 -- 172.21.15.131:0/292098984 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330579706 0 0) 0x7f53b40cbb00 con 0x7f53b4159810 2023-12-26T21:56:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.273+0000 7f53b8b9a700 1 --2- 172.21.15.131:0/292098984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 0x7f53b40c5440 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f53b4089f40 tx=0x7f53a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb33b74013d465a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.273+0000 7f53abfff700 1 -- 172.21.15.131:0/292098984 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4011630 con 0x7f53b40cf390 2023-12-26T21:56:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.273+0000 7f53abfff700 1 -- 172.21.15.131:0/292098984 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a4011790 con 0x7f53b40cf390 2023-12-26T21:56:26.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.273+0000 7f53abfff700 1 -- 172.21.15.131:0/292098984 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4011b30 con 0x7f53b40cf390 2023-12-26T21:56:26.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.274+0000 7f53bb5ff700 1 -- 172.21.15.131:0/292098984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 msgr2=0x7f53b40c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:26.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.274+0000 7f53bb5ff700 1 --2- 172.21.15.131:0/292098984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 0x7f53b40c5440 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f53b4089f40 tx=0x7f53a40091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.274+0000 7f53bb5ff700 1 -- 172.21.15.131:0/292098984 shutdown_connections 2023-12-26T21:56:26.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.274+0000 7f53bb5ff700 1 --2- 172.21.15.131:0/292098984 >> v2:172.21.15.190:3300/0 conn(0x7f53b40cd0a0 0x7f53b40cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.274+0000 7f53bb5ff700 1 --2- 172.21.15.131:0/292098984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 0x7f53b40c5440 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.274+0000 7f53bb5ff700 1 -- 172.21.15.131:0/292098984 >> 172.21.15.131:0/292098984 conn(0x7f53b4099950 msgr2=0x7f53b406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:26.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.274+0000 7f53bb5ff700 1 -- 172.21.15.131:0/292098984 shutdown_connections 2023-12-26T21:56:26.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.274+0000 7f53bb5ff700 1 -- 172.21.15.131:0/292098984 wait complete. 2023-12-26T21:56:26.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.276+0000 7f53bb5ff700 1 Processor -- start 2023-12-26T21:56:26.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.276+0000 7f53bb5ff700 1 -- start start 2023-12-26T21:56:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.277+0000 7f53bb5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b40cd0a0 0x7f53b41471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.277+0000 7f53bb5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 0x7f53b41697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.277+0000 7f53bb5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4159810 0x7f53b40b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.277+0000 7f53bb5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b41476f0 con 0x7f53b40cf390 2023-12-26T21:56:26.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.277+0000 7f53bb5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4169d00 con 0x7f53b4159810 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.277+0000 7f53bb5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40b5af0 con 0x7f53b40cd0a0 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b8b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 0x7f53b41697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b8b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 0x7f53b41697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53042/0 (socket says 172.21.15.131:53042) 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b8b9a700 1 -- 172.21.15.131:0/2925106023 learned_addr learned my addr 172.21.15.131:0/2925106023 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b939b700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b40cd0a0 0x7f53b41471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b9b9c700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4159810 0x7f53b40b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b8b9a700 1 -- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b40cd0a0 msgr2=0x7f53b41471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b8b9a700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b40cd0a0 0x7f53b41471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b8b9a700 1 -- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4159810 msgr2=0x7f53b40b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b8b9a700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4159810 0x7f53b40b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b8b9a700 1 -- 172.21.15.131:0/2925106023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a4009420 con 0x7f53b40cf390 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b939b700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b40cd0a0 0x7f53b41471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b9b9c700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4159810 0x7f53b40b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.278+0000 7f53b8b9a700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 0x7f53b41697c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f53a4013000 tx=0x7f53a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.279+0000 7f53a97fa700 1 -- 172.21.15.131:0/2925106023 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4011b10 con 0x7f53b40cf390 2023-12-26T21:56:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.279+0000 7f53a97fa700 1 -- 172.21.15.131:0/2925106023 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a40253f0 con 0x7f53b40cf390 2023-12-26T21:56:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.279+0000 7f53a97fa700 1 -- 172.21.15.131:0/2925106023 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4025790 con 0x7f53b40cf390 2023-12-26T21:56:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.279+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b416ac90 con 0x7f53b40cf390 2023-12-26T21:56:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.279+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b416a1f0 con 0x7f53b40cf390 2023-12-26T21:56:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.280+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b40717f0 con 0x7f53b40cf390 2023-12-26T21:56:26.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.281+0000 7f53a97fa700 1 -- 172.21.15.131:0/2925106023 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f53a4025b80 con 0x7f53b40cf390 2023-12-26T21:56:26.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.281+0000 7f53a97fa700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5398066f30 0x7f53980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:26.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.281+0000 7f53a97fa700 1 -- 172.21.15.131:0/2925106023 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5273+0+0 (secure 0 0 0) 0x7f53a4085c60 con 0x7f53b40cf390 2023-12-26T21:56:26.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.282+0000 7f53b939b700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5398066f30 0x7f53980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:26.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.282+0000 7f53b939b700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5398066f30 0x7f53980693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53b0001890 tx=0x7f53b0009040 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.285+0000 7f53a97fa700 1 -- 172.21.15.131:0/2925106023 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53a4053080 con 0x7f53b40cf390 2023-12-26T21:56:26.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:26.477+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f53b40cdd60 con 0x7f53b40cf390 2023-12-26T21:56:27.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.300+0000 7f53a97fa700 1 -- 172.21.15.131:0/2925106023 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v136) v1 ==== 138+0+0 (secure 0 0 0) 0x7f53a40150b0 con 0x7f53b40cf390 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.303+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5398066f30 msgr2=0x7f53980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.303+0000 7f53bb5ff700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5398066f30 0x7f53980693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53b0001890 tx=0x7f53b0009040 comp rx=0 tx=0).stop 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.303+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 msgr2=0x7f53b41697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.303+0000 7f53bb5ff700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 0x7f53b41697c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f53a4013000 tx=0x7f53a4017000 comp rx=0 tx=0).stop 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.303+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 shutdown_connections 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.303+0000 7f53bb5ff700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4159810 0x7f53b40b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.303+0000 7f53bb5ff700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b40cd0a0 0x7f53b41471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.303+0000 7f53bb5ff700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5398066f30 0x7f53980693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.303+0000 7f53bb5ff700 1 --2- 172.21.15.131:0/2925106023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53b40cf390 0x7f53b41697c0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.304+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 >> 172.21.15.131:0/2925106023 conn(0x7f53b4099950 msgr2=0x7f53b4153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:27.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.304+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 shutdown_connections 2023-12-26T21:56:27.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.304+0000 7f53bb5ff700 1 -- 172.21.15.131:0/2925106023 wait complete. 2023-12-26T21:56:27.308 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T21:56:27.326 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T21:56:27.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.529+0000 7f9f8bbff700 1 Processor -- start 2023-12-26T21:56:27.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.529+0000 7f9f8bbff700 1 -- start start 2023-12-26T21:56:27.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.529+0000 7f9f8bbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f84069d30 0x7f9f8406e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:27.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.529+0000 7f9f8bbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f840c6b30 con 0x7f9f840adf60 2023-12-26T21:56:27.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.529+0000 7f9f8bbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f840c6e10 con 0x7f9f8406e2c0 2023-12-26T21:56:27.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.529+0000 7f9f8bbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f841561e0 con 0x7f9f84069d30 2023-12-26T21:56:27.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.529+0000 7f9f8a19c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f84069d30 0x7f9f8406e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:27.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.530+0000 7f9f8a19c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f84069d30 0x7f9f8406e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53052/0 (socket says 172.21.15.131:53052) 2023-12-26T21:56:27.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.530+0000 7f9f8999b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9f840adf60 0x7f9f840ac040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48872/0 (socket says 172.21.15.131:48872) 2023-12-26T21:56:27.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.530+0000 7f9f8a19c700 1 -- 172.21.15.131:0/1935640066 learned_addr learned my addr 172.21.15.131:0/1935640066 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:27.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.530+0000 7f9f8a19c700 1 -- 172.21.15.131:0/1935640066 >> v1:172.21.15.190:6789/0 conn(0x7f9f8406e2c0 legacy=0x7f9f84069620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:27.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.530+0000 7f9f8a19c700 1 -- 172.21.15.131:0/1935640066 >> v1:172.21.15.131:6789/0 conn(0x7f9f840adf60 legacy=0x7f9f840ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:27.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.530+0000 7f9f8a19c700 1 -- 172.21.15.131:0/1935640066 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f840c70f0 con 0x7f9f84069d30 2023-12-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.530+0000 7f9f88999700 1 -- 172.21.15.131:0/1935640066 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706621024 0 0) 0x7f9f840c6b30 con 0x7f9f840adf60 2023-12-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.530+0000 7f9f8a19c700 1 --2- 172.21.15.131:0/1935640066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f84069d30 0x7f9f8406e760 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009f50 tx=0x7f9f7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68deda45da1e5c67 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f88999700 1 -- 172.21.15.131:0/1935640066 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0092e0 con 0x7f9f84069d30 2023-12-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f88999700 1 -- 172.21.15.131:0/1935640066 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c00ba70 con 0x7f9f84069d30 2023-12-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f88999700 1 -- 172.21.15.131:0/1935640066 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c00be30 con 0x7f9f84069d30 2023-12-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f8bbff700 1 -- 172.21.15.131:0/1935640066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f84069d30 msgr2=0x7f9f8406e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f8bbff700 1 --2- 172.21.15.131:0/1935640066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f84069d30 0x7f9f8406e760 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009f50 tx=0x7f9f7c009840 comp rx=0 tx=0).stop 2023-12-26T21:56:27.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f8bbff700 1 -- 172.21.15.131:0/1935640066 shutdown_connections 2023-12-26T21:56:27.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f8bbff700 1 --2- 172.21.15.131:0/1935640066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f84069d30 0x7f9f8406e760 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:27.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f8bbff700 1 -- 172.21.15.131:0/1935640066 >> 172.21.15.131:0/1935640066 conn(0x7f9f84068040 msgr2=0x7f9f840cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:27.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f8bbff700 1 -- 172.21.15.131:0/1935640066 shutdown_connections 2023-12-26T21:56:27.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.531+0000 7f9f8bbff700 1 -- 172.21.15.131:0/1935640066 wait complete. 2023-12-26T21:56:27.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.533+0000 7f9f8bbff700 1 Processor -- start 2023-12-26T21:56:27.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.533+0000 7f9f8bbff700 1 -- start start 2023-12-26T21:56:27.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8bbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f84069d30 0x7f9f84168f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:27.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8bbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f8406e2c0 0x7f9f840bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:27.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f840adf60 0x7f9f8415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:27.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8bbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8416a3c0 con 0x7f9f8406e2c0 2023-12-26T21:56:27.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8bbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8416a850 con 0x7f9f84069d30 2023-12-26T21:56:27.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f841697b0 con 0x7f9f840adf60 2023-12-26T21:56:27.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8919a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f8406e2c0 0x7f9f840bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:27.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8919a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f8406e2c0 0x7f9f840bfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53068/0 (socket says 172.21.15.131:53068) 2023-12-26T21:56:27.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8919a700 1 -- 172.21.15.131:0/4065656314 learned_addr learned my addr 172.21.15.131:0/4065656314 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:27.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8a19c700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f840adf60 0x7f9f8415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:27.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8999b700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f84069d30 0x7f9f84168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:27.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8919a700 1 -- 172.21.15.131:0/4065656314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f840adf60 msgr2=0x7f9f8415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:27.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8919a700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f840adf60 0x7f9f8415dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:27.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8919a700 1 -- 172.21.15.131:0/4065656314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f84069d30 msgr2=0x7f9f84168f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8919a700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f84069d30 0x7f9f84168f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.534+0000 7f9f8919a700 1 -- 172.21.15.131:0/4065656314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c009ae0 con 0x7f9f8406e2c0 2023-12-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.535+0000 7f9f8919a700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f8406e2c0 0x7f9f840bfa20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c590 tx=0x7f9f7400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.535+0000 7f9f79ffb700 1 -- 172.21.15.131:0/4065656314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f74011590 con 0x7f9f8406e2c0 2023-12-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.535+0000 7f9f79ffb700 1 -- 172.21.15.131:0/4065656314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f740116f0 con 0x7f9f8406e2c0 2023-12-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.535+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f84169a90 con 0x7f9f8406e2c0 2023-12-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.535+0000 7f9f79ffb700 1 -- 172.21.15.131:0/4065656314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f74011ab0 con 0x7f9f8406e2c0 2023-12-26T21:56:27.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.535+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8416a050 con 0x7f9f8406e2c0 2023-12-26T21:56:27.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.536+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f840657e0 con 0x7f9f8406e2c0 2023-12-26T21:56:27.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.537+0000 7f9f79ffb700 1 -- 172.21.15.131:0/4065656314 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9f7401b3b0 con 0x7f9f8406e2c0 2023-12-26T21:56:27.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.538+0000 7f9f79ffb700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f68066f30 0x7f9f680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:27.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.538+0000 7f9f79ffb700 1 -- 172.21.15.131:0/4065656314 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5643+0+0 (secure 0 0 0) 0x7f9f74083b40 con 0x7f9f8406e2c0 2023-12-26T21:56:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.538+0000 7f9f8999b700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f68066f30 0x7f9f680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.539+0000 7f9f8999b700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f68066f30 0x7f9f680693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f80007fd0 tx=0x7f9f80007db0 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:27.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.543+0000 7f9f79ffb700 1 -- 172.21.15.131:0/4065656314 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f74050d50 con 0x7f9f8406e2c0 2023-12-26T21:56:27.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:27.737+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9f8416a210 con 0x7f9f8406e2c0 2023-12-26T21:56:28.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.310+0000 7f9f79ffb700 1 -- 172.21.15.131:0/4065656314 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v137) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9f74017070 con 0x7f9f8406e2c0 2023-12-26T21:56:28.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.313+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f68066f30 msgr2=0x7f9f680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:28.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.313+0000 7f9f8bbff700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f68066f30 0x7f9f680693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f80007fd0 tx=0x7f9f80007db0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.313+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f8406e2c0 msgr2=0x7f9f840bfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:28.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.313+0000 7f9f8bbff700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f8406e2c0 0x7f9f840bfa20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c590 tx=0x7f9f7400c950 comp rx=0 tx=0).stop 2023-12-26T21:56:28.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.314+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 shutdown_connections 2023-12-26T21:56:28.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.314+0000 7f9f8bbff700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f840adf60 0x7f9f8415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.314+0000 7f9f8bbff700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f84069d30 0x7f9f84168f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.314+0000 7f9f8bbff700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f68066f30 0x7f9f680693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.314+0000 7f9f8bbff700 1 --2- 172.21.15.131:0/4065656314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f8406e2c0 0x7f9f840bfa20 secure :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c590 tx=0x7f9f7400c950 comp rx=0 tx=0).stop 2023-12-26T21:56:28.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.314+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 >> 172.21.15.131:0/4065656314 conn(0x7f9f84068040 msgr2=0x7f9f840c8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:28.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.315+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 shutdown_connections 2023-12-26T21:56:28.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.315+0000 7f9f8bbff700 1 -- 172.21.15.131:0/4065656314 wait complete. 2023-12-26T21:56:28.318 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T21:56:28.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T21:56:28.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.539+0000 7fafedfff700 1 Processor -- start 2023-12-26T21:56:28.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.540+0000 7fafedfff700 1 -- start start 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafedfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fafe80c8820 0x7fafe80ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafedfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8bf0 0x7fafe80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafedfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fafe8069d10 0x7fafe80b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafedfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe8157a80 con 0x7fafe8069d10 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafedfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe80cac50 con 0x7fafe80c8820 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafedfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe80caf30 con 0x7fafe80c8bf0 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafe7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fafe8069d10 0x7fafe80b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafe7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fafe8069d10 0x7fafe80b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53080/0 (socket says 172.21.15.131:53080) 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafe7fff700 1 -- 172.21.15.131:0/42888788 learned_addr learned my addr 172.21.15.131:0/42888788 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafe6ffd700 1 --2- 172.21.15.131:0/42888788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8bf0 0x7fafe80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.541+0000 7fafe77fe700 1 --2- 172.21.15.131:0/42888788 >> v2:172.21.15.190:3300/0 conn(0x7fafe80c8820 0x7fafe80ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:28.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.542+0000 7fafe7fff700 1 -- 172.21.15.131:0/42888788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8bf0 msgr2=0x7fafe80b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:28.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.542+0000 7fafe7fff700 1 --2- 172.21.15.131:0/42888788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8bf0 0x7fafe80b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.542+0000 7fafe7fff700 1 -- 172.21.15.131:0/42888788 >> v2:172.21.15.190:3300/0 conn(0x7fafe80c8820 msgr2=0x7fafe80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:28.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.542+0000 7fafe7fff700 1 --2- 172.21.15.131:0/42888788 >> v2:172.21.15.190:3300/0 conn(0x7fafe80c8820 0x7fafe80ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.542+0000 7fafe7fff700 1 -- 172.21.15.131:0/42888788 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe80cb210 con 0x7fafe8069d10 2023-12-26T21:56:28.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.542+0000 7fafe7fff700 1 --2- 172.21.15.131:0/42888788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 0x7fafe80b1820 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fafe8064ea0 tx=0x7fafe0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89906ab4aa8ccae server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:28.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.542+0000 7fafe67fc700 1 -- 172.21.15.131:0/42888788 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe000e070 con 0x7fafe8069d10 2023-12-26T21:56:28.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.542+0000 7fafe67fc700 1 -- 172.21.15.131:0/42888788 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe000b970 con 0x7fafe8069d10 2023-12-26T21:56:28.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.542+0000 7fafe67fc700 1 -- 172.21.15.131:0/42888788 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe00092e0 con 0x7fafe8069d10 2023-12-26T21:56:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.543+0000 7fafedfff700 1 -- 172.21.15.131:0/42888788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 msgr2=0x7fafe80b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.543+0000 7fafedfff700 1 --2- 172.21.15.131:0/42888788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 0x7fafe80b1820 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fafe8064ea0 tx=0x7fafe0009840 comp rx=0 tx=0).stop 2023-12-26T21:56:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.543+0000 7fafedfff700 1 -- 172.21.15.131:0/42888788 shutdown_connections 2023-12-26T21:56:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.543+0000 7fafedfff700 1 --2- 172.21.15.131:0/42888788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 0x7fafe80b1820 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.543+0000 7fafedfff700 1 --2- 172.21.15.131:0/42888788 >> v2:172.21.15.190:3300/0 conn(0x7fafe80c8820 0x7fafe80ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.543+0000 7fafedfff700 1 --2- 172.21.15.131:0/42888788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8bf0 0x7fafe80b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.543+0000 7fafedfff700 1 -- 172.21.15.131:0/42888788 >> 172.21.15.131:0/42888788 conn(0x7fafe8068040 msgr2=0x7fafe8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.543+0000 7fafedfff700 1 -- 172.21.15.131:0/42888788 shutdown_connections 2023-12-26T21:56:28.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.543+0000 7fafedfff700 1 -- 172.21.15.131:0/42888788 wait complete. 2023-12-26T21:56:28.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.544+0000 7fafedfff700 1 Processor -- start 2023-12-26T21:56:28.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.545+0000 7fafedfff700 1 -- start start 2023-12-26T21:56:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafedfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 0x7fafe8151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafedfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8820 0x7fafe813a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafedfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe80c8bf0 0x7fafe81551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafedfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe813ace0 con 0x7fafe8069d10 2023-12-26T21:56:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafedfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe8151070 con 0x7fafe80c8bf0 2023-12-26T21:56:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafedfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe8150460 con 0x7fafe80c8820 2023-12-26T21:56:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 0x7fafe8151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 0x7fafe8151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53084/0 (socket says 172.21.15.131:53084) 2023-12-26T21:56:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe77fe700 1 -- 172.21.15.131:0/1545520696 learned_addr learned my addr 172.21.15.131:0/1545520696 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe7fff700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe80c8bf0 0x7fafe81551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe6ffd700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8820 0x7fafe813a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe77fe700 1 -- 172.21.15.131:0/1545520696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8820 msgr2=0x7fafe813a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe77fe700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8820 0x7fafe813a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe77fe700 1 -- 172.21.15.131:0/1545520696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe80c8bf0 msgr2=0x7fafe81551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe77fe700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe80c8bf0 0x7fafe81551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe77fe700 1 -- 172.21.15.131:0/1545520696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe0009ae0 con 0x7fafe8069d10 2023-12-26T21:56:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.546+0000 7fafe77fe700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 0x7fafe8151500 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fafd800ec80 tx=0x7fafd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.547+0000 7fafcf7fe700 1 -- 172.21.15.131:0/1545520696 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd800c9d0 con 0x7fafe8069d10 2023-12-26T21:56:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.547+0000 7fafcf7fe700 1 -- 172.21.15.131:0/1545520696 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd800cb30 con 0x7fafe8069d10 2023-12-26T21:56:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.547+0000 7fafcf7fe700 1 -- 172.21.15.131:0/1545520696 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8013470 con 0x7fafe8069d10 2023-12-26T21:56:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.547+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe8150740 con 0x7fafe8069d10 2023-12-26T21:56:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.547+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe8150d00 con 0x7fafe8069d10 2023-12-26T21:56:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.548+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe80657e0 con 0x7fafe8069d10 2023-12-26T21:56:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.550+0000 7fafcf7fe700 1 -- 172.21.15.131:0/1545520696 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fafd8013820 con 0x7fafe8069d10 2023-12-26T21:56:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.550+0000 7fafcf7fe700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fafd4066f30 0x7fafd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.551+0000 7fafcf7fe700 1 -- 172.21.15.131:0/1545520696 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fafd80851d0 con 0x7fafe8069d10 2023-12-26T21:56:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.551+0000 7fafe6ffd700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fafd4066f30 0x7fafd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.551+0000 7fafe6ffd700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fafd4066f30 0x7fafd40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafe8156db0 tx=0x7fafdc008040 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.554+0000 7fafcf7fe700 1 -- 172.21.15.131:0/1545520696 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafd8052300 con 0x7fafe8069d10 2023-12-26T21:56:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:28.791+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fafe8151350 con 0x7fafe8069d10 2023-12-26T21:56:29.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.320+0000 7fafcf7fe700 1 -- 172.21.15.131:0/1545520696 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7fafd8055920 con 0x7fafe8069d10 2023-12-26T21:56:29.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.325+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fafd4066f30 msgr2=0x7fafd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.325+0000 7fafedfff700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fafd4066f30 0x7fafd40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafe8156db0 tx=0x7fafdc008040 comp rx=0 tx=0).stop 2023-12-26T21:56:29.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.325+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 msgr2=0x7fafe8151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.325+0000 7fafedfff700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 0x7fafe8151500 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fafd800ec80 tx=0x7fafd80091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.326+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 shutdown_connections 2023-12-26T21:56:29.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.326+0000 7fafedfff700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafe80c8bf0 0x7fafe81551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.326+0000 7fafedfff700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fafd4066f30 0x7fafd40693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.326+0000 7fafedfff700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe8069d10 0x7fafe8151500 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.326+0000 7fafedfff700 1 --2- 172.21.15.131:0/1545520696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafe80c8820 0x7fafe813a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.326+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 >> 172.21.15.131:0/1545520696 conn(0x7fafe8068040 msgr2=0x7fafe80b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:29.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.326+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 shutdown_connections 2023-12-26T21:56:29.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.326+0000 7fafedfff700 1 -- 172.21.15.131:0/1545520696 wait complete. 2023-12-26T21:56:29.328 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 16 and data pool 17 2023-12-26T21:56:29.347 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:56:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.538+0000 7fa1267ff700 1 Processor -- start 2023-12-26T21:56:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.538+0000 7fa1267ff700 1 -- start start 2023-12-26T21:56:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.539+0000 7fa1267ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1200c8820 0x7fa1200ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.539+0000 7fa1267ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200b19f0 0x7fa120161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.539+0000 7fa1267ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa120157a80 con 0x7fa1200c8bf0 2023-12-26T21:56:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.539+0000 7fa1267ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1200cac50 con 0x7fa1200c8820 2023-12-26T21:56:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.539+0000 7fa1267ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200caf30 con 0x7fa1200b19f0 2023-12-26T21:56:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa11ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1200c8820 0x7fa1200ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa124d9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200b19f0 0x7fa120161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa11ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1200c8820 0x7fa1200ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:56282/0 (socket says 172.21.15.131:56282) 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa124d9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200b19f0 0x7fa120161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:52036/0 (socket says 172.21.15.131:52036) 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa11ffff700 1 -- 172.21.15.131:0/537944650 learned_addr learned my addr 172.21.15.131:0/537944650 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa11f7fe700 1 --1- 172.21.15.131:0/537944650 >> v1:172.21.15.190:6789/0 conn(0x7fa1200c8bf0 0x7fa1200b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:37152/0 (socket says 172.21.15.131:37152) 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa11effd700 1 -- 172.21.15.131:0/537944650 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909254952 0 0) 0x7fa120157a80 con 0x7fa1200c8bf0 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa11effd700 1 -- 172.21.15.131:0/537944650 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa108003570 con 0x7fa1200c8bf0 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa124d9c700 1 -- 172.21.15.131:0/537944650 >> v2:172.21.15.190:3300/0 conn(0x7fa1200c8820 msgr2=0x7fa1200ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa124d9c700 1 --2- 172.21.15.131:0/537944650 >> v2:172.21.15.190:3300/0 conn(0x7fa1200c8820 0x7fa1200ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa124d9c700 1 -- 172.21.15.131:0/537944650 >> v1:172.21.15.190:6789/0 conn(0x7fa1200c8bf0 legacy=0x7fa1200b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa124d9c700 1 -- 172.21.15.131:0/537944650 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1200cb210 con 0x7fa1200b19f0 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.540+0000 7fa11ffff700 1 --2- 172.21.15.131:0/537944650 >> v2:172.21.15.190:3300/0 conn(0x7fa1200c8820 0x7fa1200ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.541+0000 7fa124d9c700 1 --2- 172.21.15.131:0/537944650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200b19f0 0x7fa120161090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa118009e60 tx=0x7fa118009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bee55be88c579d5b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.541+0000 7fa11effd700 1 -- 172.21.15.131:0/537944650 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800e070 con 0x7fa1200b19f0 2023-12-26T21:56:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.541+0000 7fa11effd700 1 -- 172.21.15.131:0/537944650 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11800bab0 con 0x7fa1200b19f0 2023-12-26T21:56:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.541+0000 7fa11effd700 1 -- 172.21.15.131:0/537944650 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800be70 con 0x7fa1200b19f0 2023-12-26T21:56:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.542+0000 7fa1267ff700 1 -- 172.21.15.131:0/537944650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200b19f0 msgr2=0x7fa120161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.542+0000 7fa1267ff700 1 --2- 172.21.15.131:0/537944650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200b19f0 0x7fa120161090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa118009e60 tx=0x7fa118009780 comp rx=0 tx=0).stop 2023-12-26T21:56:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.542+0000 7fa1267ff700 1 -- 172.21.15.131:0/537944650 shutdown_connections 2023-12-26T21:56:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.542+0000 7fa1267ff700 1 --2- 172.21.15.131:0/537944650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200b19f0 0x7fa120161090 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.542+0000 7fa1267ff700 1 --2- 172.21.15.131:0/537944650 >> v2:172.21.15.190:3300/0 conn(0x7fa1200c8820 0x7fa1200ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.542+0000 7fa1267ff700 1 -- 172.21.15.131:0/537944650 >> 172.21.15.131:0/537944650 conn(0x7fa120068040 msgr2=0x7fa120069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.542+0000 7fa1267ff700 1 -- 172.21.15.131:0/537944650 shutdown_connections 2023-12-26T21:56:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.542+0000 7fa1267ff700 1 -- 172.21.15.131:0/537944650 wait complete. 2023-12-26T21:56:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.543+0000 7fa1267ff700 1 Processor -- start 2023-12-26T21:56:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.543+0000 7fa1267ff700 1 -- start start 2023-12-26T21:56:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.544+0000 7fa1267ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1200b19f0 0x7fa12015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.544+0000 7fa1267ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200c8820 0x7fa120152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.544+0000 7fa1267ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1200c8bf0 0x7fa1201696d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.544+0000 7fa1267ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200af0d0 con 0x7fa1200c8bf0 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.544+0000 7fa1267ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12015dd10 con 0x7fa1200b19f0 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.544+0000 7fa1267ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200af320 con 0x7fa1200c8820 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa124d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1200c8bf0 0x7fa1201696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa124d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1200c8bf0 0x7fa1201696d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53088/0 (socket says 172.21.15.131:53088) 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa124d9c700 1 -- 172.21.15.131:0/113199125 learned_addr learned my addr 172.21.15.131:0/113199125 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa11f7fe700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200c8820 0x7fa120152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa11ffff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1200b19f0 0x7fa12015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa124d9c700 1 -- 172.21.15.131:0/113199125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200c8820 msgr2=0x7fa120152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa124d9c700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200c8820 0x7fa120152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa124d9c700 1 -- 172.21.15.131:0/113199125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1200b19f0 msgr2=0x7fa12015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa124d9c700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1200b19f0 0x7fa12015c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa124d9c700 1 -- 172.21.15.131:0/113199125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100094f0 con 0x7fa1200c8bf0 2023-12-26T21:56:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.545+0000 7fa124d9c700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1200c8bf0 0x7fa1201696d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa118013000 tx=0x7fa118019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.546+0000 7fa107fff700 1 -- 172.21.15.131:0/113199125 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118012e10 con 0x7fa1200c8bf0 2023-12-26T21:56:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.546+0000 7fa107fff700 1 -- 172.21.15.131:0/113199125 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa118027470 con 0x7fa1200c8bf0 2023-12-26T21:56:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.546+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa118009a20 con 0x7fa1200c8bf0 2023-12-26T21:56:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.546+0000 7fa107fff700 1 -- 172.21.15.131:0/113199125 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118027830 con 0x7fa1200c8bf0 2023-12-26T21:56:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.546+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1200af8e0 con 0x7fa1200c8bf0 2023-12-26T21:56:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.547+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1200657e0 con 0x7fa1200c8bf0 2023-12-26T21:56:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.548+0000 7fa107fff700 1 -- 172.21.15.131:0/113199125 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa118024070 con 0x7fa1200c8bf0 2023-12-26T21:56:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.548+0000 7fa107fff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa10c066f30 0x7fa10c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.549+0000 7fa107fff700 1 -- 172.21.15.131:0/113199125 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fa1180870c0 con 0x7fa1200c8bf0 2023-12-26T21:56:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.549+0000 7fa11ffff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa10c066f30 0x7fa10c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.549+0000 7fa11ffff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa10c066f30 0x7fa10c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa110006d20 tx=0x7fa110009250 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:29.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.553+0000 7fa107fff700 1 -- 172.21.15.131:0/113199125 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1180540f0 con 0x7fa1200c8bf0 2023-12-26T21:56:29.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.744+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1200afb00 con 0x7fa1200c8bf0 2023-12-26T21:56:29.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.745+0000 7fa107fff700 1 -- 172.21.15.131:0/113199125 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16066 (secure 0 0 0) 0x7fa118057710 con 0x7fa1200c8bf0 2023-12-26T21:56:29.745 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:29.745 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":138,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:56:28.792286+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:56:26.517640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-26T21:56:27.777447+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa10c066f30 msgr2=0x7fa10c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa10c066f30 0x7fa10c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa110006d20 tx=0x7fa110009250 comp rx=0 tx=0).stop 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1200c8bf0 msgr2=0x7fa1201696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1200c8bf0 0x7fa1201696d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa118013000 tx=0x7fa118019000 comp rx=0 tx=0).stop 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 shutdown_connections 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa10c066f30 0x7fa10c0693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1200c8bf0 0x7fa1201696d0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1200b19f0 0x7fa12015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 --2- 172.21.15.131:0/113199125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1200c8820 0x7fa120152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 >> 172.21.15.131:0/113199125 conn(0x7fa120068040 msgr2=0x7fa12006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:29.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 shutdown_connections 2023-12-26T21:56:29.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.747+0000 7fa1267ff700 1 -- 172.21.15.131:0/113199125 wait complete. 2023-12-26T21:56:29.765 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:56:29.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-12-26T21:56:29.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.954+0000 7f5210bff700 1 Processor -- start 2023-12-26T21:56:29.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.955+0000 7f5210bff700 1 -- start start 2023-12-26T21:56:29.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.956+0000 7f5210bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 0x7f520c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.956+0000 7f5210bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f520c0cc100 0x7f520c0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.956+0000 7f5210bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520c0686c0 con 0x7f520c13c510 2023-12-26T21:56:29.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.956+0000 7f5210bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520c069520 con 0x7f520c0cc100 2023-12-26T21:56:29.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.956+0000 7f5210bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c068910 con 0x7f520c06bfd0 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f520a59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f520c13c510 0x7f520c06b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48888/0 (socket says 172.21.15.131:48888) 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f520a59c700 1 -- 172.21.15.131:0/3676859078 learned_addr learned my addr 172.21.15.131:0/3676859078 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f5209d9b700 1 --2- 172.21.15.131:0/3676859078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 0x7f520c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f520ad9d700 1 --2- 172.21.15.131:0/3676859078 >> v2:172.21.15.190:3300/0 conn(0x7f520c0cc100 0x7f520c0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f520959a700 1 -- 172.21.15.131:0/3676859078 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248173294 0 0) 0x7f520c0686c0 con 0x7f520c13c510 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f520959a700 1 -- 172.21.15.131:0/3676859078 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f4003570 con 0x7f520c13c510 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f5209d9b700 1 -- 172.21.15.131:0/3676859078 >> v2:172.21.15.190:3300/0 conn(0x7f520c0cc100 msgr2=0x7f520c0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f5209d9b700 1 --2- 172.21.15.131:0/3676859078 >> v2:172.21.15.190:3300/0 conn(0x7f520c0cc100 0x7f520c0cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f5209d9b700 1 -- 172.21.15.131:0/3676859078 >> v1:172.21.15.131:6789/0 conn(0x7f520c13c510 legacy=0x7f520c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.957+0000 7f5209d9b700 1 -- 172.21.15.131:0/3676859078 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c068bf0 con 0x7f520c06bfd0 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f5209d9b700 1 --2- 172.21.15.131:0/3676859078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 0x7f520c0c8a20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52000098e0 tx=0x7f52000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b612921225d1f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f520959a700 1 -- 172.21.15.131:0/3676859078 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200011630 con 0x7f520c06bfd0 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f520959a700 1 -- 172.21.15.131:0/3676859078 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200011790 con 0x7f520c06bfd0 2023-12-26T21:56:29.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f520959a700 1 -- 172.21.15.131:0/3676859078 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200011b30 con 0x7f520c06bfd0 2023-12-26T21:56:29.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f5210bff700 1 -- 172.21.15.131:0/3676859078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 msgr2=0x7f520c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f5210bff700 1 --2- 172.21.15.131:0/3676859078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 0x7f520c0c8a20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52000098e0 tx=0x7f52000091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f5210bff700 1 -- 172.21.15.131:0/3676859078 shutdown_connections 2023-12-26T21:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f5210bff700 1 --2- 172.21.15.131:0/3676859078 >> v2:172.21.15.190:3300/0 conn(0x7f520c0cc100 0x7f520c0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f5210bff700 1 --2- 172.21.15.131:0/3676859078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 0x7f520c0c8a20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.958+0000 7f5210bff700 1 -- 172.21.15.131:0/3676859078 >> 172.21.15.131:0/3676859078 conn(0x7f520c0ce3e0 msgr2=0x7f520c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.959+0000 7f5210bff700 1 -- 172.21.15.131:0/3676859078 shutdown_connections 2023-12-26T21:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.959+0000 7f5210bff700 1 -- 172.21.15.131:0/3676859078 wait complete. 2023-12-26T21:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.960+0000 7f5210bff700 1 Processor -- start 2023-12-26T21:56:29.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.960+0000 7f5210bff700 1 -- start start 2023-12-26T21:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f5210bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 0x7f520c0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f5210bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0cc100 0x7f520c0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f5210bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520c13c510 0x7f520c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f5210bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c15a290 con 0x7f520c13c510 2023-12-26T21:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f5210bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c15a570 con 0x7f520c0cc100 2023-12-26T21:56:29.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f5210bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c15a850 con 0x7f520c06bfd0 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f520ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520c13c510 0x7f520c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f520ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520c13c510 0x7f520c15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53092/0 (socket says 172.21.15.131:53092) 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f520ad9d700 1 -- 172.21.15.131:0/1641302161 learned_addr learned my addr 172.21.15.131:0/1641302161 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f5209d9b700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0cc100 0x7f520c0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.961+0000 7f520a59c700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 0x7f520c0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f520ad9d700 1 -- 172.21.15.131:0/1641302161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 msgr2=0x7f520c0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f520ad9d700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 0x7f520c0b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f520ad9d700 1 -- 172.21.15.131:0/1641302161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0cc100 msgr2=0x7f520c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f520ad9d700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0cc100 0x7f520c0b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:29.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f520ad9d700 1 -- 172.21.15.131:0/1641302161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5200009420 con 0x7f520c13c510 2023-12-26T21:56:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f520ad9d700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520c13c510 0x7f520c15dcb0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f5204009e90 tx=0x7f520400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f51fa7fc700 1 -- 172.21.15.131:0/1641302161 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520400cb70 con 0x7f520c13c510 2023-12-26T21:56:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f51fa7fc700 1 -- 172.21.15.131:0/1641302161 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520400f040 con 0x7f520c13c510 2023-12-26T21:56:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520c15ab30 con 0x7f520c13c510 2023-12-26T21:56:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f51fa7fc700 1 -- 172.21.15.131:0/1641302161 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204014480 con 0x7f520c13c510 2023-12-26T21:56:29.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.962+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520c159d20 con 0x7f520c13c510 2023-12-26T21:56:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.963+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520c15afd0 con 0x7f520c13c510 2023-12-26T21:56:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.964+0000 7f51fa7fc700 1 -- 172.21.15.131:0/1641302161 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5204014830 con 0x7f520c13c510 2023-12-26T21:56:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.965+0000 7f51fa7fc700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51ec066f30 0x7f51ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.965+0000 7f51fa7fc700 1 -- 172.21.15.131:0/1641302161 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f5204085140 con 0x7f520c13c510 2023-12-26T21:56:29.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.966+0000 7f520a59c700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51ec066f30 0x7f51ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:29.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.966+0000 7f520a59c700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51ec066f30 0x7f51ec0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f520c15eea0 tx=0x7f51fc009040 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:29.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:29.969+0000 7f51fa7fc700 1 -- 172.21.15.131:0/1641302161 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5204052170 con 0x7f520c13c510 2023-12-26T21:56:30.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.160+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f520c0657e0 con 0x7f520c13c510 2023-12-26T21:56:30.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.161+0000 7f51fa7fc700 1 -- 172.21.15.131:0/1641302161 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16066 (secure 0 0 0) 0x7f5204055790 con 0x7f520c13c510 2023-12-26T21:56:30.161 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:30.161 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":138,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:56:28.792286+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:56:26.517640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-26T21:56:27.777447+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.163+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51ec066f30 msgr2=0x7f51ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.163+0000 7f5210bff700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51ec066f30 0x7f51ec0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f520c15eea0 tx=0x7f51fc009040 comp rx=0 tx=0).stop 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.163+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520c13c510 msgr2=0x7f520c15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.164+0000 7f5210bff700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520c13c510 0x7f520c15dcb0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f5204009e90 tx=0x7f520400b040 comp rx=0 tx=0).stop 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.164+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 shutdown_connections 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.164+0000 7f5210bff700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f51ec066f30 0x7f51ec0693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.164+0000 7f5210bff700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f520c13c510 0x7f520c15dcb0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.164+0000 7f5210bff700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f520c06bfd0 0x7f520c0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.164+0000 7f5210bff700 1 --2- 172.21.15.131:0/1641302161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f520c0cc100 0x7f520c0b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.164+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 >> 172.21.15.131:0/1641302161 conn(0x7f520c0ce3e0 msgr2=0x7f520c152460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.164+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 shutdown_connections 2023-12-26T21:56:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.164+0000 7f5210bff700 1 -- 172.21.15.131:0/1641302161 wait complete. 2023-12-26T21:56:30.182 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T21:56:30.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.379+0000 7f9a03bff700 1 Processor -- start 2023-12-26T21:56:30.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.379+0000 7f9a03bff700 1 -- start start 2023-12-26T21:56:30.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.380+0000 7f9a03bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f99fc159600 0x7f99fc0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:30.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.380+0000 7f9a03bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0d01e0 0x7f99fc0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:30.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a03bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99fc0cfd50 con 0x7f99fc1599d0 2023-12-26T21:56:30.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a03bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99fc0b1280 con 0x7f99fc159600 2023-12-26T21:56:30.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a03bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0cf140 con 0x7f99fc0d01e0 2023-12-26T21:56:30.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a0199b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f99fc159600 0x7f99fc0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:30.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a0119a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f99fc1599d0 0x7f99fc15dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48896/0 (socket says 172.21.15.131:48896) 2023-12-26T21:56:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a0119a700 1 -- 172.21.15.131:0/356590477 learned_addr learned my addr 172.21.15.131:0/356590477 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a0199b700 1 -- 172.21.15.131:0/356590477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0d01e0 msgr2=0x7f99fc0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a0199b700 1 --2- 172.21.15.131:0/356590477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0d01e0 0x7f99fc0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a0199b700 1 -- 172.21.15.131:0/356590477 >> v1:172.21.15.131:6789/0 conn(0x7f99fc1599d0 legacy=0x7f99fc15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a0199b700 1 -- 172.21.15.131:0/356590477 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0cf420 con 0x7f99fc159600 2023-12-26T21:56:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.381+0000 7f9a00999700 1 -- 172.21.15.131:0/356590477 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721808438 0 0) 0x7f99fc0cfd50 con 0x7f99fc1599d0 2023-12-26T21:56:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a0199b700 1 --2- 172.21.15.131:0/356590477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc159600 0x7f99fc0ceb60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f99f80098d0 tx=0x7f99f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f99e86de4aaa29 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a00999700 1 -- 172.21.15.131:0/356590477 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8009e70 con 0x7f99fc159600 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a00999700 1 -- 172.21.15.131:0/356590477 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f800b860 con 0x7f99fc159600 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a00999700 1 -- 172.21.15.131:0/356590477 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800bc40 con 0x7f99fc159600 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a03bff700 1 -- 172.21.15.131:0/356590477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc159600 msgr2=0x7f99fc0ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a03bff700 1 --2- 172.21.15.131:0/356590477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc159600 0x7f99fc0ceb60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f99f80098d0 tx=0x7f99f80091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a03bff700 1 -- 172.21.15.131:0/356590477 shutdown_connections 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a03bff700 1 --2- 172.21.15.131:0/356590477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0d01e0 0x7f99fc0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a03bff700 1 --2- 172.21.15.131:0/356590477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc159600 0x7f99fc0ceb60 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a03bff700 1 -- 172.21.15.131:0/356590477 >> 172.21.15.131:0/356590477 conn(0x7f99fc06ece0 msgr2=0x7f99fc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a03bff700 1 -- 172.21.15.131:0/356590477 shutdown_connections 2023-12-26T21:56:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.382+0000 7f9a03bff700 1 -- 172.21.15.131:0/356590477 wait complete. 2023-12-26T21:56:30.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.384+0000 7f9a03bff700 1 Processor -- start 2023-12-26T21:56:30.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.384+0000 7f9a03bff700 1 -- start start 2023-12-26T21:56:30.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a03bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0d01e0 0x7f99fc0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:30.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a03bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc159600 0x7f99fc0b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:30.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a03bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc15e060 0x7f99fc0c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:30.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a03bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0caa60 con 0x7f99fc15e060 2023-12-26T21:56:30.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a03bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0b3800 con 0x7f99fc159600 2023-12-26T21:56:30.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a03bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0b61d0 con 0x7f99fc0d01e0 2023-12-26T21:56:30.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a0219c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc15e060 0x7f99fc0c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a0219c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc15e060 0x7f99fc0c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53112/0 (socket says 172.21.15.131:53112) 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a0219c700 1 -- 172.21.15.131:0/3241970739 learned_addr learned my addr 172.21.15.131:0/3241970739 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a0119a700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc159600 0x7f99fc0b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.385+0000 7f9a0199b700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0d01e0 0x7f99fc0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f9a0219c700 1 -- 172.21.15.131:0/3241970739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0d01e0 msgr2=0x7f99fc0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f9a0219c700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0d01e0 0x7f99fc0ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f9a0219c700 1 -- 172.21.15.131:0/3241970739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc159600 msgr2=0x7f99fc0b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f9a0219c700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc159600 0x7f99fc0b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f9a0219c700 1 -- 172.21.15.131:0/3241970739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f80093c0 con 0x7f99fc15e060 2023-12-26T21:56:30.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f9a0219c700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc15e060 0x7f99fc0c68b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f99f400bb30 tx=0x7f99f40097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:30.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f99f1ffb700 1 -- 172.21.15.131:0/3241970739 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400c7c0 con 0x7f99fc15e060 2023-12-26T21:56:30.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f99f1ffb700 1 -- 172.21.15.131:0/3241970739 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f400c920 con 0x7f99fc15e060 2023-12-26T21:56:30.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f99f1ffb700 1 -- 172.21.15.131:0/3241970739 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400cce0 con 0x7f99fc15e060 2023-12-26T21:56:30.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc0b3a70 con 0x7f99fc15e060 2023-12-26T21:56:30.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.386+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc0b4030 con 0x7f99fc15e060 2023-12-26T21:56:30.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.387+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc0b41f0 con 0x7f99fc15e060 2023-12-26T21:56:30.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.389+0000 7f99f1ffb700 1 -- 172.21.15.131:0/3241970739 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f99f4014620 con 0x7f99fc15e060 2023-12-26T21:56:30.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.389+0000 7f99f1ffb700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f99e0066f30 0x7f99e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:30.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.389+0000 7f99f1ffb700 1 -- 172.21.15.131:0/3241970739 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f99f4084640 con 0x7f99fc15e060 2023-12-26T21:56:30.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.390+0000 7f9a0199b700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f99e0066f30 0x7f99e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:30.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.390+0000 7f9a0199b700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f99e0066f30 0x7f99e00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009fd0 tx=0x7f99f8017000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:30.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.393+0000 7f99f1ffb700 1 -- 172.21.15.131:0/3241970739 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99f4051710 con 0x7f99fc15e060 2023-12-26T21:56:30.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:30.631+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f99fc088420 con 0x7f99fc15e060 2023-12-26T21:56:31.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.334+0000 7f99f1ffb700 1 -- 172.21.15.131:0/3241970739 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) v1 ==== 118+0+0 (secure 0 0 0) 0x7f99f4054d30 con 0x7f99fc15e060 2023-12-26T21:56:31.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f99e0066f30 msgr2=0x7f99e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f99e0066f30 0x7f99e00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009fd0 tx=0x7f99f8017000 comp rx=0 tx=0).stop 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc15e060 msgr2=0x7f99fc0c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc15e060 0x7f99fc0c68b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f99f400bb30 tx=0x7f99f40097e0 comp rx=0 tx=0).stop 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 shutdown_connections 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f99e0066f30 0x7f99e00693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99fc15e060 0x7f99fc0c68b0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99fc0d01e0 0x7f99fc0ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 --2- 172.21.15.131:0/3241970739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99fc159600 0x7f99fc0b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 >> 172.21.15.131:0/3241970739 conn(0x7f99fc06ece0 msgr2=0x7f99fc0cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 shutdown_connections 2023-12-26T21:56:31.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.338+0000 7f9a03bff700 1 -- 172.21.15.131:0/3241970739 wait complete. 2023-12-26T21:56:31.358 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T21:56:31.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.555+0000 7fd1ab5ff700 1 Processor -- start 2023-12-26T21:56:31.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.556+0000 7fd1ab5ff700 1 -- start start 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.557+0000 7fd1ab5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd1a406e480 0x7fd1a40abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1ab5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a415a8c0 con 0x7fd1a40aa480 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1ab5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a4159f90 con 0x7fd1a4159820 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1ab5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a415ad50 con 0x7fd1a406e480 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1a8b9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd1a406e480 0x7fd1a40abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1a8b9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd1a406e480 0x7fd1a40abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53124/0 (socket says 172.21.15.131:53124) 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1a8b9a700 1 -- 172.21.15.131:0/1051335464 learned_addr learned my addr 172.21.15.131:0/1051335464 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1a9b9c700 1 --1- 172.21.15.131:0/1051335464 >> v1:172.21.15.131:6789/0 conn(0x7fd1a40aa480 0x7fd1a40675b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48912/0 (socket says 172.21.15.131:48912) 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1a8b9a700 1 -- 172.21.15.131:0/1051335464 >> v1:172.21.15.190:6789/0 conn(0x7fd1a4159820 legacy=0x7fd1a406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1a8b9a700 1 -- 172.21.15.131:0/1051335464 >> v1:172.21.15.131:6789/0 conn(0x7fd1a40aa480 legacy=0x7fd1a40675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd1a8b9a700 1 -- 172.21.15.131:0/1051335464 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a415a270 con 0x7fd1a406e480 2023-12-26T21:56:31.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.558+0000 7fd19bfff700 1 -- 172.21.15.131:0/1051335464 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546753870 0 0) 0x7fd1a415a8c0 con 0x7fd1a40aa480 2023-12-26T21:56:31.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd1a8b9a700 1 --2- 172.21.15.131:0/1051335464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 0x7fd1a40abdb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd1940099b0 tx=0x7fd194009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1489278e1ee36fc3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:31.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd19bfff700 1 -- 172.21.15.131:0/1051335464 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19400b820 con 0x7fd1a406e480 2023-12-26T21:56:31.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd19bfff700 1 -- 172.21.15.131:0/1051335464 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19400b980 con 0x7fd1a406e480 2023-12-26T21:56:31.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd19bfff700 1 -- 172.21.15.131:0/1051335464 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19400bd20 con 0x7fd1a406e480 2023-12-26T21:56:31.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/1051335464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 msgr2=0x7fd1a40abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:31.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd1ab5ff700 1 --2- 172.21.15.131:0/1051335464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 0x7fd1a40abdb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd1940099b0 tx=0x7fd194009250 comp rx=0 tx=0).stop 2023-12-26T21:56:31.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/1051335464 shutdown_connections 2023-12-26T21:56:31.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd1ab5ff700 1 --2- 172.21.15.131:0/1051335464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 0x7fd1a40abdb0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:31.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/1051335464 >> 172.21.15.131:0/1051335464 conn(0x7fd1a406c290 msgr2=0x7fd1a40ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:31.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.559+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/1051335464 shutdown_connections 2023-12-26T21:56:31.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.560+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/1051335464 wait complete. 2023-12-26T21:56:31.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.561+0000 7fd1ab5ff700 1 Processor -- start 2023-12-26T21:56:31.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.561+0000 7fd1ab5ff700 1 -- start start 2023-12-26T21:56:31.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.562+0000 7fd1ab5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 0x7fd1a4155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.562+0000 7fd1ab5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40aa480 0x7fd1a40c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.562+0000 7fd1ab5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a4159820 0x7fd1a40c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.562+0000 7fd1ab5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40c2680 con 0x7fd1a406e480 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.562+0000 7fd1ab5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40c2960 con 0x7fd1a40aa480 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.562+0000 7fd1ab5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40c2c40 con 0x7fd1a4159820 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.562+0000 7fd1a939b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 0x7fd1a4155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.562+0000 7fd1a939b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 0x7fd1a4155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53128/0 (socket says 172.21.15.131:53128) 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.562+0000 7fd1a939b700 1 -- 172.21.15.131:0/3124012192 learned_addr learned my addr 172.21.15.131:0/3124012192 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1a939b700 1 -- 172.21.15.131:0/3124012192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a4159820 msgr2=0x7fd1a40c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1a939b700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a4159820 0x7fd1a40c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1a939b700 1 -- 172.21.15.131:0/3124012192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40aa480 msgr2=0x7fd1a40c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1a939b700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40aa480 0x7fd1a40c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1a939b700 1 -- 172.21.15.131:0/3124012192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940094f0 con 0x7fd1a406e480 2023-12-26T21:56:31.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1a939b700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 0x7fd1a4155fd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000b5e0 tx=0x7fd1a0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:31.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1997fa700 1 -- 172.21.15.131:0/3124012192 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000f510 con 0x7fd1a406e480 2023-12-26T21:56:31.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1997fa700 1 -- 172.21.15.131:0/3124012192 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a000f670 con 0x7fd1a406e480 2023-12-26T21:56:31.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a40c2f20 con 0x7fd1a406e480 2023-12-26T21:56:31.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1997fa700 1 -- 172.21.15.131:0/3124012192 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000fa30 con 0x7fd1a406e480 2023-12-26T21:56:31.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.563+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a4156510 con 0x7fd1a406e480 2023-12-26T21:56:31.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.564+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a40657e0 con 0x7fd1a406e480 2023-12-26T21:56:31.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.565+0000 7fd1997fa700 1 -- 172.21.15.131:0/3124012192 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd1a001a400 con 0x7fd1a406e480 2023-12-26T21:56:31.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.566+0000 7fd1997fa700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd188066f30 0x7fd1880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:31.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.566+0000 7fd1997fa700 1 -- 172.21.15.131:0/3124012192 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fd1a0084200 con 0x7fd1a406e480 2023-12-26T21:56:31.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.566+0000 7fd1a8b9a700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd188066f30 0x7fd1880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:31.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.567+0000 7fd1a8b9a700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd188066f30 0x7fd1880693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd1940099b0 tx=0x7fd194016000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:31.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.570+0000 7fd1997fa700 1 -- 172.21.15.131:0/3124012192 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1a0051350 con 0x7fd1a406e480 2023-12-26T21:56:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:31.805+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd1a415a550 con 0x7fd1a406e480 2023-12-26T21:56:32.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.340+0000 7fd1997fa700 1 -- 172.21.15.131:0/3124012192 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1a0054970 con 0x7fd1a406e480 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd188066f30 msgr2=0x7fd1880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd188066f30 0x7fd1880693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd1940099b0 tx=0x7fd194016000 comp rx=0 tx=0).stop 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 msgr2=0x7fd1a4155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 0x7fd1a4155fd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000b5e0 tx=0x7fd1a0009120 comp rx=0 tx=0).stop 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 shutdown_connections 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a4159820 0x7fd1a40c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd188066f30 0x7fd1880693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a406e480 0x7fd1a4155fd0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 --2- 172.21.15.131:0/3124012192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40aa480 0x7fd1a40c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 >> 172.21.15.131:0/3124012192 conn(0x7fd1a406c290 msgr2=0x7fd1a4156ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:32.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 shutdown_connections 2023-12-26T21:56:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.343+0000 7fd1ab5ff700 1 -- 172.21.15.131:0/3124012192 wait complete. 2023-12-26T21:56:32.364 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:56:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.564+0000 7f0c49fff700 1 Processor -- start 2023-12-26T21:56:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.564+0000 7f0c49fff700 1 -- start start 2023-12-26T21:56:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.565+0000 7f0c49fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c440c8820 0x7f0c440ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.565+0000 7f0c49fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0c440c8bf0 0x7f0c440b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.565+0000 7f0c49fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c44157a80 con 0x7f0c44069d10 2023-12-26T21:56:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.565+0000 7f0c49fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c440cac50 con 0x7f0c440c8bf0 2023-12-26T21:56:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.565+0000 7f0c49fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440caf30 con 0x7f0c440c8820 2023-12-26T21:56:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.565+0000 7f0c42ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0c440c8bf0 0x7f0c440b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.565+0000 7f0c42ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0c440c8bf0 0x7f0c440b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53142/0 (socket says 172.21.15.131:53142) 2023-12-26T21:56:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.565+0000 7f0c42ffd700 1 -- 172.21.15.131:0/3443159624 learned_addr learned my addr 172.21.15.131:0/3443159624 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.566+0000 7f0c437fe700 1 --2- 172.21.15.131:0/3443159624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c440c8820 0x7f0c440ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.566+0000 7f0c42ffd700 1 -- 172.21.15.131:0/3443159624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c440c8820 msgr2=0x7f0c440ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.566+0000 7f0c42ffd700 1 --2- 172.21.15.131:0/3443159624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c440c8820 0x7f0c440ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.566+0000 7f0c42ffd700 1 -- 172.21.15.131:0/3443159624 >> v1:172.21.15.190:6789/0 conn(0x7f0c44069d10 legacy=0x7f0c44157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.566+0000 7f0c42ffd700 1 -- 172.21.15.131:0/3443159624 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440cb210 con 0x7f0c440c8bf0 2023-12-26T21:56:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.566+0000 7f0c437fe700 1 --2- 172.21.15.131:0/3443159624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c440c8820 0x7f0c440ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.566+0000 7f0c42ffd700 1 --2- 172.21.15.131:0/3443159624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8bf0 0x7f0c440b12e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0c380099b0 tx=0x7f0c38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c2058470e8c0d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c427fc700 1 -- 172.21.15.131:0/3443159624 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800b860 con 0x7f0c440c8bf0 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c427fc700 1 -- 172.21.15.131:0/3443159624 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3800b9c0 con 0x7f0c440c8bf0 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c427fc700 1 -- 172.21.15.131:0/3443159624 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800bd60 con 0x7f0c440c8bf0 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c49fff700 1 -- 172.21.15.131:0/3443159624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8bf0 msgr2=0x7f0c440b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c49fff700 1 --2- 172.21.15.131:0/3443159624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8bf0 0x7f0c440b12e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0c380099b0 tx=0x7f0c38009250 comp rx=0 tx=0).stop 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c49fff700 1 -- 172.21.15.131:0/3443159624 shutdown_connections 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c49fff700 1 --2- 172.21.15.131:0/3443159624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c440c8820 0x7f0c440ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c49fff700 1 --2- 172.21.15.131:0/3443159624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8bf0 0x7f0c440b12e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c49fff700 1 -- 172.21.15.131:0/3443159624 >> 172.21.15.131:0/3443159624 conn(0x7f0c44068040 msgr2=0x7f0c44069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c49fff700 1 -- 172.21.15.131:0/3443159624 shutdown_connections 2023-12-26T21:56:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.567+0000 7f0c49fff700 1 -- 172.21.15.131:0/3443159624 wait complete. 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.569+0000 7f0c49fff700 1 Processor -- start 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.569+0000 7f0c49fff700 1 -- start start 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c49fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c44069d10 0x7f0c44169c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c49fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8820 0x7f0c4416c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c49fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c440c8bf0 0x7f0c440c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c49fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440af4f0 con 0x7f0c440c8820 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c49fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440af7d0 con 0x7f0c440c8bf0 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c49fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440afab0 con 0x7f0c44069d10 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c42ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8820 0x7f0c4416c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c42ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8820 0x7f0c4416c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53156/0 (socket says 172.21.15.131:53156) 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c42ffd700 1 -- 172.21.15.131:0/2401386329 learned_addr learned my addr 172.21.15.131:0/2401386329 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c437fe700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c44069d10 0x7f0c44169c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.570+0000 7f0c43fff700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c440c8bf0 0x7f0c440c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c42ffd700 1 -- 172.21.15.131:0/2401386329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c44069d10 msgr2=0x7f0c44169c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c42ffd700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c44069d10 0x7f0c44169c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c42ffd700 1 -- 172.21.15.131:0/2401386329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c440c8bf0 msgr2=0x7f0c440c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c42ffd700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c440c8bf0 0x7f0c440c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c42ffd700 1 -- 172.21.15.131:0/2401386329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c380094f0 con 0x7f0c440c8820 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c42ffd700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8820 0x7f0c4416c910 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009290 tx=0x7f0c38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c2b7fe700 1 -- 172.21.15.131:0/2401386329 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38026480 con 0x7f0c440c8820 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c2b7fe700 1 -- 172.21.15.131:0/2401386329 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3801f070 con 0x7f0c440c8820 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c2b7fe700 1 -- 172.21.15.131:0/2401386329 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38026830 con 0x7f0c440c8820 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c440afd90 con 0x7f0c440c8820 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.571+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4416a1b0 con 0x7f0c440c8820 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.572+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c440b0230 con 0x7f0c440c8820 2023-12-26T21:56:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.573+0000 7f0c2b7fe700 1 -- 172.21.15.131:0/2401386329 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0c38026ba0 con 0x7f0c440c8820 2023-12-26T21:56:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.574+0000 7f0c2b7fe700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0c30066f30 0x7f0c300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.574+0000 7f0c2b7fe700 1 -- 172.21.15.131:0/2401386329 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f0c38021070 con 0x7f0c440c8820 2023-12-26T21:56:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.574+0000 7f0c437fe700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0c30066f30 0x7f0c300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.575+0000 7f0c437fe700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0c30066f30 0x7f0c300693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0c34006f00 tx=0x7f0c34008040 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:32.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.578+0000 7f0c2b7fe700 1 -- 172.21.15.131:0/2401386329 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c38053610 con 0x7f0c440c8820 2023-12-26T21:56:32.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.814+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c440cba00 con 0x7f0c440c8820 2023-12-26T21:56:32.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.814+0000 7f0c2b7fe700 1 -- 172.21.15.131:0/2401386329 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f0c38056c30 con 0x7f0c440c8820 2023-12-26T21:56:32.815 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:32.815 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3746500235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3746500235},{"type":"v1","addr":"172.21.15.190:6835","nonce":3746500235}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8115,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2030200373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2030200373},{"type":"v1","addr":"172.21.15.131:6833","nonce":2030200373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2894985738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2894985738},{"type":"v1","addr":"172.21.15.190:6834","nonce":2894985738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:56:28.792645+0000","modified":"2023-12-26T21:56:32.333009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8142":{"gid":8142,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1841321761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1841321761},{"type":"v1","addr":"172.21.15.131:6835","nonce":1841321761}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-12-26T21:56:32.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.816+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0c30066f30 msgr2=0x7f0c300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:32.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.816+0000 7f0c49fff700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0c30066f30 0x7f0c300693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0c34006f00 tx=0x7f0c34008040 comp rx=0 tx=0).stop 2023-12-26T21:56:32.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.816+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8820 msgr2=0x7f0c4416c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:32.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.816+0000 7f0c49fff700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8820 0x7f0c4416c910 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009290 tx=0x7f0c38017000 comp rx=0 tx=0).stop 2023-12-26T21:56:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.817+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 shutdown_connections 2023-12-26T21:56:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.817+0000 7f0c49fff700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c440c8bf0 0x7f0c440c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.817+0000 7f0c49fff700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c44069d10 0x7f0c44169c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.817+0000 7f0c49fff700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0c30066f30 0x7f0c300693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.817+0000 7f0c49fff700 1 --2- 172.21.15.131:0/2401386329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c440c8820 0x7f0c4416c910 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.817+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 >> 172.21.15.131:0/2401386329 conn(0x7f0c44068040 msgr2=0x7f0c4414c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.817+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 shutdown_connections 2023-12-26T21:56:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:32.817+0000 7f0c49fff700 1 -- 172.21.15.131:0/2401386329 wait complete. 2023-12-26T21:56:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 66 2023-12-26T21:56:32.835 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T21:56:33.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.026+0000 7f8976bff700 1 Processor -- start 2023-12-26T21:56:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.026+0000 7f8976bff700 1 -- start start 2023-12-26T21:56:33.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.027+0000 7f8976bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89700cb320 0x7f8970069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.027+0000 7f8976bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f897006a1e0 0x7f89700b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.027+0000 7f8976bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8970156940 con 0x7f89700b0c90 2023-12-26T21:56:33.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.027+0000 7f8976bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89700c8cc0 con 0x7f897006a1e0 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.027+0000 7f8976bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89700ad700 con 0x7f89700cb320 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.027+0000 7f897519c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f89700b0c90 0x7f89700aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48922/0 (socket says 172.21.15.131:48922) 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.027+0000 7f897519c700 1 -- 172.21.15.131:0/3839308672 learned_addr learned my addr 172.21.15.131:0/3839308672 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f897499b700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89700cb320 0x7f8970069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896ffff700 1 --2- 172.21.15.131:0/3839308672 >> v2:172.21.15.190:3300/0 conn(0x7f897006a1e0 0x7f89700b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405439694 0 0) 0x7f8970156940 con 0x7f89700b0c90 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8960003570 con 0x7f89700b0c90 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1629256986 0 0) 0x7f8960003570 con 0x7f89700b0c90 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89700cb320 msgr2=0x7f8970069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89700cb320 0x7f8970069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 >> v2:172.21.15.190:3300/0 conn(0x7f897006a1e0 msgr2=0x7f89700b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 --2- 172.21.15.131:0/3839308672 >> v2:172.21.15.190:3300/0 conn(0x7f897006a1e0 0x7f89700b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89700ae500 con 0x7f89700b0c90 2023-12-26T21:56:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f89680017f0 con 0x7f89700b0c90 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896ffff700 1 --2- 172.21.15.131:0/3839308672 >> v2:172.21.15.190:3300/0 conn(0x7f897006a1e0 0x7f89700b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 >> v1:172.21.15.131:6789/0 conn(0x7f89700b0c90 legacy=0x7f89700aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8960003ae0 0x7f8960003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8960004810 0x7f8960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89700cb320 0x7f89600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89700c8cc0 con 0x7f8960003ae0 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896000dca0 con 0x7f8960004810 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.028+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896000de60 con 0x7f89700cb320 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.029+0000 7f897499b700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8960003ae0 0x7f8960003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.029+0000 7f896ffff700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8960004810 0x7f8960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.029+0000 7f897519c700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89700cb320 0x7f89600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.029+0000 7f897499b700 1 -- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89700cb320 msgr2=0x7f89600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.029+0000 7f897499b700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89700cb320 0x7f89600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.029+0000 7f897499b700 1 -- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8960004810 msgr2=0x7f8960004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.029+0000 7f897499b700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8960004810 0x7f8960004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.029+0000 7f897499b700 1 -- 172.21.15.131:0/3839308672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8960003570 con 0x7f8960003ae0 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.029+0000 7f897499b700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8960003ae0 0x7f8960003ef0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8964009c00 tx=0x7f89640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5171ba28bd5a00 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896400e710 con 0x7f8960003ae0 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896400e870 con 0x7f8960003ae0 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f896f7fe700 1 -- 172.21.15.131:0/3839308672 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896400ec30 con 0x7f8960003ae0 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f8976bff700 1 -- 172.21.15.131:0/3839308672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8960003ae0 msgr2=0x7f8960003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f8976bff700 1 --2- 172.21.15.131:0/3839308672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8960003ae0 0x7f8960003ef0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8964009c00 tx=0x7f89640091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f897519c700 1 -- 172.21.15.131:0/3839308672 reap_dead start 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f8976bff700 1 -- 172.21.15.131:0/3839308672 shutdown_connections 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f8976bff700 1 -- 172.21.15.131:0/3839308672 >> 172.21.15.131:0/3839308672 conn(0x7f8970068040 msgr2=0x7f8970069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f8976bff700 1 -- 172.21.15.131:0/3839308672 shutdown_connections 2023-12-26T21:56:33.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.030+0000 7f8976bff700 1 -- 172.21.15.131:0/3839308672 wait complete. 2023-12-26T21:56:33.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.032+0000 7f8976bff700 1 Processor -- start 2023-12-26T21:56:33.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.032+0000 7f8976bff700 1 -- start start 2023-12-26T21:56:33.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f8976bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8960003ae0 0x7f897014baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f8976bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f897014d3c0 0x7f897013ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f8976bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970168b80 0x7f8970168ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f8976bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8970169530 con 0x7f897014d3c0 2023-12-26T21:56:33.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f8976bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897014c320 con 0x7f8970168b80 2023-12-26T21:56:33.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f8976bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897014c600 con 0x7f8960003ae0 2023-12-26T21:56:33.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f896ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f897014d3c0 0x7f897013ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f896ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f897014d3c0 0x7f897013ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53172/0 (socket says 172.21.15.131:53172) 2023-12-26T21:56:33.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f896ffff700 1 -- 172.21.15.131:0/1918958140 learned_addr learned my addr 172.21.15.131:0/1918958140 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:33.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f896ffff700 1 -- 172.21.15.131:0/1918958140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8960003ae0 msgr2=0x7f897014baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f896ffff700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8960003ae0 0x7f897014baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f896ffff700 1 -- 172.21.15.131:0/1918958140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970168b80 msgr2=0x7f8970168ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f896ffff700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970168b80 0x7f8970168ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f896ffff700 1 -- 172.21.15.131:0/1918958140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897014c8e0 con 0x7f897014d3c0 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.033+0000 7f896ffff700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f897014d3c0 0x7f897013ec40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f895c011d70 tx=0x7f895c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.034+0000 7f896cff9700 1 -- 172.21.15.131:0/1918958140 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c00fcd0 con 0x7f897014d3c0 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.034+0000 7f896cff9700 1 -- 172.21.15.131:0/1918958140 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895c00fe30 con 0x7f897014d3c0 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.034+0000 7f896cff9700 1 -- 172.21.15.131:0/1918958140 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c014670 con 0x7f897014d3c0 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.034+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897014cbc0 con 0x7f897014d3c0 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.034+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897014c030 con 0x7f897014d3c0 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.035+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89700657e0 con 0x7f897014d3c0 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.036+0000 7f896cff9700 1 -- 172.21.15.131:0/1918958140 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f895c014aa0 con 0x7f897014d3c0 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.036+0000 7f896cff9700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8954066f30 0x7f89540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.036+0000 7f896cff9700 1 -- 172.21.15.131:0/1918958140 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f895c018030 con 0x7f897014d3c0 2023-12-26T21:56:33.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.037+0000 7f897499b700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8954066f30 0x7f89540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.037+0000 7f897499b700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8954066f30 0x7f89540693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89640091e0 tx=0x7f896401a000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:33.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.040+0000 7f896cff9700 1 -- 172.21.15.131:0/1918958140 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f895c052790 con 0x7f897014d3c0 2023-12-26T21:56:33.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.234+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8970159d20 con 0x7f897014d3c0 2023-12-26T21:56:33.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.235+0000 7f896cff9700 1 -- 172.21.15.131:0/1918958140 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16053 (secure 0 0 0) 0x7f895c006cb0 con 0x7f897014d3c0 2023-12-26T21:56:33.235 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:33.235 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":139,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T21:56:30.272736+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:56:26.517640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-26T21:56:27.777447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T21:56:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8954066f30 msgr2=0x7f89540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8954066f30 0x7f89540693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89640091e0 tx=0x7f896401a000 comp rx=0 tx=0).stop 2023-12-26T21:56:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f897014d3c0 msgr2=0x7f897013ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f897014d3c0 0x7f897013ec40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f895c011d70 tx=0x7f895c0091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 shutdown_connections 2023-12-26T21:56:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8970168b80 0x7f8970168ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8960003ae0 0x7f897014baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8954066f30 0x7f89540693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 --2- 172.21.15.131:0/1918958140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f897014d3c0 0x7f897013ec40 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.237+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 >> 172.21.15.131:0/1918958140 conn(0x7f8970068040 msgr2=0x7f897006e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:33.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.238+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 shutdown_connections 2023-12-26T21:56:33.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.238+0000 7f8976bff700 1 -- 172.21.15.131:0/1918958140 wait complete. 2023-12-26T21:56:33.256 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T21:56:33.256 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:56:33.290 INFO:tasks.ceph.mds.a.smithi131.stderr:2023-12-26T21:56:33.291+0000 7f8289ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T21:56:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.448+0000 7f81f39ff700 1 Processor -- start 2023-12-26T21:56:33.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.449+0000 7f81f39ff700 1 -- start start 2023-12-26T21:56:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.450+0000 7f81f39ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81ec06bfd0 0x7f81ec0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.450+0000 7f81f39ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec0cc100 0x7f81ec0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.450+0000 7f81f39ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec0686c0 con 0x7f81ec13c510 2023-12-26T21:56:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.450+0000 7f81f39ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec069520 con 0x7f81ec06bfd0 2023-12-26T21:56:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.450+0000 7f81f39ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec068910 con 0x7f81ec0cc100 2023-12-26T21:56:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.450+0000 7f81f0f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81ec06bfd0 0x7f81ec0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.450+0000 7f81f179b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f81ec13c510 0x7f81ec06b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48938/0 (socket says 172.21.15.131:48938) 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.450+0000 7f81f179b700 1 -- 172.21.15.131:0/887478057 learned_addr learned my addr 172.21.15.131:0/887478057 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.450+0000 7f81f1f9c700 1 --2- 172.21.15.131:0/887478057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec0cc100 0x7f81ec0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81f0f9a700 1 -- 172.21.15.131:0/887478057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec0cc100 msgr2=0x7f81ec0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81f0f9a700 1 --2- 172.21.15.131:0/887478057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec0cc100 0x7f81ec0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81f0f9a700 1 -- 172.21.15.131:0/887478057 >> v1:172.21.15.131:6789/0 conn(0x7f81ec13c510 legacy=0x7f81ec06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81f0f9a700 1 -- 172.21.15.131:0/887478057 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec068bf0 con 0x7f81ec06bfd0 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81f1f9c700 1 --2- 172.21.15.131:0/887478057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec0cc100 0x7f81ec0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81e3fff700 1 -- 172.21.15.131:0/887478057 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315130401 0 0) 0x7f81ec0686c0 con 0x7f81ec13c510 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81f0f9a700 1 --2- 172.21.15.131:0/887478057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 0x7f81ec0c8a20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0098e0 tx=0x7f81dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12adb49f49a4296b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81e3fff700 1 -- 172.21.15.131:0/887478057 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc009e80 con 0x7f81ec06bfd0 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81e3fff700 1 -- 172.21.15.131:0/887478057 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc00b860 con 0x7f81ec06bfd0 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.451+0000 7f81e3fff700 1 -- 172.21.15.131:0/887478057 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00bbe0 con 0x7f81ec06bfd0 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.452+0000 7f81f39ff700 1 -- 172.21.15.131:0/887478057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 msgr2=0x7f81ec0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.452+0000 7f81f39ff700 1 --2- 172.21.15.131:0/887478057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 0x7f81ec0c8a20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0098e0 tx=0x7f81dc0091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.452+0000 7f81f39ff700 1 -- 172.21.15.131:0/887478057 shutdown_connections 2023-12-26T21:56:33.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.452+0000 7f81f39ff700 1 --2- 172.21.15.131:0/887478057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec0cc100 0x7f81ec0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.452+0000 7f81f39ff700 1 --2- 172.21.15.131:0/887478057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 0x7f81ec0c8a20 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.452+0000 7f81f39ff700 1 -- 172.21.15.131:0/887478057 >> 172.21.15.131:0/887478057 conn(0x7f81ec0ce3e0 msgr2=0x7f81ec0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:33.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.452+0000 7f81f39ff700 1 -- 172.21.15.131:0/887478057 shutdown_connections 2023-12-26T21:56:33.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.452+0000 7f81f39ff700 1 -- 172.21.15.131:0/887478057 wait complete. 2023-12-26T21:56:33.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.453+0000 7f81f39ff700 1 Processor -- start 2023-12-26T21:56:33.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.453+0000 7f81f39ff700 1 -- start start 2023-12-26T21:56:33.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.454+0000 7f81f39ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 0x7f81ec156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.454+0000 7f81f39ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ec0cc100 0x7f81ec0b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.454+0000 7f81f39ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec13c510 0x7f81ec0cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.454+0000 7f81f39ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec154550 con 0x7f81ec06bfd0 2023-12-26T21:56:33.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.454+0000 7f81f39ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0b1a20 con 0x7f81ec0cc100 2023-12-26T21:56:33.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.454+0000 7f81f39ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0b1180 con 0x7f81ec13c510 2023-12-26T21:56:33.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.454+0000 7f81f179b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 0x7f81ec156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.454+0000 7f81f179b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 0x7f81ec156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53188/0 (socket says 172.21.15.131:53188) 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.454+0000 7f81f179b700 1 -- 172.21.15.131:0/3210930917 learned_addr learned my addr 172.21.15.131:0/3210930917 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f1f9c700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec13c510 0x7f81ec0cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f0f9a700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ec0cc100 0x7f81ec0b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f179b700 1 -- 172.21.15.131:0/3210930917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec13c510 msgr2=0x7f81ec0cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f179b700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec13c510 0x7f81ec0cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f179b700 1 -- 172.21.15.131:0/3210930917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ec0cc100 msgr2=0x7f81ec0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f179b700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ec0cc100 0x7f81ec0b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f179b700 1 -- 172.21.15.131:0/3210930917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc009420 con 0x7f81ec06bfd0 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f0f9a700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ec0cc100 0x7f81ec0b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f179b700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 0x7f81ec156ad0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f81e800c790 tx=0x7f81e8007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81e17fa700 1 -- 172.21.15.131:0/3210930917 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8001ea0 con 0x7f81ec06bfd0 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81e17fa700 1 -- 172.21.15.131:0/3210930917 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e80072e0 con 0x7f81ec06bfd0 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec0b1460 con 0x7f81ec06bfd0 2023-12-26T21:56:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81e17fa700 1 -- 172.21.15.131:0/3210930917 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8007680 con 0x7f81ec06bfd0 2023-12-26T21:56:33.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.455+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec157010 con 0x7f81ec06bfd0 2023-12-26T21:56:33.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.457+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec154340 con 0x7f81ec06bfd0 2023-12-26T21:56:33.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.457+0000 7f81e17fa700 1 -- 172.21.15.131:0/3210930917 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f81e8005a00 con 0x7f81ec06bfd0 2023-12-26T21:56:33.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.459+0000 7f81e17fa700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f81d0066f30 0x7f81d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.459+0000 7f81e17fa700 1 -- 172.21.15.131:0/3210930917 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f81e8083ac0 con 0x7f81ec06bfd0 2023-12-26T21:56:33.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.459+0000 7f81f0f9a700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f81d0066f30 0x7f81d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.460+0000 7f81f0f9a700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f81d0066f30 0x7f81d00693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f81ec06c3d0 tx=0x7f81dc017000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:33.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.463+0000 7f81e17fa700 1 -- 172.21.15.131:0/3210930917 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81e804b5a0 con 0x7f81ec06bfd0 2023-12-26T21:56:33.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.652+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81ec154690 con 0x7f81d0066f30 2023-12-26T21:56:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.662+0000 7f81e17fa700 1 -- 172.21.15.131:0/3210930917 <== mgr.7591 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289341 (secure 0 0 0) 0x7f81dc00b9d0 con 0x7f81d0066f30 2023-12-26T21:56:33.666 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:33.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f81d0066f30 msgr2=0x7f81d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f81d0066f30 0x7f81d00693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f81ec06c3d0 tx=0x7f81dc017000 comp rx=0 tx=0).stop 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 msgr2=0x7f81ec156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 0x7f81ec156ad0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f81e800c790 tx=0x7f81e8007c80 comp rx=0 tx=0).stop 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 shutdown_connections 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ec13c510 0x7f81ec0cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f81d0066f30 0x7f81d00693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81ec06bfd0 0x7f81ec156ad0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 --2- 172.21.15.131:0/3210930917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ec0cc100 0x7f81ec0b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 >> 172.21.15.131:0/3210930917 conn(0x7f81ec0ce3e0 msgr2=0x7f81ec0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.666+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 shutdown_connections 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.667+0000 7f81f39ff700 1 -- 172.21.15.131:0/3210930917 wait complete. 2023-12-26T21:56:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:56:33.684 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-26T21:56:32.859266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2454364,"kb_used_data":14620,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752520356,"statfs":{"total":773094113280,"available":770580844544,"internally_reserved":0,"allocated":14970880,"data_stored":7931512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.559629"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.306326+0000","last_change":"2023-12-26T21:56:29.302600+0000","last_active":"2023-12-26T21:56:29.306326+0000","last_peered":"2023-12-26T21:56:29.306326+0000","last_clean":"2023-12-26T21:56:29.306326+0000","last_became_active":"2023-12-26T21:56:29.298324+0000","last_became_peered":"2023-12-26T21:56:29.298324+0000","last_unstale":"2023-12-26T21:56:29.306326+0000","last_undegraded":"2023-12-26T21:56:29.306326+0000","last_fullsized":"2023-12-26T21:56:29.306326+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:13:24.218551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.302669+0000","last_change":"2023-12-26T21:56:29.300051+0000","last_active":"2023-12-26T21:56:29.302669+0000","last_peered":"2023-12-26T21:56:29.302669+0000","last_clean":"2023-12-26T21:56:29.302669+0000","last_became_active":"2023-12-26T21:56:29.299800+0000","last_became_peered":"2023-12-26T21:56:29.299800+0000","last_unstale":"2023-12-26T21:56:29.302669+0000","last_undegraded":"2023-12-26T21:56:29.302669+0000","last_fullsized":"2023-12-26T21:56:29.302669+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:01:07.411436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.302349+0000","last_change":"2023-12-26T21:56:29.299977+0000","last_active":"2023-12-26T21:56:29.302349+0000","last_peered":"2023-12-26T21:56:29.302349+0000","last_clean":"2023-12-26T21:56:29.302349+0000","last_became_active":"2023-12-26T21:56:29.299683+0000","last_became_peered":"2023-12-26T21:56:29.299683+0000","last_unstale":"2023-12-26T21:56:29.302349+0000","last_undegraded":"2023-12-26T21:56:29.302349+0000","last_fullsized":"2023-12-26T21:56:29.302349+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:30:42.736271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296985+0000","last_change":"2023-12-26T21:56:29.287378+0000","last_active":"2023-12-26T21:56:29.296985+0000","last_peered":"2023-12-26T21:56:29.296985+0000","last_clean":"2023-12-26T21:56:29.296985+0000","last_became_active":"2023-12-26T21:56:29.286765+0000","last_became_peered":"2023-12-26T21:56:29.286765+0000","last_unstale":"2023-12-26T21:56:29.296985+0000","last_undegraded":"2023-12-26T21:56:29.296985+0000","last_fullsized":"2023-12-26T21:56:29.296985+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:52.074115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.309928+0000","last_change":"2023-12-26T21:56:29.305090+0000","last_active":"2023-12-26T21:56:29.309928+0000","last_peered":"2023-12-26T21:56:29.309928+0000","last_clean":"2023-12-26T21:56:29.309928+0000","last_became_active":"2023-12-26T21:56:29.304787+0000","last_became_peered":"2023-12-26T21:56:29.304787+0000","last_unstale":"2023-12-26T21:56:29.309928+0000","last_undegraded":"2023-12-26T21:56:29.309928+0000","last_fullsized":"2023-12-26T21:56:29.309928+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:49:28.072941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.307945+0000","last_change":"2023-12-26T21:56:29.302132+0000","last_active":"2023-12-26T21:56:29.307945+0000","last_peered":"2023-12-26T21:56:29.307945+0000","last_clean":"2023-12-26T21:56:29.307945+0000","last_became_active":"2023-12-26T21:56:29.301816+0000","last_became_peered":"2023-12-26T21:56:29.301816+0000","last_unstale":"2023-12-26T21:56:29.307945+0000","last_undegraded":"2023-12-26T21:56:29.307945+0000","last_fullsized":"2023-12-26T21:56:29.307945+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:13:49.718420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296184+0000","last_change":"2023-12-26T21:56:29.292318+0000","last_active":"2023-12-26T21:56:29.296184+0000","last_peered":"2023-12-26T21:56:29.296184+0000","last_clean":"2023-12-26T21:56:29.296184+0000","last_became_active":"2023-12-26T21:56:29.292007+0000","last_became_peered":"2023-12-26T21:56:29.292007+0000","last_unstale":"2023-12-26T21:56:29.296184+0000","last_undegraded":"2023-12-26T21:56:29.296184+0000","last_fullsized":"2023-12-26T21:56:29.296184+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:07:27.210531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.291235+0000","last_change":"2023-12-26T21:56:29.290342+0000","last_active":"2023-12-26T21:56:29.291235+0000","last_peered":"2023-12-26T21:56:29.291235+0000","last_clean":"2023-12-26T21:56:29.291235+0000","last_became_active":"2023-12-26T21:56:29.290037+0000","last_became_peered":"2023-12-26T21:56:29.290037+0000","last_unstale":"2023-12-26T21:56:29.291235+0000","last_undegraded":"2023-12-26T21:56:29.291235+0000","last_fullsized":"2023-12-26T21:56:29.291235+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:58:23.621550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308463+0000","last_change":"2023-12-26T21:56:29.306806+0000","last_active":"2023-12-26T21:56:29.308463+0000","last_peered":"2023-12-26T21:56:29.308463+0000","last_clean":"2023-12-26T21:56:29.308463+0000","last_became_active":"2023-12-26T21:56:29.306521+0000","last_became_peered":"2023-12-26T21:56:29.306521+0000","last_unstale":"2023-12-26T21:56:29.308463+0000","last_undegraded":"2023-12-26T21:56:29.308463+0000","last_fullsized":"2023-12-26T21:56:29.308463+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:31:48.703988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.304459+0000","last_change":"2023-12-26T21:56:29.299269+0000","last_active":"2023-12-26T21:56:29.304459+0000","last_peered":"2023-12-26T21:56:29.304459+0000","last_clean":"2023-12-26T21:56:29.304459+0000","last_became_active":"2023-12-26T21:56:29.298977+0000","last_became_peered":"2023-12-26T21:56:29.298977+0000","last_unstale":"2023-12-26T21:56:29.304459+0000","last_undegraded":"2023-12-26T21:56:29.304459+0000","last_fullsized":"2023-12-26T21:56:29.304459+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:25:34.253454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.294192+0000","last_change":"2023-12-26T21:56:29.291917+0000","last_active":"2023-12-26T21:56:29.294192+0000","last_peered":"2023-12-26T21:56:29.294192+0000","last_clean":"2023-12-26T21:56:29.294192+0000","last_became_active":"2023-12-26T21:56:29.291637+0000","last_became_peered":"2023-12-26T21:56:29.291637+0000","last_unstale":"2023-12-26T21:56:29.294192+0000","last_undegraded":"2023-12-26T21:56:29.294192+0000","last_fullsized":"2023-12-26T21:56:29.294192+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:57:49.627306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.304901+0000","last_change":"2023-12-26T21:56:29.300686+0000","last_active":"2023-12-26T21:56:29.304901+0000","last_peered":"2023-12-26T21:56:29.304901+0000","last_clean":"2023-12-26T21:56:29.304901+0000","last_became_active":"2023-12-26T21:56:29.300378+0000","last_became_peered":"2023-12-26T21:56:29.300378+0000","last_unstale":"2023-12-26T21:56:29.304901+0000","last_undegraded":"2023-12-26T21:56:29.304901+0000","last_fullsized":"2023-12-26T21:56:29.304901+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:41:28.661110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.299584+0000","last_change":"2023-12-26T21:56:29.295451+0000","last_active":"2023-12-26T21:56:29.299584+0000","last_peered":"2023-12-26T21:56:29.299584+0000","last_clean":"2023-12-26T21:56:29.299584+0000","last_became_active":"2023-12-26T21:56:29.293088+0000","last_became_peered":"2023-12-26T21:56:29.293088+0000","last_unstale":"2023-12-26T21:56:29.299584+0000","last_undegraded":"2023-12-26T21:56:29.299584+0000","last_fullsized":"2023-12-26T21:56:29.299584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:38:28.059515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.298203+0000","last_change":"2023-12-26T21:56:29.295002+0000","last_active":"2023-12-26T21:56:29.298203+0000","last_peered":"2023-12-26T21:56:29.298203+0000","last_clean":"2023-12-26T21:56:29.298203+0000","last_became_active":"2023-12-26T21:56:29.291508+0000","last_became_peered":"2023-12-26T21:56:29.291508+0000","last_unstale":"2023-12-26T21:56:29.298203+0000","last_undegraded":"2023-12-26T21:56:29.298203+0000","last_fullsized":"2023-12-26T21:56:29.298203+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:15.376677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308474+0000","last_change":"2023-12-26T21:56:29.306657+0000","last_active":"2023-12-26T21:56:29.308474+0000","last_peered":"2023-12-26T21:56:29.308474+0000","last_clean":"2023-12-26T21:56:29.308474+0000","last_became_active":"2023-12-26T21:56:29.306390+0000","last_became_peered":"2023-12-26T21:56:29.306390+0000","last_unstale":"2023-12-26T21:56:29.308474+0000","last_undegraded":"2023-12-26T21:56:29.308474+0000","last_fullsized":"2023-12-26T21:56:29.308474+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:31:12.310074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.291780+0000","last_change":"2023-12-26T21:56:29.284827+0000","last_active":"2023-12-26T21:56:29.291780+0000","last_peered":"2023-12-26T21:56:29.291780+0000","last_clean":"2023-12-26T21:56:29.291780+0000","last_became_active":"2023-12-26T21:56:29.284439+0000","last_became_peered":"2023-12-26T21:56:29.284439+0000","last_unstale":"2023-12-26T21:56:29.291780+0000","last_undegraded":"2023-12-26T21:56:29.291780+0000","last_fullsized":"2023-12-26T21:56:29.291780+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:03.057577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303607+0000","last_change":"2023-12-26T21:56:29.297224+0000","last_active":"2023-12-26T21:56:29.303607+0000","last_peered":"2023-12-26T21:56:29.303607+0000","last_clean":"2023-12-26T21:56:29.303607+0000","last_became_active":"2023-12-26T21:56:29.296887+0000","last_became_peered":"2023-12-26T21:56:29.296887+0000","last_unstale":"2023-12-26T21:56:29.303607+0000","last_undegraded":"2023-12-26T21:56:29.303607+0000","last_fullsized":"2023-12-26T21:56:29.303607+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:27:49.727129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303513+0000","last_change":"2023-12-26T21:56:29.301589+0000","last_active":"2023-12-26T21:56:29.303513+0000","last_peered":"2023-12-26T21:56:29.303513+0000","last_clean":"2023-12-26T21:56:29.303513+0000","last_became_active":"2023-12-26T21:56:29.301317+0000","last_became_peered":"2023-12-26T21:56:29.301317+0000","last_unstale":"2023-12-26T21:56:29.303513+0000","last_undegraded":"2023-12-26T21:56:29.303513+0000","last_fullsized":"2023-12-26T21:56:29.303513+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:55:59.668636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308525+0000","last_change":"2023-12-26T21:56:29.304832+0000","last_active":"2023-12-26T21:56:29.308525+0000","last_peered":"2023-12-26T21:56:29.308525+0000","last_clean":"2023-12-26T21:56:29.308525+0000","last_became_active":"2023-12-26T21:56:29.304567+0000","last_became_peered":"2023-12-26T21:56:29.304567+0000","last_unstale":"2023-12-26T21:56:29.308525+0000","last_undegraded":"2023-12-26T21:56:29.308525+0000","last_fullsized":"2023-12-26T21:56:29.308525+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:32.394353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303545+0000","last_change":"2023-12-26T21:56:29.300938+0000","last_active":"2023-12-26T21:56:29.303545+0000","last_peered":"2023-12-26T21:56:29.303545+0000","last_clean":"2023-12-26T21:56:29.303545+0000","last_became_active":"2023-12-26T21:56:29.299464+0000","last_became_peered":"2023-12-26T21:56:29.299464+0000","last_unstale":"2023-12-26T21:56:29.303545+0000","last_undegraded":"2023-12-26T21:56:29.303545+0000","last_fullsized":"2023-12-26T21:56:29.303545+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:01:08.499642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.291188+0000","last_change":"2023-12-26T21:56:29.284598+0000","last_active":"2023-12-26T21:56:29.291188+0000","last_peered":"2023-12-26T21:56:29.291188+0000","last_clean":"2023-12-26T21:56:29.291188+0000","last_became_active":"2023-12-26T21:56:29.284308+0000","last_became_peered":"2023-12-26T21:56:29.284308+0000","last_unstale":"2023-12-26T21:56:29.291188+0000","last_undegraded":"2023-12-26T21:56:29.291188+0000","last_fullsized":"2023-12-26T21:56:29.291188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:53:44.515057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.298083+0000","last_change":"2023-12-26T21:56:29.293702+0000","last_active":"2023-12-26T21:56:29.298083+0000","last_peered":"2023-12-26T21:56:29.298083+0000","last_clean":"2023-12-26T21:56:29.298083+0000","last_became_active":"2023-12-26T21:56:29.293344+0000","last_became_peered":"2023-12-26T21:56:29.293344+0000","last_unstale":"2023-12-26T21:56:29.298083+0000","last_undegraded":"2023-12-26T21:56:29.298083+0000","last_fullsized":"2023-12-26T21:56:29.298083+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:50:33.880355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296146+0000","last_change":"2023-12-26T21:56:29.293633+0000","last_active":"2023-12-26T21:56:29.296146+0000","last_peered":"2023-12-26T21:56:29.296146+0000","last_clean":"2023-12-26T21:56:29.296146+0000","last_became_active":"2023-12-26T21:56:29.293346+0000","last_became_peered":"2023-12-26T21:56:29.293346+0000","last_unstale":"2023-12-26T21:56:29.296146+0000","last_undegraded":"2023-12-26T21:56:29.296146+0000","last_fullsized":"2023-12-26T21:56:29.296146+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:13:49.718420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.307648+0000","last_change":"2023-12-26T21:56:29.297091+0000","last_active":"2023-12-26T21:56:29.307648+0000","last_peered":"2023-12-26T21:56:29.307648+0000","last_clean":"2023-12-26T21:56:29.307648+0000","last_became_active":"2023-12-26T21:56:29.296771+0000","last_became_peered":"2023-12-26T21:56:29.296771+0000","last_unstale":"2023-12-26T21:56:29.307648+0000","last_undegraded":"2023-12-26T21:56:29.307648+0000","last_fullsized":"2023-12-26T21:56:29.307648+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:15:05.819700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.301821+0000","last_change":"2023-12-26T21:56:29.297791+0000","last_active":"2023-12-26T21:56:29.301821+0000","last_peered":"2023-12-26T21:56:29.301821+0000","last_clean":"2023-12-26T21:56:29.301821+0000","last_became_active":"2023-12-26T21:56:29.294050+0000","last_became_peered":"2023-12-26T21:56:29.294050+0000","last_unstale":"2023-12-26T21:56:29.301821+0000","last_undegraded":"2023-12-26T21:56:29.301821+0000","last_fullsized":"2023-12-26T21:56:29.301821+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:14.983804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.305868+0000","last_change":"2023-12-26T21:56:29.297762+0000","last_active":"2023-12-26T21:56:29.305868+0000","last_peered":"2023-12-26T21:56:29.305868+0000","last_clean":"2023-12-26T21:56:29.305868+0000","last_became_active":"2023-12-26T21:56:29.297506+0000","last_became_peered":"2023-12-26T21:56:29.297506+0000","last_unstale":"2023-12-26T21:56:29.305868+0000","last_undegraded":"2023-12-26T21:56:29.305868+0000","last_fullsized":"2023-12-26T21:56:29.305868+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:24.216953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.306777+0000","last_change":"2023-12-26T21:56:29.301104+0000","last_active":"2023-12-26T21:56:29.306777+0000","last_peered":"2023-12-26T21:56:29.306777+0000","last_clean":"2023-12-26T21:56:29.306777+0000","last_became_active":"2023-12-26T21:56:29.300816+0000","last_became_peered":"2023-12-26T21:56:29.300816+0000","last_unstale":"2023-12-26T21:56:29.306777+0000","last_undegraded":"2023-12-26T21:56:29.306777+0000","last_fullsized":"2023-12-26T21:56:29.306777+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:39:35.247764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303790+0000","last_change":"2023-12-26T21:56:29.297806+0000","last_active":"2023-12-26T21:56:29.303790+0000","last_peered":"2023-12-26T21:56:29.303790+0000","last_clean":"2023-12-26T21:56:29.303790+0000","last_became_active":"2023-12-26T21:56:29.297557+0000","last_became_peered":"2023-12-26T21:56:29.297557+0000","last_unstale":"2023-12-26T21:56:29.303790+0000","last_undegraded":"2023-12-26T21:56:29.303790+0000","last_fullsized":"2023-12-26T21:56:29.303790+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:59:51.699509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296178+0000","last_change":"2023-12-26T21:56:29.292069+0000","last_active":"2023-12-26T21:56:29.296178+0000","last_peered":"2023-12-26T21:56:29.296178+0000","last_clean":"2023-12-26T21:56:29.296178+0000","last_became_active":"2023-12-26T21:56:29.291819+0000","last_became_peered":"2023-12-26T21:56:29.291819+0000","last_unstale":"2023-12-26T21:56:29.296178+0000","last_undegraded":"2023-12-26T21:56:29.296178+0000","last_fullsized":"2023-12-26T21:56:29.296178+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:20:11.430124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303455+0000","last_change":"2023-12-26T21:56:29.299065+0000","last_active":"2023-12-26T21:56:29.303455+0000","last_peered":"2023-12-26T21:56:29.303455+0000","last_clean":"2023-12-26T21:56:29.303455+0000","last_became_active":"2023-12-26T21:56:29.298809+0000","last_became_peered":"2023-12-26T21:56:29.298809+0000","last_unstale":"2023-12-26T21:56:29.303455+0000","last_undegraded":"2023-12-26T21:56:29.303455+0000","last_fullsized":"2023-12-26T21:56:29.303455+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:33:40.337183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.306009+0000","last_change":"2023-12-26T21:56:29.301094+0000","last_active":"2023-12-26T21:56:29.306009+0000","last_peered":"2023-12-26T21:56:29.306009+0000","last_clean":"2023-12-26T21:56:29.306009+0000","last_became_active":"2023-12-26T21:56:29.300655+0000","last_became_peered":"2023-12-26T21:56:29.300655+0000","last_unstale":"2023-12-26T21:56:29.306009+0000","last_undegraded":"2023-12-26T21:56:29.306009+0000","last_fullsized":"2023-12-26T21:56:29.306009+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:18.076626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.304642+0000","last_change":"2023-12-26T21:56:29.301384+0000","last_active":"2023-12-26T21:56:29.304642+0000","last_peered":"2023-12-26T21:56:29.304642+0000","last_clean":"2023-12-26T21:56:29.304642+0000","last_became_active":"2023-12-26T21:56:29.298288+0000","last_became_peered":"2023-12-26T21:56:29.298288+0000","last_unstale":"2023-12-26T21:56:29.304642+0000","last_undegraded":"2023-12-26T21:56:29.304642+0000","last_fullsized":"2023-12-26T21:56:29.304642+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:50.244766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.306626+0000","last_change":"2023-12-26T21:56:29.303308+0000","last_active":"2023-12-26T21:56:29.306626+0000","last_peered":"2023-12-26T21:56:29.306626+0000","last_clean":"2023-12-26T21:56:29.306626+0000","last_became_active":"2023-12-26T21:56:29.301194+0000","last_became_peered":"2023-12-26T21:56:29.301194+0000","last_unstale":"2023-12-26T21:56:29.306626+0000","last_undegraded":"2023-12-26T21:56:29.306626+0000","last_fullsized":"2023-12-26T21:56:29.306626+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:05.988465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"138'4","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329343+0000","last_change":"2023-12-26T21:56:28.329200+0000","last_active":"2023-12-26T21:56:29.329343+0000","last_peered":"2023-12-26T21:56:29.329343+0000","last_clean":"2023-12-26T21:56:29.329343+0000","last_became_active":"2023-12-26T21:56:28.328941+0000","last_became_peered":"2023-12-26T21:56:28.328941+0000","last_unstale":"2023-12-26T21:56:29.329343+0000","last_undegraded":"2023-12-26T21:56:29.329343+0000","last_fullsized":"2023-12-26T21:56:29.329343+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:17.070266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.298995+0000","last_change":"2023-12-26T21:56:29.294745+0000","last_active":"2023-12-26T21:56:29.298995+0000","last_peered":"2023-12-26T21:56:29.298995+0000","last_clean":"2023-12-26T21:56:29.298995+0000","last_became_active":"2023-12-26T21:56:29.294467+0000","last_became_peered":"2023-12-26T21:56:29.294467+0000","last_unstale":"2023-12-26T21:56:29.298995+0000","last_undegraded":"2023-12-26T21:56:29.298995+0000","last_fullsized":"2023-12-26T21:56:29.298995+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:48.124399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.326673+0000","last_change":"2023-12-26T21:56:28.335859+0000","last_active":"2023-12-26T21:56:29.326673+0000","last_peered":"2023-12-26T21:56:29.326673+0000","last_clean":"2023-12-26T21:56:29.326673+0000","last_became_active":"2023-12-26T21:56:28.335471+0000","last_became_peered":"2023-12-26T21:56:28.335471+0000","last_unstale":"2023-12-26T21:56:29.326673+0000","last_undegraded":"2023-12-26T21:56:29.326673+0000","last_fullsized":"2023-12-26T21:56:29.326673+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:49:27.066581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303813+0000","last_change":"2023-12-26T21:56:29.299927+0000","last_active":"2023-12-26T21:56:29.303813+0000","last_peered":"2023-12-26T21:56:29.303813+0000","last_clean":"2023-12-26T21:56:29.303813+0000","last_became_active":"2023-12-26T21:56:29.299658+0000","last_became_peered":"2023-12-26T21:56:29.299658+0000","last_unstale":"2023-12-26T21:56:29.303813+0000","last_undegraded":"2023-12-26T21:56:29.303813+0000","last_fullsized":"2023-12-26T21:56:29.303813+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:55:11.959879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329398+0000","last_change":"2023-12-26T21:56:28.333793+0000","last_active":"2023-12-26T21:56:29.329398+0000","last_peered":"2023-12-26T21:56:29.329398+0000","last_clean":"2023-12-26T21:56:29.329398+0000","last_became_active":"2023-12-26T21:56:28.333519+0000","last_became_peered":"2023-12-26T21:56:28.333519+0000","last_unstale":"2023-12-26T21:56:29.329398+0000","last_undegraded":"2023-12-26T21:56:29.329398+0000","last_fullsized":"2023-12-26T21:56:29.329398+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:05.348656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308507+0000","last_change":"2023-12-26T21:56:29.304767+0000","last_active":"2023-12-26T21:56:29.308507+0000","last_peered":"2023-12-26T21:56:29.308507+0000","last_clean":"2023-12-26T21:56:29.308507+0000","last_became_active":"2023-12-26T21:56:29.304560+0000","last_became_peered":"2023-12-26T21:56:29.304560+0000","last_unstale":"2023-12-26T21:56:29.308507+0000","last_undegraded":"2023-12-26T21:56:29.308507+0000","last_fullsized":"2023-12-26T21:56:29.308507+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:28.678021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.331907+0000","last_change":"2023-12-26T21:56:28.333975+0000","last_active":"2023-12-26T21:56:29.331907+0000","last_peered":"2023-12-26T21:56:29.331907+0000","last_clean":"2023-12-26T21:56:29.331907+0000","last_became_active":"2023-12-26T21:56:28.333614+0000","last_became_peered":"2023-12-26T21:56:28.333614+0000","last_unstale":"2023-12-26T21:56:29.331907+0000","last_undegraded":"2023-12-26T21:56:29.331907+0000","last_fullsized":"2023-12-26T21:56:29.331907+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:49:27.066581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.305665+0000","last_change":"2023-12-26T21:56:29.297498+0000","last_active":"2023-12-26T21:56:29.305665+0000","last_peered":"2023-12-26T21:56:29.305665+0000","last_clean":"2023-12-26T21:56:29.305665+0000","last_became_active":"2023-12-26T21:56:29.296677+0000","last_became_peered":"2023-12-26T21:56:29.296677+0000","last_unstale":"2023-12-26T21:56:29.305665+0000","last_undegraded":"2023-12-26T21:56:29.305665+0000","last_fullsized":"2023-12-26T21:56:29.305665+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:39:00.944849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329114+0000","last_change":"2023-12-26T21:56:28.337058+0000","last_active":"2023-12-26T21:56:29.329114+0000","last_peered":"2023-12-26T21:56:29.329114+0000","last_clean":"2023-12-26T21:56:29.329114+0000","last_became_active":"2023-12-26T21:56:28.336682+0000","last_became_peered":"2023-12-26T21:56:28.336682+0000","last_unstale":"2023-12-26T21:56:29.329114+0000","last_undegraded":"2023-12-26T21:56:29.329114+0000","last_fullsized":"2023-12-26T21:56:29.329114+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:22:13.902192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.291208+0000","last_change":"2023-12-26T21:56:29.286935+0000","last_active":"2023-12-26T21:56:29.291208+0000","last_peered":"2023-12-26T21:56:29.291208+0000","last_clean":"2023-12-26T21:56:29.291208+0000","last_became_active":"2023-12-26T21:56:29.284293+0000","last_became_peered":"2023-12-26T21:56:29.284293+0000","last_unstale":"2023-12-26T21:56:29.291208+0000","last_undegraded":"2023-12-26T21:56:29.291208+0000","last_fullsized":"2023-12-26T21:56:29.291208+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:49:10.975962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.327801+0000","last_change":"2023-12-26T21:56:28.328243+0000","last_active":"2023-12-26T21:56:29.327801+0000","last_peered":"2023-12-26T21:56:29.327801+0000","last_clean":"2023-12-26T21:56:29.327801+0000","last_became_active":"2023-12-26T21:56:28.327298+0000","last_became_peered":"2023-12-26T21:56:28.327298+0000","last_unstale":"2023-12-26T21:56:29.327801+0000","last_undegraded":"2023-12-26T21:56:29.327801+0000","last_fullsized":"2023-12-26T21:56:29.327801+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:17.956980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.299533+0000","last_change":"2023-12-26T21:56:29.291896+0000","last_active":"2023-12-26T21:56:29.299533+0000","last_peered":"2023-12-26T21:56:29.299533+0000","last_clean":"2023-12-26T21:56:29.299533+0000","last_became_active":"2023-12-26T21:56:29.291520+0000","last_became_peered":"2023-12-26T21:56:29.291520+0000","last_unstale":"2023-12-26T21:56:29.299533+0000","last_undegraded":"2023-12-26T21:56:29.299533+0000","last_fullsized":"2023-12-26T21:56:29.299533+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:12.386199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.278711+0000","last_change":"2023-12-26T21:56:28.326837+0000","last_active":"2023-12-26T21:56:29.278711+0000","last_peered":"2023-12-26T21:56:29.278711+0000","last_clean":"2023-12-26T21:56:29.278711+0000","last_became_active":"2023-12-26T21:56:28.326583+0000","last_became_peered":"2023-12-26T21:56:28.326583+0000","last_unstale":"2023-12-26T21:56:29.278711+0000","last_undegraded":"2023-12-26T21:56:29.278711+0000","last_fullsized":"2023-12-26T21:56:29.278711+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:04:27.405594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.292398+0000","last_change":"2023-12-26T21:56:29.290560+0000","last_active":"2023-12-26T21:56:29.292398+0000","last_peered":"2023-12-26T21:56:29.292398+0000","last_clean":"2023-12-26T21:56:29.292398+0000","last_became_active":"2023-12-26T21:56:29.289803+0000","last_became_peered":"2023-12-26T21:56:29.289803+0000","last_unstale":"2023-12-26T21:56:29.292398+0000","last_undegraded":"2023-12-26T21:56:29.292398+0000","last_fullsized":"2023-12-26T21:56:29.292398+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:00:09.663521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.280515+0000","last_change":"2023-12-26T21:56:28.337306+0000","last_active":"2023-12-26T21:56:29.280515+0000","last_peered":"2023-12-26T21:56:29.280515+0000","last_clean":"2023-12-26T21:56:29.280515+0000","last_became_active":"2023-12-26T21:56:28.336945+0000","last_became_peered":"2023-12-26T21:56:28.336945+0000","last_unstale":"2023-12-26T21:56:29.280515+0000","last_undegraded":"2023-12-26T21:56:29.280515+0000","last_fullsized":"2023-12-26T21:56:29.280515+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:18.422751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.305990+0000","last_change":"2023-12-26T21:56:29.303119+0000","last_active":"2023-12-26T21:56:29.305990+0000","last_peered":"2023-12-26T21:56:29.305990+0000","last_clean":"2023-12-26T21:56:29.305990+0000","last_became_active":"2023-12-26T21:56:29.302040+0000","last_became_peered":"2023-12-26T21:56:29.302040+0000","last_unstale":"2023-12-26T21:56:29.305990+0000","last_undegraded":"2023-12-26T21:56:29.305990+0000","last_fullsized":"2023-12-26T21:56:29.305990+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:44:43.541452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.279701+0000","last_change":"2023-12-26T21:56:28.332621+0000","last_active":"2023-12-26T21:56:29.279701+0000","last_peered":"2023-12-26T21:56:29.279701+0000","last_clean":"2023-12-26T21:56:29.279701+0000","last_became_active":"2023-12-26T21:56:28.332294+0000","last_became_peered":"2023-12-26T21:56:28.332294+0000","last_unstale":"2023-12-26T21:56:29.279701+0000","last_undegraded":"2023-12-26T21:56:29.279701+0000","last_fullsized":"2023-12-26T21:56:29.279701+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:49.238406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.293318+0000","last_change":"2023-12-26T21:56:29.287202+0000","last_active":"2023-12-26T21:56:29.293318+0000","last_peered":"2023-12-26T21:56:29.293318+0000","last_clean":"2023-12-26T21:56:29.293318+0000","last_became_active":"2023-12-26T21:56:29.286859+0000","last_became_peered":"2023-12-26T21:56:29.286859+0000","last_unstale":"2023-12-26T21:56:29.293318+0000","last_undegraded":"2023-12-26T21:56:29.293318+0000","last_fullsized":"2023-12-26T21:56:29.293318+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:27:34.201449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.280218+0000","last_change":"2023-12-26T21:56:28.324714+0000","last_active":"2023-12-26T21:56:29.280218+0000","last_peered":"2023-12-26T21:56:29.280218+0000","last_clean":"2023-12-26T21:56:29.280218+0000","last_became_active":"2023-12-26T21:56:28.324394+0000","last_became_peered":"2023-12-26T21:56:28.324394+0000","last_unstale":"2023-12-26T21:56:29.280218+0000","last_undegraded":"2023-12-26T21:56:29.280218+0000","last_fullsized":"2023-12-26T21:56:29.280218+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:30:26.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.300029+0000","last_change":"2023-12-26T21:56:29.296619+0000","last_active":"2023-12-26T21:56:29.300029+0000","last_peered":"2023-12-26T21:56:29.300029+0000","last_clean":"2023-12-26T21:56:29.300029+0000","last_became_active":"2023-12-26T21:56:29.296355+0000","last_became_peered":"2023-12-26T21:56:29.296355+0000","last_unstale":"2023-12-26T21:56:29.300029+0000","last_undegraded":"2023-12-26T21:56:29.300029+0000","last_fullsized":"2023-12-26T21:56:29.300029+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:38:16.041172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"138'3","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.331541+0000","last_change":"2023-12-26T21:56:28.330566+0000","last_active":"2023-12-26T21:56:29.331541+0000","last_peered":"2023-12-26T21:56:29.331541+0000","last_clean":"2023-12-26T21:56:29.331541+0000","last_became_active":"2023-12-26T21:56:28.330300+0000","last_became_peered":"2023-12-26T21:56:28.330300+0000","last_unstale":"2023-12-26T21:56:29.331541+0000","last_undegraded":"2023-12-26T21:56:29.331541+0000","last_fullsized":"2023-12-26T21:56:29.331541+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:08:47.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303330+0000","last_change":"2023-12-26T21:56:29.298988+0000","last_active":"2023-12-26T21:56:29.303330+0000","last_peered":"2023-12-26T21:56:29.303330+0000","last_clean":"2023-12-26T21:56:29.303330+0000","last_became_active":"2023-12-26T21:56:29.298717+0000","last_became_peered":"2023-12-26T21:56:29.298717+0000","last_unstale":"2023-12-26T21:56:29.303330+0000","last_undegraded":"2023-12-26T21:56:29.303330+0000","last_fullsized":"2023-12-26T21:56:29.303330+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:16:49.690202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"138'6","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.333593+0000","last_change":"2023-12-26T21:56:28.333459+0000","last_active":"2023-12-26T21:56:29.333593+0000","last_peered":"2023-12-26T21:56:29.333593+0000","last_clean":"2023-12-26T21:56:29.333593+0000","last_became_active":"2023-12-26T21:56:28.332790+0000","last_became_peered":"2023-12-26T21:56:28.332790+0000","last_unstale":"2023-12-26T21:56:29.333593+0000","last_undegraded":"2023-12-26T21:56:29.333593+0000","last_fullsized":"2023-12-26T21:56:29.333593+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:55:49.457721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.289684+0000","last_change":"2023-12-26T21:56:29.284491+0000","last_active":"2023-12-26T21:56:29.289684+0000","last_peered":"2023-12-26T21:56:29.289684+0000","last_clean":"2023-12-26T21:56:29.289684+0000","last_became_active":"2023-12-26T21:56:29.284083+0000","last_became_peered":"2023-12-26T21:56:29.284083+0000","last_unstale":"2023-12-26T21:56:29.289684+0000","last_undegraded":"2023-12-26T21:56:29.289684+0000","last_fullsized":"2023-12-26T21:56:29.289684+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:06.355015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329577+0000","last_change":"2023-12-26T21:56:28.331301+0000","last_active":"2023-12-26T21:56:29.329577+0000","last_peered":"2023-12-26T21:56:29.329577+0000","last_clean":"2023-12-26T21:56:29.329577+0000","last_became_active":"2023-12-26T21:56:28.330897+0000","last_became_peered":"2023-12-26T21:56:28.330897+0000","last_unstale":"2023-12-26T21:56:29.329577+0000","last_undegraded":"2023-12-26T21:56:29.329577+0000","last_fullsized":"2023-12-26T21:56:29.329577+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:57.220980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.298107+0000","last_change":"2023-12-26T21:56:29.291431+0000","last_active":"2023-12-26T21:56:29.298107+0000","last_peered":"2023-12-26T21:56:29.298107+0000","last_clean":"2023-12-26T21:56:29.298107+0000","last_became_active":"2023-12-26T21:56:29.290993+0000","last_became_peered":"2023-12-26T21:56:29.290993+0000","last_unstale":"2023-12-26T21:56:29.298107+0000","last_undegraded":"2023-12-26T21:56:29.298107+0000","last_fullsized":"2023-12-26T21:56:29.298107+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:58:31.017354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.292810+0000","last_change":"2023-12-26T21:56:28.330437+0000","last_active":"2023-12-26T21:56:29.292810+0000","last_peered":"2023-12-26T21:56:29.292810+0000","last_clean":"2023-12-26T21:56:29.292810+0000","last_became_active":"2023-12-26T21:56:28.330195+0000","last_became_peered":"2023-12-26T21:56:28.330195+0000","last_unstale":"2023-12-26T21:56:29.292810+0000","last_undegraded":"2023-12-26T21:56:29.292810+0000","last_fullsized":"2023-12-26T21:56:29.292810+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:27:07.378809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.302703+0000","last_change":"2023-12-26T21:56:29.295052+0000","last_active":"2023-12-26T21:56:29.302703+0000","last_peered":"2023-12-26T21:56:29.302703+0000","last_clean":"2023-12-26T21:56:29.302703+0000","last_became_active":"2023-12-26T21:56:29.294712+0000","last_became_peered":"2023-12-26T21:56:29.294712+0000","last_unstale":"2023-12-26T21:56:29.302703+0000","last_undegraded":"2023-12-26T21:56:29.302703+0000","last_fullsized":"2023-12-26T21:56:29.302703+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:30:59.091981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"138'8","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.334337+0000","last_change":"2023-12-26T21:56:28.323723+0000","last_active":"2023-12-26T21:56:29.334337+0000","last_peered":"2023-12-26T21:56:29.334337+0000","last_clean":"2023-12-26T21:56:29.334337+0000","last_became_active":"2023-12-26T21:56:28.323368+0000","last_became_peered":"2023-12-26T21:56:28.323368+0000","last_unstale":"2023-12-26T21:56:29.334337+0000","last_undegraded":"2023-12-26T21:56:29.334337+0000","last_fullsized":"2023-12-26T21:56:29.334337+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:54.707404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303502+0000","last_change":"2023-12-26T21:56:29.299169+0000","last_active":"2023-12-26T21:56:29.303502+0000","last_peered":"2023-12-26T21:56:29.303502+0000","last_clean":"2023-12-26T21:56:29.303502+0000","last_became_active":"2023-12-26T21:56:29.298882+0000","last_became_peered":"2023-12-26T21:56:29.298882+0000","last_unstale":"2023-12-26T21:56:29.303502+0000","last_undegraded":"2023-12-26T21:56:29.303502+0000","last_fullsized":"2023-12-26T21:56:29.303502+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:00:52.782353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329108+0000","last_change":"2023-12-26T21:56:28.338162+0000","last_active":"2023-12-26T21:56:29.329108+0000","last_peered":"2023-12-26T21:56:29.329108+0000","last_clean":"2023-12-26T21:56:29.329108+0000","last_became_active":"2023-12-26T21:56:28.337763+0000","last_became_peered":"2023-12-26T21:56:28.337763+0000","last_unstale":"2023-12-26T21:56:29.329108+0000","last_undegraded":"2023-12-26T21:56:29.329108+0000","last_fullsized":"2023-12-26T21:56:29.329108+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:20:10.423765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.307847+0000","last_change":"2023-12-26T21:56:29.304794+0000","last_active":"2023-12-26T21:56:29.307847+0000","last_peered":"2023-12-26T21:56:29.307847+0000","last_clean":"2023-12-26T21:56:29.307847+0000","last_became_active":"2023-12-26T21:56:29.304492+0000","last_became_peered":"2023-12-26T21:56:29.304492+0000","last_unstale":"2023-12-26T21:56:29.307847+0000","last_undegraded":"2023-12-26T21:56:29.307847+0000","last_fullsized":"2023-12-26T21:56:29.307847+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:33:10.959030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.327566+0000","last_change":"2023-12-26T21:56:28.328508+0000","last_active":"2023-12-26T21:56:29.327566+0000","last_peered":"2023-12-26T21:56:29.327566+0000","last_clean":"2023-12-26T21:56:29.327566+0000","last_became_active":"2023-12-26T21:56:28.327745+0000","last_became_peered":"2023-12-26T21:56:28.327745+0000","last_unstale":"2023-12-26T21:56:29.327566+0000","last_undegraded":"2023-12-26T21:56:29.327566+0000","last_fullsized":"2023-12-26T21:56:29.327566+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:03.449268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303289+0000","last_change":"2023-12-26T21:56:29.298392+0000","last_active":"2023-12-26T21:56:29.303289+0000","last_peered":"2023-12-26T21:56:29.303289+0000","last_clean":"2023-12-26T21:56:29.303289+0000","last_became_active":"2023-12-26T21:56:29.298095+0000","last_became_peered":"2023-12-26T21:56:29.298095+0000","last_unstale":"2023-12-26T21:56:29.303289+0000","last_undegraded":"2023-12-26T21:56:29.303289+0000","last_fullsized":"2023-12-26T21:56:29.303289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:27:10.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329145+0000","last_change":"2023-12-26T21:56:28.336955+0000","last_active":"2023-12-26T21:56:29.329145+0000","last_peered":"2023-12-26T21:56:29.329145+0000","last_clean":"2023-12-26T21:56:29.329145+0000","last_became_active":"2023-12-26T21:56:28.336641+0000","last_became_peered":"2023-12-26T21:56:28.336641+0000","last_unstale":"2023-12-26T21:56:29.329145+0000","last_undegraded":"2023-12-26T21:56:29.329145+0000","last_fullsized":"2023-12-26T21:56:29.329145+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:31:11.303715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303718+0000","last_change":"2023-12-26T21:56:29.300764+0000","last_active":"2023-12-26T21:56:29.303718+0000","last_peered":"2023-12-26T21:56:29.303718+0000","last_clean":"2023-12-26T21:56:29.303718+0000","last_became_active":"2023-12-26T21:56:29.297808+0000","last_became_peered":"2023-12-26T21:56:29.297808+0000","last_unstale":"2023-12-26T21:56:29.303718+0000","last_undegraded":"2023-12-26T21:56:29.303718+0000","last_fullsized":"2023-12-26T21:56:29.303718+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:26:19.047598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.283204+0000","last_change":"2023-12-26T21:56:28.327827+0000","last_active":"2023-12-26T21:56:29.283204+0000","last_peered":"2023-12-26T21:56:29.283204+0000","last_clean":"2023-12-26T21:56:29.283204+0000","last_became_active":"2023-12-26T21:56:28.327529+0000","last_became_peered":"2023-12-26T21:56:28.327529+0000","last_unstale":"2023-12-26T21:56:29.283204+0000","last_undegraded":"2023-12-26T21:56:29.283204+0000","last_fullsized":"2023-12-26T21:56:29.283204+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:04.982105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296746+0000","last_change":"2023-12-26T21:56:29.295378+0000","last_active":"2023-12-26T21:56:29.296746+0000","last_peered":"2023-12-26T21:56:29.296746+0000","last_clean":"2023-12-26T21:56:29.296746+0000","last_became_active":"2023-12-26T21:56:29.295013+0000","last_became_peered":"2023-12-26T21:56:29.295013+0000","last_unstale":"2023-12-26T21:56:29.296746+0000","last_undegraded":"2023-12-26T21:56:29.296746+0000","last_fullsized":"2023-12-26T21:56:29.296746+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:40:30.129228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329465+0000","last_change":"2023-12-26T21:56:28.326256+0000","last_active":"2023-12-26T21:56:29.329465+0000","last_peered":"2023-12-26T21:56:29.329465+0000","last_clean":"2023-12-26T21:56:29.329465+0000","last_became_active":"2023-12-26T21:56:28.325963+0000","last_became_peered":"2023-12-26T21:56:28.325963+0000","last_unstale":"2023-12-26T21:56:29.329465+0000","last_undegraded":"2023-12-26T21:56:29.329465+0000","last_fullsized":"2023-12-26T21:56:29.329465+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:42:28.838829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.300437+0000","last_change":"2023-12-26T21:56:29.296764+0000","last_active":"2023-12-26T21:56:29.300437+0000","last_peered":"2023-12-26T21:56:29.300437+0000","last_clean":"2023-12-26T21:56:29.300437+0000","last_became_active":"2023-12-26T21:56:29.296461+0000","last_became_peered":"2023-12-26T21:56:29.296461+0000","last_unstale":"2023-12-26T21:56:29.300437+0000","last_undegraded":"2023-12-26T21:56:29.300437+0000","last_fullsized":"2023-12-26T21:56:29.300437+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:25:48.978620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.293176+0000","last_change":"2023-12-26T21:56:28.331745+0000","last_active":"2023-12-26T21:56:29.293176+0000","last_peered":"2023-12-26T21:56:29.293176+0000","last_clean":"2023-12-26T21:56:29.293176+0000","last_became_active":"2023-12-26T21:56:28.331435+0000","last_became_peered":"2023-12-26T21:56:28.331435+0000","last_unstale":"2023-12-26T21:56:29.293176+0000","last_undegraded":"2023-12-26T21:56:29.293176+0000","last_fullsized":"2023-12-26T21:56:29.293176+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:46:15.614750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.297762+0000","last_change":"2023-12-26T21:56:29.295685+0000","last_active":"2023-12-26T21:56:29.297762+0000","last_peered":"2023-12-26T21:56:29.297762+0000","last_clean":"2023-12-26T21:56:29.297762+0000","last_became_active":"2023-12-26T21:56:29.295434+0000","last_became_peered":"2023-12-26T21:56:29.295434+0000","last_unstale":"2023-12-26T21:56:29.297762+0000","last_undegraded":"2023-12-26T21:56:29.297762+0000","last_fullsized":"2023-12-26T21:56:29.297762+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:44:46.461518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.280736+0000","last_change":"2023-12-26T21:56:28.326271+0000","last_active":"2023-12-26T21:56:29.280736+0000","last_peered":"2023-12-26T21:56:29.280736+0000","last_clean":"2023-12-26T21:56:29.280736+0000","last_became_active":"2023-12-26T21:56:28.325985+0000","last_became_peered":"2023-12-26T21:56:28.325985+0000","last_unstale":"2023-12-26T21:56:29.280736+0000","last_undegraded":"2023-12-26T21:56:29.280736+0000","last_fullsized":"2023-12-26T21:56:29.280736+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:22.509771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303870+0000","last_change":"2023-12-26T21:56:29.293095+0000","last_active":"2023-12-26T21:56:29.303870+0000","last_peered":"2023-12-26T21:56:29.303870+0000","last_clean":"2023-12-26T21:56:29.303870+0000","last_became_active":"2023-12-26T21:56:29.292238+0000","last_became_peered":"2023-12-26T21:56:29.292238+0000","last_unstale":"2023-12-26T21:56:29.303870+0000","last_undegraded":"2023-12-26T21:56:29.303870+0000","last_fullsized":"2023-12-26T21:56:29.303870+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:23.516130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.281117+0000","last_change":"2023-12-26T21:56:28.329073+0000","last_active":"2023-12-26T21:56:29.281117+0000","last_peered":"2023-12-26T21:56:29.281117+0000","last_clean":"2023-12-26T21:56:29.281117+0000","last_became_active":"2023-12-26T21:56:28.328734+0000","last_became_peered":"2023-12-26T21:56:28.328734+0000","last_unstale":"2023-12-26T21:56:29.281117+0000","last_undegraded":"2023-12-26T21:56:29.281117+0000","last_fullsized":"2023-12-26T21:56:29.281117+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:02.051217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303375+0000","last_change":"2023-12-26T21:56:29.300660+0000","last_active":"2023-12-26T21:56:29.303375+0000","last_peered":"2023-12-26T21:56:29.303375+0000","last_clean":"2023-12-26T21:56:29.303375+0000","last_became_active":"2023-12-26T21:56:29.299380+0000","last_became_peered":"2023-12-26T21:56:29.299380+0000","last_unstale":"2023-12-26T21:56:29.303375+0000","last_undegraded":"2023-12-26T21:56:29.303375+0000","last_fullsized":"2023-12-26T21:56:29.303375+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:29.719927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.332090+0000","last_change":"2023-12-26T21:56:28.327417+0000","last_active":"2023-12-26T21:56:29.332090+0000","last_peered":"2023-12-26T21:56:29.332090+0000","last_clean":"2023-12-26T21:56:29.332090+0000","last_became_active":"2023-12-26T21:56:28.327124+0000","last_became_peered":"2023-12-26T21:56:28.327124+0000","last_unstale":"2023-12-26T21:56:29.332090+0000","last_undegraded":"2023-12-26T21:56:29.332090+0000","last_fullsized":"2023-12-26T21:56:29.332090+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:20:29.860954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.299715+0000","last_change":"2023-12-26T21:56:29.295969+0000","last_active":"2023-12-26T21:56:29.299715+0000","last_peered":"2023-12-26T21:56:29.299715+0000","last_clean":"2023-12-26T21:56:29.299715+0000","last_became_active":"2023-12-26T21:56:29.295700+0000","last_became_peered":"2023-12-26T21:56:29.295700+0000","last_unstale":"2023-12-26T21:56:29.299715+0000","last_undegraded":"2023-12-26T21:56:29.299715+0000","last_fullsized":"2023-12-26T21:56:29.299715+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.043412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.279332+0000","last_change":"2023-12-26T21:56:28.338142+0000","last_active":"2023-12-26T21:56:29.279332+0000","last_peered":"2023-12-26T21:56:29.279332+0000","last_clean":"2023-12-26T21:56:29.279332+0000","last_became_active":"2023-12-26T21:56:28.337880+0000","last_became_peered":"2023-12-26T21:56:28.337880+0000","last_unstale":"2023-12-26T21:56:29.279332+0000","last_undegraded":"2023-12-26T21:56:29.279332+0000","last_fullsized":"2023-12-26T21:56:29.279332+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:24:37.620686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.309940+0000","last_change":"2023-12-26T21:56:29.307621+0000","last_active":"2023-12-26T21:56:29.309940+0000","last_peered":"2023-12-26T21:56:29.309940+0000","last_clean":"2023-12-26T21:56:29.309940+0000","last_became_active":"2023-12-26T21:56:29.307276+0000","last_became_peered":"2023-12-26T21:56:29.307276+0000","last_unstale":"2023-12-26T21:56:29.309940+0000","last_undegraded":"2023-12-26T21:56:29.309940+0000","last_fullsized":"2023-12-26T21:56:29.309940+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:56:47.190533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.279888+0000","last_change":"2023-12-26T21:56:28.329399+0000","last_active":"2023-12-26T21:56:29.279888+0000","last_peered":"2023-12-26T21:56:29.279888+0000","last_clean":"2023-12-26T21:56:29.279888+0000","last_became_active":"2023-12-26T21:56:28.329153+0000","last_became_peered":"2023-12-26T21:56:28.329153+0000","last_unstale":"2023-12-26T21:56:29.279888+0000","last_undegraded":"2023-12-26T21:56:29.279888+0000","last_fullsized":"2023-12-26T21:56:29.279888+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:33:42.903245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303699+0000","last_change":"2023-12-26T21:56:29.300461+0000","last_active":"2023-12-26T21:56:29.303699+0000","last_peered":"2023-12-26T21:56:29.303699+0000","last_clean":"2023-12-26T21:56:29.303699+0000","last_became_active":"2023-12-26T21:56:29.297571+0000","last_became_peered":"2023-12-26T21:56:29.297571+0000","last_unstale":"2023-12-26T21:56:29.303699+0000","last_undegraded":"2023-12-26T21:56:29.303699+0000","last_fullsized":"2023-12-26T21:56:29.303699+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:15:53.820020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.280516+0000","last_change":"2023-12-26T21:56:28.339259+0000","last_active":"2023-12-26T21:56:29.280516+0000","last_peered":"2023-12-26T21:56:29.280516+0000","last_clean":"2023-12-26T21:56:29.280516+0000","last_became_active":"2023-12-26T21:56:28.338970+0000","last_became_peered":"2023-12-26T21:56:28.338970+0000","last_unstale":"2023-12-26T21:56:29.280516+0000","last_undegraded":"2023-12-26T21:56:29.280516+0000","last_fullsized":"2023-12-26T21:56:29.280516+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:11.379840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.300500+0000","last_change":"2023-12-26T21:56:29.297105+0000","last_active":"2023-12-26T21:56:29.300500+0000","last_peered":"2023-12-26T21:56:29.300500+0000","last_clean":"2023-12-26T21:56:29.300500+0000","last_became_active":"2023-12-26T21:56:29.296755+0000","last_became_peered":"2023-12-26T21:56:29.296755+0000","last_unstale":"2023-12-26T21:56:29.300500+0000","last_undegraded":"2023-12-26T21:56:29.300500+0000","last_fullsized":"2023-12-26T21:56:29.300500+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:26:11.061541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.278931+0000","last_change":"2023-12-26T21:56:28.331641+0000","last_active":"2023-12-26T21:56:29.278931+0000","last_peered":"2023-12-26T21:56:29.278931+0000","last_clean":"2023-12-26T21:56:29.278931+0000","last_became_active":"2023-12-26T21:56:28.331163+0000","last_became_peered":"2023-12-26T21:56:28.331163+0000","last_unstale":"2023-12-26T21:56:29.278931+0000","last_undegraded":"2023-12-26T21:56:29.278931+0000","last_fullsized":"2023-12-26T21:56:29.278931+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:33:40.215631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.305962+0000","last_change":"2023-12-26T21:56:29.302255+0000","last_active":"2023-12-26T21:56:29.305962+0000","last_peered":"2023-12-26T21:56:29.305962+0000","last_clean":"2023-12-26T21:56:29.305962+0000","last_became_active":"2023-12-26T21:56:29.299701+0000","last_became_peered":"2023-12-26T21:56:29.299701+0000","last_unstale":"2023-12-26T21:56:29.305962+0000","last_undegraded":"2023-12-26T21:56:29.305962+0000","last_fullsized":"2023-12-26T21:56:29.305962+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:09:59.751369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.283315+0000","last_change":"2023-12-26T21:56:28.332448+0000","last_active":"2023-12-26T21:56:29.283315+0000","last_peered":"2023-12-26T21:56:29.283315+0000","last_clean":"2023-12-26T21:56:29.283315+0000","last_became_active":"2023-12-26T21:56:28.331754+0000","last_became_peered":"2023-12-26T21:56:28.331754+0000","last_unstale":"2023-12-26T21:56:29.283315+0000","last_undegraded":"2023-12-26T21:56:29.283315+0000","last_fullsized":"2023-12-26T21:56:29.283315+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:56:46.184174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.297744+0000","last_change":"2023-12-26T21:56:29.294983+0000","last_active":"2023-12-26T21:56:29.297744+0000","last_peered":"2023-12-26T21:56:29.297744+0000","last_clean":"2023-12-26T21:56:29.297744+0000","last_became_active":"2023-12-26T21:56:29.294639+0000","last_became_peered":"2023-12-26T21:56:29.294639+0000","last_unstale":"2023-12-26T21:56:29.297744+0000","last_undegraded":"2023-12-26T21:56:29.297744+0000","last_fullsized":"2023-12-26T21:56:29.297744+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:48.115508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.294324+0000","last_change":"2023-12-26T21:56:28.323720+0000","last_active":"2023-12-26T21:56:29.294324+0000","last_peered":"2023-12-26T21:56:29.294324+0000","last_clean":"2023-12-26T21:56:29.294324+0000","last_became_active":"2023-12-26T21:56:28.323148+0000","last_became_peered":"2023-12-26T21:56:28.323148+0000","last_unstale":"2023-12-26T21:56:29.294324+0000","last_undegraded":"2023-12-26T21:56:29.294324+0000","last_fullsized":"2023-12-26T21:56:29.294324+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:22.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308242+0000","last_change":"2023-12-26T21:56:29.301983+0000","last_active":"2023-12-26T21:56:29.308242+0000","last_peered":"2023-12-26T21:56:29.308242+0000","last_clean":"2023-12-26T21:56:29.308242+0000","last_became_active":"2023-12-26T21:56:29.301689+0000","last_became_peered":"2023-12-26T21:56:29.301689+0000","last_unstale":"2023-12-26T21:56:29.308242+0000","last_undegraded":"2023-12-26T21:56:29.308242+0000","last_fullsized":"2023-12-26T21:56:29.308242+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:47:05.820173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.285032+0000","last_change":"2023-12-26T21:56:28.338737+0000","last_active":"2023-12-26T21:56:29.285032+0000","last_peered":"2023-12-26T21:56:29.285032+0000","last_clean":"2023-12-26T21:56:29.285032+0000","last_became_active":"2023-12-26T21:56:28.338068+0000","last_became_peered":"2023-12-26T21:56:28.338068+0000","last_unstale":"2023-12-26T21:56:29.285032+0000","last_undegraded":"2023-12-26T21:56:29.285032+0000","last_fullsized":"2023-12-26T21:56:29.285032+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:59:23.706823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.297188+0000","last_change":"2023-12-26T21:56:29.287118+0000","last_active":"2023-12-26T21:56:29.297188+0000","last_peered":"2023-12-26T21:56:29.297188+0000","last_clean":"2023-12-26T21:56:29.297188+0000","last_became_active":"2023-12-26T21:56:29.286800+0000","last_became_peered":"2023-12-26T21:56:29.286800+0000","last_unstale":"2023-12-26T21:56:29.297188+0000","last_undegraded":"2023-12-26T21:56:29.297188+0000","last_fullsized":"2023-12-26T21:56:29.297188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:26.376471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.283891+0000","last_change":"2023-12-26T21:56:28.333716+0000","last_active":"2023-12-26T21:56:29.283891+0000","last_peered":"2023-12-26T21:56:29.283891+0000","last_clean":"2023-12-26T21:56:29.283891+0000","last_became_active":"2023-12-26T21:56:28.333433+0000","last_became_peered":"2023-12-26T21:56:28.333433+0000","last_unstale":"2023-12-26T21:56:29.283891+0000","last_undegraded":"2023-12-26T21:56:29.283891+0000","last_fullsized":"2023-12-26T21:56:29.283891+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:49.238406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"128'249","reported_seq":737,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.288817+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:56:29.288817+0000","last_peered":"2023-12-26T21:56:29.288817+0000","last_clean":"2023-12-26T21:56:29.288817+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:56:29.288817+0000","last_undegraded":"2023-12-26T21:56:29.288817+0000","last_fullsized":"2023-12-26T21:56:29.288817+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303080,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068760,"statfs":{"total":96636764160,"available":96326410240,"internally_reserved":0,"allocated":1736704,"data_stored":845769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66000000000000003,"15min":0.69499999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.66500000000000004},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39300000000000002},"max":{"1min":0.89100000000000001,"5min":0.94099999999999995,"15min":0.98299999999999998},"last":0.59099999999999997}]},{"osd":1,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64200000000000002,"15min":0.67200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.85899999999999999,"15min":1},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67600000000000005,"15min":0.70399999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.34200000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71699999999999997}]},{"osd":2,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.71499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.434},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.629},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66200000000000003,"15min":0.68400000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.434},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.55800000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.82799999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91400000000000003,"5min":1.024,"15min":1.0569999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.311,"5min":0.309,"15min":0.309},"max":{"1min":0.81599999999999995,"5min":1.036,"15min":1.036},"last":0.77900000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.82499999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94099999999999995,"5min":1.117,"15min":1.117},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.48899999999999999,"5min":0.45800000000000002,"15min":0.44},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.749}]}]},{"osd":6,"up_from":15,"seq":64424509673,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307156,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064684,"statfs":{"total":96636764160,"available":96322236416,"internally_reserved":0,"allocated":1716224,"data_stored":844109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66400000000000003,"15min":0.70099999999999996},"min":{"1min":0.49399999999999999,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.82199999999999995,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68899999999999995,"15min":0.70899999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65100000000000002,"15min":0.68799999999999994},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.879,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.40200000000000002},"max":{"1min":0.83499999999999996,"5min":0.95899999999999996,"15min":0.97599999999999998},"last":0.65700000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.70299999999999996},"min":{"1min":0.48299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81899999999999995,"5min":0.93200000000000005,"15min":0.97099999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.52900000000000003,"5min":0.48399999999999999,"15min":0.46400000000000002},"max":{"1min":0.85099999999999998,"5min":0.93999999999999995,"15min":0.94699999999999995},"last":0.626}]},{"osd":3,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69199999999999995,"15min":0.71999999999999997},"min":{"1min":0.51900000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85399999999999998,"5min":0.92000000000000004,"15min":1.026},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71999999999999997,"15min":0.747},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.42799999999999999},"max":{"1min":0.83799999999999997,"5min":0.96099999999999997,"15min":1.0129999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.84299999999999997,"5min":0.91200000000000003,"15min":1.0529999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.97299999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":1.2609999999999999},"last":0.77500000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72199999999999998,"15min":0.70799999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.438},"max":{"1min":0.89800000000000002,"5min":1.008,"15min":1.0920000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.97399999999999998,"15min":1.0920000000000001},"last":0.753}]}]},{"osd":1,"up_from":15,"seq":64424509673,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311832,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060008,"statfs":{"total":96636764160,"available":96317448192,"internally_reserved":0,"allocated":2310144,"data_stored":1434477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46100000000000002,"15min":0.45200000000000001},"min":{"1min":0.29599999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.872,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.5},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.58399999999999996,"5min":1.353,"15min":1.353},"last":0.38}]},{"osd":2,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.47999999999999998},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78000000000000003,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.40300000000000002,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.93400000000000005,"5min":1.488,"15min":1.488},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56599999999999995,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35799999999999998},"max":{"1min":0.97299999999999998,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.73799999999999999,"5min":1.506,"15min":1.506},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70199999999999996,"15min":0.73899999999999999},"min":{"1min":0.51600000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88300000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.76400000000000001,"15min":0.752},"min":{"1min":0.52400000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1319999999999999,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.57699999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.54000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.1020000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75800000000000001,"15min":0.79800000000000004},"min":{"1min":0.58499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.119,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.746}]},{"osd":6,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76300000000000001,"15min":0.79800000000000004},"min":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.48399999999999999},"max":{"1min":1.012,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78200000000000003,"15min":0.80300000000000005},"min":{"1min":0.59099999999999997,"5min":0.56499999999999995,"15min":0.54400000000000004},"max":{"1min":1.1000000000000001,"5min":1.615,"15min":1.615},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.753,"15min":0.77400000000000002},"min":{"1min":0.53600000000000003,"5min":0.48999999999999999,"15min":0.46400000000000002},"max":{"1min":1.0189999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.54900000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91700000000000004,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.78600000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509673,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303068,"kb_used_data":1684,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068772,"statfs":{"total":96636764160,"available":96326422528,"internally_reserved":0,"allocated":1724416,"data_stored":844733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.245},"max":{"1min":0.63500000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44900000000000001,"15min":0.47499999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":0.69699999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45800000000000002,"15min":0.48499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.73099999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.497,"15min":0.51500000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52300000000000002,"15min":0.53900000000000003},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66800000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58599999999999997,"15min":0.60399999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71999999999999997,"5min":0.86499999999999999,"15min":0.86699999999999999},"last":0.47999999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70899999999999996,"15min":0.74099999999999999},"min":{"1min":0.51800000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93400000000000005,"5min":1.226,"15min":1.2749999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72799999999999998,"15min":0.75900000000000001},"min":{"1min":0.53800000000000003,"5min":0.47899999999999998,"15min":0.47699999999999998},"max":{"1min":0.90100000000000002,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76700000000000002,"15min":0.79200000000000004},"min":{"1min":0.52400000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.002,"5min":1.1339999999999999,"15min":1.153},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.53400000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0489999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.77200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.96299999999999997,"5min":1.093,"15min":1.214},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.78200000000000003},"min":{"1min":0.52700000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.95299999999999996,"5min":1.169,"15min":1.169},"last":0.627}]},{"osd":7,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.79200000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.036,"5min":1.252,"15min":1.252},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.749,"15min":0.78500000000000003},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.016,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.55800000000000005}]}]},{"osd":2,"up_from":14,"seq":60129542378,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307168,"kb_used_data":1688,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064672,"statfs":{"total":96636764160,"available":96322224128,"internally_reserved":0,"allocated":1728512,"data_stored":844719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44800000000000001,"15min":0.45100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.24299999999999999},"max":{"1min":0.58399999999999996,"5min":0.70399999999999996,"15min":0.85999999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.50900000000000001,"15min":0.52200000000000002},"min":{"1min":0.36499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.83899999999999997},"last":0.442}]},{"osd":1,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.35199999999999998,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.74199999999999999,"5min":1.056,"15min":1.056},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58999999999999997,"15min":0.59999999999999998},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83599999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83399999999999996,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.371}]},{"osd":4,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.751,"15min":0.76200000000000001},"min":{"1min":0.55100000000000005,"5min":0.54600000000000004,"15min":0.38300000000000001},"max":{"1min":0.94499999999999995,"5min":1.3,"15min":1.3},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75,"15min":0.76400000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95799999999999996,"5min":1.048,"15min":1.1759999999999999},"last":0.64600000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.75,"15min":0.76400000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.502},"max":{"1min":0.89500000000000002,"5min":1.25,"15min":1.25},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":0.98799999999999999,"5min":1.002,"15min":1.1319999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.50700000000000001},"max":{"1min":0.93000000000000005,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.61199999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.0009999999999999,"5min":1.202,"15min":1.272},"last":0.67000000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76200000000000001,"15min":0.77500000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":1.1060000000000001,"15min":1.1220000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79400000000000004,"15min":0.81299999999999994},"min":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.54300000000000004},"max":{"1min":1.0329999999999999,"5min":1.232,"15min":1.232},"last":0.747}]}]},{"osd":3,"up_from":15,"seq":64424509674,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311252,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060588,"statfs":{"total":96636764160,"available":96318042112,"internally_reserved":0,"allocated":1716224,"data_stored":844109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.246,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.315},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.52200000000000002,"15min":0.495},"min":{"1min":0.32000000000000001,"5min":0.316,"15min":0.30299999999999999},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.33400000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.89400000000000002,"5min":1.468,"15min":1.468},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55600000000000005,"15min":0.53500000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.3940000000000001,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.377},"max":{"1min":1.4419999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.42199999999999999},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.48299999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.754,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.437,"15min":0.437},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":5.7229999999999999,"5min":5.7229999999999999,"15min":5.7229999999999999},"last":0.79200000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.80600000000000005,"15min":0.78500000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51500000000000001},"max":{"1min":5.6799999999999997,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57999999999999996,"5min":0.52600000000000002,"15min":0.51700000000000002},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.81000000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.81100000000000005,"15min":0.80400000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":3.8250000000000002,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.55700000000000005},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.82999999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.3420000000000001,"5min":3.3420000000000001,"15min":3.3420000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.81699999999999995,"15min":0.80800000000000005},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":0.75600000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542377,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307172,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064668,"statfs":{"total":96636764160,"available":96322220032,"internally_reserved":0,"allocated":1732608,"data_stored":845735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.44800000000000001,"5min":0.439,"15min":0.307},"max":{"1min":0.89800000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68700000000000006,"15min":0.70199999999999996},"min":{"1min":0.44,"5min":0.436,"15min":0.371},"max":{"1min":0.91400000000000003,"5min":1.006,"15min":1.008},"last":0.61099999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.311},"max":{"1min":0.96499999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.40000000000000002},"max":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82199999999999995}]},{"osd":2,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.51200000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.878},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68300000000000005,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.38800000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63300000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70799999999999996,"15min":0.71199999999999997},"min":{"1min":0.57399999999999995,"5min":0.496,"15min":0.45300000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.97599999999999998},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.438},"max":{"1min":0.91400000000000003,"5min":1.046,"15min":1.046},"last":0.78000000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.432,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84999999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64200000000000002,"15min":0.61499999999999999},"min":{"1min":0.432,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85299999999999998,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.92200000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.67300000000000004},"min":{"1min":0.50900000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.91300000000000003,"5min":1.133,"15min":1.133},"last":0.93700000000000006}]},{"osd":7,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.002},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.61399999999999999,"5min":0.54300000000000004,"15min":0.48499999999999999},"max":{"1min":0.878,"5min":1.266,"15min":1.266},"last":0.96199999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509672,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303636,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068204,"statfs":{"total":96636764160,"available":96325840896,"internally_reserved":0,"allocated":2306048,"data_stored":1427861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.68100000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.27300000000000002},"max":{"1min":0.83599999999999997,"5min":0.89800000000000002,"15min":0.995},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.32400000000000001},"max":{"1min":0.89900000000000002,"5min":0.91600000000000004,"15min":1.0369999999999999},"last":0.90700000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.439,"15min":0.34000000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.04},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66500000000000004,"15min":0.69599999999999995},"min":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.36899999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.83499999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68200000000000005,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.3029999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.68100000000000005,"15min":0.69999999999999996},"min":{"1min":0.46999999999999997,"5min":0.45800000000000002,"15min":0.38800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.72599999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.54800000000000004,"5min":0.504,"15min":0.39200000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.0640000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69799999999999995,"15min":0.71999999999999997},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.47799999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93899999999999995},"last":0.85199999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.48899999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.92200000000000004,"5min":1.01,"15min":1.01},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.40600000000000003,"5min":0.35699999999999998,"15min":0.35499999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.496,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.68300000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.012,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:56:33.690 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T21:56:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.915+0000 7efd0ddff700 1 Processor -- start 2023-12-26T21:56:33.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.915+0000 7efd0ddff700 1 -- start start 2023-12-26T21:56:33.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.916+0000 7efd0ddff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efd080cc810 0x7efd080c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.916+0000 7efd0ddff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd080686c0 con 0x7efd0806bfd0 2023-12-26T21:56:33.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.917+0000 7efd0ddff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd08069520 con 0x7efd0813c510 2023-12-26T21:56:33.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.917+0000 7efd0ddff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd08068910 con 0x7efd080cc810 2023-12-26T21:56:33.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.917+0000 7efd07fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efd080cc810 0x7efd080c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.917+0000 7efd06ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efd0806bfd0 0x7efd080cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48950/0 (socket says 172.21.15.131:48950) 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.917+0000 7efd06ffd700 1 -- 172.21.15.131:0/1712303253 learned_addr learned my addr 172.21.15.131:0/1712303253 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.917+0000 7efd07fff700 1 -- 172.21.15.131:0/1712303253 >> v1:172.21.15.190:6789/0 conn(0x7efd0813c510 legacy=0x7efd0806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.917+0000 7efd07fff700 1 -- 172.21.15.131:0/1712303253 >> v1:172.21.15.131:6789/0 conn(0x7efd0806bfd0 legacy=0x7efd080cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.917+0000 7efd07fff700 1 -- 172.21.15.131:0/1712303253 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd08068bf0 con 0x7efd080cc810 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd07fff700 1 --2- 172.21.15.131:0/1712303253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 0x7efd080c8a20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7efd00009e60 tx=0x7efd00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91acf3c39cbde1fb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd067fc700 1 -- 172.21.15.131:0/1712303253 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0000e070 con 0x7efd080cc810 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd067fc700 1 -- 172.21.15.131:0/1712303253 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0000bc30 con 0x7efd080cc810 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd067fc700 1 -- 172.21.15.131:0/1712303253 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00012430 con 0x7efd080cc810 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd0ddff700 1 -- 172.21.15.131:0/1712303253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 msgr2=0x7efd080c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd0ddff700 1 --2- 172.21.15.131:0/1712303253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 0x7efd080c8a20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7efd00009e60 tx=0x7efd00009780 comp rx=0 tx=0).stop 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd0ddff700 1 -- 172.21.15.131:0/1712303253 shutdown_connections 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd0ddff700 1 --2- 172.21.15.131:0/1712303253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 0x7efd080c8a20 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd0ddff700 1 -- 172.21.15.131:0/1712303253 >> 172.21.15.131:0/1712303253 conn(0x7efd080ce3e0 msgr2=0x7efd080ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:33.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd0ddff700 1 -- 172.21.15.131:0/1712303253 shutdown_connections 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.918+0000 7efd0ddff700 1 -- 172.21.15.131:0/1712303253 wait complete. 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.920+0000 7efd0ddff700 1 Processor -- start 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.920+0000 7efd0ddff700 1 -- start start 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd0ddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd0806bfd0 0x7efd08159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd0ddff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 0x7efd0815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd0ddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd0813c510 0x7efd080b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd0ddff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd080b1180 con 0x7efd080cc810 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd0ddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd080b1a20 con 0x7efd0813c510 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd0ddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0813f4a0 con 0x7efd0806bfd0 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd06ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 0x7efd0815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd06ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 0x7efd0815bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53200/0 (socket says 172.21.15.131:53200) 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd06ffd700 1 -- 172.21.15.131:0/1852656406 learned_addr learned my addr 172.21.15.131:0/1852656406 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd07fff700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd0813c510 0x7efd080b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.921+0000 7efd077fe700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd0806bfd0 0x7efd08159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efd06ffd700 1 -- 172.21.15.131:0/1852656406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd0806bfd0 msgr2=0x7efd08159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efd06ffd700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd0806bfd0 0x7efd08159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efd06ffd700 1 -- 172.21.15.131:0/1852656406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd0813c510 msgr2=0x7efd080b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efd06ffd700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd0813c510 0x7efd080b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efd06ffd700 1 -- 172.21.15.131:0/1852656406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd00009a20 con 0x7efd080cc810 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efd06ffd700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 0x7efd0815bb40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00c5c0 tx=0x7efcfc001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efcef7fe700 1 -- 172.21.15.131:0/1852656406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc005b30 con 0x7efd080cc810 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efcef7fe700 1 -- 172.21.15.131:0/1852656406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc005cb0 con 0x7efd080cc810 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efcef7fe700 1 -- 172.21.15.131:0/1852656406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc007320 con 0x7efd080cc810 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd080b13d0 con 0x7efd080cc810 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.922+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0815c080 con 0x7efd080cc810 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.923+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd080657e0 con 0x7efd080cc810 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.924+0000 7efcef7fe700 1 -- 172.21.15.131:0/1852656406 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efcfc01b3b0 con 0x7efd080cc810 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.925+0000 7efcef7fe700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efcf4066f30 0x7efcf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:33.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.925+0000 7efcef7fe700 1 -- 172.21.15.131:0/1852656406 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7efcfc083ac0 con 0x7efd080cc810 2023-12-26T21:56:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.925+0000 7efd077fe700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efcf4066f30 0x7efcf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.926+0000 7efd077fe700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efcf4066f30 0x7efcf40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efcf8000fb0 tx=0x7efcf8007e90 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:33.929+0000 7efcef7fe700 1 -- 172.21.15.131:0/1852656406 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcfc050af0 con 0x7efd080cc810 2023-12-26T21:56:34.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.114+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd080b1b60 con 0x7efcf4066f30 2023-12-26T21:56:34.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.123+0000 7efcef7fe700 1 -- 172.21.15.131:0/1852656406 <== mgr.7591 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289341 (secure 0 0 0) 0x7efd080b1b60 con 0x7efcf4066f30 2023-12-26T21:56:34.128 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.126+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efcf4066f30 msgr2=0x7efcf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efcf4066f30 0x7efcf40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efcf8000fb0 tx=0x7efcf8007e90 comp rx=0 tx=0).stop 2023-12-26T21:56:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 msgr2=0x7efd0815bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 0x7efd0815bb40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00c5c0 tx=0x7efcfc001020 comp rx=0 tx=0).stop 2023-12-26T21:56:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 shutdown_connections 2023-12-26T21:56:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd0813c510 0x7efd080b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd0806bfd0 0x7efd08159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efcf4066f30 0x7efcf40693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 --2- 172.21.15.131:0/1852656406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd080cc810 0x7efd0815bb40 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 >> 172.21.15.131:0/1852656406 conn(0x7efd080ce3e0 msgr2=0x7efd0815eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:34.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 shutdown_connections 2023-12-26T21:56:34.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.127+0000 7efd0ddff700 1 -- 172.21.15.131:0/1852656406 wait complete. 2023-12-26T21:56:34.132 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T21:56:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-26T21:56:32.859266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2454364,"kb_used_data":14620,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752520356,"statfs":{"total":773094113280,"available":770580844544,"internally_reserved":0,"allocated":14970880,"data_stored":7931512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.559629"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.306326+0000","last_change":"2023-12-26T21:56:29.302600+0000","last_active":"2023-12-26T21:56:29.306326+0000","last_peered":"2023-12-26T21:56:29.306326+0000","last_clean":"2023-12-26T21:56:29.306326+0000","last_became_active":"2023-12-26T21:56:29.298324+0000","last_became_peered":"2023-12-26T21:56:29.298324+0000","last_unstale":"2023-12-26T21:56:29.306326+0000","last_undegraded":"2023-12-26T21:56:29.306326+0000","last_fullsized":"2023-12-26T21:56:29.306326+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:13:24.218551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.302669+0000","last_change":"2023-12-26T21:56:29.300051+0000","last_active":"2023-12-26T21:56:29.302669+0000","last_peered":"2023-12-26T21:56:29.302669+0000","last_clean":"2023-12-26T21:56:29.302669+0000","last_became_active":"2023-12-26T21:56:29.299800+0000","last_became_peered":"2023-12-26T21:56:29.299800+0000","last_unstale":"2023-12-26T21:56:29.302669+0000","last_undegraded":"2023-12-26T21:56:29.302669+0000","last_fullsized":"2023-12-26T21:56:29.302669+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:01:07.411436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.302349+0000","last_change":"2023-12-26T21:56:29.299977+0000","last_active":"2023-12-26T21:56:29.302349+0000","last_peered":"2023-12-26T21:56:29.302349+0000","last_clean":"2023-12-26T21:56:29.302349+0000","last_became_active":"2023-12-26T21:56:29.299683+0000","last_became_peered":"2023-12-26T21:56:29.299683+0000","last_unstale":"2023-12-26T21:56:29.302349+0000","last_undegraded":"2023-12-26T21:56:29.302349+0000","last_fullsized":"2023-12-26T21:56:29.302349+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:30:42.736271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296985+0000","last_change":"2023-12-26T21:56:29.287378+0000","last_active":"2023-12-26T21:56:29.296985+0000","last_peered":"2023-12-26T21:56:29.296985+0000","last_clean":"2023-12-26T21:56:29.296985+0000","last_became_active":"2023-12-26T21:56:29.286765+0000","last_became_peered":"2023-12-26T21:56:29.286765+0000","last_unstale":"2023-12-26T21:56:29.296985+0000","last_undegraded":"2023-12-26T21:56:29.296985+0000","last_fullsized":"2023-12-26T21:56:29.296985+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:54:52.074115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.309928+0000","last_change":"2023-12-26T21:56:29.305090+0000","last_active":"2023-12-26T21:56:29.309928+0000","last_peered":"2023-12-26T21:56:29.309928+0000","last_clean":"2023-12-26T21:56:29.309928+0000","last_became_active":"2023-12-26T21:56:29.304787+0000","last_became_peered":"2023-12-26T21:56:29.304787+0000","last_unstale":"2023-12-26T21:56:29.309928+0000","last_undegraded":"2023-12-26T21:56:29.309928+0000","last_fullsized":"2023-12-26T21:56:29.309928+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:49:28.072941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.307945+0000","last_change":"2023-12-26T21:56:29.302132+0000","last_active":"2023-12-26T21:56:29.307945+0000","last_peered":"2023-12-26T21:56:29.307945+0000","last_clean":"2023-12-26T21:56:29.307945+0000","last_became_active":"2023-12-26T21:56:29.301816+0000","last_became_peered":"2023-12-26T21:56:29.301816+0000","last_unstale":"2023-12-26T21:56:29.307945+0000","last_undegraded":"2023-12-26T21:56:29.307945+0000","last_fullsized":"2023-12-26T21:56:29.307945+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:13:49.718420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296184+0000","last_change":"2023-12-26T21:56:29.292318+0000","last_active":"2023-12-26T21:56:29.296184+0000","last_peered":"2023-12-26T21:56:29.296184+0000","last_clean":"2023-12-26T21:56:29.296184+0000","last_became_active":"2023-12-26T21:56:29.292007+0000","last_became_peered":"2023-12-26T21:56:29.292007+0000","last_unstale":"2023-12-26T21:56:29.296184+0000","last_undegraded":"2023-12-26T21:56:29.296184+0000","last_fullsized":"2023-12-26T21:56:29.296184+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:07:27.210531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.291235+0000","last_change":"2023-12-26T21:56:29.290342+0000","last_active":"2023-12-26T21:56:29.291235+0000","last_peered":"2023-12-26T21:56:29.291235+0000","last_clean":"2023-12-26T21:56:29.291235+0000","last_became_active":"2023-12-26T21:56:29.290037+0000","last_became_peered":"2023-12-26T21:56:29.290037+0000","last_unstale":"2023-12-26T21:56:29.291235+0000","last_undegraded":"2023-12-26T21:56:29.291235+0000","last_fullsized":"2023-12-26T21:56:29.291235+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:58:23.621550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308463+0000","last_change":"2023-12-26T21:56:29.306806+0000","last_active":"2023-12-26T21:56:29.308463+0000","last_peered":"2023-12-26T21:56:29.308463+0000","last_clean":"2023-12-26T21:56:29.308463+0000","last_became_active":"2023-12-26T21:56:29.306521+0000","last_became_peered":"2023-12-26T21:56:29.306521+0000","last_unstale":"2023-12-26T21:56:29.308463+0000","last_undegraded":"2023-12-26T21:56:29.308463+0000","last_fullsized":"2023-12-26T21:56:29.308463+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:31:48.703988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.304459+0000","last_change":"2023-12-26T21:56:29.299269+0000","last_active":"2023-12-26T21:56:29.304459+0000","last_peered":"2023-12-26T21:56:29.304459+0000","last_clean":"2023-12-26T21:56:29.304459+0000","last_became_active":"2023-12-26T21:56:29.298977+0000","last_became_peered":"2023-12-26T21:56:29.298977+0000","last_unstale":"2023-12-26T21:56:29.304459+0000","last_undegraded":"2023-12-26T21:56:29.304459+0000","last_fullsized":"2023-12-26T21:56:29.304459+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:25:34.253454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.294192+0000","last_change":"2023-12-26T21:56:29.291917+0000","last_active":"2023-12-26T21:56:29.294192+0000","last_peered":"2023-12-26T21:56:29.294192+0000","last_clean":"2023-12-26T21:56:29.294192+0000","last_became_active":"2023-12-26T21:56:29.291637+0000","last_became_peered":"2023-12-26T21:56:29.291637+0000","last_unstale":"2023-12-26T21:56:29.294192+0000","last_undegraded":"2023-12-26T21:56:29.294192+0000","last_fullsized":"2023-12-26T21:56:29.294192+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:57:49.627306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.304901+0000","last_change":"2023-12-26T21:56:29.300686+0000","last_active":"2023-12-26T21:56:29.304901+0000","last_peered":"2023-12-26T21:56:29.304901+0000","last_clean":"2023-12-26T21:56:29.304901+0000","last_became_active":"2023-12-26T21:56:29.300378+0000","last_became_peered":"2023-12-26T21:56:29.300378+0000","last_unstale":"2023-12-26T21:56:29.304901+0000","last_undegraded":"2023-12-26T21:56:29.304901+0000","last_fullsized":"2023-12-26T21:56:29.304901+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:41:28.661110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.299584+0000","last_change":"2023-12-26T21:56:29.295451+0000","last_active":"2023-12-26T21:56:29.299584+0000","last_peered":"2023-12-26T21:56:29.299584+0000","last_clean":"2023-12-26T21:56:29.299584+0000","last_became_active":"2023-12-26T21:56:29.293088+0000","last_became_peered":"2023-12-26T21:56:29.293088+0000","last_unstale":"2023-12-26T21:56:29.299584+0000","last_undegraded":"2023-12-26T21:56:29.299584+0000","last_fullsized":"2023-12-26T21:56:29.299584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:38:28.059515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.298203+0000","last_change":"2023-12-26T21:56:29.295002+0000","last_active":"2023-12-26T21:56:29.298203+0000","last_peered":"2023-12-26T21:56:29.298203+0000","last_clean":"2023-12-26T21:56:29.298203+0000","last_became_active":"2023-12-26T21:56:29.291508+0000","last_became_peered":"2023-12-26T21:56:29.291508+0000","last_unstale":"2023-12-26T21:56:29.298203+0000","last_undegraded":"2023-12-26T21:56:29.298203+0000","last_fullsized":"2023-12-26T21:56:29.298203+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:15.376677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308474+0000","last_change":"2023-12-26T21:56:29.306657+0000","last_active":"2023-12-26T21:56:29.308474+0000","last_peered":"2023-12-26T21:56:29.308474+0000","last_clean":"2023-12-26T21:56:29.308474+0000","last_became_active":"2023-12-26T21:56:29.306390+0000","last_became_peered":"2023-12-26T21:56:29.306390+0000","last_unstale":"2023-12-26T21:56:29.308474+0000","last_undegraded":"2023-12-26T21:56:29.308474+0000","last_fullsized":"2023-12-26T21:56:29.308474+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:31:12.310074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.291780+0000","last_change":"2023-12-26T21:56:29.284827+0000","last_active":"2023-12-26T21:56:29.291780+0000","last_peered":"2023-12-26T21:56:29.291780+0000","last_clean":"2023-12-26T21:56:29.291780+0000","last_became_active":"2023-12-26T21:56:29.284439+0000","last_became_peered":"2023-12-26T21:56:29.284439+0000","last_unstale":"2023-12-26T21:56:29.291780+0000","last_undegraded":"2023-12-26T21:56:29.291780+0000","last_fullsized":"2023-12-26T21:56:29.291780+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:03.057577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303607+0000","last_change":"2023-12-26T21:56:29.297224+0000","last_active":"2023-12-26T21:56:29.303607+0000","last_peered":"2023-12-26T21:56:29.303607+0000","last_clean":"2023-12-26T21:56:29.303607+0000","last_became_active":"2023-12-26T21:56:29.296887+0000","last_became_peered":"2023-12-26T21:56:29.296887+0000","last_unstale":"2023-12-26T21:56:29.303607+0000","last_undegraded":"2023-12-26T21:56:29.303607+0000","last_fullsized":"2023-12-26T21:56:29.303607+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:27:49.727129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303513+0000","last_change":"2023-12-26T21:56:29.301589+0000","last_active":"2023-12-26T21:56:29.303513+0000","last_peered":"2023-12-26T21:56:29.303513+0000","last_clean":"2023-12-26T21:56:29.303513+0000","last_became_active":"2023-12-26T21:56:29.301317+0000","last_became_peered":"2023-12-26T21:56:29.301317+0000","last_unstale":"2023-12-26T21:56:29.303513+0000","last_undegraded":"2023-12-26T21:56:29.303513+0000","last_fullsized":"2023-12-26T21:56:29.303513+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:55:59.668636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308525+0000","last_change":"2023-12-26T21:56:29.304832+0000","last_active":"2023-12-26T21:56:29.308525+0000","last_peered":"2023-12-26T21:56:29.308525+0000","last_clean":"2023-12-26T21:56:29.308525+0000","last_became_active":"2023-12-26T21:56:29.304567+0000","last_became_peered":"2023-12-26T21:56:29.304567+0000","last_unstale":"2023-12-26T21:56:29.308525+0000","last_undegraded":"2023-12-26T21:56:29.308525+0000","last_fullsized":"2023-12-26T21:56:29.308525+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:16:32.394353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303545+0000","last_change":"2023-12-26T21:56:29.300938+0000","last_active":"2023-12-26T21:56:29.303545+0000","last_peered":"2023-12-26T21:56:29.303545+0000","last_clean":"2023-12-26T21:56:29.303545+0000","last_became_active":"2023-12-26T21:56:29.299464+0000","last_became_peered":"2023-12-26T21:56:29.299464+0000","last_unstale":"2023-12-26T21:56:29.303545+0000","last_undegraded":"2023-12-26T21:56:29.303545+0000","last_fullsized":"2023-12-26T21:56:29.303545+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:01:08.499642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.291188+0000","last_change":"2023-12-26T21:56:29.284598+0000","last_active":"2023-12-26T21:56:29.291188+0000","last_peered":"2023-12-26T21:56:29.291188+0000","last_clean":"2023-12-26T21:56:29.291188+0000","last_became_active":"2023-12-26T21:56:29.284308+0000","last_became_peered":"2023-12-26T21:56:29.284308+0000","last_unstale":"2023-12-26T21:56:29.291188+0000","last_undegraded":"2023-12-26T21:56:29.291188+0000","last_fullsized":"2023-12-26T21:56:29.291188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:53:44.515057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.298083+0000","last_change":"2023-12-26T21:56:29.293702+0000","last_active":"2023-12-26T21:56:29.298083+0000","last_peered":"2023-12-26T21:56:29.298083+0000","last_clean":"2023-12-26T21:56:29.298083+0000","last_became_active":"2023-12-26T21:56:29.293344+0000","last_became_peered":"2023-12-26T21:56:29.293344+0000","last_unstale":"2023-12-26T21:56:29.298083+0000","last_undegraded":"2023-12-26T21:56:29.298083+0000","last_fullsized":"2023-12-26T21:56:29.298083+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:50:33.880355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296146+0000","last_change":"2023-12-26T21:56:29.293633+0000","last_active":"2023-12-26T21:56:29.296146+0000","last_peered":"2023-12-26T21:56:29.296146+0000","last_clean":"2023-12-26T21:56:29.296146+0000","last_became_active":"2023-12-26T21:56:29.293346+0000","last_became_peered":"2023-12-26T21:56:29.293346+0000","last_unstale":"2023-12-26T21:56:29.296146+0000","last_undegraded":"2023-12-26T21:56:29.296146+0000","last_fullsized":"2023-12-26T21:56:29.296146+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:13:49.718420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.307648+0000","last_change":"2023-12-26T21:56:29.297091+0000","last_active":"2023-12-26T21:56:29.307648+0000","last_peered":"2023-12-26T21:56:29.307648+0000","last_clean":"2023-12-26T21:56:29.307648+0000","last_became_active":"2023-12-26T21:56:29.296771+0000","last_became_peered":"2023-12-26T21:56:29.296771+0000","last_unstale":"2023-12-26T21:56:29.307648+0000","last_undegraded":"2023-12-26T21:56:29.307648+0000","last_fullsized":"2023-12-26T21:56:29.307648+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:15:05.819700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.301821+0000","last_change":"2023-12-26T21:56:29.297791+0000","last_active":"2023-12-26T21:56:29.301821+0000","last_peered":"2023-12-26T21:56:29.301821+0000","last_clean":"2023-12-26T21:56:29.301821+0000","last_became_active":"2023-12-26T21:56:29.294050+0000","last_became_peered":"2023-12-26T21:56:29.294050+0000","last_unstale":"2023-12-26T21:56:29.301821+0000","last_undegraded":"2023-12-26T21:56:29.301821+0000","last_fullsized":"2023-12-26T21:56:29.301821+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:14.983804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.305868+0000","last_change":"2023-12-26T21:56:29.297762+0000","last_active":"2023-12-26T21:56:29.305868+0000","last_peered":"2023-12-26T21:56:29.305868+0000","last_clean":"2023-12-26T21:56:29.305868+0000","last_became_active":"2023-12-26T21:56:29.297506+0000","last_became_peered":"2023-12-26T21:56:29.297506+0000","last_unstale":"2023-12-26T21:56:29.305868+0000","last_undegraded":"2023-12-26T21:56:29.305868+0000","last_fullsized":"2023-12-26T21:56:29.305868+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:24.216953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.306777+0000","last_change":"2023-12-26T21:56:29.301104+0000","last_active":"2023-12-26T21:56:29.306777+0000","last_peered":"2023-12-26T21:56:29.306777+0000","last_clean":"2023-12-26T21:56:29.306777+0000","last_became_active":"2023-12-26T21:56:29.300816+0000","last_became_peered":"2023-12-26T21:56:29.300816+0000","last_unstale":"2023-12-26T21:56:29.306777+0000","last_undegraded":"2023-12-26T21:56:29.306777+0000","last_fullsized":"2023-12-26T21:56:29.306777+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:39:35.247764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303790+0000","last_change":"2023-12-26T21:56:29.297806+0000","last_active":"2023-12-26T21:56:29.303790+0000","last_peered":"2023-12-26T21:56:29.303790+0000","last_clean":"2023-12-26T21:56:29.303790+0000","last_became_active":"2023-12-26T21:56:29.297557+0000","last_became_peered":"2023-12-26T21:56:29.297557+0000","last_unstale":"2023-12-26T21:56:29.303790+0000","last_undegraded":"2023-12-26T21:56:29.303790+0000","last_fullsized":"2023-12-26T21:56:29.303790+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:59:51.699509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296178+0000","last_change":"2023-12-26T21:56:29.292069+0000","last_active":"2023-12-26T21:56:29.296178+0000","last_peered":"2023-12-26T21:56:29.296178+0000","last_clean":"2023-12-26T21:56:29.296178+0000","last_became_active":"2023-12-26T21:56:29.291819+0000","last_became_peered":"2023-12-26T21:56:29.291819+0000","last_unstale":"2023-12-26T21:56:29.296178+0000","last_undegraded":"2023-12-26T21:56:29.296178+0000","last_fullsized":"2023-12-26T21:56:29.296178+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:20:11.430124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303455+0000","last_change":"2023-12-26T21:56:29.299065+0000","last_active":"2023-12-26T21:56:29.303455+0000","last_peered":"2023-12-26T21:56:29.303455+0000","last_clean":"2023-12-26T21:56:29.303455+0000","last_became_active":"2023-12-26T21:56:29.298809+0000","last_became_peered":"2023-12-26T21:56:29.298809+0000","last_unstale":"2023-12-26T21:56:29.303455+0000","last_undegraded":"2023-12-26T21:56:29.303455+0000","last_fullsized":"2023-12-26T21:56:29.303455+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:33:40.337183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.306009+0000","last_change":"2023-12-26T21:56:29.301094+0000","last_active":"2023-12-26T21:56:29.306009+0000","last_peered":"2023-12-26T21:56:29.306009+0000","last_clean":"2023-12-26T21:56:29.306009+0000","last_became_active":"2023-12-26T21:56:29.300655+0000","last_became_peered":"2023-12-26T21:56:29.300655+0000","last_unstale":"2023-12-26T21:56:29.306009+0000","last_undegraded":"2023-12-26T21:56:29.306009+0000","last_fullsized":"2023-12-26T21:56:29.306009+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:18.076626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.304642+0000","last_change":"2023-12-26T21:56:29.301384+0000","last_active":"2023-12-26T21:56:29.304642+0000","last_peered":"2023-12-26T21:56:29.304642+0000","last_clean":"2023-12-26T21:56:29.304642+0000","last_became_active":"2023-12-26T21:56:29.298288+0000","last_became_peered":"2023-12-26T21:56:29.298288+0000","last_unstale":"2023-12-26T21:56:29.304642+0000","last_undegraded":"2023-12-26T21:56:29.304642+0000","last_fullsized":"2023-12-26T21:56:29.304642+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:50.244766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.306626+0000","last_change":"2023-12-26T21:56:29.303308+0000","last_active":"2023-12-26T21:56:29.306626+0000","last_peered":"2023-12-26T21:56:29.306626+0000","last_clean":"2023-12-26T21:56:29.306626+0000","last_became_active":"2023-12-26T21:56:29.301194+0000","last_became_peered":"2023-12-26T21:56:29.301194+0000","last_unstale":"2023-12-26T21:56:29.306626+0000","last_undegraded":"2023-12-26T21:56:29.306626+0000","last_fullsized":"2023-12-26T21:56:29.306626+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:05.988465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"138'4","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329343+0000","last_change":"2023-12-26T21:56:28.329200+0000","last_active":"2023-12-26T21:56:29.329343+0000","last_peered":"2023-12-26T21:56:29.329343+0000","last_clean":"2023-12-26T21:56:29.329343+0000","last_became_active":"2023-12-26T21:56:28.328941+0000","last_became_peered":"2023-12-26T21:56:28.328941+0000","last_unstale":"2023-12-26T21:56:29.329343+0000","last_undegraded":"2023-12-26T21:56:29.329343+0000","last_fullsized":"2023-12-26T21:56:29.329343+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:33:17.070266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.298995+0000","last_change":"2023-12-26T21:56:29.294745+0000","last_active":"2023-12-26T21:56:29.298995+0000","last_peered":"2023-12-26T21:56:29.298995+0000","last_clean":"2023-12-26T21:56:29.298995+0000","last_became_active":"2023-12-26T21:56:29.294467+0000","last_became_peered":"2023-12-26T21:56:29.294467+0000","last_unstale":"2023-12-26T21:56:29.298995+0000","last_undegraded":"2023-12-26T21:56:29.298995+0000","last_fullsized":"2023-12-26T21:56:29.298995+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:48.124399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.326673+0000","last_change":"2023-12-26T21:56:28.335859+0000","last_active":"2023-12-26T21:56:29.326673+0000","last_peered":"2023-12-26T21:56:29.326673+0000","last_clean":"2023-12-26T21:56:29.326673+0000","last_became_active":"2023-12-26T21:56:28.335471+0000","last_became_peered":"2023-12-26T21:56:28.335471+0000","last_unstale":"2023-12-26T21:56:29.326673+0000","last_undegraded":"2023-12-26T21:56:29.326673+0000","last_fullsized":"2023-12-26T21:56:29.326673+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:49:27.066581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303813+0000","last_change":"2023-12-26T21:56:29.299927+0000","last_active":"2023-12-26T21:56:29.303813+0000","last_peered":"2023-12-26T21:56:29.303813+0000","last_clean":"2023-12-26T21:56:29.303813+0000","last_became_active":"2023-12-26T21:56:29.299658+0000","last_became_peered":"2023-12-26T21:56:29.299658+0000","last_unstale":"2023-12-26T21:56:29.303813+0000","last_undegraded":"2023-12-26T21:56:29.303813+0000","last_fullsized":"2023-12-26T21:56:29.303813+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:55:11.959879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329398+0000","last_change":"2023-12-26T21:56:28.333793+0000","last_active":"2023-12-26T21:56:29.329398+0000","last_peered":"2023-12-26T21:56:29.329398+0000","last_clean":"2023-12-26T21:56:29.329398+0000","last_became_active":"2023-12-26T21:56:28.333519+0000","last_became_peered":"2023-12-26T21:56:28.333519+0000","last_unstale":"2023-12-26T21:56:29.329398+0000","last_undegraded":"2023-12-26T21:56:29.329398+0000","last_fullsized":"2023-12-26T21:56:29.329398+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:05.348656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308507+0000","last_change":"2023-12-26T21:56:29.304767+0000","last_active":"2023-12-26T21:56:29.308507+0000","last_peered":"2023-12-26T21:56:29.308507+0000","last_clean":"2023-12-26T21:56:29.308507+0000","last_became_active":"2023-12-26T21:56:29.304560+0000","last_became_peered":"2023-12-26T21:56:29.304560+0000","last_unstale":"2023-12-26T21:56:29.308507+0000","last_undegraded":"2023-12-26T21:56:29.308507+0000","last_fullsized":"2023-12-26T21:56:29.308507+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:19:28.678021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.331907+0000","last_change":"2023-12-26T21:56:28.333975+0000","last_active":"2023-12-26T21:56:29.331907+0000","last_peered":"2023-12-26T21:56:29.331907+0000","last_clean":"2023-12-26T21:56:29.331907+0000","last_became_active":"2023-12-26T21:56:28.333614+0000","last_became_peered":"2023-12-26T21:56:28.333614+0000","last_unstale":"2023-12-26T21:56:29.331907+0000","last_undegraded":"2023-12-26T21:56:29.331907+0000","last_fullsized":"2023-12-26T21:56:29.331907+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:49:27.066581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.305665+0000","last_change":"2023-12-26T21:56:29.297498+0000","last_active":"2023-12-26T21:56:29.305665+0000","last_peered":"2023-12-26T21:56:29.305665+0000","last_clean":"2023-12-26T21:56:29.305665+0000","last_became_active":"2023-12-26T21:56:29.296677+0000","last_became_peered":"2023-12-26T21:56:29.296677+0000","last_unstale":"2023-12-26T21:56:29.305665+0000","last_undegraded":"2023-12-26T21:56:29.305665+0000","last_fullsized":"2023-12-26T21:56:29.305665+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:39:00.944849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329114+0000","last_change":"2023-12-26T21:56:28.337058+0000","last_active":"2023-12-26T21:56:29.329114+0000","last_peered":"2023-12-26T21:56:29.329114+0000","last_clean":"2023-12-26T21:56:29.329114+0000","last_became_active":"2023-12-26T21:56:28.336682+0000","last_became_peered":"2023-12-26T21:56:28.336682+0000","last_unstale":"2023-12-26T21:56:29.329114+0000","last_undegraded":"2023-12-26T21:56:29.329114+0000","last_fullsized":"2023-12-26T21:56:29.329114+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:22:13.902192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.291208+0000","last_change":"2023-12-26T21:56:29.286935+0000","last_active":"2023-12-26T21:56:29.291208+0000","last_peered":"2023-12-26T21:56:29.291208+0000","last_clean":"2023-12-26T21:56:29.291208+0000","last_became_active":"2023-12-26T21:56:29.284293+0000","last_became_peered":"2023-12-26T21:56:29.284293+0000","last_unstale":"2023-12-26T21:56:29.291208+0000","last_undegraded":"2023-12-26T21:56:29.291208+0000","last_fullsized":"2023-12-26T21:56:29.291208+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:49:10.975962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.327801+0000","last_change":"2023-12-26T21:56:28.328243+0000","last_active":"2023-12-26T21:56:29.327801+0000","last_peered":"2023-12-26T21:56:29.327801+0000","last_clean":"2023-12-26T21:56:29.327801+0000","last_became_active":"2023-12-26T21:56:28.327298+0000","last_became_peered":"2023-12-26T21:56:28.327298+0000","last_unstale":"2023-12-26T21:56:29.327801+0000","last_undegraded":"2023-12-26T21:56:29.327801+0000","last_fullsized":"2023-12-26T21:56:29.327801+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:29:17.956980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.299533+0000","last_change":"2023-12-26T21:56:29.291896+0000","last_active":"2023-12-26T21:56:29.299533+0000","last_peered":"2023-12-26T21:56:29.299533+0000","last_clean":"2023-12-26T21:56:29.299533+0000","last_became_active":"2023-12-26T21:56:29.291520+0000","last_became_peered":"2023-12-26T21:56:29.291520+0000","last_unstale":"2023-12-26T21:56:29.299533+0000","last_undegraded":"2023-12-26T21:56:29.299533+0000","last_fullsized":"2023-12-26T21:56:29.299533+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:12.386199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.278711+0000","last_change":"2023-12-26T21:56:28.326837+0000","last_active":"2023-12-26T21:56:29.278711+0000","last_peered":"2023-12-26T21:56:29.278711+0000","last_clean":"2023-12-26T21:56:29.278711+0000","last_became_active":"2023-12-26T21:56:28.326583+0000","last_became_peered":"2023-12-26T21:56:28.326583+0000","last_unstale":"2023-12-26T21:56:29.278711+0000","last_undegraded":"2023-12-26T21:56:29.278711+0000","last_fullsized":"2023-12-26T21:56:29.278711+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:04:27.405594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.292398+0000","last_change":"2023-12-26T21:56:29.290560+0000","last_active":"2023-12-26T21:56:29.292398+0000","last_peered":"2023-12-26T21:56:29.292398+0000","last_clean":"2023-12-26T21:56:29.292398+0000","last_became_active":"2023-12-26T21:56:29.289803+0000","last_became_peered":"2023-12-26T21:56:29.289803+0000","last_unstale":"2023-12-26T21:56:29.292398+0000","last_undegraded":"2023-12-26T21:56:29.292398+0000","last_fullsized":"2023-12-26T21:56:29.292398+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:00:09.663521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.280515+0000","last_change":"2023-12-26T21:56:28.337306+0000","last_active":"2023-12-26T21:56:29.280515+0000","last_peered":"2023-12-26T21:56:29.280515+0000","last_clean":"2023-12-26T21:56:29.280515+0000","last_became_active":"2023-12-26T21:56:28.336945+0000","last_became_peered":"2023-12-26T21:56:28.336945+0000","last_unstale":"2023-12-26T21:56:29.280515+0000","last_undegraded":"2023-12-26T21:56:29.280515+0000","last_fullsized":"2023-12-26T21:56:29.280515+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:18.422751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.305990+0000","last_change":"2023-12-26T21:56:29.303119+0000","last_active":"2023-12-26T21:56:29.305990+0000","last_peered":"2023-12-26T21:56:29.305990+0000","last_clean":"2023-12-26T21:56:29.305990+0000","last_became_active":"2023-12-26T21:56:29.302040+0000","last_became_peered":"2023-12-26T21:56:29.302040+0000","last_unstale":"2023-12-26T21:56:29.305990+0000","last_undegraded":"2023-12-26T21:56:29.305990+0000","last_fullsized":"2023-12-26T21:56:29.305990+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:44:43.541452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.279701+0000","last_change":"2023-12-26T21:56:28.332621+0000","last_active":"2023-12-26T21:56:29.279701+0000","last_peered":"2023-12-26T21:56:29.279701+0000","last_clean":"2023-12-26T21:56:29.279701+0000","last_became_active":"2023-12-26T21:56:28.332294+0000","last_became_peered":"2023-12-26T21:56:28.332294+0000","last_unstale":"2023-12-26T21:56:29.279701+0000","last_undegraded":"2023-12-26T21:56:29.279701+0000","last_fullsized":"2023-12-26T21:56:29.279701+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:49.238406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.293318+0000","last_change":"2023-12-26T21:56:29.287202+0000","last_active":"2023-12-26T21:56:29.293318+0000","last_peered":"2023-12-26T21:56:29.293318+0000","last_clean":"2023-12-26T21:56:29.293318+0000","last_became_active":"2023-12-26T21:56:29.286859+0000","last_became_peered":"2023-12-26T21:56:29.286859+0000","last_unstale":"2023-12-26T21:56:29.293318+0000","last_undegraded":"2023-12-26T21:56:29.293318+0000","last_fullsized":"2023-12-26T21:56:29.293318+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:27:34.201449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.280218+0000","last_change":"2023-12-26T21:56:28.324714+0000","last_active":"2023-12-26T21:56:29.280218+0000","last_peered":"2023-12-26T21:56:29.280218+0000","last_clean":"2023-12-26T21:56:29.280218+0000","last_became_active":"2023-12-26T21:56:28.324394+0000","last_became_peered":"2023-12-26T21:56:28.324394+0000","last_unstale":"2023-12-26T21:56:29.280218+0000","last_undegraded":"2023-12-26T21:56:29.280218+0000","last_fullsized":"2023-12-26T21:56:29.280218+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:30:26.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.300029+0000","last_change":"2023-12-26T21:56:29.296619+0000","last_active":"2023-12-26T21:56:29.300029+0000","last_peered":"2023-12-26T21:56:29.300029+0000","last_clean":"2023-12-26T21:56:29.300029+0000","last_became_active":"2023-12-26T21:56:29.296355+0000","last_became_peered":"2023-12-26T21:56:29.296355+0000","last_unstale":"2023-12-26T21:56:29.300029+0000","last_undegraded":"2023-12-26T21:56:29.300029+0000","last_fullsized":"2023-12-26T21:56:29.300029+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:38:16.041172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"138'3","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.331541+0000","last_change":"2023-12-26T21:56:28.330566+0000","last_active":"2023-12-26T21:56:29.331541+0000","last_peered":"2023-12-26T21:56:29.331541+0000","last_clean":"2023-12-26T21:56:29.331541+0000","last_became_active":"2023-12-26T21:56:28.330300+0000","last_became_peered":"2023-12-26T21:56:28.330300+0000","last_unstale":"2023-12-26T21:56:29.331541+0000","last_undegraded":"2023-12-26T21:56:29.331541+0000","last_fullsized":"2023-12-26T21:56:29.331541+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:08:47.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303330+0000","last_change":"2023-12-26T21:56:29.298988+0000","last_active":"2023-12-26T21:56:29.303330+0000","last_peered":"2023-12-26T21:56:29.303330+0000","last_clean":"2023-12-26T21:56:29.303330+0000","last_became_active":"2023-12-26T21:56:29.298717+0000","last_became_peered":"2023-12-26T21:56:29.298717+0000","last_unstale":"2023-12-26T21:56:29.303330+0000","last_undegraded":"2023-12-26T21:56:29.303330+0000","last_fullsized":"2023-12-26T21:56:29.303330+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:16:49.690202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"138'6","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.333593+0000","last_change":"2023-12-26T21:56:28.333459+0000","last_active":"2023-12-26T21:56:29.333593+0000","last_peered":"2023-12-26T21:56:29.333593+0000","last_clean":"2023-12-26T21:56:29.333593+0000","last_became_active":"2023-12-26T21:56:28.332790+0000","last_became_peered":"2023-12-26T21:56:28.332790+0000","last_unstale":"2023-12-26T21:56:29.333593+0000","last_undegraded":"2023-12-26T21:56:29.333593+0000","last_fullsized":"2023-12-26T21:56:29.333593+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:55:49.457721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.289684+0000","last_change":"2023-12-26T21:56:29.284491+0000","last_active":"2023-12-26T21:56:29.289684+0000","last_peered":"2023-12-26T21:56:29.289684+0000","last_clean":"2023-12-26T21:56:29.289684+0000","last_became_active":"2023-12-26T21:56:29.284083+0000","last_became_peered":"2023-12-26T21:56:29.284083+0000","last_unstale":"2023-12-26T21:56:29.289684+0000","last_undegraded":"2023-12-26T21:56:29.289684+0000","last_fullsized":"2023-12-26T21:56:29.289684+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:06.355015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329577+0000","last_change":"2023-12-26T21:56:28.331301+0000","last_active":"2023-12-26T21:56:29.329577+0000","last_peered":"2023-12-26T21:56:29.329577+0000","last_clean":"2023-12-26T21:56:29.329577+0000","last_became_active":"2023-12-26T21:56:28.330897+0000","last_became_peered":"2023-12-26T21:56:28.330897+0000","last_unstale":"2023-12-26T21:56:29.329577+0000","last_undegraded":"2023-12-26T21:56:29.329577+0000","last_fullsized":"2023-12-26T21:56:29.329577+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:57.220980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.298107+0000","last_change":"2023-12-26T21:56:29.291431+0000","last_active":"2023-12-26T21:56:29.298107+0000","last_peered":"2023-12-26T21:56:29.298107+0000","last_clean":"2023-12-26T21:56:29.298107+0000","last_became_active":"2023-12-26T21:56:29.290993+0000","last_became_peered":"2023-12-26T21:56:29.290993+0000","last_unstale":"2023-12-26T21:56:29.298107+0000","last_undegraded":"2023-12-26T21:56:29.298107+0000","last_fullsized":"2023-12-26T21:56:29.298107+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:58:31.017354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.292810+0000","last_change":"2023-12-26T21:56:28.330437+0000","last_active":"2023-12-26T21:56:29.292810+0000","last_peered":"2023-12-26T21:56:29.292810+0000","last_clean":"2023-12-26T21:56:29.292810+0000","last_became_active":"2023-12-26T21:56:28.330195+0000","last_became_peered":"2023-12-26T21:56:28.330195+0000","last_unstale":"2023-12-26T21:56:29.292810+0000","last_undegraded":"2023-12-26T21:56:29.292810+0000","last_fullsized":"2023-12-26T21:56:29.292810+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:27:07.378809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.302703+0000","last_change":"2023-12-26T21:56:29.295052+0000","last_active":"2023-12-26T21:56:29.302703+0000","last_peered":"2023-12-26T21:56:29.302703+0000","last_clean":"2023-12-26T21:56:29.302703+0000","last_became_active":"2023-12-26T21:56:29.294712+0000","last_became_peered":"2023-12-26T21:56:29.294712+0000","last_unstale":"2023-12-26T21:56:29.302703+0000","last_undegraded":"2023-12-26T21:56:29.302703+0000","last_fullsized":"2023-12-26T21:56:29.302703+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:30:59.091981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"138'8","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.334337+0000","last_change":"2023-12-26T21:56:28.323723+0000","last_active":"2023-12-26T21:56:29.334337+0000","last_peered":"2023-12-26T21:56:29.334337+0000","last_clean":"2023-12-26T21:56:29.334337+0000","last_became_active":"2023-12-26T21:56:28.323368+0000","last_became_peered":"2023-12-26T21:56:28.323368+0000","last_unstale":"2023-12-26T21:56:29.334337+0000","last_undegraded":"2023-12-26T21:56:29.334337+0000","last_fullsized":"2023-12-26T21:56:29.334337+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:54.707404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303502+0000","last_change":"2023-12-26T21:56:29.299169+0000","last_active":"2023-12-26T21:56:29.303502+0000","last_peered":"2023-12-26T21:56:29.303502+0000","last_clean":"2023-12-26T21:56:29.303502+0000","last_became_active":"2023-12-26T21:56:29.298882+0000","last_became_peered":"2023-12-26T21:56:29.298882+0000","last_unstale":"2023-12-26T21:56:29.303502+0000","last_undegraded":"2023-12-26T21:56:29.303502+0000","last_fullsized":"2023-12-26T21:56:29.303502+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:00:52.782353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329108+0000","last_change":"2023-12-26T21:56:28.338162+0000","last_active":"2023-12-26T21:56:29.329108+0000","last_peered":"2023-12-26T21:56:29.329108+0000","last_clean":"2023-12-26T21:56:29.329108+0000","last_became_active":"2023-12-26T21:56:28.337763+0000","last_became_peered":"2023-12-26T21:56:28.337763+0000","last_unstale":"2023-12-26T21:56:29.329108+0000","last_undegraded":"2023-12-26T21:56:29.329108+0000","last_fullsized":"2023-12-26T21:56:29.329108+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:20:10.423765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.307847+0000","last_change":"2023-12-26T21:56:29.304794+0000","last_active":"2023-12-26T21:56:29.307847+0000","last_peered":"2023-12-26T21:56:29.307847+0000","last_clean":"2023-12-26T21:56:29.307847+0000","last_became_active":"2023-12-26T21:56:29.304492+0000","last_became_peered":"2023-12-26T21:56:29.304492+0000","last_unstale":"2023-12-26T21:56:29.307847+0000","last_undegraded":"2023-12-26T21:56:29.307847+0000","last_fullsized":"2023-12-26T21:56:29.307847+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:33:10.959030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.327566+0000","last_change":"2023-12-26T21:56:28.328508+0000","last_active":"2023-12-26T21:56:29.327566+0000","last_peered":"2023-12-26T21:56:29.327566+0000","last_clean":"2023-12-26T21:56:29.327566+0000","last_became_active":"2023-12-26T21:56:28.327745+0000","last_became_peered":"2023-12-26T21:56:28.327745+0000","last_unstale":"2023-12-26T21:56:29.327566+0000","last_undegraded":"2023-12-26T21:56:29.327566+0000","last_fullsized":"2023-12-26T21:56:29.327566+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:05:03.449268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303289+0000","last_change":"2023-12-26T21:56:29.298392+0000","last_active":"2023-12-26T21:56:29.303289+0000","last_peered":"2023-12-26T21:56:29.303289+0000","last_clean":"2023-12-26T21:56:29.303289+0000","last_became_active":"2023-12-26T21:56:29.298095+0000","last_became_peered":"2023-12-26T21:56:29.298095+0000","last_unstale":"2023-12-26T21:56:29.303289+0000","last_undegraded":"2023-12-26T21:56:29.303289+0000","last_fullsized":"2023-12-26T21:56:29.303289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:27:10.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329145+0000","last_change":"2023-12-26T21:56:28.336955+0000","last_active":"2023-12-26T21:56:29.329145+0000","last_peered":"2023-12-26T21:56:29.329145+0000","last_clean":"2023-12-26T21:56:29.329145+0000","last_became_active":"2023-12-26T21:56:28.336641+0000","last_became_peered":"2023-12-26T21:56:28.336641+0000","last_unstale":"2023-12-26T21:56:29.329145+0000","last_undegraded":"2023-12-26T21:56:29.329145+0000","last_fullsized":"2023-12-26T21:56:29.329145+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:31:11.303715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303718+0000","last_change":"2023-12-26T21:56:29.300764+0000","last_active":"2023-12-26T21:56:29.303718+0000","last_peered":"2023-12-26T21:56:29.303718+0000","last_clean":"2023-12-26T21:56:29.303718+0000","last_became_active":"2023-12-26T21:56:29.297808+0000","last_became_peered":"2023-12-26T21:56:29.297808+0000","last_unstale":"2023-12-26T21:56:29.303718+0000","last_undegraded":"2023-12-26T21:56:29.303718+0000","last_fullsized":"2023-12-26T21:56:29.303718+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:26:19.047598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.283204+0000","last_change":"2023-12-26T21:56:28.327827+0000","last_active":"2023-12-26T21:56:29.283204+0000","last_peered":"2023-12-26T21:56:29.283204+0000","last_clean":"2023-12-26T21:56:29.283204+0000","last_became_active":"2023-12-26T21:56:28.327529+0000","last_became_peered":"2023-12-26T21:56:28.327529+0000","last_unstale":"2023-12-26T21:56:29.283204+0000","last_undegraded":"2023-12-26T21:56:29.283204+0000","last_fullsized":"2023-12-26T21:56:29.283204+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:04.982105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.296746+0000","last_change":"2023-12-26T21:56:29.295378+0000","last_active":"2023-12-26T21:56:29.296746+0000","last_peered":"2023-12-26T21:56:29.296746+0000","last_clean":"2023-12-26T21:56:29.296746+0000","last_became_active":"2023-12-26T21:56:29.295013+0000","last_became_peered":"2023-12-26T21:56:29.295013+0000","last_unstale":"2023-12-26T21:56:29.296746+0000","last_undegraded":"2023-12-26T21:56:29.296746+0000","last_fullsized":"2023-12-26T21:56:29.296746+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:40:30.129228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.329465+0000","last_change":"2023-12-26T21:56:28.326256+0000","last_active":"2023-12-26T21:56:29.329465+0000","last_peered":"2023-12-26T21:56:29.329465+0000","last_clean":"2023-12-26T21:56:29.329465+0000","last_became_active":"2023-12-26T21:56:28.325963+0000","last_became_peered":"2023-12-26T21:56:28.325963+0000","last_unstale":"2023-12-26T21:56:29.329465+0000","last_undegraded":"2023-12-26T21:56:29.329465+0000","last_fullsized":"2023-12-26T21:56:29.329465+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:42:28.838829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.300437+0000","last_change":"2023-12-26T21:56:29.296764+0000","last_active":"2023-12-26T21:56:29.300437+0000","last_peered":"2023-12-26T21:56:29.300437+0000","last_clean":"2023-12-26T21:56:29.300437+0000","last_became_active":"2023-12-26T21:56:29.296461+0000","last_became_peered":"2023-12-26T21:56:29.296461+0000","last_unstale":"2023-12-26T21:56:29.300437+0000","last_undegraded":"2023-12-26T21:56:29.300437+0000","last_fullsized":"2023-12-26T21:56:29.300437+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:25:48.978620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.293176+0000","last_change":"2023-12-26T21:56:28.331745+0000","last_active":"2023-12-26T21:56:29.293176+0000","last_peered":"2023-12-26T21:56:29.293176+0000","last_clean":"2023-12-26T21:56:29.293176+0000","last_became_active":"2023-12-26T21:56:28.331435+0000","last_became_peered":"2023-12-26T21:56:28.331435+0000","last_unstale":"2023-12-26T21:56:29.293176+0000","last_undegraded":"2023-12-26T21:56:29.293176+0000","last_fullsized":"2023-12-26T21:56:29.293176+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:46:15.614750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.297762+0000","last_change":"2023-12-26T21:56:29.295685+0000","last_active":"2023-12-26T21:56:29.297762+0000","last_peered":"2023-12-26T21:56:29.297762+0000","last_clean":"2023-12-26T21:56:29.297762+0000","last_became_active":"2023-12-26T21:56:29.295434+0000","last_became_peered":"2023-12-26T21:56:29.295434+0000","last_unstale":"2023-12-26T21:56:29.297762+0000","last_undegraded":"2023-12-26T21:56:29.297762+0000","last_fullsized":"2023-12-26T21:56:29.297762+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:44:46.461518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.280736+0000","last_change":"2023-12-26T21:56:28.326271+0000","last_active":"2023-12-26T21:56:29.280736+0000","last_peered":"2023-12-26T21:56:29.280736+0000","last_clean":"2023-12-26T21:56:29.280736+0000","last_became_active":"2023-12-26T21:56:28.325985+0000","last_became_peered":"2023-12-26T21:56:28.325985+0000","last_unstale":"2023-12-26T21:56:29.280736+0000","last_undegraded":"2023-12-26T21:56:29.280736+0000","last_fullsized":"2023-12-26T21:56:29.280736+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:22.509771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303870+0000","last_change":"2023-12-26T21:56:29.293095+0000","last_active":"2023-12-26T21:56:29.303870+0000","last_peered":"2023-12-26T21:56:29.303870+0000","last_clean":"2023-12-26T21:56:29.303870+0000","last_became_active":"2023-12-26T21:56:29.292238+0000","last_became_peered":"2023-12-26T21:56:29.292238+0000","last_unstale":"2023-12-26T21:56:29.303870+0000","last_undegraded":"2023-12-26T21:56:29.303870+0000","last_fullsized":"2023-12-26T21:56:29.303870+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:23.516130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.281117+0000","last_change":"2023-12-26T21:56:28.329073+0000","last_active":"2023-12-26T21:56:29.281117+0000","last_peered":"2023-12-26T21:56:29.281117+0000","last_clean":"2023-12-26T21:56:29.281117+0000","last_became_active":"2023-12-26T21:56:28.328734+0000","last_became_peered":"2023-12-26T21:56:28.328734+0000","last_unstale":"2023-12-26T21:56:29.281117+0000","last_undegraded":"2023-12-26T21:56:29.281117+0000","last_fullsized":"2023-12-26T21:56:29.281117+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:02.051217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303375+0000","last_change":"2023-12-26T21:56:29.300660+0000","last_active":"2023-12-26T21:56:29.303375+0000","last_peered":"2023-12-26T21:56:29.303375+0000","last_clean":"2023-12-26T21:56:29.303375+0000","last_became_active":"2023-12-26T21:56:29.299380+0000","last_became_peered":"2023-12-26T21:56:29.299380+0000","last_unstale":"2023-12-26T21:56:29.303375+0000","last_undegraded":"2023-12-26T21:56:29.303375+0000","last_fullsized":"2023-12-26T21:56:29.303375+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:00:29.719927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.332090+0000","last_change":"2023-12-26T21:56:28.327417+0000","last_active":"2023-12-26T21:56:29.332090+0000","last_peered":"2023-12-26T21:56:29.332090+0000","last_clean":"2023-12-26T21:56:29.332090+0000","last_became_active":"2023-12-26T21:56:28.327124+0000","last_became_peered":"2023-12-26T21:56:28.327124+0000","last_unstale":"2023-12-26T21:56:29.332090+0000","last_undegraded":"2023-12-26T21:56:29.332090+0000","last_fullsized":"2023-12-26T21:56:29.332090+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:20:29.860954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.299715+0000","last_change":"2023-12-26T21:56:29.295969+0000","last_active":"2023-12-26T21:56:29.299715+0000","last_peered":"2023-12-26T21:56:29.299715+0000","last_clean":"2023-12-26T21:56:29.299715+0000","last_became_active":"2023-12-26T21:56:29.295700+0000","last_became_peered":"2023-12-26T21:56:29.295700+0000","last_unstale":"2023-12-26T21:56:29.299715+0000","last_undegraded":"2023-12-26T21:56:29.299715+0000","last_fullsized":"2023-12-26T21:56:29.299715+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:10:16.043412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.279332+0000","last_change":"2023-12-26T21:56:28.338142+0000","last_active":"2023-12-26T21:56:29.279332+0000","last_peered":"2023-12-26T21:56:29.279332+0000","last_clean":"2023-12-26T21:56:29.279332+0000","last_became_active":"2023-12-26T21:56:28.337880+0000","last_became_peered":"2023-12-26T21:56:28.337880+0000","last_unstale":"2023-12-26T21:56:29.279332+0000","last_undegraded":"2023-12-26T21:56:29.279332+0000","last_fullsized":"2023-12-26T21:56:29.279332+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:24:37.620686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.309940+0000","last_change":"2023-12-26T21:56:29.307621+0000","last_active":"2023-12-26T21:56:29.309940+0000","last_peered":"2023-12-26T21:56:29.309940+0000","last_clean":"2023-12-26T21:56:29.309940+0000","last_became_active":"2023-12-26T21:56:29.307276+0000","last_became_peered":"2023-12-26T21:56:29.307276+0000","last_unstale":"2023-12-26T21:56:29.309940+0000","last_undegraded":"2023-12-26T21:56:29.309940+0000","last_fullsized":"2023-12-26T21:56:29.309940+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:56:47.190533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.279888+0000","last_change":"2023-12-26T21:56:28.329399+0000","last_active":"2023-12-26T21:56:29.279888+0000","last_peered":"2023-12-26T21:56:29.279888+0000","last_clean":"2023-12-26T21:56:29.279888+0000","last_became_active":"2023-12-26T21:56:28.329153+0000","last_became_peered":"2023-12-26T21:56:28.329153+0000","last_unstale":"2023-12-26T21:56:29.279888+0000","last_undegraded":"2023-12-26T21:56:29.279888+0000","last_fullsized":"2023-12-26T21:56:29.279888+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:33:42.903245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.303699+0000","last_change":"2023-12-26T21:56:29.300461+0000","last_active":"2023-12-26T21:56:29.303699+0000","last_peered":"2023-12-26T21:56:29.303699+0000","last_clean":"2023-12-26T21:56:29.303699+0000","last_became_active":"2023-12-26T21:56:29.297571+0000","last_became_peered":"2023-12-26T21:56:29.297571+0000","last_unstale":"2023-12-26T21:56:29.303699+0000","last_undegraded":"2023-12-26T21:56:29.303699+0000","last_fullsized":"2023-12-26T21:56:29.303699+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:15:53.820020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.280516+0000","last_change":"2023-12-26T21:56:28.339259+0000","last_active":"2023-12-26T21:56:29.280516+0000","last_peered":"2023-12-26T21:56:29.280516+0000","last_clean":"2023-12-26T21:56:29.280516+0000","last_became_active":"2023-12-26T21:56:28.338970+0000","last_became_peered":"2023-12-26T21:56:28.338970+0000","last_unstale":"2023-12-26T21:56:29.280516+0000","last_undegraded":"2023-12-26T21:56:29.280516+0000","last_fullsized":"2023-12-26T21:56:29.280516+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:13:11.379840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.300500+0000","last_change":"2023-12-26T21:56:29.297105+0000","last_active":"2023-12-26T21:56:29.300500+0000","last_peered":"2023-12-26T21:56:29.300500+0000","last_clean":"2023-12-26T21:56:29.300500+0000","last_became_active":"2023-12-26T21:56:29.296755+0000","last_became_peered":"2023-12-26T21:56:29.296755+0000","last_unstale":"2023-12-26T21:56:29.300500+0000","last_undegraded":"2023-12-26T21:56:29.300500+0000","last_fullsized":"2023-12-26T21:56:29.300500+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:26:11.061541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.278931+0000","last_change":"2023-12-26T21:56:28.331641+0000","last_active":"2023-12-26T21:56:29.278931+0000","last_peered":"2023-12-26T21:56:29.278931+0000","last_clean":"2023-12-26T21:56:29.278931+0000","last_became_active":"2023-12-26T21:56:28.331163+0000","last_became_peered":"2023-12-26T21:56:28.331163+0000","last_unstale":"2023-12-26T21:56:29.278931+0000","last_undegraded":"2023-12-26T21:56:29.278931+0000","last_fullsized":"2023-12-26T21:56:29.278931+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:33:40.215631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.305962+0000","last_change":"2023-12-26T21:56:29.302255+0000","last_active":"2023-12-26T21:56:29.305962+0000","last_peered":"2023-12-26T21:56:29.305962+0000","last_clean":"2023-12-26T21:56:29.305962+0000","last_became_active":"2023-12-26T21:56:29.299701+0000","last_became_peered":"2023-12-26T21:56:29.299701+0000","last_unstale":"2023-12-26T21:56:29.305962+0000","last_undegraded":"2023-12-26T21:56:29.305962+0000","last_fullsized":"2023-12-26T21:56:29.305962+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:09:59.751369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.283315+0000","last_change":"2023-12-26T21:56:28.332448+0000","last_active":"2023-12-26T21:56:29.283315+0000","last_peered":"2023-12-26T21:56:29.283315+0000","last_clean":"2023-12-26T21:56:29.283315+0000","last_became_active":"2023-12-26T21:56:28.331754+0000","last_became_peered":"2023-12-26T21:56:28.331754+0000","last_unstale":"2023-12-26T21:56:29.283315+0000","last_undegraded":"2023-12-26T21:56:29.283315+0000","last_fullsized":"2023-12-26T21:56:29.283315+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:56:46.184174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.297744+0000","last_change":"2023-12-26T21:56:29.294983+0000","last_active":"2023-12-26T21:56:29.297744+0000","last_peered":"2023-12-26T21:56:29.297744+0000","last_clean":"2023-12-26T21:56:29.297744+0000","last_became_active":"2023-12-26T21:56:29.294639+0000","last_became_peered":"2023-12-26T21:56:29.294639+0000","last_unstale":"2023-12-26T21:56:29.297744+0000","last_undegraded":"2023-12-26T21:56:29.297744+0000","last_fullsized":"2023-12-26T21:56:29.297744+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:50:48.115508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.294324+0000","last_change":"2023-12-26T21:56:28.323720+0000","last_active":"2023-12-26T21:56:29.294324+0000","last_peered":"2023-12-26T21:56:29.294324+0000","last_clean":"2023-12-26T21:56:29.294324+0000","last_became_active":"2023-12-26T21:56:28.323148+0000","last_became_peered":"2023-12-26T21:56:28.323148+0000","last_unstale":"2023-12-26T21:56:29.294324+0000","last_undegraded":"2023-12-26T21:56:29.294324+0000","last_fullsized":"2023-12-26T21:56:29.294324+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:22.069346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.308242+0000","last_change":"2023-12-26T21:56:29.301983+0000","last_active":"2023-12-26T21:56:29.308242+0000","last_peered":"2023-12-26T21:56:29.308242+0000","last_clean":"2023-12-26T21:56:29.308242+0000","last_became_active":"2023-12-26T21:56:29.301689+0000","last_became_peered":"2023-12-26T21:56:29.301689+0000","last_unstale":"2023-12-26T21:56:29.308242+0000","last_undegraded":"2023-12-26T21:56:29.308242+0000","last_fullsized":"2023-12-26T21:56:29.308242+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:47:05.820173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.285032+0000","last_change":"2023-12-26T21:56:28.338737+0000","last_active":"2023-12-26T21:56:29.285032+0000","last_peered":"2023-12-26T21:56:29.285032+0000","last_clean":"2023-12-26T21:56:29.285032+0000","last_became_active":"2023-12-26T21:56:28.338068+0000","last_became_peered":"2023-12-26T21:56:28.338068+0000","last_unstale":"2023-12-26T21:56:29.285032+0000","last_undegraded":"2023-12-26T21:56:29.285032+0000","last_fullsized":"2023-12-26T21:56:29.285032+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:59:23.706823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.297188+0000","last_change":"2023-12-26T21:56:29.287118+0000","last_active":"2023-12-26T21:56:29.297188+0000","last_peered":"2023-12-26T21:56:29.297188+0000","last_clean":"2023-12-26T21:56:29.297188+0000","last_became_active":"2023-12-26T21:56:29.286800+0000","last_became_peered":"2023-12-26T21:56:29.286800+0000","last_unstale":"2023-12-26T21:56:29.297188+0000","last_undegraded":"2023-12-26T21:56:29.297188+0000","last_fullsized":"2023-12-26T21:56:29.297188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:28.299844+0000","last_clean_scrub_stamp":"2023-12-26T21:56:28.299844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:26.376471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.283891+0000","last_change":"2023-12-26T21:56:28.333716+0000","last_active":"2023-12-26T21:56:29.283891+0000","last_peered":"2023-12-26T21:56:29.283891+0000","last_clean":"2023-12-26T21:56:29.283891+0000","last_became_active":"2023-12-26T21:56:28.333433+0000","last_became_peered":"2023-12-26T21:56:28.333433+0000","last_unstale":"2023-12-26T21:56:29.283891+0000","last_undegraded":"2023-12-26T21:56:29.283891+0000","last_fullsized":"2023-12-26T21:56:29.283891+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:56:27.293485+0000","last_clean_scrub_stamp":"2023-12-26T21:56:27.293485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:34:49.238406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"128'249","reported_seq":737,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-26T21:56:29.288817+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T21:56:29.288817+0000","last_peered":"2023-12-26T21:56:29.288817+0000","last_clean":"2023-12-26T21:56:29.288817+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T21:56:29.288817+0000","last_undegraded":"2023-12-26T21:56:29.288817+0000","last_fullsized":"2023-12-26T21:56:29.288817+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303080,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068760,"statfs":{"total":96636764160,"available":96326410240,"internally_reserved":0,"allocated":1736704,"data_stored":845769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66000000000000003,"15min":0.69499999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.96099999999999997,"15min":1.0780000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.66500000000000004},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39300000000000002},"max":{"1min":0.89100000000000001,"5min":0.94099999999999995,"15min":0.98299999999999998},"last":0.59099999999999997}]},{"osd":1,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64200000000000002,"15min":0.67200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80000000000000004,"5min":0.85899999999999999,"15min":1},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67600000000000005,"15min":0.70399999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.34200000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71699999999999997}]},{"osd":2,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.71499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.434},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.629},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66200000000000003,"15min":0.68400000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.434},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.55800000000000005}]},{"osd":3,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.82799999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91400000000000003,"5min":1.024,"15min":1.0569999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.311,"5min":0.309,"15min":0.309},"max":{"1min":0.81599999999999995,"5min":1.036,"15min":1.036},"last":0.77900000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.82499999999999996}]},{"osd":6,"last update":"Tue Dec 26 21:56:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94099999999999995,"5min":1.117,"15min":1.117},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.48899999999999999,"5min":0.45800000000000002,"15min":0.44},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.749}]}]},{"osd":6,"up_from":15,"seq":64424509673,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307156,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064684,"statfs":{"total":96636764160,"available":96322236416,"internally_reserved":0,"allocated":1716224,"data_stored":844109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66400000000000003,"15min":0.70099999999999996},"min":{"1min":0.49399999999999999,"5min":0.40600000000000003,"15min":0.35399999999999998},"max":{"1min":0.82199999999999995,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68899999999999995,"15min":0.70899999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":0.96499999999999997,"15min":0.98199999999999998},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65100000000000002,"15min":0.68799999999999994},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.879,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.40200000000000002},"max":{"1min":0.83499999999999996,"5min":0.95899999999999996,"15min":0.97599999999999998},"last":0.65700000000000003}]},{"osd":2,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66500000000000004,"15min":0.70299999999999996},"min":{"1min":0.48299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81899999999999995,"5min":0.93200000000000005,"15min":0.97099999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.52900000000000003,"5min":0.48399999999999999,"15min":0.46400000000000002},"max":{"1min":0.85099999999999998,"5min":0.93999999999999995,"15min":0.94699999999999995},"last":0.626}]},{"osd":3,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69199999999999995,"15min":0.71999999999999997},"min":{"1min":0.51900000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85399999999999998,"5min":0.92000000000000004,"15min":1.026},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71999999999999997,"15min":0.747},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.42799999999999999},"max":{"1min":0.83799999999999997,"5min":0.96099999999999997,"15min":1.0129999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.84299999999999997,"5min":0.91200000000000003,"15min":1.0529999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.97299999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":1.2609999999999999},"last":0.77500000000000002}]},{"osd":7,"last update":"Tue Dec 26 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72199999999999998,"15min":0.70799999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.438},"max":{"1min":0.89800000000000002,"5min":1.008,"15min":1.0920000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.97399999999999998,"15min":1.0920000000000001},"last":0.753}]}]},{"osd":1,"up_from":15,"seq":64424509673,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311832,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060008,"statfs":{"total":96636764160,"available":96317448192,"internally_reserved":0,"allocated":2310144,"data_stored":1434477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46100000000000002,"15min":0.45200000000000001},"min":{"1min":0.29599999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.872,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.5},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48299999999999998,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.58399999999999996,"5min":1.353,"15min":1.353},"last":0.38}]},{"osd":2,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.47999999999999998},"min":{"1min":0.30299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78000000000000003,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.40300000000000002,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.93400000000000005,"5min":1.488,"15min":1.488},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56599999999999995,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35799999999999998},"max":{"1min":0.97299999999999998,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.73799999999999999,"5min":1.506,"15min":1.506},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70199999999999996,"15min":0.73899999999999999},"min":{"1min":0.51600000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88300000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.76400000000000001,"15min":0.752},"min":{"1min":0.52400000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1319999999999999,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":0.57699999999999996}]},{"osd":5,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.54000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.1020000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75800000000000001,"15min":0.79800000000000004},"min":{"1min":0.58499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.119,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.746}]},{"osd":6,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76300000000000001,"15min":0.79800000000000004},"min":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.48399999999999999},"max":{"1min":1.012,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78200000000000003,"15min":0.80300000000000005},"min":{"1min":0.59099999999999997,"5min":0.56499999999999995,"15min":0.54400000000000004},"max":{"1min":1.1000000000000001,"5min":1.615,"15min":1.615},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.753,"15min":0.77400000000000002},"min":{"1min":0.53600000000000003,"5min":0.48999999999999999,"15min":0.46400000000000002},"max":{"1min":1.0189999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.54900000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91700000000000004,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.78600000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509673,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303068,"kb_used_data":1684,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068772,"statfs":{"total":96636764160,"available":96326422528,"internally_reserved":0,"allocated":1724416,"data_stored":844733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.245},"max":{"1min":0.63500000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44900000000000001,"15min":0.47499999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":0.69699999999999995,"5min":0.69999999999999996,"15min":0.80000000000000004},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45800000000000002,"15min":0.48499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.73099999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.497,"15min":0.51500000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.74399999999999999,"15min":0.76200000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52300000000000002,"15min":0.53900000000000003},"min":{"1min":0.309,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66800000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58599999999999997,"15min":0.60399999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71999999999999997,"5min":0.86499999999999999,"15min":0.86699999999999999},"last":0.47999999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70899999999999996,"15min":0.74099999999999999},"min":{"1min":0.51800000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93400000000000005,"5min":1.226,"15min":1.2749999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72799999999999998,"15min":0.75900000000000001},"min":{"1min":0.53800000000000003,"5min":0.47899999999999998,"15min":0.47699999999999998},"max":{"1min":0.90100000000000002,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76700000000000002,"15min":0.79200000000000004},"min":{"1min":0.52400000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.002,"5min":1.1339999999999999,"15min":1.153},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.80500000000000005},"min":{"1min":0.53400000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0489999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.77200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.45300000000000001},"max":{"1min":0.96299999999999997,"5min":1.093,"15min":1.214},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.78200000000000003},"min":{"1min":0.52700000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.95299999999999996,"5min":1.169,"15min":1.169},"last":0.627}]},{"osd":7,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.79200000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.036,"5min":1.252,"15min":1.252},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.749,"15min":0.78500000000000003},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.016,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.55800000000000005}]}]},{"osd":2,"up_from":14,"seq":60129542378,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307168,"kb_used_data":1688,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064672,"statfs":{"total":96636764160,"available":96322224128,"internally_reserved":0,"allocated":1728512,"data_stored":844719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44800000000000001,"15min":0.45100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.24299999999999999},"max":{"1min":0.58399999999999996,"5min":0.70399999999999996,"15min":0.85999999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.50900000000000001,"15min":0.52200000000000002},"min":{"1min":0.36499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.83899999999999997},"last":0.442}]},{"osd":1,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.35199999999999998,"5min":0.317,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.74199999999999999,"5min":1.056,"15min":1.056},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58999999999999997,"15min":0.59999999999999998},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83599999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83399999999999996,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.371}]},{"osd":4,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.751,"15min":0.76200000000000001},"min":{"1min":0.55100000000000005,"5min":0.54600000000000004,"15min":0.38300000000000001},"max":{"1min":0.94499999999999995,"5min":1.3,"15min":1.3},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75,"15min":0.76400000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95799999999999996,"5min":1.048,"15min":1.1759999999999999},"last":0.64600000000000002}]},{"osd":5,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.75,"15min":0.76400000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.502},"max":{"1min":0.89500000000000002,"5min":1.25,"15min":1.25},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":0.98799999999999999,"5min":1.002,"15min":1.1319999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.50700000000000001},"max":{"1min":0.93000000000000005,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.61199999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.0009999999999999,"5min":1.202,"15min":1.272},"last":0.67000000000000004}]},{"osd":7,"last update":"Tue Dec 26 21:56:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76200000000000001,"15min":0.77500000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":1.1060000000000001,"15min":1.1220000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79400000000000004,"15min":0.81299999999999994},"min":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.54300000000000004},"max":{"1min":1.0329999999999999,"5min":1.232,"15min":1.232},"last":0.747}]}]},{"osd":3,"up_from":15,"seq":64424509674,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311252,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060588,"statfs":{"total":96636764160,"available":96318042112,"internally_reserved":0,"allocated":1716224,"data_stored":844109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.246,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.315},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.52200000000000002,"15min":0.495},"min":{"1min":0.32000000000000001,"5min":0.316,"15min":0.30299999999999999},"max":{"1min":3.7839999999999998,"5min":3.7839999999999998,"15min":3.7839999999999998},"last":0.33400000000000002}]},{"osd":1,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.89400000000000002,"5min":1.468,"15min":1.468},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55600000000000005,"15min":0.53500000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.3940000000000001,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.377},"max":{"1min":1.4419999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.66400000000000003,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.42199999999999999},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.48299999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.754,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.437,"15min":0.437},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.77700000000000002,"15min":0.76800000000000002},"min":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.45900000000000002},"max":{"1min":5.7229999999999999,"5min":5.7229999999999999,"15min":5.7229999999999999},"last":0.79200000000000004}]},{"osd":5,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.80600000000000005,"15min":0.78500000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51500000000000001},"max":{"1min":5.6799999999999997,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57999999999999996,"5min":0.52600000000000002,"15min":0.51700000000000002},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.81000000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.81100000000000005,"15min":0.80400000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46000000000000002},"max":{"1min":3.8250000000000002,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.55700000000000005},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.82999999999999996}]},{"osd":7,"last update":"Tue Dec 26 21:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":3.3420000000000001,"5min":3.3420000000000001,"15min":3.3420000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.81699999999999995,"15min":0.80800000000000005},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":0.75600000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542377,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307172,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064668,"statfs":{"total":96636764160,"available":96322220032,"internally_reserved":0,"allocated":1732608,"data_stored":845735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.44800000000000001,"5min":0.439,"15min":0.307},"max":{"1min":0.89800000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68700000000000006,"15min":0.70199999999999996},"min":{"1min":0.44,"5min":0.436,"15min":0.371},"max":{"1min":0.91400000000000003,"5min":1.006,"15min":1.008},"last":0.61099999999999999}]},{"osd":1,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.311},"max":{"1min":0.96499999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.40000000000000002},"max":{"1min":0.93700000000000006,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82199999999999995}]},{"osd":2,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.51200000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.878},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68300000000000005,"15min":0.69599999999999995},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.38800000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63300000000000001}]},{"osd":3,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70799999999999996,"15min":0.71199999999999997},"min":{"1min":0.57399999999999995,"5min":0.496,"15min":0.45300000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.97599999999999998},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.442,"15min":0.438},"max":{"1min":0.91400000000000003,"5min":1.046,"15min":1.046},"last":0.78000000000000003}]},{"osd":5,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.432,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84999999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64200000000000002,"15min":0.61499999999999999},"min":{"1min":0.432,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85299999999999998,"5min":0.91700000000000004,"15min":0.92600000000000005},"last":0.92200000000000004}]},{"osd":6,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.67300000000000004},"min":{"1min":0.50900000000000001,"5min":0.44600000000000001,"15min":0.438},"max":{"1min":0.91300000000000003,"5min":1.133,"15min":1.133},"last":0.93700000000000006}]},{"osd":7,"last update":"Tue Dec 26 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.45800000000000002},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.002},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.61399999999999999,"5min":0.54300000000000004,"15min":0.48499999999999999},"max":{"1min":0.878,"5min":1.266,"15min":1.266},"last":0.96199999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509672,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303636,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068204,"statfs":{"total":96636764160,"available":96325840896,"internally_reserved":0,"allocated":2306048,"data_stored":1427861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.68100000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.27300000000000002},"max":{"1min":0.83599999999999997,"5min":0.89800000000000002,"15min":0.995},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66900000000000004,"15min":0.68700000000000006},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.32400000000000001},"max":{"1min":0.89900000000000002,"5min":0.91600000000000004,"15min":1.0369999999999999},"last":0.90700000000000003}]},{"osd":1,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.439,"15min":0.34000000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.04},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66500000000000004,"15min":0.69599999999999995},"min":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.36899999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.83499999999999996}]},{"osd":2,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68200000000000005,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.3029999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.68100000000000005,"15min":0.69999999999999996},"min":{"1min":0.46999999999999997,"5min":0.45800000000000002,"15min":0.38800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.72599999999999998}]},{"osd":3,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.54800000000000004,"5min":0.504,"15min":0.39200000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.0640000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69799999999999995,"15min":0.71999999999999997},"min":{"1min":0.499,"5min":0.49099999999999999,"15min":0.47799999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93899999999999995},"last":0.85199999999999998}]},{"osd":4,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.34000000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.48899999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.92200000000000004,"5min":1.01,"15min":1.01},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.40600000000000003,"5min":0.35699999999999998,"15min":0.35499999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.496,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Dec 26 21:56:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.68300000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.012,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T21:56:34.152 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T21:56:34.152 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T21:56:34.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.383+0000 7f2535dff700 1 Processor -- start 2023-12-26T21:56:34.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.383+0000 7f2535dff700 1 -- start start 2023-12-26T21:56:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.384+0000 7f2535dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300adf60 0x7f25300cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.384+0000 7f2535dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25300c6b30 con 0x7f25300ac4b0 2023-12-26T21:56:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.384+0000 7f2535dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25300c6e10 con 0x7f25300ac040 2023-12-26T21:56:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.384+0000 7f2535dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25301561e0 con 0x7f25300adf60 2023-12-26T21:56:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.385+0000 7f252ffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f25300ac4b0 0x7f2530069620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48958/0 (socket says 172.21.15.131:48958) 2023-12-26T21:56:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.385+0000 7f252ffff700 1 -- 172.21.15.131:0/2229474252 learned_addr learned my addr 172.21.15.131:0/2229474252 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.385+0000 7f252f7fe700 1 --2- 172.21.15.131:0/2229474252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300adf60 0x7f25300cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.385+0000 7f252effd700 1 -- 172.21.15.131:0/2229474252 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580593071 0 0) 0x7f25300c6b30 con 0x7f25300ac4b0 2023-12-26T21:56:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.385+0000 7f252effd700 1 -- 172.21.15.131:0/2229474252 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f251c003570 con 0x7f25300ac4b0 2023-12-26T21:56:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.385+0000 7f252f7fe700 1 -- 172.21.15.131:0/2229474252 >> v1:172.21.15.190:6789/0 conn(0x7f25300ac040 legacy=0x7f253006e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.385+0000 7f252effd700 1 -- 172.21.15.131:0/2229474252 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166235336 0 0) 0x7f25300c6e10 con 0x7f25300ac040 2023-12-26T21:56:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.385+0000 7f252f7fe700 1 -- 172.21.15.131:0/2229474252 >> v1:172.21.15.131:6789/0 conn(0x7f25300ac4b0 legacy=0x7f2530069620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.385+0000 7f252f7fe700 1 -- 172.21.15.131:0/2229474252 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25300c70f0 con 0x7f25300adf60 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.386+0000 7f252f7fe700 1 --2- 172.21.15.131:0/2229474252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300adf60 0x7f25300cc8c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2520009910 tx=0x7f25200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=441a4d94a54eb6e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.386+0000 7f252effd700 1 -- 172.21.15.131:0/2229474252 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252000b680 con 0x7f25300adf60 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.386+0000 7f252effd700 1 -- 172.21.15.131:0/2229474252 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252000b7e0 con 0x7f25300adf60 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.386+0000 7f252effd700 1 -- 172.21.15.131:0/2229474252 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252000bba0 con 0x7f25300adf60 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.386+0000 7f2535dff700 1 -- 172.21.15.131:0/2229474252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300adf60 msgr2=0x7f25300cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.386+0000 7f2535dff700 1 --2- 172.21.15.131:0/2229474252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300adf60 0x7f25300cc8c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2520009910 tx=0x7f25200091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.387+0000 7f2535dff700 1 -- 172.21.15.131:0/2229474252 shutdown_connections 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.387+0000 7f2535dff700 1 --2- 172.21.15.131:0/2229474252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300adf60 0x7f25300cc8c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.387+0000 7f2535dff700 1 -- 172.21.15.131:0/2229474252 >> 172.21.15.131:0/2229474252 conn(0x7f2530068040 msgr2=0x7f25300cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.387+0000 7f2535dff700 1 -- 172.21.15.131:0/2229474252 shutdown_connections 2023-12-26T21:56:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.387+0000 7f2535dff700 1 -- 172.21.15.131:0/2229474252 wait complete. 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.388+0000 7f2535dff700 1 Processor -- start 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.388+0000 7f2535dff700 1 -- start start 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.389+0000 7f2535dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25300ac040 0x7f25300a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.389+0000 7f2535dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300ac4b0 0x7f25300c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.389+0000 7f2535dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25300adf60 0x7f25300c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.389+0000 7f2535dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2530148100 con 0x7f25300ac040 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.389+0000 7f2535dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25300a9fa0 con 0x7f25300adf60 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.389+0000 7f2535dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25300aa280 con 0x7f25300ac4b0 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.389+0000 7f252f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25300ac040 0x7f25300a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.389+0000 7f252f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25300ac040 0x7f25300a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53206/0 (socket says 172.21.15.131:53206) 2023-12-26T21:56:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.389+0000 7f252f7fe700 1 -- 172.21.15.131:0/469769463 learned_addr learned my addr 172.21.15.131:0/469769463 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f252f7fe700 1 -- 172.21.15.131:0/469769463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300ac4b0 msgr2=0x7f25300c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f252f7fe700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300ac4b0 0x7f25300c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f252f7fe700 1 -- 172.21.15.131:0/469769463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25300adf60 msgr2=0x7f25300c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f252f7fe700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25300adf60 0x7f25300c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f252f7fe700 1 -- 172.21.15.131:0/469769463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520009420 con 0x7f25300ac040 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f252f7fe700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25300ac040 0x7f25300a98c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2530064a60 tx=0x7f2520017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f25277fe700 1 -- 172.21.15.131:0/469769463 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252000bb60 con 0x7f25300ac040 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f25277fe700 1 -- 172.21.15.131:0/469769463 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2520027450 con 0x7f25300ac040 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25300aa560 con 0x7f25300ac040 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f25277fe700 1 -- 172.21.15.131:0/469769463 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520027810 con 0x7f25300ac040 2023-12-26T21:56:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.390+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25300aab20 con 0x7f25300ac040 2023-12-26T21:56:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.391+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25300657e0 con 0x7f25300ac040 2023-12-26T21:56:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.392+0000 7f25277fe700 1 -- 172.21.15.131:0/469769463 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2520027b80 con 0x7f25300ac040 2023-12-26T21:56:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.393+0000 7f25277fe700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2510066f30 0x7f25100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.393+0000 7f25277fe700 1 -- 172.21.15.131:0/469769463 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f2520086740 con 0x7f25300ac040 2023-12-26T21:56:34.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.393+0000 7f2527fff700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2510066f30 0x7f25100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:34.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.394+0000 7f2527fff700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2510066f30 0x7f25100693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2530154fe0 tx=0x7f2518007b40 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:34.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.398+0000 7f25277fe700 1 -- 172.21.15.131:0/469769463 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2520053770 con 0x7f25300ac040 2023-12-26T21:56:34.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.669+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f25300aad10 con 0x7f25300ac040 2023-12-26T21:56:34.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.675+0000 7f25277fe700 1 -- 172.21.15.131:0/469769463 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2520056d90 con 0x7f25300ac040 2023-12-26T21:56:34.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.677+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2510066f30 msgr2=0x7f25100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.677+0000 7f2535dff700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2510066f30 0x7f25100693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2530154fe0 tx=0x7f2518007b40 comp rx=0 tx=0).stop 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.677+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25300ac040 msgr2=0x7f25300a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.677+0000 7f2535dff700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25300ac040 0x7f25300a98c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2530064a60 tx=0x7f2520017000 comp rx=0 tx=0).stop 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.678+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 shutdown_connections 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.678+0000 7f2535dff700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25300adf60 0x7f25300c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.678+0000 7f2535dff700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2510066f30 0x7f25100693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.678+0000 7f2535dff700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25300ac040 0x7f25300a98c0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.678+0000 7f2535dff700 1 --2- 172.21.15.131:0/469769463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25300ac4b0 0x7f25300c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.678+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 >> 172.21.15.131:0/469769463 conn(0x7f2530068040 msgr2=0x7f25300aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.678+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 shutdown_connections 2023-12-26T21:56:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.678+0000 7f2535dff700 1 -- 172.21.15.131:0/469769463 wait complete. 2023-12-26T21:56:34.679 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T21:56:34.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T21:56:34.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.886+0000 7fb2d7fff700 1 Processor -- start 2023-12-26T21:56:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.886+0000 7fb2d7fff700 1 -- start start 2023-12-26T21:56:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.887+0000 7fb2d7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb2d80b6020 0x7fb2d815a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.887+0000 7fb2d7fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb2d815af90 0x7fb2d80b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.887+0000 7fb2d7fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d815c0a0 con 0x7fb2d80b4100 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.887+0000 7fb2d7fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d80b63f0 con 0x7fb2d80b6020 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.887+0000 7fb2d7fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d81598e0 con 0x7fb2d815af90 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.887+0000 7fb2d6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb2d80b6020 0x7fb2d815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.888+0000 7fb2d6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb2d80b6020 0x7fb2d815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53202/0 (socket says 172.21.15.131:53202) 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.888+0000 7fb2d6ffd700 1 -- 172.21.15.131:0/2188702145 learned_addr learned my addr 172.21.15.131:0/2188702145 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.888+0000 7fb2d67fc700 1 --2- 172.21.15.131:0/2188702145 >> v2:172.21.15.190:3300/0 conn(0x7fb2d815af90 0x7fb2d80b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.888+0000 7fb2d6ffd700 1 -- 172.21.15.131:0/2188702145 >> v2:172.21.15.190:3300/0 conn(0x7fb2d815af90 msgr2=0x7fb2d80b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.888+0000 7fb2d6ffd700 1 --2- 172.21.15.131:0/2188702145 >> v2:172.21.15.190:3300/0 conn(0x7fb2d815af90 0x7fb2d80b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.888+0000 7fb2d6ffd700 1 -- 172.21.15.131:0/2188702145 >> v1:172.21.15.190:6789/0 conn(0x7fb2d80b4100 legacy=0x7fb2d815c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.888+0000 7fb2d6ffd700 1 -- 172.21.15.131:0/2188702145 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d815b490 con 0x7fb2d80b6020 2023-12-26T21:56:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.888+0000 7fb2d6ffd700 1 --2- 172.21.15.131:0/2188702145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 0x7fb2d815a630 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80d1540 tx=0x7fb2c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3178a26a07fbe6af server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.888+0000 7fb2d5ffb700 1 -- 172.21.15.131:0/2188702145 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800b860 con 0x7fb2d80b6020 2023-12-26T21:56:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d5ffb700 1 -- 172.21.15.131:0/2188702145 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c800b9c0 con 0x7fb2d80b6020 2023-12-26T21:56:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d5ffb700 1 -- 172.21.15.131:0/2188702145 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800bd60 con 0x7fb2d80b6020 2023-12-26T21:56:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2188702145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 msgr2=0x7fb2d815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d7fff700 1 --2- 172.21.15.131:0/2188702145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 0x7fb2d815a630 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80d1540 tx=0x7fb2c8009250 comp rx=0 tx=0).stop 2023-12-26T21:56:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2188702145 shutdown_connections 2023-12-26T21:56:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d7fff700 1 --2- 172.21.15.131:0/2188702145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 0x7fb2d815a630 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d7fff700 1 --2- 172.21.15.131:0/2188702145 >> v2:172.21.15.190:3300/0 conn(0x7fb2d815af90 0x7fb2d80b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2188702145 >> 172.21.15.131:0/2188702145 conn(0x7fb2d806ece0 msgr2=0x7fb2d806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2188702145 shutdown_connections 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.889+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2188702145 wait complete. 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.891+0000 7fb2d7fff700 1 Processor -- start 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.891+0000 7fb2d7fff700 1 -- start start 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d80b4100 0x7fb2d81583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 0x7fb2d815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d815af90 0x7fb2d80c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d7fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80c8f60 con 0x7fb2d80b6020 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d7fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d815f2f0 con 0x7fb2d80b4100 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d7fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d815e6e0 con 0x7fb2d815af90 2023-12-26T21:56:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d67fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 0x7fb2d815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d67fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 0x7fb2d815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53210/0 (socket says 172.21.15.131:53210) 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d67fc700 1 -- 172.21.15.131:0/2565962438 learned_addr learned my addr 172.21.15.131:0/2565962438 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d6ffd700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d80b4100 0x7fb2d81583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d77fe700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d815af90 0x7fb2d80c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d67fc700 1 -- 172.21.15.131:0/2565962438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d815af90 msgr2=0x7fb2d80c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d67fc700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d815af90 0x7fb2d80c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d67fc700 1 -- 172.21.15.131:0/2565962438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d80b4100 msgr2=0x7fb2d81583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d67fc700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d80b4100 0x7fb2d81583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d67fc700 1 -- 172.21.15.131:0/2565962438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80094f0 con 0x7fb2d80b6020 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.892+0000 7fb2d6ffd700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d80b4100 0x7fb2d81583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.893+0000 7fb2d67fc700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 0x7fb2d815f780 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00d860 tx=0x7fb2cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.893+0000 7fb2beffd700 1 -- 172.21.15.131:0/2565962438 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0125f0 con 0x7fb2d80b6020 2023-12-26T21:56:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.893+0000 7fb2beffd700 1 -- 172.21.15.131:0/2565962438 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc012750 con 0x7fb2d80b6020 2023-12-26T21:56:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.893+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d815e9c0 con 0x7fb2d80b6020 2023-12-26T21:56:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.893+0000 7fb2beffd700 1 -- 172.21.15.131:0/2565962438 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc012b10 con 0x7fb2d80b6020 2023-12-26T21:56:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.893+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d815ef80 con 0x7fb2d80b6020 2023-12-26T21:56:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.894+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d815f4a0 con 0x7fb2d80b6020 2023-12-26T21:56:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.895+0000 7fb2beffd700 1 -- 172.21.15.131:0/2565962438 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb2cc01e460 con 0x7fb2d80b6020 2023-12-26T21:56:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.896+0000 7fb2beffd700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c4066f30 0x7fb2c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.896+0000 7fb2beffd700 1 -- 172.21.15.131:0/2565962438 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fb2cc084c40 con 0x7fb2d80b6020 2023-12-26T21:56:34.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.896+0000 7fb2d6ffd700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c4066f30 0x7fb2c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:34.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.897+0000 7fb2d6ffd700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c4066f30 0x7fb2c40693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80d1540 tx=0x7fb2c8017000 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:34.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:34.900+0000 7fb2beffd700 1 -- 172.21.15.131:0/2565962438 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2d815f4a0 con 0x7fb2d80b6020 2023-12-26T21:56:35.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.135+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2d815f110 con 0x7fb2d80b6020 2023-12-26T21:56:35.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.135+0000 7fb2beffd700 1 -- 172.21.15.131:0/2565962438 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fb2cc051d10 con 0x7fb2d80b6020 2023-12-26T21:56:35.135 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:35.135 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3746500235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3746500235},{"type":"v1","addr":"172.21.15.190:6835","nonce":3746500235}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8115,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2030200373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2030200373},{"type":"v1","addr":"172.21.15.131:6833","nonce":2030200373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2894985738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2894985738},{"type":"v1","addr":"172.21.15.190:6834","nonce":2894985738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:56:28.792645+0000","modified":"2023-12-26T21:56:32.333009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8142":{"gid":8142,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1841321761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1841321761},{"type":"v1","addr":"172.21.15.131:6835","nonce":1841321761}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-12-26T21:56:35.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c4066f30 msgr2=0x7fb2c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c4066f30 0x7fb2c40693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80d1540 tx=0x7fb2c8017000 comp rx=0 tx=0).stop 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 msgr2=0x7fb2d815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 0x7fb2d815f780 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00d860 tx=0x7fb2cc008040 comp rx=0 tx=0).stop 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 shutdown_connections 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d815af90 0x7fb2d80c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d80b4100 0x7fb2d81583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c4066f30 0x7fb2c40693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 --2- 172.21.15.131:0/2565962438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d80b6020 0x7fb2d815f780 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 >> 172.21.15.131:0/2565962438 conn(0x7fb2d806ece0 msgr2=0x7fb2d80d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 shutdown_connections 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.138+0000 7fb2d7fff700 1 -- 172.21.15.131:0/2565962438 wait complete. 2023-12-26T21:56:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 66 2023-12-26T21:56:35.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T21:56:28.792645+0000', 'modified': '2023-12-26T21:56:32.333009+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8142}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8142': {'gid': 8142, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6835/1841321761', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1841321761}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1841321761}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T21:56:35.559 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T21:56:35.559 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-26T21:56:35.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.750+0000 7fadc83ff700 1 Processor -- start 2023-12-26T21:56:35.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.750+0000 7fadc83ff700 1 -- start start 2023-12-26T21:56:35.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.751+0000 7fadc83ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc00b0bf0 0x7fadc00c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.751+0000 7fadc83ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc0156940 con 0x7fadc00b0750 2023-12-26T21:56:35.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.751+0000 7fadc83ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc00ae500 con 0x7fadc00cb320 2023-12-26T21:56:35.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.751+0000 7fadc83ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00c8cc0 con 0x7fadc00b0bf0 2023-12-26T21:56:35.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc599a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fadc00b0750 0x7fadc00c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51596/0 (socket says 172.21.15.131:51596) 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc599a700 1 -- 172.21.15.131:0/980573539 learned_addr learned my addr 172.21.15.131:0/980573539 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc699c700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc00b0bf0 0x7fadc00c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200060911 0 0) 0x7fadc0156940 con 0x7fadc00b0750 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadac003570 con 0x7fadc00b0750 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2741246370 0 0) 0x7fadac003570 con 0x7fadc00b0750 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc00b0bf0 msgr2=0x7fadc00c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc00b0bf0 0x7fadc00c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 >> v1:172.21.15.190:6789/0 conn(0x7fadc00cb320 legacy=0x7fadc0069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00b0280 con 0x7fadc00b0750 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fadb0001470 con 0x7fadc00b0750 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 >> v1:172.21.15.131:6789/0 conn(0x7fadc00b0750 legacy=0x7fadc00c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.752+0000 7fadc5199700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc00b0bf0 0x7fadac003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc5199700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac004010 0x7fadac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc5199700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac004d50 0x7fadac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00c8cc0 con 0x7fadac004010 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac00bbf0 con 0x7fadac004d50 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac00dfc0 con 0x7fadc00b0bf0 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc599a700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac004010 0x7fadac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc619b700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc00b0bf0 0x7fadac003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc699c700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac004d50 0x7fadac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc599a700 1 -- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc00b0bf0 msgr2=0x7fadac003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc599a700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc00b0bf0 0x7fadac003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc599a700 1 -- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac004d50 msgr2=0x7fadac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc599a700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac004d50 0x7fadac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc599a700 1 -- 172.21.15.131:0/980573539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac003570 con 0x7fadac004010 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.753+0000 7fadc599a700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac004010 0x7fadac004810 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c390 tx=0x7fadb0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45309831ddeea060 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb000cc60 con 0x7fadac004010 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb000cdc0 con 0x7fadac004010 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc5199700 1 -- 172.21.15.131:0/980573539 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0005b70 con 0x7fadac004010 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc83ff700 1 -- 172.21.15.131:0/980573539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac004010 msgr2=0x7fadac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc83ff700 1 --2- 172.21.15.131:0/980573539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac004010 0x7fadac004810 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c390 tx=0x7fadb0001e50 comp rx=0 tx=0).stop 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc699c700 1 -- 172.21.15.131:0/980573539 reap_dead start 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc83ff700 1 -- 172.21.15.131:0/980573539 shutdown_connections 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc83ff700 1 -- 172.21.15.131:0/980573539 >> 172.21.15.131:0/980573539 conn(0x7fadc0068040 msgr2=0x7fadc0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc83ff700 1 -- 172.21.15.131:0/980573539 shutdown_connections 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.754+0000 7fadc83ff700 1 -- 172.21.15.131:0/980573539 wait complete. 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.756+0000 7fadc83ff700 1 Processor -- start 2023-12-26T21:56:35.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.756+0000 7fadc83ff700 1 -- start start 2023-12-26T21:56:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc83ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac004010 0x7fadc0166390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc83ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc01598c0 0x7fadc0157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc83ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc00a64d0 0x7fadc00a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc83ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00a54a0 con 0x7fadc00a64d0 2023-12-26T21:56:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc83ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc0159d10 con 0x7fadac004010 2023-12-26T21:56:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc83ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00a5710 con 0x7fadc01598c0 2023-12-26T21:56:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc00a64d0 0x7fadc00a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc00a64d0 0x7fadc00a7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53234/0 (socket says 172.21.15.131:53234) 2023-12-26T21:56:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc699c700 1 -- 172.21.15.131:0/3447147236 learned_addr learned my addr 172.21.15.131:0/3447147236 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc599a700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc01598c0 0x7fadc0157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc619b700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac004010 0x7fadc0166390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc699c700 1 -- 172.21.15.131:0/3447147236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc01598c0 msgr2=0x7fadc0157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc699c700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc01598c0 0x7fadc0157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc699c700 1 -- 172.21.15.131:0/3447147236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac004010 msgr2=0x7fadc0166390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc699c700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac004010 0x7fadc0166390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.757+0000 7fadc699c700 1 -- 172.21.15.131:0/3447147236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00a59f0 con 0x7fadc00a64d0 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.758+0000 7fadc699c700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc00a64d0 0x7fadc00a7740 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fadc0064440 tx=0x7fadb800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.758+0000 7fadb67fc700 1 -- 172.21.15.131:0/3447147236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb800e040 con 0x7fadc00a64d0 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.758+0000 7fadb67fc700 1 -- 172.21.15.131:0/3447147236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb800f040 con 0x7fadc00a64d0 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.758+0000 7fadb67fc700 1 -- 172.21.15.131:0/3447147236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb800fd70 con 0x7fadc00a64d0 2023-12-26T21:56:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.758+0000 7fadc83ff700 1 -- 172.21.15.131:0/3447147236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc00a5cd0 con 0x7fadc00a64d0 2023-12-26T21:56:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.758+0000 7fadc83ff700 1 -- 172.21.15.131:0/3447147236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc0158500 con 0x7fadc00a64d0 2023-12-26T21:56:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.760+0000 7fadb67fc700 1 -- 172.21.15.131:0/3447147236 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fadb8015640 con 0x7fadc00a64d0 2023-12-26T21:56:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.761+0000 7fadb67fc700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fada4066f60 0x7fada4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.761+0000 7fadb67fc700 1 -- 172.21.15.131:0/3447147236 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fadb8084e90 con 0x7fadc00a64d0 2023-12-26T21:56:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.761+0000 7fadc619b700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fada4066f60 0x7fada4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.761+0000 7fadc619b700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fada4066f60 0x7fada4069410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fadc0158a80 tx=0x7fadbc007d30 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.763+0000 7fadd6bf2b80 1 Processor -- start 2023-12-26T21:56:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.763+0000 7fadd6bf2b80 1 -- start start 2023-12-26T21:56:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.764+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 0x555b58890370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.764+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5888bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.764+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 0x555b588ac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.764+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b587be3e0 con 0x555b588908b0 2023-12-26T21:56:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.764+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b587bbbe0 con 0x555b5888c170 2023-12-26T21:56:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.764+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b587b79d0 con 0x555b5888ffa0 2023-12-26T21:56:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc599a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5888bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc599a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5888bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53240/0 (socket says 172.21.15.131:53240) 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc599a700 1 -- 172.21.15.131:0/1397143990 learned_addr learned my addr 172.21.15.131:0/1397143990 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc619b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 0x555b58890370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc699c700 1 --2- 172.21.15.131:0/1397143990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 0x555b588ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc599a700 1 -- 172.21.15.131:0/1397143990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 msgr2=0x555b58890370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc599a700 1 --2- 172.21.15.131:0/1397143990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 0x555b58890370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc599a700 1 -- 172.21.15.131:0/1397143990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 msgr2=0x555b588ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc599a700 1 --2- 172.21.15.131:0/1397143990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 0x555b588ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc599a700 1 -- 172.21.15.131:0/1397143990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b5864b360 con 0x555b588908b0 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fadc599a700 1 --2- 172.21.15.131:0/1397143990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5888bc30 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c0e0 tx=0x7fadb0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aec3ac1aa693d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fada37fe700 1 -- 172.21.15.131:0/1397143990 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0001b70 con 0x555b588908b0 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fada37fe700 1 -- 172.21.15.131:0/1397143990 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb0007ac0 con 0x555b588908b0 2023-12-26T21:56:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.765+0000 7fada37fe700 1 -- 172.21.15.131:0/1397143990 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0001430 con 0x555b588908b0 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.766+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/1397143990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 msgr2=0x555b5888bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.766+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/1397143990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5888bc30 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c0e0 tx=0x7fadb0015000 comp rx=0 tx=0).stop 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.766+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/1397143990 shutdown_connections 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.766+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/1397143990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 0x555b588ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.766+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/1397143990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 0x555b58890370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.766+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/1397143990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5888bc30 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.766+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/1397143990 >> 172.21.15.131:0/1397143990 conn(0x555b58866440 msgr2=0x555b588b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.766+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/1397143990 shutdown_connections 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.766+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/1397143990 wait complete. 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.767+0000 7fadd6bf2b80 1 -- start start 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.767+0000 7fadd6bf2b80 1 Processor -- start 2023-12-26T21:56:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.768+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 0x555b5888e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.768+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 0x555b5888a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.768+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.768+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b587641b0 con 0x555b588908b0 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.768+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b58763e50 con 0x555b5888ffa0 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.768+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b586b0e40 con 0x555b5888c170 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.768+0000 7fadd6bf2b80 10 client.0 _ll_register_callbacks cb 0x555b586b37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.768+0000 7fadc699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc599a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 0x555b5888a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53250/0 (socket says 172.21.15.131:53250) 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc619b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 0x555b5888e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc699c700 1 -- 172.21.15.131:0/441508982 learned_addr learned my addr 172.21.15.131:0/441508982 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc699c700 1 -- 172.21.15.131:0/441508982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 msgr2=0x555b5888e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc699c700 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 0x555b5888e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc699c700 1 -- 172.21.15.131:0/441508982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 msgr2=0x555b5888a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc699c700 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 0x555b5888a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc699c700 1 -- 172.21.15.131:0/441508982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac003570 con 0x555b588908b0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadc699c700 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588b1dc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fadb80832a0 tx=0x7fadb8023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598bf8e782d5cbde server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fada0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fada0ff9700 1 -- 172.21.15.131:0/441508982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8055c20 con 0x555b588908b0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fada0ff9700 1 -- 172.21.15.131:0/441508982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fadb8058e70 con 0x555b588908b0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.769+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/441508982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b58838eb0 con 0x555b588908b0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fada0ff9700 1 -- 172.21.15.131:0/441508982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb805cbc0 con 0x555b588908b0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fadd6bf2b80 10 client.8280 fetch_fsmap learned FSMap version 66 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/441508982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x555b585f7b10 con 0x555b588908b0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fada0ff9700 1 -- 172.21.15.131:0/441508982 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b58838eb0 con 0x555b588908b0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fada0ff9700 1 -- 172.21.15.131:0/441508982 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 66) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fadb805c730 con 0x555b588908b0 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fadd6bf2b80 10 client.8280 fetch_fsmap finished waiting for FSMap version 66 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fadd6bf2b80 10 client.8280 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-26T21:56:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fadd6bf2b80 20 client.8280 populate_metadata read hostname 'smithi131' 2023-12-26T21:56:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] conn(0x555b588af6c0 0x555b588a7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fadd6bf2b80 4 client.8280 mds_command: new command op to 8142 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T21:56:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/441508982 --> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b5878e880 con 0x555b588af6c0 2023-12-26T21:56:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.770+0000 7fadc619b700 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] conn(0x555b588af6c0 0x555b588a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.771+0000 7fada0ff9700 10 client.8280 ms_handle_connect on v2:172.21.15.131:6834/1841321761 2023-12-26T21:56:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.771+0000 7fadc619b700 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] conn(0x555b588af6c0 0x555b588a7580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5d6dd954382f98d server_cookie=8e37cd312a23c096 in_seq=0 out_seq=0 2023-12-26T21:56:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.771+0000 7fada0ff9700 1 -- 172.21.15.131:0/441508982 <== mds.0 v2:172.21.15.131:6834/1841321761 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x555b5878e880 con 0x555b588af6c0 2023-12-26T21:56:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.771+0000 7fada0ff9700 10 client.8280 handle_command_reply: tid=0 2023-12-26T21:56:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.771+0000 7fadd6bf2b80 1 client.8280 shutdown 2023-12-26T21:56:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/441508982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 msgr2=0x555b588b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588b1dc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fadb80832a0 tx=0x7fadb8023040 comp rx=0 tx=0).stop 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/441508982 shutdown_connections 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] conn(0x555b588af6c0 0x555b588a7580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588b1dc0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 0x555b5888e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/441508982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 0x555b5888a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fada0ff9700 0 client.8280 ms_handle_reset on v2:172.21.15.131:6834/1841321761 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/441508982 >> 172.21.15.131:0/441508982 conn(0x555b58866440 msgr2=0x555b588ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/441508982 shutdown_connections 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/441508982 wait complete. 2023-12-26T21:56:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.772+0000 7fadd6bf2b80 20 client.8280 trim_cache size 0 max 16384 2023-12-26T21:56:35.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.789+0000 7fadd6bf2b80 1 Processor -- start 2023-12-26T21:56:35.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.789+0000 7fadd6bf2b80 1 -- start start 2023-12-26T21:56:35.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 0x555b58889570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 0x555b58889ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5885dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b5888b9e0 con 0x555b588908b0 2023-12-26T21:56:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b588656c0 con 0x555b5888c170 2023-12-26T21:56:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b5885ffc0 con 0x555b5888ffa0 2023-12-26T21:56:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadc699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5885dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadc699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5885dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53256/0 (socket says 172.21.15.131:53256) 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadc699c700 1 -- 172.21.15.131:0/3946847346 learned_addr learned my addr 172.21.15.131:0/3946847346 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadc599a700 1 --2- 172.21.15.131:0/3946847346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 0x555b58889ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadc619b700 1 --2- 172.21.15.131:0/3946847346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 0x555b58889570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.790+0000 7fadc699c700 1 -- 172.21.15.131:0/3946847346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 msgr2=0x555b58889ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadc699c700 1 --2- 172.21.15.131:0/3946847346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 0x555b58889ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadc699c700 1 -- 172.21.15.131:0/3946847346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 msgr2=0x555b58889570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadc699c700 1 --2- 172.21.15.131:0/3946847346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 0x555b58889570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadc699c700 1 -- 172.21.15.131:0/3946847346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b588b2c50 con 0x555b588908b0 2023-12-26T21:56:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadc699c700 1 --2- 172.21.15.131:0/3946847346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5885dee0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fadb8053f70 tx=0x7fadb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4af3d23e6f8fff server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fada3fff700 1 -- 172.21.15.131:0/3946847346 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8058e70 con 0x555b588908b0 2023-12-26T21:56:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fada3fff700 1 -- 172.21.15.131:0/3946847346 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb8055c20 con 0x555b588908b0 2023-12-26T21:56:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fada3fff700 1 -- 172.21.15.131:0/3946847346 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb805cbc0 con 0x555b588908b0 2023-12-26T21:56:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/3946847346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 msgr2=0x555b5885dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/3946847346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5885dee0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fadb8053f70 tx=0x7fadb8017000 comp rx=0 tx=0).stop 2023-12-26T21:56:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/3946847346 shutdown_connections 2023-12-26T21:56:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/3946847346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b5885dee0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/3946847346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888c170 0x555b58889570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/3946847346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888ffa0 0x555b58889ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.791+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/3946847346 >> 172.21.15.131:0/3946847346 conn(0x555b58866440 msgr2=0x555b588ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.792+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/3946847346 shutdown_connections 2023-12-26T21:56:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.792+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/3946847346 wait complete. 2023-12-26T21:56:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.792+0000 7fadd6bf2b80 1 -- start start 2023-12-26T21:56:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.793+0000 7fadd6bf2b80 1 Processor -- start 2023-12-26T21:56:35.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.794+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 0x555b588cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.794+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 0x555b589169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.794+0000 7fadd6bf2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588a7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.794+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b588cea20 con 0x555b588908b0 2023-12-26T21:56:35.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.794+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b588cefa0 con 0x555b5888ffa0 2023-12-26T21:56:35.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.794+0000 7fadd6bf2b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b588cec30 con 0x555b5888c170 2023-12-26T21:56:35.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.794+0000 7fadd6bf2b80 10 client.0 _ll_register_callbacks cb 0x555b586b37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T21:56:35.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc699c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588a7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53262/0 (socket says 172.21.15.131:53262) 2023-12-26T21:56:35.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc699c700 1 -- 172.21.15.131:0/4188596588 learned_addr learned my addr 172.21.15.131:0/4188596588 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc619b700 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 0x555b588cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc599a700 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 0x555b589169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc699c700 1 -- 172.21.15.131:0/4188596588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 msgr2=0x555b588cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc699c700 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 0x555b588cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc699c700 1 -- 172.21.15.131:0/4188596588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 msgr2=0x555b589169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc699c700 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 0x555b589169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc699c700 1 -- 172.21.15.131:0/4188596588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b585f7b10 con 0x555b588908b0 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadc699c700 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588a7280 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fadb8052640 tx=0x7fadb805b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47368862fd8f6543 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fada0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T21:56:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fada0ff9700 1 -- 172.21.15.131:0/4188596588 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb805aed0 con 0x555b588908b0 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fada0ff9700 1 -- 172.21.15.131:0/4188596588 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fadb8058e70 con 0x555b588908b0 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.795+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/4188596588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadc0157590 con 0x555b588908b0 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fada0ff9700 1 -- 172.21.15.131:0/4188596588 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb805cbc0 con 0x555b588908b0 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fadd6bf2b80 10 client.8286 fetch_fsmap learned FSMap version 66 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/4188596588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x555b58786290 con 0x555b588908b0 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fada0ff9700 1 -- 172.21.15.131:0/4188596588 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc0157590 con 0x555b588908b0 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fada0ff9700 1 -- 172.21.15.131:0/4188596588 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 66) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fadb8051a10 con 0x555b588908b0 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fadd6bf2b80 10 client.8286 fetch_fsmap finished waiting for FSMap version 66 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fadd6bf2b80 10 client.8286 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fadd6bf2b80 20 client.8286 populate_metadata read hostname 'smithi131' 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] conn(0x555b588887e0 0x555b58888bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fadd6bf2b80 4 client.8286 mds_command: new command op to 8142 tid=0[{"prefix": "status"}] 2023-12-26T21:56:35.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/4188596588 --> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555b5885c490 con 0x555b588887e0 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.796+0000 7fadc619b700 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] conn(0x555b588887e0 0x555b58888bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadc619b700 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] conn(0x555b588887e0 0x555b58888bb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0425bdf6847a92 server_cookie=d729a53581978f28 in_seq=0 out_seq=0 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fada0ff9700 10 client.8286 ms_handle_connect on v2:172.21.15.131:6834/1841321761 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fada0ff9700 1 -- 172.21.15.131:0/4188596588 <== mds.0 v2:172.21.15.131:6834/1841321761 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x555b5885c490 con 0x555b588887e0 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fada0ff9700 10 client.8286 handle_command_reply: tid=0 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 client.8286 shutdown 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/4188596588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 msgr2=0x555b588a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588a7280 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fadb8052640 tx=0x7fadb805b050 comp rx=0 tx=0).stop 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/4188596588 shutdown_connections 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.131:6834/1841321761,v1:172.21.15.131:6835/1841321761] conn(0x555b588887e0 0x555b58888bb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b588908b0 0x555b588a7280 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b5888c170 0x555b588cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fada0ff9700 0 client.8286 ms_handle_reset on v2:172.21.15.131:6834/1841321761 2023-12-26T21:56:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 --2- 172.21.15.131:0/4188596588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b5888ffa0 0x555b589169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/4188596588 >> 172.21.15.131:0/4188596588 conn(0x555b58866440 msgr2=0x555b588a6170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/4188596588 shutdown_connections 2023-12-26T21:56:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 1 -- 172.21.15.131:0/4188596588 wait complete. 2023-12-26T21:56:35.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.797+0000 7fadd6bf2b80 20 client.8286 trim_cache size 0 max 16384 2023-12-26T21:56:35.806 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T21:56:35.806 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T21:56:35.806 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8142, 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.4792173210000001, 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 66, 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 139, 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 139, 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 12.63852529 2023-12-26T21:56:35.807 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T21:56:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 -- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fada4066f60 msgr2=0x7fada4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fada4066f60 0x7fada4069410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fadc0158a80 tx=0x7fadbc007d30 comp rx=0 tx=0).stop 2023-12-26T21:56:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 -- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc00a64d0 msgr2=0x7fadc00a7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc00a64d0 0x7fadc00a7740 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fadc0064440 tx=0x7fadb800b040 comp rx=0 tx=0).stop 2023-12-26T21:56:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 -- 172.21.15.131:0/3447147236 shutdown_connections 2023-12-26T21:56:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fada4066f60 0x7fada4069410 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadc00a64d0 0x7fadc00a7740 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadac004010 0x7fadc0166390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 --2- 172.21.15.131:0/3447147236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc01598c0 0x7fadc0157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:35.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 -- 172.21.15.131:0/3447147236 >> 172.21.15.131:0/3447147236 conn(0x7fadc0068040 msgr2=0x7fadc006dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:35.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 -- 172.21.15.131:0/3447147236 shutdown_connections 2023-12-26T21:56:35.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:35.798+0000 7fadc83ff700 1 -- 172.21.15.131:0/3447147236 wait complete. 2023-12-26T21:56:35.815 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T21:56:36.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.004+0000 7facd79ff700 1 Processor -- start 2023-12-26T21:56:36.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.004+0000 7facd79ff700 1 -- start start 2023-12-26T21:56:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.005+0000 7facd79ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd00b5310 0x7facd00b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.006+0000 7facd79ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd00afcd0 con 0x7facd0159a70 2023-12-26T21:56:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.006+0000 7facd79ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd00b3e70 con 0x7facd01596a0 2023-12-26T21:56:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.006+0000 7facd79ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd00b0f20 con 0x7facd00b5310 2023-12-26T21:56:36.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.006+0000 7facd4f9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7facd0159a70 0x7facd00b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51612/0 (socket says 172.21.15.131:51612) 2023-12-26T21:56:36.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.006+0000 7facd4f9a700 1 -- 172.21.15.131:0/1163502822 learned_addr learned my addr 172.21.15.131:0/1163502822 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:36.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.006+0000 7facd5f9c700 1 --2- 172.21.15.131:0/1163502822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd00b5310 0x7facd00b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:36.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.006+0000 7facc7fff700 1 -- 172.21.15.131:0/1163502822 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238633565 0 0) 0x7facd00afcd0 con 0x7facd0159a70 2023-12-26T21:56:36.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.006+0000 7facc7fff700 1 -- 172.21.15.131:0/1163502822 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facbc003570 con 0x7facd0159a70 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facc7fff700 1 -- 172.21.15.131:0/1163502822 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825786591 0 0) 0x7facd00b3e70 con 0x7facd01596a0 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facc7fff700 1 -- 172.21.15.131:0/1163502822 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd00afcd0 con 0x7facd01596a0 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facc7fff700 1 -- 172.21.15.131:0/1163502822 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740542296 0 0) 0x7facbc003570 con 0x7facd0159a70 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facd5f9c700 1 -- 172.21.15.131:0/1163502822 >> v1:172.21.15.190:6789/0 conn(0x7facd01596a0 legacy=0x7facd00b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facd5f9c700 1 -- 172.21.15.131:0/1163502822 >> v1:172.21.15.131:6789/0 conn(0x7facd0159a70 legacy=0x7facd00b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facd5f9c700 1 -- 172.21.15.131:0/1163502822 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00b3540 con 0x7facd00b5310 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facd5f9c700 1 --2- 172.21.15.131:0/1163502822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd00b5310 0x7facd00b0870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7facc8009e90 tx=0x7facc8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2b640f7cbf128ba server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facc7fff700 1 -- 172.21.15.131:0/1163502822 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc800e070 con 0x7facd00b5310 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facc7fff700 1 -- 172.21.15.131:0/1163502822 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc800b870 con 0x7facd00b5310 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.007+0000 7facc7fff700 1 -- 172.21.15.131:0/1163502822 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc00013f0 con 0x7facd00b5310 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.008+0000 7facd79ff700 1 -- 172.21.15.131:0/1163502822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd00b5310 msgr2=0x7facd00b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.008+0000 7facd79ff700 1 --2- 172.21.15.131:0/1163502822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd00b5310 0x7facd00b0870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7facc8009e90 tx=0x7facc8009780 comp rx=0 tx=0).stop 2023-12-26T21:56:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.008+0000 7facd79ff700 1 -- 172.21.15.131:0/1163502822 shutdown_connections 2023-12-26T21:56:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.008+0000 7facd79ff700 1 --2- 172.21.15.131:0/1163502822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd00b5310 0x7facd00b0870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.008+0000 7facd79ff700 1 -- 172.21.15.131:0/1163502822 >> 172.21.15.131:0/1163502822 conn(0x7facd006ece0 msgr2=0x7facd006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.008+0000 7facd79ff700 1 -- 172.21.15.131:0/1163502822 shutdown_connections 2023-12-26T21:56:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.008+0000 7facd79ff700 1 -- 172.21.15.131:0/1163502822 wait complete. 2023-12-26T21:56:36.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.009+0000 7facd79ff700 1 Processor -- start 2023-12-26T21:56:36.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.010+0000 7facd79ff700 1 -- start start 2023-12-26T21:56:36.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.010+0000 7facd79ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facd00b5310 0x7facd0168e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:36.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.010+0000 7facd79ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd01596a0 0x7facd00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:36.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.010+0000 7facd79ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd0159a70 0x7facd00cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:36.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.010+0000 7facd79ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd0155d40 con 0x7facd00b5310 2023-12-26T21:56:36.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.010+0000 7facd79ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd0156020 con 0x7facd01596a0 2023-12-26T21:56:36.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.010+0000 7facd79ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd0156300 con 0x7facd0159a70 2023-12-26T21:56:36.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facd00b5310 0x7facd0168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:36.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facd00b5310 0x7facd0168e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53278/0 (socket says 172.21.15.131:53278) 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd579b700 1 -- 172.21.15.131:0/1254834220 learned_addr learned my addr 172.21.15.131:0/1254834220 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd5f9c700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd0159a70 0x7facd00cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd4f9a700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd01596a0 0x7facd00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd579b700 1 -- 172.21.15.131:0/1254834220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd0159a70 msgr2=0x7facd00cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd579b700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd0159a70 0x7facd00cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd579b700 1 -- 172.21.15.131:0/1254834220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd01596a0 msgr2=0x7facd00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd579b700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd01596a0 0x7facd00cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd579b700 1 -- 172.21.15.131:0/1254834220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8009a20 con 0x7facd00b5310 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd4f9a700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd01596a0 0x7facd00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:36.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.011+0000 7facd579b700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facd00b5310 0x7facd0168e40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7faccc00c7c0 tx=0x7faccc007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.012+0000 7facc57fa700 1 -- 172.21.15.131:0/1254834220 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc010510 con 0x7facd00b5310 2023-12-26T21:56:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.012+0000 7facc57fa700 1 -- 172.21.15.131:0/1254834220 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc010670 con 0x7facd00b5310 2023-12-26T21:56:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.012+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd01565e0 con 0x7facd00b5310 2023-12-26T21:56:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.012+0000 7facc57fa700 1 -- 172.21.15.131:0/1254834220 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc010a30 con 0x7facd00b5310 2023-12-26T21:56:36.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.012+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd0169380 con 0x7facd00b5310 2023-12-26T21:56:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.013+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd00657e0 con 0x7facd00b5310 2023-12-26T21:56:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.014+0000 7facc57fa700 1 -- 172.21.15.131:0/1254834220 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faccc01b3b0 con 0x7facd00b5310 2023-12-26T21:56:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.014+0000 7facc57fa700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7facb4066f30 0x7facb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.014+0000 7facc57fa700 1 -- 172.21.15.131:0/1254834220 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7faccc083e80 con 0x7facd00b5310 2023-12-26T21:56:36.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.015+0000 7facd4f9a700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7facb4066f30 0x7facb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:36.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.015+0000 7facd4f9a700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7facb4066f30 0x7facb40693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facc0001ed0 tx=0x7facc0001820 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:36.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.018+0000 7facc57fa700 1 -- 172.21.15.131:0/1254834220 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faccc050f50 con 0x7facd00b5310 2023-12-26T21:56:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.253+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7facd0155e80 con 0x7facd00b5310 2023-12-26T21:56:36.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.254+0000 7facc57fa700 1 -- 172.21.15.131:0/1254834220 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7facd0155e80 con 0x7facd00b5310 2023-12-26T21:56:36.254 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7facb4066f30 msgr2=0x7facb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7facb4066f30 0x7facb40693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facc0001ed0 tx=0x7facc0001820 comp rx=0 tx=0).stop 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facd00b5310 msgr2=0x7facd0168e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facd00b5310 0x7facd0168e40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7faccc00c7c0 tx=0x7faccc007e50 comp rx=0 tx=0).stop 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 shutdown_connections 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd0159a70 0x7facd00cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7facb4066f30 0x7facb40693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facd00b5310 0x7facd0168e40 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 --2- 172.21.15.131:0/1254834220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd01596a0 0x7facd00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 >> 172.21.15.131:0/1254834220 conn(0x7facd006ece0 msgr2=0x7facd00d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:36.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 shutdown_connections 2023-12-26T21:56:36.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:36.256+0000 7facd79ff700 1 -- 172.21.15.131:0/1254834220 wait complete. 2023-12-26T21:56:36.273 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T21:56:36.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T21:56:36.274 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T21:56:36.274 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T21:56:36.274 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T21:56:36.274 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T21:56:36.274 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T21:56:36.274 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T21:56:36.274 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:56:36.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:56:36.274 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T21:56:36.318 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T21:56:36.318 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T21:56:36.319 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41260sec preferred_lft 41260sec 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:56:36.320 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T21:56:36.321 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T21:56:36.321 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:56:36.321 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T21:56:36.321 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T21:56:36.321 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T21:56:36.321 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T21:56:36.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:56:36.321 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T21:56:36.426 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T21:56:36.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:56:36.430 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T21:56:36.477 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T21:56:36.477 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T21:56:36.477 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T21:56:36.477 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T21:56:36.477 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T21:56:36.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:56:36.478 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T21:56:36.478 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T21:56:36.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T21:56:36.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T21:56:36.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T21:56:36.478 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T21:56:36.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:56:36.687 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T21:56:36.729 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T21:56:36.730 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T21:56:36.731 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:56:36.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:56:36.731 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T21:56:36.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:56:36.840 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T21:56:36.884 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T21:56:36.885 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T21:56:36.885 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:56:36.968 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T21:56:36.968 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T21:56:36.968 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:56:37.106 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:56:37.106 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:56:37.134 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T21:56:37.134 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T21:56:37.135 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T21:56:37.135 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T21:56:37.135 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T21:56:37.139 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T21:56:37.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T21:56:37.139 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T21:56:37.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T21:56:37.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.404+0000 7f7796dff700 1 Processor -- start 2023-12-26T21:56:37.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.405+0000 7f7796dff700 1 -- start start 2023-12-26T21:56:37.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.406+0000 7f7796dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f779006bfd0 0x7f77900c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.406+0000 7f7796dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f77900cc100 0x7f77900cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.406+0000 7f7796dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77900686c0 con 0x7f779013c510 2023-12-26T21:56:37.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.406+0000 7f7796dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7790069520 con 0x7f77900cc100 2023-12-26T21:56:37.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.406+0000 7f7796dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7790068910 con 0x7f779006bfd0 2023-12-26T21:56:37.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.406+0000 7f779539c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f77900cc100 0x7f77900cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.406+0000 7f779539c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f77900cc100 0x7f77900cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53286/0 (socket says 172.21.15.131:53286) 2023-12-26T21:56:37.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.406+0000 7f779539c700 1 -- 172.21.15.131:0/2692058486 learned_addr learned my addr 172.21.15.131:0/2692058486 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:37.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.406+0000 7f778ffff700 1 --2- 172.21.15.131:0/2692058486 >> v2:172.21.15.190:3300/0 conn(0x7f779006bfd0 0x7f77900c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.407+0000 7f779539c700 1 -- 172.21.15.131:0/2692058486 >> v2:172.21.15.190:3300/0 conn(0x7f779006bfd0 msgr2=0x7f77900c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:37.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.407+0000 7f779539c700 1 --2- 172.21.15.131:0/2692058486 >> v2:172.21.15.190:3300/0 conn(0x7f779006bfd0 0x7f77900c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.407+0000 7f779539c700 1 -- 172.21.15.131:0/2692058486 >> v1:172.21.15.190:6789/0 conn(0x7f779013c510 legacy=0x7f779006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.407+0000 7f779539c700 1 -- 172.21.15.131:0/2692058486 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7790068bf0 con 0x7f77900cc100 2023-12-26T21:56:37.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.407+0000 7f778ffff700 1 --2- 172.21.15.131:0/2692058486 >> v2:172.21.15.190:3300/0 conn(0x7f779006bfd0 0x7f77900c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T21:56:37.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.407+0000 7f779539c700 1 --2- 172.21.15.131:0/2692058486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 0x7f77900cc520 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f77900b43e0 tx=0x7f7788009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ca984aee7f2021 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:37.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.407+0000 7f778f7fe700 1 -- 172.21.15.131:0/2692058486 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778800e070 con 0x7f77900cc100 2023-12-26T21:56:37.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.407+0000 7f778f7fe700 1 -- 172.21.15.131:0/2692058486 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778800ba50 con 0x7f77900cc100 2023-12-26T21:56:37.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.407+0000 7f778f7fe700 1 -- 172.21.15.131:0/2692058486 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77880092e0 con 0x7f77900cc100 2023-12-26T21:56:37.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.408+0000 7f7796dff700 1 -- 172.21.15.131:0/2692058486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 msgr2=0x7f77900cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.408+0000 7f7796dff700 1 --2- 172.21.15.131:0/2692058486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 0x7f77900cc520 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f77900b43e0 tx=0x7f7788009840 comp rx=0 tx=0).stop 2023-12-26T21:56:37.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.408+0000 7f7796dff700 1 -- 172.21.15.131:0/2692058486 shutdown_connections 2023-12-26T21:56:37.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.408+0000 7f7796dff700 1 --2- 172.21.15.131:0/2692058486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 0x7f77900cc520 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.408+0000 7f7796dff700 1 --2- 172.21.15.131:0/2692058486 >> v2:172.21.15.190:3300/0 conn(0x7f779006bfd0 0x7f77900c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.408+0000 7f7796dff700 1 -- 172.21.15.131:0/2692058486 >> 172.21.15.131:0/2692058486 conn(0x7f77900ce3e0 msgr2=0x7f77900ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:37.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.408+0000 7f7796dff700 1 -- 172.21.15.131:0/2692058486 shutdown_connections 2023-12-26T21:56:37.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.408+0000 7f7796dff700 1 -- 172.21.15.131:0/2692058486 wait complete. 2023-12-26T21:56:37.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.409+0000 7f7796dff700 1 Processor -- start 2023-12-26T21:56:37.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.409+0000 7f7796dff700 1 -- start start 2023-12-26T21:56:37.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.410+0000 7f7796dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f779006bfd0 0x7f779016d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.410+0000 7f7796dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 0x7f779013bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.410+0000 7f7796dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f779013c510 0x7f77900b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.410+0000 7f7796dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779015d240 con 0x7f77900cc100 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.410+0000 7f7796dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779015d520 con 0x7f779013c510 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.410+0000 7f7796dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77901529c0 con 0x7f779006bfd0 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 0x7f779013bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 0x7f779013bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53292/0 (socket says 172.21.15.131:53292) 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778ffff700 1 -- 172.21.15.131:0/843244010 learned_addr learned my addr 172.21.15.131:0/843244010 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f7794b9b700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f779006bfd0 0x7f779016d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f779539c700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f779013c510 0x7f77900b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778ffff700 1 -- 172.21.15.131:0/843244010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f779006bfd0 msgr2=0x7f779016d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778ffff700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f779006bfd0 0x7f779016d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778ffff700 1 -- 172.21.15.131:0/843244010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f779013c510 msgr2=0x7f77900b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778ffff700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f779013c510 0x7f77900b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778ffff700 1 -- 172.21.15.131:0/843244010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788009ae0 con 0x7f77900cc100 2023-12-26T21:56:37.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778ffff700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 0x7f779013bcb0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f777c00d8c0 tx=0x7f777c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:37.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.411+0000 7f778cff9700 1 -- 172.21.15.131:0/843244010 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0125f0 con 0x7f77900cc100 2023-12-26T21:56:37.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.412+0000 7f778cff9700 1 -- 172.21.15.131:0/843244010 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c012750 con 0x7f77900cc100 2023-12-26T21:56:37.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.412+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779016d9d0 con 0x7f77900cc100 2023-12-26T21:56:37.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.412+0000 7f778cff9700 1 -- 172.21.15.131:0/843244010 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c012b10 con 0x7f77900cc100 2023-12-26T21:56:37.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.412+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779016df90 con 0x7f77900cc100 2023-12-26T21:56:37.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.413+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779015d6b0 con 0x7f77900cc100 2023-12-26T21:56:37.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.413+0000 7f778cff9700 1 -- 172.21.15.131:0/843244010 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f777c014430 con 0x7f77900cc100 2023-12-26T21:56:37.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.414+0000 7f778cff9700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7774066f30 0x7f77740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.414+0000 7f778cff9700 1 -- 172.21.15.131:0/843244010 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f777c084a50 con 0x7f77900cc100 2023-12-26T21:56:37.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.415+0000 7f7794b9b700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7774066f30 0x7f77740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.415+0000 7f7794b9b700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7774066f30 0x7f77740693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7784007fd0 tx=0x7f7784007eb0 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:37.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.418+0000 7f778cff9700 1 -- 172.21.15.131:0/843244010 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f777c051a80 con 0x7f77900cc100 2023-12-26T21:56:37.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.638+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f77900657e0 con 0x7f7774066f30 2023-12-26T21:56:37.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.639+0000 7f778cff9700 1 -- 172.21.15.131:0/843244010 <== mgr.7591 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f77900657e0 con 0x7f7774066f30 2023-12-26T21:56:37.639 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T21:56:37.639 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T21:56:37.639 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T21:56:37.639 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T21:56:37.639 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T21:56:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7774066f30 msgr2=0x7f77740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7774066f30 0x7f77740693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7784007fd0 tx=0x7f7784007eb0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 msgr2=0x7f779013bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 0x7f779013bcb0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f777c00d8c0 tx=0x7f777c008040 comp rx=0 tx=0).stop 2023-12-26T21:56:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 shutdown_connections 2023-12-26T21:56:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f779013c510 0x7f77900b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f779006bfd0 0x7f779016d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7774066f30 0x7f77740693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 --2- 172.21.15.131:0/843244010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77900cc100 0x7f779013bcb0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 >> 172.21.15.131:0/843244010 conn(0x7f77900ce3e0 msgr2=0x7f77900c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 shutdown_connections 2023-12-26T21:56:37.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.642+0000 7f7796dff700 1 -- 172.21.15.131:0/843244010 wait complete. 2023-12-26T21:56:37.659 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T21:56:37.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.860+0000 7fe7ac9ff700 1 Processor -- start 2023-12-26T21:56:37.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.860+0000 7fe7ac9ff700 1 -- start start 2023-12-26T21:56:37.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.861+0000 7fe7ac9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7a80cb320 0x7fe7a8069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7ac9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7a80b0e60 0x7fe7a80c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7ac9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a8156940 con 0x7fe7a806a1e0 2023-12-26T21:56:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7ac9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80c8cc0 con 0x7fe7a80b0e60 2023-12-26T21:56:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7ac9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80ad700 con 0x7fe7a80cb320 2023-12-26T21:56:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7a6d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7a80b0e60 0x7fe7a80c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7a5d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe7a806a1e0 0x7fe7a80b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51614/0 (socket says 172.21.15.131:51614) 2023-12-26T21:56:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7a5d9b700 1 -- 172.21.15.131:0/1224591588 learned_addr learned my addr 172.21.15.131:0/1224591588 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7a659c700 1 --2- 172.21.15.131:0/1224591588 >> v2:172.21.15.190:3300/0 conn(0x7fe7a80cb320 0x7fe7a8069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7a6d9d700 1 -- 172.21.15.131:0/1224591588 >> v2:172.21.15.190:3300/0 conn(0x7fe7a80cb320 msgr2=0x7fe7a8069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7a6d9d700 1 --2- 172.21.15.131:0/1224591588 >> v2:172.21.15.190:3300/0 conn(0x7fe7a80cb320 0x7fe7a8069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7a6d9d700 1 -- 172.21.15.131:0/1224591588 >> v1:172.21.15.131:6789/0 conn(0x7fe7a806a1e0 legacy=0x7fe7a80b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7a6d9d700 1 -- 172.21.15.131:0/1224591588 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80ae500 con 0x7fe7a80b0e60 2023-12-26T21:56:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.862+0000 7fe7a559a700 1 -- 172.21.15.131:0/1224591588 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823528086 0 0) 0x7fe7a8156940 con 0x7fe7a806a1e0 2023-12-26T21:56:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.863+0000 7fe7a6d9d700 1 --2- 172.21.15.131:0/1224591588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a80b0e60 0x7fe7a80c9400 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8064410 tx=0x7fe7a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71def2c680af7ded server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.863+0000 7fe7a559a700 1 -- 172.21.15.131:0/1224591588 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a000e070 con 0x7fe7a80b0e60 2023-12-26T21:56:37.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.863+0000 7fe7a559a700 1 -- 172.21.15.131:0/1224591588 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a000bb60 con 0x7fe7a80b0e60 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.863+0000 7fe7a559a700 1 -- 172.21.15.131:0/1224591588 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0012430 con 0x7fe7a80b0e60 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.863+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1224591588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a80b0e60 msgr2=0x7fe7a80c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.863+0000 7fe7ac9ff700 1 --2- 172.21.15.131:0/1224591588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a80b0e60 0x7fe7a80c9400 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8064410 tx=0x7fe7a0009780 comp rx=0 tx=0).stop 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.864+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1224591588 shutdown_connections 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.864+0000 7fe7ac9ff700 1 --2- 172.21.15.131:0/1224591588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a80b0e60 0x7fe7a80c9400 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.864+0000 7fe7ac9ff700 1 --2- 172.21.15.131:0/1224591588 >> v2:172.21.15.190:3300/0 conn(0x7fe7a80cb320 0x7fe7a8069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.864+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1224591588 >> 172.21.15.131:0/1224591588 conn(0x7fe7a8068040 msgr2=0x7fe7a8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.864+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1224591588 shutdown_connections 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.864+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1224591588 wait complete. 2023-12-26T21:56:37.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.865+0000 7fe7ac9ff700 1 Processor -- start 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.865+0000 7fe7ac9ff700 1 -- start start 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.866+0000 7fe7ac9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a806a1e0 0x7fe7a8169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.866+0000 7fe7ac9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a80b0e60 0x7fe7a8163230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.866+0000 7fe7ac9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cb320 0x7fe7a80abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.866+0000 7fe7ac9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80a6240 con 0x7fe7a806a1e0 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.866+0000 7fe7ac9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a816a420 con 0x7fe7a80b0e60 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.866+0000 7fe7ac9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a816a680 con 0x7fe7a80cb320 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.866+0000 7fe7a659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a806a1e0 0x7fe7a8169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.866+0000 7fe7a659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a806a1e0 0x7fe7a8169ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53306/0 (socket says 172.21.15.131:53306) 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.866+0000 7fe7a659c700 1 -- 172.21.15.131:0/1781537139 learned_addr learned my addr 172.21.15.131:0/1781537139 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7a6d9d700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cb320 0x7fe7a80abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7a5d9b700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a80b0e60 0x7fe7a8163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7a659c700 1 -- 172.21.15.131:0/1781537139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cb320 msgr2=0x7fe7a80abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7a659c700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cb320 0x7fe7a80abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7a659c700 1 -- 172.21.15.131:0/1781537139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a80b0e60 msgr2=0x7fe7a8163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:37.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7a659c700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a80b0e60 0x7fe7a8163230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:37.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7a659c700 1 -- 172.21.15.131:0/1781537139 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0009a20 con 0x7fe7a806a1e0 2023-12-26T21:56:37.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7a659c700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a806a1e0 0x7fe7a8169ee0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe79800e930 tx=0x7fe7980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:37.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7967fc700 1 -- 172.21.15.131:0/1781537139 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79800eed0 con 0x7fe7a806a1e0 2023-12-26T21:56:37.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7967fc700 1 -- 172.21.15.131:0/1781537139 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79800c850 con 0x7fe7a806a1e0 2023-12-26T21:56:37.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a816d070 con 0x7fe7a806a1e0 2023-12-26T21:56:37.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7967fc700 1 -- 172.21.15.131:0/1781537139 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79800cbf0 con 0x7fe7a806a1e0 2023-12-26T21:56:37.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.867+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a816d630 con 0x7fe7a806a1e0 2023-12-26T21:56:37.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.868+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a816d970 con 0x7fe7a806a1e0 2023-12-26T21:56:37.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.869+0000 7fe7967fc700 1 -- 172.21.15.131:0/1781537139 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 62) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe798013470 con 0x7fe7a806a1e0 2023-12-26T21:56:37.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.870+0000 7fe7967fc700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe788066f30 0x7fe7880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:37.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.870+0000 7fe7967fc700 1 -- 172.21.15.131:0/1781537139 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fe7980848b0 con 0x7fe7a806a1e0 2023-12-26T21:56:37.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.870+0000 7fe7a5d9b700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe788066f30 0x7fe7880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:37.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.871+0000 7fe7a5d9b700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe788066f30 0x7fe7880693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8154a20 tx=0x7fe79c009040 comp rx=0 tx=0).ready entity=mgr.7591 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:37.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:37.874+0000 7fe7967fc700 1 -- 172.21.15.131:0/1781537139 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe798051980 con 0x7fe7a806a1e0 2023-12-26T21:56:38.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.119+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe7a816a9b0 con 0x7fe7a806a1e0 2023-12-26T21:56:38.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.365+0000 7fe7967fc700 1 -- 172.21.15.131:0/1781537139 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 63) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fe798051d50 con 0x7fe7a806a1e0 2023-12-26T21:56:38.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.366+0000 7fe7967fc700 1 -- 172.21.15.131:0/1781537139 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v63) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe798051980 con 0x7fe7a806a1e0 2023-12-26T21:56:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe788066f30 msgr2=0x7fe7880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe788066f30 0x7fe7880693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8154a20 tx=0x7fe79c009040 comp rx=0 tx=0).stop 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a806a1e0 msgr2=0x7fe7a8169ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a806a1e0 0x7fe7a8169ee0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe79800e930 tx=0x7fe7980091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 shutdown_connections 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cb320 0x7fe7a80abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe788066f30 0x7fe7880693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7a806a1e0 0x7fe7a8169ee0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 --2- 172.21.15.131:0/1781537139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a80b0e60 0x7fe7a8163230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 >> 172.21.15.131:0/1781537139 conn(0x7fe7a8068040 msgr2=0x7fe7a8154c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 shutdown_connections 2023-12-26T21:56:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.369+0000 7fe7ac9ff700 1 -- 172.21.15.131:0/1781537139 wait complete. 2023-12-26T21:56:38.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T21:56:38.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.583+0000 7f18dddff700 1 Processor -- start 2023-12-26T21:56:38.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.583+0000 7f18dddff700 1 -- start start 2023-12-26T21:56:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.584+0000 7f18dddff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f18d80cb320 0x7f18d8069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.584+0000 7f18dddff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18d806a1e0 0x7f18d80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.584+0000 7f18dddff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d8156940 con 0x7f18d80b0c90 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.584+0000 7f18dddff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d80c8cc0 con 0x7f18d806a1e0 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.584+0000 7f18dddff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d80ad700 con 0x7f18d80cb320 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.584+0000 7f18d6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18d806a1e0 0x7f18d80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.585+0000 7f18d6ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18d806a1e0 0x7f18d80b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53320/0 (socket says 172.21.15.131:53320) 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.585+0000 7f18d6ffd700 1 -- 172.21.15.131:0/4220548438 learned_addr learned my addr 172.21.15.131:0/4220548438 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.585+0000 7f18d77fe700 1 --2- 172.21.15.131:0/4220548438 >> v2:172.21.15.190:3300/0 conn(0x7f18d80cb320 0x7f18d8069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.585+0000 7f18d6ffd700 1 -- 172.21.15.131:0/4220548438 >> v2:172.21.15.190:3300/0 conn(0x7f18d80cb320 msgr2=0x7f18d8069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.585+0000 7f18d6ffd700 1 --2- 172.21.15.131:0/4220548438 >> v2:172.21.15.190:3300/0 conn(0x7f18d80cb320 0x7f18d8069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.585+0000 7f18d6ffd700 1 -- 172.21.15.131:0/4220548438 >> v1:172.21.15.190:6789/0 conn(0x7f18d80b0c90 legacy=0x7f18d80aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.585+0000 7f18d6ffd700 1 -- 172.21.15.131:0/4220548438 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d80ae500 con 0x7f18d806a1e0 2023-12-26T21:56:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.585+0000 7f18d77fe700 1 --2- 172.21.15.131:0/4220548438 >> v2:172.21.15.190:3300/0 conn(0x7f18d80cb320 0x7f18d8069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:38.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.585+0000 7f18d6ffd700 1 --2- 172.21.15.131:0/4220548438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d806a1e0 0x7f18d80b0750 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f18d8064410 tx=0x7f18cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff565aec7f8125e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18d67fc700 1 -- 172.21.15.131:0/4220548438 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc009ea0 con 0x7f18d806a1e0 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18d67fc700 1 -- 172.21.15.131:0/4220548438 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc00b820 con 0x7f18d806a1e0 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18d67fc700 1 -- 172.21.15.131:0/4220548438 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00bbc0 con 0x7f18d806a1e0 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18dddff700 1 -- 172.21.15.131:0/4220548438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d806a1e0 msgr2=0x7f18d80b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18dddff700 1 --2- 172.21.15.131:0/4220548438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d806a1e0 0x7f18d80b0750 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f18d8064410 tx=0x7f18cc009250 comp rx=0 tx=0).stop 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18dddff700 1 -- 172.21.15.131:0/4220548438 shutdown_connections 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18dddff700 1 --2- 172.21.15.131:0/4220548438 >> v2:172.21.15.190:3300/0 conn(0x7f18d80cb320 0x7f18d8069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18dddff700 1 --2- 172.21.15.131:0/4220548438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d806a1e0 0x7f18d80b0750 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18dddff700 1 -- 172.21.15.131:0/4220548438 >> 172.21.15.131:0/4220548438 conn(0x7f18d8068040 msgr2=0x7f18d8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.586+0000 7f18dddff700 1 -- 172.21.15.131:0/4220548438 shutdown_connections 2023-12-26T21:56:38.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.587+0000 7f18dddff700 1 -- 172.21.15.131:0/4220548438 wait complete. 2023-12-26T21:56:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.588+0000 7f18dddff700 1 Processor -- start 2023-12-26T21:56:38.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.588+0000 7f18dddff700 1 -- start start 2023-12-26T21:56:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18dddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d806a1e0 0x7f18d80c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18dddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18d80b0c90 0x7f18d81699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18dddff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d80cb320 0x7f18d80a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18dddff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80c7b60 con 0x7f18d80cb320 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18dddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d816a320 con 0x7f18d80b0c90 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18dddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d816a600 con 0x7f18d806a1e0 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18d7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d80cb320 0x7f18d80a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18d77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d806a1e0 0x7f18d80c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18d6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18d80b0c90 0x7f18d81699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18d7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d80cb320 0x7f18d80a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53324/0 (socket says 172.21.15.131:53324) 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.589+0000 7f18d7fff700 1 -- 172.21.15.131:0/670686915 learned_addr learned my addr 172.21.15.131:0/670686915 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18d77fe700 1 -- 172.21.15.131:0/670686915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18d80b0c90 msgr2=0x7f18d81699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18d77fe700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18d80b0c90 0x7f18d81699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18d77fe700 1 -- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d80cb320 msgr2=0x7f18d80a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18d77fe700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d80cb320 0x7f18d80a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18d77fe700 1 -- 172.21.15.131:0/670686915 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8009420 con 0x7f18d806a1e0 2023-12-26T21:56:38.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18d77fe700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d806a1e0 0x7f18d80c7620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18c800ea00 tx=0x7f18c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18bf7fe700 1 -- 172.21.15.131:0/670686915 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800c6a0 con 0x7f18d806a1e0 2023-12-26T21:56:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18bf7fe700 1 -- 172.21.15.131:0/670686915 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c800c800 con 0x7f18d806a1e0 2023-12-26T21:56:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc0094f0 con 0x7f18d806a1e0 2023-12-26T21:56:38.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.590+0000 7f18bf7fe700 1 -- 172.21.15.131:0/670686915 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800cba0 con 0x7f18d806a1e0 2023-12-26T21:56:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.591+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d80a5520 con 0x7f18d806a1e0 2023-12-26T21:56:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.592+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d80657e0 con 0x7f18d806a1e0 2023-12-26T21:56:38.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.593+0000 7f18bf7fe700 1 -- 172.21.15.131:0/670686915 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 63) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f18c8013470 con 0x7f18d806a1e0 2023-12-26T21:56:38.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.593+0000 7f18bf7fe700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f18c4066f80 0x7f18c4069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:38.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.594+0000 7f18bf7fe700 1 -- 172.21.15.131:0/670686915 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f18c8084940 con 0x7f18d806a1e0 2023-12-26T21:56:38.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.594+0000 7f18d6ffd700 1 -- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f18c4066f80 msgr2=0x7f18c4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:38.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.594+0000 7f18d6ffd700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f18c4066f80 0x7f18c4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:38.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.598+0000 7f18bf7fe700 1 -- 172.21.15.131:0/670686915 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18c8051970 con 0x7f18d806a1e0 2023-12-26T21:56:38.794 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:38.793+0000 7f3abe2b11c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:56:38.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.795+0000 7f18d6ffd700 1 -- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f18c4066f80 msgr2=0x7f18c4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:38.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.795+0000 7f18d6ffd700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f18c4066f80 0x7f18c4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:38.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.799+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f18d80cbce0 con 0x7f18d806a1e0 2023-12-26T21:56:38.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.805+0000 7f18bf7fe700 1 -- 172.21.15.131:0/670686915 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f18c8054f90 con 0x7f18d806a1e0 2023-12-26T21:56:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f18c4066f80 msgr2=0x7f18c4069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f18c4066f80 0x7f18c4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d806a1e0 msgr2=0x7f18d80c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d806a1e0 0x7f18d80c7620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18c800ea00 tx=0x7f18c80091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 shutdown_connections 2023-12-26T21:56:38.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f18c4066f80 0x7f18c4069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d80cb320 0x7f18d80a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d806a1e0 0x7f18d80c7620 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 --2- 172.21.15.131:0/670686915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18d80b0c90 0x7f18d81699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:38.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 >> 172.21.15.131:0/670686915 conn(0x7f18d8068040 msgr2=0x7f18d80cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:38.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 shutdown_connections 2023-12-26T21:56:38.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:38.807+0000 7f18dddff700 1 -- 172.21.15.131:0/670686915 wait complete. 2023-12-26T21:56:38.817 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:38.817+0000 7fc2beadc1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T21:56:38.826 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T21:56:38.900 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:38.899+0000 7f3abe2b11c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:56:38.938 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:38.937+0000 7fc2beadc1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T21:56:38.988 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:38.987+0000 7f3abe2b11c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:56:39.033 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:39.031+0000 7fc2beadc1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T21:56:39.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.031+0000 7ff26abff700 1 Processor -- start 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.031+0000 7ff26abff700 1 -- start start 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.032+0000 7ff26abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff26413c510 0x7ff2640c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.032+0000 7ff26abff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff26406b8c0 0x7ff26406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff26abff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26406c270 con 0x7ff2640cc100 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff26abff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2640686c0 con 0x7ff26406b8c0 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff26abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff264069520 con 0x7ff26413c510 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff263fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff26406b8c0 0x7ff26406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff26919c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff2640cc100 0x7ff2640cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51636/0 (socket says 172.21.15.131:51636) 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff26919c700 1 -- 172.21.15.131:0/1163201284 learned_addr learned my addr 172.21.15.131:0/1163201284 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff26899b700 1 --2- 172.21.15.131:0/1163201284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff26413c510 0x7ff2640c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff263fff700 1 -- 172.21.15.131:0/1163201284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff26413c510 msgr2=0x7ff2640c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff263fff700 1 --2- 172.21.15.131:0/1163201284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff26413c510 0x7ff2640c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff263fff700 1 -- 172.21.15.131:0/1163201284 >> v1:172.21.15.131:6789/0 conn(0x7ff2640cc100 legacy=0x7ff2640cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff263fff700 1 -- 172.21.15.131:0/1163201284 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff264068910 con 0x7ff26406b8c0 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.033+0000 7ff2637fe700 1 -- 172.21.15.131:0/1163201284 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105503640 0 0) 0x7ff26406c270 con 0x7ff2640cc100 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.034+0000 7ff263fff700 1 --2- 172.21.15.131:0/1163201284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26406b8c0 0x7ff26406bd30 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff2500098e0 tx=0x7ff2500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00debb203a5ffe server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.034+0000 7ff2637fe700 1 -- 172.21.15.131:0/1163201284 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000b640 con 0x7ff26406b8c0 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.034+0000 7ff2637fe700 1 -- 172.21.15.131:0/1163201284 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25000b7a0 con 0x7ff26406b8c0 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.034+0000 7ff2637fe700 1 -- 172.21.15.131:0/1163201284 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25000bb40 con 0x7ff26406b8c0 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.034+0000 7ff26abff700 1 -- 172.21.15.131:0/1163201284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26406b8c0 msgr2=0x7ff26406bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.034+0000 7ff26abff700 1 --2- 172.21.15.131:0/1163201284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26406b8c0 0x7ff26406bd30 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff2500098e0 tx=0x7ff2500091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.035+0000 7ff26abff700 1 -- 172.21.15.131:0/1163201284 shutdown_connections 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.035+0000 7ff26abff700 1 --2- 172.21.15.131:0/1163201284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff26413c510 0x7ff2640c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.035+0000 7ff26abff700 1 --2- 172.21.15.131:0/1163201284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26406b8c0 0x7ff26406bd30 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.035+0000 7ff26abff700 1 -- 172.21.15.131:0/1163201284 >> 172.21.15.131:0/1163201284 conn(0x7ff2640ce3e0 msgr2=0x7ff2640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.035+0000 7ff26abff700 1 -- 172.21.15.131:0/1163201284 shutdown_connections 2023-12-26T21:56:39.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.035+0000 7ff26abff700 1 -- 172.21.15.131:0/1163201284 wait complete. 2023-12-26T21:56:39.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.036+0000 7ff26abff700 1 Processor -- start 2023-12-26T21:56:39.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.036+0000 7ff26abff700 1 -- start start 2023-12-26T21:56:39.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.037+0000 7ff26abff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff26406b8c0 0x7ff264139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.037+0000 7ff26abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2640cc100 0x7ff26415bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.037+0000 7ff26abff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26413c510 0x7ff2641590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.037+0000 7ff26abff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26414fee0 con 0x7ff26413c510 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.037+0000 7ff26abff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2641504a0 con 0x7ff26406b8c0 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.037+0000 7ff26abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26415c6b0 con 0x7ff2640cc100 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26413c510 0x7ff2641590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26899b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff26406b8c0 0x7ff264139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff263fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2640cc100 0x7ff26415bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26899b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff26406b8c0 0x7ff264139dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:37028/0 (socket says 172.21.15.131:37028) 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26899b700 1 -- 172.21.15.131:0/3094232772 learned_addr learned my addr 172.21.15.131:0/3094232772 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26899b700 1 -- 172.21.15.131:0/3094232772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2640cc100 msgr2=0x7ff26415bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26899b700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2640cc100 0x7ff26415bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26899b700 1 -- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26413c510 msgr2=0x7ff2641590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26899b700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26413c510 0x7ff2641590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26899b700 1 -- 172.21.15.131:0/3094232772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff250009420 con 0x7ff26406b8c0 2023-12-26T21:56:39.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26919c700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26413c510 0x7ff2641590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T21:56:39.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff26899b700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff26406b8c0 0x7ff264139dc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff25800ec80 tx=0x7ff258009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:39.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.038+0000 7ff260ff9700 1 -- 172.21.15.131:0/3094232772 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25800c850 con 0x7ff26406b8c0 2023-12-26T21:56:39.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.039+0000 7ff260ff9700 1 -- 172.21.15.131:0/3094232772 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25800c9b0 con 0x7ff26406b8c0 2023-12-26T21:56:39.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.039+0000 7ff260ff9700 1 -- 172.21.15.131:0/3094232772 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25800cd30 con 0x7ff26406b8c0 2023-12-26T21:56:39.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.039+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26415c990 con 0x7ff26406b8c0 2023-12-26T21:56:39.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.039+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26413a300 con 0x7ff26406b8c0 2023-12-26T21:56:39.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.040+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2640657e0 con 0x7ff26406b8c0 2023-12-26T21:56:39.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.040+0000 7ff260ff9700 1 -- 172.21.15.131:0/3094232772 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 63) v1 ==== 84759+0+0 (secure 0 0 0) 0x7ff258013680 con 0x7ff26406b8c0 2023-12-26T21:56:39.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.041+0000 7ff260ff9700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff248066f80 0x7ff248069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.041+0000 7ff260ff9700 1 -- 172.21.15.131:0/3094232772 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7ff258084d30 con 0x7ff26406b8c0 2023-12-26T21:56:39.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.041+0000 7ff263fff700 1 -- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff248066f80 msgr2=0x7ff248069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:39.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.041+0000 7ff263fff700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff248066f80 0x7ff248069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:39.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.046+0000 7ff260ff9700 1 -- 172.21.15.131:0/3094232772 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff258051d60 con 0x7ff26406b8c0 2023-12-26T21:56:39.072 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:39.070+0000 7f3abe2b11c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:56:39.118 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:39.118+0000 7fc2beadc1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T21:56:39.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.242+0000 7ff263fff700 1 -- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff248066f80 msgr2=0x7ff248069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:39.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.242+0000 7ff263fff700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff248066f80 0x7ff248069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:39.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.247+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ff264068bf0 con 0x7ff26406b8c0 2023-12-26T21:56:39.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.252+0000 7ff260ff9700 1 -- 172.21.15.131:0/3094232772 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff2580210c0 con 0x7ff26406b8c0 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff248066f80 msgr2=0x7ff248069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff248066f80 0x7ff248069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff26406b8c0 msgr2=0x7ff264139dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff26406b8c0 0x7ff264139dc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff25800ec80 tx=0x7ff258009120 comp rx=0 tx=0).stop 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 shutdown_connections 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff248066f80 0x7ff248069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff26413c510 0x7ff2641590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff26406b8c0 0x7ff264139dc0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 --2- 172.21.15.131:0/3094232772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2640cc100 0x7ff26415bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 >> 172.21.15.131:0/3094232772 conn(0x7ff2640ce3e0 msgr2=0x7ff264153960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 shutdown_connections 2023-12-26T21:56:39.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.254+0000 7ff26abff700 1 -- 172.21.15.131:0/3094232772 wait complete. 2023-12-26T21:56:39.262 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:39.261+0000 7f3abe2b11c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:56:39.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T21:56:39.273 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-26T21:56:39.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1 1M 2023-12-26T21:56:39.326 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:39.325+0000 7fc2beadc1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:56:39.440 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:39.439+0000 7f3abe2b11c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:56:39.504 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:39.504+0000 7fc2beadc1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T21:56:39.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.520+0000 7faf30dff700 1 Processor -- start 2023-12-26T21:56:39.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.520+0000 7faf30dff700 1 -- start start 2023-12-26T21:56:39.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.521+0000 7faf30dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faf2c156730 0x7faf2c0c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.521+0000 7faf30dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c0acfc0 con 0x7faf2c06ee80 2023-12-26T21:56:39.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.521+0000 7faf30dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c0c91f0 con 0x7faf2c156360 2023-12-26T21:56:39.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.521+0000 7faf30dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2c0c94d0 con 0x7faf2c156730 2023-12-26T21:56:39.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.522+0000 7faf2ad9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7faf2c06ee80 0x7faf2c0c89d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51640/0 (socket says 172.21.15.131:51640) 2023-12-26T21:56:39.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.522+0000 7faf2ad9d700 1 -- 172.21.15.131:0/311026073 learned_addr learned my addr 172.21.15.131:0/311026073 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:39.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.522+0000 7faf29d9b700 1 --2- 172.21.15.131:0/311026073 >> v2:172.21.15.190:3300/0 conn(0x7faf2c156730 0x7faf2c0c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:39.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.522+0000 7faf2959a700 1 -- 172.21.15.131:0/311026073 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623646850 0 0) 0x7faf2c0acfc0 con 0x7faf2c06ee80 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.522+0000 7faf2959a700 1 -- 172.21.15.131:0/311026073 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf14003570 con 0x7faf2c06ee80 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.522+0000 7faf29d9b700 1 -- 172.21.15.131:0/311026073 >> v1:172.21.15.190:6789/0 conn(0x7faf2c156360 legacy=0x7faf2c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.522+0000 7faf2959a700 1 -- 172.21.15.131:0/311026073 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293408121 0 0) 0x7faf2c0c91f0 con 0x7faf2c156360 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.522+0000 7faf29d9b700 1 -- 172.21.15.131:0/311026073 >> v1:172.21.15.131:6789/0 conn(0x7faf2c06ee80 legacy=0x7faf2c0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.522+0000 7faf29d9b700 1 -- 172.21.15.131:0/311026073 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0acc50 con 0x7faf2c156730 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.523+0000 7faf29d9b700 1 --2- 172.21.15.131:0/311026073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2c156730 0x7faf2c0c8490 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faf200098e0 tx=0x7faf200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10080714be2ac98 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.523+0000 7faf2959a700 1 -- 172.21.15.131:0/311026073 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000b640 con 0x7faf2c156730 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.523+0000 7faf2959a700 1 -- 172.21.15.131:0/311026073 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2000b7a0 con 0x7faf2c156730 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.523+0000 7faf2959a700 1 -- 172.21.15.131:0/311026073 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000bb40 con 0x7faf2c156730 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.523+0000 7faf30dff700 1 -- 172.21.15.131:0/311026073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2c156730 msgr2=0x7faf2c0c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.523+0000 7faf30dff700 1 --2- 172.21.15.131:0/311026073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2c156730 0x7faf2c0c8490 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faf200098e0 tx=0x7faf200091a0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.524+0000 7faf30dff700 1 -- 172.21.15.131:0/311026073 shutdown_connections 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.524+0000 7faf30dff700 1 --2- 172.21.15.131:0/311026073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2c156730 0x7faf2c0c8490 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.524+0000 7faf30dff700 1 -- 172.21.15.131:0/311026073 >> 172.21.15.131:0/311026073 conn(0x7faf2c0675b0 msgr2=0x7faf2c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:39.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.524+0000 7faf30dff700 1 -- 172.21.15.131:0/311026073 shutdown_connections 2023-12-26T21:56:39.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.524+0000 7faf30dff700 1 -- 172.21.15.131:0/311026073 wait complete. 2023-12-26T21:56:39.526 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:39.524+0000 7f3abe2b11c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:56:39.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.525+0000 7faf30dff700 1 Processor -- start 2023-12-26T21:56:39.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.525+0000 7faf30dff700 1 -- start start 2023-12-26T21:56:39.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.526+0000 7faf30dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2c06ee80 0x7faf2c1551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.526+0000 7faf30dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c156360 0x7faf2c0aa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.526+0000 7faf30dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2c156730 0x7faf2c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.526+0000 7faf30dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c15b360 con 0x7faf2c156360 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.526+0000 7faf30dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c1556f0 con 0x7faf2c156730 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.526+0000 7faf30dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0aad90 con 0x7faf2c06ee80 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf29d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c156360 0x7faf2c0aa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf29d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c156360 0x7faf2c0aa2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53354/0 (socket says 172.21.15.131:53354) 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf2a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2c06ee80 0x7faf2c1551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf29d9b700 1 -- 172.21.15.131:0/3412373603 learned_addr learned my addr 172.21.15.131:0/3412373603 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf2ad9d700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2c156730 0x7faf2c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf29d9b700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2c06ee80 msgr2=0x7faf2c1551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf29d9b700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2c06ee80 0x7faf2c1551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf29d9b700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2c156730 msgr2=0x7faf2c157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf29d9b700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2c156730 0x7faf2c157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:39.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf29d9b700 1 -- 172.21.15.131:0/3412373603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20009420 con 0x7faf2c156360 2023-12-26T21:56:39.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.527+0000 7faf29d9b700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c156360 0x7faf2c0aa2d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7faf200093f0 tx=0x7faf20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:39.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.528+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2001f070 con 0x7faf2c156360 2023-12-26T21:56:39.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.528+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf20000bc0 con 0x7faf2c156360 2023-12-26T21:56:39.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.528+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c0ab070 con 0x7faf2c156360 2023-12-26T21:56:39.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.528+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20025420 con 0x7faf2c156360 2023-12-26T21:56:39.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.528+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c157db0 con 0x7faf2c156360 2023-12-26T21:56:39.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.529+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c15b510 con 0x7faf2c156360 2023-12-26T21:56:39.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.530+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 63) v1 ==== 84759+0+0 (secure 0 0 0) 0x7faf20023070 con 0x7faf2c156360 2023-12-26T21:56:39.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.531+0000 7faf1a7fc700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:39.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.531+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6125+0+0 (secure 0 0 0) 0x7faf20085a30 con 0x7faf2c156360 2023-12-26T21:56:39.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.531+0000 7faf2a59c700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 msgr2=0x7faf0c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:39.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.531+0000 7faf2a59c700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T21:56:39.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.538+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf2c15b510 con 0x7faf2c156360 2023-12-26T21:56:39.596 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:39.595+0000 7fc2beadc1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T21:56:39.671 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:39.670+0000 7f3abe2b11c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:56:39.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.732+0000 7faf2a59c700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 msgr2=0x7faf0c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:39.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.732+0000 7faf2a59c700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T21:56:39.747 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:39.746+0000 7fc2beadc1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T21:56:39.762 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:39.760+0000 7f3abe2b11c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:56:39.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:39.764+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7faf2c15ae70 con 0x7faf0c066f80 2023-12-26T21:56:39.838 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:39.838+0000 7fc2beadc1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T21:56:39.944 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:39.942+0000 7f3abe2b11c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:56:40.021 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:40.020+0000 7fc2beadc1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T21:56:40.094 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:40.093+0000 7f3abe2b11c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:56:40.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:40.133+0000 7faf2a59c700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 msgr2=0x7faf0c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:40.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:40.133+0000 7faf2a59c700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T21:56:40.170 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:40.170+0000 7fc2beadc1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T21:56:40.474 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:40.474+0000 7f3abe2b11c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:56:40.554 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:40.553+0000 7fc2beadc1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T21:56:40.739 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:40.739+0000 7f3abe2b11c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:56:40.818 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:40.818+0000 7fc2beadc1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T21:56:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:40.934+0000 7faf2a59c700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 msgr2=0x7faf0c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:40.934+0000 7faf2a59c700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T21:56:41.023 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:41.023+0000 7f3abe2b11c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:56:41.096 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:41.095+0000 7fc2beadc1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T21:56:41.112 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:41.112+0000 7f3abe2b11c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:56:41.184 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:41.184+0000 7fc2beadc1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T21:56:41.846 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:41.846+0000 7f3abe2b11c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:56:41.934 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:41.934+0000 7fc2beadc1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T21:56:41.940 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:41.940+0000 7f3abe2b11c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:56:42.029 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:42.028+0000 7fc2beadc1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T21:56:42.092 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:42.091+0000 7f3abe2b11c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:56:42.181 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:42.181+0000 7fc2beadc1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T21:56:42.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:42.536+0000 7faf2a59c700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 msgr2=0x7faf0c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T21:56:42.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:42.536+0000 7faf2a59c700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T21:56:42.716 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:42.715+0000 7f3abe2b11c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:56:42.803 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:42.802+0000 7fc2beadc1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T21:56:42.903 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:42.903+0000 7f3abe2b11c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:56:42.991 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:42.991+0000 7fc2beadc1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T21:56:43.092 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:43.092+0000 7f3abe2b11c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:56:43.180 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:43.180+0000 7fc2beadc1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T21:56:43.931 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:43.931+0000 7f3abe2b11c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:56:44.024 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.023+0000 7fc2beadc1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T21:56:44.206 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:44.205+0000 7f3abe2b11c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:56:44.300 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.299+0000 7fc2beadc1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T21:56:44.300 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:44.300+0000 7f3abe2b11c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:56:44.395 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.394+0000 7fc2beadc1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T21:56:44.501 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T21:56:44.500+0000 7f3abe2b11c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:56:44.593 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.593+0000 7fc2beadc1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T21:56:44.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:44.602+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 64) v1 ==== 84470+0+0 (secure 0 0 0) 0x7faf20052e30 con 0x7faf2c156360 2023-12-26T21:56:44.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:44.603+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 msgr2=0x7faf0c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T21:56:44.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:44.603+0000 7faf1a7fc700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:44.738 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.737+0000 7fc230471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:44.738 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.737+0000 7fc230471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:44.738 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.737+0000 7fc230471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:44.738 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.737+0000 7fc230471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:44.738 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.737+0000 7fc230471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:44.738 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.737+0000 7fc233477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:44.739 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.738+0000 7fc233477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:44.739 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.738+0000 7fc233477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:44.739 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.738+0000 7fc233477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:44.739 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:56:44.738+0000 7fc233477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:56:45.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.608+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 65) v1 ==== 84597+0+0 (secure 0 0 0) 0x7faf2005d230 con 0x7faf2c156360 2023-12-26T21:56:45.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.608+0000 7faf1a7fc700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:45.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.608+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7faf0c00a3f0 con 0x7faf0c066f80 2023-12-26T21:56:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.612+0000 7faf2a59c700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:45.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.616+0000 7faf2a59c700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf1c007fa0 tx=0x7faf1c007ce0 comp rx=0 tx=0).ready entity=mgr.7921 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:45.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.675+0000 7faf1a7fc700 1 -- 172.21.15.131:0/3412373603 <== mgr.7921 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7faf0c00a3f0 con 0x7faf0c066f80 2023-12-26T21:56:45.676 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 msgr2=0x7faf0c069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf1c007fa0 tx=0x7faf1c007ce0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c156360 msgr2=0x7faf2c0aa2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c156360 0x7faf2c0aa2d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7faf200093f0 tx=0x7faf20017000 comp rx=0 tx=0).stop 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 shutdown_connections 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf2c156730 0x7faf2c157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf2c06ee80 0x7faf2c1551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faf0c066f80 0x7faf0c069430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 --2- 172.21.15.131:0/3412373603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c156360 0x7faf2c0aa2d0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 >> 172.21.15.131:0/3412373603 conn(0x7faf2c0675b0 msgr2=0x7faf2c067980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 shutdown_connections 2023-12-26T21:56:45.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.678+0000 7faf30dff700 1 -- 172.21.15.131:0/3412373603 wait complete. 2023-12-26T21:56:45.698 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-12-26T21:56:45.698 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs /snap_test_dir1 55n 2023-12-26T21:56:45.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.894+0000 7f602a5ff700 1 Processor -- start 2023-12-26T21:56:45.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.894+0000 7f602a5ff700 1 -- start start 2023-12-26T21:56:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f602a5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f602413c510 0x7f60240c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f602a5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60240cc810 0x7f602406bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f602a5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60240686c0 con 0x7f602406b8c0 2023-12-26T21:56:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f602a5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602406c280 con 0x7f602413c510 2023-12-26T21:56:45.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f602a5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6024069520 con 0x7f60240cc810 2023-12-26T21:56:45.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f601bfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f602406b8c0 0x7f60240cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32942/0 (socket says 172.21.15.131:32942) 2023-12-26T21:56:45.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f601bfff700 1 -- 172.21.15.131:0/3723544195 learned_addr learned my addr 172.21.15.131:0/3723544195 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T21:56:45.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f6023fff700 1 --2- 172.21.15.131:0/3723544195 >> v2:172.21.15.190:3300/0 conn(0x7f602413c510 0x7f60240c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:45.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f6028b9c700 1 --2- 172.21.15.131:0/3723544195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60240cc810 0x7f602406bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:45.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f60237fe700 1 -- 172.21.15.131:0/3723544195 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676437664 0 0) 0x7f60240686c0 con 0x7f602406b8c0 2023-12-26T21:56:45.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f60237fe700 1 -- 172.21.15.131:0/3723544195 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6010003570 con 0x7f602406b8c0 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f6028b9c700 1 -- 172.21.15.131:0/3723544195 >> v2:172.21.15.190:3300/0 conn(0x7f602413c510 msgr2=0x7f60240c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f6028b9c700 1 --2- 172.21.15.131:0/3723544195 >> v2:172.21.15.190:3300/0 conn(0x7f602413c510 0x7f60240c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.896+0000 7f6028b9c700 1 -- 172.21.15.131:0/3723544195 >> v1:172.21.15.131:6789/0 conn(0x7f602406b8c0 legacy=0x7f60240cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.897+0000 7f6028b9c700 1 -- 172.21.15.131:0/3723544195 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6024068910 con 0x7f60240cc810 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.897+0000 7f6028b9c700 1 --2- 172.21.15.131:0/3723544195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60240cc810 0x7f602406bd40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f601c009e60 tx=0x7f601c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61631aee5d6f3eef server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.897+0000 7f60237fe700 1 -- 172.21.15.131:0/3723544195 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00e070 con 0x7f60240cc810 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.897+0000 7f60237fe700 1 -- 172.21.15.131:0/3723544195 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00bab0 con 0x7f60240cc810 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.897+0000 7f60237fe700 1 -- 172.21.15.131:0/3723544195 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00be70 con 0x7f60240cc810 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.897+0000 7f602a5ff700 1 -- 172.21.15.131:0/3723544195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60240cc810 msgr2=0x7f602406bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.897+0000 7f602a5ff700 1 --2- 172.21.15.131:0/3723544195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60240cc810 0x7f602406bd40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f601c009e60 tx=0x7f601c009780 comp rx=0 tx=0).stop 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.898+0000 7f602a5ff700 1 -- 172.21.15.131:0/3723544195 shutdown_connections 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.898+0000 7f602a5ff700 1 --2- 172.21.15.131:0/3723544195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60240cc810 0x7f602406bd40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.898+0000 7f602a5ff700 1 --2- 172.21.15.131:0/3723544195 >> v2:172.21.15.190:3300/0 conn(0x7f602413c510 0x7f60240c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.898+0000 7f602a5ff700 1 -- 172.21.15.131:0/3723544195 >> 172.21.15.131:0/3723544195 conn(0x7f60240ce3e0 msgr2=0x7f60240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:45.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.898+0000 7f602a5ff700 1 -- 172.21.15.131:0/3723544195 shutdown_connections 2023-12-26T21:56:45.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.898+0000 7f602a5ff700 1 -- 172.21.15.131:0/3723544195 wait complete. 2023-12-26T21:56:45.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.900+0000 7f602a5ff700 1 Processor -- start 2023-12-26T21:56:45.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.900+0000 7f602a5ff700 1 -- start start 2023-12-26T21:56:45.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.901+0000 7f602a5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f602406b8c0 0x7f602413eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.901+0000 7f602a5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60240cc810 0x7f6024153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.901+0000 7f602a5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f602413c510 0x7f602413dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.901+0000 7f602a5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6024153d90 con 0x7f60240cc810 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.901+0000 7f602a5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60241549e0 con 0x7f602406b8c0 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.901+0000 7f602a5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6024154400 con 0x7f602413c510 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.901+0000 7f601bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60240cc810 0x7f6024153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.901+0000 7f601bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60240cc810 0x7f6024153850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52532/0 (socket says 172.21.15.131:52532) 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f601bfff700 1 -- 172.21.15.131:0/3777558335 learned_addr learned my addr 172.21.15.131:0/3777558335 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f6023fff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f602406b8c0 0x7f602413eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:45.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f6028b9c700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f602413c510 0x7f602413dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f601bfff700 1 -- 172.21.15.131:0/3777558335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f602413c510 msgr2=0x7f602413dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f601bfff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f602413c510 0x7f602413dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f601bfff700 1 -- 172.21.15.131:0/3777558335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f602406b8c0 msgr2=0x7f602413eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f601bfff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f602406b8c0 0x7f602413eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f601bfff700 1 -- 172.21.15.131:0/3777558335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c009a20 con 0x7f60240cc810 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f601bfff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60240cc810 0x7f6024153850 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f600c00c5c0 tx=0x7f600c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f6020ff9700 1 -- 172.21.15.131:0/3777558335 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c005b30 con 0x7f60240cc810 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f6020ff9700 1 -- 172.21.15.131:0/3777558335 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c005cb0 con 0x7f60240cc810 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f6020ff9700 1 -- 172.21.15.131:0/3777558335 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c007320 con 0x7f60240cc810 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.902+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602413f510 con 0x7f60240cc810 2023-12-26T21:56:45.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.903+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602413e1c0 con 0x7f60240cc810 2023-12-26T21:56:45.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.904+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6024154cc0 con 0x7f60240cc810 2023-12-26T21:56:45.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.904+0000 7f6020ff9700 1 -- 172.21.15.131:0/3777558335 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 65) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f600c01b3b0 con 0x7f60240cc810 2023-12-26T21:56:45.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.905+0000 7f6020ff9700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6004066dd0 0x7f6004069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:45.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.905+0000 7f6020ff9700 1 -- 172.21.15.131:0/3777558335 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6383+0+0 (secure 0 0 0) 0x7f600c0839a0 con 0x7f60240cc810 2023-12-26T21:56:45.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.906+0000 7f6023fff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6004066dd0 0x7f6004069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:45.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.906+0000 7f6023fff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6004066dd0 0x7f6004069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6014009da0 tx=0x7f6014009120 comp rx=0 tx=0).ready entity=mgr.7921 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:45.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:45.910+0000 7f6020ff9700 1 -- 172.21.15.131:0/3777558335 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f600c050b50 con 0x7f60240cc810 2023-12-26T21:56:46.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.127+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f602406c8d0 con 0x7f6004066dd0 2023-12-26T21:56:46.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.178+0000 7f6020ff9700 1 -- 172.21.15.131:0/3777558335 <== mgr.7921 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f602406c8d0 con 0x7f6004066dd0 2023-12-26T21:56:46.179 INFO:teuthology.orchestra.run.smithi131.stdout:Retention added to path /snap_test_dir1 2023-12-26T21:56:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6004066dd0 msgr2=0x7f6004069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6004066dd0 0x7f6004069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6014009da0 tx=0x7f6014009120 comp rx=0 tx=0).stop 2023-12-26T21:56:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60240cc810 msgr2=0x7f6024153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60240cc810 0x7f6024153850 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f600c00c5c0 tx=0x7f600c007e00 comp rx=0 tx=0).stop 2023-12-26T21:56:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 shutdown_connections 2023-12-26T21:56:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f602413c510 0x7f602413dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f602406b8c0 0x7f602413eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6004066dd0 0x7f6004069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 --2- 172.21.15.131:0/3777558335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60240cc810 0x7f6024153850 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 >> 172.21.15.131:0/3777558335 conn(0x7f60240ce3e0 msgr2=0x7f60240c3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:46.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 shutdown_connections 2023-12-26T21:56:46.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.181+0000 7f602a5ff700 1 -- 172.21.15.131:0/3777558335 wait complete. 2023-12-26T21:56:46.199 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-12-26T21:56:46.200 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-12-26T21:56:46.200 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json /snap_test_dir1 2023-12-26T21:56:46.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.393+0000 7f9f6b1ff700 1 Processor -- start 2023-12-26T21:56:46.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.393+0000 7f9f6b1ff700 1 -- start start 2023-12-26T21:56:46.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.394+0000 7f9f6b1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f641593f0 0x7f9f640abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:46.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.394+0000 7f9f6b1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 0x7f9f6406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:46.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.394+0000 7f9f6b1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f64067ff0 con 0x7f9f641597c0 2023-12-26T21:56:46.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.394+0000 7f9f6b1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f64068480 con 0x7f9f641593f0 2023-12-26T21:56:46.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.394+0000 7f9f6b1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6406a820 con 0x7f9f64069e60 2023-12-26T21:56:46.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.394+0000 7f9f68f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f641593f0 0x7f9f640abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:46.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.394+0000 7f9f68f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f641593f0 0x7f9f640abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52542/0 (socket says 172.21.15.131:52542) 2023-12-26T21:56:46.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.394+0000 7f9f68f9b700 1 -- 172.21.15.131:0/3394029972 learned_addr learned my addr 172.21.15.131:0/3394029972 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:46.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.394+0000 7f9f6979c700 1 --2- 172.21.15.131:0/3394029972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 0x7f9f6406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:46.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.395+0000 7f9f68f9b700 1 -- 172.21.15.131:0/3394029972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 msgr2=0x7f9f6406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T21:56:46.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.395+0000 7f9f68f9b700 1 --2- 172.21.15.131:0/3394029972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 0x7f9f6406a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.395+0000 7f9f68f9b700 1 -- 172.21.15.131:0/3394029972 >> v1:172.21.15.190:6789/0 conn(0x7f9f641597c0 legacy=0x7f9f640aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:46.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.395+0000 7f9f68f9b700 1 -- 172.21.15.131:0/3394029972 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640ac350 con 0x7f9f641593f0 2023-12-26T21:56:46.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.395+0000 7f9f68f9b700 1 --2- 172.21.15.131:0/3394029972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641593f0 0x7f9f640abc90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f9f580099b0 tx=0x7f9f58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab1234fbc4943e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.395+0000 7f9f637fe700 1 -- 172.21.15.131:0/3394029972 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58009ea0 con 0x7f9f641593f0 2023-12-26T21:56:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.395+0000 7f9f637fe700 1 -- 172.21.15.131:0/3394029972 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5800b860 con 0x7f9f641593f0 2023-12-26T21:56:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.395+0000 7f9f637fe700 1 -- 172.21.15.131:0/3394029972 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800bc00 con 0x7f9f641593f0 2023-12-26T21:56:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.396+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/3394029972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641593f0 msgr2=0x7f9f640abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.396+0000 7f9f6b1ff700 1 --2- 172.21.15.131:0/3394029972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641593f0 0x7f9f640abc90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f9f580099b0 tx=0x7f9f58009250 comp rx=0 tx=0).stop 2023-12-26T21:56:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.396+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/3394029972 shutdown_connections 2023-12-26T21:56:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.396+0000 7f9f6b1ff700 1 --2- 172.21.15.131:0/3394029972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 0x7f9f6406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.396+0000 7f9f6b1ff700 1 --2- 172.21.15.131:0/3394029972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641593f0 0x7f9f640abc90 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.396+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/3394029972 >> 172.21.15.131:0/3394029972 conn(0x7f9f6406b710 msgr2=0x7f9f6406bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.396+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/3394029972 shutdown_connections 2023-12-26T21:56:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.396+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/3394029972 wait complete. 2023-12-26T21:56:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.397+0000 7f9f6b1ff700 1 Processor -- start 2023-12-26T21:56:46.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.398+0000 7f9f6b1ff700 1 -- start start 2023-12-26T21:56:46.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.398+0000 7f9f6b1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 0x7f9f6416aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6b1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641593f0 0x7f9f6416cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6b1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641597c0 0x7f9f64169dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6b1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640b0700 con 0x7f9f641597c0 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6b1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640b0cc0 con 0x7f9f641593f0 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6b1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64159f60 con 0x7f9f64069e60 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6979c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641597c0 0x7f9f64169dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6979c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641597c0 0x7f9f64169dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52556/0 (socket says 172.21.15.131:52556) 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6979c700 1 -- 172.21.15.131:0/2193544941 learned_addr learned my addr 172.21.15.131:0/2193544941 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f68f9b700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 0x7f9f6416aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f63fff700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641593f0 0x7f9f6416cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6979c700 1 -- 172.21.15.131:0/2193544941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 msgr2=0x7f9f6416aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:46.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6979c700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 0x7f9f6416aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6979c700 1 -- 172.21.15.131:0/2193544941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641593f0 msgr2=0x7f9f6416cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6979c700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641593f0 0x7f9f6416cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.399+0000 7f9f6979c700 1 -- 172.21.15.131:0/2193544941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580094f0 con 0x7f9f641597c0 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.400+0000 7f9f6979c700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641597c0 0x7f9f64169dc0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009ec0 tx=0x7f9f5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.400+0000 7f9f60ff9700 1 -- 172.21.15.131:0/2193544941 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00cb70 con 0x7f9f641597c0 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.400+0000 7f9f60ff9700 1 -- 172.21.15.131:0/2193544941 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c00f040 con 0x7f9f641597c0 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.400+0000 7f9f60ff9700 1 -- 172.21.15.131:0/2193544941 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c014480 con 0x7f9f641597c0 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.400+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6416a950 con 0x7f9f641597c0 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.400+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f640b0950 con 0x7f9f641597c0 2023-12-26T21:56:46.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.401+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f640b0e50 con 0x7f9f641597c0 2023-12-26T21:56:46.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.402+0000 7f9f60ff9700 1 -- 172.21.15.131:0/2193544941 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 65) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f9f5c014830 con 0x7f9f641597c0 2023-12-26T21:56:46.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.403+0000 7f9f60ff9700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f48066dd0 0x7f9f48069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T21:56:46.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.403+0000 7f9f60ff9700 1 -- 172.21.15.131:0/2193544941 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6383+0+0 (secure 0 0 0) 0x7f9f5c084d80 con 0x7f9f641597c0 2023-12-26T21:56:46.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.403+0000 7f9f68f9b700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f48066dd0 0x7f9f48069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T21:56:46.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.404+0000 7f9f68f9b700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f48066dd0 0x7f9f48069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9f580099b0 tx=0x7f9f58017000 comp rx=0 tx=0).ready entity=mgr.7921 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T21:56:46.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.407+0000 7f9f60ff9700 1 -- 172.21.15.131:0/2193544941 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f5c052000 con 0x7f9f641597c0 2023-12-26T21:56:46.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.614+0000 7f9f60ff9700 1 -- 172.21.15.131:0/2193544941 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 66) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f9f5c0523d0 con 0x7f9f641597c0 2023-12-26T21:56:46.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.629+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f640717f0 con 0x7f9f48066dd0 2023-12-26T21:56:46.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.637+0000 7f9f60ff9700 1 -- 172.21.15.131:0/2193544941 <== mgr.7921 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f9f640717f0 con 0x7f9f48066dd0 2023-12-26T21:56:46.638 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T21:56:46.638 INFO:teuthology.orchestra.run.smithi131.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-12-26T21:56:46.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f48066dd0 msgr2=0x7f9f48069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:46.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f48066dd0 0x7f9f48069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9f580099b0 tx=0x7f9f58017000 comp rx=0 tx=0).stop 2023-12-26T21:56:46.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641597c0 msgr2=0x7f9f64169dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T21:56:46.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641597c0 0x7f9f64169dc0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009ec0 tx=0x7f9f5c00b040 comp rx=0 tx=0).stop 2023-12-26T21:56:46.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 shutdown_connections 2023-12-26T21:56:46.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9f48066dd0 0x7f9f48069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f641597c0 0x7f9f64169dc0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64069e60 0x7f9f6416aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 --2- 172.21.15.131:0/2193544941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641593f0 0x7f9f6416cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T21:56:46.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.639+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 >> 172.21.15.131:0/2193544941 conn(0x7f9f6406b710 msgr2=0x7f9f6406bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T21:56:46.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.640+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 shutdown_connections 2023-12-26T21:56:46.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T21:56:46.640+0000 7f9f6b1ff700 1 -- 172.21.15.131:0/2193544941 wait complete. 2023-12-26T21:56:46.658 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-12-26T21:56:46.658 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-12-26T21:56:49.557 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:56:49.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:57:00.192 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:57:00.191+0000 7fc222455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:57:00.192 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:57:00.191+0000 7fc222455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:57:00.192 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:57:00.191+0000 7fc222455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:57:00.192 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:57:00.191+0000 7fc222455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:57:00.192 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T21:57:00.191+0000 7fc222455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T21:57:19.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:57:19.693 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:57:49.812 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:57:49.817 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:58:19.932 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:58:19.937 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:58:50.136 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:58:50.142 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:59:20.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:59:20.470 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:59:50.573 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T21:59:50.580 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:00:20.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:00:20.980 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:00:51.266 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:00:51.272 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:01:21.395 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:01:21.401 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:01:51.509 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:01:51.514 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:02:21.708 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:02:21.714 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:02:51.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:02:51.959 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:03:22.233 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:03:22.238 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:03:52.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:03:52.552 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:04:22.660 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:04:22.666 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:04:52.777 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:04:52.782 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:05:22.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:05:22.901 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:05:53.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:05:53.250 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:06:23.372 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:06:23.377 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:06:53.662 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:06:53.668 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:07:23.841 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:07:23.846 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:07:53.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:07:53.968 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:08:24.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:08:24.099 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:08:54.207 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:08:54.212 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:09:24.321 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:09:24.334 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:09:54.442 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:09:54.447 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:10:24.631 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:10:24.637 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:10:54.752 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:10:54.758 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:11:24.889 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:11:24.894 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:11:55.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:11:55.031 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:12:25.139 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:12:25.144 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:12:55.255 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:12:55.260 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:13:25.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:13:25.587 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:13:55.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:13:55.762 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:14:26.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:14:26.102 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:14:56.208 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:14:56.214 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:15:26.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:15:26.361 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:15:56.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:15:56.690 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:16:27.028 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:16:27.034 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:16:57.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:16:57.816 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:17:27.925 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:17:27.930 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:17:58.040 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:17:58.046 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:18:28.154 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:18:28.159 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:18:58.266 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:18:58.271 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:19:28.381 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:19:28.387 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:19:58.504 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:19:58.509 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:20:28.617 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:20:28.623 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:20:58.781 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:20:58.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:21:28.936 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:21:28.942 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:21:59.158 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:21:59.164 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:22:29.479 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:22:29.485 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:22:59.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:22:59.816 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:23:29.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:23:29.946 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:24:00.065 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:24:00.070 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:24:30.183 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:24:30.189 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:25:00.300 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:25:00.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:25:30.544 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:25:30.550 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:26:00.685 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:26:00.690 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:26:31.008 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:26:31.013 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:27:01.126 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:27:01.132 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:27:31.251 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:27:31.256 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:28:01.407 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:28:01.413 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:28:31.785 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:28:31.790 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:29:02.089 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:29:02.095 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:29:32.448 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:29:32.453 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:30:02.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:30:02.730 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:30:32.838 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:30:32.843 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:31:02.994 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:31:03.000 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:31:33.282 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:31:33.288 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:32:03.396 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:32:03.402 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:32:33.527 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:32:33.532 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:33:03.966 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:33:03.972 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:33:34.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:33:34.094 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:34:04.204 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:34:04.210 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:34:34.318 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:34:34.325 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:35:04.432 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:35:04.438 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:35:34.593 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:35:34.599 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:36:04.710 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:36:04.716 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:36:35.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:36:35.035 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:37:05.227 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:37:05.234 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:37:35.341 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:37:35.347 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:38:05.479 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:38:05.484 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:38:35.600 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:38:35.606 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:39:05.716 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:39:05.722 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:39:35.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:39:35.842 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:40:05.982 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:40:05.987 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:40:36.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:40:36.105 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:41:06.230 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:41:06.236 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:41:36.366 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:41:36.374 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:42:06.496 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:42:06.503 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:42:36.609 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:42:36.614 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:43:06.727 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:43:06.732 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:43:36.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:43:36.855 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:44:06.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:44:06.969 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:44:37.081 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:44:37.087 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:45:07.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:45:07.205 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:45:37.608 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:45:37.615 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:46:07.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:46:07.736 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:46:38.035 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:46:38.041 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:47:08.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:47:08.154 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:47:38.270 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:47:38.275 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:48:08.399 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:48:08.405 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:48:38.529 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:48:38.534 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:49:08.644 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:49:08.651 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:49:38.765 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:49:38.770 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:50:08.876 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:50:08.881 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:50:39.026 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:50:39.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:51:09.150 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:51:09.158 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:51:39.305 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:51:39.315 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:52:09.427 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:52:09.434 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:52:15.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:15.660 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T22:52:15.715 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_58_00_UTC 2023-12-26T22:52:15.716 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_59_00_UTC 2023-12-26T22:52:15.716 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_00_00_UTC 2023-12-26T22:52:15.716 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_01_00_UTC 2023-12-26T22:52:15.716 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_02_00_UTC 2023-12-26T22:52:15.716 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_03_00_UTC 2023-12-26T22:52:15.716 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_04_00_UTC 2023-12-26T22:52:15.716 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_05_00_UTC 2023-12-26T22:52:15.716 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_06_00_UTC 2023-12-26T22:52:15.716 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_07_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_08_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_09_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_10_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_11_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_12_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_13_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_14_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_15_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_16_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_17_00_UTC 2023-12-26T22:52:15.717 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_18_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_19_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_20_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_21_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_22_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_23_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_24_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_25_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_26_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_27_00_UTC 2023-12-26T22:52:15.718 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_28_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_29_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_30_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_31_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_32_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_33_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_34_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_35_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_36_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_37_00_UTC 2023-12-26T22:52:15.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_38_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_39_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_40_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_41_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_42_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_43_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_44_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_45_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_46_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_47_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_48_00_UTC 2023-12-26T22:52:15.720 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_49_00_UTC 2023-12-26T22:52:15.721 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_50_00_UTC 2023-12-26T22:52:15.721 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_51_00_UTC 2023-12-26T22:52:15.721 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_52_00_UTC 2023-12-26T22:52:15.721 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-21_58_00_UTC', 'scheduled-2023-12-26-21_59_00_UTC', 'scheduled-2023-12-26-22_00_00_UTC', 'scheduled-2023-12-26-22_01_00_UTC', 'scheduled-2023-12-26-22_02_00_UTC', 'scheduled-2023-12-26-22_03_00_UTC', 'scheduled-2023-12-26-22_04_00_UTC', 'scheduled-2023-12-26-22_05_00_UTC', 'scheduled-2023-12-26-22_06_00_UTC', 'scheduled-2023-12-26-22_07_00_UTC', 'scheduled-2023-12-26-22_08_00_UTC', 'scheduled-2023-12-26-22_09_00_UTC', 'scheduled-2023-12-26-22_10_00_UTC', 'scheduled-2023-12-26-22_11_00_UTC', 'scheduled-2023-12-26-22_12_00_UTC', 'scheduled-2023-12-26-22_13_00_UTC', 'scheduled-2023-12-26-22_14_00_UTC', 'scheduled-2023-12-26-22_15_00_UTC', 'scheduled-2023-12-26-22_16_00_UTC', 'scheduled-2023-12-26-22_17_00_UTC', 'scheduled-2023-12-26-22_18_00_UTC', 'scheduled-2023-12-26-22_19_00_UTC', 'scheduled-2023-12-26-22_20_00_UTC', 'scheduled-2023-12-26-22_21_00_UTC', 'scheduled-2023-12-26-22_22_00_UTC', 'scheduled-2023-12-26-22_23_00_UTC', 'scheduled-2023-12-26-22_24_00_UTC', 'scheduled-2023-12-26-22_25_00_UTC', 'scheduled-2023-12-26-22_26_00_UTC', 'scheduled-2023-12-26-22_27_00_UTC', 'scheduled-2023-12-26-22_28_00_UTC', 'scheduled-2023-12-26-22_29_00_UTC', 'scheduled-2023-12-26-22_30_00_UTC', 'scheduled-2023-12-26-22_31_00_UTC', 'scheduled-2023-12-26-22_32_00_UTC', 'scheduled-2023-12-26-22_33_00_UTC', 'scheduled-2023-12-26-22_34_00_UTC', 'scheduled-2023-12-26-22_35_00_UTC', 'scheduled-2023-12-26-22_36_00_UTC', 'scheduled-2023-12-26-22_37_00_UTC', 'scheduled-2023-12-26-22_38_00_UTC', 'scheduled-2023-12-26-22_39_00_UTC', 'scheduled-2023-12-26-22_40_00_UTC', 'scheduled-2023-12-26-22_41_00_UTC', 'scheduled-2023-12-26-22_42_00_UTC', 'scheduled-2023-12-26-22_43_00_UTC', 'scheduled-2023-12-26-22_44_00_UTC', 'scheduled-2023-12-26-22_45_00_UTC', 'scheduled-2023-12-26-22_46_00_UTC', 'scheduled-2023-12-26-22_47_00_UTC', 'scheduled-2023-12-26-22_48_00_UTC', 'scheduled-2023-12-26-22_49_00_UTC', 'scheduled-2023-12-26-22_50_00_UTC', 'scheduled-2023-12-26-22_51_00_UTC', 'scheduled-2023-12-26-22_52_00_UTC'] 2023-12-26T22:52:15.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1 2023-12-26T22:52:15.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.920+0000 7f3884fff700 1 Processor -- start 2023-12-26T22:52:15.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.920+0000 7f3884fff700 1 -- start start 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.922+0000 7f3884fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3880157730 0x7f38800ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.922+0000 7f3884fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f38800ae550 0x7f38800c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.922+0000 7f3884fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388006e110 con 0x7f3880157360 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.922+0000 7f3884fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38800c6930 con 0x7f3880157730 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.922+0000 7f3884fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38800ae990 con 0x7f38800ae550 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.922+0000 7f387dd9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3880157730 0x7f38800ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.922+0000 7f387e59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3880157360 0x7f388015ba10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48688/0 (socket says 172.21.15.131:48688) 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.922+0000 7f387e59c700 1 -- 172.21.15.131:0/4251713929 learned_addr learned my addr 172.21.15.131:0/4251713929 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.922+0000 7f387ed9d700 1 --2- 172.21.15.131:0/4251713929 >> v2:172.21.15.190:3300/0 conn(0x7f38800ae550 0x7f38800c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.923+0000 7f387dd9b700 1 -- 172.21.15.131:0/4251713929 >> v2:172.21.15.190:3300/0 conn(0x7f38800ae550 msgr2=0x7f38800c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.923+0000 7f387dd9b700 1 --2- 172.21.15.131:0/4251713929 >> v2:172.21.15.190:3300/0 conn(0x7f38800ae550 0x7f38800c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.923+0000 7f387dd9b700 1 -- 172.21.15.131:0/4251713929 >> v1:172.21.15.131:6789/0 conn(0x7f3880157360 legacy=0x7f388015ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.923+0000 7f387d59a700 1 -- 172.21.15.131:0/4251713929 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161193674 0 0) 0x7f388006e110 con 0x7f3880157360 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.923+0000 7f387dd9b700 1 -- 172.21.15.131:0/4251713929 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38800c6500 con 0x7f3880157730 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.923+0000 7f387dd9b700 1 --2- 172.21.15.131:0/4251713929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157730 0x7f38800ae010 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f388006b9a0 tx=0x7f38740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5797af03f2d17f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.923+0000 7f387d59a700 1 -- 172.21.15.131:0/4251713929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400b6b0 con 0x7f3880157730 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.923+0000 7f387d59a700 1 -- 172.21.15.131:0/4251713929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387400b810 con 0x7f3880157730 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.923+0000 7f387d59a700 1 -- 172.21.15.131:0/4251713929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400bbb0 con 0x7f3880157730 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.924+0000 7f3884fff700 1 -- 172.21.15.131:0/4251713929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157730 msgr2=0x7f38800ae010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.924+0000 7f3884fff700 1 --2- 172.21.15.131:0/4251713929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157730 0x7f38800ae010 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f388006b9a0 tx=0x7f38740091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:15.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.924+0000 7f3884fff700 1 -- 172.21.15.131:0/4251713929 shutdown_connections 2023-12-26T22:52:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.924+0000 7f3884fff700 1 --2- 172.21.15.131:0/4251713929 >> v2:172.21.15.190:3300/0 conn(0x7f38800ae550 0x7f38800c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.924+0000 7f3884fff700 1 --2- 172.21.15.131:0/4251713929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157730 0x7f38800ae010 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.924+0000 7f3884fff700 1 -- 172.21.15.131:0/4251713929 >> 172.21.15.131:0/4251713929 conn(0x7f3880068020 msgr2=0x7f38800c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.924+0000 7f3884fff700 1 -- 172.21.15.131:0/4251713929 shutdown_connections 2023-12-26T22:52:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.924+0000 7f3884fff700 1 -- 172.21.15.131:0/4251713929 wait complete. 2023-12-26T22:52:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.925+0000 7f3884fff700 1 Processor -- start 2023-12-26T22:52:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.926+0000 7f3884fff700 1 -- start start 2023-12-26T22:52:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.926+0000 7f3884fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38800ae550 0x7f388014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.926+0000 7f3884fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157360 0x7f38800af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.926+0000 7f3884fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3880157730 0x7f38800c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.926+0000 7f3884fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3880167360 con 0x7f3880157360 2023-12-26T22:52:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.926+0000 7f3884fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3880158170 con 0x7f38800ae550 2023-12-26T22:52:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.926+0000 7f3884fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388014fac0 con 0x7f3880157730 2023-12-26T22:52:15.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38800ae550 0x7f388014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157360 0x7f38800af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157360 0x7f38800af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57590/0 (socket says 172.21.15.131:57590) 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38800ae550 0x7f388014f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:55792/0 (socket says 172.21.15.131:55792) 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387dd9b700 1 -- 172.21.15.131:0/1962624764 learned_addr learned my addr 172.21.15.131:0/1962624764 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387ed9d700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3880157730 0x7f38800c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387dd9b700 1 -- 172.21.15.131:0/1962624764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3880157730 msgr2=0x7f38800c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387dd9b700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3880157730 0x7f38800c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387dd9b700 1 -- 172.21.15.131:0/1962624764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38800ae550 msgr2=0x7f388014f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387dd9b700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38800ae550 0x7f388014f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387dd9b700 1 -- 172.21.15.131:0/1962624764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874009420 con 0x7f3880157360 2023-12-26T22:52:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.927+0000 7f387dd9b700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157360 0x7f38800af900 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f387400b0e0 tx=0x7f3874017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.928+0000 7f386e7fc700 1 -- 172.21.15.131:0/1962624764 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874000d10 con 0x7f3880157360 2023-12-26T22:52:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.928+0000 7f386e7fc700 1 -- 172.21.15.131:0/1962624764 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387400bb90 con 0x7f3880157360 2023-12-26T22:52:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.928+0000 7f386e7fc700 1 -- 172.21.15.131:0/1962624764 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874011690 con 0x7f3880157360 2023-12-26T22:52:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.928+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38800afe40 con 0x7f3880157360 2023-12-26T22:52:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.928+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38800b0d90 con 0x7f3880157360 2023-12-26T22:52:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.929+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3880158320 con 0x7f3880157360 2023-12-26T22:52:15.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.930+0000 7f386e7fc700 1 -- 172.21.15.131:0/1962624764 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 68) v1 ==== 84873+0+0 (secure 0 0 0) 0x7f3874011ac0 con 0x7f3880157360 2023-12-26T22:52:15.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.931+0000 7f386e7fc700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f38600670a0 0x7f3860069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:15.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.931+0000 7f386e7fc700 1 -- 172.21.15.131:0/1962624764 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f3874086210 con 0x7f3880157360 2023-12-26T22:52:15.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.931+0000 7f387e59c700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f38600670a0 0x7f3860069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:15.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.931+0000 7f387e59c700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f38600670a0 0x7f3860069550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38800b20c0 tx=0x7f3870007d30 comp rx=0 tx=0).ready entity=mgr.7921 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:15.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:15.935+0000 7f386e7fc700 1 -- 172.21.15.131:0/1962624764 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3880158320 con 0x7f3880157360 2023-12-26T22:52:16.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.156+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3880158320 con 0x7f38600670a0 2023-12-26T22:52:16.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.165+0000 7f386e7fc700 1 -- 172.21.15.131:0/1962624764 <== mgr.7921 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f3880158320 con 0x7f38600670a0 2023-12-26T22:52:16.165 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:56:45", "first": "2023-12-26T21:57:00", "last": "2023-12-26T22:52:00", "last_pruned": "2023-12-26T22:52:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-12-26T22:52:16.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.167+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f38600670a0 msgr2=0x7f3860069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.167+0000 7f3884fff700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f38600670a0 0x7f3860069550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38800b20c0 tx=0x7f3870007d30 comp rx=0 tx=0).stop 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.167+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157360 msgr2=0x7f38800af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.167+0000 7f3884fff700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157360 0x7f38800af900 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f387400b0e0 tx=0x7f3874017000 comp rx=0 tx=0).stop 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.168+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 shutdown_connections 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.168+0000 7f3884fff700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3880157730 0x7f38800c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.168+0000 7f3884fff700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38800ae550 0x7f388014f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.168+0000 7f3884fff700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f38600670a0 0x7f3860069550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.168+0000 7f3884fff700 1 --2- 172.21.15.131:0/1962624764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3880157360 0x7f38800af900 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.168+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 >> 172.21.15.131:0/1962624764 conn(0x7f3880068020 msgr2=0x7f38801504f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:16.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.168+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 shutdown_connections 2023-12-26T22:52:16.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.168+0000 7f3884fff700 1 -- 172.21.15.131:0/1962624764 wait complete. 2023-12-26T22:52:16.185 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T21:56:45", "first": "2023-12-26T21:57:00", "last": "2023-12-26T22:52:00", "last_pruned": "2023-12-26T22:52:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-12-26T22:52:16.186 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T21:56:45', 'first': '2023-12-26T21:57:00', 'last': '2023-12-26T22:52:00', 'last_pruned': '2023-12-26T22:52:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-12-26T22:52:16.186 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-12-26T22:52:16.186 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-12-26T22:52:16.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1 2023-12-26T22:52:16.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.382+0000 7f3e81dff700 1 Processor -- start 2023-12-26T22:52:16.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.382+0000 7f3e81dff700 1 -- start start 2023-12-26T22:52:16.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.383+0000 7f3e81dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3e7c156730 0x7f3e7c0c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:16.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.383+0000 7f3e81dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c06ee80 0x7f3e7c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:16.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.383+0000 7f3e81dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c0acfc0 con 0x7f3e7c156360 2023-12-26T22:52:16.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.383+0000 7f3e81dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c0c91f0 con 0x7f3e7c156730 2023-12-26T22:52:16.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.383+0000 7f3e81dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0c94d0 con 0x7f3e7c06ee80 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.383+0000 7f3e7affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3e7c156730 0x7f3e7c0c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.383+0000 7f3e7affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3e7c156730 0x7f3e7c0c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57594/0 (socket says 172.21.15.131:57594) 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.383+0000 7f3e7affd700 1 -- 172.21.15.131:0/2807844987 learned_addr learned my addr 172.21.15.131:0/2807844987 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7bfff700 1 --2- 172.21.15.131:0/2807844987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c06ee80 0x7f3e7c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7affd700 1 -- 172.21.15.131:0/2807844987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c06ee80 msgr2=0x7f3e7c0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7affd700 1 --2- 172.21.15.131:0/2807844987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c06ee80 0x7f3e7c0c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7affd700 1 -- 172.21.15.131:0/2807844987 >> v1:172.21.15.190:6789/0 conn(0x7f3e7c156360 legacy=0x7f3e7c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7affd700 1 -- 172.21.15.131:0/2807844987 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0acc50 con 0x7f3e7c156730 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7a7fc700 1 -- 172.21.15.131:0/2807844987 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241368596 0 0) 0x7f3e7c0acfc0 con 0x7f3e7c156360 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7affd700 1 --2- 172.21.15.131:0/2807844987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c156730 0x7f3e7c0c8490 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000bc50 tx=0x7f3e7000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56fcf24e9553b1e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:16.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7a7fc700 1 -- 172.21.15.131:0/2807844987 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000dbf0 con 0x7f3e7c156730 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7a7fc700 1 -- 172.21.15.131:0/2807844987 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7000dd50 con 0x7f3e7c156730 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.384+0000 7f3e7a7fc700 1 -- 172.21.15.131:0/2807844987 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e700196b0 con 0x7f3e7c156730 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.385+0000 7f3e81dff700 1 -- 172.21.15.131:0/2807844987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c156730 msgr2=0x7f3e7c0c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.385+0000 7f3e81dff700 1 --2- 172.21.15.131:0/2807844987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c156730 0x7f3e7c0c8490 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000bc50 tx=0x7f3e7000b4f0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.385+0000 7f3e81dff700 1 -- 172.21.15.131:0/2807844987 shutdown_connections 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.385+0000 7f3e81dff700 1 --2- 172.21.15.131:0/2807844987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c06ee80 0x7f3e7c0c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.385+0000 7f3e81dff700 1 --2- 172.21.15.131:0/2807844987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c156730 0x7f3e7c0c8490 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.385+0000 7f3e81dff700 1 -- 172.21.15.131:0/2807844987 >> 172.21.15.131:0/2807844987 conn(0x7f3e7c0675b0 msgr2=0x7f3e7c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.385+0000 7f3e81dff700 1 -- 172.21.15.131:0/2807844987 shutdown_connections 2023-12-26T22:52:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.385+0000 7f3e81dff700 1 -- 172.21.15.131:0/2807844987 wait complete. 2023-12-26T22:52:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.387+0000 7f3e81dff700 1 Processor -- start 2023-12-26T22:52:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.387+0000 7f3e81dff700 1 -- start start 2023-12-26T22:52:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e81dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c06ee80 0x7f3e7c0c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e81dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e7c156360 0x7f3e7c150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e81dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c156730 0x7f3e7c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e81dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b0330 con 0x7f3e7c06ee80 2023-12-26T22:52:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e81dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b0000 con 0x7f3e7c156360 2023-12-26T22:52:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e81dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0c4b80 con 0x7f3e7c156730 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c06ee80 0x7f3e7c0c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c06ee80 0x7f3e7c0c4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57598/0 (socket says 172.21.15.131:57598) 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7b7fe700 1 -- 172.21.15.131:0/4265243404 learned_addr learned my addr 172.21.15.131:0/4265243404 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7affd700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e7c156360 0x7f3e7c150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7bfff700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c156730 0x7f3e7c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7b7fe700 1 -- 172.21.15.131:0/4265243404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c156730 msgr2=0x7f3e7c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7b7fe700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c156730 0x7f3e7c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7b7fe700 1 -- 172.21.15.131:0/4265243404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e7c156360 msgr2=0x7f3e7c150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7b7fe700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e7c156360 0x7f3e7c150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.388+0000 7f3e7b7fe700 1 -- 172.21.15.131:0/4265243404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7000b790 con 0x7f3e7c06ee80 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.389+0000 7f3e7b7fe700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c06ee80 0x7f3e7c0c4230 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c7e0 tx=0x7f3e6c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.389+0000 7f3e637fe700 1 -- 172.21.15.131:0/4265243404 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c011590 con 0x7f3e7c06ee80 2023-12-26T22:52:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.389+0000 7f3e637fe700 1 -- 172.21.15.131:0/4265243404 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c011710 con 0x7f3e7c06ee80 2023-12-26T22:52:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.389+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c0c4e60 con 0x7f3e7c06ee80 2023-12-26T22:52:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.389+0000 7f3e637fe700 1 -- 172.21.15.131:0/4265243404 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c011ad0 con 0x7f3e7c06ee80 2023-12-26T22:52:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.389+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c0c4770 con 0x7f3e7c06ee80 2023-12-26T22:52:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.390+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c0b0610 con 0x7f3e7c06ee80 2023-12-26T22:52:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.391+0000 7f3e637fe700 1 -- 172.21.15.131:0/4265243404 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 68) v1 ==== 84873+0+0 (secure 0 0 0) 0x7f3e6c01b3b0 con 0x7f3e7c06ee80 2023-12-26T22:52:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.392+0000 7f3e637fe700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3e680670a0 0x7f3e68069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.392+0000 7f3e637fe700 1 -- 172.21.15.131:0/4265243404 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f3e6c083fa0 con 0x7f3e7c06ee80 2023-12-26T22:52:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.392+0000 7f3e7affd700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3e680670a0 0x7f3e68069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.393+0000 7f3e7affd700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3e680670a0 0x7f3e68069550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000bc50 tx=0x7f3e7001b000 comp rx=0 tx=0).ready entity=mgr.7921 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:16.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.396+0000 7f3e637fe700 1 -- 172.21.15.131:0/4265243404 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e6c050df0 con 0x7f3e7c06ee80 2023-12-26T22:52:16.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.615+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e7c0b0470 con 0x7f3e680670a0 2023-12-26T22:52:16.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.682+0000 7f3e637fe700 1 -- 172.21.15.131:0/4265243404 <== mgr.7921 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7f3e70000ce0 con 0x7f3e680670a0 2023-12-26T22:52:16.681 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1 2023-12-26T22:52:16.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3e680670a0 msgr2=0x7f3e68069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:16.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3e680670a0 0x7f3e68069550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000bc50 tx=0x7f3e7001b000 comp rx=0 tx=0).stop 2023-12-26T22:52:16.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c06ee80 msgr2=0x7f3e7c0c4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:16.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c06ee80 0x7f3e7c0c4230 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c7e0 tx=0x7f3e6c007df0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 shutdown_connections 2023-12-26T22:52:16.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e7c156730 0x7f3e7c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3e680670a0 0x7f3e68069550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e7c06ee80 0x7f3e7c0c4230 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 --2- 172.21.15.131:0/4265243404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e7c156360 0x7f3e7c150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:16.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 >> 172.21.15.131:0/4265243404 conn(0x7f3e7c0675b0 msgr2=0x7f3e7c153f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:16.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 shutdown_connections 2023-12-26T22:52:16.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:16.684+0000 7f3e81dff700 1 -- 172.21.15.131:0/4265243404 wait complete. 2023-12-26T22:52:16.702 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-12-26T22:52:16.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:16.703 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T22:52:16.749 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_58_00_UTC 2023-12-26T22:52:16.749 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-21_59_00_UTC 2023-12-26T22:52:16.749 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_00_00_UTC 2023-12-26T22:52:16.749 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_01_00_UTC 2023-12-26T22:52:16.749 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_02_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_03_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_04_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_05_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_06_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_07_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_08_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_09_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_10_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_11_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_12_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_13_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_14_00_UTC 2023-12-26T22:52:16.750 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_15_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_16_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_17_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_18_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_19_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_20_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_21_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_22_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_23_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_24_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_25_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_26_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_27_00_UTC 2023-12-26T22:52:16.751 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_28_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_29_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_30_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_31_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_32_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_33_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_34_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_35_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_36_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_37_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_38_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_39_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_40_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_41_00_UTC 2023-12-26T22:52:16.752 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_42_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_43_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_44_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_45_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_46_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_47_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_48_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_49_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_50_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_51_00_UTC 2023-12-26T22:52:16.753 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_52_00_UTC 2023-12-26T22:52:16.754 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-21_58_00_UTC 2023-12-26T22:52:16.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:16.754 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-21_58_00_UTC) 2023-12-26T22:52:16.845 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-21_59_00_UTC 2023-12-26T22:52:16.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:16.845 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-21_59_00_UTC) 2023-12-26T22:52:16.895 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_00_00_UTC 2023-12-26T22:52:16.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:16.895 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_00_00_UTC) 2023-12-26T22:52:16.945 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_01_00_UTC 2023-12-26T22:52:16.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:16.945 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_01_00_UTC) 2023-12-26T22:52:16.995 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_02_00_UTC 2023-12-26T22:52:16.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:16.995 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_02_00_UTC) 2023-12-26T22:52:17.046 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_03_00_UTC 2023-12-26T22:52:17.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.046 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_03_00_UTC) 2023-12-26T22:52:17.098 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_04_00_UTC 2023-12-26T22:52:17.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.098 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_04_00_UTC) 2023-12-26T22:52:17.149 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_05_00_UTC 2023-12-26T22:52:17.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.149 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_05_00_UTC) 2023-12-26T22:52:17.200 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_06_00_UTC 2023-12-26T22:52:17.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.201 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_06_00_UTC) 2023-12-26T22:52:17.252 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_07_00_UTC 2023-12-26T22:52:17.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.252 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_07_00_UTC) 2023-12-26T22:52:17.303 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_08_00_UTC 2023-12-26T22:52:17.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.303 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_08_00_UTC) 2023-12-26T22:52:17.355 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_09_00_UTC 2023-12-26T22:52:17.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.355 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_09_00_UTC) 2023-12-26T22:52:17.406 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_10_00_UTC 2023-12-26T22:52:17.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.406 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_10_00_UTC) 2023-12-26T22:52:17.457 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_11_00_UTC 2023-12-26T22:52:17.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.457 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_11_00_UTC) 2023-12-26T22:52:17.508 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_12_00_UTC 2023-12-26T22:52:17.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.508 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_12_00_UTC) 2023-12-26T22:52:17.558 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_13_00_UTC 2023-12-26T22:52:17.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.558 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_13_00_UTC) 2023-12-26T22:52:17.609 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_14_00_UTC 2023-12-26T22:52:17.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.610 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_14_00_UTC) 2023-12-26T22:52:17.659 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_15_00_UTC 2023-12-26T22:52:17.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.660 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_15_00_UTC) 2023-12-26T22:52:17.710 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_16_00_UTC 2023-12-26T22:52:17.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.710 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_16_00_UTC) 2023-12-26T22:52:17.760 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_17_00_UTC 2023-12-26T22:52:17.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.761 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_17_00_UTC) 2023-12-26T22:52:17.812 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_18_00_UTC 2023-12-26T22:52:17.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.812 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_18_00_UTC) 2023-12-26T22:52:17.862 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_19_00_UTC 2023-12-26T22:52:17.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.862 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_19_00_UTC) 2023-12-26T22:52:17.913 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_20_00_UTC 2023-12-26T22:52:17.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.913 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_20_00_UTC) 2023-12-26T22:52:17.965 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_21_00_UTC 2023-12-26T22:52:17.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:17.965 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_21_00_UTC) 2023-12-26T22:52:18.016 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_22_00_UTC 2023-12-26T22:52:18.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.016 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_22_00_UTC) 2023-12-26T22:52:18.066 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_23_00_UTC 2023-12-26T22:52:18.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.067 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_23_00_UTC) 2023-12-26T22:52:18.117 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_24_00_UTC 2023-12-26T22:52:18.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.117 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_24_00_UTC) 2023-12-26T22:52:18.169 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_25_00_UTC 2023-12-26T22:52:18.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.169 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_25_00_UTC) 2023-12-26T22:52:18.219 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_26_00_UTC 2023-12-26T22:52:18.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.219 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_26_00_UTC) 2023-12-26T22:52:18.269 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_27_00_UTC 2023-12-26T22:52:18.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.269 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_27_00_UTC) 2023-12-26T22:52:18.320 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_28_00_UTC 2023-12-26T22:52:18.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.320 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_28_00_UTC) 2023-12-26T22:52:18.370 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_29_00_UTC 2023-12-26T22:52:18.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.371 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_29_00_UTC) 2023-12-26T22:52:18.421 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_30_00_UTC 2023-12-26T22:52:18.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.422 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_30_00_UTC) 2023-12-26T22:52:18.471 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_31_00_UTC 2023-12-26T22:52:18.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.472 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_31_00_UTC) 2023-12-26T22:52:18.522 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_32_00_UTC 2023-12-26T22:52:18.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.523 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_32_00_UTC) 2023-12-26T22:52:18.572 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_33_00_UTC 2023-12-26T22:52:18.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.573 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_33_00_UTC) 2023-12-26T22:52:18.623 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_34_00_UTC 2023-12-26T22:52:18.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.623 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_34_00_UTC) 2023-12-26T22:52:18.673 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_35_00_UTC 2023-12-26T22:52:18.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.673 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_35_00_UTC) 2023-12-26T22:52:18.723 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_36_00_UTC 2023-12-26T22:52:18.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.724 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_36_00_UTC) 2023-12-26T22:52:18.774 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_37_00_UTC 2023-12-26T22:52:18.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.774 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_37_00_UTC) 2023-12-26T22:52:18.824 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_38_00_UTC 2023-12-26T22:52:18.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.824 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_38_00_UTC) 2023-12-26T22:52:18.876 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_39_00_UTC 2023-12-26T22:52:18.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.876 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_39_00_UTC) 2023-12-26T22:52:18.926 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_40_00_UTC 2023-12-26T22:52:18.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.927 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_40_00_UTC) 2023-12-26T22:52:18.976 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_41_00_UTC 2023-12-26T22:52:18.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:18.976 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_41_00_UTC) 2023-12-26T22:52:19.026 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_42_00_UTC 2023-12-26T22:52:19.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.026 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_42_00_UTC) 2023-12-26T22:52:19.076 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_43_00_UTC 2023-12-26T22:52:19.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.076 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_43_00_UTC) 2023-12-26T22:52:19.126 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_44_00_UTC 2023-12-26T22:52:19.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.127 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_44_00_UTC) 2023-12-26T22:52:19.216 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_45_00_UTC 2023-12-26T22:52:19.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.216 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_45_00_UTC) 2023-12-26T22:52:19.315 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_46_00_UTC 2023-12-26T22:52:19.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.316 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_46_00_UTC) 2023-12-26T22:52:19.400 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_47_00_UTC 2023-12-26T22:52:19.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.400 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_47_00_UTC) 2023-12-26T22:52:19.472 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_48_00_UTC 2023-12-26T22:52:19.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.473 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_48_00_UTC) 2023-12-26T22:52:19.528 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_49_00_UTC 2023-12-26T22:52:19.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.528 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_49_00_UTC) 2023-12-26T22:52:19.578 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_50_00_UTC 2023-12-26T22:52:19.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.578 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_50_00_UTC) 2023-12-26T22:52:19.627 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_51_00_UTC 2023-12-26T22:52:19.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.627 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_51_00_UTC) 2023-12-26T22:52:19.676 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-22_52_00_UTC 2023-12-26T22:52:19.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.676 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-22_52_00_UTC) 2023-12-26T22:52:19.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:19.726 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-26T22:52:19.774 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T22:52:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.007+0000 7fb2ebfff700 1 Processor -- start 2023-12-26T22:52:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.007+0000 7fb2ebfff700 1 -- start start 2023-12-26T22:52:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.008+0000 7fb2ebfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e40c8bf0 0x7fb2e40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.008+0000 7fb2ebfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb2e4069d10 0x7fb2e40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.008+0000 7fb2ebfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e40caa10 con 0x7fb2e40c8820 2023-12-26T22:52:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.008+0000 7fb2ebfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e4157a80 con 0x7fb2e4069d10 2023-12-26T22:52:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.008+0000 7fb2ebfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40cac50 con 0x7fb2e40c8bf0 2023-12-26T22:52:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2e9d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb2e40c8820 0x7fb2e40ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48694/0 (socket says 172.21.15.131:48694) 2023-12-26T22:52:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2e9d9b700 1 -- 172.21.15.131:0/1490335644 learned_addr learned my addr 172.21.15.131:0/1490335644 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2ea59c700 1 --2- 172.21.15.131:0/1490335644 >> v2:172.21.15.131:3300/0 conn(0x7fb2e4069d10 0x7fb2e40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2e959a700 1 --2- 172.21.15.131:0/1490335644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e40c8bf0 0x7fb2e40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2e8d99700 1 -- 172.21.15.131:0/1490335644 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113766942 0 0) 0x7fb2e0001110 con 0x7fb2e40c8820 2023-12-26T22:52:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2e8d99700 1 -- 172.21.15.131:0/1490335644 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d0003570 con 0x7fb2e40c8820 2023-12-26T22:52:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2ea59c700 1 -- 172.21.15.131:0/1490335644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e40c8bf0 msgr2=0x7fb2e40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2ea59c700 1 --2- 172.21.15.131:0/1490335644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e40c8bf0 0x7fb2e40b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2ea59c700 1 -- 172.21.15.131:0/1490335644 >> v1:172.21.15.131:6789/0 conn(0x7fb2e40c8820 legacy=0x7fb2e40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.009+0000 7fb2ea59c700 1 -- 172.21.15.131:0/1490335644 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e40caf30 con 0x7fb2e4069d10 2023-12-26T22:52:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2ea59c700 1 --2- 172.21.15.131:0/1490335644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e4069d10 0x7fb2e40b1820 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009e60 tx=0x7fb2dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6834938d86b5c289 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2e8d99700 1 -- 172.21.15.131:0/1490335644 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00e070 con 0x7fb2e4069d10 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2e8d99700 1 -- 172.21.15.131:0/1490335644 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc00bab0 con 0x7fb2e4069d10 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2e8d99700 1 -- 172.21.15.131:0/1490335644 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00be70 con 0x7fb2e4069d10 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1490335644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e4069d10 msgr2=0x7fb2e40b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2ebfff700 1 --2- 172.21.15.131:0/1490335644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e4069d10 0x7fb2e40b1820 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009e60 tx=0x7fb2dc009780 comp rx=0 tx=0).stop 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1490335644 shutdown_connections 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2ebfff700 1 --2- 172.21.15.131:0/1490335644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e4069d10 0x7fb2e40b1820 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2ebfff700 1 --2- 172.21.15.131:0/1490335644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e40c8bf0 0x7fb2e40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1490335644 >> 172.21.15.131:0/1490335644 conn(0x7fb2e4068040 msgr2=0x7fb2e4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1490335644 shutdown_connections 2023-12-26T22:52:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.010+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1490335644 wait complete. 2023-12-26T22:52:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.012+0000 7fb2ebfff700 1 Processor -- start 2023-12-26T22:52:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.012+0000 7fb2ebfff700 1 -- start start 2023-12-26T22:52:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2ebfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e4069d10 0x7fb2e4159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2ebfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e40c8820 0x7fb2e4155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2ebfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2e40c8bf0 0x7fb2e4156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2ebfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c5430 con 0x7fb2e40c8820 2023-12-26T22:52:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2ebfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c5fb0 con 0x7fb2e40c8bf0 2023-12-26T22:52:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2ebfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c6440 con 0x7fb2e4069d10 2023-12-26T22:52:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e40c8820 0x7fb2e4155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e40c8820 0x7fb2e4155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57612/0 (socket says 172.21.15.131:57612) 2023-12-26T22:52:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e959a700 1 -- 172.21.15.131:0/1900282150 learned_addr learned my addr 172.21.15.131:0/1900282150 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e9d9b700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e4069d10 0x7fb2e4159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2ea59c700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2e40c8bf0 0x7fb2e4156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e959a700 1 -- 172.21.15.131:0/1900282150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e4069d10 msgr2=0x7fb2e4159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e959a700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e4069d10 0x7fb2e4159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e959a700 1 -- 172.21.15.131:0/1900282150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2e40c8bf0 msgr2=0x7fb2e4156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e959a700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2e40c8bf0 0x7fb2e4156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e959a700 1 -- 172.21.15.131:0/1900282150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc009a20 con 0x7fb2e40c8820 2023-12-26T22:52:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.013+0000 7fb2e9d9b700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e4069d10 0x7fb2e4159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.014+0000 7fb2e959a700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e40c8820 0x7fb2e4155c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400ec80 tx=0x7fb2d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.014+0000 7fb2d9ffb700 1 -- 172.21.15.131:0/1900282150 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400c9d0 con 0x7fb2e40c8820 2023-12-26T22:52:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.014+0000 7fb2d9ffb700 1 -- 172.21.15.131:0/1900282150 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d400cb30 con 0x7fb2e40c8820 2023-12-26T22:52:20.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.014+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e40c5680 con 0x7fb2e40c8820 2023-12-26T22:52:20.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.014+0000 7fb2d9ffb700 1 -- 172.21.15.131:0/1900282150 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4013470 con 0x7fb2e40c8820 2023-12-26T22:52:20.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.014+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e40c5c40 con 0x7fb2e40c8820 2023-12-26T22:52:20.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.015+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e40657e0 con 0x7fb2e40c8820 2023-12-26T22:52:20.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.016+0000 7fb2d9ffb700 1 -- 172.21.15.131:0/1900282150 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 68) v1 ==== 84873+0+0 (secure 0 0 0) 0x7fb2d4013820 con 0x7fb2e40c8820 2023-12-26T22:52:20.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.017+0000 7fb2d9ffb700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c80670a0 0x7fb2c8069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:20.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.017+0000 7fb2d9ffb700 1 -- 172.21.15.131:0/1900282150 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fb2d4084e00 con 0x7fb2e40c8820 2023-12-26T22:52:20.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.018+0000 7fb2e9d9b700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c80670a0 0x7fb2c8069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:20.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.018+0000 7fb2e9d9b700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c80670a0 0x7fb2c8069550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0001270 tx=0x7fb2e0009040 comp rx=0 tx=0).ready entity=mgr.7921 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:20.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.021+0000 7fb2d9ffb700 1 -- 172.21.15.131:0/1900282150 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2d4051b80 con 0x7fb2e40c8820 2023-12-26T22:52:20.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:20.363+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb2e40c60f0 con 0x7fb2e40c8820 2023-12-26T22:52:21.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.147+0000 7fb2d9ffb700 1 -- 172.21.15.131:0/1900282150 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fb2d4051f50 con 0x7fb2e40c8820 2023-12-26T22:52:21.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.150+0000 7fb2d9ffb700 1 -- 172.21.15.131:0/1900282150 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v69) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb2d4051b80 con 0x7fb2e40c8820 2023-12-26T22:52:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c80670a0 msgr2=0x7fb2c8069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c80670a0 0x7fb2c8069550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0001270 tx=0x7fb2e0009040 comp rx=0 tx=0).stop 2023-12-26T22:52:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e40c8820 msgr2=0x7fb2e4155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e40c8820 0x7fb2e4155c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400ec80 tx=0x7fb2d40091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 shutdown_connections 2023-12-26T22:52:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2e40c8bf0 0x7fb2e4156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:21.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2e4069d10 0x7fb2e4159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:21.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb2c80670a0 0x7fb2c8069550 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:21.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 --2- 172.21.15.131:0/1900282150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2e40c8820 0x7fb2e4155c40 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:21.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 >> 172.21.15.131:0/1900282150 conn(0x7fb2e4068040 msgr2=0x7fb2e40ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:21.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 shutdown_connections 2023-12-26T22:52:21.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.154+0000 7fb2ebfff700 1 -- 172.21.15.131:0/1900282150 wait complete. 2023-12-26T22:52:21.189 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T22:52:21.190 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T22:52:21.330 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T22:52:21.438 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T22:52:21.438 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T22:52:21.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:21.438 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T22:52:21.527 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T22:52:21.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:52:21.527 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T22:52:21.572 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:52:21.644 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:21.645+0000 7fd31b8981c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T22:52:21.676 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:21.677+0000 7f4e1b65d1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T22:52:21.748 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:21.749+0000 7fd31b8981c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T22:52:21.786 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:21.786+0000 7f4e1b65d1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T22:52:21.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.806+0000 7f13257ff700 1 Processor -- start 2023-12-26T22:52:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.806+0000 7f13257ff700 1 -- start start 2023-12-26T22:52:21.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.808+0000 7f13257ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f13200b07f0 0x7f13201554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:21.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.808+0000 7f13257ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13200c8bf0 con 0x7f1320159920 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.808+0000 7f13257ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13200c8e60 con 0x7f1320159cf0 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.808+0000 7f13257ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13200c9140 con 0x7f13200b07f0 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.809+0000 7f131e7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f13200b07f0 0x7f13201554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.809+0000 7f131effd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1320159920 0x7f13200b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48696/0 (socket says 172.21.15.131:48696) 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.809+0000 7f131effd700 1 -- 172.21.15.131:0/3997441719 learned_addr learned my addr 172.21.15.131:0/3997441719 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.809+0000 7f131e7fc700 1 -- 172.21.15.131:0/3997441719 >> v1:172.21.15.190:6789/0 conn(0x7f1320159cf0 legacy=0x7f1320155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.809+0000 7f131e7fc700 1 -- 172.21.15.131:0/3997441719 >> v1:172.21.15.131:6789/0 conn(0x7f1320159920 legacy=0x7f13200b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.809+0000 7f131e7fc700 1 -- 172.21.15.131:0/3997441719 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13200c9420 con 0x7f13200b07f0 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.809+0000 7f131e7fc700 1 --2- 172.21.15.131:0/3997441719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13200b07f0 0x7f13201554d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f131400bb90 tx=0x7f131400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8813a25765a89e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:21.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.810+0000 7f131dffb700 1 -- 172.21.15.131:0/3997441719 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131400da90 con 0x7f13200b07f0 2023-12-26T22:52:21.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.810+0000 7f131dffb700 1 -- 172.21.15.131:0/3997441719 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131400dbf0 con 0x7f13200b07f0 2023-12-26T22:52:21.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.810+0000 7f131dffb700 1 -- 172.21.15.131:0/3997441719 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314017410 con 0x7f13200b07f0 2023-12-26T22:52:21.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.810+0000 7f13257ff700 1 -- 172.21.15.131:0/3997441719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13200b07f0 msgr2=0x7f13201554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:21.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.810+0000 7f13257ff700 1 --2- 172.21.15.131:0/3997441719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13200b07f0 0x7f13201554d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f131400bb90 tx=0x7f131400b430 comp rx=0 tx=0).stop 2023-12-26T22:52:21.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.811+0000 7f13257ff700 1 -- 172.21.15.131:0/3997441719 shutdown_connections 2023-12-26T22:52:21.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.811+0000 7f13257ff700 1 --2- 172.21.15.131:0/3997441719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13200b07f0 0x7f13201554d0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:21.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.811+0000 7f13257ff700 1 -- 172.21.15.131:0/3997441719 >> 172.21.15.131:0/3997441719 conn(0x7f1320068040 msgr2=0x7f1320069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:21.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.811+0000 7f13257ff700 1 -- 172.21.15.131:0/3997441719 shutdown_connections 2023-12-26T22:52:21.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.811+0000 7f13257ff700 1 -- 172.21.15.131:0/3997441719 wait complete. 2023-12-26T22:52:21.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.812+0000 7f13257ff700 1 Processor -- start 2023-12-26T22:52:21.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.812+0000 7f13257ff700 1 -- start start 2023-12-26T22:52:21.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.813+0000 7f13257ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320159920 0x7f1320164fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:21.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.813+0000 7f13257ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1320159cf0 0x7f13200c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.813+0000 7f13257ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13200c14d0 0x7f13200c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.813+0000 7f13257ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1320141760 con 0x7f1320159920 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.813+0000 7f13257ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200c3310 con 0x7f13200c14d0 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.813+0000 7f13257ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13200c1c50 con 0x7f1320159cf0 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.813+0000 7f131effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320159920 0x7f1320164fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320159920 0x7f1320164fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57642/0 (socket says 172.21.15.131:57642) 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131effd700 1 -- 172.21.15.131:0/4190121395 learned_addr learned my addr 172.21.15.131:0/4190121395 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131e7fc700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1320159cf0 0x7f13200c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131f7fe700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13200c14d0 0x7f13200c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131effd700 1 -- 172.21.15.131:0/4190121395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1320159cf0 msgr2=0x7f13200c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131effd700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1320159cf0 0x7f13200c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131effd700 1 -- 172.21.15.131:0/4190121395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13200c14d0 msgr2=0x7f13200c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131effd700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13200c14d0 0x7f13200c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131effd700 1 -- 172.21.15.131:0/4190121395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131400b6d0 con 0x7f1320159920 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f131effd700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320159920 0x7f1320164fd0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f131000c7c0 tx=0x7f1310007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.814+0000 7f1306ffd700 1 -- 172.21.15.131:0/4190121395 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310005b30 con 0x7f1320159920 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.815+0000 7f1306ffd700 1 -- 172.21.15.131:0/4190121395 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1310005c90 con 0x7f1320159920 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.815+0000 7f1306ffd700 1 -- 172.21.15.131:0/4190121395 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310004a80 con 0x7f1320159920 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.815+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13200c1f30 con 0x7f1320159920 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.815+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13200c2fa0 con 0x7f1320159920 2023-12-26T22:52:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.816+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13200657e0 con 0x7f1320159920 2023-12-26T22:52:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.817+0000 7f1306ffd700 1 -- 172.21.15.131:0/4190121395 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f13100070d0 con 0x7f1320159920 2023-12-26T22:52:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.817+0000 7f1306ffd700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f130c067050 0x7f130c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.817+0000 7f131e7fc700 1 -- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f130c067050 msgr2=0x7f130c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.817+0000 7f1306ffd700 1 -- 172.21.15.131:0/4190121395 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f1310083d80 con 0x7f1320159920 2023-12-26T22:52:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.818+0000 7f131e7fc700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f130c067050 0x7f130c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:21.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:21.822+0000 7f1306ffd700 1 -- 172.21.15.131:0/4190121395 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f131004b5a0 con 0x7f1320159920 2023-12-26T22:52:21.838 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:21.839+0000 7fd31b8981c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T22:52:21.881 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:21.882+0000 7f4e1b65d1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T22:52:21.922 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:21.922+0000 7fd31b8981c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T22:52:21.965 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:21.965+0000 7f4e1b65d1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T22:52:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.019+0000 7f131e7fc700 1 -- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f130c067050 msgr2=0x7f130c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.019+0000 7f131e7fc700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f130c067050 0x7f130c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:22.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.065+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13200c9700 con 0x7f1320159920 2023-12-26T22:52:22.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.066+0000 7f1306ffd700 1 -- 172.21.15.131:0/4190121395 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f1310050b00 con 0x7f1320159920 2023-12-26T22:52:22.065 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:22.066 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3746500235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3746500235},{"type":"v1","addr":"172.21.15.190:6835","nonce":3746500235}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8115,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2030200373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2030200373},{"type":"v1","addr":"172.21.15.131:6833","nonce":2030200373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2894985738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2894985738},{"type":"v1","addr":"172.21.15.190:6834","nonce":2894985738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:56:28.792645+0000","modified":"2023-12-26T21:56:32.333009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8142":{"gid":8142,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1841321761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1841321761},{"type":"v1","addr":"172.21.15.131:6835","nonce":1841321761}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-12-26T22:52:22.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f130c067050 msgr2=0x7f130c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:22.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f130c067050 0x7f130c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320159920 msgr2=0x7f1320164fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320159920 0x7f1320164fd0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f131000c7c0 tx=0x7f1310007f70 comp rx=0 tx=0).stop 2023-12-26T22:52:22.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 shutdown_connections 2023-12-26T22:52:22.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13200c14d0 0x7f13200c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f130c067050 0x7f130c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1320159920 0x7f1320164fd0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 --2- 172.21.15.131:0/4190121395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1320159cf0 0x7f13200c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 >> 172.21.15.131:0/4190121395 conn(0x7f1320068040 msgr2=0x7f13200c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:22.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.068+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 shutdown_connections 2023-12-26T22:52:22.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.069+0000 7f13257ff700 1 -- 172.21.15.131:0/4190121395 wait complete. 2023-12-26T22:52:22.069 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 66 2023-12-26T22:52:22.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:52:22.111 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:22.111+0000 7fd31b8981c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:52:22.162 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:22.163+0000 7f4e1b65d1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:52:22.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.280+0000 7f5cbedff700 1 Processor -- start 2023-12-26T22:52:22.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.281+0000 7f5cbedff700 1 -- start start 2023-12-26T22:52:22.281 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:22.282+0000 7fd31b8981c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T22:52:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.283+0000 7f5cbedff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5cb8159920 0x7f5cb80b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.283+0000 7f5cbedff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb80c8bf0 con 0x7f5cb8155be0 2023-12-26T22:52:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.283+0000 7f5cbedff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb80c8e60 con 0x7f5cb8159cf0 2023-12-26T22:52:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.283+0000 7f5cbedff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb80c9140 con 0x7f5cb8159920 2023-12-26T22:52:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.283+0000 7f5cbcb9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5cb8159920 0x7f5cb80b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.283+0000 7f5cbcb9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5cb8159920 0x7f5cb80b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57646/0 (socket says 172.21.15.131:57646) 2023-12-26T22:52:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.283+0000 7f5cbd39c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5cb8155be0 0x7f5cb80c9c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48712/0 (socket says 172.21.15.131:48712) 2023-12-26T22:52:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.283+0000 7f5cbcb9b700 1 -- 172.21.15.131:0/1243038418 learned_addr learned my addr 172.21.15.131:0/1243038418 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:22.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.283+0000 7f5cb7fff700 1 --1- 172.21.15.131:0/1243038418 >> v1:172.21.15.190:6789/0 conn(0x7f5cb8159cf0 0x7f5cb81554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:56584/0 (socket says 172.21.15.131:56584) 2023-12-26T22:52:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.284+0000 7f5cbcb9b700 1 -- 172.21.15.131:0/1243038418 >> v1:172.21.15.190:6789/0 conn(0x7f5cb8159cf0 legacy=0x7f5cb81554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.284+0000 7f5cbcb9b700 1 -- 172.21.15.131:0/1243038418 >> v1:172.21.15.131:6789/0 conn(0x7f5cb8155be0 legacy=0x7f5cb80c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.284+0000 7f5cbcb9b700 1 -- 172.21.15.131:0/1243038418 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb80c9420 con 0x7f5cb8159920 2023-12-26T22:52:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.284+0000 7f5cbcb9b700 1 --2- 172.21.15.131:0/1243038418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8159920 0x7f5cb80b00e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00aa50 tx=0x7f5cac00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5dfc6e7558d966 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.284+0000 7f5cb77fe700 1 -- 172.21.15.131:0/1243038418 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00b990 con 0x7f5cb8159920 2023-12-26T22:52:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.284+0000 7f5cb77fe700 1 -- 172.21.15.131:0/1243038418 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac00baf0 con 0x7f5cb8159920 2023-12-26T22:52:22.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.284+0000 7f5cb77fe700 1 -- 172.21.15.131:0/1243038418 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00be90 con 0x7f5cb8159920 2023-12-26T22:52:22.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.285+0000 7f5cbedff700 1 -- 172.21.15.131:0/1243038418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8159920 msgr2=0x7f5cb80b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.285+0000 7f5cbedff700 1 --2- 172.21.15.131:0/1243038418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8159920 0x7f5cb80b00e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00aa50 tx=0x7f5cac00a330 comp rx=0 tx=0).stop 2023-12-26T22:52:22.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.285+0000 7f5cbedff700 1 -- 172.21.15.131:0/1243038418 shutdown_connections 2023-12-26T22:52:22.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.285+0000 7f5cbedff700 1 --2- 172.21.15.131:0/1243038418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8159920 0x7f5cb80b00e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.285+0000 7f5cbedff700 1 -- 172.21.15.131:0/1243038418 >> 172.21.15.131:0/1243038418 conn(0x7f5cb8068040 msgr2=0x7f5cb8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:22.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.285+0000 7f5cbedff700 1 -- 172.21.15.131:0/1243038418 shutdown_connections 2023-12-26T22:52:22.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.285+0000 7f5cbedff700 1 -- 172.21.15.131:0/1243038418 wait complete. 2023-12-26T22:52:22.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.287+0000 7f5cbedff700 1 Processor -- start 2023-12-26T22:52:22.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.288+0000 7f5cbedff700 1 -- start start 2023-12-26T22:52:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cbedff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8155be0 0x7f5cb813f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cbedff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb8159920 0x7f5cb80c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cbedff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cb8159cf0 0x7f5cb806e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cbedff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb8150870 con 0x7f5cb8155be0 2023-12-26T22:52:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cbedff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb8151110 con 0x7f5cb8159920 2023-12-26T22:52:22.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cbedff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb80c7160 con 0x7f5cb8159cf0 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cbcb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8155be0 0x7f5cb813f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cbcb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8155be0 0x7f5cb813f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57656/0 (socket says 172.21.15.131:57656) 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cbcb9b700 1 -- 172.21.15.131:0/2291103979 learned_addr learned my addr 172.21.15.131:0/2291103979 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.289+0000 7f5cb7fff700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb8159920 0x7f5cb80c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cbd39c700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cb8159cf0 0x7f5cb806e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cbcb9b700 1 -- 172.21.15.131:0/2291103979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cb8159cf0 msgr2=0x7f5cb806e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cbcb9b700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cb8159cf0 0x7f5cb806e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cbcb9b700 1 -- 172.21.15.131:0/2291103979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb8159920 msgr2=0x7f5cb80c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cbcb9b700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb8159920 0x7f5cb80c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cbcb9b700 1 -- 172.21.15.131:0/2291103979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac00a590 con 0x7f5cb8155be0 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cbcb9b700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8155be0 0x7f5cb813f210 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80688f0 tx=0x7f5cac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:22.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cb4ff9700 1 -- 172.21.15.131:0/2291103979 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac01f070 con 0x7f5cb8155be0 2023-12-26T22:52:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cb4ff9700 1 -- 172.21.15.131:0/2291103979 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac024460 con 0x7f5cb8155be0 2023-12-26T22:52:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb8150ac0 con 0x7f5cb8155be0 2023-12-26T22:52:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.290+0000 7f5cb4ff9700 1 -- 172.21.15.131:0/2291103979 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac024820 con 0x7f5cb8155be0 2023-12-26T22:52:22.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.291+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb813f750 con 0x7f5cb8155be0 2023-12-26T22:52:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.292+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb80657e0 con 0x7f5cb8155be0 2023-12-26T22:52:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.292+0000 7f5cb4ff9700 1 -- 172.21.15.131:0/2291103979 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f5cac024b90 con 0x7f5cb8155be0 2023-12-26T22:52:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.293+0000 7f5cb4ff9700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c9c067050 0x7f5c9c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.293+0000 7f5cb4ff9700 1 -- 172.21.15.131:0/2291103979 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f5cac0864b0 con 0x7f5cb8155be0 2023-12-26T22:52:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.293+0000 7f5cb7fff700 1 -- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c9c067050 msgr2=0x7f5c9c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:22.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.293+0000 7f5cb7fff700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c9c067050 0x7f5c9c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:22.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.301+0000 7f5cb4ff9700 1 -- 172.21.15.131:0/2291103979 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5cac053120 con 0x7f5cb8155be0 2023-12-26T22:52:22.348 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:22.349+0000 7f4e1b65d1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T22:52:22.365 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:22.366+0000 7fd31b8981c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T22:52:22.435 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:22.436+0000 7f4e1b65d1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T22:52:22.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.494+0000 7f5cb7fff700 1 -- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c9c067050 msgr2=0x7f5c9c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:22.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.494+0000 7f5cb7fff700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c9c067050 0x7f5c9c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:22.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.506+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cb80c9700 con 0x7f5cb8155be0 2023-12-26T22:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.507+0000 7f5cb4ff9700 1 -- 172.21.15.131:0/2291103979 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16688 (secure 0 0 0) 0x7f5cac00b040 con 0x7f5cb8155be0 2023-12-26T22:52:22.507 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:22.507 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":147,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:52:21.142799+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:56:26.517640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-26T21:56:27.777447+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":147,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"17.3f","mappings":[{"from":6,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6837/39288":"2023-12-27T21:56:44.596879+0000","172.21.15.131:6836/39288":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/3801233358":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/3315291990":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/2652223535":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/727308574":"2023-12-27T21:56:44.596879+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":48,"length":9},{"begin":104,"length":9}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c9c067050 msgr2=0x7f5c9c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c9c067050 0x7f5c9c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8155be0 msgr2=0x7f5cb813f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8155be0 0x7f5cb813f210 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80688f0 tx=0x7f5cac019000 comp rx=0 tx=0).stop 2023-12-26T22:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 shutdown_connections 2023-12-26T22:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cb8159cf0 0x7f5cb806e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c9c067050 0x7f5c9c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cb8155be0 0x7f5cb813f210 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 --2- 172.21.15.131:0/2291103979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cb8159920 0x7f5cb80c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.509+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 >> 172.21.15.131:0/2291103979 conn(0x7f5cb8068040 msgr2=0x7f5cb8069360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:22.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.510+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 shutdown_connections 2023-12-26T22:52:22.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.510+0000 7f5cbedff700 1 -- 172.21.15.131:0/2291103979 wait complete. 2023-12-26T22:52:22.511 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:22.511+0000 7fd31b8981c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T22:52:22.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T21:56:26.517640+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T21:56:27.777447+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 147, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T22:52:22.528 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T22:52:22.595 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:22.595+0000 7f4e1b65d1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T22:52:22.602 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:22.602+0000 7fd31b8981c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T22:52:22.690 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:22.690+0000 7f4e1b65d1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T22:52:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.726+0000 7ff262dff700 1 Processor -- start 2023-12-26T22:52:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.726+0000 7ff262dff700 1 -- start start 2023-12-26T22:52:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.727+0000 7ff262dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff25c15d240 0x7ff25c0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff262dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c0a98b0 con 0x7ff25c0ad390 2023-12-26T22:52:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff262dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c0ad790 con 0x7ff25c1595e0 2023-12-26T22:52:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff262dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0add40 con 0x7ff25c15d240 2023-12-26T22:52:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff25bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff25c15d240 0x7ff25c0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff25bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff25c15d240 0x7ff25c0ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57666/0 (socket says 172.21.15.131:57666) 2023-12-26T22:52:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff26139c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff25c0ad390 0x7ff25c0aa020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48720/0 (socket says 172.21.15.131:48720) 2023-12-26T22:52:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff25bfff700 1 -- 172.21.15.131:0/1459468218 learned_addr learned my addr 172.21.15.131:0/1459468218 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff25bfff700 1 -- 172.21.15.131:0/1459468218 >> v1:172.21.15.190:6789/0 conn(0x7ff25c1595e0 legacy=0x7ff25c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff25bfff700 1 -- 172.21.15.131:0/1459468218 >> v1:172.21.15.131:6789/0 conn(0x7ff25c0ad390 legacy=0x7ff25c0aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.728+0000 7ff25bfff700 1 -- 172.21.15.131:0/1459468218 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c15a500 con 0x7ff25c15d240 2023-12-26T22:52:22.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.729+0000 7ff25bfff700 1 --2- 172.21.15.131:0/1459468218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c15d240 0x7ff25c0ace50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff2480099b0 tx=0x7ff248009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59af8d76c37c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.729+0000 7ff25b7fe700 1 -- 172.21.15.131:0/1459468218 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24800b860 con 0x7ff25c15d240 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.729+0000 7ff25b7fe700 1 -- 172.21.15.131:0/1459468218 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24800b9c0 con 0x7ff25c15d240 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.729+0000 7ff25b7fe700 1 -- 172.21.15.131:0/1459468218 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24800bd60 con 0x7ff25c15d240 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.729+0000 7ff262dff700 1 -- 172.21.15.131:0/1459468218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c15d240 msgr2=0x7ff25c0ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.729+0000 7ff262dff700 1 --2- 172.21.15.131:0/1459468218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c15d240 0x7ff25c0ace50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff2480099b0 tx=0x7ff248009250 comp rx=0 tx=0).stop 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.730+0000 7ff262dff700 1 -- 172.21.15.131:0/1459468218 shutdown_connections 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.730+0000 7ff262dff700 1 --2- 172.21.15.131:0/1459468218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c15d240 0x7ff25c0ace50 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.730+0000 7ff262dff700 1 -- 172.21.15.131:0/1459468218 >> 172.21.15.131:0/1459468218 conn(0x7ff25c068040 msgr2=0x7ff25c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.730+0000 7ff262dff700 1 -- 172.21.15.131:0/1459468218 shutdown_connections 2023-12-26T22:52:22.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.730+0000 7ff262dff700 1 -- 172.21.15.131:0/1459468218 wait complete. 2023-12-26T22:52:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.731+0000 7ff262dff700 1 Processor -- start 2023-12-26T22:52:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.731+0000 7ff262dff700 1 -- start start 2023-12-26T22:52:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.732+0000 7ff262dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff25c0ad390 0x7ff25c150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.732+0000 7ff262dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c1595e0 0x7ff25c14eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.732+0000 7ff262dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c15d240 0x7ff25c14f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.732+0000 7ff262dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c1505b0 con 0x7ff25c1595e0 2023-12-26T22:52:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.732+0000 7ff262dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c154c30 con 0x7ff25c15d240 2023-12-26T22:52:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.732+0000 7ff262dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c154ef0 con 0x7ff25c0ad390 2023-12-26T22:52:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.732+0000 7ff25bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c1595e0 0x7ff25c14eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.732+0000 7ff25bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c1595e0 0x7ff25c14eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57670/0 (socket says 172.21.15.131:57670) 2023-12-26T22:52:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.732+0000 7ff25bfff700 1 -- 172.21.15.131:0/1254324472 learned_addr learned my addr 172.21.15.131:0/1254324472 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff26139c700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c15d240 0x7ff25c14f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff260b9b700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff25c0ad390 0x7ff25c150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff25bfff700 1 -- 172.21.15.131:0/1254324472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff25c0ad390 msgr2=0x7ff25c150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff25bfff700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff25c0ad390 0x7ff25c150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff25bfff700 1 -- 172.21.15.131:0/1254324472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c15d240 msgr2=0x7ff25c14f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff25bfff700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c15d240 0x7ff25c14f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff25bfff700 1 -- 172.21.15.131:0/1254324472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2480094f0 con 0x7ff25c1595e0 2023-12-26T22:52:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff25bfff700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c1595e0 0x7ff25c14eee0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff248009290 tx=0x7ff248017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff258ff9700 1 -- 172.21.15.131:0/1254324472 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24801a630 con 0x7ff25c1595e0 2023-12-26T22:52:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.733+0000 7ff258ff9700 1 -- 172.21.15.131:0/1254324472 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24801a790 con 0x7ff25c1595e0 2023-12-26T22:52:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.734+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c14fd10 con 0x7ff25c1595e0 2023-12-26T22:52:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.734+0000 7ff258ff9700 1 -- 172.21.15.131:0/1254324472 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24801ab10 con 0x7ff25c1595e0 2023-12-26T22:52:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.734+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c1502d0 con 0x7ff25c1595e0 2023-12-26T22:52:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.735+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c155060 con 0x7ff25c1595e0 2023-12-26T22:52:22.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.736+0000 7ff258ff9700 1 -- 172.21.15.131:0/1254324472 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7ff24802d440 con 0x7ff25c1595e0 2023-12-26T22:52:22.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.737+0000 7ff258ff9700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff240067050 0x7ff240069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:22.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.737+0000 7ff260b9b700 1 -- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff240067050 msgr2=0x7ff240069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:22.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.737+0000 7ff258ff9700 1 -- 172.21.15.131:0/1254324472 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6529+0+0 (secure 0 0 0) 0x7ff248086620 con 0x7ff25c1595e0 2023-12-26T22:52:22.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.737+0000 7ff260b9b700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff240067050 0x7ff240069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.741+0000 7ff258ff9700 1 -- 172.21.15.131:0/1254324472 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2480534c0 con 0x7ff25c1595e0 2023-12-26T22:52:22.782 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:22.783+0000 7fd31b8981c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T22:52:22.887 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:22.888+0000 7f4e1b65d1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T22:52:22.932 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:22.932+0000 7fd31b8981c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T22:52:22.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.938+0000 7ff260b9b700 1 -- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff240067050 msgr2=0x7ff240069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:22.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.938+0000 7ff260b9b700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff240067050 0x7ff240069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:22.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.977+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff25c0657e0 con 0x7ff25c1595e0 2023-12-26T22:52:22.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.977+0000 7ff258ff9700 1 -- 172.21.15.131:0/1254324472 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff248056ae0 con 0x7ff25c1595e0 2023-12-26T22:52:22.977 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T22:52:22.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.979+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff240067050 msgr2=0x7ff240069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:22.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.979+0000 7ff262dff700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff240067050 0x7ff240069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.979+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c1595e0 msgr2=0x7ff25c14eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:22.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.979+0000 7ff262dff700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c1595e0 0x7ff25c14eee0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff248009290 tx=0x7ff248017000 comp rx=0 tx=0).stop 2023-12-26T22:52:22.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.980+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 shutdown_connections 2023-12-26T22:52:22.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.980+0000 7ff262dff700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff25c15d240 0x7ff25c14f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.980+0000 7ff262dff700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff25c0ad390 0x7ff25c150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.980+0000 7ff262dff700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff240067050 0x7ff240069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.980+0000 7ff262dff700 1 --2- 172.21.15.131:0/1254324472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff25c1595e0 0x7ff25c14eee0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:22.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.980+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 >> 172.21.15.131:0/1254324472 conn(0x7ff25c068040 msgr2=0x7ff25c0aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:22.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.980+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 shutdown_connections 2023-12-26T22:52:22.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:22.980+0000 7ff262dff700 1 -- 172.21.15.131:0/1254324472 wait complete. 2023-12-26T22:52:22.998 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T22:52:23.047 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:23.048+0000 7f4e1b65d1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T22:52:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.195+0000 7fb6937ff700 1 Processor -- start 2023-12-26T22:52:23.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.195+0000 7fb6937ff700 1 -- start start 2023-12-26T22:52:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb6937ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb68c06b8c0 0x7fb68c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb6937ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb68c06bd40 0x7fb68c0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb6937ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68c15ba70 con 0x7fb68c158030 2023-12-26T22:52:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb6937ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68c15bf00 con 0x7fb68c06bd40 2023-12-26T22:52:23.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb6937ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68c06c2a0 con 0x7fb68c06b8c0 2023-12-26T22:52:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb691d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb68c06bd40 0x7fb68c0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb690d9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb68c06b8c0 0x7fb68c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb691d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb68c06bd40 0x7fb68c0c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57680/0 (socket says 172.21.15.131:57680) 2023-12-26T22:52:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb691d9c700 1 -- 172.21.15.131:0/1897077595 learned_addr learned my addr 172.21.15.131:0/1897077595 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.197+0000 7fb683fff700 1 -- 172.21.15.131:0/1897077595 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121658826 0 0) 0x7fb68c15ba70 con 0x7fb68c158030 2023-12-26T22:52:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb683fff700 1 -- 172.21.15.131:0/1897077595 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb678003570 con 0x7fb68c158030 2023-12-26T22:52:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb691d9c700 1 -- 172.21.15.131:0/1897077595 >> v2:172.21.15.190:3300/0 conn(0x7fb68c06b8c0 msgr2=0x7fb68c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb691d9c700 1 --2- 172.21.15.131:0/1897077595 >> v2:172.21.15.190:3300/0 conn(0x7fb68c06b8c0 0x7fb68c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb691d9c700 1 -- 172.21.15.131:0/1897077595 >> v1:172.21.15.190:6789/0 conn(0x7fb68c158030 legacy=0x7fb68c158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb691d9c700 1 -- 172.21.15.131:0/1897077595 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c15b140 con 0x7fb68c06bd40 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb690d9a700 1 --2- 172.21.15.131:0/1897077595 >> v2:172.21.15.190:3300/0 conn(0x7fb68c06b8c0 0x7fb68c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb691d9c700 1 --2- 172.21.15.131:0/1897077595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06bd40 0x7fb68c0c8f60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0c89a0 tx=0x7fb684009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcb8b0944fe7aa6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb683fff700 1 -- 172.21.15.131:0/1897077595 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68400e070 con 0x7fb68c06bd40 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb683fff700 1 -- 172.21.15.131:0/1897077595 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68400b970 con 0x7fb68c06bd40 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.198+0000 7fb683fff700 1 -- 172.21.15.131:0/1897077595 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840092e0 con 0x7fb68c06bd40 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.199+0000 7fb6937ff700 1 -- 172.21.15.131:0/1897077595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06bd40 msgr2=0x7fb68c0c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.199+0000 7fb6937ff700 1 --2- 172.21.15.131:0/1897077595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06bd40 0x7fb68c0c8f60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0c89a0 tx=0x7fb684009840 comp rx=0 tx=0).stop 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.199+0000 7fb6937ff700 1 -- 172.21.15.131:0/1897077595 shutdown_connections 2023-12-26T22:52:23.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.199+0000 7fb6937ff700 1 --2- 172.21.15.131:0/1897077595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06bd40 0x7fb68c0c8f60 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.199+0000 7fb6937ff700 1 --2- 172.21.15.131:0/1897077595 >> v2:172.21.15.190:3300/0 conn(0x7fb68c06b8c0 0x7fb68c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.199+0000 7fb6937ff700 1 -- 172.21.15.131:0/1897077595 >> 172.21.15.131:0/1897077595 conn(0x7fb68c0ce3e0 msgr2=0x7fb68c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:23.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.199+0000 7fb6937ff700 1 -- 172.21.15.131:0/1897077595 shutdown_connections 2023-12-26T22:52:23.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.199+0000 7fb6937ff700 1 -- 172.21.15.131:0/1897077595 wait complete. 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.200+0000 7fb6937ff700 1 Processor -- start 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.201+0000 7fb6937ff700 1 -- start start 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.201+0000 7fb6937ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06b8c0 0x7fb68c0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.201+0000 7fb6937ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb68c06bd40 0x7fb68c14eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.201+0000 7fb6937ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb68c158030 0x7fb68c14bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.202+0000 7fb6937ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c14f0e0 con 0x7fb68c06b8c0 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.202+0000 7fb6937ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c069000 con 0x7fb68c158030 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.202+0000 7fb6937ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c14f5a0 con 0x7fb68c06bd40 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.202+0000 7fb69159b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06b8c0 0x7fb68c0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:23.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.202+0000 7fb69159b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06b8c0 0x7fb68c0c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57686/0 (socket says 172.21.15.131:57686) 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.202+0000 7fb69159b700 1 -- 172.21.15.131:0/3957598770 learned_addr learned my addr 172.21.15.131:0/3957598770 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.202+0000 7fb691d9c700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb68c158030 0x7fb68c14bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.202+0000 7fb690d9a700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb68c06bd40 0x7fb68c14eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb69159b700 1 -- 172.21.15.131:0/3957598770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb68c06bd40 msgr2=0x7fb68c14eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb69159b700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb68c06bd40 0x7fb68c14eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb69159b700 1 -- 172.21.15.131:0/3957598770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb68c158030 msgr2=0x7fb68c14bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb69159b700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb68c158030 0x7fb68c14bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb69159b700 1 -- 172.21.15.131:0/3957598770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c009420 con 0x7fb68c06b8c0 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb69159b700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06b8c0 0x7fb68c0c83a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb68800c560 tx=0x7fb688007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb6817fa700 1 -- 172.21.15.131:0/3957598770 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688010510 con 0x7fb68c06b8c0 2023-12-26T22:52:23.206 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb6817fa700 1 -- 172.21.15.131:0/3957598770 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688010670 con 0x7fb68c06b8c0 2023-12-26T22:52:23.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb684009ae0 con 0x7fb68c06b8c0 2023-12-26T22:52:23.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb6817fa700 1 -- 172.21.15.131:0/3957598770 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688010a30 con 0x7fb68c06b8c0 2023-12-26T22:52:23.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.203+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c14fb60 con 0x7fb68c06b8c0 2023-12-26T22:52:23.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.204+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c0657e0 con 0x7fb68c06b8c0 2023-12-26T22:52:23.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.205+0000 7fb6817fa700 1 -- 172.21.15.131:0/3957598770 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fb688002050 con 0x7fb68c06b8c0 2023-12-26T22:52:23.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.206+0000 7fb6817fa700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb670067050 0x7fb670069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.206+0000 7fb6817fa700 1 -- 172.21.15.131:0/3957598770 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fb688084370 con 0x7fb68c06b8c0 2023-12-26T22:52:23.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.206+0000 7fb690d9a700 1 -- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb670067050 msgr2=0x7fb670069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:23.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.206+0000 7fb690d9a700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb670067050 0x7fb670069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:23.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.210+0000 7fb6817fa700 1 -- 172.21.15.131:0/3957598770 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6880510f0 con 0x7fb68c06b8c0 2023-12-26T22:52:23.315 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:23.315+0000 7fd31b8981c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T22:52:23.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.407+0000 7fb690d9a700 1 -- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb670067050 msgr2=0x7fb670069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:23.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.407+0000 7fb690d9a700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb670067050 0x7fb670069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:23.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.411+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb68c15b420 con 0x7fb68c06b8c0 2023-12-26T22:52:23.411 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.412+0000 7fb6817fa700 1 -- 172.21.15.131:0/3957598770 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v147) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb688054710 con 0x7fb68c06b8c0 2023-12-26T22:52:23.411 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T22:52:23.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb670067050 msgr2=0x7fb670069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:23.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb670067050 0x7fb670069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06b8c0 msgr2=0x7fb68c0c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06b8c0 0x7fb68c0c83a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb68800c560 tx=0x7fb688007e00 comp rx=0 tx=0).stop 2023-12-26T22:52:23.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 shutdown_connections 2023-12-26T22:52:23.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb68c158030 0x7fb68c14bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb670067050 0x7fb670069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb68c06b8c0 0x7fb68c0c83a0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 --2- 172.21.15.131:0/3957598770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb68c06bd40 0x7fb68c14eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 >> 172.21.15.131:0/3957598770 conn(0x7fb68c0ce3e0 msgr2=0x7fb68c14cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:23.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.414+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 shutdown_connections 2023-12-26T22:52:23.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.415+0000 7fb6937ff700 1 -- 172.21.15.131:0/3957598770 wait complete. 2023-12-26T22:52:23.433 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T22:52:23.459 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:23.460+0000 7f4e1b65d1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T22:52:23.577 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:23.578+0000 7fd31b8981c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T22:52:23.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.635+0000 7ff3151ff700 1 Processor -- start 2023-12-26T22:52:23.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.635+0000 7ff3151ff700 1 -- start start 2023-12-26T22:52:23.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.637+0000 7ff3151ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3100b07f0 0x7ff3101554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.637+0000 7ff3151ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3100c8bf0 con 0x7ff310159cf0 2023-12-26T22:52:23.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.637+0000 7ff3151ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3100c8e60 con 0x7ff310159920 2023-12-26T22:52:23.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.637+0000 7ff3151ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3100c9140 con 0x7ff3100b07f0 2023-12-26T22:52:23.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30e59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3100b07f0 0x7ff3101554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:23.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30f59e700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff310159cf0 0x7ff310155a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48722/0 (socket says 172.21.15.131:48722) 2023-12-26T22:52:23.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30f59e700 1 -- 172.21.15.131:0/3440367899 learned_addr learned my addr 172.21.15.131:0/3440367899 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:23.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30ed9d700 1 --1- 172.21.15.131:0/3440367899 >> v1:172.21.15.190:6789/0 conn(0x7ff310159920 0x7ff3100b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:56602/0 (socket says 172.21.15.131:56602) 2023-12-26T22:52:23.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30dd9b700 1 -- 172.21.15.131:0/3440367899 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686803545 0 0) 0x7ff3100c8bf0 con 0x7ff310159cf0 2023-12-26T22:52:23.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30dd9b700 1 -- 172.21.15.131:0/3440367899 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f8004680 con 0x7ff310159cf0 2023-12-26T22:52:23.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30dd9b700 1 -- 172.21.15.131:0/3440367899 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495611854 0 0) 0x7ff3100c8e60 con 0x7ff310159920 2023-12-26T22:52:23.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30e59c700 1 -- 172.21.15.131:0/3440367899 >> v1:172.21.15.190:6789/0 conn(0x7ff310159920 legacy=0x7ff3100b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30e59c700 1 -- 172.21.15.131:0/3440367899 >> v1:172.21.15.131:6789/0 conn(0x7ff310159cf0 legacy=0x7ff310155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.638+0000 7ff30e59c700 1 -- 172.21.15.131:0/3440367899 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100c9420 con 0x7ff3100b07f0 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff30e59c700 1 --2- 172.21.15.131:0/3440367899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3100b07f0 0x7ff3101554d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff3040098e0 tx=0x7ff3040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bda559de72a93f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff30dd9b700 1 -- 172.21.15.131:0/3440367899 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400b640 con 0x7ff3100b07f0 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff30dd9b700 1 -- 172.21.15.131:0/3440367899 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30400b7a0 con 0x7ff3100b07f0 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff30dd9b700 1 -- 172.21.15.131:0/3440367899 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400bb40 con 0x7ff3100b07f0 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff3151ff700 1 -- 172.21.15.131:0/3440367899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3100b07f0 msgr2=0x7ff3101554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff3151ff700 1 --2- 172.21.15.131:0/3440367899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3100b07f0 0x7ff3101554d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff3040098e0 tx=0x7ff3040091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff3151ff700 1 -- 172.21.15.131:0/3440367899 shutdown_connections 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff3151ff700 1 --2- 172.21.15.131:0/3440367899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3100b07f0 0x7ff3101554d0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff3151ff700 1 -- 172.21.15.131:0/3440367899 >> 172.21.15.131:0/3440367899 conn(0x7ff310068040 msgr2=0x7ff310069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.639+0000 7ff3151ff700 1 -- 172.21.15.131:0/3440367899 shutdown_connections 2023-12-26T22:52:23.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.640+0000 7ff3151ff700 1 -- 172.21.15.131:0/3440367899 wait complete. 2023-12-26T22:52:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.641+0000 7ff3151ff700 1 Processor -- start 2023-12-26T22:52:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.641+0000 7ff3151ff700 1 -- start start 2023-12-26T22:52:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff3151ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff310159920 0x7ff31014f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff3151ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff310159cf0 0x7ff31015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff3151ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff310167950 0x7ff31015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff3151ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31015a7e0 con 0x7ff310159cf0 2023-12-26T22:52:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff3151ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31015b080 con 0x7ff310159920 2023-12-26T22:52:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff3151ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3101531a0 con 0x7ff310167950 2023-12-26T22:52:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff30e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff310159cf0 0x7ff31015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff30ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff310159920 0x7ff31014f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff30f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff310167950 0x7ff31015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff30e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff310159cf0 0x7ff31015d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57708/0 (socket says 172.21.15.131:57708) 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff30e59c700 1 -- 172.21.15.131:0/2608913511 learned_addr learned my addr 172.21.15.131:0/2608913511 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.642+0000 7ff30ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff310159920 0x7ff31014f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:55902/0 (socket says 172.21.15.131:55902) 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff30e59c700 1 -- 172.21.15.131:0/2608913511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff310167950 msgr2=0x7ff31015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff30e59c700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff310167950 0x7ff31015d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff30e59c700 1 -- 172.21.15.131:0/2608913511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff310159920 msgr2=0x7ff31014f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff30e59c700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff310159920 0x7ff31014f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff30e59c700 1 -- 172.21.15.131:0/2608913511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304009420 con 0x7ff310159cf0 2023-12-26T22:52:23.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff30e59c700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff310159cf0 0x7ff31015d070 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff304013000 tx=0x7ff304017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:23.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff2f6ffd700 1 -- 172.21.15.131:0/2608913511 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30401f070 con 0x7ff310159cf0 2023-12-26T22:52:23.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff2f6ffd700 1 -- 172.21.15.131:0/2608913511 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff304002f70 con 0x7ff310159cf0 2023-12-26T22:52:23.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31015aa30 con 0x7ff310159cf0 2023-12-26T22:52:23.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff2f6ffd700 1 -- 172.21.15.131:0/2608913511 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400bb40 con 0x7ff310159cf0 2023-12-26T22:52:23.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.643+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31014fd40 con 0x7ff310159cf0 2023-12-26T22:52:23.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.645+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31015b230 con 0x7ff310159cf0 2023-12-26T22:52:23.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.646+0000 7ff2f6ffd700 1 -- 172.21.15.131:0/2608913511 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7ff304011630 con 0x7ff310159cf0 2023-12-26T22:52:23.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.647+0000 7ff2f6ffd700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff2fc067050 0x7ff2fc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:23.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.647+0000 7ff2f6ffd700 1 -- 172.21.15.131:0/2608913511 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6529+0+0 (secure 0 0 0) 0x7ff304085b20 con 0x7ff310159cf0 2023-12-26T22:52:23.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.647+0000 7ff30ed9d700 1 -- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff2fc067050 msgr2=0x7ff2fc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:23.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.647+0000 7ff30ed9d700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff2fc067050 0x7ff2fc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:23.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.651+0000 7ff2f6ffd700 1 -- 172.21.15.131:0/2608913511 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff31015b230 con 0x7ff310159cf0 2023-12-26T22:52:23.745 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:23.746+0000 7f4e1b65d1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T22:52:23.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.848+0000 7ff30ed9d700 1 -- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff2fc067050 msgr2=0x7ff2fc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:23.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.848+0000 7ff30ed9d700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff2fc067050 0x7ff2fc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:23.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.853+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff31015b1f0 con 0x7ff310159cf0 2023-12-26T22:52:23.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.853+0000 7ff2f6ffd700 1 -- 172.21.15.131:0/2608913511 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v147) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3040528a0 con 0x7ff310159cf0 2023-12-26T22:52:23.852 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T22:52:23.854 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:23.855+0000 7fd31b8981c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T22:52:23.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff2fc067050 msgr2=0x7ff2fc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff2fc067050 0x7ff2fc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff310159cf0 msgr2=0x7ff31015d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff310159cf0 0x7ff31015d070 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff304013000 tx=0x7ff304017000 comp rx=0 tx=0).stop 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 shutdown_connections 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff310167950 0x7ff31015d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff310159920 0x7ff31014f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff2fc067050 0x7ff2fc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 --2- 172.21.15.131:0/2608913511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff310159cf0 0x7ff31015d070 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 >> 172.21.15.131:0/2608913511 conn(0x7ff310068040 msgr2=0x7ff310069360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 shutdown_connections 2023-12-26T22:52:23.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:23.856+0000 7ff3151ff700 1 -- 172.21.15.131:0/2608913511 wait complete. 2023-12-26T22:52:23.874 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:52:23.942 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:23.943+0000 7fd31b8981c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T22:52:24.035 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:24.035+0000 7f4e1b65d1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T22:52:24.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.109+0000 7fc2395ff700 1 Processor -- start 2023-12-26T22:52:24.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.109+0000 7fc2395ff700 1 -- start start 2023-12-26T22:52:24.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2395ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2340cb320 0x7fc234069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2395ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc23406a1e0 0x7fc2340b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2395ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc234156940 con 0x7fc2340b0c90 2023-12-26T22:52:24.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2395ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2340c8cc0 con 0x7fc23406a1e0 2023-12-26T22:52:24.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2395ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340ad700 con 0x7fc2340cb320 2023-12-26T22:52:24.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2327fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc23406a1e0 0x7fc2340b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:24.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2327fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc23406a1e0 0x7fc2340b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57710/0 (socket says 172.21.15.131:57710) 2023-12-26T22:52:24.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2327fc700 1 -- 172.21.15.131:0/2682030777 learned_addr learned my addr 172.21.15.131:0/2682030777 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:24.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc232ffd700 1 --2- 172.21.15.131:0/2682030777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2340cb320 0x7fc234069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:24.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2327fc700 1 -- 172.21.15.131:0/2682030777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2340cb320 msgr2=0x7fc234069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:24.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2327fc700 1 --2- 172.21.15.131:0/2682030777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2340cb320 0x7fc234069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2327fc700 1 -- 172.21.15.131:0/2682030777 >> v1:172.21.15.190:6789/0 conn(0x7fc2340b0c90 legacy=0x7fc2340aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.111+0000 7fc2327fc700 1 -- 172.21.15.131:0/2682030777 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2340ae500 con 0x7fc23406a1e0 2023-12-26T22:52:24.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.112+0000 7fc2327fc700 1 --2- 172.21.15.131:0/2682030777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23406a1e0 0x7fc2340b0750 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc2280099b0 tx=0x7fc228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65eca74b9005af1e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.112+0000 7fc231ffb700 1 -- 172.21.15.131:0/2682030777 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228009ea0 con 0x7fc23406a1e0 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.112+0000 7fc231ffb700 1 -- 172.21.15.131:0/2682030777 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22800b860 con 0x7fc23406a1e0 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.112+0000 7fc231ffb700 1 -- 172.21.15.131:0/2682030777 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22800bc00 con 0x7fc23406a1e0 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.112+0000 7fc2395ff700 1 -- 172.21.15.131:0/2682030777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23406a1e0 msgr2=0x7fc2340b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.112+0000 7fc2395ff700 1 --2- 172.21.15.131:0/2682030777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23406a1e0 0x7fc2340b0750 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc2280099b0 tx=0x7fc228009250 comp rx=0 tx=0).stop 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.113+0000 7fc2395ff700 1 -- 172.21.15.131:0/2682030777 shutdown_connections 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.113+0000 7fc2395ff700 1 --2- 172.21.15.131:0/2682030777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2340cb320 0x7fc234069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.113+0000 7fc2395ff700 1 --2- 172.21.15.131:0/2682030777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23406a1e0 0x7fc2340b0750 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.113+0000 7fc2395ff700 1 -- 172.21.15.131:0/2682030777 >> 172.21.15.131:0/2682030777 conn(0x7fc234068040 msgr2=0x7fc234069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.113+0000 7fc2395ff700 1 -- 172.21.15.131:0/2682030777 shutdown_connections 2023-12-26T22:52:24.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.113+0000 7fc2395ff700 1 -- 172.21.15.131:0/2682030777 wait complete. 2023-12-26T22:52:24.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.114+0000 7fc2395ff700 1 Processor -- start 2023-12-26T22:52:24.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.114+0000 7fc2395ff700 1 -- start start 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc2395ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc23406a1e0 0x7fc234169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc2395ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2340b0c90 0x7fc2341527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc2395ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340cb320 0x7fc2340c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc2395ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23416a130 con 0x7fc2340cb320 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc2395ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23415f130 con 0x7fc2340b0c90 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc2395ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc234150620 con 0x7fc23406a1e0 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc232ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc23406a1e0 0x7fc234169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc2337fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340cb320 0x7fc2340c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc232ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc23406a1e0 0x7fc234169bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:58504/0 (socket says 172.21.15.131:58504) 2023-12-26T22:52:24.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.115+0000 7fc232ffd700 1 -- 172.21.15.131:0/220853796 learned_addr learned my addr 172.21.15.131:0/220853796 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:24.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.116+0000 7fc2327fc700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2340b0c90 0x7fc2341527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:24.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.116+0000 7fc232ffd700 1 -- 172.21.15.131:0/220853796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2340b0c90 msgr2=0x7fc2341527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.116+0000 7fc232ffd700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2340b0c90 0x7fc2341527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.116+0000 7fc232ffd700 1 -- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340cb320 msgr2=0x7fc2340c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.116+0000 7fc232ffd700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340cb320 0x7fc2340c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.116+0000 7fc232ffd700 1 -- 172.21.15.131:0/220853796 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280094f0 con 0x7fc23406a1e0 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.116+0000 7fc2337fe700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340cb320 0x7fc2340c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.116+0000 7fc232ffd700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc23406a1e0 0x7fc234169bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e930 tx=0x7fc2240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.116+0000 7fc21affd700 1 -- 172.21.15.131:0/220853796 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224013680 con 0x7fc23406a1e0 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.117+0000 7fc21affd700 1 -- 172.21.15.131:0/220853796 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2240137e0 con 0x7fc23406a1e0 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.117+0000 7fc21affd700 1 -- 172.21.15.131:0/220853796 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224013b80 con 0x7fc23406a1e0 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.117+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23416cc30 con 0x7fc23406a1e0 2023-12-26T22:52:24.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.117+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23416d1f0 con 0x7fc23406a1e0 2023-12-26T22:52:24.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.118+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2340657e0 con 0x7fc23406a1e0 2023-12-26T22:52:24.118 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.119+0000 7fc21affd700 1 -- 172.21.15.131:0/220853796 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fc22400c3e0 con 0x7fc23406a1e0 2023-12-26T22:52:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.119+0000 7fc21affd700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc220067050 0x7fc220069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.120+0000 7fc2327fc700 1 -- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc220067050 msgr2=0x7fc220069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.120+0000 7fc21affd700 1 -- 172.21.15.131:0/220853796 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fc224084b00 con 0x7fc23406a1e0 2023-12-26T22:52:24.119 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.120+0000 7fc2327fc700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc220067050 0x7fc220069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:24.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.124+0000 7fc21affd700 1 -- 172.21.15.131:0/220853796 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc224086e60 con 0x7fc23406a1e0 2023-12-26T22:52:24.130 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:24.131+0000 7f4e1b65d1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T22:52:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.321+0000 7fc2327fc700 1 -- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc220067050 msgr2=0x7fc220069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:24.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.321+0000 7fc2327fc700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc220067050 0x7fc220069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:24.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.376+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2340c6dd0 con 0x7fc23406a1e0 2023-12-26T22:52:24.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.377+0000 7fc21affd700 1 -- 172.21.15.131:0/220853796 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fc224051880 con 0x7fc23406a1e0 2023-12-26T22:52:24.377 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:24.377 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3746500235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3746500235},{"type":"v1","addr":"172.21.15.190:6835","nonce":3746500235}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8115,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2030200373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2030200373},{"type":"v1","addr":"172.21.15.131:6833","nonce":2030200373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2894985738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2894985738},{"type":"v1","addr":"172.21.15.190:6834","nonce":2894985738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:56:28.792645+0000","modified":"2023-12-26T21:56:32.333009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8142":{"gid":8142,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1841321761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1841321761},{"type":"v1","addr":"172.21.15.131:6835","nonce":1841321761}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-12-26T22:52:24.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc220067050 msgr2=0x7fc220069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:24.379 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc220067050 0x7fc220069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc23406a1e0 msgr2=0x7fc234169bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc23406a1e0 0x7fc234169bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e930 tx=0x7fc2240091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 shutdown_connections 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc220067050 0x7fc220069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2340cb320 0x7fc2340c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc23406a1e0 0x7fc234169bf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 --2- 172.21.15.131:0/220853796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2340b0c90 0x7fc2341527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 >> 172.21.15.131:0/220853796 conn(0x7fc234068040 msgr2=0x7fc234151e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 shutdown_connections 2023-12-26T22:52:24.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.380+0000 7fc2395ff700 1 -- 172.21.15.131:0/220853796 wait complete. 2023-12-26T22:52:24.381 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 66 2023-12-26T22:52:24.398 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:52:24.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.591+0000 7ff6833ff700 1 Processor -- start 2023-12-26T22:52:24.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.591+0000 7ff6833ff700 1 -- start start 2023-12-26T22:52:24.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.592+0000 7ff6833ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff67c06bfd0 0x7ff67c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff6833ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c0686c0 con 0x7ff67c13c510 2023-12-26T22:52:24.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff6833ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c069520 con 0x7ff67c0cc100 2023-12-26T22:52:24.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff6833ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c068910 con 0x7ff67c06bfd0 2023-12-26T22:52:24.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff68119b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff67c13c510 0x7ff67c06b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48730/0 (socket says 172.21.15.131:48730) 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff68119b700 1 -- 172.21.15.131:0/1393235390 learned_addr learned my addr 172.21.15.131:0/1393235390 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff68099a700 1 --2- 172.21.15.131:0/1393235390 >> v2:172.21.15.190:3300/0 conn(0x7ff67c06bfd0 0x7ff67c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff673fff700 1 -- 172.21.15.131:0/1393235390 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943181626 0 0) 0x7ff67c0686c0 con 0x7ff67c13c510 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff673fff700 1 -- 172.21.15.131:0/1393235390 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff668003570 con 0x7ff67c13c510 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff673fff700 1 -- 172.21.15.131:0/1393235390 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877733003 0 0) 0x7ff67c069520 con 0x7ff67c0cc100 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.593+0000 7ff673fff700 1 -- 172.21.15.131:0/1393235390 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff67c0686c0 con 0x7ff67c0cc100 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.594+0000 7ff673fff700 1 -- 172.21.15.131:0/1393235390 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3587090969 0 0) 0x7ff668003570 con 0x7ff67c13c510 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.594+0000 7ff68099a700 1 -- 172.21.15.131:0/1393235390 >> v1:172.21.15.190:6789/0 conn(0x7ff67c0cc100 legacy=0x7ff67c0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.594+0000 7ff68099a700 1 -- 172.21.15.131:0/1393235390 >> v1:172.21.15.131:6789/0 conn(0x7ff67c13c510 legacy=0x7ff67c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.594+0000 7ff68099a700 1 -- 172.21.15.131:0/1393235390 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c068bf0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.594+0000 7ff68099a700 1 --2- 172.21.15.131:0/1393235390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c06bfd0 0x7ff67c0c8a20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0098e0 tx=0x7ff66c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a099f8c4b6d1283a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:24.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.594+0000 7ff673fff700 1 -- 172.21.15.131:0/1393235390 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780015f0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.594+0000 7ff673fff700 1 -- 172.21.15.131:0/1393235390 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c011420 con 0x7ff67c06bfd0 2023-12-26T22:52:24.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.594+0000 7ff673fff700 1 -- 172.21.15.131:0/1393235390 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0117c0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.595+0000 7ff6833ff700 1 -- 172.21.15.131:0/1393235390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c06bfd0 msgr2=0x7ff67c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.595+0000 7ff6833ff700 1 --2- 172.21.15.131:0/1393235390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c06bfd0 0x7ff67c0c8a20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0098e0 tx=0x7ff66c0091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.595+0000 7ff6833ff700 1 -- 172.21.15.131:0/1393235390 shutdown_connections 2023-12-26T22:52:24.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.595+0000 7ff6833ff700 1 --2- 172.21.15.131:0/1393235390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c06bfd0 0x7ff67c0c8a20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.595+0000 7ff6833ff700 1 -- 172.21.15.131:0/1393235390 >> 172.21.15.131:0/1393235390 conn(0x7ff67c0ce3e0 msgr2=0x7ff67c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:24.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.595+0000 7ff6833ff700 1 -- 172.21.15.131:0/1393235390 shutdown_connections 2023-12-26T22:52:24.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.595+0000 7ff6833ff700 1 -- 172.21.15.131:0/1393235390 wait complete. 2023-12-26T22:52:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.596+0000 7ff6833ff700 1 Processor -- start 2023-12-26T22:52:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.596+0000 7ff6833ff700 1 -- start start 2023-12-26T22:52:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.597+0000 7ff6833ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c06bfd0 0x7ff67c13be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.597+0000 7ff6833ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0cc100 0x7ff67c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.597+0000 7ff6833ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c13c510 0x7ff67c0c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.597+0000 7ff6833ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c13c3c0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.597+0000 7ff6833ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0b27f0 con 0x7ff67c0cc100 2023-12-26T22:52:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.597+0000 7ff6833ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c15a890 con 0x7ff67c13c510 2023-12-26T22:52:24.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68119b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c06bfd0 0x7ff67c13be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:24.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68099a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0cc100 0x7ff67c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:24.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68119b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c06bfd0 0x7ff67c13be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57736/0 (socket says 172.21.15.131:57736) 2023-12-26T22:52:24.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68119b700 1 -- 172.21.15.131:0/2145051534 learned_addr learned my addr 172.21.15.131:0/2145051534 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:24.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68199c700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c13c510 0x7ff67c0c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68119b700 1 -- 172.21.15.131:0/2145051534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c13c510 msgr2=0x7ff67c0c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68119b700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c13c510 0x7ff67c0c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68119b700 1 -- 172.21.15.131:0/2145051534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0cc100 msgr2=0x7ff67c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68119b700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0cc100 0x7ff67c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68119b700 1 -- 172.21.15.131:0/2145051534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c009420 con 0x7ff67c06bfd0 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68099a700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0cc100 0x7ff67c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.598+0000 7ff68119b700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c06bfd0 0x7ff67c13be80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff67800cb90 tx=0x7ff6780016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.599+0000 7ff6717fa700 1 -- 172.21.15.131:0/2145051534 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780070f0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.599+0000 7ff6717fa700 1 -- 172.21.15.131:0/2145051534 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678007290 con 0x7ff67c06bfd0 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.599+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c0b1900 con 0x7ff67c06bfd0 2023-12-26T22:52:24.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.599+0000 7ff6717fa700 1 -- 172.21.15.131:0/2145051534 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678007650 con 0x7ff67c06bfd0 2023-12-26T22:52:24.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.599+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c0b1ec0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.600+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c0657e0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.602+0000 7ff6717fa700 1 -- 172.21.15.131:0/2145051534 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7ff678005a00 con 0x7ff67c06bfd0 2023-12-26T22:52:24.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.602+0000 7ff6717fa700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff660067050 0x7ff660069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:24.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.603+0000 7ff6717fa700 1 -- 172.21.15.131:0/2145051534 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6529+0+0 (secure 0 0 0) 0x7ff6780846d0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.603+0000 7ff68099a700 1 -- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff660067050 msgr2=0x7ff660069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:24.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.603+0000 7ff68099a700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff660067050 0x7ff660069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:24.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.606+0000 7ff6717fa700 1 -- 172.21.15.131:0/2145051534 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6780514f0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.679 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:24.680+0000 7fd31b8981c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T22:52:24.771 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:24.772+0000 7fd31b8981c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T22:52:24.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.802+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff67c068ed0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.803+0000 7ff6717fa700 1 -- 172.21.15.131:0/2145051534 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16688 (secure 0 0 0) 0x7ff678005cb0 con 0x7ff67c06bfd0 2023-12-26T22:52:24.803 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:24.803 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":148,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:52:24.158434+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:56:26.517640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-26T21:56:27.777447+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":148,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"17.3f","mappings":[{"from":6,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6837/39288":"2023-12-27T21:56:44.596879+0000","172.21.15.131:6836/39288":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/3801233358":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/3315291990":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/2652223535":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/727308574":"2023-12-27T21:56:44.596879+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:52:24.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.804+0000 7ff68099a700 1 -- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff660067050 msgr2=0x7ff660069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:24.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.804+0000 7ff68099a700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff660067050 0x7ff660069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff660067050 msgr2=0x7ff660069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff660067050 0x7ff660069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c06bfd0 msgr2=0x7ff67c13be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c06bfd0 0x7ff67c13be80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff67800cb90 tx=0x7ff6780016e0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 shutdown_connections 2023-12-26T22:52:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c13c510 0x7ff67c0c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff660067050 0x7ff660069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff67c06bfd0 0x7ff67c13be80 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 --2- 172.21.15.131:0/2145051534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0cc100 0x7ff67c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 >> 172.21.15.131:0/2145051534 conn(0x7ff67c0ce3e0 msgr2=0x7ff67c155b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.805+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 shutdown_connections 2023-12-26T22:52:24.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:24.806+0000 7ff6833ff700 1 -- 172.21.15.131:0/2145051534 wait complete. 2023-12-26T22:52:24.824 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T22:52:24.824 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:52:24.908 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:24.908+0000 7f4e1b65d1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T22:52:24.923 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:24.924+0000 7fd31b8981c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T22:52:25.007 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:25.008+0000 7f4e1b65d1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T22:52:25.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.020+0000 7fa1ce9ff700 1 Processor -- start 2023-12-26T22:52:25.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.020+0000 7fa1ce9ff700 1 -- start start 2023-12-26T22:52:25.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1ce9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa1c80c8bf0 0x7fa1c80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1ce9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1c8069d10 0x7fa1c80b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1ce9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c80caa10 con 0x7fa1c80c8820 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1ce9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c8157a80 con 0x7fa1c80c8bf0 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1ce9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c80cac50 con 0x7fa1c8069d10 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1c77fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa1c80c8bf0 0x7fa1c80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1c7fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa1c80c8820 0x7fa1c80ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33886/0 (socket says 172.21.15.131:33886) 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1c7fff700 1 -- 172.21.15.131:0/708297735 learned_addr learned my addr 172.21.15.131:0/708297735 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1ccf9c700 1 --2- 172.21.15.131:0/708297735 >> v2:172.21.15.190:3300/0 conn(0x7fa1c8069d10 0x7fa1c80b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1c77fe700 1 -- 172.21.15.131:0/708297735 >> v2:172.21.15.190:3300/0 conn(0x7fa1c8069d10 msgr2=0x7fa1c80b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1c77fe700 1 --2- 172.21.15.131:0/708297735 >> v2:172.21.15.190:3300/0 conn(0x7fa1c8069d10 0x7fa1c80b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1c77fe700 1 -- 172.21.15.131:0/708297735 >> v1:172.21.15.131:6789/0 conn(0x7fa1c80c8820 legacy=0x7fa1c80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1c77fe700 1 -- 172.21.15.131:0/708297735 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c80caf30 con 0x7fa1c80c8bf0 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1ccf9c700 1 --2- 172.21.15.131:0/708297735 >> v2:172.21.15.190:3300/0 conn(0x7fa1c8069d10 0x7fa1c80b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.022+0000 7fa1c6ffd700 1 -- 172.21.15.131:0/708297735 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239237986 0 0) 0x7fa1b8001110 con 0x7fa1c80c8820 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.023+0000 7fa1c77fe700 1 --2- 172.21.15.131:0/708297735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c80c8bf0 0x7fa1c80b12e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8070f80 tx=0x7fa1bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1a643a60eebd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:25.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.023+0000 7fa1c6ffd700 1 -- 172.21.15.131:0/708297735 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc00b6b0 con 0x7fa1c80c8bf0 2023-12-26T22:52:25.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.023+0000 7fa1c6ffd700 1 -- 172.21.15.131:0/708297735 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc00b810 con 0x7fa1c80c8bf0 2023-12-26T22:52:25.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.023+0000 7fa1c6ffd700 1 -- 172.21.15.131:0/708297735 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc00bbb0 con 0x7fa1c80c8bf0 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.023+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/708297735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c80c8bf0 msgr2=0x7fa1c80b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.023+0000 7fa1ce9ff700 1 --2- 172.21.15.131:0/708297735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c80c8bf0 0x7fa1c80b12e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8070f80 tx=0x7fa1bc0091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.024+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/708297735 shutdown_connections 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.024+0000 7fa1ce9ff700 1 --2- 172.21.15.131:0/708297735 >> v2:172.21.15.190:3300/0 conn(0x7fa1c8069d10 0x7fa1c80b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.024+0000 7fa1ce9ff700 1 --2- 172.21.15.131:0/708297735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c80c8bf0 0x7fa1c80b12e0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.024+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/708297735 >> 172.21.15.131:0/708297735 conn(0x7fa1c8068040 msgr2=0x7fa1c8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.024+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/708297735 shutdown_connections 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.024+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/708297735 wait complete. 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.025+0000 7fa1ce9ff700 1 Processor -- start 2023-12-26T22:52:25.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.025+0000 7fa1ce9ff700 1 -- start start 2023-12-26T22:52:25.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.026+0000 7fa1ce9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c8069d10 0x7fa1c806ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.026+0000 7fa1ce9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c80c8820 0x7fa1c8150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.026+0000 7fa1ce9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c80c8bf0 0x7fa1c81587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.026+0000 7fa1ce9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c806e5e0 con 0x7fa1c8069d10 2023-12-26T22:52:25.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.026+0000 7fa1ce9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c8158d20 con 0x7fa1c80c8820 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.026+0000 7fa1ce9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c8153860 con 0x7fa1c80c8bf0 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.026+0000 7fa1c7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c8069d10 0x7fa1c806ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1c7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c8069d10 0x7fa1c806ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46220/0 (socket says 172.21.15.131:46220) 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1c7fff700 1 -- 172.21.15.131:0/4242246598 learned_addr learned my addr 172.21.15.131:0/4242246598 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1c77fe700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c80c8820 0x7fa1c8150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1ccf9c700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c80c8bf0 0x7fa1c81587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1c7fff700 1 -- 172.21.15.131:0/4242246598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c80c8bf0 msgr2=0x7fa1c81587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1c7fff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c80c8bf0 0x7fa1c81587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1c7fff700 1 -- 172.21.15.131:0/4242246598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c80c8820 msgr2=0x7fa1c8150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1c7fff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c80c8820 0x7fa1c8150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1c7fff700 1 -- 172.21.15.131:0/4242246598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1bc009420 con 0x7fa1c8069d10 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1c7fff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c8069d10 0x7fa1c806ea70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800c810 tx=0x7fa1b8007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.027+0000 7fa1affff700 1 -- 172.21.15.131:0/4242246598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8005b30 con 0x7fa1c8069d10 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.028+0000 7fa1affff700 1 -- 172.21.15.131:0/4242246598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b8005c90 con 0x7fa1c8069d10 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.028+0000 7fa1affff700 1 -- 172.21.15.131:0/4242246598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8007300 con 0x7fa1c8069d10 2023-12-26T22:52:25.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.028+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c8153580 con 0x7fa1c8069d10 2023-12-26T22:52:25.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.028+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c806e270 con 0x7fa1c8069d10 2023-12-26T22:52:25.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.029+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c80657e0 con 0x7fa1c8069d10 2023-12-26T22:52:25.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.030+0000 7fa1affff700 1 -- 172.21.15.131:0/4242246598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fa1b801b3b0 con 0x7fa1c8069d10 2023-12-26T22:52:25.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.030+0000 7fa1affff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa1b4067050 0x7fa1b4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.031+0000 7fa1affff700 1 -- 172.21.15.131:0/4242246598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fa1b8083db0 con 0x7fa1c8069d10 2023-12-26T22:52:25.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.031+0000 7fa1c77fe700 1 -- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa1b4067050 msgr2=0x7fa1b4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:25.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.031+0000 7fa1c77fe700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa1b4067050 0x7fa1b4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:25.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.038+0000 7fa1affff700 1 -- 172.21.15.131:0/4242246598 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1b8088070 con 0x7fa1c8069d10 2023-12-26T22:52:25.176 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:25.176+0000 7f4e1b65d1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T22:52:25.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.232+0000 7fa1c77fe700 1 -- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa1b4067050 msgr2=0x7fa1b4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:25.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.232+0000 7fa1c77fe700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa1b4067050 0x7fa1b4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:25.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.289+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1c80caa10 con 0x7fa1c8069d10 2023-12-26T22:52:25.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.289+0000 7fa1affff700 1 -- 172.21.15.131:0/4242246598 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fa1c80caa10 con 0x7fa1c8069d10 2023-12-26T22:52:25.289 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:25.289 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3746500235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3746500235},{"type":"v1","addr":"172.21.15.190:6835","nonce":3746500235}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8115,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2030200373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2030200373},{"type":"v1","addr":"172.21.15.131:6833","nonce":2030200373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2894985738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2894985738},{"type":"v1","addr":"172.21.15.190:6834","nonce":2894985738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:56:28.792645+0000","modified":"2023-12-26T21:56:32.333009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8142":{"gid":8142,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1841321761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1841321761},{"type":"v1","addr":"172.21.15.131:6835","nonce":1841321761}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-12-26T22:52:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.291+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa1b4067050 msgr2=0x7fa1b4069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.291+0000 7fa1ce9ff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa1b4067050 0x7fa1b4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.291+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c8069d10 msgr2=0x7fa1c806ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.291+0000 7fa1ce9ff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c8069d10 0x7fa1c806ea70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800c810 tx=0x7fa1b8007c50 comp rx=0 tx=0).stop 2023-12-26T22:52:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.291+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 shutdown_connections 2023-12-26T22:52:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.292+0000 7fa1ce9ff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c80c8bf0 0x7fa1c81587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.292+0000 7fa1ce9ff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa1b4067050 0x7fa1b4069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.292+0000 7fa1ce9ff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1c8069d10 0x7fa1c806ea70 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.292+0000 7fa1ce9ff700 1 --2- 172.21.15.131:0/4242246598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c80c8820 0x7fa1c8150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.292+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 >> 172.21.15.131:0/4242246598 conn(0x7fa1c8068040 msgr2=0x7fa1c8151960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.292+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 shutdown_connections 2023-12-26T22:52:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.292+0000 7fa1ce9ff700 1 -- 172.21.15.131:0/4242246598 wait complete. 2023-12-26T22:52:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 66 2023-12-26T22:52:25.309 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:52:25.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.505+0000 7f75e27ff700 1 Processor -- start 2023-12-26T22:52:25.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.506+0000 7f75e27ff700 1 -- start start 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75e27ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75dc1595e0 0x7f75dc1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75e27ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 0x7f75dc159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75e27ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc0a98b0 con 0x7f75dc0ace50 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75e27ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75dc0add40 con 0x7f75dc1595e0 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75e27ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc15a500 con 0x7f75dc15d240 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75dbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75dc1595e0 0x7f75dc1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75dbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75dc1595e0 0x7f75dc1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46224/0 (socket says 172.21.15.131:46224) 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75dbfff700 1 -- 172.21.15.131:0/2950761926 learned_addr learned my addr 172.21.15.131:0/2950761926 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75e0d9c700 1 --1- 172.21.15.131:0/2950761926 >> v1:172.21.15.131:6789/0 conn(0x7f75dc0ace50 0x7f75dc0abb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33888/0 (socket says 172.21.15.131:33888) 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.507+0000 7f75db7fe700 1 --2- 172.21.15.131:0/2950761926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 0x7f75dc159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75dbfff700 1 -- 172.21.15.131:0/2950761926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 msgr2=0x7f75dc159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:25.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75dbfff700 1 --2- 172.21.15.131:0/2950761926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 0x7f75dc159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75dbfff700 1 -- 172.21.15.131:0/2950761926 >> v1:172.21.15.131:6789/0 conn(0x7f75dc0ace50 legacy=0x7f75dc0abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75dbfff700 1 -- 172.21.15.131:0/2950761926 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc15a7e0 con 0x7f75dc1595e0 2023-12-26T22:52:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75daffd700 1 -- 172.21.15.131:0/2950761926 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887917737 0 0) 0x7f75dc0a98b0 con 0x7f75dc0ace50 2023-12-26T22:52:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75db7fe700 1 --2- 172.21.15.131:0/2950761926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 0x7f75dc159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75dbfff700 1 --2- 172.21.15.131:0/2950761926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc1595e0 0x7f75dc1599b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0099b0 tx=0x7f75cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196787985250974b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75daffd700 1 -- 172.21.15.131:0/2950761926 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00b860 con 0x7f75dc1595e0 2023-12-26T22:52:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75daffd700 1 -- 172.21.15.131:0/2950761926 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc00b9c0 con 0x7f75dc1595e0 2023-12-26T22:52:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.508+0000 7f75daffd700 1 -- 172.21.15.131:0/2950761926 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00bd60 con 0x7f75dc1595e0 2023-12-26T22:52:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.509+0000 7f75e27ff700 1 -- 172.21.15.131:0/2950761926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc1595e0 msgr2=0x7f75dc1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.509+0000 7f75e27ff700 1 --2- 172.21.15.131:0/2950761926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc1595e0 0x7f75dc1599b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0099b0 tx=0x7f75cc009250 comp rx=0 tx=0).stop 2023-12-26T22:52:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.509+0000 7f75e27ff700 1 -- 172.21.15.131:0/2950761926 shutdown_connections 2023-12-26T22:52:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.509+0000 7f75e27ff700 1 --2- 172.21.15.131:0/2950761926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc1595e0 0x7f75dc1599b0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.509+0000 7f75e27ff700 1 --2- 172.21.15.131:0/2950761926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 0x7f75dc159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.509+0000 7f75e27ff700 1 -- 172.21.15.131:0/2950761926 >> 172.21.15.131:0/2950761926 conn(0x7f75dc068040 msgr2=0x7f75dc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.509+0000 7f75e27ff700 1 -- 172.21.15.131:0/2950761926 shutdown_connections 2023-12-26T22:52:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.509+0000 7f75e27ff700 1 -- 172.21.15.131:0/2950761926 wait complete. 2023-12-26T22:52:25.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.511+0000 7f75e27ff700 1 Processor -- start 2023-12-26T22:52:25.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.511+0000 7f75e27ff700 1 -- start start 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75e27ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc0ace50 0x7f75dc0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75e27ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75dc1595e0 0x7f75dc16ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75e27ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 0x7f75dc0c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75e27ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0cbfa0 con 0x7f75dc0ace50 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75e27ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0c1f60 con 0x7f75dc1595e0 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75e27ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc16b260 con 0x7f75dc15d240 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75dbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc0ace50 0x7f75dc0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75dbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc0ace50 0x7f75dc0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46232/0 (socket says 172.21.15.131:46232) 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75dbfff700 1 -- 172.21.15.131:0/606524249 learned_addr learned my addr 172.21.15.131:0/606524249 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75e0d9c700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 0x7f75dc0c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.512+0000 7f75db7fe700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75dc1595e0 0x7f75dc16ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75dbfff700 1 -- 172.21.15.131:0/606524249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 msgr2=0x7f75dc0c23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75dbfff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 0x7f75dc0c23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75dbfff700 1 -- 172.21.15.131:0/606524249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75dc1595e0 msgr2=0x7f75dc16ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75dbfff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75dc1595e0 0x7f75dc16ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75dbfff700 1 -- 172.21.15.131:0/606524249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc0094f0 con 0x7f75dc0ace50 2023-12-26T22:52:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75dbfff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc0ace50 0x7f75dc0cba60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009290 tx=0x7f75cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75c3fff700 1 -- 172.21.15.131:0/606524249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc011e40 con 0x7f75dc0ace50 2023-12-26T22:52:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75c3fff700 1 -- 172.21.15.131:0/606524249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc01f070 con 0x7f75dc0ace50 2023-12-26T22:52:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75c3fff700 1 -- 172.21.15.131:0/606524249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0266d0 con 0x7f75dc0ace50 2023-12-26T22:52:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.513+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc0b0950 con 0x7f75dc0ace50 2023-12-26T22:52:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.514+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc0c1400 con 0x7f75dc0ace50 2023-12-26T22:52:25.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.515+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc0657e0 con 0x7f75dc0ace50 2023-12-26T22:52:25.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.515+0000 7f75c3fff700 1 -- 172.21.15.131:0/606524249 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f75cc01a430 con 0x7f75dc0ace50 2023-12-26T22:52:25.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.517+0000 7f75c3fff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f75c8067050 0x7f75c8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.517+0000 7f75db7fe700 1 -- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f75c8067050 msgr2=0x7f75c8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:25.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.517+0000 7f75c3fff700 1 -- 172.21.15.131:0/606524249 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f75cc021070 con 0x7f75dc0ace50 2023-12-26T22:52:25.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.517+0000 7f75db7fe700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f75c8067050 0x7f75c8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:25.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.521+0000 7f75c3fff700 1 -- 172.21.15.131:0/606524249 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75cc053600 con 0x7f75dc0ace50 2023-12-26T22:52:25.550 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:25.551+0000 7fd31b8981c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T22:52:25.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.718+0000 7f75db7fe700 1 -- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f75c8067050 msgr2=0x7f75c8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:25.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.718+0000 7f75db7fe700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f75c8067050 0x7f75c8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:25.736 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:25.737+0000 7fd31b8981c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T22:52:25.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.770+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75dc15aac0 con 0x7f75dc0ace50 2023-12-26T22:52:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.771+0000 7f75c3fff700 1 -- 172.21.15.131:0/606524249 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f75cc056c20 con 0x7f75dc0ace50 2023-12-26T22:52:25.770 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:25.770 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3746500235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3746500235},{"type":"v1","addr":"172.21.15.190:6835","nonce":3746500235}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8115,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2030200373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2030200373},{"type":"v1","addr":"172.21.15.131:6833","nonce":2030200373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2894985738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2894985738},{"type":"v1","addr":"172.21.15.190:6834","nonce":2894985738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T21:56:28.792645+0000","modified":"2023-12-26T21:56:32.333009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8142":{"gid":8142,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/1841321761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1841321761},{"type":"v1","addr":"172.21.15.131:6835","nonce":1841321761}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-12-26T22:52:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f75c8067050 msgr2=0x7f75c8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f75c8067050 0x7f75c8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc0ace50 msgr2=0x7f75dc0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc0ace50 0x7f75dc0cba60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009290 tx=0x7f75cc017000 comp rx=0 tx=0).stop 2023-12-26T22:52:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 shutdown_connections 2023-12-26T22:52:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75dc15d240 0x7f75dc0c23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f75c8067050 0x7f75c8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75dc0ace50 0x7f75dc0cba60 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 --2- 172.21.15.131:0/606524249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75dc1595e0 0x7f75dc16ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.773+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 >> 172.21.15.131:0/606524249 conn(0x7f75dc068040 msgr2=0x7f75dc153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.774+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 shutdown_connections 2023-12-26T22:52:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.774+0000 7f75e27ff700 1 -- 172.21.15.131:0/606524249 wait complete. 2023-12-26T22:52:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 66 2023-12-26T22:52:25.792 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:52:25.841 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:25.841+0000 7f4e1b65d1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T22:52:25.926 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:25.926+0000 7fd31b8981c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T22:52:25.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.991+0000 7f362a1ff700 1 Processor -- start 2023-12-26T22:52:25.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.991+0000 7f362a1ff700 1 -- start start 2023-12-26T22:52:25.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.992+0000 7f362a1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 0x7f36240cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.992+0000 7f362a1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36240b0550 0x7f362415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.992+0000 7f362a1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36240b0d70 con 0x7f36240b00e0 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.992+0000 7f362a1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36240680f0 con 0x7f36240b0550 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.992+0000 7f362a1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36240675b0 con 0x7f36240cc580 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.993+0000 7f3622ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f36240b00e0 0x7f362406e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33894/0 (socket says 172.21.15.131:33894) 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.993+0000 7f3622ffd700 1 -- 172.21.15.131:0/3779862458 learned_addr learned my addr 172.21.15.131:0/3779862458 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.993+0000 7f3623fff700 1 --2- 172.21.15.131:0/3779862458 >> v2:172.21.15.131:3300/0 conn(0x7f36240b0550 0x7f362415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.993+0000 7f36237fe700 1 --2- 172.21.15.131:0/3779862458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 0x7f36240cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.993+0000 7f3623fff700 1 -- 172.21.15.131:0/3779862458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 msgr2=0x7f36240cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.993+0000 7f3623fff700 1 --2- 172.21.15.131:0/3779862458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 0x7f36240cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.993+0000 7f3623fff700 1 -- 172.21.15.131:0/3779862458 >> v1:172.21.15.131:6789/0 conn(0x7f36240b00e0 legacy=0x7f362406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.993+0000 7f3623fff700 1 -- 172.21.15.131:0/3779862458 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624159c30 con 0x7f36240b0550 2023-12-26T22:52:25.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.993+0000 7f3623fff700 1 --2- 172.21.15.131:0/3779862458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b0550 0x7f362415d9b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f361c009e60 tx=0x7f361c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93dfa81c31af90f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:25.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f36227fc700 1 -- 172.21.15.131:0/3779862458 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00e070 con 0x7f36240b0550 2023-12-26T22:52:25.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f36227fc700 1 -- 172.21.15.131:0/3779862458 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c00bb60 con 0x7f36240b0550 2023-12-26T22:52:25.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f36227fc700 1 -- 172.21.15.131:0/3779862458 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c012430 con 0x7f36240b0550 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f362a1ff700 1 -- 172.21.15.131:0/3779862458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b0550 msgr2=0x7f362415d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f362a1ff700 1 --2- 172.21.15.131:0/3779862458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b0550 0x7f362415d9b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f361c009e60 tx=0x7f361c009780 comp rx=0 tx=0).stop 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f362a1ff700 1 -- 172.21.15.131:0/3779862458 shutdown_connections 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f362a1ff700 1 --2- 172.21.15.131:0/3779862458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b0550 0x7f362415d9b0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f362a1ff700 1 --2- 172.21.15.131:0/3779862458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 0x7f36240cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f362a1ff700 1 -- 172.21.15.131:0/3779862458 >> 172.21.15.131:0/3779862458 conn(0x7f36240686c0 msgr2=0x7f3624068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f362a1ff700 1 -- 172.21.15.131:0/3779862458 shutdown_connections 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.994+0000 7f362a1ff700 1 -- 172.21.15.131:0/3779862458 wait complete. 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.996+0000 7f362a1ff700 1 Processor -- start 2023-12-26T22:52:25.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.996+0000 7f362a1ff700 1 -- start start 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f362a1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b00e0 0x7f3624148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f362a1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36240b0550 0x7f36240c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f362a1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 0x7f36240c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f362a1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36240ae960 con 0x7f36240b00e0 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f362a1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36240c2240 con 0x7f36240b0550 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f362a1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36240c2520 con 0x7f36240cc580 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f36237fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b00e0 0x7f3624148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f36237fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b00e0 0x7f3624148c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46244/0 (socket says 172.21.15.131:46244) 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f36237fe700 1 -- 172.21.15.131:0/254136879 learned_addr learned my addr 172.21.15.131:0/254136879 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:25.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f3622ffd700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36240b0550 0x7f36240c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f3623fff700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 0x7f36240c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:25.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f36237fe700 1 -- 172.21.15.131:0/254136879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 msgr2=0x7f36240c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f36237fe700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 0x7f36240c4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.997+0000 7f36237fe700 1 -- 172.21.15.131:0/254136879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36240b0550 msgr2=0x7f36240c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:25.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.998+0000 7f36237fe700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36240b0550 0x7f36240c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:25.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.998+0000 7f36237fe700 1 -- 172.21.15.131:0/254136879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c009a20 con 0x7f36240b00e0 2023-12-26T22:52:25.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.998+0000 7f3623fff700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 0x7f36240c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:25.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.998+0000 7f36237fe700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b00e0 0x7f3624148c40 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f361400e930 tx=0x7f36140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:25.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.998+0000 7f360b7fe700 1 -- 172.21.15.131:0/254136879 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614013680 con 0x7f36240b00e0 2023-12-26T22:52:25.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.998+0000 7f360b7fe700 1 -- 172.21.15.131:0/254136879 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140137e0 con 0x7f36240b00e0 2023-12-26T22:52:25.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.998+0000 7f360b7fe700 1 -- 172.21.15.131:0/254136879 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614013b80 con 0x7f36240b00e0 2023-12-26T22:52:25.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.998+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36240c2800 con 0x7f36240b00e0 2023-12-26T22:52:25.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.998+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3624149180 con 0x7f36240b00e0 2023-12-26T22:52:25.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:25.999+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36240657e0 con 0x7f36240b00e0 2023-12-26T22:52:26.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.001+0000 7f360b7fe700 1 -- 172.21.15.131:0/254136879 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f361400c460 con 0x7f36240b00e0 2023-12-26T22:52:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.001+0000 7f360b7fe700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3610067050 0x7f3610069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.002+0000 7f360b7fe700 1 -- 172.21.15.131:0/254136879 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f3614084b60 con 0x7f36240b00e0 2023-12-26T22:52:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.002+0000 7f3622ffd700 1 -- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3610067050 msgr2=0x7f3610069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.002+0000 7f3622ffd700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3610067050 0x7f3610069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:26.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.005+0000 7f360b7fe700 1 -- 172.21.15.131:0/254136879 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3614051980 con 0x7f36240b00e0 2023-12-26T22:52:26.042 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:26.042+0000 7f4e1b65d1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T22:52:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.203+0000 7f3622ffd700 1 -- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3610067050 msgr2=0x7f3610069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.203+0000 7f3622ffd700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3610067050 0x7f3610069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:26.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.209+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3624067850 con 0x7f36240b00e0 2023-12-26T22:52:26.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.210+0000 7f360b7fe700 1 -- 172.21.15.131:0/254136879 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16688 (secure 0 0 0) 0x7f3614054fa0 con 0x7f36240b00e0 2023-12-26T22:52:26.210 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:26.210 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":148,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:52:24.158434+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-26T21:56:26.517640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-26T21:56:27.777447+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":148,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"17.3f","mappings":[{"from":6,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6837/39288":"2023-12-27T21:56:44.596879+0000","172.21.15.131:6836/39288":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/3801233358":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/3315291990":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/2652223535":"2023-12-27T21:56:44.596879+0000","172.21.15.131:0/727308574":"2023-12-27T21:56:44.596879+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.212+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3610067050 msgr2=0x7f3610069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.212+0000 7f362a1ff700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3610067050 0x7f3610069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.212+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b00e0 msgr2=0x7f3624148c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.212+0000 7f362a1ff700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b00e0 0x7f3624148c40 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f361400e930 tx=0x7f36140091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.212+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 shutdown_connections 2023-12-26T22:52:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.213+0000 7f362a1ff700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36240cc580 0x7f36240c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.213+0000 7f362a1ff700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3610067050 0x7f3610069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.213+0000 7f362a1ff700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36240b00e0 0x7f3624148c40 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.213+0000 7f362a1ff700 1 --2- 172.21.15.131:0/254136879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36240b0550 0x7f36240c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.213+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 >> 172.21.15.131:0/254136879 conn(0x7f36240686c0 msgr2=0x7f36240a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:26.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.213+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 shutdown_connections 2023-12-26T22:52:26.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.213+0000 7f362a1ff700 1 -- 172.21.15.131:0/254136879 wait complete. 2023-12-26T22:52:26.231 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T22:52:26.245 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:26.245+0000 7f4e1b65d1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T22:52:26.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.432+0000 7fc37a9ff700 1 Processor -- start 2023-12-26T22:52:26.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.432+0000 7fc37a9ff700 1 -- start start 2023-12-26T22:52:26.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.433+0000 7fc37a9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc3741595e0 0x7fc3741599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.433+0000 7fc37a9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3740a98b0 con 0x7fc37415d240 2023-12-26T22:52:26.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.433+0000 7fc37a9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37415a500 con 0x7fc3740ad560 2023-12-26T22:52:26.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.433+0000 7fc37a9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3740add40 con 0x7fc3741595e0 2023-12-26T22:52:26.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.433+0000 7fc373fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc3741595e0 0x7fc3741599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.433+0000 7fc3737fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc37415d240 0x7fc3740ace50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33908/0 (socket says 172.21.15.131:33908) 2023-12-26T22:52:26.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.433+0000 7fc373fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc3741595e0 0x7fc3741599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46248/0 (socket says 172.21.15.131:46248) 2023-12-26T22:52:26.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.433+0000 7fc3737fe700 1 -- 172.21.15.131:0/3929174989 learned_addr learned my addr 172.21.15.131:0/3929174989 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:26.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.434+0000 7fc373fff700 1 -- 172.21.15.131:0/3929174989 >> v1:172.21.15.190:6789/0 conn(0x7fc3740ad560 legacy=0x7fc3740aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.434+0000 7fc373fff700 1 -- 172.21.15.131:0/3929174989 >> v1:172.21.15.131:6789/0 conn(0x7fc37415d240 legacy=0x7fc3740ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.434+0000 7fc373fff700 1 -- 172.21.15.131:0/3929174989 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37415a7e0 con 0x7fc3741595e0 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.434+0000 7fc373fff700 1 --2- 172.21.15.131:0/3929174989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3741595e0 0x7fc3741599b0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc3640098e0 tx=0x7fc3640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbc51a12c38b018 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.434+0000 7fc372ffd700 1 -- 172.21.15.131:0/3929174989 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364009e80 con 0x7fc3741595e0 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.434+0000 7fc372ffd700 1 -- 172.21.15.131:0/3929174989 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36400b860 con 0x7fc3741595e0 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.434+0000 7fc372ffd700 1 -- 172.21.15.131:0/3929174989 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36400bbe0 con 0x7fc3741595e0 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.435+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3929174989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3741595e0 msgr2=0x7fc3741599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.435+0000 7fc37a9ff700 1 --2- 172.21.15.131:0/3929174989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3741595e0 0x7fc3741599b0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc3640098e0 tx=0x7fc3640091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.435+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3929174989 shutdown_connections 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.435+0000 7fc37a9ff700 1 --2- 172.21.15.131:0/3929174989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3741595e0 0x7fc3741599b0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.435+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3929174989 >> 172.21.15.131:0/3929174989 conn(0x7fc374068040 msgr2=0x7fc374069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.435+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3929174989 shutdown_connections 2023-12-26T22:52:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.435+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3929174989 wait complete. 2023-12-26T22:52:26.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.437+0000 7fc37a9ff700 1 Processor -- start 2023-12-26T22:52:26.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.437+0000 7fc37a9ff700 1 -- start start 2023-12-26T22:52:26.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc37a9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3740ad560 0x7fc3740cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc37a9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3741595e0 0x7fc37416b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc37a9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37415d240 0x7fc37415c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc37a9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c3a80 con 0x7fc37415d240 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc37a9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37415c2c0 con 0x7fc3740ad560 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc37a9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740cbe80 con 0x7fc3741595e0 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc378f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37415d240 0x7fc37415c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc373fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3740ad560 0x7fc3740cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc373fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3740ad560 0x7fc3740cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:48530/0 (socket says 172.21.15.131:48530) 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc373fff700 1 -- 172.21.15.131:0/3080191597 learned_addr learned my addr 172.21.15.131:0/3080191597 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc3737fe700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3741595e0 0x7fc37416b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc373fff700 1 -- 172.21.15.131:0/3080191597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3741595e0 msgr2=0x7fc37416b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc373fff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3741595e0 0x7fc37416b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc373fff700 1 -- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37415d240 msgr2=0x7fc37415c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc373fff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37415d240 0x7fc37415c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc373fff700 1 -- 172.21.15.131:0/3080191597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364009420 con 0x7fc3740ad560 2023-12-26T22:52:26.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.438+0000 7fc378f9c700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37415d240 0x7fc37415c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.439+0000 7fc373fff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3740ad560 0x7fc3740cb940 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc36400b3c0 tx=0x7fc364017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.439+0000 7fc35bfff700 1 -- 172.21.15.131:0/3080191597 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36400b9d0 con 0x7fc3740ad560 2023-12-26T22:52:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.439+0000 7fc35bfff700 1 -- 172.21.15.131:0/3080191597 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36400bb30 con 0x7fc3740ad560 2023-12-26T22:52:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.439+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37415b990 con 0x7fc3740ad560 2023-12-26T22:52:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.439+0000 7fc35bfff700 1 -- 172.21.15.131:0/3080191597 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364025650 con 0x7fc3740ad560 2023-12-26T22:52:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.439+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37415bf50 con 0x7fc3740ad560 2023-12-26T22:52:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.440+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37415c5a0 con 0x7fc3740ad560 2023-12-26T22:52:26.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.441+0000 7fc35bfff700 1 -- 172.21.15.131:0/3080191597 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fc364023070 con 0x7fc3740ad560 2023-12-26T22:52:26.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.442+0000 7fc35bfff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc360067050 0x7fc360069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.442+0000 7fc35bfff700 1 -- 172.21.15.131:0/3080191597 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fc364085fe0 con 0x7fc3740ad560 2023-12-26T22:52:26.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.442+0000 7fc3737fe700 1 -- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc360067050 msgr2=0x7fc360069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:26.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.442+0000 7fc3737fe700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc360067050 0x7fc360069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.452+0000 7fc35bfff700 1 -- 172.21.15.131:0/3080191597 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc364052da0 con 0x7fc3740ad560 2023-12-26T22:52:26.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.643+0000 7fc3737fe700 1 -- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc360067050 msgr2=0x7fc360069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:26.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.643+0000 7fc3737fe700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc360067050 0x7fc360069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.698+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc3740657e0 con 0x7fc3740ad560 2023-12-26T22:52:26.698 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T22:52:26.699+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T22:52:26.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.706+0000 7fc35bfff700 1 -- 172.21.15.131:0/3080191597 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc3640563c0 con 0x7fc3740ad560 2023-12-26T22:52:26.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc360067050 msgr2=0x7fc360069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:26.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc360067050 0x7fc360069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3740ad560 msgr2=0x7fc3740cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3740ad560 0x7fc3740cb940 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc36400b3c0 tx=0x7fc364017000 comp rx=0 tx=0).stop 2023-12-26T22:52:26.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 shutdown_connections 2023-12-26T22:52:26.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc360067050 0x7fc360069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37415d240 0x7fc37415c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3740ad560 0x7fc3740cb940 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 --2- 172.21.15.131:0/3080191597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3741595e0 0x7fc37416b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 >> 172.21.15.131:0/3080191597 conn(0x7fc374068040 msgr2=0x7fc374157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:26.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 shutdown_connections 2023-12-26T22:52:26.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.709+0000 7fc37a9ff700 1 -- 172.21.15.131:0/3080191597 wait complete. 2023-12-26T22:52:26.710 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T22:52:26.728 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T22:52:26.764 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T22:52:26.779 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:26.780+0000 7fd31b8981c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T22:52:26.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.960+0000 7fa0bc5ff700 1 Processor -- start 2023-12-26T22:52:26.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.961+0000 7fa0bc5ff700 1 -- start start 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0bc5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0b406f0e0 0x7fa0b406f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0bc5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa0b415ba50 0x7fa0b4157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0bc5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b4157f90 0x7fa0b40b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0bc5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b4158af0 con 0x7fa0b406f0e0 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0bc5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b415b5c0 con 0x7fa0b415ba50 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0bc5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4158530 con 0x7fa0b4157f90 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0ba39b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0b406f0e0 0x7fa0b406f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0ba39b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0b406f0e0 0x7fa0b406f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46284/0 (socket says 172.21.15.131:46284) 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0ba39b700 1 -- 172.21.15.131:0/4282399837 learned_addr learned my addr 172.21.15.131:0/4282399837 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0b9b9a700 1 --2- 172.21.15.131:0/4282399837 >> v2:172.21.15.190:3300/0 conn(0x7fa0b415ba50 0x7fa0b4157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.962+0000 7fa0bab9c700 1 --2- 172.21.15.131:0/4282399837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b4157f90 0x7fa0b40b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.963+0000 7fa0ba39b700 1 -- 172.21.15.131:0/4282399837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b4157f90 msgr2=0x7fa0b40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.963+0000 7fa0ba39b700 1 --2- 172.21.15.131:0/4282399837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b4157f90 0x7fa0b40b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.963+0000 7fa0ba39b700 1 -- 172.21.15.131:0/4282399837 >> v2:172.21.15.190:3300/0 conn(0x7fa0b415ba50 msgr2=0x7fa0b4157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:26.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.963+0000 7fa0ba39b700 1 --2- 172.21.15.131:0/4282399837 >> v2:172.21.15.190:3300/0 conn(0x7fa0b415ba50 0x7fa0b4157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.963+0000 7fa0ba39b700 1 -- 172.21.15.131:0/4282399837 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4158810 con 0x7fa0b406f0e0 2023-12-26T22:52:26.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.963+0000 7fa0ba39b700 1 --2- 172.21.15.131:0/4282399837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b406f0e0 0x7fa0b406f4b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40ccaa0 tx=0x7fa0b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad7431f29944bdb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:26.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.963+0000 7fa0b9399700 1 -- 172.21.15.131:0/4282399837 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000b640 con 0x7fa0b406f0e0 2023-12-26T22:52:26.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.963+0000 7fa0b9399700 1 -- 172.21.15.131:0/4282399837 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b000b7a0 con 0x7fa0b406f0e0 2023-12-26T22:52:26.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.963+0000 7fa0b9399700 1 -- 172.21.15.131:0/4282399837 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000bb60 con 0x7fa0b406f0e0 2023-12-26T22:52:26.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.964+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/4282399837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b406f0e0 msgr2=0x7fa0b406f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.964+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/4282399837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b406f0e0 0x7fa0b406f4b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40ccaa0 tx=0x7fa0b0009250 comp rx=0 tx=0).stop 2023-12-26T22:52:26.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.964+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/4282399837 shutdown_connections 2023-12-26T22:52:26.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.964+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/4282399837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b406f0e0 0x7fa0b406f4b0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.964+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/4282399837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b4157f90 0x7fa0b40b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.964+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/4282399837 >> v2:172.21.15.190:3300/0 conn(0x7fa0b415ba50 0x7fa0b4157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.964+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/4282399837 >> 172.21.15.131:0/4282399837 conn(0x7fa0b40686c0 msgr2=0x7fa0b4068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:26.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.964+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/4282399837 shutdown_connections 2023-12-26T22:52:26.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.964+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/4282399837 wait complete. 2023-12-26T22:52:26.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.966+0000 7fa0bc5ff700 1 Processor -- start 2023-12-26T22:52:26.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.967+0000 7fa0bc5ff700 1 -- start start 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0bc5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406f0e0 0x7fa0b40a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0bc5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b4157f90 0x7fa0b413fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0bc5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b415ba50 0x7fa0b40b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0bc5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40aaae0 con 0x7fa0b4157f90 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0bc5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40b1bd0 con 0x7fa0b415ba50 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0bc5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40a9ed0 con 0x7fa0b406f0e0 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0b9b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b4157f90 0x7fa0b413fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0b9b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b4157f90 0x7fa0b413fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46294/0 (socket says 172.21.15.131:46294) 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0b9b9a700 1 -- 172.21.15.131:0/3267922129 learned_addr learned my addr 172.21.15.131:0/3267922129 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:26.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0ba39b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406f0e0 0x7fa0b40a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.968+0000 7fa0bab9c700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b415ba50 0x7fa0b40b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0b9b9a700 1 -- 172.21.15.131:0/3267922129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406f0e0 msgr2=0x7fa0b40a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0b9b9a700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406f0e0 0x7fa0b40a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0b9b9a700 1 -- 172.21.15.131:0/3267922129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b415ba50 msgr2=0x7fa0b40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0b9b9a700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b415ba50 0x7fa0b40b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0b9b9a700 1 -- 172.21.15.131:0/3267922129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b00094f0 con 0x7fa0b4157f90 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0b9b9a700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b4157f90 0x7fa0b413fef0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400e930 tx=0x7fa0a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3267922129 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4013680 con 0x7fa0b4157f90 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3267922129 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a40137e0 con 0x7fa0b4157f90 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b40aa1b0 con 0x7fa0b4157f90 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.969+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3267922129 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4013b80 con 0x7fa0b4157f90 2023-12-26T22:52:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.970+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b40aa770 con 0x7fa0b4157f90 2023-12-26T22:52:26.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.970+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b40657e0 con 0x7fa0b4157f90 2023-12-26T22:52:26.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.972+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3267922129 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 69) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fa0a400c3e0 con 0x7fa0b4157f90 2023-12-26T22:52:26.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.973+0000 7fa0aa7fc700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 0x7fa098069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:26.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.973+0000 7fa0ba39b700 1 -- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 msgr2=0x7fa098069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:26.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.973+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3267922129 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6571+0+0 (secure 0 0 0) 0x7fa0a4084b00 con 0x7fa0b4157f90 2023-12-26T22:52:26.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.973+0000 7fa0ba39b700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 0x7fa098069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:26.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:26.976+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3267922129 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0a4051900 con 0x7fa0b4157f90 2023-12-26T22:52:27.058 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:27.059+0000 7fd31b8981c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:52:27.147 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:27.147+0000 7f4e1b65d1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T22:52:27.151 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:27.152+0000 7fd31b8981c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T22:52:27.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.174+0000 7fa0ba39b700 1 -- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 msgr2=0x7fa098069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:27.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.174+0000 7fa0ba39b700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 0x7fa098069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:27.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.222+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa0b40aac20 con 0x7fa0b4157f90 2023-12-26T22:52:27.348 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:27.348+0000 7fd31b8981c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T22:52:27.430 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:27.431+0000 7f4e1b65d1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:52:27.525 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:27.526+0000 7f4e1b65d1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T22:52:27.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.575+0000 7fa0ba39b700 1 -- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 msgr2=0x7fa098069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:27.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.575+0000 7fa0ba39b700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 0x7fa098069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T22:52:27.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.704+0000 7fa0aa7fc700 1 -- 172.21.15.131:0/3267922129 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa0a4054f20 con 0x7fa0b4157f90 2023-12-26T22:52:27.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 msgr2=0x7fa098069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:27.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 0x7fa098069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b4157f90 msgr2=0x7fa0b413fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:27.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b4157f90 0x7fa0b413fef0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400e930 tx=0x7fa0a40091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 shutdown_connections 2023-12-26T22:52:27.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0b415ba50 0x7fa0b40b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0b406f0e0 0x7fa0b40a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fa098067050 0x7fa098069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 --2- 172.21.15.131:0/3267922129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0b4157f90 0x7fa0b413fef0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 >> 172.21.15.131:0/3267922129 conn(0x7fa0b40686c0 msgr2=0x7fa0b4159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:27.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 shutdown_connections 2023-12-26T22:52:27.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.707+0000 7fa0bc5ff700 1 -- 172.21.15.131:0/3267922129 wait complete. 2023-12-26T22:52:27.726 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T22:52:27.726 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:27.726+0000 7f4e1b65d1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T22:52:27.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.938+0000 7f43333ff700 1 Processor -- start 2023-12-26T22:52:27.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.938+0000 7f43333ff700 1 -- start start 2023-12-26T22:52:27.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f43333ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f432c142f00 0x7f432c069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:27.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f43333ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f432c0cc590 0x7f432c0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:27.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f43333ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 0x7f432c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:27.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f43333ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c0cb7c0 con 0x7f432c142f00 2023-12-26T22:52:27.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f43333ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c069640 con 0x7f432c0cc590 2023-12-26T22:52:27.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f43333ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c068750 con 0x7f432c0cb340 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433119b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f432c142f00 0x7f432c069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433119b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f432c142f00 0x7f432c069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46376/0 (socket says 172.21.15.131:46376) 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433119b700 1 -- 172.21.15.131:0/2889135162 learned_addr learned my addr 172.21.15.131:0/2889135162 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433099a700 1 --2- 172.21.15.131:0/2889135162 >> v2:172.21.15.190:3300/0 conn(0x7f432c0cc590 0x7f432c0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433199c700 1 --2- 172.21.15.131:0/2889135162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 0x7f432c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433119b700 1 -- 172.21.15.131:0/2889135162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 msgr2=0x7f432c0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433119b700 1 --2- 172.21.15.131:0/2889135162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 0x7f432c0c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433119b700 1 -- 172.21.15.131:0/2889135162 >> v2:172.21.15.190:3300/0 conn(0x7f432c0cc590 msgr2=0x7f432c0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433119b700 1 --2- 172.21.15.131:0/2889135162 >> v2:172.21.15.190:3300/0 conn(0x7f432c0cc590 0x7f432c0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433119b700 1 -- 172.21.15.131:0/2889135162 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c068a30 con 0x7f432c142f00 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.940+0000 7f433099a700 1 --2- 172.21.15.131:0/2889135162 >> v2:172.21.15.190:3300/0 conn(0x7f432c0cc590 0x7f432c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.941+0000 7f433119b700 1 --2- 172.21.15.131:0/2889135162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 0x7f432c069ad0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f432c06eeb0 tx=0x7f4328009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea698bbdc52e98 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.941+0000 7f4323fff700 1 -- 172.21.15.131:0/2889135162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800b860 con 0x7f432c142f00 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.941+0000 7f4323fff700 1 -- 172.21.15.131:0/2889135162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432800b9c0 con 0x7f432c142f00 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.941+0000 7f4323fff700 1 -- 172.21.15.131:0/2889135162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800bd60 con 0x7f432c142f00 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.941+0000 7f43333ff700 1 -- 172.21.15.131:0/2889135162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 msgr2=0x7f432c069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.941+0000 7f43333ff700 1 --2- 172.21.15.131:0/2889135162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 0x7f432c069ad0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f432c06eeb0 tx=0x7f4328009250 comp rx=0 tx=0).stop 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.942+0000 7f43333ff700 1 -- 172.21.15.131:0/2889135162 shutdown_connections 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.942+0000 7f43333ff700 1 --2- 172.21.15.131:0/2889135162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 0x7f432c069ad0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.942+0000 7f43333ff700 1 --2- 172.21.15.131:0/2889135162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 0x7f432c0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.942+0000 7f43333ff700 1 --2- 172.21.15.131:0/2889135162 >> v2:172.21.15.190:3300/0 conn(0x7f432c0cc590 0x7f432c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.942+0000 7f43333ff700 1 -- 172.21.15.131:0/2889135162 >> 172.21.15.131:0/2889135162 conn(0x7f432c063cd0 msgr2=0x7f432c06b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:27.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.942+0000 7f43333ff700 1 -- 172.21.15.131:0/2889135162 shutdown_connections 2023-12-26T22:52:27.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.942+0000 7f43333ff700 1 -- 172.21.15.131:0/2889135162 wait complete. 2023-12-26T22:52:27.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.943+0000 7f43333ff700 1 Processor -- start 2023-12-26T22:52:27.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.943+0000 7f43333ff700 1 -- start start 2023-12-26T22:52:27.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.944+0000 7f43333ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 0x7f432c13fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:27.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.944+0000 7f43333ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c0cc590 0x7f432c0c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:27.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.944+0000 7f43333ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 0x7f432c0b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:27.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.944+0000 7f43333ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c16a3d0 con 0x7f432c142f00 2023-12-26T22:52:27.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.944+0000 7f43333ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c140050 con 0x7f432c0cc590 2023-12-26T22:52:27.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.944+0000 7f43333ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0c59f0 con 0x7f432c0cb340 2023-12-26T22:52:27.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.944+0000 7f433199c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 0x7f432c0b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:27.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.944+0000 7f433199c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 0x7f432c0b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46378/0 (socket says 172.21.15.131:46378) 2023-12-26T22:52:27.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.944+0000 7f433199c700 1 -- 172.21.15.131:0/3376910653 learned_addr learned my addr 172.21.15.131:0/3376910653 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:27.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f433119b700 1 --2- 172.21.15.131:0/3376910653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 0x7f432c13fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:27.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f433099a700 1 --2- 172.21.15.131:0/3376910653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c0cc590 0x7f432c0c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:27.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f433199c700 1 -- 172.21.15.131:0/3376910653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 msgr2=0x7f432c13fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:27.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f433199c700 1 --2- 172.21.15.131:0/3376910653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 0x7f432c13fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f433199c700 1 -- 172.21.15.131:0/3376910653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c0cc590 msgr2=0x7f432c0c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f433199c700 1 --2- 172.21.15.131:0/3376910653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c0cc590 0x7f432c0c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f433199c700 1 -- 172.21.15.131:0/3376910653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280094f0 con 0x7f432c142f00 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f433199c700 1 --2- 172.21.15.131:0/3376910653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 0x7f432c0b2d60 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4324009fd0 tx=0x7f4324009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f43217fa700 1 -- 172.21.15.131:0/3376910653 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432400ed50 con 0x7f432c142f00 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f43217fa700 1 -- 172.21.15.131:0/3376910653 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432400eeb0 con 0x7f432c142f00 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f43333ff700 1 -- 172.21.15.131:0/3376910653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c0c5c70 con 0x7f432c142f00 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f43217fa700 1 -- 172.21.15.131:0/3376910653 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324014650 con 0x7f432c142f00 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.945+0000 7f43333ff700 1 -- 172.21.15.131:0/3376910653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c0c6230 con 0x7f432c142f00 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.946+0000 7f43333ff700 1 -- 172.21.15.131:0/3376910653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432c16a510 con 0x7f432c142f00 2023-12-26T22:52:27.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.947+0000 7f43217fa700 1 -- 172.21.15.131:0/3376910653 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 70) v1 ==== 84453+0+0 (secure 0 0 0) 0x7f43240149c0 con 0x7f432c142f00 2023-12-26T22:52:27.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.948+0000 7f43217fa700 1 -- 172.21.15.131:0/3376910653 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6829+0+0 (secure 0 0 0) 0x7f4324084d50 con 0x7f432c142f00 2023-12-26T22:52:27.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:27.952+0000 7f43217fa700 1 -- 172.21.15.131:0/3376910653 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f432c16a510 con 0x7f432c142f00 2023-12-26T22:52:28.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.146+0000 7f43333ff700 1 -- 172.21.15.131:0/3376910653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f432c16a510 con 0x7f432c142f00 2023-12-26T22:52:28.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.741+0000 7f43217fa700 1 -- 172.21.15.131:0/3376910653 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4324051e30 con 0x7f432c142f00 2023-12-26T22:52:28.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.745+0000 7f43333ff700 1 -- 172.21.15.131:0/3376910653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 msgr2=0x7f432c0b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.745+0000 7f43333ff700 1 --2- 172.21.15.131:0/3376910653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 0x7f432c0b2d60 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4324009fd0 tx=0x7f4324009900 comp rx=0 tx=0).stop 2023-12-26T22:52:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.745+0000 7f43333ff700 1 -- 172.21.15.131:0/3376910653 shutdown_connections 2023-12-26T22:52:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.745+0000 7f43333ff700 1 --2- 172.21.15.131:0/3376910653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f432c142f00 0x7f432c0b2d60 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.745+0000 7f43333ff700 1 --2- 172.21.15.131:0/3376910653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f432c0cb340 0x7f432c13fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.745+0000 7f43333ff700 1 --2- 172.21.15.131:0/3376910653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432c0cc590 0x7f432c0c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.745+0000 7f43333ff700 1 -- 172.21.15.131:0/3376910653 >> 172.21.15.131:0/3376910653 conn(0x7f432c063cd0 msgr2=0x7f432c15c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.746+0000 7f43333ff700 1 -- 172.21.15.131:0/3376910653 shutdown_connections 2023-12-26T22:52:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.746+0000 7f43333ff700 1 -- 172.21.15.131:0/3376910653 wait complete. 2023-12-26T22:52:28.747 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T22:52:28.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T22:52:28.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.971+0000 7f25b79ff700 1 Processor -- start 2023-12-26T22:52:28.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.972+0000 7f25b79ff700 1 -- start start 2023-12-26T22:52:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.973+0000 7f25b79ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25b0159810 0x7f25b00c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.973+0000 7f25b79ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b00cbb00 con 0x7f25b00cec80 2023-12-26T22:52:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.973+0000 7f25b79ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b00ce040 con 0x7f25b00cd7b0 2023-12-26T22:52:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.973+0000 7f25b79ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00cb2a0 con 0x7f25b0159810 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.973+0000 7f25b579b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25b0159810 0x7f25b00c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.973+0000 7f25b4f9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f25b00cec80 0x7f25b00cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33916/0 (socket says 172.21.15.131:33916) 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.973+0000 7f25b579b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25b0159810 0x7f25b00c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46388/0 (socket says 172.21.15.131:46388) 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.973+0000 7f25b4f9a700 1 -- 172.21.15.131:0/2301957689 learned_addr learned my addr 172.21.15.131:0/2301957689 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25b579b700 1 -- 172.21.15.131:0/2301957689 >> v1:172.21.15.190:6789/0 conn(0x7f25b00cd7b0 legacy=0x7f25b00cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25b579b700 1 -- 172.21.15.131:0/2301957689 >> v1:172.21.15.131:6789/0 conn(0x7f25b00cec80 legacy=0x7f25b00cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25b579b700 1 -- 172.21.15.131:0/2301957689 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0159be0 con 0x7f25b0159810 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25a7fff700 1 -- 172.21.15.131:0/2301957689 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526700713 0 0) 0x7f25b00cbb00 con 0x7f25b00cec80 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25b579b700 1 --2- 172.21.15.131:0/2301957689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b0159810 0x7f25b00c5440 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0098d0 tx=0x7f25ac009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee2b816b1c0d33c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25a7fff700 1 -- 172.21.15.131:0/2301957689 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac009e70 con 0x7f25b0159810 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25a7fff700 1 -- 172.21.15.131:0/2301957689 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac005c10 con 0x7f25b0159810 2023-12-26T22:52:28.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25a7fff700 1 -- 172.21.15.131:0/2301957689 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac00b610 con 0x7f25b0159810 2023-12-26T22:52:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25b79ff700 1 -- 172.21.15.131:0/2301957689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b0159810 msgr2=0x7f25b00c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.974+0000 7f25b79ff700 1 --2- 172.21.15.131:0/2301957689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b0159810 0x7f25b00c5440 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0098d0 tx=0x7f25ac009120 comp rx=0 tx=0).stop 2023-12-26T22:52:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.975+0000 7f25b79ff700 1 -- 172.21.15.131:0/2301957689 shutdown_connections 2023-12-26T22:52:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.975+0000 7f25b79ff700 1 --2- 172.21.15.131:0/2301957689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b0159810 0x7f25b00c5440 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.975+0000 7f25b79ff700 1 -- 172.21.15.131:0/2301957689 >> 172.21.15.131:0/2301957689 conn(0x7f25b0099950 msgr2=0x7f25b006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.975+0000 7f25b79ff700 1 -- 172.21.15.131:0/2301957689 shutdown_connections 2023-12-26T22:52:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.975+0000 7f25b79ff700 1 -- 172.21.15.131:0/2301957689 wait complete. 2023-12-26T22:52:28.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.976+0000 7f25b79ff700 1 Processor -- start 2023-12-26T22:52:28.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.976+0000 7f25b79ff700 1 -- start start 2023-12-26T22:52:28.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.977+0000 7f25b79ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b00cd7b0 0x7f25b014a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:28.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.977+0000 7f25b79ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b00cec80 0x7f25b00c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:28.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.977+0000 7f25b79ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b0159810 0x7f25b015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:28.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.977+0000 7f25b79ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b0156a10 con 0x7f25b00cec80 2023-12-26T22:52:28.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.977+0000 7f25b79ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b0155e00 con 0x7f25b00cd7b0 2023-12-26T22:52:28.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.977+0000 7f25b79ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b01560e0 con 0x7f25b0159810 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.977+0000 7f25b4f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b00cec80 0x7f25b00c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.977+0000 7f25b4f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b00cec80 0x7f25b00c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46396/0 (socket says 172.21.15.131:46396) 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.977+0000 7f25b4f9a700 1 -- 172.21.15.131:0/2468511150 learned_addr learned my addr 172.21.15.131:0/2468511150 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b579b700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b00cd7b0 0x7f25b014a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b5f9c700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b0159810 0x7f25b015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b4f9a700 1 -- 172.21.15.131:0/2468511150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b0159810 msgr2=0x7f25b015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b4f9a700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b0159810 0x7f25b015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b4f9a700 1 -- 172.21.15.131:0/2468511150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b00cd7b0 msgr2=0x7f25b014a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b4f9a700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b00cd7b0 0x7f25b014a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b4f9a700 1 -- 172.21.15.131:0/2468511150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac0093c0 con 0x7f25b00cec80 2023-12-26T22:52:28.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b579b700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b00cd7b0 0x7f25b014a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:28.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b4f9a700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b00cec80 0x7f25b00c7750 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c7e0 tx=0x7f25a0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:28.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25a57fa700 1 -- 172.21.15.131:0/2468511150 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0005b30 con 0x7f25b00cec80 2023-12-26T22:52:28.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25a57fa700 1 -- 172.21.15.131:0/2468511150 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a0005c90 con 0x7f25b00cec80 2023-12-26T22:52:28.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25a57fa700 1 -- 172.21.15.131:0/2468511150 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0007300 con 0x7f25b00cec80 2023-12-26T22:52:28.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.978+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b01563c0 con 0x7f25b00cec80 2023-12-26T22:52:28.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.979+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b014a720 con 0x7f25b00cec80 2023-12-26T22:52:28.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.980+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b00717f0 con 0x7f25b00cec80 2023-12-26T22:52:28.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.980+0000 7f25a57fa700 1 -- 172.21.15.131:0/2468511150 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 71) v1 ==== 84580+0+0 (secure 0 0 0) 0x7f25a001b400 con 0x7f25b00cec80 2023-12-26T22:52:28.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.982+0000 7f25a57fa700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2594066d80 0x7f2594069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:28.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.982+0000 7f25a57fa700 1 -- 172.21.15.131:0/2468511150 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6362+0+0 (secure 0 0 0) 0x7f25a0083ff0 con 0x7f25b00cec80 2023-12-26T22:52:28.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.982+0000 7f25b579b700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2594066d80 0x7f2594069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:28.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.982+0000 7f25b579b700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2594066d80 0x7f2594069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009390 tx=0x7f25ac015000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:28.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:28.986+0000 7f25a57fa700 1 -- 172.21.15.131:0/2468511150 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25a0051230 con 0x7f25b00cec80 2023-12-26T22:52:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.188+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f25b0156b50 con 0x7f25b00cec80 2023-12-26T22:52:29.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.752+0000 7f25a57fa700 1 -- 172.21.15.131:0/2468511150 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) v1 ==== 169+0+0 (secure 0 0 0) 0x7f25b0156b50 con 0x7f25b00cec80 2023-12-26T22:52:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2594066d80 msgr2=0x7f2594069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2594066d80 0x7f2594069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009390 tx=0x7f25ac015000 comp rx=0 tx=0).stop 2023-12-26T22:52:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b00cec80 msgr2=0x7f25b00c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b00cec80 0x7f25b00c7750 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c7e0 tx=0x7f25a0007df0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 shutdown_connections 2023-12-26T22:52:29.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b0159810 0x7f25b015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b00cd7b0 0x7f25b014a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2594066d80 0x7f2594069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 --2- 172.21.15.131:0/2468511150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b00cec80 0x7f25b00c7750 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.756+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 >> 172.21.15.131:0/2468511150 conn(0x7f25b0099950 msgr2=0x7f25b00c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:29.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.757+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 shutdown_connections 2023-12-26T22:52:29.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.757+0000 7f25b79ff700 1 -- 172.21.15.131:0/2468511150 wait complete. 2023-12-26T22:52:29.758 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T22:52:29.781 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T22:52:29.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.975+0000 7f5c737ff700 1 Processor -- start 2023-12-26T22:52:29.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.975+0000 7f5c737ff700 1 -- start start 2023-12-26T22:52:29.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.976+0000 7f5c737ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c6c159a70 0x7f5c6c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:29.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.976+0000 7f5c737ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5c6c0b2a90 0x7f5c6c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:29.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.976+0000 7f5c737ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c0afcd0 con 0x7f5c6c1596a0 2023-12-26T22:52:29.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.976+0000 7f5c737ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0b3e70 con 0x7f5c6c0b2a90 2023-12-26T22:52:29.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.976+0000 7f5c737ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0b0f20 con 0x7f5c6c159a70 2023-12-26T22:52:29.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c7159b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5c6c1596a0 0x7f5c6c0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33918/0 (socket says 172.21.15.131:33918) 2023-12-26T22:52:29.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c7159b700 1 -- 172.21.15.131:0/3534831949 learned_addr learned my addr 172.21.15.131:0/3534831949 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:29.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c71d9c700 1 --2- 172.21.15.131:0/3534831949 >> v2:172.21.15.131:3300/0 conn(0x7f5c6c0b2a90 0x7f5c6c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c70d9a700 1 --2- 172.21.15.131:0/3534831949 >> v2:172.21.15.190:3300/0 conn(0x7f5c6c159a70 0x7f5c6c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c63fff700 1 -- 172.21.15.131:0/3534831949 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127282253 0 0) 0x7f5c6c0afcd0 con 0x7f5c6c1596a0 2023-12-26T22:52:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c63fff700 1 -- 172.21.15.131:0/3534831949 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c58003570 con 0x7f5c6c1596a0 2023-12-26T22:52:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c71d9c700 1 -- 172.21.15.131:0/3534831949 >> v2:172.21.15.190:3300/0 conn(0x7f5c6c159a70 msgr2=0x7f5c6c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c71d9c700 1 --2- 172.21.15.131:0/3534831949 >> v2:172.21.15.190:3300/0 conn(0x7f5c6c159a70 0x7f5c6c0b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c71d9c700 1 -- 172.21.15.131:0/3534831949 >> v1:172.21.15.131:6789/0 conn(0x7f5c6c1596a0 legacy=0x7f5c6c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c71d9c700 1 -- 172.21.15.131:0/3534831949 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0b3540 con 0x7f5c6c0b2a90 2023-12-26T22:52:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.977+0000 7f5c71d9c700 1 --2- 172.21.15.131:0/3534831949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c0b2a90 0x7f5c6c0b06a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0642f0 tx=0x7f5c64009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bbad588ff6bce1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c63fff700 1 -- 172.21.15.131:0/3534831949 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400e070 con 0x7f5c6c0b2a90 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c63fff700 1 -- 172.21.15.131:0/3534831949 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6400bb60 con 0x7f5c6c0b2a90 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c63fff700 1 -- 172.21.15.131:0/3534831949 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64012430 con 0x7f5c6c0b2a90 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c737ff700 1 -- 172.21.15.131:0/3534831949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c0b2a90 msgr2=0x7f5c6c0b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c737ff700 1 --2- 172.21.15.131:0/3534831949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c0b2a90 0x7f5c6c0b06a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0642f0 tx=0x7f5c64009780 comp rx=0 tx=0).stop 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c737ff700 1 -- 172.21.15.131:0/3534831949 shutdown_connections 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c737ff700 1 --2- 172.21.15.131:0/3534831949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c0b2a90 0x7f5c6c0b06a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c737ff700 1 --2- 172.21.15.131:0/3534831949 >> v2:172.21.15.190:3300/0 conn(0x7f5c6c159a70 0x7f5c6c0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c737ff700 1 -- 172.21.15.131:0/3534831949 >> 172.21.15.131:0/3534831949 conn(0x7f5c6c06ece0 msgr2=0x7f5c6c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c737ff700 1 -- 172.21.15.131:0/3534831949 shutdown_connections 2023-12-26T22:52:29.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.978+0000 7f5c737ff700 1 -- 172.21.15.131:0/3534831949 wait complete. 2023-12-26T22:52:29.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.980+0000 7f5c737ff700 1 Processor -- start 2023-12-26T22:52:29.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.980+0000 7f5c737ff700 1 -- start start 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c737ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c6c0b2a90 0x7f5c6c154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c737ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c1596a0 0x7f5c6c157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c737ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c6c159a70 0x7f5c6c152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c737ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1549c0 con 0x7f5c6c1596a0 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c737ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1584e0 con 0x7f5c6c0b2a90 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c737ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0c4d50 con 0x7f5c6c159a70 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c70d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c1596a0 0x7f5c6c157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c70d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c1596a0 0x7f5c6c157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46424/0 (socket says 172.21.15.131:46424) 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c70d9a700 1 -- 172.21.15.131:0/993482407 learned_addr learned my addr 172.21.15.131:0/993482407 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c7159b700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c6c0b2a90 0x7f5c6c154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.981+0000 7f5c71d9c700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c6c159a70 0x7f5c6c152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c70d9a700 1 -- 172.21.15.131:0/993482407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c6c159a70 msgr2=0x7f5c6c152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c70d9a700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c6c159a70 0x7f5c6c152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c70d9a700 1 -- 172.21.15.131:0/993482407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c6c0b2a90 msgr2=0x7f5c6c154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c70d9a700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c6c0b2a90 0x7f5c6c154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c70d9a700 1 -- 172.21.15.131:0/993482407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64009a20 con 0x7f5c6c1596a0 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c70d9a700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c1596a0 0x7f5c6c157fa0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d890 tx=0x7f5c5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c617fa700 1 -- 172.21.15.131:0/993482407 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0125f0 con 0x7f5c6c1596a0 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c617fa700 1 -- 172.21.15.131:0/993482407 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c012750 con 0x7f5c6c1596a0 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c154bb0 con 0x7f5c6c1596a0 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c617fa700 1 -- 172.21.15.131:0/993482407 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c012b10 con 0x7f5c6c1596a0 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.982+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c155170 con 0x7f5c6c1596a0 2023-12-26T22:52:29.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.983+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c0c5030 con 0x7f5c6c1596a0 2023-12-26T22:52:29.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.984+0000 7f5c617fa700 1 -- 172.21.15.131:0/993482407 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5c5c01e460 con 0x7f5c6c1596a0 2023-12-26T22:52:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.985+0000 7f5c617fa700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c50066f30 0x7f5c500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.986+0000 7f5c617fa700 1 -- 172.21.15.131:0/993482407 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f5c5c084b30 con 0x7f5c6c1596a0 2023-12-26T22:52:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.986+0000 7f5c7159b700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c50066f30 0x7f5c500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.986+0000 7f5c7159b700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c50066f30 0x7f5c500693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007fd0 tx=0x7f5c68009040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:29.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:29.990+0000 7f5c617fa700 1 -- 172.21.15.131:0/993482407 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c5c051c30 con 0x7f5c6c1596a0 2023-12-26T22:52:30.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.187+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5c6c0c4e90 con 0x7f5c6c1596a0 2023-12-26T22:52:30.190 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.191+0000 7f5c617fa700 1 -- 172.21.15.131:0/993482407 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5c6c0c4e90 con 0x7f5c6c1596a0 2023-12-26T22:52:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c50066f30 msgr2=0x7f5c500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c50066f30 0x7f5c500693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007fd0 tx=0x7f5c68009040 comp rx=0 tx=0).stop 2023-12-26T22:52:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c1596a0 msgr2=0x7f5c6c157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c1596a0 0x7f5c6c157fa0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d890 tx=0x7f5c5c008040 comp rx=0 tx=0).stop 2023-12-26T22:52:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 shutdown_connections 2023-12-26T22:52:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c6c159a70 0x7f5c6c152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c6c0b2a90 0x7f5c6c154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5c50066f30 0x7f5c500693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 --2- 172.21.15.131:0/993482407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c6c1596a0 0x7f5c6c157fa0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 >> 172.21.15.131:0/993482407 conn(0x7f5c6c06ece0 msgr2=0x7f5c6c0b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:30.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 shutdown_connections 2023-12-26T22:52:30.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.193+0000 7f5c737ff700 1 -- 172.21.15.131:0/993482407 wait complete. 2023-12-26T22:52:30.210 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T22:52:30.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.409+0000 7f17c51ff700 1 Processor -- start 2023-12-26T22:52:30.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.409+0000 7f17c51ff700 1 -- start start 2023-12-26T22:52:30.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17c51ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17c0159920 0x7f17c00b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17c51ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c0155be0 0x7f17c00b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17c51ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c00c8bf0 con 0x7f17c0159cf0 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17c51ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c00c8e60 con 0x7f17c0159920 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17c51ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00c9140 con 0x7f17c0155be0 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17be59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f17c0159cf0 0x7f17c01554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33922/0 (socket says 172.21.15.131:33922) 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17be59c700 1 -- 172.21.15.131:0/2692138705 learned_addr learned my addr 172.21.15.131:0/2692138705 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17bed9d700 1 --2- 172.21.15.131:0/2692138705 >> v2:172.21.15.190:3300/0 conn(0x7f17c0159920 0x7f17c00b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17bf59e700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c0155be0 0x7f17c00b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069845720 0 0) 0x7f17c00c8bf0 con 0x7f17c0159cf0 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.411+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17a8003570 con 0x7f17c0159cf0 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1530968894 0 0) 0x7f17a8003570 con 0x7f17c0159cf0 2023-12-26T22:52:30.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c0155be0 msgr2=0x7f17c00b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c0155be0 0x7f17c00b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 >> v2:172.21.15.190:3300/0 conn(0x7f17c0159920 msgr2=0x7f17c00b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 --2- 172.21.15.131:0/2692138705 >> v2:172.21.15.190:3300/0 conn(0x7f17c0159920 0x7f17c00b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00c9420 con 0x7f17c0159cf0 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f17b4001470 con 0x7f17c0159cf0 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bed9d700 1 --2- 172.21.15.131:0/2692138705 >> v2:172.21.15.190:3300/0 conn(0x7f17c0159920 0x7f17c00b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 >> v1:172.21.15.131:6789/0 conn(0x7f17c0159cf0 legacy=0x7f17c01554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17a8003b10 0x7f17a8003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c0155be0 0x7f17a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a8004d50 0x7f17a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00c8e60 con 0x7f17a8004d50 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00c9140 con 0x7f17a8003b10 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a800bb40 con 0x7f17c0155be0 2023-12-26T22:52:30.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bf59e700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a8004d50 0x7f17a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17be59c700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c0155be0 0x7f17a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.412+0000 7f17bed9d700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17a8003b10 0x7f17a8003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.413+0000 7f17bf59e700 1 -- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c0155be0 msgr2=0x7f17a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.413+0000 7f17bf59e700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c0155be0 0x7f17a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.413+0000 7f17bf59e700 1 -- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17a8003b10 msgr2=0x7f17a8003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.413+0000 7f17bf59e700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17a8003b10 0x7f17a8003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.413+0000 7f17bf59e700 1 -- 172.21.15.131:0/2692138705 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a8003570 con 0x7f17a8004d50 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.413+0000 7f17bf59e700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a8004d50 0x7f17a80051c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e210 tx=0x7f17b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16dc25a142e4de49 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.413+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800ea60 con 0x7f17a8004d50 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.413+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b800f040 con 0x7f17a8004d50 2023-12-26T22:52:30.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.413+0000 7f17bdd9b700 1 -- 172.21.15.131:0/2692138705 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800ee10 con 0x7f17a8004d50 2023-12-26T22:52:30.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.414+0000 7f17c51ff700 1 -- 172.21.15.131:0/2692138705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a8004d50 msgr2=0x7f17a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.414+0000 7f17c51ff700 1 --2- 172.21.15.131:0/2692138705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a8004d50 0x7f17a80051c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e210 tx=0x7f17b800b040 comp rx=0 tx=0).stop 2023-12-26T22:52:30.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.414+0000 7f17bf59e700 1 -- 172.21.15.131:0/2692138705 reap_dead start 2023-12-26T22:52:30.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.414+0000 7f17c51ff700 1 -- 172.21.15.131:0/2692138705 shutdown_connections 2023-12-26T22:52:30.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.414+0000 7f17c51ff700 1 -- 172.21.15.131:0/2692138705 >> 172.21.15.131:0/2692138705 conn(0x7f17c0068040 msgr2=0x7f17c0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:30.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.414+0000 7f17c51ff700 1 -- 172.21.15.131:0/2692138705 shutdown_connections 2023-12-26T22:52:30.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.414+0000 7f17c51ff700 1 -- 172.21.15.131:0/2692138705 wait complete. 2023-12-26T22:52:30.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.415+0000 7f17c51ff700 1 Processor -- start 2023-12-26T22:52:30.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.415+0000 7f17c51ff700 1 -- start start 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.416+0000 7f17c51ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c015df00 0x7f17c015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.416+0000 7f17c51ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c014e630 0x7f17c014eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.416+0000 7f17c51ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c015a4b0 0x7f17c015a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.416+0000 7f17c51ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c014efe0 con 0x7f17c015a4b0 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.416+0000 7f17c51ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0150a60 con 0x7f17c015df00 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.416+0000 7f17c51ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c014fe50 con 0x7f17c014e630 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.416+0000 7f17bf59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c015a4b0 0x7f17c015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.416+0000 7f17bf59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c015a4b0 0x7f17c015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46444/0 (socket says 172.21.15.131:46444) 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.416+0000 7f17bf59e700 1 -- 172.21.15.131:0/1133206347 learned_addr learned my addr 172.21.15.131:0/1133206347 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17bed9d700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c015df00 0x7f17c015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17be59c700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c014e630 0x7f17c014eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17bf59e700 1 -- 172.21.15.131:0/1133206347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c014e630 msgr2=0x7f17c014eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17bf59e700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c014e630 0x7f17c014eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.419 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17bf59e700 1 -- 172.21.15.131:0/1133206347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c015df00 msgr2=0x7f17c015bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17bf59e700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c015df00 0x7f17c015bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17bf59e700 1 -- 172.21.15.131:0/1133206347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c0150130 con 0x7f17c015a4b0 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17be59c700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c014e630 0x7f17c014eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17bf59e700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c015a4b0 0x7f17c015a930 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f17b80094d0 tx=0x7f17b801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17a6ffd700 1 -- 172.21.15.131:0/1133206347 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b801e640 con 0x7f17c015a4b0 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17a6ffd700 1 -- 172.21.15.131:0/1133206347 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8022070 con 0x7f17c015a4b0 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17a6ffd700 1 -- 172.21.15.131:0/1133206347 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b801e9d0 con 0x7f17c015a4b0 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.417+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c0150410 con 0x7f17c015a4b0 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.418+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c015c060 con 0x7f17c015a4b0 2023-12-26T22:52:30.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.419+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c00657e0 con 0x7f17c015a4b0 2023-12-26T22:52:30.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.420+0000 7f17a6ffd700 1 -- 172.21.15.131:0/1133206347 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f17b8003040 con 0x7f17c015a4b0 2023-12-26T22:52:30.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.421+0000 7f17a6ffd700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f17ac066f30 0x7f17ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.421+0000 7f17a6ffd700 1 -- 172.21.15.131:0/1133206347 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f17b8024070 con 0x7f17c015a4b0 2023-12-26T22:52:30.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.421+0000 7f17bed9d700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f17ac066f30 0x7f17ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.421+0000 7f17bed9d700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f17ac066f30 0x7f17ac0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f17c0159620 tx=0x7f17b0008040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:30.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.424+0000 7f17a6ffd700 1 -- 172.21.15.131:0/1133206347 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17b8054320 con 0x7f17c015a4b0 2023-12-26T22:52:30.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.616+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f17c0150bd0 con 0x7f17c015a4b0 2023-12-26T22:52:30.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.620+0000 7f17a6ffd700 1 -- 172.21.15.131:0/1133206347 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v50) v1 ==== 115+0+0 (secure 0 0 0) 0x7f17b8009780 con 0x7f17c015a4b0 2023-12-26T22:52:30.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f17ac066f30 msgr2=0x7f17ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f17ac066f30 0x7f17ac0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f17c0159620 tx=0x7f17b0008040 comp rx=0 tx=0).stop 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c015a4b0 msgr2=0x7f17c015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c015a4b0 0x7f17c015a930 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f17b80094d0 tx=0x7f17b801a000 comp rx=0 tx=0).stop 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 shutdown_connections 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f17ac066f30 0x7f17ac0693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17c015a4b0 0x7f17c015a930 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c015df00 0x7f17c015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 --2- 172.21.15.131:0/1133206347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c014e630 0x7f17c014eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 >> 172.21.15.131:0/1133206347 conn(0x7f17c0068040 msgr2=0x7f17c00cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 shutdown_connections 2023-12-26T22:52:30.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.623+0000 7f17c51ff700 1 -- 172.21.15.131:0/1133206347 wait complete. 2023-12-26T22:52:30.641 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T22:52:30.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.833+0000 7f03e4dff700 1 Processor -- start 2023-12-26T22:52:30.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.834+0000 7f03e4dff700 1 -- start start 2023-12-26T22:52:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.835+0000 7f03e4dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03e00d01e0 0x7f03e00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.835+0000 7f03e4dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e00cfd50 con 0x7f03e0159600 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.835+0000 7f03e4dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e00b1280 con 0x7f03e01599d0 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.835+0000 7f03e4dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e00cf140 con 0x7f03e00d01e0 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.835+0000 7f03de59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f03e0159600 0x7f03e015dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33930/0 (socket says 172.21.15.131:33930) 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.835+0000 7f03de59c700 1 -- 172.21.15.131:0/3801874274 learned_addr learned my addr 172.21.15.131:0/3801874274 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.835+0000 7f03ded9d700 1 --2- 172.21.15.131:0/3801874274 >> v2:172.21.15.131:3300/0 conn(0x7f03e00d01e0 0x7f03e00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.836+0000 7f03dd59a700 1 -- 172.21.15.131:0/3801874274 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771810528 0 0) 0x7f03e00cfd50 con 0x7f03e0159600 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.836+0000 7f03dd59a700 1 -- 172.21.15.131:0/3801874274 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c8003570 con 0x7f03e0159600 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.836+0000 7f03ded9d700 1 -- 172.21.15.131:0/3801874274 >> v1:172.21.15.190:6789/0 conn(0x7f03e01599d0 legacy=0x7f03e00ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.836+0000 7f03ded9d700 1 -- 172.21.15.131:0/3801874274 >> v1:172.21.15.131:6789/0 conn(0x7f03e0159600 legacy=0x7f03e015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.836+0000 7f03ded9d700 1 -- 172.21.15.131:0/3801874274 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00cf420 con 0x7f03e00d01e0 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.836+0000 7f03ded9d700 1 --2- 172.21.15.131:0/3801874274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 0x7f03e00ceb60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009e60 tx=0x7f03d8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c80bb235d22ca4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.836+0000 7f03dd59a700 1 -- 172.21.15.131:0/3801874274 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800e070 con 0x7f03e00d01e0 2023-12-26T22:52:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.836+0000 7f03dd59a700 1 -- 172.21.15.131:0/3801874274 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d800bb60 con 0x7f03e00d01e0 2023-12-26T22:52:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.836+0000 7f03dd59a700 1 -- 172.21.15.131:0/3801874274 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8012430 con 0x7f03e00d01e0 2023-12-26T22:52:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.837+0000 7f03e4dff700 1 -- 172.21.15.131:0/3801874274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 msgr2=0x7f03e00ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.837+0000 7f03e4dff700 1 --2- 172.21.15.131:0/3801874274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 0x7f03e00ceb60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009e60 tx=0x7f03d8009780 comp rx=0 tx=0).stop 2023-12-26T22:52:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.837+0000 7f03e4dff700 1 -- 172.21.15.131:0/3801874274 shutdown_connections 2023-12-26T22:52:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.837+0000 7f03e4dff700 1 --2- 172.21.15.131:0/3801874274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 0x7f03e00ceb60 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.837+0000 7f03e4dff700 1 -- 172.21.15.131:0/3801874274 >> 172.21.15.131:0/3801874274 conn(0x7f03e006ece0 msgr2=0x7f03e006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.837+0000 7f03e4dff700 1 -- 172.21.15.131:0/3801874274 shutdown_connections 2023-12-26T22:52:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.837+0000 7f03e4dff700 1 -- 172.21.15.131:0/3801874274 wait complete. 2023-12-26T22:52:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.838+0000 7f03e4dff700 1 Processor -- start 2023-12-26T22:52:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.838+0000 7f03e4dff700 1 -- start start 2023-12-26T22:52:30.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.839+0000 7f03e4dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 0x7f03e015be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.839+0000 7f03e4dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e0159600 0x7f03e015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.839+0000 7f03e4dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03e015a5c0 0x7f03e00c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.839+0000 7f03e4dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e015ba00 con 0x7f03e00d01e0 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.839+0000 7f03e4dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e0156d20 con 0x7f03e0159600 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.839+0000 7f03e4dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e015adf0 con 0x7f03e015a5c0 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.839+0000 7f03de59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 0x7f03e015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.839+0000 7f03de59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 0x7f03e015be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46458/0 (socket says 172.21.15.131:46458) 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.839+0000 7f03de59c700 1 -- 172.21.15.131:0/1467835511 learned_addr learned my addr 172.21.15.131:0/1467835511 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03ddd9b700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e0159600 0x7f03e015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03ded9d700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03e015a5c0 0x7f03e00c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03de59c700 1 -- 172.21.15.131:0/1467835511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03e015a5c0 msgr2=0x7f03e00c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03de59c700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03e015a5c0 0x7f03e00c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03de59c700 1 -- 172.21.15.131:0/1467835511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e0159600 msgr2=0x7f03e015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03de59c700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e0159600 0x7f03e015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03de59c700 1 -- 172.21.15.131:0/1467835511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d8009a20 con 0x7f03e00d01e0 2023-12-26T22:52:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03de59c700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 0x7f03e015be90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f03d000c790 tx=0x7f03d0007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03ce7fc700 1 -- 172.21.15.131:0/1467835511 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0007300 con 0x7f03e00d01e0 2023-12-26T22:52:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03ce7fc700 1 -- 172.21.15.131:0/1467835511 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0007460 con 0x7f03e00d01e0 2023-12-26T22:52:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e015b0d0 con 0x7f03e00d01e0 2023-12-26T22:52:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03ce7fc700 1 -- 172.21.15.131:0/1467835511 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d00057f0 con 0x7f03e00d01e0 2023-12-26T22:52:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.840+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e015b690 con 0x7f03e00d01e0 2023-12-26T22:52:30.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.841+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e00657e0 con 0x7f03e00d01e0 2023-12-26T22:52:30.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.842+0000 7f03ce7fc700 1 -- 172.21.15.131:0/1467835511 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f03d0005c20 con 0x7f03e00d01e0 2023-12-26T22:52:30.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.843+0000 7f03ce7fc700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f03c0066fb0 0x7f03c0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:30.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.843+0000 7f03ce7fc700 1 -- 172.21.15.131:0/1467835511 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f03d0084160 con 0x7f03e00d01e0 2023-12-26T22:52:30.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.844+0000 7f03ddd9b700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f03c0066fb0 0x7f03c0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:30.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.844+0000 7f03ddd9b700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f03c0066fb0 0x7f03c0069460 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f03e00d1600 tx=0x7f03d4009040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:30.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:30.847+0000 7f03ce7fc700 1 -- 172.21.15.131:0/1467835511 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03d00512e0 con 0x7f03e00d01e0 2023-12-26T22:52:31.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.039+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f03e00cf700 con 0x7f03e00d01e0 2023-12-26T22:52:31.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.045+0000 7f03ce7fc700 1 -- 172.21.15.131:0/1467835511 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v51) v1 ==== 120+0+0 (secure 0 0 0) 0x7f03d0054900 con 0x7f03e00d01e0 2023-12-26T22:52:31.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f03c0066fb0 msgr2=0x7f03c0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f03c0066fb0 0x7f03c0069460 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f03e00d1600 tx=0x7f03d4009040 comp rx=0 tx=0).stop 2023-12-26T22:52:31.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 msgr2=0x7f03e015be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 0x7f03e015be90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f03d000c790 tx=0x7f03d0007c80 comp rx=0 tx=0).stop 2023-12-26T22:52:31.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 shutdown_connections 2023-12-26T22:52:31.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03e015a5c0 0x7f03e00c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f03c0066fb0 0x7f03c0069460 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e00d01e0 0x7f03e015be90 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 --2- 172.21.15.131:0/1467835511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e0159600 0x7f03e015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 >> 172.21.15.131:0/1467835511 conn(0x7f03e006ece0 msgr2=0x7f03e0068580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:31.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 shutdown_connections 2023-12-26T22:52:31.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.048+0000 7f03e4dff700 1 -- 172.21.15.131:0/1467835511 wait complete. 2023-12-26T22:52:31.066 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-12-26T22:52:31.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.258+0000 7fc3c33ff700 1 Processor -- start 2023-12-26T22:52:31.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.258+0000 7fc3c33ff700 1 -- start start 2023-12-26T22:52:31.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.259+0000 7fc3c33ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc1596a0 0x7fc3bc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.259+0000 7fc3c33ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc3bc159a70 0x7fc3bc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.259+0000 7fc3c33ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3bc0afcd0 con 0x7fc3bc0b0160 2023-12-26T22:52:31.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.259+0000 7fc3c33ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3bc0b3e70 con 0x7fc3bc159a70 2023-12-26T22:52:31.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.259+0000 7fc3c33ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0b0cd0 con 0x7fc3bc1596a0 2023-12-26T22:52:31.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c099a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc3bc159a70 0x7fc3bc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c119b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc1596a0 0x7fc3bc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c099a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc3bc159a70 0x7fc3bc0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:48726/0 (socket says 172.21.15.131:48726) 2023-12-26T22:52:31.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c119b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc1596a0 0x7fc3bc0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:50260/0 (socket says 172.21.15.131:50260) 2023-12-26T22:52:31.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c099a700 1 -- 172.21.15.131:0/8452485 learned_addr learned my addr 172.21.15.131:0/8452485 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:31.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c199c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fc3bc0b0160 0x7fc3bc0b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:38344/0 (socket says 172.21.15.131:38344) 2023-12-26T22:52:31.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c099a700 1 -- 172.21.15.131:0/8452485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc1596a0 msgr2=0x7fc3bc0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:31.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c099a700 1 --2- 172.21.15.131:0/8452485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc1596a0 0x7fc3bc0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c099a700 1 -- 172.21.15.131:0/8452485 >> v1:172.21.15.190:6789/0 conn(0x7fc3bc0b0160 legacy=0x7fc3bc0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c099a700 1 -- 172.21.15.131:0/8452485 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0b2e00 con 0x7fc3bc159a70 2023-12-26T22:52:31.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.260+0000 7fc3c099a700 1 --2- 172.21.15.131:0/8452485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3bc159a70 0x7fc3bc0b28c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0099b0 tx=0x7fc3ac009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa16e295e092fd05 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:31.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3b3fff700 1 -- 172.21.15.131:0/8452485 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac009ea0 con 0x7fc3bc159a70 2023-12-26T22:52:31.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3b3fff700 1 -- 172.21.15.131:0/8452485 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac00b860 con 0x7fc3bc159a70 2023-12-26T22:52:31.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3b3fff700 1 -- 172.21.15.131:0/8452485 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00bc00 con 0x7fc3bc159a70 2023-12-26T22:52:31.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3c33ff700 1 -- 172.21.15.131:0/8452485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3bc159a70 msgr2=0x7fc3bc0b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3c33ff700 1 --2- 172.21.15.131:0/8452485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3bc159a70 0x7fc3bc0b28c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0099b0 tx=0x7fc3ac009250 comp rx=0 tx=0).stop 2023-12-26T22:52:31.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3c33ff700 1 -- 172.21.15.131:0/8452485 shutdown_connections 2023-12-26T22:52:31.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3c33ff700 1 --2- 172.21.15.131:0/8452485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc1596a0 0x7fc3bc0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3c33ff700 1 --2- 172.21.15.131:0/8452485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3bc159a70 0x7fc3bc0b28c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3c33ff700 1 -- 172.21.15.131:0/8452485 >> 172.21.15.131:0/8452485 conn(0x7fc3bc06ece0 msgr2=0x7fc3bc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:31.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3c33ff700 1 -- 172.21.15.131:0/8452485 shutdown_connections 2023-12-26T22:52:31.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.261+0000 7fc3c33ff700 1 -- 172.21.15.131:0/8452485 wait complete. 2023-12-26T22:52:31.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.263+0000 7fc3c33ff700 1 Processor -- start 2023-12-26T22:52:31.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.263+0000 7fc3c33ff700 1 -- start start 2023-12-26T22:52:31.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c33ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3bc0b0160 0x7fc3bc0c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c33ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3bc1596a0 0x7fc3bc15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c33ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc159a70 0x7fc3bc156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c33ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc15c3b0 con 0x7fc3bc1596a0 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c33ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0c45f0 con 0x7fc3bc0b0160 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c33ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0c48d0 con 0x7fc3bc159a70 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3bc1596a0 0x7fc3bc15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3bc1596a0 0x7fc3bc15be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46472/0 (socket says 172.21.15.131:46472) 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c099a700 1 -- 172.21.15.131:0/36900914 learned_addr learned my addr 172.21.15.131:0/36900914 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c119b700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3bc0b0160 0x7fc3bc0c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.264+0000 7fc3c199c700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc159a70 0x7fc3bc156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3c099a700 1 -- 172.21.15.131:0/36900914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc159a70 msgr2=0x7fc3bc156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3c099a700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc159a70 0x7fc3bc156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3c099a700 1 -- 172.21.15.131:0/36900914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3bc0b0160 msgr2=0x7fc3bc0c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3c099a700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3bc0b0160 0x7fc3bc0c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3c099a700 1 -- 172.21.15.131:0/36900914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac0094f0 con 0x7fc3bc1596a0 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3c199c700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc159a70 0x7fc3bc156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3c099a700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3bc1596a0 0x7fc3bc15be70 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc089710 tx=0x7fc3ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3b17fa700 1 -- 172.21.15.131:0/36900914 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00bbe0 con 0x7fc3bc1596a0 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3b17fa700 1 -- 172.21.15.131:0/36900914 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac011420 con 0x7fc3bc1596a0 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3b17fa700 1 -- 172.21.15.131:0/36900914 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0117c0 con 0x7fc3bc1596a0 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3bc0c4bb0 con 0x7fc3bc1596a0 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.265+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3bc157080 con 0x7fc3bc1596a0 2023-12-26T22:52:31.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.266+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3bc16d3f0 con 0x7fc3bc1596a0 2023-12-26T22:52:31.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.267+0000 7fc3b17fa700 1 -- 172.21.15.131:0/36900914 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc3ac011bf0 con 0x7fc3bc1596a0 2023-12-26T22:52:31.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.268+0000 7fc3b17fa700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc3a0066f30 0x7fc3a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.268+0000 7fc3b17fa700 1 -- 172.21.15.131:0/36900914 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5919+0+0 (secure 0 0 0) 0x7fc3ac085f40 con 0x7fc3bc1596a0 2023-12-26T22:52:31.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.268+0000 7fc3c119b700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc3a0066f30 0x7fc3a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.269+0000 7fc3c119b700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc3a0066f30 0x7fc3a00693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009db0 tx=0x7fc3b80091a0 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:31.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.272+0000 7fc3b17fa700 1 -- 172.21.15.131:0/36900914 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3ac053040 con 0x7fc3bc1596a0 2023-12-26T22:52:31.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.541+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fc3bc16d580 con 0x7fc3bc1596a0 2023-12-26T22:52:31.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.768+0000 7fc3b17fa700 1 -- 172.21.15.131:0/36900914 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2438) v1 ==== 178+0+0 (secure 0 0 0) 0x7fc3ac056660 con 0x7fc3bc1596a0 2023-12-26T22:52:31.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.770+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc3a0066f30 msgr2=0x7fc3a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.770+0000 7fc3c33ff700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc3a0066f30 0x7fc3a00693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009db0 tx=0x7fc3b80091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.770+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3bc1596a0 msgr2=0x7fc3bc15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.770+0000 7fc3c33ff700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3bc1596a0 0x7fc3bc15be70 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc089710 tx=0x7fc3ac017000 comp rx=0 tx=0).stop 2023-12-26T22:52:31.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.771+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 shutdown_connections 2023-12-26T22:52:31.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.771+0000 7fc3c33ff700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3bc159a70 0x7fc3bc156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.771+0000 7fc3c33ff700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3bc0b0160 0x7fc3bc0c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.771+0000 7fc3c33ff700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc3a0066f30 0x7fc3a00693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.771+0000 7fc3c33ff700 1 --2- 172.21.15.131:0/36900914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3bc1596a0 0x7fc3bc15be70 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.771+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 >> 172.21.15.131:0/36900914 conn(0x7fc3bc06ece0 msgr2=0x7fc3bc1589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:31.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.771+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 shutdown_connections 2023-12-26T22:52:31.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.771+0000 7fc3c33ff700 1 -- 172.21.15.131:0/36900914 wait complete. 2023-12-26T22:52:31.789 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-12-26T22:52:31.789 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-12-26T22:52:31.789 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T22:52:31.790 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-12-26T22:52:31.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.980+0000 7fbf24dff700 1 Processor -- start 2023-12-26T22:52:31.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.980+0000 7fbf24dff700 1 -- start start 2023-12-26T22:52:31.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.981+0000 7fbf24dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf200cb320 0x7fbf20069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.981+0000 7fbf24dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbf200b0e60 0x7fbf200c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.981+0000 7fbf24dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf20156940 con 0x7fbf2006a1e0 2023-12-26T22:52:31.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.981+0000 7fbf24dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf200c8cc0 con 0x7fbf200cb320 2023-12-26T22:52:31.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.981+0000 7fbf24dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf200ad700 con 0x7fbf200b0e60 2023-12-26T22:52:31.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.981+0000 7fbf1e59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf200cb320 0x7fbf20069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.981+0000 7fbf1e59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf200cb320 0x7fbf20069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46486/0 (socket says 172.21.15.131:46486) 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.981+0000 7fbf1e59c700 1 -- 172.21.15.131:0/2991255880 learned_addr learned my addr 172.21.15.131:0/2991255880 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.982+0000 7fbf1ed9d700 1 --2- 172.21.15.131:0/2991255880 >> v2:172.21.15.190:3300/0 conn(0x7fbf200b0e60 0x7fbf200c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.982+0000 7fbf1e59c700 1 -- 172.21.15.131:0/2991255880 >> v2:172.21.15.190:3300/0 conn(0x7fbf200b0e60 msgr2=0x7fbf200c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.982+0000 7fbf1e59c700 1 --2- 172.21.15.131:0/2991255880 >> v2:172.21.15.190:3300/0 conn(0x7fbf200b0e60 0x7fbf200c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.982+0000 7fbf1e59c700 1 -- 172.21.15.131:0/2991255880 >> v1:172.21.15.190:6789/0 conn(0x7fbf2006a1e0 legacy=0x7fbf200b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.982+0000 7fbf1e59c700 1 -- 172.21.15.131:0/2991255880 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf200ae500 con 0x7fbf200cb320 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.982+0000 7fbf1e59c700 1 --2- 172.21.15.131:0/2991255880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200cb320 0x7fbf20069ca0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fbf2006fb50 tx=0x7fbf10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e939d39101fe1c48 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.982+0000 7fbf1d59a700 1 -- 172.21.15.131:0/2991255880 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000b820 con 0x7fbf200cb320 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.982+0000 7fbf1d59a700 1 -- 172.21.15.131:0/2991255880 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1000b980 con 0x7fbf200cb320 2023-12-26T22:52:31.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.982+0000 7fbf1d59a700 1 -- 172.21.15.131:0/2991255880 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000bd20 con 0x7fbf200cb320 2023-12-26T22:52:31.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.983+0000 7fbf24dff700 1 -- 172.21.15.131:0/2991255880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200cb320 msgr2=0x7fbf20069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.983+0000 7fbf24dff700 1 --2- 172.21.15.131:0/2991255880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200cb320 0x7fbf20069ca0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fbf2006fb50 tx=0x7fbf10009250 comp rx=0 tx=0).stop 2023-12-26T22:52:31.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.983+0000 7fbf24dff700 1 -- 172.21.15.131:0/2991255880 shutdown_connections 2023-12-26T22:52:31.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.983+0000 7fbf24dff700 1 --2- 172.21.15.131:0/2991255880 >> v2:172.21.15.190:3300/0 conn(0x7fbf200b0e60 0x7fbf200c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.983+0000 7fbf24dff700 1 --2- 172.21.15.131:0/2991255880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200cb320 0x7fbf20069ca0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.983+0000 7fbf24dff700 1 -- 172.21.15.131:0/2991255880 >> 172.21.15.131:0/2991255880 conn(0x7fbf20068040 msgr2=0x7fbf20069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:31.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.983+0000 7fbf24dff700 1 -- 172.21.15.131:0/2991255880 shutdown_connections 2023-12-26T22:52:31.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.983+0000 7fbf24dff700 1 -- 172.21.15.131:0/2991255880 wait complete. 2023-12-26T22:52:31.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.984+0000 7fbf24dff700 1 Processor -- start 2023-12-26T22:52:31.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.985+0000 7fbf24dff700 1 -- start start 2023-12-26T22:52:31.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.985+0000 7fbf24dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf2006a1e0 0x7fbf2014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.985+0000 7fbf24dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200b0e60 0x7fbf200ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.985+0000 7fbf24dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf200cb320 0x7fbf200ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.985+0000 7fbf24dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf20152a30 con 0x7fbf200b0e60 2023-12-26T22:52:31.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.985+0000 7fbf24dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf20152dd0 con 0x7fbf2006a1e0 2023-12-26T22:52:31.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.985+0000 7fbf24dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf20152190 con 0x7fbf200cb320 2023-12-26T22:52:31.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200b0e60 0x7fbf200ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200b0e60 0x7fbf200ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46488/0 (socket says 172.21.15.131:46488) 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1dd9b700 1 -- 172.21.15.131:0/1424917909 learned_addr learned my addr 172.21.15.131:0/1424917909 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1e59c700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf2006a1e0 0x7fbf2014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1ed9d700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf200cb320 0x7fbf200ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1dd9b700 1 -- 172.21.15.131:0/1424917909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf200cb320 msgr2=0x7fbf200ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1dd9b700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf200cb320 0x7fbf200ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1dd9b700 1 -- 172.21.15.131:0/1424917909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf2006a1e0 msgr2=0x7fbf2014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1dd9b700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf2006a1e0 0x7fbf2014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.986+0000 7fbf1dd9b700 1 -- 172.21.15.131:0/1424917909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100094f0 con 0x7fbf200b0e60 2023-12-26T22:52:31.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.987+0000 7fbf1dd9b700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200b0e60 0x7fbf200ca3d0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400c590 tx=0x7fbf14007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:31.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.987+0000 7fbf0e7fc700 1 -- 172.21.15.131:0/1424917909 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011590 con 0x7fbf200b0e60 2023-12-26T22:52:31.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.987+0000 7fbf0e7fc700 1 -- 172.21.15.131:0/1424917909 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf140116f0 con 0x7fbf200b0e60 2023-12-26T22:52:31.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.987+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf20152470 con 0x7fbf200b0e60 2023-12-26T22:52:31.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.987+0000 7fbf0e7fc700 1 -- 172.21.15.131:0/1424917909 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011ab0 con 0x7fbf200b0e60 2023-12-26T22:52:31.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.987+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2014e5a0 con 0x7fbf200b0e60 2023-12-26T22:52:31.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.988+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf20152ba0 con 0x7fbf200b0e60 2023-12-26T22:52:31.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.989+0000 7fbf0e7fc700 1 -- 172.21.15.131:0/1424917909 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbf1401c460 con 0x7fbf200b0e60 2023-12-26T22:52:31.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.990+0000 7fbf0e7fc700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbf00066f30 0x7fbf000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:31.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.990+0000 7fbf0e7fc700 1 -- 172.21.15.131:0/1424917909 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5919+0+0 (secure 0 0 0) 0x7fbf14084250 con 0x7fbf200b0e60 2023-12-26T22:52:31.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.990+0000 7fbf1e59c700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbf00066f30 0x7fbf000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:31.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.991+0000 7fbf1e59c700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbf00066f30 0x7fbf000693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbf2006fb50 tx=0x7fbf10016000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:31.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:31.994+0000 7fbf0e7fc700 1 -- 172.21.15.131:0/1424917909 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf140513f0 con 0x7fbf200b0e60 2023-12-26T22:52:32.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.263+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7fbf20152f40 con 0x7fbf200b0e60 2023-12-26T22:52:32.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.768+0000 7fbf0e7fc700 1 -- 172.21.15.131:0/1424917909 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2439) v1 ==== 177+0+0 (secure 0 0 0) 0x7fbf14054a10 con 0x7fbf200b0e60 2023-12-26T22:52:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbf00066f30 msgr2=0x7fbf000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbf00066f30 0x7fbf000693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbf2006fb50 tx=0x7fbf10016000 comp rx=0 tx=0).stop 2023-12-26T22:52:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200b0e60 msgr2=0x7fbf200ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200b0e60 0x7fbf200ca3d0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400c590 tx=0x7fbf14007f30 comp rx=0 tx=0).stop 2023-12-26T22:52:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 shutdown_connections 2023-12-26T22:52:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf200cb320 0x7fbf200ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf2006a1e0 0x7fbf2014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:32.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbf00066f30 0x7fbf000693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:32.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 --2- 172.21.15.131:0/1424917909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf200b0e60 0x7fbf200ca3d0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:32.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 >> 172.21.15.131:0/1424917909 conn(0x7fbf20068040 msgr2=0x7fbf2014ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:32.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 shutdown_connections 2023-12-26T22:52:32.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.771+0000 7fbf24dff700 1 -- 172.21.15.131:0/1424917909 wait complete. 2023-12-26T22:52:32.791 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T22:52:32.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.987+0000 7f3b439ff700 1 Processor -- start 2023-12-26T22:52:32.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.987+0000 7f3b439ff700 1 -- start start 2023-12-26T22:52:32.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.988+0000 7f3b439ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c159820 0x7f3b3c0abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:32.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.988+0000 7f3b439ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b3c0aa480 0x7f3b3c06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:32.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.988+0000 7f3b439ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3c15a8c0 con 0x7f3b3c06e2b0 2023-12-26T22:52:32.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.988+0000 7f3b439ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3c15ad50 con 0x7f3b3c0aa480 2023-12-26T22:52:32.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.988+0000 7f3b439ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c06e730 con 0x7f3b3c159820 2023-12-26T22:52:32.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b40f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b3c0aa480 0x7f3b3c06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:32.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b40f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b3c0aa480 0x7f3b3c06dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46502/0 (socket says 172.21.15.131:46502) 2023-12-26T22:52:32.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b40f9a700 1 -- 172.21.15.131:0/865243280 learned_addr learned my addr 172.21.15.131:0/865243280 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:32.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b4179b700 1 --2- 172.21.15.131:0/865243280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c159820 0x7f3b3c0abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:32.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b40f9a700 1 -- 172.21.15.131:0/865243280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c159820 msgr2=0x7f3b3c0abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b40f9a700 1 --2- 172.21.15.131:0/865243280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c159820 0x7f3b3c0abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b40f9a700 1 -- 172.21.15.131:0/865243280 >> v1:172.21.15.190:6789/0 conn(0x7f3b3c06e2b0 legacy=0x7f3b3c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b40f9a700 1 -- 172.21.15.131:0/865243280 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c159f90 con 0x7f3b3c0aa480 2023-12-26T22:52:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b4179b700 1 --2- 172.21.15.131:0/865243280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c159820 0x7f3b3c0abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.989+0000 7f3b40f9a700 1 --2- 172.21.15.131:0/865243280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 0x7f3b3c06dd70 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0099e0 tx=0x7f3b2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9e77f14732c489 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b33fff700 1 -- 172.21.15.131:0/865243280 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00b7e0 con 0x7f3b3c0aa480 2023-12-26T22:52:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b33fff700 1 -- 172.21.15.131:0/865243280 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c00b940 con 0x7f3b3c0aa480 2023-12-26T22:52:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b33fff700 1 -- 172.21.15.131:0/865243280 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00bce0 con 0x7f3b3c0aa480 2023-12-26T22:52:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b439ff700 1 -- 172.21.15.131:0/865243280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 msgr2=0x7f3b3c06dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:32.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b439ff700 1 --2- 172.21.15.131:0/865243280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 0x7f3b3c06dd70 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0099e0 tx=0x7f3b2c009250 comp rx=0 tx=0).stop 2023-12-26T22:52:32.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b439ff700 1 -- 172.21.15.131:0/865243280 shutdown_connections 2023-12-26T22:52:32.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b439ff700 1 --2- 172.21.15.131:0/865243280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c159820 0x7f3b3c0abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:32.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b439ff700 1 --2- 172.21.15.131:0/865243280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 0x7f3b3c06dd70 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:32.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b439ff700 1 -- 172.21.15.131:0/865243280 >> 172.21.15.131:0/865243280 conn(0x7f3b3c06c290 msgr2=0x7f3b3c0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:32.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b439ff700 1 -- 172.21.15.131:0/865243280 shutdown_connections 2023-12-26T22:52:32.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.990+0000 7f3b439ff700 1 -- 172.21.15.131:0/865243280 wait complete. 2023-12-26T22:52:32.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.992+0000 7f3b439ff700 1 Processor -- start 2023-12-26T22:52:32.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.992+0000 7f3b439ff700 1 -- start start 2023-12-26T22:52:32.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b439ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c06e2b0 0x7f3b3c1410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b439ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 0x7f3b3c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b439ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3c159820 0x7f3b3c067830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b439ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c16aee0 con 0x7f3b3c0aa480 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b439ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c16b880 con 0x7f3b3c159820 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b439ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c1415e0 con 0x7f3b3c06e2b0 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b40f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 0x7f3b3c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b40f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 0x7f3b3c15eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46510/0 (socket says 172.21.15.131:46510) 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b40f9a700 1 -- 172.21.15.131:0/3352306700 learned_addr learned my addr 172.21.15.131:0/3352306700 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b4179b700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c06e2b0 0x7f3b3c1410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b40f9a700 1 -- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c06e2b0 msgr2=0x7f3b3c1410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b40f9a700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c06e2b0 0x7f3b3c1410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b40f9a700 1 -- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3c159820 msgr2=0x7f3b3c067830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b41f9c700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3c159820 0x7f3b3c067830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b40f9a700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3c159820 0x7f3b3c067830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b40f9a700 1 -- 172.21.15.131:0/3352306700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c0094f0 con 0x7f3b3c0aa480 2023-12-26T22:52:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b4179b700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c06e2b0 0x7f3b3c1410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.993+0000 7f3b41f9c700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3c159820 0x7f3b3c067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.994+0000 7f3b40f9a700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 0x7f3b3c15eef0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009290 tx=0x7f3b2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.994+0000 7f3b317fa700 1 -- 172.21.15.131:0/3352306700 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c011d50 con 0x7f3b3c0aa480 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.994+0000 7f3b317fa700 1 -- 172.21.15.131:0/3352306700 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c020070 con 0x7f3b3c0aa480 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.994+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c15f430 con 0x7f3b3c0aa480 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.994+0000 7f3b317fa700 1 -- 172.21.15.131:0/3352306700 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c01b6a0 con 0x7f3b3c0aa480 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.994+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c16b230 con 0x7f3b3c0aa480 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.995+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c16b090 con 0x7f3b3c0aa480 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.996+0000 7f3b317fa700 1 -- 172.21.15.131:0/3352306700 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3b2c022030 con 0x7f3b3c0aa480 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.997+0000 7f3b317fa700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b20066f30 0x7f3b200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.997+0000 7f3b317fa700 1 -- 172.21.15.131:0/3352306700 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f3b2c086650 con 0x7f3b3c0aa480 2023-12-26T22:52:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.997+0000 7f3b4179b700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b20066f30 0x7f3b200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:32.998+0000 7f3b4179b700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b20066f30 0x7f3b200693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b38008b20 tx=0x7f3b38008040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.001+0000 7f3b317fa700 1 -- 172.21.15.131:0/3352306700 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b3c16b090 con 0x7f3b3c0aa480 2023-12-26T22:52:33.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.195+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3b3c16b090 con 0x7f3b3c0aa480 2023-12-26T22:52:33.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.201+0000 7f3b317fa700 1 -- 172.21.15.131:0/3352306700 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3b3c16b090 con 0x7f3b3c0aa480 2023-12-26T22:52:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b20066f30 msgr2=0x7f3b200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b20066f30 0x7f3b200693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b38008b20 tx=0x7f3b38008040 comp rx=0 tx=0).stop 2023-12-26T22:52:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 msgr2=0x7f3b3c15eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 0x7f3b3c15eef0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009290 tx=0x7f3b2c016000 comp rx=0 tx=0).stop 2023-12-26T22:52:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 shutdown_connections 2023-12-26T22:52:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3c159820 0x7f3b3c067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.204 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c06e2b0 0x7f3b3c1410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b20066f30 0x7f3b200693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 --2- 172.21.15.131:0/3352306700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3c0aa480 0x7f3b3c15eef0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 >> 172.21.15.131:0/3352306700 conn(0x7f3b3c06c290 msgr2=0x7f3b3c157840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 shutdown_connections 2023-12-26T22:52:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.204+0000 7f3b439ff700 1 -- 172.21.15.131:0/3352306700 wait complete. 2023-12-26T22:52:33.221 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T22:52:33.221 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T22:52:33.310 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T22:52:33.402 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T22:52:33.402 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T22:52:33.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.592+0000 7fb2641ff700 1 Processor -- start 2023-12-26T22:52:33.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.592+0000 7fb2641ff700 1 -- start start 2023-12-26T22:52:33.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.593+0000 7fb2641ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb25c13c510 0x7fb25c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.593+0000 7fb2641ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c0cc810 0x7fb25c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.593+0000 7fb2641ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25c0686c0 con 0x7fb25c06b8c0 2023-12-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.593+0000 7fb2641ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c06c280 con 0x7fb25c13c510 2023-12-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.593+0000 7fb2641ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c069520 con 0x7fb25c0cc810 2023-12-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb26179a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb25c06b8c0 0x7fb25c0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33946/0 (socket says 172.21.15.131:33946) 2023-12-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb26179a700 1 -- 172.21.15.131:0/2653442216 learned_addr learned my addr 172.21.15.131:0/2653442216 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb261f9b700 1 --2- 172.21.15.131:0/2653442216 >> v2:172.21.15.190:3300/0 conn(0x7fb25c13c510 0x7fb25c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb26279c700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c0cc810 0x7fb25c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359974157 0 0) 0x7fb25c0686c0 con 0x7fb25c06b8c0 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb248003570 con 0x7fb25c06b8c0 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 925776842 0 0) 0x7fb248003570 con 0x7fb25c06b8c0 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c0cc810 msgr2=0x7fb25c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c0cc810 0x7fb25c06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 >> v2:172.21.15.190:3300/0 conn(0x7fb25c13c510 msgr2=0x7fb25c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 --2- 172.21.15.131:0/2653442216 >> v2:172.21.15.190:3300/0 conn(0x7fb25c13c510 0x7fb25c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c068910 con 0x7fb25c06b8c0 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fb24c001470 con 0x7fb25c06b8c0 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.594+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 >> v1:172.21.15.131:6789/0 conn(0x7fb25c06b8c0 legacy=0x7fb25c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb260f99700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c0cc810 0x7fb248003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb260f99700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2480040b0 0x7fb248004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb260f99700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb2480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c06c280 con 0x7fb248004d50 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c069520 con 0x7fb2480040b0 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24800bb60 con 0x7fb25c0cc810 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb26279c700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb2480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb26179a700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2480040b0 0x7fb248004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb261f9b700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c0cc810 0x7fb248003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb26279c700 1 -- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c0cc810 msgr2=0x7fb248003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb26279c700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c0cc810 0x7fb248003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb26279c700 1 -- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2480040b0 msgr2=0x7fb248004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb26279c700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2480040b0 0x7fb248004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.595+0000 7fb26279c700 1 -- 172.21.15.131:0/2653442216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb248003570 con 0x7fb248004d50 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb26279c700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb2480051c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb2540094d0 tx=0x7fb25400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6aa7e43ebe90a66 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400ea90 con 0x7fb248004d50 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25400f040 con 0x7fb248004d50 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb260f99700 1 -- 172.21.15.131:0/2653442216 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400ee20 con 0x7fb248004d50 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb2641ff700 1 -- 172.21.15.131:0/2653442216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 msgr2=0x7fb2480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb2641ff700 1 --2- 172.21.15.131:0/2653442216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb2480051c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb2540094d0 tx=0x7fb25400b040 comp rx=0 tx=0).stop 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb26279c700 1 -- 172.21.15.131:0/2653442216 reap_dead start 2023-12-26T22:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb2641ff700 1 -- 172.21.15.131:0/2653442216 shutdown_connections 2023-12-26T22:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb2641ff700 1 -- 172.21.15.131:0/2653442216 >> 172.21.15.131:0/2653442216 conn(0x7fb25c0ce3e0 msgr2=0x7fb25c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb2641ff700 1 -- 172.21.15.131:0/2653442216 shutdown_connections 2023-12-26T22:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.596+0000 7fb2641ff700 1 -- 172.21.15.131:0/2653442216 wait complete. 2023-12-26T22:52:33.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.598+0000 7fb2641ff700 1 Processor -- start 2023-12-26T22:52:33.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.598+0000 7fb2641ff700 1 -- start start 2023-12-26T22:52:33.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb2641ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb25c163310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb2641ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c15d9b0 0x7fb25c15de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb2641ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c15aff0 0x7fb25c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb2641ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c15e360 con 0x7fb248004d50 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb2641ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c163850 con 0x7fb25c15aff0 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb2641ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c15c960 con 0x7fb25c15d9b0 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb261f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb25c163310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb261f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb25c163310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46514/0 (socket says 172.21.15.131:46514) 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb261f9b700 1 -- 172.21.15.131:0/456346971 learned_addr learned my addr 172.21.15.131:0/456346971 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb26179a700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c15d9b0 0x7fb25c15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb26279c700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c15aff0 0x7fb25c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb261f9b700 1 -- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c15d9b0 msgr2=0x7fb25c15de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb261f9b700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c15d9b0 0x7fb25c15de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb261f9b700 1 -- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c15aff0 msgr2=0x7fb25c15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb261f9b700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c15aff0 0x7fb25c15b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb261f9b700 1 -- 172.21.15.131:0/456346971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c15bd50 con 0x7fb248004d50 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb26179a700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c15d9b0 0x7fb25c15de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.599+0000 7fb26279c700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c15aff0 0x7fb25c15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.600+0000 7fb261f9b700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb25c163310 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb258011a40 tx=0x7fb258009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.600+0000 7fb251ffb700 1 -- 172.21.15.131:0/456346971 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258005c10 con 0x7fb248004d50 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.600+0000 7fb251ffb700 1 -- 172.21.15.131:0/456346971 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb258005d70 con 0x7fb248004d50 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.600+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c15c030 con 0x7fb248004d50 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.600+0000 7fb251ffb700 1 -- 172.21.15.131:0/456346971 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25800f960 con 0x7fb248004d50 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.600+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c15c5f0 con 0x7fb248004d50 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.601+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c0657e0 con 0x7fb248004d50 2023-12-26T22:52:33.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.602+0000 7fb251ffb700 1 -- 172.21.15.131:0/456346971 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb25800fd90 con 0x7fb248004d50 2023-12-26T22:52:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.603+0000 7fb251ffb700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb240066f30 0x7fb2400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.603+0000 7fb251ffb700 1 -- 172.21.15.131:0/456346971 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5919+0+0 (secure 0 0 0) 0x7fb258084fd0 con 0x7fb248004d50 2023-12-26T22:52:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.603+0000 7fb26179a700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb240066f30 0x7fb2400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.604+0000 7fb26179a700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb240066f30 0x7fb2400693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0015a0 tx=0x7fb24c009040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.607+0000 7fb251ffb700 1 -- 172.21.15.131:0/456346971 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb258052170 con 0x7fb248004d50 2023-12-26T22:52:33.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.849+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb25c068bf0 con 0x7fb248004d50 2023-12-26T22:52:33.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.849+0000 7fb251ffb700 1 -- 172.21.15.131:0/456346971 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fb258021df0 con 0x7fb248004d50 2023-12-26T22:52:33.849 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:33.849 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3746500235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3746500235},{"type":"v1","addr":"172.21.15.190:6835","nonce":3746500235}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1336505415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1336505415},{"type":"v1","addr":"172.21.15.131:6835","nonce":1336505415}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":8115,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2030200373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2030200373},{"type":"v1","addr":"172.21.15.131:6833","nonce":2030200373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2894985738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2894985738},{"type":"v1","addr":"172.21.15.190:6834","nonce":2894985738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[]} 2023-12-26T22:52:33.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb240066f30 msgr2=0x7fb2400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb240066f30 0x7fb2400693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0015a0 tx=0x7fb24c009040 comp rx=0 tx=0).stop 2023-12-26T22:52:33.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 msgr2=0x7fb25c163310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:33.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb25c163310 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb258011a40 tx=0x7fb258009120 comp rx=0 tx=0).stop 2023-12-26T22:52:33.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 shutdown_connections 2023-12-26T22:52:33.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb25c15aff0 0x7fb25c15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb240066f30 0x7fb2400693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb248004d50 0x7fb25c163310 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 --2- 172.21.15.131:0/456346971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb25c15d9b0 0x7fb25c15de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:33.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 >> 172.21.15.131:0/456346971 conn(0x7fb25c0ce3e0 msgr2=0x7fb25c0c3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:33.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 shutdown_connections 2023-12-26T22:52:33.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:33.852+0000 7fb2641ff700 1 -- 172.21.15.131:0/456346971 wait complete. 2023-12-26T22:52:33.853 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 68 2023-12-26T22:52:33.870 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T22:52:33.870 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T22:52:33.870 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T22:52:33.870 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T22:52:33.871 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T22:52:33.871 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T22:52:33.871 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T22:52:33.871 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T22:52:33.871 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T22:52:33.872 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T22:52:33.872 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T22:52:33.872 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T22:52:33.872 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T22:52:33.872 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T22:52:33.873 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T22:52:33.873 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T22:52:33.885 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T22:52:33.885 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T22:52:33.890 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T22:52:33.890 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T22:52:33.897 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T22:52:33.897 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T22:52:33.898 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T22:52:33.898 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T22:52:33.933 INFO:tasks.ceph.mds.a:Started 2023-12-26T22:52:33.934 DEBUG:teuthology.parallel:result is None 2023-12-26T22:52:33.937 INFO:tasks.ceph.mds.c:Started 2023-12-26T22:52:33.937 DEBUG:teuthology.parallel:result is None 2023-12-26T22:52:33.943 INFO:tasks.ceph.mds.b:Started 2023-12-26T22:52:33.943 DEBUG:teuthology.parallel:result is None 2023-12-26T22:52:33.946 INFO:tasks.ceph.mds.d:Started 2023-12-26T22:52:33.946 DEBUG:teuthology.parallel:result is None 2023-12-26T22:52:33.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T22:52:34.076 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T22:52:34.076 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T22:52:34.089 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T22:52:34.092 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T22:52:34.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.150+0000 7f66b67ff700 1 Processor -- start 2023-12-26T22:52:34.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.150+0000 7f66b67ff700 1 -- start start 2023-12-26T22:52:34.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.151+0000 7f66b67ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b0159560 0x7f66b015d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:34.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.151+0000 7f66b67ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66b006a1a0 0x7f66b0068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:34.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.151+0000 7f66b67ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b0068a80 con 0x7f66b0159930 2023-12-26T22:52:34.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.151+0000 7f66b67ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b00cac50 con 0x7f66b006a1a0 2023-12-26T22:52:34.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.151+0000 7f66b67ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b015da00 con 0x7f66b0159560 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.151+0000 7f66b4d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66b006a1a0 0x7f66b0068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.151+0000 7f66b4d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66b006a1a0 0x7f66b0068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46554/0 (socket says 172.21.15.131:46554) 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66b4d9c700 1 -- 172.21.15.131:0/226076023 learned_addr learned my addr 172.21.15.131:0/226076023 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.151+0000 7f66af7fe700 1 --1- 172.21.15.131:0/226076023 >> v1:172.21.15.131:6789/0 conn(0x7f66b0159930 0x7f66b00b11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33954/0 (socket says 172.21.15.131:33954) 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66affff700 1 --2- 172.21.15.131:0/226076023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b0159560 0x7f66b015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66b4d9c700 1 -- 172.21.15.131:0/226076023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b0159560 msgr2=0x7f66b015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66b4d9c700 1 --2- 172.21.15.131:0/226076023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b0159560 0x7f66b015d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66b4d9c700 1 -- 172.21.15.131:0/226076023 >> v1:172.21.15.131:6789/0 conn(0x7f66b0159930 legacy=0x7f66b00b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66b4d9c700 1 -- 172.21.15.131:0/226076023 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b015dce0 con 0x7f66b006a1a0 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66affff700 1 --2- 172.21.15.131:0/226076023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b0159560 0x7f66b015d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66aeffd700 1 -- 172.21.15.131:0/226076023 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163536423 0 0) 0x7f66b0068a80 con 0x7f66b0159930 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66b4d9c700 1 --2- 172.21.15.131:0/226076023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b006a1a0 0x7f66b0068f10 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009f20 tx=0x7f66a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ec1e7fcbdfb9d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66aeffd700 1 -- 172.21.15.131:0/226076023 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800e070 con 0x7f66b006a1a0 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66aeffd700 1 -- 172.21.15.131:0/226076023 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a800ba80 con 0x7f66b006a1a0 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.152+0000 7f66aeffd700 1 -- 172.21.15.131:0/226076023 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a80092e0 con 0x7f66b006a1a0 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.153+0000 7f66b67ff700 1 -- 172.21.15.131:0/226076023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b006a1a0 msgr2=0x7f66b0068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.153+0000 7f66b67ff700 1 --2- 172.21.15.131:0/226076023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b006a1a0 0x7f66b0068f10 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009f20 tx=0x7f66a8009840 comp rx=0 tx=0).stop 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.153+0000 7f66b67ff700 1 -- 172.21.15.131:0/226076023 shutdown_connections 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.153+0000 7f66b67ff700 1 --2- 172.21.15.131:0/226076023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b006a1a0 0x7f66b0068f10 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.153+0000 7f66b67ff700 1 --2- 172.21.15.131:0/226076023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b0159560 0x7f66b015d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.153+0000 7f66b67ff700 1 -- 172.21.15.131:0/226076023 >> 172.21.15.131:0/226076023 conn(0x7f66b00ce3d0 msgr2=0x7f66b00ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.153+0000 7f66b67ff700 1 -- 172.21.15.131:0/226076023 shutdown_connections 2023-12-26T22:52:34.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.153+0000 7f66b67ff700 1 -- 172.21.15.131:0/226076023 wait complete. 2023-12-26T22:52:34.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.155+0000 7f66b67ff700 1 Processor -- start 2023-12-26T22:52:34.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.155+0000 7f66b67ff700 1 -- start start 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.155+0000 7f66b67ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b006a1a0 0x7f66b0157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66b67ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b0159560 0x7f66b015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66b67ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b0159930 0x7f66b015c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66b67ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b00c2d90 con 0x7f66b0159560 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66b67ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b00c2180 con 0x7f66b0159930 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66b67ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b00c2460 con 0x7f66b006a1a0 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66af7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b0159560 0x7f66b015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66af7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b0159560 0x7f66b015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46562/0 (socket says 172.21.15.131:46562) 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66af7fe700 1 -- 172.21.15.131:0/3625545326 learned_addr learned my addr 172.21.15.131:0/3625545326 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66affff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b006a1a0 0x7f66b0157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66b4d9c700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b0159930 0x7f66b015c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66af7fe700 1 -- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b006a1a0 msgr2=0x7f66b0157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66af7fe700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b006a1a0 0x7f66b0157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66af7fe700 1 -- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b0159930 msgr2=0x7f66b015c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66af7fe700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b0159930 0x7f66b015c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66af7fe700 1 -- 172.21.15.131:0/3625545326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8009ae0 con 0x7f66b0159560 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66b4d9c700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b0159930 0x7f66b015c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66affff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b006a1a0 0x7f66b0157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.156+0000 7f66af7fe700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b0159560 0x7f66b015c430 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f66a40011e0 tx=0x7f66a4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.157+0000 7f6697fff700 1 -- 172.21.15.131:0/3625545326 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4007300 con 0x7f66b0159560 2023-12-26T22:52:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.157+0000 7f6697fff700 1 -- 172.21.15.131:0/3625545326 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a4007460 con 0x7f66b0159560 2023-12-26T22:52:34.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.157+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b00c2740 con 0x7f66b0159560 2023-12-26T22:52:34.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.157+0000 7f6697fff700 1 -- 172.21.15.131:0/3625545326 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4004830 con 0x7f66b0159560 2023-12-26T22:52:34.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.157+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b00c1e30 con 0x7f66b0159560 2023-12-26T22:52:34.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.158+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b00c2f40 con 0x7f66b0159560 2023-12-26T22:52:34.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.159+0000 7f6697fff700 1 -- 172.21.15.131:0/3625545326 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f66a4004be0 con 0x7f66b0159560 2023-12-26T22:52:34.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.160+0000 7f6697fff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f669c066f30 0x7f669c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:34.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.160+0000 7f6697fff700 1 -- 172.21.15.131:0/3625545326 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f66a4083bb0 con 0x7f66b0159560 2023-12-26T22:52:34.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.160+0000 7f66affff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f669c066f30 0x7f669c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:34.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.160+0000 7f66affff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f669c066f30 0x7f669c0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006fd0 tx=0x7f66a0008040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:34.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.164+0000 7f6697fff700 1 -- 172.21.15.131:0/3625545326 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66b00c2f40 con 0x7f66b0159560 2023-12-26T22:52:34.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:34.376+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f66b00657e0 con 0x7f66b0159560 2023-12-26T22:52:35.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.217+0000 7f6697fff700 1 -- 172.21.15.131:0/3625545326 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v157) v1 ==== 97+0+0 (secure 0 0 0) 0x7f66a4018020 con 0x7f66b0159560 2023-12-26T22:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f669c066f30 msgr2=0x7f669c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f669c066f30 0x7f669c0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006fd0 tx=0x7f66a0008040 comp rx=0 tx=0).stop 2023-12-26T22:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b0159560 msgr2=0x7f66b015c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b0159560 0x7f66b015c430 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f66a40011e0 tx=0x7f66a4007e10 comp rx=0 tx=0).stop 2023-12-26T22:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 shutdown_connections 2023-12-26T22:52:35.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b0159930 0x7f66b015c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b006a1a0 0x7f66b0157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f669c066f30 0x7f669c0693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 --2- 172.21.15.131:0/3625545326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b0159560 0x7f66b015c430 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 >> 172.21.15.131:0/3625545326 conn(0x7f66b00ce3d0 msgr2=0x7f66b0158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:35.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 shutdown_connections 2023-12-26T22:52:35.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.220+0000 7f66b67ff700 1 -- 172.21.15.131:0/3625545326 wait complete. 2023-12-26T22:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T22:52:35.239 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T22:52:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.432+0000 7f45b51ff700 1 Processor -- start 2023-12-26T22:52:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.433+0000 7f45b51ff700 1 -- start start 2023-12-26T22:52:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.434+0000 7f45b51ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f45b00b0750 0x7f45b00b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.434+0000 7f45b51ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b0156940 con 0x7f45b00c9400 2023-12-26T22:52:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.434+0000 7f45b51ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b00ae500 con 0x7f45b00cb320 2023-12-26T22:52:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.434+0000 7f45b51ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b00c8cc0 con 0x7f45b00b0750 2023-12-26T22:52:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.434+0000 7f45af59e700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f45b00c9400 0x7f45b00c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58830/0 (socket says 172.21.15.131:58830) 2023-12-26T22:52:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.434+0000 7f45af59e700 1 -- 172.21.15.131:0/2650169984 learned_addr learned my addr 172.21.15.131:0/2650169984 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.434+0000 7f45ae59c700 1 --2- 172.21.15.131:0/2650169984 >> v2:172.21.15.190:3300/0 conn(0x7f45b00b0750 0x7f45b00b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.434+0000 7f45add9b700 1 -- 172.21.15.131:0/2650169984 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732658779 0 0) 0x7f45b0156940 con 0x7f45b00c9400 2023-12-26T22:52:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45add9b700 1 -- 172.21.15.131:0/2650169984 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4598003570 con 0x7f45b00c9400 2023-12-26T22:52:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45add9b700 1 -- 172.21.15.131:0/2650169984 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645710068 0 0) 0x7f45b00ae500 con 0x7f45b00cb320 2023-12-26T22:52:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45add9b700 1 -- 172.21.15.131:0/2650169984 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b0156940 con 0x7f45b00cb320 2023-12-26T22:52:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45ae59c700 1 -- 172.21.15.131:0/2650169984 >> v1:172.21.15.190:6789/0 conn(0x7f45b00cb320 legacy=0x7f45b0069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45add9b700 1 -- 172.21.15.131:0/2650169984 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3650042247 0 0) 0x7f4598003570 con 0x7f45b00c9400 2023-12-26T22:52:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45ae59c700 1 -- 172.21.15.131:0/2650169984 >> v1:172.21.15.131:6789/0 conn(0x7f45b00c9400 legacy=0x7f45b00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45ae59c700 1 -- 172.21.15.131:0/2650169984 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b00b0280 con 0x7f45b00b0750 2023-12-26T22:52:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45ae59c700 1 --2- 172.21.15.131:0/2650169984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b00b0750 0x7f45b00b0bf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f45a40098e0 tx=0x7f45a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=150522ae343ba928 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45add9b700 1 -- 172.21.15.131:0/2650169984 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a80017f0 con 0x7f45b00b0750 2023-12-26T22:52:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45add9b700 1 -- 172.21.15.131:0/2650169984 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a4011420 con 0x7f45b00b0750 2023-12-26T22:52:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.435+0000 7f45add9b700 1 -- 172.21.15.131:0/2650169984 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a40117a0 con 0x7f45b00b0750 2023-12-26T22:52:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.436+0000 7f45b51ff700 1 -- 172.21.15.131:0/2650169984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b00b0750 msgr2=0x7f45b00b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.436+0000 7f45b51ff700 1 --2- 172.21.15.131:0/2650169984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b00b0750 0x7f45b00b0bf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f45a40098e0 tx=0x7f45a40091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.436+0000 7f45b51ff700 1 -- 172.21.15.131:0/2650169984 shutdown_connections 2023-12-26T22:52:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.436+0000 7f45b51ff700 1 --2- 172.21.15.131:0/2650169984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b00b0750 0x7f45b00b0bf0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.436+0000 7f45b51ff700 1 -- 172.21.15.131:0/2650169984 >> 172.21.15.131:0/2650169984 conn(0x7f45b0068040 msgr2=0x7f45b0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.436+0000 7f45b51ff700 1 -- 172.21.15.131:0/2650169984 shutdown_connections 2023-12-26T22:52:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.436+0000 7f45b51ff700 1 -- 172.21.15.131:0/2650169984 wait complete. 2023-12-26T22:52:35.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.438+0000 7f45b51ff700 1 Processor -- start 2023-12-26T22:52:35.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.438+0000 7f45b51ff700 1 -- start start 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45b51ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b00b0750 0x7f45b0147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45b51ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b00c9400 0x7f45b0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45b51ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b00cb320 0x7f45b0069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45b51ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0168200 con 0x7f45b00c9400 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45b51ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0167630 con 0x7f45b00b0750 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45b51ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01678d0 con 0x7f45b00cb320 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45ae59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b00c9400 0x7f45b0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45ae59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b00c9400 0x7f45b0069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57310/0 (socket says 172.21.15.131:57310) 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45ae59c700 1 -- 172.21.15.131:0/2550980998 learned_addr learned my addr 172.21.15.131:0/2550980998 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45aed9d700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b00b0750 0x7f45b0147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45af59e700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b00cb320 0x7f45b0069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45ae59c700 1 -- 172.21.15.131:0/2550980998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b00cb320 msgr2=0x7f45b0069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45ae59c700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b00cb320 0x7f45b0069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45ae59c700 1 -- 172.21.15.131:0/2550980998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b00b0750 msgr2=0x7f45b0147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45ae59c700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b00b0750 0x7f45b0147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45ae59c700 1 -- 172.21.15.131:0/2550980998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a4009420 con 0x7f45b00c9400 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.439+0000 7f45af59e700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b00cb320 0x7f45b0069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.440+0000 7f45ae59c700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b00c9400 0x7f45b0069590 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f45a40093f0 tx=0x7f45a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.440+0000 7f4596ffd700 1 -- 172.21.15.131:0/2550980998 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a401cc90 con 0x7f45b00c9400 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.440+0000 7f4596ffd700 1 -- 172.21.15.131:0/2550980998 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a401ce10 con 0x7f45b00c9400 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.440+0000 7f4596ffd700 1 -- 172.21.15.131:0/2550980998 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a401a6a0 con 0x7f45b00c9400 2023-12-26T22:52:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.440+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b0167bb0 con 0x7f45b00c9400 2023-12-26T22:52:35.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.440+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b0148050 con 0x7f45b00c9400 2023-12-26T22:52:35.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.441+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b0148240 con 0x7f45b00c9400 2023-12-26T22:52:35.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.442+0000 7f4596ffd700 1 -- 172.21.15.131:0/2550980998 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f45a401aa90 con 0x7f45b00c9400 2023-12-26T22:52:35.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.443+0000 7f4596ffd700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.443+0000 7f4596ffd700 1 -- 172.21.15.131:0/2550980998 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f45a4085ea0 con 0x7f45b00c9400 2023-12-26T22:52:35.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.443+0000 7f45aed9d700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.445 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.444+0000 7f45aed9d700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f459c066f30 0x7f459c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f45a00037e0 tx=0x7f45a0009040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:35.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.447+0000 7f4596ffd700 1 -- 172.21.15.131:0/2550980998 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45b0148240 con 0x7f45b00c9400 2023-12-26T22:52:35.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.713+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f45b0148240 con 0x7f45b00c9400 2023-12-26T22:52:35.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.713+0000 7f4596ffd700 1 -- 172.21.15.131:0/2550980998 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v40) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f45b0148240 con 0x7f45b00c9400 2023-12-26T22:52:35.713 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:35.713 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T22:52:35.713 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T22:52:35.713 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.713 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T22:52:35.713 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T22:52:35.713 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:52:35.714 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T22:52:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.717 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T22:52:35.718 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.719 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T22:52:35.720 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.721 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T22:52:35.723 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T22:52:35.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.725 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T22:52:35.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T22:52:35.727 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:52:35.727 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.727 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:35.727 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T22:52:35.727 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T22:52:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f459c066f30 msgr2=0x7f459c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f459c066f30 0x7f459c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f45a00037e0 tx=0x7f45a0009040 comp rx=0 tx=0).stop 2023-12-26T22:52:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b00c9400 msgr2=0x7f45b0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b00c9400 0x7f45b0069590 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f45a40093f0 tx=0x7f45a4016000 comp rx=0 tx=0).stop 2023-12-26T22:52:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 shutdown_connections 2023-12-26T22:52:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b00cb320 0x7f45b0069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b00b0750 0x7f45b0147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 --2- 172.21.15.131:0/2550980998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b00c9400 0x7f45b0069590 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 >> 172.21.15.131:0/2550980998 conn(0x7f45b0068040 msgr2=0x7f45b015bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:35.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 shutdown_connections 2023-12-26T22:52:35.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.716+0000 7f45b51ff700 1 -- 172.21.15.131:0/2550980998 wait complete. 2023-12-26T22:52:35.734 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:52:35.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.925+0000 7fb845fff700 1 Processor -- start 2023-12-26T22:52:35.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.925+0000 7fb845fff700 1 -- start start 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.926+0000 7fb845fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb840159600 0x7fb8400ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.926+0000 7fb845fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8400d01e0 0x7fb8400ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.926+0000 7fb845fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8400cfd50 con 0x7fb8401599d0 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.926+0000 7fb845fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8400b1280 con 0x7fb840159600 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.926+0000 7fb845fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400cf140 con 0x7fb8400d01e0 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.926+0000 7fb83f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb840159600 0x7fb8400ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.926+0000 7fb83f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb840159600 0x7fb8400ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57326/0 (socket says 172.21.15.131:57326) 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.926+0000 7fb83f7fe700 1 -- 172.21.15.131:0/1210573818 learned_addr learned my addr 172.21.15.131:0/1210573818 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.926+0000 7fb837fff700 1 --1- 172.21.15.131:0/1210573818 >> v1:172.21.15.131:6789/0 conn(0x7fb8401599d0 0x7fb84015dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58844/0 (socket says 172.21.15.131:58844) 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83ffff700 1 --2- 172.21.15.131:0/1210573818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8400d01e0 0x7fb8400ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83f7fe700 1 -- 172.21.15.131:0/1210573818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8400d01e0 msgr2=0x7fb8400ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83f7fe700 1 --2- 172.21.15.131:0/1210573818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8400d01e0 0x7fb8400ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83f7fe700 1 -- 172.21.15.131:0/1210573818 >> v1:172.21.15.131:6789/0 conn(0x7fb8401599d0 legacy=0x7fb84015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83f7fe700 1 -- 172.21.15.131:0/1210573818 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8400cf420 con 0x7fb840159600 2023-12-26T22:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83effd700 1 -- 172.21.15.131:0/1210573818 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238682121 0 0) 0x7fb8400cfd50 con 0x7fb8401599d0 2023-12-26T22:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83f7fe700 1 --2- 172.21.15.131:0/1210573818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840159600 0x7fb8400ceb60 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb830009990 tx=0x7fb830009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace2507e489e7cf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83effd700 1 -- 172.21.15.131:0/1210573818 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830009e80 con 0x7fb840159600 2023-12-26T22:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83effd700 1 -- 172.21.15.131:0/1210573818 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb830005c10 con 0x7fb840159600 2023-12-26T22:52:35.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.927+0000 7fb83effd700 1 -- 172.21.15.131:0/1210573818 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000b650 con 0x7fb840159600 2023-12-26T22:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.928+0000 7fb845fff700 1 -- 172.21.15.131:0/1210573818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840159600 msgr2=0x7fb8400ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.928+0000 7fb845fff700 1 --2- 172.21.15.131:0/1210573818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840159600 0x7fb8400ceb60 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb830009990 tx=0x7fb830009220 comp rx=0 tx=0).stop 2023-12-26T22:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.928+0000 7fb845fff700 1 -- 172.21.15.131:0/1210573818 shutdown_connections 2023-12-26T22:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.928+0000 7fb845fff700 1 --2- 172.21.15.131:0/1210573818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8400d01e0 0x7fb8400ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.928+0000 7fb845fff700 1 --2- 172.21.15.131:0/1210573818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840159600 0x7fb8400ceb60 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.928+0000 7fb845fff700 1 -- 172.21.15.131:0/1210573818 >> 172.21.15.131:0/1210573818 conn(0x7fb84006ece0 msgr2=0x7fb84006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:35.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.928+0000 7fb845fff700 1 -- 172.21.15.131:0/1210573818 shutdown_connections 2023-12-26T22:52:35.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.928+0000 7fb845fff700 1 -- 172.21.15.131:0/1210573818 wait complete. 2023-12-26T22:52:35.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.929+0000 7fb845fff700 1 Processor -- start 2023-12-26T22:52:35.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.930+0000 7fb845fff700 1 -- start start 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.930+0000 7fb845fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8400d01e0 0x7fb840154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.930+0000 7fb845fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb840159600 0x7fb84015bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.930+0000 7fb845fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840154590 0x7fb8400b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.930+0000 7fb845fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84015c380 con 0x7fb840154590 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.930+0000 7fb845fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84016d700 con 0x7fb8400d01e0 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.930+0000 7fb845fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84015c0b0 con 0x7fb840159600 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840154590 0x7fb8400b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840154590 0x7fb8400b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57342/0 (socket says 172.21.15.131:57342) 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83ffff700 1 -- 172.21.15.131:0/1077015147 learned_addr learned my addr 172.21.15.131:0/1077015147 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb837fff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb840159600 0x7fb84015bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83f7fe700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8400d01e0 0x7fb840154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83ffff700 1 -- 172.21.15.131:0/1077015147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb840159600 msgr2=0x7fb84015bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83ffff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb840159600 0x7fb84015bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83ffff700 1 -- 172.21.15.131:0/1077015147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8400d01e0 msgr2=0x7fb840154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83ffff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8400d01e0 0x7fb840154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83ffff700 1 -- 172.21.15.131:0/1077015147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb830009480 con 0x7fb840154590 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83f7fe700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8400d01e0 0x7fb840154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.931+0000 7fb83ffff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840154590 0x7fb8400b1ff0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb838009ec0 tx=0x7fb83800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.932+0000 7fb8377fe700 1 -- 172.21.15.131:0/1077015147 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800cc20 con 0x7fb840154590 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.932+0000 7fb8377fe700 1 -- 172.21.15.131:0/1077015147 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83800f040 con 0x7fb840154590 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.932+0000 7fb8377fe700 1 -- 172.21.15.131:0/1077015147 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838014480 con 0x7fb840154590 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.932+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84016cdd0 con 0x7fb840154590 2023-12-26T22:52:35.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.932+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84016d390 con 0x7fb840154590 2023-12-26T22:52:35.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.933+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84016d890 con 0x7fb840154590 2023-12-26T22:52:35.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.934+0000 7fb8377fe700 1 -- 172.21.15.131:0/1077015147 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb838014810 con 0x7fb840154590 2023-12-26T22:52:35.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.935+0000 7fb8377fe700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb820066f60 0x7fb820069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:35.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.935+0000 7fb8377fe700 1 -- 172.21.15.131:0/1077015147 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5317+0+0 (secure 0 0 0) 0x7fb838084de0 con 0x7fb840154590 2023-12-26T22:52:35.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.935+0000 7fb83f7fe700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb820066f60 0x7fb820069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:35.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.936+0000 7fb83f7fe700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb820066f60 0x7fb820069410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb830009260 tx=0x7fb830016000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:35.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:35.939+0000 7fb8377fe700 1 -- 172.21.15.131:0/1077015147 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8380521d0 con 0x7fb840154590 2023-12-26T22:52:36.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.132+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8400657e0 con 0x7fb840154590 2023-12-26T22:52:36.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.133+0000 7fb8377fe700 1 -- 172.21.15.131:0/1077015147 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+12904 (secure 0 0 0) 0x7fb838018090 con 0x7fb840154590 2023-12-26T22:52:36.133 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:36.133 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":157,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:52:35.214585+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:52:36.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb820066f60 msgr2=0x7fb820069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb820066f60 0x7fb820069410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb830009260 tx=0x7fb830016000 comp rx=0 tx=0).stop 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840154590 msgr2=0x7fb8400b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840154590 0x7fb8400b1ff0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb838009ec0 tx=0x7fb83800b040 comp rx=0 tx=0).stop 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 shutdown_connections 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb820066f60 0x7fb820069410 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb840154590 0x7fb8400b1ff0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8400d01e0 0x7fb840154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 --2- 172.21.15.131:0/1077015147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb840159600 0x7fb84015bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 >> 172.21.15.131:0/1077015147 conn(0x7fb84006ece0 msgr2=0x7fb840068580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 shutdown_connections 2023-12-26T22:52:36.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.136+0000 7fb845fff700 1 -- 172.21.15.131:0/1077015147 wait complete. 2023-12-26T22:52:36.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T22:52:36.154 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T22:52:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.348+0000 7fab547ff700 1 Processor -- start 2023-12-26T22:52:36.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.348+0000 7fab547ff700 1 -- start start 2023-12-26T22:52:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.349+0000 7fab547ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fab4c13c510 0x7fab4c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.349+0000 7fab547ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4c0686c0 con 0x7fab4c06b8c0 2023-12-26T22:52:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.349+0000 7fab547ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4c069520 con 0x7fab4c0cc810 2023-12-26T22:52:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.349+0000 7fab547ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4c068910 con 0x7fab4c13c510 2023-12-26T22:52:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.349+0000 7fab51d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fab4c06b8c0 0x7fab4c0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58858/0 (socket says 172.21.15.131:58858) 2023-12-26T22:52:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.349+0000 7fab51d9a700 1 -- 172.21.15.131:0/3633772990 learned_addr learned my addr 172.21.15.131:0/3633772990 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:36.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.349+0000 7fab5259b700 1 --2- 172.21.15.131:0/3633772990 >> v2:172.21.15.190:3300/0 conn(0x7fab4c13c510 0x7fab4c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.350+0000 7fab51599700 1 -- 172.21.15.131:0/3633772990 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002684430 0 0) 0x7fab4c0686c0 con 0x7fab4c06b8c0 2023-12-26T22:52:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.350+0000 7fab51599700 1 -- 172.21.15.131:0/3633772990 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab38003570 con 0x7fab4c06b8c0 2023-12-26T22:52:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.350+0000 7fab51599700 1 -- 172.21.15.131:0/3633772990 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607153010 0 0) 0x7fab4c069520 con 0x7fab4c0cc810 2023-12-26T22:52:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.350+0000 7fab51599700 1 -- 172.21.15.131:0/3633772990 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab4c0686c0 con 0x7fab4c0cc810 2023-12-26T22:52:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.350+0000 7fab5259b700 1 -- 172.21.15.131:0/3633772990 >> v1:172.21.15.190:6789/0 conn(0x7fab4c0cc810 legacy=0x7fab4c0699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.350+0000 7fab5259b700 1 -- 172.21.15.131:0/3633772990 >> v1:172.21.15.131:6789/0 conn(0x7fab4c06b8c0 legacy=0x7fab4c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.350+0000 7fab51599700 1 -- 172.21.15.131:0/3633772990 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1867829697 0 0) 0x7fab38003570 con 0x7fab4c06b8c0 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.350+0000 7fab5259b700 1 -- 172.21.15.131:0/3633772990 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c068bf0 con 0x7fab4c13c510 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.350+0000 7fab5259b700 1 --2- 172.21.15.131:0/3633772990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab4c13c510 0x7fab4c0c8a20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fab480098d0 tx=0x7fab48009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d5be955c5156c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab51599700 1 -- 172.21.15.131:0/3633772990 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48005c10 con 0x7fab4c13c510 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab51599700 1 -- 172.21.15.131:0/3633772990 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab48005d70 con 0x7fab4c13c510 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab51599700 1 -- 172.21.15.131:0/3633772990 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48006260 con 0x7fab4c13c510 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab547ff700 1 -- 172.21.15.131:0/3633772990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab4c13c510 msgr2=0x7fab4c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab547ff700 1 --2- 172.21.15.131:0/3633772990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab4c13c510 0x7fab4c0c8a20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fab480098d0 tx=0x7fab48009120 comp rx=0 tx=0).stop 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab547ff700 1 -- 172.21.15.131:0/3633772990 shutdown_connections 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab547ff700 1 --2- 172.21.15.131:0/3633772990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab4c13c510 0x7fab4c0c8a20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab547ff700 1 -- 172.21.15.131:0/3633772990 >> 172.21.15.131:0/3633772990 conn(0x7fab4c0ce3e0 msgr2=0x7fab4c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab547ff700 1 -- 172.21.15.131:0/3633772990 shutdown_connections 2023-12-26T22:52:36.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.351+0000 7fab547ff700 1 -- 172.21.15.131:0/3633772990 wait complete. 2023-12-26T22:52:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.353+0000 7fab547ff700 1 Processor -- start 2023-12-26T22:52:36.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.354+0000 7fab547ff700 1 -- start start 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab547ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab4c06b8c0 0x7fab4c13bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab547ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab4c0cc810 0x7fab4c159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab547ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab4c13c510 0x7fab4c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab547ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c15cb60 con 0x7fab4c06b8c0 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab547ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c0b1f50 con 0x7fab4c0cc810 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab547ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c0b2230 con 0x7fab4c13c510 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab5259b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab4c06b8c0 0x7fab4c13bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab5259b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab4c06b8c0 0x7fab4c13bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57350/0 (socket says 172.21.15.131:57350) 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab5259b700 1 -- 172.21.15.131:0/3579761407 learned_addr learned my addr 172.21.15.131:0/3579761407 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab52d9c700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab4c13c510 0x7fab4c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab51d9a700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab4c0cc810 0x7fab4c159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab5259b700 1 -- 172.21.15.131:0/3579761407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab4c13c510 msgr2=0x7fab4c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab5259b700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab4c13c510 0x7fab4c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab5259b700 1 -- 172.21.15.131:0/3579761407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab4c0cc810 msgr2=0x7fab4c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab5259b700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab4c0cc810 0x7fab4c159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.355+0000 7fab5259b700 1 -- 172.21.15.131:0/3579761407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480093c0 con 0x7fab4c06b8c0 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.356+0000 7fab5259b700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab4c06b8c0 0x7fab4c13bb70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fab48009160 tx=0x7fab48016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.356+0000 7fab427fc700 1 -- 172.21.15.131:0/3579761407 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4801d9e0 con 0x7fab4c06b8c0 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.356+0000 7fab427fc700 1 -- 172.21.15.131:0/3579761407 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4801db60 con 0x7fab4c06b8c0 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.356+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c0b2510 con 0x7fab4c06b8c0 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.356+0000 7fab427fc700 1 -- 172.21.15.131:0/3579761407 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4801b480 con 0x7fab4c06b8c0 2023-12-26T22:52:36.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.356+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4c13c0b0 con 0x7fab4c06b8c0 2023-12-26T22:52:36.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.357+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4c0b29b0 con 0x7fab4c06b8c0 2023-12-26T22:52:36.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.358+0000 7fab427fc700 1 -- 172.21.15.131:0/3579761407 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fab4801b7f0 con 0x7fab4c06b8c0 2023-12-26T22:52:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.359+0000 7fab427fc700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fab30066f30 0x7fab300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.359+0000 7fab427fc700 1 -- 172.21.15.131:0/3579761407 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5317+0+0 (secure 0 0 0) 0x7fab48086390 con 0x7fab4c06b8c0 2023-12-26T22:52:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.359+0000 7fab51d9a700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fab30066f30 0x7fab300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.360+0000 7fab51d9a700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fab30066f30 0x7fab300693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0b12a0 tx=0x7fab3c007bd0 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.363+0000 7fab427fc700 1 -- 172.21.15.131:0/3579761407 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab48053780 con 0x7fab4c06b8c0 2023-12-26T22:52:36.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.555+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab4c0657e0 con 0x7fab4c06b8c0 2023-12-26T22:52:36.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.556+0000 7fab427fc700 1 -- 172.21.15.131:0/3579761407 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab48056da0 con 0x7fab4c06b8c0 2023-12-26T22:52:36.556 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T22:52:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fab30066f30 msgr2=0x7fab300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fab30066f30 0x7fab300693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0b12a0 tx=0x7fab3c007bd0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab4c06b8c0 msgr2=0x7fab4c13bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab4c06b8c0 0x7fab4c13bb70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fab48009160 tx=0x7fab48016000 comp rx=0 tx=0).stop 2023-12-26T22:52:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 shutdown_connections 2023-12-26T22:52:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab4c13c510 0x7fab4c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fab30066f30 0x7fab300693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab4c06b8c0 0x7fab4c13bb70 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 --2- 172.21.15.131:0/3579761407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab4c0cc810 0x7fab4c159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.558+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 >> 172.21.15.131:0/3579761407 conn(0x7fab4c0ce3e0 msgr2=0x7fab4c152480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:36.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.559+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 shutdown_connections 2023-12-26T22:52:36.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.559+0000 7fab547ff700 1 -- 172.21.15.131:0/3579761407 wait complete. 2023-12-26T22:52:36.576 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T22:52:36.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.767+0000 7f93899ff700 1 Processor -- start 2023-12-26T22:52:36.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.767+0000 7f93899ff700 1 -- start start 2023-12-26T22:52:36.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.768+0000 7f93899ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f93841596a0 0x7f93840b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.768+0000 7f93899ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9384159a70 0x7f93840b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.768+0000 7f93899ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f93840b0160 0x7f93840b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.768+0000 7f93899ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93840afcd0 con 0x7f93841596a0 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.768+0000 7f93899ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93840b3e70 con 0x7f93840b0160 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.768+0000 7f93899ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93840b2e00 con 0x7f9384159a70 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.768+0000 7f9382ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f93841596a0 0x7f93840b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.768+0000 7f9382ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f93841596a0 0x7f93840b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57358/0 (socket says 172.21.15.131:57358) 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.768+0000 7f9382ffd700 1 -- 172.21.15.131:0/1604663957 learned_addr learned my addr 172.21.15.131:0/1604663957 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f93827fc700 1 --2- 172.21.15.131:0/1604663957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9384159a70 0x7f93840b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f93837fe700 1 --2- 172.21.15.131:0/1604663957 >> v2:172.21.15.190:3300/0 conn(0x7f93840b0160 0x7f93840b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f9382ffd700 1 -- 172.21.15.131:0/1604663957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9384159a70 msgr2=0x7f93840b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f9382ffd700 1 --2- 172.21.15.131:0/1604663957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9384159a70 0x7f93840b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f9382ffd700 1 -- 172.21.15.131:0/1604663957 >> v2:172.21.15.190:3300/0 conn(0x7f93840b0160 msgr2=0x7f93840b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f9382ffd700 1 --2- 172.21.15.131:0/1604663957 >> v2:172.21.15.190:3300/0 conn(0x7f93840b0160 0x7f93840b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f9382ffd700 1 -- 172.21.15.131:0/1604663957 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93840b0f20 con 0x7f93841596a0 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f93837fe700 1 --2- 172.21.15.131:0/1604663957 >> v2:172.21.15.190:3300/0 conn(0x7f93840b0160 0x7f93840b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f9382ffd700 1 --2- 172.21.15.131:0/1604663957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93841596a0 0x7f93840b2380 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f93840cdef0 tx=0x7f9374009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61343c6454024cdb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f9381ffb700 1 -- 172.21.15.131:0/1604663957 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374009e70 con 0x7f93841596a0 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f9381ffb700 1 -- 172.21.15.131:0/1604663957 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937400b860 con 0x7f93841596a0 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.769+0000 7f9381ffb700 1 -- 172.21.15.131:0/1604663957 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400bc00 con 0x7f93841596a0 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.770+0000 7f93899ff700 1 -- 172.21.15.131:0/1604663957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93841596a0 msgr2=0x7f93840b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.770+0000 7f93899ff700 1 --2- 172.21.15.131:0/1604663957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93841596a0 0x7f93840b2380 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f93840cdef0 tx=0x7f9374009250 comp rx=0 tx=0).stop 2023-12-26T22:52:36.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.770+0000 7f93899ff700 1 -- 172.21.15.131:0/1604663957 shutdown_connections 2023-12-26T22:52:36.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.770+0000 7f93899ff700 1 --2- 172.21.15.131:0/1604663957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93841596a0 0x7f93840b2380 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.770+0000 7f93899ff700 1 --2- 172.21.15.131:0/1604663957 >> v2:172.21.15.190:3300/0 conn(0x7f93840b0160 0x7f93840b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.770+0000 7f93899ff700 1 --2- 172.21.15.131:0/1604663957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9384159a70 0x7f93840b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.770+0000 7f93899ff700 1 -- 172.21.15.131:0/1604663957 >> 172.21.15.131:0/1604663957 conn(0x7f938406ece0 msgr2=0x7f938406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:36.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.770+0000 7f93899ff700 1 -- 172.21.15.131:0/1604663957 shutdown_connections 2023-12-26T22:52:36.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.770+0000 7f93899ff700 1 -- 172.21.15.131:0/1604663957 wait complete. 2023-12-26T22:52:36.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.771+0000 7f93899ff700 1 Processor -- start 2023-12-26T22:52:36.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.772+0000 7f93899ff700 1 -- start start 2023-12-26T22:52:36.776 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.772+0000 7f93899ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93840b0160 0x7f93840c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.772+0000 7f93899ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93841596a0 0x7f9384169900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.772+0000 7f93899ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9384159a70 0x7f93840c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.772+0000 7f93899ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384169e40 con 0x7f93840b0160 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.772+0000 7f93899ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93840c4910 con 0x7f93841596a0 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.772+0000 7f93899ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93840ca760 con 0x7f9384159a70 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f9382ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93840b0160 0x7f93840c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f9382ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93840b0160 0x7f93840c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57364/0 (socket says 172.21.15.131:57364) 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f9382ffd700 1 -- 172.21.15.131:0/3099450366 learned_addr learned my addr 172.21.15.131:0/3099450366 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f9382ffd700 1 -- 172.21.15.131:0/3099450366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9384159a70 msgr2=0x7f93840c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f9382ffd700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9384159a70 0x7f93840c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f9382ffd700 1 -- 172.21.15.131:0/3099450366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93841596a0 msgr2=0x7f9384169900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f9382ffd700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93841596a0 0x7f9384169900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f9382ffd700 1 -- 172.21.15.131:0/3099450366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93740094f0 con 0x7f93840b0160 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f9382ffd700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93840b0160 0x7f93840c78a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9374009290 tx=0x7f9374017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f936affd700 1 -- 172.21.15.131:0/3099450366 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400bbe0 con 0x7f93840b0160 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f936affd700 1 -- 172.21.15.131:0/3099450366 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937401f070 con 0x7f93840b0160 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.773+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93840ca480 con 0x7f93840b0160 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.774+0000 7f936affd700 1 -- 172.21.15.131:0/3099450366 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374026690 con 0x7f93840b0160 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.774+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93840c45a0 con 0x7f93840b0160 2023-12-26T22:52:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.775+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93840657e0 con 0x7f93840b0160 2023-12-26T22:52:36.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.776+0000 7f936affd700 1 -- 172.21.15.131:0/3099450366 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f937401a430 con 0x7f93840b0160 2023-12-26T22:52:36.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.777+0000 7f936affd700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9370066f30 0x7f93700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:36.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.777+0000 7f936affd700 1 -- 172.21.15.131:0/3099450366 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f9374021070 con 0x7f93840b0160 2023-12-26T22:52:36.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.777+0000 7f93827fc700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9370066f30 0x7f93700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:36.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.778+0000 7f93827fc700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9370066f30 0x7f93700693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f93840af3a0 tx=0x7f93780091a0 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:36.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.782+0000 7f936affd700 1 -- 172.21.15.131:0/3099450366 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93740b75a0 con 0x7f93840b0160 2023-12-26T22:52:36.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.975+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f93840b3540 con 0x7f93840b0160 2023-12-26T22:52:36.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.975+0000 7f936affd700 1 -- 172.21.15.131:0/3099450366 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v157) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9374015090 con 0x7f93840b0160 2023-12-26T22:52:36.975 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:36.975 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T22:52:36.975 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:36.975 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T22:52:36.975 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T22:52:36.976 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:36.976 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T22:52:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.977+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9370066f30 msgr2=0x7f93700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.977+0000 7f93899ff700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9370066f30 0x7f93700693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f93840af3a0 tx=0x7f93780091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.977+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93840b0160 msgr2=0x7f93840c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.977+0000 7f93899ff700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93840b0160 0x7f93840c78a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9374009290 tx=0x7f9374017000 comp rx=0 tx=0).stop 2023-12-26T22:52:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.978+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 shutdown_connections 2023-12-26T22:52:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.978+0000 7f93899ff700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9384159a70 0x7f93840c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.978+0000 7f93899ff700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9370066f30 0x7f93700693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.978+0000 7f93899ff700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93840b0160 0x7f93840c78a0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.978+0000 7f93899ff700 1 --2- 172.21.15.131:0/3099450366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93841596a0 0x7f9384169900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.978+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 >> 172.21.15.131:0/3099450366 conn(0x7f938406ece0 msgr2=0x7f93840cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.978+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 shutdown_connections 2023-12-26T22:52:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:36.978+0000 7f93899ff700 1 -- 172.21.15.131:0/3099450366 wait complete. 2023-12-26T22:52:36.995 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T22:52:36.995 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T22:52:37.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.189+0000 7fc7fb1ff700 1 Processor -- start 2023-12-26T22:52:37.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.189+0000 7fc7fb1ff700 1 -- start start 2023-12-26T22:52:37.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.190+0000 7fc7fb1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f406a570 0x7fc7f40abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:37.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.190+0000 7fc7fb1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f4067ff0 con 0x7fc7f41597c0 2023-12-26T22:52:37.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.190+0000 7fc7fb1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f40ac350 con 0x7fc7f41593f0 2023-12-26T22:52:37.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.190+0000 7fc7fb1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4068480 con 0x7fc7f406a570 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f3fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc7f41597c0 0x7fc7f4069e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58872/0 (socket says 172.21.15.131:58872) 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f3fff700 1 -- 172.21.15.131:0/328744230 learned_addr learned my addr 172.21.15.131:0/328744230 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f979c700 1 --2- 172.21.15.131:0/328744230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f406a570 0x7fc7f40abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f37fe700 1 -- 172.21.15.131:0/328744230 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927275167 0 0) 0x7fc7f4067ff0 con 0x7fc7f41597c0 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f37fe700 1 -- 172.21.15.131:0/328744230 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e4003570 con 0x7fc7f41597c0 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f37fe700 1 -- 172.21.15.131:0/328744230 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457479 0 0) 0x7fc7f40ac350 con 0x7fc7f41593f0 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f37fe700 1 -- 172.21.15.131:0/328744230 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f4067ff0 con 0x7fc7f41593f0 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f979c700 1 -- 172.21.15.131:0/328744230 >> v1:172.21.15.190:6789/0 conn(0x7fc7f41593f0 legacy=0x7fc7f40aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f979c700 1 -- 172.21.15.131:0/328744230 >> v1:172.21.15.131:6789/0 conn(0x7fc7f41597c0 legacy=0x7fc7f4069e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:37.193 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.191+0000 7fc7f979c700 1 -- 172.21.15.131:0/328744230 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40af8c0 con 0x7fc7f406a570 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7f979c700 1 --2- 172.21.15.131:0/328744230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f406a570 0x7fc7f40abc90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009e90 tx=0x7fc7ec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a1ebb2346bf441 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7f37fe700 1 -- 172.21.15.131:0/328744230 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00e070 con 0x7fc7f406a570 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7f37fe700 1 -- 172.21.15.131:0/328744230 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec00bb80 con 0x7fc7f406a570 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7f37fe700 1 -- 172.21.15.131:0/328744230 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec012430 con 0x7fc7f406a570 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/328744230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f406a570 msgr2=0x7fc7f40abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7fb1ff700 1 --2- 172.21.15.131:0/328744230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f406a570 0x7fc7f40abc90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009e90 tx=0x7fc7ec009780 comp rx=0 tx=0).stop 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/328744230 shutdown_connections 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7fb1ff700 1 --2- 172.21.15.131:0/328744230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f406a570 0x7fc7f40abc90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/328744230 >> 172.21.15.131:0/328744230 conn(0x7fc7f406b710 msgr2=0x7fc7f406bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/328744230 shutdown_connections 2023-12-26T22:52:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.192+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/328744230 wait complete. 2023-12-26T22:52:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.194+0000 7fc7fb1ff700 1 Processor -- start 2023-12-26T22:52:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.194+0000 7fc7fb1ff700 1 -- start start 2023-12-26T22:52:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7fb1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f406a570 0x7fc7f41541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7fb1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f41593f0 0x7fc7f416cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7fb1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f41597c0 0x7fc7f416d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7fb1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4155ab0 con 0x7fc7f406a570 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7fb1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4154a10 con 0x7fc7f41593f0 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7fb1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4154cf0 con 0x7fc7f41597c0 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7f8f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f406a570 0x7fc7f41541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7f8f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f406a570 0x7fc7f41541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57372/0 (socket says 172.21.15.131:57372) 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7f8f9b700 1 -- 172.21.15.131:0/2781584406 learned_addr learned my addr 172.21.15.131:0/2781584406 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7f8f9b700 1 -- 172.21.15.131:0/2781584406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f41597c0 msgr2=0x7fc7f416d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.195+0000 7fc7f8f9b700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f41597c0 0x7fc7f416d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.196+0000 7fc7f8f9b700 1 -- 172.21.15.131:0/2781584406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f41593f0 msgr2=0x7fc7f416cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.196+0000 7fc7f8f9b700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f41593f0 0x7fc7f416cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.196+0000 7fc7f8f9b700 1 -- 172.21.15.131:0/2781584406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec009a20 con 0x7fc7f406a570 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.196+0000 7fc7f8f9b700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f406a570 0x7fc7f41541c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800c790 tx=0x7fc7e8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.196+0000 7fc7f0ff9700 1 -- 172.21.15.131:0/2781584406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8010510 con 0x7fc7f406a570 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.196+0000 7fc7f0ff9700 1 -- 172.21.15.131:0/2781584406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e8010670 con 0x7fc7f406a570 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.196+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f4154fd0 con 0x7fc7f406a570 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.196+0000 7fc7f0ff9700 1 -- 172.21.15.131:0/2781584406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8010a10 con 0x7fc7f406a570 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.196+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f4155590 con 0x7fc7f406a570 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.197+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f40717f0 con 0x7fc7f406a570 2023-12-26T22:52:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.198+0000 7fc7f0ff9700 1 -- 172.21.15.131:0/2781584406 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc7e801b400 con 0x7fc7f406a570 2023-12-26T22:52:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.199+0000 7fc7f0ff9700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc7d8066f30 0x7fc7d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.199+0000 7fc7f0ff9700 1 -- 172.21.15.131:0/2781584406 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5317+0+0 (secure 0 0 0) 0x7fc7e80841a0 con 0x7fc7f406a570 2023-12-26T22:52:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.199+0000 7fc7f3fff700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc7d8066f30 0x7fc7d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.200+0000 7fc7f3fff700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc7d8066f30 0x7fc7d80693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00058d0 tx=0x7fc7e00079d0 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:37.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.203+0000 7fc7f0ff9700 1 -- 172.21.15.131:0/2781584406 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7e8051570 con 0x7fc7f406a570 2023-12-26T22:52:37.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.397+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc7f4063250 con 0x7fc7f406a570 2023-12-26T22:52:37.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.784+0000 7fc7f0ff9700 1 -- 172.21.15.131:0/2781584406 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v158) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc7e8054b90 con 0x7fc7f406a570 2023-12-26T22:52:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.788+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc7d8066f30 msgr2=0x7fc7d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.788+0000 7fc7fb1ff700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc7d8066f30 0x7fc7d80693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00058d0 tx=0x7fc7e00079d0 comp rx=0 tx=0).stop 2023-12-26T22:52:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.788+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f406a570 msgr2=0x7fc7f41541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.788+0000 7fc7fb1ff700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f406a570 0x7fc7f41541c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800c790 tx=0x7fc7e8007e10 comp rx=0 tx=0).stop 2023-12-26T22:52:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.789+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 shutdown_connections 2023-12-26T22:52:37.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.789+0000 7fc7fb1ff700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7f41597c0 0x7fc7f416d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:37.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.789+0000 7fc7fb1ff700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc7d8066f30 0x7fc7d80693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:37.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.789+0000 7fc7fb1ff700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f406a570 0x7fc7f41541c0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:37.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.789+0000 7fc7fb1ff700 1 --2- 172.21.15.131:0/2781584406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7f41593f0 0x7fc7f416cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:37.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.789+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 >> 172.21.15.131:0/2781584406 conn(0x7fc7f406b710 msgr2=0x7fc7f4156610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:37.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.789+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 shutdown_connections 2023-12-26T22:52:37.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:37.790+0000 7fc7fb1ff700 1 -- 172.21.15.131:0/2781584406 wait complete. 2023-12-26T22:52:37.791 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T22:52:37.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T22:52:38.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.008+0000 7f9244bff700 1 Processor -- start 2023-12-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.009+0000 7f9244bff700 1 -- start start 2023-12-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f9244bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9240157050 0x7f92400686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f9244bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9240068c00 0x7f92400cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f9244bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92400caed0 con 0x7f9240157420 2023-12-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f9244bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92400c9520 con 0x7f9240157050 2023-12-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f9244bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92400ca8e0 con 0x7f9240068c00 2023-12-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f923e59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9240157050 0x7f92400686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f923e59c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9240157050 0x7f92400686c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57376/0 (socket says 172.21.15.131:57376) 2023-12-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f923e59c700 1 -- 172.21.15.131:0/3025748078 learned_addr learned my addr 172.21.15.131:0/3025748078 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f923ed9d700 1 --2- 172.21.15.131:0/3025748078 >> v2:172.21.15.190:3300/0 conn(0x7f9240068c00 0x7f92400cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f923e59c700 1 -- 172.21.15.131:0/3025748078 >> v2:172.21.15.190:3300/0 conn(0x7f9240068c00 msgr2=0x7f92400cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f923e59c700 1 --2- 172.21.15.131:0/3025748078 >> v2:172.21.15.190:3300/0 conn(0x7f9240068c00 0x7f92400cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f923e59c700 1 -- 172.21.15.131:0/3025748078 >> v1:172.21.15.190:6789/0 conn(0x7f9240157420 legacy=0x7f9240159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.010+0000 7f923e59c700 1 -- 172.21.15.131:0/3025748078 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924015a6c0 con 0x7f9240157050 2023-12-26T22:52:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f923e59c700 1 --2- 172.21.15.131:0/3025748078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157050 0x7f92400686c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f9240069bb0 tx=0x7f9230009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db8203a8fb844aa server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f923dd9b700 1 -- 172.21.15.131:0/3025748078 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000b820 con 0x7f9240157050 2023-12-26T22:52:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f923dd9b700 1 -- 172.21.15.131:0/3025748078 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923000b980 con 0x7f9240157050 2023-12-26T22:52:38.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f923dd9b700 1 -- 172.21.15.131:0/3025748078 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000bd20 con 0x7f9240157050 2023-12-26T22:52:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f9244bff700 1 -- 172.21.15.131:0/3025748078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157050 msgr2=0x7f92400686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f9244bff700 1 --2- 172.21.15.131:0/3025748078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157050 0x7f92400686c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f9240069bb0 tx=0x7f9230009250 comp rx=0 tx=0).stop 2023-12-26T22:52:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f9244bff700 1 -- 172.21.15.131:0/3025748078 shutdown_connections 2023-12-26T22:52:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f9244bff700 1 --2- 172.21.15.131:0/3025748078 >> v2:172.21.15.190:3300/0 conn(0x7f9240068c00 0x7f92400cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f9244bff700 1 --2- 172.21.15.131:0/3025748078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157050 0x7f92400686c0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.011+0000 7f9244bff700 1 -- 172.21.15.131:0/3025748078 >> 172.21.15.131:0/3025748078 conn(0x7f924006dc80 msgr2=0x7f92400675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.012+0000 7f9244bff700 1 -- 172.21.15.131:0/3025748078 shutdown_connections 2023-12-26T22:52:38.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.012+0000 7f9244bff700 1 -- 172.21.15.131:0/3025748078 wait complete. 2023-12-26T22:52:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.013+0000 7f9244bff700 1 Processor -- start 2023-12-26T22:52:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.013+0000 7f9244bff700 1 -- start start 2023-12-26T22:52:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.014+0000 7f9244bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9240068c00 0x7f92400c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.014+0000 7f9244bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9240157050 0x7f92400b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.014+0000 7f9244bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157420 0x7f9240069030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.014+0000 7f9244bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92400b40f0 con 0x7f9240157420 2023-12-26T22:52:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.014+0000 7f9244bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92400b34e0 con 0x7f9240157050 2023-12-26T22:52:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.014+0000 7f9244bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92400b37c0 con 0x7f9240068c00 2023-12-26T22:52:38.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.014+0000 7f923ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157420 0x7f9240069030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.014+0000 7f923ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157420 0x7f9240069030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57380/0 (socket says 172.21.15.131:57380) 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.014+0000 7f923ed9d700 1 -- 172.21.15.131:0/721063253 learned_addr learned my addr 172.21.15.131:0/721063253 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f9237fff700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9240157050 0x7f92400b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f923e59c700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9240068c00 0x7f92400c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f923ed9d700 1 -- 172.21.15.131:0/721063253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9240068c00 msgr2=0x7f92400c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f923ed9d700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9240068c00 0x7f92400c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f923ed9d700 1 -- 172.21.15.131:0/721063253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9240157050 msgr2=0x7f92400b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f923ed9d700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9240157050 0x7f92400b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f923ed9d700 1 -- 172.21.15.131:0/721063253 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92300094f0 con 0x7f9240157420 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f9237fff700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9240157050 0x7f92400b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:38.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f923ed9d700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157420 0x7f9240069030 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f923800cd10 tx=0x7f9238007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f92367fc700 1 -- 172.21.15.131:0/721063253 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800aac0 con 0x7f9240157420 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f92367fc700 1 -- 172.21.15.131:0/721063253 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923800ac20 con 0x7f9240157420 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92400b3aa0 con 0x7f9240157420 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f92367fc700 1 -- 172.21.15.131:0/721063253 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238018480 con 0x7f9240157420 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.015+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9240069570 con 0x7f9240157420 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.016+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924016abf0 con 0x7f9240157420 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.017+0000 7f92367fc700 1 -- 172.21.15.131:0/721063253 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f923800d040 con 0x7f9240157420 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.018+0000 7f92367fc700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9220066f30 0x7f92200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.018+0000 7f92367fc700 1 -- 172.21.15.131:0/721063253 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f9238013070 con 0x7f9240157420 2023-12-26T22:52:38.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.018+0000 7f923e59c700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9220066f30 0x7f92200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:38.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.019+0000 7f923e59c700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9220066f30 0x7f92200693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9240069bb0 tx=0x7f9230016000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:38.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.023+0000 7f92367fc700 1 -- 172.21.15.131:0/721063253 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f924016abf0 con 0x7f9240157420 2023-12-26T22:52:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.218+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f924016abf0 con 0x7f9240157420 2023-12-26T22:52:38.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.789+0000 7f92367fc700 1 -- 172.21.15.131:0/721063253 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v159) v1 ==== 170+0+0 (secure 0 0 0) 0x7f924016abf0 con 0x7f9240157420 2023-12-26T22:52:38.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.795+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9220066f30 msgr2=0x7f92200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:38.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.795+0000 7f9244bff700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9220066f30 0x7f92200693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9240069bb0 tx=0x7f9230016000 comp rx=0 tx=0).stop 2023-12-26T22:52:38.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.795+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157420 msgr2=0x7f9240069030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:38.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.795+0000 7f9244bff700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157420 0x7f9240069030 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f923800cd10 tx=0x7f9238007770 comp rx=0 tx=0).stop 2023-12-26T22:52:38.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.798+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 shutdown_connections 2023-12-26T22:52:38.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.798+0000 7f9244bff700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9220066f30 0x7f92200693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:38.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.798+0000 7f9244bff700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9240157420 0x7f9240069030 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:38.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.798+0000 7f9244bff700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9240068c00 0x7f92400c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:38.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.798+0000 7f9244bff700 1 --2- 172.21.15.131:0/721063253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9240157050 0x7f92400b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:38.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.798+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 >> 172.21.15.131:0/721063253 conn(0x7f924006dc80 msgr2=0x7f92400cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:38.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.799+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 shutdown_connections 2023-12-26T22:52:38.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:38.799+0000 7f9244bff700 1 -- 172.21.15.131:0/721063253 wait complete. 2023-12-26T22:52:38.802 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T22:52:38.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T22:52:39.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.020+0000 7f2016dff700 1 Processor -- start 2023-12-26T22:52:39.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.020+0000 7f2016dff700 1 -- start start 2023-12-26T22:52:39.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.021+0000 7f2016dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f201006bfd0 0x7f20100c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.021+0000 7f2016dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20100cc100 0x7f20100cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.021+0000 7f2016dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20100686c0 con 0x7f201013c510 2023-12-26T22:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.021+0000 7f2016dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2010069520 con 0x7f20100cc100 2023-12-26T22:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.021+0000 7f2016dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2010068910 con 0x7f201006bfd0 2023-12-26T22:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.021+0000 7f2014b9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f201013c510 0x7f201006b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58886/0 (socket says 172.21.15.131:58886) 2023-12-26T22:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f2014b9b700 1 -- 172.21.15.131:0/1454294185 learned_addr learned my addr 172.21.15.131:0/1454294185 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f201539c700 1 --2- 172.21.15.131:0/1454294185 >> v2:172.21.15.190:3300/0 conn(0x7f20100cc100 0x7f20100cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200ffff700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f201006bfd0 0x7f20100c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:39.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986094631 0 0) 0x7f20100686c0 con 0x7f201013c510 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2000003570 con 0x7f201013c510 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2795814233 0 0) 0x7f2000003570 con 0x7f201013c510 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f201006bfd0 msgr2=0x7f20100c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f201006bfd0 0x7f20100c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 >> v2:172.21.15.190:3300/0 conn(0x7f20100cc100 msgr2=0x7f20100cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 --2- 172.21.15.131:0/1454294185 >> v2:172.21.15.190:3300/0 conn(0x7f20100cc100 0x7f20100cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010068bf0 con 0x7f201013c510 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f20040015f0 con 0x7f201013c510 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 >> v1:172.21.15.131:6789/0 conn(0x7f201013c510 legacy=0x7f201006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2000003a60 0x7f2000003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:39.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f2000004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f201006bfd0 0x7f20000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010069520 con 0x7f2000004810 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010068910 con 0x7f2000003a60 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.022+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200000ba60 con 0x7f201006bfd0 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f200ffff700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f2000004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f201539c700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f201006bfd0 0x7f20000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f2014b9b700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2000003a60 0x7f2000003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f200ffff700 1 -- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f201006bfd0 msgr2=0x7f20000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f200ffff700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f201006bfd0 0x7f20000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f200ffff700 1 -- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2000003a60 msgr2=0x7f2000003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:39.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f200ffff700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2000003a60 0x7f2000003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f200ffff700 1 -- 172.21.15.131:0/1454294185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000003570 con 0x7f2000004810 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f2014b9b700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2000003a60 0x7f2000003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.023+0000 7f200ffff700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f2000004c80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009c30 tx=0x7f1ffc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ada78918edbb8f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00e710 con 0x7f2000004810 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc00e870 con 0x7f2000004810 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f200f7fe700 1 -- 172.21.15.131:0/1454294185 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00ec30 con 0x7f2000004810 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f2016dff700 1 -- 172.21.15.131:0/1454294185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 msgr2=0x7f2000004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f2016dff700 1 --2- 172.21.15.131:0/1454294185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f2000004c80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009c30 tx=0x7f1ffc0091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f201539c700 1 -- 172.21.15.131:0/1454294185 reap_dead start 2023-12-26T22:52:39.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f2016dff700 1 -- 172.21.15.131:0/1454294185 shutdown_connections 2023-12-26T22:52:39.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f2016dff700 1 -- 172.21.15.131:0/1454294185 >> 172.21.15.131:0/1454294185 conn(0x7f20100ce3e0 msgr2=0x7f20100ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:39.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f2016dff700 1 -- 172.21.15.131:0/1454294185 shutdown_connections 2023-12-26T22:52:39.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.024+0000 7f2016dff700 1 -- 172.21.15.131:0/1454294185 wait complete. 2023-12-26T22:52:39.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.026+0000 7f2016dff700 1 Processor -- start 2023-12-26T22:52:39.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.026+0000 7f2016dff700 1 -- start start 2023-12-26T22:52:39.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2016dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f201013a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:39.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2016dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f201015a5c0 0x7f201015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2016dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20101588a0 0x7f2010158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2016dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010159280 con 0x7f2000004810 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2016dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201015b760 con 0x7f201015a5c0 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2016dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20100b25f0 con 0x7f20101588a0 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2014b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f201013a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2014b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f201013a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57396/0 (socket says 172.21.15.131:57396) 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2014b9b700 1 -- 172.21.15.131:0/4201492260 learned_addr learned my addr 172.21.15.131:0/4201492260 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f200ffff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f201015a5c0 0x7f201015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2014b9b700 1 -- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20101588a0 msgr2=0x7f2010158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f201539c700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20101588a0 0x7f2010158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:39.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2014b9b700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20101588a0 0x7f2010158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2014b9b700 1 -- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f201015a5c0 msgr2=0x7f201015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2014b9b700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f201015a5c0 0x7f201015aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f2014b9b700 1 -- 172.21.15.131:0/4201492260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100b14d0 con 0x7f2000004810 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f201539c700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20101588a0 0x7f2010158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.027+0000 7f200ffff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f201015a5c0 0x7f201015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.028+0000 7f2014b9b700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f201013a810 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f200400ebc0 tx=0x7f2004009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.028+0000 7f200cff9700 1 -- 172.21.15.131:0/4201492260 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20040072e0 con 0x7f2000004810 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.028+0000 7f200cff9700 1 -- 172.21.15.131:0/4201492260 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2004007460 con 0x7f2000004810 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.028+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201015af70 con 0x7f2000004810 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.028+0000 7f200cff9700 1 -- 172.21.15.131:0/4201492260 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20040057f0 con 0x7f2000004810 2023-12-26T22:52:39.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.028+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20100b2b70 con 0x7f2000004810 2023-12-26T22:52:39.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.029+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20100657e0 con 0x7f2000004810 2023-12-26T22:52:39.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.031+0000 7f200cff9700 1 -- 172.21.15.131:0/4201492260 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2004005b60 con 0x7f2000004810 2023-12-26T22:52:39.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.032+0000 7f200cff9700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1ff4067000 0x7f1ff40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:39.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.032+0000 7f200cff9700 1 -- 172.21.15.131:0/4201492260 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f20040852c0 con 0x7f2000004810 2023-12-26T22:52:39.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.032+0000 7f200ffff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1ff4067000 0x7f1ff40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:39.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.033+0000 7f200ffff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1ff4067000 0x7f1ff40694b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f201016cdb0 tx=0x7f1ffc01a000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:39.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.036+0000 7f200cff9700 1 -- 172.21.15.131:0/4201492260 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2004052420 con 0x7f2000004810 2023-12-26T22:52:39.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.269+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2010068ed0 con 0x7f2000004810 2023-12-26T22:52:39.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:52:39.553 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:52:39.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.821+0000 7f200cff9700 1 -- 172.21.15.131:0/4201492260 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2004055a40 con 0x7f2000004810 2023-12-26T22:52:39.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.830+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1ff4067000 msgr2=0x7f1ff40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:39.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.830+0000 7f2016dff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1ff4067000 0x7f1ff40694b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f201016cdb0 tx=0x7f1ffc01a000 comp rx=0 tx=0).stop 2023-12-26T22:52:39.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.830+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 msgr2=0x7f201013a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:39.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.830+0000 7f2016dff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f201013a810 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f200400ebc0 tx=0x7f2004009040 comp rx=0 tx=0).stop 2023-12-26T22:52:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.834+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 shutdown_connections 2023-12-26T22:52:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.834+0000 7f2016dff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20101588a0 0x7f2010158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.834+0000 7f2016dff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1ff4067000 0x7f1ff40694b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.834+0000 7f2016dff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2000004810 0x7f201013a810 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.834+0000 7f2016dff700 1 --2- 172.21.15.131:0/4201492260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f201015a5c0 0x7f201015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.834+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 >> 172.21.15.131:0/4201492260 conn(0x7f20100ce3e0 msgr2=0x7f20100c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.835+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 shutdown_connections 2023-12-26T22:52:39.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:39.835+0000 7f2016dff700 1 -- 172.21.15.131:0/4201492260 wait complete. 2023-12-26T22:52:39.839 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 18 and data pool 19 2023-12-26T22:52:39.859 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:52:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.048+0000 7f53533ff700 1 Processor -- start 2023-12-26T22:52:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.048+0000 7f53533ff700 1 -- start start 2023-12-26T22:52:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f53533ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f534c13c510 0x7f534c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f53533ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f534c06b8c0 0x7f534c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f53533ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c0cc100 0x7f534c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f53533ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534c06c280 con 0x7f534c06b8c0 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f53533ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534c0686c0 con 0x7f534c13c510 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f53533ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534c069520 con 0x7f534c0cc100 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f535099a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f534c06b8c0 0x7f534c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f535099a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f534c06b8c0 0x7f534c06bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57406/0 (socket says 172.21.15.131:57406) 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f535099a700 1 -- 172.21.15.131:0/1687034142 learned_addr learned my addr 172.21.15.131:0/1687034142 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.050+0000 7f535119b700 1 --2- 172.21.15.131:0/1687034142 >> v2:172.21.15.190:3300/0 conn(0x7f534c13c510 0x7f534c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f535099a700 1 -- 172.21.15.131:0/1687034142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c0cc100 msgr2=0x7f534c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f535199c700 1 --2- 172.21.15.131:0/1687034142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c0cc100 0x7f534c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f535099a700 1 --2- 172.21.15.131:0/1687034142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c0cc100 0x7f534c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f535099a700 1 -- 172.21.15.131:0/1687034142 >> v2:172.21.15.190:3300/0 conn(0x7f534c13c510 msgr2=0x7f534c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f535099a700 1 --2- 172.21.15.131:0/1687034142 >> v2:172.21.15.190:3300/0 conn(0x7f534c13c510 0x7f534c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f535099a700 1 -- 172.21.15.131:0/1687034142 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c068910 con 0x7f534c06b8c0 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f535119b700 1 --2- 172.21.15.131:0/1687034142 >> v2:172.21.15.190:3300/0 conn(0x7f534c13c510 0x7f534c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f535199c700 1 --2- 172.21.15.131:0/1687034142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c0cc100 0x7f534c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f535099a700 1 --2- 172.21.15.131:0/1687034142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c06b8c0 0x7f534c06bd40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f534c06fcc0 tx=0x7f533c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc7f6a6db072c9c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f5343fff700 1 -- 172.21.15.131:0/1687034142 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c00b860 con 0x7f534c06b8c0 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f5343fff700 1 -- 172.21.15.131:0/1687034142 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c00b9c0 con 0x7f534c06b8c0 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.051+0000 7f5343fff700 1 -- 172.21.15.131:0/1687034142 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c00bd60 con 0x7f534c06b8c0 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.052+0000 7f53533ff700 1 -- 172.21.15.131:0/1687034142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c06b8c0 msgr2=0x7f534c06bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.052+0000 7f53533ff700 1 --2- 172.21.15.131:0/1687034142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c06b8c0 0x7f534c06bd40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f534c06fcc0 tx=0x7f533c009250 comp rx=0 tx=0).stop 2023-12-26T22:52:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.052+0000 7f53533ff700 1 -- 172.21.15.131:0/1687034142 shutdown_connections 2023-12-26T22:52:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.052+0000 7f53533ff700 1 --2- 172.21.15.131:0/1687034142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c0cc100 0x7f534c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.052+0000 7f53533ff700 1 --2- 172.21.15.131:0/1687034142 >> v2:172.21.15.190:3300/0 conn(0x7f534c13c510 0x7f534c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.052+0000 7f53533ff700 1 --2- 172.21.15.131:0/1687034142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c06b8c0 0x7f534c06bd40 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.052+0000 7f53533ff700 1 -- 172.21.15.131:0/1687034142 >> 172.21.15.131:0/1687034142 conn(0x7f534c0ce3e0 msgr2=0x7f534c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.052+0000 7f53533ff700 1 -- 172.21.15.131:0/1687034142 shutdown_connections 2023-12-26T22:52:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.052+0000 7f53533ff700 1 -- 172.21.15.131:0/1687034142 wait complete. 2023-12-26T22:52:40.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.053+0000 7f53533ff700 1 Processor -- start 2023-12-26T22:52:40.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.053+0000 7f53533ff700 1 -- start start 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.054+0000 7f53533ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c06b8c0 0x7f534c13caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.054+0000 7f53533ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f534c0cc100 0x7f534c13b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.054+0000 7f53533ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c13c510 0x7f534c15b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.054+0000 7f53533ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c13b5b0 con 0x7f534c13c510 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.054+0000 7f53533ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c16dac0 con 0x7f534c0cc100 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.054+0000 7f53533ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534c16d190 con 0x7f534c06b8c0 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535199c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c13c510 0x7f534c15b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535199c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c13c510 0x7f534c15b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57418/0 (socket says 172.21.15.131:57418) 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535099a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f534c0cc100 0x7f534c13b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535199c700 1 -- 172.21.15.131:0/1807014995 learned_addr learned my addr 172.21.15.131:0/1807014995 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535119b700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c06b8c0 0x7f534c13caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535199c700 1 -- 172.21.15.131:0/1807014995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c06b8c0 msgr2=0x7f534c13caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535199c700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c06b8c0 0x7f534c13caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535199c700 1 -- 172.21.15.131:0/1807014995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f534c0cc100 msgr2=0x7f534c13b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535199c700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f534c0cc100 0x7f534c13b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535199c700 1 -- 172.21.15.131:0/1807014995 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c0094f0 con 0x7f534c13c510 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f535199c700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c13c510 0x7f534c15b400 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f534400a380 tx=0x7f5344007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.055+0000 7f53417fa700 1 -- 172.21.15.131:0/1807014995 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344015440 con 0x7f534c13c510 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.056+0000 7f53417fa700 1 -- 172.21.15.131:0/1807014995 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53440155a0 con 0x7f534c13c510 2023-12-26T22:52:40.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.056+0000 7f53417fa700 1 -- 172.21.15.131:0/1807014995 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344015960 con 0x7f534c13c510 2023-12-26T22:52:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.056+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c16d470 con 0x7f534c13c510 2023-12-26T22:52:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.056+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c15b940 con 0x7f534c13c510 2023-12-26T22:52:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.057+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534c16dc30 con 0x7f534c13c510 2023-12-26T22:52:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.058+0000 7f53417fa700 1 -- 172.21.15.131:0/1807014995 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5344015d50 con 0x7f534c13c510 2023-12-26T22:52:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.058+0000 7f53417fa700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5330066f30 0x7f53300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.059+0000 7f53417fa700 1 -- 172.21.15.131:0/1807014995 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f5344084f90 con 0x7f534c13c510 2023-12-26T22:52:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.059+0000 7f535119b700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5330066f30 0x7f53300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.059+0000 7f535119b700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5330066f30 0x7f53300693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f534800ad40 tx=0x7f5348008040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:40.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.062+0000 7f53417fa700 1 -- 172.21.15.131:0/1807014995 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5344052030 con 0x7f534c13c510 2023-12-26T22:52:40.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.260+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f534c0657e0 con 0x7f534c13c510 2023-12-26T22:52:40.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.261+0000 7f53417fa700 1 -- 172.21.15.131:0/1807014995 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f534401a090 con 0x7f534c13c510 2023-12-26T22:52:40.261 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:40.261 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":160,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:52:39.271545+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:52:37.440528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-26T22:52:38.259993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:52:40.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5330066f30 msgr2=0x7f53300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5330066f30 0x7f53300693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f534800ad40 tx=0x7f5348008040 comp rx=0 tx=0).stop 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c13c510 msgr2=0x7f534c15b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c13c510 0x7f534c15b400 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f534400a380 tx=0x7f5344007900 comp rx=0 tx=0).stop 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 shutdown_connections 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5330066f30 0x7f53300693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f534c13c510 0x7f534c15b400 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f534c06b8c0 0x7f534c13caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 --2- 172.21.15.131:0/1807014995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f534c0cc100 0x7f534c13b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.263+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 >> 172.21.15.131:0/1807014995 conn(0x7f534c0ce3e0 msgr2=0x7f534c0cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.264+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 shutdown_connections 2023-12-26T22:52:40.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.264+0000 7f53533ff700 1 -- 172.21.15.131:0/1807014995 wait complete. 2023-12-26T22:52:40.282 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:52:40.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.471+0000 7f34223ff700 1 Processor -- start 2023-12-26T22:52:40.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.472+0000 7f34223ff700 1 -- start start 2023-12-26T22:52:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f34223ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f341c159600 0x7f341c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f34223ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f341c0d01e0 0x7f341c0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f34223ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c0cfd50 con 0x7f341c1599d0 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f34223ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c0b1280 con 0x7f341c0d01e0 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f34223ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c0cf140 con 0x7f341c159600 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f342099c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f341c0d01e0 0x7f341c0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f342099c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f341c0d01e0 0x7f341c0ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57426/0 (socket says 172.21.15.131:57426) 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f342099c700 1 -- 172.21.15.131:0/2072721794 learned_addr learned my addr 172.21.15.131:0/2072721794 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f341bfff700 1 --2- 172.21.15.131:0/2072721794 >> v2:172.21.15.190:3300/0 conn(0x7f341c159600 0x7f341c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.473+0000 7f342099c700 1 -- 172.21.15.131:0/2072721794 >> v2:172.21.15.190:3300/0 conn(0x7f341c159600 msgr2=0x7f341c0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.474+0000 7f342099c700 1 --2- 172.21.15.131:0/2072721794 >> v2:172.21.15.190:3300/0 conn(0x7f341c159600 0x7f341c0ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.474+0000 7f342099c700 1 -- 172.21.15.131:0/2072721794 >> v1:172.21.15.190:6789/0 conn(0x7f341c1599d0 legacy=0x7f341c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.474+0000 7f342099c700 1 -- 172.21.15.131:0/2072721794 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c0cf420 con 0x7f341c0d01e0 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.474+0000 7f342099c700 1 --2- 172.21.15.131:0/2072721794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c0d01e0 0x7f341c0ccf00 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f341c089850 tx=0x7f3414009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6dd5469fe68095 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.474+0000 7f341affd700 1 -- 172.21.15.131:0/2072721794 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341400e070 con 0x7f341c0d01e0 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.474+0000 7f341affd700 1 -- 172.21.15.131:0/2072721794 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341400ba20 con 0x7f341c0d01e0 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.474+0000 7f341affd700 1 -- 172.21.15.131:0/2072721794 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140092e0 con 0x7f341c0d01e0 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.475+0000 7f34223ff700 1 -- 172.21.15.131:0/2072721794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c0d01e0 msgr2=0x7f341c0ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.475+0000 7f34223ff700 1 --2- 172.21.15.131:0/2072721794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c0d01e0 0x7f341c0ccf00 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f341c089850 tx=0x7f3414009840 comp rx=0 tx=0).stop 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.475+0000 7f34223ff700 1 -- 172.21.15.131:0/2072721794 shutdown_connections 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.475+0000 7f34223ff700 1 --2- 172.21.15.131:0/2072721794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c0d01e0 0x7f341c0ccf00 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.475+0000 7f34223ff700 1 --2- 172.21.15.131:0/2072721794 >> v2:172.21.15.190:3300/0 conn(0x7f341c159600 0x7f341c0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.475+0000 7f34223ff700 1 -- 172.21.15.131:0/2072721794 >> 172.21.15.131:0/2072721794 conn(0x7f341c06ece0 msgr2=0x7f341c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:40.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.475+0000 7f34223ff700 1 -- 172.21.15.131:0/2072721794 shutdown_connections 2023-12-26T22:52:40.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.475+0000 7f34223ff700 1 -- 172.21.15.131:0/2072721794 wait complete. 2023-12-26T22:52:40.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.476+0000 7f34223ff700 1 Processor -- start 2023-12-26T22:52:40.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.476+0000 7f34223ff700 1 -- start start 2023-12-26T22:52:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.477+0000 7f34223ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f341c0d01e0 0x7f341c0af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.477+0000 7f34223ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c159600 0x7f341c0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.477+0000 7f34223ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f341c0c85f0 0x7f341c16c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.477+0000 7f34223ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c16d630 con 0x7f341c159600 2023-12-26T22:52:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.477+0000 7f34223ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c16a3a0 con 0x7f341c0c85f0 2023-12-26T22:52:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.477+0000 7f34223ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c8a40 con 0x7f341c0d01e0 2023-12-26T22:52:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.477+0000 7f341b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c159600 0x7f341c0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.477+0000 7f341b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c159600 0x7f341c0c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57428/0 (socket says 172.21.15.131:57428) 2023-12-26T22:52:40.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.477+0000 7f341b7fe700 1 -- 172.21.15.131:0/1426742492 learned_addr learned my addr 172.21.15.131:0/1426742492 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f341bfff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f341c0d01e0 0x7f341c0af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f342099c700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f341c0c85f0 0x7f341c16c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f341b7fe700 1 -- 172.21.15.131:0/1426742492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f341c0d01e0 msgr2=0x7f341c0af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f341b7fe700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f341c0d01e0 0x7f341c0af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f341b7fe700 1 -- 172.21.15.131:0/1426742492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f341c0c85f0 msgr2=0x7f341c16c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f341b7fe700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f341c0c85f0 0x7f341c16c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f341b7fe700 1 -- 172.21.15.131:0/1426742492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3414009ae0 con 0x7f341c159600 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f341b7fe700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c159600 0x7f341c0c80b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f341000c510 tx=0x7f341000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f3403fff700 1 -- 172.21.15.131:0/1426742492 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410011590 con 0x7f341c159600 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f3403fff700 1 -- 172.21.15.131:0/1426742492 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410011710 con 0x7f341c159600 2023-12-26T22:52:40.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c0afc90 con 0x7f341c159600 2023-12-26T22:52:40.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.478+0000 7f3403fff700 1 -- 172.21.15.131:0/1426742492 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410011ad0 con 0x7f341c159600 2023-12-26T22:52:40.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.479+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c0c8f70 con 0x7f341c159600 2023-12-26T22:52:40.482 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.479+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c0657e0 con 0x7f341c159600 2023-12-26T22:52:40.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.480+0000 7f3403fff700 1 -- 172.21.15.131:0/1426742492 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f341001b3b0 con 0x7f341c159600 2023-12-26T22:52:40.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.481+0000 7f3403fff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3408066f30 0x7f34080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.481+0000 7f3403fff700 1 -- 172.21.15.131:0/1426742492 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f3410083b10 con 0x7f341c159600 2023-12-26T22:52:40.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.482+0000 7f341bfff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3408066f30 0x7f34080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.483 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.482+0000 7f341bfff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3408066f30 0x7f34080693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f340c009da0 tx=0x7f340c009120 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:40.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.486+0000 7f3403fff700 1 -- 172.21.15.131:0/1426742492 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3410050b10 con 0x7f341c159600 2023-12-26T22:52:40.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.679+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f341c0cf700 con 0x7f341c159600 2023-12-26T22:52:40.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.680+0000 7f3403fff700 1 -- 172.21.15.131:0/1426742492 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f3410017070 con 0x7f341c159600 2023-12-26T22:52:40.680 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:40.680 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":160,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:52:39.271545+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:52:37.440528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-26T22:52:38.259993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3408066f30 msgr2=0x7f34080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3408066f30 0x7f34080693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f340c009da0 tx=0x7f340c009120 comp rx=0 tx=0).stop 2023-12-26T22:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c159600 msgr2=0x7f341c0c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c159600 0x7f341c0c80b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f341000c510 tx=0x7f341000c820 comp rx=0 tx=0).stop 2023-12-26T22:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 shutdown_connections 2023-12-26T22:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f341c0c85f0 0x7f341c16c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f341c0d01e0 0x7f341c0af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3408066f30 0x7f34080693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 --2- 172.21.15.131:0/1426742492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f341c159600 0x7f341c0c80b0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 >> 172.21.15.131:0/1426742492 conn(0x7f341c06ece0 msgr2=0x7f341c158040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 shutdown_connections 2023-12-26T22:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.682+0000 7f34223ff700 1 -- 172.21.15.131:0/1426742492 wait complete. 2023-12-26T22:52:40.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T22:52:40.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.898+0000 7facacfff700 1 Processor -- start 2023-12-26T22:52:40.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.898+0000 7facacfff700 1 -- start start 2023-12-26T22:52:40.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.899+0000 7facacfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faca806bfd0 0x7faca80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.899+0000 7facacfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca80cc100 0x7faca80cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.899+0000 7facacfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca80686c0 con 0x7faca813c510 2023-12-26T22:52:40.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.899+0000 7facacfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca8069520 con 0x7faca806bfd0 2023-12-26T22:52:40.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.899+0000 7facacfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca8068910 con 0x7faca80cc100 2023-12-26T22:52:40.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.900+0000 7faca5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faca806bfd0 0x7faca80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.900+0000 7faca5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faca806bfd0 0x7faca80c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:52930/0 (socket says 172.21.15.131:52930) 2023-12-26T22:52:40.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.900+0000 7faca5d9b700 1 -- 172.21.15.131:0/1023072931 learned_addr learned my addr 172.21.15.131:0/1023072931 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:40.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.900+0000 7faca6d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca80cc100 0x7faca80cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.900+0000 7faca5d9b700 1 -- 172.21.15.131:0/1023072931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca80cc100 msgr2=0x7faca80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:40.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.900+0000 7faca5d9b700 1 --2- 172.21.15.131:0/1023072931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca80cc100 0x7faca80cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.900+0000 7faca5d9b700 1 -- 172.21.15.131:0/1023072931 >> v1:172.21.15.190:6789/0 conn(0x7faca813c510 legacy=0x7faca806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.900+0000 7faca5d9b700 1 -- 172.21.15.131:0/1023072931 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8068bf0 con 0x7faca806bfd0 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.901+0000 7faca5d9b700 1 --2- 172.21.15.131:0/1023072931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca806bfd0 0x7faca80c8a20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0099b0 tx=0x7fac9c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=488f5c9755d359ac server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.901+0000 7faca559a700 1 -- 172.21.15.131:0/1023072931 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c009ea0 con 0x7faca806bfd0 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.901+0000 7faca559a700 1 -- 172.21.15.131:0/1023072931 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c00b860 con 0x7faca806bfd0 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.901+0000 7faca559a700 1 -- 172.21.15.131:0/1023072931 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00bc00 con 0x7faca806bfd0 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.902+0000 7facacfff700 1 -- 172.21.15.131:0/1023072931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca806bfd0 msgr2=0x7faca80c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.902+0000 7facacfff700 1 --2- 172.21.15.131:0/1023072931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca806bfd0 0x7faca80c8a20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0099b0 tx=0x7fac9c009250 comp rx=0 tx=0).stop 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.902+0000 7facacfff700 1 -- 172.21.15.131:0/1023072931 shutdown_connections 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.902+0000 7facacfff700 1 --2- 172.21.15.131:0/1023072931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca80cc100 0x7faca80cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.902+0000 7facacfff700 1 --2- 172.21.15.131:0/1023072931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca806bfd0 0x7faca80c8a20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.902+0000 7facacfff700 1 -- 172.21.15.131:0/1023072931 >> 172.21.15.131:0/1023072931 conn(0x7faca80ce3e0 msgr2=0x7faca80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:40.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.902+0000 7facacfff700 1 -- 172.21.15.131:0/1023072931 shutdown_connections 2023-12-26T22:52:40.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.902+0000 7facacfff700 1 -- 172.21.15.131:0/1023072931 wait complete. 2023-12-26T22:52:40.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.903+0000 7facacfff700 1 Processor -- start 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.903+0000 7facacfff700 1 -- start start 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.904+0000 7facacfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faca806bfd0 0x7faca80c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.904+0000 7facacfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca80cc100 0x7faca80c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.904+0000 7facacfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca813c510 0x7faca80c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.904+0000 7facacfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca80b1180 con 0x7faca806bfd0 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.904+0000 7facacfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca80b1d00 con 0x7faca80cc100 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.904+0000 7facacfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca80b13d0 con 0x7faca813c510 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faca806bfd0 0x7faca80c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca5d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca80cc100 0x7faca80c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faca806bfd0 0x7faca80c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57436/0 (socket says 172.21.15.131:57436) 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca659c700 1 -- 172.21.15.131:0/2316699967 learned_addr learned my addr 172.21.15.131:0/2316699967 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca6d9d700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca813c510 0x7faca80c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca659c700 1 -- 172.21.15.131:0/2316699967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca813c510 msgr2=0x7faca80c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca659c700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca813c510 0x7faca80c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca659c700 1 -- 172.21.15.131:0/2316699967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca80cc100 msgr2=0x7faca80c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca659c700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca80cc100 0x7faca80c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca659c700 1 -- 172.21.15.131:0/2316699967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c0094f0 con 0x7faca806bfd0 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca5d9b700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca80cc100 0x7faca80c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.905+0000 7faca659c700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faca806bfd0 0x7faca80c5a10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c7c0 tx=0x7fac98007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:40.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.906+0000 7fac967fc700 1 -- 172.21.15.131:0/2316699967 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98011590 con 0x7faca806bfd0 2023-12-26T22:52:40.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.906+0000 7fac967fc700 1 -- 172.21.15.131:0/2316699967 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98011710 con 0x7faca806bfd0 2023-12-26T22:52:40.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.906+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca80b16b0 con 0x7faca806bfd0 2023-12-26T22:52:40.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.906+0000 7fac967fc700 1 -- 172.21.15.131:0/2316699967 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98011ad0 con 0x7faca806bfd0 2023-12-26T22:52:40.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.906+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca80c5f50 con 0x7faca806bfd0 2023-12-26T22:52:40.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.907+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca80b1fe0 con 0x7faca806bfd0 2023-12-26T22:52:40.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.908+0000 7fac967fc700 1 -- 172.21.15.131:0/2316699967 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fac9801b3b0 con 0x7faca806bfd0 2023-12-26T22:52:40.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.908+0000 7fac967fc700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fac88066f30 0x7fac880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:40.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.909+0000 7fac967fc700 1 -- 172.21.15.131:0/2316699967 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fac98083a60 con 0x7faca806bfd0 2023-12-26T22:52:40.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.909+0000 7faca5d9b700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fac88066f30 0x7fac880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:40.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.909+0000 7faca5d9b700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fac88066f30 0x7fac880693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0099b0 tx=0x7fac9c017000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:40.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:40.912+0000 7fac967fc700 1 -- 172.21.15.131:0/2316699967 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac98050bb0 con 0x7faca806bfd0 2023-12-26T22:52:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.153+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faca80657e0 con 0x7faca806bfd0 2023-12-26T22:52:41.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.825+0000 7fac967fc700 1 -- 172.21.15.131:0/2316699967 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) v1 ==== 118+0+0 (secure 0 0 0) 0x7fac98017070 con 0x7faca806bfd0 2023-12-26T22:52:41.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fac88066f30 msgr2=0x7fac880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:41.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fac88066f30 0x7fac880693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0099b0 tx=0x7fac9c017000 comp rx=0 tx=0).stop 2023-12-26T22:52:41.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faca806bfd0 msgr2=0x7faca80c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:41.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faca806bfd0 0x7faca80c5a10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c7c0 tx=0x7fac98007f70 comp rx=0 tx=0).stop 2023-12-26T22:52:41.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 shutdown_connections 2023-12-26T22:52:41.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faca813c510 0x7faca80c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:41.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fac88066f30 0x7fac880693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:41.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faca806bfd0 0x7faca80c5a10 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:41.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 --2- 172.21.15.131:0/2316699967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faca80cc100 0x7faca80c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:41.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 >> 172.21.15.131:0/2316699967 conn(0x7faca80ce3e0 msgr2=0x7faca80c4380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:41.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 shutdown_connections 2023-12-26T22:52:41.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:41.828+0000 7facacfff700 1 -- 172.21.15.131:0/2316699967 wait complete. 2023-12-26T22:52:41.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T22:52:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.046+0000 7f026dfff700 1 Processor -- start 2023-12-26T22:52:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.047+0000 7f026dfff700 1 -- start start 2023-12-26T22:52:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.048+0000 7f026dfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f02680c8820 0x7f02680ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.048+0000 7f026dfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f02680b19f0 0x7f0268161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.048+0000 7f026dfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0268157a80 con 0x7f02680c8bf0 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.048+0000 7f026dfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680cac50 con 0x7f02680c8820 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.048+0000 7f026dfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680caf30 con 0x7f02680b19f0 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.048+0000 7f02677fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f02680c8820 0x7f02680ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.048+0000 7f02677fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f02680c8820 0x7f02680ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57444/0 (socket says 172.21.15.131:57444) 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.048+0000 7f02677fe700 1 -- 172.21.15.131:0/138729802 learned_addr learned my addr 172.21.15.131:0/138729802 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.048+0000 7f0267fff700 1 --2- 172.21.15.131:0/138729802 >> v2:172.21.15.190:3300/0 conn(0x7f02680b19f0 0x7f0268161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.049+0000 7f02677fe700 1 -- 172.21.15.131:0/138729802 >> v2:172.21.15.190:3300/0 conn(0x7f02680b19f0 msgr2=0x7f0268161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.049+0000 7f02677fe700 1 --2- 172.21.15.131:0/138729802 >> v2:172.21.15.190:3300/0 conn(0x7f02680b19f0 0x7f0268161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.049+0000 7f02677fe700 1 -- 172.21.15.131:0/138729802 >> v1:172.21.15.190:6789/0 conn(0x7f02680c8bf0 legacy=0x7f02680b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:42.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.049+0000 7f02677fe700 1 -- 172.21.15.131:0/138729802 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680cb210 con 0x7f02680c8820 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.049+0000 7f02677fe700 1 --2- 172.21.15.131:0/138729802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680c8820 0x7f02680ca300 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0268070f80 tx=0x7f0258009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e41bee363aff8f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.049+0000 7f02667fc700 1 -- 172.21.15.131:0/138729802 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025800b820 con 0x7f02680c8820 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.049+0000 7f02667fc700 1 -- 172.21.15.131:0/138729802 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025800b980 con 0x7f02680c8820 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.049+0000 7f02667fc700 1 -- 172.21.15.131:0/138729802 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025800bd20 con 0x7f02680c8820 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.050+0000 7f026dfff700 1 -- 172.21.15.131:0/138729802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680c8820 msgr2=0x7f02680ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.050+0000 7f026dfff700 1 --2- 172.21.15.131:0/138729802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680c8820 0x7f02680ca300 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0268070f80 tx=0x7f0258009250 comp rx=0 tx=0).stop 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.050+0000 7f026dfff700 1 -- 172.21.15.131:0/138729802 shutdown_connections 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.050+0000 7f026dfff700 1 --2- 172.21.15.131:0/138729802 >> v2:172.21.15.190:3300/0 conn(0x7f02680b19f0 0x7f0268161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.050+0000 7f026dfff700 1 --2- 172.21.15.131:0/138729802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680c8820 0x7f02680ca300 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.050+0000 7f026dfff700 1 -- 172.21.15.131:0/138729802 >> 172.21.15.131:0/138729802 conn(0x7f0268068040 msgr2=0x7f0268069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.050+0000 7f026dfff700 1 -- 172.21.15.131:0/138729802 shutdown_connections 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.050+0000 7f026dfff700 1 -- 172.21.15.131:0/138729802 wait complete. 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.051+0000 7f026dfff700 1 Processor -- start 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.052+0000 7f026dfff700 1 -- start start 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.052+0000 7f026dfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680b19f0 0x7f0268153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.052+0000 7f026dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02680c8820 0x7f02681547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.052+0000 7f026dfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02680c8bf0 0x7f02681586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.052+0000 7f026dfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0268150510 con 0x7f02680b19f0 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.052+0000 7f026dfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0268154360 con 0x7f02680c8bf0 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.052+0000 7f026dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0268154d30 con 0x7f02680c8820 2023-12-26T22:52:42.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f02677fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680b19f0 0x7f0268153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f02677fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680b19f0 0x7f0268153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57448/0 (socket says 172.21.15.131:57448) 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f02677fe700 1 -- 172.21.15.131:0/2013942960 learned_addr learned my addr 172.21.15.131:0/2013942960 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f0266ffd700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02680c8820 0x7f02681547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f0267fff700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02680c8bf0 0x7f02681586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f02677fe700 1 -- 172.21.15.131:0/2013942960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02680c8820 msgr2=0x7f02681547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f02677fe700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02680c8820 0x7f02681547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f02677fe700 1 -- 172.21.15.131:0/2013942960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02680c8bf0 msgr2=0x7f02681586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f02677fe700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02680c8bf0 0x7f02681586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f02677fe700 1 -- 172.21.15.131:0/2013942960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02580094f0 con 0x7f02680b19f0 2023-12-26T22:52:42.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f0267fff700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02680c8bf0 0x7f02681586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f02677fe700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680b19f0 0x7f0268153090 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0258009290 tx=0x7f0258016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f024f7fe700 1 -- 172.21.15.131:0/2013942960 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258011e20 con 0x7f02680b19f0 2023-12-26T22:52:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.053+0000 7f024f7fe700 1 -- 172.21.15.131:0/2013942960 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0258020070 con 0x7f02680b19f0 2023-12-26T22:52:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.054+0000 7f024f7fe700 1 -- 172.21.15.131:0/2013942960 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025801b6a0 con 0x7f02680b19f0 2023-12-26T22:52:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.054+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0268153750 con 0x7f02680b19f0 2023-12-26T22:52:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.054+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0268153ff0 con 0x7f02680b19f0 2023-12-26T22:52:42.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.055+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0268153b40 con 0x7f02680b19f0 2023-12-26T22:52:42.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.056+0000 7f024f7fe700 1 -- 172.21.15.131:0/2013942960 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0258022030 con 0x7f02680b19f0 2023-12-26T22:52:42.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.056+0000 7f024f7fe700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0254066f30 0x7f02540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.057+0000 7f024f7fe700 1 -- 172.21.15.131:0/2013942960 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f0258085f00 con 0x7f02680b19f0 2023-12-26T22:52:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.057+0000 7f0266ffd700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0254066f30 0x7f02540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.057+0000 7f0266ffd700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0254066f30 0x7f02540693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0268150920 tx=0x7f025c009040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.061+0000 7f024f7fe700 1 -- 172.21.15.131:0/2013942960 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0258052f00 con 0x7f02680b19f0 2023-12-26T22:52:42.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.295+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f02680657e0 con 0x7f02680b19f0 2023-12-26T22:52:42.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.830+0000 7f024f7fe700 1 -- 172.21.15.131:0/2013942960 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0258056520 con 0x7f02680b19f0 2023-12-26T22:52:42.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0254066f30 msgr2=0x7f02540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:42.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0254066f30 0x7f02540693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0268150920 tx=0x7f025c009040 comp rx=0 tx=0).stop 2023-12-26T22:52:42.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680b19f0 msgr2=0x7f0268153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:42.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680b19f0 0x7f0268153090 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0258009290 tx=0x7f0258016000 comp rx=0 tx=0).stop 2023-12-26T22:52:42.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 shutdown_connections 2023-12-26T22:52:42.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02680c8bf0 0x7f02681586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:42.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0254066f30 0x7f02540693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:42.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02680b19f0 0x7f0268153090 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:42.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 --2- 172.21.15.131:0/2013942960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02680c8820 0x7f02681547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:42.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 >> 172.21.15.131:0/2013942960 conn(0x7f0268068040 msgr2=0x7f02680c2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:42.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 shutdown_connections 2023-12-26T22:52:42.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:42.833+0000 7f026dfff700 1 -- 172.21.15.131:0/2013942960 wait complete. 2023-12-26T22:52:42.852 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:52:43.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.044+0000 7fd8b1dff700 1 Processor -- start 2023-12-26T22:52:43.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.044+0000 7fd8b1dff700 1 -- start start 2023-12-26T22:52:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.045+0000 7fd8b1dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd8ac0cb320 0x7fd8ac069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.045+0000 7fd8b1dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8ac0b0e60 0x7fd8ac0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.045+0000 7fd8b1dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ac156940 con 0x7fd8ac06a1e0 2023-12-26T22:52:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.045+0000 7fd8b1dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ac0c8cc0 con 0x7fd8ac0cb320 2023-12-26T22:52:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.045+0000 7fd8b1dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ac0ad700 con 0x7fd8ac0b0e60 2023-12-26T22:52:43.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8ab7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd8ac0cb320 0x7fd8ac069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8aaffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd8ac06a1e0 0x7fd8ac0b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58888/0 (socket says 172.21.15.131:58888) 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8aaffd700 1 -- 172.21.15.131:0/2250522651 learned_addr learned my addr 172.21.15.131:0/2250522651 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8ab7fe700 1 -- 172.21.15.131:0/2250522651 >> v2:172.21.15.190:3300/0 conn(0x7fd8ac0b0e60 msgr2=0x7fd8ac0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8abfff700 1 --2- 172.21.15.131:0/2250522651 >> v2:172.21.15.190:3300/0 conn(0x7fd8ac0b0e60 0x7fd8ac0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8aa7fc700 1 -- 172.21.15.131:0/2250522651 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696249106 0 0) 0x7fd8ac156940 con 0x7fd8ac06a1e0 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8ab7fe700 1 --2- 172.21.15.131:0/2250522651 >> v2:172.21.15.190:3300/0 conn(0x7fd8ac0b0e60 0x7fd8ac0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8ab7fe700 1 -- 172.21.15.131:0/2250522651 >> v1:172.21.15.131:6789/0 conn(0x7fd8ac06a1e0 legacy=0x7fd8ac0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8ab7fe700 1 -- 172.21.15.131:0/2250522651 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac0ae500 con 0x7fd8ac0cb320 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8abfff700 1 --2- 172.21.15.131:0/2250522651 >> v2:172.21.15.190:3300/0 conn(0x7fd8ac0b0e60 0x7fd8ac0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:43.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.046+0000 7fd8ab7fe700 1 --2- 172.21.15.131:0/2250522651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 0x7fd8ac069ca0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac06fb50 tx=0x7fd89c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2afba2d9314e4b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8aa7fc700 1 -- 172.21.15.131:0/2250522651 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00b6b0 con 0x7fd8ac0cb320 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8aa7fc700 1 -- 172.21.15.131:0/2250522651 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c00b810 con 0x7fd8ac0cb320 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8aa7fc700 1 -- 172.21.15.131:0/2250522651 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00bbb0 con 0x7fd8ac0cb320 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2250522651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 msgr2=0x7fd8ac069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8b1dff700 1 --2- 172.21.15.131:0/2250522651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 0x7fd8ac069ca0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac06fb50 tx=0x7fd89c0091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2250522651 shutdown_connections 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8b1dff700 1 --2- 172.21.15.131:0/2250522651 >> v2:172.21.15.190:3300/0 conn(0x7fd8ac0b0e60 0x7fd8ac0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8b1dff700 1 --2- 172.21.15.131:0/2250522651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 0x7fd8ac069ca0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2250522651 >> 172.21.15.131:0/2250522651 conn(0x7fd8ac068040 msgr2=0x7fd8ac069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:43.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2250522651 shutdown_connections 2023-12-26T22:52:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.047+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2250522651 wait complete. 2023-12-26T22:52:43.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.049+0000 7fd8b1dff700 1 Processor -- start 2023-12-26T22:52:43.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.049+0000 7fd8b1dff700 1 -- start start 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8b1dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ac06a1e0 0x7fd8ac152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8b1dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ac0b0e60 0x7fd8ac15d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8b1dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 0x7fd8ac14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8b1dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac15a530 con 0x7fd8ac0cb320 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8b1dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac15a810 con 0x7fd8ac06a1e0 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8b1dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac15aaf0 con 0x7fd8ac0b0e60 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 0x7fd8ac14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 0x7fd8ac14f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57460/0 (socket says 172.21.15.131:57460) 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8abfff700 1 -- 172.21.15.131:0/2813585908 learned_addr learned my addr 172.21.15.131:0/2813585908 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:43.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8ab7fe700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ac06a1e0 0x7fd8ac152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8aaffd700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ac0b0e60 0x7fd8ac15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8abfff700 1 -- 172.21.15.131:0/2813585908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ac0b0e60 msgr2=0x7fd8ac15d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8abfff700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ac0b0e60 0x7fd8ac15d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8abfff700 1 -- 172.21.15.131:0/2813585908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ac06a1e0 msgr2=0x7fd8ac152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8abfff700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ac06a1e0 0x7fd8ac152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8abfff700 1 -- 172.21.15.131:0/2813585908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c009420 con 0x7fd8ac0cb320 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.050+0000 7fd8ab7fe700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ac06a1e0 0x7fd8ac152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.051+0000 7fd8abfff700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 0x7fd8ac14f940 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400cfb0 tx=0x7fd8a40078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.051+0000 7fd8937fe700 1 -- 172.21.15.131:0/2813585908 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4015440 con 0x7fd8ac0cb320 2023-12-26T22:52:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.051+0000 7fd8937fe700 1 -- 172.21.15.131:0/2813585908 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a40155a0 con 0x7fd8ac0cb320 2023-12-26T22:52:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.051+0000 7fd8937fe700 1 -- 172.21.15.131:0/2813585908 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4015960 con 0x7fd8ac0cb320 2023-12-26T22:52:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.051+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ac15add0 con 0x7fd8ac0cb320 2023-12-26T22:52:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.051+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac167550 con 0x7fd8ac0cb320 2023-12-26T22:52:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.052+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac167770 con 0x7fd8ac0cb320 2023-12-26T22:52:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.053+0000 7fd8937fe700 1 -- 172.21.15.131:0/2813585908 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd8a4003020 con 0x7fd8ac0cb320 2023-12-26T22:52:43.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.054+0000 7fd8937fe700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd898066f30 0x7fd8980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.054+0000 7fd8937fe700 1 -- 172.21.15.131:0/2813585908 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd8a4084f80 con 0x7fd8ac0cb320 2023-12-26T22:52:43.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.054+0000 7fd8ab7fe700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd898066f30 0x7fd8980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.055+0000 7fd8ab7fe700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd898066f30 0x7fd8980693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0091e0 tx=0x7fd89c017000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:43.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.058+0000 7fd8937fe700 1 -- 172.21.15.131:0/2813585908 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8a404d5a0 con 0x7fd8ac0cb320 2023-12-26T22:52:43.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.292+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8ac0657e0 con 0x7fd8ac0cb320 2023-12-26T22:52:43.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.293+0000 7fd8937fe700 1 -- 172.21.15.131:0/2813585908 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fd8a4051f80 con 0x7fd8ac0cb320 2023-12-26T22:52:43.292 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:43.293 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2085575956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2085575956},{"type":"v1","addr":"172.21.15.190:6834","nonce":2085575956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2129957224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2129957224},{"type":"v1","addr":"172.21.15.190:6835","nonce":2129957224}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3810273571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3810273571},{"type":"v1","addr":"172.21.15.131:6834","nonce":3810273571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:52:39.271885+0000","modified":"2023-12-26T22:52:42.824355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8628":{"gid":8628,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2240171052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2240171052},{"type":"v1","addr":"172.21.15.131:6835","nonce":2240171052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-12-26T22:52:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd898066f30 msgr2=0x7fd8980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd898066f30 0x7fd8980693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0091e0 tx=0x7fd89c017000 comp rx=0 tx=0).stop 2023-12-26T22:52:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 msgr2=0x7fd8ac14f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 0x7fd8ac14f940 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400cfb0 tx=0x7fd8a40078c0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 shutdown_connections 2023-12-26T22:52:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd898066f30 0x7fd8980693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ac0cb320 0x7fd8ac14f940 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ac06a1e0 0x7fd8ac152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 --2- 172.21.15.131:0/2813585908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ac0b0e60 0x7fd8ac15d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 >> 172.21.15.131:0/2813585908 conn(0x7fd8ac068040 msgr2=0x7fd8ac1571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 shutdown_connections 2023-12-26T22:52:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.295+0000 7fd8b1dff700 1 -- 172.21.15.131:0/2813585908 wait complete. 2023-12-26T22:52:43.297 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 73 2023-12-26T22:52:43.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:52:43.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.503+0000 7f54359ff700 1 Processor -- start 2023-12-26T22:52:43.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.503+0000 7f54359ff700 1 -- start start 2023-12-26T22:52:43.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.504+0000 7f54359ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f543013c510 0x7f54300c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.504+0000 7f54359ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f543006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.504+0000 7f54359ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54300686c0 con 0x7f54300cc100 2023-12-26T22:52:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.504+0000 7f54359ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5430069520 con 0x7f543013c510 2023-12-26T22:52:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.504+0000 7f54359ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543006c250 con 0x7f543006b8c0 2023-12-26T22:52:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542effd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f543013c510 0x7f54300c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542e7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f543006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542effd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f543013c510 0x7f54300c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:52990/0 (socket says 172.21.15.131:52990) 2023-12-26T22:52:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542f7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f54300cc100 0x7f54300cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:58412/0 (socket says 172.21.15.131:58412) 2023-12-26T22:52:43.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542e7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f543006bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:55528/0 (socket says 172.21.15.131:55528) 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542effd700 1 -- 172.21.15.131:0/2419170155 learned_addr learned my addr 172.21.15.131:0/2419170155 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542e7fc700 1 -- 172.21.15.131:0/2419170155 >> v2:172.21.15.190:3300/0 conn(0x7f543013c510 msgr2=0x7f54300c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542e7fc700 1 --2- 172.21.15.131:0/2419170155 >> v2:172.21.15.190:3300/0 conn(0x7f543013c510 0x7f54300c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542e7fc700 1 -- 172.21.15.131:0/2419170155 >> v1:172.21.15.190:6789/0 conn(0x7f54300cc100 legacy=0x7f54300cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.505+0000 7f542e7fc700 1 -- 172.21.15.131:0/2419170155 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5430068910 con 0x7f543006b8c0 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.506+0000 7f542e7fc700 1 --2- 172.21.15.131:0/2419170155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f543006bd10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54240098e0 tx=0x7f54240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be2460ad93ffcc87 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.506+0000 7f542dffb700 1 -- 172.21.15.131:0/2419170155 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400b640 con 0x7f543006b8c0 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.506+0000 7f542dffb700 1 -- 172.21.15.131:0/2419170155 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542400b7a0 con 0x7f543006b8c0 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.506+0000 7f542dffb700 1 -- 172.21.15.131:0/2419170155 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400bb40 con 0x7f543006b8c0 2023-12-26T22:52:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.506+0000 7f54359ff700 1 -- 172.21.15.131:0/2419170155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 msgr2=0x7f543006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.506+0000 7f54359ff700 1 --2- 172.21.15.131:0/2419170155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f543006bd10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54240098e0 tx=0x7f54240091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.507+0000 7f54359ff700 1 -- 172.21.15.131:0/2419170155 shutdown_connections 2023-12-26T22:52:43.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.507+0000 7f54359ff700 1 --2- 172.21.15.131:0/2419170155 >> v2:172.21.15.190:3300/0 conn(0x7f543013c510 0x7f54300c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.507+0000 7f54359ff700 1 --2- 172.21.15.131:0/2419170155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f543006bd10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.507+0000 7f54359ff700 1 -- 172.21.15.131:0/2419170155 >> 172.21.15.131:0/2419170155 conn(0x7f54300ce3e0 msgr2=0x7f54300ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:43.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.507+0000 7f54359ff700 1 -- 172.21.15.131:0/2419170155 shutdown_connections 2023-12-26T22:52:43.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.507+0000 7f54359ff700 1 -- 172.21.15.131:0/2419170155 wait complete. 2023-12-26T22:52:43.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.508+0000 7f54359ff700 1 Processor -- start 2023-12-26T22:52:43.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.508+0000 7f54359ff700 1 -- start start 2023-12-26T22:52:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f54359ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f54300c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f54359ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54300cc100 0x7f5430154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f54359ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f543013c510 0x7f5430156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f54359ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54301553e0 con 0x7f543013c510 2023-12-26T22:52:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f54359ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5430169610 con 0x7f54300cc100 2023-12-26T22:52:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f54359ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54301698d0 con 0x7f543006b8c0 2023-12-26T22:52:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f542f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f543013c510 0x7f5430156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f542f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f543013c510 0x7f5430156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57466/0 (socket says 172.21.15.131:57466) 2023-12-26T22:52:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f542f7fe700 1 -- 172.21.15.131:0/218452633 learned_addr learned my addr 172.21.15.131:0/218452633 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f542e7fc700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54300cc100 0x7f5430154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.509+0000 7f542effd700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f54300c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f542f7fe700 1 -- 172.21.15.131:0/218452633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 msgr2=0x7f54300c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f542f7fe700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f54300c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f542f7fe700 1 -- 172.21.15.131:0/218452633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54300cc100 msgr2=0x7f5430154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f542f7fe700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54300cc100 0x7f5430154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f542f7fe700 1 -- 172.21.15.131:0/218452633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424009420 con 0x7f543013c510 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f542e7fc700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54300cc100 0x7f5430154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f542f7fe700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f543013c510 0x7f5430156800 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f542800cd00 tx=0x7f5428001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f5416ffd700 1 -- 172.21.15.131:0/218452633 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800ce60 con 0x7f543013c510 2023-12-26T22:52:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f5416ffd700 1 -- 172.21.15.131:0/218452633 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428005df0 con 0x7f543013c510 2023-12-26T22:52:43.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f5416ffd700 1 -- 172.21.15.131:0/218452633 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800ab00 con 0x7f543013c510 2023-12-26T22:52:43.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5430169bb0 con 0x7f543013c510 2023-12-26T22:52:43.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.510+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543016a170 con 0x7f543013c510 2023-12-26T22:52:43.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.511+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54300657e0 con 0x7f543013c510 2023-12-26T22:52:43.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.512+0000 7f5416ffd700 1 -- 172.21.15.131:0/218452633 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f54280070f0 con 0x7f543013c510 2023-12-26T22:52:43.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.513+0000 7f5416ffd700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f541c066f30 0x7f541c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.513+0000 7f5416ffd700 1 -- 172.21.15.131:0/218452633 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f5428084710 con 0x7f543013c510 2023-12-26T22:52:43.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.513+0000 7f542effd700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f541c066f30 0x7f541c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.514+0000 7f542effd700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f541c066f30 0x7f541c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5430152d90 tx=0x7f5420009220 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:43.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.517+0000 7f5416ffd700 1 -- 172.21.15.131:0/218452633 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5428051830 con 0x7f543013c510 2023-12-26T22:52:43.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.712+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5430068bf0 con 0x7f543013c510 2023-12-26T22:52:43.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.713+0000 7f5416ffd700 1 -- 172.21.15.131:0/218452633 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16124 (secure 0 0 0) 0x7f5428054e50 con 0x7f543013c510 2023-12-26T22:52:43.712 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:43.713 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":161,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:52:40.810546+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:52:37.440528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-26T22:52:38.259993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:52:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f541c066f30 msgr2=0x7f541c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f541c066f30 0x7f541c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5430152d90 tx=0x7f5420009220 comp rx=0 tx=0).stop 2023-12-26T22:52:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f543013c510 msgr2=0x7f5430156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f543013c510 0x7f5430156800 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f542800cd00 tx=0x7f5428001b10 comp rx=0 tx=0).stop 2023-12-26T22:52:43.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 shutdown_connections 2023-12-26T22:52:43.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f541c066f30 0x7f541c0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f543013c510 0x7f5430156800 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f543006b8c0 0x7f54300c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 --2- 172.21.15.131:0/218452633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54300cc100 0x7f5430154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 >> 172.21.15.131:0/218452633 conn(0x7f54300ce3e0 msgr2=0x7f543006cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:43.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 shutdown_connections 2023-12-26T22:52:43.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.715+0000 7f54359ff700 1 -- 172.21.15.131:0/218452633 wait complete. 2023-12-26T22:52:43.734 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T22:52:43.735 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T22:52:43.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.929+0000 7f9e49bff700 1 Processor -- start 2023-12-26T22:52:43.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.929+0000 7f9e49bff700 1 -- start start 2023-12-26T22:52:43.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.930+0000 7f9e49bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9e440cc980 0x7f9e440b0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.930+0000 7f9e49bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9e440b0e80 0x7f9e440c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.930+0000 7f9e49bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e440b04b0 con 0x7f9e440acc60 2023-12-26T22:52:43.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.930+0000 7f9e49bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e440af640 con 0x7f9e440b0e80 2023-12-26T22:52:43.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.930+0000 7f9e49bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e440af8a0 con 0x7f9e440cc980 2023-12-26T22:52:43.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.930+0000 7f9e43fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9e440b0e80 0x7f9e440c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.930+0000 7f9e437fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9e440acc60 0x7f9e440aef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58904/0 (socket says 172.21.15.131:58904) 2023-12-26T22:52:43.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.930+0000 7f9e437fe700 1 -- 172.21.15.131:0/2160019392 learned_addr learned my addr 172.21.15.131:0/2160019392 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:43.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e42ffd700 1 --2- 172.21.15.131:0/2160019392 >> v2:172.21.15.190:3300/0 conn(0x7f9e440cc980 0x7f9e440b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e427fc700 1 -- 172.21.15.131:0/2160019392 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351484569 0 0) 0x7f9e440b04b0 con 0x7f9e440acc60 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e427fc700 1 -- 172.21.15.131:0/2160019392 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e2c003570 con 0x7f9e440acc60 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e43fff700 1 -- 172.21.15.131:0/2160019392 >> v2:172.21.15.190:3300/0 conn(0x7f9e440cc980 msgr2=0x7f9e440b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e43fff700 1 --2- 172.21.15.131:0/2160019392 >> v2:172.21.15.190:3300/0 conn(0x7f9e440cc980 0x7f9e440b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e43fff700 1 -- 172.21.15.131:0/2160019392 >> v1:172.21.15.131:6789/0 conn(0x7f9e440acc60 legacy=0x7f9e440aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e43fff700 1 -- 172.21.15.131:0/2160019392 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440afb80 con 0x7f9e440b0e80 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e42ffd700 1 --2- 172.21.15.131:0/2160019392 >> v2:172.21.15.190:3300/0 conn(0x7f9e440cc980 0x7f9e440b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e43fff700 1 --2- 172.21.15.131:0/2160019392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440b0e80 0x7f9e440c90a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9e440cb650 tx=0x7f9e3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e132489bd55127 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e427fc700 1 -- 172.21.15.131:0/2160019392 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00e070 con 0x7f9e440b0e80 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e427fc700 1 -- 172.21.15.131:0/2160019392 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c00bb60 con 0x7f9e440b0e80 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.931+0000 7f9e427fc700 1 -- 172.21.15.131:0/2160019392 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c012430 con 0x7f9e440b0e80 2023-12-26T22:52:43.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.932+0000 7f9e49bff700 1 -- 172.21.15.131:0/2160019392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440b0e80 msgr2=0x7f9e440c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.932+0000 7f9e49bff700 1 --2- 172.21.15.131:0/2160019392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440b0e80 0x7f9e440c90a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9e440cb650 tx=0x7f9e3c009780 comp rx=0 tx=0).stop 2023-12-26T22:52:43.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.932+0000 7f9e49bff700 1 -- 172.21.15.131:0/2160019392 shutdown_connections 2023-12-26T22:52:43.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.932+0000 7f9e49bff700 1 --2- 172.21.15.131:0/2160019392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440b0e80 0x7f9e440c90a0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.932+0000 7f9e49bff700 1 --2- 172.21.15.131:0/2160019392 >> v2:172.21.15.190:3300/0 conn(0x7f9e440cc980 0x7f9e440b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.932+0000 7f9e49bff700 1 -- 172.21.15.131:0/2160019392 >> 172.21.15.131:0/2160019392 conn(0x7f9e44068040 msgr2=0x7f9e440cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:43.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.932+0000 7f9e49bff700 1 -- 172.21.15.131:0/2160019392 shutdown_connections 2023-12-26T22:52:43.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.932+0000 7f9e49bff700 1 -- 172.21.15.131:0/2160019392 wait complete. 2023-12-26T22:52:43.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.934+0000 7f9e49bff700 1 Processor -- start 2023-12-26T22:52:43.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.934+0000 7f9e49bff700 1 -- start start 2023-12-26T22:52:43.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e49bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e440acc60 0x7f9e44145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e49bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e440b0e80 0x7f9e4414b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e49bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440cc980 0x7f9e4414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e49bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44167210 con 0x7f9e440cc980 2023-12-26T22:52:43.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e49bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44148d80 con 0x7f9e440b0e80 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e49bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4414f350 con 0x7f9e440acc60 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440cc980 0x7f9e4414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e43fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440cc980 0x7f9e4414bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57490/0 (socket says 172.21.15.131:57490) 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e43fff700 1 -- 172.21.15.131:0/2327070312 learned_addr learned my addr 172.21.15.131:0/2327070312 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e42ffd700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e440b0e80 0x7f9e4414b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e437fe700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e440acc60 0x7f9e44145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e43fff700 1 -- 172.21.15.131:0/2327070312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e440acc60 msgr2=0x7f9e44145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e43fff700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e440acc60 0x7f9e44145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e43fff700 1 -- 172.21.15.131:0/2327070312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e440b0e80 msgr2=0x7f9e4414b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e43fff700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e440b0e80 0x7f9e4414b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:43.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e43fff700 1 -- 172.21.15.131:0/2327070312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c009a20 con 0x7f9e440cc980 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.935+0000 7f9e437fe700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e440acc60 0x7f9e44145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.936+0000 7f9e43fff700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440cc980 0x7f9e4414bb10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0097c0 tx=0x7f9e3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.936+0000 7f9e2b7fe700 1 -- 172.21.15.131:0/2327070312 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c027470 con 0x7f9e440cc980 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.936+0000 7f9e2b7fe700 1 -- 172.21.15.131:0/2327070312 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c027630 con 0x7f9e440cc980 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.936+0000 7f9e2b7fe700 1 -- 172.21.15.131:0/2327070312 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0279f0 con 0x7f9e440cc980 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.936+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e44149020 con 0x7f9e440cc980 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.936+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e44145c90 con 0x7f9e440cc980 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.937+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e44149540 con 0x7f9e440cc980 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.938+0000 7f9e2b7fe700 1 -- 172.21.15.131:0/2327070312 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9e3c024070 con 0x7f9e440cc980 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.939+0000 7f9e2b7fe700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e30066f30 0x7f9e300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.939+0000 7f9e2b7fe700 1 -- 172.21.15.131:0/2327070312 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f9e3c086f10 con 0x7f9e440cc980 2023-12-26T22:52:43.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.939+0000 7f9e437fe700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e30066f30 0x7f9e300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:43.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.939+0000 7f9e437fe700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e30066f30 0x7f9e300693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e34001360 tx=0x7f9e34009040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:43.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:43.943+0000 7f9e2b7fe700 1 -- 172.21.15.131:0/2327070312 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e44149540 con 0x7f9e440cc980 2023-12-26T22:52:44.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.131+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e44149540 con 0x7f9e30066f30 2023-12-26T22:52:44.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.141+0000 7f9e2b7fe700 1 -- 172.21.15.131:0/2327070312 <== mgr.8436 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278547 (secure 0 0 0) 0x7f9e44149540 con 0x7f9e30066f30 2023-12-26T22:52:44.145 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:44.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e30066f30 msgr2=0x7f9e300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e30066f30 0x7f9e300693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e34001360 tx=0x7f9e34009040 comp rx=0 tx=0).stop 2023-12-26T22:52:44.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440cc980 msgr2=0x7f9e4414bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440cc980 0x7f9e4414bb10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0097c0 tx=0x7f9e3c019000 comp rx=0 tx=0).stop 2023-12-26T22:52:44.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 shutdown_connections 2023-12-26T22:52:44.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e30066f30 0x7f9e300693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e440cc980 0x7f9e4414bb10 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e440acc60 0x7f9e44145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 --2- 172.21.15.131:0/2327070312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e440b0e80 0x7f9e4414b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 >> 172.21.15.131:0/2327070312 conn(0x7f9e44068040 msgr2=0x7f9e4406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:44.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 shutdown_connections 2023-12-26T22:52:44.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.145+0000 7f9e49bff700 1 -- 172.21.15.131:0/2327070312 wait complete. 2023-12-26T22:52:44.150 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T22:52:44.162 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T22:52:43.756350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2460296,"kb_used_data":16456,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752514424,"statfs":{"total":773094113280,"available":770574770176,"internally_reserved":0,"allocated":16850944,"data_stored":8950944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.969957"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.795074+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:44:32.467424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789950+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:39:08.748882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793419+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:14.172792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790531+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:00:23.547354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790084+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:34:45.392051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787558+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:07:24.409370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789746+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:52:43.991067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790069+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:29.330660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.791013+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:05:31.638926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787570+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:06:21.026051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793326+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:28.315292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789952+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:24:30.781539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906969+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:47:17.114623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793875+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:05:31.638926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794441+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:11:46.085192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790533+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:39.067839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793394+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:39.067839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789816+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:30.704146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794553+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:08:55.020611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789888+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:08:28.814418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.786936+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:36:26.953614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906938+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:35.313337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793253+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:22:22.603851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787474+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:32:44.835523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794862+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:06.257273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787248+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:14:31.472805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906902+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:11:49.316037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790637+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:06.765554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.791485+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:34.546507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793323+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:42:21.268550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906899+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:12:18.480091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794953+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:57.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"150'271","reported_seq":833,"reported_epoch":157,"state":"active+clean","last_fresh":"2023-12-26T22:52:37.385427+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T22:52:37.385427+0000","last_peered":"2023-12-26T22:52:37.385427+0000","last_clean":"2023-12-26T22:52:37.385427+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T22:52:37.385427+0000","last_undegraded":"2023-12-26T22:52:37.385427+0000","last_fullsized":"2023-12-26T22:52:37.385427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.71299999999999997},"min":{"1min":0.53500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.41999999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.42499999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.73399999999999999},"min":{"1min":0.53400000000000003,"5min":0.46000000000000002,"15min":0.45700000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.41499999999999998},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.52500000000000002}]},{"osd":3,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80300000000000005,"15min":0.79200000000000004},"min":{"1min":0.63600000000000001,"5min":0.58899999999999997,"15min":0.47899999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.42799999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.749},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.34599999999999997},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.47799999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.44700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.39000000000000001},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":0.68500000000000005}]}]},{"osd":6,"up_from":15,"seq":64424510345,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.98899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.92100000000000004,"5min":0.95399999999999996,"15min":0.98799999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.50600000000000001},"max":{"1min":0.93300000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.95899999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77800000000000002},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.52800000000000002},"max":{"1min":0.95299999999999996,"5min":1.012,"15min":1.0249999999999999},"last":0.77700000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":0.998,"15min":0.998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.48099999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.871,"5min":0.995,"15min":0.995},"last":0.83499999999999996}]},{"osd":5,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42499999999999999},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70699999999999996,"15min":0.69799999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.93999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97199999999999998,"5min":1.05,"15min":1.05},"last":0.90100000000000002}]}]},{"osd":1,"up_from":15,"seq":64424510345,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312068,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059772,"statfs":{"total":96636764160,"available":96317206528,"internally_reserved":0,"allocated":2551808,"data_stored":1561644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.47199999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.88400000000000001},"last":0.317},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.35999999999999999,"5min":0.32300000000000001,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.81000000000000005,"15min":0.84499999999999997},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.32400000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.752},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.87,"15min":0.87},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.38700000000000001},"max":{"1min":0.80600000000000005,"5min":0.877,"15min":0.90900000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.38200000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.61399999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":0.97799999999999998,"5min":1.044,"15min":1.163},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.121},"last":0.82799999999999996}]},{"osd":5,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.82299999999999995},"min":{"1min":0.69399999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.028,"5min":1.0820000000000001,"15min":1.1419999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83499999999999996,"15min":0.83199999999999996},"min":{"1min":0.68100000000000005,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.04,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.2450000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.84199999999999997,"15min":0.83999999999999997},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.1259999999999999,"15min":1.2270000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.55700000000000005},"max":{"1min":0.99299999999999999,"5min":1.107,"15min":1.2689999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.497},"max":{"1min":1.0529999999999999,"5min":1.0660000000000001,"15min":1.1890000000000001},"last":0.73999999999999999}]}]},{"osd":0,"up_from":15,"seq":64424510345,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.28199999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.63400000000000001,"5min":0.73799999999999999,"15min":0.86599999999999999},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.627,"5min":0.70499999999999996,"15min":0.88800000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72899999999999998,"5min":0.81899999999999995,"15min":0.90500000000000003},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.35499999999999998},"max":{"1min":0.72699999999999998,"5min":0.80900000000000005,"15min":0.86799999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.51200000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.92200000000000004},"last":0.497}]},{"osd":4,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":1.0649999999999999,"5min":1.103,"15min":1.145},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.65800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0640000000000001,"5min":1.1299999999999999,"15min":1.218},"last":0.81299999999999994}]},{"osd":5,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.83299999999999996},"min":{"1min":0.65400000000000003,"5min":0.58099999999999996,"15min":0.51600000000000001},"max":{"1min":1.0489999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83799999999999997,"15min":0.83399999999999996},"min":{"1min":0.64300000000000002,"5min":0.56599999999999995,"15min":0.50900000000000001},"max":{"1min":1.081,"5min":1.218,"15min":1.218},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.58999999999999997},"max":{"1min":1.0229999999999999,"5min":1.1499999999999999,"15min":1.1659999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.55500000000000005},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.2350000000000001},"last":0.79800000000000004}]},{"osd":7,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82099999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.54500000000000004},"max":{"1min":1.073,"5min":1.165,"15min":1.254},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81200000000000006},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.256,"15min":1.3200000000000001},"last":0.76100000000000001}]}]},{"osd":2,"up_from":14,"seq":60129543050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29499999999999998},"max":{"1min":0.66700000000000004,"5min":1.075,"15min":1.075},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.439,"5min":0.42499999999999999,"15min":0.378},"max":{"1min":0.73199999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32900000000000001},"max":{"1min":0.751,"5min":1.079,"15min":1.079},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34100000000000003},"max":{"1min":0.65900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":1.129,"15min":1.2490000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.83899999999999997,"5min":0.84999999999999998,"15min":1.198},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.626,"5min":0.59199999999999997,"15min":0.52500000000000002},"max":{"1min":1.177,"5min":1.222,"15min":1.222},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.50600000000000001},"max":{"1min":1.044,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.88100000000000001}]},{"osd":5,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.48499999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.22},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82799999999999996,"15min":0.80900000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57099999999999995},"max":{"1min":1.095,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.78800000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.56100000000000005},"max":{"1min":1.075,"5min":1.075,"15min":1.2470000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.67600000000000005,"5min":0.625,"15min":0.58699999999999997},"max":{"1min":1.0649999999999999,"5min":1.298,"15min":1.298},"last":0.77100000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":1.0389999999999999,"5min":1.056,"15min":1.1899999999999999},"last":0.755},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84299999999999997},"min":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2350000000000001,"5min":1.367,"15min":1.367},"last":0.72599999999999998}]}]},{"osd":3,"up_from":15,"seq":64424510346,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299200,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072640,"statfs":{"total":96636764160,"available":96330383360,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28000000000000003},"max":{"1min":0.64600000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.93799999999999994},"last":0.34599999999999997}]},{"osd":1,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":1.0449999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.752,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.622,"15min":0.625},"min":{"1min":0.441,"5min":0.441,"15min":0.40000000000000002},"max":{"1min":0.73799999999999999,"5min":1.516,"15min":1.516},"last":0.504},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.49399999999999999,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.879,"5min":1.395,"15min":1.395},"last":0.72799999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.03,"5min":1.413,"15min":1.413},"last":0.63},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79800000000000004,"15min":0.78200000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46999999999999997},"max":{"1min":1.0980000000000001,"5min":1.163,"15min":1.1850000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.60099999999999998,"5min":0.54500000000000004,"15min":0.52200000000000002},"max":{"1min":1.002,"5min":1.054,"15min":1.1359999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82099999999999995,"15min":0.79900000000000004},"min":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.53100000000000003},"max":{"1min":1.054,"5min":1.202,"15min":1.2270000000000001},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.66100000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.0880000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60399999999999998},"max":{"1min":1.101,"5min":1.1140000000000001,"15min":1.165},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82399999999999995,"15min":0.81999999999999995},"min":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.55300000000000005},"max":{"1min":0.998,"5min":1.321,"15min":1.321},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.52000000000000002},"max":{"1min":1.0780000000000001,"5min":1.466,"15min":1.466},"last":0.59399999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543049,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.52300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.90400000000000003,"15min":0.99299999999999999},"last":0.436},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.99199999999999999,"15min":1.0640000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53900000000000003,"5min":0.48699999999999999,"15min":0.47099999999999997},"max":{"1min":0.90500000000000003,"5min":1.01,"15min":1.0309999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72999999999999998,"15min":0.73799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90000000000000002,"5min":0.98599999999999999,"15min":1.04},"last":0.81200000000000006}]},{"osd":2,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":0.90800000000000003,"5min":0.93400000000000005,"15min":0.98599999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96399999999999997,"5min":0.97799999999999998,"15min":0.99399999999999999},"last":0.59099999999999997}]},{"osd":3,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.46300000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75800000000000001,"15min":0.76700000000000002},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.503},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.78100000000000003}]},{"osd":5,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.97399999999999998},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.35699999999999998},"max":{"1min":0.87,"5min":0.94799999999999995,"15min":0.98699999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":1.0149999999999999},"last":0.85199999999999998}]},{"osd":7,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.44900000000000001},"max":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":1.0449999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.89300000000000002}]}]},{"osd":5,"up_from":15,"seq":64424510344,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312068,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059772,"statfs":{"total":96636764160,"available":96317206528,"internally_reserved":0,"allocated":2551808,"data_stored":1561644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40100000000000002},"max":{"1min":0.96199999999999997,"5min":1.071,"15min":1.071},"last":0.59799999999999998}]},{"osd":1,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.58199999999999996,"5min":0.50600000000000001,"15min":0.46200000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.2269999999999999,"5min":3.2269999999999999,"15min":3.2269999999999999},"last":0.63100000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91500000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.75600000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9199999999999999,"5min":2.9199999999999999,"15min":2.9199999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.97499999999999998}]},{"osd":6,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.0169999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.48499999999999999,"15min":0.45300000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.93400000000000005}]},{"osd":7,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95099999999999996,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.95099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T22:52:44.168 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T22:52:44.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.393+0000 7f7630dff700 1 Processor -- start 2023-12-26T22:52:44.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.393+0000 7f7630dff700 1 -- start start 2023-12-26T22:52:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.394+0000 7f7630dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f762c159a70 0x7f762c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.394+0000 7f7630dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c0b2a90 0x7f762c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.394+0000 7f7630dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762c0afcd0 con 0x7f762c1596a0 2023-12-26T22:52:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.394+0000 7f7630dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762c0b3e70 con 0x7f762c159a70 2023-12-26T22:52:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.394+0000 7f7630dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762c0b0f20 con 0x7f762c0b2a90 2023-12-26T22:52:44.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f762a59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f762c1596a0 0x7f762c0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58908/0 (socket says 172.21.15.131:58908) 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f762a59c700 1 -- 172.21.15.131:0/198300117 learned_addr learned my addr 172.21.15.131:0/198300117 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f7629d9b700 1 --2- 172.21.15.131:0/198300117 >> v2:172.21.15.190:3300/0 conn(0x7f762c159a70 0x7f762c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f762ad9d700 1 --2- 172.21.15.131:0/198300117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c0b2a90 0x7f762c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f762959a700 1 -- 172.21.15.131:0/198300117 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601323477 0 0) 0x7f762c0afcd0 con 0x7f762c1596a0 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f762959a700 1 -- 172.21.15.131:0/198300117 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7614003570 con 0x7f762c1596a0 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f762ad9d700 1 -- 172.21.15.131:0/198300117 >> v2:172.21.15.190:3300/0 conn(0x7f762c159a70 msgr2=0x7f762c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f762ad9d700 1 --2- 172.21.15.131:0/198300117 >> v2:172.21.15.190:3300/0 conn(0x7f762c159a70 0x7f762c0b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f762ad9d700 1 -- 172.21.15.131:0/198300117 >> v1:172.21.15.131:6789/0 conn(0x7f762c1596a0 legacy=0x7f762c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f762ad9d700 1 -- 172.21.15.131:0/198300117 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c0b3540 con 0x7f762c0b2a90 2023-12-26T22:52:44.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.395+0000 7f7629d9b700 1 --2- 172.21.15.131:0/198300117 >> v2:172.21.15.190:3300/0 conn(0x7f762c159a70 0x7f762c0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f762ad9d700 1 --2- 172.21.15.131:0/198300117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c0b2a90 0x7f762c0b06a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7624009e60 tx=0x7f7624009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf8d771d62e4e62 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f762959a700 1 -- 172.21.15.131:0/198300117 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400e070 con 0x7f762c0b2a90 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f762959a700 1 -- 172.21.15.131:0/198300117 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762400bab0 con 0x7f762c0b2a90 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f762959a700 1 -- 172.21.15.131:0/198300117 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400be30 con 0x7f762c0b2a90 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f7630dff700 1 -- 172.21.15.131:0/198300117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c0b2a90 msgr2=0x7f762c0b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f7630dff700 1 --2- 172.21.15.131:0/198300117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c0b2a90 0x7f762c0b06a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7624009e60 tx=0x7f7624009780 comp rx=0 tx=0).stop 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f7630dff700 1 -- 172.21.15.131:0/198300117 shutdown_connections 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f7630dff700 1 --2- 172.21.15.131:0/198300117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c0b2a90 0x7f762c0b06a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f7630dff700 1 --2- 172.21.15.131:0/198300117 >> v2:172.21.15.190:3300/0 conn(0x7f762c159a70 0x7f762c0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.396+0000 7f7630dff700 1 -- 172.21.15.131:0/198300117 >> 172.21.15.131:0/198300117 conn(0x7f762c06ece0 msgr2=0x7f762c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:44.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.397+0000 7f7630dff700 1 -- 172.21.15.131:0/198300117 shutdown_connections 2023-12-26T22:52:44.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.397+0000 7f7630dff700 1 -- 172.21.15.131:0/198300117 wait complete. 2023-12-26T22:52:44.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.398+0000 7f7630dff700 1 Processor -- start 2023-12-26T22:52:44.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.398+0000 7f7630dff700 1 -- start start 2023-12-26T22:52:44.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.399+0000 7f7630dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f762c0b2a90 0x7f762c154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.399+0000 7f7630dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762c1596a0 0x7f762c15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.399+0000 7f7630dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c159a70 0x7f762c157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.399+0000 7f7630dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c0cd420 con 0x7f762c0b2a90 2023-12-26T22:52:44.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.399+0000 7f7630dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c0cc810 con 0x7f762c1596a0 2023-12-26T22:52:44.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.399+0000 7f7630dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762c0ccaf0 con 0x7f762c159a70 2023-12-26T22:52:44.402 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.399+0000 7f762a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f762c0b2a90 0x7f762c154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.399+0000 7f762a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f762c0b2a90 0x7f762c154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57492/0 (socket says 172.21.15.131:57492) 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.399+0000 7f762a59c700 1 -- 172.21.15.131:0/2616973557 learned_addr learned my addr 172.21.15.131:0/2616973557 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f7629d9b700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762c1596a0 0x7f762c15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f762ad9d700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c159a70 0x7f762c157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f762a59c700 1 -- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c159a70 msgr2=0x7f762c157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f762a59c700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c159a70 0x7f762c157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f762a59c700 1 -- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762c1596a0 msgr2=0x7f762c15a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f762a59c700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762c1596a0 0x7f762c15a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f762a59c700 1 -- 172.21.15.131:0/2616973557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7620009420 con 0x7f762c0b2a90 2023-12-26T22:52:44.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f762ad9d700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c159a70 0x7f762c157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:44.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f7629d9b700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762c1596a0 0x7f762c15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:44.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f762a59c700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f762c0b2a90 0x7f762c154f10 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f761c0011e0 tx=0x7f761c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:44.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f761a7fc700 1 -- 172.21.15.131:0/2616973557 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c005b30 con 0x7f762c0b2a90 2023-12-26T22:52:44.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f761a7fc700 1 -- 172.21.15.131:0/2616973557 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c005c90 con 0x7f762c0b2a90 2023-12-26T22:52:44.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f761a7fc700 1 -- 172.21.15.131:0/2616973557 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c0072e0 con 0x7f762c0b2a90 2023-12-26T22:52:44.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7624009a20 con 0x7f762c0b2a90 2023-12-26T22:52:44.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.400+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762c0cd0b0 con 0x7f762c0b2a90 2023-12-26T22:52:44.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.401+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762c0657e0 con 0x7f762c0b2a90 2023-12-26T22:52:44.404 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.402+0000 7f761a7fc700 1 -- 172.21.15.131:0/2616973557 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f761c007650 con 0x7f762c0b2a90 2023-12-26T22:52:44.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.403+0000 7f761a7fc700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f760c066f30 0x7f760c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.405 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.403+0000 7f761a7fc700 1 -- 172.21.15.131:0/2616973557 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f761c083bb0 con 0x7f762c0b2a90 2023-12-26T22:52:44.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.403+0000 7f7629d9b700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f760c066f30 0x7f760c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.404+0000 7f7629d9b700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f760c066f30 0x7f760c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76200093f0 tx=0x7f7620008040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:44.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.407+0000 7f761a7fc700 1 -- 172.21.15.131:0/2616973557 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f761c04b5a0 con 0x7f762c0b2a90 2023-12-26T22:52:44.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.593+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f762c0cd560 con 0x7f760c066f30 2023-12-26T22:52:44.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.601+0000 7f761a7fc700 1 -- 172.21.15.131:0/2616973557 <== mgr.8436 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278547 (secure 0 0 0) 0x7f7620010950 con 0x7f760c066f30 2023-12-26T22:52:44.605 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:44.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.605+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f760c066f30 msgr2=0x7f760c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.605+0000 7f7630dff700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f760c066f30 0x7f760c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76200093f0 tx=0x7f7620008040 comp rx=0 tx=0).stop 2023-12-26T22:52:44.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.605+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f762c0b2a90 msgr2=0x7f762c154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.605+0000 7f7630dff700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f762c0b2a90 0x7f762c154f10 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f761c0011e0 tx=0x7f761c007df0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.606+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 shutdown_connections 2023-12-26T22:52:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.606+0000 7f7630dff700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f762c159a70 0x7f762c157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.606+0000 7f7630dff700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f760c066f30 0x7f760c0693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.606+0000 7f7630dff700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f762c0b2a90 0x7f762c154f10 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.606+0000 7f7630dff700 1 --2- 172.21.15.131:0/2616973557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f762c1596a0 0x7f762c15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.606+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 >> 172.21.15.131:0/2616973557 conn(0x7f762c06ece0 msgr2=0x7f762c0cf180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.606+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 shutdown_connections 2023-12-26T22:52:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.606+0000 7f7630dff700 1 -- 172.21.15.131:0/2616973557 wait complete. 2023-12-26T22:52:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T22:52:44.623 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T22:52:43.756350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2460296,"kb_used_data":16456,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752514424,"statfs":{"total":773094113280,"available":770574770176,"internally_reserved":0,"allocated":16850944,"data_stored":8950944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.969957"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.795074+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:44:32.467424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789950+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:39:08.748882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793419+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:14.172792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790531+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:00:23.547354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790084+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:34:45.392051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787558+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:07:24.409370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789746+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:52:43.991067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790069+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:29.330660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.791013+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:05:31.638926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787570+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:06:21.026051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793326+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:28.315292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789952+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:24:30.781539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906969+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:47:17.114623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793875+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:05:31.638926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794441+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:11:46.085192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790533+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:39.067839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793394+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:39.067839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789816+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:30.704146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794553+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:08:55.020611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789888+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:08:28.814418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.786936+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:36:26.953614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906938+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:35.313337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793253+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:22:22.603851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787474+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:32:44.835523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794862+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:06.257273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787248+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:14:31.472805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906902+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:11:49.316037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790637+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:06.765554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.791485+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:34.546507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793323+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:42:21.268550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906899+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:12:18.480091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794953+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:57.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"150'271","reported_seq":833,"reported_epoch":157,"state":"active+clean","last_fresh":"2023-12-26T22:52:37.385427+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T22:52:37.385427+0000","last_peered":"2023-12-26T22:52:37.385427+0000","last_clean":"2023-12-26T22:52:37.385427+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T22:52:37.385427+0000","last_undegraded":"2023-12-26T22:52:37.385427+0000","last_fullsized":"2023-12-26T22:52:37.385427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.71299999999999997},"min":{"1min":0.53500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.41999999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.42499999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.73399999999999999},"min":{"1min":0.53400000000000003,"5min":0.46000000000000002,"15min":0.45700000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.41499999999999998},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.52500000000000002}]},{"osd":3,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80300000000000005,"15min":0.79200000000000004},"min":{"1min":0.63600000000000001,"5min":0.58899999999999997,"15min":0.47899999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.42799999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.749},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.34599999999999997},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.47799999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.44700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.39000000000000001},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":0.68500000000000005}]}]},{"osd":6,"up_from":15,"seq":64424510345,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.98899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.92100000000000004,"5min":0.95399999999999996,"15min":0.98799999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.50600000000000001},"max":{"1min":0.93300000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.95899999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77800000000000002},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.52800000000000002},"max":{"1min":0.95299999999999996,"5min":1.012,"15min":1.0249999999999999},"last":0.77700000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":0.998,"15min":0.998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.48099999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.871,"5min":0.995,"15min":0.995},"last":0.83499999999999996}]},{"osd":5,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42499999999999999},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70699999999999996,"15min":0.69799999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.93999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97199999999999998,"5min":1.05,"15min":1.05},"last":0.90100000000000002}]}]},{"osd":1,"up_from":15,"seq":64424510345,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312068,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059772,"statfs":{"total":96636764160,"available":96317206528,"internally_reserved":0,"allocated":2551808,"data_stored":1561644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.47199999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.88400000000000001},"last":0.317},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.35999999999999999,"5min":0.32300000000000001,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.81000000000000005,"15min":0.84499999999999997},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.32400000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.752},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.87,"15min":0.87},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.38700000000000001},"max":{"1min":0.80600000000000005,"5min":0.877,"15min":0.90900000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.38200000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.61399999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":0.97799999999999998,"5min":1.044,"15min":1.163},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.121},"last":0.82799999999999996}]},{"osd":5,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.82299999999999995},"min":{"1min":0.69399999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.028,"5min":1.0820000000000001,"15min":1.1419999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83499999999999996,"15min":0.83199999999999996},"min":{"1min":0.68100000000000005,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.04,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.2450000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.84199999999999997,"15min":0.83999999999999997},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.1259999999999999,"15min":1.2270000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.55700000000000005},"max":{"1min":0.99299999999999999,"5min":1.107,"15min":1.2689999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.497},"max":{"1min":1.0529999999999999,"5min":1.0660000000000001,"15min":1.1890000000000001},"last":0.73999999999999999}]}]},{"osd":0,"up_from":15,"seq":64424510345,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.28199999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.63400000000000001,"5min":0.73799999999999999,"15min":0.86599999999999999},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.627,"5min":0.70499999999999996,"15min":0.88800000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72899999999999998,"5min":0.81899999999999995,"15min":0.90500000000000003},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.35499999999999998},"max":{"1min":0.72699999999999998,"5min":0.80900000000000005,"15min":0.86799999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.51200000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.92200000000000004},"last":0.497}]},{"osd":4,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":1.0649999999999999,"5min":1.103,"15min":1.145},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.65800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0640000000000001,"5min":1.1299999999999999,"15min":1.218},"last":0.81299999999999994}]},{"osd":5,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.83299999999999996},"min":{"1min":0.65400000000000003,"5min":0.58099999999999996,"15min":0.51600000000000001},"max":{"1min":1.0489999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83799999999999997,"15min":0.83399999999999996},"min":{"1min":0.64300000000000002,"5min":0.56599999999999995,"15min":0.50900000000000001},"max":{"1min":1.081,"5min":1.218,"15min":1.218},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.58999999999999997},"max":{"1min":1.0229999999999999,"5min":1.1499999999999999,"15min":1.1659999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.55500000000000005},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.2350000000000001},"last":0.79800000000000004}]},{"osd":7,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82099999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.54500000000000004},"max":{"1min":1.073,"5min":1.165,"15min":1.254},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81200000000000006},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.256,"15min":1.3200000000000001},"last":0.76100000000000001}]}]},{"osd":2,"up_from":14,"seq":60129543050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29499999999999998},"max":{"1min":0.66700000000000004,"5min":1.075,"15min":1.075},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.439,"5min":0.42499999999999999,"15min":0.378},"max":{"1min":0.73199999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32900000000000001},"max":{"1min":0.751,"5min":1.079,"15min":1.079},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34100000000000003},"max":{"1min":0.65900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":1.129,"15min":1.2490000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.83899999999999997,"5min":0.84999999999999998,"15min":1.198},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.626,"5min":0.59199999999999997,"15min":0.52500000000000002},"max":{"1min":1.177,"5min":1.222,"15min":1.222},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.50600000000000001},"max":{"1min":1.044,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.88100000000000001}]},{"osd":5,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.48499999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.22},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82799999999999996,"15min":0.80900000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57099999999999995},"max":{"1min":1.095,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.78800000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.56100000000000005},"max":{"1min":1.075,"5min":1.075,"15min":1.2470000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.67600000000000005,"5min":0.625,"15min":0.58699999999999997},"max":{"1min":1.0649999999999999,"5min":1.298,"15min":1.298},"last":0.77100000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":1.0389999999999999,"5min":1.056,"15min":1.1899999999999999},"last":0.755},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84299999999999997},"min":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2350000000000001,"5min":1.367,"15min":1.367},"last":0.72599999999999998}]}]},{"osd":3,"up_from":15,"seq":64424510346,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299200,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072640,"statfs":{"total":96636764160,"available":96330383360,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28000000000000003},"max":{"1min":0.64600000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.93799999999999994},"last":0.34599999999999997}]},{"osd":1,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":1.0449999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.752,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.622,"15min":0.625},"min":{"1min":0.441,"5min":0.441,"15min":0.40000000000000002},"max":{"1min":0.73799999999999999,"5min":1.516,"15min":1.516},"last":0.504},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.49399999999999999,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.879,"5min":1.395,"15min":1.395},"last":0.72799999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.03,"5min":1.413,"15min":1.413},"last":0.63},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79800000000000004,"15min":0.78200000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46999999999999997},"max":{"1min":1.0980000000000001,"5min":1.163,"15min":1.1850000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.60099999999999998,"5min":0.54500000000000004,"15min":0.52200000000000002},"max":{"1min":1.002,"5min":1.054,"15min":1.1359999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82099999999999995,"15min":0.79900000000000004},"min":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.53100000000000003},"max":{"1min":1.054,"5min":1.202,"15min":1.2270000000000001},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.66100000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.0880000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60399999999999998},"max":{"1min":1.101,"5min":1.1140000000000001,"15min":1.165},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82399999999999995,"15min":0.81999999999999995},"min":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.55300000000000005},"max":{"1min":0.998,"5min":1.321,"15min":1.321},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.52000000000000002},"max":{"1min":1.0780000000000001,"5min":1.466,"15min":1.466},"last":0.59399999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543049,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.52300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.90400000000000003,"15min":0.99299999999999999},"last":0.436},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.99199999999999999,"15min":1.0640000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53900000000000003,"5min":0.48699999999999999,"15min":0.47099999999999997},"max":{"1min":0.90500000000000003,"5min":1.01,"15min":1.0309999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72999999999999998,"15min":0.73799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90000000000000002,"5min":0.98599999999999999,"15min":1.04},"last":0.81200000000000006}]},{"osd":2,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":0.90800000000000003,"5min":0.93400000000000005,"15min":0.98599999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96399999999999997,"5min":0.97799999999999998,"15min":0.99399999999999999},"last":0.59099999999999997}]},{"osd":3,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.46300000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75800000000000001,"15min":0.76700000000000002},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.503},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.78100000000000003}]},{"osd":5,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.97399999999999998},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.35699999999999998},"max":{"1min":0.87,"5min":0.94799999999999995,"15min":0.98699999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":1.0149999999999999},"last":0.85199999999999998}]},{"osd":7,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.44900000000000001},"max":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":1.0449999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.89300000000000002}]}]},{"osd":5,"up_from":15,"seq":64424510344,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312068,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059772,"statfs":{"total":96636764160,"available":96317206528,"internally_reserved":0,"allocated":2551808,"data_stored":1561644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40100000000000002},"max":{"1min":0.96199999999999997,"5min":1.071,"15min":1.071},"last":0.59799999999999998}]},{"osd":1,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.58199999999999996,"5min":0.50600000000000001,"15min":0.46200000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.2269999999999999,"5min":3.2269999999999999,"15min":3.2269999999999999},"last":0.63100000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91500000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.75600000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9199999999999999,"5min":2.9199999999999999,"15min":2.9199999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.97499999999999998}]},{"osd":6,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.0169999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.48499999999999999,"15min":0.45300000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.93400000000000005}]},{"osd":7,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95099999999999996,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.95099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T22:52:44.628 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T22:52:44.779 INFO:tasks.ceph.mds.a.smithi131.stderr:2023-12-26T22:52:44.779+0000 7fefcd3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T22:52:44.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.855+0000 7f61b2bff700 1 Processor -- start 2023-12-26T22:52:44.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.855+0000 7f61b2bff700 1 -- start start 2023-12-26T22:52:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.856+0000 7f61b2bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f61ac0cac30 0x7f61ac155be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.856+0000 7f61b2bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac0c8bf0 con 0x7f61ac0b07f0 2023-12-26T22:52:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.856+0000 7f61b2bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac0c8e60 con 0x7f61ac159920 2023-12-26T22:52:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.856+0000 7f61b2bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac0c9140 con 0x7f61ac0cac30 2023-12-26T22:52:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.857+0000 7f61abfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f61ac0b07f0 0x7f61ac1554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41022/0 (socket says 172.21.15.131:41022) 2023-12-26T22:52:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.857+0000 7f61abfff700 1 -- 172.21.15.131:0/3316789757 learned_addr learned my addr 172.21.15.131:0/3316789757 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.857+0000 7f61b119c700 1 --2- 172.21.15.131:0/3316789757 >> v2:172.21.15.190:3300/0 conn(0x7f61ac0cac30 0x7f61ac155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.857+0000 7f61ab7fe700 1 -- 172.21.15.131:0/3316789757 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589079807 0 0) 0x7f61ac0c8bf0 con 0x7f61ac0b07f0 2023-12-26T22:52:44.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.857+0000 7f61ab7fe700 1 -- 172.21.15.131:0/3316789757 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f619c004680 con 0x7f61ac0b07f0 2023-12-26T22:52:44.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.857+0000 7f61b119c700 1 -- 172.21.15.131:0/3316789757 >> v1:172.21.15.190:6789/0 conn(0x7f61ac159920 legacy=0x7f61ac0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.857+0000 7f61b119c700 1 -- 172.21.15.131:0/3316789757 >> v1:172.21.15.131:6789/0 conn(0x7f61ac0b07f0 legacy=0x7f61ac1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.857+0000 7f61ab7fe700 1 -- 172.21.15.131:0/3316789757 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1888081398 0 0) 0x7f619c004680 con 0x7f61ac0b07f0 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.857+0000 7f61b119c700 1 -- 172.21.15.131:0/3316789757 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac0c9420 con 0x7f61ac0cac30 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.858+0000 7f61b119c700 1 --2- 172.21.15.131:0/3316789757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61ac0cac30 0x7f61ac155be0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009e60 tx=0x7f61a4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30edeba1e48f8bf server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.858+0000 7f61ab7fe700 1 -- 172.21.15.131:0/3316789757 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a400e070 con 0x7f61ac0cac30 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.858+0000 7f61ab7fe700 1 -- 172.21.15.131:0/3316789757 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a400bb60 con 0x7f61ac0cac30 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.858+0000 7f61ab7fe700 1 -- 172.21.15.131:0/3316789757 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4012430 con 0x7f61ac0cac30 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.858+0000 7f61b2bff700 1 -- 172.21.15.131:0/3316789757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61ac0cac30 msgr2=0x7f61ac155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.858+0000 7f61b2bff700 1 --2- 172.21.15.131:0/3316789757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61ac0cac30 0x7f61ac155be0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009e60 tx=0x7f61a4009780 comp rx=0 tx=0).stop 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.858+0000 7f61b2bff700 1 -- 172.21.15.131:0/3316789757 shutdown_connections 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.858+0000 7f61b2bff700 1 --2- 172.21.15.131:0/3316789757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61ac0cac30 0x7f61ac155be0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.858+0000 7f61b2bff700 1 -- 172.21.15.131:0/3316789757 >> 172.21.15.131:0/3316789757 conn(0x7f61ac068040 msgr2=0x7f61ac069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.859+0000 7f61b2bff700 1 -- 172.21.15.131:0/3316789757 shutdown_connections 2023-12-26T22:52:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.859+0000 7f61b2bff700 1 -- 172.21.15.131:0/3316789757 wait complete. 2023-12-26T22:52:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.860+0000 7f61b2bff700 1 Processor -- start 2023-12-26T22:52:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.860+0000 7f61b2bff700 1 -- start start 2023-12-26T22:52:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61b2bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61ac0cac30 0x7f61ac0c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61b2bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61ac159920 0x7f61ac0c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61b2bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61ac166da0 0x7f61ac15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61b2bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c1f90 con 0x7f61ac159920 2023-12-26T22:52:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61b2bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c7f00 con 0x7f61ac166da0 2023-12-26T22:52:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61b2bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c7070 con 0x7f61ac0cac30 2023-12-26T22:52:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61ac159920 0x7f61ac0c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61ac159920 0x7f61ac0c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35534/0 (socket says 172.21.15.131:35534) 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61abfff700 1 -- 172.21.15.131:0/3178190592 learned_addr learned my addr 172.21.15.131:0/3178190592 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61b119c700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61ac166da0 0x7f61ac15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.861+0000 7f61b099b700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61ac0cac30 0x7f61ac0c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61abfff700 1 -- 172.21.15.131:0/3178190592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61ac0cac30 msgr2=0x7f61ac0c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61abfff700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61ac0cac30 0x7f61ac0c1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61abfff700 1 -- 172.21.15.131:0/3178190592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61ac166da0 msgr2=0x7f61ac15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61abfff700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61ac166da0 0x7f61ac15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61abfff700 1 -- 172.21.15.131:0/3178190592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a4009a20 con 0x7f61ac159920 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61abfff700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61ac159920 0x7f61ac0c6b30 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f619800c810 tx=0x7f6198007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61a8ff9700 1 -- 172.21.15.131:0/3178190592 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198007360 con 0x7f61ac159920 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61a8ff9700 1 -- 172.21.15.131:0/3178190592 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61980074c0 con 0x7f61ac159920 2023-12-26T22:52:44.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac0c72f0 con 0x7f61ac159920 2023-12-26T22:52:44.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61a8ff9700 1 -- 172.21.15.131:0/3178190592 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198004830 con 0x7f61ac159920 2023-12-26T22:52:44.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.862+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac0c78b0 con 0x7f61ac159920 2023-12-26T22:52:44.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.863+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac0c2170 con 0x7f61ac159920 2023-12-26T22:52:44.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.864+0000 7f61a8ff9700 1 -- 172.21.15.131:0/3178190592 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6198004c00 con 0x7f61ac159920 2023-12-26T22:52:44.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.865+0000 7f61a8ff9700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6190066f30 0x7f61900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:44.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.865+0000 7f61a8ff9700 1 -- 172.21.15.131:0/3178190592 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f6198084390 con 0x7f61ac159920 2023-12-26T22:52:44.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.865+0000 7f61b099b700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6190066f30 0x7f61900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:44.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.866+0000 7f61b099b700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6190066f30 0x7f61900693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61a00039e0 tx=0x7f61a0007ec0 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:44.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:44.869+0000 7f61a8ff9700 1 -- 172.21.15.131:0/3178190592 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61ac0c2170 con 0x7f61ac159920 2023-12-26T22:52:45.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.059+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61ac0c2170 con 0x7f6190066f30 2023-12-26T22:52:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.067+0000 7f61a8ff9700 1 -- 172.21.15.131:0/3178190592 <== mgr.8436 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278547 (secure 0 0 0) 0x7f61ac0c2170 con 0x7f6190066f30 2023-12-26T22:52:45.071 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:45.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6190066f30 msgr2=0x7f61900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6190066f30 0x7f61900693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61a00039e0 tx=0x7f61a0007ec0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61ac159920 msgr2=0x7f61ac0c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61ac159920 0x7f61ac0c6b30 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f619800c810 tx=0x7f6198007b50 comp rx=0 tx=0).stop 2023-12-26T22:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 shutdown_connections 2023-12-26T22:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61ac166da0 0x7f61ac15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61ac0cac30 0x7f61ac0c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6190066f30 0x7f61900693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 --2- 172.21.15.131:0/3178190592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61ac159920 0x7f61ac0c6b30 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 >> 172.21.15.131:0/3178190592 conn(0x7f61ac068040 msgr2=0x7f61ac1561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 shutdown_connections 2023-12-26T22:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.071+0000 7f61b2bff700 1 -- 172.21.15.131:0/3178190592 wait complete. 2023-12-26T22:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T22:52:45.089 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T22:52:43.756350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2460296,"kb_used_data":16456,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752514424,"statfs":{"total":773094113280,"available":770574770176,"internally_reserved":0,"allocated":16850944,"data_stored":8950944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.969957"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.795074+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:44:32.467424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789950+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:39:08.748882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793419+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:14.172792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790531+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:00:23.547354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790084+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:34:45.392051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787558+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:07:24.409370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789746+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:52:43.991067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790069+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:05:29.330660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.791013+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:05:31.638926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787570+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:06:21.026051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793326+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:28.315292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789952+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:24:30.781539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906969+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:47:17.114623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793875+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:05:31.638926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794441+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:11:46.085192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790533+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:39.067839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793394+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:39.067839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789816+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:53:30.704146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794553+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:08:55.020611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.789888+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:08:28.814418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.786936+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:36:26.953614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906938+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:49:35.313337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793253+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:22:22.603851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787474+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:32:44.835523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794862+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:26:06.257273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.787248+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:14:31.472805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906902+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:11:49.316037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.790637+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:06.765554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.791485+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:34.546507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.793323+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:42:21.268550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.906899+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:12:18.480091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T22:52:38.783519+0000","last_change":"2023-12-26T22:52:38.783519+0000","last_active":"2023-12-26T22:52:38.783519+0000","last_peered":"2023-12-26T22:52:38.783519+0000","last_clean":"2023-12-26T22:52:38.783519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:38.783519+0000","last_undegraded":"2023-12-26T22:52:38.783519+0000","last_fullsized":"2023-12-26T22:52:38.783519+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T22:52:37.777351+0000","last_change":"2023-12-26T22:52:37.794953+0000","last_active":"2023-12-26T22:52:37.777351+0000","last_peered":"2023-12-26T22:52:37.777351+0000","last_clean":"2023-12-26T22:52:37.777351+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T22:52:37.777351+0000","last_undegraded":"2023-12-26T22:52:37.777351+0000","last_fullsized":"2023-12-26T22:52:37.777351+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:57.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"150'271","reported_seq":833,"reported_epoch":157,"state":"active+clean","last_fresh":"2023-12-26T22:52:37.385427+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T22:52:37.385427+0000","last_peered":"2023-12-26T22:52:37.385427+0000","last_clean":"2023-12-26T22:52:37.385427+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T22:52:37.385427+0000","last_undegraded":"2023-12-26T22:52:37.385427+0000","last_fullsized":"2023-12-26T22:52:37.385427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.71299999999999997},"min":{"1min":0.53500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.41999999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.42499999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.73399999999999999},"min":{"1min":0.53400000000000003,"5min":0.46000000000000002,"15min":0.45700000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.41499999999999998},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.52500000000000002}]},{"osd":3,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80300000000000005,"15min":0.79200000000000004},"min":{"1min":0.63600000000000001,"5min":0.58899999999999997,"15min":0.47899999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.42799999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.749},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.34599999999999997},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.47799999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.44700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.39000000000000001},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":0.68500000000000005}]}]},{"osd":6,"up_from":15,"seq":64424510345,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.98899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.92100000000000004,"5min":0.95399999999999996,"15min":0.98799999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.50600000000000001},"max":{"1min":0.93300000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.95899999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77800000000000002},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.52800000000000002},"max":{"1min":0.95299999999999996,"5min":1.012,"15min":1.0249999999999999},"last":0.77700000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":0.998,"15min":0.998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.48099999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.871,"5min":0.995,"15min":0.995},"last":0.83499999999999996}]},{"osd":5,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42499999999999999},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70699999999999996,"15min":0.69799999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.93999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97199999999999998,"5min":1.05,"15min":1.05},"last":0.90100000000000002}]}]},{"osd":1,"up_from":15,"seq":64424510345,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312068,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059772,"statfs":{"total":96636764160,"available":96317206528,"internally_reserved":0,"allocated":2551808,"data_stored":1561644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.47199999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.88400000000000001},"last":0.317},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.35999999999999999,"5min":0.32300000000000001,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.81000000000000005,"15min":0.84499999999999997},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.32400000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.752},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.87,"15min":0.87},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.38700000000000001},"max":{"1min":0.80600000000000005,"5min":0.877,"15min":0.90900000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.38200000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.61399999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":0.97799999999999998,"5min":1.044,"15min":1.163},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.121},"last":0.82799999999999996}]},{"osd":5,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.82299999999999995},"min":{"1min":0.69399999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.028,"5min":1.0820000000000001,"15min":1.1419999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83499999999999996,"15min":0.83199999999999996},"min":{"1min":0.68100000000000005,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.04,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.2450000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.84199999999999997,"15min":0.83999999999999997},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.1259999999999999,"15min":1.2270000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.55700000000000005},"max":{"1min":0.99299999999999999,"5min":1.107,"15min":1.2689999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.497},"max":{"1min":1.0529999999999999,"5min":1.0660000000000001,"15min":1.1890000000000001},"last":0.73999999999999999}]}]},{"osd":0,"up_from":15,"seq":64424510345,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.28199999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.63400000000000001,"5min":0.73799999999999999,"15min":0.86599999999999999},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.627,"5min":0.70499999999999996,"15min":0.88800000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72899999999999998,"5min":0.81899999999999995,"15min":0.90500000000000003},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.35499999999999998},"max":{"1min":0.72699999999999998,"5min":0.80900000000000005,"15min":0.86799999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.51200000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.92200000000000004},"last":0.497}]},{"osd":4,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":1.0649999999999999,"5min":1.103,"15min":1.145},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.65800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0640000000000001,"5min":1.1299999999999999,"15min":1.218},"last":0.81299999999999994}]},{"osd":5,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.83299999999999996},"min":{"1min":0.65400000000000003,"5min":0.58099999999999996,"15min":0.51600000000000001},"max":{"1min":1.0489999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83799999999999997,"15min":0.83399999999999996},"min":{"1min":0.64300000000000002,"5min":0.56599999999999995,"15min":0.50900000000000001},"max":{"1min":1.081,"5min":1.218,"15min":1.218},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.58999999999999997},"max":{"1min":1.0229999999999999,"5min":1.1499999999999999,"15min":1.1659999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.55500000000000005},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.2350000000000001},"last":0.79800000000000004}]},{"osd":7,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82099999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.54500000000000004},"max":{"1min":1.073,"5min":1.165,"15min":1.254},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81200000000000006},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.256,"15min":1.3200000000000001},"last":0.76100000000000001}]}]},{"osd":2,"up_from":14,"seq":60129543050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29499999999999998},"max":{"1min":0.66700000000000004,"5min":1.075,"15min":1.075},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.439,"5min":0.42499999999999999,"15min":0.378},"max":{"1min":0.73199999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32900000000000001},"max":{"1min":0.751,"5min":1.079,"15min":1.079},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34100000000000003},"max":{"1min":0.65900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":1.129,"15min":1.2490000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.83899999999999997,"5min":0.84999999999999998,"15min":1.198},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.626,"5min":0.59199999999999997,"15min":0.52500000000000002},"max":{"1min":1.177,"5min":1.222,"15min":1.222},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.50600000000000001},"max":{"1min":1.044,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.88100000000000001}]},{"osd":5,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.48499999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.22},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82799999999999996,"15min":0.80900000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57099999999999995},"max":{"1min":1.095,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.78800000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.56100000000000005},"max":{"1min":1.075,"5min":1.075,"15min":1.2470000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.67600000000000005,"5min":0.625,"15min":0.58699999999999997},"max":{"1min":1.0649999999999999,"5min":1.298,"15min":1.298},"last":0.77100000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":1.0389999999999999,"5min":1.056,"15min":1.1899999999999999},"last":0.755},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84299999999999997},"min":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2350000000000001,"5min":1.367,"15min":1.367},"last":0.72599999999999998}]}]},{"osd":3,"up_from":15,"seq":64424510346,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299200,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072640,"statfs":{"total":96636764160,"available":96330383360,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28000000000000003},"max":{"1min":0.64600000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.93799999999999994},"last":0.34599999999999997}]},{"osd":1,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":1.0449999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.752,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.622,"15min":0.625},"min":{"1min":0.441,"5min":0.441,"15min":0.40000000000000002},"max":{"1min":0.73799999999999999,"5min":1.516,"15min":1.516},"last":0.504},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.49399999999999999,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.879,"5min":1.395,"15min":1.395},"last":0.72799999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.03,"5min":1.413,"15min":1.413},"last":0.63},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79800000000000004,"15min":0.78200000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46999999999999997},"max":{"1min":1.0980000000000001,"5min":1.163,"15min":1.1850000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.60099999999999998,"5min":0.54500000000000004,"15min":0.52200000000000002},"max":{"1min":1.002,"5min":1.054,"15min":1.1359999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82099999999999995,"15min":0.79900000000000004},"min":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.53100000000000003},"max":{"1min":1.054,"5min":1.202,"15min":1.2270000000000001},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.66100000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.0880000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60399999999999998},"max":{"1min":1.101,"5min":1.1140000000000001,"15min":1.165},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82399999999999995,"15min":0.81999999999999995},"min":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.55300000000000005},"max":{"1min":0.998,"5min":1.321,"15min":1.321},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.52000000000000002},"max":{"1min":1.0780000000000001,"5min":1.466,"15min":1.466},"last":0.59399999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543049,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307392,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064448,"statfs":{"total":96636764160,"available":96321994752,"internally_reserved":0,"allocated":1957888,"data_stored":971276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.52300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.90400000000000003,"15min":0.99299999999999999},"last":0.436},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.99199999999999999,"15min":1.0640000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53900000000000003,"5min":0.48699999999999999,"15min":0.47099999999999997},"max":{"1min":0.90500000000000003,"5min":1.01,"15min":1.0309999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72999999999999998,"15min":0.73799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90000000000000002,"5min":0.98599999999999999,"15min":1.04},"last":0.81200000000000006}]},{"osd":2,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":0.90800000000000003,"5min":0.93400000000000005,"15min":0.98599999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96399999999999997,"5min":0.97799999999999998,"15min":0.99399999999999999},"last":0.59099999999999997}]},{"osd":3,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.46300000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75800000000000001,"15min":0.76700000000000002},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.503},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.78100000000000003}]},{"osd":5,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.97399999999999998},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.35699999999999998},"max":{"1min":0.87,"5min":0.94799999999999995,"15min":0.98699999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":1.0149999999999999},"last":0.85199999999999998}]},{"osd":7,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.44900000000000001},"max":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":1.0449999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.89300000000000002}]}]},{"osd":5,"up_from":15,"seq":64424510344,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312068,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059772,"statfs":{"total":96636764160,"available":96317206528,"internally_reserved":0,"allocated":2551808,"data_stored":1561644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40100000000000002},"max":{"1min":0.96199999999999997,"5min":1.071,"15min":1.071},"last":0.59799999999999998}]},{"osd":1,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.58199999999999996,"5min":0.50600000000000001,"15min":0.46200000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.2269999999999999,"5min":3.2269999999999999,"15min":3.2269999999999999},"last":0.63100000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91500000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.75600000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9199999999999999,"5min":2.9199999999999999,"15min":2.9199999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.97499999999999998}]},{"osd":6,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.0169999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.48499999999999999,"15min":0.45300000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.93400000000000005}]},{"osd":7,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95099999999999996,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.95099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T22:52:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3f is not active+clean 2023-12-26T22:52:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3e is not active+clean 2023-12-26T22:52:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3d is not active+clean 2023-12-26T22:52:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3c is not active+clean 2023-12-26T22:52:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3b is not active+clean 2023-12-26T22:52:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3a is not active+clean 2023-12-26T22:52:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.39 is not active+clean 2023-12-26T22:52:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.38 is not active+clean 2023-12-26T22:52:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.37 is not active+clean 2023-12-26T22:52:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.36 is not active+clean 2023-12-26T22:52:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.35 is not active+clean 2023-12-26T22:52:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.34 is not active+clean 2023-12-26T22:52:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.33 is not active+clean 2023-12-26T22:52:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.32 is not active+clean 2023-12-26T22:52:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.31 is not active+clean 2023-12-26T22:52:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.30 is not active+clean 2023-12-26T22:52:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2f is not active+clean 2023-12-26T22:52:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2e is not active+clean 2023-12-26T22:52:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2d is not active+clean 2023-12-26T22:52:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2c is not active+clean 2023-12-26T22:52:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2b is not active+clean 2023-12-26T22:52:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2a is not active+clean 2023-12-26T22:52:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.29 is not active+clean 2023-12-26T22:52:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.28 is not active+clean 2023-12-26T22:52:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.27 is not active+clean 2023-12-26T22:52:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.26 is not active+clean 2023-12-26T22:52:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.25 is not active+clean 2023-12-26T22:52:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.24 is not active+clean 2023-12-26T22:52:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.23 is not active+clean 2023-12-26T22:52:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.22 is not active+clean 2023-12-26T22:52:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.21 is not active+clean 2023-12-26T22:52:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.20 is not active+clean 2023-12-26T22:52:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1e is not active+clean 2023-12-26T22:52:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.795074+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:44:32.467424+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1f is not active+clean 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1e is not active+clean 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.789950+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:39:08.748882+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2023-12-26T22:52:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.793419+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:47:14.172792+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T22:52:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2023-12-26T22:52:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2023-12-26T22:52:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.790531+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:00:23.547354+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T22:52:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1a is not active+clean 2023-12-26T22:52:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2023-12-26T22:52:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.790084+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:34:45.392051+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1b is not active+clean 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.787558+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:07:24.409370+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.18 is not active+clean 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.789746+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T04:52:43.991067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.19 is not active+clean 2023-12-26T22:52:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.18 is not active+clean 2023-12-26T22:52:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.790069+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:05:29.330660+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T22:52:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.16 is not active+clean 2023-12-26T22:52:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.17 is not active+clean 2023-12-26T22:52:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.791013+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:05:31.638926+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T22:52:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.17 is not active+clean 2023-12-26T22:52:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2023-12-26T22:52:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.787570+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:06:21.026051+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T22:52:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2023-12-26T22:52:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2023-12-26T22:52:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.793326+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:13:28.315292+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T22:52:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2023-12-26T22:52:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2023-12-26T22:52:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.789952+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:24:30.781539+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T22:52:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.13 is not active+clean 2023-12-26T22:52:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.12 is not active+clean 2023-12-26T22:52:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.906969+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:47:17.114623+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T22:52:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.10 is not active+clean 2023-12-26T22:52:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2023-12-26T22:52:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.793875+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:05:31.638926+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T22:52:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.11 is not active+clean 2023-12-26T22:52:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.10 is not active+clean 2023-12-26T22:52:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.794441+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T07:11:46.085192+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T22:52:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.e is not active+clean 2023-12-26T22:52:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2023-12-26T22:52:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.790533+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:03:39.067839+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T22:52:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.f is not active+clean 2023-12-26T22:52:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.793394+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:03:39.067839+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.c is not active+clean 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.789816+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:53:30.704146+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.d is not active+clean 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2023-12-26T22:52:45.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.794553+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:08:55.020611+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T22:52:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.a is not active+clean 2023-12-26T22:52:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.b is not active+clean 2023-12-26T22:52:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.789888+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:08:28.814418+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T22:52:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.b is not active+clean 2023-12-26T22:52:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2023-12-26T22:52:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.786936+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:36:26.953614+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T22:52:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.8 is not active+clean 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.906938+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:49:35.313337+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.793253+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:22:22.603851+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.6 is not active+clean 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.7 is not active+clean 2023-12-26T22:52:45.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.787474+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T03:32:44.835523+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T22:52:45.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.7 is not active+clean 2023-12-26T22:52:45.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2023-12-26T22:52:45.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.794862+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:26:06.257273+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T22:52:45.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.4 is not active+clean 2023-12-26T22:52:45.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.5 is not active+clean 2023-12-26T22:52:45.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.787248+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:14:31.472805+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.5 is not active+clean 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.906902+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:11:49.316037+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.790637+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:07:06.765554+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3 is not active+clean 2023-12-26T22:52:45.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.791485+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:52:34.546507+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.0 is not active+clean 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.793323+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:42:21.268550+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1 is not active+clean 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2023-12-26T22:52:45.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.906899+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:12:18.480091+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T22:52:45.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2023-12-26T22:52:45.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T22:52:38.783519+0000', 'last_change': '2023-12-26T22:52:38.783519+0000', 'last_active': '2023-12-26T22:52:38.783519+0000', 'last_peered': '2023-12-26T22:52:38.783519+0000', 'last_clean': '2023-12-26T22:52:38.783519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:38.783519+0000', 'last_undegraded': '2023-12-26T22:52:38.783519+0000', 'last_fullsized': '2023-12-26T22:52:38.783519+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:38.783519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T22:52:45.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2023-12-26T22:52:45.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T22:52:37.777351+0000', 'last_change': '2023-12-26T22:52:37.794953+0000', 'last_active': '2023-12-26T22:52:37.777351+0000', 'last_peered': '2023-12-26T22:52:37.777351+0000', 'last_clean': '2023-12-26T22:52:37.777351+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T22:52:37.777351+0000', 'last_undegraded': '2023-12-26T22:52:37.777351+0000', 'last_fullsized': '2023-12-26T22:52:37.777351+0000', 'mapping_epoch': 158, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 158, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'last_clean_scrub_stamp': '2023-12-26T22:52:37.777351+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:45:57.359936+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T22:52:45.115 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-26T22:52:45.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.317+0000 7faabbbff700 1 Processor -- start 2023-12-26T22:52:45.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.317+0000 7faabbbff700 1 -- start start 2023-12-26T22:52:45.319 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.318+0000 7faabbbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faab41596a0 0x7faab40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.318+0000 7faabbbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab40afcd0 con 0x7faab4159a70 2023-12-26T22:52:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.318+0000 7faabbbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab40b3e70 con 0x7faab40b0870 2023-12-26T22:52:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.318+0000 7faabbbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab40b0cb0 con 0x7faab41596a0 2023-12-26T22:52:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.318+0000 7faab999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faab41596a0 0x7faab40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.318+0000 7faab919a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7faab4159a70 0x7faab40b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41026/0 (socket says 172.21.15.131:41026) 2023-12-26T22:52:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.318+0000 7faab919a700 1 -- 172.21.15.131:0/573554212 learned_addr learned my addr 172.21.15.131:0/573554212 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.318+0000 7faab999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faab41596a0 0x7faab40b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35550/0 (socket says 172.21.15.131:35550) 2023-12-26T22:52:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.319+0000 7faab999b700 1 -- 172.21.15.131:0/573554212 >> v1:172.21.15.190:6789/0 conn(0x7faab40b0870 legacy=0x7faab40b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.319+0000 7faab999b700 1 -- 172.21.15.131:0/573554212 >> v1:172.21.15.131:6789/0 conn(0x7faab4159a70 legacy=0x7faab40b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.319+0000 7faab999b700 1 -- 172.21.15.131:0/573554212 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab40b0f20 con 0x7faab41596a0 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.319+0000 7faab8999700 1 -- 172.21.15.131:0/573554212 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784525701 0 0) 0x7faab40afcd0 con 0x7faab4159a70 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.319+0000 7faab999b700 1 --2- 172.21.15.131:0/573554212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 0x7faab40b2380 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7faab00098e0 tx=0x7faab00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a73232a1049121c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.319+0000 7faab8999700 1 -- 172.21.15.131:0/573554212 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0009e80 con 0x7faab41596a0 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.319+0000 7faab8999700 1 -- 172.21.15.131:0/573554212 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab000b820 con 0x7faab41596a0 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.319+0000 7faab8999700 1 -- 172.21.15.131:0/573554212 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000bba0 con 0x7faab41596a0 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.320+0000 7faabbbff700 1 -- 172.21.15.131:0/573554212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 msgr2=0x7faab40b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.320+0000 7faabbbff700 1 --2- 172.21.15.131:0/573554212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 0x7faab40b2380 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7faab00098e0 tx=0x7faab00091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.320+0000 7faabbbff700 1 -- 172.21.15.131:0/573554212 shutdown_connections 2023-12-26T22:52:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.320+0000 7faabbbff700 1 --2- 172.21.15.131:0/573554212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 0x7faab40b2380 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.320+0000 7faabbbff700 1 -- 172.21.15.131:0/573554212 >> 172.21.15.131:0/573554212 conn(0x7faab406ece0 msgr2=0x7faab406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.320+0000 7faabbbff700 1 -- 172.21.15.131:0/573554212 shutdown_connections 2023-12-26T22:52:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.320+0000 7faabbbff700 1 -- 172.21.15.131:0/573554212 wait complete. 2023-12-26T22:52:45.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.321+0000 7faabbbff700 1 Processor -- start 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.321+0000 7faabbbff700 1 -- start start 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.322+0000 7faabbbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faab40b0870 0x7faab4155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.322+0000 7faabbbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 0x7faab415a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.322+0000 7faabbbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab4159a70 0x7faab40b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.322+0000 7faabbbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab40c9b40 con 0x7faab41596a0 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.322+0000 7faabbbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab4156430 con 0x7faab4159a70 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.322+0000 7faabbbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab415ad60 con 0x7faab40b0870 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab919a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 0x7faab415a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab919a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 0x7faab415a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35562/0 (socket says 172.21.15.131:35562) 2023-12-26T22:52:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab919a700 1 -- 172.21.15.131:0/89861621 learned_addr learned my addr 172.21.15.131:0/89861621 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab999b700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faab40b0870 0x7faab4155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faaba19c700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab4159a70 0x7faab40b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab919a700 1 -- 172.21.15.131:0/89861621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faab40b0870 msgr2=0x7faab4155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab919a700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faab40b0870 0x7faab4155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab919a700 1 -- 172.21.15.131:0/89861621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab4159a70 msgr2=0x7faab40b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab919a700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab4159a70 0x7faab40b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab919a700 1 -- 172.21.15.131:0/89861621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0009420 con 0x7faab41596a0 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faab919a700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 0x7faab415a820 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c7e0 tx=0x7faaa4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faaa9ffb700 1 -- 172.21.15.131:0/89861621 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4005b30 con 0x7faab41596a0 2023-12-26T22:52:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faaa9ffb700 1 -- 172.21.15.131:0/89861621 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa4005c90 con 0x7faab41596a0 2023-12-26T22:52:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab415b6c0 con 0x7faab41596a0 2023-12-26T22:52:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.323+0000 7faaa9ffb700 1 -- 172.21.15.131:0/89861621 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4007300 con 0x7faab41596a0 2023-12-26T22:52:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.324+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab415bc80 con 0x7faab41596a0 2023-12-26T22:52:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.325+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab40657e0 con 0x7faab41596a0 2023-12-26T22:52:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.325+0000 7faaa9ffb700 1 -- 172.21.15.131:0/89861621 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faaa40076b0 con 0x7faab41596a0 2023-12-26T22:52:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.327+0000 7faaa9ffb700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa98066f30 0x7faa980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.327+0000 7faaa9ffb700 1 -- 172.21.15.131:0/89861621 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7faaa4083b80 con 0x7faab41596a0 2023-12-26T22:52:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.327+0000 7faab999b700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa98066f30 0x7faa980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.328+0000 7faab999b700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa98066f30 0x7faa980693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faab00030c0 tx=0x7faab0016000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:45.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.331+0000 7faaa9ffb700 1 -- 172.21.15.131:0/89861621 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faaa4050c20 con 0x7faab41596a0 2023-12-26T22:52:45.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.597+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7faab40b3540 con 0x7faab41596a0 2023-12-26T22:52:45.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.598+0000 7faaa9ffb700 1 -- 172.21.15.131:0/89861621 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1114 (secure 0 0 0) 0x7faaa4017470 con 0x7faab41596a0 2023-12-26T22:52:45.598 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:45.598 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"ab603302-d88b-429a-be64-92b3c298adad","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4581,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":161,"num_osds":8,"num_up_osds":8,"osd_up_since":1703626596,"num_in_osds":8,"osd_in_since":1703626587,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2519343104,"bytes_avail":770574770176,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":73,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":8628}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-12-26T22:42:47.352505+0000","services":{}},"progress_events":{}} 2023-12-26T22:52:45.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa98066f30 msgr2=0x7faa980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa98066f30 0x7faa980693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faab00030c0 tx=0x7faab0016000 comp rx=0 tx=0).stop 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 msgr2=0x7faab415a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 0x7faab415a820 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c7e0 tx=0x7faaa4007df0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 shutdown_connections 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faab4159a70 0x7faab40b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faab40b0870 0x7faab4155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faa98066f30 0x7faa980693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 --2- 172.21.15.131:0/89861621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faab41596a0 0x7faab415a820 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 >> 172.21.15.131:0/89861621 conn(0x7faab406ece0 msgr2=0x7faab40cd210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 shutdown_connections 2023-12-26T22:52:45.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.600+0000 7faabbbff700 1 -- 172.21.15.131:0/89861621 wait complete. 2023-12-26T22:52:45.617 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-26T22:52:45.618 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T22:52:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.809+0000 7f1b39bff700 1 Processor -- start 2023-12-26T22:52:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.809+0000 7f1b39bff700 1 -- start start 2023-12-26T22:52:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.810+0000 7f1b39bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1b34153bd0 0x7f1b340b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.810+0000 7f1b39bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b340b1c10 0x7f1b34068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.810+0000 7f1b39bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b3415d250 con 0x7f1b340b3b70 2023-12-26T22:52:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.810+0000 7f1b39bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b3415e260 con 0x7f1b340b1c10 2023-12-26T22:52:45.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.810+0000 7f1b39bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b340b2070 con 0x7f1b34153bd0 2023-12-26T22:52:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.810+0000 7f1b33fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b340b1c10 0x7f1b34068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.810+0000 7f1b33fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b340b1c10 0x7f1b34068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35578/0 (socket says 172.21.15.131:35578) 2023-12-26T22:52:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.810+0000 7f1b33fff700 1 -- 172.21.15.131:0/2917426590 learned_addr learned my addr 172.21.15.131:0/2917426590 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.810+0000 7f1b32ffd700 1 --1- 172.21.15.131:0/2917426590 >> v1:172.21.15.131:6789/0 conn(0x7f1b340b3b70 0x7f1b340b1500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41030/0 (socket says 172.21.15.131:41030) 2023-12-26T22:52:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b337fe700 1 --2- 172.21.15.131:0/2917426590 >> v2:172.21.15.190:3300/0 conn(0x7f1b34153bd0 0x7f1b340b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b33fff700 1 -- 172.21.15.131:0/2917426590 >> v2:172.21.15.190:3300/0 conn(0x7f1b34153bd0 msgr2=0x7f1b340b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b33fff700 1 --2- 172.21.15.131:0/2917426590 >> v2:172.21.15.190:3300/0 conn(0x7f1b34153bd0 0x7f1b340b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b33fff700 1 -- 172.21.15.131:0/2917426590 >> v1:172.21.15.131:6789/0 conn(0x7f1b340b3b70 legacy=0x7f1b340b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b33fff700 1 -- 172.21.15.131:0/2917426590 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3415d4a0 con 0x7f1b340b1c10 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b337fe700 1 --2- 172.21.15.131:0/2917426590 >> v2:172.21.15.190:3300/0 conn(0x7f1b34153bd0 0x7f1b340b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b327fc700 1 -- 172.21.15.131:0/2917426590 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668080069 0 0) 0x7f1b3415d250 con 0x7f1b340b3b70 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b33fff700 1 --2- 172.21.15.131:0/2917426590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b1c10 0x7f1b34068920 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f1b3406f150 tx=0x7f1b2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675339a1748dff6a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b327fc700 1 -- 172.21.15.131:0/2917426590 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00e070 con 0x7f1b340b1c10 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b327fc700 1 -- 172.21.15.131:0/2917426590 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c00ba40 con 0x7f1b340b1c10 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.811+0000 7f1b327fc700 1 -- 172.21.15.131:0/2917426590 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0092e0 con 0x7f1b340b1c10 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.812+0000 7f1b39bff700 1 -- 172.21.15.131:0/2917426590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b1c10 msgr2=0x7f1b34068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.812+0000 7f1b39bff700 1 --2- 172.21.15.131:0/2917426590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b1c10 0x7f1b34068920 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f1b3406f150 tx=0x7f1b2c009840 comp rx=0 tx=0).stop 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.812+0000 7f1b39bff700 1 -- 172.21.15.131:0/2917426590 shutdown_connections 2023-12-26T22:52:45.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.812+0000 7f1b39bff700 1 --2- 172.21.15.131:0/2917426590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b1c10 0x7f1b34068920 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.812+0000 7f1b39bff700 1 --2- 172.21.15.131:0/2917426590 >> v2:172.21.15.190:3300/0 conn(0x7f1b34153bd0 0x7f1b340b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.812+0000 7f1b39bff700 1 -- 172.21.15.131:0/2917426590 >> 172.21.15.131:0/2917426590 conn(0x7f1b34063d00 msgr2=0x7f1b3406b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:45.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.812+0000 7f1b39bff700 1 -- 172.21.15.131:0/2917426590 shutdown_connections 2023-12-26T22:52:45.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.812+0000 7f1b39bff700 1 -- 172.21.15.131:0/2917426590 wait complete. 2023-12-26T22:52:45.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.814+0000 7f1b39bff700 1 Processor -- start 2023-12-26T22:52:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.814+0000 7f1b39bff700 1 -- start start 2023-12-26T22:52:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.814+0000 7f1b39bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340b1c10 0x7f1b3413be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b39bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b3b70 0x7f1b3413afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b39bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b34153bd0 0x7f1b3413b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b39bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3414a340 con 0x7f1b340b3b70 2023-12-26T22:52:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b39bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b34139f20 con 0x7f1b340b1c10 2023-12-26T22:52:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b39bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3413a200 con 0x7f1b34153bd0 2023-12-26T22:52:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b32ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b3b70 0x7f1b3413afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b32ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b3b70 0x7f1b3413afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35592/0 (socket says 172.21.15.131:35592) 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b32ffd700 1 -- 172.21.15.131:0/3439973196 learned_addr learned my addr 172.21.15.131:0/3439973196 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b337fe700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340b1c10 0x7f1b3413be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b33fff700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b34153bd0 0x7f1b3413b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b32ffd700 1 -- 172.21.15.131:0/3439973196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b34153bd0 msgr2=0x7f1b3413b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b32ffd700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b34153bd0 0x7f1b3413b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b32ffd700 1 -- 172.21.15.131:0/3439973196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340b1c10 msgr2=0x7f1b3413be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b32ffd700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340b1c10 0x7f1b3413be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b32ffd700 1 -- 172.21.15.131:0/3439973196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c009ae0 con 0x7f1b340b3b70 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.815+0000 7f1b32ffd700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b3b70 0x7f1b3413afc0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1b280011e0 tx=0x7f1b28007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.816+0000 7f1b1b7fe700 1 -- 172.21.15.131:0/3439973196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28005a70 con 0x7f1b340b3b70 2023-12-26T22:52:45.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.816+0000 7f1b1b7fe700 1 -- 172.21.15.131:0/3439973196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b28005bd0 con 0x7f1b340b3b70 2023-12-26T22:52:45.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.816+0000 7f1b1b7fe700 1 -- 172.21.15.131:0/3439973196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b280070d0 con 0x7f1b340b3b70 2023-12-26T22:52:45.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.816+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3413a4e0 con 0x7f1b340b3b70 2023-12-26T22:52:45.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.816+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3413aaa0 con 0x7f1b340b3b70 2023-12-26T22:52:45.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.817+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3413acc0 con 0x7f1b340b3b70 2023-12-26T22:52:45.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.818+0000 7f1b1b7fe700 1 -- 172.21.15.131:0/3439973196 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1b28007480 con 0x7f1b340b3b70 2023-12-26T22:52:45.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.819+0000 7f1b1b7fe700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b20066f30 0x7f1b200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:45.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.819+0000 7f1b1b7fe700 1 -- 172.21.15.131:0/3439973196 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f1b28083bb0 con 0x7f1b340b3b70 2023-12-26T22:52:45.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.819+0000 7f1b337fe700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b20066f30 0x7f1b200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:45.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.820+0000 7f1b337fe700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b20066f30 0x7f1b200693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b24006f00 tx=0x7f1b24008040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:45.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:45.823+0000 7f1b1b7fe700 1 -- 172.21.15.131:0/3439973196 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b28050c50 con 0x7f1b340b3b70 2023-12-26T22:52:46.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.012+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b340657e0 con 0x7f1b20066f30 2023-12-26T22:52:46.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.021+0000 7f1b1b7fe700 1 -- 172.21.15.131:0/3439973196 <== mgr.8436 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289470 (secure 0 0 0) 0x7f1b340657e0 con 0x7f1b20066f30 2023-12-26T22:52:46.025 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:46.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b20066f30 msgr2=0x7f1b200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b20066f30 0x7f1b200693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b24006f00 tx=0x7f1b24008040 comp rx=0 tx=0).stop 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b3b70 msgr2=0x7f1b3413afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b3b70 0x7f1b3413afc0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1b280011e0 tx=0x7f1b28007e10 comp rx=0 tx=0).stop 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 shutdown_connections 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b34153bd0 0x7f1b3413b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b340b1c10 0x7f1b3413be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1b20066f30 0x7f1b200693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 --2- 172.21.15.131:0/3439973196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b340b3b70 0x7f1b3413afc0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 >> 172.21.15.131:0/3439973196 conn(0x7f1b34063d00 msgr2=0x7f1b341511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 shutdown_connections 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:46.025+0000 7f1b39bff700 1 -- 172.21.15.131:0/3439973196 wait complete. 2023-12-26T22:52:46.028 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T22:52:46.042 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-26T22:52:45.756687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464744,"kb_used_data":16808,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752509976,"statfs":{"total":773094113280,"available":770570215424,"internally_reserved":0,"allocated":17211392,"data_stored":9121204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.966396"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839246+0000","last_change":"2023-12-26T22:52:39.835959+0000","last_active":"2023-12-26T22:52:39.839246+0000","last_peered":"2023-12-26T22:52:39.839246+0000","last_clean":"2023-12-26T22:52:39.839246+0000","last_became_active":"2023-12-26T22:52:39.835701+0000","last_became_peered":"2023-12-26T22:52:39.835701+0000","last_unstale":"2023-12-26T22:52:39.839246+0000","last_undegraded":"2023-12-26T22:52:39.839246+0000","last_fullsized":"2023-12-26T22:52:39.839246+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:04:34.951035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829859+0000","last_change":"2023-12-26T22:52:39.825281+0000","last_active":"2023-12-26T22:52:39.829859+0000","last_peered":"2023-12-26T22:52:39.829859+0000","last_clean":"2023-12-26T22:52:39.829859+0000","last_became_active":"2023-12-26T22:52:39.824828+0000","last_became_peered":"2023-12-26T22:52:39.824828+0000","last_unstale":"2023-12-26T22:52:39.829859+0000","last_undegraded":"2023-12-26T22:52:39.829859+0000","last_fullsized":"2023-12-26T22:52:39.829859+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:58:39.999445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.834845+0000","last_change":"2023-12-26T22:52:39.830474+0000","last_active":"2023-12-26T22:52:39.834845+0000","last_peered":"2023-12-26T22:52:39.834845+0000","last_clean":"2023-12-26T22:52:39.834845+0000","last_became_active":"2023-12-26T22:52:39.830201+0000","last_became_peered":"2023-12-26T22:52:39.830201+0000","last_unstale":"2023-12-26T22:52:39.834845+0000","last_undegraded":"2023-12-26T22:52:39.834845+0000","last_fullsized":"2023-12-26T22:52:39.834845+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:31:16.761360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.842530+0000","last_change":"2023-12-26T22:52:39.837252+0000","last_active":"2023-12-26T22:52:39.842530+0000","last_peered":"2023-12-26T22:52:39.842530+0000","last_clean":"2023-12-26T22:52:39.842530+0000","last_became_active":"2023-12-26T22:52:39.836849+0000","last_became_peered":"2023-12-26T22:52:39.836849+0000","last_unstale":"2023-12-26T22:52:39.842530+0000","last_undegraded":"2023-12-26T22:52:39.842530+0000","last_fullsized":"2023-12-26T22:52:39.842530+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:39:42.771599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.844550+0000","last_change":"2023-12-26T22:52:39.838540+0000","last_active":"2023-12-26T22:52:39.844550+0000","last_peered":"2023-12-26T22:52:39.844550+0000","last_clean":"2023-12-26T22:52:39.844550+0000","last_became_active":"2023-12-26T22:52:39.838299+0000","last_became_peered":"2023-12-26T22:52:39.838299+0000","last_unstale":"2023-12-26T22:52:39.844550+0000","last_undegraded":"2023-12-26T22:52:39.844550+0000","last_fullsized":"2023-12-26T22:52:39.844550+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:20:35.952591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839595+0000","last_change":"2023-12-26T22:52:39.836133+0000","last_active":"2023-12-26T22:52:39.839595+0000","last_peered":"2023-12-26T22:52:39.839595+0000","last_clean":"2023-12-26T22:52:39.839595+0000","last_became_active":"2023-12-26T22:52:39.835795+0000","last_became_peered":"2023-12-26T22:52:39.835795+0000","last_unstale":"2023-12-26T22:52:39.839595+0000","last_undegraded":"2023-12-26T22:52:39.839595+0000","last_fullsized":"2023-12-26T22:52:39.839595+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:56:24.308560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827733+0000","last_change":"2023-12-26T22:52:39.824058+0000","last_active":"2023-12-26T22:52:39.827733+0000","last_peered":"2023-12-26T22:52:39.827733+0000","last_clean":"2023-12-26T22:52:39.827733+0000","last_became_active":"2023-12-26T22:52:39.823758+0000","last_became_peered":"2023-12-26T22:52:39.823758+0000","last_unstale":"2023-12-26T22:52:39.827733+0000","last_undegraded":"2023-12-26T22:52:39.827733+0000","last_fullsized":"2023-12-26T22:52:39.827733+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:08:09.135608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.822165+0000","last_change":"2023-12-26T22:52:39.818061+0000","last_active":"2023-12-26T22:52:39.822165+0000","last_peered":"2023-12-26T22:52:39.822165+0000","last_clean":"2023-12-26T22:52:39.822165+0000","last_became_active":"2023-12-26T22:52:39.817784+0000","last_became_peered":"2023-12-26T22:52:39.817784+0000","last_unstale":"2023-12-26T22:52:39.822165+0000","last_undegraded":"2023-12-26T22:52:39.822165+0000","last_fullsized":"2023-12-26T22:52:39.822165+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:52:15.221910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835113+0000","last_change":"2023-12-26T22:52:39.831201+0000","last_active":"2023-12-26T22:52:39.835113+0000","last_peered":"2023-12-26T22:52:39.835113+0000","last_clean":"2023-12-26T22:52:39.835113+0000","last_became_active":"2023-12-26T22:52:39.830972+0000","last_became_peered":"2023-12-26T22:52:39.830972+0000","last_unstale":"2023-12-26T22:52:39.835113+0000","last_undegraded":"2023-12-26T22:52:39.835113+0000","last_fullsized":"2023-12-26T22:52:39.835113+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:39:23.078653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839274+0000","last_change":"2023-12-26T22:52:39.836097+0000","last_active":"2023-12-26T22:52:39.839274+0000","last_peered":"2023-12-26T22:52:39.839274+0000","last_clean":"2023-12-26T22:52:39.839274+0000","last_became_active":"2023-12-26T22:52:39.835650+0000","last_became_peered":"2023-12-26T22:52:39.835650+0000","last_unstale":"2023-12-26T22:52:39.839274+0000","last_undegraded":"2023-12-26T22:52:39.839274+0000","last_fullsized":"2023-12-26T22:52:39.839274+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:08:09.135608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.825845+0000","last_change":"2023-12-26T22:52:39.820024+0000","last_active":"2023-12-26T22:52:39.825845+0000","last_peered":"2023-12-26T22:52:39.825845+0000","last_clean":"2023-12-26T22:52:39.825845+0000","last_became_active":"2023-12-26T22:52:39.819743+0000","last_became_peered":"2023-12-26T22:52:39.819743+0000","last_unstale":"2023-12-26T22:52:39.825845+0000","last_undegraded":"2023-12-26T22:52:39.825845+0000","last_fullsized":"2023-12-26T22:52:39.825845+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:16:14.117868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.840318+0000","last_change":"2023-12-26T22:52:39.836477+0000","last_active":"2023-12-26T22:52:39.840318+0000","last_peered":"2023-12-26T22:52:39.840318+0000","last_clean":"2023-12-26T22:52:39.840318+0000","last_became_active":"2023-12-26T22:52:39.834537+0000","last_became_peered":"2023-12-26T22:52:39.834537+0000","last_unstale":"2023-12-26T22:52:39.840318+0000","last_undegraded":"2023-12-26T22:52:39.840318+0000","last_fullsized":"2023-12-26T22:52:39.840318+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:19:49.658340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827869+0000","last_change":"2023-12-26T22:52:39.823718+0000","last_active":"2023-12-26T22:52:39.827869+0000","last_peered":"2023-12-26T22:52:39.827869+0000","last_clean":"2023-12-26T22:52:39.827869+0000","last_became_active":"2023-12-26T22:52:39.821803+0000","last_became_peered":"2023-12-26T22:52:39.821803+0000","last_unstale":"2023-12-26T22:52:39.827869+0000","last_undegraded":"2023-12-26T22:52:39.827869+0000","last_fullsized":"2023-12-26T22:52:39.827869+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:32:48.632110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830026+0000","last_change":"2023-12-26T22:52:39.824548+0000","last_active":"2023-12-26T22:52:39.830026+0000","last_peered":"2023-12-26T22:52:39.830026+0000","last_clean":"2023-12-26T22:52:39.830026+0000","last_became_active":"2023-12-26T22:52:39.824279+0000","last_became_peered":"2023-12-26T22:52:39.824279+0000","last_unstale":"2023-12-26T22:52:39.830026+0000","last_undegraded":"2023-12-26T22:52:39.830026+0000","last_fullsized":"2023-12-26T22:52:39.830026+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:42:22.274717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830409+0000","last_change":"2023-12-26T22:52:39.823317+0000","last_active":"2023-12-26T22:52:39.830409+0000","last_peered":"2023-12-26T22:52:39.830409+0000","last_clean":"2023-12-26T22:52:39.830409+0000","last_became_active":"2023-12-26T22:52:39.823023+0000","last_became_peered":"2023-12-26T22:52:39.823023+0000","last_unstale":"2023-12-26T22:52:39.830409+0000","last_undegraded":"2023-12-26T22:52:39.830409+0000","last_fullsized":"2023-12-26T22:52:39.830409+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:55:10.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831122+0000","last_change":"2023-12-26T22:52:39.829027+0000","last_active":"2023-12-26T22:52:39.831122+0000","last_peered":"2023-12-26T22:52:39.831122+0000","last_clean":"2023-12-26T22:52:39.831122+0000","last_became_active":"2023-12-26T22:52:39.828259+0000","last_became_peered":"2023-12-26T22:52:39.828259+0000","last_unstale":"2023-12-26T22:52:39.831122+0000","last_undegraded":"2023-12-26T22:52:39.831122+0000","last_fullsized":"2023-12-26T22:52:39.831122+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:17.014057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.834941+0000","last_change":"2023-12-26T22:52:39.830730+0000","last_active":"2023-12-26T22:52:39.834941+0000","last_peered":"2023-12-26T22:52:39.834941+0000","last_clean":"2023-12-26T22:52:39.834941+0000","last_became_active":"2023-12-26T22:52:39.830490+0000","last_became_peered":"2023-12-26T22:52:39.830490+0000","last_unstale":"2023-12-26T22:52:39.834941+0000","last_undegraded":"2023-12-26T22:52:39.834941+0000","last_fullsized":"2023-12-26T22:52:39.834941+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:15:04.617894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830420+0000","last_change":"2023-12-26T22:52:39.823248+0000","last_active":"2023-12-26T22:52:39.830420+0000","last_peered":"2023-12-26T22:52:39.830420+0000","last_clean":"2023-12-26T22:52:39.830420+0000","last_became_active":"2023-12-26T22:52:39.822871+0000","last_became_peered":"2023-12-26T22:52:39.822871+0000","last_unstale":"2023-12-26T22:52:39.830420+0000","last_undegraded":"2023-12-26T22:52:39.830420+0000","last_fullsized":"2023-12-26T22:52:39.830420+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:53.548137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827280+0000","last_change":"2023-12-26T22:52:39.821141+0000","last_active":"2023-12-26T22:52:39.827280+0000","last_peered":"2023-12-26T22:52:39.827280+0000","last_clean":"2023-12-26T22:52:39.827280+0000","last_became_active":"2023-12-26T22:52:39.820850+0000","last_became_peered":"2023-12-26T22:52:39.820850+0000","last_unstale":"2023-12-26T22:52:39.827280+0000","last_undegraded":"2023-12-26T22:52:39.827280+0000","last_fullsized":"2023-12-26T22:52:39.827280+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:55:10.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830521+0000","last_change":"2023-12-26T22:52:39.823571+0000","last_active":"2023-12-26T22:52:39.830521+0000","last_peered":"2023-12-26T22:52:39.830521+0000","last_clean":"2023-12-26T22:52:39.830521+0000","last_became_active":"2023-12-26T22:52:39.823275+0000","last_became_peered":"2023-12-26T22:52:39.823275+0000","last_unstale":"2023-12-26T22:52:39.830521+0000","last_undegraded":"2023-12-26T22:52:39.830521+0000","last_fullsized":"2023-12-26T22:52:39.830521+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:01.162816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835865+0000","last_change":"2023-12-26T22:52:39.829908+0000","last_active":"2023-12-26T22:52:39.835865+0000","last_peered":"2023-12-26T22:52:39.835865+0000","last_clean":"2023-12-26T22:52:39.835865+0000","last_became_active":"2023-12-26T22:52:39.829435+0000","last_became_peered":"2023-12-26T22:52:39.829435+0000","last_unstale":"2023-12-26T22:52:39.835865+0000","last_undegraded":"2023-12-26T22:52:39.835865+0000","last_fullsized":"2023-12-26T22:52:39.835865+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:04.626344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831087+0000","last_change":"2023-12-26T22:52:39.827559+0000","last_active":"2023-12-26T22:52:39.831087+0000","last_peered":"2023-12-26T22:52:39.831087+0000","last_clean":"2023-12-26T22:52:39.831087+0000","last_became_active":"2023-12-26T22:52:39.827269+0000","last_became_peered":"2023-12-26T22:52:39.827269+0000","last_unstale":"2023-12-26T22:52:39.831087+0000","last_undegraded":"2023-12-26T22:52:39.831087+0000","last_fullsized":"2023-12-26T22:52:39.831087+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:42:15.920688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.834921+0000","last_change":"2023-12-26T22:52:39.830038+0000","last_active":"2023-12-26T22:52:39.834921+0000","last_peered":"2023-12-26T22:52:39.834921+0000","last_clean":"2023-12-26T22:52:39.834921+0000","last_became_active":"2023-12-26T22:52:39.829684+0000","last_became_peered":"2023-12-26T22:52:39.829684+0000","last_unstale":"2023-12-26T22:52:39.834921+0000","last_undegraded":"2023-12-26T22:52:39.834921+0000","last_fullsized":"2023-12-26T22:52:39.834921+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:40.074006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830492+0000","last_change":"2023-12-26T22:52:39.824345+0000","last_active":"2023-12-26T22:52:39.830492+0000","last_peered":"2023-12-26T22:52:39.830492+0000","last_clean":"2023-12-26T22:52:39.830492+0000","last_became_active":"2023-12-26T22:52:39.824092+0000","last_became_peered":"2023-12-26T22:52:39.824092+0000","last_unstale":"2023-12-26T22:52:39.830492+0000","last_undegraded":"2023-12-26T22:52:39.830492+0000","last_fullsized":"2023-12-26T22:52:39.830492+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:42:19.294384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.825944+0000","last_change":"2023-12-26T22:52:39.820236+0000","last_active":"2023-12-26T22:52:39.825944+0000","last_peered":"2023-12-26T22:52:39.825944+0000","last_clean":"2023-12-26T22:52:39.825944+0000","last_became_active":"2023-12-26T22:52:39.819800+0000","last_became_peered":"2023-12-26T22:52:39.819800+0000","last_unstale":"2023-12-26T22:52:39.825944+0000","last_undegraded":"2023-12-26T22:52:39.825944+0000","last_fullsized":"2023-12-26T22:52:39.825944+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:41:42.506695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816985+0000","last_change":"2023-12-26T22:52:39.813032+0000","last_active":"2023-12-26T22:52:39.816985+0000","last_peered":"2023-12-26T22:52:39.816985+0000","last_clean":"2023-12-26T22:52:39.816985+0000","last_became_active":"2023-12-26T22:52:39.812742+0000","last_became_peered":"2023-12-26T22:52:39.812742+0000","last_unstale":"2023-12-26T22:52:39.816985+0000","last_undegraded":"2023-12-26T22:52:39.816985+0000","last_fullsized":"2023-12-26T22:52:39.816985+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:34.964257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.842248+0000","last_change":"2023-12-26T22:52:39.838105+0000","last_active":"2023-12-26T22:52:39.842248+0000","last_peered":"2023-12-26T22:52:39.842248+0000","last_clean":"2023-12-26T22:52:39.842248+0000","last_became_active":"2023-12-26T22:52:39.837836+0000","last_became_peered":"2023-12-26T22:52:39.837836+0000","last_unstale":"2023-12-26T22:52:39.842248+0000","last_undegraded":"2023-12-26T22:52:39.842248+0000","last_fullsized":"2023-12-26T22:52:39.842248+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:40.074006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.828691+0000","last_change":"2023-12-26T22:52:39.823463+0000","last_active":"2023-12-26T22:52:39.828691+0000","last_peered":"2023-12-26T22:52:39.828691+0000","last_clean":"2023-12-26T22:52:39.828691+0000","last_became_active":"2023-12-26T22:52:39.820090+0000","last_became_peered":"2023-12-26T22:52:39.820090+0000","last_unstale":"2023-12-26T22:52:39.828691+0000","last_undegraded":"2023-12-26T22:52:39.828691+0000","last_fullsized":"2023-12-26T22:52:39.828691+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:22:23.610018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827737+0000","last_change":"2023-12-26T22:52:39.821258+0000","last_active":"2023-12-26T22:52:39.827737+0000","last_peered":"2023-12-26T22:52:39.827737+0000","last_clean":"2023-12-26T22:52:39.827737+0000","last_became_active":"2023-12-26T22:52:39.820952+0000","last_became_peered":"2023-12-26T22:52:39.820952+0000","last_unstale":"2023-12-26T22:52:39.827737+0000","last_undegraded":"2023-12-26T22:52:39.827737+0000","last_fullsized":"2023-12-26T22:52:39.827737+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:09:39.835895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829859+0000","last_change":"2023-12-26T22:52:39.821885+0000","last_active":"2023-12-26T22:52:39.829859+0000","last_peered":"2023-12-26T22:52:39.829859+0000","last_clean":"2023-12-26T22:52:39.829859+0000","last_became_active":"2023-12-26T22:52:39.821616+0000","last_became_peered":"2023-12-26T22:52:39.821616+0000","last_unstale":"2023-12-26T22:52:39.829859+0000","last_undegraded":"2023-12-26T22:52:39.829859+0000","last_fullsized":"2023-12-26T22:52:39.829859+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:48:29.597572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.820290+0000","last_change":"2023-12-26T22:52:39.815915+0000","last_active":"2023-12-26T22:52:39.820290+0000","last_peered":"2023-12-26T22:52:39.820290+0000","last_clean":"2023-12-26T22:52:39.820290+0000","last_became_active":"2023-12-26T22:52:39.815128+0000","last_became_peered":"2023-12-26T22:52:39.815128+0000","last_unstale":"2023-12-26T22:52:39.820290+0000","last_undegraded":"2023-12-26T22:52:39.820290+0000","last_fullsized":"2023-12-26T22:52:39.820290+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:03:42.705883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827905+0000","last_change":"2023-12-26T22:52:39.819345+0000","last_active":"2023-12-26T22:52:39.827905+0000","last_peered":"2023-12-26T22:52:39.827905+0000","last_clean":"2023-12-26T22:52:39.827905+0000","last_became_active":"2023-12-26T22:52:39.818164+0000","last_became_peered":"2023-12-26T22:52:39.818164+0000","last_unstale":"2023-12-26T22:52:39.827905+0000","last_undegraded":"2023-12-26T22:52:39.827905+0000","last_fullsized":"2023-12-26T22:52:39.827905+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:28.267144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830859+0000","last_change":"2023-12-26T22:52:39.828680+0000","last_active":"2023-12-26T22:52:39.830859+0000","last_peered":"2023-12-26T22:52:39.830859+0000","last_clean":"2023-12-26T22:52:39.830859+0000","last_became_active":"2023-12-26T22:52:39.828268+0000","last_became_peered":"2023-12-26T22:52:39.828268+0000","last_unstale":"2023-12-26T22:52:39.830859+0000","last_undegraded":"2023-12-26T22:52:39.830859+0000","last_fullsized":"2023-12-26T22:52:39.830859+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:29:59.360259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"160'4","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.843570+0000","last_change":"2023-12-26T22:52:38.806860+0000","last_active":"2023-12-26T22:52:39.843570+0000","last_peered":"2023-12-26T22:52:39.843570+0000","last_clean":"2023-12-26T22:52:39.843570+0000","last_became_active":"2023-12-26T22:52:38.806568+0000","last_became_peered":"2023-12-26T22:52:38.806568+0000","last_unstale":"2023-12-26T22:52:39.843570+0000","last_undegraded":"2023-12-26T22:52:39.843570+0000","last_fullsized":"2023-12-26T22:52:39.843570+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:33.958090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835179+0000","last_change":"2023-12-26T22:52:39.831602+0000","last_active":"2023-12-26T22:52:39.835179+0000","last_peered":"2023-12-26T22:52:39.835179+0000","last_clean":"2023-12-26T22:52:39.835179+0000","last_became_active":"2023-12-26T22:52:39.830106+0000","last_became_peered":"2023-12-26T22:52:39.830106+0000","last_unstale":"2023-12-26T22:52:39.835179+0000","last_undegraded":"2023-12-26T22:52:39.835179+0000","last_fullsized":"2023-12-26T22:52:39.835179+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:02:44.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.840698+0000","last_change":"2023-12-26T22:52:38.815087+0000","last_active":"2023-12-26T22:52:39.840698+0000","last_peered":"2023-12-26T22:52:39.840698+0000","last_clean":"2023-12-26T22:52:39.840698+0000","last_became_active":"2023-12-26T22:52:38.814815+0000","last_became_peered":"2023-12-26T22:52:38.814815+0000","last_unstale":"2023-12-26T22:52:39.840698+0000","last_undegraded":"2023-12-26T22:52:39.840698+0000","last_fullsized":"2023-12-26T22:52:39.840698+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:08:57.128212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835068+0000","last_change":"2023-12-26T22:52:39.832243+0000","last_active":"2023-12-26T22:52:39.835068+0000","last_peered":"2023-12-26T22:52:39.835068+0000","last_clean":"2023-12-26T22:52:39.835068+0000","last_became_active":"2023-12-26T22:52:39.831987+0000","last_became_peered":"2023-12-26T22:52:39.831987+0000","last_unstale":"2023-12-26T22:52:39.835068+0000","last_undegraded":"2023-12-26T22:52:39.835068+0000","last_fullsized":"2023-12-26T22:52:39.835068+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:45:54.661560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839828+0000","last_change":"2023-12-26T22:52:38.806358+0000","last_active":"2023-12-26T22:52:39.839828+0000","last_peered":"2023-12-26T22:52:39.839828+0000","last_clean":"2023-12-26T22:52:39.839828+0000","last_became_active":"2023-12-26T22:52:38.805477+0000","last_became_peered":"2023-12-26T22:52:38.805477+0000","last_unstale":"2023-12-26T22:52:39.839828+0000","last_undegraded":"2023-12-26T22:52:39.839828+0000","last_fullsized":"2023-12-26T22:52:39.839828+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:11:46.085192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835560+0000","last_change":"2023-12-26T22:52:39.828732+0000","last_active":"2023-12-26T22:52:39.835560+0000","last_peered":"2023-12-26T22:52:39.835560+0000","last_clean":"2023-12-26T22:52:39.835560+0000","last_became_active":"2023-12-26T22:52:39.828494+0000","last_became_peered":"2023-12-26T22:52:39.828494+0000","last_unstale":"2023-12-26T22:52:39.835560+0000","last_undegraded":"2023-12-26T22:52:39.835560+0000","last_fullsized":"2023-12-26T22:52:39.835560+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:10:42.560357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839564+0000","last_change":"2023-12-26T22:52:38.808323+0000","last_active":"2023-12-26T22:52:39.839564+0000","last_peered":"2023-12-26T22:52:39.839564+0000","last_clean":"2023-12-26T22:52:39.839564+0000","last_became_active":"2023-12-26T22:52:38.808070+0000","last_became_peered":"2023-12-26T22:52:38.808070+0000","last_unstale":"2023-12-26T22:52:39.839564+0000","last_undegraded":"2023-12-26T22:52:39.839564+0000","last_fullsized":"2023-12-26T22:52:39.839564+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:33:16.614000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829088+0000","last_change":"2023-12-26T22:52:39.820509+0000","last_active":"2023-12-26T22:52:39.829088+0000","last_peered":"2023-12-26T22:52:39.829088+0000","last_clean":"2023-12-26T22:52:39.829088+0000","last_became_active":"2023-12-26T22:52:39.819503+0000","last_became_peered":"2023-12-26T22:52:39.819503+0000","last_unstale":"2023-12-26T22:52:39.829088+0000","last_undegraded":"2023-12-26T22:52:39.829088+0000","last_fullsized":"2023-12-26T22:52:39.829088+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:25:51.946981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835140+0000","last_change":"2023-12-26T22:52:38.810524+0000","last_active":"2023-12-26T22:52:39.835140+0000","last_peered":"2023-12-26T22:52:39.835140+0000","last_clean":"2023-12-26T22:52:39.835140+0000","last_became_active":"2023-12-26T22:52:38.810220+0000","last_became_peered":"2023-12-26T22:52:38.810220+0000","last_unstale":"2023-12-26T22:52:39.835140+0000","last_undegraded":"2023-12-26T22:52:39.835140+0000","last_fullsized":"2023-12-26T22:52:39.835140+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:27:09.195955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835848+0000","last_change":"2023-12-26T22:52:39.829725+0000","last_active":"2023-12-26T22:52:39.835848+0000","last_peered":"2023-12-26T22:52:39.835848+0000","last_clean":"2023-12-26T22:52:39.835848+0000","last_became_active":"2023-12-26T22:52:39.829140+0000","last_became_peered":"2023-12-26T22:52:39.829140+0000","last_unstale":"2023-12-26T22:52:39.835848+0000","last_undegraded":"2023-12-26T22:52:39.835848+0000","last_fullsized":"2023-12-26T22:52:39.835848+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:01.395499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.837404+0000","last_change":"2023-12-26T22:52:38.806633+0000","last_active":"2023-12-26T22:52:39.837404+0000","last_peered":"2023-12-26T22:52:39.837404+0000","last_clean":"2023-12-26T22:52:39.837404+0000","last_became_active":"2023-12-26T22:52:38.806290+0000","last_became_peered":"2023-12-26T22:52:38.806290+0000","last_unstale":"2023-12-26T22:52:39.837404+0000","last_undegraded":"2023-12-26T22:52:39.837404+0000","last_fullsized":"2023-12-26T22:52:39.837404+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:24:26.280738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835282+0000","last_change":"2023-12-26T22:52:39.825551+0000","last_active":"2023-12-26T22:52:39.835282+0000","last_peered":"2023-12-26T22:52:39.835282+0000","last_clean":"2023-12-26T22:52:39.835282+0000","last_became_active":"2023-12-26T22:52:39.825173+0000","last_became_peered":"2023-12-26T22:52:39.825173+0000","last_unstale":"2023-12-26T22:52:39.835282+0000","last_undegraded":"2023-12-26T22:52:39.835282+0000","last_fullsized":"2023-12-26T22:52:39.835282+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:39:03.541353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.817091+0000","last_change":"2023-12-26T22:52:38.803339+0000","last_active":"2023-12-26T22:52:39.817091+0000","last_peered":"2023-12-26T22:52:39.817091+0000","last_clean":"2023-12-26T22:52:39.817091+0000","last_became_active":"2023-12-26T22:52:38.803004+0000","last_became_peered":"2023-12-26T22:52:38.803004+0000","last_unstale":"2023-12-26T22:52:39.817091+0000","last_undegraded":"2023-12-26T22:52:39.817091+0000","last_fullsized":"2023-12-26T22:52:39.817091+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:18.261824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829181+0000","last_change":"2023-12-26T22:52:39.821697+0000","last_active":"2023-12-26T22:52:39.829181+0000","last_peered":"2023-12-26T22:52:39.829181+0000","last_clean":"2023-12-26T22:52:39.829181+0000","last_became_active":"2023-12-26T22:52:39.821425+0000","last_became_peered":"2023-12-26T22:52:39.821425+0000","last_unstale":"2023-12-26T22:52:39.829181+0000","last_undegraded":"2023-12-26T22:52:39.829181+0000","last_fullsized":"2023-12-26T22:52:39.829181+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:53.548137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.807541+0000","last_change":"2023-12-26T22:52:38.807376+0000","last_active":"2023-12-26T22:52:39.807541+0000","last_peered":"2023-12-26T22:52:39.807541+0000","last_clean":"2023-12-26T22:52:39.807541+0000","last_became_active":"2023-12-26T22:52:38.807095+0000","last_became_peered":"2023-12-26T22:52:38.807095+0000","last_unstale":"2023-12-26T22:52:39.807541+0000","last_undegraded":"2023-12-26T22:52:39.807541+0000","last_fullsized":"2023-12-26T22:52:39.807541+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:32.892492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829837+0000","last_change":"2023-12-26T22:52:39.825099+0000","last_active":"2023-12-26T22:52:39.829837+0000","last_peered":"2023-12-26T22:52:39.829837+0000","last_clean":"2023-12-26T22:52:39.829837+0000","last_became_active":"2023-12-26T22:52:39.824467+0000","last_became_peered":"2023-12-26T22:52:39.824467+0000","last_unstale":"2023-12-26T22:52:39.829837+0000","last_undegraded":"2023-12-26T22:52:39.829837+0000","last_fullsized":"2023-12-26T22:52:39.829837+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:11:00.599172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.837812+0000","last_change":"2023-12-26T22:52:38.814539+0000","last_active":"2023-12-26T22:52:39.837812+0000","last_peered":"2023-12-26T22:52:39.837812+0000","last_clean":"2023-12-26T22:52:39.837812+0000","last_became_active":"2023-12-26T22:52:38.814146+0000","last_became_peered":"2023-12-26T22:52:38.814146+0000","last_unstale":"2023-12-26T22:52:39.837812+0000","last_undegraded":"2023-12-26T22:52:39.837812+0000","last_fullsized":"2023-12-26T22:52:39.837812+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:25:12.341665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.832295+0000","last_change":"2023-12-26T22:52:39.826311+0000","last_active":"2023-12-26T22:52:39.832295+0000","last_peered":"2023-12-26T22:52:39.832295+0000","last_clean":"2023-12-26T22:52:39.832295+0000","last_became_active":"2023-12-26T22:52:39.825969+0000","last_became_peered":"2023-12-26T22:52:39.825969+0000","last_unstale":"2023-12-26T22:52:39.832295+0000","last_undegraded":"2023-12-26T22:52:39.832295+0000","last_fullsized":"2023-12-26T22:52:39.832295+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:04:34.951035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.810433+0000","last_change":"2023-12-26T22:52:38.812350+0000","last_active":"2023-12-26T22:52:39.810433+0000","last_peered":"2023-12-26T22:52:39.810433+0000","last_clean":"2023-12-26T22:52:39.810433+0000","last_became_active":"2023-12-26T22:52:38.811753+0000","last_became_peered":"2023-12-26T22:52:38.811753+0000","last_unstale":"2023-12-26T22:52:39.810433+0000","last_undegraded":"2023-12-26T22:52:39.810433+0000","last_fullsized":"2023-12-26T22:52:39.810433+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:33:21.821521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.836591+0000","last_change":"2023-12-26T22:52:39.830070+0000","last_active":"2023-12-26T22:52:39.836591+0000","last_peered":"2023-12-26T22:52:39.836591+0000","last_clean":"2023-12-26T22:52:39.836591+0000","last_became_active":"2023-12-26T22:52:39.828908+0000","last_became_peered":"2023-12-26T22:52:39.828908+0000","last_unstale":"2023-12-26T22:52:39.836591+0000","last_undegraded":"2023-12-26T22:52:39.836591+0000","last_fullsized":"2023-12-26T22:52:39.836591+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:57:48.698783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"160'3","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.844604+0000","last_change":"2023-12-26T22:52:38.810927+0000","last_active":"2023-12-26T22:52:39.844604+0000","last_peered":"2023-12-26T22:52:39.844604+0000","last_clean":"2023-12-26T22:52:39.844604+0000","last_became_active":"2023-12-26T22:52:38.810723+0000","last_became_peered":"2023-12-26T22:52:38.810723+0000","last_unstale":"2023-12-26T22:52:39.844604+0000","last_undegraded":"2023-12-26T22:52:39.844604+0000","last_fullsized":"2023-12-26T22:52:39.844604+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:48:28.591405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.841233+0000","last_change":"2023-12-26T22:52:39.839005+0000","last_active":"2023-12-26T22:52:39.841233+0000","last_peered":"2023-12-26T22:52:39.841233+0000","last_clean":"2023-12-26T22:52:39.841233+0000","last_became_active":"2023-12-26T22:52:39.838719+0000","last_became_peered":"2023-12-26T22:52:39.838719+0000","last_unstale":"2023-12-26T22:52:39.841233+0000","last_undegraded":"2023-12-26T22:52:39.841233+0000","last_fullsized":"2023-12-26T22:52:39.841233+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:05:32.645093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"160'6","reported_seq":16,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.845257+0000","last_change":"2023-12-26T22:52:38.814433+0000","last_active":"2023-12-26T22:52:39.845257+0000","last_peered":"2023-12-26T22:52:39.845257+0000","last_clean":"2023-12-26T22:52:39.845257+0000","last_became_active":"2023-12-26T22:52:38.814178+0000","last_became_peered":"2023-12-26T22:52:38.814178+0000","last_unstale":"2023-12-26T22:52:39.845257+0000","last_undegraded":"2023-12-26T22:52:39.845257+0000","last_fullsized":"2023-12-26T22:52:39.845257+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:04:43.464355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829686+0000","last_change":"2023-12-26T22:52:39.826438+0000","last_active":"2023-12-26T22:52:39.829686+0000","last_peered":"2023-12-26T22:52:39.829686+0000","last_clean":"2023-12-26T22:52:39.829686+0000","last_became_active":"2023-12-26T22:52:39.826179+0000","last_became_peered":"2023-12-26T22:52:39.826179+0000","last_unstale":"2023-12-26T22:52:39.829686+0000","last_undegraded":"2023-12-26T22:52:39.829686+0000","last_fullsized":"2023-12-26T22:52:39.829686+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:28:36.449693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816573+0000","last_change":"2023-12-26T22:52:38.813005+0000","last_active":"2023-12-26T22:52:39.816573+0000","last_peered":"2023-12-26T22:52:39.816573+0000","last_clean":"2023-12-26T22:52:39.816573+0000","last_became_active":"2023-12-26T22:52:38.812682+0000","last_became_peered":"2023-12-26T22:52:38.812682+0000","last_unstale":"2023-12-26T22:52:39.816573+0000","last_undegraded":"2023-12-26T22:52:39.816573+0000","last_fullsized":"2023-12-26T22:52:39.816573+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:41.915923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827204+0000","last_change":"2023-12-26T22:52:39.823437+0000","last_active":"2023-12-26T22:52:39.827204+0000","last_peered":"2023-12-26T22:52:39.827204+0000","last_clean":"2023-12-26T22:52:39.827204+0000","last_became_active":"2023-12-26T22:52:39.822757+0000","last_became_peered":"2023-12-26T22:52:39.822757+0000","last_unstale":"2023-12-26T22:52:39.827204+0000","last_undegraded":"2023-12-26T22:52:39.827204+0000","last_fullsized":"2023-12-26T22:52:39.827204+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:03:42.705883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816772+0000","last_change":"2023-12-26T22:52:38.814992+0000","last_active":"2023-12-26T22:52:39.816772+0000","last_peered":"2023-12-26T22:52:39.816772+0000","last_clean":"2023-12-26T22:52:39.816772+0000","last_became_active":"2023-12-26T22:52:38.814440+0000","last_became_peered":"2023-12-26T22:52:38.814440+0000","last_unstale":"2023-12-26T22:52:39.816772+0000","last_undegraded":"2023-12-26T22:52:39.816772+0000","last_fullsized":"2023-12-26T22:52:39.816772+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:57.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835524+0000","last_change":"2023-12-26T22:52:39.826671+0000","last_active":"2023-12-26T22:52:39.835524+0000","last_peered":"2023-12-26T22:52:39.835524+0000","last_clean":"2023-12-26T22:52:39.835524+0000","last_became_active":"2023-12-26T22:52:39.826266+0000","last_became_peered":"2023-12-26T22:52:39.826266+0000","last_unstale":"2023-12-26T22:52:39.835524+0000","last_undegraded":"2023-12-26T22:52:39.835524+0000","last_fullsized":"2023-12-26T22:52:39.835524+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:08:05.207585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"160'8","reported_seq":18,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.846674+0000","last_change":"2023-12-26T22:52:38.808832+0000","last_active":"2023-12-26T22:52:39.846674+0000","last_peered":"2023-12-26T22:52:39.846674+0000","last_clean":"2023-12-26T22:52:39.846674+0000","last_became_active":"2023-12-26T22:52:38.808579+0000","last_became_peered":"2023-12-26T22:52:38.808579+0000","last_unstale":"2023-12-26T22:52:39.846674+0000","last_undegraded":"2023-12-26T22:52:39.846674+0000","last_fullsized":"2023-12-26T22:52:39.846674+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:19:51.678937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.826071+0000","last_change":"2023-12-26T22:52:39.820681+0000","last_active":"2023-12-26T22:52:39.826071+0000","last_peered":"2023-12-26T22:52:39.826071+0000","last_clean":"2023-12-26T22:52:39.826071+0000","last_became_active":"2023-12-26T22:52:39.820426+0000","last_became_peered":"2023-12-26T22:52:39.820426+0000","last_unstale":"2023-12-26T22:52:39.826071+0000","last_undegraded":"2023-12-26T22:52:39.826071+0000","last_fullsized":"2023-12-26T22:52:39.826071+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:28.894419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.844903+0000","last_change":"2023-12-26T22:52:38.810594+0000","last_active":"2023-12-26T22:52:39.844903+0000","last_peered":"2023-12-26T22:52:39.844903+0000","last_clean":"2023-12-26T22:52:39.844903+0000","last_became_active":"2023-12-26T22:52:38.810277+0000","last_became_peered":"2023-12-26T22:52:38.810277+0000","last_unstale":"2023-12-26T22:52:39.844903+0000","last_undegraded":"2023-12-26T22:52:39.844903+0000","last_fullsized":"2023-12-26T22:52:39.844903+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:48:59.823930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827471+0000","last_change":"2023-12-26T22:52:39.823481+0000","last_active":"2023-12-26T22:52:39.827471+0000","last_peered":"2023-12-26T22:52:39.827471+0000","last_clean":"2023-12-26T22:52:39.827471+0000","last_became_active":"2023-12-26T22:52:39.821403+0000","last_became_peered":"2023-12-26T22:52:39.821403+0000","last_unstale":"2023-12-26T22:52:39.827471+0000","last_undegraded":"2023-12-26T22:52:39.827471+0000","last_fullsized":"2023-12-26T22:52:39.827471+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:39:51.930454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839904+0000","last_change":"2023-12-26T22:52:38.812266+0000","last_active":"2023-12-26T22:52:39.839904+0000","last_peered":"2023-12-26T22:52:39.839904+0000","last_clean":"2023-12-26T22:52:39.839904+0000","last_became_active":"2023-12-26T22:52:38.811998+0000","last_became_peered":"2023-12-26T22:52:38.811998+0000","last_unstale":"2023-12-26T22:52:39.839904+0000","last_undegraded":"2023-12-26T22:52:39.839904+0000","last_fullsized":"2023-12-26T22:52:39.839904+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:25:12.341665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829604+0000","last_change":"2023-12-26T22:52:39.825068+0000","last_active":"2023-12-26T22:52:39.829604+0000","last_peered":"2023-12-26T22:52:39.829604+0000","last_clean":"2023-12-26T22:52:39.829604+0000","last_became_active":"2023-12-26T22:52:39.824791+0000","last_became_peered":"2023-12-26T22:52:39.824791+0000","last_unstale":"2023-12-26T22:52:39.829604+0000","last_undegraded":"2023-12-26T22:52:39.829604+0000","last_fullsized":"2023-12-26T22:52:39.829604+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:01.162816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.841145+0000","last_change":"2023-12-26T22:52:38.806206+0000","last_active":"2023-12-26T22:52:39.841145+0000","last_peered":"2023-12-26T22:52:39.841145+0000","last_clean":"2023-12-26T22:52:39.841145+0000","last_became_active":"2023-12-26T22:52:38.805935+0000","last_became_peered":"2023-12-26T22:52:38.805935+0000","last_unstale":"2023-12-26T22:52:39.841145+0000","last_undegraded":"2023-12-26T22:52:39.841145+0000","last_fullsized":"2023-12-26T22:52:39.841145+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:14:07.043456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831353+0000","last_change":"2023-12-26T22:52:39.827749+0000","last_active":"2023-12-26T22:52:39.831353+0000","last_peered":"2023-12-26T22:52:39.831353+0000","last_clean":"2023-12-26T22:52:39.831353+0000","last_became_active":"2023-12-26T22:52:39.826618+0000","last_became_peered":"2023-12-26T22:52:39.826618+0000","last_unstale":"2023-12-26T22:52:39.831353+0000","last_undegraded":"2023-12-26T22:52:39.831353+0000","last_fullsized":"2023-12-26T22:52:39.831353+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:25:51.946981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.818459+0000","last_change":"2023-12-26T22:52:38.810348+0000","last_active":"2023-12-26T22:52:39.818459+0000","last_peered":"2023-12-26T22:52:39.818459+0000","last_clean":"2023-12-26T22:52:39.818459+0000","last_became_active":"2023-12-26T22:52:38.809954+0000","last_became_peered":"2023-12-26T22:52:38.809954+0000","last_unstale":"2023-12-26T22:52:39.818459+0000","last_undegraded":"2023-12-26T22:52:39.818459+0000","last_fullsized":"2023-12-26T22:52:39.818459+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:55:44.465224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827636+0000","last_change":"2023-12-26T22:52:39.821415+0000","last_active":"2023-12-26T22:52:39.827636+0000","last_peered":"2023-12-26T22:52:39.827636+0000","last_clean":"2023-12-26T22:52:39.827636+0000","last_became_active":"2023-12-26T22:52:39.820875+0000","last_became_peered":"2023-12-26T22:52:39.820875+0000","last_unstale":"2023-12-26T22:52:39.827636+0000","last_undegraded":"2023-12-26T22:52:39.827636+0000","last_fullsized":"2023-12-26T22:52:39.827636+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:33:37.716984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.840914+0000","last_change":"2023-12-26T22:52:38.809885+0000","last_active":"2023-12-26T22:52:39.840914+0000","last_peered":"2023-12-26T22:52:39.840914+0000","last_clean":"2023-12-26T22:52:39.840914+0000","last_became_active":"2023-12-26T22:52:38.809635+0000","last_became_peered":"2023-12-26T22:52:38.809635+0000","last_unstale":"2023-12-26T22:52:39.840914+0000","last_undegraded":"2023-12-26T22:52:39.840914+0000","last_fullsized":"2023-12-26T22:52:39.840914+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:55:18.333216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.823518+0000","last_change":"2023-12-26T22:52:39.818870+0000","last_active":"2023-12-26T22:52:39.823518+0000","last_peered":"2023-12-26T22:52:39.823518+0000","last_clean":"2023-12-26T22:52:39.823518+0000","last_became_active":"2023-12-26T22:52:39.818671+0000","last_became_peered":"2023-12-26T22:52:39.818671+0000","last_unstale":"2023-12-26T22:52:39.823518+0000","last_undegraded":"2023-12-26T22:52:39.823518+0000","last_fullsized":"2023-12-26T22:52:39.823518+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:15.178959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.810082+0000","last_change":"2023-12-26T22:52:38.809776+0000","last_active":"2023-12-26T22:52:39.810082+0000","last_peered":"2023-12-26T22:52:39.810082+0000","last_clean":"2023-12-26T22:52:39.810082+0000","last_became_active":"2023-12-26T22:52:38.809482+0000","last_became_peered":"2023-12-26T22:52:38.809482+0000","last_unstale":"2023-12-26T22:52:39.810082+0000","last_undegraded":"2023-12-26T22:52:39.810082+0000","last_fullsized":"2023-12-26T22:52:39.810082+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:24:57.454731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829716+0000","last_change":"2023-12-26T22:52:39.827080+0000","last_active":"2023-12-26T22:52:39.829716+0000","last_peered":"2023-12-26T22:52:39.829716+0000","last_clean":"2023-12-26T22:52:39.829716+0000","last_became_active":"2023-12-26T22:52:39.823150+0000","last_became_peered":"2023-12-26T22:52:39.823150+0000","last_unstale":"2023-12-26T22:52:39.829716+0000","last_undegraded":"2023-12-26T22:52:39.829716+0000","last_fullsized":"2023-12-26T22:52:39.829716+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:22:23.610018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.814515+0000","last_change":"2023-12-26T22:52:38.815616+0000","last_active":"2023-12-26T22:52:39.814515+0000","last_peered":"2023-12-26T22:52:39.814515+0000","last_clean":"2023-12-26T22:52:39.814515+0000","last_became_active":"2023-12-26T22:52:38.814947+0000","last_became_peered":"2023-12-26T22:52:38.814947+0000","last_unstale":"2023-12-26T22:52:39.814515+0000","last_undegraded":"2023-12-26T22:52:39.814515+0000","last_fullsized":"2023-12-26T22:52:39.814515+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:44:43.034508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835225+0000","last_change":"2023-12-26T22:52:39.826508+0000","last_active":"2023-12-26T22:52:39.835225+0000","last_peered":"2023-12-26T22:52:39.835225+0000","last_clean":"2023-12-26T22:52:39.835225+0000","last_became_active":"2023-12-26T22:52:39.825995+0000","last_became_peered":"2023-12-26T22:52:39.825995+0000","last_unstale":"2023-12-26T22:52:39.835225+0000","last_undegraded":"2023-12-26T22:52:39.835225+0000","last_fullsized":"2023-12-26T22:52:39.835225+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:48.367833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.813598+0000","last_change":"2023-12-26T22:52:38.804764+0000","last_active":"2023-12-26T22:52:39.813598+0000","last_peered":"2023-12-26T22:52:39.813598+0000","last_clean":"2023-12-26T22:52:39.813598+0000","last_became_active":"2023-12-26T22:52:38.802950+0000","last_became_peered":"2023-12-26T22:52:38.802950+0000","last_unstale":"2023-12-26T22:52:39.813598+0000","last_undegraded":"2023-12-26T22:52:39.813598+0000","last_fullsized":"2023-12-26T22:52:39.813598+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:34.546507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.844054+0000","last_change":"2023-12-26T22:52:39.838171+0000","last_active":"2023-12-26T22:52:39.844054+0000","last_peered":"2023-12-26T22:52:39.844054+0000","last_clean":"2023-12-26T22:52:39.844054+0000","last_became_active":"2023-12-26T22:52:39.837896+0000","last_became_peered":"2023-12-26T22:52:39.837896+0000","last_unstale":"2023-12-26T22:52:39.844054+0000","last_undegraded":"2023-12-26T22:52:39.844054+0000","last_fullsized":"2023-12-26T22:52:39.844054+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:35.552674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.843398+0000","last_change":"2023-12-26T22:52:38.813554+0000","last_active":"2023-12-26T22:52:39.843398+0000","last_peered":"2023-12-26T22:52:39.843398+0000","last_clean":"2023-12-26T22:52:39.843398+0000","last_became_active":"2023-12-26T22:52:38.813064+0000","last_became_peered":"2023-12-26T22:52:38.813064+0000","last_unstale":"2023-12-26T22:52:39.843398+0000","last_undegraded":"2023-12-26T22:52:39.843398+0000","last_fullsized":"2023-12-26T22:52:39.843398+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:45:55.560511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830982+0000","last_change":"2023-12-26T22:52:39.822333+0000","last_active":"2023-12-26T22:52:39.830982+0000","last_peered":"2023-12-26T22:52:39.830982+0000","last_clean":"2023-12-26T22:52:39.830982+0000","last_became_active":"2023-12-26T22:52:39.822057+0000","last_became_peered":"2023-12-26T22:52:39.822057+0000","last_unstale":"2023-12-26T22:52:39.830982+0000","last_undegraded":"2023-12-26T22:52:39.830982+0000","last_fullsized":"2023-12-26T22:52:39.830982+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:42:19.294384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.817553+0000","last_change":"2023-12-26T22:52:38.814408+0000","last_active":"2023-12-26T22:52:39.817553+0000","last_peered":"2023-12-26T22:52:39.817553+0000","last_clean":"2023-12-26T22:52:39.817553+0000","last_became_active":"2023-12-26T22:52:38.814109+0000","last_became_peered":"2023-12-26T22:52:38.814109+0000","last_unstale":"2023-12-26T22:52:39.817553+0000","last_undegraded":"2023-12-26T22:52:39.817553+0000","last_fullsized":"2023-12-26T22:52:39.817553+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:10:59.593004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.819818+0000","last_change":"2023-12-26T22:52:39.818053+0000","last_active":"2023-12-26T22:52:39.819818+0000","last_peered":"2023-12-26T22:52:39.819818+0000","last_clean":"2023-12-26T22:52:39.819818+0000","last_became_active":"2023-12-26T22:52:39.817771+0000","last_became_peered":"2023-12-26T22:52:39.817771+0000","last_unstale":"2023-12-26T22:52:39.819818+0000","last_undegraded":"2023-12-26T22:52:39.819818+0000","last_fullsized":"2023-12-26T22:52:39.819818+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:15.178959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.810250+0000","last_change":"2023-12-26T22:52:38.808077+0000","last_active":"2023-12-26T22:52:39.810250+0000","last_peered":"2023-12-26T22:52:39.810250+0000","last_clean":"2023-12-26T22:52:39.810250+0000","last_became_active":"2023-12-26T22:52:38.807769+0000","last_became_peered":"2023-12-26T22:52:38.807769+0000","last_unstale":"2023-12-26T22:52:39.810250+0000","last_undegraded":"2023-12-26T22:52:39.810250+0000","last_fullsized":"2023-12-26T22:52:39.810250+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:28.315292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.832083+0000","last_change":"2023-12-26T22:52:39.824274+0000","last_active":"2023-12-26T22:52:39.832083+0000","last_peered":"2023-12-26T22:52:39.832083+0000","last_clean":"2023-12-26T22:52:39.832083+0000","last_became_active":"2023-12-26T22:52:39.823980+0000","last_became_peered":"2023-12-26T22:52:39.823980+0000","last_unstale":"2023-12-26T22:52:39.832083+0000","last_undegraded":"2023-12-26T22:52:39.832083+0000","last_fullsized":"2023-12-26T22:52:39.832083+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:22.008831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.812021+0000","last_change":"2023-12-26T22:52:38.810987+0000","last_active":"2023-12-26T22:52:39.812021+0000","last_peered":"2023-12-26T22:52:39.812021+0000","last_clean":"2023-12-26T22:52:39.812021+0000","last_became_active":"2023-12-26T22:52:38.810686+0000","last_became_peered":"2023-12-26T22:52:38.810686+0000","last_unstale":"2023-12-26T22:52:39.812021+0000","last_undegraded":"2023-12-26T22:52:39.812021+0000","last_fullsized":"2023-12-26T22:52:39.812021+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:43.884256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.841126+0000","last_change":"2023-12-26T22:52:39.837844+0000","last_active":"2023-12-26T22:52:39.841126+0000","last_peered":"2023-12-26T22:52:39.841126+0000","last_clean":"2023-12-26T22:52:39.841126+0000","last_became_active":"2023-12-26T22:52:39.837571+0000","last_became_peered":"2023-12-26T22:52:39.837571+0000","last_unstale":"2023-12-26T22:52:39.841126+0000","last_undegraded":"2023-12-26T22:52:39.841126+0000","last_fullsized":"2023-12-26T22:52:39.841126+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:30:44.630844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.807598+0000","last_change":"2023-12-26T22:52:38.807859+0000","last_active":"2023-12-26T22:52:39.807598+0000","last_peered":"2023-12-26T22:52:39.807598+0000","last_clean":"2023-12-26T22:52:39.807598+0000","last_became_active":"2023-12-26T22:52:38.807571+0000","last_became_peered":"2023-12-26T22:52:38.807571+0000","last_unstale":"2023-12-26T22:52:39.807598+0000","last_undegraded":"2023-12-26T22:52:39.807598+0000","last_fullsized":"2023-12-26T22:52:39.807598+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:34:26.672326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835883+0000","last_change":"2023-12-26T22:52:39.832401+0000","last_active":"2023-12-26T22:52:39.835883+0000","last_peered":"2023-12-26T22:52:39.835883+0000","last_clean":"2023-12-26T22:52:39.835883+0000","last_became_active":"2023-12-26T22:52:39.832092+0000","last_became_peered":"2023-12-26T22:52:39.832092+0000","last_unstale":"2023-12-26T22:52:39.835883+0000","last_undegraded":"2023-12-26T22:52:39.835883+0000","last_fullsized":"2023-12-26T22:52:39.835883+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:07:28.230904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835638+0000","last_change":"2023-12-26T22:52:38.812709+0000","last_active":"2023-12-26T22:52:39.835638+0000","last_peered":"2023-12-26T22:52:39.835638+0000","last_clean":"2023-12-26T22:52:39.835638+0000","last_became_active":"2023-12-26T22:52:38.811995+0000","last_became_peered":"2023-12-26T22:52:38.811995+0000","last_unstale":"2023-12-26T22:52:39.835638+0000","last_undegraded":"2023-12-26T22:52:39.835638+0000","last_fullsized":"2023-12-26T22:52:39.835638+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:45:50.223763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831432+0000","last_change":"2023-12-26T22:52:39.827794+0000","last_active":"2023-12-26T22:52:39.831432+0000","last_peered":"2023-12-26T22:52:39.831432+0000","last_clean":"2023-12-26T22:52:39.831432+0000","last_became_active":"2023-12-26T22:52:39.827510+0000","last_became_peered":"2023-12-26T22:52:39.827510+0000","last_unstale":"2023-12-26T22:52:39.831432+0000","last_undegraded":"2023-12-26T22:52:39.831432+0000","last_fullsized":"2023-12-26T22:52:39.831432+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:42:15.920688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816856+0000","last_change":"2023-12-26T22:52:38.815488+0000","last_active":"2023-12-26T22:52:39.816856+0000","last_peered":"2023-12-26T22:52:39.816856+0000","last_clean":"2023-12-26T22:52:39.816856+0000","last_became_active":"2023-12-26T22:52:38.814801+0000","last_became_peered":"2023-12-26T22:52:38.814801+0000","last_unstale":"2023-12-26T22:52:39.816856+0000","last_undegraded":"2023-12-26T22:52:39.816856+0000","last_fullsized":"2023-12-26T22:52:39.816856+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:48:59.823930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829757+0000","last_change":"2023-12-26T22:52:39.826008+0000","last_active":"2023-12-26T22:52:39.829757+0000","last_peered":"2023-12-26T22:52:39.829757+0000","last_clean":"2023-12-26T22:52:39.829757+0000","last_became_active":"2023-12-26T22:52:39.822643+0000","last_became_peered":"2023-12-26T22:52:39.822643+0000","last_unstale":"2023-12-26T22:52:39.829757+0000","last_undegraded":"2023-12-26T22:52:39.829757+0000","last_fullsized":"2023-12-26T22:52:39.829757+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:17.910688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816311+0000","last_change":"2023-12-26T22:52:38.812787+0000","last_active":"2023-12-26T22:52:39.816311+0000","last_peered":"2023-12-26T22:52:39.816311+0000","last_clean":"2023-12-26T22:52:39.816311+0000","last_became_active":"2023-12-26T22:52:38.812432+0000","last_became_peered":"2023-12-26T22:52:38.812432+0000","last_unstale":"2023-12-26T22:52:39.816311+0000","last_undegraded":"2023-12-26T22:52:39.816311+0000","last_fullsized":"2023-12-26T22:52:39.816311+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:53:58.318395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831408+0000","last_change":"2023-12-26T22:52:39.823016+0000","last_active":"2023-12-26T22:52:39.831408+0000","last_peered":"2023-12-26T22:52:39.831408+0000","last_clean":"2023-12-26T22:52:39.831408+0000","last_became_active":"2023-12-26T22:52:39.822779+0000","last_became_peered":"2023-12-26T22:52:39.822779+0000","last_unstale":"2023-12-26T22:52:39.831408+0000","last_undegraded":"2023-12-26T22:52:39.831408+0000","last_fullsized":"2023-12-26T22:52:39.831408+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:09:59.482783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.845299+0000","last_change":"2023-12-26T22:52:38.809342+0000","last_active":"2023-12-26T22:52:39.845299+0000","last_peered":"2023-12-26T22:52:39.845299+0000","last_clean":"2023-12-26T22:52:39.845299+0000","last_became_active":"2023-12-26T22:52:38.808972+0000","last_became_peered":"2023-12-26T22:52:38.808972+0000","last_unstale":"2023-12-26T22:52:39.845299+0000","last_undegraded":"2023-12-26T22:52:39.845299+0000","last_fullsized":"2023-12-26T22:52:39.845299+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:47:17.900980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"150'271","reported_seq":836,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.806642+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T22:52:39.806642+0000","last_peered":"2023-12-26T22:52:39.806642+0000","last_clean":"2023-12-26T22:52:39.806642+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T22:52:39.806642+0000","last_undegraded":"2023-12-26T22:52:39.806642+0000","last_fullsized":"2023-12-26T22:52:39.806642+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543050,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":1952,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":1998848,"data_stored":992016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.71299999999999997},"min":{"1min":0.53500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.41999999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.68000000000000005}]},{"osd":1,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.42499999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.90200000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.73399999999999999},"min":{"1min":0.53400000000000003,"5min":0.46000000000000002,"15min":0.45700000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.41499999999999998},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.88200000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80300000000000005,"15min":0.79200000000000004},"min":{"1min":0.63600000000000001,"5min":0.58899999999999997,"15min":0.47899999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.42799999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.73499999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.80800000000000005}]},{"osd":5,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.34599999999999997},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.47799999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.94299999999999995}]},{"osd":6,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.44700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.39000000000000001},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":0.65600000000000003}]}]},{"osd":6,"up_from":15,"seq":64424510346,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307428,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064412,"statfs":{"total":96636764160,"available":96321957888,"internally_reserved":0,"allocated":1994752,"data_stored":991982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.98899999999999999},"last":0.876},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.92100000000000004,"5min":0.95399999999999996,"15min":0.98799999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.50600000000000001},"max":{"1min":0.93300000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.626}]},{"osd":2,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.95899999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77800000000000002},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.52800000000000002},"max":{"1min":0.95299999999999996,"5min":1.012,"15min":1.0249999999999999},"last":0.68500000000000005}]},{"osd":4,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":0.998,"15min":0.998},"last":0.505},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.48099999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.871,"5min":0.995,"15min":0.995},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42499999999999999},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70699999999999996,"15min":0.69799999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81499999999999995}]},{"osd":7,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.93999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97199999999999998,"5min":1.05,"15min":1.05},"last":0.90100000000000002}]}]},{"osd":1,"up_from":15,"seq":64424510346,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312120,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059720,"statfs":{"total":96636764160,"available":96317153280,"internally_reserved":0,"allocated":2605056,"data_stored":1583908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.47199999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.88400000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.35999999999999999,"5min":0.32300000000000001,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.81000000000000005,"15min":0.84499999999999997},"last":0.45200000000000001}]},{"osd":2,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.32400000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.752},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.87,"15min":0.87},"last":0.46500000000000002}]},{"osd":3,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.38700000000000001},"max":{"1min":0.80600000000000005,"5min":0.877,"15min":0.90900000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.38200000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":0.97799999999999998,"5min":1.044,"15min":1.163},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.121},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.82299999999999995},"min":{"1min":0.69399999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.028,"5min":1.0820000000000001,"15min":1.1419999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83499999999999996,"15min":0.83199999999999996},"min":{"1min":0.68100000000000005,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.04,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.2450000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.84199999999999997,"15min":0.83999999999999997},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.1259999999999999,"15min":1.2270000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.55700000000000005},"max":{"1min":0.99299999999999999,"5min":1.107,"15min":1.2689999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.497},"max":{"1min":1.0529999999999999,"5min":1.0660000000000001,"15min":1.1890000000000001},"last":0.70099999999999996}]}]},{"osd":0,"up_from":15,"seq":64424510346,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307448,"kb_used_data":1968,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064392,"statfs":{"total":96636764160,"available":96321937408,"internally_reserved":0,"allocated":2015232,"data_stored":993574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.28199999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.63400000000000001,"5min":0.73799999999999999,"15min":0.86599999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.627,"5min":0.70499999999999996,"15min":0.88800000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72899999999999998,"5min":0.81899999999999995,"15min":0.90500000000000003},"last":0.39000000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.35499999999999998},"max":{"1min":0.72699999999999998,"5min":0.80900000000000005,"15min":0.86799999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.51200000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.92200000000000004},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":1.0649999999999999,"5min":1.103,"15min":1.145},"last":0.75},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.65800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0640000000000001,"5min":1.1299999999999999,"15min":1.218},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.83299999999999996},"min":{"1min":0.65400000000000003,"5min":0.58099999999999996,"15min":0.51600000000000001},"max":{"1min":1.0489999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83799999999999997,"15min":0.83399999999999996},"min":{"1min":0.64300000000000002,"5min":0.56599999999999995,"15min":0.50900000000000001},"max":{"1min":1.081,"5min":1.218,"15min":1.218},"last":0.93100000000000005}]},{"osd":6,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.58999999999999997},"max":{"1min":1.0229999999999999,"5min":1.1499999999999999,"15min":1.1659999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.55500000000000005},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.2350000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82099999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.54500000000000004},"max":{"1min":1.073,"5min":1.165,"15min":1.254},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81200000000000006},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.256,"15min":1.3200000000000001},"last":0.87}]}]},{"osd":2,"up_from":14,"seq":60129543051,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307436,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064404,"statfs":{"total":96636764160,"available":96321949696,"internally_reserved":0,"allocated":2002944,"data_stored":992626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29499999999999998},"max":{"1min":0.66700000000000004,"5min":1.075,"15min":1.075},"last":0.443},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.439,"5min":0.42499999999999999,"15min":0.378},"max":{"1min":0.73199999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32900000000000001},"max":{"1min":0.751,"5min":1.079,"15min":1.079},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34100000000000003},"max":{"1min":0.65900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.34100000000000003}]},{"osd":3,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":1.129,"15min":1.2490000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.83899999999999997,"5min":0.84999999999999998,"15min":1.198},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.626,"5min":0.59199999999999997,"15min":0.52500000000000002},"max":{"1min":1.177,"5min":1.222,"15min":1.222},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.50600000000000001},"max":{"1min":1.044,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.48499999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.22},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82799999999999996,"15min":0.80900000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57099999999999995},"max":{"1min":1.095,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.78800000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.56100000000000005},"max":{"1min":1.075,"5min":1.075,"15min":1.2470000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.67600000000000005,"5min":0.625,"15min":0.58699999999999997},"max":{"1min":1.0649999999999999,"5min":1.298,"15min":1.298},"last":0.80800000000000005}]},{"osd":7,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":1.0389999999999999,"5min":1.056,"15min":1.1899999999999999},"last":0.754},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84299999999999997},"min":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2350000000000001,"5min":1.367,"15min":1.367},"last":0.70199999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510347,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299240,"kb_used_data":1952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072600,"statfs":{"total":96636764160,"available":96330342400,"internally_reserved":0,"allocated":1998848,"data_stored":992052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28000000000000003},"max":{"1min":0.64600000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.93799999999999994},"last":0.44400000000000001}]},{"osd":1,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":1.0449999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.752,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.622,"15min":0.625},"min":{"1min":0.441,"5min":0.441,"15min":0.40000000000000002},"max":{"1min":0.73799999999999999,"5min":1.516,"15min":1.516},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.49399999999999999,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.879,"5min":1.395,"15min":1.395},"last":0.502}]},{"osd":4,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.03,"5min":1.413,"15min":1.413},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79800000000000004,"15min":0.78200000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46999999999999997},"max":{"1min":1.0980000000000001,"5min":1.163,"15min":1.1850000000000001},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.60099999999999998,"5min":0.54500000000000004,"15min":0.52200000000000002},"max":{"1min":1.002,"5min":1.054,"15min":1.1359999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82099999999999995,"15min":0.79900000000000004},"min":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.53100000000000003},"max":{"1min":1.054,"5min":1.202,"15min":1.2270000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.66100000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.0880000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60399999999999998},"max":{"1min":1.101,"5min":1.1140000000000001,"15min":1.165},"last":0.84499999999999997}]},{"osd":7,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82399999999999995,"15min":0.81999999999999995},"min":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.55300000000000005},"max":{"1min":0.998,"5min":1.321,"15min":1.321},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.52000000000000002},"max":{"1min":1.0780000000000001,"5min":1.466,"15min":1.466},"last":0.89200000000000002}]}]},{"osd":4,"up_from":14,"seq":60129543050,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311524,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060316,"statfs":{"total":96636764160,"available":96317763584,"internally_reserved":0,"allocated":1994752,"data_stored":991982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.52300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.90400000000000003,"15min":0.99299999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.99199999999999999,"15min":1.0640000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53900000000000003,"5min":0.48699999999999999,"15min":0.47099999999999997},"max":{"1min":0.90500000000000003,"5min":1.01,"15min":1.0309999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72999999999999998,"15min":0.73799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90000000000000002,"5min":0.98599999999999999,"15min":1.04},"last":0.64300000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":0.90800000000000003,"5min":0.93400000000000005,"15min":0.98599999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96399999999999997,"5min":0.97799999999999998,"15min":0.99399999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.46300000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75800000000000001,"15min":0.76700000000000002},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.503},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.97399999999999998},"last":0.79900000000000004}]},{"osd":6,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.35699999999999998},"max":{"1min":0.87,"5min":0.94799999999999995,"15min":0.98699999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":1.0149999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.44900000000000001},"max":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":1.0449999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.751}]}]},{"osd":5,"up_from":15,"seq":64424510345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312116,"kb_used_data":2540,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059724,"statfs":{"total":96636764160,"available":96317157376,"internally_reserved":0,"allocated":2600960,"data_stored":1583064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40100000000000002},"max":{"1min":0.96199999999999997,"5min":1.071,"15min":1.071},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.499},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.58199999999999996,"5min":0.50600000000000001,"15min":0.46200000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.62},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.2269999999999999,"5min":3.2269999999999999,"15min":3.2269999999999999},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91500000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.75600000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9199999999999999,"5min":2.9199999999999999,"15min":2.9199999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.66600000000000004}]},{"osd":6,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.0169999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.48499999999999999,"15min":0.45300000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.751}]},{"osd":7,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95099999999999996,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T22:52:49.048 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T22:52:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.249+0000 7fcfbadff700 1 Processor -- start 2023-12-26T22:52:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.249+0000 7fcfbadff700 1 -- start start 2023-12-26T22:52:49.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.250+0000 7fcfbadff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcfb40cb320 0x7fcfb4069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.250+0000 7fcfbadff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb4156940 con 0x7fcfb40b0e60 2023-12-26T22:52:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.250+0000 7fcfbadff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb40ae500 con 0x7fcfb406a1e0 2023-12-26T22:52:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.250+0000 7fcfbadff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb40c8cc0 con 0x7fcfb40cb320 2023-12-26T22:52:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb939c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcfb40b0e60 0x7fcfb40c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41038/0 (socket says 172.21.15.131:41038) 2023-12-26T22:52:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb939c700 1 -- 172.21.15.131:0/2644136855 learned_addr learned my addr 172.21.15.131:0/2644136855 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb8b9b700 1 --2- 172.21.15.131:0/2644136855 >> v2:172.21.15.190:3300/0 conn(0x7fcfb40cb320 0x7fcfb4069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31951025 0 0) 0x7fcfb4156940 con 0x7fcfb40b0e60 2023-12-26T22:52:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa4003570 con 0x7fcfb40b0e60 2023-12-26T22:52:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3615900836 0 0) 0x7fcfa4003570 con 0x7fcfb40b0e60 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 >> v2:172.21.15.190:3300/0 conn(0x7fcfb40cb320 msgr2=0x7fcfb4069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb37fe700 1 --2- 172.21.15.131:0/2644136855 >> v2:172.21.15.190:3300/0 conn(0x7fcfb40cb320 0x7fcfb4069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 >> v1:172.21.15.190:6789/0 conn(0x7fcfb406a1e0 legacy=0x7fcfb40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40b0280 con 0x7fcfb40b0e60 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.251+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fcfac0017f0 con 0x7fcfb40b0e60 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 >> v1:172.21.15.131:6789/0 conn(0x7fcfb40b0e60 legacy=0x7fcfb40c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb37fe700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4003ab0 0x7fcfa4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb37fe700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa4004810 0x7fcfa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb37fe700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfa4005210 0x7fcfa400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c8cc0 con 0x7fcfa4005210 2023-12-26T22:52:49.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa400c130 con 0x7fcfa4003ab0 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa400e500 con 0x7fcfa4004810 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb939c700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfa4005210 0x7fcfa400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb8b9b700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4003ab0 0x7fcfa4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb3fff700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa4004810 0x7fcfa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb939c700 1 -- 172.21.15.131:0/2644136855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa4004810 msgr2=0x7fcfa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb939c700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa4004810 0x7fcfa4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb939c700 1 -- 172.21.15.131:0/2644136855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4003ab0 msgr2=0x7fcfa4003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb939c700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4003ab0 0x7fcfa4003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb939c700 1 -- 172.21.15.131:0/2644136855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4003570 con 0x7fcfa4005210 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.252+0000 7fcfb939c700 1 -- 172.21.15.131:0/2644136855 reap_dead start 2023-12-26T22:52:49.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfb939c700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfa4005210 0x7fcfa400ba60 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40b1260 tx=0x7fcfb40ae470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cf656242d8864d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0070d0 con 0x7fcfa4005210 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac007250 con 0x7fcfa4005210 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfb37fe700 1 -- 172.21.15.131:0/2644136855 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac007610 con 0x7fcfa4005210 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfbadff700 1 -- 172.21.15.131:0/2644136855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfa4005210 msgr2=0x7fcfa400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfbadff700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfa4005210 0x7fcfa400ba60 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40b1260 tx=0x7fcfb40ae470 comp rx=0 tx=0).stop 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfbadff700 1 -- 172.21.15.131:0/2644136855 shutdown_connections 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfbadff700 1 --2- 172.21.15.131:0/2644136855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfa4005210 0x7fcfa400ba60 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfbadff700 1 -- 172.21.15.131:0/2644136855 >> 172.21.15.131:0/2644136855 conn(0x7fcfb4068040 msgr2=0x7fcfb4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfbadff700 1 -- 172.21.15.131:0/2644136855 shutdown_connections 2023-12-26T22:52:49.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.253+0000 7fcfbadff700 1 -- 172.21.15.131:0/2644136855 wait complete. 2023-12-26T22:52:49.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.255+0000 7fcfbadff700 1 Processor -- start 2023-12-26T22:52:49.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.255+0000 7fcfbadff700 1 -- start start 2023-12-26T22:52:49.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfbadff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4005210 0x7fcfb4145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfbadff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfb415b720 0x7fcfb40c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfbadff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb415bb90 0x7fcfb416cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfbadff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c5a00 con 0x7fcfb415bb90 2023-12-26T22:52:49.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfbadff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c6860 con 0x7fcfa4005210 2023-12-26T22:52:49.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfbadff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb416dba0 con 0x7fcfb415b720 2023-12-26T22:52:49.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfb939c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb415bb90 0x7fcfb416cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfb939c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb415bb90 0x7fcfb416cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35604/0 (socket says 172.21.15.131:35604) 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfb939c700 1 -- 172.21.15.131:0/1538467858 learned_addr learned my addr 172.21.15.131:0/1538467858 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfb3fff700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfb415b720 0x7fcfb40c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.256+0000 7fcfb8b9b700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4005210 0x7fcfb4145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfb939c700 1 -- 172.21.15.131:0/1538467858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfb415b720 msgr2=0x7fcfb40c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfb939c700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfb415b720 0x7fcfb40c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfb939c700 1 -- 172.21.15.131:0/1538467858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4005210 msgr2=0x7fcfb4145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfb939c700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4005210 0x7fcfb4145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfb939c700 1 -- 172.21.15.131:0/1538467858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb4146200 con 0x7fcfb415bb90 2023-12-26T22:52:49.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfb939c700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb415bb90 0x7fcfb416cda0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00afd0 tx=0x7fcfac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfb0ff9700 1 -- 172.21.15.131:0/1538467858 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac007030 con 0x7fcfb415bb90 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfb0ff9700 1 -- 172.21.15.131:0/1538467858 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac007980 con 0x7fcfb415bb90 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb40c5c50 con 0x7fcfb415bb90 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfb0ff9700 1 -- 172.21.15.131:0/1538467858 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac007610 con 0x7fcfb415bb90 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.257+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb40c6210 con 0x7fcfb415bb90 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.258+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb40c69d0 con 0x7fcfb415bb90 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.259+0000 7fcfb0ff9700 1 -- 172.21.15.131:0/1538467858 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcfac01f430 con 0x7fcfb415bb90 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.260+0000 7fcfb0ff9700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf98066f30 0x7fcf980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.260+0000 7fcfb0ff9700 1 -- 172.21.15.131:0/1538467858 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fcfac085b50 con 0x7fcfb415bb90 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.260+0000 7fcfb8b9b700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf98066f30 0x7fcf980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.261+0000 7fcfb8b9b700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf98066f30 0x7fcf980693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8006f00 tx=0x7fcfa80091a0 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.264+0000 7fcfb0ff9700 1 -- 172.21.15.131:0/1538467858 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfac052c00 con 0x7fcfb415bb90 2023-12-26T22:52:49.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.453+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfb40657e0 con 0x7fcf98066f30 2023-12-26T22:52:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.461+0000 7fcfb0ff9700 1 -- 172.21.15.131:0/1538467858 <== mgr.8436 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289470 (secure 0 0 0) 0x7fcfb40657e0 con 0x7fcf98066f30 2023-12-26T22:52:49.465 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf98066f30 msgr2=0x7fcf980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf98066f30 0x7fcf980693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8006f00 tx=0x7fcfa80091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb415bb90 msgr2=0x7fcfb416cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb415bb90 0x7fcfb416cda0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00afd0 tx=0x7fcfac016000 comp rx=0 tx=0).stop 2023-12-26T22:52:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 shutdown_connections 2023-12-26T22:52:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcf98066f30 0x7fcf980693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb415bb90 0x7fcfb416cda0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4005210 0x7fcfb4145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 --2- 172.21.15.131:0/1538467858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfb415b720 0x7fcfb40c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 >> 172.21.15.131:0/1538467858 conn(0x7fcfb4068040 msgr2=0x7fcfb40c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 shutdown_connections 2023-12-26T22:52:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.466+0000 7fcfbadff700 1 -- 172.21.15.131:0/1538467858 wait complete. 2023-12-26T22:52:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T22:52:49.483 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-26T22:52:47.757197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2464744,"kb_used_data":16808,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752509976,"statfs":{"total":773094113280,"available":770570215424,"internally_reserved":0,"allocated":17211392,"data_stored":9121204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001619"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839246+0000","last_change":"2023-12-26T22:52:39.835959+0000","last_active":"2023-12-26T22:52:39.839246+0000","last_peered":"2023-12-26T22:52:39.839246+0000","last_clean":"2023-12-26T22:52:39.839246+0000","last_became_active":"2023-12-26T22:52:39.835701+0000","last_became_peered":"2023-12-26T22:52:39.835701+0000","last_unstale":"2023-12-26T22:52:39.839246+0000","last_undegraded":"2023-12-26T22:52:39.839246+0000","last_fullsized":"2023-12-26T22:52:39.839246+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:04:34.951035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829859+0000","last_change":"2023-12-26T22:52:39.825281+0000","last_active":"2023-12-26T22:52:39.829859+0000","last_peered":"2023-12-26T22:52:39.829859+0000","last_clean":"2023-12-26T22:52:39.829859+0000","last_became_active":"2023-12-26T22:52:39.824828+0000","last_became_peered":"2023-12-26T22:52:39.824828+0000","last_unstale":"2023-12-26T22:52:39.829859+0000","last_undegraded":"2023-12-26T22:52:39.829859+0000","last_fullsized":"2023-12-26T22:52:39.829859+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:58:39.999445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.834845+0000","last_change":"2023-12-26T22:52:39.830474+0000","last_active":"2023-12-26T22:52:39.834845+0000","last_peered":"2023-12-26T22:52:39.834845+0000","last_clean":"2023-12-26T22:52:39.834845+0000","last_became_active":"2023-12-26T22:52:39.830201+0000","last_became_peered":"2023-12-26T22:52:39.830201+0000","last_unstale":"2023-12-26T22:52:39.834845+0000","last_undegraded":"2023-12-26T22:52:39.834845+0000","last_fullsized":"2023-12-26T22:52:39.834845+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:31:16.761360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.842530+0000","last_change":"2023-12-26T22:52:39.837252+0000","last_active":"2023-12-26T22:52:39.842530+0000","last_peered":"2023-12-26T22:52:39.842530+0000","last_clean":"2023-12-26T22:52:39.842530+0000","last_became_active":"2023-12-26T22:52:39.836849+0000","last_became_peered":"2023-12-26T22:52:39.836849+0000","last_unstale":"2023-12-26T22:52:39.842530+0000","last_undegraded":"2023-12-26T22:52:39.842530+0000","last_fullsized":"2023-12-26T22:52:39.842530+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:39:42.771599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.844550+0000","last_change":"2023-12-26T22:52:39.838540+0000","last_active":"2023-12-26T22:52:39.844550+0000","last_peered":"2023-12-26T22:52:39.844550+0000","last_clean":"2023-12-26T22:52:39.844550+0000","last_became_active":"2023-12-26T22:52:39.838299+0000","last_became_peered":"2023-12-26T22:52:39.838299+0000","last_unstale":"2023-12-26T22:52:39.844550+0000","last_undegraded":"2023-12-26T22:52:39.844550+0000","last_fullsized":"2023-12-26T22:52:39.844550+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:20:35.952591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839595+0000","last_change":"2023-12-26T22:52:39.836133+0000","last_active":"2023-12-26T22:52:39.839595+0000","last_peered":"2023-12-26T22:52:39.839595+0000","last_clean":"2023-12-26T22:52:39.839595+0000","last_became_active":"2023-12-26T22:52:39.835795+0000","last_became_peered":"2023-12-26T22:52:39.835795+0000","last_unstale":"2023-12-26T22:52:39.839595+0000","last_undegraded":"2023-12-26T22:52:39.839595+0000","last_fullsized":"2023-12-26T22:52:39.839595+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:56:24.308560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827733+0000","last_change":"2023-12-26T22:52:39.824058+0000","last_active":"2023-12-26T22:52:39.827733+0000","last_peered":"2023-12-26T22:52:39.827733+0000","last_clean":"2023-12-26T22:52:39.827733+0000","last_became_active":"2023-12-26T22:52:39.823758+0000","last_became_peered":"2023-12-26T22:52:39.823758+0000","last_unstale":"2023-12-26T22:52:39.827733+0000","last_undegraded":"2023-12-26T22:52:39.827733+0000","last_fullsized":"2023-12-26T22:52:39.827733+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:08:09.135608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.822165+0000","last_change":"2023-12-26T22:52:39.818061+0000","last_active":"2023-12-26T22:52:39.822165+0000","last_peered":"2023-12-26T22:52:39.822165+0000","last_clean":"2023-12-26T22:52:39.822165+0000","last_became_active":"2023-12-26T22:52:39.817784+0000","last_became_peered":"2023-12-26T22:52:39.817784+0000","last_unstale":"2023-12-26T22:52:39.822165+0000","last_undegraded":"2023-12-26T22:52:39.822165+0000","last_fullsized":"2023-12-26T22:52:39.822165+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:52:15.221910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835113+0000","last_change":"2023-12-26T22:52:39.831201+0000","last_active":"2023-12-26T22:52:39.835113+0000","last_peered":"2023-12-26T22:52:39.835113+0000","last_clean":"2023-12-26T22:52:39.835113+0000","last_became_active":"2023-12-26T22:52:39.830972+0000","last_became_peered":"2023-12-26T22:52:39.830972+0000","last_unstale":"2023-12-26T22:52:39.835113+0000","last_undegraded":"2023-12-26T22:52:39.835113+0000","last_fullsized":"2023-12-26T22:52:39.835113+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:39:23.078653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839274+0000","last_change":"2023-12-26T22:52:39.836097+0000","last_active":"2023-12-26T22:52:39.839274+0000","last_peered":"2023-12-26T22:52:39.839274+0000","last_clean":"2023-12-26T22:52:39.839274+0000","last_became_active":"2023-12-26T22:52:39.835650+0000","last_became_peered":"2023-12-26T22:52:39.835650+0000","last_unstale":"2023-12-26T22:52:39.839274+0000","last_undegraded":"2023-12-26T22:52:39.839274+0000","last_fullsized":"2023-12-26T22:52:39.839274+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:08:09.135608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.825845+0000","last_change":"2023-12-26T22:52:39.820024+0000","last_active":"2023-12-26T22:52:39.825845+0000","last_peered":"2023-12-26T22:52:39.825845+0000","last_clean":"2023-12-26T22:52:39.825845+0000","last_became_active":"2023-12-26T22:52:39.819743+0000","last_became_peered":"2023-12-26T22:52:39.819743+0000","last_unstale":"2023-12-26T22:52:39.825845+0000","last_undegraded":"2023-12-26T22:52:39.825845+0000","last_fullsized":"2023-12-26T22:52:39.825845+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:16:14.117868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.840318+0000","last_change":"2023-12-26T22:52:39.836477+0000","last_active":"2023-12-26T22:52:39.840318+0000","last_peered":"2023-12-26T22:52:39.840318+0000","last_clean":"2023-12-26T22:52:39.840318+0000","last_became_active":"2023-12-26T22:52:39.834537+0000","last_became_peered":"2023-12-26T22:52:39.834537+0000","last_unstale":"2023-12-26T22:52:39.840318+0000","last_undegraded":"2023-12-26T22:52:39.840318+0000","last_fullsized":"2023-12-26T22:52:39.840318+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:19:49.658340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827869+0000","last_change":"2023-12-26T22:52:39.823718+0000","last_active":"2023-12-26T22:52:39.827869+0000","last_peered":"2023-12-26T22:52:39.827869+0000","last_clean":"2023-12-26T22:52:39.827869+0000","last_became_active":"2023-12-26T22:52:39.821803+0000","last_became_peered":"2023-12-26T22:52:39.821803+0000","last_unstale":"2023-12-26T22:52:39.827869+0000","last_undegraded":"2023-12-26T22:52:39.827869+0000","last_fullsized":"2023-12-26T22:52:39.827869+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:32:48.632110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830026+0000","last_change":"2023-12-26T22:52:39.824548+0000","last_active":"2023-12-26T22:52:39.830026+0000","last_peered":"2023-12-26T22:52:39.830026+0000","last_clean":"2023-12-26T22:52:39.830026+0000","last_became_active":"2023-12-26T22:52:39.824279+0000","last_became_peered":"2023-12-26T22:52:39.824279+0000","last_unstale":"2023-12-26T22:52:39.830026+0000","last_undegraded":"2023-12-26T22:52:39.830026+0000","last_fullsized":"2023-12-26T22:52:39.830026+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:42:22.274717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830409+0000","last_change":"2023-12-26T22:52:39.823317+0000","last_active":"2023-12-26T22:52:39.830409+0000","last_peered":"2023-12-26T22:52:39.830409+0000","last_clean":"2023-12-26T22:52:39.830409+0000","last_became_active":"2023-12-26T22:52:39.823023+0000","last_became_peered":"2023-12-26T22:52:39.823023+0000","last_unstale":"2023-12-26T22:52:39.830409+0000","last_undegraded":"2023-12-26T22:52:39.830409+0000","last_fullsized":"2023-12-26T22:52:39.830409+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:55:10.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831122+0000","last_change":"2023-12-26T22:52:39.829027+0000","last_active":"2023-12-26T22:52:39.831122+0000","last_peered":"2023-12-26T22:52:39.831122+0000","last_clean":"2023-12-26T22:52:39.831122+0000","last_became_active":"2023-12-26T22:52:39.828259+0000","last_became_peered":"2023-12-26T22:52:39.828259+0000","last_unstale":"2023-12-26T22:52:39.831122+0000","last_undegraded":"2023-12-26T22:52:39.831122+0000","last_fullsized":"2023-12-26T22:52:39.831122+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:07:17.014057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.834941+0000","last_change":"2023-12-26T22:52:39.830730+0000","last_active":"2023-12-26T22:52:39.834941+0000","last_peered":"2023-12-26T22:52:39.834941+0000","last_clean":"2023-12-26T22:52:39.834941+0000","last_became_active":"2023-12-26T22:52:39.830490+0000","last_became_peered":"2023-12-26T22:52:39.830490+0000","last_unstale":"2023-12-26T22:52:39.834941+0000","last_undegraded":"2023-12-26T22:52:39.834941+0000","last_fullsized":"2023-12-26T22:52:39.834941+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:15:04.617894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830420+0000","last_change":"2023-12-26T22:52:39.823248+0000","last_active":"2023-12-26T22:52:39.830420+0000","last_peered":"2023-12-26T22:52:39.830420+0000","last_clean":"2023-12-26T22:52:39.830420+0000","last_became_active":"2023-12-26T22:52:39.822871+0000","last_became_peered":"2023-12-26T22:52:39.822871+0000","last_unstale":"2023-12-26T22:52:39.830420+0000","last_undegraded":"2023-12-26T22:52:39.830420+0000","last_fullsized":"2023-12-26T22:52:39.830420+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:53.548137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827280+0000","last_change":"2023-12-26T22:52:39.821141+0000","last_active":"2023-12-26T22:52:39.827280+0000","last_peered":"2023-12-26T22:52:39.827280+0000","last_clean":"2023-12-26T22:52:39.827280+0000","last_became_active":"2023-12-26T22:52:39.820850+0000","last_became_peered":"2023-12-26T22:52:39.820850+0000","last_unstale":"2023-12-26T22:52:39.827280+0000","last_undegraded":"2023-12-26T22:52:39.827280+0000","last_fullsized":"2023-12-26T22:52:39.827280+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:55:10.847693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830521+0000","last_change":"2023-12-26T22:52:39.823571+0000","last_active":"2023-12-26T22:52:39.830521+0000","last_peered":"2023-12-26T22:52:39.830521+0000","last_clean":"2023-12-26T22:52:39.830521+0000","last_became_active":"2023-12-26T22:52:39.823275+0000","last_became_peered":"2023-12-26T22:52:39.823275+0000","last_unstale":"2023-12-26T22:52:39.830521+0000","last_undegraded":"2023-12-26T22:52:39.830521+0000","last_fullsized":"2023-12-26T22:52:39.830521+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:01.162816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835865+0000","last_change":"2023-12-26T22:52:39.829908+0000","last_active":"2023-12-26T22:52:39.835865+0000","last_peered":"2023-12-26T22:52:39.835865+0000","last_clean":"2023-12-26T22:52:39.835865+0000","last_became_active":"2023-12-26T22:52:39.829435+0000","last_became_peered":"2023-12-26T22:52:39.829435+0000","last_unstale":"2023-12-26T22:52:39.835865+0000","last_undegraded":"2023-12-26T22:52:39.835865+0000","last_fullsized":"2023-12-26T22:52:39.835865+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:02:04.626344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831087+0000","last_change":"2023-12-26T22:52:39.827559+0000","last_active":"2023-12-26T22:52:39.831087+0000","last_peered":"2023-12-26T22:52:39.831087+0000","last_clean":"2023-12-26T22:52:39.831087+0000","last_became_active":"2023-12-26T22:52:39.827269+0000","last_became_peered":"2023-12-26T22:52:39.827269+0000","last_unstale":"2023-12-26T22:52:39.831087+0000","last_undegraded":"2023-12-26T22:52:39.831087+0000","last_fullsized":"2023-12-26T22:52:39.831087+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:42:15.920688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.834921+0000","last_change":"2023-12-26T22:52:39.830038+0000","last_active":"2023-12-26T22:52:39.834921+0000","last_peered":"2023-12-26T22:52:39.834921+0000","last_clean":"2023-12-26T22:52:39.834921+0000","last_became_active":"2023-12-26T22:52:39.829684+0000","last_became_peered":"2023-12-26T22:52:39.829684+0000","last_unstale":"2023-12-26T22:52:39.834921+0000","last_undegraded":"2023-12-26T22:52:39.834921+0000","last_fullsized":"2023-12-26T22:52:39.834921+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:40.074006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830492+0000","last_change":"2023-12-26T22:52:39.824345+0000","last_active":"2023-12-26T22:52:39.830492+0000","last_peered":"2023-12-26T22:52:39.830492+0000","last_clean":"2023-12-26T22:52:39.830492+0000","last_became_active":"2023-12-26T22:52:39.824092+0000","last_became_peered":"2023-12-26T22:52:39.824092+0000","last_unstale":"2023-12-26T22:52:39.830492+0000","last_undegraded":"2023-12-26T22:52:39.830492+0000","last_fullsized":"2023-12-26T22:52:39.830492+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:42:19.294384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.825944+0000","last_change":"2023-12-26T22:52:39.820236+0000","last_active":"2023-12-26T22:52:39.825944+0000","last_peered":"2023-12-26T22:52:39.825944+0000","last_clean":"2023-12-26T22:52:39.825944+0000","last_became_active":"2023-12-26T22:52:39.819800+0000","last_became_peered":"2023-12-26T22:52:39.819800+0000","last_unstale":"2023-12-26T22:52:39.825944+0000","last_undegraded":"2023-12-26T22:52:39.825944+0000","last_fullsized":"2023-12-26T22:52:39.825944+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:41:42.506695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816985+0000","last_change":"2023-12-26T22:52:39.813032+0000","last_active":"2023-12-26T22:52:39.816985+0000","last_peered":"2023-12-26T22:52:39.816985+0000","last_clean":"2023-12-26T22:52:39.816985+0000","last_became_active":"2023-12-26T22:52:39.812742+0000","last_became_peered":"2023-12-26T22:52:39.812742+0000","last_unstale":"2023-12-26T22:52:39.816985+0000","last_undegraded":"2023-12-26T22:52:39.816985+0000","last_fullsized":"2023-12-26T22:52:39.816985+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:34.964257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.842248+0000","last_change":"2023-12-26T22:52:39.838105+0000","last_active":"2023-12-26T22:52:39.842248+0000","last_peered":"2023-12-26T22:52:39.842248+0000","last_clean":"2023-12-26T22:52:39.842248+0000","last_became_active":"2023-12-26T22:52:39.837836+0000","last_became_peered":"2023-12-26T22:52:39.837836+0000","last_unstale":"2023-12-26T22:52:39.842248+0000","last_undegraded":"2023-12-26T22:52:39.842248+0000","last_fullsized":"2023-12-26T22:52:39.842248+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:03:40.074006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.828691+0000","last_change":"2023-12-26T22:52:39.823463+0000","last_active":"2023-12-26T22:52:39.828691+0000","last_peered":"2023-12-26T22:52:39.828691+0000","last_clean":"2023-12-26T22:52:39.828691+0000","last_became_active":"2023-12-26T22:52:39.820090+0000","last_became_peered":"2023-12-26T22:52:39.820090+0000","last_unstale":"2023-12-26T22:52:39.828691+0000","last_undegraded":"2023-12-26T22:52:39.828691+0000","last_fullsized":"2023-12-26T22:52:39.828691+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:22:23.610018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827737+0000","last_change":"2023-12-26T22:52:39.821258+0000","last_active":"2023-12-26T22:52:39.827737+0000","last_peered":"2023-12-26T22:52:39.827737+0000","last_clean":"2023-12-26T22:52:39.827737+0000","last_became_active":"2023-12-26T22:52:39.820952+0000","last_became_peered":"2023-12-26T22:52:39.820952+0000","last_unstale":"2023-12-26T22:52:39.827737+0000","last_undegraded":"2023-12-26T22:52:39.827737+0000","last_fullsized":"2023-12-26T22:52:39.827737+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:09:39.835895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829859+0000","last_change":"2023-12-26T22:52:39.821885+0000","last_active":"2023-12-26T22:52:39.829859+0000","last_peered":"2023-12-26T22:52:39.829859+0000","last_clean":"2023-12-26T22:52:39.829859+0000","last_became_active":"2023-12-26T22:52:39.821616+0000","last_became_peered":"2023-12-26T22:52:39.821616+0000","last_unstale":"2023-12-26T22:52:39.829859+0000","last_undegraded":"2023-12-26T22:52:39.829859+0000","last_fullsized":"2023-12-26T22:52:39.829859+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:48:29.597572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.820290+0000","last_change":"2023-12-26T22:52:39.815915+0000","last_active":"2023-12-26T22:52:39.820290+0000","last_peered":"2023-12-26T22:52:39.820290+0000","last_clean":"2023-12-26T22:52:39.820290+0000","last_became_active":"2023-12-26T22:52:39.815128+0000","last_became_peered":"2023-12-26T22:52:39.815128+0000","last_unstale":"2023-12-26T22:52:39.820290+0000","last_undegraded":"2023-12-26T22:52:39.820290+0000","last_fullsized":"2023-12-26T22:52:39.820290+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:03:42.705883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827905+0000","last_change":"2023-12-26T22:52:39.819345+0000","last_active":"2023-12-26T22:52:39.827905+0000","last_peered":"2023-12-26T22:52:39.827905+0000","last_clean":"2023-12-26T22:52:39.827905+0000","last_became_active":"2023-12-26T22:52:39.818164+0000","last_became_peered":"2023-12-26T22:52:39.818164+0000","last_unstale":"2023-12-26T22:52:39.827905+0000","last_undegraded":"2023-12-26T22:52:39.827905+0000","last_fullsized":"2023-12-26T22:52:39.827905+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:28.267144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830859+0000","last_change":"2023-12-26T22:52:39.828680+0000","last_active":"2023-12-26T22:52:39.830859+0000","last_peered":"2023-12-26T22:52:39.830859+0000","last_clean":"2023-12-26T22:52:39.830859+0000","last_became_active":"2023-12-26T22:52:39.828268+0000","last_became_peered":"2023-12-26T22:52:39.828268+0000","last_unstale":"2023-12-26T22:52:39.830859+0000","last_undegraded":"2023-12-26T22:52:39.830859+0000","last_fullsized":"2023-12-26T22:52:39.830859+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:29:59.360259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"160'4","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.843570+0000","last_change":"2023-12-26T22:52:38.806860+0000","last_active":"2023-12-26T22:52:39.843570+0000","last_peered":"2023-12-26T22:52:39.843570+0000","last_clean":"2023-12-26T22:52:39.843570+0000","last_became_active":"2023-12-26T22:52:38.806568+0000","last_became_peered":"2023-12-26T22:52:38.806568+0000","last_unstale":"2023-12-26T22:52:39.843570+0000","last_undegraded":"2023-12-26T22:52:39.843570+0000","last_fullsized":"2023-12-26T22:52:39.843570+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:27:33.958090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835179+0000","last_change":"2023-12-26T22:52:39.831602+0000","last_active":"2023-12-26T22:52:39.835179+0000","last_peered":"2023-12-26T22:52:39.835179+0000","last_clean":"2023-12-26T22:52:39.835179+0000","last_became_active":"2023-12-26T22:52:39.830106+0000","last_became_peered":"2023-12-26T22:52:39.830106+0000","last_unstale":"2023-12-26T22:52:39.835179+0000","last_undegraded":"2023-12-26T22:52:39.835179+0000","last_fullsized":"2023-12-26T22:52:39.835179+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:02:44.956745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.840698+0000","last_change":"2023-12-26T22:52:38.815087+0000","last_active":"2023-12-26T22:52:39.840698+0000","last_peered":"2023-12-26T22:52:39.840698+0000","last_clean":"2023-12-26T22:52:39.840698+0000","last_became_active":"2023-12-26T22:52:38.814815+0000","last_became_peered":"2023-12-26T22:52:38.814815+0000","last_unstale":"2023-12-26T22:52:39.840698+0000","last_undegraded":"2023-12-26T22:52:39.840698+0000","last_fullsized":"2023-12-26T22:52:39.840698+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:08:57.128212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835068+0000","last_change":"2023-12-26T22:52:39.832243+0000","last_active":"2023-12-26T22:52:39.835068+0000","last_peered":"2023-12-26T22:52:39.835068+0000","last_clean":"2023-12-26T22:52:39.835068+0000","last_became_active":"2023-12-26T22:52:39.831987+0000","last_became_peered":"2023-12-26T22:52:39.831987+0000","last_unstale":"2023-12-26T22:52:39.835068+0000","last_undegraded":"2023-12-26T22:52:39.835068+0000","last_fullsized":"2023-12-26T22:52:39.835068+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:45:54.661560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839828+0000","last_change":"2023-12-26T22:52:38.806358+0000","last_active":"2023-12-26T22:52:39.839828+0000","last_peered":"2023-12-26T22:52:39.839828+0000","last_clean":"2023-12-26T22:52:39.839828+0000","last_became_active":"2023-12-26T22:52:38.805477+0000","last_became_peered":"2023-12-26T22:52:38.805477+0000","last_unstale":"2023-12-26T22:52:39.839828+0000","last_undegraded":"2023-12-26T22:52:39.839828+0000","last_fullsized":"2023-12-26T22:52:39.839828+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:11:46.085192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835560+0000","last_change":"2023-12-26T22:52:39.828732+0000","last_active":"2023-12-26T22:52:39.835560+0000","last_peered":"2023-12-26T22:52:39.835560+0000","last_clean":"2023-12-26T22:52:39.835560+0000","last_became_active":"2023-12-26T22:52:39.828494+0000","last_became_peered":"2023-12-26T22:52:39.828494+0000","last_unstale":"2023-12-26T22:52:39.835560+0000","last_undegraded":"2023-12-26T22:52:39.835560+0000","last_fullsized":"2023-12-26T22:52:39.835560+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:10:42.560357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839564+0000","last_change":"2023-12-26T22:52:38.808323+0000","last_active":"2023-12-26T22:52:39.839564+0000","last_peered":"2023-12-26T22:52:39.839564+0000","last_clean":"2023-12-26T22:52:39.839564+0000","last_became_active":"2023-12-26T22:52:38.808070+0000","last_became_peered":"2023-12-26T22:52:38.808070+0000","last_unstale":"2023-12-26T22:52:39.839564+0000","last_undegraded":"2023-12-26T22:52:39.839564+0000","last_fullsized":"2023-12-26T22:52:39.839564+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:33:16.614000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829088+0000","last_change":"2023-12-26T22:52:39.820509+0000","last_active":"2023-12-26T22:52:39.829088+0000","last_peered":"2023-12-26T22:52:39.829088+0000","last_clean":"2023-12-26T22:52:39.829088+0000","last_became_active":"2023-12-26T22:52:39.819503+0000","last_became_peered":"2023-12-26T22:52:39.819503+0000","last_unstale":"2023-12-26T22:52:39.829088+0000","last_undegraded":"2023-12-26T22:52:39.829088+0000","last_fullsized":"2023-12-26T22:52:39.829088+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:25:51.946981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835140+0000","last_change":"2023-12-26T22:52:38.810524+0000","last_active":"2023-12-26T22:52:39.835140+0000","last_peered":"2023-12-26T22:52:39.835140+0000","last_clean":"2023-12-26T22:52:39.835140+0000","last_became_active":"2023-12-26T22:52:38.810220+0000","last_became_peered":"2023-12-26T22:52:38.810220+0000","last_unstale":"2023-12-26T22:52:39.835140+0000","last_undegraded":"2023-12-26T22:52:39.835140+0000","last_fullsized":"2023-12-26T22:52:39.835140+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:27:09.195955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835848+0000","last_change":"2023-12-26T22:52:39.829725+0000","last_active":"2023-12-26T22:52:39.835848+0000","last_peered":"2023-12-26T22:52:39.835848+0000","last_clean":"2023-12-26T22:52:39.835848+0000","last_became_active":"2023-12-26T22:52:39.829140+0000","last_became_peered":"2023-12-26T22:52:39.829140+0000","last_unstale":"2023-12-26T22:52:39.835848+0000","last_undegraded":"2023-12-26T22:52:39.835848+0000","last_fullsized":"2023-12-26T22:52:39.835848+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:01.395499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.837404+0000","last_change":"2023-12-26T22:52:38.806633+0000","last_active":"2023-12-26T22:52:39.837404+0000","last_peered":"2023-12-26T22:52:39.837404+0000","last_clean":"2023-12-26T22:52:39.837404+0000","last_became_active":"2023-12-26T22:52:38.806290+0000","last_became_peered":"2023-12-26T22:52:38.806290+0000","last_unstale":"2023-12-26T22:52:39.837404+0000","last_undegraded":"2023-12-26T22:52:39.837404+0000","last_fullsized":"2023-12-26T22:52:39.837404+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:24:26.280738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835282+0000","last_change":"2023-12-26T22:52:39.825551+0000","last_active":"2023-12-26T22:52:39.835282+0000","last_peered":"2023-12-26T22:52:39.835282+0000","last_clean":"2023-12-26T22:52:39.835282+0000","last_became_active":"2023-12-26T22:52:39.825173+0000","last_became_peered":"2023-12-26T22:52:39.825173+0000","last_unstale":"2023-12-26T22:52:39.835282+0000","last_undegraded":"2023-12-26T22:52:39.835282+0000","last_fullsized":"2023-12-26T22:52:39.835282+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:39:03.541353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.817091+0000","last_change":"2023-12-26T22:52:38.803339+0000","last_active":"2023-12-26T22:52:39.817091+0000","last_peered":"2023-12-26T22:52:39.817091+0000","last_clean":"2023-12-26T22:52:39.817091+0000","last_became_active":"2023-12-26T22:52:38.803004+0000","last_became_peered":"2023-12-26T22:52:38.803004+0000","last_unstale":"2023-12-26T22:52:39.817091+0000","last_undegraded":"2023-12-26T22:52:39.817091+0000","last_fullsized":"2023-12-26T22:52:39.817091+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:18.261824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829181+0000","last_change":"2023-12-26T22:52:39.821697+0000","last_active":"2023-12-26T22:52:39.829181+0000","last_peered":"2023-12-26T22:52:39.829181+0000","last_clean":"2023-12-26T22:52:39.829181+0000","last_became_active":"2023-12-26T22:52:39.821425+0000","last_became_peered":"2023-12-26T22:52:39.821425+0000","last_unstale":"2023-12-26T22:52:39.829181+0000","last_undegraded":"2023-12-26T22:52:39.829181+0000","last_fullsized":"2023-12-26T22:52:39.829181+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:35:53.548137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.807541+0000","last_change":"2023-12-26T22:52:38.807376+0000","last_active":"2023-12-26T22:52:39.807541+0000","last_peered":"2023-12-26T22:52:39.807541+0000","last_clean":"2023-12-26T22:52:39.807541+0000","last_became_active":"2023-12-26T22:52:38.807095+0000","last_became_peered":"2023-12-26T22:52:38.807095+0000","last_unstale":"2023-12-26T22:52:39.807541+0000","last_undegraded":"2023-12-26T22:52:39.807541+0000","last_fullsized":"2023-12-26T22:52:39.807541+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:47:32.892492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829837+0000","last_change":"2023-12-26T22:52:39.825099+0000","last_active":"2023-12-26T22:52:39.829837+0000","last_peered":"2023-12-26T22:52:39.829837+0000","last_clean":"2023-12-26T22:52:39.829837+0000","last_became_active":"2023-12-26T22:52:39.824467+0000","last_became_peered":"2023-12-26T22:52:39.824467+0000","last_unstale":"2023-12-26T22:52:39.829837+0000","last_undegraded":"2023-12-26T22:52:39.829837+0000","last_fullsized":"2023-12-26T22:52:39.829837+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:11:00.599172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.837812+0000","last_change":"2023-12-26T22:52:38.814539+0000","last_active":"2023-12-26T22:52:39.837812+0000","last_peered":"2023-12-26T22:52:39.837812+0000","last_clean":"2023-12-26T22:52:39.837812+0000","last_became_active":"2023-12-26T22:52:38.814146+0000","last_became_peered":"2023-12-26T22:52:38.814146+0000","last_unstale":"2023-12-26T22:52:39.837812+0000","last_undegraded":"2023-12-26T22:52:39.837812+0000","last_fullsized":"2023-12-26T22:52:39.837812+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:25:12.341665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.832295+0000","last_change":"2023-12-26T22:52:39.826311+0000","last_active":"2023-12-26T22:52:39.832295+0000","last_peered":"2023-12-26T22:52:39.832295+0000","last_clean":"2023-12-26T22:52:39.832295+0000","last_became_active":"2023-12-26T22:52:39.825969+0000","last_became_peered":"2023-12-26T22:52:39.825969+0000","last_unstale":"2023-12-26T22:52:39.832295+0000","last_undegraded":"2023-12-26T22:52:39.832295+0000","last_fullsized":"2023-12-26T22:52:39.832295+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:04:34.951035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.810433+0000","last_change":"2023-12-26T22:52:38.812350+0000","last_active":"2023-12-26T22:52:39.810433+0000","last_peered":"2023-12-26T22:52:39.810433+0000","last_clean":"2023-12-26T22:52:39.810433+0000","last_became_active":"2023-12-26T22:52:38.811753+0000","last_became_peered":"2023-12-26T22:52:38.811753+0000","last_unstale":"2023-12-26T22:52:39.810433+0000","last_undegraded":"2023-12-26T22:52:39.810433+0000","last_fullsized":"2023-12-26T22:52:39.810433+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:33:21.821521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.836591+0000","last_change":"2023-12-26T22:52:39.830070+0000","last_active":"2023-12-26T22:52:39.836591+0000","last_peered":"2023-12-26T22:52:39.836591+0000","last_clean":"2023-12-26T22:52:39.836591+0000","last_became_active":"2023-12-26T22:52:39.828908+0000","last_became_peered":"2023-12-26T22:52:39.828908+0000","last_unstale":"2023-12-26T22:52:39.836591+0000","last_undegraded":"2023-12-26T22:52:39.836591+0000","last_fullsized":"2023-12-26T22:52:39.836591+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:57:48.698783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"160'3","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.844604+0000","last_change":"2023-12-26T22:52:38.810927+0000","last_active":"2023-12-26T22:52:39.844604+0000","last_peered":"2023-12-26T22:52:39.844604+0000","last_clean":"2023-12-26T22:52:39.844604+0000","last_became_active":"2023-12-26T22:52:38.810723+0000","last_became_peered":"2023-12-26T22:52:38.810723+0000","last_unstale":"2023-12-26T22:52:39.844604+0000","last_undegraded":"2023-12-26T22:52:39.844604+0000","last_fullsized":"2023-12-26T22:52:39.844604+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:48:28.591405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.841233+0000","last_change":"2023-12-26T22:52:39.839005+0000","last_active":"2023-12-26T22:52:39.841233+0000","last_peered":"2023-12-26T22:52:39.841233+0000","last_clean":"2023-12-26T22:52:39.841233+0000","last_became_active":"2023-12-26T22:52:39.838719+0000","last_became_peered":"2023-12-26T22:52:39.838719+0000","last_unstale":"2023-12-26T22:52:39.841233+0000","last_undegraded":"2023-12-26T22:52:39.841233+0000","last_fullsized":"2023-12-26T22:52:39.841233+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:05:32.645093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"160'6","reported_seq":16,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.845257+0000","last_change":"2023-12-26T22:52:38.814433+0000","last_active":"2023-12-26T22:52:39.845257+0000","last_peered":"2023-12-26T22:52:39.845257+0000","last_clean":"2023-12-26T22:52:39.845257+0000","last_became_active":"2023-12-26T22:52:38.814178+0000","last_became_peered":"2023-12-26T22:52:38.814178+0000","last_unstale":"2023-12-26T22:52:39.845257+0000","last_undegraded":"2023-12-26T22:52:39.845257+0000","last_fullsized":"2023-12-26T22:52:39.845257+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:04:43.464355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829686+0000","last_change":"2023-12-26T22:52:39.826438+0000","last_active":"2023-12-26T22:52:39.829686+0000","last_peered":"2023-12-26T22:52:39.829686+0000","last_clean":"2023-12-26T22:52:39.829686+0000","last_became_active":"2023-12-26T22:52:39.826179+0000","last_became_peered":"2023-12-26T22:52:39.826179+0000","last_unstale":"2023-12-26T22:52:39.829686+0000","last_undegraded":"2023-12-26T22:52:39.829686+0000","last_fullsized":"2023-12-26T22:52:39.829686+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:28:36.449693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816573+0000","last_change":"2023-12-26T22:52:38.813005+0000","last_active":"2023-12-26T22:52:39.816573+0000","last_peered":"2023-12-26T22:52:39.816573+0000","last_clean":"2023-12-26T22:52:39.816573+0000","last_became_active":"2023-12-26T22:52:38.812682+0000","last_became_peered":"2023-12-26T22:52:38.812682+0000","last_unstale":"2023-12-26T22:52:39.816573+0000","last_undegraded":"2023-12-26T22:52:39.816573+0000","last_fullsized":"2023-12-26T22:52:39.816573+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:41.915923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827204+0000","last_change":"2023-12-26T22:52:39.823437+0000","last_active":"2023-12-26T22:52:39.827204+0000","last_peered":"2023-12-26T22:52:39.827204+0000","last_clean":"2023-12-26T22:52:39.827204+0000","last_became_active":"2023-12-26T22:52:39.822757+0000","last_became_peered":"2023-12-26T22:52:39.822757+0000","last_unstale":"2023-12-26T22:52:39.827204+0000","last_undegraded":"2023-12-26T22:52:39.827204+0000","last_fullsized":"2023-12-26T22:52:39.827204+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:03:42.705883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816772+0000","last_change":"2023-12-26T22:52:38.814992+0000","last_active":"2023-12-26T22:52:39.816772+0000","last_peered":"2023-12-26T22:52:39.816772+0000","last_clean":"2023-12-26T22:52:39.816772+0000","last_became_active":"2023-12-26T22:52:38.814440+0000","last_became_peered":"2023-12-26T22:52:38.814440+0000","last_unstale":"2023-12-26T22:52:39.816772+0000","last_undegraded":"2023-12-26T22:52:39.816772+0000","last_fullsized":"2023-12-26T22:52:39.816772+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:45:57.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835524+0000","last_change":"2023-12-26T22:52:39.826671+0000","last_active":"2023-12-26T22:52:39.835524+0000","last_peered":"2023-12-26T22:52:39.835524+0000","last_clean":"2023-12-26T22:52:39.835524+0000","last_became_active":"2023-12-26T22:52:39.826266+0000","last_became_peered":"2023-12-26T22:52:39.826266+0000","last_unstale":"2023-12-26T22:52:39.835524+0000","last_undegraded":"2023-12-26T22:52:39.835524+0000","last_fullsized":"2023-12-26T22:52:39.835524+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:08:05.207585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"160'8","reported_seq":18,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.846674+0000","last_change":"2023-12-26T22:52:38.808832+0000","last_active":"2023-12-26T22:52:39.846674+0000","last_peered":"2023-12-26T22:52:39.846674+0000","last_clean":"2023-12-26T22:52:39.846674+0000","last_became_active":"2023-12-26T22:52:38.808579+0000","last_became_peered":"2023-12-26T22:52:38.808579+0000","last_unstale":"2023-12-26T22:52:39.846674+0000","last_undegraded":"2023-12-26T22:52:39.846674+0000","last_fullsized":"2023-12-26T22:52:39.846674+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:19:51.678937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.826071+0000","last_change":"2023-12-26T22:52:39.820681+0000","last_active":"2023-12-26T22:52:39.826071+0000","last_peered":"2023-12-26T22:52:39.826071+0000","last_clean":"2023-12-26T22:52:39.826071+0000","last_became_active":"2023-12-26T22:52:39.820426+0000","last_became_peered":"2023-12-26T22:52:39.820426+0000","last_unstale":"2023-12-26T22:52:39.826071+0000","last_undegraded":"2023-12-26T22:52:39.826071+0000","last_fullsized":"2023-12-26T22:52:39.826071+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:40:28.894419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.844903+0000","last_change":"2023-12-26T22:52:38.810594+0000","last_active":"2023-12-26T22:52:39.844903+0000","last_peered":"2023-12-26T22:52:39.844903+0000","last_clean":"2023-12-26T22:52:39.844903+0000","last_became_active":"2023-12-26T22:52:38.810277+0000","last_became_peered":"2023-12-26T22:52:38.810277+0000","last_unstale":"2023-12-26T22:52:39.844903+0000","last_undegraded":"2023-12-26T22:52:39.844903+0000","last_fullsized":"2023-12-26T22:52:39.844903+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:48:59.823930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827471+0000","last_change":"2023-12-26T22:52:39.823481+0000","last_active":"2023-12-26T22:52:39.827471+0000","last_peered":"2023-12-26T22:52:39.827471+0000","last_clean":"2023-12-26T22:52:39.827471+0000","last_became_active":"2023-12-26T22:52:39.821403+0000","last_became_peered":"2023-12-26T22:52:39.821403+0000","last_unstale":"2023-12-26T22:52:39.827471+0000","last_undegraded":"2023-12-26T22:52:39.827471+0000","last_fullsized":"2023-12-26T22:52:39.827471+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:39:51.930454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.839904+0000","last_change":"2023-12-26T22:52:38.812266+0000","last_active":"2023-12-26T22:52:39.839904+0000","last_peered":"2023-12-26T22:52:39.839904+0000","last_clean":"2023-12-26T22:52:39.839904+0000","last_became_active":"2023-12-26T22:52:38.811998+0000","last_became_peered":"2023-12-26T22:52:38.811998+0000","last_unstale":"2023-12-26T22:52:39.839904+0000","last_undegraded":"2023-12-26T22:52:39.839904+0000","last_fullsized":"2023-12-26T22:52:39.839904+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:25:12.341665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829604+0000","last_change":"2023-12-26T22:52:39.825068+0000","last_active":"2023-12-26T22:52:39.829604+0000","last_peered":"2023-12-26T22:52:39.829604+0000","last_clean":"2023-12-26T22:52:39.829604+0000","last_became_active":"2023-12-26T22:52:39.824791+0000","last_became_peered":"2023-12-26T22:52:39.824791+0000","last_unstale":"2023-12-26T22:52:39.829604+0000","last_undegraded":"2023-12-26T22:52:39.829604+0000","last_fullsized":"2023-12-26T22:52:39.829604+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:18:01.162816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"160'1","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.841145+0000","last_change":"2023-12-26T22:52:38.806206+0000","last_active":"2023-12-26T22:52:39.841145+0000","last_peered":"2023-12-26T22:52:39.841145+0000","last_clean":"2023-12-26T22:52:39.841145+0000","last_became_active":"2023-12-26T22:52:38.805935+0000","last_became_peered":"2023-12-26T22:52:38.805935+0000","last_unstale":"2023-12-26T22:52:39.841145+0000","last_undegraded":"2023-12-26T22:52:39.841145+0000","last_fullsized":"2023-12-26T22:52:39.841145+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:14:07.043456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831353+0000","last_change":"2023-12-26T22:52:39.827749+0000","last_active":"2023-12-26T22:52:39.831353+0000","last_peered":"2023-12-26T22:52:39.831353+0000","last_clean":"2023-12-26T22:52:39.831353+0000","last_became_active":"2023-12-26T22:52:39.826618+0000","last_became_peered":"2023-12-26T22:52:39.826618+0000","last_unstale":"2023-12-26T22:52:39.831353+0000","last_undegraded":"2023-12-26T22:52:39.831353+0000","last_fullsized":"2023-12-26T22:52:39.831353+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:25:51.946981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.818459+0000","last_change":"2023-12-26T22:52:38.810348+0000","last_active":"2023-12-26T22:52:39.818459+0000","last_peered":"2023-12-26T22:52:39.818459+0000","last_clean":"2023-12-26T22:52:39.818459+0000","last_became_active":"2023-12-26T22:52:38.809954+0000","last_became_peered":"2023-12-26T22:52:38.809954+0000","last_unstale":"2023-12-26T22:52:39.818459+0000","last_undegraded":"2023-12-26T22:52:39.818459+0000","last_fullsized":"2023-12-26T22:52:39.818459+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:55:44.465224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.827636+0000","last_change":"2023-12-26T22:52:39.821415+0000","last_active":"2023-12-26T22:52:39.827636+0000","last_peered":"2023-12-26T22:52:39.827636+0000","last_clean":"2023-12-26T22:52:39.827636+0000","last_became_active":"2023-12-26T22:52:39.820875+0000","last_became_peered":"2023-12-26T22:52:39.820875+0000","last_unstale":"2023-12-26T22:52:39.827636+0000","last_undegraded":"2023-12-26T22:52:39.827636+0000","last_fullsized":"2023-12-26T22:52:39.827636+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:33:37.716984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.840914+0000","last_change":"2023-12-26T22:52:38.809885+0000","last_active":"2023-12-26T22:52:39.840914+0000","last_peered":"2023-12-26T22:52:39.840914+0000","last_clean":"2023-12-26T22:52:39.840914+0000","last_became_active":"2023-12-26T22:52:38.809635+0000","last_became_peered":"2023-12-26T22:52:38.809635+0000","last_unstale":"2023-12-26T22:52:39.840914+0000","last_undegraded":"2023-12-26T22:52:39.840914+0000","last_fullsized":"2023-12-26T22:52:39.840914+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:55:18.333216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.823518+0000","last_change":"2023-12-26T22:52:39.818870+0000","last_active":"2023-12-26T22:52:39.823518+0000","last_peered":"2023-12-26T22:52:39.823518+0000","last_clean":"2023-12-26T22:52:39.823518+0000","last_became_active":"2023-12-26T22:52:39.818671+0000","last_became_peered":"2023-12-26T22:52:39.818671+0000","last_unstale":"2023-12-26T22:52:39.823518+0000","last_undegraded":"2023-12-26T22:52:39.823518+0000","last_fullsized":"2023-12-26T22:52:39.823518+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:15.178959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.810082+0000","last_change":"2023-12-26T22:52:38.809776+0000","last_active":"2023-12-26T22:52:39.810082+0000","last_peered":"2023-12-26T22:52:39.810082+0000","last_clean":"2023-12-26T22:52:39.810082+0000","last_became_active":"2023-12-26T22:52:38.809482+0000","last_became_peered":"2023-12-26T22:52:38.809482+0000","last_unstale":"2023-12-26T22:52:39.810082+0000","last_undegraded":"2023-12-26T22:52:39.810082+0000","last_fullsized":"2023-12-26T22:52:39.810082+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:24:57.454731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829716+0000","last_change":"2023-12-26T22:52:39.827080+0000","last_active":"2023-12-26T22:52:39.829716+0000","last_peered":"2023-12-26T22:52:39.829716+0000","last_clean":"2023-12-26T22:52:39.829716+0000","last_became_active":"2023-12-26T22:52:39.823150+0000","last_became_peered":"2023-12-26T22:52:39.823150+0000","last_unstale":"2023-12-26T22:52:39.829716+0000","last_undegraded":"2023-12-26T22:52:39.829716+0000","last_fullsized":"2023-12-26T22:52:39.829716+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:22:23.610018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.814515+0000","last_change":"2023-12-26T22:52:38.815616+0000","last_active":"2023-12-26T22:52:39.814515+0000","last_peered":"2023-12-26T22:52:39.814515+0000","last_clean":"2023-12-26T22:52:39.814515+0000","last_became_active":"2023-12-26T22:52:38.814947+0000","last_became_peered":"2023-12-26T22:52:38.814947+0000","last_unstale":"2023-12-26T22:52:39.814515+0000","last_undegraded":"2023-12-26T22:52:39.814515+0000","last_fullsized":"2023-12-26T22:52:39.814515+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:44:43.034508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835225+0000","last_change":"2023-12-26T22:52:39.826508+0000","last_active":"2023-12-26T22:52:39.835225+0000","last_peered":"2023-12-26T22:52:39.835225+0000","last_clean":"2023-12-26T22:52:39.835225+0000","last_became_active":"2023-12-26T22:52:39.825995+0000","last_became_peered":"2023-12-26T22:52:39.825995+0000","last_unstale":"2023-12-26T22:52:39.835225+0000","last_undegraded":"2023-12-26T22:52:39.835225+0000","last_fullsized":"2023-12-26T22:52:39.835225+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:08:48.367833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.813598+0000","last_change":"2023-12-26T22:52:38.804764+0000","last_active":"2023-12-26T22:52:39.813598+0000","last_peered":"2023-12-26T22:52:39.813598+0000","last_clean":"2023-12-26T22:52:39.813598+0000","last_became_active":"2023-12-26T22:52:38.802950+0000","last_became_peered":"2023-12-26T22:52:38.802950+0000","last_unstale":"2023-12-26T22:52:39.813598+0000","last_undegraded":"2023-12-26T22:52:39.813598+0000","last_fullsized":"2023-12-26T22:52:39.813598+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:34.546507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.844054+0000","last_change":"2023-12-26T22:52:39.838171+0000","last_active":"2023-12-26T22:52:39.844054+0000","last_peered":"2023-12-26T22:52:39.844054+0000","last_clean":"2023-12-26T22:52:39.844054+0000","last_became_active":"2023-12-26T22:52:39.837896+0000","last_became_peered":"2023-12-26T22:52:39.837896+0000","last_unstale":"2023-12-26T22:52:39.844054+0000","last_undegraded":"2023-12-26T22:52:39.844054+0000","last_fullsized":"2023-12-26T22:52:39.844054+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:35.552674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.843398+0000","last_change":"2023-12-26T22:52:38.813554+0000","last_active":"2023-12-26T22:52:39.843398+0000","last_peered":"2023-12-26T22:52:39.843398+0000","last_clean":"2023-12-26T22:52:39.843398+0000","last_became_active":"2023-12-26T22:52:38.813064+0000","last_became_peered":"2023-12-26T22:52:38.813064+0000","last_unstale":"2023-12-26T22:52:39.843398+0000","last_undegraded":"2023-12-26T22:52:39.843398+0000","last_fullsized":"2023-12-26T22:52:39.843398+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:45:55.560511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.830982+0000","last_change":"2023-12-26T22:52:39.822333+0000","last_active":"2023-12-26T22:52:39.830982+0000","last_peered":"2023-12-26T22:52:39.830982+0000","last_clean":"2023-12-26T22:52:39.830982+0000","last_became_active":"2023-12-26T22:52:39.822057+0000","last_became_peered":"2023-12-26T22:52:39.822057+0000","last_unstale":"2023-12-26T22:52:39.830982+0000","last_undegraded":"2023-12-26T22:52:39.830982+0000","last_fullsized":"2023-12-26T22:52:39.830982+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:42:19.294384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.817553+0000","last_change":"2023-12-26T22:52:38.814408+0000","last_active":"2023-12-26T22:52:39.817553+0000","last_peered":"2023-12-26T22:52:39.817553+0000","last_clean":"2023-12-26T22:52:39.817553+0000","last_became_active":"2023-12-26T22:52:38.814109+0000","last_became_peered":"2023-12-26T22:52:38.814109+0000","last_unstale":"2023-12-26T22:52:39.817553+0000","last_undegraded":"2023-12-26T22:52:39.817553+0000","last_fullsized":"2023-12-26T22:52:39.817553+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:10:59.593004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.819818+0000","last_change":"2023-12-26T22:52:39.818053+0000","last_active":"2023-12-26T22:52:39.819818+0000","last_peered":"2023-12-26T22:52:39.819818+0000","last_clean":"2023-12-26T22:52:39.819818+0000","last_became_active":"2023-12-26T22:52:39.817771+0000","last_became_peered":"2023-12-26T22:52:39.817771+0000","last_unstale":"2023-12-26T22:52:39.819818+0000","last_undegraded":"2023-12-26T22:52:39.819818+0000","last_fullsized":"2023-12-26T22:52:39.819818+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:47:15.178959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.810250+0000","last_change":"2023-12-26T22:52:38.808077+0000","last_active":"2023-12-26T22:52:39.810250+0000","last_peered":"2023-12-26T22:52:39.810250+0000","last_clean":"2023-12-26T22:52:39.810250+0000","last_became_active":"2023-12-26T22:52:38.807769+0000","last_became_peered":"2023-12-26T22:52:38.807769+0000","last_unstale":"2023-12-26T22:52:39.810250+0000","last_undegraded":"2023-12-26T22:52:39.810250+0000","last_fullsized":"2023-12-26T22:52:39.810250+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:13:28.315292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.832083+0000","last_change":"2023-12-26T22:52:39.824274+0000","last_active":"2023-12-26T22:52:39.832083+0000","last_peered":"2023-12-26T22:52:39.832083+0000","last_clean":"2023-12-26T22:52:39.832083+0000","last_became_active":"2023-12-26T22:52:39.823980+0000","last_became_peered":"2023-12-26T22:52:39.823980+0000","last_unstale":"2023-12-26T22:52:39.832083+0000","last_undegraded":"2023-12-26T22:52:39.832083+0000","last_fullsized":"2023-12-26T22:52:39.832083+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:22.008831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.812021+0000","last_change":"2023-12-26T22:52:38.810987+0000","last_active":"2023-12-26T22:52:39.812021+0000","last_peered":"2023-12-26T22:52:39.812021+0000","last_clean":"2023-12-26T22:52:39.812021+0000","last_became_active":"2023-12-26T22:52:38.810686+0000","last_became_peered":"2023-12-26T22:52:38.810686+0000","last_unstale":"2023-12-26T22:52:39.812021+0000","last_undegraded":"2023-12-26T22:52:39.812021+0000","last_fullsized":"2023-12-26T22:52:39.812021+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:43.884256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.841126+0000","last_change":"2023-12-26T22:52:39.837844+0000","last_active":"2023-12-26T22:52:39.841126+0000","last_peered":"2023-12-26T22:52:39.841126+0000","last_clean":"2023-12-26T22:52:39.841126+0000","last_became_active":"2023-12-26T22:52:39.837571+0000","last_became_peered":"2023-12-26T22:52:39.837571+0000","last_unstale":"2023-12-26T22:52:39.841126+0000","last_undegraded":"2023-12-26T22:52:39.841126+0000","last_fullsized":"2023-12-26T22:52:39.841126+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:30:44.630844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.807598+0000","last_change":"2023-12-26T22:52:38.807859+0000","last_active":"2023-12-26T22:52:39.807598+0000","last_peered":"2023-12-26T22:52:39.807598+0000","last_clean":"2023-12-26T22:52:39.807598+0000","last_became_active":"2023-12-26T22:52:38.807571+0000","last_became_peered":"2023-12-26T22:52:38.807571+0000","last_unstale":"2023-12-26T22:52:39.807598+0000","last_undegraded":"2023-12-26T22:52:39.807598+0000","last_fullsized":"2023-12-26T22:52:39.807598+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:34:26.672326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835883+0000","last_change":"2023-12-26T22:52:39.832401+0000","last_active":"2023-12-26T22:52:39.835883+0000","last_peered":"2023-12-26T22:52:39.835883+0000","last_clean":"2023-12-26T22:52:39.835883+0000","last_became_active":"2023-12-26T22:52:39.832092+0000","last_became_peered":"2023-12-26T22:52:39.832092+0000","last_unstale":"2023-12-26T22:52:39.835883+0000","last_undegraded":"2023-12-26T22:52:39.835883+0000","last_fullsized":"2023-12-26T22:52:39.835883+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:07:28.230904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.835638+0000","last_change":"2023-12-26T22:52:38.812709+0000","last_active":"2023-12-26T22:52:39.835638+0000","last_peered":"2023-12-26T22:52:39.835638+0000","last_clean":"2023-12-26T22:52:39.835638+0000","last_became_active":"2023-12-26T22:52:38.811995+0000","last_became_peered":"2023-12-26T22:52:38.811995+0000","last_unstale":"2023-12-26T22:52:39.835638+0000","last_undegraded":"2023-12-26T22:52:39.835638+0000","last_fullsized":"2023-12-26T22:52:39.835638+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:45:50.223763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831432+0000","last_change":"2023-12-26T22:52:39.827794+0000","last_active":"2023-12-26T22:52:39.831432+0000","last_peered":"2023-12-26T22:52:39.831432+0000","last_clean":"2023-12-26T22:52:39.831432+0000","last_became_active":"2023-12-26T22:52:39.827510+0000","last_became_peered":"2023-12-26T22:52:39.827510+0000","last_unstale":"2023-12-26T22:52:39.831432+0000","last_undegraded":"2023-12-26T22:52:39.831432+0000","last_fullsized":"2023-12-26T22:52:39.831432+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:42:15.920688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816856+0000","last_change":"2023-12-26T22:52:38.815488+0000","last_active":"2023-12-26T22:52:39.816856+0000","last_peered":"2023-12-26T22:52:39.816856+0000","last_clean":"2023-12-26T22:52:39.816856+0000","last_became_active":"2023-12-26T22:52:38.814801+0000","last_became_peered":"2023-12-26T22:52:38.814801+0000","last_unstale":"2023-12-26T22:52:39.816856+0000","last_undegraded":"2023-12-26T22:52:39.816856+0000","last_fullsized":"2023-12-26T22:52:39.816856+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:48:59.823930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.829757+0000","last_change":"2023-12-26T22:52:39.826008+0000","last_active":"2023-12-26T22:52:39.829757+0000","last_peered":"2023-12-26T22:52:39.829757+0000","last_clean":"2023-12-26T22:52:39.829757+0000","last_became_active":"2023-12-26T22:52:39.822643+0000","last_became_peered":"2023-12-26T22:52:39.822643+0000","last_unstale":"2023-12-26T22:52:39.829757+0000","last_undegraded":"2023-12-26T22:52:39.829757+0000","last_fullsized":"2023-12-26T22:52:39.829757+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:17.910688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.816311+0000","last_change":"2023-12-26T22:52:38.812787+0000","last_active":"2023-12-26T22:52:39.816311+0000","last_peered":"2023-12-26T22:52:39.816311+0000","last_clean":"2023-12-26T22:52:39.816311+0000","last_became_active":"2023-12-26T22:52:38.812432+0000","last_became_peered":"2023-12-26T22:52:38.812432+0000","last_unstale":"2023-12-26T22:52:39.816311+0000","last_undegraded":"2023-12-26T22:52:39.816311+0000","last_fullsized":"2023-12-26T22:52:39.816311+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T22:53:58.318395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.831408+0000","last_change":"2023-12-26T22:52:39.823016+0000","last_active":"2023-12-26T22:52:39.831408+0000","last_peered":"2023-12-26T22:52:39.831408+0000","last_clean":"2023-12-26T22:52:39.831408+0000","last_became_active":"2023-12-26T22:52:39.822779+0000","last_became_peered":"2023-12-26T22:52:39.822779+0000","last_unstale":"2023-12-26T22:52:39.831408+0000","last_undegraded":"2023-12-26T22:52:39.831408+0000","last_fullsized":"2023-12-26T22:52:39.831408+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:38.783519+0000","last_clean_scrub_stamp":"2023-12-26T22:52:38.783519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:09:59.482783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"160'2","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.845299+0000","last_change":"2023-12-26T22:52:38.809342+0000","last_active":"2023-12-26T22:52:39.845299+0000","last_peered":"2023-12-26T22:52:39.845299+0000","last_clean":"2023-12-26T22:52:39.845299+0000","last_became_active":"2023-12-26T22:52:38.808972+0000","last_became_peered":"2023-12-26T22:52:38.808972+0000","last_unstale":"2023-12-26T22:52:39.845299+0000","last_undegraded":"2023-12-26T22:52:39.845299+0000","last_fullsized":"2023-12-26T22:52:39.845299+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:52:37.777351+0000","last_clean_scrub_stamp":"2023-12-26T22:52:37.777351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:47:17.900980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"150'271","reported_seq":836,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-12-26T22:52:39.806642+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T22:52:39.806642+0000","last_peered":"2023-12-26T22:52:39.806642+0000","last_clean":"2023-12-26T22:52:39.806642+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T22:52:39.806642+0000","last_undegraded":"2023-12-26T22:52:39.806642+0000","last_fullsized":"2023-12-26T22:52:39.806642+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543050,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":1952,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":1998848,"data_stored":992016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.71299999999999997},"min":{"1min":0.53500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.41999999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.68000000000000005}]},{"osd":1,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.42499999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.90200000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.73399999999999999},"min":{"1min":0.53400000000000003,"5min":0.46000000000000002,"15min":0.45700000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.41499999999999998},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.88200000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80300000000000005,"15min":0.79200000000000004},"min":{"1min":0.63600000000000001,"5min":0.58899999999999997,"15min":0.47899999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.42799999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.73499999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.80800000000000005}]},{"osd":5,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64400000000000002,"15min":0.628},"min":{"1min":0.44400000000000001,"5min":0.38900000000000001,"15min":0.34599999999999997},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.47799999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.94299999999999995}]},{"osd":6,"last update":"Tue Dec 26 22:51:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.44700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.39000000000000001},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":0.65600000000000003}]}]},{"osd":6,"up_from":15,"seq":64424510346,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307428,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064412,"statfs":{"total":96636764160,"available":96321957888,"internally_reserved":0,"allocated":1994752,"data_stored":991982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.98899999999999999},"last":0.876},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.92100000000000004,"5min":0.95399999999999996,"15min":0.98799999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.50600000000000001},"max":{"1min":0.93300000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.626}]},{"osd":2,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.95899999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77800000000000002},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.52800000000000002},"max":{"1min":0.95299999999999996,"5min":1.012,"15min":1.0249999999999999},"last":0.68500000000000005}]},{"osd":4,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":0.998,"15min":0.998},"last":0.505},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.48099999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.871,"5min":0.995,"15min":0.995},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42499999999999999},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70699999999999996,"15min":0.69799999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81499999999999995}]},{"osd":7,"last update":"Tue Dec 26 22:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.93999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97199999999999998,"5min":1.05,"15min":1.05},"last":0.90100000000000002}]}]},{"osd":1,"up_from":15,"seq":64424510346,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312120,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059720,"statfs":{"total":96636764160,"available":96317153280,"internally_reserved":0,"allocated":2605056,"data_stored":1583908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.47199999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63600000000000001,"5min":0.79900000000000004,"15min":0.88400000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.35999999999999999,"5min":0.32300000000000001,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.81000000000000005,"15min":0.84499999999999997},"last":0.45200000000000001}]},{"osd":2,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.32400000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.752},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73299999999999998,"5min":0.87,"15min":0.87},"last":0.46500000000000002}]},{"osd":3,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.38700000000000001},"max":{"1min":0.80600000000000005,"5min":0.877,"15min":0.90900000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.38200000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":0.97799999999999998,"5min":1.044,"15min":1.163},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.121},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.82299999999999995},"min":{"1min":0.69399999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.028,"5min":1.0820000000000001,"15min":1.1419999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83499999999999996,"15min":0.83199999999999996},"min":{"1min":0.68100000000000005,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.04,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.2450000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.84199999999999997,"15min":0.83999999999999997},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.1259999999999999,"15min":1.2270000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.55700000000000005},"max":{"1min":0.99299999999999999,"5min":1.107,"15min":1.2689999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.497},"max":{"1min":1.0529999999999999,"5min":1.0660000000000001,"15min":1.1890000000000001},"last":0.70099999999999996}]}]},{"osd":0,"up_from":15,"seq":64424510346,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307448,"kb_used_data":1968,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064392,"statfs":{"total":96636764160,"available":96321937408,"internally_reserved":0,"allocated":2015232,"data_stored":993574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.28199999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.63400000000000001,"5min":0.73799999999999999,"15min":0.86599999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.627,"5min":0.70499999999999996,"15min":0.88800000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72899999999999998,"5min":0.81899999999999995,"15min":0.90500000000000003},"last":0.39000000000000001}]},{"osd":3,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.35499999999999998},"max":{"1min":0.72699999999999998,"5min":0.80900000000000005,"15min":0.86799999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.51200000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.92200000000000004},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":1.0649999999999999,"5min":1.103,"15min":1.145},"last":0.75},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.65800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0640000000000001,"5min":1.1299999999999999,"15min":1.218},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.83299999999999996},"min":{"1min":0.65400000000000003,"5min":0.58099999999999996,"15min":0.51600000000000001},"max":{"1min":1.0489999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83799999999999997,"15min":0.83399999999999996},"min":{"1min":0.64300000000000002,"5min":0.56599999999999995,"15min":0.50900000000000001},"max":{"1min":1.081,"5min":1.218,"15min":1.218},"last":0.93100000000000005}]},{"osd":6,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.58999999999999997},"max":{"1min":1.0229999999999999,"5min":1.1499999999999999,"15min":1.1659999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80400000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.55500000000000005},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.2350000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Tue Dec 26 22:52:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82099999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.54500000000000004},"max":{"1min":1.073,"5min":1.165,"15min":1.254},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81200000000000006},"min":{"1min":0.65100000000000002,"5min":0.58799999999999997,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.256,"15min":1.3200000000000001},"last":0.87}]}]},{"osd":2,"up_from":14,"seq":60129543051,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307436,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064404,"statfs":{"total":96636764160,"available":96321949696,"internally_reserved":0,"allocated":2002944,"data_stored":992626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.29499999999999998},"max":{"1min":0.66700000000000004,"5min":1.075,"15min":1.075},"last":0.443},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.439,"5min":0.42499999999999999,"15min":0.378},"max":{"1min":0.73199999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32900000000000001},"max":{"1min":0.751,"5min":1.079,"15min":1.079},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.501},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34100000000000003},"max":{"1min":0.65900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.34100000000000003}]},{"osd":3,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":1.129,"15min":1.2490000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.83899999999999997,"5min":0.84999999999999998,"15min":1.198},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.626,"5min":0.59199999999999997,"15min":0.52500000000000002},"max":{"1min":1.177,"5min":1.222,"15min":1.222},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.50600000000000001},"max":{"1min":1.044,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.48499999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.22},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82799999999999996,"15min":0.80900000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57099999999999995},"max":{"1min":1.095,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.78800000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.56100000000000005},"max":{"1min":1.075,"5min":1.075,"15min":1.2470000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.67600000000000005,"5min":0.625,"15min":0.58699999999999997},"max":{"1min":1.0649999999999999,"5min":1.298,"15min":1.298},"last":0.80800000000000005}]},{"osd":7,"last update":"Tue Dec 26 22:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":1.0389999999999999,"5min":1.056,"15min":1.1899999999999999},"last":0.754},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.84299999999999997},"min":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2350000000000001,"5min":1.367,"15min":1.367},"last":0.70199999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510347,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299240,"kb_used_data":1952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072600,"statfs":{"total":96636764160,"available":96330342400,"internally_reserved":0,"allocated":1998848,"data_stored":992052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28000000000000003},"max":{"1min":0.64600000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.93799999999999994},"last":0.44400000000000001}]},{"osd":1,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":1.0449999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.379},"max":{"1min":0.752,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.622,"15min":0.625},"min":{"1min":0.441,"5min":0.441,"15min":0.40000000000000002},"max":{"1min":0.73799999999999999,"5min":1.516,"15min":1.516},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.49399999999999999,"5min":0.47599999999999998,"15min":0.41999999999999998},"max":{"1min":0.879,"5min":1.395,"15min":1.395},"last":0.502}]},{"osd":4,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.03,"5min":1.413,"15min":1.413},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79800000000000004,"15min":0.78200000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46999999999999997},"max":{"1min":1.0980000000000001,"5min":1.163,"15min":1.1850000000000001},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.60099999999999998,"5min":0.54500000000000004,"15min":0.52200000000000002},"max":{"1min":1.002,"5min":1.054,"15min":1.1359999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82099999999999995,"15min":0.79900000000000004},"min":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.53100000000000003},"max":{"1min":1.054,"5min":1.202,"15min":1.2270000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.66100000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.0880000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60399999999999998},"max":{"1min":1.101,"5min":1.1140000000000001,"15min":1.165},"last":0.84499999999999997}]},{"osd":7,"last update":"Tue Dec 26 22:51:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82399999999999995,"15min":0.81999999999999995},"min":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.55300000000000005},"max":{"1min":0.998,"5min":1.321,"15min":1.321},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.58299999999999996,"5min":0.56499999999999995,"15min":0.52000000000000002},"max":{"1min":1.0780000000000001,"5min":1.466,"15min":1.466},"last":0.89200000000000002}]}]},{"osd":4,"up_from":14,"seq":60129543050,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311524,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060316,"statfs":{"total":96636764160,"available":96317763584,"internally_reserved":0,"allocated":1994752,"data_stored":991982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.52300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.90400000000000003,"15min":0.99299999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.99199999999999999,"15min":1.0640000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53900000000000003,"5min":0.48699999999999999,"15min":0.47099999999999997},"max":{"1min":0.90500000000000003,"5min":1.01,"15min":1.0309999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72999999999999998,"15min":0.73799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90000000000000002,"5min":0.98599999999999999,"15min":1.04},"last":0.64300000000000002}]},{"osd":2,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":0.90800000000000003,"5min":0.93400000000000005,"15min":0.98599999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96399999999999997,"5min":0.97799999999999998,"15min":0.99399999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.46300000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75800000000000001,"15min":0.76700000000000002},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.503},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.97399999999999998},"last":0.79900000000000004}]},{"osd":6,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.35699999999999998},"max":{"1min":0.87,"5min":0.94799999999999995,"15min":0.98699999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":1.0149999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Dec 26 22:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.44900000000000001},"max":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":1.0449999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.751}]}]},{"osd":5,"up_from":15,"seq":64424510345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312116,"kb_used_data":2540,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059724,"statfs":{"total":96636764160,"available":96317157376,"internally_reserved":0,"allocated":2600960,"data_stored":1583064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40100000000000002},"max":{"1min":0.96199999999999997,"5min":1.071,"15min":1.071},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.499},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.58199999999999996,"5min":0.50600000000000001,"15min":0.46200000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.62},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.2269999999999999,"5min":3.2269999999999999,"15min":3.2269999999999999},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91500000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.75600000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9199999999999999,"5min":2.9199999999999999,"15min":2.9199999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.66600000000000004}]},{"osd":6,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.0169999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.48499999999999999,"15min":0.45300000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.751}]},{"osd":7,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95099999999999996,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T22:52:49.488 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T22:52:49.488 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T22:52:49.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.720+0000 7f539bdff700 1 Processor -- start 2023-12-26T22:52:49.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.720+0000 7f539bdff700 1 -- start start 2023-12-26T22:52:49.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.721+0000 7f539bdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5394159a20 0x7f53940ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.721+0000 7f539bdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53940ccbc0 con 0x7f53940cf260 2023-12-26T22:52:49.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.721+0000 7f539bdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53940cf6a0 con 0x7f5394159650 2023-12-26T22:52:49.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.721+0000 7f539bdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53940cbfb0 con 0x7f5394159a20 2023-12-26T22:52:49.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.722+0000 7f539a39c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f53940cf260 0x7f53940c55d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41040/0 (socket says 172.21.15.131:41040) 2023-12-26T22:52:49.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.722+0000 7f539a39c700 1 -- 172.21.15.131:0/3374921605 learned_addr learned my addr 172.21.15.131:0/3374921605 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.722+0000 7f539939a700 1 --2- 172.21.15.131:0/3374921605 >> v2:172.21.15.190:3300/0 conn(0x7f5394159a20 0x7f53940ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.722+0000 7f5398b99700 1 -- 172.21.15.131:0/3374921605 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621701701 0 0) 0x7f53940ccbc0 con 0x7f53940cf260 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.722+0000 7f5398b99700 1 -- 172.21.15.131:0/3374921605 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5380003570 con 0x7f53940cf260 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.722+0000 7f539939a700 1 -- 172.21.15.131:0/3374921605 >> v1:172.21.15.190:6789/0 conn(0x7f5394159650 legacy=0x7f53940cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.722+0000 7f539939a700 1 -- 172.21.15.131:0/3374921605 >> v1:172.21.15.131:6789/0 conn(0x7f53940cf260 legacy=0x7f53940c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.722+0000 7f539939a700 1 -- 172.21.15.131:0/3374921605 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940cc290 con 0x7f5394159a20 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.722+0000 7f539939a700 1 --2- 172.21.15.131:0/3374921605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5394159a20 0x7f53940ced20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f53840098e0 tx=0x7f53840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53eef5078ab1c0e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f5398b99700 1 -- 172.21.15.131:0/3374921605 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538400b640 con 0x7f5394159a20 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f5398b99700 1 -- 172.21.15.131:0/3374921605 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538400b7a0 con 0x7f5394159a20 2023-12-26T22:52:49.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f5398b99700 1 -- 172.21.15.131:0/3374921605 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538400bb40 con 0x7f5394159a20 2023-12-26T22:52:49.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f539bdff700 1 -- 172.21.15.131:0/3374921605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5394159a20 msgr2=0x7f53940ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f539bdff700 1 --2- 172.21.15.131:0/3374921605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5394159a20 0x7f53940ced20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f53840098e0 tx=0x7f53840091a0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f539bdff700 1 -- 172.21.15.131:0/3374921605 shutdown_connections 2023-12-26T22:52:49.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f539bdff700 1 --2- 172.21.15.131:0/3374921605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5394159a20 0x7f53940ced20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f539bdff700 1 -- 172.21.15.131:0/3374921605 >> 172.21.15.131:0/3374921605 conn(0x7f5394099980 msgr2=0x7f539406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:49.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f539bdff700 1 -- 172.21.15.131:0/3374921605 shutdown_connections 2023-12-26T22:52:49.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.723+0000 7f539bdff700 1 -- 172.21.15.131:0/3374921605 wait complete. 2023-12-26T22:52:49.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.725+0000 7f539bdff700 1 Processor -- start 2023-12-26T22:52:49.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.725+0000 7f539bdff700 1 -- start start 2023-12-26T22:52:49.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f539bdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53940cf260 0x7f539416a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f539bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5394159650 0x7f539409d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f539bdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5394159a20 0x7f5394156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f539bdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539409d040 con 0x7f53940cf260 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f539bdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539409a4b0 con 0x7f5394159a20 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f539bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539416ae50 con 0x7f5394159650 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f5399b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53940cf260 0x7f539416a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f5399b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53940cf260 0x7f539416a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35608/0 (socket says 172.21.15.131:35608) 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f5399b9b700 1 -- 172.21.15.131:0/1517374098 learned_addr learned my addr 172.21.15.131:0/1517374098 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f539a39c700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5394159a20 0x7f5394156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f539939a700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5394159650 0x7f539409d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f5399b9b700 1 -- 172.21.15.131:0/1517374098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5394159650 msgr2=0x7f539409d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.726+0000 7f5399b9b700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5394159650 0x7f539409d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f5399b9b700 1 -- 172.21.15.131:0/1517374098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5394159a20 msgr2=0x7f5394156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f5399b9b700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5394159a20 0x7f5394156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f5399b9b700 1 -- 172.21.15.131:0/1517374098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5384009420 con 0x7f53940cf260 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f539939a700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5394159650 0x7f539409d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f5399b9b700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53940cf260 0x7f539416a910 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f539000c7c0 tx=0x7f5390007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f5389ffb700 1 -- 172.21.15.131:0/1517374098 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390011590 con 0x7f53940cf260 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f5389ffb700 1 -- 172.21.15.131:0/1517374098 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5390011710 con 0x7f53940cf260 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f5389ffb700 1 -- 172.21.15.131:0/1517374098 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390011ad0 con 0x7f53940cf260 2023-12-26T22:52:49.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539409c430 con 0x7f53940cf260 2023-12-26T22:52:49.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.727+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539409c9f0 con 0x7f53940cf260 2023-12-26T22:52:49.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.728+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53940657e0 con 0x7f53940cf260 2023-12-26T22:52:49.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.729+0000 7f5389ffb700 1 -- 172.21.15.131:0/1517374098 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f539001b3b0 con 0x7f53940cf260 2023-12-26T22:52:49.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.730+0000 7f5389ffb700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5378066f30 0x7f53780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:49.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.730+0000 7f5389ffb700 1 -- 172.21.15.131:0/1517374098 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f5390083cb0 con 0x7f53940cf260 2023-12-26T22:52:49.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.730+0000 7f539939a700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5378066f30 0x7f53780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:49.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.731+0000 7f539939a700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5378066f30 0x7f53780693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53840098e0 tx=0x7f5384017000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:49.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:49.734+0000 7f5389ffb700 1 -- 172.21.15.131:0/1517374098 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5390050d60 con 0x7f53940cf260 2023-12-26T22:52:50.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.001+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f53940cc570 con 0x7f53940cf260 2023-12-26T22:52:50.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.006+0000 7f5389ffb700 1 -- 172.21.15.131:0/1517374098 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v41) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5390017070 con 0x7f53940cf260 2023-12-26T22:52:50.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5378066f30 msgr2=0x7f53780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:50.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5378066f30 0x7f53780693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53840098e0 tx=0x7f5384017000 comp rx=0 tx=0).stop 2023-12-26T22:52:50.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53940cf260 msgr2=0x7f539416a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:50.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53940cf260 0x7f539416a910 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f539000c7c0 tx=0x7f5390007f70 comp rx=0 tx=0).stop 2023-12-26T22:52:50.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 shutdown_connections 2023-12-26T22:52:50.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5394159a20 0x7f5394156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5378066f30 0x7f53780693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53940cf260 0x7f539416a910 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 --2- 172.21.15.131:0/1517374098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5394159650 0x7f539409d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 >> 172.21.15.131:0/1517374098 conn(0x7f5394099980 msgr2=0x7f53940ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:50.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 shutdown_connections 2023-12-26T22:52:50.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.008+0000 7f539bdff700 1 -- 172.21.15.131:0/1517374098 wait complete. 2023-12-26T22:52:50.008 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T22:52:50.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:52:50.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.218+0000 7f2da67ff700 1 Processor -- start 2023-12-26T22:52:50.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.219+0000 7f2da67ff700 1 -- start start 2023-12-26T22:52:50.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2da67ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2da006bfd0 0x7f2da00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:50.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2da67ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da00cc100 0x7f2da00cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:50.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2da67ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da00686c0 con 0x7f2da013c510 2023-12-26T22:52:50.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2da67ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da0069520 con 0x7f2da006bfd0 2023-12-26T22:52:50.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2da67ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0068910 con 0x7f2da00cc100 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2d9f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2da006bfd0 0x7f2da00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2d9f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2da006bfd0 0x7f2da00c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35618/0 (socket says 172.21.15.131:35618) 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2d9ffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2da013c510 0x7f2da006b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41054/0 (socket says 172.21.15.131:41054) 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2d9f7fe700 1 -- 172.21.15.131:0/2759541280 learned_addr learned my addr 172.21.15.131:0/2759541280 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.220+0000 7f2da4d9c700 1 --2- 172.21.15.131:0/2759541280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da00cc100 0x7f2da00cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.221+0000 7f2d9f7fe700 1 -- 172.21.15.131:0/2759541280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da00cc100 msgr2=0x7f2da00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.221+0000 7f2d9f7fe700 1 --2- 172.21.15.131:0/2759541280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da00cc100 0x7f2da00cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.221+0000 7f2d9f7fe700 1 -- 172.21.15.131:0/2759541280 >> v1:172.21.15.131:6789/0 conn(0x7f2da013c510 legacy=0x7f2da006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.221+0000 7f2d9f7fe700 1 -- 172.21.15.131:0/2759541280 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0068bf0 con 0x7f2da006bfd0 2023-12-26T22:52:50.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.221+0000 7f2d9effd700 1 -- 172.21.15.131:0/2759541280 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208984420 0 0) 0x7f2da00686c0 con 0x7f2da013c510 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.221+0000 7f2d9f7fe700 1 --2- 172.21.15.131:0/2759541280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 0x7f2da00c8a20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2d940099b0 tx=0x7f2d94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c095623c1c766f60 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.221+0000 7f2d9effd700 1 -- 172.21.15.131:0/2759541280 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400b860 con 0x7f2da006bfd0 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.221+0000 7f2d9effd700 1 -- 172.21.15.131:0/2759541280 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9400b9c0 con 0x7f2da006bfd0 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.221+0000 7f2d9effd700 1 -- 172.21.15.131:0/2759541280 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400bd60 con 0x7f2da006bfd0 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.222+0000 7f2da67ff700 1 -- 172.21.15.131:0/2759541280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 msgr2=0x7f2da00c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.222+0000 7f2da67ff700 1 --2- 172.21.15.131:0/2759541280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 0x7f2da00c8a20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2d940099b0 tx=0x7f2d94009250 comp rx=0 tx=0).stop 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.222+0000 7f2da67ff700 1 -- 172.21.15.131:0/2759541280 shutdown_connections 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.222+0000 7f2da67ff700 1 --2- 172.21.15.131:0/2759541280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da00cc100 0x7f2da00cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.222+0000 7f2da67ff700 1 --2- 172.21.15.131:0/2759541280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 0x7f2da00c8a20 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.222+0000 7f2da67ff700 1 -- 172.21.15.131:0/2759541280 >> 172.21.15.131:0/2759541280 conn(0x7f2da00ce3e0 msgr2=0x7f2da00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:50.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.222+0000 7f2da67ff700 1 -- 172.21.15.131:0/2759541280 shutdown_connections 2023-12-26T22:52:50.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.222+0000 7f2da67ff700 1 -- 172.21.15.131:0/2759541280 wait complete. 2023-12-26T22:52:50.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.223+0000 7f2da67ff700 1 Processor -- start 2023-12-26T22:52:50.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.223+0000 7f2da67ff700 1 -- start start 2023-12-26T22:52:50.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.224+0000 7f2da67ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 0x7f2da0153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:50.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.224+0000 7f2da67ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da00cc100 0x7f2da0153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:50.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.224+0000 7f2da67ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da013c510 0x7f2da016daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:50.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.224+0000 7f2da67ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0153dc0 con 0x7f2da006bfd0 2023-12-26T22:52:50.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.224+0000 7f2da67ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0156bb0 con 0x7f2da00cc100 2023-12-26T22:52:50.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.224+0000 7f2da67ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0156e90 con 0x7f2da013c510 2023-12-26T22:52:50.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.224+0000 7f2d9ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 0x7f2da0153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.224+0000 7f2d9ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 0x7f2da0153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35634/0 (socket says 172.21.15.131:35634) 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d9ffff700 1 -- 172.21.15.131:0/3641396000 learned_addr learned my addr 172.21.15.131:0/3641396000 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d9f7fe700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da00cc100 0x7f2da0153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2da4d9c700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da013c510 0x7f2da016daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d9ffff700 1 -- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da013c510 msgr2=0x7f2da016daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d9ffff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da013c510 0x7f2da016daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d9ffff700 1 -- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da00cc100 msgr2=0x7f2da0153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d9ffff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da00cc100 0x7f2da0153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d9ffff700 1 -- 172.21.15.131:0/3641396000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d940094f0 con 0x7f2da006bfd0 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2da4d9c700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da013c510 0x7f2da016daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:50.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d9f7fe700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da00cc100 0x7f2da0153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d9ffff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 0x7f2da0153340 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000c810 tx=0x7f2d90007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d87fff700 1 -- 172.21.15.131:0/3641396000 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90007300 con 0x7f2da006bfd0 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d87fff700 1 -- 172.21.15.131:0/3641396000 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90007460 con 0x7f2da006bfd0 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da00b1220 con 0x7f2da006bfd0 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2d87fff700 1 -- 172.21.15.131:0/3641396000 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d900057f0 con 0x7f2da006bfd0 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.225+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da00b17e0 con 0x7f2da006bfd0 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.226+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da00657e0 con 0x7f2da006bfd0 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.227+0000 7f2d87fff700 1 -- 172.21.15.131:0/3641396000 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2d90005bc0 con 0x7f2da006bfd0 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.228+0000 7f2d87fff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d8c066f30 0x7f2d8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:50.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.228+0000 7f2d87fff700 1 -- 172.21.15.131:0/3641396000 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f2d900842b0 con 0x7f2da006bfd0 2023-12-26T22:52:50.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.228+0000 7f2d9f7fe700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d8c066f30 0x7f2d8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:50.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.229+0000 7f2d9f7fe700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d8c066f30 0x7f2d8c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2d940099b0 tx=0x7f2d94017000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:50.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.233+0000 7f2d87fff700 1 -- 172.21.15.131:0/3641396000 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d90051330 con 0x7f2da006bfd0 2023-12-26T22:52:50.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.473+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2da00b19d0 con 0x7f2da006bfd0 2023-12-26T22:52:50.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.473+0000 7f2d87fff700 1 -- 172.21.15.131:0/3641396000 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f2d90054950 con 0x7f2da006bfd0 2023-12-26T22:52:50.472 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:52:50.472 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2085575956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2085575956},{"type":"v1","addr":"172.21.15.190:6834","nonce":2085575956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2129957224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2129957224},{"type":"v1","addr":"172.21.15.190:6835","nonce":2129957224}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3810273571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3810273571},{"type":"v1","addr":"172.21.15.131:6834","nonce":3810273571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:52:39.271885+0000","modified":"2023-12-26T22:52:42.824355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8628":{"gid":8628,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2240171052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2240171052},{"type":"v1","addr":"172.21.15.131:6835","nonce":2240171052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-12-26T22:52:50.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d8c066f30 msgr2=0x7f2d8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d8c066f30 0x7f2d8c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2d940099b0 tx=0x7f2d94017000 comp rx=0 tx=0).stop 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 msgr2=0x7f2da0153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 0x7f2da0153340 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000c810 tx=0x7f2d90007d20 comp rx=0 tx=0).stop 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 shutdown_connections 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da013c510 0x7f2da016daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f2d8c066f30 0x7f2d8c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2da006bfd0 0x7f2da0153340 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 --2- 172.21.15.131:0/3641396000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da00cc100 0x7f2da0153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 >> 172.21.15.131:0/3641396000 conn(0x7f2da00ce3e0 msgr2=0x7f2da013aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 shutdown_connections 2023-12-26T22:52:50.475 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:50.476+0000 7f2da67ff700 1 -- 172.21.15.131:0/3641396000 wait complete. 2023-12-26T22:52:50.476 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 73 2023-12-26T22:52:50.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T22:52:39.271885+0000', 'modified': '2023-12-26T22:52:42.824355+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8628': {'gid': 8628, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6835/2240171052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 2240171052}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 2240171052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T22:52:50.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T22:52:50.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-26T22:52:51.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.087+0000 7f93279ff700 1 Processor -- start 2023-12-26T22:52:51.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.087+0000 7f93279ff700 1 -- start start 2023-12-26T22:52:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f93279ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932015d9b0 0x7f932015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f93279ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93200b07f0 con 0x7f93200cc580 2023-12-26T22:52:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f93279ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93200b0d70 con 0x7f93200cc950 2023-12-26T22:52:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f93279ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93200680f0 con 0x7f932015d9b0 2023-12-26T22:52:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f932579b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f93200cc580 0x7f93200b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41068/0 (socket says 172.21.15.131:41068) 2023-12-26T22:52:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f932579b700 1 -- 172.21.15.131:0/2666361411 learned_addr learned my addr 172.21.15.131:0/2666361411 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f9325f9c700 1 --2- 172.21.15.131:0/2666361411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932015d9b0 0x7f932015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f9317fff700 1 -- 172.21.15.131:0/2666361411 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199000256 0 0) 0x7f931c001110 con 0x7f93200cc580 2023-12-26T22:52:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f9317fff700 1 -- 172.21.15.131:0/2666361411 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f930c003570 con 0x7f93200cc580 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f9317fff700 1 -- 172.21.15.131:0/2666361411 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639962765 0 0) 0x7f93200b0d70 con 0x7f93200cc950 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.089+0000 7f9317fff700 1 -- 172.21.15.131:0/2666361411 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f931c001110 con 0x7f93200cc950 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.090+0000 7f9325f9c700 1 -- 172.21.15.131:0/2666361411 >> v1:172.21.15.190:6789/0 conn(0x7f93200cc950 legacy=0x7f932006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.090+0000 7f9325f9c700 1 -- 172.21.15.131:0/2666361411 >> v1:172.21.15.131:6789/0 conn(0x7f93200cc580 legacy=0x7f93200b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.090+0000 7f9325f9c700 1 -- 172.21.15.131:0/2666361411 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200675b0 con 0x7f932015d9b0 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.090+0000 7f9325f9c700 1 --2- 172.21.15.131:0/2666361411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932015d9b0 0x7f932015de30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9318009e90 tx=0x7f9318009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c2dd8f39b78f93 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.090+0000 7f9317fff700 1 -- 172.21.15.131:0/2666361411 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931800e070 con 0x7f932015d9b0 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.090+0000 7f9317fff700 1 -- 172.21.15.131:0/2666361411 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931800bb80 con 0x7f932015d9b0 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.090+0000 7f9317fff700 1 -- 172.21.15.131:0/2666361411 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318012430 con 0x7f932015d9b0 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.091+0000 7f93279ff700 1 -- 172.21.15.131:0/2666361411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932015d9b0 msgr2=0x7f932015de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.091+0000 7f93279ff700 1 --2- 172.21.15.131:0/2666361411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932015d9b0 0x7f932015de30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9318009e90 tx=0x7f9318009780 comp rx=0 tx=0).stop 2023-12-26T22:52:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.091+0000 7f93279ff700 1 -- 172.21.15.131:0/2666361411 shutdown_connections 2023-12-26T22:52:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.091+0000 7f93279ff700 1 --2- 172.21.15.131:0/2666361411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932015d9b0 0x7f932015de30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.091+0000 7f93279ff700 1 -- 172.21.15.131:0/2666361411 >> 172.21.15.131:0/2666361411 conn(0x7f93200686c0 msgr2=0x7f9320068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.091+0000 7f93279ff700 1 -- 172.21.15.131:0/2666361411 shutdown_connections 2023-12-26T22:52:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.091+0000 7f93279ff700 1 -- 172.21.15.131:0/2666361411 wait complete. 2023-12-26T22:52:51.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.093+0000 7f93279ff700 1 Processor -- start 2023-12-26T22:52:51.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.093+0000 7f93279ff700 1 -- start start 2023-12-26T22:52:51.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f93279ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93200cc580 0x7f9320159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f93279ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93200cc950 0x7f932013bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f93279ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932015d9b0 0x7f932016a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f93279ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932015c6c0 con 0x7f932015d9b0 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f93279ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932013c2e0 con 0x7f93200cc950 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f93279ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932015a490 con 0x7f93200cc580 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f9325f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932015d9b0 0x7f932016a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f932579b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93200cc580 0x7f9320159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f9325f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932015d9b0 0x7f932016a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35646/0 (socket says 172.21.15.131:35646) 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f932579b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93200cc580 0x7f9320159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:40938/0 (socket says 172.21.15.131:40938) 2023-12-26T22:52:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f9325f9c700 1 -- 172.21.15.131:0/3250058627 learned_addr learned my addr 172.21.15.131:0/3250058627 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f9324f9a700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93200cc950 0x7f932013bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f9325f9c700 1 -- 172.21.15.131:0/3250058627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93200cc580 msgr2=0x7f9320159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f9325f9c700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93200cc580 0x7f9320159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f9325f9c700 1 -- 172.21.15.131:0/3250058627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93200cc950 msgr2=0x7f932013bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f9325f9c700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93200cc950 0x7f932013bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.094+0000 7f9325f9c700 1 -- 172.21.15.131:0/3250058627 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9318009a20 con 0x7f932015d9b0 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.095+0000 7f9325f9c700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932015d9b0 0x7f932016a870 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f931800b610 tx=0x7f9318019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.095+0000 7f93157fa700 1 -- 172.21.15.131:0/3250058627 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318026480 con 0x7f932015d9b0 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.095+0000 7f93157fa700 1 -- 172.21.15.131:0/3250058627 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9318026600 con 0x7f932015d9b0 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.095+0000 7f93279ff700 1 -- 172.21.15.131:0/3250058627 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93200ae7f0 con 0x7f932015d9b0 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.095+0000 7f93157fa700 1 -- 172.21.15.131:0/3250058627 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180269c0 con 0x7f932015d9b0 2023-12-26T22:52:51.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.095+0000 7f93279ff700 1 -- 172.21.15.131:0/3250058627 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932016adb0 con 0x7f932015d9b0 2023-12-26T22:52:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.097+0000 7f93157fa700 1 -- 172.21.15.131:0/3250058627 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f931801c400 con 0x7f932015d9b0 2023-12-26T22:52:51.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.098+0000 7f93157fa700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9304066f30 0x7f93040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.098+0000 7f93157fa700 1 -- 172.21.15.131:0/3250058627 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f9318086600 con 0x7f932015d9b0 2023-12-26T22:52:51.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.098+0000 7f932579b700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9304066f30 0x7f93040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.099+0000 7f932579b700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9304066f30 0x7f93040693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f931c007fd0 tx=0x7f931c009040 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.100+0000 7f93360fbb80 1 Processor -- start 2023-12-26T22:52:51.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.100+0000 7f93360fbb80 1 -- start start 2023-12-26T22:52:51.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07afa0 0x55643a07b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07b8b0 0x55643a076c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a077170 0x55643a097360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f93360fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556439fa93e0 con 0x55643a07b8b0 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f93360fbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556439fa6be0 con 0x55643a077170 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f93360fbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556439fa29d0 con 0x55643a07afa0 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f9324f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07b8b0 0x55643a076c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f9324f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07b8b0 0x55643a076c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35652/0 (socket says 172.21.15.131:35652) 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f9324f9a700 1 -- 172.21.15.131:0/4047461327 learned_addr learned my addr 172.21.15.131:0/4047461327 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f932579b700 1 --2- 172.21.15.131:0/4047461327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07afa0 0x55643a07b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f9325f9c700 1 --2- 172.21.15.131:0/4047461327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a077170 0x55643a097360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f9324f9a700 1 -- 172.21.15.131:0/4047461327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07afa0 msgr2=0x55643a07b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f9324f9a700 1 --2- 172.21.15.131:0/4047461327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07afa0 0x55643a07b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f9324f9a700 1 -- 172.21.15.131:0/4047461327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a077170 msgr2=0x55643a097360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f9324f9a700 1 --2- 172.21.15.131:0/4047461327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a077170 0x55643a097360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.101+0000 7f9324f9a700 1 -- 172.21.15.131:0/4047461327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556439e36360 con 0x55643a07b8b0 2023-12-26T22:52:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.102+0000 7f9324f9a700 1 --2- 172.21.15.131:0/4047461327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07b8b0 0x55643a076c30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f93100038e0 tx=0x7f9310007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfd01326ec084a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.102+0000 7f930a7fc700 1 -- 172.21.15.131:0/4047461327 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310003520 con 0x55643a07b8b0 2023-12-26T22:52:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.102+0000 7f930a7fc700 1 -- 172.21.15.131:0/4047461327 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310002840 con 0x55643a07b8b0 2023-12-26T22:52:51.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.102+0000 7f930a7fc700 1 -- 172.21.15.131:0/4047461327 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310002bc0 con 0x55643a07b8b0 2023-12-26T22:52:51.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.102+0000 7f93360fbb80 1 -- 172.21.15.131:0/4047461327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07b8b0 msgr2=0x55643a076c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.102+0000 7f93360fbb80 1 --2- 172.21.15.131:0/4047461327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07b8b0 0x55643a076c30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f93100038e0 tx=0x7f9310007d10 comp rx=0 tx=0).stop 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.103+0000 7f93360fbb80 1 -- 172.21.15.131:0/4047461327 shutdown_connections 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.103+0000 7f93360fbb80 1 --2- 172.21.15.131:0/4047461327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a077170 0x55643a097360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.103+0000 7f93360fbb80 1 --2- 172.21.15.131:0/4047461327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07afa0 0x55643a07b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.103+0000 7f93360fbb80 1 --2- 172.21.15.131:0/4047461327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07b8b0 0x55643a076c30 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.103+0000 7f93360fbb80 1 -- 172.21.15.131:0/4047461327 >> 172.21.15.131:0/4047461327 conn(0x55643a051440 msgr2=0x55643a09cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.103+0000 7f93360fbb80 1 -- 172.21.15.131:0/4047461327 shutdown_connections 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.103+0000 7f93360fbb80 1 -- 172.21.15.131:0/4047461327 wait complete. 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.103+0000 7f93360fbb80 1 -- start start 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.104+0000 7f93360fbb80 1 Processor -- start 2023-12-26T22:52:51.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.104+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a078940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.104+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 0x55643a094130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.104+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a079eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f93360fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x556439f4f1b0 con 0x55643a077170 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f93360fbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556439f4ee50 con 0x55643a07afa0 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f93360fbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556439e9be40 con 0x55643a07b8b0 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f93360fbb80 10 client.0 _ll_register_callbacks cb 0x556439e9e7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f932579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a078940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f932579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a078940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35662/0 (socket says 172.21.15.131:35662) 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f932579b700 1 -- 172.21.15.131:0/1702859555 learned_addr learned my addr 172.21.15.131:0/1702859555 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f9325f9c700 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a079eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f9324f9a700 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 0x55643a094130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f932579b700 1 -- 172.21.15.131:0/1702859555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 msgr2=0x55643a079eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f932579b700 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a079eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f932579b700 1 -- 172.21.15.131:0/1702859555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 msgr2=0x55643a094130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f932579b700 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 0x55643a094130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f932579b700 1 -- 172.21.15.131:0/1702859555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c001110 con 0x55643a077170 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f932579b700 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a078940 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f931c014900 tx=0x7f931c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d49eb8deae97965 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.105+0000 7f92ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f92ff7fe700 1 -- 172.21.15.131:0/1702859555 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c021400 con 0x55643a077170 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f92ff7fe700 1 -- 172.21.15.131:0/1702859555 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f931c01c040 con 0x55643a077170 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f93360fbb80 1 -- 172.21.15.131:0/1702859555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55643a023eb0 con 0x55643a077170 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f92ff7fe700 1 -- 172.21.15.131:0/1702859555 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f931c021770 con 0x55643a077170 2023-12-26T22:52:51.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f93360fbb80 10 client.8799 fetch_fsmap learned FSMap version 73 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f93360fbb80 1 -- 172.21.15.131:0/1702859555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x556439de2b10 con 0x55643a077170 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f92ff7fe700 1 -- 172.21.15.131:0/1702859555 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55643a023eb0 con 0x55643a077170 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f92ff7fe700 1 -- 172.21.15.131:0/1702859555 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f931c0218d0 con 0x55643a077170 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f93360fbb80 10 client.8799 fetch_fsmap finished waiting for FSMap version 73 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f93360fbb80 10 client.8799 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f93360fbb80 20 client.8799 populate_metadata read hostname 'smithi131' 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f93360fbb80 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] conn(0x55643a092810 0x55643a092be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f93360fbb80 4 client.8799 mds_command: new command op to 8628 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.106+0000 7f93360fbb80 1 -- 172.21.15.131:0/1702859555 --> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556439f79880 con 0x55643a092810 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.107+0000 7f9324f9a700 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] conn(0x55643a092810 0x55643a092be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.107+0000 7f9324f9a700 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] conn(0x55643a092810 0x55643a092be0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268e100457435800 server_cookie=6b90357fe311ad94 in_seq=0 out_seq=0 2023-12-26T22:52:51.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.107+0000 7f92ff7fe700 10 client.8799 ms_handle_connect on v2:172.21.15.131:6833/2240171052 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f92ff7fe700 1 -- 172.21.15.131:0/1702859555 <== mds.0 v2:172.21.15.131:6833/2240171052 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556439f79880 con 0x55643a092810 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f92ff7fe700 10 client.8799 handle_command_reply: tid=0 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 client.8799 shutdown 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 -- 172.21.15.131:0/1702859555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 msgr2=0x55643a078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a078940 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f931c014900 tx=0x7f931c015000 comp rx=0 tx=0).stop 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 -- 172.21.15.131:0/1702859555 shutdown_connections 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] conn(0x55643a092810 0x55643a092be0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a079eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a078940 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 --2- 172.21.15.131:0/1702859555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 0x55643a094130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f92ff7fe700 0 client.8799 ms_handle_reset on v2:172.21.15.131:6833/2240171052 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 -- 172.21.15.131:0/1702859555 >> 172.21.15.131:0/1702859555 conn(0x55643a051440 msgr2=0x55643a055c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 -- 172.21.15.131:0/1702859555 shutdown_connections 2023-12-26T22:52:51.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.108+0000 7f93360fbb80 1 -- 172.21.15.131:0/1702859555 wait complete. 2023-12-26T22:52:51.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.109+0000 7f93360fbb80 20 client.8799 trim_cache size 0 max 16384 2023-12-26T22:52:51.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.126+0000 7f93360fbb80 1 Processor -- start 2023-12-26T22:52:51.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.126+0000 7f93360fbb80 1 -- start start 2023-12-26T22:52:51.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.126+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a077170 0x55643a055c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07afa0 0x55643a0822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a082830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f93360fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643a092170 con 0x55643a07afa0 2023-12-26T22:52:51.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f93360fbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643a0512d0 con 0x55643a077170 2023-12-26T22:52:51.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f93360fbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55643a053a00 con 0x55643a07b8b0 2023-12-26T22:52:51.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f9324f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07afa0 0x55643a0822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f9324f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07afa0 0x55643a0822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35666/0 (socket says 172.21.15.131:35666) 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f9324f9a700 1 -- 172.21.15.131:0/296473860 learned_addr learned my addr 172.21.15.131:0/296473860 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f9324f9a700 1 -- 172.21.15.131:0/296473860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 msgr2=0x55643a082830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f9324f9a700 1 --2- 172.21.15.131:0/296473860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a082830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f9324f9a700 1 -- 172.21.15.131:0/296473860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a077170 msgr2=0x55643a055c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f9324f9a700 1 --2- 172.21.15.131:0/296473860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a077170 0x55643a055c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f9324f9a700 1 -- 172.21.15.131:0/296473860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556439f71290 con 0x55643a07afa0 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.127+0000 7f9324f9a700 1 --2- 172.21.15.131:0/296473860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07afa0 0x55643a0822f0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9310003880 tx=0x7f9310020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2981e3e1779672 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f930affd700 1 -- 172.21.15.131:0/296473860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310025070 con 0x55643a07afa0 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f930affd700 1 -- 172.21.15.131:0/296473860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310026040 con 0x55643a07afa0 2023-12-26T22:52:51.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f930affd700 1 -- 172.21.15.131:0/296473860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310029250 con 0x55643a07afa0 2023-12-26T22:52:51.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f93360fbb80 1 -- 172.21.15.131:0/296473860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07afa0 msgr2=0x55643a0822f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f93360fbb80 1 --2- 172.21.15.131:0/296473860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07afa0 0x55643a0822f0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9310003880 tx=0x7f9310020040 comp rx=0 tx=0).stop 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f93360fbb80 1 -- 172.21.15.131:0/296473860 shutdown_connections 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f93360fbb80 1 --2- 172.21.15.131:0/296473860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a082830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f93360fbb80 1 --2- 172.21.15.131:0/296473860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a077170 0x55643a055c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f93360fbb80 1 --2- 172.21.15.131:0/296473860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a07afa0 0x55643a0822f0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f93360fbb80 1 -- 172.21.15.131:0/296473860 >> 172.21.15.131:0/296473860 conn(0x55643a051440 msgr2=0x55643a095060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f93360fbb80 1 -- 172.21.15.131:0/296473860 shutdown_connections 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.128+0000 7f93360fbb80 1 -- 172.21.15.131:0/296473860 wait complete. 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.129+0000 7f93360fbb80 1 -- start start 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.129+0000 7f93360fbb80 1 Processor -- start 2023-12-26T22:52:51.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a0a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 0x55643a0822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f93360fbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a082830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f93360fbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643a049b20 con 0x55643a077170 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f93360fbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643a049070 con 0x55643a07afa0 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f93360fbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55643a074c30 con 0x55643a07b8b0 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f93360fbb80 10 client.0 _ll_register_callbacks cb 0x7f931c014450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f932579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a0a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f932579b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a0a7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35674/0 (socket says 172.21.15.131:35674) 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f9324f9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 0x55643a0822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.130+0000 7f932579b700 1 -- 172.21.15.131:0/2091806362 learned_addr learned my addr 172.21.15.131:0/2091806362 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f9325f9c700 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a082830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f932579b700 1 -- 172.21.15.131:0/2091806362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 msgr2=0x55643a082830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f932579b700 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a082830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f932579b700 1 -- 172.21.15.131:0/2091806362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 msgr2=0x55643a0822f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f932579b700 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 0x55643a0822f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f932579b700 1 -- 172.21.15.131:0/2091806362 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556439e36360 con 0x55643a077170 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f932579b700 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a0a7eb0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f931c021ef0 tx=0x7f931c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faa1badf356cee6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f92ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f92ff7fe700 1 -- 172.21.15.131:0/2091806362 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c014c10 con 0x55643a077170 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f92ff7fe700 1 -- 172.21.15.131:0/2091806362 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f931c014500 con 0x55643a077170 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f93360fbb80 1 -- 172.21.15.131:0/2091806362 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9320156f90 con 0x55643a077170 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f92ff7fe700 1 -- 172.21.15.131:0/2091806362 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f931c032600 con 0x55643a077170 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f93360fbb80 10 client.8805 fetch_fsmap learned FSMap version 73 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.131+0000 7f93360fbb80 1 -- 172.21.15.131:0/2091806362 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x556439e3ac80 con 0x55643a077170 2023-12-26T22:52:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.132+0000 7f92ff7fe700 1 -- 172.21.15.131:0/2091806362 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9320156f90 con 0x55643a077170 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.132+0000 7f92ff7fe700 1 -- 172.21.15.131:0/2091806362 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 73) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f931c03c3f0 con 0x55643a077170 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.132+0000 7f93360fbb80 10 client.8805 fetch_fsmap finished waiting for FSMap version 73 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.132+0000 7f93360fbb80 10 client.8805 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.132+0000 7f93360fbb80 20 client.8805 populate_metadata read hostname 'smithi131' 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.132+0000 7f93360fbb80 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] conn(0x55643a0987d0 0x55643a09b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.132+0000 7f93360fbb80 4 client.8805 mds_command: new command op to 8628 tid=0[{"prefix": "status"}] 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.132+0000 7f93360fbb80 1 -- 172.21.15.131:0/2091806362 --> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55643a047490 con 0x55643a0987d0 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.132+0000 7f9324f9a700 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] conn(0x55643a0987d0 0x55643a09b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f9324f9a700 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] conn(0x55643a0987d0 0x55643a09b750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688fa5ffbb796dda server_cookie=92c10b29348596ed in_seq=0 out_seq=0 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f92ff7fe700 10 client.8805 ms_handle_connect on v2:172.21.15.131:6833/2240171052 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f92ff7fe700 1 -- 172.21.15.131:0/2091806362 <== mds.0 v2:172.21.15.131:6833/2240171052 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55643a047490 con 0x55643a0987d0 2023-12-26T22:52:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f92ff7fe700 10 client.8805 handle_command_reply: tid=0 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 client.8805 shutdown 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 -- 172.21.15.131:0/2091806362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 msgr2=0x55643a0a7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a0a7eb0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f931c021ef0 tx=0x7f931c019000 comp rx=0 tx=0).stop 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 -- 172.21.15.131:0/2091806362 shutdown_connections 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.131:6833/2240171052,v1:172.21.15.131:6835/2240171052] conn(0x55643a0987d0 0x55643a09b750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55643a07b8b0 0x55643a082830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55643a077170 0x55643a0a7eb0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f92ff7fe700 0 client.8805 ms_handle_reset on v2:172.21.15.131:6833/2240171052 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 --2- 172.21.15.131:0/2091806362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55643a07afa0 0x55643a0822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 -- 172.21.15.131:0/2091806362 >> 172.21.15.131:0/2091806362 conn(0x55643a051440 msgr2=0x55643a055c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 -- 172.21.15.131:0/2091806362 shutdown_connections 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 1 -- 172.21.15.131:0/2091806362 wait complete. 2023-12-26T22:52:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.133+0000 7f93360fbb80 20 client.8805 trim_cache size 0 max 16384 2023-12-26T22:52:51.138 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T22:52:51.138 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T22:52:51.138 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T22:52:51.138 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8628, 2023-12-26T22:52:51.138 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T22:52:51.138 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T22:52:51.138 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T22:52:51.138 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 11.321398335, 2023-12-26T22:52:51.138 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 73, 2023-12-26T22:52:51.139 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 161, 2023-12-26T22:52:51.139 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 161, 2023-12-26T22:52:51.139 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 17.051241927 2023-12-26T22:52:51.139 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 -- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9304066f30 msgr2=0x7f93040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9304066f30 0x7f93040693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f931c007fd0 tx=0x7f931c009040 comp rx=0 tx=0).stop 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 -- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932015d9b0 msgr2=0x7f932016a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932015d9b0 0x7f932016a870 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f931800b610 tx=0x7f9318019000 comp rx=0 tx=0).stop 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 -- 172.21.15.131:0/3250058627 shutdown_connections 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9304066f30 0x7f93040693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932015d9b0 0x7f932016a870 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93200cc580 0x7f9320159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 --2- 172.21.15.131:0/3250058627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93200cc950 0x7f932013bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 -- 172.21.15.131:0/3250058627 >> 172.21.15.131:0/3250058627 conn(0x7f93200686c0 msgr2=0x7f93200b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 -- 172.21.15.131:0/3250058627 shutdown_connections 2023-12-26T22:52:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.134+0000 7f93279ff700 1 -- 172.21.15.131:0/3250058627 wait complete. 2023-12-26T22:52:51.151 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T22:52:51.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.340+0000 7f8c0c9ff700 1 Processor -- start 2023-12-26T22:52:51.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.340+0000 7f8c0c9ff700 1 -- start start 2023-12-26T22:52:51.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.341+0000 7f8c0c9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8c080aa480 0x7f8c080abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c0c9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0815a8c0 con 0x7f8c08159820 2023-12-26T22:52:51.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c0c9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c08159f90 con 0x7f8c0806e480 2023-12-26T22:52:51.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c0c9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0815ad50 con 0x7f8c080aa480 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c0659c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8c08159820 0x7f8c0806dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41078/0 (socket says 172.21.15.131:41078) 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c0659c700 1 -- 172.21.15.131:0/1777799713 learned_addr learned my addr 172.21.15.131:0/1777799713 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c06d9d700 1 --2- 172.21.15.131:0/1777799713 >> v2:172.21.15.131:3300/0 conn(0x7f8c080aa480 0x7f8c080abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c05d9b700 1 -- 172.21.15.131:0/1777799713 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297928200 0 0) 0x7f8c0815a8c0 con 0x7f8c08159820 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c06d9d700 1 -- 172.21.15.131:0/1777799713 >> v1:172.21.15.190:6789/0 conn(0x7f8c0806e480 legacy=0x7f8c080675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c06d9d700 1 -- 172.21.15.131:0/1777799713 >> v1:172.21.15.131:6789/0 conn(0x7f8c08159820 legacy=0x7f8c0806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.342+0000 7f8c06d9d700 1 -- 172.21.15.131:0/1777799713 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0815a270 con 0x7f8c080aa480 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c06d9d700 1 --2- 172.21.15.131:0/1777799713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c080aa480 0x7f8c080abdb0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009e60 tx=0x7f8c00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c3c30deb68e2da server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c05d9b700 1 -- 172.21.15.131:0/1777799713 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0000e070 con 0x7f8c080aa480 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c05d9b700 1 -- 172.21.15.131:0/1777799713 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0000bc30 con 0x7f8c080aa480 2023-12-26T22:52:51.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c05d9b700 1 -- 172.21.15.131:0/1777799713 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00012430 con 0x7f8c080aa480 2023-12-26T22:52:51.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1777799713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c080aa480 msgr2=0x7f8c080abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c0c9ff700 1 --2- 172.21.15.131:0/1777799713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c080aa480 0x7f8c080abdb0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009e60 tx=0x7f8c00009780 comp rx=0 tx=0).stop 2023-12-26T22:52:51.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1777799713 shutdown_connections 2023-12-26T22:52:51.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c0c9ff700 1 --2- 172.21.15.131:0/1777799713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c080aa480 0x7f8c080abdb0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1777799713 >> 172.21.15.131:0/1777799713 conn(0x7f8c0806c290 msgr2=0x7f8c080ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:51.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1777799713 shutdown_connections 2023-12-26T22:52:51.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.343+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1777799713 wait complete. 2023-12-26T22:52:51.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.345+0000 7f8c0c9ff700 1 Processor -- start 2023-12-26T22:52:51.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.345+0000 7f8c0c9ff700 1 -- start start 2023-12-26T22:52:51.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c0c9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0806e480 0x7f8c0814a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c0c9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c080aa480 0x7f8c080c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c0c9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c08159820 0x7f8c08069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c0c9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080c32c0 con 0x7f8c08159820 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c0c9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080c26b0 con 0x7f8c080aa480 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c0c9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080c2990 con 0x7f8c0806e480 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c06d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c08159820 0x7f8c08069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c06d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c08159820 0x7f8c08069d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35704/0 (socket says 172.21.15.131:35704) 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c06d9d700 1 -- 172.21.15.131:0/1348499596 learned_addr learned my addr 172.21.15.131:0/1348499596 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8bfffff700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c080aa480 0x7f8c080c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c0659c700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0806e480 0x7f8c0814a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c06d9d700 1 -- 172.21.15.131:0/1348499596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0806e480 msgr2=0x7f8c0814a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c06d9d700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0806e480 0x7f8c0814a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.346+0000 7f8c06d9d700 1 -- 172.21.15.131:0/1348499596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c080aa480 msgr2=0x7f8c080c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.347+0000 7f8c06d9d700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c080aa480 0x7f8c080c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.347+0000 7f8c06d9d700 1 -- 172.21.15.131:0/1348499596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c00009a20 con 0x7f8c08159820 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.347+0000 7f8c06d9d700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c08159820 0x7f8c08069d80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8c00013000 tx=0x7f8c00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.347+0000 7f8bfe7fc700 1 -- 172.21.15.131:0/1348499596 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0001b420 con 0x7f8c08159820 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.347+0000 7f8bfe7fc700 1 -- 172.21.15.131:0/1348499596 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0001b580 con 0x7f8c08159820 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.347+0000 7f8bfe7fc700 1 -- 172.21.15.131:0/1348499596 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0001b920 con 0x7f8c08159820 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.347+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c080c2c70 con 0x7f8c08159820 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.347+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0814aa40 con 0x7f8c08159820 2023-12-26T22:52:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.348+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c080c3430 con 0x7f8c08159820 2023-12-26T22:52:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.349+0000 7f8bfe7fc700 1 -- 172.21.15.131:0/1348499596 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8c0001bc90 con 0x7f8c08159820 2023-12-26T22:52:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.350+0000 7f8bfe7fc700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8be8066f30 0x7f8be80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.351+0000 7f8bfe7fc700 1 -- 172.21.15.131:0/1348499596 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f8c00086500 con 0x7f8c08159820 2023-12-26T22:52:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.351+0000 7f8c0659c700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8be8066f30 0x7f8be80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.351+0000 7f8c0659c700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8be8066f30 0x7f8be80693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8007160 tx=0x7f8bf8007f50 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.355+0000 7f8bfe7fc700 1 -- 172.21.15.131:0/1348499596 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c00053500 con 0x7f8c08159820 2023-12-26T22:52:51.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.590+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8c080657e0 con 0x7f8c08159820 2023-12-26T22:52:51.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.591+0000 7f8bfe7fc700 1 -- 172.21.15.131:0/1348499596 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8c0002a060 con 0x7f8c08159820 2023-12-26T22:52:51.590 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T22:52:51.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8be8066f30 msgr2=0x7f8be80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8be8066f30 0x7f8be80693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8007160 tx=0x7f8bf8007f50 comp rx=0 tx=0).stop 2023-12-26T22:52:51.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c08159820 msgr2=0x7f8c08069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:51.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c08159820 0x7f8c08069d80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8c00013000 tx=0x7f8c00019000 comp rx=0 tx=0).stop 2023-12-26T22:52:51.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 shutdown_connections 2023-12-26T22:52:51.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8be8066f30 0x7f8be80693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c08159820 0x7f8c08069d80 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0806e480 0x7f8c0814a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 --2- 172.21.15.131:0/1348499596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c080aa480 0x7f8c080c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:51.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 >> 172.21.15.131:0/1348499596 conn(0x7f8c0806c290 msgr2=0x7f8c080ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:51.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 shutdown_connections 2023-12-26T22:52:51.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:51.593+0000 7f8c0c9ff700 1 -- 172.21.15.131:0/1348499596 wait complete. 2023-12-26T22:52:51.610 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T22:52:51.610 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T22:52:51.610 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T22:52:51.610 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T22:52:51.610 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T22:52:51.610 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T22:52:51.610 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T22:52:51.610 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T22:52:51.610 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T22:52:51.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:52:51.611 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T22:52:51.655 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T22:52:51.655 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T22:52:51.656 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 37884sec preferred_lft 37884sec 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T22:52:51.657 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T22:52:51.658 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T22:52:51.659 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:52:51.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:52:51.659 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T22:52:51.762 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T22:52:51.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:52:51.766 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T22:52:51.812 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T22:52:51.813 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T22:52:51.813 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T22:52:51.813 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T22:52:51.813 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T22:52:51.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:52:51.814 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T22:52:51.814 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T22:52:51.815 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T22:52:51.815 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T22:52:51.815 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T22:52:51.815 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T22:52:51.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:52:51.985 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T22:52:52.029 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T22:52:52.029 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T22:52:52.029 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T22:52:52.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:52:52.029 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T22:52:52.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T22:52:52.139 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T22:52:52.181 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T22:52:52.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T22:52:52.181 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T22:52:52.263 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T22:52:52.263 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T22:52:52.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T22:52:52.400 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T22:52:52.400 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T22:52:52.428 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T22:52:52.428 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T22:52:52.428 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T22:52:52.428 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T22:52:52.428 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T22:52:52.434 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T22:52:52.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:52:52.434 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T22:52:52.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T22:52:52.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.695+0000 7f58433ff700 1 Processor -- start 2023-12-26T22:52:52.693 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.695+0000 7f58433ff700 1 -- start start 2023-12-26T22:52:52.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.697+0000 7f58433ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f583c0b07f0 0x7f583c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:52.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.697+0000 7f58433ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159cf0 0x7f583c155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:52.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.697+0000 7f58433ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c0c8bf0 con 0x7f583c159920 2023-12-26T22:52:52.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.697+0000 7f58433ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c155f50 con 0x7f583c0b07f0 2023-12-26T22:52:52.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.697+0000 7f58433ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0c8e60 con 0x7f583c159cf0 2023-12-26T22:52:52.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.697+0000 7f584099a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f583c0b07f0 0x7f583c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:52.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.697+0000 7f584199c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159cf0 0x7f583c155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:52.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.697+0000 7f584099a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f583c0b07f0 0x7f583c1554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:44318/0 (socket says 172.21.15.131:44318) 2023-12-26T22:52:52.697 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.697+0000 7f584099a700 1 -- 172.21.15.131:0/591557040 learned_addr learned my addr 172.21.15.131:0/591557040 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:52.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.698+0000 7f584099a700 1 -- 172.21.15.131:0/591557040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159cf0 msgr2=0x7f583c155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:52.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.698+0000 7f584099a700 1 --2- 172.21.15.131:0/591557040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159cf0 0x7f583c155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.698+0000 7f584099a700 1 -- 172.21.15.131:0/591557040 >> v1:172.21.15.190:6789/0 conn(0x7f583c159920 legacy=0x7f583c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:52.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.698+0000 7f584099a700 1 -- 172.21.15.131:0/591557040 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c0c9140 con 0x7f583c0b07f0 2023-12-26T22:52:52.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.698+0000 7f584199c700 1 --2- 172.21.15.131:0/591557040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159cf0 0x7f583c155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:52.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.698+0000 7f584099a700 1 --2- 172.21.15.131:0/591557040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c0b07f0 0x7f583c1554d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f582c00bc40 tx=0x7f582c00b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ca2b7aecb24192 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:52.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.698+0000 7f5833fff700 1 -- 172.21.15.131:0/591557040 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c00da90 con 0x7f583c0b07f0 2023-12-26T22:52:52.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.698+0000 7f5833fff700 1 -- 172.21.15.131:0/591557040 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c00dbf0 con 0x7f583c0b07f0 2023-12-26T22:52:52.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.698+0000 7f5833fff700 1 -- 172.21.15.131:0/591557040 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c017410 con 0x7f583c0b07f0 2023-12-26T22:52:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.699+0000 7f58433ff700 1 -- 172.21.15.131:0/591557040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c0b07f0 msgr2=0x7f583c1554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.699+0000 7f58433ff700 1 --2- 172.21.15.131:0/591557040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c0b07f0 0x7f583c1554d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f582c00bc40 tx=0x7f582c00b4e0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.699+0000 7f58433ff700 1 -- 172.21.15.131:0/591557040 shutdown_connections 2023-12-26T22:52:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.699+0000 7f58433ff700 1 --2- 172.21.15.131:0/591557040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159cf0 0x7f583c155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.699+0000 7f58433ff700 1 --2- 172.21.15.131:0/591557040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c0b07f0 0x7f583c1554d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.699+0000 7f58433ff700 1 -- 172.21.15.131:0/591557040 >> 172.21.15.131:0/591557040 conn(0x7f583c068040 msgr2=0x7f583c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:52.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.699+0000 7f58433ff700 1 -- 172.21.15.131:0/591557040 shutdown_connections 2023-12-26T22:52:52.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.699+0000 7f58433ff700 1 -- 172.21.15.131:0/591557040 wait complete. 2023-12-26T22:52:52.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.701+0000 7f58433ff700 1 Processor -- start 2023-12-26T22:52:52.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.701+0000 7f58433ff700 1 -- start start 2023-12-26T22:52:52.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.702+0000 7f58433ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159920 0x7f583c1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:52.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.702+0000 7f58433ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c159cf0 0x7f583c15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:52.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.702+0000 7f58433ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f583c149b80 0x7f583c14a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:52.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.702+0000 7f58433ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c136740 con 0x7f583c149b80 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.702+0000 7f58433ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c14e710 con 0x7f583c159cf0 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.702+0000 7f58433ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c14e3e0 con 0x7f583c159920 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584199c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f583c149b80 0x7f583c14a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584119b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159920 0x7f583c1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584199c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f583c149b80 0x7f583c14a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35710/0 (socket says 172.21.15.131:35710) 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584199c700 1 -- 172.21.15.131:0/488478774 learned_addr learned my addr 172.21.15.131:0/488478774 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584099a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c159cf0 0x7f583c15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584199c700 1 -- 172.21.15.131:0/488478774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159920 msgr2=0x7f583c1503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584199c700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159920 0x7f583c1503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584199c700 1 -- 172.21.15.131:0/488478774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c159cf0 msgr2=0x7f583c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584199c700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c159cf0 0x7f583c15ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584199c700 1 -- 172.21.15.131:0/488478774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5834009a20 con 0x7f583c149b80 2023-12-26T22:52:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584099a700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c159cf0 0x7f583c15ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.703+0000 7f584199c700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f583c149b80 0x7f583c14a650 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5834009fc0 tx=0x7f583400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.704+0000 7f58317fa700 1 -- 172.21.15.131:0/488478774 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583400cbd0 con 0x7f583c149b80 2023-12-26T22:52:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.704+0000 7f58317fa700 1 -- 172.21.15.131:0/488478774 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583400f040 con 0x7f583c149b80 2023-12-26T22:52:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.704+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c00b780 con 0x7f583c149b80 2023-12-26T22:52:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.704+0000 7f58317fa700 1 -- 172.21.15.131:0/488478774 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834014480 con 0x7f583c149b80 2023-12-26T22:52:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.704+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c15ba20 con 0x7f583c149b80 2023-12-26T22:52:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.705+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583c0657e0 con 0x7f583c149b80 2023-12-26T22:52:52.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.706+0000 7f58317fa700 1 -- 172.21.15.131:0/488478774 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5834003020 con 0x7f583c149b80 2023-12-26T22:52:52.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.706+0000 7f58317fa700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5820066f30 0x7f58200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:52.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.707+0000 7f58317fa700 1 -- 172.21.15.131:0/488478774 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f5834085140 con 0x7f583c149b80 2023-12-26T22:52:52.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.707+0000 7f584119b700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5820066f30 0x7f58200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:52.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.707+0000 7f584119b700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5820066f30 0x7f58200693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58380039f0 tx=0x7f5838007ec0 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:52.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.711+0000 7f58317fa700 1 -- 172.21.15.131:0/488478774 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5834052140 con 0x7f583c149b80 2023-12-26T22:52:52.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.932+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f583c14e8c0 con 0x7f5820066f30 2023-12-26T22:52:52.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.934+0000 7f58317fa700 1 -- 172.21.15.131:0/488478774 <== mgr.8436 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f583c14e8c0 con 0x7f5820066f30 2023-12-26T22:52:52.933 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T22:52:52.933 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:52:52.933 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T22:52:52.933 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:52:52.933 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T22:52:52.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5820066f30 msgr2=0x7f58200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:52.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5820066f30 0x7f58200693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58380039f0 tx=0x7f5838007ec0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f583c149b80 msgr2=0x7f583c14a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f583c149b80 0x7f583c14a650 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5834009fc0 tx=0x7f583400b040 comp rx=0 tx=0).stop 2023-12-26T22:52:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 shutdown_connections 2023-12-26T22:52:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5820066f30 0x7f58200693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f583c149b80 0x7f583c14a650 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c159920 0x7f583c1503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 --2- 172.21.15.131:0/488478774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c159cf0 0x7f583c15ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 >> 172.21.15.131:0/488478774 conn(0x7f583c068040 msgr2=0x7f583c06e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 shutdown_connections 2023-12-26T22:52:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:52.936+0000 7f58433ff700 1 -- 172.21.15.131:0/488478774 wait complete. 2023-12-26T22:52:52.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T22:52:53.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.147+0000 7f837afff700 1 Processor -- start 2023-12-26T22:52:53.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.147+0000 7f837afff700 1 -- start start 2023-12-26T22:52:53.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.148+0000 7f837afff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f83741596a0 0x7f83740b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:53.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.148+0000 7f837afff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8374159a70 0x7f83740b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:53.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.148+0000 7f837afff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f83740b0160 0x7f83740b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:53.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.148+0000 7f837afff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83740afcd0 con 0x7f83741596a0 2023-12-26T22:52:53.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.148+0000 7f837afff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83740b3e70 con 0x7f83740b0160 2023-12-26T22:52:53.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.148+0000 7f837afff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b2e00 con 0x7f8374159a70 2023-12-26T22:52:53.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8378d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f83741596a0 0x7f83740b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:53.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8373fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8374159a70 0x7f83740b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:53.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8378d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f83741596a0 0x7f83740b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35714/0 (socket says 172.21.15.131:35714) 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8378d9b700 1 -- 172.21.15.131:0/1464682154 learned_addr learned my addr 172.21.15.131:0/1464682154 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f837959c700 1 --2- 172.21.15.131:0/1464682154 >> v2:172.21.15.190:3300/0 conn(0x7f83740b0160 0x7f83740b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8378d9b700 1 -- 172.21.15.131:0/1464682154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8374159a70 msgr2=0x7f83740b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8378d9b700 1 --2- 172.21.15.131:0/1464682154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8374159a70 0x7f83740b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8378d9b700 1 -- 172.21.15.131:0/1464682154 >> v2:172.21.15.190:3300/0 conn(0x7f83740b0160 msgr2=0x7f83740b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8378d9b700 1 --2- 172.21.15.131:0/1464682154 >> v2:172.21.15.190:3300/0 conn(0x7f83740b0160 0x7f83740b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8378d9b700 1 -- 172.21.15.131:0/1464682154 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83740b0f20 con 0x7f83741596a0 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8373fff700 1 --2- 172.21.15.131:0/1464682154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8374159a70 0x7f83740b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f837959c700 1 --2- 172.21.15.131:0/1464682154 >> v2:172.21.15.190:3300/0 conn(0x7f83740b0160 0x7f83740b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:53.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.149+0000 7f8378d9b700 1 --2- 172.21.15.131:0/1464682154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83741596a0 0x7f83740b2380 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f83740cdef0 tx=0x7f8368009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731e914d6b32673b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f83737fe700 1 -- 172.21.15.131:0/1464682154 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836800b640 con 0x7f83741596a0 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f83737fe700 1 -- 172.21.15.131:0/1464682154 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836800b7a0 con 0x7f83741596a0 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f83737fe700 1 -- 172.21.15.131:0/1464682154 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836800bb60 con 0x7f83741596a0 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f837afff700 1 -- 172.21.15.131:0/1464682154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83741596a0 msgr2=0x7f83740b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f837afff700 1 --2- 172.21.15.131:0/1464682154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83741596a0 0x7f83740b2380 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f83740cdef0 tx=0x7f8368009250 comp rx=0 tx=0).stop 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f837afff700 1 -- 172.21.15.131:0/1464682154 shutdown_connections 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f837afff700 1 --2- 172.21.15.131:0/1464682154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83741596a0 0x7f83740b2380 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f837afff700 1 --2- 172.21.15.131:0/1464682154 >> v2:172.21.15.190:3300/0 conn(0x7f83740b0160 0x7f83740b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f837afff700 1 --2- 172.21.15.131:0/1464682154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8374159a70 0x7f83740b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:53.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.150+0000 7f837afff700 1 -- 172.21.15.131:0/1464682154 >> 172.21.15.131:0/1464682154 conn(0x7f837406ece0 msgr2=0x7f837406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:53.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.151+0000 7f837afff700 1 -- 172.21.15.131:0/1464682154 shutdown_connections 2023-12-26T22:52:53.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.151+0000 7f837afff700 1 -- 172.21.15.131:0/1464682154 wait complete. 2023-12-26T22:52:53.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.153+0000 7f837afff700 1 Processor -- start 2023-12-26T22:52:53.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.153+0000 7f837afff700 1 -- start start 2023-12-26T22:52:53.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f837afff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83740b0160 0x7f837416ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:53.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f837afff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83741596a0 0x7f83740c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:53.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f837afff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8374159a70 0x7f8374155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:53.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f837afff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740c8ea0 con 0x7f8374159a70 2023-12-26T22:52:53.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f837afff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740c7bd0 con 0x7f83740b0160 2023-12-26T22:52:53.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f837afff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83740c8a10 con 0x7f83741596a0 2023-12-26T22:52:53.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f837959c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8374159a70 0x7f8374155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:53.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f837959c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8374159a70 0x7f8374155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35726/0 (socket says 172.21.15.131:35726) 2023-12-26T22:52:53.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f837959c700 1 -- 172.21.15.131:0/3381914656 learned_addr learned my addr 172.21.15.131:0/3381914656 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:53.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f8373fff700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83741596a0 0x7f83740c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:53.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.154+0000 7f8378d9b700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83740b0160 0x7f837416ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:53.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f837959c700 1 -- 172.21.15.131:0/3381914656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83741596a0 msgr2=0x7f83740c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:53.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f837959c700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83741596a0 0x7f83740c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:53.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f837959c700 1 -- 172.21.15.131:0/3381914656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83740b0160 msgr2=0x7f837416ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:53.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f837959c700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83740b0160 0x7f837416ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:53.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f837959c700 1 -- 172.21.15.131:0/3381914656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c009a20 con 0x7f8374159a70 2023-12-26T22:52:53.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f837959c700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8374159a70 0x7f8374155bc0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f836c009fc0 tx=0x7f836c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:53.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f8370ff9700 1 -- 172.21.15.131:0/3381914656 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c00cbd0 con 0x7f8374159a70 2023-12-26T22:52:53.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f8370ff9700 1 -- 172.21.15.131:0/3381914656 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836c00f040 con 0x7f8374159a70 2023-12-26T22:52:53.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8360009420 con 0x7f8374159a70 2023-12-26T22:52:53.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f8370ff9700 1 -- 172.21.15.131:0/3381914656 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c014480 con 0x7f8374159a70 2023-12-26T22:52:53.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.155+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83680094f0 con 0x7f8374159a70 2023-12-26T22:52:53.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.156+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83740c8cf0 con 0x7f8374159a70 2023-12-26T22:52:53.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.157+0000 7f8370ff9700 1 -- 172.21.15.131:0/3381914656 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 72) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f836c003020 con 0x7f8374159a70 2023-12-26T22:52:53.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.159+0000 7f8370ff9700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8358066f30 0x7f83580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:53.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.159+0000 7f8370ff9700 1 -- 172.21.15.131:0/3381914656 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f836c084ff0 con 0x7f8374159a70 2023-12-26T22:52:53.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.159+0000 7f8378d9b700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8358066f30 0x7f83580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:53.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.159+0000 7f8378d9b700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8358066f30 0x7f83580693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f83740cdef0 tx=0x7f8368017000 comp rx=0 tx=0).ready entity=mgr.8436 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:53.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.163+0000 7f8370ff9700 1 -- 172.21.15.131:0/3381914656 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f836c052090 con 0x7f8374159a70 2023-12-26T22:52:53.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:53.409+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f83740657e0 con 0x7f8374159a70 2023-12-26T22:52:54.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.158+0000 7f8370ff9700 1 -- 172.21.15.131:0/3381914656 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 73) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f836c052460 con 0x7f8374159a70 2023-12-26T22:52:54.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.160+0000 7f8370ff9700 1 -- 172.21.15.131:0/3381914656 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v73) v1 ==== 92+0+0 (secure 0 0 0) 0x7f836c055b20 con 0x7f8374159a70 2023-12-26T22:52:54.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.163+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8358066f30 msgr2=0x7f83580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.163+0000 7f837afff700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8358066f30 0x7f83580693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f83740cdef0 tx=0x7f8368017000 comp rx=0 tx=0).stop 2023-12-26T22:52:54.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.163+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8374159a70 msgr2=0x7f8374155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.163+0000 7f837afff700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8374159a70 0x7f8374155bc0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f836c009fc0 tx=0x7f836c00b040 comp rx=0 tx=0).stop 2023-12-26T22:52:54.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.164+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 shutdown_connections 2023-12-26T22:52:54.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.164+0000 7f837afff700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f8358066f30 0x7f83580693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.164+0000 7f837afff700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8374159a70 0x7f8374155bc0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.164+0000 7f837afff700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83740b0160 0x7f837416ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.164+0000 7f837afff700 1 --2- 172.21.15.131:0/3381914656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83741596a0 0x7f83740c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.164+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 >> 172.21.15.131:0/3381914656 conn(0x7f837406ece0 msgr2=0x7f8374158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:54.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.164+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 shutdown_connections 2023-12-26T22:52:54.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.164+0000 7f837afff700 1 -- 172.21.15.131:0/3381914656 wait complete. 2023-12-26T22:52:54.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T22:52:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.382+0000 7f441519d700 1 Processor -- start 2023-12-26T22:52:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.382+0000 7f441519d700 1 -- start start 2023-12-26T22:52:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.383+0000 7f441519d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f44100c6880 0x7f44100c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.383+0000 7f441519d700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100c13c0 con 0x7f44100c6c50 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.383+0000 7f441519d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100c0d90 con 0x7f4410157410 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.383+0000 7f441519d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4410157850 con 0x7f44100c6880 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.383+0000 7f440ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f44100c6880 0x7f44100c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.384+0000 7f440ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f44100c6880 0x7f44100c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35740/0 (socket says 172.21.15.131:35740) 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.384+0000 7f440ffff700 1 -- 172.21.15.131:0/163332043 learned_addr learned my addr 172.21.15.131:0/163332043 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.384+0000 7f440f7fe700 1 --1- 172.21.15.131:0/163332043 >> v1:172.21.15.131:6789/0 conn(0x7f44100c6c50 0x7f4410156d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41088/0 (socket says 172.21.15.131:41088) 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.384+0000 7f441499c700 1 --1- 172.21.15.131:0/163332043 >> v1:172.21.15.190:6789/0 conn(0x7f4410157410 0x7f44100c1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:46510/0 (socket says 172.21.15.131:46510) 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.384+0000 7f440ffff700 1 -- 172.21.15.131:0/163332043 >> v1:172.21.15.190:6789/0 conn(0x7f4410157410 legacy=0x7f44100c1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.384+0000 7f440ffff700 1 -- 172.21.15.131:0/163332043 >> v1:172.21.15.131:6789/0 conn(0x7f44100c6c50 legacy=0x7f4410156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.384+0000 7f440effd700 1 -- 172.21.15.131:0/163332043 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475862521 0 0) 0x7f44100c0d90 con 0x7f4410157410 2023-12-26T22:52:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.384+0000 7f440ffff700 1 -- 172.21.15.131:0/163332043 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44100c87f0 con 0x7f44100c6880 2023-12-26T22:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.384+0000 7f440ffff700 1 --2- 172.21.15.131:0/163332043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6880 0x7f44100c82b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f44000099b0 tx=0x7f4400009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e30a3ad1dd38676 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.385+0000 7f440effd700 1 -- 172.21.15.131:0/163332043 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000b860 con 0x7f44100c6880 2023-12-26T22:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.385+0000 7f440effd700 1 -- 172.21.15.131:0/163332043 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440000b9c0 con 0x7f44100c6880 2023-12-26T22:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.385+0000 7f440effd700 1 -- 172.21.15.131:0/163332043 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000bd60 con 0x7f44100c6880 2023-12-26T22:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.385+0000 7f441519d700 1 -- 172.21.15.131:0/163332043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6880 msgr2=0x7f44100c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.385+0000 7f441519d700 1 --2- 172.21.15.131:0/163332043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6880 0x7f44100c82b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f44000099b0 tx=0x7f4400009250 comp rx=0 tx=0).stop 2023-12-26T22:52:54.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.385+0000 7f441519d700 1 -- 172.21.15.131:0/163332043 shutdown_connections 2023-12-26T22:52:54.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.385+0000 7f441519d700 1 --2- 172.21.15.131:0/163332043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6880 0x7f44100c82b0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.385+0000 7f441519d700 1 -- 172.21.15.131:0/163332043 >> 172.21.15.131:0/163332043 conn(0x7f4410068080 msgr2=0x7f44100cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:54.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.386+0000 7f441519d700 1 -- 172.21.15.131:0/163332043 shutdown_connections 2023-12-26T22:52:54.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.386+0000 7f441519d700 1 -- 172.21.15.131:0/163332043 wait complete. 2023-12-26T22:52:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.387+0000 7f441519d700 1 Processor -- start 2023-12-26T22:52:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.387+0000 7f441519d700 1 -- start start 2023-12-26T22:52:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.388+0000 7f441519d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44100c6880 0x7f44100b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.388+0000 7f441519d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6c50 0x7f44100984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.388+0000 7f441519d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4410157410 0x7f44100989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.388+0000 7f441519d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44100b1710 con 0x7f44100c6c50 2023-12-26T22:52:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.388+0000 7f441519d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44100c38d0 con 0x7f44100c6880 2023-12-26T22:52:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.388+0000 7f441519d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44100b1150 con 0x7f4410157410 2023-12-26T22:52:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.388+0000 7f440f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6c50 0x7f44100984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:54.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6c50 0x7f44100984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35750/0 (socket says 172.21.15.131:35750) 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440f7fe700 1 -- 172.21.15.131:0/2620322396 learned_addr learned my addr 172.21.15.131:0/2620322396 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440ffff700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44100c6880 0x7f44100b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f441499c700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4410157410 0x7f44100989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440f7fe700 1 -- 172.21.15.131:0/2620322396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4410157410 msgr2=0x7f44100989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440f7fe700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4410157410 0x7f44100989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440f7fe700 1 -- 172.21.15.131:0/2620322396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44100c6880 msgr2=0x7f44100b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440f7fe700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44100c6880 0x7f44100b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440f7fe700 1 -- 172.21.15.131:0/2620322396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000094f0 con 0x7f44100c6c50 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440ffff700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44100c6880 0x7f44100b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.389+0000 7f440f7fe700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6c50 0x7f44100984a0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f440400c7c0 tx=0x7f4404007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.390+0000 7f43f7fff700 1 -- 172.21.15.131:0/2620322396 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404005a70 con 0x7f44100c6c50 2023-12-26T22:52:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.390+0000 7f43f7fff700 1 -- 172.21.15.131:0/2620322396 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404005bd0 con 0x7f44100c6c50 2023-12-26T22:52:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.390+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44100b1430 con 0x7f44100c6c50 2023-12-26T22:52:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.390+0000 7f43f7fff700 1 -- 172.21.15.131:0/2620322396 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44040070d0 con 0x7f44100c6c50 2023-12-26T22:52:54.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.390+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44100996d0 con 0x7f44100c6c50 2023-12-26T22:52:54.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.391+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44100998f0 con 0x7f44100c6c50 2023-12-26T22:52:54.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.392+0000 7f43f7fff700 1 -- 172.21.15.131:0/2620322396 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 73) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f4404007440 con 0x7f44100c6c50 2023-12-26T22:52:54.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.393+0000 7f43f7fff700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43fc066f80 0x7f43fc069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.393+0000 7f43f7fff700 1 -- 172.21.15.131:0/2620322396 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f4404083c00 con 0x7f44100c6c50 2023-12-26T22:52:54.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.393+0000 7f440ffff700 1 -- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43fc066f80 msgr2=0x7f43fc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:54.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.393+0000 7f440ffff700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43fc066f80 0x7f43fc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:54.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.401+0000 7f43f7fff700 1 -- 172.21.15.131:0/2620322396 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44100998f0 con 0x7f44100c6c50 2023-12-26T22:52:54.589 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:54.590+0000 7fe95bc721c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T22:52:54.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.594+0000 7f440ffff700 1 -- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43fc066f80 msgr2=0x7f43fc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:54.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.594+0000 7f440ffff700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43fc066f80 0x7f43fc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:54.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.602+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f44100998f0 con 0x7f44100c6c50 2023-12-26T22:52:54.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.606+0000 7f43f7fff700 1 -- 172.21.15.131:0/2620322396 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f44100998f0 con 0x7f44100c6c50 2023-12-26T22:52:54.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.608+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43fc066f80 msgr2=0x7f43fc069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:54.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.608+0000 7f441519d700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43fc066f80 0x7f43fc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6c50 msgr2=0x7f44100984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6c50 0x7f44100984a0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f440400c7c0 tx=0x7f4404007f70 comp rx=0 tx=0).stop 2023-12-26T22:52:54.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 shutdown_connections 2023-12-26T22:52:54.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4410157410 0x7f44100989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44100c6880 0x7f44100b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f43fc066f80 0x7f43fc069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 --2- 172.21.15.131:0/2620322396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44100c6c50 0x7f44100984a0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 >> 172.21.15.131:0/2620322396 conn(0x7f4410068080 msgr2=0x7f4410069800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:54.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 shutdown_connections 2023-12-26T22:52:54.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.609+0000 7f441519d700 1 -- 172.21.15.131:0/2620322396 wait complete. 2023-12-26T22:52:54.612 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:54.614+0000 7f8b36af31c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T22:52:54.626 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T22:52:54.694 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:54.695+0000 7fe95bc721c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T22:52:54.723 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:54.724+0000 7f8b36af31c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T22:52:54.783 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:54.784+0000 7fe95bc721c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T22:52:54.816 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:54.818+0000 7f8b36af31c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T22:52:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.824+0000 7f23dc1ff700 1 Processor -- start 2023-12-26T22:52:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.824+0000 7f23dc1ff700 1 -- start start 2023-12-26T22:52:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.826+0000 7f23dc1ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23d40b19f0 0x7f23d41620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.826+0000 7f23dc1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d40caa10 con 0x7f23d40c8820 2023-12-26T22:52:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.826+0000 7f23dc1ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d4157a80 con 0x7f23d40c8bf0 2023-12-26T22:52:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.826+0000 7f23dc1ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d40cac50 con 0x7f23d40b19f0 2023-12-26T22:52:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.826+0000 7f23d9f9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f23d40c8820 0x7f23d40ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41096/0 (socket says 172.21.15.131:41096) 2023-12-26T22:52:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.826+0000 7f23d9f9b700 1 -- 172.21.15.131:0/2218409828 learned_addr learned my addr 172.21.15.131:0/2218409828 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:52:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.826+0000 7f23da79c700 1 --2- 172.21.15.131:0/2218409828 >> v2:172.21.15.131:3300/0 conn(0x7f23d40b19f0 0x7f23d41620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:54.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.827+0000 7f23da79c700 1 -- 172.21.15.131:0/2218409828 >> v1:172.21.15.190:6789/0 conn(0x7f23d40c8bf0 legacy=0x7f23d40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.827+0000 7f23da79c700 1 -- 172.21.15.131:0/2218409828 >> v1:172.21.15.131:6789/0 conn(0x7f23d40c8820 legacy=0x7f23d40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.827+0000 7f23da79c700 1 -- 172.21.15.131:0/2218409828 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40caf30 con 0x7f23d40b19f0 2023-12-26T22:52:54.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.827+0000 7f23d8f99700 1 -- 172.21.15.131:0/2218409828 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795585993 0 0) 0x7f23d0001110 con 0x7f23d40c8820 2023-12-26T22:52:54.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.827+0000 7f23da79c700 1 --2- 172.21.15.131:0/2218409828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 0x7f23d41620a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009e60 tx=0x7f23cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59b797d6e1f0dd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.827+0000 7f23d8f99700 1 -- 172.21.15.131:0/2218409828 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc00e070 con 0x7f23d40b19f0 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.828+0000 7f23d8f99700 1 -- 172.21.15.131:0/2218409828 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc00bc30 con 0x7f23d40b19f0 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.828+0000 7f23d8f99700 1 -- 172.21.15.131:0/2218409828 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc012430 con 0x7f23d40b19f0 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.828+0000 7f23dc1ff700 1 -- 172.21.15.131:0/2218409828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 msgr2=0x7f23d41620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.828+0000 7f23dc1ff700 1 --2- 172.21.15.131:0/2218409828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 0x7f23d41620a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009e60 tx=0x7f23cc009780 comp rx=0 tx=0).stop 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.828+0000 7f23dc1ff700 1 -- 172.21.15.131:0/2218409828 shutdown_connections 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.828+0000 7f23dc1ff700 1 --2- 172.21.15.131:0/2218409828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 0x7f23d41620a0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.828+0000 7f23dc1ff700 1 -- 172.21.15.131:0/2218409828 >> 172.21.15.131:0/2218409828 conn(0x7f23d4068040 msgr2=0x7f23d4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.828+0000 7f23dc1ff700 1 -- 172.21.15.131:0/2218409828 shutdown_connections 2023-12-26T22:52:54.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.828+0000 7f23dc1ff700 1 -- 172.21.15.131:0/2218409828 wait complete. 2023-12-26T22:52:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.830+0000 7f23dc1ff700 1 Processor -- start 2023-12-26T22:52:54.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.831+0000 7f23dc1ff700 1 -- start start 2023-12-26T22:52:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23dc1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 0x7f23d4168350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23dc1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23d40c8820 0x7f23d4169f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23dc1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23d40c8bf0 0x7f23d4167290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23dc1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4168890 con 0x7f23d40b19f0 2023-12-26T22:52:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23dc1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40c65e0 con 0x7f23d40c8bf0 2023-12-26T22:52:54.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23dc1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40c5cb0 con 0x7f23d40c8820 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23d9f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 0x7f23d4168350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23d9f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 0x7f23d4168350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35762/0 (socket says 172.21.15.131:35762) 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23d9f9b700 1 -- 172.21.15.131:0/3264297394 learned_addr learned my addr 172.21.15.131:0/3264297394 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23da79c700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23d40c8bf0 0x7f23d4167290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.832+0000 7f23d979a700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23d40c8820 0x7f23d4169f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23d9f9b700 1 -- 172.21.15.131:0/3264297394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23d40c8820 msgr2=0x7f23d4169f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23d9f9b700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23d40c8820 0x7f23d4169f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23d9f9b700 1 -- 172.21.15.131:0/3264297394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23d40c8bf0 msgr2=0x7f23d4167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23d9f9b700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23d40c8bf0 0x7f23d4167290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:54.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23d9f9b700 1 -- 172.21.15.131:0/3264297394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc009a20 con 0x7f23d40b19f0 2023-12-26T22:52:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23d9f9b700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 0x7f23d4168350 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c840 tx=0x7f23d0007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23c9ffb700 1 -- 172.21.15.131:0/3264297394 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0005b30 con 0x7f23d40b19f0 2023-12-26T22:52:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23c9ffb700 1 -- 172.21.15.131:0/3264297394 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d0005c90 con 0x7f23d40b19f0 2023-12-26T22:52:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23c9ffb700 1 -- 172.21.15.131:0/3264297394 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0007300 con 0x7f23d40b19f0 2023-12-26T22:52:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d40c5f90 con 0x7f23d40b19f0 2023-12-26T22:52:54.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.833+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d40c58f0 con 0x7f23d40b19f0 2023-12-26T22:52:54.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.835+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d40c68c0 con 0x7f23d40b19f0 2023-12-26T22:52:54.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.836+0000 7f23c9ffb700 1 -- 172.21.15.131:0/3264297394 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 73) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f23d001b3b0 con 0x7f23d40b19f0 2023-12-26T22:52:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.838+0000 7f23c9ffb700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f23b8066f80 0x7f23b8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.838+0000 7f23d979a700 1 -- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f23b8066f80 msgr2=0x7f23b8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.838+0000 7f23c9ffb700 1 -- 172.21.15.131:0/3264297394 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f23d0083b80 con 0x7f23d40b19f0 2023-12-26T22:52:54.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.838+0000 7f23d979a700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f23b8066f80 0x7f23b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:54.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:54.842+0000 7f23c9ffb700 1 -- 172.21.15.131:0/3264297394 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23d0050b80 con 0x7f23d40b19f0 2023-12-26T22:52:54.869 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:54.870+0000 7fe95bc721c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T22:52:54.904 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:54.906+0000 7f8b36af31c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T22:52:55.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.039+0000 7f23d979a700 1 -- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f23b8066f80 msgr2=0x7f23b8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:55.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.039+0000 7f23d979a700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f23b8066f80 0x7f23b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:55.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.039+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f23d40caa10 con 0x7f23d40b19f0 2023-12-26T22:52:55.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.043+0000 7f23c9ffb700 1 -- 172.21.15.131:0/3264297394 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7f23d40caa10 con 0x7f23d40b19f0 2023-12-26T22:52:55.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.045+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f23b8066f80 msgr2=0x7f23b8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:52:55.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.045+0000 7f23dc1ff700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f23b8066f80 0x7f23b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.045+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 msgr2=0x7f23d4168350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:55.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.045+0000 7f23dc1ff700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 0x7f23d4168350 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c840 tx=0x7f23d0007c50 comp rx=0 tx=0).stop 2023-12-26T22:52:55.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.046+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 shutdown_connections 2023-12-26T22:52:55.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.046+0000 7f23dc1ff700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23d40c8bf0 0x7f23d4167290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.046+0000 7f23dc1ff700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f23b8066f80 0x7f23b8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.046+0000 7f23dc1ff700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d40b19f0 0x7f23d4168350 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.046+0000 7f23dc1ff700 1 --2- 172.21.15.131:0/3264297394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23d40c8820 0x7f23d4169f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.046+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 >> 172.21.15.131:0/3264297394 conn(0x7f23d4068040 msgr2=0x7f23d4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:55.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.046+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 shutdown_connections 2023-12-26T22:52:55.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.046+0000 7f23dc1ff700 1 -- 172.21.15.131:0/3264297394 wait complete. 2023-12-26T22:52:55.059 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:55.060+0000 7fe95bc721c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:52:55.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:52:55.063 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-26T22:52:55.102 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:55.103+0000 7f8b36af31c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:52:55.111 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2023-12-26T22:52:55.231 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:55.231+0000 7fe95bc721c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T22:52:55.286 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:55.287+0000 7f8b36af31c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T22:52:55.313 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:55.314+0000 7fe95bc721c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T22:52:55.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.357+0000 7f9e765ff700 1 Processor -- start 2023-12-26T22:52:55.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.357+0000 7f9e765ff700 1 -- start start 2023-12-26T22:52:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.358+0000 7f9e765ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 0x7f9e70069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.358+0000 7f9e765ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9e700cc590 0x7f9e700cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.358+0000 7f9e765ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e70069640 con 0x7f9e700cb340 2023-12-26T22:52:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.358+0000 7f9e765ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e700cb7c0 con 0x7f9e700cc590 2023-12-26T22:52:55.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.358+0000 7f9e765ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e70068750 con 0x7f9e70142f00 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.359+0000 7f9e6f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9e700cc590 0x7f9e700cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.359+0000 7f9e6f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9e700cc590 0x7f9e700cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46250/0 (socket says 172.21.15.131:46250) 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.359+0000 7f9e6f7fe700 1 -- 172.21.15.131:0/2892577846 learned_addr learned my addr 172.21.15.131:0/2892577846 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.359+0000 7f9e6ffff700 1 --2- 172.21.15.131:0/2892577846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 0x7f9e70069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.359+0000 7f9e6f7fe700 1 -- 172.21.15.131:0/2892577846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 msgr2=0x7f9e70069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.359+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/2892577846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 0x7f9e70069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.359+0000 7f9e6f7fe700 1 -- 172.21.15.131:0/2892577846 >> v1:172.21.15.190:6789/0 conn(0x7f9e700cb340 legacy=0x7f9e700c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.359+0000 7f9e6f7fe700 1 -- 172.21.15.131:0/2892577846 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e70068a30 con 0x7f9e700cc590 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.359+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/2892577846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cc590 0x7f9e700cae00 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f9e640099b0 tx=0x7f9e64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31033c28d2e4296 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:55.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e6effd700 1 -- 172.21.15.131:0/2892577846 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64009ea0 con 0x7f9e700cc590 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e6effd700 1 -- 172.21.15.131:0/2892577846 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6400b860 con 0x7f9e700cc590 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e6effd700 1 -- 172.21.15.131:0/2892577846 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6400bc00 con 0x7f9e700cc590 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e765ff700 1 -- 172.21.15.131:0/2892577846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cc590 msgr2=0x7f9e700cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e765ff700 1 --2- 172.21.15.131:0/2892577846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cc590 0x7f9e700cae00 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f9e640099b0 tx=0x7f9e64009250 comp rx=0 tx=0).stop 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e765ff700 1 -- 172.21.15.131:0/2892577846 shutdown_connections 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e765ff700 1 --2- 172.21.15.131:0/2892577846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 0x7f9e70069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e765ff700 1 --2- 172.21.15.131:0/2892577846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cc590 0x7f9e700cae00 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e765ff700 1 -- 172.21.15.131:0/2892577846 >> 172.21.15.131:0/2892577846 conn(0x7f9e70063cd0 msgr2=0x7f9e7006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.360+0000 7f9e765ff700 1 -- 172.21.15.131:0/2892577846 shutdown_connections 2023-12-26T22:52:55.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.361+0000 7f9e765ff700 1 -- 172.21.15.131:0/2892577846 wait complete. 2023-12-26T22:52:55.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.362+0000 7f9e765ff700 1 Processor -- start 2023-12-26T22:52:55.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.362+0000 7f9e765ff700 1 -- start start 2023-12-26T22:52:55.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.363+0000 7f9e765ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cb340 0x7f9e700c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:55.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.363+0000 7f9e765ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e700cc590 0x7f9e7015f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:55.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.363+0000 7f9e765ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 0x7f9e7015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.363+0000 7f9e765ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7016a060 con 0x7f9e700cb340 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.363+0000 7f9e765ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e70169d50 con 0x7f9e700cc590 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.363+0000 7f9e765ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700c7ce0 con 0x7f9e70142f00 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cb340 0x7f9e700c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cb340 0x7f9e700c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46252/0 (socket says 172.21.15.131:46252) 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6ffff700 1 -- 172.21.15.131:0/1696156804 learned_addr learned my addr 172.21.15.131:0/1696156804 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e700cc590 0x7f9e7015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e74b9c700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 0x7f9e7015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6ffff700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 msgr2=0x7f9e7015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6ffff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 0x7f9e7015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6ffff700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e700cc590 msgr2=0x7f9e7015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:52:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6ffff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e700cc590 0x7f9e7015f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:52:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6ffff700 1 -- 172.21.15.131:0/1696156804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640094f0 con 0x7f9e700cb340 2023-12-26T22:52:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.364+0000 7f9e6ffff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cb340 0x7f9e700c7370 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000e960 tx=0x7f9e600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:52:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.365+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60013680 con 0x7f9e700cb340 2023-12-26T22:52:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.365+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e600137e0 con 0x7f9e700cb340 2023-12-26T22:52:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.365+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e700c7fc0 con 0x7f9e700cb340 2023-12-26T22:52:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.365+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60013b80 con 0x7f9e700cb340 2023-12-26T22:52:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.365+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e700c9bd0 con 0x7f9e700cb340 2023-12-26T22:52:55.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.366+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e700c8460 con 0x7f9e700cb340 2023-12-26T22:52:55.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.368+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 73) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f9e600123d0 con 0x7f9e700cb340 2023-12-26T22:52:55.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.368+0000 7f9e57fff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:52:55.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.368+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f9e60084910 con 0x7f9e700cb340 2023-12-26T22:52:55.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.368+0000 7f9e6f7fe700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 msgr2=0x7f9e5c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:55.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.369+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:52:55.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.377+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e60051910 con 0x7f9e700cb340 2023-12-26T22:52:55.378 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:55.380+0000 7f8b36af31c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T22:52:55.460 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:55.460+0000 7fe95bc721c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T22:52:55.527 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:55.528+0000 7f8b36af31c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T22:52:55.550 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:55.551+0000 7fe95bc721c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T22:52:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.570+0000 7f9e6f7fe700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 msgr2=0x7f9e5c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:55.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.570+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:52:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.600+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e7016a1a0 con 0x7f9e5c066f80 2023-12-26T22:52:55.617 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:55.619+0000 7f8b36af31c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T22:52:55.732 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:55.733+0000 7fe95bc721c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T22:52:55.798 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:55.800+0000 7f8b36af31c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T22:52:55.885 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:55.886+0000 7fe95bc721c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T22:52:55.951 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:55.952+0000 7f8b36af31c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T22:52:55.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.971+0000 7f9e6f7fe700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 msgr2=0x7f9e5c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:55.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:55.971+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T22:52:56.266 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:56.267+0000 7fe95bc721c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T22:52:56.332 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:56.334+0000 7f8b36af31c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T22:52:56.530 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:56.531+0000 7fe95bc721c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T22:52:56.597 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:56.599+0000 7f8b36af31c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T22:52:56.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:56.772+0000 7f9e6f7fe700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 msgr2=0x7f9e5c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:56.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:56.772+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T22:52:56.807 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:56.807+0000 7fe95bc721c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T22:52:56.874 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:56.875+0000 7f8b36af31c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T22:52:56.895 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:56.896+0000 7fe95bc721c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T22:52:56.963 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:56.965+0000 7f8b36af31c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T22:52:57.628 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:57.629+0000 7fe95bc721c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T22:52:57.692 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:57.693+0000 7f8b36af31c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T22:52:57.720 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:57.721+0000 7fe95bc721c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T22:52:57.785 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:57.786+0000 7f8b36af31c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T22:52:57.870 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:57.871+0000 7fe95bc721c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T22:52:57.937 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:57.938+0000 7f8b36af31c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T22:52:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:58.373+0000 7f9e6f7fe700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 msgr2=0x7f9e5c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:52:58.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:52:58.373+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T22:52:58.487 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:58.488+0000 7fe95bc721c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T22:52:58.559 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:58.560+0000 7f8b36af31c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T22:52:58.675 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:58.676+0000 7fe95bc721c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T22:52:58.747 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:58.747+0000 7f8b36af31c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T22:52:58.865 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:58.866+0000 7fe95bc721c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T22:52:58.937 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:58.938+0000 7f8b36af31c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T22:52:59.704 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:59.705+0000 7fe95bc721c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T22:52:59.785 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:52:59.785+0000 7f8b36af31c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T22:52:59.977 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:52:59.978+0000 7fe95bc721c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:00.060 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.061+0000 7f8b36af31c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:00.070 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:00.071+0000 7fe95bc721c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:00.155 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.155+0000 7f8b36af31c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:00.267 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:00.268+0000 7fe95bc721c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T22:53:00.349 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.349+0000 7f8b36af31c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T22:53:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:00.361+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 74) v1 ==== 84470+0+0 (secure 0 0 0) 0x7f9e60051ce0 con 0x7f9e700cb340 2023-12-26T22:53:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:00.361+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 msgr2=0x7f9e5c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:00.361+0000 7f9e57fff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:00.482 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.482+0000 7f8aa8471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:00.482 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.482+0000 7f8aa8471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:00.482 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.482+0000 7f8aa8471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:00.482 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.482+0000 7f8aa8471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:00.482 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.482+0000 7f8aa8471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:00.483 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.482+0000 7f8aab477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:00.483 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.483+0000 7f8aab477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:00.483 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.483+0000 7f8aab477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:00.483 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.483+0000 7f8aab477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:00.483 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:00.483+0000 7f8aab477700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.366+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 75) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f9e60055830 con 0x7f9e700cb340 2023-12-26T22:53:01.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.367+0000 7f9e57fff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:01.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.367+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e5c00a3f0 con 0x7f9e5c066f80 2023-12-26T22:53:01.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.369+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.370+0000 7f9e6f7fe700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9e640099b0 tx=0x7f9e64017000 comp rx=0 tx=0).ready entity=mgr.8838 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:01.439 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.440+0000 7f9e57fff700 1 -- 172.21.15.131:0/1696156804 <== mgr.8838 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9e5c00a3f0 con 0x7f9e5c066f80 2023-12-26T22:53:01.440 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path snap_test_dir1 2023-12-26T22:53:01.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 msgr2=0x7f9e5c069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:01.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9e640099b0 tx=0x7f9e64017000 comp rx=0 tx=0).stop 2023-12-26T22:53:01.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cb340 msgr2=0x7f9e700c7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cb340 0x7f9e700c7370 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000e960 tx=0x7f9e600091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 shutdown_connections 2023-12-26T22:53:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e70142f00 0x7f9e7015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9e5c066f80 0x7f9e5c069430 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e700cb340 0x7f9e700c7370 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 --2- 172.21.15.131:0/1696156804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e700cc590 0x7f9e7015f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 >> 172.21.15.131:0/1696156804 conn(0x7f9e70063cd0 msgr2=0x7f9e700b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 shutdown_connections 2023-12-26T22:53:01.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.443+0000 7f9e765ff700 1 -- 172.21.15.131:0/1696156804 wait complete. 2023-12-26T22:53:01.462 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-26T22:53:01.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-12-26T22:53:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.659+0000 7f3c7d7ff700 1 Processor -- start 2023-12-26T22:53:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.659+0000 7f3c7d7ff700 1 -- start start 2023-12-26T22:53:01.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.660+0000 7f3c7d7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3c78159810 0x7f3c780c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:01.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.660+0000 7f3c7d7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3c780cec80 0x7f3c780cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.660+0000 7f3c7d7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cd0a0 0x7f3c780cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.660+0000 7f3c7d7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c780cb2a0 con 0x7f3c78159810 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c7d7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c780cf640 con 0x7f3c780cec80 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c7d7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780cbb00 con 0x7f3c780cd0a0 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c76ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3c78159810 0x7f3c780c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c76ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3c78159810 0x7f3c780c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46412/0 (socket says 172.21.15.131:46412) 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c76ffd700 1 -- 172.21.15.131:0/3862849648 learned_addr learned my addr 172.21.15.131:0/3862849648 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c767fc700 1 --2- 172.21.15.131:0/3862849648 >> v2:172.21.15.190:3300/0 conn(0x7f3c780cec80 0x7f3c780cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c777fe700 1 --2- 172.21.15.131:0/3862849648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cd0a0 0x7f3c780cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c76ffd700 1 -- 172.21.15.131:0/3862849648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cd0a0 msgr2=0x7f3c780cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c76ffd700 1 --2- 172.21.15.131:0/3862849648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cd0a0 0x7f3c780cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c76ffd700 1 -- 172.21.15.131:0/3862849648 >> v2:172.21.15.190:3300/0 conn(0x7f3c780cec80 msgr2=0x7f3c780cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c76ffd700 1 --2- 172.21.15.131:0/3862849648 >> v2:172.21.15.190:3300/0 conn(0x7f3c780cec80 0x7f3c780cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.661+0000 7f3c76ffd700 1 -- 172.21.15.131:0/3862849648 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c780ce040 con 0x7f3c78159810 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c76ffd700 1 --2- 172.21.15.131:0/3862849648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c78159810 0x7f3c780c5440 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3c78089a80 tx=0x7f3c68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3093ce5bad8b572 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c75ffb700 1 -- 172.21.15.131:0/3862849648 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800b860 con 0x7f3c78159810 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c75ffb700 1 -- 172.21.15.131:0/3862849648 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6800b9c0 con 0x7f3c78159810 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c75ffb700 1 -- 172.21.15.131:0/3862849648 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800bd60 con 0x7f3c78159810 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/3862849648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c78159810 msgr2=0x7f3c780c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/3862849648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c78159810 0x7f3c780c5440 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3c78089a80 tx=0x7f3c68009250 comp rx=0 tx=0).stop 2023-12-26T22:53:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/3862849648 shutdown_connections 2023-12-26T22:53:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/3862849648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c78159810 0x7f3c780c5440 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/3862849648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cd0a0 0x7f3c780cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/3862849648 >> v2:172.21.15.190:3300/0 conn(0x7f3c780cec80 0x7f3c780cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/3862849648 >> 172.21.15.131:0/3862849648 conn(0x7f3c78099950 msgr2=0x7f3c7806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/3862849648 shutdown_connections 2023-12-26T22:53:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.662+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/3862849648 wait complete. 2023-12-26T22:53:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.664+0000 7f3c7d7ff700 1 Processor -- start 2023-12-26T22:53:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.664+0000 7f3c7d7ff700 1 -- start start 2023-12-26T22:53:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c7d7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c780cd0a0 0x7f3c7816db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c7d7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cec80 0x7f3c78153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c7d7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c78159810 0x7f3c78153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c7d7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7814dca0 con 0x7f3c780cd0a0 2023-12-26T22:53:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c7d7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c781513a0 con 0x7f3c78159810 2023-12-26T22:53:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c7d7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c781515f0 con 0x7f3c780cec80 2023-12-26T22:53:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c76ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c780cd0a0 0x7f3c7816db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c76ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c780cd0a0 0x7f3c7816db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46420/0 (socket says 172.21.15.131:46420) 2023-12-26T22:53:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c76ffd700 1 -- 172.21.15.131:0/1767740061 learned_addr learned my addr 172.21.15.131:0/1767740061 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c767fc700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cec80 0x7f3c78153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c777fe700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c78159810 0x7f3c78153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c76ffd700 1 -- 172.21.15.131:0/1767740061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cec80 msgr2=0x7f3c78153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c76ffd700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cec80 0x7f3c78153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c76ffd700 1 -- 172.21.15.131:0/1767740061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c78159810 msgr2=0x7f3c78153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c76ffd700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c78159810 0x7f3c78153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c76ffd700 1 -- 172.21.15.131:0/1767740061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68009490 con 0x7f3c780cd0a0 2023-12-26T22:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.665+0000 7f3c777fe700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c78159810 0x7f3c78153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.666+0000 7f3c76ffd700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c780cd0a0 0x7f3c7816db00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006fd0 tx=0x7f3c68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.666+0000 7f3c5effd700 1 -- 172.21.15.131:0/1767740061 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800b860 con 0x7f3c780cd0a0 2023-12-26T22:53:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.666+0000 7f3c5effd700 1 -- 172.21.15.131:0/1767740061 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c68011de0 con 0x7f3c780cd0a0 2023-12-26T22:53:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.666+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c781518d0 con 0x7f3c780cd0a0 2023-12-26T22:53:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.666+0000 7f3c5effd700 1 -- 172.21.15.131:0/1767740061 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68024690 con 0x7f3c780cd0a0 2023-12-26T22:53:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.666+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c78151e90 con 0x7f3c780cd0a0 2023-12-26T22:53:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.667+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c780ceac0 con 0x7f3c780cd0a0 2023-12-26T22:53:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.668+0000 7f3c5effd700 1 -- 172.21.15.131:0/1767740061 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 75) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f3c6802d440 con 0x7f3c780cd0a0 2023-12-26T22:53:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.669+0000 7f3c5effd700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3c64066dd0 0x7f3c64069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.669+0000 7f3c5effd700 1 -- 172.21.15.131:0/1767740061 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f3c68085ea0 con 0x7f3c780cd0a0 2023-12-26T22:53:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.669+0000 7f3c767fc700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3c64066dd0 0x7f3c64069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.670+0000 7f3c767fc700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3c64066dd0 0x7f3c64069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3c780b4c40 tx=0x7f3c6c008040 comp rx=0 tx=0).ready entity=mgr.8838 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:01.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.673+0000 7f3c5effd700 1 -- 172.21.15.131:0/1767740061 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c680530a0 con 0x7f3c780cd0a0 2023-12-26T22:53:01.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.889+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c780717f0 con 0x7f3c64066dd0 2023-12-26T22:53:01.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.946+0000 7f3c5effd700 1 -- 172.21.15.131:0/1767740061 <== mgr.8838 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3c780717f0 con 0x7f3c64066dd0 2023-12-26T22:53:01.946 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path snap_test_dir1 2023-12-26T22:53:01.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.948+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3c64066dd0 msgr2=0x7f3c64069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:01.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.948+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3c64066dd0 0x7f3c64069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3c780b4c40 tx=0x7f3c6c008040 comp rx=0 tx=0).stop 2023-12-26T22:53:01.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.948+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c780cd0a0 msgr2=0x7f3c7816db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:01.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.948+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c780cd0a0 0x7f3c7816db00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006fd0 tx=0x7f3c68017000 comp rx=0 tx=0).stop 2023-12-26T22:53:01.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.949+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 shutdown_connections 2023-12-26T22:53:01.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.949+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c78159810 0x7f3c78153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.949+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3c64066dd0 0x7f3c64069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.949+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c780cd0a0 0x7f3c7816db00 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.949+0000 7f3c7d7ff700 1 --2- 172.21.15.131:0/1767740061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c780cec80 0x7f3c78153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:01.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.949+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 >> 172.21.15.131:0/1767740061 conn(0x7f3c78099950 msgr2=0x7f3c780c6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:01.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.949+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 shutdown_connections 2023-12-26T22:53:01.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:01.949+0000 7f3c7d7ff700 1 -- 172.21.15.131:0/1767740061 wait complete. 2023-12-26T22:53:01.966 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-26T22:53:01.967 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-12-26T22:53:02.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.160+0000 7f74439ff700 1 Processor -- start 2023-12-26T22:53:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.160+0000 7f74439ff700 1 -- start start 2023-12-26T22:53:02.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.161+0000 7f74439ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c156730 0x7f743c0c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.161+0000 7f74439ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f743c06ee80 0x7f743c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.161+0000 7f74439ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c0acfc0 con 0x7f743c156360 2023-12-26T22:53:02.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.161+0000 7f74439ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c0c91f0 con 0x7f743c06ee80 2023-12-26T22:53:02.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.161+0000 7f74439ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0c94d0 con 0x7f743c156730 2023-12-26T22:53:02.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.161+0000 7f744179b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f743c156360 0x7f743c06e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33204/0 (socket says 172.21.15.131:33204) 2023-12-26T22:53:02.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.161+0000 7f744179b700 1 -- 172.21.15.131:0/3595201571 learned_addr learned my addr 172.21.15.131:0/3595201571 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:02.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7440f9a700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c156730 0x7f743c0c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7441f9c700 1 --2- 172.21.15.131:0/3595201571 >> v2:172.21.15.190:3300/0 conn(0x7f743c06ee80 0x7f743c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772011067 0 0) 0x7f743c0acfc0 con 0x7f743c156360 2023-12-26T22:53:02.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7428003570 con 0x7f743c156360 2023-12-26T22:53:02.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1849019921 0 0) 0x7f7428003570 con 0x7f743c156360 2023-12-26T22:53:02.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c156730 msgr2=0x7f743c0c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:02.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c156730 0x7f743c0c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 >> v2:172.21.15.190:3300/0 conn(0x7f743c06ee80 msgr2=0x7f743c0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:02.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 --2- 172.21.15.131:0/3595201571 >> v2:172.21.15.190:3300/0 conn(0x7f743c06ee80 0x7f743c0c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0acc50 con 0x7f743c156360 2023-12-26T22:53:02.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f7438001470 con 0x7f743c156360 2023-12-26T22:53:02.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 >> v1:172.21.15.131:6789/0 conn(0x7f743c156360 legacy=0x7f743c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:02.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428003a60 0x7f7428003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7428004810 0x7f7428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c156730 0x7f74280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0c91f0 con 0x7f7428004810 2023-12-26T22:53:02.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0c94d0 con 0x7f7428003a60 2023-12-26T22:53:02.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.162+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742800ba60 con 0x7f743c156730 2023-12-26T22:53:02.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7440f9a700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7428004810 0x7f7428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f744179b700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428003a60 0x7f7428003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7441f9c700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c156730 0x7f74280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7440f9a700 1 -- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c156730 msgr2=0x7f74280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:02.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7440f9a700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c156730 0x7f74280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7440f9a700 1 -- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428003a60 msgr2=0x7f7428003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:02.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7440f9a700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428003a60 0x7f7428003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7440f9a700 1 -- 172.21.15.131:0/3595201571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428003570 con 0x7f7428004810 2023-12-26T22:53:02.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7441f9c700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c156730 0x7f74280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:02.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7440f9a700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7428004810 0x7f7428004c80 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f742c009c30 tx=0x7f742c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc24afee28a4b49f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:02.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c00e710 con 0x7f7428004810 2023-12-26T22:53:02.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c00e870 con 0x7f7428004810 2023-12-26T22:53:02.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.163+0000 7f7433fff700 1 -- 172.21.15.131:0/3595201571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c00ec30 con 0x7f7428004810 2023-12-26T22:53:02.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.164+0000 7f74439ff700 1 -- 172.21.15.131:0/3595201571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7428004810 msgr2=0x7f7428004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:02.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.164+0000 7f74439ff700 1 --2- 172.21.15.131:0/3595201571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7428004810 0x7f7428004c80 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f742c009c30 tx=0x7f742c0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.164+0000 7f7441f9c700 1 -- 172.21.15.131:0/3595201571 reap_dead start 2023-12-26T22:53:02.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.164+0000 7f74439ff700 1 -- 172.21.15.131:0/3595201571 shutdown_connections 2023-12-26T22:53:02.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.164+0000 7f74439ff700 1 -- 172.21.15.131:0/3595201571 >> 172.21.15.131:0/3595201571 conn(0x7f743c0675b0 msgr2=0x7f743c0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:02.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.164+0000 7f74439ff700 1 -- 172.21.15.131:0/3595201571 shutdown_connections 2023-12-26T22:53:02.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.164+0000 7f74439ff700 1 -- 172.21.15.131:0/3595201571 wait complete. 2023-12-26T22:53:02.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.166+0000 7f74439ff700 1 Processor -- start 2023-12-26T22:53:02.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.166+0000 7f74439ff700 1 -- start start 2023-12-26T22:53:02.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f74439ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428004810 0x7f743c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f74439ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f743c0aef90 0x7f743c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f74439ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0b0ec0 0x7f743c0b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f74439ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0b1e10 con 0x7f743c0aef90 2023-12-26T22:53:02.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f74439ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c15ddd0 con 0x7f7428004810 2023-12-26T22:53:02.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f74439ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c15e8f0 con 0x7f743c0b0ec0 2023-12-26T22:53:02.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f7440f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f743c0aef90 0x7f743c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f7440f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f743c0aef90 0x7f743c06e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46434/0 (socket says 172.21.15.131:46434) 2023-12-26T22:53:02.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f7440f9a700 1 -- 172.21.15.131:0/2361987383 learned_addr learned my addr 172.21.15.131:0/2361987383 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:02.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f744179b700 1 --2- 172.21.15.131:0/2361987383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428004810 0x7f743c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f7441f9c700 1 --2- 172.21.15.131:0/2361987383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0b0ec0 0x7f743c0b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f7440f9a700 1 -- 172.21.15.131:0/2361987383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0b0ec0 msgr2=0x7f743c0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:02.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f7440f9a700 1 --2- 172.21.15.131:0/2361987383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0b0ec0 0x7f743c0b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f7440f9a700 1 -- 172.21.15.131:0/2361987383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428004810 msgr2=0x7f743c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:02.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f7440f9a700 1 --2- 172.21.15.131:0/2361987383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428004810 0x7f743c06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.167+0000 7f7440f9a700 1 -- 172.21.15.131:0/2361987383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c15dfc0 con 0x7f743c0aef90 2023-12-26T22:53:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.168+0000 7f7440f9a700 1 --2- 172.21.15.131:0/2361987383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f743c0aef90 0x7f743c06e3c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f74280046d0 tx=0x7f742c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.168+0000 7f74317fa700 1 -- 172.21.15.131:0/2361987383 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c022070 con 0x7f743c0aef90 2023-12-26T22:53:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.168+0000 7f74317fa700 1 -- 172.21.15.131:0/2361987383 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c00e9e0 con 0x7f743c0aef90 2023-12-26T22:53:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.168+0000 7f74317fa700 1 -- 172.21.15.131:0/2361987383 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c01c450 con 0x7f743c0aef90 2023-12-26T22:53:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.168+0000 7f74439ff700 1 -- 172.21.15.131:0/2361987383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c15e2a0 con 0x7f743c0aef90 2023-12-26T22:53:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.168+0000 7f74439ff700 1 -- 172.21.15.131:0/2361987383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c15ed80 con 0x7f743c0aef90 2023-12-26T22:53:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.169+0000 7f74439ff700 1 -- 172.21.15.131:0/2361987383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c0717f0 con 0x7f743c0aef90 2023-12-26T22:53:02.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.170+0000 7f74317fa700 1 -- 172.21.15.131:0/2361987383 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 75) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f742c01c860 con 0x7f743c0aef90 2023-12-26T22:53:02.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.171+0000 7f74317fa700 1 --2- 172.21.15.131:0/2361987383 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7420066dd0 0x7f7420069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.171+0000 7f74317fa700 1 -- 172.21.15.131:0/2361987383 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f742c085cd0 con 0x7f743c0aef90 2023-12-26T22:53:02.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.171+0000 7f744179b700 1 --2- 172.21.15.131:0/2361987383 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7420066dd0 0x7f7420069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.171+0000 7f744179b700 1 --2- 172.21.15.131:0/2361987383 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7420066dd0 0x7f7420069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7438001d20 tx=0x7f7438009040 comp rx=0 tx=0).ready entity=mgr.8838 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:02.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.175+0000 7f74317fa700 1 -- 172.21.15.131:0/2361987383 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f742c052e50 con 0x7f743c0aef90 2023-12-26T22:53:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.377+0000 7f74317fa700 1 -- 172.21.15.131:0/2361987383 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 76) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f742c053220 con 0x7f743c0aef90 2023-12-26T22:53:02.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.399+0000 7f74439ff700 1 -- 172.21.15.131:0/2361987383 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f743c0c8f10 con 0x7f7420066dd0 2023-12-26T22:53:02.405 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:02.406+0000 7f8ab2485700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-26T22:53:02.406 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.406+0000 7f74317fa700 1 -- 172.21.15.131:0/2361987383 <== mgr.8838 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f743c0c8f10 con 0x7f7420066dd0 2023-12-26T22:53:02.407 INFO:teuthology.orchestra.run.smithi131.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-26T22:53:02.416 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-26T22:53:02.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:53:02.417 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-26T22:53:02.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T22:53:02.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.696+0000 7fb1851ff700 1 Processor -- start 2023-12-26T22:53:02.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.696+0000 7fb1851ff700 1 -- start start 2023-12-26T22:53:02.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.697+0000 7fb1851ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb180159820 0x7fb1800abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.697+0000 7fb1851ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1800aa480 0x7fb18006dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.697+0000 7fb1851ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18015a8c0 con 0x7fb18006e2b0 2023-12-26T22:53:02.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.697+0000 7fb1851ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18015ad50 con 0x7fb1800aa480 2023-12-26T22:53:02.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.697+0000 7fb1851ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18006e730 con 0x7fb180159820 2023-12-26T22:53:02.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.697+0000 7fb17f59e700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb18006e2b0 0x7fb18015d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33212/0 (socket says 172.21.15.131:33212) 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.697+0000 7fb17f59e700 1 -- 172.21.15.131:0/1824034523 learned_addr learned my addr 172.21.15.131:0/1824034523 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17ed9d700 1 --2- 172.21.15.131:0/1824034523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb180159820 0x7fb1800abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17e59c700 1 --2- 172.21.15.131:0/1824034523 >> v2:172.21.15.190:3300/0 conn(0x7fb1800aa480 0x7fb18006dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17dd9b700 1 -- 172.21.15.131:0/1824034523 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359662167 0 0) 0x7fb18015a8c0 con 0x7fb18006e2b0 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17dd9b700 1 -- 172.21.15.131:0/1824034523 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb168003570 con 0x7fb18006e2b0 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17e59c700 1 -- 172.21.15.131:0/1824034523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb180159820 msgr2=0x7fb1800abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17e59c700 1 --2- 172.21.15.131:0/1824034523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb180159820 0x7fb1800abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17e59c700 1 -- 172.21.15.131:0/1824034523 >> v1:172.21.15.131:6789/0 conn(0x7fb18006e2b0 legacy=0x7fb18015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17e59c700 1 -- 172.21.15.131:0/1824034523 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180159f90 con 0x7fb1800aa480 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17ed9d700 1 --2- 172.21.15.131:0/1824034523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb180159820 0x7fb1800abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.698+0000 7fb17e59c700 1 --2- 172.21.15.131:0/1824034523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1800aa480 0x7fb18006dd70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb174009910 tx=0x7fb1740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25d73cb594491714 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb17dd9b700 1 -- 172.21.15.131:0/1824034523 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174011630 con 0x7fb1800aa480 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb17dd9b700 1 -- 172.21.15.131:0/1824034523 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174011790 con 0x7fb1800aa480 2023-12-26T22:53:02.699 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb17dd9b700 1 -- 172.21.15.131:0/1824034523 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174011b30 con 0x7fb1800aa480 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb1851ff700 1 -- 172.21.15.131:0/1824034523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1800aa480 msgr2=0x7fb18006dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb1851ff700 1 --2- 172.21.15.131:0/1824034523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1800aa480 0x7fb18006dd70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb174009910 tx=0x7fb1740091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb1851ff700 1 -- 172.21.15.131:0/1824034523 shutdown_connections 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb1851ff700 1 --2- 172.21.15.131:0/1824034523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb180159820 0x7fb1800abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb1851ff700 1 --2- 172.21.15.131:0/1824034523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1800aa480 0x7fb18006dd70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb1851ff700 1 -- 172.21.15.131:0/1824034523 >> 172.21.15.131:0/1824034523 conn(0x7fb18006c290 msgr2=0x7fb1800ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb1851ff700 1 -- 172.21.15.131:0/1824034523 shutdown_connections 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.699+0000 7fb1851ff700 1 -- 172.21.15.131:0/1824034523 wait complete. 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.701+0000 7fb1851ff700 1 Processor -- start 2023-12-26T22:53:02.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.701+0000 7fb1851ff700 1 -- start start 2023-12-26T22:53:02.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb1851ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18006e2b0 0x7fb18014d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb1851ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1800aa480 0x7fb18016c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb1851ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb180159820 0x7fb180157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb1851ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18014fe80 con 0x7fb180159820 2023-12-26T22:53:02.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb1851ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18014d8f0 con 0x7fb18006e2b0 2023-12-26T22:53:02.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb1851ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18014f270 con 0x7fb1800aa480 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb17f59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb180159820 0x7fb180157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb17f59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb180159820 0x7fb180157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46446/0 (socket says 172.21.15.131:46446) 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb17f59e700 1 -- 172.21.15.131:0/1911744769 learned_addr learned my addr 172.21.15.131:0/1911744769 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb17e59c700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1800aa480 0x7fb18016c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.702+0000 7fb17ed9d700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18006e2b0 0x7fb18014d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb17f59e700 1 -- 172.21.15.131:0/1911744769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1800aa480 msgr2=0x7fb18016c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb17f59e700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1800aa480 0x7fb18016c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb17f59e700 1 -- 172.21.15.131:0/1911744769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18006e2b0 msgr2=0x7fb18014d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb17f59e700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18006e2b0 0x7fb18014d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb17f59e700 1 -- 172.21.15.131:0/1911744769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170009420 con 0x7fb180159820 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb17e59c700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1800aa480 0x7fb18016c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:02.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb17f59e700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb180159820 0x7fb180157390 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb178001460 tx=0x7fb178001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:02.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb166ffd700 1 -- 172.21.15.131:0/1911744769 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17800a8a0 con 0x7fb180159820 2023-12-26T22:53:02.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb166ffd700 1 -- 172.21.15.131:0/1911744769 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17800aa00 con 0x7fb180159820 2023-12-26T22:53:02.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb166ffd700 1 -- 172.21.15.131:0/1911744769 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17800ad80 con 0x7fb180159820 2023-12-26T22:53:02.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb174009420 con 0x7fb180159820 2023-12-26T22:53:02.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.703+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18014f830 con 0x7fb180159820 2023-12-26T22:53:02.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.704+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb180150030 con 0x7fb180159820 2023-12-26T22:53:02.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.705+0000 7fb166ffd700 1 -- 172.21.15.131:0/1911744769 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 76) v1 ==== 84918+0+0 (secure 0 0 0) 0x7fb178007300 con 0x7fb180159820 2023-12-26T22:53:02.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.706+0000 7fb166ffd700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb16c067130 0x7fb16c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:02.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.706+0000 7fb166ffd700 1 -- 172.21.15.131:0/1911744769 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fb1780850a0 con 0x7fb180159820 2023-12-26T22:53:02.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.706+0000 7fb17ed9d700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb16c067130 0x7fb16c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:02.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.707+0000 7fb17ed9d700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb16c067130 0x7fb16c0695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb170009f40 tx=0x7fb1700091a0 comp rx=0 tx=0).ready entity=mgr.8838 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:02.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.711+0000 7fb166ffd700 1 -- 172.21.15.131:0/1911744769 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb180150030 con 0x7fb180159820 2023-12-26T22:53:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:02.961+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb180150030 con 0x7fb180159820 2023-12-26T22:53:03.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.382+0000 7fb166ffd700 1 -- 172.21.15.131:0/1911744769 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fb1780520e0 con 0x7fb180159820 2023-12-26T22:53:03.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.384+0000 7fb166ffd700 1 -- 172.21.15.131:0/1911744769 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v77) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb1780557a0 con 0x7fb180159820 2023-12-26T22:53:03.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb16c067130 msgr2=0x7fb16c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:03.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb16c067130 0x7fb16c0695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb170009f40 tx=0x7fb1700091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb180159820 msgr2=0x7fb180157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:03.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb180159820 0x7fb180157390 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb178001460 tx=0x7fb178001a40 comp rx=0 tx=0).stop 2023-12-26T22:53:03.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 shutdown_connections 2023-12-26T22:53:03.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb16c067130 0x7fb16c0695e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb180159820 0x7fb180157390 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb18006e2b0 0x7fb18014d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 --2- 172.21.15.131:0/1911744769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1800aa480 0x7fb18016c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.387+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 >> 172.21.15.131:0/1911744769 conn(0x7fb18006c290 msgr2=0x7fb1800ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:03.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.388+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 shutdown_connections 2023-12-26T22:53:03.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.388+0000 7fb1851ff700 1 -- 172.21.15.131:0/1911744769 wait complete. 2023-12-26T22:53:03.407 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T22:53:03.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T22:53:03.507 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T22:53:03.593 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T22:53:03.593 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T22:53:03.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:53:03.593 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T22:53:03.672 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T22:53:03.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:53:03.672 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T22:53:03.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:53:03.827 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:03.828+0000 7fbfb53371c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T22:53:03.857 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:03.857+0000 7f3da90621c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T22:53:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.927+0000 7ff2357ff700 1 Processor -- start 2023-12-26T22:53:03.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.927+0000 7ff2357ff700 1 -- start start 2023-12-26T22:53:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff2357ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff2300c8820 0x7ff2300ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff2357ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300b19f0 0x7ff230161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff2357ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff230157a80 con 0x7ff2300c8bf0 2023-12-26T22:53:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff2357ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2300cac50 con 0x7ff2300c8820 2023-12-26T22:53:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff2357ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300caf30 con 0x7ff2300b19f0 2023-12-26T22:53:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff22effd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff2300c8820 0x7ff2300ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff22effd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff2300c8820 0x7ff2300ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46482/0 (socket says 172.21.15.131:46482) 2023-12-26T22:53:03.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff22effd700 1 -- 172.21.15.131:0/3056755808 learned_addr learned my addr 172.21.15.131:0/3056755808 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff22e7fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff2300c8bf0 0x7ff2300b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33238/0 (socket says 172.21.15.131:33238) 2023-12-26T22:53:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.929+0000 7ff22f7fe700 1 --2- 172.21.15.131:0/3056755808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300b19f0 0x7ff230161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.930+0000 7ff22effd700 1 -- 172.21.15.131:0/3056755808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300b19f0 msgr2=0x7ff230161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.930+0000 7ff22effd700 1 --2- 172.21.15.131:0/3056755808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300b19f0 0x7ff230161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.930+0000 7ff22effd700 1 -- 172.21.15.131:0/3056755808 >> v1:172.21.15.131:6789/0 conn(0x7ff2300c8bf0 legacy=0x7ff2300b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.930+0000 7ff22effd700 1 -- 172.21.15.131:0/3056755808 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2300cb210 con 0x7ff2300c8820 2023-12-26T22:53:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.930+0000 7ff22dffb700 1 -- 172.21.15.131:0/3056755808 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955592810 0 0) 0x7ff230157a80 con 0x7ff2300c8bf0 2023-12-26T22:53:03.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.930+0000 7ff22effd700 1 --2- 172.21.15.131:0/3056755808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300c8820 0x7ff2300ca300 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff2200099b0 tx=0x7ff220009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e56c21bca8b7fe server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.930+0000 7ff22dffb700 1 -- 172.21.15.131:0/3056755808 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220009ea0 con 0x7ff2300c8820 2023-12-26T22:53:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.930+0000 7ff22dffb700 1 -- 172.21.15.131:0/3056755808 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22000b860 con 0x7ff2300c8820 2023-12-26T22:53:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.930+0000 7ff22dffb700 1 -- 172.21.15.131:0/3056755808 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000bc00 con 0x7ff2300c8820 2023-12-26T22:53:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.931+0000 7ff2357ff700 1 -- 172.21.15.131:0/3056755808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300c8820 msgr2=0x7ff2300ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.931+0000 7ff2357ff700 1 --2- 172.21.15.131:0/3056755808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300c8820 0x7ff2300ca300 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff2200099b0 tx=0x7ff220009250 comp rx=0 tx=0).stop 2023-12-26T22:53:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.931+0000 7ff2357ff700 1 -- 172.21.15.131:0/3056755808 shutdown_connections 2023-12-26T22:53:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.931+0000 7ff2357ff700 1 --2- 172.21.15.131:0/3056755808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300b19f0 0x7ff230161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.931+0000 7ff2357ff700 1 --2- 172.21.15.131:0/3056755808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300c8820 0x7ff2300ca300 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.931+0000 7ff2357ff700 1 -- 172.21.15.131:0/3056755808 >> 172.21.15.131:0/3056755808 conn(0x7ff230068040 msgr2=0x7ff230069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:03.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.931+0000 7ff2357ff700 1 -- 172.21.15.131:0/3056755808 shutdown_connections 2023-12-26T22:53:03.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.931+0000 7ff2357ff700 1 -- 172.21.15.131:0/3056755808 wait complete. 2023-12-26T22:53:03.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.932+0000 7ff2357ff700 1 Processor -- start 2023-12-26T22:53:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.933+0000 7ff2357ff700 1 -- start start 2023-12-26T22:53:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.933+0000 7ff2357ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300b19f0 0x7ff2300b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.933+0000 7ff2357ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300c8820 0x7ff230146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.933+0000 7ff2357ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300c8bf0 0x7ff2301692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.933+0000 7ff2357ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23016a2f0 con 0x7ff2300b19f0 2023-12-26T22:53:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.933+0000 7ff2357ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff230147270 con 0x7ff2300c8bf0 2023-12-26T22:53:03.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.933+0000 7ff2357ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff230169820 con 0x7ff2300c8820 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300b19f0 0x7ff2300b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300b19f0 0x7ff2300b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46490/0 (socket says 172.21.15.131:46490) 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22effd700 1 -- 172.21.15.131:0/1902975182 learned_addr learned my addr 172.21.15.131:0/1902975182 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22e7fc700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300c8820 0x7ff230146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22f7fe700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300c8bf0 0x7ff2301692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22effd700 1 -- 172.21.15.131:0/1902975182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300c8820 msgr2=0x7ff230146d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22effd700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300c8820 0x7ff230146d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22effd700 1 -- 172.21.15.131:0/1902975182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300c8bf0 msgr2=0x7ff2301692e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22effd700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300c8bf0 0x7ff2301692e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:03.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.934+0000 7ff22effd700 1 -- 172.21.15.131:0/1902975182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2200094f0 con 0x7ff2300b19f0 2023-12-26T22:53:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.935+0000 7ff22effd700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300b19f0 0x7ff2300b0d70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff22000b3c0 tx=0x7ff220017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.935+0000 7ff216ffd700 1 -- 172.21.15.131:0/1902975182 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220000d10 con 0x7ff2300b19f0 2023-12-26T22:53:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.935+0000 7ff216ffd700 1 -- 172.21.15.131:0/1902975182 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22000b9d0 con 0x7ff2300b19f0 2023-12-26T22:53:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.935+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2300af300 con 0x7ff2300b19f0 2023-12-26T22:53:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.935+0000 7ff216ffd700 1 -- 172.21.15.131:0/1902975182 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220025420 con 0x7ff2300b19f0 2023-12-26T22:53:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.935+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff230169a50 con 0x7ff2300b19f0 2023-12-26T22:53:03.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.936+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2300af0d0 con 0x7ff2300b19f0 2023-12-26T22:53:03.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.937+0000 7ff216ffd700 1 -- 172.21.15.131:0/1902975182 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7ff220023070 con 0x7ff2300b19f0 2023-12-26T22:53:03.940 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:03.935+0000 7fbfb53371c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T22:53:03.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.938+0000 7ff216ffd700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff21c0670e0 0x7ff21c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:03.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.938+0000 7ff216ffd700 1 -- 172.21.15.131:0/1902975182 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff220086060 con 0x7ff2300b19f0 2023-12-26T22:53:03.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.938+0000 7ff22e7fc700 1 -- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff21c0670e0 msgr2=0x7ff21c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:03.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.938+0000 7ff22e7fc700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff21c0670e0 0x7ff21c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:03.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:03.942+0000 7ff216ffd700 1 -- 172.21.15.131:0/1902975182 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2300af0d0 con 0x7ff2300b19f0 2023-12-26T22:53:03.968 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:03.968+0000 7f3da90621c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T22:53:04.025 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:04.025+0000 7fbfb53371c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T22:53:04.060 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:04.061+0000 7f3da90621c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T22:53:04.108 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:04.109+0000 7fbfb53371c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T22:53:04.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.139+0000 7ff22e7fc700 1 -- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff21c0670e0 msgr2=0x7ff21c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:04.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.139+0000 7ff22e7fc700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff21c0670e0 0x7ff21c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:04.146 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:04.147+0000 7f3da90621c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T22:53:04.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.191+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2300af0d0 con 0x7ff2300b19f0 2023-12-26T22:53:04.191 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.191+0000 7ff216ffd700 1 -- 172.21.15.131:0/1902975182 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4611 (secure 0 0 0) 0x7ff2300af0d0 con 0x7ff2300b19f0 2023-12-26T22:53:04.191 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:04.191 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2085575956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2085575956},{"type":"v1","addr":"172.21.15.190:6834","nonce":2085575956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2129957224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2129957224},{"type":"v1","addr":"172.21.15.190:6835","nonce":2129957224}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3810273571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3810273571},{"type":"v1","addr":"172.21.15.131:6834","nonce":3810273571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:52:39.271885+0000","modified":"2023-12-26T22:52:42.824355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8628":{"gid":8628,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2240171052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2240171052},{"type":"v1","addr":"172.21.15.131:6835","nonce":2240171052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-12-26T22:53:04.194 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff21c0670e0 msgr2=0x7ff21c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff21c0670e0 0x7ff21c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300b19f0 msgr2=0x7ff2300b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300b19f0 0x7ff2300b0d70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff22000b3c0 tx=0x7ff220017000 comp rx=0 tx=0).stop 2023-12-26T22:53:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 shutdown_connections 2023-12-26T22:53:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300c8bf0 0x7ff2301692e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff21c0670e0 0x7ff21c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2300b19f0 0x7ff2300b0d70 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.195 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 --2- 172.21.15.131:0/1902975182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300c8820 0x7ff230146d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 >> 172.21.15.131:0/1902975182 conn(0x7ff230068040 msgr2=0x7ff2300c1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 shutdown_connections 2023-12-26T22:53:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.194+0000 7ff2357ff700 1 -- 172.21.15.131:0/1902975182 wait complete. 2023-12-26T22:53:04.196 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 73 2023-12-26T22:53:04.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:53:04.298 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:04.299+0000 7fbfb53371c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:04.350 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:04.350+0000 7f3da90621c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:04.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.410+0000 7f9d223ff700 1 Processor -- start 2023-12-26T22:53:04.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.410+0000 7f9d223ff700 1 -- start start 2023-12-26T22:53:04.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.411+0000 7f9d223ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9d1c1599d0 0x7f9d1c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.411+0000 7f9d223ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0d01e0 0x7f9d1c0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.411+0000 7f9d223ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1c0cfd50 con 0x7f9d1c159600 2023-12-26T22:53:04.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.411+0000 7f9d223ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c0b1280 con 0x7f9d1c1599d0 2023-12-26T22:53:04.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.411+0000 7f9d223ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0cf140 con 0x7f9d1c0d01e0 2023-12-26T22:53:04.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.411+0000 7f9d2099c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0d01e0 0x7f9d1c0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d1bfff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9d1c159600 0x7f9d1c15dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:59736/0 (socket says 172.21.15.131:59736) 2023-12-26T22:53:04.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d1bfff700 1 -- 172.21.15.131:0/4006138170 learned_addr learned my addr 172.21.15.131:0/4006138170 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:04.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d1b7fe700 1 --2- 172.21.15.131:0/4006138170 >> v2:172.21.15.190:3300/0 conn(0x7f9d1c1599d0 0x7f9d1c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d1affd700 1 -- 172.21.15.131:0/4006138170 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067071687 0 0) 0x7f9d1c0cfd50 con 0x7f9d1c159600 2023-12-26T22:53:04.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d1affd700 1 -- 172.21.15.131:0/4006138170 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d04003570 con 0x7f9d1c159600 2023-12-26T22:53:04.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d2099c700 1 -- 172.21.15.131:0/4006138170 >> v2:172.21.15.190:3300/0 conn(0x7f9d1c1599d0 msgr2=0x7f9d1c0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d2099c700 1 --2- 172.21.15.131:0/4006138170 >> v2:172.21.15.190:3300/0 conn(0x7f9d1c1599d0 0x7f9d1c0ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d2099c700 1 -- 172.21.15.131:0/4006138170 >> v1:172.21.15.190:6789/0 conn(0x7f9d1c159600 legacy=0x7f9d1c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d2099c700 1 -- 172.21.15.131:0/4006138170 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c0cf420 con 0x7f9d1c0d01e0 2023-12-26T22:53:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d1b7fe700 1 --2- 172.21.15.131:0/4006138170 >> v2:172.21.15.190:3300/0 conn(0x7f9d1c1599d0 0x7f9d1c0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.412+0000 7f9d2099c700 1 --2- 172.21.15.131:0/4006138170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0d01e0 0x7f9d1c0ccf00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009e60 tx=0x7f9d14009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b581f81679880160 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d1affd700 1 -- 172.21.15.131:0/4006138170 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400e070 con 0x7f9d1c0d01e0 2023-12-26T22:53:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d1affd700 1 -- 172.21.15.131:0/4006138170 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400bab0 con 0x7f9d1c0d01e0 2023-12-26T22:53:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d1affd700 1 -- 172.21.15.131:0/4006138170 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400be70 con 0x7f9d1c0d01e0 2023-12-26T22:53:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d223ff700 1 -- 172.21.15.131:0/4006138170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0d01e0 msgr2=0x7f9d1c0ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:04.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d223ff700 1 --2- 172.21.15.131:0/4006138170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0d01e0 0x7f9d1c0ccf00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009e60 tx=0x7f9d14009780 comp rx=0 tx=0).stop 2023-12-26T22:53:04.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d223ff700 1 -- 172.21.15.131:0/4006138170 shutdown_connections 2023-12-26T22:53:04.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d223ff700 1 --2- 172.21.15.131:0/4006138170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0d01e0 0x7f9d1c0ccf00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d223ff700 1 --2- 172.21.15.131:0/4006138170 >> v2:172.21.15.190:3300/0 conn(0x7f9d1c1599d0 0x7f9d1c0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d223ff700 1 -- 172.21.15.131:0/4006138170 >> 172.21.15.131:0/4006138170 conn(0x7f9d1c06ece0 msgr2=0x7f9d1c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:04.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d223ff700 1 -- 172.21.15.131:0/4006138170 shutdown_connections 2023-12-26T22:53:04.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.413+0000 7f9d223ff700 1 -- 172.21.15.131:0/4006138170 wait complete. 2023-12-26T22:53:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.415+0000 7f9d223ff700 1 Processor -- start 2023-12-26T22:53:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.415+0000 7f9d223ff700 1 -- start start 2023-12-26T22:53:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d223ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0d01e0 0x7f9d1c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d223ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c159600 0x7f9d1c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d223ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c0c8ab0 0x7f9d1c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d223ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c15bb70 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d223ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0c7a10 con 0x7f9d1c0d01e0 2023-12-26T22:53:04.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d223ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0c7cf0 con 0x7f9d1c159600 2023-12-26T22:53:04.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d2099c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c0c8ab0 0x7f9d1c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c159600 0x7f9d1c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d2099c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c0c8ab0 0x7f9d1c0c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46504/0 (socket says 172.21.15.131:46504) 2023-12-26T22:53:04.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d2099c700 1 -- 172.21.15.131:0/787248099 learned_addr learned my addr 172.21.15.131:0/787248099 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:04.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.416+0000 7f9d1bfff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0d01e0 0x7f9d1c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d2099c700 1 -- 172.21.15.131:0/787248099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c159600 msgr2=0x7f9d1c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d2099c700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c159600 0x7f9d1c0c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d2099c700 1 -- 172.21.15.131:0/787248099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0d01e0 msgr2=0x7f9d1c15a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d2099c700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0d01e0 0x7f9d1c15a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d2099c700 1 -- 172.21.15.131:0/787248099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009420 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d2099c700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c0c8ab0 0x7f9d1c0c8f50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f9d14013000 tx=0x7f9d14019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d03fff700 1 -- 172.21.15.131:0/787248099 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14012e10 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d03fff700 1 -- 172.21.15.131:0/787248099 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d14027470 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d14009a20 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d03fff700 1 -- 172.21.15.131:0/787248099 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14027830 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.417+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c0c82b0 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.419+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c0657e0 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.419+0000 7f9d03fff700 1 -- 172.21.15.131:0/787248099 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f9d14024070 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.420+0000 7f9d03fff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9d080670e0 0x7f9d08069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.420+0000 7f9d03fff700 1 -- 172.21.15.131:0/787248099 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f9d140875e0 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.420+0000 7f9d1bfff700 1 -- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9d080670e0 msgr2=0x7f9d08069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:04.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.420+0000 7f9d1bfff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9d080670e0 0x7f9d08069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:04.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.424+0000 7f9d03fff700 1 -- 172.21.15.131:0/787248099 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d14054260 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.472 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:04.473+0000 7fbfb53371c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T22:53:04.532 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:04.532+0000 7f3da90621c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T22:53:04.556 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:04.557+0000 7fbfb53371c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T22:53:04.617 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:04.617+0000 7f3da90621c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T22:53:04.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.621+0000 7f9d1bfff700 1 -- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9d080670e0 msgr2=0x7f9d08069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:04.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.621+0000 7f9d1bfff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9d080670e0 0x7f9d08069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:04.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.624+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d1c0cf700 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.626+0000 7f9d03fff700 1 -- 172.21.15.131:0/787248099 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16497 (secure 0 0 0) 0x7f9d14057880 con 0x7f9d1c0c8ab0 2023-12-26T22:53:04.626 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:04.626 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":162,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:53:00.352922+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:52:37.440528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-26T22:52:38.259993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3597309391":"2023-12-27T22:53:00.352891+0000","172.21.15.131:0/3207268993":"2023-12-27T22:53:00.352891+0000","172.21.15.131:6837/39288":"2023-12-27T22:53:00.352891+0000","172.21.15.131:6836/39288":"2023-12-27T22:53:00.352891+0000","172.21.15.131:0/1148444260":"2023-12-27T22:53:00.352891+0000","172.21.15.131:0/2850349354":"2023-12-27T22:53:00.352891+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:53:04.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9d080670e0 msgr2=0x7f9d08069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:04.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9d080670e0 0x7f9d08069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c0c8ab0 msgr2=0x7f9d1c0c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:04.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c0c8ab0 0x7f9d1c0c8f50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f9d14013000 tx=0x7f9d14019000 comp rx=0 tx=0).stop 2023-12-26T22:53:04.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 shutdown_connections 2023-12-26T22:53:04.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9d080670e0 0x7f9d08069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c0c8ab0 0x7f9d1c0c8f50 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0d01e0 0x7f9d1c15a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 --2- 172.21.15.131:0/787248099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c159600 0x7f9d1c0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.628+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 >> 172.21.15.131:0/787248099 conn(0x7f9d1c06ece0 msgr2=0x7f9d1c0caf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:04.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.629+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 shutdown_connections 2023-12-26T22:53:04.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.629+0000 7f9d223ff700 1 -- 172.21.15.131:0/787248099 wait complete. 2023-12-26T22:53:04.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T22:52:37.440528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T22:52:38.259993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T22:53:04.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T22:53:04.701 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:04.702+0000 7fbfb53371c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:04.772 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:04.773+0000 7f3da90621c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:04.792 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:04.793+0000 7fbfb53371c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T22:53:04.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.846+0000 7f5d66dff700 1 Processor -- start 2023-12-26T22:53:04.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.846+0000 7f5d66dff700 1 -- start start 2023-12-26T22:53:04.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.847+0000 7f5d66dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5d601595e0 0x7f5d601599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.847+0000 7f5d66dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d6015d240 0x7f5d60159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.847+0000 7f5d66dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d600ace50 0x7f5d600ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d66dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d600add40 con 0x7f5d601595e0 2023-12-26T22:53:04.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d66dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d600ad7f0 con 0x7f5d6015d240 2023-12-26T22:53:04.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d66dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600a98b0 con 0x7f5d600ace50 2023-12-26T22:53:04.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d64b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5d601595e0 0x7f5d601599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d64b9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5d601595e0 0x7f5d601599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46518/0 (socket says 172.21.15.131:46518) 2023-12-26T22:53:04.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d64b9b700 1 -- 172.21.15.131:0/1786815681 learned_addr learned my addr 172.21.15.131:0/1786815681 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:04.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d5ffff700 1 --2- 172.21.15.131:0/1786815681 >> v2:172.21.15.190:3300/0 conn(0x7f5d6015d240 0x7f5d60159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d6539c700 1 --2- 172.21.15.131:0/1786815681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d600ace50 0x7f5d600ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d64b9b700 1 -- 172.21.15.131:0/1786815681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d600ace50 msgr2=0x7f5d600ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:04.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d64b9b700 1 --2- 172.21.15.131:0/1786815681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d600ace50 0x7f5d600ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d64b9b700 1 -- 172.21.15.131:0/1786815681 >> v2:172.21.15.190:3300/0 conn(0x7f5d6015d240 msgr2=0x7f5d60159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d64b9b700 1 --2- 172.21.15.131:0/1786815681 >> v2:172.21.15.190:3300/0 conn(0x7f5d6015d240 0x7f5d60159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d64b9b700 1 -- 172.21.15.131:0/1786815681 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6015a500 con 0x7f5d601595e0 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d5ffff700 1 --2- 172.21.15.131:0/1786815681 >> v2:172.21.15.190:3300/0 conn(0x7f5d6015d240 0x7f5d60159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.848+0000 7f5d64b9b700 1 --2- 172.21.15.131:0/1786815681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d601595e0 0x7f5d601599b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5d6006a640 tx=0x7f5d54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fab2cf19f7beaa0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.849+0000 7f5d5f7fe700 1 -- 172.21.15.131:0/1786815681 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400b860 con 0x7f5d601595e0 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.849+0000 7f5d5f7fe700 1 -- 172.21.15.131:0/1786815681 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5400b9c0 con 0x7f5d601595e0 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.849+0000 7f5d5f7fe700 1 -- 172.21.15.131:0/1786815681 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400bd60 con 0x7f5d601595e0 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.849+0000 7f5d66dff700 1 -- 172.21.15.131:0/1786815681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d601595e0 msgr2=0x7f5d601599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.849+0000 7f5d66dff700 1 --2- 172.21.15.131:0/1786815681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d601595e0 0x7f5d601599b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5d6006a640 tx=0x7f5d54009250 comp rx=0 tx=0).stop 2023-12-26T22:53:04.852 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.849+0000 7f5d66dff700 1 -- 172.21.15.131:0/1786815681 shutdown_connections 2023-12-26T22:53:04.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.849+0000 7f5d66dff700 1 --2- 172.21.15.131:0/1786815681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d601595e0 0x7f5d601599b0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.849+0000 7f5d66dff700 1 --2- 172.21.15.131:0/1786815681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d600ace50 0x7f5d600ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.849+0000 7f5d66dff700 1 --2- 172.21.15.131:0/1786815681 >> v2:172.21.15.190:3300/0 conn(0x7f5d6015d240 0x7f5d60159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.850+0000 7f5d66dff700 1 -- 172.21.15.131:0/1786815681 >> 172.21.15.131:0/1786815681 conn(0x7f5d60068040 msgr2=0x7f5d60069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:04.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.850+0000 7f5d66dff700 1 -- 172.21.15.131:0/1786815681 shutdown_connections 2023-12-26T22:53:04.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.850+0000 7f5d66dff700 1 -- 172.21.15.131:0/1786815681 wait complete. 2023-12-26T22:53:04.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.852+0000 7f5d66dff700 1 Processor -- start 2023-12-26T22:53:04.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.852+0000 7f5d66dff700 1 -- start start 2023-12-26T22:53:04.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.853+0000 7f5d66dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d600ace50 0x7f5d60152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.853+0000 7f5d66dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d601595e0 0x7f5d600b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.853+0000 7f5d66dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d6015d240 0x7f5d600c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d66dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600b11c0 con 0x7f5d600ace50 2023-12-26T22:53:04.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d66dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600b05b0 con 0x7f5d601595e0 2023-12-26T22:53:04.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d66dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600b0890 con 0x7f5d6015d240 2023-12-26T22:53:04.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d64b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d600ace50 0x7f5d60152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d64b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d600ace50 0x7f5d60152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46528/0 (socket says 172.21.15.131:46528) 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d64b9b700 1 -- 172.21.15.131:0/3890189785 learned_addr learned my addr 172.21.15.131:0/3890189785 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d5ffff700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d601595e0 0x7f5d600b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d6539c700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d6015d240 0x7f5d600c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d64b9b700 1 -- 172.21.15.131:0/3890189785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d6015d240 msgr2=0x7f5d600c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d64b9b700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d6015d240 0x7f5d600c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d64b9b700 1 -- 172.21.15.131:0/3890189785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d601595e0 msgr2=0x7f5d600b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d64b9b700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d601595e0 0x7f5d600b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d64b9b700 1 -- 172.21.15.131:0/3890189785 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540094f0 con 0x7f5d600ace50 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d6539c700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d6015d240 0x7f5d600c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:04.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.854+0000 7f5d64b9b700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d600ace50 0x7f5d60152fb0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009290 tx=0x7f5d54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:04.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.855+0000 7f5d5cff9700 1 -- 172.21.15.131:0/3890189785 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54026480 con 0x7f5d600ace50 2023-12-26T22:53:04.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.855+0000 7f5d5cff9700 1 -- 172.21.15.131:0/3890189785 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5401f070 con 0x7f5d600ace50 2023-12-26T22:53:04.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.855+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d600b0b70 con 0x7f5d600ace50 2023-12-26T22:53:04.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.855+0000 7f5d5cff9700 1 -- 172.21.15.131:0/3890189785 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54026830 con 0x7f5d600ace50 2023-12-26T22:53:04.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.855+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d601534f0 con 0x7f5d600ace50 2023-12-26T22:53:04.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.856+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d600657e0 con 0x7f5d600ace50 2023-12-26T22:53:04.859 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.857+0000 7f5d5cff9700 1 -- 172.21.15.131:0/3890189785 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f5d54026ba0 con 0x7f5d600ace50 2023-12-26T22:53:04.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.858+0000 7f5d5cff9700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5d440670e0 0x7f5d44069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:04.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.858+0000 7f5d5cff9700 1 -- 172.21.15.131:0/3890189785 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5d54021070 con 0x7f5d600ace50 2023-12-26T22:53:04.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.858+0000 7f5d5ffff700 1 -- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5d440670e0 msgr2=0x7f5d44069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:04.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.858+0000 7f5d5ffff700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5d440670e0 0x7f5d44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:04.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:04.866+0000 7f5d5cff9700 1 -- 172.21.15.131:0/3890189785 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d540536a0 con 0x7f5d600ace50 2023-12-26T22:53:04.871 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:04.871+0000 7f3da90621c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T22:53:04.974 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:04.975+0000 7fbfb53371c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T22:53:05.058 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:05.058+0000 7f3da90621c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T22:53:05.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.059+0000 7f5d5ffff700 1 -- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5d440670e0 msgr2=0x7f5d44069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:05.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.059+0000 7f5d5ffff700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5d440670e0 0x7f5d44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:05.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.102+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d6015a7e0 con 0x7f5d600ace50 2023-12-26T22:53:05.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.102+0000 7f5d5cff9700 1 -- 172.21.15.131:0/3890189785 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d5402b020 con 0x7f5d600ace50 2023-12-26T22:53:05.102 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T22:53:05.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5d440670e0 msgr2=0x7f5d44069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:05.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5d440670e0 0x7f5d44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d600ace50 msgr2=0x7f5d60152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d600ace50 0x7f5d60152fb0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009290 tx=0x7f5d54017000 comp rx=0 tx=0).stop 2023-12-26T22:53:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 shutdown_connections 2023-12-26T22:53:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d6015d240 0x7f5d600c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5d440670e0 0x7f5d44069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d600ace50 0x7f5d60152fb0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 --2- 172.21.15.131:0/3890189785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d601595e0 0x7f5d600b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 >> 172.21.15.131:0/3890189785 conn(0x7f5d60068040 msgr2=0x7f5d600ab020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 shutdown_connections 2023-12-26T22:53:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.105+0000 7f5d66dff700 1 -- 172.21.15.131:0/3890189785 wait complete. 2023-12-26T22:53:05.123 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:05.124+0000 7fbfb53371c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T22:53:05.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T22:53:05.217 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:05.218+0000 7f3da90621c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T22:53:05.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.324+0000 7f3ccd9ff700 1 Processor -- start 2023-12-26T22:53:05.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.324+0000 7f3ccd9ff700 1 -- start start 2023-12-26T22:53:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.325+0000 7f3ccd9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3cc80c8bf0 0x7f3cc80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.325+0000 7f3ccd9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8069d10 0x7f3cc80b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.325+0000 7f3ccd9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc80caa10 con 0x7f3cc80c8820 2023-12-26T22:53:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.325+0000 7f3ccd9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc8157a80 con 0x7f3cc80c8bf0 2023-12-26T22:53:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.325+0000 7f3ccd9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80cac50 con 0x7f3cc8069d10 2023-12-26T22:53:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.325+0000 7f3cc77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8069d10 0x7f3cc80b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.325+0000 7f3cc6ffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f3cc80c8820 0x7f3cc80ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:56116/0 (socket says 172.21.15.131:56116) 2023-12-26T22:53:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc6ffd700 1 -- 172.21.15.131:0/687240530 learned_addr learned my addr 172.21.15.131:0/687240530 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc67fc700 1 --2- 172.21.15.131:0/687240530 >> v2:172.21.15.190:3300/0 conn(0x7f3cc80c8bf0 0x7f3cc80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc5ffb700 1 -- 172.21.15.131:0/687240530 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245313938 0 0) 0x7f3cb8001110 con 0x7f3cc80c8820 2023-12-26T22:53:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc5ffb700 1 -- 172.21.15.131:0/687240530 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb0003570 con 0x7f3cc80c8820 2023-12-26T22:53:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc67fc700 1 -- 172.21.15.131:0/687240530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8069d10 msgr2=0x7f3cc80b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:05.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc67fc700 1 --2- 172.21.15.131:0/687240530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8069d10 0x7f3cc80b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc67fc700 1 -- 172.21.15.131:0/687240530 >> v1:172.21.15.190:6789/0 conn(0x7f3cc80c8820 legacy=0x7f3cc80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc67fc700 1 -- 172.21.15.131:0/687240530 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80caf30 con 0x7f3cc80c8bf0 2023-12-26T22:53:05.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc77fe700 1 --2- 172.21.15.131:0/687240530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8069d10 0x7f3cc80b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:05.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.326+0000 7f3cc67fc700 1 --2- 172.21.15.131:0/687240530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80c8bf0 0x7f3cc80b12e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0098e0 tx=0x7f3cbc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63f2ef05b48a96db server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:05.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3cc5ffb700 1 -- 172.21.15.131:0/687240530 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011630 con 0x7f3cc80c8bf0 2023-12-26T22:53:05.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3cc5ffb700 1 -- 172.21.15.131:0/687240530 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc011790 con 0x7f3cc80c8bf0 2023-12-26T22:53:05.328 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3cc5ffb700 1 -- 172.21.15.131:0/687240530 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011b30 con 0x7f3cc80c8bf0 2023-12-26T22:53:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/687240530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80c8bf0 msgr2=0x7f3cc80b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3ccd9ff700 1 --2- 172.21.15.131:0/687240530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80c8bf0 0x7f3cc80b12e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0098e0 tx=0x7f3cbc0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/687240530 shutdown_connections 2023-12-26T22:53:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3ccd9ff700 1 --2- 172.21.15.131:0/687240530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8069d10 0x7f3cc80b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3ccd9ff700 1 --2- 172.21.15.131:0/687240530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80c8bf0 0x7f3cc80b12e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/687240530 >> 172.21.15.131:0/687240530 conn(0x7f3cc8068040 msgr2=0x7f3cc8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/687240530 shutdown_connections 2023-12-26T22:53:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.327+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/687240530 wait complete. 2023-12-26T22:53:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.329+0000 7f3ccd9ff700 1 Processor -- start 2023-12-26T22:53:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.329+0000 7f3ccd9ff700 1 -- start start 2023-12-26T22:53:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3ccd9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc8069d10 0x7f3cc8153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3ccd9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc80c8820 0x7f3cc81608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3ccd9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80c8bf0 0x7f3cc814d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3ccd9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8160de0 con 0x7f3cc80c8820 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3ccd9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc81535d0 con 0x7f3cc8069d10 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3ccd9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc81529c0 con 0x7f3cc80c8bf0 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3cc67fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc80c8820 0x7f3cc81608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3cc67fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc80c8820 0x7f3cc81608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50354/0 (socket says 172.21.15.131:50354) 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3cc67fc700 1 -- 172.21.15.131:0/3403599994 learned_addr learned my addr 172.21.15.131:0/3403599994 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3cc6ffd700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc8069d10 0x7f3cc8153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.330+0000 7f3cc77fe700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80c8bf0 0x7f3cc814d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3cc67fc700 1 -- 172.21.15.131:0/3403599994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80c8bf0 msgr2=0x7f3cc814d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3cc67fc700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80c8bf0 0x7f3cc814d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3cc67fc700 1 -- 172.21.15.131:0/3403599994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc8069d10 msgr2=0x7f3cc8153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3cc67fc700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc8069d10 0x7f3cc8153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3cc67fc700 1 -- 172.21.15.131:0/3403599994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0009a20 con 0x7f3cc80c8820 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3cc67fc700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc80c8820 0x7f3cc81608a0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc013000 tx=0x7f3cbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3caeffd700 1 -- 172.21.15.131:0/3403599994 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011900 con 0x7f3cc80c8820 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3caeffd700 1 -- 172.21.15.131:0/3403599994 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc011a60 con 0x7f3cc80c8820 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc009420 con 0x7f3cc80c8820 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3caeffd700 1 -- 172.21.15.131:0/3403599994 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc025670 con 0x7f3cc80c8820 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.331+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc8152f80 con 0x7f3cc80c8820 2023-12-26T22:53:05.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.332+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc8153740 con 0x7f3cc80c8820 2023-12-26T22:53:05.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.333+0000 7f3caeffd700 1 -- 172.21.15.131:0/3403599994 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f3cbc023070 con 0x7f3cc80c8820 2023-12-26T22:53:05.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.334+0000 7f3caeffd700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3cb40670e0 0x7f3cb4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.334+0000 7f3caeffd700 1 -- 172.21.15.131:0/3403599994 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f3cbc085f20 con 0x7f3cc80c8820 2023-12-26T22:53:05.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.334+0000 7f3cc6ffd700 1 -- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3cb40670e0 msgr2=0x7f3cb4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:05.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.334+0000 7f3cc6ffd700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3cb40670e0 0x7f3cb4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:05.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.339+0000 7f3caeffd700 1 -- 172.21.15.131:0/3403599994 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cbc052d00 con 0x7f3cc80c8820 2023-12-26T22:53:05.508 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:05.509+0000 7fbfb53371c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T22:53:05.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.535+0000 7f3cc6ffd700 1 -- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3cb40670e0 msgr2=0x7f3cb4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:05.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.535+0000 7f3cc6ffd700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3cb40670e0 0x7f3cb4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:05.539 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.540+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3cc80657e0 con 0x7f3cc80c8820 2023-12-26T22:53:05.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.540+0000 7f3caeffd700 1 -- 172.21.15.131:0/3403599994 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3cbc02a020 con 0x7f3cc80c8820 2023-12-26T22:53:05.540 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T22:53:05.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3cb40670e0 msgr2=0x7f3cb4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:05.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3cb40670e0 0x7f3cb4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc80c8820 msgr2=0x7f3cc81608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc80c8820 0x7f3cc81608a0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc013000 tx=0x7f3cbc017000 comp rx=0 tx=0).stop 2023-12-26T22:53:05.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 shutdown_connections 2023-12-26T22:53:05.543 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80c8bf0 0x7f3cc814d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc8069d10 0x7f3cc8153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3cb40670e0 0x7f3cb4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 --2- 172.21.15.131:0/3403599994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc80c8820 0x7f3cc81608a0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 >> 172.21.15.131:0/3403599994 conn(0x7f3cc8068040 msgr2=0x7f3cc80704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:05.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 shutdown_connections 2023-12-26T22:53:05.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.543+0000 7f3ccd9ff700 1 -- 172.21.15.131:0/3403599994 wait complete. 2023-12-26T22:53:05.561 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T22:53:05.624 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:05.624+0000 7f3da90621c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T22:53:05.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.758+0000 7ff1853ff700 1 Processor -- start 2023-12-26T22:53:05.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.758+0000 7ff1853ff700 1 -- start start 2023-12-26T22:53:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.760+0000 7ff1853ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff180159920 0x7ff1800b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.760+0000 7ff1853ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff180159cf0 0x7ff1800b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.760+0000 7ff1853ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1800c8bf0 con 0x7ff1801554d0 2023-12-26T22:53:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.760+0000 7ff1853ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1800c8e60 con 0x7ff180159cf0 2023-12-26T22:53:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.760+0000 7ff1853ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800c9140 con 0x7ff180159920 2023-12-26T22:53:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.760+0000 7ff17effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff180159920 0x7ff1800b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.760+0000 7ff17e7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff180159cf0 0x7ff1800b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.760+0000 7ff17f7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff1801554d0 0x7ff180155930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:56128/0 (socket says 172.21.15.131:56128) 2023-12-26T22:53:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.760+0000 7ff17f7fe700 1 -- 172.21.15.131:0/2927697145 learned_addr learned my addr 172.21.15.131:0/2927697145 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17dffb700 1 -- 172.21.15.131:0/2927697145 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937142928 0 0) 0x7ff1800c8bf0 con 0x7ff1801554d0 2023-12-26T22:53:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17dffb700 1 -- 172.21.15.131:0/2927697145 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff168003570 con 0x7ff1801554d0 2023-12-26T22:53:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17e7fc700 1 -- 172.21.15.131:0/2927697145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff180159920 msgr2=0x7ff1800b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17e7fc700 1 --2- 172.21.15.131:0/2927697145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff180159920 0x7ff1800b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17e7fc700 1 -- 172.21.15.131:0/2927697145 >> v1:172.21.15.190:6789/0 conn(0x7ff1801554d0 legacy=0x7ff180155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17e7fc700 1 -- 172.21.15.131:0/2927697145 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1800c9420 con 0x7ff180159cf0 2023-12-26T22:53:05.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17effd700 1 --2- 172.21.15.131:0/2927697145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff180159920 0x7ff1800b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17e7fc700 1 --2- 172.21.15.131:0/2927697145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180159cf0 0x7ff1800b0620 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff1740098e0 tx=0x7ff1740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93379b2102771d17 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17dffb700 1 -- 172.21.15.131:0/2927697145 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174011630 con 0x7ff180159cf0 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17dffb700 1 -- 172.21.15.131:0/2927697145 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174011790 con 0x7ff180159cf0 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.761+0000 7ff17dffb700 1 -- 172.21.15.131:0/2927697145 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174011b30 con 0x7ff180159cf0 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.762+0000 7ff1853ff700 1 -- 172.21.15.131:0/2927697145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180159cf0 msgr2=0x7ff1800b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.762+0000 7ff1853ff700 1 --2- 172.21.15.131:0/2927697145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180159cf0 0x7ff1800b0620 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff1740098e0 tx=0x7ff1740091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.762+0000 7ff1853ff700 1 -- 172.21.15.131:0/2927697145 shutdown_connections 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.762+0000 7ff1853ff700 1 --2- 172.21.15.131:0/2927697145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff180159920 0x7ff1800b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.762+0000 7ff1853ff700 1 --2- 172.21.15.131:0/2927697145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180159cf0 0x7ff1800b0620 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.762+0000 7ff1853ff700 1 -- 172.21.15.131:0/2927697145 >> 172.21.15.131:0/2927697145 conn(0x7ff180068040 msgr2=0x7ff180069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:05.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.762+0000 7ff1853ff700 1 -- 172.21.15.131:0/2927697145 shutdown_connections 2023-12-26T22:53:05.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.762+0000 7ff1853ff700 1 -- 172.21.15.131:0/2927697145 wait complete. 2023-12-26T22:53:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.764+0000 7ff1853ff700 1 Processor -- start 2023-12-26T22:53:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.764+0000 7ff1853ff700 1 -- start start 2023-12-26T22:53:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff1853ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1801554d0 0x7ff180156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff1853ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff180159920 0x7ff180154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff1853ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180159cf0 0x7ff18015e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff1853ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff180155130 con 0x7ff180159920 2023-12-26T22:53:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff1853ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18015d650 con 0x7ff180159cf0 2023-12-26T22:53:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff1853ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff180158740 con 0x7ff1801554d0 2023-12-26T22:53:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff180159920 0x7ff180154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff180159920 0x7ff180154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50362/0 (socket says 172.21.15.131:50362) 2023-12-26T22:53:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17e7fc700 1 -- 172.21.15.131:0/2107005074 learned_addr learned my addr 172.21.15.131:0/2107005074 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17f7fe700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180159cf0 0x7ff18015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17effd700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1801554d0 0x7ff180156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17e7fc700 1 -- 172.21.15.131:0/2107005074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1801554d0 msgr2=0x7ff180156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17e7fc700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1801554d0 0x7ff180156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17e7fc700 1 -- 172.21.15.131:0/2107005074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180159cf0 msgr2=0x7ff18015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17e7fc700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180159cf0 0x7ff18015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.765+0000 7ff17e7fc700 1 -- 172.21.15.131:0/2107005074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17000a5d0 con 0x7ff180159920 2023-12-26T22:53:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.766+0000 7ff17e7fc700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff180159920 0x7ff180154bf0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff1800688f0 tx=0x7ff174017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.766+0000 7ff166ffd700 1 -- 172.21.15.131:0/2107005074 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174011900 con 0x7ff180159920 2023-12-26T22:53:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.766+0000 7ff166ffd700 1 -- 172.21.15.131:0/2107005074 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174011a60 con 0x7ff180159920 2023-12-26T22:53:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.766+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff174009420 con 0x7ff180159920 2023-12-26T22:53:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.766+0000 7ff166ffd700 1 -- 172.21.15.131:0/2107005074 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174025640 con 0x7ff180159920 2023-12-26T22:53:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.766+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18015d860 con 0x7ff180159920 2023-12-26T22:53:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.767+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18006f800 con 0x7ff180159920 2023-12-26T22:53:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.768+0000 7ff166ffd700 1 -- 172.21.15.131:0/2107005074 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7ff174025a30 con 0x7ff180159920 2023-12-26T22:53:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.769+0000 7ff166ffd700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff16c0670e0 0x7ff16c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.769+0000 7ff166ffd700 1 -- 172.21.15.131:0/2107005074 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff174086200 con 0x7ff180159920 2023-12-26T22:53:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.769+0000 7ff17effd700 1 -- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff16c0670e0 msgr2=0x7ff16c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.769+0000 7ff17effd700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff16c0670e0 0x7ff16c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.773+0000 7ff166ffd700 1 -- 172.21.15.131:0/2107005074 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff174052f20 con 0x7ff180159920 2023-12-26T22:53:05.775 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:05.776+0000 7fbfb53371c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T22:53:05.931 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:05.931+0000 7f3da90621c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T22:53:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.970+0000 7ff17effd700 1 -- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff16c0670e0 msgr2=0x7ff16c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:05.970+0000 7ff17effd700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff16c0670e0 0x7ff16c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.009+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1801588d0 con 0x7ff180159920 2023-12-26T22:53:06.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.010+0000 7ff166ffd700 1 -- 172.21.15.131:0/2107005074 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff174056540 con 0x7ff180159920 2023-12-26T22:53:06.009 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T22:53:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff16c0670e0 msgr2=0x7ff16c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff16c0670e0 0x7ff16c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff180159920 msgr2=0x7ff180154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff180159920 0x7ff180154bf0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff1800688f0 tx=0x7ff174017000 comp rx=0 tx=0).stop 2023-12-26T22:53:06.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 shutdown_connections 2023-12-26T22:53:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180159cf0 0x7ff18015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1801554d0 0x7ff180156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff16c0670e0 0x7ff16c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 --2- 172.21.15.131:0/2107005074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff180159920 0x7ff180154bf0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 >> 172.21.15.131:0/2107005074 conn(0x7ff180068040 msgr2=0x7ff1800b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 shutdown_connections 2023-12-26T22:53:06.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.012+0000 7ff1853ff700 1 -- 172.21.15.131:0/2107005074 wait complete. 2023-12-26T22:53:06.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:53:06.055 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:06.056+0000 7fbfb53371c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T22:53:06.145 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:06.146+0000 7fbfb53371c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T22:53:06.222 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:06.222+0000 7f3da90621c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T22:53:06.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.228+0000 7fd638bff700 1 Processor -- start 2023-12-26T22:53:06.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.228+0000 7fd638bff700 1 -- start start 2023-12-26T22:53:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.229+0000 7fd638bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6340c8820 0x7fd6340ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.229+0000 7fd638bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6340c8bf0 0x7fd6340b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.229+0000 7fd638bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd634069d10 0x7fd6340b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.229+0000 7fd638bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd634157a80 con 0x7fd6340c8820 2023-12-26T22:53:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.229+0000 7fd638bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6340cac50 con 0x7fd634069d10 2023-12-26T22:53:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.229+0000 7fd638bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340caf30 con 0x7fd6340c8bf0 2023-12-26T22:53:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.229+0000 7fd63259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6340c8820 0x7fd6340ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd63259c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6340c8820 0x7fd6340ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50374/0 (socket says 172.21.15.131:50374) 2023-12-26T22:53:06.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd63259c700 1 -- 172.21.15.131:0/1204269446 learned_addr learned my addr 172.21.15.131:0/1204269446 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:06.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd632d9d700 1 --2- 172.21.15.131:0/1204269446 >> v2:172.21.15.190:3300/0 conn(0x7fd634069d10 0x7fd6340b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd631d9b700 1 --2- 172.21.15.131:0/1204269446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6340c8bf0 0x7fd6340b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd63259c700 1 -- 172.21.15.131:0/1204269446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6340c8bf0 msgr2=0x7fd6340b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:06.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd63259c700 1 --2- 172.21.15.131:0/1204269446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6340c8bf0 0x7fd6340b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd63259c700 1 -- 172.21.15.131:0/1204269446 >> v2:172.21.15.190:3300/0 conn(0x7fd634069d10 msgr2=0x7fd6340b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:06.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd63259c700 1 --2- 172.21.15.131:0/1204269446 >> v2:172.21.15.190:3300/0 conn(0x7fd634069d10 0x7fd6340b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd63259c700 1 -- 172.21.15.131:0/1204269446 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6340cb210 con 0x7fd6340c8820 2023-12-26T22:53:06.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd63259c700 1 --2- 172.21.15.131:0/1204269446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 0x7fd6340ca300 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fd63406b560 tx=0x7fd624009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebe22ba4ebe39d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:06.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.230+0000 7fd63159a700 1 -- 172.21.15.131:0/1204269446 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400b640 con 0x7fd6340c8820 2023-12-26T22:53:06.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd63159a700 1 -- 172.21.15.131:0/1204269446 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62400b7a0 con 0x7fd6340c8820 2023-12-26T22:53:06.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd63159a700 1 -- 172.21.15.131:0/1204269446 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400bb60 con 0x7fd6340c8820 2023-12-26T22:53:06.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd638bff700 1 -- 172.21.15.131:0/1204269446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 msgr2=0x7fd6340ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd638bff700 1 --2- 172.21.15.131:0/1204269446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 0x7fd6340ca300 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fd63406b560 tx=0x7fd624009250 comp rx=0 tx=0).stop 2023-12-26T22:53:06.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd638bff700 1 -- 172.21.15.131:0/1204269446 shutdown_connections 2023-12-26T22:53:06.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd638bff700 1 --2- 172.21.15.131:0/1204269446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 0x7fd6340ca300 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd638bff700 1 --2- 172.21.15.131:0/1204269446 >> v2:172.21.15.190:3300/0 conn(0x7fd634069d10 0x7fd6340b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd638bff700 1 --2- 172.21.15.131:0/1204269446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6340c8bf0 0x7fd6340b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd638bff700 1 -- 172.21.15.131:0/1204269446 >> 172.21.15.131:0/1204269446 conn(0x7fd634068040 msgr2=0x7fd634069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:06.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd638bff700 1 -- 172.21.15.131:0/1204269446 shutdown_connections 2023-12-26T22:53:06.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.231+0000 7fd638bff700 1 -- 172.21.15.131:0/1204269446 wait complete. 2023-12-26T22:53:06.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.233+0000 7fd638bff700 1 Processor -- start 2023-12-26T22:53:06.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.233+0000 7fd638bff700 1 -- start start 2023-12-26T22:53:06.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd638bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd634069d10 0x7fd6340a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd638bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 0x7fd6340c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd638bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6340c8bf0 0x7fd6340c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd638bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340addb0 con 0x7fd6340c8820 2023-12-26T22:53:06.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd638bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340c2f10 con 0x7fd6340c8bf0 2023-12-26T22:53:06.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd638bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340ad7f0 con 0x7fd634069d10 2023-12-26T22:53:06.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd631d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 0x7fd6340c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd631d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 0x7fd6340c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50390/0 (socket says 172.21.15.131:50390) 2023-12-26T22:53:06.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd631d9b700 1 -- 172.21.15.131:0/2573567912 learned_addr learned my addr 172.21.15.131:0/2573567912 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:06.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd63259c700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd634069d10 0x7fd6340a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd632d9d700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6340c8bf0 0x7fd6340c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd631d9b700 1 -- 172.21.15.131:0/2573567912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd634069d10 msgr2=0x7fd6340a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd631d9b700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd634069d10 0x7fd6340a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd631d9b700 1 -- 172.21.15.131:0/2573567912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6340c8bf0 msgr2=0x7fd6340c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd631d9b700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6340c8bf0 0x7fd6340c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.234+0000 7fd631d9b700 1 -- 172.21.15.131:0/2573567912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6240094f0 con 0x7fd6340c8820 2023-12-26T22:53:06.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.235+0000 7fd631d9b700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 0x7fd6340c6260 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fd62800ec80 tx=0x7fd6280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:06.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.235+0000 7fd6227fc700 1 -- 172.21.15.131:0/2573567912 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62800c850 con 0x7fd6340c8820 2023-12-26T22:53:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.235+0000 7fd6227fc700 1 -- 172.21.15.131:0/2573567912 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62800c9d0 con 0x7fd6340c8820 2023-12-26T22:53:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.235+0000 7fd6227fc700 1 -- 172.21.15.131:0/2573567912 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62800cd90 con 0x7fd6340c8820 2023-12-26T22:53:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.235+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6340adad0 con 0x7fd6340c8820 2023-12-26T22:53:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.235+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6340a63c0 con 0x7fd6340c8820 2023-12-26T22:53:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.236+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63406be30 con 0x7fd6340c8820 2023-12-26T22:53:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.237+0000 7fd6227fc700 1 -- 172.21.15.131:0/2573567912 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fd628013680 con 0x7fd6340c8820 2023-12-26T22:53:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.238+0000 7fd6227fc700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd6140670e0 0x7fd614069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.238+0000 7fd6227fc700 1 -- 172.21.15.131:0/2573567912 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fd628084d50 con 0x7fd6340c8820 2023-12-26T22:53:06.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.238+0000 7fd63259c700 1 -- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd6140670e0 msgr2=0x7fd614069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:06.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.238+0000 7fd63259c700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd6140670e0 0x7fd614069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:06.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.242+0000 7fd6227fc700 1 -- 172.21.15.131:0/2573567912 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd628088030 con 0x7fd6340c8820 2023-12-26T22:53:06.317 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:06.318+0000 7f3da90621c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T22:53:06.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.439+0000 7fd63259c700 1 -- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd6140670e0 msgr2=0x7fd614069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:06.438 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.439+0000 7fd63259c700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd6140670e0 0x7fd614069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:06.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.488+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6340b20b0 con 0x7fd6340c8820 2023-12-26T22:53:06.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.489+0000 7fd6227fc700 1 -- 172.21.15.131:0/2573567912 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fd628051a70 con 0x7fd6340c8820 2023-12-26T22:53:06.488 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:06.488 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2085575956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2085575956},{"type":"v1","addr":"172.21.15.190:6834","nonce":2085575956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2129957224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2129957224},{"type":"v1","addr":"172.21.15.190:6835","nonce":2129957224}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3810273571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3810273571},{"type":"v1","addr":"172.21.15.131:6834","nonce":3810273571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:52:39.271885+0000","modified":"2023-12-26T22:52:42.824355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8628":{"gid":8628,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2240171052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2240171052},{"type":"v1","addr":"172.21.15.131:6835","nonce":2240171052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-12-26T22:53:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.492+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd6140670e0 msgr2=0x7fd614069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.492+0000 7fd638bff700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd6140670e0 0x7fd614069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.492+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 msgr2=0x7fd6340c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.492+0000 7fd638bff700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 0x7fd6340c6260 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fd62800ec80 tx=0x7fd6280091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.492+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 shutdown_connections 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.492+0000 7fd638bff700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6340c8bf0 0x7fd6340c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.492+0000 7fd638bff700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd634069d10 0x7fd6340a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.492+0000 7fd638bff700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fd6140670e0 0x7fd614069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.492+0000 7fd638bff700 1 --2- 172.21.15.131:0/2573567912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6340c8820 0x7fd6340c6260 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.493+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 >> 172.21.15.131:0/2573567912 conn(0x7fd634068040 msgr2=0x7fd6341567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.493+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 shutdown_connections 2023-12-26T22:53:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.493+0000 7fd638bff700 1 -- 172.21.15.131:0/2573567912 wait complete. 2023-12-26T22:53:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 73 2023-12-26T22:53:06.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:53:06.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.712+0000 7ff5db5ff700 1 Processor -- start 2023-12-26T22:53:06.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.712+0000 7ff5db5ff700 1 -- start start 2023-12-26T22:53:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.713+0000 7ff5db5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff5d41595e0 0x7ff5d41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.713+0000 7ff5db5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d40a98b0 con 0x7ff5d40ad560 2023-12-26T22:53:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.713+0000 7ff5db5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d415a500 con 0x7ff5d415d240 2023-12-26T22:53:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.713+0000 7ff5db5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d40add40 con 0x7ff5d41595e0 2023-12-26T22:53:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.714+0000 7ff5d939b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff5d41595e0 0x7ff5d41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.714+0000 7ff5d9b9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff5d40ad560 0x7ff5d40aa020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37202/0 (socket says 172.21.15.131:37202) 2023-12-26T22:53:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.714+0000 7ff5d9b9c700 1 -- 172.21.15.131:0/4187059388 learned_addr learned my addr 172.21.15.131:0/4187059388 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.714+0000 7ff5cbfff700 1 -- 172.21.15.131:0/4187059388 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191877110 0 0) 0x7ff5d40a98b0 con 0x7ff5d40ad560 2023-12-26T22:53:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.714+0000 7ff5d939b700 1 -- 172.21.15.131:0/4187059388 >> v1:172.21.15.190:6789/0 conn(0x7ff5d415d240 legacy=0x7ff5d40ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.714+0000 7ff5d939b700 1 -- 172.21.15.131:0/4187059388 >> v1:172.21.15.131:6789/0 conn(0x7ff5d40ad560 legacy=0x7ff5d40aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.714+0000 7ff5d939b700 1 -- 172.21.15.131:0/4187059388 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d415a7e0 con 0x7ff5d41595e0 2023-12-26T22:53:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.714+0000 7ff5d939b700 1 --2- 172.21.15.131:0/4187059388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 0x7ff5d41599b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00098e0 tx=0x7ff5d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece643774f196c2e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5cbfff700 1 -- 172.21.15.131:0/4187059388 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0009e80 con 0x7ff5d41595e0 2023-12-26T22:53:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5cbfff700 1 -- 172.21.15.131:0/4187059388 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d000b860 con 0x7ff5d41595e0 2023-12-26T22:53:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5cbfff700 1 -- 172.21.15.131:0/4187059388 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000bbe0 con 0x7ff5d41595e0 2023-12-26T22:53:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5db5ff700 1 -- 172.21.15.131:0/4187059388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 msgr2=0x7ff5d41599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5db5ff700 1 --2- 172.21.15.131:0/4187059388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 0x7ff5d41599b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00098e0 tx=0x7ff5d00091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5db5ff700 1 -- 172.21.15.131:0/4187059388 shutdown_connections 2023-12-26T22:53:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5db5ff700 1 --2- 172.21.15.131:0/4187059388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 0x7ff5d41599b0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5db5ff700 1 -- 172.21.15.131:0/4187059388 >> 172.21.15.131:0/4187059388 conn(0x7ff5d4068040 msgr2=0x7ff5d4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5db5ff700 1 -- 172.21.15.131:0/4187059388 shutdown_connections 2023-12-26T22:53:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.715+0000 7ff5db5ff700 1 -- 172.21.15.131:0/4187059388 wait complete. 2023-12-26T22:53:06.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.717+0000 7ff5db5ff700 1 Processor -- start 2023-12-26T22:53:06.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.717+0000 7ff5db5ff700 1 -- start start 2023-12-26T22:53:06.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5db5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d40ad560 0x7ff5d416ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5db5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 0x7ff5d40cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5db5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d415d240 0x7ff5d40c2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5db5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d416b2a0 con 0x7ff5d41595e0 2023-12-26T22:53:06.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5db5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40c1f80 con 0x7ff5d40ad560 2023-12-26T22:53:06.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5db5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40cbe50 con 0x7ff5d415d240 2023-12-26T22:53:06.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5d8b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 0x7ff5d40cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5d939b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d40ad560 0x7ff5d416ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5d8b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 0x7ff5d40cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50404/0 (socket says 172.21.15.131:50404) 2023-12-26T22:53:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5d8b9a700 1 -- 172.21.15.131:0/2501836394 learned_addr learned my addr 172.21.15.131:0/2501836394 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.718+0000 7ff5d9b9c700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d415d240 0x7ff5d40c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5d8b9a700 1 -- 172.21.15.131:0/2501836394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d415d240 msgr2=0x7ff5d40c2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5d8b9a700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d415d240 0x7ff5d40c2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5d8b9a700 1 -- 172.21.15.131:0/2501836394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d40ad560 msgr2=0x7ff5d416ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5d8b9a700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d40ad560 0x7ff5d416ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5d8b9a700 1 -- 172.21.15.131:0/2501836394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d0009420 con 0x7ff5d41595e0 2023-12-26T22:53:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5d9b9c700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d415d240 0x7ff5d40c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5d939b700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d40ad560 0x7ff5d416ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5d8b9a700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 0x7ff5d40cb910 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400c4e0 tx=0x7ff5c400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5c97fa700 1 -- 172.21.15.131:0/2501836394 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4011590 con 0x7ff5d41595e0 2023-12-26T22:53:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5c97fa700 1 -- 172.21.15.131:0/2501836394 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c4011710 con 0x7ff5d41595e0 2023-12-26T22:53:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d40c1370 con 0x7ff5d41595e0 2023-12-26T22:53:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5c97fa700 1 -- 172.21.15.131:0/2501836394 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4011ad0 con 0x7ff5d41595e0 2023-12-26T22:53:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.719+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d40c1930 con 0x7ff5d41595e0 2023-12-26T22:53:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.721+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d40657e0 con 0x7ff5d41595e0 2023-12-26T22:53:06.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.721+0000 7ff5c97fa700 1 -- 172.21.15.131:0/2501836394 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7ff5c401b3b0 con 0x7ff5d41595e0 2023-12-26T22:53:06.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.722+0000 7ff5c97fa700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff5b80670e0 0x7ff5b8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:06.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.722+0000 7ff5c97fa700 1 -- 172.21.15.131:0/2501836394 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff5c4083f90 con 0x7ff5d41595e0 2023-12-26T22:53:06.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.722+0000 7ff5d939b700 1 -- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff5b80670e0 msgr2=0x7ff5b8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:06.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.722+0000 7ff5d939b700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff5b80670e0 0x7ff5b8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:06.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.726+0000 7ff5c97fa700 1 -- 172.21.15.131:0/2501836394 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5c4050cb0 con 0x7ff5d41595e0 2023-12-26T22:53:06.879 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:06.880+0000 7fbfb53371c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T22:53:06.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.923+0000 7ff5d939b700 1 -- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff5b80670e0 msgr2=0x7ff5b8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:06.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.923+0000 7ff5d939b700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff5b80670e0 0x7ff5b8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.931+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5d415aac0 con 0x7ff5d41595e0 2023-12-26T22:53:06.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.932+0000 7ff5c97fa700 1 -- 172.21.15.131:0/2501836394 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16497 (secure 0 0 0) 0x7ff5c4017070 con 0x7ff5d41595e0 2023-12-26T22:53:06.932 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:06.932 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":162,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:53:00.352922+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:52:37.440528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-26T22:52:38.259993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3597309391":"2023-12-27T22:53:00.352891+0000","172.21.15.131:0/3207268993":"2023-12-27T22:53:00.352891+0000","172.21.15.131:6837/39288":"2023-12-27T22:53:00.352891+0000","172.21.15.131:6836/39288":"2023-12-27T22:53:00.352891+0000","172.21.15.131:0/1148444260":"2023-12-27T22:53:00.352891+0000","172.21.15.131:0/2850349354":"2023-12-27T22:53:00.352891+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:53:06.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff5b80670e0 msgr2=0x7ff5b8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:06.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff5b80670e0 0x7ff5b8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 msgr2=0x7ff5d40cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:06.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 0x7ff5d40cb910 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400c4e0 tx=0x7ff5c400c7f0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 shutdown_connections 2023-12-26T22:53:06.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5d415d240 0x7ff5d40c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5d40ad560 0x7ff5d416ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff5b80670e0 0x7ff5b8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 --2- 172.21.15.131:0/2501836394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5d41595e0 0x7ff5d40cb910 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:06.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 >> 172.21.15.131:0/2501836394 conn(0x7ff5d4068040 msgr2=0x7ff5d40aadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:06.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 shutdown_connections 2023-12-26T22:53:06.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:06.935+0000 7ff5db5ff700 1 -- 172.21.15.131:0/2501836394 wait complete. 2023-12-26T22:53:06.954 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T22:53:06.954 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:53:06.973 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:06.974+0000 7fbfb53371c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T22:53:07.112 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:07.112+0000 7f3da90621c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T22:53:07.126 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:07.127+0000 7fbfb53371c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T22:53:07.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.148+0000 7fe7ed7ff700 1 Processor -- start 2023-12-26T22:53:07.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.148+0000 7fe7ed7ff700 1 -- start start 2023-12-26T22:53:07.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.149+0000 7fe7ed7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7e81596b0 0x7fe7e8067f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.149+0000 7fe7ed7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7e8159a80 0x7fe7e80c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.149+0000 7fe7ed7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e815ac30 con 0x7fe7e80c7df0 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.149+0000 7fe7ed7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e80c8200 con 0x7fe7e8159a80 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.149+0000 7fe7ed7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e8068480 con 0x7fe7e81596b0 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e67fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7e8159a80 0x7fe7e80c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e77fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe7e80c7df0 0x7fe7e806a6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37204/0 (socket says 172.21.15.131:37204) 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e67fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe7e8159a80 0x7fe7e80c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50412/0 (socket says 172.21.15.131:50412) 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e6ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7e81596b0 0x7fe7e8067f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e77fe700 1 -- 172.21.15.131:0/664251065 learned_addr learned my addr 172.21.15.131:0/664251065 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e67fc700 1 -- 172.21.15.131:0/664251065 >> v2:172.21.15.190:3300/0 conn(0x7fe7e81596b0 msgr2=0x7fe7e8067f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:07.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e67fc700 1 --2- 172.21.15.131:0/664251065 >> v2:172.21.15.190:3300/0 conn(0x7fe7e81596b0 0x7fe7e8067f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e67fc700 1 -- 172.21.15.131:0/664251065 >> v1:172.21.15.131:6789/0 conn(0x7fe7e80c7df0 legacy=0x7fe7e806a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e67fc700 1 -- 172.21.15.131:0/664251065 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e815a020 con 0x7fe7e8159a80 2023-12-26T22:53:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e6ffd700 1 --2- 172.21.15.131:0/664251065 >> v2:172.21.15.190:3300/0 conn(0x7fe7e81596b0 0x7fe7e8067f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.150+0000 7fe7e5ffb700 1 -- 172.21.15.131:0/664251065 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243391283 0 0) 0x7fe7e815ac30 con 0x7fe7e80c7df0 2023-12-26T22:53:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.151+0000 7fe7e67fc700 1 --2- 172.21.15.131:0/664251065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e8159a80 0x7fe7e80c78b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe7e806a230 tx=0x7fe7dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf12281cc5f4f9a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.151+0000 7fe7e5ffb700 1 -- 172.21.15.131:0/664251065 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc011630 con 0x7fe7e8159a80 2023-12-26T22:53:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.151+0000 7fe7e5ffb700 1 -- 172.21.15.131:0/664251065 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc011790 con 0x7fe7e8159a80 2023-12-26T22:53:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.151+0000 7fe7e5ffb700 1 -- 172.21.15.131:0/664251065 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc011b30 con 0x7fe7e8159a80 2023-12-26T22:53:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.151+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/664251065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e8159a80 msgr2=0x7fe7e80c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.151+0000 7fe7ed7ff700 1 --2- 172.21.15.131:0/664251065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e8159a80 0x7fe7e80c78b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe7e806a230 tx=0x7fe7dc0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.152+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/664251065 shutdown_connections 2023-12-26T22:53:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.152+0000 7fe7ed7ff700 1 --2- 172.21.15.131:0/664251065 >> v2:172.21.15.190:3300/0 conn(0x7fe7e81596b0 0x7fe7e8067f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.152+0000 7fe7ed7ff700 1 --2- 172.21.15.131:0/664251065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e8159a80 0x7fe7e80c78b0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.152+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/664251065 >> 172.21.15.131:0/664251065 conn(0x7fe7e80686c0 msgr2=0x7fe7e8068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.152+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/664251065 shutdown_connections 2023-12-26T22:53:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.152+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/664251065 wait complete. 2023-12-26T22:53:07.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.153+0000 7fe7ed7ff700 1 Processor -- start 2023-12-26T22:53:07.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.153+0000 7fe7ed7ff700 1 -- start start 2023-12-26T22:53:07.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.154+0000 7fe7ed7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e80c7df0 0x7fe7e80c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.154+0000 7fe7ed7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e81596b0 0x7fe7e8156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.154+0000 7fe7ed7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e8159a80 0x7fe7e80b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.154+0000 7fe7ed7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e815d540 con 0x7fe7e8159a80 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.154+0000 7fe7ed7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e815c930 con 0x7fe7e81596b0 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.154+0000 7fe7ed7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e815cc10 con 0x7fe7e80c7df0 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e67fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e81596b0 0x7fe7e8156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e8159a80 0x7fe7e80b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e67fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e81596b0 0x7fe7e8156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:35794/0 (socket says 172.21.15.131:35794) 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e67fc700 1 -- 172.21.15.131:0/1548127390 learned_addr learned my addr 172.21.15.131:0/1548127390 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e6ffd700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e80c7df0 0x7fe7e80c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e67fc700 1 -- 172.21.15.131:0/1548127390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e80c7df0 msgr2=0x7fe7e80c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e67fc700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e80c7df0 0x7fe7e80c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e67fc700 1 -- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e8159a80 msgr2=0x7fe7e80b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e67fc700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e8159a80 0x7fe7e80b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e67fc700 1 -- 172.21.15.131:0/1548127390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d8009420 con 0x7fe7e81596b0 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.155+0000 7fe7e67fc700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e81596b0 0x7fe7e8156720 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe7e806dfa0 tx=0x7fe7dc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.156+0000 7fe7ceffd700 1 -- 172.21.15.131:0/1548127390 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc000d10 con 0x7fe7e81596b0 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.156+0000 7fe7ceffd700 1 -- 172.21.15.131:0/1548127390 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc011b10 con 0x7fe7e81596b0 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.156+0000 7fe7ceffd700 1 -- 172.21.15.131:0/1548127390 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc025640 con 0x7fe7e81596b0 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.156+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc009420 con 0x7fe7e81596b0 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.156+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e815d1d0 con 0x7fe7e81596b0 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.157+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e815d6f0 con 0x7fe7e81596b0 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.158+0000 7fe7ceffd700 1 -- 172.21.15.131:0/1548127390 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fe7dc025a30 con 0x7fe7e81596b0 2023-12-26T22:53:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.158+0000 7fe7ceffd700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe7d40670e0 0x7fe7d4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.158+0000 7fe7ceffd700 1 -- 172.21.15.131:0/1548127390 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fe7dc0863c0 con 0x7fe7e81596b0 2023-12-26T22:53:07.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.158+0000 7fe7e6ffd700 1 -- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe7d40670e0 msgr2=0x7fe7d4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:07.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.158+0000 7fe7e6ffd700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe7d40670e0 0x7fe7d4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:07.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.163+0000 7fe7ceffd700 1 -- 172.21.15.131:0/1548127390 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7e815d6f0 con 0x7fe7e81596b0 2023-12-26T22:53:07.212 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:07.212+0000 7f3da90621c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T22:53:07.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.360+0000 7fe7e6ffd700 1 -- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe7d40670e0 msgr2=0x7fe7d4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:07.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.360+0000 7fe7e6ffd700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe7d40670e0 0x7fe7d4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:07.370 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:07.370+0000 7f3da90621c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T22:53:07.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.409+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7e815d6b0 con 0x7fe7e81596b0 2023-12-26T22:53:07.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.410+0000 7fe7ceffd700 1 -- 172.21.15.131:0/1548127390 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fe7dc053040 con 0x7fe7e81596b0 2023-12-26T22:53:07.410 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:07.410 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2085575956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2085575956},{"type":"v1","addr":"172.21.15.190:6834","nonce":2085575956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2129957224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2129957224},{"type":"v1","addr":"172.21.15.190:6835","nonce":2129957224}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3810273571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3810273571},{"type":"v1","addr":"172.21.15.131:6834","nonce":3810273571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:52:39.271885+0000","modified":"2023-12-26T22:52:42.824355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8628":{"gid":8628,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2240171052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2240171052},{"type":"v1","addr":"172.21.15.131:6835","nonce":2240171052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-12-26T22:53:07.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe7d40670e0 msgr2=0x7fe7d4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:07.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe7d40670e0 0x7fe7d4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e81596b0 msgr2=0x7fe7e8156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e81596b0 0x7fe7e8156720 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe7e806dfa0 tx=0x7fe7dc017000 comp rx=0 tx=0).stop 2023-12-26T22:53:07.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 shutdown_connections 2023-12-26T22:53:07.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fe7d40670e0 0x7fe7d4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7e8159a80 0x7fe7e80b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e80c7df0 0x7fe7e80c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 --2- 172.21.15.131:0/1548127390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e81596b0 0x7fe7e8156720 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 >> 172.21.15.131:0/1548127390 conn(0x7fe7e80686c0 msgr2=0x7fe7e80cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:07.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.414+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 shutdown_connections 2023-12-26T22:53:07.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.415+0000 7fe7ed7ff700 1 -- 172.21.15.131:0/1548127390 wait complete. 2023-12-26T22:53:07.416 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 73 2023-12-26T22:53:07.433 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:53:07.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.637+0000 7ff7dbbff700 1 Processor -- start 2023-12-26T22:53:07.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.637+0000 7ff7dbbff700 1 -- start start 2023-12-26T22:53:07.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.638+0000 7ff7dbbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d4159820 0x7ff7d40abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.638+0000 7ff7dbbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff7d40aa480 0x7ff7d406dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.638+0000 7ff7dbbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d415a8c0 con 0x7ff7d406e2b0 2023-12-26T22:53:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.638+0000 7ff7dbbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d415ad50 con 0x7ff7d40aa480 2023-12-26T22:53:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.638+0000 7ff7dbbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d406e730 con 0x7ff7d4159820 2023-12-26T22:53:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7d919a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff7d40aa480 0x7ff7d406dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7da19c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff7d406e2b0 0x7ff7d415d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37218/0 (socket says 172.21.15.131:37218) 2023-12-26T22:53:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7d919a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff7d40aa480 0x7ff7d406dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50432/0 (socket says 172.21.15.131:50432) 2023-12-26T22:53:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7da19c700 1 -- 172.21.15.131:0/3105642604 learned_addr learned my addr 172.21.15.131:0/3105642604 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7d999b700 1 --2- 172.21.15.131:0/3105642604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d4159820 0x7ff7d40abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7d919a700 1 -- 172.21.15.131:0/3105642604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d4159820 msgr2=0x7ff7d40abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7d919a700 1 --2- 172.21.15.131:0/3105642604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d4159820 0x7ff7d40abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7d919a700 1 -- 172.21.15.131:0/3105642604 >> v1:172.21.15.131:6789/0 conn(0x7ff7d406e2b0 legacy=0x7ff7d415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7d919a700 1 -- 172.21.15.131:0/3105642604 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4159f90 con 0x7ff7d40aa480 2023-12-26T22:53:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7d8999700 1 -- 172.21.15.131:0/3105642604 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713070702 0 0) 0x7ff7d415a8c0 con 0x7ff7d406e2b0 2023-12-26T22:53:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.639+0000 7ff7d919a700 1 --2- 172.21.15.131:0/3105642604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d40aa480 0x7ff7d406dd70 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009910 tx=0x7ff7c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bd2214251cb16e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7d8999700 1 -- 172.21.15.131:0/3105642604 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400b640 con 0x7ff7d40aa480 2023-12-26T22:53:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7d8999700 1 -- 172.21.15.131:0/3105642604 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c400b7a0 con 0x7ff7d40aa480 2023-12-26T22:53:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7d8999700 1 -- 172.21.15.131:0/3105642604 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400bb60 con 0x7ff7d40aa480 2023-12-26T22:53:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7dbbff700 1 -- 172.21.15.131:0/3105642604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d40aa480 msgr2=0x7ff7d406dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7dbbff700 1 --2- 172.21.15.131:0/3105642604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d40aa480 0x7ff7d406dd70 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009910 tx=0x7ff7c40091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7dbbff700 1 -- 172.21.15.131:0/3105642604 shutdown_connections 2023-12-26T22:53:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7dbbff700 1 --2- 172.21.15.131:0/3105642604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d4159820 0x7ff7d40abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7dbbff700 1 --2- 172.21.15.131:0/3105642604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d40aa480 0x7ff7d406dd70 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7dbbff700 1 -- 172.21.15.131:0/3105642604 >> 172.21.15.131:0/3105642604 conn(0x7ff7d406c290 msgr2=0x7ff7d40ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7dbbff700 1 -- 172.21.15.131:0/3105642604 shutdown_connections 2023-12-26T22:53:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.640+0000 7ff7dbbff700 1 -- 172.21.15.131:0/3105642604 wait complete. 2023-12-26T22:53:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.642+0000 7ff7dbbff700 1 Processor -- start 2023-12-26T22:53:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.642+0000 7ff7dbbff700 1 -- start start 2023-12-26T22:53:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7dbbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d406e2b0 0x7ff7d416afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7dbbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40aa480 0x7ff7d416dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7dbbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4159820 0x7ff7d40cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7dbbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d416b5f0 con 0x7ff7d4159820 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7dbbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d416e020 con 0x7ff7d406e2b0 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7dbbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d416b8b0 con 0x7ff7d40aa480 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7da19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4159820 0x7ff7d40cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7d919a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40aa480 0x7ff7d416dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7da19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4159820 0x7ff7d40cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50440/0 (socket says 172.21.15.131:50440) 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7da19c700 1 -- 172.21.15.131:0/1802957438 learned_addr learned my addr 172.21.15.131:0/1802957438 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.643+0000 7ff7d999b700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d406e2b0 0x7ff7d416afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7da19c700 1 -- 172.21.15.131:0/1802957438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40aa480 msgr2=0x7ff7d416dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7da19c700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40aa480 0x7ff7d416dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7da19c700 1 -- 172.21.15.131:0/1802957438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d406e2b0 msgr2=0x7ff7d416afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7da19c700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d406e2b0 0x7ff7d416afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7da19c700 1 -- 172.21.15.131:0/1802957438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c4009420 con 0x7ff7d4159820 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7d919a700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40aa480 0x7ff7d416dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7da19c700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4159820 0x7ff7d40cc0b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00cbf0 tx=0x7ff7cc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7c9ffb700 1 -- 172.21.15.131:0/1802957438 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00a8b0 con 0x7ff7d4159820 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7c9ffb700 1 -- 172.21.15.131:0/1802957438 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc00aa10 con 0x7ff7d4159820 2023-12-26T22:53:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7c9ffb700 1 -- 172.21.15.131:0/1802957438 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00add0 con 0x7ff7d4159820 2023-12-26T22:53:07.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d416e250 con 0x7ff7d4159820 2023-12-26T22:53:07.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.644+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d416e810 con 0x7ff7d4159820 2023-12-26T22:53:07.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.645+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d416ba40 con 0x7ff7d4159820 2023-12-26T22:53:07.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.647+0000 7ff7c9ffb700 1 -- 172.21.15.131:0/1802957438 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7ff7cc007300 con 0x7ff7d4159820 2023-12-26T22:53:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.648+0000 7ff7c9ffb700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff7b80670e0 0x7ff7b8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.648+0000 7ff7c9ffb700 1 -- 172.21.15.131:0/1802957438 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff7cc084e90 con 0x7ff7d4159820 2023-12-26T22:53:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.648+0000 7ff7d999b700 1 -- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff7b80670e0 msgr2=0x7ff7b8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:07.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.648+0000 7ff7d999b700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff7b80670e0 0x7ff7b8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:07.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.651+0000 7ff7c9ffb700 1 -- 172.21.15.131:0/1802957438 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7cc051b90 con 0x7ff7d4159820 2023-12-26T22:53:07.746 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:07.747+0000 7fbfb53371c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T22:53:07.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.849+0000 7ff7d999b700 1 -- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff7b80670e0 msgr2=0x7ff7b8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:07.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.849+0000 7ff7d999b700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff7b80670e0 0x7ff7b8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.893+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7d40657e0 con 0x7ff7d4159820 2023-12-26T22:53:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.894+0000 7ff7c9ffb700 1 -- 172.21.15.131:0/1802957438 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4611 (secure 0 0 0) 0x7ff7cc0551b0 con 0x7ff7d4159820 2023-12-26T22:53:07.894 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:07.894 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2085575956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2085575956},{"type":"v1","addr":"172.21.15.190:6834","nonce":2085575956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2129957224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2129957224},{"type":"v1","addr":"172.21.15.190:6835","nonce":2129957224}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3810273571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3810273571},{"type":"v1","addr":"172.21.15.131:6834","nonce":3810273571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:52:39.271885+0000","modified":"2023-12-26T22:52:42.824355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8628":{"gid":8628,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2240171052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2240171052},{"type":"v1","addr":"172.21.15.131:6835","nonce":2240171052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-12-26T22:53:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff7b80670e0 msgr2=0x7ff7b8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff7b80670e0 0x7ff7b8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4159820 msgr2=0x7ff7d40cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4159820 0x7ff7d40cc0b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00cbf0 tx=0x7ff7cc0019b0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 shutdown_connections 2023-12-26T22:53:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ff7b80670e0 0x7ff7b8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7d4159820 0x7ff7d40cc0b0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d406e2b0 0x7ff7d416afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 --2- 172.21.15.131:0/1802957438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d40aa480 0x7ff7d416dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 >> 172.21.15.131:0/1802957438 conn(0x7ff7d406c290 msgr2=0x7ff7d4158730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 shutdown_connections 2023-12-26T22:53:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:07.897+0000 7ff7dbbff700 1 -- 172.21.15.131:0/1802957438 wait complete. 2023-12-26T22:53:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 73 2023-12-26T22:53:07.916 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:53:07.933 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:07.934+0000 7fbfb53371c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T22:53:08.036 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:08.036+0000 7f3da90621c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T22:53:08.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.110+0000 7f5ecc3ff700 1 Processor -- start 2023-12-26T22:53:08.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.111+0000 7f5ecc3ff700 1 -- start start 2023-12-26T22:53:08.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.112+0000 7f5ecc3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5ec406e480 0x7f5ec40abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.112+0000 7f5ecc3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ec40aa480 0x7f5ec40675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.112+0000 7f5ecc3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec415a8c0 con 0x7f5ec4159820 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.112+0000 7f5ecc3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec415ad50 con 0x7f5ec40aa480 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.112+0000 7f5ecc3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec4159f90 con 0x7f5ec406e480 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5eca99c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ec40aa480 0x7f5ec40675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5eca99c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ec40aa480 0x7f5ec40675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50442/0 (socket says 172.21.15.131:50442) 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5eca99c700 1 -- 172.21.15.131:0/935614609 learned_addr learned my addr 172.21.15.131:0/935614609 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5ec999a700 1 --2- 172.21.15.131:0/935614609 >> v2:172.21.15.190:3300/0 conn(0x7f5ec406e480 0x7f5ec40abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5eca99c700 1 -- 172.21.15.131:0/935614609 >> v2:172.21.15.190:3300/0 conn(0x7f5ec406e480 msgr2=0x7f5ec40abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5eca99c700 1 --2- 172.21.15.131:0/935614609 >> v2:172.21.15.190:3300/0 conn(0x7f5ec406e480 0x7f5ec40abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5eca99c700 1 -- 172.21.15.131:0/935614609 >> v1:172.21.15.190:6789/0 conn(0x7f5ec4159820 legacy=0x7f5ec406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5eca99c700 1 -- 172.21.15.131:0/935614609 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec415a270 con 0x7f5ec40aa480 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5ec9199700 1 -- 172.21.15.131:0/935614609 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427007143 0 0) 0x7f5ec415a8c0 con 0x7f5ec4159820 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.113+0000 7f5eca99c700 1 --2- 172.21.15.131:0/935614609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40aa480 0x7f5ec40675b0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40af3f0 tx=0x7f5ebc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbde9a2f374b2b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ec9199700 1 -- 172.21.15.131:0/935614609 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00e070 con 0x7f5ec40aa480 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ec9199700 1 -- 172.21.15.131:0/935614609 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc00ba20 con 0x7f5ec40aa480 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ec9199700 1 -- 172.21.15.131:0/935614609 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0092e0 con 0x7f5ec40aa480 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/935614609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40aa480 msgr2=0x7f5ec40675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ecc3ff700 1 --2- 172.21.15.131:0/935614609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40aa480 0x7f5ec40675b0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40af3f0 tx=0x7f5ebc009840 comp rx=0 tx=0).stop 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/935614609 shutdown_connections 2023-12-26T22:53:08.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ecc3ff700 1 --2- 172.21.15.131:0/935614609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40aa480 0x7f5ec40675b0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ecc3ff700 1 --2- 172.21.15.131:0/935614609 >> v2:172.21.15.190:3300/0 conn(0x7f5ec406e480 0x7f5ec40abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/935614609 >> 172.21.15.131:0/935614609 conn(0x7f5ec406c290 msgr2=0x7f5ec40ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:08.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.114+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/935614609 shutdown_connections 2023-12-26T22:53:08.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.115+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/935614609 wait complete. 2023-12-26T22:53:08.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.116+0000 7f5ecc3ff700 1 Processor -- start 2023-12-26T22:53:08.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.116+0000 7f5ecc3ff700 1 -- start start 2023-12-26T22:53:08.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5ecc3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec406e480 0x7f5ec416be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5ecc3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ec40aa480 0x7f5ec4147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5ecc3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec4159820 0x7f5ec4147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5ecc3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec4148390 con 0x7f5ec4159820 2023-12-26T22:53:08.120 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5ecc3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40b1870 con 0x7f5ec406e480 2023-12-26T22:53:08.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5ecc3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40b1b50 con 0x7f5ec40aa480 2023-12-26T22:53:08.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5eca99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec4159820 0x7f5ec4147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5eca99c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec4159820 0x7f5ec4147950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50450/0 (socket says 172.21.15.131:50450) 2023-12-26T22:53:08.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5eca99c700 1 -- 172.21.15.131:0/1085231996 learned_addr learned my addr 172.21.15.131:0/1085231996 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:08.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5ec999a700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ec40aa480 0x7f5ec4147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.117+0000 7f5eca19b700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec406e480 0x7f5ec416be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5eca99c700 1 -- 172.21.15.131:0/1085231996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ec40aa480 msgr2=0x7f5ec4147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5eca99c700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ec40aa480 0x7f5ec4147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5eca99c700 1 -- 172.21.15.131:0/1085231996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec406e480 msgr2=0x7f5ec416be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5eca99c700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec406e480 0x7f5ec416be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5eca99c700 1 -- 172.21.15.131:0/1085231996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc009ae0 con 0x7f5ec4159820 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5eca99c700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec4159820 0x7f5ec4147950 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009880 tx=0x7f5ebc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5eba7fc700 1 -- 172.21.15.131:0/1085231996 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc008a80 con 0x7f5ec4159820 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5eba7fc700 1 -- 172.21.15.131:0/1085231996 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc008c00 con 0x7f5ec4159820 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5eba7fc700 1 -- 172.21.15.131:0/1085231996 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc026460 con 0x7f5ec4159820 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec40b1e30 con 0x7f5ec4159820 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.118+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec4158730 con 0x7f5ec4159820 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.119+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec40657e0 con 0x7f5ec4159820 2023-12-26T22:53:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.120+0000 7f5eba7fc700 1 -- 172.21.15.131:0/1085231996 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f5ebc003040 con 0x7f5ec4159820 2023-12-26T22:53:08.123 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:08.123+0000 7fbfb53371c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T22:53:08.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.121+0000 7f5eba7fc700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5ea80670e0 0x7f5ea8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.121+0000 7f5eba7fc700 1 -- 172.21.15.131:0/1085231996 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5ebc086a70 con 0x7f5ec4159820 2023-12-26T22:53:08.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.121+0000 7f5eca19b700 1 -- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5ea80670e0 msgr2=0x7f5ea8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:08.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.121+0000 7f5eca19b700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5ea80670e0 0x7f5ea8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:08.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.125+0000 7f5eba7fc700 1 -- 172.21.15.131:0/1085231996 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ebc053680 con 0x7f5ec4159820 2023-12-26T22:53:08.236 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:08.236+0000 7f3da90621c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T22:53:08.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.322+0000 7f5eca19b700 1 -- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5ea80670e0 msgr2=0x7f5ea8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:08.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.322+0000 7f5eca19b700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5ea80670e0 0x7f5ea8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.326+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ec415a550 con 0x7f5ec4159820 2023-12-26T22:53:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.327+0000 7f5eba7fc700 1 -- 172.21.15.131:0/1085231996 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16497 (secure 0 0 0) 0x7f5ebc021090 con 0x7f5ec4159820 2023-12-26T22:53:08.327 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:08.328 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":162,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:53:00.352922+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:52:37.440528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-26T22:52:38.259993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3597309391":"2023-12-27T22:53:00.352891+0000","172.21.15.131:0/3207268993":"2023-12-27T22:53:00.352891+0000","172.21.15.131:6837/39288":"2023-12-27T22:53:00.352891+0000","172.21.15.131:6836/39288":"2023-12-27T22:53:00.352891+0000","172.21.15.131:0/1148444260":"2023-12-27T22:53:00.352891+0000","172.21.15.131:0/2850349354":"2023-12-27T22:53:00.352891+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:53:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.329+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5ea80670e0 msgr2=0x7f5ea8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.329+0000 7f5ecc3ff700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5ea80670e0 0x7f5ea8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec4159820 msgr2=0x7f5ec4147950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec4159820 0x7f5ec4147950 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009880 tx=0x7f5ebc018000 comp rx=0 tx=0).stop 2023-12-26T22:53:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 shutdown_connections 2023-12-26T22:53:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f5ea80670e0 0x7f5ea8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec4159820 0x7f5ec4147950 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ec406e480 0x7f5ec416be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 --2- 172.21.15.131:0/1085231996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ec40aa480 0x7f5ec4147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 >> 172.21.15.131:0/1085231996 conn(0x7f5ec406c290 msgr2=0x7f5ec4069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 shutdown_connections 2023-12-26T22:53:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.330+0000 7f5ecc3ff700 1 -- 172.21.15.131:0/1085231996 wait complete. 2023-12-26T22:53:08.348 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T22:53:08.442 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:08.443+0000 7f3da90621c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T22:53:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.548+0000 7fcbb95ff700 1 Processor -- start 2023-12-26T22:53:08.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.548+0000 7fcbb95ff700 1 -- start start 2023-12-26T22:53:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.549+0000 7fcbb95ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcbb4156290 0x7fcbb40c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.549+0000 7fcbb95ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb4156660 0x7fcbb40af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.549+0000 7fcbb95ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb4067640 con 0x7fcbb40b0fe0 2023-12-26T22:53:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.549+0000 7fcbb95ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb40b0240 con 0x7fcbb4156290 2023-12-26T22:53:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.549+0000 7fcbb95ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40b0520 con 0x7fcbb4156660 2023-12-26T22:53:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb37fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcbb40b0fe0 0x7fcbb415a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37230/0 (socket says 172.21.15.131:37230) 2023-12-26T22:53:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb37fe700 1 -- 172.21.15.131:0/1700641917 learned_addr learned my addr 172.21.15.131:0/1700641917 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb27fc700 1 --2- 172.21.15.131:0/1700641917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb4156660 0x7fcbb40af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb2ffd700 1 --2- 172.21.15.131:0/1700641917 >> v2:172.21.15.190:3300/0 conn(0x7fcbb4156290 0x7fcbb40c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb27fc700 1 -- 172.21.15.131:0/1700641917 >> v2:172.21.15.190:3300/0 conn(0x7fcbb4156290 msgr2=0x7fcbb40c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb27fc700 1 --2- 172.21.15.131:0/1700641917 >> v2:172.21.15.190:3300/0 conn(0x7fcbb4156290 0x7fcbb40c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb1ffb700 1 -- 172.21.15.131:0/1700641917 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425173454 0 0) 0x7fcbb4067640 con 0x7fcbb40b0fe0 2023-12-26T22:53:08.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb27fc700 1 -- 172.21.15.131:0/1700641917 >> v1:172.21.15.131:6789/0 conn(0x7fcbb40b0fe0 legacy=0x7fcbb415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb27fc700 1 -- 172.21.15.131:0/1700641917 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40b0800 con 0x7fcbb4156660 2023-12-26T22:53:08.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.550+0000 7fcbb27fc700 1 --2- 172.21.15.131:0/1700641917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb4156660 0x7fcbb40af940 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcba80098e0 tx=0x7fcba80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2afd92a02f7f835 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:08.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb1ffb700 1 -- 172.21.15.131:0/1700641917 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800b640 con 0x7fcbb4156660 2023-12-26T22:53:08.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb1ffb700 1 -- 172.21.15.131:0/1700641917 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba800b7a0 con 0x7fcbb4156660 2023-12-26T22:53:08.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb1ffb700 1 -- 172.21.15.131:0/1700641917 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800bb40 con 0x7fcbb4156660 2023-12-26T22:53:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb95ff700 1 -- 172.21.15.131:0/1700641917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb4156660 msgr2=0x7fcbb40af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb95ff700 1 --2- 172.21.15.131:0/1700641917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb4156660 0x7fcbb40af940 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcba80098e0 tx=0x7fcba80091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb95ff700 1 -- 172.21.15.131:0/1700641917 shutdown_connections 2023-12-26T22:53:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb95ff700 1 --2- 172.21.15.131:0/1700641917 >> v2:172.21.15.190:3300/0 conn(0x7fcbb4156290 0x7fcbb40c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb95ff700 1 --2- 172.21.15.131:0/1700641917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb4156660 0x7fcbb40af940 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb95ff700 1 -- 172.21.15.131:0/1700641917 >> 172.21.15.131:0/1700641917 conn(0x7fcbb406c290 msgr2=0x7fcbb40ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.551+0000 7fcbb95ff700 1 -- 172.21.15.131:0/1700641917 shutdown_connections 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.552+0000 7fcbb95ff700 1 -- 172.21.15.131:0/1700641917 wait complete. 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.553+0000 7fcbb95ff700 1 Processor -- start 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.553+0000 7fcbb95ff700 1 -- start start 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb95ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb40b0fe0 0x7fcbb4152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb95ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbb4156290 0x7fcbb41574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb95ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb4156660 0x7fcbb41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb95ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb4153740 con 0x7fcbb4156660 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb95ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb416a3c0 con 0x7fcbb4156290 2023-12-26T22:53:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb95ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb415d080 con 0x7fcbb40b0fe0 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb4156660 0x7fcbb41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb27fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbb4156290 0x7fcbb41574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb4156660 0x7fcbb41579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50466/0 (socket says 172.21.15.131:50466) 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb37fe700 1 -- 172.21.15.131:0/3887578419 learned_addr learned my addr 172.21.15.131:0/3887578419 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.554+0000 7fcbb2ffd700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb40b0fe0 0x7fcbb4152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb27fc700 1 -- 172.21.15.131:0/3887578419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb40b0fe0 msgr2=0x7fcbb4152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb27fc700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb40b0fe0 0x7fcbb4152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb27fc700 1 -- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb4156660 msgr2=0x7fcbb41579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb27fc700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb4156660 0x7fcbb41579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb27fc700 1 -- 172.21.15.131:0/3887578419 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba8009420 con 0x7fcbb4156290 2023-12-26T22:53:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb37fe700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb4156660 0x7fcbb41579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb2ffd700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb40b0fe0 0x7fcbb4152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb27fc700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbb4156290 0x7fcbb41574a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcba8016000 tx=0x7fcba8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcb9affd700 1 -- 172.21.15.131:0/3887578419 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800bb20 con 0x7fcbb4156290 2023-12-26T22:53:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcb9affd700 1 -- 172.21.15.131:0/3887578419 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba8000c90 con 0x7fcbb4156290 2023-12-26T22:53:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcb9affd700 1 -- 172.21.15.131:0/3887578419 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80246a0 con 0x7fcbb4156290 2023-12-26T22:53:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb415d360 con 0x7fcbb4156290 2023-12-26T22:53:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.555+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb416a6a0 con 0x7fcbb4156290 2023-12-26T22:53:08.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.557+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb415d1c0 con 0x7fcbb4156290 2023-12-26T22:53:08.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.557+0000 7fcb9affd700 1 -- 172.21.15.131:0/3887578419 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fcba8024a90 con 0x7fcbb4156290 2023-12-26T22:53:08.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.558+0000 7fcb9affd700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcba00670e0 0x7fcba0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:08.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.558+0000 7fcb9affd700 1 -- 172.21.15.131:0/3887578419 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fcba8086200 con 0x7fcbb4156290 2023-12-26T22:53:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.558+0000 7fcbb2ffd700 1 -- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcba00670e0 msgr2=0x7fcba0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.558+0000 7fcbb2ffd700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcba00670e0 0x7fcba0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:08.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.562+0000 7fcb9affd700 1 -- 172.21.15.131:0/3887578419 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbb415d1c0 con 0x7fcbb4156290 2023-12-26T22:53:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.759+0000 7fcbb2ffd700 1 -- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcba00670e0 msgr2=0x7fcba0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.759+0000 7fcbb2ffd700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcba00670e0 0x7fcba0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:08.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.801+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcbb415d1c0 con 0x7fcbb4156290 2023-12-26T22:53:08.802 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T22:53:08.802+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T22:53:08.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.808+0000 7fcb9affd700 1 -- 172.21.15.131:0/3887578419 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcbb415d1c0 con 0x7fcbb4156290 2023-12-26T22:53:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcba00670e0 msgr2=0x7fcba0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcba00670e0 0x7fcba0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbb4156290 msgr2=0x7fcbb41574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbb4156290 0x7fcbb41574a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcba8016000 tx=0x7fcba8017000 comp rx=0 tx=0).stop 2023-12-26T22:53:08.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 shutdown_connections 2023-12-26T22:53:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fcba00670e0 0x7fcba0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb4156660 0x7fcbb41579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbb40b0fe0 0x7fcbb4152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 --2- 172.21.15.131:0/3887578419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbb4156290 0x7fcbb41574a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 >> 172.21.15.131:0/3887578419 conn(0x7fcbb406c290 msgr2=0x7fcbb4155470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 shutdown_connections 2023-12-26T22:53:08.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:08.811+0000 7fcbb95ff700 1 -- 172.21.15.131:0/3887578419 wait complete. 2023-12-26T22:53:08.813 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T22:53:08.813 INFO:tasks.ceph.mds.a.smithi131.stderr: -2291> 2023-12-26T22:52:44.779+0000 7fefcd3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T22:53:08.831 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T22:53:08.858 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T22:53:08.970 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:08.971+0000 7fbfb53371c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T22:53:09.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.036+0000 7fbc8ebff700 1 Processor -- start 2023-12-26T22:53:09.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.036+0000 7fbc8ebff700 1 -- start start 2023-12-26T22:53:09.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc8ebff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbc881573f0 0x7fbc880af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:09.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc8ebff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbc881577c0 0x7fbc8815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:09.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc8ebff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 0x7fbc880aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:09.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc8ebff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc880ca830 con 0x7fbc881577c0 2023-12-26T22:53:09.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc8ebff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc880af1b0 con 0x7fbc881573f0 2023-12-26T22:53:09.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc8ebff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88067870 con 0x7fbc8815b700 2023-12-26T22:53:09.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc87fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbc881577c0 0x7fbc8815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:09.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc87fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbc881577c0 0x7fbc8815b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50490/0 (socket says 172.21.15.131:50490) 2023-12-26T22:53:09.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc87fff700 1 -- 172.21.15.131:0/1134493784 learned_addr learned my addr 172.21.15.131:0/1134493784 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:09.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc8c99b700 1 --2- 172.21.15.131:0/1134493784 >> v2:172.21.15.190:3300/0 conn(0x7fbc881573f0 0x7fbc880af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:09.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.037+0000 7fbc8d19c700 1 --2- 172.21.15.131:0/1134493784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 0x7fbc880aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:09.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.038+0000 7fbc87fff700 1 -- 172.21.15.131:0/1134493784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 msgr2=0x7fbc880aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:09.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.038+0000 7fbc87fff700 1 --2- 172.21.15.131:0/1134493784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 0x7fbc880aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.038+0000 7fbc87fff700 1 -- 172.21.15.131:0/1134493784 >> v2:172.21.15.190:3300/0 conn(0x7fbc881573f0 msgr2=0x7fbc880af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:09.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.038+0000 7fbc87fff700 1 --2- 172.21.15.131:0/1134493784 >> v2:172.21.15.190:3300/0 conn(0x7fbc881573f0 0x7fbc880af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.038+0000 7fbc87fff700 1 -- 172.21.15.131:0/1134493784 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880aebd0 con 0x7fbc881577c0 2023-12-26T22:53:09.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.038+0000 7fbc8c99b700 1 --2- 172.21.15.131:0/1134493784 >> v2:172.21.15.190:3300/0 conn(0x7fbc881573f0 0x7fbc880af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:09.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.038+0000 7fbc87fff700 1 --2- 172.21.15.131:0/1134493784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 0x7fbc8815b1c0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fbc8806de30 tx=0x7fbc74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0910b8a82ffd83 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:09.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.038+0000 7fbc877fe700 1 -- 172.21.15.131:0/1134493784 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400b640 con 0x7fbc881577c0 2023-12-26T22:53:09.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc877fe700 1 -- 172.21.15.131:0/1134493784 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7400b7a0 con 0x7fbc881577c0 2023-12-26T22:53:09.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc877fe700 1 -- 172.21.15.131:0/1134493784 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400bb60 con 0x7fbc881577c0 2023-12-26T22:53:09.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc8ebff700 1 -- 172.21.15.131:0/1134493784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 msgr2=0x7fbc8815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:09.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/1134493784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 0x7fbc8815b1c0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fbc8806de30 tx=0x7fbc74009250 comp rx=0 tx=0).stop 2023-12-26T22:53:09.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc8ebff700 1 -- 172.21.15.131:0/1134493784 shutdown_connections 2023-12-26T22:53:09.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/1134493784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 0x7fbc880aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/1134493784 >> v2:172.21.15.190:3300/0 conn(0x7fbc881573f0 0x7fbc880af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/1134493784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 0x7fbc8815b1c0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc8ebff700 1 -- 172.21.15.131:0/1134493784 >> 172.21.15.131:0/1134493784 conn(0x7fbc880686c0 msgr2=0x7fbc88068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc8ebff700 1 -- 172.21.15.131:0/1134493784 shutdown_connections 2023-12-26T22:53:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.039+0000 7fbc8ebff700 1 -- 172.21.15.131:0/1134493784 wait complete. 2023-12-26T22:53:09.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.041+0000 7fbc8ebff700 1 Processor -- start 2023-12-26T22:53:09.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.041+0000 7fbc8ebff700 1 -- start start 2023-12-26T22:53:09.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc8ebff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc881573f0 0x7fbc8814e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:09.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc8ebff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 0x7fbc880b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:09.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc8ebff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 0x7fbc880cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc8ebff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88158380 con 0x7fbc881577c0 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc8ebff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880b2c40 con 0x7fbc881573f0 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc8ebff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880b2f20 con 0x7fbc8815b700 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc87fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 0x7fbc880b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc8c99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc881573f0 0x7fbc8814e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc87fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 0x7fbc880b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50498/0 (socket says 172.21.15.131:50498) 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc8c99b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc881573f0 0x7fbc8814e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:35850/0 (socket says 172.21.15.131:35850) 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc87fff700 1 -- 172.21.15.131:0/3551831657 learned_addr learned my addr 172.21.15.131:0/3551831657 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.042+0000 7fbc8d19c700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 0x7fbc880cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:09.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc8c99b700 1 -- 172.21.15.131:0/3551831657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 msgr2=0x7fbc880cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc8c99b700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 0x7fbc880cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc8c99b700 1 -- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 msgr2=0x7fbc880b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc8c99b700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 0x7fbc880b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc8c99b700 1 -- 172.21.15.131:0/3551831657 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c009420 con 0x7fbc881573f0 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc8c99b700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc881573f0 0x7fbc8814e710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c010db0 tx=0x7fbc7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc84ff9700 1 -- 172.21.15.131:0/3551831657 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c015070 con 0x7fbc881573f0 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc84ff9700 1 -- 172.21.15.131:0/3551831657 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c00bb20 con 0x7fbc881573f0 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc740094f0 con 0x7fbc881573f0 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc84ff9700 1 -- 172.21.15.131:0/3551831657 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c019410 con 0x7fbc881573f0 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.043+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc88158010 con 0x7fbc881573f0 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.045+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc880657e0 con 0x7fbc881573f0 2023-12-26T22:53:09.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.045+0000 7fbc84ff9700 1 -- 172.21.15.131:0/3551831657 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 77) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fbc7c019800 con 0x7fbc881573f0 2023-12-26T22:53:09.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.046+0000 7fbc84ff9700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 0x7fbc6c0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:09.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.046+0000 7fbc87fff700 1 -- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 msgr2=0x7fbc6c0695c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:09.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.046+0000 7fbc84ff9700 1 -- 172.21.15.131:0/3551831657 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fbc7c088a20 con 0x7fbc881573f0 2023-12-26T22:53:09.049 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.046+0000 7fbc87fff700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 0x7fbc6c0695c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:09.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.050+0000 7fbc84ff9700 1 -- 172.21.15.131:0/3551831657 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc7c0556c0 con 0x7fbc881573f0 2023-12-26T22:53:09.246 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:09.247+0000 7fbfb53371c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:09.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.247+0000 7fbc87fff700 1 -- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 msgr2=0x7fbc6c0695c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:09.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.247+0000 7fbc87fff700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 0x7fbc6c0695c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:09.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.296+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbc8814ef90 con 0x7fbc881573f0 2023-12-26T22:53:09.319 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:09.320+0000 7f3da90621c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T22:53:09.339 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:09.340+0000 7fbfb53371c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:09.534 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:09.535+0000 7fbfb53371c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T22:53:09.598 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:09.598+0000 7f3da90621c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:09.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.648+0000 7fbc87fff700 1 -- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 msgr2=0x7fbc6c0695c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:09.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.648+0000 7fbc87fff700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 0x7fbc6c0695c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T22:53:09.669 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:53:09.675 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:53:09.695 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:09.695+0000 7f3da90621c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:09.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.808+0000 7fbc84ff9700 1 -- 172.21.15.131:0/3551831657 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbc7c058ce0 con 0x7fbc881573f0 2023-12-26T22:53:09.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 msgr2=0x7fbc6c0695c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 0x7fbc6c0695c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc881573f0 msgr2=0x7fbc8814e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc881573f0 0x7fbc8814e710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c010db0 tx=0x7fbc7c0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 shutdown_connections 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc8815b700 0x7fbc880cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc881573f0 0x7fbc8814e710 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fbc6c067110 0x7fbc6c0695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 --2- 172.21.15.131:0/3551831657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc881577c0 0x7fbc880b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 >> 172.21.15.131:0/3551831657 conn(0x7fbc880686c0 msgr2=0x7fbc88155560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 shutdown_connections 2023-12-26T22:53:09.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:09.811+0000 7fbc8ebff700 1 -- 172.21.15.131:0/3551831657 wait complete. 2023-12-26T22:53:09.830 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T22:53:09.906 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:09.907+0000 7f3da90621c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T22:53:10.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.086+0000 7f071e7ff700 1 Processor -- start 2023-12-26T22:53:10.090 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.086+0000 7f071e7ff700 1 -- start start 2023-12-26T22:53:10.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f071e7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f07180adcb0 0x7f07180abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:10.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f071e7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07180ae080 0x7f0718069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:10.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f071e7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0718069a60 0x7f071806df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:10.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f071e7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07180c7430 con 0x7f07180adcb0 2023-12-26T22:53:10.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f071e7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07180ac460 con 0x7f0718069a60 2023-12-26T22:53:10.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f071e7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0718069ee0 con 0x7f07180ae080 2023-12-26T22:53:10.091 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f0717fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f07180adcb0 0x7f07180abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f071cd9c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0718069a60 0x7f071806df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f071cd9c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0718069a60 0x7f071806df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:35940/0 (socket says 172.21.15.131:35940) 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f0717fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f07180adcb0 0x7f07180abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50596/0 (socket says 172.21.15.131:50596) 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f071cd9c700 1 -- 172.21.15.131:0/2401912410 learned_addr learned my addr 172.21.15.131:0/2401912410 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.087+0000 7f07177fe700 1 --2- 172.21.15.131:0/2401912410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07180ae080 0x7f0718069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.088+0000 7f071cd9c700 1 -- 172.21.15.131:0/2401912410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07180ae080 msgr2=0x7f0718069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.088+0000 7f071cd9c700 1 --2- 172.21.15.131:0/2401912410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07180ae080 0x7f0718069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.088+0000 7f071cd9c700 1 -- 172.21.15.131:0/2401912410 >> v2:172.21.15.131:3300/0 conn(0x7f07180adcb0 msgr2=0x7f07180abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.088+0000 7f071cd9c700 1 --2- 172.21.15.131:0/2401912410 >> v2:172.21.15.131:3300/0 conn(0x7f07180adcb0 0x7f07180abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:10.092 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.088+0000 7f071cd9c700 1 -- 172.21.15.131:0/2401912410 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07180c6820 con 0x7f0718069a60 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.088+0000 7f071cd9c700 1 --2- 172.21.15.131:0/2401912410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0718069a60 0x7f071806df30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f07180c9c90 tx=0x7f0710009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dac168ee34b7b7d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.088+0000 7f0716ffd700 1 -- 172.21.15.131:0/2401912410 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07100092e0 con 0x7f0718069a60 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.088+0000 7f0716ffd700 1 -- 172.21.15.131:0/2401912410 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071000db80 con 0x7f0718069a60 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.088+0000 7f0716ffd700 1 -- 172.21.15.131:0/2401912410 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710011460 con 0x7f0718069a60 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.089+0000 7f071e7ff700 1 -- 172.21.15.131:0/2401912410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0718069a60 msgr2=0x7f071806df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.089+0000 7f071e7ff700 1 --2- 172.21.15.131:0/2401912410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0718069a60 0x7f071806df30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f07180c9c90 tx=0x7f0710009840 comp rx=0 tx=0).stop 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.089+0000 7f071e7ff700 1 -- 172.21.15.131:0/2401912410 shutdown_connections 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.089+0000 7f071e7ff700 1 --2- 172.21.15.131:0/2401912410 >> v2:172.21.15.131:3300/0 conn(0x7f07180adcb0 0x7f07180abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.089+0000 7f071e7ff700 1 --2- 172.21.15.131:0/2401912410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0718069a60 0x7f071806df30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:10.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.089+0000 7f071e7ff700 1 --2- 172.21.15.131:0/2401912410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07180ae080 0x7f0718069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:10.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.089+0000 7f071e7ff700 1 -- 172.21.15.131:0/2401912410 >> 172.21.15.131:0/2401912410 conn(0x7f07180675b0 msgr2=0x7f07180679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:10.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.089+0000 7f071e7ff700 1 -- 172.21.15.131:0/2401912410 shutdown_connections 2023-12-26T22:53:10.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.089+0000 7f071e7ff700 1 -- 172.21.15.131:0/2401912410 wait complete. 2023-12-26T22:53:10.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.090+0000 7f071e7ff700 1 Processor -- start 2023-12-26T22:53:10.094 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.091+0000 7f071e7ff700 1 -- start start 2023-12-26T22:53:10.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.091+0000 7f071e7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0718069a60 0x7f07180c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:10.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.091+0000 7f071e7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07180adcb0 0x7f07180c1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:10.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f071e7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07180ae080 0x7f0718169370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:10.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f071e7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0718169ec0 con 0x7f07180adcb0 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f071e7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071816a1a0 con 0x7f07180ae080 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f071e7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07180c5220 con 0x7f0718069a60 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f07177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07180adcb0 0x7f07180c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f07177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07180adcb0 0x7f07180c1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50608/0 (socket says 172.21.15.131:50608) 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f07177fe700 1 -- 172.21.15.131:0/232398354 learned_addr learned my addr 172.21.15.131:0/232398354 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f0717fff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0718069a60 0x7f07180c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f071cd9c700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07180ae080 0x7f0718169370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f07177fe700 1 -- 172.21.15.131:0/232398354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0718069a60 msgr2=0x7f07180c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f07177fe700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0718069a60 0x7f07180c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:10.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f07177fe700 1 -- 172.21.15.131:0/232398354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07180ae080 msgr2=0x7f0718169370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f07177fe700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07180ae080 0x7f0718169370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f07177fe700 1 -- 172.21.15.131:0/232398354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0710009ae0 con 0x7f07180adcb0 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f071cd9c700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07180ae080 0x7f0718169370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.092+0000 7f07177fe700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07180adcb0 0x7f07180c1680 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f070c00ea70 tx=0x7f070c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.093+0000 7f06fffff700 1 -- 172.21.15.131:0/232398354 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00c920 con 0x7f07180adcb0 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.093+0000 7f06fffff700 1 -- 172.21.15.131:0/232398354 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c00caa0 con 0x7f07180adcb0 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.093+0000 7f06fffff700 1 -- 172.21.15.131:0/232398354 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00ce60 con 0x7f07180adcb0 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.093+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07180c5500 con 0x7f07180adcb0 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.093+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071816cc40 con 0x7f07180adcb0 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.094+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071816a3e0 con 0x7f07180adcb0 2023-12-26T22:53:10.099 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.095+0000 7f06fffff700 1 -- 172.21.15.131:0/232398354 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 78) v1 ==== 84453+0+0 (secure 0 0 0) 0x7f070c013680 con 0x7f07180adcb0 2023-12-26T22:53:10.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.096+0000 7f06fffff700 1 -- 172.21.15.131:0/232398354 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f070c084850 con 0x7f07180adcb0 2023-12-26T22:53:10.100 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.100+0000 7f06fffff700 1 -- 172.21.15.131:0/232398354 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f071816a3e0 con 0x7f07180adcb0 2023-12-26T22:53:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.296+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f071816a3e0 con 0x7f07180adcb0 2023-12-26T22:53:10.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.921+0000 7f06fffff700 1 -- 172.21.15.131:0/232398354 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 79) v1 ==== 84580+0+0 (secure 0 0 0) 0x7f070c0514e0 con 0x7f07180adcb0 2023-12-26T22:53:10.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.921+0000 7f06fffff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f070406cad0 0x7f070406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:10.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.922+0000 7f0717fff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f070406cad0 0x7f070406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:10.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:10.923+0000 7f0717fff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f070406cad0 0x7f070406eeb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0708009d40 tx=0x7f0708008040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:11.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.125+0000 7f06fffff700 1 -- 172.21.15.131:0/232398354 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v165) v1 ==== 181+0+0 (secure 0 0 0) 0x7f070c07d420 con 0x7f07180adcb0 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f070406cad0 msgr2=0x7f070406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f070406cad0 0x7f070406eeb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0708009d40 tx=0x7f0708008040 comp rx=0 tx=0).stop 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07180adcb0 msgr2=0x7f07180c1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07180adcb0 0x7f07180c1680 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f070c00ea70 tx=0x7f070c0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 shutdown_connections 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07180ae080 0x7f0718169370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0718069a60 0x7f07180c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f070406cad0 0x7f070406eeb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 --2- 172.21.15.131:0/232398354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07180adcb0 0x7f07180c1680 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 >> 172.21.15.131:0/232398354 conn(0x7f07180675b0 msgr2=0x7f07180cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:11.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 shutdown_connections 2023-12-26T22:53:11.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.127+0000 7f071e7ff700 1 -- 172.21.15.131:0/232398354 wait complete. 2023-12-26T22:53:11.128 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T22:53:11.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T22:53:11.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.382+0000 7f39d45ff700 1 Processor -- start 2023-12-26T22:53:11.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.382+0000 7f39d45ff700 1 -- start start 2023-12-26T22:53:11.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.383+0000 7f39d45ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f39cc0c9d00 0x7f39cc0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:11.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.383+0000 7f39d45ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc06eef0 0x7f39cc06f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:11.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.383+0000 7f39d45ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39cc06ea60 con 0x7f39cc13fca0 2023-12-26T22:53:11.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.383+0000 7f39d45ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39cc06e490 con 0x7f39cc0c9d00 2023-12-26T22:53:11.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.383+0000 7f39d45ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc140070 con 0x7f39cc06eef0 2023-12-26T22:53:11.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d1b9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f39cc0c9d00 0x7f39cc0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:11.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d2b9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc06eef0 0x7f39cc06f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:11.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d2b9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc06eef0 0x7f39cc06f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:45430/0 (socket says 172.21.15.131:45430) 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d2b9c700 1 -- 172.21.15.131:0/4196399801 learned_addr learned my addr 172.21.15.131:0/4196399801 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d1b9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f39cc0c9d00 0x7f39cc0ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:35946/0 (socket says 172.21.15.131:35946) 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d239b700 1 --1- 172.21.15.131:0/4196399801 >> v1:172.21.15.190:6789/0 conn(0x7f39cc13fca0 0x7f39cc159f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:56170/0 (socket says 172.21.15.131:56170) 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d2b9c700 1 -- 172.21.15.131:0/4196399801 >> v2:172.21.15.190:3300/0 conn(0x7f39cc0c9d00 msgr2=0x7f39cc0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d2b9c700 1 --2- 172.21.15.131:0/4196399801 >> v2:172.21.15.190:3300/0 conn(0x7f39cc0c9d00 0x7f39cc0ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d2b9c700 1 -- 172.21.15.131:0/4196399801 >> v1:172.21.15.190:6789/0 conn(0x7f39cc13fca0 legacy=0x7f39cc159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d2b9c700 1 -- 172.21.15.131:0/4196399801 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc15a680 con 0x7f39cc06eef0 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.384+0000 7f39d1b9a700 1 --2- 172.21.15.131:0/4196399801 >> v2:172.21.15.190:3300/0 conn(0x7f39cc0c9d00 0x7f39cc0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.385+0000 7f39d2b9c700 1 --2- 172.21.15.131:0/4196399801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc06eef0 0x7f39cc06f330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39c400d1a0 tx=0x7f39c400ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899695cd54318cee server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:11.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.385+0000 7f39d1399700 1 -- 172.21.15.131:0/4196399801 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4010070 con 0x7f39cc06eef0 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.385+0000 7f39d1399700 1 -- 172.21.15.131:0/4196399801 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c400de10 con 0x7f39cc06eef0 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.385+0000 7f39d1399700 1 -- 172.21.15.131:0/4196399801 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c400b320 con 0x7f39cc06eef0 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.386+0000 7f39d45ff700 1 -- 172.21.15.131:0/4196399801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc06eef0 msgr2=0x7f39cc06f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.386+0000 7f39d45ff700 1 --2- 172.21.15.131:0/4196399801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc06eef0 0x7f39cc06f330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39c400d1a0 tx=0x7f39c400ba50 comp rx=0 tx=0).stop 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.386+0000 7f39d45ff700 1 -- 172.21.15.131:0/4196399801 shutdown_connections 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.386+0000 7f39d45ff700 1 --2- 172.21.15.131:0/4196399801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc06eef0 0x7f39cc06f330 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.386+0000 7f39d45ff700 1 --2- 172.21.15.131:0/4196399801 >> v2:172.21.15.190:3300/0 conn(0x7f39cc0c9d00 0x7f39cc0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.386+0000 7f39d45ff700 1 -- 172.21.15.131:0/4196399801 >> 172.21.15.131:0/4196399801 conn(0x7f39cc068020 msgr2=0x7f39cc0c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.386+0000 7f39d45ff700 1 -- 172.21.15.131:0/4196399801 shutdown_connections 2023-12-26T22:53:11.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.386+0000 7f39d45ff700 1 -- 172.21.15.131:0/4196399801 wait complete. 2023-12-26T22:53:11.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.387+0000 7f39d45ff700 1 Processor -- start 2023-12-26T22:53:11.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.387+0000 7f39d45ff700 1 -- start start 2023-12-26T22:53:11.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.388+0000 7f39d45ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39cc06eef0 0x7f39cc0af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:11.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.388+0000 7f39d45ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc0c9d00 0x7f39cc13e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:11.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.388+0000 7f39d45ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc13fca0 0x7f39cc0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:11.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.388+0000 7f39d45ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0af870 con 0x7f39cc06eef0 2023-12-26T22:53:11.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.388+0000 7f39d45ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0afb50 con 0x7f39cc0c9d00 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.388+0000 7f39d45ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0afe30 con 0x7f39cc13fca0 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.388+0000 7f39d239b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39cc06eef0 0x7f39cc0af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.388+0000 7f39d239b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39cc06eef0 0x7f39cc0af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50614/0 (socket says 172.21.15.131:50614) 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.388+0000 7f39d239b700 1 -- 172.21.15.131:0/3064378330 learned_addr learned my addr 172.21.15.131:0/3064378330 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d2b9c700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc13fca0 0x7f39cc0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d1b9a700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc0c9d00 0x7f39cc13e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d239b700 1 -- 172.21.15.131:0/3064378330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc13fca0 msgr2=0x7f39cc0ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d239b700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc13fca0 0x7f39cc0ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d239b700 1 -- 172.21.15.131:0/3064378330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc0c9d00 msgr2=0x7f39cc13e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:11.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d239b700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc0c9d00 0x7f39cc13e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d239b700 1 -- 172.21.15.131:0/3064378330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc009420 con 0x7f39cc06eef0 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d239b700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39cc06eef0 0x7f39cc0af1e0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c7c0 tx=0x7f39c8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39c27fc700 1 -- 172.21.15.131:0/3064378330 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8005a70 con 0x7f39cc06eef0 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39c27fc700 1 -- 172.21.15.131:0/3064378330 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c8005bd0 con 0x7f39cc06eef0 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39c27fc700 1 -- 172.21.15.131:0/3064378330 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c80070d0 con 0x7f39cc06eef0 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c400bcf0 con 0x7f39cc06eef0 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.389+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39cc0b03f0 con 0x7f39cc06eef0 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.390+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39cc065850 con 0x7f39cc06eef0 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.391+0000 7f39c27fc700 1 -- 172.21.15.131:0/3064378330 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 79) v1 ==== 84580+0+0 (secure 0 0 0) 0x7f39c8007480 con 0x7f39cc06eef0 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.392+0000 7f39c27fc700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f39b0066d80 0x7f39b0069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.392+0000 7f39c27fc700 1 -- 172.21.15.131:0/3064378330 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6376+0+0 (secure 0 0 0) 0x7f39c8083890 con 0x7f39cc06eef0 2023-12-26T22:53:11.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.393+0000 7f39d1b9a700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f39b0066d80 0x7f39b0069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:11.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.393+0000 7f39d1b9a700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f39b0066d80 0x7f39b0069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0ab690 tx=0x7f39bc0091a0 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:11.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.397+0000 7f39c27fc700 1 -- 172.21.15.131:0/3064378330 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39c8050b10 con 0x7f39cc06eef0 2023-12-26T22:53:11.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:11.589+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f39cc0b0600 con 0x7f39cc06eef0 2023-12-26T22:53:12.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.129+0000 7f39c27fc700 1 -- 172.21.15.131:0/3064378330 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f39c8050ee0 con 0x7f39cc06eef0 2023-12-26T22:53:12.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.136+0000 7f39c27fc700 1 -- 172.21.15.131:0/3064378330 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v166) v1 ==== 169+0+0 (secure 0 0 0) 0x7f39c80545a0 con 0x7f39cc06eef0 2023-12-26T22:53:12.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f39b0066d80 msgr2=0x7f39b0069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f39b0066d80 0x7f39b0069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0ab690 tx=0x7f39bc0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39cc06eef0 msgr2=0x7f39cc0af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39cc06eef0 0x7f39cc0af1e0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c7c0 tx=0x7f39c8007f70 comp rx=0 tx=0).stop 2023-12-26T22:53:12.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 shutdown_connections 2023-12-26T22:53:12.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc13fca0 0x7f39cc0ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f39b0066d80 0x7f39b0069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39cc06eef0 0x7f39cc0af1e0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 --2- 172.21.15.131:0/3064378330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc0c9d00 0x7f39cc13e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 >> 172.21.15.131:0/3064378330 conn(0x7f39cc068020 msgr2=0x7f39cc13b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:12.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 shutdown_connections 2023-12-26T22:53:12.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.140+0000 7f39d45ff700 1 -- 172.21.15.131:0/3064378330 wait complete. 2023-12-26T22:53:12.141 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T22:53:12.159 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T22:53:12.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.351+0000 7fb5acfff700 1 Processor -- start 2023-12-26T22:53:12.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.351+0000 7fb5acfff700 1 -- start start 2023-12-26T22:53:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5acfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5a81599d0 0x7fb5a80ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5acfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a80d01e0 0x7fb5a80ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5acfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a80cfd50 con 0x7fb5a8159600 2023-12-26T22:53:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5acfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a80b1280 con 0x7fb5a81599d0 2023-12-26T22:53:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5acfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80cf140 con 0x7fb5a80d01e0 2023-12-26T22:53:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5a5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5a81599d0 0x7fb5a80ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5a6d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a80d01e0 0x7fb5a80ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5a5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5a81599d0 0x7fb5a80ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:35964/0 (socket says 172.21.15.131:35964) 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5a6d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a80d01e0 0x7fb5a80ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:45450/0 (socket says 172.21.15.131:45450) 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.353+0000 7fb5a5d9b700 1 -- 172.21.15.131:0/3261397635 learned_addr learned my addr 172.21.15.131:0/3261397635 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.354+0000 7fb5a5d9b700 1 -- 172.21.15.131:0/3261397635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a80d01e0 msgr2=0x7fb5a80ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.354+0000 7fb5a5d9b700 1 --2- 172.21.15.131:0/3261397635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a80d01e0 0x7fb5a80ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.354+0000 7fb5a5d9b700 1 -- 172.21.15.131:0/3261397635 >> v1:172.21.15.190:6789/0 conn(0x7fb5a8159600 legacy=0x7fb5a815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.354+0000 7fb5a5d9b700 1 -- 172.21.15.131:0/3261397635 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80cf420 con 0x7fb5a81599d0 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.354+0000 7fb5a559a700 1 -- 172.21.15.131:0/3261397635 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084969947 0 0) 0x7fb5a80cfd50 con 0x7fb5a8159600 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.354+0000 7fb5a5d9b700 1 --2- 172.21.15.131:0/3261397635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a81599d0 0x7fb5a80ceb60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0099b0 tx=0x7fb59c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b01add9ba72d6347 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.354+0000 7fb5a559a700 1 -- 172.21.15.131:0/3261397635 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c009ea0 con 0x7fb5a81599d0 2023-12-26T22:53:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.354+0000 7fb5a559a700 1 -- 172.21.15.131:0/3261397635 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c00b860 con 0x7fb5a81599d0 2023-12-26T22:53:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.354+0000 7fb5a559a700 1 -- 172.21.15.131:0/3261397635 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00bc00 con 0x7fb5a81599d0 2023-12-26T22:53:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.355+0000 7fb5acfff700 1 -- 172.21.15.131:0/3261397635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a81599d0 msgr2=0x7fb5a80ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.355+0000 7fb5acfff700 1 --2- 172.21.15.131:0/3261397635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a81599d0 0x7fb5a80ceb60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0099b0 tx=0x7fb59c009250 comp rx=0 tx=0).stop 2023-12-26T22:53:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.355+0000 7fb5acfff700 1 -- 172.21.15.131:0/3261397635 shutdown_connections 2023-12-26T22:53:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.355+0000 7fb5acfff700 1 --2- 172.21.15.131:0/3261397635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a80d01e0 0x7fb5a80ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.355+0000 7fb5acfff700 1 --2- 172.21.15.131:0/3261397635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a81599d0 0x7fb5a80ceb60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.355+0000 7fb5acfff700 1 -- 172.21.15.131:0/3261397635 >> 172.21.15.131:0/3261397635 conn(0x7fb5a806ece0 msgr2=0x7fb5a806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.355+0000 7fb5acfff700 1 -- 172.21.15.131:0/3261397635 shutdown_connections 2023-12-26T22:53:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.355+0000 7fb5acfff700 1 -- 172.21.15.131:0/3261397635 wait complete. 2023-12-26T22:53:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.356+0000 7fb5acfff700 1 Processor -- start 2023-12-26T22:53:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.357+0000 7fb5acfff700 1 -- start start 2023-12-26T22:53:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.357+0000 7fb5acfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5a80d01e0 0x7fb5a814bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.357+0000 7fb5acfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8159600 0x7fb5a80b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.357+0000 7fb5acfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a80c68c0 0x7fb5a80c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.357+0000 7fb5acfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80c7280 con 0x7fb5a80d01e0 2023-12-26T22:53:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.357+0000 7fb5acfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a814d020 con 0x7fb5a80c68c0 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.357+0000 7fb5acfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a814c410 con 0x7fb5a8159600 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5a80d01e0 0x7fb5a814bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5a80d01e0 0x7fb5a814bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50622/0 (socket says 172.21.15.131:50622) 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a659c700 1 -- 172.21.15.131:0/3293119814 learned_addr learned my addr 172.21.15.131:0/3293119814 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a5d9b700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8159600 0x7fb5a80b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a6d9d700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a80c68c0 0x7fb5a80c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a659c700 1 -- 172.21.15.131:0/3293119814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8159600 msgr2=0x7fb5a80b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a659c700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8159600 0x7fb5a80b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a659c700 1 -- 172.21.15.131:0/3293119814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a80c68c0 msgr2=0x7fb5a80c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a659c700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a80c68c0 0x7fb5a80c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.358+0000 7fb5a659c700 1 -- 172.21.15.131:0/3293119814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0094f0 con 0x7fb5a80d01e0 2023-12-26T22:53:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.359+0000 7fb5a659c700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5a80d01e0 0x7fb5a814bc70 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c810 tx=0x7fb598007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.359+0000 7fb5967fc700 1 -- 172.21.15.131:0/3293119814 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598011590 con 0x7fb5a80d01e0 2023-12-26T22:53:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.359+0000 7fb5967fc700 1 -- 172.21.15.131:0/3293119814 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb598011710 con 0x7fb5a80d01e0 2023-12-26T22:53:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.359+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a814c6f0 con 0x7fb5a80d01e0 2023-12-26T22:53:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.359+0000 7fb5967fc700 1 -- 172.21.15.131:0/3293119814 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598011ad0 con 0x7fb5a80d01e0 2023-12-26T22:53:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.359+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a814ccb0 con 0x7fb5a80d01e0 2023-12-26T22:53:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.360+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a80657e0 con 0x7fb5a80d01e0 2023-12-26T22:53:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.361+0000 7fb5967fc700 1 -- 172.21.15.131:0/3293119814 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb59801b3b0 con 0x7fb5a80d01e0 2023-12-26T22:53:12.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.362+0000 7fb5967fc700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb588066f30 0x7fb5880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.362+0000 7fb5967fc700 1 -- 172.21.15.131:0/3293119814 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fb598083ab0 con 0x7fb5a80d01e0 2023-12-26T22:53:12.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.362+0000 7fb5a5d9b700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb588066f30 0x7fb5880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.362+0000 7fb5a5d9b700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb588066f30 0x7fb5880693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0099b0 tx=0x7fb59c017000 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:12.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.366+0000 7fb5967fc700 1 -- 172.21.15.131:0/3293119814 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb598050cd0 con 0x7fb5a80d01e0 2023-12-26T22:53:12.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.557+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb5a80cf700 con 0x7fb5a80d01e0 2023-12-26T22:53:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.562+0000 7fb5967fc700 1 -- 172.21.15.131:0/3293119814 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb598017020 con 0x7fb5a80d01e0 2023-12-26T22:53:12.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb588066f30 msgr2=0x7fb5880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb588066f30 0x7fb5880693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0099b0 tx=0x7fb59c017000 comp rx=0 tx=0).stop 2023-12-26T22:53:12.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5a80d01e0 msgr2=0x7fb5a814bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5a80d01e0 0x7fb5a814bc70 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c810 tx=0x7fb598007dd0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 shutdown_connections 2023-12-26T22:53:12.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5a80c68c0 0x7fb5a80c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb588066f30 0x7fb5880693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5a80d01e0 0x7fb5a814bc70 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 --2- 172.21.15.131:0/3293119814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5a8159600 0x7fb5a80b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 >> 172.21.15.131:0/3293119814 conn(0x7fb5a806ece0 msgr2=0x7fb5a80cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:12.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 shutdown_connections 2023-12-26T22:53:12.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.564+0000 7fb5acfff700 1 -- 172.21.15.131:0/3293119814 wait complete. 2023-12-26T22:53:12.583 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T22:53:12.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.774+0000 7ffaba5ff700 1 Processor -- start 2023-12-26T22:53:12.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.774+0000 7ffaba5ff700 1 -- start start 2023-12-26T22:53:12.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.775+0000 7ffaba5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40b0750 0x7ffab40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.775+0000 7ffaba5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffab40c9400 0x7ffab40c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.775+0000 7ffaba5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab4156940 con 0x7ffab40cb320 2023-12-26T22:53:12.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.775+0000 7ffaba5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab40c8cc0 con 0x7ffab40c9400 2023-12-26T22:53:12.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.775+0000 7ffaba5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40ad700 con 0x7ffab40b0750 2023-12-26T22:53:12.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab3fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ffab40cb320 0x7ffab4069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37246/0 (socket says 172.21.15.131:37246) 2023-12-26T22:53:12.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab3fff700 1 -- 172.21.15.131:0/1518111422 learned_addr learned my addr 172.21.15.131:0/1518111422 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab37fe700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40b0750 0x7ffab40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593715061 0 0) 0x7ffab4156940 con 0x7ffab40cb320 2023-12-26T22:53:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c003570 con 0x7ffab40cb320 2023-12-26T22:53:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab8b9c700 1 --2- 172.21.15.131:0/1518111422 >> v2:172.21.15.190:3300/0 conn(0x7ffab40c9400 0x7ffab40c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2541655826 0 0) 0x7ffa9c003570 con 0x7ffab40cb320 2023-12-26T22:53:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40b0750 msgr2=0x7ffab40b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40b0750 0x7ffab40b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 >> v2:172.21.15.190:3300/0 conn(0x7ffab40c9400 msgr2=0x7ffab40c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 --2- 172.21.15.131:0/1518111422 >> v2:172.21.15.190:3300/0 conn(0x7ffab40c9400 0x7ffab40c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40b1100 con 0x7ffab40cb320 2023-12-26T22:53:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7ffaa4001470 con 0x7ffab40cb320 2023-12-26T22:53:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.776+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 >> v1:172.21.15.131:6789/0 conn(0x7ffab40cb320 legacy=0x7ffab4069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab2ffd700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40b0750 0x7ffa9c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab2ffd700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c003fe0 0x7ffa9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab2ffd700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa9c004d50 0x7ffa9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40c8cc0 con 0x7ffa9c004d50 2023-12-26T22:53:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c00dc70 con 0x7ffa9c003fe0 2023-12-26T22:53:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c00dde0 con 0x7ffab40b0750 2023-12-26T22:53:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab8b9c700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa9c004d50 0x7ffa9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab3fff700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40b0750 0x7ffa9c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab37fe700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c003fe0 0x7ffa9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab8b9c700 1 -- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40b0750 msgr2=0x7ffa9c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab8b9c700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40b0750 0x7ffa9c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab8b9c700 1 -- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c003fe0 msgr2=0x7ffa9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab8b9c700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c003fe0 0x7ffa9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab8b9c700 1 -- 172.21.15.131:0/1518111422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c003570 con 0x7ffa9c004d50 2023-12-26T22:53:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.777+0000 7ffab8b9c700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa9c004d50 0x7ffa9c0051c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00e210 tx=0x7ffaac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f411a91d16a885df server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00ea60 con 0x7ffa9c004d50 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac00f040 con 0x7ffa9c004d50 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffab2ffd700 1 -- 172.21.15.131:0/1518111422 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00ee10 con 0x7ffa9c004d50 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1518111422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa9c004d50 msgr2=0x7ffa9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffaba5ff700 1 --2- 172.21.15.131:0/1518111422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa9c004d50 0x7ffa9c0051c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00e210 tx=0x7ffaac00b040 comp rx=0 tx=0).stop 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffab8b9c700 1 -- 172.21.15.131:0/1518111422 reap_dead start 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1518111422 shutdown_connections 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1518111422 >> 172.21.15.131:0/1518111422 conn(0x7ffab4068040 msgr2=0x7ffab4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1518111422 shutdown_connections 2023-12-26T22:53:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.778+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1518111422 wait complete. 2023-12-26T22:53:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.780+0000 7ffaba5ff700 1 Processor -- start 2023-12-26T22:53:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.781+0000 7ffaba5ff700 1 -- start start 2023-12-26T22:53:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffaba5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c004d50 0x7ffab4153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffaba5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffab4153ed0 0x7ffab4164be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffaba5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab4165120 0x7ffab41655a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffaba5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab4154340 con 0x7ffab4153ed0 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffaba5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40a5760 con 0x7ffa9c004d50 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffaba5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab4154550 con 0x7ffab4165120 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffab4153ed0 0x7ffab4164be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffab4153ed0 0x7ffab4164be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50638/0 (socket says 172.21.15.131:50638) 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab37fe700 1 -- 172.21.15.131:0/1844758901 learned_addr learned my addr 172.21.15.131:0/1844758901 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab3fff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c004d50 0x7ffab4153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab8b9c700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab4165120 0x7ffab41655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab37fe700 1 -- 172.21.15.131:0/1844758901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab4165120 msgr2=0x7ffab41655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab37fe700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab4165120 0x7ffab41655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab37fe700 1 -- 172.21.15.131:0/1844758901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c004d50 msgr2=0x7ffab4153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab37fe700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c004d50 0x7ffab4153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab37fe700 1 -- 172.21.15.131:0/1844758901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab4154830 con 0x7ffab4153ed0 2023-12-26T22:53:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.782+0000 7ffab3fff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c004d50 0x7ffab4153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.783+0000 7ffab37fe700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffab4153ed0 0x7ffab4164be0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8006ac0 tx=0x7ffaa80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.783+0000 7ffa9bfff700 1 -- 172.21.15.131:0/1844758901 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800f940 con 0x7ffab4153ed0 2023-12-26T22:53:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.783+0000 7ffa9bfff700 1 -- 172.21.15.131:0/1844758901 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa800faa0 con 0x7ffab4153ed0 2023-12-26T22:53:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.783+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab4156e00 con 0x7ffab4153ed0 2023-12-26T22:53:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.783+0000 7ffa9bfff700 1 -- 172.21.15.131:0/1844758901 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800fe20 con 0x7ffab4153ed0 2023-12-26T22:53:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.783+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab41573c0 con 0x7ffab4153ed0 2023-12-26T22:53:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.784+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab4157730 con 0x7ffab4153ed0 2023-12-26T22:53:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.785+0000 7ffa9bfff700 1 -- 172.21.15.131:0/1844758901 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffaa8014650 con 0x7ffab4153ed0 2023-12-26T22:53:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.786+0000 7ffa9bfff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffaa0066f30 0x7ffaa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.786+0000 7ffa9bfff700 1 -- 172.21.15.131:0/1844758901 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5962+0+0 (secure 0 0 0) 0x7ffaa8018030 con 0x7ffab4153ed0 2023-12-26T22:53:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.786+0000 7ffab3fff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffaa0066f30 0x7ffaa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.786+0000 7ffab3fff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffaa0066f30 0x7ffaa00693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40015a0 tx=0x7ffaa4009040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.790+0000 7ffa9bfff700 1 -- 172.21.15.131:0/1844758901 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffaa8052240 con 0x7ffab4153ed0 2023-12-26T22:53:12.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.995+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ffab40657e0 con 0x7ffab4153ed0 2023-12-26T22:53:12.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:12.999+0000 7ffa9bfff700 1 -- 172.21.15.131:0/1844758901 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v56) v1 ==== 115+0+0 (secure 0 0 0) 0x7ffaa80220b0 con 0x7ffab4153ed0 2023-12-26T22:53:13.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffaa0066f30 msgr2=0x7ffaa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffaa0066f30 0x7ffaa00693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40015a0 tx=0x7ffaa4009040 comp rx=0 tx=0).stop 2023-12-26T22:53:13.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffab4153ed0 msgr2=0x7ffab4164be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffab4153ed0 0x7ffab4164be0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8006ac0 tx=0x7ffaa80091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 shutdown_connections 2023-12-26T22:53:13.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab4165120 0x7ffab41655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa9c004d50 0x7ffab4153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffaa0066f30 0x7ffaa00693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 --2- 172.21.15.131:0/1844758901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffab4153ed0 0x7ffab4164be0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 >> 172.21.15.131:0/1844758901 conn(0x7ffab4068040 msgr2=0x7ffab40acef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:13.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 shutdown_connections 2023-12-26T22:53:13.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.002+0000 7ffaba5ff700 1 -- 172.21.15.131:0/1844758901 wait complete. 2023-12-26T22:53:13.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T22:53:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.216+0000 7fc2085ff700 1 Processor -- start 2023-12-26T22:53:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.216+0000 7fc2085ff700 1 -- start start 2023-12-26T22:53:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc2085ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000cb320 0x7fc200069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc2085ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc2000b0e60 0x7fc2000c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc2085ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc200156940 con 0x7fc20006a1e0 2023-12-26T22:53:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc2085ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2000c8cc0 con 0x7fc2000b0e60 2023-12-26T22:53:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc2085ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2000ad700 con 0x7fc2000cb320 2023-12-26T22:53:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc205b9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc20006a1e0 0x7fc2000b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37262/0 (socket says 172.21.15.131:37262) 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc205b9a700 1 -- 172.21.15.131:0/1037548034 learned_addr learned my addr 172.21.15.131:0/1037548034 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc20639b700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000cb320 0x7fc200069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc206b9c700 1 --2- 172.21.15.131:0/1037548034 >> v2:172.21.15.190:3300/0 conn(0x7fc2000b0e60 0x7fc2000c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.218+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077945821 0 0) 0x7fc200156940 con 0x7fc20006a1e0 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ec003570 con 0x7fc20006a1e0 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749869962 0 0) 0x7fc1ec003570 con 0x7fc20006a1e0 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000cb320 msgr2=0x7fc200069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000cb320 0x7fc200069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 >> v2:172.21.15.190:3300/0 conn(0x7fc2000b0e60 msgr2=0x7fc2000c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 --2- 172.21.15.131:0/1037548034 >> v2:172.21.15.190:3300/0 conn(0x7fc2000b0e60 0x7fc2000c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2000ae500 con 0x7fc20006a1e0 2023-12-26T22:53:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fc1f0001470 con 0x7fc20006a1e0 2023-12-26T22:53:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 >> v1:172.21.15.131:6789/0 conn(0x7fc20006a1e0 legacy=0x7fc2000b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1ec003a60 0x7fc1ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000cb320 0x7fc1ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1ec004d50 0x7fc1ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2000c8cc0 con 0x7fc1ec004d50 2023-12-26T22:53:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec00dc70 con 0x7fc1ec003a60 2023-12-26T22:53:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec00ddb0 con 0x7fc2000cb320 2023-12-26T22:53:13.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc206b9c700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1ec004d50 0x7fc1ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc205b9a700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000cb320 0x7fc1ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.219+0000 7fc20639b700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1ec003a60 0x7fc1ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.220+0000 7fc206b9c700 1 -- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000cb320 msgr2=0x7fc1ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:13.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.220+0000 7fc206b9c700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000cb320 0x7fc1ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.220+0000 7fc206b9c700 1 -- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1ec003a60 msgr2=0x7fc1ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:13.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.220+0000 7fc206b9c700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1ec003a60 0x7fc1ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.220+0000 7fc206b9c700 1 -- 172.21.15.131:0/1037548034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec003570 con 0x7fc1ec004d50 2023-12-26T22:53:13.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.220+0000 7fc206b9c700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1ec004d50 0x7fc1ec0051c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc2000b1260 tx=0x7fc1f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb650290bb97e30 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.220+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800ea00 con 0x7fc1ec004d50 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.220+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f800f040 con 0x7fc1ec004d50 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.220+0000 7fc205399700 1 -- 172.21.15.131:0/1037548034 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800ed90 con 0x7fc1ec004d50 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.221+0000 7fc2085ff700 1 -- 172.21.15.131:0/1037548034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1ec004d50 msgr2=0x7fc1ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.221+0000 7fc2085ff700 1 --2- 172.21.15.131:0/1037548034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1ec004d50 0x7fc1ec0051c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc2000b1260 tx=0x7fc1f800b040 comp rx=0 tx=0).stop 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.221+0000 7fc206b9c700 1 -- 172.21.15.131:0/1037548034 reap_dead start 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.221+0000 7fc2085ff700 1 -- 172.21.15.131:0/1037548034 shutdown_connections 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.221+0000 7fc2085ff700 1 -- 172.21.15.131:0/1037548034 >> 172.21.15.131:0/1037548034 conn(0x7fc200068040 msgr2=0x7fc200069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.221+0000 7fc2085ff700 1 -- 172.21.15.131:0/1037548034 shutdown_connections 2023-12-26T22:53:13.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.221+0000 7fc2085ff700 1 -- 172.21.15.131:0/1037548034 wait complete. 2023-12-26T22:53:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.222+0000 7fc2085ff700 1 Processor -- start 2023-12-26T22:53:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.222+0000 7fc2085ff700 1 -- start start 2023-12-26T22:53:13.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.223+0000 7fc2085ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc20014c8f0 0x7fc2000c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.223+0000 7fc2085ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc20014f2b0 0x7fc20006de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.223+0000 7fc2085ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e3a0 0x7fc20014d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.223+0000 7fc2085ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20006e7f0 con 0x7fc20006e3a0 2023-12-26T22:53:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.223+0000 7fc2085ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20016a510 con 0x7fc20014f2b0 2023-12-26T22:53:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.223+0000 7fc2085ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20014d2d0 con 0x7fc20014c8f0 2023-12-26T22:53:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.223+0000 7fc206b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e3a0 0x7fc20014d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.223+0000 7fc206b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e3a0 0x7fc20014d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50644/0 (socket says 172.21.15.131:50644) 2023-12-26T22:53:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.223+0000 7fc206b9c700 1 -- 172.21.15.131:0/4272979977 learned_addr learned my addr 172.21.15.131:0/4272979977 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc205b9a700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc20014f2b0 0x7fc20006de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc20639b700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc20014c8f0 0x7fc2000c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc206b9c700 1 -- 172.21.15.131:0/4272979977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc20014c8f0 msgr2=0x7fc2000c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc206b9c700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc20014c8f0 0x7fc2000c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc206b9c700 1 -- 172.21.15.131:0/4272979977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc20014f2b0 msgr2=0x7fc20006de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc206b9c700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc20014f2b0 0x7fc20006de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc206b9c700 1 -- 172.21.15.131:0/4272979977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200169900 con 0x7fc20006e3a0 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc20639b700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc20014c8f0 0x7fc2000c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc206b9c700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e3a0 0x7fc20014d9d0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc2000690c0 tx=0x7fc1f801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc1f67fc700 1 -- 172.21.15.131:0/4272979977 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800ee80 con 0x7fc20006e3a0 2023-12-26T22:53:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc1f67fc700 1 -- 172.21.15.131:0/4272979977 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f801d630 con 0x7fc20006e3a0 2023-12-26T22:53:13.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc1f67fc700 1 -- 172.21.15.131:0/4272979977 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f801d9f0 con 0x7fc20006e3a0 2023-12-26T22:53:13.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc200169be0 con 0x7fc20006e3a0 2023-12-26T22:53:13.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.224+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20016a1a0 con 0x7fc20006e3a0 2023-12-26T22:53:13.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.225+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2000657e0 con 0x7fc20006e3a0 2023-12-26T22:53:13.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.226+0000 7fc1f67fc700 1 -- 172.21.15.131:0/4272979977 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc1f8003040 con 0x7fc20006e3a0 2023-12-26T22:53:13.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.227+0000 7fc1f67fc700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc1e4066f30 0x7fc1e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.227+0000 7fc1f67fc700 1 -- 172.21.15.131:0/4272979977 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc1f80857e0 con 0x7fc20006e3a0 2023-12-26T22:53:13.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.227+0000 7fc20639b700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc1e4066f30 0x7fc1e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.232 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.228+0000 7fc20639b700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc1e4066f30 0x7fc1e40693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0093f0 tx=0x7fc1fc008040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:13.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.231+0000 7fc1f67fc700 1 -- 172.21.15.131:0/4272979977 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1f8052950 con 0x7fc20006e3a0 2023-12-26T22:53:13.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.423+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fc20014d6f0 con 0x7fc20006e3a0 2023-12-26T22:53:13.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.427+0000 7fc1f67fc700 1 -- 172.21.15.131:0/4272979977 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v57) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc20014d6f0 con 0x7fc20006e3a0 2023-12-26T22:53:13.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc1e4066f30 msgr2=0x7fc1e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc1e4066f30 0x7fc1e40693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0093f0 tx=0x7fc1fc008040 comp rx=0 tx=0).stop 2023-12-26T22:53:13.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e3a0 msgr2=0x7fc20014d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e3a0 0x7fc20014d9d0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc2000690c0 tx=0x7fc1f801a000 comp rx=0 tx=0).stop 2023-12-26T22:53:13.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 shutdown_connections 2023-12-26T22:53:13.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc1e4066f30 0x7fc1e40693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e3a0 0x7fc20014d9d0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc20014c8f0 0x7fc2000c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 --2- 172.21.15.131:0/4272979977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc20014f2b0 0x7fc20006de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 >> 172.21.15.131:0/4272979977 conn(0x7fc200068040 msgr2=0x7fc2000c9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:13.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 shutdown_connections 2023-12-26T22:53:13.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.430+0000 7fc2085ff700 1 -- 172.21.15.131:0/4272979977 wait complete. 2023-12-26T22:53:13.448 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-12-26T22:53:13.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.638+0000 7f0b74bff700 1 Processor -- start 2023-12-26T22:53:13.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.638+0000 7f0b74bff700 1 -- start start 2023-12-26T22:53:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.640+0000 7f0b74bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b7013c510 0x7f0b700c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.640+0000 7f0b74bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b700686c0 con 0x7f0b7006b8c0 2023-12-26T22:53:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.640+0000 7f0b74bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b70069520 con 0x7f0b700cc810 2023-12-26T22:53:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.640+0000 7f0b74bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b70068910 con 0x7f0b7013c510 2023-12-26T22:53:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.640+0000 7f0b6dd9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0b7006b8c0 0x7f0b700cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37270/0 (socket says 172.21.15.131:37270) 2023-12-26T22:53:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.640+0000 7f0b6dd9b700 1 -- 172.21.15.131:0/3334986676 learned_addr learned my addr 172.21.15.131:0/3334986676 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.640+0000 7f0b6e59c700 1 --2- 172.21.15.131:0/3334986676 >> v2:172.21.15.190:3300/0 conn(0x7f0b7013c510 0x7f0b700c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.640+0000 7f0b6d59a700 1 -- 172.21.15.131:0/3334986676 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376697084 0 0) 0x7f0b700686c0 con 0x7f0b7006b8c0 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.640+0000 7f0b6d59a700 1 -- 172.21.15.131:0/3334986676 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b58003570 con 0x7f0b7006b8c0 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.641+0000 7f0b6d59a700 1 -- 172.21.15.131:0/3334986676 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705015107 0 0) 0x7f0b70069520 con 0x7f0b700cc810 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.641+0000 7f0b6d59a700 1 -- 172.21.15.131:0/3334986676 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b700686c0 con 0x7f0b700cc810 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.641+0000 7f0b6e59c700 1 -- 172.21.15.131:0/3334986676 >> v1:172.21.15.190:6789/0 conn(0x7f0b700cc810 legacy=0x7f0b700699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.641+0000 7f0b6e59c700 1 -- 172.21.15.131:0/3334986676 >> v1:172.21.15.131:6789/0 conn(0x7f0b7006b8c0 legacy=0x7f0b700cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.641+0000 7f0b6e59c700 1 -- 172.21.15.131:0/3334986676 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70068bf0 con 0x7f0b7013c510 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.641+0000 7f0b6e59c700 1 --2- 172.21.15.131:0/3334986676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 0x7f0b700c8a20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098d0 tx=0x7f0b60009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=929d99738716d7d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.641+0000 7f0b6d59a700 1 -- 172.21.15.131:0/3334986676 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60005c10 con 0x7f0b7013c510 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.641+0000 7f0b6d59a700 1 -- 172.21.15.131:0/3334986676 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b60005d70 con 0x7f0b7013c510 2023-12-26T22:53:13.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.641+0000 7f0b6d59a700 1 -- 172.21.15.131:0/3334986676 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60006260 con 0x7f0b7013c510 2023-12-26T22:53:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.642+0000 7f0b74bff700 1 -- 172.21.15.131:0/3334986676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 msgr2=0x7f0b700c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.642+0000 7f0b74bff700 1 --2- 172.21.15.131:0/3334986676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 0x7f0b700c8a20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098d0 tx=0x7f0b60009120 comp rx=0 tx=0).stop 2023-12-26T22:53:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.642+0000 7f0b74bff700 1 -- 172.21.15.131:0/3334986676 shutdown_connections 2023-12-26T22:53:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.642+0000 7f0b74bff700 1 --2- 172.21.15.131:0/3334986676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 0x7f0b700c8a20 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.642+0000 7f0b74bff700 1 -- 172.21.15.131:0/3334986676 >> 172.21.15.131:0/3334986676 conn(0x7f0b700ce3e0 msgr2=0x7f0b700ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.642+0000 7f0b74bff700 1 -- 172.21.15.131:0/3334986676 shutdown_connections 2023-12-26T22:53:13.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.642+0000 7f0b74bff700 1 -- 172.21.15.131:0/3334986676 wait complete. 2023-12-26T22:53:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.643+0000 7f0b74bff700 1 Processor -- start 2023-12-26T22:53:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.643+0000 7f0b74bff700 1 -- start start 2023-12-26T22:53:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.644+0000 7f0b74bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b7006b8c0 0x7f0b7013b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.644+0000 7f0b74bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b700cc810 0x7f0b7016a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.644+0000 7f0b74bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 0x7f0b700c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.644+0000 7f0b74bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7013bec0 con 0x7f0b7006b8c0 2023-12-26T22:53:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.644+0000 7f0b74bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700c8020 con 0x7f0b7013c510 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.644+0000 7f0b74bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7013c2b0 con 0x7f0b700cc810 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.644+0000 7f0b6e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b7006b8c0 0x7f0b7013b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.644+0000 7f0b6e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b7006b8c0 0x7f0b7013b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50658/0 (socket says 172.21.15.131:50658) 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.644+0000 7f0b6e59c700 1 -- 172.21.15.131:0/3748298988 learned_addr learned my addr 172.21.15.131:0/3748298988 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6dd9b700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b700cc810 0x7f0b7016a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6ed9d700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 0x7f0b700c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6e59c700 1 -- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b700cc810 msgr2=0x7f0b7016a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6e59c700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b700cc810 0x7f0b7016a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6e59c700 1 -- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 msgr2=0x7f0b700c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6e59c700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 0x7f0b700c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6e59c700 1 -- 172.21.15.131:0/3748298988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b600093c0 con 0x7f0b7006b8c0 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6dd9b700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b700cc810 0x7f0b7016a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6ed9d700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 0x7f0b700c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b6e59c700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b7006b8c0 0x7f0b7013b980 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009160 tx=0x7f0b60016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b5e7fc700 1 -- 172.21.15.131:0/3748298988 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6001d9e0 con 0x7f0b7006b8c0 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b5e7fc700 1 -- 172.21.15.131:0/3748298988 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6001db40 con 0x7f0b7006b8c0 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7016a760 con 0x7f0b7006b8c0 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.645+0000 7f0b5e7fc700 1 -- 172.21.15.131:0/3748298988 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6001b480 con 0x7f0b7006b8c0 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.646+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b700c79d0 con 0x7f0b7006b8c0 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.646+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b700657e0 con 0x7f0b7006b8c0 2023-12-26T22:53:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.647+0000 7f0b5e7fc700 1 -- 172.21.15.131:0/3748298988 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0b6001b7f0 con 0x7f0b7006b8c0 2023-12-26T22:53:13.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.648+0000 7f0b5e7fc700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0b50066f30 0x7f0b500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:13.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.648+0000 7f0b5e7fc700 1 -- 172.21.15.131:0/3748298988 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0b600863a0 con 0x7f0b7006b8c0 2023-12-26T22:53:13.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.648+0000 7f0b6dd9b700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0b50066f30 0x7f0b500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:13.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.649+0000 7f0b6dd9b700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0b50066f30 0x7f0b500693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0b64007fd0 tx=0x7f0b64009040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:13.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.652+0000 7f0b5e7fc700 1 -- 172.21.15.131:0/3748298988 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b60053510 con 0x7f0b7006b8c0 2023-12-26T22:53:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:13.920+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f0b70068ed0 con 0x7f0b7006b8c0 2023-12-26T22:53:14.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.131+0000 7f0b5e7fc700 1 -- 172.21.15.131:0/3748298988 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2473) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0b6002a0f0 con 0x7f0b7006b8c0 2023-12-26T22:53:14.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0b50066f30 msgr2=0x7f0b500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:14.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0b50066f30 0x7f0b500693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0b64007fd0 tx=0x7f0b64009040 comp rx=0 tx=0).stop 2023-12-26T22:53:14.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b7006b8c0 msgr2=0x7f0b7013b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:14.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b7006b8c0 0x7f0b7013b980 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009160 tx=0x7f0b60016000 comp rx=0 tx=0).stop 2023-12-26T22:53:14.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 shutdown_connections 2023-12-26T22:53:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b7013c510 0x7f0b700c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0b50066f30 0x7f0b500693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b7006b8c0 0x7f0b7013b980 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 --2- 172.21.15.131:0/3748298988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b700cc810 0x7f0b7016a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 >> 172.21.15.131:0/3748298988 conn(0x7f0b700ce3e0 msgr2=0x7f0b701524f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 shutdown_connections 2023-12-26T22:53:14.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.134+0000 7f0b74bff700 1 -- 172.21.15.131:0/3748298988 wait complete. 2023-12-26T22:53:14.152 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-12-26T22:53:14.152 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-12-26T22:53:14.152 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T22:53:14.153 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-12-26T22:53:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.344+0000 7f9d1c9ff700 1 Processor -- start 2023-12-26T22:53:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.345+0000 7f9d1c9ff700 1 -- start start 2023-12-26T22:53:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1c9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9d18159560 0x7f9d1815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1c9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9d18159930 0x7f9d180b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1c9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d18068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1c9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d18068a80 con 0x7f9d18159560 2023-12-26T22:53:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1c9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d180cac50 con 0x7f9d18159930 2023-12-26T22:53:14.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1c9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1815da00 con 0x7f9d1806a1a0 2023-12-26T22:53:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1659c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9d18159560 0x7f9d1815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1659c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9d18159560 0x7f9d1815d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50674/0 (socket says 172.21.15.131:50674) 2023-12-26T22:53:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1659c700 1 -- 172.21.15.131:0/3626008046 learned_addr learned my addr 172.21.15.131:0/3626008046 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d15d9b700 1 --2- 172.21.15.131:0/3626008046 >> v2:172.21.15.190:3300/0 conn(0x7f9d18159930 0x7f9d180b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d16d9d700 1 --2- 172.21.15.131:0/3626008046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d18068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1659c700 1 -- 172.21.15.131:0/3626008046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 msgr2=0x7f9d18068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1659c700 1 --2- 172.21.15.131:0/3626008046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d18068f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.346+0000 7f9d1659c700 1 -- 172.21.15.131:0/3626008046 >> v2:172.21.15.190:3300/0 conn(0x7f9d18159930 msgr2=0x7f9d180b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.347+0000 7f9d1659c700 1 --2- 172.21.15.131:0/3626008046 >> v2:172.21.15.190:3300/0 conn(0x7f9d18159930 0x7f9d180b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.347+0000 7f9d1659c700 1 -- 172.21.15.131:0/3626008046 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1815dce0 con 0x7f9d18159560 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.347+0000 7f9d16d9d700 1 --2- 172.21.15.131:0/3626008046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d18068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.347+0000 7f9d15d9b700 1 --2- 172.21.15.131:0/3626008046 >> v2:172.21.15.190:3300/0 conn(0x7f9d18159930 0x7f9d180b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.347+0000 7f9d1659c700 1 --2- 172.21.15.131:0/3626008046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 0x7f9d1815d300 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f9d180c1ba0 tx=0x7f9d08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb8e0379e2fe377 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.347+0000 7f9d1559a700 1 -- 172.21.15.131:0/3626008046 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800b860 con 0x7f9d18159560 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.347+0000 7f9d1559a700 1 -- 172.21.15.131:0/3626008046 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0800b9c0 con 0x7f9d18159560 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.347+0000 7f9d1559a700 1 -- 172.21.15.131:0/3626008046 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800bd60 con 0x7f9d18159560 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.348+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/3626008046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 msgr2=0x7f9d1815d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.348+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/3626008046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 0x7f9d1815d300 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f9d180c1ba0 tx=0x7f9d08009250 comp rx=0 tx=0).stop 2023-12-26T22:53:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.348+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/3626008046 shutdown_connections 2023-12-26T22:53:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.348+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/3626008046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 0x7f9d1815d300 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.348+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/3626008046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d18068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.348+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/3626008046 >> v2:172.21.15.190:3300/0 conn(0x7f9d18159930 0x7f9d180b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.348+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/3626008046 >> 172.21.15.131:0/3626008046 conn(0x7f9d180ce3d0 msgr2=0x7f9d180ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.348+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/3626008046 shutdown_connections 2023-12-26T22:53:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.348+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/3626008046 wait complete. 2023-12-26T22:53:14.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.349+0000 7f9d1c9ff700 1 Processor -- start 2023-12-26T22:53:14.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.349+0000 7f9d1c9ff700 1 -- start start 2023-12-26T22:53:14.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.350+0000 7f9d1c9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d180b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:14.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.350+0000 7f9d1c9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 0x7f9d1815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:14.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.350+0000 7f9d1c9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d18159930 0x7f9d1815b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:14.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.350+0000 7f9d1c9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1815c800 con 0x7f9d18159560 2023-12-26T22:53:14.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.350+0000 7f9d1c9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180c2ab0 con 0x7f9d18159930 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.350+0000 7f9d1c9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1815bbf0 con 0x7f9d1806a1a0 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.350+0000 7f9d15d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 0x7f9d1815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d15d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 0x7f9d1815b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50680/0 (socket says 172.21.15.131:50680) 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d15d9b700 1 -- 172.21.15.131:0/1840804901 learned_addr learned my addr 172.21.15.131:0/1840804901 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d16d9d700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d18159930 0x7f9d1815b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d1659c700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d180b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d15d9b700 1 -- 172.21.15.131:0/1840804901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 msgr2=0x7f9d180b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d15d9b700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d180b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d15d9b700 1 -- 172.21.15.131:0/1840804901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d18159930 msgr2=0x7f9d1815b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:14.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d15d9b700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d18159930 0x7f9d1815b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d15d9b700 1 -- 172.21.15.131:0/1840804901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d080094f0 con 0x7f9d18159560 2023-12-26T22:53:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d1659c700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d180b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d15d9b700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 0x7f9d1815b080 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00e850 tx=0x7f9d0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d067fc700 1 -- 172.21.15.131:0/1840804901 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c013410 con 0x7f9d18159560 2023-12-26T22:53:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d067fc700 1 -- 172.21.15.131:0/1840804901 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c013570 con 0x7f9d18159560 2023-12-26T22:53:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d067fc700 1 -- 172.21.15.131:0/1840804901 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c013910 con 0x7f9d18159560 2023-12-26T22:53:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.351+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1815bed0 con 0x7f9d18159560 2023-12-26T22:53:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.352+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1815c490 con 0x7f9d18159560 2023-12-26T22:53:14.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.353+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d180657e0 con 0x7f9d18159560 2023-12-26T22:53:14.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.353+0000 7f9d067fc700 1 -- 172.21.15.131:0/1840804901 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9d0c013d00 con 0x7f9d18159560 2023-12-26T22:53:14.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.354+0000 7f9d067fc700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9cf8066f30 0x7f9cf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:14.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.354+0000 7f9d067fc700 1 -- 172.21.15.131:0/1840804901 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f9d0c084330 con 0x7f9d18159560 2023-12-26T22:53:14.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.354+0000 7f9d1659c700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9cf8066f30 0x7f9cf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:14.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.355+0000 7f9d1659c700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9cf8066f30 0x7f9cf80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9d180c1ba0 tx=0x7f9d08017000 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:14.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.358+0000 7f9d067fc700 1 -- 172.21.15.131:0/1840804901 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d0c0514a0 con 0x7f9d18159560 2023-12-26T22:53:14.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:14.631+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f9d1815c940 con 0x7f9d18159560 2023-12-26T22:53:15.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.142+0000 7f9d067fc700 1 -- 172.21.15.131:0/1840804901 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2474) v1 ==== 168+0+0 (secure 0 0 0) 0x7f9d0c054ac0 con 0x7f9d18159560 2023-12-26T22:53:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.144+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9cf8066f30 msgr2=0x7f9cf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.144+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9cf8066f30 0x7f9cf80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9d180c1ba0 tx=0x7f9d08017000 comp rx=0 tx=0).stop 2023-12-26T22:53:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.144+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 msgr2=0x7f9d1815b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.144+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 0x7f9d1815b080 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00e850 tx=0x7f9d0c008040 comp rx=0 tx=0).stop 2023-12-26T22:53:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.145+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 shutdown_connections 2023-12-26T22:53:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.145+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d18159930 0x7f9d1815b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.145+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1806a1a0 0x7f9d180b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.145+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f9cf8066f30 0x7f9cf80693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.145+0000 7f9d1c9ff700 1 --2- 172.21.15.131:0/1840804901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d18159560 0x7f9d1815b080 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.145+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 >> 172.21.15.131:0/1840804901 conn(0x7f9d180ce3d0 msgr2=0x7f9d18157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:15.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.145+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 shutdown_connections 2023-12-26T22:53:15.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.145+0000 7f9d1c9ff700 1 -- 172.21.15.131:0/1840804901 wait complete. 2023-12-26T22:53:15.163 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T22:53:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.356+0000 7f19821ff700 1 Processor -- start 2023-12-26T22:53:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.357+0000 7f19821ff700 1 -- start start 2023-12-26T22:53:15.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.358+0000 7f19821ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f197c159920 0x7f197c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.358+0000 7f19821ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f197c0ced90 0x7f197c0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.358+0000 7f19821ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c0c6810 con 0x7f197c0c5420 2023-12-26T22:53:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.358+0000 7f19821ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197c15a230 con 0x7f197c0ced90 2023-12-26T22:53:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.358+0000 7f19821ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197c0cf6d0 con 0x7f197c159920 2023-12-26T22:53:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.358+0000 7f197bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f197c0ced90 0x7f197c0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.358+0000 7f197bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f197c0ced90 0x7f197c0cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60268/0 (socket says 172.21.15.131:60268) 2023-12-26T22:53:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.358+0000 7f197bfff700 1 -- 172.21.15.131:0/4194540820 learned_addr learned my addr 172.21.15.131:0/4194540820 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.358+0000 7f197b7fe700 1 --2- 172.21.15.131:0/4194540820 >> v2:172.21.15.190:3300/0 conn(0x7f197c159920 0x7f197c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.359+0000 7f197bfff700 1 -- 172.21.15.131:0/4194540820 >> v2:172.21.15.190:3300/0 conn(0x7f197c159920 msgr2=0x7f197c159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.359+0000 7f197bfff700 1 --2- 172.21.15.131:0/4194540820 >> v2:172.21.15.190:3300/0 conn(0x7f197c159920 0x7f197c159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.359+0000 7f197bfff700 1 -- 172.21.15.131:0/4194540820 >> v1:172.21.15.190:6789/0 conn(0x7f197c0c5420 legacy=0x7f197c0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.359+0000 7f197bfff700 1 -- 172.21.15.131:0/4194540820 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0c5c00 con 0x7f197c0ced90 2023-12-26T22:53:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.359+0000 7f197bfff700 1 --2- 172.21.15.131:0/4194540820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0ced90 0x7f197c0cf190 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f197c088ec0 tx=0x7f1974009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69c0726a0c85f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.359+0000 7f197a7fc700 1 -- 172.21.15.131:0/4194540820 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400e070 con 0x7f197c0ced90 2023-12-26T22:53:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.359+0000 7f197a7fc700 1 -- 172.21.15.131:0/4194540820 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197400ba20 con 0x7f197c0ced90 2023-12-26T22:53:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.359+0000 7f197a7fc700 1 -- 172.21.15.131:0/4194540820 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740092e0 con 0x7f197c0ced90 2023-12-26T22:53:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.360+0000 7f19821ff700 1 -- 172.21.15.131:0/4194540820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0ced90 msgr2=0x7f197c0cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.360+0000 7f19821ff700 1 --2- 172.21.15.131:0/4194540820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0ced90 0x7f197c0cf190 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f197c088ec0 tx=0x7f1974009840 comp rx=0 tx=0).stop 2023-12-26T22:53:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.360+0000 7f19821ff700 1 -- 172.21.15.131:0/4194540820 shutdown_connections 2023-12-26T22:53:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.360+0000 7f19821ff700 1 --2- 172.21.15.131:0/4194540820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0ced90 0x7f197c0cf190 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.360+0000 7f19821ff700 1 --2- 172.21.15.131:0/4194540820 >> v2:172.21.15.190:3300/0 conn(0x7f197c159920 0x7f197c159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.360+0000 7f19821ff700 1 -- 172.21.15.131:0/4194540820 >> 172.21.15.131:0/4194540820 conn(0x7f197c099980 msgr2=0x7f197c06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.360+0000 7f19821ff700 1 -- 172.21.15.131:0/4194540820 shutdown_connections 2023-12-26T22:53:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.360+0000 7f19821ff700 1 -- 172.21.15.131:0/4194540820 wait complete. 2023-12-26T22:53:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.361+0000 7f19821ff700 1 Processor -- start 2023-12-26T22:53:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.362+0000 7f19821ff700 1 -- start start 2023-12-26T22:53:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.362+0000 7f19821ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0c5420 0x7f197c0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.362+0000 7f19821ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f197c0ced90 0x7f197c0c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.362+0000 7f19821ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f197c159920 0x7f197c154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.362+0000 7f19821ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c169990 con 0x7f197c0c5420 2023-12-26T22:53:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.362+0000 7f19821ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c169640 con 0x7f197c0ced90 2023-12-26T22:53:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.362+0000 7f19821ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0c7e30 con 0x7f197c159920 2023-12-26T22:53:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0c5420 0x7f197c0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0c5420 0x7f197c0c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60282/0 (socket says 172.21.15.131:60282) 2023-12-26T22:53:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197b7fe700 1 -- 172.21.15.131:0/3102384441 learned_addr learned my addr 172.21.15.131:0/3102384441 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197affd700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f197c0ced90 0x7f197c0c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197bfff700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f197c159920 0x7f197c154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197b7fe700 1 -- 172.21.15.131:0/3102384441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f197c159920 msgr2=0x7f197c154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197b7fe700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f197c159920 0x7f197c154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197b7fe700 1 -- 172.21.15.131:0/3102384441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f197c0ced90 msgr2=0x7f197c0c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197b7fe700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f197c0ced90 0x7f197c0c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197b7fe700 1 -- 172.21.15.131:0/3102384441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974009ae0 con 0x7f197c0c5420 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197affd700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f197c0ced90 0x7f197c0c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f197b7fe700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0c5420 0x7f197c0c9200 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f196c00ec80 tx=0x7f196c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f19637fe700 1 -- 172.21.15.131:0/3102384441 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c00c9d0 con 0x7f197c0c5420 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.363+0000 7f19637fe700 1 -- 172.21.15.131:0/3102384441 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c00cb30 con 0x7f197c0c5420 2023-12-26T22:53:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.364+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c0c8110 con 0x7f197c0c5420 2023-12-26T22:53:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.364+0000 7f19637fe700 1 -- 172.21.15.131:0/3102384441 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c013470 con 0x7f197c0c5420 2023-12-26T22:53:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.364+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c155490 con 0x7f197c0c5420 2023-12-26T22:53:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.365+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c0657e0 con 0x7f197c0c5420 2023-12-26T22:53:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.365+0000 7f19637fe700 1 -- 172.21.15.131:0/3102384441 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f196c013860 con 0x7f197c0c5420 2023-12-26T22:53:15.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.366+0000 7f19637fe700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1968066f30 0x7f19680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.366+0000 7f19637fe700 1 -- 172.21.15.131:0/3102384441 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f196c084d20 con 0x7f197c0c5420 2023-12-26T22:53:15.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.367+0000 7f197affd700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1968066f30 0x7f19680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.367+0000 7f197affd700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1968066f30 0x7f19680693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1970007fd0 tx=0x7f1970009040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:15.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.370+0000 7f19637fe700 1 -- 172.21.15.131:0/3102384441 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f196c051e90 con 0x7f197c0c5420 2023-12-26T22:53:15.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.561+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f197c0c6d40 con 0x7f197c0c5420 2023-12-26T22:53:15.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.566+0000 7f19637fe700 1 -- 172.21.15.131:0/3102384441 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f196c0170c0 con 0x7f197c0c5420 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1968066f30 msgr2=0x7f19680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1968066f30 0x7f19680693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1970007fd0 tx=0x7f1970009040 comp rx=0 tx=0).stop 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0c5420 msgr2=0x7f197c0c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0c5420 0x7f197c0c9200 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f196c00ec80 tx=0x7f196c009120 comp rx=0 tx=0).stop 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 shutdown_connections 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f197c159920 0x7f197c154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1968066f30 0x7f19680693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f197c0c5420 0x7f197c0c9200 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 --2- 172.21.15.131:0/3102384441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f197c0ced90 0x7f197c0c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 >> 172.21.15.131:0/3102384441 conn(0x7f197c099980 msgr2=0x7f197c0685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:15.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 shutdown_connections 2023-12-26T22:53:15.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.568+0000 7f19821ff700 1 -- 172.21.15.131:0/3102384441 wait complete. 2023-12-26T22:53:15.586 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T22:53:15.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T22:53:15.674 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T22:53:15.784 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T22:53:15.785 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T22:53:15.786 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:53:15.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.979+0000 7f0a743ff700 1 Processor -- start 2023-12-26T22:53:15.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.979+0000 7f0a743ff700 1 -- start start 2023-12-26T22:53:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.980+0000 7f0a743ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a6c13c510 0x7f0a6c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.980+0000 7f0a743ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c0cc810 0x7f0a6c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.980+0000 7f0a743ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c0686c0 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.980+0000 7f0a743ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6c06c280 con 0x7f0a6c13c510 2023-12-26T22:53:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.980+0000 7f0a743ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c069520 con 0x7f0a6c0cc810 2023-12-26T22:53:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.980+0000 7f0a7219b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a6c13c510 0x7f0a6c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.980+0000 7f0a7219b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a6c13c510 0x7f0a6c0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60286/0 (socket says 172.21.15.131:60286) 2023-12-26T22:53:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.980+0000 7f0a7219b700 1 -- 172.21.15.131:0/192919686 learned_addr learned my addr 172.21.15.131:0/192919686 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.980+0000 7f0a7299c700 1 --2- 172.21.15.131:0/192919686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c0cc810 0x7f0a6c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.981+0000 7f0a7219b700 1 -- 172.21.15.131:0/192919686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c0cc810 msgr2=0x7f0a6c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:15.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.981+0000 7f0a7219b700 1 --2- 172.21.15.131:0/192919686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c0cc810 0x7f0a6c06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.981+0000 7f0a7219b700 1 -- 172.21.15.131:0/192919686 >> v1:172.21.15.190:6789/0 conn(0x7f0a6c06b8c0 legacy=0x7f0a6c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.981+0000 7f0a7219b700 1 -- 172.21.15.131:0/192919686 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c068910 con 0x7f0a6c13c510 2023-12-26T22:53:15.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.981+0000 7f0a7219b700 1 --2- 172.21.15.131:0/192919686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c13c510 0x7f0a6c0c8a20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009990 tx=0x7f0a68009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad330eb93177366 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:15.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.981+0000 7f0a71199700 1 -- 172.21.15.131:0/192919686 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68005c10 con 0x7f0a6c13c510 2023-12-26T22:53:15.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.981+0000 7f0a71199700 1 -- 172.21.15.131:0/192919686 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a68005d70 con 0x7f0a6c13c510 2023-12-26T22:53:15.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.981+0000 7f0a71199700 1 -- 172.21.15.131:0/192919686 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800b880 con 0x7f0a6c13c510 2023-12-26T22:53:15.983 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.982+0000 7f0a743ff700 1 -- 172.21.15.131:0/192919686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c13c510 msgr2=0x7f0a6c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.982+0000 7f0a743ff700 1 --2- 172.21.15.131:0/192919686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c13c510 0x7f0a6c0c8a20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009990 tx=0x7f0a68009220 comp rx=0 tx=0).stop 2023-12-26T22:53:15.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.982+0000 7f0a743ff700 1 -- 172.21.15.131:0/192919686 shutdown_connections 2023-12-26T22:53:15.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.982+0000 7f0a743ff700 1 --2- 172.21.15.131:0/192919686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c0cc810 0x7f0a6c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.982+0000 7f0a743ff700 1 --2- 172.21.15.131:0/192919686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c13c510 0x7f0a6c0c8a20 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.982+0000 7f0a743ff700 1 -- 172.21.15.131:0/192919686 >> 172.21.15.131:0/192919686 conn(0x7f0a6c0ce3e0 msgr2=0x7f0a6c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:15.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.982+0000 7f0a743ff700 1 -- 172.21.15.131:0/192919686 shutdown_connections 2023-12-26T22:53:15.984 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.982+0000 7f0a743ff700 1 -- 172.21.15.131:0/192919686 wait complete. 2023-12-26T22:53:15.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.983+0000 7f0a743ff700 1 Processor -- start 2023-12-26T22:53:15.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.983+0000 7f0a743ff700 1 -- start start 2023-12-26T22:53:15.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.984+0000 7f0a743ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c06b8c0 0x7f0a6c15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.984+0000 7f0a743ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6c0cc810 0x7f0a6c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.984+0000 7f0a743ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c13c510 0x7f0a6c0c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.984+0000 7f0a743ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c16aaa0 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.984+0000 7f0a743ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c169bd0 con 0x7f0a6c0cc810 2023-12-26T22:53:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.984+0000 7f0a743ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c169e90 con 0x7f0a6c13c510 2023-12-26T22:53:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7219b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c06b8c0 0x7f0a6c15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7219b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c06b8c0 0x7f0a6c15b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60288/0 (socket says 172.21.15.131:60288) 2023-12-26T22:53:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7219b700 1 -- 172.21.15.131:0/2634251740 learned_addr learned my addr 172.21.15.131:0/2634251740 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7199a700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6c0cc810 0x7f0a6c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7299c700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c13c510 0x7f0a6c0c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7219b700 1 -- 172.21.15.131:0/2634251740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c13c510 msgr2=0x7f0a6c0c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7219b700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c13c510 0x7f0a6c0c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7219b700 1 -- 172.21.15.131:0/2634251740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6c0cc810 msgr2=0x7f0a6c15b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7219b700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6c0cc810 0x7f0a6c15b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7219b700 1 -- 172.21.15.131:0/2634251740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a68009480 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7299c700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c13c510 0x7f0a6c0c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a7219b700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c06b8c0 0x7f0a6c15b490 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f0a68005d40 tx=0x7f0a68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.985+0000 7f0a627fc700 1 -- 172.21.15.131:0/2634251740 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a680119b0 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.986+0000 7f0a627fc700 1 -- 172.21.15.131:0/2634251740 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a68011b10 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.986+0000 7f0a627fc700 1 -- 172.21.15.131:0/2634251740 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68011e90 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.986+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c16a170 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.986+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c16a730 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.987+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c150ac0 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.988+0000 7f0a627fc700 1 -- 172.21.15.131:0/2634251740 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0a6801b640 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.988+0000 7f0a627fc700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0a50066f30 0x7f0a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:15.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.988+0000 7f0a627fc700 1 -- 172.21.15.131:0/2634251740 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0a680860b0 con 0x7f0a6c06b8c0 2023-12-26T22:53:15.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.989+0000 7f0a7199a700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0a50066f30 0x7f0a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:15.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.989+0000 7f0a7199a700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0a50066f30 0x7f0a500693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0ca400 tx=0x7f0a5c007ec0 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:15.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:15.992+0000 7f0a627fc700 1 -- 172.21.15.131:0/2634251740 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a68053180 con 0x7f0a6c06b8c0 2023-12-26T22:53:16.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.230+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a6c0657e0 con 0x7f0a6c06b8c0 2023-12-26T22:53:16.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.230+0000 7f0a627fc700 1 -- 172.21.15.131:0/2634251740 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f0a6802a090 con 0x7f0a6c06b8c0 2023-12-26T22:53:16.230 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:16.230 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2085575956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2085575956},{"type":"v1","addr":"172.21.15.190:6834","nonce":2085575956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2129957224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2129957224},{"type":"v1","addr":"172.21.15.190:6835","nonce":2129957224}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":8625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3810273571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3810273571},{"type":"v1","addr":"172.21.15.131:6834","nonce":3810273571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/199102307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":199102307},{"type":"v1","addr":"172.21.15.131:6835","nonce":199102307}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[]} 2023-12-26T22:53:16.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0a50066f30 msgr2=0x7f0a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:16.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0a50066f30 0x7f0a500693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0ca400 tx=0x7f0a5c007ec0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c06b8c0 msgr2=0x7f0a6c15b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:16.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c06b8c0 0x7f0a6c15b490 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f0a68005d40 tx=0x7f0a68016000 comp rx=0 tx=0).stop 2023-12-26T22:53:16.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 shutdown_connections 2023-12-26T22:53:16.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a6c13c510 0x7f0a6c0c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0a50066f30 0x7f0a500693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a6c06b8c0 0x7f0a6c15b490 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 --2- 172.21.15.131:0/2634251740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a6c0cc810 0x7f0a6c15b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 >> 172.21.15.131:0/2634251740 conn(0x7f0a6c0ce3e0 msgr2=0x7f0a6c13b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:16.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 shutdown_connections 2023-12-26T22:53:16.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.233+0000 7f0a743ff700 1 -- 172.21.15.131:0/2634251740 wait complete. 2023-12-26T22:53:16.234 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 75 2023-12-26T22:53:16.251 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T22:53:16.251 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T22:53:16.251 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T22:53:16.251 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T22:53:16.251 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T22:53:16.252 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T22:53:16.252 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T22:53:16.252 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T22:53:16.252 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T22:53:16.252 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T22:53:16.253 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T22:53:16.253 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T22:53:16.253 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T22:53:16.253 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T22:53:16.253 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T22:53:16.254 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T22:53:16.268 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T22:53:16.268 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T22:53:16.270 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T22:53:16.270 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T22:53:16.272 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T22:53:16.272 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T22:53:16.273 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T22:53:16.273 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T22:53:16.314 INFO:tasks.ceph.mds.a:Started 2023-12-26T22:53:16.314 DEBUG:teuthology.parallel:result is None 2023-12-26T22:53:16.316 INFO:tasks.ceph.mds.c:Started 2023-12-26T22:53:16.316 DEBUG:teuthology.parallel:result is None 2023-12-26T22:53:16.317 INFO:tasks.ceph.mds.b:Started 2023-12-26T22:53:16.317 DEBUG:teuthology.parallel:result is None 2023-12-26T22:53:16.320 INFO:tasks.ceph.mds.d:Started 2023-12-26T22:53:16.320 DEBUG:teuthology.parallel:result is None 2023-12-26T22:53:16.320 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T22:53:16.456 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T22:53:16.457 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T22:53:16.463 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T22:53:16.464 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T22:53:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.522+0000 7f7c15dff700 1 Processor -- start 2023-12-26T22:53:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.522+0000 7f7c15dff700 1 -- start start 2023-12-26T22:53:16.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.523+0000 7f7c15dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c100cb320 0x7f7c10069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:16.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.523+0000 7f7c15dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7c100b0e60 0x7f7c100c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.523+0000 7f7c15dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c10156940 con 0x7f7c1006a1e0 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.523+0000 7f7c15dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c100c8cc0 con 0x7f7c100b0e60 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.523+0000 7f7c15dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c100ad700 con 0x7f7c100cb320 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7c100b0e60 0x7f7c100c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0ffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7c100b0e60 0x7f7c100c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60318/0 (socket says 172.21.15.131:60318) 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0ffff700 1 -- 172.21.15.131:0/757198877 learned_addr learned my addr 172.21.15.131:0/757198877 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0effd700 1 --1- 172.21.15.131:0/757198877 >> v1:172.21.15.131:6789/0 conn(0x7f7c1006a1e0 0x7f7c100b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40290/0 (socket says 172.21.15.131:40290) 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0f7fe700 1 --2- 172.21.15.131:0/757198877 >> v2:172.21.15.190:3300/0 conn(0x7f7c100cb320 0x7f7c10069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0ffff700 1 -- 172.21.15.131:0/757198877 >> v2:172.21.15.190:3300/0 conn(0x7f7c100cb320 msgr2=0x7f7c10069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0ffff700 1 --2- 172.21.15.131:0/757198877 >> v2:172.21.15.190:3300/0 conn(0x7f7c100cb320 0x7f7c10069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0ffff700 1 -- 172.21.15.131:0/757198877 >> v1:172.21.15.131:6789/0 conn(0x7f7c1006a1e0 legacy=0x7f7c100b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0ffff700 1 -- 172.21.15.131:0/757198877 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c100ae500 con 0x7f7c100b0e60 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.524+0000 7f7c0ffff700 1 --2- 172.21.15.131:0/757198877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c100b0e60 0x7f7c100c9400 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7c10064410 tx=0x7f7c08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc96c4e92a5b36a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c0e7fc700 1 -- 172.21.15.131:0/757198877 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800e070 con 0x7f7c100b0e60 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c0e7fc700 1 -- 172.21.15.131:0/757198877 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0800ba50 con 0x7f7c100b0e60 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c0e7fc700 1 -- 172.21.15.131:0/757198877 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080092e0 con 0x7f7c100b0e60 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c15dff700 1 -- 172.21.15.131:0/757198877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c100b0e60 msgr2=0x7f7c100c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c15dff700 1 --2- 172.21.15.131:0/757198877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c100b0e60 0x7f7c100c9400 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7c10064410 tx=0x7f7c08009840 comp rx=0 tx=0).stop 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c15dff700 1 -- 172.21.15.131:0/757198877 shutdown_connections 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c15dff700 1 --2- 172.21.15.131:0/757198877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c100b0e60 0x7f7c100c9400 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c15dff700 1 --2- 172.21.15.131:0/757198877 >> v2:172.21.15.190:3300/0 conn(0x7f7c100cb320 0x7f7c10069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c15dff700 1 -- 172.21.15.131:0/757198877 >> 172.21.15.131:0/757198877 conn(0x7f7c10068040 msgr2=0x7f7c10069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c15dff700 1 -- 172.21.15.131:0/757198877 shutdown_connections 2023-12-26T22:53:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.525+0000 7f7c15dff700 1 -- 172.21.15.131:0/757198877 wait complete. 2023-12-26T22:53:16.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.527+0000 7f7c15dff700 1 Processor -- start 2023-12-26T22:53:16.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.527+0000 7f7c15dff700 1 -- start start 2023-12-26T22:53:16.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c15dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c1006a1e0 0x7f7c10153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:16.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c15dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c100b0e60 0x7f7c1015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:16.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c15dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c100cb320 0x7f7c100abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c15dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1015ee40 con 0x7f7c1006a1e0 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c15dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1015a720 con 0x7f7c100cb320 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c15dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1015a110 con 0x7f7c100b0e60 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c1006a1e0 0x7f7c10153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c1006a1e0 0x7f7c10153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60322/0 (socket says 172.21.15.131:60322) 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0f7fe700 1 -- 172.21.15.131:0/1906257674 learned_addr learned my addr 172.21.15.131:0/1906257674 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0effd700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c100b0e60 0x7f7c1015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0ffff700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c100cb320 0x7f7c100abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0f7fe700 1 -- 172.21.15.131:0/1906257674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c100b0e60 msgr2=0x7f7c1015e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:16.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0f7fe700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c100b0e60 0x7f7c1015e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0f7fe700 1 -- 172.21.15.131:0/1906257674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c100cb320 msgr2=0x7f7c100abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0f7fe700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c100cb320 0x7f7c100abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.528+0000 7f7c0f7fe700 1 -- 172.21.15.131:0/1906257674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c08009ae0 con 0x7f7c1006a1e0 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.529+0000 7f7c0f7fe700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c1006a1e0 0x7f7c10153e10 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000d860 tx=0x7f7c00008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.529+0000 7f7bf77fe700 1 -- 172.21.15.131:0/1906257674 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00009610 con 0x7f7c1006a1e0 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.529+0000 7f7bf77fe700 1 -- 172.21.15.131:0/1906257674 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c00009770 con 0x7f7c1006a1e0 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.529+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c10154350 con 0x7f7c1006a1e0 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.529+0000 7f7bf77fe700 1 -- 172.21.15.131:0/1906257674 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00009b30 con 0x7f7c1006a1e0 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.529+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1015a3b0 con 0x7f7c1006a1e0 2023-12-26T22:53:16.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.530+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c10159f70 con 0x7f7c1006a1e0 2023-12-26T22:53:16.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.531+0000 7f7bf77fe700 1 -- 172.21.15.131:0/1906257674 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7c0001e460 con 0x7f7c1006a1e0 2023-12-26T22:53:16.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.532+0000 7f7bf77fe700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7bfc066f30 0x7f7bfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:16.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.532+0000 7f7bf77fe700 1 -- 172.21.15.131:0/1906257674 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f7c00084ca0 con 0x7f7c1006a1e0 2023-12-26T22:53:16.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.532+0000 7f7c0effd700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7bfc066f30 0x7f7bfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:16.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.533+0000 7f7c0effd700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7bfc066f30 0x7f7bfc0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7c04007fd0 tx=0x7f7c04007d50 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:16.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.536+0000 7f7bf77fe700 1 -- 172.21.15.131:0/1906257674 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c10159f70 con 0x7f7c1006a1e0 2023-12-26T22:53:16.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:16.749+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7c10159f70 con 0x7f7c1006a1e0 2023-12-26T22:53:17.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.582+0000 7f7bf77fe700 1 -- 172.21.15.131:0/1906257674 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v171) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7c10159f70 con 0x7f7c1006a1e0 2023-12-26T22:53:17.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.584+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7bfc066f30 msgr2=0x7f7bfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.584+0000 7f7c15dff700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7bfc066f30 0x7f7bfc0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7c04007fd0 tx=0x7f7c04007d50 comp rx=0 tx=0).stop 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c1006a1e0 msgr2=0x7f7c10153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c1006a1e0 0x7f7c10153e10 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000d860 tx=0x7f7c00008040 comp rx=0 tx=0).stop 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 shutdown_connections 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c100cb320 0x7f7c100abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7bfc066f30 0x7f7bfc0693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c1006a1e0 0x7f7c10153e10 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 --2- 172.21.15.131:0/1906257674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c100b0e60 0x7f7c1015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 >> 172.21.15.131:0/1906257674 conn(0x7f7c10068040 msgr2=0x7f7c10155c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:17.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 shutdown_connections 2023-12-26T22:53:17.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.585+0000 7f7c15dff700 1 -- 172.21.15.131:0/1906257674 wait complete. 2023-12-26T22:53:17.586 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T22:53:17.604 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T22:53:17.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.794+0000 7f07559ff700 1 Processor -- start 2023-12-26T22:53:17.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.794+0000 7f07559ff700 1 -- start start 2023-12-26T22:53:17.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.795+0000 7f07559ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0750159560 0x7f075015d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.795+0000 7f07559ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0750159930 0x7f07500b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.795+0000 7f07559ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f075006a1a0 0x7f0750068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.795+0000 7f07559ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0750068a80 con 0x7f075006a1a0 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.795+0000 7f07559ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07500cac50 con 0x7f0750159930 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.795+0000 7f07559ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075015da00 con 0x7f0750159560 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.795+0000 7f074f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f075006a1a0 0x7f0750068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.795+0000 7f074f7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f075006a1a0 0x7f0750068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60334/0 (socket says 172.21.15.131:60334) 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.795+0000 7f074f7fe700 1 -- 172.21.15.131:0/2684393606 learned_addr learned my addr 172.21.15.131:0/2684393606 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074effd700 1 --2- 172.21.15.131:0/2684393606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0750159560 0x7f075015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074e7fc700 1 --2- 172.21.15.131:0/2684393606 >> v2:172.21.15.190:3300/0 conn(0x7f0750159930 0x7f07500b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:17.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074f7fe700 1 -- 172.21.15.131:0/2684393606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0750159560 msgr2=0x7f075015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:17.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074f7fe700 1 --2- 172.21.15.131:0/2684393606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0750159560 0x7f075015d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074f7fe700 1 -- 172.21.15.131:0/2684393606 >> v2:172.21.15.190:3300/0 conn(0x7f0750159930 msgr2=0x7f07500b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:17.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074f7fe700 1 --2- 172.21.15.131:0/2684393606 >> v2:172.21.15.190:3300/0 conn(0x7f0750159930 0x7f07500b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074f7fe700 1 -- 172.21.15.131:0/2684393606 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075015dce0 con 0x7f075006a1a0 2023-12-26T22:53:17.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074f7fe700 1 --2- 172.21.15.131:0/2684393606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f075006a1a0 0x7f0750068f10 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f075006e810 tx=0x7f0748009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbce3b581207597 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:17.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074dffb700 1 -- 172.21.15.131:0/2684393606 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074800e070 con 0x7f075006a1a0 2023-12-26T22:53:17.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074dffb700 1 -- 172.21.15.131:0/2684393606 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074800ba50 con 0x7f075006a1a0 2023-12-26T22:53:17.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.796+0000 7f074dffb700 1 -- 172.21.15.131:0/2684393606 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07480092e0 con 0x7f075006a1a0 2023-12-26T22:53:17.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.797+0000 7f07559ff700 1 -- 172.21.15.131:0/2684393606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f075006a1a0 msgr2=0x7f0750068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:17.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.797+0000 7f07559ff700 1 --2- 172.21.15.131:0/2684393606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f075006a1a0 0x7f0750068f10 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f075006e810 tx=0x7f0748009840 comp rx=0 tx=0).stop 2023-12-26T22:53:17.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.797+0000 7f07559ff700 1 -- 172.21.15.131:0/2684393606 shutdown_connections 2023-12-26T22:53:17.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.797+0000 7f07559ff700 1 --2- 172.21.15.131:0/2684393606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f075006a1a0 0x7f0750068f10 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.797+0000 7f07559ff700 1 --2- 172.21.15.131:0/2684393606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0750159560 0x7f075015d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.797+0000 7f07559ff700 1 --2- 172.21.15.131:0/2684393606 >> v2:172.21.15.190:3300/0 conn(0x7f0750159930 0x7f07500b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.797+0000 7f07559ff700 1 -- 172.21.15.131:0/2684393606 >> 172.21.15.131:0/2684393606 conn(0x7f07500ce3d0 msgr2=0x7f07500ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:17.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.797+0000 7f07559ff700 1 -- 172.21.15.131:0/2684393606 shutdown_connections 2023-12-26T22:53:17.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.797+0000 7f07559ff700 1 -- 172.21.15.131:0/2684393606 wait complete. 2023-12-26T22:53:17.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.798+0000 7f07559ff700 1 Processor -- start 2023-12-26T22:53:17.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.798+0000 7f07559ff700 1 -- start start 2023-12-26T22:53:17.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.799+0000 7f07559ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f075006a1a0 0x7f07500c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:17.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.799+0000 7f07559ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0750159560 0x7f075015adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.799+0000 7f07559ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0750159930 0x7f07500b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.799+0000 7f07559ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075015b300 con 0x7f0750159560 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.799+0000 7f07559ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075015b4f0 con 0x7f0750159930 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.799+0000 7f07559ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075015b7d0 con 0x7f075006a1a0 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0750159560 0x7f075015adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0750159560 0x7f075015adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60336/0 (socket says 172.21.15.131:60336) 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074e7fc700 1 -- 172.21.15.131:0/2680614800 learned_addr learned my addr 172.21.15.131:0/2680614800 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074effd700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f075006a1a0 0x7f07500c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074f7fe700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0750159930 0x7f07500b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074e7fc700 1 -- 172.21.15.131:0/2680614800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f075006a1a0 msgr2=0x7f07500c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074e7fc700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f075006a1a0 0x7f07500c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074e7fc700 1 -- 172.21.15.131:0/2680614800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0750159930 msgr2=0x7f07500b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:17.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074e7fc700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0750159930 0x7f07500b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:17.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074e7fc700 1 -- 172.21.15.131:0/2680614800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0748009ae0 con 0x7f0750159560 2023-12-26T22:53:17.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.800+0000 7f074e7fc700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0750159560 0x7f075015adc0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f074400d890 tx=0x7f0744008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:17.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.801+0000 7f0736ffd700 1 -- 172.21.15.131:0/2680614800 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07440125f0 con 0x7f0750159560 2023-12-26T22:53:17.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.801+0000 7f0736ffd700 1 -- 172.21.15.131:0/2680614800 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0744012750 con 0x7f0750159560 2023-12-26T22:53:17.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.801+0000 7f0736ffd700 1 -- 172.21.15.131:0/2680614800 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744012af0 con 0x7f0750159560 2023-12-26T22:53:17.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.801+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075015bab0 con 0x7f0750159560 2023-12-26T22:53:17.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.801+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075015c070 con 0x7f0750159560 2023-12-26T22:53:17.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.802+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07500657e0 con 0x7f0750159560 2023-12-26T22:53:17.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.803+0000 7f0736ffd700 1 -- 172.21.15.131:0/2680614800 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f074401e460 con 0x7f0750159560 2023-12-26T22:53:17.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.803+0000 7f0736ffd700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f073c066f30 0x7f073c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:17.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.804+0000 7f0736ffd700 1 -- 172.21.15.131:0/2680614800 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f07440849c0 con 0x7f0750159560 2023-12-26T22:53:17.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.804+0000 7f074effd700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f073c066f30 0x7f073c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:17.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.804+0000 7f074effd700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f073c066f30 0x7f073c0693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0740006fd0 tx=0x7f0740008040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:17.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:17.807+0000 7f0736ffd700 1 -- 172.21.15.131:0/2680614800 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0744051d10 con 0x7f0750159560 2023-12-26T22:53:18.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.076+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0750068e10 con 0x7f0750159560 2023-12-26T22:53:18.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.077+0000 7f0736ffd700 1 -- 172.21.15.131:0/2680614800 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v41) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0744055330 con 0x7f0750159560 2023-12-26T22:53:18.077 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:18.077 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T22:53:18.077 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T22:53:18.077 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.077 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T22:53:18.077 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T22:53:18.077 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.078 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:53:18.078 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:53:18.078 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:53:18.078 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.078 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.078 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.078 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T22:53:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T22:53:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:53:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:53:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:53:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.079 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T22:53:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T22:53:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:53:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:53:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:53:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.081 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.082 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.082 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.082 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T22:53:18.082 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T22:53:18.082 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.082 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:53:18.082 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:53:18.082 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.082 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.083 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.084 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T22:53:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T22:53:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:53:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:53:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.085 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.086 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T22:53:18.086 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T22:53:18.086 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.086 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:53:18.086 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:53:18.086 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.086 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.086 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.086 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.087 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T22:53:18.088 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.089 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T22:53:18.090 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T22:53:18.091 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T22:53:18.092 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.093 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:18.094 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T22:53:18.095 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T22:53:18.095 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T22:53:18.095 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T22:53:18.095 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T22:53:18.095 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T22:53:18.095 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.095 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:18.095 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T22:53:18.095 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T22:53:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f073c066f30 msgr2=0x7f073c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f073c066f30 0x7f073c0693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0740006fd0 tx=0x7f0740008040 comp rx=0 tx=0).stop 2023-12-26T22:53:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0750159560 msgr2=0x7f075015adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0750159560 0x7f075015adc0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f074400d890 tx=0x7f0744008040 comp rx=0 tx=0).stop 2023-12-26T22:53:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 shutdown_connections 2023-12-26T22:53:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0750159930 0x7f07500b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f075006a1a0 0x7f07500c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f073c066f30 0x7f073c0693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 --2- 172.21.15.131:0/2680614800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0750159560 0x7f075015adc0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 >> 172.21.15.131:0/2680614800 conn(0x7f07500ce3d0 msgr2=0x7f07500b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 shutdown_connections 2023-12-26T22:53:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.079+0000 7f07559ff700 1 -- 172.21.15.131:0/2680614800 wait complete. 2023-12-26T22:53:18.099 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:53:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.290+0000 7fc64b9ff700 1 Processor -- start 2023-12-26T22:53:18.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.290+0000 7fc64b9ff700 1 -- start start 2023-12-26T22:53:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.291+0000 7fc64b9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc6440cb320 0x7fc644069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.291+0000 7fc64b9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc644156940 con 0x7fc6440b0e60 2023-12-26T22:53:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.291+0000 7fc64b9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6440ae500 con 0x7fc64406a1e0 2023-12-26T22:53:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.291+0000 7fc64b9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6440c8cc0 con 0x7fc6440cb320 2023-12-26T22:53:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.291+0000 7fc64979b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc6440cb320 0x7fc644069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.291+0000 7fc64979b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc6440cb320 0x7fc644069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60350/0 (socket says 172.21.15.131:60350) 2023-12-26T22:53:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.291+0000 7fc64979b700 1 -- 172.21.15.131:0/3781713571 learned_addr learned my addr 172.21.15.131:0/3781713571 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.292+0000 7fc64979b700 1 -- 172.21.15.131:0/3781713571 >> v1:172.21.15.190:6789/0 conn(0x7fc64406a1e0 legacy=0x7fc6440b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.292+0000 7fc64979b700 1 -- 172.21.15.131:0/3781713571 >> v1:172.21.15.131:6789/0 conn(0x7fc6440b0e60 legacy=0x7fc6440c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.292+0000 7fc64979b700 1 -- 172.21.15.131:0/3781713571 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440b0280 con 0x7fc6440cb320 2023-12-26T22:53:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.292+0000 7fc64979b700 1 --2- 172.21.15.131:0/3781713571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6440cb320 0x7fc644069ca0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc6400099b0 tx=0x7fc640009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa68f124f5deaf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.292+0000 7fc63bfff700 1 -- 172.21.15.131:0/3781713571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000b860 con 0x7fc6440cb320 2023-12-26T22:53:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.292+0000 7fc63bfff700 1 -- 172.21.15.131:0/3781713571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64000b9c0 con 0x7fc6440cb320 2023-12-26T22:53:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.292+0000 7fc63bfff700 1 -- 172.21.15.131:0/3781713571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000bd60 con 0x7fc6440cb320 2023-12-26T22:53:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.293+0000 7fc64b9ff700 1 -- 172.21.15.131:0/3781713571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6440cb320 msgr2=0x7fc644069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.293+0000 7fc64b9ff700 1 --2- 172.21.15.131:0/3781713571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6440cb320 0x7fc644069ca0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc6400099b0 tx=0x7fc640009250 comp rx=0 tx=0).stop 2023-12-26T22:53:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.293+0000 7fc64b9ff700 1 -- 172.21.15.131:0/3781713571 shutdown_connections 2023-12-26T22:53:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.293+0000 7fc64b9ff700 1 --2- 172.21.15.131:0/3781713571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6440cb320 0x7fc644069ca0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.293+0000 7fc64b9ff700 1 -- 172.21.15.131:0/3781713571 >> 172.21.15.131:0/3781713571 conn(0x7fc644068040 msgr2=0x7fc644069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.293+0000 7fc64b9ff700 1 -- 172.21.15.131:0/3781713571 shutdown_connections 2023-12-26T22:53:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.293+0000 7fc64b9ff700 1 -- 172.21.15.131:0/3781713571 wait complete. 2023-12-26T22:53:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.294+0000 7fc64b9ff700 1 Processor -- start 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.295+0000 7fc64b9ff700 1 -- start start 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.295+0000 7fc64b9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc64406a1e0 0x7fc644153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.295+0000 7fc64b9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6440b0e60 0x7fc64414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.295+0000 7fc64b9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6440cb320 0x7fc644159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.295+0000 7fc64b9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc644154300 con 0x7fc64406a1e0 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.295+0000 7fc64b9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc644152f70 con 0x7fc6440b0e60 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.295+0000 7fc64b9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc644153250 con 0x7fc6440cb320 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc64979b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc64406a1e0 0x7fc644153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc64979b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc64406a1e0 0x7fc644153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60362/0 (socket says 172.21.15.131:60362) 2023-12-26T22:53:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc64979b700 1 -- 172.21.15.131:0/2619805689 learned_addr learned my addr 172.21.15.131:0/2619805689 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc649f9c700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6440cb320 0x7fc644159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc648f9a700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6440b0e60 0x7fc64414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc64979b700 1 -- 172.21.15.131:0/2619805689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6440cb320 msgr2=0x7fc644159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc64979b700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6440cb320 0x7fc644159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc64979b700 1 -- 172.21.15.131:0/2619805689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6440b0e60 msgr2=0x7fc64414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc64979b700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6440b0e60 0x7fc64414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc64979b700 1 -- 172.21.15.131:0/2619805689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6400094f0 con 0x7fc64406a1e0 2023-12-26T22:53:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.296+0000 7fc64979b700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc64406a1e0 0x7fc644153dc0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc644159f40 tx=0x7fc640017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.297+0000 7fc6397fa700 1 -- 172.21.15.131:0/2619805689 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640011ea0 con 0x7fc64406a1e0 2023-12-26T22:53:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.297+0000 7fc6397fa700 1 -- 172.21.15.131:0/2619805689 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64001a630 con 0x7fc64406a1e0 2023-12-26T22:53:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.297+0000 7fc6397fa700 1 -- 172.21.15.131:0/2619805689 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64001a9f0 con 0x7fc64406a1e0 2023-12-26T22:53:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.297+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc644153530 con 0x7fc64406a1e0 2023-12-26T22:53:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.297+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64414ebd0 con 0x7fc64406a1e0 2023-12-26T22:53:18.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.298+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6440657e0 con 0x7fc64406a1e0 2023-12-26T22:53:18.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.299+0000 7fc6397fa700 1 -- 172.21.15.131:0/2619805689 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc640026450 con 0x7fc64406a1e0 2023-12-26T22:53:18.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.300+0000 7fc6397fa700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc628066f30 0x7fc6280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.300+0000 7fc6397fa700 1 -- 172.21.15.131:0/2619805689 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5317+0+0 (secure 0 0 0) 0x7fc640085f90 con 0x7fc64406a1e0 2023-12-26T22:53:18.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.300+0000 7fc648f9a700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc628066f30 0x7fc6280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.301+0000 7fc648f9a700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc628066f30 0x7fc6280693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc64415f310 tx=0x7fc634009040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:18.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.304+0000 7fc6397fa700 1 -- 172.21.15.131:0/2619805689 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6400b75a0 con 0x7fc64406a1e0 2023-12-26T22:53:18.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.502+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6441544a0 con 0x7fc64406a1e0 2023-12-26T22:53:18.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.503+0000 7fc6397fa700 1 -- 172.21.15.131:0/2619805689 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+12904 (secure 0 0 0) 0x7fc640015090 con 0x7fc64406a1e0 2023-12-26T22:53:18.503 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:18.503 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":171,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:53:17.579623+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:53:18.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.505+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc628066f30 msgr2=0x7fc6280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.505+0000 7fc64b9ff700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc628066f30 0x7fc6280693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc64415f310 tx=0x7fc634009040 comp rx=0 tx=0).stop 2023-12-26T22:53:18.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc64406a1e0 msgr2=0x7fc644153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc64406a1e0 0x7fc644153dc0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc644159f40 tx=0x7fc640017000 comp rx=0 tx=0).stop 2023-12-26T22:53:18.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 shutdown_connections 2023-12-26T22:53:18.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6440cb320 0x7fc644159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fc628066f30 0x7fc6280693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc64406a1e0 0x7fc644153dc0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 --2- 172.21.15.131:0/2619805689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6440b0e60 0x7fc64414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 >> 172.21.15.131:0/2619805689 conn(0x7fc644068040 msgr2=0x7fc644152180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 shutdown_connections 2023-12-26T22:53:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.506+0000 7fc64b9ff700 1 -- 172.21.15.131:0/2619805689 wait complete. 2023-12-26T22:53:18.524 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T22:53:18.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T22:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.716+0000 7facfe3ff700 1 Processor -- start 2023-12-26T22:53:18.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.716+0000 7facfe3ff700 1 -- start start 2023-12-26T22:53:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facfe3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7facf806bfd0 0x7facf80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facfe3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf80cc100 0x7facf80cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facfe3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf80686c0 con 0x7facf813c510 2023-12-26T22:53:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facfe3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf8069520 con 0x7facf806bfd0 2023-12-26T22:53:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facfe3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf8068910 con 0x7facf80cc100 2023-12-26T22:53:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facf77fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7facf806bfd0 0x7facf80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facfc99c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf80cc100 0x7facf80cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facfc99c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf80cc100 0x7facf80cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:43490/0 (socket says 172.21.15.131:43490) 2023-12-26T22:53:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facfc99c700 1 -- 172.21.15.131:0/3391797655 learned_addr learned my addr 172.21.15.131:0/3391797655 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:18.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facf77fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7facf806bfd0 0x7facf80c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:36588/0 (socket says 172.21.15.131:36588) 2023-12-26T22:53:18.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.718+0000 7facf7fff700 1 --1- 172.21.15.131:0/3391797655 >> v1:172.21.15.190:6789/0 conn(0x7facf813c510 0x7facf806b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:33668/0 (socket says 172.21.15.131:33668) 2023-12-26T22:53:18.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.719+0000 7facfc99c700 1 -- 172.21.15.131:0/3391797655 >> v2:172.21.15.190:3300/0 conn(0x7facf806bfd0 msgr2=0x7facf80c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:18.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.719+0000 7facfc99c700 1 --2- 172.21.15.131:0/3391797655 >> v2:172.21.15.190:3300/0 conn(0x7facf806bfd0 0x7facf80c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.719+0000 7facfc99c700 1 -- 172.21.15.131:0/3391797655 >> v1:172.21.15.190:6789/0 conn(0x7facf813c510 legacy=0x7facf806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.719+0000 7facfc99c700 1 -- 172.21.15.131:0/3391797655 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8068bf0 con 0x7facf80cc100 2023-12-26T22:53:18.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.719+0000 7facf77fe700 1 --2- 172.21.15.131:0/3391797655 >> v2:172.21.15.190:3300/0 conn(0x7facf806bfd0 0x7facf80c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:18.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.719+0000 7facfc99c700 1 --2- 172.21.15.131:0/3391797655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf80cc100 0x7facf80cc560 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7facf0009f20 tx=0x7facf0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e242ced39a528e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:18.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.719+0000 7facf6ffd700 1 -- 172.21.15.131:0/3391797655 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf000e070 con 0x7facf80cc100 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.719+0000 7facf6ffd700 1 -- 172.21.15.131:0/3391797655 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf000b9d0 con 0x7facf80cc100 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.719+0000 7facf6ffd700 1 -- 172.21.15.131:0/3391797655 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00092e0 con 0x7facf80cc100 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.720+0000 7facfe3ff700 1 -- 172.21.15.131:0/3391797655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf80cc100 msgr2=0x7facf80cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.720+0000 7facfe3ff700 1 --2- 172.21.15.131:0/3391797655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf80cc100 0x7facf80cc560 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7facf0009f20 tx=0x7facf0009840 comp rx=0 tx=0).stop 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.720+0000 7facfe3ff700 1 -- 172.21.15.131:0/3391797655 shutdown_connections 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.720+0000 7facfe3ff700 1 --2- 172.21.15.131:0/3391797655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf80cc100 0x7facf80cc560 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.720+0000 7facfe3ff700 1 --2- 172.21.15.131:0/3391797655 >> v2:172.21.15.190:3300/0 conn(0x7facf806bfd0 0x7facf80c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.720+0000 7facfe3ff700 1 -- 172.21.15.131:0/3391797655 >> 172.21.15.131:0/3391797655 conn(0x7facf80ce3e0 msgr2=0x7facf80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.720+0000 7facfe3ff700 1 -- 172.21.15.131:0/3391797655 shutdown_connections 2023-12-26T22:53:18.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.720+0000 7facfe3ff700 1 -- 172.21.15.131:0/3391797655 wait complete. 2023-12-26T22:53:18.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.721+0000 7facfe3ff700 1 Processor -- start 2023-12-26T22:53:18.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.721+0000 7facfe3ff700 1 -- start start 2023-12-26T22:53:18.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.722+0000 7facfe3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf806bfd0 0x7facf813be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.722+0000 7facfe3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf80cc100 0x7facf80c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.722+0000 7facfe3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf813c510 0x7facf80c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.722+0000 7facfe3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf813c3b0 con 0x7facf80cc100 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.722+0000 7facfe3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80c7eb0 con 0x7facf813c510 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.722+0000 7facfe3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf80c72a0 con 0x7facf806bfd0 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf80cc100 0x7facf80c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf80cc100 0x7facf80c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60368/0 (socket says 172.21.15.131:60368) 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf77fe700 1 -- 172.21.15.131:0/2550676840 learned_addr learned my addr 172.21.15.131:0/2550676840 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facfc99c700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf813c510 0x7facf80c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf7fff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf806bfd0 0x7facf813be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf77fe700 1 -- 172.21.15.131:0/2550676840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf806bfd0 msgr2=0x7facf813be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf77fe700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf806bfd0 0x7facf813be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf77fe700 1 -- 172.21.15.131:0/2550676840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf813c510 msgr2=0x7facf80c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf77fe700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf813c510 0x7facf80c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf77fe700 1 -- 172.21.15.131:0/2550676840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec009420 con 0x7facf80cc100 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facfc99c700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf813c510 0x7facf80c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.723+0000 7facf77fe700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf80cc100 0x7facf80c68c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7facec00ec80 tx=0x7facec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.724+0000 7facdffff700 1 -- 172.21.15.131:0/2550676840 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00c850 con 0x7facf80cc100 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.724+0000 7facdffff700 1 -- 172.21.15.131:0/2550676840 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec00c9b0 con 0x7facf80cc100 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.724+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf0009ae0 con 0x7facf80cc100 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.724+0000 7facdffff700 1 -- 172.21.15.131:0/2550676840 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00cd50 con 0x7facf80cc100 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.724+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf80c7860 con 0x7facf80cc100 2023-12-26T22:53:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.725+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf80657e0 con 0x7facf80cc100 2023-12-26T22:53:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.726+0000 7facdffff700 1 -- 172.21.15.131:0/2550676840 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7facec013680 con 0x7facf80cc100 2023-12-26T22:53:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.726+0000 7facdffff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7face4066f30 0x7face40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.726+0000 7facdffff700 1 -- 172.21.15.131:0/2550676840 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5317+0+0 (secure 0 0 0) 0x7facec084440 con 0x7facf80cc100 2023-12-26T22:53:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.727+0000 7facf7fff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7face4066f30 0x7face40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.727+0000 7facf7fff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7face4066f30 0x7face40693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7face8007060 tx=0x7face8007ea0 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:18.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.730+0000 7facdffff700 1 -- 172.21.15.131:0/2550676840 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facec051790 con 0x7facf80cc100 2023-12-26T22:53:18.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.922+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facf8068ed0 con 0x7facf80cc100 2023-12-26T22:53:18.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.923+0000 7facdffff700 1 -- 172.21.15.131:0/2550676840 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7facec082e60 con 0x7facf80cc100 2023-12-26T22:53:18.923 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T22:53:18.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7face4066f30 msgr2=0x7face40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7face4066f30 0x7face40693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7face8007060 tx=0x7face8007ea0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf80cc100 msgr2=0x7facf80c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:18.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf80cc100 0x7facf80c68c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7facec00ec80 tx=0x7facec0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 shutdown_connections 2023-12-26T22:53:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf813c510 0x7facf80c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf806bfd0 0x7facf813be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7face4066f30 0x7face40693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 --2- 172.21.15.131:0/2550676840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf80cc100 0x7facf80c68c0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 >> 172.21.15.131:0/2550676840 conn(0x7facf80ce3e0 msgr2=0x7facf806cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 shutdown_connections 2023-12-26T22:53:18.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:18.925+0000 7facfe3ff700 1 -- 172.21.15.131:0/2550676840 wait complete. 2023-12-26T22:53:18.944 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T22:53:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.134+0000 7f01a99ff700 1 Processor -- start 2023-12-26T22:53:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.134+0000 7f01a99ff700 1 -- start start 2023-12-26T22:53:19.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.135+0000 7f01a99ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f01a40c8820 0x7f01a40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.135+0000 7f01a99ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f01a40c8bf0 0x7f01a40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.135+0000 7f01a99ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a4069d10 0x7f01a40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.135+0000 7f01a99ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a4157a80 con 0x7f01a40c8bf0 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.135+0000 7f01a99ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a40cac50 con 0x7f01a40c8820 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.135+0000 7f01a99ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40caf30 con 0x7f01a4069d10 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.135+0000 7f01a27fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f01a40c8bf0 0x7f01a40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a27fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f01a40c8bf0 0x7f01a40b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60376/0 (socket says 172.21.15.131:60376) 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a27fc700 1 -- 172.21.15.131:0/3251239843 learned_addr learned my addr 172.21.15.131:0/3251239843 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a37fe700 1 --2- 172.21.15.131:0/3251239843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a4069d10 0x7f01a40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a2ffd700 1 --2- 172.21.15.131:0/3251239843 >> v2:172.21.15.190:3300/0 conn(0x7f01a40c8820 0x7f01a40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a27fc700 1 -- 172.21.15.131:0/3251239843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a4069d10 msgr2=0x7f01a40b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a27fc700 1 --2- 172.21.15.131:0/3251239843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a4069d10 0x7f01a40b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a27fc700 1 -- 172.21.15.131:0/3251239843 >> v2:172.21.15.190:3300/0 conn(0x7f01a40c8820 msgr2=0x7f01a40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a27fc700 1 --2- 172.21.15.131:0/3251239843 >> v2:172.21.15.190:3300/0 conn(0x7f01a40c8820 0x7f01a40ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a27fc700 1 -- 172.21.15.131:0/3251239843 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a40cb210 con 0x7f01a40c8bf0 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a37fe700 1 --2- 172.21.15.131:0/3251239843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a4069d10 0x7f01a40b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a27fc700 1 --2- 172.21.15.131:0/3251239843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 0x7f01a40b12e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f01a4064ea0 tx=0x7f0198009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4b7f86a65370 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.136+0000 7f01a1ffb700 1 -- 172.21.15.131:0/3251239843 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198009e70 con 0x7f01a40c8bf0 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a1ffb700 1 -- 172.21.15.131:0/3251239843 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019800b860 con 0x7f01a40c8bf0 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a1ffb700 1 -- 172.21.15.131:0/3251239843 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800bc00 con 0x7f01a40c8bf0 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a99ff700 1 -- 172.21.15.131:0/3251239843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 msgr2=0x7f01a40b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a99ff700 1 --2- 172.21.15.131:0/3251239843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 0x7f01a40b12e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f01a4064ea0 tx=0x7f0198009250 comp rx=0 tx=0).stop 2023-12-26T22:53:19.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a99ff700 1 -- 172.21.15.131:0/3251239843 shutdown_connections 2023-12-26T22:53:19.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a99ff700 1 --2- 172.21.15.131:0/3251239843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a4069d10 0x7f01a40b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a99ff700 1 --2- 172.21.15.131:0/3251239843 >> v2:172.21.15.190:3300/0 conn(0x7f01a40c8820 0x7f01a40ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a99ff700 1 --2- 172.21.15.131:0/3251239843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 0x7f01a40b12e0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a99ff700 1 -- 172.21.15.131:0/3251239843 >> 172.21.15.131:0/3251239843 conn(0x7f01a4068040 msgr2=0x7f01a4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:19.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a99ff700 1 -- 172.21.15.131:0/3251239843 shutdown_connections 2023-12-26T22:53:19.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.137+0000 7f01a99ff700 1 -- 172.21.15.131:0/3251239843 wait complete. 2023-12-26T22:53:19.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.139+0000 7f01a99ff700 1 Processor -- start 2023-12-26T22:53:19.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.139+0000 7f01a99ff700 1 -- start start 2023-12-26T22:53:19.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a99ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01a4069d10 0x7f01a40a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a99ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a40c8820 0x7f01a4154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a99ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 0x7f01a40b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a99ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40b0640 con 0x7f01a40c8bf0 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a99ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40a6750 con 0x7f01a4069d10 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a99ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4154ad0 con 0x7f01a40c8820 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 0x7f01a40b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 0x7f01a40b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60388/0 (socket says 172.21.15.131:60388) 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a37fe700 1 -- 172.21.15.131:0/2273115024 learned_addr learned my addr 172.21.15.131:0/2273115024 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a2ffd700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01a4069d10 0x7f01a40a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a37fe700 1 -- 172.21.15.131:0/2273115024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a40c8820 msgr2=0x7f01a4154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a37fe700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a40c8820 0x7f01a4154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a37fe700 1 -- 172.21.15.131:0/2273115024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01a4069d10 msgr2=0x7f01a40a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a37fe700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01a4069d10 0x7f01a40a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a37fe700 1 -- 172.21.15.131:0/2273115024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01980094f0 con 0x7f01a40c8bf0 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.140+0000 7f01a2ffd700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01a4069d10 0x7f01a40a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.141+0000 7f01a37fe700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 0x7f01a40b0ad0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f019c009fc0 tx=0x7f019c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.141+0000 7f018affd700 1 -- 172.21.15.131:0/2273115024 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c00dd40 con 0x7f01a40c8bf0 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.141+0000 7f018affd700 1 -- 172.21.15.131:0/2273115024 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c00f040 con 0x7f01a40c8bf0 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.141+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a40b0010 con 0x7f01a40c8bf0 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.141+0000 7f018affd700 1 -- 172.21.15.131:0/2273115024 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c015690 con 0x7f01a40c8bf0 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.141+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a40b02d0 con 0x7f01a40c8bf0 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.142+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a40657e0 con 0x7f01a40c8bf0 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.143+0000 7f018affd700 1 -- 172.21.15.131:0/2273115024 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f019c01e3b0 con 0x7f01a40c8bf0 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.144+0000 7f018affd700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0190066f30 0x7f01900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.144+0000 7f018affd700 1 -- 172.21.15.131:0/2273115024 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f019c084900 con 0x7f01a40c8bf0 2023-12-26T22:53:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.144+0000 7f01a2ffd700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0190066f30 0x7f01900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.144+0000 7f01a2ffd700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0190066f30 0x7f01900693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01940093f0 tx=0x7f0194008040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:19.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.147+0000 7f018affd700 1 -- 172.21.15.131:0/2273115024 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f019c051cf0 con 0x7f01a40c8bf0 2023-12-26T22:53:19.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.341+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f01a40b0780 con 0x7f01a40c8bf0 2023-12-26T22:53:19.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.341+0000 7f018affd700 1 -- 172.21.15.131:0/2273115024 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v171) v1 ==== 84+0+65 (secure 0 0 0) 0x7f019c01a090 con 0x7f01a40c8bf0 2023-12-26T22:53:19.341 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:19.341 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T22:53:19.342 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:19.342 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T22:53:19.342 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T22:53:19.342 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:19.342 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T22:53:19.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0190066f30 msgr2=0x7f01900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:19.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0190066f30 0x7f01900693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01940093f0 tx=0x7f0194008040 comp rx=0 tx=0).stop 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 msgr2=0x7f01a40b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 0x7f01a40b0ad0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f019c009fc0 tx=0x7f019c0099b0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 shutdown_connections 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f0190066f30 0x7f01900693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01a40c8bf0 0x7f01a40b0ad0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01a4069d10 0x7f01a40a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 --2- 172.21.15.131:0/2273115024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01a40c8820 0x7f01a4154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.343+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 >> 172.21.15.131:0/2273115024 conn(0x7f01a4068040 msgr2=0x7f01a40ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.344+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 shutdown_connections 2023-12-26T22:53:19.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.344+0000 7f01a99ff700 1 -- 172.21.15.131:0/2273115024 wait complete. 2023-12-26T22:53:19.362 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T22:53:19.362 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T22:53:19.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.556+0000 7f64517ff700 1 Processor -- start 2023-12-26T22:53:19.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.556+0000 7f64517ff700 1 -- start start 2023-12-26T22:53:19.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.557+0000 7f64517ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c1597c0 0x7f644c0abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.557+0000 7f64517ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f644c069e60 0x7f644c06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.557+0000 7f64517ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c067ff0 con 0x7f644c1593f0 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.557+0000 7f64517ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644c068480 con 0x7f644c069e60 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.557+0000 7f64517ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c06a820 con 0x7f644c1597c0 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f644affd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f644c1593f0 0x7f644c0aa270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40314/0 (socket says 172.21.15.131:40314) 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f644affd700 1 -- 172.21.15.131:0/2309977432 learned_addr learned my addr 172.21.15.131:0/2309977432 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f644b7fe700 1 --2- 172.21.15.131:0/2309977432 >> v2:172.21.15.190:3300/0 conn(0x7f644c069e60 0x7f644c06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f644a7fc700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c1597c0 0x7f644c0abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030271521 0 0) 0x7f644c067ff0 con 0x7f644c1593f0 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6434003570 con 0x7f644c1593f0 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 238409558 0 0) 0x7f6434003570 con 0x7f644c1593f0 2023-12-26T22:53:19.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c1597c0 msgr2=0x7f644c0abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c1597c0 0x7f644c0abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 >> v2:172.21.15.190:3300/0 conn(0x7f644c069e60 msgr2=0x7f644c06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 --2- 172.21.15.131:0/2309977432 >> v2:172.21.15.190:3300/0 conn(0x7f644c069e60 0x7f644c06a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0ac350 con 0x7f644c1593f0 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f643c001470 con 0x7f644c1593f0 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 >> v1:172.21.15.131:6789/0 conn(0x7f644c1593f0 legacy=0x7f644c0aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c1597c0 0x7f6434003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f6434004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434004d50 0x7f64340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c068480 con 0x7f6434003fe0 2023-12-26T22:53:19.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c06a820 con 0x7f6434004d50 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.558+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643400ba90 con 0x7f644c1597c0 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644b7fe700 1 --2- 172.21.15.131:0/2309977432 >> v2:172.21.15.190:3300/0 conn(0x7f644c069e60 0x7f644c06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644a7fc700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f6434004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644affd700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c1597c0 0x7f6434003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644b7fe700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434004d50 0x7f64340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644a7fc700 1 -- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c1597c0 msgr2=0x7f6434003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644a7fc700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c1597c0 0x7f6434003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644a7fc700 1 -- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434004d50 msgr2=0x7f64340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644a7fc700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434004d50 0x7f64340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644a7fc700 1 -- 172.21.15.131:0/2309977432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434003570 con 0x7f6434003fe0 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644b7fe700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434004d50 0x7f64340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.559+0000 7f644a7fc700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f6434004810 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6440009c00 tx=0x7f64400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6c04c769abd4ec server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644000e6b0 con 0x7f6434003fe0 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644000e810 con 0x7f6434003fe0 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f6449ffb700 1 -- 172.21.15.131:0/2309977432 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644000ebd0 con 0x7f6434003fe0 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f64517ff700 1 -- 172.21.15.131:0/2309977432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 msgr2=0x7f6434004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f64517ff700 1 --2- 172.21.15.131:0/2309977432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f6434004810 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6440009c00 tx=0x7f64400091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f644b7fe700 1 -- 172.21.15.131:0/2309977432 reap_dead start 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f64517ff700 1 -- 172.21.15.131:0/2309977432 shutdown_connections 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f64517ff700 1 -- 172.21.15.131:0/2309977432 >> 172.21.15.131:0/2309977432 conn(0x7f644c06b710 msgr2=0x7f644c06bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:19.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f64517ff700 1 -- 172.21.15.131:0/2309977432 shutdown_connections 2023-12-26T22:53:19.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.560+0000 7f64517ff700 1 -- 172.21.15.131:0/2309977432 wait complete. 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.562+0000 7f64517ff700 1 Processor -- start 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.562+0000 7f64517ff700 1 -- start start 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f64517ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f644c0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f64517ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f644c14fa10 0x7f644c13e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f64517ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c0ac4e0 0x7f644c0ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f64517ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0acea0 con 0x7f6434003fe0 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f64517ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c150a00 con 0x7f644c14fa10 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f64517ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0b0420 con 0x7f644c0ac4e0 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f644c0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f644c0afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60410/0 (socket says 172.21.15.131:60410) 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644affd700 1 -- 172.21.15.131:0/554920113 learned_addr learned my addr 172.21.15.131:0/554920113 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644a7fc700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f644c14fa10 0x7f644c13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644b7fe700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c0ac4e0 0x7f644c0ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644affd700 1 -- 172.21.15.131:0/554920113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c0ac4e0 msgr2=0x7f644c0ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644affd700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c0ac4e0 0x7f644c0ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644affd700 1 -- 172.21.15.131:0/554920113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f644c14fa10 msgr2=0x7f644c13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644affd700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f644c14fa10 0x7f644c13e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.563+0000 7f644affd700 1 -- 172.21.15.131:0/554920113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c13ea20 con 0x7f6434003fe0 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.564+0000 7f644affd700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f644c0afc30 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f643c00eb70 tx=0x7f644c0675b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.564+0000 7f6432ffd700 1 -- 172.21.15.131:0/554920113 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c005e70 con 0x7f6434003fe0 2023-12-26T22:53:19.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.564+0000 7f6432ffd700 1 -- 172.21.15.131:0/554920113 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c00c5d0 con 0x7f6434003fe0 2023-12-26T22:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.564+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c150160 con 0x7f6434003fe0 2023-12-26T22:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.564+0000 7f6432ffd700 1 -- 172.21.15.131:0/554920113 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00c9b0 con 0x7f6434003fe0 2023-12-26T22:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.564+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c150720 con 0x7f6434003fe0 2023-12-26T22:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.565+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c0717f0 con 0x7f6434003fe0 2023-12-26T22:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.566+0000 7f6432ffd700 1 -- 172.21.15.131:0/554920113 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f643c00cd20 con 0x7f6434003fe0 2023-12-26T22:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.567+0000 7f6432ffd700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6438066f30 0x7f64380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.567+0000 7f6432ffd700 1 -- 172.21.15.131:0/554920113 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f643c084080 con 0x7f6434003fe0 2023-12-26T22:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.567+0000 7f644a7fc700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6438066f30 0x7f64380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.568+0000 7f644a7fc700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6438066f30 0x7f64380693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f64400091e0 tx=0x7f644001a000 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:19.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.571+0000 7f6432ffd700 1 -- 172.21.15.131:0/554920113 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f643c051470 con 0x7f6434003fe0 2023-12-26T22:53:19.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:19.763+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f644c0af8c0 con 0x7f6434003fe0 2023-12-26T22:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.164+0000 7f6432ffd700 1 -- 172.21.15.131:0/554920113 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v172) v1 ==== 138+0+0 (secure 0 0 0) 0x7f643c054a90 con 0x7f6434003fe0 2023-12-26T22:53:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6438066f30 msgr2=0x7f64380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6438066f30 0x7f64380693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f64400091e0 tx=0x7f644001a000 comp rx=0 tx=0).stop 2023-12-26T22:53:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 msgr2=0x7f644c0afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f644c0afc30 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f643c00eb70 tx=0x7f644c0675b0 comp rx=0 tx=0).stop 2023-12-26T22:53:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 shutdown_connections 2023-12-26T22:53:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f644c0ac4e0 0x7f644c0ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6438066f30 0x7f64380693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434003fe0 0x7f644c0afc30 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:20.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 --2- 172.21.15.131:0/554920113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f644c14fa10 0x7f644c13e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:20.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 >> 172.21.15.131:0/554920113 conn(0x7f644c06b710 msgr2=0x7f644c151760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:20.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 shutdown_connections 2023-12-26T22:53:20.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.167+0000 7f64517ff700 1 -- 172.21.15.131:0/554920113 wait complete. 2023-12-26T22:53:20.168 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T22:53:20.189 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T22:53:20.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.385+0000 7f09e29ff700 1 Processor -- start 2023-12-26T22:53:20.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.385+0000 7f09e29ff700 1 -- start start 2023-12-26T22:53:20.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.386+0000 7f09e29ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09dc069620 0x7f09dc0699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:20.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.386+0000 7f09e29ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09dc157b30 con 0x7f09dc06dd90 2023-12-26T22:53:20.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.386+0000 7f09e29ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09dc06f450 con 0x7f09dc06e230 2023-12-26T22:53:20.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.386+0000 7f09e29ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09dc06e650 con 0x7f09dc069620 2023-12-26T22:53:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.386+0000 7f09dbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09dc069620 0x7f09dc0699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.386+0000 7f09dbfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09dc069620 0x7f09dc0699f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60426/0 (socket says 172.21.15.131:60426) 2023-12-26T22:53:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.386+0000 7f09dbfff700 1 -- 172.21.15.131:0/997549653 learned_addr learned my addr 172.21.15.131:0/997549653 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.386+0000 7f09db7fe700 1 --1- 172.21.15.131:0/997549653 >> v1:172.21.15.131:6789/0 conn(0x7f09dc06dd90 0x7f09dc0ad9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40316/0 (socket says 172.21.15.131:40316) 2023-12-26T22:53:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.387+0000 7f09dbfff700 1 -- 172.21.15.131:0/997549653 >> v1:172.21.15.190:6789/0 conn(0x7f09dc06e230 legacy=0x7f09dc0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.387+0000 7f09dbfff700 1 -- 172.21.15.131:0/997549653 >> v1:172.21.15.131:6789/0 conn(0x7f09dc06dd90 legacy=0x7f09dc0ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.387+0000 7f09dbfff700 1 -- 172.21.15.131:0/997549653 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc06f8e0 con 0x7f09dc069620 2023-12-26T22:53:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.387+0000 7f09dbfff700 1 --2- 172.21.15.131:0/997549653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc069620 0x7f09dc0699f0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0099b0 tx=0x7f09cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f7aa7d5dd83c50 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.387+0000 7f09daffd700 1 -- 172.21.15.131:0/997549653 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00b860 con 0x7f09dc069620 2023-12-26T22:53:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.387+0000 7f09daffd700 1 -- 172.21.15.131:0/997549653 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc00b9c0 con 0x7f09dc069620 2023-12-26T22:53:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.387+0000 7f09daffd700 1 -- 172.21.15.131:0/997549653 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00bd60 con 0x7f09dc069620 2023-12-26T22:53:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.388+0000 7f09e29ff700 1 -- 172.21.15.131:0/997549653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc069620 msgr2=0x7f09dc0699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.388+0000 7f09e29ff700 1 --2- 172.21.15.131:0/997549653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc069620 0x7f09dc0699f0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0099b0 tx=0x7f09cc009250 comp rx=0 tx=0).stop 2023-12-26T22:53:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.388+0000 7f09e29ff700 1 -- 172.21.15.131:0/997549653 shutdown_connections 2023-12-26T22:53:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.388+0000 7f09e29ff700 1 --2- 172.21.15.131:0/997549653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc069620 0x7f09dc0699f0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.388+0000 7f09e29ff700 1 -- 172.21.15.131:0/997549653 >> 172.21.15.131:0/997549653 conn(0x7f09dc068040 msgr2=0x7f09dc0cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.388+0000 7f09e29ff700 1 -- 172.21.15.131:0/997549653 shutdown_connections 2023-12-26T22:53:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.388+0000 7f09e29ff700 1 -- 172.21.15.131:0/997549653 wait complete. 2023-12-26T22:53:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.389+0000 7f09e29ff700 1 Processor -- start 2023-12-26T22:53:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.389+0000 7f09e29ff700 1 -- start start 2023-12-26T22:53:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.390+0000 7f09e29ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09dc069620 0x7f09dc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.390+0000 7f09e29ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc06dd90 0x7f09dc148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.390+0000 7f09e29ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09dc06e230 0x7f09dc15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.390+0000 7f09e29ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc16dd10 con 0x7f09dc06dd90 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.390+0000 7f09e29ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc16d100 con 0x7f09dc069620 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.390+0000 7f09e29ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc16d3e0 con 0x7f09dc06e230 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09db7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc06dd90 0x7f09dc148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09db7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc06dd90 0x7f09dc148580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60440/0 (socket says 172.21.15.131:60440) 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09db7fe700 1 -- 172.21.15.131:0/820626697 learned_addr learned my addr 172.21.15.131:0/820626697 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09dbfff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09dc069620 0x7f09dc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09e0f9c700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09dc06e230 0x7f09dc15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09db7fe700 1 -- 172.21.15.131:0/820626697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09dc06e230 msgr2=0x7f09dc15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09db7fe700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09dc06e230 0x7f09dc15ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09db7fe700 1 -- 172.21.15.131:0/820626697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09dc069620 msgr2=0x7f09dc0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09db7fe700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09dc069620 0x7f09dc0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09db7fe700 1 -- 172.21.15.131:0/820626697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0094f0 con 0x7f09dc06dd90 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09db7fe700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc06dd90 0x7f09dc148580 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f09d000c7c0 tx=0x7f09d0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.391+0000 7f09c3fff700 1 -- 172.21.15.131:0/820626697 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0005a70 con 0x7f09dc06dd90 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.392+0000 7f09c3fff700 1 -- 172.21.15.131:0/820626697 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d0005bd0 con 0x7f09dc06dd90 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.392+0000 7f09c3fff700 1 -- 172.21.15.131:0/820626697 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d00070d0 con 0x7f09dc06dd90 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.392+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc16d6c0 con 0x7f09dc06dd90 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.392+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc148ac0 con 0x7f09dc06dd90 2023-12-26T22:53:20.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.393+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc0657e0 con 0x7f09dc06dd90 2023-12-26T22:53:20.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.394+0000 7f09c3fff700 1 -- 172.21.15.131:0/820626697 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f09d0007480 con 0x7f09dc06dd90 2023-12-26T22:53:20.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.395+0000 7f09c3fff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09c8066f30 0x7f09c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:20.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.395+0000 7f09c3fff700 1 -- 172.21.15.131:0/820626697 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f09d0083930 con 0x7f09dc06dd90 2023-12-26T22:53:20.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.395+0000 7f09dbfff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09c8066f30 0x7f09c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:20.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.396+0000 7f09dbfff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09c8066f30 0x7f09c80693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0099b0 tx=0x7f09cc017000 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:20.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.400+0000 7f09c3fff700 1 -- 172.21.15.131:0/820626697 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09d0050bb0 con 0x7f09dc06dd90 2023-12-26T22:53:20.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:20.594+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f09dc069f30 con 0x7f09dc06dd90 2023-12-26T22:53:21.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.171+0000 7f09c3fff700 1 -- 172.21.15.131:0/820626697 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v173) v1 ==== 170+0+0 (secure 0 0 0) 0x7f09d0017020 con 0x7f09dc06dd90 2023-12-26T22:53:21.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.178+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09c8066f30 msgr2=0x7f09c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:21.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.178+0000 7f09e29ff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09c8066f30 0x7f09c80693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0099b0 tx=0x7f09cc017000 comp rx=0 tx=0).stop 2023-12-26T22:53:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.178+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc06dd90 msgr2=0x7f09dc148580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.178+0000 7f09e29ff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc06dd90 0x7f09dc148580 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f09d000c7c0 tx=0x7f09d0007f70 comp rx=0 tx=0).stop 2023-12-26T22:53:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.180+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 shutdown_connections 2023-12-26T22:53:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.180+0000 7f09e29ff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09dc06e230 0x7f09dc15ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.180+0000 7f09e29ff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09dc069620 0x7f09dc0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.180+0000 7f09e29ff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f09c8066f30 0x7f09c80693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.180+0000 7f09e29ff700 1 --2- 172.21.15.131:0/820626697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09dc06dd90 0x7f09dc148580 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:21.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.180+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 >> 172.21.15.131:0/820626697 conn(0x7f09dc068040 msgr2=0x7f09dc0cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:21.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.183+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 shutdown_connections 2023-12-26T22:53:21.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.183+0000 7f09e29ff700 1 -- 172.21.15.131:0/820626697 wait complete. 2023-12-26T22:53:21.189 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T22:53:21.207 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T22:53:21.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.403+0000 7f4f815ff700 1 Processor -- start 2023-12-26T22:53:21.403 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.404+0000 7f4f815ff700 1 -- start start 2023-12-26T22:53:21.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.405+0000 7f4f815ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 0x7f4f7c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:21.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.405+0000 7f4f815ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c0b0d70 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.405+0000 7f4f815ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c0675b0 con 0x7f4f7c0b0550 2023-12-26T22:53:21.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.405+0000 7f4f815ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c0680f0 con 0x7f4f7c0cc580 2023-12-26T22:53:21.407 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.405+0000 7f4f7a7fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4f7c0b00e0 0x7f4f7c06e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40332/0 (socket says 172.21.15.131:40332) 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.405+0000 7f4f7a7fc700 1 -- 172.21.15.131:0/2294457556 learned_addr learned my addr 172.21.15.131:0/2294457556 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.405+0000 7f4f7affd700 1 --2- 172.21.15.131:0/2294457556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 0x7f4f7c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f79ffb700 1 -- 172.21.15.131:0/2294457556 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155260399 0 0) 0x7f4f7c0b0d70 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f79ffb700 1 -- 172.21.15.131:0/2294457556 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f64003570 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f79ffb700 1 -- 172.21.15.131:0/2294457556 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888993053 0 0) 0x7f4f7c0675b0 con 0x7f4f7c0b0550 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f79ffb700 1 -- 172.21.15.131:0/2294457556 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7c0b0d70 con 0x7f4f7c0b0550 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f7affd700 1 -- 172.21.15.131:0/2294457556 >> v1:172.21.15.190:6789/0 conn(0x7f4f7c0b0550 legacy=0x7f4f7c15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f7affd700 1 -- 172.21.15.131:0/2294457556 >> v1:172.21.15.131:6789/0 conn(0x7f4f7c0b00e0 legacy=0x7f4f7c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f7affd700 1 -- 172.21.15.131:0/2294457556 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c159c30 con 0x7f4f7c0cc580 2023-12-26T22:53:21.408 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f7affd700 1 --2- 172.21.15.131:0/2294457556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 0x7f4f7c0cc950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009910 tx=0x7f4f6c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66d5e203ac7cd13d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:21.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f79ffb700 1 -- 172.21.15.131:0/2294457556 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c011630 con 0x7f4f7c0cc580 2023-12-26T22:53:21.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.406+0000 7f4f79ffb700 1 -- 172.21.15.131:0/2294457556 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c011790 con 0x7f4f7c0cc580 2023-12-26T22:53:21.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.407+0000 7f4f79ffb700 1 -- 172.21.15.131:0/2294457556 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c011b10 con 0x7f4f7c0cc580 2023-12-26T22:53:21.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.407+0000 7f4f815ff700 1 -- 172.21.15.131:0/2294457556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 msgr2=0x7f4f7c0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:21.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.407+0000 7f4f815ff700 1 --2- 172.21.15.131:0/2294457556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 0x7f4f7c0cc950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009910 tx=0x7f4f6c0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:21.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.407+0000 7f4f815ff700 1 -- 172.21.15.131:0/2294457556 shutdown_connections 2023-12-26T22:53:21.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.407+0000 7f4f815ff700 1 --2- 172.21.15.131:0/2294457556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 0x7f4f7c0cc950 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:21.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.407+0000 7f4f815ff700 1 -- 172.21.15.131:0/2294457556 >> 172.21.15.131:0/2294457556 conn(0x7f4f7c0686c0 msgr2=0x7f4f7c068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:21.409 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.407+0000 7f4f815ff700 1 -- 172.21.15.131:0/2294457556 shutdown_connections 2023-12-26T22:53:21.410 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.407+0000 7f4f815ff700 1 -- 172.21.15.131:0/2294457556 wait complete. 2023-12-26T22:53:21.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.409+0000 7f4f815ff700 1 Processor -- start 2023-12-26T22:53:21.412 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.409+0000 7f4f815ff700 1 -- start start 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f815ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f7c0b00e0 0x7f4f7c16c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f815ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0b0550 0x7f4f7c169890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f815ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 0x7f4f7c169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f815ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c15b370 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f815ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c15b930 con 0x7f4f7c0b0550 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f815ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c16a3c0 con 0x7f4f7c0cc580 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f7c0b00e0 0x7f4f7c16c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f7c0b00e0 0x7f4f7c16c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60442/0 (socket says 172.21.15.131:60442) 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7affd700 1 -- 172.21.15.131:0/3858909210 learned_addr learned my addr 172.21.15.131:0/3858909210 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:21.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7a7fc700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0b0550 0x7f4f7c169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7b7fe700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 0x7f4f7c169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7affd700 1 -- 172.21.15.131:0/3858909210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 msgr2=0x7f4f7c169dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7affd700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 0x7f4f7c169dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7affd700 1 -- 172.21.15.131:0/3858909210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0b0550 msgr2=0x7f4f7c169890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7affd700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0b0550 0x7f4f7c169890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.410+0000 7f4f7affd700 1 -- 172.21.15.131:0/3858909210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c009420 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.411+0000 7f4f7affd700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f7c0b00e0 0x7f4f7c16c8a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0b0f40 tx=0x7f4f6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.411+0000 7f4f62ffd700 1 -- 172.21.15.131:0/3858909210 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c011b10 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.411+0000 7f4f62ffd700 1 -- 172.21.15.131:0/3858909210 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c000c90 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.414 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.411+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c16a6a0 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.411+0000 7f4f62ffd700 1 -- 172.21.15.131:0/3858909210 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c026680 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.411+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7c15dad0 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.412+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7c0657e0 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.415+0000 7f4f62ffd700 1 -- 172.21.15.131:0/3858909210 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4f6c0193b0 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.415+0000 7f4f62ffd700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f68066f30 0x7f4f680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:21.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.415+0000 7f4f62ffd700 1 -- 172.21.15.131:0/3858909210 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4f6c086030 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.416+0000 7f4f7a7fc700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f68066f30 0x7f4f680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:21.416 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.416+0000 7f4f7a7fc700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f68066f30 0x7f4f680693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c15e210 tx=0x7f4f70009040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:21.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.419+0000 7f4f62ffd700 1 -- 172.21.15.131:0/3858909210 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f6c053290 con 0x7f4f7c0b00e0 2023-12-26T22:53:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:21.653+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4f7c067850 con 0x7f4f7c0b00e0 2023-12-26T22:53:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.188+0000 7f4f62ffd700 1 -- 172.21.15.131:0/3858909210 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v77) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4f6c0150c0 con 0x7f4f7c0b00e0 2023-12-26T22:53:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f68066f30 msgr2=0x7f4f680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f68066f30 0x7f4f680693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c15e210 tx=0x7f4f70009040 comp rx=0 tx=0).stop 2023-12-26T22:53:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f7c0b00e0 msgr2=0x7f4f7c16c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f7c0b00e0 0x7f4f7c16c8a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0b0f40 tx=0x7f4f6c017000 comp rx=0 tx=0).stop 2023-12-26T22:53:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 shutdown_connections 2023-12-26T22:53:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0cc580 0x7f4f7c169dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f4f68066f30 0x7f4f680693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f7c0b00e0 0x7f4f7c16c8a0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 --2- 172.21.15.131:0/3858909210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0b0550 0x7f4f7c169890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.198+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 >> 172.21.15.131:0/3858909210 conn(0x7f4f7c0686c0 msgr2=0x7f4f7c0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:22.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.200+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 shutdown_connections 2023-12-26T22:53:22.200 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.201+0000 7f4f815ff700 1 -- 172.21.15.131:0/3858909210 wait complete. 2023-12-26T22:53:22.206 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 20 and data pool 21 2023-12-26T22:53:22.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:53:22.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.426+0000 7eff2ddff700 1 Processor -- start 2023-12-26T22:53:22.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.426+0000 7eff2ddff700 1 -- start start 2023-12-26T22:53:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.427+0000 7eff2ddff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7eff2806bfd0 0x7eff280c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.427+0000 7eff2ddff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7eff280cc100 0x7eff280cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.427+0000 7eff2ddff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff280686c0 con 0x7eff2813c510 2023-12-26T22:53:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.427+0000 7eff2ddff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff28069520 con 0x7eff280cc100 2023-12-26T22:53:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.427+0000 7eff2ddff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff28068910 con 0x7eff2806bfd0 2023-12-26T22:53:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.428+0000 7eff27fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7eff280cc100 0x7eff280cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.428+0000 7eff27fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7eff280cc100 0x7eff280cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60452/0 (socket says 172.21.15.131:60452) 2023-12-26T22:53:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.428+0000 7eff27fff700 1 -- 172.21.15.131:0/3155375043 learned_addr learned my addr 172.21.15.131:0/3155375043 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.428+0000 7eff26ffd700 1 --2- 172.21.15.131:0/3155375043 >> v2:172.21.15.190:3300/0 conn(0x7eff2806bfd0 0x7eff280c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.428+0000 7eff27fff700 1 -- 172.21.15.131:0/3155375043 >> v2:172.21.15.190:3300/0 conn(0x7eff2806bfd0 msgr2=0x7eff280c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.428+0000 7eff27fff700 1 --2- 172.21.15.131:0/3155375043 >> v2:172.21.15.190:3300/0 conn(0x7eff2806bfd0 0x7eff280c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.428+0000 7eff27fff700 1 -- 172.21.15.131:0/3155375043 >> v1:172.21.15.190:6789/0 conn(0x7eff2813c510 legacy=0x7eff2806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.428+0000 7eff27fff700 1 -- 172.21.15.131:0/3155375043 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff28068bf0 con 0x7eff280cc100 2023-12-26T22:53:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.428+0000 7eff27fff700 1 --2- 172.21.15.131:0/3155375043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff280cc100 0x7eff280cc520 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7eff280b43e0 tx=0x7eff20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2190bcccc4a1422 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff267fc700 1 -- 172.21.15.131:0/3155375043 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000e070 con 0x7eff280cc100 2023-12-26T22:53:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff267fc700 1 -- 172.21.15.131:0/3155375043 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2000ba50 con 0x7eff280cc100 2023-12-26T22:53:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff267fc700 1 -- 172.21.15.131:0/3155375043 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff200092e0 con 0x7eff280cc100 2023-12-26T22:53:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff2ddff700 1 -- 172.21.15.131:0/3155375043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff280cc100 msgr2=0x7eff280cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff2ddff700 1 --2- 172.21.15.131:0/3155375043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff280cc100 0x7eff280cc520 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7eff280b43e0 tx=0x7eff20009840 comp rx=0 tx=0).stop 2023-12-26T22:53:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff2ddff700 1 -- 172.21.15.131:0/3155375043 shutdown_connections 2023-12-26T22:53:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff2ddff700 1 --2- 172.21.15.131:0/3155375043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff280cc100 0x7eff280cc520 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff2ddff700 1 --2- 172.21.15.131:0/3155375043 >> v2:172.21.15.190:3300/0 conn(0x7eff2806bfd0 0x7eff280c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff2ddff700 1 -- 172.21.15.131:0/3155375043 >> 172.21.15.131:0/3155375043 conn(0x7eff280ce3e0 msgr2=0x7eff280ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff2ddff700 1 -- 172.21.15.131:0/3155375043 shutdown_connections 2023-12-26T22:53:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.429+0000 7eff2ddff700 1 -- 172.21.15.131:0/3155375043 wait complete. 2023-12-26T22:53:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.431+0000 7eff2ddff700 1 Processor -- start 2023-12-26T22:53:22.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.431+0000 7eff2ddff700 1 -- start start 2023-12-26T22:53:22.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff2ddff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2806bfd0 0x7eff2813c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff2ddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff280cc100 0x7eff2813a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff2ddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff2813c510 0x7eff2816d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff2ddff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff28155b00 con 0x7eff2806bfd0 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff2ddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff280b2b30 con 0x7eff280cc100 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff2ddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2813ac80 con 0x7eff2813c510 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2806bfd0 0x7eff2813c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2806bfd0 0x7eff2813c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60464/0 (socket says 172.21.15.131:60464) 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff277fe700 1 -- 172.21.15.131:0/3078863012 learned_addr learned my addr 172.21.15.131:0/3078863012 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff27fff700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff2813c510 0x7eff2816d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.432+0000 7eff26ffd700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff280cc100 0x7eff2813a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff277fe700 1 -- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff2813c510 msgr2=0x7eff2816d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff277fe700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff2813c510 0x7eff2816d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff277fe700 1 -- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff280cc100 msgr2=0x7eff2813a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff277fe700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff280cc100 0x7eff2813a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff277fe700 1 -- 172.21.15.131:0/3078863012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff20009ae0 con 0x7eff2806bfd0 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff27fff700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff2813c510 0x7eff2816d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:22.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff26ffd700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff280cc100 0x7eff2813a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:22.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff277fe700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2806bfd0 0x7eff2813c8e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7eff1800c7f0 tx=0x7eff18007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:22.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff0f7fe700 1 -- 172.21.15.131:0/3078863012 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18011590 con 0x7eff2806bfd0 2023-12-26T22:53:22.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff0f7fe700 1 -- 172.21.15.131:0/3078863012 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff18011710 con 0x7eff2806bfd0 2023-12-26T22:53:22.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff0f7fe700 1 -- 172.21.15.131:0/3078863012 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18011ad0 con 0x7eff2806bfd0 2023-12-26T22:53:22.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2816ddd0 con 0x7eff2806bfd0 2023-12-26T22:53:22.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.433+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff280b2200 con 0x7eff2806bfd0 2023-12-26T22:53:22.434 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.434+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff280657e0 con 0x7eff2806bfd0 2023-12-26T22:53:22.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.435+0000 7eff0f7fe700 1 -- 172.21.15.131:0/3078863012 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7eff1801b3b0 con 0x7eff2806bfd0 2023-12-26T22:53:22.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.436+0000 7eff0f7fe700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7eff14066f30 0x7eff140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.436+0000 7eff0f7fe700 1 -- 172.21.15.131:0/3078863012 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6169+0+0 (secure 0 0 0) 0x7eff18083c00 con 0x7eff2806bfd0 2023-12-26T22:53:22.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.436+0000 7eff26ffd700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7eff14066f30 0x7eff140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.437+0000 7eff26ffd700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7eff14066f30 0x7eff140693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff280b1690 tx=0x7eff1c008040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.440+0000 7eff0f7fe700 1 -- 172.21.15.131:0/3078863012 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff1808a080 con 0x7eff2806bfd0 2023-12-26T22:53:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.635+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff28068ed0 con 0x7eff2806bfd0 2023-12-26T22:53:22.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.636+0000 7eff0f7fe700 1 -- 172.21.15.131:0/3078863012 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+16137 (secure 0 0 0) 0x7eff18017070 con 0x7eff2806bfd0 2023-12-26T22:53:22.636 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:22.636 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":174,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:53:21.654960+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:53:19.805015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-26T22:53:20.636358+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:53:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7eff14066f30 msgr2=0x7eff140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7eff14066f30 0x7eff140693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff280b1690 tx=0x7eff1c008040 comp rx=0 tx=0).stop 2023-12-26T22:53:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2806bfd0 msgr2=0x7eff2813c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2806bfd0 0x7eff2813c8e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7eff1800c7f0 tx=0x7eff18007f70 comp rx=0 tx=0).stop 2023-12-26T22:53:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 shutdown_connections 2023-12-26T22:53:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff2813c510 0x7eff2816d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7eff14066f30 0x7eff140693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2806bfd0 0x7eff2813c8e0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 --2- 172.21.15.131:0/3078863012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff280cc100 0x7eff2813a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.638+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 >> 172.21.15.131:0/3078863012 conn(0x7eff280ce3e0 msgr2=0x7eff28153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.639+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 shutdown_connections 2023-12-26T22:53:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.639+0000 7eff2ddff700 1 -- 172.21.15.131:0/3078863012 wait complete. 2023-12-26T22:53:22.656 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:53:22.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.845+0000 7f16087ff700 1 Processor -- start 2023-12-26T22:53:22.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.845+0000 7f16087ff700 1 -- start start 2023-12-26T22:53:22.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.846+0000 7f16087ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16001596a0 0x7f16000b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.846+0000 7f16087ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f16000b0870 0x7f16000b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.846+0000 7f16087ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16000afcd0 con 0x7f1600159a70 2023-12-26T22:53:22.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.846+0000 7f16087ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16000b3e70 con 0x7f16000b0870 2023-12-26T22:53:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.846+0000 7f16087ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16000b0cd0 con 0x7f16001596a0 2023-12-26T22:53:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f1606d9c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f16000b0870 0x7f16000b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f160659b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16001596a0 0x7f16000b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f1606d9c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f16000b0870 0x7f16000b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:36682/0 (socket says 172.21.15.131:36682) 2023-12-26T22:53:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f1606d9c700 1 -- 172.21.15.131:0/2142895230 learned_addr learned my addr 172.21.15.131:0/2142895230 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f160659b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16001596a0 0x7f16000b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:43584/0 (socket says 172.21.15.131:43584) 2023-12-26T22:53:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f1605d9a700 1 --1- 172.21.15.131:0/2142895230 >> v1:172.21.15.190:6789/0 conn(0x7f1600159a70 0x7f16000b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:33704/0 (socket says 172.21.15.131:33704) 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f1606d9c700 1 -- 172.21.15.131:0/2142895230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16001596a0 msgr2=0x7f16000b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f1606d9c700 1 --2- 172.21.15.131:0/2142895230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16001596a0 0x7f16000b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f1606d9c700 1 -- 172.21.15.131:0/2142895230 >> v1:172.21.15.190:6789/0 conn(0x7f1600159a70 legacy=0x7f16000b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.847+0000 7f1606d9c700 1 -- 172.21.15.131:0/2142895230 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16000b2e00 con 0x7f16000b0870 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.848+0000 7f1606d9c700 1 --2- 172.21.15.131:0/2142895230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16000b0870 0x7f16000b28c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009f20 tx=0x7f15f8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff26c85e19c21ead server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.848+0000 7f1605599700 1 -- 172.21.15.131:0/2142895230 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800e070 con 0x7f16000b0870 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.848+0000 7f1605599700 1 -- 172.21.15.131:0/2142895230 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f800b9d0 con 0x7f16000b0870 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.848+0000 7f1605599700 1 -- 172.21.15.131:0/2142895230 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f80092e0 con 0x7f16000b0870 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.848+0000 7f16087ff700 1 -- 172.21.15.131:0/2142895230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16000b0870 msgr2=0x7f16000b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.848+0000 7f16087ff700 1 --2- 172.21.15.131:0/2142895230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16000b0870 0x7f16000b28c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009f20 tx=0x7f15f8009840 comp rx=0 tx=0).stop 2023-12-26T22:53:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.848+0000 7f16087ff700 1 -- 172.21.15.131:0/2142895230 shutdown_connections 2023-12-26T22:53:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.848+0000 7f16087ff700 1 --2- 172.21.15.131:0/2142895230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16000b0870 0x7f16000b28c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.848+0000 7f16087ff700 1 --2- 172.21.15.131:0/2142895230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16001596a0 0x7f16000b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.849+0000 7f16087ff700 1 -- 172.21.15.131:0/2142895230 >> 172.21.15.131:0/2142895230 conn(0x7f160006ece0 msgr2=0x7f160006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.849+0000 7f16087ff700 1 -- 172.21.15.131:0/2142895230 shutdown_connections 2023-12-26T22:53:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.849+0000 7f16087ff700 1 -- 172.21.15.131:0/2142895230 wait complete. 2023-12-26T22:53:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.851+0000 7f16087ff700 1 Processor -- start 2023-12-26T22:53:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.851+0000 7f16087ff700 1 -- start start 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f16087ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16000b0870 0x7f1600158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f16087ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16001596a0 0x7f1600169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f16087ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1600159a70 0x7f16000c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f16087ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16000c6d90 con 0x7f1600159a70 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f16087ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160015d2e0 con 0x7f16001596a0 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f16087ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160016a420 con 0x7f16000b0870 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f1606d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1600159a70 0x7f16000c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f1606d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1600159a70 0x7f16000c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60480/0 (socket says 172.21.15.131:60480) 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f1606d9c700 1 -- 172.21.15.131:0/3065605638 learned_addr learned my addr 172.21.15.131:0/3065605638 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f160659b700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16000b0870 0x7f1600158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.852+0000 7f1605d9a700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16001596a0 0x7f1600169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f1606d9c700 1 -- 172.21.15.131:0/3065605638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16000b0870 msgr2=0x7f1600158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f1606d9c700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16000b0870 0x7f1600158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f1606d9c700 1 -- 172.21.15.131:0/3065605638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16001596a0 msgr2=0x7f1600169ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f1606d9c700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16001596a0 0x7f1600169ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f1606d9c700 1 -- 172.21.15.131:0/3065605638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8009ae0 con 0x7f1600159a70 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f1606d9c700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1600159a70 0x7f16000c9730 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f1600089710 tx=0x7f15f8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f15f67fc700 1 -- 172.21.15.131:0/3065605638 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8008b10 con 0x7f1600159a70 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f15f67fc700 1 -- 172.21.15.131:0/3065605638 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f8008cb0 con 0x7f1600159a70 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f15f67fc700 1 -- 172.21.15.131:0/3065605638 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f801d670 con 0x7f1600159a70 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.853+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16000c6fe0 con 0x7f1600159a70 2023-12-26T22:53:22.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.854+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160015d860 con 0x7f1600159a70 2023-12-26T22:53:22.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.854+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16000657e0 con 0x7f1600159a70 2023-12-26T22:53:22.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.856+0000 7f15f67fc700 1 -- 172.21.15.131:0/3065605638 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f15f8003040 con 0x7f1600159a70 2023-12-26T22:53:22.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.856+0000 7f15f67fc700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f15e4066f30 0x7f15e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:22.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.856+0000 7f15f67fc700 1 -- 172.21.15.131:0/3065605638 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f15f8086310 con 0x7f1600159a70 2023-12-26T22:53:22.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.857+0000 7f160659b700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f15e4066f30 0x7f15e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:22.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.857+0000 7f160659b700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f15e4066f30 0x7f15e40693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009db0 tx=0x7f15fc0091a0 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:22.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:22.860+0000 7f15f67fc700 1 -- 172.21.15.131:0/3065605638 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15f8053310 con 0x7f1600159a70 2023-12-26T22:53:23.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.056+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16000cac70 con 0x7f1600159a70 2023-12-26T22:53:23.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.057+0000 7f15f67fc700 1 -- 172.21.15.131:0/3065605638 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f15f8056930 con 0x7f1600159a70 2023-12-26T22:53:23.056 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:23.057 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":174,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:53:21.654960+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:53:19.805015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-26T22:53:20.636358+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:53:23.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f15e4066f30 msgr2=0x7f15e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:23.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f15e4066f30 0x7f15e40693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009db0 tx=0x7f15fc0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1600159a70 msgr2=0x7f16000c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:23.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1600159a70 0x7f16000c9730 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f1600089710 tx=0x7f15f8018000 comp rx=0 tx=0).stop 2023-12-26T22:53:23.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 shutdown_connections 2023-12-26T22:53:23.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f15e4066f30 0x7f15e40693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1600159a70 0x7f16000c9730 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16000b0870 0x7f1600158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 --2- 172.21.15.131:0/3065605638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16001596a0 0x7f1600169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 >> 172.21.15.131:0/3065605638 conn(0x7f160006ece0 msgr2=0x7f16000cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:23.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 shutdown_connections 2023-12-26T22:53:23.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.059+0000 7f16087ff700 1 -- 172.21.15.131:0/3065605638 wait complete. 2023-12-26T22:53:23.077 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T22:53:23.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.277+0000 7f64ab5ff700 1 Processor -- start 2023-12-26T22:53:23.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.277+0000 7f64ab5ff700 1 -- start start 2023-12-26T22:53:23.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.278+0000 7f64ab5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f64a40cd100 0x7f64a40cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:23.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.278+0000 7f64ab5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f64a4159900 0x7f64a40cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:23.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.278+0000 7f64ab5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a415db90 con 0x7f64a40cd4d0 2023-12-26T22:53:23.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.278+0000 7f64ab5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a40cbe50 con 0x7f64a4159900 2023-12-26T22:53:23.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.278+0000 7f64ab5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a415d330 con 0x7f64a40cd100 2023-12-26T22:53:23.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.278+0000 7f64a9b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f64a4159900 0x7f64a40cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:23.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.278+0000 7f64a9b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f64a4159900 0x7f64a40cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60484/0 (socket says 172.21.15.131:60484) 2023-12-26T22:53:23.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.278+0000 7f64a9b9c700 1 -- 172.21.15.131:0/333876207 learned_addr learned my addr 172.21.15.131:0/333876207 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:23.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.279+0000 7f64a939b700 1 --2- 172.21.15.131:0/333876207 >> v2:172.21.15.190:3300/0 conn(0x7f64a40cd100 0x7f64a40cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:23.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.279+0000 7f64a9b9c700 1 -- 172.21.15.131:0/333876207 >> v2:172.21.15.190:3300/0 conn(0x7f64a40cd100 msgr2=0x7f64a40cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:23.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.279+0000 7f64a9b9c700 1 --2- 172.21.15.131:0/333876207 >> v2:172.21.15.190:3300/0 conn(0x7f64a40cd100 0x7f64a40cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.279+0000 7f64a9b9c700 1 -- 172.21.15.131:0/333876207 >> v1:172.21.15.190:6789/0 conn(0x7f64a40cd4d0 legacy=0x7f64a41591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:23.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.279+0000 7f64a9b9c700 1 -- 172.21.15.131:0/333876207 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a40cbb70 con 0x7f64a4159900 2023-12-26T22:53:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.279+0000 7f64a9b9c700 1 --2- 172.21.15.131:0/333876207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a4159900 0x7f64a40cadc0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f64a4089280 tx=0x7f649c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b90cae38faf7b60 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.279+0000 7f649bfff700 1 -- 172.21.15.131:0/333876207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c00e070 con 0x7f64a4159900 2023-12-26T22:53:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.279+0000 7f649bfff700 1 -- 172.21.15.131:0/333876207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c00ba20 con 0x7f64a4159900 2023-12-26T22:53:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.279+0000 7f649bfff700 1 -- 172.21.15.131:0/333876207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c0092e0 con 0x7f64a4159900 2023-12-26T22:53:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.280+0000 7f64ab5ff700 1 -- 172.21.15.131:0/333876207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a4159900 msgr2=0x7f64a40cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.280+0000 7f64ab5ff700 1 --2- 172.21.15.131:0/333876207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a4159900 0x7f64a40cadc0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f64a4089280 tx=0x7f649c009840 comp rx=0 tx=0).stop 2023-12-26T22:53:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.280+0000 7f64ab5ff700 1 -- 172.21.15.131:0/333876207 shutdown_connections 2023-12-26T22:53:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.280+0000 7f64ab5ff700 1 --2- 172.21.15.131:0/333876207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a4159900 0x7f64a40cadc0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.280+0000 7f64ab5ff700 1 --2- 172.21.15.131:0/333876207 >> v2:172.21.15.190:3300/0 conn(0x7f64a40cd100 0x7f64a40cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.280+0000 7f64ab5ff700 1 -- 172.21.15.131:0/333876207 >> 172.21.15.131:0/333876207 conn(0x7f64a406ece0 msgr2=0x7f64a406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:23.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.280+0000 7f64ab5ff700 1 -- 172.21.15.131:0/333876207 shutdown_connections 2023-12-26T22:53:23.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.280+0000 7f64ab5ff700 1 -- 172.21.15.131:0/333876207 wait complete. 2023-12-26T22:53:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.281+0000 7f64ab5ff700 1 Processor -- start 2023-12-26T22:53:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.282+0000 7f64ab5ff700 1 -- start start 2023-12-26T22:53:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.282+0000 7f64ab5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64a40cd100 0x7f64a41489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.282+0000 7f64ab5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a40cd4d0 0x7f64a40b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64ab5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64a4159900 0x7f64a4152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64ab5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a416cf30 con 0x7f64a40cd4d0 2023-12-26T22:53:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64ab5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a416dc80 con 0x7f64a4159900 2023-12-26T22:53:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64ab5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a416d1a0 con 0x7f64a40cd100 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a8b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a40cd4d0 0x7f64a40b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a939b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64a40cd100 0x7f64a41489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a8b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a40cd4d0 0x7f64a40b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60498/0 (socket says 172.21.15.131:60498) 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a939b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64a40cd100 0x7f64a41489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:43592/0 (socket says 172.21.15.131:43592) 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a8b9a700 1 -- 172.21.15.131:0/3835921732 learned_addr learned my addr 172.21.15.131:0/3835921732 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a9b9c700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64a4159900 0x7f64a4152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a8b9a700 1 -- 172.21.15.131:0/3835921732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64a40cd100 msgr2=0x7f64a41489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a8b9a700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64a40cd100 0x7f64a41489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a8b9a700 1 -- 172.21.15.131:0/3835921732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64a4159900 msgr2=0x7f64a4152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a8b9a700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64a4159900 0x7f64a4152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.283+0000 7f64a8b9a700 1 -- 172.21.15.131:0/3835921732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c009ae0 con 0x7f64a40cd4d0 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.284+0000 7f64a8b9a700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a40cd4d0 0x7f64a40b4910 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f649400c590 tx=0x7f649400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.284+0000 7f64997fa700 1 -- 172.21.15.131:0/3835921732 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494011590 con 0x7f64a40cd4d0 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.284+0000 7f64997fa700 1 -- 172.21.15.131:0/3835921732 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6494011710 con 0x7f64a40cd4d0 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.284+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a416d480 con 0x7f64a40cd4d0 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.284+0000 7f64997fa700 1 -- 172.21.15.131:0/3835921732 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494011ad0 con 0x7f64a40cd4d0 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.284+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a416d810 con 0x7f64a40cd4d0 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.285+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a40657e0 con 0x7f64a40cd4d0 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.286+0000 7f64997fa700 1 -- 172.21.15.131:0/3835921732 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f649401b3b0 con 0x7f64a40cd4d0 2023-12-26T22:53:23.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.287+0000 7f64997fa700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6488066f30 0x7f64880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:23.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.287+0000 7f64997fa700 1 -- 172.21.15.131:0/3835921732 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f6494083c30 con 0x7f64a40cd4d0 2023-12-26T22:53:23.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.287+0000 7f64a939b700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6488066f30 0x7f64880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:23.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.288+0000 7f64a939b700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6488066f30 0x7f64880693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009db0 tx=0x7f64a00091a0 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:23.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.291+0000 7f64997fa700 1 -- 172.21.15.131:0/3835921732 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6494050c30 con 0x7f64a40cd4d0 2023-12-26T22:53:23.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:23.529+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f64a416d9a0 con 0x7f64a40cd4d0 2023-12-26T22:53:24.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.219+0000 7f64997fa700 1 -- 172.21.15.131:0/3835921732 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) v1 ==== 118+0+0 (secure 0 0 0) 0x7f64a416d9a0 con 0x7f64a40cd4d0 2023-12-26T22:53:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6488066f30 msgr2=0x7f64880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6488066f30 0x7f64880693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009db0 tx=0x7f64a00091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a40cd4d0 msgr2=0x7f64a40b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a40cd4d0 0x7f64a40b4910 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f649400c590 tx=0x7f649400c8a0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 shutdown_connections 2023-12-26T22:53:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64a4159900 0x7f64a4152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64a40cd100 0x7f64a41489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6488066f30 0x7f64880693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 --2- 172.21.15.131:0/3835921732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64a40cd4d0 0x7f64a40b4910 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 >> 172.21.15.131:0/3835921732 conn(0x7f64a406ece0 msgr2=0x7f64a40cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 shutdown_connections 2023-12-26T22:53:24.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.222+0000 7f64ab5ff700 1 -- 172.21.15.131:0/3835921732 wait complete. 2023-12-26T22:53:24.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T22:53:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.440+0000 7fab18fff700 1 Processor -- start 2023-12-26T22:53:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.440+0000 7fab18fff700 1 -- start start 2023-12-26T22:53:24.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.441+0000 7fab18fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fab141599d0 0x7fab140ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:24.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.441+0000 7fab18fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab140cfd50 con 0x7fab14159600 2023-12-26T22:53:24.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.441+0000 7fab18fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab140b1280 con 0x7fab140d01e0 2023-12-26T22:53:24.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.441+0000 7fab18fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab140cf140 con 0x7fab141599d0 2023-12-26T22:53:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1259c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fab14159600 0x7fab1415dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40344/0 (socket says 172.21.15.131:40344) 2023-12-26T22:53:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1259c700 1 -- 172.21.15.131:0/2680552768 learned_addr learned my addr 172.21.15.131:0/2680552768 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab11d9b700 1 --2- 172.21.15.131:0/2680552768 >> v2:172.21.15.190:3300/0 conn(0x7fab141599d0 0x7fab140ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867676947 0 0) 0x7fab140cfd50 con 0x7fab14159600 2023-12-26T22:53:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faafc003570 con 0x7fab14159600 2023-12-26T22:53:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6642786 0 0) 0x7fab140b1280 con 0x7fab140d01e0 2023-12-26T22:53:24.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab140cfd50 con 0x7fab140d01e0 2023-12-26T22:53:24.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799297935 0 0) 0x7faafc003570 con 0x7fab14159600 2023-12-26T22:53:24.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 >> v2:172.21.15.190:3300/0 conn(0x7fab141599d0 msgr2=0x7fab140ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:24.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 --2- 172.21.15.131:0/2680552768 >> v2:172.21.15.190:3300/0 conn(0x7fab141599d0 0x7fab140ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 >> v1:172.21.15.190:6789/0 conn(0x7fab140d01e0 legacy=0x7fab140ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:24.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140cf420 con 0x7fab14159600 2023-12-26T22:53:24.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.442+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fab040015f0 con 0x7fab14159600 2023-12-26T22:53:24.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 >> v1:172.21.15.131:6789/0 conn(0x7fab14159600 legacy=0x7fab1415dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:24.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab1159a700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc003c40 0x7faafc004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:24.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab1159a700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7faafc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:24.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab1159a700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc005210 0x7faafc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:24.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab140cf140 con 0x7faafc004810 2023-12-26T22:53:24.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc00c130 con 0x7faafc005210 2023-12-26T22:53:24.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc00c2c0 con 0x7faafc003c40 2023-12-26T22:53:24.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab11d9b700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7faafc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:24.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab1259c700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc003c40 0x7faafc004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:24.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab12d9d700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc005210 0x7faafc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:24.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab11d9b700 1 -- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc003c40 msgr2=0x7faafc004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:24.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab11d9b700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc003c40 0x7faafc004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab11d9b700 1 -- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc005210 msgr2=0x7faafc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:24.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab11d9b700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc005210 0x7faafc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab11d9b700 1 -- 172.21.15.131:0/2680552768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc003570 con 0x7faafc004810 2023-12-26T22:53:24.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab1259c700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc003c40 0x7faafc004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:24.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab12d9d700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc005210 0x7faafc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:24.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.443+0000 7fab12d9d700 1 -- 172.21.15.131:0/2680552768 reap_dead start 2023-12-26T22:53:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab11d9b700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7faafc004c80 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fab08009c30 tx=0x7fab080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6e6f7a50fb086c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080095b0 con 0x7faafc004810 2023-12-26T22:53:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0800e710 con 0x7faafc004810 2023-12-26T22:53:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab1159a700 1 -- 172.21.15.131:0/2680552768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800ead0 con 0x7faafc004810 2023-12-26T22:53:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab18fff700 1 -- 172.21.15.131:0/2680552768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 msgr2=0x7faafc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab18fff700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7faafc004c80 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fab08009c30 tx=0x7fab080091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab18fff700 1 -- 172.21.15.131:0/2680552768 shutdown_connections 2023-12-26T22:53:24.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab18fff700 1 --2- 172.21.15.131:0/2680552768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7faafc004c80 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab18fff700 1 -- 172.21.15.131:0/2680552768 >> 172.21.15.131:0/2680552768 conn(0x7fab1406ece0 msgr2=0x7fab1406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab18fff700 1 -- 172.21.15.131:0/2680552768 shutdown_connections 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.444+0000 7fab18fff700 1 -- 172.21.15.131:0/2680552768 wait complete. 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.446+0000 7fab18fff700 1 Processor -- start 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.446+0000 7fab18fff700 1 -- start start 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab18fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7fab1414e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab18fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab140b3390 0x7fab14156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab18fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab140b3800 0x7fab141572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab18fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab140b3d50 con 0x7faafc004810 2023-12-26T22:53:24.455 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab18fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab140b4030 con 0x7fab140b3390 2023-12-26T22:53:24.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab18fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab140b4310 con 0x7fab140b3800 2023-12-26T22:53:24.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab1259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7fab1414e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:24.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab1259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7fab1414e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60510/0 (socket says 172.21.15.131:60510) 2023-12-26T22:53:24.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab1259c700 1 -- 172.21.15.131:0/2940710275 learned_addr learned my addr 172.21.15.131:0/2940710275 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:24.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab12d9d700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab140b3800 0x7fab141572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:24.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab11d9b700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab140b3390 0x7fab14156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:24.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab1259c700 1 -- 172.21.15.131:0/2940710275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab140b3800 msgr2=0x7fab141572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:24.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab1259c700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab140b3800 0x7fab141572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab1259c700 1 -- 172.21.15.131:0/2940710275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab140b3390 msgr2=0x7fab14156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab1259c700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab140b3390 0x7fab14156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.447+0000 7fab1259c700 1 -- 172.21.15.131:0/2940710275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1414f680 con 0x7faafc004810 2023-12-26T22:53:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.448+0000 7fab1259c700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7fab1414e4b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fab04001580 tx=0x7fab04009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.448+0000 7fab027fc700 1 -- 172.21.15.131:0/2940710275 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040072e0 con 0x7faafc004810 2023-12-26T22:53:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.448+0000 7fab027fc700 1 -- 172.21.15.131:0/2940710275 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab04007440 con 0x7faafc004810 2023-12-26T22:53:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.448+0000 7fab027fc700 1 -- 172.21.15.131:0/2940710275 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040057f0 con 0x7faafc004810 2023-12-26T22:53:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.448+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1414f330 con 0x7faafc004810 2023-12-26T22:53:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.448+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1415bd40 con 0x7faafc004810 2023-12-26T22:53:24.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.449+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1415bf30 con 0x7faafc004810 2023-12-26T22:53:24.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.450+0000 7fab027fc700 1 -- 172.21.15.131:0/2940710275 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fab04005b80 con 0x7faafc004810 2023-12-26T22:53:24.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.451+0000 7fab027fc700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faaf4066f60 0x7faaf4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:24.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.451+0000 7fab027fc700 1 -- 172.21.15.131:0/2940710275 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fab04084150 con 0x7faafc004810 2023-12-26T22:53:24.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.451+0000 7fab11d9b700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faaf4066f60 0x7faaf4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:24.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.452+0000 7fab11d9b700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faaf4066f60 0x7faaf4069410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab140cf680 tx=0x7fab0801a000 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:24.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.455+0000 7fab027fc700 1 -- 172.21.15.131:0/2940710275 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab1415bf30 con 0x7faafc004810 2023-12-26T22:53:24.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:24.690+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fab1415bf30 con 0x7faafc004810 2023-12-26T22:53:25.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.220+0000 7fab027fc700 1 -- 172.21.15.131:0/2940710275 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab1415bf30 con 0x7faafc004810 2023-12-26T22:53:25.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faaf4066f60 msgr2=0x7faaf4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faaf4066f60 0x7faaf4069410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab140cf680 tx=0x7fab0801a000 comp rx=0 tx=0).stop 2023-12-26T22:53:25.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 msgr2=0x7fab1414e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7fab1414e4b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fab04001580 tx=0x7fab04009040 comp rx=0 tx=0).stop 2023-12-26T22:53:25.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 shutdown_connections 2023-12-26T22:53:25.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab140b3800 0x7fab141572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7faaf4066f60 0x7faaf4069410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faafc004810 0x7fab1414e4b0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 --2- 172.21.15.131:0/2940710275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab140b3390 0x7fab14156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 >> 172.21.15.131:0/2940710275 conn(0x7fab1406ece0 msgr2=0x7fab140caca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:25.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 shutdown_connections 2023-12-26T22:53:25.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.223+0000 7fab18fff700 1 -- 172.21.15.131:0/2940710275 wait complete. 2023-12-26T22:53:25.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:53:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.436+0000 7f766f7ff700 1 Processor -- start 2023-12-26T22:53:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.436+0000 7f766f7ff700 1 -- start start 2023-12-26T22:53:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.437+0000 7f766f7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f76680cb320 0x7f7668069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.437+0000 7f766f7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f766806a1e0 0x7f76680b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.437+0000 7f766f7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680b0c90 0x7f76680aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.437+0000 7f766f7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76680c8cc0 con 0x7f76680cb320 2023-12-26T22:53:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.437+0000 7f766f7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76680ad700 con 0x7f766806a1e0 2023-12-26T22:53:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.437+0000 7f766f7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76680b1110 con 0x7f76680b0c90 2023-12-26T22:53:25.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.437+0000 7f766d59b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f76680cb320 0x7f7668069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.437+0000 7f766d59b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f76680cb320 0x7f7668069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48084/0 (socket says 172.21.15.131:48084) 2023-12-26T22:53:25.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.437+0000 7f766d59b700 1 -- 172.21.15.131:0/3910332678 learned_addr learned my addr 172.21.15.131:0/3910332678 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:25.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766d59b700 1 -- 172.21.15.131:0/3910332678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680b0c90 msgr2=0x7f76680aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:25.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766dd9c700 1 --2- 172.21.15.131:0/3910332678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680b0c90 0x7f76680aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766cd9a700 1 --2- 172.21.15.131:0/3910332678 >> v2:172.21.15.190:3300/0 conn(0x7f766806a1e0 0x7f76680b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.441 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766d59b700 1 --2- 172.21.15.131:0/3910332678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680b0c90 0x7f76680aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766d59b700 1 -- 172.21.15.131:0/3910332678 >> v2:172.21.15.190:3300/0 conn(0x7f766806a1e0 msgr2=0x7f76680b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766d59b700 1 --2- 172.21.15.131:0/3910332678 >> v2:172.21.15.190:3300/0 conn(0x7f766806a1e0 0x7f76680b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766d59b700 1 -- 172.21.15.131:0/3910332678 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7668156940 con 0x7f76680cb320 2023-12-26T22:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766cd9a700 1 --2- 172.21.15.131:0/3910332678 >> v2:172.21.15.190:3300/0 conn(0x7f766806a1e0 0x7f76680b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766dd9c700 1 --2- 172.21.15.131:0/3910332678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680b0c90 0x7f76680aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f766d59b700 1 --2- 172.21.15.131:0/3910332678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76680cb320 0x7f7668069ca0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7668064700 tx=0x7f7664009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8cecd4d1efd256 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f765ffff700 1 -- 172.21.15.131:0/3910332678 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400b820 con 0x7f76680cb320 2023-12-26T22:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f765ffff700 1 -- 172.21.15.131:0/3910332678 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400b980 con 0x7f76680cb320 2023-12-26T22:53:25.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.438+0000 7f765ffff700 1 -- 172.21.15.131:0/3910332678 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400bd20 con 0x7f76680cb320 2023-12-26T22:53:25.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.439+0000 7f766f7ff700 1 -- 172.21.15.131:0/3910332678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76680cb320 msgr2=0x7f7668069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.439+0000 7f766f7ff700 1 --2- 172.21.15.131:0/3910332678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76680cb320 0x7f7668069ca0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7668064700 tx=0x7f7664009250 comp rx=0 tx=0).stop 2023-12-26T22:53:25.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.439+0000 7f766f7ff700 1 -- 172.21.15.131:0/3910332678 shutdown_connections 2023-12-26T22:53:25.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.439+0000 7f766f7ff700 1 --2- 172.21.15.131:0/3910332678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76680cb320 0x7f7668069ca0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.439+0000 7f766f7ff700 1 --2- 172.21.15.131:0/3910332678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680b0c90 0x7f76680aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.443 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.439+0000 7f766f7ff700 1 --2- 172.21.15.131:0/3910332678 >> v2:172.21.15.190:3300/0 conn(0x7f766806a1e0 0x7f76680b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.439+0000 7f766f7ff700 1 -- 172.21.15.131:0/3910332678 >> 172.21.15.131:0/3910332678 conn(0x7f7668068040 msgr2=0x7f7668069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:25.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.439+0000 7f766f7ff700 1 -- 172.21.15.131:0/3910332678 shutdown_connections 2023-12-26T22:53:25.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.439+0000 7f766f7ff700 1 -- 172.21.15.131:0/3910332678 wait complete. 2023-12-26T22:53:25.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.440+0000 7f766f7ff700 1 Processor -- start 2023-12-26T22:53:25.444 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.441+0000 7f766f7ff700 1 -- start start 2023-12-26T22:53:25.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.441+0000 7f766f7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f766806a1e0 0x7f766816a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.441+0000 7f766f7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76680b0c90 0x7f766806e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.441+0000 7f766f7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680cb320 0x7f7668144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.441+0000 7f766f7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766806e4b0 con 0x7f766806a1e0 2023-12-26T22:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766f7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766816a900 con 0x7f76680b0c90 2023-12-26T22:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766f7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766806ee80 con 0x7f76680cb320 2023-12-26T22:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766d59b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f766806a1e0 0x7f766816a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766d59b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f766806a1e0 0x7f766816a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48096/0 (socket says 172.21.15.131:48096) 2023-12-26T22:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766d59b700 1 -- 172.21.15.131:0/2110317693 learned_addr learned my addr 172.21.15.131:0/2110317693 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766d59b700 1 -- 172.21.15.131:0/2110317693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680cb320 msgr2=0x7f7668144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766d59b700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680cb320 0x7f7668144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766d59b700 1 -- 172.21.15.131:0/2110317693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76680b0c90 msgr2=0x7f766806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766d59b700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76680b0c90 0x7f766806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766d59b700 1 -- 172.21.15.131:0/2110317693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640094f0 con 0x7f766806a1e0 2023-12-26T22:53:25.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.442+0000 7f766d59b700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f766806a1e0 0x7f766816a3c0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7664009290 tx=0x7f7664017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:25.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.443+0000 7f765d7fa700 1 -- 172.21.15.131:0/2110317693 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766401a420 con 0x7f766806a1e0 2023-12-26T22:53:25.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.443+0000 7f765d7fa700 1 -- 172.21.15.131:0/2110317693 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766401a580 con 0x7f766806a1e0 2023-12-26T22:53:25.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.443+0000 7f765d7fa700 1 -- 172.21.15.131:0/2110317693 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766401a940 con 0x7f766806a1e0 2023-12-26T22:53:25.449 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.443+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766806d8a0 con 0x7f766806a1e0 2023-12-26T22:53:25.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.443+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766806de60 con 0x7f766806a1e0 2023-12-26T22:53:25.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.444+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76680ae7e0 con 0x7f766806a1e0 2023-12-26T22:53:25.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.445+0000 7f765d7fa700 1 -- 172.21.15.131:0/2110317693 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f76640253f0 con 0x7f766806a1e0 2023-12-26T22:53:25.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.445+0000 7f765d7fa700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f764c066f30 0x7f764c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.446+0000 7f765d7fa700 1 -- 172.21.15.131:0/2110317693 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f76640860f0 con 0x7f766806a1e0 2023-12-26T22:53:25.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.446+0000 7f766cd9a700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f764c066f30 0x7f764c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.450 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.446+0000 7f766cd9a700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f764c066f30 0x7f764c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f76680ad6d0 tx=0x7f7658008040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:25.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.450+0000 7f765d7fa700 1 -- 172.21.15.131:0/2110317693 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7664053190 con 0x7f766806a1e0 2023-12-26T22:53:25.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.686+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76680c8e00 con 0x7f766806a1e0 2023-12-26T22:53:25.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.686+0000 7f765d7fa700 1 -- 172.21.15.131:0/2110317693 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f76640567b0 con 0x7f766806a1e0 2023-12-26T22:53:25.686 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:25.686 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2545294570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2545294570},{"type":"v1","addr":"172.21.15.190:6835","nonce":2545294570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1800615106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1800615106},{"type":"v1","addr":"172.21.15.131:6834","nonce":1800615106}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3279484094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3279484094},{"type":"v1","addr":"172.21.15.131:6835","nonce":3279484094}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:53:21.655334+0000","modified":"2023-12-26T22:53:25.218600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9105":{"gid":9105,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/716462328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":716462328},{"type":"v1","addr":"172.21.15.190:6834","nonce":716462328}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-12-26T22:53:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.688+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f764c066f30 msgr2=0x7f764c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.688+0000 7f766f7ff700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f764c066f30 0x7f764c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f76680ad6d0 tx=0x7f7658008040 comp rx=0 tx=0).stop 2023-12-26T22:53:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.688+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f766806a1e0 msgr2=0x7f766816a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.688+0000 7f766f7ff700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f766806a1e0 0x7f766816a3c0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7664009290 tx=0x7f7664017000 comp rx=0 tx=0).stop 2023-12-26T22:53:25.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.689+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 shutdown_connections 2023-12-26T22:53:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.689+0000 7f766f7ff700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76680cb320 0x7f7668144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.689+0000 7f766f7ff700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f764c066f30 0x7f764c0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.689+0000 7f766f7ff700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f766806a1e0 0x7f766816a3c0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.689+0000 7f766f7ff700 1 --2- 172.21.15.131:0/2110317693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76680b0c90 0x7f766806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.689+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 >> 172.21.15.131:0/2110317693 conn(0x7f7668068040 msgr2=0x7f76680adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.689+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 shutdown_connections 2023-12-26T22:53:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.689+0000 7f766f7ff700 1 -- 172.21.15.131:0/2110317693 wait complete. 2023-12-26T22:53:25.691 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 81 2023-12-26T22:53:25.707 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T22:53:25.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.897+0000 7efd7dfff700 1 Processor -- start 2023-12-26T22:53:25.901 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.897+0000 7efd7dfff700 1 -- start start 2023-12-26T22:53:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd7dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780cb320 0x7efd78069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd7dfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efd7806a1e0 0x7efd780b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd7dfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd78156940 con 0x7efd780b0c90 2023-12-26T22:53:25.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd7dfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd780c8cc0 con 0x7efd7806a1e0 2023-12-26T22:53:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd7dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd780ad700 con 0x7efd780cb320 2023-12-26T22:53:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd77fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efd780b0c90 0x7efd780aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56302/0 (socket says 172.21.15.131:56302) 2023-12-26T22:53:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd77fff700 1 -- 172.21.15.131:0/3809537581 learned_addr learned my addr 172.21.15.131:0/3809537581 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd6ffff700 1 --2- 172.21.15.131:0/3809537581 >> v2:172.21.15.190:3300/0 conn(0x7efd7806a1e0 0x7efd780b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd777fe700 1 --2- 172.21.15.131:0/3809537581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780cb320 0x7efd78069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd76ffd700 1 -- 172.21.15.131:0/3809537581 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117603345 0 0) 0x7efd78156940 con 0x7efd780b0c90 2023-12-26T22:53:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.899+0000 7efd76ffd700 1 -- 172.21.15.131:0/3809537581 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd64003570 con 0x7efd780b0c90 2023-12-26T22:53:25.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.900+0000 7efd6ffff700 1 -- 172.21.15.131:0/3809537581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780cb320 msgr2=0x7efd78069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.900+0000 7efd6ffff700 1 --2- 172.21.15.131:0/3809537581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780cb320 0x7efd78069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.900+0000 7efd6ffff700 1 -- 172.21.15.131:0/3809537581 >> v1:172.21.15.131:6789/0 conn(0x7efd780b0c90 legacy=0x7efd780aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.900+0000 7efd6ffff700 1 -- 172.21.15.131:0/3809537581 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd780ae500 con 0x7efd7806a1e0 2023-12-26T22:53:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.900+0000 7efd777fe700 1 --2- 172.21.15.131:0/3809537581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780cb320 0x7efd78069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.900+0000 7efd6ffff700 1 --2- 172.21.15.131:0/3809537581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7806a1e0 0x7efd780b0750 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efd600098e0 tx=0x7efd600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14d1a23d4a9eb549 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.900+0000 7efd76ffd700 1 -- 172.21.15.131:0/3809537581 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60011630 con 0x7efd7806a1e0 2023-12-26T22:53:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.900+0000 7efd76ffd700 1 -- 172.21.15.131:0/3809537581 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd60011790 con 0x7efd7806a1e0 2023-12-26T22:53:25.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.900+0000 7efd76ffd700 1 -- 172.21.15.131:0/3809537581 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60011b30 con 0x7efd7806a1e0 2023-12-26T22:53:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.901+0000 7efd7dfff700 1 -- 172.21.15.131:0/3809537581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7806a1e0 msgr2=0x7efd780b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.901+0000 7efd7dfff700 1 --2- 172.21.15.131:0/3809537581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7806a1e0 0x7efd780b0750 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efd600098e0 tx=0x7efd600091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.901+0000 7efd7dfff700 1 -- 172.21.15.131:0/3809537581 shutdown_connections 2023-12-26T22:53:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.901+0000 7efd7dfff700 1 --2- 172.21.15.131:0/3809537581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780cb320 0x7efd78069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.901+0000 7efd7dfff700 1 --2- 172.21.15.131:0/3809537581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7806a1e0 0x7efd780b0750 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.901+0000 7efd7dfff700 1 -- 172.21.15.131:0/3809537581 >> 172.21.15.131:0/3809537581 conn(0x7efd78068040 msgr2=0x7efd78069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.901+0000 7efd7dfff700 1 -- 172.21.15.131:0/3809537581 shutdown_connections 2023-12-26T22:53:25.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.901+0000 7efd7dfff700 1 -- 172.21.15.131:0/3809537581 wait complete. 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.902+0000 7efd7dfff700 1 Processor -- start 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.902+0000 7efd7dfff700 1 -- start start 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.903+0000 7efd7dfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7806a1e0 0x7efd7806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.903+0000 7efd7dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780b0c90 0x7efd781698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.903+0000 7efd7dfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd780cb320 0x7efd78169de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.903+0000 7efd7dfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7806eb40 con 0x7efd780cb320 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.903+0000 7efd7dfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7806df30 con 0x7efd7806a1e0 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.903+0000 7efd7dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7806e210 con 0x7efd780b0c90 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd77fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd780cb320 0x7efd78169de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd77fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd780cb320 0x7efd78169de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48108/0 (socket says 172.21.15.131:48108) 2023-12-26T22:53:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd77fff700 1 -- 172.21.15.131:0/2400537051 learned_addr learned my addr 172.21.15.131:0/2400537051 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd777fe700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7806a1e0 0x7efd7806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd6ffff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780b0c90 0x7efd781698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd77fff700 1 -- 172.21.15.131:0/2400537051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780b0c90 msgr2=0x7efd781698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd77fff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780b0c90 0x7efd781698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd77fff700 1 -- 172.21.15.131:0/2400537051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7806a1e0 msgr2=0x7efd7806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd77fff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7806a1e0 0x7efd7806d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:25.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd77fff700 1 -- 172.21.15.131:0/2400537051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68009420 con 0x7efd780cb320 2023-12-26T22:53:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.904+0000 7efd77fff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd780cb320 0x7efd78169de0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7efd70001460 tx=0x7efd70001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.905+0000 7efd6f7fe700 1 -- 172.21.15.131:0/2400537051 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7000a890 con 0x7efd780cb320 2023-12-26T22:53:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.905+0000 7efd6f7fe700 1 -- 172.21.15.131:0/2400537051 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7000a9f0 con 0x7efd780cb320 2023-12-26T22:53:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.905+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd60009420 con 0x7efd780cb320 2023-12-26T22:53:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.905+0000 7efd6f7fe700 1 -- 172.21.15.131:0/2400537051 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7000ad70 con 0x7efd780cb320 2023-12-26T22:53:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.905+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7806e7d0 con 0x7efd780cb320 2023-12-26T22:53:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.906+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd780657e0 con 0x7efd780cb320 2023-12-26T22:53:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.907+0000 7efd6f7fe700 1 -- 172.21.15.131:0/2400537051 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efd70007300 con 0x7efd780cb320 2023-12-26T22:53:25.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.907+0000 7efd6f7fe700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efd58066f30 0x7efd580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:25.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.907+0000 7efd6f7fe700 1 -- 172.21.15.131:0/2400537051 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7efd70084b30 con 0x7efd780cb320 2023-12-26T22:53:25.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.908+0000 7efd777fe700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efd58066f30 0x7efd580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:25.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.908+0000 7efd777fe700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efd58066f30 0x7efd580693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd780caa20 tx=0x7efd680091a0 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:25.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:25.911+0000 7efd6f7fe700 1 -- 172.21.15.131:0/2400537051 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd70051ba0 con 0x7efd780cb320 2023-12-26T22:53:26.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.105+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd7806ec80 con 0x7efd780cb320 2023-12-26T22:53:26.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.106+0000 7efd6f7fe700 1 -- 172.21.15.131:0/2400537051 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+16124 (secure 0 0 0) 0x7efd700551c0 con 0x7efd780cb320 2023-12-26T22:53:26.106 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:26.106 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":175,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T22:53:23.195703+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:53:19.805015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-26T22:53:20.636358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T22:53:26.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.108+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efd58066f30 msgr2=0x7efd580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.108+0000 7efd7dfff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efd58066f30 0x7efd580693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd780caa20 tx=0x7efd680091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.108+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd780cb320 msgr2=0x7efd78169de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.109 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.108+0000 7efd7dfff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd780cb320 0x7efd78169de0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7efd70001460 tx=0x7efd70001ae0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.109+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 shutdown_connections 2023-12-26T22:53:26.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.109+0000 7efd7dfff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7efd58066f30 0x7efd580693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.109+0000 7efd7dfff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd780cb320 0x7efd78169de0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.109+0000 7efd7dfff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd7806a1e0 0x7efd7806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.109+0000 7efd7dfff700 1 --2- 172.21.15.131:0/2400537051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd780b0c90 0x7efd781698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.109+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 >> 172.21.15.131:0/2400537051 conn(0x7efd78068040 msgr2=0x7efd781577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:26.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.109+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 shutdown_connections 2023-12-26T22:53:26.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.109+0000 7efd7dfff700 1 -- 172.21.15.131:0/2400537051 wait complete. 2023-12-26T22:53:26.127 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T22:53:26.127 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T22:53:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.318+0000 7f142dfff700 1 Processor -- start 2023-12-26T22:53:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.319+0000 7f142dfff700 1 -- start start 2023-12-26T22:53:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f142dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14281596a0 0x7f14280b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f142dfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14280b0870 0x7f14280b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f142dfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14280afcd0 con 0x7f1428159a70 2023-12-26T22:53:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f142dfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14280b3e70 con 0x7f14280b0870 2023-12-26T22:53:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f142dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14280b0cd0 con 0x7f14281596a0 2023-12-26T22:53:26.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f1427fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14280b0870 0x7f14280b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f1427fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14280b0870 0x7f14280b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48124/0 (socket says 172.21.15.131:48124) 2023-12-26T22:53:26.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f1427fff700 1 -- 172.21.15.131:0/2975769277 learned_addr learned my addr 172.21.15.131:0/2975769277 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:26.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f14277fe700 1 --2- 172.21.15.131:0/2975769277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14281596a0 0x7f14280b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f1427fff700 1 -- 172.21.15.131:0/2975769277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14281596a0 msgr2=0x7f14280b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:26.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f1427fff700 1 --2- 172.21.15.131:0/2975769277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14281596a0 0x7f14280b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.320+0000 7f1427fff700 1 -- 172.21.15.131:0/2975769277 >> v1:172.21.15.190:6789/0 conn(0x7f1428159a70 legacy=0x7f14280b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:26.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.321+0000 7f1427fff700 1 -- 172.21.15.131:0/2975769277 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14280b2e00 con 0x7f14280b0870 2023-12-26T22:53:26.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.321+0000 7f14277fe700 1 --2- 172.21.15.131:0/2975769277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14281596a0 0x7f14280b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.321+0000 7f1427fff700 1 --2- 172.21.15.131:0/2975769277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14280b0870 0x7f14280b28c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f1420009f20 tx=0x7f1420009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d54df02e8ca818 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.321+0000 7f14267fc700 1 -- 172.21.15.131:0/2975769277 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142000e070 con 0x7f14280b0870 2023-12-26T22:53:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.321+0000 7f14267fc700 1 -- 172.21.15.131:0/2975769277 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142000ba40 con 0x7f14280b0870 2023-12-26T22:53:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.321+0000 7f14267fc700 1 -- 172.21.15.131:0/2975769277 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14200092e0 con 0x7f14280b0870 2023-12-26T22:53:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.322+0000 7f142dfff700 1 -- 172.21.15.131:0/2975769277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14280b0870 msgr2=0x7f14280b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.322+0000 7f142dfff700 1 --2- 172.21.15.131:0/2975769277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14280b0870 0x7f14280b28c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f1420009f20 tx=0x7f1420009840 comp rx=0 tx=0).stop 2023-12-26T22:53:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.322+0000 7f142dfff700 1 -- 172.21.15.131:0/2975769277 shutdown_connections 2023-12-26T22:53:26.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.322+0000 7f142dfff700 1 --2- 172.21.15.131:0/2975769277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14280b0870 0x7f14280b28c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.322+0000 7f142dfff700 1 --2- 172.21.15.131:0/2975769277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14281596a0 0x7f14280b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.322+0000 7f142dfff700 1 -- 172.21.15.131:0/2975769277 >> 172.21.15.131:0/2975769277 conn(0x7f142806ece0 msgr2=0x7f142806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.322+0000 7f142dfff700 1 -- 172.21.15.131:0/2975769277 shutdown_connections 2023-12-26T22:53:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.322+0000 7f142dfff700 1 -- 172.21.15.131:0/2975769277 wait complete. 2023-12-26T22:53:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.323+0000 7f142dfff700 1 Processor -- start 2023-12-26T22:53:26.326 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.323+0000 7f142dfff700 1 -- start start 2023-12-26T22:53:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.324+0000 7f142dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14280b0870 0x7f142815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.324+0000 7f142dfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14281596a0 0x7f142815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.324+0000 7f142dfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1428159a70 0x7f142815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.324+0000 7f142dfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142815f160 con 0x7f1428159a70 2023-12-26T22:53:26.330 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.324+0000 7f142dfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142815e550 con 0x7f14281596a0 2023-12-26T22:53:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.324+0000 7f142dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142815e830 con 0x7f14280b0870 2023-12-26T22:53:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.324+0000 7f1427fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1428159a70 0x7f142815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f1427fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1428159a70 0x7f142815ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48138/0 (socket says 172.21.15.131:48138) 2023-12-26T22:53:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f1427fff700 1 -- 172.21.15.131:0/126379657 learned_addr learned my addr 172.21.15.131:0/126379657 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f14277fe700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14280b0870 0x7f142815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f1426ffd700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14281596a0 0x7f142815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f1427fff700 1 -- 172.21.15.131:0/126379657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14280b0870 msgr2=0x7f142815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f1427fff700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14280b0870 0x7f142815dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.331 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f1427fff700 1 -- 172.21.15.131:0/126379657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14281596a0 msgr2=0x7f142815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f1427fff700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14281596a0 0x7f142815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f1427fff700 1 -- 172.21.15.131:0/126379657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420009ae0 con 0x7f1428159a70 2023-12-26T22:53:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f1427fff700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1428159a70 0x7f142815ca00 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f1428089710 tx=0x7f1420017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f140f7fe700 1 -- 172.21.15.131:0/126379657 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420012d00 con 0x7f1428159a70 2023-12-26T22:53:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f140f7fe700 1 -- 172.21.15.131:0/126379657 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1420012e60 con 0x7f1428159a70 2023-12-26T22:53:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142815eb10 con 0x7f1428159a70 2023-12-26T22:53:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.325+0000 7f140f7fe700 1 -- 172.21.15.131:0/126379657 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142001d670 con 0x7f1428159a70 2023-12-26T22:53:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.326+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142815e140 con 0x7f1428159a70 2023-12-26T22:53:26.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.327+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14280657e0 con 0x7f1428159a70 2023-12-26T22:53:26.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.327+0000 7f140f7fe700 1 -- 172.21.15.131:0/126379657 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f142001da60 con 0x7f1428159a70 2023-12-26T22:53:26.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.328+0000 7f140f7fe700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1414066f30 0x7f14140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.328+0000 7f140f7fe700 1 -- 172.21.15.131:0/126379657 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f1420085fb0 con 0x7f1428159a70 2023-12-26T22:53:26.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.328+0000 7f14277fe700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1414066f30 0x7f14140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.329+0000 7f14277fe700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1414066f30 0x7f14140693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f141800adc0 tx=0x7f1418006f90 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:26.334 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.332+0000 7f140f7fe700 1 -- 172.21.15.131:0/126379657 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14200b75a0 con 0x7f1428159a70 2023-12-26T22:53:26.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.522+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14280cac70 con 0x7f1414066f30 2023-12-26T22:53:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.532+0000 7f140f7fe700 1 -- 172.21.15.131:0/126379657 <== mgr.8455 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289601 (secure 0 0 0) 0x7f14280cac70 con 0x7f1414066f30 2023-12-26T22:53:26.538 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:26.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.536+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1414066f30 msgr2=0x7f14140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.536+0000 7f142dfff700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1414066f30 0x7f14140693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f141800adc0 tx=0x7f1418006f90 comp rx=0 tx=0).stop 2023-12-26T22:53:26.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.536+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1428159a70 msgr2=0x7f142815ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.536+0000 7f142dfff700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1428159a70 0x7f142815ca00 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f1428089710 tx=0x7f1420017000 comp rx=0 tx=0).stop 2023-12-26T22:53:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.537+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 shutdown_connections 2023-12-26T22:53:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.537+0000 7f142dfff700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f1414066f30 0x7f14140693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.537+0000 7f142dfff700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1428159a70 0x7f142815ca00 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.537+0000 7f142dfff700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14280b0870 0x7f142815dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.537+0000 7f142dfff700 1 --2- 172.21.15.131:0/126379657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14281596a0 0x7f142815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.537+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 >> 172.21.15.131:0/126379657 conn(0x7f142806ece0 msgr2=0x7f14280cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.537+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 shutdown_connections 2023-12-26T22:53:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.537+0000 7f142dfff700 1 -- 172.21.15.131:0/126379657 wait complete. 2023-12-26T22:53:26.547 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T22:53:26.554 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T22:53:25.934257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2470248,"kb_used_data":18216,"kb_used_omap":0,"kb_used_meta":2451968,"kb_avail":752504472,"statfs":{"total":773094113280,"available":770564579328,"internally_reserved":0,"allocated":18653184,"data_stored":9859412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2510815232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.770485"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222507+0000","last_change":"2023-12-26T22:53:22.218719+0000","last_active":"2023-12-26T22:53:22.222507+0000","last_peered":"2023-12-26T22:53:22.222507+0000","last_clean":"2023-12-26T22:53:22.222507+0000","last_became_active":"2023-12-26T22:53:22.216707+0000","last_became_peered":"2023-12-26T22:53:22.216707+0000","last_unstale":"2023-12-26T22:53:22.222507+0000","last_undegraded":"2023-12-26T22:53:22.222507+0000","last_fullsized":"2023-12-26T22:53:22.222507+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:21.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215945+0000","last_change":"2023-12-26T22:53:22.213713+0000","last_active":"2023-12-26T22:53:22.215945+0000","last_peered":"2023-12-26T22:53:22.215945+0000","last_clean":"2023-12-26T22:53:22.215945+0000","last_became_active":"2023-12-26T22:53:22.213116+0000","last_became_peered":"2023-12-26T22:53:22.213116+0000","last_unstale":"2023-12-26T22:53:22.215945+0000","last_undegraded":"2023-12-26T22:53:22.215945+0000","last_fullsized":"2023-12-26T22:53:22.215945+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:47.075927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226570+0000","last_change":"2023-12-26T22:53:22.222197+0000","last_active":"2023-12-26T22:53:22.226570+0000","last_peered":"2023-12-26T22:53:22.226570+0000","last_clean":"2023-12-26T22:53:22.226570+0000","last_became_active":"2023-12-26T22:53:22.221145+0000","last_became_peered":"2023-12-26T22:53:22.221145+0000","last_unstale":"2023-12-26T22:53:22.226570+0000","last_undegraded":"2023-12-26T22:53:22.226570+0000","last_fullsized":"2023-12-26T22:53:22.226570+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:42:11.111318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227579+0000","last_change":"2023-12-26T22:53:22.225552+0000","last_active":"2023-12-26T22:53:22.227579+0000","last_peered":"2023-12-26T22:53:22.227579+0000","last_clean":"2023-12-26T22:53:22.227579+0000","last_became_active":"2023-12-26T22:53:22.225275+0000","last_became_peered":"2023-12-26T22:53:22.225275+0000","last_unstale":"2023-12-26T22:53:22.227579+0000","last_undegraded":"2023-12-26T22:53:22.227579+0000","last_fullsized":"2023-12-26T22:53:22.227579+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:46:20.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.207676+0000","last_change":"2023-12-26T22:53:22.202198+0000","last_active":"2023-12-26T22:53:22.207676+0000","last_peered":"2023-12-26T22:53:22.207676+0000","last_clean":"2023-12-26T22:53:22.207676+0000","last_became_active":"2023-12-26T22:53:22.201894+0000","last_became_peered":"2023-12-26T22:53:22.201894+0000","last_unstale":"2023-12-26T22:53:22.207676+0000","last_undegraded":"2023-12-26T22:53:22.207676+0000","last_fullsized":"2023-12-26T22:53:22.207676+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:03:27.336647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222599+0000","last_change":"2023-12-26T22:53:22.221789+0000","last_active":"2023-12-26T22:53:22.222599+0000","last_peered":"2023-12-26T22:53:22.222599+0000","last_clean":"2023-12-26T22:53:22.222599+0000","last_became_active":"2023-12-26T22:53:22.220854+0000","last_became_peered":"2023-12-26T22:53:22.220854+0000","last_unstale":"2023-12-26T22:53:22.222599+0000","last_undegraded":"2023-12-26T22:53:22.222599+0000","last_fullsized":"2023-12-26T22:53:22.222599+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:15:10.647047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.209035+0000","last_change":"2023-12-26T22:53:22.204079+0000","last_active":"2023-12-26T22:53:22.209035+0000","last_peered":"2023-12-26T22:53:22.209035+0000","last_clean":"2023-12-26T22:53:22.209035+0000","last_became_active":"2023-12-26T22:53:22.203814+0000","last_became_peered":"2023-12-26T22:53:22.203814+0000","last_unstale":"2023-12-26T22:53:22.209035+0000","last_undegraded":"2023-12-26T22:53:22.209035+0000","last_fullsized":"2023-12-26T22:53:22.209035+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:11:20.118109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222695+0000","last_change":"2023-12-26T22:53:22.219496+0000","last_active":"2023-12-26T22:53:22.222695+0000","last_peered":"2023-12-26T22:53:22.222695+0000","last_clean":"2023-12-26T22:53:22.222695+0000","last_became_active":"2023-12-26T22:53:22.216460+0000","last_became_peered":"2023-12-26T22:53:22.216460+0000","last_unstale":"2023-12-26T22:53:22.222695+0000","last_undegraded":"2023-12-26T22:53:22.222695+0000","last_fullsized":"2023-12-26T22:53:22.222695+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:00:02.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.211364+0000","last_change":"2023-12-26T22:53:22.208422+0000","last_active":"2023-12-26T22:53:22.211364+0000","last_peered":"2023-12-26T22:53:22.211364+0000","last_clean":"2023-12-26T22:53:22.211364+0000","last_became_active":"2023-12-26T22:53:22.208133+0000","last_became_peered":"2023-12-26T22:53:22.208133+0000","last_unstale":"2023-12-26T22:53:22.211364+0000","last_undegraded":"2023-12-26T22:53:22.211364+0000","last_fullsized":"2023-12-26T22:53:22.211364+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:40:05.458555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226649+0000","last_change":"2023-12-26T22:53:22.222498+0000","last_active":"2023-12-26T22:53:22.226649+0000","last_peered":"2023-12-26T22:53:22.226649+0000","last_clean":"2023-12-26T22:53:22.226649+0000","last_became_active":"2023-12-26T22:53:22.220712+0000","last_became_peered":"2023-12-26T22:53:22.220712+0000","last_unstale":"2023-12-26T22:53:22.226649+0000","last_undegraded":"2023-12-26T22:53:22.226649+0000","last_fullsized":"2023-12-26T22:53:22.226649+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:40:36.155981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.211717+0000","last_change":"2023-12-26T22:53:22.209184+0000","last_active":"2023-12-26T22:53:22.211717+0000","last_peered":"2023-12-26T22:53:22.211717+0000","last_clean":"2023-12-26T22:53:22.211717+0000","last_became_active":"2023-12-26T22:53:22.208890+0000","last_became_peered":"2023-12-26T22:53:22.208890+0000","last_unstale":"2023-12-26T22:53:22.211717+0000","last_undegraded":"2023-12-26T22:53:22.211717+0000","last_fullsized":"2023-12-26T22:53:22.211717+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:46:37.041463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.216836+0000","last_change":"2023-12-26T22:53:22.209841+0000","last_active":"2023-12-26T22:53:22.216836+0000","last_peered":"2023-12-26T22:53:22.216836+0000","last_clean":"2023-12-26T22:53:22.216836+0000","last_became_active":"2023-12-26T22:53:22.209492+0000","last_became_peered":"2023-12-26T22:53:22.209492+0000","last_unstale":"2023-12-26T22:53:22.216836+0000","last_undegraded":"2023-12-26T22:53:22.216836+0000","last_fullsized":"2023-12-26T22:53:22.216836+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:08:38.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.206443+0000","last_change":"2023-12-26T22:53:22.201377+0000","last_active":"2023-12-26T22:53:22.206443+0000","last_peered":"2023-12-26T22:53:22.206443+0000","last_clean":"2023-12-26T22:53:22.206443+0000","last_became_active":"2023-12-26T22:53:22.200053+0000","last_became_peered":"2023-12-26T22:53:22.200053+0000","last_unstale":"2023-12-26T22:53:22.206443+0000","last_undegraded":"2023-12-26T22:53:22.206443+0000","last_fullsized":"2023-12-26T22:53:22.206443+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:41.862685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226608+0000","last_change":"2023-12-26T22:53:22.221280+0000","last_active":"2023-12-26T22:53:22.226608+0000","last_peered":"2023-12-26T22:53:22.226608+0000","last_clean":"2023-12-26T22:53:22.226608+0000","last_became_active":"2023-12-26T22:53:22.220915+0000","last_became_peered":"2023-12-26T22:53:22.220915+0000","last_unstale":"2023-12-26T22:53:22.226608+0000","last_undegraded":"2023-12-26T22:53:22.226608+0000","last_fullsized":"2023-12-26T22:53:22.226608+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:38:25.089802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215809+0000","last_change":"2023-12-26T22:53:22.213531+0000","last_active":"2023-12-26T22:53:22.215809+0000","last_peered":"2023-12-26T22:53:22.215809+0000","last_clean":"2023-12-26T22:53:22.215809+0000","last_became_active":"2023-12-26T22:53:22.213236+0000","last_became_peered":"2023-12-26T22:53:22.213236+0000","last_unstale":"2023-12-26T22:53:22.215809+0000","last_undegraded":"2023-12-26T22:53:22.215809+0000","last_fullsized":"2023-12-26T22:53:22.215809+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:33:04.388734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.206665+0000","last_change":"2023-12-26T22:53:22.201444+0000","last_active":"2023-12-26T22:53:22.206665+0000","last_peered":"2023-12-26T22:53:22.206665+0000","last_clean":"2023-12-26T22:53:22.206665+0000","last_became_active":"2023-12-26T22:53:22.201175+0000","last_became_peered":"2023-12-26T22:53:22.201175+0000","last_unstale":"2023-12-26T22:53:22.206665+0000","last_undegraded":"2023-12-26T22:53:22.206665+0000","last_fullsized":"2023-12-26T22:53:22.206665+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:03.484408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.228979+0000","last_change":"2023-12-26T22:53:22.227064+0000","last_active":"2023-12-26T22:53:22.228979+0000","last_peered":"2023-12-26T22:53:22.228979+0000","last_clean":"2023-12-26T22:53:22.228979+0000","last_became_active":"2023-12-26T22:53:22.224360+0000","last_became_peered":"2023-12-26T22:53:22.224360+0000","last_unstale":"2023-12-26T22:53:22.228979+0000","last_undegraded":"2023-12-26T22:53:22.228979+0000","last_fullsized":"2023-12-26T22:53:22.228979+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:01:39.571028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215916+0000","last_change":"2023-12-26T22:53:22.212661+0000","last_active":"2023-12-26T22:53:22.215916+0000","last_peered":"2023-12-26T22:53:22.215916+0000","last_clean":"2023-12-26T22:53:22.215916+0000","last_became_active":"2023-12-26T22:53:22.212405+0000","last_became_peered":"2023-12-26T22:53:22.212405+0000","last_unstale":"2023-12-26T22:53:22.215916+0000","last_undegraded":"2023-12-26T22:53:22.215916+0000","last_fullsized":"2023-12-26T22:53:22.215916+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:15:10.647047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220429+0000","last_change":"2023-12-26T22:53:22.211452+0000","last_active":"2023-12-26T22:53:22.220429+0000","last_peered":"2023-12-26T22:53:22.220429+0000","last_clean":"2023-12-26T22:53:22.220429+0000","last_became_active":"2023-12-26T22:53:22.211193+0000","last_became_peered":"2023-12-26T22:53:22.211193+0000","last_unstale":"2023-12-26T22:53:22.220429+0000","last_undegraded":"2023-12-26T22:53:22.220429+0000","last_fullsized":"2023-12-26T22:53:22.220429+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:37:07.113041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220431+0000","last_change":"2023-12-26T22:53:22.209747+0000","last_active":"2023-12-26T22:53:22.220431+0000","last_peered":"2023-12-26T22:53:22.220431+0000","last_clean":"2023-12-26T22:53:22.220431+0000","last_became_active":"2023-12-26T22:53:22.209476+0000","last_became_peered":"2023-12-26T22:53:22.209476+0000","last_unstale":"2023-12-26T22:53:22.220431+0000","last_undegraded":"2023-12-26T22:53:22.220431+0000","last_fullsized":"2023-12-26T22:53:22.220431+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:43:14.771756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226933+0000","last_change":"2023-12-26T22:53:22.223611+0000","last_active":"2023-12-26T22:53:22.226933+0000","last_peered":"2023-12-26T22:53:22.226933+0000","last_clean":"2023-12-26T22:53:22.226933+0000","last_became_active":"2023-12-26T22:53:22.223350+0000","last_became_peered":"2023-12-26T22:53:22.223350+0000","last_unstale":"2023-12-26T22:53:22.226933+0000","last_undegraded":"2023-12-26T22:53:22.226933+0000","last_fullsized":"2023-12-26T22:53:22.226933+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:04:24.295277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.217605+0000","last_change":"2023-12-26T22:53:22.210953+0000","last_active":"2023-12-26T22:53:22.217605+0000","last_peered":"2023-12-26T22:53:22.217605+0000","last_clean":"2023-12-26T22:53:22.217605+0000","last_became_active":"2023-12-26T22:53:22.210684+0000","last_became_peered":"2023-12-26T22:53:22.210684+0000","last_unstale":"2023-12-26T22:53:22.217605+0000","last_undegraded":"2023-12-26T22:53:22.217605+0000","last_fullsized":"2023-12-26T22:53:22.217605+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:26.829735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222276+0000","last_change":"2023-12-26T22:53:22.216903+0000","last_active":"2023-12-26T22:53:22.222276+0000","last_peered":"2023-12-26T22:53:22.222276+0000","last_clean":"2023-12-26T22:53:22.222276+0000","last_became_active":"2023-12-26T22:53:22.216623+0000","last_became_peered":"2023-12-26T22:53:22.216623+0000","last_unstale":"2023-12-26T22:53:22.222276+0000","last_undegraded":"2023-12-26T22:53:22.222276+0000","last_fullsized":"2023-12-26T22:53:22.222276+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:59:45.374415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.211413+0000","last_change":"2023-12-26T22:53:22.205001+0000","last_active":"2023-12-26T22:53:22.211413+0000","last_peered":"2023-12-26T22:53:22.211413+0000","last_clean":"2023-12-26T22:53:22.211413+0000","last_became_active":"2023-12-26T22:53:22.204732+0000","last_became_peered":"2023-12-26T22:53:22.204732+0000","last_unstale":"2023-12-26T22:53:22.211413+0000","last_undegraded":"2023-12-26T22:53:22.211413+0000","last_fullsized":"2023-12-26T22:53:22.211413+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:40:05.458555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.213422+0000","last_change":"2023-12-26T22:53:22.206415+0000","last_active":"2023-12-26T22:53:22.213422+0000","last_peered":"2023-12-26T22:53:22.213422+0000","last_clean":"2023-12-26T22:53:22.213422+0000","last_became_active":"2023-12-26T22:53:22.206105+0000","last_became_peered":"2023-12-26T22:53:22.206105+0000","last_unstale":"2023-12-26T22:53:22.213422+0000","last_undegraded":"2023-12-26T22:53:22.213422+0000","last_fullsized":"2023-12-26T22:53:22.213422+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:23:52.675851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.221081+0000","last_change":"2023-12-26T22:53:22.215818+0000","last_active":"2023-12-26T22:53:22.221081+0000","last_peered":"2023-12-26T22:53:22.221081+0000","last_clean":"2023-12-26T22:53:22.221081+0000","last_became_active":"2023-12-26T22:53:22.214058+0000","last_became_peered":"2023-12-26T22:53:22.214058+0000","last_unstale":"2023-12-26T22:53:22.221081+0000","last_undegraded":"2023-12-26T22:53:22.221081+0000","last_fullsized":"2023-12-26T22:53:22.221081+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:16:24.302086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212508+0000","last_change":"2023-12-26T22:53:22.205604+0000","last_active":"2023-12-26T22:53:22.212508+0000","last_peered":"2023-12-26T22:53:22.212508+0000","last_clean":"2023-12-26T22:53:22.212508+0000","last_became_active":"2023-12-26T22:53:22.205335+0000","last_became_peered":"2023-12-26T22:53:22.205335+0000","last_unstale":"2023-12-26T22:53:22.212508+0000","last_undegraded":"2023-12-26T22:53:22.212508+0000","last_fullsized":"2023-12-26T22:53:22.212508+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:30:20.461910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.214579+0000","last_change":"2023-12-26T22:53:22.212711+0000","last_active":"2023-12-26T22:53:22.214579+0000","last_peered":"2023-12-26T22:53:22.214579+0000","last_clean":"2023-12-26T22:53:22.214579+0000","last_became_active":"2023-12-26T22:53:22.212237+0000","last_became_peered":"2023-12-26T22:53:22.212237+0000","last_unstale":"2023-12-26T22:53:22.214579+0000","last_undegraded":"2023-12-26T22:53:22.214579+0000","last_fullsized":"2023-12-26T22:53:22.214579+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:16:24.302086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220461+0000","last_change":"2023-12-26T22:53:22.211279+0000","last_active":"2023-12-26T22:53:22.220461+0000","last_peered":"2023-12-26T22:53:22.220461+0000","last_clean":"2023-12-26T22:53:22.220461+0000","last_became_active":"2023-12-26T22:53:22.211028+0000","last_became_peered":"2023-12-26T22:53:22.211028+0000","last_unstale":"2023-12-26T22:53:22.220461+0000","last_undegraded":"2023-12-26T22:53:22.220461+0000","last_fullsized":"2023-12-26T22:53:22.220461+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:38:15.775091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.217628+0000","last_change":"2023-12-26T22:53:22.210994+0000","last_active":"2023-12-26T22:53:22.217628+0000","last_peered":"2023-12-26T22:53:22.217628+0000","last_clean":"2023-12-26T22:53:22.217628+0000","last_became_active":"2023-12-26T22:53:22.210707+0000","last_became_peered":"2023-12-26T22:53:22.210707+0000","last_unstale":"2023-12-26T22:53:22.217628+0000","last_undegraded":"2023-12-26T22:53:22.217628+0000","last_fullsized":"2023-12-26T22:53:22.217628+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:28:18.916576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212240+0000","last_change":"2023-12-26T22:53:22.202279+0000","last_active":"2023-12-26T22:53:22.212240+0000","last_peered":"2023-12-26T22:53:22.212240+0000","last_clean":"2023-12-26T22:53:22.212240+0000","last_became_active":"2023-12-26T22:53:22.202024+0000","last_became_peered":"2023-12-26T22:53:22.202024+0000","last_unstale":"2023-12-26T22:53:22.212240+0000","last_undegraded":"2023-12-26T22:53:22.212240+0000","last_fullsized":"2023-12-26T22:53:22.212240+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:33:12.690776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212994+0000","last_change":"2023-12-26T22:53:22.204223+0000","last_active":"2023-12-26T22:53:22.212994+0000","last_peered":"2023-12-26T22:53:22.212994+0000","last_clean":"2023-12-26T22:53:22.212994+0000","last_became_active":"2023-12-26T22:53:22.203560+0000","last_became_peered":"2023-12-26T22:53:22.203560+0000","last_unstale":"2023-12-26T22:53:22.212994+0000","last_undegraded":"2023-12-26T22:53:22.212994+0000","last_fullsized":"2023-12-26T22:53:22.212994+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:51:39.335927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.206642+0000","last_change":"2023-12-26T22:53:22.202612+0000","last_active":"2023-12-26T22:53:22.206642+0000","last_peered":"2023-12-26T22:53:22.206642+0000","last_clean":"2023-12-26T22:53:22.206642+0000","last_became_active":"2023-12-26T22:53:22.202092+0000","last_became_peered":"2023-12-26T22:53:22.202092+0000","last_unstale":"2023-12-26T22:53:22.206642+0000","last_undegraded":"2023-12-26T22:53:22.206642+0000","last_fullsized":"2023-12-26T22:53:22.206642+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:46:37.041463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"174'4","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.228289+0000","last_change":"2023-12-26T22:53:21.180537+0000","last_active":"2023-12-26T22:53:22.228289+0000","last_peered":"2023-12-26T22:53:22.228289+0000","last_clean":"2023-12-26T22:53:22.228289+0000","last_became_active":"2023-12-26T22:53:21.180268+0000","last_became_peered":"2023-12-26T22:53:21.180268+0000","last_unstale":"2023-12-26T22:53:22.228289+0000","last_undegraded":"2023-12-26T22:53:22.228289+0000","last_fullsized":"2023-12-26T22:53:22.228289+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:04:13.843581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215502+0000","last_change":"2023-12-26T22:53:22.212871+0000","last_active":"2023-12-26T22:53:22.215502+0000","last_peered":"2023-12-26T22:53:22.215502+0000","last_clean":"2023-12-26T22:53:22.215502+0000","last_became_active":"2023-12-26T22:53:22.212026+0000","last_became_peered":"2023-12-26T22:53:22.212026+0000","last_unstale":"2023-12-26T22:53:22.215502+0000","last_undegraded":"2023-12-26T22:53:22.215502+0000","last_fullsized":"2023-12-26T22:53:22.215502+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:54:07.840473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223543+0000","last_change":"2023-12-26T22:53:21.186408+0000","last_active":"2023-12-26T22:53:22.223543+0000","last_peered":"2023-12-26T22:53:22.223543+0000","last_clean":"2023-12-26T22:53:22.223543+0000","last_became_active":"2023-12-26T22:53:21.181260+0000","last_became_peered":"2023-12-26T22:53:21.181260+0000","last_unstale":"2023-12-26T22:53:22.223543+0000","last_undegraded":"2023-12-26T22:53:22.223543+0000","last_fullsized":"2023-12-26T22:53:22.223543+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:11.585142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215267+0000","last_change":"2023-12-26T22:53:22.213679+0000","last_active":"2023-12-26T22:53:22.215267+0000","last_peered":"2023-12-26T22:53:22.215267+0000","last_clean":"2023-12-26T22:53:22.215267+0000","last_became_active":"2023-12-26T22:53:22.212846+0000","last_became_peered":"2023-12-26T22:53:22.212846+0000","last_unstale":"2023-12-26T22:53:22.215267+0000","last_undegraded":"2023-12-26T22:53:22.215267+0000","last_fullsized":"2023-12-26T22:53:22.215267+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:41:24.790254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223851+0000","last_change":"2023-12-26T22:53:21.189072+0000","last_active":"2023-12-26T22:53:22.223851+0000","last_peered":"2023-12-26T22:53:22.223851+0000","last_clean":"2023-12-26T22:53:22.223851+0000","last_became_active":"2023-12-26T22:53:21.188804+0000","last_became_peered":"2023-12-26T22:53:21.188804+0000","last_unstale":"2023-12-26T22:53:22.223851+0000","last_undegraded":"2023-12-26T22:53:22.223851+0000","last_fullsized":"2023-12-26T22:53:22.223851+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:20:14.854866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227927+0000","last_change":"2023-12-26T22:53:22.226257+0000","last_active":"2023-12-26T22:53:22.227927+0000","last_peered":"2023-12-26T22:53:22.227927+0000","last_clean":"2023-12-26T22:53:22.227927+0000","last_became_active":"2023-12-26T22:53:22.225308+0000","last_became_peered":"2023-12-26T22:53:22.225308+0000","last_unstale":"2023-12-26T22:53:22.227927+0000","last_undegraded":"2023-12-26T22:53:22.227927+0000","last_fullsized":"2023-12-26T22:53:22.227927+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:44:01.231663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223683+0000","last_change":"2023-12-26T22:53:21.184599+0000","last_active":"2023-12-26T22:53:22.223683+0000","last_peered":"2023-12-26T22:53:22.223683+0000","last_clean":"2023-12-26T22:53:22.223683+0000","last_became_active":"2023-12-26T22:53:21.182198+0000","last_became_peered":"2023-12-26T22:53:21.182198+0000","last_unstale":"2023-12-26T22:53:22.223683+0000","last_undegraded":"2023-12-26T22:53:22.223683+0000","last_fullsized":"2023-12-26T22:53:22.223683+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:44.576855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212524+0000","last_change":"2023-12-26T22:53:22.204357+0000","last_active":"2023-12-26T22:53:22.212524+0000","last_peered":"2023-12-26T22:53:22.212524+0000","last_clean":"2023-12-26T22:53:22.212524+0000","last_became_active":"2023-12-26T22:53:22.204074+0000","last_became_peered":"2023-12-26T22:53:22.204074+0000","last_unstale":"2023-12-26T22:53:22.212524+0000","last_undegraded":"2023-12-26T22:53:22.212524+0000","last_fullsized":"2023-12-26T22:53:22.212524+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:38:15.775091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.216522+0000","last_change":"2023-12-26T22:53:21.190029+0000","last_active":"2023-12-26T22:53:22.216522+0000","last_peered":"2023-12-26T22:53:22.216522+0000","last_clean":"2023-12-26T22:53:22.216522+0000","last_became_active":"2023-12-26T22:53:21.189760+0000","last_became_peered":"2023-12-26T22:53:21.189760+0000","last_unstale":"2023-12-26T22:53:22.216522+0000","last_undegraded":"2023-12-26T22:53:22.216522+0000","last_fullsized":"2023-12-26T22:53:22.216522+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:58:01.085416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.230403+0000","last_change":"2023-12-26T22:53:22.227280+0000","last_active":"2023-12-26T22:53:22.230403+0000","last_peered":"2023-12-26T22:53:22.230403+0000","last_clean":"2023-12-26T22:53:22.230403+0000","last_became_active":"2023-12-26T22:53:22.226925+0000","last_became_peered":"2023-12-26T22:53:22.226925+0000","last_unstale":"2023-12-26T22:53:22.230403+0000","last_undegraded":"2023-12-26T22:53:22.230403+0000","last_fullsized":"2023-12-26T22:53:22.230403+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:58:27.027125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.221338+0000","last_change":"2023-12-26T22:53:21.188316+0000","last_active":"2023-12-26T22:53:22.221338+0000","last_peered":"2023-12-26T22:53:22.221338+0000","last_clean":"2023-12-26T22:53:22.221338+0000","last_became_active":"2023-12-26T22:53:21.186623+0000","last_became_peered":"2023-12-26T22:53:21.186623+0000","last_unstale":"2023-12-26T22:53:22.221338+0000","last_undegraded":"2023-12-26T22:53:22.221338+0000","last_fullsized":"2023-12-26T22:53:22.221338+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:56:00.710341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.201360+0000","last_change":"2023-12-26T22:53:22.199282+0000","last_active":"2023-12-26T22:53:22.201360+0000","last_peered":"2023-12-26T22:53:22.201360+0000","last_clean":"2023-12-26T22:53:22.201360+0000","last_became_active":"2023-12-26T22:53:22.199009+0000","last_became_peered":"2023-12-26T22:53:22.199009+0000","last_unstale":"2023-12-26T22:53:22.201360+0000","last_undegraded":"2023-12-26T22:53:22.201360+0000","last_fullsized":"2023-12-26T22:53:22.201360+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:47.328793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.204651+0000","last_change":"2023-12-26T22:53:21.190195+0000","last_active":"2023-12-26T22:53:22.204651+0000","last_peered":"2023-12-26T22:53:22.204651+0000","last_clean":"2023-12-26T22:53:22.204651+0000","last_became_active":"2023-12-26T22:53:21.188813+0000","last_became_peered":"2023-12-26T22:53:21.188813+0000","last_unstale":"2023-12-26T22:53:22.204651+0000","last_undegraded":"2023-12-26T22:53:22.204651+0000","last_fullsized":"2023-12-26T22:53:22.204651+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:14:49.420581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222255+0000","last_change":"2023-12-26T22:53:22.216518+0000","last_active":"2023-12-26T22:53:22.222255+0000","last_peered":"2023-12-26T22:53:22.222255+0000","last_clean":"2023-12-26T22:53:22.222255+0000","last_became_active":"2023-12-26T22:53:22.216099+0000","last_became_peered":"2023-12-26T22:53:22.216099+0000","last_unstale":"2023-12-26T22:53:22.222255+0000","last_undegraded":"2023-12-26T22:53:22.222255+0000","last_fullsized":"2023-12-26T22:53:22.222255+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:46:32.353169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.213471+0000","last_change":"2023-12-26T22:53:21.178758+0000","last_active":"2023-12-26T22:53:22.213471+0000","last_peered":"2023-12-26T22:53:22.213471+0000","last_clean":"2023-12-26T22:53:22.213471+0000","last_became_active":"2023-12-26T22:53:21.178506+0000","last_became_peered":"2023-12-26T22:53:21.178506+0000","last_unstale":"2023-12-26T22:53:22.213471+0000","last_undegraded":"2023-12-26T22:53:22.213471+0000","last_fullsized":"2023-12-26T22:53:22.213471+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:55:31.344157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.219163+0000","last_change":"2023-12-26T22:53:22.210228+0000","last_active":"2023-12-26T22:53:22.219163+0000","last_peered":"2023-12-26T22:53:22.219163+0000","last_clean":"2023-12-26T22:53:22.219163+0000","last_became_active":"2023-12-26T22:53:22.209949+0000","last_became_peered":"2023-12-26T22:53:22.209949+0000","last_unstale":"2023-12-26T22:53:22.219163+0000","last_undegraded":"2023-12-26T22:53:22.219163+0000","last_fullsized":"2023-12-26T22:53:22.219163+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:39:20.300743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.205453+0000","last_change":"2023-12-26T22:53:21.185824+0000","last_active":"2023-12-26T22:53:22.205453+0000","last_peered":"2023-12-26T22:53:22.205453+0000","last_clean":"2023-12-26T22:53:22.205453+0000","last_became_active":"2023-12-26T22:53:21.185084+0000","last_became_peered":"2023-12-26T22:53:21.185084+0000","last_unstale":"2023-12-26T22:53:22.205453+0000","last_undegraded":"2023-12-26T22:53:22.205453+0000","last_fullsized":"2023-12-26T22:53:22.205453+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:33:26.481774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226824+0000","last_change":"2023-12-26T22:53:22.224425+0000","last_active":"2023-12-26T22:53:22.226824+0000","last_peered":"2023-12-26T22:53:22.226824+0000","last_clean":"2023-12-26T22:53:22.226824+0000","last_became_active":"2023-12-26T22:53:22.224109+0000","last_became_peered":"2023-12-26T22:53:22.224109+0000","last_unstale":"2023-12-26T22:53:22.226824+0000","last_undegraded":"2023-12-26T22:53:22.226824+0000","last_fullsized":"2023-12-26T22:53:22.226824+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:05:53.082200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.194555+0000","last_change":"2023-12-26T22:53:21.192124+0000","last_active":"2023-12-26T22:53:22.194555+0000","last_peered":"2023-12-26T22:53:22.194555+0000","last_clean":"2023-12-26T22:53:22.194555+0000","last_became_active":"2023-12-26T22:53:21.191138+0000","last_became_peered":"2023-12-26T22:53:21.191138+0000","last_unstale":"2023-12-26T22:53:22.194555+0000","last_undegraded":"2023-12-26T22:53:22.194555+0000","last_fullsized":"2023-12-26T22:53:22.194555+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:24:00.638948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.216857+0000","last_change":"2023-12-26T22:53:22.214251+0000","last_active":"2023-12-26T22:53:22.216857+0000","last_peered":"2023-12-26T22:53:22.216857+0000","last_clean":"2023-12-26T22:53:22.216857+0000","last_became_active":"2023-12-26T22:53:22.213841+0000","last_became_peered":"2023-12-26T22:53:22.213841+0000","last_unstale":"2023-12-26T22:53:22.216857+0000","last_undegraded":"2023-12-26T22:53:22.216857+0000","last_fullsized":"2023-12-26T22:53:22.216857+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:32:49.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"174'6","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.225245+0000","last_change":"2023-12-26T22:53:21.189013+0000","last_active":"2023-12-26T22:53:22.225245+0000","last_peered":"2023-12-26T22:53:22.225245+0000","last_clean":"2023-12-26T22:53:22.225245+0000","last_became_active":"2023-12-26T22:53:21.187801+0000","last_became_peered":"2023-12-26T22:53:21.187801+0000","last_unstale":"2023-12-26T22:53:22.225245+0000","last_undegraded":"2023-12-26T22:53:22.225245+0000","last_fullsized":"2023-12-26T22:53:22.225245+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:40.853740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.219699+0000","last_change":"2023-12-26T22:53:22.212762+0000","last_active":"2023-12-26T22:53:22.219699+0000","last_peered":"2023-12-26T22:53:22.219699+0000","last_clean":"2023-12-26T22:53:22.219699+0000","last_became_active":"2023-12-26T22:53:22.211143+0000","last_became_peered":"2023-12-26T22:53:22.211143+0000","last_unstale":"2023-12-26T22:53:22.219699+0000","last_undegraded":"2023-12-26T22:53:22.219699+0000","last_fullsized":"2023-12-26T22:53:22.219699+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:38:13.416933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223997+0000","last_change":"2023-12-26T22:53:21.187785+0000","last_active":"2023-12-26T22:53:22.223997+0000","last_peered":"2023-12-26T22:53:22.223997+0000","last_clean":"2023-12-26T22:53:22.223997+0000","last_became_active":"2023-12-26T22:53:21.187458+0000","last_became_peered":"2023-12-26T22:53:21.187458+0000","last_unstale":"2023-12-26T22:53:22.223997+0000","last_undegraded":"2023-12-26T22:53:22.223997+0000","last_fullsized":"2023-12-26T22:53:22.223997+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:39.312979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220483+0000","last_change":"2023-12-26T22:53:22.211703+0000","last_active":"2023-12-26T22:53:22.220483+0000","last_peered":"2023-12-26T22:53:22.220483+0000","last_clean":"2023-12-26T22:53:22.220483+0000","last_became_active":"2023-12-26T22:53:22.209722+0000","last_became_peered":"2023-12-26T22:53:22.209722+0000","last_unstale":"2023-12-26T22:53:22.220483+0000","last_undegraded":"2023-12-26T22:53:22.220483+0000","last_fullsized":"2023-12-26T22:53:22.220483+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:33:12.690776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.202025+0000","last_change":"2023-12-26T22:53:21.191509+0000","last_active":"2023-12-26T22:53:22.202025+0000","last_peered":"2023-12-26T22:53:22.202025+0000","last_clean":"2023-12-26T22:53:22.202025+0000","last_became_active":"2023-12-26T22:53:21.190722+0000","last_became_peered":"2023-12-26T22:53:21.190722+0000","last_unstale":"2023-12-26T22:53:22.202025+0000","last_undegraded":"2023-12-26T22:53:22.202025+0000","last_fullsized":"2023-12-26T22:53:22.202025+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:32:48.375268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227856+0000","last_change":"2023-12-26T22:53:22.226206+0000","last_active":"2023-12-26T22:53:22.227856+0000","last_peered":"2023-12-26T22:53:22.227856+0000","last_clean":"2023-12-26T22:53:22.227856+0000","last_became_active":"2023-12-26T22:53:22.224842+0000","last_became_peered":"2023-12-26T22:53:22.224842+0000","last_unstale":"2023-12-26T22:53:22.227856+0000","last_undegraded":"2023-12-26T22:53:22.227856+0000","last_fullsized":"2023-12-26T22:53:22.227856+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:25:41.646188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.192328+0000","last_change":"2023-12-26T22:53:21.183612+0000","last_active":"2023-12-26T22:53:22.192328+0000","last_peered":"2023-12-26T22:53:22.192328+0000","last_clean":"2023-12-26T22:53:22.192328+0000","last_became_active":"2023-12-26T22:53:21.183207+0000","last_became_peered":"2023-12-26T22:53:21.183207+0000","last_unstale":"2023-12-26T22:53:22.192328+0000","last_undegraded":"2023-12-26T22:53:22.192328+0000","last_fullsized":"2023-12-26T22:53:22.192328+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:38.281030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222041+0000","last_change":"2023-12-26T22:53:22.216354+0000","last_active":"2023-12-26T22:53:22.222041+0000","last_peered":"2023-12-26T22:53:22.222041+0000","last_clean":"2023-12-26T22:53:22.222041+0000","last_became_active":"2023-12-26T22:53:22.215484+0000","last_became_peered":"2023-12-26T22:53:22.215484+0000","last_unstale":"2023-12-26T22:53:22.222041+0000","last_undegraded":"2023-12-26T22:53:22.222041+0000","last_fullsized":"2023-12-26T22:53:22.222041+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:18:00.845307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"174'8","reported_seq":18,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.224105+0000","last_change":"2023-12-26T22:53:21.183791+0000","last_active":"2023-12-26T22:53:22.224105+0000","last_peered":"2023-12-26T22:53:22.224105+0000","last_clean":"2023-12-26T22:53:22.224105+0000","last_became_active":"2023-12-26T22:53:21.183533+0000","last_became_peered":"2023-12-26T22:53:21.183533+0000","last_unstale":"2023-12-26T22:53:22.224105+0000","last_undegraded":"2023-12-26T22:53:22.224105+0000","last_fullsized":"2023-12-26T22:53:22.224105+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:38.281030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215937+0000","last_change":"2023-12-26T22:53:22.213174+0000","last_active":"2023-12-26T22:53:22.215937+0000","last_peered":"2023-12-26T22:53:22.215937+0000","last_clean":"2023-12-26T22:53:22.215937+0000","last_became_active":"2023-12-26T22:53:22.212914+0000","last_became_peered":"2023-12-26T22:53:22.212914+0000","last_unstale":"2023-12-26T22:53:22.215937+0000","last_undegraded":"2023-12-26T22:53:22.215937+0000","last_fullsized":"2023-12-26T22:53:22.215937+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:15:46.997797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223111+0000","last_change":"2023-12-26T22:53:21.184831+0000","last_active":"2023-12-26T22:53:22.223111+0000","last_peered":"2023-12-26T22:53:22.223111+0000","last_clean":"2023-12-26T22:53:22.223111+0000","last_became_active":"2023-12-26T22:53:21.184552+0000","last_became_peered":"2023-12-26T22:53:21.184552+0000","last_unstale":"2023-12-26T22:53:22.223111+0000","last_undegraded":"2023-12-26T22:53:22.223111+0000","last_fullsized":"2023-12-26T22:53:22.223111+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:16:23.293140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.213397+0000","last_change":"2023-12-26T22:53:22.204614+0000","last_active":"2023-12-26T22:53:22.213397+0000","last_peered":"2023-12-26T22:53:22.213397+0000","last_clean":"2023-12-26T22:53:22.213397+0000","last_became_active":"2023-12-26T22:53:22.204336+0000","last_became_peered":"2023-12-26T22:53:22.204336+0000","last_unstale":"2023-12-26T22:53:22.213397+0000","last_undegraded":"2023-12-26T22:53:22.213397+0000","last_fullsized":"2023-12-26T22:53:22.213397+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:00:02.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.225045+0000","last_change":"2023-12-26T22:53:21.180443+0000","last_active":"2023-12-26T22:53:22.225045+0000","last_peered":"2023-12-26T22:53:22.225045+0000","last_clean":"2023-12-26T22:53:22.225045+0000","last_became_active":"2023-12-26T22:53:21.180118+0000","last_became_peered":"2023-12-26T22:53:21.180118+0000","last_unstale":"2023-12-26T22:53:22.225045+0000","last_undegraded":"2023-12-26T22:53:22.225045+0000","last_fullsized":"2023-12-26T22:53:22.225045+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:41:12.747519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.208997+0000","last_change":"2023-12-26T22:53:22.204833+0000","last_active":"2023-12-26T22:53:22.208997+0000","last_peered":"2023-12-26T22:53:22.208997+0000","last_clean":"2023-12-26T22:53:22.208997+0000","last_became_active":"2023-12-26T22:53:22.204535+0000","last_became_peered":"2023-12-26T22:53:22.204535+0000","last_unstale":"2023-12-26T22:53:22.208997+0000","last_undegraded":"2023-12-26T22:53:22.208997+0000","last_fullsized":"2023-12-26T22:53:22.208997+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:31:59.141262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.214170+0000","last_change":"2023-12-26T22:53:21.193608+0000","last_active":"2023-12-26T22:53:22.214170+0000","last_peered":"2023-12-26T22:53:22.214170+0000","last_clean":"2023-12-26T22:53:22.214170+0000","last_became_active":"2023-12-26T22:53:21.193231+0000","last_became_peered":"2023-12-26T22:53:21.193231+0000","last_unstale":"2023-12-26T22:53:22.214170+0000","last_undegraded":"2023-12-26T22:53:22.214170+0000","last_fullsized":"2023-12-26T22:53:22.214170+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:03:26.327702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.211698+0000","last_change":"2023-12-26T22:53:22.208605+0000","last_active":"2023-12-26T22:53:22.211698+0000","last_peered":"2023-12-26T22:53:22.211698+0000","last_clean":"2023-12-26T22:53:22.211698+0000","last_became_active":"2023-12-26T22:53:22.208319+0000","last_became_peered":"2023-12-26T22:53:22.208319+0000","last_unstale":"2023-12-26T22:53:22.211698+0000","last_undegraded":"2023-12-26T22:53:22.211698+0000","last_fullsized":"2023-12-26T22:53:22.211698+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:33:04.388734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.192202+0000","last_change":"2023-12-26T22:53:21.189279+0000","last_active":"2023-12-26T22:53:22.192202+0000","last_peered":"2023-12-26T22:53:22.192202+0000","last_clean":"2023-12-26T22:53:22.192202+0000","last_became_active":"2023-12-26T22:53:21.189000+0000","last_became_peered":"2023-12-26T22:53:21.189000+0000","last_unstale":"2023-12-26T22:53:22.192202+0000","last_undegraded":"2023-12-26T22:53:22.192202+0000","last_fullsized":"2023-12-26T22:53:22.192202+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:21:40.023765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.221151+0000","last_change":"2023-12-26T22:53:22.217522+0000","last_active":"2023-12-26T22:53:22.221151+0000","last_peered":"2023-12-26T22:53:22.221151+0000","last_clean":"2023-12-26T22:53:22.221151+0000","last_became_active":"2023-12-26T22:53:22.216906+0000","last_became_peered":"2023-12-26T22:53:22.216906+0000","last_unstale":"2023-12-26T22:53:22.221151+0000","last_undegraded":"2023-12-26T22:53:22.221151+0000","last_fullsized":"2023-12-26T22:53:22.221151+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:02:31.293948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222002+0000","last_change":"2023-12-26T22:53:21.185896+0000","last_active":"2023-12-26T22:53:22.222002+0000","last_peered":"2023-12-26T22:53:22.222002+0000","last_clean":"2023-12-26T22:53:22.222002+0000","last_became_active":"2023-12-26T22:53:21.181249+0000","last_became_peered":"2023-12-26T22:53:21.181249+0000","last_unstale":"2023-12-26T22:53:22.222002+0000","last_undegraded":"2023-12-26T22:53:22.222002+0000","last_fullsized":"2023-12-26T22:53:22.222002+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:56:00.710341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212486+0000","last_change":"2023-12-26T22:53:22.205721+0000","last_active":"2023-12-26T22:53:22.212486+0000","last_peered":"2023-12-26T22:53:22.212486+0000","last_clean":"2023-12-26T22:53:22.212486+0000","last_became_active":"2023-12-26T22:53:22.203612+0000","last_became_peered":"2023-12-26T22:53:22.203612+0000","last_unstale":"2023-12-26T22:53:22.212486+0000","last_undegraded":"2023-12-26T22:53:22.212486+0000","last_fullsized":"2023-12-26T22:53:22.212486+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:21.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.190543+0000","last_change":"2023-12-26T22:53:21.189867+0000","last_active":"2023-12-26T22:53:22.190543+0000","last_peered":"2023-12-26T22:53:22.190543+0000","last_clean":"2023-12-26T22:53:22.190543+0000","last_became_active":"2023-12-26T22:53:21.189379+0000","last_became_peered":"2023-12-26T22:53:21.189379+0000","last_unstale":"2023-12-26T22:53:22.190543+0000","last_undegraded":"2023-12-26T22:53:22.190543+0000","last_fullsized":"2023-12-26T22:53:22.190543+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:58.789465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220180+0000","last_change":"2023-12-26T22:53:22.212321+0000","last_active":"2023-12-26T22:53:22.220180+0000","last_peered":"2023-12-26T22:53:22.220180+0000","last_clean":"2023-12-26T22:53:22.220180+0000","last_became_active":"2023-12-26T22:53:22.211339+0000","last_became_peered":"2023-12-26T22:53:22.211339+0000","last_unstale":"2023-12-26T22:53:22.220180+0000","last_undegraded":"2023-12-26T22:53:22.220180+0000","last_fullsized":"2023-12-26T22:53:22.220180+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:44:03.598595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.190526+0000","last_change":"2023-12-26T22:53:21.185640+0000","last_active":"2023-12-26T22:53:22.190526+0000","last_peered":"2023-12-26T22:53:22.190526+0000","last_clean":"2023-12-26T22:53:22.190526+0000","last_became_active":"2023-12-26T22:53:21.185345+0000","last_became_peered":"2023-12-26T22:53:21.185345+0000","last_unstale":"2023-12-26T22:53:22.190526+0000","last_undegraded":"2023-12-26T22:53:22.190526+0000","last_fullsized":"2023-12-26T22:53:22.190526+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:47:02.377466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212266+0000","last_change":"2023-12-26T22:53:22.202290+0000","last_active":"2023-12-26T22:53:22.212266+0000","last_peered":"2023-12-26T22:53:22.212266+0000","last_clean":"2023-12-26T22:53:22.212266+0000","last_became_active":"2023-12-26T22:53:22.202068+0000","last_became_peered":"2023-12-26T22:53:22.202068+0000","last_unstale":"2023-12-26T22:53:22.212266+0000","last_undegraded":"2023-12-26T22:53:22.212266+0000","last_fullsized":"2023-12-26T22:53:22.212266+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:37:07.113041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.189744+0000","last_change":"2023-12-26T22:53:21.190990+0000","last_active":"2023-12-26T22:53:22.189744+0000","last_peered":"2023-12-26T22:53:22.189744+0000","last_clean":"2023-12-26T22:53:22.189744+0000","last_became_active":"2023-12-26T22:53:21.189321+0000","last_became_peered":"2023-12-26T22:53:21.189321+0000","last_unstale":"2023-12-26T22:53:22.189744+0000","last_undegraded":"2023-12-26T22:53:22.189744+0000","last_fullsized":"2023-12-26T22:53:22.189744+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:16:34.121807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227545+0000","last_change":"2023-12-26T22:53:22.225409+0000","last_active":"2023-12-26T22:53:22.227545+0000","last_peered":"2023-12-26T22:53:22.227545+0000","last_clean":"2023-12-26T22:53:22.227545+0000","last_became_active":"2023-12-26T22:53:22.225113+0000","last_became_peered":"2023-12-26T22:53:22.225113+0000","last_unstale":"2023-12-26T22:53:22.227545+0000","last_undegraded":"2023-12-26T22:53:22.227545+0000","last_fullsized":"2023-12-26T22:53:22.227545+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:50:24.470636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215282+0000","last_change":"2023-12-26T22:53:21.190321+0000","last_active":"2023-12-26T22:53:22.215282+0000","last_peered":"2023-12-26T22:53:22.215282+0000","last_clean":"2023-12-26T22:53:22.215282+0000","last_became_active":"2023-12-26T22:53:21.190022+0000","last_became_peered":"2023-12-26T22:53:21.190022+0000","last_unstale":"2023-12-26T22:53:22.215282+0000","last_undegraded":"2023-12-26T22:53:22.215282+0000","last_fullsized":"2023-12-26T22:53:22.215282+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:13:32.726081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227553+0000","last_change":"2023-12-26T22:53:22.225501+0000","last_active":"2023-12-26T22:53:22.227553+0000","last_peered":"2023-12-26T22:53:22.227553+0000","last_clean":"2023-12-26T22:53:22.227553+0000","last_became_active":"2023-12-26T22:53:22.225027+0000","last_became_peered":"2023-12-26T22:53:22.225027+0000","last_unstale":"2023-12-26T22:53:22.227553+0000","last_undegraded":"2023-12-26T22:53:22.227553+0000","last_fullsized":"2023-12-26T22:53:22.227553+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:45.617260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.193127+0000","last_change":"2023-12-26T22:53:21.184348+0000","last_active":"2023-12-26T22:53:22.193127+0000","last_peered":"2023-12-26T22:53:22.193127+0000","last_clean":"2023-12-26T22:53:22.193127+0000","last_became_active":"2023-12-26T22:53:21.184059+0000","last_became_peered":"2023-12-26T22:53:21.184059+0000","last_unstale":"2023-12-26T22:53:22.193127+0000","last_undegraded":"2023-12-26T22:53:22.193127+0000","last_fullsized":"2023-12-26T22:53:22.193127+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:58:01.085416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222231+0000","last_change":"2023-12-26T22:53:22.218858+0000","last_active":"2023-12-26T22:53:22.222231+0000","last_peered":"2023-12-26T22:53:22.222231+0000","last_clean":"2023-12-26T22:53:22.222231+0000","last_became_active":"2023-12-26T22:53:22.218493+0000","last_became_peered":"2023-12-26T22:53:22.218493+0000","last_unstale":"2023-12-26T22:53:22.222231+0000","last_undegraded":"2023-12-26T22:53:22.222231+0000","last_fullsized":"2023-12-26T22:53:22.222231+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:47:03.386411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.203408+0000","last_change":"2023-12-26T22:53:21.188465+0000","last_active":"2023-12-26T22:53:22.203408+0000","last_peered":"2023-12-26T22:53:22.203408+0000","last_clean":"2023-12-26T22:53:22.203408+0000","last_became_active":"2023-12-26T22:53:21.188074+0000","last_became_peered":"2023-12-26T22:53:21.188074+0000","last_unstale":"2023-12-26T22:53:22.203408+0000","last_undegraded":"2023-12-26T22:53:22.203408+0000","last_fullsized":"2023-12-26T22:53:22.203408+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:31:58.132317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.221826+0000","last_change":"2023-12-26T22:53:22.218790+0000","last_active":"2023-12-26T22:53:22.221826+0000","last_peered":"2023-12-26T22:53:22.221826+0000","last_clean":"2023-12-26T22:53:22.221826+0000","last_became_active":"2023-12-26T22:53:22.216634+0000","last_became_peered":"2023-12-26T22:53:22.216634+0000","last_unstale":"2023-12-26T22:53:22.221826+0000","last_undegraded":"2023-12-26T22:53:22.221826+0000","last_fullsized":"2023-12-26T22:53:22.221826+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:47.328793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.205377+0000","last_change":"2023-12-26T22:53:21.193433+0000","last_active":"2023-12-26T22:53:22.205377+0000","last_peered":"2023-12-26T22:53:22.205377+0000","last_clean":"2023-12-26T22:53:22.205377+0000","last_became_active":"2023-12-26T22:53:21.192925+0000","last_became_peered":"2023-12-26T22:53:21.192925+0000","last_unstale":"2023-12-26T22:53:22.205377+0000","last_undegraded":"2023-12-26T22:53:22.205377+0000","last_fullsized":"2023-12-26T22:53:22.205377+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:52:52.523353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212467+0000","last_change":"2023-12-26T22:53:22.206219+0000","last_active":"2023-12-26T22:53:22.212467+0000","last_peered":"2023-12-26T22:53:22.212467+0000","last_clean":"2023-12-26T22:53:22.212467+0000","last_became_active":"2023-12-26T22:53:22.205942+0000","last_became_peered":"2023-12-26T22:53:22.205942+0000","last_unstale":"2023-12-26T22:53:22.212467+0000","last_undegraded":"2023-12-26T22:53:22.212467+0000","last_fullsized":"2023-12-26T22:53:22.212467+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:10:30.959829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.192241+0000","last_change":"2023-12-26T22:53:21.182336+0000","last_active":"2023-12-26T22:53:22.192241+0000","last_peered":"2023-12-26T22:53:22.192241+0000","last_clean":"2023-12-26T22:53:22.192241+0000","last_became_active":"2023-12-26T22:53:21.182004+0000","last_became_peered":"2023-12-26T22:53:21.182004+0000","last_unstale":"2023-12-26T22:53:22.192241+0000","last_undegraded":"2023-12-26T22:53:22.192241+0000","last_fullsized":"2023-12-26T22:53:22.192241+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:08:16.127123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212783+0000","last_change":"2023-12-26T22:53:22.207411+0000","last_active":"2023-12-26T22:53:22.212783+0000","last_peered":"2023-12-26T22:53:22.212783+0000","last_clean":"2023-12-26T22:53:22.212783+0000","last_became_active":"2023-12-26T22:53:22.207115+0000","last_became_peered":"2023-12-26T22:53:22.207115+0000","last_unstale":"2023-12-26T22:53:22.212783+0000","last_undegraded":"2023-12-26T22:53:22.212783+0000","last_fullsized":"2023-12-26T22:53:22.212783+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:15:41.634326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.213674+0000","last_change":"2023-12-26T22:53:21.188343+0000","last_active":"2023-12-26T22:53:22.213674+0000","last_peered":"2023-12-26T22:53:22.213674+0000","last_clean":"2023-12-26T22:53:22.213674+0000","last_became_active":"2023-12-26T22:53:21.187901+0000","last_became_peered":"2023-12-26T22:53:21.187901+0000","last_unstale":"2023-12-26T22:53:22.213674+0000","last_undegraded":"2023-12-26T22:53:22.213674+0000","last_fullsized":"2023-12-26T22:53:22.213674+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:35:10.928485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223771+0000","last_change":"2023-12-26T22:53:22.218970+0000","last_active":"2023-12-26T22:53:22.223771+0000","last_peered":"2023-12-26T22:53:22.223771+0000","last_clean":"2023-12-26T22:53:22.223771+0000","last_became_active":"2023-12-26T22:53:22.218659+0000","last_became_peered":"2023-12-26T22:53:22.218659+0000","last_unstale":"2023-12-26T22:53:22.223771+0000","last_undegraded":"2023-12-26T22:53:22.223771+0000","last_fullsized":"2023-12-26T22:53:22.223771+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:37:49.053144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.201373+0000","last_change":"2023-12-26T22:53:21.189485+0000","last_active":"2023-12-26T22:53:22.201373+0000","last_peered":"2023-12-26T22:53:22.201373+0000","last_clean":"2023-12-26T22:53:22.201373+0000","last_became_active":"2023-12-26T22:53:21.189196+0000","last_became_peered":"2023-12-26T22:53:21.189196+0000","last_unstale":"2023-12-26T22:53:22.201373+0000","last_undegraded":"2023-12-26T22:53:22.201373+0000","last_fullsized":"2023-12-26T22:53:22.201373+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:01:08.802761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226879+0000","last_change":"2023-12-26T22:53:22.223240+0000","last_active":"2023-12-26T22:53:22.226879+0000","last_peered":"2023-12-26T22:53:22.226879+0000","last_clean":"2023-12-26T22:53:22.226879+0000","last_became_active":"2023-12-26T22:53:22.222927+0000","last_became_peered":"2023-12-26T22:53:22.222927+0000","last_unstale":"2023-12-26T22:53:22.226879+0000","last_undegraded":"2023-12-26T22:53:22.226879+0000","last_fullsized":"2023-12-26T22:53:22.226879+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:22:06.940568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.204299+0000","last_change":"2023-12-26T22:53:21.184118+0000","last_active":"2023-12-26T22:53:22.204299+0000","last_peered":"2023-12-26T22:53:22.204299+0000","last_clean":"2023-12-26T22:53:22.204299+0000","last_became_active":"2023-12-26T22:53:21.181129+0000","last_became_peered":"2023-12-26T22:53:21.181129+0000","last_unstale":"2023-12-26T22:53:22.204299+0000","last_undegraded":"2023-12-26T22:53:22.204299+0000","last_fullsized":"2023-12-26T22:53:22.204299+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:09:39.505337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.225670+0000","last_change":"2023-12-26T22:53:22.224828+0000","last_active":"2023-12-26T22:53:22.225670+0000","last_peered":"2023-12-26T22:53:22.225670+0000","last_clean":"2023-12-26T22:53:22.225670+0000","last_became_active":"2023-12-26T22:53:22.224467+0000","last_became_peered":"2023-12-26T22:53:22.224467+0000","last_unstale":"2023-12-26T22:53:22.225670+0000","last_undegraded":"2023-12-26T22:53:22.225670+0000","last_fullsized":"2023-12-26T22:53:22.225670+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:34:31.962868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"174'3","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.217584+0000","last_change":"2023-12-26T22:53:21.190558+0000","last_active":"2023-12-26T22:53:22.217584+0000","last_peered":"2023-12-26T22:53:22.217584+0000","last_clean":"2023-12-26T22:53:22.217584+0000","last_became_active":"2023-12-26T22:53:21.190308+0000","last_became_peered":"2023-12-26T22:53:21.190308+0000","last_unstale":"2023-12-26T22:53:22.217584+0000","last_undegraded":"2023-12-26T22:53:22.217584+0000","last_fullsized":"2023-12-26T22:53:22.217584+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:40:21.381550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"164'293","reported_seq":927,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.192865+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T22:53:22.192865+0000","last_peered":"2023-12-26T22:53:22.192865+0000","last_clean":"2023-12-26T22:53:22.192865+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T22:53:22.192865+0000","last_undegraded":"2023-12-26T22:53:22.192865+0000","last_fullsized":"2023-12-26T22:53:22.192865+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543057,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307604,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064236,"statfs":{"total":96636764160,"available":96321777664,"internally_reserved":0,"allocated":2174976,"data_stored":1084258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78900000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.5},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.68600000000000005,"15min":0.68899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.41999999999999998},"max":{"1min":0.77100000000000002,"5min":1.02,"15min":1.02},"last":0.45800000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":1.601,"15min":1.601},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73499999999999999,"15min":0.746},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.441},"max":{"1min":0.90200000000000002,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.82499999999999996,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41499999999999998},"max":{"1min":0.88200000000000001,"5min":1.548,"15min":1.548},"last":0.48899999999999999}]},{"osd":3,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.629,"5min":0.58899999999999997,"15min":0.47899999999999998},"max":{"1min":0.88600000000000001,"5min":1.242,"15min":1.242},"last":0.754},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.42799999999999999},"max":{"1min":0.81699999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.74099999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40699999999999997},"max":{"1min":0.84599999999999997,"5min":1.113,"15min":1.113},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46300000000000002,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.34599999999999997},"max":{"1min":0.82699999999999996,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.59499999999999997,"5min":0.55100000000000005,"15min":0.47799999999999998},"max":{"1min":0.94299999999999995,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.79300000000000004}]},{"osd":6,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70399999999999996,"15min":0.69299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85999999999999999,"5min":2.609,"15min":2.609},"last":0.628},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":2.996,"15min":2.996},"last":0.72099999999999997}]}]},{"osd":6,"up_from":15,"seq":64424510353,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307604,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064236,"statfs":{"total":96636764160,"available":96321777664,"internally_reserved":0,"allocated":2174976,"data_stored":1084258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.77100000000000002,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":4.2409999999999997,"5min":4.2409999999999997,"15min":4.2409999999999997},"last":0.69299999999999995}]},{"osd":1,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73099999999999998,"15min":0.72199999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98799999999999999},"last":0.628},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.752},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.50600000000000001},"max":{"1min":0.91800000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.46700000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.65900000000000003}]},{"osd":3,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.80600000000000005,"15min":0.77000000000000002},"min":{"1min":0.57199999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.3369999999999997,"5min":4.3369999999999997,"15min":4.3369999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97999999999999998,"5min":1.012,"15min":1.012},"last":0.74099999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.998,"15min":0.998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.47199999999999998,"5min":0.45500000000000002,"15min":0.40899999999999997},"max":{"1min":0.89400000000000002,"5min":0.92700000000000005,"15min":0.995},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.95999999999999996},"last":0.498},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69899999999999995,"15min":0.69599999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45200000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.47799999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.53000000000000003}]}]},{"osd":1,"up_from":15,"seq":64424510353,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312284,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059556,"statfs":{"total":96636764160,"available":96316985344,"internally_reserved":0,"allocated":2772992,"data_stored":1674696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44800000000000001,"15min":0.46899999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58599999999999997,"5min":0.79900000000000004,"15min":0.88400000000000001},"last":0.495},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.316},"max":{"1min":0.67000000000000004,"5min":0.81000000000000005,"15min":0.84499999999999997},"last":0.53100000000000003}]},{"osd":2,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69799999999999995,"5min":0.87,"15min":0.87},"last":0.59699999999999998}]},{"osd":3,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":0.81000000000000005,"5min":0.877,"15min":0.89800000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83299999999999996,"5min":0.98699999999999999,"15min":1.163},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.745,"15min":0.75600000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":0.91000000000000003,"5min":1.0289999999999999,"15min":1.121},"last":0.81000000000000005}]},{"osd":5,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.80600000000000005,"15min":0.81499999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.502},"max":{"1min":0.93500000000000005,"5min":1.0820000000000001,"15min":1.1419999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.81699999999999995,"15min":0.82499999999999996},"min":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.875,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.81999999999999995,"15min":0.82599999999999996},"min":{"1min":0.56499999999999995,"5min":0.56299999999999994,"15min":0.55000000000000004},"max":{"1min":0.92600000000000005,"5min":1.0760000000000001,"15min":1.2450000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.81200000000000006,"15min":0.83099999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89200000000000002,"5min":1.1259999999999999,"15min":1.2270000000000001},"last":0.77600000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.80100000000000005,"15min":0.81299999999999994},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.55700000000000005},"max":{"1min":0.94499999999999995,"5min":1.107,"15min":1.2689999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.80400000000000005,"15min":0.81599999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.497},"max":{"1min":0.96599999999999997,"5min":1.0660000000000001,"15min":1.1890000000000001},"last":0.64200000000000002}]}]},{"osd":0,"up_from":15,"seq":64424510353,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307608,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064232,"statfs":{"total":96636764160,"available":96321773568,"internally_reserved":0,"allocated":2179072,"data_stored":1084348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.56799999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.439},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.307},"max":{"1min":0.63500000000000001,"5min":0.72399999999999998,"15min":0.86599999999999999},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.504,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64100000000000001,"5min":0.70199999999999996,"15min":0.88800000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.90500000000000003},"last":0.41299999999999998}]},{"osd":3,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.86799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.63300000000000001,"15min":0.63600000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.755,"5min":0.82799999999999996,"15min":0.92200000000000004},"last":0.46400000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":0.873,"5min":1.103,"15min":1.145},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.76400000000000001,"15min":0.78000000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.82799999999999996,"5min":1.1299999999999999,"15min":1.218},"last":0.63100000000000001}]},{"osd":5,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.81299999999999994,"15min":0.82299999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92700000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.81799999999999995,"15min":0.82999999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.50900000000000001},"max":{"1min":0.93100000000000005,"5min":1.218,"15min":1.218},"last":0.77800000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.89100000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78600000000000003,"15min":0.79900000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.83199999999999996,"5min":1.0780000000000001,"15min":1.2350000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.81200000000000006,"15min":0.81499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.90900000000000003,"5min":1.165,"15min":1.254},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.80600000000000005,"15min":0.80900000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.87,"5min":1.256,"15min":1.3200000000000001},"last":0.74399999999999999}]}]},{"osd":2,"up_from":14,"seq":60129543058,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307608,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064232,"statfs":{"total":96636764160,"available":96321773568,"internally_reserved":0,"allocated":2179072,"data_stored":1084280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29499999999999998},"max":{"1min":0.61199999999999999,"5min":1.075,"15min":1.075},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.54400000000000004,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.378},"max":{"1min":0.59399999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.69599999999999995,"5min":1.079,"15min":1.079},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.498},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66300000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.505}]},{"osd":3,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.625},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73499999999999999,"5min":1.129,"15min":1.2490000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.748,"5min":0.84999999999999998,"15min":1.198},"last":0.57299999999999995}]},{"osd":4,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.52500000000000002},"max":{"1min":1.02,"5min":1.177,"15min":1.222},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79300000000000004,"15min":0.78400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.50600000000000001},"max":{"1min":0.97099999999999997,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54300000000000004},"max":{"1min":0.88400000000000001,"5min":1.206,"15min":1.22},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81699999999999995,"15min":0.80600000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.98799999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56100000000000005},"max":{"1min":1.006,"5min":1.075,"15min":1.2470000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.84899999999999998,"15min":0.83399999999999996},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.58699999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.298},"last":0.90300000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.81100000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.56699999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.1899999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.84299999999999997,"15min":0.84099999999999997},"min":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0509999999999999,"5min":1.2350000000000001,"15min":1.367},"last":0.79700000000000004}]}]},{"osd":3,"up_from":15,"seq":64424510354,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303516,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068324,"statfs":{"total":96636764160,"available":96325963776,"internally_reserved":0,"allocated":2183168,"data_stored":1084846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51700000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.47699999999999998,"15min":0.49199999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.31},"max":{"1min":0.55100000000000005,"5min":0.85299999999999998,"15min":0.93799999999999994},"last":0.41099999999999998}]},{"osd":1,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47999999999999998,"15min":0.48499999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.0449999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60799999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59199999999999997}]},{"osd":2,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40000000000000002},"max":{"1min":0.80500000000000005,"5min":1.516,"15min":1.516},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85399999999999998,"5min":1.395,"15min":1.395},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.754,"15min":0.75},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79900000000000004,"5min":1.413,"15min":1.413},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.46999999999999997},"max":{"1min":0.91400000000000003,"5min":1.163,"15min":1.1850000000000001},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.52200000000000002},"max":{"1min":0.84499999999999997,"5min":1.054,"15min":1.1359999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.79800000000000004,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.84899999999999998,"5min":1.202,"15min":1.202},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.80700000000000005},"min":{"1min":0.55900000000000005,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":0.88600000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.83199999999999996,"15min":0.83399999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93000000000000005,"5min":1.1140000000000001,"15min":1.163},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90300000000000002,"5min":1.321,"15min":1.321},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":0.80300000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52000000000000002},"max":{"1min":0.94099999999999995,"5min":1.466,"15min":1.466},"last":0.73899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129543057,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311720,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060120,"statfs":{"total":96636764160,"available":96317562880,"internally_reserved":0,"allocated":2195456,"data_stored":1086370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68000000000000005,"15min":0.70499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.79700000000000004,"5min":0.90400000000000003,"15min":0.99299999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70799999999999996,"15min":0.73499999999999999},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.86599999999999999,"5min":0.99199999999999999,"15min":1.0640000000000001},"last":0.71899999999999997}]},{"osd":1,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.72399999999999998,"15min":0.746},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94099999999999995,"5min":1.01,"15min":1.0309999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81200000000000006,"5min":0.98599999999999999,"15min":1.04},"last":0.63700000000000001}]},{"osd":2,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.72099999999999997,"15min":0.747},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88400000000000001,"5min":0.93400000000000005,"15min":0.98599999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.70799999999999996,"15min":0.72199999999999998},"min":{"1min":0.49199999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.749,"5min":0.97799999999999998,"15min":0.99399999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72699999999999998,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.48799999999999999,"15min":0.46300000000000002},"max":{"1min":0.84699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.752,"15min":0.76300000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90000000000000002,"5min":1.018,"15min":1.018},"last":0.78300000000000003}]},{"osd":5,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59499999999999997,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80500000000000005,"5min":0.93100000000000005,"15min":0.97399999999999998},"last":0.443}]},{"osd":6,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.62,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84199999999999997,"5min":0.93200000000000005,"15min":0.98699999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.67900000000000005},"min":{"1min":0.46800000000000003,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.86099999999999999,"5min":0.90300000000000002,"15min":1.0149999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.70899999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47599999999999998},"max":{"1min":0.88500000000000001,"5min":0.93500000000000005,"15min":1.0449999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.49399999999999999},"max":{"1min":0.89400000000000002,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.58299999999999996}]}]},{"osd":5,"up_from":15,"seq":64424510352,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312304,"kb_used_data":2728,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059536,"statfs":{"total":96636764160,"available":96316964864,"internally_reserved":0,"allocated":2793472,"data_stored":1676356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40100000000000002},"max":{"1min":0.96199999999999997,"5min":1.071,"15min":1.071},"last":0.71399999999999997}]},{"osd":1,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.62}]},{"osd":2,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.58199999999999996,"5min":0.50600000000000001,"15min":0.46200000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.2269999999999999,"5min":3.2269999999999999,"15min":3.2269999999999999},"last":0.67800000000000005}]},{"osd":3,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91500000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.751},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.75600000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9199999999999999,"5min":2.9199999999999999,"15min":2.9199999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.64200000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.0169999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.48499999999999999,"15min":0.45300000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95099999999999996,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.66700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T22:53:26.562 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T22:53:26.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.784+0000 7f42e19ff700 1 Processor -- start 2023-12-26T22:53:26.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.784+0000 7f42e19ff700 1 -- start start 2023-12-26T22:53:26.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.785+0000 7f42e19ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc159930 0x7f42dc15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.785+0000 7f42e19ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f42dc06a1a0 0x7f42dc068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.785+0000 7f42e19ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc068a80 con 0x7f42dc159560 2023-12-26T22:53:26.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.785+0000 7f42e19ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc0cac50 con 0x7f42dc06a1a0 2023-12-26T22:53:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.785+0000 7f42e19ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc15da00 con 0x7f42dc159930 2023-12-26T22:53:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.785+0000 7f42daffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f42dc159560 0x7f42dc0b11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56310/0 (socket says 172.21.15.131:56310) 2023-12-26T22:53:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.785+0000 7f42daffd700 1 -- 172.21.15.131:0/661916309 learned_addr learned my addr 172.21.15.131:0/661916309 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42da7fc700 1 --2- 172.21.15.131:0/661916309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc159930 0x7f42dc15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42db7fe700 1 --2- 172.21.15.131:0/661916309 >> v2:172.21.15.190:3300/0 conn(0x7f42dc06a1a0 0x7f42dc068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42d9ffb700 1 -- 172.21.15.131:0/661916309 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420586699 0 0) 0x7f42dc068a80 con 0x7f42dc159560 2023-12-26T22:53:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42d9ffb700 1 -- 172.21.15.131:0/661916309 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c4003570 con 0x7f42dc159560 2023-12-26T22:53:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42db7fe700 1 -- 172.21.15.131:0/661916309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc159930 msgr2=0x7f42dc15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:26.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42db7fe700 1 --2- 172.21.15.131:0/661916309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc159930 0x7f42dc15d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42db7fe700 1 -- 172.21.15.131:0/661916309 >> v1:172.21.15.131:6789/0 conn(0x7f42dc159560 legacy=0x7f42dc0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42db7fe700 1 -- 172.21.15.131:0/661916309 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc15dce0 con 0x7f42dc06a1a0 2023-12-26T22:53:26.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42da7fc700 1 --2- 172.21.15.131:0/661916309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc159930 0x7f42dc15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:26.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.786+0000 7f42db7fe700 1 --2- 172.21.15.131:0/661916309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc06a1a0 0x7f42dc068f10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009e60 tx=0x7f42d4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8667f48f0c3433bf server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:26.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42d9ffb700 1 -- 172.21.15.131:0/661916309 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d400e070 con 0x7f42dc06a1a0 2023-12-26T22:53:26.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42d9ffb700 1 -- 172.21.15.131:0/661916309 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d400bab0 con 0x7f42dc06a1a0 2023-12-26T22:53:26.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42d9ffb700 1 -- 172.21.15.131:0/661916309 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d400be70 con 0x7f42dc06a1a0 2023-12-26T22:53:26.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42e19ff700 1 -- 172.21.15.131:0/661916309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc06a1a0 msgr2=0x7f42dc068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42e19ff700 1 --2- 172.21.15.131:0/661916309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc06a1a0 0x7f42dc068f10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009e60 tx=0x7f42d4009780 comp rx=0 tx=0).stop 2023-12-26T22:53:26.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42e19ff700 1 -- 172.21.15.131:0/661916309 shutdown_connections 2023-12-26T22:53:26.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42e19ff700 1 --2- 172.21.15.131:0/661916309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc06a1a0 0x7f42dc068f10 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42e19ff700 1 --2- 172.21.15.131:0/661916309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc159930 0x7f42dc15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42e19ff700 1 -- 172.21.15.131:0/661916309 >> 172.21.15.131:0/661916309 conn(0x7f42dc0ce3d0 msgr2=0x7f42dc0ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:26.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42e19ff700 1 -- 172.21.15.131:0/661916309 shutdown_connections 2023-12-26T22:53:26.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.787+0000 7f42e19ff700 1 -- 172.21.15.131:0/661916309 wait complete. 2023-12-26T22:53:26.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.789+0000 7f42e19ff700 1 Processor -- start 2023-12-26T22:53:26.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.790+0000 7f42e19ff700 1 -- start start 2023-12-26T22:53:26.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42e19ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc06a1a0 0x7f42dc15bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42e19ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42dc159560 0x7f42dc0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42e19ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc159930 0x7f42dc156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42e19ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc15a210 con 0x7f42dc159560 2023-12-26T22:53:26.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42e19ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc15a4f0 con 0x7f42dc159930 2023-12-26T22:53:26.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42e19ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc15a7d0 con 0x7f42dc06a1a0 2023-12-26T22:53:26.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42da7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42dc159560 0x7f42dc0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42da7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42dc159560 0x7f42dc0cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48154/0 (socket says 172.21.15.131:48154) 2023-12-26T22:53:26.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42da7fc700 1 -- 172.21.15.131:0/1420079848 learned_addr learned my addr 172.21.15.131:0/1420079848 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:26.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42db7fe700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc159930 0x7f42dc156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.791+0000 7f42daffd700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc06a1a0 0x7f42dc15bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42da7fc700 1 -- 172.21.15.131:0/1420079848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc06a1a0 msgr2=0x7f42dc15bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42da7fc700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc06a1a0 0x7f42dc15bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42da7fc700 1 -- 172.21.15.131:0/1420079848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc159930 msgr2=0x7f42dc156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:26.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42da7fc700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc159930 0x7f42dc156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:26.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42da7fc700 1 -- 172.21.15.131:0/1420079848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0009420 con 0x7f42dc159560 2023-12-26T22:53:26.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42da7fc700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42dc159560 0x7f42dc0cd090 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f42d000ec80 tx=0x7f42d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:26.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42c2ffd700 1 -- 172.21.15.131:0/1420079848 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000c920 con 0x7f42dc159560 2023-12-26T22:53:26.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42c2ffd700 1 -- 172.21.15.131:0/1420079848 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d000ca80 con 0x7f42dc159560 2023-12-26T22:53:26.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42c2ffd700 1 -- 172.21.15.131:0/1420079848 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000ce00 con 0x7f42dc159560 2023-12-26T22:53:26.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d4009a20 con 0x7f42dc159560 2023-12-26T22:53:26.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.792+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc0c23f0 con 0x7f42dc159560 2023-12-26T22:53:26.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.794+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc0c26a0 con 0x7f42dc159560 2023-12-26T22:53:26.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.794+0000 7f42c2ffd700 1 -- 172.21.15.131:0/1420079848 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f42d0013680 con 0x7f42dc159560 2023-12-26T22:53:26.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.795+0000 7f42c2ffd700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f42c8066f30 0x7f42c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:26.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.795+0000 7f42c2ffd700 1 -- 172.21.15.131:0/1420079848 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f42d0084ae0 con 0x7f42dc159560 2023-12-26T22:53:26.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.795+0000 7f42daffd700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f42c8066f30 0x7f42c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:26.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.796+0000 7f42daffd700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f42c8066f30 0x7f42c80693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f42cc007fd0 tx=0x7f42cc009040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:26.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.799+0000 7f42c2ffd700 1 -- 172.21.15.131:0/1420079848 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42dc0c26a0 con 0x7f42dc159560 2023-12-26T22:53:26.987 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.987+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42dc0c26a0 con 0x7f42c8066f30 2023-12-26T22:53:26.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:26.995+0000 7f42c2ffd700 1 -- 172.21.15.131:0/1420079848 <== mgr.8455 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289601 (secure 0 0 0) 0x7f42dc0c26a0 con 0x7f42c8066f30 2023-12-26T22:53:27.000 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:27.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f42c8066f30 msgr2=0x7f42c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f42c8066f30 0x7f42c80693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f42cc007fd0 tx=0x7f42cc009040 comp rx=0 tx=0).stop 2023-12-26T22:53:27.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42dc159560 msgr2=0x7f42dc0cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42dc159560 0x7f42dc0cd090 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f42d000ec80 tx=0x7f42d00091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 shutdown_connections 2023-12-26T22:53:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42dc159930 0x7f42dc156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42dc06a1a0 0x7f42dc15bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f42c8066f30 0x7f42c80693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 --2- 172.21.15.131:0/1420079848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42dc159560 0x7f42dc0cd090 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 >> 172.21.15.131:0/1420079848 conn(0x7f42dc0ce3d0 msgr2=0x7f42dc158430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 shutdown_connections 2023-12-26T22:53:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.000+0000 7f42e19ff700 1 -- 172.21.15.131:0/1420079848 wait complete. 2023-12-26T22:53:27.004 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T22:53:27.018 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T22:53:25.934257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2470248,"kb_used_data":18216,"kb_used_omap":0,"kb_used_meta":2451968,"kb_avail":752504472,"statfs":{"total":773094113280,"available":770564579328,"internally_reserved":0,"allocated":18653184,"data_stored":9859412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2510815232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.770485"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222507+0000","last_change":"2023-12-26T22:53:22.218719+0000","last_active":"2023-12-26T22:53:22.222507+0000","last_peered":"2023-12-26T22:53:22.222507+0000","last_clean":"2023-12-26T22:53:22.222507+0000","last_became_active":"2023-12-26T22:53:22.216707+0000","last_became_peered":"2023-12-26T22:53:22.216707+0000","last_unstale":"2023-12-26T22:53:22.222507+0000","last_undegraded":"2023-12-26T22:53:22.222507+0000","last_fullsized":"2023-12-26T22:53:22.222507+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:21.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215945+0000","last_change":"2023-12-26T22:53:22.213713+0000","last_active":"2023-12-26T22:53:22.215945+0000","last_peered":"2023-12-26T22:53:22.215945+0000","last_clean":"2023-12-26T22:53:22.215945+0000","last_became_active":"2023-12-26T22:53:22.213116+0000","last_became_peered":"2023-12-26T22:53:22.213116+0000","last_unstale":"2023-12-26T22:53:22.215945+0000","last_undegraded":"2023-12-26T22:53:22.215945+0000","last_fullsized":"2023-12-26T22:53:22.215945+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:26:47.075927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226570+0000","last_change":"2023-12-26T22:53:22.222197+0000","last_active":"2023-12-26T22:53:22.226570+0000","last_peered":"2023-12-26T22:53:22.226570+0000","last_clean":"2023-12-26T22:53:22.226570+0000","last_became_active":"2023-12-26T22:53:22.221145+0000","last_became_peered":"2023-12-26T22:53:22.221145+0000","last_unstale":"2023-12-26T22:53:22.226570+0000","last_undegraded":"2023-12-26T22:53:22.226570+0000","last_fullsized":"2023-12-26T22:53:22.226570+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:42:11.111318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227579+0000","last_change":"2023-12-26T22:53:22.225552+0000","last_active":"2023-12-26T22:53:22.227579+0000","last_peered":"2023-12-26T22:53:22.227579+0000","last_clean":"2023-12-26T22:53:22.227579+0000","last_became_active":"2023-12-26T22:53:22.225275+0000","last_became_peered":"2023-12-26T22:53:22.225275+0000","last_unstale":"2023-12-26T22:53:22.227579+0000","last_undegraded":"2023-12-26T22:53:22.227579+0000","last_fullsized":"2023-12-26T22:53:22.227579+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:46:20.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.207676+0000","last_change":"2023-12-26T22:53:22.202198+0000","last_active":"2023-12-26T22:53:22.207676+0000","last_peered":"2023-12-26T22:53:22.207676+0000","last_clean":"2023-12-26T22:53:22.207676+0000","last_became_active":"2023-12-26T22:53:22.201894+0000","last_became_peered":"2023-12-26T22:53:22.201894+0000","last_unstale":"2023-12-26T22:53:22.207676+0000","last_undegraded":"2023-12-26T22:53:22.207676+0000","last_fullsized":"2023-12-26T22:53:22.207676+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:03:27.336647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222599+0000","last_change":"2023-12-26T22:53:22.221789+0000","last_active":"2023-12-26T22:53:22.222599+0000","last_peered":"2023-12-26T22:53:22.222599+0000","last_clean":"2023-12-26T22:53:22.222599+0000","last_became_active":"2023-12-26T22:53:22.220854+0000","last_became_peered":"2023-12-26T22:53:22.220854+0000","last_unstale":"2023-12-26T22:53:22.222599+0000","last_undegraded":"2023-12-26T22:53:22.222599+0000","last_fullsized":"2023-12-26T22:53:22.222599+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:15:10.647047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.209035+0000","last_change":"2023-12-26T22:53:22.204079+0000","last_active":"2023-12-26T22:53:22.209035+0000","last_peered":"2023-12-26T22:53:22.209035+0000","last_clean":"2023-12-26T22:53:22.209035+0000","last_became_active":"2023-12-26T22:53:22.203814+0000","last_became_peered":"2023-12-26T22:53:22.203814+0000","last_unstale":"2023-12-26T22:53:22.209035+0000","last_undegraded":"2023-12-26T22:53:22.209035+0000","last_fullsized":"2023-12-26T22:53:22.209035+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:11:20.118109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222695+0000","last_change":"2023-12-26T22:53:22.219496+0000","last_active":"2023-12-26T22:53:22.222695+0000","last_peered":"2023-12-26T22:53:22.222695+0000","last_clean":"2023-12-26T22:53:22.222695+0000","last_became_active":"2023-12-26T22:53:22.216460+0000","last_became_peered":"2023-12-26T22:53:22.216460+0000","last_unstale":"2023-12-26T22:53:22.222695+0000","last_undegraded":"2023-12-26T22:53:22.222695+0000","last_fullsized":"2023-12-26T22:53:22.222695+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:00:02.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.211364+0000","last_change":"2023-12-26T22:53:22.208422+0000","last_active":"2023-12-26T22:53:22.211364+0000","last_peered":"2023-12-26T22:53:22.211364+0000","last_clean":"2023-12-26T22:53:22.211364+0000","last_became_active":"2023-12-26T22:53:22.208133+0000","last_became_peered":"2023-12-26T22:53:22.208133+0000","last_unstale":"2023-12-26T22:53:22.211364+0000","last_undegraded":"2023-12-26T22:53:22.211364+0000","last_fullsized":"2023-12-26T22:53:22.211364+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:40:05.458555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226649+0000","last_change":"2023-12-26T22:53:22.222498+0000","last_active":"2023-12-26T22:53:22.226649+0000","last_peered":"2023-12-26T22:53:22.226649+0000","last_clean":"2023-12-26T22:53:22.226649+0000","last_became_active":"2023-12-26T22:53:22.220712+0000","last_became_peered":"2023-12-26T22:53:22.220712+0000","last_unstale":"2023-12-26T22:53:22.226649+0000","last_undegraded":"2023-12-26T22:53:22.226649+0000","last_fullsized":"2023-12-26T22:53:22.226649+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:40:36.155981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.211717+0000","last_change":"2023-12-26T22:53:22.209184+0000","last_active":"2023-12-26T22:53:22.211717+0000","last_peered":"2023-12-26T22:53:22.211717+0000","last_clean":"2023-12-26T22:53:22.211717+0000","last_became_active":"2023-12-26T22:53:22.208890+0000","last_became_peered":"2023-12-26T22:53:22.208890+0000","last_unstale":"2023-12-26T22:53:22.211717+0000","last_undegraded":"2023-12-26T22:53:22.211717+0000","last_fullsized":"2023-12-26T22:53:22.211717+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:46:37.041463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.216836+0000","last_change":"2023-12-26T22:53:22.209841+0000","last_active":"2023-12-26T22:53:22.216836+0000","last_peered":"2023-12-26T22:53:22.216836+0000","last_clean":"2023-12-26T22:53:22.216836+0000","last_became_active":"2023-12-26T22:53:22.209492+0000","last_became_peered":"2023-12-26T22:53:22.209492+0000","last_unstale":"2023-12-26T22:53:22.216836+0000","last_undegraded":"2023-12-26T22:53:22.216836+0000","last_fullsized":"2023-12-26T22:53:22.216836+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:08:38.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.206443+0000","last_change":"2023-12-26T22:53:22.201377+0000","last_active":"2023-12-26T22:53:22.206443+0000","last_peered":"2023-12-26T22:53:22.206443+0000","last_clean":"2023-12-26T22:53:22.206443+0000","last_became_active":"2023-12-26T22:53:22.200053+0000","last_became_peered":"2023-12-26T22:53:22.200053+0000","last_unstale":"2023-12-26T22:53:22.206443+0000","last_undegraded":"2023-12-26T22:53:22.206443+0000","last_fullsized":"2023-12-26T22:53:22.206443+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:41.862685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226608+0000","last_change":"2023-12-26T22:53:22.221280+0000","last_active":"2023-12-26T22:53:22.226608+0000","last_peered":"2023-12-26T22:53:22.226608+0000","last_clean":"2023-12-26T22:53:22.226608+0000","last_became_active":"2023-12-26T22:53:22.220915+0000","last_became_peered":"2023-12-26T22:53:22.220915+0000","last_unstale":"2023-12-26T22:53:22.226608+0000","last_undegraded":"2023-12-26T22:53:22.226608+0000","last_fullsized":"2023-12-26T22:53:22.226608+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:38:25.089802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215809+0000","last_change":"2023-12-26T22:53:22.213531+0000","last_active":"2023-12-26T22:53:22.215809+0000","last_peered":"2023-12-26T22:53:22.215809+0000","last_clean":"2023-12-26T22:53:22.215809+0000","last_became_active":"2023-12-26T22:53:22.213236+0000","last_became_peered":"2023-12-26T22:53:22.213236+0000","last_unstale":"2023-12-26T22:53:22.215809+0000","last_undegraded":"2023-12-26T22:53:22.215809+0000","last_fullsized":"2023-12-26T22:53:22.215809+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:33:04.388734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.206665+0000","last_change":"2023-12-26T22:53:22.201444+0000","last_active":"2023-12-26T22:53:22.206665+0000","last_peered":"2023-12-26T22:53:22.206665+0000","last_clean":"2023-12-26T22:53:22.206665+0000","last_became_active":"2023-12-26T22:53:22.201175+0000","last_became_peered":"2023-12-26T22:53:22.201175+0000","last_unstale":"2023-12-26T22:53:22.206665+0000","last_undegraded":"2023-12-26T22:53:22.206665+0000","last_fullsized":"2023-12-26T22:53:22.206665+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:03.484408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.228979+0000","last_change":"2023-12-26T22:53:22.227064+0000","last_active":"2023-12-26T22:53:22.228979+0000","last_peered":"2023-12-26T22:53:22.228979+0000","last_clean":"2023-12-26T22:53:22.228979+0000","last_became_active":"2023-12-26T22:53:22.224360+0000","last_became_peered":"2023-12-26T22:53:22.224360+0000","last_unstale":"2023-12-26T22:53:22.228979+0000","last_undegraded":"2023-12-26T22:53:22.228979+0000","last_fullsized":"2023-12-26T22:53:22.228979+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:01:39.571028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215916+0000","last_change":"2023-12-26T22:53:22.212661+0000","last_active":"2023-12-26T22:53:22.215916+0000","last_peered":"2023-12-26T22:53:22.215916+0000","last_clean":"2023-12-26T22:53:22.215916+0000","last_became_active":"2023-12-26T22:53:22.212405+0000","last_became_peered":"2023-12-26T22:53:22.212405+0000","last_unstale":"2023-12-26T22:53:22.215916+0000","last_undegraded":"2023-12-26T22:53:22.215916+0000","last_fullsized":"2023-12-26T22:53:22.215916+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:15:10.647047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220429+0000","last_change":"2023-12-26T22:53:22.211452+0000","last_active":"2023-12-26T22:53:22.220429+0000","last_peered":"2023-12-26T22:53:22.220429+0000","last_clean":"2023-12-26T22:53:22.220429+0000","last_became_active":"2023-12-26T22:53:22.211193+0000","last_became_peered":"2023-12-26T22:53:22.211193+0000","last_unstale":"2023-12-26T22:53:22.220429+0000","last_undegraded":"2023-12-26T22:53:22.220429+0000","last_fullsized":"2023-12-26T22:53:22.220429+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:37:07.113041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220431+0000","last_change":"2023-12-26T22:53:22.209747+0000","last_active":"2023-12-26T22:53:22.220431+0000","last_peered":"2023-12-26T22:53:22.220431+0000","last_clean":"2023-12-26T22:53:22.220431+0000","last_became_active":"2023-12-26T22:53:22.209476+0000","last_became_peered":"2023-12-26T22:53:22.209476+0000","last_unstale":"2023-12-26T22:53:22.220431+0000","last_undegraded":"2023-12-26T22:53:22.220431+0000","last_fullsized":"2023-12-26T22:53:22.220431+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:43:14.771756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226933+0000","last_change":"2023-12-26T22:53:22.223611+0000","last_active":"2023-12-26T22:53:22.226933+0000","last_peered":"2023-12-26T22:53:22.226933+0000","last_clean":"2023-12-26T22:53:22.226933+0000","last_became_active":"2023-12-26T22:53:22.223350+0000","last_became_peered":"2023-12-26T22:53:22.223350+0000","last_unstale":"2023-12-26T22:53:22.226933+0000","last_undegraded":"2023-12-26T22:53:22.226933+0000","last_fullsized":"2023-12-26T22:53:22.226933+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:04:24.295277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.217605+0000","last_change":"2023-12-26T22:53:22.210953+0000","last_active":"2023-12-26T22:53:22.217605+0000","last_peered":"2023-12-26T22:53:22.217605+0000","last_clean":"2023-12-26T22:53:22.217605+0000","last_became_active":"2023-12-26T22:53:22.210684+0000","last_became_peered":"2023-12-26T22:53:22.210684+0000","last_unstale":"2023-12-26T22:53:22.217605+0000","last_undegraded":"2023-12-26T22:53:22.217605+0000","last_fullsized":"2023-12-26T22:53:22.217605+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:52:26.829735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222276+0000","last_change":"2023-12-26T22:53:22.216903+0000","last_active":"2023-12-26T22:53:22.222276+0000","last_peered":"2023-12-26T22:53:22.222276+0000","last_clean":"2023-12-26T22:53:22.222276+0000","last_became_active":"2023-12-26T22:53:22.216623+0000","last_became_peered":"2023-12-26T22:53:22.216623+0000","last_unstale":"2023-12-26T22:53:22.222276+0000","last_undegraded":"2023-12-26T22:53:22.222276+0000","last_fullsized":"2023-12-26T22:53:22.222276+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:59:45.374415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.211413+0000","last_change":"2023-12-26T22:53:22.205001+0000","last_active":"2023-12-26T22:53:22.211413+0000","last_peered":"2023-12-26T22:53:22.211413+0000","last_clean":"2023-12-26T22:53:22.211413+0000","last_became_active":"2023-12-26T22:53:22.204732+0000","last_became_peered":"2023-12-26T22:53:22.204732+0000","last_unstale":"2023-12-26T22:53:22.211413+0000","last_undegraded":"2023-12-26T22:53:22.211413+0000","last_fullsized":"2023-12-26T22:53:22.211413+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:40:05.458555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.213422+0000","last_change":"2023-12-26T22:53:22.206415+0000","last_active":"2023-12-26T22:53:22.213422+0000","last_peered":"2023-12-26T22:53:22.213422+0000","last_clean":"2023-12-26T22:53:22.213422+0000","last_became_active":"2023-12-26T22:53:22.206105+0000","last_became_peered":"2023-12-26T22:53:22.206105+0000","last_unstale":"2023-12-26T22:53:22.213422+0000","last_undegraded":"2023-12-26T22:53:22.213422+0000","last_fullsized":"2023-12-26T22:53:22.213422+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:23:52.675851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.221081+0000","last_change":"2023-12-26T22:53:22.215818+0000","last_active":"2023-12-26T22:53:22.221081+0000","last_peered":"2023-12-26T22:53:22.221081+0000","last_clean":"2023-12-26T22:53:22.221081+0000","last_became_active":"2023-12-26T22:53:22.214058+0000","last_became_peered":"2023-12-26T22:53:22.214058+0000","last_unstale":"2023-12-26T22:53:22.221081+0000","last_undegraded":"2023-12-26T22:53:22.221081+0000","last_fullsized":"2023-12-26T22:53:22.221081+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:16:24.302086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212508+0000","last_change":"2023-12-26T22:53:22.205604+0000","last_active":"2023-12-26T22:53:22.212508+0000","last_peered":"2023-12-26T22:53:22.212508+0000","last_clean":"2023-12-26T22:53:22.212508+0000","last_became_active":"2023-12-26T22:53:22.205335+0000","last_became_peered":"2023-12-26T22:53:22.205335+0000","last_unstale":"2023-12-26T22:53:22.212508+0000","last_undegraded":"2023-12-26T22:53:22.212508+0000","last_fullsized":"2023-12-26T22:53:22.212508+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:30:20.461910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.214579+0000","last_change":"2023-12-26T22:53:22.212711+0000","last_active":"2023-12-26T22:53:22.214579+0000","last_peered":"2023-12-26T22:53:22.214579+0000","last_clean":"2023-12-26T22:53:22.214579+0000","last_became_active":"2023-12-26T22:53:22.212237+0000","last_became_peered":"2023-12-26T22:53:22.212237+0000","last_unstale":"2023-12-26T22:53:22.214579+0000","last_undegraded":"2023-12-26T22:53:22.214579+0000","last_fullsized":"2023-12-26T22:53:22.214579+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:16:24.302086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220461+0000","last_change":"2023-12-26T22:53:22.211279+0000","last_active":"2023-12-26T22:53:22.220461+0000","last_peered":"2023-12-26T22:53:22.220461+0000","last_clean":"2023-12-26T22:53:22.220461+0000","last_became_active":"2023-12-26T22:53:22.211028+0000","last_became_peered":"2023-12-26T22:53:22.211028+0000","last_unstale":"2023-12-26T22:53:22.220461+0000","last_undegraded":"2023-12-26T22:53:22.220461+0000","last_fullsized":"2023-12-26T22:53:22.220461+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:38:15.775091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.217628+0000","last_change":"2023-12-26T22:53:22.210994+0000","last_active":"2023-12-26T22:53:22.217628+0000","last_peered":"2023-12-26T22:53:22.217628+0000","last_clean":"2023-12-26T22:53:22.217628+0000","last_became_active":"2023-12-26T22:53:22.210707+0000","last_became_peered":"2023-12-26T22:53:22.210707+0000","last_unstale":"2023-12-26T22:53:22.217628+0000","last_undegraded":"2023-12-26T22:53:22.217628+0000","last_fullsized":"2023-12-26T22:53:22.217628+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:28:18.916576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212240+0000","last_change":"2023-12-26T22:53:22.202279+0000","last_active":"2023-12-26T22:53:22.212240+0000","last_peered":"2023-12-26T22:53:22.212240+0000","last_clean":"2023-12-26T22:53:22.212240+0000","last_became_active":"2023-12-26T22:53:22.202024+0000","last_became_peered":"2023-12-26T22:53:22.202024+0000","last_unstale":"2023-12-26T22:53:22.212240+0000","last_undegraded":"2023-12-26T22:53:22.212240+0000","last_fullsized":"2023-12-26T22:53:22.212240+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:33:12.690776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212994+0000","last_change":"2023-12-26T22:53:22.204223+0000","last_active":"2023-12-26T22:53:22.212994+0000","last_peered":"2023-12-26T22:53:22.212994+0000","last_clean":"2023-12-26T22:53:22.212994+0000","last_became_active":"2023-12-26T22:53:22.203560+0000","last_became_peered":"2023-12-26T22:53:22.203560+0000","last_unstale":"2023-12-26T22:53:22.212994+0000","last_undegraded":"2023-12-26T22:53:22.212994+0000","last_fullsized":"2023-12-26T22:53:22.212994+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:51:39.335927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.206642+0000","last_change":"2023-12-26T22:53:22.202612+0000","last_active":"2023-12-26T22:53:22.206642+0000","last_peered":"2023-12-26T22:53:22.206642+0000","last_clean":"2023-12-26T22:53:22.206642+0000","last_became_active":"2023-12-26T22:53:22.202092+0000","last_became_peered":"2023-12-26T22:53:22.202092+0000","last_unstale":"2023-12-26T22:53:22.206642+0000","last_undegraded":"2023-12-26T22:53:22.206642+0000","last_fullsized":"2023-12-26T22:53:22.206642+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:46:37.041463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"174'4","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.228289+0000","last_change":"2023-12-26T22:53:21.180537+0000","last_active":"2023-12-26T22:53:22.228289+0000","last_peered":"2023-12-26T22:53:22.228289+0000","last_clean":"2023-12-26T22:53:22.228289+0000","last_became_active":"2023-12-26T22:53:21.180268+0000","last_became_peered":"2023-12-26T22:53:21.180268+0000","last_unstale":"2023-12-26T22:53:22.228289+0000","last_undegraded":"2023-12-26T22:53:22.228289+0000","last_fullsized":"2023-12-26T22:53:22.228289+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:04:13.843581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215502+0000","last_change":"2023-12-26T22:53:22.212871+0000","last_active":"2023-12-26T22:53:22.215502+0000","last_peered":"2023-12-26T22:53:22.215502+0000","last_clean":"2023-12-26T22:53:22.215502+0000","last_became_active":"2023-12-26T22:53:22.212026+0000","last_became_peered":"2023-12-26T22:53:22.212026+0000","last_unstale":"2023-12-26T22:53:22.215502+0000","last_undegraded":"2023-12-26T22:53:22.215502+0000","last_fullsized":"2023-12-26T22:53:22.215502+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:54:07.840473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223543+0000","last_change":"2023-12-26T22:53:21.186408+0000","last_active":"2023-12-26T22:53:22.223543+0000","last_peered":"2023-12-26T22:53:22.223543+0000","last_clean":"2023-12-26T22:53:22.223543+0000","last_became_active":"2023-12-26T22:53:21.181260+0000","last_became_peered":"2023-12-26T22:53:21.181260+0000","last_unstale":"2023-12-26T22:53:22.223543+0000","last_undegraded":"2023-12-26T22:53:22.223543+0000","last_fullsized":"2023-12-26T22:53:22.223543+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:11.585142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215267+0000","last_change":"2023-12-26T22:53:22.213679+0000","last_active":"2023-12-26T22:53:22.215267+0000","last_peered":"2023-12-26T22:53:22.215267+0000","last_clean":"2023-12-26T22:53:22.215267+0000","last_became_active":"2023-12-26T22:53:22.212846+0000","last_became_peered":"2023-12-26T22:53:22.212846+0000","last_unstale":"2023-12-26T22:53:22.215267+0000","last_undegraded":"2023-12-26T22:53:22.215267+0000","last_fullsized":"2023-12-26T22:53:22.215267+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:41:24.790254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223851+0000","last_change":"2023-12-26T22:53:21.189072+0000","last_active":"2023-12-26T22:53:22.223851+0000","last_peered":"2023-12-26T22:53:22.223851+0000","last_clean":"2023-12-26T22:53:22.223851+0000","last_became_active":"2023-12-26T22:53:21.188804+0000","last_became_peered":"2023-12-26T22:53:21.188804+0000","last_unstale":"2023-12-26T22:53:22.223851+0000","last_undegraded":"2023-12-26T22:53:22.223851+0000","last_fullsized":"2023-12-26T22:53:22.223851+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:20:14.854866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227927+0000","last_change":"2023-12-26T22:53:22.226257+0000","last_active":"2023-12-26T22:53:22.227927+0000","last_peered":"2023-12-26T22:53:22.227927+0000","last_clean":"2023-12-26T22:53:22.227927+0000","last_became_active":"2023-12-26T22:53:22.225308+0000","last_became_peered":"2023-12-26T22:53:22.225308+0000","last_unstale":"2023-12-26T22:53:22.227927+0000","last_undegraded":"2023-12-26T22:53:22.227927+0000","last_fullsized":"2023-12-26T22:53:22.227927+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:44:01.231663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223683+0000","last_change":"2023-12-26T22:53:21.184599+0000","last_active":"2023-12-26T22:53:22.223683+0000","last_peered":"2023-12-26T22:53:22.223683+0000","last_clean":"2023-12-26T22:53:22.223683+0000","last_became_active":"2023-12-26T22:53:21.182198+0000","last_became_peered":"2023-12-26T22:53:21.182198+0000","last_unstale":"2023-12-26T22:53:22.223683+0000","last_undegraded":"2023-12-26T22:53:22.223683+0000","last_fullsized":"2023-12-26T22:53:22.223683+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:46:44.576855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212524+0000","last_change":"2023-12-26T22:53:22.204357+0000","last_active":"2023-12-26T22:53:22.212524+0000","last_peered":"2023-12-26T22:53:22.212524+0000","last_clean":"2023-12-26T22:53:22.212524+0000","last_became_active":"2023-12-26T22:53:22.204074+0000","last_became_peered":"2023-12-26T22:53:22.204074+0000","last_unstale":"2023-12-26T22:53:22.212524+0000","last_undegraded":"2023-12-26T22:53:22.212524+0000","last_fullsized":"2023-12-26T22:53:22.212524+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:38:15.775091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.216522+0000","last_change":"2023-12-26T22:53:21.190029+0000","last_active":"2023-12-26T22:53:22.216522+0000","last_peered":"2023-12-26T22:53:22.216522+0000","last_clean":"2023-12-26T22:53:22.216522+0000","last_became_active":"2023-12-26T22:53:21.189760+0000","last_became_peered":"2023-12-26T22:53:21.189760+0000","last_unstale":"2023-12-26T22:53:22.216522+0000","last_undegraded":"2023-12-26T22:53:22.216522+0000","last_fullsized":"2023-12-26T22:53:22.216522+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:58:01.085416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.230403+0000","last_change":"2023-12-26T22:53:22.227280+0000","last_active":"2023-12-26T22:53:22.230403+0000","last_peered":"2023-12-26T22:53:22.230403+0000","last_clean":"2023-12-26T22:53:22.230403+0000","last_became_active":"2023-12-26T22:53:22.226925+0000","last_became_peered":"2023-12-26T22:53:22.226925+0000","last_unstale":"2023-12-26T22:53:22.230403+0000","last_undegraded":"2023-12-26T22:53:22.230403+0000","last_fullsized":"2023-12-26T22:53:22.230403+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:58:27.027125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.221338+0000","last_change":"2023-12-26T22:53:21.188316+0000","last_active":"2023-12-26T22:53:22.221338+0000","last_peered":"2023-12-26T22:53:22.221338+0000","last_clean":"2023-12-26T22:53:22.221338+0000","last_became_active":"2023-12-26T22:53:21.186623+0000","last_became_peered":"2023-12-26T22:53:21.186623+0000","last_unstale":"2023-12-26T22:53:22.221338+0000","last_undegraded":"2023-12-26T22:53:22.221338+0000","last_fullsized":"2023-12-26T22:53:22.221338+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:56:00.710341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.201360+0000","last_change":"2023-12-26T22:53:22.199282+0000","last_active":"2023-12-26T22:53:22.201360+0000","last_peered":"2023-12-26T22:53:22.201360+0000","last_clean":"2023-12-26T22:53:22.201360+0000","last_became_active":"2023-12-26T22:53:22.199009+0000","last_became_peered":"2023-12-26T22:53:22.199009+0000","last_unstale":"2023-12-26T22:53:22.201360+0000","last_undegraded":"2023-12-26T22:53:22.201360+0000","last_fullsized":"2023-12-26T22:53:22.201360+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:47.328793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.204651+0000","last_change":"2023-12-26T22:53:21.190195+0000","last_active":"2023-12-26T22:53:22.204651+0000","last_peered":"2023-12-26T22:53:22.204651+0000","last_clean":"2023-12-26T22:53:22.204651+0000","last_became_active":"2023-12-26T22:53:21.188813+0000","last_became_peered":"2023-12-26T22:53:21.188813+0000","last_unstale":"2023-12-26T22:53:22.204651+0000","last_undegraded":"2023-12-26T22:53:22.204651+0000","last_fullsized":"2023-12-26T22:53:22.204651+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:14:49.420581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222255+0000","last_change":"2023-12-26T22:53:22.216518+0000","last_active":"2023-12-26T22:53:22.222255+0000","last_peered":"2023-12-26T22:53:22.222255+0000","last_clean":"2023-12-26T22:53:22.222255+0000","last_became_active":"2023-12-26T22:53:22.216099+0000","last_became_peered":"2023-12-26T22:53:22.216099+0000","last_unstale":"2023-12-26T22:53:22.222255+0000","last_undegraded":"2023-12-26T22:53:22.222255+0000","last_fullsized":"2023-12-26T22:53:22.222255+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:46:32.353169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.213471+0000","last_change":"2023-12-26T22:53:21.178758+0000","last_active":"2023-12-26T22:53:22.213471+0000","last_peered":"2023-12-26T22:53:22.213471+0000","last_clean":"2023-12-26T22:53:22.213471+0000","last_became_active":"2023-12-26T22:53:21.178506+0000","last_became_peered":"2023-12-26T22:53:21.178506+0000","last_unstale":"2023-12-26T22:53:22.213471+0000","last_undegraded":"2023-12-26T22:53:22.213471+0000","last_fullsized":"2023-12-26T22:53:22.213471+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:55:31.344157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.219163+0000","last_change":"2023-12-26T22:53:22.210228+0000","last_active":"2023-12-26T22:53:22.219163+0000","last_peered":"2023-12-26T22:53:22.219163+0000","last_clean":"2023-12-26T22:53:22.219163+0000","last_became_active":"2023-12-26T22:53:22.209949+0000","last_became_peered":"2023-12-26T22:53:22.209949+0000","last_unstale":"2023-12-26T22:53:22.219163+0000","last_undegraded":"2023-12-26T22:53:22.219163+0000","last_fullsized":"2023-12-26T22:53:22.219163+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:39:20.300743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.205453+0000","last_change":"2023-12-26T22:53:21.185824+0000","last_active":"2023-12-26T22:53:22.205453+0000","last_peered":"2023-12-26T22:53:22.205453+0000","last_clean":"2023-12-26T22:53:22.205453+0000","last_became_active":"2023-12-26T22:53:21.185084+0000","last_became_peered":"2023-12-26T22:53:21.185084+0000","last_unstale":"2023-12-26T22:53:22.205453+0000","last_undegraded":"2023-12-26T22:53:22.205453+0000","last_fullsized":"2023-12-26T22:53:22.205453+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:33:26.481774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226824+0000","last_change":"2023-12-26T22:53:22.224425+0000","last_active":"2023-12-26T22:53:22.226824+0000","last_peered":"2023-12-26T22:53:22.226824+0000","last_clean":"2023-12-26T22:53:22.226824+0000","last_became_active":"2023-12-26T22:53:22.224109+0000","last_became_peered":"2023-12-26T22:53:22.224109+0000","last_unstale":"2023-12-26T22:53:22.226824+0000","last_undegraded":"2023-12-26T22:53:22.226824+0000","last_fullsized":"2023-12-26T22:53:22.226824+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:05:53.082200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.194555+0000","last_change":"2023-12-26T22:53:21.192124+0000","last_active":"2023-12-26T22:53:22.194555+0000","last_peered":"2023-12-26T22:53:22.194555+0000","last_clean":"2023-12-26T22:53:22.194555+0000","last_became_active":"2023-12-26T22:53:21.191138+0000","last_became_peered":"2023-12-26T22:53:21.191138+0000","last_unstale":"2023-12-26T22:53:22.194555+0000","last_undegraded":"2023-12-26T22:53:22.194555+0000","last_fullsized":"2023-12-26T22:53:22.194555+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:24:00.638948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.216857+0000","last_change":"2023-12-26T22:53:22.214251+0000","last_active":"2023-12-26T22:53:22.216857+0000","last_peered":"2023-12-26T22:53:22.216857+0000","last_clean":"2023-12-26T22:53:22.216857+0000","last_became_active":"2023-12-26T22:53:22.213841+0000","last_became_peered":"2023-12-26T22:53:22.213841+0000","last_unstale":"2023-12-26T22:53:22.216857+0000","last_undegraded":"2023-12-26T22:53:22.216857+0000","last_fullsized":"2023-12-26T22:53:22.216857+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:32:49.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"174'6","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.225245+0000","last_change":"2023-12-26T22:53:21.189013+0000","last_active":"2023-12-26T22:53:22.225245+0000","last_peered":"2023-12-26T22:53:22.225245+0000","last_clean":"2023-12-26T22:53:22.225245+0000","last_became_active":"2023-12-26T22:53:21.187801+0000","last_became_peered":"2023-12-26T22:53:21.187801+0000","last_unstale":"2023-12-26T22:53:22.225245+0000","last_undegraded":"2023-12-26T22:53:22.225245+0000","last_fullsized":"2023-12-26T22:53:22.225245+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:40.853740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.219699+0000","last_change":"2023-12-26T22:53:22.212762+0000","last_active":"2023-12-26T22:53:22.219699+0000","last_peered":"2023-12-26T22:53:22.219699+0000","last_clean":"2023-12-26T22:53:22.219699+0000","last_became_active":"2023-12-26T22:53:22.211143+0000","last_became_peered":"2023-12-26T22:53:22.211143+0000","last_unstale":"2023-12-26T22:53:22.219699+0000","last_undegraded":"2023-12-26T22:53:22.219699+0000","last_fullsized":"2023-12-26T22:53:22.219699+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:38:13.416933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223997+0000","last_change":"2023-12-26T22:53:21.187785+0000","last_active":"2023-12-26T22:53:22.223997+0000","last_peered":"2023-12-26T22:53:22.223997+0000","last_clean":"2023-12-26T22:53:22.223997+0000","last_became_active":"2023-12-26T22:53:21.187458+0000","last_became_peered":"2023-12-26T22:53:21.187458+0000","last_unstale":"2023-12-26T22:53:22.223997+0000","last_undegraded":"2023-12-26T22:53:22.223997+0000","last_fullsized":"2023-12-26T22:53:22.223997+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:39.312979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220483+0000","last_change":"2023-12-26T22:53:22.211703+0000","last_active":"2023-12-26T22:53:22.220483+0000","last_peered":"2023-12-26T22:53:22.220483+0000","last_clean":"2023-12-26T22:53:22.220483+0000","last_became_active":"2023-12-26T22:53:22.209722+0000","last_became_peered":"2023-12-26T22:53:22.209722+0000","last_unstale":"2023-12-26T22:53:22.220483+0000","last_undegraded":"2023-12-26T22:53:22.220483+0000","last_fullsized":"2023-12-26T22:53:22.220483+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:33:12.690776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.202025+0000","last_change":"2023-12-26T22:53:21.191509+0000","last_active":"2023-12-26T22:53:22.202025+0000","last_peered":"2023-12-26T22:53:22.202025+0000","last_clean":"2023-12-26T22:53:22.202025+0000","last_became_active":"2023-12-26T22:53:21.190722+0000","last_became_peered":"2023-12-26T22:53:21.190722+0000","last_unstale":"2023-12-26T22:53:22.202025+0000","last_undegraded":"2023-12-26T22:53:22.202025+0000","last_fullsized":"2023-12-26T22:53:22.202025+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:32:48.375268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227856+0000","last_change":"2023-12-26T22:53:22.226206+0000","last_active":"2023-12-26T22:53:22.227856+0000","last_peered":"2023-12-26T22:53:22.227856+0000","last_clean":"2023-12-26T22:53:22.227856+0000","last_became_active":"2023-12-26T22:53:22.224842+0000","last_became_peered":"2023-12-26T22:53:22.224842+0000","last_unstale":"2023-12-26T22:53:22.227856+0000","last_undegraded":"2023-12-26T22:53:22.227856+0000","last_fullsized":"2023-12-26T22:53:22.227856+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:25:41.646188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.192328+0000","last_change":"2023-12-26T22:53:21.183612+0000","last_active":"2023-12-26T22:53:22.192328+0000","last_peered":"2023-12-26T22:53:22.192328+0000","last_clean":"2023-12-26T22:53:22.192328+0000","last_became_active":"2023-12-26T22:53:21.183207+0000","last_became_peered":"2023-12-26T22:53:21.183207+0000","last_unstale":"2023-12-26T22:53:22.192328+0000","last_undegraded":"2023-12-26T22:53:22.192328+0000","last_fullsized":"2023-12-26T22:53:22.192328+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:38.281030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222041+0000","last_change":"2023-12-26T22:53:22.216354+0000","last_active":"2023-12-26T22:53:22.222041+0000","last_peered":"2023-12-26T22:53:22.222041+0000","last_clean":"2023-12-26T22:53:22.222041+0000","last_became_active":"2023-12-26T22:53:22.215484+0000","last_became_peered":"2023-12-26T22:53:22.215484+0000","last_unstale":"2023-12-26T22:53:22.222041+0000","last_undegraded":"2023-12-26T22:53:22.222041+0000","last_fullsized":"2023-12-26T22:53:22.222041+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:18:00.845307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"174'8","reported_seq":18,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.224105+0000","last_change":"2023-12-26T22:53:21.183791+0000","last_active":"2023-12-26T22:53:22.224105+0000","last_peered":"2023-12-26T22:53:22.224105+0000","last_clean":"2023-12-26T22:53:22.224105+0000","last_became_active":"2023-12-26T22:53:21.183533+0000","last_became_peered":"2023-12-26T22:53:21.183533+0000","last_unstale":"2023-12-26T22:53:22.224105+0000","last_undegraded":"2023-12-26T22:53:22.224105+0000","last_fullsized":"2023-12-26T22:53:22.224105+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:36:38.281030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215937+0000","last_change":"2023-12-26T22:53:22.213174+0000","last_active":"2023-12-26T22:53:22.215937+0000","last_peered":"2023-12-26T22:53:22.215937+0000","last_clean":"2023-12-26T22:53:22.215937+0000","last_became_active":"2023-12-26T22:53:22.212914+0000","last_became_peered":"2023-12-26T22:53:22.212914+0000","last_unstale":"2023-12-26T22:53:22.215937+0000","last_undegraded":"2023-12-26T22:53:22.215937+0000","last_fullsized":"2023-12-26T22:53:22.215937+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:15:46.997797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223111+0000","last_change":"2023-12-26T22:53:21.184831+0000","last_active":"2023-12-26T22:53:22.223111+0000","last_peered":"2023-12-26T22:53:22.223111+0000","last_clean":"2023-12-26T22:53:22.223111+0000","last_became_active":"2023-12-26T22:53:21.184552+0000","last_became_peered":"2023-12-26T22:53:21.184552+0000","last_unstale":"2023-12-26T22:53:22.223111+0000","last_undegraded":"2023-12-26T22:53:22.223111+0000","last_fullsized":"2023-12-26T22:53:22.223111+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:16:23.293140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.213397+0000","last_change":"2023-12-26T22:53:22.204614+0000","last_active":"2023-12-26T22:53:22.213397+0000","last_peered":"2023-12-26T22:53:22.213397+0000","last_clean":"2023-12-26T22:53:22.213397+0000","last_became_active":"2023-12-26T22:53:22.204336+0000","last_became_peered":"2023-12-26T22:53:22.204336+0000","last_unstale":"2023-12-26T22:53:22.213397+0000","last_undegraded":"2023-12-26T22:53:22.213397+0000","last_fullsized":"2023-12-26T22:53:22.213397+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:00:02.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.225045+0000","last_change":"2023-12-26T22:53:21.180443+0000","last_active":"2023-12-26T22:53:22.225045+0000","last_peered":"2023-12-26T22:53:22.225045+0000","last_clean":"2023-12-26T22:53:22.225045+0000","last_became_active":"2023-12-26T22:53:21.180118+0000","last_became_peered":"2023-12-26T22:53:21.180118+0000","last_unstale":"2023-12-26T22:53:22.225045+0000","last_undegraded":"2023-12-26T22:53:22.225045+0000","last_fullsized":"2023-12-26T22:53:22.225045+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:41:12.747519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.208997+0000","last_change":"2023-12-26T22:53:22.204833+0000","last_active":"2023-12-26T22:53:22.208997+0000","last_peered":"2023-12-26T22:53:22.208997+0000","last_clean":"2023-12-26T22:53:22.208997+0000","last_became_active":"2023-12-26T22:53:22.204535+0000","last_became_peered":"2023-12-26T22:53:22.204535+0000","last_unstale":"2023-12-26T22:53:22.208997+0000","last_undegraded":"2023-12-26T22:53:22.208997+0000","last_fullsized":"2023-12-26T22:53:22.208997+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:31:59.141262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"174'1","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.214170+0000","last_change":"2023-12-26T22:53:21.193608+0000","last_active":"2023-12-26T22:53:22.214170+0000","last_peered":"2023-12-26T22:53:22.214170+0000","last_clean":"2023-12-26T22:53:22.214170+0000","last_became_active":"2023-12-26T22:53:21.193231+0000","last_became_peered":"2023-12-26T22:53:21.193231+0000","last_unstale":"2023-12-26T22:53:22.214170+0000","last_undegraded":"2023-12-26T22:53:22.214170+0000","last_fullsized":"2023-12-26T22:53:22.214170+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:03:26.327702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.211698+0000","last_change":"2023-12-26T22:53:22.208605+0000","last_active":"2023-12-26T22:53:22.211698+0000","last_peered":"2023-12-26T22:53:22.211698+0000","last_clean":"2023-12-26T22:53:22.211698+0000","last_became_active":"2023-12-26T22:53:22.208319+0000","last_became_peered":"2023-12-26T22:53:22.208319+0000","last_unstale":"2023-12-26T22:53:22.211698+0000","last_undegraded":"2023-12-26T22:53:22.211698+0000","last_fullsized":"2023-12-26T22:53:22.211698+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:33:04.388734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.192202+0000","last_change":"2023-12-26T22:53:21.189279+0000","last_active":"2023-12-26T22:53:22.192202+0000","last_peered":"2023-12-26T22:53:22.192202+0000","last_clean":"2023-12-26T22:53:22.192202+0000","last_became_active":"2023-12-26T22:53:21.189000+0000","last_became_peered":"2023-12-26T22:53:21.189000+0000","last_unstale":"2023-12-26T22:53:22.192202+0000","last_undegraded":"2023-12-26T22:53:22.192202+0000","last_fullsized":"2023-12-26T22:53:22.192202+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:21:40.023765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.221151+0000","last_change":"2023-12-26T22:53:22.217522+0000","last_active":"2023-12-26T22:53:22.221151+0000","last_peered":"2023-12-26T22:53:22.221151+0000","last_clean":"2023-12-26T22:53:22.221151+0000","last_became_active":"2023-12-26T22:53:22.216906+0000","last_became_peered":"2023-12-26T22:53:22.216906+0000","last_unstale":"2023-12-26T22:53:22.221151+0000","last_undegraded":"2023-12-26T22:53:22.221151+0000","last_fullsized":"2023-12-26T22:53:22.221151+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:02:31.293948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222002+0000","last_change":"2023-12-26T22:53:21.185896+0000","last_active":"2023-12-26T22:53:22.222002+0000","last_peered":"2023-12-26T22:53:22.222002+0000","last_clean":"2023-12-26T22:53:22.222002+0000","last_became_active":"2023-12-26T22:53:21.181249+0000","last_became_peered":"2023-12-26T22:53:21.181249+0000","last_unstale":"2023-12-26T22:53:22.222002+0000","last_undegraded":"2023-12-26T22:53:22.222002+0000","last_fullsized":"2023-12-26T22:53:22.222002+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:56:00.710341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212486+0000","last_change":"2023-12-26T22:53:22.205721+0000","last_active":"2023-12-26T22:53:22.212486+0000","last_peered":"2023-12-26T22:53:22.212486+0000","last_clean":"2023-12-26T22:53:22.212486+0000","last_became_active":"2023-12-26T22:53:22.203612+0000","last_became_peered":"2023-12-26T22:53:22.203612+0000","last_unstale":"2023-12-26T22:53:22.212486+0000","last_undegraded":"2023-12-26T22:53:22.212486+0000","last_fullsized":"2023-12-26T22:53:22.212486+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:21.867292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.190543+0000","last_change":"2023-12-26T22:53:21.189867+0000","last_active":"2023-12-26T22:53:22.190543+0000","last_peered":"2023-12-26T22:53:22.190543+0000","last_clean":"2023-12-26T22:53:22.190543+0000","last_became_active":"2023-12-26T22:53:21.189379+0000","last_became_peered":"2023-12-26T22:53:21.189379+0000","last_unstale":"2023-12-26T22:53:22.190543+0000","last_undegraded":"2023-12-26T22:53:22.190543+0000","last_fullsized":"2023-12-26T22:53:22.190543+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:06:58.789465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.220180+0000","last_change":"2023-12-26T22:53:22.212321+0000","last_active":"2023-12-26T22:53:22.220180+0000","last_peered":"2023-12-26T22:53:22.220180+0000","last_clean":"2023-12-26T22:53:22.220180+0000","last_became_active":"2023-12-26T22:53:22.211339+0000","last_became_peered":"2023-12-26T22:53:22.211339+0000","last_unstale":"2023-12-26T22:53:22.220180+0000","last_undegraded":"2023-12-26T22:53:22.220180+0000","last_fullsized":"2023-12-26T22:53:22.220180+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:44:03.598595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.190526+0000","last_change":"2023-12-26T22:53:21.185640+0000","last_active":"2023-12-26T22:53:22.190526+0000","last_peered":"2023-12-26T22:53:22.190526+0000","last_clean":"2023-12-26T22:53:22.190526+0000","last_became_active":"2023-12-26T22:53:21.185345+0000","last_became_peered":"2023-12-26T22:53:21.185345+0000","last_unstale":"2023-12-26T22:53:22.190526+0000","last_undegraded":"2023-12-26T22:53:22.190526+0000","last_fullsized":"2023-12-26T22:53:22.190526+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:47:02.377466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212266+0000","last_change":"2023-12-26T22:53:22.202290+0000","last_active":"2023-12-26T22:53:22.212266+0000","last_peered":"2023-12-26T22:53:22.212266+0000","last_clean":"2023-12-26T22:53:22.212266+0000","last_became_active":"2023-12-26T22:53:22.202068+0000","last_became_peered":"2023-12-26T22:53:22.202068+0000","last_unstale":"2023-12-26T22:53:22.212266+0000","last_undegraded":"2023-12-26T22:53:22.212266+0000","last_fullsized":"2023-12-26T22:53:22.212266+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:37:07.113041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.189744+0000","last_change":"2023-12-26T22:53:21.190990+0000","last_active":"2023-12-26T22:53:22.189744+0000","last_peered":"2023-12-26T22:53:22.189744+0000","last_clean":"2023-12-26T22:53:22.189744+0000","last_became_active":"2023-12-26T22:53:21.189321+0000","last_became_peered":"2023-12-26T22:53:21.189321+0000","last_unstale":"2023-12-26T22:53:22.189744+0000","last_undegraded":"2023-12-26T22:53:22.189744+0000","last_fullsized":"2023-12-26T22:53:22.189744+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:16:34.121807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227545+0000","last_change":"2023-12-26T22:53:22.225409+0000","last_active":"2023-12-26T22:53:22.227545+0000","last_peered":"2023-12-26T22:53:22.227545+0000","last_clean":"2023-12-26T22:53:22.227545+0000","last_became_active":"2023-12-26T22:53:22.225113+0000","last_became_peered":"2023-12-26T22:53:22.225113+0000","last_unstale":"2023-12-26T22:53:22.227545+0000","last_undegraded":"2023-12-26T22:53:22.227545+0000","last_fullsized":"2023-12-26T22:53:22.227545+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:50:24.470636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"174'2","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.215282+0000","last_change":"2023-12-26T22:53:21.190321+0000","last_active":"2023-12-26T22:53:22.215282+0000","last_peered":"2023-12-26T22:53:22.215282+0000","last_clean":"2023-12-26T22:53:22.215282+0000","last_became_active":"2023-12-26T22:53:21.190022+0000","last_became_peered":"2023-12-26T22:53:21.190022+0000","last_unstale":"2023-12-26T22:53:22.215282+0000","last_undegraded":"2023-12-26T22:53:22.215282+0000","last_fullsized":"2023-12-26T22:53:22.215282+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:13:32.726081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.227553+0000","last_change":"2023-12-26T22:53:22.225501+0000","last_active":"2023-12-26T22:53:22.227553+0000","last_peered":"2023-12-26T22:53:22.227553+0000","last_clean":"2023-12-26T22:53:22.227553+0000","last_became_active":"2023-12-26T22:53:22.225027+0000","last_became_peered":"2023-12-26T22:53:22.225027+0000","last_unstale":"2023-12-26T22:53:22.227553+0000","last_undegraded":"2023-12-26T22:53:22.227553+0000","last_fullsized":"2023-12-26T22:53:22.227553+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:32:45.617260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.193127+0000","last_change":"2023-12-26T22:53:21.184348+0000","last_active":"2023-12-26T22:53:22.193127+0000","last_peered":"2023-12-26T22:53:22.193127+0000","last_clean":"2023-12-26T22:53:22.193127+0000","last_became_active":"2023-12-26T22:53:21.184059+0000","last_became_peered":"2023-12-26T22:53:21.184059+0000","last_unstale":"2023-12-26T22:53:22.193127+0000","last_undegraded":"2023-12-26T22:53:22.193127+0000","last_fullsized":"2023-12-26T22:53:22.193127+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:58:01.085416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.222231+0000","last_change":"2023-12-26T22:53:22.218858+0000","last_active":"2023-12-26T22:53:22.222231+0000","last_peered":"2023-12-26T22:53:22.222231+0000","last_clean":"2023-12-26T22:53:22.222231+0000","last_became_active":"2023-12-26T22:53:22.218493+0000","last_became_peered":"2023-12-26T22:53:22.218493+0000","last_unstale":"2023-12-26T22:53:22.222231+0000","last_undegraded":"2023-12-26T22:53:22.222231+0000","last_fullsized":"2023-12-26T22:53:22.222231+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:47:03.386411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.203408+0000","last_change":"2023-12-26T22:53:21.188465+0000","last_active":"2023-12-26T22:53:22.203408+0000","last_peered":"2023-12-26T22:53:22.203408+0000","last_clean":"2023-12-26T22:53:22.203408+0000","last_became_active":"2023-12-26T22:53:21.188074+0000","last_became_peered":"2023-12-26T22:53:21.188074+0000","last_unstale":"2023-12-26T22:53:22.203408+0000","last_undegraded":"2023-12-26T22:53:22.203408+0000","last_fullsized":"2023-12-26T22:53:22.203408+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:31:58.132317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.221826+0000","last_change":"2023-12-26T22:53:22.218790+0000","last_active":"2023-12-26T22:53:22.221826+0000","last_peered":"2023-12-26T22:53:22.221826+0000","last_clean":"2023-12-26T22:53:22.221826+0000","last_became_active":"2023-12-26T22:53:22.216634+0000","last_became_peered":"2023-12-26T22:53:22.216634+0000","last_unstale":"2023-12-26T22:53:22.221826+0000","last_undegraded":"2023-12-26T22:53:22.221826+0000","last_fullsized":"2023-12-26T22:53:22.221826+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:50:47.328793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.205377+0000","last_change":"2023-12-26T22:53:21.193433+0000","last_active":"2023-12-26T22:53:22.205377+0000","last_peered":"2023-12-26T22:53:22.205377+0000","last_clean":"2023-12-26T22:53:22.205377+0000","last_became_active":"2023-12-26T22:53:21.192925+0000","last_became_peered":"2023-12-26T22:53:21.192925+0000","last_unstale":"2023-12-26T22:53:22.205377+0000","last_undegraded":"2023-12-26T22:53:22.205377+0000","last_fullsized":"2023-12-26T22:53:22.205377+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:52:52.523353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212467+0000","last_change":"2023-12-26T22:53:22.206219+0000","last_active":"2023-12-26T22:53:22.212467+0000","last_peered":"2023-12-26T22:53:22.212467+0000","last_clean":"2023-12-26T22:53:22.212467+0000","last_became_active":"2023-12-26T22:53:22.205942+0000","last_became_peered":"2023-12-26T22:53:22.205942+0000","last_unstale":"2023-12-26T22:53:22.212467+0000","last_undegraded":"2023-12-26T22:53:22.212467+0000","last_fullsized":"2023-12-26T22:53:22.212467+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:10:30.959829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.192241+0000","last_change":"2023-12-26T22:53:21.182336+0000","last_active":"2023-12-26T22:53:22.192241+0000","last_peered":"2023-12-26T22:53:22.192241+0000","last_clean":"2023-12-26T22:53:22.192241+0000","last_became_active":"2023-12-26T22:53:21.182004+0000","last_became_peered":"2023-12-26T22:53:21.182004+0000","last_unstale":"2023-12-26T22:53:22.192241+0000","last_undegraded":"2023-12-26T22:53:22.192241+0000","last_fullsized":"2023-12-26T22:53:22.192241+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:08:16.127123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.212783+0000","last_change":"2023-12-26T22:53:22.207411+0000","last_active":"2023-12-26T22:53:22.212783+0000","last_peered":"2023-12-26T22:53:22.212783+0000","last_clean":"2023-12-26T22:53:22.212783+0000","last_became_active":"2023-12-26T22:53:22.207115+0000","last_became_peered":"2023-12-26T22:53:22.207115+0000","last_unstale":"2023-12-26T22:53:22.212783+0000","last_undegraded":"2023-12-26T22:53:22.212783+0000","last_fullsized":"2023-12-26T22:53:22.212783+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:15:41.634326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.213674+0000","last_change":"2023-12-26T22:53:21.188343+0000","last_active":"2023-12-26T22:53:22.213674+0000","last_peered":"2023-12-26T22:53:22.213674+0000","last_clean":"2023-12-26T22:53:22.213674+0000","last_became_active":"2023-12-26T22:53:21.187901+0000","last_became_peered":"2023-12-26T22:53:21.187901+0000","last_unstale":"2023-12-26T22:53:22.213674+0000","last_undegraded":"2023-12-26T22:53:22.213674+0000","last_fullsized":"2023-12-26T22:53:22.213674+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:35:10.928485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.223771+0000","last_change":"2023-12-26T22:53:22.218970+0000","last_active":"2023-12-26T22:53:22.223771+0000","last_peered":"2023-12-26T22:53:22.223771+0000","last_clean":"2023-12-26T22:53:22.223771+0000","last_became_active":"2023-12-26T22:53:22.218659+0000","last_became_peered":"2023-12-26T22:53:22.218659+0000","last_unstale":"2023-12-26T22:53:22.223771+0000","last_undegraded":"2023-12-26T22:53:22.223771+0000","last_fullsized":"2023-12-26T22:53:22.223771+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:37:49.053144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.201373+0000","last_change":"2023-12-26T22:53:21.189485+0000","last_active":"2023-12-26T22:53:22.201373+0000","last_peered":"2023-12-26T22:53:22.201373+0000","last_clean":"2023-12-26T22:53:22.201373+0000","last_became_active":"2023-12-26T22:53:21.189196+0000","last_became_peered":"2023-12-26T22:53:21.189196+0000","last_unstale":"2023-12-26T22:53:22.201373+0000","last_undegraded":"2023-12-26T22:53:22.201373+0000","last_fullsized":"2023-12-26T22:53:22.201373+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:01:08.802761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.226879+0000","last_change":"2023-12-26T22:53:22.223240+0000","last_active":"2023-12-26T22:53:22.226879+0000","last_peered":"2023-12-26T22:53:22.226879+0000","last_clean":"2023-12-26T22:53:22.226879+0000","last_became_active":"2023-12-26T22:53:22.222927+0000","last_became_peered":"2023-12-26T22:53:22.222927+0000","last_unstale":"2023-12-26T22:53:22.226879+0000","last_undegraded":"2023-12-26T22:53:22.226879+0000","last_fullsized":"2023-12-26T22:53:22.226879+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:22:06.940568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.204299+0000","last_change":"2023-12-26T22:53:21.184118+0000","last_active":"2023-12-26T22:53:22.204299+0000","last_peered":"2023-12-26T22:53:22.204299+0000","last_clean":"2023-12-26T22:53:22.204299+0000","last_became_active":"2023-12-26T22:53:21.181129+0000","last_became_peered":"2023-12-26T22:53:21.181129+0000","last_unstale":"2023-12-26T22:53:22.204299+0000","last_undegraded":"2023-12-26T22:53:22.204299+0000","last_fullsized":"2023-12-26T22:53:22.204299+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:09:39.505337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.225670+0000","last_change":"2023-12-26T22:53:22.224828+0000","last_active":"2023-12-26T22:53:22.225670+0000","last_peered":"2023-12-26T22:53:22.225670+0000","last_clean":"2023-12-26T22:53:22.225670+0000","last_became_active":"2023-12-26T22:53:22.224467+0000","last_became_peered":"2023-12-26T22:53:22.224467+0000","last_unstale":"2023-12-26T22:53:22.225670+0000","last_undegraded":"2023-12-26T22:53:22.225670+0000","last_fullsized":"2023-12-26T22:53:22.225670+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:21.163421+0000","last_clean_scrub_stamp":"2023-12-26T22:53:21.163421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:34:31.962868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"174'3","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.217584+0000","last_change":"2023-12-26T22:53:21.190558+0000","last_active":"2023-12-26T22:53:22.217584+0000","last_peered":"2023-12-26T22:53:22.217584+0000","last_clean":"2023-12-26T22:53:22.217584+0000","last_became_active":"2023-12-26T22:53:21.190308+0000","last_became_peered":"2023-12-26T22:53:21.190308+0000","last_unstale":"2023-12-26T22:53:22.217584+0000","last_undegraded":"2023-12-26T22:53:22.217584+0000","last_fullsized":"2023-12-26T22:53:22.217584+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T22:53:20.154476+0000","last_clean_scrub_stamp":"2023-12-26T22:53:20.154476+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:40:21.381550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"164'293","reported_seq":927,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-26T22:53:22.192865+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T22:53:22.192865+0000","last_peered":"2023-12-26T22:53:22.192865+0000","last_clean":"2023-12-26T22:53:22.192865+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T22:53:22.192865+0000","last_undegraded":"2023-12-26T22:53:22.192865+0000","last_fullsized":"2023-12-26T22:53:22.192865+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543057,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307604,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064236,"statfs":{"total":96636764160,"available":96321777664,"internally_reserved":0,"allocated":2174976,"data_stored":1084258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78900000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.5},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.68600000000000005,"15min":0.68899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.41999999999999998},"max":{"1min":0.77100000000000002,"5min":1.02,"15min":1.02},"last":0.45800000000000002}]},{"osd":1,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":1.601,"15min":1.601},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73499999999999999,"15min":0.746},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.441},"max":{"1min":0.90200000000000002,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.82499999999999996,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41499999999999998},"max":{"1min":0.88200000000000001,"5min":1.548,"15min":1.548},"last":0.48899999999999999}]},{"osd":3,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.629,"5min":0.58899999999999997,"15min":0.47899999999999998},"max":{"1min":0.88600000000000001,"5min":1.242,"15min":1.242},"last":0.754},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.42799999999999999},"max":{"1min":0.81699999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.74099999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40699999999999997},"max":{"1min":0.84599999999999997,"5min":1.113,"15min":1.113},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46300000000000002,"5min":0.38500000000000001,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.34599999999999997},"max":{"1min":0.82699999999999996,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.59499999999999997,"5min":0.55100000000000005,"15min":0.47799999999999998},"max":{"1min":0.94299999999999995,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.79300000000000004}]},{"osd":6,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70399999999999996,"15min":0.69299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85999999999999999,"5min":2.609,"15min":2.609},"last":0.628},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":2.996,"15min":2.996},"last":0.72099999999999997}]}]},{"osd":6,"up_from":15,"seq":64424510353,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307604,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064236,"statfs":{"total":96636764160,"available":96321777664,"internally_reserved":0,"allocated":2174976,"data_stored":1084258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.77100000000000002,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":4.2409999999999997,"5min":4.2409999999999997,"15min":4.2409999999999997},"last":0.69299999999999995}]},{"osd":1,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73099999999999998,"15min":0.72199999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98799999999999999},"last":0.628},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.752},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.50600000000000001},"max":{"1min":0.91800000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.46700000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.65900000000000003}]},{"osd":3,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.80600000000000005,"15min":0.77000000000000002},"min":{"1min":0.57199999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.3369999999999997,"5min":4.3369999999999997,"15min":4.3369999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97999999999999998,"5min":1.012,"15min":1.012},"last":0.74099999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.998,"15min":0.998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.47199999999999998,"5min":0.45500000000000002,"15min":0.40899999999999997},"max":{"1min":0.89400000000000002,"5min":0.92700000000000005,"15min":0.995},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.95999999999999996},"last":0.498},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69899999999999995,"15min":0.69599999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Dec 26 22:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45200000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.47799999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.53000000000000003}]}]},{"osd":1,"up_from":15,"seq":64424510353,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312284,"kb_used_data":2708,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059556,"statfs":{"total":96636764160,"available":96316985344,"internally_reserved":0,"allocated":2772992,"data_stored":1674696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44800000000000001,"15min":0.46899999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58599999999999997,"5min":0.79900000000000004,"15min":0.88400000000000001},"last":0.495},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.316},"max":{"1min":0.67000000000000004,"5min":0.81000000000000005,"15min":0.84499999999999997},"last":0.53100000000000003}]},{"osd":2,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.69799999999999995,"5min":0.87,"15min":0.87},"last":0.59699999999999998}]},{"osd":3,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":0.81000000000000005,"5min":0.877,"15min":0.89800000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77200000000000002,"5min":0.82399999999999995,"15min":0.84499999999999997},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83299999999999996,"5min":0.98699999999999999,"15min":1.163},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.745,"15min":0.75600000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":0.91000000000000003,"5min":1.0289999999999999,"15min":1.121},"last":0.81000000000000005}]},{"osd":5,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.80600000000000005,"15min":0.81499999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.502},"max":{"1min":0.93500000000000005,"5min":1.0820000000000001,"15min":1.1419999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.81699999999999995,"15min":0.82499999999999996},"min":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.875,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.81999999999999995,"15min":0.82599999999999996},"min":{"1min":0.56499999999999995,"5min":0.56299999999999994,"15min":0.55000000000000004},"max":{"1min":0.92600000000000005,"5min":1.0760000000000001,"15min":1.2450000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.81200000000000006,"15min":0.83099999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89200000000000002,"5min":1.1259999999999999,"15min":1.2270000000000001},"last":0.77600000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.80100000000000005,"15min":0.81299999999999994},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.55700000000000005},"max":{"1min":0.94499999999999995,"5min":1.107,"15min":1.2689999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.80400000000000005,"15min":0.81599999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.497},"max":{"1min":0.96599999999999997,"5min":1.0660000000000001,"15min":1.1890000000000001},"last":0.64200000000000002}]}]},{"osd":0,"up_from":15,"seq":64424510353,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307608,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064232,"statfs":{"total":96636764160,"available":96321773568,"internally_reserved":0,"allocated":2179072,"data_stored":1084348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.56799999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.439},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.307},"max":{"1min":0.63500000000000001,"5min":0.72399999999999998,"15min":0.86599999999999999},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.504,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64100000000000001,"5min":0.70199999999999996,"15min":0.88800000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.90500000000000003},"last":0.41299999999999998}]},{"osd":3,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.35499999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.86799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.63300000000000001,"15min":0.63600000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.755,"5min":0.82799999999999996,"15min":0.92200000000000004},"last":0.46400000000000002}]},{"osd":4,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":0.873,"5min":1.103,"15min":1.145},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.76400000000000001,"15min":0.78000000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.82799999999999996,"5min":1.1299999999999999,"15min":1.218},"last":0.63100000000000001}]},{"osd":5,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.81299999999999994,"15min":0.82299999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92700000000000005,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.81799999999999995,"15min":0.82999999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.50900000000000001},"max":{"1min":0.93100000000000005,"5min":1.218,"15min":1.218},"last":0.77800000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.89100000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78600000000000003,"15min":0.79900000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.83199999999999996,"5min":1.0780000000000001,"15min":1.2350000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.81200000000000006,"15min":0.81499999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.90900000000000003,"5min":1.165,"15min":1.254},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.80600000000000005,"15min":0.80900000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.87,"5min":1.256,"15min":1.3200000000000001},"last":0.74399999999999999}]}]},{"osd":2,"up_from":14,"seq":60129543058,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307608,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064232,"statfs":{"total":96636764160,"available":96321773568,"internally_reserved":0,"allocated":2179072,"data_stored":1084280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29499999999999998},"max":{"1min":0.61199999999999999,"5min":1.075,"15min":1.075},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.54400000000000004,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.378},"max":{"1min":0.59399999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.69599999999999995,"5min":1.079,"15min":1.079},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.498},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66300000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.505}]},{"osd":3,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.625},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73499999999999999,"5min":1.129,"15min":1.2490000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.41299999999999998,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.748,"5min":0.84999999999999998,"15min":1.198},"last":0.57299999999999995}]},{"osd":4,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.52500000000000002},"max":{"1min":1.02,"5min":1.177,"15min":1.222},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79300000000000004,"15min":0.78400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.50600000000000001},"max":{"1min":0.97099999999999997,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54300000000000004},"max":{"1min":0.88400000000000001,"5min":1.206,"15min":1.22},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81699999999999995,"15min":0.80600000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.98799999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56100000000000005},"max":{"1min":1.006,"5min":1.075,"15min":1.2470000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.84899999999999998,"15min":0.83399999999999996},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.58699999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.298},"last":0.90300000000000002}]},{"osd":7,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.81100000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.56699999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.1899999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.84299999999999997,"15min":0.84099999999999997},"min":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0509999999999999,"5min":1.2350000000000001,"15min":1.367},"last":0.79700000000000004}]}]},{"osd":3,"up_from":15,"seq":64424510354,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303516,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068324,"statfs":{"total":96636764160,"available":96325963776,"internally_reserved":0,"allocated":2183168,"data_stored":1084846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51700000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.47699999999999998,"15min":0.49199999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.31},"max":{"1min":0.55100000000000005,"5min":0.85299999999999998,"15min":0.93799999999999994},"last":0.41099999999999998}]},{"osd":1,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47999999999999998,"15min":0.48499999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.0449999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60799999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59199999999999997}]},{"osd":2,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40000000000000002},"max":{"1min":0.80500000000000005,"5min":1.516,"15min":1.516},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85399999999999998,"5min":1.395,"15min":1.395},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.754,"15min":0.75},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79900000000000004,"5min":1.413,"15min":1.413},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.46999999999999997},"max":{"1min":0.91400000000000003,"5min":1.163,"15min":1.1850000000000001},"last":0.57199999999999995}]},{"osd":5,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.76200000000000001,"15min":0.77300000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.52200000000000002},"max":{"1min":0.84499999999999997,"5min":1.054,"15min":1.1359999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.79800000000000004,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.84899999999999998,"5min":1.202,"15min":1.202},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.80700000000000005},"min":{"1min":0.55900000000000005,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":0.88600000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.83199999999999996,"15min":0.83399999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.93000000000000005,"5min":1.1140000000000001,"15min":1.163},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Dec 26 22:52:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90300000000000002,"5min":1.321,"15min":1.321},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81000000000000005,"15min":0.80300000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52000000000000002},"max":{"1min":0.94099999999999995,"5min":1.466,"15min":1.466},"last":0.73899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129543057,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311720,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060120,"statfs":{"total":96636764160,"available":96317562880,"internally_reserved":0,"allocated":2195456,"data_stored":1086370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68000000000000005,"15min":0.70499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.79700000000000004,"5min":0.90400000000000003,"15min":0.99299999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70799999999999996,"15min":0.73499999999999999},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.86599999999999999,"5min":0.99199999999999999,"15min":1.0640000000000001},"last":0.71899999999999997}]},{"osd":1,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.72399999999999998,"15min":0.746},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94099999999999995,"5min":1.01,"15min":1.0309999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81200000000000006,"5min":0.98599999999999999,"15min":1.04},"last":0.63700000000000001}]},{"osd":2,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.72099999999999997,"15min":0.747},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88400000000000001,"5min":0.93400000000000005,"15min":0.98599999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.70799999999999996,"15min":0.72199999999999998},"min":{"1min":0.49199999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.749,"5min":0.97799999999999998,"15min":0.99399999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72699999999999998,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.48799999999999999,"15min":0.46300000000000002},"max":{"1min":0.84699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.752,"15min":0.76300000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90000000000000002,"5min":1.018,"15min":1.018},"last":0.78300000000000003}]},{"osd":5,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59499999999999997,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80500000000000005,"5min":0.93100000000000005,"15min":0.97399999999999998},"last":0.443}]},{"osd":6,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.62,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84199999999999997,"5min":0.93200000000000005,"15min":0.98699999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.67900000000000005},"min":{"1min":0.46800000000000003,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.86099999999999999,"5min":0.90300000000000002,"15min":1.0149999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Tue Dec 26 22:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.70899999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47599999999999998},"max":{"1min":0.88500000000000001,"5min":0.93500000000000005,"15min":1.0449999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.49399999999999999},"max":{"1min":0.89400000000000002,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.58299999999999996}]}]},{"osd":5,"up_from":15,"seq":64424510352,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312304,"kb_used_data":2728,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059536,"statfs":{"total":96636764160,"available":96316964864,"internally_reserved":0,"allocated":2793472,"data_stored":1676356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40100000000000002},"max":{"1min":0.96199999999999997,"5min":1.071,"15min":1.071},"last":0.71399999999999997}]},{"osd":1,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.62}]},{"osd":2,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.58199999999999996,"5min":0.50600000000000001,"15min":0.46200000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.2269999999999999,"5min":3.2269999999999999,"15min":3.2269999999999999},"last":0.67800000000000005}]},{"osd":3,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91500000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.751},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75900000000000001,"15min":0.75600000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9199999999999999,"5min":2.9199999999999999,"15min":2.9199999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":1.018,"15min":1.018},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.64200000000000002}]},{"osd":6,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.0169999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.48499999999999999,"15min":0.45300000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Dec 26 22:52:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":3.1720000000000002,"5min":3.1720000000000002,"15min":3.1720000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95099999999999996,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.66700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T22:53:27.024 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T22:53:27.024 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T22:53:27.158 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-12-26T22:53:27.157+0000 7f88879f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T22:53:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.256+0000 7f3999bff700 1 Processor -- start 2023-12-26T22:53:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.256+0000 7f3999bff700 1 -- start start 2023-12-26T22:53:27.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f3999bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3994069620 0x7f39940699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f3999bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f399406dd90 0x7f399406e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f3999bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940ad9f0 0x7f39940ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f3999bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3994157b30 con 0x7f399406dd90 2023-12-26T22:53:27.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f3999bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399406f450 con 0x7f3994069620 2023-12-26T22:53:27.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f3999bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399406f8e0 con 0x7f39940ad9f0 2023-12-26T22:53:27.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f398bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f399406dd90 0x7f399406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f398bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f399406dd90 0x7f399406e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48170/0 (socket says 172.21.15.131:48170) 2023-12-26T22:53:27.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f398bfff700 1 -- 172.21.15.131:0/3493271724 learned_addr learned my addr 172.21.15.131:0/3493271724 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:27.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f39937fe700 1 --2- 172.21.15.131:0/3493271724 >> v2:172.21.15.190:3300/0 conn(0x7f3994069620 0x7f39940699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.257+0000 7f3993fff700 1 --2- 172.21.15.131:0/3493271724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940ad9f0 0x7f39940ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.258+0000 7f398bfff700 1 -- 172.21.15.131:0/3493271724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940ad9f0 msgr2=0x7f39940ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:27.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.258+0000 7f398bfff700 1 --2- 172.21.15.131:0/3493271724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940ad9f0 0x7f39940ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.258+0000 7f398bfff700 1 -- 172.21.15.131:0/3493271724 >> v2:172.21.15.190:3300/0 conn(0x7f3994069620 msgr2=0x7f39940699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:27.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.258+0000 7f398bfff700 1 --2- 172.21.15.131:0/3493271724 >> v2:172.21.15.190:3300/0 conn(0x7f3994069620 0x7f39940699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.258+0000 7f398bfff700 1 -- 172.21.15.131:0/3493271724 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3994069f30 con 0x7f399406dd90 2023-12-26T22:53:27.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.258+0000 7f398bfff700 1 --2- 172.21.15.131:0/3493271724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399406dd90 0x7f399406e230 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f39940ccb80 tx=0x7f397c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc722d80269151 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:27.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.258+0000 7f3992ffd700 1 -- 172.21.15.131:0/3493271724 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c00b640 con 0x7f399406dd90 2023-12-26T22:53:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.258+0000 7f3992ffd700 1 -- 172.21.15.131:0/3493271724 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c00b7a0 con 0x7f399406dd90 2023-12-26T22:53:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.258+0000 7f3992ffd700 1 -- 172.21.15.131:0/3493271724 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c00bb60 con 0x7f399406dd90 2023-12-26T22:53:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.259+0000 7f3999bff700 1 -- 172.21.15.131:0/3493271724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399406dd90 msgr2=0x7f399406e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.259+0000 7f3999bff700 1 --2- 172.21.15.131:0/3493271724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399406dd90 0x7f399406e230 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f39940ccb80 tx=0x7f397c009250 comp rx=0 tx=0).stop 2023-12-26T22:53:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.259+0000 7f3999bff700 1 -- 172.21.15.131:0/3493271724 shutdown_connections 2023-12-26T22:53:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.259+0000 7f3999bff700 1 --2- 172.21.15.131:0/3493271724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940ad9f0 0x7f39940ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.259+0000 7f3999bff700 1 --2- 172.21.15.131:0/3493271724 >> v2:172.21.15.190:3300/0 conn(0x7f3994069620 0x7f39940699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.259+0000 7f3999bff700 1 --2- 172.21.15.131:0/3493271724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399406dd90 0x7f399406e230 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.259+0000 7f3999bff700 1 -- 172.21.15.131:0/3493271724 >> 172.21.15.131:0/3493271724 conn(0x7f3994068040 msgr2=0x7f39940cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.259+0000 7f3999bff700 1 -- 172.21.15.131:0/3493271724 shutdown_connections 2023-12-26T22:53:27.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.259+0000 7f3999bff700 1 -- 172.21.15.131:0/3493271724 wait complete. 2023-12-26T22:53:27.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.260+0000 7f3999bff700 1 Processor -- start 2023-12-26T22:53:27.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.261+0000 7f3999bff700 1 -- start start 2023-12-26T22:53:27.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.261+0000 7f3999bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3994069620 0x7f399415c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3999bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399406dd90 0x7f39940c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3999bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39940ad9f0 0x7f39940c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3999bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940a5bd0 con 0x7f39940ad9f0 2023-12-26T22:53:27.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3999bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c7f10 con 0x7f3994069620 2023-12-26T22:53:27.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3999bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c81f0 con 0x7f399406dd90 2023-12-26T22:53:27.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3993fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39940ad9f0 0x7f39940c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3993fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39940ad9f0 0x7f39940c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48178/0 (socket says 172.21.15.131:48178) 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3993fff700 1 -- 172.21.15.131:0/2659834443 learned_addr learned my addr 172.21.15.131:0/2659834443 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f398bfff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399406dd90 0x7f39940c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3993fff700 1 -- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399406dd90 msgr2=0x7f39940c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3993fff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399406dd90 0x7f39940c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3993fff700 1 -- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3994069620 msgr2=0x7f399415c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f39937fe700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3994069620 0x7f399415c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3993fff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3994069620 0x7f399415c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3993fff700 1 -- 172.21.15.131:0/2659834443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0094f0 con 0x7f39940ad9f0 2023-12-26T22:53:27.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f398bfff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399406dd90 0x7f39940c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:27.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f39937fe700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3994069620 0x7f399415c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:27.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.262+0000 7f3993fff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39940ad9f0 0x7f39940c76a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f398c009df0 tx=0x7f398c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:27.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.263+0000 7f398b7fe700 1 -- 172.21.15.131:0/2659834443 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c00cb50 con 0x7f39940ad9f0 2023-12-26T22:53:27.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.263+0000 7f398b7fe700 1 -- 172.21.15.131:0/2659834443 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c00f040 con 0x7f39940ad9f0 2023-12-26T22:53:27.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.263+0000 7f398b7fe700 1 -- 172.21.15.131:0/2659834443 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c014480 con 0x7f39940ad9f0 2023-12-26T22:53:27.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.263+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39940c84d0 con 0x7f39940ad9f0 2023-12-26T22:53:27.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.263+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39940c7be0 con 0x7f39940ad9f0 2023-12-26T22:53:27.269 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.264+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39940a5d80 con 0x7f39940ad9f0 2023-12-26T22:53:27.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.265+0000 7f398b7fe700 1 -- 172.21.15.131:0/2659834443 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f398c014830 con 0x7f39940ad9f0 2023-12-26T22:53:27.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.266+0000 7f398b7fe700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3974066f30 0x7f39740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.266+0000 7f398b7fe700 1 -- 172.21.15.131:0/2659834443 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f398c085340 con 0x7f39940ad9f0 2023-12-26T22:53:27.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.266+0000 7f39937fe700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3974066f30 0x7f39740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.267+0000 7f39937fe700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3974066f30 0x7f39740693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3984006f00 tx=0x7f3984008040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:27.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.270+0000 7f398b7fe700 1 -- 172.21.15.131:0/2659834443 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39940a5d80 con 0x7f39940ad9f0 2023-12-26T22:53:27.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.538+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f39940657e0 con 0x7f39940ad9f0 2023-12-26T22:53:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.544+0000 7f398b7fe700 1 -- 172.21.15.131:0/2659834443 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v43) v1 ==== 190+0+0 (secure 0 0 0) 0x7f398c055e70 con 0x7f39940ad9f0 2023-12-26T22:53:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3974066f30 msgr2=0x7f39740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3974066f30 0x7f39740693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3984006f00 tx=0x7f3984008040 comp rx=0 tx=0).stop 2023-12-26T22:53:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39940ad9f0 msgr2=0x7f39940c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39940ad9f0 0x7f39940c76a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f398c009df0 tx=0x7f398c00b040 comp rx=0 tx=0).stop 2023-12-26T22:53:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 shutdown_connections 2023-12-26T22:53:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3974066f30 0x7f39740693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39940ad9f0 0x7f39940c76a0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3994069620 0x7f399415c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 --2- 172.21.15.131:0/2659834443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399406dd90 0x7f39940c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 >> 172.21.15.131:0/2659834443 conn(0x7f3994068040 msgr2=0x7f39940af240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 shutdown_connections 2023-12-26T22:53:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.547+0000 7f3999bff700 1 -- 172.21.15.131:0/2659834443 wait complete. 2023-12-26T22:53:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T22:53:27.567 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T22:53:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.757+0000 7fb537bff700 1 Processor -- start 2023-12-26T22:53:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.757+0000 7fb537bff700 1 -- start start 2023-12-26T22:53:27.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.758+0000 7fb537bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb53013c510 0x7fb5300c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.758+0000 7fb537bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5300cc810 0x7fb53006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.758+0000 7fb537bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5300686c0 con 0x7fb53006b8c0 2023-12-26T22:53:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.758+0000 7fb537bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53006c280 con 0x7fb53013c510 2023-12-26T22:53:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.758+0000 7fb537bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb530069520 con 0x7fb5300cc810 2023-12-26T22:53:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.758+0000 7fb53599b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb53013c510 0x7fb5300c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.758+0000 7fb53599b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb53013c510 0x7fb5300c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48194/0 (socket says 172.21.15.131:48194) 2023-12-26T22:53:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.758+0000 7fb53599b700 1 -- 172.21.15.131:0/2702511168 learned_addr learned my addr 172.21.15.131:0/2702511168 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.758+0000 7fb53519a700 1 --1- 172.21.15.131:0/2702511168 >> v1:172.21.15.131:6789/0 conn(0x7fb53006b8c0 0x7fb5300cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56312/0 (socket says 172.21.15.131:56312) 2023-12-26T22:53:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb53619c700 1 --2- 172.21.15.131:0/2702511168 >> v2:172.21.15.190:3300/0 conn(0x7fb5300cc810 0x7fb53006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb53599b700 1 -- 172.21.15.131:0/2702511168 >> v2:172.21.15.190:3300/0 conn(0x7fb5300cc810 msgr2=0x7fb53006bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb53599b700 1 --2- 172.21.15.131:0/2702511168 >> v2:172.21.15.190:3300/0 conn(0x7fb5300cc810 0x7fb53006bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb53599b700 1 -- 172.21.15.131:0/2702511168 >> v1:172.21.15.131:6789/0 conn(0x7fb53006b8c0 legacy=0x7fb5300cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb53599b700 1 -- 172.21.15.131:0/2702511168 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb530068910 con 0x7fb53013c510 2023-12-26T22:53:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb53619c700 1 --2- 172.21.15.131:0/2702511168 >> v2:172.21.15.190:3300/0 conn(0x7fb5300cc810 0x7fb53006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb53599b700 1 --2- 172.21.15.131:0/2702511168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53013c510 0x7fb5300c8a20 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb53006f6a0 tx=0x7fb52c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b451ec2c0ba1196 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb534999700 1 -- 172.21.15.131:0/2702511168 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c005c10 con 0x7fb53013c510 2023-12-26T22:53:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb534999700 1 -- 172.21.15.131:0/2702511168 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c005d70 con 0x7fb53013c510 2023-12-26T22:53:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.759+0000 7fb534999700 1 -- 172.21.15.131:0/2702511168 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c00b880 con 0x7fb53013c510 2023-12-26T22:53:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.760+0000 7fb537bff700 1 -- 172.21.15.131:0/2702511168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53013c510 msgr2=0x7fb5300c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.760+0000 7fb537bff700 1 --2- 172.21.15.131:0/2702511168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53013c510 0x7fb5300c8a20 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb53006f6a0 tx=0x7fb52c009220 comp rx=0 tx=0).stop 2023-12-26T22:53:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.760+0000 7fb537bff700 1 -- 172.21.15.131:0/2702511168 shutdown_connections 2023-12-26T22:53:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.760+0000 7fb537bff700 1 --2- 172.21.15.131:0/2702511168 >> v2:172.21.15.190:3300/0 conn(0x7fb5300cc810 0x7fb53006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.760+0000 7fb537bff700 1 --2- 172.21.15.131:0/2702511168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb53013c510 0x7fb5300c8a20 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.760+0000 7fb537bff700 1 -- 172.21.15.131:0/2702511168 >> 172.21.15.131:0/2702511168 conn(0x7fb5300ce3e0 msgr2=0x7fb5300ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.760+0000 7fb537bff700 1 -- 172.21.15.131:0/2702511168 shutdown_connections 2023-12-26T22:53:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.760+0000 7fb537bff700 1 -- 172.21.15.131:0/2702511168 wait complete. 2023-12-26T22:53:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.762+0000 7fb537bff700 1 Processor -- start 2023-12-26T22:53:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.762+0000 7fb537bff700 1 -- start start 2023-12-26T22:53:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb537bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53006b8c0 0x7fb53013a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb537bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5300cc810 0x7fb53013b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb537bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb53013c510 0x7fb5300c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb537bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5300c6610 con 0x7fb5300cc810 2023-12-26T22:53:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb537bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5300c5a00 con 0x7fb53013c510 2023-12-26T22:53:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb537bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5300c5ce0 con 0x7fb53006b8c0 2023-12-26T22:53:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53519a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5300cc810 0x7fb53013b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53519a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5300cc810 0x7fb53013b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48210/0 (socket says 172.21.15.131:48210) 2023-12-26T22:53:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53519a700 1 -- 172.21.15.131:0/3265586104 learned_addr learned my addr 172.21.15.131:0/3265586104 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53599b700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53006b8c0 0x7fb53013a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53619c700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb53013c510 0x7fb5300c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53519a700 1 -- 172.21.15.131:0/3265586104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53006b8c0 msgr2=0x7fb53013a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53519a700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53006b8c0 0x7fb53013a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53519a700 1 -- 172.21.15.131:0/3265586104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb53013c510 msgr2=0x7fb5300c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53519a700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb53013c510 0x7fb5300c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53519a700 1 -- 172.21.15.131:0/3265586104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c009480 con 0x7fb5300cc810 2023-12-26T22:53:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.763+0000 7fb53519a700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5300cc810 0x7fb53013b970 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c560 tx=0x7fb520007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.764+0000 7fb525ffb700 1 -- 172.21.15.131:0/3265586104 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520005a70 con 0x7fb5300cc810 2023-12-26T22:53:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.764+0000 7fb525ffb700 1 -- 172.21.15.131:0/3265586104 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb520005bd0 con 0x7fb5300cc810 2023-12-26T22:53:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.764+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5300c5fc0 con 0x7fb5300cc810 2023-12-26T22:53:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.764+0000 7fb525ffb700 1 -- 172.21.15.131:0/3265586104 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5200070d0 con 0x7fb5300cc810 2023-12-26T22:53:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.764+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53013aa30 con 0x7fb5300cc810 2023-12-26T22:53:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.765+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53006e080 con 0x7fb5300cc810 2023-12-26T22:53:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.766+0000 7fb525ffb700 1 -- 172.21.15.131:0/3265586104 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb520007480 con 0x7fb5300cc810 2023-12-26T22:53:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.767+0000 7fb525ffb700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb514066f30 0x7fb5140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.767+0000 7fb525ffb700 1 -- 172.21.15.131:0/3265586104 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb520083c80 con 0x7fb5300cc810 2023-12-26T22:53:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.767+0000 7fb53599b700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb514066f30 0x7fb5140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.767+0000 7fb53599b700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb514066f30 0x7fb5140693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb5300cf460 tx=0x7fb52c016000 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:27.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:27.770+0000 7fb525ffb700 1 -- 172.21.15.131:0/3265586104 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb520050c80 con 0x7fb5300cc810 2023-12-26T22:53:28.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.007+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5300657e0 con 0x7fb5300cc810 2023-12-26T22:53:28.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.007+0000 7fb525ffb700 1 -- 172.21.15.131:0/3265586104 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4610 (secure 0 0 0) 0x7fb520017020 con 0x7fb5300cc810 2023-12-26T22:53:28.007 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T22:53:28.008 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2545294570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2545294570},{"type":"v1","addr":"172.21.15.190:6835","nonce":2545294570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1800615106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1800615106},{"type":"v1","addr":"172.21.15.131:6834","nonce":1800615106}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3279484094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3279484094},{"type":"v1","addr":"172.21.15.131:6835","nonce":3279484094}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:53:21.655334+0000","modified":"2023-12-26T22:53:25.218600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9105":{"gid":9105,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/716462328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":716462328},{"type":"v1","addr":"172.21.15.190:6834","nonce":716462328}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-12-26T22:53:28.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb514066f30 msgr2=0x7fb5140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb514066f30 0x7fb5140693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb5300cf460 tx=0x7fb52c016000 comp rx=0 tx=0).stop 2023-12-26T22:53:28.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5300cc810 msgr2=0x7fb53013b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5300cc810 0x7fb53013b970 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c560 tx=0x7fb520007f70 comp rx=0 tx=0).stop 2023-12-26T22:53:28.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 shutdown_connections 2023-12-26T22:53:28.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb53013c510 0x7fb5300c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53006b8c0 0x7fb53013a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7fb514066f30 0x7fb5140693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 --2- 172.21.15.131:0/3265586104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5300cc810 0x7fb53013b970 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 >> 172.21.15.131:0/3265586104 conn(0x7fb5300ce3e0 msgr2=0x7fb530156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:28.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 shutdown_connections 2023-12-26T22:53:28.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.010+0000 7fb537bff700 1 -- 172.21.15.131:0/3265586104 wait complete. 2023-12-26T22:53:28.012 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 81 2023-12-26T22:53:28.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T22:53:21.655334+0000', 'modified': '2023-12-26T22:53:25.218600+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9105}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9105': {'gid': 9105, 'name': 'b', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6834/716462328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 716462328}, {'type': 'v1', 'addr': '172.21.15.190:6834', 'nonce': 716462328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T22:53:28.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T22:53:28.432 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-26T22:53:28.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.626+0000 7f71cc5ff700 1 Processor -- start 2023-12-26T22:53:28.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.626+0000 7f71cc5ff700 1 -- start start 2023-12-26T22:53:28.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.627+0000 7f71cc5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.627+0000 7f71cc5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f71c406b8c0 0x7f71c406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.627+0000 7f71cc5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c406c270 con 0x7f71c40cc100 2023-12-26T22:53:28.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.627+0000 7f71cc5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40686c0 con 0x7f71c406b8c0 2023-12-26T22:53:28.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.627+0000 7f71cc5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4069520 con 0x7f71c413c510 2023-12-26T22:53:28.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71c9b9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f71c406b8c0 0x7f71c406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71ca39b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71cab9c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f71c40cc100 0x7f71c40cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:34986/0 (socket says 172.21.15.131:34986) 2023-12-26T22:53:28.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71cab9c700 1 -- 172.21.15.131:0/1829345458 learned_addr learned my addr 172.21.15.131:0/1829345458 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71c9399700 1 -- 172.21.15.131:0/1829345458 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242483681 0 0) 0x7f71c406c270 con 0x7f71c40cc100 2023-12-26T22:53:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71c9399700 1 -- 172.21.15.131:0/1829345458 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b0003570 con 0x7f71c40cc100 2023-12-26T22:53:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71ca39b700 1 -- 172.21.15.131:0/1829345458 >> v2:172.21.15.190:3300/0 conn(0x7f71c406b8c0 msgr2=0x7f71c406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71ca39b700 1 --2- 172.21.15.131:0/1829345458 >> v2:172.21.15.190:3300/0 conn(0x7f71c406b8c0 0x7f71c406bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71ca39b700 1 -- 172.21.15.131:0/1829345458 >> v1:172.21.15.190:6789/0 conn(0x7f71c40cc100 legacy=0x7f71c40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71ca39b700 1 -- 172.21.15.131:0/1829345458 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c4068910 con 0x7f71c413c510 2023-12-26T22:53:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.628+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/1829345458 >> v2:172.21.15.190:3300/0 conn(0x7f71c406b8c0 0x7f71c406bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71ca39b700 1 --2- 172.21.15.131:0/1829345458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40c8a20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f71c00098d0 tx=0x7f71c0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbd943401a3a9216 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71c9399700 1 -- 172.21.15.131:0/1829345458 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c0005c10 con 0x7f71c413c510 2023-12-26T22:53:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71c9399700 1 -- 172.21.15.131:0/1829345458 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c0005d70 con 0x7f71c413c510 2023-12-26T22:53:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71c9399700 1 -- 172.21.15.131:0/1829345458 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c0006260 con 0x7f71c413c510 2023-12-26T22:53:28.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1829345458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 msgr2=0x7f71c40c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71cc5ff700 1 --2- 172.21.15.131:0/1829345458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40c8a20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f71c00098d0 tx=0x7f71c0009120 comp rx=0 tx=0).stop 2023-12-26T22:53:28.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1829345458 shutdown_connections 2023-12-26T22:53:28.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71cc5ff700 1 --2- 172.21.15.131:0/1829345458 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40c8a20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71cc5ff700 1 --2- 172.21.15.131:0/1829345458 >> v2:172.21.15.190:3300/0 conn(0x7f71c406b8c0 0x7f71c406bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.629+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1829345458 >> 172.21.15.131:0/1829345458 conn(0x7f71c40ce3e0 msgr2=0x7f71c40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:28.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.630+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1829345458 shutdown_connections 2023-12-26T22:53:28.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.630+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1829345458 wait complete. 2023-12-26T22:53:28.635 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.631+0000 7f71cc5ff700 1 Processor -- start 2023-12-26T22:53:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.631+0000 7f71cc5ff700 1 -- start start 2023-12-26T22:53:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71cc5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c406b8c0 0x7f71c413be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71cc5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c40cc100 0x7f71c40b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71cc5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71cc5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c413c390 con 0x7f71c406b8c0 2023-12-26T22:53:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71cc5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c415e0c0 con 0x7f71c40cc100 2023-12-26T22:53:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71cc5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c415e3a0 con 0x7f71c413c510 2023-12-26T22:53:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71ca39b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c406b8c0 0x7f71c413be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71ca39b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c406b8c0 0x7f71c413be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48212/0 (socket says 172.21.15.131:48212) 2023-12-26T22:53:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71ca39b700 1 -- 172.21.15.131:0/1475106488 learned_addr learned my addr 172.21.15.131:0/1475106488 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71cab9c700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.632+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c40cc100 0x7f71c40b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71ca39b700 1 -- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 msgr2=0x7f71c40b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71ca39b700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71ca39b700 1 -- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c40cc100 msgr2=0x7f71c40b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71ca39b700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c40cc100 0x7f71c40b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71ca39b700 1 -- 172.21.15.131:0/1475106488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b4009420 con 0x7f71c406b8c0 2023-12-26T22:53:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c40cc100 0x7f71c40b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71cab9c700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71ca39b700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c406b8c0 0x7f71c413be50 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009ec0 tx=0x7f71c0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71ba7fc700 1 -- 172.21.15.131:0/1475106488 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c001d6b0 con 0x7f71c406b8c0 2023-12-26T22:53:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71ba7fc700 1 -- 172.21.15.131:0/1475106488 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c001d810 con 0x7f71c406b8c0 2023-12-26T22:53:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71ba7fc700 1 -- 172.21.15.131:0/1475106488 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c001dbd0 con 0x7f71c406b8c0 2023-12-26T22:53:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1475106488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c00093c0 con 0x7f71c406b8c0 2023-12-26T22:53:28.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.633+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1475106488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c415e960 con 0x7f71c406b8c0 2023-12-26T22:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.635+0000 7f71ba7fc700 1 -- 172.21.15.131:0/1475106488 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f71c001a420 con 0x7f71c406b8c0 2023-12-26T22:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.636+0000 7f71ba7fc700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f71a8066f30 0x7f71a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.636+0000 7f71ba7fc700 1 -- 172.21.15.131:0/1475106488 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f71c0085cd0 con 0x7f71c406b8c0 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.637+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f71a8066f30 0x7f71a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.637+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f71a8066f30 0x7f71a80693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71c40c58d0 tx=0x7f71b40091a0 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.638+0000 7f71dacedb80 1 Processor -- start 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.638+0000 7f71dacedb80 1 -- start start 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc421370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc4218b0 0x55a3cc41cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 0x55a3cc43d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71dacedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc34f3e0 con 0x55a3cc420fa0 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71dacedb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc34cbe0 con 0x55a3cc4218b0 2023-12-26T22:53:28.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71dacedb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc3489d0 con 0x55a3cc41d170 2023-12-26T22:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71ca39b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc421370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71ca39b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc421370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48216/0 (socket says 172.21.15.131:48216) 2023-12-26T22:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71ca39b700 1 -- 172.21.15.131:0/2222385830 learned_addr learned my addr 172.21.15.131:0/2222385830 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71cab9c700 1 --2- 172.21.15.131:0/2222385830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 0x55a3cc43d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.639+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/2222385830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc4218b0 0x55a3cc41cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.640+0000 7f71ca39b700 1 -- 172.21.15.131:0/2222385830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 msgr2=0x55a3cc43d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.640+0000 7f71ca39b700 1 --2- 172.21.15.131:0/2222385830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 0x55a3cc43d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.640+0000 7f71ca39b700 1 -- 172.21.15.131:0/2222385830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc4218b0 msgr2=0x55a3cc41cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.640+0000 7f71ca39b700 1 --2- 172.21.15.131:0/2222385830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc4218b0 0x55a3cc41cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.640+0000 7f71ca39b700 1 -- 172.21.15.131:0/2222385830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3cc1dc360 con 0x55a3cc420fa0 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.640+0000 7f71ca39b700 1 --2- 172.21.15.131:0/2222385830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc421370 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f71c0056320 tx=0x7f71c0083ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad00c8d168ffffb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.640+0000 7f71a77fe700 1 -- 172.21.15.131:0/2222385830 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c0056990 con 0x55a3cc420fa0 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.640+0000 7f71a77fe700 1 -- 172.21.15.131:0/2222385830 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c0059be0 con 0x55a3cc420fa0 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.640+0000 7f71a77fe700 1 -- 172.21.15.131:0/2222385830 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c005d930 con 0x55a3cc420fa0 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.641+0000 7f71dacedb80 1 -- 172.21.15.131:0/2222385830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 msgr2=0x55a3cc421370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.641+0000 7f71dacedb80 1 --2- 172.21.15.131:0/2222385830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc421370 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f71c0056320 tx=0x7f71c0083ee0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.641+0000 7f71dacedb80 1 -- 172.21.15.131:0/2222385830 shutdown_connections 2023-12-26T22:53:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.641+0000 7f71dacedb80 1 --2- 172.21.15.131:0/2222385830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 0x55a3cc43d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.641+0000 7f71dacedb80 1 --2- 172.21.15.131:0/2222385830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc421370 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.641+0000 7f71dacedb80 1 --2- 172.21.15.131:0/2222385830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc4218b0 0x55a3cc41cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.641+0000 7f71dacedb80 1 -- 172.21.15.131:0/2222385830 >> 172.21.15.131:0/2222385830 conn(0x55a3cc3f7440 msgr2=0x55a3cc442dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.641+0000 7f71dacedb80 1 -- 172.21.15.131:0/2222385830 shutdown_connections 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.641+0000 7f71dacedb80 1 -- 172.21.15.131:0/2222385830 wait complete. 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.642+0000 7f71dacedb80 1 -- start start 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.642+0000 7f71dacedb80 1 Processor -- start 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc41d170 0x55a3cc439ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc420fa0 0x55a3cc41bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc4218b0 0x55a3cc41ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71dacedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc2f51b0 con 0x55a3cc41d170 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71dacedb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc2f4e50 con 0x55a3cc4218b0 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71dacedb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc241e40 con 0x55a3cc420fa0 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71dacedb80 10 client.0 _ll_register_callbacks cb 0x55a3cc2447b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71ca39b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc41d170 0x55a3cc439ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71ca39b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc41d170 0x55a3cc439ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48222/0 (socket says 172.21.15.131:48222) 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71ca39b700 1 -- 172.21.15.131:0/1160396723 learned_addr learned my addr 172.21.15.131:0/1160396723 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71ca39b700 1 -- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc420fa0 msgr2=0x55a3cc41bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71ca39b700 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc420fa0 0x55a3cc41bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71ca39b700 1 -- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc4218b0 msgr2=0x55a3cc41ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71ca39b700 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc4218b0 0x55a3cc41ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.643+0000 7f71ca39b700 1 -- 172.21.15.131:0/1160396723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c415e960 con 0x55a3cc41d170 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71ca39b700 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc41d170 0x55a3cc439ef0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f71c006cfd0 tx=0x7f71c006d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c885257198a9457b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71a4ff9700 1 -- 172.21.15.131:0/1160396723 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c005d930 con 0x55a3cc41d170 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71a4ff9700 1 -- 172.21.15.131:0/1160396723 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f71c0055e80 con 0x55a3cc41d170 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71dacedb80 1 -- 172.21.15.131:0/1160396723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3cc3c9eb0 con 0x55a3cc41d170 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71a4ff9700 1 -- 172.21.15.131:0/1160396723 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71c005d4a0 con 0x55a3cc41d170 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71dacedb80 10 client.9225 fetch_fsmap learned FSMap version 81 2023-12-26T22:53:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71dacedb80 1 -- 172.21.15.131:0/1160396723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55a3cc188b10 con 0x55a3cc41d170 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71a4ff9700 1 -- 172.21.15.131:0/1160396723 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3cc3c9eb0 con 0x55a3cc41d170 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71a4ff9700 1 -- 172.21.15.131:0/1160396723 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 81) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f71c00722e0 con 0x55a3cc41d170 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71dacedb80 10 client.9225 fetch_fsmap finished waiting for FSMap version 81 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71dacedb80 10 client.9225 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71dacedb80 20 client.9225 populate_metadata read hostname 'smithi131' 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71dacedb80 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] conn(0x55a3cc451ef0 0x55a3cc4522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71dacedb80 4 client.9225 mds_command: new command op to 9105 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.644+0000 7f71dacedb80 1 -- 172.21.15.131:0/1160396723 --> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3cc31f880 con 0x55a3cc451ef0 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.645+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] conn(0x55a3cc451ef0 0x55a3cc4522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.645+0000 7f71a4ff9700 10 client.9225 ms_handle_connect on v2:172.21.15.190:6832/716462328 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.645+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] conn(0x55a3cc451ef0 0x55a3cc4522c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f6234c2d397384 server_cookie=aa6d32ef24a652a in_seq=0 out_seq=0 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.646+0000 7f71a4ff9700 1 -- 172.21.15.131:0/1160396723 <== mds.0 v2:172.21.15.190:6832/716462328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a3cc31f880 con 0x55a3cc451ef0 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.646+0000 7f71a4ff9700 10 client.9225 handle_command_reply: tid=0 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.646+0000 7f71dacedb80 1 client.9225 shutdown 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 -- 172.21.15.131:0/1160396723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc41d170 msgr2=0x55a3cc439ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc41d170 0x55a3cc439ef0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f71c006cfd0 tx=0x7f71c006d000 comp rx=0 tx=0).stop 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 -- 172.21.15.131:0/1160396723 shutdown_connections 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] conn(0x55a3cc451ef0 0x55a3cc4522c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc4218b0 0x55a3cc41ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc41d170 0x55a3cc439ef0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 --2- 172.21.15.131:0/1160396723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc420fa0 0x55a3cc41bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71a4ff9700 0 client.9225 ms_handle_reset on v2:172.21.15.190:6832/716462328 2023-12-26T22:53:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 -- 172.21.15.131:0/1160396723 >> 172.21.15.131:0/1160396723 conn(0x55a3cc3f7440 msgr2=0x55a3cc3f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 -- 172.21.15.131:0/1160396723 shutdown_connections 2023-12-26T22:53:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 1 -- 172.21.15.131:0/1160396723 wait complete. 2023-12-26T22:53:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.647+0000 7f71dacedb80 20 client.9225 trim_cache size 0 max 16384 2023-12-26T22:53:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.664+0000 7f71dacedb80 1 Processor -- start 2023-12-26T22:53:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.665+0000 7f71dacedb80 1 -- start start 2023-12-26T22:53:28.667 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc41d170 0x55a3cc3f5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc419f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc4218b0 0x55a3cc41a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71dacedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc4216e0 con 0x55a3cc420fa0 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71dacedb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc41e840 con 0x55a3cc41d170 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71dacedb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc3ef760 con 0x55a3cc4218b0 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71c9b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc419f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71c9b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc419f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48236/0 (socket says 172.21.15.131:48236) 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71c9b9a700 1 -- 172.21.15.131:0/2101203475 learned_addr learned my addr 172.21.15.131:0/2101203475 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71cab9c700 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc4218b0 0x55a3cc41a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71ca39b700 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc41d170 0x55a3cc3f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71c9b9a700 1 -- 172.21.15.131:0/2101203475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc4218b0 msgr2=0x55a3cc41a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc4218b0 0x55a3cc41a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71c9b9a700 1 -- 172.21.15.131:0/2101203475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc41d170 msgr2=0x55a3cc3f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc41d170 0x55a3cc3f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71c9b9a700 1 -- 172.21.15.131:0/2101203475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3cc317290 con 0x55a3cc420fa0 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.666+0000 7f71cab9c700 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc4218b0 0x55a3cc41a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71c9b9a700 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc419f40 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x55a3cc422290 tx=0x7f71b4028250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9d02e167dfad7e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71a7fff700 1 -- 172.21.15.131:0/2101203475 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4028070 con 0x55a3cc420fa0 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71a7fff700 1 -- 172.21.15.131:0/2101203475 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4023560 con 0x55a3cc420fa0 2023-12-26T22:53:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71a7fff700 1 -- 172.21.15.131:0/2101203475 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4023900 con 0x55a3cc420fa0 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71dacedb80 1 -- 172.21.15.131:0/2101203475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 msgr2=0x55a3cc419f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71dacedb80 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc419f40 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x55a3cc422290 tx=0x7f71b4028250 comp rx=0 tx=0).stop 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71dacedb80 1 -- 172.21.15.131:0/2101203475 shutdown_connections 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71dacedb80 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc4218b0 0x55a3cc41a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71dacedb80 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc41d170 0x55a3cc3f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71dacedb80 1 --2- 172.21.15.131:0/2101203475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc420fa0 0x55a3cc419f40 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71dacedb80 1 -- 172.21.15.131:0/2101203475 >> 172.21.15.131:0/2101203475 conn(0x55a3cc3f7440 msgr2=0x55a3cc3eefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71dacedb80 1 -- 172.21.15.131:0/2101203475 shutdown_connections 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.667+0000 7f71dacedb80 1 -- 172.21.15.131:0/2101203475 wait complete. 2023-12-26T22:53:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.668+0000 7f71dacedb80 1 -- start start 2023-12-26T22:53:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.668+0000 7f71dacedb80 1 Processor -- start 2023-12-26T22:53:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.669+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 0x55a3cc419e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.669+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc420fa0 0x55a3cc41a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.669+0000 7f71dacedb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc4218b0 0x55a3cc441750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.669+0000 7f71dacedb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc4427f0 con 0x55a3cc4218b0 2023-12-26T22:53:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.669+0000 7f71dacedb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc41c720 con 0x55a3cc420fa0 2023-12-26T22:53:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.669+0000 7f71dacedb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc458520 con 0x55a3cc41d170 2023-12-26T22:53:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.669+0000 7f71dacedb80 10 client.0 _ll_register_callbacks cb 0x7f71b4011dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T22:53:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71cab9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc4218b0 0x55a3cc441750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71cab9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc4218b0 0x55a3cc441750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48252/0 (socket says 172.21.15.131:48252) 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71c9b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc420fa0 0x55a3cc41a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71cab9c700 1 -- 172.21.15.131:0/901935727 learned_addr learned my addr 172.21.15.131:0/901935727 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71ca39b700 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 0x55a3cc419e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71cab9c700 1 -- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 msgr2=0x55a3cc419e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71cab9c700 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 0x55a3cc419e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71cab9c700 1 -- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc420fa0 msgr2=0x55a3cc41a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71cab9c700 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc420fa0 0x55a3cc41a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71cab9c700 1 -- 172.21.15.131:0/901935727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b4014600 con 0x55a3cc4218b0 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71cab9c700 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc4218b0 0x55a3cc441750 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0143f0 tx=0x7f71bc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d066a366c355926 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T22:53:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71a4ff9700 1 -- 172.21.15.131:0/901935727 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc016420 con 0x55a3cc4218b0 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71a4ff9700 1 -- 172.21.15.131:0/901935727 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f71bc016580 con 0x55a3cc4218b0 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.670+0000 7f71dacedb80 1 -- 172.21.15.131:0/901935727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71c4151c50 con 0x55a3cc4218b0 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71a4ff9700 1 -- 172.21.15.131:0/901935727 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71bc016950 con 0x55a3cc4218b0 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71dacedb80 10 client.9231 fetch_fsmap learned FSMap version 81 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71dacedb80 1 -- 172.21.15.131:0/901935727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55a3cc1e0c80 con 0x55a3cc4218b0 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71a4ff9700 1 -- 172.21.15.131:0/901935727 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4151c50 con 0x55a3cc4218b0 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71a4ff9700 1 -- 172.21.15.131:0/901935727 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 81) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f71bc01c020 con 0x55a3cc4218b0 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71dacedb80 10 client.9231 fetch_fsmap finished waiting for FSMap version 81 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71dacedb80 10 client.9231 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71dacedb80 20 client.9231 populate_metadata read hostname 'smithi131' 2023-12-26T22:53:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71dacedb80 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] conn(0x55a3cc441c90 0x55a3cc43d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71dacedb80 4 client.9231 mds_command: new command op to 9105 tid=0[{"prefix": "status"}] 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.671+0000 7f71dacedb80 1 -- 172.21.15.131:0/901935727 --> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a3cc3ed490 con 0x55a3cc441c90 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.672+0000 7f71ca39b700 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] conn(0x55a3cc441c90 0x55a3cc43d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.672+0000 7f71ca39b700 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] conn(0x55a3cc441c90 0x55a3cc43d720 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c9b0abd7b516874 server_cookie=6c4dd4882242bea6 in_seq=0 out_seq=0 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.672+0000 7f71a4ff9700 10 client.9231 ms_handle_connect on v2:172.21.15.190:6832/716462328 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.672+0000 7f71a4ff9700 1 -- 172.21.15.131:0/901935727 <== mds.0 v2:172.21.15.190:6832/716462328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55a3cc3ed490 con 0x55a3cc441c90 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.672+0000 7f71a4ff9700 10 client.9231 handle_command_reply: tid=0 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.672+0000 7f71dacedb80 1 client.9231 shutdown 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.672+0000 7f71dacedb80 1 -- 172.21.15.131:0/901935727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc4218b0 msgr2=0x55a3cc441750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.672+0000 7f71dacedb80 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc4218b0 0x55a3cc441750 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0143f0 tx=0x7f71bc009040 comp rx=0 tx=0).stop 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71dacedb80 1 -- 172.21.15.131:0/901935727 shutdown_connections 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71dacedb80 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a3cc4218b0 0x55a3cc441750 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71dacedb80 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a3cc41d170 0x55a3cc419e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71dacedb80 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:6832/716462328,v1:172.21.15.190:6834/716462328] conn(0x55a3cc441c90 0x55a3cc43d720 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71dacedb80 1 --2- 172.21.15.131:0/901935727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3cc420fa0 0x55a3cc41a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71dacedb80 1 -- 172.21.15.131:0/901935727 >> 172.21.15.131:0/901935727 conn(0x55a3cc3f7440 msgr2=0x55a3cc3f47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71a4ff9700 0 client.9231 ms_handle_reset on v2:172.21.15.190:6832/716462328 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71dacedb80 1 -- 172.21.15.131:0/901935727 shutdown_connections 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71dacedb80 1 -- 172.21.15.131:0/901935727 wait complete. 2023-12-26T22:53:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71dacedb80 20 client.9231 trim_cache size 0 max 16384 2023-12-26T22:53:28.680 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T22:53:28.680 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T22:53:28.680 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T22:53:28.680 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 9105, 2023-12-26T22:53:28.680 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T22:53:28.680 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T22:53:28.680 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T22:53:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.4871534439999996, 2023-12-26T22:53:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 81, 2023-12-26T22:53:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 175, 2023-12-26T22:53:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 175, 2023-12-26T22:53:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 12.203685738000001 2023-12-26T22:53:28.681 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T22:53:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f71a8066f30 msgr2=0x7f71a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71cc5ff700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f71a8066f30 0x7f71a80693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71c40c58d0 tx=0x7f71b40091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c406b8c0 msgr2=0x7f71c413be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.673+0000 7f71cc5ff700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c406b8c0 0x7f71c413be50 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009ec0 tx=0x7f71c0016000 comp rx=0 tx=0).stop 2023-12-26T22:53:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.674+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1475106488 shutdown_connections 2023-12-26T22:53:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.674+0000 7f71cc5ff700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71c413c510 0x7f71c40b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.674+0000 7f71cc5ff700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f71a8066f30 0x7f71a80693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.674+0000 7f71cc5ff700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c406b8c0 0x7f71c413be50 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.674+0000 7f71cc5ff700 1 --2- 172.21.15.131:0/1475106488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71c40cc100 0x7f71c40b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.674+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1475106488 >> 172.21.15.131:0/1475106488 conn(0x7f71c40ce3e0 msgr2=0x7f71c41524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.674+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1475106488 shutdown_connections 2023-12-26T22:53:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.674+0000 7f71cc5ff700 1 -- 172.21.15.131:0/1475106488 wait complete. 2023-12-26T22:53:28.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T22:53:28.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.879+0000 7f6e4bbff700 1 Processor -- start 2023-12-26T22:53:28.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.879+0000 7f6e4bbff700 1 -- start start 2023-12-26T22:53:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.880+0000 7f6e4bbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6e44159820 0x7f6e440abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.880+0000 7f6e4bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e440aa480 0x7f6e4406dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.880+0000 7f6e4bbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e4406e2b0 0x7f6e4415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.880+0000 7f6e4bbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4415ad50 con 0x7f6e44159820 2023-12-26T22:53:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.880+0000 7f6e4bbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4406e730 con 0x7f6e4406e2b0 2023-12-26T22:53:28.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.880+0000 7f6e4bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415a8c0 con 0x7f6e440aa480 2023-12-26T22:53:28.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6e44159820 0x7f6e440abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6e44159820 0x7f6e440abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48268/0 (socket says 172.21.15.131:48268) 2023-12-26T22:53:28.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4999b700 1 -- 172.21.15.131:0/3330428953 learned_addr learned my addr 172.21.15.131:0/3330428953 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:28.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4919a700 1 --2- 172.21.15.131:0/3330428953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e440aa480 0x7f6e4406dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4a19c700 1 --2- 172.21.15.131:0/3330428953 >> v2:172.21.15.190:3300/0 conn(0x7f6e4406e2b0 0x7f6e4415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4999b700 1 -- 172.21.15.131:0/3330428953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e440aa480 msgr2=0x7f6e4406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4999b700 1 --2- 172.21.15.131:0/3330428953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e440aa480 0x7f6e4406dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4999b700 1 -- 172.21.15.131:0/3330428953 >> v2:172.21.15.190:3300/0 conn(0x7f6e4406e2b0 msgr2=0x7f6e4415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4999b700 1 --2- 172.21.15.131:0/3330428953 >> v2:172.21.15.190:3300/0 conn(0x7f6e4406e2b0 0x7f6e4415d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4999b700 1 -- 172.21.15.131:0/3330428953 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e44159f90 con 0x7f6e44159820 2023-12-26T22:53:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e4999b700 1 --2- 172.21.15.131:0/3330428953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 0x7f6e440abdb0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f6e4406cbd0 tx=0x7f6e40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9746512d35e0265e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.881+0000 7f6e48999700 1 -- 172.21.15.131:0/3330428953 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000b820 con 0x7f6e44159820 2023-12-26T22:53:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e48999700 1 -- 172.21.15.131:0/3330428953 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4000b980 con 0x7f6e44159820 2023-12-26T22:53:28.885 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e48999700 1 -- 172.21.15.131:0/3330428953 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000bd20 con 0x7f6e44159820 2023-12-26T22:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e4bbff700 1 -- 172.21.15.131:0/3330428953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 msgr2=0x7f6e440abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/3330428953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 0x7f6e440abdb0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f6e4406cbd0 tx=0x7f6e40009250 comp rx=0 tx=0).stop 2023-12-26T22:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e4bbff700 1 -- 172.21.15.131:0/3330428953 shutdown_connections 2023-12-26T22:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/3330428953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 0x7f6e440abdb0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/3330428953 >> v2:172.21.15.190:3300/0 conn(0x7f6e4406e2b0 0x7f6e4415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/3330428953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e440aa480 0x7f6e4406dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e4bbff700 1 -- 172.21.15.131:0/3330428953 >> 172.21.15.131:0/3330428953 conn(0x7f6e4406c290 msgr2=0x7f6e440ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:28.886 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e4bbff700 1 -- 172.21.15.131:0/3330428953 shutdown_connections 2023-12-26T22:53:28.887 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.882+0000 7f6e4bbff700 1 -- 172.21.15.131:0/3330428953 wait complete. 2023-12-26T22:53:28.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.884+0000 7f6e4bbff700 1 Processor -- start 2023-12-26T22:53:28.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.885+0000 7f6e4bbff700 1 -- start start 2023-12-26T22:53:28.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4406e2b0 0x7f6e44145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4bbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e440aa480 0x7f6e4415af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4bbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 0x7f6e4415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4bbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b0060 con 0x7f6e44159820 2023-12-26T22:53:28.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4bbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b0620 con 0x7f6e440aa480 2023-12-26T22:53:28.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44145a80 con 0x7f6e4406e2b0 2023-12-26T22:53:28.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4a19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 0x7f6e4415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4a19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 0x7f6e4415b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48284/0 (socket says 172.21.15.131:48284) 2023-12-26T22:53:28.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4a19c700 1 -- 172.21.15.131:0/1744650527 learned_addr learned my addr 172.21.15.131:0/1744650527 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:28.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4999b700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4406e2b0 0x7f6e44145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4919a700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e440aa480 0x7f6e4415af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4a19c700 1 -- 172.21.15.131:0/1744650527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4406e2b0 msgr2=0x7f6e44145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4a19c700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4406e2b0 0x7f6e44145540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.892 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4a19c700 1 -- 172.21.15.131:0/1744650527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e440aa480 msgr2=0x7f6e4415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:28.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4a19c700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e440aa480 0x7f6e4415af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:28.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.886+0000 7f6e4a19c700 1 -- 172.21.15.131:0/1744650527 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400094f0 con 0x7f6e44159820 2023-12-26T22:53:28.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.887+0000 7f6e4a19c700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 0x7f6e4415b470 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c007f20 tx=0x7f6e3c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.887+0000 7f6e39ffb700 1 -- 172.21.15.131:0/1744650527 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00eca0 con 0x7f6e44159820 2023-12-26T22:53:28.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.887+0000 7f6e39ffb700 1 -- 172.21.15.131:0/1744650527 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c00ee20 con 0x7f6e44159820 2023-12-26T22:53:28.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.887+0000 7f6e39ffb700 1 -- 172.21.15.131:0/1744650527 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c014670 con 0x7f6e44159820 2023-12-26T22:53:28.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.887+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e440c2940 con 0x7f6e44159820 2023-12-26T22:53:28.893 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.887+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e440b02b0 con 0x7f6e44159820 2023-12-26T22:53:28.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.888+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e440b0790 con 0x7f6e44159820 2023-12-26T22:53:28.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.889+0000 7f6e39ffb700 1 -- 172.21.15.131:0/1744650527 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6e3c0149e0 con 0x7f6e44159820 2023-12-26T22:53:28.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.890+0000 7f6e39ffb700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6e28066f30 0x7f6e280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:28.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.890+0000 7f6e39ffb700 1 -- 172.21.15.131:0/1744650527 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f6e3c084eb0 con 0x7f6e44159820 2023-12-26T22:53:28.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.890+0000 7f6e4999b700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6e28066f30 0x7f6e280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:28.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.891+0000 7f6e4999b700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6e28066f30 0x7f6e280693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e4406cbd0 tx=0x7f6e40017000 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:28.895 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:28.894+0000 7f6e39ffb700 1 -- 172.21.15.131:0/1744650527 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e3c051eb0 con 0x7f6e44159820 2023-12-26T22:53:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.127+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6e4415a270 con 0x7f6e44159820 2023-12-26T22:53:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.128+0000 7f6e39ffb700 1 -- 172.21.15.131:0/1744650527 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6e3c0554d0 con 0x7f6e44159820 2023-12-26T22:53:29.128 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T22:53:29.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6e28066f30 msgr2=0x7f6e280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:29.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6e28066f30 0x7f6e280693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e4406cbd0 tx=0x7f6e40017000 comp rx=0 tx=0).stop 2023-12-26T22:53:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 msgr2=0x7f6e4415b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 0x7f6e4415b470 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c007f20 tx=0x7f6e3c009860 comp rx=0 tx=0).stop 2023-12-26T22:53:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 shutdown_connections 2023-12-26T22:53:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6e28066f30 0x7f6e280693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e44159820 0x7f6e4415b470 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4406e2b0 0x7f6e44145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 --2- 172.21.15.131:0/1744650527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e440aa480 0x7f6e4415af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 >> 172.21.15.131:0/1744650527 conn(0x7f6e4406c290 msgr2=0x7f6e4415c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 shutdown_connections 2023-12-26T22:53:29.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:29.130+0000 7f6e4bbff700 1 -- 172.21.15.131:0/1744650527 wait complete. 2023-12-26T22:53:29.148 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T22:53:29.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T22:53:29.148 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T22:53:29.148 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T22:53:29.148 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T22:53:29.148 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T22:53:29.148 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T22:53:29.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T22:53:29.149 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T22:53:29.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:53:29.149 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T22:53:29.195 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T22:53:29.195 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T22:53:29.195 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T22:53:29.195 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:53:29.195 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T22:53:29.196 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:53:29.196 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T22:53:29.196 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T22:53:29.196 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T22:53:29.196 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T22:53:29.196 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T22:53:29.196 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T22:53:29.196 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T22:53:29.196 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T22:53:29.197 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T22:53:29.197 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T22:53:29.197 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T22:53:29.197 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 37847sec preferred_lft 37847sec 2023-12-26T22:53:29.197 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T22:53:29.197 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:53:29.197 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:53:29.198 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T22:53:29.199 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T22:53:29.199 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T22:53:29.199 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T22:53:29.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:53:29.199 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T22:53:29.301 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T22:53:29.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:53:29.304 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T22:53:29.351 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T22:53:29.352 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T22:53:29.352 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T22:53:29.352 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T22:53:29.352 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T22:53:29.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:53:29.353 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T22:53:29.353 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T22:53:29.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T22:53:29.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T22:53:29.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T22:53:29.353 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T22:53:29.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:53:29.564 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T22:53:29.609 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T22:53:29.609 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T22:53:29.610 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T22:53:29.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:53:29.610 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T22:53:29.718 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T22:53:29.719 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T22:53:29.762 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T22:53:29.762 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T22:53:29.762 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T22:53:29.845 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T22:53:29.846 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T22:53:29.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T22:53:29.983 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T22:53:29.983 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T22:53:30.012 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T22:53:30.012 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T22:53:30.012 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T22:53:30.012 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T22:53:30.012 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T22:53:30.018 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T22:53:30.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T22:53:30.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T22:53:30.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T22:53:30.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.283+0000 7f70b21ff700 1 Processor -- start 2023-12-26T22:53:30.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.283+0000 7f70b21ff700 1 -- start start 2023-12-26T22:53:30.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70b21ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f70ac06bfd0 0x7f70ac0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70b21ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ac0686c0 con 0x7f70ac13c510 2023-12-26T22:53:30.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70b21ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ac069520 con 0x7f70ac0cc100 2023-12-26T22:53:30.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70b21ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ac068910 con 0x7f70ac06bfd0 2023-12-26T22:53:30.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70aaffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f70ac06bfd0 0x7f70ac0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70aaffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f70ac06bfd0 0x7f70ac0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48298/0 (socket says 172.21.15.131:48298) 2023-12-26T22:53:30.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70aaffd700 1 -- 172.21.15.131:0/2894983452 learned_addr learned my addr 172.21.15.131:0/2894983452 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:30.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70aaffd700 1 -- 172.21.15.131:0/2894983452 >> v1:172.21.15.190:6789/0 conn(0x7f70ac0cc100 legacy=0x7f70ac0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:30.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70aaffd700 1 -- 172.21.15.131:0/2894983452 >> v1:172.21.15.131:6789/0 conn(0x7f70ac13c510 legacy=0x7f70ac06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.285+0000 7f70aaffd700 1 -- 172.21.15.131:0/2894983452 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac068bf0 con 0x7f70ac06bfd0 2023-12-26T22:53:30.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.286+0000 7f70aaffd700 1 --2- 172.21.15.131:0/2894983452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac06bfd0 0x7f70ac0c8a20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f70a00099b0 tx=0x7f70a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721159ebda097f78 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:30.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.286+0000 7f70aa7fc700 1 -- 172.21.15.131:0/2894983452 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a000b820 con 0x7f70ac06bfd0 2023-12-26T22:53:30.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.286+0000 7f70aa7fc700 1 -- 172.21.15.131:0/2894983452 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a000b980 con 0x7f70ac06bfd0 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.286+0000 7f70aa7fc700 1 -- 172.21.15.131:0/2894983452 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a000bd20 con 0x7f70ac06bfd0 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.286+0000 7f70b21ff700 1 -- 172.21.15.131:0/2894983452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac06bfd0 msgr2=0x7f70ac0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.286+0000 7f70b21ff700 1 --2- 172.21.15.131:0/2894983452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac06bfd0 0x7f70ac0c8a20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f70a00099b0 tx=0x7f70a0009250 comp rx=0 tx=0).stop 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.287+0000 7f70b21ff700 1 -- 172.21.15.131:0/2894983452 shutdown_connections 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.287+0000 7f70b21ff700 1 --2- 172.21.15.131:0/2894983452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac06bfd0 0x7f70ac0c8a20 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.287+0000 7f70b21ff700 1 -- 172.21.15.131:0/2894983452 >> 172.21.15.131:0/2894983452 conn(0x7f70ac0ce3e0 msgr2=0x7f70ac0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.287+0000 7f70b21ff700 1 -- 172.21.15.131:0/2894983452 shutdown_connections 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.287+0000 7f70b21ff700 1 -- 172.21.15.131:0/2894983452 wait complete. 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.288+0000 7f70b21ff700 1 Processor -- start 2023-12-26T22:53:30.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.288+0000 7f70b21ff700 1 -- start start 2023-12-26T22:53:30.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.289+0000 7f70b21ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70ac06bfd0 0x7f70ac15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.289+0000 7f70b21ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac0cc100 0x7f70ac0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.289+0000 7f70b21ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70ac13c510 0x7f70ac16da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.289+0000 7f70b21ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac15f160 con 0x7f70ac0cc100 2023-12-26T22:53:30.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.289+0000 7f70b21ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0b3380 con 0x7f70ac13c510 2023-12-26T22:53:30.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.289+0000 7f70b21ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0b2770 con 0x7f70ac06bfd0 2023-12-26T22:53:30.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.289+0000 7f70aaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac0cc100 0x7f70ac0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.289+0000 7f70aaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac0cc100 0x7f70ac0c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48304/0 (socket says 172.21.15.131:48304) 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.289+0000 7f70aaffd700 1 -- 172.21.15.131:0/3096019208 learned_addr learned my addr 172.21.15.131:0/3096019208 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70abfff700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70ac13c510 0x7f70ac16da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70ab7fe700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70ac06bfd0 0x7f70ac15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70aaffd700 1 -- 172.21.15.131:0/3096019208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70ac06bfd0 msgr2=0x7f70ac15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70aaffd700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70ac06bfd0 0x7f70ac15ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70aaffd700 1 -- 172.21.15.131:0/3096019208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70ac13c510 msgr2=0x7f70ac16da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70aaffd700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70ac13c510 0x7f70ac16da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70aaffd700 1 -- 172.21.15.131:0/3096019208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a00094f0 con 0x7f70ac0cc100 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70ab7fe700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70ac06bfd0 0x7f70ac15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70aaffd700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac0cc100 0x7f70ac0c63b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f70a000c010 tx=0x7f70a0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:30.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70937fe700 1 -- 172.21.15.131:0/3096019208 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011e10 con 0x7f70ac0cc100 2023-12-26T22:53:30.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70937fe700 1 -- 172.21.15.131:0/3096019208 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a0020070 con 0x7f70ac0cc100 2023-12-26T22:53:30.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac0b2a50 con 0x7f70ac0cc100 2023-12-26T22:53:30.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.290+0000 7f70937fe700 1 -- 172.21.15.131:0/3096019208 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a001b6a0 con 0x7f70ac0cc100 2023-12-26T22:53:30.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.291+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac0b3010 con 0x7f70ac0cc100 2023-12-26T22:53:30.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.292+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac0657e0 con 0x7f70ac0cc100 2023-12-26T22:53:30.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.292+0000 7f70937fe700 1 -- 172.21.15.131:0/3096019208 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f70a0022030 con 0x7f70ac0cc100 2023-12-26T22:53:30.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.293+0000 7f70937fe700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7098066f30 0x7f70980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.293+0000 7f70937fe700 1 -- 172.21.15.131:0/3096019208 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f70a0086720 con 0x7f70ac0cc100 2023-12-26T22:53:30.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.293+0000 7f70ab7fe700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7098066f30 0x7f70980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.294+0000 7f70ab7fe700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7098066f30 0x7f70980693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f709c001830 tx=0x7f709c009040 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:30.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.298+0000 7f70937fe700 1 -- 172.21.15.131:0/3096019208 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70a0053720 con 0x7f70ac0cc100 2023-12-26T22:53:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.516+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f70ac0657e0 con 0x7f7098066f30 2023-12-26T22:53:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.517+0000 7f70937fe700 1 -- 172.21.15.131:0/3096019208 <== mgr.8455 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f70ac0657e0 con 0x7f7098066f30 2023-12-26T22:53:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T22:53:30.517 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T22:53:30.517 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T22:53:30.517 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T22:53:30.517 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T22:53:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.519+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7098066f30 msgr2=0x7f70980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.519+0000 7f70b21ff700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7098066f30 0x7f70980693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f709c001830 tx=0x7f709c009040 comp rx=0 tx=0).stop 2023-12-26T22:53:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.519+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac0cc100 msgr2=0x7f70ac0c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.519+0000 7f70b21ff700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac0cc100 0x7f70ac0c63b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f70a000c010 tx=0x7f70a0016000 comp rx=0 tx=0).stop 2023-12-26T22:53:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.519+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 shutdown_connections 2023-12-26T22:53:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.519+0000 7f70b21ff700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70ac13c510 0x7f70ac16da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.520+0000 7f70b21ff700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70ac06bfd0 0x7f70ac15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.520+0000 7f70b21ff700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f7098066f30 0x7f70980693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.520+0000 7f70b21ff700 1 --2- 172.21.15.131:0/3096019208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70ac0cc100 0x7f70ac0c63b0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.520+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 >> 172.21.15.131:0/3096019208 conn(0x7f70ac0ce3e0 msgr2=0x7f70ac13a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.520+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 shutdown_connections 2023-12-26T22:53:30.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.520+0000 7f70b21ff700 1 -- 172.21.15.131:0/3096019208 wait complete. 2023-12-26T22:53:30.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T22:53:30.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.728+0000 7f2a203ff700 1 Processor -- start 2023-12-26T22:53:30.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.729+0000 7f2a203ff700 1 -- start start 2023-12-26T22:53:30.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a203ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a18159a70 0x7f2a180b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a203ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2a180b2a90 0x7f2a180b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a203ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a180afcd0 con 0x7f2a181596a0 2023-12-26T22:53:30.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a203ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a180b3e70 con 0x7f2a18159a70 2023-12-26T22:53:30.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a203ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a180b0f20 con 0x7f2a180b2a90 2023-12-26T22:53:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a1d99a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a18159a70 0x7f2a180b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a1d99a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a18159a70 0x7f2a180b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48316/0 (socket says 172.21.15.131:48316) 2023-12-26T22:53:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a1d99a700 1 -- 172.21.15.131:0/2278381782 learned_addr learned my addr 172.21.15.131:0/2278381782 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a1e99c700 1 --2- 172.21.15.131:0/2278381782 >> v2:172.21.15.190:3300/0 conn(0x7f2a180b2a90 0x7f2a180b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a1d99a700 1 -- 172.21.15.131:0/2278381782 >> v2:172.21.15.190:3300/0 conn(0x7f2a180b2a90 msgr2=0x7f2a180b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a1d99a700 1 --2- 172.21.15.131:0/2278381782 >> v2:172.21.15.190:3300/0 conn(0x7f2a180b2a90 0x7f2a180b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a1d99a700 1 -- 172.21.15.131:0/2278381782 >> v1:172.21.15.131:6789/0 conn(0x7f2a181596a0 legacy=0x7f2a180b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.730+0000 7f2a1d99a700 1 -- 172.21.15.131:0/2278381782 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a180b3540 con 0x7f2a18159a70 2023-12-26T22:53:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.731+0000 7f2a1d99a700 1 --2- 172.21.15.131:0/2278381782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a18159a70 0x7f2a180b0160 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f2a180b66b0 tx=0x7f2a08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b248689f167a5f06 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.731+0000 7f2a1d199700 1 -- 172.21.15.131:0/2278381782 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800b860 con 0x7f2a18159a70 2023-12-26T22:53:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.731+0000 7f2a1d199700 1 -- 172.21.15.131:0/2278381782 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0800b9c0 con 0x7f2a18159a70 2023-12-26T22:53:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.731+0000 7f2a1d199700 1 -- 172.21.15.131:0/2278381782 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800bd60 con 0x7f2a18159a70 2023-12-26T22:53:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.731+0000 7f2a203ff700 1 -- 172.21.15.131:0/2278381782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a18159a70 msgr2=0x7f2a180b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.731+0000 7f2a203ff700 1 --2- 172.21.15.131:0/2278381782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a18159a70 0x7f2a180b0160 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f2a180b66b0 tx=0x7f2a08009250 comp rx=0 tx=0).stop 2023-12-26T22:53:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.732+0000 7f2a203ff700 1 -- 172.21.15.131:0/2278381782 shutdown_connections 2023-12-26T22:53:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.732+0000 7f2a203ff700 1 --2- 172.21.15.131:0/2278381782 >> v2:172.21.15.190:3300/0 conn(0x7f2a180b2a90 0x7f2a180b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.732+0000 7f2a203ff700 1 --2- 172.21.15.131:0/2278381782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a18159a70 0x7f2a180b0160 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.732+0000 7f2a203ff700 1 -- 172.21.15.131:0/2278381782 >> 172.21.15.131:0/2278381782 conn(0x7f2a1806ece0 msgr2=0x7f2a1806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:30.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.732+0000 7f2a203ff700 1 -- 172.21.15.131:0/2278381782 shutdown_connections 2023-12-26T22:53:30.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.732+0000 7f2a203ff700 1 -- 172.21.15.131:0/2278381782 wait complete. 2023-12-26T22:53:30.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.733+0000 7f2a203ff700 1 Processor -- start 2023-12-26T22:53:30.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.733+0000 7f2a203ff700 1 -- start start 2023-12-26T22:53:30.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.734+0000 7f2a203ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a180b2a90 0x7f2a1814a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.734+0000 7f2a203ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a181596a0 0x7f2a180c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.734+0000 7f2a203ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a18159a70 0x7f2a1815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.734+0000 7f2a203ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180c67a0 con 0x7f2a180b2a90 2023-12-26T22:53:30.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.734+0000 7f2a203ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180c5b90 con 0x7f2a181596a0 2023-12-26T22:53:30.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.734+0000 7f2a203ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180c5e70 con 0x7f2a18159a70 2023-12-26T22:53:30.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.734+0000 7f2a1e19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a180b2a90 0x7f2a1814a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.734+0000 7f2a1e19b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a180b2a90 0x7f2a1814a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48332/0 (socket says 172.21.15.131:48332) 2023-12-26T22:53:30.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.734+0000 7f2a1e19b700 1 -- 172.21.15.131:0/2061999436 learned_addr learned my addr 172.21.15.131:0/2061999436 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:30.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a1e99c700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a18159a70 0x7f2a1815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a1d99a700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a181596a0 0x7f2a180c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a1e19b700 1 -- 172.21.15.131:0/2061999436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a18159a70 msgr2=0x7f2a1815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a1e19b700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a18159a70 0x7f2a1815d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a1e19b700 1 -- 172.21.15.131:0/2061999436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a181596a0 msgr2=0x7f2a180c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:30.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a1e19b700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a181596a0 0x7f2a180c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:30.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a1e19b700 1 -- 172.21.15.131:0/2061999436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a080094f0 con 0x7f2a180b2a90 2023-12-26T22:53:30.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a1e19b700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a180b2a90 0x7f2a1814a910 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400c590 tx=0x7f2a1400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:30.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a0e7fc700 1 -- 172.21.15.131:0/2061999436 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14011590 con 0x7f2a180b2a90 2023-12-26T22:53:30.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a0e7fc700 1 -- 172.21.15.131:0/2061999436 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a14011710 con 0x7f2a180b2a90 2023-12-26T22:53:30.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a0e7fc700 1 -- 172.21.15.131:0/2061999436 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14011ad0 con 0x7f2a180b2a90 2023-12-26T22:53:30.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.735+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a180c6150 con 0x7f2a180b2a90 2023-12-26T22:53:30.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.736+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1815cb00 con 0x7f2a180b2a90 2023-12-26T22:53:30.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.736+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a180657e0 con 0x7f2a180b2a90 2023-12-26T22:53:30.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.737+0000 7f2a0e7fc700 1 -- 172.21.15.131:0/2061999436 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 80) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2a1401b3b0 con 0x7f2a180b2a90 2023-12-26T22:53:30.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.738+0000 7f2a0e7fc700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f29fc066f30 0x7f29fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:30.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.738+0000 7f2a0e7fc700 1 -- 172.21.15.131:0/2061999436 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f2a14083c10 con 0x7f2a180b2a90 2023-12-26T22:53:30.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.739+0000 7f2a1d99a700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f29fc066f30 0x7f29fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:30.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.739+0000 7f2a1d99a700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f29fc066f30 0x7f29fc0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2a180b66b0 tx=0x7f2a08017000 comp rx=0 tx=0).ready entity=mgr.8455 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:30.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.742+0000 7f2a0e7fc700 1 -- 172.21.15.131:0/2061999436 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a14050c10 con 0x7f2a180b2a90 2023-12-26T22:53:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:30.988+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2a180b3820 con 0x7f2a180b2a90 2023-12-26T22:53:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.243+0000 7f2a0e7fc700 1 -- 172.21.15.131:0/2061999436 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 81) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f2a14050fe0 con 0x7f2a180b2a90 2023-12-26T22:53:31.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.244+0000 7f2a0e7fc700 1 -- 172.21.15.131:0/2061999436 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v81) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2a1407c600 con 0x7f2a180b2a90 2023-12-26T22:53:31.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f29fc066f30 msgr2=0x7f29fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f29fc066f30 0x7f29fc0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2a180b66b0 tx=0x7f2a08017000 comp rx=0 tx=0).stop 2023-12-26T22:53:31.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a180b2a90 msgr2=0x7f2a1814a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a180b2a90 0x7f2a1814a910 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400c590 tx=0x7f2a1400c8a0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 shutdown_connections 2023-12-26T22:53:31.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a18159a70 0x7f2a1815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f29fc066f30 0x7f29fc0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a180b2a90 0x7f2a1814a910 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 --2- 172.21.15.131:0/2061999436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a181596a0 0x7f2a180c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 >> 172.21.15.131:0/2061999436 conn(0x7f2a1806ece0 msgr2=0x7f2a180cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:31.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 shutdown_connections 2023-12-26T22:53:31.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.247+0000 7f2a203ff700 1 -- 172.21.15.131:0/2061999436 wait complete. 2023-12-26T22:53:31.267 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T22:53:31.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.474+0000 7ffb3d3ff700 1 Processor -- start 2023-12-26T22:53:31.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.474+0000 7ffb3d3ff700 1 -- start start 2023-12-26T22:53:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.476+0000 7ffb3d3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb38159900 0x7ffb380cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.476+0000 7ffb3d3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffb380cd4d0 0x7ffb380cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.476+0000 7ffb3d3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3815db90 con 0x7ffb380cd100 2023-12-26T22:53:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.476+0000 7ffb3d3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb380cbe50 con 0x7ffb380cd4d0 2023-12-26T22:53:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.476+0000 7ffb3d3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3815d330 con 0x7ffb38159900 2023-12-26T22:53:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.476+0000 7ffb377fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffb380cd4d0 0x7ffb380cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb367fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb38159900 0x7ffb380cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb36ffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ffb380cd100 0x7ffb381591f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:35008/0 (socket says 172.21.15.131:35008) 2023-12-26T22:53:31.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb36ffd700 1 -- 172.21.15.131:0/2513063314 learned_addr learned my addr 172.21.15.131:0/2513063314 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:31.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb367fc700 1 -- 172.21.15.131:0/2513063314 >> v2:172.21.15.131:3300/0 conn(0x7ffb380cd4d0 msgr2=0x7ffb380cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:31.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb367fc700 1 --2- 172.21.15.131:0/2513063314 >> v2:172.21.15.131:3300/0 conn(0x7ffb380cd4d0 0x7ffb380cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb367fc700 1 -- 172.21.15.131:0/2513063314 >> v1:172.21.15.190:6789/0 conn(0x7ffb380cd100 legacy=0x7ffb381591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb367fc700 1 -- 172.21.15.131:0/2513063314 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380cbb70 con 0x7ffb38159900 2023-12-26T22:53:31.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb35ffb700 1 -- 172.21.15.131:0/2513063314 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948851887 0 0) 0x7ffb3815db90 con 0x7ffb380cd100 2023-12-26T22:53:31.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb367fc700 1 --2- 172.21.15.131:0/2513063314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb38159900 0x7ffb380cb590 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffb380cf900 tx=0x7ffb2c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eecdf2756a9dfd8e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:31.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.477+0000 7ffb35ffb700 1 -- 172.21.15.131:0/2513063314 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c011630 con 0x7ffb38159900 2023-12-26T22:53:31.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.478+0000 7ffb35ffb700 1 -- 172.21.15.131:0/2513063314 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c011790 con 0x7ffb38159900 2023-12-26T22:53:31.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.478+0000 7ffb35ffb700 1 -- 172.21.15.131:0/2513063314 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c011b30 con 0x7ffb38159900 2023-12-26T22:53:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.478+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/2513063314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb38159900 msgr2=0x7ffb380cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.478+0000 7ffb3d3ff700 1 --2- 172.21.15.131:0/2513063314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb38159900 0x7ffb380cb590 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffb380cf900 tx=0x7ffb2c0091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.479+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/2513063314 shutdown_connections 2023-12-26T22:53:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.479+0000 7ffb3d3ff700 1 --2- 172.21.15.131:0/2513063314 >> v2:172.21.15.131:3300/0 conn(0x7ffb380cd4d0 0x7ffb380cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.479+0000 7ffb3d3ff700 1 --2- 172.21.15.131:0/2513063314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb38159900 0x7ffb380cb590 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.479+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/2513063314 >> 172.21.15.131:0/2513063314 conn(0x7ffb3806ece0 msgr2=0x7ffb3806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.479+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/2513063314 shutdown_connections 2023-12-26T22:53:31.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.479+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/2513063314 wait complete. 2023-12-26T22:53:31.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.481+0000 7ffb3d3ff700 1 Processor -- start 2023-12-26T22:53:31.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.481+0000 7ffb3d3ff700 1 -- start start 2023-12-26T22:53:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb3d3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb380cd100 0x7ffb38169fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb3d3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb380cd4d0 0x7ffb3815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb3d3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb38159900 0x7ffb380b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb3d3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3816a510 con 0x7ffb380cd100 2023-12-26T22:53:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb3d3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3815cf00 con 0x7ffb380cd4d0 2023-12-26T22:53:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb3d3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3815cc40 con 0x7ffb38159900 2023-12-26T22:53:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb36ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb380cd100 0x7ffb38169fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb36ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb380cd100 0x7ffb38169fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48374/0 (socket says 172.21.15.131:48374) 2023-12-26T22:53:31.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb36ffd700 1 -- 172.21.15.131:0/3779204890 learned_addr learned my addr 172.21.15.131:0/3779204890 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb367fc700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb380cd4d0 0x7ffb3815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.483+0000 7ffb377fe700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb38159900 0x7ffb380b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb367fc700 1 -- 172.21.15.131:0/3779204890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb38159900 msgr2=0x7ffb380b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb367fc700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb38159900 0x7ffb380b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb367fc700 1 -- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb380cd100 msgr2=0x7ffb38169fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb367fc700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb380cd100 0x7ffb38169fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb367fc700 1 -- 172.21.15.131:0/3779204890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c009420 con 0x7ffb380cd4d0 2023-12-26T22:53:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb367fc700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb380cd4d0 0x7ffb3815c700 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c016000 tx=0x7ffb2c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:31.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb1effd700 1 -- 172.21.15.131:0/3779204890 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c000d10 con 0x7ffb380cd4d0 2023-12-26T22:53:31.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb1effd700 1 -- 172.21.15.131:0/3779204890 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c011b10 con 0x7ffb380cd4d0 2023-12-26T22:53:31.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb1effd700 1 -- 172.21.15.131:0/3779204890 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c025640 con 0x7ffb380cd4d0 2023-12-26T22:53:31.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb380c8c60 con 0x7ffb380cd4d0 2023-12-26T22:53:31.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.484+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb380c9220 con 0x7ffb380cd4d0 2023-12-26T22:53:31.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.485+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb380657e0 con 0x7ffb380cd4d0 2023-12-26T22:53:31.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.486+0000 7ffb1effd700 1 -- 172.21.15.131:0/3779204890 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 81) v1 ==== 84759+0+0 (secure 0 0 0) 0x7ffb2c0193b0 con 0x7ffb380cd4d0 2023-12-26T22:53:31.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.487+0000 7ffb1effd700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffb24066f80 0x7ffb24069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.487+0000 7ffb1effd700 1 -- 172.21.15.131:0/3779204890 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7ffb2c0860a0 con 0x7ffb380cd4d0 2023-12-26T22:53:31.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.487+0000 7ffb36ffd700 1 -- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffb24066f80 msgr2=0x7ffb24069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:31.488 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.487+0000 7ffb36ffd700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffb24066f80 0x7ffb24069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:31.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.491+0000 7ffb1effd700 1 -- 172.21.15.131:0/3779204890 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb2c0530a0 con 0x7ffb380cd4d0 2023-12-26T22:53:31.674 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:31.674+0000 7fe6ce68e1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T22:53:31.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.688+0000 7ffb36ffd700 1 -- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffb24066f80 msgr2=0x7ffb24069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.688+0000 7ffb36ffd700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffb24066f80 0x7ffb24069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:31.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.711+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ffb380cf5a0 con 0x7ffb380cd4d0 2023-12-26T22:53:31.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.715+0000 7ffb1effd700 1 -- 172.21.15.131:0/3779204890 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffb2c0566c0 con 0x7ffb380cd4d0 2023-12-26T22:53:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffb24066f80 msgr2=0x7ffb24069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffb24066f80 0x7ffb24069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb380cd4d0 msgr2=0x7ffb3815c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb380cd4d0 0x7ffb3815c700 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c016000 tx=0x7ffb2c017000 comp rx=0 tx=0).stop 2023-12-26T22:53:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 shutdown_connections 2023-12-26T22:53:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb38159900 0x7ffb380b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7ffb24066f80 0x7ffb24069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb380cd100 0x7ffb38169fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 --2- 172.21.15.131:0/3779204890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb380cd4d0 0x7ffb3815c700 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 >> 172.21.15.131:0/3779204890 conn(0x7ffb3806ece0 msgr2=0x7ffb381544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 shutdown_connections 2023-12-26T22:53:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.718+0000 7ffb3d3ff700 1 -- 172.21.15.131:0/3779204890 wait complete. 2023-12-26T22:53:31.726 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:31.727+0000 7fad718331c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T22:53:31.737 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T22:53:31.777 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:31.778+0000 7fe6ce68e1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T22:53:31.839 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:31.839+0000 7fad718331c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T22:53:31.869 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:31.867+0000 7fe6ce68e1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T22:53:31.931 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:31.931+0000 7fad718331c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T22:53:31.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.933+0000 7f17b57ff700 1 Processor -- start 2023-12-26T22:53:31.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.933+0000 7f17b57ff700 1 -- start start 2023-12-26T22:53:31.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.935+0000 7f17b57ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17b00b07f0 0x7f17b01554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.935+0000 7f17b57ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b00c8bf0 con 0x7f17b0159920 2023-12-26T22:53:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.935+0000 7f17b57ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b00c8e60 con 0x7f17b0159cf0 2023-12-26T22:53:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.935+0000 7f17b57ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b00c9140 con 0x7f17b00b07f0 2023-12-26T22:53:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.935+0000 7f17aeffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f17b0159920 0x7f17b00b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56348/0 (socket says 172.21.15.131:56348) 2023-12-26T22:53:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.935+0000 7f17aeffd700 1 -- 172.21.15.131:0/2909800092 learned_addr learned my addr 172.21.15.131:0/2909800092 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.935+0000 7f17ae7fc700 1 --2- 172.21.15.131:0/2909800092 >> v2:172.21.15.190:3300/0 conn(0x7f17b00b07f0 0x7f17b01554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.935+0000 7f17adffb700 1 -- 172.21.15.131:0/2909800092 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349598528 0 0) 0x7f17b00c8bf0 con 0x7f17b0159920 2023-12-26T22:53:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17adffb700 1 -- 172.21.15.131:0/2909800092 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1798004680 con 0x7f17b0159920 2023-12-26T22:53:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17adffb700 1 -- 172.21.15.131:0/2909800092 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833730770 0 0) 0x7f17b00c8e60 con 0x7f17b0159cf0 2023-12-26T22:53:31.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17adffb700 1 -- 172.21.15.131:0/2909800092 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17b00c8bf0 con 0x7f17b0159cf0 2023-12-26T22:53:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17ae7fc700 1 -- 172.21.15.131:0/2909800092 >> v1:172.21.15.190:6789/0 conn(0x7f17b0159cf0 legacy=0x7f17b0155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17ae7fc700 1 -- 172.21.15.131:0/2909800092 >> v1:172.21.15.131:6789/0 conn(0x7f17b0159920 legacy=0x7f17b00b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17ae7fc700 1 -- 172.21.15.131:0/2909800092 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00c9420 con 0x7f17b00b07f0 2023-12-26T22:53:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17ae7fc700 1 --2- 172.21.15.131:0/2909800092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b00b07f0 0x7f17b01554d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f17a40098e0 tx=0x7f17a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e4eab1a4477186 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17adffb700 1 -- 172.21.15.131:0/2909800092 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4011630 con 0x7f17b00b07f0 2023-12-26T22:53:31.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17adffb700 1 -- 172.21.15.131:0/2909800092 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a4011790 con 0x7f17b00b07f0 2023-12-26T22:53:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.936+0000 7f17adffb700 1 -- 172.21.15.131:0/2909800092 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4011b30 con 0x7f17b00b07f0 2023-12-26T22:53:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.937+0000 7f17b57ff700 1 -- 172.21.15.131:0/2909800092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b00b07f0 msgr2=0x7f17b01554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.937+0000 7f17b57ff700 1 --2- 172.21.15.131:0/2909800092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b00b07f0 0x7f17b01554d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f17a40098e0 tx=0x7f17a40091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.937+0000 7f17b57ff700 1 -- 172.21.15.131:0/2909800092 shutdown_connections 2023-12-26T22:53:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.937+0000 7f17b57ff700 1 --2- 172.21.15.131:0/2909800092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b00b07f0 0x7f17b01554d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.937+0000 7f17b57ff700 1 -- 172.21.15.131:0/2909800092 >> 172.21.15.131:0/2909800092 conn(0x7f17b0068040 msgr2=0x7f17b0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:31.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.937+0000 7f17b57ff700 1 -- 172.21.15.131:0/2909800092 shutdown_connections 2023-12-26T22:53:31.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.937+0000 7f17b57ff700 1 -- 172.21.15.131:0/2909800092 wait complete. 2023-12-26T22:53:31.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.938+0000 7f17b57ff700 1 Processor -- start 2023-12-26T22:53:31.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.939+0000 7f17b57ff700 1 -- start start 2023-12-26T22:53:31.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.939+0000 7f17b57ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b0159920 0x7f17b006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.939+0000 7f17b57ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b0159cf0 0x7f17b01569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.939+0000 7f17b57ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0156f20 0x7f17b0155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.939+0000 7f17b57ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0157710 con 0x7f17b0159cf0 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.939+0000 7f17b57ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b01579f0 con 0x7f17b0156f20 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.939+0000 7f17b57ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0157cd0 con 0x7f17b0159920 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17ae7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b0159cf0 0x7f17b01569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17aeffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b0159920 0x7f17b006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17aeffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b0159920 0x7f17b006ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:55706/0 (socket says 172.21.15.131:55706) 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17aeffd700 1 -- 172.21.15.131:0/2134501576 learned_addr learned my addr 172.21.15.131:0/2134501576 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17af7fe700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0156f20 0x7f17b0155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17ae7fc700 1 -- 172.21.15.131:0/2134501576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b0159920 msgr2=0x7f17b006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17ae7fc700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b0159920 0x7f17b006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17ae7fc700 1 -- 172.21.15.131:0/2134501576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0156f20 msgr2=0x7f17b0155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17ae7fc700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0156f20 0x7f17b0155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17ae7fc700 1 -- 172.21.15.131:0/2134501576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a4009420 con 0x7f17b0159cf0 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.940+0000 7f17af7fe700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0156f20 0x7f17b0155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.941+0000 7f17ae7fc700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b0159cf0 0x7f17b01569e0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f17a4016000 tx=0x7f17a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.941+0000 7f1796ffd700 1 -- 172.21.15.131:0/2134501576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a401f070 con 0x7f17b0159cf0 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.941+0000 7f1796ffd700 1 -- 172.21.15.131:0/2134501576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a4011900 con 0x7f17b0159cf0 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.941+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b0157fb0 con 0x7f17b0159cf0 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.941+0000 7f1796ffd700 1 -- 172.21.15.131:0/2134501576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4023410 con 0x7f17b0159cf0 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.941+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b01573a0 con 0x7f17b0159cf0 2023-12-26T22:53:31.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.942+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b00657e0 con 0x7f17b0159cf0 2023-12-26T22:53:31.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.943+0000 7f1796ffd700 1 -- 172.21.15.131:0/2134501576 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 81) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f17a4023820 con 0x7f17b0159cf0 2023-12-26T22:53:31.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.944+0000 7f1796ffd700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f179c066f80 0x7f179c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:31.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.944+0000 7f1796ffd700 1 -- 172.21.15.131:0/2134501576 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f17a40858a0 con 0x7f17b0159cf0 2023-12-26T22:53:31.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.944+0000 7f17aeffd700 1 -- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f179c066f80 msgr2=0x7f179c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:31.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.944+0000 7f17aeffd700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f179c066f80 0x7f179c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:31.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:31.948+0000 7f1796ffd700 1 -- 172.21.15.131:0/2134501576 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17a4052940 con 0x7f17b0159cf0 2023-12-26T22:53:31.948 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:31.949+0000 7fe6ce68e1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T22:53:32.017 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:32.018+0000 7fad718331c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T22:53:32.140 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:32.140+0000 7fe6ce68e1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:32.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.145+0000 7f17aeffd700 1 -- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f179c066f80 msgr2=0x7f179c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:32.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.145+0000 7f17aeffd700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f179c066f80 0x7f179c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:32.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.147+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f17b0157e10 con 0x7f17b0159cf0 2023-12-26T22:53:32.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.151+0000 7f1796ffd700 1 -- 172.21.15.131:0/2134501576 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7f17b0157e10 con 0x7f17b0159cf0 2023-12-26T22:53:32.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f179c066f80 msgr2=0x7f179c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:32.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f179c066f80 0x7f179c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b0159cf0 msgr2=0x7f17b01569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:32.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b0159cf0 0x7f17b01569e0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f17a4016000 tx=0x7f17a4017000 comp rx=0 tx=0).stop 2023-12-26T22:53:32.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 shutdown_connections 2023-12-26T22:53:32.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b0156f20 0x7f17b0155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17b0159920 0x7f17b006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f179c066f80 0x7f179c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 --2- 172.21.15.131:0/2134501576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b0159cf0 0x7f17b01569e0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 >> 172.21.15.131:0/2134501576 conn(0x7f17b0068040 msgr2=0x7f17b00cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:32.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 shutdown_connections 2023-12-26T22:53:32.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.154+0000 7f17b57ff700 1 -- 172.21.15.131:0/2134501576 wait complete. 2023-12-26T22:53:32.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:53:32.173 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-26T22:53:32.213 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:32.214+0000 7fad718331c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:32.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T22:53:32.221 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-12-26T22:53:32.308 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2023-12-26T22:53:32.312 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:32.312+0000 7fe6ce68e1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T22:53:32.393 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:32.394+0000 7fad718331c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T22:53:32.394 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:32.395+0000 7fe6ce68e1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T22:53:32.483 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:32.484+0000 7fad718331c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T22:53:32.542 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:32.543+0000 7fe6ce68e1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:32.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.549+0000 7f639e7ff700 1 Processor -- start 2023-12-26T22:53:32.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.550+0000 7f639e7ff700 1 -- start start 2023-12-26T22:53:32.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.552+0000 7f639e7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f63981597e0 0x7f63980675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:32.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.552+0000 7f639e7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63980aa1f0 0x7f639815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:32.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.552+0000 7f639e7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639815a750 con 0x7f6398067af0 2023-12-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.552+0000 7f639e7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6398159bb0 con 0x7f63981597e0 2023-12-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.552+0000 7f639e7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6398068070 con 0x7f63980aa1f0 2023-12-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f638ffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6398067af0 0x7f63980a9ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56360/0 (socket says 172.21.15.131:56360) 2023-12-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f638ffff700 1 -- 172.21.15.131:0/2164773572 learned_addr learned my addr 172.21.15.131:0/2164773572 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f6397fff700 1 --2- 172.21.15.131:0/2164773572 >> v2:172.21.15.190:3300/0 conn(0x7f63981597e0 0x7f63980675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f639cd9c700 1 --2- 172.21.15.131:0/2164773572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63980aa1f0 0x7f639815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f63977fe700 1 -- 172.21.15.131:0/2164773572 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655422581 0 0) 0x7f639815a750 con 0x7f6398067af0 2023-12-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f63977fe700 1 -- 172.21.15.131:0/2164773572 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6384003570 con 0x7f6398067af0 2023-12-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f6397fff700 1 -- 172.21.15.131:0/2164773572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63980aa1f0 msgr2=0x7f639815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f6397fff700 1 --2- 172.21.15.131:0/2164773572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63980aa1f0 0x7f639815d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f6397fff700 1 -- 172.21.15.131:0/2164773572 >> v1:172.21.15.131:6789/0 conn(0x7f6398067af0 legacy=0x7f63980a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f6397fff700 1 -- 172.21.15.131:0/2164773572 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6398159e20 con 0x7f63981597e0 2023-12-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f639cd9c700 1 --2- 172.21.15.131:0/2164773572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63980aa1f0 0x7f639815d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.553+0000 7f6397fff700 1 --2- 172.21.15.131:0/2164773572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63981597e0 0x7f63980675b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f63880098e0 tx=0x7f63880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c4ef1de749e00a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.554+0000 7f63977fe700 1 -- 172.21.15.131:0/2164773572 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388011630 con 0x7f63981597e0 2023-12-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.554+0000 7f63977fe700 1 -- 172.21.15.131:0/2164773572 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6388011790 con 0x7f63981597e0 2023-12-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.554+0000 7f63977fe700 1 -- 172.21.15.131:0/2164773572 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388011b30 con 0x7f63981597e0 2023-12-26T22:53:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.555+0000 7f639e7ff700 1 -- 172.21.15.131:0/2164773572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63981597e0 msgr2=0x7f63980675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.555+0000 7f639e7ff700 1 --2- 172.21.15.131:0/2164773572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63981597e0 0x7f63980675b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f63880098e0 tx=0x7f63880091a0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.555+0000 7f639e7ff700 1 -- 172.21.15.131:0/2164773572 shutdown_connections 2023-12-26T22:53:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.555+0000 7f639e7ff700 1 --2- 172.21.15.131:0/2164773572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63980aa1f0 0x7f639815d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.555+0000 7f639e7ff700 1 --2- 172.21.15.131:0/2164773572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63981597e0 0x7f63980675b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.555+0000 7f639e7ff700 1 -- 172.21.15.131:0/2164773572 >> 172.21.15.131:0/2164773572 conn(0x7f639806b1c0 msgr2=0x7f63980aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.555+0000 7f639e7ff700 1 -- 172.21.15.131:0/2164773572 shutdown_connections 2023-12-26T22:53:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.555+0000 7f639e7ff700 1 -- 172.21.15.131:0/2164773572 wait complete. 2023-12-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.558+0000 7f639e7ff700 1 Processor -- start 2023-12-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.558+0000 7f639e7ff700 1 -- start start 2023-12-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.559+0000 7f639e7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6398067af0 0x7f639814adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.559+0000 7f639e7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63980aa1f0 0x7f63981540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.559+0000 7f639e7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63981597e0 0x7f639806ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.559+0000 7f639e7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398169f90 con 0x7f63981597e0 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.559+0000 7f639e7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639814b330 con 0x7f63980aa1f0 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.559+0000 7f639e7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639816adb0 con 0x7f6398067af0 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f639cd9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63981597e0 0x7f639806ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f638ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63980aa1f0 0x7f63981540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f6397fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6398067af0 0x7f639814adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f638ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63980aa1f0 0x7f63981540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:52430/0 (socket says 172.21.15.131:52430) 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f638ffff700 1 -- 172.21.15.131:0/179394336 learned_addr learned my addr 172.21.15.131:0/179394336 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f638ffff700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6398067af0 msgr2=0x7f639814adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f638ffff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6398067af0 0x7f639814adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f638ffff700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63981597e0 msgr2=0x7f639806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f638ffff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63981597e0 0x7f639806ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f638ffff700 1 -- 172.21.15.131:0/179394336 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390009a20 con 0x7f63980aa1f0 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f6397fff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6398067af0 0x7f639814adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.560+0000 7f638ffff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63980aa1f0 0x7f63981540b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f638000c590 tx=0x7f6380007e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.561+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380005b30 con 0x7f63980aa1f0 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.561+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6380005cb0 con 0x7f63980aa1f0 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.561+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380007320 con 0x7f63980aa1f0 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.561+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6388009420 con 0x7f63980aa1f0 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.561+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639816a480 con 0x7f63980aa1f0 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.562+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639816af20 con 0x7f63980aa1f0 2023-12-26T22:53:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.563+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 81) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f638001b3b0 con 0x7f63980aa1f0 2023-12-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.563+0000 7f6394ff9700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f6378069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.563+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f6380083c80 con 0x7f63980aa1f0 2023-12-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.563+0000 7f6397fff700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 msgr2=0x7f6378069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.563+0000 7f6397fff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f6378069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T22:53:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.567+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6380050c80 con 0x7f63980aa1f0 2023-12-26T22:53:32.634 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:32.634+0000 7fe6ce68e1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T22:53:32.640 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:32.640+0000 7fad718331c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:32.734 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:32.734+0000 7fad718331c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T22:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.764+0000 7f6397fff700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 msgr2=0x7f6378069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.764+0000 7f6397fff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f6378069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T22:53:32.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:32.795+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f63980717f0 con 0x7f6378066f80 2023-12-26T22:53:32.815 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:32.815+0000 7fe6ce68e1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T22:53:32.918 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:32.919+0000 7fad718331c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T22:53:32.963 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:32.964+0000 7fe6ce68e1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T22:53:33.068 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:33.068+0000 7fad718331c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T22:53:33.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:33.165+0000 7f6397fff700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 msgr2=0x7f6378069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:33.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:33.165+0000 7f6397fff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f6378069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T22:53:33.341 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:33.341+0000 7fe6ce68e1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T22:53:33.449 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:33.451+0000 7fad718331c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T22:53:33.602 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:33.603+0000 7fe6ce68e1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T22:53:33.713 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:33.714+0000 7fad718331c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T22:53:33.877 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:33.878+0000 7fe6ce68e1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T22:53:33.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:33.967+0000 7f6397fff700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 msgr2=0x7f6378069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:33.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:33.967+0000 7f6397fff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f6378069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T22:53:33.966 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:33.966+0000 7fe6ce68e1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T22:53:33.991 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:33.993+0000 7fad718331c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T22:53:34.080 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:34.082+0000 7fad718331c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T22:53:34.691 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:34.691+0000 7fe6ce68e1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T22:53:34.784 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:34.784+0000 7fe6ce68e1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T22:53:34.816 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:34.818+0000 7fad718331c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T22:53:34.911 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:34.912+0000 7fad718331c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T22:53:34.933 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:34.934+0000 7fe6ce68e1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T22:53:35.064 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:35.065+0000 7fad718331c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T22:53:35.553 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:35.553+0000 7fe6ce68e1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T22:53:35.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:35.568+0000 7f6397fff700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 msgr2=0x7f6378069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/39288 2023-12-26T22:53:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:35.568+0000 7f6397fff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f6378069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T22:53:35.687 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:35.688+0000 7fad718331c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T22:53:35.738 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:35.738+0000 7fe6ce68e1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T22:53:35.877 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:35.878+0000 7fad718331c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T22:53:35.926 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:35.926+0000 7fe6ce68e1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T22:53:36.068 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:36.069+0000 7fad718331c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T22:53:36.761 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:36.762+0000 7fe6ce68e1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T22:53:36.916 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:36.918+0000 7fad718331c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T22:53:37.035 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:37.036+0000 7fe6ce68e1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:37.127 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:37.128+0000 7fe6ce68e1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:37.196 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.197+0000 7fad718331c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T22:53:37.290 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.292+0000 7fad718331c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T22:53:37.324 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T22:53:37.325+0000 7fe6ce68e1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T22:53:37.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:37.386+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mgrmap(e 82) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f6380051050 con 0x7f63980aa1f0 2023-12-26T22:53:37.492 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.493+0000 7fad718331c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T22:53:37.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:37.505+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mgrmap(e 83) v1 ==== 84470+0+0 (secure 0 0 0) 0x7f6380053e30 con 0x7f63980aa1f0 2023-12-26T22:53:37.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:37.505+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 msgr2=0x7f6378069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T22:53:37.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:37.505+0000 7f6394ff9700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f6378069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:37.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face63b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:37.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face63b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:37.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face63b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:37.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face63b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:37.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face63b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:37.627 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face33b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:37.628 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face33b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:37.628 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face33b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:37.628 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face33b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:37.628 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:53:37.628+0000 7face33b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:53:38.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.514+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mon.1 v2:172.21.15.190:3300/0 9 ==== mgrmap(e 84) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f6380051920 con 0x7f63980aa1f0 2023-12-26T22:53:38.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.514+0000 7f6394ff9700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f63780693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:38.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.514+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f63980717f0 con 0x7f6378066f80 2023-12-26T22:53:38.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.515+0000 7f6397fff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f63780693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:38.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.517+0000 7f6397fff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f63780693b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f63880091e0 tx=0x7f6388017000 comp rx=0 tx=0).ready entity=mgr.9273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.576+0000 7f6394ff9700 1 -- 172.21.15.131:0/179394336 <== mgr.9273 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f63980717f0 con 0x7f6378066f80 2023-12-26T22:53:38.574 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-12-26T22:53:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 msgr2=0x7f63780693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f63780693b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f63880091e0 tx=0x7f6388017000 comp rx=0 tx=0).stop 2023-12-26T22:53:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63980aa1f0 msgr2=0x7f63981540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63980aa1f0 0x7f63981540b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f638000c590 tx=0x7f6380007e00 comp rx=0 tx=0).stop 2023-12-26T22:53:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 shutdown_connections 2023-12-26T22:53:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6378066f80 0x7f63780693b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63981597e0 0x7f639806ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6398067af0 0x7f639814adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 --2- 172.21.15.131:0/179394336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63980aa1f0 0x7f63981540b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 >> 172.21.15.131:0/179394336 conn(0x7f639806b1c0 msgr2=0x7f639815c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 shutdown_connections 2023-12-26T22:53:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.578+0000 7f639e7ff700 1 -- 172.21.15.131:0/179394336 wait complete. 2023-12-26T22:53:38.596 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-12-26T22:53:38.597 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2023-12-26T22:53:38.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.832+0000 7f688abff700 1 Processor -- start 2023-12-26T22:53:38.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.832+0000 7f688abff700 1 -- start start 2023-12-26T22:53:38.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.833+0000 7f688abff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6884159810 0x7f68840c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:38.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.833+0000 7f688abff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68840cec80 0x7f68840cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:38.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.833+0000 7f688abff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68840cbb00 con 0x7f68840cd0a0 2023-12-26T22:53:38.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688abff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68840cb2a0 con 0x7f6884159810 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688abff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68840cf640 con 0x7f68840cec80 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688899b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6884159810 0x7f68840c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688899b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6884159810 0x7f68840c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42084/0 (socket says 172.21.15.131:42084) 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688899b700 1 -- 172.21.15.131:0/3137260576 learned_addr learned my addr 172.21.15.131:0/3137260576 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f6883fff700 1 --2- 172.21.15.131:0/3137260576 >> v2:172.21.15.190:3300/0 conn(0x7f68840cec80 0x7f68840cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688899b700 1 -- 172.21.15.131:0/3137260576 >> v2:172.21.15.190:3300/0 conn(0x7f68840cec80 msgr2=0x7f68840cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688899b700 1 --2- 172.21.15.131:0/3137260576 >> v2:172.21.15.190:3300/0 conn(0x7f68840cec80 0x7f68840cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688899b700 1 -- 172.21.15.131:0/3137260576 >> v1:172.21.15.131:6789/0 conn(0x7f68840cd0a0 legacy=0x7f68840cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688899b700 1 -- 172.21.15.131:0/3137260576 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68840ce040 con 0x7f6884159810 2023-12-26T22:53:38.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f6883fff700 1 --2- 172.21.15.131:0/3137260576 >> v2:172.21.15.190:3300/0 conn(0x7f68840cec80 0x7f68840cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.834+0000 7f688899b700 1 --2- 172.21.15.131:0/3137260576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 0x7f68840c5440 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f6884089a80 tx=0x7f6878009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3cb7fe0fb73b65 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f68837fe700 1 -- 172.21.15.131:0/3137260576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878005c10 con 0x7f6884159810 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f68837fe700 1 -- 172.21.15.131:0/3137260576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878005d70 con 0x7f6884159810 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f68837fe700 1 -- 172.21.15.131:0/3137260576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687800b880 con 0x7f6884159810 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f688abff700 1 -- 172.21.15.131:0/3137260576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 msgr2=0x7f68840c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f688abff700 1 --2- 172.21.15.131:0/3137260576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 0x7f68840c5440 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f6884089a80 tx=0x7f6878009220 comp rx=0 tx=0).stop 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f688abff700 1 -- 172.21.15.131:0/3137260576 shutdown_connections 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f688abff700 1 --2- 172.21.15.131:0/3137260576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 0x7f68840c5440 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f688abff700 1 --2- 172.21.15.131:0/3137260576 >> v2:172.21.15.190:3300/0 conn(0x7f68840cec80 0x7f68840cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:38.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f688abff700 1 -- 172.21.15.131:0/3137260576 >> 172.21.15.131:0/3137260576 conn(0x7f6884099950 msgr2=0x7f688406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:38.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f688abff700 1 -- 172.21.15.131:0/3137260576 shutdown_connections 2023-12-26T22:53:38.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.835+0000 7f688abff700 1 -- 172.21.15.131:0/3137260576 wait complete. 2023-12-26T22:53:38.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.837+0000 7f688abff700 1 Processor -- start 2023-12-26T22:53:38.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.838+0000 7f688abff700 1 -- start start 2023-12-26T22:53:38.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688abff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68840cd0a0 0x7f68840b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:38.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68840cec80 0x7f688415e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:38.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688abff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 0x7f6884169490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:38.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688abff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6884154460 con 0x7f6884159810 2023-12-26T22:53:38.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688abff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68840c7750 con 0x7f68840cd0a0 2023-12-26T22:53:38.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68840c7a30 con 0x7f68840cec80 2023-12-26T22:53:38.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 0x7f6884169490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:38.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 0x7f6884169490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42094/0 (socket says 172.21.15.131:42094) 2023-12-26T22:53:38.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688919c700 1 -- 172.21.15.131:0/8967247 learned_addr learned my addr 172.21.15.131:0/8967247 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.839+0000 7f688899b700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68840cd0a0 0x7f68840b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f6883fff700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68840cec80 0x7f688415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f688919c700 1 -- 172.21.15.131:0/8967247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68840cec80 msgr2=0x7f688415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f688919c700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68840cec80 0x7f688415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f688919c700 1 -- 172.21.15.131:0/8967247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68840cd0a0 msgr2=0x7f68840b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f688919c700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68840cd0a0 0x7f68840b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f688919c700 1 -- 172.21.15.131:0/8967247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6878009480 con 0x7f6884159810 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f688919c700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 0x7f6884169490 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f687c00c840 tx=0x7f687c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f6880ff9700 1 -- 172.21.15.131:0/8967247 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c012630 con 0x7f6884159810 2023-12-26T22:53:38.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f6880ff9700 1 -- 172.21.15.131:0/8967247 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c0127b0 con 0x7f6884159810 2023-12-26T22:53:38.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68840c7d10 con 0x7f6884159810 2023-12-26T22:53:38.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f6880ff9700 1 -- 172.21.15.131:0/8967247 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c012b70 con 0x7f6884159810 2023-12-26T22:53:38.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.840+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68840c82d0 con 0x7f6884159810 2023-12-26T22:53:38.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.841+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68840717f0 con 0x7f6884159810 2023-12-26T22:53:38.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.842+0000 7f6880ff9700 1 -- 172.21.15.131:0/8967247 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 84) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f687c003030 con 0x7f6884159810 2023-12-26T22:53:38.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.843+0000 7f6880ff9700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6868066dd0 0x7f6868069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T22:53:38.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.843+0000 7f6880ff9700 1 -- 172.21.15.131:0/8967247 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f687c0845a0 con 0x7f6884159810 2023-12-26T22:53:38.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.843+0000 7f688899b700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6868066dd0 0x7f6868069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T22:53:38.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.844+0000 7f688899b700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6868066dd0 0x7f6868069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6878009260 tx=0x7f6878016000 comp rx=0 tx=0).ready entity=mgr.9273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T22:53:38.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:38.847+0000 7f6880ff9700 1 -- 172.21.15.131:0/8967247 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f687c051720 con 0x7f6884159810 2023-12-26T22:53:39.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.062+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f68840c8460 con 0x7f6868066dd0 2023-12-26T22:53:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.114+0000 7f6880ff9700 1 -- 172.21.15.131:0/8967247 <== mgr.9273 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f68840c8460 con 0x7f6868066dd0 2023-12-26T22:53:39.113 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-12-26T22:53:39.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6868066dd0 msgr2=0x7f6868069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:39.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6868066dd0 0x7f6868069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6878009260 tx=0x7f6878016000 comp rx=0 tx=0).stop 2023-12-26T22:53:39.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 msgr2=0x7f6884169490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T22:53:39.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 0x7f6884169490 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f687c00c840 tx=0x7f687c001670 comp rx=0 tx=0).stop 2023-12-26T22:53:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 shutdown_connections 2023-12-26T22:53:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f6868066dd0 0x7f6868069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6884159810 0x7f6884169490 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68840cd0a0 0x7f68840b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 --2- 172.21.15.131:0/8967247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68840cec80 0x7f688415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T22:53:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 >> 172.21.15.131:0/8967247 conn(0x7f6884099950 msgr2=0x7f688415c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T22:53:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 shutdown_connections 2023-12-26T22:53:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T22:53:39.116+0000 7f688abff700 1 -- 172.21.15.131:0/8967247 wait complete. 2023-12-26T22:53:39.133 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-12-26T22:53:39.783 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:53:39.789 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:54:00.127 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:54:00.127+0000 7facd5395700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:54:00.127 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:54:00.127+0000 7facd5395700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:54:00.127 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:54:00.127+0000 7facd5395700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:54:00.127 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:54:00.127+0000 7facd5395700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:54:00.127 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T22:54:00.127+0000 7facd5395700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T22:54:09.906 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:54:09.913 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:54:40.029 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:54:40.035 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:55:10.151 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:55:10.156 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:55:40.273 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:55:40.279 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:56:10.397 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:56:10.403 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:56:40.578 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:56:40.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:57:10.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:57:10.736 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:57:40.856 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:57:40.871 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:58:11.145 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:58:11.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:58:41.448 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:58:41.455 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:59:11.569 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:59:11.574 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:59:41.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T22:59:41.698 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:00:11.812 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:00:11.818 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:00:41.943 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:00:41.948 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:01:12.083 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:01:12.088 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:01:42.221 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:01:42.227 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:02:12.428 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:02:12.435 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:02:42.550 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:02:42.556 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:03:12.672 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:03:12.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:03:42.794 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:03:42.800 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:04:12.912 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:04:12.918 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:04:16.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:04:16.134 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_54_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_55_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_56_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_57_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_58_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_59_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_00_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_01_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_02_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_03_00_UTC 2023-12-26T23:04:16.186 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_04_00_UTC 2023-12-26T23:04:16.187 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-22_54_00_UTC', 'scheduled-2023-12-26-22_55_00_UTC', 'scheduled-2023-12-26-22_56_00_UTC', 'scheduled-2023-12-26-22_57_00_UTC', 'scheduled-2023-12-26-22_58_00_UTC', 'scheduled-2023-12-26-22_59_00_UTC', 'scheduled-2023-12-26-23_00_00_UTC', 'scheduled-2023-12-26-23_01_00_UTC', 'scheduled-2023-12-26-23_02_00_UTC', 'scheduled-2023-12-26-23_03_00_UTC', 'scheduled-2023-12-26-23_04_00_UTC'] 2023-12-26T23:04:16.187 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-12-26T23:04:16.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.387+0000 7f36e05ff700 1 Processor -- start 2023-12-26T23:04:16.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.387+0000 7f36e05ff700 1 -- start start 2023-12-26T23:04:16.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.388+0000 7f36e05ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d8156730 0x7f36d80c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.388+0000 7f36e05ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36d806ee80 0x7f36d80c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.388+0000 7f36e05ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d80acfc0 con 0x7f36d8156360 2023-12-26T23:04:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.388+0000 7f36e05ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d80c91f0 con 0x7f36d806ee80 2023-12-26T23:04:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.388+0000 7f36e05ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d80c94d0 con 0x7f36d8156730 2023-12-26T23:04:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.388+0000 7f36deb9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36d806ee80 0x7f36d80c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36deb9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36d806ee80 0x7f36d80c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53484/0 (socket says 172.21.15.131:53484) 2023-12-26T23:04:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36deb9c700 1 -- 172.21.15.131:0/1412843186 learned_addr learned my addr 172.21.15.131:0/1412843186 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:04:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36ddb9a700 1 --2- 172.21.15.131:0/1412843186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d8156730 0x7f36d80c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36deb9c700 1 -- 172.21.15.131:0/1412843186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d8156730 msgr2=0x7f36d80c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:04:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36deb9c700 1 --2- 172.21.15.131:0/1412843186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d8156730 0x7f36d80c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36deb9c700 1 -- 172.21.15.131:0/1412843186 >> v1:172.21.15.131:6789/0 conn(0x7f36d8156360 legacy=0x7f36d806e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36deb9c700 1 -- 172.21.15.131:0/1412843186 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d80acc50 con 0x7f36d806ee80 2023-12-26T23:04:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36ddb9a700 1 --2- 172.21.15.131:0/1412843186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d8156730 0x7f36d80c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:04:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36deb9c700 1 --2- 172.21.15.131:0/1412843186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d806ee80 0x7f36d80c89d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009f20 tx=0x7f36d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4eb27fb419e759 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:04:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36dd399700 1 -- 172.21.15.131:0/1412843186 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d000e070 con 0x7f36d806ee80 2023-12-26T23:04:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36dd399700 1 -- 172.21.15.131:0/1412843186 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d000ba80 con 0x7f36d806ee80 2023-12-26T23:04:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.389+0000 7f36dd399700 1 -- 172.21.15.131:0/1412843186 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d00092e0 con 0x7f36d806ee80 2023-12-26T23:04:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.390+0000 7f36e05ff700 1 -- 172.21.15.131:0/1412843186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d806ee80 msgr2=0x7f36d80c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.390+0000 7f36e05ff700 1 --2- 172.21.15.131:0/1412843186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d806ee80 0x7f36d80c89d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009f20 tx=0x7f36d0009840 comp rx=0 tx=0).stop 2023-12-26T23:04:16.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.390+0000 7f36e05ff700 1 -- 172.21.15.131:0/1412843186 shutdown_connections 2023-12-26T23:04:16.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.390+0000 7f36e05ff700 1 --2- 172.21.15.131:0/1412843186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d806ee80 0x7f36d80c89d0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.390+0000 7f36e05ff700 1 --2- 172.21.15.131:0/1412843186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d8156730 0x7f36d80c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.390+0000 7f36e05ff700 1 -- 172.21.15.131:0/1412843186 >> 172.21.15.131:0/1412843186 conn(0x7f36d80675b0 msgr2=0x7f36d80679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:04:16.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.390+0000 7f36e05ff700 1 -- 172.21.15.131:0/1412843186 shutdown_connections 2023-12-26T23:04:16.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.390+0000 7f36e05ff700 1 -- 172.21.15.131:0/1412843186 wait complete. 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.392+0000 7f36e05ff700 1 Processor -- start 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.392+0000 7f36e05ff700 1 -- start start 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36e05ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d806ee80 0x7f36d80c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36e05ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d8156360 0x7f36d816ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36e05ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36d8156730 0x7f36d816d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36e05ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d80c4ca0 con 0x7f36d8156360 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36e05ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d80c4090 con 0x7f36d8156730 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36e05ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d80c4370 con 0x7f36d806ee80 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36ddb9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d8156360 0x7f36d816ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36ddb9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d8156360 0x7f36d816ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53494/0 (socket says 172.21.15.131:53494) 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36ddb9a700 1 -- 172.21.15.131:0/552712079 learned_addr learned my addr 172.21.15.131:0/552712079 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36deb9c700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36d8156730 0x7f36d816d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.393+0000 7f36de39b700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d806ee80 0x7f36d80c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36ddb9a700 1 -- 172.21.15.131:0/552712079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d806ee80 msgr2=0x7f36d80c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36ddb9a700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d806ee80 0x7f36d80c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36ddb9a700 1 -- 172.21.15.131:0/552712079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36d8156730 msgr2=0x7f36d816d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36ddb9a700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36d8156730 0x7f36d816d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36ddb9a700 1 -- 172.21.15.131:0/552712079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d0009ae0 con 0x7f36d8156360 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36ddb9a700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d8156360 0x7f36d816ce80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f36c800c850 tx=0x7f36c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36ce7fc700 1 -- 172.21.15.131:0/552712079 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8011600 con 0x7f36d8156360 2023-12-26T23:04:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36ce7fc700 1 -- 172.21.15.131:0/552712079 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c8011760 con 0x7f36d8156360 2023-12-26T23:04:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d80c4650 con 0x7f36d8156360 2023-12-26T23:04:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36ce7fc700 1 -- 172.21.15.131:0/552712079 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8011b00 con 0x7f36d8156360 2023-12-26T23:04:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.394+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d80c5710 con 0x7f36d8156360 2023-12-26T23:04:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.395+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d80717f0 con 0x7f36d8156360 2023-12-26T23:04:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.396+0000 7f36ce7fc700 1 -- 172.21.15.131:0/552712079 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 87) v1 ==== 84873+0+0 (secure 0 0 0) 0x7f36c8018350 con 0x7f36d8156360 2023-12-26T23:04:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.397+0000 7f36ce7fc700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36bc0670a0 0x7f36bc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.397+0000 7f36ce7fc700 1 -- 172.21.15.131:0/552712079 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f36c8084a70 con 0x7f36d8156360 2023-12-26T23:04:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.397+0000 7f36de39b700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36bc0670a0 0x7f36bc069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.398+0000 7f36de39b700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36bc0670a0 0x7f36bc069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f36d8151400 tx=0x7f36d4009040 comp rx=0 tx=0).ready entity=mgr.9273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:04:16.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.402+0000 7f36ce7fc700 1 -- 172.21.15.131:0/552712079 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36c8051710 con 0x7f36d8156360 2023-12-26T23:04:16.614 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.616+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 --> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36d80c8f10 con 0x7f36bc0670a0 2023-12-26T23:04:16.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.625+0000 7f36ce7fc700 1 -- 172.21.15.131:0/552712079 <== mgr.9273 v2:172.21.15.131:6836/39288 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f36d80c8f10 con 0x7f36bc0670a0 2023-12-26T23:04:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:04:16.624 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T22:53:38", "first": "2023-12-26T22:54:00", "last": "2023-12-26T23:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-12-26T23:04:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36bc0670a0 msgr2=0x7f36bc069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.626 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36bc0670a0 0x7f36bc069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f36d8151400 tx=0x7f36d4009040 comp rx=0 tx=0).stop 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d8156360 msgr2=0x7f36d816ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d8156360 0x7f36d816ce80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f36c800c850 tx=0x7f36c8008040 comp rx=0 tx=0).stop 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 shutdown_connections 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36d8156730 0x7f36d816d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36d806ee80 0x7f36d80c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f36bc0670a0 0x7f36bc069550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 --2- 172.21.15.131:0/552712079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36d8156360 0x7f36d816ce80 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 >> 172.21.15.131:0/552712079 conn(0x7f36d80675b0 msgr2=0x7f36d806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 shutdown_connections 2023-12-26T23:04:16.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.627+0000 7f36e05ff700 1 -- 172.21.15.131:0/552712079 wait complete. 2023-12-26T23:04:16.644 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T22:53:38", "first": "2023-12-26T22:54:00", "last": "2023-12-26T23:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-12-26T23:04:16.644 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T22:53:38', 'first': '2023-12-26T22:54:00', 'last': '2023-12-26T23:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-12-26T23:04:16.644 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-26T23:04:16.645 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-26T23:04:16.645 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-26T23:04:16.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.835+0000 7f3b7b5ff700 1 Processor -- start 2023-12-26T23:04:16.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.835+0000 7f3b7b5ff700 1 -- start start 2023-12-26T23:04:16.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b7b5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b740b0bf0 0x7f3b740c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b7b5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b74156940 con 0x7f3b740b0750 2023-12-26T23:04:16.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b7b5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b740ae500 con 0x7f3b740cb320 2023-12-26T23:04:16.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b7b5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b740c8cc0 con 0x7f3b740b0bf0 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b79b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b740b0bf0 0x7f3b740c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b79b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b740b0bf0 0x7f3b740c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53504/0 (socket says 172.21.15.131:53504) 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b79b9c700 1 -- 172.21.15.131:0/161292958 learned_addr learned my addr 172.21.15.131:0/161292958 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b79b9c700 1 -- 172.21.15.131:0/161292958 >> v1:172.21.15.190:6789/0 conn(0x7f3b740cb320 legacy=0x7f3b74069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b79b9c700 1 -- 172.21.15.131:0/161292958 >> v1:172.21.15.131:6789/0 conn(0x7f3b740b0750 legacy=0x7f3b740c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.837+0000 7f3b79b9c700 1 -- 172.21.15.131:0/161292958 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b740b0280 con 0x7f3b740b0bf0 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b79b9c700 1 --2- 172.21.15.131:0/161292958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 0x7f3b740c9b10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009f20 tx=0x7f3b6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7508d092d74943c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b6bfff700 1 -- 172.21.15.131:0/161292958 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00e070 con 0x7f3b740b0bf0 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b6bfff700 1 -- 172.21.15.131:0/161292958 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c00ba80 con 0x7f3b740b0bf0 2023-12-26T23:04:16.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b6bfff700 1 -- 172.21.15.131:0/161292958 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0092e0 con 0x7f3b740b0bf0 2023-12-26T23:04:16.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/161292958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 msgr2=0x7f3b740c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b7b5ff700 1 --2- 172.21.15.131:0/161292958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 0x7f3b740c9b10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009f20 tx=0x7f3b6c009840 comp rx=0 tx=0).stop 2023-12-26T23:04:16.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/161292958 shutdown_connections 2023-12-26T23:04:16.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b7b5ff700 1 --2- 172.21.15.131:0/161292958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 0x7f3b740c9b10 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/161292958 >> 172.21.15.131:0/161292958 conn(0x7f3b74068040 msgr2=0x7f3b74069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:04:16.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.838+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/161292958 shutdown_connections 2023-12-26T23:04:16.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.839+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/161292958 wait complete. 2023-12-26T23:04:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.840+0000 7f3b7b5ff700 1 Processor -- start 2023-12-26T23:04:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.840+0000 7f3b7b5ff700 1 -- start start 2023-12-26T23:04:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b7b5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b740b0750 0x7f3b7414d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b7b5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 0x7f3b74158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b7b5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b740cb320 0x7f3b741585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b7b5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7413a3c0 con 0x7f3b740b0bf0 2023-12-26T23:04:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b7b5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7413ab50 con 0x7f3b740b0750 2023-12-26T23:04:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b7b5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7413ae30 con 0x7f3b740cb320 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b78b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 0x7f3b74158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b78b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 0x7f3b74158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53510/0 (socket says 172.21.15.131:53510) 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b78b9a700 1 -- 172.21.15.131:0/1448500858 learned_addr learned my addr 172.21.15.131:0/1448500858 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b78b9a700 1 -- 172.21.15.131:0/1448500858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b740cb320 msgr2=0x7f3b741585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.841+0000 7f3b7939b700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b740b0750 0x7f3b7414d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b78b9a700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b740cb320 0x7f3b741585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b78b9a700 1 -- 172.21.15.131:0/1448500858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b740b0750 msgr2=0x7f3b7414d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b78b9a700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b740b0750 0x7f3b7414d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b78b9a700 1 -- 172.21.15.131:0/1448500858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c009ae0 con 0x7f3b740b0bf0 2023-12-26T23:04:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b7939b700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b740b0750 0x7f3b7414d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:04:16.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b78b9a700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 0x7f3b74158080 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c590 tx=0x7f3b6400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:04:16.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b697fa700 1 -- 172.21.15.131:0/1448500858 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64011590 con 0x7f3b740b0bf0 2023-12-26T23:04:16.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b697fa700 1 -- 172.21.15.131:0/1448500858 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b640116f0 con 0x7f3b740b0bf0 2023-12-26T23:04:16.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b697fa700 1 -- 172.21.15.131:0/1448500858 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64011ab0 con 0x7f3b740b0bf0 2023-12-26T23:04:16.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7413b110 con 0x7f3b740b0bf0 2023-12-26T23:04:16.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.842+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7413b6d0 con 0x7f3b740b0bf0 2023-12-26T23:04:16.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.843+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b74070df0 con 0x7f3b740b0bf0 2023-12-26T23:04:16.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.844+0000 7f3b697fa700 1 -- 172.21.15.131:0/1448500858 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 87) v1 ==== 84873+0+0 (secure 0 0 0) 0x7f3b6401b3b0 con 0x7f3b740b0bf0 2023-12-26T23:04:16.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.845+0000 7f3b697fa700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b580670d0 0x7f3b58069580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:04:16.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.845+0000 7f3b697fa700 1 -- 172.21.15.131:0/1448500858 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f3b64084070 con 0x7f3b740b0bf0 2023-12-26T23:04:16.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.845+0000 7f3b7939b700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b580670d0 0x7f3b58069580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:04:16.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.846+0000 7f3b7939b700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b580670d0 0x7f3b58069580 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b74155dd0 tx=0x7f3b70009040 comp rx=0 tx=0).ready entity=mgr.9273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:04:16.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:16.849+0000 7f3b697fa700 1 -- 172.21.15.131:0/1448500858 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b64050e20 con 0x7f3b740b0bf0 2023-12-26T23:04:17.095 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.097+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3b740657e0 con 0x7f3b740b0bf0 2023-12-26T23:04:17.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.100+0000 7f3b697fa700 1 -- 172.21.15.131:0/1448500858 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v87) v1 ==== 74+0+163914 (secure 0 0 0) 0x7f3b64017020 con 0x7f3b740b0bf0 2023-12-26T23:04:17.102 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:04:17.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.103+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b580670d0 msgr2=0x7f3b58069580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:17.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.103+0000 7f3b7b5ff700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b580670d0 0x7f3b58069580 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3b74155dd0 tx=0x7f3b70009040 comp rx=0 tx=0).stop 2023-12-26T23:04:17.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.103+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 msgr2=0x7f3b74158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:04:17.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.103+0000 7f3b7b5ff700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 0x7f3b74158080 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c590 tx=0x7f3b6400c950 comp rx=0 tx=0).stop 2023-12-26T23:04:17.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.104+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 shutdown_connections 2023-12-26T23:04:17.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.104+0000 7f3b7b5ff700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b740cb320 0x7f3b741585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:17.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.104+0000 7f3b7b5ff700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b740b0750 0x7f3b7414d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:17.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.104+0000 7f3b7b5ff700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:6836/39288,v1:172.21.15.131:6837/39288] conn(0x7f3b580670d0 0x7f3b58069580 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:17.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.104+0000 7f3b7b5ff700 1 --2- 172.21.15.131:0/1448500858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b740b0bf0 0x7f3b74158080 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:04:17.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.104+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 >> 172.21.15.131:0/1448500858 conn(0x7f3b74068040 msgr2=0x7f3b74156f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:04:17.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.104+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 shutdown_connections 2023-12-26T23:04:17.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:04:17.104+0000 7f3b7b5ff700 1 -- 172.21.15.131:0/1448500858 wait complete. 2023-12-26T23:04:17.120 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":87,"active_gid":9273,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":39288},{"type":"v1","addr":"172.21.15.131:6837","nonce":39288}]},"active_addr":"172.21.15.131:6837/39288","active_change":"2023-12-26T22:53:37.495619+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":7757,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":176,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":3168215713}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1073482886}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":659813654}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":109159886}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":470407650}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1083345344}]}]} 2023-12-26T23:04:17.163 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: y 2023-12-26T23:04:17.163 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-26T23:04:17.164 INFO:tasks.ceph.mgr.y:Stopping old one... 2023-12-26T23:04:17.164 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-26T23:04:17.164 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:04:17.234 INFO:tasks.ceph.mgr.y:Stopped 2023-12-26T23:04:17.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-26T23:04:17.239 INFO:tasks.ceph.mgr.y:Started 2023-12-26T23:04:17.726 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:17.728+0000 7f1d84e891c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:04:17.831 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:17.832+0000 7f1d84e891c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:04:17.922 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:17.924+0000 7f1d84e891c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:04:18.004 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:18.006+0000 7f1d84e891c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:04:18.194 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:18.196+0000 7f1d84e891c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:04:18.370 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:18.372+0000 7f1d84e891c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:04:18.455 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:18.456+0000 7f1d84e891c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:04:18.602 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:18.604+0000 7f1d84e891c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:04:18.695 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:18.697+0000 7f1d84e891c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:04:18.879 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:18.880+0000 7f1d84e891c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:04:19.029 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:19.030+0000 7f1d84e891c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:04:19.414 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:19.415+0000 7f1d84e891c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:04:19.682 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:19.684+0000 7f1d84e891c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:04:19.963 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:19.965+0000 7f1d84e891c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:04:20.052 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:20.054+0000 7f1d84e891c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:04:20.788 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:20.790+0000 7f1d84e891c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:04:20.883 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:20.884+0000 7f1d84e891c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:04:21.036 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:21.038+0000 7f1d84e891c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:04:21.659 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:21.660+0000 7f1d84e891c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:04:21.850 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:21.851+0000 7f1d84e891c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:04:22.040 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:22.042+0000 7f1d84e891c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:04:22.891 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:22.892+0000 7f1d84e891c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:04:23.168 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.170+0000 7f1d84e891c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:04:23.263 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.264+0000 7f1d84e891c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:04:23.462 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.464+0000 7f1d84e891c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:04:23.629 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf5fb0700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:23.629 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf5fb0700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:23.629 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf5fb0700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:23.629 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf5fb0700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:23.629 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf5fb0700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:23.629 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf8fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:23.629 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf8fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:23.630 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf8fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:23.630 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf8fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:23.630 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:04:23.630+0000 7f1cf8fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:04:43.289 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:04:43.294 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:05:00.626 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:05:00.626+0000 7f1d00fc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:05:00.626 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:05:00.626+0000 7f1d00fc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:05:00.626 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:05:00.626+0000 7f1d00fc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:05:00.626 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:05:00.626+0000 7f1d00fc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:05:00.626 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:05:00.626+0000 7f1d00fc6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:05:13.399 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:05:13.405 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:05:43.658 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:05:43.665 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:06:13.772 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:06:13.779 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:06:44.233 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:06:44.240 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:07:14.346 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:07:14.352 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:07:44.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:07:44.478 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:08:14.611 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:08:14.617 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:08:44.734 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:08:44.743 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:09:14.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:09:14.867 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:09:17.241 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2023-12-26T23:09:17.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.448+0000 7f46addff700 1 Processor -- start 2023-12-26T23:09:17.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.448+0000 7f46addff700 1 -- start start 2023-12-26T23:09:17.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.449+0000 7f46addff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f46a8156360 0x7f46a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:17.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.449+0000 7f46addff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a80acfc0 con 0x7f46a80c8ba0 2023-12-26T23:09:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.449+0000 7f46addff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a80c91f0 con 0x7f46a8156730 2023-12-26T23:09:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.449+0000 7f46addff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a80c9000 con 0x7f46a8156360 2023-12-26T23:09:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.450+0000 7f46a77fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f46a8156360 0x7f46a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.450+0000 7f46a7fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f46a80c8ba0 0x7f46a80ad450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45972/0 (socket says 172.21.15.131:45972) 2023-12-26T23:09:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.450+0000 7f46a7fff700 1 -- 172.21.15.131:0/2018870296 learned_addr learned my addr 172.21.15.131:0/2018870296 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.450+0000 7f46a77fe700 1 -- 172.21.15.131:0/2018870296 >> v1:172.21.15.190:6789/0 conn(0x7f46a8156730 legacy=0x7f46a80c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.450+0000 7f46a77fe700 1 -- 172.21.15.131:0/2018870296 >> v1:172.21.15.131:6789/0 conn(0x7f46a80c8ba0 legacy=0x7f46a80ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.450+0000 7f46a77fe700 1 -- 172.21.15.131:0/2018870296 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a80c94d0 con 0x7f46a8156360 2023-12-26T23:09:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.450+0000 7f46a67fc700 1 -- 172.21.15.131:0/2018870296 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809771784 0 0) 0x7f46a80acfc0 con 0x7f46a80c8ba0 2023-12-26T23:09:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.450+0000 7f46a77fe700 1 --2- 172.21.15.131:0/2018870296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156360 0x7f46a806e770 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f46980098e0 tx=0x7f46980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1c2db8418e1213 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46a67fc700 1 -- 172.21.15.131:0/2018870296 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698009e80 con 0x7f46a8156360 2023-12-26T23:09:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46a67fc700 1 -- 172.21.15.131:0/2018870296 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469800b860 con 0x7f46a8156360 2023-12-26T23:09:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46a67fc700 1 -- 172.21.15.131:0/2018870296 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800bbe0 con 0x7f46a8156360 2023-12-26T23:09:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46addff700 1 -- 172.21.15.131:0/2018870296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156360 msgr2=0x7f46a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46addff700 1 --2- 172.21.15.131:0/2018870296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156360 0x7f46a806e770 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f46980098e0 tx=0x7f46980091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46addff700 1 -- 172.21.15.131:0/2018870296 shutdown_connections 2023-12-26T23:09:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46addff700 1 --2- 172.21.15.131:0/2018870296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156360 0x7f46a806e770 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46addff700 1 -- 172.21.15.131:0/2018870296 >> 172.21.15.131:0/2018870296 conn(0x7f46a80675b0 msgr2=0x7f46a80679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:17.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46addff700 1 -- 172.21.15.131:0/2018870296 shutdown_connections 2023-12-26T23:09:17.454 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.451+0000 7f46addff700 1 -- 172.21.15.131:0/2018870296 wait complete. 2023-12-26T23:09:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.453+0000 7f46addff700 1 Processor -- start 2023-12-26T23:09:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.453+0000 7f46addff700 1 -- start start 2023-12-26T23:09:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46addff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a80c8ba0 0x7f46a80c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46addff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a8156360 0x7f46a8153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46addff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156730 0x7f46a80b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46addff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a8158620 con 0x7f46a8156730 2023-12-26T23:09:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46addff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a8154060 con 0x7f46a8156360 2023-12-26T23:09:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46addff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80b0010 con 0x7f46a80c8ba0 2023-12-26T23:09:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156730 0x7f46a80b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156730 0x7f46a80b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47482/0 (socket says 172.21.15.131:47482) 2023-12-26T23:09:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a7fff700 1 -- 172.21.15.131:0/2795808514 learned_addr learned my addr 172.21.15.131:0/2795808514 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a6ffd700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a8156360 0x7f46a8153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a77fe700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a80c8ba0 0x7f46a80c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a7fff700 1 -- 172.21.15.131:0/2795808514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a80c8ba0 msgr2=0x7f46a80c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a7fff700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a80c8ba0 0x7f46a80c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a7fff700 1 -- 172.21.15.131:0/2795808514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a8156360 msgr2=0x7f46a8153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a7fff700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a8156360 0x7f46a8153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.454+0000 7f46a7fff700 1 -- 172.21.15.131:0/2795808514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4698009420 con 0x7f46a8156730 2023-12-26T23:09:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.455+0000 7f46a7fff700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156730 0x7f46a80b13d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f46a000cc90 tx=0x7f46a0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.455+0000 7f468f7fe700 1 -- 172.21.15.131:0/2795808514 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a000cea0 con 0x7f46a8156730 2023-12-26T23:09:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.455+0000 7f468f7fe700 1 -- 172.21.15.131:0/2795808514 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a000ab10 con 0x7f46a8156730 2023-12-26T23:09:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.455+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a80b02f0 con 0x7f46a8156730 2023-12-26T23:09:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.455+0000 7f468f7fe700 1 -- 172.21.15.131:0/2795808514 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a00070f0 con 0x7f46a8156730 2023-12-26T23:09:17.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.455+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a8157f60 con 0x7f46a8156730 2023-12-26T23:09:17.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.456+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a80717f0 con 0x7f46a8156730 2023-12-26T23:09:17.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.457+0000 7f468f7fe700 1 -- 172.21.15.131:0/2795808514 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 92) v1 ==== 84873+0+0 (secure 0 0 0) 0x7f46a00074a0 con 0x7f46a8156730 2023-12-26T23:09:17.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.458+0000 7f468f7fe700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f46940670a0 0x7f4694069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:17.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.458+0000 7f468f7fe700 1 -- 172.21.15.131:0/2795808514 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f46a0084ff0 con 0x7f46a8156730 2023-12-26T23:09:17.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.458+0000 7f46a77fe700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f46940670a0 0x7f4694069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:17.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.458+0000 7f46a77fe700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f46940670a0 0x7f4694069550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f46980030c0 tx=0x7f4698016000 comp rx=0 tx=0).ready entity=mgr.9366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:17.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.462+0000 7f468f7fe700 1 -- 172.21.15.131:0/2795808514 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46a0051d40 con 0x7f46a8156730 2023-12-26T23:09:17.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.683+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f46a8158760 con 0x7f46940670a0 2023-12-26T23:09:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.754+0000 7f468f7fe700 1 -- 172.21.15.131:0/2795808514 <== mgr.9366 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f46a8158760 con 0x7f46940670a0 2023-12-26T23:09:17.754 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-12-26T23:09:17.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f46940670a0 msgr2=0x7f4694069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:17.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f46940670a0 0x7f4694069550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f46980030c0 tx=0x7f4698016000 comp rx=0 tx=0).stop 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156730 msgr2=0x7f46a80b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156730 0x7f46a80b13d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f46a000cc90 tx=0x7f46a0001a80 comp rx=0 tx=0).stop 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 shutdown_connections 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f46940670a0 0x7f4694069550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46a8156730 0x7f46a80b13d0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a80c8ba0 0x7f46a80c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 --2- 172.21.15.131:0/2795808514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a8156360 0x7f46a8153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 >> 172.21.15.131:0/2795808514 conn(0x7f46a80675b0 msgr2=0x7f46a8067980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.756+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 shutdown_connections 2023-12-26T23:09:17.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:17.757+0000 7f46addff700 1 -- 172.21.15.131:0/2795808514 wait complete. 2023-12-26T23:09:17.775 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-12-26T23:09:17.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:17.776 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-26T23:09:17.821 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_54_00_UTC 2023-12-26T23:09:17.821 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_55_00_UTC 2023-12-26T23:09:17.821 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_56_00_UTC 2023-12-26T23:09:17.821 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_57_00_UTC 2023-12-26T23:09:17.821 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_58_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_59_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_00_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_01_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_02_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_03_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_04_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_05_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_06_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_07_00_UTC 2023-12-26T23:09:17.822 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_08_00_UTC 2023-12-26T23:09:17.823 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_09_00_UTC 2023-12-26T23:09:17.824 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-26-22_54_00_UTC', 'scheduled-2023-12-26-22_55_00_UTC', 'scheduled-2023-12-26-22_56_00_UTC', 'scheduled-2023-12-26-22_57_00_UTC', 'scheduled-2023-12-26-22_58_00_UTC', 'scheduled-2023-12-26-22_59_00_UTC', 'scheduled-2023-12-26-23_00_00_UTC', 'scheduled-2023-12-26-23_01_00_UTC', 'scheduled-2023-12-26-23_02_00_UTC', 'scheduled-2023-12-26-23_03_00_UTC', 'scheduled-2023-12-26-23_04_00_UTC', 'scheduled-2023-12-26-23_05_00_UTC', 'scheduled-2023-12-26-23_06_00_UTC', 'scheduled-2023-12-26-23_07_00_UTC', 'scheduled-2023-12-26-23_08_00_UTC', 'scheduled-2023-12-26-23_09_00_UTC'] 2023-12-26T23:09:17.824 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-12-26T23:09:18.056 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.056+0000 7f63ba9ff700 1 Processor -- start 2023-12-26T23:09:18.057 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.057+0000 7f63ba9ff700 1 -- start start 2023-12-26T23:09:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63ba9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41597e0 0x7f63b40675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63ba9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63b4067af0 0x7f63b40a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63ba9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b415a750 con 0x7f63b40aa020 2023-12-26T23:09:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63ba9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b4159bb0 con 0x7f63b4067af0 2023-12-26T23:09:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63ba9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40aa440 con 0x7f63b41597e0 2023-12-26T23:09:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63b37fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63b4067af0 0x7f63b40a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63b37fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63b4067af0 0x7f63b40a9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47490/0 (socket says 172.21.15.131:47490) 2023-12-26T23:09:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63b37fe700 1 -- 172.21.15.131:0/3461205948 learned_addr learned my addr 172.21.15.131:0/3461205948 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63b8f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f63b40aa020 0x7f63b406e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45980/0 (socket says 172.21.15.131:45980) 2023-12-26T23:09:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63b3fff700 1 --2- 172.21.15.131:0/3461205948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41597e0 0x7f63b40675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63b37fe700 1 -- 172.21.15.131:0/3461205948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41597e0 msgr2=0x7f63b40675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.058+0000 7f63b37fe700 1 --2- 172.21.15.131:0/3461205948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41597e0 0x7f63b40675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.059+0000 7f63b37fe700 1 -- 172.21.15.131:0/3461205948 >> v1:172.21.15.131:6789/0 conn(0x7f63b40aa020 legacy=0x7f63b406e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.059+0000 7f63b37fe700 1 -- 172.21.15.131:0/3461205948 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4068070 con 0x7f63b4067af0 2023-12-26T23:09:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.059+0000 7f63b2ffd700 1 -- 172.21.15.131:0/3461205948 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206536439 0 0) 0x7f63b415a750 con 0x7f63b40aa020 2023-12-26T23:09:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.059+0000 7f63b37fe700 1 --2- 172.21.15.131:0/3461205948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4067af0 0x7f63b40a9ae0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f63a80099b0 tx=0x7f63a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b4a6432b4eaf1c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.059+0000 7f63b2ffd700 1 -- 172.21.15.131:0/3461205948 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a800b860 con 0x7f63b4067af0 2023-12-26T23:09:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.059+0000 7f63b2ffd700 1 -- 172.21.15.131:0/3461205948 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a800b9c0 con 0x7f63b4067af0 2023-12-26T23:09:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.059+0000 7f63b2ffd700 1 -- 172.21.15.131:0/3461205948 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a800bd60 con 0x7f63b4067af0 2023-12-26T23:09:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.059+0000 7f63ba9ff700 1 -- 172.21.15.131:0/3461205948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4067af0 msgr2=0x7f63b40a9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.059+0000 7f63ba9ff700 1 --2- 172.21.15.131:0/3461205948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4067af0 0x7f63b40a9ae0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f63a80099b0 tx=0x7f63a8009250 comp rx=0 tx=0).stop 2023-12-26T23:09:18.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.060+0000 7f63ba9ff700 1 -- 172.21.15.131:0/3461205948 shutdown_connections 2023-12-26T23:09:18.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.060+0000 7f63ba9ff700 1 --2- 172.21.15.131:0/3461205948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41597e0 0x7f63b40675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.060+0000 7f63ba9ff700 1 --2- 172.21.15.131:0/3461205948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4067af0 0x7f63b40a9ae0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.060+0000 7f63ba9ff700 1 -- 172.21.15.131:0/3461205948 >> 172.21.15.131:0/3461205948 conn(0x7f63b406b1c0 msgr2=0x7f63b40aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:18.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.060+0000 7f63ba9ff700 1 -- 172.21.15.131:0/3461205948 shutdown_connections 2023-12-26T23:09:18.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.060+0000 7f63ba9ff700 1 -- 172.21.15.131:0/3461205948 wait complete. 2023-12-26T23:09:18.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.061+0000 7f63ba9ff700 1 Processor -- start 2023-12-26T23:09:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.061+0000 7f63ba9ff700 1 -- start start 2023-12-26T23:09:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.062+0000 7f63ba9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4067af0 0x7f63b4142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.062+0000 7f63ba9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40aa020 0x7f63b40b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.062+0000 7f63ba9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b41597e0 0x7f63b4156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.062+0000 7f63ba9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4143330 con 0x7f63b41597e0 2023-12-26T23:09:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.062+0000 7f63ba9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b416d300 con 0x7f63b40aa020 2023-12-26T23:09:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.062+0000 7f63ba9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b416de80 con 0x7f63b4067af0 2023-12-26T23:09:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b8f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b41597e0 0x7f63b4156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4067af0 0x7f63b4142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4067af0 0x7f63b4142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:52994/0 (socket says 172.21.15.131:52994) 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b3fff700 1 -- 172.21.15.131:0/2170450518 learned_addr learned my addr 172.21.15.131:0/2170450518 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b37fe700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40aa020 0x7f63b40b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b8f9c700 1 -- 172.21.15.131:0/2170450518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4067af0 msgr2=0x7f63b4142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b8f9c700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4067af0 0x7f63b4142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b8f9c700 1 -- 172.21.15.131:0/2170450518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40aa020 msgr2=0x7f63b40b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b8f9c700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40aa020 0x7f63b40b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b8f9c700 1 -- 172.21.15.131:0/2170450518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80094f0 con 0x7f63b41597e0 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b3fff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4067af0 0x7f63b4142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f63b8f9c700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b41597e0 0x7f63b4156b70 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00cc90 tx=0x7f63ac001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f639bfff700 1 -- 172.21.15.131:0/2170450518 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac005df0 con 0x7f63b41597e0 2023-12-26T23:09:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.063+0000 7f639bfff700 1 -- 172.21.15.131:0/2170450518 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac00a6a0 con 0x7f63b41597e0 2023-12-26T23:09:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.064+0000 7f639bfff700 1 -- 172.21.15.131:0/2170450518 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00aa60 con 0x7f63b41597e0 2023-12-26T23:09:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.064+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b40b1550 con 0x7f63b41597e0 2023-12-26T23:09:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.064+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b416d830 con 0x7f63b41597e0 2023-12-26T23:09:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.065+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b40717f0 con 0x7f63b41597e0 2023-12-26T23:09:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.066+0000 7f639bfff700 1 -- 172.21.15.131:0/2170450518 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 92) v1 ==== 84873+0+0 (secure 0 0 0) 0x7f63ac0070f0 con 0x7f63b41597e0 2023-12-26T23:09:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.066+0000 7f639bfff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f63a00670a0 0x7f63a0069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.066+0000 7f639bfff700 1 -- 172.21.15.131:0/2170450518 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f63ac084ae0 con 0x7f63b41597e0 2023-12-26T23:09:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.067+0000 7f63b3fff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f63a00670a0 0x7f63a0069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.067+0000 7f63b3fff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f63a00670a0 0x7f63a0069550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f63a4006fd0 tx=0x7f63a4008040 comp rx=0 tx=0).ready entity=mgr.9366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:18.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.070+0000 7f639bfff700 1 -- 172.21.15.131:0/2170450518 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63ac051960 con 0x7f63b41597e0 2023-12-26T23:09:18.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.286+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63b40678d0 con 0x7f63a00670a0 2023-12-26T23:09:18.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.294+0000 7f639bfff700 1 -- 172.21.15.131:0/2170450518 <== mgr.9366 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f63b40678d0 con 0x7f63a00670a0 2023-12-26T23:09:18.295 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:18.295 INFO:teuthology.orchestra.run.smithi131.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T22:53:38", "first": "2023-12-26T22:54:00", "last": "2023-12-26T23:09:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-12-26T23:09:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f63a00670a0 msgr2=0x7f63a0069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f63a00670a0 0x7f63a0069550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f63a4006fd0 tx=0x7f63a4008040 comp rx=0 tx=0).stop 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b41597e0 msgr2=0x7f63b4156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b41597e0 0x7f63b4156b70 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00cc90 tx=0x7f63ac001a60 comp rx=0 tx=0).stop 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 shutdown_connections 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f63a00670a0 0x7f63a0069550 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b41597e0 0x7f63b4156b70 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4067af0 0x7f63b4142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 --2- 172.21.15.131:0/2170450518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40aa020 0x7f63b40b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 >> 172.21.15.131:0/2170450518 conn(0x7f63b406b1c0 msgr2=0x7f63b41543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 shutdown_connections 2023-12-26T23:09:18.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.297+0000 7f63ba9ff700 1 -- 172.21.15.131:0/2170450518 wait complete. 2023-12-26T23:09:18.315 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-26T00:00:00", "created": "2023-12-26T22:53:38", "first": "2023-12-26T22:54:00", "last": "2023-12-26T23:09:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-12-26T23:09:18.316 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-12-26T00:00:00', 'created': '2023-12-26T22:53:38', 'first': '2023-12-26T22:54:00', 'last': '2023-12-26T23:09:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-12-26T23:09:18.316 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-12-26T23:09:18.316 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-12-26T23:09:18.316 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2023-12-26T23:09:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.513+0000 7f8c155ff700 1 Processor -- start 2023-12-26T23:09:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.513+0000 7f8c155ff700 1 -- start start 2023-12-26T23:09:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c155ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c10142f00 0x7f8c10069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c155ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8c100cc590 0x7f8c100cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c155ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c100cb340 0x7f8c100c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c155ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c100cb7c0 con 0x7f8c100cc590 2023-12-26T23:09:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c155ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c10069640 con 0x7f8c100cb340 2023-12-26T23:09:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c155ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10068750 con 0x7f8c10142f00 2023-12-26T23:09:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0e7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8c100cc590 0x7f8c100cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0e7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8c100cc590 0x7f8c100cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47510/0 (socket says 172.21.15.131:47510) 2023-12-26T23:09:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0e7fc700 1 -- 172.21.15.131:0/3288590400 learned_addr learned my addr 172.21.15.131:0/3288590400 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0effd700 1 --2- 172.21.15.131:0/3288590400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c10142f00 0x7f8c10069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0f7fe700 1 --2- 172.21.15.131:0/3288590400 >> v2:172.21.15.190:3300/0 conn(0x7f8c100cb340 0x7f8c100c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0e7fc700 1 -- 172.21.15.131:0/3288590400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c10142f00 msgr2=0x7f8c10069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0e7fc700 1 --2- 172.21.15.131:0/3288590400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c10142f00 0x7f8c10069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0e7fc700 1 -- 172.21.15.131:0/3288590400 >> v2:172.21.15.190:3300/0 conn(0x7f8c100cb340 msgr2=0x7f8c100c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0e7fc700 1 --2- 172.21.15.131:0/3288590400 >> v2:172.21.15.190:3300/0 conn(0x7f8c100cb340 0x7f8c100c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0e7fc700 1 -- 172.21.15.131:0/3288590400 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c10068a30 con 0x7f8c100cc590 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0f7fe700 1 --2- 172.21.15.131:0/3288590400 >> v2:172.21.15.190:3300/0 conn(0x7f8c100cb340 0x7f8c100c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.515+0000 7f8c0effd700 1 --2- 172.21.15.131:0/3288590400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c10142f00 0x7f8c10069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c0e7fc700 1 --2- 172.21.15.131:0/3288590400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c100cc590 0x7f8c100cae00 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f8c1006f560 tx=0x7f8c04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28dbc1f84403a21 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c0dffb700 1 -- 172.21.15.131:0/3288590400 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400b860 con 0x7f8c100cc590 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c0dffb700 1 -- 172.21.15.131:0/3288590400 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0400b9c0 con 0x7f8c100cc590 2023-12-26T23:09:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c0dffb700 1 -- 172.21.15.131:0/3288590400 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400bd60 con 0x7f8c100cc590 2023-12-26T23:09:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c155ff700 1 -- 172.21.15.131:0/3288590400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c100cc590 msgr2=0x7f8c100cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c155ff700 1 --2- 172.21.15.131:0/3288590400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c100cc590 0x7f8c100cae00 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f8c1006f560 tx=0x7f8c04009250 comp rx=0 tx=0).stop 2023-12-26T23:09:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c155ff700 1 -- 172.21.15.131:0/3288590400 shutdown_connections 2023-12-26T23:09:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c155ff700 1 --2- 172.21.15.131:0/3288590400 >> v2:172.21.15.190:3300/0 conn(0x7f8c100cb340 0x7f8c100c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c155ff700 1 --2- 172.21.15.131:0/3288590400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c10142f00 0x7f8c10069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c155ff700 1 --2- 172.21.15.131:0/3288590400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c100cc590 0x7f8c100cae00 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c155ff700 1 -- 172.21.15.131:0/3288590400 >> 172.21.15.131:0/3288590400 conn(0x7f8c10063cd0 msgr2=0x7f8c1006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.516+0000 7f8c155ff700 1 -- 172.21.15.131:0/3288590400 shutdown_connections 2023-12-26T23:09:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.517+0000 7f8c155ff700 1 -- 172.21.15.131:0/3288590400 wait complete. 2023-12-26T23:09:18.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.518+0000 7f8c155ff700 1 Processor -- start 2023-12-26T23:09:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.518+0000 7f8c155ff700 1 -- start start 2023-12-26T23:09:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c155ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb340 0x7f8c100a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c155ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c100cc590 0x7f8c100b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c155ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c10142f00 0x7f8c100b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c155ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100aaf80 con 0x7f8c10142f00 2023-12-26T23:09:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c155ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100aa370 con 0x7f8c100cc590 2023-12-26T23:09:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c155ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100aa650 con 0x7f8c100cb340 2023-12-26T23:09:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c10142f00 0x7f8c100b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c10142f00 0x7f8c100b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47526/0 (socket says 172.21.15.131:47526) 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c0f7fe700 1 -- 172.21.15.131:0/2680666379 learned_addr learned my addr 172.21.15.131:0/2680666379 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c0effd700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb340 0x7f8c100a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.519+0000 7f8c0e7fc700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c100cc590 0x7f8c100b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8c0f7fe700 1 -- 172.21.15.131:0/2680666379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb340 msgr2=0x7f8c100a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8c0f7fe700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb340 0x7f8c100a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8c0f7fe700 1 -- 172.21.15.131:0/2680666379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c100cc590 msgr2=0x7f8c100b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8c0f7fe700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c100cc590 0x7f8c100b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8c0f7fe700 1 -- 172.21.15.131:0/2680666379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040094f0 con 0x7f8c10142f00 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8c0f7fe700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c10142f00 0x7f8c100b09c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800d180 tx=0x7f8c08009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8bf6ffd700 1 -- 172.21.15.131:0/2680666379 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800de20 con 0x7f8c10142f00 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8bf6ffd700 1 -- 172.21.15.131:0/2680666379 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0800f040 con 0x7f8c10142f00 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c100aa930 con 0x7f8c10142f00 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8bf6ffd700 1 -- 172.21.15.131:0/2680666379 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08015690 con 0x7f8c10142f00 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.520+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c10169b60 con 0x7f8c10142f00 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.521+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c100717f0 con 0x7f8c10142f00 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.522+0000 7f8bf6ffd700 1 -- 172.21.15.131:0/2680666379 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 92) v1 ==== 84873+0+0 (secure 0 0 0) 0x7f8c0801e3b0 con 0x7f8c10142f00 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.523+0000 7f8bf6ffd700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8bfc0670a0 0x7f8bfc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.523+0000 7f8bf6ffd700 1 -- 172.21.15.131:0/2680666379 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f8c08085320 con 0x7f8c10142f00 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.523+0000 7f8c0effd700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8bfc0670a0 0x7f8bfc069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.524+0000 7f8c0effd700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8bfc0670a0 0x7f8bfc069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c00006fd0 tx=0x7f8c00008040 comp rx=0 tx=0).ready entity=mgr.9366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:18.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.527+0000 7f8bf6ffd700 1 -- 172.21.15.131:0/2680666379 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c08051ec0 con 0x7f8c10142f00 2023-12-26T23:09:18.745 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.745+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c100ab0c0 con 0x7f8bfc0670a0 2023-12-26T23:09:18.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.820+0000 7f8bf6ffd700 1 -- 172.21.15.131:0/2680666379 <== mgr.9366 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f8c100ab0c0 con 0x7f8bfc0670a0 2023-12-26T23:09:18.821 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-12-26T23:09:18.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8bfc0670a0 msgr2=0x7f8bfc069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8bfc0670a0 0x7f8bfc069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c00006fd0 tx=0x7f8c00008040 comp rx=0 tx=0).stop 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c10142f00 msgr2=0x7f8c100b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c10142f00 0x7f8c100b09c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800d180 tx=0x7f8c08009a50 comp rx=0 tx=0).stop 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 shutdown_connections 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8bfc0670a0 0x7f8bfc069550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c10142f00 0x7f8c100b09c0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c100cb340 0x7f8c100a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 --2- 172.21.15.131:0/2680666379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c100cc590 0x7f8c100b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 >> 172.21.15.131:0/2680666379 conn(0x7f8c10063cd0 msgr2=0x7f8c10141c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 shutdown_connections 2023-12-26T23:09:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:18.823+0000 7f8c155ff700 1 -- 172.21.15.131:0/2680666379 wait complete. 2023-12-26T23:09:18.841 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-12-26T23:09:18.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:18.841 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-26T23:09:18.887 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_54_00_UTC 2023-12-26T23:09:18.887 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_55_00_UTC 2023-12-26T23:09:18.887 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_56_00_UTC 2023-12-26T23:09:18.887 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_57_00_UTC 2023-12-26T23:09:18.887 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_58_00_UTC 2023-12-26T23:09:18.887 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-22_59_00_UTC 2023-12-26T23:09:18.887 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_00_00_UTC 2023-12-26T23:09:18.887 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_01_00_UTC 2023-12-26T23:09:18.887 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_02_00_UTC 2023-12-26T23:09:18.888 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_03_00_UTC 2023-12-26T23:09:18.888 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_04_00_UTC 2023-12-26T23:09:18.888 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_05_00_UTC 2023-12-26T23:09:18.888 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_06_00_UTC 2023-12-26T23:09:18.888 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_07_00_UTC 2023-12-26T23:09:18.888 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_08_00_UTC 2023-12-26T23:09:18.888 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_09_00_UTC 2023-12-26T23:09:18.889 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_54_00_UTC 2023-12-26T23:09:18.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:18.890 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_54_00_UTC) 2023-12-26T23:09:18.979 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_55_00_UTC 2023-12-26T23:09:18.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:18.979 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_55_00_UTC) 2023-12-26T23:09:19.028 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_56_00_UTC 2023-12-26T23:09:19.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.029 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_56_00_UTC) 2023-12-26T23:09:19.078 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_57_00_UTC 2023-12-26T23:09:19.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.078 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_57_00_UTC) 2023-12-26T23:09:19.127 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_58_00_UTC 2023-12-26T23:09:19.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.127 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_58_00_UTC) 2023-12-26T23:09:19.175 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_59_00_UTC 2023-12-26T23:09:19.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.176 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-22_59_00_UTC) 2023-12-26T23:09:19.225 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_00_00_UTC 2023-12-26T23:09:19.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.225 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_00_00_UTC) 2023-12-26T23:09:19.274 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_01_00_UTC 2023-12-26T23:09:19.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.274 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_01_00_UTC) 2023-12-26T23:09:19.324 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_02_00_UTC 2023-12-26T23:09:19.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.324 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_02_00_UTC) 2023-12-26T23:09:19.373 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_03_00_UTC 2023-12-26T23:09:19.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.373 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_03_00_UTC) 2023-12-26T23:09:19.422 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_04_00_UTC 2023-12-26T23:09:19.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.422 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_04_00_UTC) 2023-12-26T23:09:19.472 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_05_00_UTC 2023-12-26T23:09:19.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.472 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_05_00_UTC) 2023-12-26T23:09:19.521 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_06_00_UTC 2023-12-26T23:09:19.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.532 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_06_00_UTC) 2023-12-26T23:09:19.582 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_07_00_UTC 2023-12-26T23:09:19.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.582 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_07_00_UTC) 2023-12-26T23:09:19.632 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_08_00_UTC 2023-12-26T23:09:19.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.632 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_08_00_UTC) 2023-12-26T23:09:19.682 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_09_00_UTC 2023-12-26T23:09:19.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.682 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-26-23_09_00_UTC) 2023-12-26T23:09:19.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:19.732 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-12-26T23:09:19.779 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T23:09:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.010+0000 7fdd541ff700 1 Processor -- start 2023-12-26T23:09:20.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.011+0000 7fdd541ff700 1 -- start start 2023-12-26T23:09:20.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.012+0000 7fdd541ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c06bfd0 0x7fdd4c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:20.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.012+0000 7fdd541ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd4c0cc100 0x7fdd4c0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd541ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4c0686c0 con 0x7fdd4c13c510 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd541ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c069520 con 0x7fdd4c0cc100 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd541ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c068910 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd51f9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdd4c13c510 0x7fdd4c06b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45992/0 (socket says 172.21.15.131:45992) 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd51f9b700 1 -- 172.21.15.131:0/813656896 learned_addr learned my addr 172.21.15.131:0/813656896 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd5279c700 1 --2- 172.21.15.131:0/813656896 >> v2:172.21.15.190:3300/0 conn(0x7fdd4c0cc100 0x7fdd4c0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd5179a700 1 --2- 172.21.15.131:0/813656896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c06bfd0 0x7fdd4c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd50f99700 1 -- 172.21.15.131:0/813656896 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253036221 0 0) 0x7fdd4c0686c0 con 0x7fdd4c13c510 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd50f99700 1 -- 172.21.15.131:0/813656896 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd38003570 con 0x7fdd4c13c510 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd5279c700 1 -- 172.21.15.131:0/813656896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c06bfd0 msgr2=0x7fdd4c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:20.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd5279c700 1 --2- 172.21.15.131:0/813656896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c06bfd0 0x7fdd4c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd5279c700 1 -- 172.21.15.131:0/813656896 >> v1:172.21.15.131:6789/0 conn(0x7fdd4c13c510 legacy=0x7fdd4c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd5279c700 1 -- 172.21.15.131:0/813656896 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c068bf0 con 0x7fdd4c0cc100 2023-12-26T23:09:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.013+0000 7fdd5179a700 1 --2- 172.21.15.131:0/813656896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c06bfd0 0x7fdd4c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.014+0000 7fdd5279c700 1 --2- 172.21.15.131:0/813656896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0cc100 0x7fdd4c0cc580 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009e60 tx=0x7fdd44009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d866bca120c3bf server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.014+0000 7fdd50f99700 1 -- 172.21.15.131:0/813656896 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400e070 con 0x7fdd4c0cc100 2023-12-26T23:09:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.014+0000 7fdd50f99700 1 -- 172.21.15.131:0/813656896 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4400bab0 con 0x7fdd4c0cc100 2023-12-26T23:09:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.014+0000 7fdd50f99700 1 -- 172.21.15.131:0/813656896 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400be30 con 0x7fdd4c0cc100 2023-12-26T23:09:20.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.015+0000 7fdd541ff700 1 -- 172.21.15.131:0/813656896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0cc100 msgr2=0x7fdd4c0cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.015+0000 7fdd541ff700 1 --2- 172.21.15.131:0/813656896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0cc100 0x7fdd4c0cc580 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009e60 tx=0x7fdd44009780 comp rx=0 tx=0).stop 2023-12-26T23:09:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.015+0000 7fdd541ff700 1 -- 172.21.15.131:0/813656896 shutdown_connections 2023-12-26T23:09:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.015+0000 7fdd541ff700 1 --2- 172.21.15.131:0/813656896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c0cc100 0x7fdd4c0cc580 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.015+0000 7fdd541ff700 1 --2- 172.21.15.131:0/813656896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c06bfd0 0x7fdd4c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.015+0000 7fdd541ff700 1 -- 172.21.15.131:0/813656896 >> 172.21.15.131:0/813656896 conn(0x7fdd4c0ce3e0 msgr2=0x7fdd4c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.015+0000 7fdd541ff700 1 -- 172.21.15.131:0/813656896 shutdown_connections 2023-12-26T23:09:20.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.015+0000 7fdd541ff700 1 -- 172.21.15.131:0/813656896 wait complete. 2023-12-26T23:09:20.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.017+0000 7fdd541ff700 1 Processor -- start 2023-12-26T23:09:20.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.017+0000 7fdd541ff700 1 -- start start 2023-12-26T23:09:20.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.018+0000 7fdd541ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c06bfd0 0x7fdd4c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:20.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.018+0000 7fdd541ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c0cc100 0x7fdd4c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:20.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd541ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c13c510 0x7fdd4c0b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:20.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd541ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c156d80 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd541ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c1563b0 con 0x7fdd4c13c510 2023-12-26T23:09:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd541ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c0b3f90 con 0x7fdd4c0cc100 2023-12-26T23:09:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd51f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c06bfd0 0x7fdd4c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd51f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c06bfd0 0x7fdd4c0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47538/0 (socket says 172.21.15.131:47538) 2023-12-26T23:09:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd51f9b700 1 -- 172.21.15.131:0/1126696008 learned_addr learned my addr 172.21.15.131:0/1126696008 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd5279c700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c13c510 0x7fdd4c0b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd5179a700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c0cc100 0x7fdd4c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd51f9b700 1 -- 172.21.15.131:0/1126696008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c0cc100 msgr2=0x7fdd4c156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd51f9b700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c0cc100 0x7fdd4c156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd51f9b700 1 -- 172.21.15.131:0/1126696008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c13c510 msgr2=0x7fdd4c0b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd51f9b700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c13c510 0x7fdd4c0b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.019+0000 7fdd51f9b700 1 -- 172.21.15.131:0/1126696008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009420 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.020+0000 7fdd51f9b700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c06bfd0 0x7fdd4c0b1b40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c790 tx=0x7fdd48007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.020+0000 7fdd41ffb700 1 -- 172.21.15.131:0/1126696008 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480072e0 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.020+0000 7fdd41ffb700 1 -- 172.21.15.131:0/1126696008 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd48007460 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.020+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd44009a20 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.020+0000 7fdd41ffb700 1 -- 172.21.15.131:0/1126696008 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480057f0 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.020+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c155a80 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.021+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c0657e0 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.022+0000 7fdd41ffb700 1 -- 172.21.15.131:0/1126696008 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 92) v1 ==== 84873+0+0 (secure 0 0 0) 0x7fdd48005bc0 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.023+0000 7fdd41ffb700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd300670a0 0x7fdd30069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.023+0000 7fdd41ffb700 1 -- 172.21.15.131:0/1126696008 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fdd48083ff0 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.023+0000 7fdd5179a700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd300670a0 0x7fdd30069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:20.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.023+0000 7fdd5179a700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd300670a0 0x7fdd30069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0c8580 tx=0x7fdd3c0091a0 comp rx=0 tx=0).ready entity=mgr.9366 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:20.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.026+0000 7fdd41ffb700 1 -- 172.21.15.131:0/1126696008 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd48050d00 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.272+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fdd4c068ed0 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.465+0000 7fdd41ffb700 1 -- 172.21.15.131:0/1126696008 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fdd480510d0 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.466+0000 7fdd41ffb700 1 -- 172.21.15.131:0/1126696008 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v93) v1 ==== 93+0+0 (secure 0 0 0) 0x7fdd48050d00 con 0x7fdd4c06bfd0 2023-12-26T23:09:20.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd300670a0 msgr2=0x7fdd30069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:20.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd300670a0 0x7fdd30069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0c8580 tx=0x7fdd3c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c06bfd0 msgr2=0x7fdd4c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c06bfd0 0x7fdd4c0b1b40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c790 tx=0x7fdd48007c60 comp rx=0 tx=0).stop 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 shutdown_connections 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd300670a0 0x7fdd30069550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd4c13c510 0x7fdd4c0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd4c06bfd0 0x7fdd4c0b1b40 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 --2- 172.21.15.131:0/1126696008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd4c0cc100 0x7fdd4c156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 >> 172.21.15.131:0/1126696008 conn(0x7fdd4c0ce3e0 msgr2=0x7fdd4c0cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 shutdown_connections 2023-12-26T23:09:20.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:20.469+0000 7fdd541ff700 1 -- 172.21.15.131:0/1126696008 wait complete. 2023-12-26T23:09:20.488 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T23:09:20.488 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:09:20.581 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:09:20.667 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T23:09:20.667 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T23:09:20.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:20.667 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T23:09:20.745 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T23:09:20.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:09:20.745 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T23:09:20.789 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:09:20.906 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:20.906+0000 7f64e85631c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:09:20.922 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:20.922+0000 7f2d11f141c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:09:21.010 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:21.011+0000 7f64e85631c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:09:21.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.022+0000 7f7e5f9ff700 1 Processor -- start 2023-12-26T23:09:21.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.022+0000 7f7e5f9ff700 1 -- start start 2023-12-26T23:09:21.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.023+0000 7f7e5f9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e580b2360 0x7f7e580caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5f9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e580aecc0 0x7f7e5815b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5f9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e58067c30 con 0x7f7e5815b260 2023-12-26T23:09:21.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5f9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e58067650 con 0x7f7e580b2360 2023-12-26T23:09:21.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5f9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5815bcd0 con 0x7f7e580aecc0 2023-12-26T23:09:21.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5cf9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e580b2360 0x7f7e580caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5d79b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7e5815b260 0x7f7e580b1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46002/0 (socket says 172.21.15.131:46002) 2023-12-26T23:09:21.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5d79b700 1 -- 172.21.15.131:0/2943930650 learned_addr learned my addr 172.21.15.131:0/2943930650 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:21.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5df9c700 1 --2- 172.21.15.131:0/2943930650 >> v2:172.21.15.190:3300/0 conn(0x7f7e580aecc0 0x7f7e5815b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5cf9a700 1 -- 172.21.15.131:0/2943930650 >> v2:172.21.15.190:3300/0 conn(0x7f7e580aecc0 msgr2=0x7f7e5815b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:21.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5cf9a700 1 --2- 172.21.15.131:0/2943930650 >> v2:172.21.15.190:3300/0 conn(0x7f7e580aecc0 0x7f7e5815b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5cf9a700 1 -- 172.21.15.131:0/2943930650 >> v1:172.21.15.131:6789/0 conn(0x7f7e5815b260 legacy=0x7f7e580b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e5cf9a700 1 -- 172.21.15.131:0/2943930650 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e580637b0 con 0x7f7e580b2360 2023-12-26T23:09:21.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.024+0000 7f7e4ffff700 1 -- 172.21.15.131:0/2943930650 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591347465 0 0) 0x7f7e58067c30 con 0x7f7e5815b260 2023-12-26T23:09:21.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.025+0000 7f7e5cf9a700 1 --2- 172.21.15.131:0/2943930650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e580b2360 0x7f7e580caae0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f7e5806fb30 tx=0x7f7e480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf19de3ffa37cfb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:21.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.025+0000 7f7e4ffff700 1 -- 172.21.15.131:0/2943930650 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4800b690 con 0x7f7e580b2360 2023-12-26T23:09:21.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.025+0000 7f7e4ffff700 1 -- 172.21.15.131:0/2943930650 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4800b7f0 con 0x7f7e580b2360 2023-12-26T23:09:21.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.025+0000 7f7e4ffff700 1 -- 172.21.15.131:0/2943930650 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4800bb90 con 0x7f7e580b2360 2023-12-26T23:09:21.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.026+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/2943930650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e580b2360 msgr2=0x7f7e580caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.026+0000 7f7e5f9ff700 1 --2- 172.21.15.131:0/2943930650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e580b2360 0x7f7e580caae0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f7e5806fb30 tx=0x7f7e480091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.026+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/2943930650 shutdown_connections 2023-12-26T23:09:21.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.026+0000 7f7e5f9ff700 1 --2- 172.21.15.131:0/2943930650 >> v2:172.21.15.190:3300/0 conn(0x7f7e580aecc0 0x7f7e5815b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.026+0000 7f7e5f9ff700 1 --2- 172.21.15.131:0/2943930650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e580b2360 0x7f7e580caae0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.026+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/2943930650 >> 172.21.15.131:0/2943930650 conn(0x7f7e5806c290 msgr2=0x7f7e580ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:21.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.026+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/2943930650 shutdown_connections 2023-12-26T23:09:21.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.026+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/2943930650 wait complete. 2023-12-26T23:09:21.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.028+0000 7f7e5f9ff700 1 Processor -- start 2023-12-26T23:09:21.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.029+0000 7f7e5f9ff700 1 -- start start 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5f9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e580aecc0 0x7f7e580b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5f9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e580b2360 0x7f7e581515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5f9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e5815b260 0x7f7e580cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5f9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580b1ae0 con 0x7f7e5815b260 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5f9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e58151b20 con 0x7f7e580b2360 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5f9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580c5cd0 con 0x7f7e580aecc0 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5df9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e5815b260 0x7f7e580cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5df9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e5815b260 0x7f7e580cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47576/0 (socket says 172.21.15.131:47576) 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5df9c700 1 -- 172.21.15.131:0/1673859701 learned_addr learned my addr 172.21.15.131:0/1673859701 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5d79b700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e580aecc0 0x7f7e580b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.030+0000 7f7e5cf9a700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e580b2360 0x7f7e581515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e5df9c700 1 -- 172.21.15.131:0/1673859701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e580aecc0 msgr2=0x7f7e580b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e5df9c700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e580aecc0 0x7f7e580b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e5df9c700 1 -- 172.21.15.131:0/1673859701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e580b2360 msgr2=0x7f7e581515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e5df9c700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e580b2360 0x7f7e581515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e5df9c700 1 -- 172.21.15.131:0/1673859701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48009420 con 0x7f7e5815b260 2023-12-26T23:09:21.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e5df9c700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e5815b260 0x7f7e580cbab0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009f20 tx=0x7f7e5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e4d7fa700 1 -- 172.21.15.131:0/1673859701 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000cb70 con 0x7f7e5815b260 2023-12-26T23:09:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e4d7fa700 1 -- 172.21.15.131:0/1673859701 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5000f040 con 0x7f7e5815b260 2023-12-26T23:09:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e4d7fa700 1 -- 172.21.15.131:0/1673859701 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50014480 con 0x7f7e5815b260 2023-12-26T23:09:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.031+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e580c83d0 con 0x7f7e5815b260 2023-12-26T23:09:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.032+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e580c5f20 con 0x7f7e5815b260 2023-12-26T23:09:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.033+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e580c8560 con 0x7f7e5815b260 2023-12-26T23:09:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.034+0000 7f7e4d7fa700 1 -- 172.21.15.131:0/1673859701 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f7e50014830 con 0x7f7e5815b260 2023-12-26T23:09:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.035+0000 7f7e4d7fa700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7e3c067050 0x7f7e3c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.035 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:21.035+0000 7f2d11f141c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:09:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.035+0000 7f7e4d7fa700 1 -- 172.21.15.131:0/1673859701 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f7e50085880 con 0x7f7e5815b260 2023-12-26T23:09:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.035+0000 7f7e5d79b700 1 -- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7e3c067050 msgr2=0x7f7e3c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.035+0000 7f7e5d79b700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7e3c067050 0x7f7e3c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:21.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.039+0000 7f7e4d7fa700 1 -- 172.21.15.131:0/1673859701 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e50052420 con 0x7f7e5815b260 2023-12-26T23:09:21.100 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:21.100+0000 7f64e85631c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:09:21.127 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:21.127+0000 7f2d11f141c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:09:21.182 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:21.182+0000 7f64e85631c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:09:21.211 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:21.212+0000 7f2d11f141c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:09:21.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.236+0000 7f7e5d79b700 1 -- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7e3c067050 msgr2=0x7f7e3c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:21.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.236+0000 7f7e5d79b700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7e3c067050 0x7f7e3c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:21.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.288+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e580657e0 con 0x7f7e5815b260 2023-12-26T23:09:21.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.288+0000 7f7e4d7fa700 1 -- 172.21.15.131:0/1673859701 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f7e50055a40 con 0x7f7e5815b260 2023-12-26T23:09:21.288 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:21.288 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2545294570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2545294570},{"type":"v1","addr":"172.21.15.190:6835","nonce":2545294570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1800615106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1800615106},{"type":"v1","addr":"172.21.15.131:6834","nonce":1800615106}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3279484094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3279484094},{"type":"v1","addr":"172.21.15.131:6835","nonce":3279484094}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:53:21.655334+0000","modified":"2023-12-26T22:53:25.218600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9105":{"gid":9105,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/716462328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":716462328},{"type":"v1","addr":"172.21.15.190:6834","nonce":716462328}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-12-26T23:09:21.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.290+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7e3c067050 msgr2=0x7f7e3c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:21.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.290+0000 7f7e5f9ff700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7e3c067050 0x7f7e3c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e5815b260 msgr2=0x7f7e580cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e5815b260 0x7f7e580cbab0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009f20 tx=0x7f7e5000b040 comp rx=0 tx=0).stop 2023-12-26T23:09:21.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 shutdown_connections 2023-12-26T23:09:21.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7e3c067050 0x7f7e3c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e5815b260 0x7f7e580cbab0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e580aecc0 0x7f7e580b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 --2- 172.21.15.131:0/1673859701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e580b2360 0x7f7e581515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 >> 172.21.15.131:0/1673859701 conn(0x7f7e5806c290 msgr2=0x7f7e58158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:21.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 shutdown_connections 2023-12-26T23:09:21.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.291+0000 7f7e5f9ff700 1 -- 172.21.15.131:0/1673859701 wait complete. 2023-12-26T23:09:21.292 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 81 2023-12-26T23:09:21.310 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:09:21.372 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:21.372+0000 7f64e85631c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:09:21.410 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:21.411+0000 7f2d11f141c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:09:21.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.502+0000 7f21d71ff700 1 Processor -- start 2023-12-26T23:09:21.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.502+0000 7f21d71ff700 1 -- start start 2023-12-26T23:09:21.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.503+0000 7f21d71ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f21d00cb320 0x7f21d0069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.503+0000 7f21d71ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 0x7f21d00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.503+0000 7f21d71ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d0156940 con 0x7f21d00b0c90 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.503+0000 7f21d71ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d00c8cc0 con 0x7f21d00cb320 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.503+0000 7f21d71ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00ad700 con 0x7f21d006a1e0 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.503+0000 7f21d4f9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f21d00cb320 0x7f21d0069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.503+0000 7f21d579c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f21d00b0c90 0x7f21d00aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46014/0 (socket says 172.21.15.131:46014) 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.503+0000 7f21d579c700 1 -- 172.21.15.131:0/4280830664 learned_addr learned my addr 172.21.15.131:0/4280830664 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.504+0000 7f21cffff700 1 --2- 172.21.15.131:0/4280830664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 0x7f21d00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.504+0000 7f21d4f9b700 1 -- 172.21.15.131:0/4280830664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 msgr2=0x7f21d00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.504+0000 7f21d4f9b700 1 --2- 172.21.15.131:0/4280830664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 0x7f21d00b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.504+0000 7f21d4f9b700 1 -- 172.21.15.131:0/4280830664 >> v1:172.21.15.131:6789/0 conn(0x7f21d00b0c90 legacy=0x7f21d00aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.504+0000 7f21d4f9b700 1 -- 172.21.15.131:0/4280830664 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d00ae500 con 0x7f21d00cb320 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.504+0000 7f21cf7fe700 1 -- 172.21.15.131:0/4280830664 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508334825 0 0) 0x7f21d0156940 con 0x7f21d00b0c90 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.504+0000 7f21d4f9b700 1 --2- 172.21.15.131:0/4280830664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 0x7f21d0069ca0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f21c40098e0 tx=0x7f21c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804219a3e63c62dd server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.504+0000 7f21cf7fe700 1 -- 172.21.15.131:0/4280830664 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400b640 con 0x7f21d00cb320 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21cf7fe700 1 -- 172.21.15.131:0/4280830664 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c400b7a0 con 0x7f21d00cb320 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21cf7fe700 1 -- 172.21.15.131:0/4280830664 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400bb40 con 0x7f21d00cb320 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21d71ff700 1 -- 172.21.15.131:0/4280830664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 msgr2=0x7f21d0069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21d71ff700 1 --2- 172.21.15.131:0/4280830664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 0x7f21d0069ca0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f21c40098e0 tx=0x7f21c40091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21d71ff700 1 -- 172.21.15.131:0/4280830664 shutdown_connections 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21d71ff700 1 --2- 172.21.15.131:0/4280830664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 0x7f21d0069ca0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21d71ff700 1 --2- 172.21.15.131:0/4280830664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 0x7f21d00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21d71ff700 1 -- 172.21.15.131:0/4280830664 >> 172.21.15.131:0/4280830664 conn(0x7f21d0068040 msgr2=0x7f21d0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21d71ff700 1 -- 172.21.15.131:0/4280830664 shutdown_connections 2023-12-26T23:09:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.505+0000 7f21d71ff700 1 -- 172.21.15.131:0/4280830664 wait complete. 2023-12-26T23:09:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.507+0000 7f21d71ff700 1 Processor -- start 2023-12-26T23:09:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.507+0000 7f21d71ff700 1 -- start start 2023-12-26T23:09:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d71ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 0x7f21d0157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d71ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d00b0c90 0x7f21d00c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d71ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 0x7f21d01544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d71ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d01568b0 con 0x7f21d00cb320 2023-12-26T23:09:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d71ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00c7e70 con 0x7f21d00b0c90 2023-12-26T23:09:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d71ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0156b50 con 0x7f21d006a1e0 2023-12-26T23:09:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d579c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 0x7f21d01544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d579c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 0x7f21d01544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47600/0 (socket says 172.21.15.131:47600) 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d579c700 1 -- 172.21.15.131:0/3865509652 learned_addr learned my addr 172.21.15.131:0/3865509652 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d4f9b700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 0x7f21d0157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21cffff700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d00b0c90 0x7f21d00c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d579c700 1 -- 172.21.15.131:0/3865509652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 msgr2=0x7f21d0157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d579c700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 0x7f21d0157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d579c700 1 -- 172.21.15.131:0/3865509652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d00b0c90 msgr2=0x7f21d00c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d579c700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d00b0c90 0x7f21d00c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.508+0000 7f21d579c700 1 -- 172.21.15.131:0/3865509652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c4009420 con 0x7f21d00cb320 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.509+0000 7f21d579c700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 0x7f21d01544e0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f21c800cc90 tx=0x7f21c8001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:21.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.509+0000 7f21ccff9700 1 -- 172.21.15.131:0/3865509652 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800a8b0 con 0x7f21d00cb320 2023-12-26T23:09:21.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.509+0000 7f21ccff9700 1 -- 172.21.15.131:0/3865509652 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c800aa10 con 0x7f21d00cb320 2023-12-26T23:09:21.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.509+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d0156e30 con 0x7f21d00cb320 2023-12-26T23:09:21.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.509+0000 7f21ccff9700 1 -- 172.21.15.131:0/3865509652 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800add0 con 0x7f21d00cb320 2023-12-26T23:09:21.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.509+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d0157890 con 0x7f21d00cb320 2023-12-26T23:09:21.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.510+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d00657e0 con 0x7f21d00cb320 2023-12-26T23:09:21.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.512+0000 7f21ccff9700 1 -- 172.21.15.131:0/3865509652 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f21c8007300 con 0x7f21d00cb320 2023-12-26T23:09:21.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.513+0000 7f21ccff9700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f21b4067050 0x7f21b4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.513+0000 7f21ccff9700 1 -- 172.21.15.131:0/3865509652 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f21c8085870 con 0x7f21d00cb320 2023-12-26T23:09:21.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.513+0000 7f21d4f9b700 1 -- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f21b4067050 msgr2=0x7f21b4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:21.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.513+0000 7f21d4f9b700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f21b4067050 0x7f21b4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.515+0000 7f21ccff9700 1 -- 172.21.15.131:0/3865509652 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21c8052720 con 0x7f21d00cb320 2023-12-26T23:09:21.542 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:21.543+0000 7f64e85631c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:09:21.604 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:21.604+0000 7f2d11f141c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:09:21.626 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:21.626+0000 7f64e85631c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:09:21.688 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:21.688+0000 7f2d11f141c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:09:21.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.714+0000 7f21d4f9b700 1 -- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f21b4067050 msgr2=0x7f21b4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:21.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.714+0000 7f21d4f9b700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f21b4067050 0x7f21b4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:21.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.730+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21d0157020 con 0x7f21d00cb320 2023-12-26T23:09:21.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.731+0000 7f21ccff9700 1 -- 172.21.15.131:0/3865509652 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+16968 (secure 0 0 0) 0x7f21c8055d40 con 0x7f21d00cb320 2023-12-26T23:09:21.732 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:21.733 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":179,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:04:23.466177+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:53:19.805015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-26T22:53:20.636358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.5","mappings":[{"from":2,"to":5}]},{"pgid":"21.3f","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/109159886":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/1073482886":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/659813654":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/3572226702":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/3168215713":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/2626934136":"2023-12-27T22:53:37.495457+0000","172.21.15.131:6836/39288":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/892276341":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/470407650":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/1083345344":"2023-12-27T23:04:23.466126+0000","172.21.15.131:6837/39288":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/1605877564":"2023-12-27T22:53:37.495457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:09:21.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f21b4067050 msgr2=0x7f21b4069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f21b4067050 0x7f21b4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 msgr2=0x7f21d01544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 0x7f21d01544e0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f21c800cc90 tx=0x7f21c8001a80 comp rx=0 tx=0).stop 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 shutdown_connections 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f21b4067050 0x7f21b4069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d00cb320 0x7f21d01544e0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21d006a1e0 0x7f21d0157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 --2- 172.21.15.131:0/3865509652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21d00b0c90 0x7f21d00c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 >> 172.21.15.131:0/3865509652 conn(0x7f21d0068040 msgr2=0x7f21d00c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 shutdown_connections 2023-12-26T23:09:21.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.734+0000 7f21d71ff700 1 -- 172.21.15.131:0/3865509652 wait complete. 2023-12-26T23:09:21.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T22:53:19.805015+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T22:53:20.636358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T23:09:21.756 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T23:09:21.774 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:21.774+0000 7f64e85631c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:09:21.843 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:21.844+0000 7f2d11f141c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:09:21.863 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:21.864+0000 7f64e85631c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:09:21.938 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:21.938+0000 7f2d11f141c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:09:21.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.951+0000 7fa1f1bff700 1 Processor -- start 2023-12-26T23:09:21.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.951+0000 7fa1f1bff700 1 -- start start 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.952+0000 7fa1f1bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1ec1595e0 0x7fa1ec1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.952+0000 7fa1f1bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa1ec0ad560 0x7fa1ec159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1f1bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec0a98b0 con 0x7fa1ec15d240 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1f1bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ec0add40 con 0x7fa1ec0ad560 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1f1bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ec15a500 con 0x7fa1ec1595e0 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1ebfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa1ec0ad560 0x7fa1ec159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1eb7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1ec1595e0 0x7fa1ec1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1eaffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa1ec15d240 0x7fa1ec0ace50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46022/0 (socket says 172.21.15.131:46022) 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1eaffd700 1 -- 172.21.15.131:0/3116834198 learned_addr learned my addr 172.21.15.131:0/3116834198 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1eb7fe700 1 -- 172.21.15.131:0/3116834198 >> v2:172.21.15.131:3300/0 conn(0x7fa1ec0ad560 msgr2=0x7fa1ec159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1eb7fe700 1 --2- 172.21.15.131:0/3116834198 >> v2:172.21.15.131:3300/0 conn(0x7fa1ec0ad560 0x7fa1ec159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1eb7fe700 1 -- 172.21.15.131:0/3116834198 >> v1:172.21.15.131:6789/0 conn(0x7fa1ec15d240 legacy=0x7fa1ec0ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1eb7fe700 1 -- 172.21.15.131:0/3116834198 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec15a7e0 con 0x7fa1ec1595e0 2023-12-26T23:09:21.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1ebfff700 1 --2- 172.21.15.131:0/3116834198 >> v2:172.21.15.131:3300/0 conn(0x7fa1ec0ad560 0x7fa1ec159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:21.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.953+0000 7fa1ea7fc700 1 -- 172.21.15.131:0/3116834198 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641835919 0 0) 0x7fa1ec0a98b0 con 0x7fa1ec15d240 2023-12-26T23:09:21.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.954+0000 7fa1eb7fe700 1 --2- 172.21.15.131:0/3116834198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec1595e0 0x7fa1ec1599b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec064440 tx=0x7fa1dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5680a098535dd4e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:21.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.954+0000 7fa1ea7fc700 1 -- 172.21.15.131:0/3116834198 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc011630 con 0x7fa1ec1595e0 2023-12-26T23:09:21.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.954+0000 7fa1ea7fc700 1 -- 172.21.15.131:0/3116834198 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc011790 con 0x7fa1ec1595e0 2023-12-26T23:09:21.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.954+0000 7fa1ea7fc700 1 -- 172.21.15.131:0/3116834198 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc011b30 con 0x7fa1ec1595e0 2023-12-26T23:09:21.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.955+0000 7fa1f1bff700 1 -- 172.21.15.131:0/3116834198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec1595e0 msgr2=0x7fa1ec1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.955+0000 7fa1f1bff700 1 --2- 172.21.15.131:0/3116834198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec1595e0 0x7fa1ec1599b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec064440 tx=0x7fa1dc0091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.955+0000 7fa1f1bff700 1 -- 172.21.15.131:0/3116834198 shutdown_connections 2023-12-26T23:09:21.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.955+0000 7fa1f1bff700 1 --2- 172.21.15.131:0/3116834198 >> v2:172.21.15.131:3300/0 conn(0x7fa1ec0ad560 0x7fa1ec159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.955+0000 7fa1f1bff700 1 --2- 172.21.15.131:0/3116834198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec1595e0 0x7fa1ec1599b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.955+0000 7fa1f1bff700 1 -- 172.21.15.131:0/3116834198 >> 172.21.15.131:0/3116834198 conn(0x7fa1ec068040 msgr2=0x7fa1ec069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:21.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.955+0000 7fa1f1bff700 1 -- 172.21.15.131:0/3116834198 shutdown_connections 2023-12-26T23:09:21.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.955+0000 7fa1f1bff700 1 -- 172.21.15.131:0/3116834198 wait complete. 2023-12-26T23:09:21.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.956+0000 7fa1f1bff700 1 Processor -- start 2023-12-26T23:09:21.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.956+0000 7fa1f1bff700 1 -- start start 2023-12-26T23:09:21.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.957+0000 7fa1f1bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1ec0ad560 0x7fa1ec0b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.957+0000 7fa1f1bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1ec1595e0 0x7fa1ec152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.957+0000 7fa1f1bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec15d240 0x7fa1ec152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.957+0000 7fa1f1bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec15b9c0 con 0x7fa1ec1595e0 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.957+0000 7fa1f1bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec15bca0 con 0x7fa1ec15d240 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.957+0000 7fa1f1bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec15bf80 con 0x7fa1ec0ad560 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eb7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1ec0ad560 0x7fa1ec0b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eb7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1ec0ad560 0x7fa1ec0b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:53092/0 (socket says 172.21.15.131:53092) 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eb7fe700 1 -- 172.21.15.131:0/2927976739 learned_addr learned my addr 172.21.15.131:0/2927976739 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eaffd700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1ec1595e0 0x7fa1ec152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1ebfff700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec15d240 0x7fa1ec152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eb7fe700 1 -- 172.21.15.131:0/2927976739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec15d240 msgr2=0x7fa1ec152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eb7fe700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec15d240 0x7fa1ec152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eb7fe700 1 -- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1ec1595e0 msgr2=0x7fa1ec152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:21.959 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eb7fe700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1ec1595e0 0x7fa1ec152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:21.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eb7fe700 1 -- 172.21.15.131:0/2927976739 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e4009a20 con 0x7fa1ec0ad560 2023-12-26T23:09:21.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1ebfff700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec15d240 0x7fa1ec152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:21.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.958+0000 7fa1eb7fe700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1ec0ad560 0x7fa1ec0b0e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec15a640 tx=0x7fa1dc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:21.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.959+0000 7fa1d37fe700 1 -- 172.21.15.131:0/2927976739 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc011b10 con 0x7fa1ec0ad560 2023-12-26T23:09:21.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.959+0000 7fa1d37fe700 1 -- 172.21.15.131:0/2927976739 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc0253f0 con 0x7fa1ec0ad560 2023-12-26T23:09:21.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.959+0000 7fa1d37fe700 1 -- 172.21.15.131:0/2927976739 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc025790 con 0x7fa1ec0ad560 2023-12-26T23:09:21.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.959+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc009420 con 0x7fa1ec0ad560 2023-12-26T23:09:21.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.959+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec15c540 con 0x7fa1ec0ad560 2023-12-26T23:09:21.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.960+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec15c6d0 con 0x7fa1ec0ad560 2023-12-26T23:09:21.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.961+0000 7fa1d37fe700 1 -- 172.21.15.131:0/2927976739 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fa1dc025b80 con 0x7fa1ec0ad560 2023-12-26T23:09:21.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.962+0000 7fa1d37fe700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa1d8067050 0x7fa1d8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:21.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.962+0000 7fa1eaffd700 1 -- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa1d8067050 msgr2=0x7fa1d8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:21.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.962+0000 7fa1d37fe700 1 -- 172.21.15.131:0/2927976739 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fa1dc086320 con 0x7fa1ec0ad560 2023-12-26T23:09:21.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.962+0000 7fa1eaffd700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa1d8067050 0x7fa1d8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:21.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:21.966+0000 7fa1d37fe700 1 -- 172.21.15.131:0/2927976739 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1dc053090 con 0x7fa1ec0ad560 2023-12-26T23:09:22.045 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:22.046+0000 7f64e85631c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:09:22.126 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:22.127+0000 7f2d11f141c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:09:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.163+0000 7fa1eaffd700 1 -- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa1d8067050 msgr2=0x7fa1d8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.163+0000 7fa1eaffd700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa1d8067050 0x7fa1d8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:22.194 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:22.195+0000 7f64e85631c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:09:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.197+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1ec0657e0 con 0x7fa1ec0ad560 2023-12-26T23:09:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.198+0000 7fa1d37fe700 1 -- 172.21.15.131:0/2927976739 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v179) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1dc0566b0 con 0x7fa1ec0ad560 2023-12-26T23:09:22.198 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T23:09:22.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.200+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa1d8067050 msgr2=0x7fa1d8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:22.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.200+0000 7fa1f1bff700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa1d8067050 0x7fa1d8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.200+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1ec0ad560 msgr2=0x7fa1ec0b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:22.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.200+0000 7fa1f1bff700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1ec0ad560 0x7fa1ec0b0e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec15a640 tx=0x7fa1dc017000 comp rx=0 tx=0).stop 2023-12-26T23:09:22.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.201+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 shutdown_connections 2023-12-26T23:09:22.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.201+0000 7fa1f1bff700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa1d8067050 0x7fa1d8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.201 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.201+0000 7fa1f1bff700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1ec15d240 0x7fa1ec152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.201+0000 7fa1f1bff700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1ec0ad560 0x7fa1ec0b0e60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.201+0000 7fa1f1bff700 1 --2- 172.21.15.131:0/2927976739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1ec1595e0 0x7fa1ec152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.201+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 >> 172.21.15.131:0/2927976739 conn(0x7fa1ec068040 msgr2=0x7fa1ec0ab020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.201+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 shutdown_connections 2023-12-26T23:09:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.201+0000 7fa1f1bff700 1 -- 172.21.15.131:0/2927976739 wait complete. 2023-12-26T23:09:22.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T23:09:22.286 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:22.287+0000 7f2d11f141c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:09:22.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.417+0000 7f2a9e3ff700 1 Processor -- start 2023-12-26T23:09:22.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.417+0000 7f2a9e3ff700 1 -- start start 2023-12-26T23:09:22.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a9e3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a980c8820 0x7f2a980ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:22.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a9e3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2a980b19f0 0x7f2a98161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:22.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a9e3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a98157a80 con 0x7f2a980c8bf0 2023-12-26T23:09:22.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a9e3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980cac50 con 0x7f2a980c8820 2023-12-26T23:09:22.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a9e3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980caf30 con 0x7f2a980b19f0 2023-12-26T23:09:22.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a97fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a980c8820 0x7f2a980ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:22.420 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a977fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2a980c8bf0 0x7f2a980b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46024/0 (socket says 172.21.15.131:46024) 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a977fe700 1 -- 172.21.15.131:0/501303400 learned_addr learned my addr 172.21.15.131:0/501303400 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a9c99c700 1 --2- 172.21.15.131:0/501303400 >> v2:172.21.15.190:3300/0 conn(0x7f2a980b19f0 0x7f2a98161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a97fff700 1 -- 172.21.15.131:0/501303400 >> v2:172.21.15.190:3300/0 conn(0x7f2a980b19f0 msgr2=0x7f2a98161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a97fff700 1 --2- 172.21.15.131:0/501303400 >> v2:172.21.15.190:3300/0 conn(0x7f2a980b19f0 0x7f2a98161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.419+0000 7f2a97fff700 1 -- 172.21.15.131:0/501303400 >> v1:172.21.15.131:6789/0 conn(0x7f2a980c8bf0 legacy=0x7f2a980b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.420+0000 7f2a97fff700 1 -- 172.21.15.131:0/501303400 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a980cb210 con 0x7f2a980c8820 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.420+0000 7f2a97fff700 1 --2- 172.21.15.131:0/501303400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980c8820 0x7f2a980ca300 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f2a98070f80 tx=0x7f2a880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ce99c69e6bca74 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.420+0000 7f2a96ffd700 1 -- 172.21.15.131:0/501303400 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88011630 con 0x7f2a980c8820 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.420+0000 7f2a96ffd700 1 -- 172.21.15.131:0/501303400 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a88011790 con 0x7f2a980c8820 2023-12-26T23:09:22.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.420+0000 7f2a96ffd700 1 -- 172.21.15.131:0/501303400 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88011b30 con 0x7f2a980c8820 2023-12-26T23:09:22.422 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.421+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/501303400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980c8820 msgr2=0x7f2a980ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.421+0000 7f2a9e3ff700 1 --2- 172.21.15.131:0/501303400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980c8820 0x7f2a980ca300 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f2a98070f80 tx=0x7f2a880091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.421+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/501303400 shutdown_connections 2023-12-26T23:09:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.421+0000 7f2a9e3ff700 1 --2- 172.21.15.131:0/501303400 >> v2:172.21.15.190:3300/0 conn(0x7f2a980b19f0 0x7f2a98161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.421+0000 7f2a9e3ff700 1 --2- 172.21.15.131:0/501303400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980c8820 0x7f2a980ca300 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.421+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/501303400 >> 172.21.15.131:0/501303400 conn(0x7f2a98068040 msgr2=0x7f2a98069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.421+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/501303400 shutdown_connections 2023-12-26T23:09:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.421+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/501303400 wait complete. 2023-12-26T23:09:22.423 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.423+0000 7f2a9e3ff700 1 Processor -- start 2023-12-26T23:09:22.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.423+0000 7f2a9e3ff700 1 -- start start 2023-12-26T23:09:22.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.424+0000 7f2a9e3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980b19f0 0x7f2a980a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.424+0000 7f2a9e3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c8820 0x7f2a9815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.424+0000 7f2a9e3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a980c8bf0 0x7f2a980b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.424+0000 7f2a9e3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980af5e0 con 0x7f2a980b19f0 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.424+0000 7f2a9e3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980af7c0 con 0x7f2a980c8820 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.424+0000 7f2a9e3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980afaa0 con 0x7f2a980c8bf0 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980b19f0 0x7f2a980a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a97fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980b19f0 0x7f2a980a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47628/0 (socket says 172.21.15.131:47628) 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a97fff700 1 -- 172.21.15.131:0/636597925 learned_addr learned my addr 172.21.15.131:0/636597925 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a977fe700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c8820 0x7f2a9815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:22.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a9c99c700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a980c8bf0 0x7f2a980b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a97fff700 1 -- 172.21.15.131:0/636597925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a980c8bf0 msgr2=0x7f2a980b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a97fff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a980c8bf0 0x7f2a980b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a97fff700 1 -- 172.21.15.131:0/636597925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c8820 msgr2=0x7f2a9815c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a97fff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c8820 0x7f2a9815c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a97fff700 1 -- 172.21.15.131:0/636597925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a88009420 con 0x7f2a980b19f0 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.425+0000 7f2a97fff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980b19f0 0x7f2a980a54a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f2a980688f0 tx=0x7f2a88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.426+0000 7f2a7ffff700 1 -- 172.21.15.131:0/636597925 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88000d10 con 0x7f2a980b19f0 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.426+0000 7f2a7ffff700 1 -- 172.21.15.131:0/636597925 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a88011b10 con 0x7f2a980b19f0 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.426+0000 7f2a7ffff700 1 -- 172.21.15.131:0/636597925 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88025640 con 0x7f2a980b19f0 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.426+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a980afd80 con 0x7f2a980b19f0 2023-12-26T23:09:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.426+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a980b0340 con 0x7f2a980b19f0 2023-12-26T23:09:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.427+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a980ae100 con 0x7f2a980b19f0 2023-12-26T23:09:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.428+0000 7f2a7ffff700 1 -- 172.21.15.131:0/636597925 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f2a88025a30 con 0x7f2a980b19f0 2023-12-26T23:09:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.429+0000 7f2a7ffff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a84067050 0x7f2a84069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.429+0000 7f2a7ffff700 1 -- 172.21.15.131:0/636597925 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f2a88086810 con 0x7f2a980b19f0 2023-12-26T23:09:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.429+0000 7f2a977fe700 1 -- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a84067050 msgr2=0x7f2a84069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:22.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.429+0000 7f2a977fe700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a84067050 0x7f2a84069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:22.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.432+0000 7f2a7ffff700 1 -- 172.21.15.131:0/636597925 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a88053540 con 0x7f2a980b19f0 2023-12-26T23:09:22.623 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:22.624+0000 7f64e85631c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:09:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.630+0000 7f2a977fe700 1 -- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a84067050 msgr2=0x7f2a84069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:22.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.630+0000 7f2a977fe700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a84067050 0x7f2a84069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:22.748 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.748+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a980657e0 con 0x7f2a980b19f0 2023-12-26T23:09:22.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.749+0000 7f2a7ffff700 1 -- 172.21.15.131:0/636597925 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v180) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a88015020 con 0x7f2a980b19f0 2023-12-26T23:09:22.749 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T23:09:22.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a84067050 msgr2=0x7f2a84069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:22.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a84067050 0x7f2a84069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980b19f0 msgr2=0x7f2a980a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:22.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980b19f0 0x7f2a980a54a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f2a980688f0 tx=0x7f2a88017000 comp rx=0 tx=0).stop 2023-12-26T23:09:22.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 shutdown_connections 2023-12-26T23:09:22.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a84067050 0x7f2a84069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a980c8bf0 0x7f2a980b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a980b19f0 0x7f2a980a54a0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 --2- 172.21.15.131:0/636597925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c8820 0x7f2a9815c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:22.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 >> 172.21.15.131:0/636597925 conn(0x7f2a98068040 msgr2=0x7f2a980c2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:22.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 shutdown_connections 2023-12-26T23:09:22.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:22.751+0000 7f2a9e3ff700 1 -- 172.21.15.131:0/636597925 wait complete. 2023-12-26T23:09:22.779 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T23:09:22.783 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:22.784+0000 7f2d11f141c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:09:22.923 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:22.924+0000 7f64e85631c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:09:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.006+0000 7f9570dff700 1 Processor -- start 2023-12-26T23:09:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.006+0000 7f9570dff700 1 -- start start 2023-12-26T23:09:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.007+0000 7f9570dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f956c159a70 0x7f956c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.007+0000 7f9570dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c0b2a90 0x7f956c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.007+0000 7f9570dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956c0afcd0 con 0x7f956c1596a0 2023-12-26T23:09:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.007+0000 7f9570dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956c0b3e70 con 0x7f956c159a70 2023-12-26T23:09:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.007+0000 7f9570dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0b0f20 con 0x7f956c0b2a90 2023-12-26T23:09:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f9569d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f956c159a70 0x7f956c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f956ad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c0b2a90 0x7f956c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f9569d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f956c159a70 0x7f956c0b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47630/0 (socket says 172.21.15.131:47630) 2023-12-26T23:09:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f9569d9b700 1 -- 172.21.15.131:0/162026583 learned_addr learned my addr 172.21.15.131:0/162026583 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f956a59c700 1 --1- 172.21.15.131:0/162026583 >> v1:172.21.15.190:6789/0 conn(0x7f956c1596a0 0x7f956c0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:50112/0 (socket says 172.21.15.131:50112) 2023-12-26T23:09:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f9569d9b700 1 -- 172.21.15.131:0/162026583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c0b2a90 msgr2=0x7f956c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f9569d9b700 1 --2- 172.21.15.131:0/162026583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c0b2a90 0x7f956c0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f9569d9b700 1 -- 172.21.15.131:0/162026583 >> v1:172.21.15.190:6789/0 conn(0x7f956c1596a0 legacy=0x7f956c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f9569d9b700 1 -- 172.21.15.131:0/162026583 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c0b3540 con 0x7f956c159a70 2023-12-26T23:09:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.008+0000 7f9569d9b700 1 --2- 172.21.15.131:0/162026583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c159a70 0x7f956c0b0160 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f95600099b0 tx=0x7f9560009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7543ea31267a85b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.009+0000 7f956959a700 1 -- 172.21.15.131:0/162026583 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560009ea0 con 0x7f956c159a70 2023-12-26T23:09:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.009+0000 7f956959a700 1 -- 172.21.15.131:0/162026583 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956000b860 con 0x7f956c159a70 2023-12-26T23:09:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.009+0000 7f956959a700 1 -- 172.21.15.131:0/162026583 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000bc00 con 0x7f956c159a70 2023-12-26T23:09:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.009+0000 7f9570dff700 1 -- 172.21.15.131:0/162026583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c159a70 msgr2=0x7f956c0b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.009+0000 7f9570dff700 1 --2- 172.21.15.131:0/162026583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c159a70 0x7f956c0b0160 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f95600099b0 tx=0x7f9560009250 comp rx=0 tx=0).stop 2023-12-26T23:09:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.009+0000 7f9570dff700 1 -- 172.21.15.131:0/162026583 shutdown_connections 2023-12-26T23:09:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.009+0000 7f9570dff700 1 --2- 172.21.15.131:0/162026583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c0b2a90 0x7f956c0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.009+0000 7f9570dff700 1 --2- 172.21.15.131:0/162026583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c159a70 0x7f956c0b0160 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.009+0000 7f9570dff700 1 -- 172.21.15.131:0/162026583 >> 172.21.15.131:0/162026583 conn(0x7f956c06ece0 msgr2=0x7f956c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.010+0000 7f9570dff700 1 -- 172.21.15.131:0/162026583 shutdown_connections 2023-12-26T23:09:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.010+0000 7f9570dff700 1 -- 172.21.15.131:0/162026583 wait complete. 2023-12-26T23:09:23.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.012+0000 7f9570dff700 1 Processor -- start 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.012+0000 7f9570dff700 1 -- start start 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f9570dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c0b2a90 0x7f956c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f9570dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f956c1596a0 0x7f956c150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f9570dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c159a70 0x7f956c16cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f9570dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0c5760 con 0x7f956c0b2a90 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f9570dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0c4b50 con 0x7f956c1596a0 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f9570dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0c4e30 con 0x7f956c159a70 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f956a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c0b2a90 0x7f956c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f9569d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f956c1596a0 0x7f956c150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f956a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c0b2a90 0x7f956c0cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47636/0 (socket says 172.21.15.131:47636) 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f956a59c700 1 -- 172.21.15.131:0/4119396258 learned_addr learned my addr 172.21.15.131:0/4119396258 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.013+0000 7f956ad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c159a70 0x7f956c16cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.014+0000 7f9569d9b700 1 -- 172.21.15.131:0/4119396258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c159a70 msgr2=0x7f956c16cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.014+0000 7f9569d9b700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c159a70 0x7f956c16cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.014+0000 7f9569d9b700 1 -- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c0b2a90 msgr2=0x7f956c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.014+0000 7f9569d9b700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c0b2a90 0x7f956c0cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.014+0000 7f9569d9b700 1 -- 172.21.15.131:0/4119396258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600094f0 con 0x7f956c1596a0 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.014+0000 7f956ad9d700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c159a70 0x7f956c16cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.014+0000 7f9569d9b700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f956c1596a0 0x7f956c150970 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f956c0892a0 tx=0x7f9560017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:23.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.015+0000 7f955a7fc700 1 -- 172.21.15.131:0/4119396258 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560000d10 con 0x7f956c1596a0 2023-12-26T23:09:23.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.015+0000 7f955a7fc700 1 -- 172.21.15.131:0/4119396258 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956000bbe0 con 0x7f956c1596a0 2023-12-26T23:09:23.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.015+0000 7f955a7fc700 1 -- 172.21.15.131:0/4119396258 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560011670 con 0x7f956c1596a0 2023-12-26T23:09:23.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.015+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956c0c5110 con 0x7f956c1596a0 2023-12-26T23:09:23.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.015+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956c0cc580 con 0x7f956c1596a0 2023-12-26T23:09:23.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.016+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956c0c58d0 con 0x7f956c1596a0 2023-12-26T23:09:23.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.018+0000 7f955a7fc700 1 -- 172.21.15.131:0/4119396258 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f9560011aa0 con 0x7f956c1596a0 2023-12-26T23:09:23.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.018+0000 7f955a7fc700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f954c067050 0x7f954c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.018+0000 7f955a7fc700 1 -- 172.21.15.131:0/4119396258 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f9560085ad0 con 0x7f956c1596a0 2023-12-26T23:09:23.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.018+0000 7f956a59c700 1 -- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f954c067050 msgr2=0x7f954c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:23.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.018+0000 7f956a59c700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f954c067050 0x7f954c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:23.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.023+0000 7f955a7fc700 1 -- 172.21.15.131:0/4119396258 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9560052870 con 0x7f956c1596a0 2023-12-26T23:09:23.097 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:23.097+0000 7f2d11f141c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:09:23.205 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:23.206+0000 7f64e85631c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:09:23.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.219+0000 7f956a59c700 1 -- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f954c067050 msgr2=0x7f954c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:23.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.219+0000 7f956a59c700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f954c067050 0x7f954c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:23.219 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.220+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f956c0c4f70 con 0x7f956c1596a0 2023-12-26T23:09:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.220+0000 7f955a7fc700 1 -- 172.21.15.131:0/4119396258 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v180) v1 ==== 100+0+11 (secure 0 0 0) 0x7f956c0c4f70 con 0x7f956c1596a0 2023-12-26T23:09:23.220 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T23:09:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f954c067050 msgr2=0x7f954c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f954c067050 0x7f954c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f956c1596a0 msgr2=0x7f956c150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f956c1596a0 0x7f956c150970 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f956c0892a0 tx=0x7f9560017000 comp rx=0 tx=0).stop 2023-12-26T23:09:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 shutdown_connections 2023-12-26T23:09:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f954c067050 0x7f954c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f956c159a70 0x7f956c16cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f956c0b2a90 0x7f956c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 --2- 172.21.15.131:0/4119396258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f956c1596a0 0x7f956c150970 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 >> 172.21.15.131:0/4119396258 conn(0x7f956c06ece0 msgr2=0x7f956c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:23.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 shutdown_connections 2023-12-26T23:09:23.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.224+0000 7f9570dff700 1 -- 172.21.15.131:0/4119396258 wait complete. 2023-12-26T23:09:23.251 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:09:23.295 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:23.296+0000 7f64e85631c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:09:23.418 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:23.417+0000 7f2d11f141c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:09:23.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.467+0000 7f20093ff700 1 Processor -- start 2023-12-26T23:09:23.467 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.467+0000 7f20093ff700 1 -- start start 2023-12-26T23:09:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.468+0000 7f20093ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f20041573f0 0x7f20040af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.468+0000 7f20093ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20040aca80 0x7f20040acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.468+0000 7f20093ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20040af1b0 con 0x7f20041577c0 2023-12-26T23:09:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.468+0000 7f20093ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20040ca830 con 0x7f20041573f0 2023-12-26T23:09:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.468+0000 7f20093ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20040ad440 con 0x7f20040aca80 2023-12-26T23:09:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.469+0000 7f2002ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f20041573f0 0x7f20040af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.469+0000 7f20037fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20040aca80 0x7f20040acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.469+0000 7f20027fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f20041577c0 0x7f200415b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46040/0 (socket says 172.21.15.131:46040) 2023-12-26T23:09:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.469+0000 7f20027fc700 1 -- 172.21.15.131:0/1761229826 learned_addr learned my addr 172.21.15.131:0/1761229826 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.469+0000 7f2002ffd700 1 -- 172.21.15.131:0/1761229826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20040aca80 msgr2=0x7f20040acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.469+0000 7f2002ffd700 1 --2- 172.21.15.131:0/1761229826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20040aca80 0x7f20040acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.469+0000 7f2002ffd700 1 -- 172.21.15.131:0/1761229826 >> v1:172.21.15.131:6789/0 conn(0x7f20041577c0 legacy=0x7f200415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.469+0000 7f2002ffd700 1 -- 172.21.15.131:0/1761229826 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2004067870 con 0x7f20041573f0 2023-12-26T23:09:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.469+0000 7f20037fe700 1 --2- 172.21.15.131:0/1761229826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20040aca80 0x7f20040acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.470+0000 7f2002ffd700 1 --2- 172.21.15.131:0/1761229826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041573f0 0x7f20040af640 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40098e0 tx=0x7f1ff40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd0ccaead1e37d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:23.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.472+0000 7f2001ffb700 1 -- 172.21.15.131:0/1761229826 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400b640 con 0x7f20041573f0 2023-12-26T23:09:23.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.472+0000 7f2001ffb700 1 -- 172.21.15.131:0/1761229826 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff400b7a0 con 0x7f20041573f0 2023-12-26T23:09:23.472 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.472+0000 7f2001ffb700 1 -- 172.21.15.131:0/1761229826 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400bb40 con 0x7f20041573f0 2023-12-26T23:09:23.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.472+0000 7f20093ff700 1 -- 172.21.15.131:0/1761229826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041573f0 msgr2=0x7f20040af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.472+0000 7f20093ff700 1 --2- 172.21.15.131:0/1761229826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041573f0 0x7f20040af640 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40098e0 tx=0x7f1ff40091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.473 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.473+0000 7f20093ff700 1 -- 172.21.15.131:0/1761229826 shutdown_connections 2023-12-26T23:09:23.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.473+0000 7f20093ff700 1 --2- 172.21.15.131:0/1761229826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20040aca80 0x7f20040acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.473+0000 7f20093ff700 1 --2- 172.21.15.131:0/1761229826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041573f0 0x7f20040af640 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.473+0000 7f20093ff700 1 -- 172.21.15.131:0/1761229826 >> 172.21.15.131:0/1761229826 conn(0x7f20040686c0 msgr2=0x7f2004068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:23.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.473+0000 7f20093ff700 1 -- 172.21.15.131:0/1761229826 shutdown_connections 2023-12-26T23:09:23.474 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.473+0000 7f20093ff700 1 -- 172.21.15.131:0/1761229826 wait complete. 2023-12-26T23:09:23.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.474+0000 7f20093ff700 1 Processor -- start 2023-12-26T23:09:23.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.474+0000 7f20093ff700 1 -- start start 2023-12-26T23:09:23.477 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.475+0000 7f20093ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20040aca80 0x7f20040b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.475+0000 7f20093ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20041573f0 0x7f20040b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.475+0000 7f20093ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041577c0 0x7f20040b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.475+0000 7f20093ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200415ac40 con 0x7f20041577c0 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.475+0000 7f20093ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20040c4db0 con 0x7f20040aca80 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.475+0000 7f20093ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20040b06e0 con 0x7f20041573f0 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f20037fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041577c0 0x7f20040b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f2002ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20040aca80 0x7f20040b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f2002ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20040aca80 0x7f20040b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:33890/0 (socket says 172.21.15.131:33890) 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f2002ffd700 1 -- 172.21.15.131:0/3913947358 learned_addr learned my addr 172.21.15.131:0/3913947358 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f20027fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20041573f0 0x7f20040b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f20037fe700 1 -- 172.21.15.131:0/3913947358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20041573f0 msgr2=0x7f20040b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f20037fe700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20041573f0 0x7f20040b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f20037fe700 1 -- 172.21.15.131:0/3913947358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20040aca80 msgr2=0x7f20040b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f20037fe700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20040aca80 0x7f20040b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f20037fe700 1 -- 172.21.15.131:0/3913947358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc009a20 con 0x7f20041577c0 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f20037fe700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041577c0 0x7f20040b0ee0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009f10 tx=0x7f1ffc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f1feaffd700 1 -- 172.21.15.131:0/3913947358 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00cb70 con 0x7f20041577c0 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.476+0000 7f1feaffd700 1 -- 172.21.15.131:0/3913947358 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc00f040 con 0x7f20041577c0 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.477+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff4009420 con 0x7f20041577c0 2023-12-26T23:09:23.479 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.477+0000 7f1feaffd700 1 -- 172.21.15.131:0/3913947358 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc014480 con 0x7f20041577c0 2023-12-26T23:09:23.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.477+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20040c5560 con 0x7f20041577c0 2023-12-26T23:09:23.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.478+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20040657e0 con 0x7f20041577c0 2023-12-26T23:09:23.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.479+0000 7f1feaffd700 1 -- 172.21.15.131:0/3913947358 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f1ffc003020 con 0x7f20041577c0 2023-12-26T23:09:23.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.480+0000 7f1feaffd700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ff0067050 0x7f1ff0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.480+0000 7f1feaffd700 1 -- 172.21.15.131:0/3913947358 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f1ffc085520 con 0x7f20041577c0 2023-12-26T23:09:23.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.480+0000 7f2002ffd700 1 -- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ff0067050 msgr2=0x7f1ff0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:23.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.480+0000 7f2002ffd700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ff0067050 0x7f1ff0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:23.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.484+0000 7f1feaffd700 1 -- 172.21.15.131:0/3913947358 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ffc052090 con 0x7f20041577c0 2023-12-26T23:09:23.515 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:23.515+0000 7f2d11f141c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:09:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.681+0000 7f2002ffd700 1 -- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ff0067050 msgr2=0x7f1ff0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.681+0000 7f2002ffd700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ff0067050 0x7f1ff0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:23.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.729+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20040cab40 con 0x7f20041577c0 2023-12-26T23:09:23.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.730+0000 7f1feaffd700 1 -- 172.21.15.131:0/3913947358 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f1ffc0556b0 con 0x7f20041577c0 2023-12-26T23:09:23.730 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:23.730 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2545294570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2545294570},{"type":"v1","addr":"172.21.15.190:6835","nonce":2545294570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1800615106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1800615106},{"type":"v1","addr":"172.21.15.131:6834","nonce":1800615106}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3279484094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3279484094},{"type":"v1","addr":"172.21.15.131:6835","nonce":3279484094}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:53:21.655334+0000","modified":"2023-12-26T22:53:25.218600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9105":{"gid":9105,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/716462328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":716462328},{"type":"v1","addr":"172.21.15.190:6834","nonce":716462328}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-12-26T23:09:23.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.733+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ff0067050 msgr2=0x7f1ff0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:23.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.733+0000 7f20093ff700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ff0067050 0x7f1ff0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041577c0 msgr2=0x7f20040b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041577c0 0x7f20040b0ee0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009f10 tx=0x7f1ffc00b040 comp rx=0 tx=0).stop 2023-12-26T23:09:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 shutdown_connections 2023-12-26T23:09:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ff0067050 0x7f1ff0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20041577c0 0x7f20040b0ee0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20040aca80 0x7f20040b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 --2- 172.21.15.131:0/3913947358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20041573f0 0x7f20040b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 >> 172.21.15.131:0/3913947358 conn(0x7f20040686c0 msgr2=0x7f200406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 shutdown_connections 2023-12-26T23:09:23.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.734+0000 7f20093ff700 1 -- 172.21.15.131:0/3913947358 wait complete. 2023-12-26T23:09:23.736 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 81 2023-12-26T23:09:23.760 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:09:23.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.951+0000 7fa99a3ff700 1 Processor -- start 2023-12-26T23:09:23.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.952+0000 7fa99a3ff700 1 -- start start 2023-12-26T23:09:23.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.953+0000 7fa99a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa99406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.953+0000 7fa99a3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa99406eb30 0x7fa99415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.953+0000 7fa99a3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940b07f0 con 0x7fa9940cc580 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.953+0000 7fa99a3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9940680f0 con 0x7fa99406eb30 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.953+0000 7fa99a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940ccda0 con 0x7fa9940cc950 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.953+0000 7fa993fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa9940cc580 0x7fa9940b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46056/0 (socket says 172.21.15.131:46056) 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.953+0000 7fa993fff700 1 -- 172.21.15.131:0/26272549 learned_addr learned my addr 172.21.15.131:0/26272549 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.953+0000 7fa99899c700 1 --2- 172.21.15.131:0/26272549 >> v2:172.21.15.131:3300/0 conn(0x7fa99406eb30 0x7fa99415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.953+0000 7fa9937fe700 1 --2- 172.21.15.131:0/26272549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa99406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.954+0000 7fa992ffd700 1 -- 172.21.15.131:0/26272549 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553986742 0 0) 0x7fa984001110 con 0x7fa9940cc580 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.954+0000 7fa992ffd700 1 -- 172.21.15.131:0/26272549 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa97c003570 con 0x7fa9940cc580 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.954+0000 7fa99899c700 1 -- 172.21.15.131:0/26272549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 msgr2=0x7fa99406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:23.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.954+0000 7fa99899c700 1 --2- 172.21.15.131:0/26272549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa99406e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.954+0000 7fa99899c700 1 -- 172.21.15.131:0/26272549 >> v1:172.21.15.131:6789/0 conn(0x7fa9940cc580 legacy=0x7fa9940b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.954+0000 7fa99899c700 1 -- 172.21.15.131:0/26272549 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940b0d70 con 0x7fa99406eb30 2023-12-26T23:09:23.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.954+0000 7fa99899c700 1 --2- 172.21.15.131:0/26272549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa99406eb30 0x7fa99415d9b0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009e90 tx=0x7fa98c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5945304035b5288 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:23.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.954+0000 7fa992ffd700 1 -- 172.21.15.131:0/26272549 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00e070 con 0x7fa99406eb30 2023-12-26T23:09:23.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa992ffd700 1 -- 172.21.15.131:0/26272549 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c00bb40 con 0x7fa99406eb30 2023-12-26T23:09:23.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa992ffd700 1 -- 172.21.15.131:0/26272549 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c012430 con 0x7fa99406eb30 2023-12-26T23:09:23.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa99a3ff700 1 -- 172.21.15.131:0/26272549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa99406eb30 msgr2=0x7fa99415d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa99a3ff700 1 --2- 172.21.15.131:0/26272549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa99406eb30 0x7fa99415d9b0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009e90 tx=0x7fa98c009780 comp rx=0 tx=0).stop 2023-12-26T23:09:23.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa99a3ff700 1 -- 172.21.15.131:0/26272549 shutdown_connections 2023-12-26T23:09:23.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa99a3ff700 1 --2- 172.21.15.131:0/26272549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa99406eb30 0x7fa99415d9b0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa99a3ff700 1 --2- 172.21.15.131:0/26272549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa99406e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa99a3ff700 1 -- 172.21.15.131:0/26272549 >> 172.21.15.131:0/26272549 conn(0x7fa9940686c0 msgr2=0x7fa994068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:23.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa99a3ff700 1 -- 172.21.15.131:0/26272549 shutdown_connections 2023-12-26T23:09:23.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.955+0000 7fa99a3ff700 1 -- 172.21.15.131:0/26272549 wait complete. 2023-12-26T23:09:23.960 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.957+0000 7fa99a3ff700 1 Processor -- start 2023-12-26T23:09:23.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.958+0000 7fa99a3ff700 1 -- start start 2023-12-26T23:09:23.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99a3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa99406eb30 0x7fa994156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99a3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9940cc580 0x7fa9940adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa9941694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99a3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940c2570 con 0x7fa9940cc580 2023-12-26T23:09:23.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99a3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940c2810 con 0x7fa99406eb30 2023-12-26T23:09:23.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa994156d70 con 0x7fa9940cc950 2023-12-26T23:09:23.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa9937fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9940cc580 0x7fa9940adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa993fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa99406eb30 0x7fa994156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99899c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa9941694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa9937fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9940cc580 0x7fa9940adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47662/0 (socket says 172.21.15.131:47662) 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99899c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa9941694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:53144/0 (socket says 172.21.15.131:53144) 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa9937fe700 1 -- 172.21.15.131:0/2306404283 learned_addr learned my addr 172.21.15.131:0/2306404283 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99899c700 1 -- 172.21.15.131:0/2306404283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa99406eb30 msgr2=0x7fa994156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99899c700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa99406eb30 0x7fa994156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99899c700 1 -- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9940cc580 msgr2=0x7fa9940adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99899c700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9940cc580 0x7fa9940adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa99899c700 1 -- 172.21.15.131:0/2306404283 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009a20 con 0x7fa9940cc950 2023-12-26T23:09:23.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.959+0000 7fa9937fe700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9940cc580 0x7fa9940adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:23.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.960+0000 7fa99899c700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa9941694b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa98c016000 tx=0x7fa98c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:23.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.960+0000 7fa97bfff700 1 -- 172.21.15.131:0/2306404283 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c027470 con 0x7fa9940cc950 2023-12-26T23:09:23.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.960+0000 7fa97bfff700 1 -- 172.21.15.131:0/2306404283 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c0275d0 con 0x7fa9940cc950 2023-12-26T23:09:23.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.960+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9940ae0b0 con 0x7fa9940cc950 2023-12-26T23:09:23.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.960+0000 7fa97bfff700 1 -- 172.21.15.131:0/2306404283 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c027990 con 0x7fa9940cc950 2023-12-26T23:09:23.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.960+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9940ae660 con 0x7fa9940cc950 2023-12-26T23:09:23.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.961+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9940c2980 con 0x7fa9940cc950 2023-12-26T23:09:23.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.962+0000 7fa97bfff700 1 -- 172.21.15.131:0/2306404283 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fa98c024070 con 0x7fa9940cc950 2023-12-26T23:09:23.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.963+0000 7fa97bfff700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa980067050 0x7fa980069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:23.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.963+0000 7fa97bfff700 1 -- 172.21.15.131:0/2306404283 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa98c087190 con 0x7fa9940cc950 2023-12-26T23:09:23.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.963+0000 7fa993fff700 1 -- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa980067050 msgr2=0x7fa980069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:23.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.963+0000 7fa993fff700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa980067050 0x7fa980069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:23.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:23.967+0000 7fa97bfff700 1 -- 172.21.15.131:0/2306404283 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa98c053e10 con 0x7fa9940cc950 2023-12-26T23:09:24.029 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:24.029+0000 7f64e85631c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:09:24.123 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:24.123+0000 7f64e85631c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:09:24.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.164+0000 7fa993fff700 1 -- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa980067050 msgr2=0x7fa980069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:24.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.164+0000 7fa993fff700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa980067050 0x7fa980069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:24.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.167+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9940657e0 con 0x7fa9940cc950 2023-12-26T23:09:24.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.168+0000 7fa97bfff700 1 -- 172.21.15.131:0/2306404283 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+17083 (secure 0 0 0) 0x7fa98c057430 con 0x7fa9940cc950 2023-12-26T23:09:24.169 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:24.169 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":180,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:09:22.463161+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:53:19.805015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-26T22:53:20.636358+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":180,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.5","mappings":[{"from":2,"to":5}]},{"pgid":"21.3f","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/109159886":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/1073482886":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/659813654":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/3572226702":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/3168215713":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/2626934136":"2023-12-27T22:53:37.495457+0000","172.21.15.131:6836/39288":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/892276341":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/470407650":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/1083345344":"2023-12-27T23:04:23.466126+0000","172.21.15.131:6837/39288":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/1605877564":"2023-12-27T22:53:37.495457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:09:24.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa980067050 msgr2=0x7fa980069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa980067050 0x7fa980069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 msgr2=0x7fa9941694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa9941694b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa98c016000 tx=0x7fa98c019000 comp rx=0 tx=0).stop 2023-12-26T23:09:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 shutdown_connections 2023-12-26T23:09:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa980067050 0x7fa980069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9940cc950 0x7fa9941694b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa99406eb30 0x7fa994156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 --2- 172.21.15.131:0/2306404283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9940cc580 0x7fa9940adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 >> 172.21.15.131:0/2306404283 conn(0x7fa9940686c0 msgr2=0x7fa9940cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 shutdown_connections 2023-12-26T23:09:24.173 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.171+0000 7fa99a3ff700 1 -- 172.21.15.131:0/2306404283 wait complete. 2023-12-26T23:09:24.190 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T23:09:24.191 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:09:24.273 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:24.273+0000 7f64e85631c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:09:24.297 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:24.297+0000 7f2d11f141c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:09:24.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.385+0000 7f857dfff700 1 Processor -- start 2023-12-26T23:09:24.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.385+0000 7f857dfff700 1 -- start start 2023-12-26T23:09:24.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.386+0000 7f857dfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f857806f4b0 0x7f8578157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.386+0000 7f857dfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8578157f90 0x7f85780b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.386+0000 7f857dfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857806f900 con 0x7f857806f0e0 2023-12-26T23:09:24.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.386+0000 7f857dfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8578158af0 con 0x7f857806f4b0 2023-12-26T23:09:24.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.386+0000 7f857dfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857815b5c0 con 0x7f8578157f90 2023-12-26T23:09:24.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f8576ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f857806f4b0 0x7f8578157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f85777fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f857806f0e0 0x7f857815ba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:50120/0 (socket says 172.21.15.131:50120) 2023-12-26T23:09:24.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f8576ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f857806f4b0 0x7f8578157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47678/0 (socket says 172.21.15.131:47678) 2023-12-26T23:09:24.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f85777fe700 1 -- 172.21.15.131:0/2524377188 learned_addr learned my addr 172.21.15.131:0/2524377188 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:24.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f8577fff700 1 --2- 172.21.15.131:0/2524377188 >> v2:172.21.15.190:3300/0 conn(0x7f8578157f90 0x7f85780b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f85767fc700 1 -- 172.21.15.131:0/2524377188 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64963470 0 0) 0x7f857806f900 con 0x7f857806f0e0 2023-12-26T23:09:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f8576ffd700 1 -- 172.21.15.131:0/2524377188 >> v2:172.21.15.190:3300/0 conn(0x7f8578157f90 msgr2=0x7f85780b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f8576ffd700 1 --2- 172.21.15.131:0/2524377188 >> v2:172.21.15.190:3300/0 conn(0x7f8578157f90 0x7f85780b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f8576ffd700 1 -- 172.21.15.131:0/2524377188 >> v1:172.21.15.190:6789/0 conn(0x7f857806f0e0 legacy=0x7f857815ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f8576ffd700 1 -- 172.21.15.131:0/2524377188 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8578158530 con 0x7f857806f4b0 2023-12-26T23:09:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.387+0000 7f8576ffd700 1 --2- 172.21.15.131:0/2524377188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 0x7f8578157a50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f8578152110 tx=0x7f856c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bbba3cec9373d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f85767fc700 1 -- 172.21.15.131:0/2524377188 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c011630 con 0x7f857806f4b0 2023-12-26T23:09:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f85767fc700 1 -- 172.21.15.131:0/2524377188 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c011790 con 0x7f857806f4b0 2023-12-26T23:09:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f85767fc700 1 -- 172.21.15.131:0/2524377188 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c011b30 con 0x7f857806f4b0 2023-12-26T23:09:24.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f857dfff700 1 -- 172.21.15.131:0/2524377188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 msgr2=0x7f8578157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f857dfff700 1 --2- 172.21.15.131:0/2524377188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 0x7f8578157a50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f8578152110 tx=0x7f856c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f857dfff700 1 -- 172.21.15.131:0/2524377188 shutdown_connections 2023-12-26T23:09:24.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f857dfff700 1 --2- 172.21.15.131:0/2524377188 >> v2:172.21.15.190:3300/0 conn(0x7f8578157f90 0x7f85780b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f857dfff700 1 --2- 172.21.15.131:0/2524377188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 0x7f8578157a50 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f857dfff700 1 -- 172.21.15.131:0/2524377188 >> 172.21.15.131:0/2524377188 conn(0x7f85780686c0 msgr2=0x7f8578068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:24.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f857dfff700 1 -- 172.21.15.131:0/2524377188 shutdown_connections 2023-12-26T23:09:24.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.388+0000 7f857dfff700 1 -- 172.21.15.131:0/2524377188 wait complete. 2023-12-26T23:09:24.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.390+0000 7f857dfff700 1 Processor -- start 2023-12-26T23:09:24.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.390+0000 7f857dfff700 1 -- start start 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f857dfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f857806f0e0 0x7f85780ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f857dfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 0x7f857814bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f857dfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8578157f90 0x7f8578159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f857dfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857814f700 con 0x7f857806f4b0 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f857dfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857815eb00 con 0x7f857806f0e0 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f857dfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857815def0 con 0x7f8578157f90 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f8576ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 0x7f857814bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f8576ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 0x7f857814bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47688/0 (socket says 172.21.15.131:47688) 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f8576ffd700 1 -- 172.21.15.131:0/1386311456 learned_addr learned my addr 172.21.15.131:0/1386311456 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:24.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f85777fe700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f857806f0e0 0x7f85780ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.391+0000 7f8577fff700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8578157f90 0x7f8578159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f8576ffd700 1 -- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8578157f90 msgr2=0x7f8578159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f8576ffd700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8578157f90 0x7f8578159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f8576ffd700 1 -- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f857806f0e0 msgr2=0x7f85780ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f8576ffd700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f857806f0e0 0x7f85780ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f8576ffd700 1 -- 172.21.15.131:0/1386311456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c009420 con 0x7f857806f4b0 2023-12-26T23:09:24.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f85777fe700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f857806f0e0 0x7f85780ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:24.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f8577fff700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8578157f90 0x7f8578159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:24.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f8576ffd700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 0x7f857814bbf0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f8578068690 tx=0x7f856c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f855f7fe700 1 -- 172.21.15.131:0/1386311456 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c011b10 con 0x7f857806f4b0 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f855f7fe700 1 -- 172.21.15.131:0/1386311456 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c0253f0 con 0x7f857806f4b0 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857815e1d0 con 0x7f857806f4b0 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f855f7fe700 1 -- 172.21.15.131:0/1386311456 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c025790 con 0x7f857806f4b0 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.392+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857815e790 con 0x7f857806f4b0 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.393+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85780657e0 con 0x7f857806f4b0 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.394+0000 7f855f7fe700 1 -- 172.21.15.131:0/1386311456 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f856c025b80 con 0x7f857806f4b0 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.396+0000 7f855f7fe700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8564067050 0x7f8564069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.396+0000 7f85777fe700 1 -- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8564067050 msgr2=0x7f8564069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:24.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.396+0000 7f855f7fe700 1 -- 172.21.15.131:0/1386311456 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f856c0864a0 con 0x7f857806f4b0 2023-12-26T23:09:24.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.396+0000 7f85777fe700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8564067050 0x7f8564069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:24.401 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:24.400+0000 7f2d11f141c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:09:24.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.400+0000 7f855f7fe700 1 -- 172.21.15.131:0/1386311456 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f856c053120 con 0x7f857806f4b0 2023-12-26T23:09:24.559 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:24.559+0000 7f2d11f141c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:09:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.597+0000 7f85777fe700 1 -- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8564067050 msgr2=0x7f8564069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:24.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.597+0000 7f85777fe700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8564067050 0x7f8564069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:24.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.646+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85781588a0 con 0x7f857806f4b0 2023-12-26T23:09:24.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.647+0000 7f855f7fe700 1 -- 172.21.15.131:0/1386311456 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f856c056740 con 0x7f857806f4b0 2023-12-26T23:09:24.653 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:24.654 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2545294570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2545294570},{"type":"v1","addr":"172.21.15.190:6835","nonce":2545294570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1800615106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1800615106},{"type":"v1","addr":"172.21.15.131:6834","nonce":1800615106}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3279484094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3279484094},{"type":"v1","addr":"172.21.15.131:6835","nonce":3279484094}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:53:21.655334+0000","modified":"2023-12-26T22:53:25.218600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9105":{"gid":9105,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/716462328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":716462328},{"type":"v1","addr":"172.21.15.190:6834","nonce":716462328}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-12-26T23:09:24.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8564067050 msgr2=0x7f8564069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:24.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8564067050 0x7f8564069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 msgr2=0x7f857814bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 0x7f857814bbf0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f8578068690 tx=0x7f856c017000 comp rx=0 tx=0).stop 2023-12-26T23:09:24.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 shutdown_connections 2023-12-26T23:09:24.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8564067050 0x7f8564069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8578157f90 0x7f8578159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f857806f0e0 0x7f85780ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 --2- 172.21.15.131:0/1386311456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f857806f4b0 0x7f857814bbf0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 >> 172.21.15.131:0/1386311456 conn(0x7f85780686c0 msgr2=0x7f857814db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.649+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 shutdown_connections 2023-12-26T23:09:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.650+0000 7f857dfff700 1 -- 172.21.15.131:0/1386311456 wait complete. 2023-12-26T23:09:24.657 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 81 2023-12-26T23:09:24.668 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:09:24.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.863+0000 7f4b913ff700 1 Processor -- start 2023-12-26T23:09:24.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.863+0000 7f4b913ff700 1 -- start start 2023-12-26T23:09:24.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.864+0000 7f4b913ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b8c15b260 0x7f4b8c0caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.864+0000 7f4b913ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c0aecc0 0x7f4b8c15b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b913ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8c067c30 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b913ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b8c067650 con 0x7f4b8c15b260 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b913ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c15bcd0 con 0x7f4b8c0aecc0 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b8affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b8c15b260 0x7f4b8c0caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b8affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4b8c15b260 0x7f4b8c0caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42034/0 (socket says 172.21.15.131:42034) 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b8affd700 1 -- 172.21.15.131:0/1150462823 learned_addr learned my addr 172.21.15.131:0/1150462823 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b8a7fc700 1 --2- 172.21.15.131:0/1150462823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c0aecc0 0x7f4b8c15b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b8affd700 1 -- 172.21.15.131:0/1150462823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c0aecc0 msgr2=0x7f4b8c15b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b8affd700 1 --2- 172.21.15.131:0/1150462823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c0aecc0 0x7f4b8c15b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b8affd700 1 -- 172.21.15.131:0/1150462823 >> v1:172.21.15.190:6789/0 conn(0x7f4b8c0c8e70 legacy=0x7f4b8c0b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b8affd700 1 -- 172.21.15.131:0/1150462823 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c0637b0 con 0x7f4b8c15b260 2023-12-26T23:09:24.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.865+0000 7f4b89ffb700 1 -- 172.21.15.131:0/1150462823 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767882549 0 0) 0x7f4b8c067c30 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.866+0000 7f4b8affd700 1 --2- 172.21.15.131:0/1150462823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c15b260 0x7f4b8c0caae0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0099b0 tx=0x7f4b7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ed08f37019e65f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.866+0000 7f4b89ffb700 1 -- 172.21.15.131:0/1150462823 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c009ea0 con 0x7f4b8c15b260 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.866+0000 7f4b89ffb700 1 -- 172.21.15.131:0/1150462823 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c00b8f0 con 0x7f4b8c15b260 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.866+0000 7f4b89ffb700 1 -- 172.21.15.131:0/1150462823 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00bc90 con 0x7f4b8c15b260 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.867+0000 7f4b913ff700 1 -- 172.21.15.131:0/1150462823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c15b260 msgr2=0x7f4b8c0caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.867+0000 7f4b913ff700 1 --2- 172.21.15.131:0/1150462823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c15b260 0x7f4b8c0caae0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0099b0 tx=0x7f4b7c009250 comp rx=0 tx=0).stop 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.867+0000 7f4b913ff700 1 -- 172.21.15.131:0/1150462823 shutdown_connections 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.867+0000 7f4b913ff700 1 --2- 172.21.15.131:0/1150462823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c15b260 0x7f4b8c0caae0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.867+0000 7f4b913ff700 1 --2- 172.21.15.131:0/1150462823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c0aecc0 0x7f4b8c15b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.867+0000 7f4b913ff700 1 -- 172.21.15.131:0/1150462823 >> 172.21.15.131:0/1150462823 conn(0x7f4b8c06c290 msgr2=0x7f4b8c0ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:24.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.867+0000 7f4b913ff700 1 -- 172.21.15.131:0/1150462823 shutdown_connections 2023-12-26T23:09:24.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.867+0000 7f4b913ff700 1 -- 172.21.15.131:0/1150462823 wait complete. 2023-12-26T23:09:24.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.869+0000 7f4b913ff700 1 Processor -- start 2023-12-26T23:09:24.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.869+0000 7f4b913ff700 1 -- start start 2023-12-26T23:09:24.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.870+0000 7f4b913ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8c0aecc0 0x7f4b8c16a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.870+0000 7f4b913ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c0c8e70 0x7f4b8c0c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.870+0000 7f4b913ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c15b260 0x7f4b8c0c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.870+0000 7f4b913ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c16be10 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.870+0000 7f4b913ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c0c26f0 con 0x7f4b8c0aecc0 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.870+0000 7f4b913ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c0c29d0 con 0x7f4b8c15b260 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c0c8e70 0x7f4b8c0c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c0c8e70 0x7f4b8c0c1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42046/0 (socket says 172.21.15.131:42046) 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8a7fc700 1 -- 172.21.15.131:0/1100994579 learned_addr learned my addr 172.21.15.131:0/1100994579 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8affd700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8c0aecc0 0x7f4b8c16a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8b7fe700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c15b260 0x7f4b8c0c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8a7fc700 1 -- 172.21.15.131:0/1100994579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c15b260 msgr2=0x7f4b8c0c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8a7fc700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c15b260 0x7f4b8c0c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8a7fc700 1 -- 172.21.15.131:0/1100994579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8c0aecc0 msgr2=0x7f4b8c16a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8a7fc700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8c0aecc0 0x7f4b8c16a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.871+0000 7f4b8a7fc700 1 -- 172.21.15.131:0/1100994579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c0094f0 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.872+0000 7f4b8a7fc700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c0c8e70 0x7f4b8c0c1b80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000ec80 tx=0x7f4b800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.872+0000 7f4b72ffd700 1 -- 172.21.15.131:0/1100994579 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000c850 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.872+0000 7f4b72ffd700 1 -- 172.21.15.131:0/1100994579 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8000c9b0 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.872+0000 7f4b72ffd700 1 -- 172.21.15.131:0/1100994579 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000cd50 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.872+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c0c2cb0 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.872+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c0c3270 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.873+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c0657e0 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.875+0000 7f4b72ffd700 1 -- 172.21.15.131:0/1100994579 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f4b80013680 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.875+0000 7f4b72ffd700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4b78067050 0x7f4b78069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.875+0000 7f4b72ffd700 1 -- 172.21.15.131:0/1100994579 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f4b80084dc0 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.876+0000 7f4b8affd700 1 -- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4b78067050 msgr2=0x7f4b78069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.876+0000 7f4b8affd700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4b78067050 0x7f4b78069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:24.880+0000 7f4b72ffd700 1 -- 172.21.15.131:0/1100994579 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b80051a40 con 0x7f4b8c0c8e70 2023-12-26T23:09:24.895 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:24.895+0000 7f64e85631c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:09:25.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.077+0000 7f4b8affd700 1 -- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4b78067050 msgr2=0x7f4b78069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:25.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.077+0000 7f4b8affd700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4b78067050 0x7f4b78069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:25.081 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:25.081+0000 7f64e85631c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:09:25.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.124+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b8c0678c0 con 0x7f4b8c0c8e70 2023-12-26T23:09:25.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.125+0000 7f4b72ffd700 1 -- 172.21.15.131:0/1100994579 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f4b80055060 con 0x7f4b8c0c8e70 2023-12-26T23:09:25.126 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:25.126 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2545294570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2545294570},{"type":"v1","addr":"172.21.15.190:6835","nonce":2545294570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1800615106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1800615106},{"type":"v1","addr":"172.21.15.131:6834","nonce":1800615106}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3279484094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3279484094},{"type":"v1","addr":"172.21.15.131:6835","nonce":3279484094}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T22:53:21.655334+0000","modified":"2023-12-26T22:53:25.218600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9105":{"gid":9105,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/716462328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":716462328},{"type":"v1","addr":"172.21.15.190:6834","nonce":716462328}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-12-26T23:09:25.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.127+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4b78067050 msgr2=0x7f4b78069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:25.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.127+0000 7f4b913ff700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4b78067050 0x7f4b78069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.127+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c0c8e70 msgr2=0x7f4b8c0c1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.127+0000 7f4b913ff700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c0c8e70 0x7f4b8c0c1b80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000ec80 tx=0x7f4b800091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.128+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 shutdown_connections 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.128+0000 7f4b913ff700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4b78067050 0x7f4b78069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.128+0000 7f4b913ff700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b8c15b260 0x7f4b8c0c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.128+0000 7f4b913ff700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b8c0aecc0 0x7f4b8c16a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.128+0000 7f4b913ff700 1 --2- 172.21.15.131:0/1100994579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b8c0c8e70 0x7f4b8c0c1b80 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.128+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 >> 172.21.15.131:0/1100994579 conn(0x7f4b8c06c290 msgr2=0x7f4b8c154690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.128+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 shutdown_connections 2023-12-26T23:09:25.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.128+0000 7f4b913ff700 1 -- 172.21.15.131:0/1100994579 wait complete. 2023-12-26T23:09:25.130 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 81 2023-12-26T23:09:25.146 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:09:25.219 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:25.219+0000 7f2d11f141c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:09:25.271 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:25.271+0000 7f64e85631c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:09:25.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.338+0000 7fd9cbfff700 1 Processor -- start 2023-12-26T23:09:25.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.339+0000 7fd9cbfff700 1 -- start start 2023-12-26T23:09:25.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.340+0000 7fd9cbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40b19f0 0x7fd9c41620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.340+0000 7fd9cbfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c40caa10 con 0x7fd9c40c8bf0 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.340+0000 7fd9cbfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c4157a80 con 0x7fd9c40c8820 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.340+0000 7fd9cbfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c40cac50 con 0x7fd9c40b19f0 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.340+0000 7fd9c959a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd9c40c8bf0 0x7fd9c40b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49320/0 (socket says 172.21.15.131:49320) 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.340+0000 7fd9c959a700 1 -- 172.21.15.131:0/1609187577 learned_addr learned my addr 172.21.15.131:0/1609187577 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.340+0000 7fd9ca59c700 1 --2- 172.21.15.131:0/1609187577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40b19f0 0x7fd9c41620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.340+0000 7fd9c8d99700 1 -- 172.21.15.131:0/1609187577 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020660946 0 0) 0x7fd9b4001110 con 0x7fd9c40c8bf0 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.340+0000 7fd9c8d99700 1 -- 172.21.15.131:0/1609187577 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b0003570 con 0x7fd9c40c8bf0 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9c8d99700 1 -- 172.21.15.131:0/1609187577 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761319440 0 0) 0x7fd9c4157a80 con 0x7fd9c40c8820 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9c8d99700 1 -- 172.21.15.131:0/1609187577 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b4001110 con 0x7fd9c40c8820 2023-12-26T23:09:25.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9ca59c700 1 -- 172.21.15.131:0/1609187577 >> v1:172.21.15.190:6789/0 conn(0x7fd9c40c8820 legacy=0x7fd9c40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9ca59c700 1 -- 172.21.15.131:0/1609187577 >> v1:172.21.15.131:6789/0 conn(0x7fd9c40c8bf0 legacy=0x7fd9c40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9c8d99700 1 -- 172.21.15.131:0/1609187577 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2976720243 0 0) 0x7fd9b0003570 con 0x7fd9c40c8bf0 2023-12-26T23:09:25.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9ca59c700 1 -- 172.21.15.131:0/1609187577 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c40caf30 con 0x7fd9c40b19f0 2023-12-26T23:09:25.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9ca59c700 1 --2- 172.21.15.131:0/1609187577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40b19f0 0x7fd9c41620a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009e90 tx=0x7fd9bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5384cfeb1acab732 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:25.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9c8d99700 1 -- 172.21.15.131:0/1609187577 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00e070 con 0x7fd9c40b19f0 2023-12-26T23:09:25.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9c8d99700 1 -- 172.21.15.131:0/1609187577 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc00b870 con 0x7fd9c40b19f0 2023-12-26T23:09:25.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.341+0000 7fd9c8d99700 1 -- 172.21.15.131:0/1609187577 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4001600 con 0x7fd9c40b19f0 2023-12-26T23:09:25.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.342+0000 7fd9cbfff700 1 -- 172.21.15.131:0/1609187577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40b19f0 msgr2=0x7fd9c41620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.342+0000 7fd9cbfff700 1 --2- 172.21.15.131:0/1609187577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40b19f0 0x7fd9c41620a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009e90 tx=0x7fd9bc009780 comp rx=0 tx=0).stop 2023-12-26T23:09:25.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.342+0000 7fd9cbfff700 1 -- 172.21.15.131:0/1609187577 shutdown_connections 2023-12-26T23:09:25.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.342+0000 7fd9cbfff700 1 --2- 172.21.15.131:0/1609187577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40b19f0 0x7fd9c41620a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.342+0000 7fd9cbfff700 1 -- 172.21.15.131:0/1609187577 >> 172.21.15.131:0/1609187577 conn(0x7fd9c4068040 msgr2=0x7fd9c4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:25.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.342+0000 7fd9cbfff700 1 -- 172.21.15.131:0/1609187577 shutdown_connections 2023-12-26T23:09:25.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.342+0000 7fd9cbfff700 1 -- 172.21.15.131:0/1609187577 wait complete. 2023-12-26T23:09:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.344+0000 7fd9cbfff700 1 Processor -- start 2023-12-26T23:09:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.344+0000 7fd9cbfff700 1 -- start start 2023-12-26T23:09:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.344+0000 7fd9cbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9c40b19f0 0x7fd9c4145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9cbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c40c8820 0x7fd9c415a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9cbfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40c8bf0 0x7fd9c4150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9cbfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c41596a0 con 0x7fd9c40c8820 2023-12-26T23:09:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9cbfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c41587f0 con 0x7fd9c40b19f0 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9cbfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c4158a90 con 0x7fd9c40c8bf0 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c40c8820 0x7fd9c415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c40c8820 0x7fd9c415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42056/0 (socket says 172.21.15.131:42056) 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c959a700 1 -- 172.21.15.131:0/75755967 learned_addr learned my addr 172.21.15.131:0/75755967 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c9d9b700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9c40b19f0 0x7fd9c4145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9ca59c700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40c8bf0 0x7fd9c4150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c959a700 1 -- 172.21.15.131:0/75755967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40c8bf0 msgr2=0x7fd9c4150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c959a700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40c8bf0 0x7fd9c4150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c959a700 1 -- 172.21.15.131:0/75755967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9c40b19f0 msgr2=0x7fd9c4145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c959a700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9c40b19f0 0x7fd9c4145e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c959a700 1 -- 172.21.15.131:0/75755967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc009a20 con 0x7fd9c40c8820 2023-12-26T23:09:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.345+0000 7fd9c9d9b700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9c40b19f0 0x7fd9c4145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.346+0000 7fd9c959a700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c40c8820 0x7fd9c415a090 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400cb60 tx=0x7fd9b4001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.346+0000 7fd9b9ffb700 1 -- 172.21.15.131:0/75755967 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400ce70 con 0x7fd9c40c8820 2023-12-26T23:09:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.346+0000 7fd9b9ffb700 1 -- 172.21.15.131:0/75755967 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b4005b30 con 0x7fd9c40c8820 2023-12-26T23:09:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.346+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c4158d70 con 0x7fd9c40c8820 2023-12-26T23:09:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.346+0000 7fd9b9ffb700 1 -- 172.21.15.131:0/75755967 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40070f0 con 0x7fd9c40c8820 2023-12-26T23:09:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.346+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c4159330 con 0x7fd9c40c8820 2023-12-26T23:09:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.347+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c40657e0 con 0x7fd9c40c8820 2023-12-26T23:09:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.348+0000 7fd9b9ffb700 1 -- 172.21.15.131:0/75755967 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fd9b40074e0 con 0x7fd9c40c8820 2023-12-26T23:09:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.349+0000 7fd9b9ffb700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd9a8067050 0x7fd9a8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.349+0000 7fd9b9ffb700 1 -- 172.21.15.131:0/75755967 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd9b4084720 con 0x7fd9c40c8820 2023-12-26T23:09:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.349+0000 7fd9c9d9b700 1 -- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd9a8067050 msgr2=0x7fd9a8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.349+0000 7fd9c9d9b700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd9a8067050 0x7fd9a8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:25.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.353+0000 7fd9b9ffb700 1 -- 172.21.15.131:0/75755967 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9b40514c0 con 0x7fd9c40c8820 2023-12-26T23:09:25.419 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:25.420+0000 7f2d11f141c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:09:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.550+0000 7fd9c9d9b700 1 -- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd9a8067050 msgr2=0x7fd9a8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.550+0000 7fd9c9d9b700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd9a8067050 0x7fd9a8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.553+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9c40caa10 con 0x7fd9c40c8820 2023-12-26T23:09:25.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.554+0000 7fd9b9ffb700 1 -- 172.21.15.131:0/75755967 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+17083 (secure 0 0 0) 0x7fd9c40caa10 con 0x7fd9c40c8820 2023-12-26T23:09:25.555 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:25.555 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":180,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:09:22.463161+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-26T22:53:19.805015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-26T22:53:20.636358+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":180,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.5","mappings":[{"from":2,"to":5}]},{"pgid":"21.3f","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/109159886":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/1073482886":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/659813654":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/3572226702":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/3168215713":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/2626934136":"2023-12-27T22:53:37.495457+0000","172.21.15.131:6836/39288":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/892276341":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/470407650":"2023-12-27T23:04:23.466126+0000","172.21.15.131:0/1083345344":"2023-12-27T23:04:23.466126+0000","172.21.15.131:6837/39288":"2023-12-27T22:53:37.495457+0000","172.21.15.131:0/1605877564":"2023-12-27T22:53:37.495457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:09:25.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd9a8067050 msgr2=0x7fd9a8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd9a8067050 0x7fd9a8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c40c8820 msgr2=0x7fd9c415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c40c8820 0x7fd9c415a090 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400cb60 tx=0x7fd9b4001b60 comp rx=0 tx=0).stop 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 shutdown_connections 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd9a8067050 0x7fd9a8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9c40c8bf0 0x7fd9c4150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9c40b19f0 0x7fd9c4145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 --2- 172.21.15.131:0/75755967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9c40c8820 0x7fd9c415a090 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 >> 172.21.15.131:0/75755967 conn(0x7fd9c4068040 msgr2=0x7fd9c414e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:25.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 shutdown_connections 2023-12-26T23:09:25.559 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.557+0000 7fd9cbfff700 1 -- 172.21.15.131:0/75755967 wait complete. 2023-12-26T23:09:25.576 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T23:09:25.618 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:25.618+0000 7f2d11f141c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:09:25.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.771+0000 7f12307ff700 1 Processor -- start 2023-12-26T23:09:25.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.771+0000 7f12307ff700 1 -- start start 2023-12-26T23:09:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.772+0000 7f12307ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f122815b260 0x7f12280caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.772+0000 7f12307ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1228067c30 con 0x7f12280b2360 2023-12-26T23:09:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.772+0000 7f12307ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1228067650 con 0x7f12280aecc0 2023-12-26T23:09:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.772+0000 7f12307ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122815bcd0 con 0x7f122815b260 2023-12-26T23:09:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.772+0000 7f122ed9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f12280b2360 0x7f12280680c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49324/0 (socket says 172.21.15.131:49324) 2023-12-26T23:09:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.772+0000 7f122e59b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f122815b260 0x7f12280caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.772+0000 7f122ed9c700 1 -- 172.21.15.131:0/1869596624 learned_addr learned my addr 172.21.15.131:0/1869596624 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.773+0000 7f122e59b700 1 -- 172.21.15.131:0/1869596624 >> v1:172.21.15.190:6789/0 conn(0x7f12280aecc0 legacy=0x7f12280b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.773+0000 7f122e59b700 1 -- 172.21.15.131:0/1869596624 >> v1:172.21.15.131:6789/0 conn(0x7f12280b2360 legacy=0x7f12280680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.773+0000 7f122e59b700 1 -- 172.21.15.131:0/1869596624 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12280637b0 con 0x7f122815b260 2023-12-26T23:09:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.773+0000 7f122e59b700 1 --2- 172.21.15.131:0/1869596624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 0x7f12280caae0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f12240098e0 tx=0x7f12240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f98e55ddf92cfd4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.773+0000 7f122d599700 1 -- 172.21.15.131:0/1869596624 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224009e80 con 0x7f122815b260 2023-12-26T23:09:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.773+0000 7f122d599700 1 -- 172.21.15.131:0/1869596624 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122400b8f0 con 0x7f122815b260 2023-12-26T23:09:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.773+0000 7f122d599700 1 -- 172.21.15.131:0/1869596624 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400bc70 con 0x7f122815b260 2023-12-26T23:09:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.774+0000 7f12307ff700 1 -- 172.21.15.131:0/1869596624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 msgr2=0x7f12280caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.774+0000 7f12307ff700 1 --2- 172.21.15.131:0/1869596624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 0x7f12280caae0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f12240098e0 tx=0x7f12240091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.774+0000 7f12307ff700 1 -- 172.21.15.131:0/1869596624 shutdown_connections 2023-12-26T23:09:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.774+0000 7f12307ff700 1 --2- 172.21.15.131:0/1869596624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 0x7f12280caae0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.774+0000 7f12307ff700 1 -- 172.21.15.131:0/1869596624 >> 172.21.15.131:0/1869596624 conn(0x7f122806c290 msgr2=0x7f12280ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.774+0000 7f12307ff700 1 -- 172.21.15.131:0/1869596624 shutdown_connections 2023-12-26T23:09:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.774+0000 7f12307ff700 1 -- 172.21.15.131:0/1869596624 wait complete. 2023-12-26T23:09:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.776+0000 7f12307ff700 1 Processor -- start 2023-12-26T23:09:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.776+0000 7f12307ff700 1 -- start start 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f12307ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12280aecc0 0x7f12280c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f12307ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12280b2360 0x7f12281689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f12307ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 0x7f12281545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f12307ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12280c7810 con 0x7f122815b260 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f12307ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228168f00 con 0x7f12280b2360 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f12307ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1228154ae0 con 0x7f12280aecc0 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122e59b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12280aecc0 0x7f12280c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122ed9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 0x7f12281545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122e59b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12280aecc0 0x7f12280c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:51294/0 (socket says 172.21.15.131:51294) 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122e59b700 1 -- 172.21.15.131:0/2114963762 learned_addr learned my addr 172.21.15.131:0/2114963762 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122dd9a700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12280b2360 0x7f12281689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122ed9c700 1 -- 172.21.15.131:0/2114963762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12280aecc0 msgr2=0x7f12280c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122ed9c700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12280aecc0 0x7f12280c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122ed9c700 1 -- 172.21.15.131:0/2114963762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12280b2360 msgr2=0x7f12281689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122ed9c700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12280b2360 0x7f12281689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122ed9c700 1 -- 172.21.15.131:0/2114963762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1224009420 con 0x7f122815b260 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.777+0000 7f122e59b700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12280aecc0 0x7f12280c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.778+0000 7f122ed9c700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 0x7f12281545a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f122000cb90 tx=0x7f12200019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.778+0000 7f121e7fc700 1 -- 172.21.15.131:0/2114963762 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122000a7e0 con 0x7f122815b260 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.778+0000 7f121e7fc700 1 -- 172.21.15.131:0/2114963762 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122000a980 con 0x7f122815b260 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.778+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12280c6c00 con 0x7f122815b260 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.778+0000 7f121e7fc700 1 -- 172.21.15.131:0/2114963762 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122000ad40 con 0x7f122815b260 2023-12-26T23:09:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.778+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12280c71c0 con 0x7f122815b260 2023-12-26T23:09:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.779+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12280c7980 con 0x7f122815b260 2023-12-26T23:09:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.780+0000 7f121e7fc700 1 -- 172.21.15.131:0/2114963762 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f1220007300 con 0x7f122815b260 2023-12-26T23:09:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.781+0000 7f121e7fc700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f120c067050 0x7f120c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.781+0000 7f121e7fc700 1 -- 172.21.15.131:0/2114963762 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f1220084cf0 con 0x7f122815b260 2023-12-26T23:09:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.781+0000 7f122e59b700 1 -- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f120c067050 msgr2=0x7f120c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.781+0000 7f122e59b700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f120c067050 0x7f120c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.785+0000 7f121e7fc700 1 -- 172.21.15.131:0/2114963762 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1220089070 con 0x7f122815b260 2023-12-26T23:09:25.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.982+0000 7f122e59b700 1 -- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f120c067050 msgr2=0x7f120c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:25.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:25.982+0000 7f122e59b700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f120c067050 0x7f120c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:26.025 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.025+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f12280657e0 con 0x7f122815b260 2023-12-26T23:09:26.026 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T23:09:26.026+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T23:09:26.030 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.030+0000 7f121e7fc700 1 -- 172.21.15.131:0/2114963762 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1220051a90 con 0x7f122815b260 2023-12-26T23:09:26.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f120c067050 msgr2=0x7f120c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:26.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f120c067050 0x7f120c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 msgr2=0x7f12281545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 0x7f12281545a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f122000cb90 tx=0x7f12200019a0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 shutdown_connections 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f120c067050 0x7f120c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f122815b260 0x7f12281545a0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12280aecc0 0x7f12280c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 --2- 172.21.15.131:0/2114963762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12280b2360 0x7f12281689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 >> 172.21.15.131:0/2114963762 conn(0x7f122806c290 msgr2=0x7f122806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 shutdown_connections 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.033+0000 7f12307ff700 1 -- 172.21.15.131:0/2114963762 wait complete. 2023-12-26T23:09:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T23:09:26.053 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T23:09:26.091 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T23:09:26.127 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:26.127+0000 7f64e85631c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:09:26.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.286+0000 7fc05ddff700 1 Processor -- start 2023-12-26T23:09:26.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.287+0000 7fc05ddff700 1 -- start start 2023-12-26T23:09:26.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.288+0000 7fc05ddff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc0580cb320 0x7fc058069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:26.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.288+0000 7fc05ddff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc05806a1e0 0x7fc0580b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:26.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.288+0000 7fc05ddff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc058156940 con 0x7fc0580b0c90 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.288+0000 7fc05ddff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0580c8cc0 con 0x7fc05806a1e0 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.288+0000 7fc05ddff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0580ad700 con 0x7fc0580cb320 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.288+0000 7fc057fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc0580b0c90 0x7fc0580aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49326/0 (socket says 172.21.15.131:49326) 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.288+0000 7fc057fff700 1 -- 172.21.15.131:0/3342462649 learned_addr learned my addr 172.21.15.131:0/3342462649 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.288+0000 7fc056ffd700 1 --2- 172.21.15.131:0/3342462649 >> v2:172.21.15.131:3300/0 conn(0x7fc05806a1e0 0x7fc0580b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.288+0000 7fc0577fe700 1 --2- 172.21.15.131:0/3342462649 >> v2:172.21.15.190:3300/0 conn(0x7fc0580cb320 0x7fc058069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.289+0000 7fc056ffd700 1 -- 172.21.15.131:0/3342462649 >> v2:172.21.15.190:3300/0 conn(0x7fc0580cb320 msgr2=0x7fc058069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.289+0000 7fc056ffd700 1 --2- 172.21.15.131:0/3342462649 >> v2:172.21.15.190:3300/0 conn(0x7fc0580cb320 0x7fc058069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.289+0000 7fc056ffd700 1 -- 172.21.15.131:0/3342462649 >> v1:172.21.15.131:6789/0 conn(0x7fc0580b0c90 legacy=0x7fc0580aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.289+0000 7fc056ffd700 1 -- 172.21.15.131:0/3342462649 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0580ae500 con 0x7fc05806a1e0 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.289+0000 7fc056ffd700 1 --2- 172.21.15.131:0/3342462649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 0x7fc0580b0750 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc058064410 tx=0x7fc04c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332e0bd55b5e6403 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.289+0000 7fc0567fc700 1 -- 172.21.15.131:0/3342462649 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c011630 con 0x7fc05806a1e0 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.289+0000 7fc0567fc700 1 -- 172.21.15.131:0/3342462649 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04c011790 con 0x7fc05806a1e0 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.289+0000 7fc0567fc700 1 -- 172.21.15.131:0/3342462649 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c011b30 con 0x7fc05806a1e0 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.290+0000 7fc05ddff700 1 -- 172.21.15.131:0/3342462649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 msgr2=0x7fc0580b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.290+0000 7fc05ddff700 1 --2- 172.21.15.131:0/3342462649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 0x7fc0580b0750 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc058064410 tx=0x7fc04c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.290+0000 7fc05ddff700 1 -- 172.21.15.131:0/3342462649 shutdown_connections 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.290+0000 7fc05ddff700 1 --2- 172.21.15.131:0/3342462649 >> v2:172.21.15.190:3300/0 conn(0x7fc0580cb320 0x7fc058069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.290+0000 7fc05ddff700 1 --2- 172.21.15.131:0/3342462649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 0x7fc0580b0750 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.290+0000 7fc05ddff700 1 -- 172.21.15.131:0/3342462649 >> 172.21.15.131:0/3342462649 conn(0x7fc058068040 msgr2=0x7fc058069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:26.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.290+0000 7fc05ddff700 1 -- 172.21.15.131:0/3342462649 shutdown_connections 2023-12-26T23:09:26.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.290+0000 7fc05ddff700 1 -- 172.21.15.131:0/3342462649 wait complete. 2023-12-26T23:09:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.291+0000 7fc05ddff700 1 Processor -- start 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.292+0000 7fc05ddff700 1 -- start start 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.292+0000 7fc05ddff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 0x7fc058155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.292+0000 7fc05ddff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580b0c90 0x7fc05816a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc05ddff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb320 0x7fc05816a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc05ddff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05816ab60 con 0x7fc05806a1e0 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc05ddff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05816ad20 con 0x7fc0580b0c90 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc05ddff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05816af90 con 0x7fc0580cb320 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc0577fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 0x7fc058155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc0577fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 0x7fc058155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42094/0 (socket says 172.21.15.131:42094) 2023-12-26T23:09:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc0577fe700 1 -- 172.21.15.131:0/1621877129 learned_addr learned my addr 172.21.15.131:0/1621877129 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:26.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc057fff700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb320 0x7fc05816a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:26.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc056ffd700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580b0c90 0x7fc05816a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:26.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc0577fe700 1 -- 172.21.15.131:0/1621877129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb320 msgr2=0x7fc05816a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:26.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc0577fe700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb320 0x7fc05816a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc0577fe700 1 -- 172.21.15.131:0/1621877129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580b0c90 msgr2=0x7fc05816a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:26.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc0577fe700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580b0c90 0x7fc05816a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:26.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc0577fe700 1 -- 172.21.15.131:0/1621877129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c009420 con 0x7fc05806a1e0 2023-12-26T23:09:26.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc056ffd700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580b0c90 0x7fc05816a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:26.297 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.293+0000 7fc0577fe700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 0x7fc058155ba0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e960 tx=0x7fc0480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:26.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.294+0000 7fc03f7fe700 1 -- 172.21.15.131:0/1621877129 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800c850 con 0x7fc05806a1e0 2023-12-26T23:09:26.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.294+0000 7fc03f7fe700 1 -- 172.21.15.131:0/1621877129 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04800c9b0 con 0x7fc05806a1e0 2023-12-26T23:09:26.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.294+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05816b270 con 0x7fc05806a1e0 2023-12-26T23:09:26.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.294+0000 7fc03f7fe700 1 -- 172.21.15.131:0/1621877129 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800cd50 con 0x7fc05806a1e0 2023-12-26T23:09:26.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.294+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05816b830 con 0x7fc05806a1e0 2023-12-26T23:09:26.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.295+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0580657e0 con 0x7fc05806a1e0 2023-12-26T23:09:26.298 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.296+0000 7fc03f7fe700 1 -- 172.21.15.131:0/1621877129 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 93) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fc048013680 con 0x7fc05806a1e0 2023-12-26T23:09:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.297+0000 7fc03f7fe700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 0x7fc044069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.297+0000 7fc03f7fe700 1 -- 172.21.15.131:0/1621877129 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fc048084d60 con 0x7fc05806a1e0 2023-12-26T23:09:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.297+0000 7fc056ffd700 1 -- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 msgr2=0x7fc044069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.297+0000 7fc056ffd700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 0x7fc044069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:26.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.301+0000 7fc03f7fe700 1 -- 172.21.15.131:0/1621877129 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc048051a40 con 0x7fc05806a1e0 2023-12-26T23:09:26.404 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:26.404+0000 7f64e85631c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:09:26.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.498+0000 7fc056ffd700 1 -- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 msgr2=0x7fc044069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:26.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.498+0000 7fc056ffd700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 0x7fc044069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:26.499 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:26.499+0000 7f64e85631c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:09:26.505 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:26.505+0000 7f2d11f141c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:09:26.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.548+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc05816bba0 con 0x7fc05806a1e0 2023-12-26T23:09:26.696 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:26.697+0000 7f64e85631c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:09:26.784 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:26.785+0000 7f2d11f141c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:09:26.879 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:26.879+0000 7f2d11f141c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:09:26.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.899+0000 7fc056ffd700 1 -- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 msgr2=0x7fc044069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:26.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:26.899+0000 7fc056ffd700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 0x7fc044069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T23:09:27.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.031+0000 7fc03f7fe700 1 -- 172.21.15.131:0/1621877129 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc048055060 con 0x7fc05806a1e0 2023-12-26T23:09:27.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 msgr2=0x7fc044069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:27.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 0x7fc044069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 msgr2=0x7fc058155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:27.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 0x7fc058155ba0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e960 tx=0x7fc0480091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 shutdown_connections 2023-12-26T23:09:27.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc044067050 0x7fc044069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0580cb320 0x7fc05816a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc05806a1e0 0x7fc058155ba0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 --2- 172.21.15.131:0/1621877129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0580b0c90 0x7fc05816a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 >> 172.21.15.131:0/1621877129 conn(0x7fc058068040 msgr2=0x7fc0580abc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:27.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 shutdown_connections 2023-12-26T23:09:27.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.034+0000 7fc05ddff700 1 -- 172.21.15.131:0/1621877129 wait complete. 2023-12-26T23:09:27.053 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T23:09:27.083 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:27.083+0000 7f2d11f141c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:09:27.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.269+0000 7ff924dff700 1 Processor -- start 2023-12-26T23:09:27.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.270+0000 7ff924dff700 1 -- start start 2023-12-26T23:09:27.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.271+0000 7ff924dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9200cb510 0x7ff920069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.271+0000 7ff924dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff920069640 con 0x7ff920142f00 2023-12-26T23:09:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.271+0000 7ff924dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff920068750 con 0x7ff9200cc590 2023-12-26T23:09:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.271+0000 7ff924dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff920068a30 con 0x7ff9200cb510 2023-12-26T23:09:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.271+0000 7ff91e59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff920142f00 0x7ff9200cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49330/0 (socket says 172.21.15.131:49330) 2023-12-26T23:09:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.271+0000 7ff91e59c700 1 -- 172.21.15.131:0/4146331133 learned_addr learned my addr 172.21.15.131:0/4146331133 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.271+0000 7ff91dd9b700 1 --2- 172.21.15.131:0/4146331133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9200cb510 0x7ff920069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.271+0000 7ff91d59a700 1 -- 172.21.15.131:0/4146331133 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441171236 0 0) 0x7ff920069640 con 0x7ff920142f00 2023-12-26T23:09:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.272+0000 7ff91d59a700 1 -- 172.21.15.131:0/4146331133 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff908003570 con 0x7ff920142f00 2023-12-26T23:09:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.272+0000 7ff91d59a700 1 -- 172.21.15.131:0/4146331133 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853069969 0 0) 0x7ff920068750 con 0x7ff9200cc590 2023-12-26T23:09:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.272+0000 7ff91dd9b700 1 -- 172.21.15.131:0/4146331133 >> v1:172.21.15.190:6789/0 conn(0x7ff9200cc590 legacy=0x7ff9200c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.272+0000 7ff91dd9b700 1 -- 172.21.15.131:0/4146331133 >> v1:172.21.15.131:6789/0 conn(0x7ff920142f00 legacy=0x7ff9200cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.272+0000 7ff91dd9b700 1 -- 172.21.15.131:0/4146331133 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920068d10 con 0x7ff9200cb510 2023-12-26T23:09:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.272+0000 7ff91dd9b700 1 --2- 172.21.15.131:0/4146331133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9200cb510 0x7ff920069ad0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff914009910 tx=0x7ff9140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95aba04916a4152b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.272+0000 7ff91d59a700 1 -- 172.21.15.131:0/4146331133 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914011630 con 0x7ff9200cb510 2023-12-26T23:09:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.272+0000 7ff91d59a700 1 -- 172.21.15.131:0/4146331133 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff914011790 con 0x7ff9200cb510 2023-12-26T23:09:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.272+0000 7ff91d59a700 1 -- 172.21.15.131:0/4146331133 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914011b50 con 0x7ff9200cb510 2023-12-26T23:09:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.273+0000 7ff924dff700 1 -- 172.21.15.131:0/4146331133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9200cb510 msgr2=0x7ff920069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.273+0000 7ff924dff700 1 --2- 172.21.15.131:0/4146331133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9200cb510 0x7ff920069ad0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff914009910 tx=0x7ff9140091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.273+0000 7ff924dff700 1 -- 172.21.15.131:0/4146331133 shutdown_connections 2023-12-26T23:09:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.273+0000 7ff924dff700 1 --2- 172.21.15.131:0/4146331133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9200cb510 0x7ff920069ad0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.273+0000 7ff924dff700 1 -- 172.21.15.131:0/4146331133 >> 172.21.15.131:0/4146331133 conn(0x7ff920063cd0 msgr2=0x7ff92006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.273+0000 7ff924dff700 1 -- 172.21.15.131:0/4146331133 shutdown_connections 2023-12-26T23:09:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.273+0000 7ff924dff700 1 -- 172.21.15.131:0/4146331133 wait complete. 2023-12-26T23:09:27.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.274+0000 7ff924dff700 1 Processor -- start 2023-12-26T23:09:27.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.275+0000 7ff924dff700 1 -- start start 2023-12-26T23:09:27.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.275+0000 7ff924dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200cb510 0x7ff92013fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:27.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.275+0000 7ff924dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9200cc590 0x7ff92016a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:27.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.275+0000 7ff924dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920142f00 0x7ff9200c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:27.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.275+0000 7ff924dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92016bf20 con 0x7ff9200cc590 2023-12-26T23:09:27.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.275+0000 7ff924dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92016b310 con 0x7ff9200cb510 2023-12-26T23:09:27.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.275+0000 7ff924dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92016b5f0 con 0x7ff920142f00 2023-12-26T23:09:27.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9200cc590 0x7ff92016a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:27.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9200cc590 0x7ff92016a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42198/0 (socket says 172.21.15.131:42198) 2023-12-26T23:09:27.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91dd9b700 1 -- 172.21.15.131:0/1129958403 learned_addr learned my addr 172.21.15.131:0/1129958403 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:27.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91e59c700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200cb510 0x7ff92013fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:27.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91ed9d700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920142f00 0x7ff9200c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:27.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91dd9b700 1 -- 172.21.15.131:0/1129958403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920142f00 msgr2=0x7ff9200c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:27.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91dd9b700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920142f00 0x7ff9200c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91dd9b700 1 -- 172.21.15.131:0/1129958403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200cb510 msgr2=0x7ff92013fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91dd9b700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200cb510 0x7ff92013fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91dd9b700 1 -- 172.21.15.131:0/1129958403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff914009420 con 0x7ff9200cc590 2023-12-26T23:09:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.276+0000 7ff91dd9b700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9200cc590 0x7ff92016a850 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7ff92006f3d0 tx=0x7ff914016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.277+0000 7ff90e7fc700 1 -- 172.21.15.131:0/1129958403 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914000d10 con 0x7ff9200cc590 2023-12-26T23:09:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.277+0000 7ff90e7fc700 1 -- 172.21.15.131:0/1129958403 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff914011b10 con 0x7ff9200cc590 2023-12-26T23:09:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.277+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92016b8d0 con 0x7ff9200cc590 2023-12-26T23:09:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.277+0000 7ff90e7fc700 1 -- 172.21.15.131:0/1129958403 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91401a6c0 con 0x7ff9200cc590 2023-12-26T23:09:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.277+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff920169b60 con 0x7ff9200cc590 2023-12-26T23:09:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.278+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92016c090 con 0x7ff9200cc590 2023-12-26T23:09:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.279+0000 7ff90e7fc700 1 -- 172.21.15.131:0/1129958403 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 94) v1 ==== 84453+0+0 (secure 0 0 0) 0x7ff91401aa30 con 0x7ff9200cc590 2023-12-26T23:09:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.279+0000 7ff90e7fc700 1 -- 172.21.15.131:0/1129958403 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 7201+0+0 (secure 0 0 0) 0x7ff914086240 con 0x7ff9200cc590 2023-12-26T23:09:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.284+0000 7ff90e7fc700 1 -- 172.21.15.131:0/1129958403 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9140531a0 con 0x7ff9200cc590 2023-12-26T23:09:27.476 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:27.476+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff9200717f0 con 0x7ff9200cc590 2023-12-26T23:09:28.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.096+0000 7ff90e7fc700 1 -- 172.21.15.131:0/1129958403 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 95) v1 ==== 84580+0+0 (secure 0 0 0) 0x7ff91401ace0 con 0x7ff9200cc590 2023-12-26T23:09:28.096 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.096+0000 7ff90e7fc700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff90006cd90 0x7ff90006f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:28.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.098+0000 7ff91e59c700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff90006cd90 0x7ff90006f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.102+0000 7ff91e59c700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff90006cd90 0x7ff90006f170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff910007fd0 tx=0x7ff910000bc0 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:28.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.492+0000 7ff90e7fc700 1 -- 172.21.15.131:0/1129958403 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v183) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff914024410 con 0x7ff9200cc590 2023-12-26T23:09:28.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.494+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff90006cd90 msgr2=0x7ff90006f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.494+0000 7ff924dff700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff90006cd90 0x7ff90006f170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff910007fd0 tx=0x7ff910000bc0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9200cc590 msgr2=0x7ff92016a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9200cc590 0x7ff92016a850 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7ff92006f3d0 tx=0x7ff914016000 comp rx=0 tx=0).stop 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 shutdown_connections 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff90006cd90 0x7ff90006f170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff920142f00 0x7ff9200c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200cb510 0x7ff92013fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 --2- 172.21.15.131:0/1129958403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9200cc590 0x7ff92016a850 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 >> 172.21.15.131:0/1129958403 conn(0x7ff920063cd0 msgr2=0x7ff92015bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:28.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 shutdown_connections 2023-12-26T23:09:28.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.495+0000 7ff924dff700 1 -- 172.21.15.131:0/1129958403 wait complete. 2023-12-26T23:09:28.498 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T23:09:28.516 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T23:09:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.750+0000 7ffad69ff700 1 Processor -- start 2023-12-26T23:09:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.750+0000 7ffad69ff700 1 -- start start 2023-12-26T23:09:28.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffad69ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad0142f00 0x7ffad0069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:28.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffad69ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffad00cc590 0x7ffad00cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:28.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffad69ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffad00cb340 0x7ffad00c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:28.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffad69ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad00cb7c0 con 0x7ffad00cc590 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffad69ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad0069640 con 0x7ffad00cb340 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffad69ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad0068750 con 0x7ffad0142f00 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffacf7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffad00cc590 0x7ffad00cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffacf7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffad00cc590 0x7ffad00cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42208/0 (socket says 172.21.15.131:42208) 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffacf7fe700 1 -- 172.21.15.131:0/341257935 learned_addr learned my addr 172.21.15.131:0/341257935 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffad4f9c700 1 --2- 172.21.15.131:0/341257935 >> v2:172.21.15.190:3300/0 conn(0x7ffad00cb340 0x7ffad00c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.752+0000 7ffacffff700 1 --2- 172.21.15.131:0/341257935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad0142f00 0x7ffad0069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.753+0000 7ffacf7fe700 1 -- 172.21.15.131:0/341257935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad0142f00 msgr2=0x7ffad0069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.753+0000 7ffacf7fe700 1 --2- 172.21.15.131:0/341257935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad0142f00 0x7ffad0069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.753+0000 7ffacf7fe700 1 -- 172.21.15.131:0/341257935 >> v2:172.21.15.190:3300/0 conn(0x7ffad00cb340 msgr2=0x7ffad00c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.753+0000 7ffacf7fe700 1 --2- 172.21.15.131:0/341257935 >> v2:172.21.15.190:3300/0 conn(0x7ffad00cb340 0x7ffad00c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.753+0000 7ffacf7fe700 1 -- 172.21.15.131:0/341257935 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad0068a30 con 0x7ffad00cc590 2023-12-26T23:09:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.753+0000 7ffacf7fe700 1 --2- 172.21.15.131:0/341257935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cc590 0x7ffad00cae00 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7ffad006f560 tx=0x7ffac4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1794b8c2eb898b35 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.753+0000 7ffaceffd700 1 -- 172.21.15.131:0/341257935 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400b640 con 0x7ffad00cc590 2023-12-26T23:09:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.753+0000 7ffaceffd700 1 -- 172.21.15.131:0/341257935 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac400b7a0 con 0x7ffad00cc590 2023-12-26T23:09:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.753+0000 7ffaceffd700 1 -- 172.21.15.131:0/341257935 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400bb60 con 0x7ffad00cc590 2023-12-26T23:09:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.754+0000 7ffad69ff700 1 -- 172.21.15.131:0/341257935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cc590 msgr2=0x7ffad00cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.754+0000 7ffad69ff700 1 --2- 172.21.15.131:0/341257935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cc590 0x7ffad00cae00 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7ffad006f560 tx=0x7ffac4009250 comp rx=0 tx=0).stop 2023-12-26T23:09:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.754+0000 7ffad69ff700 1 -- 172.21.15.131:0/341257935 shutdown_connections 2023-12-26T23:09:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.754+0000 7ffad69ff700 1 --2- 172.21.15.131:0/341257935 >> v2:172.21.15.190:3300/0 conn(0x7ffad00cb340 0x7ffad00c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.754+0000 7ffad69ff700 1 --2- 172.21.15.131:0/341257935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad0142f00 0x7ffad0069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.754+0000 7ffad69ff700 1 --2- 172.21.15.131:0/341257935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cc590 0x7ffad00cae00 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.754+0000 7ffad69ff700 1 -- 172.21.15.131:0/341257935 >> 172.21.15.131:0/341257935 conn(0x7ffad0063cd0 msgr2=0x7ffad006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.754+0000 7ffad69ff700 1 -- 172.21.15.131:0/341257935 shutdown_connections 2023-12-26T23:09:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.754+0000 7ffad69ff700 1 -- 172.21.15.131:0/341257935 wait complete. 2023-12-26T23:09:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.755+0000 7ffad69ff700 1 Processor -- start 2023-12-26T23:09:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.755+0000 7ffad69ff700 1 -- start start 2023-12-26T23:09:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.756+0000 7ffad69ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cb340 0x7ffad00c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.756+0000 7ffad69ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad00cc590 0x7ffad00c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.756+0000 7ffad69ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0142f00 0x7ffad015fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.756+0000 7ffad69ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00a9ff0 con 0x7ffad00cb340 2023-12-26T23:09:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.756+0000 7ffad69ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00c9910 con 0x7ffad0142f00 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.756+0000 7ffad69ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00c9b00 con 0x7ffad00cc590 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cb340 0x7ffad00c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cb340 0x7ffad00c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42210/0 (socket says 172.21.15.131:42210) 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacffff700 1 -- 172.21.15.131:0/2516166618 learned_addr learned my addr 172.21.15.131:0/2516166618 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacf7fe700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad00cc590 0x7ffad00c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffad4f9c700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0142f00 0x7ffad015fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacffff700 1 -- 172.21.15.131:0/2516166618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad00cc590 msgr2=0x7ffad00c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacffff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad00cc590 0x7ffad00c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacffff700 1 -- 172.21.15.131:0/2516166618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0142f00 msgr2=0x7ffad015fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacffff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0142f00 0x7ffad015fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacffff700 1 -- 172.21.15.131:0/2516166618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40094f0 con 0x7ffad00cb340 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffad4f9c700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0142f00 0x7ffad015fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffacffff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cb340 0x7ffad00c6100 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ffac000ec80 tx=0x7ffac00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffab7fff700 1 -- 172.21.15.131:0/2516166618 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac000c850 con 0x7ffad00cb340 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.757+0000 7ffab7fff700 1 -- 172.21.15.131:0/2516166618 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac000c9d0 con 0x7ffad00cb340 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.758+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad00c9e40 con 0x7ffad00cb340 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.758+0000 7ffab7fff700 1 -- 172.21.15.131:0/2516166618 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac000cd90 con 0x7ffad00cb340 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.758+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad00aa4e0 con 0x7ffad00cb340 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.759+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad00aa8c0 con 0x7ffad00cb340 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.759+0000 7ffab7fff700 1 -- 172.21.15.131:0/2516166618 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 95) v1 ==== 84580+0+0 (secure 0 0 0) 0x7ffac0013680 con 0x7ffad00cb340 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.760+0000 7ffab7fff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ffabc066d80 0x7ffabc069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.760+0000 7ffab7fff700 1 -- 172.21.15.131:0/2516166618 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6763+0+0 (secure 0 0 0) 0x7ffac00849d0 con 0x7ffad00cb340 2023-12-26T23:09:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.761+0000 7ffacf7fe700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ffabc066d80 0x7ffabc069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.761+0000 7ffacf7fe700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ffabc066d80 0x7ffabc069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffad006f560 tx=0x7ffac4017000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.766+0000 7ffab7fff700 1 -- 172.21.15.131:0/2516166618 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffac0051a00 con 0x7ffad00cb340 2023-12-26T23:09:28.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:28.963+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffad00717f0 con 0x7ffad00cb340 2023-12-26T23:09:29.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.499+0000 7ffab7fff700 1 -- 172.21.15.131:0/2516166618 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffac0051dd0 con 0x7ffad00cb340 2023-12-26T23:09:29.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.501+0000 7ffab7fff700 1 -- 172.21.15.131:0/2516166618 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v184) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffac0051a00 con 0x7ffad00cb340 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ffabc066d80 msgr2=0x7ffabc069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ffabc066d80 0x7ffabc069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffad006f560 tx=0x7ffac4017000 comp rx=0 tx=0).stop 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cb340 msgr2=0x7ffad00c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cb340 0x7ffad00c6100 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ffac000ec80 tx=0x7ffac00091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 shutdown_connections 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ffabc066d80 0x7ffabc069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad0142f00 0x7ffad015fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffad00cb340 0x7ffad00c6100 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 --2- 172.21.15.131:0/2516166618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad00cc590 0x7ffad00c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.505+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 >> 172.21.15.131:0/2516166618 conn(0x7ffad0063cd0 msgr2=0x7ffad00c4380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.506+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 shutdown_connections 2023-12-26T23:09:29.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.506+0000 7ffad69ff700 1 -- 172.21.15.131:0/2516166618 wait complete. 2023-12-26T23:09:29.507 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T23:09:29.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T23:09:29.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.716+0000 7feb51bff700 1 Processor -- start 2023-12-26T23:09:29.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.716+0000 7feb51bff700 1 -- start start 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.717+0000 7feb51bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb4c1596a0 0x7feb4c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.717+0000 7feb51bff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feb4c159a70 0x7feb4c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb51bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4c0afcd0 con 0x7feb4c0b0160 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb51bff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4c0b3e70 con 0x7feb4c159a70 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb51bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4c0b2e00 con 0x7feb4c1596a0 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb4affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feb4c159a70 0x7feb4c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb4affd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feb4c159a70 0x7feb4c0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42212/0 (socket says 172.21.15.131:42212) 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb4affd700 1 -- 172.21.15.131:0/555735881 learned_addr learned my addr 172.21.15.131:0/555735881 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb4b7fe700 1 --2- 172.21.15.131:0/555735881 >> v2:172.21.15.190:3300/0 conn(0x7feb4c1596a0 0x7feb4c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb4affd700 1 -- 172.21.15.131:0/555735881 >> v2:172.21.15.190:3300/0 conn(0x7feb4c1596a0 msgr2=0x7feb4c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb4affd700 1 --2- 172.21.15.131:0/555735881 >> v2:172.21.15.190:3300/0 conn(0x7feb4c1596a0 0x7feb4c0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb4affd700 1 -- 172.21.15.131:0/555735881 >> v1:172.21.15.190:6789/0 conn(0x7feb4c0b0160 legacy=0x7feb4c0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.718+0000 7feb4affd700 1 -- 172.21.15.131:0/555735881 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0b0cb0 con 0x7feb4c159a70 2023-12-26T23:09:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.719+0000 7feb4affd700 1 --2- 172.21.15.131:0/555735881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c159a70 0x7feb4c0b28c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0892a0 tx=0x7feb40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6125a416588c96 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.719+0000 7feb4a7fc700 1 -- 172.21.15.131:0/555735881 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4000b860 con 0x7feb4c159a70 2023-12-26T23:09:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.719+0000 7feb4a7fc700 1 -- 172.21.15.131:0/555735881 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4000b9c0 con 0x7feb4c159a70 2023-12-26T23:09:29.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.719+0000 7feb4a7fc700 1 -- 172.21.15.131:0/555735881 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4000bd60 con 0x7feb4c159a70 2023-12-26T23:09:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.719+0000 7feb51bff700 1 -- 172.21.15.131:0/555735881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c159a70 msgr2=0x7feb4c0b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.719+0000 7feb51bff700 1 --2- 172.21.15.131:0/555735881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c159a70 0x7feb4c0b28c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0892a0 tx=0x7feb40009250 comp rx=0 tx=0).stop 2023-12-26T23:09:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.720+0000 7feb51bff700 1 -- 172.21.15.131:0/555735881 shutdown_connections 2023-12-26T23:09:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.720+0000 7feb51bff700 1 --2- 172.21.15.131:0/555735881 >> v2:172.21.15.190:3300/0 conn(0x7feb4c1596a0 0x7feb4c0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.720+0000 7feb51bff700 1 --2- 172.21.15.131:0/555735881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c159a70 0x7feb4c0b28c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.720+0000 7feb51bff700 1 -- 172.21.15.131:0/555735881 >> 172.21.15.131:0/555735881 conn(0x7feb4c06ece0 msgr2=0x7feb4c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.720+0000 7feb51bff700 1 -- 172.21.15.131:0/555735881 shutdown_connections 2023-12-26T23:09:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.720+0000 7feb51bff700 1 -- 172.21.15.131:0/555735881 wait complete. 2023-12-26T23:09:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.721+0000 7feb51bff700 1 Processor -- start 2023-12-26T23:09:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.721+0000 7feb51bff700 1 -- start start 2023-12-26T23:09:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.722+0000 7feb51bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c0b0160 0x7feb4c155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.722+0000 7feb51bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb4c1596a0 0x7feb4c0c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.722+0000 7feb51bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb4c159a70 0x7feb4c0c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.722+0000 7feb51bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c7680 con 0x7feb4c0b0160 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.722+0000 7feb51bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c7960 con 0x7feb4c159a70 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.722+0000 7feb51bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c7c40 con 0x7feb4c1596a0 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.722+0000 7feb4b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c0b0160 0x7feb4c155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.722+0000 7feb4b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c0b0160 0x7feb4c155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42224/0 (socket says 172.21.15.131:42224) 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.722+0000 7feb4b7fe700 1 -- 172.21.15.131:0/1835206463 learned_addr learned my addr 172.21.15.131:0/1835206463 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb4affd700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb4c1596a0 0x7feb4c0c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb4bfff700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb4c159a70 0x7feb4c0c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb4b7fe700 1 -- 172.21.15.131:0/1835206463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb4c1596a0 msgr2=0x7feb4c0c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb4b7fe700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb4c1596a0 0x7feb4c0c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb4b7fe700 1 -- 172.21.15.131:0/1835206463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb4c159a70 msgr2=0x7feb4c0c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb4b7fe700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb4c159a70 0x7feb4c0c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb4b7fe700 1 -- 172.21.15.131:0/1835206463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400094f0 con 0x7feb4c0b0160 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb4bfff700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb4c159a70 0x7feb4c0c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb4b7fe700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c0b0160 0x7feb4c155900 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00d890 tx=0x7feb3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb337fe700 1 -- 172.21.15.131:0/1835206463 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c0125f0 con 0x7feb4c0b0160 2023-12-26T23:09:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb337fe700 1 -- 172.21.15.131:0/1835206463 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c012770 con 0x7feb4c0b0160 2023-12-26T23:09:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c0c7f20 con 0x7feb4c0b0160 2023-12-26T23:09:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb337fe700 1 -- 172.21.15.131:0/1835206463 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c012b30 con 0x7feb4c0b0160 2023-12-26T23:09:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.723+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c155e40 con 0x7feb4c0b0160 2023-12-26T23:09:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.724+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c0c8570 con 0x7feb4c0b0160 2023-12-26T23:09:29.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.725+0000 7feb337fe700 1 -- 172.21.15.131:0/1835206463 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feb3c01e460 con 0x7feb4c0b0160 2023-12-26T23:09:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.726+0000 7feb337fe700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7feb38066f30 0x7feb380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.727+0000 7feb337fe700 1 -- 172.21.15.131:0/1835206463 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6291+0+0 (secure 0 0 0) 0x7feb3c084e30 con 0x7feb4c0b0160 2023-12-26T23:09:29.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.727+0000 7feb4affd700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7feb38066f30 0x7feb380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.727+0000 7feb4affd700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7feb38066f30 0x7feb380693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0892a0 tx=0x7feb40017000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.731+0000 7feb337fe700 1 -- 172.21.15.131:0/1835206463 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb3c051e50 con 0x7feb4c0b0160 2023-12-26T23:09:29.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.922+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7feb4c1571e0 con 0x7feb4c0b0160 2023-12-26T23:09:29.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.927+0000 7feb337fe700 1 -- 172.21.15.131:0/1835206463 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7feb3c055470 con 0x7feb4c0b0160 2023-12-26T23:09:29.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7feb38066f30 msgr2=0x7feb380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:29.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7feb38066f30 0x7feb380693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0892a0 tx=0x7feb40017000 comp rx=0 tx=0).stop 2023-12-26T23:09:29.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c0b0160 msgr2=0x7feb4c155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:29.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c0b0160 0x7feb4c155900 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00d890 tx=0x7feb3c008040 comp rx=0 tx=0).stop 2023-12-26T23:09:29.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 shutdown_connections 2023-12-26T23:09:29.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7feb38066f30 0x7feb380693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb4c159a70 0x7feb4c0c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb4c0b0160 0x7feb4c155900 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 --2- 172.21.15.131:0/1835206463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb4c1596a0 0x7feb4c0c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:29.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 >> 172.21.15.131:0/1835206463 conn(0x7feb4c06ece0 msgr2=0x7feb4c157900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:29.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 shutdown_connections 2023-12-26T23:09:29.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:29.929+0000 7feb51bff700 1 -- 172.21.15.131:0/1835206463 wait complete. 2023-12-26T23:09:29.946 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T23:09:30.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.147+0000 7f80ba3ff700 1 Processor -- start 2023-12-26T23:09:30.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.147+0000 7f80ba3ff700 1 -- start start 2023-12-26T23:09:30.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.148+0000 7f80ba3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f80b40b07f0 0x7f80b41554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.148+0000 7f80ba3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80b4159cf0 0x7f80b4155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.148+0000 7f80ba3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b40c8bf0 con 0x7f80b4159920 2023-12-26T23:09:30.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.148+0000 7f80ba3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b4155f50 con 0x7f80b4159cf0 2023-12-26T23:09:30.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.148+0000 7f80ba3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b40c8e60 con 0x7f80b40b07f0 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.148+0000 7f80b899c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80b4159cf0 0x7f80b4155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b37fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f80b40b07f0 0x7f80b41554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b899c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80b4159cf0 0x7f80b4155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42230/0 (socket says 172.21.15.131:42230) 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b899c700 1 -- 172.21.15.131:0/2444621609 learned_addr learned my addr 172.21.15.131:0/2444621609 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b3fff700 1 --1- 172.21.15.131:0/2444621609 >> v1:172.21.15.190:6789/0 conn(0x7f80b4159920 0x7f80b40b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:43972/0 (socket says 172.21.15.131:43972) 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b899c700 1 -- 172.21.15.131:0/2444621609 >> v2:172.21.15.190:3300/0 conn(0x7f80b40b07f0 msgr2=0x7f80b41554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b899c700 1 --2- 172.21.15.131:0/2444621609 >> v2:172.21.15.190:3300/0 conn(0x7f80b40b07f0 0x7f80b41554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b899c700 1 -- 172.21.15.131:0/2444621609 >> v1:172.21.15.190:6789/0 conn(0x7f80b4159920 legacy=0x7f80b40b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b899c700 1 -- 172.21.15.131:0/2444621609 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b40c9140 con 0x7f80b4159cf0 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b899c700 1 --2- 172.21.15.131:0/2444621609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 0x7f80b4155a10 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f80b40b16b0 tx=0x7f80ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25679ce0553aa26a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b2ffd700 1 -- 172.21.15.131:0/2444621609 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00e070 con 0x7f80b4159cf0 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b2ffd700 1 -- 172.21.15.131:0/2444621609 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac00ba20 con 0x7f80b4159cf0 2023-12-26T23:09:30.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.149+0000 7f80b2ffd700 1 -- 172.21.15.131:0/2444621609 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0092e0 con 0x7f80b4159cf0 2023-12-26T23:09:30.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.150+0000 7f80ba3ff700 1 -- 172.21.15.131:0/2444621609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 msgr2=0x7f80b4155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.150+0000 7f80ba3ff700 1 --2- 172.21.15.131:0/2444621609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 0x7f80b4155a10 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f80b40b16b0 tx=0x7f80ac009840 comp rx=0 tx=0).stop 2023-12-26T23:09:30.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.150+0000 7f80ba3ff700 1 -- 172.21.15.131:0/2444621609 shutdown_connections 2023-12-26T23:09:30.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.150+0000 7f80ba3ff700 1 --2- 172.21.15.131:0/2444621609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 0x7f80b4155a10 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.150+0000 7f80ba3ff700 1 --2- 172.21.15.131:0/2444621609 >> v2:172.21.15.190:3300/0 conn(0x7f80b40b07f0 0x7f80b41554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.150+0000 7f80ba3ff700 1 -- 172.21.15.131:0/2444621609 >> 172.21.15.131:0/2444621609 conn(0x7f80b4068040 msgr2=0x7f80b4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:30.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.150+0000 7f80ba3ff700 1 -- 172.21.15.131:0/2444621609 shutdown_connections 2023-12-26T23:09:30.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.150+0000 7f80ba3ff700 1 -- 172.21.15.131:0/2444621609 wait complete. 2023-12-26T23:09:30.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.152+0000 7f80ba3ff700 1 Processor -- start 2023-12-26T23:09:30.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.152+0000 7f80ba3ff700 1 -- start start 2023-12-26T23:09:30.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80ba3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80b4159920 0x7f80b4154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80ba3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 0x7f80b4154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80ba3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80b4154e40 0x7f80b40c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80ba3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41547a0 con 0x7f80b4159cf0 2023-12-26T23:09:30.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80ba3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4156120 con 0x7f80b4154e40 2023-12-26T23:09:30.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80ba3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4156610 con 0x7f80b4159920 2023-12-26T23:09:30.154 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 0x7f80b4154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80b37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 0x7f80b4154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42246/0 (socket says 172.21.15.131:42246) 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80b37fe700 1 -- 172.21.15.131:0/529504137 learned_addr learned my addr 172.21.15.131:0/529504137 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80b37fe700 1 -- 172.21.15.131:0/529504137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80b4159920 msgr2=0x7f80b4154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80b37fe700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80b4159920 0x7f80b4154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80b37fe700 1 -- 172.21.15.131:0/529504137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80b4154e40 msgr2=0x7f80b40c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80b37fe700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80b4154e40 0x7f80b40c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80b37fe700 1 -- 172.21.15.131:0/529504137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac009ae0 con 0x7f80b4159cf0 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.153+0000 7f80b37fe700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 0x7f80b4154900 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f80a8006b00 tx=0x7f80a800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.154+0000 7f809bfff700 1 -- 172.21.15.131:0/529504137 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800dde0 con 0x7f80b4159cf0 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.154+0000 7f809bfff700 1 -- 172.21.15.131:0/529504137 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a8010440 con 0x7f80b4159cf0 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.154+0000 7f809bfff700 1 -- 172.21.15.131:0/529504137 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8010800 con 0x7f80b4159cf0 2023-12-26T23:09:30.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.154+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b4156330 con 0x7f80b4159cf0 2023-12-26T23:09:30.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.154+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b4156e60 con 0x7f80b4159cf0 2023-12-26T23:09:30.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.155+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b41602d0 con 0x7f80b4159cf0 2023-12-26T23:09:30.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.156+0000 7f809bfff700 1 -- 172.21.15.131:0/529504137 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f80a8010b70 con 0x7f80b4159cf0 2023-12-26T23:09:30.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.157+0000 7f809bfff700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f80a0066f30 0x7f80a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.157+0000 7f809bfff700 1 -- 172.21.15.131:0/529504137 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6291+0+0 (secure 0 0 0) 0x7f80a8084f10 con 0x7f80b4159cf0 2023-12-26T23:09:30.157 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.157+0000 7f80b3fff700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f80a0066f30 0x7f80a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.158+0000 7f80b3fff700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f80a0066f30 0x7f80a00693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f80b40c85f0 tx=0x7f80a4007b30 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:30.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.161+0000 7f809bfff700 1 -- 172.21.15.131:0/529504137 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80b41602d0 con 0x7f80b4159cf0 2023-12-26T23:09:30.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.356+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f80b41602d0 con 0x7f80b4159cf0 2023-12-26T23:09:30.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.360+0000 7f809bfff700 1 -- 172.21.15.131:0/529504137 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v62) v1 ==== 115+0+0 (secure 0 0 0) 0x7f80b41602d0 con 0x7f80b4159cf0 2023-12-26T23:09:30.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f80a0066f30 msgr2=0x7f80a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f80a0066f30 0x7f80a00693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f80b40c85f0 tx=0x7f80a4007b30 comp rx=0 tx=0).stop 2023-12-26T23:09:30.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 msgr2=0x7f80b4154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 0x7f80b4154900 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f80a8006b00 tx=0x7f80a800b430 comp rx=0 tx=0).stop 2023-12-26T23:09:30.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 shutdown_connections 2023-12-26T23:09:30.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f80a0066f30 0x7f80a00693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80b4154e40 0x7f80b40c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80b4159920 0x7f80b4154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 --2- 172.21.15.131:0/529504137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80b4159cf0 0x7f80b4154900 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 >> 172.21.15.131:0/529504137 conn(0x7f80b4068040 msgr2=0x7f80b40ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:30.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 shutdown_connections 2023-12-26T23:09:30.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.363+0000 7f80ba3ff700 1 -- 172.21.15.131:0/529504137 wait complete. 2023-12-26T23:09:30.382 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T23:09:30.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.574+0000 7fe2aebff700 1 Processor -- start 2023-12-26T23:09:30.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.575+0000 7fe2aebff700 1 -- start start 2023-12-26T23:09:30.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.576+0000 7fe2aebff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2a80b0750 0x7fe2a80b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.576+0000 7fe2aebff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2a80c9400 0x7fe2a80c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.576+0000 7fe2aebff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a8156940 con 0x7fe2a80cb320 2023-12-26T23:09:30.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.576+0000 7fe2aebff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a80c8cc0 con 0x7fe2a80b0750 2023-12-26T23:09:30.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.576+0000 7fe2aebff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a80ad700 con 0x7fe2a80c9400 2023-12-26T23:09:30.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.576+0000 7fe2a7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2a80b0750 0x7fe2a80b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.576+0000 7fe2a7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2a80b0750 0x7fe2a80b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42248/0 (socket says 172.21.15.131:42248) 2023-12-26T23:09:30.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.576+0000 7fe2a7fff700 1 -- 172.21.15.131:0/598525090 learned_addr learned my addr 172.21.15.131:0/598525090 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:30.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.576+0000 7fe2ad19c700 1 --2- 172.21.15.131:0/598525090 >> v2:172.21.15.190:3300/0 conn(0x7fe2a80c9400 0x7fe2a80c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.577+0000 7fe2a7fff700 1 -- 172.21.15.131:0/598525090 >> v2:172.21.15.190:3300/0 conn(0x7fe2a80c9400 msgr2=0x7fe2a80c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:30.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.577+0000 7fe2a7fff700 1 --2- 172.21.15.131:0/598525090 >> v2:172.21.15.190:3300/0 conn(0x7fe2a80c9400 0x7fe2a80c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.577+0000 7fe2a7fff700 1 -- 172.21.15.131:0/598525090 >> v1:172.21.15.190:6789/0 conn(0x7fe2a80cb320 legacy=0x7fe2a8069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.577+0000 7fe2a7fff700 1 -- 172.21.15.131:0/598525090 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a80ae500 con 0x7fe2a80b0750 2023-12-26T23:09:30.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.577+0000 7fe2a7fff700 1 --2- 172.21.15.131:0/598525090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80b0750 0x7fe2a80b0bf0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe2a806fb50 tx=0x7fe29400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7ebdc523c26c9d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:30.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.577+0000 7fe2a77fe700 1 -- 172.21.15.131:0/598525090 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29400be60 con 0x7fe2a80b0750 2023-12-26T23:09:30.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.577+0000 7fe2a77fe700 1 -- 172.21.15.131:0/598525090 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29400ddc0 con 0x7fe2a80b0750 2023-12-26T23:09:30.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.577+0000 7fe2a77fe700 1 -- 172.21.15.131:0/598525090 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2940196b0 con 0x7fe2a80b0750 2023-12-26T23:09:30.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.578+0000 7fe2aebff700 1 -- 172.21.15.131:0/598525090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80b0750 msgr2=0x7fe2a80b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.578+0000 7fe2aebff700 1 --2- 172.21.15.131:0/598525090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80b0750 0x7fe2a80b0bf0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe2a806fb50 tx=0x7fe29400b4f0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.578+0000 7fe2aebff700 1 -- 172.21.15.131:0/598525090 shutdown_connections 2023-12-26T23:09:30.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.578+0000 7fe2aebff700 1 --2- 172.21.15.131:0/598525090 >> v2:172.21.15.190:3300/0 conn(0x7fe2a80c9400 0x7fe2a80c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.579 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.578+0000 7fe2aebff700 1 --2- 172.21.15.131:0/598525090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80b0750 0x7fe2a80b0bf0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.578+0000 7fe2aebff700 1 -- 172.21.15.131:0/598525090 >> 172.21.15.131:0/598525090 conn(0x7fe2a8068040 msgr2=0x7fe2a8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:30.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.578+0000 7fe2aebff700 1 -- 172.21.15.131:0/598525090 shutdown_connections 2023-12-26T23:09:30.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.578+0000 7fe2aebff700 1 -- 172.21.15.131:0/598525090 wait complete. 2023-12-26T23:09:30.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.579+0000 7fe2aebff700 1 Processor -- start 2023-12-26T23:09:30.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.579+0000 7fe2aebff700 1 -- start start 2023-12-26T23:09:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.580+0000 7fe2aebff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2a80b0750 0x7fe2a81528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.580+0000 7fe2aebff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80c9400 0x7fe2a8151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.580+0000 7fe2aebff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2a80cb320 0x7fe2a80c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.580+0000 7fe2aebff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a816a860 con 0x7fe2a80c9400 2023-12-26T23:09:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.580+0000 7fe2aebff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a816a2a0 con 0x7fe2a80b0750 2023-12-26T23:09:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.580+0000 7fe2aebff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a816a580 con 0x7fe2a80cb320 2023-12-26T23:09:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.580+0000 7fe2a7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80c9400 0x7fe2a8151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80c9400 0x7fe2a8151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42254/0 (socket says 172.21.15.131:42254) 2023-12-26T23:09:30.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a7fff700 1 -- 172.21.15.131:0/1871581549 learned_addr learned my addr 172.21.15.131:0/1871581549 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2ac99b700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2a80b0750 0x7fe2a81528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2ad19c700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2a80cb320 0x7fe2a80c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a7fff700 1 -- 172.21.15.131:0/1871581549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2a80cb320 msgr2=0x7fe2a80c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a7fff700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2a80cb320 0x7fe2a80c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a7fff700 1 -- 172.21.15.131:0/1871581549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2a80b0750 msgr2=0x7fe2a81528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a7fff700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2a80b0750 0x7fe2a81528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a7fff700 1 -- 172.21.15.131:0/1871581549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29400b790 con 0x7fe2a80c9400 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a7fff700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80c9400 0x7fe2a8151fa0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b760 tx=0x7fe29401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a4ff9700 1 -- 172.21.15.131:0/1871581549 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294010680 con 0x7fe2a80c9400 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.581+0000 7fe2a4ff9700 1 -- 172.21.15.131:0/1871581549 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294010800 con 0x7fe2a80c9400 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.582+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a8152ed0 con 0x7fe2a80c9400 2023-12-26T23:09:30.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.582+0000 7fe2a4ff9700 1 -- 172.21.15.131:0/1871581549 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294010bc0 con 0x7fe2a80c9400 2023-12-26T23:09:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.582+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a81524e0 con 0x7fe2a80c9400 2023-12-26T23:09:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.583+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a80657e0 con 0x7fe2a80c9400 2023-12-26T23:09:30.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.583+0000 7fe2a4ff9700 1 -- 172.21.15.131:0/1871581549 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe2940233f0 con 0x7fe2a80c9400 2023-12-26T23:09:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.584+0000 7fe2a4ff9700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe28c066f30 0x7fe28c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.584+0000 7fe2a4ff9700 1 -- 172.21.15.131:0/1871581549 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6291+0+0 (secure 0 0 0) 0x7fe294014070 con 0x7fe2a80c9400 2023-12-26T23:09:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.585+0000 7fe2ac99b700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe28c066f30 0x7fe28c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:30.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.585+0000 7fe2ac99b700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe28c066f30 0x7fe28c0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80c7fd0 tx=0x7fe29c009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:30.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.589+0000 7fe2a4ff9700 1 -- 172.21.15.131:0/1871581549 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2940535c0 con 0x7fe2a80c9400 2023-12-26T23:09:30.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.784+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fe2a81533f0 con 0x7fe2a80c9400 2023-12-26T23:09:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.788+0000 7fe2a4ff9700 1 -- 172.21.15.131:0/1871581549 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v63) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe2a81533f0 con 0x7fe2a80c9400 2023-12-26T23:09:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe28c066f30 msgr2=0x7fe28c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe28c066f30 0x7fe28c0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80c7fd0 tx=0x7fe29c009040 comp rx=0 tx=0).stop 2023-12-26T23:09:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80c9400 msgr2=0x7fe2a8151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80c9400 0x7fe2a8151fa0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b760 tx=0x7fe29401b000 comp rx=0 tx=0).stop 2023-12-26T23:09:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 shutdown_connections 2023-12-26T23:09:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe28c066f30 0x7fe28c0693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2a80cb320 0x7fe2a80c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2a80b0750 0x7fe2a81528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 --2- 172.21.15.131:0/1871581549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2a80c9400 0x7fe2a8151fa0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.790+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 >> 172.21.15.131:0/1871581549 conn(0x7fe2a8068040 msgr2=0x7fe2a81559b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.791+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 shutdown_connections 2023-12-26T23:09:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.791+0000 7fe2aebff700 1 -- 172.21.15.131:0/1871581549 wait complete. 2023-12-26T23:09:30.809 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-12-26T23:09:31.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.997+0000 7fb79bbff700 1 Processor -- start 2023-12-26T23:09:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.997+0000 7fb79bbff700 1 -- start start 2023-12-26T23:09:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79bbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb794159820 0x7fb7940abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 0x7fb79406dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79bbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79415a8c0 con 0x7fb79406e2b0 2023-12-26T23:09:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79bbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79415ad50 con 0x7fb794159820 2023-12-26T23:09:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79406e730 con 0x7fb7940aa480 2023-12-26T23:09:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb794159820 0x7fb7940abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79a19c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb79406e2b0 0x7fb79415d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49346/0 (socket says 172.21.15.131:49346) 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79a19c700 1 -- 172.21.15.131:0/3690331604 learned_addr learned my addr 172.21.15.131:0/3690331604 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79919a700 1 --2- 172.21.15.131:0/3690331604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 0x7fb79406dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79999b700 1 -- 172.21.15.131:0/3690331604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 msgr2=0x7fb79406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79999b700 1 --2- 172.21.15.131:0/3690331604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 0x7fb79406dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79999b700 1 -- 172.21.15.131:0/3690331604 >> v1:172.21.15.131:6789/0 conn(0x7fb79406e2b0 legacy=0x7fb79415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb79999b700 1 -- 172.21.15.131:0/3690331604 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794159f90 con 0x7fb794159820 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:30.999+0000 7fb798999700 1 -- 172.21.15.131:0/3690331604 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146093171 0 0) 0x7fb79415a8c0 con 0x7fb79406e2b0 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb79999b700 1 --2- 172.21.15.131:0/3690331604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 0x7fb7940abdb0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fb790009910 tx=0x7fb7900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc01ec13f45fdb88 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb798999700 1 -- 172.21.15.131:0/3690331604 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000b820 con 0x7fb794159820 2023-12-26T23:09:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb798999700 1 -- 172.21.15.131:0/3690331604 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79000b980 con 0x7fb794159820 2023-12-26T23:09:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb798999700 1 -- 172.21.15.131:0/3690331604 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000bd20 con 0x7fb794159820 2023-12-26T23:09:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb79bbff700 1 -- 172.21.15.131:0/3690331604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 msgr2=0x7fb7940abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb79bbff700 1 --2- 172.21.15.131:0/3690331604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 0x7fb7940abdb0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fb790009910 tx=0x7fb7900091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb79bbff700 1 -- 172.21.15.131:0/3690331604 shutdown_connections 2023-12-26T23:09:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb79bbff700 1 --2- 172.21.15.131:0/3690331604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 0x7fb7940abdb0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb79bbff700 1 --2- 172.21.15.131:0/3690331604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 0x7fb79406dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.000+0000 7fb79bbff700 1 -- 172.21.15.131:0/3690331604 >> 172.21.15.131:0/3690331604 conn(0x7fb79406c290 msgr2=0x7fb7940ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.001+0000 7fb79bbff700 1 -- 172.21.15.131:0/3690331604 shutdown_connections 2023-12-26T23:09:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.001+0000 7fb79bbff700 1 -- 172.21.15.131:0/3690331604 wait complete. 2023-12-26T23:09:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.002+0000 7fb79bbff700 1 Processor -- start 2023-12-26T23:09:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.002+0000 7fb79bbff700 1 -- start start 2023-12-26T23:09:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79bbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb79406e2b0 0x7fb7940c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 0x7fb7940c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79bbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 0x7fb7940c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79bbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940b1ae0 con 0x7fb794159820 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79bbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940b0ed0 con 0x7fb79406e2b0 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940b11b0 con 0x7fb7940aa480 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79a19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 0x7fb7940c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79a19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 0x7fb7940c2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42258/0 (socket says 172.21.15.131:42258) 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79a19c700 1 -- 172.21.15.131:0/3392315296 learned_addr learned my addr 172.21.15.131:0/3392315296 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79919a700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 0x7fb7940c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.003+0000 7fb79999b700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb79406e2b0 0x7fb7940c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79a19c700 1 -- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 msgr2=0x7fb7940c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79a19c700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 0x7fb7940c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79a19c700 1 -- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb79406e2b0 msgr2=0x7fb7940c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79a19c700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb79406e2b0 0x7fb7940c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79a19c700 1 -- 172.21.15.131:0/3392315296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790009420 con 0x7fb794159820 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79999b700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb79406e2b0 0x7fb7940c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79919a700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 0x7fb7940c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79a19c700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 0x7fb7940c2a80 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00cc10 tx=0x7fb78c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb789ffb700 1 -- 172.21.15.131:0/3392315296 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00a930 con 0x7fb794159820 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb789ffb700 1 -- 172.21.15.131:0/3392315296 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c00aab0 con 0x7fb794159820 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7940b1490 con 0x7fb794159820 2023-12-26T23:09:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb789ffb700 1 -- 172.21.15.131:0/3392315296 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00ae70 con 0x7fb794159820 2023-12-26T23:09:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.004+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7940c5230 con 0x7fb794159820 2023-12-26T23:09:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.005+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7940657e0 con 0x7fb794159820 2023-12-26T23:09:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.006+0000 7fb789ffb700 1 -- 172.21.15.131:0/3392315296 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb78c007300 con 0x7fb794159820 2023-12-26T23:09:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.007+0000 7fb789ffb700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb778066f30 0x7fb7780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.007+0000 7fb789ffb700 1 -- 172.21.15.131:0/3392315296 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6291+0+0 (secure 0 0 0) 0x7fb78c084bb0 con 0x7fb794159820 2023-12-26T23:09:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.007+0000 7fb79999b700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb778066f30 0x7fb7780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.008+0000 7fb79999b700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb778066f30 0x7fb7780693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb790009860 tx=0x7fb790017000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:31.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.011+0000 7fb789ffb700 1 -- 172.21.15.131:0/3392315296 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb78c051bb0 con 0x7fb794159820 2023-12-26T23:09:31.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.279+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fb7940aec10 con 0x7fb794159820 2023-12-26T23:09:31.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.502+0000 7fb789ffb700 1 -- 172.21.15.131:0/3392315296 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2978) v1 ==== 165+0+0 (secure 0 0 0) 0x7fb78c0551d0 con 0x7fb794159820 2023-12-26T23:09:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.504+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb778066f30 msgr2=0x7fb7780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.504+0000 7fb79bbff700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb778066f30 0x7fb7780693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb790009860 tx=0x7fb790017000 comp rx=0 tx=0).stop 2023-12-26T23:09:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.504+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 msgr2=0x7fb7940c2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.504+0000 7fb79bbff700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 0x7fb7940c2a80 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00cc10 tx=0x7fb78c001970 comp rx=0 tx=0).stop 2023-12-26T23:09:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.505+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 shutdown_connections 2023-12-26T23:09:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.505+0000 7fb79bbff700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb778066f30 0x7fb7780693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.505+0000 7fb79bbff700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb794159820 0x7fb7940c2a80 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.505+0000 7fb79bbff700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb79406e2b0 0x7fb7940c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.505+0000 7fb79bbff700 1 --2- 172.21.15.131:0/3392315296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940aa480 0x7fb7940c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.505+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 >> 172.21.15.131:0/3392315296 conn(0x7fb79406c290 msgr2=0x7fb794156630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.505+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 shutdown_connections 2023-12-26T23:09:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.505+0000 7fb79bbff700 1 -- 172.21.15.131:0/3392315296 wait complete. 2023-12-26T23:09:31.522 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-12-26T23:09:31.523 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-12-26T23:09:31.523 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-26T23:09:31.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-12-26T23:09:31.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.714+0000 7f1a325ff700 1 Processor -- start 2023-12-26T23:09:31.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.714+0000 7f1a325ff700 1 -- start start 2023-12-26T23:09:31.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a325ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1a2c1596a0 0x7f1a2c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a325ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2c0afcd0 con 0x7f1a2c0b0870 2023-12-26T23:09:31.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a325ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2c0b3e70 con 0x7f1a2c159a70 2023-12-26T23:09:31.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a325ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c0b0cb0 con 0x7f1a2c1596a0 2023-12-26T23:09:31.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a2bfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1a2c1596a0 0x7f1a2c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:31.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a30b9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1a2c0b0870 0x7f1a2c0b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49352/0 (socket says 172.21.15.131:49352) 2023-12-26T23:09:31.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a30b9c700 1 -- 172.21.15.131:0/2243367122 learned_addr learned my addr 172.21.15.131:0/2243367122 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:31.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a2bfff700 1 -- 172.21.15.131:0/2243367122 >> v1:172.21.15.190:6789/0 conn(0x7f1a2c159a70 legacy=0x7f1a2c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:31.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a2bfff700 1 -- 172.21.15.131:0/2243367122 >> v1:172.21.15.131:6789/0 conn(0x7f1a2c0b0870 legacy=0x7f1a2c0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a2bfff700 1 -- 172.21.15.131:0/2243367122 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c0b0f20 con 0x7f1a2c1596a0 2023-12-26T23:09:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.716+0000 7f1a2affd700 1 -- 172.21.15.131:0/2243367122 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369504905 0 0) 0x7f1a2c0afcd0 con 0x7f1a2c0b0870 2023-12-26T23:09:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.717+0000 7f1a2bfff700 1 --2- 172.21.15.131:0/2243367122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c1596a0 0x7f1a2c0b2380 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0098e0 tx=0x7f1a1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818f278fb4a22e6b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.717+0000 7f1a2affd700 1 -- 172.21.15.131:0/2243367122 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c009e80 con 0x7f1a2c1596a0 2023-12-26T23:09:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.717+0000 7f1a2affd700 1 -- 172.21.15.131:0/2243367122 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c00b820 con 0x7f1a2c1596a0 2023-12-26T23:09:31.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.717+0000 7f1a2affd700 1 -- 172.21.15.131:0/2243367122 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00bba0 con 0x7f1a2c1596a0 2023-12-26T23:09:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.717+0000 7f1a325ff700 1 -- 172.21.15.131:0/2243367122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c1596a0 msgr2=0x7f1a2c0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.717+0000 7f1a325ff700 1 --2- 172.21.15.131:0/2243367122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c1596a0 0x7f1a2c0b2380 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0098e0 tx=0x7f1a1c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.718+0000 7f1a325ff700 1 -- 172.21.15.131:0/2243367122 shutdown_connections 2023-12-26T23:09:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.718+0000 7f1a325ff700 1 --2- 172.21.15.131:0/2243367122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c1596a0 0x7f1a2c0b2380 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.718+0000 7f1a325ff700 1 -- 172.21.15.131:0/2243367122 >> 172.21.15.131:0/2243367122 conn(0x7f1a2c06ece0 msgr2=0x7f1a2c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:31.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.718+0000 7f1a325ff700 1 -- 172.21.15.131:0/2243367122 shutdown_connections 2023-12-26T23:09:31.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.718+0000 7f1a325ff700 1 -- 172.21.15.131:0/2243367122 wait complete. 2023-12-26T23:09:31.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.719+0000 7f1a325ff700 1 Processor -- start 2023-12-26T23:09:31.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.719+0000 7f1a325ff700 1 -- start start 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.720+0000 7f1a325ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c0b0870 0x7f1a2c0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.720+0000 7f1a325ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a2c1596a0 0x7f1a2c0cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.720+0000 7f1a325ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a2c159a70 0x7f1a2c169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.720+0000 7f1a325ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c151340 con 0x7f1a2c0b0870 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.720+0000 7f1a325ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0cb740 con 0x7f1a2c159a70 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.720+0000 7f1a325ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0cc560 con 0x7f1a2c1596a0 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.720+0000 7f1a2bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c0b0870 0x7f1a2c0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.720+0000 7f1a2bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c0b0870 0x7f1a2c0ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42276/0 (socket says 172.21.15.131:42276) 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.720+0000 7f1a2bfff700 1 -- 172.21.15.131:0/3241291069 learned_addr learned my addr 172.21.15.131:0/3241291069 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a2bfff700 1 -- 172.21.15.131:0/3241291069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a2c1596a0 msgr2=0x7f1a2c0cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a2bfff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a2c1596a0 0x7f1a2c0cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a2bfff700 1 -- 172.21.15.131:0/3241291069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a2c159a70 msgr2=0x7f1a2c169a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a2bfff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a2c159a70 0x7f1a2c169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a2bfff700 1 -- 172.21.15.131:0/3241291069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c009420 con 0x7f1a2c0b0870 2023-12-26T23:09:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a2bfff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c0b0870 0x7f1a2c0ca140 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c089710 tx=0x7f1a1c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a13fff700 1 -- 172.21.15.131:0/3241291069 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00bb60 con 0x7f1a2c0b0870 2023-12-26T23:09:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a13fff700 1 -- 172.21.15.131:0/3241291069 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c01b680 con 0x7f1a2c0b0870 2023-12-26T23:09:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a13fff700 1 -- 172.21.15.131:0/3241291069 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c01ba40 con 0x7f1a2c0b0870 2023-12-26T23:09:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c0cb950 con 0x7f1a2c0b0870 2023-12-26T23:09:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.721+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c0cbf10 con 0x7f1a2c0b0870 2023-12-26T23:09:31.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.722+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c0ca680 con 0x7f1a2c0b0870 2023-12-26T23:09:31.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.723+0000 7f1a13fff700 1 -- 172.21.15.131:0/3241291069 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1a1c01c450 con 0x7f1a2c0b0870 2023-12-26T23:09:31.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.724+0000 7f1a13fff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1a18066f30 0x7f1a180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:31.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.724+0000 7f1a13fff700 1 -- 172.21.15.131:0/3241291069 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6291+0+0 (secure 0 0 0) 0x7f1a1c0866d0 con 0x7f1a2c0b0870 2023-12-26T23:09:31.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.724+0000 7f1a2b7fe700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1a18066f30 0x7f1a180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:31.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.725+0000 7f1a2b7fe700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1a18066f30 0x7f1a180693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1a20007fd0 tx=0x7f1a20007b20 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.728+0000 7f1a13fff700 1 -- 172.21.15.131:0/3241291069 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a1c053650 con 0x7f1a2c0b0870 2023-12-26T23:09:31.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:31.997+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f1a2c0657e0 con 0x7f1a2c0b0870 2023-12-26T23:09:32.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.503+0000 7f1a13fff700 1 -- 172.21.15.131:0/3241291069 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2979) v1 ==== 166+0+0 (secure 0 0 0) 0x7f1a1c056c70 con 0x7f1a2c0b0870 2023-12-26T23:09:32.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1a18066f30 msgr2=0x7f1a180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:32.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1a18066f30 0x7f1a180693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1a20007fd0 tx=0x7f1a20007b20 comp rx=0 tx=0).stop 2023-12-26T23:09:32.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c0b0870 msgr2=0x7f1a2c0ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:32.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c0b0870 0x7f1a2c0ca140 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c089710 tx=0x7f1a1c016000 comp rx=0 tx=0).stop 2023-12-26T23:09:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 shutdown_connections 2023-12-26T23:09:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1a18066f30 0x7f1a180693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a2c159a70 0x7f1a2c169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a2c0b0870 0x7f1a2c0ca140 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 --2- 172.21.15.131:0/3241291069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a2c1596a0 0x7f1a2c0cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 >> 172.21.15.131:0/3241291069 conn(0x7f1a2c06ece0 msgr2=0x7f1a2c0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.506+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 shutdown_connections 2023-12-26T23:09:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.507+0000 7f1a325ff700 1 -- 172.21.15.131:0/3241291069 wait complete. 2023-12-26T23:09:32.527 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T23:09:32.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.721+0000 7f2d769ff700 1 Processor -- start 2023-12-26T23:09:32.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.721+0000 7f2d769ff700 1 -- start start 2023-12-26T23:09:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.722+0000 7f2d769ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d700c8bf0 0x7f2d700b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:32.726 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.722+0000 7f2d769ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d700caa10 con 0x7f2d70069d10 2023-12-26T23:09:32.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.722+0000 7f2d769ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d70157a80 con 0x7f2d700c8820 2023-12-26T23:09:32.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.722+0000 7f2d769ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700cac50 con 0x7f2d700c8bf0 2023-12-26T23:09:32.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d74f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2d70069d10 0x7f2d70161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49356/0 (socket says 172.21.15.131:49356) 2023-12-26T23:09:32.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d74f9c700 1 -- 172.21.15.131:0/1005002191 learned_addr learned my addr 172.21.15.131:0/1005002191 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:32.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6f7fe700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d700c8bf0 0x7f2d700b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:32.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338220837 0 0) 0x7f2d68001490 con 0x7f2d70069d10 2023-12-26T23:09:32.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d58003570 con 0x7f2d70069d10 2023-12-26T23:09:32.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1914246622 0 0) 0x7f2d58003570 con 0x7f2d70069d10 2023-12-26T23:09:32.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d700c8bf0 msgr2=0x7f2d700b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:32.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6effd700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d700c8bf0 0x7f2d700b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 >> v1:172.21.15.190:6789/0 conn(0x7f2d700c8820 legacy=0x7f2d700ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:32.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700caf30 con 0x7f2d70069d10 2023-12-26T23:09:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f2d68001980 con 0x7f2d70069d10 2023-12-26T23:09:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.723+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 >> v1:172.21.15.131:6789/0 conn(0x7f2d70069d10 legacy=0x7f2d70161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6effd700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d58003a90 0x7f2d58003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6effd700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d58004810 0x7f2d58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6effd700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d700c8bf0 0x7f2d580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700cac50 con 0x7f2d58003a90 2023-12-26T23:09:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800bb10 con 0x7f2d58004810 2023-12-26T23:09:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800bcd0 con 0x7f2d700c8bf0 2023-12-26T23:09:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6ffff700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d58003a90 0x7f2d58003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6f7fe700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d58004810 0x7f2d58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d74f9c700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d700c8bf0 0x7f2d580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6ffff700 1 -- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d700c8bf0 msgr2=0x7f2d580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6ffff700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d700c8bf0 0x7f2d580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6ffff700 1 -- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d58004810 msgr2=0x7f2d58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6ffff700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d58004810 0x7f2d58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.724+0000 7f2d6ffff700 1 -- 172.21.15.131:0/1005002191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58003570 con 0x7f2d58003a90 2023-12-26T23:09:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d6ffff700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d58003a90 0x7f2d58003e60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2d600019a0 tx=0x7f2d60007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c954bfa990968280 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:32.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000c5d0 con 0x7f2d58003a90 2023-12-26T23:09:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6000c730 con 0x7f2d58003a90 2023-12-26T23:09:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d6effd700 1 -- 172.21.15.131:0/1005002191 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000cad0 con 0x7f2d58003a90 2023-12-26T23:09:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d769ff700 1 -- 172.21.15.131:0/1005002191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d58003a90 msgr2=0x7f2d58003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d769ff700 1 --2- 172.21.15.131:0/1005002191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d58003a90 0x7f2d58003e60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2d600019a0 tx=0x7f2d60007d10 comp rx=0 tx=0).stop 2023-12-26T23:09:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d74f9c700 1 -- 172.21.15.131:0/1005002191 reap_dead start 2023-12-26T23:09:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d769ff700 1 -- 172.21.15.131:0/1005002191 shutdown_connections 2023-12-26T23:09:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d769ff700 1 -- 172.21.15.131:0/1005002191 >> 172.21.15.131:0/1005002191 conn(0x7f2d70068040 msgr2=0x7f2d70069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d769ff700 1 -- 172.21.15.131:0/1005002191 shutdown_connections 2023-12-26T23:09:32.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.725+0000 7f2d769ff700 1 -- 172.21.15.131:0/1005002191 wait complete. 2023-12-26T23:09:32.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.727+0000 7f2d769ff700 1 Processor -- start 2023-12-26T23:09:32.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.727+0000 7f2d769ff700 1 -- start start 2023-12-26T23:09:32.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d769ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d58003a90 0x7f2d700acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:32.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d769ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d70141c00 0x7f2d700c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:32.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d769ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d700c78f0 0x7f2d700ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:32.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d769ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700c7d60 con 0x7f2d70141c00 2023-12-26T23:09:32.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d769ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700aca30 con 0x7f2d700c78f0 2023-12-26T23:09:32.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d769ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700abe20 con 0x7f2d58003a90 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d6f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d70141c00 0x7f2d700c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d6f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d70141c00 0x7f2d700c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42294/0 (socket says 172.21.15.131:42294) 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d6f7fe700 1 -- 172.21.15.131:0/1725798162 learned_addr learned my addr 172.21.15.131:0/1725798162 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d74f9c700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d700c78f0 0x7f2d700ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.728+0000 7f2d6ffff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d58003a90 0x7f2d700acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d6f7fe700 1 -- 172.21.15.131:0/1725798162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d58003a90 msgr2=0x7f2d700acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d6f7fe700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d58003a90 0x7f2d700acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d6f7fe700 1 -- 172.21.15.131:0/1725798162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d700c78f0 msgr2=0x7f2d700ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d6f7fe700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d700c78f0 0x7f2d700ab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d6f7fe700 1 -- 172.21.15.131:0/1725798162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700ac100 con 0x7f2d70141c00 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d6f7fe700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d70141c00 0x7f2d700c73b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2d64011d40 tx=0x7f2d640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d57fff700 1 -- 172.21.15.131:0/1725798162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400fac0 con 0x7f2d70141c00 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d57fff700 1 -- 172.21.15.131:0/1725798162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400fc20 con 0x7f2d70141c00 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d57fff700 1 -- 172.21.15.131:0/1725798162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64014440 con 0x7f2d70141c00 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d700ac3e0 con 0x7f2d70141c00 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.729+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d70142070 con 0x7f2d70141c00 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.730+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d700ac880 con 0x7f2d70141c00 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.732+0000 7f2d57fff700 1 -- 172.21.15.131:0/1725798162 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2d64014870 con 0x7f2d70141c00 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.733+0000 7f2d57fff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2d5c066f30 0x7f2d5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:32.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.733+0000 7f2d57fff700 1 -- 172.21.15.131:0/1725798162 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6291+0+0 (secure 0 0 0) 0x7f2d64018030 con 0x7f2d70141c00 2023-12-26T23:09:32.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.733+0000 7f2d6ffff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2d5c066f30 0x7f2d5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:32.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.733+0000 7f2d6ffff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2d5c066f30 0x7f2d5c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2d60000fd0 tx=0x7f2d60015000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:32.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.736+0000 7f2d57fff700 1 -- 172.21.15.131:0/1725798162 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d64052210 con 0x7f2d70141c00 2023-12-26T23:09:32.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.929+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2d700acba0 con 0x7f2d70141c00 2023-12-26T23:09:32.933 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.933+0000 7f2d57fff700 1 -- 172.21.15.131:0/1725798162 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2d64059e60 con 0x7f2d70141c00 2023-12-26T23:09:32.936 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2d5c066f30 msgr2=0x7f2d5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2d5c066f30 0x7f2d5c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2d60000fd0 tx=0x7f2d60015000 comp rx=0 tx=0).stop 2023-12-26T23:09:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d70141c00 msgr2=0x7f2d700c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d70141c00 0x7f2d700c73b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2d64011d40 tx=0x7f2d640091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 shutdown_connections 2023-12-26T23:09:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2d5c066f30 0x7f2d5c0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d700c78f0 0x7f2d700ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d58003a90 0x7f2d700acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.937 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 --2- 172.21.15.131:0/1725798162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d70141c00 0x7f2d700c73b0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:32.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 >> 172.21.15.131:0/1725798162 conn(0x7f2d70068040 msgr2=0x7f2d700cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:32.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 shutdown_connections 2023-12-26T23:09:32.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:32.936+0000 7f2d769ff700 1 -- 172.21.15.131:0/1725798162 wait complete. 2023-12-26T23:09:32.955 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T23:09:32.956 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:09:33.054 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:09:33.158 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T23:09:33.158 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T23:09:33.158 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:09:33.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.347+0000 7f08833ff700 1 Processor -- start 2023-12-26T23:09:33.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.347+0000 7f08833ff700 1 -- start start 2023-12-26T23:09:33.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.348+0000 7f08833ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f087c0cc580 0x7f087c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:33.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.348+0000 7f08833ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f087c0b0560 0x7f087c15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.348+0000 7f08833ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c0b0d70 con 0x7f087c0b00e0 2023-12-26T23:09:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.348+0000 7f08833ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c0680f0 con 0x7f087c0b0560 2023-12-26T23:09:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.348+0000 7f08833ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c0675b0 con 0x7f087c0cc580 2023-12-26T23:09:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088199c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f087c0b0560 0x7f087c15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088199c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f087c0b0560 0x7f087c15d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42296/0 (socket says 172.21.15.131:42296) 2023-12-26T23:09:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088199c700 1 -- 172.21.15.131:0/234505110 learned_addr learned my addr 172.21.15.131:0/234505110 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088099a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f087c0b00e0 0x7f087c06e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49360/0 (socket says 172.21.15.131:49360) 2023-12-26T23:09:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088119b700 1 --2- 172.21.15.131:0/234505110 >> v2:172.21.15.190:3300/0 conn(0x7f087c0cc580 0x7f087c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088199c700 1 -- 172.21.15.131:0/234505110 >> v2:172.21.15.190:3300/0 conn(0x7f087c0cc580 msgr2=0x7f087c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088199c700 1 --2- 172.21.15.131:0/234505110 >> v2:172.21.15.190:3300/0 conn(0x7f087c0cc580 0x7f087c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088199c700 1 -- 172.21.15.131:0/234505110 >> v1:172.21.15.131:6789/0 conn(0x7f087c0b00e0 legacy=0x7f087c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088199c700 1 -- 172.21.15.131:0/234505110 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c159c30 con 0x7f087c0b0560 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f0873fff700 1 -- 172.21.15.131:0/234505110 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954989271 0 0) 0x7f087c0b0d70 con 0x7f087c0b00e0 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.349+0000 7f088199c700 1 --2- 172.21.15.131:0/234505110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 0x7f087c15d9b0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f087c06b740 tx=0x7f0874009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0b49ef201a03c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f0873fff700 1 -- 172.21.15.131:0/234505110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087400e070 con 0x7f087c0b0560 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f0873fff700 1 -- 172.21.15.131:0/234505110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087400ba20 con 0x7f087c0b0560 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f0873fff700 1 -- 172.21.15.131:0/234505110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08740092e0 con 0x7f087c0b0560 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f08833ff700 1 -- 172.21.15.131:0/234505110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 msgr2=0x7f087c15d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:33.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f08833ff700 1 --2- 172.21.15.131:0/234505110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 0x7f087c15d9b0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f087c06b740 tx=0x7f0874009840 comp rx=0 tx=0).stop 2023-12-26T23:09:33.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f08833ff700 1 -- 172.21.15.131:0/234505110 shutdown_connections 2023-12-26T23:09:33.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f08833ff700 1 --2- 172.21.15.131:0/234505110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 0x7f087c15d9b0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:33.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f08833ff700 1 --2- 172.21.15.131:0/234505110 >> v2:172.21.15.190:3300/0 conn(0x7f087c0cc580 0x7f087c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:33.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f08833ff700 1 -- 172.21.15.131:0/234505110 >> 172.21.15.131:0/234505110 conn(0x7f087c0686c0 msgr2=0x7f087c068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:33.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f08833ff700 1 -- 172.21.15.131:0/234505110 shutdown_connections 2023-12-26T23:09:33.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.350+0000 7f08833ff700 1 -- 172.21.15.131:0/234505110 wait complete. 2023-12-26T23:09:33.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.352+0000 7f08833ff700 1 Processor -- start 2023-12-26T23:09:33.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.353+0000 7f08833ff700 1 -- start start 2023-12-26T23:09:33.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f08833ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f087c0b00e0 0x7f087c169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:33.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f08833ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 0x7f087c0acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f08833ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087c0cc580 0x7f087c0c4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f08833ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0ae620 con 0x7f087c0b0560 2023-12-26T23:09:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f08833ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0ae190 con 0x7f087c0b00e0 2023-12-26T23:09:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f08833ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0ad580 con 0x7f087c0cc580 2023-12-26T23:09:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 0x7f087c0acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088099a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 0x7f087c0acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42310/0 (socket says 172.21.15.131:42310) 2023-12-26T23:09:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088099a700 1 -- 172.21.15.131:0/853589986 learned_addr learned my addr 172.21.15.131:0/853589986 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088119b700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f087c0b00e0 0x7f087c169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:33.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088199c700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087c0cc580 0x7f087c0c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088099a700 1 -- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087c0cc580 msgr2=0x7f087c0c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088099a700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087c0cc580 0x7f087c0c4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088099a700 1 -- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f087c0b00e0 msgr2=0x7f087c169fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088099a700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f087c0b00e0 0x7f087c169fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088099a700 1 -- 172.21.15.131:0/853589986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0874009ae0 con 0x7f087c0b0560 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088119b700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f087c0b00e0 0x7f087c169fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.354+0000 7f088199c700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087c0cc580 0x7f087c0c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.355+0000 7f088099a700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 0x7f087c0acba0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c7e0 tx=0x7f086c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.355+0000 7f08717fa700 1 -- 172.21.15.131:0/853589986 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c007300 con 0x7f087c0b0560 2023-12-26T23:09:33.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.355+0000 7f08717fa700 1 -- 172.21.15.131:0/853589986 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c007460 con 0x7f087c0b0560 2023-12-26T23:09:33.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.355+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c0ad860 con 0x7f087c0b0560 2023-12-26T23:09:33.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.355+0000 7f08717fa700 1 -- 172.21.15.131:0/853589986 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c004830 con 0x7f087c0b0560 2023-12-26T23:09:33.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.355+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c0ade20 con 0x7f087c0b0560 2023-12-26T23:09:33.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.356+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c0ae340 con 0x7f087c0b0560 2023-12-26T23:09:33.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.357+0000 7f08717fa700 1 -- 172.21.15.131:0/853589986 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f086c004be0 con 0x7f087c0b0560 2023-12-26T23:09:33.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.358+0000 7f08717fa700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0860066f30 0x7f08600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:33.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.358+0000 7f08717fa700 1 -- 172.21.15.131:0/853589986 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6291+0+0 (secure 0 0 0) 0x7f086c083d60 con 0x7f087c0b0560 2023-12-26T23:09:33.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.358+0000 7f088119b700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0860066f30 0x7f08600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:33.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.359+0000 7f088119b700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0860066f30 0x7f08600693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0878006f00 tx=0x7f0878008040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:33.361 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.362+0000 7f08717fa700 1 -- 172.21.15.131:0/853589986 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f087c0ae340 con 0x7f087c0b0560 2023-12-26T23:09:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.599+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f087c0ae340 con 0x7f087c0b0560 2023-12-26T23:09:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.599+0000 7f08717fa700 1 -- 172.21.15.131:0/853589986 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f087c0ae340 con 0x7f087c0b0560 2023-12-26T23:09:33.599 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:33.600 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2545294570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2545294570},{"type":"v1","addr":"172.21.15.190:6835","nonce":2545294570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9031,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4101223120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4101223120},{"type":"v1","addr":"172.21.15.190:6834","nonce":4101223120}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1800615106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1800615106},{"type":"v1","addr":"172.21.15.131:6834","nonce":1800615106}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3279484094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3279484094},{"type":"v1","addr":"172.21.15.131:6835","nonce":3279484094}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[]} 2023-12-26T23:09:33.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.601+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0860066f30 msgr2=0x7f08600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:33.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.601+0000 7f08833ff700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0860066f30 0x7f08600693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0878006f00 tx=0x7f0878008040 comp rx=0 tx=0).stop 2023-12-26T23:09:33.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.601+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 msgr2=0x7f087c0acba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:33.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.601+0000 7f08833ff700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 0x7f087c0acba0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c7e0 tx=0x7f086c007e10 comp rx=0 tx=0).stop 2023-12-26T23:09:33.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.602+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 shutdown_connections 2023-12-26T23:09:33.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.602+0000 7f08833ff700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0860066f30 0x7f08600693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:33.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.602+0000 7f08833ff700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087c0cc580 0x7f087c0c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:33.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.602+0000 7f08833ff700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f087c0b00e0 0x7f087c169fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:33.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.602+0000 7f08833ff700 1 --2- 172.21.15.131:0/853589986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087c0b0560 0x7f087c0acba0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:33.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.602+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 >> 172.21.15.131:0/853589986 conn(0x7f087c0686c0 msgr2=0x7f087c0cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:33.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.602+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 shutdown_connections 2023-12-26T23:09:33.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:33.602+0000 7f08833ff700 1 -- 172.21.15.131:0/853589986 wait complete. 2023-12-26T23:09:33.604 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 83 2023-12-26T23:09:33.620 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T23:09:33.620 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T23:09:33.620 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T23:09:33.620 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:09:33.621 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T23:09:33.621 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T23:09:33.621 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T23:09:33.621 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:09:33.622 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T23:09:33.622 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T23:09:33.622 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T23:09:33.622 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:09:33.622 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T23:09:33.622 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T23:09:33.623 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T23:09:33.623 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:09:33.637 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T23:09:33.637 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T23:09:33.639 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T23:09:33.639 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T23:09:33.641 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T23:09:33.641 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T23:09:33.682 INFO:tasks.ceph.mds.c:Started 2023-12-26T23:09:33.683 DEBUG:teuthology.parallel:result is None 2023-12-26T23:09:33.684 INFO:tasks.ceph.mds.b:Started 2023-12-26T23:09:33.685 DEBUG:teuthology.parallel:result is None 2023-12-26T23:09:33.688 INFO:tasks.ceph.mds.d:Started 2023-12-26T23:09:33.689 DEBUG:teuthology.parallel:result is None 2023-12-26T23:09:33.817 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T23:09:33.832 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T23:09:33.832 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T23:09:39.637 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T23:09:39.638 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T23:09:39.644 INFO:tasks.ceph.mds.a:Started 2023-12-26T23:09:39.645 DEBUG:teuthology.parallel:result is None 2023-12-26T23:09:39.645 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T23:09:39.791 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T23:09:39.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.848+0000 7f61d99ff700 1 Processor -- start 2023-12-26T23:09:39.848 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.849+0000 7f61d99ff700 1 -- start start 2023-12-26T23:09:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.850+0000 7f61d99ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f61d40c8820 0x7f61d40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.850+0000 7f61d99ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c8bf0 0x7f61d40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.850+0000 7f61d99ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d4157a80 con 0x7f61d4069d10 2023-12-26T23:09:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.850+0000 7f61d99ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d40cac50 con 0x7f61d40c8820 2023-12-26T23:09:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.850+0000 7f61d99ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40caf30 con 0x7f61d40c8bf0 2023-12-26T23:09:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.850+0000 7f61d37fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f61d4069d10 0x7f61d4157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60536/0 (socket says 172.21.15.131:60536) 2023-12-26T23:09:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.850+0000 7f61d37fe700 1 -- 172.21.15.131:0/744115047 learned_addr learned my addr 172.21.15.131:0/744115047 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.850+0000 7f61d2ffd700 1 --2- 172.21.15.131:0/744115047 >> v2:172.21.15.190:3300/0 conn(0x7f61d40c8820 0x7f61d40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d27fc700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c8bf0 0x7f61d40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038813010 0 0) 0x7f61d4157a80 con 0x7f61d4069d10 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61bc003570 con 0x7f61d4069d10 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2736910992 0 0) 0x7f61bc003570 con 0x7f61d4069d10 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c8bf0 msgr2=0x7f61d40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c8bf0 0x7f61d40b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 >> v2:172.21.15.190:3300/0 conn(0x7f61d40c8820 msgr2=0x7f61d40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 --2- 172.21.15.131:0/744115047 >> v2:172.21.15.190:3300/0 conn(0x7f61d40c8820 0x7f61d40ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40cb210 con 0x7f61d4069d10 2023-12-26T23:09:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f61cc0017f0 con 0x7f61d4069d10 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 >> v1:172.21.15.131:6789/0 conn(0x7f61d4069d10 legacy=0x7f61d4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c8bf0 0x7f61bc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61bc004d50 0x7f61bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40cac50 con 0x7f61bc003fe0 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40caf30 con 0x7f61bc004d50 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc00ba90 con 0x7f61d40c8bf0 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d27fc700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d37fe700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61bc004d50 0x7f61bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:39.855 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.851+0000 7f61d2ffd700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c8bf0 0x7f61bc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.852+0000 7f61d27fc700 1 -- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c8bf0 msgr2=0x7f61bc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.852+0000 7f61d27fc700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c8bf0 0x7f61bc003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.852+0000 7f61d27fc700 1 -- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61bc004d50 msgr2=0x7f61bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.852+0000 7f61d27fc700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61bc004d50 0x7f61bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.852+0000 7f61d27fc700 1 -- 172.21.15.131:0/744115047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc003570 con 0x7f61bc003fe0 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.852+0000 7f61d27fc700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61bc004810 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009c00 tx=0x7f61c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdd96e85a48641c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.852+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800e6b0 con 0x7f61bc003fe0 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.852+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c800e810 con 0x7f61bc003fe0 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.852+0000 7f61d1ffb700 1 -- 172.21.15.131:0/744115047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800ebd0 con 0x7f61bc003fe0 2023-12-26T23:09:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.853+0000 7f61d99ff700 1 -- 172.21.15.131:0/744115047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 msgr2=0x7f61bc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.853+0000 7f61d99ff700 1 --2- 172.21.15.131:0/744115047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61bc004810 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009c00 tx=0x7f61c80091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.853+0000 7f61d37fe700 1 -- 172.21.15.131:0/744115047 reap_dead start 2023-12-26T23:09:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.853+0000 7f61d99ff700 1 -- 172.21.15.131:0/744115047 shutdown_connections 2023-12-26T23:09:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.853+0000 7f61d99ff700 1 -- 172.21.15.131:0/744115047 >> 172.21.15.131:0/744115047 conn(0x7f61d4068040 msgr2=0x7f61d4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.853+0000 7f61d99ff700 1 -- 172.21.15.131:0/744115047 shutdown_connections 2023-12-26T23:09:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.853+0000 7f61d99ff700 1 -- 172.21.15.131:0/744115047 wait complete. 2023-12-26T23:09:39.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.854+0000 7f61d99ff700 1 Processor -- start 2023-12-26T23:09:39.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.855+0000 7f61d99ff700 1 -- start start 2023-12-26T23:09:39.860 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.855+0000 7f61d99ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61d4147c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d99ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d40a65f0 0x7f61d40a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d99ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c6d60 0x7f61d40c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d99ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d416a620 con 0x7f61bc003fe0 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d99ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40c76f0 con 0x7f61d40a65f0 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d99ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d4169a10 con 0x7f61d40c6d60 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61d4147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61d4147c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60522/0 (socket says 172.21.15.131:60522) 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d2ffd700 1 -- 172.21.15.131:0/3230948120 learned_addr learned my addr 172.21.15.131:0/3230948120 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d37fe700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c6d60 0x7f61d40c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d27fc700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d40a65f0 0x7f61d40a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d2ffd700 1 -- 172.21.15.131:0/3230948120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c6d60 msgr2=0x7f61d40c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d2ffd700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c6d60 0x7f61d40c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d2ffd700 1 -- 172.21.15.131:0/3230948120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d40a65f0 msgr2=0x7f61d40a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d2ffd700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d40a65f0 0x7f61d40a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d2ffd700 1 -- 172.21.15.131:0/3230948120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4169cf0 con 0x7f61bc003fe0 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d27fc700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d40a65f0 0x7f61d40a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.856+0000 7f61d2ffd700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61d4147c60 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f61c4011d40 tx=0x7f61c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.857+0000 7f61baffd700 1 -- 172.21.15.131:0/3230948120 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c400fac0 con 0x7f61bc003fe0 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.857+0000 7f61baffd700 1 -- 172.21.15.131:0/3230948120 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c400fc20 con 0x7f61bc003fe0 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.857+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d4169fd0 con 0x7f61bc003fe0 2023-12-26T23:09:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.857+0000 7f61baffd700 1 -- 172.21.15.131:0/3230948120 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4014440 con 0x7f61bc003fe0 2023-12-26T23:09:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.857+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d41481a0 con 0x7f61bc003fe0 2023-12-26T23:09:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.858+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d415abb0 con 0x7f61bc003fe0 2023-12-26T23:09:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.859+0000 7f61baffd700 1 -- 172.21.15.131:0/3230948120 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f61c4014870 con 0x7f61bc003fe0 2023-12-26T23:09:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.859+0000 7f61baffd700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61c0066f30 0x7f61c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.860+0000 7f61baffd700 1 -- 172.21.15.131:0/3230948120 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6291+0+0 (secure 0 0 0) 0x7f61c4018030 con 0x7f61bc003fe0 2023-12-26T23:09:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.860+0000 7f61d27fc700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61c0066f30 0x7f61c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.860+0000 7f61d27fc700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61c0066f30 0x7f61c00693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f61c80091e0 tx=0x7f61c801a000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:39.864 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:39.864+0000 7f61baffd700 1 -- 172.21.15.131:0/3230948120 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61c4052260 con 0x7f61bc003fe0 2023-12-26T23:09:39.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-12-26T23:09:40.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:40.073+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f61d40657e0 con 0x7f61bc003fe0 2023-12-26T23:09:41.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.372+0000 7f61baffd700 1 -- 172.21.15.131:0/3230948120 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v189) v1 ==== 97+0+0 (secure 0 0 0) 0x7f61c4055880 con 0x7f61bc003fe0 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.375+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61c0066f30 msgr2=0x7f61c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.375+0000 7f61d99ff700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61c0066f30 0x7f61c00693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f61c80091e0 tx=0x7f61c801a000 comp rx=0 tx=0).stop 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.375+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 msgr2=0x7f61d4147c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.375+0000 7f61d99ff700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61d4147c60 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f61c4011d40 tx=0x7f61c40091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.376+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 shutdown_connections 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.376+0000 7f61d99ff700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61c0066f30 0x7f61c00693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.376+0000 7f61d99ff700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d40c6d60 0x7f61d40c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.376+0000 7f61d99ff700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61bc003fe0 0x7f61d4147c60 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.376+0000 7f61d99ff700 1 --2- 172.21.15.131:0/3230948120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d40a65f0 0x7f61d40a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.376+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 >> 172.21.15.131:0/3230948120 conn(0x7f61d4068040 msgr2=0x7f61d40c17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:41.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.376+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 shutdown_connections 2023-12-26T23:09:41.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.376+0000 7f61d99ff700 1 -- 172.21.15.131:0/3230948120 wait complete. 2023-12-26T23:09:41.376 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T23:09:41.395 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T23:09:41.589 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.591+0000 7f599bfff700 1 Processor -- start 2023-12-26T23:09:41.590 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.591+0000 7f599bfff700 1 -- start start 2023-12-26T23:09:41.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.592+0000 7f599bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 0x7f59940b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:41.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.592+0000 7f599bfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59940caa10 con 0x7f5994069d10 2023-12-26T23:09:41.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.592+0000 7f599bfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5994157a80 con 0x7f59940c8820 2023-12-26T23:09:41.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.592+0000 7f599bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59940cac50 con 0x7f59940c8bf0 2023-12-26T23:09:41.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.592+0000 7f599a59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5994069d10 0x7f5994161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60544/0 (socket says 172.21.15.131:60544) 2023-12-26T23:09:41.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.592+0000 7f599a59c700 1 -- 172.21.15.131:0/3931252597 learned_addr learned my addr 172.21.15.131:0/3931252597 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:41.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f599959a700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 0x7f59940b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:41.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733126598 0 0) 0x7f598c001490 con 0x7f5994069d10 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5980003570 con 0x7f5994069d10 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115679874 0 0) 0x7f5994157a80 con 0x7f59940c8820 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f598c001490 con 0x7f59940c8820 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3047795355 0 0) 0x7f5980003570 con 0x7f5994069d10 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 msgr2=0x7f59940b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 0x7f59940b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 >> v1:172.21.15.190:6789/0 conn(0x7f59940c8820 legacy=0x7f59940ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59940caf30 con 0x7f5994069d10 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f598c001980 con 0x7f5994069d10 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f599959a700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 0x7f59940b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 >> v1:172.21.15.131:6789/0 conn(0x7f5994069d10 legacy=0x7f5994161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5980003c20 0x7f5980004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 0x7f5980004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5980004d50 0x7f59800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:41.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59940cac50 con 0x7f5980004d50 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598000bb10 con 0x7f5980003c20 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.593+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598000bcd0 con 0x7f59940c8bf0 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f5999d9b700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5980003c20 0x7f5980004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f599959a700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 0x7f5980004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f599a59c700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5980004d50 0x7f59800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f599a59c700 1 -- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 msgr2=0x7f5980004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f599a59c700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 0x7f5980004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f599a59c700 1 -- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5980003c20 msgr2=0x7f5980004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f599a59c700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5980003c20 0x7f5980004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f599a59c700 1 -- 172.21.15.131:0/3931252597 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984009420 con 0x7f5980004d50 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f599959a700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59940c8bf0 0x7f5980004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f5999d9b700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5980003c20 0x7f5980004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f599a59c700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5980004d50 0x7f59800051c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f598c00aa70 tx=0x7f598c00a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a9c9eb9b6badac server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:41.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c00a400 con 0x7f5980004d50 2023-12-26T23:09:41.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c00c5d0 con 0x7f5980004d50 2023-12-26T23:09:41.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.594+0000 7f5998d99700 1 -- 172.21.15.131:0/3931252597 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c00c990 con 0x7f5980004d50 2023-12-26T23:09:41.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.595+0000 7f599bfff700 1 -- 172.21.15.131:0/3931252597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5980004d50 msgr2=0x7f59800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:41.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.595+0000 7f599bfff700 1 --2- 172.21.15.131:0/3931252597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5980004d50 0x7f59800051c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f598c00aa70 tx=0x7f598c00a0f0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.595+0000 7f599a59c700 1 -- 172.21.15.131:0/3931252597 reap_dead start 2023-12-26T23:09:41.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.595+0000 7f599bfff700 1 -- 172.21.15.131:0/3931252597 shutdown_connections 2023-12-26T23:09:41.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.595+0000 7f599bfff700 1 -- 172.21.15.131:0/3931252597 >> 172.21.15.131:0/3931252597 conn(0x7f5994068040 msgr2=0x7f5994069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:41.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.595+0000 7f599bfff700 1 -- 172.21.15.131:0/3931252597 shutdown_connections 2023-12-26T23:09:41.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.595+0000 7f599bfff700 1 -- 172.21.15.131:0/3931252597 wait complete. 2023-12-26T23:09:41.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.596+0000 7f599bfff700 1 Processor -- start 2023-12-26T23:09:41.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.597+0000 7f599bfff700 1 -- start start 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.597+0000 7f599bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004d50 0x7f5994157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.597+0000 7f599bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5994142ad0 0x7f5994158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.597+0000 7f599bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f599415e830 0x7f599415eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.597+0000 7f599bfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5994156a90 con 0x7f5994142ad0 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.597+0000 7f599bfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599416af60 con 0x7f599415e830 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.597+0000 7f599bfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599415f1e0 con 0x7f5980004d50 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5994142ad0 0x7f5994158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599959a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5994142ad0 0x7f5994158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60542/0 (socket says 172.21.15.131:60542) 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599959a700 1 -- 172.21.15.131:0/2026655804 learned_addr learned my addr 172.21.15.131:0/2026655804 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f5999d9b700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004d50 0x7f5994157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599a59c700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f599415e830 0x7f599415eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599959a700 1 -- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004d50 msgr2=0x7f5994157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599959a700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004d50 0x7f5994157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599959a700 1 -- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f599415e830 msgr2=0x7f599415eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599959a700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f599415e830 0x7f599415eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599959a700 1 -- 172.21.15.131:0/2026655804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5994158a10 con 0x7f5994142ad0 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f5999d9b700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004d50 0x7f5994157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599a59c700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f599415e830 0x7f599415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f599959a700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5994142ad0 0x7f5994158370 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f5984011eb0 tx=0x7f59840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.598+0000 7f5989ffb700 1 -- 172.21.15.131:0/2026655804 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598400fe70 con 0x7f5994142ad0 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.599+0000 7f5989ffb700 1 -- 172.21.15.131:0/2026655804 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598401d480 con 0x7f5994142ad0 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.599+0000 7f5989ffb700 1 -- 172.21.15.131:0/2026655804 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598401d800 con 0x7f5994142ad0 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.599+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599416a910 con 0x7f5994142ad0 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.599+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5994158cf0 con 0x7f5994142ad0 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.600+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599416b0d0 con 0x7f5994142ad0 2023-12-26T23:09:41.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.601+0000 7f5989ffb700 1 -- 172.21.15.131:0/2026655804 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f598401dbf0 con 0x7f5994142ad0 2023-12-26T23:09:41.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.601+0000 7f5989ffb700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5978066f30 0x7f59780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:41.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.601+0000 7f5989ffb700 1 -- 172.21.15.131:0/2026655804 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5345+0+0 (secure 0 0 0) 0x7f5984018070 con 0x7f5994142ad0 2023-12-26T23:09:41.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.602+0000 7f5999d9b700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5978066f30 0x7f59780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:41.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.602+0000 7f5999d9b700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5978066f30 0x7f59780693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5990007bc0 tx=0x7f5990009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:41.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.605+0000 7f5989ffb700 1 -- 172.21.15.131:0/2026655804 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59840528a0 con 0x7f5994142ad0 2023-12-26T23:09:41.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.870+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f59940657e0 con 0x7f5994142ad0 2023-12-26T23:09:41.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.870+0000 7f5989ffb700 1 -- 172.21.15.131:0/2026655804 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v48) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5984055ec0 con 0x7f5994142ad0 2023-12-26T23:09:41.869 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:41.869 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:09:41.869 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T23:09:41.869 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.869 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:09:41.870 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.871 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:09:41.872 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T23:09:41.873 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.874 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T23:09:41.876 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.877 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T23:09:41.878 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T23:09:41.879 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T23:09:41.880 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.881 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:41.882 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T23:09:41.882 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.872+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5978066f30 msgr2=0x7f59780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.872+0000 7f599bfff700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5978066f30 0x7f59780693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5990007bc0 tx=0x7f5990009040 comp rx=0 tx=0).stop 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5994142ad0 msgr2=0x7f5994158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5994142ad0 0x7f5994158370 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f5984011eb0 tx=0x7f59840091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 shutdown_connections 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5978066f30 0x7f59780693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f599415e830 0x7f599415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004d50 0x7f5994157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 --2- 172.21.15.131:0/2026655804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5994142ad0 0x7f5994158370 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 >> 172.21.15.131:0/2026655804 conn(0x7f5994068040 msgr2=0x7f59940cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 shutdown_connections 2023-12-26T23:09:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:41.873+0000 7f599bfff700 1 -- 172.21.15.131:0/2026655804 wait complete. 2023-12-26T23:09:41.890 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:09:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.080+0000 7f5dfe5ff700 1 Processor -- start 2023-12-26T23:09:42.079 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.080+0000 7f5dfe5ff700 1 -- start start 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.081+0000 7f5dfe5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5df80c8bf0 0x7f5df80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.081+0000 7f5dfe5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df8069d10 0x7f5df80b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.081+0000 7f5dfe5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df80caa10 con 0x7f5df80c8820 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.081+0000 7f5dfe5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df8157a80 con 0x7f5df80c8bf0 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.081+0000 7f5dfe5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80cac50 con 0x7f5df8069d10 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.081+0000 7f5df77fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5df80c8bf0 0x7f5df80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5df77fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5df80c8bf0 0x7f5df80b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60558/0 (socket says 172.21.15.131:60558) 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5df77fe700 1 -- 172.21.15.131:0/553259379 learned_addr learned my addr 172.21.15.131:0/553259379 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5dfcb9c700 1 --2- 172.21.15.131:0/553259379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df8069d10 0x7f5df80b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5df77fe700 1 -- 172.21.15.131:0/553259379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df8069d10 msgr2=0x7f5df80b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5df77fe700 1 --2- 172.21.15.131:0/553259379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df8069d10 0x7f5df80b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5df77fe700 1 -- 172.21.15.131:0/553259379 >> v1:172.21.15.190:6789/0 conn(0x7f5df80c8820 legacy=0x7f5df80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5df77fe700 1 -- 172.21.15.131:0/553259379 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80caf30 con 0x7f5df80c8bf0 2023-12-26T23:09:42.082 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5df77fe700 1 --2- 172.21.15.131:0/553259379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 0x7f5df80b12e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0099b0 tx=0x7f5dec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33532316e0fda198 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5df6ffd700 1 -- 172.21.15.131:0/553259379 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00b860 con 0x7f5df80c8bf0 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.082+0000 7f5df6ffd700 1 -- 172.21.15.131:0/553259379 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec00b9c0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.083+0000 7f5df6ffd700 1 -- 172.21.15.131:0/553259379 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00bd60 con 0x7f5df80c8bf0 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.083+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/553259379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 msgr2=0x7f5df80b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.083+0000 7f5dfe5ff700 1 --2- 172.21.15.131:0/553259379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 0x7f5df80b12e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0099b0 tx=0x7f5dec009250 comp rx=0 tx=0).stop 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.083+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/553259379 shutdown_connections 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.083+0000 7f5dfe5ff700 1 --2- 172.21.15.131:0/553259379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df8069d10 0x7f5df80b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.083+0000 7f5dfe5ff700 1 --2- 172.21.15.131:0/553259379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 0x7f5df80b12e0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.083+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/553259379 >> 172.21.15.131:0/553259379 conn(0x7f5df8068040 msgr2=0x7f5df8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.083+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/553259379 shutdown_connections 2023-12-26T23:09:42.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.083+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/553259379 wait complete. 2023-12-26T23:09:42.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.085+0000 7f5dfe5ff700 1 Processor -- start 2023-12-26T23:09:42.084 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.085+0000 7f5dfe5ff700 1 -- start start 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfe5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df8069d10 0x7f5df80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfe5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df80c8820 0x7f5df81534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfe5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 0x7f5df816ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfe5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80b08b0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfe5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80b1140 con 0x7f5df8069d10 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfe5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8169d80 con 0x7f5df80c8820 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfcb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 0x7f5df816ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfcb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 0x7f5df816ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60568/0 (socket says 172.21.15.131:60568) 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfcb9c700 1 -- 172.21.15.131:0/1226457298 learned_addr learned my addr 172.21.15.131:0/1226457298 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5df77fe700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df80c8820 0x7f5df81534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5df7fff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df8069d10 0x7f5df80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfcb9c700 1 -- 172.21.15.131:0/1226457298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df80c8820 msgr2=0x7f5df81534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfcb9c700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df80c8820 0x7f5df81534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfcb9c700 1 -- 172.21.15.131:0/1226457298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df8069d10 msgr2=0x7f5df80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfcb9c700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df8069d10 0x7f5df80b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfcb9c700 1 -- 172.21.15.131:0/1226457298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec0094f0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5df7fff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df8069d10 0x7f5df80b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.086+0000 7f5dfcb9c700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 0x7f5df816ad90 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f5df0007f20 tx=0x7f5df0009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.087+0000 7f5ddffff700 1 -- 172.21.15.131:0/1226457298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df000eca0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.087+0000 7f5ddffff700 1 -- 172.21.15.131:0/1226457298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df000ee20 con 0x7f5df80c8bf0 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.087+0000 7f5ddffff700 1 -- 172.21.15.131:0/1226457298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0014670 con 0x7f5df80c8bf0 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.087+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df80af5d0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.087+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df80afb90 con 0x7f5df80c8bf0 2023-12-26T23:09:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.088+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df816a060 con 0x7f5df80c8bf0 2023-12-26T23:09:42.088 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.089+0000 7f5ddffff700 1 -- 172.21.15.131:0/1226457298 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5df00149e0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.090+0000 7f5ddffff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5de4066f30 0x7f5de40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.090+0000 7f5ddffff700 1 -- 172.21.15.131:0/1226457298 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5345+0+0 (secure 0 0 0) 0x7f5df0084ba0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.090+0000 7f5df7fff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5de4066f30 0x7f5de40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.089 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.091+0000 7f5df7fff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5de4066f30 0x7f5de40693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5df8155f90 tx=0x7f5de8009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:42.093 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.094+0000 7f5ddffff700 1 -- 172.21.15.131:0/1226457298 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5df0051ed0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.291+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5df80657e0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.292+0000 7f5ddffff700 1 -- 172.21.15.131:0/1226457298 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+12950 (secure 0 0 0) 0x7f5df00554f0 con 0x7f5df80c8bf0 2023-12-26T23:09:42.291 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:42.291 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":189,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:09:41.367746+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.294+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5de4066f30 msgr2=0x7f5de40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.294+0000 7f5dfe5ff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5de4066f30 0x7f5de40693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5df8155f90 tx=0x7f5de8009040 comp rx=0 tx=0).stop 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.294+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 msgr2=0x7f5df816ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.294+0000 7f5dfe5ff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 0x7f5df816ad90 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f5df0007f20 tx=0x7f5df0009860 comp rx=0 tx=0).stop 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.295+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 shutdown_connections 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.295+0000 7f5dfe5ff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5de4066f30 0x7f5de40693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.295+0000 7f5dfe5ff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5df80c8bf0 0x7f5df816ad90 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.295+0000 7f5dfe5ff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df8069d10 0x7f5df80b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.295+0000 7f5dfe5ff700 1 --2- 172.21.15.131:0/1226457298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df80c8820 0x7f5df81534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.295+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 >> 172.21.15.131:0/1226457298 conn(0x7f5df8068040 msgr2=0x7f5df8158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:42.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.295+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 shutdown_connections 2023-12-26T23:09:42.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.295+0000 7f5dfe5ff700 1 -- 172.21.15.131:0/1226457298 wait complete. 2023-12-26T23:09:42.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T23:09:42.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T23:09:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.505+0000 7fb55a5ff700 1 Processor -- start 2023-12-26T23:09:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.505+0000 7fb55a5ff700 1 -- start start 2023-12-26T23:09:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb55a5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb55413c510 0x7fb5540c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb55a5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb55406b8c0 0x7fb55406bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb55a5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 0x7fb5540cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb55a5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55406c280 con 0x7fb55413c510 2023-12-26T23:09:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb55a5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540686c0 con 0x7fb55406b8c0 2023-12-26T23:09:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb55a5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb554069520 con 0x7fb5540cc100 2023-12-26T23:09:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb553fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb55413c510 0x7fb5540c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb553fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb55413c510 0x7fb5540c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60578/0 (socket says 172.21.15.131:60578) 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb553fff700 1 -- 172.21.15.131:0/1138165692 learned_addr learned my addr 172.21.15.131:0/1138165692 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb5537fe700 1 --2- 172.21.15.131:0/1138165692 >> v2:172.21.15.190:3300/0 conn(0x7fb55406b8c0 0x7fb55406bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.506+0000 7fb558b9c700 1 --2- 172.21.15.131:0/1138165692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 0x7fb5540cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.507+0000 7fb553fff700 1 -- 172.21.15.131:0/1138165692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 msgr2=0x7fb5540cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.507+0000 7fb553fff700 1 --2- 172.21.15.131:0/1138165692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 0x7fb5540cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.507+0000 7fb553fff700 1 -- 172.21.15.131:0/1138165692 >> v2:172.21.15.190:3300/0 conn(0x7fb55406b8c0 msgr2=0x7fb55406bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.507+0000 7fb553fff700 1 --2- 172.21.15.131:0/1138165692 >> v2:172.21.15.190:3300/0 conn(0x7fb55406b8c0 0x7fb55406bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.507+0000 7fb553fff700 1 -- 172.21.15.131:0/1138165692 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554068910 con 0x7fb55413c510 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.507+0000 7fb553fff700 1 --2- 172.21.15.131:0/1138165692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 0x7fb5540c8a20 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fb5540b43e0 tx=0x7fb544009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23da14fbf5c7aceb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.507+0000 7fb552ffd700 1 -- 172.21.15.131:0/1138165692 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544005c10 con 0x7fb55413c510 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.507+0000 7fb552ffd700 1 -- 172.21.15.131:0/1138165692 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544005d70 con 0x7fb55413c510 2023-12-26T23:09:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.507+0000 7fb552ffd700 1 -- 172.21.15.131:0/1138165692 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544006260 con 0x7fb55413c510 2023-12-26T23:09:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.508+0000 7fb55a5ff700 1 -- 172.21.15.131:0/1138165692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 msgr2=0x7fb5540c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.508+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/1138165692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 0x7fb5540c8a20 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fb5540b43e0 tx=0x7fb544009220 comp rx=0 tx=0).stop 2023-12-26T23:09:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.508+0000 7fb55a5ff700 1 -- 172.21.15.131:0/1138165692 shutdown_connections 2023-12-26T23:09:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.508+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/1138165692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 0x7fb5540c8a20 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.508+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/1138165692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 0x7fb5540cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.508+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/1138165692 >> v2:172.21.15.190:3300/0 conn(0x7fb55406b8c0 0x7fb55406bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.508+0000 7fb55a5ff700 1 -- 172.21.15.131:0/1138165692 >> 172.21.15.131:0/1138165692 conn(0x7fb5540ce3e0 msgr2=0x7fb5540ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.508+0000 7fb55a5ff700 1 -- 172.21.15.131:0/1138165692 shutdown_connections 2023-12-26T23:09:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.508+0000 7fb55a5ff700 1 -- 172.21.15.131:0/1138165692 wait complete. 2023-12-26T23:09:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.509+0000 7fb55a5ff700 1 Processor -- start 2023-12-26T23:09:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.510+0000 7fb55a5ff700 1 -- start start 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.510+0000 7fb55a5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55406b8c0 0x7fb55413bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.510+0000 7fb55a5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 0x7fb554154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.510+0000 7fb55a5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 0x7fb5541548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb55a5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540b2090 con 0x7fb55413c510 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb55a5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55415ea30 con 0x7fb55406b8c0 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb55a5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55415ed10 con 0x7fb5540cc100 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb558b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 0x7fb5541548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb558b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 0x7fb5541548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60592/0 (socket says 172.21.15.131:60592) 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb558b9c700 1 -- 172.21.15.131:0/250133520 learned_addr learned my addr 172.21.15.131:0/250133520 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb553fff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55406b8c0 0x7fb55413bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb5537fe700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 0x7fb554154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb558b9c700 1 -- 172.21.15.131:0/250133520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 msgr2=0x7fb554154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb558b9c700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 0x7fb554154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb558b9c700 1 -- 172.21.15.131:0/250133520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55406b8c0 msgr2=0x7fb55413bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb558b9c700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55406b8c0 0x7fb55413bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb558b9c700 1 -- 172.21.15.131:0/250133520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544009480 con 0x7fb55413c510 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.511+0000 7fb558b9c700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 0x7fb5541548a0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009e90 tx=0x7fb54c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.512+0000 7fb53bfff700 1 -- 172.21.15.131:0/250133520 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00cb70 con 0x7fb55413c510 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.512+0000 7fb53bfff700 1 -- 172.21.15.131:0/250133520 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c00f040 con 0x7fb55413c510 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.512+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5540b2330 con 0x7fb55413c510 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.512+0000 7fb53bfff700 1 -- 172.21.15.131:0/250133520 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c014480 con 0x7fb55413c510 2023-12-26T23:09:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.512+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55413c290 con 0x7fb55413c510 2023-12-26T23:09:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.513+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5540657e0 con 0x7fb55413c510 2023-12-26T23:09:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.514+0000 7fb53bfff700 1 -- 172.21.15.131:0/250133520 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb54c014830 con 0x7fb55413c510 2023-12-26T23:09:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.515+0000 7fb53bfff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb540066f30 0x7fb5400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.515+0000 7fb53bfff700 1 -- 172.21.15.131:0/250133520 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5345+0+0 (secure 0 0 0) 0x7fb54c084d50 con 0x7fb55413c510 2023-12-26T23:09:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.515+0000 7fb553fff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb540066f30 0x7fb5400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.515+0000 7fb553fff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb540066f30 0x7fb5400693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb544009260 tx=0x7fb544016000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:42.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.519+0000 7fb53bfff700 1 -- 172.21.15.131:0/250133520 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb54c088030 con 0x7fb55413c510 2023-12-26T23:09:42.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.709+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5540b2850 con 0x7fb55413c510 2023-12-26T23:09:42.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.710+0000 7fb53bfff700 1 -- 172.21.15.131:0/250133520 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5540b2850 con 0x7fb55413c510 2023-12-26T23:09:42.709 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb540066f30 msgr2=0x7fb5400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb540066f30 0x7fb5400693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb544009260 tx=0x7fb544016000 comp rx=0 tx=0).stop 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 msgr2=0x7fb5541548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 0x7fb5541548a0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009e90 tx=0x7fb54c00b040 comp rx=0 tx=0).stop 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 shutdown_connections 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb540066f30 0x7fb5400693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb55413c510 0x7fb5541548a0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55406b8c0 0x7fb55413bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 --2- 172.21.15.131:0/250133520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5540cc100 0x7fb554154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.712+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 >> 172.21.15.131:0/250133520 conn(0x7fb5540ce3e0 msgr2=0x7fb5541524f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.713+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 shutdown_connections 2023-12-26T23:09:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.713+0000 7fb55a5ff700 1 -- 172.21.15.131:0/250133520 wait complete. 2023-12-26T23:09:42.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T23:09:42.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.920+0000 7f7547dff700 1 Processor -- start 2023-12-26T23:09:42.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.920+0000 7f7547dff700 1 -- start start 2023-12-26T23:09:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.921+0000 7f7547dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd100 0x7f75400cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.921+0000 7f7547dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75400cd4d0 0x7f75400cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.921+0000 7f7547dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754015db90 con 0x7f75401591f0 2023-12-26T23:09:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.921+0000 7f7547dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75400cbe50 con 0x7f75400cd4d0 2023-12-26T23:09:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.921+0000 7f7547dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754015d330 con 0x7f75400cd100 2023-12-26T23:09:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.921+0000 7f754539a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75400cd4d0 0x7f75400cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.921+0000 7f754539a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75400cd4d0 0x7f75400cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60604/0 (socket says 172.21.15.131:60604) 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f754539a700 1 -- 172.21.15.131:0/2242342968 learned_addr learned my addr 172.21.15.131:0/2242342968 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f7545b9b700 1 --2- 172.21.15.131:0/2242342968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd100 0x7f75400cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f754539a700 1 -- 172.21.15.131:0/2242342968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd100 msgr2=0x7f75400cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f754539a700 1 --2- 172.21.15.131:0/2242342968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd100 0x7f75400cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f754539a700 1 -- 172.21.15.131:0/2242342968 >> v1:172.21.15.190:6789/0 conn(0x7f75401591f0 legacy=0x7f7540159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f754539a700 1 -- 172.21.15.131:0/2242342968 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400cbb70 con 0x7f75400cd4d0 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f754539a700 1 --2- 172.21.15.131:0/2242342968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd4d0 0x7f75400cadc0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f75300099b0 tx=0x7f7530009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c94f213faee94c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f7544b99700 1 -- 172.21.15.131:0/2242342968 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530009ea0 con 0x7f75400cd4d0 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f7544b99700 1 -- 172.21.15.131:0/2242342968 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753000b860 con 0x7f75400cd4d0 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.922+0000 7f7544b99700 1 -- 172.21.15.131:0/2242342968 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753000bc00 con 0x7f75400cd4d0 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.923+0000 7f7547dff700 1 -- 172.21.15.131:0/2242342968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd4d0 msgr2=0x7f75400cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.923+0000 7f7547dff700 1 --2- 172.21.15.131:0/2242342968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd4d0 0x7f75400cadc0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f75300099b0 tx=0x7f7530009250 comp rx=0 tx=0).stop 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.923+0000 7f7547dff700 1 -- 172.21.15.131:0/2242342968 shutdown_connections 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.923+0000 7f7547dff700 1 --2- 172.21.15.131:0/2242342968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd100 0x7f75400cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.923+0000 7f7547dff700 1 --2- 172.21.15.131:0/2242342968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd4d0 0x7f75400cadc0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.923+0000 7f7547dff700 1 -- 172.21.15.131:0/2242342968 >> 172.21.15.131:0/2242342968 conn(0x7f754006ece0 msgr2=0x7f754006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:42.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.923+0000 7f7547dff700 1 -- 172.21.15.131:0/2242342968 shutdown_connections 2023-12-26T23:09:42.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.923+0000 7f7547dff700 1 -- 172.21.15.131:0/2242342968 wait complete. 2023-12-26T23:09:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.924+0000 7f7547dff700 1 Processor -- start 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.925+0000 7f7547dff700 1 -- start start 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.925+0000 7f7547dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd100 0x7f7540151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.925+0000 7f7547dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd4d0 0x7f75400c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.925+0000 7f7547dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75401591f0 0x7f75400c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.925+0000 7f7547dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754015b3c0 con 0x7f75400cd100 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.925+0000 7f7547dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754015b550 con 0x7f75401591f0 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.925+0000 7f7547dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75400b5830 con 0x7f75400cd4d0 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7545b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd100 0x7f7540151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7545b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd100 0x7f7540151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60606/0 (socket says 172.21.15.131:60606) 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7545b9b700 1 -- 172.21.15.131:0/491628958 learned_addr learned my addr 172.21.15.131:0/491628958 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f754539a700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd4d0 0x7f75400c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f754639c700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75401591f0 0x7f75400c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7545b9b700 1 -- 172.21.15.131:0/491628958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd4d0 msgr2=0x7f75400c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7545b9b700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd4d0 0x7f75400c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7545b9b700 1 -- 172.21.15.131:0/491628958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75401591f0 msgr2=0x7f75400c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7545b9b700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75401591f0 0x7f75400c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7545b9b700 1 -- 172.21.15.131:0/491628958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75300094f0 con 0x7f75400cd100 2023-12-26T23:09:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7545b9b700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd100 0x7f7540151550 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f753c00ec80 tx=0x7f753c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.926+0000 7f7535ffb700 1 -- 172.21.15.131:0/491628958 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c00c9d0 con 0x7f75400cd100 2023-12-26T23:09:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.927+0000 7f7535ffb700 1 -- 172.21.15.131:0/491628958 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c00cb30 con 0x7f75400cd100 2023-12-26T23:09:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.927+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75400b5b10 con 0x7f75400cd100 2023-12-26T23:09:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.927+0000 7f7535ffb700 1 -- 172.21.15.131:0/491628958 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c013470 con 0x7f75400cd100 2023-12-26T23:09:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.927+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75400b60d0 con 0x7f75400cd100 2023-12-26T23:09:42.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.928+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75400657e0 con 0x7f75400cd100 2023-12-26T23:09:42.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.929+0000 7f7535ffb700 1 -- 172.21.15.131:0/491628958 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f753c013820 con 0x7f75400cd100 2023-12-26T23:09:42.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.929+0000 7f7535ffb700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7524066f30 0x7f75240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:42.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.930+0000 7f7535ffb700 1 -- 172.21.15.131:0/491628958 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5345+0+0 (secure 0 0 0) 0x7f753c084860 con 0x7f75400cd100 2023-12-26T23:09:42.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.930+0000 7f754539a700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7524066f30 0x7f75240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:42.929 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.930+0000 7f754539a700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7524066f30 0x7f75240693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f75300099b0 tx=0x7f7530017000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:42.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:42.933+0000 7f7535ffb700 1 -- 172.21.15.131:0/491628958 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f753c051b90 con 0x7f75400cd100 2023-12-26T23:09:43.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.131+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f75400b1300 con 0x7f75400cd100 2023-12-26T23:09:43.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.132+0000 7f7535ffb700 1 -- 172.21.15.131:0/491628958 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v189) v1 ==== 84+0+65 (secure 0 0 0) 0x7f753c0551b0 con 0x7f75400cd100 2023-12-26T23:09:43.131 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:43.131 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T23:09:43.131 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:43.131 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T23:09:43.131 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T23:09:43.131 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:43.132 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T23:09:43.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7524066f30 msgr2=0x7f75240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:43.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7524066f30 0x7f75240693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f75300099b0 tx=0x7f7530017000 comp rx=0 tx=0).stop 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd100 msgr2=0x7f7540151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd100 0x7f7540151550 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f753c00ec80 tx=0x7f753c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 shutdown_connections 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7524066f30 0x7f75240693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75401591f0 0x7f75400c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75400cd100 0x7f7540151550 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 --2- 172.21.15.131:0/491628958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75400cd4d0 0x7f75400c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 >> 172.21.15.131:0/491628958 conn(0x7f754006ece0 msgr2=0x7f754006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 shutdown_connections 2023-12-26T23:09:43.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.134+0000 7f7547dff700 1 -- 172.21.15.131:0/491628958 wait complete. 2023-12-26T23:09:43.151 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T23:09:43.151 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T23:09:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.347+0000 7f58c1fff700 1 Processor -- start 2023-12-26T23:09:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.347+0000 7f58c1fff700 1 -- start start 2023-12-26T23:09:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.348+0000 7f58c1fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f58bc0c8d40 0x7f58bc15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.348+0000 7f58c1fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58bc156e50 0x7f58bc1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.348+0000 7f58c1fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc0af040 con 0x7f58bc0c9110 2023-12-26T23:09:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.348+0000 7f58c1fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc0ae0e0 con 0x7f58bc0c8d40 2023-12-26T23:09:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.348+0000 7f58c1fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc0ae3c0 con 0x7f58bc156e50 2023-12-26T23:09:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.348+0000 7f58bb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f58bc0c8d40 0x7f58bc15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.348+0000 7f58bb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f58bc0c8d40 0x7f58bc15bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60622/0 (socket says 172.21.15.131:60622) 2023-12-26T23:09:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.348+0000 7f58bb7fe700 1 -- 172.21.15.131:0/61819514 learned_addr learned my addr 172.21.15.131:0/61819514 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58bbfff700 1 --2- 172.21.15.131:0/61819514 >> v2:172.21.15.190:3300/0 conn(0x7f58bc156e50 0x7f58bc1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58bb7fe700 1 -- 172.21.15.131:0/61819514 >> v2:172.21.15.190:3300/0 conn(0x7f58bc156e50 msgr2=0x7f58bc1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58bb7fe700 1 --2- 172.21.15.131:0/61819514 >> v2:172.21.15.190:3300/0 conn(0x7f58bc156e50 0x7f58bc1572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58bb7fe700 1 -- 172.21.15.131:0/61819514 >> v1:172.21.15.190:6789/0 conn(0x7f58bc0c9110 legacy=0x7f58bc0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58bb7fe700 1 -- 172.21.15.131:0/61819514 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc0ae6a0 con 0x7f58bc0c8d40 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58bbfff700 1 --2- 172.21.15.131:0/61819514 >> v2:172.21.15.190:3300/0 conn(0x7f58bc156e50 0x7f58bc1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58bb7fe700 1 --2- 172.21.15.131:0/61819514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc0c8d40 0x7f58bc15bc30 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0ac5c0 tx=0x7f58ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb4b8021c67ed4d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58baffd700 1 -- 172.21.15.131:0/61819514 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00b820 con 0x7f58bc0c8d40 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58baffd700 1 -- 172.21.15.131:0/61819514 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00b980 con 0x7f58bc0c8d40 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.349+0000 7f58baffd700 1 -- 172.21.15.131:0/61819514 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00bd20 con 0x7f58bc0c8d40 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.350+0000 7f58c1fff700 1 -- 172.21.15.131:0/61819514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc0c8d40 msgr2=0x7f58bc15bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.350+0000 7f58c1fff700 1 --2- 172.21.15.131:0/61819514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc0c8d40 0x7f58bc15bc30 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0ac5c0 tx=0x7f58ac009250 comp rx=0 tx=0).stop 2023-12-26T23:09:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.350+0000 7f58c1fff700 1 -- 172.21.15.131:0/61819514 shutdown_connections 2023-12-26T23:09:43.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.350+0000 7f58c1fff700 1 --2- 172.21.15.131:0/61819514 >> v2:172.21.15.190:3300/0 conn(0x7f58bc156e50 0x7f58bc1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.350+0000 7f58c1fff700 1 --2- 172.21.15.131:0/61819514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc0c8d40 0x7f58bc15bc30 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.350+0000 7f58c1fff700 1 -- 172.21.15.131:0/61819514 >> 172.21.15.131:0/61819514 conn(0x7f58bc0675b0 msgr2=0x7f58bc0679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:43.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.350+0000 7f58c1fff700 1 -- 172.21.15.131:0/61819514 shutdown_connections 2023-12-26T23:09:43.351 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.350+0000 7f58c1fff700 1 -- 172.21.15.131:0/61819514 wait complete. 2023-12-26T23:09:43.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.351+0000 7f58c1fff700 1 Processor -- start 2023-12-26T23:09:43.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.351+0000 7f58c1fff700 1 -- start start 2023-12-26T23:09:43.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.352+0000 7f58c1fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58bc0c8d40 0x7f58bc0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:43.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.352+0000 7f58c1fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58bc0c9110 0x7f58bc0c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.352+0000 7f58c1fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc156e50 0x7f58bc16a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.352+0000 7f58c1fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0abca0 con 0x7f58bc156e50 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.352+0000 7f58c1fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0abf80 con 0x7f58bc0c9110 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.352+0000 7f58c1fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0ac260 con 0x7f58bc0c8d40 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc156e50 0x7f58bc16a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc156e50 0x7f58bc16a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60636/0 (socket says 172.21.15.131:60636) 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bb7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58bc0c8d40 0x7f58bc0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bbfff700 1 -- 172.21.15.131:0/82259472 learned_addr learned my addr 172.21.15.131:0/82259472 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58b3fff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58bc0c9110 0x7f58bc0c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bbfff700 1 -- 172.21.15.131:0/82259472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58bc0c8d40 msgr2=0x7f58bc0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bbfff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58bc0c8d40 0x7f58bc0c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bbfff700 1 -- 172.21.15.131:0/82259472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58bc0c9110 msgr2=0x7f58bc0c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:43.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bbfff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58bc0c9110 0x7f58bc0c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:43.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bbfff700 1 -- 172.21.15.131:0/82259472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0094f0 con 0x7f58bc156e50 2023-12-26T23:09:43.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.353+0000 7f58bbfff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc156e50 0x7f58bc16a5d0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f58b400cef0 tx=0x7f58b4007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:43.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.354+0000 7f58b37fe700 1 -- 172.21.15.131:0/82259472 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b400ad20 con 0x7f58bc156e50 2023-12-26T23:09:43.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.354+0000 7f58b37fe700 1 -- 172.21.15.131:0/82259472 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4016420 con 0x7f58bc156e50 2023-12-26T23:09:43.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.354+0000 7f58b37fe700 1 -- 172.21.15.131:0/82259472 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b400aea0 con 0x7f58bc156e50 2023-12-26T23:09:43.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.354+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc16a0d0 con 0x7f58bc156e50 2023-12-26T23:09:43.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.354+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc0c5bd0 con 0x7f58bc156e50 2023-12-26T23:09:43.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.355+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc0717f0 con 0x7f58bc156e50 2023-12-26T23:09:43.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.356+0000 7f58b37fe700 1 -- 172.21.15.131:0/82259472 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f58b4016a70 con 0x7f58bc156e50 2023-12-26T23:09:43.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.356+0000 7f58b37fe700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f589c066f30 0x7f589c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:43.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.357+0000 7f58b37fe700 1 -- 172.21.15.131:0/82259472 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5345+0+0 (secure 0 0 0) 0x7f58b4012070 con 0x7f58bc156e50 2023-12-26T23:09:43.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.357+0000 7f58bb7fe700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f589c066f30 0x7f589c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:43.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.357+0000 7f58bb7fe700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f589c066f30 0x7f589c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0ac5c0 tx=0x7f58ac016000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:43.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.360+0000 7f58b37fe700 1 -- 172.21.15.131:0/82259472 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58b404e5a0 con 0x7f58bc156e50 2023-12-26T23:09:43.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:43.557+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f58bc0ae980 con 0x7f58bc156e50 2023-12-26T23:09:44.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.562+0000 7f58b37fe700 1 -- 172.21.15.131:0/82259472 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v190) v1 ==== 138+0+0 (secure 0 0 0) 0x7f58b4051fc0 con 0x7f58bc156e50 2023-12-26T23:09:44.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f589c066f30 msgr2=0x7f589c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:44.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f589c066f30 0x7f589c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0ac5c0 tx=0x7f58ac016000 comp rx=0 tx=0).stop 2023-12-26T23:09:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc156e50 msgr2=0x7f58bc16a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc156e50 0x7f58bc16a5d0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f58b400cef0 tx=0x7f58b4007860 comp rx=0 tx=0).stop 2023-12-26T23:09:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 shutdown_connections 2023-12-26T23:09:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f589c066f30 0x7f589c0693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58bc156e50 0x7f58bc16a5d0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58bc0c8d40 0x7f58bc0c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 --2- 172.21.15.131:0/82259472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58bc0c9110 0x7f58bc0c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 >> 172.21.15.131:0/82259472 conn(0x7f58bc0675b0 msgr2=0x7f58bc154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:44.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 shutdown_connections 2023-12-26T23:09:44.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.568+0000 7f58c1fff700 1 -- 172.21.15.131:0/82259472 wait complete. 2023-12-26T23:09:44.570 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T23:09:44.589 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T23:09:44.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.790+0000 7fc8b93ff700 1 Processor -- start 2023-12-26T23:09:44.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.791+0000 7fc8b93ff700 1 -- start start 2023-12-26T23:09:44.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b93ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8b4157050 0x7fc8b40686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:44.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b93ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 0x7fc8b4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:44.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b93ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc8b4068c00 0x7fc8b40cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:44.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b93ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40c9520 con 0x7fc8b4068c00 2023-12-26T23:09:44.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b93ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40caed0 con 0x7fc8b4157050 2023-12-26T23:09:44.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b93ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40ca8e0 con 0x7fc8b4157420 2023-12-26T23:09:44.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b37fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc8b4068c00 0x7fc8b40cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:44.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b37fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc8b4068c00 0x7fc8b40cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60650/0 (socket says 172.21.15.131:60650) 2023-12-26T23:09:44.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b37fe700 1 -- 172.21.15.131:0/3467123227 learned_addr learned my addr 172.21.15.131:0/3467123227 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b2ffd700 1 --2- 172.21.15.131:0/3467123227 >> v2:172.21.15.190:3300/0 conn(0x7fc8b4157050 0x7fc8b40686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.792+0000 7fc8b27fc700 1 --2- 172.21.15.131:0/3467123227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 0x7fc8b4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.793+0000 7fc8b37fe700 1 -- 172.21.15.131:0/3467123227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 msgr2=0x7fc8b4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.793+0000 7fc8b37fe700 1 --2- 172.21.15.131:0/3467123227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 0x7fc8b4159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.793+0000 7fc8b37fe700 1 -- 172.21.15.131:0/3467123227 >> v2:172.21.15.190:3300/0 conn(0x7fc8b4157050 msgr2=0x7fc8b40686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.793+0000 7fc8b37fe700 1 --2- 172.21.15.131:0/3467123227 >> v2:172.21.15.190:3300/0 conn(0x7fc8b4157050 0x7fc8b40686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.793+0000 7fc8b37fe700 1 -- 172.21.15.131:0/3467123227 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b415a6c0 con 0x7fc8b4068c00 2023-12-26T23:09:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.793+0000 7fc8b37fe700 1 --2- 172.21.15.131:0/3467123227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 0x7fc8b40cb360 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fc8b406f3a0 tx=0x7fc8ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78d22f8e7b4e428 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.793+0000 7fc8b1ffb700 1 -- 172.21.15.131:0/3467123227 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac00e070 con 0x7fc8b4068c00 2023-12-26T23:09:44.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.793+0000 7fc8b1ffb700 1 -- 172.21.15.131:0/3467123227 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac00b970 con 0x7fc8b4068c00 2023-12-26T23:09:44.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.793+0000 7fc8b1ffb700 1 -- 172.21.15.131:0/3467123227 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0092e0 con 0x7fc8b4068c00 2023-12-26T23:09:44.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.794+0000 7fc8b93ff700 1 -- 172.21.15.131:0/3467123227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 msgr2=0x7fc8b40cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:44.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.794+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/3467123227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 0x7fc8b40cb360 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fc8b406f3a0 tx=0x7fc8ac009840 comp rx=0 tx=0).stop 2023-12-26T23:09:44.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.794+0000 7fc8b93ff700 1 -- 172.21.15.131:0/3467123227 shutdown_connections 2023-12-26T23:09:44.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.794+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/3467123227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 0x7fc8b40cb360 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.794+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/3467123227 >> v2:172.21.15.190:3300/0 conn(0x7fc8b4157050 0x7fc8b40686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.794+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/3467123227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 0x7fc8b4159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.794+0000 7fc8b93ff700 1 -- 172.21.15.131:0/3467123227 >> 172.21.15.131:0/3467123227 conn(0x7fc8b406dc80 msgr2=0x7fc8b40675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:44.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.794+0000 7fc8b93ff700 1 -- 172.21.15.131:0/3467123227 shutdown_connections 2023-12-26T23:09:44.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.794+0000 7fc8b93ff700 1 -- 172.21.15.131:0/3467123227 wait complete. 2023-12-26T23:09:44.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.795+0000 7fc8b93ff700 1 Processor -- start 2023-12-26T23:09:44.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.796+0000 7fc8b93ff700 1 -- start start 2023-12-26T23:09:44.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.796+0000 7fc8b93ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 0x7fc8b40b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.796+0000 7fc8b93ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8b4157050 0x7fc8b4159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.796+0000 7fc8b93ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 0x7fc8b4154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.796+0000 7fc8b93ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40c64b0 con 0x7fc8b4068c00 2023-12-26T23:09:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.796+0000 7fc8b93ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40c58a0 con 0x7fc8b4157050 2023-12-26T23:09:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.796+0000 7fc8b93ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40c5b80 con 0x7fc8b4157420 2023-12-26T23:09:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 0x7fc8b40b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 0x7fc8b40b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60666/0 (socket says 172.21.15.131:60666) 2023-12-26T23:09:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b2ffd700 1 -- 172.21.15.131:0/1536652944 learned_addr learned my addr 172.21.15.131:0/1536652944 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b37fe700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 0x7fc8b4154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b27fc700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8b4157050 0x7fc8b4159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b2ffd700 1 -- 172.21.15.131:0/1536652944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 msgr2=0x7fc8b4154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b2ffd700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 0x7fc8b4154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b2ffd700 1 -- 172.21.15.131:0/1536652944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8b4157050 msgr2=0x7fc8b4159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b2ffd700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8b4157050 0x7fc8b4159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b2ffd700 1 -- 172.21.15.131:0/1536652944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac009ae0 con 0x7fc8b4068c00 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b27fc700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8b4157050 0x7fc8b4159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc8b2ffd700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 0x7fc8b40b3a10 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400ec80 tx=0x7fc8a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.797+0000 7fc89affd700 1 -- 172.21.15.131:0/1536652944 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400c850 con 0x7fc8b4068c00 2023-12-26T23:09:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.798+0000 7fc89affd700 1 -- 172.21.15.131:0/1536652944 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a400c9b0 con 0x7fc8b4068c00 2023-12-26T23:09:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.798+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b40c5e60 con 0x7fc8b4068c00 2023-12-26T23:09:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.798+0000 7fc89affd700 1 -- 172.21.15.131:0/1536652944 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400cd50 con 0x7fc8b4068c00 2023-12-26T23:09:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.798+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b40b3f50 con 0x7fc8b4068c00 2023-12-26T23:09:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.799+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b40c26a0 con 0x7fc8b4068c00 2023-12-26T23:09:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.800+0000 7fc89affd700 1 -- 172.21.15.131:0/1536652944 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc8a4013680 con 0x7fc8b4068c00 2023-12-26T23:09:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.801+0000 7fc89affd700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc8a0066f30 0x7fc8a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.801+0000 7fc89affd700 1 -- 172.21.15.131:0/1536652944 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5715+0+0 (secure 0 0 0) 0x7fc8a4084ff0 con 0x7fc8b4068c00 2023-12-26T23:09:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.801+0000 7fc8b27fc700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc8a0066f30 0x7fc8a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.802+0000 7fc8b27fc700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc8a0066f30 0x7fc8a00693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4155660 tx=0x7fc8a8008040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:44.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.805+0000 7fc89affd700 1 -- 172.21.15.131:0/1536652944 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8a4052270 con 0x7fc8b4068c00 2023-12-26T23:09:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:44.997+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc8b40657e0 con 0x7fc8b4068c00 2023-12-26T23:09:45.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:09:45.036 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:09:45.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.568+0000 7fc89affd700 1 -- 172.21.15.131:0/1536652944 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v191) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc8a4055890 con 0x7fc8b4068c00 2023-12-26T23:09:45.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.572+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc8a0066f30 msgr2=0x7fc8a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:45.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.572+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc8a0066f30 0x7fc8a00693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4155660 tx=0x7fc8a8008040 comp rx=0 tx=0).stop 2023-12-26T23:09:45.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.572+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 msgr2=0x7fc8b40b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:45.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.572+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 0x7fc8b40b3a10 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400ec80 tx=0x7fc8a40091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.573+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 shutdown_connections 2023-12-26T23:09:45.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.573+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc8a0066f30 0x7fc8a00693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.573+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8b4157420 0x7fc8b4154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.573+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8b4068c00 0x7fc8b40b3a10 secure :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400ec80 tx=0x7fc8a40091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.573+0000 7fc8b93ff700 1 --2- 172.21.15.131:0/1536652944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8b4157050 0x7fc8b4159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.573+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 >> 172.21.15.131:0/1536652944 conn(0x7fc8b406dc80 msgr2=0x7fc8b4069030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:45.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.575+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 shutdown_connections 2023-12-26T23:09:45.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.576+0000 7fc8b93ff700 1 -- 172.21.15.131:0/1536652944 wait complete. 2023-12-26T23:09:45.582 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T23:09:45.603 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T23:09:45.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.797+0000 7f0c9b9ff700 1 Processor -- start 2023-12-26T23:09:45.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.797+0000 7f0c9b9ff700 1 -- start start 2023-12-26T23:09:45.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.798+0000 7f0c9b9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0c940cb320 0x7f0c94069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:45.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.798+0000 7f0c9b9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0c940b0e60 0x7f0c940c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:45.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.798+0000 7f0c9b9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c94156940 con 0x7f0c9406a1e0 2023-12-26T23:09:45.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.798+0000 7f0c9b9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c940c8cc0 con 0x7f0c940cb320 2023-12-26T23:09:45.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.798+0000 7f0c9b9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c940ad700 con 0x7f0c940b0e60 2023-12-26T23:09:45.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.798+0000 7f0c9979b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0c940cb320 0x7f0c94069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.798+0000 7f0c9979b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0c940cb320 0x7f0c94069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52256/0 (socket says 172.21.15.131:52256) 2023-12-26T23:09:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.798+0000 7f0c9979b700 1 -- 172.21.15.131:0/3338851883 learned_addr learned my addr 172.21.15.131:0/3338851883 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.799+0000 7f0c9979b700 1 -- 172.21.15.131:0/3338851883 >> v2:172.21.15.190:3300/0 conn(0x7f0c940b0e60 msgr2=0x7f0c940c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.799+0000 7f0c99f9c700 1 --2- 172.21.15.131:0/3338851883 >> v2:172.21.15.190:3300/0 conn(0x7f0c940b0e60 0x7f0c940c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.799+0000 7f0c9979b700 1 --2- 172.21.15.131:0/3338851883 >> v2:172.21.15.190:3300/0 conn(0x7f0c940b0e60 0x7f0c940c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.799+0000 7f0c9979b700 1 -- 172.21.15.131:0/3338851883 >> v1:172.21.15.190:6789/0 conn(0x7f0c9406a1e0 legacy=0x7f0c940b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.799+0000 7f0c9979b700 1 -- 172.21.15.131:0/3338851883 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c940ae500 con 0x7f0c940cb320 2023-12-26T23:09:45.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.799+0000 7f0c9979b700 1 --2- 172.21.15.131:0/3338851883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940cb320 0x7f0c94069ca0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f0c9406fb50 tx=0x7f0c90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98accafd0dd48c48 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.799+0000 7f0c8bfff700 1 -- 172.21.15.131:0/3338851883 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000b820 con 0x7f0c940cb320 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.799+0000 7f0c8bfff700 1 -- 172.21.15.131:0/3338851883 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9000b980 con 0x7f0c940cb320 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.799+0000 7f0c8bfff700 1 -- 172.21.15.131:0/3338851883 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000bd20 con 0x7f0c940cb320 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.800+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3338851883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940cb320 msgr2=0x7f0c94069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.800+0000 7f0c9b9ff700 1 --2- 172.21.15.131:0/3338851883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940cb320 0x7f0c94069ca0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f0c9406fb50 tx=0x7f0c90009250 comp rx=0 tx=0).stop 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.800+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3338851883 shutdown_connections 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.800+0000 7f0c9b9ff700 1 --2- 172.21.15.131:0/3338851883 >> v2:172.21.15.190:3300/0 conn(0x7f0c940b0e60 0x7f0c940c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.800+0000 7f0c9b9ff700 1 --2- 172.21.15.131:0/3338851883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940cb320 0x7f0c94069ca0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.800+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3338851883 >> 172.21.15.131:0/3338851883 conn(0x7f0c94068040 msgr2=0x7f0c94069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:45.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.800+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3338851883 shutdown_connections 2023-12-26T23:09:45.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.800+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3338851883 wait complete. 2023-12-26T23:09:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.801+0000 7f0c9b9ff700 1 Processor -- start 2023-12-26T23:09:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.802+0000 7f0c9b9ff700 1 -- start start 2023-12-26T23:09:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.802+0000 7f0c9b9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c9406a1e0 0x7f0c941426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.802+0000 7f0c9b9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940b0e60 0x7f0c94158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.802+0000 7f0c9b9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c940cb320 0x7f0c940c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.802+0000 7f0c9b9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c94159a70 con 0x7f0c940b0e60 2023-12-26T23:09:45.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.802+0000 7f0c9b9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c94142c10 con 0x7f0c9406a1e0 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.802+0000 7f0c9b9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c941594b0 con 0x7f0c940cb320 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c98f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940b0e60 0x7f0c94158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c98f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940b0e60 0x7f0c94158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52272/0 (socket says 172.21.15.131:52272) 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c98f9a700 1 -- 172.21.15.131:0/3229922447 learned_addr learned my addr 172.21.15.131:0/3229922447 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c99f9c700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c940cb320 0x7f0c940c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c9979b700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c9406a1e0 0x7f0c941426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c98f9a700 1 -- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c940cb320 msgr2=0x7f0c940c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c98f9a700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c940cb320 0x7f0c940c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c98f9a700 1 -- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c9406a1e0 msgr2=0x7f0c941426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c98f9a700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c9406a1e0 0x7f0c941426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:45.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c98f9a700 1 -- 172.21.15.131:0/3229922447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c900094f0 con 0x7f0c940b0e60 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c99f9c700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c940cb320 0x7f0c940c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c9979b700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c9406a1e0 0x7f0c941426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.803+0000 7f0c98f9a700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940b0e60 0x7f0c94158c80 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400c590 tx=0x7f0c84007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.804+0000 7f0c897fa700 1 -- 172.21.15.131:0/3229922447 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84011590 con 0x7f0c940b0e60 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.804+0000 7f0c897fa700 1 -- 172.21.15.131:0/3229922447 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c84011730 con 0x7f0c940b0e60 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.804+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c94159790 con 0x7f0c940b0e60 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.804+0000 7f0c897fa700 1 -- 172.21.15.131:0/3229922447 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84011af0 con 0x7f0c940b0e60 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.804+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c940c13a0 con 0x7f0c940b0e60 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.805+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c940657e0 con 0x7f0c940b0e60 2023-12-26T23:09:45.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.806+0000 7f0c897fa700 1 -- 172.21.15.131:0/3229922447 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0c84006820 con 0x7f0c940b0e60 2023-12-26T23:09:45.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.807+0000 7f0c897fa700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c78066f30 0x7f0c780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:45.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.807+0000 7f0c897fa700 1 -- 172.21.15.131:0/3229922447 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f0c84084ff0 con 0x7f0c940b0e60 2023-12-26T23:09:45.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.808+0000 7f0c9979b700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c78066f30 0x7f0c780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:45.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.808+0000 7f0c9979b700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c78066f30 0x7f0c780693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c9406fb50 tx=0x7f0c90016000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:45.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:45.811+0000 7f0c897fa700 1 -- 172.21.15.131:0/3229922447 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c840520d0 con 0x7f0c940b0e60 2023-12-26T23:09:46.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.043+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0c940c8e00 con 0x7f0c940b0e60 2023-12-26T23:09:46.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.567+0000 7f0c897fa700 1 -- 172.21.15.131:0/3229922447 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v86) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0c840556f0 con 0x7f0c940b0e60 2023-12-26T23:09:46.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.572+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c78066f30 msgr2=0x7f0c780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:46.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.572+0000 7f0c9b9ff700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c78066f30 0x7f0c780693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c9406fb50 tx=0x7f0c90016000 comp rx=0 tx=0).stop 2023-12-26T23:09:46.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.572+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940b0e60 msgr2=0x7f0c94158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:46.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.572+0000 7f0c9b9ff700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940b0e60 0x7f0c94158c80 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400c590 tx=0x7f0c84007f30 comp rx=0 tx=0).stop 2023-12-26T23:09:46.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.573+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 shutdown_connections 2023-12-26T23:09:46.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.573+0000 7f0c9b9ff700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c78066f30 0x7f0c780693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:46.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.573+0000 7f0c9b9ff700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c940cb320 0x7f0c940c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:46.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.573+0000 7f0c9b9ff700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c9406a1e0 0x7f0c941426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:46.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.573+0000 7f0c9b9ff700 1 --2- 172.21.15.131:0/3229922447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c940b0e60 0x7f0c94158c80 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:46.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.573+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 >> 172.21.15.131:0/3229922447 conn(0x7f0c94068040 msgr2=0x7f0c9406e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:46.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.574+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 shutdown_connections 2023-12-26T23:09:46.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.575+0000 7f0c9b9ff700 1 -- 172.21.15.131:0/3229922447 wait complete. 2023-12-26T23:09:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 22 and data pool 23 2023-12-26T23:09:46.611 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:09:46.642 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-12-26T23:09:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.802+0000 7f61d73ff700 1 Processor -- start 2023-12-26T23:09:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.802+0000 7f61d73ff700 1 -- start start 2023-12-26T23:09:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.803+0000 7f61d73ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f61d00cc580 0x7f61d00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.803+0000 7f61d73ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00b0560 0x7f61d015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.803+0000 7f61d73ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00b0d70 con 0x7f61d00b00e0 2023-12-26T23:09:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.803+0000 7f61d73ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d00680f0 con 0x7f61d00cc580 2023-12-26T23:09:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.803+0000 7f61d73ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00675b0 con 0x7f61d00b0560 2023-12-26T23:09:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61d599c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00b0560 0x7f61d015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61d499a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f61d00b00e0 0x7f61d006e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:35780/0 (socket says 172.21.15.131:35780) 2023-12-26T23:09:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61d499a700 1 -- 172.21.15.131:0/2739325088 learned_addr learned my addr 172.21.15.131:0/2739325088 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61d519b700 1 --2- 172.21.15.131:0/2739325088 >> v2:172.21.15.190:3300/0 conn(0x7f61d00cc580 0x7f61d00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61c7fff700 1 -- 172.21.15.131:0/2739325088 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416882755 0 0) 0x7f61d00b0d70 con 0x7f61d00b00e0 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61c7fff700 1 -- 172.21.15.131:0/2739325088 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61bc003570 con 0x7f61d00b00e0 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61d599c700 1 -- 172.21.15.131:0/2739325088 >> v2:172.21.15.190:3300/0 conn(0x7f61d00cc580 msgr2=0x7f61d00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61d599c700 1 --2- 172.21.15.131:0/2739325088 >> v2:172.21.15.190:3300/0 conn(0x7f61d00cc580 0x7f61d00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61d599c700 1 -- 172.21.15.131:0/2739325088 >> v1:172.21.15.190:6789/0 conn(0x7f61d00b00e0 legacy=0x7f61d006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61d599c700 1 -- 172.21.15.131:0/2739325088 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d0159c30 con 0x7f61d00b0560 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.804+0000 7f61d519b700 1 --2- 172.21.15.131:0/2739325088 >> v2:172.21.15.190:3300/0 conn(0x7f61d00cc580 0x7f61d00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.805+0000 7f61d599c700 1 --2- 172.21.15.131:0/2739325088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00b0560 0x7f61d015d9b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009e60 tx=0x7f61c8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63fac6ea3a821c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.805+0000 7f61c7fff700 1 -- 172.21.15.131:0/2739325088 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800e070 con 0x7f61d00b0560 2023-12-26T23:09:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.805+0000 7f61c7fff700 1 -- 172.21.15.131:0/2739325088 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c800bab0 con 0x7f61d00b0560 2023-12-26T23:09:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.805+0000 7f61c7fff700 1 -- 172.21.15.131:0/2739325088 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800be30 con 0x7f61d00b0560 2023-12-26T23:09:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.805+0000 7f61d73ff700 1 -- 172.21.15.131:0/2739325088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00b0560 msgr2=0x7f61d015d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.805+0000 7f61d73ff700 1 --2- 172.21.15.131:0/2739325088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00b0560 0x7f61d015d9b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009e60 tx=0x7f61c8009780 comp rx=0 tx=0).stop 2023-12-26T23:09:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.806+0000 7f61d73ff700 1 -- 172.21.15.131:0/2739325088 shutdown_connections 2023-12-26T23:09:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.806+0000 7f61d73ff700 1 --2- 172.21.15.131:0/2739325088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00b0560 0x7f61d015d9b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.806+0000 7f61d73ff700 1 --2- 172.21.15.131:0/2739325088 >> v2:172.21.15.190:3300/0 conn(0x7f61d00cc580 0x7f61d00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.806+0000 7f61d73ff700 1 -- 172.21.15.131:0/2739325088 >> 172.21.15.131:0/2739325088 conn(0x7f61d00686c0 msgr2=0x7f61d0068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.806+0000 7f61d73ff700 1 -- 172.21.15.131:0/2739325088 shutdown_connections 2023-12-26T23:09:46.808 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.806+0000 7f61d73ff700 1 -- 172.21.15.131:0/2739325088 wait complete. 2023-12-26T23:09:46.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.807+0000 7f61d73ff700 1 Processor -- start 2023-12-26T23:09:46.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.807+0000 7f61d73ff700 1 -- start start 2023-12-26T23:09:46.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.808+0000 7f61d73ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61d00b00e0 0x7f61d015a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.808+0000 7f61d73ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d00b0560 0x7f61d014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.808+0000 7f61d73ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00cc580 0x7f61d014d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.808+0000 7f61d73ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00ae1f0 con 0x7f61d00b00e0 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.808+0000 7f61d73ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00ad570 con 0x7f61d00b0560 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.808+0000 7f61d73ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00ad850 con 0x7f61d00cc580 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.808+0000 7f61d519b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61d00b00e0 0x7f61d015a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.808+0000 7f61d519b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61d00b00e0 0x7f61d015a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52284/0 (socket says 172.21.15.131:52284) 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.808+0000 7f61d519b700 1 -- 172.21.15.131:0/962849604 learned_addr learned my addr 172.21.15.131:0/962849604 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d599c700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00cc580 0x7f61d014d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d499a700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d00b0560 0x7f61d014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:46.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d519b700 1 -- 172.21.15.131:0/962849604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00cc580 msgr2=0x7f61d014d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d519b700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00cc580 0x7f61d014d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d519b700 1 -- 172.21.15.131:0/962849604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d00b0560 msgr2=0x7f61d014cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d519b700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d00b0560 0x7f61d014cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d519b700 1 -- 172.21.15.131:0/962849604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc009420 con 0x7f61d00b00e0 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d519b700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61d00b00e0 0x7f61d015a9e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00ec80 tx=0x7f61cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61c57fa700 1 -- 172.21.15.131:0/962849604 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00c9d0 con 0x7f61d00b00e0 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61c57fa700 1 -- 172.21.15.131:0/962849604 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc00cb30 con 0x7f61d00b00e0 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c8009a20 con 0x7f61d00b00e0 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61c57fa700 1 -- 172.21.15.131:0/962849604 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00ceb0 con 0x7f61d00b00e0 2023-12-26T23:09:46.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.809+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d00ade10 con 0x7f61d00b00e0 2023-12-26T23:09:46.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.810+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d00657e0 con 0x7f61d00b00e0 2023-12-26T23:09:46.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.811+0000 7f61c57fa700 1 -- 172.21.15.131:0/962849604 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f61cc013680 con 0x7f61d00b00e0 2023-12-26T23:09:46.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.812+0000 7f61c57fa700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61b4066f30 0x7f61b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:46.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.812+0000 7f61c57fa700 1 -- 172.21.15.131:0/962849604 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f61cc084c10 con 0x7f61d00b00e0 2023-12-26T23:09:46.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.812+0000 7f61d499a700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61b4066f30 0x7f61b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:46.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.813+0000 7f61d499a700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61b4066f30 0x7f61b40693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f61d00b2120 tx=0x7f61c0009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:46.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:46.816+0000 7f61c57fa700 1 -- 172.21.15.131:0/962849604 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61cc051bf0 con 0x7f61d00b00e0 2023-12-26T23:09:47.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.011+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61d0067850 con 0x7f61d00b00e0 2023-12-26T23:09:47.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.012+0000 7f61c57fa700 1 -- 172.21.15.131:0/962849604 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+16183 (secure 0 0 0) 0x7f61cc055210 con 0x7f61d00b00e0 2023-12-26T23:09:47.013 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:47.013 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":192,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:09:46.045524+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:09:43.598995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-26T23:09:45.039854+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:09:47.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61b4066f30 msgr2=0x7f61b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61b4066f30 0x7f61b40693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f61d00b2120 tx=0x7f61c0009040 comp rx=0 tx=0).stop 2023-12-26T23:09:47.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61d00b00e0 msgr2=0x7f61d015a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61d00b00e0 0x7f61d015a9e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00ec80 tx=0x7f61cc0091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 shutdown_connections 2023-12-26T23:09:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f61b4066f30 0x7f61b40693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61d00cc580 0x7f61d014d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61d00b00e0 0x7f61d015a9e0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 --2- 172.21.15.131:0/962849604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61d00b0560 0x7f61d014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 >> 172.21.15.131:0/962849604 conn(0x7f61d00686c0 msgr2=0x7f61d00a9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 shutdown_connections 2023-12-26T23:09:47.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.015+0000 7f61d73ff700 1 -- 172.21.15.131:0/962849604 wait complete. 2023-12-26T23:09:47.033 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:09:47.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.224+0000 7f14fabff700 1 Processor -- start 2023-12-26T23:09:47.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.224+0000 7f14fabff700 1 -- start start 2023-12-26T23:09:47.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.225+0000 7f14fabff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f415d240 0x7f14f40ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.225+0000 7f14fabff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14f40ad390 0x7f14f40aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.226 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.225+0000 7f14fabff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f40a98b0 con 0x7f14f41595e0 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.225+0000 7f14fabff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f40add40 con 0x7f14f40ad390 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.225+0000 7f14fabff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f40ad7b0 con 0x7f14f415d240 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.225+0000 7f14f919c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14f40ad390 0x7f14f40aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.225+0000 7f14f919c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14f40ad390 0x7f14f40aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52300/0 (socket says 172.21.15.131:52300) 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.225+0000 7f14f919c700 1 -- 172.21.15.131:0/3745933703 learned_addr learned my addr 172.21.15.131:0/3745933703 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.225+0000 7f14f3fff700 1 --2- 172.21.15.131:0/3745933703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f415d240 0x7f14f40ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.226+0000 7f14f919c700 1 -- 172.21.15.131:0/3745933703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f415d240 msgr2=0x7f14f40ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.226+0000 7f14f919c700 1 --2- 172.21.15.131:0/3745933703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f415d240 0x7f14f40ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.226+0000 7f14f919c700 1 -- 172.21.15.131:0/3745933703 >> v1:172.21.15.190:6789/0 conn(0x7f14f41595e0 legacy=0x7f14f41599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.226+0000 7f14f919c700 1 -- 172.21.15.131:0/3745933703 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f415a500 con 0x7f14f40ad390 2023-12-26T23:09:47.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.226+0000 7f14f919c700 1 --2- 172.21.15.131:0/3745933703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f40ad390 0x7f14f40aa020 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009f20 tx=0x7f14ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12bd460645c2c71 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:47.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.226+0000 7f14f37fe700 1 -- 172.21.15.131:0/3745933703 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec00e070 con 0x7f14f40ad390 2023-12-26T23:09:47.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.226+0000 7f14f37fe700 1 -- 172.21.15.131:0/3745933703 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec00b9d0 con 0x7f14f40ad390 2023-12-26T23:09:47.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.226+0000 7f14f37fe700 1 -- 172.21.15.131:0/3745933703 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0092e0 con 0x7f14f40ad390 2023-12-26T23:09:47.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.227+0000 7f14fabff700 1 -- 172.21.15.131:0/3745933703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f40ad390 msgr2=0x7f14f40aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.227+0000 7f14fabff700 1 --2- 172.21.15.131:0/3745933703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f40ad390 0x7f14f40aa020 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009f20 tx=0x7f14ec009840 comp rx=0 tx=0).stop 2023-12-26T23:09:47.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.227+0000 7f14fabff700 1 -- 172.21.15.131:0/3745933703 shutdown_connections 2023-12-26T23:09:47.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.227+0000 7f14fabff700 1 --2- 172.21.15.131:0/3745933703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f40ad390 0x7f14f40aa020 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.227+0000 7f14fabff700 1 --2- 172.21.15.131:0/3745933703 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f415d240 0x7f14f40ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.227+0000 7f14fabff700 1 -- 172.21.15.131:0/3745933703 >> 172.21.15.131:0/3745933703 conn(0x7f14f4068040 msgr2=0x7f14f4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:47.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.227+0000 7f14fabff700 1 -- 172.21.15.131:0/3745933703 shutdown_connections 2023-12-26T23:09:47.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.227+0000 7f14fabff700 1 -- 172.21.15.131:0/3745933703 wait complete. 2023-12-26T23:09:47.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.229+0000 7f14fabff700 1 Processor -- start 2023-12-26T23:09:47.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.229+0000 7f14fabff700 1 -- start start 2023-12-26T23:09:47.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.230+0000 7f14fabff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f40ad390 0x7f14f40af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.230+0000 7f14fabff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14f41595e0 0x7f14f4150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.230+0000 7f14fabff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f415d240 0x7f14f4147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.230+0000 7f14fabff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f41553c0 con 0x7f14f415d240 2023-12-26T23:09:47.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.230+0000 7f14fabff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f4151090 con 0x7f14f41595e0 2023-12-26T23:09:47.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.230+0000 7f14fabff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f4148280 con 0x7f14f40ad390 2023-12-26T23:09:47.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f415d240 0x7f14f4147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f919c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f415d240 0x7f14f4147d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52314/0 (socket says 172.21.15.131:52314) 2023-12-26T23:09:47.233 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f919c700 1 -- 172.21.15.131:0/568847720 learned_addr learned my addr 172.21.15.131:0/568847720 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f3fff700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14f41595e0 0x7f14f4150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f899b700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f40ad390 0x7f14f40af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f919c700 1 -- 172.21.15.131:0/568847720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f40ad390 msgr2=0x7f14f40af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f919c700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f40ad390 0x7f14f40af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f919c700 1 -- 172.21.15.131:0/568847720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14f41595e0 msgr2=0x7f14f4150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f919c700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14f41595e0 0x7f14f4150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f919c700 1 -- 172.21.15.131:0/568847720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec009ae0 con 0x7f14f415d240 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.231+0000 7f14f919c700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f415d240 0x7f14f4147d40 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00b610 tx=0x7f14ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.232+0000 7f14f0ff9700 1 -- 172.21.15.131:0/568847720 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec008ad0 con 0x7f14f415d240 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.232+0000 7f14f0ff9700 1 -- 172.21.15.131:0/568847720 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec008c50 con 0x7f14f415d240 2023-12-26T23:09:47.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.232+0000 7f14f0ff9700 1 -- 172.21.15.131:0/568847720 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec026460 con 0x7f14f415d240 2023-12-26T23:09:47.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.232+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f40afba0 con 0x7f14f415d240 2023-12-26T23:09:47.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.232+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f40b0160 con 0x7f14f415d240 2023-12-26T23:09:47.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.233+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f4155190 con 0x7f14f415d240 2023-12-26T23:09:47.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.234+0000 7f14f0ff9700 1 -- 172.21.15.131:0/568847720 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f14ec003040 con 0x7f14f415d240 2023-12-26T23:09:47.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.234+0000 7f14f0ff9700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f14d8066f30 0x7f14d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.235+0000 7f14f0ff9700 1 -- 172.21.15.131:0/568847720 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f14ec085f90 con 0x7f14f415d240 2023-12-26T23:09:47.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.235+0000 7f14f899b700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f14d8066f30 0x7f14d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.235+0000 7f14f899b700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f14d8066f30 0x7f14d80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14e8003a80 tx=0x7f14e8007ec0 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:47.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.239+0000 7f14f0ff9700 1 -- 172.21.15.131:0/568847720 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14f4155190 con 0x7f14f415d240 2023-12-26T23:09:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.432+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14f4155190 con 0x7f14f415d240 2023-12-26T23:09:47.433 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.433+0000 7f14f0ff9700 1 -- 172.21.15.131:0/568847720 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+16183 (secure 0 0 0) 0x7f14f4155190 con 0x7f14f415d240 2023-12-26T23:09:47.433 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:47.433 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":192,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:09:46.045524+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:09:43.598995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-26T23:09:45.039854+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:09:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.435+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f14d8066f30 msgr2=0x7f14d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.435+0000 7f14fabff700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f14d8066f30 0x7f14d80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14e8003a80 tx=0x7f14e8007ec0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.435+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f415d240 msgr2=0x7f14f4147d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.435+0000 7f14fabff700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f415d240 0x7f14f4147d40 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00b610 tx=0x7f14ec018000 comp rx=0 tx=0).stop 2023-12-26T23:09:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.435+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 shutdown_connections 2023-12-26T23:09:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.435+0000 7f14fabff700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f14d8066f30 0x7f14d80693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.435+0000 7f14fabff700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f415d240 0x7f14f4147d40 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.435+0000 7f14fabff700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f40ad390 0x7f14f40af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.435+0000 7f14fabff700 1 --2- 172.21.15.131:0/568847720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14f41595e0 0x7f14f4150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.436+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 >> 172.21.15.131:0/568847720 conn(0x7f14f4068040 msgr2=0x7f14f4156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.436+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 shutdown_connections 2023-12-26T23:09:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.436+0000 7f14fabff700 1 -- 172.21.15.131:0/568847720 wait complete. 2023-12-26T23:09:47.454 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T23:09:47.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.653+0000 7fb1577ff700 1 Processor -- start 2023-12-26T23:09:47.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.653+0000 7fb1577ff700 1 -- start start 2023-12-26T23:09:47.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.654+0000 7fb1577ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb15006bfd0 0x7fb1500c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.654+0000 7fb1577ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1500cc100 0x7fb1500cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.654+0000 7fb1577ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1500686c0 con 0x7fb15013c510 2023-12-26T23:09:47.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.654+0000 7fb1577ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb150069520 con 0x7fb1500cc100 2023-12-26T23:09:47.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.654+0000 7fb1577ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb150068910 con 0x7fb15006bfd0 2023-12-26T23:09:47.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb155d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1500cc100 0x7fb1500cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb155d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1500cc100 0x7fb1500cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52328/0 (socket says 172.21.15.131:52328) 2023-12-26T23:09:47.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb155d9c700 1 -- 172.21.15.131:0/4144626843 learned_addr learned my addr 172.21.15.131:0/4144626843 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:47.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb15559b700 1 --1- 172.21.15.131:0/4144626843 >> v1:172.21.15.131:6789/0 conn(0x7fb15013c510 0x7fb15006b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55086/0 (socket says 172.21.15.131:55086) 2023-12-26T23:09:47.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb154d9a700 1 --2- 172.21.15.131:0/4144626843 >> v2:172.21.15.190:3300/0 conn(0x7fb15006bfd0 0x7fb1500c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb155d9c700 1 -- 172.21.15.131:0/4144626843 >> v2:172.21.15.190:3300/0 conn(0x7fb15006bfd0 msgr2=0x7fb1500c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb155d9c700 1 --2- 172.21.15.131:0/4144626843 >> v2:172.21.15.190:3300/0 conn(0x7fb15006bfd0 0x7fb1500c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb155d9c700 1 -- 172.21.15.131:0/4144626843 >> v1:172.21.15.131:6789/0 conn(0x7fb15013c510 legacy=0x7fb15006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb155d9c700 1 -- 172.21.15.131:0/4144626843 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb150068bf0 con 0x7fb1500cc100 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb147fff700 1 -- 172.21.15.131:0/4144626843 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243284151 0 0) 0x7fb1500686c0 con 0x7fb15013c510 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb155d9c700 1 --2- 172.21.15.131:0/4144626843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 0x7fb1500cc520 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb1500b43e0 tx=0x7fb148009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ecba3631b8bfd4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.655+0000 7fb147fff700 1 -- 172.21.15.131:0/4144626843 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800e070 con 0x7fb1500cc100 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb147fff700 1 -- 172.21.15.131:0/4144626843 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800ba50 con 0x7fb1500cc100 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb147fff700 1 -- 172.21.15.131:0/4144626843 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1480092e0 con 0x7fb1500cc100 2023-12-26T23:09:47.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb1577ff700 1 -- 172.21.15.131:0/4144626843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 msgr2=0x7fb1500cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb1577ff700 1 --2- 172.21.15.131:0/4144626843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 0x7fb1500cc520 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb1500b43e0 tx=0x7fb148009840 comp rx=0 tx=0).stop 2023-12-26T23:09:47.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb1577ff700 1 -- 172.21.15.131:0/4144626843 shutdown_connections 2023-12-26T23:09:47.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb1577ff700 1 --2- 172.21.15.131:0/4144626843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 0x7fb1500cc520 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb1577ff700 1 --2- 172.21.15.131:0/4144626843 >> v2:172.21.15.190:3300/0 conn(0x7fb15006bfd0 0x7fb1500c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb1577ff700 1 -- 172.21.15.131:0/4144626843 >> 172.21.15.131:0/4144626843 conn(0x7fb1500ce3e0 msgr2=0x7fb1500ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:47.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb1577ff700 1 -- 172.21.15.131:0/4144626843 shutdown_connections 2023-12-26T23:09:47.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.656+0000 7fb1577ff700 1 -- 172.21.15.131:0/4144626843 wait complete. 2023-12-26T23:09:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.658+0000 7fb1577ff700 1 Processor -- start 2023-12-26T23:09:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.658+0000 7fb1577ff700 1 -- start start 2023-12-26T23:09:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb1577ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb15006bfd0 0x7fb15016da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb1577ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 0x7fb15013ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb1577ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb15013c510 0x7fb150159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb1577ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15015bf60 con 0x7fb1500cc100 2023-12-26T23:09:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb1577ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1500b27b0 con 0x7fb15013c510 2023-12-26T23:09:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb1577ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15015a260 con 0x7fb15006bfd0 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb154d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 0x7fb15013ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb154d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 0x7fb15013ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52330/0 (socket says 172.21.15.131:52330) 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb154d9a700 1 -- 172.21.15.131:0/3804352867 learned_addr learned my addr 172.21.15.131:0/3804352867 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb15559b700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb15006bfd0 0x7fb15016da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb155d9c700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb15013c510 0x7fb150159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb154d9a700 1 -- 172.21.15.131:0/3804352867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb15006bfd0 msgr2=0x7fb15016da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb154d9a700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb15006bfd0 0x7fb15016da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb154d9a700 1 -- 172.21.15.131:0/3804352867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb15013c510 msgr2=0x7fb150159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb154d9a700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb15013c510 0x7fb150159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:47.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb154d9a700 1 -- 172.21.15.131:0/3804352867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb148009ae0 con 0x7fb1500cc100 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.659+0000 7fb155d9c700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb15013c510 0x7fb150159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.660+0000 7fb154d9a700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 0x7fb15013ed40 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb14000d860 tx=0x7fb140008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.660+0000 7fb1457fa700 1 -- 172.21.15.131:0/3804352867 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1400125f0 con 0x7fb1500cc100 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.660+0000 7fb1457fa700 1 -- 172.21.15.131:0/3804352867 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb140012770 con 0x7fb1500cc100 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.660+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15015c1b0 con 0x7fb1500cc100 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.660+0000 7fb1457fa700 1 -- 172.21.15.131:0/3804352867 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140012b30 con 0x7fb1500cc100 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.660+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1500b2440 con 0x7fb1500cc100 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.661+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1500b2920 con 0x7fb1500cc100 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.662+0000 7fb1457fa700 1 -- 172.21.15.131:0/3804352867 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb14001e460 con 0x7fb1500cc100 2023-12-26T23:09:47.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.663+0000 7fb1457fa700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb134066f30 0x7fb1340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:47.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.663+0000 7fb1457fa700 1 -- 172.21.15.131:0/3804352867 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fb140084c50 con 0x7fb1500cc100 2023-12-26T23:09:47.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.663+0000 7fb15559b700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb134066f30 0x7fb1340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:47.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.664+0000 7fb15559b700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb134066f30 0x7fb1340693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0012a0 tx=0x7fb14c009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:47.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.667+0000 7fb1457fa700 1 -- 172.21.15.131:0/3804352867 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb14008a020 con 0x7fb1500cc100 2023-12-26T23:09:47.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:47.902+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb1500657e0 con 0x7fb1500cc100 2023-12-26T23:09:48.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.597+0000 7fb1457fa700 1 -- 172.21.15.131:0/3804352867 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb140055760 con 0x7fb1500cc100 2023-12-26T23:09:48.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb134066f30 msgr2=0x7fb1340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:48.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb134066f30 0x7fb1340693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0012a0 tx=0x7fb14c009040 comp rx=0 tx=0).stop 2023-12-26T23:09:48.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 msgr2=0x7fb15013ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:48.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 0x7fb15013ed40 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb14000d860 tx=0x7fb140008040 comp rx=0 tx=0).stop 2023-12-26T23:09:48.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 shutdown_connections 2023-12-26T23:09:48.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb134066f30 0x7fb1340693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:48.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb15013c510 0x7fb150159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:48.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb15006bfd0 0x7fb15016da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:48.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 --2- 172.21.15.131:0/3804352867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1500cc100 0x7fb15013ed40 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:48.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 >> 172.21.15.131:0/3804352867 conn(0x7fb1500ce3e0 msgr2=0x7fb150159260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:48.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 shutdown_connections 2023-12-26T23:09:48.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.600+0000 7fb1577ff700 1 -- 172.21.15.131:0/3804352867 wait complete. 2023-12-26T23:09:48.622 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T23:09:48.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.817+0000 7fb4d75ff700 1 Processor -- start 2023-12-26T23:09:48.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.817+0000 7fb4d75ff700 1 -- start start 2023-12-26T23:09:48.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.818+0000 7fb4d75ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb4d0159a70 0x7fb4d00b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:48.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.818+0000 7fb4d75ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb4d00b2a90 0x7fb4d00b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:48.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.818+0000 7fb4d75ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d00afcd0 con 0x7fb4d01596a0 2023-12-26T23:09:48.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.818+0000 7fb4d75ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d00b3e70 con 0x7fb4d00b2a90 2023-12-26T23:09:48.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.818+0000 7fb4d75ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d00b0f20 con 0x7fb4d0159a70 2023-12-26T23:09:48.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4d5b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb4d00b2a90 0x7fb4d00b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:48.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4d5b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb4d00b2a90 0x7fb4d00b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52344/0 (socket says 172.21.15.131:52344) 2023-12-26T23:09:48.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4d5b9c700 1 -- 172.21.15.131:0/3805553628 learned_addr learned my addr 172.21.15.131:0/3805553628 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4d4b9a700 1 --2- 172.21.15.131:0/3805553628 >> v2:172.21.15.190:3300/0 conn(0x7fb4d0159a70 0x7fb4d00b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4d5b9c700 1 -- 172.21.15.131:0/3805553628 >> v2:172.21.15.190:3300/0 conn(0x7fb4d0159a70 msgr2=0x7fb4d00b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4d5b9c700 1 --2- 172.21.15.131:0/3805553628 >> v2:172.21.15.190:3300/0 conn(0x7fb4d0159a70 0x7fb4d00b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4d5b9c700 1 -- 172.21.15.131:0/3805553628 >> v1:172.21.15.190:6789/0 conn(0x7fb4d01596a0 legacy=0x7fb4d00b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4d5b9c700 1 -- 172.21.15.131:0/3805553628 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d00b3540 con 0x7fb4d00b2a90 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4d5b9c700 1 --2- 172.21.15.131:0/3805553628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d00b2a90 0x7fb4d00b06a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00642f0 tx=0x7fb4c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7adc64184aef3eda server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.819+0000 7fb4c7fff700 1 -- 172.21.15.131:0/3805553628 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800e070 con 0x7fb4d00b2a90 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4c7fff700 1 -- 172.21.15.131:0/3805553628 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c800ba50 con 0x7fb4d00b2a90 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4c7fff700 1 -- 172.21.15.131:0/3805553628 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80092e0 con 0x7fb4d00b2a90 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4d75ff700 1 -- 172.21.15.131:0/3805553628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d00b2a90 msgr2=0x7fb4d00b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:48.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4d75ff700 1 --2- 172.21.15.131:0/3805553628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d00b2a90 0x7fb4d00b06a0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00642f0 tx=0x7fb4c8009840 comp rx=0 tx=0).stop 2023-12-26T23:09:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4d75ff700 1 -- 172.21.15.131:0/3805553628 shutdown_connections 2023-12-26T23:09:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4d75ff700 1 --2- 172.21.15.131:0/3805553628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d00b2a90 0x7fb4d00b06a0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4d75ff700 1 --2- 172.21.15.131:0/3805553628 >> v2:172.21.15.190:3300/0 conn(0x7fb4d0159a70 0x7fb4d00b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4d75ff700 1 -- 172.21.15.131:0/3805553628 >> 172.21.15.131:0/3805553628 conn(0x7fb4d006ece0 msgr2=0x7fb4d006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4d75ff700 1 -- 172.21.15.131:0/3805553628 shutdown_connections 2023-12-26T23:09:48.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.820+0000 7fb4d75ff700 1 -- 172.21.15.131:0/3805553628 wait complete. 2023-12-26T23:09:48.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.822+0000 7fb4d75ff700 1 Processor -- start 2023-12-26T23:09:48.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.822+0000 7fb4d75ff700 1 -- start start 2023-12-26T23:09:48.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d75ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d00b2a90 0x7fb4d015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:48.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d75ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d01596a0 0x7fb4d0156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:48.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d75ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0159a70 0x7fb4d01580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d75ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0154440 con 0x7fb4d01596a0 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d75ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0154140 con 0x7fb4d0159a70 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d75ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0157600 con 0x7fb4d00b2a90 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d4b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d01596a0 0x7fb4d0156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d4b9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d01596a0 0x7fb4d0156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52348/0 (socket says 172.21.15.131:52348) 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d4b9a700 1 -- 172.21.15.131:0/1320951982 learned_addr learned my addr 172.21.15.131:0/1320951982 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d5b9c700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0159a70 0x7fb4d01580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.823+0000 7fb4d539b700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d00b2a90 0x7fb4d015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4d4b9a700 1 -- 172.21.15.131:0/1320951982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d00b2a90 msgr2=0x7fb4d015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:48.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4d4b9a700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d00b2a90 0x7fb4d015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4d4b9a700 1 -- 172.21.15.131:0/1320951982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0159a70 msgr2=0x7fb4d01580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4d4b9a700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0159a70 0x7fb4d01580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4d4b9a700 1 -- 172.21.15.131:0/1320951982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c8009ae0 con 0x7fb4d01596a0 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4d539b700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d00b2a90 0x7fb4d015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4d4b9a700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d01596a0 0x7fb4d0156d20 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000d860 tx=0x7fb4c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4c57fa700 1 -- 172.21.15.131:0/1320951982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00125f0 con 0x7fb4d01596a0 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4c57fa700 1 -- 172.21.15.131:0/1320951982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c0012750 con 0x7fb4d01596a0 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d01578e0 con 0x7fb4d01596a0 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4c57fa700 1 -- 172.21.15.131:0/1320951982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0012b10 con 0x7fb4d01596a0 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.824+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d0157260 con 0x7fb4d01596a0 2023-12-26T23:09:48.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.825+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d01545f0 con 0x7fb4d01596a0 2023-12-26T23:09:48.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.826+0000 7fb4c57fa700 1 -- 172.21.15.131:0/1320951982 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb4c001e460 con 0x7fb4d01596a0 2023-12-26T23:09:48.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.827+0000 7fb4c57fa700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb4b4066f30 0x7fb4b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:48.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.827+0000 7fb4c57fa700 1 -- 172.21.15.131:0/1320951982 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fb4c0084e50 con 0x7fb4d01596a0 2023-12-26T23:09:48.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.827+0000 7fb4d539b700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb4b4066f30 0x7fb4b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:48.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.828+0000 7fb4d539b700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb4b4066f30 0x7fb4b40693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4d015cbd0 tx=0x7fb4cc009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:48.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:48.831+0000 7fb4c57fa700 1 -- 172.21.15.131:0/1320951982 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4d01545f0 con 0x7fb4d01596a0 2023-12-26T23:09:49.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.070+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb4d00657e0 con 0x7fb4d01596a0 2023-12-26T23:09:49.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.604+0000 7fb4c57fa700 1 -- 172.21.15.131:0/1320951982 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb4c0055630 con 0x7fb4d01596a0 2023-12-26T23:09:49.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb4b4066f30 msgr2=0x7fb4b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:49.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb4b4066f30 0x7fb4b40693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4d015cbd0 tx=0x7fb4cc009040 comp rx=0 tx=0).stop 2023-12-26T23:09:49.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d01596a0 msgr2=0x7fb4d0156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:49.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d01596a0 0x7fb4d0156d20 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000d860 tx=0x7fb4c0008040 comp rx=0 tx=0).stop 2023-12-26T23:09:49.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 shutdown_connections 2023-12-26T23:09:49.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb4b4066f30 0x7fb4b40693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:49.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4d0159a70 0x7fb4d01580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:49.607 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4d00b2a90 0x7fb4d015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:49.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 --2- 172.21.15.131:0/1320951982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4d01596a0 0x7fb4d0156d20 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:49.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 >> 172.21.15.131:0/1320951982 conn(0x7fb4d006ece0 msgr2=0x7fb4d00cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:49.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 shutdown_connections 2023-12-26T23:09:49.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.607+0000 7fb4d75ff700 1 -- 172.21.15.131:0/1320951982 wait complete. 2023-12-26T23:09:49.626 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:09:49.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.815+0000 7f8d747ff700 1 Processor -- start 2023-12-26T23:09:49.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.815+0000 7f8d747ff700 1 -- start start 2023-12-26T23:09:49.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.816+0000 7f8d747ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c13c510 0x7f8d6c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:49.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.816+0000 7f8d747ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8d6c06b8c0 0x7f8d6c06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:49.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.816+0000 7f8d747ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d6c06c270 con 0x7f8d6c0cc100 2023-12-26T23:09:49.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.816+0000 7f8d747ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d6c0686c0 con 0x7f8d6c06b8c0 2023-12-26T23:09:49.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.816+0000 7f8d747ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c069520 con 0x7f8d6c13c510 2023-12-26T23:09:49.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d71d9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8d6c06b8c0 0x7f8d6c06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:49.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d7259b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c13c510 0x7f8d6c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d72d9c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8d6c0cc100 0x7f8d6c0cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:35818/0 (socket says 172.21.15.131:35818) 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d72d9c700 1 -- 172.21.15.131:0/2595240790 learned_addr learned my addr 172.21.15.131:0/2595240790 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d71599700 1 -- 172.21.15.131:0/2595240790 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812900779 0 0) 0x7f8d6c06c270 con 0x7f8d6c0cc100 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d7259b700 1 -- 172.21.15.131:0/2595240790 >> v2:172.21.15.190:3300/0 conn(0x7f8d6c06b8c0 msgr2=0x7f8d6c06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d7259b700 1 --2- 172.21.15.131:0/2595240790 >> v2:172.21.15.190:3300/0 conn(0x7f8d6c06b8c0 0x7f8d6c06bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d7259b700 1 -- 172.21.15.131:0/2595240790 >> v1:172.21.15.190:6789/0 conn(0x7f8d6c0cc100 legacy=0x7f8d6c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d7259b700 1 -- 172.21.15.131:0/2595240790 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c068910 con 0x7f8d6c13c510 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d71d9a700 1 --2- 172.21.15.131:0/2595240790 >> v2:172.21.15.190:3300/0 conn(0x7f8d6c06b8c0 0x7f8d6c06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.817+0000 7f8d7259b700 1 --2- 172.21.15.131:0/2595240790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c13c510 0x7f8d6c0c8a20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8d680098d0 tx=0x7f8d68009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b79fbc54518e7778 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d71599700 1 -- 172.21.15.131:0/2595240790 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68005c10 con 0x7f8d6c13c510 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d71599700 1 -- 172.21.15.131:0/2595240790 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68005d70 con 0x7f8d6c13c510 2023-12-26T23:09:49.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d71599700 1 -- 172.21.15.131:0/2595240790 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68006280 con 0x7f8d6c13c510 2023-12-26T23:09:49.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d747ff700 1 -- 172.21.15.131:0/2595240790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c13c510 msgr2=0x7f8d6c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:49.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d747ff700 1 --2- 172.21.15.131:0/2595240790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c13c510 0x7f8d6c0c8a20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8d680098d0 tx=0x7f8d68009120 comp rx=0 tx=0).stop 2023-12-26T23:09:49.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d747ff700 1 -- 172.21.15.131:0/2595240790 shutdown_connections 2023-12-26T23:09:49.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d747ff700 1 --2- 172.21.15.131:0/2595240790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c13c510 0x7f8d6c0c8a20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:49.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d747ff700 1 --2- 172.21.15.131:0/2595240790 >> v2:172.21.15.190:3300/0 conn(0x7f8d6c06b8c0 0x7f8d6c06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:49.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d747ff700 1 -- 172.21.15.131:0/2595240790 >> 172.21.15.131:0/2595240790 conn(0x7f8d6c0ce3e0 msgr2=0x7f8d6c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:49.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d747ff700 1 -- 172.21.15.131:0/2595240790 shutdown_connections 2023-12-26T23:09:49.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.818+0000 7f8d747ff700 1 -- 172.21.15.131:0/2595240790 wait complete. 2023-12-26T23:09:49.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.820+0000 7f8d747ff700 1 Processor -- start 2023-12-26T23:09:49.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.820+0000 7f8d747ff700 1 -- start start 2023-12-26T23:09:49.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d747ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c06b8c0 0x7f8d6c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:49.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d747ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d6c0cc100 0x7f8d6c16d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:49.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d747ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c13c510 0x7f8d6c15be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:49.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d747ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c13bec0 con 0x7f8d6c13c510 2023-12-26T23:09:49.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d747ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c1589e0 con 0x7f8d6c0cc100 2023-12-26T23:09:49.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d747ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c16dd00 con 0x7f8d6c06b8c0 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d72d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c13c510 0x7f8d6c15be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d72d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c13c510 0x7f8d6c15be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52360/0 (socket says 172.21.15.131:52360) 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d72d9c700 1 -- 172.21.15.131:0/551473786 learned_addr learned my addr 172.21.15.131:0/551473786 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d71d9a700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d6c0cc100 0x7f8d6c16d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.821+0000 7f8d7259b700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c06b8c0 0x7f8d6c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d72d9c700 1 -- 172.21.15.131:0/551473786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c06b8c0 msgr2=0x7f8d6c158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d72d9c700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c06b8c0 0x7f8d6c158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d72d9c700 1 -- 172.21.15.131:0/551473786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d6c0cc100 msgr2=0x7f8d6c16d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d72d9c700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d6c0cc100 0x7f8d6c16d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d72d9c700 1 -- 172.21.15.131:0/551473786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c009420 con 0x7f8d6c13c510 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d71d9a700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d6c0cc100 0x7f8d6c16d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d72d9c700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c13c510 0x7f8d6c15be80 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400cc90 tx=0x7f8d64001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d627fc700 1 -- 172.21.15.131:0/551473786 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400cea0 con 0x7f8d6c13c510 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d627fc700 1 -- 172.21.15.131:0/551473786 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6400ab70 con 0x7f8d6c13c510 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d680093c0 con 0x7f8d6c13c510 2023-12-26T23:09:49.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d627fc700 1 -- 172.21.15.131:0/551473786 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640070f0 con 0x7f8d6c13c510 2023-12-26T23:09:49.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.822+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6c1580b0 con 0x7f8d6c13c510 2023-12-26T23:09:49.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.823+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6c0cf0d0 con 0x7f8d6c13c510 2023-12-26T23:09:49.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.824+0000 7f8d627fc700 1 -- 172.21.15.131:0/551473786 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8d640074a0 con 0x7f8d6c13c510 2023-12-26T23:09:49.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.825+0000 7f8d627fc700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8d50066f30 0x7f8d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:49.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.825+0000 7f8d627fc700 1 -- 172.21.15.131:0/551473786 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f8d64084c00 con 0x7f8d6c13c510 2023-12-26T23:09:49.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.825+0000 7f8d7259b700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8d50066f30 0x7f8d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:49.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.826+0000 7f8d7259b700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8d50066f30 0x7f8d500693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009160 tx=0x7f8d68016000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:49.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:49.829+0000 7f8d627fc700 1 -- 172.21.15.131:0/551473786 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d6408b080 con 0x7f8d6c13c510 2023-12-26T23:09:50.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.064+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d6c158b50 con 0x7f8d6c13c510 2023-12-26T23:09:50.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.065+0000 7f8d627fc700 1 -- 172.21.15.131:0/551473786 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4613 (secure 0 0 0) 0x7f8d64051d00 con 0x7f8d6c13c510 2023-12-26T23:09:50.065 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:50.065 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2070005366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2070005366},{"type":"v1","addr":"172.21.15.190:6835","nonce":2070005366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3800715315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3800715315},{"type":"v1","addr":"172.21.15.190:6834","nonce":3800715315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9645,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3865566903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3865566903},{"type":"v1","addr":"172.21.15.131:6833","nonce":3865566903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:09:46.045975+0000","modified":"2023-12-26T23:09:49.597663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9666":{"gid":9666,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2967199312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2967199312},{"type":"v1","addr":"172.21.15.131:6835","nonce":2967199312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-12-26T23:09:50.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8d50066f30 msgr2=0x7f8d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8d50066f30 0x7f8d500693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009160 tx=0x7f8d68016000 comp rx=0 tx=0).stop 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c13c510 msgr2=0x7f8d6c15be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c13c510 0x7f8d6c15be80 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400cc90 tx=0x7f8d64001a80 comp rx=0 tx=0).stop 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 shutdown_connections 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8d50066f30 0x7f8d500693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d6c13c510 0x7f8d6c15be80 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d6c06b8c0 0x7f8d6c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 --2- 172.21.15.131:0/551473786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d6c0cc100 0x7f8d6c16d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 >> 172.21.15.131:0/551473786 conn(0x7f8d6c0ce3e0 msgr2=0x7f8d6c0cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 shutdown_connections 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.067+0000 7f8d747ff700 1 -- 172.21.15.131:0/551473786 wait complete. 2023-12-26T23:09:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 90 2023-12-26T23:09:50.085 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:09:50.274 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.274+0000 7fc52cbff700 1 Processor -- start 2023-12-26T23:09:50.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.275+0000 7fc52cbff700 1 -- start start 2023-12-26T23:09:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc52cbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5281596a0 0x7fc5280b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc52cbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc5280b0870 0x7fc5280b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc52cbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280afcd0 con 0x7fc528159a70 2023-12-26T23:09:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc52cbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280b3e70 con 0x7fc5280b0870 2023-12-26T23:09:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc52cbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280b0cd0 con 0x7fc5281596a0 2023-12-26T23:09:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc525d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc528159a70 0x7fc5280b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55090/0 (socket says 172.21.15.131:55090) 2023-12-26T23:09:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc525d9b700 1 -- 172.21.15.131:0/154164364 learned_addr learned my addr 172.21.15.131:0/154164364 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:50.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc526d9d700 1 --2- 172.21.15.131:0/154164364 >> v2:172.21.15.190:3300/0 conn(0x7fc5280b0870 0x7fc5280b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc52659c700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5281596a0 0x7fc5280b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.276+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686020958 0 0) 0x7fc5280afcd0 con 0x7fc528159a70 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc510003570 con 0x7fc528159a70 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1784553855 0 0) 0x7fc510003570 con 0x7fc528159a70 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5281596a0 msgr2=0x7fc5280b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5281596a0 0x7fc5280b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 >> v2:172.21.15.190:3300/0 conn(0x7fc5280b0870 msgr2=0x7fc5280b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 --2- 172.21.15.131:0/154164364 >> v2:172.21.15.190:3300/0 conn(0x7fc5280b0870 0x7fc5280b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280b2e00 con 0x7fc528159a70 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fc51c001470 con 0x7fc528159a70 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 >> v1:172.21.15.131:6789/0 conn(0x7fc528159a70 legacy=0x7fc5280b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003a60 0x7fc510003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510004810 0x7fc510004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5281596a0 0x7fc5100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280b3e70 con 0x7fc510003a60 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280b0cd0 con 0x7fc510004810 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51000ba60 con 0x7fc5281596a0 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc52659c700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003a60 0x7fc510003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.277+0000 7fc525d9b700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510004810 0x7fc510004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc526d9d700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5281596a0 0x7fc5100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc52659c700 1 -- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5281596a0 msgr2=0x7fc5100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc52659c700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5281596a0 0x7fc5100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc52659c700 1 -- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510004810 msgr2=0x7fc510004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:50.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc52659c700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510004810 0x7fc510004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc52659c700 1 -- 172.21.15.131:0/154164364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc510003570 con 0x7fc510003a60 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc526d9d700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5281596a0 0x7fc5100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc525d9b700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510004810 0x7fc510004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc52659c700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003a60 0x7fc510003e70 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fc518009c30 tx=0x7fc5180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8cc74256ac6373 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51800e710 con 0x7fc510003a60 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51800e870 con 0x7fc510003a60 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.278+0000 7fc52559a700 1 -- 172.21.15.131:0/154164364 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51800ec30 con 0x7fc510003a60 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.279+0000 7fc52cbff700 1 -- 172.21.15.131:0/154164364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003a60 msgr2=0x7fc510003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.279+0000 7fc52cbff700 1 --2- 172.21.15.131:0/154164364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003a60 0x7fc510003e70 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fc518009c30 tx=0x7fc5180091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.279+0000 7fc526d9d700 1 -- 172.21.15.131:0/154164364 reap_dead start 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.279+0000 7fc52cbff700 1 -- 172.21.15.131:0/154164364 shutdown_connections 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.279+0000 7fc52cbff700 1 -- 172.21.15.131:0/154164364 >> 172.21.15.131:0/154164364 conn(0x7fc52806ece0 msgr2=0x7fc52806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:50.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.279+0000 7fc52cbff700 1 -- 172.21.15.131:0/154164364 shutdown_connections 2023-12-26T23:09:50.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.279+0000 7fc52cbff700 1 -- 172.21.15.131:0/154164364 wait complete. 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.280+0000 7fc52cbff700 1 Processor -- start 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.281+0000 7fc52cbff700 1 -- start start 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.281+0000 7fc52cbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510003a60 0x7fc52815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.281+0000 7fc52cbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5280b4460 0x7fc5280b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.281+0000 7fc52cbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc528159fb0 0x7fc52815a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.281+0000 7fc52cbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280ca5e0 con 0x7fc5280b4460 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.281+0000 7fc52cbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52815ce90 con 0x7fc510003a60 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.281+0000 7fc52cbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280b4e10 con 0x7fc528159fb0 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc525d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5280b4460 0x7fc5280b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc525d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5280b4460 0x7fc5280b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52374/0 (socket says 172.21.15.131:52374) 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc525d9b700 1 -- 172.21.15.131:0/1118364257 learned_addr learned my addr 172.21.15.131:0/1118364257 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc526d9d700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc528159fb0 0x7fc52815a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc52659c700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510003a60 0x7fc52815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc525d9b700 1 -- 172.21.15.131:0/1118364257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc528159fb0 msgr2=0x7fc52815a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc525d9b700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc528159fb0 0x7fc52815a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc525d9b700 1 -- 172.21.15.131:0/1118364257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510003a60 msgr2=0x7fc52815c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc525d9b700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510003a60 0x7fc52815c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc525d9b700 1 -- 172.21.15.131:0/1118364257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52816ac00 con 0x7fc5280b4460 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc525d9b700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5280b4460 0x7fc5280b48d0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00ec20 tx=0x7fc5280b3fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.282+0000 7fc5167fc700 1 -- 172.21.15.131:0/1118364257 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00c760 con 0x7fc5280b4460 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.283+0000 7fc5167fc700 1 -- 172.21.15.131:0/1118364257 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00c8c0 con 0x7fc5280b4460 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.283+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52815a960 con 0x7fc5280b4460 2023-12-26T23:09:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.283+0000 7fc5167fc700 1 -- 172.21.15.131:0/1118364257 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00cc40 con 0x7fc5280b4460 2023-12-26T23:09:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.283+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52816a2d0 con 0x7fc5280b4460 2023-12-26T23:09:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.284+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5280657e0 con 0x7fc5280b4460 2023-12-26T23:09:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.285+0000 7fc5167fc700 1 -- 172.21.15.131:0/1118364257 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc51c01d460 con 0x7fc5280b4460 2023-12-26T23:09:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.285+0000 7fc5167fc700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc508066f30 0x7fc5080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.285+0000 7fc5167fc700 1 -- 172.21.15.131:0/1118364257 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fc51c084680 con 0x7fc5280b4460 2023-12-26T23:09:50.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.286+0000 7fc52659c700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc508066f30 0x7fc5080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.286+0000 7fc52659c700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc508066f30 0x7fc5080693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc5180091e0 tx=0x7fc51801a000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:50.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.290+0000 7fc5167fc700 1 -- 172.21.15.131:0/1118364257 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc51c051700 con 0x7fc5280b4460 2023-12-26T23:09:50.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.487+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc52816ad90 con 0x7fc5280b4460 2023-12-26T23:09:50.487 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.488+0000 7fc5167fc700 1 -- 172.21.15.131:0/1118364257 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16170 (secure 0 0 0) 0x7fc51c054d20 con 0x7fc5280b4460 2023-12-26T23:09:50.488 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:50.488 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":193,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:09:47.578970+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:09:43.598995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-26T23:09:45.039854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:09:50.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc508066f30 msgr2=0x7fc5080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc508066f30 0x7fc5080693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc5180091e0 tx=0x7fc51801a000 comp rx=0 tx=0).stop 2023-12-26T23:09:50.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5280b4460 msgr2=0x7fc5280b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5280b4460 0x7fc5280b48d0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00ec20 tx=0x7fc5280b3fb0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 shutdown_connections 2023-12-26T23:09:50.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc508066f30 0x7fc5080693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc528159fb0 0x7fc52815a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc510003a60 0x7fc52815c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 --2- 172.21.15.131:0/1118364257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5280b4460 0x7fc5280b48d0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 >> 172.21.15.131:0/1118364257 conn(0x7fc52806ece0 msgr2=0x7fc52815d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:50.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 shutdown_connections 2023-12-26T23:09:50.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.490+0000 7fc52cbff700 1 -- 172.21.15.131:0/1118364257 wait complete. 2023-12-26T23:09:50.508 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T23:09:50.509 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:09:50.530 INFO:tasks.ceph.mds.a.smithi131.stderr:2023-12-26T23:09:50.531+0000 7f45131f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T23:09:50.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.701+0000 7fa7a29ff700 1 Processor -- start 2023-12-26T23:09:50.701 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.701+0000 7fa7a29ff700 1 -- start start 2023-12-26T23:09:50.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.702+0000 7fa7a29ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 0x7fa79c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.702+0000 7fa7a29ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa79c0c9400 0x7fa79c0c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.704 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.702+0000 7fa7a29ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79c156940 con 0x7fa79c0cb320 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.702+0000 7fa7a29ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa79c0c8cc0 con 0x7fa79c0c9400 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.702+0000 7fa7a29ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0ad700 con 0x7fa79c0b0750 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.702+0000 7fa7a0f9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa79c0c9400 0x7fa79c0c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.702+0000 7fa7a0f9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa79c0c9400 0x7fa79c0c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52386/0 (socket says 172.21.15.131:52386) 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.702+0000 7fa7a0f9c700 1 -- 172.21.15.131:0/3392235940 learned_addr learned my addr 172.21.15.131:0/3392235940 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa79b7fe700 1 --2- 172.21.15.131:0/3392235940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 0x7fa79c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa7a0f9c700 1 -- 172.21.15.131:0/3392235940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 msgr2=0x7fa79c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa7a0f9c700 1 --2- 172.21.15.131:0/3392235940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 0x7fa79c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa7a0f9c700 1 -- 172.21.15.131:0/3392235940 >> v1:172.21.15.190:6789/0 conn(0x7fa79c0cb320 legacy=0x7fa79c069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa7a0f9c700 1 -- 172.21.15.131:0/3392235940 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c0b1100 con 0x7fa79c0c9400 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa79affd700 1 -- 172.21.15.131:0/3392235940 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070214077 0 0) 0x7fa79c156940 con 0x7fa79c0cb320 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa7a0f9c700 1 --2- 172.21.15.131:0/3392235940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0c9400 0x7fa79c0c9880 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fa79400d1a0 tx=0x7fa79400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf3658741522b97 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa79affd700 1 -- 172.21.15.131:0/3392235940 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794010070 con 0x7fa79c0c9400 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa79affd700 1 -- 172.21.15.131:0/3392235940 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79400de10 con 0x7fa79c0c9400 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.703+0000 7fa79affd700 1 -- 172.21.15.131:0/3392235940 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79400b320 con 0x7fa79c0c9400 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.704+0000 7fa7a29ff700 1 -- 172.21.15.131:0/3392235940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0c9400 msgr2=0x7fa79c0c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.704+0000 7fa7a29ff700 1 --2- 172.21.15.131:0/3392235940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0c9400 0x7fa79c0c9880 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fa79400d1a0 tx=0x7fa79400ba50 comp rx=0 tx=0).stop 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.704+0000 7fa7a29ff700 1 -- 172.21.15.131:0/3392235940 shutdown_connections 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.704+0000 7fa7a29ff700 1 --2- 172.21.15.131:0/3392235940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0c9400 0x7fa79c0c9880 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.704+0000 7fa7a29ff700 1 --2- 172.21.15.131:0/3392235940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 0x7fa79c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.704+0000 7fa7a29ff700 1 -- 172.21.15.131:0/3392235940 >> 172.21.15.131:0/3392235940 conn(0x7fa79c068040 msgr2=0x7fa79c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.704+0000 7fa7a29ff700 1 -- 172.21.15.131:0/3392235940 shutdown_connections 2023-12-26T23:09:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.704+0000 7fa7a29ff700 1 -- 172.21.15.131:0/3392235940 wait complete. 2023-12-26T23:09:50.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.706+0000 7fa7a29ff700 1 Processor -- start 2023-12-26T23:09:50.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.706+0000 7fa7a29ff700 1 -- start start 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a29ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 0x7fa79c0c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a29ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c0c9400 0x7fa79c156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a29ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0cb320 0x7fa79c16cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a29ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c158120 con 0x7fa79c0cb320 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a29ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0c7030 con 0x7fa79c0c9400 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a29ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c15ad60 con 0x7fa79c0b0750 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a0f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0cb320 0x7fa79c16cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a0f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0cb320 0x7fa79c16cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52402/0 (socket says 172.21.15.131:52402) 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a0f9c700 1 -- 172.21.15.131:0/2311552803 learned_addr learned my addr 172.21.15.131:0/2311552803 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:50.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa79bfff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 0x7fa79c0c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa79b7fe700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c0c9400 0x7fa79c156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a0f9c700 1 -- 172.21.15.131:0/2311552803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 msgr2=0x7fa79c0c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a0f9c700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 0x7fa79c0c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a0f9c700 1 -- 172.21.15.131:0/2311552803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c0c9400 msgr2=0x7fa79c156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a0f9c700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c0c9400 0x7fa79c156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.708+0000 7fa7a0f9c700 1 -- 172.21.15.131:0/2311552803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79400bcf0 con 0x7fa79c0cb320 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.709+0000 7fa7a0f9c700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0cb320 0x7fa79c16cba0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0688f0 tx=0x7fa79401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.709+0000 7fa783fff700 1 -- 172.21.15.131:0/2311552803 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79400ab90 con 0x7fa79c0cb320 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.709+0000 7fa783fff700 1 -- 172.21.15.131:0/2311552803 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79400ad30 con 0x7fa79c0cb320 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.709+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c15b040 con 0x7fa79c0cb320 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.709+0000 7fa783fff700 1 -- 172.21.15.131:0/2311552803 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79400aea0 con 0x7fa79c0cb320 2023-12-26T23:09:50.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.709+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c15b600 con 0x7fa79c0cb320 2023-12-26T23:09:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.710+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79c0adb00 con 0x7fa79c0cb320 2023-12-26T23:09:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.711+0000 7fa783fff700 1 -- 172.21.15.131:0/2311552803 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa794003040 con 0x7fa79c0cb320 2023-12-26T23:09:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.712+0000 7fa783fff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa788066f30 0x7fa7880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.712+0000 7fa783fff700 1 -- 172.21.15.131:0/2311552803 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fa7940863c0 con 0x7fa79c0cb320 2023-12-26T23:09:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.712+0000 7fa79bfff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa788066f30 0x7fa7880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.713+0000 7fa79bfff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa788066f30 0x7fa7880693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0c8480 tx=0x7fa78c009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:50.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.717+0000 7fa783fff700 1 -- 172.21.15.131:0/2311552803 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7940533a0 con 0x7fa79c0cb320 2023-12-26T23:09:50.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.906+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa79c0657e0 con 0x7fa788066f30 2023-12-26T23:09:50.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.916+0000 7fa783fff700 1 -- 172.21.15.131:0/2311552803 <== mgr.9456 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289630 (secure 0 0 0) 0x7fa79c0657e0 con 0x7fa788066f30 2023-12-26T23:09:50.921 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:50.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa788066f30 msgr2=0x7fa7880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa788066f30 0x7fa7880693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0c8480 tx=0x7fa78c009040 comp rx=0 tx=0).stop 2023-12-26T23:09:50.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0cb320 msgr2=0x7fa79c16cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:50.925 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0cb320 0x7fa79c16cba0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0688f0 tx=0x7fa79401a000 comp rx=0 tx=0).stop 2023-12-26T23:09:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 shutdown_connections 2023-12-26T23:09:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa788066f30 0x7fa7880693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79c0cb320 0x7fa79c16cba0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa79c0b0750 0x7fa79c0c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 --2- 172.21.15.131:0/2311552803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa79c0c9400 0x7fa79c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 >> 172.21.15.131:0/2311552803 conn(0x7fa79c068040 msgr2=0x7fa79c153c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 shutdown_connections 2023-12-26T23:09:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:50.921+0000 7fa7a29ff700 1 -- 172.21.15.131:0/2311552803 wait complete. 2023-12-26T23:09:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:09:50.939 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-26T23:09:49.110251+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2467884,"kb_used_data":19948,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752506836,"statfs":{"total":773094113280,"available":770567000064,"internally_reserved":0,"allocated":20426752,"data_stored":10813704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.551931"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607300+0000","last_change":"2023-12-26T23:09:46.604583+0000","last_active":"2023-12-26T23:09:46.607300+0000","last_peered":"2023-12-26T23:09:46.607300+0000","last_clean":"2023-12-26T23:09:46.607300+0000","last_became_active":"2023-12-26T23:09:46.604290+0000","last_became_peered":"2023-12-26T23:09:46.604290+0000","last_unstale":"2023-12-26T23:09:46.607300+0000","last_undegraded":"2023-12-26T23:09:46.607300+0000","last_fullsized":"2023-12-26T23:09:46.607300+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:28:42.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607140+0000","last_change":"2023-12-26T23:09:46.604724+0000","last_active":"2023-12-26T23:09:46.607140+0000","last_peered":"2023-12-26T23:09:46.607140+0000","last_clean":"2023-12-26T23:09:46.607140+0000","last_became_active":"2023-12-26T23:09:46.604419+0000","last_became_peered":"2023-12-26T23:09:46.604419+0000","last_unstale":"2023-12-26T23:09:46.607140+0000","last_undegraded":"2023-12-26T23:09:46.607140+0000","last_fullsized":"2023-12-26T23:09:46.607140+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:39:28.252771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606847+0000","last_change":"2023-12-26T23:09:46.601125+0000","last_active":"2023-12-26T23:09:46.606847+0000","last_peered":"2023-12-26T23:09:46.606847+0000","last_clean":"2023-12-26T23:09:46.606847+0000","last_became_active":"2023-12-26T23:09:46.600809+0000","last_became_peered":"2023-12-26T23:09:46.600809+0000","last_unstale":"2023-12-26T23:09:46.606847+0000","last_undegraded":"2023-12-26T23:09:46.606847+0000","last_fullsized":"2023-12-26T23:09:46.606847+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:06:10.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603561+0000","last_change":"2023-12-26T23:09:46.593583+0000","last_active":"2023-12-26T23:09:46.603561+0000","last_peered":"2023-12-26T23:09:46.603561+0000","last_clean":"2023-12-26T23:09:46.603561+0000","last_became_active":"2023-12-26T23:09:46.592826+0000","last_became_peered":"2023-12-26T23:09:46.592826+0000","last_unstale":"2023-12-26T23:09:46.603561+0000","last_undegraded":"2023-12-26T23:09:46.603561+0000","last_fullsized":"2023-12-26T23:09:46.603561+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:35.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.593116+0000","last_change":"2023-12-26T23:09:46.586477+0000","last_active":"2023-12-26T23:09:46.593116+0000","last_peered":"2023-12-26T23:09:46.593116+0000","last_clean":"2023-12-26T23:09:46.593116+0000","last_became_active":"2023-12-26T23:09:46.585043+0000","last_became_peered":"2023-12-26T23:09:46.585043+0000","last_unstale":"2023-12-26T23:09:46.593116+0000","last_undegraded":"2023-12-26T23:09:46.593116+0000","last_fullsized":"2023-12-26T23:09:46.593116+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:07:11.137583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602175+0000","last_change":"2023-12-26T23:09:46.598405+0000","last_active":"2023-12-26T23:09:46.602175+0000","last_peered":"2023-12-26T23:09:46.602175+0000","last_clean":"2023-12-26T23:09:46.602175+0000","last_became_active":"2023-12-26T23:09:46.598081+0000","last_became_peered":"2023-12-26T23:09:46.598081+0000","last_unstale":"2023-12-26T23:09:46.602175+0000","last_undegraded":"2023-12-26T23:09:46.602175+0000","last_fullsized":"2023-12-26T23:09:46.602175+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:32:54.019368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602944+0000","last_change":"2023-12-26T23:09:46.599347+0000","last_active":"2023-12-26T23:09:46.602944+0000","last_peered":"2023-12-26T23:09:46.602944+0000","last_clean":"2023-12-26T23:09:46.602944+0000","last_became_active":"2023-12-26T23:09:46.598821+0000","last_became_peered":"2023-12-26T23:09:46.598821+0000","last_unstale":"2023-12-26T23:09:46.602944+0000","last_undegraded":"2023-12-26T23:09:46.602944+0000","last_fullsized":"2023-12-26T23:09:46.602944+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:14:21.007369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.598028+0000","last_change":"2023-12-26T23:09:46.588940+0000","last_active":"2023-12-26T23:09:46.598028+0000","last_peered":"2023-12-26T23:09:46.598028+0000","last_clean":"2023-12-26T23:09:46.598028+0000","last_became_active":"2023-12-26T23:09:46.588660+0000","last_became_peered":"2023-12-26T23:09:46.588660+0000","last_unstale":"2023-12-26T23:09:46.598028+0000","last_undegraded":"2023-12-26T23:09:46.598028+0000","last_fullsized":"2023-12-26T23:09:46.598028+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:04:25.759214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606681+0000","last_change":"2023-12-26T23:09:46.599553+0000","last_active":"2023-12-26T23:09:46.606681+0000","last_peered":"2023-12-26T23:09:46.606681+0000","last_clean":"2023-12-26T23:09:46.606681+0000","last_became_active":"2023-12-26T23:09:46.599288+0000","last_became_peered":"2023-12-26T23:09:46.599288+0000","last_unstale":"2023-12-26T23:09:46.606681+0000","last_undegraded":"2023-12-26T23:09:46.606681+0000","last_fullsized":"2023-12-26T23:09:46.606681+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:30.266221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596988+0000","last_change":"2023-12-26T23:09:46.588421+0000","last_active":"2023-12-26T23:09:46.596988+0000","last_peered":"2023-12-26T23:09:46.596988+0000","last_clean":"2023-12-26T23:09:46.596988+0000","last_became_active":"2023-12-26T23:09:46.587928+0000","last_became_peered":"2023-12-26T23:09:46.587928+0000","last_unstale":"2023-12-26T23:09:46.596988+0000","last_undegraded":"2023-12-26T23:09:46.596988+0000","last_fullsized":"2023-12-26T23:09:46.596988+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:15:17.735808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.598803+0000","last_change":"2023-12-26T23:09:46.594067+0000","last_active":"2023-12-26T23:09:46.598803+0000","last_peered":"2023-12-26T23:09:46.598803+0000","last_clean":"2023-12-26T23:09:46.598803+0000","last_became_active":"2023-12-26T23:09:46.593176+0000","last_became_peered":"2023-12-26T23:09:46.593176+0000","last_unstale":"2023-12-26T23:09:46.598803+0000","last_undegraded":"2023-12-26T23:09:46.598803+0000","last_fullsized":"2023-12-26T23:09:46.598803+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:05.643738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.601107+0000","last_change":"2023-12-26T23:09:46.589699+0000","last_active":"2023-12-26T23:09:46.601107+0000","last_peered":"2023-12-26T23:09:46.601107+0000","last_clean":"2023-12-26T23:09:46.601107+0000","last_became_active":"2023-12-26T23:09:46.587539+0000","last_became_peered":"2023-12-26T23:09:46.587539+0000","last_unstale":"2023-12-26T23:09:46.601107+0000","last_undegraded":"2023-12-26T23:09:46.601107+0000","last_fullsized":"2023-12-26T23:09:46.601107+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:54:55.628676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602714+0000","last_change":"2023-12-26T23:09:46.593850+0000","last_active":"2023-12-26T23:09:46.602714+0000","last_peered":"2023-12-26T23:09:46.602714+0000","last_clean":"2023-12-26T23:09:46.602714+0000","last_became_active":"2023-12-26T23:09:46.592837+0000","last_became_peered":"2023-12-26T23:09:46.592837+0000","last_unstale":"2023-12-26T23:09:46.602714+0000","last_undegraded":"2023-12-26T23:09:46.602714+0000","last_fullsized":"2023-12-26T23:09:46.602714+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:31:57.686453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596693+0000","last_change":"2023-12-26T23:09:46.589078+0000","last_active":"2023-12-26T23:09:46.596693+0000","last_peered":"2023-12-26T23:09:46.596693+0000","last_clean":"2023-12-26T23:09:46.596693+0000","last_became_active":"2023-12-26T23:09:46.588822+0000","last_became_peered":"2023-12-26T23:09:46.588822+0000","last_unstale":"2023-12-26T23:09:46.596693+0000","last_undegraded":"2023-12-26T23:09:46.596693+0000","last_fullsized":"2023-12-26T23:09:46.596693+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:33:38.292911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.597377+0000","last_change":"2023-12-26T23:09:46.590757+0000","last_active":"2023-12-26T23:09:46.597377+0000","last_peered":"2023-12-26T23:09:46.597377+0000","last_clean":"2023-12-26T23:09:46.597377+0000","last_became_active":"2023-12-26T23:09:46.590454+0000","last_became_peered":"2023-12-26T23:09:46.590454+0000","last_unstale":"2023-12-26T23:09:46.597377+0000","last_undegraded":"2023-12-26T23:09:46.597377+0000","last_fullsized":"2023-12-26T23:09:46.597377+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:42.286153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596419+0000","last_change":"2023-12-26T23:09:46.593070+0000","last_active":"2023-12-26T23:09:46.596419+0000","last_peered":"2023-12-26T23:09:46.596419+0000","last_clean":"2023-12-26T23:09:46.596419+0000","last_became_active":"2023-12-26T23:09:46.591746+0000","last_became_peered":"2023-12-26T23:09:46.591746+0000","last_unstale":"2023-12-26T23:09:46.596419+0000","last_undegraded":"2023-12-26T23:09:46.596419+0000","last_fullsized":"2023-12-26T23:09:46.596419+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:33:06.116398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.587534+0000","last_change":"2023-12-26T23:09:46.581145+0000","last_active":"2023-12-26T23:09:46.587534+0000","last_peered":"2023-12-26T23:09:46.587534+0000","last_clean":"2023-12-26T23:09:46.587534+0000","last_became_active":"2023-12-26T23:09:46.580240+0000","last_became_peered":"2023-12-26T23:09:46.580240+0000","last_unstale":"2023-12-26T23:09:46.587534+0000","last_undegraded":"2023-12-26T23:09:46.587534+0000","last_fullsized":"2023-12-26T23:09:46.587534+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:11:39.652050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604136+0000","last_change":"2023-12-26T23:09:46.598701+0000","last_active":"2023-12-26T23:09:46.604136+0000","last_peered":"2023-12-26T23:09:46.604136+0000","last_clean":"2023-12-26T23:09:46.604136+0000","last_became_active":"2023-12-26T23:09:46.598385+0000","last_became_peered":"2023-12-26T23:09:46.598385+0000","last_unstale":"2023-12-26T23:09:46.604136+0000","last_undegraded":"2023-12-26T23:09:46.604136+0000","last_fullsized":"2023-12-26T23:09:46.604136+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:43.770033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603208+0000","last_change":"2023-12-26T23:09:46.597227+0000","last_active":"2023-12-26T23:09:46.603208+0000","last_peered":"2023-12-26T23:09:46.603208+0000","last_clean":"2023-12-26T23:09:46.603208+0000","last_became_active":"2023-12-26T23:09:46.596951+0000","last_became_peered":"2023-12-26T23:09:46.596951+0000","last_unstale":"2023-12-26T23:09:46.603208+0000","last_undegraded":"2023-12-26T23:09:46.603208+0000","last_fullsized":"2023-12-26T23:09:46.603208+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:55.348285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602628+0000","last_change":"2023-12-26T23:09:46.594826+0000","last_active":"2023-12-26T23:09:46.602628+0000","last_peered":"2023-12-26T23:09:46.602628+0000","last_clean":"2023-12-26T23:09:46.602628+0000","last_became_active":"2023-12-26T23:09:46.594579+0000","last_became_peered":"2023-12-26T23:09:46.594579+0000","last_unstale":"2023-12-26T23:09:46.602628+0000","last_undegraded":"2023-12-26T23:09:46.602628+0000","last_fullsized":"2023-12-26T23:09:46.602628+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:08:43.055929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.587099+0000","last_change":"2023-12-26T23:09:46.583496+0000","last_active":"2023-12-26T23:09:46.587099+0000","last_peered":"2023-12-26T23:09:46.587099+0000","last_clean":"2023-12-26T23:09:46.587099+0000","last_became_active":"2023-12-26T23:09:46.583232+0000","last_became_peered":"2023-12-26T23:09:46.583232+0000","last_unstale":"2023-12-26T23:09:46.587099+0000","last_undegraded":"2023-12-26T23:09:46.587099+0000","last_fullsized":"2023-12-26T23:09:46.587099+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:27:42.382215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600874+0000","last_change":"2023-12-26T23:09:46.589655+0000","last_active":"2023-12-26T23:09:46.600874+0000","last_peered":"2023-12-26T23:09:46.600874+0000","last_clean":"2023-12-26T23:09:46.600874+0000","last_became_active":"2023-12-26T23:09:46.589412+0000","last_became_peered":"2023-12-26T23:09:46.589412+0000","last_unstale":"2023-12-26T23:09:46.600874+0000","last_undegraded":"2023-12-26T23:09:46.600874+0000","last_fullsized":"2023-12-26T23:09:46.600874+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:59:57.577667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607004+0000","last_change":"2023-12-26T23:09:46.601505+0000","last_active":"2023-12-26T23:09:46.607004+0000","last_peered":"2023-12-26T23:09:46.607004+0000","last_clean":"2023-12-26T23:09:46.607004+0000","last_became_active":"2023-12-26T23:09:46.601205+0000","last_became_peered":"2023-12-26T23:09:46.601205+0000","last_unstale":"2023-12-26T23:09:46.607004+0000","last_undegraded":"2023-12-26T23:09:46.607004+0000","last_fullsized":"2023-12-26T23:09:46.607004+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:33:38.292911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.592394+0000","last_change":"2023-12-26T23:09:46.583704+0000","last_active":"2023-12-26T23:09:46.592394+0000","last_peered":"2023-12-26T23:09:46.592394+0000","last_clean":"2023-12-26T23:09:46.592394+0000","last_became_active":"2023-12-26T23:09:46.581921+0000","last_became_peered":"2023-12-26T23:09:46.581921+0000","last_unstale":"2023-12-26T23:09:46.592394+0000","last_undegraded":"2023-12-26T23:09:46.592394+0000","last_fullsized":"2023-12-26T23:09:46.592394+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:05:17.387272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603435+0000","last_change":"2023-12-26T23:09:46.600528+0000","last_active":"2023-12-26T23:09:46.603435+0000","last_peered":"2023-12-26T23:09:46.603435+0000","last_clean":"2023-12-26T23:09:46.603435+0000","last_became_active":"2023-12-26T23:09:46.598038+0000","last_became_peered":"2023-12-26T23:09:46.598038+0000","last_unstale":"2023-12-26T23:09:46.603435+0000","last_undegraded":"2023-12-26T23:09:46.603435+0000","last_fullsized":"2023-12-26T23:09:46.603435+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:46:38.005388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600294+0000","last_change":"2023-12-26T23:09:46.593840+0000","last_active":"2023-12-26T23:09:46.600294+0000","last_peered":"2023-12-26T23:09:46.600294+0000","last_clean":"2023-12-26T23:09:46.600294+0000","last_became_active":"2023-12-26T23:09:46.593054+0000","last_became_peered":"2023-12-26T23:09:46.593054+0000","last_unstale":"2023-12-26T23:09:46.600294+0000","last_undegraded":"2023-12-26T23:09:46.600294+0000","last_fullsized":"2023-12-26T23:09:46.600294+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:41:08.550325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.593960+0000","last_change":"2023-12-26T23:09:46.586639+0000","last_active":"2023-12-26T23:09:46.593960+0000","last_peered":"2023-12-26T23:09:46.593960+0000","last_clean":"2023-12-26T23:09:46.593960+0000","last_became_active":"2023-12-26T23:09:46.586369+0000","last_became_peered":"2023-12-26T23:09:46.586369+0000","last_unstale":"2023-12-26T23:09:46.593960+0000","last_undegraded":"2023-12-26T23:09:46.593960+0000","last_fullsized":"2023-12-26T23:09:46.593960+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:14:21.007369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600500+0000","last_change":"2023-12-26T23:09:46.594599+0000","last_active":"2023-12-26T23:09:46.600500+0000","last_peered":"2023-12-26T23:09:46.600500+0000","last_clean":"2023-12-26T23:09:46.600500+0000","last_became_active":"2023-12-26T23:09:46.594310+0000","last_became_peered":"2023-12-26T23:09:46.594310+0000","last_unstale":"2023-12-26T23:09:46.600500+0000","last_undegraded":"2023-12-26T23:09:46.600500+0000","last_fullsized":"2023-12-26T23:09:46.600500+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:18:06.593586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606832+0000","last_change":"2023-12-26T23:09:46.600088+0000","last_active":"2023-12-26T23:09:46.606832+0000","last_peered":"2023-12-26T23:09:46.606832+0000","last_clean":"2023-12-26T23:09:46.606832+0000","last_became_active":"2023-12-26T23:09:46.599658+0000","last_became_peered":"2023-12-26T23:09:46.599658+0000","last_unstale":"2023-12-26T23:09:46.606832+0000","last_undegraded":"2023-12-26T23:09:46.606832+0000","last_fullsized":"2023-12-26T23:09:46.606832+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:18:50.623844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596672+0000","last_change":"2023-12-26T23:09:46.585342+0000","last_active":"2023-12-26T23:09:46.596672+0000","last_peered":"2023-12-26T23:09:46.596672+0000","last_clean":"2023-12-26T23:09:46.596672+0000","last_became_active":"2023-12-26T23:09:46.585080+0000","last_became_peered":"2023-12-26T23:09:46.585080+0000","last_unstale":"2023-12-26T23:09:46.596672+0000","last_undegraded":"2023-12-26T23:09:46.596672+0000","last_fullsized":"2023-12-26T23:09:46.596672+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:41:18.206867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607013+0000","last_change":"2023-12-26T23:09:46.602417+0000","last_active":"2023-12-26T23:09:46.607013+0000","last_peered":"2023-12-26T23:09:46.607013+0000","last_clean":"2023-12-26T23:09:46.607013+0000","last_became_active":"2023-12-26T23:09:46.602165+0000","last_became_peered":"2023-12-26T23:09:46.602165+0000","last_unstale":"2023-12-26T23:09:46.607013+0000","last_undegraded":"2023-12-26T23:09:46.607013+0000","last_fullsized":"2023-12-26T23:09:46.607013+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:38:01.154494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603248+0000","last_change":"2023-12-26T23:09:46.599267+0000","last_active":"2023-12-26T23:09:46.603248+0000","last_peered":"2023-12-26T23:09:46.603248+0000","last_clean":"2023-12-26T23:09:46.603248+0000","last_became_active":"2023-12-26T23:09:46.599008+0000","last_became_peered":"2023-12-26T23:09:46.599008+0000","last_unstale":"2023-12-26T23:09:46.603248+0000","last_undegraded":"2023-12-26T23:09:46.603248+0000","last_fullsized":"2023-12-26T23:09:46.603248+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:58:36.562969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604161+0000","last_change":"2023-12-26T23:09:46.599826+0000","last_active":"2023-12-26T23:09:46.604161+0000","last_peered":"2023-12-26T23:09:46.604161+0000","last_clean":"2023-12-26T23:09:46.604161+0000","last_became_active":"2023-12-26T23:09:46.598055+0000","last_became_peered":"2023-12-26T23:09:46.598055+0000","last_unstale":"2023-12-26T23:09:46.604161+0000","last_undegraded":"2023-12-26T23:09:46.604161+0000","last_fullsized":"2023-12-26T23:09:46.604161+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:21:26.253582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"192'4","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607536+0000","last_change":"2023-12-26T23:09:45.589924+0000","last_active":"2023-12-26T23:09:46.607536+0000","last_peered":"2023-12-26T23:09:46.607536+0000","last_clean":"2023-12-26T23:09:46.607536+0000","last_became_active":"2023-12-26T23:09:45.589660+0000","last_became_peered":"2023-12-26T23:09:45.589660+0000","last_unstale":"2023-12-26T23:09:46.607536+0000","last_undegraded":"2023-12-26T23:09:46.607536+0000","last_fullsized":"2023-12-26T23:09:46.607536+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:04:27.424684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599125+0000","last_change":"2023-12-26T23:09:46.595424+0000","last_active":"2023-12-26T23:09:46.599125+0000","last_peered":"2023-12-26T23:09:46.599125+0000","last_clean":"2023-12-26T23:09:46.599125+0000","last_became_active":"2023-12-26T23:09:46.595133+0000","last_became_peered":"2023-12-26T23:09:46.595133+0000","last_unstale":"2023-12-26T23:09:46.599125+0000","last_undegraded":"2023-12-26T23:09:46.599125+0000","last_fullsized":"2023-12-26T23:09:46.599125+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:47:13.668855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.588539+0000","last_change":"2023-12-26T23:09:45.582327+0000","last_active":"2023-12-26T23:09:46.588539+0000","last_peered":"2023-12-26T23:09:46.588539+0000","last_clean":"2023-12-26T23:09:46.588539+0000","last_became_active":"2023-12-26T23:09:45.582075+0000","last_became_peered":"2023-12-26T23:09:45.582075+0000","last_unstale":"2023-12-26T23:09:46.588539+0000","last_undegraded":"2023-12-26T23:09:46.588539+0000","last_fullsized":"2023-12-26T23:09:46.588539+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:43:03.184792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602984+0000","last_change":"2023-12-26T23:09:46.595260+0000","last_active":"2023-12-26T23:09:46.602984+0000","last_peered":"2023-12-26T23:09:46.602984+0000","last_clean":"2023-12-26T23:09:46.602984+0000","last_became_active":"2023-12-26T23:09:46.594948+0000","last_became_peered":"2023-12-26T23:09:46.594948+0000","last_unstale":"2023-12-26T23:09:46.602984+0000","last_undegraded":"2023-12-26T23:09:46.602984+0000","last_fullsized":"2023-12-26T23:09:46.602984+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:32:55.217157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.590140+0000","last_change":"2023-12-26T23:09:45.575120+0000","last_active":"2023-12-26T23:09:46.590140+0000","last_peered":"2023-12-26T23:09:46.590140+0000","last_clean":"2023-12-26T23:09:46.590140+0000","last_became_active":"2023-12-26T23:09:45.574842+0000","last_became_peered":"2023-12-26T23:09:45.574842+0000","last_unstale":"2023-12-26T23:09:46.590140+0000","last_undegraded":"2023-12-26T23:09:46.590140+0000","last_fullsized":"2023-12-26T23:09:46.590140+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:27:11.943098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602906+0000","last_change":"2023-12-26T23:09:46.598110+0000","last_active":"2023-12-26T23:09:46.602906+0000","last_peered":"2023-12-26T23:09:46.602906+0000","last_clean":"2023-12-26T23:09:46.602906+0000","last_became_active":"2023-12-26T23:09:46.597849+0000","last_became_peered":"2023-12-26T23:09:46.597849+0000","last_unstale":"2023-12-26T23:09:46.602906+0000","last_undegraded":"2023-12-26T23:09:46.602906+0000","last_fullsized":"2023-12-26T23:09:46.602906+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:35:21.934564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599731+0000","last_change":"2023-12-26T23:09:45.586785+0000","last_active":"2023-12-26T23:09:46.599731+0000","last_peered":"2023-12-26T23:09:46.599731+0000","last_clean":"2023-12-26T23:09:46.599731+0000","last_became_active":"2023-12-26T23:09:45.586349+0000","last_became_peered":"2023-12-26T23:09:45.586349+0000","last_unstale":"2023-12-26T23:09:46.599731+0000","last_undegraded":"2023-12-26T23:09:46.599731+0000","last_fullsized":"2023-12-26T23:09:46.599731+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:59:17.893604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603880+0000","last_change":"2023-12-26T23:09:46.593732+0000","last_active":"2023-12-26T23:09:46.603880+0000","last_peered":"2023-12-26T23:09:46.603880+0000","last_clean":"2023-12-26T23:09:46.603880+0000","last_became_active":"2023-12-26T23:09:46.593471+0000","last_became_peered":"2023-12-26T23:09:46.593471+0000","last_unstale":"2023-12-26T23:09:46.603880+0000","last_undegraded":"2023-12-26T23:09:46.603880+0000","last_fullsized":"2023-12-26T23:09:46.603880+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:18:03.529861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604309+0000","last_change":"2023-12-26T23:09:45.589451+0000","last_active":"2023-12-26T23:09:46.604309+0000","last_peered":"2023-12-26T23:09:46.604309+0000","last_clean":"2023-12-26T23:09:46.604309+0000","last_became_active":"2023-12-26T23:09:45.589180+0000","last_became_peered":"2023-12-26T23:09:45.589180+0000","last_unstale":"2023-12-26T23:09:46.604309+0000","last_undegraded":"2023-12-26T23:09:46.604309+0000","last_fullsized":"2023-12-26T23:09:46.604309+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:40:50.286051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603764+0000","last_change":"2023-12-26T23:09:46.596499+0000","last_active":"2023-12-26T23:09:46.603764+0000","last_peered":"2023-12-26T23:09:46.603764+0000","last_clean":"2023-12-26T23:09:46.603764+0000","last_became_active":"2023-12-26T23:09:46.596239+0000","last_became_peered":"2023-12-26T23:09:46.596239+0000","last_unstale":"2023-12-26T23:09:46.603764+0000","last_undegraded":"2023-12-26T23:09:46.603764+0000","last_fullsized":"2023-12-26T23:09:46.603764+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:09.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.601031+0000","last_change":"2023-12-26T23:09:45.586765+0000","last_active":"2023-12-26T23:09:46.601031+0000","last_peered":"2023-12-26T23:09:46.601031+0000","last_clean":"2023-12-26T23:09:46.601031+0000","last_became_active":"2023-12-26T23:09:45.586468+0000","last_became_peered":"2023-12-26T23:09:45.586468+0000","last_unstale":"2023-12-26T23:09:46.601031+0000","last_undegraded":"2023-12-26T23:09:46.601031+0000","last_fullsized":"2023-12-26T23:09:46.601031+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:51:08.741674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603864+0000","last_change":"2023-12-26T23:09:46.597661+0000","last_active":"2023-12-26T23:09:46.603864+0000","last_peered":"2023-12-26T23:09:46.603864+0000","last_clean":"2023-12-26T23:09:46.603864+0000","last_became_active":"2023-12-26T23:09:46.597380+0000","last_became_peered":"2023-12-26T23:09:46.597380+0000","last_unstale":"2023-12-26T23:09:46.603864+0000","last_undegraded":"2023-12-26T23:09:46.603864+0000","last_fullsized":"2023-12-26T23:09:46.603864+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:42.286153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.594120+0000","last_change":"2023-12-26T23:09:45.590794+0000","last_active":"2023-12-26T23:09:46.594120+0000","last_peered":"2023-12-26T23:09:46.594120+0000","last_clean":"2023-12-26T23:09:46.594120+0000","last_became_active":"2023-12-26T23:09:45.590380+0000","last_became_peered":"2023-12-26T23:09:45.590380+0000","last_unstale":"2023-12-26T23:09:46.594120+0000","last_undegraded":"2023-12-26T23:09:46.594120+0000","last_fullsized":"2023-12-26T23:09:46.594120+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:15:46.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.601012+0000","last_change":"2023-12-26T23:09:46.593072+0000","last_active":"2023-12-26T23:09:46.601012+0000","last_peered":"2023-12-26T23:09:46.601012+0000","last_clean":"2023-12-26T23:09:46.601012+0000","last_became_active":"2023-12-26T23:09:46.592763+0000","last_became_peered":"2023-12-26T23:09:46.592763+0000","last_unstale":"2023-12-26T23:09:46.601012+0000","last_undegraded":"2023-12-26T23:09:46.601012+0000","last_fullsized":"2023-12-26T23:09:46.601012+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:29:00.694839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.582093+0000","last_change":"2023-12-26T23:09:45.582292+0000","last_active":"2023-12-26T23:09:46.582093+0000","last_peered":"2023-12-26T23:09:46.582093+0000","last_clean":"2023-12-26T23:09:46.582093+0000","last_became_active":"2023-12-26T23:09:45.582031+0000","last_became_peered":"2023-12-26T23:09:45.582031+0000","last_unstale":"2023-12-26T23:09:46.582093+0000","last_undegraded":"2023-12-26T23:09:46.582093+0000","last_fullsized":"2023-12-26T23:09:46.582093+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:58:06.657883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.591506+0000","last_change":"2023-12-26T23:09:46.586759+0000","last_active":"2023-12-26T23:09:46.591506+0000","last_peered":"2023-12-26T23:09:46.591506+0000","last_clean":"2023-12-26T23:09:46.591506+0000","last_became_active":"2023-12-26T23:09:46.586480+0000","last_became_peered":"2023-12-26T23:09:46.586480+0000","last_unstale":"2023-12-26T23:09:46.591506+0000","last_undegraded":"2023-12-26T23:09:46.591506+0000","last_fullsized":"2023-12-26T23:09:46.591506+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:05.643738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.570073+0000","last_change":"2023-12-26T23:09:45.585069+0000","last_active":"2023-12-26T23:09:46.570073+0000","last_peered":"2023-12-26T23:09:46.570073+0000","last_clean":"2023-12-26T23:09:46.570073+0000","last_became_active":"2023-12-26T23:09:45.583718+0000","last_became_peered":"2023-12-26T23:09:45.583718+0000","last_unstale":"2023-12-26T23:09:46.570073+0000","last_undegraded":"2023-12-26T23:09:46.570073+0000","last_fullsized":"2023-12-26T23:09:46.570073+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:02:31.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.595522+0000","last_change":"2023-12-26T23:09:46.591348+0000","last_active":"2023-12-26T23:09:46.595522+0000","last_peered":"2023-12-26T23:09:46.595522+0000","last_clean":"2023-12-26T23:09:46.595522+0000","last_became_active":"2023-12-26T23:09:46.591002+0000","last_became_peered":"2023-12-26T23:09:46.591002+0000","last_unstale":"2023-12-26T23:09:46.595522+0000","last_undegraded":"2023-12-26T23:09:46.595522+0000","last_fullsized":"2023-12-26T23:09:46.595522+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:59:57.577667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"192'3","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602421+0000","last_change":"2023-12-26T23:09:45.579845+0000","last_active":"2023-12-26T23:09:46.602421+0000","last_peered":"2023-12-26T23:09:46.602421+0000","last_clean":"2023-12-26T23:09:46.602421+0000","last_became_active":"2023-12-26T23:09:45.579485+0000","last_became_peered":"2023-12-26T23:09:45.579485+0000","last_unstale":"2023-12-26T23:09:46.602421+0000","last_undegraded":"2023-12-26T23:09:46.602421+0000","last_fullsized":"2023-12-26T23:09:46.602421+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:56:08.086287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606826+0000","last_change":"2023-12-26T23:09:46.601064+0000","last_active":"2023-12-26T23:09:46.606826+0000","last_peered":"2023-12-26T23:09:46.606826+0000","last_clean":"2023-12-26T23:09:46.606826+0000","last_became_active":"2023-12-26T23:09:46.600489+0000","last_became_peered":"2023-12-26T23:09:46.600489+0000","last_unstale":"2023-12-26T23:09:46.606826+0000","last_undegraded":"2023-12-26T23:09:46.606826+0000","last_fullsized":"2023-12-26T23:09:46.606826+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:52:12.397718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"192'6","reported_seq":16,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607610+0000","last_change":"2023-12-26T23:09:45.576823+0000","last_active":"2023-12-26T23:09:46.607610+0000","last_peered":"2023-12-26T23:09:46.607610+0000","last_clean":"2023-12-26T23:09:46.607610+0000","last_became_active":"2023-12-26T23:09:45.576486+0000","last_became_peered":"2023-12-26T23:09:45.576486+0000","last_unstale":"2023-12-26T23:09:46.607610+0000","last_undegraded":"2023-12-26T23:09:46.607610+0000","last_fullsized":"2023-12-26T23:09:46.607610+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:25:59.254836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.595388+0000","last_change":"2023-12-26T23:09:46.589966+0000","last_active":"2023-12-26T23:09:46.595388+0000","last_peered":"2023-12-26T23:09:46.595388+0000","last_clean":"2023-12-26T23:09:46.595388+0000","last_became_active":"2023-12-26T23:09:46.589697+0000","last_became_peered":"2023-12-26T23:09:46.589697+0000","last_unstale":"2023-12-26T23:09:46.595388+0000","last_undegraded":"2023-12-26T23:09:46.595388+0000","last_fullsized":"2023-12-26T23:09:46.595388+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:17:12.820879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599056+0000","last_change":"2023-12-26T23:09:45.583454+0000","last_active":"2023-12-26T23:09:46.599056+0000","last_peered":"2023-12-26T23:09:46.599056+0000","last_clean":"2023-12-26T23:09:46.599056+0000","last_became_active":"2023-12-26T23:09:45.583195+0000","last_became_peered":"2023-12-26T23:09:45.583195+0000","last_unstale":"2023-12-26T23:09:46.599056+0000","last_undegraded":"2023-12-26T23:09:46.599056+0000","last_fullsized":"2023-12-26T23:09:46.599056+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:46:17.437641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606870+0000","last_change":"2023-12-26T23:09:46.600684+0000","last_active":"2023-12-26T23:09:46.606870+0000","last_peered":"2023-12-26T23:09:46.606870+0000","last_clean":"2023-12-26T23:09:46.606870+0000","last_became_active":"2023-12-26T23:09:46.600238+0000","last_became_peered":"2023-12-26T23:09:46.600238+0000","last_unstale":"2023-12-26T23:09:46.606870+0000","last_undegraded":"2023-12-26T23:09:46.606870+0000","last_fullsized":"2023-12-26T23:09:46.606870+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:04:29.108988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.570516+0000","last_change":"2023-12-26T23:09:45.580860+0000","last_active":"2023-12-26T23:09:46.570516+0000","last_peered":"2023-12-26T23:09:46.570516+0000","last_clean":"2023-12-26T23:09:46.570516+0000","last_became_active":"2023-12-26T23:09:45.579589+0000","last_became_peered":"2023-12-26T23:09:45.579589+0000","last_unstale":"2023-12-26T23:09:46.570516+0000","last_undegraded":"2023-12-26T23:09:46.570516+0000","last_fullsized":"2023-12-26T23:09:46.570516+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:10.474047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.586974+0000","last_change":"2023-12-26T23:09:46.582547+0000","last_active":"2023-12-26T23:09:46.586974+0000","last_peered":"2023-12-26T23:09:46.586974+0000","last_clean":"2023-12-26T23:09:46.586974+0000","last_became_active":"2023-12-26T23:09:46.581757+0000","last_became_peered":"2023-12-26T23:09:46.581757+0000","last_unstale":"2023-12-26T23:09:46.586974+0000","last_undegraded":"2023-12-26T23:09:46.586974+0000","last_fullsized":"2023-12-26T23:09:46.586974+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:42:01.311312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.582427+0000","last_change":"2023-12-26T23:09:45.588483+0000","last_active":"2023-12-26T23:09:46.582427+0000","last_peered":"2023-12-26T23:09:46.582427+0000","last_clean":"2023-12-26T23:09:46.582427+0000","last_became_active":"2023-12-26T23:09:45.586903+0000","last_became_peered":"2023-12-26T23:09:45.586903+0000","last_unstale":"2023-12-26T23:09:46.582427+0000","last_undegraded":"2023-12-26T23:09:46.582427+0000","last_fullsized":"2023-12-26T23:09:46.582427+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:39.956735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.597360+0000","last_change":"2023-12-26T23:09:46.587107+0000","last_active":"2023-12-26T23:09:46.597360+0000","last_peered":"2023-12-26T23:09:46.597360+0000","last_clean":"2023-12-26T23:09:46.597360+0000","last_became_active":"2023-12-26T23:09:46.586868+0000","last_became_peered":"2023-12-26T23:09:46.586868+0000","last_unstale":"2023-12-26T23:09:46.597360+0000","last_undegraded":"2023-12-26T23:09:46.597360+0000","last_fullsized":"2023-12-26T23:09:46.597360+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:32:47.514453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"192'8","reported_seq":18,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599708+0000","last_change":"2023-12-26T23:09:45.578748+0000","last_active":"2023-12-26T23:09:46.599708+0000","last_peered":"2023-12-26T23:09:46.599708+0000","last_clean":"2023-12-26T23:09:46.599708+0000","last_became_active":"2023-12-26T23:09:45.578489+0000","last_became_peered":"2023-12-26T23:09:45.578489+0000","last_unstale":"2023-12-26T23:09:46.599708+0000","last_undegraded":"2023-12-26T23:09:46.599708+0000","last_fullsized":"2023-12-26T23:09:46.599708+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:18:14.831411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.594256+0000","last_change":"2023-12-26T23:09:46.587699+0000","last_active":"2023-12-26T23:09:46.594256+0000","last_peered":"2023-12-26T23:09:46.594256+0000","last_clean":"2023-12-26T23:09:46.594256+0000","last_became_active":"2023-12-26T23:09:46.587415+0000","last_became_peered":"2023-12-26T23:09:46.587415+0000","last_unstale":"2023-12-26T23:09:46.594256+0000","last_undegraded":"2023-12-26T23:09:46.594256+0000","last_fullsized":"2023-12-26T23:09:46.594256+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:46:38.005388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.590103+0000","last_change":"2023-12-26T23:09:45.578820+0000","last_active":"2023-12-26T23:09:46.590103+0000","last_peered":"2023-12-26T23:09:46.590103+0000","last_clean":"2023-12-26T23:09:46.590103+0000","last_became_active":"2023-12-26T23:09:45.578559+0000","last_became_peered":"2023-12-26T23:09:45.578559+0000","last_unstale":"2023-12-26T23:09:46.590103+0000","last_undegraded":"2023-12-26T23:09:46.590103+0000","last_fullsized":"2023-12-26T23:09:46.590103+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:22:50.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604769+0000","last_change":"2023-12-26T23:09:46.602463+0000","last_active":"2023-12-26T23:09:46.604769+0000","last_peered":"2023-12-26T23:09:46.604769+0000","last_clean":"2023-12-26T23:09:46.604769+0000","last_became_active":"2023-12-26T23:09:46.601615+0000","last_became_peered":"2023-12-26T23:09:46.601615+0000","last_unstale":"2023-12-26T23:09:46.604769+0000","last_undegraded":"2023-12-26T23:09:46.604769+0000","last_fullsized":"2023-12-26T23:09:46.604769+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:22.512723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.588280+0000","last_change":"2023-12-26T23:09:45.586961+0000","last_active":"2023-12-26T23:09:46.588280+0000","last_peered":"2023-12-26T23:09:46.588280+0000","last_clean":"2023-12-26T23:09:46.588280+0000","last_became_active":"2023-12-26T23:09:45.586607+0000","last_became_peered":"2023-12-26T23:09:45.586607+0000","last_unstale":"2023-12-26T23:09:46.588280+0000","last_undegraded":"2023-12-26T23:09:46.588280+0000","last_fullsized":"2023-12-26T23:09:46.588280+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:04:56.455080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603829+0000","last_change":"2023-12-26T23:09:46.596716+0000","last_active":"2023-12-26T23:09:46.603829+0000","last_peered":"2023-12-26T23:09:46.603829+0000","last_clean":"2023-12-26T23:09:46.603829+0000","last_became_active":"2023-12-26T23:09:46.595878+0000","last_became_peered":"2023-12-26T23:09:46.595878+0000","last_unstale":"2023-12-26T23:09:46.603829+0000","last_undegraded":"2023-12-26T23:09:46.603829+0000","last_fullsized":"2023-12-26T23:09:46.603829+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:25:56.200695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.598162+0000","last_change":"2023-12-26T23:09:45.588685+0000","last_active":"2023-12-26T23:09:46.598162+0000","last_peered":"2023-12-26T23:09:46.598162+0000","last_clean":"2023-12-26T23:09:46.598162+0000","last_became_active":"2023-12-26T23:09:45.587100+0000","last_became_peered":"2023-12-26T23:09:45.587100+0000","last_unstale":"2023-12-26T23:09:46.598162+0000","last_undegraded":"2023-12-26T23:09:46.598162+0000","last_fullsized":"2023-12-26T23:09:46.598162+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:05:08.591870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607432+0000","last_change":"2023-12-26T23:09:46.604089+0000","last_active":"2023-12-26T23:09:46.607432+0000","last_peered":"2023-12-26T23:09:46.607432+0000","last_clean":"2023-12-26T23:09:46.607432+0000","last_became_active":"2023-12-26T23:09:46.601440+0000","last_became_peered":"2023-12-26T23:09:46.601440+0000","last_unstale":"2023-12-26T23:09:46.607432+0000","last_undegraded":"2023-12-26T23:09:46.607432+0000","last_fullsized":"2023-12-26T23:09:46.607432+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:22:44.998972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.576998+0000","last_change":"2023-12-26T23:09:45.585513+0000","last_active":"2023-12-26T23:09:46.576998+0000","last_peered":"2023-12-26T23:09:46.576998+0000","last_clean":"2023-12-26T23:09:46.576998+0000","last_became_active":"2023-12-26T23:09:45.584696+0000","last_became_peered":"2023-12-26T23:09:45.584696+0000","last_unstale":"2023-12-26T23:09:46.576998+0000","last_undegraded":"2023-12-26T23:09:46.576998+0000","last_fullsized":"2023-12-26T23:09:46.576998+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:52:28.177857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607095+0000","last_change":"2023-12-26T23:09:46.603095+0000","last_active":"2023-12-26T23:09:46.607095+0000","last_peered":"2023-12-26T23:09:46.607095+0000","last_clean":"2023-12-26T23:09:46.607095+0000","last_became_active":"2023-12-26T23:09:46.602847+0000","last_became_peered":"2023-12-26T23:09:46.602847+0000","last_unstale":"2023-12-26T23:09:46.607095+0000","last_undegraded":"2023-12-26T23:09:46.607095+0000","last_fullsized":"2023-12-26T23:09:46.607095+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:40:31.701871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603284+0000","last_change":"2023-12-26T23:09:45.585675+0000","last_active":"2023-12-26T23:09:46.603284+0000","last_peered":"2023-12-26T23:09:46.603284+0000","last_clean":"2023-12-26T23:09:46.603284+0000","last_became_active":"2023-12-26T23:09:45.585461+0000","last_became_peered":"2023-12-26T23:09:45.585461+0000","last_unstale":"2023-12-26T23:09:46.603284+0000","last_undegraded":"2023-12-26T23:09:46.603284+0000","last_fullsized":"2023-12-26T23:09:46.603284+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:51:59.410791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607420+0000","last_change":"2023-12-26T23:09:46.603900+0000","last_active":"2023-12-26T23:09:46.607420+0000","last_peered":"2023-12-26T23:09:46.607420+0000","last_clean":"2023-12-26T23:09:46.607420+0000","last_became_active":"2023-12-26T23:09:46.603348+0000","last_became_peered":"2023-12-26T23:09:46.603348+0000","last_unstale":"2023-12-26T23:09:46.607420+0000","last_undegraded":"2023-12-26T23:09:46.607420+0000","last_fullsized":"2023-12-26T23:09:46.607420+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:18:28.979877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.576612+0000","last_change":"2023-12-26T23:09:45.574162+0000","last_active":"2023-12-26T23:09:46.576612+0000","last_peered":"2023-12-26T23:09:46.576612+0000","last_clean":"2023-12-26T23:09:46.576612+0000","last_became_active":"2023-12-26T23:09:45.573894+0000","last_became_peered":"2023-12-26T23:09:45.573894+0000","last_unstale":"2023-12-26T23:09:46.576612+0000","last_undegraded":"2023-12-26T23:09:46.576612+0000","last_fullsized":"2023-12-26T23:09:46.576612+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:26.471388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606898+0000","last_change":"2023-12-26T23:09:46.601902+0000","last_active":"2023-12-26T23:09:46.606898+0000","last_peered":"2023-12-26T23:09:46.606898+0000","last_clean":"2023-12-26T23:09:46.606898+0000","last_became_active":"2023-12-26T23:09:46.601540+0000","last_became_peered":"2023-12-26T23:09:46.601540+0000","last_unstale":"2023-12-26T23:09:46.606898+0000","last_undegraded":"2023-12-26T23:09:46.606898+0000","last_fullsized":"2023-12-26T23:09:46.606898+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:47.888770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.578037+0000","last_change":"2023-12-26T23:09:45.589363+0000","last_active":"2023-12-26T23:09:46.578037+0000","last_peered":"2023-12-26T23:09:46.578037+0000","last_clean":"2023-12-26T23:09:46.578037+0000","last_became_active":"2023-12-26T23:09:45.589107+0000","last_became_peered":"2023-12-26T23:09:45.589107+0000","last_unstale":"2023-12-26T23:09:46.578037+0000","last_undegraded":"2023-12-26T23:09:46.578037+0000","last_fullsized":"2023-12-26T23:09:46.578037+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:57.785145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600165+0000","last_change":"2023-12-26T23:09:46.594497+0000","last_active":"2023-12-26T23:09:46.600165+0000","last_peered":"2023-12-26T23:09:46.600165+0000","last_clean":"2023-12-26T23:09:46.600165+0000","last_became_active":"2023-12-26T23:09:46.594179+0000","last_became_peered":"2023-12-26T23:09:46.594179+0000","last_unstale":"2023-12-26T23:09:46.600165+0000","last_undegraded":"2023-12-26T23:09:46.600165+0000","last_fullsized":"2023-12-26T23:09:46.600165+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:05:17.387272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.576832+0000","last_change":"2023-12-26T23:09:45.584699+0000","last_active":"2023-12-26T23:09:46.576832+0000","last_peered":"2023-12-26T23:09:46.576832+0000","last_clean":"2023-12-26T23:09:46.576832+0000","last_became_active":"2023-12-26T23:09:45.584460+0000","last_became_peered":"2023-12-26T23:09:45.584460+0000","last_unstale":"2023-12-26T23:09:46.576832+0000","last_undegraded":"2023-12-26T23:09:46.576832+0000","last_fullsized":"2023-12-26T23:09:46.576832+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:29:54.385431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.597131+0000","last_change":"2023-12-26T23:09:46.592203+0000","last_active":"2023-12-26T23:09:46.597131+0000","last_peered":"2023-12-26T23:09:46.597131+0000","last_clean":"2023-12-26T23:09:46.597131+0000","last_became_active":"2023-12-26T23:09:46.590871+0000","last_became_peered":"2023-12-26T23:09:46.590871+0000","last_unstale":"2023-12-26T23:09:46.597131+0000","last_undegraded":"2023-12-26T23:09:46.597131+0000","last_fullsized":"2023-12-26T23:09:46.597131+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:41:48.083462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596066+0000","last_change":"2023-12-26T23:09:45.577660+0000","last_active":"2023-12-26T23:09:46.596066+0000","last_peered":"2023-12-26T23:09:46.596066+0000","last_clean":"2023-12-26T23:09:46.596066+0000","last_became_active":"2023-12-26T23:09:45.577235+0000","last_became_peered":"2023-12-26T23:09:45.577235+0000","last_unstale":"2023-12-26T23:09:46.596066+0000","last_undegraded":"2023-12-26T23:09:46.596066+0000","last_fullsized":"2023-12-26T23:09:46.596066+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:27:36.878163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604790+0000","last_change":"2023-12-26T23:09:46.603013+0000","last_active":"2023-12-26T23:09:46.604790+0000","last_peered":"2023-12-26T23:09:46.604790+0000","last_clean":"2023-12-26T23:09:46.604790+0000","last_became_active":"2023-12-26T23:09:46.602749+0000","last_became_peered":"2023-12-26T23:09:46.602749+0000","last_unstale":"2023-12-26T23:09:46.604790+0000","last_undegraded":"2023-12-26T23:09:46.604790+0000","last_fullsized":"2023-12-26T23:09:46.604790+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:27:42.382215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.577040+0000","last_change":"2023-12-26T23:09:45.590876+0000","last_active":"2023-12-26T23:09:46.577040+0000","last_peered":"2023-12-26T23:09:46.577040+0000","last_clean":"2023-12-26T23:09:46.577040+0000","last_became_active":"2023-12-26T23:09:45.590610+0000","last_became_peered":"2023-12-26T23:09:45.590610+0000","last_unstale":"2023-12-26T23:09:46.577040+0000","last_undegraded":"2023-12-26T23:09:46.577040+0000","last_fullsized":"2023-12-26T23:09:46.577040+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:17.072154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603727+0000","last_change":"2023-12-26T23:09:46.596171+0000","last_active":"2023-12-26T23:09:46.603727+0000","last_peered":"2023-12-26T23:09:46.603727+0000","last_clean":"2023-12-26T23:09:46.603727+0000","last_became_active":"2023-12-26T23:09:46.593136+0000","last_became_peered":"2023-12-26T23:09:46.593136+0000","last_unstale":"2023-12-26T23:09:46.603727+0000","last_undegraded":"2023-12-26T23:09:46.603727+0000","last_fullsized":"2023-12-26T23:09:46.603727+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:32:55.217157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.595547+0000","last_change":"2023-12-26T23:09:45.589622+0000","last_active":"2023-12-26T23:09:46.595547+0000","last_peered":"2023-12-26T23:09:46.595547+0000","last_clean":"2023-12-26T23:09:46.595547+0000","last_became_active":"2023-12-26T23:09:45.589332+0000","last_became_peered":"2023-12-26T23:09:45.589332+0000","last_unstale":"2023-12-26T23:09:46.595547+0000","last_undegraded":"2023-12-26T23:09:46.595547+0000","last_fullsized":"2023-12-26T23:09:46.595547+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:17.394434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600980+0000","last_change":"2023-12-26T23:09:46.592752+0000","last_active":"2023-12-26T23:09:46.600980+0000","last_peered":"2023-12-26T23:09:46.600980+0000","last_clean":"2023-12-26T23:09:46.600980+0000","last_became_active":"2023-12-26T23:09:46.592308+0000","last_became_peered":"2023-12-26T23:09:46.592308+0000","last_unstale":"2023-12-26T23:09:46.600980+0000","last_undegraded":"2023-12-26T23:09:46.600980+0000","last_fullsized":"2023-12-26T23:09:46.600980+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:34:04.189487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.584407+0000","last_change":"2023-12-26T23:09:45.586964+0000","last_active":"2023-12-26T23:09:46.584407+0000","last_peered":"2023-12-26T23:09:46.584407+0000","last_clean":"2023-12-26T23:09:46.584407+0000","last_became_active":"2023-12-26T23:09:45.586065+0000","last_became_peered":"2023-12-26T23:09:45.586065+0000","last_unstale":"2023-12-26T23:09:46.584407+0000","last_undegraded":"2023-12-26T23:09:46.584407+0000","last_fullsized":"2023-12-26T23:09:46.584407+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:13:26.740447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602197+0000","last_change":"2023-12-26T23:09:46.599444+0000","last_active":"2023-12-26T23:09:46.602197+0000","last_peered":"2023-12-26T23:09:46.602197+0000","last_clean":"2023-12-26T23:09:46.602197+0000","last_became_active":"2023-12-26T23:09:46.598681+0000","last_became_peered":"2023-12-26T23:09:46.598681+0000","last_unstale":"2023-12-26T23:09:46.602197+0000","last_undegraded":"2023-12-26T23:09:46.602197+0000","last_fullsized":"2023-12-26T23:09:46.602197+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:59:20.100619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.574706+0000","last_change":"2023-12-26T23:09:45.586115+0000","last_active":"2023-12-26T23:09:46.574706+0000","last_peered":"2023-12-26T23:09:46.574706+0000","last_clean":"2023-12-26T23:09:46.574706+0000","last_became_active":"2023-12-26T23:09:45.585883+0000","last_became_peered":"2023-12-26T23:09:45.585883+0000","last_unstale":"2023-12-26T23:09:46.574706+0000","last_undegraded":"2023-12-26T23:09:46.574706+0000","last_fullsized":"2023-12-26T23:09:46.574706+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:59:03.590907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606706+0000","last_change":"2023-12-26T23:09:46.599927+0000","last_active":"2023-12-26T23:09:46.606706+0000","last_peered":"2023-12-26T23:09:46.606706+0000","last_clean":"2023-12-26T23:09:46.606706+0000","last_became_active":"2023-12-26T23:09:46.599339+0000","last_became_peered":"2023-12-26T23:09:46.599339+0000","last_unstale":"2023-12-26T23:09:46.606706+0000","last_undegraded":"2023-12-26T23:09:46.606706+0000","last_fullsized":"2023-12-26T23:09:46.606706+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:15:09.620999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.582592+0000","last_change":"2023-12-26T23:09:45.588345+0000","last_active":"2023-12-26T23:09:46.582592+0000","last_peered":"2023-12-26T23:09:46.582592+0000","last_clean":"2023-12-26T23:09:46.582592+0000","last_became_active":"2023-12-26T23:09:45.587978+0000","last_became_peered":"2023-12-26T23:09:45.587978+0000","last_unstale":"2023-12-26T23:09:46.582592+0000","last_undegraded":"2023-12-26T23:09:46.582592+0000","last_fullsized":"2023-12-26T23:09:46.582592+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:20:17.363109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603037+0000","last_change":"2023-12-26T23:09:46.599500+0000","last_active":"2023-12-26T23:09:46.603037+0000","last_peered":"2023-12-26T23:09:46.603037+0000","last_clean":"2023-12-26T23:09:46.603037+0000","last_became_active":"2023-12-26T23:09:46.599046+0000","last_became_peered":"2023-12-26T23:09:46.599046+0000","last_unstale":"2023-12-26T23:09:46.603037+0000","last_undegraded":"2023-12-26T23:09:46.603037+0000","last_fullsized":"2023-12-26T23:09:46.603037+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:07:11.137583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.594200+0000","last_change":"2023-12-26T23:09:45.583854+0000","last_active":"2023-12-26T23:09:46.594200+0000","last_peered":"2023-12-26T23:09:46.594200+0000","last_clean":"2023-12-26T23:09:46.594200+0000","last_became_active":"2023-12-26T23:09:45.583603+0000","last_became_peered":"2023-12-26T23:09:45.583603+0000","last_unstale":"2023-12-26T23:09:46.594200+0000","last_undegraded":"2023-12-26T23:09:46.594200+0000","last_fullsized":"2023-12-26T23:09:46.594200+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:48:39.313836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599117+0000","last_change":"2023-12-26T23:09:46.595346+0000","last_active":"2023-12-26T23:09:46.599117+0000","last_peered":"2023-12-26T23:09:46.599117+0000","last_clean":"2023-12-26T23:09:46.599117+0000","last_became_active":"2023-12-26T23:09:46.595058+0000","last_became_peered":"2023-12-26T23:09:46.595058+0000","last_unstale":"2023-12-26T23:09:46.599117+0000","last_undegraded":"2023-12-26T23:09:46.599117+0000","last_fullsized":"2023-12-26T23:09:46.599117+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:31:03.025875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.593741+0000","last_change":"2023-12-26T23:09:45.590066+0000","last_active":"2023-12-26T23:09:46.593741+0000","last_peered":"2023-12-26T23:09:46.593741+0000","last_clean":"2023-12-26T23:09:46.593741+0000","last_became_active":"2023-12-26T23:09:45.588052+0000","last_became_peered":"2023-12-26T23:09:45.588052+0000","last_unstale":"2023-12-26T23:09:46.593741+0000","last_undegraded":"2023-12-26T23:09:46.593741+0000","last_fullsized":"2023-12-26T23:09:46.593741+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:34:22.541440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604773+0000","last_change":"2023-12-26T23:09:46.602552+0000","last_active":"2023-12-26T23:09:46.604773+0000","last_peered":"2023-12-26T23:09:46.604773+0000","last_clean":"2023-12-26T23:09:46.604773+0000","last_became_active":"2023-12-26T23:09:46.602164+0000","last_became_peered":"2023-12-26T23:09:46.602164+0000","last_unstale":"2023-12-26T23:09:46.604773+0000","last_undegraded":"2023-12-26T23:09:46.604773+0000","last_fullsized":"2023-12-26T23:09:46.604773+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:46:26.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.573735+0000","last_change":"2023-12-26T23:09:45.585289+0000","last_active":"2023-12-26T23:09:46.573735+0000","last_peered":"2023-12-26T23:09:46.573735+0000","last_clean":"2023-12-26T23:09:46.573735+0000","last_became_active":"2023-12-26T23:09:45.584881+0000","last_became_peered":"2023-12-26T23:09:45.584881+0000","last_unstale":"2023-12-26T23:09:46.573735+0000","last_undegraded":"2023-12-26T23:09:46.573735+0000","last_fullsized":"2023-12-26T23:09:46.573735+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:39.956735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"182'326","reported_seq":1059,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.573212+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:09:46.573212+0000","last_peered":"2023-12-26T23:09:46.573212+0000","last_clean":"2023-12-26T23:09:46.573212+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:09:46.573212+0000","last_undegraded":"2023-12-26T23:09:46.573212+0000","last_fullsized":"2023-12-26T23:09:46.573212+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543251,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307816,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064024,"statfs":{"total":96636764160,"available":96321560576,"internally_reserved":0,"allocated":2392064,"data_stored":1201002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.54600000000000004,"5min":0.436,"15min":0.372},"max":{"1min":0.92200000000000004,"5min":0.96299999999999997,"15min":1.048},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.60099999999999998,"5min":0.45200000000000001,"15min":0.40899999999999997},"max":{"1min":0.90600000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60899999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.51100000000000001,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.91100000000000003,"5min":0.98599999999999999,"15min":0.995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":0.88400000000000001,"5min":0.96599999999999997,"15min":1.0449999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73999999999999999,"15min":0.74099999999999999},"min":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.89800000000000002,"5min":0.94199999999999995,"15min":1.0049999999999999},"last":0.628}]},{"osd":3,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91400000000000003,"5min":1.0229999999999999,"15min":1.0760000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":0.97599999999999998,"15min":0.98199999999999998},"last":0.66300000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.71799999999999997,"5min":0.90400000000000003,"15min":0.91400000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.39600000000000002,"15min":0.38},"max":{"1min":0.753,"5min":0.83699999999999997,"15min":0.93000000000000005},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.42299999999999999,"15min":0.39400000000000002},"max":{"1min":0.73299999999999998,"5min":0.89100000000000001,"15min":0.98799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89000000000000001,"5min":0.998,"15min":1.042},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66700000000000004,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.45100000000000001},"max":{"1min":0.82499999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.56000000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.85599999999999998,"5min":0.96599999999999997,"15min":1.0129999999999999},"last":0.58999999999999997}]}]},{"osd":6,"up_from":15,"seq":64424510547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307820,"kb_used_data":2340,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064020,"statfs":{"total":96636764160,"available":96321556480,"internally_reserved":0,"allocated":2396160,"data_stored":1201036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.47099999999999997,"5min":0.438,"15min":0.32800000000000001},"max":{"1min":0.82299999999999995,"5min":0.96599999999999997,"15min":1.1080000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.61699999999999999,"5min":0.498,"15min":0.41799999999999998},"max":{"1min":0.92400000000000004,"5min":0.94299999999999995,"15min":1.0680000000000001},"last":0.72299999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.50600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":1.046},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.746,"15min":0.754},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95999999999999996,"5min":0.98099999999999998,"15min":1.0580000000000001},"last":0.70599999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.61299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89100000000000001,"5min":0.98899999999999999,"15min":1.071},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.63,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94599999999999995,"5min":0.95299999999999996,"15min":1.0680000000000001},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75800000000000001,"15min":0.76300000000000001},"min":{"1min":0.58699999999999997,"5min":0.496,"15min":0.47799999999999998},"max":{"1min":0.92800000000000005,"5min":1.024,"15min":1.024},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":0.91700000000000004,"5min":1,"15min":1.032},"last":0.68899999999999995}]},{"osd":4,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.432,"15min":0.39400000000000002},"max":{"1min":0.90500000000000003,"5min":0.92600000000000005,"15min":1.1599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.48499999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92100000000000004,"5min":0.96399999999999997,"15min":1.111},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":1.0149999999999999,"15min":1.0860000000000001},"last":0.57499999999999996}]},{"osd":7,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.48199999999999998,"5min":0.443,"15min":0.42999999999999999},"max":{"1min":0.89700000000000002,"5min":1.03,"15min":1.1419999999999999},"last":0.626},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95999999999999996,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.59699999999999998}]}]},{"osd":1,"up_from":15,"seq":64424510547,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316624,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055216,"statfs":{"total":96636764160,"available":96312541184,"internally_reserved":0,"allocated":3022848,"data_stored":1799868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.32100000000000001,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.60099999999999998,"5min":0.68700000000000006,"15min":0.81399999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.79200000000000004},"last":0.496}]},{"osd":2,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.33800000000000002},"max":{"1min":0.61599999999999999,"5min":0.753,"15min":0.84099999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.35899999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.91200000000000003},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.41399999999999998},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.32700000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.93600000000000005},"last":0.52700000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.55100000000000005,"5min":0.53900000000000003,"15min":0.499},"max":{"1min":0.97399999999999998,"5min":1.103,"15min":1.103},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78800000000000003,"15min":0.77100000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.48599999999999999},"max":{"1min":1.0369999999999999,"5min":1.1739999999999999,"15min":1.2410000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81100000000000005,"15min":0.80800000000000005},"min":{"1min":0.63400000000000001,"5min":0.55000000000000004,"15min":0.50700000000000001},"max":{"1min":1.008,"5min":1.0669999999999999,"15min":1.22},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.1240000000000001,"15min":1.1839999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82399999999999995,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.52600000000000002},"max":{"1min":1.0589999999999999,"5min":1.087,"15min":1.1950000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85399999999999998,"15min":0.83899999999999997},"min":{"1min":0.625,"5min":0.625,"15min":0.54200000000000004},"max":{"1min":1.0089999999999999,"5min":1.2090000000000001,"15min":1.218},"last":0.79000000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.0469999999999999,"5min":1.056,"15min":1.125},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83099999999999996,"15min":0.82199999999999995},"min":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.55400000000000005},"max":{"1min":1.0229999999999999,"5min":1.1459999999999999,"15min":1.218},"last":0.88900000000000001}]}]},{"osd":0,"up_from":15,"seq":64424510547,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307832,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064008,"statfs":{"total":96636764160,"available":96321544192,"internally_reserved":0,"allocated":2408448,"data_stored":1207852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.45400000000000001},"min":{"1min":0.32600000000000001,"5min":0.30199999999999999,"15min":0.26600000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.499,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.313},"max":{"1min":0.63500000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.371}]},{"osd":2,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.505,"15min":0.51300000000000001},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.30599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54000000000000004,"15min":0.54700000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.38200000000000001},"max":{"1min":0.71899999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.434}]},{"osd":3,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.67600000000000005,"5min":0.90200000000000002,"15min":1.3839999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.45300000000000001},"max":{"1min":0.83099999999999996,"5min":0.85599999999999998,"15min":1.012},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.51700000000000002},"max":{"1min":0.92600000000000005,"5min":1.0840000000000001,"15min":1.2250000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96799999999999997,"5min":1.0409999999999999,"15min":1.113},"last":0.64400000000000002}]},{"osd":5,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79900000000000004,"15min":0.82099999999999995},"min":{"1min":0.622,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.014,"5min":1.0920000000000001,"15min":1.157},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97999999999999998,"5min":1.0740000000000001,"15min":1.282},"last":0.81799999999999995}]},{"osd":6,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78900000000000003,"15min":0.80800000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.90800000000000003,"5min":1.0529999999999999,"15min":1.2749999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.1299999999999999,"15min":1.3129999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.58699999999999997,"5min":0.57199999999999995,"15min":0.53900000000000003},"max":{"1min":0.95299999999999996,"5min":1.173,"15min":1.256},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98399999999999999,"5min":1.194,"15min":1.3360000000000001},"last":0.85799999999999998}]}]},{"osd":2,"up_from":14,"seq":60129543252,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311928,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059912,"statfs":{"total":96636764160,"available":96317349888,"internally_reserved":0,"allocated":2408448,"data_stored":1208336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48799999999999999,"15min":0.47299999999999998},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":0.72399999999999998,"15min":0.84599999999999997},"last":0.316},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.41099999999999998},"max":{"1min":0.65200000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.4219999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.373,"5min":0.36799999999999999,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":0.75600000000000001,"15min":0.91200000000000003},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.497},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.755},"last":1.9630000000000001}]},{"osd":3,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.627,"15min":0.622},"min":{"1min":0.498,"5min":0.45900000000000002,"15min":0.441},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.96299999999999997},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.83199999999999996,"5min":0.89600000000000002,"15min":1.056},"last":0.89300000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79800000000000004,"15min":0.79600000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.046,"5min":1.427,"15min":1.427},"last":1.8},{"interface":"front","average":{"1min":0.745,"5min":0.80100000000000005,"15min":0.79400000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.54000000000000004},"max":{"1min":0.93100000000000005,"5min":1.1719999999999999,"15min":1.2070000000000001},"last":1.222}]},{"osd":5,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.80600000000000005,"15min":0.79500000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.50800000000000001},"max":{"1min":0.92700000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.923},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82899999999999996,"15min":0.81200000000000006},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.98999999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.1599999999999999}]},{"osd":6,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81499999999999995,"15min":0.79500000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.52400000000000002},"max":{"1min":0.95799999999999996,"5min":1.272,"15min":1.272},"last":0.755},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.53300000000000003},"max":{"1min":1.004,"5min":1.1040000000000001,"15min":1.1830000000000001},"last":1.8859999999999999}]},{"osd":7,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84199999999999997,"15min":0.82099999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.56399999999999995},"max":{"1min":1.0249999999999999,"5min":1.161,"15min":1.1619999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.85199999999999998,"15min":0.84099999999999997},"min":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.55100000000000005},"max":{"1min":1.0780000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.841}]}]},{"osd":3,"up_from":15,"seq":64424510548,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303736,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068104,"statfs":{"total":96636764160,"available":96325738496,"internally_reserved":0,"allocated":2408448,"data_stored":1202560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.46500000000000002},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.246},"max":{"1min":0.66700000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.316},"max":{"1min":0.63300000000000001,"5min":0.74199999999999999,"15min":0.82299999999999995},"last":0.316}]},{"osd":1,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.67500000000000004,"15min":0.753},"last":0.372},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.307},"max":{"1min":0.67200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.47999999999999998,"5min":0.442,"15min":0.372},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66300000000000003,"15min":0.65800000000000003},"min":{"1min":0.52900000000000003,"5min":0.49099999999999999,"15min":0.41499999999999998},"max":{"1min":0.93500000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.159,"5min":1.177,"15min":1.177},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":1.0329999999999999,"15min":1.081},"last":0.501}]},{"osd":5,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56000000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0980000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.056,"5min":1.0860000000000001,"15min":1.1839999999999999},"last":0.48999999999999999}]},{"osd":6,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.52000000000000002},"max":{"1min":1.042,"5min":1.1359999999999999,"15min":1.147},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61099999999999999},"max":{"1min":1.0569999999999999,"5min":1.101,"15min":1.167},"last":0.64200000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.024,"5min":1.141,"15min":1.214},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.83699999999999997,"15min":0.83299999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1180000000000001,"5min":1.21,"15min":1.21},"last":0.58599999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543251,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311916,"kb_used_data":2340,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059924,"statfs":{"total":96636764160,"available":96317362176,"internally_reserved":0,"allocated":2396160,"data_stored":1201036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.40799999999999997},"max":{"1min":0.872,"5min":0.90500000000000003,"15min":1.0489999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.52200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.83699999999999997,"5min":1.008,"15min":1.087},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.73899999999999999,"15min":0.74199999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79000000000000004,"5min":0.997,"15min":1.0600000000000001},"last":0.751},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":0.82499999999999996,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.53000000000000003,"5min":0.47199999999999998,"15min":0.46500000000000002},"max":{"1min":0.89600000000000002,"5min":0.99199999999999999,"15min":1.0109999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81599999999999995,"5min":0.91700000000000004,"15min":0.998},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73899999999999999,"15min":0.73399999999999999},"min":{"1min":0.56299999999999994,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76400000000000001,"15min":0.76700000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":0.89400000000000002,"5min":1.0469999999999999,"15min":1.0589999999999999},"last":0.623}]},{"osd":5,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.63400000000000001,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96299999999999997,"5min":0.96799999999999997,"15min":0.98999999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.48999999999999999,"5min":0.42099999999999999,"15min":0.38500000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.024},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71699999999999997,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47099999999999997},"max":{"1min":0.96999999999999997,"5min":0.98099999999999998,"15min":0.98799999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.499,"5min":0.495,"15min":0.44500000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76800000000000002,"15min":0.752},"min":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.68899999999999995}]}]},{"osd":5,"up_from":15,"seq":64424510546,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300212,"kb_used_data":2924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071628,"statfs":{"total":96636764160,"available":96329347072,"internally_reserved":0,"allocated":2994176,"data_stored":1792014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.56399999999999995,"5min":0.45700000000000002,"15min":0.434},"max":{"1min":0.89100000000000001,"5min":0.96799999999999997,"15min":0.998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.57999999999999996,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.88500000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.57899999999999996,"5min":0.46300000000000002,"15min":0.41899999999999998},"max":{"1min":0.97699999999999998,"5min":0.98299999999999998,"15min":1.0580000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.74399999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":1.016,"15min":1.016},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.59199999999999997,"5min":0.46300000000000002,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":1.012,"15min":1.022},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.57799999999999996,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.88800000000000001,"5min":0.92800000000000005,"15min":0.99199999999999999},"last":0.66100000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76100000000000001,"15min":0.75600000000000001},"min":{"1min":0.59799999999999998,"5min":0.53700000000000003,"15min":0.498},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.60199999999999998,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.72799999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.622,"15min":0.59499999999999997},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.35899999999999999},"max":{"1min":0.82499999999999996,"5min":0.91600000000000004,"15min":1.0309999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":0.81200000000000006,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.78800000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.90000000000000002,"5min":0.97799999999999998,"15min":1.042},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78800000000000003,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.93100000000000005,"5min":1.0309999999999999,"15min":1.0820000000000001},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.71799999999999997,"15min":0.69899999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.75,"5min":0.996,"15min":0.996},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.75,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.48699999999999999,"15min":0.46300000000000002},"max":{"1min":0.96199999999999997,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:09:50.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:09:51.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.171+0000 7efc509ff700 1 Processor -- start 2023-12-26T23:09:51.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.171+0000 7efc509ff700 1 -- start start 2023-12-26T23:09:51.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.172+0000 7efc509ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c154110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.172+0000 7efc509ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c1505a0 con 0x7efc4c156350 2023-12-26T23:09:51.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.172+0000 7efc509ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c1586c0 con 0x7efc4c0b04e0 2023-12-26T23:09:51.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.172+0000 7efc509ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0b0960 con 0x7efc4c153cd0 2023-12-26T23:09:51.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4a59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efc4c156350 0x7efc4c157260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55104/0 (socket says 172.21.15.131:55104) 2023-12-26T23:09:51.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4a59c700 1 -- 172.21.15.131:0/4156049986 learned_addr learned my addr 172.21.15.131:0/4156049986 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:51.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4ad9d700 1 --2- 172.21.15.131:0/4156049986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4959a700 1 -- 172.21.15.131:0/4156049986 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021509853 0 0) 0x7efc3c001290 con 0x7efc4c156350 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4959a700 1 -- 172.21.15.131:0/4156049986 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc34004680 con 0x7efc4c156350 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4959a700 1 -- 172.21.15.131:0/4156049986 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505819979 0 0) 0x7efc4c1586c0 con 0x7efc4c0b04e0 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4959a700 1 -- 172.21.15.131:0/4156049986 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc3c001290 con 0x7efc4c0b04e0 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4ad9d700 1 -- 172.21.15.131:0/4156049986 >> v1:172.21.15.190:6789/0 conn(0x7efc4c0b04e0 legacy=0x7efc4c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4959a700 1 -- 172.21.15.131:0/4156049986 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 573279087 0 0) 0x7efc34004680 con 0x7efc4c156350 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4ad9d700 1 -- 172.21.15.131:0/4156049986 >> v1:172.21.15.131:6789/0 conn(0x7efc4c156350 legacy=0x7efc4c157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.173+0000 7efc4ad9d700 1 -- 172.21.15.131:0/4156049986 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c157ab0 con 0x7efc4c153cd0 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc4ad9d700 1 --2- 172.21.15.131:0/4156049986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c154110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efc44009e90 tx=0x7efc44009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b61f250494ffcfb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc4959a700 1 -- 172.21.15.131:0/4156049986 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4400e070 con 0x7efc4c153cd0 2023-12-26T23:09:51.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc4959a700 1 -- 172.21.15.131:0/4156049986 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4400b870 con 0x7efc4c153cd0 2023-12-26T23:09:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc4959a700 1 -- 172.21.15.131:0/4156049986 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c001780 con 0x7efc4c153cd0 2023-12-26T23:09:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc509ff700 1 -- 172.21.15.131:0/4156049986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 msgr2=0x7efc4c154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc509ff700 1 --2- 172.21.15.131:0/4156049986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c154110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efc44009e90 tx=0x7efc44009780 comp rx=0 tx=0).stop 2023-12-26T23:09:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc509ff700 1 -- 172.21.15.131:0/4156049986 shutdown_connections 2023-12-26T23:09:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc509ff700 1 --2- 172.21.15.131:0/4156049986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c154110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc509ff700 1 -- 172.21.15.131:0/4156049986 >> 172.21.15.131:0/4156049986 conn(0x7efc4c068040 msgr2=0x7efc4c0cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.174+0000 7efc509ff700 1 -- 172.21.15.131:0/4156049986 shutdown_connections 2023-12-26T23:09:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.175+0000 7efc509ff700 1 -- 172.21.15.131:0/4156049986 wait complete. 2023-12-26T23:09:51.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.176+0000 7efc509ff700 1 Processor -- start 2023-12-26T23:09:51.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.176+0000 7efc509ff700 1 -- start start 2023-12-26T23:09:51.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.177+0000 7efc509ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c0b04e0 0x7efc4c0af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.177+0000 7efc509ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.177+0000 7efc509ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc4c156350 0x7efc4c16d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.177+0000 7efc509ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c16e1a0 con 0x7efc4c156350 2023-12-26T23:09:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.177+0000 7efc509ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c15adf0 con 0x7efc4c0b04e0 2023-12-26T23:09:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.177+0000 7efc509ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c16dbe0 con 0x7efc4c153cd0 2023-12-26T23:09:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.177+0000 7efc4ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc4c156350 0x7efc4c16d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.177+0000 7efc4ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc4c156350 0x7efc4c16d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52410/0 (socket says 172.21.15.131:52410) 2023-12-26T23:09:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.177+0000 7efc4ad9d700 1 -- 172.21.15.131:0/3571113843 learned_addr learned my addr 172.21.15.131:0/3571113843 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc4a59c700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c0b04e0 0x7efc4c0af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc4ad9d700 1 -- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 msgr2=0x7efc4c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc49d9b700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc4ad9d700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc4ad9d700 1 -- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c0b04e0 msgr2=0x7efc4c0af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc4ad9d700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c0b04e0 0x7efc4c0af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc4ad9d700 1 -- 172.21.15.131:0/3571113843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc44009a20 con 0x7efc4c156350 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc4a59c700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c0b04e0 0x7efc4c0af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc49d9b700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc4ad9d700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc4c156350 0x7efc4c16d200 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7efc4c070fa0 tx=0x7efc44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:51.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc3a7fc700 1 -- 172.21.15.131:0/3571113843 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44012d10 con 0x7efc4c156350 2023-12-26T23:09:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc3a7fc700 1 -- 172.21.15.131:0/3571113843 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44012e90 con 0x7efc4c156350 2023-12-26T23:09:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c16dec0 con 0x7efc4c156350 2023-12-26T23:09:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc3a7fc700 1 -- 172.21.15.131:0/3571113843 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4401e6a0 con 0x7efc4c156350 2023-12-26T23:09:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.178+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c150890 con 0x7efc4c156350 2023-12-26T23:09:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.179+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c15af80 con 0x7efc4c156350 2023-12-26T23:09:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.180+0000 7efc3a7fc700 1 -- 172.21.15.131:0/3571113843 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efc4401ea50 con 0x7efc4c156350 2023-12-26T23:09:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.181+0000 7efc3a7fc700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7efc2c066f30 0x7efc2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.181+0000 7efc3a7fc700 1 -- 172.21.15.131:0/3571113843 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7efc44086cb0 con 0x7efc4c156350 2023-12-26T23:09:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.181+0000 7efc4a59c700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7efc2c066f30 0x7efc2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.182+0000 7efc4a59c700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7efc2c066f30 0x7efc2c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc3c001170 tx=0x7efc3c009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.185+0000 7efc3a7fc700 1 -- 172.21.15.131:0/3571113843 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc44053d30 con 0x7efc4c156350 2023-12-26T23:09:51.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.373+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc4c0657e0 con 0x7efc2c066f30 2023-12-26T23:09:51.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.381+0000 7efc3a7fc700 1 -- 172.21.15.131:0/3571113843 <== mgr.9456 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289630 (secure 0 0 0) 0x7efc4c0657e0 con 0x7efc2c066f30 2023-12-26T23:09:51.386 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:51.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.385+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7efc2c066f30 msgr2=0x7efc2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.385+0000 7efc509ff700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7efc2c066f30 0x7efc2c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc3c001170 tx=0x7efc3c009040 comp rx=0 tx=0).stop 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.385+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc4c156350 msgr2=0x7efc4c16d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.385+0000 7efc509ff700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc4c156350 0x7efc4c16d200 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7efc4c070fa0 tx=0x7efc44017000 comp rx=0 tx=0).stop 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.386+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 shutdown_connections 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.386+0000 7efc509ff700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7efc2c066f30 0x7efc2c0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.386+0000 7efc509ff700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc4c156350 0x7efc4c16d200 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.386+0000 7efc509ff700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c0b04e0 0x7efc4c0af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.386+0000 7efc509ff700 1 --2- 172.21.15.131:0/3571113843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4c153cd0 0x7efc4c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.386+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 >> 172.21.15.131:0/3571113843 conn(0x7efc4c068040 msgr2=0x7efc4c157240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.386+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 shutdown_connections 2023-12-26T23:09:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.386+0000 7efc509ff700 1 -- 172.21.15.131:0/3571113843 wait complete. 2023-12-26T23:09:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:09:51.404 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-26T23:09:51.110659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2467884,"kb_used_data":19948,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752506836,"statfs":{"total":773094113280,"available":770567000064,"internally_reserved":0,"allocated":20426752,"data_stored":10813704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001226"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607300+0000","last_change":"2023-12-26T23:09:46.604583+0000","last_active":"2023-12-26T23:09:46.607300+0000","last_peered":"2023-12-26T23:09:46.607300+0000","last_clean":"2023-12-26T23:09:46.607300+0000","last_became_active":"2023-12-26T23:09:46.604290+0000","last_became_peered":"2023-12-26T23:09:46.604290+0000","last_unstale":"2023-12-26T23:09:46.607300+0000","last_undegraded":"2023-12-26T23:09:46.607300+0000","last_fullsized":"2023-12-26T23:09:46.607300+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:28:42.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607140+0000","last_change":"2023-12-26T23:09:46.604724+0000","last_active":"2023-12-26T23:09:46.607140+0000","last_peered":"2023-12-26T23:09:46.607140+0000","last_clean":"2023-12-26T23:09:46.607140+0000","last_became_active":"2023-12-26T23:09:46.604419+0000","last_became_peered":"2023-12-26T23:09:46.604419+0000","last_unstale":"2023-12-26T23:09:46.607140+0000","last_undegraded":"2023-12-26T23:09:46.607140+0000","last_fullsized":"2023-12-26T23:09:46.607140+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:39:28.252771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606847+0000","last_change":"2023-12-26T23:09:46.601125+0000","last_active":"2023-12-26T23:09:46.606847+0000","last_peered":"2023-12-26T23:09:46.606847+0000","last_clean":"2023-12-26T23:09:46.606847+0000","last_became_active":"2023-12-26T23:09:46.600809+0000","last_became_peered":"2023-12-26T23:09:46.600809+0000","last_unstale":"2023-12-26T23:09:46.606847+0000","last_undegraded":"2023-12-26T23:09:46.606847+0000","last_fullsized":"2023-12-26T23:09:46.606847+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:06:10.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603561+0000","last_change":"2023-12-26T23:09:46.593583+0000","last_active":"2023-12-26T23:09:46.603561+0000","last_peered":"2023-12-26T23:09:46.603561+0000","last_clean":"2023-12-26T23:09:46.603561+0000","last_became_active":"2023-12-26T23:09:46.592826+0000","last_became_peered":"2023-12-26T23:09:46.592826+0000","last_unstale":"2023-12-26T23:09:46.603561+0000","last_undegraded":"2023-12-26T23:09:46.603561+0000","last_fullsized":"2023-12-26T23:09:46.603561+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:35:35.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.593116+0000","last_change":"2023-12-26T23:09:46.586477+0000","last_active":"2023-12-26T23:09:46.593116+0000","last_peered":"2023-12-26T23:09:46.593116+0000","last_clean":"2023-12-26T23:09:46.593116+0000","last_became_active":"2023-12-26T23:09:46.585043+0000","last_became_peered":"2023-12-26T23:09:46.585043+0000","last_unstale":"2023-12-26T23:09:46.593116+0000","last_undegraded":"2023-12-26T23:09:46.593116+0000","last_fullsized":"2023-12-26T23:09:46.593116+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:07:11.137583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602175+0000","last_change":"2023-12-26T23:09:46.598405+0000","last_active":"2023-12-26T23:09:46.602175+0000","last_peered":"2023-12-26T23:09:46.602175+0000","last_clean":"2023-12-26T23:09:46.602175+0000","last_became_active":"2023-12-26T23:09:46.598081+0000","last_became_peered":"2023-12-26T23:09:46.598081+0000","last_unstale":"2023-12-26T23:09:46.602175+0000","last_undegraded":"2023-12-26T23:09:46.602175+0000","last_fullsized":"2023-12-26T23:09:46.602175+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:32:54.019368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602944+0000","last_change":"2023-12-26T23:09:46.599347+0000","last_active":"2023-12-26T23:09:46.602944+0000","last_peered":"2023-12-26T23:09:46.602944+0000","last_clean":"2023-12-26T23:09:46.602944+0000","last_became_active":"2023-12-26T23:09:46.598821+0000","last_became_peered":"2023-12-26T23:09:46.598821+0000","last_unstale":"2023-12-26T23:09:46.602944+0000","last_undegraded":"2023-12-26T23:09:46.602944+0000","last_fullsized":"2023-12-26T23:09:46.602944+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:14:21.007369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.598028+0000","last_change":"2023-12-26T23:09:46.588940+0000","last_active":"2023-12-26T23:09:46.598028+0000","last_peered":"2023-12-26T23:09:46.598028+0000","last_clean":"2023-12-26T23:09:46.598028+0000","last_became_active":"2023-12-26T23:09:46.588660+0000","last_became_peered":"2023-12-26T23:09:46.588660+0000","last_unstale":"2023-12-26T23:09:46.598028+0000","last_undegraded":"2023-12-26T23:09:46.598028+0000","last_fullsized":"2023-12-26T23:09:46.598028+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:04:25.759214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606681+0000","last_change":"2023-12-26T23:09:46.599553+0000","last_active":"2023-12-26T23:09:46.606681+0000","last_peered":"2023-12-26T23:09:46.606681+0000","last_clean":"2023-12-26T23:09:46.606681+0000","last_became_active":"2023-12-26T23:09:46.599288+0000","last_became_peered":"2023-12-26T23:09:46.599288+0000","last_unstale":"2023-12-26T23:09:46.606681+0000","last_undegraded":"2023-12-26T23:09:46.606681+0000","last_fullsized":"2023-12-26T23:09:46.606681+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:20:30.266221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596988+0000","last_change":"2023-12-26T23:09:46.588421+0000","last_active":"2023-12-26T23:09:46.596988+0000","last_peered":"2023-12-26T23:09:46.596988+0000","last_clean":"2023-12-26T23:09:46.596988+0000","last_became_active":"2023-12-26T23:09:46.587928+0000","last_became_peered":"2023-12-26T23:09:46.587928+0000","last_unstale":"2023-12-26T23:09:46.596988+0000","last_undegraded":"2023-12-26T23:09:46.596988+0000","last_fullsized":"2023-12-26T23:09:46.596988+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:15:17.735808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.598803+0000","last_change":"2023-12-26T23:09:46.594067+0000","last_active":"2023-12-26T23:09:46.598803+0000","last_peered":"2023-12-26T23:09:46.598803+0000","last_clean":"2023-12-26T23:09:46.598803+0000","last_became_active":"2023-12-26T23:09:46.593176+0000","last_became_peered":"2023-12-26T23:09:46.593176+0000","last_unstale":"2023-12-26T23:09:46.598803+0000","last_undegraded":"2023-12-26T23:09:46.598803+0000","last_fullsized":"2023-12-26T23:09:46.598803+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:05.643738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.601107+0000","last_change":"2023-12-26T23:09:46.589699+0000","last_active":"2023-12-26T23:09:46.601107+0000","last_peered":"2023-12-26T23:09:46.601107+0000","last_clean":"2023-12-26T23:09:46.601107+0000","last_became_active":"2023-12-26T23:09:46.587539+0000","last_became_peered":"2023-12-26T23:09:46.587539+0000","last_unstale":"2023-12-26T23:09:46.601107+0000","last_undegraded":"2023-12-26T23:09:46.601107+0000","last_fullsized":"2023-12-26T23:09:46.601107+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:54:55.628676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602714+0000","last_change":"2023-12-26T23:09:46.593850+0000","last_active":"2023-12-26T23:09:46.602714+0000","last_peered":"2023-12-26T23:09:46.602714+0000","last_clean":"2023-12-26T23:09:46.602714+0000","last_became_active":"2023-12-26T23:09:46.592837+0000","last_became_peered":"2023-12-26T23:09:46.592837+0000","last_unstale":"2023-12-26T23:09:46.602714+0000","last_undegraded":"2023-12-26T23:09:46.602714+0000","last_fullsized":"2023-12-26T23:09:46.602714+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:31:57.686453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596693+0000","last_change":"2023-12-26T23:09:46.589078+0000","last_active":"2023-12-26T23:09:46.596693+0000","last_peered":"2023-12-26T23:09:46.596693+0000","last_clean":"2023-12-26T23:09:46.596693+0000","last_became_active":"2023-12-26T23:09:46.588822+0000","last_became_peered":"2023-12-26T23:09:46.588822+0000","last_unstale":"2023-12-26T23:09:46.596693+0000","last_undegraded":"2023-12-26T23:09:46.596693+0000","last_fullsized":"2023-12-26T23:09:46.596693+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:33:38.292911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.597377+0000","last_change":"2023-12-26T23:09:46.590757+0000","last_active":"2023-12-26T23:09:46.597377+0000","last_peered":"2023-12-26T23:09:46.597377+0000","last_clean":"2023-12-26T23:09:46.597377+0000","last_became_active":"2023-12-26T23:09:46.590454+0000","last_became_peered":"2023-12-26T23:09:46.590454+0000","last_unstale":"2023-12-26T23:09:46.597377+0000","last_undegraded":"2023-12-26T23:09:46.597377+0000","last_fullsized":"2023-12-26T23:09:46.597377+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:42.286153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596419+0000","last_change":"2023-12-26T23:09:46.593070+0000","last_active":"2023-12-26T23:09:46.596419+0000","last_peered":"2023-12-26T23:09:46.596419+0000","last_clean":"2023-12-26T23:09:46.596419+0000","last_became_active":"2023-12-26T23:09:46.591746+0000","last_became_peered":"2023-12-26T23:09:46.591746+0000","last_unstale":"2023-12-26T23:09:46.596419+0000","last_undegraded":"2023-12-26T23:09:46.596419+0000","last_fullsized":"2023-12-26T23:09:46.596419+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:33:06.116398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.587534+0000","last_change":"2023-12-26T23:09:46.581145+0000","last_active":"2023-12-26T23:09:46.587534+0000","last_peered":"2023-12-26T23:09:46.587534+0000","last_clean":"2023-12-26T23:09:46.587534+0000","last_became_active":"2023-12-26T23:09:46.580240+0000","last_became_peered":"2023-12-26T23:09:46.580240+0000","last_unstale":"2023-12-26T23:09:46.587534+0000","last_undegraded":"2023-12-26T23:09:46.587534+0000","last_fullsized":"2023-12-26T23:09:46.587534+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:11:39.652050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604136+0000","last_change":"2023-12-26T23:09:46.598701+0000","last_active":"2023-12-26T23:09:46.604136+0000","last_peered":"2023-12-26T23:09:46.604136+0000","last_clean":"2023-12-26T23:09:46.604136+0000","last_became_active":"2023-12-26T23:09:46.598385+0000","last_became_peered":"2023-12-26T23:09:46.598385+0000","last_unstale":"2023-12-26T23:09:46.604136+0000","last_undegraded":"2023-12-26T23:09:46.604136+0000","last_fullsized":"2023-12-26T23:09:46.604136+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:23:43.770033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603208+0000","last_change":"2023-12-26T23:09:46.597227+0000","last_active":"2023-12-26T23:09:46.603208+0000","last_peered":"2023-12-26T23:09:46.603208+0000","last_clean":"2023-12-26T23:09:46.603208+0000","last_became_active":"2023-12-26T23:09:46.596951+0000","last_became_peered":"2023-12-26T23:09:46.596951+0000","last_unstale":"2023-12-26T23:09:46.603208+0000","last_undegraded":"2023-12-26T23:09:46.603208+0000","last_fullsized":"2023-12-26T23:09:46.603208+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:17:55.348285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602628+0000","last_change":"2023-12-26T23:09:46.594826+0000","last_active":"2023-12-26T23:09:46.602628+0000","last_peered":"2023-12-26T23:09:46.602628+0000","last_clean":"2023-12-26T23:09:46.602628+0000","last_became_active":"2023-12-26T23:09:46.594579+0000","last_became_peered":"2023-12-26T23:09:46.594579+0000","last_unstale":"2023-12-26T23:09:46.602628+0000","last_undegraded":"2023-12-26T23:09:46.602628+0000","last_fullsized":"2023-12-26T23:09:46.602628+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:08:43.055929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.587099+0000","last_change":"2023-12-26T23:09:46.583496+0000","last_active":"2023-12-26T23:09:46.587099+0000","last_peered":"2023-12-26T23:09:46.587099+0000","last_clean":"2023-12-26T23:09:46.587099+0000","last_became_active":"2023-12-26T23:09:46.583232+0000","last_became_peered":"2023-12-26T23:09:46.583232+0000","last_unstale":"2023-12-26T23:09:46.587099+0000","last_undegraded":"2023-12-26T23:09:46.587099+0000","last_fullsized":"2023-12-26T23:09:46.587099+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:27:42.382215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600874+0000","last_change":"2023-12-26T23:09:46.589655+0000","last_active":"2023-12-26T23:09:46.600874+0000","last_peered":"2023-12-26T23:09:46.600874+0000","last_clean":"2023-12-26T23:09:46.600874+0000","last_became_active":"2023-12-26T23:09:46.589412+0000","last_became_peered":"2023-12-26T23:09:46.589412+0000","last_unstale":"2023-12-26T23:09:46.600874+0000","last_undegraded":"2023-12-26T23:09:46.600874+0000","last_fullsized":"2023-12-26T23:09:46.600874+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:59:57.577667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607004+0000","last_change":"2023-12-26T23:09:46.601505+0000","last_active":"2023-12-26T23:09:46.607004+0000","last_peered":"2023-12-26T23:09:46.607004+0000","last_clean":"2023-12-26T23:09:46.607004+0000","last_became_active":"2023-12-26T23:09:46.601205+0000","last_became_peered":"2023-12-26T23:09:46.601205+0000","last_unstale":"2023-12-26T23:09:46.607004+0000","last_undegraded":"2023-12-26T23:09:46.607004+0000","last_fullsized":"2023-12-26T23:09:46.607004+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:33:38.292911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.592394+0000","last_change":"2023-12-26T23:09:46.583704+0000","last_active":"2023-12-26T23:09:46.592394+0000","last_peered":"2023-12-26T23:09:46.592394+0000","last_clean":"2023-12-26T23:09:46.592394+0000","last_became_active":"2023-12-26T23:09:46.581921+0000","last_became_peered":"2023-12-26T23:09:46.581921+0000","last_unstale":"2023-12-26T23:09:46.592394+0000","last_undegraded":"2023-12-26T23:09:46.592394+0000","last_fullsized":"2023-12-26T23:09:46.592394+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:05:17.387272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603435+0000","last_change":"2023-12-26T23:09:46.600528+0000","last_active":"2023-12-26T23:09:46.603435+0000","last_peered":"2023-12-26T23:09:46.603435+0000","last_clean":"2023-12-26T23:09:46.603435+0000","last_became_active":"2023-12-26T23:09:46.598038+0000","last_became_peered":"2023-12-26T23:09:46.598038+0000","last_unstale":"2023-12-26T23:09:46.603435+0000","last_undegraded":"2023-12-26T23:09:46.603435+0000","last_fullsized":"2023-12-26T23:09:46.603435+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:46:38.005388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600294+0000","last_change":"2023-12-26T23:09:46.593840+0000","last_active":"2023-12-26T23:09:46.600294+0000","last_peered":"2023-12-26T23:09:46.600294+0000","last_clean":"2023-12-26T23:09:46.600294+0000","last_became_active":"2023-12-26T23:09:46.593054+0000","last_became_peered":"2023-12-26T23:09:46.593054+0000","last_unstale":"2023-12-26T23:09:46.600294+0000","last_undegraded":"2023-12-26T23:09:46.600294+0000","last_fullsized":"2023-12-26T23:09:46.600294+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:41:08.550325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.593960+0000","last_change":"2023-12-26T23:09:46.586639+0000","last_active":"2023-12-26T23:09:46.593960+0000","last_peered":"2023-12-26T23:09:46.593960+0000","last_clean":"2023-12-26T23:09:46.593960+0000","last_became_active":"2023-12-26T23:09:46.586369+0000","last_became_peered":"2023-12-26T23:09:46.586369+0000","last_unstale":"2023-12-26T23:09:46.593960+0000","last_undegraded":"2023-12-26T23:09:46.593960+0000","last_fullsized":"2023-12-26T23:09:46.593960+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:14:21.007369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600500+0000","last_change":"2023-12-26T23:09:46.594599+0000","last_active":"2023-12-26T23:09:46.600500+0000","last_peered":"2023-12-26T23:09:46.600500+0000","last_clean":"2023-12-26T23:09:46.600500+0000","last_became_active":"2023-12-26T23:09:46.594310+0000","last_became_peered":"2023-12-26T23:09:46.594310+0000","last_unstale":"2023-12-26T23:09:46.600500+0000","last_undegraded":"2023-12-26T23:09:46.600500+0000","last_fullsized":"2023-12-26T23:09:46.600500+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:18:06.593586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606832+0000","last_change":"2023-12-26T23:09:46.600088+0000","last_active":"2023-12-26T23:09:46.606832+0000","last_peered":"2023-12-26T23:09:46.606832+0000","last_clean":"2023-12-26T23:09:46.606832+0000","last_became_active":"2023-12-26T23:09:46.599658+0000","last_became_peered":"2023-12-26T23:09:46.599658+0000","last_unstale":"2023-12-26T23:09:46.606832+0000","last_undegraded":"2023-12-26T23:09:46.606832+0000","last_fullsized":"2023-12-26T23:09:46.606832+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:18:50.623844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596672+0000","last_change":"2023-12-26T23:09:46.585342+0000","last_active":"2023-12-26T23:09:46.596672+0000","last_peered":"2023-12-26T23:09:46.596672+0000","last_clean":"2023-12-26T23:09:46.596672+0000","last_became_active":"2023-12-26T23:09:46.585080+0000","last_became_peered":"2023-12-26T23:09:46.585080+0000","last_unstale":"2023-12-26T23:09:46.596672+0000","last_undegraded":"2023-12-26T23:09:46.596672+0000","last_fullsized":"2023-12-26T23:09:46.596672+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:41:18.206867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607013+0000","last_change":"2023-12-26T23:09:46.602417+0000","last_active":"2023-12-26T23:09:46.607013+0000","last_peered":"2023-12-26T23:09:46.607013+0000","last_clean":"2023-12-26T23:09:46.607013+0000","last_became_active":"2023-12-26T23:09:46.602165+0000","last_became_peered":"2023-12-26T23:09:46.602165+0000","last_unstale":"2023-12-26T23:09:46.607013+0000","last_undegraded":"2023-12-26T23:09:46.607013+0000","last_fullsized":"2023-12-26T23:09:46.607013+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:38:01.154494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603248+0000","last_change":"2023-12-26T23:09:46.599267+0000","last_active":"2023-12-26T23:09:46.603248+0000","last_peered":"2023-12-26T23:09:46.603248+0000","last_clean":"2023-12-26T23:09:46.603248+0000","last_became_active":"2023-12-26T23:09:46.599008+0000","last_became_peered":"2023-12-26T23:09:46.599008+0000","last_unstale":"2023-12-26T23:09:46.603248+0000","last_undegraded":"2023-12-26T23:09:46.603248+0000","last_fullsized":"2023-12-26T23:09:46.603248+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:58:36.562969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604161+0000","last_change":"2023-12-26T23:09:46.599826+0000","last_active":"2023-12-26T23:09:46.604161+0000","last_peered":"2023-12-26T23:09:46.604161+0000","last_clean":"2023-12-26T23:09:46.604161+0000","last_became_active":"2023-12-26T23:09:46.598055+0000","last_became_peered":"2023-12-26T23:09:46.598055+0000","last_unstale":"2023-12-26T23:09:46.604161+0000","last_undegraded":"2023-12-26T23:09:46.604161+0000","last_fullsized":"2023-12-26T23:09:46.604161+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:21:26.253582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"192'4","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607536+0000","last_change":"2023-12-26T23:09:45.589924+0000","last_active":"2023-12-26T23:09:46.607536+0000","last_peered":"2023-12-26T23:09:46.607536+0000","last_clean":"2023-12-26T23:09:46.607536+0000","last_became_active":"2023-12-26T23:09:45.589660+0000","last_became_peered":"2023-12-26T23:09:45.589660+0000","last_unstale":"2023-12-26T23:09:46.607536+0000","last_undegraded":"2023-12-26T23:09:46.607536+0000","last_fullsized":"2023-12-26T23:09:46.607536+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:04:27.424684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599125+0000","last_change":"2023-12-26T23:09:46.595424+0000","last_active":"2023-12-26T23:09:46.599125+0000","last_peered":"2023-12-26T23:09:46.599125+0000","last_clean":"2023-12-26T23:09:46.599125+0000","last_became_active":"2023-12-26T23:09:46.595133+0000","last_became_peered":"2023-12-26T23:09:46.595133+0000","last_unstale":"2023-12-26T23:09:46.599125+0000","last_undegraded":"2023-12-26T23:09:46.599125+0000","last_fullsized":"2023-12-26T23:09:46.599125+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:47:13.668855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.588539+0000","last_change":"2023-12-26T23:09:45.582327+0000","last_active":"2023-12-26T23:09:46.588539+0000","last_peered":"2023-12-26T23:09:46.588539+0000","last_clean":"2023-12-26T23:09:46.588539+0000","last_became_active":"2023-12-26T23:09:45.582075+0000","last_became_peered":"2023-12-26T23:09:45.582075+0000","last_unstale":"2023-12-26T23:09:46.588539+0000","last_undegraded":"2023-12-26T23:09:46.588539+0000","last_fullsized":"2023-12-26T23:09:46.588539+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:43:03.184792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602984+0000","last_change":"2023-12-26T23:09:46.595260+0000","last_active":"2023-12-26T23:09:46.602984+0000","last_peered":"2023-12-26T23:09:46.602984+0000","last_clean":"2023-12-26T23:09:46.602984+0000","last_became_active":"2023-12-26T23:09:46.594948+0000","last_became_peered":"2023-12-26T23:09:46.594948+0000","last_unstale":"2023-12-26T23:09:46.602984+0000","last_undegraded":"2023-12-26T23:09:46.602984+0000","last_fullsized":"2023-12-26T23:09:46.602984+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:32:55.217157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.590140+0000","last_change":"2023-12-26T23:09:45.575120+0000","last_active":"2023-12-26T23:09:46.590140+0000","last_peered":"2023-12-26T23:09:46.590140+0000","last_clean":"2023-12-26T23:09:46.590140+0000","last_became_active":"2023-12-26T23:09:45.574842+0000","last_became_peered":"2023-12-26T23:09:45.574842+0000","last_unstale":"2023-12-26T23:09:46.590140+0000","last_undegraded":"2023-12-26T23:09:46.590140+0000","last_fullsized":"2023-12-26T23:09:46.590140+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:27:11.943098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602906+0000","last_change":"2023-12-26T23:09:46.598110+0000","last_active":"2023-12-26T23:09:46.602906+0000","last_peered":"2023-12-26T23:09:46.602906+0000","last_clean":"2023-12-26T23:09:46.602906+0000","last_became_active":"2023-12-26T23:09:46.597849+0000","last_became_peered":"2023-12-26T23:09:46.597849+0000","last_unstale":"2023-12-26T23:09:46.602906+0000","last_undegraded":"2023-12-26T23:09:46.602906+0000","last_fullsized":"2023-12-26T23:09:46.602906+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:35:21.934564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599731+0000","last_change":"2023-12-26T23:09:45.586785+0000","last_active":"2023-12-26T23:09:46.599731+0000","last_peered":"2023-12-26T23:09:46.599731+0000","last_clean":"2023-12-26T23:09:46.599731+0000","last_became_active":"2023-12-26T23:09:45.586349+0000","last_became_peered":"2023-12-26T23:09:45.586349+0000","last_unstale":"2023-12-26T23:09:46.599731+0000","last_undegraded":"2023-12-26T23:09:46.599731+0000","last_fullsized":"2023-12-26T23:09:46.599731+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:59:17.893604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603880+0000","last_change":"2023-12-26T23:09:46.593732+0000","last_active":"2023-12-26T23:09:46.603880+0000","last_peered":"2023-12-26T23:09:46.603880+0000","last_clean":"2023-12-26T23:09:46.603880+0000","last_became_active":"2023-12-26T23:09:46.593471+0000","last_became_peered":"2023-12-26T23:09:46.593471+0000","last_unstale":"2023-12-26T23:09:46.603880+0000","last_undegraded":"2023-12-26T23:09:46.603880+0000","last_fullsized":"2023-12-26T23:09:46.603880+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:18:03.529861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604309+0000","last_change":"2023-12-26T23:09:45.589451+0000","last_active":"2023-12-26T23:09:46.604309+0000","last_peered":"2023-12-26T23:09:46.604309+0000","last_clean":"2023-12-26T23:09:46.604309+0000","last_became_active":"2023-12-26T23:09:45.589180+0000","last_became_peered":"2023-12-26T23:09:45.589180+0000","last_unstale":"2023-12-26T23:09:46.604309+0000","last_undegraded":"2023-12-26T23:09:46.604309+0000","last_fullsized":"2023-12-26T23:09:46.604309+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:40:50.286051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603764+0000","last_change":"2023-12-26T23:09:46.596499+0000","last_active":"2023-12-26T23:09:46.603764+0000","last_peered":"2023-12-26T23:09:46.603764+0000","last_clean":"2023-12-26T23:09:46.603764+0000","last_became_active":"2023-12-26T23:09:46.596239+0000","last_became_peered":"2023-12-26T23:09:46.596239+0000","last_unstale":"2023-12-26T23:09:46.603764+0000","last_undegraded":"2023-12-26T23:09:46.603764+0000","last_fullsized":"2023-12-26T23:09:46.603764+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:09.617984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.601031+0000","last_change":"2023-12-26T23:09:45.586765+0000","last_active":"2023-12-26T23:09:46.601031+0000","last_peered":"2023-12-26T23:09:46.601031+0000","last_clean":"2023-12-26T23:09:46.601031+0000","last_became_active":"2023-12-26T23:09:45.586468+0000","last_became_peered":"2023-12-26T23:09:45.586468+0000","last_unstale":"2023-12-26T23:09:46.601031+0000","last_undegraded":"2023-12-26T23:09:46.601031+0000","last_fullsized":"2023-12-26T23:09:46.601031+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:51:08.741674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603864+0000","last_change":"2023-12-26T23:09:46.597661+0000","last_active":"2023-12-26T23:09:46.603864+0000","last_peered":"2023-12-26T23:09:46.603864+0000","last_clean":"2023-12-26T23:09:46.603864+0000","last_became_active":"2023-12-26T23:09:46.597380+0000","last_became_peered":"2023-12-26T23:09:46.597380+0000","last_unstale":"2023-12-26T23:09:46.603864+0000","last_undegraded":"2023-12-26T23:09:46.603864+0000","last_fullsized":"2023-12-26T23:09:46.603864+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:42.286153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.594120+0000","last_change":"2023-12-26T23:09:45.590794+0000","last_active":"2023-12-26T23:09:46.594120+0000","last_peered":"2023-12-26T23:09:46.594120+0000","last_clean":"2023-12-26T23:09:46.594120+0000","last_became_active":"2023-12-26T23:09:45.590380+0000","last_became_peered":"2023-12-26T23:09:45.590380+0000","last_unstale":"2023-12-26T23:09:46.594120+0000","last_undegraded":"2023-12-26T23:09:46.594120+0000","last_fullsized":"2023-12-26T23:09:46.594120+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:15:46.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.601012+0000","last_change":"2023-12-26T23:09:46.593072+0000","last_active":"2023-12-26T23:09:46.601012+0000","last_peered":"2023-12-26T23:09:46.601012+0000","last_clean":"2023-12-26T23:09:46.601012+0000","last_became_active":"2023-12-26T23:09:46.592763+0000","last_became_peered":"2023-12-26T23:09:46.592763+0000","last_unstale":"2023-12-26T23:09:46.601012+0000","last_undegraded":"2023-12-26T23:09:46.601012+0000","last_fullsized":"2023-12-26T23:09:46.601012+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:29:00.694839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.582093+0000","last_change":"2023-12-26T23:09:45.582292+0000","last_active":"2023-12-26T23:09:46.582093+0000","last_peered":"2023-12-26T23:09:46.582093+0000","last_clean":"2023-12-26T23:09:46.582093+0000","last_became_active":"2023-12-26T23:09:45.582031+0000","last_became_peered":"2023-12-26T23:09:45.582031+0000","last_unstale":"2023-12-26T23:09:46.582093+0000","last_undegraded":"2023-12-26T23:09:46.582093+0000","last_fullsized":"2023-12-26T23:09:46.582093+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:58:06.657883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.591506+0000","last_change":"2023-12-26T23:09:46.586759+0000","last_active":"2023-12-26T23:09:46.591506+0000","last_peered":"2023-12-26T23:09:46.591506+0000","last_clean":"2023-12-26T23:09:46.591506+0000","last_became_active":"2023-12-26T23:09:46.586480+0000","last_became_peered":"2023-12-26T23:09:46.586480+0000","last_unstale":"2023-12-26T23:09:46.591506+0000","last_undegraded":"2023-12-26T23:09:46.591506+0000","last_fullsized":"2023-12-26T23:09:46.591506+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:05.643738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.570073+0000","last_change":"2023-12-26T23:09:45.585069+0000","last_active":"2023-12-26T23:09:46.570073+0000","last_peered":"2023-12-26T23:09:46.570073+0000","last_clean":"2023-12-26T23:09:46.570073+0000","last_became_active":"2023-12-26T23:09:45.583718+0000","last_became_peered":"2023-12-26T23:09:45.583718+0000","last_unstale":"2023-12-26T23:09:46.570073+0000","last_undegraded":"2023-12-26T23:09:46.570073+0000","last_fullsized":"2023-12-26T23:09:46.570073+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:02:31.996982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.595522+0000","last_change":"2023-12-26T23:09:46.591348+0000","last_active":"2023-12-26T23:09:46.595522+0000","last_peered":"2023-12-26T23:09:46.595522+0000","last_clean":"2023-12-26T23:09:46.595522+0000","last_became_active":"2023-12-26T23:09:46.591002+0000","last_became_peered":"2023-12-26T23:09:46.591002+0000","last_unstale":"2023-12-26T23:09:46.595522+0000","last_undegraded":"2023-12-26T23:09:46.595522+0000","last_fullsized":"2023-12-26T23:09:46.595522+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:59:57.577667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"192'3","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602421+0000","last_change":"2023-12-26T23:09:45.579845+0000","last_active":"2023-12-26T23:09:46.602421+0000","last_peered":"2023-12-26T23:09:46.602421+0000","last_clean":"2023-12-26T23:09:46.602421+0000","last_became_active":"2023-12-26T23:09:45.579485+0000","last_became_peered":"2023-12-26T23:09:45.579485+0000","last_unstale":"2023-12-26T23:09:46.602421+0000","last_undegraded":"2023-12-26T23:09:46.602421+0000","last_fullsized":"2023-12-26T23:09:46.602421+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:56:08.086287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606826+0000","last_change":"2023-12-26T23:09:46.601064+0000","last_active":"2023-12-26T23:09:46.606826+0000","last_peered":"2023-12-26T23:09:46.606826+0000","last_clean":"2023-12-26T23:09:46.606826+0000","last_became_active":"2023-12-26T23:09:46.600489+0000","last_became_peered":"2023-12-26T23:09:46.600489+0000","last_unstale":"2023-12-26T23:09:46.606826+0000","last_undegraded":"2023-12-26T23:09:46.606826+0000","last_fullsized":"2023-12-26T23:09:46.606826+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:52:12.397718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"192'6","reported_seq":16,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607610+0000","last_change":"2023-12-26T23:09:45.576823+0000","last_active":"2023-12-26T23:09:46.607610+0000","last_peered":"2023-12-26T23:09:46.607610+0000","last_clean":"2023-12-26T23:09:46.607610+0000","last_became_active":"2023-12-26T23:09:45.576486+0000","last_became_peered":"2023-12-26T23:09:45.576486+0000","last_unstale":"2023-12-26T23:09:46.607610+0000","last_undegraded":"2023-12-26T23:09:46.607610+0000","last_fullsized":"2023-12-26T23:09:46.607610+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:25:59.254836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.595388+0000","last_change":"2023-12-26T23:09:46.589966+0000","last_active":"2023-12-26T23:09:46.595388+0000","last_peered":"2023-12-26T23:09:46.595388+0000","last_clean":"2023-12-26T23:09:46.595388+0000","last_became_active":"2023-12-26T23:09:46.589697+0000","last_became_peered":"2023-12-26T23:09:46.589697+0000","last_unstale":"2023-12-26T23:09:46.595388+0000","last_undegraded":"2023-12-26T23:09:46.595388+0000","last_fullsized":"2023-12-26T23:09:46.595388+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:17:12.820879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599056+0000","last_change":"2023-12-26T23:09:45.583454+0000","last_active":"2023-12-26T23:09:46.599056+0000","last_peered":"2023-12-26T23:09:46.599056+0000","last_clean":"2023-12-26T23:09:46.599056+0000","last_became_active":"2023-12-26T23:09:45.583195+0000","last_became_peered":"2023-12-26T23:09:45.583195+0000","last_unstale":"2023-12-26T23:09:46.599056+0000","last_undegraded":"2023-12-26T23:09:46.599056+0000","last_fullsized":"2023-12-26T23:09:46.599056+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:46:17.437641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606870+0000","last_change":"2023-12-26T23:09:46.600684+0000","last_active":"2023-12-26T23:09:46.606870+0000","last_peered":"2023-12-26T23:09:46.606870+0000","last_clean":"2023-12-26T23:09:46.606870+0000","last_became_active":"2023-12-26T23:09:46.600238+0000","last_became_peered":"2023-12-26T23:09:46.600238+0000","last_unstale":"2023-12-26T23:09:46.606870+0000","last_undegraded":"2023-12-26T23:09:46.606870+0000","last_fullsized":"2023-12-26T23:09:46.606870+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:04:29.108988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.570516+0000","last_change":"2023-12-26T23:09:45.580860+0000","last_active":"2023-12-26T23:09:46.570516+0000","last_peered":"2023-12-26T23:09:46.570516+0000","last_clean":"2023-12-26T23:09:46.570516+0000","last_became_active":"2023-12-26T23:09:45.579589+0000","last_became_peered":"2023-12-26T23:09:45.579589+0000","last_unstale":"2023-12-26T23:09:46.570516+0000","last_undegraded":"2023-12-26T23:09:46.570516+0000","last_fullsized":"2023-12-26T23:09:46.570516+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:10:10.474047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.586974+0000","last_change":"2023-12-26T23:09:46.582547+0000","last_active":"2023-12-26T23:09:46.586974+0000","last_peered":"2023-12-26T23:09:46.586974+0000","last_clean":"2023-12-26T23:09:46.586974+0000","last_became_active":"2023-12-26T23:09:46.581757+0000","last_became_peered":"2023-12-26T23:09:46.581757+0000","last_unstale":"2023-12-26T23:09:46.586974+0000","last_undegraded":"2023-12-26T23:09:46.586974+0000","last_fullsized":"2023-12-26T23:09:46.586974+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:42:01.311312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.582427+0000","last_change":"2023-12-26T23:09:45.588483+0000","last_active":"2023-12-26T23:09:46.582427+0000","last_peered":"2023-12-26T23:09:46.582427+0000","last_clean":"2023-12-26T23:09:46.582427+0000","last_became_active":"2023-12-26T23:09:45.586903+0000","last_became_peered":"2023-12-26T23:09:45.586903+0000","last_unstale":"2023-12-26T23:09:46.582427+0000","last_undegraded":"2023-12-26T23:09:46.582427+0000","last_fullsized":"2023-12-26T23:09:46.582427+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:39.956735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.597360+0000","last_change":"2023-12-26T23:09:46.587107+0000","last_active":"2023-12-26T23:09:46.597360+0000","last_peered":"2023-12-26T23:09:46.597360+0000","last_clean":"2023-12-26T23:09:46.597360+0000","last_became_active":"2023-12-26T23:09:46.586868+0000","last_became_peered":"2023-12-26T23:09:46.586868+0000","last_unstale":"2023-12-26T23:09:46.597360+0000","last_undegraded":"2023-12-26T23:09:46.597360+0000","last_fullsized":"2023-12-26T23:09:46.597360+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:32:47.514453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"192'8","reported_seq":18,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599708+0000","last_change":"2023-12-26T23:09:45.578748+0000","last_active":"2023-12-26T23:09:46.599708+0000","last_peered":"2023-12-26T23:09:46.599708+0000","last_clean":"2023-12-26T23:09:46.599708+0000","last_became_active":"2023-12-26T23:09:45.578489+0000","last_became_peered":"2023-12-26T23:09:45.578489+0000","last_unstale":"2023-12-26T23:09:46.599708+0000","last_undegraded":"2023-12-26T23:09:46.599708+0000","last_fullsized":"2023-12-26T23:09:46.599708+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:18:14.831411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.594256+0000","last_change":"2023-12-26T23:09:46.587699+0000","last_active":"2023-12-26T23:09:46.594256+0000","last_peered":"2023-12-26T23:09:46.594256+0000","last_clean":"2023-12-26T23:09:46.594256+0000","last_became_active":"2023-12-26T23:09:46.587415+0000","last_became_peered":"2023-12-26T23:09:46.587415+0000","last_unstale":"2023-12-26T23:09:46.594256+0000","last_undegraded":"2023-12-26T23:09:46.594256+0000","last_fullsized":"2023-12-26T23:09:46.594256+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:46:38.005388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.590103+0000","last_change":"2023-12-26T23:09:45.578820+0000","last_active":"2023-12-26T23:09:46.590103+0000","last_peered":"2023-12-26T23:09:46.590103+0000","last_clean":"2023-12-26T23:09:46.590103+0000","last_became_active":"2023-12-26T23:09:45.578559+0000","last_became_peered":"2023-12-26T23:09:45.578559+0000","last_unstale":"2023-12-26T23:09:46.590103+0000","last_undegraded":"2023-12-26T23:09:46.590103+0000","last_fullsized":"2023-12-26T23:09:46.590103+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:22:50.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604769+0000","last_change":"2023-12-26T23:09:46.602463+0000","last_active":"2023-12-26T23:09:46.604769+0000","last_peered":"2023-12-26T23:09:46.604769+0000","last_clean":"2023-12-26T23:09:46.604769+0000","last_became_active":"2023-12-26T23:09:46.601615+0000","last_became_peered":"2023-12-26T23:09:46.601615+0000","last_unstale":"2023-12-26T23:09:46.604769+0000","last_undegraded":"2023-12-26T23:09:46.604769+0000","last_fullsized":"2023-12-26T23:09:46.604769+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:22.512723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.588280+0000","last_change":"2023-12-26T23:09:45.586961+0000","last_active":"2023-12-26T23:09:46.588280+0000","last_peered":"2023-12-26T23:09:46.588280+0000","last_clean":"2023-12-26T23:09:46.588280+0000","last_became_active":"2023-12-26T23:09:45.586607+0000","last_became_peered":"2023-12-26T23:09:45.586607+0000","last_unstale":"2023-12-26T23:09:46.588280+0000","last_undegraded":"2023-12-26T23:09:46.588280+0000","last_fullsized":"2023-12-26T23:09:46.588280+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:04:56.455080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603829+0000","last_change":"2023-12-26T23:09:46.596716+0000","last_active":"2023-12-26T23:09:46.603829+0000","last_peered":"2023-12-26T23:09:46.603829+0000","last_clean":"2023-12-26T23:09:46.603829+0000","last_became_active":"2023-12-26T23:09:46.595878+0000","last_became_peered":"2023-12-26T23:09:46.595878+0000","last_unstale":"2023-12-26T23:09:46.603829+0000","last_undegraded":"2023-12-26T23:09:46.603829+0000","last_fullsized":"2023-12-26T23:09:46.603829+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:25:56.200695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.598162+0000","last_change":"2023-12-26T23:09:45.588685+0000","last_active":"2023-12-26T23:09:46.598162+0000","last_peered":"2023-12-26T23:09:46.598162+0000","last_clean":"2023-12-26T23:09:46.598162+0000","last_became_active":"2023-12-26T23:09:45.587100+0000","last_became_peered":"2023-12-26T23:09:45.587100+0000","last_unstale":"2023-12-26T23:09:46.598162+0000","last_undegraded":"2023-12-26T23:09:46.598162+0000","last_fullsized":"2023-12-26T23:09:46.598162+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:05:08.591870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607432+0000","last_change":"2023-12-26T23:09:46.604089+0000","last_active":"2023-12-26T23:09:46.607432+0000","last_peered":"2023-12-26T23:09:46.607432+0000","last_clean":"2023-12-26T23:09:46.607432+0000","last_became_active":"2023-12-26T23:09:46.601440+0000","last_became_peered":"2023-12-26T23:09:46.601440+0000","last_unstale":"2023-12-26T23:09:46.607432+0000","last_undegraded":"2023-12-26T23:09:46.607432+0000","last_fullsized":"2023-12-26T23:09:46.607432+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:22:44.998972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.576998+0000","last_change":"2023-12-26T23:09:45.585513+0000","last_active":"2023-12-26T23:09:46.576998+0000","last_peered":"2023-12-26T23:09:46.576998+0000","last_clean":"2023-12-26T23:09:46.576998+0000","last_became_active":"2023-12-26T23:09:45.584696+0000","last_became_peered":"2023-12-26T23:09:45.584696+0000","last_unstale":"2023-12-26T23:09:46.576998+0000","last_undegraded":"2023-12-26T23:09:46.576998+0000","last_fullsized":"2023-12-26T23:09:46.576998+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:52:28.177857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607095+0000","last_change":"2023-12-26T23:09:46.603095+0000","last_active":"2023-12-26T23:09:46.607095+0000","last_peered":"2023-12-26T23:09:46.607095+0000","last_clean":"2023-12-26T23:09:46.607095+0000","last_became_active":"2023-12-26T23:09:46.602847+0000","last_became_peered":"2023-12-26T23:09:46.602847+0000","last_unstale":"2023-12-26T23:09:46.607095+0000","last_undegraded":"2023-12-26T23:09:46.607095+0000","last_fullsized":"2023-12-26T23:09:46.607095+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:40:31.701871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603284+0000","last_change":"2023-12-26T23:09:45.585675+0000","last_active":"2023-12-26T23:09:46.603284+0000","last_peered":"2023-12-26T23:09:46.603284+0000","last_clean":"2023-12-26T23:09:46.603284+0000","last_became_active":"2023-12-26T23:09:45.585461+0000","last_became_peered":"2023-12-26T23:09:45.585461+0000","last_unstale":"2023-12-26T23:09:46.603284+0000","last_undegraded":"2023-12-26T23:09:46.603284+0000","last_fullsized":"2023-12-26T23:09:46.603284+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:51:59.410791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.607420+0000","last_change":"2023-12-26T23:09:46.603900+0000","last_active":"2023-12-26T23:09:46.607420+0000","last_peered":"2023-12-26T23:09:46.607420+0000","last_clean":"2023-12-26T23:09:46.607420+0000","last_became_active":"2023-12-26T23:09:46.603348+0000","last_became_peered":"2023-12-26T23:09:46.603348+0000","last_unstale":"2023-12-26T23:09:46.607420+0000","last_undegraded":"2023-12-26T23:09:46.607420+0000","last_fullsized":"2023-12-26T23:09:46.607420+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:18:28.979877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.576612+0000","last_change":"2023-12-26T23:09:45.574162+0000","last_active":"2023-12-26T23:09:46.576612+0000","last_peered":"2023-12-26T23:09:46.576612+0000","last_clean":"2023-12-26T23:09:46.576612+0000","last_became_active":"2023-12-26T23:09:45.573894+0000","last_became_peered":"2023-12-26T23:09:45.573894+0000","last_unstale":"2023-12-26T23:09:46.576612+0000","last_undegraded":"2023-12-26T23:09:46.576612+0000","last_fullsized":"2023-12-26T23:09:46.576612+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:03:26.471388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606898+0000","last_change":"2023-12-26T23:09:46.601902+0000","last_active":"2023-12-26T23:09:46.606898+0000","last_peered":"2023-12-26T23:09:46.606898+0000","last_clean":"2023-12-26T23:09:46.606898+0000","last_became_active":"2023-12-26T23:09:46.601540+0000","last_became_peered":"2023-12-26T23:09:46.601540+0000","last_unstale":"2023-12-26T23:09:46.606898+0000","last_undegraded":"2023-12-26T23:09:46.606898+0000","last_fullsized":"2023-12-26T23:09:46.606898+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:11:47.888770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.578037+0000","last_change":"2023-12-26T23:09:45.589363+0000","last_active":"2023-12-26T23:09:46.578037+0000","last_peered":"2023-12-26T23:09:46.578037+0000","last_clean":"2023-12-26T23:09:46.578037+0000","last_became_active":"2023-12-26T23:09:45.589107+0000","last_became_peered":"2023-12-26T23:09:45.589107+0000","last_unstale":"2023-12-26T23:09:46.578037+0000","last_undegraded":"2023-12-26T23:09:46.578037+0000","last_fullsized":"2023-12-26T23:09:46.578037+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:41:57.785145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600165+0000","last_change":"2023-12-26T23:09:46.594497+0000","last_active":"2023-12-26T23:09:46.600165+0000","last_peered":"2023-12-26T23:09:46.600165+0000","last_clean":"2023-12-26T23:09:46.600165+0000","last_became_active":"2023-12-26T23:09:46.594179+0000","last_became_peered":"2023-12-26T23:09:46.594179+0000","last_unstale":"2023-12-26T23:09:46.600165+0000","last_undegraded":"2023-12-26T23:09:46.600165+0000","last_fullsized":"2023-12-26T23:09:46.600165+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:05:17.387272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.576832+0000","last_change":"2023-12-26T23:09:45.584699+0000","last_active":"2023-12-26T23:09:46.576832+0000","last_peered":"2023-12-26T23:09:46.576832+0000","last_clean":"2023-12-26T23:09:46.576832+0000","last_became_active":"2023-12-26T23:09:45.584460+0000","last_became_peered":"2023-12-26T23:09:45.584460+0000","last_unstale":"2023-12-26T23:09:46.576832+0000","last_undegraded":"2023-12-26T23:09:46.576832+0000","last_fullsized":"2023-12-26T23:09:46.576832+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:29:54.385431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.597131+0000","last_change":"2023-12-26T23:09:46.592203+0000","last_active":"2023-12-26T23:09:46.597131+0000","last_peered":"2023-12-26T23:09:46.597131+0000","last_clean":"2023-12-26T23:09:46.597131+0000","last_became_active":"2023-12-26T23:09:46.590871+0000","last_became_peered":"2023-12-26T23:09:46.590871+0000","last_unstale":"2023-12-26T23:09:46.597131+0000","last_undegraded":"2023-12-26T23:09:46.597131+0000","last_fullsized":"2023-12-26T23:09:46.597131+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:41:48.083462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.596066+0000","last_change":"2023-12-26T23:09:45.577660+0000","last_active":"2023-12-26T23:09:46.596066+0000","last_peered":"2023-12-26T23:09:46.596066+0000","last_clean":"2023-12-26T23:09:46.596066+0000","last_became_active":"2023-12-26T23:09:45.577235+0000","last_became_peered":"2023-12-26T23:09:45.577235+0000","last_unstale":"2023-12-26T23:09:46.596066+0000","last_undegraded":"2023-12-26T23:09:46.596066+0000","last_fullsized":"2023-12-26T23:09:46.596066+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:27:36.878163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604790+0000","last_change":"2023-12-26T23:09:46.603013+0000","last_active":"2023-12-26T23:09:46.604790+0000","last_peered":"2023-12-26T23:09:46.604790+0000","last_clean":"2023-12-26T23:09:46.604790+0000","last_became_active":"2023-12-26T23:09:46.602749+0000","last_became_peered":"2023-12-26T23:09:46.602749+0000","last_unstale":"2023-12-26T23:09:46.604790+0000","last_undegraded":"2023-12-26T23:09:46.604790+0000","last_fullsized":"2023-12-26T23:09:46.604790+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:27:42.382215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.577040+0000","last_change":"2023-12-26T23:09:45.590876+0000","last_active":"2023-12-26T23:09:46.577040+0000","last_peered":"2023-12-26T23:09:46.577040+0000","last_clean":"2023-12-26T23:09:46.577040+0000","last_became_active":"2023-12-26T23:09:45.590610+0000","last_became_peered":"2023-12-26T23:09:45.590610+0000","last_unstale":"2023-12-26T23:09:46.577040+0000","last_undegraded":"2023-12-26T23:09:46.577040+0000","last_fullsized":"2023-12-26T23:09:46.577040+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:17.072154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603727+0000","last_change":"2023-12-26T23:09:46.596171+0000","last_active":"2023-12-26T23:09:46.603727+0000","last_peered":"2023-12-26T23:09:46.603727+0000","last_clean":"2023-12-26T23:09:46.603727+0000","last_became_active":"2023-12-26T23:09:46.593136+0000","last_became_peered":"2023-12-26T23:09:46.593136+0000","last_unstale":"2023-12-26T23:09:46.603727+0000","last_undegraded":"2023-12-26T23:09:46.603727+0000","last_fullsized":"2023-12-26T23:09:46.603727+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:32:55.217157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.595547+0000","last_change":"2023-12-26T23:09:45.589622+0000","last_active":"2023-12-26T23:09:46.595547+0000","last_peered":"2023-12-26T23:09:46.595547+0000","last_clean":"2023-12-26T23:09:46.595547+0000","last_became_active":"2023-12-26T23:09:45.589332+0000","last_became_peered":"2023-12-26T23:09:45.589332+0000","last_unstale":"2023-12-26T23:09:46.595547+0000","last_undegraded":"2023-12-26T23:09:46.595547+0000","last_fullsized":"2023-12-26T23:09:46.595547+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:17.394434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.600980+0000","last_change":"2023-12-26T23:09:46.592752+0000","last_active":"2023-12-26T23:09:46.600980+0000","last_peered":"2023-12-26T23:09:46.600980+0000","last_clean":"2023-12-26T23:09:46.600980+0000","last_became_active":"2023-12-26T23:09:46.592308+0000","last_became_peered":"2023-12-26T23:09:46.592308+0000","last_unstale":"2023-12-26T23:09:46.600980+0000","last_undegraded":"2023-12-26T23:09:46.600980+0000","last_fullsized":"2023-12-26T23:09:46.600980+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:34:04.189487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.584407+0000","last_change":"2023-12-26T23:09:45.586964+0000","last_active":"2023-12-26T23:09:46.584407+0000","last_peered":"2023-12-26T23:09:46.584407+0000","last_clean":"2023-12-26T23:09:46.584407+0000","last_became_active":"2023-12-26T23:09:45.586065+0000","last_became_peered":"2023-12-26T23:09:45.586065+0000","last_unstale":"2023-12-26T23:09:46.584407+0000","last_undegraded":"2023-12-26T23:09:46.584407+0000","last_fullsized":"2023-12-26T23:09:46.584407+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:13:26.740447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.602197+0000","last_change":"2023-12-26T23:09:46.599444+0000","last_active":"2023-12-26T23:09:46.602197+0000","last_peered":"2023-12-26T23:09:46.602197+0000","last_clean":"2023-12-26T23:09:46.602197+0000","last_became_active":"2023-12-26T23:09:46.598681+0000","last_became_peered":"2023-12-26T23:09:46.598681+0000","last_unstale":"2023-12-26T23:09:46.602197+0000","last_undegraded":"2023-12-26T23:09:46.602197+0000","last_fullsized":"2023-12-26T23:09:46.602197+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:59:20.100619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.574706+0000","last_change":"2023-12-26T23:09:45.586115+0000","last_active":"2023-12-26T23:09:46.574706+0000","last_peered":"2023-12-26T23:09:46.574706+0000","last_clean":"2023-12-26T23:09:46.574706+0000","last_became_active":"2023-12-26T23:09:45.585883+0000","last_became_peered":"2023-12-26T23:09:45.585883+0000","last_unstale":"2023-12-26T23:09:46.574706+0000","last_undegraded":"2023-12-26T23:09:46.574706+0000","last_fullsized":"2023-12-26T23:09:46.574706+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:59:03.590907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.606706+0000","last_change":"2023-12-26T23:09:46.599927+0000","last_active":"2023-12-26T23:09:46.606706+0000","last_peered":"2023-12-26T23:09:46.606706+0000","last_clean":"2023-12-26T23:09:46.606706+0000","last_became_active":"2023-12-26T23:09:46.599339+0000","last_became_peered":"2023-12-26T23:09:46.599339+0000","last_unstale":"2023-12-26T23:09:46.606706+0000","last_undegraded":"2023-12-26T23:09:46.606706+0000","last_fullsized":"2023-12-26T23:09:46.606706+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:15:09.620999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.582592+0000","last_change":"2023-12-26T23:09:45.588345+0000","last_active":"2023-12-26T23:09:46.582592+0000","last_peered":"2023-12-26T23:09:46.582592+0000","last_clean":"2023-12-26T23:09:46.582592+0000","last_became_active":"2023-12-26T23:09:45.587978+0000","last_became_peered":"2023-12-26T23:09:45.587978+0000","last_unstale":"2023-12-26T23:09:46.582592+0000","last_undegraded":"2023-12-26T23:09:46.582592+0000","last_fullsized":"2023-12-26T23:09:46.582592+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:20:17.363109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.603037+0000","last_change":"2023-12-26T23:09:46.599500+0000","last_active":"2023-12-26T23:09:46.603037+0000","last_peered":"2023-12-26T23:09:46.603037+0000","last_clean":"2023-12-26T23:09:46.603037+0000","last_became_active":"2023-12-26T23:09:46.599046+0000","last_became_peered":"2023-12-26T23:09:46.599046+0000","last_unstale":"2023-12-26T23:09:46.603037+0000","last_undegraded":"2023-12-26T23:09:46.603037+0000","last_fullsized":"2023-12-26T23:09:46.603037+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:07:11.137583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.594200+0000","last_change":"2023-12-26T23:09:45.583854+0000","last_active":"2023-12-26T23:09:46.594200+0000","last_peered":"2023-12-26T23:09:46.594200+0000","last_clean":"2023-12-26T23:09:46.594200+0000","last_became_active":"2023-12-26T23:09:45.583603+0000","last_became_peered":"2023-12-26T23:09:45.583603+0000","last_unstale":"2023-12-26T23:09:46.594200+0000","last_undegraded":"2023-12-26T23:09:46.594200+0000","last_fullsized":"2023-12-26T23:09:46.594200+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:48:39.313836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.599117+0000","last_change":"2023-12-26T23:09:46.595346+0000","last_active":"2023-12-26T23:09:46.599117+0000","last_peered":"2023-12-26T23:09:46.599117+0000","last_clean":"2023-12-26T23:09:46.599117+0000","last_became_active":"2023-12-26T23:09:46.595058+0000","last_became_peered":"2023-12-26T23:09:46.595058+0000","last_unstale":"2023-12-26T23:09:46.599117+0000","last_undegraded":"2023-12-26T23:09:46.599117+0000","last_fullsized":"2023-12-26T23:09:46.599117+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:31:03.025875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.593741+0000","last_change":"2023-12-26T23:09:45.590066+0000","last_active":"2023-12-26T23:09:46.593741+0000","last_peered":"2023-12-26T23:09:46.593741+0000","last_clean":"2023-12-26T23:09:46.593741+0000","last_became_active":"2023-12-26T23:09:45.588052+0000","last_became_peered":"2023-12-26T23:09:45.588052+0000","last_unstale":"2023-12-26T23:09:46.593741+0000","last_undegraded":"2023-12-26T23:09:46.593741+0000","last_fullsized":"2023-12-26T23:09:46.593741+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:34:22.541440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.604773+0000","last_change":"2023-12-26T23:09:46.602552+0000","last_active":"2023-12-26T23:09:46.604773+0000","last_peered":"2023-12-26T23:09:46.604773+0000","last_clean":"2023-12-26T23:09:46.604773+0000","last_became_active":"2023-12-26T23:09:46.602164+0000","last_became_peered":"2023-12-26T23:09:46.602164+0000","last_unstale":"2023-12-26T23:09:46.604773+0000","last_undegraded":"2023-12-26T23:09:46.604773+0000","last_fullsized":"2023-12-26T23:09:46.604773+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:45.557714+0000","last_clean_scrub_stamp":"2023-12-26T23:09:45.557714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:46:26.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.573735+0000","last_change":"2023-12-26T23:09:45.585289+0000","last_active":"2023-12-26T23:09:46.573735+0000","last_peered":"2023-12-26T23:09:46.573735+0000","last_clean":"2023-12-26T23:09:46.573735+0000","last_became_active":"2023-12-26T23:09:45.584881+0000","last_became_peered":"2023-12-26T23:09:45.584881+0000","last_unstale":"2023-12-26T23:09:46.573735+0000","last_undegraded":"2023-12-26T23:09:46.573735+0000","last_fullsized":"2023-12-26T23:09:46.573735+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:09:44.546024+0000","last_clean_scrub_stamp":"2023-12-26T23:09:44.546024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:33:39.956735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"182'326","reported_seq":1059,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-26T23:09:46.573212+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:09:46.573212+0000","last_peered":"2023-12-26T23:09:46.573212+0000","last_clean":"2023-12-26T23:09:46.573212+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:09:46.573212+0000","last_undegraded":"2023-12-26T23:09:46.573212+0000","last_fullsized":"2023-12-26T23:09:46.573212+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543251,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307816,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064024,"statfs":{"total":96636764160,"available":96321560576,"internally_reserved":0,"allocated":2392064,"data_stored":1201002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.54600000000000004,"5min":0.436,"15min":0.372},"max":{"1min":0.92200000000000004,"5min":0.96299999999999997,"15min":1.048},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.60099999999999998,"5min":0.45200000000000001,"15min":0.40899999999999997},"max":{"1min":0.90600000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60899999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.51100000000000001,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.91100000000000003,"5min":0.98599999999999999,"15min":0.995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":0.88400000000000001,"5min":0.96599999999999997,"15min":1.0449999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73999999999999999,"15min":0.74099999999999999},"min":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":1.0089999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.89800000000000002,"5min":0.94199999999999995,"15min":1.0049999999999999},"last":0.628}]},{"osd":3,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91400000000000003,"5min":1.0229999999999999,"15min":1.0760000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":0.97599999999999998,"15min":0.98199999999999998},"last":0.66300000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.71799999999999997,"5min":0.90400000000000003,"15min":0.91400000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57099999999999995,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.39600000000000002,"15min":0.38},"max":{"1min":0.753,"5min":0.83699999999999997,"15min":0.93000000000000005},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.42299999999999999,"15min":0.39400000000000002},"max":{"1min":0.73299999999999998,"5min":0.89100000000000001,"15min":0.98799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89000000000000001,"5min":0.998,"15min":1.042},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66700000000000004,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.45100000000000001},"max":{"1min":0.82499999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.56000000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.85599999999999998,"5min":0.96599999999999997,"15min":1.0129999999999999},"last":0.58999999999999997}]}]},{"osd":6,"up_from":15,"seq":64424510547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307820,"kb_used_data":2340,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064020,"statfs":{"total":96636764160,"available":96321556480,"internally_reserved":0,"allocated":2396160,"data_stored":1201036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.47099999999999997,"5min":0.438,"15min":0.32800000000000001},"max":{"1min":0.82299999999999995,"5min":0.96599999999999997,"15min":1.1080000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.61699999999999999,"5min":0.498,"15min":0.41799999999999998},"max":{"1min":0.92400000000000004,"5min":0.94299999999999995,"15min":1.0680000000000001},"last":0.72299999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.50600000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":1.046},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.746,"15min":0.754},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95999999999999996,"5min":0.98099999999999998,"15min":1.0580000000000001},"last":0.70599999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.61299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89100000000000001,"5min":0.98899999999999999,"15min":1.071},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.63,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94599999999999995,"5min":0.95299999999999996,"15min":1.0680000000000001},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75800000000000001,"15min":0.76300000000000001},"min":{"1min":0.58699999999999997,"5min":0.496,"15min":0.47799999999999998},"max":{"1min":0.92800000000000005,"5min":1.024,"15min":1.024},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":0.91700000000000004,"5min":1,"15min":1.032},"last":0.68899999999999995}]},{"osd":4,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.432,"15min":0.39400000000000002},"max":{"1min":0.90500000000000003,"5min":0.92600000000000005,"15min":1.1599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.48499999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92100000000000004,"5min":0.96399999999999997,"15min":1.111},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":1.0149999999999999,"15min":1.0860000000000001},"last":0.57499999999999996}]},{"osd":7,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.48199999999999998,"5min":0.443,"15min":0.42999999999999999},"max":{"1min":0.89700000000000002,"5min":1.03,"15min":1.1419999999999999},"last":0.626},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.497,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95999999999999996,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.59699999999999998}]}]},{"osd":1,"up_from":15,"seq":64424510547,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316624,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055216,"statfs":{"total":96636764160,"available":96312541184,"internally_reserved":0,"allocated":3022848,"data_stored":1799868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.32100000000000001,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.60099999999999998,"5min":0.68700000000000006,"15min":0.81399999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.79200000000000004},"last":0.496}]},{"osd":2,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.33800000000000002},"max":{"1min":0.61599999999999999,"5min":0.753,"15min":0.84099999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.35899999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.91200000000000003},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.41399999999999998},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.32700000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.93600000000000005},"last":0.52700000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.55100000000000005,"5min":0.53900000000000003,"15min":0.499},"max":{"1min":0.97399999999999998,"5min":1.103,"15min":1.103},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78800000000000003,"15min":0.77100000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.48599999999999999},"max":{"1min":1.0369999999999999,"5min":1.1739999999999999,"15min":1.2410000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81100000000000005,"15min":0.80800000000000005},"min":{"1min":0.63400000000000001,"5min":0.55000000000000004,"15min":0.50700000000000001},"max":{"1min":1.008,"5min":1.0669999999999999,"15min":1.22},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.1240000000000001,"15min":1.1839999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82399999999999995,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.52600000000000002},"max":{"1min":1.0589999999999999,"5min":1.087,"15min":1.1950000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85399999999999998,"15min":0.83899999999999997},"min":{"1min":0.625,"5min":0.625,"15min":0.54200000000000004},"max":{"1min":1.0089999999999999,"5min":1.2090000000000001,"15min":1.218},"last":0.79000000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.0469999999999999,"5min":1.056,"15min":1.125},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83099999999999996,"15min":0.82199999999999995},"min":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.55400000000000005},"max":{"1min":1.0229999999999999,"5min":1.1459999999999999,"15min":1.218},"last":0.88900000000000001}]}]},{"osd":0,"up_from":15,"seq":64424510547,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307832,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064008,"statfs":{"total":96636764160,"available":96321544192,"internally_reserved":0,"allocated":2408448,"data_stored":1207852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.45400000000000001},"min":{"1min":0.32600000000000001,"5min":0.30199999999999999,"15min":0.26600000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.499,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.313},"max":{"1min":0.63500000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.371}]},{"osd":2,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.505,"15min":0.51300000000000001},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.30599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54000000000000004,"15min":0.54700000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.38200000000000001},"max":{"1min":0.71899999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.434}]},{"osd":3,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.32800000000000001},"max":{"1min":0.67600000000000005,"5min":0.90200000000000002,"15min":1.3839999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.45300000000000001},"max":{"1min":0.83099999999999996,"5min":0.85599999999999998,"15min":1.012},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.51700000000000002},"max":{"1min":0.92600000000000005,"5min":1.0840000000000001,"15min":1.2250000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96799999999999997,"5min":1.0409999999999999,"15min":1.113},"last":0.64400000000000002}]},{"osd":5,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79900000000000004,"15min":0.82099999999999995},"min":{"1min":0.622,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.014,"5min":1.0920000000000001,"15min":1.157},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.84199999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97999999999999998,"5min":1.0740000000000001,"15min":1.282},"last":0.81799999999999995}]},{"osd":6,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78900000000000003,"15min":0.80800000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.90800000000000003,"5min":1.0529999999999999,"15min":1.2749999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.1299999999999999,"15min":1.3129999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Tue Dec 26 23:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.58699999999999997,"5min":0.57199999999999995,"15min":0.53900000000000003},"max":{"1min":0.95299999999999996,"5min":1.173,"15min":1.256},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98399999999999999,"5min":1.194,"15min":1.3360000000000001},"last":0.85799999999999998}]}]},{"osd":2,"up_from":14,"seq":60129543252,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311928,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059912,"statfs":{"total":96636764160,"available":96317349888,"internally_reserved":0,"allocated":2408448,"data_stored":1208336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48799999999999999,"15min":0.47299999999999998},"min":{"1min":0.38800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":0.72399999999999998,"15min":0.84599999999999997},"last":0.316},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.41099999999999998},"max":{"1min":0.65200000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.4219999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.373,"5min":0.36799999999999999,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":0.75600000000000001,"15min":0.91200000000000003},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.497},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.755},"last":1.9630000000000001}]},{"osd":3,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.627,"15min":0.622},"min":{"1min":0.498,"5min":0.45900000000000002,"15min":0.441},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.96299999999999997},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.83199999999999996,"5min":0.89600000000000002,"15min":1.056},"last":0.89300000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79800000000000004,"15min":0.79600000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.046,"5min":1.427,"15min":1.427},"last":1.8},{"interface":"front","average":{"1min":0.745,"5min":0.80100000000000005,"15min":0.79400000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.54000000000000004},"max":{"1min":0.93100000000000005,"5min":1.1719999999999999,"15min":1.2070000000000001},"last":1.222}]},{"osd":5,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.80600000000000005,"15min":0.79500000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.50800000000000001},"max":{"1min":0.92700000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.923},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82899999999999996,"15min":0.81200000000000006},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.98999999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.1599999999999999}]},{"osd":6,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81499999999999995,"15min":0.79500000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.52400000000000002},"max":{"1min":0.95799999999999996,"5min":1.272,"15min":1.272},"last":0.755},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.53300000000000003},"max":{"1min":1.004,"5min":1.1040000000000001,"15min":1.1830000000000001},"last":1.8859999999999999}]},{"osd":7,"last update":"Tue Dec 26 23:09:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84199999999999997,"15min":0.82099999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.56399999999999995},"max":{"1min":1.0249999999999999,"5min":1.161,"15min":1.1619999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.85199999999999998,"15min":0.84099999999999997},"min":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.55100000000000005},"max":{"1min":1.0780000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.841}]}]},{"osd":3,"up_from":15,"seq":64424510548,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303736,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068104,"statfs":{"total":96636764160,"available":96325738496,"internally_reserved":0,"allocated":2408448,"data_stored":1202560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.46500000000000002},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.246},"max":{"1min":0.66700000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.316},"max":{"1min":0.63300000000000001,"5min":0.74199999999999999,"15min":0.82299999999999995},"last":0.316}]},{"osd":1,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.67500000000000004,"15min":0.753},"last":0.372},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.307},"max":{"1min":0.67200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.47999999999999998,"5min":0.442,"15min":0.372},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66300000000000003,"15min":0.65800000000000003},"min":{"1min":0.52900000000000003,"5min":0.49099999999999999,"15min":0.41499999999999998},"max":{"1min":0.93500000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.159,"5min":1.177,"15min":1.177},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":1.0329999999999999,"15min":1.081},"last":0.501}]},{"osd":5,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56000000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0980000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.056,"5min":1.0860000000000001,"15min":1.1839999999999999},"last":0.48999999999999999}]},{"osd":6,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.52000000000000002},"max":{"1min":1.042,"5min":1.1359999999999999,"15min":1.147},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61099999999999999},"max":{"1min":1.0569999999999999,"5min":1.101,"15min":1.167},"last":0.64200000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.58399999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.024,"5min":1.141,"15min":1.214},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.83699999999999997,"15min":0.83299999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1180000000000001,"5min":1.21,"15min":1.21},"last":0.58599999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543251,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311916,"kb_used_data":2340,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059924,"statfs":{"total":96636764160,"available":96317362176,"internally_reserved":0,"allocated":2396160,"data_stored":1201036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.40799999999999997},"max":{"1min":0.872,"5min":0.90500000000000003,"15min":1.0489999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.52200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.83699999999999997,"5min":1.008,"15min":1.087},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.73899999999999999,"15min":0.74199999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79000000000000004,"5min":0.997,"15min":1.0600000000000001},"last":0.751},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":0.82499999999999996,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.53000000000000003,"5min":0.47199999999999998,"15min":0.46500000000000002},"max":{"1min":0.89600000000000002,"5min":0.99199999999999999,"15min":1.0109999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81599999999999995,"5min":0.91700000000000004,"15min":0.998},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73899999999999999,"15min":0.73399999999999999},"min":{"1min":0.56299999999999994,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76400000000000001,"15min":0.76700000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":0.89400000000000002,"5min":1.0469999999999999,"15min":1.0589999999999999},"last":0.623}]},{"osd":5,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.63400000000000001,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96299999999999997,"5min":0.96799999999999997,"15min":0.98999999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.48999999999999999,"5min":0.42099999999999999,"15min":0.38500000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.024},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71699999999999997,"15min":0.70199999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47099999999999997},"max":{"1min":0.96999999999999997,"5min":0.98099999999999998,"15min":0.98799999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.499,"5min":0.495,"15min":0.44500000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76800000000000002,"15min":0.752},"min":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.68899999999999995}]}]},{"osd":5,"up_from":15,"seq":64424510546,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300212,"kb_used_data":2924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071628,"statfs":{"total":96636764160,"available":96329347072,"internally_reserved":0,"allocated":2994176,"data_stored":1792014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.56399999999999995,"5min":0.45700000000000002,"15min":0.434},"max":{"1min":0.89100000000000001,"5min":0.96799999999999997,"15min":0.998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.57999999999999996,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.88500000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.57899999999999996,"5min":0.46300000000000002,"15min":0.41899999999999998},"max":{"1min":0.97699999999999998,"5min":0.98299999999999998,"15min":1.0580000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.74399999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":1.016,"15min":1.016},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.59199999999999997,"5min":0.46300000000000002,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":1.012,"15min":1.022},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.57799999999999996,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.88800000000000001,"5min":0.92800000000000005,"15min":0.99199999999999999},"last":0.66100000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76100000000000001,"15min":0.75600000000000001},"min":{"1min":0.59799999999999998,"5min":0.53700000000000003,"15min":0.498},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.60199999999999998,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.72799999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.622,"15min":0.59499999999999997},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.35899999999999999},"max":{"1min":0.82499999999999996,"5min":0.91600000000000004,"15min":1.0309999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":0.81200000000000006,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.78800000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.39900000000000002},"max":{"1min":0.90000000000000002,"5min":0.97799999999999998,"15min":1.042},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78800000000000003,"15min":0.76400000000000001},"min":{"1min":0.64300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.93100000000000005,"5min":1.0309999999999999,"15min":1.0820000000000001},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.71799999999999997,"15min":0.69899999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.75,"5min":0.996,"15min":0.996},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.75,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.48699999999999999,"15min":0.46300000000000002},"max":{"1min":0.96199999999999997,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:09:51.409 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T23:09:51.409 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T23:09:51.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.644+0000 7f6fc7fff700 1 Processor -- start 2023-12-26T23:09:51.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.645+0000 7f6fc7fff700 1 -- start start 2023-12-26T23:09:51.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.646+0000 7f6fc7fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6fc0159650 0x7f6fc00cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.646+0000 7f6fc7fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6fc0159a20 0x7f6fc00cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.646+0000 7f6fc7fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc00ccbc0 con 0x7f6fc00ced20 2023-12-26T23:09:51.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.646+0000 7f6fc7fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc00cbfb0 con 0x7f6fc0159a20 2023-12-26T23:09:51.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.646+0000 7f6fc7fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc00cc290 con 0x7f6fc0159650 2023-12-26T23:09:51.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.646+0000 7f6fc559a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6fc0159a20 0x7f6fc00cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.646+0000 7f6fc559a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6fc0159a20 0x7f6fc00cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52418/0 (socket says 172.21.15.131:52418) 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.646+0000 7f6fc559a700 1 -- 172.21.15.131:0/2630354249 learned_addr learned my addr 172.21.15.131:0/2630354249 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.646+0000 7f6fc5d9b700 1 --2- 172.21.15.131:0/2630354249 >> v2:172.21.15.190:3300/0 conn(0x7f6fc0159650 0x7f6fc00cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.647+0000 7f6fc559a700 1 -- 172.21.15.131:0/2630354249 >> v2:172.21.15.190:3300/0 conn(0x7f6fc0159650 msgr2=0x7f6fc00cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.647+0000 7f6fc559a700 1 --2- 172.21.15.131:0/2630354249 >> v2:172.21.15.190:3300/0 conn(0x7f6fc0159650 0x7f6fc00cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.647+0000 7f6fc559a700 1 -- 172.21.15.131:0/2630354249 >> v1:172.21.15.190:6789/0 conn(0x7f6fc00ced20 legacy=0x7f6fc015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.647+0000 7f6fc559a700 1 -- 172.21.15.131:0/2630354249 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00cc570 con 0x7f6fc0159a20 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.647+0000 7f6fc559a700 1 --2- 172.21.15.131:0/2630354249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159a20 0x7f6fc00cb9a0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0088ef0 tx=0x7f6fb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f7daec64afb8ce server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.647+0000 7f6fc4d99700 1 -- 172.21.15.131:0/2630354249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0009e70 con 0x7f6fc0159a20 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.647+0000 7f6fc4d99700 1 -- 172.21.15.131:0/2630354249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb000b860 con 0x7f6fc0159a20 2023-12-26T23:09:51.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.647+0000 7f6fc4d99700 1 -- 172.21.15.131:0/2630354249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000bc00 con 0x7f6fc0159a20 2023-12-26T23:09:51.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.648+0000 7f6fc7fff700 1 -- 172.21.15.131:0/2630354249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159a20 msgr2=0x7f6fc00cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.648+0000 7f6fc7fff700 1 --2- 172.21.15.131:0/2630354249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159a20 0x7f6fc00cb9a0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0088ef0 tx=0x7f6fb0009250 comp rx=0 tx=0).stop 2023-12-26T23:09:51.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.648+0000 7f6fc7fff700 1 -- 172.21.15.131:0/2630354249 shutdown_connections 2023-12-26T23:09:51.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.648+0000 7f6fc7fff700 1 --2- 172.21.15.131:0/2630354249 >> v2:172.21.15.190:3300/0 conn(0x7f6fc0159650 0x7f6fc00cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.648+0000 7f6fc7fff700 1 --2- 172.21.15.131:0/2630354249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159a20 0x7f6fc00cb9a0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.648+0000 7f6fc7fff700 1 -- 172.21.15.131:0/2630354249 >> 172.21.15.131:0/2630354249 conn(0x7f6fc0099980 msgr2=0x7f6fc006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:51.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.648+0000 7f6fc7fff700 1 -- 172.21.15.131:0/2630354249 shutdown_connections 2023-12-26T23:09:51.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.648+0000 7f6fc7fff700 1 -- 172.21.15.131:0/2630354249 wait complete. 2023-12-26T23:09:51.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.649+0000 7f6fc7fff700 1 Processor -- start 2023-12-26T23:09:51.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.649+0000 7f6fc7fff700 1 -- start start 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.650+0000 7f6fc7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc00ced20 0x7f6fc0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.650+0000 7f6fc7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159650 0x7f6fc015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.650+0000 7f6fc7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fc0159a20 0x7f6fc0146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.650+0000 7f6fc7fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc015afe0 con 0x7f6fc0159650 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.650+0000 7f6fc7fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc015b470 con 0x7f6fc00ced20 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.650+0000 7f6fc7fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc015a3d0 con 0x7f6fc0159a20 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc559a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159650 0x7f6fc015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc5d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc00ced20 0x7f6fc0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc559a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159650 0x7f6fc015bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52422/0 (socket says 172.21.15.131:52422) 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc559a700 1 -- 172.21.15.131:0/1054545709 learned_addr learned my addr 172.21.15.131:0/1054545709 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc659c700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fc0159a20 0x7f6fc0146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc559a700 1 -- 172.21.15.131:0/1054545709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fc0159a20 msgr2=0x7f6fc0146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc559a700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fc0159a20 0x7f6fc0146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc559a700 1 -- 172.21.15.131:0/1054545709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc00ced20 msgr2=0x7f6fc0159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc559a700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc00ced20 0x7f6fc0159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc559a700 1 -- 172.21.15.131:0/1054545709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb00094f0 con 0x7f6fc0159650 2023-12-26T23:09:51.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fc559a700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159650 0x7f6fc015bc10 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00d1690 tx=0x7f6fb0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:51.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.651+0000 7f6fb5ffb700 1 -- 172.21.15.131:0/1054545709 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000b9d0 con 0x7f6fc0159650 2023-12-26T23:09:51.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.652+0000 7f6fb5ffb700 1 -- 172.21.15.131:0/1054545709 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb000bb30 con 0x7f6fc0159650 2023-12-26T23:09:51.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.652+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc015a6b0 con 0x7f6fc0159650 2023-12-26T23:09:51.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.652+0000 7f6fb5ffb700 1 -- 172.21.15.131:0/1054545709 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0025670 con 0x7f6fc0159650 2023-12-26T23:09:51.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.652+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc015ac70 con 0x7f6fc0159650 2023-12-26T23:09:51.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.653+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc00657e0 con 0x7f6fc0159650 2023-12-26T23:09:51.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.653+0000 7f6fb5ffb700 1 -- 172.21.15.131:0/1054545709 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6fb0023070 con 0x7f6fc0159650 2023-12-26T23:09:51.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.654+0000 7f6fb5ffb700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fa4066f30 0x7f6fa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:51.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.654+0000 7f6fb5ffb700 1 -- 172.21.15.131:0/1054545709 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f6fb0085ba0 con 0x7f6fc0159650 2023-12-26T23:09:51.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.655+0000 7f6fc5d9b700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fa4066f30 0x7f6fa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:51.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.655+0000 7f6fc5d9b700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fa4066f30 0x7f6fa40693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009da0 tx=0x7f6fbc009120 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:51.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.658+0000 7f6fb5ffb700 1 -- 172.21.15.131:0/1054545709 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fb0052c60 con 0x7f6fc0159650 2023-12-26T23:09:51.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.924+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6fc00cc850 con 0x7f6fc0159650 2023-12-26T23:09:51.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.929+0000 7f6fb5ffb700 1 -- 172.21.15.131:0/1054545709 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v49) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6fb0056240 con 0x7f6fc0159650 2023-12-26T23:09:51.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fa4066f30 msgr2=0x7f6fa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fa4066f30 0x7f6fa40693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009da0 tx=0x7f6fbc009120 comp rx=0 tx=0).stop 2023-12-26T23:09:51.931 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159650 msgr2=0x7f6fc015bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159650 0x7f6fc015bc10 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00d1690 tx=0x7f6fb0017000 comp rx=0 tx=0).stop 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 shutdown_connections 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fa4066f30 0x7f6fa40693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fc0159a20 0x7f6fc0146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fc00ced20 0x7f6fc0159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 --2- 172.21.15.131:0/1054545709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc0159650 0x7f6fc015bc10 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 >> 172.21.15.131:0/1054545709 conn(0x7f6fc0099980 msgr2=0x7f6fc0158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 shutdown_connections 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:51.931+0000 7f6fc7fff700 1 -- 172.21.15.131:0/1054545709 wait complete. 2023-12-26T23:09:51.932 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T23:09:51.949 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:09:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.139+0000 7fcd279ff700 1 Processor -- start 2023-12-26T23:09:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.139+0000 7fcd279ff700 1 -- start start 2023-12-26T23:09:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.140+0000 7fcd279ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 0x7fcd200b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.140+0000 7fcd279ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd20069d10 0x7fcd200b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.140+0000 7fcd279ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd200caa10 con 0x7fcd200c8820 2023-12-26T23:09:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.140+0000 7fcd279ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd20157a80 con 0x7fcd20069d10 2023-12-26T23:09:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.140+0000 7fcd279ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200cac50 con 0x7fcd200c8bf0 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.140+0000 7fcd2579b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcd200c8820 0x7fcd200ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55116/0 (socket says 172.21.15.131:55116) 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.140+0000 7fcd2579b700 1 -- 172.21.15.131:0/3236191883 learned_addr learned my addr 172.21.15.131:0/3236191883 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.140+0000 7fcd25f9c700 1 --2- 172.21.15.131:0/3236191883 >> v2:172.21.15.131:3300/0 conn(0x7fcd20069d10 0x7fcd200b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.140+0000 7fcd24f9a700 1 --2- 172.21.15.131:0/3236191883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 0x7fcd200b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd17fff700 1 -- 172.21.15.131:0/3236191883 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706673900 0 0) 0x7fcd1c001110 con 0x7fcd200c8820 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd17fff700 1 -- 172.21.15.131:0/3236191883 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd0c003570 con 0x7fcd200c8820 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd25f9c700 1 -- 172.21.15.131:0/3236191883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 msgr2=0x7fcd200b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd25f9c700 1 --2- 172.21.15.131:0/3236191883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 0x7fcd200b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd25f9c700 1 -- 172.21.15.131:0/3236191883 >> v1:172.21.15.131:6789/0 conn(0x7fcd200c8820 legacy=0x7fcd200ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd25f9c700 1 -- 172.21.15.131:0/3236191883 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd200caf30 con 0x7fcd20069d10 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd25f9c700 1 --2- 172.21.15.131:0/3236191883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd20069d10 0x7fcd200b1820 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009e60 tx=0x7fcd18009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448b45d194fdaff7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd17fff700 1 -- 172.21.15.131:0/3236191883 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1800e070 con 0x7fcd20069d10 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd17fff700 1 -- 172.21.15.131:0/3236191883 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1800bab0 con 0x7fcd20069d10 2023-12-26T23:09:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.141+0000 7fcd17fff700 1 -- 172.21.15.131:0/3236191883 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1800be70 con 0x7fcd20069d10 2023-12-26T23:09:52.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.142+0000 7fcd279ff700 1 -- 172.21.15.131:0/3236191883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd20069d10 msgr2=0x7fcd200b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:52.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.142+0000 7fcd279ff700 1 --2- 172.21.15.131:0/3236191883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd20069d10 0x7fcd200b1820 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009e60 tx=0x7fcd18009780 comp rx=0 tx=0).stop 2023-12-26T23:09:52.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.142+0000 7fcd279ff700 1 -- 172.21.15.131:0/3236191883 shutdown_connections 2023-12-26T23:09:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.142+0000 7fcd279ff700 1 --2- 172.21.15.131:0/3236191883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd20069d10 0x7fcd200b1820 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.142+0000 7fcd279ff700 1 --2- 172.21.15.131:0/3236191883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 0x7fcd200b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.142+0000 7fcd279ff700 1 -- 172.21.15.131:0/3236191883 >> 172.21.15.131:0/3236191883 conn(0x7fcd20068040 msgr2=0x7fcd20069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.142+0000 7fcd279ff700 1 -- 172.21.15.131:0/3236191883 shutdown_connections 2023-12-26T23:09:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.142+0000 7fcd279ff700 1 -- 172.21.15.131:0/3236191883 wait complete. 2023-12-26T23:09:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.144+0000 7fcd279ff700 1 Processor -- start 2023-12-26T23:09:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.144+0000 7fcd279ff700 1 -- start start 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd279ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd20069d10 0x7fcd20168c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd279ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd200c8820 0x7fcd20159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd279ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 0x7fcd2015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd279ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20169500 con 0x7fcd200c8820 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd279ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201691d0 con 0x7fcd20069d10 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd279ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2015e0e0 con 0x7fcd200c8bf0 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd24f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd200c8820 0x7fcd20159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd24f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd200c8820 0x7fcd20159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52448/0 (socket says 172.21.15.131:52448) 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd24f9a700 1 -- 172.21.15.131:0/798654339 learned_addr learned my addr 172.21.15.131:0/798654339 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd2579b700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd20069d10 0x7fcd20168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd25f9c700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 0x7fcd2015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd24f9a700 1 -- 172.21.15.131:0/798654339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 msgr2=0x7fcd2015a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd24f9a700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 0x7fcd2015a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd24f9a700 1 -- 172.21.15.131:0/798654339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd20069d10 msgr2=0x7fcd20168c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd24f9a700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd20069d10 0x7fcd20168c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.145+0000 7fcd24f9a700 1 -- 172.21.15.131:0/798654339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd18009a20 con 0x7fcd200c8820 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.146+0000 7fcd24f9a700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd200c8820 0x7fcd20159f80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ec80 tx=0x7fcd100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.146+0000 7fcd157fa700 1 -- 172.21.15.131:0/798654339 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000c9d0 con 0x7fcd200c8820 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.146+0000 7fcd157fa700 1 -- 172.21.15.131:0/798654339 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000cb30 con 0x7fcd200c8820 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.146+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2015e3c0 con 0x7fcd200c8820 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.146+0000 7fcd157fa700 1 -- 172.21.15.131:0/798654339 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10013470 con 0x7fcd200c8820 2023-12-26T23:09:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.146+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2015e980 con 0x7fcd200c8820 2023-12-26T23:09:52.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.147+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd200657e0 con 0x7fcd200c8820 2023-12-26T23:09:52.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.148+0000 7fcd157fa700 1 -- 172.21.15.131:0/798654339 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcd10013820 con 0x7fcd200c8820 2023-12-26T23:09:52.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.149+0000 7fcd157fa700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcd04066f30 0x7fcd040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:52.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.149+0000 7fcd157fa700 1 -- 172.21.15.131:0/798654339 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fcd10084ab0 con 0x7fcd200c8820 2023-12-26T23:09:52.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.149+0000 7fcd2579b700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcd04066f30 0x7fcd040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:52.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.150+0000 7fcd2579b700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcd04066f30 0x7fcd040693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c001270 tx=0x7fcd1c009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:52.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.153+0000 7fcd157fa700 1 -- 172.21.15.131:0/798654339 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd10051b30 con 0x7fcd200c8820 2023-12-26T23:09:52.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.392+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd20169640 con 0x7fcd200c8820 2023-12-26T23:09:52.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.392+0000 7fcd157fa700 1 -- 172.21.15.131:0/798654339 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4613 (secure 0 0 0) 0x7fcd10055150 con 0x7fcd200c8820 2023-12-26T23:09:52.393 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:09:52.393 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2070005366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2070005366},{"type":"v1","addr":"172.21.15.190:6835","nonce":2070005366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3800715315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3800715315},{"type":"v1","addr":"172.21.15.190:6834","nonce":3800715315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9645,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3865566903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3865566903},{"type":"v1","addr":"172.21.15.131:6833","nonce":3865566903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:09:46.045975+0000","modified":"2023-12-26T23:09:49.597663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9666":{"gid":9666,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2967199312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2967199312},{"type":"v1","addr":"172.21.15.131:6835","nonce":2967199312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-12-26T23:09:52.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcd04066f30 msgr2=0x7fcd040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:52.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcd04066f30 0x7fcd040693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c001270 tx=0x7fcd1c009040 comp rx=0 tx=0).stop 2023-12-26T23:09:52.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd200c8820 msgr2=0x7fcd20159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:52.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd200c8820 0x7fcd20159f80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ec80 tx=0x7fcd100091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 shutdown_connections 2023-12-26T23:09:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcd04066f30 0x7fcd040693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200c8bf0 0x7fcd2015a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd20069d10 0x7fcd20168c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 --2- 172.21.15.131:0/798654339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd200c8820 0x7fcd20159f80 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 >> 172.21.15.131:0/798654339 conn(0x7fcd20068040 msgr2=0x7fcd200b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 shutdown_connections 2023-12-26T23:09:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:52.395+0000 7fcd279ff700 1 -- 172.21.15.131:0/798654339 wait complete. 2023-12-26T23:09:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 90 2023-12-26T23:09:52.817 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T23:09:46.045975+0000', 'modified': '2023-12-26T23:09:49.597663+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9666}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9666': {'gid': 9666, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6835/2967199312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 2967199312}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 2967199312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T23:09:52.817 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T23:09:52.817 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-26T23:09:53.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.009+0000 7fdf87bff700 1 Processor -- start 2023-12-26T23:09:53.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.010+0000 7fdf87bff700 1 -- start start 2023-12-26T23:09:53.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.011+0000 7fdf87bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf800cac30 0x7fdf80155be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.011+0000 7fdf87bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf800c8bf0 con 0x7fdf800b07f0 2023-12-26T23:09:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.011+0000 7fdf87bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf800c8e60 con 0x7fdf80159920 2023-12-26T23:09:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.011+0000 7fdf87bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800c9140 con 0x7fdf800cac30 2023-12-26T23:09:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.011+0000 7fdf8519a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdf800b07f0 0x7fdf801554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55130/0 (socket says 172.21.15.131:55130) 2023-12-26T23:09:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.011+0000 7fdf8519a700 1 -- 172.21.15.131:0/61347231 learned_addr learned my addr 172.21.15.131:0/61347231 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.011+0000 7fdf8619c700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf800cac30 0x7fdf80155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640668726 0 0) 0x7fdf800c8bf0 con 0x7fdf800b07f0 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf6c004680 con 0x7fdf800b07f0 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289697743 0 0) 0x7fdf800c8e60 con 0x7fdf80159920 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf800c8bf0 con 0x7fdf80159920 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4188638950 0 0) 0x7fdf6c004680 con 0x7fdf800b07f0 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf800cac30 msgr2=0x7fdf80155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf800cac30 0x7fdf80155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 >> v1:172.21.15.190:6789/0 conn(0x7fdf80159920 legacy=0x7fdf800b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800c9420 con 0x7fdf800b07f0 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fdf70001470 con 0x7fdf800b07f0 2023-12-26T23:09:53.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf8619c700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf800cac30 0x7fdf80155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 >> v1:172.21.15.131:6789/0 conn(0x7fdf800b07f0 legacy=0x7fdf801554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c004d30 0x7fdf6c005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf800cac30 0x7fdf6c005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c005e60 0x7fdf6c0062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800c9140 con 0x7fdf6c004d30 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c00cc80 con 0x7fdf6c005e60 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c003510 con 0x7fdf800cac30 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf8599b700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c004d30 0x7fdf6c005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf8519a700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf800cac30 0x7fdf6c005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.012+0000 7fdf8619c700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c005e60 0x7fdf6c0062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.013+0000 7fdf8599b700 1 -- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf800cac30 msgr2=0x7fdf6c005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.013+0000 7fdf8599b700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf800cac30 0x7fdf6c005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.013+0000 7fdf8599b700 1 -- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c005e60 msgr2=0x7fdf6c0062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.013+0000 7fdf8599b700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c005e60 0x7fdf6c0062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.013+0000 7fdf8599b700 1 -- 172.21.15.131:0/61347231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009a20 con 0x7fdf6c004d30 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.013+0000 7fdf8599b700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c004d30 0x7fdf6c005120 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c001a10 tx=0x7fdf7c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444f331d460dea0c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.013+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00c5d0 con 0x7fdf6c004d30 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.013+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c00c730 con 0x7fdf6c004d30 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.013+0000 7fdf84999700 1 -- 172.21.15.131:0/61347231 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00cad0 con 0x7fdf6c004d30 2023-12-26T23:09:53.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.014+0000 7fdf87bff700 1 -- 172.21.15.131:0/61347231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c004d30 msgr2=0x7fdf6c005120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.014+0000 7fdf87bff700 1 --2- 172.21.15.131:0/61347231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c004d30 0x7fdf6c005120 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c001a10 tx=0x7fdf7c007d30 comp rx=0 tx=0).stop 2023-12-26T23:09:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.014+0000 7fdf8619c700 1 -- 172.21.15.131:0/61347231 reap_dead start 2023-12-26T23:09:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.014+0000 7fdf87bff700 1 -- 172.21.15.131:0/61347231 shutdown_connections 2023-12-26T23:09:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.014+0000 7fdf87bff700 1 -- 172.21.15.131:0/61347231 >> 172.21.15.131:0/61347231 conn(0x7fdf80068040 msgr2=0x7fdf80069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.014+0000 7fdf87bff700 1 -- 172.21.15.131:0/61347231 shutdown_connections 2023-12-26T23:09:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.014+0000 7fdf87bff700 1 -- 172.21.15.131:0/61347231 wait complete. 2023-12-26T23:09:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.015+0000 7fdf87bff700 1 Processor -- start 2023-12-26T23:09:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.015+0000 7fdf87bff700 1 -- start start 2023-12-26T23:09:53.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.016+0000 7fdf87bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c004d30 0x7fdf80158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.016+0000 7fdf87bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf80149930 0x7fdf8014d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.016+0000 7fdf87bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf800c6ab0 0x7fdf800c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.016+0000 7fdf87bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80159210 con 0x7fdf800c6ab0 2023-12-26T23:09:53.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.016+0000 7fdf87bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800c6010 con 0x7fdf6c004d30 2023-12-26T23:09:53.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.016+0000 7fdf87bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80158f30 con 0x7fdf80149930 2023-12-26T23:09:53.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf800c6ab0 0x7fdf800c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf800c6ab0 0x7fdf800c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52470/0 (socket says 172.21.15.131:52470) 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8619c700 1 -- 172.21.15.131:0/3199584870 learned_addr learned my addr 172.21.15.131:0/3199584870 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8599b700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c004d30 0x7fdf80158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8519a700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf80149930 0x7fdf8014d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8619c700 1 -- 172.21.15.131:0/3199584870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf80149930 msgr2=0x7fdf8014d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8619c700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf80149930 0x7fdf8014d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8619c700 1 -- 172.21.15.131:0/3199584870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c004d30 msgr2=0x7fdf80158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8619c700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c004d30 0x7fdf80158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8619c700 1 -- 172.21.15.131:0/3199584870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800c62b0 con 0x7fdf800c6ab0 2023-12-26T23:09:53.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8599b700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c004d30 0x7fdf80158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf8619c700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf800c6ab0 0x7fdf800c6f20 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fdf80064440 tx=0x7fdf7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.017+0000 7fdf75ffb700 1 -- 172.21.15.131:0/3199584870 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800e040 con 0x7fdf800c6ab0 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.018+0000 7fdf75ffb700 1 -- 172.21.15.131:0/3199584870 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7800f040 con 0x7fdf800c6ab0 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.018+0000 7fdf75ffb700 1 -- 172.21.15.131:0/3199584870 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78015430 con 0x7fdf800c6ab0 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.018+0000 7fdf87bff700 1 -- 172.21.15.131:0/3199584870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf800c6590 con 0x7fdf800c6ab0 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.018+0000 7fdf87bff700 1 -- 172.21.15.131:0/3199584870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf80149da0 con 0x7fdf800c6ab0 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.020+0000 7fdf75ffb700 1 -- 172.21.15.131:0/3199584870 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdf78015820 con 0x7fdf800c6ab0 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.020+0000 7fdf75ffb700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdf64066f30 0x7fdf640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.021+0000 7fdf75ffb700 1 -- 172.21.15.131:0/3199584870 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fdf78085330 con 0x7fdf800c6ab0 2023-12-26T23:09:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.021+0000 7fdf8599b700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdf64066f30 0x7fdf640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.021+0000 7fdf8599b700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdf64066f30 0x7fdf640693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c008fd0 tx=0x7fdf7c015000 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.022+0000 7fdf963e2b80 1 Processor -- start 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.023+0000 7fdf963e2b80 1 -- start start 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.023+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 0x560308acd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.023+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308ac8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.023+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308ae9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.023+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603089fb3e0 con 0x560308ac9170 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.023+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603089f8be0 con 0x560308accfa0 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.023+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603089f49d0 con 0x560308acd8b0 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308ae9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8619c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308ae9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52476/0 (socket says 172.21.15.131:52476) 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8619c700 1 -- 172.21.15.131:0/751131454 learned_addr learned my addr 172.21.15.131:0/751131454 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8519a700 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308ac8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8599b700 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 0x560308acd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8619c700 1 -- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 msgr2=0x560308ac8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8619c700 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308ac8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8619c700 1 -- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 msgr2=0x560308acd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8619c700 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 0x560308acd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8619c700 1 -- 172.21.15.131:0/751131454 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560308888360 con 0x560308ac9170 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8599b700 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 0x560308acd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8519a700 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308ac8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf8619c700 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308ae9360 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fdf780835c0 tx=0x7fdf78083520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8f263835f16dd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf62ffd700 1 -- 172.21.15.131:0/751131454 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78052310 con 0x560308ac9170 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf62ffd700 1 -- 172.21.15.131:0/751131454 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf78059030 con 0x560308ac9170 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.024+0000 7fdf62ffd700 1 -- 172.21.15.131:0/751131454 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78055fd0 con 0x560308ac9170 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.025+0000 7fdf963e2b80 1 -- 172.21.15.131:0/751131454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 msgr2=0x560308ae9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.025+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308ae9360 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fdf780835c0 tx=0x7fdf78083520 comp rx=0 tx=0).stop 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.025+0000 7fdf963e2b80 1 -- 172.21.15.131:0/751131454 shutdown_connections 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.025+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308ae9360 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.025+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 0x560308acd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.025+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/751131454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308ac8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.025+0000 7fdf963e2b80 1 -- 172.21.15.131:0/751131454 >> 172.21.15.131:0/751131454 conn(0x560308aa3440 msgr2=0x560308aeedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:53.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.025+0000 7fdf963e2b80 1 -- 172.21.15.131:0/751131454 shutdown_connections 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.025+0000 7fdf963e2b80 1 -- 172.21.15.131:0/751131454 wait complete. 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.026+0000 7fdf963e2b80 1 -- start start 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.026+0000 7fdf963e2b80 1 Processor -- start 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308afabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 0x560308afe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308afe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603089a11b0 con 0x560308ac9170 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603089a0e50 con 0x560308accfa0 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603088ede40 con 0x560308acd8b0 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf963e2b80 10 client.0 _ll_register_callbacks cb 0x5603088f07b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T23:09:53.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf8599b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308afabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf8599b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308afabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52484/0 (socket says 172.21.15.131:52484) 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf8599b700 1 -- 172.21.15.131:0/1095315196 learned_addr learned my addr 172.21.15.131:0/1095315196 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf8619c700 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308afe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.027+0000 7fdf8519a700 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 0x560308afe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf8599b700 1 -- 172.21.15.131:0/1095315196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 msgr2=0x560308afe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf8599b700 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308afe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf8599b700 1 -- 172.21.15.131:0/1095315196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 msgr2=0x560308afe1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf8599b700 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 0x560308afe1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf8599b700 1 -- 172.21.15.131:0/1095315196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf80149da0 con 0x560308ac9170 2023-12-26T23:09:53.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf8599b700 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308afabc0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00cf40 tx=0x7fdf7c021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c6609210c3472c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1095315196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c012780 con 0x560308ac9170 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1095315196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fdf7c025070 con 0x560308ac9170 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.028+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1095315196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560308a75eb0 con 0x560308ac9170 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1095315196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf7c012af0 con 0x560308ac9170 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf963e2b80 10 client.9786 fetch_fsmap learned FSMap version 90 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1095315196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x560308834b10 con 0x560308ac9170 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1095315196 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560308a75eb0 con 0x560308ac9170 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1095315196 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdf7c031020 con 0x560308ac9170 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf963e2b80 10 client.9786 fetch_fsmap finished waiting for FSMap version 90 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf963e2b80 10 client.9786 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-26T23:09:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf963e2b80 20 client.9786 populate_metadata read hostname 'smithi131' 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] conn(0x560308afd8a0 0x560308ac9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf963e2b80 4 client.9786 mds_command: new command op to 9666 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1095315196 --> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603089cb880 con 0x560308afd8a0 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.029+0000 7fdf8519a700 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] conn(0x560308afd8a0 0x560308ac9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.030+0000 7fdf8519a700 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] conn(0x560308afd8a0 0x560308ac9df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1bbc50dd649e18 server_cookie=ac026d30f69660cb in_seq=0 out_seq=0 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.030+0000 7fdf5bfff700 10 client.9786 ms_handle_connect on v2:172.21.15.131:6834/2967199312 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.030+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1095315196 <== mds.0 v2:172.21.15.131:6834/2967199312 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5603089cb880 con 0x560308afd8a0 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.030+0000 7fdf5bfff700 10 client.9786 handle_command_reply: tid=0 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 client.9786 shutdown 2023-12-26T23:09:53.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1095315196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 msgr2=0x560308afabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308afabc0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00cf40 tx=0x7fdf7c021070 comp rx=0 tx=0).stop 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1095315196 shutdown_connections 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] conn(0x560308afd8a0 0x560308ac9df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308afe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308ac9170 0x560308afabc0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1095315196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308accfa0 0x560308afe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf5bfff700 0 client.9786 ms_handle_reset on v2:172.21.15.131:6834/2967199312 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1095315196 >> 172.21.15.131:0/1095315196 conn(0x560308aa3440 msgr2=0x560308aa7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1095315196 shutdown_connections 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1095315196 wait complete. 2023-12-26T23:09:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.031+0000 7fdf963e2b80 20 client.9786 trim_cache size 0 max 16384 2023-12-26T23:09:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.048+0000 7fdf963e2b80 1 Processor -- start 2023-12-26T23:09:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.048+0000 7fdf963e2b80 1 -- start start 2023-12-26T23:09:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.049+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308ac9170 0x560308ac9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.049+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.049+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308af9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.049+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560308ae5c20 con 0x560308accfa0 2023-12-26T23:09:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.049+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560308acd490 con 0x560308ac9170 2023-12-26T23:09:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.049+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560308aa5a20 con 0x560308acd8b0 2023-12-26T23:09:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.049+0000 7fdf8519a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf8519a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52490/0 (socket says 172.21.15.131:52490) 2023-12-26T23:09:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf8519a700 1 -- 172.21.15.131:0/48624931 learned_addr learned my addr 172.21.15.131:0/48624931 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf8519a700 1 -- 172.21.15.131:0/48624931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 msgr2=0x560308af9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf8519a700 1 --2- 172.21.15.131:0/48624931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308af9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf8519a700 1 -- 172.21.15.131:0/48624931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308ac9170 msgr2=0x560308ac9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf8519a700 1 --2- 172.21.15.131:0/48624931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308ac9170 0x560308ac9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf8519a700 1 -- 172.21.15.131:0/48624931 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560308aa2990 con 0x560308accfa0 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf8519a700 1 --2- 172.21.15.131:0/48624931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aca330 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fdf70001120 tx=0x7fdf70006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df48b1a1bdecb45a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf637fe700 1 -- 172.21.15.131:0/48624931 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70004d30 con 0x560308accfa0 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf637fe700 1 -- 172.21.15.131:0/48624931 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf70004e90 con 0x560308accfa0 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.050+0000 7fdf637fe700 1 -- 172.21.15.131:0/48624931 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700076d0 con 0x560308accfa0 2023-12-26T23:09:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.051+0000 7fdf963e2b80 1 -- 172.21.15.131:0/48624931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 msgr2=0x560308aca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.051+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/48624931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aca330 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fdf70001120 tx=0x7fdf70006040 comp rx=0 tx=0).stop 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.051+0000 7fdf963e2b80 1 -- 172.21.15.131:0/48624931 shutdown_connections 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.051+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/48624931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308acd8b0 0x560308af9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.051+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/48624931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308ac9170 0x560308ac9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.051+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/48624931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aca330 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.051+0000 7fdf963e2b80 1 -- 172.21.15.131:0/48624931 >> 172.21.15.131:0/48624931 conn(0x560308aa3440 msgr2=0x560308aa3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.051+0000 7fdf963e2b80 1 -- 172.21.15.131:0/48624931 shutdown_connections 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.051+0000 7fdf963e2b80 1 -- 172.21.15.131:0/48624931 wait complete. 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.052+0000 7fdf963e2b80 1 -- start start 2023-12-26T23:09:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.052+0000 7fdf963e2b80 1 Processor -- start 2023-12-26T23:09:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308ac9170 0x560308ae2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf963e2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308acd8b0 0x560308ac9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x560308ae7260 con 0x560308accfa0 2023-12-26T23:09:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560308b515f0 con 0x560308acd8b0 2023-12-26T23:09:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf963e2b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560308aa6850 con 0x560308ac9170 2023-12-26T23:09:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf963e2b80 10 client.0 _ll_register_callbacks cb 0x7fdf78009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8519a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8519a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aead00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52506/0 (socket says 172.21.15.131:52506) 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8519a700 1 -- 172.21.15.131:0/1931362911 learned_addr learned my addr 172.21.15.131:0/1931362911 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8599b700 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308ac9170 0x560308ae2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8619c700 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308acd8b0 0x560308ac9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8519a700 1 -- 172.21.15.131:0/1931362911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308ac9170 msgr2=0x560308ae2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8519a700 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308ac9170 0x560308ae2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8519a700 1 -- 172.21.15.131:0/1931362911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308acd8b0 msgr2=0x560308ac9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8519a700 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308acd8b0 0x560308ac9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8519a700 1 -- 172.21.15.131:0/1931362911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf800c9420 con 0x560308accfa0 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.054+0000 7fdf8599b700 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308ac9170 0x560308ae2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf8519a700 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aead00 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fdf7001d220 tx=0x7fdf7001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4568385e6525ce8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1931362911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70017850 con 0x560308accfa0 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1931362911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fdf7002ccc0 con 0x560308accfa0 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1931362911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf80069bd0 con 0x560308accfa0 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1931362911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf7002ce80 con 0x560308accfa0 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf963e2b80 10 client.9792 fetch_fsmap learned FSMap version 90 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1931362911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5603089c3290 con 0x560308accfa0 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1931362911 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf80069bd0 con 0x560308accfa0 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1931362911 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdf70017bc0 con 0x560308accfa0 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf963e2b80 10 client.9792 fetch_fsmap finished waiting for FSMap version 90 2023-12-26T23:09:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf963e2b80 10 client.9792 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf963e2b80 20 client.9792 populate_metadata read hostname 'smithi131' 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] conn(0x560308b507a0 0x560308b50b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf963e2b80 4 client.9792 mds_command: new command op to 9666 tid=0[{"prefix": "status"}] 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.055+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1931362911 --> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560308a99490 con 0x560308b507a0 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.056+0000 7fdf8599b700 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] conn(0x560308b507a0 0x560308b50b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.056+0000 7fdf8599b700 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] conn(0x560308b507a0 0x560308b50b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86664f98b45060fc server_cookie=e249d38db09c49a7 in_seq=0 out_seq=0 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.056+0000 7fdf5bfff700 10 client.9792 ms_handle_connect on v2:172.21.15.131:6834/2967199312 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.056+0000 7fdf5bfff700 1 -- 172.21.15.131:0/1931362911 <== mds.0 v2:172.21.15.131:6834/2967199312 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x560308a99490 con 0x560308b507a0 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.056+0000 7fdf5bfff700 10 client.9792 handle_command_reply: tid=0 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.056+0000 7fdf963e2b80 1 client.9792 shutdown 2023-12-26T23:09:53.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.056+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1931362911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 msgr2=0x560308aead00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.056+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aead00 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fdf7001d220 tx=0x7fdf7001a000 comp rx=0 tx=0).stop 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1931362911 shutdown_connections 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560308acd8b0 0x560308ac9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.131:6834/2967199312,v1:172.21.15.131:6835/2967199312] conn(0x560308b507a0 0x560308b50b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560308ac9170 0x560308ae2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf963e2b80 1 --2- 172.21.15.131:0/1931362911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560308accfa0 0x560308aead00 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf5bfff700 0 client.9792 ms_handle_reset on v2:172.21.15.131:6834/2967199312 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1931362911 >> 172.21.15.131:0/1931362911 conn(0x560308aa3440 msgr2=0x560308ae6940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1931362911 shutdown_connections 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf963e2b80 1 -- 172.21.15.131:0/1931362911 wait complete. 2023-12-26T23:09:53.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf963e2b80 20 client.9792 trim_cache size 0 max 16384 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 9666, 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.4913153939999999, 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 90, 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 193, 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 193, 2023-12-26T23:09:53.064 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 13.261014443000001 2023-12-26T23:09:53.065 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:09:53.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf87bff700 1 -- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdf64066f30 msgr2=0x7fdf640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf87bff700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdf64066f30 0x7fdf640693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c008fd0 tx=0x7fdf7c015000 comp rx=0 tx=0).stop 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf87bff700 1 -- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf800c6ab0 msgr2=0x7fdf800c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf87bff700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf800c6ab0 0x7fdf800c6f20 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fdf80064440 tx=0x7fdf7800b040 comp rx=0 tx=0).stop 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf87bff700 1 -- 172.21.15.131:0/3199584870 shutdown_connections 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf87bff700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdf64066f30 0x7fdf640693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.057+0000 7fdf87bff700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf800c6ab0 0x7fdf800c6f20 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.058+0000 7fdf87bff700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6c004d30 0x7fdf80158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.058+0000 7fdf87bff700 1 --2- 172.21.15.131:0/3199584870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf80149930 0x7fdf8014d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.058+0000 7fdf87bff700 1 -- 172.21.15.131:0/3199584870 >> 172.21.15.131:0/3199584870 conn(0x7fdf80068040 msgr2=0x7fdf800aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.058+0000 7fdf87bff700 1 -- 172.21.15.131:0/3199584870 shutdown_connections 2023-12-26T23:09:53.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.058+0000 7fdf87bff700 1 -- 172.21.15.131:0/3199584870 wait complete. 2023-12-26T23:09:53.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T23:09:53.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.265+0000 7f3ecfbff700 1 Processor -- start 2023-12-26T23:09:53.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.265+0000 7f3ecfbff700 1 -- start start 2023-12-26T23:09:53.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.266+0000 7f3ecfbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 0x7f3ec8069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.266+0000 7f3ecfbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3ec806a1e0 0x7f3ec80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.266+0000 7f3ecfbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3ec80b0c90 0x7f3ec80aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.266+0000 7f3ecfbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec80c8cc0 con 0x7f3ec80b0c90 2023-12-26T23:09:53.270 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.266+0000 7f3ecfbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec80ad700 con 0x7f3ec806a1e0 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.266+0000 7f3ecfbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec80b1110 con 0x7f3ec80cb320 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ece19c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3ec80b0c90 0x7f3ec80aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ece19c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3ec80b0c90 0x7f3ec80aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52520/0 (socket says 172.21.15.131:52520) 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ece19c700 1 -- 172.21.15.131:0/3780936391 learned_addr learned my addr 172.21.15.131:0/3780936391 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ecd19a700 1 --2- 172.21.15.131:0/3780936391 >> v2:172.21.15.190:3300/0 conn(0x7f3ec806a1e0 0x7f3ec80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ecd99b700 1 --2- 172.21.15.131:0/3780936391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 0x7f3ec8069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ece19c700 1 -- 172.21.15.131:0/3780936391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 msgr2=0x7f3ec8069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ece19c700 1 --2- 172.21.15.131:0/3780936391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 0x7f3ec8069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ece19c700 1 -- 172.21.15.131:0/3780936391 >> v2:172.21.15.190:3300/0 conn(0x7f3ec806a1e0 msgr2=0x7f3ec80b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:53.271 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ece19c700 1 --2- 172.21.15.131:0/3780936391 >> v2:172.21.15.190:3300/0 conn(0x7f3ec806a1e0 0x7f3ec80b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ece19c700 1 -- 172.21.15.131:0/3780936391 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec8156940 con 0x7f3ec80b0c90 2023-12-26T23:09:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ecd99b700 1 --2- 172.21.15.131:0/3780936391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 0x7f3ec8069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.267+0000 7f3ece19c700 1 --2- 172.21.15.131:0/3780936391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec80b0c90 0x7f3ec80aefd0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8068920 tx=0x7f3ec000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea4cb95afd309a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecc999700 1 -- 172.21.15.131:0/3780936391 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0010070 con 0x7f3ec80b0c90 2023-12-26T23:09:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecc999700 1 -- 172.21.15.131:0/3780936391 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec000dde0 con 0x7f3ec80b0c90 2023-12-26T23:09:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecc999700 1 -- 172.21.15.131:0/3780936391 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec000b320 con 0x7f3ec80b0c90 2023-12-26T23:09:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecfbff700 1 -- 172.21.15.131:0/3780936391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec80b0c90 msgr2=0x7f3ec80aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/3780936391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec80b0c90 0x7f3ec80aefd0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8068920 tx=0x7f3ec000ba50 comp rx=0 tx=0).stop 2023-12-26T23:09:53.272 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecfbff700 1 -- 172.21.15.131:0/3780936391 shutdown_connections 2023-12-26T23:09:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/3780936391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec80b0c90 0x7f3ec80aefd0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/3780936391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 0x7f3ec8069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/3780936391 >> v2:172.21.15.190:3300/0 conn(0x7f3ec806a1e0 0x7f3ec80b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecfbff700 1 -- 172.21.15.131:0/3780936391 >> 172.21.15.131:0/3780936391 conn(0x7f3ec8068040 msgr2=0x7f3ec8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecfbff700 1 -- 172.21.15.131:0/3780936391 shutdown_connections 2023-12-26T23:09:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.268+0000 7f3ecfbff700 1 -- 172.21.15.131:0/3780936391 wait complete. 2023-12-26T23:09:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.270+0000 7f3ecfbff700 1 Processor -- start 2023-12-26T23:09:53.273 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.270+0000 7f3ecfbff700 1 -- start start 2023-12-26T23:09:53.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecfbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec806a1e0 0x7f3ec81680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecfbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec80b0c90 0x7f3ec80acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecfbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 0x7f3ec80c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecfbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec80c3910 con 0x7f3ec806a1e0 2023-12-26T23:09:53.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecfbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec81689b0 con 0x7f3ec80b0c90 2023-12-26T23:09:53.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecfbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8168c90 con 0x7f3ec80cb320 2023-12-26T23:09:53.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecd99b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec806a1e0 0x7f3ec81680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecd99b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec806a1e0 0x7f3ec81680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52532/0 (socket says 172.21.15.131:52532) 2023-12-26T23:09:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecd99b700 1 -- 172.21.15.131:0/4076052470 learned_addr learned my addr 172.21.15.131:0/4076052470 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ece19c700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 0x7f3ec80c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecd19a700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec80b0c90 0x7f3ec80acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecd99b700 1 -- 172.21.15.131:0/4076052470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 msgr2=0x7f3ec80c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecd99b700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 0x7f3ec80c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecd99b700 1 -- 172.21.15.131:0/4076052470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec80b0c90 msgr2=0x7f3ec80acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecd99b700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec80b0c90 0x7f3ec80acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.271+0000 7f3ecd99b700 1 -- 172.21.15.131:0/4076052470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec000bcf0 con 0x7f3ec806a1e0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.272+0000 7f3ecd99b700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec806a1e0 0x7f3ec81680b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400d860 tx=0x7f3ec4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.272+0000 7f3ebdffb700 1 -- 172.21.15.131:0/4076052470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40125f0 con 0x7f3ec806a1e0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.272+0000 7f3ebdffb700 1 -- 172.21.15.131:0/4076052470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4012750 con 0x7f3ec806a1e0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.272+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec80c35a0 con 0x7f3ec806a1e0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.272+0000 7f3ebdffb700 1 -- 172.21.15.131:0/4076052470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4012ad0 con 0x7f3ec806a1e0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.272+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec80ad100 con 0x7f3ec806a1e0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.273+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec80657e0 con 0x7f3ec806a1e0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.274+0000 7f3ebdffb700 1 -- 172.21.15.131:0/4076052470 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3ec4014430 con 0x7f3ec806a1e0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.275+0000 7f3ebdffb700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3eac066f30 0x7f3eac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.275+0000 7f3ebdffb700 1 -- 172.21.15.131:0/4076052470 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f3ec4084b70 con 0x7f3ec806a1e0 2023-12-26T23:09:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.275+0000 7f3ecd19a700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3eac066f30 0x7f3eac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:53.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.276+0000 7f3ecd19a700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3eac066f30 0x7f3eac0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80c2ca0 tx=0x7f3eb8008040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:53.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.279+0000 7f3ebdffb700 1 -- 172.21.15.131:0/4076052470 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ec4055680 con 0x7f3ec806a1e0 2023-12-26T23:09:53.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.515+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ec80685c0 con 0x7f3ec806a1e0 2023-12-26T23:09:53.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.516+0000 7f3ebdffb700 1 -- 172.21.15.131:0/4076052470 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3ec4055210 con 0x7f3ec806a1e0 2023-12-26T23:09:53.516 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T23:09:53.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3eac066f30 msgr2=0x7f3eac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3eac066f30 0x7f3eac0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80c2ca0 tx=0x7f3eb8008040 comp rx=0 tx=0).stop 2023-12-26T23:09:53.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec806a1e0 msgr2=0x7f3ec81680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:53.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec806a1e0 0x7f3ec81680b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400d860 tx=0x7f3ec4008040 comp rx=0 tx=0).stop 2023-12-26T23:09:53.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 shutdown_connections 2023-12-26T23:09:53.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3eac066f30 0x7f3eac0693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ec80cb320 0x7f3ec80c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ec806a1e0 0x7f3ec81680b0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 --2- 172.21.15.131:0/4076052470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec80b0c90 0x7f3ec80acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:53.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 >> 172.21.15.131:0/4076052470 conn(0x7f3ec8068040 msgr2=0x7f3ec80ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:53.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 shutdown_connections 2023-12-26T23:09:53.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:53.518+0000 7f3ecfbff700 1 -- 172.21.15.131:0/4076052470 wait complete. 2023-12-26T23:09:53.536 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T23:09:53.536 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T23:09:53.536 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T23:09:53.536 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T23:09:53.536 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T23:09:53.537 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T23:09:53.537 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T23:09:53.537 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T23:09:53.537 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T23:09:53.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:09:53.537 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T23:09:53.582 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T23:09:53.583 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T23:09:53.583 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T23:09:53.583 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:09:53.583 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T23:09:53.583 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:09:53.583 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T23:09:53.583 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T23:09:53.583 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 36862sec preferred_lft 36862sec 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T23:09:53.584 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T23:09:53.585 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:09:53.586 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T23:09:53.586 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T23:09:53.586 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T23:09:53.586 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:09:53.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:09:53.586 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T23:09:53.691 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T23:09:53.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:09:53.695 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T23:09:53.741 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T23:09:53.741 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T23:09:53.741 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T23:09:53.741 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T23:09:53.741 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T23:09:53.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:09:53.743 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T23:09:53.743 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T23:09:53.743 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T23:09:53.743 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T23:09:53.743 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T23:09:53.743 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T23:09:53.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:09:53.954 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T23:09:53.999 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T23:09:53.999 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T23:09:53.999 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T23:09:53.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:09:53.999 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T23:09:54.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T23:09:54.108 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T23:09:54.152 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T23:09:54.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T23:09:54.152 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T23:09:54.239 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T23:09:54.239 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T23:09:54.239 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T23:09:54.378 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T23:09:54.378 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T23:09:54.407 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T23:09:54.407 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T23:09:54.407 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T23:09:54.407 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T23:09:54.407 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T23:09:54.412 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T23:09:54.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:09:54.412 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T23:09:54.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T23:09:54.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.678+0000 7f48d49ff700 1 Processor -- start 2023-12-26T23:09:54.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.678+0000 7f48d49ff700 1 -- start start 2023-12-26T23:09:54.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.679+0000 7f48d49ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48d00b0750 0x7f48d00b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:54.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.679+0000 7f48d49ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48d00c9400 0x7f48d00c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:54.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.679+0000 7f48d49ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d0156940 con 0x7f48d00cb320 2023-12-26T23:09:54.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.679+0000 7f48d49ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d00c8cc0 con 0x7f48d00c9400 2023-12-26T23:09:54.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.679+0000 7f48d49ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d00ad700 con 0x7f48d00b0750 2023-12-26T23:09:54.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48ce59c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f48d00cb320 0x7f48d0069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55144/0 (socket says 172.21.15.131:55144) 2023-12-26T23:09:54.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48ce59c700 1 -- 172.21.15.131:0/2142354273 learned_addr learned my addr 172.21.15.131:0/2142354273 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:54.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48ced9d700 1 --2- 172.21.15.131:0/2142354273 >> v2:172.21.15.131:3300/0 conn(0x7f48d00c9400 0x7f48d00c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:54.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48cdd9b700 1 --2- 172.21.15.131:0/2142354273 >> v2:172.21.15.190:3300/0 conn(0x7f48d00b0750 0x7f48d00b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:54.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48cd59a700 1 -- 172.21.15.131:0/2142354273 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782040681 0 0) 0x7f48d0156940 con 0x7f48d00cb320 2023-12-26T23:09:54.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48cd59a700 1 -- 172.21.15.131:0/2142354273 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b8003570 con 0x7f48d00cb320 2023-12-26T23:09:54.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48ced9d700 1 -- 172.21.15.131:0/2142354273 >> v2:172.21.15.190:3300/0 conn(0x7f48d00b0750 msgr2=0x7f48d00b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:54.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48ced9d700 1 --2- 172.21.15.131:0/2142354273 >> v2:172.21.15.190:3300/0 conn(0x7f48d00b0750 0x7f48d00b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:54.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48ced9d700 1 -- 172.21.15.131:0/2142354273 >> v1:172.21.15.131:6789/0 conn(0x7f48d00cb320 legacy=0x7f48d0069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:54.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48ced9d700 1 -- 172.21.15.131:0/2142354273 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d00ae500 con 0x7f48d00c9400 2023-12-26T23:09:54.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.680+0000 7f48ced9d700 1 --2- 172.21.15.131:0/2142354273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00c9400 0x7f48d00c9840 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f48d0064700 tx=0x7f48c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bb38cec2311133 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:54.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48cd59a700 1 -- 172.21.15.131:0/2142354273 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800e070 con 0x7f48d00c9400 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48cd59a700 1 -- 172.21.15.131:0/2142354273 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c800ba70 con 0x7f48d00c9400 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48cd59a700 1 -- 172.21.15.131:0/2142354273 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800be30 con 0x7f48d00c9400 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48d49ff700 1 -- 172.21.15.131:0/2142354273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00c9400 msgr2=0x7f48d00c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48d49ff700 1 --2- 172.21.15.131:0/2142354273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00c9400 0x7f48d00c9840 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f48d0064700 tx=0x7f48c8009780 comp rx=0 tx=0).stop 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48d49ff700 1 -- 172.21.15.131:0/2142354273 shutdown_connections 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48d49ff700 1 --2- 172.21.15.131:0/2142354273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00c9400 0x7f48d00c9840 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48d49ff700 1 --2- 172.21.15.131:0/2142354273 >> v2:172.21.15.190:3300/0 conn(0x7f48d00b0750 0x7f48d00b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48d49ff700 1 -- 172.21.15.131:0/2142354273 >> 172.21.15.131:0/2142354273 conn(0x7f48d0068040 msgr2=0x7f48d0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48d49ff700 1 -- 172.21.15.131:0/2142354273 shutdown_connections 2023-12-26T23:09:54.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.681+0000 7f48d49ff700 1 -- 172.21.15.131:0/2142354273 wait complete. 2023-12-26T23:09:54.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.683+0000 7f48d49ff700 1 Processor -- start 2023-12-26T23:09:54.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.683+0000 7f48d49ff700 1 -- start start 2023-12-26T23:09:54.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48d49ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00b0750 0x7f48d0147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:54.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48d49ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d00c9400 0x7f48d015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:54.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48d49ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00cb320 0x7f48d00cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:54.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48d49ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0157b90 con 0x7f48d00cb320 2023-12-26T23:09:54.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48d49ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0156af0 con 0x7f48d00b0750 2023-12-26T23:09:54.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48d49ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0156dd0 con 0x7f48d00c9400 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ced9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00cb320 0x7f48d00cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ced9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00cb320 0x7f48d00cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52550/0 (socket says 172.21.15.131:52550) 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ced9d700 1 -- 172.21.15.131:0/2517794110 learned_addr learned my addr 172.21.15.131:0/2517794110 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ce59c700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00b0750 0x7f48d0147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48cdd9b700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d00c9400 0x7f48d015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ced9d700 1 -- 172.21.15.131:0/2517794110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d00c9400 msgr2=0x7f48d015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ced9d700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d00c9400 0x7f48d015deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ced9d700 1 -- 172.21.15.131:0/2517794110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00b0750 msgr2=0x7f48d0147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ced9d700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00b0750 0x7f48d0147cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ced9d700 1 -- 172.21.15.131:0/2517794110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c8009a20 con 0x7f48d00cb320 2023-12-26T23:09:54.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.684+0000 7f48ce59c700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00b0750 0x7f48d0147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.685+0000 7f48ced9d700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00cb320 0x7f48d00cc110 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f48d00688f0 tx=0x7f48c8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.685+0000 7f48be7fc700 1 -- 172.21.15.131:0/2517794110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8027470 con 0x7f48d00cb320 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.685+0000 7f48be7fc700 1 -- 172.21.15.131:0/2517794110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c80275f0 con 0x7f48d00cb320 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.685+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d01570b0 con 0x7f48d00cb320 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.685+0000 7f48be7fc700 1 -- 172.21.15.131:0/2517794110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80279b0 con 0x7f48d00cb320 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.685+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d0157670 con 0x7f48d00cb320 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.686+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d01579e0 con 0x7f48d00cb320 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.687+0000 7f48be7fc700 1 -- 172.21.15.131:0/2517794110 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f48c8024070 con 0x7f48d00cb320 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.688+0000 7f48be7fc700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f48b0066f30 0x7f48b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.688+0000 7f48be7fc700 1 -- 172.21.15.131:0/2517794110 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f48c8086ed0 con 0x7f48d00cb320 2023-12-26T23:09:54.690 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.688+0000 7f48ce59c700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f48b0066f30 0x7f48b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:54.691 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.689+0000 7f48ce59c700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f48b0066f30 0x7f48b00693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48c00058d0 tx=0x7f48c0009040 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:54.692 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.692+0000 7f48be7fc700 1 -- 172.21.15.131:0/2517794110 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48c8053f50 con 0x7f48d00cb320 2023-12-26T23:09:54.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.913+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f48d00657e0 con 0x7f48b0066f30 2023-12-26T23:09:54.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.914+0000 7f48be7fc700 1 -- 172.21.15.131:0/2517794110 <== mgr.9456 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f48d00657e0 con 0x7f48b0066f30 2023-12-26T23:09:54.915 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T23:09:54.915 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:09:54.915 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T23:09:54.915 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:09:54.915 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T23:09:54.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f48b0066f30 msgr2=0x7f48b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:54.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f48b0066f30 0x7f48b00693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48c00058d0 tx=0x7f48c0009040 comp rx=0 tx=0).stop 2023-12-26T23:09:54.917 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00cb320 msgr2=0x7f48d00cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00cb320 0x7f48d00cc110 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f48d00688f0 tx=0x7f48c8019000 comp rx=0 tx=0).stop 2023-12-26T23:09:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 shutdown_connections 2023-12-26T23:09:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f48b0066f30 0x7f48b00693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48d00cb320 0x7f48d00cc110 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00b0750 0x7f48d0147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 --2- 172.21.15.131:0/2517794110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d00c9400 0x7f48d015deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 >> 172.21.15.131:0/2517794110 conn(0x7f48d0068040 msgr2=0x7f48d00ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 shutdown_connections 2023-12-26T23:09:54.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:54.917+0000 7f48d49ff700 1 -- 172.21.15.131:0/2517794110 wait complete. 2023-12-26T23:09:54.936 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T23:09:55.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.128+0000 7fc26bdff700 1 Processor -- start 2023-12-26T23:09:55.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.128+0000 7fc26bdff700 1 -- start start 2023-12-26T23:09:55.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc26bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26413c510 0x7fc2640c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:55.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc26bdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc26406b8c0 0x7fc26406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:55.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc26bdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26406c270 con 0x7fc2640cc100 2023-12-26T23:09:55.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc26bdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2640686c0 con 0x7fc26406b8c0 2023-12-26T23:09:55.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc26bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc264069520 con 0x7fc26413c510 2023-12-26T23:09:55.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc269b9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26413c510 0x7fc2640c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:55.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc26939a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc26406b8c0 0x7fc26406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:55.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc269b9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26413c510 0x7fc2640c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:48968/0 (socket says 172.21.15.131:48968) 2023-12-26T23:09:55.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc269b9b700 1 -- 172.21.15.131:0/1910235790 learned_addr learned my addr 172.21.15.131:0/1910235790 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:55.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.130+0000 7fc26a39c700 1 --1- 172.21.15.131:0/1910235790 >> v1:172.21.15.190:6789/0 conn(0x7fc2640cc100 0x7fc2640cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:53508/0 (socket says 172.21.15.131:53508) 2023-12-26T23:09:55.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.131+0000 7fc269b9b700 1 -- 172.21.15.131:0/1910235790 >> v2:172.21.15.190:3300/0 conn(0x7fc26406b8c0 msgr2=0x7fc26406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:55.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.131+0000 7fc269b9b700 1 --2- 172.21.15.131:0/1910235790 >> v2:172.21.15.190:3300/0 conn(0x7fc26406b8c0 0x7fc26406bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:55.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.131+0000 7fc269b9b700 1 -- 172.21.15.131:0/1910235790 >> v1:172.21.15.190:6789/0 conn(0x7fc2640cc100 legacy=0x7fc2640cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:55.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.131+0000 7fc269b9b700 1 -- 172.21.15.131:0/1910235790 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc264068910 con 0x7fc26413c510 2023-12-26T23:09:55.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.131+0000 7fc269b9b700 1 --2- 172.21.15.131:0/1910235790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26413c510 0x7fc2640c8a20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc260009f70 tx=0x7fc260009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dd4e87c9284d3b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:55.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.131+0000 7fc268b99700 1 -- 172.21.15.131:0/1910235790 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26000e070 con 0x7fc26413c510 2023-12-26T23:09:55.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.131+0000 7fc268b99700 1 -- 172.21.15.131:0/1910235790 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26000ba20 con 0x7fc26413c510 2023-12-26T23:09:55.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.132+0000 7fc268b99700 1 -- 172.21.15.131:0/1910235790 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2600092e0 con 0x7fc26413c510 2023-12-26T23:09:55.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.132+0000 7fc26bdff700 1 -- 172.21.15.131:0/1910235790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26413c510 msgr2=0x7fc2640c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:55.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.132+0000 7fc26bdff700 1 --2- 172.21.15.131:0/1910235790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26413c510 0x7fc2640c8a20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc260009f70 tx=0x7fc260009840 comp rx=0 tx=0).stop 2023-12-26T23:09:55.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.132+0000 7fc26bdff700 1 -- 172.21.15.131:0/1910235790 shutdown_connections 2023-12-26T23:09:55.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.132+0000 7fc26bdff700 1 --2- 172.21.15.131:0/1910235790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26413c510 0x7fc2640c8a20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:55.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.132+0000 7fc26bdff700 1 --2- 172.21.15.131:0/1910235790 >> v2:172.21.15.190:3300/0 conn(0x7fc26406b8c0 0x7fc26406bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:55.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.132+0000 7fc26bdff700 1 -- 172.21.15.131:0/1910235790 >> 172.21.15.131:0/1910235790 conn(0x7fc2640ce3e0 msgr2=0x7fc2640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:55.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.132+0000 7fc26bdff700 1 -- 172.21.15.131:0/1910235790 shutdown_connections 2023-12-26T23:09:55.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.132+0000 7fc26bdff700 1 -- 172.21.15.131:0/1910235790 wait complete. 2023-12-26T23:09:55.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.134+0000 7fc26bdff700 1 Processor -- start 2023-12-26T23:09:55.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.134+0000 7fc26bdff700 1 -- start start 2023-12-26T23:09:55.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc26bdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc26406b8c0 0x7fc26413bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:55.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc26bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2640cc100 0x7fc26414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:55.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc26bdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26413c510 0x7fc26415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:55.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc26bdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640b1180 con 0x7fc26406b8c0 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc26bdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26414c840 con 0x7fc26413c510 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc26bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26414cb20 con 0x7fc2640cc100 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc269b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc26406b8c0 0x7fc26413bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc269b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc26406b8c0 0x7fc26413bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54070/0 (socket says 172.21.15.131:54070) 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc269b9b700 1 -- 172.21.15.131:0/129328314 learned_addr learned my addr 172.21.15.131:0/129328314 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc26a39c700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26413c510 0x7fc26415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc26939a700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2640cc100 0x7fc26414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc269b9b700 1 -- 172.21.15.131:0/129328314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2640cc100 msgr2=0x7fc26414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc269b9b700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2640cc100 0x7fc26414bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:55.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc269b9b700 1 -- 172.21.15.131:0/129328314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26413c510 msgr2=0x7fc26415e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc269b9b700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26413c510 0x7fc26415e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.135+0000 7fc269b9b700 1 -- 172.21.15.131:0/129328314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009ae0 con 0x7fc26406b8c0 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.136+0000 7fc269b9b700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc26406b8c0 0x7fc26413bf80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc26000b6c0 tx=0x7fc260018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.136+0000 7fc259ffb700 1 -- 172.21.15.131:0/129328314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260007b90 con 0x7fc26406b8c0 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.136+0000 7fc259ffb700 1 -- 172.21.15.131:0/129328314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc260007cf0 con 0x7fc26406b8c0 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.136+0000 7fc259ffb700 1 -- 172.21.15.131:0/129328314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26001e680 con 0x7fc26406b8c0 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.136+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26414ce00 con 0x7fc26406b8c0 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.136+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26414b090 con 0x7fc26406b8c0 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.137+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2640657e0 con 0x7fc26406b8c0 2023-12-26T23:09:55.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.138+0000 7fc259ffb700 1 -- 172.21.15.131:0/129328314 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 96) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc260003040 con 0x7fc26406b8c0 2023-12-26T23:09:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.139+0000 7fc259ffb700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc248066f30 0x7fc2480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:55.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.139+0000 7fc259ffb700 1 -- 172.21.15.131:0/129328314 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fc260086280 con 0x7fc26406b8c0 2023-12-26T23:09:55.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.139+0000 7fc26939a700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc248066f30 0x7fc2480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:55.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.139+0000 7fc26939a700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc248066f30 0x7fc2480693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc2641598e0 tx=0x7fc2540091a0 comp rx=0 tx=0).ready entity=mgr.9456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:55.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.143+0000 7fc259ffb700 1 -- 172.21.15.131:0/129328314 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc260053300 con 0x7fc26406b8c0 2023-12-26T23:09:55.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:55.393+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc26415d020 con 0x7fc26406b8c0 2023-12-26T23:09:56.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.385+0000 7fc259ffb700 1 -- 172.21.15.131:0/129328314 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 97) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fc2600536d0 con 0x7fc26406b8c0 2023-12-26T23:09:56.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.387+0000 7fc259ffb700 1 -- 172.21.15.131:0/129328314 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v97) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc260056d90 con 0x7fc26406b8c0 2023-12-26T23:09:56.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc248066f30 msgr2=0x7fc2480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:56.390 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc248066f30 0x7fc2480693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc2641598e0 tx=0x7fc2540091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc26406b8c0 msgr2=0x7fc26413bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc26406b8c0 0x7fc26413bf80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc26000b6c0 tx=0x7fc260018000 comp rx=0 tx=0).stop 2023-12-26T23:09:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 shutdown_connections 2023-12-26T23:09:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc248066f30 0x7fc2480693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26413c510 0x7fc26415e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc26406b8c0 0x7fc26413bf80 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 --2- 172.21.15.131:0/129328314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2640cc100 0x7fc26414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 >> 172.21.15.131:0/129328314 conn(0x7fc2640ce3e0 msgr2=0x7fc2640c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:56.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 shutdown_connections 2023-12-26T23:09:56.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.390+0000 7fc26bdff700 1 -- 172.21.15.131:0/129328314 wait complete. 2023-12-26T23:09:56.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T23:09:56.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.611+0000 7f40b1dff700 1 Processor -- start 2023-12-26T23:09:56.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.611+0000 7f40b1dff700 1 -- start start 2023-12-26T23:09:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.612+0000 7f40b1dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f40ac0cc580 0x7f40ac0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.612+0000 7f40b1dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f40ac0b00e0 0x7f40ac06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.612+0000 7f40b1dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac0b0d70 con 0x7f40ac0b0560 2023-12-26T23:09:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.612+0000 7f40b1dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac0680f0 con 0x7f40ac0cc580 2023-12-26T23:09:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.612+0000 7f40b1dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac0675b0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.613+0000 7f40ab7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f40ac0cc580 0x7f40ac0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.613+0000 7f40ab7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f40ac0cc580 0x7f40ac0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54100/0 (socket says 172.21.15.131:54100) 2023-12-26T23:09:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.613+0000 7f40ab7fe700 1 -- 172.21.15.131:0/2842571903 learned_addr learned my addr 172.21.15.131:0/2842571903 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.613+0000 7f40aaffd700 1 --2- 172.21.15.131:0/2842571903 >> v2:172.21.15.190:3300/0 conn(0x7f40ac0b00e0 0x7f40ac06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.613+0000 7f40ab7fe700 1 -- 172.21.15.131:0/2842571903 >> v2:172.21.15.190:3300/0 conn(0x7f40ac0b00e0 msgr2=0x7f40ac06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.613+0000 7f40ab7fe700 1 --2- 172.21.15.131:0/2842571903 >> v2:172.21.15.190:3300/0 conn(0x7f40ac0b00e0 0x7f40ac06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.613+0000 7f40ab7fe700 1 -- 172.21.15.131:0/2842571903 >> v1:172.21.15.190:6789/0 conn(0x7f40ac0b0560 legacy=0x7f40ac15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.613+0000 7f40ab7fe700 1 -- 172.21.15.131:0/2842571903 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac159c30 con 0x7f40ac0cc580 2023-12-26T23:09:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.614+0000 7f40ab7fe700 1 --2- 172.21.15.131:0/2842571903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 0x7f40ac0cc950 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f40ac06a140 tx=0x7f409c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57496dda4e71f0cb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:56.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.614+0000 7f40aa7fc700 1 -- 172.21.15.131:0/2842571903 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c009e70 con 0x7f40ac0cc580 2023-12-26T23:09:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.614+0000 7f40aa7fc700 1 -- 172.21.15.131:0/2842571903 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c00b860 con 0x7f40ac0cc580 2023-12-26T23:09:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.614+0000 7f40aa7fc700 1 -- 172.21.15.131:0/2842571903 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c00bc00 con 0x7f40ac0cc580 2023-12-26T23:09:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.614+0000 7f40b1dff700 1 -- 172.21.15.131:0/2842571903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 msgr2=0x7f40ac0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.614+0000 7f40b1dff700 1 --2- 172.21.15.131:0/2842571903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 0x7f40ac0cc950 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f40ac06a140 tx=0x7f409c009250 comp rx=0 tx=0).stop 2023-12-26T23:09:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.615+0000 7f40b1dff700 1 -- 172.21.15.131:0/2842571903 shutdown_connections 2023-12-26T23:09:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.615+0000 7f40b1dff700 1 --2- 172.21.15.131:0/2842571903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 0x7f40ac0cc950 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.615+0000 7f40b1dff700 1 --2- 172.21.15.131:0/2842571903 >> v2:172.21.15.190:3300/0 conn(0x7f40ac0b00e0 0x7f40ac06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.615+0000 7f40b1dff700 1 -- 172.21.15.131:0/2842571903 >> 172.21.15.131:0/2842571903 conn(0x7f40ac0686c0 msgr2=0x7f40ac068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:56.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.615+0000 7f40b1dff700 1 -- 172.21.15.131:0/2842571903 shutdown_connections 2023-12-26T23:09:56.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.615+0000 7f40b1dff700 1 -- 172.21.15.131:0/2842571903 wait complete. 2023-12-26T23:09:56.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.616+0000 7f40b1dff700 1 Processor -- start 2023-12-26T23:09:56.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.616+0000 7f40b1dff700 1 -- start start 2023-12-26T23:09:56.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40b1dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40ac0b00e0 0x7f40ac149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:56.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40b1dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40ac0b0560 0x7f40ac0c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:56.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40b1dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 0x7f40ac0c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:56.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40b1dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac15cd40 con 0x7f40ac0cc580 2023-12-26T23:09:56.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40b1dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac14a3a0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40b1dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac15ca60 con 0x7f40ac0b0560 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40ab7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40ac0b00e0 0x7f40ac149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 0x7f40ac0c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40ab7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40ac0b00e0 0x7f40ac149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:49510/0 (socket says 172.21.15.131:49510) 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.617+0000 7f40ab7fe700 1 -- 172.21.15.131:0/2303843054 learned_addr learned my addr 172.21.15.131:0/2303843054 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40aaffd700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40ac0b0560 0x7f40ac0c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40ab7fe700 1 -- 172.21.15.131:0/2303843054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40ac0b0560 msgr2=0x7f40ac0c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40ab7fe700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40ac0b0560 0x7f40ac0c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40ab7fe700 1 -- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 msgr2=0x7f40ac0c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40ab7fe700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 0x7f40ac0c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40ab7fe700 1 -- 172.21.15.131:0/2303843054 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0094f0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40abfff700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 0x7f40ac0c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40ab7fe700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40ac0b00e0 0x7f40ac149e60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f40ac06b770 tx=0x7f409c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40937fe700 1 -- 172.21.15.131:0/2303843054 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c00bbe0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40937fe700 1 -- 172.21.15.131:0/2303843054 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c011420 con 0x7f40ac0b00e0 2023-12-26T23:09:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.618+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac0adf70 con 0x7f40ac0b00e0 2023-12-26T23:09:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.619+0000 7f40937fe700 1 -- 172.21.15.131:0/2303843054 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c0117c0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.619+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac0ae530 con 0x7f40ac0b00e0 2023-12-26T23:09:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.620+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac0ae8a0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.621+0000 7f40937fe700 1 -- 172.21.15.131:0/2303843054 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 97) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f409c011bf0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.621+0000 7f40937fe700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4098066f80 0x7f4098069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.621+0000 7f40937fe700 1 -- 172.21.15.131:0/2303843054 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f409c085bc0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.621+0000 7f40aaffd700 1 -- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4098066f80 msgr2=0x7f4098069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:56.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.621+0000 7f40aaffd700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4098066f80 0x7f4098069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:56.625 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.626+0000 7f40937fe700 1 -- 172.21.15.131:0/2303843054 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f409c052ba0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.817 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:56.818+0000 7f217c26e1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:09:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.822+0000 7f40aaffd700 1 -- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4098066f80 msgr2=0x7f4098069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.822+0000 7f40aaffd700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4098066f80 0x7f4098069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:56.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.828+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f40ac0657e0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.835+0000 7f40937fe700 1 -- 172.21.15.131:0/2303843054 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f409c0561c0 con 0x7f40ac0b00e0 2023-12-26T23:09:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4098066f80 msgr2=0x7f4098069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4098066f80 0x7f4098069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40ac0b00e0 msgr2=0x7f40ac149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40ac0b00e0 0x7f40ac149e60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f40ac06b770 tx=0x7f409c017000 comp rx=0 tx=0).stop 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 shutdown_connections 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4098066f80 0x7f4098069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40ac0cc580 0x7f40ac0c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40ac0b00e0 0x7f40ac149e60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 --2- 172.21.15.131:0/2303843054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40ac0b0560 0x7f40ac0c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 >> 172.21.15.131:0/2303843054 conn(0x7f40ac0686c0 msgr2=0x7f40ac157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:56.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 shutdown_connections 2023-12-26T23:09:56.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:56.838+0000 7f40b1dff700 1 -- 172.21.15.131:0/2303843054 wait complete. 2023-12-26T23:09:56.841 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:56.841+0000 7fb55caf41c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:09:56.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T23:09:56.922 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:56.922+0000 7f217c26e1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:09:56.963 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:56.963+0000 7fb55caf41c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:09:57.009 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:57.010+0000 7f217c26e1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:09:57.056 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:57.056+0000 7fb55caf41c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:09:57.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.065+0000 7ff855bff700 1 Processor -- start 2023-12-26T23:09:57.065 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.065+0000 7ff855bff700 1 -- start start 2023-12-26T23:09:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff855bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff8500c8bf0 0x7ff8500b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff855bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8500caa10 con 0x7ff8500c8820 2023-12-26T23:09:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff855bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff850157a80 con 0x7ff850069d10 2023-12-26T23:09:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff855bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8500cac50 con 0x7ff8500c8bf0 2023-12-26T23:09:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff84f7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff8500c8820 0x7ff8500ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34424/0 (socket says 172.21.15.131:34424) 2023-12-26T23:09:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff84f7fe700 1 -- 172.21.15.131:0/2055517360 learned_addr learned my addr 172.21.15.131:0/2055517360 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff847fff700 1 --2- 172.21.15.131:0/2055517360 >> v2:172.21.15.190:3300/0 conn(0x7ff8500c8bf0 0x7ff8500b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:57.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff84effd700 1 -- 172.21.15.131:0/2055517360 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986009776 0 0) 0x7ff840001110 con 0x7ff8500c8820 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff84effd700 1 -- 172.21.15.131:0/2055517360 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff83c003570 con 0x7ff8500c8820 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff84effd700 1 -- 172.21.15.131:0/2055517360 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010063260 0 0) 0x7ff850157a80 con 0x7ff850069d10 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff84effd700 1 -- 172.21.15.131:0/2055517360 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff840001110 con 0x7ff850069d10 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff847fff700 1 -- 172.21.15.131:0/2055517360 >> v1:172.21.15.190:6789/0 conn(0x7ff850069d10 legacy=0x7ff850161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff847fff700 1 -- 172.21.15.131:0/2055517360 >> v1:172.21.15.131:6789/0 conn(0x7ff8500c8820 legacy=0x7ff8500ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.067+0000 7ff847fff700 1 -- 172.21.15.131:0/2055517360 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8500caf30 con 0x7ff8500c8bf0 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.068+0000 7ff847fff700 1 --2- 172.21.15.131:0/2055517360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8500c8bf0 0x7ff8500b12e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff8380098e0 tx=0x7ff8380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddf99076df47655 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.068+0000 7ff84effd700 1 -- 172.21.15.131:0/2055517360 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838011630 con 0x7ff8500c8bf0 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.068+0000 7ff84effd700 1 -- 172.21.15.131:0/2055517360 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff838011790 con 0x7ff8500c8bf0 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.068+0000 7ff84effd700 1 -- 172.21.15.131:0/2055517360 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838011b30 con 0x7ff8500c8bf0 2023-12-26T23:09:57.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.069+0000 7ff855bff700 1 -- 172.21.15.131:0/2055517360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8500c8bf0 msgr2=0x7ff8500b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.069+0000 7ff855bff700 1 --2- 172.21.15.131:0/2055517360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8500c8bf0 0x7ff8500b12e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff8380098e0 tx=0x7ff8380091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.069+0000 7ff855bff700 1 -- 172.21.15.131:0/2055517360 shutdown_connections 2023-12-26T23:09:57.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.069+0000 7ff855bff700 1 --2- 172.21.15.131:0/2055517360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8500c8bf0 0x7ff8500b12e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.069+0000 7ff855bff700 1 -- 172.21.15.131:0/2055517360 >> 172.21.15.131:0/2055517360 conn(0x7ff850068040 msgr2=0x7ff850069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:57.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.069+0000 7ff855bff700 1 -- 172.21.15.131:0/2055517360 shutdown_connections 2023-12-26T23:09:57.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.069+0000 7ff855bff700 1 -- 172.21.15.131:0/2055517360 wait complete. 2023-12-26T23:09:57.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.070+0000 7ff855bff700 1 Processor -- start 2023-12-26T23:09:57.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.070+0000 7ff855bff700 1 -- start start 2023-12-26T23:09:57.073 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.071+0000 7ff855bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff850069d10 0x7ff85014b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.071+0000 7ff855bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8500c8820 0x7ff850152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.071+0000 7ff855bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8500c8bf0 0x7ff8500ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.071+0000 7ff855bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85014dcf0 con 0x7ff8500c8bf0 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.071+0000 7ff855bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8501531e0 con 0x7ff850069d10 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.071+0000 7ff855bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff850153480 con 0x7ff8500c8820 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8500c8bf0 0x7ff8500ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff850069d10 0x7ff85014b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8500c8bf0 0x7ff8500ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54118/0 (socket says 172.21.15.131:54118) 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84ffff700 1 -- 172.21.15.131:0/1698760173 learned_addr learned my addr 172.21.15.131:0/1698760173 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff847fff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8500c8820 0x7ff850152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84ffff700 1 -- 172.21.15.131:0/1698760173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8500c8820 msgr2=0x7ff850152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84ffff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8500c8820 0x7ff850152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84ffff700 1 -- 172.21.15.131:0/1698760173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff850069d10 msgr2=0x7ff85014b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84ffff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff850069d10 0x7ff85014b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84ffff700 1 -- 172.21.15.131:0/1698760173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff838009420 con 0x7ff8500c8bf0 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff84ffff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8500c8bf0 0x7ff8500ae060 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c860 tx=0x7ff848002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff8477fe700 1 -- 172.21.15.131:0/1698760173 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848002030 con 0x7ff8500c8bf0 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff8477fe700 1 -- 172.21.15.131:0/1698760173 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8480021b0 con 0x7ff8500c8bf0 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff850153760 con 0x7ff8500c8bf0 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.072+0000 7ff8477fe700 1 -- 172.21.15.131:0/1698760173 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848012630 con 0x7ff8500c8bf0 2023-12-26T23:09:57.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.073+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8501508d0 con 0x7ff8500c8bf0 2023-12-26T23:09:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.074+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8501539a0 con 0x7ff8500c8bf0 2023-12-26T23:09:57.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.074+0000 7ff8477fe700 1 -- 172.21.15.131:0/1698760173 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 97) v1 ==== 84759+0+0 (secure 0 0 0) 0x7ff848012a20 con 0x7ff8500c8bf0 2023-12-26T23:09:57.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.076+0000 7ff8477fe700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff830066f80 0x7ff830069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.076+0000 7ff84f7fe700 1 -- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff830066f80 msgr2=0x7ff830069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:57.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.076+0000 7ff8477fe700 1 -- 172.21.15.131:0/1698760173 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7ff848084770 con 0x7ff8500c8bf0 2023-12-26T23:09:57.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.076+0000 7ff84f7fe700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff830066f80 0x7ff830069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:57.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.080+0000 7ff8477fe700 1 -- 172.21.15.131:0/1698760173 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8501539a0 con 0x7ff8500c8bf0 2023-12-26T23:09:57.092 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:57.092+0000 7f217c26e1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:09:57.180 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:57.180+0000 7fb55caf41c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:09:57.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.277+0000 7ff84f7fe700 1 -- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff830066f80 msgr2=0x7ff830069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:57.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.277+0000 7ff84f7fe700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff830066f80 0x7ff830069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:57.280 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:57.280+0000 7f217c26e1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:09:57.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.342+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ff8501539a0 con 0x7ff8500c8bf0 2023-12-26T23:09:57.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.346+0000 7ff8477fe700 1 -- 172.21.15.131:0/1698760173 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff8501539a0 con 0x7ff8500c8bf0 2023-12-26T23:09:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff830066f80 msgr2=0x7ff830069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:09:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff830066f80 0x7ff830069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8500c8bf0 msgr2=0x7ff8500ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8500c8bf0 0x7ff8500ae060 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c860 tx=0x7ff848002700 comp rx=0 tx=0).stop 2023-12-26T23:09:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 shutdown_connections 2023-12-26T23:09:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff830066f80 0x7ff830069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8500c8bf0 0x7ff8500ae060 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff850069d10 0x7ff85014b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 --2- 172.21.15.131:0/1698760173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8500c8820 0x7ff850152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.348+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 >> 172.21.15.131:0/1698760173 conn(0x7ff850068040 msgr2=0x7ff850156870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.349+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 shutdown_connections 2023-12-26T23:09:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.349+0000 7ff855bff700 1 -- 172.21.15.131:0/1698760173 wait complete. 2023-12-26T23:09:57.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:09:57.374 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-26T23:09:57.390 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:57.389+0000 7fb55caf41c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:09:57.423 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-12-26T23:09:57.452 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:57.452+0000 7f217c26e1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:09:57.537 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:57.536+0000 7f217c26e1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:09:57.577 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:57.577+0000 7fb55caf41c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:09:57.665 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:57.665+0000 7fb55caf41c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:09:57.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.670+0000 7fe86b9ff700 1 Processor -- start 2023-12-26T23:09:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.671+0000 7fe86b9ff700 1 -- start start 2023-12-26T23:09:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.672+0000 7fe86b9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe8641576f0 0x7fe8640b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.672+0000 7fe86b9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe8640b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.672+0000 7fe86b9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86415b350 con 0x7fe864157320 2023-12-26T23:09:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.672+0000 7fe86b9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86406cf80 con 0x7fe8641576f0 2023-12-26T23:09:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.672+0000 7fe86b9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86406fa90 con 0x7fe8640caf60 2023-12-26T23:09:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.672+0000 7fe868f9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe8641576f0 0x7fe8640b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.672+0000 7fe86979b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe864157320 0x7fe86406e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:53530/0 (socket says 172.21.15.131:53530) 2023-12-26T23:09:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.672+0000 7fe869f9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe8640b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.672+0000 7fe86979b700 1 -- 172.21.15.131:0/3226190770 learned_addr learned my addr 172.21.15.131:0/3226190770 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:09:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.673+0000 7fe85bfff700 1 -- 172.21.15.131:0/3226190770 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144200104 0 0) 0x7fe86415b350 con 0x7fe864157320 2023-12-26T23:09:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.673+0000 7fe85bfff700 1 -- 172.21.15.131:0/3226190770 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe850003570 con 0x7fe864157320 2023-12-26T23:09:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.673+0000 7fe868f9a700 1 -- 172.21.15.131:0/3226190770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 msgr2=0x7fe8640b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:09:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.673+0000 7fe868f9a700 1 --2- 172.21.15.131:0/3226190770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe8640b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.673+0000 7fe868f9a700 1 -- 172.21.15.131:0/3226190770 >> v1:172.21.15.190:6789/0 conn(0x7fe864157320 legacy=0x7fe86406e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.673+0000 7fe868f9a700 1 -- 172.21.15.131:0/3226190770 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86406f1f0 con 0x7fe8641576f0 2023-12-26T23:09:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.673+0000 7fe869f9c700 1 --2- 172.21.15.131:0/3226190770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe8640b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.673+0000 7fe868f9a700 1 --2- 172.21.15.131:0/3226190770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 0x7fe8640b23d0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe8540098e0 tx=0x7fe8540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1522ed64fbd172d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.673+0000 7fe85bfff700 1 -- 172.21.15.131:0/3226190770 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854011630 con 0x7fe8641576f0 2023-12-26T23:09:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe85bfff700 1 -- 172.21.15.131:0/3226190770 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe854011790 con 0x7fe8641576f0 2023-12-26T23:09:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe85bfff700 1 -- 172.21.15.131:0/3226190770 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854011b30 con 0x7fe8641576f0 2023-12-26T23:09:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3226190770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 msgr2=0x7fe8640b23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe86b9ff700 1 --2- 172.21.15.131:0/3226190770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 0x7fe8640b23d0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe8540098e0 tx=0x7fe8540091a0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3226190770 shutdown_connections 2023-12-26T23:09:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe86b9ff700 1 --2- 172.21.15.131:0/3226190770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe8640b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe86b9ff700 1 --2- 172.21.15.131:0/3226190770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 0x7fe8640b23d0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3226190770 >> 172.21.15.131:0/3226190770 conn(0x7fe86406b1c0 msgr2=0x7fe8640aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:09:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3226190770 shutdown_connections 2023-12-26T23:09:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.674+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3226190770 wait complete. 2023-12-26T23:09:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.676+0000 7fe86b9ff700 1 Processor -- start 2023-12-26T23:09:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.677+0000 7fe86b9ff700 1 -- start start 2023-12-26T23:09:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe86b9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe864153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe86b9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe864157320 0x7fe8640c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe86b9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 0x7fe8640aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe86b9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640c3160 con 0x7fe8641576f0 2023-12-26T23:09:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe86b9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640c9780 con 0x7fe864157320 2023-12-26T23:09:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe86b9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640c2b50 con 0x7fe8640caf60 2023-12-26T23:09:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe869f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 0x7fe8640aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe868f9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe864157320 0x7fe8640c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe86979b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe864153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:09:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe869f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 0x7fe8640aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54136/0 (socket says 172.21.15.131:54136) 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.678+0000 7fe869f9c700 1 -- 172.21.15.131:0/3254035263 learned_addr learned my addr 172.21.15.131:0/3254035263 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe869f9c700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 msgr2=0x7fe864153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe869f9c700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe864153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe869f9c700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe864157320 msgr2=0x7fe8640c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe869f9c700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe864157320 0x7fe8640c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe869f9c700 1 -- 172.21.15.131:0/3254035263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c009a20 con 0x7fe8641576f0 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe86979b700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe864153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe868f9a700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe864157320 0x7fe8640c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe869f9c700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 0x7fe8640aa880 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009f60 tx=0x7fe85c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:09:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00cb70 con 0x7fe8641576f0 2023-12-26T23:09:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c00f040 con 0x7fe8641576f0 2023-12-26T23:09:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c014480 con 0x7fe8641576f0 2023-12-26T23:09:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe854009420 con 0x7fe8641576f0 2023-12-26T23:09:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.679+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86415a550 con 0x7fe8641576f0 2023-12-26T23:09:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.681+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe864158510 con 0x7fe8641576f0 2023-12-26T23:09:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.681+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 97) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fe85c003020 con 0x7fe8641576f0 2023-12-26T23:09:57.685 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:57.682+0000 7f217c26e1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:09:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.682+0000 7fe8597fa700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe848069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:09:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.682+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fe85c085140 con 0x7fe8641576f0 2023-12-26T23:09:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.682+0000 7fe86979b700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 msgr2=0x7fe848069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.682+0000 7fe86979b700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe848069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:09:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.686+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe864158510 con 0x7fe8641576f0 2023-12-26T23:09:57.772 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:57.772+0000 7f217c26e1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:09:57.819 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:57.819+0000 7fb55caf41c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:09:57.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.883+0000 7fe86979b700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 msgr2=0x7fe848069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:57.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.883+0000 7fe86979b700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe848069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:09:57.913 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:57.913+0000 7fb55caf41c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:09:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:57.918+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe864158510 con 0x7fe848066f80 2023-12-26T23:09:57.951 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:57.952+0000 7f217c26e1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:09:58.095 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:58.095+0000 7fb55caf41c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:09:58.102 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:58.102+0000 7f217c26e1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:09:58.245 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:58.245+0000 7fb55caf41c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:09:58.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:58.284+0000 7fe86979b700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 msgr2=0x7fe848069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:58.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:58.284+0000 7fe86979b700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe848069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T23:09:58.482 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:58.483+0000 7f217c26e1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:09:58.630 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:58.630+0000 7fb55caf41c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:09:58.748 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:58.748+0000 7f217c26e1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:09:58.897 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:58.897+0000 7fb55caf41c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:09:59.024 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:59.025+0000 7f217c26e1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:09:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:59.085+0000 7fe86979b700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 msgr2=0x7fe848069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:09:59.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:09:59.085+0000 7fe86979b700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe848069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T23:09:59.112 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:59.112+0000 7f217c26e1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:09:59.178 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:59.177+0000 7fb55caf41c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:09:59.266 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:09:59.266+0000 7fb55caf41c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:09:59.845 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:59.846+0000 7f217c26e1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:09:59.939 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:09:59.939+0000 7f217c26e1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:10:00.004 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:00.004+0000 7fb55caf41c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:10:00.092 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:10:00.092+0000 7f217c26e1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:10:00.100 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:00.100+0000 7fb55caf41c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:10:00.251 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:00.251+0000 7fb55caf41c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:10:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:00.687+0000 7fe86979b700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 msgr2=0x7fe848069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:10:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:00.687+0000 7fe86979b700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe848069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T23:10:00.714 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:10:00.714+0000 7f217c26e1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:10:00.875 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:00.875+0000 7fb55caf41c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:10:00.901 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:10:00.901+0000 7f217c26e1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:10:01.067 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:01.067+0000 7fb55caf41c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:10:01.090 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:10:01.090+0000 7f217c26e1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:10:01.258 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:01.258+0000 7fb55caf41c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:10:01.943 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:10:01.943+0000 7f217c26e1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:10:02.111 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.111+0000 7fb55caf41c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:10:02.217 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:10:02.218+0000 7f217c26e1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:10:02.310 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:10:02.310+0000 7f217c26e1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:10:02.390 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.390+0000 7fb55caf41c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:10:02.484 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.484+0000 7fb55caf41c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:10:02.507 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:10:02.507+0000 7f217c26e1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:10:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:02.565+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 98) v1 ==== 84759+0+0 (secure 0 0 0) 0x7fe85c0524f0 con 0x7fe8641576f0 2023-12-26T23:10:02.690 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.690+0000 7fb55caf41c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:10:02.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:02.700+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 99) v1 ==== 84470+0+0 (secure 0 0 0) 0x7fe85c0552d0 con 0x7fe8641576f0 2023-12-26T23:10:02.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:02.700+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 msgr2=0x7fe848069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:10:02.700 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:02.700+0000 7fe8597fa700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe848069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.824+0000 7fb4ce471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.824+0000 7fb4ce471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.824+0000 7fb4ce471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.824+0000 7fb4ce471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.824+0000 7fb4ce471700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.825+0000 7fb4d0c76700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.825+0000 7fb4d0c76700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.825+0000 7fb4d0c76700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.825+0000 7fb4d0c76700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:02.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:10:02.825+0000 7fb4d0c76700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:10:03.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.704+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mon.0 v2:172.21.15.131:3300/0 9 ==== mgrmap(e 100) v1 ==== 84597+0+0 (secure 0 0 0) 0x7fe85c04e2c0 con 0x7fe8641576f0 2023-12-26T23:10:03.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.705+0000 7fe8597fa700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe8480693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:03.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.705+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe864158510 con 0x7fe848066f80 2023-12-26T23:10:03.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.708+0000 7fe86979b700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe8480693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:03.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.713+0000 7fe86979b700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe8480693b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe860007fd0 tx=0x7fe860009040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:10:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.780+0000 7fe8597fa700 1 -- 172.21.15.131:0/3254035263 <== mgr.9277 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe864158510 con 0x7fe848066f80 2023-12-26T23:10:03.780 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path snap_test_dir1 2023-12-26T23:10:03.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 msgr2=0x7fe8480693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:03.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe8480693b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe860007fd0 tx=0x7fe860009040 comp rx=0 tx=0).stop 2023-12-26T23:10:03.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 msgr2=0x7fe8640aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:03.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 0x7fe8640aa880 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009f60 tx=0x7fe85c00b040 comp rx=0 tx=0).stop 2023-12-26T23:10:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 shutdown_connections 2023-12-26T23:10:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe848066f80 0x7fe8480693b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8641576f0 0x7fe8640aa880 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8640caf60 0x7fe864153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 --2- 172.21.15.131:0/3254035263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe864157320 0x7fe8640c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 >> 172.21.15.131:0/3254035263 conn(0x7fe86406b1c0 msgr2=0x7fe8640cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:10:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 shutdown_connections 2023-12-26T23:10:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:03.782+0000 7fe86b9ff700 1 -- 172.21.15.131:0/3254035263 wait complete. 2023-12-26T23:10:03.802 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-26T23:10:03.803 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-12-26T23:10:04.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.006+0000 7f028d3ff700 1 Processor -- start 2023-12-26T23:10:04.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.006+0000 7f028d3ff700 1 -- start start 2023-12-26T23:10:04.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.007+0000 7f028d3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f02880c8bf0 0x7f02880b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.007+0000 7f028d3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0288069d10 0x7f02880b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.007+0000 7f028d3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02880caa10 con 0x7f02880c8820 2023-12-26T23:10:04.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.007+0000 7f028d3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0288157a80 con 0x7f0288069d10 2023-12-26T23:10:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.007+0000 7f028d3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02880cac50 con 0x7f02880c8bf0 2023-12-26T23:10:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.007+0000 7f02877fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0288069d10 0x7f02880b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.007+0000 7f02877fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0288069d10 0x7f02880b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54282/0 (socket says 172.21.15.131:54282) 2023-12-26T23:10:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f02877fe700 1 -- 172.21.15.131:0/3397958395 learned_addr learned my addr 172.21.15.131:0/3397958395 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:10:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f02867fc700 1 --2- 172.21.15.131:0/3397958395 >> v2:172.21.15.190:3300/0 conn(0x7f02880c8bf0 0x7f02880b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f02877fe700 1 -- 172.21.15.131:0/3397958395 >> v2:172.21.15.190:3300/0 conn(0x7f02880c8bf0 msgr2=0x7f02880b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:10:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f02877fe700 1 --2- 172.21.15.131:0/3397958395 >> v2:172.21.15.190:3300/0 conn(0x7f02880c8bf0 0x7f02880b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f02877fe700 1 -- 172.21.15.131:0/3397958395 >> v1:172.21.15.190:6789/0 conn(0x7f02880c8820 legacy=0x7f02880ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f02877fe700 1 -- 172.21.15.131:0/3397958395 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880caf30 con 0x7f0288069d10 2023-12-26T23:10:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f02867fc700 1 --2- 172.21.15.131:0/3397958395 >> v2:172.21.15.190:3300/0 conn(0x7f02880c8bf0 0x7f02880b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:10:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f02877fe700 1 --2- 172.21.15.131:0/3397958395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0288069d10 0x7f02880b1820 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f02880646b0 tx=0x7f0280009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaf3024c449aa7b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:10:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f0285ffb700 1 -- 172.21.15.131:0/3397958395 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000e070 con 0x7f0288069d10 2023-12-26T23:10:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f0285ffb700 1 -- 172.21.15.131:0/3397958395 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028000ba50 con 0x7f0288069d10 2023-12-26T23:10:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.008+0000 7f0285ffb700 1 -- 172.21.15.131:0/3397958395 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800092e0 con 0x7f0288069d10 2023-12-26T23:10:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.009+0000 7f028d3ff700 1 -- 172.21.15.131:0/3397958395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0288069d10 msgr2=0x7f02880b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.009+0000 7f028d3ff700 1 --2- 172.21.15.131:0/3397958395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0288069d10 0x7f02880b1820 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f02880646b0 tx=0x7f0280009840 comp rx=0 tx=0).stop 2023-12-26T23:10:04.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.009+0000 7f028d3ff700 1 -- 172.21.15.131:0/3397958395 shutdown_connections 2023-12-26T23:10:04.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.009+0000 7f028d3ff700 1 --2- 172.21.15.131:0/3397958395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0288069d10 0x7f02880b1820 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.009+0000 7f028d3ff700 1 --2- 172.21.15.131:0/3397958395 >> v2:172.21.15.190:3300/0 conn(0x7f02880c8bf0 0x7f02880b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.009+0000 7f028d3ff700 1 -- 172.21.15.131:0/3397958395 >> 172.21.15.131:0/3397958395 conn(0x7f0288068040 msgr2=0x7f0288069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:10:04.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.009+0000 7f028d3ff700 1 -- 172.21.15.131:0/3397958395 shutdown_connections 2023-12-26T23:10:04.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.009+0000 7f028d3ff700 1 -- 172.21.15.131:0/3397958395 wait complete. 2023-12-26T23:10:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.011+0000 7f028d3ff700 1 Processor -- start 2023-12-26T23:10:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.011+0000 7f028d3ff700 1 -- start start 2023-12-26T23:10:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f028d3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0288069d10 0x7f02880c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f028d3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02880c8820 0x7f028815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f028d3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02880c8bf0 0x7f0288152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f028d3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028815c500 con 0x7f02880c8820 2023-12-26T23:10:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f028d3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028815d340 con 0x7f0288069d10 2023-12-26T23:10:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f028d3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028815c730 con 0x7f02880c8bf0 2023-12-26T23:10:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f02867fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02880c8820 0x7f028815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f02867fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02880c8820 0x7f028815bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54286/0 (socket says 172.21.15.131:54286) 2023-12-26T23:10:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f02867fc700 1 -- 172.21.15.131:0/3419818355 learned_addr learned my addr 172.21.15.131:0/3419818355 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:10:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.012+0000 7f0286ffd700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0288069d10 0x7f02880c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f02877fe700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02880c8bf0 0x7f0288152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f02867fc700 1 -- 172.21.15.131:0/3419818355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02880c8bf0 msgr2=0x7f0288152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f02867fc700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02880c8bf0 0x7f0288152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f02867fc700 1 -- 172.21.15.131:0/3419818355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0288069d10 msgr2=0x7f02880c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f02867fc700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0288069d10 0x7f02880c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f02867fc700 1 -- 172.21.15.131:0/3419818355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280009ae0 con 0x7f02880c8820 2023-12-26T23:10:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f02867fc700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02880c8820 0x7f028815bfc0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e8a0 tx=0x7f027c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:10:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f026effd700 1 -- 172.21.15.131:0/3419818355 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c013410 con 0x7f02880c8820 2023-12-26T23:10:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f026effd700 1 -- 172.21.15.131:0/3419818355 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c013570 con 0x7f02880c8820 2023-12-26T23:10:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f026effd700 1 -- 172.21.15.131:0/3419818355 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c013930 con 0x7f02880c8820 2023-12-26T23:10:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.013+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028815ca10 con 0x7f02880c8820 2023-12-26T23:10:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.014+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028815cfd0 con 0x7f02880c8820 2023-12-26T23:10:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.015+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02880657e0 con 0x7f02880c8820 2023-12-26T23:10:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.015+0000 7f026effd700 1 -- 172.21.15.131:0/3419818355 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 100) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f027c013ca0 con 0x7f02880c8820 2023-12-26T23:10:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.016+0000 7f026effd700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0274066dd0 0x7f0274069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.016+0000 7f026effd700 1 -- 172.21.15.131:0/3419818355 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6455+0+0 (secure 0 0 0) 0x7f027c0842b0 con 0x7f02880c8820 2023-12-26T23:10:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.016+0000 7f0286ffd700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0274066dd0 0x7f0274069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.017+0000 7f0286ffd700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0274066dd0 0x7f0274069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0278003a80 tx=0x7f0278007ec0 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:10:04.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.020+0000 7f026effd700 1 -- 172.21.15.131:0/3419818355 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f027c054ea0 con 0x7f02880c8820 2023-12-26T23:10:04.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.239+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f028815d480 con 0x7f0274066dd0 2023-12-26T23:10:04.290 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.290+0000 7f026effd700 1 -- 172.21.15.131:0/3419818355 <== mgr.9277 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f0278003710 con 0x7f0274066dd0 2023-12-26T23:10:04.290 INFO:teuthology.orchestra.run.smithi131.stdout:Retention added to path snap_test_dir1 2023-12-26T23:10:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0274066dd0 msgr2=0x7f0274069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0274066dd0 0x7f0274069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0278003a80 tx=0x7f0278007ec0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02880c8820 msgr2=0x7f028815bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02880c8820 0x7f028815bfc0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e8a0 tx=0x7f027c008040 comp rx=0 tx=0).stop 2023-12-26T23:10:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 shutdown_connections 2023-12-26T23:10:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0274066dd0 0x7f0274069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02880c8bf0 0x7f0288152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0288069d10 0x7f02880c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 --2- 172.21.15.131:0/3419818355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02880c8820 0x7f028815bfc0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 >> 172.21.15.131:0/3419818355 conn(0x7f0288068040 msgr2=0x7f028806d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:10:04.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 shutdown_connections 2023-12-26T23:10:04.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.292+0000 7f028d3ff700 1 -- 172.21.15.131:0/3419818355 wait complete. 2023-12-26T23:10:04.311 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-12-26T23:10:04.311 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-26-23_11_00 in ~56s... 2023-12-26T23:10:04.311 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-12-26T23:10:04.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-12-26T23:10:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.507+0000 7f3af13ff700 1 Processor -- start 2023-12-26T23:10:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.507+0000 7f3af13ff700 1 -- start start 2023-12-26T23:10:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.508+0000 7f3af13ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 0x7f3aec0c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3af13ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3aec0cec80 0x7f3aec0cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3af13ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3aec0cd0a0 0x7f3aec0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3af13ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aec0cf660 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3af13ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aec0cb2a0 con 0x7f3aec0cec80 2023-12-26T23:10:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3af13ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec0cbb00 con 0x7f3aec159810 2023-12-26T23:10:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3aec0cd0a0 0x7f3aec0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeaffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 0x7f3aec0c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3aec0cd0a0 0x7f3aec0cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54292/0 (socket says 172.21.15.131:54292) 2023-12-26T23:10:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeaffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 0x7f3aec0c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:49150/0 (socket says 172.21.15.131:49150) 2023-12-26T23:10:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeb7fe700 1 -- 172.21.15.131:0/3319480693 learned_addr learned my addr 172.21.15.131:0/3319480693 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:10:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aea7fc700 1 --2- 172.21.15.131:0/3319480693 >> v2:172.21.15.190:3300/0 conn(0x7f3aec0cec80 0x7f3aec0cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeb7fe700 1 -- 172.21.15.131:0/3319480693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 msgr2=0x7f3aec0c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:10:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeb7fe700 1 --2- 172.21.15.131:0/3319480693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 0x7f3aec0c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeb7fe700 1 -- 172.21.15.131:0/3319480693 >> v2:172.21.15.190:3300/0 conn(0x7f3aec0cec80 msgr2=0x7f3aec0cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:10:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeb7fe700 1 --2- 172.21.15.131:0/3319480693 >> v2:172.21.15.190:3300/0 conn(0x7f3aec0cec80 0x7f3aec0cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aeb7fe700 1 -- 172.21.15.131:0/3319480693 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0ce040 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.509+0000 7f3aea7fc700 1 --2- 172.21.15.131:0/3319480693 >> v2:172.21.15.190:3300/0 conn(0x7f3aec0cec80 0x7f3aec0cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:10:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.510+0000 7f3aeb7fe700 1 --2- 172.21.15.131:0/3319480693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 0x7f3aec0cd4e0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3aec088a50 tx=0x7f3ae4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03a553fea7f7885 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:10:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.510+0000 7f3ae9ffb700 1 -- 172.21.15.131:0/3319480693 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400e070 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.510+0000 7f3ae9ffb700 1 -- 172.21.15.131:0/3319480693 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae400ba20 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.510+0000 7f3ae9ffb700 1 -- 172.21.15.131:0/3319480693 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40092e0 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.511+0000 7f3af13ff700 1 -- 172.21.15.131:0/3319480693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 msgr2=0x7f3aec0cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.511+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3319480693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 0x7f3aec0cd4e0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3aec088a50 tx=0x7f3ae4009840 comp rx=0 tx=0).stop 2023-12-26T23:10:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.511+0000 7f3af13ff700 1 -- 172.21.15.131:0/3319480693 shutdown_connections 2023-12-26T23:10:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.511+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3319480693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 0x7f3aec0cd4e0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.511+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3319480693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 0x7f3aec0c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.511+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3319480693 >> v2:172.21.15.190:3300/0 conn(0x7f3aec0cec80 0x7f3aec0cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.511+0000 7f3af13ff700 1 -- 172.21.15.131:0/3319480693 >> 172.21.15.131:0/3319480693 conn(0x7f3aec099950 msgr2=0x7f3aec06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:10:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.511+0000 7f3af13ff700 1 -- 172.21.15.131:0/3319480693 shutdown_connections 2023-12-26T23:10:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.511+0000 7f3af13ff700 1 -- 172.21.15.131:0/3319480693 wait complete. 2023-12-26T23:10:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.512+0000 7f3af13ff700 1 Processor -- start 2023-12-26T23:10:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.512+0000 7f3af13ff700 1 -- start start 2023-12-26T23:10:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.513+0000 7f3af13ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 0x7f3aec1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.513+0000 7f3af13ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aec0cec80 0x7f3aec0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.513+0000 7f3af13ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 0x7f3aec15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.513+0000 7f3af13ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec1558f0 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.513+0000 7f3af13ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec155d90 con 0x7f3aec0cec80 2023-12-26T23:10:04.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.513+0000 7f3af13ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec156070 con 0x7f3aec159810 2023-12-26T23:10:04.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 0x7f3aec1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 0x7f3aec1553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54296/0 (socket says 172.21.15.131:54296) 2023-12-26T23:10:04.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeaffd700 1 -- 172.21.15.131:0/3700699200 learned_addr learned my addr 172.21.15.131:0/3700699200 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:10:04.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aea7fc700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aec0cec80 0x7f3aec0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeb7fe700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 0x7f3aec15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeaffd700 1 -- 172.21.15.131:0/3700699200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 msgr2=0x7f3aec15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeaffd700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 0x7f3aec15bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeaffd700 1 -- 172.21.15.131:0/3700699200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aec0cec80 msgr2=0x7f3aec0b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeaffd700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aec0cec80 0x7f3aec0b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeaffd700 1 -- 172.21.15.131:0/3700699200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae4009ae0 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aea7fc700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aec0cec80 0x7f3aec0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:10:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3aeaffd700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 0x7f3aec1553b0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00ec80 tx=0x7f3adc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:10:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3ad2ffd700 1 -- 172.21.15.131:0/3700699200 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00c9d0 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.514+0000 7f3ad2ffd700 1 -- 172.21.15.131:0/3700699200 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc00cb30 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.515+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aec156350 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.523 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.515+0000 7f3ad2ffd700 1 -- 172.21.15.131:0/3700699200 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc013470 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.515+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aec156910 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.516+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aec156ad0 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.516+0000 7f3ad2ffd700 1 -- 172.21.15.131:0/3700699200 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 100) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f3adc013820 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.517+0000 7f3ad2ffd700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3ad8066dd0 0x7f3ad8069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:10:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.517+0000 7f3ad2ffd700 1 -- 172.21.15.131:0/3700699200 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6455+0+0 (secure 0 0 0) 0x7f3adc084d40 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.517+0000 7f3aea7fc700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3ad8066dd0 0x7f3ad8069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:10:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.518+0000 7f3aea7fc700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3ad8066dd0 0x7f3ad8069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006f00 tx=0x7f3ae0008040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:10:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.521+0000 7f3ad2ffd700 1 -- 172.21.15.131:0/3700699200 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3adc051ea0 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.714+0000 7f3ad2ffd700 1 -- 172.21.15.131:0/3700699200 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 101) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f3adc052270 con 0x7f3aec0cd0a0 2023-12-26T23:10:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.741+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3aec0717f0 con 0x7f3ad8066dd0 2023-12-26T23:10:04.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.749+0000 7f3ad2ffd700 1 -- 172.21.15.131:0/3700699200 <== mgr.9277 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f3aec0717f0 con 0x7f3ad8066dd0 2023-12-26T23:10:04.749 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:10:04.749 INFO:teuthology.orchestra.run.smithi131.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-26T23:10:04.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3ad8066dd0 msgr2=0x7f3ad8069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3ad8066dd0 0x7f3ad8069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006f00 tx=0x7f3ae0008040 comp rx=0 tx=0).stop 2023-12-26T23:10:04.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 msgr2=0x7f3aec1553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:10:04.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 0x7f3aec1553b0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00ec80 tx=0x7f3adc009120 comp rx=0 tx=0).stop 2023-12-26T23:10:04.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 shutdown_connections 2023-12-26T23:10:04.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3ad8066dd0 0x7f3ad8069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aec159810 0x7f3aec15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3aec0cd0a0 0x7f3aec1553b0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 --2- 172.21.15.131:0/3700699200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aec0cec80 0x7f3aec0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:10:04.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 >> 172.21.15.131:0/3700699200 conn(0x7f3aec099950 msgr2=0x7f3aec0c6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:10:04.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 shutdown_connections 2023-12-26T23:10:04.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:10:04.751+0000 7f3af13ff700 1 -- 172.21.15.131:0/3700699200 wait complete. 2023-12-26T23:10:04.770 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-26T23:10:04.770 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-12-26T23:10:04.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:04.770 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:04.815 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:04.816 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:04.816 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:05.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:05.817 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:05.865 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:05.865 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:05.865 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:06.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:06.866 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:06.917 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:06.917 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:06.917 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:07.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:07.919 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:07.970 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:07.970 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:07.970 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:08.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:08.972 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:09.024 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:09.024 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:09.024 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:10.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:10.026 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:10.077 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:10.077 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:10.077 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:11.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:11.078 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:11.129 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:11.129 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:11.129 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:12.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:12.130 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:12.181 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:12.181 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:12.181 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:13.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:13.182 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:13.236 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:13.236 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:13.236 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:14.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:14.239 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:14.291 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:14.291 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:14.291 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:15.167 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:10:15.174 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:10:15.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:15.292 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:15.338 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:15.339 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:15.339 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:16.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:16.340 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:16.391 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:16.391 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:16.391 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:17.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:17.393 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:17.443 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:17.443 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:17.443 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:18.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:18.445 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:18.495 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:18.496 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:18.496 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:19.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:19.497 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:19.547 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:19.547 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:19.548 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:20.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:20.549 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:20.599 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:20.599 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:20.599 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:21.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:21.600 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:21.651 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:21.651 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:21.651 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:22.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:22.653 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:22.704 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:22.704 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:22.704 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:23.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:23.705 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:23.756 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:23.757 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:23.757 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:24.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:24.758 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:24.809 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:24.809 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:24.810 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:25.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:25.811 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:25.862 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:25.862 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:25.862 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:26.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:26.863 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:26.914 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:26.914 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:26.914 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:27.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:27.916 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:27.967 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:27.967 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:27.968 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:28.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:28.968 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:29.020 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:29.020 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:29.020 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:30.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:30.021 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:30.071 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:30.071 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:30.071 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:31.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:31.072 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:31.122 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:31.122 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:31.122 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:32.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:32.124 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:32.175 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:32.175 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:32.175 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:33.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:33.176 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:33.230 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:33.230 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:33.230 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:34.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:34.232 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:34.283 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:34.283 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:34.283 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:35.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:35.287 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:35.337 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:35.338 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:35.338 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:36.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:36.340 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:36.391 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:36.391 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:36.391 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:37.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:37.392 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:37.443 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:37.443 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:37.444 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:38.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:38.445 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:38.495 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:38.495 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:38.495 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:39.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:39.497 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:39.549 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:39.549 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:39.549 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:40.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:40.550 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:40.603 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:40.603 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:40.604 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:41.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:41.605 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:41.655 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:41.655 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:41.655 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:42.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:42.657 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:42.709 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:42.709 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:42.709 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:43.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:43.709 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:43.760 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:43.760 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:43.760 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:44.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:44.762 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:44.813 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:44.813 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:44.813 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:45.438 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:10:45.444 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:10:45.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:45.815 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:45.863 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:45.864 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:45.864 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:46.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:46.865 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:46.915 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:46.915 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:46.915 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:47.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:47.916 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:47.967 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:47.967 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:47.967 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:48.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:48.969 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:49.019 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:49.019 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:49.020 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:50.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:50.021 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:50.071 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:50.071 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:50.071 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:51.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:51.072 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:51.123 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:51.123 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:51.123 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:52.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:52.125 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:52.175 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:52.175 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:52.175 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:53.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:53.177 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:53.227 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:53.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:53.228 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:54.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:54.229 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:54.279 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:54.279 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:54.279 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:55.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:55.280 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:55.330 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:55.330 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:55.331 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:56.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:56.332 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:56.383 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:56.383 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:56.383 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:57.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:57.385 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:57.435 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:57.435 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:57.435 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:58.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:58.436 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:58.487 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:58.487 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:58.487 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:10:59.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:10:59.489 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:10:59.539 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-26T23:10:59.540 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:10:59.540 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:00.302 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:11:00.302+0000 7fb4c0455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:11:00.302 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:11:00.302+0000 7fb4c0455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:11:00.302 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:11:00.302+0000 7fb4c0455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:11:00.302 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:11:00.302+0000 7fb4c0455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:11:00.302 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:11:00.302+0000 7fb4c0455700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:11:00.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:00.541 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:00.588 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:00.589 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:00.589 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:00.589 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:00.589 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-26-23_11_00_UTC'] 2023-12-26T23:11:00.589 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703632204.3114383, now = 1703632260.5896592, timo = 56 2023-12-26T23:11:01.590 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-12-26-23_11_00 in ~116s... 2023-12-26T23:11:01.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:01.591 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:01.640 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:01.642 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:01.642 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:01.642 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:02.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:02.643 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:02.692 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:02.694 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:02.694 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:02.694 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:03.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:03.696 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:03.746 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:03.748 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:03.748 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:03.748 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:04.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:04.749 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:04.799 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:04.800 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:04.800 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:04.800 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:05.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:05.802 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:05.851 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:05.852 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:05.852 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:05.852 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:06.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:06.854 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:06.904 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:06.906 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:06.906 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:06.906 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:07.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:07.907 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:07.957 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:07.958 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:07.958 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:07.958 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:08.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:08.959 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:09.009 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:09.010 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:09.010 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:09.010 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:10.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:10.012 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:10.061 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:10.062 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:10.062 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:10.062 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:11.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:11.064 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:11.113 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:11.114 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:11.114 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:11.114 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:12.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:12.116 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:12.165 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:12.167 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:12.167 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:12.167 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:13.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:13.168 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:13.218 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:13.220 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:13.220 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:13.220 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:14.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:14.221 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:14.272 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:14.273 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:14.274 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:14.274 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:15.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:15.275 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:15.325 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:15.326 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:15.326 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:15.326 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:15.553 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:11:15.559 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:11:16.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:16.327 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:16.377 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:16.378 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:16.378 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:16.379 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:17.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:17.380 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:17.429 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:17.430 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:17.430 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:17.430 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:18.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:18.431 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:18.480 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:18.481 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:18.481 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:18.481 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:19.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:19.482 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:19.531 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:19.532 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:19.532 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:19.532 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:20.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:20.533 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:20.580 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:20.581 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:20.581 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:20.581 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:21.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:21.582 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:21.632 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:21.633 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:21.633 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:21.633 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:22.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:22.634 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:22.683 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:22.684 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:22.684 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:22.684 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:23.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:23.686 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:23.736 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:23.737 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:23.737 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:23.737 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:24.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:24.738 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:24.788 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:24.789 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:24.790 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:24.790 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:25.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:25.791 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:25.840 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:25.841 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:25.841 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:25.842 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:26.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:26.842 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:26.893 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:26.894 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:26.894 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:26.894 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:27.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:27.895 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:27.945 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:27.946 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:27.947 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:27.947 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:28.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:28.948 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:28.997 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:28.999 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:28.999 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:28.999 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:30.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:30.000 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:30.049 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:30.051 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:30.051 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:30.051 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:31.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:31.052 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:31.101 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:31.102 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:31.102 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:31.102 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:32.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:32.103 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:32.151 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:32.152 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:32.153 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:32.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:33.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:33.154 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:33.203 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:33.205 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:33.205 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:33.205 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:34.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:34.206 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:34.256 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:34.257 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:34.257 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:34.257 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:35.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:35.259 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:35.308 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:35.309 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:35.309 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:35.309 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:36.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:36.310 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:36.359 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:36.360 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:36.360 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:36.361 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:37.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:37.362 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:37.410 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:37.411 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:37.412 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:37.412 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:38.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:38.413 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:38.463 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:38.464 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:38.464 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:38.464 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:39.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:39.465 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:39.514 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:39.515 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:39.515 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:39.515 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:40.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:40.516 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:40.565 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:40.567 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:40.567 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:40.567 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:41.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:41.568 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:41.617 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:41.618 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:41.618 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:41.618 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:42.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:42.619 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:42.668 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:42.669 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:42.669 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:42.670 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:43.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:43.670 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:43.719 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:43.720 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:43.720 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:43.720 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:44.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:44.722 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:44.770 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:44.771 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:44.771 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:44.771 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:45.686 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:11:45.691 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:11:45.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:45.773 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:45.819 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:45.820 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:45.820 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:45.820 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:46.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:46.821 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:46.871 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:46.872 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:46.872 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:46.872 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:47.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:47.873 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:47.922 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:47.924 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:47.924 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:47.924 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:48.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:48.925 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:48.975 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:48.976 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:48.976 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:48.976 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:49.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:49.978 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:50.027 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:50.029 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:50.029 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:50.029 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:51.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:51.030 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:51.080 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:51.081 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:51.081 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:51.081 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:52.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:52.082 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:52.132 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:52.133 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:52.133 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:52.133 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:53.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:53.134 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:53.183 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:53.184 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:53.184 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:53.184 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:54.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:54.185 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:54.234 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:54.236 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:54.236 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:54.236 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:55.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:55.237 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:55.286 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:55.288 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:55.288 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:55.288 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:56.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:56.289 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:56.338 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:56.339 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:56.339 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:56.339 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:57.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:57.341 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:57.390 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:57.391 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:57.392 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:57.392 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:58.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:58.393 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:58.442 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:58.443 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:58.443 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:58.443 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:11:59.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:11:59.445 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:11:59.493 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_11_00_UTC 2023-12-26T23:11:59.495 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:11:59.495 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-26T23:11:59.495 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-26T23:12:00.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:12:00.497 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:12:00.542 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_12_00_UTC 2023-12-26T23:12:00.543 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-26-23_12_00_UTC'} 2023-12-26T23:12:00.543 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-26-23_12_00_UTC'} 2023-12-26T23:12:00.543 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-12-26-23_11_00_UTC'} 2023-12-26T23:12:00.543 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-12-26-23_11_00_UTC'] 2023-12-26T23:12:00.543 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703632204.3114383, now = 1703632320.5435207, timo = 116 2023-12-26T23:12:01.544 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-12-26T23:12:01.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.749+0000 7fdffa7ff700 1 Processor -- start 2023-12-26T23:12:01.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.750+0000 7fdffa7ff700 1 -- start start 2023-12-26T23:12:01.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.751+0000 7fdffa7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdff4143130 0x7fdff406a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:01.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.751+0000 7fdffa7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff406e660 con 0x7fdff4142d60 2023-12-26T23:12:01.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.751+0000 7fdffa7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff406ac80 con 0x7fdff40c78c0 2023-12-26T23:12:01.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.751+0000 7fdffa7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff40cdb50 con 0x7fdff4143130 2023-12-26T23:12:01.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.751+0000 7fdff3fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdff4142d60 0x7fdff40cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35730/0 (socket says 172.21.15.131:35730) 2023-12-26T23:12:01.750 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.751+0000 7fdff3fff700 1 -- 172.21.15.131:0/2612917251 learned_addr learned my addr 172.21.15.131:0/2612917251 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.751+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474510926 0 0) 0x7fdff406e660 con 0x7fdff4142d60 2023-12-26T23:12:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.751+0000 7fdff37fe700 1 --2- 172.21.15.131:0/2612917251 >> v2:172.21.15.190:3300/0 conn(0x7fdff4143130 0x7fdff406a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.751+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfdc003570 con 0x7fdff4142d60 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4239762186 0 0) 0x7fdfdc003570 con 0x7fdff4142d60 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 >> v2:172.21.15.190:3300/0 conn(0x7fdff4143130 msgr2=0x7fdff406a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 --2- 172.21.15.131:0/2612917251 >> v2:172.21.15.190:3300/0 conn(0x7fdff4143130 0x7fdff406a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 >> v1:172.21.15.190:6789/0 conn(0x7fdff40c78c0 legacy=0x7fdff406eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff406af60 con 0x7fdff4142d60 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fdfe4001470 con 0x7fdff4142d60 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 >> v1:172.21.15.131:6789/0 conn(0x7fdff4142d60 legacy=0x7fdff40cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc003ab0 0x7fdfdc003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfdc004810 0x7fdfdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc005210 0x7fdfdc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc00c130 con 0x7fdfdc005210 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc00c270 con 0x7fdfdc003ab0 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc00c430 con 0x7fdfdc004810 2023-12-26T23:12:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff8d9c700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc005210 0x7fdfdc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff3fff700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc003ab0 0x7fdfdc003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.752+0000 7fdff37fe700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfdc004810 0x7fdfdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff8d9c700 1 -- 172.21.15.131:0/2612917251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfdc004810 msgr2=0x7fdfdc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff8d9c700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfdc004810 0x7fdfdc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff8d9c700 1 -- 172.21.15.131:0/2612917251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc003ab0 msgr2=0x7fdfdc003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff8d9c700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfdc003ab0 0x7fdfdc003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff8d9c700 1 -- 172.21.15.131:0/2612917251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc003570 con 0x7fdfdc005210 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff8d9c700 1 -- 172.21.15.131:0/2612917251 reap_dead start 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff8d9c700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc005210 0x7fdfdc00ba60 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fdfec001fc0 tx=0x7fdfec001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670cca9cd56258af server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec001950 con 0x7fdfdc005210 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec00c8b0 con 0x7fdfdc005210 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.753+0000 7fdff2ffd700 1 -- 172.21.15.131:0/2612917251 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00cc90 con 0x7fdfdc005210 2023-12-26T23:12:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.754+0000 7fdffa7ff700 1 -- 172.21.15.131:0/2612917251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc005210 msgr2=0x7fdfdc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:01.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.754+0000 7fdffa7ff700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc005210 0x7fdfdc00ba60 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fdfec001fc0 tx=0x7fdfec001640 comp rx=0 tx=0).stop 2023-12-26T23:12:01.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.754+0000 7fdffa7ff700 1 -- 172.21.15.131:0/2612917251 shutdown_connections 2023-12-26T23:12:01.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.754+0000 7fdffa7ff700 1 --2- 172.21.15.131:0/2612917251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfdc005210 0x7fdfdc00ba60 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:01.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.754+0000 7fdffa7ff700 1 -- 172.21.15.131:0/2612917251 >> 172.21.15.131:0/2612917251 conn(0x7fdff4069140 msgr2=0x7fdff40cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:01.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.754+0000 7fdffa7ff700 1 -- 172.21.15.131:0/2612917251 shutdown_connections 2023-12-26T23:12:01.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.754+0000 7fdffa7ff700 1 -- 172.21.15.131:0/2612917251 wait complete. 2023-12-26T23:12:01.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.755+0000 7fdffa7ff700 1 Processor -- start 2023-12-26T23:12:01.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.755+0000 7fdffa7ff700 1 -- start start 2023-12-26T23:12:01.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.756+0000 7fdffa7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfdc005210 0x7fdff41695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:01.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.756+0000 7fdffa7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff413abd0 0x7fdff40b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.756+0000 7fdffa7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff413b040 0x7fdff406f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.756+0000 7fdffa7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff416cf30 con 0x7fdff413abd0 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.756+0000 7fdffa7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff416d210 con 0x7fdff413b040 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.756+0000 7fdffa7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff416d4f0 con 0x7fdfdc005210 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff413abd0 0x7fdff40b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff413abd0 0x7fdff40b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56988/0 (socket says 172.21.15.131:56988) 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff37fe700 1 -- 172.21.15.131:0/46338941 learned_addr learned my addr 172.21.15.131:0/46338941 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff3fff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfdc005210 0x7fdff41695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff8d9c700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff413b040 0x7fdff406f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff37fe700 1 -- 172.21.15.131:0/46338941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfdc005210 msgr2=0x7fdff41695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff37fe700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfdc005210 0x7fdff41695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff37fe700 1 -- 172.21.15.131:0/46338941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff413b040 msgr2=0x7fdff406f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff37fe700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff413b040 0x7fdff406f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:01.759 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff37fe700 1 -- 172.21.15.131:0/46338941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff416d7d0 con 0x7fdff413abd0 2023-12-26T23:12:01.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdff37fe700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff413abd0 0x7fdff40b1ca0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8006ac0 tx=0x7fdfe80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:01.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdfdbfff700 1 -- 172.21.15.131:0/46338941 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8011ec0 con 0x7fdff413abd0 2023-12-26T23:12:01.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdfdbfff700 1 -- 172.21.15.131:0/46338941 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe800f940 con 0x7fdff413abd0 2023-12-26T23:12:01.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdfdbfff700 1 -- 172.21.15.131:0/46338941 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800fcc0 con 0x7fdff413abd0 2023-12-26T23:12:01.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.757+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff4169b30 con 0x7fdff413abd0 2023-12-26T23:12:01.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.758+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff40b21e0 con 0x7fdff413abd0 2023-12-26T23:12:01.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.759+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff413b560 con 0x7fdff413abd0 2023-12-26T23:12:01.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.760+0000 7fdfdbfff700 1 -- 172.21.15.131:0/46338941 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 103) v1 ==== 84873+0+0 (secure 0 0 0) 0x7fdfe8014650 con 0x7fdff413abd0 2023-12-26T23:12:01.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.761+0000 7fdfdbfff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdfe00670a0 0x7fdfe0069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:01.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.761+0000 7fdfdbfff700 1 -- 172.21.15.131:0/46338941 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6484+0+0 (secure 0 0 0) 0x7fdfe8018030 con 0x7fdff413abd0 2023-12-26T23:12:01.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.761+0000 7fdff3fff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdfe00670a0 0x7fdfe0069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:01.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.761+0000 7fdff3fff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdfe00670a0 0x7fdfe0069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4000e50 tx=0x7fdff4159ca0 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.765+0000 7fdfdbfff700 1 -- 172.21.15.131:0/46338941 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdff413b560 con 0x7fdff413abd0 2023-12-26T23:12:01.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:01.983+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdff416d380 con 0x7fdfe00670a0 2023-12-26T23:12:02.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.048+0000 7fdfdbfff700 1 -- 172.21.15.131:0/46338941 <== mgr.9277 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fdff416d380 con 0x7fdfe00670a0 2023-12-26T23:12:02.047 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path snap_test_dir1 2023-12-26T23:12:02.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.050+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdfe00670a0 msgr2=0x7fdfe0069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:02.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.050+0000 7fdffa7ff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdfe00670a0 0x7fdfe0069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4000e50 tx=0x7fdff4159ca0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.050 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff413abd0 msgr2=0x7fdff40b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff413abd0 0x7fdff40b1ca0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8006ac0 tx=0x7fdfe80091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 shutdown_connections 2023-12-26T23:12:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdfe00670a0 0x7fdfe0069550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff413b040 0x7fdff406f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfdc005210 0x7fdff41695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 --2- 172.21.15.131:0/46338941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff413abd0 0x7fdff40b1ca0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 >> 172.21.15.131:0/46338941 conn(0x7fdff4069140 msgr2=0x7fdff40c2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 shutdown_connections 2023-12-26T23:12:02.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.051+0000 7fdffa7ff700 1 -- 172.21.15.131:0/46338941 wait complete. 2023-12-26T23:12:02.068 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-26T23:12:02.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:12:02.068 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:12:02.112 INFO:teuthology.orchestra.run.smithi131.stdout:scheduled-2023-12-26-23_12_00_UTC 2023-12-26T23:12:02.113 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-26-23_12_00_UTC 2023-12-26T23:12:02.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:12:02.113 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-26-23_12_00_UTC) 2023-12-26T23:12:02.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:12:02.203 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-26T23:12:02.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T23:12:02.480 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.481+0000 7f1130fff700 1 Processor -- start 2023-12-26T23:12:02.481 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.481+0000 7f1130fff700 1 -- start start 2023-12-26T23:12:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.482+0000 7f1130fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159920 0x7f112c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.482+0000 7f1130fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f112c155be0 0x7f112c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.482+0000 7f1130fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112c0c8bf0 con 0x7f112c159cf0 2023-12-26T23:12:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.482+0000 7f1130fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112c0c8e60 con 0x7f112c155be0 2023-12-26T23:12:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.483+0000 7f1130fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0c9140 con 0x7f112c159920 2023-12-26T23:12:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.483+0000 7f112ad9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f112c155be0 0x7f112c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.483+0000 7f112ad9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f112c155be0 0x7f112c0b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56994/0 (socket says 172.21.15.131:56994) 2023-12-26T23:12:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.483+0000 7f112ad9d700 1 -- 172.21.15.131:0/4129390347 learned_addr learned my addr 172.21.15.131:0/4129390347 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.483+0000 7f112a59c700 1 --2- 172.21.15.131:0/4129390347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159920 0x7f112c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.483+0000 7f112ad9d700 1 -- 172.21.15.131:0/4129390347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159920 msgr2=0x7f112c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.483+0000 7f112ad9d700 1 --2- 172.21.15.131:0/4129390347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159920 0x7f112c0b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.483+0000 7f112ad9d700 1 -- 172.21.15.131:0/4129390347 >> v1:172.21.15.190:6789/0 conn(0x7f112c159cf0 legacy=0x7f112c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.483+0000 7f112ad9d700 1 -- 172.21.15.131:0/4129390347 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c0c9420 con 0x7f112c155be0 2023-12-26T23:12:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f112ad9d700 1 --2- 172.21.15.131:0/4129390347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 0x7f112c0b0620 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1124009f20 tx=0x7f1124009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f327185b4be242 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1129d9b700 1 -- 172.21.15.131:0/4129390347 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11240092e0 con 0x7f112c155be0 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1129d9b700 1 -- 172.21.15.131:0/4129390347 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112400ba40 con 0x7f112c155be0 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1129d9b700 1 -- 172.21.15.131:0/4129390347 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400be00 con 0x7f112c155be0 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1130fff700 1 -- 172.21.15.131:0/4129390347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 msgr2=0x7f112c0b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1130fff700 1 --2- 172.21.15.131:0/4129390347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 0x7f112c0b0620 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1124009f20 tx=0x7f1124009840 comp rx=0 tx=0).stop 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1130fff700 1 -- 172.21.15.131:0/4129390347 shutdown_connections 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1130fff700 1 --2- 172.21.15.131:0/4129390347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 0x7f112c0b0620 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1130fff700 1 --2- 172.21.15.131:0/4129390347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159920 0x7f112c0b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1130fff700 1 -- 172.21.15.131:0/4129390347 >> 172.21.15.131:0/4129390347 conn(0x7f112c068040 msgr2=0x7f112c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.484+0000 7f1130fff700 1 -- 172.21.15.131:0/4129390347 shutdown_connections 2023-12-26T23:12:02.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.485+0000 7f1130fff700 1 -- 172.21.15.131:0/4129390347 wait complete. 2023-12-26T23:12:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.486+0000 7f1130fff700 1 Processor -- start 2023-12-26T23:12:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.486+0000 7f1130fff700 1 -- start start 2023-12-26T23:12:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f1130fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 0x7f112c15d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f1130fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f112c159920 0x7f112c143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f1130fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159cf0 0x7f112c143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f1130fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c145640 con 0x7f112c155be0 2023-12-26T23:12:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f1130fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c144a30 con 0x7f112c159920 2023-12-26T23:12:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f1130fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c144d10 con 0x7f112c159cf0 2023-12-26T23:12:02.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f112a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 0x7f112c15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f112a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 0x7f112c15d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57010/0 (socket says 172.21.15.131:57010) 2023-12-26T23:12:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f112a59c700 1 -- 172.21.15.131:0/1811172766 learned_addr learned my addr 172.21.15.131:0/1811172766 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f112ad9d700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159cf0 0x7f112c143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.487+0000 7f1123fff700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f112c159920 0x7f112c143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f112a59c700 1 -- 172.21.15.131:0/1811172766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159cf0 msgr2=0x7f112c143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f112a59c700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159cf0 0x7f112c143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f112a59c700 1 -- 172.21.15.131:0/1811172766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f112c159920 msgr2=0x7f112c143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f112a59c700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f112c159920 0x7f112c143880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:02.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f112a59c700 1 -- 172.21.15.131:0/1811172766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1124009ae0 con 0x7f112c155be0 2023-12-26T23:12:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f112a59c700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 0x7f112c15d850 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f111c00d890 tx=0x7f111c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f11227fc700 1 -- 172.21.15.131:0/1811172766 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c0125f0 con 0x7f112c155be0 2023-12-26T23:12:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f11227fc700 1 -- 172.21.15.131:0/1811172766 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c012770 con 0x7f112c155be0 2023-12-26T23:12:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c144ff0 con 0x7f112c155be0 2023-12-26T23:12:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f11227fc700 1 -- 172.21.15.131:0/1811172766 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c012b30 con 0x7f112c155be0 2023-12-26T23:12:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.488+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c15dd90 con 0x7f112c155be0 2023-12-26T23:12:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.489+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112c15dfb0 con 0x7f112c155be0 2023-12-26T23:12:02.492 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.490+0000 7f11227fc700 1 -- 172.21.15.131:0/1811172766 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 103) v1 ==== 84873+0+0 (secure 0 0 0) 0x7f111c01e460 con 0x7f112c155be0 2023-12-26T23:12:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.491+0000 7f11227fc700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f110c0670a0 0x7f110c069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.491+0000 7f11227fc700 1 -- 172.21.15.131:0/1811172766 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f111c085090 con 0x7f112c155be0 2023-12-26T23:12:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.491+0000 7f1123fff700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f110c0670a0 0x7f110c069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:02.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.492+0000 7f1123fff700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f110c0670a0 0x7f110c069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f112c0c4340 tx=0x7f1114009060 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:02.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.495+0000 7f11227fc700 1 -- 172.21.15.131:0/1811172766 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f111c051e40 con 0x7f112c155be0 2023-12-26T23:12:02.749 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:02.749+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f112c0657e0 con 0x7f112c155be0 2023-12-26T23:12:03.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.516+0000 7f11227fc700 1 -- 172.21.15.131:0/1811172766 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f111c052210 con 0x7f112c155be0 2023-12-26T23:12:03.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.517+0000 7f11227fc700 1 -- 172.21.15.131:0/1811172766 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v104) v1 ==== 93+0+0 (secure 0 0 0) 0x7f111c0558d0 con 0x7f112c155be0 2023-12-26T23:12:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f110c0670a0 msgr2=0x7f110c069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f110c0670a0 0x7f110c069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f112c0c4340 tx=0x7f1114009060 comp rx=0 tx=0).stop 2023-12-26T23:12:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 msgr2=0x7f112c15d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 0x7f112c15d850 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f111c00d890 tx=0x7f111c008040 comp rx=0 tx=0).stop 2023-12-26T23:12:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 shutdown_connections 2023-12-26T23:12:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f110c0670a0 0x7f110c069550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f112c159cf0 0x7f112c143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f112c155be0 0x7f112c15d850 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 --2- 172.21.15.131:0/1811172766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f112c159920 0x7f112c143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 >> 172.21.15.131:0/1811172766 conn(0x7f112c068040 msgr2=0x7f112c0aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 shutdown_connections 2023-12-26T23:12:03.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:03.520+0000 7f1130fff700 1 -- 172.21.15.131:0/1811172766 wait complete. 2023-12-26T23:12:03.541 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T23:12:03.541 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:12:03.651 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:12:03.740 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T23:12:03.740 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T23:12:03.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:12:03.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T23:12:03.820 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T23:12:03.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:03.820 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T23:12:03.865 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:03.952 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:03.952+0000 7fd47a88c1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:12:03.977 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:03.977+0000 7fdbc28391c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:12:04.056 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:04.057+0000 7fd47a88c1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:12:04.087 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:04.087+0000 7fdbc28391c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:12:04.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.100+0000 7fdc019ff700 1 Processor -- start 2023-12-26T23:12:04.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.101+0000 7fdc019ff700 1 -- start start 2023-12-26T23:12:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.103+0000 7fdc019ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdbfc0cac30 0x7fdbfc155be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.103+0000 7fdc019ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbfc0c8bf0 con 0x7fdbfc0b07f0 2023-12-26T23:12:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.103+0000 7fdc019ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbfc0c8e60 con 0x7fdbfc159920 2023-12-26T23:12:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.103+0000 7fdc019ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbfc0c9140 con 0x7fdbfc0cac30 2023-12-26T23:12:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.103+0000 7fdbfa7fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdbfc0b07f0 0x7fdbfc1554d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35738/0 (socket says 172.21.15.131:35738) 2023-12-26T23:12:04.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.103+0000 7fdbfa7fc700 1 -- 172.21.15.131:0/3043519546 learned_addr learned my addr 172.21.15.131:0/3043519546 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.103+0000 7fdbfb7fe700 1 --2- 172.21.15.131:0/3043519546 >> v2:172.21.15.131:3300/0 conn(0x7fdbfc0cac30 0x7fdbfc155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbf9ffb700 1 -- 172.21.15.131:0/3043519546 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682874572 0 0) 0x7fdbfc0c8bf0 con 0x7fdbfc0b07f0 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbf9ffb700 1 -- 172.21.15.131:0/3043519546 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbe4003570 con 0x7fdbfc0b07f0 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbf9ffb700 1 -- 172.21.15.131:0/3043519546 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177070139 0 0) 0x7fdbfc0c8e60 con 0x7fdbfc159920 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbf9ffb700 1 -- 172.21.15.131:0/3043519546 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbfc0c8bf0 con 0x7fdbfc159920 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbfb7fe700 1 -- 172.21.15.131:0/3043519546 >> v1:172.21.15.190:6789/0 conn(0x7fdbfc159920 legacy=0x7fdbfc0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbfb7fe700 1 -- 172.21.15.131:0/3043519546 >> v1:172.21.15.131:6789/0 conn(0x7fdbfc0b07f0 legacy=0x7fdbfc1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbfb7fe700 1 -- 172.21.15.131:0/3043519546 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0c9420 con 0x7fdbfc0cac30 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbfb7fe700 1 --2- 172.21.15.131:0/3043519546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 0x7fdbfc155be0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009e60 tx=0x7fdbf4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202aa1e39d2ada8e server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:04.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbf9ffb700 1 -- 172.21.15.131:0/3043519546 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400e070 con 0x7fdbfc0cac30 2023-12-26T23:12:04.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbf9ffb700 1 -- 172.21.15.131:0/3043519546 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf400bab0 con 0x7fdbfc0cac30 2023-12-26T23:12:04.107 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.104+0000 7fdbf9ffb700 1 -- 172.21.15.131:0/3043519546 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400be70 con 0x7fdbfc0cac30 2023-12-26T23:12:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.105+0000 7fdc019ff700 1 -- 172.21.15.131:0/3043519546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 msgr2=0x7fdbfc155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.105+0000 7fdc019ff700 1 --2- 172.21.15.131:0/3043519546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 0x7fdbfc155be0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009e60 tx=0x7fdbf4009780 comp rx=0 tx=0).stop 2023-12-26T23:12:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.105+0000 7fdc019ff700 1 -- 172.21.15.131:0/3043519546 shutdown_connections 2023-12-26T23:12:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.105+0000 7fdc019ff700 1 --2- 172.21.15.131:0/3043519546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 0x7fdbfc155be0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.105+0000 7fdc019ff700 1 -- 172.21.15.131:0/3043519546 >> 172.21.15.131:0/3043519546 conn(0x7fdbfc068040 msgr2=0x7fdbfc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.105+0000 7fdc019ff700 1 -- 172.21.15.131:0/3043519546 shutdown_connections 2023-12-26T23:12:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.106+0000 7fdc019ff700 1 -- 172.21.15.131:0/3043519546 wait complete. 2023-12-26T23:12:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.107+0000 7fdc019ff700 1 Processor -- start 2023-12-26T23:12:04.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.107+0000 7fdc019ff700 1 -- start start 2023-12-26T23:12:04.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdc019ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 0x7fdbfc154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdc019ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc159920 0x7fdbfc0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdc019ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc06e990 0x7fdbfc0c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdc019ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc154e90 con 0x7fdbfc0cac30 2023-12-26T23:12:04.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdc019ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc0c5db0 con 0x7fdbfc06e990 2023-12-26T23:12:04.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdc019ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc0c6090 con 0x7fdbfc159920 2023-12-26T23:12:04.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdbfaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 0x7fdbfc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:04.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdbfaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 0x7fdbfc154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57034/0 (socket says 172.21.15.131:57034) 2023-12-26T23:12:04.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdbfaffd700 1 -- 172.21.15.131:0/4070330393 learned_addr learned my addr 172.21.15.131:0/4070330393 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:04.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdbfa7fc700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc159920 0x7fdbfc0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:04.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.108+0000 7fdbfb7fe700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc06e990 0x7fdbfc0c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:04.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdbfaffd700 1 -- 172.21.15.131:0/4070330393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc159920 msgr2=0x7fdbfc0acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdbfaffd700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc159920 0x7fdbfc0acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdbfaffd700 1 -- 172.21.15.131:0/4070330393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc06e990 msgr2=0x7fdbfc0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdbfaffd700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc06e990 0x7fdbfc0c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdbfaffd700 1 -- 172.21.15.131:0/4070330393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf4009a20 con 0x7fdbfc0cac30 2023-12-26T23:12:04.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdbfaffd700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 0x7fdbfc154950 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c580 tx=0x7fdbec007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:04.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdbe2ffd700 1 -- 172.21.15.131:0/4070330393 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec010510 con 0x7fdbfc0cac30 2023-12-26T23:12:04.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdbe2ffd700 1 -- 172.21.15.131:0/4070330393 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec010670 con 0x7fdbfc0cac30 2023-12-26T23:12:04.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdbe2ffd700 1 -- 172.21.15.131:0/4070330393 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec010a30 con 0x7fdbfc0cac30 2023-12-26T23:12:04.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc0c6370 con 0x7fdbfc0cac30 2023-12-26T23:12:04.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.109+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc0ad280 con 0x7fdbfc0cac30 2023-12-26T23:12:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.110+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbfc15ca90 con 0x7fdbfc0cac30 2023-12-26T23:12:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.111+0000 7fdbe2ffd700 1 -- 172.21.15.131:0/4070330393 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fdbec01b400 con 0x7fdbfc0cac30 2023-12-26T23:12:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.113+0000 7fdbe2ffd700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdbe8067050 0x7fdbe8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.113+0000 7fdbfa7fc700 1 -- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdbe8067050 msgr2=0x7fdbe8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.113+0000 7fdbe2ffd700 1 -- 172.21.15.131:0/4070330393 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6484+0+0 (secure 0 0 0) 0x7fdbec084530 con 0x7fdbfc0cac30 2023-12-26T23:12:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.113+0000 7fdbfa7fc700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdbe8067050 0x7fdbe8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:04.117 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.117+0000 7fdbe2ffd700 1 -- 172.21.15.131:0/4070330393 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbec051380 con 0x7fdbfc0cac30 2023-12-26T23:12:04.147 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:04.147+0000 7fd47a88c1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:12:04.183 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:04.183+0000 7fdbc28391c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:12:04.229 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:04.230+0000 7fd47a88c1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:12:04.269 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:04.268+0000 7fdbc28391c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:12:04.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.314+0000 7fdbfa7fc700 1 -- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdbe8067050 msgr2=0x7fdbe8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:04.314 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.314+0000 7fdbfa7fc700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdbe8067050 0x7fdbe8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:04.364 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.364+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbfc0657e0 con 0x7fdbfc0cac30 2023-12-26T23:12:04.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.364+0000 7fdbe2ffd700 1 -- 172.21.15.131:0/4070330393 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4613 (secure 0 0 0) 0x7fdbec0549a0 con 0x7fdbfc0cac30 2023-12-26T23:12:04.365 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:04.365 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2070005366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2070005366},{"type":"v1","addr":"172.21.15.190:6835","nonce":2070005366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3800715315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3800715315},{"type":"v1","addr":"172.21.15.190:6834","nonce":3800715315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9645,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3865566903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3865566903},{"type":"v1","addr":"172.21.15.131:6833","nonce":3865566903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:09:46.045975+0000","modified":"2023-12-26T23:09:49.597663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9666":{"gid":9666,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2967199312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2967199312},{"type":"v1","addr":"172.21.15.131:6835","nonce":2967199312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-12-26T23:12:04.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdbe8067050 msgr2=0x7fdbe8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:04.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdbe8067050 0x7fdbe8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 msgr2=0x7fdbfc154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 0x7fdbfc154950 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c580 tx=0x7fdbec007e10 comp rx=0 tx=0).stop 2023-12-26T23:12:04.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 shutdown_connections 2023-12-26T23:12:04.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdbe8067050 0x7fdbe8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbfc06e990 0x7fdbfc0c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbfc0cac30 0x7fdbfc154950 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 --2- 172.21.15.131:0/4070330393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbfc159920 0x7fdbfc0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 >> 172.21.15.131:0/4070330393 conn(0x7fdbfc068040 msgr2=0x7fdbfc156140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:04.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 shutdown_connections 2023-12-26T23:12:04.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.367+0000 7fdc019ff700 1 -- 172.21.15.131:0/4070330393 wait complete. 2023-12-26T23:12:04.369 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 90 2023-12-26T23:12:04.387 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:04.417 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:04.417+0000 7fd47a88c1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:04.470 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:04.470+0000 7fdbc28391c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:04.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.581+0000 7f94d69ff700 1 Processor -- start 2023-12-26T23:12:04.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.581+0000 7f94d69ff700 1 -- start start 2023-12-26T23:12:04.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.582+0000 7f94d69ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94d013c510 0x7f94d00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.582+0000 7f94d69ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 0x7f94d006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.582+0000 7f94d69ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d00686c0 con 0x7f94d006b8c0 2023-12-26T23:12:04.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.582+0000 7f94d69ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d006c280 con 0x7f94d013c510 2023-12-26T23:12:04.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.582+0000 7f94d69ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d0069520 con 0x7f94d00cc810 2023-12-26T23:12:04.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94cffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94d013c510 0x7f94d00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:04.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94d4f9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 0x7f94d006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:04.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94cffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94d013c510 0x7f94d00c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57044/0 (socket says 172.21.15.131:57044) 2023-12-26T23:12:04.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94cffff700 1 -- 172.21.15.131:0/536350705 learned_addr learned my addr 172.21.15.131:0/536350705 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:04.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94cf7fe700 1 --1- 172.21.15.131:0/536350705 >> v1:172.21.15.190:6789/0 conn(0x7f94d006b8c0 0x7f94d00cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:36416/0 (socket says 172.21.15.131:36416) 2023-12-26T23:12:04.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94cffff700 1 -- 172.21.15.131:0/536350705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 msgr2=0x7f94d006bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:04.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94cffff700 1 --2- 172.21.15.131:0/536350705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 0x7f94d006bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94cffff700 1 -- 172.21.15.131:0/536350705 >> v1:172.21.15.190:6789/0 conn(0x7f94d006b8c0 legacy=0x7f94d00cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94cffff700 1 -- 172.21.15.131:0/536350705 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0068910 con 0x7f94d013c510 2023-12-26T23:12:04.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94ceffd700 1 -- 172.21.15.131:0/536350705 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10525235 0 0) 0x7f94d00686c0 con 0x7f94d006b8c0 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.583+0000 7f94cffff700 1 --2- 172.21.15.131:0/536350705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 0x7f94d00c8a20 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009990 tx=0x7f94c0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6a0a9f4e19811d server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94ceffd700 1 -- 172.21.15.131:0/536350705 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0009e80 con 0x7f94d013c510 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94ceffd700 1 -- 172.21.15.131:0/536350705 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c0005c10 con 0x7f94d013c510 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94ceffd700 1 -- 172.21.15.131:0/536350705 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c000b650 con 0x7f94d013c510 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94d69ff700 1 -- 172.21.15.131:0/536350705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 msgr2=0x7f94d00c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94d69ff700 1 --2- 172.21.15.131:0/536350705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 0x7f94d00c8a20 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009990 tx=0x7f94c0009220 comp rx=0 tx=0).stop 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94d69ff700 1 -- 172.21.15.131:0/536350705 shutdown_connections 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94d69ff700 1 --2- 172.21.15.131:0/536350705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 0x7f94d006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94d69ff700 1 --2- 172.21.15.131:0/536350705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 0x7f94d00c8a20 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.587 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94d69ff700 1 -- 172.21.15.131:0/536350705 >> 172.21.15.131:0/536350705 conn(0x7f94d00ce3e0 msgr2=0x7f94d00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:04.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94d69ff700 1 -- 172.21.15.131:0/536350705 shutdown_connections 2023-12-26T23:12:04.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.584+0000 7f94d69ff700 1 -- 172.21.15.131:0/536350705 wait complete. 2023-12-26T23:12:04.588 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:04.587+0000 7fd47a88c1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:12:04.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.587+0000 7f94d69ff700 1 Processor -- start 2023-12-26T23:12:04.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.587+0000 7f94d69ff700 1 -- start start 2023-12-26T23:12:04.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.588+0000 7f94d69ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d006b8c0 0x7f94d00b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.588+0000 7f94d69ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 0x7f94d00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.588+0000 7f94d69ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 0x7f94d00b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.588+0000 7f94d69ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00b3b10 con 0x7f94d013c510 2023-12-26T23:12:04.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.588+0000 7f94d69ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d015b250 con 0x7f94d006b8c0 2023-12-26T23:12:04.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.588+0000 7f94d69ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d015a680 con 0x7f94d00cc810 2023-12-26T23:12:04.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94d4f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 0x7f94d00b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:04.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94cffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d006b8c0 0x7f94d00b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:04.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94d4f9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 0x7f94d00b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57050/0 (socket says 172.21.15.131:57050) 2023-12-26T23:12:04.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94d4f9c700 1 -- 172.21.15.131:0/3418824477 learned_addr learned my addr 172.21.15.131:0/3418824477 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:04.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94cf7fe700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 0x7f94d00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:04.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94d4f9c700 1 -- 172.21.15.131:0/3418824477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 msgr2=0x7f94d00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94d4f9c700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 0x7f94d00b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94d4f9c700 1 -- 172.21.15.131:0/3418824477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d006b8c0 msgr2=0x7f94d00b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94d4f9c700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d006b8c0 0x7f94d00b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94d4f9c700 1 -- 172.21.15.131:0/3418824477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c0009480 con 0x7f94d013c510 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.589+0000 7f94d4f9c700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 0x7f94d00b35d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009e90 tx=0x7f94c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.590+0000 7f94b7fff700 1 -- 172.21.15.131:0/3418824477 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800cb70 con 0x7f94d013c510 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.590+0000 7f94b7fff700 1 -- 172.21.15.131:0/3418824477 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c800f040 con 0x7f94d013c510 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.590+0000 7f94b7fff700 1 -- 172.21.15.131:0/3418824477 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8014480 con 0x7f94d013c510 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.590+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d015a920 con 0x7f94d013c510 2023-12-26T23:12:04.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.590+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d015aee0 con 0x7f94d013c510 2023-12-26T23:12:04.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.591+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d00657e0 con 0x7f94d013c510 2023-12-26T23:12:04.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.592+0000 7f94b7fff700 1 -- 172.21.15.131:0/3418824477 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f94c8014830 con 0x7f94d013c510 2023-12-26T23:12:04.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.593+0000 7f94b7fff700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f94bc067050 0x7f94bc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:04.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.593+0000 7f94b7fff700 1 -- 172.21.15.131:0/3418824477 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f94c8085780 con 0x7f94d013c510 2023-12-26T23:12:04.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.593+0000 7f94cffff700 1 -- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f94bc067050 msgr2=0x7f94bc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:04.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.593+0000 7f94cffff700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f94bc067050 0x7f94bc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:04.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.597+0000 7f94b7fff700 1 -- 172.21.15.131:0/3418824477 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94c80524c0 con 0x7f94d013c510 2023-12-26T23:12:04.653 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:04.652+0000 7fdbc28391c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:12:04.671 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:04.671+0000 7fd47a88c1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:12:04.739 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:04.739+0000 7fdbc28391c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:12:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.794+0000 7f94cffff700 1 -- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f94bc067050 msgr2=0x7f94bc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.794+0000 7f94cffff700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f94bc067050 0x7f94bc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:04.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.801+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94d0068bf0 con 0x7f94d013c510 2023-12-26T23:12:04.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.802+0000 7f94b7fff700 1 -- 172.21.15.131:0/3418824477 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+16592 (secure 0 0 0) 0x7f94c8055ae0 con 0x7f94d013c510 2023-12-26T23:12:04.803 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:04.803 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":196,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:11:04.499190+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:09:43.598995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-26T23:09:45.039854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.29","mappings":[{"from":5,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1645000428":"2023-12-27T23:10:02.694004+0000","172.21.15.131:0/280306402":"2023-12-27T23:10:02.694004+0000","172.21.15.131:0/2488529326":"2023-12-27T23:10:02.694004+0000","172.21.15.131:6837/92228":"2023-12-27T23:10:02.694004+0000","172.21.15.131:6836/92228":"2023-12-27T23:10:02.694004+0000","172.21.15.131:0/2520330293":"2023-12-27T23:10:02.694004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:04.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f94bc067050 msgr2=0x7f94bc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:04.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f94bc067050 0x7f94bc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 msgr2=0x7f94d00b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:04.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 0x7f94d00b35d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009e90 tx=0x7f94c800b040 comp rx=0 tx=0).stop 2023-12-26T23:12:04.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 shutdown_connections 2023-12-26T23:12:04.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f94bc067050 0x7f94bc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94d013c510 0x7f94d00b35d0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d006b8c0 0x7f94d00b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 --2- 172.21.15.131:0/3418824477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d00cc810 0x7f94d00b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:04.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 >> 172.21.15.131:0/3418824477 conn(0x7f94d00ce3e0 msgr2=0x7f94d015e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:04.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 shutdown_connections 2023-12-26T23:12:04.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:04.805+0000 7f94d69ff700 1 -- 172.21.15.131:0/3418824477 wait complete. 2023-12-26T23:12:04.817 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:04.817+0000 7fd47a88c1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:04.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T23:09:43.598995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '192', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T23:09:45.039854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T23:12:04.825 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T23:12:04.905 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:04.905+0000 7fdbc28391c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:04.908 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:04.908+0000 7fd47a88c1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:12:05.002 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:05.002+0000 7fdbc28391c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:12:05.026 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.026+0000 7fc29e9ff700 1 Processor -- start 2023-12-26T23:12:05.027 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.026+0000 7fc29e9ff700 1 -- start start 2023-12-26T23:12:05.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.027+0000 7fc29e9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc298159a70 0x7fc2980b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.027+0000 7fc29e9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2980afcd0 con 0x7fc2981596a0 2023-12-26T23:12:05.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.027+0000 7fc29e9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2980b3e70 con 0x7fc2980b2a90 2023-12-26T23:12:05.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.027+0000 7fc29e9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980b0f20 con 0x7fc298159a70 2023-12-26T23:12:05.031 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc297fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc2981596a0 0x7fc2980b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34054/0 (socket says 172.21.15.131:34054) 2023-12-26T23:12:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc297fff700 1 -- 172.21.15.131:0/564069554 learned_addr learned my addr 172.21.15.131:0/564069554 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc2977fe700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc298159a70 0x7fc2980b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648749764 0 0) 0x7fc2980afcd0 con 0x7fc2981596a0 2023-12-26T23:12:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc280003570 con 0x7fc2981596a0 2023-12-26T23:12:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 925601016 0 0) 0x7fc280003570 con 0x7fc2981596a0 2023-12-26T23:12:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc298159a70 msgr2=0x7fc2980b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc296ffd700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc298159a70 0x7fc2980b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 >> v1:172.21.15.190:6789/0 conn(0x7fc2980b2a90 legacy=0x7fc2980b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:05.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2980b3540 con 0x7fc2981596a0 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fc288001470 con 0x7fc2981596a0 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.028+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 >> v1:172.21.15.131:6789/0 conn(0x7fc2981596a0 legacy=0x7fc2980b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc296ffd700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc280003a90 0x7fc280003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc296ffd700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc280004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc296ffd700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc298159a70 0x7fc2800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980b0f20 con 0x7fc280004810 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28000bb40 con 0x7fc280003a90 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28000bd00 con 0x7fc298159a70 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc2977fe700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc280004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc2977fe700 1 -- 172.21.15.131:0/564069554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc298159a70 msgr2=0x7fc2800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc2977fe700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc298159a70 0x7fc2800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc2977fe700 1 -- 172.21.15.131:0/564069554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc280003a90 msgr2=0x7fc280003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc2977fe700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc280003a90 0x7fc280003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc2977fe700 1 -- 172.21.15.131:0/564069554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc280003570 con 0x7fc280004810 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.029+0000 7fc2977fe700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc280004c80 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009c00 tx=0x7fc28c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb019bf9014ae1b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00e710 con 0x7fc280004810 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c00e870 con 0x7fc280004810 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc296ffd700 1 -- 172.21.15.131:0/564069554 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00ec30 con 0x7fc280004810 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc29e9ff700 1 -- 172.21.15.131:0/564069554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 msgr2=0x7fc280004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.034 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc29e9ff700 1 --2- 172.21.15.131:0/564069554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc280004c80 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009c00 tx=0x7fc28c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc29cf9c700 1 -- 172.21.15.131:0/564069554 reap_dead start 2023-12-26T23:12:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc29e9ff700 1 -- 172.21.15.131:0/564069554 shutdown_connections 2023-12-26T23:12:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc29e9ff700 1 -- 172.21.15.131:0/564069554 >> 172.21.15.131:0/564069554 conn(0x7fc29806ece0 msgr2=0x7fc29806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc29e9ff700 1 -- 172.21.15.131:0/564069554 shutdown_connections 2023-12-26T23:12:05.035 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.030+0000 7fc29e9ff700 1 -- 172.21.15.131:0/564069554 wait complete. 2023-12-26T23:12:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.033+0000 7fc29e9ff700 1 Processor -- start 2023-12-26T23:12:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.033+0000 7fc29e9ff700 1 -- start start 2023-12-26T23:12:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.034+0000 7fc29e9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc29816d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.034+0000 7fc29e9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2980cfa60 0x7fc2980b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.034+0000 7fc29e9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2981505f0 0x7fc298150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc29e9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29816def0 con 0x7fc280004810 2023-12-26T23:12:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc29e9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc298150fb0 con 0x7fc2980cfa60 2023-12-26T23:12:05.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc29e9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980d01d0 con 0x7fc2981505f0 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc297fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc29816d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc297fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc29816d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55536/0 (socket says 172.21.15.131:55536) 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc297fff700 1 -- 172.21.15.131:0/3247219011 learned_addr learned my addr 172.21.15.131:0/3247219011 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc2977fe700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2980cfa60 0x7fc2980b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc29cf9c700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2981505f0 0x7fc298150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc297fff700 1 -- 172.21.15.131:0/3247219011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2981505f0 msgr2=0x7fc298150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc297fff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2981505f0 0x7fc298150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc297fff700 1 -- 172.21.15.131:0/3247219011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2980cfa60 msgr2=0x7fc2980b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc297fff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2980cfa60 0x7fc2980b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc297fff700 1 -- 172.21.15.131:0/3247219011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2980d04b0 con 0x7fc280004810 2023-12-26T23:12:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.035+0000 7fc297fff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc29816d9b0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fc28800cb10 tx=0x7fc2980b3de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.036+0000 7fc27ffff700 1 -- 172.21.15.131:0/3247219011 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288007150 con 0x7fc280004810 2023-12-26T23:12:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.036+0000 7fc27ffff700 1 -- 172.21.15.131:0/3247219011 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2880072b0 con 0x7fc280004810 2023-12-26T23:12:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.036+0000 7fc27ffff700 1 -- 172.21.15.131:0/3247219011 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288007670 con 0x7fc280004810 2023-12-26T23:12:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.036+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2980d0790 con 0x7fc280004810 2023-12-26T23:12:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.036+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2980d0d50 con 0x7fc280004810 2023-12-26T23:12:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.037+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2980657e0 con 0x7fc280004810 2023-12-26T23:12:05.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.038+0000 7fc27ffff700 1 -- 172.21.15.131:0/3247219011 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fc288004a40 con 0x7fc280004810 2023-12-26T23:12:05.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.038+0000 7fc27ffff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc284067050 0x7fc284069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.039+0000 7fc2977fe700 1 -- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc284067050 msgr2=0x7fc284069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:05.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.039+0000 7fc27ffff700 1 -- 172.21.15.131:0/3247219011 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6484+0+0 (secure 0 0 0) 0x7fc288083fc0 con 0x7fc280004810 2023-12-26T23:12:05.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.039+0000 7fc2977fe700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc284067050 0x7fc284069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:05.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.042+0000 7fc27ffff700 1 -- 172.21.15.131:0/3247219011 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc28804c5a0 con 0x7fc280004810 2023-12-26T23:12:05.091 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:05.092+0000 7fd47a88c1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:12:05.194 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:05.194+0000 7fdbc28391c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:12:05.240 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:05.240+0000 7fd47a88c1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:12:05.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.240+0000 7fc2977fe700 1 -- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc284067050 msgr2=0x7fc284069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:05.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.240+0000 7fc2977fe700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc284067050 0x7fc284069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.279+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2980b3820 con 0x7fc280004810 2023-12-26T23:12:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.279+0000 7fc27ffff700 1 -- 172.21.15.131:0/3247219011 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v196) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc288018020 con 0x7fc280004810 2023-12-26T23:12:05.280 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T23:12:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc284067050 msgr2=0x7fc284069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc284067050 0x7fc284069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 msgr2=0x7fc29816d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc29816d9b0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fc28800cb10 tx=0x7fc2980b3de0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 shutdown_connections 2023-12-26T23:12:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc284067050 0x7fc284069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2981505f0 0x7fc298150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280004810 0x7fc29816d9b0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 --2- 172.21.15.131:0/3247219011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2980cfa60 0x7fc2980b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 >> 172.21.15.131:0/3247219011 conn(0x7fc29806ece0 msgr2=0x7fc2980ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:05.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 shutdown_connections 2023-12-26T23:12:05.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.282+0000 7fc29e9ff700 1 -- 172.21.15.131:0/3247219011 wait complete. 2023-12-26T23:12:05.301 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T23:12:05.357 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:05.357+0000 7fdbc28391c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:12:05.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.502+0000 7f00b19ff700 1 Processor -- start 2023-12-26T23:12:05.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.502+0000 7f00b19ff700 1 -- start start 2023-12-26T23:12:05.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00b19ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f00ac159920 0x7f00ac0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00b19ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f00ac159cf0 0x7f00ac0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00b19ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac1554d0 0x7f00ac155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00b19ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac0c8bf0 con 0x7f00ac159920 2023-12-26T23:12:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00b19ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac0c8e60 con 0x7f00ac159cf0 2023-12-26T23:12:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00b19ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0c9140 con 0x7f00ac1554d0 2023-12-26T23:12:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00aaffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f00ac159920 0x7f00ac0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00aaffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f00ac159920 0x7f00ac0b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55550/0 (socket says 172.21.15.131:55550) 2023-12-26T23:12:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00aaffd700 1 -- 172.21.15.131:0/2841199402 learned_addr learned my addr 172.21.15.131:0/2841199402 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00ab7fe700 1 --2- 172.21.15.131:0/2841199402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac1554d0 0x7f00ac155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.504+0000 7f00aa7fc700 1 --2- 172.21.15.131:0/2841199402 >> v2:172.21.15.190:3300/0 conn(0x7f00ac159cf0 0x7f00ac0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.505+0000 7f00aaffd700 1 -- 172.21.15.131:0/2841199402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac1554d0 msgr2=0x7f00ac155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.505+0000 7f00aaffd700 1 --2- 172.21.15.131:0/2841199402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac1554d0 0x7f00ac155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.505+0000 7f00aaffd700 1 -- 172.21.15.131:0/2841199402 >> v2:172.21.15.190:3300/0 conn(0x7f00ac159cf0 msgr2=0x7f00ac0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.505+0000 7f00aaffd700 1 --2- 172.21.15.131:0/2841199402 >> v2:172.21.15.190:3300/0 conn(0x7f00ac159cf0 0x7f00ac0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.505+0000 7f00aaffd700 1 -- 172.21.15.131:0/2841199402 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0c9420 con 0x7f00ac159920 2023-12-26T23:12:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.505+0000 7f00aaffd700 1 --2- 172.21.15.131:0/2841199402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159920 0x7f00ac0b00e0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0b0b60 tx=0x7f009c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a734e86468395341 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.505+0000 7f00a9ffb700 1 -- 172.21.15.131:0/2841199402 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c00db00 con 0x7f00ac159920 2023-12-26T23:12:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.505+0000 7f00a9ffb700 1 -- 172.21.15.131:0/2841199402 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c00dc60 con 0x7f00ac159920 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.505+0000 7f00a9ffb700 1 -- 172.21.15.131:0/2841199402 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c019480 con 0x7f00ac159920 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.506+0000 7f00b19ff700 1 -- 172.21.15.131:0/2841199402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159920 msgr2=0x7f00ac0b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.506+0000 7f00b19ff700 1 --2- 172.21.15.131:0/2841199402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159920 0x7f00ac0b00e0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0b0b60 tx=0x7f009c00b4f0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.506+0000 7f00b19ff700 1 -- 172.21.15.131:0/2841199402 shutdown_connections 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.506+0000 7f00b19ff700 1 --2- 172.21.15.131:0/2841199402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159920 0x7f00ac0b00e0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.506+0000 7f00b19ff700 1 --2- 172.21.15.131:0/2841199402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac1554d0 0x7f00ac155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.506+0000 7f00b19ff700 1 --2- 172.21.15.131:0/2841199402 >> v2:172.21.15.190:3300/0 conn(0x7f00ac159cf0 0x7f00ac0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.506+0000 7f00b19ff700 1 -- 172.21.15.131:0/2841199402 >> 172.21.15.131:0/2841199402 conn(0x7f00ac068040 msgr2=0x7f00ac069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.506+0000 7f00b19ff700 1 -- 172.21.15.131:0/2841199402 shutdown_connections 2023-12-26T23:12:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.506+0000 7f00b19ff700 1 -- 172.21.15.131:0/2841199402 wait complete. 2023-12-26T23:12:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.508+0000 7f00b19ff700 1 Processor -- start 2023-12-26T23:12:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.508+0000 7f00b19ff700 1 -- start start 2023-12-26T23:12:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00b19ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac1554d0 0x7f00ac0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00b19ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac159920 0x7f00ac0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00b19ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159cf0 0x7f00ac169df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00b19ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac16bc90 con 0x7f00ac159cf0 2023-12-26T23:12:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00b19ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac16b0c0 con 0x7f00ac1554d0 2023-12-26T23:12:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00b19ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac167a90 con 0x7f00ac159920 2023-12-26T23:12:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00aa7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac159920 0x7f00ac0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00aaffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac1554d0 0x7f00ac0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00aa7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac159920 0x7f00ac0c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:59172/0 (socket says 172.21.15.131:59172) 2023-12-26T23:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00ab7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159cf0 0x7f00ac169df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.509+0000 7f00aa7fc700 1 -- 172.21.15.131:0/3900627934 learned_addr learned my addr 172.21.15.131:0/3900627934 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.510+0000 7f00aa7fc700 1 -- 172.21.15.131:0/3900627934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac1554d0 msgr2=0x7f00ac0c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.510+0000 7f00aa7fc700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac1554d0 0x7f00ac0c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.510+0000 7f00aa7fc700 1 -- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159cf0 msgr2=0x7f00ac169df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.510+0000 7f00aa7fc700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159cf0 0x7f00ac169df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.510+0000 7f00aa7fc700 1 -- 172.21.15.131:0/3900627934 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c00b790 con 0x7f00ac159920 2023-12-26T23:12:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.510+0000 7f00aaffd700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac1554d0 0x7f00ac0c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.510+0000 7f00ab7fe700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159cf0 0x7f00ac169df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.510+0000 7f00aa7fc700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac159920 0x7f00ac0c6870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e930 tx=0x7f00a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:05.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.517+0000 7f0092ffd700 1 -- 172.21.15.131:0/3900627934 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0013680 con 0x7f00ac159920 2023-12-26T23:12:05.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.517+0000 7f0092ffd700 1 -- 172.21.15.131:0/3900627934 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a00137e0 con 0x7f00ac159920 2023-12-26T23:12:05.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.517+0000 7f0092ffd700 1 -- 172.21.15.131:0/3900627934 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0013b80 con 0x7f00ac159920 2023-12-26T23:12:05.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.517+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac16b360 con 0x7f00ac159920 2023-12-26T23:12:05.518 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.517+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac16b920 con 0x7f00ac159920 2023-12-26T23:12:05.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.518+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac0657e0 con 0x7f00ac159920 2023-12-26T23:12:05.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.519+0000 7f0092ffd700 1 -- 172.21.15.131:0/3900627934 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f00a000c3e0 con 0x7f00ac159920 2023-12-26T23:12:05.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.520+0000 7f0092ffd700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0098067120 0x7f00980695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.520+0000 7f00aaffd700 1 -- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0098067120 msgr2=0x7f00980695d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:05.520 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.520+0000 7f00aaffd700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0098067120 0x7f00980695d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:05.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.527+0000 7f0092ffd700 1 -- 172.21.15.131:0/3900627934 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6540+0+0 (secure 0 0 0) 0x7f00a0089080 con 0x7f00ac159920 2023-12-26T23:12:05.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.533+0000 7f0092ffd700 1 -- 172.21.15.131:0/3900627934 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00a00518c0 con 0x7f00ac159920 2023-12-26T23:12:05.625 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:05.625+0000 7fd47a88c1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:12:05.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.721+0000 7f00aaffd700 1 -- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0098067120 msgr2=0x7f00980695d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:05.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.721+0000 7f00aaffd700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0098067120 0x7f00980695d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:05.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.739+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00ac16bdd0 con 0x7f00ac159920 2023-12-26T23:12:05.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.740+0000 7f0092ffd700 1 -- 172.21.15.131:0/3900627934 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v197) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00a0054ee0 con 0x7f00ac159920 2023-12-26T23:12:05.740 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T23:12:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0098067120 msgr2=0x7f00980695d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0098067120 0x7f00980695d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac159920 msgr2=0x7f00ac0c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac159920 0x7f00ac0c6870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e930 tx=0x7f00a00091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 shutdown_connections 2023-12-26T23:12:05.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0098067120 0x7f00980695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00ac159cf0 0x7f00ac169df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ac1554d0 0x7f00ac0c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 --2- 172.21.15.131:0/3900627934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ac159920 0x7f00ac0c6870 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 >> 172.21.15.131:0/3900627934 conn(0x7f00ac068040 msgr2=0x7f00ac069360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 shutdown_connections 2023-12-26T23:12:05.744 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.742+0000 7f00b19ff700 1 -- 172.21.15.131:0/3900627934 wait complete. 2023-12-26T23:12:05.761 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T23:12:05.777 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:05.777+0000 7fdbc28391c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:12:05.896 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:05.896+0000 7fd47a88c1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:12:05.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.957+0000 7f62699ff700 1 Processor -- start 2023-12-26T23:12:05.958 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.958+0000 7f62699ff700 1 -- start start 2023-12-26T23:12:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f62699ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f626413c510 0x7f62640c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f62699ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 0x7f626406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f62699ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62640cc100 0x7f62640cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f62699ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62640686c0 con 0x7f626413c510 2023-12-26T23:12:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f62699ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6264069520 con 0x7f62640cc100 2023-12-26T23:12:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f62699ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626406c250 con 0x7f626406b8c0 2023-12-26T23:12:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f6262ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f626413c510 0x7f62640c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f6262ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f626413c510 0x7f62640c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55566/0 (socket says 172.21.15.131:55566) 2023-12-26T23:12:05.962 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f6262ffd700 1 -- 172.21.15.131:0/988804267 learned_addr learned my addr 172.21.15.131:0/988804267 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f62637fe700 1 --2- 172.21.15.131:0/988804267 >> v2:172.21.15.190:3300/0 conn(0x7f62640cc100 0x7f62640cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.959+0000 7f62627fc700 1 --2- 172.21.15.131:0/988804267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 0x7f626406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f6262ffd700 1 -- 172.21.15.131:0/988804267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 msgr2=0x7f626406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f6262ffd700 1 --2- 172.21.15.131:0/988804267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 0x7f626406bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f6262ffd700 1 -- 172.21.15.131:0/988804267 >> v2:172.21.15.190:3300/0 conn(0x7f62640cc100 msgr2=0x7f62640cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f6262ffd700 1 --2- 172.21.15.131:0/988804267 >> v2:172.21.15.190:3300/0 conn(0x7f62640cc100 0x7f62640cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f6262ffd700 1 -- 172.21.15.131:0/988804267 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6264068910 con 0x7f626413c510 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f62637fe700 1 --2- 172.21.15.131:0/988804267 >> v2:172.21.15.190:3300/0 conn(0x7f62640cc100 0x7f62640cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f6262ffd700 1 --2- 172.21.15.131:0/988804267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f626413c510 0x7f62640c8a20 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f626406f130 tx=0x7f6254009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8794b1a048705f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:05.963 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f6261ffb700 1 -- 172.21.15.131:0/988804267 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254005c10 con 0x7f626413c510 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f6261ffb700 1 -- 172.21.15.131:0/988804267 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6254005d70 con 0x7f626413c510 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.960+0000 7f6261ffb700 1 -- 172.21.15.131:0/988804267 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254006260 con 0x7f626413c510 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.961+0000 7f62699ff700 1 -- 172.21.15.131:0/988804267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f626413c510 msgr2=0x7f62640c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.961+0000 7f62699ff700 1 --2- 172.21.15.131:0/988804267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f626413c510 0x7f62640c8a20 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f626406f130 tx=0x7f6254009220 comp rx=0 tx=0).stop 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.961+0000 7f62699ff700 1 -- 172.21.15.131:0/988804267 shutdown_connections 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.961+0000 7f62699ff700 1 --2- 172.21.15.131:0/988804267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f626413c510 0x7f62640c8a20 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.961+0000 7f62699ff700 1 --2- 172.21.15.131:0/988804267 >> v2:172.21.15.190:3300/0 conn(0x7f62640cc100 0x7f62640cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.961+0000 7f62699ff700 1 --2- 172.21.15.131:0/988804267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 0x7f626406bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.961+0000 7f62699ff700 1 -- 172.21.15.131:0/988804267 >> 172.21.15.131:0/988804267 conn(0x7f62640ce3e0 msgr2=0x7f62640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.961+0000 7f62699ff700 1 -- 172.21.15.131:0/988804267 shutdown_connections 2023-12-26T23:12:05.964 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.961+0000 7f62699ff700 1 -- 172.21.15.131:0/988804267 wait complete. 2023-12-26T23:12:05.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.963+0000 7f62699ff700 1 Processor -- start 2023-12-26T23:12:05.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.963+0000 7f62699ff700 1 -- start start 2023-12-26T23:12:05.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62699ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 0x7f62640b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62699ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62640cc100 0x7f6264154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62699ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626413c510 0x7f626415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62699ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626415ce70 con 0x7f62640cc100 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62699ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626415d2e0 con 0x7f626413c510 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62699ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626415d5c0 con 0x7f626406b8c0 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62627fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62640cc100 0x7f6264154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f6262ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 0x7f62640b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62627fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62640cc100 0x7f6264154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55570/0 (socket says 172.21.15.131:55570) 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62627fc700 1 -- 172.21.15.131:0/3194473623 learned_addr learned my addr 172.21.15.131:0/3194473623 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62637fe700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626413c510 0x7f626415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:05.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62627fc700 1 -- 172.21.15.131:0/3194473623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 msgr2=0x7f62640b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62627fc700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 0x7f62640b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62627fc700 1 -- 172.21.15.131:0/3194473623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626413c510 msgr2=0x7f626415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62627fc700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626413c510 0x7f626415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f62627fc700 1 -- 172.21.15.131:0/3194473623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c009a20 con 0x7f62640cc100 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.964+0000 7f6262ffd700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 0x7f62640b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.965+0000 7f62627fc700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62640cc100 0x7f6264154df0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f625800e950 tx=0x7f62580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.965+0000 7f624affd700 1 -- 172.21.15.131:0/3194473623 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258013680 con 0x7f62640cc100 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.965+0000 7f624affd700 1 -- 172.21.15.131:0/3194473623 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62580137e0 con 0x7f62640cc100 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.965+0000 7f624affd700 1 -- 172.21.15.131:0/3194473623 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258013ba0 con 0x7f62640cc100 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.965+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6254009480 con 0x7f62640cc100 2023-12-26T23:12:05.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.965+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626415db80 con 0x7f62640cc100 2023-12-26T23:12:05.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.966+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62640657e0 con 0x7f62640cc100 2023-12-26T23:12:05.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.967+0000 7f624affd700 1 -- 172.21.15.131:0/3194473623 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f625800c490 con 0x7f62640cc100 2023-12-26T23:12:05.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.968+0000 7f624affd700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6250067050 0x7f6250069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:05.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.969+0000 7f6262ffd700 1 -- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6250067050 msgr2=0x7f6250069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:05.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.969+0000 7f624affd700 1 -- 172.21.15.131:0/3194473623 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6540+0+0 (secure 0 0 0) 0x7f6258084ba0 con 0x7f62640cc100 2023-12-26T23:12:05.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.969+0000 7f6262ffd700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6250067050 0x7f6250069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:05.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:05.973+0000 7f624affd700 1 -- 172.21.15.131:0/3194473623 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62580519e0 con 0x7f62640cc100 2023-12-26T23:12:06.059 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:06.059+0000 7fdbc28391c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:12:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.170+0000 7f6262ffd700 1 -- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6250067050 msgr2=0x7f6250069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:06.170 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.170+0000 7f6262ffd700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6250067050 0x7f6250069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:06.177 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:06.177+0000 7fd47a88c1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:12:06.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.178+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f626415dda0 con 0x7f62640cc100 2023-12-26T23:12:06.179 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.179+0000 7f624affd700 1 -- 172.21.15.131:0/3194473623 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v197) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6258055000 con 0x7f62640cc100 2023-12-26T23:12:06.179 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T23:12:06.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.181+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6250067050 msgr2=0x7f6250069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:06.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.181+0000 7f62699ff700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6250067050 0x7f6250069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.181+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62640cc100 msgr2=0x7f6264154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.181+0000 7f62699ff700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62640cc100 0x7f6264154df0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f625800e950 tx=0x7f62580091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.181+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 shutdown_connections 2023-12-26T23:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.181+0000 7f62699ff700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6250067050 0x7f6250069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.181+0000 7f62699ff700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f626413c510 0x7f626415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.182+0000 7f62699ff700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f626406b8c0 0x7f62640b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.182+0000 7f62699ff700 1 --2- 172.21.15.131:0/3194473623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62640cc100 0x7f6264154df0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.182+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 >> 172.21.15.131:0/3194473623 conn(0x7f62640ce3e0 msgr2=0x7f62640c4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.182+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 shutdown_connections 2023-12-26T23:12:06.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.182+0000 7f62699ff700 1 -- 172.21.15.131:0/3194473623 wait complete. 2023-12-26T23:12:06.203 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:06.265 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:06.266+0000 7fd47a88c1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:12:06.361 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:06.361+0000 7fdbc28391c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:12:06.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.451+0000 7f95a2dff700 1 Processor -- start 2023-12-26T23:12:06.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.451+0000 7f95a2dff700 1 -- start start 2023-12-26T23:12:06.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.452+0000 7f95a2dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f959c159a70 0x7f959c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.452+0000 7f95a2dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959c0afcd0 con 0x7f959c1596a0 2023-12-26T23:12:06.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.452+0000 7f95a2dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959c0b3e70 con 0x7f959c0b2a90 2023-12-26T23:12:06.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.452+0000 7f95a2dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959c0b0f20 con 0x7f959c159a70 2023-12-26T23:12:06.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.452+0000 7f95a0b9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f959c1596a0 0x7f959c0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34064/0 (socket says 172.21.15.131:34064) 2023-12-26T23:12:06.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f95a0b9b700 1 -- 172.21.15.131:0/3587767399 learned_addr learned my addr 172.21.15.131:0/3587767399 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:06.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959bfff700 1 --2- 172.21.15.131:0/3587767399 >> v2:172.21.15.190:3300/0 conn(0x7f959c159a70 0x7f959c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710399303 0 0) 0x7f959c0afcd0 con 0x7f959c1596a0 2023-12-26T23:12:06.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f958c003570 con 0x7f959c1596a0 2023-12-26T23:12:06.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904558692 0 0) 0x7f959c0b3e70 con 0x7f959c0b2a90 2023-12-26T23:12:06.458 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f959c0afcd0 con 0x7f959c0b2a90 2023-12-26T23:12:06.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143544183 0 0) 0x7f958c003570 con 0x7f959c1596a0 2023-12-26T23:12:06.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 >> v2:172.21.15.190:3300/0 conn(0x7f959c159a70 msgr2=0x7f959c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:06.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 --2- 172.21.15.131:0/3587767399 >> v2:172.21.15.190:3300/0 conn(0x7f959c159a70 0x7f959c0b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 >> v1:172.21.15.190:6789/0 conn(0x7f959c0b2a90 legacy=0x7f959c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0b3540 con 0x7f959c1596a0 2023-12-26T23:12:06.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f9590001470 con 0x7f959c1596a0 2023-12-26T23:12:06.459 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 >> v1:172.21.15.131:6789/0 conn(0x7f959c1596a0 legacy=0x7f959c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.453+0000 7f959b7fe700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f958c003c40 0x7f958c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f959b7fe700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c004810 0x7f958c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f959b7fe700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f958c005210 0x7f958c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0b0f20 con 0x7f958c005210 2023-12-26T23:12:06.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c00c130 con 0x7f958c004810 2023-12-26T23:12:06.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958c00c2c0 con 0x7f958c003c40 2023-12-26T23:12:06.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f95a139c700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f958c005210 0x7f958c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f95a0b9b700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f958c003c40 0x7f958c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f959bfff700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c004810 0x7f958c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f95a139c700 1 -- 172.21.15.131:0/3587767399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f958c003c40 msgr2=0x7f958c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f95a139c700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f958c003c40 0x7f958c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f95a139c700 1 -- 172.21.15.131:0/3587767399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c004810 msgr2=0x7f958c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f95a139c700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c004810 0x7f958c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f95a139c700 1 -- 172.21.15.131:0/3587767399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c003570 con 0x7f958c005210 2023-12-26T23:12:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f95a139c700 1 -- 172.21.15.131:0/3587767399 reap_dead start 2023-12-26T23:12:06.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.454+0000 7f95a139c700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f958c005210 0x7f958c00ba60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9594002150 tx=0x7f95940017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257362e15ef461c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959400c940 con 0x7f958c005210 2023-12-26T23:12:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959400caa0 con 0x7f958c005210 2023-12-26T23:12:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f959b7fe700 1 -- 172.21.15.131:0/3587767399 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959400cc10 con 0x7f958c005210 2023-12-26T23:12:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f95a2dff700 1 -- 172.21.15.131:0/3587767399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f958c005210 msgr2=0x7f958c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f95a2dff700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f958c005210 0x7f958c00ba60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9594002150 tx=0x7f95940017a0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.462 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f95a2dff700 1 -- 172.21.15.131:0/3587767399 shutdown_connections 2023-12-26T23:12:06.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f95a2dff700 1 --2- 172.21.15.131:0/3587767399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f958c005210 0x7f958c00ba60 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f95a2dff700 1 -- 172.21.15.131:0/3587767399 >> 172.21.15.131:0/3587767399 conn(0x7f959c06ece0 msgr2=0x7f959c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:06.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f95a2dff700 1 -- 172.21.15.131:0/3587767399 shutdown_connections 2023-12-26T23:12:06.463 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.455+0000 7f95a2dff700 1 -- 172.21.15.131:0/3587767399 wait complete. 2023-12-26T23:12:06.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.458+0000 7f95a2dff700 1 Processor -- start 2023-12-26T23:12:06.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.458+0000 7f95a2dff700 1 -- start start 2023-12-26T23:12:06.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.459+0000 7f95a2dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c005210 0x7f959c15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.459+0000 7f95a2dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f959c0c8ab0 0x7f959c1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.459+0000 7f95a2dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f959c0cf590 0x7f959c0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f95a2dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0c9fa0 con 0x7f959c0c8ab0 2023-12-26T23:12:06.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f95a2dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0cff40 con 0x7f958c005210 2023-12-26T23:12:06.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f95a2dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0c9320 con 0x7f959c0cf590 2023-12-26T23:12:06.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f959bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f959c0c8ab0 0x7f959c1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f95a0b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c005210 0x7f959c15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f95a0b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c005210 0x7f959c15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:33716/0 (socket says 172.21.15.131:33716) 2023-12-26T23:12:06.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f959bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f959c0c8ab0 0x7f959c1509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55590/0 (socket says 172.21.15.131:55590) 2023-12-26T23:12:06.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f95a0b9b700 1 -- 172.21.15.131:0/462600985 learned_addr learned my addr 172.21.15.131:0/462600985 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:06.466 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f95a139c700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f959c0cf590 0x7f959c0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.467 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:06.460+0000 7fdbc28391c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:12:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f959bfff700 1 -- 172.21.15.131:0/462600985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f959c0cf590 msgr2=0x7f959c0cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f959bfff700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f959c0cf590 0x7f959c0cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f959bfff700 1 -- 172.21.15.131:0/462600985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c005210 msgr2=0x7f959c15ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f959bfff700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c005210 0x7f959c15ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.460+0000 7f959bfff700 1 -- 172.21.15.131:0/462600985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0c9600 con 0x7f959c0c8ab0 2023-12-26T23:12:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.461+0000 7f959bfff700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f959c0c8ab0 0x7f959c1509e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f95880119f0 tx=0x7f95880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.461+0000 7f9598ff9700 1 -- 172.21.15.131:0/462600985 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800f790 con 0x7f959c0c8ab0 2023-12-26T23:12:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.461+0000 7f9598ff9700 1 -- 172.21.15.131:0/462600985 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958800f8f0 con 0x7f959c0c8ab0 2023-12-26T23:12:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.461+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c0c98e0 con 0x7f959c0c8ab0 2023-12-26T23:12:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.461+0000 7f9598ff9700 1 -- 172.21.15.131:0/462600985 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800fc90 con 0x7f959c0c8ab0 2023-12-26T23:12:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.461+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c150f20 con 0x7f959c0c8ab0 2023-12-26T23:12:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.462+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c0657e0 con 0x7f959c0c8ab0 2023-12-26T23:12:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.464+0000 7f9598ff9700 1 -- 172.21.15.131:0/462600985 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f9588014440 con 0x7f959c0c8ab0 2023-12-26T23:12:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.465+0000 7f9598ff9700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9580067050 0x7f9580069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.465+0000 7f9598ff9700 1 -- 172.21.15.131:0/462600985 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6540+0+0 (secure 0 0 0) 0x7f9588018030 con 0x7f959c0c8ab0 2023-12-26T23:12:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.465+0000 7f95a0b9b700 1 -- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9580067050 msgr2=0x7f9580069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.465+0000 7f95a0b9b700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9580067050 0x7f9580069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.469+0000 7f9598ff9700 1 -- 172.21.15.131:0/462600985 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95880520a0 con 0x7f959c0c8ab0 2023-12-26T23:12:06.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.666+0000 7f95a0b9b700 1 -- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9580067050 msgr2=0x7f9580069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:06.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.666+0000 7f95a0b9b700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9580067050 0x7f9580069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:06.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.719+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f959c0b3820 con 0x7f959c0c8ab0 2023-12-26T23:12:06.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.719+0000 7f9598ff9700 1 -- 172.21.15.131:0/462600985 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4613 (secure 0 0 0) 0x7f95880556c0 con 0x7f959c0c8ab0 2023-12-26T23:12:06.720 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:06.720 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2070005366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2070005366},{"type":"v1","addr":"172.21.15.190:6835","nonce":2070005366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3800715315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3800715315},{"type":"v1","addr":"172.21.15.190:6834","nonce":3800715315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9645,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3865566903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3865566903},{"type":"v1","addr":"172.21.15.131:6833","nonce":3865566903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:09:46.045975+0000","modified":"2023-12-26T23:09:49.597663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9666":{"gid":9666,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2967199312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2967199312},{"type":"v1","addr":"172.21.15.131:6835","nonce":2967199312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-12-26T23:12:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.721+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9580067050 msgr2=0x7f9580069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.721+0000 7f95a2dff700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9580067050 0x7f9580069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.721+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f959c0c8ab0 msgr2=0x7f959c1509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.721+0000 7f95a2dff700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f959c0c8ab0 0x7f959c1509e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f95880119f0 tx=0x7f95880091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.721+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 shutdown_connections 2023-12-26T23:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.722+0000 7f95a2dff700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9580067050 0x7f9580069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.722+0000 7f95a2dff700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f959c0cf590 0x7f959c0cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.722+0000 7f95a2dff700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f958c005210 0x7f959c15ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.722+0000 7f95a2dff700 1 --2- 172.21.15.131:0/462600985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f959c0c8ab0 0x7f959c1509e0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.722+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 >> 172.21.15.131:0/462600985 conn(0x7f959c06ece0 msgr2=0x7f959c158690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:06.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.722+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 shutdown_connections 2023-12-26T23:12:06.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.722+0000 7f95a2dff700 1 -- 172.21.15.131:0/462600985 wait complete. 2023-12-26T23:12:06.724 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 90 2023-12-26T23:12:06.741 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:06.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.934+0000 7f4aeb7ff700 1 Processor -- start 2023-12-26T23:12:06.935 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.934+0000 7f4aeb7ff700 1 -- start start 2023-12-26T23:12:06.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.935+0000 7f4aeb7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae413c510 0x7f4ae40c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.935+0000 7f4aeb7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ae406b8c0 0x7f4ae406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.935+0000 7f4aeb7ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ae40cc100 0x7f4ae40cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.935+0000 7f4aeb7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae406c270 con 0x7f4ae406b8c0 2023-12-26T23:12:06.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.935+0000 7f4aeb7ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae40686c0 con 0x7f4ae40cc100 2023-12-26T23:12:06.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.935+0000 7f4aeb7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4069520 con 0x7f4ae413c510 2023-12-26T23:12:06.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae8d9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ae406b8c0 0x7f4ae406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae8d9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ae406b8c0 0x7f4ae406bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55598/0 (socket says 172.21.15.131:55598) 2023-12-26T23:12:06.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae8d9a700 1 -- 172.21.15.131:0/546801211 learned_addr learned my addr 172.21.15.131:0/546801211 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:06.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae959b700 1 --2- 172.21.15.131:0/546801211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae413c510 0x7f4ae40c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae9d9c700 1 --2- 172.21.15.131:0/546801211 >> v2:172.21.15.190:3300/0 conn(0x7f4ae40cc100 0x7f4ae40cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae8d9a700 1 -- 172.21.15.131:0/546801211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae413c510 msgr2=0x7f4ae40c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae8d9a700 1 --2- 172.21.15.131:0/546801211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae413c510 0x7f4ae40c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae8d9a700 1 -- 172.21.15.131:0/546801211 >> v2:172.21.15.190:3300/0 conn(0x7f4ae40cc100 msgr2=0x7f4ae40cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae8d9a700 1 --2- 172.21.15.131:0/546801211 >> v2:172.21.15.190:3300/0 conn(0x7f4ae40cc100 0x7f4ae40cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.936+0000 7f4ae8d9a700 1 -- 172.21.15.131:0/546801211 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae4068910 con 0x7f4ae406b8c0 2023-12-26T23:12:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.937+0000 7f4ae8d9a700 1 --2- 172.21.15.131:0/546801211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 0x7f4ae406bd30 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f4ae406f130 tx=0x7f4ad4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e034ac854385a58f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.937+0000 7f4adbfff700 1 -- 172.21.15.131:0/546801211 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400b640 con 0x7f4ae406b8c0 2023-12-26T23:12:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.937+0000 7f4adbfff700 1 -- 172.21.15.131:0/546801211 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad400b7a0 con 0x7f4ae406b8c0 2023-12-26T23:12:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.937+0000 7f4adbfff700 1 -- 172.21.15.131:0/546801211 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400bb60 con 0x7f4ae406b8c0 2023-12-26T23:12:06.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.937+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/546801211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 msgr2=0x7f4ae406bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.937+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/546801211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 0x7f4ae406bd30 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f4ae406f130 tx=0x7f4ad4009250 comp rx=0 tx=0).stop 2023-12-26T23:12:06.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.938+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/546801211 shutdown_connections 2023-12-26T23:12:06.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.938+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/546801211 >> v2:172.21.15.190:3300/0 conn(0x7f4ae40cc100 0x7f4ae40cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.938+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/546801211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae413c510 0x7f4ae40c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.938+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/546801211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 0x7f4ae406bd30 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.938+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/546801211 >> 172.21.15.131:0/546801211 conn(0x7f4ae40ce3e0 msgr2=0x7f4ae40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:06.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.938+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/546801211 shutdown_connections 2023-12-26T23:12:06.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.938+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/546801211 wait complete. 2023-12-26T23:12:06.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.939+0000 7f4aeb7ff700 1 Processor -- start 2023-12-26T23:12:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.939+0000 7f4aeb7ff700 1 -- start start 2023-12-26T23:12:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.940+0000 7f4aeb7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 0x7f4ae413bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.940+0000 7f4aeb7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae40cc100 0x7f4ae4154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.940+0000 7f4aeb7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae413c510 0x7f4ae4158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.940+0000 7f4aeb7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40b3340 con 0x7f4ae406b8c0 2023-12-26T23:12:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.940+0000 7f4aeb7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4155730 con 0x7f4ae413c510 2023-12-26T23:12:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.940+0000 7f4aeb7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41548a0 con 0x7f4ae40cc100 2023-12-26T23:12:06.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae959b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 0x7f4ae413bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae959b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 0x7f4ae413bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55600/0 (socket says 172.21.15.131:55600) 2023-12-26T23:12:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae959b700 1 -- 172.21.15.131:0/3609184314 learned_addr learned my addr 172.21.15.131:0/3609184314 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae8d9a700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae40cc100 0x7f4ae4154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae9d9c700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae413c510 0x7f4ae4158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae959b700 1 -- 172.21.15.131:0/3609184314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae40cc100 msgr2=0x7f4ae4154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae959b700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae40cc100 0x7f4ae4154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae959b700 1 -- 172.21.15.131:0/3609184314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae413c510 msgr2=0x7f4ae4158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:06.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae959b700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae413c510 0x7f4ae4158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae959b700 1 -- 172.21.15.131:0/3609184314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad40094f0 con 0x7f4ae406b8c0 2023-12-26T23:12:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae8d9a700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae40cc100 0x7f4ae4154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.941+0000 7f4ae959b700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 0x7f4ae413bf80 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000ecb0 tx=0x7f4ae0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.942+0000 7f4ad97fa700 1 -- 172.21.15.131:0/3609184314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000c850 con 0x7f4ae406b8c0 2023-12-26T23:12:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.942+0000 7f4ad97fa700 1 -- 172.21.15.131:0/3609184314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae000c9b0 con 0x7f4ae406b8c0 2023-12-26T23:12:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.942+0000 7f4ad97fa700 1 -- 172.21.15.131:0/3609184314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000cd30 con 0x7f4ae406b8c0 2023-12-26T23:12:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.942+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae4154b20 con 0x7f4ae406b8c0 2023-12-26T23:12:06.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.942+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae41550e0 con 0x7f4ae406b8c0 2023-12-26T23:12:06.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.943+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae40657e0 con 0x7f4ae406b8c0 2023-12-26T23:12:06.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.944+0000 7f4ad97fa700 1 -- 172.21.15.131:0/3609184314 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f4ae0013680 con 0x7f4ae406b8c0 2023-12-26T23:12:06.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.945+0000 7f4ad97fa700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4ac8067050 0x7f4ac8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:06.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.945+0000 7f4ad97fa700 1 -- 172.21.15.131:0/3609184314 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6540+0+0 (secure 0 0 0) 0x7f4ae0084ff0 con 0x7f4ae406b8c0 2023-12-26T23:12:06.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.945+0000 7f4ae8d9a700 1 -- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4ac8067050 msgr2=0x7f4ac8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:06.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.945+0000 7f4ae8d9a700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4ac8067050 0x7f4ac8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:06.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:06.949+0000 7f4ad97fa700 1 -- 172.21.15.131:0/3609184314 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ae0051e10 con 0x7f4ae406b8c0 2023-12-26T23:12:07.006 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:07.006+0000 7fd47a88c1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:12:07.099 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:07.099+0000 7fd47a88c1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:12:07.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.146+0000 7f4ae8d9a700 1 -- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4ac8067050 msgr2=0x7f4ac8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:07.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.146+0000 7f4ae8d9a700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4ac8067050 0x7f4ac8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:07.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.155+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ae4140440 con 0x7f4ae406b8c0 2023-12-26T23:12:07.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.156+0000 7f4ad97fa700 1 -- 172.21.15.131:0/3609184314 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16704 (secure 0 0 0) 0x7f4ae0017090 con 0x7f4ae406b8c0 2023-12-26T23:12:07.156 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:07.157 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":197,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:05.509837+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:09:43.598995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-26T23:09:45.039854+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":197,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.29","mappings":[{"from":5,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1645000428":"2023-12-27T23:10:02.694004+0000","172.21.15.131:0/280306402":"2023-12-27T23:10:02.694004+0000","172.21.15.131:0/2488529326":"2023-12-27T23:10:02.694004+0000","172.21.15.131:6837/92228":"2023-12-27T23:10:02.694004+0000","172.21.15.131:6836/92228":"2023-12-27T23:10:02.694004+0000","172.21.15.131:0/2520330293":"2023-12-27T23:10:02.694004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4ac8067050 msgr2=0x7f4ac8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4ac8067050 0x7f4ac8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 msgr2=0x7f4ae413bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 0x7f4ae413bf80 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000ecb0 tx=0x7f4ae0009120 comp rx=0 tx=0).stop 2023-12-26T23:12:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 shutdown_connections 2023-12-26T23:12:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4ac8067050 0x7f4ac8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ae413c510 0x7f4ae4158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ae406b8c0 0x7f4ae413bf80 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 --2- 172.21.15.131:0/3609184314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ae40cc100 0x7f4ae4154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 >> 172.21.15.131:0/3609184314 conn(0x7f4ae40ce3e0 msgr2=0x7f4ae40c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.158+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 shutdown_connections 2023-12-26T23:12:07.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.159+0000 7f4aeb7ff700 1 -- 172.21.15.131:0/3609184314 wait complete. 2023-12-26T23:12:07.178 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T23:12:07.178 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:07.242 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:07.242+0000 7fdbc28391c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:12:07.252 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:07.252+0000 7fd47a88c1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:12:07.342 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:07.342+0000 7fdbc28391c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:12:07.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.372+0000 7fa699fff700 1 Processor -- start 2023-12-26T23:12:07.372 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.372+0000 7fa699fff700 1 -- start start 2023-12-26T23:12:07.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.373+0000 7fa699fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa694159920 0x7fa6940b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.373+0000 7fa699fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159cf0 0x7fa6940b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.373+0000 7fa699fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6940c8bf0 con 0x7fa6941554d0 2023-12-26T23:12:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.373+0000 7fa699fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6940c8e60 con 0x7fa694159920 2023-12-26T23:12:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.373+0000 7fa699fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940c9140 con 0x7fa694159cf0 2023-12-26T23:12:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159cf0 0x7fa6940b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159cf0 0x7fa6940b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:59236/0 (socket says 172.21.15.131:59236) 2023-12-26T23:12:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa692ffd700 1 -- 172.21.15.131:0/2438246731 learned_addr learned my addr 172.21.15.131:0/2438246731 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa693fff700 1 --1- 172.21.15.131:0/2438246731 >> v1:172.21.15.190:6789/0 conn(0x7fa6941554d0 0x7fa694155930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:54470/0 (socket says 172.21.15.131:54470) 2023-12-26T23:12:07.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa6937fe700 1 --2- 172.21.15.131:0/2438246731 >> v2:172.21.15.190:3300/0 conn(0x7fa694159920 0x7fa6940b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa6927fc700 1 -- 172.21.15.131:0/2438246731 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240958887 0 0) 0x7fa6940c8bf0 con 0x7fa6941554d0 2023-12-26T23:12:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa692ffd700 1 -- 172.21.15.131:0/2438246731 >> v2:172.21.15.190:3300/0 conn(0x7fa694159920 msgr2=0x7fa6940b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa692ffd700 1 --2- 172.21.15.131:0/2438246731 >> v2:172.21.15.190:3300/0 conn(0x7fa694159920 0x7fa6940b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa692ffd700 1 -- 172.21.15.131:0/2438246731 >> v1:172.21.15.190:6789/0 conn(0x7fa6941554d0 legacy=0x7fa694155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.374+0000 7fa692ffd700 1 -- 172.21.15.131:0/2438246731 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6940c9420 con 0x7fa694159cf0 2023-12-26T23:12:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.375+0000 7fa692ffd700 1 --2- 172.21.15.131:0/2438246731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159cf0 0x7fa6940b0620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa68800bc50 tx=0x7fa68800b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8499f42d1c9ca88 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.375+0000 7fa6927fc700 1 -- 172.21.15.131:0/2438246731 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800dbf0 con 0x7fa694159cf0 2023-12-26T23:12:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.375+0000 7fa6927fc700 1 -- 172.21.15.131:0/2438246731 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68800dd50 con 0x7fa694159cf0 2023-12-26T23:12:07.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.375+0000 7fa6927fc700 1 -- 172.21.15.131:0/2438246731 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6880196b0 con 0x7fa694159cf0 2023-12-26T23:12:07.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.376+0000 7fa699fff700 1 -- 172.21.15.131:0/2438246731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159cf0 msgr2=0x7fa6940b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.376+0000 7fa699fff700 1 --2- 172.21.15.131:0/2438246731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159cf0 0x7fa6940b0620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa68800bc50 tx=0x7fa68800b4f0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.376+0000 7fa699fff700 1 -- 172.21.15.131:0/2438246731 shutdown_connections 2023-12-26T23:12:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.376+0000 7fa699fff700 1 --2- 172.21.15.131:0/2438246731 >> v2:172.21.15.190:3300/0 conn(0x7fa694159920 0x7fa6940b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.376+0000 7fa699fff700 1 --2- 172.21.15.131:0/2438246731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159cf0 0x7fa6940b0620 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.376+0000 7fa699fff700 1 -- 172.21.15.131:0/2438246731 >> 172.21.15.131:0/2438246731 conn(0x7fa694068040 msgr2=0x7fa694069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.376+0000 7fa699fff700 1 -- 172.21.15.131:0/2438246731 shutdown_connections 2023-12-26T23:12:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.376+0000 7fa699fff700 1 -- 172.21.15.131:0/2438246731 wait complete. 2023-12-26T23:12:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.377+0000 7fa699fff700 1 Processor -- start 2023-12-26T23:12:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.377+0000 7fa699fff700 1 -- start start 2023-12-26T23:12:07.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.378+0000 7fa699fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6941554d0 0x7fa69413ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.378+0000 7fa699fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159920 0x7fa69415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.378+0000 7fa699fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa694159cf0 0x7fa69415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.378+0000 7fa699fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940c62a0 con 0x7fa694159cf0 2023-12-26T23:12:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.378+0000 7fa699fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940c5690 con 0x7fa6941554d0 2023-12-26T23:12:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.378+0000 7fa699fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940c5970 con 0x7fa694159920 2023-12-26T23:12:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa693fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa694159cf0 0x7fa69415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159920 0x7fa69415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159920 0x7fa69415a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:59238/0 (socket says 172.21.15.131:59238) 2023-12-26T23:12:07.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa692ffd700 1 -- 172.21.15.131:0/3079838762 learned_addr learned my addr 172.21.15.131:0/3079838762 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa6937fe700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6941554d0 0x7fa69413ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa692ffd700 1 -- 172.21.15.131:0/3079838762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6941554d0 msgr2=0x7fa69413ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa692ffd700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6941554d0 0x7fa69413ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa692ffd700 1 -- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa694159cf0 msgr2=0x7fa69415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa692ffd700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa694159cf0 0x7fa69415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa692ffd700 1 -- 172.21.15.131:0/3079838762 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68800b790 con 0x7fa694159920 2023-12-26T23:12:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.379+0000 7fa692ffd700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159920 0x7fa69415a4b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa68800d680 tx=0x7fa68801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:07.384 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.380+0000 7fa67b7fe700 1 -- 172.21.15.131:0/3079838762 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688010680 con 0x7fa694159920 2023-12-26T23:12:07.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.380+0000 7fa67b7fe700 1 -- 172.21.15.131:0/3079838762 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68801e040 con 0x7fa694159920 2023-12-26T23:12:07.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.380+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6940c5c50 con 0x7fa694159920 2023-12-26T23:12:07.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.380+0000 7fa67b7fe700 1 -- 172.21.15.131:0/3079838762 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688010a50 con 0x7fa694159920 2023-12-26T23:12:07.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.380+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69413e310 con 0x7fa694159920 2023-12-26T23:12:07.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.381+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6940c6410 con 0x7fa694159920 2023-12-26T23:12:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.382+0000 7fa67b7fe700 1 -- 172.21.15.131:0/3079838762 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fa68802c430 con 0x7fa694159920 2023-12-26T23:12:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.382+0000 7fa67b7fe700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa680067050 0x7fa680069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.382+0000 7fa67b7fe700 1 -- 172.21.15.131:0/3079838762 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6540+0+0 (secure 0 0 0) 0x7fa688014070 con 0x7fa694159920 2023-12-26T23:12:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.382+0000 7fa6937fe700 1 -- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa680067050 msgr2=0x7fa680069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.383+0000 7fa6937fe700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa680067050 0x7fa680069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:07.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.386+0000 7fa67b7fe700 1 -- 172.21.15.131:0/3079838762 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa688053850 con 0x7fa694159920 2023-12-26T23:12:07.505 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:07.505+0000 7fdbc28391c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:12:07.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.584+0000 7fa6937fe700 1 -- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa680067050 msgr2=0x7fa680069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:07.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.584+0000 7fa6937fe700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa680067050 0x7fa680069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:07.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.637+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6940657e0 con 0x7fa694159920 2023-12-26T23:12:07.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.637+0000 7fa67b7fe700 1 -- 172.21.15.131:0/3079838762 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4613 (secure 0 0 0) 0x7fa688056e70 con 0x7fa694159920 2023-12-26T23:12:07.638 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:07.638 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2070005366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2070005366},{"type":"v1","addr":"172.21.15.190:6835","nonce":2070005366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3800715315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3800715315},{"type":"v1","addr":"172.21.15.190:6834","nonce":3800715315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9645,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3865566903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3865566903},{"type":"v1","addr":"172.21.15.131:6833","nonce":3865566903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:09:46.045975+0000","modified":"2023-12-26T23:09:49.597663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9666":{"gid":9666,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2967199312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2967199312},{"type":"v1","addr":"172.21.15.131:6835","nonce":2967199312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-12-26T23:12:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa680067050 msgr2=0x7fa680069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa680067050 0x7fa680069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159920 msgr2=0x7fa69415a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159920 0x7fa69415a4b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa68800d680 tx=0x7fa68801b000 comp rx=0 tx=0).stop 2023-12-26T23:12:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 shutdown_connections 2023-12-26T23:12:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa680067050 0x7fa680069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa694159cf0 0x7fa69415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6941554d0 0x7fa69413ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 --2- 172.21.15.131:0/3079838762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa694159920 0x7fa69415a4b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 >> 172.21.15.131:0/3079838762 conn(0x7fa694068040 msgr2=0x7fa6940aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 shutdown_connections 2023-12-26T23:12:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.640+0000 7fa699fff700 1 -- 172.21.15.131:0/3079838762 wait complete. 2023-12-26T23:12:07.643 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 90 2023-12-26T23:12:07.659 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.861+0000 7fc2171ff700 1 Processor -- start 2023-12-26T23:12:07.861 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.861+0000 7fc2171ff700 1 -- start start 2023-12-26T23:12:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.862+0000 7fc2171ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc2100ad560 0x7fc2101645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.862+0000 7fc2171ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2100a98b0 con 0x7fc21015d240 2023-12-26T23:12:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.862+0000 7fc2171ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21015a500 con 0x7fc2101595e0 2023-12-26T23:12:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.862+0000 7fc2171ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2100add40 con 0x7fc2100ad560 2023-12-26T23:12:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20ffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc21015d240 0x7fc2100ace50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34066/0 (socket says 172.21.15.131:34066) 2023-12-26T23:12:07.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20ffff700 1 -- 172.21.15.131:0/2648715688 learned_addr learned my addr 172.21.15.131:0/2648715688 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc21579c700 1 --2- 172.21.15.131:0/2648715688 >> v2:172.21.15.190:3300/0 conn(0x7fc2100ad560 0x7fc2101645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765312828 0 0) 0x7fc2100a98b0 con 0x7fc21015d240 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc200003570 con 0x7fc21015d240 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608181489 0 0) 0x7fc21015a500 con 0x7fc2101595e0 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2100a98b0 con 0x7fc2101595e0 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3526587291 0 0) 0x7fc200003570 con 0x7fc21015d240 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 >> v2:172.21.15.190:3300/0 conn(0x7fc2100ad560 msgr2=0x7fc2101645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20f7fe700 1 --2- 172.21.15.131:0/2648715688 >> v2:172.21.15.190:3300/0 conn(0x7fc2100ad560 0x7fc2101645e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 >> v1:172.21.15.190:6789/0 conn(0x7fc2101595e0 legacy=0x7fc2101599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.863+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21015a7e0 con 0x7fc21015d240 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fc1fc001470 con 0x7fc21015d240 2023-12-26T23:12:07.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 >> v1:172.21.15.131:6789/0 conn(0x7fc21015d240 legacy=0x7fc2100ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc20f7fe700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc200003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc20f7fe700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc200004810 0x7fc200004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc20f7fe700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000051c0 0x7fc20000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100add40 con 0x7fc200003bf0 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20000c0b0 con 0x7fc200004810 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20000c270 con 0x7fc2000051c0 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc214f9b700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc200003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc21579c700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000051c0 0x7fc20000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc20ffff700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc200004810 0x7fc200004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc214f9b700 1 -- 172.21.15.131:0/2648715688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000051c0 msgr2=0x7fc20000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc214f9b700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2000051c0 0x7fc20000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc214f9b700 1 -- 172.21.15.131:0/2648715688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc200004810 msgr2=0x7fc200004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc214f9b700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc200004810 0x7fc200004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc214f9b700 1 -- 172.21.15.131:0/2648715688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200003570 con 0x7fc200003bf0 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.864+0000 7fc21579c700 1 -- 172.21.15.131:0/2648715688 reap_dead start 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.865+0000 7fc214f9b700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc200003fe0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc204001a10 tx=0x7fc204007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80c75c033e0312 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.865+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20400c5d0 con 0x7fc200003bf0 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.865+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20400c730 con 0x7fc200003bf0 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.865+0000 7fc20f7fe700 1 -- 172.21.15.131:0/2648715688 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20400caf0 con 0x7fc200003bf0 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.866+0000 7fc2171ff700 1 -- 172.21.15.131:0/2648715688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 msgr2=0x7fc200003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.866+0000 7fc2171ff700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc200003fe0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc204001a10 tx=0x7fc204007d30 comp rx=0 tx=0).stop 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.866+0000 7fc2171ff700 1 -- 172.21.15.131:0/2648715688 shutdown_connections 2023-12-26T23:12:07.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.866+0000 7fc2171ff700 1 --2- 172.21.15.131:0/2648715688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc200003fe0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.866+0000 7fc2171ff700 1 -- 172.21.15.131:0/2648715688 >> 172.21.15.131:0/2648715688 conn(0x7fc210068040 msgr2=0x7fc210069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:07.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.866+0000 7fc2171ff700 1 -- 172.21.15.131:0/2648715688 shutdown_connections 2023-12-26T23:12:07.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.866+0000 7fc2171ff700 1 -- 172.21.15.131:0/2648715688 wait complete. 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.868+0000 7fc2171ff700 1 Processor -- start 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.868+0000 7fc2171ff700 1 -- start start 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc2171ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc2100c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc2171ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc210169f50 0x7fc21016a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc2171ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc210152a00 0x7fc210152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc2171ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100af580 con 0x7fc200003bf0 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc2171ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100afad0 con 0x7fc210169f50 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc2171ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2101533b0 con 0x7fc210152a00 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc214f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc2100c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc214f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc2100c1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55632/0 (socket says 172.21.15.131:55632) 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc214f9b700 1 -- 172.21.15.131:0/549336183 learned_addr learned my addr 172.21.15.131:0/549336183 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc21579c700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc210152a00 0x7fc210152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc20ffff700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc210169f50 0x7fc21016a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.870+0000 7fc21579c700 1 -- 172.21.15.131:0/549336183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc210169f50 msgr2=0x7fc21016a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc21579c700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc210169f50 0x7fc21016a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc21579c700 1 -- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 msgr2=0x7fc2100c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc21579c700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc2100c1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc21579c700 1 -- 172.21.15.131:0/549336183 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21016aa00 con 0x7fc210152a00 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc214f9b700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc2100c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc21579c700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc210152a00 0x7fc210152e70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc21006a640 tx=0x7fc20800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:07.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc20cff9700 1 -- 172.21.15.131:0/549336183 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800e040 con 0x7fc210152a00 2023-12-26T23:12:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc20cff9700 1 -- 172.21.15.131:0/549336183 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20800f040 con 0x7fc210152a00 2023-12-26T23:12:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21016ace0 con 0x7fc210152a00 2023-12-26T23:12:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc20cff9700 1 -- 172.21.15.131:0/549336183 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800fd70 con 0x7fc210152a00 2023-12-26T23:12:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.871+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2100cbc70 con 0x7fc210152a00 2023-12-26T23:12:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.872+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2100657e0 con 0x7fc210152a00 2023-12-26T23:12:07.875 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:07.874+0000 7fd47a88c1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:12:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.873+0000 7fc20cff9700 1 -- 172.21.15.131:0/549336183 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fc208015640 con 0x7fc210152a00 2023-12-26T23:12:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.874+0000 7fc20cff9700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc1f4067050 0x7fc1f4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.874+0000 7fc20cff9700 1 -- 172.21.15.131:0/549336183 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6540+0+0 (secure 0 0 0) 0x7fc208085280 con 0x7fc210152a00 2023-12-26T23:12:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.874+0000 7fc214f9b700 1 -- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc1f4067050 msgr2=0x7fc1f4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:07.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.874+0000 7fc214f9b700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc1f4067050 0x7fc1f4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:07.878+0000 7fc20cff9700 1 -- 172.21.15.131:0/549336183 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc208051ef0 con 0x7fc210152a00 2023-12-26T23:12:08.061 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:08.061+0000 7fd47a88c1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:12:08.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.075+0000 7fc214f9b700 1 -- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc1f4067050 msgr2=0x7fc1f4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:08.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.075+0000 7fc214f9b700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc1f4067050 0x7fc1f4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:08.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.126+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc21016aed0 con 0x7fc210152a00 2023-12-26T23:12:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.127+0000 7fc20cff9700 1 -- 172.21.15.131:0/549336183 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4613 (secure 0 0 0) 0x7fc208055510 con 0x7fc210152a00 2023-12-26T23:12:08.127 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:08.128 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2070005366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2070005366},{"type":"v1","addr":"172.21.15.190:6835","nonce":2070005366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3800715315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3800715315},{"type":"v1","addr":"172.21.15.190:6834","nonce":3800715315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9645,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3865566903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3865566903},{"type":"v1","addr":"172.21.15.131:6833","nonce":3865566903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:09:46.045975+0000","modified":"2023-12-26T23:09:49.597663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9666":{"gid":9666,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2967199312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2967199312},{"type":"v1","addr":"172.21.15.131:6835","nonce":2967199312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-12-26T23:12:08.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc1f4067050 msgr2=0x7fc1f4069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:08.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc1f4067050 0x7fc1f4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc210152a00 msgr2=0x7fc210152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc210152a00 0x7fc210152e70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc21006a640 tx=0x7fc20800b040 comp rx=0 tx=0).stop 2023-12-26T23:12:08.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 shutdown_connections 2023-12-26T23:12:08.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc1f4067050 0x7fc1f4069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc210152a00 0x7fc210152e70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200003bf0 0x7fc2100c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 --2- 172.21.15.131:0/549336183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc210169f50 0x7fc21016a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.129+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 >> 172.21.15.131:0/549336183 conn(0x7fc210068040 msgr2=0x7fc2100b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:08.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.130+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 shutdown_connections 2023-12-26T23:12:08.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.130+0000 7fc2171ff700 1 -- 172.21.15.131:0/549336183 wait complete. 2023-12-26T23:12:08.132 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 90 2023-12-26T23:12:08.149 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:08.169 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:08.169+0000 7fdbc28391c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:12:08.251 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:08.251+0000 7fd47a88c1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:12:08.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.341+0000 7fe1637ff700 1 Processor -- start 2023-12-26T23:12:08.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.342+0000 7fe1637ff700 1 -- start start 2023-12-26T23:12:08.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe1637ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe15c13c510 0x7fe15c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe1637ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe15c0cc810 0x7fe15c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe1637ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15c0686c0 con 0x7fe15c06b8c0 2023-12-26T23:12:08.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe1637ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c06c280 con 0x7fe15c13c510 2023-12-26T23:12:08.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe1637ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c069520 con 0x7fe15c0cc810 2023-12-26T23:12:08.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe16159b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe15c13c510 0x7fe15c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe160d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe15c06b8c0 0x7fe15c0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34082/0 (socket says 172.21.15.131:34082) 2023-12-26T23:12:08.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe160d9a700 1 -- 172.21.15.131:0/1797913913 learned_addr learned my addr 172.21.15.131:0/1797913913 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:08.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe161d9c700 1 --2- 172.21.15.131:0/1797913913 >> v2:172.21.15.190:3300/0 conn(0x7fe15c0cc810 0x7fe15c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe16159b700 1 -- 172.21.15.131:0/1797913913 >> v2:172.21.15.190:3300/0 conn(0x7fe15c0cc810 msgr2=0x7fe15c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe16159b700 1 --2- 172.21.15.131:0/1797913913 >> v2:172.21.15.190:3300/0 conn(0x7fe15c0cc810 0x7fe15c06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe153fff700 1 -- 172.21.15.131:0/1797913913 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159659651 0 0) 0x7fe15c0686c0 con 0x7fe15c06b8c0 2023-12-26T23:12:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe16159b700 1 -- 172.21.15.131:0/1797913913 >> v1:172.21.15.131:6789/0 conn(0x7fe15c06b8c0 legacy=0x7fe15c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.344+0000 7fe16159b700 1 -- 172.21.15.131:0/1797913913 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c068910 con 0x7fe15c13c510 2023-12-26T23:12:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.345+0000 7fe16159b700 1 --2- 172.21.15.131:0/1797913913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 0x7fe15c0c8a20 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fe15c06f6a0 tx=0x7fe158009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc07612560dbb786 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.345+0000 7fe153fff700 1 -- 172.21.15.131:0/1797913913 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158005c10 con 0x7fe15c13c510 2023-12-26T23:12:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.345+0000 7fe153fff700 1 -- 172.21.15.131:0/1797913913 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158005d70 con 0x7fe15c13c510 2023-12-26T23:12:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.345+0000 7fe153fff700 1 -- 172.21.15.131:0/1797913913 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158006260 con 0x7fe15c13c510 2023-12-26T23:12:08.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.346+0000 7fe1637ff700 1 -- 172.21.15.131:0/1797913913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 msgr2=0x7fe15c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.346+0000 7fe1637ff700 1 --2- 172.21.15.131:0/1797913913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 0x7fe15c0c8a20 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fe15c06f6a0 tx=0x7fe158009120 comp rx=0 tx=0).stop 2023-12-26T23:12:08.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.346+0000 7fe1637ff700 1 -- 172.21.15.131:0/1797913913 shutdown_connections 2023-12-26T23:12:08.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.346+0000 7fe1637ff700 1 --2- 172.21.15.131:0/1797913913 >> v2:172.21.15.190:3300/0 conn(0x7fe15c0cc810 0x7fe15c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.346+0000 7fe1637ff700 1 --2- 172.21.15.131:0/1797913913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 0x7fe15c0c8a20 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.346+0000 7fe1637ff700 1 -- 172.21.15.131:0/1797913913 >> 172.21.15.131:0/1797913913 conn(0x7fe15c0ce3e0 msgr2=0x7fe15c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:08.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.346+0000 7fe1637ff700 1 -- 172.21.15.131:0/1797913913 shutdown_connections 2023-12-26T23:12:08.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.346+0000 7fe1637ff700 1 -- 172.21.15.131:0/1797913913 wait complete. 2023-12-26T23:12:08.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.348+0000 7fe1637ff700 1 Processor -- start 2023-12-26T23:12:08.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.348+0000 7fe1637ff700 1 -- start start 2023-12-26T23:12:08.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe1637ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe15c06b8c0 0x7fe15c15e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe1637ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe15c0cc810 0x7fe15c0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe1637ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 0x7fe15c0b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe1637ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c0b4020 con 0x7fe15c13c510 2023-12-26T23:12:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe1637ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c16d7f0 con 0x7fe15c06b8c0 2023-12-26T23:12:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe1637ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c15d790 con 0x7fe15c0cc810 2023-12-26T23:12:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe161d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 0x7fe15c0b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe160d9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe15c0cc810 0x7fe15c0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe161d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 0x7fe15c0b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55638/0 (socket says 172.21.15.131:55638) 2023-12-26T23:12:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe161d9c700 1 -- 172.21.15.131:0/2528056647 learned_addr learned my addr 172.21.15.131:0/2528056647 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.350+0000 7fe16159b700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe15c06b8c0 0x7fe15c15e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe161d9c700 1 -- 172.21.15.131:0/2528056647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe15c0cc810 msgr2=0x7fe15c0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe161d9c700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe15c0cc810 0x7fe15c0b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe161d9c700 1 -- 172.21.15.131:0/2528056647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe15c06b8c0 msgr2=0x7fe15c15e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe161d9c700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe15c06b8c0 0x7fe15c15e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe161d9c700 1 -- 172.21.15.131:0/2528056647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580093c0 con 0x7fe15c13c510 2023-12-26T23:12:08.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe161d9c700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 0x7fe15c0b15e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fe154009fa0 tx=0x7fe154009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:08.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe1517fa700 1 -- 172.21.15.131:0/2528056647 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15400ebc0 con 0x7fe15c13c510 2023-12-26T23:12:08.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe1517fa700 1 -- 172.21.15.131:0/2528056647 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15400ed20 con 0x7fe15c13c510 2023-12-26T23:12:08.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe1517fa700 1 -- 172.21.15.131:0/2528056647 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154014670 con 0x7fe15c13c510 2023-12-26T23:12:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.351+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c15da70 con 0x7fe15c13c510 2023-12-26T23:12:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.352+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c15e310 con 0x7fe15c13c510 2023-12-26T23:12:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.353+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c15e4d0 con 0x7fe15c13c510 2023-12-26T23:12:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.353+0000 7fe1517fa700 1 -- 172.21.15.131:0/2528056647 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fe1540149e0 con 0x7fe15c13c510 2023-12-26T23:12:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.354+0000 7fe1517fa700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe140067050 0x7fe140069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.354+0000 7fe1517fa700 1 -- 172.21.15.131:0/2528056647 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6540+0+0 (secure 0 0 0) 0x7fe154085090 con 0x7fe15c13c510 2023-12-26T23:12:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.354+0000 7fe16159b700 1 -- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe140067050 msgr2=0x7fe140069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.354+0000 7fe16159b700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe140067050 0x7fe140069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:08.358 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.358+0000 7fe1517fa700 1 -- 172.21.15.131:0/2528056647 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe154051eb0 con 0x7fe15c13c510 2023-12-26T23:12:08.371 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:08.371+0000 7fdbc28391c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:12:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.556+0000 7fe16159b700 1 -- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe140067050 msgr2=0x7fe140069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.556+0000 7fe16159b700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe140067050 0x7fe140069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.559+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe15c0657e0 con 0x7fe15c13c510 2023-12-26T23:12:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.560+0000 7fe1517fa700 1 -- 172.21.15.131:0/2528056647 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16704 (secure 0 0 0) 0x7fe1540554d0 con 0x7fe15c13c510 2023-12-26T23:12:08.561 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:08.561 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":197,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:05.509837+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:09:43.598995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-26T23:09:45.039854+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":197,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.29","mappings":[{"from":5,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1645000428":"2023-12-27T23:10:02.694004+0000","172.21.15.131:0/280306402":"2023-12-27T23:10:02.694004+0000","172.21.15.131:0/2488529326":"2023-12-27T23:10:02.694004+0000","172.21.15.131:6837/92228":"2023-12-27T23:10:02.694004+0000","172.21.15.131:6836/92228":"2023-12-27T23:10:02.694004+0000","172.21.15.131:0/2520330293":"2023-12-27T23:10:02.694004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe140067050 msgr2=0x7fe140069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe140067050 0x7fe140069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 msgr2=0x7fe15c0b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 0x7fe15c0b15e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fe154009fa0 tx=0x7fe154009900 comp rx=0 tx=0).stop 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 shutdown_connections 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe140067050 0x7fe140069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe15c13c510 0x7fe15c0b15e0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe15c06b8c0 0x7fe15c15e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 --2- 172.21.15.131:0/2528056647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe15c0cc810 0x7fe15c0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 >> 172.21.15.131:0/2528056647 conn(0x7fe15c0ce3e0 msgr2=0x7fe15c1557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:08.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 shutdown_connections 2023-12-26T23:12:08.565 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.563+0000 7fe1637ff700 1 -- 172.21.15.131:0/2528056647 wait complete. 2023-12-26T23:12:08.567 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:08.567+0000 7fdbc28391c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:12:08.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T23:12:08.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.816+0000 7fd60cdff700 1 Processor -- start 2023-12-26T23:12:08.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.816+0000 7fd60cdff700 1 -- start start 2023-12-26T23:12:08.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd60cdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd608159a70 0x7fd6080b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd60cdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6080b2a90 0x7fd6080b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd60cdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6080afcd0 con 0x7fd6081596a0 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd60cdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6080b3e70 con 0x7fd6080b2a90 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd60cdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6080b0f20 con 0x7fd608159a70 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd606d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6080b2a90 0x7fd6080b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd606d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6080b2a90 0x7fd6080b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55642/0 (socket says 172.21.15.131:55642) 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd606d9d700 1 -- 172.21.15.131:0/3004594553 learned_addr learned my addr 172.21.15.131:0/3004594553 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd605d9b700 1 --2- 172.21.15.131:0/3004594553 >> v2:172.21.15.190:3300/0 conn(0x7fd608159a70 0x7fd6080b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd606d9d700 1 -- 172.21.15.131:0/3004594553 >> v2:172.21.15.190:3300/0 conn(0x7fd608159a70 msgr2=0x7fd6080b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd606d9d700 1 --2- 172.21.15.131:0/3004594553 >> v2:172.21.15.190:3300/0 conn(0x7fd608159a70 0x7fd6080b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd606d9d700 1 -- 172.21.15.131:0/3004594553 >> v1:172.21.15.190:6789/0 conn(0x7fd6081596a0 legacy=0x7fd6080b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.820 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.819+0000 7fd606d9d700 1 -- 172.21.15.131:0/3004594553 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080b3540 con 0x7fd6080b2a90 2023-12-26T23:12:08.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.818+0000 7fd60559a700 1 -- 172.21.15.131:0/3004594553 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22477256 0 0) 0x7fd6080afcd0 con 0x7fd6081596a0 2023-12-26T23:12:08.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.819+0000 7fd606d9d700 1 --2- 172.21.15.131:0/3004594553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6080b2a90 0x7fd6080b06a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fd6080642f0 tx=0x7fd600009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5911f172edbf0634 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:08.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.819+0000 7fd60559a700 1 -- 172.21.15.131:0/3004594553 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60000e070 con 0x7fd6080b2a90 2023-12-26T23:12:08.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.819+0000 7fd60559a700 1 -- 172.21.15.131:0/3004594553 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60000ba20 con 0x7fd6080b2a90 2023-12-26T23:12:08.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.819+0000 7fd60559a700 1 -- 172.21.15.131:0/3004594553 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000092e0 con 0x7fd6080b2a90 2023-12-26T23:12:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.820+0000 7fd60cdff700 1 -- 172.21.15.131:0/3004594553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6080b2a90 msgr2=0x7fd6080b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.820+0000 7fd60cdff700 1 --2- 172.21.15.131:0/3004594553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6080b2a90 0x7fd6080b06a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fd6080642f0 tx=0x7fd600009840 comp rx=0 tx=0).stop 2023-12-26T23:12:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.820+0000 7fd60cdff700 1 -- 172.21.15.131:0/3004594553 shutdown_connections 2023-12-26T23:12:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.820+0000 7fd60cdff700 1 --2- 172.21.15.131:0/3004594553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6080b2a90 0x7fd6080b06a0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.820+0000 7fd60cdff700 1 --2- 172.21.15.131:0/3004594553 >> v2:172.21.15.190:3300/0 conn(0x7fd608159a70 0x7fd6080b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.820+0000 7fd60cdff700 1 -- 172.21.15.131:0/3004594553 >> 172.21.15.131:0/3004594553 conn(0x7fd60806ece0 msgr2=0x7fd60806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.820+0000 7fd60cdff700 1 -- 172.21.15.131:0/3004594553 shutdown_connections 2023-12-26T23:12:08.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.820+0000 7fd60cdff700 1 -- 172.21.15.131:0/3004594553 wait complete. 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.821+0000 7fd60cdff700 1 Processor -- start 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.822+0000 7fd60cdff700 1 -- start start 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.822+0000 7fd60cdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6080b2a90 0x7fd6080c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.822+0000 7fd60cdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6081596a0 0x7fd6080cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.822+0000 7fd60cdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd608159a70 0x7fd6080cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.822+0000 7fd60cdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080c8d10 con 0x7fd6081596a0 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.822+0000 7fd60cdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080c9890 con 0x7fd6080b2a90 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.822+0000 7fd60cdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080c8f60 con 0x7fd608159a70 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd605d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6081596a0 0x7fd6080cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd605d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6081596a0 0x7fd6080cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55652/0 (socket says 172.21.15.131:55652) 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd605d9b700 1 -- 172.21.15.131:0/4285882545 learned_addr learned my addr 172.21.15.131:0/4285882545 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd606d9d700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd608159a70 0x7fd6080cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd60659c700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6080b2a90 0x7fd6080c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd605d9b700 1 -- 172.21.15.131:0/4285882545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd608159a70 msgr2=0x7fd6080cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd605d9b700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd608159a70 0x7fd6080cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd605d9b700 1 -- 172.21.15.131:0/4285882545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6080b2a90 msgr2=0x7fd6080c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd605d9b700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6080b2a90 0x7fd6080c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd605d9b700 1 -- 172.21.15.131:0/4285882545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd600009ae0 con 0x7fd6081596a0 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.823+0000 7fd605d9b700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6081596a0 0x7fd6080cb200 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e960 tx=0x7fd5fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.824+0000 7fd5f67fc700 1 -- 172.21.15.131:0/4285882545 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc013680 con 0x7fd6081596a0 2023-12-26T23:12:08.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.824+0000 7fd5f67fc700 1 -- 172.21.15.131:0/4285882545 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc0137e0 con 0x7fd6081596a0 2023-12-26T23:12:08.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.824+0000 7fd5f67fc700 1 -- 172.21.15.131:0/4285882545 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc013b80 con 0x7fd6081596a0 2023-12-26T23:12:08.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.824+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6080c9240 con 0x7fd6081596a0 2023-12-26T23:12:08.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.824+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6080ca260 con 0x7fd6081596a0 2023-12-26T23:12:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.825+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6080657e0 con 0x7fd6081596a0 2023-12-26T23:12:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.826+0000 7fd5f67fc700 1 -- 172.21.15.131:0/4285882545 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7fd5fc0123d0 con 0x7fd6081596a0 2023-12-26T23:12:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.827+0000 7fd5f67fc700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5e8067050 0x7fd5e8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.827+0000 7fd5f67fc700 1 -- 172.21.15.131:0/4285882545 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6540+0+0 (secure 0 0 0) 0x7fd5fc084ba0 con 0x7fd6081596a0 2023-12-26T23:12:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.827+0000 7fd60659c700 1 -- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5e8067050 msgr2=0x7fd5e8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:08.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.827+0000 7fd60659c700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5e8067050 0x7fd5e8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:08.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:08.833+0000 7fd5f67fc700 1 -- 172.21.15.131:0/4285882545 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5fc051920 con 0x7fd6081596a0 2023-12-26T23:12:09.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.028+0000 7fd60659c700 1 -- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5e8067050 msgr2=0x7fd5e8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:09.028 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.028+0000 7fd60659c700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5e8067050 0x7fd5e8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:09.076 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.076+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd6080c99d0 con 0x7fd6081596a0 2023-12-26T23:12:09.076 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T23:12:09.076+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T23:12:09.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.081+0000 7fd5f67fc700 1 -- 172.21.15.131:0/4285882545 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd5fc054f40 con 0x7fd6081596a0 2023-12-26T23:12:09.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5e8067050 msgr2=0x7fd5e8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:09.085 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5e8067050 0x7fd5e8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6081596a0 msgr2=0x7fd6080cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6081596a0 0x7fd6080cb200 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e960 tx=0x7fd5fc0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 shutdown_connections 2023-12-26T23:12:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5e8067050 0x7fd5e8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd608159a70 0x7fd6080cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6080b2a90 0x7fd6080c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 --2- 172.21.15.131:0/4285882545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6081596a0 0x7fd6080cb200 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.084+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 >> 172.21.15.131:0/4285882545 conn(0x7fd60806ece0 msgr2=0x7fd6080cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:09.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.085+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 shutdown_connections 2023-12-26T23:12:09.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.085+0000 7fd60cdff700 1 -- 172.21.15.131:0/4285882545 wait complete. 2023-12-26T23:12:09.087 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T23:12:09.096 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:09.096+0000 7fd47a88c1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:12:09.105 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T23:12:09.143 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T23:12:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.302+0000 7f970dbff700 1 Processor -- start 2023-12-26T23:12:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.302+0000 7f970dbff700 1 -- start start 2023-12-26T23:12:09.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f970dbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f970806e480 0x7f97080abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:09.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f970dbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97080aa480 0x7f97080675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:09.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f970dbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970815a8c0 con 0x7f9708159820 2023-12-26T23:12:09.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f970dbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970815ad50 con 0x7f970806e480 2023-12-26T23:12:09.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f970dbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9708159f90 con 0x7f97080aa480 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f9706ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f970806e480 0x7f97080abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f97077fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9708159820 0x7f970806dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34094/0 (socket says 172.21.15.131:34094) 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f97077fe700 1 -- 172.21.15.131:0/541276540 learned_addr learned my addr 172.21.15.131:0/541276540 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f9707fff700 1 --2- 172.21.15.131:0/541276540 >> v2:172.21.15.190:3300/0 conn(0x7f97080aa480 0x7f97080675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f9706ffd700 1 -- 172.21.15.131:0/541276540 >> v2:172.21.15.190:3300/0 conn(0x7f97080aa480 msgr2=0x7f97080675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f9706ffd700 1 --2- 172.21.15.131:0/541276540 >> v2:172.21.15.190:3300/0 conn(0x7f97080aa480 0x7f97080675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f9706ffd700 1 -- 172.21.15.131:0/541276540 >> v1:172.21.15.131:6789/0 conn(0x7f9708159820 legacy=0x7f970806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f9706ffd700 1 -- 172.21.15.131:0/541276540 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970815a270 con 0x7f970806e480 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f9707fff700 1 --2- 172.21.15.131:0/541276540 >> v2:172.21.15.190:3300/0 conn(0x7f97080aa480 0x7f97080675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.304+0000 7f97067fc700 1 -- 172.21.15.131:0/541276540 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852672699 0 0) 0x7f970815a8c0 con 0x7f9708159820 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.305+0000 7f9706ffd700 1 --2- 172.21.15.131:0/541276540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f970806e480 0x7f97080abdb0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f970806cbd0 tx=0x7f96fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cada811b733dc504 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.305+0000 7f97067fc700 1 -- 172.21.15.131:0/541276540 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00b6b0 con 0x7f970806e480 2023-12-26T23:12:09.306 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.305+0000 7f97067fc700 1 -- 172.21.15.131:0/541276540 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96fc00b810 con 0x7f970806e480 2023-12-26T23:12:09.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.305+0000 7f97067fc700 1 -- 172.21.15.131:0/541276540 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00bbb0 con 0x7f970806e480 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.305+0000 7f970dbff700 1 -- 172.21.15.131:0/541276540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f970806e480 msgr2=0x7f97080abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.305+0000 7f970dbff700 1 --2- 172.21.15.131:0/541276540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f970806e480 0x7f97080abdb0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f970806cbd0 tx=0x7f96fc0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.306+0000 7f970dbff700 1 -- 172.21.15.131:0/541276540 shutdown_connections 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.306+0000 7f970dbff700 1 --2- 172.21.15.131:0/541276540 >> v2:172.21.15.190:3300/0 conn(0x7f97080aa480 0x7f97080675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.306+0000 7f970dbff700 1 --2- 172.21.15.131:0/541276540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f970806e480 0x7f97080abdb0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.306+0000 7f970dbff700 1 -- 172.21.15.131:0/541276540 >> 172.21.15.131:0/541276540 conn(0x7f970806c290 msgr2=0x7f97080ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.306+0000 7f970dbff700 1 -- 172.21.15.131:0/541276540 shutdown_connections 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.306+0000 7f970dbff700 1 -- 172.21.15.131:0/541276540 wait complete. 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.307+0000 7f970dbff700 1 Processor -- start 2023-12-26T23:12:09.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.307+0000 7f970dbff700 1 -- start start 2023-12-26T23:12:09.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f970dbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970806e480 0x7f9708143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:09.310 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f970dbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97080aa480 0x7f9708158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f970dbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708159820 0x7f97081589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f970dbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97080c2290 con 0x7f97080aa480 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f970dbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97080ae4f0 con 0x7f970806e480 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f970dbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97080c1960 con 0x7f9708159820 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f9706ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97080aa480 0x7f9708158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f9706ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97080aa480 0x7f9708158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55678/0 (socket says 172.21.15.131:55678) 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f9706ffd700 1 -- 172.21.15.131:0/148676654 learned_addr learned my addr 172.21.15.131:0/148676654 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.308+0000 7f97077fe700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970806e480 0x7f9708143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f9707fff700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708159820 0x7f97081589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f9706ffd700 1 -- 172.21.15.131:0/148676654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708159820 msgr2=0x7f97081589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f9706ffd700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708159820 0x7f97081589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f9706ffd700 1 -- 172.21.15.131:0/148676654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970806e480 msgr2=0x7f9708143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:09.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f9706ffd700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970806e480 0x7f9708143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:09.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f9706ffd700 1 -- 172.21.15.131:0/148676654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc009420 con 0x7f97080aa480 2023-12-26T23:12:09.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f9706ffd700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97080aa480 0x7f9708158490 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0093f0 tx=0x7f96fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:09.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f96ef7fe700 1 -- 172.21.15.131:0/148676654 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00bb90 con 0x7f97080aa480 2023-12-26T23:12:09.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f96ef7fe700 1 -- 172.21.15.131:0/148676654 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96fc01be00 con 0x7f97080aa480 2023-12-26T23:12:09.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f970dbff700 1 -- 172.21.15.131:0/148676654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97080c1c40 con 0x7f97080aa480 2023-12-26T23:12:09.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.309+0000 7f96ef7fe700 1 -- 172.21.15.131:0/148676654 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc011650 con 0x7f97080aa480 2023-12-26T23:12:09.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.310+0000 7f970dbff700 1 -- 172.21.15.131:0/148676654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9708144190 con 0x7f97080aa480 2023-12-26T23:12:09.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.311+0000 7f970dbff700 1 -- 172.21.15.131:0/148676654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97080657e0 con 0x7f97080aa480 2023-12-26T23:12:09.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.311+0000 7f96ef7fe700 1 -- 172.21.15.131:0/148676654 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 104) v1 ==== 84856+0+0 (secure 0 0 0) 0x7f96fc011a40 con 0x7f97080aa480 2023-12-26T23:12:09.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.312+0000 7f96ef7fe700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 0x7f96f4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:09.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.312+0000 7f96ef7fe700 1 -- 172.21.15.131:0/148676654 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6598+0+0 (secure 0 0 0) 0x7f96fc085f10 con 0x7f97080aa480 2023-12-26T23:12:09.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.312+0000 7f97077fe700 1 -- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 msgr2=0x7f96f4069530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:09.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.313+0000 7f97077fe700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 0x7f96f4069530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:09.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.316+0000 7f96ef7fe700 1 -- 172.21.15.131:0/148676654 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96fc052cd0 con 0x7f97080aa480 2023-12-26T23:12:09.373 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:09.373+0000 7fd47a88c1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:09.462 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:09.463+0000 7fdbc28391c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:12:09.466 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:09.466+0000 7fd47a88c1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:09.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.514+0000 7f97077fe700 1 -- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 msgr2=0x7f96f4069530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:09.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.514+0000 7f97077fe700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 0x7f96f4069530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:09.561 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.561+0000 7f970dbff700 1 -- 172.21.15.131:0/148676654 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f970815a550 con 0x7f97080aa480 2023-12-26T23:12:09.667 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:09.667+0000 7fd47a88c1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:12:09.750 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:09.750+0000 7fdbc28391c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:09.845 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:09.845+0000 7fdbc28391c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:09.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.915+0000 7f97077fe700 1 -- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 msgr2=0x7f96f4069530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:09.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:09.915+0000 7f97077fe700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 0x7f96f4069530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T23:12:10.044 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:10.044+0000 7fdbc28391c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:12:10.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.052+0000 7f96ef7fe700 1 -- 172.21.15.131:0/148676654 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 105) v1 ==== 84453+0+0 (secure 0 0 0) 0x7f96fc0530a0 con 0x7f97080aa480 2023-12-26T23:12:10.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.052+0000 7f96ef7fe700 1 -- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 msgr2=0x7f96f4069530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:10.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.052+0000 7f96ef7fe700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 0x7f96f4069530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.083 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.082+0000 7f96ef7fe700 1 -- 172.21.15.131:0/148676654 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7f96fc059620 con 0x7f97080aa480 2023-12-26T23:12:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.085+0000 7f970dbff700 1 -- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97080aa480 msgr2=0x7f9708158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.085+0000 7f970dbff700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97080aa480 0x7f9708158490 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0093f0 tx=0x7f96fc017000 comp rx=0 tx=0).stop 2023-12-26T23:12:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.085+0000 7f970dbff700 1 -- 172.21.15.131:0/148676654 shutdown_connections 2023-12-26T23:12:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.085+0000 7f970dbff700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96f4067080 0x7f96f4069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.085+0000 7f970dbff700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9708159820 0x7f97081589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.085+0000 7f970dbff700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f970806e480 0x7f9708143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.085+0000 7f970dbff700 1 --2- 172.21.15.131:0/148676654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97080aa480 0x7f9708158490 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.085+0000 7f970dbff700 1 -- 172.21.15.131:0/148676654 >> 172.21.15.131:0/148676654 conn(0x7f970806c290 msgr2=0x7f9708156500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.086+0000 7f970dbff700 1 -- 172.21.15.131:0/148676654 shutdown_connections 2023-12-26T23:12:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.086+0000 7f970dbff700 1 -- 172.21.15.131:0/148676654 wait complete. 2023-12-26T23:12:10.111 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T23:12:10.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.343+0000 7f7cecfff700 1 Processor -- start 2023-12-26T23:12:10.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.344+0000 7f7cecfff700 1 -- start start 2023-12-26T23:12:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7cecfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 0x7f7ce815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7cecfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ce8156e50 0x7f7ce81572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7cecfff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce80af040 con 0x7f7ce80c8d40 2023-12-26T23:12:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7cecfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce80ae0e0 con 0x7f7ce8156e50 2023-12-26T23:12:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7cecfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80ae3c0 con 0x7f7ce80c9110 2023-12-26T23:12:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7ce659c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7ce80c8d40 0x7f7ce80ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34110/0 (socket says 172.21.15.131:34110) 2023-12-26T23:12:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7ce659c700 1 -- 172.21.15.131:0/2291724350 learned_addr learned my addr 172.21.15.131:0/2291724350 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7ce6d9d700 1 --2- 172.21.15.131:0/2291724350 >> v2:172.21.15.190:3300/0 conn(0x7f7ce8156e50 0x7f7ce81572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7ce5d9b700 1 --2- 172.21.15.131:0/2291724350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 0x7f7ce815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.345+0000 7f7ce559a700 1 -- 172.21.15.131:0/2291724350 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140498537 0 0) 0x7f7ce80af040 con 0x7f7ce80c8d40 2023-12-26T23:12:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.346+0000 7f7ce559a700 1 -- 172.21.15.131:0/2291724350 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cd0003570 con 0x7f7ce80c8d40 2023-12-26T23:12:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.346+0000 7f7ce6d9d700 1 -- 172.21.15.131:0/2291724350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 msgr2=0x7f7ce815bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.346+0000 7f7ce6d9d700 1 --2- 172.21.15.131:0/2291724350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 0x7f7ce815bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.346+0000 7f7ce6d9d700 1 -- 172.21.15.131:0/2291724350 >> v1:172.21.15.131:6789/0 conn(0x7f7ce80c8d40 legacy=0x7f7ce80ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.346+0000 7f7ce6d9d700 1 -- 172.21.15.131:0/2291724350 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce80ae6a0 con 0x7f7ce8156e50 2023-12-26T23:12:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.346+0000 7f7ce6d9d700 1 --2- 172.21.15.131:0/2291724350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 0x7f7ce81572d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009e60 tx=0x7f7ce0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57042c4a4a145b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.346+0000 7f7ce559a700 1 -- 172.21.15.131:0/2291724350 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000e070 con 0x7f7ce8156e50 2023-12-26T23:12:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.346+0000 7f7ce559a700 1 -- 172.21.15.131:0/2291724350 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce000bab0 con 0x7f7ce8156e50 2023-12-26T23:12:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.346+0000 7f7ce559a700 1 -- 172.21.15.131:0/2291724350 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000be70 con 0x7f7ce8156e50 2023-12-26T23:12:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.347+0000 7f7cecfff700 1 -- 172.21.15.131:0/2291724350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 msgr2=0x7f7ce81572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.347+0000 7f7cecfff700 1 --2- 172.21.15.131:0/2291724350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 0x7f7ce81572d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009e60 tx=0x7f7ce0009780 comp rx=0 tx=0).stop 2023-12-26T23:12:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.347+0000 7f7cecfff700 1 -- 172.21.15.131:0/2291724350 shutdown_connections 2023-12-26T23:12:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.347+0000 7f7cecfff700 1 --2- 172.21.15.131:0/2291724350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 0x7f7ce81572d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.347+0000 7f7cecfff700 1 --2- 172.21.15.131:0/2291724350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 0x7f7ce815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.347+0000 7f7cecfff700 1 -- 172.21.15.131:0/2291724350 >> 172.21.15.131:0/2291724350 conn(0x7f7ce80675b0 msgr2=0x7f7ce80679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.347+0000 7f7cecfff700 1 -- 172.21.15.131:0/2291724350 shutdown_connections 2023-12-26T23:12:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.347+0000 7f7cecfff700 1 -- 172.21.15.131:0/2291724350 wait complete. 2023-12-26T23:12:10.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.348+0000 7f7cecfff700 1 Processor -- start 2023-12-26T23:12:10.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.348+0000 7f7cecfff700 1 -- start start 2023-12-26T23:12:10.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.349+0000 7f7cecfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce80c8d40 0x7f7ce80c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:10.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.349+0000 7f7cecfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 0x7f7ce814aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:10.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.349+0000 7f7cecfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 0x7f7ce806e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:10.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.349+0000 7f7cecfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80c8530 con 0x7f7ce80c8d40 2023-12-26T23:12:10.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.349+0000 7f7cecfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80c0b20 con 0x7f7ce8156e50 2023-12-26T23:12:10.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.349+0000 7f7cecfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80c0e00 con 0x7f7ce80c9110 2023-12-26T23:12:10.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce80c8d40 0x7f7ce80c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:10.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce80c8d40 0x7f7ce80c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55792/0 (socket says 172.21.15.131:55792) 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce659c700 1 -- 172.21.15.131:0/1234192878 learned_addr learned my addr 172.21.15.131:0/1234192878 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce6d9d700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 0x7f7ce806e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce5d9b700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 0x7f7ce814aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce659c700 1 -- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 msgr2=0x7f7ce814aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce659c700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 0x7f7ce814aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce659c700 1 -- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 msgr2=0x7f7ce806e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce659c700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 0x7f7ce806e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce659c700 1 -- 172.21.15.131:0/1234192878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce0009a20 con 0x7f7ce80c8d40 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce5d9b700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 0x7f7ce814aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:10.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce6d9d700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 0x7f7ce806e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7ce659c700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce80c8d40 0x7f7ce80c7ff0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800c7c0 tx=0x7f7cd8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.350+0000 7f7cd67fc700 1 -- 172.21.15.131:0/1234192878 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8005b30 con 0x7f7ce80c8d40 2023-12-26T23:12:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.351+0000 7f7cd67fc700 1 -- 172.21.15.131:0/1234192878 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd8005c90 con 0x7f7ce80c8d40 2023-12-26T23:12:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.351+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce80c10e0 con 0x7f7ce80c8d40 2023-12-26T23:12:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.351+0000 7f7cd67fc700 1 -- 172.21.15.131:0/1234192878 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8007340 con 0x7f7ce80c8d40 2023-12-26T23:12:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.351+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce80c16a0 con 0x7f7ce80c8d40 2023-12-26T23:12:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.352+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce80c1cf0 con 0x7f7ce80c8d40 2023-12-26T23:12:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.353+0000 7f7cd67fc700 1 -- 172.21.15.131:0/1234192878 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 105) v1 ==== 84453+0+0 (secure 0 0 0) 0x7f7cd801b400 con 0x7f7ce80c8d40 2023-12-26T23:12:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.353+0000 7f7cd67fc700 1 -- 172.21.15.131:0/1234192878 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6856+0+0 (secure 0 0 0) 0x7f7cd80840f0 con 0x7f7ce80c8d40 2023-12-26T23:12:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.357+0000 7f7cd67fc700 1 -- 172.21.15.131:0/1234192878 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cd8051230 con 0x7f7ce80c8d40 2023-12-26T23:12:10.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:10.554+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7ce80717f0 con 0x7f7ce80c8d40 2023-12-26T23:12:11.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.060+0000 7f7cd67fc700 1 -- 172.21.15.131:0/1234192878 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 106) v1 ==== 84580+0+0 (secure 0 0 0) 0x7f7cd8050ce0 con 0x7f7ce80c8d40 2023-12-26T23:12:11.060 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.060+0000 7f7cd67fc700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7cc806cbe0 0x7f7cc806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:11.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.062+0000 7f7ce5d9b700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7cc806cbe0 0x7f7cc806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:11.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.063+0000 7f7ce5d9b700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7cc806cbe0 0x7f7cc806efc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80c7f80 tx=0x7f7cdc008040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:11.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.533+0000 7f7cd67fc700 1 -- 172.21.15.131:0/1234192878 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v200) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7cd8080820 con 0x7f7ce80c8d40 2023-12-26T23:12:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7cc806cbe0 msgr2=0x7f7cc806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7cc806cbe0 0x7f7cc806efc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80c7f80 tx=0x7f7cdc008040 comp rx=0 tx=0).stop 2023-12-26T23:12:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce80c8d40 msgr2=0x7f7ce80c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce80c8d40 0x7f7ce80c7ff0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800c7c0 tx=0x7f7cd8007e00 comp rx=0 tx=0).stop 2023-12-26T23:12:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 shutdown_connections 2023-12-26T23:12:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7cc806cbe0 0x7f7cc806efc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:11.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ce8156e50 0x7f7ce806e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:11.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce80c8d40 0x7f7ce80c7ff0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:11.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 --2- 172.21.15.131:0/1234192878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ce80c9110 0x7f7ce814aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:11.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 >> 172.21.15.131:0/1234192878 conn(0x7f7ce80675b0 msgr2=0x7f7ce8069520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:11.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 shutdown_connections 2023-12-26T23:12:11.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.536+0000 7f7cecfff700 1 -- 172.21.15.131:0/1234192878 wait complete. 2023-12-26T23:12:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T23:12:11.562 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T23:12:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.760+0000 7fecdfdff700 1 Processor -- start 2023-12-26T23:12:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.760+0000 7fecdfdff700 1 -- start start 2023-12-26T23:12:11.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.761+0000 7fecdfdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fecd8156730 0x7fecd80c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:11.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.761+0000 7fecdfdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fecd806ee80 0x7fecd80c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.761+0000 7fecdfdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd80acfc0 con 0x7fecd8156360 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.761+0000 7fecdfdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd80c91f0 con 0x7fecd806ee80 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.761+0000 7fecdfdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd80c94d0 con 0x7fecd8156730 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.761+0000 7fecde39c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fecd806ee80 0x7fecd80c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.761+0000 7fecde39c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fecd806ee80 0x7fecd80c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55800/0 (socket says 172.21.15.131:55800) 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.761+0000 7fecde39c700 1 -- 172.21.15.131:0/1611746945 learned_addr learned my addr 172.21.15.131:0/1611746945 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.761+0000 7fecddb9b700 1 --1- 172.21.15.131:0/1611746945 >> v1:172.21.15.131:6789/0 conn(0x7fecd8156360 0x7fecd806e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34112/0 (socket says 172.21.15.131:34112) 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.762+0000 7fecdd39a700 1 --2- 172.21.15.131:0/1611746945 >> v2:172.21.15.190:3300/0 conn(0x7fecd8156730 0x7fecd80c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.762+0000 7fecde39c700 1 -- 172.21.15.131:0/1611746945 >> v2:172.21.15.190:3300/0 conn(0x7fecd8156730 msgr2=0x7fecd80c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:11.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.762+0000 7fecde39c700 1 --2- 172.21.15.131:0/1611746945 >> v2:172.21.15.190:3300/0 conn(0x7fecd8156730 0x7fecd80c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.762+0000 7fecde39c700 1 -- 172.21.15.131:0/1611746945 >> v1:172.21.15.131:6789/0 conn(0x7fecd8156360 legacy=0x7fecd806e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.762+0000 7fecde39c700 1 -- 172.21.15.131:0/1611746945 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd80acc50 con 0x7fecd806ee80 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.762+0000 7fecde39c700 1 --2- 172.21.15.131:0/1611746945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 0x7fecd80c89d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fecd8071110 tx=0x7fecd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb9ee94f0d5ea53 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.762+0000 7fecdcb99700 1 -- 172.21.15.131:0/1611746945 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd000e070 con 0x7fecd806ee80 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.762+0000 7fecdcb99700 1 -- 172.21.15.131:0/1611746945 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd000ba50 con 0x7fecd806ee80 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.762+0000 7fecdcb99700 1 -- 172.21.15.131:0/1611746945 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00092e0 con 0x7fecd806ee80 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.763+0000 7fecdfdff700 1 -- 172.21.15.131:0/1611746945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 msgr2=0x7fecd80c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.763+0000 7fecdfdff700 1 --2- 172.21.15.131:0/1611746945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 0x7fecd80c89d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fecd8071110 tx=0x7fecd0009840 comp rx=0 tx=0).stop 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.763+0000 7fecdfdff700 1 -- 172.21.15.131:0/1611746945 shutdown_connections 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.763+0000 7fecdfdff700 1 --2- 172.21.15.131:0/1611746945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 0x7fecd80c89d0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.763+0000 7fecdfdff700 1 --2- 172.21.15.131:0/1611746945 >> v2:172.21.15.190:3300/0 conn(0x7fecd8156730 0x7fecd80c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:11.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.763+0000 7fecdfdff700 1 -- 172.21.15.131:0/1611746945 >> 172.21.15.131:0/1611746945 conn(0x7fecd80675b0 msgr2=0x7fecd80679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:11.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.763+0000 7fecdfdff700 1 -- 172.21.15.131:0/1611746945 shutdown_connections 2023-12-26T23:12:11.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.763+0000 7fecdfdff700 1 -- 172.21.15.131:0/1611746945 wait complete. 2023-12-26T23:12:11.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.764+0000 7fecdfdff700 1 Processor -- start 2023-12-26T23:12:11.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.765+0000 7fecdfdff700 1 -- start start 2023-12-26T23:12:11.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.765+0000 7fecdfdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 0x7fecd8153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:11.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.765+0000 7fecdfdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecd8156360 0x7fecd8156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:11.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.765+0000 7fecdfdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecd8156730 0x7fecd80b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:11.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.765+0000 7fecdfdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd80c40e0 con 0x7fecd806ee80 2023-12-26T23:12:11.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.765+0000 7fecdfdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd80c31f0 con 0x7fecd8156360 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.765+0000 7fecdfdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd80c34d0 con 0x7fecd8156730 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecddb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 0x7fecd8153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecddb9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 0x7fecd8153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55808/0 (socket says 172.21.15.131:55808) 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecddb9b700 1 -- 172.21.15.131:0/3021014570 learned_addr learned my addr 172.21.15.131:0/3021014570 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecdd39a700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecd8156360 0x7fecd8156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecde39c700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecd8156730 0x7fecd80b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecddb9b700 1 -- 172.21.15.131:0/3021014570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecd8156730 msgr2=0x7fecd80b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecddb9b700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecd8156730 0x7fecd80b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecddb9b700 1 -- 172.21.15.131:0/3021014570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecd8156360 msgr2=0x7fecd8156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:11.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecddb9b700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecd8156360 0x7fecd8156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:11.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecddb9b700 1 -- 172.21.15.131:0/3021014570 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0009ae0 con 0x7fecd806ee80 2023-12-26T23:12:11.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7fecddb9b700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 0x7fecd8153450 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c7c0 tx=0x7fecd4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:11.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.766+0000 7feccdffb700 1 -- 172.21.15.131:0/3021014570 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4005a70 con 0x7fecd806ee80 2023-12-26T23:12:11.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.767+0000 7feccdffb700 1 -- 172.21.15.131:0/3021014570 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd4005bf0 con 0x7fecd806ee80 2023-12-26T23:12:11.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.767+0000 7feccdffb700 1 -- 172.21.15.131:0/3021014570 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40070f0 con 0x7fecd806ee80 2023-12-26T23:12:11.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.767+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd80c37b0 con 0x7fecd806ee80 2023-12-26T23:12:11.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.767+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd80c3d70 con 0x7fecd806ee80 2023-12-26T23:12:11.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.768+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd80717f0 con 0x7fecd806ee80 2023-12-26T23:12:11.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.769+0000 7feccdffb700 1 -- 172.21.15.131:0/3021014570 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 106) v1 ==== 84580+0+0 (secure 0 0 0) 0x7fecd4007460 con 0x7fecd806ee80 2023-12-26T23:12:11.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.769+0000 7feccdffb700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fecbc066d80 0x7fecbc069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:11.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.770+0000 7feccdffb700 1 -- 172.21.15.131:0/3021014570 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6418+0+0 (secure 0 0 0) 0x7fecd4083fb0 con 0x7fecd806ee80 2023-12-26T23:12:11.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.770+0000 7fecdd39a700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fecbc066d80 0x7fecbc069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:11.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.770+0000 7fecdd39a700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fecbc066d80 0x7fecbc069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fecd816cb60 tx=0x7fecc8008040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:11.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.774+0000 7feccdffb700 1 -- 172.21.15.131:0/3021014570 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecd40511b0 con 0x7fecd806ee80 2023-12-26T23:12:11.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:11.967+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fecd80cd080 con 0x7fecd806ee80 2023-12-26T23:12:12.538 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.538+0000 7feccdffb700 1 -- 172.21.15.131:0/3021014570 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fecd405a080 con 0x7fecd806ee80 2023-12-26T23:12:12.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.552+0000 7feccdffb700 1 -- 172.21.15.131:0/3021014570 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v201) v1 ==== 169+0+0 (secure 0 0 0) 0x7fecd40511b0 con 0x7fecd806ee80 2023-12-26T23:12:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fecbc066d80 msgr2=0x7fecbc069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fecbc066d80 0x7fecbc069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fecd816cb60 tx=0x7fecc8008040 comp rx=0 tx=0).stop 2023-12-26T23:12:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 msgr2=0x7fecd8153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 0x7fecd8153450 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c7c0 tx=0x7fecd4007f70 comp rx=0 tx=0).stop 2023-12-26T23:12:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 shutdown_connections 2023-12-26T23:12:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fecbc066d80 0x7fecbc069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:12.562 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecd8156730 0x7fecd80b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:12.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecd806ee80 0x7fecd8153450 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:12.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 --2- 172.21.15.131:0/3021014570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecd8156360 0x7fecd8156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:12.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.561+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 >> 172.21.15.131:0/3021014570 conn(0x7fecd80675b0 msgr2=0x7fecd806dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:12.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.562+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 shutdown_connections 2023-12-26T23:12:12.564 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.562+0000 7fecdfdff700 1 -- 172.21.15.131:0/3021014570 wait complete. 2023-12-26T23:12:12.566 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T23:12:12.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T23:12:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.785+0000 7f118d7ff700 1 Processor -- start 2023-12-26T23:12:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.785+0000 7f118d7ff700 1 -- start start 2023-12-26T23:12:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.786+0000 7f118d7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 0x7f11880ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.786+0000 7f118d7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f11880c8bf0 0x7f11880b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.786+0000 7f118d7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1188157a80 con 0x7f1188069d10 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.786+0000 7f118d7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11880cac50 con 0x7f11880c8bf0 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.786+0000 7f118d7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11880caf30 con 0x7f11880c8820 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f11867fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f11880c8bf0 0x7f11880b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f11867fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f11880c8bf0 0x7f11880b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55814/0 (socket says 172.21.15.131:55814) 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f11867fc700 1 -- 172.21.15.131:0/333218260 learned_addr learned my addr 172.21.15.131:0/333218260 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f11867fc700 1 -- 172.21.15.131:0/333218260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 msgr2=0x7f11880ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f1186ffd700 1 --2- 172.21.15.131:0/333218260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 0x7f11880ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f11867fc700 1 --2- 172.21.15.131:0/333218260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 0x7f11880ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f11867fc700 1 -- 172.21.15.131:0/333218260 >> v1:172.21.15.190:6789/0 conn(0x7f1188069d10 legacy=0x7f1188157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f11867fc700 1 -- 172.21.15.131:0/333218260 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11880cb210 con 0x7f11880c8bf0 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f1186ffd700 1 --2- 172.21.15.131:0/333218260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 0x7f11880ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.787+0000 7f11867fc700 1 --2- 172.21.15.131:0/333218260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 0x7f11880b12e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f117c0099b0 tx=0x7f117c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc07fe87119a3e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f1185ffb700 1 -- 172.21.15.131:0/333218260 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00b820 con 0x7f11880c8bf0 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f1185ffb700 1 -- 172.21.15.131:0/333218260 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00b980 con 0x7f11880c8bf0 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f1185ffb700 1 -- 172.21.15.131:0/333218260 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00bd20 con 0x7f11880c8bf0 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f118d7ff700 1 -- 172.21.15.131:0/333218260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 msgr2=0x7f11880b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f118d7ff700 1 --2- 172.21.15.131:0/333218260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 0x7f11880b12e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f117c0099b0 tx=0x7f117c009250 comp rx=0 tx=0).stop 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f118d7ff700 1 -- 172.21.15.131:0/333218260 shutdown_connections 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f118d7ff700 1 --2- 172.21.15.131:0/333218260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 0x7f11880ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f118d7ff700 1 --2- 172.21.15.131:0/333218260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 0x7f11880b12e0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f118d7ff700 1 -- 172.21.15.131:0/333218260 >> 172.21.15.131:0/333218260 conn(0x7f1188068040 msgr2=0x7f1188069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f118d7ff700 1 -- 172.21.15.131:0/333218260 shutdown_connections 2023-12-26T23:12:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.788+0000 7f118d7ff700 1 -- 172.21.15.131:0/333218260 wait complete. 2023-12-26T23:12:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.790+0000 7f118d7ff700 1 Processor -- start 2023-12-26T23:12:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.790+0000 7f118d7ff700 1 -- start start 2023-12-26T23:12:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f118d7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1188069d10 0x7f11880add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f118d7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 0x7f11880af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f118d7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 0x7f11880a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f118d7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880b05e0 con 0x7f11880c8bf0 2023-12-26T23:12:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f118d7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880ae250 con 0x7f1188069d10 2023-12-26T23:12:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f118d7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11880b0a70 con 0x7f11880c8820 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f11877fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 0x7f11880a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f11877fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 0x7f11880a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55824/0 (socket says 172.21.15.131:55824) 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f11877fe700 1 -- 172.21.15.131:0/423822453 learned_addr learned my addr 172.21.15.131:0/423822453 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f11867fc700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 0x7f11880af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.791+0000 7f1186ffd700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1188069d10 0x7f11880add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f11877fe700 1 -- 172.21.15.131:0/423822453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 msgr2=0x7f11880af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f11877fe700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 0x7f11880af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f11877fe700 1 -- 172.21.15.131:0/423822453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1188069d10 msgr2=0x7f11880add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f11877fe700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1188069d10 0x7f11880add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f11877fe700 1 -- 172.21.15.131:0/423822453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0094f0 con 0x7f11880c8bf0 2023-12-26T23:12:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f11877fe700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 0x7f11880a54a0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f118000bc60 tx=0x7f1180009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f116effd700 1 -- 172.21.15.131:0/423822453 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000c840 con 0x7f11880c8bf0 2023-12-26T23:12:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f116effd700 1 -- 172.21.15.131:0/423822453 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118000c9a0 con 0x7f11880c8bf0 2023-12-26T23:12:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f116effd700 1 -- 172.21.15.131:0/423822453 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000cda0 con 0x7f11880c8bf0 2023-12-26T23:12:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11880af9d0 con 0x7f11880c8bf0 2023-12-26T23:12:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.792+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11880aff90 con 0x7f11880c8bf0 2023-12-26T23:12:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.793+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11880657e0 con 0x7f11880c8bf0 2023-12-26T23:12:12.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.794+0000 7f116effd700 1 -- 172.21.15.131:0/423822453 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1180014620 con 0x7f11880c8bf0 2023-12-26T23:12:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.795+0000 7f116effd700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1174066f30 0x7f11740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.795+0000 7f116effd700 1 -- 172.21.15.131:0/423822453 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f1180084630 con 0x7f11880c8bf0 2023-12-26T23:12:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.795+0000 7f1186ffd700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1174066f30 0x7f11740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:12.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.796+0000 7f1186ffd700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1174066f30 0x7f11740693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1178006fd0 tx=0x7f1178008040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:12.799 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.799+0000 7f116effd700 1 -- 172.21.15.131:0/423822453 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11800516f0 con 0x7f11880c8bf0 2023-12-26T23:12:12.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.991+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f11880b0720 con 0x7f11880c8bf0 2023-12-26T23:12:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.996+0000 7f116effd700 1 -- 172.21.15.131:0/423822453 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1180054d10 con 0x7f11880c8bf0 2023-12-26T23:12:12.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1174066f30 msgr2=0x7f11740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:12.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1174066f30 0x7f11740693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1178006fd0 tx=0x7f1178008040 comp rx=0 tx=0).stop 2023-12-26T23:12:12.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 msgr2=0x7f11880a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:12.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 0x7f11880a54a0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f118000bc60 tx=0x7f1180009820 comp rx=0 tx=0).stop 2023-12-26T23:12:12.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 shutdown_connections 2023-12-26T23:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1174066f30 0x7f11740693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11880c8bf0 0x7f11880a54a0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1188069d10 0x7f11880add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 --2- 172.21.15.131:0/423822453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11880c8820 0x7f11880af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 >> 172.21.15.131:0/423822453 conn(0x7f1188068040 msgr2=0x7f1188159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 shutdown_connections 2023-12-26T23:12:13.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:12.998+0000 7f118d7ff700 1 -- 172.21.15.131:0/423822453 wait complete. 2023-12-26T23:12:13.017 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T23:12:13.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.209+0000 7f73487ff700 1 Processor -- start 2023-12-26T23:12:13.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.209+0000 7f73487ff700 1 -- start start 2023-12-26T23:12:13.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f73487ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400c8820 0x7f73400ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f73487ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73400c8bf0 0x7f73400b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f73487ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7340069d10 0x7f73400b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f73487ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7340157a80 con 0x7f7340069d10 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f73487ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73400cac50 con 0x7f73400c8bf0 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f73487ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73400caf30 con 0x7f73400c8820 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f7346d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7340069d10 0x7f73400b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f7346d9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7340069d10 0x7f73400b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55832/0 (socket says 172.21.15.131:55832) 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f7346d9c700 1 -- 172.21.15.131:0/433184023 learned_addr learned my addr 172.21.15.131:0/433184023 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f7345d9a700 1 --2- 172.21.15.131:0/433184023 >> v2:172.21.15.190:3300/0 conn(0x7f73400c8bf0 0x7f73400b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f734659b700 1 --2- 172.21.15.131:0/433184023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400c8820 0x7f73400ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f7346d9c700 1 -- 172.21.15.131:0/433184023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400c8820 msgr2=0x7f73400ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f7346d9c700 1 --2- 172.21.15.131:0/433184023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400c8820 0x7f73400ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f7346d9c700 1 -- 172.21.15.131:0/433184023 >> v2:172.21.15.190:3300/0 conn(0x7f73400c8bf0 msgr2=0x7f73400b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.211+0000 7f7346d9c700 1 --2- 172.21.15.131:0/433184023 >> v2:172.21.15.190:3300/0 conn(0x7f73400c8bf0 0x7f73400b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.212+0000 7f7346d9c700 1 -- 172.21.15.131:0/433184023 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73400cb210 con 0x7f7340069d10 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.212+0000 7f7345d9a700 1 --2- 172.21.15.131:0/433184023 >> v2:172.21.15.190:3300/0 conn(0x7f73400c8bf0 0x7f73400b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.212+0000 7f7346d9c700 1 --2- 172.21.15.131:0/433184023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7340069d10 0x7f73400b1820 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f7340068920 tx=0x7f7338009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c5e0b4b531fc99 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.212+0000 7f7345599700 1 -- 172.21.15.131:0/433184023 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733800e070 con 0x7f7340069d10 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.212+0000 7f7345599700 1 -- 172.21.15.131:0/433184023 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733800ba50 con 0x7f7340069d10 2023-12-26T23:12:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.212+0000 7f7345599700 1 -- 172.21.15.131:0/433184023 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380092e0 con 0x7f7340069d10 2023-12-26T23:12:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.213+0000 7f73487ff700 1 -- 172.21.15.131:0/433184023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7340069d10 msgr2=0x7f73400b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.213+0000 7f73487ff700 1 --2- 172.21.15.131:0/433184023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7340069d10 0x7f73400b1820 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f7340068920 tx=0x7f7338009840 comp rx=0 tx=0).stop 2023-12-26T23:12:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.213+0000 7f73487ff700 1 -- 172.21.15.131:0/433184023 shutdown_connections 2023-12-26T23:12:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.213+0000 7f73487ff700 1 --2- 172.21.15.131:0/433184023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7340069d10 0x7f73400b1820 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.213+0000 7f73487ff700 1 --2- 172.21.15.131:0/433184023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400c8820 0x7f73400ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.213+0000 7f73487ff700 1 --2- 172.21.15.131:0/433184023 >> v2:172.21.15.190:3300/0 conn(0x7f73400c8bf0 0x7f73400b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.213+0000 7f73487ff700 1 -- 172.21.15.131:0/433184023 >> 172.21.15.131:0/433184023 conn(0x7f7340068040 msgr2=0x7f7340069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.213+0000 7f73487ff700 1 -- 172.21.15.131:0/433184023 shutdown_connections 2023-12-26T23:12:13.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.213+0000 7f73487ff700 1 -- 172.21.15.131:0/433184023 wait complete. 2023-12-26T23:12:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.214+0000 7f73487ff700 1 Processor -- start 2023-12-26T23:12:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.214+0000 7f73487ff700 1 -- start start 2023-12-26T23:12:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f73487ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7340069d10 0x7f734015b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f73487ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73400c8820 0x7f73400af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f73487ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73400c8bf0 0x7f73400acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f73487ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73400b03d0 con 0x7f73400c8820 2023-12-26T23:12:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f73487ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73400af7e0 con 0x7f73400c8bf0 2023-12-26T23:12:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f73487ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73400afaa0 con 0x7f7340069d10 2023-12-26T23:12:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f7345d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73400c8820 0x7f73400af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f7345d9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73400c8820 0x7f73400af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55840/0 (socket says 172.21.15.131:55840) 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f7345d9a700 1 -- 172.21.15.131:0/1101767271 learned_addr learned my addr 172.21.15.131:0/1101767271 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.215+0000 7f734659b700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7340069d10 0x7f734015b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f7346d9c700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73400c8bf0 0x7f73400acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f7345d9a700 1 -- 172.21.15.131:0/1101767271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7340069d10 msgr2=0x7f734015b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f7345d9a700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7340069d10 0x7f734015b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f7345d9a700 1 -- 172.21.15.131:0/1101767271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73400c8bf0 msgr2=0x7f73400acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f7345d9a700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73400c8bf0 0x7f73400acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f7345d9a700 1 -- 172.21.15.131:0/1101767271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7338009ae0 con 0x7f73400c8820 2023-12-26T23:12:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f7345d9a700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73400c8820 0x7f73400af0d0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f733000e8d0 tx=0x7f7330008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f73367fc700 1 -- 172.21.15.131:0/1101767271 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330013410 con 0x7f73400c8820 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f73367fc700 1 -- 172.21.15.131:0/1101767271 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7330013570 con 0x7f73400c8820 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73400afd80 con 0x7f73400c8820 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.216+0000 7f73367fc700 1 -- 172.21.15.131:0/1101767271 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330013910 con 0x7f73400c8820 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.217+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73400ad150 con 0x7f73400c8820 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.218+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73400b0540 con 0x7f73400c8820 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.218+0000 7f73367fc700 1 -- 172.21.15.131:0/1101767271 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7330013d00 con 0x7f73400c8820 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.219+0000 7f73367fc700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7324066f30 0x7f73240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.219+0000 7f73367fc700 1 -- 172.21.15.131:0/1101767271 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f7330084430 con 0x7f73400c8820 2023-12-26T23:12:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.219+0000 7f734659b700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7324066f30 0x7f73240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.220+0000 7f734659b700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7324066f30 0x7f73240693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f733c006fd0 tx=0x7f733c008040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.224+0000 7f73367fc700 1 -- 172.21.15.131:0/1101767271 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73300514f0 con 0x7f73400c8820 2023-12-26T23:12:13.415 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.415+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f73400657e0 con 0x7f73400c8820 2023-12-26T23:12:13.421 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.421+0000 7f73367fc700 1 -- 172.21.15.131:0/1101767271 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v68) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7330054b10 con 0x7f73400c8820 2023-12-26T23:12:13.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.423+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7324066f30 msgr2=0x7f73240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.424 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.423+0000 7f73487ff700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7324066f30 0x7f73240693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f733c006fd0 tx=0x7f733c008040 comp rx=0 tx=0).stop 2023-12-26T23:12:13.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.423+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73400c8820 msgr2=0x7f73400af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.424+0000 7f73487ff700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73400c8820 0x7f73400af0d0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f733000e8d0 tx=0x7f7330008040 comp rx=0 tx=0).stop 2023-12-26T23:12:13.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.424+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 shutdown_connections 2023-12-26T23:12:13.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.424+0000 7f73487ff700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7324066f30 0x7f73240693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.424+0000 7f73487ff700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73400c8bf0 0x7f73400acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.424+0000 7f73487ff700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7340069d10 0x7f734015b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.424+0000 7f73487ff700 1 --2- 172.21.15.131:0/1101767271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73400c8820 0x7f73400af0d0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.425 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.424+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 >> 172.21.15.131:0/1101767271 conn(0x7f7340068040 msgr2=0x7f73400c1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:13.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.424+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 shutdown_connections 2023-12-26T23:12:13.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.424+0000 7f73487ff700 1 -- 172.21.15.131:0/1101767271 wait complete. 2023-12-26T23:12:13.442 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T23:12:13.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.635+0000 7f9cd53ff700 1 Processor -- start 2023-12-26T23:12:13.636 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.635+0000 7f9cd53ff700 1 -- start start 2023-12-26T23:12:13.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.636+0000 7f9cd53ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9cd00cb320 0x7f9cd0069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cd53ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9cd006a1e0 0x7f9cd00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cd53ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00b0c90 0x7f9cd00aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cd53ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd00c8cc0 con 0x7f9cd00cb320 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cd53ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd00ad700 con 0x7f9cd006a1e0 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cd53ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd00b1110 con 0x7f9cd00b0c90 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cceffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9cd00cb320 0x7f9cd0069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cceffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9cd00cb320 0x7f9cd0069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55848/0 (socket says 172.21.15.131:55848) 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cceffd700 1 -- 172.21.15.131:0/3421918849 learned_addr learned my addr 172.21.15.131:0/3421918849 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9ccf7fe700 1 --2- 172.21.15.131:0/3421918849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00b0c90 0x7f9cd00aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cce7fc700 1 --2- 172.21.15.131:0/3421918849 >> v2:172.21.15.190:3300/0 conn(0x7f9cd006a1e0 0x7f9cd00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cceffd700 1 -- 172.21.15.131:0/3421918849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00b0c90 msgr2=0x7f9cd00aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:13.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cceffd700 1 --2- 172.21.15.131:0/3421918849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00b0c90 0x7f9cd00aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cceffd700 1 -- 172.21.15.131:0/3421918849 >> v2:172.21.15.190:3300/0 conn(0x7f9cd006a1e0 msgr2=0x7f9cd00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cceffd700 1 --2- 172.21.15.131:0/3421918849 >> v2:172.21.15.190:3300/0 conn(0x7f9cd006a1e0 0x7f9cd00b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cceffd700 1 -- 172.21.15.131:0/3421918849 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd0156940 con 0x7f9cd00cb320 2023-12-26T23:12:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.637+0000 7f9cce7fc700 1 --2- 172.21.15.131:0/3421918849 >> v2:172.21.15.190:3300/0 conn(0x7f9cd006a1e0 0x7f9cd00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9cceffd700 1 --2- 172.21.15.131:0/3421918849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 0x7f9cd0069ca0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0064700 tx=0x7f9cc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52bb54f9c6dd4b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9ccdffb700 1 -- 172.21.15.131:0/3421918849 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000b820 con 0x7f9cd00cb320 2023-12-26T23:12:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9ccdffb700 1 -- 172.21.15.131:0/3421918849 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc000b980 con 0x7f9cd00cb320 2023-12-26T23:12:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9ccdffb700 1 -- 172.21.15.131:0/3421918849 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000bd20 con 0x7f9cd00cb320 2023-12-26T23:12:13.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9cd53ff700 1 -- 172.21.15.131:0/3421918849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 msgr2=0x7f9cd0069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/3421918849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 0x7f9cd0069ca0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0064700 tx=0x7f9cc0009250 comp rx=0 tx=0).stop 2023-12-26T23:12:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9cd53ff700 1 -- 172.21.15.131:0/3421918849 shutdown_connections 2023-12-26T23:12:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/3421918849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 0x7f9cd0069ca0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/3421918849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd00b0c90 0x7f9cd00aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.638+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/3421918849 >> v2:172.21.15.190:3300/0 conn(0x7f9cd006a1e0 0x7f9cd00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.639+0000 7f9cd53ff700 1 -- 172.21.15.131:0/3421918849 >> 172.21.15.131:0/3421918849 conn(0x7f9cd0068040 msgr2=0x7f9cd0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.639+0000 7f9cd53ff700 1 -- 172.21.15.131:0/3421918849 shutdown_connections 2023-12-26T23:12:13.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.639+0000 7f9cd53ff700 1 -- 172.21.15.131:0/3421918849 wait complete. 2023-12-26T23:12:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.640+0000 7f9cd53ff700 1 Processor -- start 2023-12-26T23:12:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.640+0000 7f9cd53ff700 1 -- start start 2023-12-26T23:12:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.641+0000 7f9cd53ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd006a1e0 0x7f9cd0142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.641+0000 7f9cd53ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00b0c90 0x7f9cd0148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.641+0000 7f9cd53ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 0x7f9cd0148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.641+0000 7f9cd53ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0147e80 con 0x7f9cd00cb320 2023-12-26T23:12:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.641+0000 7f9cd53ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0163230 con 0x7f9cd00b0c90 2023-12-26T23:12:13.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.641+0000 7f9cd53ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01478c0 con 0x7f9cd006a1e0 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.641+0000 7f9ccf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 0x7f9cd0148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.641+0000 7f9ccf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 0x7f9cd0148a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55856/0 (socket says 172.21.15.131:55856) 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.641+0000 7f9ccf7fe700 1 -- 172.21.15.131:0/1959990317 learned_addr learned my addr 172.21.15.131:0/1959990317 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9cceffd700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd006a1e0 0x7f9cd0142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9cce7fc700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00b0c90 0x7f9cd0148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9ccf7fe700 1 -- 172.21.15.131:0/1959990317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd006a1e0 msgr2=0x7f9cd0142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9ccf7fe700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd006a1e0 0x7f9cd0142e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9ccf7fe700 1 -- 172.21.15.131:0/1959990317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00b0c90 msgr2=0x7f9cd0148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9ccf7fe700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00b0c90 0x7f9cd0148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9ccf7fe700 1 -- 172.21.15.131:0/1959990317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc00094f0 con 0x7f9cd00cb320 2023-12-26T23:12:13.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9cce7fc700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00b0c90 0x7f9cd0148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9ccf7fe700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 0x7f9cd0148a60 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009fa0 tx=0x7f9cc8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9cb6ffd700 1 -- 172.21.15.131:0/1959990317 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc800ed20 con 0x7f9cd00cb320 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9cb6ffd700 1 -- 172.21.15.131:0/1959990317 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc800ee80 con 0x7f9cd00cb320 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd0147ba0 con 0x7f9cd00cb320 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9cb6ffd700 1 -- 172.21.15.131:0/1959990317 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8014670 con 0x7f9cd00cb320 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.642+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd0163720 con 0x7f9cd00cb320 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.643+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd006b5b0 con 0x7f9cd00cb320 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.644+0000 7f9cb6ffd700 1 -- 172.21.15.131:0/1959990317 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9cc80149e0 con 0x7f9cd00cb320 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.645+0000 7f9cb6ffd700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9cbc066f30 0x7f9cbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.646+0000 7f9cb6ffd700 1 -- 172.21.15.131:0/1959990317 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f9cc8084dc0 con 0x7f9cd00cb320 2023-12-26T23:12:13.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.646+0000 7f9cceffd700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9cbc066f30 0x7f9cbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:13.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.646+0000 7f9cceffd700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9cbc066f30 0x7f9cbc0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0064700 tx=0x7f9cc0017000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:13.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.650+0000 7f9cb6ffd700 1 -- 172.21.15.131:0/1959990317 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cc8051f20 con 0x7f9cd00cb320 2023-12-26T23:12:13.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.842+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f9cd0163940 con 0x7f9cd00cb320 2023-12-26T23:12:13.847 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.846+0000 7f9cb6ffd700 1 -- 172.21.15.131:0/1959990317 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v69) v1 ==== 120+0+0 (secure 0 0 0) 0x7f9cc8055540 con 0x7f9cd00cb320 2023-12-26T23:12:13.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9cbc066f30 msgr2=0x7f9cbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9cbc066f30 0x7f9cbc0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0064700 tx=0x7f9cc0017000 comp rx=0 tx=0).stop 2023-12-26T23:12:13.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 msgr2=0x7f9cd0148a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:13.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 0x7f9cd0148a60 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009fa0 tx=0x7f9cc8009900 comp rx=0 tx=0).stop 2023-12-26T23:12:13.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 shutdown_connections 2023-12-26T23:12:13.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9cbc066f30 0x7f9cbc0693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd00cb320 0x7f9cd0148a60 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cd006a1e0 0x7f9cd0142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 --2- 172.21.15.131:0/1959990317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cd00b0c90 0x7f9cd0148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:13.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 >> 172.21.15.131:0/1959990317 conn(0x7f9cd0068040 msgr2=0x7f9cd0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:13.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 shutdown_connections 2023-12-26T23:12:13.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:13.849+0000 7f9cd53ff700 1 -- 172.21.15.131:0/1959990317 wait complete. 2023-12-26T23:12:13.868 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-12-26T23:12:14.058 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.058+0000 7fa9387ff700 1 Processor -- start 2023-12-26T23:12:14.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.058+0000 7fa9387ff700 1 -- start start 2023-12-26T23:12:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.059+0000 7fa9387ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa9300b5310 0x7fa9300b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.059+0000 7fa9387ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9300afcd0 con 0x7fa930159a70 2023-12-26T23:12:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.059+0000 7fa9387ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9300b3e70 con 0x7fa9301596a0 2023-12-26T23:12:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.059+0000 7fa9387ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9300b0f20 con 0x7fa9300b5310 2023-12-26T23:12:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.060+0000 7fa935d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa930159a70 0x7fa9300b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34122/0 (socket says 172.21.15.131:34122) 2023-12-26T23:12:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.060+0000 7fa935d9a700 1 -- 172.21.15.131:0/2998518078 learned_addr learned my addr 172.21.15.131:0/2998518078 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.060+0000 7fa936d9c700 1 --2- 172.21.15.131:0/2998518078 >> v2:172.21.15.131:3300/0 conn(0x7fa9300b5310 0x7fa9300b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.060+0000 7fa935599700 1 -- 172.21.15.131:0/2998518078 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939714778 0 0) 0x7fa9300afcd0 con 0x7fa930159a70 2023-12-26T23:12:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.060+0000 7fa935599700 1 -- 172.21.15.131:0/2998518078 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa91c003570 con 0x7fa930159a70 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.060+0000 7fa936d9c700 1 -- 172.21.15.131:0/2998518078 >> v1:172.21.15.190:6789/0 conn(0x7fa9301596a0 legacy=0x7fa9300b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.060+0000 7fa936d9c700 1 -- 172.21.15.131:0/2998518078 >> v1:172.21.15.131:6789/0 conn(0x7fa930159a70 legacy=0x7fa9300b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.060+0000 7fa936d9c700 1 -- 172.21.15.131:0/2998518078 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300b3540 con 0x7fa9300b5310 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.060+0000 7fa936d9c700 1 --2- 172.21.15.131:0/2998518078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 0x7fa9300b0870 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa928009e60 tx=0x7fa928009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c363b73281952502 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa935599700 1 -- 172.21.15.131:0/2998518078 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92800e070 con 0x7fa9300b5310 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa935599700 1 -- 172.21.15.131:0/2998518078 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92800bb60 con 0x7fa9300b5310 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa935599700 1 -- 172.21.15.131:0/2998518078 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928012430 con 0x7fa9300b5310 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa9387ff700 1 -- 172.21.15.131:0/2998518078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 msgr2=0x7fa9300b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa9387ff700 1 --2- 172.21.15.131:0/2998518078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 0x7fa9300b0870 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa928009e60 tx=0x7fa928009780 comp rx=0 tx=0).stop 2023-12-26T23:12:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa9387ff700 1 -- 172.21.15.131:0/2998518078 shutdown_connections 2023-12-26T23:12:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa9387ff700 1 --2- 172.21.15.131:0/2998518078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 0x7fa9300b0870 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa9387ff700 1 -- 172.21.15.131:0/2998518078 >> 172.21.15.131:0/2998518078 conn(0x7fa93006ece0 msgr2=0x7fa93006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa9387ff700 1 -- 172.21.15.131:0/2998518078 shutdown_connections 2023-12-26T23:12:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.061+0000 7fa9387ff700 1 -- 172.21.15.131:0/2998518078 wait complete. 2023-12-26T23:12:14.066 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.063+0000 7fa9387ff700 1 Processor -- start 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.063+0000 7fa9387ff700 1 -- start start 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa9387ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 0x7fa930169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa9387ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9301596a0 0x7fa93014db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa9387ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa930159a70 0x7fa9300c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa9387ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300c4f00 con 0x7fa9300b5310 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa9387ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93014e080 con 0x7fa9301596a0 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa9387ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300c4010 con 0x7fa930159a70 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa93659b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 0x7fa930169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa93659b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 0x7fa930169dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55868/0 (socket says 172.21.15.131:55868) 2023-12-26T23:12:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa93659b700 1 -- 172.21.15.131:0/789324604 learned_addr learned my addr 172.21.15.131:0/789324604 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:14.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa935d9a700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9301596a0 0x7fa93014db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa936d9c700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa930159a70 0x7fa9300c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa93659b700 1 -- 172.21.15.131:0/789324604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa930159a70 msgr2=0x7fa9300c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa93659b700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa930159a70 0x7fa9300c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa93659b700 1 -- 172.21.15.131:0/789324604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9301596a0 msgr2=0x7fa93014db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa93659b700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9301596a0 0x7fa93014db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.064+0000 7fa93659b700 1 -- 172.21.15.131:0/789324604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928009a20 con 0x7fa9300b5310 2023-12-26T23:12:14.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.065+0000 7fa93659b700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 0x7fa930169dd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c5c0 tx=0x7fa92c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:14.068 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.065+0000 7fa9267fc700 1 -- 172.21.15.131:0/789324604 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c011590 con 0x7fa9300b5310 2023-12-26T23:12:14.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.065+0000 7fa9267fc700 1 -- 172.21.15.131:0/789324604 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c011710 con 0x7fa9300b5310 2023-12-26T23:12:14.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.065+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9300c42f0 con 0x7fa9300b5310 2023-12-26T23:12:14.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.065+0000 7fa9267fc700 1 -- 172.21.15.131:0/789324604 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c011ad0 con 0x7fa9300b5310 2023-12-26T23:12:14.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.065+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9300c48b0 con 0x7fa9300b5310 2023-12-26T23:12:14.069 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.066+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9300657e0 con 0x7fa9300b5310 2023-12-26T23:12:14.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.067+0000 7fa9267fc700 1 -- 172.21.15.131:0/789324604 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa92c012420 con 0x7fa9300b5310 2023-12-26T23:12:14.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.068+0000 7fa9267fc700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa914066f30 0x7fa9140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.068+0000 7fa9267fc700 1 -- 172.21.15.131:0/789324604 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5975+0+0 (secure 0 0 0) 0x7fa92c083a80 con 0x7fa9300b5310 2023-12-26T23:12:14.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.068+0000 7fa935d9a700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa914066f30 0x7fa9140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.070 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.069+0000 7fa935d9a700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa914066f30 0x7fa9140693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa9300af840 tx=0x7fa920009040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:14.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.072+0000 7fa9267fc700 1 -- 172.21.15.131:0/789324604 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa92c04b5a0 con 0x7fa9300b5310 2023-12-26T23:12:14.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.342+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7fa9300b3820 con 0x7fa9300b5310 2023-12-26T23:12:14.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.549+0000 7fa9267fc700 1 -- 172.21.15.131:0/789324604 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3076) v1 ==== 163+0+0 (secure 0 0 0) 0x7fa92c017070 con 0x7fa9300b5310 2023-12-26T23:12:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa914066f30 msgr2=0x7fa9140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa914066f30 0x7fa9140693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa9300af840 tx=0x7fa920009040 comp rx=0 tx=0).stop 2023-12-26T23:12:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 msgr2=0x7fa930169dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 0x7fa930169dd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c5c0 tx=0x7fa92c00c980 comp rx=0 tx=0).stop 2023-12-26T23:12:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 shutdown_connections 2023-12-26T23:12:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa914066f30 0x7fa9140693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa930159a70 0x7fa9300c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9300b5310 0x7fa930169dd0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 --2- 172.21.15.131:0/789324604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9301596a0 0x7fa93014db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.551+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 >> 172.21.15.131:0/789324604 conn(0x7fa93006ece0 msgr2=0x7fa9300b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:14.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.552+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 shutdown_connections 2023-12-26T23:12:14.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.552+0000 7fa9387ff700 1 -- 172.21.15.131:0/789324604 wait complete. 2023-12-26T23:12:14.570 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-12-26T23:12:14.570 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-12-26T23:12:14.570 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-12-26T23:12:14.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-12-26T23:12:14.760 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.760+0000 7f28c3dff700 1 Processor -- start 2023-12-26T23:12:14.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.760+0000 7f28c3dff700 1 -- start start 2023-12-26T23:12:14.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c3dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f28bc0cb320 0x7f28bc069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c3dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f28bc0b0e60 0x7f28bc0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c3dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28bc156940 con 0x7f28bc06a1e0 2023-12-26T23:12:14.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c3dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc0c8cc0 con 0x7f28bc0b0e60 2023-12-26T23:12:14.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c3dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc0ad700 con 0x7f28bc0cb320 2023-12-26T23:12:14.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c239c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f28bc0b0e60 0x7f28bc0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c139a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f28bc06a1e0 0x7f28bc0b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34124/0 (socket says 172.21.15.131:34124) 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c139a700 1 -- 172.21.15.131:0/2789505987 learned_addr learned my addr 172.21.15.131:0/2789505987 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c1b9b700 1 --2- 172.21.15.131:0/2789505987 >> v2:172.21.15.190:3300/0 conn(0x7f28bc0cb320 0x7f28bc069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c239c700 1 -- 172.21.15.131:0/2789505987 >> v2:172.21.15.190:3300/0 conn(0x7f28bc0cb320 msgr2=0x7f28bc069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c239c700 1 --2- 172.21.15.131:0/2789505987 >> v2:172.21.15.190:3300/0 conn(0x7f28bc0cb320 0x7f28bc069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c239c700 1 -- 172.21.15.131:0/2789505987 >> v1:172.21.15.131:6789/0 conn(0x7f28bc06a1e0 legacy=0x7f28bc0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c239c700 1 -- 172.21.15.131:0/2789505987 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc0ae500 con 0x7f28bc0b0e60 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.763+0000 7f28c1b9b700 1 --2- 172.21.15.131:0/2789505987 >> v2:172.21.15.190:3300/0 conn(0x7f28bc0cb320 0x7f28bc069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.762+0000 7f28c0b99700 1 -- 172.21.15.131:0/2789505987 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687999329 0 0) 0x7f28bc156940 con 0x7f28bc06a1e0 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.763+0000 7f28c239c700 1 --2- 172.21.15.131:0/2789505987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 0x7f28bc0c9400 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f28bc066d80 tx=0x7f28b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6c206557d6a0c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.763+0000 7f28c0b99700 1 -- 172.21.15.131:0/2789505987 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b400e070 con 0x7f28bc0b0e60 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.763+0000 7f28c0b99700 1 -- 172.21.15.131:0/2789505987 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b400bc30 con 0x7f28bc0b0e60 2023-12-26T23:12:14.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.763+0000 7f28c0b99700 1 -- 172.21.15.131:0/2789505987 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4012430 con 0x7f28bc0b0e60 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.763+0000 7f28c3dff700 1 -- 172.21.15.131:0/2789505987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 msgr2=0x7f28bc0c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.763+0000 7f28c3dff700 1 --2- 172.21.15.131:0/2789505987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 0x7f28bc0c9400 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f28bc066d80 tx=0x7f28b4009780 comp rx=0 tx=0).stop 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.764+0000 7f28c3dff700 1 -- 172.21.15.131:0/2789505987 shutdown_connections 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.764+0000 7f28c3dff700 1 --2- 172.21.15.131:0/2789505987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 0x7f28bc0c9400 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.764+0000 7f28c3dff700 1 --2- 172.21.15.131:0/2789505987 >> v2:172.21.15.190:3300/0 conn(0x7f28bc0cb320 0x7f28bc069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.764+0000 7f28c3dff700 1 -- 172.21.15.131:0/2789505987 >> 172.21.15.131:0/2789505987 conn(0x7f28bc068040 msgr2=0x7f28bc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.764+0000 7f28c3dff700 1 -- 172.21.15.131:0/2789505987 shutdown_connections 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.764+0000 7f28c3dff700 1 -- 172.21.15.131:0/2789505987 wait complete. 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.765+0000 7f28c3dff700 1 Processor -- start 2023-12-26T23:12:14.766 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.765+0000 7f28c3dff700 1 -- start start 2023-12-26T23:12:14.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c3dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28bc06a1e0 0x7f28bc1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c3dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 0x7f28bc0abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c3dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28bc0cb320 0x7f28bc0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c3dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0c2320 con 0x7f28bc0b0e60 2023-12-26T23:12:14.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c3dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0c1430 con 0x7f28bc06a1e0 2023-12-26T23:12:14.768 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c3dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0c1710 con 0x7f28bc0cb320 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c139a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 0x7f28bc0abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c139a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 0x7f28bc0abc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55876/0 (socket says 172.21.15.131:55876) 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c139a700 1 -- 172.21.15.131:0/1729353990 learned_addr learned my addr 172.21.15.131:0/1729353990 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.766+0000 7f28c239c700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28bc0cb320 0x7f28bc0ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c1b9b700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28bc06a1e0 0x7f28bc1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c139a700 1 -- 172.21.15.131:0/1729353990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28bc0cb320 msgr2=0x7f28bc0ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c139a700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28bc0cb320 0x7f28bc0ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c139a700 1 -- 172.21.15.131:0/1729353990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28bc06a1e0 msgr2=0x7f28bc1598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c139a700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28bc06a1e0 0x7f28bc1598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c139a700 1 -- 172.21.15.131:0/1729353990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b4009a20 con 0x7f28bc0b0e60 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c1b9b700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28bc06a1e0 0x7f28bc1598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c139a700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 0x7f28bc0abc00 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00c810 tx=0x7f28ac007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:14.769 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28b1ffb700 1 -- 172.21.15.131:0/1729353990 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac005c00 con 0x7f28bc0b0e60 2023-12-26T23:12:14.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28b1ffb700 1 -- 172.21.15.131:0/1729353990 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac005d60 con 0x7f28bc0b0e60 2023-12-26T23:12:14.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28b1ffb700 1 -- 172.21.15.131:0/1729353990 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0072e0 con 0x7f28bc0b0e60 2023-12-26T23:12:14.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc0c19f0 con 0x7f28bc0b0e60 2023-12-26T23:12:14.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.767+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc0c1fb0 con 0x7f28bc0b0e60 2023-12-26T23:12:14.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.769+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc0c2490 con 0x7f28bc0b0e60 2023-12-26T23:12:14.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.769+0000 7f28b1ffb700 1 -- 172.21.15.131:0/1729353990 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f28ac007690 con 0x7f28bc0b0e60 2023-12-26T23:12:14.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.770+0000 7f28b1ffb700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f28a0066f30 0x7f28a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:14.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.770+0000 7f28b1ffb700 1 -- 172.21.15.131:0/1729353990 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f28ac083a60 con 0x7f28bc0b0e60 2023-12-26T23:12:14.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.771+0000 7f28c1b9b700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f28a0066f30 0x7f28a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:14.771 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.771+0000 7f28c1b9b700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f28a0066f30 0x7f28a00693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006f00 tx=0x7f28b8008040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:14.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:14.774+0000 7f28b1ffb700 1 -- 172.21.15.131:0/1729353990 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28ac04b5a0 con 0x7f28bc0b0e60 2023-12-26T23:12:15.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.045+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f28bc0657e0 con 0x7f28bc0b0e60 2023-12-26T23:12:15.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.550+0000 7f28b1ffb700 1 -- 172.21.15.131:0/1729353990 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3077) v1 ==== 158+0+0 (secure 0 0 0) 0x7f28ac017020 con 0x7f28bc0b0e60 2023-12-26T23:12:15.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f28a0066f30 msgr2=0x7f28a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:15.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f28a0066f30 0x7f28a00693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006f00 tx=0x7f28b8008040 comp rx=0 tx=0).stop 2023-12-26T23:12:15.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 msgr2=0x7f28bc0abc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:15.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 0x7f28bc0abc00 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00c810 tx=0x7f28ac007db0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 shutdown_connections 2023-12-26T23:12:15.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f28a0066f30 0x7f28a00693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28bc0cb320 0x7f28bc0ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28bc06a1e0 0x7f28bc1598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 --2- 172.21.15.131:0/1729353990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28bc0b0e60 0x7f28bc0abc00 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 >> 172.21.15.131:0/1729353990 conn(0x7f28bc068040 msgr2=0x7f28bc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:15.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 shutdown_connections 2023-12-26T23:12:15.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.553+0000 7f28c3dff700 1 -- 172.21.15.131:0/1729353990 wait complete. 2023-12-26T23:12:15.573 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-26T23:12:15.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.767+0000 7f918fbff700 1 Processor -- start 2023-12-26T23:12:15.767 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.767+0000 7f918fbff700 1 -- start start 2023-12-26T23:12:15.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.769+0000 7f918fbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f91880cc810 0x7f91880c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:15.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.769+0000 7f918fbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91880686c0 con 0x7f918806bfd0 2023-12-26T23:12:15.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.769+0000 7f918fbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9188069520 con 0x7f918813c510 2023-12-26T23:12:15.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.769+0000 7f918fbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9188068910 con 0x7f91880cc810 2023-12-26T23:12:15.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.769+0000 7f918d19a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f918806bfd0 0x7f91880cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52452/0 (socket says 172.21.15.131:52452) 2023-12-26T23:12:15.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.769+0000 7f918d19a700 1 -- 172.21.15.131:0/4202405959 learned_addr learned my addr 172.21.15.131:0/4202405959 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:15.772 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.769+0000 7f918e19c700 1 --2- 172.21.15.131:0/4202405959 >> v2:172.21.15.190:3300/0 conn(0x7f91880cc810 0x7f91880c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.769+0000 7f918c999700 1 -- 172.21.15.131:0/4202405959 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376879115 0 0) 0x7f91880686c0 con 0x7f918806bfd0 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.769+0000 7f918c999700 1 -- 172.21.15.131:0/4202405959 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9174003570 con 0x7f918806bfd0 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.770+0000 7f918c999700 1 -- 172.21.15.131:0/4202405959 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390561070 0 0) 0x7f9188069520 con 0x7f918813c510 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.770+0000 7f918c999700 1 -- 172.21.15.131:0/4202405959 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91880686c0 con 0x7f918813c510 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.770+0000 7f918e19c700 1 -- 172.21.15.131:0/4202405959 >> v1:172.21.15.190:6789/0 conn(0x7f918813c510 legacy=0x7f918806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.770+0000 7f918e19c700 1 -- 172.21.15.131:0/4202405959 >> v1:172.21.15.131:6789/0 conn(0x7f918806bfd0 legacy=0x7f91880cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.770+0000 7f918e19c700 1 -- 172.21.15.131:0/4202405959 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9188068bf0 con 0x7f91880cc810 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.770+0000 7f918e19c700 1 --2- 172.21.15.131:0/4202405959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91880cc810 0x7f91880c8a20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9180009e60 tx=0x7f9180009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d2e48347e8bf34 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.770+0000 7f918c999700 1 -- 172.21.15.131:0/4202405959 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918000e070 con 0x7f91880cc810 2023-12-26T23:12:15.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.770+0000 7f918c999700 1 -- 172.21.15.131:0/4202405959 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918000bab0 con 0x7f91880cc810 2023-12-26T23:12:15.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.770+0000 7f918c999700 1 -- 172.21.15.131:0/4202405959 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918000be70 con 0x7f91880cc810 2023-12-26T23:12:15.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.771+0000 7f918fbff700 1 -- 172.21.15.131:0/4202405959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91880cc810 msgr2=0x7f91880c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:15.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.771+0000 7f918fbff700 1 --2- 172.21.15.131:0/4202405959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91880cc810 0x7f91880c8a20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9180009e60 tx=0x7f9180009780 comp rx=0 tx=0).stop 2023-12-26T23:12:15.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.771+0000 7f918fbff700 1 -- 172.21.15.131:0/4202405959 shutdown_connections 2023-12-26T23:12:15.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.771+0000 7f918fbff700 1 --2- 172.21.15.131:0/4202405959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91880cc810 0x7f91880c8a20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.771+0000 7f918fbff700 1 -- 172.21.15.131:0/4202405959 >> 172.21.15.131:0/4202405959 conn(0x7f91880ce3e0 msgr2=0x7f91880ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:15.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.771+0000 7f918fbff700 1 -- 172.21.15.131:0/4202405959 shutdown_connections 2023-12-26T23:12:15.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.771+0000 7f918fbff700 1 -- 172.21.15.131:0/4202405959 wait complete. 2023-12-26T23:12:15.777 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.772+0000 7f918fbff700 1 Processor -- start 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.772+0000 7f918fbff700 1 -- start start 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.773+0000 7f918fbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918806bfd0 0x7f9188153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.773+0000 7f918fbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91880cc810 0x7f91880c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.773+0000 7f918fbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918813c510 0x7f91880c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.773+0000 7f918fbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880b2220 con 0x7f918813c510 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.773+0000 7f918fbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9188154050 con 0x7f918806bfd0 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.773+0000 7f918fbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880b1180 con 0x7f91880cc810 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918e19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918813c510 0x7f91880c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918e19c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918813c510 0x7f91880c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60034/0 (socket says 172.21.15.131:60034) 2023-12-26T23:12:15.778 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918e19c700 1 -- 172.21.15.131:0/3435286680 learned_addr learned my addr 172.21.15.131:0/3435286680 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918d99b700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918806bfd0 0x7f9188153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918d19a700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91880cc810 0x7f91880c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918e19c700 1 -- 172.21.15.131:0/3435286680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91880cc810 msgr2=0x7f91880c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918e19c700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91880cc810 0x7f91880c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918e19c700 1 -- 172.21.15.131:0/3435286680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918806bfd0 msgr2=0x7f9188153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918e19c700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918806bfd0 0x7f9188153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918e19c700 1 -- 172.21.15.131:0/3435286680 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9180009a20 con 0x7f918813c510 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.774+0000 7f918e19c700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918813c510 0x7f91880c9760 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f918806fe40 tx=0x7f9180019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.775+0000 7f917dffb700 1 -- 172.21.15.131:0/3435286680 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180012db0 con 0x7f918813c510 2023-12-26T23:12:15.779 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.775+0000 7f917dffb700 1 -- 172.21.15.131:0/3435286680 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180026480 con 0x7f918813c510 2023-12-26T23:12:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.775+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91880b1460 con 0x7f918813c510 2023-12-26T23:12:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.775+0000 7f917dffb700 1 -- 172.21.15.131:0/3435286680 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180026840 con 0x7f918813c510 2023-12-26T23:12:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.775+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91880b1a20 con 0x7f918813c510 2023-12-26T23:12:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.776+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91880657e0 con 0x7f918813c510 2023-12-26T23:12:15.780 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.777+0000 7f917dffb700 1 -- 172.21.15.131:0/3435286680 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f918001c400 con 0x7f918813c510 2023-12-26T23:12:15.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.778+0000 7f917dffb700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f916c066f30 0x7f916c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:15.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.778+0000 7f917dffb700 1 -- 172.21.15.131:0/3435286680 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f9180086580 con 0x7f918813c510 2023-12-26T23:12:15.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.778+0000 7f918d99b700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f916c066f30 0x7f916c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:15.781 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.779+0000 7f918d99b700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f916c066f30 0x7f916c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9184003ae0 tx=0x7f9184007d40 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:15.783 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.783+0000 7f917dffb700 1 -- 172.21.15.131:0/3435286680 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9180053640 con 0x7f918813c510 2023-12-26T23:12:15.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:12:15.809 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:12:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.981+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9188068ed0 con 0x7f918813c510 2023-12-26T23:12:15.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.985+0000 7f917dffb700 1 -- 172.21.15.131:0/3435286680 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9180056c60 con 0x7f918813c510 2023-12-26T23:12:15.988 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f916c066f30 msgr2=0x7f916c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f916c066f30 0x7f916c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9184003ae0 tx=0x7f9184007d40 comp rx=0 tx=0).stop 2023-12-26T23:12:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918813c510 msgr2=0x7f91880c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918813c510 0x7f91880c9760 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f918806fe40 tx=0x7f9180019000 comp rx=0 tx=0).stop 2023-12-26T23:12:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 shutdown_connections 2023-12-26T23:12:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f916c066f30 0x7f916c0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918813c510 0x7f91880c9760 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918806bfd0 0x7f9188153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 --2- 172.21.15.131:0/3435286680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91880cc810 0x7f91880c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 >> 172.21.15.131:0/3435286680 conn(0x7f91880ce3e0 msgr2=0x7f918813adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 shutdown_connections 2023-12-26T23:12:15.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:15.988+0000 7f918fbff700 1 -- 172.21.15.131:0/3435286680 wait complete. 2023-12-26T23:12:16.006 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T23:12:16.006 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:12:16.102 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:12:16.197 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T23:12:16.198 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-26T23:12:16.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.387+0000 7fa24d3ff700 1 Processor -- start 2023-12-26T23:12:16.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.387+0000 7fa24d3ff700 1 -- start start 2023-12-26T23:12:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.388+0000 7fa24d3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa2480b0bf0 0x7fa2480c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.388+0000 7fa24d3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa248156940 con 0x7fa2480cb320 2023-12-26T23:12:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.388+0000 7fa24d3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2480ae500 con 0x7fa2480b0750 2023-12-26T23:12:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.388+0000 7fa24d3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2480c8cc0 con 0x7fa2480b0bf0 2023-12-26T23:12:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa246ffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa2480cb320 0x7fa248069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52468/0 (socket says 172.21.15.131:52468) 2023-12-26T23:12:16.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa246ffd700 1 -- 172.21.15.131:0/1329780531 learned_addr learned my addr 172.21.15.131:0/1329780531 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa2477fe700 1 --2- 172.21.15.131:0/1329780531 >> v2:172.21.15.190:3300/0 conn(0x7fa2480b0bf0 0x7fa2480c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa245ffb700 1 -- 172.21.15.131:0/1329780531 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470125204 0 0) 0x7fa248156940 con 0x7fa2480cb320 2023-12-26T23:12:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa245ffb700 1 -- 172.21.15.131:0/1329780531 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa230003570 con 0x7fa2480cb320 2023-12-26T23:12:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa245ffb700 1 -- 172.21.15.131:0/1329780531 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24351205 0 0) 0x7fa2480ae500 con 0x7fa2480b0750 2023-12-26T23:12:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa245ffb700 1 -- 172.21.15.131:0/1329780531 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa248156940 con 0x7fa2480b0750 2023-12-26T23:12:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa245ffb700 1 -- 172.21.15.131:0/1329780531 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2859711136 0 0) 0x7fa230003570 con 0x7fa2480cb320 2023-12-26T23:12:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa2477fe700 1 -- 172.21.15.131:0/1329780531 >> v1:172.21.15.190:6789/0 conn(0x7fa2480b0750 legacy=0x7fa2480c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa2477fe700 1 -- 172.21.15.131:0/1329780531 >> v1:172.21.15.131:6789/0 conn(0x7fa2480cb320 legacy=0x7fa248069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.392 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.389+0000 7fa2477fe700 1 -- 172.21.15.131:0/1329780531 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2480b0280 con 0x7fa2480b0bf0 2023-12-26T23:12:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.390+0000 7fa2477fe700 1 --2- 172.21.15.131:0/1329780531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2480b0bf0 0x7fa2480c9b10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa240009e60 tx=0x7fa240009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=139c38be9c323855 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.390+0000 7fa245ffb700 1 -- 172.21.15.131:0/1329780531 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000e070 con 0x7fa2480b0bf0 2023-12-26T23:12:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.390+0000 7fa245ffb700 1 -- 172.21.15.131:0/1329780531 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24000b7a0 con 0x7fa2480b0bf0 2023-12-26T23:12:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.390+0000 7fa245ffb700 1 -- 172.21.15.131:0/1329780531 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238001470 con 0x7fa2480b0bf0 2023-12-26T23:12:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.390+0000 7fa24d3ff700 1 -- 172.21.15.131:0/1329780531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2480b0bf0 msgr2=0x7fa2480c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.390+0000 7fa24d3ff700 1 --2- 172.21.15.131:0/1329780531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2480b0bf0 0x7fa2480c9b10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa240009e60 tx=0x7fa240009780 comp rx=0 tx=0).stop 2023-12-26T23:12:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.390+0000 7fa24d3ff700 1 -- 172.21.15.131:0/1329780531 shutdown_connections 2023-12-26T23:12:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.390+0000 7fa24d3ff700 1 --2- 172.21.15.131:0/1329780531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2480b0bf0 0x7fa2480c9b10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.390+0000 7fa24d3ff700 1 -- 172.21.15.131:0/1329780531 >> 172.21.15.131:0/1329780531 conn(0x7fa248068040 msgr2=0x7fa248069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:16.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.391+0000 7fa24d3ff700 1 -- 172.21.15.131:0/1329780531 shutdown_connections 2023-12-26T23:12:16.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.391+0000 7fa24d3ff700 1 -- 172.21.15.131:0/1329780531 wait complete. 2023-12-26T23:12:16.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.392+0000 7fa24d3ff700 1 Processor -- start 2023-12-26T23:12:16.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.392+0000 7fa24d3ff700 1 -- start start 2023-12-26T23:12:16.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa24d3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2480b0750 0x7fa24815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa24d3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2480b0bf0 0x7fa24815b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa24d3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2480cb320 0x7fa248157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa24d3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa248159f30 con 0x7fa2480cb320 2023-12-26T23:12:16.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa24d3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24815a1d0 con 0x7fa2480b0750 2023-12-26T23:12:16.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa24d3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24815a4b0 con 0x7fa2480b0bf0 2023-12-26T23:12:16.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa2477fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2480cb320 0x7fa248157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.397 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa2477fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2480cb320 0x7fa248157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60044/0 (socket says 172.21.15.131:60044) 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa2477fe700 1 -- 172.21.15.131:0/4287371347 learned_addr learned my addr 172.21.15.131:0/4287371347 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa246ffd700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2480b0750 0x7fa24815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.393+0000 7fa2467fc700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2480b0bf0 0x7fa24815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa2477fe700 1 -- 172.21.15.131:0/4287371347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2480b0bf0 msgr2=0x7fa24815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa2477fe700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2480b0bf0 0x7fa24815b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa2477fe700 1 -- 172.21.15.131:0/4287371347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2480b0750 msgr2=0x7fa24815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa2477fe700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2480b0750 0x7fa24815c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa2477fe700 1 -- 172.21.15.131:0/4287371347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa240009a20 con 0x7fa2480cb320 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa2477fe700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2480cb320 0x7fa248157b50 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fa2480688f0 tx=0x7fa240017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa22effd700 1 -- 172.21.15.131:0/4287371347 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240012c40 con 0x7fa2480cb320 2023-12-26T23:12:16.398 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa22effd700 1 -- 172.21.15.131:0/4287371347 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa240012da0 con 0x7fa2480cb320 2023-12-26T23:12:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa22effd700 1 -- 172.21.15.131:0/4287371347 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24001e680 con 0x7fa2480cb320 2023-12-26T23:12:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24815a790 con 0x7fa2480cb320 2023-12-26T23:12:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.394+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24815ad50 con 0x7fa2480cb320 2023-12-26T23:12:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.395+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24815b0c0 con 0x7fa2480cb320 2023-12-26T23:12:16.399 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.396+0000 7fa22effd700 1 -- 172.21.15.131:0/4287371347 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa24001e9f0 con 0x7fa2480cb320 2023-12-26T23:12:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.397+0000 7fa22effd700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa234066f30 0x7fa2340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.397+0000 7fa22effd700 1 -- 172.21.15.131:0/4287371347 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5975+0+0 (secure 0 0 0) 0x7fa240086790 con 0x7fa2480cb320 2023-12-26T23:12:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.397+0000 7fa246ffd700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa234066f30 0x7fa2340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.398+0000 7fa246ffd700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa234066f30 0x7fa2340693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa24806f3d0 tx=0x7fa238000bc0 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:16.401 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.401+0000 7fa22effd700 1 -- 172.21.15.131:0/4287371347 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2400538f0 con 0x7fa2480cb320 2023-12-26T23:12:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.639+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2480657e0 con 0x7fa2480cb320 2023-12-26T23:12:16.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.639+0000 7fa22effd700 1 -- 172.21.15.131:0/4287371347 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+3542 (secure 0 0 0) 0x7fa240016df0 con 0x7fa2480cb320 2023-12-26T23:12:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:16.640 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8021,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2070005366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2070005366},{"type":"v1","addr":"172.21.15.190:6835","nonce":2070005366}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3800715315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3800715315},{"type":"v1","addr":"172.21.15.190:6834","nonce":3800715315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":9645,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3865566903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3865566903},{"type":"v1","addr":"172.21.15.131:6833","nonce":3865566903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":10002,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3971318694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3971318694},{"type":"v1","addr":"172.21.15.131:6835","nonce":3971318694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[]} 2023-12-26T23:12:16.642 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa234066f30 msgr2=0x7fa2340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa234066f30 0x7fa2340693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa24806f3d0 tx=0x7fa238000bc0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2480cb320 msgr2=0x7fa248157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2480cb320 0x7fa248157b50 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fa2480688f0 tx=0x7fa240017000 comp rx=0 tx=0).stop 2023-12-26T23:12:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 shutdown_connections 2023-12-26T23:12:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa234066f30 0x7fa2340693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2480cb320 0x7fa248157b50 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2480b0750 0x7fa24815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 --2- 172.21.15.131:0/4287371347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2480b0bf0 0x7fa24815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 >> 172.21.15.131:0/4287371347 conn(0x7fa248068040 msgr2=0x7fa248155c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:16.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 shutdown_connections 2023-12-26T23:12:16.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.642+0000 7fa24d3ff700 1 -- 172.21.15.131:0/4287371347 wait complete. 2023-12-26T23:12:16.644 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 92 2023-12-26T23:12:16.661 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-26T23:12:16.661 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-26T23:12:16.661 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T23:12:16.661 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:12:16.662 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-26T23:12:16.662 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-26T23:12:16.662 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T23:12:16.662 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:12:16.662 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-26T23:12:16.662 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-26T23:12:16.663 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T23:12:16.663 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:12:16.663 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-26T23:12:16.663 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-26T23:12:16.663 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T23:12:16.664 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:12:16.675 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T23:12:16.675 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-26T23:12:16.677 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T23:12:16.677 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-26T23:12:16.680 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T23:12:16.680 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-26T23:12:16.682 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T23:12:16.682 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-26T23:12:16.722 INFO:tasks.ceph.mds.a:Started 2023-12-26T23:12:16.723 DEBUG:teuthology.parallel:result is None 2023-12-26T23:12:16.726 INFO:tasks.ceph.mds.b:Started 2023-12-26T23:12:16.726 DEBUG:teuthology.parallel:result is None 2023-12-26T23:12:16.727 INFO:tasks.ceph.mds.c:Started 2023-12-26T23:12:16.727 DEBUG:teuthology.parallel:result is None 2023-12-26T23:12:16.730 INFO:tasks.ceph.mds.d:Started 2023-12-26T23:12:16.730 DEBUG:teuthology.parallel:result is None 2023-12-26T23:12:16.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-26T23:12:16.865 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2023-12-26T23:12:16.865 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2023-12-26T23:12:16.870 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-12-26T23:12:16.870 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T23:12:16.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.934+0000 7f53a3dff700 1 Processor -- start 2023-12-26T23:12:16.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.934+0000 7f53a3dff700 1 -- start start 2023-12-26T23:12:16.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.935+0000 7f53a3dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f539c06bfd0 0x7f539c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.935+0000 7f53a3dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539c0686c0 con 0x7f539c0cc100 2023-12-26T23:12:16.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.935+0000 7f53a3dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539c069520 con 0x7f539c13c510 2023-12-26T23:12:16.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.935+0000 7f53a3dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f539c068910 con 0x7f539c06bfd0 2023-12-26T23:12:16.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a139a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f539c06bfd0 0x7f539c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a139a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f539c06bfd0 0x7f539c0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60078/0 (socket says 172.21.15.131:60078) 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a139a700 1 -- 172.21.15.131:0/4130881853 learned_addr learned my addr 172.21.15.131:0/4130881853 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a239c700 1 --1- 172.21.15.131:0/4130881853 >> v1:172.21.15.131:6789/0 conn(0x7f539c0cc100 0x7f539c0cc520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52482/0 (socket says 172.21.15.131:52482) 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a1b9b700 1 --1- 172.21.15.131:0/4130881853 >> v1:172.21.15.190:6789/0 conn(0x7f539c13c510 0x7f539c06b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:44884/0 (socket says 172.21.15.131:44884) 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a139a700 1 -- 172.21.15.131:0/4130881853 >> v1:172.21.15.190:6789/0 conn(0x7f539c13c510 legacy=0x7f539c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a139a700 1 -- 172.21.15.131:0/4130881853 >> v1:172.21.15.131:6789/0 conn(0x7f539c0cc100 legacy=0x7f539c0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a139a700 1 -- 172.21.15.131:0/4130881853 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c068bf0 con 0x7f539c06bfd0 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a0b99700 1 -- 172.21.15.131:0/4130881853 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347167123 0 0) 0x7f539c069520 con 0x7f539c13c510 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a139a700 1 --2- 172.21.15.131:0/4130881853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 0x7f539c0c8a20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f538c0099b0 tx=0x7f538c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc35f3808335f2e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a0b99700 1 -- 172.21.15.131:0/4130881853 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00b860 con 0x7f539c06bfd0 2023-12-26T23:12:16.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.936+0000 7f53a0b99700 1 -- 172.21.15.131:0/4130881853 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c00b9c0 con 0x7f539c06bfd0 2023-12-26T23:12:16.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.937+0000 7f53a0b99700 1 -- 172.21.15.131:0/4130881853 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00bd60 con 0x7f539c06bfd0 2023-12-26T23:12:16.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.937+0000 7f53a3dff700 1 -- 172.21.15.131:0/4130881853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 msgr2=0x7f539c0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.937+0000 7f53a3dff700 1 --2- 172.21.15.131:0/4130881853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 0x7f539c0c8a20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f538c0099b0 tx=0x7f538c009250 comp rx=0 tx=0).stop 2023-12-26T23:12:16.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.937+0000 7f53a3dff700 1 -- 172.21.15.131:0/4130881853 shutdown_connections 2023-12-26T23:12:16.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.937+0000 7f53a3dff700 1 --2- 172.21.15.131:0/4130881853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 0x7f539c0c8a20 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.937+0000 7f53a3dff700 1 -- 172.21.15.131:0/4130881853 >> 172.21.15.131:0/4130881853 conn(0x7f539c0ce3e0 msgr2=0x7f539c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:16.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.937+0000 7f53a3dff700 1 -- 172.21.15.131:0/4130881853 shutdown_connections 2023-12-26T23:12:16.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.937+0000 7f53a3dff700 1 -- 172.21.15.131:0/4130881853 wait complete. 2023-12-26T23:12:16.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.939+0000 7f53a3dff700 1 Processor -- start 2023-12-26T23:12:16.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.939+0000 7f53a3dff700 1 -- start start 2023-12-26T23:12:16.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a3dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 0x7f539c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a3dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f539c0cc100 0x7f539c13b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a3dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f539c13c510 0x7f539c13ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a3dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c0b1e20 con 0x7f539c06bfd0 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a3dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c15ad10 con 0x7f539c13c510 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a3dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c15c6d0 con 0x7f539c0cc100 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a1b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 0x7f539c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a1b9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 0x7f539c15a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60086/0 (socket says 172.21.15.131:60086) 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a1b9b700 1 -- 172.21.15.131:0/470008843 learned_addr learned my addr 172.21.15.131:0/470008843 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a239c700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f539c13c510 0x7f539c13ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a139a700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f539c0cc100 0x7f539c13b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a1b9b700 1 -- 172.21.15.131:0/470008843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f539c0cc100 msgr2=0x7f539c13b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a1b9b700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f539c0cc100 0x7f539c13b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a1b9b700 1 -- 172.21.15.131:0/470008843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f539c13c510 msgr2=0x7f539c13ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a1b9b700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f539c13c510 0x7f539c13ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.940+0000 7f53a1b9b700 1 -- 172.21.15.131:0/470008843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c0094f0 con 0x7f539c06bfd0 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.941+0000 7f53a1b9b700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 0x7f539c15a730 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f539800c810 tx=0x7f5398007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.941+0000 7f5391ffb700 1 -- 172.21.15.131:0/470008843 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398005b90 con 0x7f539c06bfd0 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.941+0000 7f5391ffb700 1 -- 172.21.15.131:0/470008843 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5398005cf0 con 0x7f539c06bfd0 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.941+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c15c9b0 con 0x7f539c06bfd0 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.941+0000 7f5391ffb700 1 -- 172.21.15.131:0/470008843 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398007320 con 0x7f539c06bfd0 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.941+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c15c310 con 0x7f539c06bfd0 2023-12-26T23:12:16.945 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.942+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539c0b20c0 con 0x7f539c06bfd0 2023-12-26T23:12:16.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.943+0000 7f5391ffb700 1 -- 172.21.15.131:0/470008843 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f539801b3b0 con 0x7f539c06bfd0 2023-12-26T23:12:16.946 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.943+0000 7f5391ffb700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5380066f30 0x7f53800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:16.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.944+0000 7f5391ffb700 1 -- 172.21.15.131:0/470008843 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f5398083ae0 con 0x7f539c06bfd0 2023-12-26T23:12:16.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.944+0000 7f53a139a700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5380066f30 0x7f53800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:16.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.944+0000 7f53a139a700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5380066f30 0x7f53800693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f538c0099b0 tx=0x7f538c017000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:16.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:16.948+0000 7f5391ffb700 1 -- 172.21.15.131:0/470008843 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5398050c40 con 0x7f539c06bfd0 2023-12-26T23:12:17.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:17.161+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f539c15c500 con 0x7f539c06bfd0 2023-12-26T23:12:18.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.015+0000 7f5391ffb700 1 -- 172.21.15.131:0/470008843 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v206) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5398017020 con 0x7f539c06bfd0 2023-12-26T23:12:18.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5380066f30 msgr2=0x7f53800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5380066f30 0x7f53800693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f538c0099b0 tx=0x7f538c017000 comp rx=0 tx=0).stop 2023-12-26T23:12:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 msgr2=0x7f539c15a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 0x7f539c15a730 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f539800c810 tx=0x7f5398007d90 comp rx=0 tx=0).stop 2023-12-26T23:12:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 shutdown_connections 2023-12-26T23:12:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f5380066f30 0x7f53800693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f539c13c510 0x7f539c13ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f539c06bfd0 0x7f539c15a730 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 --2- 172.21.15.131:0/470008843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f539c0cc100 0x7f539c13b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 >> 172.21.15.131:0/470008843 conn(0x7f539c0ce3e0 msgr2=0x7f539c0c4580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:18.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 shutdown_connections 2023-12-26T23:12:18.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.019+0000 7f53a3dff700 1 -- 172.21.15.131:0/470008843 wait complete. 2023-12-26T23:12:18.021 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2023-12-26T23:12:18.040 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-26T23:12:18.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.234+0000 7fa60abff700 1 Processor -- start 2023-12-26T23:12:18.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.234+0000 7fa60abff700 1 -- start start 2023-12-26T23:12:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.235+0000 7fa60abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6041596a0 0x7fa6040b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.235+0000 7fa60abff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa604159a70 0x7fa6040b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.235+0000 7fa60abff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040afcd0 con 0x7fa6040b0160 2023-12-26T23:12:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.235+0000 7fa60abff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6040b3e70 con 0x7fa604159a70 2023-12-26T23:12:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.235+0000 7fa60abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040b0cd0 con 0x7fa6041596a0 2023-12-26T23:12:18.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.236+0000 7fa603fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa604159a70 0x7fa6040b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.236+0000 7fa60899b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6041596a0 0x7fa6040b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.236+0000 7fa60919c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa6040b0160 0x7fa6040b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:44886/0 (socket says 172.21.15.131:44886) 2023-12-26T23:12:18.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.236+0000 7fa60919c700 1 -- 172.21.15.131:0/345813098 learned_addr learned my addr 172.21.15.131:0/345813098 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:18.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.237+0000 7fa6037fe700 1 -- 172.21.15.131:0/345813098 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216747821 0 0) 0x7fa6040afcd0 con 0x7fa6040b0160 2023-12-26T23:12:18.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.237+0000 7fa60899b700 1 -- 172.21.15.131:0/345813098 >> v2:172.21.15.190:3300/0 conn(0x7fa604159a70 msgr2=0x7fa6040b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:18.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.237+0000 7fa60899b700 1 --2- 172.21.15.131:0/345813098 >> v2:172.21.15.190:3300/0 conn(0x7fa604159a70 0x7fa6040b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.239 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.237+0000 7fa60899b700 1 -- 172.21.15.131:0/345813098 >> v1:172.21.15.190:6789/0 conn(0x7fa6040b0160 legacy=0x7fa6040b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.237+0000 7fa60899b700 1 -- 172.21.15.131:0/345813098 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040b2e00 con 0x7fa6041596a0 2023-12-26T23:12:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.237+0000 7fa60899b700 1 --2- 172.21.15.131:0/345813098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6041596a0 0x7fa6040b2380 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80098e0 tx=0x7fa5f80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67207278e9f6b401 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.237+0000 7fa6037fe700 1 -- 172.21.15.131:0/345813098 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800b640 con 0x7fa6041596a0 2023-12-26T23:12:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.237+0000 7fa6037fe700 1 -- 172.21.15.131:0/345813098 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f800b7a0 con 0x7fa6041596a0 2023-12-26T23:12:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.237+0000 7fa6037fe700 1 -- 172.21.15.131:0/345813098 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800bb40 con 0x7fa6041596a0 2023-12-26T23:12:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.238+0000 7fa60abff700 1 -- 172.21.15.131:0/345813098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6041596a0 msgr2=0x7fa6040b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.238+0000 7fa60abff700 1 --2- 172.21.15.131:0/345813098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6041596a0 0x7fa6040b2380 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80098e0 tx=0x7fa5f80091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.238+0000 7fa60abff700 1 -- 172.21.15.131:0/345813098 shutdown_connections 2023-12-26T23:12:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.238+0000 7fa60abff700 1 --2- 172.21.15.131:0/345813098 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6041596a0 0x7fa6040b2380 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.238+0000 7fa60abff700 1 --2- 172.21.15.131:0/345813098 >> v2:172.21.15.190:3300/0 conn(0x7fa604159a70 0x7fa6040b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.238+0000 7fa60abff700 1 -- 172.21.15.131:0/345813098 >> 172.21.15.131:0/345813098 conn(0x7fa60406ece0 msgr2=0x7fa60406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:18.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.238+0000 7fa60abff700 1 -- 172.21.15.131:0/345813098 shutdown_connections 2023-12-26T23:12:18.241 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.238+0000 7fa60abff700 1 -- 172.21.15.131:0/345813098 wait complete. 2023-12-26T23:12:18.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.239+0000 7fa60abff700 1 Processor -- start 2023-12-26T23:12:18.244 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.239+0000 7fa60abff700 1 -- start start 2023-12-26T23:12:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.240+0000 7fa60abff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040b0160 0x7fa60415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.240+0000 7fa60abff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6041596a0 0x7fa6040c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.240+0000 7fa60abff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa604159a70 0x7fa604157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.240+0000 7fa60abff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040c92b0 con 0x7fa6041596a0 2023-12-26T23:12:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.240+0000 7fa60abff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040cdad0 con 0x7fa604159a70 2023-12-26T23:12:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.240+0000 7fa60abff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60415ae60 con 0x7fa6040b0160 2023-12-26T23:12:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.240+0000 7fa603fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6041596a0 0x7fa6040c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa603fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6041596a0 0x7fa6040c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60088/0 (socket says 172.21.15.131:60088) 2023-12-26T23:12:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa603fff700 1 -- 172.21.15.131:0/1669667212 learned_addr learned my addr 172.21.15.131:0/1669667212 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa60899b700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040b0160 0x7fa60415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa60919c700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa604159a70 0x7fa604157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa603fff700 1 -- 172.21.15.131:0/1669667212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040b0160 msgr2=0x7fa60415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa603fff700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040b0160 0x7fa60415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa603fff700 1 -- 172.21.15.131:0/1669667212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa604159a70 msgr2=0x7fa604157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa603fff700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa604159a70 0x7fa604157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa603fff700 1 -- 172.21.15.131:0/1669667212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009420 con 0x7fa6041596a0 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa60919c700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa604159a70 0x7fa604157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa603fff700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6041596a0 0x7fa6040c4010 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000e930 tx=0x7fa5f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa600ff9700 1 -- 172.21.15.131:0/1669667212 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0013680 con 0x7fa6041596a0 2023-12-26T23:12:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa600ff9700 1 -- 172.21.15.131:0/1669667212 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f00137e0 con 0x7fa6041596a0 2023-12-26T23:12:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa600ff9700 1 -- 172.21.15.131:0/1669667212 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0013b80 con 0x7fa6041596a0 2023-12-26T23:12:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.241+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6040c8980 con 0x7fa6041596a0 2023-12-26T23:12:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.242+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6040c8f40 con 0x7fa6041596a0 2023-12-26T23:12:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.242+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6040c9460 con 0x7fa6041596a0 2023-12-26T23:12:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.243+0000 7fa600ff9700 1 -- 172.21.15.131:0/1669667212 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa5f000c460 con 0x7fa6041596a0 2023-12-26T23:12:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.244+0000 7fa600ff9700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa5e8066f30 0x7fa5e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.244+0000 7fa600ff9700 1 -- 172.21.15.131:0/1669667212 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5373+0+0 (secure 0 0 0) 0x7fa5f00845c0 con 0x7fa6041596a0 2023-12-26T23:12:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.245+0000 7fa60899b700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa5e8066f30 0x7fa5e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.245+0000 7fa60899b700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa5e8066f30 0x7fa5e80693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80093f0 tx=0x7fa5f8017000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:18.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.248+0000 7fa600ff9700 1 -- 172.21.15.131:0/1669667212 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6040c9460 con 0x7fa6041596a0 2023-12-26T23:12:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.514+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa6040657e0 con 0x7fa6041596a0 2023-12-26T23:12:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.514+0000 7fa600ff9700 1 -- 172.21.15.131:0/1669667212 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v49) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa5f00518e0 con 0x7fa6041596a0 2023-12-26T23:12:18.515 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:18.515 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:12:18.515 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2023-12-26T23:12:18.515 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlf1LuOhAA6I4KyQYXc5arU3u6vO3MVw==", 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.516 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlGrXSFRAAakarDAfWItaoAiqN39i9GQ==", 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.517 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlzsqCChAAS5dJdw7mPiLMLe44HQbUlQ==", 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.518 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tltz0NIRAAAaqKhS+60Xcu2HnIdY43fw==", 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.519 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA3R4tlbJMeHhAAbQVi4LWFAfcfIRFkRJTOOg==", 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA6R4tlVxhgIhAAdYZwNhyEwBsjBTNmJGVEGw==", 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2023-12-26T23:12:18.521 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA9R4tlUEioJRAAILNgcoNgNTr265wIqKsDfA==", 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBAR4tllStDKRAAFvwLuu2o31Pm0IxVCKSdNQ==", 2023-12-26T23:12:18.522 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBHR4tl4FuOAxAA54id9LM2fj2cFBjKM1GWkw==", 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBKR4tlEnkJCBAAiYP+/ru74P+mTBY/lcMv7A==", 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:12:18.524 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBNR4tlsq0eCxAAy8Ih6FhwMUOxktR9Lds+iA==", 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.525 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQBQR4tleXs7DhAAmAZPCmZ3I9ptzfki20QCgA==", 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.526 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tl89/EKhAAP9mR5dBYspqJVnn5G+XbMA==", 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlikMmORAA6D4K/UpNUUD3FInuUwvWYA==", 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.527 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAzR4tlW7nvMRAAqPH/Cyl5HCHJtBZJsovVJw==", 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T23:12:18.528 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQA0R4tlXKauBBAAE4SaTH5RPkVyMMaWt1GxmA==", 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2023-12-26T23:12:18.529 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tl+YAnBBAAfUC8B/BeYsn7p3XY/TF8hg==", 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2023-12-26T23:12:18.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tlWo2dMxAAM6Ivyctg6zb22Hg23eGdJw==", 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.531 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAyR4tljZktKhAA8X299KuitRj2CDEjLO4PCQ==", 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2023-12-26T23:12:18.532 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:12:18.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa5e8066f30 msgr2=0x7fa5e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa5e8066f30 0x7fa5e80693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80093f0 tx=0x7fa5f8017000 comp rx=0 tx=0).stop 2023-12-26T23:12:18.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6041596a0 msgr2=0x7fa6040c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6041596a0 0x7fa6040c4010 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000e930 tx=0x7fa5f00091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 shutdown_connections 2023-12-26T23:12:18.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa5e8066f30 0x7fa5e80693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa604159a70 0x7fa604157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040b0160 0x7fa60415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 --2- 172.21.15.131:0/1669667212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6041596a0 0x7fa6040c4010 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 >> 172.21.15.131:0/1669667212 conn(0x7fa60406ece0 msgr2=0x7fa6040ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:18.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 shutdown_connections 2023-12-26T23:12:18.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.517+0000 7fa60abff700 1 -- 172.21.15.131:0/1669667212 wait complete. 2023-12-26T23:12:18.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.727+0000 7f04991ff700 1 Processor -- start 2023-12-26T23:12:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.727+0000 7f04991ff700 1 -- start start 2023-12-26T23:12:18.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.728+0000 7f04991ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04940c8820 0x7f04940ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.728+0000 7f04991ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0494157a80 con 0x7f04940b19f0 2023-12-26T23:12:18.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.728+0000 7f04991ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04940cac50 con 0x7f04940c8bf0 2023-12-26T23:12:18.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.728+0000 7f04991ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04940caf30 con 0x7f04940c8820 2023-12-26T23:12:18.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.728+0000 7f049359e700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f04940b19f0 0x7f0494161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52496/0 (socket says 172.21.15.131:52496) 2023-12-26T23:12:18.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.728+0000 7f049359e700 1 -- 172.21.15.131:0/1151102340 learned_addr learned my addr 172.21.15.131:0/1151102340 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:18.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.728+0000 7f0492d9d700 1 --2- 172.21.15.131:0/1151102340 >> v2:172.21.15.190:3300/0 conn(0x7f04940c8820 0x7f04940ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.729+0000 7f049259c700 1 -- 172.21.15.131:0/1151102340 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779895568 0 0) 0x7f0494157a80 con 0x7f04940b19f0 2023-12-26T23:12:18.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.729+0000 7f049259c700 1 -- 172.21.15.131:0/1151102340 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0480003570 con 0x7f04940b19f0 2023-12-26T23:12:18.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.729+0000 7f0492d9d700 1 -- 172.21.15.131:0/1151102340 >> v1:172.21.15.190:6789/0 conn(0x7f04940c8bf0 legacy=0x7f04940b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.729+0000 7f0492d9d700 1 -- 172.21.15.131:0/1151102340 >> v1:172.21.15.131:6789/0 conn(0x7f04940b19f0 legacy=0x7f0494161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.729+0000 7f0492d9d700 1 -- 172.21.15.131:0/1151102340 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940cb210 con 0x7f04940c8820 2023-12-26T23:12:18.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.729+0000 7f0492d9d700 1 --2- 172.21.15.131:0/1151102340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04940c8820 0x7f04940ca300 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f04840098e0 tx=0x7f04840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c110cd2567dcaca server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:18.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f049259c700 1 -- 172.21.15.131:0/1151102340 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048400b640 con 0x7f04940c8820 2023-12-26T23:12:18.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f049259c700 1 -- 172.21.15.131:0/1151102340 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048400b7a0 con 0x7f04940c8820 2023-12-26T23:12:18.731 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f049259c700 1 -- 172.21.15.131:0/1151102340 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048400bb20 con 0x7f04940c8820 2023-12-26T23:12:18.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f04991ff700 1 -- 172.21.15.131:0/1151102340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04940c8820 msgr2=0x7f04940ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f04991ff700 1 --2- 172.21.15.131:0/1151102340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04940c8820 0x7f04940ca300 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f04840098e0 tx=0x7f04840091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f04991ff700 1 -- 172.21.15.131:0/1151102340 shutdown_connections 2023-12-26T23:12:18.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f04991ff700 1 --2- 172.21.15.131:0/1151102340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04940c8820 0x7f04940ca300 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f04991ff700 1 -- 172.21.15.131:0/1151102340 >> 172.21.15.131:0/1151102340 conn(0x7f0494068040 msgr2=0x7f0494069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:18.735 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f04991ff700 1 -- 172.21.15.131:0/1151102340 shutdown_connections 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.730+0000 7f04991ff700 1 -- 172.21.15.131:0/1151102340 wait complete. 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.732+0000 7f04991ff700 1 Processor -- start 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.732+0000 7f04991ff700 1 -- start start 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f04991ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04940b19f0 0x7f04940afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f04991ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04940c8820 0x7f049415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f04991ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04940c8bf0 0x7f04941697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f04991ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940af0d0 con 0x7f04940c8bf0 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f04991ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940b10c0 con 0x7f04940c8820 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f04991ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049416a340 con 0x7f04940b19f0 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f049359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04940c8bf0 0x7f04941697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.736 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f049359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04940c8bf0 0x7f04941697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60090/0 (socket says 172.21.15.131:60090) 2023-12-26T23:12:18.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f049359e700 1 -- 172.21.15.131:0/2113008036 learned_addr learned my addr 172.21.15.131:0/2113008036 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:18.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f0492d9d700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04940b19f0 0x7f04940afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f048bfff700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04940c8820 0x7f049415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f049359e700 1 -- 172.21.15.131:0/2113008036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04940b19f0 msgr2=0x7f04940afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f049359e700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04940b19f0 0x7f04940afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f049359e700 1 -- 172.21.15.131:0/2113008036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04940c8820 msgr2=0x7f049415c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f049359e700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04940c8820 0x7f049415c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f049359e700 1 -- 172.21.15.131:0/2113008036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0484009420 con 0x7f04940c8bf0 2023-12-26T23:12:18.737 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.733+0000 7f0492d9d700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04940b19f0 0x7f04940afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:18.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.734+0000 7f049359e700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04940c8bf0 0x7f04941697c0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f048c00cc40 tx=0x7f048c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:18.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.734+0000 7f048affd700 1 -- 172.21.15.131:0/2113008036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00a960 con 0x7f04940c8bf0 2023-12-26T23:12:18.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.734+0000 7f048affd700 1 -- 172.21.15.131:0/2113008036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c00aac0 con 0x7f04940c8bf0 2023-12-26T23:12:18.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.734+0000 7f048affd700 1 -- 172.21.15.131:0/2113008036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00ae40 con 0x7f04940c8bf0 2023-12-26T23:12:18.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.734+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049416a600 con 0x7f04940c8bf0 2023-12-26T23:12:18.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.734+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04940af600 con 0x7f04940c8bf0 2023-12-26T23:12:18.738 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.735+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04940af970 con 0x7f04940c8bf0 2023-12-26T23:12:18.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.736+0000 7f048affd700 1 -- 172.21.15.131:0/2113008036 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f048c007300 con 0x7f04940c8bf0 2023-12-26T23:12:18.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.737+0000 7f048affd700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0474066f30 0x7f04740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:18.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.737+0000 7f048affd700 1 -- 172.21.15.131:0/2113008036 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f048c0848e0 con 0x7f04940c8bf0 2023-12-26T23:12:18.739 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.738+0000 7f0492d9d700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0474066f30 0x7f04740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:18.740 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.738+0000 7f0492d9d700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0474066f30 0x7f04740693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04840093f0 tx=0x7f0484017000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:18.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.741+0000 7f048affd700 1 -- 172.21.15.131:0/2113008036 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f048c051c00 con 0x7f04940c8bf0 2023-12-26T23:12:18.938 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.938+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04940657e0 con 0x7f04940c8bf0 2023-12-26T23:12:18.939 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.939+0000 7f048affd700 1 -- 172.21.15.131:0/2113008036 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+12995 (secure 0 0 0) 0x7f048c055220 con 0x7f04940c8bf0 2023-12-26T23:12:18.940 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:18.940 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":206,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:18.011568+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0474066f30 msgr2=0x7f04740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0474066f30 0x7f04740693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04840093f0 tx=0x7f0484017000 comp rx=0 tx=0).stop 2023-12-26T23:12:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04940c8bf0 msgr2=0x7f04941697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04940c8bf0 0x7f04941697c0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f048c00cc40 tx=0x7f048c001a00 comp rx=0 tx=0).stop 2023-12-26T23:12:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 shutdown_connections 2023-12-26T23:12:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0474066f30 0x7f04740693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04940c8bf0 0x7f04941697c0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04940b19f0 0x7f04940afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 --2- 172.21.15.131:0/2113008036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04940c8820 0x7f049415c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 >> 172.21.15.131:0/2113008036 conn(0x7f0494068040 msgr2=0x7f0494156580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 shutdown_connections 2023-12-26T23:12:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:18.942+0000 7f04991ff700 1 -- 172.21.15.131:0/2113008036 wait complete. 2023-12-26T23:12:18.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-12-26T23:12:18.961 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T23:12:19.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.155+0000 7f544f9ff700 1 Processor -- start 2023-12-26T23:12:19.155 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.155+0000 7f544f9ff700 1 -- start start 2023-12-26T23:12:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.156+0000 7f544f9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f544813c510 0x7f54480c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.156+0000 7f544f9ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f54480cc810 0x7f544806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.156+0000 7f544f9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54480686c0 con 0x7f544806b8c0 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.156+0000 7f544f9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f544806c280 con 0x7f544813c510 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.156+0000 7f544f9ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5448069520 con 0x7f54480cc810 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f544d79b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f544813c510 0x7f54480c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f544d79b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f544813c510 0x7f54480c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60104/0 (socket says 172.21.15.131:60104) 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f544d79b700 1 -- 172.21.15.131:0/772575975 learned_addr learned my addr 172.21.15.131:0/772575975 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f544df9c700 1 --2- 172.21.15.131:0/772575975 >> v2:172.21.15.190:3300/0 conn(0x7f54480cc810 0x7f544806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f544d79b700 1 -- 172.21.15.131:0/772575975 >> v2:172.21.15.190:3300/0 conn(0x7f54480cc810 msgr2=0x7f544806bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f544d79b700 1 --2- 172.21.15.131:0/772575975 >> v2:172.21.15.190:3300/0 conn(0x7f54480cc810 0x7f544806bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f544d79b700 1 -- 172.21.15.131:0/772575975 >> v1:172.21.15.190:6789/0 conn(0x7f544806b8c0 legacy=0x7f54480cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f544d79b700 1 -- 172.21.15.131:0/772575975 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5448068910 con 0x7f544813c510 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f544d79b700 1 --2- 172.21.15.131:0/772575975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f544813c510 0x7f54480c8a20 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f544806f6a0 tx=0x7f5444009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c95e3aab6d61878 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.157+0000 7f543ffff700 1 -- 172.21.15.131:0/772575975 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444005c10 con 0x7f544813c510 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f543ffff700 1 -- 172.21.15.131:0/772575975 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5444005d70 con 0x7f544813c510 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f543ffff700 1 -- 172.21.15.131:0/772575975 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544400b880 con 0x7f544813c510 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f544f9ff700 1 -- 172.21.15.131:0/772575975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f544813c510 msgr2=0x7f54480c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f544f9ff700 1 --2- 172.21.15.131:0/772575975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f544813c510 0x7f54480c8a20 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f544806f6a0 tx=0x7f5444009220 comp rx=0 tx=0).stop 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f544f9ff700 1 -- 172.21.15.131:0/772575975 shutdown_connections 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f544f9ff700 1 --2- 172.21.15.131:0/772575975 >> v2:172.21.15.190:3300/0 conn(0x7f54480cc810 0x7f544806bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f544f9ff700 1 --2- 172.21.15.131:0/772575975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f544813c510 0x7f54480c8a20 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.160 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f544f9ff700 1 -- 172.21.15.131:0/772575975 >> 172.21.15.131:0/772575975 conn(0x7f54480ce3e0 msgr2=0x7f54480ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:19.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f544f9ff700 1 -- 172.21.15.131:0/772575975 shutdown_connections 2023-12-26T23:12:19.161 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.158+0000 7f544f9ff700 1 -- 172.21.15.131:0/772575975 wait complete. 2023-12-26T23:12:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.160+0000 7f544f9ff700 1 Processor -- start 2023-12-26T23:12:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.160+0000 7f544f9ff700 1 -- start start 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544f9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544806b8c0 0x7f5448151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544f9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54480cc810 0x7f54480c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544f9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544813c510 0x7f54480c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544f9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544815b740 con 0x7f54480cc810 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544f9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544815ab30 con 0x7f544813c510 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544f9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544815ae10 con 0x7f544806b8c0 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544cf9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54480cc810 0x7f54480c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544cf9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54480cc810 0x7f54480c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60120/0 (socket says 172.21.15.131:60120) 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544cf9a700 1 -- 172.21.15.131:0/4196105126 learned_addr learned my addr 172.21.15.131:0/4196105126 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544cf9a700 1 -- 172.21.15.131:0/4196105126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544806b8c0 msgr2=0x7f5448151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544cf9a700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544806b8c0 0x7f5448151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544cf9a700 1 -- 172.21.15.131:0/4196105126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544813c510 msgr2=0x7f54480c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544cf9a700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544813c510 0x7f54480c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.161+0000 7f544cf9a700 1 -- 172.21.15.131:0/4196105126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444009480 con 0x7f54480cc810 2023-12-26T23:12:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.162+0000 7f544cf9a700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54480cc810 0x7f54480c3e70 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f543800c4e0 tx=0x7f543800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.162+0000 7f543d7fa700 1 -- 172.21.15.131:0/4196105126 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438011590 con 0x7f54480cc810 2023-12-26T23:12:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.162+0000 7f543d7fa700 1 -- 172.21.15.131:0/4196105126 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5438011710 con 0x7f54480cc810 2023-12-26T23:12:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.162+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544815b0f0 con 0x7f54480cc810 2023-12-26T23:12:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.162+0000 7f543d7fa700 1 -- 172.21.15.131:0/4196105126 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438011ad0 con 0x7f54480cc810 2023-12-26T23:12:19.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.162+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5448159dd0 con 0x7f54480cc810 2023-12-26T23:12:19.166 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.163+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54480657e0 con 0x7f54480cc810 2023-12-26T23:12:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.164+0000 7f543d7fa700 1 -- 172.21.15.131:0/4196105126 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f543801b3b0 con 0x7f54480cc810 2023-12-26T23:12:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.165+0000 7f543d7fa700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f542c066f30 0x7f542c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.165+0000 7f543d7fa700 1 -- 172.21.15.131:0/4196105126 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f54380839c0 con 0x7f54480cc810 2023-12-26T23:12:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.165+0000 7f544d79b700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f542c066f30 0x7f542c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.166+0000 7f544d79b700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f542c066f30 0x7f542c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5444009260 tx=0x7f5444016000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:19.169 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.169+0000 7f543d7fa700 1 -- 172.21.15.131:0/4196105126 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5438050ce0 con 0x7f54480cc810 2023-12-26T23:12:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.359+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f544815b880 con 0x7f54480cc810 2023-12-26T23:12:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.359+0000 7f543d7fa700 1 -- 172.21.15.131:0/4196105126 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v206) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5438017070 con 0x7f54480cc810 2023-12-26T23:12:19.359 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T23:12:19.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.361+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f542c066f30 msgr2=0x7f542c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.361+0000 7f544f9ff700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f542c066f30 0x7f542c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5444009260 tx=0x7f5444016000 comp rx=0 tx=0).stop 2023-12-26T23:12:19.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.361+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54480cc810 msgr2=0x7f54480c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.361+0000 7f544f9ff700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54480cc810 0x7f54480c3e70 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f543800c4e0 tx=0x7f543800c8a0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.362+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 shutdown_connections 2023-12-26T23:12:19.362 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.362+0000 7f544f9ff700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f542c066f30 0x7f542c0693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.362+0000 7f544f9ff700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544813c510 0x7f54480c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.362+0000 7f544f9ff700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544806b8c0 0x7f5448151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.362+0000 7f544f9ff700 1 --2- 172.21.15.131:0/4196105126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54480cc810 0x7f54480c3e70 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.362+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 >> 172.21.15.131:0/4196105126 conn(0x7f54480ce3e0 msgr2=0x7f54480cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:19.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.362+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 shutdown_connections 2023-12-26T23:12:19.363 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.362+0000 7f544f9ff700 1 -- 172.21.15.131:0/4196105126 wait complete. 2023-12-26T23:12:19.380 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-26T23:12:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.570+0000 7fcf37dff700 1 Processor -- start 2023-12-26T23:12:19.570 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.570+0000 7fcf37dff700 1 -- start start 2023-12-26T23:12:19.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.571+0000 7fcf37dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf30159a70 0x7fcf300b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.571+0000 7fcf37dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf300b2a90 0x7fcf300b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.571+0000 7fcf37dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf300afcd0 con 0x7fcf301596a0 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.571+0000 7fcf37dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf300b3e70 con 0x7fcf300b2a90 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.571+0000 7fcf37dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300b0f20 con 0x7fcf30159a70 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.571+0000 7fcf35b9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcf301596a0 0x7fcf300b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52500/0 (socket says 172.21.15.131:52500) 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.571+0000 7fcf35b9b700 1 -- 172.21.15.131:0/678077569 learned_addr learned my addr 172.21.15.131:0/678077569 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.571+0000 7fcf3539a700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf30159a70 0x7fcf300b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.571+0000 7fcf3639c700 1 --2- 172.21.15.131:0/678077569 >> v2:172.21.15.190:3300/0 conn(0x7fcf300b2a90 0x7fcf300b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894180331 0 0) 0x7fcf300afcd0 con 0x7fcf301596a0 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf1c003570 con 0x7fcf301596a0 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538890186 0 0) 0x7fcf1c003570 con 0x7fcf301596a0 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf30159a70 msgr2=0x7fcf300b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:19.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf30159a70 0x7fcf300b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 >> v2:172.21.15.190:3300/0 conn(0x7fcf300b2a90 msgr2=0x7fcf300b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 --2- 172.21.15.131:0/678077569 >> v2:172.21.15.190:3300/0 conn(0x7fcf300b2a90 0x7fcf300b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf300b3540 con 0x7fcf301596a0 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fcf2c001470 con 0x7fcf301596a0 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf3639c700 1 --2- 172.21.15.131:0/678077569 >> v2:172.21.15.190:3300/0 conn(0x7fcf300b2a90 0x7fcf300b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 >> v1:172.21.15.131:6789/0 conn(0x7fcf301596a0 legacy=0x7fcf300b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c003a60 0x7fcf1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf30159a70 0x7fcf1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf1c004d50 0x7fcf1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300b3e70 con 0x7fcf1c004d50 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300b0f20 con 0x7fcf1c003a60 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c00ba60 con 0x7fcf30159a70 2023-12-26T23:12:19.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf3639c700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf1c004d50 0x7fcf1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf35b9b700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c003a60 0x7fcf1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.572+0000 7fcf3539a700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf30159a70 0x7fcf1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.573+0000 7fcf3639c700 1 -- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf30159a70 msgr2=0x7fcf1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.573+0000 7fcf3639c700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf30159a70 0x7fcf1c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.573+0000 7fcf3639c700 1 -- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c003a60 msgr2=0x7fcf1c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.573+0000 7fcf3639c700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c003a60 0x7fcf1c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.573+0000 7fcf3639c700 1 -- 172.21.15.131:0/678077569 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c003570 con 0x7fcf1c004d50 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.573+0000 7fcf3639c700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf1c004d50 0x7fcf1c0051c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800e3c0 tx=0x7fcf2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dcaa98eb15fe96 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.573+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2800ec10 con 0x7fcf1c004d50 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.573+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2800f040 con 0x7fcf1c004d50 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.573+0000 7fcf34b99700 1 -- 172.21.15.131:0/678077569 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28014480 con 0x7fcf1c004d50 2023-12-26T23:12:19.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.574+0000 7fcf37dff700 1 -- 172.21.15.131:0/678077569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf1c004d50 msgr2=0x7fcf1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.574+0000 7fcf37dff700 1 --2- 172.21.15.131:0/678077569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf1c004d50 0x7fcf1c0051c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800e3c0 tx=0x7fcf2800b040 comp rx=0 tx=0).stop 2023-12-26T23:12:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.574+0000 7fcf3639c700 1 -- 172.21.15.131:0/678077569 reap_dead start 2023-12-26T23:12:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.574+0000 7fcf37dff700 1 -- 172.21.15.131:0/678077569 shutdown_connections 2023-12-26T23:12:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.574+0000 7fcf37dff700 1 -- 172.21.15.131:0/678077569 >> 172.21.15.131:0/678077569 conn(0x7fcf3006ece0 msgr2=0x7fcf3006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.574+0000 7fcf37dff700 1 -- 172.21.15.131:0/678077569 shutdown_connections 2023-12-26T23:12:19.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.574+0000 7fcf37dff700 1 -- 172.21.15.131:0/678077569 wait complete. 2023-12-26T23:12:19.580 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.575+0000 7fcf37dff700 1 Processor -- start 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.575+0000 7fcf37dff700 1 -- start start 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.576+0000 7fcf37dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c004d50 0x7fcf300ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.576+0000 7fcf37dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3016d5a0 0x7fcf3016aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.576+0000 7fcf37dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3016af80 0x7fcf3015e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.576+0000 7fcf37dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf301545e0 con 0x7fcf3016d5a0 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.576+0000 7fcf37dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3016dcf0 con 0x7fcf1c004d50 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.576+0000 7fcf37dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3016dfd0 con 0x7fcf3016af80 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.576+0000 7fcf3539a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3016d5a0 0x7fcf3016aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3539a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3016d5a0 0x7fcf3016aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60138/0 (socket says 172.21.15.131:60138) 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3539a700 1 -- 172.21.15.131:0/2522435780 learned_addr learned my addr 172.21.15.131:0/2522435780 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf35b9b700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c004d50 0x7fcf300ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3639c700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3016af80 0x7fcf3015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3539a700 1 -- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3016af80 msgr2=0x7fcf3015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3539a700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3016af80 0x7fcf3015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3539a700 1 -- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c004d50 msgr2=0x7fcf300ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3539a700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c004d50 0x7fcf300ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3539a700 1 -- 172.21.15.131:0/2522435780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3016e2b0 con 0x7fcf3016d5a0 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf35b9b700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c004d50 0x7fcf300ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3639c700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3016af80 0x7fcf3015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf3539a700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3016d5a0 0x7fcf3016aa40 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fcf20006ac0 tx=0x7fcf200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf25ffb700 1 -- 172.21.15.131:0/2522435780 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000fa10 con 0x7fcf3016d5a0 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf25ffb700 1 -- 172.21.15.131:0/2522435780 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2000fb70 con 0x7fcf3016d5a0 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf25ffb700 1 -- 172.21.15.131:0/2522435780 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20014440 con 0x7fcf3016d5a0 2023-12-26T23:12:19.582 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.577+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3016da10 con 0x7fcf3016d5a0 2023-12-26T23:12:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.578+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3015e9c0 con 0x7fcf3016d5a0 2023-12-26T23:12:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.579+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf30154420 con 0x7fcf3016d5a0 2023-12-26T23:12:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.580+0000 7fcf25ffb700 1 -- 172.21.15.131:0/2522435780 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcf20014870 con 0x7fcf3016d5a0 2023-12-26T23:12:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.581+0000 7fcf25ffb700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcf14066f30 0x7fcf140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.581+0000 7fcf25ffb700 1 -- 172.21.15.131:0/2522435780 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5373+0+0 (secure 0 0 0) 0x7fcf20018030 con 0x7fcf3016d5a0 2023-12-26T23:12:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.581+0000 7fcf35b9b700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcf14066f30 0x7fcf140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.582+0000 7fcf35b9b700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcf14066f30 0x7fcf140693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0079e0 tx=0x7fcf2c009040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:19.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.585+0000 7fcf25ffb700 1 -- 172.21.15.131:0/2522435780 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf30154420 con 0x7fcf3016d5a0 2023-12-26T23:12:19.784 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.784+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcf30154420 con 0x7fcf3016d5a0 2023-12-26T23:12:19.785 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.784+0000 7fcf25ffb700 1 -- 172.21.15.131:0/2522435780 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v206) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcf30154420 con 0x7fcf3016d5a0 2023-12-26T23:12:19.785 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:19.785 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T23:12:19.785 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:19.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2023-12-26T23:12:19.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2023-12-26T23:12:19.785 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:19.785 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcf14066f30 msgr2=0x7fcf140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcf14066f30 0x7fcf140693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0079e0 tx=0x7fcf2c009040 comp rx=0 tx=0).stop 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3016d5a0 msgr2=0x7fcf3016aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3016d5a0 0x7fcf3016aa40 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fcf20006ac0 tx=0x7fcf200091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 shutdown_connections 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fcf14066f30 0x7fcf140693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3016af80 0x7fcf3015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf1c004d50 0x7fcf300ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 --2- 172.21.15.131:0/2522435780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf3016d5a0 0x7fcf3016aa40 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 >> 172.21.15.131:0/2522435780 conn(0x7fcf3006ece0 msgr2=0x7fcf300b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 shutdown_connections 2023-12-26T23:12:19.787 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.786+0000 7fcf37dff700 1 -- 172.21.15.131:0/2522435780 wait complete. 2023-12-26T23:12:19.804 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-26T23:12:19.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-26T23:12:19.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.998+0000 7f49af5ff700 1 Processor -- start 2023-12-26T23:12:19.999 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:19.999+0000 7f49af5ff700 1 -- start start 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.000+0000 7f49af5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49a81593f0 0x7f49a80abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.000+0000 7f49af5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 0x7f49a806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.000+0000 7f49af5ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a8067ff0 con 0x7f49a81597c0 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.000+0000 7f49af5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a8068480 con 0x7f49a81593f0 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.000+0000 7f49af5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a806a820 con 0x7f49a8069e60 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.000+0000 7f49ad39b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49a81593f0 0x7f49a80abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.000+0000 7f49acb9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f49a81597c0 0x7f49a80aa270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52514/0 (socket says 172.21.15.131:52514) 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.000+0000 7f49acb9a700 1 -- 172.21.15.131:0/2867270401 learned_addr learned my addr 172.21.15.131:0/2867270401 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.000+0000 7f49adb9c700 1 --2- 172.21.15.131:0/2867270401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 0x7f49a806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.001+0000 7f49ad39b700 1 -- 172.21.15.131:0/2867270401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 msgr2=0x7f49a806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.001+0000 7f49ad39b700 1 --2- 172.21.15.131:0/2867270401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 0x7f49a806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.001+0000 7f49ad39b700 1 -- 172.21.15.131:0/2867270401 >> v1:172.21.15.131:6789/0 conn(0x7f49a81597c0 legacy=0x7f49a80aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.001+0000 7f49ad39b700 1 -- 172.21.15.131:0/2867270401 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80ac350 con 0x7f49a81593f0 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.001+0000 7f499ffff700 1 -- 172.21.15.131:0/2867270401 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075221313 0 0) 0x7f49a8067ff0 con 0x7f49a81597c0 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.001+0000 7f49ad39b700 1 --2- 172.21.15.131:0/2867270401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81593f0 0x7f49a80abc90 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f49a40098e0 tx=0x7f49a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89392743e9c6ba3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.001+0000 7f499ffff700 1 -- 172.21.15.131:0/2867270401 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4009e80 con 0x7f49a81593f0 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.001+0000 7f499ffff700 1 -- 172.21.15.131:0/2867270401 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a400b860 con 0x7f49a81593f0 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.001+0000 7f499ffff700 1 -- 172.21.15.131:0/2867270401 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a400bbe0 con 0x7f49a81593f0 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.002+0000 7f49af5ff700 1 -- 172.21.15.131:0/2867270401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81593f0 msgr2=0x7f49a80abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.002+0000 7f49af5ff700 1 --2- 172.21.15.131:0/2867270401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81593f0 0x7f49a80abc90 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f49a40098e0 tx=0x7f49a40091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.002+0000 7f49af5ff700 1 -- 172.21.15.131:0/2867270401 shutdown_connections 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.002+0000 7f49af5ff700 1 --2- 172.21.15.131:0/2867270401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 0x7f49a806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.002+0000 7f49af5ff700 1 --2- 172.21.15.131:0/2867270401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81593f0 0x7f49a80abc90 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.002+0000 7f49af5ff700 1 -- 172.21.15.131:0/2867270401 >> 172.21.15.131:0/2867270401 conn(0x7f49a806b710 msgr2=0x7f49a806bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.002+0000 7f49af5ff700 1 -- 172.21.15.131:0/2867270401 shutdown_connections 2023-12-26T23:12:20.003 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.002+0000 7f49af5ff700 1 -- 172.21.15.131:0/2867270401 wait complete. 2023-12-26T23:12:20.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.004+0000 7f49af5ff700 1 Processor -- start 2023-12-26T23:12:20.004 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.004+0000 7f49af5ff700 1 -- start start 2023-12-26T23:12:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.005+0000 7f49af5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 0x7f49a81531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.005+0000 7f49af5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49a81593f0 0x7f49a80ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.005+0000 7f49af5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81597c0 0x7f49a8159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.005+0000 7f49af5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a815b970 con 0x7f49a81597c0 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.005+0000 7f49af5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a815a690 con 0x7f49a81593f0 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.005+0000 7f49af5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a815a970 con 0x7f49a8069e60 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.005+0000 7f49adb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81597c0 0x7f49a8159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49adb9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81597c0 0x7f49a8159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60150/0 (socket says 172.21.15.131:60150) 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49adb9c700 1 -- 172.21.15.131:0/2315421097 learned_addr learned my addr 172.21.15.131:0/2315421097 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49ad39b700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 0x7f49a81531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49acb9a700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49a81593f0 0x7f49a80ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49adb9c700 1 -- 172.21.15.131:0/2315421097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 msgr2=0x7f49a81531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49adb9c700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 0x7f49a81531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49adb9c700 1 -- 172.21.15.131:0/2315421097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49a81593f0 msgr2=0x7f49a80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49adb9c700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49a81593f0 0x7f49a80ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49adb9c700 1 -- 172.21.15.131:0/2315421097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4009420 con 0x7f49a81597c0 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49acb9a700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49a81593f0 0x7f49a80ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49adb9c700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81597c0 0x7f49a8159cb0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fa0 tx=0x7f49a0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f499d7fa700 1 -- 172.21.15.131:0/2315421097 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a000ec70 con 0x7f49a81597c0 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f499d7fa700 1 -- 172.21.15.131:0/2315421097 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a000edd0 con 0x7f49a81597c0 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f499d7fa700 1 -- 172.21.15.131:0/2315421097 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0014670 con 0x7f49a81597c0 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.006+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a815ac50 con 0x7f49a81597c0 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.007+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a816a120 con 0x7f49a81597c0 2023-12-26T23:12:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.008+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a80717f0 con 0x7f49a81597c0 2023-12-26T23:12:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.008+0000 7f499d7fa700 1 -- 172.21.15.131:0/2315421097 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f49a00149e0 con 0x7f49a81597c0 2023-12-26T23:12:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.009+0000 7f499d7fa700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f498c066f30 0x7f498c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.009+0000 7f499d7fa700 1 -- 172.21.15.131:0/2315421097 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f49a0084b20 con 0x7f49a81597c0 2023-12-26T23:12:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.010+0000 7f49ad39b700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f498c066f30 0x7f498c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.010+0000 7f49ad39b700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f498c066f30 0x7f498c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49a40091e0 tx=0x7f49a4017000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:20.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.014+0000 7f499d7fa700 1 -- 172.21.15.131:0/2315421097 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49a0051ee0 con 0x7f49a81597c0 2023-12-26T23:12:20.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.207+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f49a80af8c0 con 0x7f49a81597c0 2023-12-26T23:12:20.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.569+0000 7f499d7fa700 1 -- 172.21.15.131:0/2315421097 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v207) v1 ==== 138+0+0 (secure 0 0 0) 0x7f49a0055500 con 0x7f49a81597c0 2023-12-26T23:12:20.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.574+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f498c066f30 msgr2=0x7f498c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.574+0000 7f49af5ff700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f498c066f30 0x7f498c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49a40091e0 tx=0x7f49a4017000 comp rx=0 tx=0).stop 2023-12-26T23:12:20.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.574+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81597c0 msgr2=0x7f49a8159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.574+0000 7f49af5ff700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81597c0 0x7f49a8159cb0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fa0 tx=0x7f49a0009900 comp rx=0 tx=0).stop 2023-12-26T23:12:20.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.574+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 shutdown_connections 2023-12-26T23:12:20.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.574+0000 7f49af5ff700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f498c066f30 0x7f498c0693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.574+0000 7f49af5ff700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49a81597c0 0x7f49a8159cb0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.574+0000 7f49af5ff700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49a8069e60 0x7f49a81531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.574+0000 7f49af5ff700 1 --2- 172.21.15.131:0/2315421097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49a81593f0 0x7f49a80ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.575 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.575+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 >> 172.21.15.131:0/2315421097 conn(0x7f49a806b710 msgr2=0x7f49a8151530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:20.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.575+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 shutdown_connections 2023-12-26T23:12:20.576 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.576+0000 7f49af5ff700 1 -- 172.21.15.131:0/2315421097 wait complete. 2023-12-26T23:12:20.577 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2023-12-26T23:12:20.594 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-26T23:12:20.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.791+0000 7f68647ff700 1 Processor -- start 2023-12-26T23:12:20.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.791+0000 7f68647ff700 1 -- start start 2023-12-26T23:12:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.792+0000 7f68647ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f685c157050 0x7f685c0686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.792+0000 7f68647ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c068c00 0x7f685c0cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.792+0000 7f68647ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c0caed0 con 0x7f685c157420 2023-12-26T23:12:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.792+0000 7f68647ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0c9520 con 0x7f685c157050 2023-12-26T23:12:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.792+0000 7f68647ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0ca8e0 con 0x7f685c068c00 2023-12-26T23:12:20.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.792+0000 7f6861d9a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f685c157420 0x7f685c159c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52518/0 (socket says 172.21.15.131:52518) 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861d9a700 1 -- 172.21.15.131:0/6243185 learned_addr learned my addr 172.21.15.131:0/6243185 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f686259b700 1 --2- 172.21.15.131:0/6243185 >> v2:172.21.15.190:3300/0 conn(0x7f685c157050 0x7f685c0686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6862d9c700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c068c00 0x7f685c0cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381452910 0 0) 0x7f685c0caed0 con 0x7f685c157420 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6848003570 con 0x7f685c157420 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1248912112 0 0) 0x7f6848003570 con 0x7f685c157420 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c068c00 msgr2=0x7f685c0cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c068c00 0x7f685c0cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 >> v2:172.21.15.190:3300/0 conn(0x7f685c157050 msgr2=0x7f685c0686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 --2- 172.21.15.131:0/6243185 >> v2:172.21.15.190:3300/0 conn(0x7f685c157050 0x7f685c0686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c15a6c0 con 0x7f685c157420 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f684c001470 con 0x7f685c157420 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 >> v1:172.21.15.131:6789/0 conn(0x7f685c157420 legacy=0x7f685c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6848003a60 0x7f6848003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848004810 0x7f6848004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c068c00 0x7f68480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c9520 con 0x7f6848003a60 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0ca8e0 con 0x7f6848004810 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.793+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684800ba60 con 0x7f685c068c00 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.794+0000 7f686259b700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6848003a60 0x7f6848003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.794+0000 7f6861d9a700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848004810 0x7f6848004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.794+0000 7f6862d9c700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c068c00 0x7f68480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.794+0000 7f686259b700 1 -- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c068c00 msgr2=0x7f68480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.794+0000 7f686259b700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c068c00 0x7f68480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.794+0000 7f686259b700 1 -- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848004810 msgr2=0x7f6848004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.794+0000 7f686259b700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848004810 0x7f6848004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.794+0000 7f686259b700 1 -- 172.21.15.131:0/6243185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848003570 con 0x7f6848003a60 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.794+0000 7f686259b700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6848003a60 0x7f6848003e70 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f6858009c30 tx=0x7f68580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616e33958e4215e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800e710 con 0x7f6848003a60 2023-12-26T23:12:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685800e870 con 0x7f6848003a60 2023-12-26T23:12:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f6861599700 1 -- 172.21.15.131:0/6243185 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800ec30 con 0x7f6848003a60 2023-12-26T23:12:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f68647ff700 1 -- 172.21.15.131:0/6243185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6848003a60 msgr2=0x7f6848003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f68647ff700 1 --2- 172.21.15.131:0/6243185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6848003a60 0x7f6848003e70 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f6858009c30 tx=0x7f68580091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f6862d9c700 1 -- 172.21.15.131:0/6243185 reap_dead start 2023-12-26T23:12:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f68647ff700 1 -- 172.21.15.131:0/6243185 shutdown_connections 2023-12-26T23:12:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f68647ff700 1 -- 172.21.15.131:0/6243185 >> 172.21.15.131:0/6243185 conn(0x7f685c06dc80 msgr2=0x7f685c0675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f68647ff700 1 -- 172.21.15.131:0/6243185 shutdown_connections 2023-12-26T23:12:20.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.795+0000 7f68647ff700 1 -- 172.21.15.131:0/6243185 wait complete. 2023-12-26T23:12:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.797+0000 7f68647ff700 1 Processor -- start 2023-12-26T23:12:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.797+0000 7f68647ff700 1 -- start start 2023-12-26T23:12:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f68647ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848003a60 0x7f685c0b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f68647ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c145a30 0x7f685c0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f68647ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0c63e0 0x7f685c0c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.800 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f68647ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c7d20 con 0x7f685c0c63e0 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f68647ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c7110 con 0x7f6848003a60 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f68647ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c73f0 con 0x7f685c145a30 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f6862d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0c63e0 0x7f685c0c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f6862d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0c63e0 0x7f685c0c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60178/0 (socket says 172.21.15.131:60178) 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f6862d9c700 1 -- 172.21.15.131:0/649050215 learned_addr learned my addr 172.21.15.131:0/649050215 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f6861d9a700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c145a30 0x7f685c0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.798+0000 7f686259b700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848003a60 0x7f685c0b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f6862d9c700 1 -- 172.21.15.131:0/649050215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c145a30 msgr2=0x7f685c0b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f6862d9c700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c145a30 0x7f685c0b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f6862d9c700 1 -- 172.21.15.131:0/649050215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848003a60 msgr2=0x7f685c0b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f6862d9c700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848003a60 0x7f685c0b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f6862d9c700 1 -- 172.21.15.131:0/649050215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0c76d0 con 0x7f685c0c63e0 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f6862d9c700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0c63e0 0x7f685c0c6850 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f68540094d0 tx=0x7f685400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:20.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f68527fc700 1 -- 172.21.15.131:0/649050215 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400e040 con 0x7f685c0c63e0 2023-12-26T23:12:20.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f68527fc700 1 -- 172.21.15.131:0/649050215 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685400f040 con 0x7f685c0c63e0 2023-12-26T23:12:20.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f68527fc700 1 -- 172.21.15.131:0/649050215 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400fd70 con 0x7f685c0c63e0 2023-12-26T23:12:20.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0c79b0 con 0x7f685c0c63e0 2023-12-26T23:12:20.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.799+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0b3160 con 0x7f685c0c63e0 2023-12-26T23:12:20.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.800+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c0c7b40 con 0x7f685c0c63e0 2023-12-26T23:12:20.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.801+0000 7f68527fc700 1 -- 172.21.15.131:0/649050215 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6854015640 con 0x7f685c0c63e0 2023-12-26T23:12:20.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.802+0000 7f68527fc700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6840066f30 0x7f68400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:20.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.802+0000 7f68527fc700 1 -- 172.21.15.131:0/649050215 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5743+0+0 (secure 0 0 0) 0x7f68540851d0 con 0x7f685c0c63e0 2023-12-26T23:12:20.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.802+0000 7f686259b700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6840066f30 0x7f68400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:20.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.803+0000 7f686259b700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6840066f30 0x7f68400693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68580091e0 tx=0x7f685801a000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:20.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:20.806+0000 7f68527fc700 1 -- 172.21.15.131:0/649050215 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68540524a0 con 0x7f685c0c63e0 2023-12-26T23:12:21.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.000+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f685c0c7ed0 con 0x7f685c0c63e0 2023-12-26T23:12:21.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.576+0000 7f68527fc700 1 -- 172.21.15.131:0/649050215 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v208) v1 ==== 170+0+0 (secure 0 0 0) 0x7f685c0c7ed0 con 0x7f685c0c63e0 2023-12-26T23:12:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6840066f30 msgr2=0x7f68400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6840066f30 0x7f68400693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68580091e0 tx=0x7f685801a000 comp rx=0 tx=0).stop 2023-12-26T23:12:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0c63e0 msgr2=0x7f685c0c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0c63e0 0x7f685c0c6850 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f68540094d0 tx=0x7f685400b040 comp rx=0 tx=0).stop 2023-12-26T23:12:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 shutdown_connections 2023-12-26T23:12:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6840066f30 0x7f68400693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:21.584 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c0c63e0 0x7f685c0c6850 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:21.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848003a60 0x7f685c0b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:21.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 --2- 172.21.15.131:0/649050215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f685c145a30 0x7f685c0b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:21.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.583+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 >> 172.21.15.131:0/649050215 conn(0x7f685c06dc80 msgr2=0x7f685c0cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:21.585 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.585+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 shutdown_connections 2023-12-26T23:12:21.586 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.585+0000 7f68647ff700 1 -- 172.21.15.131:0/649050215 wait complete. 2023-12-26T23:12:21.596 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2023-12-26T23:12:21.617 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-26T23:12:21.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.812+0000 7f9c08fff700 1 Processor -- start 2023-12-26T23:12:21.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.812+0000 7f9c08fff700 1 -- start start 2023-12-26T23:12:21.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.813+0000 7f9c08fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9c040cb320 0x7f9c04069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.813+0000 7f9c08fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 0x7f9c040c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.813+0000 7f9c08fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c04156940 con 0x7f9c0406a1e0 2023-12-26T23:12:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.813+0000 7f9c08fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c040c8cc0 con 0x7f9c040cb320 2023-12-26T23:12:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.813+0000 7f9c08fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040ad700 con 0x7f9c040b0e60 2023-12-26T23:12:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.813+0000 7f9c01d9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9c0406a1e0 0x7f9c040b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52532/0 (socket says 172.21.15.131:52532) 2023-12-26T23:12:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.813+0000 7f9c01d9b700 1 -- 172.21.15.131:0/2625884617 learned_addr learned my addr 172.21.15.131:0/2625884617 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c0259c700 1 --2- 172.21.15.131:0/2625884617 >> v2:172.21.15.190:3300/0 conn(0x7f9c040cb320 0x7f9c04069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:21.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c02d9d700 1 --2- 172.21.15.131:0/2625884617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 0x7f9c040c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c0159a700 1 -- 172.21.15.131:0/2625884617 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417911825 0 0) 0x7f9c04156940 con 0x7f9c0406a1e0 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c0159a700 1 -- 172.21.15.131:0/2625884617 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bec003570 con 0x7f9c0406a1e0 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c0259c700 1 -- 172.21.15.131:0/2625884617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 msgr2=0x7f9c040c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c0259c700 1 --2- 172.21.15.131:0/2625884617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 0x7f9c040c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c0259c700 1 -- 172.21.15.131:0/2625884617 >> v1:172.21.15.131:6789/0 conn(0x7f9c0406a1e0 legacy=0x7f9c040b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c0259c700 1 -- 172.21.15.131:0/2625884617 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040ae500 con 0x7f9c040cb320 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c02d9d700 1 --2- 172.21.15.131:0/2625884617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 0x7f9c040c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.814+0000 7f9c0259c700 1 --2- 172.21.15.131:0/2625884617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c040cb320 0x7f9c04069ca0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400bb90 tx=0x7f9bf400b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2810e3383bba09 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c0159a700 1 -- 172.21.15.131:0/2625884617 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400da70 con 0x7f9c040cb320 2023-12-26T23:12:21.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c0159a700 1 -- 172.21.15.131:0/2625884617 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf400dbd0 con 0x7f9c040cb320 2023-12-26T23:12:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c0159a700 1 -- 172.21.15.131:0/2625884617 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4019480 con 0x7f9c040cb320 2023-12-26T23:12:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c08fff700 1 -- 172.21.15.131:0/2625884617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c040cb320 msgr2=0x7f9c04069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c08fff700 1 --2- 172.21.15.131:0/2625884617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c040cb320 0x7f9c04069ca0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400bb90 tx=0x7f9bf400b430 comp rx=0 tx=0).stop 2023-12-26T23:12:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c08fff700 1 -- 172.21.15.131:0/2625884617 shutdown_connections 2023-12-26T23:12:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c08fff700 1 --2- 172.21.15.131:0/2625884617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 0x7f9c040c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c08fff700 1 --2- 172.21.15.131:0/2625884617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c040cb320 0x7f9c04069ca0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c08fff700 1 -- 172.21.15.131:0/2625884617 >> 172.21.15.131:0/2625884617 conn(0x7f9c04068040 msgr2=0x7f9c04069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c08fff700 1 -- 172.21.15.131:0/2625884617 shutdown_connections 2023-12-26T23:12:21.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.815+0000 7f9c08fff700 1 -- 172.21.15.131:0/2625884617 wait complete. 2023-12-26T23:12:21.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.817+0000 7f9c08fff700 1 Processor -- start 2023-12-26T23:12:21.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.817+0000 7f9c08fff700 1 -- start start 2023-12-26T23:12:21.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c08fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c0406a1e0 0x7f9c04144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:21.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c08fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 0x7f9c04158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:21.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c08fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c040cb320 0x7f9c04158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:21.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c08fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04152770 con 0x7f9c040cb320 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c08fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040c1480 con 0x7f9c0406a1e0 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c08fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041451e0 con 0x7f9c040b0e60 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c02d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c040cb320 0x7f9c04158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c02d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c040cb320 0x7f9c04158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60186/0 (socket says 172.21.15.131:60186) 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c02d9d700 1 -- 172.21.15.131:0/1211300579 learned_addr learned my addr 172.21.15.131:0/1211300579 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c0259c700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c0406a1e0 0x7f9c04144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.818+0000 7f9c01d9b700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 0x7f9c04158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9c02d9d700 1 -- 172.21.15.131:0/1211300579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 msgr2=0x7f9c04158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9c02d9d700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 0x7f9c04158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:21.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9c02d9d700 1 -- 172.21.15.131:0/1211300579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c0406a1e0 msgr2=0x7f9c04144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:21.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9c02d9d700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c0406a1e0 0x7f9c04144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:21.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9c02d9d700 1 -- 172.21.15.131:0/1211300579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc009a20 con 0x7f9c040cb320 2023-12-26T23:12:21.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9c02d9d700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c040cb320 0x7f9c04158a80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009f30 tx=0x7f9bfc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:21.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9bf27fc700 1 -- 172.21.15.131:0/1211300579 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00cb70 con 0x7f9c040cb320 2023-12-26T23:12:21.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9bf27fc700 1 -- 172.21.15.131:0/1211300579 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc00f040 con 0x7f9c040cb320 2023-12-26T23:12:21.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf400b6d0 con 0x7f9c040cb320 2023-12-26T23:12:21.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9bf27fc700 1 -- 172.21.15.131:0/1211300579 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc014480 con 0x7f9c040cb320 2023-12-26T23:12:21.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.819+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c041577d0 con 0x7f9c040cb320 2023-12-26T23:12:21.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.820+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c041579f0 con 0x7f9c040cb320 2023-12-26T23:12:21.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.821+0000 7f9bf27fc700 1 -- 172.21.15.131:0/1211300579 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9bfc0147f0 con 0x7f9c040cb320 2023-12-26T23:12:21.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.822+0000 7f9bf27fc700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9be4066f30 0x7f9be40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:21.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.822+0000 7f9bf27fc700 1 -- 172.21.15.131:0/1211300579 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f9bfc085990 con 0x7f9c040cb320 2023-12-26T23:12:21.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.823+0000 7f9c0259c700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9be4066f30 0x7f9be40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:21.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.823+0000 7f9c0259c700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9be4066f30 0x7f9be40693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400b470 tx=0x7f9bf401b000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:21.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:21.826+0000 7f9bf27fc700 1 -- 172.21.15.131:0/1211300579 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bfc052a40 con 0x7f9c040cb320 2023-12-26T23:12:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.062+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9c040657e0 con 0x7f9c040cb320 2023-12-26T23:12:22.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.595+0000 7f9bf27fc700 1 -- 172.21.15.131:0/1211300579 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v94) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9bfc056060 con 0x7f9c040cb320 2023-12-26T23:12:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.603+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9be4066f30 msgr2=0x7f9be40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.603+0000 7f9c08fff700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9be4066f30 0x7f9be40693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400b470 tx=0x7f9bf401b000 comp rx=0 tx=0).stop 2023-12-26T23:12:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.603+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c040cb320 msgr2=0x7f9c04158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:22.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.603+0000 7f9c08fff700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c040cb320 0x7f9c04158a80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009f30 tx=0x7f9bfc00b040 comp rx=0 tx=0).stop 2023-12-26T23:12:22.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.605+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 shutdown_connections 2023-12-26T23:12:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.605+0000 7f9c08fff700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9be4066f30 0x7f9be40693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.605+0000 7f9c08fff700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c040cb320 0x7f9c04158a80 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.605+0000 7f9c08fff700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c0406a1e0 0x7f9c04144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.605+0000 7f9c08fff700 1 --2- 172.21.15.131:0/1211300579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c040b0e60 0x7f9c04158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.605+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 >> 172.21.15.131:0/1211300579 conn(0x7f9c04068040 msgr2=0x7f9c040c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.605+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 shutdown_connections 2023-12-26T23:12:22.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.605+0000 7f9c08fff700 1 -- 172.21.15.131:0/1211300579 wait complete. 2023-12-26T23:12:22.612 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 24 and data pool 25 2023-12-26T23:12:22.635 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:22.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.826+0000 7f34455ff700 1 Processor -- start 2023-12-26T23:12:22.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.827+0000 7f34455ff700 1 -- start start 2023-12-26T23:12:22.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f34455ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8820 0x7f34400ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:22.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f34455ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34400b19f0 0x7f3440161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:22.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f34455ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3440157a80 con 0x7f34400c8bf0 2023-12-26T23:12:22.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f34455ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34400cac50 con 0x7f34400b19f0 2023-12-26T23:12:22.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f34455ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34400caf30 con 0x7f34400c8820 2023-12-26T23:12:22.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f343effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8820 0x7f34400ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:22.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f343f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34400b19f0 0x7f3440161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:22.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f343effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8820 0x7f34400ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:48728/0 (socket says 172.21.15.131:48728) 2023-12-26T23:12:22.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f343f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34400b19f0 0x7f3440161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:36184/0 (socket says 172.21.15.131:36184) 2023-12-26T23:12:22.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f343effd700 1 -- 172.21.15.131:0/954748068 learned_addr learned my addr 172.21.15.131:0/954748068 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:22.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.828+0000 7f343e7fc700 1 --1- 172.21.15.131:0/954748068 >> v1:172.21.15.190:6789/0 conn(0x7f34400c8bf0 0x7f34400b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:44916/0 (socket says 172.21.15.131:44916) 2023-12-26T23:12:22.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.829+0000 7f343effd700 1 -- 172.21.15.131:0/954748068 >> v2:172.21.15.190:3300/0 conn(0x7f34400b19f0 msgr2=0x7f3440161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:22.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.829+0000 7f343effd700 1 --2- 172.21.15.131:0/954748068 >> v2:172.21.15.190:3300/0 conn(0x7f34400b19f0 0x7f3440161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:22.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.829+0000 7f343effd700 1 -- 172.21.15.131:0/954748068 >> v1:172.21.15.190:6789/0 conn(0x7f34400c8bf0 legacy=0x7f34400b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:22.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.829+0000 7f343effd700 1 -- 172.21.15.131:0/954748068 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34400cb210 con 0x7f34400c8820 2023-12-26T23:12:22.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.829+0000 7f343dffb700 1 -- 172.21.15.131:0/954748068 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533866451 0 0) 0x7f3440157a80 con 0x7f34400c8bf0 2023-12-26T23:12:22.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.829+0000 7f343f7fe700 1 --2- 172.21.15.131:0/954748068 >> v2:172.21.15.190:3300/0 conn(0x7f34400b19f0 0x7f3440161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.829+0000 7f343effd700 1 --2- 172.21.15.131:0/954748068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8820 0x7f34400ca300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f34300099b0 tx=0x7f3430009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2387c9cb4183dd6c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.829+0000 7f343dffb700 1 -- 172.21.15.131:0/954748068 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430009ea0 con 0x7f34400c8820 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.829+0000 7f343dffb700 1 -- 172.21.15.131:0/954748068 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343000b860 con 0x7f34400c8820 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.830+0000 7f343dffb700 1 -- 172.21.15.131:0/954748068 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000bc00 con 0x7f34400c8820 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.830+0000 7f34455ff700 1 -- 172.21.15.131:0/954748068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8820 msgr2=0x7f34400ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.830+0000 7f34455ff700 1 --2- 172.21.15.131:0/954748068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8820 0x7f34400ca300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f34300099b0 tx=0x7f3430009250 comp rx=0 tx=0).stop 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.830+0000 7f34455ff700 1 -- 172.21.15.131:0/954748068 shutdown_connections 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.830+0000 7f34455ff700 1 --2- 172.21.15.131:0/954748068 >> v2:172.21.15.190:3300/0 conn(0x7f34400b19f0 0x7f3440161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.830+0000 7f34455ff700 1 --2- 172.21.15.131:0/954748068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8820 0x7f34400ca300 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.830+0000 7f34455ff700 1 -- 172.21.15.131:0/954748068 >> 172.21.15.131:0/954748068 conn(0x7f3440068040 msgr2=0x7f3440069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:22.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.830+0000 7f34455ff700 1 -- 172.21.15.131:0/954748068 shutdown_connections 2023-12-26T23:12:22.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.830+0000 7f34455ff700 1 -- 172.21.15.131:0/954748068 wait complete. 2023-12-26T23:12:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.832+0000 7f34455ff700 1 Processor -- start 2023-12-26T23:12:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.832+0000 7f34455ff700 1 -- start start 2023-12-26T23:12:22.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f34455ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34400b19f0 0x7f344014d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f34455ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34400c8820 0x7f34400c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f34455ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8bf0 0x7f3440166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f34455ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34401539a0 con 0x7f34400b19f0 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f34455ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344014d690 con 0x7f34400c8820 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f34455ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3440152d90 con 0x7f34400c8bf0 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34400b19f0 0x7f344014d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34400b19f0 0x7f344014d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60188/0 (socket says 172.21.15.131:60188) 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343effd700 1 -- 172.21.15.131:0/1628219707 learned_addr learned my addr 172.21.15.131:0/1628219707 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343e7fc700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34400c8820 0x7f34400c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343f7fe700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8bf0 0x7f3440166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343effd700 1 -- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8bf0 msgr2=0x7f3440166250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343effd700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8bf0 0x7f3440166250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343effd700 1 -- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34400c8820 msgr2=0x7f34400c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343effd700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34400c8820 0x7f34400c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343effd700 1 -- 172.21.15.131:0/1628219707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3438009a20 con 0x7f34400b19f0 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343e7fc700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34400c8820 0x7f34400c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.833+0000 7f343f7fe700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8bf0 0x7f3440166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.834+0000 7f343effd700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34400b19f0 0x7f344014d150 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f3430006fd0 tx=0x7f3430017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.834+0000 7f3426ffd700 1 -- 172.21.15.131:0/1628219707 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000b9d0 con 0x7f34400b19f0 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.834+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34300094f0 con 0x7f34400b19f0 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.834+0000 7f3426ffd700 1 -- 172.21.15.131:0/1628219707 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343000bb30 con 0x7f34400b19f0 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.834+0000 7f3426ffd700 1 -- 172.21.15.131:0/1628219707 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430025670 con 0x7f34400b19f0 2023-12-26T23:12:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.834+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3440153350 con 0x7f34400b19f0 2023-12-26T23:12:22.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.835+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34400657e0 con 0x7f34400b19f0 2023-12-26T23:12:22.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.836+0000 7f3426ffd700 1 -- 172.21.15.131:0/1628219707 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3430023070 con 0x7f34400b19f0 2023-12-26T23:12:22.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.837+0000 7f3426ffd700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f342c066f30 0x7f342c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.837+0000 7f3426ffd700 1 -- 172.21.15.131:0/1628219707 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f3430085eb0 con 0x7f34400b19f0 2023-12-26T23:12:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.837+0000 7f343e7fc700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f342c066f30 0x7f342c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.838+0000 7f343e7fc700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f342c066f30 0x7f342c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f344014d8f0 tx=0x7f34340016c0 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:22.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:22.841+0000 7f3426ffd700 1 -- 172.21.15.131:0/1628219707 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3430052e70 con 0x7f34400b19f0 2023-12-26T23:12:23.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.036+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34400cb4f0 con 0x7f34400b19f0 2023-12-26T23:12:23.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.037+0000 7f3426ffd700 1 -- 172.21.15.131:0/1628219707 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+16228 (secure 0 0 0) 0x7f3430015090 con 0x7f34400b19f0 2023-12-26T23:12:23.037 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:23.037 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":209,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:22.064239+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:12:20.250102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-26T23:12:21.042553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:23.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f342c066f30 msgr2=0x7f342c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f342c066f30 0x7f342c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f344014d8f0 tx=0x7f34340016c0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34400b19f0 msgr2=0x7f344014d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34400b19f0 0x7f344014d150 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f3430006fd0 tx=0x7f3430017000 comp rx=0 tx=0).stop 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 shutdown_connections 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f342c066f30 0x7f342c0693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34400c8bf0 0x7f3440166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34400b19f0 0x7f344014d150 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 --2- 172.21.15.131:0/1628219707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34400c8820 0x7f34400c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 >> 172.21.15.131:0/1628219707 conn(0x7f3440068040 msgr2=0x7f3440156540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 shutdown_connections 2023-12-26T23:12:23.040 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.039+0000 7f34455ff700 1 -- 172.21.15.131:0/1628219707 wait complete. 2023-12-26T23:12:23.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:23.247 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.247+0000 7f39a4fff700 1 Processor -- start 2023-12-26T23:12:23.248 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.247+0000 7f39a4fff700 1 -- start start 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.248+0000 7f39a4fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f39a00cb320 0x7f39a0069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.248+0000 7f39a4fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a00c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.248+0000 7f39a4fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a0156940 con 0x7f39a006a1e0 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.248+0000 7f39a4fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a00c8cc0 con 0x7f39a00cb320 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.248+0000 7f39a4fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00ad700 con 0x7f39a00b0e60 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.248+0000 7f399e59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f39a00cb320 0x7f39a0069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.248+0000 7f399ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a00c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.249+0000 7f399ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a00c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:48750/0 (socket says 172.21.15.131:48750) 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.249+0000 7f399e59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f39a00cb320 0x7f39a0069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:36206/0 (socket says 172.21.15.131:36206) 2023-12-26T23:12:23.251 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.249+0000 7f399ed9d700 1 -- 172.21.15.131:0/1227620043 learned_addr learned my addr 172.21.15.131:0/1227620043 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.249+0000 7f399ed9d700 1 -- 172.21.15.131:0/1227620043 >> v2:172.21.15.190:3300/0 conn(0x7f39a00cb320 msgr2=0x7f39a0069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.249+0000 7f399ed9d700 1 --2- 172.21.15.131:0/1227620043 >> v2:172.21.15.190:3300/0 conn(0x7f39a00cb320 0x7f39a0069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.249+0000 7f399ed9d700 1 -- 172.21.15.131:0/1227620043 >> v1:172.21.15.190:6789/0 conn(0x7f39a006a1e0 legacy=0x7f39a00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.249+0000 7f399ed9d700 1 -- 172.21.15.131:0/1227620043 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00ae500 con 0x7f39a00b0e60 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.249+0000 7f399d59a700 1 -- 172.21.15.131:0/1227620043 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053764384 0 0) 0x7f39a0156940 con 0x7f39a006a1e0 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.249+0000 7f399ed9d700 1 --2- 172.21.15.131:0/1227620043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a00c9400 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3998009f20 tx=0x7f3998009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aadbb52ad139da3b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f399d59a700 1 -- 172.21.15.131:0/1227620043 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800e070 con 0x7f39a00b0e60 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f399d59a700 1 -- 172.21.15.131:0/1227620043 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800b9d0 con 0x7f39a00b0e60 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f399d59a700 1 -- 172.21.15.131:0/1227620043 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39980092e0 con 0x7f39a00b0e60 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f39a4fff700 1 -- 172.21.15.131:0/1227620043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 msgr2=0x7f39a00c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.252 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f39a4fff700 1 --2- 172.21.15.131:0/1227620043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a00c9400 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3998009f20 tx=0x7f3998009840 comp rx=0 tx=0).stop 2023-12-26T23:12:23.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f39a4fff700 1 -- 172.21.15.131:0/1227620043 shutdown_connections 2023-12-26T23:12:23.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f39a4fff700 1 --2- 172.21.15.131:0/1227620043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a00c9400 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f39a4fff700 1 --2- 172.21.15.131:0/1227620043 >> v2:172.21.15.190:3300/0 conn(0x7f39a00cb320 0x7f39a0069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f39a4fff700 1 -- 172.21.15.131:0/1227620043 >> 172.21.15.131:0/1227620043 conn(0x7f39a0068040 msgr2=0x7f39a0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:23.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f39a4fff700 1 -- 172.21.15.131:0/1227620043 shutdown_connections 2023-12-26T23:12:23.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.250+0000 7f39a4fff700 1 -- 172.21.15.131:0/1227620043 wait complete. 2023-12-26T23:12:23.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.252+0000 7f39a4fff700 1 Processor -- start 2023-12-26T23:12:23.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.252+0000 7f39a4fff700 1 -- start start 2023-12-26T23:12:23.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f39a4fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39a006a1e0 0x7f39a015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f39a4fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a0148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f39a4fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39a00cb320 0x7f39a0169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f39a4fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a01492b0 con 0x7f39a006a1e0 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f39a4fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a015d2c0 con 0x7f39a00cb320 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f39a4fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a016a4f0 con 0x7f39a00b0e60 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f399e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39a006a1e0 0x7f39a015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f399e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39a006a1e0 0x7f39a015bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60200/0 (socket says 172.21.15.131:60200) 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f399e59c700 1 -- 172.21.15.131:0/3097051675 learned_addr learned my addr 172.21.15.131:0/3097051675 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f399ed9d700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39a00cb320 0x7f39a0169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f399dd9b700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a0148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f399e59c700 1 -- 172.21.15.131:0/3097051675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 msgr2=0x7f39a0148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f399e59c700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a0148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f399e59c700 1 -- 172.21.15.131:0/3097051675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39a00cb320 msgr2=0x7f39a0169fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.253+0000 7f399e59c700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39a00cb320 0x7f39a0169fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.254+0000 7f399e59c700 1 -- 172.21.15.131:0/3097051675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998009ae0 con 0x7f39a006a1e0 2023-12-26T23:12:23.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.254+0000 7f399e59c700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39a006a1e0 0x7f39a015bd90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f399000e930 tx=0x7f39900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:23.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.254+0000 7f398e7fc700 1 -- 172.21.15.131:0/3097051675 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990013680 con 0x7f39a006a1e0 2023-12-26T23:12:23.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.254+0000 7f398e7fc700 1 -- 172.21.15.131:0/3097051675 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39900137e0 con 0x7f39a006a1e0 2023-12-26T23:12:23.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.254+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a015c370 con 0x7f39a006a1e0 2023-12-26T23:12:23.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.254+0000 7f398e7fc700 1 -- 172.21.15.131:0/3097051675 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990013b80 con 0x7f39a006a1e0 2023-12-26T23:12:23.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.254+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a015c930 con 0x7f39a006a1e0 2023-12-26T23:12:23.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.255+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a00657e0 con 0x7f39a006a1e0 2023-12-26T23:12:23.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.256+0000 7f398e7fc700 1 -- 172.21.15.131:0/3097051675 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f399000c3e0 con 0x7f39a006a1e0 2023-12-26T23:12:23.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.258+0000 7f398e7fc700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3980066f30 0x7f39800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.258+0000 7f398e7fc700 1 -- 172.21.15.131:0/3097051675 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f3990084830 con 0x7f39a006a1e0 2023-12-26T23:12:23.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.258+0000 7f399dd9b700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3980066f30 0x7f39800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.259+0000 7f399dd9b700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3980066f30 0x7f39800693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39a015b350 tx=0x7f3994007d30 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:23.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.262+0000 7f398e7fc700 1 -- 172.21.15.131:0/3097051675 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3990051890 con 0x7f39a006a1e0 2023-12-26T23:12:23.456 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.456+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39a015d5a0 con 0x7f39a006a1e0 2023-12-26T23:12:23.457 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.457+0000 7f398e7fc700 1 -- 172.21.15.131:0/3097051675 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+16228 (secure 0 0 0) 0x7f3990054eb0 con 0x7f39a006a1e0 2023-12-26T23:12:23.457 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:23.458 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":209,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:22.064239+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:12:20.250102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-26T23:12:21.042553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:23.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3980066f30 msgr2=0x7f39800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3980066f30 0x7f39800693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39a015b350 tx=0x7f3994007d30 comp rx=0 tx=0).stop 2023-12-26T23:12:23.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39a006a1e0 msgr2=0x7f39a015bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39a006a1e0 0x7f39a015bd90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f399000e930 tx=0x7f39900091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.460 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 shutdown_connections 2023-12-26T23:12:23.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3980066f30 0x7f39800693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39a00cb320 0x7f39a0169fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39a006a1e0 0x7f39a015bd90 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 --2- 172.21.15.131:0/3097051675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a00b0e60 0x7f39a0148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 >> 172.21.15.131:0/3097051675 conn(0x7f39a0068040 msgr2=0x7f39a006e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:23.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 shutdown_connections 2023-12-26T23:12:23.461 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.459+0000 7f39a4fff700 1 -- 172.21.15.131:0/3097051675 wait complete. 2023-12-26T23:12:23.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-26T23:12:23.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.675+0000 7feebcfff700 1 Processor -- start 2023-12-26T23:12:23.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.675+0000 7feebcfff700 1 -- start start 2023-12-26T23:12:23.678 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.676+0000 7feebcfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feeb80cb320 0x7feeb8069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.676+0000 7feebcfff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feeb806a1e0 0x7feeb80b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.676+0000 7feebcfff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb8156940 con 0x7feeb80b0c90 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.676+0000 7feebcfff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb80c8cc0 con 0x7feeb806a1e0 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.676+0000 7feebcfff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb80ad700 con 0x7feeb80cb320 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.677+0000 7feeb5d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feeb806a1e0 0x7feeb80b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.677+0000 7feeb5d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feeb806a1e0 0x7feeb80b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60202/0 (socket says 172.21.15.131:60202) 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.677+0000 7feeb5d9b700 1 -- 172.21.15.131:0/1860759421 learned_addr learned my addr 172.21.15.131:0/1860759421 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.677+0000 7feeb659c700 1 --2- 172.21.15.131:0/1860759421 >> v2:172.21.15.190:3300/0 conn(0x7feeb80cb320 0x7feeb8069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.677+0000 7feeb5d9b700 1 -- 172.21.15.131:0/1860759421 >> v2:172.21.15.190:3300/0 conn(0x7feeb80cb320 msgr2=0x7feeb8069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.677+0000 7feeb5d9b700 1 --2- 172.21.15.131:0/1860759421 >> v2:172.21.15.190:3300/0 conn(0x7feeb80cb320 0x7feeb8069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.677+0000 7feeb5d9b700 1 -- 172.21.15.131:0/1860759421 >> v1:172.21.15.190:6789/0 conn(0x7feeb80b0c90 legacy=0x7feeb80aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.677+0000 7feeb5d9b700 1 -- 172.21.15.131:0/1860759421 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80ae500 con 0x7feeb806a1e0 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.677+0000 7feeb5d9b700 1 --2- 172.21.15.131:0/1860759421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 0x7feeb80b0750 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7feeb8064410 tx=0x7feeac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0b1040c2856e4a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feeb559a700 1 -- 172.21.15.131:0/1860759421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac00b7e0 con 0x7feeb806a1e0 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feeb559a700 1 -- 172.21.15.131:0/1860759421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac00b940 con 0x7feeb806a1e0 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feeb559a700 1 -- 172.21.15.131:0/1860759421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac00bce0 con 0x7feeb806a1e0 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feebcfff700 1 -- 172.21.15.131:0/1860759421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 msgr2=0x7feeb80b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feebcfff700 1 --2- 172.21.15.131:0/1860759421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 0x7feeb80b0750 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7feeb8064410 tx=0x7feeac009250 comp rx=0 tx=0).stop 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feebcfff700 1 -- 172.21.15.131:0/1860759421 shutdown_connections 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feebcfff700 1 --2- 172.21.15.131:0/1860759421 >> v2:172.21.15.190:3300/0 conn(0x7feeb80cb320 0x7feeb8069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feebcfff700 1 --2- 172.21.15.131:0/1860759421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 0x7feeb80b0750 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feebcfff700 1 -- 172.21.15.131:0/1860759421 >> 172.21.15.131:0/1860759421 conn(0x7feeb8068040 msgr2=0x7feeb8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feebcfff700 1 -- 172.21.15.131:0/1860759421 shutdown_connections 2023-12-26T23:12:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.678+0000 7feebcfff700 1 -- 172.21.15.131:0/1860759421 wait complete. 2023-12-26T23:12:23.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.680+0000 7feebcfff700 1 Processor -- start 2023-12-26T23:12:23.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.680+0000 7feebcfff700 1 -- start start 2023-12-26T23:12:23.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feebcfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 0x7feeb815e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feebcfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb80b0c90 0x7feeb8169af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feebcfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb80cb320 0x7feeb80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feebcfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815a160 con 0x7feeb806a1e0 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feebcfff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815e860 con 0x7feeb80b0c90 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feebcfff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815eb40 con 0x7feeb80cb320 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feeb659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 0x7feeb815e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feeb659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 0x7feeb815e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60216/0 (socket says 172.21.15.131:60216) 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feeb659c700 1 -- 172.21.15.131:0/1464734828 learned_addr learned my addr 172.21.15.131:0/1464734828 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feeb659c700 1 -- 172.21.15.131:0/1464734828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb80cb320 msgr2=0x7feeb80abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feeb659c700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb80cb320 0x7feeb80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feeb659c700 1 -- 172.21.15.131:0/1464734828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb80b0c90 msgr2=0x7feeb8169af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feeb659c700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb80b0c90 0x7feeb8169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.681+0000 7feeb659c700 1 -- 172.21.15.131:0/1464734828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0094f0 con 0x7feeb806a1e0 2023-12-26T23:12:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.682+0000 7feeb659c700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 0x7feeb815e0f0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7feea800d860 tx=0x7feea8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.682+0000 7feea67fc700 1 -- 172.21.15.131:0/1464734828 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80125f0 con 0x7feeb806a1e0 2023-12-26T23:12:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.682+0000 7feea67fc700 1 -- 172.21.15.131:0/1464734828 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8012750 con 0x7feeb806a1e0 2023-12-26T23:12:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.682+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb815a470 con 0x7feeb806a1e0 2023-12-26T23:12:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.682+0000 7feea67fc700 1 -- 172.21.15.131:0/1464734828 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8012b10 con 0x7feeb806a1e0 2023-12-26T23:12:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.682+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb816a030 con 0x7feeb806a1e0 2023-12-26T23:12:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.683+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb815a910 con 0x7feeb806a1e0 2023-12-26T23:12:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.684+0000 7feea67fc700 1 -- 172.21.15.131:0/1464734828 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feea801e460 con 0x7feeb806a1e0 2023-12-26T23:12:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.685+0000 7feea67fc700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fee98066f30 0x7fee980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.685+0000 7feea67fc700 1 -- 172.21.15.131:0/1464734828 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7feea8084ea0 con 0x7feeb806a1e0 2023-12-26T23:12:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.685+0000 7feeb5d9b700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fee98066f30 0x7fee980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:23.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.686+0000 7feeb5d9b700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fee98066f30 0x7fee980693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feeb8064410 tx=0x7feeac016000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:23.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.689+0000 7feea67fc700 1 -- 172.21.15.131:0/1464734828 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feea80558f0 con 0x7feeb806a1e0 2023-12-26T23:12:23.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:23.923+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feeb80657e0 con 0x7feeb806a1e0 2023-12-26T23:12:24.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.606+0000 7feea67fc700 1 -- 172.21.15.131:0/1464734828 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v97) v1 ==== 118+0+0 (secure 0 0 0) 0x7feea8055480 con 0x7feeb806a1e0 2023-12-26T23:12:24.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.608+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fee98066f30 msgr2=0x7fee980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:24.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.608+0000 7feebcfff700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fee98066f30 0x7fee980693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feeb8064410 tx=0x7feeac016000 comp rx=0 tx=0).stop 2023-12-26T23:12:24.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.608+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 msgr2=0x7feeb815e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:24.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.608+0000 7feebcfff700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 0x7feeb815e0f0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7feea800d860 tx=0x7feea8008040 comp rx=0 tx=0).stop 2023-12-26T23:12:24.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.609+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 shutdown_connections 2023-12-26T23:12:24.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.609+0000 7feebcfff700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fee98066f30 0x7fee980693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:24.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.609+0000 7feebcfff700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeb80cb320 0x7feeb80abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:24.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.609+0000 7feebcfff700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feeb806a1e0 0x7feeb815e0f0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:24.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.609+0000 7feebcfff700 1 --2- 172.21.15.131:0/1464734828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeb80b0c90 0x7feeb8169af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:24.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.609+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 >> 172.21.15.131:0/1464734828 conn(0x7feeb8068040 msgr2=0x7feeb80cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:24.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.609+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 shutdown_connections 2023-12-26T23:12:24.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.609+0000 7feebcfff700 1 -- 172.21.15.131:0/1464734828 wait complete. 2023-12-26T23:12:24.628 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-26T23:12:24.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.821+0000 7f4fe93ff700 1 Processor -- start 2023-12-26T23:12:24.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.821+0000 7f4fe93ff700 1 -- start start 2023-12-26T23:12:24.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.822+0000 7f4fe93ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4fe41596a0 0x7f4fe40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:24.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.822+0000 7f4fe93ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 0x7f4fe40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.822+0000 7f4fe93ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe40afcd0 con 0x7f4fe40b0160 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.822+0000 7f4fe93ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40b3e70 con 0x7f4fe41596a0 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.822+0000 7f4fe93ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40b0cd0 con 0x7f4fe4159a70 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.822+0000 7f4fe2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4fe41596a0 0x7f4fe40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.822+0000 7f4fe2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4fe41596a0 0x7f4fe40b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60226/0 (socket says 172.21.15.131:60226) 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.822+0000 7f4fe2ffd700 1 -- 172.21.15.131:0/3195505625 learned_addr learned my addr 172.21.15.131:0/3195505625 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.823+0000 7f4fe27fc700 1 --2- 172.21.15.131:0/3195505625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 0x7f4fe40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.823+0000 7f4fe2ffd700 1 -- 172.21.15.131:0/3195505625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 msgr2=0x7f4fe40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.823+0000 7f4fe2ffd700 1 --2- 172.21.15.131:0/3195505625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 0x7f4fe40b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.823+0000 7f4fe2ffd700 1 -- 172.21.15.131:0/3195505625 >> v1:172.21.15.190:6789/0 conn(0x7f4fe40b0160 legacy=0x7f4fe40b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.823+0000 7f4fe2ffd700 1 -- 172.21.15.131:0/3195505625 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40b2e00 con 0x7f4fe41596a0 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.823+0000 7f4fe2ffd700 1 --2- 172.21.15.131:0/3195505625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 0x7f4fe40b2380 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40099b0 tx=0x7f4fd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c5154446db8ecd server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.823+0000 7f4fe1ffb700 1 -- 172.21.15.131:0/3195505625 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400b860 con 0x7f4fe41596a0 2023-12-26T23:12:24.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.823+0000 7f4fe1ffb700 1 -- 172.21.15.131:0/3195505625 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd400b9c0 con 0x7f4fe41596a0 2023-12-26T23:12:24.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.823+0000 7f4fe1ffb700 1 -- 172.21.15.131:0/3195505625 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400bd60 con 0x7f4fe41596a0 2023-12-26T23:12:24.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.824+0000 7f4fe93ff700 1 -- 172.21.15.131:0/3195505625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 msgr2=0x7f4fe40b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:24.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.824+0000 7f4fe93ff700 1 --2- 172.21.15.131:0/3195505625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 0x7f4fe40b2380 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40099b0 tx=0x7f4fd4009250 comp rx=0 tx=0).stop 2023-12-26T23:12:24.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.824+0000 7f4fe93ff700 1 -- 172.21.15.131:0/3195505625 shutdown_connections 2023-12-26T23:12:24.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.824+0000 7f4fe93ff700 1 --2- 172.21.15.131:0/3195505625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 0x7f4fe40b2380 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:24.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.824+0000 7f4fe93ff700 1 --2- 172.21.15.131:0/3195505625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 0x7f4fe40b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:24.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.824+0000 7f4fe93ff700 1 -- 172.21.15.131:0/3195505625 >> 172.21.15.131:0/3195505625 conn(0x7f4fe406ece0 msgr2=0x7f4fe406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:24.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.824+0000 7f4fe93ff700 1 -- 172.21.15.131:0/3195505625 shutdown_connections 2023-12-26T23:12:24.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.824+0000 7f4fe93ff700 1 -- 172.21.15.131:0/3195505625 wait complete. 2023-12-26T23:12:24.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.825+0000 7f4fe93ff700 1 Processor -- start 2023-12-26T23:12:24.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.826+0000 7f4fe93ff700 1 -- start start 2023-12-26T23:12:24.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.826+0000 7f4fe93ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe40b0160 0x7f4fe4157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:24.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.826+0000 7f4fe93ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 0x7f4fe40c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:24.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.826+0000 7f4fe93ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 0x7f4fe415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:24.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe93ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4158350 con 0x7f4fe41596a0 2023-12-26T23:12:24.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe93ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c4910 con 0x7f4fe40b0160 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe93ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c4bf0 con 0x7f4fe4159a70 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe27fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 0x7f4fe40c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe27fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 0x7f4fe40c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60228/0 (socket says 172.21.15.131:60228) 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe27fc700 1 -- 172.21.15.131:0/616117707 learned_addr learned my addr 172.21.15.131:0/616117707 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe2ffd700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe40b0160 0x7f4fe4157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe37fe700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 0x7f4fe415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe27fc700 1 -- 172.21.15.131:0/616117707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 msgr2=0x7f4fe415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe27fc700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 0x7f4fe415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe27fc700 1 -- 172.21.15.131:0/616117707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe40b0160 msgr2=0x7f4fe4157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe27fc700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe40b0160 0x7f4fe4157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe27fc700 1 -- 172.21.15.131:0/616117707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd40094f0 con 0x7f4fe41596a0 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.827+0000 7f4fe27fc700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 0x7f4fe40c4010 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800d860 tx=0x7f4fd8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.828+0000 7f4fcaffd700 1 -- 172.21.15.131:0/616117707 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8009610 con 0x7f4fe41596a0 2023-12-26T23:12:24.829 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.828+0000 7f4fcaffd700 1 -- 172.21.15.131:0/616117707 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd8009770 con 0x7f4fe41596a0 2023-12-26T23:12:24.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.828+0000 7f4fcaffd700 1 -- 172.21.15.131:0/616117707 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8009b30 con 0x7f4fe41596a0 2023-12-26T23:12:24.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.828+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe40c4ed0 con 0x7f4fe41596a0 2023-12-26T23:12:24.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.828+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe40c4550 con 0x7f4fe41596a0 2023-12-26T23:12:24.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.829+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe40c7940 con 0x7f4fe41596a0 2023-12-26T23:12:24.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.830+0000 7f4fcaffd700 1 -- 172.21.15.131:0/616117707 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4fd801e460 con 0x7f4fe41596a0 2023-12-26T23:12:24.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.830+0000 7f4fcaffd700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4fd0066f30 0x7f4fd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:24.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.831+0000 7f4fcaffd700 1 -- 172.21.15.131:0/616117707 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f4fd8084dd0 con 0x7f4fe41596a0 2023-12-26T23:12:24.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.831+0000 7f4fe2ffd700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4fd0066f30 0x7f4fd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:24.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.831+0000 7f4fe2ffd700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4fd0066f30 0x7f4fd00693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40099b0 tx=0x7f4fd4017000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:24.834 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:24.834+0000 7f4fcaffd700 1 -- 172.21.15.131:0/616117707 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fd8051d90 con 0x7f4fe41596a0 2023-12-26T23:12:25.072 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.072+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4fe40c75e0 con 0x7f4fe41596a0 2023-12-26T23:12:25.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.612+0000 7f4fcaffd700 1 -- 172.21.15.131:0/616117707 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v98) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fe40c75e0 con 0x7f4fe41596a0 2023-12-26T23:12:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4fd0066f30 msgr2=0x7f4fd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4fd0066f30 0x7f4fd00693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40099b0 tx=0x7f4fd4017000 comp rx=0 tx=0).stop 2023-12-26T23:12:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 msgr2=0x7f4fe40c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 0x7f4fe40c4010 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800d860 tx=0x7f4fd8008040 comp rx=0 tx=0).stop 2023-12-26T23:12:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 shutdown_connections 2023-12-26T23:12:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4fd0066f30 0x7f4fd00693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fe4159a70 0x7f4fe415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe40b0160 0x7f4fe4157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 --2- 172.21.15.131:0/616117707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe41596a0 0x7f4fe40c4010 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 >> 172.21.15.131:0/616117707 conn(0x7f4fe406ece0 msgr2=0x7f4fe4158b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 shutdown_connections 2023-12-26T23:12:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.615+0000 7f4fe93ff700 1 -- 172.21.15.131:0/616117707 wait complete. 2023-12-26T23:12:25.635 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:25.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.828+0000 7fc76d5ff700 1 Processor -- start 2023-12-26T23:12:25.828 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.828+0000 7fc76d5ff700 1 -- start start 2023-12-26T23:12:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.829+0000 7fc76d5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc76813c510 0x7fc7680c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.829+0000 7fc76d5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76806b8c0 0x7fc76806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.829+0000 7fc76d5ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7680686c0 con 0x7fc7680cc100 2023-12-26T23:12:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.829+0000 7fc76d5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc768069520 con 0x7fc76813c510 2023-12-26T23:12:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.829+0000 7fc76d5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76806c250 con 0x7fc76806b8c0 2023-12-26T23:12:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc766ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc76813c510 0x7fc7680c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc766ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc76813c510 0x7fc7680c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55122/0 (socket says 172.21.15.131:55122) 2023-12-26T23:12:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc766ffd700 1 -- 172.21.15.131:0/1870434309 learned_addr learned my addr 172.21.15.131:0/1870434309 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:25.830 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc7667fc700 1 --2- 172.21.15.131:0/1870434309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76806b8c0 0x7fc76806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:25.831 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc766ffd700 1 -- 172.21.15.131:0/1870434309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76806b8c0 msgr2=0x7fc76806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc766ffd700 1 --2- 172.21.15.131:0/1870434309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76806b8c0 0x7fc76806bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc766ffd700 1 -- 172.21.15.131:0/1870434309 >> v1:172.21.15.190:6789/0 conn(0x7fc7680cc100 legacy=0x7fc7680cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc766ffd700 1 -- 172.21.15.131:0/1870434309 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc768068910 con 0x7fc76813c510 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc766ffd700 1 --2- 172.21.15.131:0/1870434309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76813c510 0x7fc7680c8a20 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fc758009990 tx=0x7fc758009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0df950269e3cac8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.830+0000 7fc765ffb700 1 -- 172.21.15.131:0/1870434309 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc758009e80 con 0x7fc76813c510 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc765ffb700 1 -- 172.21.15.131:0/1870434309 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc758005c10 con 0x7fc76813c510 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc765ffb700 1 -- 172.21.15.131:0/1870434309 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75800b650 con 0x7fc76813c510 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc76d5ff700 1 -- 172.21.15.131:0/1870434309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76813c510 msgr2=0x7fc7680c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc76d5ff700 1 --2- 172.21.15.131:0/1870434309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76813c510 0x7fc7680c8a20 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fc758009990 tx=0x7fc758009220 comp rx=0 tx=0).stop 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc76d5ff700 1 -- 172.21.15.131:0/1870434309 shutdown_connections 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc76d5ff700 1 --2- 172.21.15.131:0/1870434309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76813c510 0x7fc7680c8a20 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc76d5ff700 1 --2- 172.21.15.131:0/1870434309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76806b8c0 0x7fc76806bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc76d5ff700 1 -- 172.21.15.131:0/1870434309 >> 172.21.15.131:0/1870434309 conn(0x7fc7680ce3e0 msgr2=0x7fc7680ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:25.832 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc76d5ff700 1 -- 172.21.15.131:0/1870434309 shutdown_connections 2023-12-26T23:12:25.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.831+0000 7fc76d5ff700 1 -- 172.21.15.131:0/1870434309 wait complete. 2023-12-26T23:12:25.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.833+0000 7fc76d5ff700 1 Processor -- start 2023-12-26T23:12:25.833 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.833+0000 7fc76d5ff700 1 -- start start 2023-12-26T23:12:25.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc76d5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc76806b8c0 0x7fc7680b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:25.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc76d5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7680cc100 0x7fc76813add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:25.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc76d5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76813c510 0x7fc7680c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc76d5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680b29f0 con 0x7fc7680cc100 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc76d5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76815c730 con 0x7fc76806b8c0 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc76d5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680c8240 con 0x7fc76813c510 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7667fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7680cc100 0x7fc76813add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7667fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7680cc100 0x7fc76813add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55126/0 (socket says 172.21.15.131:55126) 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7667fc700 1 -- 172.21.15.131:0/2942624387 learned_addr learned my addr 172.21.15.131:0/2942624387 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc766ffd700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc76806b8c0 0x7fc7680b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7667fc700 1 -- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76813c510 msgr2=0x7fc7680c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7677fe700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76813c510 0x7fc7680c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7667fc700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76813c510 0x7fc7680c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7667fc700 1 -- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc76806b8c0 msgr2=0x7fc7680b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7667fc700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc76806b8c0 0x7fc7680b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7667fc700 1 -- 172.21.15.131:0/2942624387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc758009480 con 0x7fc7680cc100 2023-12-26T23:12:25.836 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc766ffd700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc76806b8c0 0x7fc7680b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:12:25.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.834+0000 7fc7677fe700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76813c510 0x7fc7680c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:12:25.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.835+0000 7fc7667fc700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7680cc100 0x7fc76813add0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e960 tx=0x7fc75c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:25.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.835+0000 7fc74effd700 1 -- 172.21.15.131:0/2942624387 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00c850 con 0x7fc7680cc100 2023-12-26T23:12:25.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.835+0000 7fc74effd700 1 -- 172.21.15.131:0/2942624387 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c00c9b0 con 0x7fc7680cc100 2023-12-26T23:12:25.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.835+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7680b23a0 con 0x7fc7680cc100 2023-12-26T23:12:25.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.835+0000 7fc74effd700 1 -- 172.21.15.131:0/2942624387 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00cd50 con 0x7fc7680cc100 2023-12-26T23:12:25.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.835+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7680b33c0 con 0x7fc7680cc100 2023-12-26T23:12:25.837 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.836+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7680657e0 con 0x7fc7680cc100 2023-12-26T23:12:25.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.837+0000 7fc74effd700 1 -- 172.21.15.131:0/2942624387 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc75c013680 con 0x7fc7680cc100 2023-12-26T23:12:25.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.838+0000 7fc74effd700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc754066f30 0x7fc7540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:25.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.838+0000 7fc74effd700 1 -- 172.21.15.131:0/2942624387 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7fc75c0849c0 con 0x7fc7680cc100 2023-12-26T23:12:25.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.838+0000 7fc766ffd700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc754066f30 0x7fc7540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:25.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.839+0000 7fc766ffd700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc754066f30 0x7fc7540693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc758009260 tx=0x7fc758016000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:25.842+0000 7fc74effd700 1 -- 172.21.15.131:0/2942624387 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc75c051980 con 0x7fc7680cc100 2023-12-26T23:12:26.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.077+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7680b2ba0 con 0x7fc7680cc100 2023-12-26T23:12:26.077 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.078+0000 7fc74effd700 1 -- 172.21.15.131:0/2942624387 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4617 (secure 0 0 0) 0x7fc7680b2ba0 con 0x7fc7680cc100 2023-12-26T23:12:26.078 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:26.078 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8372,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2599990460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2599990460},{"type":"v1","addr":"172.21.15.190:6835","nonce":2599990460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":9565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3465421941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3465421941},{"type":"v1","addr":"172.21.15.131:6834","nonce":3465421941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":10122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3595865527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3595865527},{"type":"v1","addr":"172.21.15.131:6835","nonce":3595865527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:12:22.064606+0000","modified":"2023-12-26T23:12:25.606067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10125":{"gid":10125,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/1917956201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1917956201},{"type":"v1","addr":"172.21.15.190:6834","nonce":1917956201}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-12-26T23:12:26.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc754066f30 msgr2=0x7fc7540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.080 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc754066f30 0x7fc7540693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc758009260 tx=0x7fc758016000 comp rx=0 tx=0).stop 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7680cc100 msgr2=0x7fc76813add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7680cc100 0x7fc76813add0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e960 tx=0x7fc75c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 shutdown_connections 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc754066f30 0x7fc7540693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc76813c510 0x7fc7680c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc76806b8c0 0x7fc7680b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 --2- 172.21.15.131:0/2942624387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7680cc100 0x7fc76813add0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 >> 172.21.15.131:0/2942624387 conn(0x7fc7680ce3e0 msgr2=0x7fc7680c4620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 shutdown_connections 2023-12-26T23:12:26.081 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.080+0000 7fc76d5ff700 1 -- 172.21.15.131:0/2942624387 wait complete. 2023-12-26T23:12:26.082 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 98 2023-12-26T23:12:26.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:26.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.288+0000 7f49331ff700 1 Processor -- start 2023-12-26T23:12:26.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.289+0000 7f49331ff700 1 -- start start 2023-12-26T23:12:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.290+0000 7f49331ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f492c159920 0x7f492c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.290+0000 7f49331ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 0x7f492c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.290+0000 7f49331ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f492c1554d0 0x7f492c155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.290+0000 7f49331ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c0c8bf0 con 0x7f492c1554d0 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.290+0000 7f49331ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c0c8e60 con 0x7f492c159920 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.290+0000 7f49331ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0c9140 con 0x7f492c159cf0 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f493179c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f492c1554d0 0x7f492c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f493179c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f492c1554d0 0x7f492c155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55140/0 (socket says 172.21.15.131:55140) 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f493179c700 1 -- 172.21.15.131:0/29700280 learned_addr learned my addr 172.21.15.131:0/29700280 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f4930f9b700 1 --2- 172.21.15.131:0/29700280 >> v2:172.21.15.190:3300/0 conn(0x7f492c159920 0x7f492c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f492bfff700 1 --2- 172.21.15.131:0/29700280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 0x7f492c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f493179c700 1 -- 172.21.15.131:0/29700280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 msgr2=0x7f492c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f493179c700 1 --2- 172.21.15.131:0/29700280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 0x7f492c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f493179c700 1 -- 172.21.15.131:0/29700280 >> v2:172.21.15.190:3300/0 conn(0x7f492c159920 msgr2=0x7f492c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:26.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f493179c700 1 --2- 172.21.15.131:0/29700280 >> v2:172.21.15.190:3300/0 conn(0x7f492c159920 0x7f492c0b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f493179c700 1 -- 172.21.15.131:0/29700280 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0c9420 con 0x7f492c1554d0 2023-12-26T23:12:26.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.291+0000 7f493179c700 1 --2- 172.21.15.131:0/29700280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 0x7f492c155930 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f492c0642a0 tx=0x7f4924009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2608681ccbb5580c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f492b7fe700 1 -- 172.21.15.131:0/29700280 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492400e070 con 0x7f492c1554d0 2023-12-26T23:12:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f492b7fe700 1 -- 172.21.15.131:0/29700280 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492400b970 con 0x7f492c1554d0 2023-12-26T23:12:26.294 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f492b7fe700 1 -- 172.21.15.131:0/29700280 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49240092e0 con 0x7f492c1554d0 2023-12-26T23:12:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f49331ff700 1 -- 172.21.15.131:0/29700280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 msgr2=0x7f492c155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f49331ff700 1 --2- 172.21.15.131:0/29700280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 0x7f492c155930 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f492c0642a0 tx=0x7f4924009840 comp rx=0 tx=0).stop 2023-12-26T23:12:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f49331ff700 1 -- 172.21.15.131:0/29700280 shutdown_connections 2023-12-26T23:12:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f49331ff700 1 --2- 172.21.15.131:0/29700280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 0x7f492c155930 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f49331ff700 1 --2- 172.21.15.131:0/29700280 >> v2:172.21.15.190:3300/0 conn(0x7f492c159920 0x7f492c0b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f49331ff700 1 --2- 172.21.15.131:0/29700280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 0x7f492c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f49331ff700 1 -- 172.21.15.131:0/29700280 >> 172.21.15.131:0/29700280 conn(0x7f492c068040 msgr2=0x7f492c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:26.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f49331ff700 1 -- 172.21.15.131:0/29700280 shutdown_connections 2023-12-26T23:12:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.292+0000 7f49331ff700 1 -- 172.21.15.131:0/29700280 wait complete. 2023-12-26T23:12:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.294+0000 7f49331ff700 1 Processor -- start 2023-12-26T23:12:26.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.294+0000 7f49331ff700 1 -- start start 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f49331ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 0x7f492c16a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f49331ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f492c159920 0x7f492c0c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f49331ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 0x7f492c0a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f49331ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c168740 con 0x7f492c1554d0 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f49331ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0c4440 con 0x7f492c159920 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f49331ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0c4720 con 0x7f492c159cf0 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f4930f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 0x7f492c16a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f4930f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 0x7f492c16a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55154/0 (socket says 172.21.15.131:55154) 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f4930f9b700 1 -- 172.21.15.131:0/4043877889 learned_addr learned my addr 172.21.15.131:0/4043877889 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f492bfff700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f492c159920 0x7f492c0c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f493179c700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 0x7f492c0a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f4930f9b700 1 -- 172.21.15.131:0/4043877889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 msgr2=0x7f492c0a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f4930f9b700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 0x7f492c0a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f4930f9b700 1 -- 172.21.15.131:0/4043877889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f492c159920 msgr2=0x7f492c0c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f4930f9b700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f492c159920 0x7f492c0c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.295+0000 7f4930f9b700 1 -- 172.21.15.131:0/4043877889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4924009ae0 con 0x7f492c1554d0 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.296+0000 7f4930f9b700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 0x7f492c16a2b0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f492000ea20 tx=0x7f492000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.296+0000 7f4928ff9700 1 -- 172.21.15.131:0/4043877889 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492000a7a0 con 0x7f492c1554d0 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.296+0000 7f4928ff9700 1 -- 172.21.15.131:0/4043877889 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492000c8f0 con 0x7f492c1554d0 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.296+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c0c4a00 con 0x7f492c1554d0 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.296+0000 7f4928ff9700 1 -- 172.21.15.131:0/4043877889 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492000cc90 con 0x7f492c1554d0 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.296+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c16a7f0 con 0x7f492c1554d0 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.297+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c0657e0 con 0x7f492c1554d0 2023-12-26T23:12:26.300 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.298+0000 7f4928ff9700 1 -- 172.21.15.131:0/4043877889 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f49200113f0 con 0x7f492c1554d0 2023-12-26T23:12:26.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.299+0000 7f4928ff9700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4910066f30 0x7f49100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.299+0000 7f4928ff9700 1 -- 172.21.15.131:0/4043877889 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f4920015030 con 0x7f492c1554d0 2023-12-26T23:12:26.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.299+0000 7f492bfff700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4910066f30 0x7f49100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.301 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.300+0000 7f492bfff700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4910066f30 0x7f49100693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f492c154b70 tx=0x7f4918008040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:26.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.303+0000 7f4928ff9700 1 -- 172.21.15.131:0/4043877889 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4920055080 con 0x7f492c1554d0 2023-12-26T23:12:26.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.497+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f492c0c9700 con 0x7f492c1554d0 2023-12-26T23:12:26.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.498+0000 7f4928ff9700 1 -- 172.21.15.131:0/4043877889 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) v1 ==== 74+0+16215 (secure 0 0 0) 0x7f4920054c10 con 0x7f492c1554d0 2023-12-26T23:12:26.499 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:26.499 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":210,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:23.589681+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:12:20.250102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-26T23:12:21.042553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:26.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4910066f30 msgr2=0x7f49100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4910066f30 0x7f49100693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f492c154b70 tx=0x7f4918008040 comp rx=0 tx=0).stop 2023-12-26T23:12:26.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 msgr2=0x7f492c16a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 0x7f492c16a2b0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f492000ea20 tx=0x7f492000a330 comp rx=0 tx=0).stop 2023-12-26T23:12:26.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 shutdown_connections 2023-12-26T23:12:26.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4910066f30 0x7f49100693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f492c159cf0 0x7f492c0a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f492c1554d0 0x7f492c16a2b0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 --2- 172.21.15.131:0/4043877889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f492c159920 0x7f492c0c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 >> 172.21.15.131:0/4043877889 conn(0x7f492c068040 msgr2=0x7f492c0aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:26.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 shutdown_connections 2023-12-26T23:12:26.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.501+0000 7f49331ff700 1 -- 172.21.15.131:0/4043877889 wait complete. 2023-12-26T23:12:26.519 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-26T23:12:26.519 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:12:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.711+0000 7f9ea57ff700 1 Processor -- start 2023-12-26T23:12:26.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.711+0000 7f9ea57ff700 1 -- start start 2023-12-26T23:12:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.712+0000 7f9ea57ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea00c8820 0x7f9ea00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.712+0000 7f9ea57ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9ea00c8bf0 0x7f9ea00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.712+0000 7f9ea57ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea0157a80 con 0x7f9ea0069d10 2023-12-26T23:12:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.712+0000 7f9ea57ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00cac50 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.712+0000 7f9ea57ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00caf30 con 0x7f9ea00c8820 2023-12-26T23:12:26.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9e7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9ea00c8bf0 0x7f9ea00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea00c8820 0x7f9ea00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9e7fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9ea00c8bf0 0x7f9ea00b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:46158/0 (socket says 172.21.15.131:46158) 2023-12-26T23:12:26.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9e7fc700 1 -- 172.21.15.131:0/1243018577 learned_addr learned my addr 172.21.15.131:0/1243018577 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:26.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9f7fe700 1 --1- 172.21.15.131:0/1243018577 >> v1:172.21.15.190:6789/0 conn(0x7f9ea0069d10 0x7f9ea0157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:44130/0 (socket says 172.21.15.131:44130) 2023-12-26T23:12:26.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9e7fc700 1 -- 172.21.15.131:0/1243018577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea00c8820 msgr2=0x7f9ea00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:26.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9e7fc700 1 --2- 172.21.15.131:0/1243018577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea00c8820 0x7f9ea00ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9e7fc700 1 -- 172.21.15.131:0/1243018577 >> v1:172.21.15.190:6789/0 conn(0x7f9ea0069d10 legacy=0x7f9ea0157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9e7fc700 1 -- 172.21.15.131:0/1243018577 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea00cb210 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.713+0000 7f9e9e7fc700 1 --2- 172.21.15.131:0/1243018577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ea00c8bf0 0x7f9ea00b12e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9e940099b0 tx=0x7f9e94009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa60357e9b3346ac server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:26.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9e9dffb700 1 -- 172.21.15.131:0/1243018577 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94009ea0 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9e9dffb700 1 -- 172.21.15.131:0/1243018577 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9400b860 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9e9dffb700 1 -- 172.21.15.131:0/1243018577 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400bc00 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9ea57ff700 1 -- 172.21.15.131:0/1243018577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ea00c8bf0 msgr2=0x7f9ea00b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9ea57ff700 1 --2- 172.21.15.131:0/1243018577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ea00c8bf0 0x7f9ea00b12e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9e940099b0 tx=0x7f9e94009250 comp rx=0 tx=0).stop 2023-12-26T23:12:26.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9ea57ff700 1 -- 172.21.15.131:0/1243018577 shutdown_connections 2023-12-26T23:12:26.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9ea57ff700 1 --2- 172.21.15.131:0/1243018577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea00c8820 0x7f9ea00ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9ea57ff700 1 --2- 172.21.15.131:0/1243018577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ea00c8bf0 0x7f9ea00b12e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9ea57ff700 1 -- 172.21.15.131:0/1243018577 >> 172.21.15.131:0/1243018577 conn(0x7f9ea0068040 msgr2=0x7f9ea0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:26.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9ea57ff700 1 -- 172.21.15.131:0/1243018577 shutdown_connections 2023-12-26T23:12:26.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.714+0000 7f9ea57ff700 1 -- 172.21.15.131:0/1243018577 wait complete. 2023-12-26T23:12:26.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.716+0000 7f9ea57ff700 1 Processor -- start 2023-12-26T23:12:26.717 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.716+0000 7f9ea57ff700 1 -- start start 2023-12-26T23:12:26.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9ea57ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea0069d10 0x7f9ea0155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9ea57ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ea00c8820 0x7f9ea0153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9ea57ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea00c8bf0 0x7f9ea016a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9ea57ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0155710 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9ea57ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01543e0 con 0x7f9ea00c8820 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9ea57ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01546c0 con 0x7f9ea0069d10 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9e9f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea00c8bf0 0x7f9ea016a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9e9f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea00c8bf0 0x7f9ea016a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55160/0 (socket says 172.21.15.131:55160) 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9e9f7fe700 1 -- 172.21.15.131:0/3697954040 learned_addr learned my addr 172.21.15.131:0/3697954040 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9e9e7fc700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ea00c8820 0x7f9ea0153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.717+0000 7f9e9effd700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea0069d10 0x7f9ea0155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9e9f7fe700 1 -- 172.21.15.131:0/3697954040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea0069d10 msgr2=0x7f9ea0155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9e9f7fe700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea0069d10 0x7f9ea0155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9e9f7fe700 1 -- 172.21.15.131:0/3697954040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ea00c8820 msgr2=0x7f9ea0153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9e9f7fe700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ea00c8820 0x7f9ea0153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9e9f7fe700 1 -- 172.21.15.131:0/3697954040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940094f0 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9e9f7fe700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea00c8bf0 0x7f9ea016a600 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800cc70 tx=0x7f9e98001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9e86ffd700 1 -- 172.21.15.131:0/3697954040 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98005df0 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9e86ffd700 1 -- 172.21.15.131:0/3697954040 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9800a5b0 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9e86ffd700 1 -- 172.21.15.131:0/3697954040 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800a930 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea01549a0 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.718+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea00c5440 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.719+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea00657e0 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.720+0000 7f9e86ffd700 1 -- 172.21.15.131:0/3697954040 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9e9800ace0 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.721+0000 7f9e86ffd700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9e8c066f30 0x7f9e8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:26.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.721+0000 7f9e86ffd700 1 -- 172.21.15.131:0/3697954040 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f9e980846d0 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.721+0000 7f9e9effd700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9e8c066f30 0x7f9e8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:26.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.722+0000 7f9e9effd700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9e8c066f30 0x7f9e8c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009d10 tx=0x7f9e900091a0 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:26.725 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.725+0000 7f9e86ffd700 1 -- 172.21.15.131:0/3697954040 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e980517b0 con 0x7f9ea00c8bf0 2023-12-26T23:12:26.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.913+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ea00cb4f0 con 0x7f9e8c066f30 2023-12-26T23:12:26.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.923+0000 7f9e86ffd700 1 -- 172.21.15.131:0/3697954040 <== mgr.9927 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278857 (secure 0 0 0) 0x7f9ea00cb4f0 con 0x7f9e8c066f30 2023-12-26T23:12:26.927 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9e8c066f30 msgr2=0x7f9e8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9e8c066f30 0x7f9e8c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009d10 tx=0x7f9e900091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea00c8bf0 msgr2=0x7f9ea016a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea00c8bf0 0x7f9ea016a600 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800cc70 tx=0x7f9e98001a70 comp rx=0 tx=0).stop 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 shutdown_connections 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9e8c066f30 0x7f9e8c0693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea00c8bf0 0x7f9ea016a600 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ea0069d10 0x7f9ea0155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 --2- 172.21.15.131:0/3697954040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ea00c8820 0x7f9ea0153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 >> 172.21.15.131:0/3697954040 conn(0x7f9ea0068040 msgr2=0x7f9ea00704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 shutdown_connections 2023-12-26T23:12:26.930 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:26.927+0000 7f9ea57ff700 1 -- 172.21.15.131:0/3697954040 wait complete. 2023-12-26T23:12:26.931 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:12:26.946 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T23:12:26.065921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452856,"kb_used_data":21304,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752521864,"statfs":{"total":773094113280,"available":770582388736,"internally_reserved":0,"allocated":21815296,"data_stored":11557076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.498011"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575758+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:15.760243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574241+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:57:09.512915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573553+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:22.340338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575297+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:09:19.634386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573139+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:34.976485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.572849+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:52:38.429194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574047+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:28:29.479031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571063+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:12:08.105016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.577061+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:58:29.785862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573092+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:29:10.284887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.572820+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:56:57.660062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574872+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:15.760243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.577774+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:54:18.679739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571723+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:59:27.380222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573280+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:21:14.188510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.572056+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:10:59.783625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571197+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:18:38.340055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570022+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:49:13.753445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570023+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:16.422448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571821+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:51:46.017125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571197+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:34.976485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571284+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:22.340338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570043+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:31:20.574628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570079+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:26.390563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575357+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:28:58.221939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575560+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:09:19.634386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571687+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:00:04.850373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573973+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:23.335081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574139+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:40:08.598399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573907+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:15:47.071802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571197+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:51:22.275494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574377+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:28:58.221939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"199'348","reported_seq":1149,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-12-26T23:12:18.018701+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:12:18.018701+0000","last_peered":"2023-12-26T23:12:18.018701+0000","last_clean":"2023-12-26T23:12:18.018701+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:12:18.018701+0000","last_undegraded":"2023-12-26T23:12:18.018701+0000","last_fullsized":"2023-12-26T23:12:18.018701+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.56699999999999995,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.85499999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.54300000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90800000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73299999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.98599999999999999,"15min":0.995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.58599999999999997,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":0.89200000000000002,"5min":0.93300000000000005,"15min":1.0449999999999999},"last":1.038}]},{"osd":2,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.55500000000000005,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.86799999999999999,"5min":0.94199999999999995,"15min":1},"last":0.69999999999999996}]},{"osd":3,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.69399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95199999999999996,"5min":1.0029999999999999,"15min":1.0760000000000001},"last":0.999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.97599999999999998,"15min":0.98199999999999998},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.91400000000000003},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.39600000000000002,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.90600000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.98799999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.57499999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92800000000000005,"5min":0.998,"15min":1.042},"last":1.1359999999999999}]},{"osd":6,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.54800000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.88800000000000001}]}]},{"osd":6,"up_from":15,"seq":64424510575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.32800000000000001},"max":{"1min":0.873,"5min":0.998,"15min":1.1080000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0680000000000001},"last":0.80400000000000005}]},{"osd":1,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":1.022,"15min":1.046},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.95999999999999996,"15min":1.0580000000000001},"last":0.745}]},{"osd":2,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91500000000000004,"15min":1.071},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":0.89800000000000002,"5min":0.97899999999999998,"15min":1.0680000000000001},"last":0.622}]},{"osd":3,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.96699999999999997,"15min":1.024},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":1.0149999999999999},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.018,"15min":1.1599999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.111},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.66900000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.67400000000000004}]}]},{"osd":1,"up_from":15,"seq":64424510575,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300380,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071460,"statfs":{"total":96636764160,"available":96329175040,"internally_reserved":0,"allocated":3166208,"data_stored":1884248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.29199999999999998,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.81399999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.50600000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.622,"5min":0.76900000000000002,"15min":0.79200000000000004},"last":0.45100000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.33800000000000002},"max":{"1min":0.58099999999999996,"5min":0.753,"15min":0.84099999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.65800000000000003,"5min":0.75700000000000001,"15min":0.91200000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.70299999999999996,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.46700000000000003,"5min":0.40600000000000003,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.872,"15min":0.93600000000000005},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.629,"5min":0.55100000000000005,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":1.103,"15min":1.103},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.79100000000000004,"15min":0.77500000000000002},"min":{"1min":0.60899999999999999,"5min":0.52500000000000002,"15min":0.48599999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.57799999999999996}]},{"osd":5,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.50700000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.83199999999999996},"min":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.1100000000000001,"5min":1.1240000000000001,"15min":1.1830000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.753,"5min":0.57599999999999996,"15min":0.52600000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.627},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.85899999999999999,"15min":0.84299999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.64700000000000002,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.218},"last":0.67400000000000004}]}]},{"osd":0,"up_from":15,"seq":64424510575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.26600000000000001},"max":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.313},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.30599999999999999},"max":{"1min":0.72899999999999998,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.88100000000000001,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88,"5min":0.91800000000000004,"15min":1.012},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.2250000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77400000000000002,"15min":0.78300000000000003},"min":{"1min":0.66400000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.113},"last":0.85999999999999999}]},{"osd":5,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80400000000000005,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0469999999999999,"5min":1.0920000000000001,"15min":1.157},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.82899999999999996,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.282},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.79500000000000004,"15min":0.80500000000000005},"min":{"1min":0.72599999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.121,"5min":1.121,"15min":1.151},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81799999999999995,"15min":0.82199999999999995},"min":{"1min":0.67500000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1020000000000001,"5min":1.173,"15min":1.256},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.64200000000000002,"5min":0.54800000000000004,"15min":0.54000000000000004},"max":{"1min":1.087,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.879}]}]},{"osd":2,"up_from":14,"seq":60129543280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312088,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059752,"statfs":{"total":96636764160,"available":96317186048,"internally_reserved":0,"allocated":2572288,"data_stored":1293880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":0.84599999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69199999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.45300000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.45800000000000002,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.64700000000000002,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.79200000000000004,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80200000000000005,"5min":0.89300000000000002,"15min":1.056},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.65600000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1180000000000001,"5min":1.8,"15min":1.8},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.59099999999999997,"15min":0.54000000000000004},"max":{"1min":1.115,"5min":1.222,"15min":1.222},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":1.079,"5min":1.923,"15min":1.923},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.087,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.55300000000000005,"15min":0.52400000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.272},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0269999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.77500000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.56399999999999995},"max":{"1min":1.042,"5min":1.1100000000000001,"15min":1.1619999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.84499999999999997},"min":{"1min":0.70999999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0740000000000001,"5min":1.841,"15min":1.841},"last":0.73599999999999999}]}]},{"osd":3,"up_from":15,"seq":64424510576,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1293880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46400000000000002,"15min":0.46100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.246},"max":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.245},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.746,"15min":0.76500000000000001},"last":0.26500000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47399999999999998,"15min":0.47999999999999998},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.753},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.307},"max":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47699999999999998}]},{"osd":2,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.751,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.079,"5min":1.177,"15min":1.177},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.58099999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.97799999999999998,"5min":1.0329999999999999,"15min":1.081},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.60499999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.0980000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.141},"last":0.63600000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.67700000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.84699999999999998},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.022,"5min":1.141,"15min":1.214},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.68700000000000006,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.71199999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312100,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059740,"statfs":{"total":96636764160,"available":96317173760,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.008,"15min":1.008},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71499999999999997,"15min":0.73699999999999999},"min":{"1min":0.55200000000000005,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":1.0329999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73299999999999998,"15min":0.75},"min":{"1min":0.58599999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.55100000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.998},"last":0.626}]},{"osd":3,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.51900000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84399999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.628,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":0.96799999999999997,"15min":0.98999999999999999},"last":0.83399999999999996}]},{"osd":6,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.73099999999999998,"5min":1.002,"15min":1.024},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.495,"15min":0.49199999999999999},"max":{"1min":0.77800000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":0.94099999999999995,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.71899999999999997}]}]},{"osd":5,"up_from":15,"seq":64424510574,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300380,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071460,"statfs":{"total":96636764160,"available":96329175040,"internally_reserved":0,"allocated":3166208,"data_stored":1884248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.60799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.92700000000000005,"15min":1.0629999999999999},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.016},"last":0.70699999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.012,"15min":1.022},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.99199999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":0.997,"15min":0.997},"last":0.74099999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93000000000000005,"15min":0.95199999999999996},"last":0.78900000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.97799999999999998,"15min":1.042},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.90600000000000003,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.90500000000000003,"15min":0.996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:12:26.951 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:12:27.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.175+0000 7f05f0bff700 1 Processor -- start 2023-12-26T23:12:27.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.176+0000 7f05f0bff700 1 -- start start 2023-12-26T23:12:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.177+0000 7f05f0bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05ec159a70 0x7f05ec0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.177+0000 7f05f0bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec0afcd0 con 0x7f05ec0b2a90 2023-12-26T23:12:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.177+0000 7f05f0bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec0b3e70 con 0x7f05ec1596a0 2023-12-26T23:12:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.177+0000 7f05f0bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec0b0f20 con 0x7f05ec159a70 2023-12-26T23:12:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.177+0000 7f05ead9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f05ec0b2a90 0x7f05ec0b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32828/0 (socket says 172.21.15.131:32828) 2023-12-26T23:12:27.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.177+0000 7f05ead9d700 1 -- 172.21.15.131:0/2823616365 learned_addr learned my addr 172.21.15.131:0/2823616365 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:27.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.177+0000 7f05e9d9b700 1 --2- 172.21.15.131:0/2823616365 >> v2:172.21.15.190:3300/0 conn(0x7f05ec159a70 0x7f05ec0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.177+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254560291 0 0) 0x7f05ec0afcd0 con 0x7f05ec0b2a90 2023-12-26T23:12:27.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05d4003570 con 0x7f05ec0b2a90 2023-12-26T23:12:27.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430295807 0 0) 0x7f05ec0b3e70 con 0x7f05ec1596a0 2023-12-26T23:12:27.180 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ec0afcd0 con 0x7f05ec1596a0 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3078321741 0 0) 0x7f05d4003570 con 0x7f05ec0b2a90 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 >> v2:172.21.15.190:3300/0 conn(0x7f05ec159a70 msgr2=0x7f05ec0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 --2- 172.21.15.131:0/2823616365 >> v2:172.21.15.190:3300/0 conn(0x7f05ec159a70 0x7f05ec0b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 >> v1:172.21.15.190:6789/0 conn(0x7f05ec1596a0 legacy=0x7f05ec0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0b3540 con 0x7f05ec0b2a90 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f05e40017f0 con 0x7f05ec0b2a90 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 >> v1:172.21.15.131:6789/0 conn(0x7f05ec0b2a90 legacy=0x7f05ec0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d4003c40 0x7f05d4004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05d4004810 0x7f05d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4005210 0x7f05d400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0b0f20 con 0x7f05d4003c40 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d400c130 con 0x7f05d4005210 2023-12-26T23:12:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d400c2c0 con 0x7f05d4004810 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05ea59c700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d4003c40 0x7f05d4004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05e9d9b700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05d4004810 0x7f05d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.178+0000 7f05ead9d700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4005210 0x7f05d400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05ea59c700 1 -- 172.21.15.131:0/2823616365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05d4004810 msgr2=0x7f05d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05ea59c700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05d4004810 0x7f05d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05ea59c700 1 -- 172.21.15.131:0/2823616365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4005210 msgr2=0x7f05d400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05ea59c700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05d4005210 0x7f05d400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05ea59c700 1 -- 172.21.15.131:0/2823616365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d4003570 con 0x7f05d4003c40 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05ead9d700 1 -- 172.21.15.131:0/2823616365 reap_dead start 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05ea59c700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d4003c40 0x7f05d4004030 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f05dc001a40 tx=0x7f05dc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c18c213ca78b441 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00c560 con 0x7f05d4003c40 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc00c6c0 con 0x7f05d4003c40 2023-12-26T23:12:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.179+0000 7f05e959a700 1 -- 172.21.15.131:0/2823616365 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00ca60 con 0x7f05d4003c40 2023-12-26T23:12:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.180+0000 7f05f0bff700 1 -- 172.21.15.131:0/2823616365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d4003c40 msgr2=0x7f05d4004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.180+0000 7f05f0bff700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d4003c40 0x7f05d4004030 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f05dc001a40 tx=0x7f05dc007d30 comp rx=0 tx=0).stop 2023-12-26T23:12:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.180+0000 7f05f0bff700 1 -- 172.21.15.131:0/2823616365 shutdown_connections 2023-12-26T23:12:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.180+0000 7f05f0bff700 1 --2- 172.21.15.131:0/2823616365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d4003c40 0x7f05d4004030 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.180+0000 7f05f0bff700 1 -- 172.21.15.131:0/2823616365 >> 172.21.15.131:0/2823616365 conn(0x7f05ec06ece0 msgr2=0x7f05ec06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.180+0000 7f05f0bff700 1 -- 172.21.15.131:0/2823616365 shutdown_connections 2023-12-26T23:12:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.180+0000 7f05f0bff700 1 -- 172.21.15.131:0/2823616365 wait complete. 2023-12-26T23:12:27.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.182+0000 7f05f0bff700 1 Processor -- start 2023-12-26T23:12:27.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.182+0000 7f05f0bff700 1 -- start start 2023-12-26T23:12:27.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.183+0000 7f05f0bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05d4003c40 0x7f05ec0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.183+0000 7f05f0bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec16a4c0 0x7f05ec16a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.183+0000 7f05f0bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ec155d80 0x7f05ec1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.183+0000 7f05f0bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec16a0a0 con 0x7f05ec16a4c0 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.183+0000 7f05f0bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec153a80 con 0x7f05ec155d80 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.183+0000 7f05f0bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec156730 con 0x7f05d4003c40 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05e9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec16a4c0 0x7f05ec16a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05e9d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec16a4c0 0x7f05ec16a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55184/0 (socket says 172.21.15.131:55184) 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05e9d9b700 1 -- 172.21.15.131:0/3693205664 learned_addr learned my addr 172.21.15.131:0/3693205664 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05ea59c700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05d4003c40 0x7f05ec0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05ead9d700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ec155d80 0x7f05ec1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05e9d9b700 1 -- 172.21.15.131:0/3693205664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05d4003c40 msgr2=0x7f05ec0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05e9d9b700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05d4003c40 0x7f05ec0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05e9d9b700 1 -- 172.21.15.131:0/3693205664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ec155d80 msgr2=0x7f05ec1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05e9d9b700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ec155d80 0x7f05ec1561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05e9d9b700 1 -- 172.21.15.131:0/3693205664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0c4010 con 0x7f05ec16a4c0 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05e9d9b700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec16a4c0 0x7f05ec16a930 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f05e0011da0 tx=0x7f05e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:27.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.184+0000 7f05da7fc700 1 -- 172.21.15.131:0/3693205664 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000f9b0 con 0x7f05ec16a4c0 2023-12-26T23:12:27.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.185+0000 7f05da7fc700 1 -- 172.21.15.131:0/3693205664 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e000fb10 con 0x7f05ec16a4c0 2023-12-26T23:12:27.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.185+0000 7f05da7fc700 1 -- 172.21.15.131:0/3693205664 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000fe90 con 0x7f05ec16a4c0 2023-12-26T23:12:27.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.185+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec0c4260 con 0x7f05ec16a4c0 2023-12-26T23:12:27.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.185+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec0c4820 con 0x7f05ec16a4c0 2023-12-26T23:12:27.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.186+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec16a1e0 con 0x7f05ec16a4c0 2023-12-26T23:12:27.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.187+0000 7f05da7fc700 1 -- 172.21.15.131:0/3693205664 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f05e0014650 con 0x7f05ec16a4c0 2023-12-26T23:12:27.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.187+0000 7f05da7fc700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05cc066f30 0x7f05cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.188+0000 7f05da7fc700 1 -- 172.21.15.131:0/3693205664 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f05e0018030 con 0x7f05ec16a4c0 2023-12-26T23:12:27.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.188+0000 7f05ea59c700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05cc066f30 0x7f05cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.188+0000 7f05ea59c700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05cc066f30 0x7f05cc0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0017f0 tx=0x7f05dc015000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:27.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.192+0000 7f05da7fc700 1 -- 172.21.15.131:0/3693205664 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05ec16a1e0 con 0x7f05ec16a4c0 2023-12-26T23:12:27.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.380+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05ec16a1e0 con 0x7f05cc066f30 2023-12-26T23:12:27.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.388+0000 7f05da7fc700 1 -- 172.21.15.131:0/3693205664 <== mgr.9927 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278857 (secure 0 0 0) 0x7f05ec16a1e0 con 0x7f05cc066f30 2023-12-26T23:12:27.392 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05cc066f30 msgr2=0x7f05cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05cc066f30 0x7f05cc0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0017f0 tx=0x7f05dc015000 comp rx=0 tx=0).stop 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec16a4c0 msgr2=0x7f05ec16a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec16a4c0 0x7f05ec16a930 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f05e0011da0 tx=0x7f05e00091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 shutdown_connections 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05cc066f30 0x7f05cc0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ec155d80 0x7f05ec1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05d4003c40 0x7f05ec0b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 --2- 172.21.15.131:0/3693205664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec16a4c0 0x7f05ec16a930 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 >> 172.21.15.131:0/3693205664 conn(0x7f05ec06ece0 msgr2=0x7f05ec0cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:27.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 shutdown_connections 2023-12-26T23:12:27.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.392+0000 7f05f0bff700 1 -- 172.21.15.131:0/3693205664 wait complete. 2023-12-26T23:12:27.395 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:12:27.410 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T23:12:26.065921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452856,"kb_used_data":21304,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752521864,"statfs":{"total":773094113280,"available":770582388736,"internally_reserved":0,"allocated":21815296,"data_stored":11557076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.498011"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575758+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:15.760243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574241+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:57:09.512915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573553+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:22.340338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575297+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:09:19.634386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573139+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:34.976485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.572849+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:52:38.429194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574047+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:28:29.479031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571063+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:12:08.105016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.577061+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:58:29.785862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573092+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:29:10.284887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.572820+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:56:57.660062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574872+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:15.760243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.577774+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:54:18.679739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571723+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:59:27.380222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573280+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:21:14.188510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.572056+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:10:59.783625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571197+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:18:38.340055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570022+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:49:13.753445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570023+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:16.422448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571821+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:51:46.017125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571197+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:34.976485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571284+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:22.340338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570043+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:31:20.574628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570079+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:26.390563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575357+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:28:58.221939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575560+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:09:19.634386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571687+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:00:04.850373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573973+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:23.335081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574139+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:40:08.598399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573907+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:15:47.071802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571197+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:51:22.275494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574377+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:28:58.221939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"199'348","reported_seq":1149,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-12-26T23:12:18.018701+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:12:18.018701+0000","last_peered":"2023-12-26T23:12:18.018701+0000","last_clean":"2023-12-26T23:12:18.018701+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:12:18.018701+0000","last_undegraded":"2023-12-26T23:12:18.018701+0000","last_fullsized":"2023-12-26T23:12:18.018701+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.56699999999999995,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.85499999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.54300000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90800000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73299999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.98599999999999999,"15min":0.995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.58599999999999997,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":0.89200000000000002,"5min":0.93300000000000005,"15min":1.0449999999999999},"last":1.038}]},{"osd":2,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.55500000000000005,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.86799999999999999,"5min":0.94199999999999995,"15min":1},"last":0.69999999999999996}]},{"osd":3,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.69399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95199999999999996,"5min":1.0029999999999999,"15min":1.0760000000000001},"last":0.999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.97599999999999998,"15min":0.98199999999999998},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.91400000000000003},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.39600000000000002,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.90600000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.98799999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.57499999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92800000000000005,"5min":0.998,"15min":1.042},"last":1.1359999999999999}]},{"osd":6,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.54800000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.88800000000000001}]}]},{"osd":6,"up_from":15,"seq":64424510575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.32800000000000001},"max":{"1min":0.873,"5min":0.998,"15min":1.1080000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0680000000000001},"last":0.80400000000000005}]},{"osd":1,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":1.022,"15min":1.046},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.95999999999999996,"15min":1.0580000000000001},"last":0.745}]},{"osd":2,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91500000000000004,"15min":1.071},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":0.89800000000000002,"5min":0.97899999999999998,"15min":1.0680000000000001},"last":0.622}]},{"osd":3,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.96699999999999997,"15min":1.024},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":1.0149999999999999},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.018,"15min":1.1599999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.111},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.66900000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.67400000000000004}]}]},{"osd":1,"up_from":15,"seq":64424510575,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300380,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071460,"statfs":{"total":96636764160,"available":96329175040,"internally_reserved":0,"allocated":3166208,"data_stored":1884248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.29199999999999998,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.81399999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.50600000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.622,"5min":0.76900000000000002,"15min":0.79200000000000004},"last":0.45100000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.33800000000000002},"max":{"1min":0.58099999999999996,"5min":0.753,"15min":0.84099999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.65800000000000003,"5min":0.75700000000000001,"15min":0.91200000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.70299999999999996,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.46700000000000003,"5min":0.40600000000000003,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.872,"15min":0.93600000000000005},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.629,"5min":0.55100000000000005,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":1.103,"15min":1.103},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.79100000000000004,"15min":0.77500000000000002},"min":{"1min":0.60899999999999999,"5min":0.52500000000000002,"15min":0.48599999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.57799999999999996}]},{"osd":5,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.50700000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.83199999999999996},"min":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.1100000000000001,"5min":1.1240000000000001,"15min":1.1830000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.753,"5min":0.57599999999999996,"15min":0.52600000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.627},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.85899999999999999,"15min":0.84299999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.64700000000000002,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.218},"last":0.67400000000000004}]}]},{"osd":0,"up_from":15,"seq":64424510575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.26600000000000001},"max":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.313},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.30599999999999999},"max":{"1min":0.72899999999999998,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.88100000000000001,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88,"5min":0.91800000000000004,"15min":1.012},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.2250000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77400000000000002,"15min":0.78300000000000003},"min":{"1min":0.66400000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.113},"last":0.85999999999999999}]},{"osd":5,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80400000000000005,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0469999999999999,"5min":1.0920000000000001,"15min":1.157},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.82899999999999996,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.282},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.79500000000000004,"15min":0.80500000000000005},"min":{"1min":0.72599999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.121,"5min":1.121,"15min":1.151},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81799999999999995,"15min":0.82199999999999995},"min":{"1min":0.67500000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1020000000000001,"5min":1.173,"15min":1.256},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.64200000000000002,"5min":0.54800000000000004,"15min":0.54000000000000004},"max":{"1min":1.087,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.879}]}]},{"osd":2,"up_from":14,"seq":60129543280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312088,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059752,"statfs":{"total":96636764160,"available":96317186048,"internally_reserved":0,"allocated":2572288,"data_stored":1293880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":0.84599999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69199999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.45300000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.45800000000000002,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.64700000000000002,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.79200000000000004,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80200000000000005,"5min":0.89300000000000002,"15min":1.056},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.65600000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1180000000000001,"5min":1.8,"15min":1.8},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.59099999999999997,"15min":0.54000000000000004},"max":{"1min":1.115,"5min":1.222,"15min":1.222},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":1.079,"5min":1.923,"15min":1.923},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.087,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.55300000000000005,"15min":0.52400000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.272},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0269999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.77500000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.56399999999999995},"max":{"1min":1.042,"5min":1.1100000000000001,"15min":1.1619999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.84499999999999997},"min":{"1min":0.70999999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0740000000000001,"5min":1.841,"15min":1.841},"last":0.73599999999999999}]}]},{"osd":3,"up_from":15,"seq":64424510576,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1293880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46400000000000002,"15min":0.46100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.246},"max":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.245},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.746,"15min":0.76500000000000001},"last":0.26500000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47399999999999998,"15min":0.47999999999999998},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.753},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.307},"max":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47699999999999998}]},{"osd":2,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.751,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.079,"5min":1.177,"15min":1.177},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.58099999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.97799999999999998,"5min":1.0329999999999999,"15min":1.081},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.60499999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.0980000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.141},"last":0.63600000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.67700000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.84699999999999998},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.022,"5min":1.141,"15min":1.214},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.68700000000000006,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.71199999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312100,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059740,"statfs":{"total":96636764160,"available":96317173760,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.008,"15min":1.008},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71499999999999997,"15min":0.73699999999999999},"min":{"1min":0.55200000000000005,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":1.0329999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73299999999999998,"15min":0.75},"min":{"1min":0.58599999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.55100000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.998},"last":0.626}]},{"osd":3,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.51900000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84399999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.628,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":0.96799999999999997,"15min":0.98999999999999999},"last":0.83399999999999996}]},{"osd":6,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.73099999999999998,"5min":1.002,"15min":1.024},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.495,"15min":0.49199999999999999},"max":{"1min":0.77800000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":0.94099999999999995,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.71899999999999997}]}]},{"osd":5,"up_from":15,"seq":64424510574,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300380,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071460,"statfs":{"total":96636764160,"available":96329175040,"internally_reserved":0,"allocated":3166208,"data_stored":1884248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.60799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.92700000000000005,"15min":1.0629999999999999},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.016},"last":0.70699999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.012,"15min":1.022},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.99199999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":0.997,"15min":0.997},"last":0.74099999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93000000000000005,"15min":0.95199999999999996},"last":0.78900000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.97799999999999998,"15min":1.042},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.90600000000000003,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.90500000000000003,"15min":0.996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:12:27.414 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:12:27.564 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-12-26T23:12:27.563+0000 7f4e285f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T23:12:27.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.641+0000 7fd526fff700 1 Processor -- start 2023-12-26T23:12:27.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.641+0000 7fd526fff700 1 -- start start 2023-12-26T23:12:27.643 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.642+0000 7fd526fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd52013c510 0x7fd5200c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.642+0000 7fd526fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 0x7fd52006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.642+0000 7fd526fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5200686c0 con 0x7fd5200cc100 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.642+0000 7fd526fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd520069520 con 0x7fd52013c510 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.642+0000 7fd526fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52006c250 con 0x7fd52006b8c0 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd524d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd52013c510 0x7fd5200c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd52559c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd5200cc100 0x7fd5200cc560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32840/0 (socket says 172.21.15.131:32840) 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd52559c700 1 -- 172.21.15.131:0/1659072422 learned_addr learned my addr 172.21.15.131:0/1659072422 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd51ffff700 1 --2- 172.21.15.131:0/1659072422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 0x7fd52006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd524d9b700 1 -- 172.21.15.131:0/1659072422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 msgr2=0x7fd52006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd524d9b700 1 --2- 172.21.15.131:0/1659072422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 0x7fd52006bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd524d9b700 1 -- 172.21.15.131:0/1659072422 >> v1:172.21.15.131:6789/0 conn(0x7fd5200cc100 legacy=0x7fd5200cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd524d9b700 1 -- 172.21.15.131:0/1659072422 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520068910 con 0x7fd52013c510 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd51f7fe700 1 -- 172.21.15.131:0/1659072422 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414011974 0 0) 0x7fd5200686c0 con 0x7fd5200cc100 2023-12-26T23:12:27.644 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd524d9b700 1 --2- 172.21.15.131:0/1659072422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 0x7fd5200c8a20 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fd5140098d0 tx=0x7fd5140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e76083ef482b93 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:27.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.643+0000 7fd51f7fe700 1 -- 172.21.15.131:0/1659072422 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514009e70 con 0x7fd52013c510 2023-12-26T23:12:27.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd51f7fe700 1 -- 172.21.15.131:0/1659072422 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51400b860 con 0x7fd52013c510 2023-12-26T23:12:27.645 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd51f7fe700 1 -- 172.21.15.131:0/1659072422 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400bc00 con 0x7fd52013c510 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd526fff700 1 -- 172.21.15.131:0/1659072422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 msgr2=0x7fd5200c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd526fff700 1 --2- 172.21.15.131:0/1659072422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 0x7fd5200c8a20 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fd5140098d0 tx=0x7fd5140091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd526fff700 1 -- 172.21.15.131:0/1659072422 shutdown_connections 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd526fff700 1 --2- 172.21.15.131:0/1659072422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 0x7fd5200c8a20 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd526fff700 1 --2- 172.21.15.131:0/1659072422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 0x7fd52006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd526fff700 1 -- 172.21.15.131:0/1659072422 >> 172.21.15.131:0/1659072422 conn(0x7fd5200ce3e0 msgr2=0x7fd5200ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd526fff700 1 -- 172.21.15.131:0/1659072422 shutdown_connections 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.644+0000 7fd526fff700 1 -- 172.21.15.131:0/1659072422 wait complete. 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.646+0000 7fd526fff700 1 Processor -- start 2023-12-26T23:12:27.646 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.646+0000 7fd526fff700 1 -- start start 2023-12-26T23:12:27.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd526fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 0x7fd52013bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd526fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5200cc100 0x7fd52016d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd526fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 0x7fd5200c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd526fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52016dee0 con 0x7fd52013c510 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd526fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52015c670 con 0x7fd5200cc100 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd526fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52015b7e0 con 0x7fd52006b8c0 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd52559c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 0x7fd5200c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd52559c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 0x7fd5200c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55196/0 (socket says 172.21.15.131:55196) 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd52559c700 1 -- 172.21.15.131:0/2512032392 learned_addr learned my addr 172.21.15.131:0/2512032392 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd524d9b700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 0x7fd52013bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd51ffff700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5200cc100 0x7fd52016d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd52559c700 1 -- 172.21.15.131:0/2512032392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 msgr2=0x7fd52013bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd52559c700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 0x7fd52013bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd52559c700 1 -- 172.21.15.131:0/2512032392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5200cc100 msgr2=0x7fd52016d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd52559c700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5200cc100 0x7fd52016d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.648 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.647+0000 7fd52559c700 1 -- 172.21.15.131:0/2512032392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5140093c0 con 0x7fd52013c510 2023-12-26T23:12:27.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.648+0000 7fd52559c700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 0x7fd5200c8e10 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fd51800ccc0 tx=0x7fd518001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:27.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.648+0000 7fd51cff9700 1 -- 172.21.15.131:0/2512032392 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51800a8a0 con 0x7fd52013c510 2023-12-26T23:12:27.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.648+0000 7fd51cff9700 1 -- 172.21.15.131:0/2512032392 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51800aa00 con 0x7fd52013c510 2023-12-26T23:12:27.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.648+0000 7fd51cff9700 1 -- 172.21.15.131:0/2512032392 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51800adc0 con 0x7fd52013c510 2023-12-26T23:12:27.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.648+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52015ba60 con 0x7fd52013c510 2023-12-26T23:12:27.649 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.648+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52015c020 con 0x7fd52013c510 2023-12-26T23:12:27.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.649+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52015c800 con 0x7fd52013c510 2023-12-26T23:12:27.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.650+0000 7fd51cff9700 1 -- 172.21.15.131:0/2512032392 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd518007300 con 0x7fd52013c510 2023-12-26T23:12:27.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.651+0000 7fd51cff9700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd504066f30 0x7fd5040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:27.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.651+0000 7fd51cff9700 1 -- 172.21.15.131:0/2512032392 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7fd518084a20 con 0x7fd52013c510 2023-12-26T23:12:27.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.651+0000 7fd524d9b700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd504066f30 0x7fd5040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:27.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.652+0000 7fd524d9b700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd504066f30 0x7fd5040693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5140091e0 tx=0x7fd514017000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:27.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.655+0000 7fd51cff9700 1 -- 172.21.15.131:0/2512032392 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd518051b00 con 0x7fd52013c510 2023-12-26T23:12:27.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.843+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5200657e0 con 0x7fd504066f30 2023-12-26T23:12:27.851 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.851+0000 7fd51cff9700 1 -- 172.21.15.131:0/2512032392 <== mgr.9927 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278857 (secure 0 0 0) 0x7fd5200657e0 con 0x7fd504066f30 2023-12-26T23:12:27.855 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd504066f30 msgr2=0x7fd5040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd504066f30 0x7fd5040693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5140091e0 tx=0x7fd514017000 comp rx=0 tx=0).stop 2023-12-26T23:12:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 msgr2=0x7fd5200c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 0x7fd5200c8e10 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fd51800ccc0 tx=0x7fd518001a80 comp rx=0 tx=0).stop 2023-12-26T23:12:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 shutdown_connections 2023-12-26T23:12:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd504066f30 0x7fd5040693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd52013c510 0x7fd5200c8e10 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd52006b8c0 0x7fd52013bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 --2- 172.21.15.131:0/2512032392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5200cc100 0x7fd52016d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:27.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.855+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 >> 172.21.15.131:0/2512032392 conn(0x7fd5200ce3e0 msgr2=0x7fd5200c4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:27.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.856+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 shutdown_connections 2023-12-26T23:12:27.858 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:27.856+0000 7fd526fff700 1 -- 172.21.15.131:0/2512032392 wait complete. 2023-12-26T23:12:27.858 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:12:27.873 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T23:12:26.065921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452856,"kb_used_data":21304,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752521864,"statfs":{"total":773094113280,"available":770582388736,"internally_reserved":0,"allocated":21815296,"data_stored":11557076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.498011"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575758+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:15.760243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574241+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:57:09.512915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573553+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:22.340338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575297+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:09:19.634386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573139+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:34.976485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.572849+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:52:38.429194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574047+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:28:29.479031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571063+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:12:08.105016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.577061+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:58:29.785862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573092+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:29:10.284887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.572820+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:56:57.660062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574872+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:15.760243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.577774+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:54:18.679739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571723+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:59:27.380222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573280+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:21:14.188510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.572056+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:10:59.783625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571197+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:18:38.340055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570022+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:49:13.753445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570023+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:21:16.422448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571821+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:51:46.017125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571197+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:02:34.976485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571284+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:24:22.340338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570043+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:31:20.574628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.570079+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:23:26.390563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575357+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:28:58.221939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.575560+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:09:19.634386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571687+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:00:04.850373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573973+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:04:23.335081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574139+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:40:08.598399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.573907+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:15:47.071802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.571197+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:51:22.275494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-26T23:12:21.567608+0000","last_change":"2023-12-26T23:12:21.567608+0000","last_active":"2023-12-26T23:12:21.567608+0000","last_peered":"2023-12-26T23:12:21.567608+0000","last_clean":"2023-12-26T23:12:21.567608+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:21.567608+0000","last_undegraded":"2023-12-26T23:12:21.567608+0000","last_fullsized":"2023-12-26T23:12:21.567608+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-12-26T23:12:20.561154+0000","last_change":"2023-12-26T23:12:20.574377+0000","last_active":"2023-12-26T23:12:20.561154+0000","last_peered":"2023-12-26T23:12:20.561154+0000","last_clean":"2023-12-26T23:12:20.561154+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-26T23:12:20.561154+0000","last_undegraded":"2023-12-26T23:12:20.561154+0000","last_fullsized":"2023-12-26T23:12:20.561154+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:28:58.221939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"199'348","reported_seq":1149,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-12-26T23:12:18.018701+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:12:18.018701+0000","last_peered":"2023-12-26T23:12:18.018701+0000","last_clean":"2023-12-26T23:12:18.018701+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:12:18.018701+0000","last_undegraded":"2023-12-26T23:12:18.018701+0000","last_fullsized":"2023-12-26T23:12:18.018701+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.56699999999999995,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.85499999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.54300000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90800000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73299999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.98599999999999999,"15min":0.995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.58599999999999997,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":0.89200000000000002,"5min":0.93300000000000005,"15min":1.0449999999999999},"last":1.038}]},{"osd":2,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.55500000000000005,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.86799999999999999,"5min":0.94199999999999995,"15min":1},"last":0.69999999999999996}]},{"osd":3,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.69399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95199999999999996,"5min":1.0029999999999999,"15min":1.0760000000000001},"last":0.999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.97599999999999998,"15min":0.98199999999999998},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.91400000000000003},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.39600000000000002,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.90600000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.98799999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.57499999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92800000000000005,"5min":0.998,"15min":1.042},"last":1.1359999999999999}]},{"osd":6,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.54800000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.88800000000000001}]}]},{"osd":6,"up_from":15,"seq":64424510575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.32800000000000001},"max":{"1min":0.873,"5min":0.998,"15min":1.1080000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0680000000000001},"last":0.80400000000000005}]},{"osd":1,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":1.022,"15min":1.046},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.95999999999999996,"15min":1.0580000000000001},"last":0.745}]},{"osd":2,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91500000000000004,"15min":1.071},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":0.89800000000000002,"5min":0.97899999999999998,"15min":1.0680000000000001},"last":0.622}]},{"osd":3,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.96699999999999997,"15min":1.024},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":1.0149999999999999},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.018,"15min":1.1599999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.111},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.66900000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.67400000000000004}]}]},{"osd":1,"up_from":15,"seq":64424510575,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300380,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071460,"statfs":{"total":96636764160,"available":96329175040,"internally_reserved":0,"allocated":3166208,"data_stored":1884248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.29199999999999998,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.81399999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.50600000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.622,"5min":0.76900000000000002,"15min":0.79200000000000004},"last":0.45100000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.33800000000000002},"max":{"1min":0.58099999999999996,"5min":0.753,"15min":0.84099999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.65800000000000003,"5min":0.75700000000000001,"15min":0.91200000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.70299999999999996,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.46700000000000003,"5min":0.40600000000000003,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.872,"15min":0.93600000000000005},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.629,"5min":0.55100000000000005,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":1.103,"15min":1.103},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.79100000000000004,"15min":0.77500000000000002},"min":{"1min":0.60899999999999999,"5min":0.52500000000000002,"15min":0.48599999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.57799999999999996}]},{"osd":5,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.50700000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.83199999999999996},"min":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.1100000000000001,"5min":1.1240000000000001,"15min":1.1830000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.753,"5min":0.57599999999999996,"15min":0.52600000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.627},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.85899999999999999,"15min":0.84299999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.64700000000000002,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.218},"last":0.67400000000000004}]}]},{"osd":0,"up_from":15,"seq":64424510575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.26600000000000001},"max":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.313},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.30599999999999999},"max":{"1min":0.72899999999999998,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.88100000000000001,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88,"5min":0.91800000000000004,"15min":1.012},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.2250000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77400000000000002,"15min":0.78300000000000003},"min":{"1min":0.66400000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.113},"last":0.85999999999999999}]},{"osd":5,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80400000000000005,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0469999999999999,"5min":1.0920000000000001,"15min":1.157},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.82899999999999996,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.282},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.79500000000000004,"15min":0.80500000000000005},"min":{"1min":0.72599999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.121,"5min":1.121,"15min":1.151},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81799999999999995,"15min":0.82199999999999995},"min":{"1min":0.67500000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1020000000000001,"5min":1.173,"15min":1.256},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.64200000000000002,"5min":0.54800000000000004,"15min":0.54000000000000004},"max":{"1min":1.087,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.879}]}]},{"osd":2,"up_from":14,"seq":60129543280,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312088,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059752,"statfs":{"total":96636764160,"available":96317186048,"internally_reserved":0,"allocated":2572288,"data_stored":1293880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":0.84599999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69199999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.45300000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.45800000000000002,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.64700000000000002,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.79200000000000004,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80200000000000005,"5min":0.89300000000000002,"15min":1.056},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.65600000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1180000000000001,"5min":1.8,"15min":1.8},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.59099999999999997,"15min":0.54000000000000004},"max":{"1min":1.115,"5min":1.222,"15min":1.222},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":1.079,"5min":1.923,"15min":1.923},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.087,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.55300000000000005,"15min":0.52400000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.272},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0269999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.77500000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.56399999999999995},"max":{"1min":1.042,"5min":1.1100000000000001,"15min":1.1619999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.84499999999999997},"min":{"1min":0.70999999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0740000000000001,"5min":1.841,"15min":1.841},"last":0.73599999999999999}]}]},{"osd":3,"up_from":15,"seq":64424510576,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303896,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067944,"statfs":{"total":96636764160,"available":96325574656,"internally_reserved":0,"allocated":2572288,"data_stored":1293880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46400000000000002,"15min":0.46100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.246},"max":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.245},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.746,"15min":0.76500000000000001},"last":0.26500000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47399999999999998,"15min":0.47999999999999998},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.753},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.307},"max":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47699999999999998}]},{"osd":2,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.751,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.079,"5min":1.177,"15min":1.177},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.58099999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.97799999999999998,"5min":1.0329999999999999,"15min":1.081},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.60499999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.0980000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.141},"last":0.63600000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.67700000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.84699999999999998},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.022,"5min":1.141,"15min":1.214},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.68700000000000006,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.71199999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312100,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059740,"statfs":{"total":96636764160,"available":96317173760,"internally_reserved":0,"allocated":2584576,"data_stored":1300205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.008,"15min":1.008},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71499999999999997,"15min":0.73699999999999999},"min":{"1min":0.55200000000000005,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":1.0329999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73299999999999998,"15min":0.75},"min":{"1min":0.58599999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.55100000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.998},"last":0.626}]},{"osd":3,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.51900000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84399999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.628,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":0.96799999999999997,"15min":0.98999999999999999},"last":0.83399999999999996}]},{"osd":6,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.73099999999999998,"5min":1.002,"15min":1.024},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.495,"15min":0.49199999999999999},"max":{"1min":0.77800000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":0.94099999999999995,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.71899999999999997}]}]},{"osd":5,"up_from":15,"seq":64424510574,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300380,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071460,"statfs":{"total":96636764160,"available":96329175040,"internally_reserved":0,"allocated":3166208,"data_stored":1884248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.60799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.92700000000000005,"15min":1.0629999999999999},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.016},"last":0.70699999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.012,"15min":1.022},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.99199999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":0.997,"15min":0.997},"last":0.74099999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93000000000000005,"15min":0.95199999999999996},"last":0.78900000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.97799999999999998,"15min":1.042},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.90600000000000003,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.90500000000000003,"15min":0.996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:12:27.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3f is not active+clean 2023-12-26T23:12:27.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3e is not active+clean 2023-12-26T23:12:27.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3d is not active+clean 2023-12-26T23:12:27.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3c is not active+clean 2023-12-26T23:12:27.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3b is not active+clean 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3a is not active+clean 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.39 is not active+clean 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.38 is not active+clean 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.37 is not active+clean 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.36 is not active+clean 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.35 is not active+clean 2023-12-26T23:12:27.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.34 is not active+clean 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.33 is not active+clean 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.32 is not active+clean 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.31 is not active+clean 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.30 is not active+clean 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2f is not active+clean 2023-12-26T23:12:27.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2e is not active+clean 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2d is not active+clean 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2c is not active+clean 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2b is not active+clean 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2a is not active+clean 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.29 is not active+clean 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.28 is not active+clean 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.27 is not active+clean 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.26 is not active+clean 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.25 is not active+clean 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.24 is not active+clean 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.23 is not active+clean 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.22 is not active+clean 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.21 is not active+clean 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.20 is not active+clean 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1e is not active+clean 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1f is not active+clean 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.575758+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:12:15.760243+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1f is not active+clean 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1e is not active+clean 2023-12-26T23:12:27.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.574241+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:57:09.512915+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1c is not active+clean 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1d is not active+clean 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.573553+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:24:22.340338+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1d is not active+clean 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1c is not active+clean 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.575297+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T10:09:19.634386+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1a is not active+clean 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1b is not active+clean 2023-12-26T23:12:27.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.573139+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:02:34.976485+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1b is not active+clean 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1a is not active+clean 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.572849+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:52:38.429194+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.19 is not active+clean 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.18 is not active+clean 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.574047+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T06:28:29.479031+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.16 is not active+clean 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.17 is not active+clean 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.571063+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T11:12:08.105016+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T23:12:27.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.17 is not active+clean 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.16 is not active+clean 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.577061+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:58:29.785862+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.14 is not active+clean 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.15 is not active+clean 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.573092+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:29:10.284887+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.15 is not active+clean 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.14 is not active+clean 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.572820+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:56:57.660062+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.12 is not active+clean 2023-12-26T23:12:27.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.13 is not active+clean 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.574872+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:12:15.760243+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.13 is not active+clean 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.12 is not active+clean 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.577774+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:54:18.679739+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.10 is not active+clean 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.11 is not active+clean 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.571723+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:59:27.380222+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.11 is not active+clean 2023-12-26T23:12:27.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.10 is not active+clean 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.573280+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:21:14.188510+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.e is not active+clean 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.f is not active+clean 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.572056+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:10:59.783625+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.f is not active+clean 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.e is not active+clean 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.571197+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:18:38.340055+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.c is not active+clean 2023-12-26T23:12:27.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.d is not active+clean 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.570022+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T05:49:13.753445+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.d is not active+clean 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.c is not active+clean 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.570023+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:21:16.422448+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.a is not active+clean 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.b is not active+clean 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.571821+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T04:51:46.017125+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.b is not active+clean 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.a is not active+clean 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.571197+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:02:34.976485+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.8 is not active+clean 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.9 is not active+clean 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.571284+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T01:24:22.340338+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.9 is not active+clean 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.8 is not active+clean 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.570043+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:31:20.574628+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.6 is not active+clean 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.7 is not active+clean 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.570079+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T00:23:26.390563+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.7 is not active+clean 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.6 is not active+clean 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.575357+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T10:28:58.221939+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.4 is not active+clean 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.5 is not active+clean 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.575560+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T10:09:19.634386+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.5 is not active+clean 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.4 is not active+clean 2023-12-26T23:12:27.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.571687+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T08:00:04.850373+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2 is not active+clean 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3 is not active+clean 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.573973+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T02:04:23.335081+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3 is not active+clean 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2 is not active+clean 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.574139+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:40:08.598399+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.0 is not active+clean 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1 is not active+clean 2023-12-26T23:12:27.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.573907+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-27T23:15:47.071802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-26T23:12:27.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1 is not active+clean 2023-12-26T23:12:27.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.0 is not active+clean 2023-12-26T23:12:27.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.571197+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T09:51:22.275494+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-26T23:12:27.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.18 is not active+clean 2023-12-26T23:12:27.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-26T23:12:21.567608+0000', 'last_change': '2023-12-26T23:12:21.567608+0000', 'last_active': '2023-12-26T23:12:21.567608+0000', 'last_peered': '2023-12-26T23:12:21.567608+0000', 'last_clean': '2023-12-26T23:12:21.567608+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:21.567608+0000', 'last_undegraded': '2023-12-26T23:12:21.567608+0000', 'last_fullsized': '2023-12-26T23:12:21.567608+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:21.567608+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-26T23:12:27.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.19 is not active+clean 2023-12-26T23:12:27.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-12-26T23:12:20.561154+0000', 'last_change': '2023-12-26T23:12:20.574377+0000', 'last_active': '2023-12-26T23:12:20.561154+0000', 'last_peered': '2023-12-26T23:12:20.561154+0000', 'last_clean': '2023-12-26T23:12:20.561154+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-26T23:12:20.561154+0000', 'last_undegraded': '2023-12-26T23:12:20.561154+0000', 'last_fullsized': '2023-12-26T23:12:20.561154+0000', 'mapping_epoch': 207, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 207, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'last_clean_scrub_stamp': '2023-12-26T23:12:20.561154+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-28T10:28:58.221939+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-26T23:12:27.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-26T23:12:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.101+0000 7fe255bff700 1 Processor -- start 2023-12-26T23:12:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.102+0000 7fe255bff700 1 -- start start 2023-12-26T23:12:28.103 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.103+0000 7fe255bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2501596a0 0x7fe2500b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.103+0000 7fe255bff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2500afcd0 con 0x7fe250159a70 2023-12-26T23:12:28.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.103+0000 7fe255bff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2500b3e70 con 0x7fe2500b0870 2023-12-26T23:12:28.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.103+0000 7fe255bff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2500b0cb0 con 0x7fe2501596a0 2023-12-26T23:12:28.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.103+0000 7fe247fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe250159a70 0x7fe2500b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32846/0 (socket says 172.21.15.131:32846) 2023-12-26T23:12:28.104 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.103+0000 7fe247fff700 1 -- 172.21.15.131:0/49331006 learned_addr learned my addr 172.21.15.131:0/49331006 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:28.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.103+0000 7fe24f7fe700 1 --2- 172.21.15.131:0/49331006 >> v2:172.21.15.190:3300/0 conn(0x7fe2501596a0 0x7fe2500b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:28.105 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.104+0000 7fe24effd700 1 -- 172.21.15.131:0/49331006 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522100633 0 0) 0x7fe2500afcd0 con 0x7fe250159a70 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.104+0000 7fe24effd700 1 -- 172.21.15.131:0/49331006 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe23c003570 con 0x7fe250159a70 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.104+0000 7fe24effd700 1 -- 172.21.15.131:0/49331006 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437285336 0 0) 0x7fe2500b3e70 con 0x7fe2500b0870 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.104+0000 7fe24f7fe700 1 -- 172.21.15.131:0/49331006 >> v1:172.21.15.190:6789/0 conn(0x7fe2500b0870 legacy=0x7fe2500b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.104+0000 7fe24f7fe700 1 -- 172.21.15.131:0/49331006 >> v1:172.21.15.131:6789/0 conn(0x7fe250159a70 legacy=0x7fe2500b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.104+0000 7fe24f7fe700 1 -- 172.21.15.131:0/49331006 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2500b0f20 con 0x7fe2501596a0 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.104+0000 7fe24f7fe700 1 --2- 172.21.15.131:0/49331006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2501596a0 0x7fe2500b2380 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe2400098e0 tx=0x7fe2400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1936bcc84c6d86f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.104+0000 7fe24effd700 1 -- 172.21.15.131:0/49331006 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000b640 con 0x7fe2501596a0 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.104+0000 7fe24effd700 1 -- 172.21.15.131:0/49331006 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24000b7a0 con 0x7fe2501596a0 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.105+0000 7fe24effd700 1 -- 172.21.15.131:0/49331006 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000bb20 con 0x7fe2501596a0 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.105+0000 7fe255bff700 1 -- 172.21.15.131:0/49331006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2501596a0 msgr2=0x7fe2500b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.105+0000 7fe255bff700 1 --2- 172.21.15.131:0/49331006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2501596a0 0x7fe2500b2380 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe2400098e0 tx=0x7fe2400091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.105+0000 7fe255bff700 1 -- 172.21.15.131:0/49331006 shutdown_connections 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.105+0000 7fe255bff700 1 --2- 172.21.15.131:0/49331006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2501596a0 0x7fe2500b2380 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.105+0000 7fe255bff700 1 -- 172.21.15.131:0/49331006 >> 172.21.15.131:0/49331006 conn(0x7fe25006ece0 msgr2=0x7fe25006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:28.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.105+0000 7fe255bff700 1 -- 172.21.15.131:0/49331006 shutdown_connections 2023-12-26T23:12:28.108 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.105+0000 7fe255bff700 1 -- 172.21.15.131:0/49331006 wait complete. 2023-12-26T23:12:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.107+0000 7fe255bff700 1 Processor -- start 2023-12-26T23:12:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.107+0000 7fe255bff700 1 -- start start 2023-12-26T23:12:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe255bff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2500b0870 0x7fe250145e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe255bff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2501596a0 0x7fe2500c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe255bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe250159a70 0x7fe2500c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe255bff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe250169c10 con 0x7fe2500b0870 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe255bff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe250169ef0 con 0x7fe250159a70 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe255bff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25016a1d0 con 0x7fe2501596a0 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe24f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2500b0870 0x7fe250145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe24f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2500b0870 0x7fe250145e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55204/0 (socket says 172.21.15.131:55204) 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe24f7fe700 1 -- 172.21.15.131:0/1846762780 learned_addr learned my addr 172.21.15.131:0/1846762780 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe24f7fe700 1 -- 172.21.15.131:0/1846762780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2501596a0 msgr2=0x7fe2500c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe24f7fe700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2501596a0 0x7fe2500c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe24f7fe700 1 -- 172.21.15.131:0/1846762780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe250159a70 msgr2=0x7fe2500c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe24f7fe700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe250159a70 0x7fe2500c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.108+0000 7fe24f7fe700 1 -- 172.21.15.131:0/1846762780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe240009420 con 0x7fe2500b0870 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.109+0000 7fe24f7fe700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2500b0870 0x7fe250145e00 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe250089710 tx=0x7fe240017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.109+0000 7fe2477fe700 1 -- 172.21.15.131:0/1846762780 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000bb20 con 0x7fe2500b0870 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.109+0000 7fe2477fe700 1 -- 172.21.15.131:0/1846762780 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24001a450 con 0x7fe2500b0870 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.109+0000 7fe2477fe700 1 -- 172.21.15.131:0/1846762780 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24001a7d0 con 0x7fe2500b0870 2023-12-26T23:12:28.111 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.109+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25016a4b0 con 0x7fe2500b0870 2023-12-26T23:12:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.109+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe250146340 con 0x7fe2500b0870 2023-12-26T23:12:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.110+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2500c9300 con 0x7fe2500b0870 2023-12-26T23:12:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.111+0000 7fe2477fe700 1 -- 172.21.15.131:0/1846762780 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe24001ac00 con 0x7fe2500b0870 2023-12-26T23:12:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.112+0000 7fe2477fe700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe230066f30 0x7fe2300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.112+0000 7fe2477fe700 1 -- 172.21.15.131:0/1846762780 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7fe2400861b0 con 0x7fe2500b0870 2023-12-26T23:12:28.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.112+0000 7fe247fff700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe230066f30 0x7fe2300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:28.113 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.113+0000 7fe247fff700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe230066f30 0x7fe2300693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2380058d0 tx=0x7fe2380079d0 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:28.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.116+0000 7fe2477fe700 1 -- 172.21.15.131:0/1846762780 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe240053220 con 0x7fe2500b0870 2023-12-26T23:12:28.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.383+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe2500657e0 con 0x7fe2500b0870 2023-12-26T23:12:28.383 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.383+0000 7fe2477fe700 1 -- 172.21.15.131:0/1846762780 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1117 (secure 0 0 0) 0x7fe240056840 con 0x7fe2500b0870 2023-12-26T23:12:28.383 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:28.383 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"ab603302-d88b-429a-be64-92b3c298adad","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5764,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":210,"num_osds":8,"num_up_osds":8,"osd_up_since":1703626596,"num_in_osds":8,"osd_in_since":1703626587,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2511724544,"bytes_avail":770582388736,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":98,"id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"d","status":"up:active","gid":10125}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2023-12-26T23:11:04.732025+0000","services":{}},"progress_events":{}} 2023-12-26T23:12:28.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe230066f30 msgr2=0x7fe2300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe230066f30 0x7fe2300693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2380058d0 tx=0x7fe2380079d0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2500b0870 msgr2=0x7fe250145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2500b0870 0x7fe250145e00 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe250089710 tx=0x7fe240017000 comp rx=0 tx=0).stop 2023-12-26T23:12:28.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 shutdown_connections 2023-12-26T23:12:28.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe230066f30 0x7fe2300693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe250159a70 0x7fe2500c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2500b0870 0x7fe250145e00 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 --2- 172.21.15.131:0/1846762780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2501596a0 0x7fe2500c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.385+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 >> 172.21.15.131:0/1846762780 conn(0x7fe25006ece0 msgr2=0x7fe250158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:28.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.386+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 shutdown_connections 2023-12-26T23:12:28.387 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.386+0000 7fe255bff700 1 -- 172.21.15.131:0/1846762780 wait complete. 2023-12-26T23:12:28.403 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-26T23:12:28.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:12:28.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.592+0000 7f7df3fff700 1 Processor -- start 2023-12-26T23:12:28.592 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.592+0000 7f7df3fff700 1 -- start start 2023-12-26T23:12:28.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.593+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0cb320 0x7f7dec069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.593+0000 7f7df3fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7dec06a1e0 0x7f7dec0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.593+0000 7f7df3fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dec156940 con 0x7f7dec0b0c90 2023-12-26T23:12:28.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.593+0000 7f7df3fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dec0c8cc0 con 0x7f7dec06a1e0 2023-12-26T23:12:28.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.593+0000 7f7df3fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec0ad700 con 0x7f7dec0cb320 2023-12-26T23:12:28.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df159a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7dec06a1e0 0x7f7dec0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:28.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df159a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7dec06a1e0 0x7f7dec0b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55212/0 (socket says 172.21.15.131:55212) 2023-12-26T23:12:28.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df159a700 1 -- 172.21.15.131:0/114301343 learned_addr learned my addr 172.21.15.131:0/114301343 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:28.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df259c700 1 --1- 172.21.15.131:0/114301343 >> v1:172.21.15.131:6789/0 conn(0x7f7dec0b0c90 0x7f7dec0aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32854/0 (socket says 172.21.15.131:32854) 2023-12-26T23:12:28.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df1d9b700 1 --2- 172.21.15.131:0/114301343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0cb320 0x7f7dec069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:28.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df159a700 1 -- 172.21.15.131:0/114301343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0cb320 msgr2=0x7f7dec069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:28.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df159a700 1 --2- 172.21.15.131:0/114301343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0cb320 0x7f7dec069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df159a700 1 -- 172.21.15.131:0/114301343 >> v1:172.21.15.131:6789/0 conn(0x7f7dec0b0c90 legacy=0x7f7dec0aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df159a700 1 -- 172.21.15.131:0/114301343 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0ae500 con 0x7f7dec06a1e0 2023-12-26T23:12:28.595 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df0d99700 1 -- 172.21.15.131:0/114301343 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201134189 0 0) 0x7f7dec156940 con 0x7f7dec0b0c90 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.594+0000 7f7df159a700 1 --2- 172.21.15.131:0/114301343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec06a1e0 0x7f7dec0b0750 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0099b0 tx=0x7f7ddc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f9d1adae841893 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df0d99700 1 -- 172.21.15.131:0/114301343 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00b860 con 0x7f7dec06a1e0 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df0d99700 1 -- 172.21.15.131:0/114301343 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc00b9c0 con 0x7f7dec06a1e0 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df0d99700 1 -- 172.21.15.131:0/114301343 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00bd60 con 0x7f7dec06a1e0 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df3fff700 1 -- 172.21.15.131:0/114301343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec06a1e0 msgr2=0x7f7dec0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df3fff700 1 --2- 172.21.15.131:0/114301343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec06a1e0 0x7f7dec0b0750 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0099b0 tx=0x7f7ddc009250 comp rx=0 tx=0).stop 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df3fff700 1 -- 172.21.15.131:0/114301343 shutdown_connections 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df3fff700 1 --2- 172.21.15.131:0/114301343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0cb320 0x7f7dec069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df3fff700 1 --2- 172.21.15.131:0/114301343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec06a1e0 0x7f7dec0b0750 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df3fff700 1 -- 172.21.15.131:0/114301343 >> 172.21.15.131:0/114301343 conn(0x7f7dec068040 msgr2=0x7f7dec069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:28.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df3fff700 1 -- 172.21.15.131:0/114301343 shutdown_connections 2023-12-26T23:12:28.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.595+0000 7f7df3fff700 1 -- 172.21.15.131:0/114301343 wait complete. 2023-12-26T23:12:28.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.597+0000 7f7df3fff700 1 Processor -- start 2023-12-26T23:12:28.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.597+0000 7f7df3fff700 1 -- start start 2023-12-26T23:12:28.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dec06a1e0 0x7f7dec138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0b0c90 0x7f7dec153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec0cb320 0x7f7dec1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df3fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec1520e0 con 0x7f7dec0cb320 2023-12-26T23:12:28.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df3fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec1523c0 con 0x7f7dec06a1e0 2023-12-26T23:12:28.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df3fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec1526a0 con 0x7f7dec0b0c90 2023-12-26T23:12:28.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec0cb320 0x7f7dec1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:28.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec0cb320 0x7f7dec1516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55224/0 (socket says 172.21.15.131:55224) 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df259c700 1 -- 172.21.15.131:0/891073465 learned_addr learned my addr 172.21.15.131:0/891073465 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df1d9b700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dec06a1e0 0x7f7dec138780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df159a700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0b0c90 0x7f7dec153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df259c700 1 -- 172.21.15.131:0/891073465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0b0c90 msgr2=0x7f7dec153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df259c700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0b0c90 0x7f7dec153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df259c700 1 -- 172.21.15.131:0/891073465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dec06a1e0 msgr2=0x7f7dec138780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df259c700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dec06a1e0 0x7f7dec138780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.598+0000 7f7df259c700 1 -- 172.21.15.131:0/891073465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0094f0 con 0x7f7dec0cb320 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.599+0000 7f7df259c700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec0cb320 0x7f7dec1516a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f7de400cd00 tx=0x7f7de4001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.599+0000 7f7de1ffb700 1 -- 172.21.15.131:0/891073465 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4005df0 con 0x7f7dec0cb320 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.599+0000 7f7de1ffb700 1 -- 172.21.15.131:0/891073465 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de400a960 con 0x7f7dec0cb320 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.599+0000 7f7de1ffb700 1 -- 172.21.15.131:0/891073465 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de400ad20 con 0x7f7dec0cb320 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.599+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dec152980 con 0x7f7dec0cb320 2023-12-26T23:12:28.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.599+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dec138cc0 con 0x7f7dec0cb320 2023-12-26T23:12:28.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.600+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dec0657e0 con 0x7f7dec0cb320 2023-12-26T23:12:28.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.601+0000 7f7de1ffb700 1 -- 172.21.15.131:0/891073465 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7de4007300 con 0x7f7dec0cb320 2023-12-26T23:12:28.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.602+0000 7f7de1ffb700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7dd0066f30 0x7f7dd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:28.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.602+0000 7f7de1ffb700 1 -- 172.21.15.131:0/891073465 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f7de40848f0 con 0x7f7dec0cb320 2023-12-26T23:12:28.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.602+0000 7f7df1d9b700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7dd0066f30 0x7f7dd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:28.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.603+0000 7f7df1d9b700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7dd0066f30 0x7f7dd00693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7dec13c120 tx=0x7f7de8008040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:28.606 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.606+0000 7f7de1ffb700 1 -- 172.21.15.131:0/891073465 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7de40519d0 con 0x7f7dec0cb320 2023-12-26T23:12:28.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.795+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7dec152530 con 0x7f7dd0066f30 2023-12-26T23:12:28.803 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.803+0000 7f7de1ffb700 1 -- 172.21.15.131:0/891073465 <== mgr.9927 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289828 (secure 0 0 0) 0x7f7dec152530 con 0x7f7dd0066f30 2023-12-26T23:12:28.807 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:28.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.807+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7dd0066f30 msgr2=0x7f7dd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7dd0066f30 0x7f7dd00693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7dec13c120 tx=0x7f7de8008040 comp rx=0 tx=0).stop 2023-12-26T23:12:28.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec0cb320 msgr2=0x7f7dec1516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec0cb320 0x7f7dec1516a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f7de400cd00 tx=0x7f7de4001ac0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 shutdown_connections 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7dd0066f30 0x7f7dd00693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dec0cb320 0x7f7dec1516a0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dec06a1e0 0x7f7dec138780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 --2- 172.21.15.131:0/891073465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dec0b0c90 0x7f7dec153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 >> 172.21.15.131:0/891073465 conn(0x7f7dec068040 msgr2=0x7f7dec157b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 shutdown_connections 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:28.808+0000 7f7df3fff700 1 -- 172.21.15.131:0/891073465 wait complete. 2023-12-26T23:12:28.810 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:12:28.825 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-26T23:12:28.066288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":623,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453256,"kb_used_data":21704,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752521464,"statfs":{"total":773094113280,"available":770581979136,"internally_reserved":0,"allocated":22224896,"data_stored":11753980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.490444"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.619284+0000","last_change":"2023-12-26T23:12:22.618470+0000","last_active":"2023-12-26T23:12:22.619284+0000","last_peered":"2023-12-26T23:12:22.619284+0000","last_clean":"2023-12-26T23:12:22.619284+0000","last_became_active":"2023-12-26T23:12:22.617303+0000","last_became_peered":"2023-12-26T23:12:22.617303+0000","last_unstale":"2023-12-26T23:12:22.619284+0000","last_undegraded":"2023-12-26T23:12:22.619284+0000","last_fullsized":"2023-12-26T23:12:22.619284+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:59:09.130170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.616029+0000","last_change":"2023-12-26T23:12:22.614091+0000","last_active":"2023-12-26T23:12:22.616029+0000","last_peered":"2023-12-26T23:12:22.616029+0000","last_clean":"2023-12-26T23:12:22.616029+0000","last_became_active":"2023-12-26T23:12:22.613796+0000","last_became_peered":"2023-12-26T23:12:22.613796+0000","last_unstale":"2023-12-26T23:12:22.616029+0000","last_undegraded":"2023-12-26T23:12:22.616029+0000","last_fullsized":"2023-12-26T23:12:22.616029+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:02:50.533341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612431+0000","last_change":"2023-12-26T23:12:22.598911+0000","last_active":"2023-12-26T23:12:22.612431+0000","last_peered":"2023-12-26T23:12:22.612431+0000","last_clean":"2023-12-26T23:12:22.612431+0000","last_became_active":"2023-12-26T23:12:22.594770+0000","last_became_peered":"2023-12-26T23:12:22.594770+0000","last_unstale":"2023-12-26T23:12:22.612431+0000","last_undegraded":"2023-12-26T23:12:22.612431+0000","last_fullsized":"2023-12-26T23:12:22.612431+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:25:21.227639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.610093+0000","last_change":"2023-12-26T23:12:22.603716+0000","last_active":"2023-12-26T23:12:22.610093+0000","last_peered":"2023-12-26T23:12:22.610093+0000","last_clean":"2023-12-26T23:12:22.610093+0000","last_became_active":"2023-12-26T23:12:22.602197+0000","last_became_peered":"2023-12-26T23:12:22.602197+0000","last_unstale":"2023-12-26T23:12:22.610093+0000","last_undegraded":"2023-12-26T23:12:22.610093+0000","last_fullsized":"2023-12-26T23:12:22.610093+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:34:42.233229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.605074+0000","last_change":"2023-12-26T23:12:22.603086+0000","last_active":"2023-12-26T23:12:22.605074+0000","last_peered":"2023-12-26T23:12:22.605074+0000","last_clean":"2023-12-26T23:12:22.605074+0000","last_became_active":"2023-12-26T23:12:22.602382+0000","last_became_peered":"2023-12-26T23:12:22.602382+0000","last_unstale":"2023-12-26T23:12:22.605074+0000","last_undegraded":"2023-12-26T23:12:22.605074+0000","last_fullsized":"2023-12-26T23:12:22.605074+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:36:44.985279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.606178+0000","last_change":"2023-12-26T23:12:22.601550+0000","last_active":"2023-12-26T23:12:22.606178+0000","last_peered":"2023-12-26T23:12:22.606178+0000","last_clean":"2023-12-26T23:12:22.606178+0000","last_became_active":"2023-12-26T23:12:22.601268+0000","last_became_peered":"2023-12-26T23:12:22.601268+0000","last_unstale":"2023-12-26T23:12:22.606178+0000","last_undegraded":"2023-12-26T23:12:22.606178+0000","last_fullsized":"2023-12-26T23:12:22.606178+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:39.054404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604731+0000","last_change":"2023-12-26T23:12:22.601334+0000","last_active":"2023-12-26T23:12:22.604731+0000","last_peered":"2023-12-26T23:12:22.604731+0000","last_clean":"2023-12-26T23:12:22.604731+0000","last_became_active":"2023-12-26T23:12:22.601066+0000","last_became_peered":"2023-12-26T23:12:22.601066+0000","last_unstale":"2023-12-26T23:12:22.604731+0000","last_undegraded":"2023-12-26T23:12:22.604731+0000","last_fullsized":"2023-12-26T23:12:22.604731+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:56:50.812363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.611742+0000","last_change":"2023-12-26T23:12:22.605627+0000","last_active":"2023-12-26T23:12:22.611742+0000","last_peered":"2023-12-26T23:12:22.611742+0000","last_clean":"2023-12-26T23:12:22.611742+0000","last_became_active":"2023-12-26T23:12:22.605384+0000","last_became_peered":"2023-12-26T23:12:22.605384+0000","last_unstale":"2023-12-26T23:12:22.611742+0000","last_undegraded":"2023-12-26T23:12:22.611742+0000","last_fullsized":"2023-12-26T23:12:22.611742+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:19:55.261546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612225+0000","last_change":"2023-12-26T23:12:22.605430+0000","last_active":"2023-12-26T23:12:22.612225+0000","last_peered":"2023-12-26T23:12:22.612225+0000","last_clean":"2023-12-26T23:12:22.612225+0000","last_became_active":"2023-12-26T23:12:22.604985+0000","last_became_peered":"2023-12-26T23:12:22.604985+0000","last_unstale":"2023-12-26T23:12:22.612225+0000","last_undegraded":"2023-12-26T23:12:22.612225+0000","last_fullsized":"2023-12-26T23:12:22.612225+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:57:10.519369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621526+0000","last_change":"2023-12-26T23:12:22.616480+0000","last_active":"2023-12-26T23:12:22.621526+0000","last_peered":"2023-12-26T23:12:22.621526+0000","last_clean":"2023-12-26T23:12:22.621526+0000","last_became_active":"2023-12-26T23:12:22.616238+0000","last_became_peered":"2023-12-26T23:12:22.616238+0000","last_unstale":"2023-12-26T23:12:22.621526+0000","last_undegraded":"2023-12-26T23:12:22.621526+0000","last_fullsized":"2023-12-26T23:12:22.621526+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:35:39.625849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608917+0000","last_change":"2023-12-26T23:12:22.601320+0000","last_active":"2023-12-26T23:12:22.608917+0000","last_peered":"2023-12-26T23:12:22.608917+0000","last_clean":"2023-12-26T23:12:22.608917+0000","last_became_active":"2023-12-26T23:12:22.601045+0000","last_became_peered":"2023-12-26T23:12:22.601045+0000","last_unstale":"2023-12-26T23:12:22.608917+0000","last_undegraded":"2023-12-26T23:12:22.608917+0000","last_fullsized":"2023-12-26T23:12:22.608917+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:08:36.371857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603091+0000","last_change":"2023-12-26T23:12:22.598407+0000","last_active":"2023-12-26T23:12:22.603091+0000","last_peered":"2023-12-26T23:12:22.603091+0000","last_clean":"2023-12-26T23:12:22.603091+0000","last_became_active":"2023-12-26T23:12:22.598072+0000","last_became_peered":"2023-12-26T23:12:22.598072+0000","last_unstale":"2023-12-26T23:12:22.603091+0000","last_undegraded":"2023-12-26T23:12:22.603091+0000","last_fullsized":"2023-12-26T23:12:22.603091+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:22.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.615088+0000","last_change":"2023-12-26T23:12:22.610046+0000","last_active":"2023-12-26T23:12:22.615088+0000","last_peered":"2023-12-26T23:12:22.615088+0000","last_clean":"2023-12-26T23:12:22.615088+0000","last_became_active":"2023-12-26T23:12:22.609779+0000","last_became_peered":"2023-12-26T23:12:22.609779+0000","last_unstale":"2023-12-26T23:12:22.615088+0000","last_undegraded":"2023-12-26T23:12:22.615088+0000","last_fullsized":"2023-12-26T23:12:22.615088+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:06:42.889746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.619149+0000","last_change":"2023-12-26T23:12:22.616561+0000","last_active":"2023-12-26T23:12:22.619149+0000","last_peered":"2023-12-26T23:12:22.619149+0000","last_clean":"2023-12-26T23:12:22.619149+0000","last_became_active":"2023-12-26T23:12:22.616312+0000","last_became_peered":"2023-12-26T23:12:22.616312+0000","last_unstale":"2023-12-26T23:12:22.619149+0000","last_undegraded":"2023-12-26T23:12:22.619149+0000","last_fullsized":"2023-12-26T23:12:22.619149+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:28:59.228392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608685+0000","last_change":"2023-12-26T23:12:22.603229+0000","last_active":"2023-12-26T23:12:22.608685+0000","last_peered":"2023-12-26T23:12:22.608685+0000","last_clean":"2023-12-26T23:12:22.608685+0000","last_became_active":"2023-12-26T23:12:22.602956+0000","last_became_peered":"2023-12-26T23:12:22.602956+0000","last_unstale":"2023-12-26T23:12:22.608685+0000","last_undegraded":"2023-12-26T23:12:22.608685+0000","last_fullsized":"2023-12-26T23:12:22.608685+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:06:35.396701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608290+0000","last_change":"2023-12-26T23:12:22.594739+0000","last_active":"2023-12-26T23:12:22.608290+0000","last_peered":"2023-12-26T23:12:22.608290+0000","last_clean":"2023-12-26T23:12:22.608290+0000","last_became_active":"2023-12-26T23:12:22.594453+0000","last_became_peered":"2023-12-26T23:12:22.594453+0000","last_unstale":"2023-12-26T23:12:22.608290+0000","last_undegraded":"2023-12-26T23:12:22.608290+0000","last_fullsized":"2023-12-26T23:12:22.608290+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:22.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621517+0000","last_change":"2023-12-26T23:12:22.620583+0000","last_active":"2023-12-26T23:12:22.621517+0000","last_peered":"2023-12-26T23:12:22.621517+0000","last_clean":"2023-12-26T23:12:22.621517+0000","last_became_active":"2023-12-26T23:12:22.619749+0000","last_became_peered":"2023-12-26T23:12:22.619749+0000","last_unstale":"2023-12-26T23:12:22.621517+0000","last_undegraded":"2023-12-26T23:12:22.621517+0000","last_fullsized":"2023-12-26T23:12:22.621517+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:26:45.615428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.610988+0000","last_change":"2023-12-26T23:12:22.604727+0000","last_active":"2023-12-26T23:12:22.610988+0000","last_peered":"2023-12-26T23:12:22.610988+0000","last_clean":"2023-12-26T23:12:22.610988+0000","last_became_active":"2023-12-26T23:12:22.604470+0000","last_became_peered":"2023-12-26T23:12:22.604470+0000","last_unstale":"2023-12-26T23:12:22.610988+0000","last_undegraded":"2023-12-26T23:12:22.610988+0000","last_fullsized":"2023-12-26T23:12:22.610988+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:22:40.411083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612470+0000","last_change":"2023-12-26T23:12:22.600667+0000","last_active":"2023-12-26T23:12:22.612470+0000","last_peered":"2023-12-26T23:12:22.612470+0000","last_clean":"2023-12-26T23:12:22.612470+0000","last_became_active":"2023-12-26T23:12:22.600410+0000","last_became_peered":"2023-12-26T23:12:22.600410+0000","last_unstale":"2023-12-26T23:12:22.612470+0000","last_undegraded":"2023-12-26T23:12:22.612470+0000","last_fullsized":"2023-12-26T23:12:22.612470+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:46:04.681070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.600465+0000","last_change":"2023-12-26T23:12:22.597138+0000","last_active":"2023-12-26T23:12:22.600465+0000","last_peered":"2023-12-26T23:12:22.600465+0000","last_clean":"2023-12-26T23:12:22.600465+0000","last_became_active":"2023-12-26T23:12:22.596819+0000","last_became_peered":"2023-12-26T23:12:22.596819+0000","last_unstale":"2023-12-26T23:12:22.600465+0000","last_undegraded":"2023-12-26T23:12:22.600465+0000","last_fullsized":"2023-12-26T23:12:22.600465+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:48.264386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.609802+0000","last_change":"2023-12-26T23:12:22.598384+0000","last_active":"2023-12-26T23:12:22.609802+0000","last_peered":"2023-12-26T23:12:22.609802+0000","last_clean":"2023-12-26T23:12:22.609802+0000","last_became_active":"2023-12-26T23:12:22.598087+0000","last_became_peered":"2023-12-26T23:12:22.598087+0000","last_unstale":"2023-12-26T23:12:22.609802+0000","last_undegraded":"2023-12-26T23:12:22.609802+0000","last_fullsized":"2023-12-26T23:12:22.609802+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:06:33.817797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604041+0000","last_change":"2023-12-26T23:12:22.598903+0000","last_active":"2023-12-26T23:12:22.604041+0000","last_peered":"2023-12-26T23:12:22.604041+0000","last_clean":"2023-12-26T23:12:22.604041+0000","last_became_active":"2023-12-26T23:12:22.598650+0000","last_became_peered":"2023-12-26T23:12:22.598650+0000","last_unstale":"2023-12-26T23:12:22.604041+0000","last_undegraded":"2023-12-26T23:12:22.604041+0000","last_fullsized":"2023-12-26T23:12:22.604041+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:00:16.909284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621055+0000","last_change":"2023-12-26T23:12:22.613831+0000","last_active":"2023-12-26T23:12:22.621055+0000","last_peered":"2023-12-26T23:12:22.621055+0000","last_clean":"2023-12-26T23:12:22.621055+0000","last_became_active":"2023-12-26T23:12:22.613551+0000","last_became_peered":"2023-12-26T23:12:22.613551+0000","last_unstale":"2023-12-26T23:12:22.621055+0000","last_undegraded":"2023-12-26T23:12:22.621055+0000","last_fullsized":"2023-12-26T23:12:22.621055+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:26:27.918216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.596011+0000","last_change":"2023-12-26T23:12:22.592199+0000","last_active":"2023-12-26T23:12:22.596011+0000","last_peered":"2023-12-26T23:12:22.596011+0000","last_clean":"2023-12-26T23:12:22.596011+0000","last_became_active":"2023-12-26T23:12:22.591939+0000","last_became_peered":"2023-12-26T23:12:22.591939+0000","last_unstale":"2023-12-26T23:12:22.596011+0000","last_undegraded":"2023-12-26T23:12:22.596011+0000","last_fullsized":"2023-12-26T23:12:22.596011+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:06:39.203393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618290+0000","last_change":"2023-12-26T23:12:22.616401+0000","last_active":"2023-12-26T23:12:22.618290+0000","last_peered":"2023-12-26T23:12:22.618290+0000","last_clean":"2023-12-26T23:12:22.618290+0000","last_became_active":"2023-12-26T23:12:22.614857+0000","last_became_peered":"2023-12-26T23:12:22.614857+0000","last_unstale":"2023-12-26T23:12:22.618290+0000","last_undegraded":"2023-12-26T23:12:22.618290+0000","last_fullsized":"2023-12-26T23:12:22.618290+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:52.603246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.605824+0000","last_change":"2023-12-26T23:12:22.600468+0000","last_active":"2023-12-26T23:12:22.605824+0000","last_peered":"2023-12-26T23:12:22.605824+0000","last_clean":"2023-12-26T23:12:22.605824+0000","last_became_active":"2023-12-26T23:12:22.599383+0000","last_became_peered":"2023-12-26T23:12:22.599383+0000","last_unstale":"2023-12-26T23:12:22.605824+0000","last_undegraded":"2023-12-26T23:12:22.605824+0000","last_fullsized":"2023-12-26T23:12:22.605824+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:46:04.681070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612452+0000","last_change":"2023-12-26T23:12:22.599229+0000","last_active":"2023-12-26T23:12:22.612452+0000","last_peered":"2023-12-26T23:12:22.612452+0000","last_clean":"2023-12-26T23:12:22.612452+0000","last_became_active":"2023-12-26T23:12:22.598549+0000","last_became_peered":"2023-12-26T23:12:22.598549+0000","last_unstale":"2023-12-26T23:12:22.612452+0000","last_undegraded":"2023-12-26T23:12:22.612452+0000","last_fullsized":"2023-12-26T23:12:22.612452+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:46.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621129+0000","last_change":"2023-12-26T23:12:22.614738+0000","last_active":"2023-12-26T23:12:22.621129+0000","last_peered":"2023-12-26T23:12:22.621129+0000","last_clean":"2023-12-26T23:12:22.621129+0000","last_became_active":"2023-12-26T23:12:22.614424+0000","last_became_peered":"2023-12-26T23:12:22.614424+0000","last_unstale":"2023-12-26T23:12:22.621129+0000","last_undegraded":"2023-12-26T23:12:22.621129+0000","last_fullsized":"2023-12-26T23:12:22.621129+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:02:50.533341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618423+0000","last_change":"2023-12-26T23:12:22.614833+0000","last_active":"2023-12-26T23:12:22.618423+0000","last_peered":"2023-12-26T23:12:22.618423+0000","last_clean":"2023-12-26T23:12:22.618423+0000","last_became_active":"2023-12-26T23:12:22.613753+0000","last_became_peered":"2023-12-26T23:12:22.613753+0000","last_unstale":"2023-12-26T23:12:22.618423+0000","last_undegraded":"2023-12-26T23:12:22.618423+0000","last_fullsized":"2023-12-26T23:12:22.618423+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:41:07.888826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.599929+0000","last_change":"2023-12-26T23:12:22.595182+0000","last_active":"2023-12-26T23:12:22.599929+0000","last_peered":"2023-12-26T23:12:22.599929+0000","last_clean":"2023-12-26T23:12:22.599929+0000","last_became_active":"2023-12-26T23:12:22.594899+0000","last_became_peered":"2023-12-26T23:12:22.594899+0000","last_unstale":"2023-12-26T23:12:22.599929+0000","last_undegraded":"2023-12-26T23:12:22.599929+0000","last_fullsized":"2023-12-26T23:12:22.599929+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:25:21.227639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604069+0000","last_change":"2023-12-26T23:12:22.599793+0000","last_active":"2023-12-26T23:12:22.604069+0000","last_peered":"2023-12-26T23:12:22.604069+0000","last_clean":"2023-12-26T23:12:22.604069+0000","last_became_active":"2023-12-26T23:12:22.599266+0000","last_became_peered":"2023-12-26T23:12:22.599266+0000","last_unstale":"2023-12-26T23:12:22.604069+0000","last_undegraded":"2023-12-26T23:12:22.604069+0000","last_fullsized":"2023-12-26T23:12:22.604069+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:21:15.194964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.611803+0000","last_change":"2023-12-26T23:12:22.605971+0000","last_active":"2023-12-26T23:12:22.611803+0000","last_peered":"2023-12-26T23:12:22.611803+0000","last_clean":"2023-12-26T23:12:22.611803+0000","last_became_active":"2023-12-26T23:12:22.605682+0000","last_became_peered":"2023-12-26T23:12:22.605682+0000","last_unstale":"2023-12-26T23:12:22.611803+0000","last_undegraded":"2023-12-26T23:12:22.611803+0000","last_fullsized":"2023-12-26T23:12:22.611803+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:40:29.805595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.607264+0000","last_change":"2023-12-26T23:12:22.601107+0000","last_active":"2023-12-26T23:12:22.607264+0000","last_peered":"2023-12-26T23:12:22.607264+0000","last_clean":"2023-12-26T23:12:22.607264+0000","last_became_active":"2023-12-26T23:12:22.600830+0000","last_became_peered":"2023-12-26T23:12:22.600830+0000","last_unstale":"2023-12-26T23:12:22.607264+0000","last_undegraded":"2023-12-26T23:12:22.607264+0000","last_fullsized":"2023-12-26T23:12:22.607264+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:28:30.485485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"209'4","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621799+0000","last_change":"2023-12-26T23:12:21.597828+0000","last_active":"2023-12-26T23:12:22.621799+0000","last_peered":"2023-12-26T23:12:22.621799+0000","last_clean":"2023-12-26T23:12:22.621799+0000","last_became_active":"2023-12-26T23:12:21.597518+0000","last_became_peered":"2023-12-26T23:12:21.597518+0000","last_unstale":"2023-12-26T23:12:22.621799+0000","last_undegraded":"2023-12-26T23:12:22.621799+0000","last_fullsized":"2023-12-26T23:12:22.621799+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:34:41.226776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621267+0000","last_change":"2023-12-26T23:12:22.615871+0000","last_active":"2023-12-26T23:12:22.621267+0000","last_peered":"2023-12-26T23:12:22.621267+0000","last_clean":"2023-12-26T23:12:22.621267+0000","last_became_active":"2023-12-26T23:12:22.613133+0000","last_became_peered":"2023-12-26T23:12:22.613133+0000","last_unstale":"2023-12-26T23:12:22.621267+0000","last_undegraded":"2023-12-26T23:12:22.621267+0000","last_fullsized":"2023-12-26T23:12:22.621267+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:20:54.097124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.615100+0000","last_change":"2023-12-26T23:12:21.596760+0000","last_active":"2023-12-26T23:12:22.615100+0000","last_peered":"2023-12-26T23:12:22.615100+0000","last_clean":"2023-12-26T23:12:22.615100+0000","last_became_active":"2023-12-26T23:12:21.596526+0000","last_became_peered":"2023-12-26T23:12:21.596526+0000","last_unstale":"2023-12-26T23:12:22.615100+0000","last_undegraded":"2023-12-26T23:12:22.615100+0000","last_fullsized":"2023-12-26T23:12:22.615100+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:34:41.226776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.613735+0000","last_change":"2023-12-26T23:12:22.610129+0000","last_active":"2023-12-26T23:12:22.613735+0000","last_peered":"2023-12-26T23:12:22.613735+0000","last_clean":"2023-12-26T23:12:22.613735+0000","last_became_active":"2023-12-26T23:12:22.609846+0000","last_became_peered":"2023-12-26T23:12:22.609846+0000","last_unstale":"2023-12-26T23:12:22.613735+0000","last_undegraded":"2023-12-26T23:12:22.613735+0000","last_fullsized":"2023-12-26T23:12:22.613735+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:20:54.097124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618351+0000","last_change":"2023-12-26T23:12:21.588180+0000","last_active":"2023-12-26T23:12:22.618351+0000","last_peered":"2023-12-26T23:12:22.618351+0000","last_clean":"2023-12-26T23:12:22.618351+0000","last_became_active":"2023-12-26T23:12:21.587108+0000","last_became_peered":"2023-12-26T23:12:21.587108+0000","last_unstale":"2023-12-26T23:12:22.618351+0000","last_undegraded":"2023-12-26T23:12:22.618351+0000","last_fullsized":"2023-12-26T23:12:22.618351+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:27.356111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618347+0000","last_change":"2023-12-26T23:12:22.616410+0000","last_active":"2023-12-26T23:12:22.618347+0000","last_peered":"2023-12-26T23:12:22.618347+0000","last_clean":"2023-12-26T23:12:22.618347+0000","last_became_active":"2023-12-26T23:12:22.616142+0000","last_became_peered":"2023-12-26T23:12:22.616142+0000","last_unstale":"2023-12-26T23:12:22.618347+0000","last_undegraded":"2023-12-26T23:12:22.618347+0000","last_fullsized":"2023-12-26T23:12:22.618347+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:16.766697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.615327+0000","last_change":"2023-12-26T23:12:21.596338+0000","last_active":"2023-12-26T23:12:22.615327+0000","last_peered":"2023-12-26T23:12:22.615327+0000","last_clean":"2023-12-26T23:12:22.615327+0000","last_became_active":"2023-12-26T23:12:21.595927+0000","last_became_peered":"2023-12-26T23:12:21.595927+0000","last_unstale":"2023-12-26T23:12:22.615327+0000","last_undegraded":"2023-12-26T23:12:22.615327+0000","last_fullsized":"2023-12-26T23:12:22.615327+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:02:26.059586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604678+0000","last_change":"2023-12-26T23:12:22.601298+0000","last_active":"2023-12-26T23:12:22.604678+0000","last_peered":"2023-12-26T23:12:22.604678+0000","last_clean":"2023-12-26T23:12:22.604678+0000","last_became_active":"2023-12-26T23:12:22.600886+0000","last_became_peered":"2023-12-26T23:12:22.600886+0000","last_unstale":"2023-12-26T23:12:22.604678+0000","last_undegraded":"2023-12-26T23:12:22.604678+0000","last_fullsized":"2023-12-26T23:12:22.604678+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:06:59.903118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618449+0000","last_change":"2023-12-26T23:12:21.587617+0000","last_active":"2023-12-26T23:12:22.618449+0000","last_peered":"2023-12-26T23:12:22.618449+0000","last_clean":"2023-12-26T23:12:22.618449+0000","last_became_active":"2023-12-26T23:12:21.586950+0000","last_became_peered":"2023-12-26T23:12:21.586950+0000","last_unstale":"2023-12-26T23:12:22.618449+0000","last_undegraded":"2023-12-26T23:12:22.618449+0000","last_fullsized":"2023-12-26T23:12:22.618449+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:45.449822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.606216+0000","last_change":"2023-12-26T23:12:22.603294+0000","last_active":"2023-12-26T23:12:22.606216+0000","last_peered":"2023-12-26T23:12:22.606216+0000","last_clean":"2023-12-26T23:12:22.606216+0000","last_became_active":"2023-12-26T23:12:22.602790+0000","last_became_peered":"2023-12-26T23:12:22.602790+0000","last_unstale":"2023-12-26T23:12:22.606216+0000","last_undegraded":"2023-12-26T23:12:22.606216+0000","last_fullsized":"2023-12-26T23:12:22.606216+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:53:38.971245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620325+0000","last_change":"2023-12-26T23:12:21.594983+0000","last_active":"2023-12-26T23:12:22.620325+0000","last_peered":"2023-12-26T23:12:22.620325+0000","last_clean":"2023-12-26T23:12:22.620325+0000","last_became_active":"2023-12-26T23:12:21.594105+0000","last_became_peered":"2023-12-26T23:12:21.594105+0000","last_unstale":"2023-12-26T23:12:22.620325+0000","last_undegraded":"2023-12-26T23:12:22.620325+0000","last_fullsized":"2023-12-26T23:12:22.620325+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:07:34.526239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608706+0000","last_change":"2023-12-26T23:12:22.604454+0000","last_active":"2023-12-26T23:12:22.608706+0000","last_peered":"2023-12-26T23:12:22.608706+0000","last_clean":"2023-12-26T23:12:22.608706+0000","last_became_active":"2023-12-26T23:12:22.604080+0000","last_became_peered":"2023-12-26T23:12:22.604080+0000","last_unstale":"2023-12-26T23:12:22.608706+0000","last_undegraded":"2023-12-26T23:12:22.608706+0000","last_fullsized":"2023-12-26T23:12:22.608706+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:27:30.344416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.588365+0000","last_change":"2023-12-26T23:12:21.600860+0000","last_active":"2023-12-26T23:12:22.588365+0000","last_peered":"2023-12-26T23:12:22.588365+0000","last_clean":"2023-12-26T23:12:22.588365+0000","last_became_active":"2023-12-26T23:12:21.599603+0000","last_became_peered":"2023-12-26T23:12:21.599603+0000","last_unstale":"2023-12-26T23:12:22.588365+0000","last_undegraded":"2023-12-26T23:12:22.588365+0000","last_fullsized":"2023-12-26T23:12:22.588365+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:00:15.902830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.611155+0000","last_change":"2023-12-26T23:12:22.602861+0000","last_active":"2023-12-26T23:12:22.611155+0000","last_peered":"2023-12-26T23:12:22.611155+0000","last_clean":"2023-12-26T23:12:22.611155+0000","last_became_active":"2023-12-26T23:12:22.601876+0000","last_became_peered":"2023-12-26T23:12:22.601876+0000","last_unstale":"2023-12-26T23:12:22.611155+0000","last_undegraded":"2023-12-26T23:12:22.611155+0000","last_fullsized":"2023-12-26T23:12:22.611155+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:14:02.597163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.584550+0000","last_change":"2023-12-26T23:12:21.592994+0000","last_active":"2023-12-26T23:12:22.584550+0000","last_peered":"2023-12-26T23:12:22.584550+0000","last_clean":"2023-12-26T23:12:22.584550+0000","last_became_active":"2023-12-26T23:12:21.592737+0000","last_became_peered":"2023-12-26T23:12:21.592737+0000","last_unstale":"2023-12-26T23:12:22.584550+0000","last_undegraded":"2023-12-26T23:12:22.584550+0000","last_fullsized":"2023-12-26T23:12:22.584550+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:53:05.285994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621289+0000","last_change":"2023-12-26T23:12:22.615955+0000","last_active":"2023-12-26T23:12:22.621289+0000","last_peered":"2023-12-26T23:12:22.621289+0000","last_clean":"2023-12-26T23:12:22.621289+0000","last_became_active":"2023-12-26T23:12:22.613156+0000","last_became_peered":"2023-12-26T23:12:22.613156+0000","last_unstale":"2023-12-26T23:12:22.621289+0000","last_undegraded":"2023-12-26T23:12:22.621289+0000","last_fullsized":"2023-12-26T23:12:22.621289+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:12:18.182031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603410+0000","last_change":"2023-12-26T23:12:21.598664+0000","last_active":"2023-12-26T23:12:22.603410+0000","last_peered":"2023-12-26T23:12:22.603410+0000","last_clean":"2023-12-26T23:12:22.603410+0000","last_became_active":"2023-12-26T23:12:21.598354+0000","last_became_peered":"2023-12-26T23:12:21.598354+0000","last_unstale":"2023-12-26T23:12:22.603410+0000","last_undegraded":"2023-12-26T23:12:22.603410+0000","last_fullsized":"2023-12-26T23:12:22.603410+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:07:57.669862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.607814+0000","last_change":"2023-12-26T23:12:22.602916+0000","last_active":"2023-12-26T23:12:22.607814+0000","last_peered":"2023-12-26T23:12:22.607814+0000","last_clean":"2023-12-26T23:12:22.607814+0000","last_became_active":"2023-12-26T23:12:22.602558+0000","last_became_peered":"2023-12-26T23:12:22.602558+0000","last_unstale":"2023-12-26T23:12:22.607814+0000","last_undegraded":"2023-12-26T23:12:22.607814+0000","last_fullsized":"2023-12-26T23:12:22.607814+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:06:33.817797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"209'3","reported_seq":13,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620387+0000","last_change":"2023-12-26T23:12:21.594707+0000","last_active":"2023-12-26T23:12:22.620387+0000","last_peered":"2023-12-26T23:12:22.620387+0000","last_clean":"2023-12-26T23:12:22.620387+0000","last_became_active":"2023-12-26T23:12:21.593719+0000","last_became_peered":"2023-12-26T23:12:21.593719+0000","last_unstale":"2023-12-26T23:12:22.620387+0000","last_undegraded":"2023-12-26T23:12:22.620387+0000","last_fullsized":"2023-12-26T23:12:22.620387+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:41:09.150367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621331+0000","last_change":"2023-12-26T23:12:22.616107+0000","last_active":"2023-12-26T23:12:22.621331+0000","last_peered":"2023-12-26T23:12:22.621331+0000","last_clean":"2023-12-26T23:12:22.621331+0000","last_became_active":"2023-12-26T23:12:22.615638+0000","last_became_peered":"2023-12-26T23:12:22.615638+0000","last_unstale":"2023-12-26T23:12:22.621331+0000","last_undegraded":"2023-12-26T23:12:22.621331+0000","last_fullsized":"2023-12-26T23:12:22.621331+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:07.058946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"209'6","reported_seq":16,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620612+0000","last_change":"2023-12-26T23:12:21.594282+0000","last_active":"2023-12-26T23:12:22.620612+0000","last_peered":"2023-12-26T23:12:22.620612+0000","last_clean":"2023-12-26T23:12:22.620612+0000","last_became_active":"2023-12-26T23:12:21.593997+0000","last_became_peered":"2023-12-26T23:12:21.593997+0000","last_unstale":"2023-12-26T23:12:22.620612+0000","last_undegraded":"2023-12-26T23:12:22.620612+0000","last_fullsized":"2023-12-26T23:12:22.620612+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:51:22.275494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621073+0000","last_change":"2023-12-26T23:12:22.614073+0000","last_active":"2023-12-26T23:12:22.621073+0000","last_peered":"2023-12-26T23:12:22.621073+0000","last_clean":"2023-12-26T23:12:22.621073+0000","last_became_active":"2023-12-26T23:12:22.613801+0000","last_became_peered":"2023-12-26T23:12:22.613801+0000","last_unstale":"2023-12-26T23:12:22.621073+0000","last_undegraded":"2023-12-26T23:12:22.621073+0000","last_fullsized":"2023-12-26T23:12:22.621073+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:45:45.843764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618480+0000","last_change":"2023-12-26T23:12:21.600730+0000","last_active":"2023-12-26T23:12:22.618480+0000","last_peered":"2023-12-26T23:12:22.618480+0000","last_clean":"2023-12-26T23:12:22.618480+0000","last_became_active":"2023-12-26T23:12:21.600494+0000","last_became_peered":"2023-12-26T23:12:21.600494+0000","last_unstale":"2023-12-26T23:12:22.618480+0000","last_undegraded":"2023-12-26T23:12:22.618480+0000","last_fullsized":"2023-12-26T23:12:22.618480+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:55:28.859094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603708+0000","last_change":"2023-12-26T23:12:22.601486+0000","last_active":"2023-12-26T23:12:22.603708+0000","last_peered":"2023-12-26T23:12:22.603708+0000","last_clean":"2023-12-26T23:12:22.603708+0000","last_became_active":"2023-12-26T23:12:22.601147+0000","last_became_peered":"2023-12-26T23:12:22.601147+0000","last_unstale":"2023-12-26T23:12:22.603708+0000","last_undegraded":"2023-12-26T23:12:22.603708+0000","last_fullsized":"2023-12-26T23:12:22.603708+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:50:31.681211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603634+0000","last_change":"2023-12-26T23:12:21.598293+0000","last_active":"2023-12-26T23:12:22.603634+0000","last_peered":"2023-12-26T23:12:22.603634+0000","last_clean":"2023-12-26T23:12:22.603634+0000","last_became_active":"2023-12-26T23:12:21.597331+0000","last_became_peered":"2023-12-26T23:12:21.597331+0000","last_unstale":"2023-12-26T23:12:22.603634+0000","last_undegraded":"2023-12-26T23:12:22.603634+0000","last_fullsized":"2023-12-26T23:12:22.603634+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:15:47.071802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.602753+0000","last_change":"2023-12-26T23:12:22.599542+0000","last_active":"2023-12-26T23:12:22.602753+0000","last_peered":"2023-12-26T23:12:22.602753+0000","last_clean":"2023-12-26T23:12:22.602753+0000","last_became_active":"2023-12-26T23:12:22.599236+0000","last_became_peered":"2023-12-26T23:12:22.599236+0000","last_unstale":"2023-12-26T23:12:22.602753+0000","last_undegraded":"2023-12-26T23:12:22.602753+0000","last_fullsized":"2023-12-26T23:12:22.602753+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:43.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.584228+0000","last_change":"2023-12-26T23:12:21.596976+0000","last_active":"2023-12-26T23:12:22.584228+0000","last_peered":"2023-12-26T23:12:22.584228+0000","last_clean":"2023-12-26T23:12:22.584228+0000","last_became_active":"2023-12-26T23:12:21.596562+0000","last_became_peered":"2023-12-26T23:12:21.596562+0000","last_unstale":"2023-12-26T23:12:22.584228+0000","last_undegraded":"2023-12-26T23:12:22.584228+0000","last_fullsized":"2023-12-26T23:12:22.584228+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:31:24.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.606281+0000","last_change":"2023-12-26T23:12:22.603629+0000","last_active":"2023-12-26T23:12:22.606281+0000","last_peered":"2023-12-26T23:12:22.606281+0000","last_clean":"2023-12-26T23:12:22.606281+0000","last_became_active":"2023-12-26T23:12:22.603139+0000","last_became_peered":"2023-12-26T23:12:22.603139+0000","last_unstale":"2023-12-26T23:12:22.606281+0000","last_undegraded":"2023-12-26T23:12:22.606281+0000","last_fullsized":"2023-12-26T23:12:22.606281+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:38:16.070134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"209'8","reported_seq":18,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618204+0000","last_change":"2023-12-26T23:12:21.599125+0000","last_active":"2023-12-26T23:12:22.618204+0000","last_peered":"2023-12-26T23:12:22.618204+0000","last_clean":"2023-12-26T23:12:22.618204+0000","last_became_active":"2023-12-26T23:12:21.598466+0000","last_became_peered":"2023-12-26T23:12:21.598466+0000","last_unstale":"2023-12-26T23:12:22.618204+0000","last_undegraded":"2023-12-26T23:12:22.618204+0000","last_fullsized":"2023-12-26T23:12:22.618204+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:48:45.758030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621303+0000","last_change":"2023-12-26T23:12:22.615839+0000","last_active":"2023-12-26T23:12:22.621303+0000","last_peered":"2023-12-26T23:12:22.621303+0000","last_clean":"2023-12-26T23:12:22.621303+0000","last_became_active":"2023-12-26T23:12:22.612485+0000","last_became_peered":"2023-12-26T23:12:22.612485+0000","last_unstale":"2023-12-26T23:12:22.621303+0000","last_undegraded":"2023-12-26T23:12:22.621303+0000","last_fullsized":"2023-12-26T23:12:22.621303+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:13:40.795030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620863+0000","last_change":"2023-12-26T23:12:21.589505+0000","last_active":"2023-12-26T23:12:22.620863+0000","last_peered":"2023-12-26T23:12:22.620863+0000","last_clean":"2023-12-26T23:12:22.620863+0000","last_became_active":"2023-12-26T23:12:21.589099+0000","last_became_peered":"2023-12-26T23:12:21.589099+0000","last_unstale":"2023-12-26T23:12:22.620863+0000","last_undegraded":"2023-12-26T23:12:22.620863+0000","last_fullsized":"2023-12-26T23:12:22.620863+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:00:59.073908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608591+0000","last_change":"2023-12-26T23:12:22.606348+0000","last_active":"2023-12-26T23:12:22.608591+0000","last_peered":"2023-12-26T23:12:22.608591+0000","last_clean":"2023-12-26T23:12:22.608591+0000","last_became_active":"2023-12-26T23:12:22.605943+0000","last_became_peered":"2023-12-26T23:12:22.605943+0000","last_unstale":"2023-12-26T23:12:22.608591+0000","last_undegraded":"2023-12-26T23:12:22.608591+0000","last_fullsized":"2023-12-26T23:12:22.608591+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:48.326459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618546+0000","last_change":"2023-12-26T23:12:21.598748+0000","last_active":"2023-12-26T23:12:22.618546+0000","last_peered":"2023-12-26T23:12:22.618546+0000","last_clean":"2023-12-26T23:12:22.618546+0000","last_became_active":"2023-12-26T23:12:21.598315+0000","last_became_peered":"2023-12-26T23:12:21.598315+0000","last_unstale":"2023-12-26T23:12:22.618546+0000","last_undegraded":"2023-12-26T23:12:22.618546+0000","last_fullsized":"2023-12-26T23:12:22.618546+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:30:13.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603689+0000","last_change":"2023-12-26T23:12:22.600650+0000","last_active":"2023-12-26T23:12:22.603689+0000","last_peered":"2023-12-26T23:12:22.603689+0000","last_clean":"2023-12-26T23:12:22.603689+0000","last_became_active":"2023-12-26T23:12:22.600058+0000","last_became_peered":"2023-12-26T23:12:22.600058+0000","last_unstale":"2023-12-26T23:12:22.603689+0000","last_undegraded":"2023-12-26T23:12:22.603689+0000","last_fullsized":"2023-12-26T23:12:22.603689+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:09:20.640840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612806+0000","last_change":"2023-12-26T23:12:21.591902+0000","last_active":"2023-12-26T23:12:22.612806+0000","last_peered":"2023-12-26T23:12:22.612806+0000","last_clean":"2023-12-26T23:12:22.612806+0000","last_became_active":"2023-12-26T23:12:21.591634+0000","last_became_peered":"2023-12-26T23:12:21.591634+0000","last_unstale":"2023-12-26T23:12:22.612806+0000","last_undegraded":"2023-12-26T23:12:22.612806+0000","last_fullsized":"2023-12-26T23:12:22.612806+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:20:51.780226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.613198+0000","last_change":"2023-12-26T23:12:22.610294+0000","last_active":"2023-12-26T23:12:22.613198+0000","last_peered":"2023-12-26T23:12:22.613198+0000","last_clean":"2023-12-26T23:12:22.613198+0000","last_became_active":"2023-12-26T23:12:22.610016+0000","last_became_peered":"2023-12-26T23:12:22.610016+0000","last_unstale":"2023-12-26T23:12:22.613198+0000","last_undegraded":"2023-12-26T23:12:22.613198+0000","last_fullsized":"2023-12-26T23:12:22.613198+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:46.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.585094+0000","last_change":"2023-12-26T23:12:21.598986+0000","last_active":"2023-12-26T23:12:22.585094+0000","last_peered":"2023-12-26T23:12:22.585094+0000","last_clean":"2023-12-26T23:12:22.585094+0000","last_became_active":"2023-12-26T23:12:21.598653+0000","last_became_peered":"2023-12-26T23:12:21.598653+0000","last_unstale":"2023-12-26T23:12:22.585094+0000","last_undegraded":"2023-12-26T23:12:22.585094+0000","last_fullsized":"2023-12-26T23:12:22.585094+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:23:22.158692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.598051+0000","last_change":"2023-12-26T23:12:22.593725+0000","last_active":"2023-12-26T23:12:22.598051+0000","last_peered":"2023-12-26T23:12:22.598051+0000","last_clean":"2023-12-26T23:12:22.598051+0000","last_became_active":"2023-12-26T23:12:22.593427+0000","last_became_peered":"2023-12-26T23:12:22.593427+0000","last_unstale":"2023-12-26T23:12:22.598051+0000","last_undegraded":"2023-12-26T23:12:22.598051+0000","last_fullsized":"2023-12-26T23:12:22.598051+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:17:09.261315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.614710+0000","last_change":"2023-12-26T23:12:21.599507+0000","last_active":"2023-12-26T23:12:22.614710+0000","last_peered":"2023-12-26T23:12:22.614710+0000","last_clean":"2023-12-26T23:12:22.614710+0000","last_became_active":"2023-12-26T23:12:21.599227+0000","last_became_peered":"2023-12-26T23:12:21.599227+0000","last_unstale":"2023-12-26T23:12:22.614710+0000","last_undegraded":"2023-12-26T23:12:22.614710+0000","last_fullsized":"2023-12-26T23:12:22.614710+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:18:38.340055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.617470+0000","last_change":"2023-12-26T23:12:22.616253+0000","last_active":"2023-12-26T23:12:22.617470+0000","last_peered":"2023-12-26T23:12:22.617470+0000","last_clean":"2023-12-26T23:12:22.617470+0000","last_became_active":"2023-12-26T23:12:22.615948+0000","last_became_peered":"2023-12-26T23:12:22.615948+0000","last_unstale":"2023-12-26T23:12:22.617470+0000","last_undegraded":"2023-12-26T23:12:22.617470+0000","last_fullsized":"2023-12-26T23:12:22.617470+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:48.592908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.589695+0000","last_change":"2023-12-26T23:12:21.599064+0000","last_active":"2023-12-26T23:12:22.589695+0000","last_peered":"2023-12-26T23:12:22.589695+0000","last_clean":"2023-12-26T23:12:22.589695+0000","last_became_active":"2023-12-26T23:12:21.598809+0000","last_became_peered":"2023-12-26T23:12:21.598809+0000","last_unstale":"2023-12-26T23:12:22.589695+0000","last_undegraded":"2023-12-26T23:12:22.589695+0000","last_fullsized":"2023-12-26T23:12:22.589695+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:45.449822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.599212+0000","last_change":"2023-12-26T23:12:22.594491+0000","last_active":"2023-12-26T23:12:22.599212+0000","last_peered":"2023-12-26T23:12:22.599212+0000","last_clean":"2023-12-26T23:12:22.599212+0000","last_became_active":"2023-12-26T23:12:22.592704+0000","last_became_peered":"2023-12-26T23:12:22.592704+0000","last_unstale":"2023-12-26T23:12:22.599212+0000","last_undegraded":"2023-12-26T23:12:22.599212+0000","last_fullsized":"2023-12-26T23:12:22.599212+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:54:32.482725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.589629+0000","last_change":"2023-12-26T23:12:21.600931+0000","last_active":"2023-12-26T23:12:22.589629+0000","last_peered":"2023-12-26T23:12:22.589629+0000","last_clean":"2023-12-26T23:12:22.589629+0000","last_became_active":"2023-12-26T23:12:21.600544+0000","last_became_peered":"2023-12-26T23:12:21.600544+0000","last_unstale":"2023-12-26T23:12:22.589629+0000","last_undegraded":"2023-12-26T23:12:22.589629+0000","last_fullsized":"2023-12-26T23:12:22.589629+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:00:59.073908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621457+0000","last_change":"2023-12-26T23:12:22.616366+0000","last_active":"2023-12-26T23:12:22.621457+0000","last_peered":"2023-12-26T23:12:22.621457+0000","last_clean":"2023-12-26T23:12:22.621457+0000","last_became_active":"2023-12-26T23:12:22.616068+0000","last_became_peered":"2023-12-26T23:12:22.616068+0000","last_unstale":"2023-12-26T23:12:22.621457+0000","last_undegraded":"2023-12-26T23:12:22.621457+0000","last_fullsized":"2023-12-26T23:12:22.621457+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:17:18.359852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.584483+0000","last_change":"2023-12-26T23:12:21.593115+0000","last_active":"2023-12-26T23:12:22.584483+0000","last_peered":"2023-12-26T23:12:22.584483+0000","last_clean":"2023-12-26T23:12:22.584483+0000","last_became_active":"2023-12-26T23:12:21.592871+0000","last_became_peered":"2023-12-26T23:12:21.592871+0000","last_unstale":"2023-12-26T23:12:22.584483+0000","last_undegraded":"2023-12-26T23:12:22.584483+0000","last_fullsized":"2023-12-26T23:12:22.584483+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:58:32.040716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.611177+0000","last_change":"2023-12-26T23:12:22.603136+0000","last_active":"2023-12-26T23:12:22.611177+0000","last_peered":"2023-12-26T23:12:22.611177+0000","last_clean":"2023-12-26T23:12:22.611177+0000","last_became_active":"2023-12-26T23:12:22.602273+0000","last_became_peered":"2023-12-26T23:12:22.602273+0000","last_unstale":"2023-12-26T23:12:22.611177+0000","last_undegraded":"2023-12-26T23:12:22.611177+0000","last_fullsized":"2023-12-26T23:12:22.611177+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:43.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620211+0000","last_change":"2023-12-26T23:12:21.598993+0000","last_active":"2023-12-26T23:12:22.620211+0000","last_peered":"2023-12-26T23:12:22.620211+0000","last_clean":"2023-12-26T23:12:22.620211+0000","last_became_active":"2023-12-26T23:12:21.598721+0000","last_became_peered":"2023-12-26T23:12:21.598721+0000","last_unstale":"2023-12-26T23:12:22.620211+0000","last_undegraded":"2023-12-26T23:12:22.620211+0000","last_fullsized":"2023-12-26T23:12:22.620211+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:19:59.693301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608111+0000","last_change":"2023-12-26T23:12:22.591386+0000","last_active":"2023-12-26T23:12:22.608111+0000","last_peered":"2023-12-26T23:12:22.608111+0000","last_clean":"2023-12-26T23:12:22.608111+0000","last_became_active":"2023-12-26T23:12:22.590472+0000","last_became_peered":"2023-12-26T23:12:22.590472+0000","last_unstale":"2023-12-26T23:12:22.608111+0000","last_undegraded":"2023-12-26T23:12:22.608111+0000","last_fullsized":"2023-12-26T23:12:22.608111+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:48.326459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.588080+0000","last_change":"2023-12-26T23:12:21.594735+0000","last_active":"2023-12-26T23:12:22.588080+0000","last_peered":"2023-12-26T23:12:22.588080+0000","last_clean":"2023-12-26T23:12:22.588080+0000","last_became_active":"2023-12-26T23:12:21.594457+0000","last_became_peered":"2023-12-26T23:12:21.594457+0000","last_unstale":"2023-12-26T23:12:22.588080+0000","last_undegraded":"2023-12-26T23:12:22.588080+0000","last_fullsized":"2023-12-26T23:12:22.588080+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:55:28.859094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608254+0000","last_change":"2023-12-26T23:12:22.595522+0000","last_active":"2023-12-26T23:12:22.608254+0000","last_peered":"2023-12-26T23:12:22.608254+0000","last_clean":"2023-12-26T23:12:22.608254+0000","last_became_active":"2023-12-26T23:12:22.595272+0000","last_became_peered":"2023-12-26T23:12:22.595272+0000","last_unstale":"2023-12-26T23:12:22.608254+0000","last_undegraded":"2023-12-26T23:12:22.608254+0000","last_fullsized":"2023-12-26T23:12:22.608254+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:41:07.888826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.587423+0000","last_change":"2023-12-26T23:12:21.594905+0000","last_active":"2023-12-26T23:12:22.587423+0000","last_peered":"2023-12-26T23:12:22.587423+0000","last_clean":"2023-12-26T23:12:22.587423+0000","last_became_active":"2023-12-26T23:12:21.594594+0000","last_became_peered":"2023-12-26T23:12:21.594594+0000","last_unstale":"2023-12-26T23:12:22.587423+0000","last_undegraded":"2023-12-26T23:12:22.587423+0000","last_fullsized":"2023-12-26T23:12:22.587423+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:02:26.059586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604857+0000","last_change":"2023-12-26T23:12:22.603023+0000","last_active":"2023-12-26T23:12:22.604857+0000","last_peered":"2023-12-26T23:12:22.604857+0000","last_clean":"2023-12-26T23:12:22.604857+0000","last_became_active":"2023-12-26T23:12:22.602614+0000","last_became_peered":"2023-12-26T23:12:22.602614+0000","last_unstale":"2023-12-26T23:12:22.604857+0000","last_undegraded":"2023-12-26T23:12:22.604857+0000","last_fullsized":"2023-12-26T23:12:22.604857+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:06:32.152177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.583856+0000","last_change":"2023-12-26T23:12:21.589618+0000","last_active":"2023-12-26T23:12:22.583856+0000","last_peered":"2023-12-26T23:12:22.583856+0000","last_clean":"2023-12-26T23:12:22.583856+0000","last_became_active":"2023-12-26T23:12:21.589304+0000","last_became_peered":"2023-12-26T23:12:21.589304+0000","last_unstale":"2023-12-26T23:12:22.583856+0000","last_undegraded":"2023-12-26T23:12:22.583856+0000","last_fullsized":"2023-12-26T23:12:22.583856+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:30:13.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.613603+0000","last_change":"2023-12-26T23:12:22.610118+0000","last_active":"2023-12-26T23:12:22.613603+0000","last_peered":"2023-12-26T23:12:22.613603+0000","last_clean":"2023-12-26T23:12:22.613603+0000","last_became_active":"2023-12-26T23:12:22.609493+0000","last_became_peered":"2023-12-26T23:12:22.609493+0000","last_unstale":"2023-12-26T23:12:22.613603+0000","last_undegraded":"2023-12-26T23:12:22.613603+0000","last_fullsized":"2023-12-26T23:12:22.613603+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:26:27.918216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.594917+0000","last_change":"2023-12-26T23:12:21.588645+0000","last_active":"2023-12-26T23:12:22.594917+0000","last_peered":"2023-12-26T23:12:22.594917+0000","last_clean":"2023-12-26T23:12:22.594917+0000","last_became_active":"2023-12-26T23:12:21.586977+0000","last_became_peered":"2023-12-26T23:12:21.586977+0000","last_unstale":"2023-12-26T23:12:22.594917+0000","last_undegraded":"2023-12-26T23:12:22.594917+0000","last_fullsized":"2023-12-26T23:12:22.594917+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:47:26.727484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.601005+0000","last_change":"2023-12-26T23:12:22.595720+0000","last_active":"2023-12-26T23:12:22.601005+0000","last_peered":"2023-12-26T23:12:22.601005+0000","last_clean":"2023-12-26T23:12:22.601005+0000","last_became_active":"2023-12-26T23:12:22.595460+0000","last_became_peered":"2023-12-26T23:12:22.595460+0000","last_unstale":"2023-12-26T23:12:22.601005+0000","last_undegraded":"2023-12-26T23:12:22.601005+0000","last_fullsized":"2023-12-26T23:12:22.601005+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:55.095066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.591100+0000","last_change":"2023-12-26T23:12:21.593706+0000","last_active":"2023-12-26T23:12:22.591100+0000","last_peered":"2023-12-26T23:12:22.591100+0000","last_clean":"2023-12-26T23:12:22.591100+0000","last_became_active":"2023-12-26T23:12:21.593437+0000","last_became_peered":"2023-12-26T23:12:21.593437+0000","last_unstale":"2023-12-26T23:12:22.591100+0000","last_undegraded":"2023-12-26T23:12:22.591100+0000","last_fullsized":"2023-12-26T23:12:22.591100+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:14.373113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.619186+0000","last_change":"2023-12-26T23:12:22.617685+0000","last_active":"2023-12-26T23:12:22.619186+0000","last_peered":"2023-12-26T23:12:22.619186+0000","last_clean":"2023-12-26T23:12:22.619186+0000","last_became_active":"2023-12-26T23:12:22.616842+0000","last_became_peered":"2023-12-26T23:12:22.616842+0000","last_unstale":"2023-12-26T23:12:22.619186+0000","last_undegraded":"2023-12-26T23:12:22.619186+0000","last_fullsized":"2023-12-26T23:12:22.619186+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:45:45.843764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.597186+0000","last_change":"2023-12-26T23:12:21.600294+0000","last_active":"2023-12-26T23:12:22.597186+0000","last_peered":"2023-12-26T23:12:22.597186+0000","last_clean":"2023-12-26T23:12:22.597186+0000","last_became_active":"2023-12-26T23:12:21.600012+0000","last_became_peered":"2023-12-26T23:12:21.600012+0000","last_unstale":"2023-12-26T23:12:22.597186+0000","last_undegraded":"2023-12-26T23:12:22.597186+0000","last_fullsized":"2023-12-26T23:12:22.597186+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:01:13.020675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.606344+0000","last_change":"2023-12-26T23:12:22.603754+0000","last_active":"2023-12-26T23:12:22.606344+0000","last_peered":"2023-12-26T23:12:22.606344+0000","last_clean":"2023-12-26T23:12:22.606344+0000","last_became_active":"2023-12-26T23:12:22.603507+0000","last_became_peered":"2023-12-26T23:12:22.603507+0000","last_unstale":"2023-12-26T23:12:22.606344+0000","last_undegraded":"2023-12-26T23:12:22.606344+0000","last_fullsized":"2023-12-26T23:12:22.606344+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:48:46.764483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.590291+0000","last_change":"2023-12-26T23:12:21.597965+0000","last_active":"2023-12-26T23:12:22.590291+0000","last_peered":"2023-12-26T23:12:22.590291+0000","last_clean":"2023-12-26T23:12:22.590291+0000","last_became_active":"2023-12-26T23:12:21.597321+0000","last_became_peered":"2023-12-26T23:12:21.597321+0000","last_unstale":"2023-12-26T23:12:22.590291+0000","last_undegraded":"2023-12-26T23:12:22.590291+0000","last_fullsized":"2023-12-26T23:12:22.590291+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:31:24.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.607612+0000","last_change":"2023-12-26T23:12:22.602610+0000","last_active":"2023-12-26T23:12:22.607612+0000","last_peered":"2023-12-26T23:12:22.607612+0000","last_clean":"2023-12-26T23:12:22.607612+0000","last_became_active":"2023-12-26T23:12:22.602027+0000","last_became_peered":"2023-12-26T23:12:22.602027+0000","last_unstale":"2023-12-26T23:12:22.607612+0000","last_undegraded":"2023-12-26T23:12:22.607612+0000","last_fullsized":"2023-12-26T23:12:22.607612+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:05:13.435725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.591824+0000","last_change":"2023-12-26T23:12:21.598921+0000","last_active":"2023-12-26T23:12:22.591824+0000","last_peered":"2023-12-26T23:12:22.591824+0000","last_clean":"2023-12-26T23:12:22.591824+0000","last_became_active":"2023-12-26T23:12:21.598690+0000","last_became_peered":"2023-12-26T23:12:21.598690+0000","last_unstale":"2023-12-26T23:12:22.591824+0000","last_undegraded":"2023-12-26T23:12:22.591824+0000","last_fullsized":"2023-12-26T23:12:22.591824+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:39.345340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"199'348","reported_seq":1152,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.597625+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:12:22.597625+0000","last_peered":"2023-12-26T23:12:22.597625+0000","last_clean":"2023-12-26T23:12:22.597625+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:12:22.597625+0000","last_undegraded":"2023-12-26T23:12:22.597625+0000","last_fullsized":"2023-12-26T23:12:22.597625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543280,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308060,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063780,"statfs":{"total":96636764160,"available":96321310720,"internally_reserved":0,"allocated":2641920,"data_stored":1322727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.56699999999999995,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.85499999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.54300000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90800000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73199999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.98599999999999999,"15min":0.995},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.58599999999999997,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":0.89200000000000002,"5min":0.93300000000000005,"15min":1.0449999999999999},"last":0.88900000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.88200000000000001,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.748},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.55500000000000005,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.86799999999999999,"5min":0.94199999999999995,"15min":1},"last":0.81399999999999995}]},{"osd":3,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.69399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95199999999999996,"5min":1.0029999999999999,"15min":1.0760000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.51100000000000001,"5min":0.498,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.97599999999999998,"15min":0.98199999999999998},"last":0.77800000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.91400000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56799999999999995,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.39600000000000002,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.55100000000000005}]},{"osd":5,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.98799999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.57499999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92800000000000005,"5min":0.998,"15min":1.042},"last":0.79500000000000004}]},{"osd":6,"last update":"Tue Dec 26 23:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.54800000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.76300000000000001}]}]},{"osd":6,"up_from":15,"seq":64424510576,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1321169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.32800000000000001},"max":{"1min":0.873,"5min":0.998,"15min":1.1080000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0680000000000001},"last":0.71099999999999997}]},{"osd":1,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":1.022,"15min":1.046},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.95999999999999996,"15min":1.0580000000000001},"last":0.77100000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91500000000000004,"15min":1.071},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":0.89800000000000002,"5min":0.97899999999999998,"15min":1.0680000000000001},"last":0.75}]},{"osd":3,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.96699999999999997,"15min":1.024},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":1.0149999999999999},"last":0.85299999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.018,"15min":1.1599999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.111},"last":0.622},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.66900000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.83899999999999997}]}]},{"osd":1,"up_from":15,"seq":64424510576,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300428,"kb_used_data":3140,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071412,"statfs":{"total":96636764160,"available":96329125888,"internally_reserved":0,"allocated":3215360,"data_stored":1911447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.29199999999999998,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.81399999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.50600000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.622,"5min":0.76900000000000002,"15min":0.79200000000000004},"last":0.45100000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.33800000000000002},"max":{"1min":0.58099999999999996,"5min":0.753,"15min":0.84099999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.65800000000000003,"5min":0.75700000000000001,"15min":0.91200000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.70299999999999996,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.46700000000000003,"5min":0.40600000000000003,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.872,"15min":0.93600000000000005},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.629,"5min":0.55100000000000005,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":1.103,"15min":1.103},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.79100000000000004,"15min":0.77500000000000002},"min":{"1min":0.60899999999999999,"5min":0.52500000000000002,"15min":0.48599999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.57799999999999996}]},{"osd":5,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.50700000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.83199999999999996},"min":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.1100000000000001,"5min":1.1240000000000001,"15min":1.1830000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.753,"5min":0.57599999999999996,"15min":0.52600000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.627},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.85899999999999999,"15min":0.84299999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.64700000000000002,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.218},"last":0.67400000000000004}]}]},{"osd":0,"up_from":15,"seq":64424510576,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308048,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063792,"statfs":{"total":96636764160,"available":96321323008,"internally_reserved":0,"allocated":2629632,"data_stored":1321667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.26600000000000001},"max":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.313},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.376}]},{"osd":2,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.30599999999999999},"max":{"1min":0.72899999999999998,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51000000000000001}]},{"osd":3,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.88100000000000001,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88,"5min":0.91800000000000004,"15min":1.012},"last":0.46800000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.2250000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77400000000000002,"15min":0.78300000000000003},"min":{"1min":0.66400000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.113},"last":0.57699999999999996}]},{"osd":5,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80400000000000005,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0469999999999999,"5min":1.0920000000000001,"15min":1.157},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.82899999999999996,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.282},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.79500000000000004,"15min":0.80500000000000005},"min":{"1min":0.72599999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.121,"5min":1.121,"15min":1.151},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.61499999999999999}]},{"osd":7,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81799999999999995,"15min":0.82199999999999995},"min":{"1min":0.67500000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1020000000000001,"5min":1.173,"15min":1.256},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.64200000000000002,"5min":0.54800000000000004,"15min":0.54000000000000004},"max":{"1min":1.087,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.72199999999999998}]}]},{"osd":2,"up_from":14,"seq":60129543281,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312140,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059700,"statfs":{"total":96636764160,"available":96317132800,"internally_reserved":0,"allocated":2625536,"data_stored":1321149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":0.84599999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69199999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.45800000000000002,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.64700000000000002,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.79200000000000004,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.505},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80200000000000005,"5min":0.89300000000000002,"15min":1.056},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.65600000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1180000000000001,"5min":1.8,"15min":1.8},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.59099999999999997,"15min":0.54000000000000004},"max":{"1min":1.115,"5min":1.222,"15min":1.222},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":1.079,"5min":1.923,"15min":1.923},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.087,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.80700000000000005}]},{"osd":6,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.55300000000000005,"15min":0.52400000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.272},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0269999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.77400000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.56399999999999995},"max":{"1min":1.042,"5min":1.1100000000000001,"15min":1.1619999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.84499999999999997},"min":{"1min":0.70999999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0740000000000001,"5min":1.841,"15min":1.841},"last":0.79900000000000004}]}]},{"osd":3,"up_from":15,"seq":64424510577,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303960,"kb_used_data":2576,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2637824,"data_stored":1322685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46400000000000002,"15min":0.46100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.246},"max":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.495},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.746,"15min":0.76500000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47399999999999998,"15min":0.47999999999999998},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.753},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.307},"max":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.751,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.079,"5min":1.177,"15min":1.177},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.58099999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.97799999999999998,"5min":1.0329999999999999,"15min":1.081},"last":0.93400000000000005}]},{"osd":5,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.60499999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.0980000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.141},"last":0.71599999999999997}]},{"osd":6,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.67700000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.84699999999999998},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.98099999999999998}]},{"osd":7,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.022,"5min":1.141,"15min":1.214},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.68700000000000006,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.748}]}]},{"osd":4,"up_from":14,"seq":60129543280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312140,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059700,"statfs":{"total":96636764160,"available":96317132800,"internally_reserved":0,"allocated":2625536,"data_stored":1321113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.008,"15min":1.008},"last":0.55300000000000005}]},{"osd":1,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71499999999999997,"15min":0.73699999999999999},"min":{"1min":0.55200000000000005,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":1.0329999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73299999999999998,"15min":0.75},"min":{"1min":0.58599999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.55100000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.998},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.51900000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84399999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.628,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":0.96799999999999997,"15min":0.98999999999999999},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.73099999999999998,"5min":1.002,"15min":1.024},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.495,"15min":0.49199999999999999},"max":{"1min":0.77800000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":0.94099999999999995,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.80600000000000005}]}]},{"osd":5,"up_from":15,"seq":64424510575,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300436,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071404,"statfs":{"total":96636764160,"available":96329117696,"internally_reserved":0,"allocated":3223552,"data_stored":1912023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.60799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.92700000000000005,"15min":1.0629999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.016},"last":0.70299999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.012,"15min":1.022},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.99199999999999999},"last":0.71699999999999997}]},{"osd":3,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":0.997,"15min":0.997},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.63},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93000000000000005,"15min":0.95199999999999996},"last":0.50700000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.97799999999999998,"15min":1.042},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.90600000000000003,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.76500000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.90500000000000003,"15min":0.996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:12:31.831 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:12:32.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.032+0000 7f700b1ff700 1 Processor -- start 2023-12-26T23:12:32.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.032+0000 7f700b1ff700 1 -- start start 2023-12-26T23:12:32.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.033+0000 7f700b1ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70040b07f0 0x7f70041554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.033+0000 7f700b1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004159cf0 0x7f7004155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.033+0000 7f700b1ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70040c8bf0 con 0x7f7004159920 2023-12-26T23:12:32.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.033+0000 7f700b1ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7004155f50 con 0x7f70040b07f0 2023-12-26T23:12:32.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.033+0000 7f700b1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040c8e60 con 0x7f7004159cf0 2023-12-26T23:12:32.036 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f7008f9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7004159920 0x7f70040b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32864/0 (socket says 172.21.15.131:32864) 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f7008f9b700 1 -- 172.21.15.131:0/4146883851 learned_addr learned my addr 172.21.15.131:0/4146883851 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f700979c700 1 --2- 172.21.15.131:0/4146883851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004159cf0 0x7f7004155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f7003fff700 1 --2- 172.21.15.131:0/4146883851 >> v2:172.21.15.190:3300/0 conn(0x7f70040b07f0 0x7f70041554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f70037fe700 1 -- 172.21.15.131:0/4146883851 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832573903 0 0) 0x7f70040c8bf0 con 0x7f7004159920 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f70037fe700 1 -- 172.21.15.131:0/4146883851 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff4003570 con 0x7f7004159920 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f700979c700 1 -- 172.21.15.131:0/4146883851 >> v2:172.21.15.190:3300/0 conn(0x7f70040b07f0 msgr2=0x7f70041554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f700979c700 1 --2- 172.21.15.131:0/4146883851 >> v2:172.21.15.190:3300/0 conn(0x7f70040b07f0 0x7f70041554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f700979c700 1 -- 172.21.15.131:0/4146883851 >> v1:172.21.15.131:6789/0 conn(0x7f7004159920 legacy=0x7f70040b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f700979c700 1 -- 172.21.15.131:0/4146883851 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70040c9140 con 0x7f7004159cf0 2023-12-26T23:12:32.037 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.034+0000 7f7003fff700 1 --2- 172.21.15.131:0/4146883851 >> v2:172.21.15.190:3300/0 conn(0x7f70040b07f0 0x7f70041554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.035+0000 7f700979c700 1 --2- 172.21.15.131:0/4146883851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004159cf0 0x7f7004155a10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009e60 tx=0x7f6ffc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88d116530adf44e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.035+0000 7f70037fe700 1 -- 172.21.15.131:0/4146883851 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00e070 con 0x7f7004159cf0 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.035+0000 7f70037fe700 1 -- 172.21.15.131:0/4146883851 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc00bab0 con 0x7f7004159cf0 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.035+0000 7f70037fe700 1 -- 172.21.15.131:0/4146883851 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00be70 con 0x7f7004159cf0 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.035+0000 7f700b1ff700 1 -- 172.21.15.131:0/4146883851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004159cf0 msgr2=0x7f7004155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.035+0000 7f700b1ff700 1 --2- 172.21.15.131:0/4146883851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004159cf0 0x7f7004155a10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009e60 tx=0x7f6ffc009780 comp rx=0 tx=0).stop 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.035+0000 7f700b1ff700 1 -- 172.21.15.131:0/4146883851 shutdown_connections 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.036+0000 7f700b1ff700 1 --2- 172.21.15.131:0/4146883851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004159cf0 0x7f7004155a10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.036+0000 7f700b1ff700 1 --2- 172.21.15.131:0/4146883851 >> v2:172.21.15.190:3300/0 conn(0x7f70040b07f0 0x7f70041554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.036+0000 7f700b1ff700 1 -- 172.21.15.131:0/4146883851 >> 172.21.15.131:0/4146883851 conn(0x7f7004068040 msgr2=0x7f7004069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.036+0000 7f700b1ff700 1 -- 172.21.15.131:0/4146883851 shutdown_connections 2023-12-26T23:12:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.036+0000 7f700b1ff700 1 -- 172.21.15.131:0/4146883851 wait complete. 2023-12-26T23:12:32.041 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.038+0000 7f700b1ff700 1 Processor -- start 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.038+0000 7f700b1ff700 1 -- start start 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f700b1ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004159920 0x7f7004156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f700b1ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7004159cf0 0x7f7004158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f700b1ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040af220 0x7f7004169e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f700b1ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70041594b0 con 0x7f7004159cf0 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f700b1ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7004156ff0 con 0x7f7004159920 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f700b1ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70041572d0 con 0x7f70040af220 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f7003fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7004159cf0 0x7f7004158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f7003fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7004159cf0 0x7f7004158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55232/0 (socket says 172.21.15.131:55232) 2023-12-26T23:12:32.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f7003fff700 1 -- 172.21.15.131:0/2733685055 learned_addr learned my addr 172.21.15.131:0/2733685055 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f7008f9b700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004159920 0x7f7004156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.039+0000 7f700979c700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040af220 0x7f7004169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f7003fff700 1 -- 172.21.15.131:0/2733685055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040af220 msgr2=0x7f7004169e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f7003fff700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040af220 0x7f7004169e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f7003fff700 1 -- 172.21.15.131:0/2733685055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004159920 msgr2=0x7f7004156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f7003fff700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004159920 0x7f7004156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f7003fff700 1 -- 172.21.15.131:0/2733685055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0009420 con 0x7f7004159cf0 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f7003fff700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7004159cf0 0x7f7004158f70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000ec80 tx=0x7f6ff00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f7000ff9700 1 -- 172.21.15.131:0/2733685055 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000c850 con 0x7f7004159cf0 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f7000ff9700 1 -- 172.21.15.131:0/2733685055 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff000c9b0 con 0x7f7004159cf0 2023-12-26T23:12:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f7000ff9700 1 -- 172.21.15.131:0/2733685055 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000cd50 con 0x7f7004159cf0 2023-12-26T23:12:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.040+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc009a20 con 0x7f7004159cf0 2023-12-26T23:12:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.041+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700414e300 con 0x7f7004159cf0 2023-12-26T23:12:32.044 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.041+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70040ccee0 con 0x7f7004159cf0 2023-12-26T23:12:32.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.042+0000 7f7000ff9700 1 -- 172.21.15.131:0/2733685055 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6ff0013680 con 0x7f7004159cf0 2023-12-26T23:12:32.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.043+0000 7f7000ff9700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fe8066f30 0x7f6fe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.043+0000 7f7000ff9700 1 -- 172.21.15.131:0/2733685055 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f6ff00848c0 con 0x7f7004159cf0 2023-12-26T23:12:32.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.043+0000 7f7008f9b700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fe8066f30 0x7f6fe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.044+0000 7f7008f9b700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fe8066f30 0x7f6fe80693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8007fd0 tx=0x7f6ff8009040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:32.048 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.048+0000 7f7000ff9700 1 -- 172.21.15.131:0/2733685055 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ff0051880 con 0x7f7004159cf0 2023-12-26T23:12:32.237 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.237+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70040657e0 con 0x7f6fe8066f30 2023-12-26T23:12:32.246 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.245+0000 7f7000ff9700 1 -- 172.21.15.131:0/2733685055 <== mgr.9927 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289657 (secure 0 0 0) 0x7f70040657e0 con 0x7f6fe8066f30 2023-12-26T23:12:32.250 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.249+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fe8066f30 msgr2=0x7f6fe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.249+0000 7f700b1ff700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fe8066f30 0x7f6fe80693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8007fd0 tx=0x7f6ff8009040 comp rx=0 tx=0).stop 2023-12-26T23:12:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.249+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7004159cf0 msgr2=0x7f7004158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.249+0000 7f700b1ff700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7004159cf0 0x7f7004158f70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000ec80 tx=0x7f6ff00091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.250+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 shutdown_connections 2023-12-26T23:12:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.250+0000 7f700b1ff700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6fe8066f30 0x7f6fe80693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.250+0000 7f700b1ff700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040af220 0x7f7004169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.250+0000 7f700b1ff700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004159920 0x7f7004156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.254 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.250+0000 7f700b1ff700 1 --2- 172.21.15.131:0/2733685055 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7004159cf0 0x7f7004158f70 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.250+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 >> 172.21.15.131:0/2733685055 conn(0x7f7004068040 msgr2=0x7f70041502d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:32.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.250+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 shutdown_connections 2023-12-26T23:12:32.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.250+0000 7f700b1ff700 1 -- 172.21.15.131:0/2733685055 wait complete. 2023-12-26T23:12:32.255 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:12:32.268 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-26T23:12:32.067280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":624,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453256,"kb_used_data":21704,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752521464,"statfs":{"total":773094113280,"available":770581979136,"internally_reserved":0,"allocated":22224896,"data_stored":11753980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.486770"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.619284+0000","last_change":"2023-12-26T23:12:22.618470+0000","last_active":"2023-12-26T23:12:22.619284+0000","last_peered":"2023-12-26T23:12:22.619284+0000","last_clean":"2023-12-26T23:12:22.619284+0000","last_became_active":"2023-12-26T23:12:22.617303+0000","last_became_peered":"2023-12-26T23:12:22.617303+0000","last_unstale":"2023-12-26T23:12:22.619284+0000","last_undegraded":"2023-12-26T23:12:22.619284+0000","last_fullsized":"2023-12-26T23:12:22.619284+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:59:09.130170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.616029+0000","last_change":"2023-12-26T23:12:22.614091+0000","last_active":"2023-12-26T23:12:22.616029+0000","last_peered":"2023-12-26T23:12:22.616029+0000","last_clean":"2023-12-26T23:12:22.616029+0000","last_became_active":"2023-12-26T23:12:22.613796+0000","last_became_peered":"2023-12-26T23:12:22.613796+0000","last_unstale":"2023-12-26T23:12:22.616029+0000","last_undegraded":"2023-12-26T23:12:22.616029+0000","last_fullsized":"2023-12-26T23:12:22.616029+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:02:50.533341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612431+0000","last_change":"2023-12-26T23:12:22.598911+0000","last_active":"2023-12-26T23:12:22.612431+0000","last_peered":"2023-12-26T23:12:22.612431+0000","last_clean":"2023-12-26T23:12:22.612431+0000","last_became_active":"2023-12-26T23:12:22.594770+0000","last_became_peered":"2023-12-26T23:12:22.594770+0000","last_unstale":"2023-12-26T23:12:22.612431+0000","last_undegraded":"2023-12-26T23:12:22.612431+0000","last_fullsized":"2023-12-26T23:12:22.612431+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:25:21.227639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.610093+0000","last_change":"2023-12-26T23:12:22.603716+0000","last_active":"2023-12-26T23:12:22.610093+0000","last_peered":"2023-12-26T23:12:22.610093+0000","last_clean":"2023-12-26T23:12:22.610093+0000","last_became_active":"2023-12-26T23:12:22.602197+0000","last_became_peered":"2023-12-26T23:12:22.602197+0000","last_unstale":"2023-12-26T23:12:22.610093+0000","last_undegraded":"2023-12-26T23:12:22.610093+0000","last_fullsized":"2023-12-26T23:12:22.610093+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:34:42.233229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.605074+0000","last_change":"2023-12-26T23:12:22.603086+0000","last_active":"2023-12-26T23:12:22.605074+0000","last_peered":"2023-12-26T23:12:22.605074+0000","last_clean":"2023-12-26T23:12:22.605074+0000","last_became_active":"2023-12-26T23:12:22.602382+0000","last_became_peered":"2023-12-26T23:12:22.602382+0000","last_unstale":"2023-12-26T23:12:22.605074+0000","last_undegraded":"2023-12-26T23:12:22.605074+0000","last_fullsized":"2023-12-26T23:12:22.605074+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:36:44.985279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.606178+0000","last_change":"2023-12-26T23:12:22.601550+0000","last_active":"2023-12-26T23:12:22.606178+0000","last_peered":"2023-12-26T23:12:22.606178+0000","last_clean":"2023-12-26T23:12:22.606178+0000","last_became_active":"2023-12-26T23:12:22.601268+0000","last_became_peered":"2023-12-26T23:12:22.601268+0000","last_unstale":"2023-12-26T23:12:22.606178+0000","last_undegraded":"2023-12-26T23:12:22.606178+0000","last_fullsized":"2023-12-26T23:12:22.606178+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:17:39.054404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604731+0000","last_change":"2023-12-26T23:12:22.601334+0000","last_active":"2023-12-26T23:12:22.604731+0000","last_peered":"2023-12-26T23:12:22.604731+0000","last_clean":"2023-12-26T23:12:22.604731+0000","last_became_active":"2023-12-26T23:12:22.601066+0000","last_became_peered":"2023-12-26T23:12:22.601066+0000","last_unstale":"2023-12-26T23:12:22.604731+0000","last_undegraded":"2023-12-26T23:12:22.604731+0000","last_fullsized":"2023-12-26T23:12:22.604731+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:56:50.812363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.611742+0000","last_change":"2023-12-26T23:12:22.605627+0000","last_active":"2023-12-26T23:12:22.611742+0000","last_peered":"2023-12-26T23:12:22.611742+0000","last_clean":"2023-12-26T23:12:22.611742+0000","last_became_active":"2023-12-26T23:12:22.605384+0000","last_became_peered":"2023-12-26T23:12:22.605384+0000","last_unstale":"2023-12-26T23:12:22.611742+0000","last_undegraded":"2023-12-26T23:12:22.611742+0000","last_fullsized":"2023-12-26T23:12:22.611742+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:19:55.261546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612225+0000","last_change":"2023-12-26T23:12:22.605430+0000","last_active":"2023-12-26T23:12:22.612225+0000","last_peered":"2023-12-26T23:12:22.612225+0000","last_clean":"2023-12-26T23:12:22.612225+0000","last_became_active":"2023-12-26T23:12:22.604985+0000","last_became_peered":"2023-12-26T23:12:22.604985+0000","last_unstale":"2023-12-26T23:12:22.612225+0000","last_undegraded":"2023-12-26T23:12:22.612225+0000","last_fullsized":"2023-12-26T23:12:22.612225+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:57:10.519369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621526+0000","last_change":"2023-12-26T23:12:22.616480+0000","last_active":"2023-12-26T23:12:22.621526+0000","last_peered":"2023-12-26T23:12:22.621526+0000","last_clean":"2023-12-26T23:12:22.621526+0000","last_became_active":"2023-12-26T23:12:22.616238+0000","last_became_peered":"2023-12-26T23:12:22.616238+0000","last_unstale":"2023-12-26T23:12:22.621526+0000","last_undegraded":"2023-12-26T23:12:22.621526+0000","last_fullsized":"2023-12-26T23:12:22.621526+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:35:39.625849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608917+0000","last_change":"2023-12-26T23:12:22.601320+0000","last_active":"2023-12-26T23:12:22.608917+0000","last_peered":"2023-12-26T23:12:22.608917+0000","last_clean":"2023-12-26T23:12:22.608917+0000","last_became_active":"2023-12-26T23:12:22.601045+0000","last_became_peered":"2023-12-26T23:12:22.601045+0000","last_unstale":"2023-12-26T23:12:22.608917+0000","last_undegraded":"2023-12-26T23:12:22.608917+0000","last_fullsized":"2023-12-26T23:12:22.608917+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:08:36.371857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603091+0000","last_change":"2023-12-26T23:12:22.598407+0000","last_active":"2023-12-26T23:12:22.603091+0000","last_peered":"2023-12-26T23:12:22.603091+0000","last_clean":"2023-12-26T23:12:22.603091+0000","last_became_active":"2023-12-26T23:12:22.598072+0000","last_became_peered":"2023-12-26T23:12:22.598072+0000","last_unstale":"2023-12-26T23:12:22.603091+0000","last_undegraded":"2023-12-26T23:12:22.603091+0000","last_fullsized":"2023-12-26T23:12:22.603091+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:22.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.615088+0000","last_change":"2023-12-26T23:12:22.610046+0000","last_active":"2023-12-26T23:12:22.615088+0000","last_peered":"2023-12-26T23:12:22.615088+0000","last_clean":"2023-12-26T23:12:22.615088+0000","last_became_active":"2023-12-26T23:12:22.609779+0000","last_became_peered":"2023-12-26T23:12:22.609779+0000","last_unstale":"2023-12-26T23:12:22.615088+0000","last_undegraded":"2023-12-26T23:12:22.615088+0000","last_fullsized":"2023-12-26T23:12:22.615088+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:06:42.889746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.619149+0000","last_change":"2023-12-26T23:12:22.616561+0000","last_active":"2023-12-26T23:12:22.619149+0000","last_peered":"2023-12-26T23:12:22.619149+0000","last_clean":"2023-12-26T23:12:22.619149+0000","last_became_active":"2023-12-26T23:12:22.616312+0000","last_became_peered":"2023-12-26T23:12:22.616312+0000","last_unstale":"2023-12-26T23:12:22.619149+0000","last_undegraded":"2023-12-26T23:12:22.619149+0000","last_fullsized":"2023-12-26T23:12:22.619149+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:28:59.228392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608685+0000","last_change":"2023-12-26T23:12:22.603229+0000","last_active":"2023-12-26T23:12:22.608685+0000","last_peered":"2023-12-26T23:12:22.608685+0000","last_clean":"2023-12-26T23:12:22.608685+0000","last_became_active":"2023-12-26T23:12:22.602956+0000","last_became_peered":"2023-12-26T23:12:22.602956+0000","last_unstale":"2023-12-26T23:12:22.608685+0000","last_undegraded":"2023-12-26T23:12:22.608685+0000","last_fullsized":"2023-12-26T23:12:22.608685+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:06:35.396701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608290+0000","last_change":"2023-12-26T23:12:22.594739+0000","last_active":"2023-12-26T23:12:22.608290+0000","last_peered":"2023-12-26T23:12:22.608290+0000","last_clean":"2023-12-26T23:12:22.608290+0000","last_became_active":"2023-12-26T23:12:22.594453+0000","last_became_peered":"2023-12-26T23:12:22.594453+0000","last_unstale":"2023-12-26T23:12:22.608290+0000","last_undegraded":"2023-12-26T23:12:22.608290+0000","last_fullsized":"2023-12-26T23:12:22.608290+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:09:22.739311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621517+0000","last_change":"2023-12-26T23:12:22.620583+0000","last_active":"2023-12-26T23:12:22.621517+0000","last_peered":"2023-12-26T23:12:22.621517+0000","last_clean":"2023-12-26T23:12:22.621517+0000","last_became_active":"2023-12-26T23:12:22.619749+0000","last_became_peered":"2023-12-26T23:12:22.619749+0000","last_unstale":"2023-12-26T23:12:22.621517+0000","last_undegraded":"2023-12-26T23:12:22.621517+0000","last_fullsized":"2023-12-26T23:12:22.621517+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:26:45.615428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.610988+0000","last_change":"2023-12-26T23:12:22.604727+0000","last_active":"2023-12-26T23:12:22.610988+0000","last_peered":"2023-12-26T23:12:22.610988+0000","last_clean":"2023-12-26T23:12:22.610988+0000","last_became_active":"2023-12-26T23:12:22.604470+0000","last_became_peered":"2023-12-26T23:12:22.604470+0000","last_unstale":"2023-12-26T23:12:22.610988+0000","last_undegraded":"2023-12-26T23:12:22.610988+0000","last_fullsized":"2023-12-26T23:12:22.610988+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:22:40.411083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612470+0000","last_change":"2023-12-26T23:12:22.600667+0000","last_active":"2023-12-26T23:12:22.612470+0000","last_peered":"2023-12-26T23:12:22.612470+0000","last_clean":"2023-12-26T23:12:22.612470+0000","last_became_active":"2023-12-26T23:12:22.600410+0000","last_became_peered":"2023-12-26T23:12:22.600410+0000","last_unstale":"2023-12-26T23:12:22.612470+0000","last_undegraded":"2023-12-26T23:12:22.612470+0000","last_fullsized":"2023-12-26T23:12:22.612470+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:46:04.681070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.600465+0000","last_change":"2023-12-26T23:12:22.597138+0000","last_active":"2023-12-26T23:12:22.600465+0000","last_peered":"2023-12-26T23:12:22.600465+0000","last_clean":"2023-12-26T23:12:22.600465+0000","last_became_active":"2023-12-26T23:12:22.596819+0000","last_became_peered":"2023-12-26T23:12:22.596819+0000","last_unstale":"2023-12-26T23:12:22.600465+0000","last_undegraded":"2023-12-26T23:12:22.600465+0000","last_fullsized":"2023-12-26T23:12:22.600465+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:34:48.264386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.609802+0000","last_change":"2023-12-26T23:12:22.598384+0000","last_active":"2023-12-26T23:12:22.609802+0000","last_peered":"2023-12-26T23:12:22.609802+0000","last_clean":"2023-12-26T23:12:22.609802+0000","last_became_active":"2023-12-26T23:12:22.598087+0000","last_became_peered":"2023-12-26T23:12:22.598087+0000","last_unstale":"2023-12-26T23:12:22.609802+0000","last_undegraded":"2023-12-26T23:12:22.609802+0000","last_fullsized":"2023-12-26T23:12:22.609802+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:06:33.817797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604041+0000","last_change":"2023-12-26T23:12:22.598903+0000","last_active":"2023-12-26T23:12:22.604041+0000","last_peered":"2023-12-26T23:12:22.604041+0000","last_clean":"2023-12-26T23:12:22.604041+0000","last_became_active":"2023-12-26T23:12:22.598650+0000","last_became_peered":"2023-12-26T23:12:22.598650+0000","last_unstale":"2023-12-26T23:12:22.604041+0000","last_undegraded":"2023-12-26T23:12:22.604041+0000","last_fullsized":"2023-12-26T23:12:22.604041+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:00:16.909284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621055+0000","last_change":"2023-12-26T23:12:22.613831+0000","last_active":"2023-12-26T23:12:22.621055+0000","last_peered":"2023-12-26T23:12:22.621055+0000","last_clean":"2023-12-26T23:12:22.621055+0000","last_became_active":"2023-12-26T23:12:22.613551+0000","last_became_peered":"2023-12-26T23:12:22.613551+0000","last_unstale":"2023-12-26T23:12:22.621055+0000","last_undegraded":"2023-12-26T23:12:22.621055+0000","last_fullsized":"2023-12-26T23:12:22.621055+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:26:27.918216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.596011+0000","last_change":"2023-12-26T23:12:22.592199+0000","last_active":"2023-12-26T23:12:22.596011+0000","last_peered":"2023-12-26T23:12:22.596011+0000","last_clean":"2023-12-26T23:12:22.596011+0000","last_became_active":"2023-12-26T23:12:22.591939+0000","last_became_peered":"2023-12-26T23:12:22.591939+0000","last_unstale":"2023-12-26T23:12:22.596011+0000","last_undegraded":"2023-12-26T23:12:22.596011+0000","last_fullsized":"2023-12-26T23:12:22.596011+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:06:39.203393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618290+0000","last_change":"2023-12-26T23:12:22.616401+0000","last_active":"2023-12-26T23:12:22.618290+0000","last_peered":"2023-12-26T23:12:22.618290+0000","last_clean":"2023-12-26T23:12:22.618290+0000","last_became_active":"2023-12-26T23:12:22.614857+0000","last_became_peered":"2023-12-26T23:12:22.614857+0000","last_unstale":"2023-12-26T23:12:22.618290+0000","last_undegraded":"2023-12-26T23:12:22.618290+0000","last_fullsized":"2023-12-26T23:12:22.618290+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:32:52.603246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.605824+0000","last_change":"2023-12-26T23:12:22.600468+0000","last_active":"2023-12-26T23:12:22.605824+0000","last_peered":"2023-12-26T23:12:22.605824+0000","last_clean":"2023-12-26T23:12:22.605824+0000","last_became_active":"2023-12-26T23:12:22.599383+0000","last_became_peered":"2023-12-26T23:12:22.599383+0000","last_unstale":"2023-12-26T23:12:22.605824+0000","last_undegraded":"2023-12-26T23:12:22.605824+0000","last_fullsized":"2023-12-26T23:12:22.605824+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:46:04.681070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612452+0000","last_change":"2023-12-26T23:12:22.599229+0000","last_active":"2023-12-26T23:12:22.612452+0000","last_peered":"2023-12-26T23:12:22.612452+0000","last_clean":"2023-12-26T23:12:22.612452+0000","last_became_active":"2023-12-26T23:12:22.598549+0000","last_became_peered":"2023-12-26T23:12:22.598549+0000","last_unstale":"2023-12-26T23:12:22.612452+0000","last_undegraded":"2023-12-26T23:12:22.612452+0000","last_fullsized":"2023-12-26T23:12:22.612452+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:46.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621129+0000","last_change":"2023-12-26T23:12:22.614738+0000","last_active":"2023-12-26T23:12:22.621129+0000","last_peered":"2023-12-26T23:12:22.621129+0000","last_clean":"2023-12-26T23:12:22.621129+0000","last_became_active":"2023-12-26T23:12:22.614424+0000","last_became_peered":"2023-12-26T23:12:22.614424+0000","last_unstale":"2023-12-26T23:12:22.621129+0000","last_undegraded":"2023-12-26T23:12:22.621129+0000","last_fullsized":"2023-12-26T23:12:22.621129+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:02:50.533341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618423+0000","last_change":"2023-12-26T23:12:22.614833+0000","last_active":"2023-12-26T23:12:22.618423+0000","last_peered":"2023-12-26T23:12:22.618423+0000","last_clean":"2023-12-26T23:12:22.618423+0000","last_became_active":"2023-12-26T23:12:22.613753+0000","last_became_peered":"2023-12-26T23:12:22.613753+0000","last_unstale":"2023-12-26T23:12:22.618423+0000","last_undegraded":"2023-12-26T23:12:22.618423+0000","last_fullsized":"2023-12-26T23:12:22.618423+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:41:07.888826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.599929+0000","last_change":"2023-12-26T23:12:22.595182+0000","last_active":"2023-12-26T23:12:22.599929+0000","last_peered":"2023-12-26T23:12:22.599929+0000","last_clean":"2023-12-26T23:12:22.599929+0000","last_became_active":"2023-12-26T23:12:22.594899+0000","last_became_peered":"2023-12-26T23:12:22.594899+0000","last_unstale":"2023-12-26T23:12:22.599929+0000","last_undegraded":"2023-12-26T23:12:22.599929+0000","last_fullsized":"2023-12-26T23:12:22.599929+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:25:21.227639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604069+0000","last_change":"2023-12-26T23:12:22.599793+0000","last_active":"2023-12-26T23:12:22.604069+0000","last_peered":"2023-12-26T23:12:22.604069+0000","last_clean":"2023-12-26T23:12:22.604069+0000","last_became_active":"2023-12-26T23:12:22.599266+0000","last_became_peered":"2023-12-26T23:12:22.599266+0000","last_unstale":"2023-12-26T23:12:22.604069+0000","last_undegraded":"2023-12-26T23:12:22.604069+0000","last_fullsized":"2023-12-26T23:12:22.604069+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:21:15.194964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.611803+0000","last_change":"2023-12-26T23:12:22.605971+0000","last_active":"2023-12-26T23:12:22.611803+0000","last_peered":"2023-12-26T23:12:22.611803+0000","last_clean":"2023-12-26T23:12:22.611803+0000","last_became_active":"2023-12-26T23:12:22.605682+0000","last_became_peered":"2023-12-26T23:12:22.605682+0000","last_unstale":"2023-12-26T23:12:22.611803+0000","last_undegraded":"2023-12-26T23:12:22.611803+0000","last_fullsized":"2023-12-26T23:12:22.611803+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:40:29.805595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.607264+0000","last_change":"2023-12-26T23:12:22.601107+0000","last_active":"2023-12-26T23:12:22.607264+0000","last_peered":"2023-12-26T23:12:22.607264+0000","last_clean":"2023-12-26T23:12:22.607264+0000","last_became_active":"2023-12-26T23:12:22.600830+0000","last_became_peered":"2023-12-26T23:12:22.600830+0000","last_unstale":"2023-12-26T23:12:22.607264+0000","last_undegraded":"2023-12-26T23:12:22.607264+0000","last_fullsized":"2023-12-26T23:12:22.607264+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:28:30.485485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"209'4","reported_seq":14,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621799+0000","last_change":"2023-12-26T23:12:21.597828+0000","last_active":"2023-12-26T23:12:22.621799+0000","last_peered":"2023-12-26T23:12:22.621799+0000","last_clean":"2023-12-26T23:12:22.621799+0000","last_became_active":"2023-12-26T23:12:21.597518+0000","last_became_peered":"2023-12-26T23:12:21.597518+0000","last_unstale":"2023-12-26T23:12:22.621799+0000","last_undegraded":"2023-12-26T23:12:22.621799+0000","last_fullsized":"2023-12-26T23:12:22.621799+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:34:41.226776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621267+0000","last_change":"2023-12-26T23:12:22.615871+0000","last_active":"2023-12-26T23:12:22.621267+0000","last_peered":"2023-12-26T23:12:22.621267+0000","last_clean":"2023-12-26T23:12:22.621267+0000","last_became_active":"2023-12-26T23:12:22.613133+0000","last_became_peered":"2023-12-26T23:12:22.613133+0000","last_unstale":"2023-12-26T23:12:22.621267+0000","last_undegraded":"2023-12-26T23:12:22.621267+0000","last_fullsized":"2023-12-26T23:12:22.621267+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:20:54.097124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.615100+0000","last_change":"2023-12-26T23:12:21.596760+0000","last_active":"2023-12-26T23:12:22.615100+0000","last_peered":"2023-12-26T23:12:22.615100+0000","last_clean":"2023-12-26T23:12:22.615100+0000","last_became_active":"2023-12-26T23:12:21.596526+0000","last_became_peered":"2023-12-26T23:12:21.596526+0000","last_unstale":"2023-12-26T23:12:22.615100+0000","last_undegraded":"2023-12-26T23:12:22.615100+0000","last_fullsized":"2023-12-26T23:12:22.615100+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:34:41.226776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.613735+0000","last_change":"2023-12-26T23:12:22.610129+0000","last_active":"2023-12-26T23:12:22.613735+0000","last_peered":"2023-12-26T23:12:22.613735+0000","last_clean":"2023-12-26T23:12:22.613735+0000","last_became_active":"2023-12-26T23:12:22.609846+0000","last_became_peered":"2023-12-26T23:12:22.609846+0000","last_unstale":"2023-12-26T23:12:22.613735+0000","last_undegraded":"2023-12-26T23:12:22.613735+0000","last_fullsized":"2023-12-26T23:12:22.613735+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:20:54.097124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618351+0000","last_change":"2023-12-26T23:12:21.588180+0000","last_active":"2023-12-26T23:12:22.618351+0000","last_peered":"2023-12-26T23:12:22.618351+0000","last_clean":"2023-12-26T23:12:22.618351+0000","last_became_active":"2023-12-26T23:12:21.587108+0000","last_became_peered":"2023-12-26T23:12:21.587108+0000","last_unstale":"2023-12-26T23:12:22.618351+0000","last_undegraded":"2023-12-26T23:12:22.618351+0000","last_fullsized":"2023-12-26T23:12:22.618351+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:40:27.356111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618347+0000","last_change":"2023-12-26T23:12:22.616410+0000","last_active":"2023-12-26T23:12:22.618347+0000","last_peered":"2023-12-26T23:12:22.618347+0000","last_clean":"2023-12-26T23:12:22.618347+0000","last_became_active":"2023-12-26T23:12:22.616142+0000","last_became_peered":"2023-12-26T23:12:22.616142+0000","last_unstale":"2023-12-26T23:12:22.618347+0000","last_undegraded":"2023-12-26T23:12:22.618347+0000","last_fullsized":"2023-12-26T23:12:22.618347+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:12:16.766697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.615327+0000","last_change":"2023-12-26T23:12:21.596338+0000","last_active":"2023-12-26T23:12:22.615327+0000","last_peered":"2023-12-26T23:12:22.615327+0000","last_clean":"2023-12-26T23:12:22.615327+0000","last_became_active":"2023-12-26T23:12:21.595927+0000","last_became_peered":"2023-12-26T23:12:21.595927+0000","last_unstale":"2023-12-26T23:12:22.615327+0000","last_undegraded":"2023-12-26T23:12:22.615327+0000","last_fullsized":"2023-12-26T23:12:22.615327+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:02:26.059586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604678+0000","last_change":"2023-12-26T23:12:22.601298+0000","last_active":"2023-12-26T23:12:22.604678+0000","last_peered":"2023-12-26T23:12:22.604678+0000","last_clean":"2023-12-26T23:12:22.604678+0000","last_became_active":"2023-12-26T23:12:22.600886+0000","last_became_peered":"2023-12-26T23:12:22.600886+0000","last_unstale":"2023-12-26T23:12:22.604678+0000","last_undegraded":"2023-12-26T23:12:22.604678+0000","last_fullsized":"2023-12-26T23:12:22.604678+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:06:59.903118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618449+0000","last_change":"2023-12-26T23:12:21.587617+0000","last_active":"2023-12-26T23:12:22.618449+0000","last_peered":"2023-12-26T23:12:22.618449+0000","last_clean":"2023-12-26T23:12:22.618449+0000","last_became_active":"2023-12-26T23:12:21.586950+0000","last_became_peered":"2023-12-26T23:12:21.586950+0000","last_unstale":"2023-12-26T23:12:22.618449+0000","last_undegraded":"2023-12-26T23:12:22.618449+0000","last_fullsized":"2023-12-26T23:12:22.618449+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:45.449822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.606216+0000","last_change":"2023-12-26T23:12:22.603294+0000","last_active":"2023-12-26T23:12:22.606216+0000","last_peered":"2023-12-26T23:12:22.606216+0000","last_clean":"2023-12-26T23:12:22.606216+0000","last_became_active":"2023-12-26T23:12:22.602790+0000","last_became_peered":"2023-12-26T23:12:22.602790+0000","last_unstale":"2023-12-26T23:12:22.606216+0000","last_undegraded":"2023-12-26T23:12:22.606216+0000","last_fullsized":"2023-12-26T23:12:22.606216+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:53:38.971245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620325+0000","last_change":"2023-12-26T23:12:21.594983+0000","last_active":"2023-12-26T23:12:22.620325+0000","last_peered":"2023-12-26T23:12:22.620325+0000","last_clean":"2023-12-26T23:12:22.620325+0000","last_became_active":"2023-12-26T23:12:21.594105+0000","last_became_peered":"2023-12-26T23:12:21.594105+0000","last_unstale":"2023-12-26T23:12:22.620325+0000","last_undegraded":"2023-12-26T23:12:22.620325+0000","last_fullsized":"2023-12-26T23:12:22.620325+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:07:34.526239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608706+0000","last_change":"2023-12-26T23:12:22.604454+0000","last_active":"2023-12-26T23:12:22.608706+0000","last_peered":"2023-12-26T23:12:22.608706+0000","last_clean":"2023-12-26T23:12:22.608706+0000","last_became_active":"2023-12-26T23:12:22.604080+0000","last_became_peered":"2023-12-26T23:12:22.604080+0000","last_unstale":"2023-12-26T23:12:22.608706+0000","last_undegraded":"2023-12-26T23:12:22.608706+0000","last_fullsized":"2023-12-26T23:12:22.608706+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:27:30.344416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.588365+0000","last_change":"2023-12-26T23:12:21.600860+0000","last_active":"2023-12-26T23:12:22.588365+0000","last_peered":"2023-12-26T23:12:22.588365+0000","last_clean":"2023-12-26T23:12:22.588365+0000","last_became_active":"2023-12-26T23:12:21.599603+0000","last_became_peered":"2023-12-26T23:12:21.599603+0000","last_unstale":"2023-12-26T23:12:22.588365+0000","last_undegraded":"2023-12-26T23:12:22.588365+0000","last_fullsized":"2023-12-26T23:12:22.588365+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:00:15.902830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.611155+0000","last_change":"2023-12-26T23:12:22.602861+0000","last_active":"2023-12-26T23:12:22.611155+0000","last_peered":"2023-12-26T23:12:22.611155+0000","last_clean":"2023-12-26T23:12:22.611155+0000","last_became_active":"2023-12-26T23:12:22.601876+0000","last_became_peered":"2023-12-26T23:12:22.601876+0000","last_unstale":"2023-12-26T23:12:22.611155+0000","last_undegraded":"2023-12-26T23:12:22.611155+0000","last_fullsized":"2023-12-26T23:12:22.611155+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:14:02.597163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.584550+0000","last_change":"2023-12-26T23:12:21.592994+0000","last_active":"2023-12-26T23:12:22.584550+0000","last_peered":"2023-12-26T23:12:22.584550+0000","last_clean":"2023-12-26T23:12:22.584550+0000","last_became_active":"2023-12-26T23:12:21.592737+0000","last_became_peered":"2023-12-26T23:12:21.592737+0000","last_unstale":"2023-12-26T23:12:22.584550+0000","last_undegraded":"2023-12-26T23:12:22.584550+0000","last_fullsized":"2023-12-26T23:12:22.584550+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:53:05.285994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621289+0000","last_change":"2023-12-26T23:12:22.615955+0000","last_active":"2023-12-26T23:12:22.621289+0000","last_peered":"2023-12-26T23:12:22.621289+0000","last_clean":"2023-12-26T23:12:22.621289+0000","last_became_active":"2023-12-26T23:12:22.613156+0000","last_became_peered":"2023-12-26T23:12:22.613156+0000","last_unstale":"2023-12-26T23:12:22.621289+0000","last_undegraded":"2023-12-26T23:12:22.621289+0000","last_fullsized":"2023-12-26T23:12:22.621289+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:12:18.182031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603410+0000","last_change":"2023-12-26T23:12:21.598664+0000","last_active":"2023-12-26T23:12:22.603410+0000","last_peered":"2023-12-26T23:12:22.603410+0000","last_clean":"2023-12-26T23:12:22.603410+0000","last_became_active":"2023-12-26T23:12:21.598354+0000","last_became_peered":"2023-12-26T23:12:21.598354+0000","last_unstale":"2023-12-26T23:12:22.603410+0000","last_undegraded":"2023-12-26T23:12:22.603410+0000","last_fullsized":"2023-12-26T23:12:22.603410+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T11:07:57.669862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.607814+0000","last_change":"2023-12-26T23:12:22.602916+0000","last_active":"2023-12-26T23:12:22.607814+0000","last_peered":"2023-12-26T23:12:22.607814+0000","last_clean":"2023-12-26T23:12:22.607814+0000","last_became_active":"2023-12-26T23:12:22.602558+0000","last_became_peered":"2023-12-26T23:12:22.602558+0000","last_unstale":"2023-12-26T23:12:22.607814+0000","last_undegraded":"2023-12-26T23:12:22.607814+0000","last_fullsized":"2023-12-26T23:12:22.607814+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:06:33.817797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"209'3","reported_seq":13,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620387+0000","last_change":"2023-12-26T23:12:21.594707+0000","last_active":"2023-12-26T23:12:22.620387+0000","last_peered":"2023-12-26T23:12:22.620387+0000","last_clean":"2023-12-26T23:12:22.620387+0000","last_became_active":"2023-12-26T23:12:21.593719+0000","last_became_peered":"2023-12-26T23:12:21.593719+0000","last_unstale":"2023-12-26T23:12:22.620387+0000","last_undegraded":"2023-12-26T23:12:22.620387+0000","last_fullsized":"2023-12-26T23:12:22.620387+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:41:09.150367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621331+0000","last_change":"2023-12-26T23:12:22.616107+0000","last_active":"2023-12-26T23:12:22.621331+0000","last_peered":"2023-12-26T23:12:22.621331+0000","last_clean":"2023-12-26T23:12:22.621331+0000","last_became_active":"2023-12-26T23:12:22.615638+0000","last_became_peered":"2023-12-26T23:12:22.615638+0000","last_unstale":"2023-12-26T23:12:22.621331+0000","last_undegraded":"2023-12-26T23:12:22.621331+0000","last_fullsized":"2023-12-26T23:12:22.621331+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:04:07.058946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"209'6","reported_seq":16,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620612+0000","last_change":"2023-12-26T23:12:21.594282+0000","last_active":"2023-12-26T23:12:22.620612+0000","last_peered":"2023-12-26T23:12:22.620612+0000","last_clean":"2023-12-26T23:12:22.620612+0000","last_became_active":"2023-12-26T23:12:21.593997+0000","last_became_peered":"2023-12-26T23:12:21.593997+0000","last_unstale":"2023-12-26T23:12:22.620612+0000","last_undegraded":"2023-12-26T23:12:22.620612+0000","last_fullsized":"2023-12-26T23:12:22.620612+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:51:22.275494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621073+0000","last_change":"2023-12-26T23:12:22.614073+0000","last_active":"2023-12-26T23:12:22.621073+0000","last_peered":"2023-12-26T23:12:22.621073+0000","last_clean":"2023-12-26T23:12:22.621073+0000","last_became_active":"2023-12-26T23:12:22.613801+0000","last_became_peered":"2023-12-26T23:12:22.613801+0000","last_unstale":"2023-12-26T23:12:22.621073+0000","last_undegraded":"2023-12-26T23:12:22.621073+0000","last_fullsized":"2023-12-26T23:12:22.621073+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:45:45.843764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618480+0000","last_change":"2023-12-26T23:12:21.600730+0000","last_active":"2023-12-26T23:12:22.618480+0000","last_peered":"2023-12-26T23:12:22.618480+0000","last_clean":"2023-12-26T23:12:22.618480+0000","last_became_active":"2023-12-26T23:12:21.600494+0000","last_became_peered":"2023-12-26T23:12:21.600494+0000","last_unstale":"2023-12-26T23:12:22.618480+0000","last_undegraded":"2023-12-26T23:12:22.618480+0000","last_fullsized":"2023-12-26T23:12:22.618480+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:55:28.859094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603708+0000","last_change":"2023-12-26T23:12:22.601486+0000","last_active":"2023-12-26T23:12:22.603708+0000","last_peered":"2023-12-26T23:12:22.603708+0000","last_clean":"2023-12-26T23:12:22.603708+0000","last_became_active":"2023-12-26T23:12:22.601147+0000","last_became_peered":"2023-12-26T23:12:22.601147+0000","last_unstale":"2023-12-26T23:12:22.603708+0000","last_undegraded":"2023-12-26T23:12:22.603708+0000","last_fullsized":"2023-12-26T23:12:22.603708+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:50:31.681211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603634+0000","last_change":"2023-12-26T23:12:21.598293+0000","last_active":"2023-12-26T23:12:22.603634+0000","last_peered":"2023-12-26T23:12:22.603634+0000","last_clean":"2023-12-26T23:12:22.603634+0000","last_became_active":"2023-12-26T23:12:21.597331+0000","last_became_peered":"2023-12-26T23:12:21.597331+0000","last_unstale":"2023-12-26T23:12:22.603634+0000","last_undegraded":"2023-12-26T23:12:22.603634+0000","last_fullsized":"2023-12-26T23:12:22.603634+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:15:47.071802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.602753+0000","last_change":"2023-12-26T23:12:22.599542+0000","last_active":"2023-12-26T23:12:22.602753+0000","last_peered":"2023-12-26T23:12:22.602753+0000","last_clean":"2023-12-26T23:12:22.602753+0000","last_became_active":"2023-12-26T23:12:22.599236+0000","last_became_peered":"2023-12-26T23:12:22.599236+0000","last_unstale":"2023-12-26T23:12:22.602753+0000","last_undegraded":"2023-12-26T23:12:22.602753+0000","last_fullsized":"2023-12-26T23:12:22.602753+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:43.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.584228+0000","last_change":"2023-12-26T23:12:21.596976+0000","last_active":"2023-12-26T23:12:22.584228+0000","last_peered":"2023-12-26T23:12:22.584228+0000","last_clean":"2023-12-26T23:12:22.584228+0000","last_became_active":"2023-12-26T23:12:21.596562+0000","last_became_peered":"2023-12-26T23:12:21.596562+0000","last_unstale":"2023-12-26T23:12:22.584228+0000","last_undegraded":"2023-12-26T23:12:22.584228+0000","last_fullsized":"2023-12-26T23:12:22.584228+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:31:24.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.606281+0000","last_change":"2023-12-26T23:12:22.603629+0000","last_active":"2023-12-26T23:12:22.606281+0000","last_peered":"2023-12-26T23:12:22.606281+0000","last_clean":"2023-12-26T23:12:22.606281+0000","last_became_active":"2023-12-26T23:12:22.603139+0000","last_became_peered":"2023-12-26T23:12:22.603139+0000","last_unstale":"2023-12-26T23:12:22.606281+0000","last_undegraded":"2023-12-26T23:12:22.606281+0000","last_fullsized":"2023-12-26T23:12:22.606281+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:38:16.070134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"209'8","reported_seq":18,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618204+0000","last_change":"2023-12-26T23:12:21.599125+0000","last_active":"2023-12-26T23:12:22.618204+0000","last_peered":"2023-12-26T23:12:22.618204+0000","last_clean":"2023-12-26T23:12:22.618204+0000","last_became_active":"2023-12-26T23:12:21.598466+0000","last_became_peered":"2023-12-26T23:12:21.598466+0000","last_unstale":"2023-12-26T23:12:22.618204+0000","last_undegraded":"2023-12-26T23:12:22.618204+0000","last_fullsized":"2023-12-26T23:12:22.618204+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:48:45.758030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621303+0000","last_change":"2023-12-26T23:12:22.615839+0000","last_active":"2023-12-26T23:12:22.621303+0000","last_peered":"2023-12-26T23:12:22.621303+0000","last_clean":"2023-12-26T23:12:22.621303+0000","last_became_active":"2023-12-26T23:12:22.612485+0000","last_became_peered":"2023-12-26T23:12:22.612485+0000","last_unstale":"2023-12-26T23:12:22.621303+0000","last_undegraded":"2023-12-26T23:12:22.621303+0000","last_fullsized":"2023-12-26T23:12:22.621303+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:13:40.795030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620863+0000","last_change":"2023-12-26T23:12:21.589505+0000","last_active":"2023-12-26T23:12:22.620863+0000","last_peered":"2023-12-26T23:12:22.620863+0000","last_clean":"2023-12-26T23:12:22.620863+0000","last_became_active":"2023-12-26T23:12:21.589099+0000","last_became_peered":"2023-12-26T23:12:21.589099+0000","last_unstale":"2023-12-26T23:12:22.620863+0000","last_undegraded":"2023-12-26T23:12:22.620863+0000","last_fullsized":"2023-12-26T23:12:22.620863+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:00:59.073908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608591+0000","last_change":"2023-12-26T23:12:22.606348+0000","last_active":"2023-12-26T23:12:22.608591+0000","last_peered":"2023-12-26T23:12:22.608591+0000","last_clean":"2023-12-26T23:12:22.608591+0000","last_became_active":"2023-12-26T23:12:22.605943+0000","last_became_peered":"2023-12-26T23:12:22.605943+0000","last_unstale":"2023-12-26T23:12:22.608591+0000","last_undegraded":"2023-12-26T23:12:22.608591+0000","last_fullsized":"2023-12-26T23:12:22.608591+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:48.326459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.618546+0000","last_change":"2023-12-26T23:12:21.598748+0000","last_active":"2023-12-26T23:12:22.618546+0000","last_peered":"2023-12-26T23:12:22.618546+0000","last_clean":"2023-12-26T23:12:22.618546+0000","last_became_active":"2023-12-26T23:12:21.598315+0000","last_became_peered":"2023-12-26T23:12:21.598315+0000","last_unstale":"2023-12-26T23:12:22.618546+0000","last_undegraded":"2023-12-26T23:12:22.618546+0000","last_fullsized":"2023-12-26T23:12:22.618546+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:30:13.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.603689+0000","last_change":"2023-12-26T23:12:22.600650+0000","last_active":"2023-12-26T23:12:22.603689+0000","last_peered":"2023-12-26T23:12:22.603689+0000","last_clean":"2023-12-26T23:12:22.603689+0000","last_became_active":"2023-12-26T23:12:22.600058+0000","last_became_peered":"2023-12-26T23:12:22.600058+0000","last_unstale":"2023-12-26T23:12:22.603689+0000","last_undegraded":"2023-12-26T23:12:22.603689+0000","last_fullsized":"2023-12-26T23:12:22.603689+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T10:09:20.640840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"209'1","reported_seq":11,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.612806+0000","last_change":"2023-12-26T23:12:21.591902+0000","last_active":"2023-12-26T23:12:22.612806+0000","last_peered":"2023-12-26T23:12:22.612806+0000","last_clean":"2023-12-26T23:12:22.612806+0000","last_became_active":"2023-12-26T23:12:21.591634+0000","last_became_peered":"2023-12-26T23:12:21.591634+0000","last_unstale":"2023-12-26T23:12:22.612806+0000","last_undegraded":"2023-12-26T23:12:22.612806+0000","last_fullsized":"2023-12-26T23:12:22.612806+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:20:51.780226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.613198+0000","last_change":"2023-12-26T23:12:22.610294+0000","last_active":"2023-12-26T23:12:22.613198+0000","last_peered":"2023-12-26T23:12:22.613198+0000","last_clean":"2023-12-26T23:12:22.613198+0000","last_became_active":"2023-12-26T23:12:22.610016+0000","last_became_peered":"2023-12-26T23:12:22.610016+0000","last_unstale":"2023-12-26T23:12:22.613198+0000","last_undegraded":"2023-12-26T23:12:22.613198+0000","last_fullsized":"2023-12-26T23:12:22.613198+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:12:46.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.585094+0000","last_change":"2023-12-26T23:12:21.598986+0000","last_active":"2023-12-26T23:12:22.585094+0000","last_peered":"2023-12-26T23:12:22.585094+0000","last_clean":"2023-12-26T23:12:22.585094+0000","last_became_active":"2023-12-26T23:12:21.598653+0000","last_became_peered":"2023-12-26T23:12:21.598653+0000","last_unstale":"2023-12-26T23:12:22.585094+0000","last_undegraded":"2023-12-26T23:12:22.585094+0000","last_fullsized":"2023-12-26T23:12:22.585094+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:23:22.158692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.598051+0000","last_change":"2023-12-26T23:12:22.593725+0000","last_active":"2023-12-26T23:12:22.598051+0000","last_peered":"2023-12-26T23:12:22.598051+0000","last_clean":"2023-12-26T23:12:22.598051+0000","last_became_active":"2023-12-26T23:12:22.593427+0000","last_became_peered":"2023-12-26T23:12:22.593427+0000","last_unstale":"2023-12-26T23:12:22.598051+0000","last_undegraded":"2023-12-26T23:12:22.598051+0000","last_fullsized":"2023-12-26T23:12:22.598051+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:17:09.261315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.614710+0000","last_change":"2023-12-26T23:12:21.599507+0000","last_active":"2023-12-26T23:12:22.614710+0000","last_peered":"2023-12-26T23:12:22.614710+0000","last_clean":"2023-12-26T23:12:22.614710+0000","last_became_active":"2023-12-26T23:12:21.599227+0000","last_became_peered":"2023-12-26T23:12:21.599227+0000","last_unstale":"2023-12-26T23:12:22.614710+0000","last_undegraded":"2023-12-26T23:12:22.614710+0000","last_fullsized":"2023-12-26T23:12:22.614710+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-27T23:18:38.340055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.617470+0000","last_change":"2023-12-26T23:12:22.616253+0000","last_active":"2023-12-26T23:12:22.617470+0000","last_peered":"2023-12-26T23:12:22.617470+0000","last_clean":"2023-12-26T23:12:22.617470+0000","last_became_active":"2023-12-26T23:12:22.615948+0000","last_became_peered":"2023-12-26T23:12:22.615948+0000","last_unstale":"2023-12-26T23:12:22.617470+0000","last_undegraded":"2023-12-26T23:12:22.617470+0000","last_fullsized":"2023-12-26T23:12:22.617470+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:22:48.592908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.589695+0000","last_change":"2023-12-26T23:12:21.599064+0000","last_active":"2023-12-26T23:12:22.589695+0000","last_peered":"2023-12-26T23:12:22.589695+0000","last_clean":"2023-12-26T23:12:22.589695+0000","last_became_active":"2023-12-26T23:12:21.598809+0000","last_became_peered":"2023-12-26T23:12:21.598809+0000","last_unstale":"2023-12-26T23:12:22.589695+0000","last_undegraded":"2023-12-26T23:12:22.589695+0000","last_fullsized":"2023-12-26T23:12:22.589695+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:06:45.449822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.599212+0000","last_change":"2023-12-26T23:12:22.594491+0000","last_active":"2023-12-26T23:12:22.599212+0000","last_peered":"2023-12-26T23:12:22.599212+0000","last_clean":"2023-12-26T23:12:22.599212+0000","last_became_active":"2023-12-26T23:12:22.592704+0000","last_became_peered":"2023-12-26T23:12:22.592704+0000","last_unstale":"2023-12-26T23:12:22.599212+0000","last_undegraded":"2023-12-26T23:12:22.599212+0000","last_fullsized":"2023-12-26T23:12:22.599212+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:54:32.482725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.589629+0000","last_change":"2023-12-26T23:12:21.600931+0000","last_active":"2023-12-26T23:12:22.589629+0000","last_peered":"2023-12-26T23:12:22.589629+0000","last_clean":"2023-12-26T23:12:22.589629+0000","last_became_active":"2023-12-26T23:12:21.600544+0000","last_became_peered":"2023-12-26T23:12:21.600544+0000","last_unstale":"2023-12-26T23:12:22.589629+0000","last_undegraded":"2023-12-26T23:12:22.589629+0000","last_fullsized":"2023-12-26T23:12:22.589629+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:00:59.073908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.621457+0000","last_change":"2023-12-26T23:12:22.616366+0000","last_active":"2023-12-26T23:12:22.621457+0000","last_peered":"2023-12-26T23:12:22.621457+0000","last_clean":"2023-12-26T23:12:22.621457+0000","last_became_active":"2023-12-26T23:12:22.616068+0000","last_became_peered":"2023-12-26T23:12:22.616068+0000","last_unstale":"2023-12-26T23:12:22.621457+0000","last_undegraded":"2023-12-26T23:12:22.621457+0000","last_fullsized":"2023-12-26T23:12:22.621457+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:17:18.359852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.584483+0000","last_change":"2023-12-26T23:12:21.593115+0000","last_active":"2023-12-26T23:12:22.584483+0000","last_peered":"2023-12-26T23:12:22.584483+0000","last_clean":"2023-12-26T23:12:22.584483+0000","last_became_active":"2023-12-26T23:12:21.592871+0000","last_became_peered":"2023-12-26T23:12:21.592871+0000","last_unstale":"2023-12-26T23:12:22.584483+0000","last_undegraded":"2023-12-26T23:12:22.584483+0000","last_fullsized":"2023-12-26T23:12:22.584483+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T02:58:32.040716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.611177+0000","last_change":"2023-12-26T23:12:22.603136+0000","last_active":"2023-12-26T23:12:22.611177+0000","last_peered":"2023-12-26T23:12:22.611177+0000","last_clean":"2023-12-26T23:12:22.611177+0000","last_became_active":"2023-12-26T23:12:22.602273+0000","last_became_peered":"2023-12-26T23:12:22.602273+0000","last_unstale":"2023-12-26T23:12:22.611177+0000","last_undegraded":"2023-12-26T23:12:22.611177+0000","last_fullsized":"2023-12-26T23:12:22.611177+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:12:43.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"209'2","reported_seq":12,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.620211+0000","last_change":"2023-12-26T23:12:21.598993+0000","last_active":"2023-12-26T23:12:22.620211+0000","last_peered":"2023-12-26T23:12:22.620211+0000","last_clean":"2023-12-26T23:12:22.620211+0000","last_became_active":"2023-12-26T23:12:21.598721+0000","last_became_peered":"2023-12-26T23:12:21.598721+0000","last_unstale":"2023-12-26T23:12:22.620211+0000","last_undegraded":"2023-12-26T23:12:22.620211+0000","last_fullsized":"2023-12-26T23:12:22.620211+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:19:59.693301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608111+0000","last_change":"2023-12-26T23:12:22.591386+0000","last_active":"2023-12-26T23:12:22.608111+0000","last_peered":"2023-12-26T23:12:22.608111+0000","last_clean":"2023-12-26T23:12:22.608111+0000","last_became_active":"2023-12-26T23:12:22.590472+0000","last_became_peered":"2023-12-26T23:12:22.590472+0000","last_unstale":"2023-12-26T23:12:22.608111+0000","last_undegraded":"2023-12-26T23:12:22.608111+0000","last_fullsized":"2023-12-26T23:12:22.608111+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:14:48.326459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"210'1","reported_seq":11,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-12-26T23:12:24.228769+0000","last_change":"2023-12-26T23:12:21.594735+0000","last_active":"2023-12-26T23:12:24.228769+0000","last_peered":"2023-12-26T23:12:24.228769+0000","last_clean":"2023-12-26T23:12:24.228769+0000","last_became_active":"2023-12-26T23:12:21.594457+0000","last_became_peered":"2023-12-26T23:12:21.594457+0000","last_unstale":"2023-12-26T23:12:24.228769+0000","last_undegraded":"2023-12-26T23:12:24.228769+0000","last_fullsized":"2023-12-26T23:12:24.228769+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:55:28.859094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.608254+0000","last_change":"2023-12-26T23:12:22.595522+0000","last_active":"2023-12-26T23:12:22.608254+0000","last_peered":"2023-12-26T23:12:22.608254+0000","last_clean":"2023-12-26T23:12:22.608254+0000","last_became_active":"2023-12-26T23:12:22.595272+0000","last_became_peered":"2023-12-26T23:12:22.595272+0000","last_unstale":"2023-12-26T23:12:22.608254+0000","last_undegraded":"2023-12-26T23:12:22.608254+0000","last_fullsized":"2023-12-26T23:12:22.608254+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T09:41:07.888826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.587423+0000","last_change":"2023-12-26T23:12:21.594905+0000","last_active":"2023-12-26T23:12:22.587423+0000","last_peered":"2023-12-26T23:12:22.587423+0000","last_clean":"2023-12-26T23:12:22.587423+0000","last_became_active":"2023-12-26T23:12:21.594594+0000","last_became_peered":"2023-12-26T23:12:21.594594+0000","last_unstale":"2023-12-26T23:12:22.587423+0000","last_undegraded":"2023-12-26T23:12:22.587423+0000","last_fullsized":"2023-12-26T23:12:22.587423+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:02:26.059586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.604857+0000","last_change":"2023-12-26T23:12:22.603023+0000","last_active":"2023-12-26T23:12:22.604857+0000","last_peered":"2023-12-26T23:12:22.604857+0000","last_clean":"2023-12-26T23:12:22.604857+0000","last_became_active":"2023-12-26T23:12:22.602614+0000","last_became_peered":"2023-12-26T23:12:22.602614+0000","last_unstale":"2023-12-26T23:12:22.604857+0000","last_undegraded":"2023-12-26T23:12:22.604857+0000","last_fullsized":"2023-12-26T23:12:22.604857+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T08:06:32.152177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.583856+0000","last_change":"2023-12-26T23:12:21.589618+0000","last_active":"2023-12-26T23:12:22.583856+0000","last_peered":"2023-12-26T23:12:22.583856+0000","last_clean":"2023-12-26T23:12:22.583856+0000","last_became_active":"2023-12-26T23:12:21.589304+0000","last_became_peered":"2023-12-26T23:12:21.589304+0000","last_unstale":"2023-12-26T23:12:22.583856+0000","last_undegraded":"2023-12-26T23:12:22.583856+0000","last_fullsized":"2023-12-26T23:12:22.583856+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:30:13.000064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.613603+0000","last_change":"2023-12-26T23:12:22.610118+0000","last_active":"2023-12-26T23:12:22.613603+0000","last_peered":"2023-12-26T23:12:22.613603+0000","last_clean":"2023-12-26T23:12:22.613603+0000","last_became_active":"2023-12-26T23:12:22.609493+0000","last_became_peered":"2023-12-26T23:12:22.609493+0000","last_unstale":"2023-12-26T23:12:22.613603+0000","last_undegraded":"2023-12-26T23:12:22.613603+0000","last_fullsized":"2023-12-26T23:12:22.613603+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T06:26:27.918216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.594917+0000","last_change":"2023-12-26T23:12:21.588645+0000","last_active":"2023-12-26T23:12:22.594917+0000","last_peered":"2023-12-26T23:12:22.594917+0000","last_clean":"2023-12-26T23:12:22.594917+0000","last_became_active":"2023-12-26T23:12:21.586977+0000","last_became_peered":"2023-12-26T23:12:21.586977+0000","last_unstale":"2023-12-26T23:12:22.594917+0000","last_undegraded":"2023-12-26T23:12:22.594917+0000","last_fullsized":"2023-12-26T23:12:22.594917+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:47:26.727484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.601005+0000","last_change":"2023-12-26T23:12:22.595720+0000","last_active":"2023-12-26T23:12:22.601005+0000","last_peered":"2023-12-26T23:12:22.601005+0000","last_clean":"2023-12-26T23:12:22.601005+0000","last_became_active":"2023-12-26T23:12:22.595460+0000","last_became_peered":"2023-12-26T23:12:22.595460+0000","last_unstale":"2023-12-26T23:12:22.601005+0000","last_undegraded":"2023-12-26T23:12:22.601005+0000","last_fullsized":"2023-12-26T23:12:22.601005+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:58:55.095066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.591100+0000","last_change":"2023-12-26T23:12:21.593706+0000","last_active":"2023-12-26T23:12:22.591100+0000","last_peered":"2023-12-26T23:12:22.591100+0000","last_clean":"2023-12-26T23:12:22.591100+0000","last_became_active":"2023-12-26T23:12:21.593437+0000","last_became_peered":"2023-12-26T23:12:21.593437+0000","last_unstale":"2023-12-26T23:12:22.591100+0000","last_undegraded":"2023-12-26T23:12:22.591100+0000","last_fullsized":"2023-12-26T23:12:22.591100+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T07:32:14.373113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.619186+0000","last_change":"2023-12-26T23:12:22.617685+0000","last_active":"2023-12-26T23:12:22.619186+0000","last_peered":"2023-12-26T23:12:22.619186+0000","last_clean":"2023-12-26T23:12:22.619186+0000","last_became_active":"2023-12-26T23:12:22.616842+0000","last_became_peered":"2023-12-26T23:12:22.616842+0000","last_unstale":"2023-12-26T23:12:22.619186+0000","last_undegraded":"2023-12-26T23:12:22.619186+0000","last_fullsized":"2023-12-26T23:12:22.619186+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T04:45:45.843764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.597186+0000","last_change":"2023-12-26T23:12:21.600294+0000","last_active":"2023-12-26T23:12:22.597186+0000","last_peered":"2023-12-26T23:12:22.597186+0000","last_clean":"2023-12-26T23:12:22.597186+0000","last_became_active":"2023-12-26T23:12:21.600012+0000","last_became_peered":"2023-12-26T23:12:21.600012+0000","last_unstale":"2023-12-26T23:12:22.597186+0000","last_undegraded":"2023-12-26T23:12:22.597186+0000","last_fullsized":"2023-12-26T23:12:22.597186+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:01:13.020675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.606344+0000","last_change":"2023-12-26T23:12:22.603754+0000","last_active":"2023-12-26T23:12:22.606344+0000","last_peered":"2023-12-26T23:12:22.606344+0000","last_clean":"2023-12-26T23:12:22.606344+0000","last_became_active":"2023-12-26T23:12:22.603507+0000","last_became_peered":"2023-12-26T23:12:22.603507+0000","last_unstale":"2023-12-26T23:12:22.606344+0000","last_undegraded":"2023-12-26T23:12:22.606344+0000","last_fullsized":"2023-12-26T23:12:22.606344+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:48:46.764483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.590291+0000","last_change":"2023-12-26T23:12:21.597965+0000","last_active":"2023-12-26T23:12:22.590291+0000","last_peered":"2023-12-26T23:12:22.590291+0000","last_clean":"2023-12-26T23:12:22.590291+0000","last_became_active":"2023-12-26T23:12:21.597321+0000","last_became_peered":"2023-12-26T23:12:21.597321+0000","last_unstale":"2023-12-26T23:12:22.590291+0000","last_undegraded":"2023-12-26T23:12:22.590291+0000","last_fullsized":"2023-12-26T23:12:22.590291+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T01:31:24.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.607612+0000","last_change":"2023-12-26T23:12:22.602610+0000","last_active":"2023-12-26T23:12:22.607612+0000","last_peered":"2023-12-26T23:12:22.607612+0000","last_clean":"2023-12-26T23:12:22.607612+0000","last_became_active":"2023-12-26T23:12:22.602027+0000","last_became_peered":"2023-12-26T23:12:22.602027+0000","last_unstale":"2023-12-26T23:12:22.607612+0000","last_undegraded":"2023-12-26T23:12:22.607612+0000","last_fullsized":"2023-12-26T23:12:22.607612+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":208,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:21.567608+0000","last_clean_scrub_stamp":"2023-12-26T23:12:21.567608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:05:13.435725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.591824+0000","last_change":"2023-12-26T23:12:21.598921+0000","last_active":"2023-12-26T23:12:22.591824+0000","last_peered":"2023-12-26T23:12:22.591824+0000","last_clean":"2023-12-26T23:12:22.591824+0000","last_became_active":"2023-12-26T23:12:21.598690+0000","last_became_peered":"2023-12-26T23:12:21.598690+0000","last_unstale":"2023-12-26T23:12:22.591824+0000","last_undegraded":"2023-12-26T23:12:22.591824+0000","last_fullsized":"2023-12-26T23:12:22.591824+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T23:12:20.561154+0000","last_clean_scrub_stamp":"2023-12-26T23:12:20.561154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T03:16:39.345340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"199'348","reported_seq":1152,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-26T23:12:22.597625+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:12:22.597625+0000","last_peered":"2023-12-26T23:12:22.597625+0000","last_clean":"2023-12-26T23:12:22.597625+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:12:22.597625+0000","last_undegraded":"2023-12-26T23:12:22.597625+0000","last_fullsized":"2023-12-26T23:12:22.597625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543281,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308060,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063780,"statfs":{"total":96636764160,"available":96321310720,"internally_reserved":0,"allocated":2641920,"data_stored":1322727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.98199999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.499}]},{"osd":1,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.498,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.72199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.0449999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.94899999999999995,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1},"last":0.66200000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0760000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.74399999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.44500000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.5,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.59699999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":1.028,"15min":1.028},"last":0.82199999999999995}]}]},{"osd":6,"up_from":15,"seq":64424510577,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1321169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.32800000000000001},"max":{"1min":0.873,"5min":0.998,"15min":1.1080000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0680000000000001},"last":0.76900000000000002}]},{"osd":1,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":1.022,"15min":1.046},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.95999999999999996,"15min":1.0580000000000001},"last":0.75600000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91500000000000004,"15min":1.071},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":0.89800000000000002,"5min":0.97899999999999998,"15min":1.0680000000000001},"last":0.73999999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.96699999999999997,"15min":1.024},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":1.0149999999999999},"last":0.72499999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.018,"15min":1.1599999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.51600000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.111},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.78100000000000003}]}]},{"osd":1,"up_from":15,"seq":64424510577,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300428,"kb_used_data":3140,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071412,"statfs":{"total":96636764160,"available":96329125888,"internally_reserved":0,"allocated":3215360,"data_stored":1911447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.29199999999999998,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.66200000000000003,"5min":0.68400000000000005,"15min":0.81399999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.50600000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.622,"5min":0.76900000000000002,"15min":0.79200000000000004},"last":0.36599999999999999}]},{"osd":2,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.33800000000000002},"max":{"1min":0.58099999999999996,"5min":0.753,"15min":0.84099999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.65800000000000003,"5min":0.75700000000000001,"15min":0.91200000000000003},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.70299999999999996,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.501},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.46700000000000003,"5min":0.40600000000000003,"15min":0.32700000000000001},"max":{"1min":0.67500000000000004,"5min":0.872,"15min":0.93600000000000005},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.629,"5min":0.55100000000000005,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":1.103,"15min":1.103},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.79100000000000004,"15min":0.77500000000000002},"min":{"1min":0.60899999999999999,"5min":0.52500000000000002,"15min":0.48599999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.72099999999999997}]},{"osd":5,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.50700000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82999999999999996,"15min":0.83199999999999996},"min":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.1100000000000001,"5min":1.1240000000000001,"15min":1.1830000000000001},"last":0.77000000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.753,"5min":0.57599999999999996,"15min":0.52600000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.85899999999999999,"15min":0.84299999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.75}]},{"osd":7,"last update":"Tue Dec 26 23:11:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.81299999999999994},"min":{"1min":0.64700000000000002,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.218},"last":0.70999999999999996}]}]},{"osd":0,"up_from":15,"seq":64424510577,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308048,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063792,"statfs":{"total":96636764160,"available":96321323008,"internally_reserved":0,"allocated":2629632,"data_stored":1321667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.26600000000000001},"max":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.502,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.313},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.30599999999999999},"max":{"1min":0.72899999999999998,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.73399999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38500000000000001}]},{"osd":3,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.88100000000000001,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88,"5min":0.91800000000000004,"15min":1.012},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.2250000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77400000000000002,"15min":0.78300000000000003},"min":{"1min":0.66400000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.113},"last":0.57099999999999995}]},{"osd":5,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80400000000000005,"15min":0.81799999999999995},"min":{"1min":0.63,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0469999999999999,"5min":1.0920000000000001,"15min":1.157},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.82899999999999996,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.282},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.79500000000000004,"15min":0.80500000000000005},"min":{"1min":0.72599999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.121,"5min":1.121,"15min":1.151},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.62}]},{"osd":7,"last update":"Tue Dec 26 23:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81799999999999995,"15min":0.82199999999999995},"min":{"1min":0.67500000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1020000000000001,"5min":1.173,"15min":1.256},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.64200000000000002,"5min":0.54800000000000004,"15min":0.54000000000000004},"max":{"1min":1.087,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.64400000000000002}]}]},{"osd":2,"up_from":14,"seq":60129543282,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312140,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059700,"statfs":{"total":96636764160,"available":96317132800,"internally_reserved":0,"allocated":2625536,"data_stored":1321149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":0.84599999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69199999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.45800000000000002,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.64700000000000002,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.79200000000000004,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80200000000000005,"5min":0.89300000000000002,"15min":1.056},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.65600000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1180000000000001,"5min":1.8,"15min":1.8},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.59099999999999997,"15min":0.54000000000000004},"max":{"1min":1.115,"5min":1.222,"15min":1.222},"last":0.875}]},{"osd":5,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":1.079,"5min":1.923,"15min":1.923},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81599999999999995,"15min":0.81100000000000005},"min":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.087,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.66500000000000004,"5min":0.55300000000000005,"15min":0.52400000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.272},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0269999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Tue Dec 26 23:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.56399999999999995},"max":{"1min":1.042,"5min":1.1100000000000001,"15min":1.1619999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.84499999999999997},"min":{"1min":0.70999999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0740000000000001,"5min":1.841,"15min":1.841},"last":0.82599999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510578,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303960,"kb_used_data":2576,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2637824,"data_stored":1322685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46400000000000002,"15min":0.46100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.246},"max":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.746,"15min":0.76500000000000001},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47399999999999998,"15min":0.47999999999999998},"min":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.753},"last":0.436},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.307},"max":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59999999999999998}]},{"osd":2,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.372},"max":{"1min":0.81799999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.501,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.751,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64100000000000001}]},{"osd":4,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.079,"5min":1.177,"15min":1.177},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.58099999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.97799999999999998,"5min":1.0329999999999999,"15min":1.081},"last":0.77900000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.60499999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.0980000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.141},"last":0.81200000000000006}]},{"osd":6,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.67700000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.84699999999999998},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.82699999999999996}]},{"osd":7,"last update":"Tue Dec 26 23:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81499999999999995,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.022,"5min":1.141,"15min":1.214},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.68700000000000006,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.98399999999999999}]}]},{"osd":4,"up_from":14,"seq":60129543281,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312140,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059700,"statfs":{"total":96636764160,"available":96317132800,"internally_reserved":0,"allocated":2625536,"data_stored":1321113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.41599999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.008,"15min":1.008},"last":0.55300000000000005}]},{"osd":1,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71499999999999997,"15min":0.73699999999999999},"min":{"1min":0.55200000000000005,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.89400000000000002,"15min":1.0329999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73299999999999998,"15min":0.75},"min":{"1min":0.58599999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.55100000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.998},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.51900000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84399999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.628,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":0.96799999999999997,"15min":0.98999999999999999},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.73099999999999998,"5min":1.002,"15min":1.024},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Dec 26 23:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.495,"15min":0.49199999999999999},"max":{"1min":0.77800000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.48399999999999999},"max":{"1min":0.94099999999999995,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.80600000000000005}]}]},{"osd":5,"up_from":15,"seq":64424510576,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300436,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071404,"statfs":{"total":96636764160,"available":96329117696,"internally_reserved":0,"allocated":3223552,"data_stored":1912023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.60799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.92700000000000005,"15min":1.0629999999999999},"last":0.749}]},{"osd":1,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.016},"last":0.67700000000000005}]},{"osd":2,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.012,"15min":1.022},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.99199999999999999},"last":0.625}]},{"osd":3,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":0.997,"15min":0.997},"last":0.73299999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93000000000000005,"15min":0.95199999999999996},"last":0.76400000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.97799999999999998,"15min":1.042},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.90600000000000003,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.90500000000000003,"15min":0.996},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.79300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:12:32.273 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-26T23:12:32.273 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-26T23:12:32.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.505+0000 7ff0953ff700 1 Processor -- start 2023-12-26T23:12:32.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.505+0000 7ff0953ff700 1 -- start start 2023-12-26T23:12:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.506+0000 7ff0953ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0900cb360 0x7ff0900cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.506+0000 7ff0953ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0900caed0 con 0x7ff090157050 2023-12-26T23:12:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.506+0000 7ff0953ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0900ca8e0 con 0x7ff090157420 2023-12-26T23:12:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.506+0000 7ff0953ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0900c9520 con 0x7ff0900cb360 2023-12-26T23:12:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.506+0000 7ff08effd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff090157050 0x7ff0900686c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32872/0 (socket says 172.21.15.131:32872) 2023-12-26T23:12:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.506+0000 7ff08effd700 1 -- 172.21.15.131:0/3337574325 learned_addr learned my addr 172.21.15.131:0/3337574325 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.506+0000 7ff08f7fe700 1 --2- 172.21.15.131:0/3337574325 >> v2:172.21.15.190:3300/0 conn(0x7ff0900cb360 0x7ff0900cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.507+0000 7ff08dffb700 1 -- 172.21.15.131:0/3337574325 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002117211 0 0) 0x7ff0900caed0 con 0x7ff090157050 2023-12-26T23:12:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.507+0000 7ff08dffb700 1 -- 172.21.15.131:0/3337574325 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff078003570 con 0x7ff090157050 2023-12-26T23:12:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.507+0000 7ff08f7fe700 1 -- 172.21.15.131:0/3337574325 >> v1:172.21.15.190:6789/0 conn(0x7ff090157420 legacy=0x7ff090159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.507+0000 7ff08dffb700 1 -- 172.21.15.131:0/3337574325 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255136621 0 0) 0x7ff078003570 con 0x7ff090157050 2023-12-26T23:12:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.507+0000 7ff08f7fe700 1 -- 172.21.15.131:0/3337574325 >> v1:172.21.15.131:6789/0 conn(0x7ff090157050 legacy=0x7ff0900686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.507+0000 7ff08f7fe700 1 -- 172.21.15.131:0/3337574325 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09015a6c0 con 0x7ff0900cb360 2023-12-26T23:12:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.507+0000 7ff08f7fe700 1 --2- 172.21.15.131:0/3337574325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0900cb360 0x7ff0900cb800 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff088009e60 tx=0x7ff088009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abd3c05586a416a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff08dffb700 1 -- 172.21.15.131:0/3337574325 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800e070 con 0x7ff0900cb360 2023-12-26T23:12:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff08dffb700 1 -- 172.21.15.131:0/3337574325 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08800b850 con 0x7ff0900cb360 2023-12-26T23:12:32.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff08dffb700 1 -- 172.21.15.131:0/3337574325 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080001470 con 0x7ff0900cb360 2023-12-26T23:12:32.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff0953ff700 1 -- 172.21.15.131:0/3337574325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0900cb360 msgr2=0x7ff0900cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff0953ff700 1 --2- 172.21.15.131:0/3337574325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0900cb360 0x7ff0900cb800 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff088009e60 tx=0x7ff088009780 comp rx=0 tx=0).stop 2023-12-26T23:12:32.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff0953ff700 1 -- 172.21.15.131:0/3337574325 shutdown_connections 2023-12-26T23:12:32.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff0953ff700 1 --2- 172.21.15.131:0/3337574325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0900cb360 0x7ff0900cb800 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff0953ff700 1 -- 172.21.15.131:0/3337574325 >> 172.21.15.131:0/3337574325 conn(0x7ff09006dc80 msgr2=0x7ff0900675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:32.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff0953ff700 1 -- 172.21.15.131:0/3337574325 shutdown_connections 2023-12-26T23:12:32.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.508+0000 7ff0953ff700 1 -- 172.21.15.131:0/3337574325 wait complete. 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.510+0000 7ff0953ff700 1 Processor -- start 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.510+0000 7ff0953ff700 1 -- start start 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff0953ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0900cb360 0x7ff090169450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff0953ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff090157050 0x7ff0900c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff0953ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090157420 0x7ff090154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff0953ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900cbd40 con 0x7ff090157420 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff0953ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090169990 con 0x7ff090157050 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff0953ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900c81b0 con 0x7ff0900cb360 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff08f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090157420 0x7ff090154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff08f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090157420 0x7ff090154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55246/0 (socket says 172.21.15.131:55246) 2023-12-26T23:12:32.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff08f7fe700 1 -- 172.21.15.131:0/1364601999 learned_addr learned my addr 172.21.15.131:0/1364601999 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff08e7fc700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff090157050 0x7ff0900c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff08effd700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0900cb360 0x7ff090169450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff08f7fe700 1 -- 172.21.15.131:0/1364601999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0900cb360 msgr2=0x7ff090169450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff08f7fe700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0900cb360 0x7ff090169450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff08f7fe700 1 -- 172.21.15.131:0/1364601999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff090157050 msgr2=0x7ff0900c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.511+0000 7ff08f7fe700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff090157050 0x7ff0900c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.512+0000 7ff08f7fe700 1 -- 172.21.15.131:0/1364601999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff088009a20 con 0x7ff090157420 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.512+0000 7ff08f7fe700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090157420 0x7ff090154e60 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff0900ce170 tx=0x7ff088017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.512+0000 7ff076ffd700 1 -- 172.21.15.131:0/1364601999 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088012b50 con 0x7ff090157420 2023-12-26T23:12:32.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.512+0000 7ff076ffd700 1 -- 172.21.15.131:0/1364601999 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff088012cb0 con 0x7ff090157420 2023-12-26T23:12:32.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.512+0000 7ff076ffd700 1 -- 172.21.15.131:0/1364601999 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08801e660 con 0x7ff090157420 2023-12-26T23:12:32.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.512+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0900b3f10 con 0x7ff090157420 2023-12-26T23:12:32.516 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.512+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0900c7ed0 con 0x7ff090157420 2023-12-26T23:12:32.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.513+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0900657e0 con 0x7ff090157420 2023-12-26T23:12:32.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.514+0000 7ff076ffd700 1 -- 172.21.15.131:0/1364601999 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff08801e9d0 con 0x7ff090157420 2023-12-26T23:12:32.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.515+0000 7ff076ffd700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff07c066f30 0x7ff07c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:32.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.515+0000 7ff076ffd700 1 -- 172.21.15.131:0/1364601999 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7ff0880868b0 con 0x7ff090157420 2023-12-26T23:12:32.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.515+0000 7ff08effd700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff07c066f30 0x7ff07c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:32.517 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.516+0000 7ff08effd700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff07c066f30 0x7ff07c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff09016c910 tx=0x7ff080000bc0 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:32.519 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.519+0000 7ff076ffd700 1 -- 172.21.15.131:0/1364601999 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff088053870 con 0x7ff090157420 2023-12-26T23:12:32.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.789+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff09015a9a0 con 0x7ff090157420 2023-12-26T23:12:32.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.794+0000 7ff076ffd700 1 -- 172.21.15.131:0/1364601999 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v51) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff088056e90 con 0x7ff090157420 2023-12-26T23:12:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff07c066f30 msgr2=0x7ff07c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff07c066f30 0x7ff07c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff09016c910 tx=0x7ff080000bc0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090157420 msgr2=0x7ff090154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090157420 0x7ff090154e60 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff0900ce170 tx=0x7ff088017000 comp rx=0 tx=0).stop 2023-12-26T23:12:32.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 shutdown_connections 2023-12-26T23:12:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff07c066f30 0x7ff07c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff090157420 0x7ff090154e60 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0900cb360 0x7ff090169450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 --2- 172.21.15.131:0/1364601999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff090157050 0x7ff0900c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 >> 172.21.15.131:0/1364601999 conn(0x7ff09006dc80 msgr2=0x7ff0900cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 shutdown_connections 2023-12-26T23:12:32.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:32.796+0000 7ff0953ff700 1 -- 172.21.15.131:0/1364601999 wait complete. 2023-12-26T23:12:32.799 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2023-12-26T23:12:32.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:33.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.005+0000 7f3201fff700 1 Processor -- start 2023-12-26T23:12:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.005+0000 7f3201fff700 1 -- start start 2023-12-26T23:12:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.006+0000 7f3201fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31fc1596a0 0x7f31fc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.006+0000 7f3201fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc0afcd0 con 0x7f31fc0b0870 2023-12-26T23:12:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f3201fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc0b3e70 con 0x7f31fc159a70 2023-12-26T23:12:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f3201fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc0b0cb0 con 0x7f31fc1596a0 2023-12-26T23:12:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31fbfff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f31fc0b0870 0x7f31fc0b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32880/0 (socket says 172.21.15.131:32880) 2023-12-26T23:12:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31fbfff700 1 -- 172.21.15.131:0/4064675299 learned_addr learned my addr 172.21.15.131:0/4064675299 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31fb7fe700 1 --2- 172.21.15.131:0/4064675299 >> v2:172.21.15.190:3300/0 conn(0x7f31fc1596a0 0x7f31fc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31faffd700 1 -- 172.21.15.131:0/4064675299 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83400632 0 0) 0x7f31fc0afcd0 con 0x7f31fc0b0870 2023-12-26T23:12:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31faffd700 1 -- 172.21.15.131:0/4064675299 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31e8003570 con 0x7f31fc0b0870 2023-12-26T23:12:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31faffd700 1 -- 172.21.15.131:0/4064675299 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732452266 0 0) 0x7f31fc0b3e70 con 0x7f31fc159a70 2023-12-26T23:12:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31faffd700 1 -- 172.21.15.131:0/4064675299 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31fc0afcd0 con 0x7f31fc159a70 2023-12-26T23:12:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31fb7fe700 1 -- 172.21.15.131:0/4064675299 >> v1:172.21.15.190:6789/0 conn(0x7f31fc159a70 legacy=0x7f31fc0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31fb7fe700 1 -- 172.21.15.131:0/4064675299 >> v1:172.21.15.131:6789/0 conn(0x7f31fc0b0870 legacy=0x7f31fc0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31faffd700 1 -- 172.21.15.131:0/4064675299 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 837987094 0 0) 0x7f31e8003570 con 0x7f31fc0b0870 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.007+0000 7f31fb7fe700 1 -- 172.21.15.131:0/4064675299 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0b0f20 con 0x7f31fc1596a0 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.008+0000 7f31fb7fe700 1 --2- 172.21.15.131:0/4064675299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc1596a0 0x7f31fc0b2380 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0098e0 tx=0x7f31ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ed74e0bb76b75c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.008+0000 7f31faffd700 1 -- 172.21.15.131:0/4064675299 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f40017f0 con 0x7f31fc1596a0 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.008+0000 7f31faffd700 1 -- 172.21.15.131:0/4064675299 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec011420 con 0x7f31fc1596a0 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.008+0000 7f31faffd700 1 -- 172.21.15.131:0/4064675299 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0117c0 con 0x7f31fc1596a0 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.009+0000 7f3201fff700 1 -- 172.21.15.131:0/4064675299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc1596a0 msgr2=0x7f31fc0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.009+0000 7f3201fff700 1 --2- 172.21.15.131:0/4064675299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc1596a0 0x7f31fc0b2380 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0098e0 tx=0x7f31ec0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.009+0000 7f3201fff700 1 -- 172.21.15.131:0/4064675299 shutdown_connections 2023-12-26T23:12:33.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.009+0000 7f3201fff700 1 --2- 172.21.15.131:0/4064675299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc1596a0 0x7f31fc0b2380 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.009+0000 7f3201fff700 1 -- 172.21.15.131:0/4064675299 >> 172.21.15.131:0/4064675299 conn(0x7f31fc06ece0 msgr2=0x7f31fc06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:33.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.009+0000 7f3201fff700 1 -- 172.21.15.131:0/4064675299 shutdown_connections 2023-12-26T23:12:33.012 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.009+0000 7f3201fff700 1 -- 172.21.15.131:0/4064675299 wait complete. 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.010+0000 7f3201fff700 1 Processor -- start 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.010+0000 7f3201fff700 1 -- start start 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.011+0000 7f3201fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0b0870 0x7f31fc0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.011+0000 7f3201fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc1596a0 0x7f31fc0c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.011+0000 7f3201fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc159a70 0x7f31fc156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.011+0000 7f3201fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0c43a0 con 0x7f31fc159a70 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.011+0000 7f3201fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc15a790 con 0x7f31fc1596a0 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.011+0000 7f3201fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0c85e0 con 0x7f31fc0b0870 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.011+0000 7f31fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc159a70 0x7f31fc156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.011+0000 7f31fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc159a70 0x7f31fc156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55260/0 (socket says 172.21.15.131:55260) 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.011+0000 7f31fbfff700 1 -- 172.21.15.131:0/1961557198 learned_addr learned my addr 172.21.15.131:0/1961557198 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31fb7fe700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0b0870 0x7f31fc0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31f3fff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc1596a0 0x7f31fc0c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31fbfff700 1 -- 172.21.15.131:0/1961557198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0b0870 msgr2=0x7f31fc0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31fbfff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0b0870 0x7f31fc0c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31fbfff700 1 -- 172.21.15.131:0/1961557198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc1596a0 msgr2=0x7f31fc0c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31fbfff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc1596a0 0x7f31fc0c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31fbfff700 1 -- 172.21.15.131:0/1961557198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec009420 con 0x7f31fc159a70 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31fbfff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc159a70 0x7f31fc156d70 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f31f400cce0 tx=0x7f31f4007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31f37fe700 1 -- 172.21.15.131:0/1961557198 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f400ce90 con 0x7f31fc159a70 2023-12-26T23:12:33.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31f37fe700 1 -- 172.21.15.131:0/1961557198 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f4004dc0 con 0x7f31fc159a70 2023-12-26T23:12:33.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc1572b0 con 0x7f31fc159a70 2023-12-26T23:12:33.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f31f37fe700 1 -- 172.21.15.131:0/1961557198 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f400aad0 con 0x7f31fc159a70 2023-12-26T23:12:33.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.012+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc159ee0 con 0x7f31fc159a70 2023-12-26T23:12:33.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.013+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31fc0657e0 con 0x7f31fc159a70 2023-12-26T23:12:33.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.014+0000 7f31f37fe700 1 -- 172.21.15.131:0/1961557198 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f31f4007150 con 0x7f31fc159a70 2023-12-26T23:12:33.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.015+0000 7f31f37fe700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f31dc066f30 0x7f31dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.015+0000 7f31f37fe700 1 -- 172.21.15.131:0/1961557198 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f31f4084220 con 0x7f31fc159a70 2023-12-26T23:12:33.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.015+0000 7f31fb7fe700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f31dc066f30 0x7f31dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.018 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.016+0000 7f31fb7fe700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f31dc066f30 0x7f31dc0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0093f0 tx=0x7f31ec016000 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.019 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.019+0000 7f31f37fe700 1 -- 172.21.15.131:0/1961557198 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31f4051300 con 0x7f31fc159a70 2023-12-26T23:12:33.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.256+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31fc15a8d0 con 0x7f31fc159a70 2023-12-26T23:12:33.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.257+0000 7f31f37fe700 1 -- 172.21.15.131:0/1961557198 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4617 (secure 0 0 0) 0x7f31f4054920 con 0x7f31fc159a70 2023-12-26T23:12:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:33.258 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8372,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2599990460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2599990460},{"type":"v1","addr":"172.21.15.190:6835","nonce":2599990460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":9565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3465421941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3465421941},{"type":"v1","addr":"172.21.15.131:6834","nonce":3465421941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":10122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3595865527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3595865527},{"type":"v1","addr":"172.21.15.131:6835","nonce":3595865527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:12:22.064606+0000","modified":"2023-12-26T23:12:25.606067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10125":{"gid":10125,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/1917956201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1917956201},{"type":"v1","addr":"172.21.15.190:6834","nonce":1917956201}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-12-26T23:12:33.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f31dc066f30 msgr2=0x7f31dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f31dc066f30 0x7f31dc0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0093f0 tx=0x7f31ec016000 comp rx=0 tx=0).stop 2023-12-26T23:12:33.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc159a70 msgr2=0x7f31fc156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc159a70 0x7f31fc156d70 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f31f400cce0 tx=0x7f31f4007d20 comp rx=0 tx=0).stop 2023-12-26T23:12:33.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 shutdown_connections 2023-12-26T23:12:33.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f31dc066f30 0x7f31dc0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc159a70 0x7f31fc156d70 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31fc0b0870 0x7f31fc0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 --2- 172.21.15.131:0/1961557198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31fc1596a0 0x7f31fc0c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 >> 172.21.15.131:0/1961557198 conn(0x7f31fc06ece0 msgr2=0x7f31fc158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:33.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 shutdown_connections 2023-12-26T23:12:33.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.260+0000 7f3201fff700 1 -- 172.21.15.131:0/1961557198 wait complete. 2023-12-26T23:12:33.262 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 98 2023-12-26T23:12:33.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-26T23:12:22.064606+0000', 'modified': '2023-12-26T23:12:25.606067+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10125}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10125': {'gid': 10125, 'name': 'd', 'rank': 0, 'incarnation': 94, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6834/1917956201', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 1917956201}, {'type': 'v1', 'addr': '172.21.15.190:6834', 'nonce': 1917956201}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-12-26T23:12:33.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-26T23:12:33.682 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-26T23:12:33.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.872+0000 7ff801fff700 1 Processor -- start 2023-12-26T23:12:33.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.872+0000 7ff801fff700 1 -- start start 2023-12-26T23:12:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.873+0000 7ff801fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 0x7ff7fc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.873+0000 7ff801fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7fc0cc100 0x7ff7fc0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.873+0000 7ff801fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7fc0686c0 con 0x7ff7fc13c510 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.873+0000 7ff801fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7fc069520 con 0x7ff7fc0cc100 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.873+0000 7ff801fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc068910 con 0x7ff7fc06bfd0 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7fb7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff7fc13c510 0x7ff7fc06b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32884/0 (socket says 172.21.15.131:32884) 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7fb7fe700 1 -- 172.21.15.131:0/2227228582 learned_addr learned my addr 172.21.15.131:0/2227228582 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7faffd700 1 --2- 172.21.15.131:0/2227228582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 0x7ff7fc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7fbfff700 1 --2- 172.21.15.131:0/2227228582 >> v2:172.21.15.190:3300/0 conn(0x7ff7fc0cc100 0x7ff7fc0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7fa7fc700 1 -- 172.21.15.131:0/2227228582 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687131026 0 0) 0x7ff7fc0686c0 con 0x7ff7fc13c510 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7fa7fc700 1 -- 172.21.15.131:0/2227228582 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e4003570 con 0x7ff7fc13c510 2023-12-26T23:12:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7fa7fc700 1 -- 172.21.15.131:0/2227228582 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 240192049 0 0) 0x7ff7e4003570 con 0x7ff7fc13c510 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7faffd700 1 -- 172.21.15.131:0/2227228582 >> v2:172.21.15.190:3300/0 conn(0x7ff7fc0cc100 msgr2=0x7ff7fc0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7faffd700 1 --2- 172.21.15.131:0/2227228582 >> v2:172.21.15.190:3300/0 conn(0x7ff7fc0cc100 0x7ff7fc0cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7faffd700 1 -- 172.21.15.131:0/2227228582 >> v1:172.21.15.131:6789/0 conn(0x7ff7fc13c510 legacy=0x7ff7fc06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7faffd700 1 -- 172.21.15.131:0/2227228582 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc068bf0 con 0x7ff7fc06bfd0 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.874+0000 7ff7fbfff700 1 --2- 172.21.15.131:0/2227228582 >> v2:172.21.15.190:3300/0 conn(0x7ff7fc0cc100 0x7ff7fc0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.875+0000 7ff7faffd700 1 --2- 172.21.15.131:0/2227228582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 0x7ff7fc0c8a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00098e0 tx=0x7ff7f00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d15cb4bef55fda3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.875+0000 7ff7fa7fc700 1 -- 172.21.15.131:0/2227228582 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0015f0 con 0x7ff7fc06bfd0 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.875+0000 7ff7fa7fc700 1 -- 172.21.15.131:0/2227228582 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f0011420 con 0x7ff7fc06bfd0 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.875+0000 7ff7fa7fc700 1 -- 172.21.15.131:0/2227228582 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00117a0 con 0x7ff7fc06bfd0 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.875+0000 7ff801fff700 1 -- 172.21.15.131:0/2227228582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 msgr2=0x7ff7fc0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.878 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.875+0000 7ff801fff700 1 --2- 172.21.15.131:0/2227228582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 0x7ff7fc0c8a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00098e0 tx=0x7ff7f00091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.875+0000 7ff801fff700 1 -- 172.21.15.131:0/2227228582 shutdown_connections 2023-12-26T23:12:33.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.875+0000 7ff801fff700 1 --2- 172.21.15.131:0/2227228582 >> v2:172.21.15.190:3300/0 conn(0x7ff7fc0cc100 0x7ff7fc0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.876+0000 7ff801fff700 1 --2- 172.21.15.131:0/2227228582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 0x7ff7fc0c8a20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.876+0000 7ff801fff700 1 -- 172.21.15.131:0/2227228582 >> 172.21.15.131:0/2227228582 conn(0x7ff7fc0ce3e0 msgr2=0x7ff7fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:33.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.876+0000 7ff801fff700 1 -- 172.21.15.131:0/2227228582 shutdown_connections 2023-12-26T23:12:33.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.876+0000 7ff801fff700 1 -- 172.21.15.131:0/2227228582 wait complete. 2023-12-26T23:12:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.877+0000 7ff801fff700 1 Processor -- start 2023-12-26T23:12:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.877+0000 7ff801fff700 1 -- start start 2023-12-26T23:12:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff801fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 0x7ff7fc0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff801fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc0cc100 0x7ff7fc0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff801fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7fc13c510 0x7ff7fc0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff801fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0b3ff0 con 0x7ff7fc0cc100 2023-12-26T23:12:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff801fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc154ff0 con 0x7ff7fc13c510 2023-12-26T23:12:33.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff801fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0b21b0 con 0x7ff7fc06bfd0 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff7faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc0cc100 0x7ff7fc0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff7faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc0cc100 0x7ff7fc0b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55268/0 (socket says 172.21.15.131:55268) 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff7faffd700 1 -- 172.21.15.131:0/1606411748 learned_addr learned my addr 172.21.15.131:0/1606411748 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.878+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 0x7ff7fc0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7fbfff700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7fc13c510 0x7ff7fc0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7faffd700 1 -- 172.21.15.131:0/1606411748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 msgr2=0x7ff7fc0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7faffd700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 0x7ff7fc0b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7faffd700 1 -- 172.21.15.131:0/1606411748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7fc13c510 msgr2=0x7ff7fc0c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7faffd700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7fc13c510 0x7ff7fc0c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7faffd700 1 -- 172.21.15.131:0/1606411748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4009a20 con 0x7ff7fc0cc100 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7faffd700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc0cc100 0x7ff7fc0b3ab0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc068a50 tx=0x7ff7f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.883 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7e37fe700 1 -- 172.21.15.131:0/1606411748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f001cba0 con 0x7ff7fc0cc100 2023-12-26T23:12:33.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7e37fe700 1 -- 172.21.15.131:0/1606411748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f001cd00 con 0x7ff7fc0cc100 2023-12-26T23:12:33.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff7e37fe700 1 -- 172.21.15.131:0/1606411748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f001a660 con 0x7ff7fc0cc100 2023-12-26T23:12:33.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.879+0000 7ff801fff700 1 -- 172.21.15.131:0/1606411748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f0009420 con 0x7ff7fc0cc100 2023-12-26T23:12:33.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.880+0000 7ff801fff700 1 -- 172.21.15.131:0/1606411748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc0b2770 con 0x7ff7fc0cc100 2023-12-26T23:12:33.884 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.881+0000 7ff7e37fe700 1 -- 172.21.15.131:0/1606411748 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff7f001aa50 con 0x7ff7fc0cc100 2023-12-26T23:12:33.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.882+0000 7ff7e37fe700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff7e8066f30 0x7ff7e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.882+0000 7ff7e37fe700 1 -- 172.21.15.131:0/1606411748 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7ff7f0085f10 con 0x7ff7fc0cc100 2023-12-26T23:12:33.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.882+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff7e8066f30 0x7ff7e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.883+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff7e8066f30 0x7ff7e80693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec000ef0 tx=0x7ff7ec001700 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.884+0000 7ff8107e4b80 1 Processor -- start 2023-12-26T23:12:33.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.884+0000 7ff8107e4b80 1 -- start start 2023-12-26T23:12:33.888 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.885+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac5c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.885+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 0x55de2ac57c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.885+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac78360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.885+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2ab8a3e0 con 0x55de2ac5bfa0 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.885+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2ab87be0 con 0x55de2ac58170 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.885+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de2ab839d0 con 0x55de2ac5c8b0 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.885+0000 7ff7fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac5c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.885+0000 7ff7fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac5c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55272/0 (socket says 172.21.15.131:55272) 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.885+0000 7ff7fb7fe700 1 -- 172.21.15.131:0/678911193 learned_addr learned my addr 172.21.15.131:0/678911193 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7faffd700 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 0x55de2ac57c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7fb7fe700 1 -- 172.21.15.131:0/678911193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 msgr2=0x55de2ac57c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7fbfff700 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac78360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 0x55de2ac57c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7fb7fe700 1 -- 172.21.15.131:0/678911193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 msgr2=0x55de2ac78360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac78360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7fb7fe700 1 -- 172.21.15.131:0/678911193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de2aa17360 con 0x55de2ac5bfa0 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7faffd700 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 0x55de2ac57c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac5c370 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00fe00 tx=0x7ff7ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e24b5b7bf9b78d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7e0ff9700 1 -- 172.21.15.131:0/678911193 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec014a70 con 0x55de2ac5bfa0 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7e0ff9700 1 -- 172.21.15.131:0/678911193 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec014bd0 con 0x55de2ac5bfa0 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.886+0000 7ff7e0ff9700 1 -- 172.21.15.131:0/678911193 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec020410 con 0x55de2ac5bfa0 2023-12-26T23:12:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.887+0000 7ff8107e4b80 1 -- 172.21.15.131:0/678911193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 msgr2=0x55de2ac5c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.887+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac5c370 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00fe00 tx=0x7ff7ec015000 comp rx=0 tx=0).stop 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.887+0000 7ff8107e4b80 1 -- 172.21.15.131:0/678911193 shutdown_connections 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.887+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac78360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.887+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac5c370 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.887+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/678911193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 0x55de2ac57c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.887+0000 7ff8107e4b80 1 -- 172.21.15.131:0/678911193 >> 172.21.15.131:0/678911193 conn(0x55de2ac32440 msgr2=0x55de2ac7ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.887+0000 7ff8107e4b80 1 -- 172.21.15.131:0/678911193 shutdown_connections 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.887+0000 7ff8107e4b80 1 -- 172.21.15.131:0/678911193 wait complete. 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.888+0000 7ff8107e4b80 1 -- start start 2023-12-26T23:12:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.888+0000 7ff8107e4b80 1 Processor -- start 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac75180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 0x55de2ac72aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5c8b0 0x55de2ac72fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2ab301b0 con 0x55de2ac5c8b0 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2ab2fe50 con 0x55de2ac58170 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de2aa7ce40 con 0x55de2ac5bfa0 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff8107e4b80 10 client.0 _ll_register_callbacks cb 0x55de2aa7f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff7fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5c8b0 0x55de2ac72fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff7fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5c8b0 0x55de2ac72fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55284/0 (socket says 172.21.15.131:55284) 2023-12-26T23:12:33.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff7fbfff700 1 -- 172.21.15.131:0/4236929551 learned_addr learned my addr 172.21.15.131:0/4236929551 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff7faffd700 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 0x55de2ac72aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.889+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac75180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7fbfff700 1 -- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 msgr2=0x55de2ac72aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7fbfff700 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 0x55de2ac72aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7fbfff700 1 -- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 msgr2=0x55de2ac75180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7fbfff700 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac75180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7fbfff700 1 -- 172.21.15.131:0/4236929551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec00f9b0 con 0x55de2ac5c8b0 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7fbfff700 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5c8b0 0x55de2ac72fe0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4007f50 tx=0x7ff7f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48125d8c3b2efefe server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/4236929551 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f400e040 con 0x55de2ac5c8b0 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/4236929551 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7ff7f4012cd0 con 0x55de2ac5c8b0 2023-12-26T23:12:33.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff8107e4b80 1 -- 172.21.15.131:0/4236929551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de2ac04eb0 con 0x55de2ac5c8b0 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/4236929551 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f4012e90 con 0x55de2ac5c8b0 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff8107e4b80 10 client.10272 fetch_fsmap learned FSMap version 98 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.890+0000 7ff8107e4b80 1 -- 172.21.15.131:0/4236929551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x55de2a9c3b10 con 0x55de2ac5c8b0 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.891+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/4236929551 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55de2ac04eb0 con 0x55de2ac5c8b0 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.891+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/4236929551 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 98) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff7f4014040 con 0x55de2ac5c8b0 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.891+0000 7ff8107e4b80 10 client.10272 fetch_fsmap finished waiting for FSMap version 98 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.891+0000 7ff8107e4b80 10 client.10272 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.891+0000 7ff8107e4b80 20 client.10272 populate_metadata read hostname 'smithi131' 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.891+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] conn(0x55de2ac5ae40 0x55de2ac5b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.891+0000 7ff8107e4b80 4 client.10272 mds_command: new command op to 10125 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.891+0000 7ff8107e4b80 1 -- 172.21.15.131:0/4236929551 --> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de2ab5a880 con 0x55de2ac5ae40 2023-12-26T23:12:33.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.891+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] conn(0x55de2ac5ae40 0x55de2ac5b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.892+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] conn(0x55de2ac5ae40 0x55de2ac5b210 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a3699f884c9656 server_cookie=42c037de28280b3d in_seq=0 out_seq=0 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.892+0000 7ff7d9ffb700 10 client.10272 ms_handle_connect on v2:172.21.15.190:6832/1917956201 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.892+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/4236929551 <== mds.0 v2:172.21.15.190:6832/1917956201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55de2ab5a880 con 0x55de2ac5ae40 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.892+0000 7ff7d9ffb700 10 client.10272 handle_command_reply: tid=0 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.892+0000 7ff8107e4b80 1 client.10272 shutdown 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 -- 172.21.15.131:0/4236929551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5c8b0 msgr2=0x55de2ac72fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5c8b0 0x55de2ac72fe0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4007f50 tx=0x7ff7f400b040 comp rx=0 tx=0).stop 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 -- 172.21.15.131:0/4236929551 shutdown_connections 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5c8b0 0x55de2ac72fe0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] conn(0x55de2ac5ae40 0x55de2ac5b210 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac75180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/4236929551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 0x55de2ac72aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 -- 172.21.15.131:0/4236929551 >> 172.21.15.131:0/4236929551 conn(0x55de2ac32440 msgr2=0x55de2ac32810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:33.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff7d9ffb700 0 client.10272 ms_handle_reset on v2:172.21.15.190:6832/1917956201 2023-12-26T23:12:33.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 -- 172.21.15.131:0/4236929551 shutdown_connections 2023-12-26T23:12:33.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 1 -- 172.21.15.131:0/4236929551 wait complete. 2023-12-26T23:12:33.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.893+0000 7ff8107e4b80 20 client.10272 trim_cache size 0 max 16384 2023-12-26T23:12:33.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.910+0000 7ff8107e4b80 1 Processor -- start 2023-12-26T23:12:33.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.910+0000 7ff8107e4b80 1 -- start start 2023-12-26T23:12:33.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.911+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.911+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.911+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 0x55de2ac63830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.911+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2ac2a0b0 con 0x55de2ac5bfa0 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.911+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2ac32810 con 0x55de2ac58170 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.911+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de2ac5a0e0 con 0x55de2ac5c8b0 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.911+0000 7ff7faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.911+0000 7ff7faffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55290/0 (socket says 172.21.15.131:55290) 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.911+0000 7ff7faffd700 1 -- 172.21.15.131:0/406634459 learned_addr learned my addr 172.21.15.131:0/406634459 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.912+0000 7ff7faffd700 1 -- 172.21.15.131:0/406634459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 msgr2=0x55de2ac63830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.912+0000 7ff7faffd700 1 --2- 172.21.15.131:0/406634459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 0x55de2ac63830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.912+0000 7ff7faffd700 1 -- 172.21.15.131:0/406634459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 msgr2=0x55de2ac589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.912+0000 7ff7faffd700 1 --2- 172.21.15.131:0/406634459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.912+0000 7ff7faffd700 1 -- 172.21.15.131:0/406634459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de2ab52290 con 0x55de2ac5bfa0 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.912+0000 7ff7faffd700 1 --2- 172.21.15.131:0/406634459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac632f0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0056c90 tx=0x7ff7f0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bb7201cb87b77c server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.912+0000 7ff7e17fa700 1 -- 172.21.15.131:0/406634459 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f005dbf0 con 0x55de2ac5bfa0 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.912+0000 7ff7e17fa700 1 -- 172.21.15.131:0/406634459 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f0054e40 con 0x55de2ac5bfa0 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.912+0000 7ff7e17fa700 1 -- 172.21.15.131:0/406634459 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0053410 con 0x55de2ac5bfa0 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 -- 172.21.15.131:0/406634459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 msgr2=0x55de2ac632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/406634459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac632f0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0056c90 tx=0x7ff7f0013040 comp rx=0 tx=0).stop 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 -- 172.21.15.131:0/406634459 shutdown_connections 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/406634459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5c8b0 0x55de2ac63830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/406634459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac58170 0x55de2ac589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/406634459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac5bfa0 0x55de2ac632f0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 -- 172.21.15.131:0/406634459 >> 172.21.15.131:0/406634459 conn(0x55de2ac32440 msgr2=0x55de2ac30790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 -- 172.21.15.131:0/406634459 shutdown_connections 2023-12-26T23:12:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 -- 172.21.15.131:0/406634459 wait complete. 2023-12-26T23:12:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.913+0000 7ff8107e4b80 1 -- start start 2023-12-26T23:12:33.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.914+0000 7ff8107e4b80 1 Processor -- start 2023-12-26T23:12:33.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac58170 0x55de2ac636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 0x55de2acdc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.920 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff8107e4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac5c8b0 0x55de2acdccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2ac79e00 con 0x55de2ac58170 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2ace2e50 con 0x55de2ac5c8b0 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff8107e4b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de2ac8f2c0 con 0x55de2ac5bfa0 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff8107e4b80 10 client.0 _ll_register_callbacks cb 0x7ff7f40078a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac58170 0x55de2ac636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac58170 0x55de2ac636d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55298/0 (socket says 172.21.15.131:55298) 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fb7fe700 1 -- 172.21.15.131:0/3452831867 learned_addr learned my addr 172.21.15.131:0/3452831867 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fbfff700 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac5c8b0 0x55de2acdccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7faffd700 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 0x55de2acdc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fb7fe700 1 -- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 msgr2=0x55de2acdc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:33.921 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 0x55de2acdc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fb7fe700 1 -- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac5c8b0 msgr2=0x55de2acdccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac5c8b0 0x55de2acdccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fb7fe700 1 -- 172.21.15.131:0/3452831867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc0b2770 con 0x55de2ac58170 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7fb7fe700 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac58170 0x55de2ac636d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec023040 tx=0x7ff7ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741c255ef74429ce server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.915+0000 7ff7d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/3452831867 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec032070 con 0x55de2ac58170 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/3452831867 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7ff7ec01c040 con 0x55de2ac58170 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff8107e4b80 1 -- 172.21.15.131:0/3452831867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7fc0c6030 con 0x55de2ac58170 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/3452831867 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ec014e90 con 0x55de2ac58170 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff8107e4b80 10 client.10278 fetch_fsmap learned FSMap version 98 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff8107e4b80 1 -- 172.21.15.131:0/3452831867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x55de2aa1bc80 con 0x55de2ac58170 2023-12-26T23:12:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/3452831867 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc0c6030 con 0x55de2ac58170 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/3452831867 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 98) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff7ec012070 con 0x55de2ac58170 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff8107e4b80 10 client.10278 fetch_fsmap finished waiting for FSMap version 98 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff8107e4b80 10 client.10278 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff8107e4b80 20 client.10278 populate_metadata read hostname 'smithi131' 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] conn(0x55de2acdf870 0x55de2acdfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff8107e4b80 4 client.10278 mds_command: new command op to 10125 tid=0[{"prefix": "status"}] 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.916+0000 7ff8107e4b80 1 -- 172.21.15.131:0/3452831867 --> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55de2ac28490 con 0x55de2acdf870 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.917+0000 7ff7faffd700 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] conn(0x55de2acdf870 0x55de2acdfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.917+0000 7ff7faffd700 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] conn(0x55de2acdf870 0x55de2acdfc40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8b00a1a181d63b server_cookie=9e09cfb6b4231d8d in_seq=0 out_seq=0 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.917+0000 7ff7d9ffb700 10 client.10278 ms_handle_connect on v2:172.21.15.190:6832/1917956201 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff7d9ffb700 1 -- 172.21.15.131:0/3452831867 <== mds.0 v2:172.21.15.190:6832/1917956201 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55de2ac28490 con 0x55de2acdf870 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff7d9ffb700 10 client.10278 handle_command_reply: tid=0 2023-12-26T23:12:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 client.10278 shutdown 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 -- 172.21.15.131:0/3452831867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac58170 msgr2=0x55de2ac636d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac58170 0x55de2ac636d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec023040 tx=0x7ff7ec019000 comp rx=0 tx=0).stop 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 -- 172.21.15.131:0/3452831867 shutdown_connections 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:6832/1917956201,v1:172.21.15.190:6834/1917956201] conn(0x55de2acdf870 0x55de2acdfc40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de2ac5c8b0 0x55de2acdccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55de2ac58170 0x55de2ac636d0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff7d9ffb700 0 client.10278 ms_handle_reset on v2:172.21.15.190:6832/1917956201 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 --2- 172.21.15.131:0/3452831867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de2ac5bfa0 0x55de2acdc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 -- 172.21.15.131:0/3452831867 >> 172.21.15.131:0/3452831867 conn(0x55de2ac32440 msgr2=0x55de2ac589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 -- 172.21.15.131:0/3452831867 shutdown_connections 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 1 -- 172.21.15.131:0/3452831867 wait complete. 2023-12-26T23:12:33.924 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.918+0000 7ff8107e4b80 20 client.10278 trim_cache size 0 max 16384 2023-12-26T23:12:33.925 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:12:33.925 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "ab603302-d88b-429a-be64-92b3c298adad", 2023-12-26T23:12:33.925 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2023-12-26T23:12:33.925 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 10125, 2023-12-26T23:12:33.925 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2023-12-26T23:12:33.925 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2023-12-26T23:12:33.925 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2023-12-26T23:12:33.925 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 11.325611289999999, 2023-12-26T23:12:33.925 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 98, 2023-12-26T23:12:33.926 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 210, 2023-12-26T23:12:33.926 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 210, 2023-12-26T23:12:33.926 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 17.043947381999999 2023-12-26T23:12:33.926 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 -- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff7e8066f30 msgr2=0x7ff7e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff7e8066f30 0x7ff7e80693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec000ef0 tx=0x7ff7ec001700 comp rx=0 tx=0).stop 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 -- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc0cc100 msgr2=0x7ff7fc0b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc0cc100 0x7ff7fc0b3ab0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc068a50 tx=0x7ff7f0016000 comp rx=0 tx=0).stop 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 -- 172.21.15.131:0/1606411748 shutdown_connections 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff7e8066f30 0x7ff7e80693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7fc13c510 0x7ff7fc0c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7fc06bfd0 0x7ff7fc0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 --2- 172.21.15.131:0/1606411748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7fc0cc100 0x7ff7fc0b3ab0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 -- 172.21.15.131:0/1606411748 >> 172.21.15.131:0/1606411748 conn(0x7ff7fc0ce3e0 msgr2=0x7ff7fc0cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:33.927 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 -- 172.21.15.131:0/1606411748 shutdown_connections 2023-12-26T23:12:33.928 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:33.919+0000 7ff801fff700 1 -- 172.21.15.131:0/1606411748 wait complete. 2023-12-26T23:12:33.936 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-26T23:12:34.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.126+0000 7f182b3ff700 1 Processor -- start 2023-12-26T23:12:34.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.126+0000 7f182b3ff700 1 -- start start 2023-12-26T23:12:34.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.127+0000 7f182b3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1824159a70 0x7f18240b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:34.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.127+0000 7f182b3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18240afcd0 con 0x7f18240b2a90 2023-12-26T23:12:34.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.127+0000 7f182b3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18240b3e70 con 0x7f18241596a0 2023-12-26T23:12:34.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.127+0000 7f182b3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240b0f20 con 0x7f1824159a70 2023-12-26T23:12:34.128 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.128+0000 7f182899a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1824159a70 0x7f18240b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:34.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.128+0000 7f182999c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f18240b2a90 0x7f18240b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:32886/0 (socket says 172.21.15.131:32886) 2023-12-26T23:12:34.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.128+0000 7f182999c700 1 -- 172.21.15.131:0/2703089208 learned_addr learned my addr 172.21.15.131:0/2703089208 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:34.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.128+0000 7f182899a700 1 -- 172.21.15.131:0/2703089208 >> v1:172.21.15.190:6789/0 conn(0x7f18241596a0 legacy=0x7f18240b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:34.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.128+0000 7f182899a700 1 -- 172.21.15.131:0/2703089208 >> v1:172.21.15.131:6789/0 conn(0x7f18240b2a90 legacy=0x7f18240b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:34.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.128+0000 7f182899a700 1 -- 172.21.15.131:0/2703089208 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240b3540 con 0x7f1824159a70 2023-12-26T23:12:34.129 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.128+0000 7f182899a700 1 --2- 172.21.15.131:0/2703089208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1824159a70 0x7f18240b0160 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f18140098e0 tx=0x7f18140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdfa2931ed9fb93 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:34.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f181bfff700 1 -- 172.21.15.131:0/2703089208 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814009e80 con 0x7f1824159a70 2023-12-26T23:12:34.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f181bfff700 1 -- 172.21.15.131:0/2703089208 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181400b860 con 0x7f1824159a70 2023-12-26T23:12:34.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f181bfff700 1 -- 172.21.15.131:0/2703089208 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400bbe0 con 0x7f1824159a70 2023-12-26T23:12:34.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f182b3ff700 1 -- 172.21.15.131:0/2703089208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1824159a70 msgr2=0x7f18240b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:34.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f182b3ff700 1 --2- 172.21.15.131:0/2703089208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1824159a70 0x7f18240b0160 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f18140098e0 tx=0x7f18140091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f182b3ff700 1 -- 172.21.15.131:0/2703089208 shutdown_connections 2023-12-26T23:12:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f182b3ff700 1 --2- 172.21.15.131:0/2703089208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1824159a70 0x7f18240b0160 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f182b3ff700 1 -- 172.21.15.131:0/2703089208 >> 172.21.15.131:0/2703089208 conn(0x7f182406ece0 msgr2=0x7f182406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f182b3ff700 1 -- 172.21.15.131:0/2703089208 shutdown_connections 2023-12-26T23:12:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.129+0000 7f182b3ff700 1 -- 172.21.15.131:0/2703089208 wait complete. 2023-12-26T23:12:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.131+0000 7f182b3ff700 1 Processor -- start 2023-12-26T23:12:34.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.131+0000 7f182b3ff700 1 -- start start 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182b3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18240b2a90 0x7f182415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182b3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18241596a0 0x7f18240cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182b3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1824159a70 0x7f18240c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182b3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240ca530 con 0x7f18241596a0 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182b3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c8ba0 con 0x7f18240b2a90 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182b3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c87d0 con 0x7f1824159a70 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182899a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18241596a0 0x7f18240cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182899a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18241596a0 0x7f18240cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55308/0 (socket says 172.21.15.131:55308) 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182899a700 1 -- 172.21.15.131:0/3052747965 learned_addr learned my addr 172.21.15.131:0/3052747965 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182919b700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18240b2a90 0x7f182415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182999c700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1824159a70 0x7f18240c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182899a700 1 -- 172.21.15.131:0/3052747965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1824159a70 msgr2=0x7f18240c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182899a700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1824159a70 0x7f18240c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182899a700 1 -- 172.21.15.131:0/3052747965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18240b2a90 msgr2=0x7f182415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182899a700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18240b2a90 0x7f182415a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182899a700 1 -- 172.21.15.131:0/3052747965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814009420 con 0x7f18241596a0 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.132+0000 7f182919b700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18240b2a90 0x7f182415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.133+0000 7f182899a700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18241596a0 0x7f18240cb860 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1824089710 tx=0x7f1814017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.133+0000 7f18197fa700 1 -- 172.21.15.131:0/3052747965 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814011420 con 0x7f18241596a0 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.133+0000 7f18197fa700 1 -- 172.21.15.131:0/3052747965 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18140115a0 con 0x7f18241596a0 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.133+0000 7f18197fa700 1 -- 172.21.15.131:0/3052747965 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814011960 con 0x7f18241596a0 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.133+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18240c8210 con 0x7f18241596a0 2023-12-26T23:12:34.135 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.133+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1824155660 con 0x7f18241596a0 2023-12-26T23:12:34.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.134+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18240c8d10 con 0x7f18241596a0 2023-12-26T23:12:34.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.135+0000 7f18197fa700 1 -- 172.21.15.131:0/3052747965 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1814011cd0 con 0x7f18241596a0 2023-12-26T23:12:34.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.136+0000 7f18197fa700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1808066f30 0x7f18080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:34.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.136+0000 7f18197fa700 1 -- 172.21.15.131:0/3052747965 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f18140861b0 con 0x7f18241596a0 2023-12-26T23:12:34.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.136+0000 7f182919b700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1808066f30 0x7f18080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:34.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.137+0000 7f182919b700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1808066f30 0x7f18080693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1820007fd0 tx=0x7f1820009040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.140+0000 7f18197fa700 1 -- 172.21.15.131:0/3052747965 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18240c8d10 con 0x7f18241596a0 2023-12-26T23:12:34.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.377+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f18240c8d10 con 0x7f18241596a0 2023-12-26T23:12:34.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.377+0000 7f18197fa700 1 -- 172.21.15.131:0/3052747965 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) v1 ==== 53+0+73 (secure 0 0 0) 0x7f18240c8d10 con 0x7f18241596a0 2023-12-26T23:12:34.378 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-26T23:12:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.379+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1808066f30 msgr2=0x7f18080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.379+0000 7f182b3ff700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1808066f30 0x7f18080693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1820007fd0 tx=0x7f1820009040 comp rx=0 tx=0).stop 2023-12-26T23:12:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.379+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18241596a0 msgr2=0x7f18240cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.379+0000 7f182b3ff700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18241596a0 0x7f18240cb860 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1824089710 tx=0x7f1814017000 comp rx=0 tx=0).stop 2023-12-26T23:12:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.380+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 shutdown_connections 2023-12-26T23:12:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.380+0000 7f182b3ff700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1808066f30 0x7f18080693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.380+0000 7f182b3ff700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1824159a70 0x7f18240c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.380+0000 7f182b3ff700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18240b2a90 0x7f182415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:34.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.380+0000 7f182b3ff700 1 --2- 172.21.15.131:0/3052747965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18241596a0 0x7f18240cb860 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:34.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.380+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 >> 172.21.15.131:0/3052747965 conn(0x7f182406ece0 msgr2=0x7f18240b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:34.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.380+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 shutdown_connections 2023-12-26T23:12:34.382 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:34.380+0000 7f182b3ff700 1 -- 172.21.15.131:0/3052747965 wait complete. 2023-12-26T23:12:34.398 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-26T23:12:34.398 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-26T23:12:34.398 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2023-12-26T23:12:34.398 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-26T23:12:34.398 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-26T23:12:34.398 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-26T23:12:34.398 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-26T23:12:34.399 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-26T23:12:34.399 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T23:12:34.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:34.399 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2023-12-26T23:12:34.445 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-26T23:12:34.445 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-26T23:12:34.445 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2023-12-26T23:12:34.445 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:12:34.445 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-26T23:12:34.446 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 36702sec preferred_lft 36702sec 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::9cd3:3ff:fe76:6816/64 scope link 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T23:12:34.447 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether da:e1:e0:85:0b:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T23:12:34.448 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::d8e1:e0ff:fe85:b2e/64 scope link 2023-12-26T23:12:34.448 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:12:34.448 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-26T23:12:34.448 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ce:77:fe:4a:4a:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T23:12:34.448 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::cc77:feff:fe4a:4a6d/64 scope link 2023-12-26T23:12:34.448 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2023-12-26T23:12:34.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:34.448 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-26T23:12:34.551 INFO:teuthology.orchestra.run.smithi131.stdout:1 2023-12-26T23:12:34.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:34.555 DEBUG:teuthology.orchestra.run.smithi131:> route 2023-12-26T23:12:34.601 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2023-12-26T23:12:34.601 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-26T23:12:34.601 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-26T23:12:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-26T23:12:34.602 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-26T23:12:34.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:34.603 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec bash -c ' 2023-12-26T23:12:34.603 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2023-12-26T23:12:34.603 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-26T23:12:34.603 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-26T23:12:34.603 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-26T23:12:34.603 DEBUG:teuthology.orchestra.run.smithi131:> ') 2023-12-26T23:12:34.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:34.775 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T23:12:34.819 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T23:12:34.819 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T23:12:34.819 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-26T23:12:34.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:34.820 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2023-12-26T23:12:34.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T23:12:34.930 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-26T23:12:34.974 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-26T23:12:34.975 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-26T23:12:34.975 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-26T23:12:35.058 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-26T23:12:35.058 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-12-26T23:12:35.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T23:12:35.197 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T23:12:35.197 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T23:12:35.227 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-26T23:12:35.227 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-26T23:12:35.227 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-12-26T23:12:35.227 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@ab603302-d88b-429a-be64-92b3c298adad.cephfs=/ 2023-12-26T23:12:35.227 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2023-12-26T23:12:35.233 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-26T23:12:35.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:35.233 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-26T23:12:35.302 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-26T23:12:35.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.494+0000 7ff78ebff700 1 Processor -- start 2023-12-26T23:12:35.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.494+0000 7ff78ebff700 1 -- start start 2023-12-26T23:12:35.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.496+0000 7ff78ebff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7880b0bf0 0x7ff7880c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.496+0000 7ff78ebff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff788156940 con 0x7ff7880b0750 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.496+0000 7ff78ebff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7880ae500 con 0x7ff7880cb320 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.496+0000 7ff78ebff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7880c8cc0 con 0x7ff7880b0bf0 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.496+0000 7ff787fff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff7880b0750 0x7ff7880c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39712/0 (socket says 172.21.15.131:39712) 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.496+0000 7ff787fff700 1 -- 172.21.15.131:0/543720351 learned_addr learned my addr 172.21.15.131:0/543720351 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.496+0000 7ff78d19c700 1 --2- 172.21.15.131:0/543720351 >> v2:172.21.15.190:3300/0 conn(0x7ff7880b0bf0 0x7ff7880c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.496+0000 7ff7877fe700 1 -- 172.21.15.131:0/543720351 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975308119 0 0) 0x7ff788156940 con 0x7ff7880b0750 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.496+0000 7ff7877fe700 1 -- 172.21.15.131:0/543720351 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff778003570 con 0x7ff7880b0750 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff7877fe700 1 -- 172.21.15.131:0/543720351 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346026110 0 0) 0x7ff7880ae500 con 0x7ff7880cb320 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff7877fe700 1 -- 172.21.15.131:0/543720351 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff788156940 con 0x7ff7880cb320 2023-12-26T23:12:35.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff7877fe700 1 -- 172.21.15.131:0/543720351 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1153947629 0 0) 0x7ff778003570 con 0x7ff7880b0750 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff78d19c700 1 -- 172.21.15.131:0/543720351 >> v1:172.21.15.190:6789/0 conn(0x7ff7880cb320 legacy=0x7ff788069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff78d19c700 1 -- 172.21.15.131:0/543720351 >> v1:172.21.15.131:6789/0 conn(0x7ff7880b0750 legacy=0x7ff7880c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff78d19c700 1 -- 172.21.15.131:0/543720351 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7880b0280 con 0x7ff7880b0bf0 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff78d19c700 1 --2- 172.21.15.131:0/543720351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7880b0bf0 0x7ff7880c9b10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff780009e60 tx=0x7ff780009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8970990058d99285 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff7877fe700 1 -- 172.21.15.131:0/543720351 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000e070 con 0x7ff7880b0bf0 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff7877fe700 1 -- 172.21.15.131:0/543720351 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78000b7a0 con 0x7ff7880b0bf0 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.497+0000 7ff7877fe700 1 -- 172.21.15.131:0/543720351 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774001470 con 0x7ff7880b0bf0 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.498+0000 7ff78ebff700 1 -- 172.21.15.131:0/543720351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7880b0bf0 msgr2=0x7ff7880c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.498+0000 7ff78ebff700 1 --2- 172.21.15.131:0/543720351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7880b0bf0 0x7ff7880c9b10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff780009e60 tx=0x7ff780009780 comp rx=0 tx=0).stop 2023-12-26T23:12:35.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.498+0000 7ff78ebff700 1 -- 172.21.15.131:0/543720351 shutdown_connections 2023-12-26T23:12:35.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.498+0000 7ff78ebff700 1 --2- 172.21.15.131:0/543720351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7880b0bf0 0x7ff7880c9b10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.498+0000 7ff78ebff700 1 -- 172.21.15.131:0/543720351 >> 172.21.15.131:0/543720351 conn(0x7ff788068040 msgr2=0x7ff788069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:35.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.498+0000 7ff78ebff700 1 -- 172.21.15.131:0/543720351 shutdown_connections 2023-12-26T23:12:35.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.498+0000 7ff78ebff700 1 -- 172.21.15.131:0/543720351 wait complete. 2023-12-26T23:12:35.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.499+0000 7ff78ebff700 1 Processor -- start 2023-12-26T23:12:35.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.500+0000 7ff78ebff700 1 -- start start 2023-12-26T23:12:35.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.500+0000 7ff78ebff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7880b0750 0x7ff7881474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.500+0000 7ff78ebff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7880b0bf0 0x7ff788150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.500+0000 7ff78ebff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7880cb320 0x7ff7880ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.500+0000 7ff78ebff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78806de40 con 0x7ff7880b0bf0 2023-12-26T23:12:35.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.500+0000 7ff78ebff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78806d580 con 0x7ff7880b0750 2023-12-26T23:12:35.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.500+0000 7ff78ebff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff788151390 con 0x7ff7880cb320 2023-12-26T23:12:35.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff787fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7880b0bf0 0x7ff788150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff787fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7880b0bf0 0x7ff788150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53658/0 (socket says 172.21.15.131:53658) 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff787fff700 1 -- 172.21.15.131:0/908121554 learned_addr learned my addr 172.21.15.131:0/908121554 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff78c99b700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7880b0750 0x7ff7881474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff78d19c700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7880cb320 0x7ff7880ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff787fff700 1 -- 172.21.15.131:0/908121554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7880cb320 msgr2=0x7ff7880ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff787fff700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7880cb320 0x7ff7880ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff787fff700 1 -- 172.21.15.131:0/908121554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7880b0750 msgr2=0x7ff7881474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff787fff700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7880b0750 0x7ff7881474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff787fff700 1 -- 172.21.15.131:0/908121554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff780009a20 con 0x7ff7880b0bf0 2023-12-26T23:12:35.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff78d19c700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7880cb320 0x7ff7880ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:35.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.501+0000 7ff787fff700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7880b0bf0 0x7ff788150e50 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff77400cbc0 tx=0x7ff774001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:35.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.502+0000 7ff784ff9700 1 -- 172.21.15.131:0/908121554 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774007e90 con 0x7ff7880b0bf0 2023-12-26T23:12:35.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.502+0000 7ff784ff9700 1 -- 172.21.15.131:0/908121554 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff774007360 con 0x7ff7880b0bf0 2023-12-26T23:12:35.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.502+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78806d7f0 con 0x7ff7880b0bf0 2023-12-26T23:12:35.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.502+0000 7ff784ff9700 1 -- 172.21.15.131:0/908121554 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774004830 con 0x7ff7880b0bf0 2023-12-26T23:12:35.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.502+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7880aaff0 con 0x7ff7880b0bf0 2023-12-26T23:12:35.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.503+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7880657e0 con 0x7ff7880b0bf0 2023-12-26T23:12:35.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.504+0000 7ff784ff9700 1 -- 172.21.15.131:0/908121554 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff774004be0 con 0x7ff7880b0bf0 2023-12-26T23:12:35.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.504+0000 7ff784ff9700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff76c066f30 0x7ff76c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.505+0000 7ff784ff9700 1 -- 172.21.15.131:0/908121554 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7ff774083b50 con 0x7ff7880b0bf0 2023-12-26T23:12:35.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.505+0000 7ff78c99b700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff76c066f30 0x7ff76c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.505+0000 7ff78c99b700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff76c066f30 0x7ff76c0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff77c007fd0 tx=0x7ff77c007d50 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:35.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.509+0000 7ff784ff9700 1 -- 172.21.15.131:0/908121554 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff774050c30 con 0x7ff7880b0bf0 2023-12-26T23:12:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.728+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff78806df80 con 0x7ff76c066f30 2023-12-26T23:12:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.729+0000 7ff784ff9700 1 -- 172.21.15.131:0/908121554 <== mgr.9927 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff78806df80 con 0x7ff76c066f30 2023-12-26T23:12:35.730 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2023-12-26T23:12:35.730 INFO:teuthology.orchestra.run.smithi131.stdout: { 2023-12-26T23:12:35.730 INFO:teuthology.orchestra.run.smithi131.stdout: "name": "cephfs" 2023-12-26T23:12:35.730 INFO:teuthology.orchestra.run.smithi131.stdout: } 2023-12-26T23:12:35.731 INFO:teuthology.orchestra.run.smithi131.stdout:] 2023-12-26T23:12:35.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff76c066f30 msgr2=0x7ff76c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff76c066f30 0x7ff76c0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff77c007fd0 tx=0x7ff77c007d50 comp rx=0 tx=0).stop 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7880b0bf0 msgr2=0x7ff788150e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7880b0bf0 0x7ff788150e50 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff77400cbc0 tx=0x7ff774001b50 comp rx=0 tx=0).stop 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 shutdown_connections 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff76c066f30 0x7ff76c0693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7880cb320 0x7ff7880ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7880b0750 0x7ff7881474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 --2- 172.21.15.131:0/908121554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7880b0bf0 0x7ff788150e50 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 >> 172.21.15.131:0/908121554 conn(0x7ff788068040 msgr2=0x7ff7880c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 shutdown_connections 2023-12-26T23:12:35.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.732+0000 7ff78ebff700 1 -- 172.21.15.131:0/908121554 wait complete. 2023-12-26T23:12:35.750 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-26T23:12:35.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.942+0000 7f2a99fff700 1 Processor -- start 2023-12-26T23:12:35.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.942+0000 7f2a99fff700 1 -- start start 2023-12-26T23:12:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a99fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940cb320 0x7f2a94069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a99fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2a940b0e60 0x7f2a940c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a99fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a94156940 con 0x7f2a9406a1e0 2023-12-26T23:12:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a99fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a940c8cc0 con 0x7f2a940b0e60 2023-12-26T23:12:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a99fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940ad700 con 0x7f2a940cb320 2023-12-26T23:12:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a93fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2a940b0e60 0x7f2a940c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a937fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940cb320 0x7f2a94069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a937fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940cb320 0x7f2a94069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:53966/0 (socket says 172.21.15.131:53966) 2023-12-26T23:12:35.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a93fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2a940b0e60 0x7f2a940c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:47956/0 (socket says 172.21.15.131:47956) 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a937fe700 1 -- 172.21.15.131:0/803307283 learned_addr learned my addr 172.21.15.131:0/803307283 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.944+0000 7f2a92ffd700 1 --1- 172.21.15.131:0/803307283 >> v1:172.21.15.190:6789/0 conn(0x7f2a9406a1e0 0x7f2a940b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:48870/0 (socket says 172.21.15.131:48870) 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a927fc700 1 -- 172.21.15.131:0/803307283 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347455304 0 0) 0x7f2a94156940 con 0x7f2a9406a1e0 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a927fc700 1 -- 172.21.15.131:0/803307283 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a7c003570 con 0x7f2a9406a1e0 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a93fff700 1 -- 172.21.15.131:0/803307283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940cb320 msgr2=0x7f2a94069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a93fff700 1 --2- 172.21.15.131:0/803307283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940cb320 0x7f2a94069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a93fff700 1 -- 172.21.15.131:0/803307283 >> v1:172.21.15.190:6789/0 conn(0x7f2a9406a1e0 legacy=0x7f2a940b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a93fff700 1 -- 172.21.15.131:0/803307283 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a940ae500 con 0x7f2a940b0e60 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a937fe700 1 --2- 172.21.15.131:0/803307283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940cb320 0x7f2a94069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a93fff700 1 --2- 172.21.15.131:0/803307283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940b0e60 0x7f2a940c9400 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00d0d0 tx=0x7f2a8c00b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f478b1ec6df340c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:35.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a927fc700 1 -- 172.21.15.131:0/803307283 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c010070 con 0x7f2a940b0e60 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a927fc700 1 -- 172.21.15.131:0/803307283 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c00b2e0 con 0x7f2a940b0e60 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.945+0000 7f2a927fc700 1 -- 172.21.15.131:0/803307283 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00dcd0 con 0x7f2a940b0e60 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.946+0000 7f2a99fff700 1 -- 172.21.15.131:0/803307283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940b0e60 msgr2=0x7f2a940c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.946+0000 7f2a99fff700 1 --2- 172.21.15.131:0/803307283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940b0e60 0x7f2a940c9400 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00d0d0 tx=0x7f2a8c00b990 comp rx=0 tx=0).stop 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.946+0000 7f2a99fff700 1 -- 172.21.15.131:0/803307283 shutdown_connections 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.946+0000 7f2a99fff700 1 --2- 172.21.15.131:0/803307283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940b0e60 0x7f2a940c9400 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.946+0000 7f2a99fff700 1 --2- 172.21.15.131:0/803307283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940cb320 0x7f2a94069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.946+0000 7f2a99fff700 1 -- 172.21.15.131:0/803307283 >> 172.21.15.131:0/803307283 conn(0x7f2a94068040 msgr2=0x7f2a94069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.946+0000 7f2a99fff700 1 -- 172.21.15.131:0/803307283 shutdown_connections 2023-12-26T23:12:35.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.946+0000 7f2a99fff700 1 -- 172.21.15.131:0/803307283 wait complete. 2023-12-26T23:12:35.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.947+0000 7f2a99fff700 1 Processor -- start 2023-12-26T23:12:35.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.948+0000 7f2a99fff700 1 -- start start 2023-12-26T23:12:35.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a99fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a9406a1e0 0x7f2a9414bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a99fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940b0e60 0x7f2a940c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a99fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940cb320 0x7f2a941690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a99fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940a63e0 con 0x7f2a9406a1e0 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a99fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9414c250 con 0x7f2a940cb320 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a99fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940a57d0 con 0x7f2a940b0e60 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a937fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a9406a1e0 0x7f2a9414bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a937fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a9406a1e0 0x7f2a9414bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53672/0 (socket says 172.21.15.131:53672) 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a937fe700 1 -- 172.21.15.131:0/1845117537 learned_addr learned my addr 172.21.15.131:0/1845117537 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a92ffd700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940b0e60 0x7f2a940c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a937fe700 1 -- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940b0e60 msgr2=0x7f2a940c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a93fff700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940cb320 0x7f2a941690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a937fe700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940b0e60 0x7f2a940c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a937fe700 1 -- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940cb320 msgr2=0x7f2a941690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a937fe700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940cb320 0x7f2a941690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a937fe700 1 -- 172.21.15.131:0/1845117537 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a840094f0 con 0x7f2a9406a1e0 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a92ffd700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940b0e60 0x7f2a940c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.949+0000 7f2a93fff700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940cb320 0x7f2a941690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.950+0000 7f2a937fe700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a9406a1e0 0x7f2a9414bd10 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400ec50 tx=0x7f2a84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.950+0000 7f2a7b7fe700 1 -- 172.21.15.131:0/1845117537 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400c9d0 con 0x7f2a9406a1e0 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.950+0000 7f2a7b7fe700 1 -- 172.21.15.131:0/1845117537 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8400cb30 con 0x7f2a9406a1e0 2023-12-26T23:12:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.950+0000 7f2a7b7fe700 1 -- 172.21.15.131:0/1845117537 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84013470 con 0x7f2a9406a1e0 2023-12-26T23:12:35.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.950+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c00bc30 con 0x7f2a9406a1e0 2023-12-26T23:12:35.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.950+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a940a5d90 con 0x7f2a9406a1e0 2023-12-26T23:12:35.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.951+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a940657e0 con 0x7f2a9406a1e0 2023-12-26T23:12:35.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.952+0000 7f2a7b7fe700 1 -- 172.21.15.131:0/1845117537 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 107) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2a84013860 con 0x7f2a9406a1e0 2023-12-26T23:12:35.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.953+0000 7f2a7b7fe700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a80066f30 0x7f2a800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:35.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.953+0000 7f2a7b7fe700 1 -- 172.21.15.131:0/1845117537 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f2a84084b50 con 0x7f2a9406a1e0 2023-12-26T23:12:35.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.953+0000 7f2a92ffd700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a80066f30 0x7f2a800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:35.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.954+0000 7f2a92ffd700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a80066f30 0x7f2a800693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2a9406f000 tx=0x7f2a88009040 comp rx=0 tx=0).ready entity=mgr.9927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:35.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:35.957+0000 7f2a7b7fe700 1 -- 172.21.15.131:0/1845117537 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a84051b10 con 0x7f2a9406a1e0 2023-12-26T23:12:36.207 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.207+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2a94153c50 con 0x7f2a9406a1e0 2023-12-26T23:12:36.447 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.446+0000 7f2a7b7fe700 1 -- 172.21.15.131:0/1845117537 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 108) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f2a84051ee0 con 0x7f2a9406a1e0 2023-12-26T23:12:36.448 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.448+0000 7f2a7b7fe700 1 -- 172.21.15.131:0/1845117537 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v108) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2a8407d500 con 0x7f2a9406a1e0 2023-12-26T23:12:36.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.450+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a80066f30 msgr2=0x7f2a800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:36.451 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.450+0000 7f2a99fff700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a80066f30 0x7f2a800693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2a9406f000 tx=0x7f2a88009040 comp rx=0 tx=0).stop 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.450+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a9406a1e0 msgr2=0x7f2a9414bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.450+0000 7f2a99fff700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a9406a1e0 0x7f2a9414bd10 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400ec50 tx=0x7f2a84009250 comp rx=0 tx=0).stop 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.450+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 shutdown_connections 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.450+0000 7f2a99fff700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2a80066f30 0x7f2a800693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.450+0000 7f2a99fff700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a940cb320 0x7f2a941690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.450+0000 7f2a99fff700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a9406a1e0 0x7f2a9414bd10 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.451+0000 7f2a99fff700 1 --2- 172.21.15.131:0/1845117537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a940b0e60 0x7f2a940c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.451+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 >> 172.21.15.131:0/1845117537 conn(0x7f2a94068040 msgr2=0x7f2a9406d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.451+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 shutdown_connections 2023-12-26T23:12:36.452 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.451+0000 7f2a99fff700 1 -- 172.21.15.131:0/1845117537 wait complete. 2023-12-26T23:12:36.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-26T23:12:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.670+0000 7f04dbbff700 1 Processor -- start 2023-12-26T23:12:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.671+0000 7f04dbbff700 1 -- start start 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04dbbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04d40cc580 0x7f04d40cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04dbbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b0560 0x7f04d415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04dbbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d40b0d70 con 0x7f04d40b00e0 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04dbbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d40680f0 con 0x7f04d40cc580 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04dbbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40675b0 con 0x7f04d40b0560 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04d999b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04d40cc580 0x7f04d40cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04da19c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b0560 0x7f04d415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04d919a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f04d40b00e0 0x7f04d406e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:48886/0 (socket says 172.21.15.131:48886) 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04da19c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b0560 0x7f04d415d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:53994/0 (socket says 172.21.15.131:53994) 2023-12-26T23:12:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04d999b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04d40cc580 0x7f04d40cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:47992/0 (socket says 172.21.15.131:47992) 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.672+0000 7f04d919a700 1 -- 172.21.15.131:0/599327652 learned_addr learned my addr 172.21.15.131:0/599327652 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04da19c700 1 -- 172.21.15.131:0/599327652 >> v2:172.21.15.190:3300/0 conn(0x7f04d40cc580 msgr2=0x7f04d40cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04da19c700 1 --2- 172.21.15.131:0/599327652 >> v2:172.21.15.190:3300/0 conn(0x7f04d40cc580 0x7f04d40cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04da19c700 1 -- 172.21.15.131:0/599327652 >> v1:172.21.15.190:6789/0 conn(0x7f04d40b00e0 legacy=0x7f04d406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04da19c700 1 -- 172.21.15.131:0/599327652 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d4159c30 con 0x7f04d40b0560 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04d999b700 1 --2- 172.21.15.131:0/599327652 >> v2:172.21.15.190:3300/0 conn(0x7f04d40cc580 0x7f04d40cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04d8999700 1 -- 172.21.15.131:0/599327652 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551549386 0 0) 0x7f04d40b0d70 con 0x7f04d40b00e0 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04da19c700 1 --2- 172.21.15.131:0/599327652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b0560 0x7f04d415d9b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009e60 tx=0x7f04cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f106bf709fedb163 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04d8999700 1 -- 172.21.15.131:0/599327652 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00e070 con 0x7f04d40b0560 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04d8999700 1 -- 172.21.15.131:0/599327652 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc00bb60 con 0x7f04d40b0560 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.673+0000 7f04d8999700 1 -- 172.21.15.131:0/599327652 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc012430 con 0x7f04d40b0560 2023-12-26T23:12:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.674+0000 7f04dbbff700 1 -- 172.21.15.131:0/599327652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b0560 msgr2=0x7f04d415d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.674+0000 7f04dbbff700 1 --2- 172.21.15.131:0/599327652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b0560 0x7f04d415d9b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009e60 tx=0x7f04cc009780 comp rx=0 tx=0).stop 2023-12-26T23:12:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.674+0000 7f04dbbff700 1 -- 172.21.15.131:0/599327652 shutdown_connections 2023-12-26T23:12:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.674+0000 7f04dbbff700 1 --2- 172.21.15.131:0/599327652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b0560 0x7f04d415d9b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.674+0000 7f04dbbff700 1 --2- 172.21.15.131:0/599327652 >> v2:172.21.15.190:3300/0 conn(0x7f04d40cc580 0x7f04d40cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.674+0000 7f04dbbff700 1 -- 172.21.15.131:0/599327652 >> 172.21.15.131:0/599327652 conn(0x7f04d40686c0 msgr2=0x7f04d4068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.674+0000 7f04dbbff700 1 -- 172.21.15.131:0/599327652 shutdown_connections 2023-12-26T23:12:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.674+0000 7f04dbbff700 1 -- 172.21.15.131:0/599327652 wait complete. 2023-12-26T23:12:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.676+0000 7f04dbbff700 1 Processor -- start 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.676+0000 7f04dbbff700 1 -- start start 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04dbbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b00e0 0x7f04d413dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04dbbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04d40b0560 0x7f04d4169aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04dbbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04d40cc580 0x7f04d40c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04dbbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d413e270 con 0x7f04d40b0560 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04dbbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d415afc0 con 0x7f04d40cc580 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04dbbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d415b2a0 con 0x7f04d40b00e0 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04d919a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04d40b0560 0x7f04d4169aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04d999b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b00e0 0x7f04d413dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04da19c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04d40cc580 0x7f04d40c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04d999b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b00e0 0x7f04d413dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:54010/0 (socket says 172.21.15.131:54010) 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04d999b700 1 -- 172.21.15.131:0/3577400247 learned_addr learned my addr 172.21.15.131:0/3577400247 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04da19c700 1 -- 172.21.15.131:0/3577400247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b00e0 msgr2=0x7f04d413dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04da19c700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b00e0 0x7f04d413dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04da19c700 1 -- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04d40b0560 msgr2=0x7f04d4169aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04da19c700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04d40b0560 0x7f04d4169aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04da19c700 1 -- 172.21.15.131:0/3577400247 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d0009420 con 0x7f04d40cc580 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.677+0000 7f04d999b700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b00e0 0x7f04d413dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.678+0000 7f04da19c700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04d40cc580 0x7f04d40c48c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f04cc016000 tx=0x7f04cc019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.678+0000 7f04c9ffb700 1 -- 172.21.15.131:0/3577400247 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc012ea0 con 0x7f04d40cc580 2023-12-26T23:12:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.678+0000 7f04c9ffb700 1 -- 172.21.15.131:0/3577400247 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc027470 con 0x7f04d40cc580 2023-12-26T23:12:36.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.678+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc009a20 con 0x7f04d40cc580 2023-12-26T23:12:36.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.678+0000 7f04c9ffb700 1 -- 172.21.15.131:0/3577400247 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc027810 con 0x7f04d40cc580 2023-12-26T23:12:36.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.678+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d415b860 con 0x7f04d40cc580 2023-12-26T23:12:36.682 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.679+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d40657e0 con 0x7f04d40cc580 2023-12-26T23:12:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.680+0000 7f04c9ffb700 1 -- 172.21.15.131:0/3577400247 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 108) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f04cc024070 con 0x7f04d40cc580 2023-12-26T23:12:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.681+0000 7f04c9ffb700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f04b8066f80 0x7f04b8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.681+0000 7f04c9ffb700 1 -- 172.21.15.131:0/3577400247 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f04cc0871c0 con 0x7f04d40cc580 2023-12-26T23:12:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.681+0000 7f04d999b700 1 -- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f04b8066f80 msgr2=0x7f04b8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.681+0000 7f04d999b700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f04b8066f80 0x7f04b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:36.689 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.689+0000 7f04c9ffb700 1 -- 172.21.15.131:0/3577400247 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04cc054200 con 0x7f04d40cc580 2023-12-26T23:12:36.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.882+0000 7f04d999b700 1 -- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f04b8066f80 msgr2=0x7f04b8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:36.882 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.882+0000 7f04d999b700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f04b8066f80 0x7f04b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.890+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f04d4067850 con 0x7f04d40cc580 2023-12-26T23:12:36.891 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:36.891+0000 7fa30a5231c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:12:36.896 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.896+0000 7f04c9ffb700 1 -- 172.21.15.131:0/3577400247 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f04cc057820 con 0x7f04d40cc580 2023-12-26T23:12:36.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f04b8066f80 msgr2=0x7f04b8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:36.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f04b8066f80 0x7f04b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04d40cc580 msgr2=0x7f04d40c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04d40cc580 0x7f04d40c48c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f04cc016000 tx=0x7f04cc019000 comp rx=0 tx=0).stop 2023-12-26T23:12:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 shutdown_connections 2023-12-26T23:12:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f04b8066f80 0x7f04b8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04d40cc580 0x7f04d40c48c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04d40b00e0 0x7f04d413dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 --2- 172.21.15.131:0/3577400247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04d40b0560 0x7f04d4169aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.898+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 >> 172.21.15.131:0/3577400247 conn(0x7f04d40686c0 msgr2=0x7f04d40cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.899+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 shutdown_connections 2023-12-26T23:12:36.900 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:36.899+0000 7f04dbbff700 1 -- 172.21.15.131:0/3577400247 wait complete. 2023-12-26T23:12:36.905 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:36.904+0000 7f0bc74851c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:12:36.918 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-26T23:12:36.997 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:36.998+0000 7fa30a5231c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:12:37.020 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:37.020+0000 7f0bc74851c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:12:37.088 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:37.088+0000 7fa30a5231c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:12:37.114 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:37.114+0000 7f0bc74851c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:12:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.120+0000 7f6329dff700 1 Processor -- start 2023-12-26T23:12:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.121+0000 7f6329dff700 1 -- start start 2023-12-26T23:12:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6329dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f632406bfd0 0x7f63240c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6329dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63240cc100 0x7f63240cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6329dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63240686c0 con 0x7f632413c510 2023-12-26T23:12:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6329dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6324069520 con 0x7f63240cc100 2023-12-26T23:12:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6329dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6324068910 con 0x7f632406bfd0 2023-12-26T23:12:37.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6323fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63240cc100 0x7f63240cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6322ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f632406bfd0 0x7f63240c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6323fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63240cc100 0x7f63240cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53710/0 (socket says 172.21.15.131:53710) 2023-12-26T23:12:37.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6322ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f632406bfd0 0x7f63240c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:48006/0 (socket says 172.21.15.131:48006) 2023-12-26T23:12:37.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f6323fff700 1 -- 172.21.15.131:0/2679205761 learned_addr learned my addr 172.21.15.131:0/2679205761 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:37.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.122+0000 7f63237fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f632413c510 0x7f632406b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:48900/0 (socket says 172.21.15.131:48900) 2023-12-26T23:12:37.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.123+0000 7f6323fff700 1 -- 172.21.15.131:0/2679205761 >> v2:172.21.15.190:3300/0 conn(0x7f632406bfd0 msgr2=0x7f63240c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:37.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.123+0000 7f6323fff700 1 --2- 172.21.15.131:0/2679205761 >> v2:172.21.15.190:3300/0 conn(0x7f632406bfd0 0x7f63240c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.125 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.123+0000 7f6323fff700 1 -- 172.21.15.131:0/2679205761 >> v1:172.21.15.190:6789/0 conn(0x7f632413c510 legacy=0x7f632406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.123+0000 7f6323fff700 1 -- 172.21.15.131:0/2679205761 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324068bf0 con 0x7f63240cc100 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.123+0000 7f6322ffd700 1 --2- 172.21.15.131:0/2679205761 >> v2:172.21.15.190:3300/0 conn(0x7f632406bfd0 0x7f63240c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.123+0000 7f6323fff700 1 --2- 172.21.15.131:0/2679205761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63240cc100 0x7f63240cc520 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f63240b43e0 tx=0x7f631c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f83d671d2d91767 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.123+0000 7f63227fc700 1 -- 172.21.15.131:0/2679205761 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c00e070 con 0x7f63240cc100 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.123+0000 7f63227fc700 1 -- 172.21.15.131:0/2679205761 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c00ba20 con 0x7f63240cc100 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.123+0000 7f63227fc700 1 -- 172.21.15.131:0/2679205761 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0092e0 con 0x7f63240cc100 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.124+0000 7f6329dff700 1 -- 172.21.15.131:0/2679205761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63240cc100 msgr2=0x7f63240cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.124+0000 7f6329dff700 1 --2- 172.21.15.131:0/2679205761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63240cc100 0x7f63240cc520 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f63240b43e0 tx=0x7f631c009840 comp rx=0 tx=0).stop 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.124+0000 7f6329dff700 1 -- 172.21.15.131:0/2679205761 shutdown_connections 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.124+0000 7f6329dff700 1 --2- 172.21.15.131:0/2679205761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63240cc100 0x7f63240cc520 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.124+0000 7f6329dff700 1 --2- 172.21.15.131:0/2679205761 >> v2:172.21.15.190:3300/0 conn(0x7f632406bfd0 0x7f63240c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.124+0000 7f6329dff700 1 -- 172.21.15.131:0/2679205761 >> 172.21.15.131:0/2679205761 conn(0x7f63240ce3e0 msgr2=0x7f63240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.124+0000 7f6329dff700 1 -- 172.21.15.131:0/2679205761 shutdown_connections 2023-12-26T23:12:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.124+0000 7f6329dff700 1 -- 172.21.15.131:0/2679205761 wait complete. 2023-12-26T23:12:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.125+0000 7f6329dff700 1 Processor -- start 2023-12-26T23:12:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.125+0000 7f6329dff700 1 -- start start 2023-12-26T23:12:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.126+0000 7f6329dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632406bfd0 0x7f6324169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.126+0000 7f6329dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63240cc100 0x7f63240c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.126+0000 7f6329dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f632413c510 0x7f63240c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.130 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.126+0000 7f6329dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63240c6c10 con 0x7f632413c510 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.126+0000 7f6329dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241699a0 con 0x7f63240cc100 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.126+0000 7f6329dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63240c6000 con 0x7f632406bfd0 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6323fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f632413c510 0x7f63240c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6323fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f632413c510 0x7f63240c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53716/0 (socket says 172.21.15.131:53716) 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6323fff700 1 -- 172.21.15.131:0/2100146531 learned_addr learned my addr 172.21.15.131:0/2100146531 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6322ffd700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63240cc100 0x7f63240c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f63237fe700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632406bfd0 0x7f6324169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6323fff700 1 -- 172.21.15.131:0/2100146531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632406bfd0 msgr2=0x7f6324169e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6323fff700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632406bfd0 0x7f6324169e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.131 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6323fff700 1 -- 172.21.15.131:0/2100146531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63240cc100 msgr2=0x7f63240c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6323fff700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63240cc100 0x7f63240c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6323fff700 1 -- 172.21.15.131:0/2100146531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6318009420 con 0x7f632413c510 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6322ffd700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63240cc100 0x7f63240c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.127+0000 7f6323fff700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f632413c510 0x7f63240c59d0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f631c009880 tx=0x7f631c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.128+0000 7f630b7fe700 1 -- 172.21.15.131:0/2100146531 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c008a80 con 0x7f632413c510 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.128+0000 7f630b7fe700 1 -- 172.21.15.131:0/2100146531 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c008c00 con 0x7f632413c510 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.128+0000 7f630b7fe700 1 -- 172.21.15.131:0/2100146531 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c026460 con 0x7f632413c510 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.128+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c009ae0 con 0x7f632413c510 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.128+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63240c65c0 con 0x7f632413c510 2023-12-26T23:12:37.132 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.129+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63240c6dc0 con 0x7f632413c510 2023-12-26T23:12:37.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.130+0000 7f630b7fe700 1 -- 172.21.15.131:0/2100146531 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 108) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f631c003040 con 0x7f632413c510 2023-12-26T23:12:37.133 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.131+0000 7f630b7fe700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6310066f80 0x7f6310069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.131+0000 7f63237fe700 1 -- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6310066f80 msgr2=0x7f6310069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:37.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.131+0000 7f630b7fe700 1 -- 172.21.15.131:0/2100146531 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f631c0860d0 con 0x7f632413c510 2023-12-26T23:12:37.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.131+0000 7f63237fe700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6310066f80 0x7f6310069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:37.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.135+0000 7f630b7fe700 1 -- 172.21.15.131:0/2100146531 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63240c6dc0 con 0x7f632413c510 2023-12-26T23:12:37.172 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:37.172+0000 7fa30a5231c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:12:37.201 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:37.201+0000 7f0bc74851c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:12:37.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.332+0000 7f63237fe700 1 -- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6310066f80 msgr2=0x7f6310069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:37.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.332+0000 7f63237fe700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6310066f80 0x7f6310069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:37.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.336+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f63240657e0 con 0x7f632413c510 2023-12-26T23:12:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.340+0000 7f630b7fe700 1 -- 172.21.15.131:0/2100146531 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f631c053130 con 0x7f632413c510 2023-12-26T23:12:37.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6310066f80 msgr2=0x7f6310069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:37.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6310066f80 0x7f6310069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f632413c510 msgr2=0x7f63240c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:37.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f632413c510 0x7f63240c59d0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f631c009880 tx=0x7f631c018000 comp rx=0 tx=0).stop 2023-12-26T23:12:37.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 shutdown_connections 2023-12-26T23:12:37.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6310066f80 0x7f6310069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f632413c510 0x7f63240c59d0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632406bfd0 0x7f6324169e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 --2- 172.21.15.131:0/2100146531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63240cc100 0x7f63240c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 >> 172.21.15.131:0/2100146531 conn(0x7f63240ce3e0 msgr2=0x7f6324159230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:37.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.342+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 shutdown_connections 2023-12-26T23:12:37.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.343+0000 7f6329dff700 1 -- 172.21.15.131:0/2100146531 wait complete. 2023-12-26T23:12:37.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:12:37.362 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-26T23:12:37.363 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:37.364+0000 7fa30a5231c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:37.400 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:37.400+0000 7f0bc74851c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:37.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-12-26T23:12:37.538 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:37.538+0000 7fa30a5231c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:12:37.581 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:37.581+0000 7f0bc74851c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:12:37.624 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:37.625+0000 7fa30a5231c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:12:37.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.650+0000 7f1f8bdff700 1 Processor -- start 2023-12-26T23:12:37.650 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.650+0000 7f1f8bdff700 1 -- start start 2023-12-26T23:12:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.651+0000 7f1f8bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 0x7f1f8415bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.651+0000 7f1f8bdff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f840c9110 0x7f1f840ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.651+0000 7f1f8bdff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f840af040 con 0x7f1f84156e50 2023-12-26T23:12:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.651+0000 7f1f8bdff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f841579e0 con 0x7f1f840c9110 2023-12-26T23:12:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.651+0000 7f1f8bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840ae0e0 con 0x7f1f840c8d40 2023-12-26T23:12:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.651+0000 7f1f8939a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f840c9110 0x7f1f840ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.651+0000 7f1f8a39c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1f84156e50 0x7f1f841572d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39734/0 (socket says 172.21.15.131:39734) 2023-12-26T23:12:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.651+0000 7f1f8a39c700 1 -- 172.21.15.131:0/1175048644 learned_addr learned my addr 172.21.15.131:0/1175048644 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:37.654 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.651+0000 7f1f8939a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f840c9110 0x7f1f840ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53726/0 (socket says 172.21.15.131:53726) 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/1175048644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 0x7f1f8415bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f8939a700 1 -- 172.21.15.131:0/1175048644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 msgr2=0x7f1f8415bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f8939a700 1 --2- 172.21.15.131:0/1175048644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 0x7f1f8415bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f8939a700 1 -- 172.21.15.131:0/1175048644 >> v1:172.21.15.131:6789/0 conn(0x7f1f84156e50 legacy=0x7f1f841572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f8939a700 1 -- 172.21.15.131:0/1175048644 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f840ae3c0 con 0x7f1f840c9110 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f88b99700 1 -- 172.21.15.131:0/1175048644 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153540185 0 0) 0x7f1f840af040 con 0x7f1f84156e50 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f8939a700 1 --2- 172.21.15.131:0/1175048644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f840c9110 0x7f1f840ad8b0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f1f740098e0 tx=0x7f1f740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143a83deca6591dd server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f88b99700 1 -- 172.21.15.131:0/1175048644 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400b640 con 0x7f1f840c9110 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f88b99700 1 -- 172.21.15.131:0/1175048644 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7400b7a0 con 0x7f1f840c9110 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.652+0000 7f1f88b99700 1 -- 172.21.15.131:0/1175048644 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400bb40 con 0x7f1f840c9110 2023-12-26T23:12:37.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.653+0000 7f1f8bdff700 1 -- 172.21.15.131:0/1175048644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f840c9110 msgr2=0x7f1f840ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.653+0000 7f1f8bdff700 1 --2- 172.21.15.131:0/1175048644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f840c9110 0x7f1f840ad8b0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f1f740098e0 tx=0x7f1f740091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.653+0000 7f1f8bdff700 1 -- 172.21.15.131:0/1175048644 shutdown_connections 2023-12-26T23:12:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.653+0000 7f1f8bdff700 1 --2- 172.21.15.131:0/1175048644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 0x7f1f8415bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.653+0000 7f1f8bdff700 1 --2- 172.21.15.131:0/1175048644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f840c9110 0x7f1f840ad8b0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.653+0000 7f1f8bdff700 1 -- 172.21.15.131:0/1175048644 >> 172.21.15.131:0/1175048644 conn(0x7f1f840675b0 msgr2=0x7f1f840679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.653+0000 7f1f8bdff700 1 -- 172.21.15.131:0/1175048644 shutdown_connections 2023-12-26T23:12:37.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.653+0000 7f1f8bdff700 1 -- 172.21.15.131:0/1175048644 wait complete. 2023-12-26T23:12:37.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.655+0000 7f1f8bdff700 1 Processor -- start 2023-12-26T23:12:37.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.655+0000 7f1f8bdff700 1 -- start start 2023-12-26T23:12:37.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 0x7f1f84145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8bdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f840c9110 0x7f1f840c07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8bdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f84156e50 0x7f1f841562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8bdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84158010 con 0x7f1f84156e50 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8bdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840c1270 con 0x7f1f840c9110 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840c1550 con 0x7f1f840c8d40 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8a39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f84156e50 0x7f1f841562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8a39c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f84156e50 0x7f1f841562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53740/0 (socket says 172.21.15.131:53740) 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8a39c700 1 -- 172.21.15.131:0/3577669943 learned_addr learned my addr 172.21.15.131:0/3577669943 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 0x7f1f84145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.656+0000 7f1f8939a700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f840c9110 0x7f1f840c07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f8a39c700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 msgr2=0x7f1f84145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:37.660 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f8a39c700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 0x7f1f84145580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f8a39c700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f840c9110 msgr2=0x7f1f840c07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:37.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f8a39c700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f840c9110 0x7f1f840c07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:37.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f8a39c700 1 -- 172.21.15.131:0/3577669943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74009420 con 0x7f1f84156e50 2023-12-26T23:12:37.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 0x7f1f84145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:37.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f8939a700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f840c9110 0x7f1f840c07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:37.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f8a39c700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f84156e50 0x7f1f841562b0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00cbf0 tx=0x7f1f7c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:37.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00a8b0 con 0x7f1f84156e50 2023-12-26T23:12:37.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c00aa10 con 0x7f1f84156e50 2023-12-26T23:12:37.661 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f84157ca0 con 0x7f1f84156e50 2023-12-26T23:12:37.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00add0 con 0x7f1f84156e50 2023-12-26T23:12:37.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.657+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f840c1e10 con 0x7f1f84156e50 2023-12-26T23:12:37.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.659+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f84158180 con 0x7f1f84156e50 2023-12-26T23:12:37.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.660+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 108) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f1f7c007300 con 0x7f1f84156e50 2023-12-26T23:12:37.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.660+0000 7f1f79ffb700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f68069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:37.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.661+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f1f7c084a10 con 0x7f1f84156e50 2023-12-26T23:12:37.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.661+0000 7f1f89b9b700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 msgr2=0x7f1f68069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:37.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.661+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f68069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:37.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.664+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f7c051af0 con 0x7f1f84156e50 2023-12-26T23:12:37.673 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:37.673+0000 7f0bc74851c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:12:37.771 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:37.771+0000 7fa30a5231c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:37.825 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:37.824+0000 7f0bc74851c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.862+0000 7f1f89b9b700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 msgr2=0x7f1f68069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:37.862 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.862+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f68069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:37.862 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:37.862+0000 7fa30a5231c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:12:37.890 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:37.890+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f84157b20 con 0x7f1f68066f80 2023-12-26T23:12:37.918 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:37.918+0000 7f0bc74851c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:12:38.044 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:38.044+0000 7fa30a5231c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:12:38.101 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:38.100+0000 7f0bc74851c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:12:38.193 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:38.193+0000 7fa30a5231c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:12:38.250 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:38.250+0000 7f0bc74851c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:12:38.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:38.263+0000 7f1f89b9b700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 msgr2=0x7f1f68069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:38.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:38.263+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f68069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T23:12:38.573 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:38.574+0000 7fa30a5231c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:12:38.637 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:38.637+0000 7f0bc74851c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:12:38.843 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:38.844+0000 7fa30a5231c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:12:38.904 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:38.904+0000 7f0bc74851c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:12:39.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:39.064+0000 7f1f89b9b700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 msgr2=0x7f1f68069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:39.064 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:39.064+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f68069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-26T23:12:39.123 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:39.124+0000 7fa30a5231c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:12:39.185 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:39.185+0000 7f0bc74851c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:12:39.213 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:39.214+0000 7fa30a5231c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:12:39.274 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:39.274+0000 7f0bc74851c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:12:39.949 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:39.949+0000 7fa30a5231c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:12:40.017 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:40.017+0000 7f0bc74851c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:12:40.043 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:40.044+0000 7fa30a5231c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:12:40.112 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:40.113+0000 7f0bc74851c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:12:40.197 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:40.197+0000 7fa30a5231c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:12:40.266 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:40.267+0000 7f0bc74851c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:12:40.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:40.667+0000 7f1f89b9b700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 msgr2=0x7f1f68069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:40.666 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:40.667+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f68069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-26T23:12:40.822 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:40.823+0000 7fa30a5231c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:12:40.898 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:40.899+0000 7f0bc74851c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:12:41.014 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:41.014+0000 7fa30a5231c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:12:41.088 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:41.089+0000 7f0bc74851c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:12:41.205 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:41.205+0000 7fa30a5231c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:12:41.279 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:41.280+0000 7f0bc74851c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:12:42.052 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:42.052+0000 7fa30a5231c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:12:42.128 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.129+0000 7f0bc74851c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:12:42.328 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:42.328+0000 7fa30a5231c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:42.407 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.408+0000 7f0bc74851c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:42.421 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:42.421+0000 7fa30a5231c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:42.502 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.503+0000 7f0bc74851c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:42.619 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:42.619+0000 7fa30a5231c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:12:42.677 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:42.678+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 109) v1 ==== 84759+0+0 (secure 0 0 0) 0x7f1f7c051ec0 con 0x7f1f84156e50 2023-12-26T23:12:42.704 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.705+0000 7f0bc74851c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:12:42.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:42.716+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 110) v1 ==== 84470+0+0 (secure 0 0 0) 0x7f1f7c055a10 con 0x7f1f84156e50 2023-12-26T23:12:42.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:42.717+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 msgr2=0x7f1f68069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:42.716 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:42.717+0000 7f1f79ffb700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f68069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:42.845 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b3bdb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:42.845 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b3bdb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:42.845 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b3bdb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:42.845 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b3bdb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:42.845 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b3bdb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:42.845 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b38db1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:42.845 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b38db1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:42.845 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b38db1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:42.846 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b38db1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:42.846 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:42.845+0000 7f0b38db1700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-26T23:12:43.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.718+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mon.0 v2:172.21.15.131:3300/0 9 ==== mgrmap(e 111) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f1f7c052790 con 0x7f1f84156e50 2023-12-26T23:12:43.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.718+0000 7f1f79ffb700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f680693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:43.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.718+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f68032aa0 con 0x7f1f68066f80 2023-12-26T23:12:43.722 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.721+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f680693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:43.724 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.725+0000 7f1f89b9b700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f680693b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009bd0 tx=0x7f1f800091a0 comp rx=0 tx=0).ready entity=mgr.9757 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.792+0000 7f1f79ffb700 1 -- 172.21.15.131:0/3577669943 <== mgr.9757 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f68032aa0 con 0x7f1f68066f80 2023-12-26T23:12:43.791 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule set for path snap_test_dir1 2023-12-26T23:12:43.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 msgr2=0x7f1f680693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:43.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f680693b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009bd0 tx=0x7f1f800091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:43.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f84156e50 msgr2=0x7f1f841562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f84156e50 0x7f1f841562b0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00cbf0 tx=0x7f1f7c0019b0 comp rx=0 tx=0).stop 2023-12-26T23:12:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 shutdown_connections 2023-12-26T23:12:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1f68066f80 0x7f1f680693b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f84156e50 0x7f1f841562b0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f840c8d40 0x7f1f84145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 --2- 172.21.15.131:0/3577669943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f840c9110 0x7f1f840c07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 >> 172.21.15.131:0/3577669943 conn(0x7f1f840675b0 msgr2=0x7f1f8406f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.794+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 shutdown_connections 2023-12-26T23:12:43.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:43.795+0000 7f1f8bdff700 1 -- 172.21.15.131:0/3577669943 wait complete. 2023-12-26T23:12:43.813 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-26T23:12:43.813 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-12-26T23:12:44.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.010+0000 7f96b69ff700 1 Processor -- start 2023-12-26T23:12:44.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.010+0000 7f96b69ff700 1 -- start start 2023-12-26T23:12:44.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.011+0000 7f96b69ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96b00c8820 0x7f96b00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.011+0000 7f96b69ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b00c8bf0 0x7f96b00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.011+0000 7f96b69ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b0157a80 con 0x7f96b0069d10 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.011+0000 7f96b69ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b00cac50 con 0x7f96b00c8820 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.011+0000 7f96b69ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00caf30 con 0x7f96b00c8bf0 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b00c8bf0 0x7f96b00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96affff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96b00c8820 0x7f96b00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b00c8bf0 0x7f96b00b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:54156/0 (socket says 172.21.15.131:54156) 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96affff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96b00c8820 0x7f96b00ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:48182/0 (socket says 172.21.15.131:48182) 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96af7fe700 1 -- 172.21.15.131:0/3274989541 learned_addr learned my addr 172.21.15.131:0/3274989541 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96b4f9c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f96b0069d10 0x7f96b0157f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:48902/0 (socket says 172.21.15.131:48902) 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96aeffd700 1 -- 172.21.15.131:0/3274989541 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057221248 0 0) 0x7f96b0157a80 con 0x7f96b0069d10 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96aeffd700 1 -- 172.21.15.131:0/3274989541 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9698003570 con 0x7f96b0069d10 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96affff700 1 -- 172.21.15.131:0/3274989541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b00c8bf0 msgr2=0x7f96b00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96affff700 1 --2- 172.21.15.131:0/3274989541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b00c8bf0 0x7f96b00b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96affff700 1 -- 172.21.15.131:0/3274989541 >> v1:172.21.15.190:6789/0 conn(0x7f96b0069d10 legacy=0x7f96b0157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96affff700 1 -- 172.21.15.131:0/3274989541 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00cb210 con 0x7f96b00c8820 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.012+0000 7f96af7fe700 1 --2- 172.21.15.131:0/3274989541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b00c8bf0 0x7f96b00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.013+0000 7f96affff700 1 --2- 172.21.15.131:0/3274989541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96b00c8820 0x7f96b00ca300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f96a00098e0 tx=0x7f96a00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c53b1b5642e5fb server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.013+0000 7f96aeffd700 1 -- 172.21.15.131:0/3274989541 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0011630 con 0x7f96b00c8820 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.013+0000 7f96aeffd700 1 -- 172.21.15.131:0/3274989541 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a0011790 con 0x7f96b00c8820 2023-12-26T23:12:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.013+0000 7f96aeffd700 1 -- 172.21.15.131:0/3274989541 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0011b30 con 0x7f96b00c8820 2023-12-26T23:12:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.014+0000 7f96b69ff700 1 -- 172.21.15.131:0/3274989541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96b00c8820 msgr2=0x7f96b00ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.014+0000 7f96b69ff700 1 --2- 172.21.15.131:0/3274989541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96b00c8820 0x7f96b00ca300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f96a00098e0 tx=0x7f96a00091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.014+0000 7f96b69ff700 1 -- 172.21.15.131:0/3274989541 shutdown_connections 2023-12-26T23:12:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.014+0000 7f96b69ff700 1 --2- 172.21.15.131:0/3274989541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96b00c8820 0x7f96b00ca300 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.014+0000 7f96b69ff700 1 --2- 172.21.15.131:0/3274989541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b00c8bf0 0x7f96b00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.014+0000 7f96b69ff700 1 -- 172.21.15.131:0/3274989541 >> 172.21.15.131:0/3274989541 conn(0x7f96b0068040 msgr2=0x7f96b0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.014+0000 7f96b69ff700 1 -- 172.21.15.131:0/3274989541 shutdown_connections 2023-12-26T23:12:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.014+0000 7f96b69ff700 1 -- 172.21.15.131:0/3274989541 wait complete. 2023-12-26T23:12:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.015+0000 7f96b69ff700 1 Processor -- start 2023-12-26T23:12:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.015+0000 7f96b69ff700 1 -- start start 2023-12-26T23:12:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.016+0000 7f96b69ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b0069d10 0x7f96b014a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.016+0000 7f96b69ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96b00c8820 0x7f96b0150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.016+0000 7f96b69ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96b00c8bf0 0x7f96b0151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.016+0000 7f96b69ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0167730 con 0x7f96b00c8820 2023-12-26T23:12:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.016+0000 7f96b69ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b014f000 con 0x7f96b00c8bf0 2023-12-26T23:12:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.016+0000 7f96b69ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b014f2e0 con 0x7f96b0069d10 2023-12-26T23:12:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.016+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96b00c8820 0x7f96b0150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.016+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96b00c8820 0x7f96b0150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53884/0 (socket says 172.21.15.131:53884) 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.016+0000 7f96af7fe700 1 -- 172.21.15.131:0/98468608 learned_addr learned my addr 172.21.15.131:0/98468608 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f96affff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b0069d10 0x7f96b014a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f96b4f9c700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96b00c8bf0 0x7f96b0151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f96af7fe700 1 -- 172.21.15.131:0/98468608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b0069d10 msgr2=0x7f96b014a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f96af7fe700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b0069d10 0x7f96b014a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f96af7fe700 1 -- 172.21.15.131:0/98468608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96b00c8bf0 msgr2=0x7f96b0151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f96af7fe700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96b00c8bf0 0x7f96b0151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f96af7fe700 1 -- 172.21.15.131:0/98468608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a40094f0 con 0x7f96b00c8820 2023-12-26T23:12:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f96af7fe700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96b00c8820 0x7f96b0150bd0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f96a400ec50 tx=0x7f96a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f9697fff700 1 -- 172.21.15.131:0/98468608 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400c850 con 0x7f96b00c8820 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.017+0000 7f9697fff700 1 -- 172.21.15.131:0/98468608 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a400c9b0 con 0x7f96b00c8820 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.018+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a0009420 con 0x7f96b00c8820 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.018+0000 7f9697fff700 1 -- 172.21.15.131:0/98468608 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400cd50 con 0x7f96b00c8820 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.018+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b014f8a0 con 0x7f96b00c8820 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.019+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b00657e0 con 0x7f96b00c8820 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.020+0000 7f9697fff700 1 -- 172.21.15.131:0/98468608 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 111) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f96a4013680 con 0x7f96b00c8820 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.020+0000 7f9697fff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f969c066dd0 0x7f969c069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.021+0000 7f9697fff700 1 -- 172.21.15.131:0/98468608 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f96a40845d0 con 0x7f96b00c8820 2023-12-26T23:12:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.021+0000 7f96affff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f969c066dd0 0x7f969c069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.021+0000 7f96affff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f969c066dd0 0x7f969c069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f96a00091e0 tx=0x7f96a0017000 comp rx=0 tx=0).ready entity=mgr.9757 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:44.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.025+0000 7f9697fff700 1 -- 172.21.15.131:0/98468608 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96a40517e0 con 0x7f96b00c8820 2023-12-26T23:12:44.243 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.244+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f96b0167870 con 0x7f969c066dd0 2023-12-26T23:12:44.304 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.304+0000 7f9697fff700 1 -- 172.21.15.131:0/98468608 <== mgr.9757 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f96b0167870 con 0x7f969c066dd0 2023-12-26T23:12:44.304 INFO:teuthology.orchestra.run.smithi131.stdout:Retention added to path snap_test_dir1 2023-12-26T23:12:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f969c066dd0 msgr2=0x7f969c069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f969c066dd0 0x7f969c069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f96a00091e0 tx=0x7f96a0017000 comp rx=0 tx=0).stop 2023-12-26T23:12:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96b00c8820 msgr2=0x7f96b0150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96b00c8820 0x7f96b0150bd0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f96a400ec50 tx=0x7f96a4009250 comp rx=0 tx=0).stop 2023-12-26T23:12:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 shutdown_connections 2023-12-26T23:12:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f969c066dd0 0x7f969c069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96b00c8bf0 0x7f96b0151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96b0069d10 0x7f96b014a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 --2- 172.21.15.131:0/98468608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96b00c8820 0x7f96b0150bd0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 >> 172.21.15.131:0/98468608 conn(0x7f96b0068040 msgr2=0x7f96b006d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 shutdown_connections 2023-12-26T23:12:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.307+0000 7f96b69ff700 1 -- 172.21.15.131:0/98468608 wait complete. 2023-12-26T23:12:44.325 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-12-26T23:12:44.331 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-26-23_13_00 in ~16s... 2023-12-26T23:12:44.331 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-12-26T23:12:44.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-12-26T23:12:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.526+0000 7f13e9dff700 1 Processor -- start 2023-12-26T23:12:44.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.527+0000 7f13e9dff700 1 -- start start 2023-12-26T23:12:44.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.528+0000 7f13e9dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f13e4156360 0x7f13e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.528+0000 7f13e9dff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f13e4156730 0x7f13e406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.528+0000 7f13e9dff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e40acfc0 con 0x7f13e40c8490 2023-12-26T23:12:44.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.528+0000 7f13e9dff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e40c9000 con 0x7f13e4156730 2023-12-26T23:12:44.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.528+0000 7f13e9dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e406f1f0 con 0x7f13e4156360 2023-12-26T23:12:44.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.528+0000 7f13e2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f13e4156730 0x7f13e406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.528+0000 7f13e2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f13e4156730 0x7f13e406ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53890/0 (socket says 172.21.15.131:53890) 2023-12-26T23:12:44.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.528+0000 7f13e2ffd700 1 -- 172.21.15.131:0/2050554311 learned_addr learned my addr 172.21.15.131:0/2050554311 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:44.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.528+0000 7f13e37fe700 1 --2- 172.21.15.131:0/2050554311 >> v2:172.21.15.190:3300/0 conn(0x7f13e4156360 0x7f13e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e27fc700 1 -- 172.21.15.131:0/2050554311 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908301435 0 0) 0x7f13e40acfc0 con 0x7f13e40c8490 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e27fc700 1 -- 172.21.15.131:0/2050554311 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13cc003570 con 0x7f13e40c8490 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e2ffd700 1 -- 172.21.15.131:0/2050554311 >> v2:172.21.15.190:3300/0 conn(0x7f13e4156360 msgr2=0x7f13e406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e2ffd700 1 --2- 172.21.15.131:0/2050554311 >> v2:172.21.15.190:3300/0 conn(0x7f13e4156360 0x7f13e406e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e2ffd700 1 -- 172.21.15.131:0/2050554311 >> v1:172.21.15.131:6789/0 conn(0x7f13e40c8490 legacy=0x7f13e40c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e2ffd700 1 -- 172.21.15.131:0/2050554311 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e40c91f0 con 0x7f13e4156730 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e2ffd700 1 --2- 172.21.15.131:0/2050554311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 0x7f13e406ecb0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f13e4069410 tx=0x7f13d800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f6ae5a6c8709bd server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e27fc700 1 -- 172.21.15.131:0/2050554311 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800db40 con 0x7f13e4156730 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e27fc700 1 -- 172.21.15.131:0/2050554311 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d800dca0 con 0x7f13e4156730 2023-12-26T23:12:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.529+0000 7f13e27fc700 1 -- 172.21.15.131:0/2050554311 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8019690 con 0x7f13e4156730 2023-12-26T23:12:44.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.530+0000 7f13e9dff700 1 -- 172.21.15.131:0/2050554311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 msgr2=0x7f13e406ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.530+0000 7f13e9dff700 1 --2- 172.21.15.131:0/2050554311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 0x7f13e406ecb0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f13e4069410 tx=0x7f13d800b4f0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.530+0000 7f13e9dff700 1 -- 172.21.15.131:0/2050554311 shutdown_connections 2023-12-26T23:12:44.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.530+0000 7f13e9dff700 1 --2- 172.21.15.131:0/2050554311 >> v2:172.21.15.190:3300/0 conn(0x7f13e4156360 0x7f13e406e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.530+0000 7f13e9dff700 1 --2- 172.21.15.131:0/2050554311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 0x7f13e406ecb0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.530+0000 7f13e9dff700 1 -- 172.21.15.131:0/2050554311 >> 172.21.15.131:0/2050554311 conn(0x7f13e40675b0 msgr2=0x7f13e40679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:44.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.530+0000 7f13e9dff700 1 -- 172.21.15.131:0/2050554311 shutdown_connections 2023-12-26T23:12:44.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.530+0000 7f13e9dff700 1 -- 172.21.15.131:0/2050554311 wait complete. 2023-12-26T23:12:44.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.531+0000 7f13e9dff700 1 Processor -- start 2023-12-26T23:12:44.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.532+0000 7f13e9dff700 1 -- start start 2023-12-26T23:12:44.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.532+0000 7f13e9dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40c8490 0x7f13e40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.532+0000 7f13e9dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13e4156360 0x7f13e4154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e9dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 0x7f13e40b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e9dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e416db10 con 0x7f13e4156730 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e9dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e4154590 con 0x7f13e40c8490 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e9dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e416cf00 con 0x7f13e4156360 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 0x7f13e40b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 0x7f13e40b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53898/0 (socket says 172.21.15.131:53898) 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e3fff700 1 -- 172.21.15.131:0/2712475550 learned_addr learned my addr 172.21.15.131:0/2712475550 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e37fe700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40c8490 0x7f13e40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e2ffd700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13e4156360 0x7f13e4154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e3fff700 1 -- 172.21.15.131:0/2712475550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13e4156360 msgr2=0x7f13e4154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e3fff700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13e4156360 0x7f13e4154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e3fff700 1 -- 172.21.15.131:0/2712475550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40c8490 msgr2=0x7f13e40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e3fff700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40c8490 0x7f13e40c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.533+0000 7f13e3fff700 1 -- 172.21.15.131:0/2712475550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d800b790 con 0x7f13e4156730 2023-12-26T23:12:44.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.534+0000 7f13e3fff700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 0x7f13e40b13d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00c910 tx=0x7f13dc0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:44.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.534+0000 7f13cb7fe700 1 -- 172.21.15.131:0/2712475550 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc012630 con 0x7f13e4156730 2023-12-26T23:12:44.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.534+0000 7f13cb7fe700 1 -- 172.21.15.131:0/2712475550 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc0127b0 con 0x7f13e4156730 2023-12-26T23:12:44.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.534+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e416d1e0 con 0x7f13e4156730 2023-12-26T23:12:44.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.534+0000 7f13cb7fe700 1 -- 172.21.15.131:0/2712475550 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc012b70 con 0x7f13e4156730 2023-12-26T23:12:44.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.534+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e416d7a0 con 0x7f13e4156730 2023-12-26T23:12:44.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.535+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e406a470 con 0x7f13e4156730 2023-12-26T23:12:44.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.536+0000 7f13cb7fe700 1 -- 172.21.15.131:0/2712475550 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 111) v1 ==== 84597+0+0 (secure 0 0 0) 0x7f13dc01b400 con 0x7f13e4156730 2023-12-26T23:12:44.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.537+0000 7f13cb7fe700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f13d0066dd0 0x7f13d0069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.537+0000 7f13cb7fe700 1 -- 172.21.15.131:0/2712475550 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f13dc084860 con 0x7f13e4156730 2023-12-26T23:12:44.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.537+0000 7f13e37fe700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f13d0066dd0 0x7f13d0069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.537+0000 7f13e37fe700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f13d0066dd0 0x7f13d0069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009db0 tx=0x7f13d40091a0 comp rx=0 tx=0).ready entity=mgr.9757 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:44.540 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.541+0000 7f13cb7fe700 1 -- 172.21.15.131:0/2712475550 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13dc051a70 con 0x7f13e4156730 2023-12-26T23:12:44.721 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.721+0000 7f13cb7fe700 1 -- 172.21.15.131:0/2712475550 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 112) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f13dc051e40 con 0x7f13e4156730 2023-12-26T23:12:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.762+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13e40717f0 con 0x7f13d0066dd0 2023-12-26T23:12:44.770 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.771+0000 7f13cb7fe700 1 -- 172.21.15.131:0/2712475550 <== mgr.9757 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f13e40717f0 con 0x7f13d0066dd0 2023-12-26T23:12:44.770 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:44.770 INFO:teuthology.orchestra.run.smithi131.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-26T23:12:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f13d0066dd0 msgr2=0x7f13d0069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f13d0066dd0 0x7f13d0069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009db0 tx=0x7f13d40091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 msgr2=0x7f13e40b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 0x7f13e40b13d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00c910 tx=0x7f13dc0027a0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 shutdown_connections 2023-12-26T23:12:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f13d0066dd0 0x7f13d0069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13e4156730 0x7f13e40b13d0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13e40c8490 0x7f13e40c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 --2- 172.21.15.131:0/2712475550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13e4156360 0x7f13e4154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 >> 172.21.15.131:0/2712475550 conn(0x7f13e40675b0 msgr2=0x7f13e40cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.773+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 shutdown_connections 2023-12-26T23:12:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.774+0000 7f13e9dff700 1 -- 172.21.15.131:0/2712475550 wait complete. 2023-12-26T23:12:44.790 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-26T23:12:44.791 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-12-26T23:12:44.791 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-12-26T23:12:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.986+0000 7f24525ff700 1 Processor -- start 2023-12-26T23:12:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.986+0000 7f24525ff700 1 -- start start 2023-12-26T23:12:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.987+0000 7f24525ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f244c0d06e0 0x7f244c159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.987+0000 7f24525ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0d0ab0 0x7f244c159650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.987+0000 7f24525ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244c15d5d0 con 0x7f244c0ceb70 2023-12-26T23:12:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.987+0000 7f24525ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c159b90 con 0x7f244c0d06e0 2023-12-26T23:12:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.987+0000 7f24525ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0b3dd0 con 0x7f244c0d0ab0 2023-12-26T23:12:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f2450b9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f244c0ceb70 0x7f244c0ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59680/0 (socket says 172.21.15.131:59680) 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f2450b9c700 1 -- 172.21.15.131:0/2506746973 learned_addr learned my addr 172.21.15.131:0/2506746973 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f244b7fe700 1 --2- 172.21.15.131:0/2506746973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0d0ab0 0x7f244c159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f244bfff700 1 --2- 172.21.15.131:0/2506746973 >> v2:172.21.15.190:3300/0 conn(0x7f244c0d06e0 0x7f244c159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f244affd700 1 -- 172.21.15.131:0/2506746973 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947056773 0 0) 0x7f244c15d5d0 con 0x7f244c0ceb70 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f244affd700 1 -- 172.21.15.131:0/2506746973 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2434003570 con 0x7f244c0ceb70 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f244bfff700 1 -- 172.21.15.131:0/2506746973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0d0ab0 msgr2=0x7f244c159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f244bfff700 1 --2- 172.21.15.131:0/2506746973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0d0ab0 0x7f244c159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f244bfff700 1 -- 172.21.15.131:0/2506746973 >> v1:172.21.15.131:6789/0 conn(0x7f244c0ceb70 legacy=0x7f244c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f244bfff700 1 -- 172.21.15.131:0/2506746973 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c0d00b0 con 0x7f244c0d06e0 2023-12-26T23:12:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.988+0000 7f244b7fe700 1 --2- 172.21.15.131:0/2506746973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0d0ab0 0x7f244c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.989+0000 7f244bfff700 1 --2- 172.21.15.131:0/2506746973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f244c0d06e0 0x7f244c159110 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f243c009910 tx=0x7f243c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa19b3dd727fc05 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.989+0000 7f244affd700 1 -- 172.21.15.131:0/2506746973 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c011630 con 0x7f244c0d06e0 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.989+0000 7f244affd700 1 -- 172.21.15.131:0/2506746973 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c011790 con 0x7f244c0d06e0 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.989+0000 7f244affd700 1 -- 172.21.15.131:0/2506746973 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c011b30 con 0x7f244c0d06e0 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.989+0000 7f24525ff700 1 -- 172.21.15.131:0/2506746973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f244c0d06e0 msgr2=0x7f244c159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.989+0000 7f24525ff700 1 --2- 172.21.15.131:0/2506746973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f244c0d06e0 0x7f244c159110 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f243c009910 tx=0x7f243c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.989+0000 7f24525ff700 1 -- 172.21.15.131:0/2506746973 shutdown_connections 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.990+0000 7f24525ff700 1 --2- 172.21.15.131:0/2506746973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f244c0d06e0 0x7f244c159110 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.990+0000 7f24525ff700 1 --2- 172.21.15.131:0/2506746973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0d0ab0 0x7f244c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.990+0000 7f24525ff700 1 -- 172.21.15.131:0/2506746973 >> 172.21.15.131:0/2506746973 conn(0x7f244c06ece0 msgr2=0x7f244c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.990+0000 7f24525ff700 1 -- 172.21.15.131:0/2506746973 shutdown_connections 2023-12-26T23:12:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.990+0000 7f24525ff700 1 -- 172.21.15.131:0/2506746973 wait complete. 2023-12-26T23:12:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.992+0000 7f24525ff700 1 Processor -- start 2023-12-26T23:12:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.992+0000 7f24525ff700 1 -- start start 2023-12-26T23:12:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f24525ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0ceb70 0x7f244c169860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f24525ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f244c0d06e0 0x7f244c1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f24525ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c0d0ab0 0x7f244c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f24525ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c15eeb0 con 0x7f244c0d0ab0 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f24525ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c15e2a0 con 0x7f244c0d06e0 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f24525ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c15e580 con 0x7f244c0ceb70 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f2450b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c0d0ab0 0x7f244c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f2450b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c0d0ab0 0x7f244c0c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34852/0 (socket says 172.21.15.131:34852) 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f2450b9c700 1 -- 172.21.15.131:0/1324170096 learned_addr learned my addr 172.21.15.131:0/1324170096 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f244bfff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0ceb70 0x7f244c169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.993+0000 7f244b7fe700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f244c0d06e0 0x7f244c1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f2450b9c700 1 -- 172.21.15.131:0/1324170096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0ceb70 msgr2=0x7f244c169860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f2450b9c700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0ceb70 0x7f244c169860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f2450b9c700 1 -- 172.21.15.131:0/1324170096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f244c0d06e0 msgr2=0x7f244c1540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f2450b9c700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f244c0d06e0 0x7f244c1540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f2450b9c700 1 -- 172.21.15.131:0/1324170096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2440009420 con 0x7f244c0d0ab0 2023-12-26T23:12:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f2450b9c700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c0d0ab0 0x7f244c0c5d50 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f2444001460 tx=0x7f2444001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f2433fff700 1 -- 172.21.15.131:0/1324170096 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244400a890 con 0x7f244c0d0ab0 2023-12-26T23:12:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f2433fff700 1 -- 172.21.15.131:0/1324170096 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244400a9f0 con 0x7f244c0d0ab0 2023-12-26T23:12:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c009420 con 0x7f244c0d0ab0 2023-12-26T23:12:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f2433fff700 1 -- 172.21.15.131:0/1324170096 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244400ad70 con 0x7f244c0d0ab0 2023-12-26T23:12:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.994+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c15eb40 con 0x7f244c0d0ab0 2023-12-26T23:12:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.995+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c15f060 con 0x7f244c0d0ab0 2023-12-26T23:12:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.996+0000 7f2433fff700 1 -- 172.21.15.131:0/1324170096 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 112) v1 ==== 84918+0+0 (secure 0 0 0) 0x7f2444007300 con 0x7f244c0d0ab0 2023-12-26T23:12:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.997+0000 7f2433fff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2438067130 0x7f24380695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.997+0000 7f2433fff700 1 -- 172.21.15.131:0/1324170096 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f2444084f60 con 0x7f244c0d0ab0 2023-12-26T23:12:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.997+0000 7f244bfff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2438067130 0x7f24380695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:44.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:44.998+0000 7f244bfff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2438067130 0x7f24380695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f243c003080 tx=0x7f243c017000 comp rx=0 tx=0).ready entity=mgr.9757 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:45.000 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.001+0000 7f2433fff700 1 -- 172.21.15.131:0/1324170096 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f244c15f060 con 0x7f244c0d0ab0 2023-12-26T23:12:45.217 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.219+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f244c0717f0 con 0x7f2438067130 2023-12-26T23:12:45.299 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.300+0000 7f2433fff700 1 -- 172.21.15.131:0/1324170096 <== mgr.9757 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f244c0717f0 con 0x7f2438067130 2023-12-26T23:12:45.299 INFO:teuthology.orchestra.run.smithi131.stdout:Schedule removed for path snap_test_dir1 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2438067130 msgr2=0x7f24380695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2438067130 0x7f24380695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f243c003080 tx=0x7f243c017000 comp rx=0 tx=0).stop 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c0d0ab0 msgr2=0x7f244c0c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c0d0ab0 0x7f244c0c5d50 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f2444001460 tx=0x7f2444001ae0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 shutdown_connections 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f2438067130 0x7f24380695e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f244c0d0ab0 0x7f244c0c5d50 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f244c0ceb70 0x7f244c169860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 --2- 172.21.15.131:0/1324170096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f244c0d06e0 0x7f244c1540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.302 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 >> 172.21.15.131:0/1324170096 conn(0x7f244c06ece0 msgr2=0x7f244c15c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:45.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 shutdown_connections 2023-12-26T23:12:45.303 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.302+0000 7f24525ff700 1 -- 172.21.15.131:0/1324170096 wait complete. 2023-12-26T23:12:45.320 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-26T23:12:45.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:12:45.321 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-26T23:12:45.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:12:45.367 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-26T23:12:45.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-26T23:12:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.648+0000 7fbb7b9ff700 1 Processor -- start 2023-12-26T23:12:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.648+0000 7fbb7b9ff700 1 -- start start 2023-12-26T23:12:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.649+0000 7fbb7b9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb740b07f0 0x7fbb741554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.649+0000 7fbb7b9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740c8bf0 con 0x7fbb74159cf0 2023-12-26T23:12:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.650+0000 7fbb7b9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740c8e60 con 0x7fbb74159920 2023-12-26T23:12:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.650+0000 7fbb7b9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740c9140 con 0x7fbb740b07f0 2023-12-26T23:12:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.650+0000 7fbb79f9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbb74159cf0 0x7fbb74155a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59688/0 (socket says 172.21.15.131:59688) 2023-12-26T23:12:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.650+0000 7fbb79f9c700 1 -- 172.21.15.131:0/1469657386 learned_addr learned my addr 172.21.15.131:0/1469657386 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.650+0000 7fbb78f9a700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb740b07f0 0x7fbb741554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.650+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205285969 0 0) 0x7fbb740c8bf0 con 0x7fbb74159cf0 2023-12-26T23:12:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.650+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb60003570 con 0x7fbb74159cf0 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.650+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2227234430 0 0) 0x7fbb60003570 con 0x7fbb74159cf0 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb740b07f0 msgr2=0x7fbb741554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb740b07f0 0x7fbb741554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 >> v1:172.21.15.190:6789/0 conn(0x7fbb74159920 legacy=0x7fbb740b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740c9420 con 0x7fbb74159cf0 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fbb6c0017f0 con 0x7fbb74159cf0 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 >> v1:172.21.15.131:6789/0 conn(0x7fbb74159cf0 legacy=0x7fbb74155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60003a90 0x7fbb60003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb60004810 0x7fbb60004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb60005180 0x7fbb6000cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740c9140 con 0x7fbb60005180 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6000d0b0 con 0x7fbb60003a90 2023-12-26T23:12:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6000d1f0 con 0x7fbb60004810 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb79f9c700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb60005180 0x7fbb6000cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb78f9a700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb60004810 0x7fbb60004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb7979b700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60003a90 0x7fbb60003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb79f9c700 1 -- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb60004810 msgr2=0x7fbb60004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb79f9c700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb60004810 0x7fbb60004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb79f9c700 1 -- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60003a90 msgr2=0x7fbb60003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb79f9c700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60003a90 0x7fbb60003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.651+0000 7fbb79f9c700 1 -- 172.21.15.131:0/1469657386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60003570 con 0x7fbb60005180 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb79f9c700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb60005180 0x7fbb6000cb70 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00a400 tx=0x7fbb6c0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c2b9d04d60c96a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0104a0 con 0x7fbb60005180 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c010600 con 0x7fbb60005180 2023-12-26T23:12:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb6bfff700 1 -- 172.21.15.131:0/1469657386 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0109c0 con 0x7fbb60005180 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/1469657386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb60005180 msgr2=0x7fbb6000cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb7b9ff700 1 --2- 172.21.15.131:0/1469657386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb60005180 0x7fbb6000cb70 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00a400 tx=0x7fbb6c0021f0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb79f9c700 1 -- 172.21.15.131:0/1469657386 reap_dead start 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/1469657386 shutdown_connections 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/1469657386 >> 172.21.15.131:0/1469657386 conn(0x7fbb74068040 msgr2=0x7fbb74069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.652+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/1469657386 shutdown_connections 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.653+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/1469657386 wait complete. 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.654+0000 7fbb7b9ff700 1 Processor -- start 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.654+0000 7fbb7b9ff700 1 -- start start 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.655+0000 7fbb7b9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60005180 0x7fbb74158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.655+0000 7fbb7b9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb740a6200 0x7fbb740a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.655+0000 7fbb7b9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb740a7740 0x7fbb740a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.655+0000 7fbb7b9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740a72b0 con 0x7fbb740a6200 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.655+0000 7fbb7b9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740a66a0 con 0x7fbb60005180 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.655+0000 7fbb7b9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740a6980 con 0x7fbb740a7740 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.655+0000 7fbb78f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb740a6200 0x7fbb740a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.655+0000 7fbb78f9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb740a6200 0x7fbb740a8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34874/0 (socket says 172.21.15.131:34874) 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.655+0000 7fbb78f9a700 1 -- 172.21.15.131:0/4003752238 learned_addr learned my addr 172.21.15.131:0/4003752238 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb7979b700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60005180 0x7fbb74158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb79f9c700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb740a7740 0x7fbb740a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb78f9a700 1 -- 172.21.15.131:0/4003752238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb740a7740 msgr2=0x7fbb740a7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb78f9a700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb740a7740 0x7fbb740a7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb78f9a700 1 -- 172.21.15.131:0/4003752238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60005180 msgr2=0x7fbb74158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb78f9a700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60005180 0x7fbb74158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb78f9a700 1 -- 172.21.15.131:0/4003752238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740a6c60 con 0x7fbb740a6200 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb7979b700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60005180 0x7fbb74158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:45.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb78f9a700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb740a6200 0x7fbb740a8d60 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fbb64006ac0 tx=0x7fbb640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:45.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb697fa700 1 -- 172.21.15.131:0/4003752238 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400fa10 con 0x7fbb740a6200 2023-12-26T23:12:45.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb697fa700 1 -- 172.21.15.131:0/4003752238 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6400fb70 con 0x7fbb740a6200 2023-12-26T23:12:45.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb740a6f40 con 0x7fbb740a6200 2023-12-26T23:12:45.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb697fa700 1 -- 172.21.15.131:0/4003752238 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64014440 con 0x7fbb740a6200 2023-12-26T23:12:45.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.656+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb740a92a0 con 0x7fbb740a6200 2023-12-26T23:12:45.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.657+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb74158af0 con 0x7fbb740a6200 2023-12-26T23:12:45.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.658+0000 7fbb697fa700 1 -- 172.21.15.131:0/4003752238 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 112) v1 ==== 84918+0+0 (secure 0 0 0) 0x7fbb64014870 con 0x7fbb740a6200 2023-12-26T23:12:45.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.659+0000 7fbb697fa700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbb58067130 0x7fbb580695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:45.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.659+0000 7fbb697fa700 1 -- 172.21.15.131:0/4003752238 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fbb64018030 con 0x7fbb740a6200 2023-12-26T23:12:45.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.659+0000 7fbb7979b700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbb58067130 0x7fbb580695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:45.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.660+0000 7fbb7979b700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbb58067130 0x7fbb580695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb70003c10 tx=0x7fbb70009040 comp rx=0 tx=0).ready entity=mgr.9757 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:45.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.663+0000 7fbb697fa700 1 -- 172.21.15.131:0/4003752238 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb640523b0 con 0x7fbb740a6200 2023-12-26T23:12:45.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:45.911+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fbb740657e0 con 0x7fbb740a6200 2023-12-26T23:12:46.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:12:46.166 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:12:46.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.742+0000 7fbb697fa700 1 -- 172.21.15.131:0/4003752238 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fbb64052780 con 0x7fbb740a6200 2023-12-26T23:12:46.742 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.743+0000 7fbb697fa700 1 -- 172.21.15.131:0/4003752238 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v113) v1 ==== 93+0+0 (secure 0 0 0) 0x7fbb640523b0 con 0x7fbb740a6200 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbb58067130 msgr2=0x7fbb580695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbb58067130 0x7fbb580695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb70003c10 tx=0x7fbb70009040 comp rx=0 tx=0).stop 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb740a6200 msgr2=0x7fbb740a8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb740a6200 0x7fbb740a8d60 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fbb64006ac0 tx=0x7fbb640091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 shutdown_connections 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbb58067130 0x7fbb580695e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb740a7740 0x7fbb740a7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb60005180 0x7fbb74158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 --2- 172.21.15.131:0/4003752238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb740a6200 0x7fbb740a8d60 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.746+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 >> 172.21.15.131:0/4003752238 conn(0x7fbb74068040 msgr2=0x7fbb740c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.747+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 shutdown_connections 2023-12-26T23:12:46.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:46.747+0000 7fbb7b9ff700 1 -- 172.21.15.131:0/4003752238 wait complete. 2023-12-26T23:12:46.765 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-26T23:12:46.765 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:12:46.874 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-26T23:12:46.969 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-26T23:12:46.969 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-26T23:12:46.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-12-26T23:12:46.969 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-26T23:12:47.050 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T23:12:47.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:47.050 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-26T23:12:47.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:47.186 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:47.187+0000 7f298870d1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:12:47.207 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:47.209+0000 7f087f3921c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-26T23:12:47.291 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:47.292+0000 7f298870d1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:12:47.315 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:47.316+0000 7f087f3921c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-26T23:12:47.332 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.333+0000 7fa0ad9ff700 1 Processor -- start 2023-12-26T23:12:47.333 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.334+0000 7fa0ad9ff700 1 -- start start 2023-12-26T23:12:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.336+0000 7fa0ad9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0a80b07f0 0x7fa0a81554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.336+0000 7fa0ad9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a80c8bf0 con 0x7fa0a8159cf0 2023-12-26T23:12:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.336+0000 7fa0ad9ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a80c8e60 con 0x7fa0a8159920 2023-12-26T23:12:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.336+0000 7fa0ad9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a80c9140 con 0x7fa0a80b07f0 2023-12-26T23:12:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.336+0000 7fa0a67fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0a80b07f0 0x7fa0a81554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.336+0000 7fa0a77fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa0a8159cf0 0x7fa0a8155a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59690/0 (socket says 172.21.15.131:59690) 2023-12-26T23:12:47.335 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.336+0000 7fa0a77fe700 1 -- 172.21.15.131:0/2720622740 learned_addr learned my addr 172.21.15.131:0/2720622740 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.337+0000 7fa0a67fc700 1 -- 172.21.15.131:0/2720622740 >> v1:172.21.15.190:6789/0 conn(0x7fa0a8159920 legacy=0x7fa0a80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.337+0000 7fa0a67fc700 1 -- 172.21.15.131:0/2720622740 >> v1:172.21.15.131:6789/0 conn(0x7fa0a8159cf0 legacy=0x7fa0a8155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.337+0000 7fa0a67fc700 1 -- 172.21.15.131:0/2720622740 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a80c9420 con 0x7fa0a80b07f0 2023-12-26T23:12:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.337+0000 7fa0a67fc700 1 --2- 172.21.15.131:0/2720622740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a80b07f0 0x7fa0a81554d0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00aa90 tx=0x7fa09c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7bfae12d3b5c56 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.337+0000 7fa0a5ffb700 1 -- 172.21.15.131:0/2720622740 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00ca40 con 0x7fa0a80b07f0 2023-12-26T23:12:47.336 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.337+0000 7fa0a5ffb700 1 -- 172.21.15.131:0/2720622740 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c00cba0 con 0x7fa0a80b07f0 2023-12-26T23:12:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.337+0000 7fa0a5ffb700 1 -- 172.21.15.131:0/2720622740 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c018400 con 0x7fa0a80b07f0 2023-12-26T23:12:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.338+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/2720622740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a80b07f0 msgr2=0x7fa0a81554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.338+0000 7fa0ad9ff700 1 --2- 172.21.15.131:0/2720622740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a80b07f0 0x7fa0a81554d0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00aa90 tx=0x7fa09c00a330 comp rx=0 tx=0).stop 2023-12-26T23:12:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.338+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/2720622740 shutdown_connections 2023-12-26T23:12:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.338+0000 7fa0ad9ff700 1 --2- 172.21.15.131:0/2720622740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a80b07f0 0x7fa0a81554d0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.338+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/2720622740 >> 172.21.15.131:0/2720622740 conn(0x7fa0a8068040 msgr2=0x7fa0a8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:47.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.338+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/2720622740 shutdown_connections 2023-12-26T23:12:47.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.338+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/2720622740 wait complete. 2023-12-26T23:12:47.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.339+0000 7fa0ad9ff700 1 Processor -- start 2023-12-26T23:12:47.338 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.339+0000 7fa0ad9ff700 1 -- start start 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.340+0000 7fa0ad9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a8159920 0x7fa0a815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.340+0000 7fa0ad9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a8159cf0 0x7fa0a8153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.340+0000 7fa0ad9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a806d5c0 0x7fa0a806da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.340+0000 7fa0ad9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a806df60 con 0x7fa0a806d5c0 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.340+0000 7fa0ad9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a815c060 con 0x7fa0a8159cf0 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.340+0000 7fa0ad9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81543c0 con 0x7fa0a8159920 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a806d5c0 0x7fa0a806da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a67fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a8159cf0 0x7fa0a8153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a67fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a8159cf0 0x7fa0a8153810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:49846/0 (socket says 172.21.15.131:49846) 2023-12-26T23:12:47.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a67fc700 1 -- 172.21.15.131:0/1588272163 learned_addr learned my addr 172.21.15.131:0/1588272163 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:47.341 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a6ffd700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a8159920 0x7fa0a815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a6ffd700 1 -- 172.21.15.131:0/1588272163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a8159cf0 msgr2=0x7fa0a8153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a6ffd700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a8159cf0 0x7fa0a8153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a6ffd700 1 -- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a806d5c0 msgr2=0x7fa0a806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a6ffd700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a806d5c0 0x7fa0a806da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a6ffd700 1 -- 172.21.15.131:0/1588272163 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c00a5d0 con 0x7fa0a8159920 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a67fc700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a8159cf0 0x7fa0a8153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.341+0000 7fa0a77fe700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a806d5c0 0x7fa0a806da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.342+0000 7fa0a6ffd700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a8159920 0x7fa0a815a750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c590 tx=0x7fa098007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.342+0000 7fa08effd700 1 -- 172.21.15.131:0/1588272163 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098011590 con 0x7fa0a8159920 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.342+0000 7fa08effd700 1 -- 172.21.15.131:0/1588272163 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098011710 con 0x7fa0a8159920 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.342+0000 7fa08effd700 1 -- 172.21.15.131:0/1588272163 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098011ad0 con 0x7fa0a8159920 2023-12-26T23:12:47.342 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.342+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a815b450 con 0x7fa0a8159920 2023-12-26T23:12:47.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.342+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a815ba10 con 0x7fa0a8159920 2023-12-26T23:12:47.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.343+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a80657e0 con 0x7fa0a8159920 2023-12-26T23:12:47.343 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.344+0000 7fa08effd700 1 -- 172.21.15.131:0/1588272163 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fa09801b3b0 con 0x7fa0a8159920 2023-12-26T23:12:47.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.345+0000 7fa08effd700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa0940670e0 0x7fa094069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.345+0000 7fa0a67fc700 1 -- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa0940670e0 msgr2=0x7fa094069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:47.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.345+0000 7fa08effd700 1 -- 172.21.15.131:0/1588272163 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fa098083f80 con 0x7fa0a8159920 2023-12-26T23:12:47.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.345+0000 7fa0a67fc700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa0940670e0 0x7fa094069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:47.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.349+0000 7fa08effd700 1 -- 172.21.15.131:0/1588272163 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa098050bc0 con 0x7fa0a8159920 2023-12-26T23:12:47.382 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:47.382+0000 7f298870d1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:12:47.409 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:47.411+0000 7f087f3921c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-26T23:12:47.465 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:47.465+0000 7f298870d1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:12:47.498 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:47.495+0000 7f087f3921c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-26T23:12:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.546+0000 7fa0a67fc700 1 -- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa0940670e0 msgr2=0x7fa094069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.546+0000 7fa0a67fc700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa0940670e0 0x7fa094069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:47.593 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.594+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0a80c9700 con 0x7fa0a8159920 2023-12-26T23:12:47.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.595+0000 7fa08effd700 1 -- 172.21.15.131:0/1588272163 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4617 (secure 0 0 0) 0x7fa098017070 con 0x7fa0a8159920 2023-12-26T23:12:47.594 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:47.594 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8372,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2599990460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2599990460},{"type":"v1","addr":"172.21.15.190:6835","nonce":2599990460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":9565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3465421941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3465421941},{"type":"v1","addr":"172.21.15.131:6834","nonce":3465421941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":10122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3595865527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3595865527},{"type":"v1","addr":"172.21.15.131:6835","nonce":3595865527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:12:22.064606+0000","modified":"2023-12-26T23:12:25.606067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10125":{"gid":10125,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/1917956201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1917956201},{"type":"v1","addr":"172.21.15.190:6834","nonce":1917956201}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-12-26T23:12:47.596 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa0940670e0 msgr2=0x7fa094069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa0940670e0 0x7fa094069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a8159920 msgr2=0x7fa0a815a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a8159920 0x7fa0a815a750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c590 tx=0x7fa098007ec0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 shutdown_connections 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fa0940670e0 0x7fa094069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0a806d5c0 0x7fa0a806da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a8159920 0x7fa0a815a750 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 --2- 172.21.15.131:0/1588272163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a8159cf0 0x7fa0a8153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 >> 172.21.15.131:0/1588272163 conn(0x7fa0a8068040 msgr2=0x7fa0a80ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 shutdown_connections 2023-12-26T23:12:47.597 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.597+0000 7fa0ad9ff700 1 -- 172.21.15.131:0/1588272163 wait complete. 2023-12-26T23:12:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 98 2023-12-26T23:12:47.616 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:47.658 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:47.659+0000 7f298870d1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:47.695 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:47.696+0000 7f087f3921c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.810+0000 7fb91cb9d700 1 Processor -- start 2023-12-26T23:12:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.810+0000 7fb91cb9d700 1 -- start start 2023-12-26T23:12:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb91cb9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb918069fe0 0x7fb91815dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb91cb9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb91806a3b0 0x7fb9180cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb91cb9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91815abe0 con 0x7fb9180cb170 2023-12-26T23:12:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb91cb9d700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9180cb590 con 0x7fb91806a3b0 2023-12-26T23:12:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb91cb9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91815e190 con 0x7fb918069fe0 2023-12-26T23:12:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb916ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb91806a3b0 0x7fb9180cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb916ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb91806a3b0 0x7fb9180cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34896/0 (socket says 172.21.15.131:34896) 2023-12-26T23:12:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb916ffd700 1 -- 172.21.15.131:0/2271858220 learned_addr learned my addr 172.21.15.131:0/2271858220 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb9177fe700 1 --2- 172.21.15.131:0/2271858220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb918069fe0 0x7fb91815dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb916ffd700 1 -- 172.21.15.131:0/2271858220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb918069fe0 msgr2=0x7fb91815dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb916ffd700 1 --2- 172.21.15.131:0/2271858220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb918069fe0 0x7fb91815dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb916ffd700 1 -- 172.21.15.131:0/2271858220 >> v1:172.21.15.190:6789/0 conn(0x7fb9180cb170 legacy=0x7fb9181536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb916ffd700 1 -- 172.21.15.131:0/2271858220 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91813c980 con 0x7fb91806a3b0 2023-12-26T23:12:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.812+0000 7fb9177fe700 1 --2- 172.21.15.131:0/2271858220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb918069fe0 0x7fb91815dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.813+0000 7fb916ffd700 1 --2- 172.21.15.131:0/2271858220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 0x7fb9180cac30 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0099b0 tx=0x7fb90c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c4791b40c77e29 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.813+0000 7fb9167fc700 1 -- 172.21.15.131:0/2271858220 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00b860 con 0x7fb91806a3b0 2023-12-26T23:12:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.813+0000 7fb9167fc700 1 -- 172.21.15.131:0/2271858220 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c00b9c0 con 0x7fb91806a3b0 2023-12-26T23:12:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.813+0000 7fb9167fc700 1 -- 172.21.15.131:0/2271858220 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00bd60 con 0x7fb91806a3b0 2023-12-26T23:12:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.814+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2271858220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 msgr2=0x7fb9180cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.814+0000 7fb91cb9d700 1 --2- 172.21.15.131:0/2271858220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 0x7fb9180cac30 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0099b0 tx=0x7fb90c009250 comp rx=0 tx=0).stop 2023-12-26T23:12:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.814+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2271858220 shutdown_connections 2023-12-26T23:12:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.814+0000 7fb91cb9d700 1 --2- 172.21.15.131:0/2271858220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb918069fe0 0x7fb91815dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.814+0000 7fb91cb9d700 1 --2- 172.21.15.131:0/2271858220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 0x7fb9180cac30 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.814+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2271858220 >> 172.21.15.131:0/2271858220 conn(0x7fb91806c150 msgr2=0x7fb918063230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.814+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2271858220 shutdown_connections 2023-12-26T23:12:47.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.814+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2271858220 wait complete. 2023-12-26T23:12:47.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.815+0000 7fb91cb9d700 1 Processor -- start 2023-12-26T23:12:47.815 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.815+0000 7fb91cb9d700 1 -- start start 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb91cb9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb918069fe0 0x7fb9180c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb91cb9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 0x7fb9180c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb91cb9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9180cb170 0x7fb9180c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb91cb9d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180c7920 con 0x7fb91806a3b0 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb91cb9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91813cc30 con 0x7fb918069fe0 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb91cb9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91816df10 con 0x7fb9180cb170 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 0x7fb9180c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb9177fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb918069fe0 0x7fb9180c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb9177fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb918069fe0 0x7fb9180c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:49848/0 (socket says 172.21.15.131:49848) 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.816+0000 7fb9177fe700 1 -- 172.21.15.131:0/2180334620 learned_addr learned my addr 172.21.15.131:0/2180334620 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:47.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb917fff700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9180cb170 0x7fb9180c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb916ffd700 1 -- 172.21.15.131:0/2180334620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9180cb170 msgr2=0x7fb9180c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb916ffd700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9180cb170 0x7fb9180c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb916ffd700 1 -- 172.21.15.131:0/2180334620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb918069fe0 msgr2=0x7fb9180c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb916ffd700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb918069fe0 0x7fb9180c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb916ffd700 1 -- 172.21.15.131:0/2180334620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c0094f0 con 0x7fb91806a3b0 2023-12-26T23:12:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb916ffd700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 0x7fb9180c5970 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb9180c8800 tx=0x7fb90c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb8ff7fe700 1 -- 172.21.15.131:0/2180334620 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c011dc0 con 0x7fb91806a3b0 2023-12-26T23:12:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb8ff7fe700 1 -- 172.21.15.131:0/2180334620 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c01f070 con 0x7fb91806a3b0 2023-12-26T23:12:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91816e1f0 con 0x7fb91806a3b0 2023-12-26T23:12:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.817+0000 7fb8ff7fe700 1 -- 172.21.15.131:0/2180334620 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0266d0 con 0x7fb91806a3b0 2023-12-26T23:12:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.818+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91816e7b0 con 0x7fb91806a3b0 2023-12-26T23:12:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.819+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91813cda0 con 0x7fb91806a3b0 2023-12-26T23:12:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.820+0000 7fb8ff7fe700 1 -- 172.21.15.131:0/2180334620 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7fb90c026ac0 con 0x7fb91806a3b0 2023-12-26T23:12:47.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.822+0000 7fb8ff7fe700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb9040670e0 0x7fb904069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:47.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.822+0000 7fb9177fe700 1 -- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb9040670e0 msgr2=0x7fb904069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:47.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.822+0000 7fb8ff7fe700 1 -- 172.21.15.131:0/2180334620 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fb90c021070 con 0x7fb91806a3b0 2023-12-26T23:12:47.821 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.822+0000 7fb9177fe700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb9040670e0 0x7fb904069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:47.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:47.826+0000 7fb8ff7fe700 1 -- 172.21.15.131:0/2180334620 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb90c053550 con 0x7fb91806a3b0 2023-12-26T23:12:47.831 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:47.831+0000 7f298870d1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:12:47.873 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:47.874+0000 7f087f3921c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-26T23:12:47.914 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:47.915+0000 7f298870d1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:12:47.959 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:47.960+0000 7f087f3921c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-26T23:12:48.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.023+0000 7fb9177fe700 1 -- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb9040670e0 msgr2=0x7fb904069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:48.022 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.023+0000 7fb9177fe700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb9040670e0 0x7fb904069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.030+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9180657e0 con 0x7fb91806a3b0 2023-12-26T23:12:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.031+0000 7fb8ff7fe700 1 -- 172.21.15.131:0/2180334620 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+16588 (secure 0 0 0) 0x7fb90c056b70 con 0x7fb91806a3b0 2023-12-26T23:12:48.030 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:48.030 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":211,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:42.707975+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:12:20.250102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-26T23:12:21.042553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3395260924":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/3886778096":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/3508392670":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/1340664010":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6837/92228":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6836/92228":"2023-12-27T23:12:42.707929+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb9040670e0 msgr2=0x7fb904069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb9040670e0 0x7fb904069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 msgr2=0x7fb9180c5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 0x7fb9180c5970 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb9180c8800 tx=0x7fb90c017000 comp rx=0 tx=0).stop 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 shutdown_connections 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb9040670e0 0x7fb904069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9180cb170 0x7fb9180c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb918069fe0 0x7fb9180c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 --2- 172.21.15.131:0/2180334620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb91806a3b0 0x7fb9180c5970 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 >> 172.21.15.131:0/2180334620 conn(0x7fb91806c150 msgr2=0x7fb918159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 shutdown_connections 2023-12-26T23:12:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.033+0000 7fb91cb9d700 1 -- 172.21.15.131:0/2180334620 wait complete. 2023-12-26T23:12:48.051 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-26T21:36:37.054865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-26T23:12:20.250102+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '209', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-26T23:12:21.042553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '210', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-26T23:12:48.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-26T23:12:48.060 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:48.061+0000 7f298870d1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:48.120 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:48.121+0000 7f087f3921c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:48.151 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:48.151+0000 7f298870d1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:12:48.215 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:48.216+0000 7f087f3921c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-26T23:12:48.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.254+0000 7f3cc99ff700 1 Processor -- start 2023-12-26T23:12:48.253 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.254+0000 7f3cc99ff700 1 -- start start 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.255+0000 7f3cc99ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3cc41595e0 0x7f3cc41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.255+0000 7f3cc99ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc415d240 0x7f3cc4159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.255+0000 7f3cc99ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3cc40ace50 0x7f3cc40ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.255+0000 7f3cc99ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc40add40 con 0x7f3cc41595e0 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.255+0000 7f3cc99ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc40ad7f0 con 0x7f3cc40ace50 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.255+0000 7f3cc99ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40a98b0 con 0x7f3cc415d240 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.255+0000 7f3cc2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3cc41595e0 0x7f3cc41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc2ffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3cc41595e0 0x7f3cc41599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34916/0 (socket says 172.21.15.131:34916) 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc2ffd700 1 -- 172.21.15.131:0/1203885340 learned_addr learned my addr 172.21.15.131:0/1203885340 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc27fc700 1 --2- 172.21.15.131:0/1203885340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc415d240 0x7f3cc4159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc37fe700 1 --2- 172.21.15.131:0/1203885340 >> v2:172.21.15.190:3300/0 conn(0x7f3cc40ace50 0x7f3cc40ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc2ffd700 1 -- 172.21.15.131:0/1203885340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc415d240 msgr2=0x7f3cc4159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc2ffd700 1 --2- 172.21.15.131:0/1203885340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc415d240 0x7f3cc4159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc2ffd700 1 -- 172.21.15.131:0/1203885340 >> v2:172.21.15.190:3300/0 conn(0x7f3cc40ace50 msgr2=0x7f3cc40ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc2ffd700 1 --2- 172.21.15.131:0/1203885340 >> v2:172.21.15.190:3300/0 conn(0x7f3cc40ace50 0x7f3cc40ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc2ffd700 1 -- 172.21.15.131:0/1203885340 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc415a500 con 0x7f3cc41595e0 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.256+0000 7f3cc2ffd700 1 --2- 172.21.15.131:0/1203885340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc41595e0 0x7f3cc41599b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f3cc406a640 tx=0x7f3cb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8d1c944c6cbbf0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc1ffb700 1 -- 172.21.15.131:0/1203885340 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb400b640 con 0x7f3cc41595e0 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc1ffb700 1 -- 172.21.15.131:0/1203885340 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb400b7a0 con 0x7f3cc41595e0 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc1ffb700 1 -- 172.21.15.131:0/1203885340 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb400bb60 con 0x7f3cc41595e0 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc99ff700 1 -- 172.21.15.131:0/1203885340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc41595e0 msgr2=0x7f3cc41599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/1203885340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc41595e0 0x7f3cc41599b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f3cc406a640 tx=0x7f3cb4009250 comp rx=0 tx=0).stop 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc99ff700 1 -- 172.21.15.131:0/1203885340 shutdown_connections 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/1203885340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc41595e0 0x7f3cc41599b0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/1203885340 >> v2:172.21.15.190:3300/0 conn(0x7f3cc40ace50 0x7f3cc40ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/1203885340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc415d240 0x7f3cc4159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc99ff700 1 -- 172.21.15.131:0/1203885340 >> 172.21.15.131:0/1203885340 conn(0x7f3cc4068040 msgr2=0x7f3cc4069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc99ff700 1 -- 172.21.15.131:0/1203885340 shutdown_connections 2023-12-26T23:12:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.257+0000 7f3cc99ff700 1 -- 172.21.15.131:0/1203885340 wait complete. 2023-12-26T23:12:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.259+0000 7f3cc99ff700 1 Processor -- start 2023-12-26T23:12:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.259+0000 7f3cc99ff700 1 -- start start 2023-12-26T23:12:48.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc99ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc40ace50 0x7f3cc4169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc99ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc41595e0 0x7f3cc416c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc99ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc415d240 0x7f3cc41553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc99ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc416ac50 con 0x7f3cc415d240 2023-12-26T23:12:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc99ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4152240 con 0x7f3cc41595e0 2023-12-26T23:12:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc99ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc416a3b0 con 0x7f3cc40ace50 2023-12-26T23:12:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc415d240 0x7f3cc41553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc37fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc415d240 0x7f3cc41553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34926/0 (socket says 172.21.15.131:34926) 2023-12-26T23:12:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc37fe700 1 -- 172.21.15.131:0/2992965013 learned_addr learned my addr 172.21.15.131:0/2992965013 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc2ffd700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc40ace50 0x7f3cc4169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.260+0000 7f3cc27fc700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc41595e0 0x7f3cc416c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3cc37fe700 1 -- 172.21.15.131:0/2992965013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc40ace50 msgr2=0x7f3cc4169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3cc37fe700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc40ace50 0x7f3cc4169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3cc37fe700 1 -- 172.21.15.131:0/2992965013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc41595e0 msgr2=0x7f3cc416c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3cc37fe700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc41595e0 0x7f3cc416c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3cc37fe700 1 -- 172.21.15.131:0/2992965013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40094f0 con 0x7f3cc415d240 2023-12-26T23:12:48.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3cc2ffd700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc40ace50 0x7f3cc4169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:48.261 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3cc37fe700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc415d240 0x7f3cc41553c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009df0 tx=0x7f3cbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3caaffd700 1 -- 172.21.15.131:0/2992965013 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00caa0 con 0x7f3cc415d240 2023-12-26T23:12:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3caaffd700 1 -- 172.21.15.131:0/2992965013 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc00f040 con 0x7f3cc415d240 2023-12-26T23:12:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc416a690 con 0x7f3cc415d240 2023-12-26T23:12:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.261+0000 7f3caaffd700 1 -- 172.21.15.131:0/2992965013 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00ce50 con 0x7f3cc415d240 2023-12-26T23:12:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.262+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc4152650 con 0x7f3cc415d240 2023-12-26T23:12:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.263+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc40657e0 con 0x7f3cc415d240 2023-12-26T23:12:48.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.264+0000 7f3caaffd700 1 -- 172.21.15.131:0/2992965013 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f3cbc014690 con 0x7f3cc415d240 2023-12-26T23:12:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.264+0000 7f3caaffd700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3cb00670e0 0x7f3cb0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.264+0000 7f3caaffd700 1 -- 172.21.15.131:0/2992965013 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f3cbc0857a0 con 0x7f3cc415d240 2023-12-26T23:12:48.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.264+0000 7f3cc2ffd700 1 -- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3cb00670e0 msgr2=0x7f3cb0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:48.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.265+0000 7f3cc2ffd700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3cb00670e0 0x7f3cb0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:48.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.268+0000 7f3caaffd700 1 -- 172.21.15.131:0/2992965013 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cbc0522d0 con 0x7f3cc415d240 2023-12-26T23:12:48.336 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:48.336+0000 7f298870d1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:12:48.405 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:48.406+0000 7f087f3921c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-26T23:12:48.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.466+0000 7f3cc2ffd700 1 -- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3cb00670e0 msgr2=0x7f3cb0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:48.465 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.466+0000 7f3cc2ffd700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3cb00670e0 0x7f3cb0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:48.485 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:48.486+0000 7f298870d1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:12:48.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.487+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3cc4152870 con 0x7f3cc415d240 2023-12-26T23:12:48.486 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.487+0000 7f3caaffd700 1 -- 172.21.15.131:0/2992965013 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v211) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3cbc0558f0 con 0x7f3cc415d240 2023-12-26T23:12:48.486 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2023-12-26T23:12:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3cb00670e0 msgr2=0x7f3cb0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3cb00670e0 0x7f3cb0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc415d240 msgr2=0x7f3cc41553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc415d240 0x7f3cc41553c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009df0 tx=0x7f3cbc00b040 comp rx=0 tx=0).stop 2023-12-26T23:12:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 shutdown_connections 2023-12-26T23:12:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3cb00670e0 0x7f3cb0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc415d240 0x7f3cc41553c0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc40ace50 0x7f3cc4169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 --2- 172.21.15.131:0/2992965013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc41595e0 0x7f3cc416c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 >> 172.21.15.131:0/2992965013 conn(0x7f3cc4068040 msgr2=0x7f3cc40ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:48.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 shutdown_connections 2023-12-26T23:12:48.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.490+0000 7f3cc99ff700 1 -- 172.21.15.131:0/2992965013 wait complete. 2023-12-26T23:12:48.507 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-26T23:12:48.562 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:48.563+0000 7f087f3921c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-26T23:12:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.704+0000 7f4146fff700 1 Processor -- start 2023-12-26T23:12:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.704+0000 7f4146fff700 1 -- start start 2023-12-26T23:12:48.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.706+0000 7f4146fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4140159820 0x7f41400abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.706+0000 7f4146fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414015a8c0 con 0x7f41400aa480 2023-12-26T23:12:48.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.706+0000 7f4146fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4140159f90 con 0x7f414006e480 2023-12-26T23:12:48.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.706+0000 7f4146fff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414015ad50 con 0x7f4140159820 2023-12-26T23:12:48.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.706+0000 7f413ffff700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f41400aa480 0x7f414006dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59706/0 (socket says 172.21.15.131:59706) 2023-12-26T23:12:48.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.706+0000 7f413ffff700 1 -- 172.21.15.131:0/4238385791 learned_addr learned my addr 172.21.15.131:0/4238385791 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:48.706 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.706+0000 7f4144d9b700 1 --2- 172.21.15.131:0/4238385791 >> v2:172.21.15.190:3300/0 conn(0x7f4140159820 0x7f41400abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.706+0000 7f413f7fe700 1 -- 172.21.15.131:0/4238385791 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106878469 0 0) 0x7f414015a8c0 con 0x7f41400aa480 2023-12-26T23:12:48.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.706+0000 7f413f7fe700 1 -- 172.21.15.131:0/4238385791 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4130003570 con 0x7f41400aa480 2023-12-26T23:12:48.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.707+0000 7f4144d9b700 1 -- 172.21.15.131:0/4238385791 >> v1:172.21.15.190:6789/0 conn(0x7f414006e480 legacy=0x7f41400675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.707 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.707+0000 7f413f7fe700 1 -- 172.21.15.131:0/4238385791 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880505757 0 0) 0x7f4140159f90 con 0x7f414006e480 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.707+0000 7f4144d9b700 1 -- 172.21.15.131:0/4238385791 >> v1:172.21.15.131:6789/0 conn(0x7f41400aa480 legacy=0x7f414006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.707+0000 7f4144d9b700 1 -- 172.21.15.131:0/4238385791 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414015a270 con 0x7f4140159820 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.707+0000 7f4144d9b700 1 --2- 172.21.15.131:0/4238385791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4140159820 0x7f41400abdb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f41340098e0 tx=0x7f41340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9fba4a18810b8f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.707+0000 7f413f7fe700 1 -- 172.21.15.131:0/4238385791 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413400b640 con 0x7f4140159820 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.707+0000 7f413f7fe700 1 -- 172.21.15.131:0/4238385791 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413400b7a0 con 0x7f4140159820 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.707+0000 7f413f7fe700 1 -- 172.21.15.131:0/4238385791 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413400bb40 con 0x7f4140159820 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.708+0000 7f4146fff700 1 -- 172.21.15.131:0/4238385791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4140159820 msgr2=0x7f41400abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.708+0000 7f4146fff700 1 --2- 172.21.15.131:0/4238385791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4140159820 0x7f41400abdb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f41340098e0 tx=0x7f41340091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.708+0000 7f4146fff700 1 -- 172.21.15.131:0/4238385791 shutdown_connections 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.708+0000 7f4146fff700 1 --2- 172.21.15.131:0/4238385791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4140159820 0x7f41400abdb0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.708+0000 7f4146fff700 1 -- 172.21.15.131:0/4238385791 >> 172.21.15.131:0/4238385791 conn(0x7f414006c290 msgr2=0x7f41400ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.708+0000 7f4146fff700 1 -- 172.21.15.131:0/4238385791 shutdown_connections 2023-12-26T23:12:48.708 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.708+0000 7f4146fff700 1 -- 172.21.15.131:0/4238385791 wait complete. 2023-12-26T23:12:48.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.710+0000 7f4146fff700 1 Processor -- start 2023-12-26T23:12:48.709 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.710+0000 7f4146fff700 1 -- start start 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f4146fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414006e480 0x7f41400c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f4146fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400aa480 0x7f4140145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f4146fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4140159820 0x7f41400b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f4146fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400c5f10 con 0x7f4140159820 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f4146fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4140145750 con 0x7f41400aa480 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f4146fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41400c1c30 con 0x7f414006e480 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f414559c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4140159820 0x7f41400b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f414559c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4140159820 0x7f41400b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34930/0 (socket says 172.21.15.131:34930) 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f414559c700 1 -- 172.21.15.131:0/2374516929 learned_addr learned my addr 172.21.15.131:0/2374516929 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f4144d9b700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414006e480 0x7f41400c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.711 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f413ffff700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400aa480 0x7f4140145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f414559c700 1 -- 172.21.15.131:0/2374516929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414006e480 msgr2=0x7f41400c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f414559c700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414006e480 0x7f41400c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f414559c700 1 -- 172.21.15.131:0/2374516929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400aa480 msgr2=0x7f4140145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f414559c700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400aa480 0x7f4140145210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.711+0000 7f414559c700 1 -- 172.21.15.131:0/2374516929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4134009420 con 0x7f4140159820 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.712+0000 7f414559c700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4140159820 0x7f41400b1950 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f413800c930 tx=0x7f41380016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.712+0000 7f413cff9700 1 -- 172.21.15.131:0/2374516929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138012630 con 0x7f4140159820 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.712+0000 7f413cff9700 1 -- 172.21.15.131:0/2374516929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41380127d0 con 0x7f4140159820 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.712+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41400c1f10 con 0x7f4140159820 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.712+0000 7f413cff9700 1 -- 172.21.15.131:0/2374516929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138012b90 con 0x7f4140159820 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.712+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41400c24d0 con 0x7f4140159820 2023-12-26T23:12:48.712 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.713+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41400657e0 con 0x7f4140159820 2023-12-26T23:12:48.713 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.714+0000 7f413cff9700 1 -- 172.21.15.131:0/2374516929 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f4138011380 con 0x7f4140159820 2023-12-26T23:12:48.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.715+0000 7f413cff9700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f41240670e0 0x7f4124069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:48.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.715+0000 7f413cff9700 1 -- 172.21.15.131:0/2374516929 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f4138084d20 con 0x7f4140159820 2023-12-26T23:12:48.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.715+0000 7f4144d9b700 1 -- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f41240670e0 msgr2=0x7f4124069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:48.714 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.715+0000 7f4144d9b700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f41240670e0 0x7f4124069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:48.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.719+0000 7f413cff9700 1 -- 172.21.15.131:0/2374516929 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4138051a00 con 0x7f4140159820 2023-12-26T23:12:48.870 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:48.870+0000 7f298870d1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:12:48.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.916+0000 7f4144d9b700 1 -- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f41240670e0 msgr2=0x7f4124069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:48.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.916+0000 7f4144d9b700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f41240670e0 0x7f4124069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:48.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.920+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f414015a550 con 0x7f4140159820 2023-12-26T23:12:48.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.920+0000 7f413cff9700 1 -- 172.21.15.131:0/2374516929 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v211) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4138055020 con 0x7f4140159820 2023-12-26T23:12:48.919 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2023-12-26T23:12:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.922+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f41240670e0 msgr2=0x7f4124069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.922+0000 7f4146fff700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f41240670e0 0x7f4124069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.922+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4140159820 msgr2=0x7f41400b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.922+0000 7f4146fff700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4140159820 0x7f41400b1950 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f413800c930 tx=0x7f41380016c0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.923+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 shutdown_connections 2023-12-26T23:12:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.923+0000 7f4146fff700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f41240670e0 0x7f4124069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.923+0000 7f4146fff700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4140159820 0x7f41400b1950 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.923+0000 7f4146fff700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414006e480 0x7f41400c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.923+0000 7f4146fff700 1 --2- 172.21.15.131:0/2374516929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400aa480 0x7f4140145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:48.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.923+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 >> 172.21.15.131:0/2374516929 conn(0x7f414006c290 msgr2=0x7f41400add70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:48.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.923+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 shutdown_connections 2023-12-26T23:12:48.923 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:48.923+0000 7f4146fff700 1 -- 172.21.15.131:0/2374516929 wait complete. 2023-12-26T23:12:48.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-26T23:12:48.969 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:48.970+0000 7f087f3921c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-26T23:12:49.137 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:49.137+0000 7f298870d1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:12:49.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.142+0000 7f1e9bdff700 1 Processor -- start 2023-12-26T23:12:49.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.142+0000 7f1e9bdff700 1 -- start start 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.143+0000 7f1e9bdff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1e940cb320 0x7f1e94069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.143+0000 7f1e9bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940b0e60 0x7f1e940c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.143+0000 7f1e9bdff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94156940 con 0x7f1e9406a1e0 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.143+0000 7f1e9bdff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e940c8cc0 con 0x7f1e940cb320 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.143+0000 7f1e9bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940ad700 con 0x7f1e940b0e60 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e99b9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1e940cb320 0x7f1e94069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e9a39c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940b0e60 0x7f1e940c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e99b9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1e940cb320 0x7f1e94069ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:49870/0 (socket says 172.21.15.131:49870) 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e99b9b700 1 -- 172.21.15.131:0/4044780781 learned_addr learned my addr 172.21.15.131:0/4044780781 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:49.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e9939a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1e9406a1e0 0x7f1e940b0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:33672/0 (socket says 172.21.15.131:33672) 2023-12-26T23:12:49.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e9a39c700 1 -- 172.21.15.131:0/4044780781 >> v2:172.21.15.190:3300/0 conn(0x7f1e940cb320 msgr2=0x7f1e94069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e9a39c700 1 --2- 172.21.15.131:0/4044780781 >> v2:172.21.15.190:3300/0 conn(0x7f1e940cb320 0x7f1e94069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e9a39c700 1 -- 172.21.15.131:0/4044780781 >> v1:172.21.15.190:6789/0 conn(0x7f1e9406a1e0 legacy=0x7f1e940b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e9a39c700 1 -- 172.21.15.131:0/4044780781 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940ae500 con 0x7f1e940b0e60 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e99b9b700 1 --2- 172.21.15.131:0/4044780781 >> v2:172.21.15.190:3300/0 conn(0x7f1e940cb320 0x7f1e94069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.144+0000 7f1e9a39c700 1 --2- 172.21.15.131:0/4044780781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940b0e60 0x7f1e940c9400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009e60 tx=0x7f1e8c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48be6a79778844d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e98b99700 1 -- 172.21.15.131:0/4044780781 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c00e070 con 0x7f1e940b0e60 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e98b99700 1 -- 172.21.15.131:0/4044780781 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c00bb60 con 0x7f1e940b0e60 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e98b99700 1 -- 172.21.15.131:0/4044780781 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c012430 con 0x7f1e940b0e60 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e9bdff700 1 -- 172.21.15.131:0/4044780781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940b0e60 msgr2=0x7f1e940c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e9bdff700 1 --2- 172.21.15.131:0/4044780781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940b0e60 0x7f1e940c9400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009e60 tx=0x7f1e8c009780 comp rx=0 tx=0).stop 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e9bdff700 1 -- 172.21.15.131:0/4044780781 shutdown_connections 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e9bdff700 1 --2- 172.21.15.131:0/4044780781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940b0e60 0x7f1e940c9400 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e9bdff700 1 --2- 172.21.15.131:0/4044780781 >> v2:172.21.15.190:3300/0 conn(0x7f1e940cb320 0x7f1e94069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e9bdff700 1 -- 172.21.15.131:0/4044780781 >> 172.21.15.131:0/4044780781 conn(0x7f1e94068040 msgr2=0x7f1e94069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:49.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e9bdff700 1 -- 172.21.15.131:0/4044780781 shutdown_connections 2023-12-26T23:12:49.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.145+0000 7f1e9bdff700 1 -- 172.21.15.131:0/4044780781 wait complete. 2023-12-26T23:12:49.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.147+0000 7f1e9bdff700 1 Processor -- start 2023-12-26T23:12:49.147 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.148+0000 7f1e9bdff700 1 -- start start 2023-12-26T23:12:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9bdff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e9406a1e0 0x7f1e940c11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9bdff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e940b0e60 0x7f1e9406e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9bdff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940cb320 0x7f1e94143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9bdff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940aa180 con 0x7f1e940b0e60 2023-12-26T23:12:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9bdff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940aa460 con 0x7f1e9406a1e0 2023-12-26T23:12:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9bdff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940aa740 con 0x7f1e940cb320 2023-12-26T23:12:49.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9939a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e940b0e60 0x7f1e9406e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9939a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e940b0e60 0x7f1e9406e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34938/0 (socket says 172.21.15.131:34938) 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9939a700 1 -- 172.21.15.131:0/458191080 learned_addr learned my addr 172.21.15.131:0/458191080 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e99b9b700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e9406a1e0 0x7f1e940c11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.149+0000 7f1e9a39c700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940cb320 0x7f1e94143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e9939a700 1 -- 172.21.15.131:0/458191080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940cb320 msgr2=0x7f1e94143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e9939a700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940cb320 0x7f1e94143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e9939a700 1 -- 172.21.15.131:0/458191080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e9406a1e0 msgr2=0x7f1e940c11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e9939a700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e9406a1e0 0x7f1e940c11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e9939a700 1 -- 172.21.15.131:0/458191080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e900094f0 con 0x7f1e940b0e60 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e99b9b700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e9406a1e0 0x7f1e940c11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e9939a700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e940b0e60 0x7f1e9406e600 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400c790 tx=0x7f1e84007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e89ffb700 1 -- 172.21.15.131:0/458191080 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84005a70 con 0x7f1e940b0e60 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e89ffb700 1 -- 172.21.15.131:0/458191080 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e84005bf0 con 0x7f1e940b0e60 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e89ffb700 1 -- 172.21.15.131:0/458191080 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840070d0 con 0x7f1e940b0e60 2023-12-26T23:12:49.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c009a20 con 0x7f1e940b0e60 2023-12-26T23:12:49.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.150+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e940aad00 con 0x7f1e940b0e60 2023-12-26T23:12:49.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.151+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e940657e0 con 0x7f1e940b0e60 2023-12-26T23:12:49.151 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.152+0000 7f1e89ffb700 1 -- 172.21.15.131:0/458191080 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f1e84007480 con 0x7f1e940b0e60 2023-12-26T23:12:49.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.153+0000 7f1e89ffb700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1e780670e0 0x7f1e78069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.153+0000 7f1e89ffb700 1 -- 172.21.15.131:0/458191080 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1e84083ff0 con 0x7f1e940b0e60 2023-12-26T23:12:49.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.153+0000 7f1e99b9b700 1 -- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1e780670e0 msgr2=0x7f1e78069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:49.152 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.153+0000 7f1e99b9b700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1e780670e0 0x7f1e78069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:49.156 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.157+0000 7f1e89ffb700 1 -- 172.21.15.131:0/458191080 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e84050c70 con 0x7f1e940b0e60 2023-12-26T23:12:49.255 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:49.256+0000 7f087f3921c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-26T23:12:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.354+0000 7f1e99b9b700 1 -- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1e780670e0 msgr2=0x7f1e78069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.354+0000 7f1e99b9b700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1e780670e0 0x7f1e78069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.357+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e940aaef0 con 0x7f1e940b0e60 2023-12-26T23:12:49.356 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.358+0000 7f1e89ffb700 1 -- 172.21.15.131:0/458191080 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v211) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e84017020 con 0x7f1e940b0e60 2023-12-26T23:12:49.357 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2023-12-26T23:12:49.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1e780670e0 msgr2=0x7f1e78069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:49.359 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1e780670e0 0x7f1e78069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e940b0e60 msgr2=0x7f1e9406e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e940b0e60 0x7f1e9406e600 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400c790 tx=0x7f1e84007f40 comp rx=0 tx=0).stop 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 shutdown_connections 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1e780670e0 0x7f1e78069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e940cb320 0x7f1e94143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e9406a1e0 0x7f1e940c11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 --2- 172.21.15.131:0/458191080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e940b0e60 0x7f1e9406e600 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 >> 172.21.15.131:0/458191080 conn(0x7f1e94068040 msgr2=0x7f1e940adca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 shutdown_connections 2023-12-26T23:12:49.360 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.360+0000 7f1e9bdff700 1 -- 172.21.15.131:0/458191080 wait complete. 2023-12-26T23:12:49.377 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:49.416 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:49.416+0000 7f298870d1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:12:49.505 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:49.505+0000 7f298870d1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:12:49.545 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:49.546+0000 7f087f3921c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-26T23:12:49.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.614+0000 7f96875ff700 1 Processor -- start 2023-12-26T23:12:49.613 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.614+0000 7f96875ff700 1 -- start start 2023-12-26T23:12:49.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.615+0000 7f96875ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968006f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.615+0000 7f96875ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f968015ba50 0x7f9680157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.615+0000 7f96875ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968015b5c0 con 0x7f9680157f90 2023-12-26T23:12:49.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.615+0000 7f96875ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9680158af0 con 0x7f968015ba50 2023-12-26T23:12:49.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.615+0000 7f96875ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9680158530 con 0x7f968006f0e0 2023-12-26T23:12:49.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9684b9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f968015ba50 0x7f9680157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9685b9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9680157f90 0x7f96800b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59720/0 (socket says 172.21.15.131:59720) 2023-12-26T23:12:49.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9685b9c700 1 -- 172.21.15.131:0/3492268451 learned_addr learned my addr 172.21.15.131:0/3492268451 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:49.615 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f968539b700 1 --2- 172.21.15.131:0/3492268451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968006f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.616 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9677fff700 1 -- 172.21.15.131:0/3492268451 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59529882 0 0) 0x7f968015b5c0 con 0x7f9680157f90 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9677fff700 1 -- 172.21.15.131:0/3492268451 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f966c003570 con 0x7f9680157f90 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9684b9a700 1 -- 172.21.15.131:0/3492268451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 msgr2=0x7f968006f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9684b9a700 1 --2- 172.21.15.131:0/3492268451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968006f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9684b9a700 1 -- 172.21.15.131:0/3492268451 >> v1:172.21.15.131:6789/0 conn(0x7f9680157f90 legacy=0x7f96800b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9684b9a700 1 -- 172.21.15.131:0/3492268451 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680158810 con 0x7f968015ba50 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f968539b700 1 --2- 172.21.15.131:0/3492268451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.616+0000 7f9684b9a700 1 --2- 172.21.15.131:0/3492268451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f968015ba50 0x7f9680157a50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f96700098e0 tx=0x7f96700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb93d425de4d9295 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.617+0000 7f9677fff700 1 -- 172.21.15.131:0/3492268451 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670011630 con 0x7f968015ba50 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.617+0000 7f9677fff700 1 -- 172.21.15.131:0/3492268451 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9670011790 con 0x7f968015ba50 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.617+0000 7f9677fff700 1 -- 172.21.15.131:0/3492268451 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670011b30 con 0x7f968015ba50 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.617+0000 7f96875ff700 1 -- 172.21.15.131:0/3492268451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f968015ba50 msgr2=0x7f9680157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.617+0000 7f96875ff700 1 --2- 172.21.15.131:0/3492268451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f968015ba50 0x7f9680157a50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f96700098e0 tx=0x7f96700091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.617+0000 7f96875ff700 1 -- 172.21.15.131:0/3492268451 shutdown_connections 2023-12-26T23:12:49.617 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.617+0000 7f96875ff700 1 --2- 172.21.15.131:0/3492268451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.617+0000 7f96875ff700 1 --2- 172.21.15.131:0/3492268451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f968015ba50 0x7f9680157a50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.617+0000 7f96875ff700 1 -- 172.21.15.131:0/3492268451 >> 172.21.15.131:0/3492268451 conn(0x7f96800686c0 msgr2=0x7f9680068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:49.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.618+0000 7f96875ff700 1 -- 172.21.15.131:0/3492268451 shutdown_connections 2023-12-26T23:12:49.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.618+0000 7f96875ff700 1 -- 172.21.15.131:0/3492268451 wait complete. 2023-12-26T23:12:49.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.619+0000 7f96875ff700 1 Processor -- start 2023-12-26T23:12:49.618 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.619+0000 7f96875ff700 1 -- start start 2023-12-26T23:12:49.619 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f96875ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968013d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f96875ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680157f90 0x7f96801505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f96875ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f968015ba50 0x7f9680156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f96875ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9680140d40 con 0x7f968015ba50 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f96875ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9680151a20 con 0x7f9680157f90 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f96875ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9680151eb0 con 0x7f968006f0e0 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f968539b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968013d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f9685b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f968015ba50 0x7f9680156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f9684b9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680157f90 0x7f96801505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f968539b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968013d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:47888/0 (socket says 172.21.15.131:47888) 2023-12-26T23:12:49.620 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.620+0000 7f968539b700 1 -- 172.21.15.131:0/3605211782 learned_addr learned my addr 172.21.15.131:0/3605211782 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:49.621 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.621+0000 7f9684b9a700 1 -- 172.21.15.131:0/3605211782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 msgr2=0x7f968013d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.621+0000 7f9684b9a700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968013d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.621+0000 7f9684b9a700 1 -- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f968015ba50 msgr2=0x7f9680156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.621+0000 7f9684b9a700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f968015ba50 0x7f9680156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.621+0000 7f9684b9a700 1 -- 172.21.15.131:0/3605211782 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c009420 con 0x7f9680157f90 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.621+0000 7f968539b700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968013d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.621+0000 7f9684b9a700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680157f90 0x7f96801505d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9670016000 tx=0x7f9670017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.621+0000 7f96757fa700 1 -- 172.21.15.131:0/3605211782 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670011900 con 0x7f9680157f90 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.621+0000 7f96757fa700 1 -- 172.21.15.131:0/3605211782 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9670011a60 con 0x7f9680157f90 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.622+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9670009420 con 0x7f9680157f90 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.622+0000 7f96757fa700 1 -- 172.21.15.131:0/3605211782 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670025640 con 0x7f9680157f90 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.622+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96801510f0 con 0x7f9680157f90 2023-12-26T23:12:49.622 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.623+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96800657e0 con 0x7f9680157f90 2023-12-26T23:12:49.623 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.624+0000 7f96757fa700 1 -- 172.21.15.131:0/3605211782 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f96700259f0 con 0x7f9680157f90 2023-12-26T23:12:49.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.625+0000 7f96757fa700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96640670e0 0x7f9664069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:49.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.625+0000 7f968539b700 1 -- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96640670e0 msgr2=0x7f9664069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:49.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.625+0000 7f96757fa700 1 -- 172.21.15.131:0/3605211782 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f9670086200 con 0x7f9680157f90 2023-12-26T23:12:49.624 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.625+0000 7f968539b700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96640670e0 0x7f9664069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:49.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.629+0000 7f96757fa700 1 -- 172.21.15.131:0/3605211782 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9670052e40 con 0x7f9680157f90 2023-12-26T23:12:49.642 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:49.643+0000 7f087f3921c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-26T23:12:49.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.826+0000 7f968539b700 1 -- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96640670e0 msgr2=0x7f9664069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:49.825 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.826+0000 7f968539b700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96640670e0 0x7f9664069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:49.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.916+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f968015b250 con 0x7f9680157f90 2023-12-26T23:12:49.916 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.917+0000 7f96757fa700 1 -- 172.21.15.131:0/3605211782 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4617 (secure 0 0 0) 0x7f9670056460 con 0x7f9680157f90 2023-12-26T23:12:49.916 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:49.916 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8372,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2599990460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2599990460},{"type":"v1","addr":"172.21.15.190:6835","nonce":2599990460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":9565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3465421941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3465421941},{"type":"v1","addr":"172.21.15.131:6834","nonce":3465421941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":10122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3595865527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3595865527},{"type":"v1","addr":"172.21.15.131:6835","nonce":3595865527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:12:22.064606+0000","modified":"2023-12-26T23:12:25.606067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10125":{"gid":10125,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/1917956201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1917956201},{"type":"v1","addr":"172.21.15.190:6834","nonce":1917956201}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-12-26T23:12:49.918 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96640670e0 msgr2=0x7f9664069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96640670e0 0x7f9664069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680157f90 msgr2=0x7f96801505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680157f90 0x7f96801505d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9670016000 tx=0x7f9670017000 comp rx=0 tx=0).stop 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 shutdown_connections 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f96640670e0 0x7f9664069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f968015ba50 0x7f9680156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968006f0e0 0x7f968013d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 --2- 172.21.15.131:0/3605211782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680157f90 0x7f96801505d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 >> 172.21.15.131:0/3605211782 conn(0x7f96800686c0 msgr2=0x7f968014de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 shutdown_connections 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:49.919+0000 7f96875ff700 1 -- 172.21.15.131:0/3605211782 wait complete. 2023-12-26T23:12:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 98 2023-12-26T23:12:49.937 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:50.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.134+0000 7f11449fe700 1 Processor -- start 2023-12-26T23:12:50.134 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.134+0000 7f11449fe700 1 -- start start 2023-12-26T23:12:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.135+0000 7f11449fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1140158950 0x7f11400ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.135+0000 7f11449fe700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11400aedd0 con 0x7f114015a3c0 2023-12-26T23:12:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.135+0000 7f11449fe700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11400af0b0 con 0x7f114015b550 2023-12-26T23:12:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.135+0000 7f11449fe700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11400675b0 con 0x7f1140158950 2023-12-26T23:12:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.135+0000 7f113ed9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f114015a3c0 0x7f114015ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59734/0 (socket says 172.21.15.131:59734) 2023-12-26T23:12:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.135+0000 7f113ed9d700 1 -- 172.21.15.131:0/1642751902 learned_addr learned my addr 172.21.15.131:0/1642751902 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.136+0000 7f113f59e700 1 --2- 172.21.15.131:0/1642751902 >> v2:172.21.15.190:3300/0 conn(0x7f1140158950 0x7f11400ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.136+0000 7f113dd9b700 1 -- 172.21.15.131:0/1642751902 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411016061 0 0) 0x7f11400aedd0 con 0x7f114015a3c0 2023-12-26T23:12:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.136+0000 7f113dd9b700 1 -- 172.21.15.131:0/1642751902 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1128003570 con 0x7f114015a3c0 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.136+0000 7f113f59e700 1 -- 172.21.15.131:0/1642751902 >> v1:172.21.15.190:6789/0 conn(0x7f114015b550 legacy=0x7f11400ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.136+0000 7f113f59e700 1 -- 172.21.15.131:0/1642751902 >> v1:172.21.15.131:6789/0 conn(0x7f114015a3c0 legacy=0x7f114015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.136+0000 7f113f59e700 1 -- 172.21.15.131:0/1642751902 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140067890 con 0x7f1140158950 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.136+0000 7f113f59e700 1 --2- 172.21.15.131:0/1642751902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1140158950 0x7f11400ae780 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1138009e60 tx=0x7f1138009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2fd5b1d6314cc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f113dd9b700 1 -- 172.21.15.131:0/1642751902 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113800e070 con 0x7f1140158950 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f113dd9b700 1 -- 172.21.15.131:0/1642751902 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113800bb60 con 0x7f1140158950 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f113dd9b700 1 -- 172.21.15.131:0/1642751902 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138012430 con 0x7f1140158950 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f11449fe700 1 -- 172.21.15.131:0/1642751902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1140158950 msgr2=0x7f11400ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f11449fe700 1 --2- 172.21.15.131:0/1642751902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1140158950 0x7f11400ae780 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1138009e60 tx=0x7f1138009780 comp rx=0 tx=0).stop 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f11449fe700 1 -- 172.21.15.131:0/1642751902 shutdown_connections 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f11449fe700 1 --2- 172.21.15.131:0/1642751902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1140158950 0x7f11400ae780 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f11449fe700 1 -- 172.21.15.131:0/1642751902 >> 172.21.15.131:0/1642751902 conn(0x7f11400cd330 msgr2=0x7f11400c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f11449fe700 1 -- 172.21.15.131:0/1642751902 shutdown_connections 2023-12-26T23:12:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.137+0000 7f11449fe700 1 -- 172.21.15.131:0/1642751902 wait complete. 2023-12-26T23:12:50.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.139+0000 7f11449fe700 1 Processor -- start 2023-12-26T23:12:50.139 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.139+0000 7f11449fe700 1 -- start start 2023-12-26T23:12:50.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f11449fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1140158950 0x7f114015d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f11449fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f114015a3c0 0x7f1140158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f11449fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114015b550 0x7f114014c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f11449fe700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114016a5c0 con 0x7f114015a3c0 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f11449fe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140169c90 con 0x7f1140158950 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f11449fe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1140169f70 con 0x7f114015b550 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f113e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f114015a3c0 0x7f1140158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f113ed9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1140158950 0x7f114015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f113e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f114015a3c0 0x7f1140158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34956/0 (socket says 172.21.15.131:34956) 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f113e59c700 1 -- 172.21.15.131:0/2376367829 learned_addr learned my addr 172.21.15.131:0/2376367829 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.140+0000 7f113f59e700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114015b550 0x7f114014c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f113e59c700 1 -- 172.21.15.131:0/2376367829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114015b550 msgr2=0x7f114014c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f113e59c700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114015b550 0x7f114014c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f113e59c700 1 -- 172.21.15.131:0/2376367829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1140158950 msgr2=0x7f114015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f113e59c700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1140158950 0x7f114015d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f113e59c700 1 -- 172.21.15.131:0/2376367829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138009a20 con 0x7f114015a3c0 2023-12-26T23:12:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f113e59c700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f114015a3c0 0x7f1140158f80 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f113400c7c0 tx=0x7f1134007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:50.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f1126ffd700 1 -- 172.21.15.131:0/2376367829 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134011590 con 0x7f114015a3c0 2023-12-26T23:12:50.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f1126ffd700 1 -- 172.21.15.131:0/2376367829 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11340116f0 con 0x7f114015a3c0 2023-12-26T23:12:50.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f1126ffd700 1 -- 172.21.15.131:0/2376367829 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134011ab0 con 0x7f114015a3c0 2023-12-26T23:12:50.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114016a250 con 0x7f114015a3c0 2023-12-26T23:12:50.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.141+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114015da90 con 0x7f114015a3c0 2023-12-26T23:12:50.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.142+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114016a730 con 0x7f114015a3c0 2023-12-26T23:12:50.143 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.143+0000 7f1126ffd700 1 -- 172.21.15.131:0/2376367829 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f113401b3b0 con 0x7f114015a3c0 2023-12-26T23:12:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.144+0000 7f1126ffd700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f112c0670e0 0x7f112c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.144+0000 7f1126ffd700 1 -- 172.21.15.131:0/2376367829 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1134083f10 con 0x7f114015a3c0 2023-12-26T23:12:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.144+0000 7f113ed9d700 1 -- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f112c0670e0 msgr2=0x7f112c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:50.145 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.144+0000 7f113ed9d700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f112c0670e0 0x7f112c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:50.148 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.148+0000 7f1126ffd700 1 -- 172.21.15.131:0/2376367829 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1134050bf0 con 0x7f114015a3c0 2023-12-26T23:12:50.243 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:50.243+0000 7f298870d1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:12:50.337 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:50.337+0000 7f298870d1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:12:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.345+0000 7f113ed9d700 1 -- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f112c0670e0 msgr2=0x7f112c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.345+0000 7f113ed9d700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f112c0670e0 0x7f112c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:50.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.348+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11400657e0 con 0x7f114015a3c0 2023-12-26T23:12:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.349+0000 7f1126ffd700 1 -- 172.21.15.131:0/2376367829 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+16588 (secure 0 0 0) 0x7f1134017070 con 0x7f114015a3c0 2023-12-26T23:12:50.349 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:50.349 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":211,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:42.707975+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:12:20.250102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-26T23:12:21.042553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3395260924":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/3886778096":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/3508392670":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/1340664010":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6837/92228":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6836/92228":"2023-12-27T23:12:42.707929+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f112c0670e0 msgr2=0x7f112c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f112c0670e0 0x7f112c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f114015a3c0 msgr2=0x7f1140158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f114015a3c0 0x7f1140158f80 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f113400c7c0 tx=0x7f1134007f50 comp rx=0 tx=0).stop 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 shutdown_connections 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f112c0670e0 0x7f112c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114015b550 0x7f114014c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1140158950 0x7f114015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 --2- 172.21.15.131:0/2376367829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f114015a3c0 0x7f1140158f80 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 >> 172.21.15.131:0/2376367829 conn(0x7f11400cd330 msgr2=0x7f1140069810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 shutdown_connections 2023-12-26T23:12:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.351+0000 7f11449fe700 1 -- 172.21.15.131:0/2376367829 wait complete. 2023-12-26T23:12:50.370 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T23:12:50.370 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:50.443 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:50.443+0000 7f087f3921c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-26T23:12:50.489 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:50.489+0000 7f298870d1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:12:50.540 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:50.540+0000 7f087f3921c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-26T23:12:50.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.563+0000 7f05b19ff700 1 Processor -- start 2023-12-26T23:12:50.563 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.563+0000 7f05b19ff700 1 -- start start 2023-12-26T23:12:50.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.565+0000 7f05b19ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f05ac0cc580 0x7f05ac0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.565+0000 7f05b19ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac0b0d70 con 0x7f05ac0b0560 2023-12-26T23:12:50.566 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.565+0000 7f05b19ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac0675b0 con 0x7f05ac0b00e0 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.565+0000 7f05b19ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac0680f0 con 0x7f05ac0cc580 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.565+0000 7f05aaffd700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f05ac0cc580 0x7f05ac0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.565+0000 7f05ab7fe700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f05ac0b0560 0x7f05ac15d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59736/0 (socket says 172.21.15.131:59736) 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.565+0000 7f05ab7fe700 1 -- 172.21.15.131:0/544230058 learned_addr learned my addr 172.21.15.131:0/544230058 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.566+0000 7f05aaffd700 1 -- 172.21.15.131:0/544230058 >> v1:172.21.15.190:6789/0 conn(0x7f05ac0b00e0 legacy=0x7f05ac06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.566+0000 7f05a9ffb700 1 -- 172.21.15.131:0/544230058 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582186634 0 0) 0x7f05ac0b0d70 con 0x7f05ac0b0560 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.566+0000 7f05aaffd700 1 -- 172.21.15.131:0/544230058 >> v1:172.21.15.131:6789/0 conn(0x7f05ac0b0560 legacy=0x7f05ac15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.566+0000 7f05aaffd700 1 -- 172.21.15.131:0/544230058 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac159c30 con 0x7f05ac0cc580 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.566+0000 7f05aaffd700 1 --2- 172.21.15.131:0/544230058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0cc580 0x7f05ac0cc950 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f059c009860 tx=0x7f059c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad435ab58d0b06f server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.566+0000 7f05a9ffb700 1 -- 172.21.15.131:0/544230058 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c011630 con 0x7f05ac0cc580 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.566+0000 7f05a9ffb700 1 -- 172.21.15.131:0/544230058 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c011790 con 0x7f05ac0cc580 2023-12-26T23:12:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.566+0000 7f05a9ffb700 1 -- 172.21.15.131:0/544230058 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c011b50 con 0x7f05ac0cc580 2023-12-26T23:12:50.568 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.567+0000 7f05b19ff700 1 -- 172.21.15.131:0/544230058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0cc580 msgr2=0x7f05ac0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.567+0000 7f05b19ff700 1 --2- 172.21.15.131:0/544230058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0cc580 0x7f05ac0cc950 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f059c009860 tx=0x7f059c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.567+0000 7f05b19ff700 1 -- 172.21.15.131:0/544230058 shutdown_connections 2023-12-26T23:12:50.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.567+0000 7f05b19ff700 1 --2- 172.21.15.131:0/544230058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0cc580 0x7f05ac0cc950 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.567+0000 7f05b19ff700 1 -- 172.21.15.131:0/544230058 >> 172.21.15.131:0/544230058 conn(0x7f05ac0686c0 msgr2=0x7f05ac068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:50.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.567+0000 7f05b19ff700 1 -- 172.21.15.131:0/544230058 shutdown_connections 2023-12-26T23:12:50.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.567+0000 7f05b19ff700 1 -- 172.21.15.131:0/544230058 wait complete. 2023-12-26T23:12:50.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.568+0000 7f05b19ff700 1 Processor -- start 2023-12-26T23:12:50.569 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.569+0000 7f05b19ff700 1 -- start start 2023-12-26T23:12:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.569+0000 7f05b19ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac0b00e0 0x7f05ac0acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.569+0000 7f05b19ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0b0560 0x7f05ac1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.569+0000 7f05b19ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0cc580 0x7f05ac153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.569+0000 7f05b19ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0add90 con 0x7f05ac0b0560 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.569+0000 7f05b19ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0ad4f0 con 0x7f05ac0b00e0 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.569+0000 7f05b19ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0ad7d0 con 0x7f05ac0cc580 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aa7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0b0560 0x7f05ac1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aa7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0b0560 0x7f05ac1522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34970/0 (socket says 172.21.15.131:34970) 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aa7fc700 1 -- 172.21.15.131:0/3891457228 learned_addr learned my addr 172.21.15.131:0/3891457228 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aaffd700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac0b00e0 0x7f05ac0acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05ab7fe700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0cc580 0x7f05ac153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aa7fc700 1 -- 172.21.15.131:0/3891457228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0cc580 msgr2=0x7f05ac153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aa7fc700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0cc580 0x7f05ac153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aa7fc700 1 -- 172.21.15.131:0/3891457228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac0b00e0 msgr2=0x7f05ac0acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aa7fc700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac0b00e0 0x7f05ac0acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aa7fc700 1 -- 172.21.15.131:0/3891457228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c009420 con 0x7f05ac0b0560 2023-12-26T23:12:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.570+0000 7f05aa7fc700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0b0560 0x7f05ac1522a0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f05a000c810 tx=0x7f05a0007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.571+0000 7f0592ffd700 1 -- 172.21.15.131:0/3891457228 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0011590 con 0x7f05ac0b0560 2023-12-26T23:12:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.571+0000 7f0592ffd700 1 -- 172.21.15.131:0/3891457228 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a00116f0 con 0x7f05ac0b0560 2023-12-26T23:12:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.571+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac0adab0 con 0x7f05ac0b0560 2023-12-26T23:12:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.571+0000 7f0592ffd700 1 -- 172.21.15.131:0/3891457228 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0011ab0 con 0x7f05ac0b0560 2023-12-26T23:12:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.571+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac0ad0e0 con 0x7f05ac0b0560 2023-12-26T23:12:50.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.572+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac159e50 con 0x7f05ac0b0560 2023-12-26T23:12:50.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.573+0000 7f0592ffd700 1 -- 172.21.15.131:0/3891457228 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f05a001b3b0 con 0x7f05ac0b0560 2023-12-26T23:12:50.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.574+0000 7f0592ffd700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05980670e0 0x7f0598069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:50.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.574+0000 7f0592ffd700 1 -- 172.21.15.131:0/3891457228 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f05a0084130 con 0x7f05ac0b0560 2023-12-26T23:12:50.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.574+0000 7f05aaffd700 1 -- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05980670e0 msgr2=0x7f0598069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:50.574 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.574+0000 7f05aaffd700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05980670e0 0x7f0598069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.578+0000 7f0592ffd700 1 -- 172.21.15.131:0/3891457228 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05ac159e50 con 0x7f05ac0b0560 2023-12-26T23:12:50.699 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:50.699+0000 7f087f3921c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-26T23:12:50.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.775+0000 7f05aaffd700 1 -- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05980670e0 msgr2=0x7f0598069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:50.775 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.775+0000 7f05aaffd700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05980670e0 0x7f0598069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:50.823 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.822+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05ac15a230 con 0x7f05ac0b0560 2023-12-26T23:12:50.824 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.823+0000 7f0592ffd700 1 -- 172.21.15.131:0/3891457228 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4617 (secure 0 0 0) 0x7f05ac15a230 con 0x7f05ac0b0560 2023-12-26T23:12:50.824 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:50.824 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8372,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2599990460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2599990460},{"type":"v1","addr":"172.21.15.190:6835","nonce":2599990460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":9565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3465421941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3465421941},{"type":"v1","addr":"172.21.15.131:6834","nonce":3465421941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":10122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3595865527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3595865527},{"type":"v1","addr":"172.21.15.131:6835","nonce":3595865527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:12:22.064606+0000","modified":"2023-12-26T23:12:25.606067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10125":{"gid":10125,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/1917956201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1917956201},{"type":"v1","addr":"172.21.15.190:6834","nonce":1917956201}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-12-26T23:12:50.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05980670e0 msgr2=0x7f0598069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:50.826 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05980670e0 0x7f0598069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0b0560 msgr2=0x7f05ac1522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0b0560 0x7f05ac1522a0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f05a000c810 tx=0x7f05a0007ee0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 shutdown_connections 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f05980670e0 0x7f0598069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ac0cc580 0x7f05ac153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ac0b00e0 0x7f05ac0acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 --2- 172.21.15.131:0/3891457228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ac0b0560 0x7f05ac1522a0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 >> 172.21.15.131:0/3891457228 conn(0x7f05ac0686c0 msgr2=0x7f05ac0b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 shutdown_connections 2023-12-26T23:12:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:50.826+0000 7f05b19ff700 1 -- 172.21.15.131:0/3891457228 wait complete. 2023-12-26T23:12:50.828 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 98 2023-12-26T23:12:50.844 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:12:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.042+0000 7f628cbff700 1 Processor -- start 2023-12-26T23:12:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.042+0000 7f628cbff700 1 -- start start 2023-12-26T23:12:51.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.043+0000 7f628cbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62880c8820 0x7f62880ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.043+0000 7f628cbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f62880c8bf0 0x7f62880b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.043+0000 7f628cbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 0x7f62880b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.043+0000 7f628cbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6288157a80 con 0x7f62880c8bf0 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f628cbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62880cac50 con 0x7f62880c8820 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f628cbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62880caf30 con 0x7f6288069d10 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f6285d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f62880c8bf0 0x7f62880b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f6285d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f62880c8bf0 0x7f62880b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34980/0 (socket says 172.21.15.131:34980) 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f628659c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62880c8820 0x7f62880ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f6285d9b700 1 -- 172.21.15.131:0/211906793 learned_addr learned my addr 172.21.15.131:0/211906793 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f6286d9d700 1 --2- 172.21.15.131:0/211906793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 0x7f62880b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f6285d9b700 1 -- 172.21.15.131:0/211906793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 msgr2=0x7f62880b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f6285d9b700 1 --2- 172.21.15.131:0/211906793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 0x7f62880b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f6285d9b700 1 -- 172.21.15.131:0/211906793 >> v2:172.21.15.190:3300/0 conn(0x7f62880c8820 msgr2=0x7f62880ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f6285d9b700 1 --2- 172.21.15.131:0/211906793 >> v2:172.21.15.190:3300/0 conn(0x7f62880c8820 0x7f62880ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f6285d9b700 1 -- 172.21.15.131:0/211906793 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62880cb210 con 0x7f62880c8bf0 2023-12-26T23:12:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.044+0000 7f628659c700 1 --2- 172.21.15.131:0/211906793 >> v2:172.21.15.190:3300/0 conn(0x7f62880c8820 0x7f62880ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.045+0000 7f6285d9b700 1 --2- 172.21.15.131:0/211906793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 0x7f62880b12e0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f6288064ea0 tx=0x7f627c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279ae573b63119cf server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.045+0000 7f628559a700 1 -- 172.21.15.131:0/211906793 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c00b640 con 0x7f62880c8bf0 2023-12-26T23:12:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.045+0000 7f628559a700 1 -- 172.21.15.131:0/211906793 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c00b7a0 con 0x7f62880c8bf0 2023-12-26T23:12:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.045+0000 7f628559a700 1 -- 172.21.15.131:0/211906793 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c00bb60 con 0x7f62880c8bf0 2023-12-26T23:12:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.046+0000 7f628cbff700 1 -- 172.21.15.131:0/211906793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 msgr2=0x7f62880b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.046+0000 7f628cbff700 1 --2- 172.21.15.131:0/211906793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 0x7f62880b12e0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f6288064ea0 tx=0x7f627c009250 comp rx=0 tx=0).stop 2023-12-26T23:12:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.046+0000 7f628cbff700 1 -- 172.21.15.131:0/211906793 shutdown_connections 2023-12-26T23:12:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.046+0000 7f628cbff700 1 --2- 172.21.15.131:0/211906793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 0x7f62880b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.046+0000 7f628cbff700 1 --2- 172.21.15.131:0/211906793 >> v2:172.21.15.190:3300/0 conn(0x7f62880c8820 0x7f62880ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.046+0000 7f628cbff700 1 --2- 172.21.15.131:0/211906793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 0x7f62880b12e0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.046+0000 7f628cbff700 1 -- 172.21.15.131:0/211906793 >> 172.21.15.131:0/211906793 conn(0x7f6288068040 msgr2=0x7f6288069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.046+0000 7f628cbff700 1 -- 172.21.15.131:0/211906793 shutdown_connections 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.046+0000 7f628cbff700 1 -- 172.21.15.131:0/211906793 wait complete. 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.048+0000 7f628cbff700 1 Processor -- start 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.048+0000 7f628cbff700 1 -- start start 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f628cbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 0x7f6288148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f628cbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62880c8820 0x7f6288152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f628cbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 0x7f62880c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f628cbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881597d0 con 0x7f62880c8bf0 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f628cbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288169030 con 0x7f62880c8820 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f628cbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62881539d0 con 0x7f6288069d10 2023-12-26T23:12:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f6286d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 0x7f62880c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f6285d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62880c8820 0x7f6288152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f6286d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 0x7f62880c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34982/0 (socket says 172.21.15.131:34982) 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f6286d9d700 1 -- 172.21.15.131:0/3960981817 learned_addr learned my addr 172.21.15.131:0/3960981817 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f628659c700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 0x7f6288148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f6286d9d700 1 -- 172.21.15.131:0/3960981817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 msgr2=0x7f6288148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f6286d9d700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 0x7f6288148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f6286d9d700 1 -- 172.21.15.131:0/3960981817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62880c8820 msgr2=0x7f6288152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f6286d9d700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62880c8820 0x7f6288152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.049+0000 7f6286d9d700 1 -- 172.21.15.131:0/3960981817 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278009420 con 0x7f62880c8bf0 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.050+0000 7f6285d9b700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62880c8820 0x7f6288152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.050+0000 7f6286d9d700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 0x7f62880c73b0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f6280009df0 tx=0x7f628000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:51.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.050+0000 7f62767fc700 1 -- 172.21.15.131:0/3960981817 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000caa0 con 0x7f62880c8bf0 2023-12-26T23:12:51.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.050+0000 7f62767fc700 1 -- 172.21.15.131:0/3960981817 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628000f040 con 0x7f62880c8bf0 2023-12-26T23:12:51.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.050+0000 7f62767fc700 1 -- 172.21.15.131:0/3960981817 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000ce70 con 0x7f62880c8bf0 2023-12-26T23:12:51.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.050+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c0094f0 con 0x7f62880c8bf0 2023-12-26T23:12:51.054 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.050+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6288159460 con 0x7f62880c8bf0 2023-12-26T23:12:51.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.051+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6288159940 con 0x7f62880c8bf0 2023-12-26T23:12:51.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.053+0000 7f62767fc700 1 -- 172.21.15.131:0/3960981817 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f6280014690 con 0x7f62880c8bf0 2023-12-26T23:12:51.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.054+0000 7f62767fc700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f62680670e0 0x7f6268069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.054+0000 7f62767fc700 1 -- 172.21.15.131:0/3960981817 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f62800854a0 con 0x7f62880c8bf0 2023-12-26T23:12:51.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.054+0000 7f628659c700 1 -- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f62680670e0 msgr2=0x7f6268069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:51.055 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.054+0000 7f628659c700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f62680670e0 0x7f6268069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:51.059 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.059+0000 7f62767fc700 1 -- 172.21.15.131:0/3960981817 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6280055a60 con 0x7f62880c8bf0 2023-12-26T23:12:51.117 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:51.117+0000 7f298870d1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:12:51.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.256+0000 7f628659c700 1 -- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f62680670e0 msgr2=0x7f6268069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:51.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.256+0000 7f628659c700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f62680670e0 0x7f6268069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:51.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.304+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62880cb4f0 con 0x7f62880c8bf0 2023-12-26T23:12:51.305 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.305+0000 7f62767fc700 1 -- 172.21.15.131:0/3960981817 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4617 (secure 0 0 0) 0x7f62800555f0 con 0x7f62880c8bf0 2023-12-26T23:12:51.306 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:51.306 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8372,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2599990460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2599990460},{"type":"v1","addr":"172.21.15.190:6835","nonce":2599990460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":9565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3465421941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3465421941},{"type":"v1","addr":"172.21.15.131:6834","nonce":3465421941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":10122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3595865527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3595865527},{"type":"v1","addr":"172.21.15.131:6835","nonce":3595865527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-26T23:12:22.064606+0000","modified":"2023-12-26T23:12:25.606067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10125":{"gid":10125,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/1917956201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1917956201},{"type":"v1","addr":"172.21.15.190:6834","nonce":1917956201}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-12-26T23:12:51.307 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:51.307+0000 7f298870d1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:12:51.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f62680670e0 msgr2=0x7f6268069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:51.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f62680670e0 0x7f6268069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 msgr2=0x7f62880c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:51.308 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 0x7f62880c73b0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f6280009df0 tx=0x7f628000b040 comp rx=0 tx=0).stop 2023-12-26T23:12:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 shutdown_connections 2023-12-26T23:12:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f62680670e0 0x7f6268069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62880c8bf0 0x7f62880c73b0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6288069d10 0x7f6288148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 --2- 172.21.15.131:0/3960981817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62880c8820 0x7f6288152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.307+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 >> 172.21.15.131:0/3960981817 conn(0x7f6288068040 msgr2=0x7f62880b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.308+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 shutdown_connections 2023-12-26T23:12:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.308+0000 7f628cbff700 1 -- 172.21.15.131:0/3960981817 wait complete. 2023-12-26T23:12:51.310 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 98 2023-12-26T23:12:51.326 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:51.349 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:51.349+0000 7f087f3921c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-26T23:12:51.497 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:51.498+0000 7f298870d1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:12:51.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.521+0000 7f1adcbff700 1 Processor -- start 2023-12-26T23:12:51.522 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.521+0000 7f1adcbff700 1 -- start start 2023-12-26T23:12:51.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.523+0000 7f1adcbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 0x7f1ad80b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.523+0000 7f1adcbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1ad8159cf0 0x7f1ad80b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.523+0000 7f1adcbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1ad8069ca0 0x7f1ad81554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.523+0000 7f1adcbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad80c8bf0 con 0x7f1ad8069ca0 2023-12-26T23:12:51.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.523+0000 7f1adcbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad80c8e60 con 0x7f1ad8159cf0 2023-12-26T23:12:51.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.523+0000 7f1adcbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80c9140 con 0x7f1ad8159920 2023-12-26T23:12:51.526 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.523+0000 7f1ad6d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1ad8069ca0 0x7f1ad81554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad6d9d700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1ad8069ca0 0x7f1ad81554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34998/0 (socket says 172.21.15.131:34998) 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad6d9d700 1 -- 172.21.15.131:0/3930108003 learned_addr learned my addr 172.21.15.131:0/3930108003 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad659c700 1 --2- 172.21.15.131:0/3930108003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 0x7f1ad80b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad5d9b700 1 --2- 172.21.15.131:0/3930108003 >> v2:172.21.15.190:3300/0 conn(0x7f1ad8159cf0 0x7f1ad80b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad6d9d700 1 -- 172.21.15.131:0/3930108003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 msgr2=0x7f1ad80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad6d9d700 1 --2- 172.21.15.131:0/3930108003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 0x7f1ad80b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad6d9d700 1 -- 172.21.15.131:0/3930108003 >> v2:172.21.15.190:3300/0 conn(0x7f1ad8159cf0 msgr2=0x7f1ad80b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad6d9d700 1 --2- 172.21.15.131:0/3930108003 >> v2:172.21.15.190:3300/0 conn(0x7f1ad8159cf0 0x7f1ad80b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad6d9d700 1 -- 172.21.15.131:0/3930108003 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80c9420 con 0x7f1ad8069ca0 2023-12-26T23:12:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad6d9d700 1 --2- 172.21.15.131:0/3930108003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8069ca0 0x7f1ad81554d0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8068920 tx=0x7f1ad000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b819cb9c6b4150 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad559a700 1 -- 172.21.15.131:0/3930108003 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0010070 con 0x7f1ad8069ca0 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad559a700 1 -- 172.21.15.131:0/3930108003 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad000dd30 con 0x7f1ad8069ca0 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.524+0000 7f1ad559a700 1 -- 172.21.15.131:0/3930108003 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000dea0 con 0x7f1ad8069ca0 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.525+0000 7f1adcbff700 1 -- 172.21.15.131:0/3930108003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8069ca0 msgr2=0x7f1ad81554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.525+0000 7f1adcbff700 1 --2- 172.21.15.131:0/3930108003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8069ca0 0x7f1ad81554d0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8068920 tx=0x7f1ad000ba50 comp rx=0 tx=0).stop 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.525+0000 7f1adcbff700 1 -- 172.21.15.131:0/3930108003 shutdown_connections 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.525+0000 7f1adcbff700 1 --2- 172.21.15.131:0/3930108003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8069ca0 0x7f1ad81554d0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.525+0000 7f1adcbff700 1 --2- 172.21.15.131:0/3930108003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 0x7f1ad80b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.525+0000 7f1adcbff700 1 --2- 172.21.15.131:0/3930108003 >> v2:172.21.15.190:3300/0 conn(0x7f1ad8159cf0 0x7f1ad80b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.525+0000 7f1adcbff700 1 -- 172.21.15.131:0/3930108003 >> 172.21.15.131:0/3930108003 conn(0x7f1ad8068040 msgr2=0x7f1ad8069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.525+0000 7f1adcbff700 1 -- 172.21.15.131:0/3930108003 shutdown_connections 2023-12-26T23:12:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.525+0000 7f1adcbff700 1 -- 172.21.15.131:0/3930108003 wait complete. 2023-12-26T23:12:51.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.527+0000 7f1adcbff700 1 Processor -- start 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.527+0000 7f1adcbff700 1 -- start start 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1adcbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8069ca0 0x7f1ad814db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1adcbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 0x7f1ad814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1adcbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8159cf0 0x7f1ad8150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1adcbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad814edc0 con 0x7f1ad8159cf0 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1adcbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad814c5f0 con 0x7f1ad8069ca0 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1adcbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad81506d0 con 0x7f1ad8159920 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad6d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8159cf0 0x7f1ad8150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad6d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8159cf0 0x7f1ad8150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35014/0 (socket says 172.21.15.131:35014) 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8069ca0 0x7f1ad814db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad6d9d700 1 -- 172.21.15.131:0/1816067502 learned_addr learned my addr 172.21.15.131:0/1816067502 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad5d9b700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 0x7f1ad814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad6d9d700 1 -- 172.21.15.131:0/1816067502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 msgr2=0x7f1ad814c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad6d9d700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 0x7f1ad814c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad6d9d700 1 -- 172.21.15.131:0/1816067502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8069ca0 msgr2=0x7f1ad814db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad6d9d700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8069ca0 0x7f1ad814db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.529+0000 7f1ad6d9d700 1 -- 172.21.15.131:0/1816067502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad000bcf0 con 0x7f1ad8159cf0 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.530+0000 7f1ad6d9d700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8159cf0 0x7f1ad8150190 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80688f0 tx=0x7f1ad001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.530+0000 7f1ac67fc700 1 -- 172.21.15.131:0/1816067502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000aa00 con 0x7f1ad8159cf0 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.530+0000 7f1ac67fc700 1 -- 172.21.15.131:0/1816067502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad000ab80 con 0x7f1ad8159cf0 2023-12-26T23:12:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.530+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad814e1b0 con 0x7f1ad8159cf0 2023-12-26T23:12:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.530+0000 7f1ac67fc700 1 -- 172.21.15.131:0/1816067502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad001e440 con 0x7f1ad8159cf0 2023-12-26T23:12:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.530+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad814e770 con 0x7f1ad8159cf0 2023-12-26T23:12:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.531+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad80657e0 con 0x7f1ad8159cf0 2023-12-26T23:12:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.532+0000 7f1ac67fc700 1 -- 172.21.15.131:0/1816067502 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f1ad0003040 con 0x7f1ad8159cf0 2023-12-26T23:12:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.533+0000 7f1ac67fc700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ab80670e0 0x7f1ab8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.533+0000 7f1ac67fc700 1 -- 172.21.15.131:0/1816067502 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1ad00863d0 con 0x7f1ad8159cf0 2023-12-26T23:12:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.533+0000 7f1ad659c700 1 -- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ab80670e0 msgr2=0x7f1ab8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.533+0000 7f1ad659c700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ab80670e0 0x7f1ab8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:51.537 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.537+0000 7f1ac67fc700 1 -- 172.21.15.131:0/1816067502 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ad0052fa0 con 0x7f1ad8159cf0 2023-12-26T23:12:51.551 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:51.551+0000 7f087f3921c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-26T23:12:51.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.734+0000 7f1ad659c700 1 -- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ab80670e0 msgr2=0x7f1ab8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:51.734 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.734+0000 7f1ad659c700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ab80670e0 0x7f1ab8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:51.741 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.741+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ad80c9700 con 0x7f1ad8159cf0 2023-12-26T23:12:51.743 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.742+0000 7f1ac67fc700 1 -- 172.21.15.131:0/1816067502 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+16588 (secure 0 0 0) 0x7f1ad00565c0 con 0x7f1ad8159cf0 2023-12-26T23:12:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:51.743 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":211,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:42.707975+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-26T23:12:20.250102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-26T23:12:21.042553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3395260924":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/3886778096":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/3508392670":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/1340664010":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6837/92228":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6836/92228":"2023-12-27T23:12:42.707929+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ab80670e0 msgr2=0x7f1ab8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ab80670e0 0x7f1ab8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8159cf0 msgr2=0x7f1ad8150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8159cf0 0x7f1ad8150190 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80688f0 tx=0x7f1ad001a000 comp rx=0 tx=0).stop 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 shutdown_connections 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1ab80670e0 0x7f1ab8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ad8159cf0 0x7f1ad8150190 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ad8069ca0 0x7f1ad814db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 --2- 172.21.15.131:0/1816067502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ad8159920 0x7f1ad814c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 >> 172.21.15.131:0/1816067502 conn(0x7f1ad8068040 msgr2=0x7f1ad81584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 shutdown_connections 2023-12-26T23:12:51.747 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:51.745+0000 7f1adcbff700 1 -- 172.21.15.131:0/1816067502 wait complete. 2023-12-26T23:12:51.750 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:51.750+0000 7f087f3921c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-26T23:12:51.765 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-26T23:12:52.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.005+0000 7f34061ff700 1 Processor -- start 2023-12-26T23:12:52.005 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.005+0000 7f34061ff700 1 -- start start 2023-12-26T23:12:52.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f34061ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 0x7f34000cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f34061ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34000b00e0 0x7f340006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f34061ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f34000b0550 0x7f340015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f34061ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34000680f0 con 0x7f34000b0550 2023-12-26T23:12:52.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f34061ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34000b0d70 con 0x7f34000b00e0 2023-12-26T23:12:52.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f34061ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000675b0 con 0x7f34000cc580 2023-12-26T23:12:52.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f33fffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f34000b0550 0x7f340015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f33feffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34000b00e0 0x7f340006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f33ff7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 0x7f34000cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f33fffff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f34000b0550 0x7f340015d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35020/0 (socket says 172.21.15.131:35020) 2023-12-26T23:12:52.008 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f33fffff700 1 -- 172.21.15.131:0/1570337326 learned_addr learned my addr 172.21.15.131:0/1570337326 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:52.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f33fffff700 1 -- 172.21.15.131:0/1570337326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 msgr2=0x7f34000cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:52.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f33fffff700 1 --2- 172.21.15.131:0/1570337326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 0x7f34000cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f33fffff700 1 -- 172.21.15.131:0/1570337326 >> v2:172.21.15.190:3300/0 conn(0x7f34000b00e0 msgr2=0x7f340006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:52.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.007+0000 7f33fffff700 1 --2- 172.21.15.131:0/1570337326 >> v2:172.21.15.190:3300/0 conn(0x7f34000b00e0 0x7f340006e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.008+0000 7f33fffff700 1 -- 172.21.15.131:0/1570337326 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3400159c30 con 0x7f34000b0550 2023-12-26T23:12:52.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.008+0000 7f33ff7fe700 1 --2- 172.21.15.131:0/1570337326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 0x7f34000cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:52.009 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.008+0000 7f33feffd700 1 --2- 172.21.15.131:0/1570337326 >> v2:172.21.15.190:3300/0 conn(0x7f34000b00e0 0x7f340006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.008+0000 7f33fffff700 1 --2- 172.21.15.131:0/1570337326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 0x7f340015d9b0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f340006b6a0 tx=0x7f33f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16aafbd20c7eaee server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.008+0000 7f33fe7fc700 1 -- 172.21.15.131:0/1570337326 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f800e070 con 0x7f34000b0550 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.008+0000 7f33fe7fc700 1 -- 172.21.15.131:0/1570337326 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f800b970 con 0x7f34000b0550 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.008+0000 7f33fe7fc700 1 -- 172.21.15.131:0/1570337326 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f80092e0 con 0x7f34000b0550 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.009+0000 7f34061ff700 1 -- 172.21.15.131:0/1570337326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 msgr2=0x7f340015d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.009+0000 7f34061ff700 1 --2- 172.21.15.131:0/1570337326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 0x7f340015d9b0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f340006b6a0 tx=0x7f33f8009840 comp rx=0 tx=0).stop 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.009+0000 7f34061ff700 1 -- 172.21.15.131:0/1570337326 shutdown_connections 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.009+0000 7f34061ff700 1 --2- 172.21.15.131:0/1570337326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 0x7f340015d9b0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.009+0000 7f34061ff700 1 --2- 172.21.15.131:0/1570337326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 0x7f34000cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.009+0000 7f34061ff700 1 --2- 172.21.15.131:0/1570337326 >> v2:172.21.15.190:3300/0 conn(0x7f34000b00e0 0x7f340006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.009+0000 7f34061ff700 1 -- 172.21.15.131:0/1570337326 >> 172.21.15.131:0/1570337326 conn(0x7f34000686c0 msgr2=0x7f3400068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.009+0000 7f34061ff700 1 -- 172.21.15.131:0/1570337326 shutdown_connections 2023-12-26T23:12:52.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.009+0000 7f34061ff700 1 -- 172.21.15.131:0/1570337326 wait complete. 2023-12-26T23:12:52.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.011+0000 7f34061ff700 1 Processor -- start 2023-12-26T23:12:52.011 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.011+0000 7f34061ff700 1 -- start start 2023-12-26T23:12:52.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.012+0000 7f34061ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34000b00e0 0x7f340014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.012+0000 7f34061ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 0x7f34000ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.012+0000 7f34061ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 0x7f34000a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.012+0000 7f34061ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000a8970 con 0x7f34000b0550 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.012+0000 7f34061ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000a5770 con 0x7f34000b00e0 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.012+0000 7f34061ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340016e640 con 0x7f34000cc580 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33feffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 0x7f34000ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33ff7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34000b00e0 0x7f340014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33ff7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34000b00e0 0x7f340014cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:50004/0 (socket says 172.21.15.131:50004) 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33feffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 0x7f34000ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35030/0 (socket says 172.21.15.131:35030) 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33ff7fe700 1 -- 172.21.15.131:0/835565108 learned_addr learned my addr 172.21.15.131:0/835565108 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33fffff700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 0x7f34000a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33ff7fe700 1 -- 172.21.15.131:0/835565108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 msgr2=0x7f34000a8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33ff7fe700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 0x7f34000a8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33ff7fe700 1 -- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 msgr2=0x7f34000ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:52.014 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33ff7fe700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 0x7f34000ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33ff7fe700 1 -- 172.21.15.131:0/835565108 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009420 con 0x7f34000b00e0 2023-12-26T23:12:52.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.013+0000 7f33feffd700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 0x7f34000ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:52.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.014+0000 7f33ff7fe700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34000b00e0 0x7f340014cd90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f33f000ec80 tx=0x7f33f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:52.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.014+0000 7f33e77fe700 1 -- 172.21.15.131:0/835565108 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000c850 con 0x7f34000b00e0 2023-12-26T23:12:52.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.014+0000 7f33e77fe700 1 -- 172.21.15.131:0/835565108 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f000c9b0 con 0x7f34000b00e0 2023-12-26T23:12:52.015 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.014+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f0009420 con 0x7f34000b00e0 2023-12-26T23:12:52.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.014+0000 7f33e77fe700 1 -- 172.21.15.131:0/835565108 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000cd30 con 0x7f34000b00e0 2023-12-26T23:12:52.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.014+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f8009ae0 con 0x7f34000b00e0 2023-12-26T23:12:52.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.015+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340016e810 con 0x7f34000b00e0 2023-12-26T23:12:52.016 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.016+0000 7f33e77fe700 1 -- 172.21.15.131:0/835565108 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f33f0005a00 con 0x7f34000b00e0 2023-12-26T23:12:52.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.017+0000 7f33e77fe700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f33ec0670e0 0x7f33ec069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.017+0000 7f33e77fe700 1 -- 172.21.15.131:0/835565108 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f33f0084e80 con 0x7f34000b00e0 2023-12-26T23:12:52.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.017+0000 7f33feffd700 1 -- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f33ec0670e0 msgr2=0x7f33ec069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:52.017 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.017+0000 7f33feffd700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f33ec0670e0 0x7f33ec069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:52.024 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.024+0000 7f33e77fe700 1 -- 172.21.15.131:0/835565108 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33f0055690 con 0x7f34000b00e0 2023-12-26T23:12:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.218+0000 7f33feffd700 1 -- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f33ec0670e0 msgr2=0x7f33ec069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.218+0000 7f33feffd700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f33ec0670e0 0x7f33ec069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.268+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f34000657e0 con 0x7f34000b00e0 2023-12-26T23:12:52.270 INFO:tasks.ceph.mon.a.smithi131.stderr:2023-12-26T23:12:52.270+0000 7fd02bdf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-26T23:12:52.275 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.275+0000 7f33e77fe700 1 -- 172.21.15.131:0/835565108 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) v1 ==== 161+0+0 (secure 0 0 0) 0x7f33f0021090 con 0x7f34000b00e0 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f33ec0670e0 msgr2=0x7f33ec069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f33ec0670e0 0x7f33ec069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34000b00e0 msgr2=0x7f340014cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34000b00e0 0x7f340014cd90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f33f000ec80 tx=0x7f33f00091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 shutdown_connections 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f33ec0670e0 0x7f33ec069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34000cc580 0x7f34000a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34000b00e0 0x7f340014cd90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 --2- 172.21.15.131:0/835565108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34000b0550 0x7f34000ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.277+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 >> 172.21.15.131:0/835565108 conn(0x7f34000686c0 msgr2=0x7f34000cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.278+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 shutdown_connections 2023-12-26T23:12:52.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.278+0000 7f34061ff700 1 -- 172.21.15.131:0/835565108 wait complete. 2023-12-26T23:12:52.279 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-26T23:12:52.280 INFO:tasks.ceph.mds.d.smithi190.stderr: -2532> 2023-12-26T23:12:27.563+0000 7f4e285f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-26T23:12:52.298 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-26T23:12:52.321 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-12-26T23:12:52.361 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:52.361+0000 7f298870d1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:12:52.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.493+0000 7f594f3ff700 1 Processor -- start 2023-12-26T23:12:52.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.493+0000 7f594f3ff700 1 -- start start 2023-12-26T23:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.494+0000 7f594f3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f59480c8bf0 0x7f59480b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.494+0000 7f594f3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5948069d10 0x7f59480b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.494+0000 7f594f3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59480caa10 con 0x7f59480c8820 2023-12-26T23:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594f3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5948157a80 con 0x7f59480c8bf0 2023-12-26T23:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594f3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59480cac50 con 0x7f5948069d10 2023-12-26T23:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594c99a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f59480c8bf0 0x7f59480b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594d19b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f59480c8820 0x7f59480ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59750/0 (socket says 172.21.15.131:59750) 2023-12-26T23:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594d19b700 1 -- 172.21.15.131:0/3413938797 learned_addr learned my addr 172.21.15.131:0/3413938797 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594d99c700 1 --2- 172.21.15.131:0/3413938797 >> v2:172.21.15.190:3300/0 conn(0x7f5948069d10 0x7f59480b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594c99a700 1 -- 172.21.15.131:0/3413938797 >> v2:172.21.15.190:3300/0 conn(0x7f5948069d10 msgr2=0x7f59480b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594c99a700 1 --2- 172.21.15.131:0/3413938797 >> v2:172.21.15.190:3300/0 conn(0x7f5948069d10 0x7f59480b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594c99a700 1 -- 172.21.15.131:0/3413938797 >> v1:172.21.15.131:6789/0 conn(0x7f59480c8820 legacy=0x7f59480ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f594c99a700 1 -- 172.21.15.131:0/3413938797 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480caf30 con 0x7f59480c8bf0 2023-12-26T23:12:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.495+0000 7f5947fff700 1 -- 172.21.15.131:0/3413938797 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493824947 0 0) 0x7f5938001110 con 0x7f59480c8820 2023-12-26T23:12:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f594c99a700 1 --2- 172.21.15.131:0/3413938797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8bf0 0x7f59480b12e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f5948070f80 tx=0x7f593c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820f188ed1f055a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f5947fff700 1 -- 172.21.15.131:0/3413938797 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00b6b0 con 0x7f59480c8bf0 2023-12-26T23:12:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f5947fff700 1 -- 172.21.15.131:0/3413938797 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c00b810 con 0x7f59480c8bf0 2023-12-26T23:12:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f5947fff700 1 -- 172.21.15.131:0/3413938797 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00bbb0 con 0x7f59480c8bf0 2023-12-26T23:12:52.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f594f3ff700 1 -- 172.21.15.131:0/3413938797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8bf0 msgr2=0x7f59480b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:52.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f594f3ff700 1 --2- 172.21.15.131:0/3413938797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8bf0 0x7f59480b12e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f5948070f80 tx=0x7f593c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f594f3ff700 1 -- 172.21.15.131:0/3413938797 shutdown_connections 2023-12-26T23:12:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f594f3ff700 1 --2- 172.21.15.131:0/3413938797 >> v2:172.21.15.190:3300/0 conn(0x7f5948069d10 0x7f59480b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f594f3ff700 1 --2- 172.21.15.131:0/3413938797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8bf0 0x7f59480b12e0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.496+0000 7f594f3ff700 1 -- 172.21.15.131:0/3413938797 >> 172.21.15.131:0/3413938797 conn(0x7f5948068040 msgr2=0x7f5948069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.497+0000 7f594f3ff700 1 -- 172.21.15.131:0/3413938797 shutdown_connections 2023-12-26T23:12:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.497+0000 7f594f3ff700 1 -- 172.21.15.131:0/3413938797 wait complete. 2023-12-26T23:12:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.498+0000 7f594f3ff700 1 Processor -- start 2023-12-26T23:12:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.498+0000 7f594f3ff700 1 -- start start 2023-12-26T23:12:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594f3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5948069d10 0x7f594814be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594f3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8820 0x7f59480ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594f3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59480c8bf0 0x7f59480ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594f3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59480c5cd0 con 0x7f59480c8820 2023-12-26T23:12:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594f3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594814c370 con 0x7f5948069d10 2023-12-26T23:12:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594f3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594816ab00 con 0x7f59480c8bf0 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594c99a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8820 0x7f59480ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594c99a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8820 0x7f59480ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35058/0 (socket says 172.21.15.131:35058) 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594c99a700 1 -- 172.21.15.131:0/3471335500 learned_addr learned my addr 172.21.15.131:0/3471335500 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594d19b700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5948069d10 0x7f594814be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.499+0000 7f594d99c700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59480c8bf0 0x7f59480ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.500+0000 7f594c99a700 1 -- 172.21.15.131:0/3471335500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59480c8bf0 msgr2=0x7f59480ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.500+0000 7f594c99a700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59480c8bf0 0x7f59480ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.500+0000 7f594c99a700 1 -- 172.21.15.131:0/3471335500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5948069d10 msgr2=0x7f594814be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.500+0000 7f594c99a700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5948069d10 0x7f594814be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.500+0000 7f594c99a700 1 -- 172.21.15.131:0/3471335500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c009420 con 0x7f59480c8820 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.500+0000 7f594d19b700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5948069d10 0x7f594814be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.500+0000 7f594c99a700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8820 0x7f59480ae030 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f593c013000 tx=0x7f593c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.500+0000 7f59457fa700 1 -- 172.21.15.131:0/3471335500 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c000d10 con 0x7f59480c8820 2023-12-26T23:12:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.501+0000 7f59457fa700 1 -- 172.21.15.131:0/3471335500 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c00bb90 con 0x7f59480c8820 2023-12-26T23:12:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.501+0000 7f594f3ff700 1 -- 172.21.15.131:0/3471335500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59480c53a0 con 0x7f59480c8820 2023-12-26T23:12:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.501+0000 7f59457fa700 1 -- 172.21.15.131:0/3471335500 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c011670 con 0x7f59480c8820 2023-12-26T23:12:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.501+0000 7f594f3ff700 1 -- 172.21.15.131:0/3471335500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59480c5960 con 0x7f59480c8820 2023-12-26T23:12:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.502+0000 7f594f3ff700 1 -- 172.21.15.131:0/3471335500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59480657e0 con 0x7f59480c8820 2023-12-26T23:12:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.503+0000 7f59457fa700 1 -- 172.21.15.131:0/3471335500 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 113) v1 ==== 84901+0+0 (secure 0 0 0) 0x7f593c011aa0 con 0x7f59480c8820 2023-12-26T23:12:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.504+0000 7f59457fa700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 0x7f5934069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.504+0000 7f59457fa700 1 -- 172.21.15.131:0/3471335500 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6569+0+0 (secure 0 0 0) 0x7f593c086560 con 0x7f59480c8820 2023-12-26T23:12:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.504+0000 7f594d19b700 1 -- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 msgr2=0x7f5934069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.504+0000 7f594d19b700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 0x7f5934069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-26T23:12:52.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.508+0000 7f59457fa700 1 -- 172.21.15.131:0/3471335500 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f593c0531f0 con 0x7f59480c8820 2023-12-26T23:12:52.642 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:52.643+0000 7f298870d1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:52.647 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:52.647+0000 7f087f3921c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-26T23:12:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.705+0000 7f594d19b700 1 -- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 msgr2=0x7f5934069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:52.705 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.705+0000 7f594d19b700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 0x7f5934069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-26T23:12:52.736 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:52.736+0000 7f298870d1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:52.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:52.753+0000 7f594f3ff700 1 -- 172.21.15.131:0/3471335500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f59480c5e10 con 0x7f59480c8820 2023-12-26T23:12:52.931 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:52.931+0000 7f087f3921c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-26T23:12:52.937 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-12-26T23:12:52.937+0000 7f298870d1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:12:53.026 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:53.026+0000 7f087f3921c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-26T23:12:53.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.106+0000 7f594d19b700 1 -- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 msgr2=0x7f5934069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6836/92228 2023-12-26T23:12:53.106 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.106+0000 7f594d19b700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 0x7f5934069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-26T23:12:53.223 INFO:tasks.ceph.mgr.y.smithi131.stderr:2023-12-26T23:12:53.223+0000 7f087f3921c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-26T23:12:53.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.230+0000 7f59457fa700 1 -- 172.21.15.131:0/3471335500 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 114) v1 ==== 84453+0+0 (secure 0 0 0) 0x7f593c011d50 con 0x7f59480c8820 2023-12-26T23:12:53.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.231+0000 7f59457fa700 1 -- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 msgr2=0x7f5934069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-26T23:12:53.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.231+0000 7f59457fa700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 0x7f5934069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.276 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.275+0000 7f59457fa700 1 -- 172.21.15.131:0/3471335500 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v100) v1 ==== 104+0+0 (secure 0 0 0) 0x7f593c027210 con 0x7f59480c8820 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.278+0000 7f594f3ff700 1 -- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8820 msgr2=0x7f59480ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.278+0000 7f594f3ff700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8820 0x7f59480ae030 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f593c013000 tx=0x7f593c017000 comp rx=0 tx=0).stop 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.278+0000 7f594f3ff700 1 -- 172.21.15.131:0/3471335500 shutdown_connections 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.278+0000 7f594f3ff700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f59340670e0 0x7f5934069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.278+0000 7f594f3ff700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59480c8bf0 0x7f59480ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.278+0000 7f594f3ff700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5948069d10 0x7f594814be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.278+0000 7f594f3ff700 1 --2- 172.21.15.131:0/3471335500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59480c8820 0x7f59480ae030 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.278+0000 7f594f3ff700 1 -- 172.21.15.131:0/3471335500 >> 172.21.15.131:0/3471335500 conn(0x7f5948068040 msgr2=0x7f5948158600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.279+0000 7f594f3ff700 1 -- 172.21.15.131:0/3471335500 shutdown_connections 2023-12-26T23:12:53.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.279+0000 7f594f3ff700 1 -- 172.21.15.131:0/3471335500 wait complete. 2023-12-26T23:12:53.302 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-26T23:12:53.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.499+0000 7f0c449ff700 1 Processor -- start 2023-12-26T23:12:53.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.499+0000 7f0c449ff700 1 -- start start 2023-12-26T23:12:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.500+0000 7f0c449ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 0x7f0c40069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.500+0000 7f0c449ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0c400cc590 0x7f0c400cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.500+0000 7f0c449ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0c400cb340 0x7f0c400c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.500+0000 7f0c449ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c400cb7c0 con 0x7f0c400cc590 2023-12-26T23:12:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.500+0000 7f0c449ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c40069640 con 0x7f0c400cb340 2023-12-26T23:12:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.500+0000 7f0c449ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c40068750 con 0x7f0c40142f00 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3dd9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0c400cc590 0x7f0c400cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3e59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 0x7f0c40069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3dd9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0c400cc590 0x7f0c400cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35120/0 (socket says 172.21.15.131:35120) 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3dd9b700 1 -- 172.21.15.131:0/2119744639 learned_addr learned my addr 172.21.15.131:0/2119744639 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3ed9d700 1 --2- 172.21.15.131:0/2119744639 >> v2:172.21.15.190:3300/0 conn(0x7f0c400cb340 0x7f0c400c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3dd9b700 1 -- 172.21.15.131:0/2119744639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 msgr2=0x7f0c40069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3dd9b700 1 --2- 172.21.15.131:0/2119744639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 0x7f0c40069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3dd9b700 1 -- 172.21.15.131:0/2119744639 >> v2:172.21.15.190:3300/0 conn(0x7f0c400cb340 msgr2=0x7f0c400c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3dd9b700 1 --2- 172.21.15.131:0/2119744639 >> v2:172.21.15.190:3300/0 conn(0x7f0c400cb340 0x7f0c400c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3dd9b700 1 -- 172.21.15.131:0/2119744639 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c40068a30 con 0x7f0c400cc590 2023-12-26T23:12:53.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.501+0000 7f0c3dd9b700 1 --2- 172.21.15.131:0/2119744639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 0x7f0c400cae00 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0c4006f560 tx=0x7f0c34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf8c0b9948a685b server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c3d59a700 1 -- 172.21.15.131:0/2119744639 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400b640 con 0x7f0c400cc590 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c3d59a700 1 -- 172.21.15.131:0/2119744639 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3400b7a0 con 0x7f0c400cc590 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c3d59a700 1 -- 172.21.15.131:0/2119744639 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400bb60 con 0x7f0c400cc590 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c449ff700 1 -- 172.21.15.131:0/2119744639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 msgr2=0x7f0c400cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2119744639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 0x7f0c400cae00 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0c4006f560 tx=0x7f0c34009250 comp rx=0 tx=0).stop 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c449ff700 1 -- 172.21.15.131:0/2119744639 shutdown_connections 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2119744639 >> v2:172.21.15.190:3300/0 conn(0x7f0c400cb340 0x7f0c400c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2119744639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 0x7f0c40069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2119744639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 0x7f0c400cae00 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c449ff700 1 -- 172.21.15.131:0/2119744639 >> 172.21.15.131:0/2119744639 conn(0x7f0c40063cd0 msgr2=0x7f0c4006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c449ff700 1 -- 172.21.15.131:0/2119744639 shutdown_connections 2023-12-26T23:12:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.502+0000 7f0c449ff700 1 -- 172.21.15.131:0/2119744639 wait complete. 2023-12-26T23:12:53.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.504+0000 7f0c449ff700 1 Processor -- start 2023-12-26T23:12:53.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.504+0000 7f0c449ff700 1 -- start start 2023-12-26T23:12:53.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c449ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400cb340 0x7f0c400b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:53.508 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c449ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 0x7f0c4013ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c449ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 0x7f0c4015f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c449ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c400c7bf0 con 0x7f0c400cc590 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c449ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c40140460 con 0x7f0c400cb340 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c449ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c400c6fe0 con 0x7f0c40142f00 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 0x7f0c4013ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3dd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 0x7f0c4013ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35130/0 (socket says 172.21.15.131:35130) 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3dd9b700 1 -- 172.21.15.131:0/2735360445 learned_addr learned my addr 172.21.15.131:0/2735360445 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3e59c700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400cb340 0x7f0c400b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3ed9d700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 0x7f0c4015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3dd9b700 1 -- 172.21.15.131:0/2735360445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 msgr2=0x7f0c4015f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3dd9b700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 0x7f0c4015f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3dd9b700 1 -- 172.21.15.131:0/2735360445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400cb340 msgr2=0x7f0c400b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3dd9b700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400cb340 0x7f0c400b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3dd9b700 1 -- 172.21.15.131:0/2735360445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c340094f0 con 0x7f0c400cc590 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.505+0000 7f0c3ed9d700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 0x7f0c4015f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.506+0000 7f0c3dd9b700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 0x7f0c4013ff20 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f0c4006f3d0 tx=0x7f0c34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.506+0000 7f0c2e7fc700 1 -- 172.21.15.131:0/2735360445 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34000d10 con 0x7f0c400cc590 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.506+0000 7f0c2e7fc700 1 -- 172.21.15.131:0/2735360445 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3400bb20 con 0x7f0c400cc590 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.506+0000 7f0c2e7fc700 1 -- 172.21.15.131:0/2735360445 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34011630 con 0x7f0c400cc590 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.506+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c400c72c0 con 0x7f0c400cc590 2023-12-26T23:12:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.506+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c400c7880 con 0x7f0c400cc590 2023-12-26T23:12:53.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.507+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c400c7d60 con 0x7f0c400cc590 2023-12-26T23:12:53.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.508+0000 7f0c2e7fc700 1 -- 172.21.15.131:0/2735360445 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 114) v1 ==== 84453+0+0 (secure 0 0 0) 0x7f0c340119a0 con 0x7f0c400cc590 2023-12-26T23:12:53.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.509+0000 7f0c2e7fc700 1 -- 172.21.15.131:0/2735360445 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6870+0+0 (secure 0 0 0) 0x7f0c34085860 con 0x7f0c400cc590 2023-12-26T23:12:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.513+0000 7f0c2e7fc700 1 -- 172.21.15.131:0/2735360445 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c34052b30 con 0x7f0c400cc590 2023-12-26T23:12:53.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:53.709+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c400717f0 con 0x7f0c400cc590 2023-12-26T23:12:54.235 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.235+0000 7f0c2e7fc700 1 -- 172.21.15.131:0/2735360445 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 115) v1 ==== 84580+0+0 (secure 0 0 0) 0x7f0c340525e0 con 0x7f0c400cc590 2023-12-26T23:12:54.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.235+0000 7f0c2e7fc700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c2006ca40 0x7f0c2006ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:54.236 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.236+0000 7f0c3e59c700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c2006ca40 0x7f0c2006ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:54.238 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.238+0000 7f0c3e59c700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c2006ca40 0x7f0c2006ee20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009ca0 tx=0x7f0c30008040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:54.277 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.277+0000 7f0c2e7fc700 1 -- 172.21.15.131:0/2735360445 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v214) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0c3402eaf0 con 0x7f0c400cc590 2023-12-26T23:12:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.280+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c2006ca40 msgr2=0x7f0c2006ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.280+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c2006ca40 0x7f0c2006ee20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009ca0 tx=0x7f0c30008040 comp rx=0 tx=0).stop 2023-12-26T23:12:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.280+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 msgr2=0x7f0c4013ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.280+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 0x7f0c4013ff20 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f0c4006f3d0 tx=0x7f0c34017000 comp rx=0 tx=0).stop 2023-12-26T23:12:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.281+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 shutdown_connections 2023-12-26T23:12:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.281+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f0c2006ca40 0x7f0c2006ee20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.281+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40142f00 0x7f0c4015f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.281+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400cb340 0x7f0c400b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.281+0000 7f0c449ff700 1 --2- 172.21.15.131:0/2735360445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c400cc590 0x7f0c4013ff20 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.281+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 >> 172.21.15.131:0/2735360445 conn(0x7f0c40063cd0 msgr2=0x7f0c400c31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:54.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.281+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 shutdown_connections 2023-12-26T23:12:54.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.281+0000 7f0c449ff700 1 -- 172.21.15.131:0/2735360445 wait complete. 2023-12-26T23:12:54.283 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2023-12-26T23:12:54.302 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-26T23:12:54.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.497+0000 7faa581ff700 1 Processor -- start 2023-12-26T23:12:54.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.497+0000 7faa581ff700 1 -- start start 2023-12-26T23:12:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.498+0000 7faa581ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faa500c9110 0x7faa5015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.498+0000 7faa581ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa500af040 con 0x7faa500c8d40 2023-12-26T23:12:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.498+0000 7faa581ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa500ae0e0 con 0x7faa50156e50 2023-12-26T23:12:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.498+0000 7faa581ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa501579c0 con 0x7faa500c9110 2023-12-26T23:12:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa55f9b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7faa500c8d40 0x7faa500ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59766/0 (socket says 172.21.15.131:59766) 2023-12-26T23:12:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa55f9b700 1 -- 172.21.15.131:0/2584040860 learned_addr learned my addr 172.21.15.131:0/2584040860 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa5579a700 1 --2- 172.21.15.131:0/2584040860 >> v2:172.21.15.190:3300/0 conn(0x7faa500c9110 0x7faa5015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542639472 0 0) 0x7faa500af040 con 0x7faa500c8d40 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa3c003570 con 0x7faa500c8d40 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979557479 0 0) 0x7faa500ae0e0 con 0x7faa50156e50 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa500af040 con 0x7faa50156e50 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1584719073 0 0) 0x7faa3c003570 con 0x7faa500c8d40 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 >> v2:172.21.15.190:3300/0 conn(0x7faa500c9110 msgr2=0x7faa5015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 --2- 172.21.15.131:0/2584040860 >> v2:172.21.15.190:3300/0 conn(0x7faa500c9110 0x7faa5015bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 >> v1:172.21.15.190:6789/0 conn(0x7faa50156e50 legacy=0x7faa501572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa500ae3c0 con 0x7faa500c8d40 2023-12-26T23:12:54.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7faa4c0013f0 con 0x7faa500c8d40 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.499+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 >> v1:172.21.15.131:6789/0 conn(0x7faa500c8d40 legacy=0x7faa500ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa54f99700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa3c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa54f99700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3c004810 0x7faa3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa54f99700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa3c005210 0x7faa3c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa501579c0 con 0x7faa3c003c40 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c00c130 con 0x7faa3c005210 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c00c2c0 con 0x7faa3c004810 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa55f9b700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa3c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa5579a700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3c004810 0x7faa3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa5679c700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa3c005210 0x7faa3c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:54.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa55f9b700 1 -- 172.21.15.131:0/2584040860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3c004810 msgr2=0x7faa3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa55f9b700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3c004810 0x7faa3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa55f9b700 1 -- 172.21.15.131:0/2584040860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa3c005210 msgr2=0x7faa3c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa55f9b700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa3c005210 0x7faa3c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa55f9b700 1 -- 172.21.15.131:0/2584040860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c003570 con 0x7faa3c003c40 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa5679c700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa3c005210 0x7faa3c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa5679c700 1 -- 172.21.15.131:0/2584040860 reap_dead start 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.500+0000 7faa55f9b700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa3c004030 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00c1c0 tx=0x7faa4c001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af97d578f4236148 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00cab0 con 0x7faa3c003c40 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c00cc30 con 0x7faa3c003c40 2023-12-26T23:12:54.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa54f99700 1 -- 172.21.15.131:0/2584040860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c005920 con 0x7faa3c003c40 2023-12-26T23:12:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa581ff700 1 -- 172.21.15.131:0/2584040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 msgr2=0x7faa3c004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa581ff700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa3c004030 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00c1c0 tx=0x7faa4c001cc0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa581ff700 1 -- 172.21.15.131:0/2584040860 shutdown_connections 2023-12-26T23:12:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa581ff700 1 --2- 172.21.15.131:0/2584040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa3c004030 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa581ff700 1 -- 172.21.15.131:0/2584040860 >> 172.21.15.131:0/2584040860 conn(0x7faa500675b0 msgr2=0x7faa500679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa581ff700 1 -- 172.21.15.131:0/2584040860 shutdown_connections 2023-12-26T23:12:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.501+0000 7faa581ff700 1 -- 172.21.15.131:0/2584040860 wait complete. 2023-12-26T23:12:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.503+0000 7faa581ff700 1 Processor -- start 2023-12-26T23:12:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.503+0000 7faa581ff700 1 -- start start 2023-12-26T23:12:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa581ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa5006e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa581ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa50168e70 0x7faa501692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa581ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa50166a40 0x7faa50166eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa581ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50169820 con 0x7faa3c003c40 2023-12-26T23:12:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa581ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5006d470 con 0x7faa50168e70 2023-12-26T23:12:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa581ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5006e2d0 con 0x7faa50166a40 2023-12-26T23:12:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa55f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa5006e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa55f9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa5006e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35150/0 (socket says 172.21.15.131:35150) 2023-12-26T23:12:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa55f9b700 1 -- 172.21.15.131:0/2687255629 learned_addr learned my addr 172.21.15.131:0/2687255629 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa5679c700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa50166a40 0x7faa50166eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa5579a700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa50168e70 0x7faa501692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa55f9b700 1 -- 172.21.15.131:0/2687255629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa50166a40 msgr2=0x7faa50166eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa55f9b700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa50166a40 0x7faa50166eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa55f9b700 1 -- 172.21.15.131:0/2687255629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa50168e70 msgr2=0x7faa501692e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa55f9b700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa50168e70 0x7faa501692e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa55f9b700 1 -- 172.21.15.131:0/2687255629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5006d6c0 con 0x7faa3c003c40 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.504+0000 7faa5679c700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa50166a40 0x7faa50166eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.505+0000 7faa55f9b700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa5006e760 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7faa4c007b50 tx=0x7faa4c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.505+0000 7faa45ffb700 1 -- 172.21.15.131:0/2687255629 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c001a00 con 0x7faa3c003c40 2023-12-26T23:12:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.505+0000 7faa45ffb700 1 -- 172.21.15.131:0/2687255629 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c000bc0 con 0x7faa3c003c40 2023-12-26T23:12:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.505+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5006d9a0 con 0x7faa3c003c40 2023-12-26T23:12:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.505+0000 7faa45ffb700 1 -- 172.21.15.131:0/2687255629 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0072e0 con 0x7faa3c003c40 2023-12-26T23:12:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.505+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5006df60 con 0x7faa3c003c40 2023-12-26T23:12:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.506+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5006e5b0 con 0x7faa3c003c40 2023-12-26T23:12:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.507+0000 7faa45ffb700 1 -- 172.21.15.131:0/2687255629 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 115) v1 ==== 84580+0+0 (secure 0 0 0) 0x7faa4c0076d0 con 0x7faa3c003c40 2023-12-26T23:12:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.508+0000 7faa45ffb700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7faa34066d80 0x7faa34069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.508+0000 7faa45ffb700 1 -- 172.21.15.131:0/2687255629 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6432+0+0 (secure 0 0 0) 0x7faa4c085090 con 0x7faa3c003c40 2023-12-26T23:12:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.508+0000 7faa5579a700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7faa34066d80 0x7faa34069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.508+0000 7faa5579a700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7faa34066d80 0x7faa34069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faa500c8960 tx=0x7faa40008040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.512+0000 7faa45ffb700 1 -- 172.21.15.131:0/2687255629 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa4c052230 con 0x7faa3c003c40 2023-12-26T23:12:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:54.709+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faa500717f0 con 0x7faa3c003c40 2023-12-26T23:12:55.279 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.278+0000 7faa45ffb700 1 -- 172.21.15.131:0/2687255629 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v215) v1 ==== 169+0+0 (secure 0 0 0) 0x7faa4c017020 con 0x7faa3c003c40 2023-12-26T23:12:55.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7faa34066d80 msgr2=0x7faa34069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7faa34066d80 0x7faa34069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faa500c8960 tx=0x7faa40008040 comp rx=0 tx=0).stop 2023-12-26T23:12:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 msgr2=0x7faa5006e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa5006e760 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7faa4c007b50 tx=0x7faa4c015000 comp rx=0 tx=0).stop 2023-12-26T23:12:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 shutdown_connections 2023-12-26T23:12:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7faa34066d80 0x7faa34069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa50166a40 0x7faa50166eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa3c003c40 0x7faa5006e760 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 --2- 172.21.15.131:0/2687255629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa50168e70 0x7faa501692e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 >> 172.21.15.131:0/2687255629 conn(0x7faa500675b0 msgr2=0x7faa500af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:55.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 shutdown_connections 2023-12-26T23:12:55.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.282+0000 7faa581ff700 1 -- 172.21.15.131:0/2687255629 wait complete. 2023-12-26T23:12:55.284 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2023-12-26T23:12:55.306 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-26T23:12:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.500+0000 7f7bb67ff700 1 Processor -- start 2023-12-26T23:12:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.501+0000 7f7bb67ff700 1 -- start start 2023-12-26T23:12:55.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.502+0000 7f7bb67ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 0x7f7bb00b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.502+0000 7f7bb67ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7bb0159cf0 0x7f7bb00b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.502+0000 7f7bb67ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb00c8bf0 con 0x7f7bb01554d0 2023-12-26T23:12:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.502+0000 7f7bb67ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb00c8e60 con 0x7f7bb0159cf0 2023-12-26T23:12:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.502+0000 7f7bb67ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00c9140 con 0x7f7bb0159920 2023-12-26T23:12:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.502+0000 7f7baf7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7bb0159cf0 0x7f7bb00b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.502+0000 7f7baf7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7bb0159cf0 0x7f7bb00b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36764/0 (socket says 172.21.15.131:36764) 2023-12-26T23:12:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.502+0000 7f7baf7fe700 1 -- 172.21.15.131:0/911182463 learned_addr learned my addr 172.21.15.131:0/911182463 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.502+0000 7f7baffff700 1 --2- 172.21.15.131:0/911182463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 0x7f7bb00b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.503+0000 7f7baf7fe700 1 -- 172.21.15.131:0/911182463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 msgr2=0x7f7bb00b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.503+0000 7f7baf7fe700 1 --2- 172.21.15.131:0/911182463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 0x7f7bb00b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.503+0000 7f7baf7fe700 1 -- 172.21.15.131:0/911182463 >> v1:172.21.15.190:6789/0 conn(0x7f7bb01554d0 legacy=0x7f7bb0155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.503+0000 7f7baf7fe700 1 -- 172.21.15.131:0/911182463 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb00c9420 con 0x7f7bb0159cf0 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.503+0000 7f7baffff700 1 --2- 172.21.15.131:0/911182463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 0x7f7bb00b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.503+0000 7f7baf7fe700 1 --2- 172.21.15.131:0/911182463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb0159cf0 0x7f7bb00b0620 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400bc80 tx=0x7f7ba400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1a0ef86131d9c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.503+0000 7f7baeffd700 1 -- 172.21.15.131:0/911182463 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400be90 con 0x7f7bb0159cf0 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.503+0000 7f7baeffd700 1 -- 172.21.15.131:0/911182463 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba400ddc0 con 0x7f7bb0159cf0 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.503+0000 7f7baeffd700 1 -- 172.21.15.131:0/911182463 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40196b0 con 0x7f7bb0159cf0 2023-12-26T23:12:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.504+0000 7f7bb67ff700 1 -- 172.21.15.131:0/911182463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb0159cf0 msgr2=0x7f7bb00b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.504+0000 7f7bb67ff700 1 --2- 172.21.15.131:0/911182463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb0159cf0 0x7f7bb00b0620 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400bc80 tx=0x7f7ba400b4f0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.504+0000 7f7bb67ff700 1 -- 172.21.15.131:0/911182463 shutdown_connections 2023-12-26T23:12:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.504+0000 7f7bb67ff700 1 --2- 172.21.15.131:0/911182463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 0x7f7bb00b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.504+0000 7f7bb67ff700 1 --2- 172.21.15.131:0/911182463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb0159cf0 0x7f7bb00b0620 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.504+0000 7f7bb67ff700 1 -- 172.21.15.131:0/911182463 >> 172.21.15.131:0/911182463 conn(0x7f7bb0068040 msgr2=0x7f7bb0069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.504+0000 7f7bb67ff700 1 -- 172.21.15.131:0/911182463 shutdown_connections 2023-12-26T23:12:55.507 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.504+0000 7f7bb67ff700 1 -- 172.21.15.131:0/911182463 wait complete. 2023-12-26T23:12:55.510 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.505+0000 7f7bb67ff700 1 Processor -- start 2023-12-26T23:12:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.505+0000 7f7bb67ff700 1 -- start start 2023-12-26T23:12:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.506+0000 7f7bb67ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb01554d0 0x7f7bb0151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.506+0000 7f7bb67ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 0x7f7bb01576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.506+0000 7f7bb67ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb0159cf0 0x7f7bb006ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.506+0000 7f7bb67ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01657e0 con 0x7f7bb01554d0 2023-12-26T23:12:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.506+0000 7f7bb67ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0165d60 con 0x7f7bb0159cf0 2023-12-26T23:12:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.506+0000 7f7bb67ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0157fe0 con 0x7f7bb0159920 2023-12-26T23:12:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.506+0000 7f7baffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb01554d0 0x7f7bb0151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.511 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7baffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb01554d0 0x7f7bb0151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36774/0 (socket says 172.21.15.131:36774) 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7baffff700 1 -- 172.21.15.131:0/3770019930 learned_addr learned my addr 172.21.15.131:0/3770019930 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7baf7fe700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 0x7f7bb01576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7bb4d9c700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb0159cf0 0x7f7bb006ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7baffff700 1 -- 172.21.15.131:0/3770019930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 msgr2=0x7f7bb01576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7baffff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 0x7f7bb01576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7baffff700 1 -- 172.21.15.131:0/3770019930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb0159cf0 msgr2=0x7f7bb006ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7baffff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb0159cf0 0x7f7bb006ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7baffff700 1 -- 172.21.15.131:0/3770019930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba400b790 con 0x7f7bb01554d0 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7bb4d9c700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb0159cf0 0x7f7bb006ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:55.512 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7baffff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb01554d0 0x7f7bb0151230 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000e8d0 tx=0x7f7ba0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:55.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7b97fff700 1 -- 172.21.15.131:0/3770019930 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0013410 con 0x7f7bb01554d0 2023-12-26T23:12:55.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7b97fff700 1 -- 172.21.15.131:0/3770019930 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba0013570 con 0x7f7bb01554d0 2023-12-26T23:12:55.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb01582c0 con 0x7f7bb01554d0 2023-12-26T23:12:55.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.507+0000 7f7b97fff700 1 -- 172.21.15.131:0/3770019930 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0013910 con 0x7f7bb01554d0 2023-12-26T23:12:55.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.508+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb0157be0 con 0x7f7bb01554d0 2023-12-26T23:12:55.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.509+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb00657e0 con 0x7f7bb01554d0 2023-12-26T23:12:55.513 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.510+0000 7f7b97fff700 1 -- 172.21.15.131:0/3770019930 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7ba0013d00 con 0x7f7bb01554d0 2023-12-26T23:12:55.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.511+0000 7f7b97fff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7b9c066f30 0x7f7b9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.511+0000 7f7b97fff700 1 -- 172.21.15.131:0/3770019930 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f7ba0084470 con 0x7f7bb01554d0 2023-12-26T23:12:55.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.511+0000 7f7baf7fe700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7b9c066f30 0x7f7b9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.514 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.512+0000 7f7baf7fe700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7b9c066f30 0x7f7b9c0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400bc80 tx=0x7f7ba401b000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:55.515 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.515+0000 7f7b97fff700 1 -- 172.21.15.131:0/3770019930 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ba00515a0 con 0x7f7bb01554d0 2023-12-26T23:12:55.710 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.709+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7bb00c9700 con 0x7f7bb01554d0 2023-12-26T23:12:55.715 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.715+0000 7f7b97fff700 1 -- 172.21.15.131:0/3770019930 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ba0054bc0 con 0x7f7bb01554d0 2023-12-26T23:12:55.718 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.717+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7b9c066f30 msgr2=0x7f7b9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.717+0000 7f7bb67ff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7b9c066f30 0x7f7b9c0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400bc80 tx=0x7f7ba401b000 comp rx=0 tx=0).stop 2023-12-26T23:12:55.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.717+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb01554d0 msgr2=0x7f7bb0151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.717+0000 7f7bb67ff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb01554d0 0x7f7bb0151230 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000e8d0 tx=0x7f7ba0008040 comp rx=0 tx=0).stop 2023-12-26T23:12:55.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.718+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 shutdown_connections 2023-12-26T23:12:55.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.718+0000 7f7bb67ff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f7b9c066f30 0x7f7b9c0693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.718+0000 7f7bb67ff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb0159cf0 0x7f7bb006ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.718+0000 7f7bb67ff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb01554d0 0x7f7bb0151230 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.719 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.718+0000 7f7bb67ff700 1 --2- 172.21.15.131:0/3770019930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb0159920 0x7f7bb01576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.718+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 >> 172.21.15.131:0/3770019930 conn(0x7f7bb0068040 msgr2=0x7f7bb0069360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:55.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.718+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 shutdown_connections 2023-12-26T23:12:55.720 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.718+0000 7f7bb67ff700 1 -- 172.21.15.131:0/3770019930 wait complete. 2023-12-26T23:12:55.735 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-26T23:12:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.934+0000 7fed8f7ff700 1 Processor -- start 2023-12-26T23:12:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.934+0000 7fed8f7ff700 1 -- start start 2023-12-26T23:12:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.935+0000 7fed8f7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed880b3d40 0x7fed880b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.935+0000 7fed8f7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8815d250 con 0x7fed88153bd0 2023-12-26T23:12:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.935+0000 7fed8f7ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8815d4a0 con 0x7fed880b1a40 2023-12-26T23:12:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.935+0000 7fed8f7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8815e260 con 0x7fed880b3d40 2023-12-26T23:12:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.935+0000 7fed8d59b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fed88153bd0 0x7fed880b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43932/0 (socket says 172.21.15.131:43932) 2023-12-26T23:12:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.935+0000 7fed8d59b700 1 -- 172.21.15.131:0/761721671 learned_addr learned my addr 172.21.15.131:0/761721671 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed8cd9a700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed880b3d40 0x7fed880b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881286984 0 0) 0x7fed8815d250 con 0x7fed88153bd0 2023-12-26T23:12:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed74003570 con 0x7fed88153bd0 2023-12-26T23:12:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1266547643 0 0) 0x7fed74003570 con 0x7fed88153bd0 2023-12-26T23:12:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed880b3d40 msgr2=0x7fed880b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed880b3d40 0x7fed880b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 >> v1:172.21.15.190:6789/0 conn(0x7fed880b1a40 legacy=0x7fed88068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8815d780 con 0x7fed88153bd0 2023-12-26T23:12:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fed84001470 con 0x7fed88153bd0 2023-12-26T23:12:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 >> v1:172.21.15.131:6789/0 conn(0x7fed88153bd0 legacy=0x7fed880b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed880b3d40 0x7fed74003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed74003fd0 0x7fed74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed74004d50 0x7fed740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7400bb10 con 0x7fed74004d50 2023-12-26T23:12:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7400bc80 con 0x7fed74003fd0 2023-12-26T23:12:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7400be40 con 0x7fed880b3d40 2023-12-26T23:12:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed8dd9c700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed74004d50 0x7fed740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.936+0000 7fed8cd9a700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed74003fd0 0x7fed74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.942 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed8d59b700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed880b3d40 0x7fed74003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed8dd9c700 1 -- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed880b3d40 msgr2=0x7fed74003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed8dd9c700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed880b3d40 0x7fed74003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed8dd9c700 1 -- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed74003fd0 msgr2=0x7fed74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed8dd9c700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed74003fd0 0x7fed74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed8dd9c700 1 -- 172.21.15.131:0/761721671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed74003570 con 0x7fed74004d50 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed8dd9c700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed74004d50 0x7fed740051c0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fed800020a0 tx=0x7fed800016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47519fbae255eb29 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800019e0 con 0x7fed74004d50 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8000c8b0 con 0x7fed74004d50 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.937+0000 7fed7ffff700 1 -- 172.21.15.131:0/761721671 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000cc50 con 0x7fed74004d50 2023-12-26T23:12:55.943 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.938+0000 7fed8f7ff700 1 -- 172.21.15.131:0/761721671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed74004d50 msgr2=0x7fed740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.938+0000 7fed8f7ff700 1 --2- 172.21.15.131:0/761721671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed74004d50 0x7fed740051c0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fed800020a0 tx=0x7fed800016d0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.938+0000 7fed8dd9c700 1 -- 172.21.15.131:0/761721671 reap_dead start 2023-12-26T23:12:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.938+0000 7fed8f7ff700 1 -- 172.21.15.131:0/761721671 shutdown_connections 2023-12-26T23:12:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.938+0000 7fed8f7ff700 1 -- 172.21.15.131:0/761721671 >> 172.21.15.131:0/761721671 conn(0x7fed88063d00 msgr2=0x7fed8806b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.938+0000 7fed8f7ff700 1 -- 172.21.15.131:0/761721671 shutdown_connections 2023-12-26T23:12:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.938+0000 7fed8f7ff700 1 -- 172.21.15.131:0/761721671 wait complete. 2023-12-26T23:12:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.939+0000 7fed8f7ff700 1 Processor -- start 2023-12-26T23:12:55.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.940+0000 7fed8f7ff700 1 -- start start 2023-12-26T23:12:55.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.940+0000 7fed8f7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed74004d50 0x7fed88155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.940+0000 7fed8f7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed88148eb0 0x7fed880cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8f7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed880cc450 0x7fed88153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8f7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed88156350 con 0x7fed88148eb0 2023-12-26T23:12:55.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8f7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8816b950 con 0x7fed880cc450 2023-12-26T23:12:55.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8f7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8816ad40 con 0x7fed74004d50 2023-12-26T23:12:55.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8cd9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed88148eb0 0x7fed880cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8cd9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed88148eb0 0x7fed880cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36782/0 (socket says 172.21.15.131:36782) 2023-12-26T23:12:55.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8cd9a700 1 -- 172.21.15.131:0/33989176 learned_addr learned my addr 172.21.15.131:0/33989176 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:55.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8dd9c700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed880cc450 0x7fed88153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8d59b700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed74004d50 0x7fed88155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8cd9a700 1 -- 172.21.15.131:0/33989176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed74004d50 msgr2=0x7fed88155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8cd9a700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed74004d50 0x7fed88155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8cd9a700 1 -- 172.21.15.131:0/33989176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed880cc450 msgr2=0x7fed88153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8cd9a700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed880cc450 0x7fed88153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8cd9a700 1 -- 172.21.15.131:0/33989176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8816b020 con 0x7fed88148eb0 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.941+0000 7fed8cd9a700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed88148eb0 0x7fed880cbf10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fed78006ac0 tx=0x7fed780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.942+0000 7fed7d7fa700 1 -- 172.21.15.131:0/33989176 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7800f8e0 con 0x7fed88148eb0 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.942+0000 7fed7d7fa700 1 -- 172.21.15.131:0/33989176 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7800fa40 con 0x7fed88148eb0 2023-12-26T23:12:55.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.942+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8816b300 con 0x7fed88148eb0 2023-12-26T23:12:55.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.942+0000 7fed7d7fa700 1 -- 172.21.15.131:0/33989176 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7800fdc0 con 0x7fed88148eb0 2023-12-26T23:12:55.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.942+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8816a9a0 con 0x7fed88148eb0 2023-12-26T23:12:55.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.943+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8814a4d0 con 0x7fed88148eb0 2023-12-26T23:12:55.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.944+0000 7fed7d7fa700 1 -- 172.21.15.131:0/33989176 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fed78014650 con 0x7fed88148eb0 2023-12-26T23:12:55.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.945+0000 7fed7d7fa700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fed6c066f30 0x7fed6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:55.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.945+0000 7fed7d7fa700 1 -- 172.21.15.131:0/33989176 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fed78018030 con 0x7fed88148eb0 2023-12-26T23:12:55.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.945+0000 7fed8d59b700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fed6c066f30 0x7fed6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:55.950 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.946+0000 7fed8d59b700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fed6c066f30 0x7fed6c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fed84007eb0 tx=0x7fed84001e70 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:55.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:55.949+0000 7fed7d7fa700 1 -- 172.21.15.131:0/33989176 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed78052110 con 0x7fed88148eb0 2023-12-26T23:12:56.141 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.141+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fed880657e0 con 0x7fed88148eb0 2023-12-26T23:12:56.146 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.146+0000 7fed7d7fa700 1 -- 172.21.15.131:0/33989176 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v74) v1 ==== 115+0+0 (secure 0 0 0) 0x7fed780220b0 con 0x7fed88148eb0 2023-12-26T23:12:56.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fed6c066f30 msgr2=0x7fed6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fed6c066f30 0x7fed6c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fed84007eb0 tx=0x7fed84001e70 comp rx=0 tx=0).stop 2023-12-26T23:12:56.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed88148eb0 msgr2=0x7fed880cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed88148eb0 0x7fed880cbf10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fed78006ac0 tx=0x7fed780091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 shutdown_connections 2023-12-26T23:12:56.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fed6c066f30 0x7fed6c0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.149 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed880cc450 0x7fed88153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed74004d50 0x7fed88155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 --2- 172.21.15.131:0/33989176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed88148eb0 0x7fed880cbf10 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 >> 172.21.15.131:0/33989176 conn(0x7fed88063d00 msgr2=0x7fed8815bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:56.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 shutdown_connections 2023-12-26T23:12:56.150 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.149+0000 7fed8f7ff700 1 -- 172.21.15.131:0/33989176 wait complete. 2023-12-26T23:12:56.167 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-26T23:12:56.365 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.365+0000 7fd1a61ff700 1 Processor -- start 2023-12-26T23:12:56.366 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.366+0000 7fd1a61ff700 1 -- start start 2023-12-26T23:12:56.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd1a61ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a0159820 0x7fd1a00abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd1a61ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1a006e480 0x7fd1a00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd1a61ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a015a8c0 con 0x7fd1a00aa480 2023-12-26T23:12:56.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd1a61ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a015ad50 con 0x7fd1a006e480 2023-12-26T23:12:56.367 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd1a61ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0159f90 con 0x7fd1a0159820 2023-12-26T23:12:56.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd19ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1a006e480 0x7fd1a00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd19f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a0159820 0x7fd1a00abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd19ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1a006e480 0x7fd1a00675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:60390/0 (socket says 172.21.15.131:60390) 2023-12-26T23:12:56.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd19ffff700 1 -- 172.21.15.131:0/2380218588 learned_addr learned my addr 172.21.15.131:0/2380218588 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:56.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.367+0000 7fd19f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a0159820 0x7fd1a00abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:52612/0 (socket says 172.21.15.131:52612) 2023-12-26T23:12:56.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.368+0000 7fd19ffff700 1 -- 172.21.15.131:0/2380218588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a0159820 msgr2=0x7fd1a00abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:56.368 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.368+0000 7fd19ffff700 1 --2- 172.21.15.131:0/2380218588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a0159820 0x7fd1a00abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.368+0000 7fd19ffff700 1 -- 172.21.15.131:0/2380218588 >> v1:172.21.15.190:6789/0 conn(0x7fd1a00aa480 legacy=0x7fd1a006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.368+0000 7fd19ffff700 1 -- 172.21.15.131:0/2380218588 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a015a270 con 0x7fd1a006e480 2023-12-26T23:12:56.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.368+0000 7fd19f7fe700 1 --2- 172.21.15.131:0/2380218588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a0159820 0x7fd1a00abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:56.369 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.368+0000 7fd19ffff700 1 --2- 172.21.15.131:0/2380218588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a006e480 0x7fd1a00675b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd198009f20 tx=0x7fd198009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbefcba9088c2e69 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.368+0000 7fd19e7fc700 1 -- 172.21.15.131:0/2380218588 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19800e070 con 0x7fd1a006e480 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd19e7fc700 1 -- 172.21.15.131:0/2380218588 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19800b9d0 con 0x7fd1a006e480 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd19e7fc700 1 -- 172.21.15.131:0/2380218588 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1980092e0 con 0x7fd1a006e480 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd1a61ff700 1 -- 172.21.15.131:0/2380218588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a006e480 msgr2=0x7fd1a00675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd1a61ff700 1 --2- 172.21.15.131:0/2380218588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a006e480 0x7fd1a00675b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd198009f20 tx=0x7fd198009840 comp rx=0 tx=0).stop 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd1a61ff700 1 -- 172.21.15.131:0/2380218588 shutdown_connections 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd1a61ff700 1 --2- 172.21.15.131:0/2380218588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a006e480 0x7fd1a00675b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd1a61ff700 1 --2- 172.21.15.131:0/2380218588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a0159820 0x7fd1a00abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd1a61ff700 1 -- 172.21.15.131:0/2380218588 >> 172.21.15.131:0/2380218588 conn(0x7fd1a006c290 msgr2=0x7fd1a00ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd1a61ff700 1 -- 172.21.15.131:0/2380218588 shutdown_connections 2023-12-26T23:12:56.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.369+0000 7fd1a61ff700 1 -- 172.21.15.131:0/2380218588 wait complete. 2023-12-26T23:12:56.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.371+0000 7fd1a61ff700 1 Processor -- start 2023-12-26T23:12:56.371 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.371+0000 7fd1a61ff700 1 -- start start 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd1a61ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a006e480 0x7fd1a0156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd1a61ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a00aa480 0x7fd1a015be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd1a61ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a0159820 0x7fd1a015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd1a61ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00c31e0 con 0x7fd1a0159820 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd1a61ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00c25d0 con 0x7fd1a00aa480 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd1a61ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00c28b0 con 0x7fd1a006e480 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd19ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a0159820 0x7fd1a015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd19ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a0159820 0x7fd1a015c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36794/0 (socket says 172.21.15.131:36794) 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd19ffff700 1 -- 172.21.15.131:0/3221912075 learned_addr learned my addr 172.21.15.131:0/3221912075 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd19f7fe700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a006e480 0x7fd1a0156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.372+0000 7fd19effd700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a00aa480 0x7fd1a015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd19ffff700 1 -- 172.21.15.131:0/3221912075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a006e480 msgr2=0x7fd1a0156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd19ffff700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a006e480 0x7fd1a0156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd19ffff700 1 -- 172.21.15.131:0/3221912075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a00aa480 msgr2=0x7fd1a015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd19ffff700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a00aa480 0x7fd1a015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd19ffff700 1 -- 172.21.15.131:0/3221912075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd190009420 con 0x7fd1a0159820 2023-12-26T23:12:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd19effd700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a00aa480 0x7fd1a015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd19ffff700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a0159820 0x7fd1a015c3b0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd1a006d0a0 tx=0x7fd198018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd1877fe700 1 -- 172.21.15.131:0/3221912075 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198008b80 con 0x7fd1a0159820 2023-12-26T23:12:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd1877fe700 1 -- 172.21.15.131:0/3221912075 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd198008ce0 con 0x7fd1a0159820 2023-12-26T23:12:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd198009ae0 con 0x7fd1a0159820 2023-12-26T23:12:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd1877fe700 1 -- 172.21.15.131:0/3221912075 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1980256b0 con 0x7fd1a0159820 2023-12-26T23:12:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.373+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a00c2e70 con 0x7fd1a0159820 2023-12-26T23:12:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.374+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a00c3390 con 0x7fd1a0159820 2023-12-26T23:12:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.375+0000 7fd1877fe700 1 -- 172.21.15.131:0/3221912075 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd198003040 con 0x7fd1a0159820 2023-12-26T23:12:56.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.376+0000 7fd1877fe700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd18c066f30 0x7fd18c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.376+0000 7fd1877fe700 1 -- 172.21.15.131:0/3221912075 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fd198085c20 con 0x7fd1a0159820 2023-12-26T23:12:56.376 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.376+0000 7fd19f7fe700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd18c066f30 0x7fd18c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.377 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.377+0000 7fd19f7fe700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd18c066f30 0x7fd18c0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd190009f40 tx=0x7fd1900091a0 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:56.380 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.380+0000 7fd1877fe700 1 -- 172.21.15.131:0/3221912075 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1a00c3390 con 0x7fd1a0159820 2023-12-26T23:12:56.573 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.573+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd1a00657e0 con 0x7fd1a0159820 2023-12-26T23:12:56.577 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.577+0000 7fd1877fe700 1 -- 172.21.15.131:0/3221912075 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v75) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd198052d50 con 0x7fd1a0159820 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd18c066f30 msgr2=0x7fd18c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd18c066f30 0x7fd18c0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd190009f40 tx=0x7fd1900091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a0159820 msgr2=0x7fd1a015c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a0159820 0x7fd1a015c3b0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd1a006d0a0 tx=0x7fd198018000 comp rx=0 tx=0).stop 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 shutdown_connections 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd18c066f30 0x7fd18c0693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1a0159820 0x7fd1a015c3b0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a006e480 0x7fd1a0156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 --2- 172.21.15.131:0/3221912075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a00aa480 0x7fd1a015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 >> 172.21.15.131:0/3221912075 conn(0x7fd1a006c290 msgr2=0x7fd1a00ac5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 shutdown_connections 2023-12-26T23:12:56.581 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.580+0000 7fd1a61ff700 1 -- 172.21.15.131:0/3221912075 wait complete. 2023-12-26T23:12:56.599 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-12-26T23:12:56.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.787+0000 7f848cbff700 1 Processor -- start 2023-12-26T23:12:56.788 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.788+0000 7f848cbff700 1 -- start start 2023-12-26T23:12:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.789+0000 7f848cbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f84880b0750 0x7f84880b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.789+0000 7f848cbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f84880c9400 0x7f84880c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.789+0000 7f848cbff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8488156940 con 0x7f84880cb320 2023-12-26T23:12:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.789+0000 7f848cbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84880c8cc0 con 0x7f84880b0750 2023-12-26T23:12:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.789+0000 7f848cbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84880ad700 con 0x7f84880c9400 2023-12-26T23:12:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.789+0000 7f8485d9b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f84880b0750 0x7f84880b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.789+0000 7f848659c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f84880cb320 0x7f8488069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43940/0 (socket says 172.21.15.131:43940) 2023-12-26T23:12:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.789+0000 7f848659c700 1 -- 172.21.15.131:0/1779824619 learned_addr learned my addr 172.21.15.131:0/1779824619 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.789+0000 7f8486d9d700 1 --2- 172.21.15.131:0/1779824619 >> v2:172.21.15.190:3300/0 conn(0x7f84880c9400 0x7f84880c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.790+0000 7f8485d9b700 1 -- 172.21.15.131:0/1779824619 >> v2:172.21.15.190:3300/0 conn(0x7f84880c9400 msgr2=0x7f84880c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.790+0000 7f8485d9b700 1 --2- 172.21.15.131:0/1779824619 >> v2:172.21.15.190:3300/0 conn(0x7f84880c9400 0x7f84880c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.790+0000 7f8485d9b700 1 -- 172.21.15.131:0/1779824619 >> v1:172.21.15.131:6789/0 conn(0x7f84880cb320 legacy=0x7f8488069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.790+0000 7f8485d9b700 1 -- 172.21.15.131:0/1779824619 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84880ae500 con 0x7f84880b0750 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.790+0000 7f848559a700 1 -- 172.21.15.131:0/1779824619 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890977001 0 0) 0x7f8488156940 con 0x7f84880cb320 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.790+0000 7f8485d9b700 1 --2- 172.21.15.131:0/1779824619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880b0750 0x7f84880b0bf0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f848806fb50 tx=0x7f847c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657bb484a323ac24 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.790+0000 7f848559a700 1 -- 172.21.15.131:0/1779824619 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c011630 con 0x7f84880b0750 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.790+0000 7f848559a700 1 -- 172.21.15.131:0/1779824619 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c011790 con 0x7f84880b0750 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.790+0000 7f848559a700 1 -- 172.21.15.131:0/1779824619 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c011b30 con 0x7f84880b0750 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.791+0000 7f848cbff700 1 -- 172.21.15.131:0/1779824619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880b0750 msgr2=0x7f84880b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.791+0000 7f848cbff700 1 --2- 172.21.15.131:0/1779824619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880b0750 0x7f84880b0bf0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f848806fb50 tx=0x7f847c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.791+0000 7f848cbff700 1 -- 172.21.15.131:0/1779824619 shutdown_connections 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.791+0000 7f848cbff700 1 --2- 172.21.15.131:0/1779824619 >> v2:172.21.15.190:3300/0 conn(0x7f84880c9400 0x7f84880c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.791+0000 7f848cbff700 1 --2- 172.21.15.131:0/1779824619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880b0750 0x7f84880b0bf0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.791+0000 7f848cbff700 1 -- 172.21.15.131:0/1779824619 >> 172.21.15.131:0/1779824619 conn(0x7f8488068040 msgr2=0x7f8488069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.791+0000 7f848cbff700 1 -- 172.21.15.131:0/1779824619 shutdown_connections 2023-12-26T23:12:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.791+0000 7f848cbff700 1 -- 172.21.15.131:0/1779824619 wait complete. 2023-12-26T23:12:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.792+0000 7f848cbff700 1 Processor -- start 2023-12-26T23:12:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.792+0000 7f848cbff700 1 -- start start 2023-12-26T23:12:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.793+0000 7f848cbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84880b0750 0x7f848806f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.793+0000 7f848cbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880c9400 0x7f84881443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.793+0000 7f848cbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84880cb320 0x7f848806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.793+0000 7f848cbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848806f540 con 0x7f84880c9400 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.793+0000 7f848cbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848806df60 con 0x7f84880b0750 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.793+0000 7f848cbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848806e240 con 0x7f84880cb320 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8485d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880c9400 0x7f84881443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8485d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880c9400 0x7f84881443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36820/0 (socket says 172.21.15.131:36820) 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8485d9b700 1 -- 172.21.15.131:0/336114108 learned_addr learned my addr 172.21.15.131:0/336114108 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f848659c700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84880b0750 0x7f848806f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8486d9d700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84880cb320 0x7f848806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8485d9b700 1 -- 172.21.15.131:0/336114108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84880cb320 msgr2=0x7f848806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8485d9b700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84880cb320 0x7f848806d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8485d9b700 1 -- 172.21.15.131:0/336114108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84880b0750 msgr2=0x7f848806f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8485d9b700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84880b0750 0x7f848806f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8485d9b700 1 -- 172.21.15.131:0/336114108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c009420 con 0x7f84880c9400 2023-12-26T23:12:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.794+0000 7f8485d9b700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880c9400 0x7f84881443c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f847c0091e0 tx=0x7f847c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.795+0000 7f84767fc700 1 -- 172.21.15.131:0/336114108 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c000d10 con 0x7f84880c9400 2023-12-26T23:12:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.795+0000 7f84767fc700 1 -- 172.21.15.131:0/336114108 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c011900 con 0x7f84880c9400 2023-12-26T23:12:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.795+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848806e520 con 0x7f84880c9400 2023-12-26T23:12:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.795+0000 7f84767fc700 1 -- 172.21.15.131:0/336114108 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c025420 con 0x7f84880c9400 2023-12-26T23:12:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.795+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848806eae0 con 0x7f84880c9400 2023-12-26T23:12:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.796+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848806ee50 con 0x7f84880c9400 2023-12-26T23:12:56.797 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.797+0000 7f84767fc700 1 -- 172.21.15.131:0/336114108 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f847c023070 con 0x7f84880c9400 2023-12-26T23:12:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.797+0000 7f84767fc700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8468066f30 0x7f84680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.798+0000 7f84767fc700 1 -- 172.21.15.131:0/336114108 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f847c085b90 con 0x7f84880c9400 2023-12-26T23:12:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.798+0000 7f848659c700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8468066f30 0x7f84680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.798+0000 7f848659c700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8468066f30 0x7f84680693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8478007160 tx=0x7f8478007f50 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:56.802+0000 7f84767fc700 1 -- 172.21.15.131:0/336114108 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f847c052c20 con 0x7f84880c9400 2023-12-26T23:12:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.074+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f84880657e0 con 0x7f84880c9400 2023-12-26T23:12:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.282+0000 7f84767fc700 1 -- 172.21.15.131:0/336114108 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3112) v1 ==== 155+0+0 (secure 0 0 0) 0x7f847c02a0e0 con 0x7f84880c9400 2023-12-26T23:12:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8468066f30 msgr2=0x7f84680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8468066f30 0x7f84680693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8478007160 tx=0x7f8478007f50 comp rx=0 tx=0).stop 2023-12-26T23:12:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880c9400 msgr2=0x7f84881443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880c9400 0x7f84881443c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f847c0091e0 tx=0x7f847c017000 comp rx=0 tx=0).stop 2023-12-26T23:12:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 shutdown_connections 2023-12-26T23:12:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f8468066f30 0x7f84680693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84880cb320 0x7f848806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:57.286 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84880b0750 0x7f848806f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:57.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 --2- 172.21.15.131:0/336114108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84880c9400 0x7f84881443c0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:57.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 >> 172.21.15.131:0/336114108 conn(0x7f8488068040 msgr2=0x7f8488155a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:57.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 shutdown_connections 2023-12-26T23:12:57.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:57.285+0000 7f848cbff700 1 -- 172.21.15.131:0/336114108 wait complete. 2023-12-26T23:12:57.304 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-12-26T23:12:57.304 INFO:tasks.cephfs_test_runner: 2023-12-26T23:12:57.304 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-26T23:12:57.304 INFO:tasks.cephfs_test_runner:Ran 11 tests in 5733.327s 2023-12-26T23:12:57.304 INFO:tasks.cephfs_test_runner: 2023-12-26T23:12:57.304 INFO:tasks.cephfs_test_runner:OK 2023-12-26T23:12:57.305 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-12-26T23:12:57.315 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-26T23:12:57.335 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-26T23:12:57.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:57.335 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T23:12:57.380 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-26T23:12:57.380 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-26T23:12:57.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:57.380 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-26T23:12:57.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:57.490 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-26T23:12:57.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:57.562 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2023-12-26T23:12:57.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:57.688 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2023-12-26T23:12:57.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:57.733 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2023-12-26T23:12:57.847 INFO:teuthology.orchestra.run.smithi131.stderr:Cannot find device "ceph-brx" 2023-12-26T23:12:57.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T23:12:57.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:57.852 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-12-26T23:12:57.902 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-26T23:12:57.902 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-26T23:12:57.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:57.902 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-26T23:12:57.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:57.976 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-26T23:12:58.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:58.050 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-12-26T23:12:58.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:58.175 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-12-26T23:12:58.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-26T23:12:58.221 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-12-26T23:12:58.335 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-12-26T23:12:58.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T23:12:58.339 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-26T23:12:58.349 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-12-26T23:12:58.349 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:12:58.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.541+0000 7f6a305ff700 1 Processor -- start 2023-12-26T23:12:58.541 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.541+0000 7f6a305ff700 1 -- start start 2023-12-26T23:12:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.542+0000 7f6a305ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a28159920 0x7f6a280b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.542+0000 7f6a305ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a28159cf0 0x7f6a280b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.542+0000 7f6a305ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a280c8bf0 con 0x7f6a281554d0 2023-12-26T23:12:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.542+0000 7f6a305ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a280c8e60 con 0x7f6a28159920 2023-12-26T23:12:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.542+0000 7f6a305ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a280c9140 con 0x7f6a28159cf0 2023-12-26T23:12:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2eb9c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6a281554d0 0x7f6a28155930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43944/0 (socket says 172.21.15.131:43944) 2023-12-26T23:12:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2eb9c700 1 -- 172.21.15.131:0/1799988580 learned_addr learned my addr 172.21.15.131:0/1799988580 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2db9a700 1 --2- 172.21.15.131:0/1799988580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a28159cf0 0x7f6a280b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2e39b700 1 --2- 172.21.15.131:0/1799988580 >> v2:172.21.15.190:3300/0 conn(0x7f6a28159920 0x7f6a280b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2d399700 1 -- 172.21.15.131:0/1799988580 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109601138 0 0) 0x7f6a280c8bf0 con 0x7f6a281554d0 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2d399700 1 -- 172.21.15.131:0/1799988580 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a14003570 con 0x7f6a281554d0 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2e39b700 1 -- 172.21.15.131:0/1799988580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a28159cf0 msgr2=0x7f6a280b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2e39b700 1 --2- 172.21.15.131:0/1799988580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a28159cf0 0x7f6a280b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2e39b700 1 -- 172.21.15.131:0/1799988580 >> v1:172.21.15.131:6789/0 conn(0x7f6a281554d0 legacy=0x7f6a28155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2e39b700 1 -- 172.21.15.131:0/1799988580 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a280c9420 con 0x7f6a28159920 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2db9a700 1 --2- 172.21.15.131:0/1799988580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a28159cf0 0x7f6a280b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.543+0000 7f6a2e39b700 1 --2- 172.21.15.131:0/1799988580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a28159920 0x7f6a280b00e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400aa90 tx=0x7f6a2400a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa39e560d548315 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a2d399700 1 -- 172.21.15.131:0/1799988580 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400c970 con 0x7f6a28159920 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a2d399700 1 -- 172.21.15.131:0/1799988580 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2400cad0 con 0x7f6a28159920 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a2d399700 1 -- 172.21.15.131:0/1799988580 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400ce70 con 0x7f6a28159920 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a305ff700 1 -- 172.21.15.131:0/1799988580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a28159920 msgr2=0x7f6a280b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a305ff700 1 --2- 172.21.15.131:0/1799988580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a28159920 0x7f6a280b00e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400aa90 tx=0x7f6a2400a330 comp rx=0 tx=0).stop 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a305ff700 1 -- 172.21.15.131:0/1799988580 shutdown_connections 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a305ff700 1 --2- 172.21.15.131:0/1799988580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a28159920 0x7f6a280b00e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a305ff700 1 --2- 172.21.15.131:0/1799988580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a28159cf0 0x7f6a280b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a305ff700 1 -- 172.21.15.131:0/1799988580 >> 172.21.15.131:0/1799988580 conn(0x7f6a28068040 msgr2=0x7f6a28069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a305ff700 1 -- 172.21.15.131:0/1799988580 shutdown_connections 2023-12-26T23:12:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.544+0000 7f6a305ff700 1 -- 172.21.15.131:0/1799988580 wait complete. 2023-12-26T23:12:58.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.546+0000 7f6a305ff700 1 Processor -- start 2023-12-26T23:12:58.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.546+0000 7f6a305ff700 1 -- start start 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a305ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a281554d0 0x7f6a2816a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a305ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a28159920 0x7f6a281560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a305ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a28159cf0 0x7f6a280c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a305ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28156610 con 0x7f6a28159920 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a305ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28152020 con 0x7f6a28159cf0 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a305ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28152300 con 0x7f6a281554d0 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a2db9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a28159920 0x7f6a281560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a2db9a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a28159920 0x7f6a281560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36822/0 (socket says 172.21.15.131:36822) 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a2db9a700 1 -- 172.21.15.131:0/3029418015 learned_addr learned my addr 172.21.15.131:0/3029418015 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a2e39b700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a281554d0 0x7f6a2816a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.547+0000 7f6a2eb9c700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a28159cf0 0x7f6a280c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a2db9a700 1 -- 172.21.15.131:0/3029418015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a281554d0 msgr2=0x7f6a2816a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a2db9a700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a281554d0 0x7f6a2816a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a2db9a700 1 -- 172.21.15.131:0/3029418015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a28159cf0 msgr2=0x7f6a280c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a2db9a700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a28159cf0 0x7f6a280c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a2db9a700 1 -- 172.21.15.131:0/3029418015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a18009420 con 0x7f6a28159920 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a2e39b700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a281554d0 0x7f6a2816a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a2db9a700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a28159920 0x7f6a281560d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800ec80 tx=0x7f6a180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a1e7fc700 1 -- 172.21.15.131:0/3029418015 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800c850 con 0x7f6a28159920 2023-12-26T23:12:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a1e7fc700 1 -- 172.21.15.131:0/3029418015 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1800c9b0 con 0x7f6a28159920 2023-12-26T23:12:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2400a5d0 con 0x7f6a28159920 2023-12-26T23:12:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a1e7fc700 1 -- 172.21.15.131:0/3029418015 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800cd30 con 0x7f6a28159920 2023-12-26T23:12:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.548+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a28169ef0 con 0x7f6a28159920 2023-12-26T23:12:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.549+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a280657e0 con 0x7f6a28159920 2023-12-26T23:12:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.550+0000 7f6a1e7fc700 1 -- 172.21.15.131:0/3029418015 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6a18013680 con 0x7f6a28159920 2023-12-26T23:12:58.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.551+0000 7f6a1e7fc700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6a0c066f30 0x7f6a0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.551+0000 7f6a1e7fc700 1 -- 172.21.15.131:0/3029418015 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f6a18084950 con 0x7f6a28159920 2023-12-26T23:12:58.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.551+0000 7f6a2e39b700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6a0c066f30 0x7f6a0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.554 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.552+0000 7f6a2e39b700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6a0c066f30 0x7f6a0c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400a370 tx=0x7f6a2401a000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:58.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.555+0000 7f6a1e7fc700 1 -- 172.21.15.131:0/3029418015 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a180519e0 con 0x7f6a28159920 2023-12-26T23:12:58.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.752+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a280c9700 con 0x7f6a28159920 2023-12-26T23:12:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.753+0000 7f6a1e7fc700 1 -- 172.21.15.131:0/3029418015 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+13938 (secure 0 0 0) 0x7f6a18055000 con 0x7f6a28159920 2023-12-26T23:12:58.753 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:58.754 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":215,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:55.276294+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2756115966":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/712424342":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/1157583804":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/296797731":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/223069446":"2023-12-27T23:12:53.226086+0000","172.21.15.190:6834/1917956201":"2023-12-27T23:12:52.271063+0000","172.21.15.131:0/1855853217":"2023-12-27T23:12:53.226086+0000","172.21.15.190:6832/1917956201":"2023-12-27T23:12:52.271063+0000","172.21.15.131:0/3395260924":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/1419722337":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/3886778096":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/3508392670":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/1340664010":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6837/92228":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6836/92228":"2023-12-27T23:12:42.707929+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:12:58.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6a0c066f30 msgr2=0x7f6a0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6a0c066f30 0x7f6a0c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400a370 tx=0x7f6a2401a000 comp rx=0 tx=0).stop 2023-12-26T23:12:58.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a28159920 msgr2=0x7f6a281560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a28159920 0x7f6a281560d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800ec80 tx=0x7f6a180091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 shutdown_connections 2023-12-26T23:12:58.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f6a0c066f30 0x7f6a0c0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a28159cf0 0x7f6a280c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a281554d0 0x7f6a2816a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 --2- 172.21.15.131:0/3029418015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a28159920 0x7f6a281560d0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 >> 172.21.15.131:0/3029418015 conn(0x7f6a28068040 msgr2=0x7f6a28069360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:58.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 shutdown_connections 2023-12-26T23:12:58.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.755+0000 7f6a305ff700 1 -- 172.21.15.131:0/3029418015 wait complete. 2023-12-26T23:12:58.773 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-26T23:12:58.774 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:12:58.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.964+0000 7f9b681ff700 1 Processor -- start 2023-12-26T23:12:58.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.965+0000 7f9b681ff700 1 -- start start 2023-12-26T23:12:58.967 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.966+0000 7f9b681ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b600c8bf0 0x7f9b600b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.966+0000 7f9b681ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b600caa10 con 0x7f9b60069d10 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.966+0000 7f9b681ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b60157a80 con 0x7f9b600c8820 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.966+0000 7f9b681ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600cac50 con 0x7f9b600c8bf0 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.966+0000 7f9b6679c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9b60069d10 0x7f9b60161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43952/0 (socket says 172.21.15.131:43952) 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.966+0000 7f9b6679c700 1 -- 172.21.15.131:0/2718176661 learned_addr learned my addr 172.21.15.131:0/2718176661 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.966+0000 7f9b6579a700 1 --2- 172.21.15.131:0/2718176661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b600c8bf0 0x7f9b600b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.966+0000 7f9b64f99700 1 -- 172.21.15.131:0/2718176661 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282352044 0 0) 0x7f9b58001490 con 0x7f9b60069d10 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b64f99700 1 -- 172.21.15.131:0/2718176661 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b4c003570 con 0x7f9b60069d10 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b64f99700 1 -- 172.21.15.131:0/2718176661 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099829337 0 0) 0x7f9b60157a80 con 0x7f9b600c8820 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b64f99700 1 -- 172.21.15.131:0/2718176661 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b58001490 con 0x7f9b600c8820 2023-12-26T23:12:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b6579a700 1 -- 172.21.15.131:0/2718176661 >> v1:172.21.15.190:6789/0 conn(0x7f9b600c8820 legacy=0x7f9b600ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b6579a700 1 -- 172.21.15.131:0/2718176661 >> v1:172.21.15.131:6789/0 conn(0x7f9b60069d10 legacy=0x7f9b60161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b6579a700 1 -- 172.21.15.131:0/2718176661 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b600caf30 con 0x7f9b600c8bf0 2023-12-26T23:12:58.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b6579a700 1 --2- 172.21.15.131:0/2718176661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b600c8bf0 0x7f9b600b12e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009910 tx=0x7f9b500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ef1ee695f4bab server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:58.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b64f99700 1 -- 172.21.15.131:0/2718176661 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50011630 con 0x7f9b600c8bf0 2023-12-26T23:12:58.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b64f99700 1 -- 172.21.15.131:0/2718176661 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b50011790 con 0x7f9b600c8bf0 2023-12-26T23:12:58.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.967+0000 7f9b64f99700 1 -- 172.21.15.131:0/2718176661 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50011b30 con 0x7f9b600c8bf0 2023-12-26T23:12:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.968+0000 7f9b681ff700 1 -- 172.21.15.131:0/2718176661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b600c8bf0 msgr2=0x7f9b600b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.968+0000 7f9b681ff700 1 --2- 172.21.15.131:0/2718176661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b600c8bf0 0x7f9b600b12e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009910 tx=0x7f9b500091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.968+0000 7f9b681ff700 1 -- 172.21.15.131:0/2718176661 shutdown_connections 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.968+0000 7f9b681ff700 1 --2- 172.21.15.131:0/2718176661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b600c8bf0 0x7f9b600b12e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.968+0000 7f9b681ff700 1 -- 172.21.15.131:0/2718176661 >> 172.21.15.131:0/2718176661 conn(0x7f9b60068040 msgr2=0x7f9b60069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.968+0000 7f9b681ff700 1 -- 172.21.15.131:0/2718176661 shutdown_connections 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.968+0000 7f9b681ff700 1 -- 172.21.15.131:0/2718176661 wait complete. 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.969+0000 7f9b681ff700 1 Processor -- start 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.969+0000 7f9b681ff700 1 -- start start 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.970+0000 7f9b681ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b60069d10 0x7f9b601684b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.970+0000 7f9b681ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b600c8820 0x7f9b600c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.970+0000 7f9b681ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b600c8bf0 0x7f9b600c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.970+0000 7f9b681ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600c3030 con 0x7f9b600c8820 2023-12-26T23:12:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.970+0000 7f9b681ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600c4550 con 0x7f9b600c8bf0 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.970+0000 7f9b681ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600c4270 con 0x7f9b60069d10 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6579a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b600c8820 0x7f9b600c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6579a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b600c8820 0x7f9b600c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36838/0 (socket says 172.21.15.131:36838) 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6579a700 1 -- 172.21.15.131:0/443914866 learned_addr learned my addr 172.21.15.131:0/443914866 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6679c700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b600c8bf0 0x7f9b600c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b65f9b700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b60069d10 0x7f9b601684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6579a700 1 -- 172.21.15.131:0/443914866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b60069d10 msgr2=0x7f9b601684b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6579a700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b60069d10 0x7f9b601684b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6579a700 1 -- 172.21.15.131:0/443914866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b600c8bf0 msgr2=0x7f9b600c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6579a700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b600c8bf0 0x7f9b600c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6579a700 1 -- 172.21.15.131:0/443914866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b50009420 con 0x7f9b600c8820 2023-12-26T23:12:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6679c700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b600c8bf0 0x7f9b600c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:12:58.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.971+0000 7f9b6579a700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b600c8820 0x7f9b600c3700 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009f00 tx=0x7f9b50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:58.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.972+0000 7f9b55ffb700 1 -- 172.21.15.131:0/443914866 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5001f070 con 0x7f9b600c8820 2023-12-26T23:12:58.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.972+0000 7f9b55ffb700 1 -- 172.21.15.131:0/443914866 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b50000bc0 con 0x7f9b600c8820 2023-12-26T23:12:58.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.972+0000 7f9b55ffb700 1 -- 172.21.15.131:0/443914866 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50025420 con 0x7f9b600c8820 2023-12-26T23:12:58.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.972+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b601679d0 con 0x7f9b600c8820 2023-12-26T23:12:58.975 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.972+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b60167f90 con 0x7f9b600c8820 2023-12-26T23:12:58.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.973+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b601681d0 con 0x7f9b600c8820 2023-12-26T23:12:58.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.974+0000 7f9b55ffb700 1 -- 172.21.15.131:0/443914866 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9b50023070 con 0x7f9b600c8820 2023-12-26T23:12:58.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.975+0000 7f9b55ffb700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9b44066f30 0x7f9b440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:58.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.975+0000 7f9b55ffb700 1 -- 172.21.15.131:0/443914866 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f9b50085a00 con 0x7f9b600c8820 2023-12-26T23:12:58.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.975+0000 7f9b65f9b700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9b44066f30 0x7f9b440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:58.976 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.976+0000 7f9b65f9b700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9b44066f30 0x7f9b440693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c007fd0 tx=0x7f9b5c007d50 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:58.979+0000 7f9b55ffb700 1 -- 172.21.15.131:0/443914866 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b601681d0 con 0x7f9b600c8820 2023-12-26T23:12:59.168 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.168+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b601681d0 con 0x7f9b44066f30 2023-12-26T23:12:59.171 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.171+0000 7f9b55ffb700 1 -- 172.21.15.131:0/443914866 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50697 (secure 0 0 0) 0x7f9b601681d0 con 0x7f9b44066f30 2023-12-26T23:12:59.172 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:59.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9b44066f30 msgr2=0x7f9b440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9b44066f30 0x7f9b440693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c007fd0 tx=0x7f9b5c007d50 comp rx=0 tx=0).stop 2023-12-26T23:12:59.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b600c8820 msgr2=0x7f9b600c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b600c8820 0x7f9b600c3700 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009f00 tx=0x7f9b50017000 comp rx=0 tx=0).stop 2023-12-26T23:12:59.174 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 shutdown_connections 2023-12-26T23:12:59.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9b44066f30 0x7f9b440693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b600c8bf0 0x7f9b600c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b60069d10 0x7f9b601684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 --2- 172.21.15.131:0/443914866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b600c8820 0x7f9b600c3700 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 >> 172.21.15.131:0/443914866 conn(0x7f9b60068040 msgr2=0x7f9b6006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:59.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 shutdown_connections 2023-12-26T23:12:59.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.173+0000 7f9b681ff700 1 -- 172.21.15.131:0/443914866 wait complete. 2023-12-26T23:12:59.176 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:12:59.192 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2023-12-26T23:12:57.243586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2454856,"kb_used_data":23304,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752519864,"statfs":{"total":773094113280,"available":770580340736,"internally_reserved":0,"allocated":23863296,"data_stored":13256694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668747,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-281,"num_write_kb":-2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.995393"},"pg_stats":[{"pgid":"1.0","version":"213'370","reported_seq":1233,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-26T23:12:53.306611+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:12:53.306611+0000","last_peered":"2023-12-26T23:12:53.306611+0000","last_clean":"2023-12-26T23:12:53.306611+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:12:53.306611+0000","last_undegraded":"2023-12-26T23:12:53.306611+0000","last_fullsized":"2023-12-26T23:12:53.306611+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543284,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308104,"kb_used_data":2624,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063736,"statfs":{"total":96636764160,"available":96321265664,"internally_reserved":0,"allocated":2686976,"data_stored":1354384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.98199999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.55100000000000005}]},{"osd":1,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.498,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.72199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.0449999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.94899999999999995,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0760000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.59999999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.44500000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63400000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.5,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.5},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.59699999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":1.028,"15min":1.028},"last":0.65100000000000002}]}]},{"osd":6,"up_from":15,"seq":64424510580,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308400,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063440,"statfs":{"total":96636764160,"available":96320962560,"internally_reserved":0,"allocated":2990080,"data_stored":1670078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.32800000000000001},"max":{"1min":0.873,"5min":0.998,"15min":1.1080000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0680000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":1.022,"15min":1.046},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.95999999999999996,"15min":1.0580000000000001},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91500000000000004,"15min":1.071},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":0.89800000000000002,"5min":0.97899999999999998,"15min":1.0680000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.96699999999999997,"15min":1.024},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":1.0149999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.018,"15min":1.1599999999999999},"last":0.438},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.111},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.55400000000000005}]},{"osd":7,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.495}]}]},{"osd":1,"up_from":15,"seq":64424510580,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300468,"kb_used_data":3180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071372,"statfs":{"total":96636764160,"available":96329084928,"internally_reserved":0,"allocated":3256320,"data_stored":1932777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.33900000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.84099999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.272,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51600000000000001}]},{"osd":4,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.53600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.103},"last":0.62},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":0.95499999999999996,"5min":1.258,"15min":1.258},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.57899999999999996,"5min":0.56000000000000005,"15min":0.51300000000000001},"max":{"1min":1.0329999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.503},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82099999999999995,"15min":0.82999999999999996},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":0.82799999999999996},"min":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0609999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.83399999999999996,"15min":0.84099999999999997},"min":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.091,"5min":1.1559999999999999,"15min":1.2090000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.82299999999999995},"min":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.218},"last":0.56999999999999995}]}]},{"osd":0,"up_from":15,"seq":64424510580,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308084,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063756,"statfs":{"total":96636764160,"available":96321286144,"internally_reserved":0,"allocated":2666496,"data_stored":1342896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.27600000000000002},"max":{"1min":0.60699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82999999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.91800000000000004,"15min":1.012},"last":0.67200000000000004}]},{"osd":4,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.2250000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.49299999999999999},"max":{"1min":1.02,"5min":1.0649999999999999,"15min":1.113},"last":0.77000000000000002}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.157},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.749,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.137,"5min":1.1459999999999999,"15min":1.282},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.085,"5min":1.121,"15min":1.151},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79600000000000004,"15min":0.80300000000000005},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93200000000000005,"5min":1.167,"15min":1.167},"last":0.78700000000000003}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0640000000000001,"5min":1.1439999999999999,"15min":1.256},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.81200000000000006},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54000000000000004},"max":{"1min":1.042,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.80400000000000005}]}]},{"osd":2,"up_from":14,"seq":60129543285,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312504,"kb_used_data":2928,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059336,"statfs":{"total":96636764160,"available":96316760064,"internally_reserved":0,"allocated":2998272,"data_stored":1670711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64100000000000001,"5min":0.67800000000000005,"15min":0.84599999999999997},"last":0.308},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64700000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.53000000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.624,"15min":0.625},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.73399999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.503},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75,"5min":0.89300000000000002,"15min":1.056},"last":0.33000000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77600000000000002,"15min":0.79500000000000004},"min":{"1min":0.56299999999999994,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83299999999999996,"5min":1.8,"15min":1.8},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.085,"5min":1.222,"15min":1.222},"last":0.66000000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.79600000000000004},"min":{"1min":0.60099999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1020000000000001,"5min":1.923,"15min":1.923},"last":0.748},{"interface":"front","average":{"1min":0.753,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95599999999999996,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.57999999999999996}]},{"osd":6,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89400000000000002,"5min":0.97799999999999998,"15min":1.272},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.69599999999999995,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0449999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.71099999999999997}]},{"osd":7,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.81299999999999994},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1619999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.65600000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.175,"5min":1.841,"15min":1.841},"last":0.627}]}]},{"osd":3,"up_from":15,"seq":64424510581,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304328,"kb_used_data":2944,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067512,"statfs":{"total":96636764160,"available":96325132288,"internally_reserved":0,"allocated":3014656,"data_stored":1682574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.48699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56999999999999995,"5min":0.746,"15min":0.76500000000000001},"last":0.443}]},{"osd":1,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47099999999999997,"15min":0.47799999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.753},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.34100000000000003},"max":{"1min":0.69399999999999995,"5min":0.77100000000000002,"15min":0.90400000000000003},"last":0.58199999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.628,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.0309999999999999},"last":0.71699999999999997}]},{"osd":4,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.113,"5min":1.159,"15min":1.177},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.93400000000000005,"5min":0.999,"15min":1.081},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.58399999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.028,"5min":1.046,"15min":1.0980000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.79900000000000004},"min":{"1min":0.53500000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.61399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.999,"5min":1.202,"15min":1.202},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":0.84399999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.177,"15min":1.177},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.53300000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.214},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0549999999999999,"5min":1.292,"15min":1.292},"last":0.68400000000000005}]}]},{"osd":4,"up_from":14,"seq":60129543284,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312176,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059664,"statfs":{"total":96636764160,"available":96317095936,"internally_reserved":0,"allocated":2662400,"data_stored":1342342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.70199999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.877,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.68400000000000005,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":0.94599999999999995,"15min":1.008},"last":0.58799999999999997}]},{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.437,"15min":0.437},"max":{"1min":0.84399999999999997,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85299999999999998,"5min":0.89000000000000001,"15min":1.0329999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82699999999999996,"5min":0.92900000000000005,"15min":0.998},"last":0.60499999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.52300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72199999999999998,"15min":0.75600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81100000000000005,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.75,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.36799999999999999},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":0.98999999999999999},"last":0.38300000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"max":{"1min":0.81899999999999995,"5min":1.002,"15min":1.024},"last":0.497},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.65200000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86599999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.435},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.54400000000000004,"5min":0.53700000000000003,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.621}]}]},{"osd":5,"up_from":15,"seq":64424510579,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300792,"kb_used_data":3504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071048,"statfs":{"total":96636764160,"available":96328753152,"internally_reserved":0,"allocated":3588096,"data_stored":2260932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.60799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.92700000000000005,"15min":1.0629999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.016},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.012,"15min":1.022},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.99199999999999999},"last":0.58999999999999997}]},{"osd":3,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":0.997,"15min":0.997},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93000000000000005,"15min":0.95199999999999996},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.97799999999999998,"15min":1.042},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.90600000000000003,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.56799999999999995}]},{"osd":7,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.90500000000000003,"15min":0.996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.496}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:12:59.194 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:12:59.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.386+0000 7fbbdc7ff700 1 Processor -- start 2023-12-26T23:12:59.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.386+0000 7fbbdc7ff700 1 -- start start 2023-12-26T23:12:59.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.387+0000 7fbbdc7ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbbd4159a70 0x7fbbd40b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.387+0000 7fbbdc7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd40b2a90 0x7fbbd40b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.387+0000 7fbbdc7ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd40afcd0 con 0x7fbbd41596a0 2023-12-26T23:12:59.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.387+0000 7fbbdc7ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd40b3e70 con 0x7fbbd4159a70 2023-12-26T23:12:59.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.387+0000 7fbbdc7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40b0f20 con 0x7fbbd40b2a90 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.387+0000 7fbbd9d9a700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbbd4159a70 0x7fbbd40b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.387+0000 7fbbda59b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbbd41596a0 0x7fbbd40b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43962/0 (socket says 172.21.15.131:43962) 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.387+0000 7fbbda59b700 1 -- 172.21.15.131:0/222551698 learned_addr learned my addr 172.21.15.131:0/222551698 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.387+0000 7fbbdad9c700 1 --2- 172.21.15.131:0/222551698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd40b2a90 0x7fbbd40b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.388+0000 7fbbd9d9a700 1 -- 172.21.15.131:0/222551698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd40b2a90 msgr2=0x7fbbd40b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.388+0000 7fbbd9d9a700 1 --2- 172.21.15.131:0/222551698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd40b2a90 0x7fbbd40b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.388+0000 7fbbd9d9a700 1 -- 172.21.15.131:0/222551698 >> v1:172.21.15.131:6789/0 conn(0x7fbbd41596a0 legacy=0x7fbbd40b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.388+0000 7fbbd9d9a700 1 -- 172.21.15.131:0/222551698 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd40b3540 con 0x7fbbd4159a70 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.388+0000 7fbbd9599700 1 -- 172.21.15.131:0/222551698 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773820664 0 0) 0x7fbbd40afcd0 con 0x7fbbd41596a0 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.388+0000 7fbbd9d9a700 1 --2- 172.21.15.131:0/222551698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 0x7fbbd40b0160 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40098e0 tx=0x7fbbc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369c5d95cd178aad server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.388+0000 7fbbd9599700 1 -- 172.21.15.131:0/222551698 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400b640 con 0x7fbbd4159a70 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.388+0000 7fbbd9599700 1 -- 172.21.15.131:0/222551698 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc400b7a0 con 0x7fbbd4159a70 2023-12-26T23:12:59.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.388+0000 7fbbd9599700 1 -- 172.21.15.131:0/222551698 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400bb40 con 0x7fbbd4159a70 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.389+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/222551698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 msgr2=0x7fbbd40b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.389+0000 7fbbdc7ff700 1 --2- 172.21.15.131:0/222551698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 0x7fbbd40b0160 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40098e0 tx=0x7fbbc40091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.389+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/222551698 shutdown_connections 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.389+0000 7fbbdc7ff700 1 --2- 172.21.15.131:0/222551698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd40b2a90 0x7fbbd40b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.389+0000 7fbbdc7ff700 1 --2- 172.21.15.131:0/222551698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 0x7fbbd40b0160 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.389+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/222551698 >> 172.21.15.131:0/222551698 conn(0x7fbbd406ece0 msgr2=0x7fbbd406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.389+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/222551698 shutdown_connections 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.389+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/222551698 wait complete. 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.390+0000 7fbbdc7ff700 1 Processor -- start 2023-12-26T23:12:59.391 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.390+0000 7fbbdc7ff700 1 -- start start 2023-12-26T23:12:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.391+0000 7fbbdc7ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd40b2a90 0x7fbbd4159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.391+0000 7fbbdc7ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd41596a0 0x7fbbd41503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.393 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.391+0000 7fbbdc7ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 0x7fbbd40c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.391+0000 7fbbdc7ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd415ca10 con 0x7fbbd4159a70 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.391+0000 7fbbdc7ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd415cc30 con 0x7fbbd40b2a90 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.391+0000 7fbbdc7ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40c9060 con 0x7fbbd41596a0 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbdad9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 0x7fbbd40c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbdad9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 0x7fbbd40c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36868/0 (socket says 172.21.15.131:36868) 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbdad9c700 1 -- 172.21.15.131:0/1394731680 learned_addr learned my addr 172.21.15.131:0/1394731680 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbd9d9a700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd41596a0 0x7fbbd41503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbda59b700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd40b2a90 0x7fbbd4159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbdad9c700 1 -- 172.21.15.131:0/1394731680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd41596a0 msgr2=0x7fbbd41503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbdad9c700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd41596a0 0x7fbbd41503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbdad9c700 1 -- 172.21.15.131:0/1394731680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd40b2a90 msgr2=0x7fbbd4159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbdad9c700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd40b2a90 0x7fbbd4159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbdad9c700 1 -- 172.21.15.131:0/1394731680 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc4009420 con 0x7fbbd4159a70 2023-12-26T23:12:59.394 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.392+0000 7fbbdad9c700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 0x7fbbd40c9860 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00ea50 tx=0x7fbbcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:59.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.393+0000 7fbbca7fc700 1 -- 172.21.15.131:0/1394731680 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc00ee90 con 0x7fbbd4159a70 2023-12-26T23:12:59.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.393+0000 7fbbca7fc700 1 -- 172.21.15.131:0/1394731680 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbcc00c850 con 0x7fbbd4159a70 2023-12-26T23:12:59.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.393+0000 7fbbca7fc700 1 -- 172.21.15.131:0/1394731680 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc00cc10 con 0x7fbbd4159a70 2023-12-26T23:12:59.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.393+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd40c9340 con 0x7fbbd4159a70 2023-12-26T23:12:59.395 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.393+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd415c510 con 0x7fbbd4159a70 2023-12-26T23:12:59.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.394+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd415ce00 con 0x7fbbd4159a70 2023-12-26T23:12:59.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.395+0000 7fbbca7fc700 1 -- 172.21.15.131:0/1394731680 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbbcc013470 con 0x7fbbd4159a70 2023-12-26T23:12:59.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.395+0000 7fbbca7fc700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbbb8066f30 0x7fbbb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.396+0000 7fbbca7fc700 1 -- 172.21.15.131:0/1394731680 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fbbcc0848a0 con 0x7fbbd4159a70 2023-12-26T23:12:59.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.396+0000 7fbbda59b700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbbb8066f30 0x7fbbb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.396 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.396+0000 7fbbda59b700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbbb8066f30 0x7fbbb80693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0002250 tx=0x7fbbd0001a20 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:59.400 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.400+0000 7fbbca7fc700 1 -- 172.21.15.131:0/1394731680 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbcc051930 con 0x7fbbd4159a70 2023-12-26T23:12:59.588 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.588+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbd40c8ef0 con 0x7fbbb8066f30 2023-12-26T23:12:59.591 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.591+0000 7fbbca7fc700 1 -- 172.21.15.131:0/1394731680 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50695 (secure 0 0 0) 0x7fbbd40c8ef0 con 0x7fbbb8066f30 2023-12-26T23:12:59.591 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbbb8066f30 msgr2=0x7fbbb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbbb8066f30 0x7fbbb80693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0002250 tx=0x7fbbd0001a20 comp rx=0 tx=0).stop 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 msgr2=0x7fbbd40c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 0x7fbbd40c9860 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00ea50 tx=0x7fbbcc0091a0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 shutdown_connections 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbbb8066f30 0x7fbbb80693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbd4159a70 0x7fbbd40c9860 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbd40b2a90 0x7fbbd4159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 --2- 172.21.15.131:0/1394731680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbd41596a0 0x7fbbd41503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 >> 172.21.15.131:0/1394731680 conn(0x7fbbd406ece0 msgr2=0x7fbbd40af440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.593+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 shutdown_connections 2023-12-26T23:12:59.594 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.594+0000 7fbbdc7ff700 1 -- 172.21.15.131:0/1394731680 wait complete. 2023-12-26T23:12:59.595 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:12:59.611 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-12-26T23:12:59.243990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2458656,"kb_used_data":23008,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752516064,"statfs":{"total":773094113280,"available":770576449536,"internally_reserved":0,"allocated":23560192,"data_stored":12943619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668747,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-281,"num_write_kb":-2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.995798"},"pg_stats":[{"pgid":"1.0","version":"213'370","reported_seq":1233,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-26T23:12:53.306611+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:12:53.306611+0000","last_peered":"2023-12-26T23:12:53.306611+0000","last_clean":"2023-12-26T23:12:53.306611+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:12:53.306611+0000","last_undegraded":"2023-12-26T23:12:53.306611+0000","last_fullsized":"2023-12-26T23:12:53.306611+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.98199999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66400000000000003}]},{"osd":1,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.498,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.72199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.0449999999999999},"last":0.81399999999999995}]},{"osd":2,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.94899999999999995,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1},"last":0.68400000000000005}]},{"osd":3,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0760000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.73599999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.44500000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.54300000000000004}]},{"osd":5,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.872}]},{"osd":6,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.5,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.59699999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":1.028,"15min":1.028},"last":0.75}]}]},{"osd":6,"up_from":15,"seq":64424510580,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308400,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063440,"statfs":{"total":96636764160,"available":96320962560,"internally_reserved":0,"allocated":2990080,"data_stored":1670078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.32800000000000001},"max":{"1min":0.873,"5min":0.998,"15min":1.1080000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0680000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":1.022,"15min":1.046},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.95999999999999996,"15min":1.0580000000000001},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91500000000000004,"15min":1.071},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":0.89800000000000002,"5min":0.97899999999999998,"15min":1.0680000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.96699999999999997,"15min":1.024},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":1.0149999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.018,"15min":1.1599999999999999},"last":0.438},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.111},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.55400000000000005}]},{"osd":7,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.495}]}]},{"osd":1,"up_from":15,"seq":64424510580,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300468,"kb_used_data":3180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071372,"statfs":{"total":96636764160,"available":96329084928,"internally_reserved":0,"allocated":3256320,"data_stored":1932777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.33900000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.84099999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.272,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51600000000000001}]},{"osd":4,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.53600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.103},"last":0.62},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":0.95499999999999996,"5min":1.258,"15min":1.258},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.57899999999999996,"5min":0.56000000000000005,"15min":0.51300000000000001},"max":{"1min":1.0329999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.503},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82099999999999995,"15min":0.82999999999999996},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":0.82799999999999996},"min":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0609999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.83399999999999996,"15min":0.84099999999999997},"min":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.091,"5min":1.1559999999999999,"15min":1.2090000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.82299999999999995},"min":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.218},"last":0.56999999999999995}]}]},{"osd":0,"up_from":15,"seq":64424510580,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308084,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063756,"statfs":{"total":96636764160,"available":96321286144,"internally_reserved":0,"allocated":2666496,"data_stored":1342896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.27600000000000002},"max":{"1min":0.60699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82999999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.91800000000000004,"15min":1.012},"last":0.67200000000000004}]},{"osd":4,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.2250000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.49299999999999999},"max":{"1min":1.02,"5min":1.0649999999999999,"15min":1.113},"last":0.77000000000000002}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.157},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.749,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.137,"5min":1.1459999999999999,"15min":1.282},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.085,"5min":1.121,"15min":1.151},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79600000000000004,"15min":0.80300000000000005},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93200000000000005,"5min":1.167,"15min":1.167},"last":0.78700000000000003}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0640000000000001,"5min":1.1439999999999999,"15min":1.256},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.81200000000000006},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54000000000000004},"max":{"1min":1.042,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.80400000000000005}]}]},{"osd":2,"up_from":14,"seq":60129543286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64100000000000001,"5min":0.67800000000000005,"15min":0.84599999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.38}]},{"osd":1,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64700000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.624,"15min":0.625},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.73399999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.502},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75,"5min":0.89300000000000002,"15min":1.056},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77600000000000002,"15min":0.79500000000000004},"min":{"1min":0.56299999999999994,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83299999999999996,"5min":1.8,"15min":1.8},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.085,"5min":1.222,"15min":1.222},"last":0.92600000000000005}]},{"osd":5,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.79600000000000004},"min":{"1min":0.60099999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1020000000000001,"5min":1.923,"15min":1.923},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95599999999999996,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.78900000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89400000000000002,"5min":0.97799999999999998,"15min":1.272},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.69599999999999995,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0449999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.998}]},{"osd":7,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.81299999999999994},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1619999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.65600000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.175,"5min":1.841,"15min":1.841},"last":0.95199999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510581,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304328,"kb_used_data":2944,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067512,"statfs":{"total":96636764160,"available":96325132288,"internally_reserved":0,"allocated":3014656,"data_stored":1682574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.48699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56999999999999995,"5min":0.746,"15min":0.76500000000000001},"last":0.443}]},{"osd":1,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47099999999999997,"15min":0.47799999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.753},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.34100000000000003},"max":{"1min":0.69399999999999995,"5min":0.77100000000000002,"15min":0.90400000000000003},"last":0.58199999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.628,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.0309999999999999},"last":0.71699999999999997}]},{"osd":4,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.113,"5min":1.159,"15min":1.177},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.93400000000000005,"5min":0.999,"15min":1.081},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.58399999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.028,"5min":1.046,"15min":1.0980000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.79900000000000004},"min":{"1min":0.53500000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.61399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.999,"5min":1.202,"15min":1.202},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":0.84399999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.177,"15min":1.177},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.53300000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.214},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0549999999999999,"5min":1.292,"15min":1.292},"last":0.68400000000000005}]}]},{"osd":4,"up_from":14,"seq":60129543284,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312176,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059664,"statfs":{"total":96636764160,"available":96317095936,"internally_reserved":0,"allocated":2662400,"data_stored":1342342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.70199999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.877,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.68400000000000005,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":0.94599999999999995,"15min":1.008},"last":0.58799999999999997}]},{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.437,"15min":0.437},"max":{"1min":0.84399999999999997,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85299999999999998,"5min":0.89000000000000001,"15min":1.0329999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82699999999999996,"5min":0.92900000000000005,"15min":0.998},"last":0.60499999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.52300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72199999999999998,"15min":0.75600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81100000000000005,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.75,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.36799999999999999},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":0.98999999999999999},"last":0.38300000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"max":{"1min":0.81899999999999995,"5min":1.002,"15min":1.024},"last":0.497},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.65200000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86599999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.435},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.54400000000000004,"5min":0.53700000000000003,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.621}]}]},{"osd":5,"up_from":15,"seq":64424510579,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300792,"kb_used_data":3504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071048,"statfs":{"total":96636764160,"available":96328753152,"internally_reserved":0,"allocated":3588096,"data_stored":2260932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.60799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.92700000000000005,"15min":1.0629999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.016},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.012,"15min":1.022},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.99199999999999999},"last":0.58999999999999997}]},{"osd":3,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":0.997,"15min":0.997},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93000000000000005,"15min":0.95199999999999996},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.97799999999999998,"15min":1.042},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.90600000000000003,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.56799999999999995}]},{"osd":7,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.90500000000000003,"15min":0.996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.496}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:12:59.612 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-26T23:12:59.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:12:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.802+0000 7f51d67ff700 1 Processor -- start 2023-12-26T23:12:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.802+0000 7f51d67ff700 1 -- start start 2023-12-26T23:12:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.803+0000 7f51d67ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d0159600 0x7f51d00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.803+0000 7f51d67ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f51d00d01e0 0x7f51d00ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51d67ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d00cfd50 con 0x7f51d01599d0 2023-12-26T23:12:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51d67ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d00b1280 con 0x7f51d00d01e0 2023-12-26T23:12:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51d67ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d00cf140 con 0x7f51d0159600 2023-12-26T23:12:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51cffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d0159600 0x7f51d00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51d4d9c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f51d00d01e0 0x7f51d00ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51cffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d0159600 0x7f51d00ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:52688/0 (socket says 172.21.15.131:52688) 2023-12-26T23:12:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51cffff700 1 -- 172.21.15.131:0/3009495146 learned_addr learned my addr 172.21.15.131:0/3009495146 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51d4d9c700 1 -- 172.21.15.131:0/3009495146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d0159600 msgr2=0x7f51d00ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:12:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51d4d9c700 1 --2- 172.21.15.131:0/3009495146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d0159600 0x7f51d00ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51d4d9c700 1 -- 172.21.15.131:0/3009495146 >> v1:172.21.15.190:6789/0 conn(0x7f51d01599d0 legacy=0x7f51d015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.804+0000 7f51d4d9c700 1 -- 172.21.15.131:0/3009495146 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d00cf420 con 0x7f51d00d01e0 2023-12-26T23:12:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.805+0000 7f51d4d9c700 1 --2- 172.21.15.131:0/3009495146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51d00d01e0 0x7f51d00ccf00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009e60 tx=0x7f51c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d96c424141c342 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.805+0000 7f51ceffd700 1 -- 172.21.15.131:0/3009495146 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c800e070 con 0x7f51d00d01e0 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.805+0000 7f51ceffd700 1 -- 172.21.15.131:0/3009495146 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c800bb60 con 0x7f51d00d01e0 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.805+0000 7f51ceffd700 1 -- 172.21.15.131:0/3009495146 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8012430 con 0x7f51d00d01e0 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.805+0000 7f51d67ff700 1 -- 172.21.15.131:0/3009495146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51d00d01e0 msgr2=0x7f51d00ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.805+0000 7f51d67ff700 1 --2- 172.21.15.131:0/3009495146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51d00d01e0 0x7f51d00ccf00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009e60 tx=0x7f51c8009780 comp rx=0 tx=0).stop 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.806+0000 7f51d67ff700 1 -- 172.21.15.131:0/3009495146 shutdown_connections 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.806+0000 7f51d67ff700 1 --2- 172.21.15.131:0/3009495146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51d00d01e0 0x7f51d00ccf00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.806+0000 7f51d67ff700 1 --2- 172.21.15.131:0/3009495146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d0159600 0x7f51d00ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.806+0000 7f51d67ff700 1 -- 172.21.15.131:0/3009495146 >> 172.21.15.131:0/3009495146 conn(0x7f51d006ece0 msgr2=0x7f51d006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.806+0000 7f51d67ff700 1 -- 172.21.15.131:0/3009495146 shutdown_connections 2023-12-26T23:12:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.806+0000 7f51d67ff700 1 -- 172.21.15.131:0/3009495146 wait complete. 2023-12-26T23:12:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.807+0000 7f51d67ff700 1 Processor -- start 2023-12-26T23:12:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.807+0000 7f51d67ff700 1 -- start start 2023-12-26T23:12:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.808+0000 7f51d67ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d00d01e0 0x7f51d016aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.808+0000 7f51d67ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51d0159600 0x7f51d016a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.808+0000 7f51d67ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d00b3390 0x7f51d00b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.808+0000 7f51d67ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d014b990 con 0x7f51d00b3390 2023-12-26T23:12:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.808+0000 7f51d67ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d00b3d40 con 0x7f51d0159600 2023-12-26T23:12:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.808+0000 7f51d67ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d014f1e0 con 0x7f51d00d01e0 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.808+0000 7f51d4d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d00b3390 0x7f51d00b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51d4d9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d00b3390 0x7f51d00b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36880/0 (socket says 172.21.15.131:36880) 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51d4d9c700 1 -- 172.21.15.131:0/2418195486 learned_addr learned my addr 172.21.15.131:0/2418195486 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51cffff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d00d01e0 0x7f51d016aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51cf7fe700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51d0159600 0x7f51d016a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51d4d9c700 1 -- 172.21.15.131:0/2418195486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d00d01e0 msgr2=0x7f51d016aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51d4d9c700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d00d01e0 0x7f51d016aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51d4d9c700 1 -- 172.21.15.131:0/2418195486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51d0159600 msgr2=0x7f51d016a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51d4d9c700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51d0159600 0x7f51d016a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:12:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51d4d9c700 1 -- 172.21.15.131:0/2418195486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8009a20 con 0x7f51d00b3390 2023-12-26T23:12:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.809+0000 7f51d4d9c700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d00b3390 0x7f51d00b3800 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f51c8013000 tx=0x7f51c8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.810+0000 7f51b7fff700 1 -- 172.21.15.131:0/2418195486 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8012ea0 con 0x7f51d00b3390 2023-12-26T23:12:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.810+0000 7f51b7fff700 1 -- 172.21.15.131:0/2418195486 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c8027470 con 0x7f51d00b3390 2023-12-26T23:12:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.810+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d00b61b0 con 0x7f51d00b3390 2023-12-26T23:12:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.810+0000 7f51b7fff700 1 -- 172.21.15.131:0/2418195486 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8027850 con 0x7f51d00b3390 2023-12-26T23:12:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.810+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d014f7e0 con 0x7f51d00b3390 2023-12-26T23:12:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.811+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d00657e0 con 0x7f51d00b3390 2023-12-26T23:12:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.812+0000 7f51b7fff700 1 -- 172.21.15.131:0/2418195486 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f51c8024070 con 0x7f51d00b3390 2023-12-26T23:12:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.812+0000 7f51b7fff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f51bc066f30 0x7f51bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:12:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.812+0000 7f51b7fff700 1 -- 172.21.15.131:0/2418195486 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f51c80871a0 con 0x7f51d00b3390 2023-12-26T23:12:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.813+0000 7f51cffff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f51bc066f30 0x7f51bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:12:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.813+0000 7f51cffff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f51bc066f30 0x7f51bc0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51d00885d0 tx=0x7f51c0009220 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:12:59.816 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:12:59.816+0000 7f51b7fff700 1 -- 172.21.15.131:0/2418195486 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51c8054230 con 0x7f51d00b3390 2023-12-26T23:13:00.007 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.007+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51d00cf700 con 0x7f51bc066f30 2023-12-26T23:13:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.010+0000 7f51b7fff700 1 -- 172.21.15.131:0/2418195486 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50695 (secure 0 0 0) 0x7f51d00cf700 con 0x7f51bc066f30 2023-12-26T23:13:00.010 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f51bc066f30 msgr2=0x7f51bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f51bc066f30 0x7f51bc0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51d00885d0 tx=0x7f51c0009220 comp rx=0 tx=0).stop 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d00b3390 msgr2=0x7f51d00b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d00b3390 0x7f51d00b3800 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f51c8013000 tx=0x7f51c8019000 comp rx=0 tx=0).stop 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 shutdown_connections 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f51bc066f30 0x7f51bc0693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d00b3390 0x7f51d00b3800 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51d00d01e0 0x7f51d016aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 --2- 172.21.15.131:0/2418195486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51d0159600 0x7f51d016a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 >> 172.21.15.131:0/2418195486 conn(0x7f51d006ece0 msgr2=0x7f51d00cabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 shutdown_connections 2023-12-26T23:13:00.013 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.012+0000 7f51d67ff700 1 -- 172.21.15.131:0/2418195486 wait complete. 2023-12-26T23:13:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:13:00.030 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-12-26T23:12:59.243990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2458656,"kb_used_data":23008,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752516064,"statfs":{"total":773094113280,"available":770576449536,"internally_reserved":0,"allocated":23560192,"data_stored":12943619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668747,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-281,"num_write_kb":-2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.995798"},"pg_stats":[{"pgid":"1.0","version":"213'370","reported_seq":1233,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-26T23:12:53.306611+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:12:53.306611+0000","last_peered":"2023-12-26T23:12:53.306611+0000","last_clean":"2023-12-26T23:12:53.306611+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:12:53.306611+0000","last_undegraded":"2023-12-26T23:12:53.306611+0000","last_fullsized":"2023-12-26T23:12:53.306611+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T05:41:41.813601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.98199999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66400000000000003}]},{"osd":1,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.498,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.72199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.0449999999999999},"last":0.81399999999999995}]},{"osd":2,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.94899999999999995,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1},"last":0.68400000000000005}]},{"osd":3,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0760000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.73599999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.44500000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.54300000000000004}]},{"osd":5,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.872}]},{"osd":6,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.5,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.59699999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":1.028,"15min":1.028},"last":0.75}]}]},{"osd":6,"up_from":15,"seq":64424510580,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308400,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063440,"statfs":{"total":96636764160,"available":96320962560,"internally_reserved":0,"allocated":2990080,"data_stored":1670078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.32800000000000001},"max":{"1min":0.873,"5min":0.998,"15min":1.1080000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0680000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":1.022,"15min":1.046},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.95999999999999996,"15min":1.0580000000000001},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91500000000000004,"15min":1.071},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":0.89800000000000002,"5min":0.97899999999999998,"15min":1.0680000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.96699999999999997,"15min":1.024},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":1.0149999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.018,"15min":1.1599999999999999},"last":0.438},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.111},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.55400000000000005}]},{"osd":7,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.495}]}]},{"osd":1,"up_from":15,"seq":64424510580,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300468,"kb_used_data":3180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071372,"statfs":{"total":96636764160,"available":96329084928,"internally_reserved":0,"allocated":3256320,"data_stored":1932777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.33900000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.84099999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.272,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51600000000000001}]},{"osd":4,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.53600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.103},"last":0.62},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":0.95499999999999996,"5min":1.258,"15min":1.258},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.57899999999999996,"5min":0.56000000000000005,"15min":0.51300000000000001},"max":{"1min":1.0329999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.503},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82099999999999995,"15min":0.82999999999999996},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":0.82799999999999996},"min":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0609999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.83399999999999996,"15min":0.84099999999999997},"min":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.091,"5min":1.1559999999999999,"15min":1.2090000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.82299999999999995},"min":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.218},"last":0.56999999999999995}]}]},{"osd":0,"up_from":15,"seq":64424510580,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308084,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063756,"statfs":{"total":96636764160,"available":96321286144,"internally_reserved":0,"allocated":2666496,"data_stored":1342896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.27600000000000002},"max":{"1min":0.60699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82999999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.91800000000000004,"15min":1.012},"last":0.67200000000000004}]},{"osd":4,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.2250000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.49299999999999999},"max":{"1min":1.02,"5min":1.0649999999999999,"15min":1.113},"last":0.77000000000000002}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.157},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.749,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.137,"5min":1.1459999999999999,"15min":1.282},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.085,"5min":1.121,"15min":1.151},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79600000000000004,"15min":0.80300000000000005},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93200000000000005,"5min":1.167,"15min":1.167},"last":0.78700000000000003}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0640000000000001,"5min":1.1439999999999999,"15min":1.256},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.81200000000000006},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54000000000000004},"max":{"1min":1.042,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.80400000000000005}]}]},{"osd":2,"up_from":14,"seq":60129543286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64100000000000001,"5min":0.67800000000000005,"15min":0.84599999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.38}]},{"osd":1,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64700000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.624,"15min":0.625},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.73399999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.502},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75,"5min":0.89300000000000002,"15min":1.056},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77600000000000002,"15min":0.79500000000000004},"min":{"1min":0.56299999999999994,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83299999999999996,"5min":1.8,"15min":1.8},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.085,"5min":1.222,"15min":1.222},"last":0.92600000000000005}]},{"osd":5,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.79600000000000004},"min":{"1min":0.60099999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1020000000000001,"5min":1.923,"15min":1.923},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95599999999999996,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.78900000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89400000000000002,"5min":0.97799999999999998,"15min":1.272},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.69599999999999995,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0449999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.998}]},{"osd":7,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.81299999999999994},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1619999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.65600000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.175,"5min":1.841,"15min":1.841},"last":0.95199999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510581,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304328,"kb_used_data":2944,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067512,"statfs":{"total":96636764160,"available":96325132288,"internally_reserved":0,"allocated":3014656,"data_stored":1682574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.48699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56999999999999995,"5min":0.746,"15min":0.76500000000000001},"last":0.443}]},{"osd":1,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47099999999999997,"15min":0.47799999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.753},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.34100000000000003},"max":{"1min":0.69399999999999995,"5min":0.77100000000000002,"15min":0.90400000000000003},"last":0.58199999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.628,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.0309999999999999},"last":0.71699999999999997}]},{"osd":4,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.113,"5min":1.159,"15min":1.177},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.93400000000000005,"5min":0.999,"15min":1.081},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.58399999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.028,"5min":1.046,"15min":1.0980000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.79900000000000004},"min":{"1min":0.53500000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.61399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.999,"5min":1.202,"15min":1.202},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":0.84399999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.177,"15min":1.177},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.53300000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.214},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0549999999999999,"5min":1.292,"15min":1.292},"last":0.68400000000000005}]}]},{"osd":4,"up_from":14,"seq":60129543284,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312176,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059664,"statfs":{"total":96636764160,"available":96317095936,"internally_reserved":0,"allocated":2662400,"data_stored":1342342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.70199999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.877,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.68400000000000005,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":0.94599999999999995,"15min":1.008},"last":0.58799999999999997}]},{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.437,"15min":0.437},"max":{"1min":0.84399999999999997,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85299999999999998,"5min":0.89000000000000001,"15min":1.0329999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82699999999999996,"5min":0.92900000000000005,"15min":0.998},"last":0.60499999999999998}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.52300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72199999999999998,"15min":0.75600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81100000000000005,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.75,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.36799999999999999},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":0.98999999999999999},"last":0.38300000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"max":{"1min":0.81899999999999995,"5min":1.002,"15min":1.024},"last":0.497},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.65200000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86599999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.435},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.54400000000000004,"5min":0.53700000000000003,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.621}]}]},{"osd":5,"up_from":15,"seq":64424510579,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300792,"kb_used_data":3504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071048,"statfs":{"total":96636764160,"available":96328753152,"internally_reserved":0,"allocated":3588096,"data_stored":2260932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.60799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.92700000000000005,"15min":1.0629999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.016},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.012,"15min":1.022},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.99199999999999999},"last":0.58999999999999997}]},{"osd":3,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":0.997,"15min":0.997},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93000000000000005,"15min":0.95199999999999996},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.97799999999999998,"15min":1.042},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.90600000000000003,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.56799999999999995}]},{"osd":7,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.90500000000000003,"15min":0.996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.496}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:13:00.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-26T23:13:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.220+0000 7ff5a37ff700 1 Processor -- start 2023-12-26T23:13:00.220 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.220+0000 7ff5a37ff700 1 -- start start 2023-12-26T23:13:00.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.221+0000 7ff5a37ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8bf0 0x7ff59c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:00.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.221+0000 7ff5a37ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59c0caa10 con 0x7ff59c0c8820 2023-12-26T23:13:00.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.221+0000 7ff5a37ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59c157a80 con 0x7ff59c069d10 2023-12-26T23:13:00.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.221+0000 7ff5a37ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c0cac50 con 0x7ff59c0c8bf0 2023-12-26T23:13:00.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff5a159b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff59c0c8820 0x7ff59c0ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43972/0 (socket says 172.21.15.131:43972) 2023-12-26T23:13:00.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff5a159b700 1 -- 172.21.15.131:0/1073957621 learned_addr learned my addr 172.21.15.131:0/1073957621 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:00.222 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff5a0d9a700 1 --2- 172.21.15.131:0/1073957621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8bf0 0x7ff59c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff593fff700 1 -- 172.21.15.131:0/1073957621 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145344128 0 0) 0x7ff598001110 con 0x7ff59c0c8820 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff593fff700 1 -- 172.21.15.131:0/1073957621 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff588003570 con 0x7ff59c0c8820 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff593fff700 1 -- 172.21.15.131:0/1073957621 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382825284 0 0) 0x7ff59c157a80 con 0x7ff59c069d10 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff593fff700 1 -- 172.21.15.131:0/1073957621 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff598001110 con 0x7ff59c069d10 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff5a0d9a700 1 -- 172.21.15.131:0/1073957621 >> v1:172.21.15.190:6789/0 conn(0x7ff59c069d10 legacy=0x7ff59c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff5a0d9a700 1 -- 172.21.15.131:0/1073957621 >> v1:172.21.15.131:6789/0 conn(0x7ff59c0c8820 legacy=0x7ff59c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.222+0000 7ff5a0d9a700 1 -- 172.21.15.131:0/1073957621 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c0caf30 con 0x7ff59c0c8bf0 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.223+0000 7ff5a0d9a700 1 --2- 172.21.15.131:0/1073957621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8bf0 0x7ff59c0b12e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009910 tx=0x7ff58c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d24c191dbae5bcf server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.223+0000 7ff593fff700 1 -- 172.21.15.131:0/1073957621 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c011630 con 0x7ff59c0c8bf0 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.223+0000 7ff593fff700 1 -- 172.21.15.131:0/1073957621 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c011790 con 0x7ff59c0c8bf0 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.223+0000 7ff593fff700 1 -- 172.21.15.131:0/1073957621 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c011b30 con 0x7ff59c0c8bf0 2023-12-26T23:13:00.224 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.223+0000 7ff5a37ff700 1 -- 172.21.15.131:0/1073957621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8bf0 msgr2=0x7ff59c0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:00.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.223+0000 7ff5a37ff700 1 --2- 172.21.15.131:0/1073957621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8bf0 0x7ff59c0b12e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009910 tx=0x7ff58c0091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.224+0000 7ff5a37ff700 1 -- 172.21.15.131:0/1073957621 shutdown_connections 2023-12-26T23:13:00.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.224+0000 7ff5a37ff700 1 --2- 172.21.15.131:0/1073957621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8bf0 0x7ff59c0b12e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.224+0000 7ff5a37ff700 1 -- 172.21.15.131:0/1073957621 >> 172.21.15.131:0/1073957621 conn(0x7ff59c068040 msgr2=0x7ff59c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:00.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.224+0000 7ff5a37ff700 1 -- 172.21.15.131:0/1073957621 shutdown_connections 2023-12-26T23:13:00.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.224+0000 7ff5a37ff700 1 -- 172.21.15.131:0/1073957621 wait complete. 2023-12-26T23:13:00.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.225+0000 7ff5a37ff700 1 Processor -- start 2023-12-26T23:13:00.225 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.225+0000 7ff5a37ff700 1 -- start start 2023-12-26T23:13:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a37ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59c069d10 0x7ff59c144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a37ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8820 0x7ff59c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a37ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59c0c8bf0 0x7ff59c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a37ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c0c1960 con 0x7ff59c069d10 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a37ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c151290 con 0x7ff59c0c8bf0 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a37ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c0c1030 con 0x7ff59c0c8820 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a159b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59c069d10 0x7ff59c144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a159b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59c069d10 0x7ff59c144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36888/0 (socket says 172.21.15.131:36888) 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a159b700 1 -- 172.21.15.131:0/3186674748 learned_addr learned my addr 172.21.15.131:0/3186674748 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a0d9a700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8820 0x7ff59c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.226+0000 7ff5a1d9c700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59c0c8bf0 0x7ff59c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5a159b700 1 -- 172.21.15.131:0/3186674748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8820 msgr2=0x7ff59c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5a159b700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8820 0x7ff59c156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5a159b700 1 -- 172.21.15.131:0/3186674748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59c0c8bf0 msgr2=0x7ff59c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:00.228 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5a159b700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59c0c8bf0 0x7ff59c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5a159b700 1 -- 172.21.15.131:0/3186674748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c009420 con 0x7ff59c069d10 2023-12-26T23:13:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5a0d9a700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8820 0x7ff59c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5a159b700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59c069d10 0x7ff59c144d30 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c790 tx=0x7ff598007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5917fa700 1 -- 172.21.15.131:0/3186674748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598005b30 con 0x7ff59c069d10 2023-12-26T23:13:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5917fa700 1 -- 172.21.15.131:0/3186674748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff598005c90 con 0x7ff59c069d10 2023-12-26T23:13:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c0c1310 con 0x7ff59c069d10 2023-12-26T23:13:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5917fa700 1 -- 172.21.15.131:0/3186674748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598007300 con 0x7ff59c069d10 2023-12-26T23:13:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.227+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c156c80 con 0x7ff59c069d10 2023-12-26T23:13:00.229 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.228+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c0657e0 con 0x7ff59c069d10 2023-12-26T23:13:00.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.229+0000 7ff5917fa700 1 -- 172.21.15.131:0/3186674748 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff59801b400 con 0x7ff59c069d10 2023-12-26T23:13:00.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.230+0000 7ff5917fa700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff580066f30 0x7ff5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:00.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.230+0000 7ff5917fa700 1 -- 172.21.15.131:0/3186674748 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7ff5980840d0 con 0x7ff59c069d10 2023-12-26T23:13:00.230 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.230+0000 7ff5a0d9a700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff580066f30 0x7ff5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:00.231 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.231+0000 7ff5a0d9a700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff580066f30 0x7ff5800693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009860 tx=0x7ff58c017000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:00.234 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.234+0000 7ff5917fa700 1 -- 172.21.15.131:0/3186674748 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff598054c70 con 0x7ff59c069d10 2023-12-26T23:13:00.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.431+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff59c0caa10 con 0x7ff59c069d10 2023-12-26T23:13:00.432 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.432+0000 7ff5917fa700 1 -- 172.21.15.131:0/3186674748 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+13938 (secure 0 0 0) 0x7ff59c0caa10 con 0x7ff59c069d10 2023-12-26T23:13:00.432 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:13:00.432 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":215,"fsid":"ab603302-d88b-429a-be64-92b3c298adad","created":"2023-12-26T21:36:24.197085+0000","modified":"2023-12-26T23:12:55.276294+0000","last_up_change":"2023-12-26T21:36:36.563433+0000","last_in_change":"2023-12-26T21:36:27.695410+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-26T21:36:37.054865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"81a77a72-fe92-4844-a340-b892cc90cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":39607},{"type":"v1","addr":"172.21.15.131:6809","nonce":39607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":39607},{"type":"v1","addr":"172.21.15.131:6811","nonce":39607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":39607},{"type":"v1","addr":"172.21.15.131:6815","nonce":39607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":39607},{"type":"v1","addr":"172.21.15.131:6813","nonce":39607}]},"public_addr":"172.21.15.131:6809/39607","cluster_addr":"172.21.15.131:6811/39607","heartbeat_back_addr":"172.21.15.131:6815/39607","heartbeat_front_addr":"172.21.15.131:6813/39607","state":["exists","up"]},{"osd":1,"uuid":"3c1f2ba2-1978-4822-8e5a-bdbdaffe9cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":39610},{"type":"v1","addr":"172.21.15.131:6818","nonce":39610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":39610},{"type":"v1","addr":"172.21.15.131:6821","nonce":39610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6825","nonce":39610},{"type":"v1","addr":"172.21.15.131:6828","nonce":39610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":39610},{"type":"v1","addr":"172.21.15.131:6824","nonce":39610}]},"public_addr":"172.21.15.131:6818/39610","cluster_addr":"172.21.15.131:6821/39610","heartbeat_back_addr":"172.21.15.131:6828/39610","heartbeat_front_addr":"172.21.15.131:6824/39610","state":["exists","up"]},{"osd":2,"uuid":"49cabd4f-9407-4b82-9bea-e959f127e69f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":39608},{"type":"v1","addr":"172.21.15.131:6801","nonce":39608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":39608},{"type":"v1","addr":"172.21.15.131:6803","nonce":39608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":39608},{"type":"v1","addr":"172.21.15.131:6807","nonce":39608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":39608},{"type":"v1","addr":"172.21.15.131:6805","nonce":39608}]},"public_addr":"172.21.15.131:6801/39608","cluster_addr":"172.21.15.131:6803/39608","heartbeat_back_addr":"172.21.15.131:6807/39608","heartbeat_front_addr":"172.21.15.131:6805/39608","state":["exists","up"]},{"osd":3,"uuid":"43c1b543-b3c5-40ad-b1df-34fcd9cd23e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":39609},{"type":"v1","addr":"172.21.15.131:6820","nonce":39609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":39609},{"type":"v1","addr":"172.21.15.131:6826","nonce":39609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":39609},{"type":"v1","addr":"172.21.15.131:6831","nonce":39609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6827","nonce":39609},{"type":"v1","addr":"172.21.15.131:6829","nonce":39609}]},"public_addr":"172.21.15.131:6820/39609","cluster_addr":"172.21.15.131:6826/39609","heartbeat_back_addr":"172.21.15.131:6831/39609","heartbeat_front_addr":"172.21.15.131:6829/39609","state":["exists","up"]},{"osd":4,"uuid":"bd6a60f8-0595-4fdb-91e3-cd1321c248a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6801","nonce":40010},{"type":"v1","addr":"172.21.15.190:6803","nonce":40010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6805","nonce":40010},{"type":"v1","addr":"172.21.15.190:6807","nonce":40010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":40010},{"type":"v1","addr":"172.21.15.190:6815","nonce":40010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6809","nonce":40010},{"type":"v1","addr":"172.21.15.190:6811","nonce":40010}]},"public_addr":"172.21.15.190:6803/40010","cluster_addr":"172.21.15.190:6807/40010","heartbeat_back_addr":"172.21.15.190:6815/40010","heartbeat_front_addr":"172.21.15.190:6811/40010","state":["exists","up"]},{"osd":5,"uuid":"ceb3f886-e96a-4cca-8136-58614b96b5d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":40012},{"type":"v1","addr":"172.21.15.190:6818","nonce":40012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":40012},{"type":"v1","addr":"172.21.15.190:6821","nonce":40012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":40012},{"type":"v1","addr":"172.21.15.190:6828","nonce":40012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":40012},{"type":"v1","addr":"172.21.15.190:6825","nonce":40012}]},"public_addr":"172.21.15.190:6818/40012","cluster_addr":"172.21.15.190:6821/40012","heartbeat_back_addr":"172.21.15.190:6828/40012","heartbeat_front_addr":"172.21.15.190:6825/40012","state":["exists","up"]},{"osd":6,"uuid":"a3915ff5-2514-4fcf-92e0-8f9144eab84c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":40035},{"type":"v1","addr":"172.21.15.190:6819","nonce":40035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":40035},{"type":"v1","addr":"172.21.15.190:6824","nonce":40035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":40035},{"type":"v1","addr":"172.21.15.190:6831","nonce":40035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":40035},{"type":"v1","addr":"172.21.15.190:6829","nonce":40035}]},"public_addr":"172.21.15.190:6819/40035","cluster_addr":"172.21.15.190:6824/40035","heartbeat_back_addr":"172.21.15.190:6831/40035","heartbeat_front_addr":"172.21.15.190:6829/40035","state":["exists","up"]},{"osd":7,"uuid":"2e7abc9f-f98a-4741-b738-321bf1e85a40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":40011},{"type":"v1","addr":"172.21.15.190:6802","nonce":40011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":40011},{"type":"v1","addr":"172.21.15.190:6806","nonce":40011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":40011},{"type":"v1","addr":"172.21.15.190:6814","nonce":40011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":40011},{"type":"v1","addr":"172.21.15.190:6810","nonce":40011}]},"public_addr":"172.21.15.190:6802/40011","cluster_addr":"172.21.15.190:6806/40011","heartbeat_back_addr":"172.21.15.190:6814/40011","heartbeat_front_addr":"172.21.15.190:6810/40011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.069154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.078807+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.779926+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.058152+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.931155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.053024+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:33.001888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-12-26T21:36:32.965704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/2756115966":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/712424342":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/1157583804":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/296797731":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/223069446":"2023-12-27T23:12:53.226086+0000","172.21.15.190:6834/1917956201":"2023-12-27T23:12:52.271063+0000","172.21.15.131:0/1855853217":"2023-12-27T23:12:53.226086+0000","172.21.15.190:6832/1917956201":"2023-12-27T23:12:52.271063+0000","172.21.15.131:0/3395260924":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/1419722337":"2023-12-27T23:12:53.226086+0000","172.21.15.131:0/3886778096":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/3508392670":"2023-12-27T23:12:42.707929+0000","172.21.15.131:0/1340664010":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6837/92228":"2023-12-27T23:12:42.707929+0000","172.21.15.131:6836/92228":"2023-12-27T23:12:42.707929+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":2},{"begin":10,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":25,"length":2},{"begin":30,"length":3},{"begin":36,"length":2},{"begin":41,"length":3},{"begin":47,"length":2},{"begin":53,"length":2},{"begin":58,"length":2},{"begin":64,"length":2},{"begin":69,"length":2},{"begin":75,"length":2},{"begin":80,"length":2},{"begin":85,"length":2},{"begin":90,"length":2},{"begin":95,"length":2},{"begin":100,"length":2},{"begin":105,"length":2},{"begin":109,"length":3},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff580066f30 msgr2=0x7ff5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff580066f30 0x7ff5800693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009860 tx=0x7ff58c017000 comp rx=0 tx=0).stop 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59c069d10 msgr2=0x7ff59c144d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59c069d10 0x7ff59c144d30 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c790 tx=0x7ff598007ea0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 shutdown_connections 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff580066f30 0x7ff5800693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff59c0c8bf0 0x7ff59c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff59c069d10 0x7ff59c144d30 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 --2- 172.21.15.131:0/3186674748 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff59c0c8820 0x7ff59c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.434+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 >> 172.21.15.131:0/3186674748 conn(0x7ff59c068040 msgr2=0x7ff59c0ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.435+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 shutdown_connections 2023-12-26T23:13:00.435 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:00.435+0000 7ff5a37ff700 1 -- 172.21.15.131:0/3186674748 wait complete. 2023-12-26T23:13:01.453 INFO:tasks.ceph:Scrubbing osd.0 2023-12-26T23:13:01.454 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-12-26T23:13:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.658+0000 7ff53b3ff700 1 Processor -- start 2023-12-26T23:13:01.659 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.658+0000 7ff53b3ff700 1 -- start start 2023-12-26T23:13:01.662 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff53b3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5340cf390 0x7ff5340c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff53b3ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5340cbb00 con 0x7ff534159810 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff53b3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5340ce040 con 0x7ff5340cd0a0 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff53b3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340cb2a0 con 0x7ff5340cf390 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff53919b700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff534159810 0x7ff5340cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43978/0 (socket says 172.21.15.131:43978) 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff53919b700 1 -- 172.21.15.131:0/3308651817 learned_addr learned my addr 172.21.15.131:0/3308651817 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff53899a700 1 --2- 172.21.15.131:0/3308651817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5340cf390 0x7ff5340c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff52bfff700 1 -- 172.21.15.131:0/3308651817 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774556697 0 0) 0x7ff5340cbb00 con 0x7ff534159810 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff52bfff700 1 -- 172.21.15.131:0/3308651817 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff520003570 con 0x7ff534159810 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.660+0000 7ff52bfff700 1 -- 172.21.15.131:0/3308651817 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943345424 0 0) 0x7ff5340ce040 con 0x7ff5340cd0a0 2023-12-26T23:13:01.663 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.661+0000 7ff52bfff700 1 -- 172.21.15.131:0/3308651817 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5340cbb00 con 0x7ff5340cd0a0 2023-12-26T23:13:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.661+0000 7ff53899a700 1 -- 172.21.15.131:0/3308651817 >> v1:172.21.15.190:6789/0 conn(0x7ff5340cd0a0 legacy=0x7ff5340cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.661+0000 7ff53899a700 1 -- 172.21.15.131:0/3308651817 >> v1:172.21.15.131:6789/0 conn(0x7ff534159810 legacy=0x7ff5340cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.661+0000 7ff52bfff700 1 -- 172.21.15.131:0/3308651817 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 650963871 0 0) 0x7ff520003570 con 0x7ff534159810 2023-12-26T23:13:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.661+0000 7ff53899a700 1 -- 172.21.15.131:0/3308651817 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff534159be0 con 0x7ff5340cf390 2023-12-26T23:13:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.661+0000 7ff53899a700 1 --2- 172.21.15.131:0/3308651817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5340cf390 0x7ff5340c5440 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff524009910 tx=0x7ff5240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff51df0ab2f6a49 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.661+0000 7ff52bfff700 1 -- 172.21.15.131:0/3308651817 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5300015f0 con 0x7ff5340cf390 2023-12-26T23:13:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.661+0000 7ff52bfff700 1 -- 172.21.15.131:0/3308651817 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff524011420 con 0x7ff5340cf390 2023-12-26T23:13:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.661+0000 7ff52bfff700 1 -- 172.21.15.131:0/3308651817 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5240117a0 con 0x7ff5340cf390 2023-12-26T23:13:01.664 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.662+0000 7ff53b3ff700 1 -- 172.21.15.131:0/3308651817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5340cf390 msgr2=0x7ff5340c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.662+0000 7ff53b3ff700 1 --2- 172.21.15.131:0/3308651817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5340cf390 0x7ff5340c5440 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff524009910 tx=0x7ff5240091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.662+0000 7ff53b3ff700 1 -- 172.21.15.131:0/3308651817 shutdown_connections 2023-12-26T23:13:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.662+0000 7ff53b3ff700 1 --2- 172.21.15.131:0/3308651817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5340cf390 0x7ff5340c5440 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.662+0000 7ff53b3ff700 1 -- 172.21.15.131:0/3308651817 >> 172.21.15.131:0/3308651817 conn(0x7ff534099950 msgr2=0x7ff53406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.662+0000 7ff53b3ff700 1 -- 172.21.15.131:0/3308651817 shutdown_connections 2023-12-26T23:13:01.665 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.662+0000 7ff53b3ff700 1 -- 172.21.15.131:0/3308651817 wait complete. 2023-12-26T23:13:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.663+0000 7ff53b3ff700 1 Processor -- start 2023-12-26T23:13:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.663+0000 7ff53b3ff700 1 -- start start 2023-12-26T23:13:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.664+0000 7ff53b3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5340cd0a0 0x7ff534153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.664+0000 7ff53b3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5340cf390 0x7ff5340c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.664+0000 7ff53b3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff534159810 0x7ff5340c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.668 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.664+0000 7ff53b3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c85b0 con 0x7ff5340cd0a0 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.664+0000 7ff53b3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340b3970 con 0x7ff5340cf390 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.664+0000 7ff53b3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c9870 con 0x7ff534159810 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.664+0000 7ff53919b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5340cd0a0 0x7ff534153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53919b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5340cd0a0 0x7ff534153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36894/0 (socket says 172.21.15.131:36894) 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53919b700 1 -- 172.21.15.131:0/2417598365 learned_addr learned my addr 172.21.15.131:0/2417598365 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53899a700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5340cf390 0x7ff5340c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53999c700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff534159810 0x7ff5340c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53919b700 1 -- 172.21.15.131:0/2417598365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff534159810 msgr2=0x7ff5340c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53919b700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff534159810 0x7ff5340c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.669 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53919b700 1 -- 172.21.15.131:0/2417598365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5340cf390 msgr2=0x7ff5340c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53919b700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5340cf390 0x7ff5340c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53919b700 1 -- 172.21.15.131:0/2417598365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff524009420 con 0x7ff5340cd0a0 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53999c700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff534159810 0x7ff5340c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53899a700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5340cf390 0x7ff5340c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53919b700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5340cd0a0 0x7ff534153580 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7ff53000cb90 tx=0x7ff530002b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff5297fa700 1 -- 172.21.15.131:0/2417598365 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530007e40 con 0x7ff5340cd0a0 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff5297fa700 1 -- 172.21.15.131:0/2417598365 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff530007300 con 0x7ff5340cd0a0 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5340c9af0 con 0x7ff5340cd0a0 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.665+0000 7ff5297fa700 1 -- 172.21.15.131:0/2417598365 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5300076a0 con 0x7ff5340cd0a0 2023-12-26T23:13:01.670 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.666+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5340ca0b0 con 0x7ff5340cd0a0 2023-12-26T23:13:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.666+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5340717f0 con 0x7ff5340cd0a0 2023-12-26T23:13:01.671 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.667+0000 7ff5297fa700 1 -- 172.21.15.131:0/2417598365 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff530005a00 con 0x7ff5340cd0a0 2023-12-26T23:13:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.668+0000 7ff5297fa700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff518066f30 0x7ff5180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.668+0000 7ff5297fa700 1 -- 172.21.15.131:0/2417598365 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7ff530084480 con 0x7ff5340cd0a0 2023-12-26T23:13:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.668+0000 7ff5297fa700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7ff51806d560 0x7ff51806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.668+0000 7ff53899a700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff518066f30 0x7ff5180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.668+0000 7ff5297fa700 1 -- 172.21.15.131:0/2417598365 --> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff518070000 con 0x7ff51806d560 2023-12-26T23:13:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.668+0000 7ff5297fa700 1 -- 172.21.15.131:0/2417598365 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=215) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5300869e0 con 0x7ff5340cd0a0 2023-12-26T23:13:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.669+0000 7ff53999c700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7ff51806d560 0x7ff51806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.669+0000 7ff53899a700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff518066f30 0x7ff5180693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff5340cf790 tx=0x7ff524016000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:01.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.669+0000 7ff53999c700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7ff51806d560 0x7ff51806f9d0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:01.673 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.670+0000 7ff5297fa700 1 -- 172.21.15.131:0/2417598365 <== osd.0 v2:172.21.15.131:6808/39607 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff518070000 con 0x7ff51806d560 2023-12-26T23:13:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.684+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 --> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff5340cdd60 con 0x7ff51806d560 2023-12-26T23:13:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff5297fa700 1 -- 172.21.15.131:0/2417598365 <== osd.0 v2:172.21.15.131:6808/39607 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff5340cdd60 con 0x7ff51806d560 2023-12-26T23:13:01.686 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:13:01.686 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-26T23:13:01.686 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:13:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7ff51806d560 msgr2=0x7ff51806f9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff53b3ff700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6808/39607,v1:172.21.15.131:6809/39607] conn(0x7ff51806d560 0x7ff51806f9d0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff518066f30 msgr2=0x7ff5180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff53b3ff700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7ff518066f30 0x7ff5180693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff5340cf790 tx=0x7ff524016000 comp rx=0 tx=0).stop 2023-12-26T23:13:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5340cd0a0 msgr2=0x7ff534153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff53b3ff700 1 --2- 172.21.15.131:0/2417598365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5340cd0a0 0x7ff534153580 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7ff53000cb90 tx=0x7ff530002b60 comp rx=0 tx=0).stop 2023-12-26T23:13:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff53999c700 1 -- 172.21.15.131:0/2417598365 reap_dead start 2023-12-26T23:13:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 shutdown_connections 2023-12-26T23:13:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.685+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 >> 172.21.15.131:0/2417598365 conn(0x7ff534099950 msgr2=0x7ff534069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.686+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 shutdown_connections 2023-12-26T23:13:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.686+0000 7ff53b3ff700 1 -- 172.21.15.131:0/2417598365 wait complete. 2023-12-26T23:13:01.703 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-12-26T23:13:01.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.893+0000 7fbc557ff700 1 Processor -- start 2023-12-26T23:13:01.894 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.894+0000 7fbc557ff700 1 -- start start 2023-12-26T23:13:01.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc557ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbc50159a70 0x7fbc500b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc557ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbc500b2a90 0x7fbc500b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc557ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc500afcd0 con 0x7fbc501596a0 2023-12-26T23:13:01.897 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc557ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc500b3e70 con 0x7fbc50159a70 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc557ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc500b0f20 con 0x7fbc500b2a90 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc4e7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbc50159a70 0x7fbc500b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc4e7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbc50159a70 0x7fbc500b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36904/0 (socket says 172.21.15.131:36904) 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc4e7fc700 1 -- 172.21.15.131:0/1293523833 learned_addr learned my addr 172.21.15.131:0/1293523833 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc4f7fe700 1 --2- 172.21.15.131:0/1293523833 >> v2:172.21.15.190:3300/0 conn(0x7fbc500b2a90 0x7fbc500b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc4e7fc700 1 -- 172.21.15.131:0/1293523833 >> v2:172.21.15.190:3300/0 conn(0x7fbc500b2a90 msgr2=0x7fbc500b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc4e7fc700 1 --2- 172.21.15.131:0/1293523833 >> v2:172.21.15.190:3300/0 conn(0x7fbc500b2a90 0x7fbc500b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc4e7fc700 1 -- 172.21.15.131:0/1293523833 >> v1:172.21.15.190:6789/0 conn(0x7fbc501596a0 legacy=0x7fbc500b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.895+0000 7fbc4e7fc700 1 -- 172.21.15.131:0/1293523833 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc500b3540 con 0x7fbc50159a70 2023-12-26T23:13:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.896+0000 7fbc4e7fc700 1 --2- 172.21.15.131:0/1293523833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc50159a70 0x7fbc500b0160 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fbc500b66b0 tx=0x7fbc44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55bac0cf58dbdb0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.896+0000 7fbc4dffb700 1 -- 172.21.15.131:0/1293523833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44009e70 con 0x7fbc50159a70 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.896+0000 7fbc4dffb700 1 -- 172.21.15.131:0/1293523833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4400b860 con 0x7fbc50159a70 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.896+0000 7fbc4dffb700 1 -- 172.21.15.131:0/1293523833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400bc00 con 0x7fbc50159a70 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.896+0000 7fbc557ff700 1 -- 172.21.15.131:0/1293523833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc50159a70 msgr2=0x7fbc500b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.896+0000 7fbc557ff700 1 --2- 172.21.15.131:0/1293523833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc50159a70 0x7fbc500b0160 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fbc500b66b0 tx=0x7fbc44009250 comp rx=0 tx=0).stop 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.897+0000 7fbc557ff700 1 -- 172.21.15.131:0/1293523833 shutdown_connections 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.897+0000 7fbc557ff700 1 --2- 172.21.15.131:0/1293523833 >> v2:172.21.15.190:3300/0 conn(0x7fbc500b2a90 0x7fbc500b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.897+0000 7fbc557ff700 1 --2- 172.21.15.131:0/1293523833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc50159a70 0x7fbc500b0160 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.897+0000 7fbc557ff700 1 -- 172.21.15.131:0/1293523833 >> 172.21.15.131:0/1293523833 conn(0x7fbc5006ece0 msgr2=0x7fbc5006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.897+0000 7fbc557ff700 1 -- 172.21.15.131:0/1293523833 shutdown_connections 2023-12-26T23:13:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.897+0000 7fbc557ff700 1 -- 172.21.15.131:0/1293523833 wait complete. 2023-12-26T23:13:01.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.898+0000 7fbc557ff700 1 Processor -- start 2023-12-26T23:13:01.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.898+0000 7fbc557ff700 1 -- start start 2023-12-26T23:13:01.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.899+0000 7fbc557ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc500b2a90 0x7fbc5015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.902 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.899+0000 7fbc557ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc501596a0 0x7fbc500c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.899+0000 7fbc557ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc50159a70 0x7fbc500c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.899+0000 7fbc557ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500c6fd0 con 0x7fbc501596a0 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.899+0000 7fbc557ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5015ac10 con 0x7fbc500b2a90 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.899+0000 7fbc557ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5015afb0 con 0x7fbc50159a70 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.899+0000 7fbc4e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc501596a0 0x7fbc500c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.899+0000 7fbc4e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc501596a0 0x7fbc500c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36910/0 (socket says 172.21.15.131:36910) 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.899+0000 7fbc4e7fc700 1 -- 172.21.15.131:0/1767148715 learned_addr learned my addr 172.21.15.131:0/1767148715 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc4f7fe700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc50159a70 0x7fbc500c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc4effd700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc500b2a90 0x7fbc5015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc4e7fc700 1 -- 172.21.15.131:0/1767148715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc50159a70 msgr2=0x7fbc500c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc4e7fc700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc50159a70 0x7fbc500c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc4e7fc700 1 -- 172.21.15.131:0/1767148715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc500b2a90 msgr2=0x7fbc5015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc4e7fc700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc500b2a90 0x7fbc5015a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc4e7fc700 1 -- 172.21.15.131:0/1767148715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440094f0 con 0x7fbc501596a0 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc4effd700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc500b2a90 0x7fbc5015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc4e7fc700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc501596a0 0x7fbc500c5f50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fbc50089710 tx=0x7fbc44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc36ffd700 1 -- 172.21.15.131:0/1767148715 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44011420 con 0x7fbc501596a0 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc36ffd700 1 -- 172.21.15.131:0/1767148715 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc440115a0 con 0x7fbc501596a0 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc500c6490 con 0x7fbc501596a0 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc36ffd700 1 -- 172.21.15.131:0/1767148715 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44011960 con 0x7fbc501596a0 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.900+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc500c6a10 con 0x7fbc501596a0 2023-12-26T23:13:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.901+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5015b120 con 0x7fbc501596a0 2023-12-26T23:13:01.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.902+0000 7fbc36ffd700 1 -- 172.21.15.131:0/1767148715 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbc44011d50 con 0x7fbc501596a0 2023-12-26T23:13:01.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.903+0000 7fbc36ffd700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbc3c066f30 0x7fbc3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.903+0000 7fbc36ffd700 1 -- 172.21.15.131:0/1767148715 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fbc44085f90 con 0x7fbc501596a0 2023-12-26T23:13:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.903+0000 7fbc4effd700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbc3c066f30 0x7fbc3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.904+0000 7fbc4effd700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbc3c066f30 0x7fbc3c0693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbc40001840 tx=0x7fbc40009040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:01.908+0000 7fbc36ffd700 1 -- 172.21.15.131:0/1767148715 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc440b75a0 con 0x7fbc501596a0 2023-12-26T23:13:02.097 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.097+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc500657e0 con 0x7fbc3c066f30 2023-12-26T23:13:02.098 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.098+0000 7fbc36ffd700 1 -- 172.21.15.131:0/1767148715 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbc500657e0 con 0x7fbc3c066f30 2023-12-26T23:13:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbc3c066f30 msgr2=0x7fbc3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbc3c066f30 0x7fbc3c0693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbc40001840 tx=0x7fbc40009040 comp rx=0 tx=0).stop 2023-12-26T23:13:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc501596a0 msgr2=0x7fbc500c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc501596a0 0x7fbc500c5f50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fbc50089710 tx=0x7fbc44017000 comp rx=0 tx=0).stop 2023-12-26T23:13:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 shutdown_connections 2023-12-26T23:13:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fbc3c066f30 0x7fbc3c0693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc50159a70 0x7fbc500c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc500b2a90 0x7fbc5015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 --2- 172.21.15.131:0/1767148715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc501596a0 0x7fbc500c5f50 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 >> 172.21.15.131:0/1767148715 conn(0x7fbc5006ece0 msgr2=0x7fbc500cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:02.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 shutdown_connections 2023-12-26T23:13:02.102 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.100+0000 7fbc557ff700 1 -- 172.21.15.131:0/1767148715 wait complete. 2023-12-26T23:13:02.102 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 0 to deep-scrub 2023-12-26T23:13:02.118 INFO:tasks.ceph:Scrubbing osd.1 2023-12-26T23:13:02.119 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-12-26T23:13:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.311+0000 7fdd2bbff700 1 Processor -- start 2023-12-26T23:13:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.311+0000 7fdd2bbff700 1 -- start start 2023-12-26T23:13:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.312+0000 7fdd2bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd24142f00 0x7fdd24069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.312+0000 7fdd2bbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd240cc590 0x7fdd240cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.312+0000 7fdd2bbff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd24069640 con 0x7fdd240cb340 2023-12-26T23:13:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.312+0000 7fdd2bbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd240cb7c0 con 0x7fdd240cc590 2023-12-26T23:13:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.312+0000 7fdd2bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd24068750 con 0x7fdd24142f00 2023-12-26T23:13:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2999b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd24142f00 0x7fdd24069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2919a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd240cc590 0x7fdd240cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2a19c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fdd240cb340 0x7fdd240c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:45838/0 (socket says 172.21.15.131:45838) 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2999b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd24142f00 0x7fdd24069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.131:52756/0 (socket says 172.21.15.131:52756) 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2999b700 1 -- 172.21.15.131:0/3845106674 learned_addr learned my addr 172.21.15.131:0/3845106674 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2919a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd240cc590 0x7fdd240cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:60510/0 (socket says 172.21.15.131:60510) 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2999b700 1 -- 172.21.15.131:0/3845106674 >> v2:172.21.15.190:3300/0 conn(0x7fdd240cc590 msgr2=0x7fdd240cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2999b700 1 --2- 172.21.15.131:0/3845106674 >> v2:172.21.15.190:3300/0 conn(0x7fdd240cc590 0x7fdd240cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd28999700 1 -- 172.21.15.131:0/3845106674 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539989435 0 0) 0x7fdd24069640 con 0x7fdd240cb340 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2999b700 1 -- 172.21.15.131:0/3845106674 >> v1:172.21.15.190:6789/0 conn(0x7fdd240cb340 legacy=0x7fdd240c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2999b700 1 -- 172.21.15.131:0/3845106674 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd24068a30 con 0x7fdd24142f00 2023-12-26T23:13:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.313+0000 7fdd2919a700 1 --2- 172.21.15.131:0/3845106674 >> v2:172.21.15.190:3300/0 conn(0x7fdd240cc590 0x7fdd240cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:13:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd2999b700 1 --2- 172.21.15.131:0/3845106674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd24142f00 0x7fdd24069ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdd200099b0 tx=0x7fdd20009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5300d11d693fe6c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd28999700 1 -- 172.21.15.131:0/3845106674 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20009ea0 con 0x7fdd24142f00 2023-12-26T23:13:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd28999700 1 -- 172.21.15.131:0/3845106674 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2000b860 con 0x7fdd24142f00 2023-12-26T23:13:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd28999700 1 -- 172.21.15.131:0/3845106674 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000bc00 con 0x7fdd24142f00 2023-12-26T23:13:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd2bbff700 1 -- 172.21.15.131:0/3845106674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd24142f00 msgr2=0x7fdd24069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd2bbff700 1 --2- 172.21.15.131:0/3845106674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd24142f00 0x7fdd24069ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdd200099b0 tx=0x7fdd20009250 comp rx=0 tx=0).stop 2023-12-26T23:13:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd2bbff700 1 -- 172.21.15.131:0/3845106674 shutdown_connections 2023-12-26T23:13:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd2bbff700 1 --2- 172.21.15.131:0/3845106674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd24142f00 0x7fdd24069ad0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd2bbff700 1 --2- 172.21.15.131:0/3845106674 >> v2:172.21.15.190:3300/0 conn(0x7fdd240cc590 0x7fdd240cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.314+0000 7fdd2bbff700 1 -- 172.21.15.131:0/3845106674 >> 172.21.15.131:0/3845106674 conn(0x7fdd24063cd0 msgr2=0x7fdd2406b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.315+0000 7fdd2bbff700 1 -- 172.21.15.131:0/3845106674 shutdown_connections 2023-12-26T23:13:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.315+0000 7fdd2bbff700 1 -- 172.21.15.131:0/3845106674 wait complete. 2023-12-26T23:13:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.316+0000 7fdd2bbff700 1 Processor -- start 2023-12-26T23:13:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.316+0000 7fdd2bbff700 1 -- start start 2023-12-26T23:13:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.317+0000 7fdd2bbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd240cb340 0x7fdd24169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.317+0000 7fdd2bbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd240cc590 0x7fdd240b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.317+0000 7fdd2bbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24142f00 0x7fdd240b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.317+0000 7fdd2bbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240b3550 con 0x7fdd240cb340 2023-12-26T23:13:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.317+0000 7fdd2bbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240c60b0 con 0x7fdd24142f00 2023-12-26T23:13:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.317+0000 7fdd2bbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd24140330 con 0x7fdd240cc590 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.317+0000 7fdd2999b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd240cb340 0x7fdd24169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.317+0000 7fdd2999b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd240cb340 0x7fdd24169a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36918/0 (socket says 172.21.15.131:36918) 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.317+0000 7fdd2999b700 1 -- 172.21.15.131:0/568220832 learned_addr learned my addr 172.21.15.131:0/568220832 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2919a700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd240cc590 0x7fdd240b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2a19c700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24142f00 0x7fdd240b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2999b700 1 -- 172.21.15.131:0/568220832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd240cc590 msgr2=0x7fdd240b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2999b700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd240cc590 0x7fdd240b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2999b700 1 -- 172.21.15.131:0/568220832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24142f00 msgr2=0x7fdd240b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2999b700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd24142f00 0x7fdd240b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2999b700 1 -- 172.21.15.131:0/568220832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14009420 con 0x7fdd240cb340 2023-12-26T23:13:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2999b700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd240cb340 0x7fdd24169a70 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006fd0 tx=0x7fdd20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd19ffb700 1 -- 172.21.15.131:0/568220832 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000bbe0 con 0x7fdd240cb340 2023-12-26T23:13:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd19ffb700 1 -- 172.21.15.131:0/568220832 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2001a450 con 0x7fdd240cb340 2023-12-26T23:13:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd19ffb700 1 -- 172.21.15.131:0/568220832 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2001a7f0 con 0x7fdd240cb340 2023-12-26T23:13:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd200094f0 con 0x7fdd240cb340 2023-12-26T23:13:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.318+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd240c6630 con 0x7fdd240cb340 2023-12-26T23:13:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.319+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd240717f0 con 0x7fdd240cb340 2023-12-26T23:13:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.320+0000 7fdd19ffb700 1 -- 172.21.15.131:0/568220832 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdd2001ab60 con 0x7fdd240cb340 2023-12-26T23:13:02.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.321+0000 7fdd19ffb700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd08066f30 0x7fdd080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.324 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.321+0000 7fdd19ffb700 1 -- 172.21.15.131:0/568220832 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fdd20086060 con 0x7fdd240cb340 2023-12-26T23:13:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.321+0000 7fdd19ffb700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7fdd0806d6d0 0x7fdd0806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.321+0000 7fdd2919a700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd08066f30 0x7fdd080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.321+0000 7fdd19ffb700 1 -- 172.21.15.131:0/568220832 --> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd08070170 con 0x7fdd0806d6d0 2023-12-26T23:13:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.321+0000 7fdd19ffb700 1 -- 172.21.15.131:0/568220832 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=215) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd20088560 con 0x7fdd240cb340 2023-12-26T23:13:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.322+0000 7fdd2a19c700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7fdd0806d6d0 0x7fdd0806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.322+0000 7fdd2919a700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd08066f30 0x7fdd080693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdd240c57d0 tx=0x7fdd14008040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.322+0000 7fdd2a19c700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7fdd0806d6d0 0x7fdd0806fb40 crc :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.323+0000 7fdd19ffb700 1 -- 172.21.15.131:0/568220832 <== osd.1 v2:172.21.15.131:6816/39610 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdd08070170 con 0x7fdd0806d6d0 2023-12-26T23:13:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.337+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 --> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdd24068d10 con 0x7fdd0806d6d0 2023-12-26T23:13:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.337+0000 7fdd19ffb700 1 -- 172.21.15.131:0/568220832 <== osd.1 v2:172.21.15.131:6816/39610 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdd24068d10 con 0x7fdd0806d6d0 2023-12-26T23:13:02.338 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:13:02.338 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-26T23:13:02.338 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:13:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.337+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7fdd0806d6d0 msgr2=0x7fdd0806fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.337+0000 7fdd2bbff700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6816/39610,v1:172.21.15.131:6818/39610] conn(0x7fdd0806d6d0 0x7fdd0806fb40 crc :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.338+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd08066f30 msgr2=0x7fdd080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.338+0000 7fdd2bbff700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fdd08066f30 0x7fdd080693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdd240c57d0 tx=0x7fdd14008040 comp rx=0 tx=0).stop 2023-12-26T23:13:02.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.338+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd240cb340 msgr2=0x7fdd24169a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.338+0000 7fdd2bbff700 1 --2- 172.21.15.131:0/568220832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd240cb340 0x7fdd24169a70 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006fd0 tx=0x7fdd20017000 comp rx=0 tx=0).stop 2023-12-26T23:13:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.338+0000 7fdd2a19c700 1 -- 172.21.15.131:0/568220832 reap_dead start 2023-12-26T23:13:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.338+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 shutdown_connections 2023-12-26T23:13:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.338+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 >> 172.21.15.131:0/568220832 conn(0x7fdd24063cd0 msgr2=0x7fdd240c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.338+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 shutdown_connections 2023-12-26T23:13:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.338+0000 7fdd2bbff700 1 -- 172.21.15.131:0/568220832 wait complete. 2023-12-26T23:13:02.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-12-26T23:13:02.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.545+0000 7f9acadff700 1 Processor -- start 2023-12-26T23:13:02.546 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.545+0000 7f9acadff700 1 -- start start 2023-12-26T23:13:02.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.546+0000 7f9acadff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9ac41596a0 0x7f9ac40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.546+0000 7f9acadff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac4159a70 0x7f9ac40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.546+0000 7f9acadff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9ac40b0160 0x7f9ac40b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.546+0000 7f9acadff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac40afcd0 con 0x7f9ac40b0160 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.546+0000 7f9acadff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac40b3e70 con 0x7f9ac41596a0 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.546+0000 7f9acadff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40b2e00 con 0x7f9ac4159a70 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac939c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9ac40b0160 0x7f9ac40b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac939c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9ac40b0160 0x7f9ac40b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36932/0 (socket says 172.21.15.131:36932) 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac939c700 1 -- 172.21.15.131:0/766884618 learned_addr learned my addr 172.21.15.131:0/766884618 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac8b9b700 1 --2- 172.21.15.131:0/766884618 >> v2:172.21.15.190:3300/0 conn(0x7f9ac41596a0 0x7f9ac40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac3fff700 1 --2- 172.21.15.131:0/766884618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac4159a70 0x7f9ac40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac939c700 1 -- 172.21.15.131:0/766884618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac4159a70 msgr2=0x7f9ac40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac939c700 1 --2- 172.21.15.131:0/766884618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac4159a70 0x7f9ac40b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac939c700 1 -- 172.21.15.131:0/766884618 >> v2:172.21.15.190:3300/0 conn(0x7f9ac41596a0 msgr2=0x7f9ac40b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac939c700 1 --2- 172.21.15.131:0/766884618 >> v2:172.21.15.190:3300/0 conn(0x7f9ac41596a0 0x7f9ac40b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.547+0000 7f9ac939c700 1 -- 172.21.15.131:0/766884618 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac40b0f20 con 0x7f9ac40b0160 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9ac939c700 1 --2- 172.21.15.131:0/766884618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac40b0160 0x7f9ac40b05c0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40892a0 tx=0x7f9abc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bd4b12a8258317 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9ac37fe700 1 -- 172.21.15.131:0/766884618 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00e070 con 0x7f9ac40b0160 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9ac37fe700 1 -- 172.21.15.131:0/766884618 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc00b970 con 0x7f9ac40b0160 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9ac37fe700 1 -- 172.21.15.131:0/766884618 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0092e0 con 0x7f9ac40b0160 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9acadff700 1 -- 172.21.15.131:0/766884618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac40b0160 msgr2=0x7f9ac40b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9acadff700 1 --2- 172.21.15.131:0/766884618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac40b0160 0x7f9ac40b05c0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40892a0 tx=0x7f9abc009840 comp rx=0 tx=0).stop 2023-12-26T23:13:02.551 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9acadff700 1 -- 172.21.15.131:0/766884618 shutdown_connections 2023-12-26T23:13:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9acadff700 1 --2- 172.21.15.131:0/766884618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac40b0160 0x7f9ac40b05c0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9acadff700 1 --2- 172.21.15.131:0/766884618 >> v2:172.21.15.190:3300/0 conn(0x7f9ac41596a0 0x7f9ac40b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9acadff700 1 --2- 172.21.15.131:0/766884618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac4159a70 0x7f9ac40b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9acadff700 1 -- 172.21.15.131:0/766884618 >> 172.21.15.131:0/766884618 conn(0x7f9ac406ece0 msgr2=0x7f9ac406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.548+0000 7f9acadff700 1 -- 172.21.15.131:0/766884618 shutdown_connections 2023-12-26T23:13:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.549+0000 7f9acadff700 1 -- 172.21.15.131:0/766884618 wait complete. 2023-12-26T23:13:02.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.550+0000 7f9acadff700 1 Processor -- start 2023-12-26T23:13:02.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.550+0000 7f9acadff700 1 -- start start 2023-12-26T23:13:02.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.551+0000 7f9acadff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac40b0160 0x7f9ac40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.551+0000 7f9acadff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ac41596a0 0x7f9ac4158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.551+0000 7f9acadff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac4159a70 0x7f9ac40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.551+0000 7f9acadff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40ceff0 con 0x7f9ac4159a70 2023-12-26T23:13:02.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.551+0000 7f9acadff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40c8430 con 0x7f9ac41596a0 2023-12-26T23:13:02.555 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.551+0000 7f9acadff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40cfab0 con 0x7f9ac40b0160 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.551+0000 7f9ac939c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac4159a70 0x7f9ac40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.551+0000 7f9ac939c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac4159a70 0x7f9ac40c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36946/0 (socket says 172.21.15.131:36946) 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.551+0000 7f9ac939c700 1 -- 172.21.15.131:0/38814327 learned_addr learned my addr 172.21.15.131:0/38814327 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac8b9b700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac40b0160 0x7f9ac40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac3fff700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ac41596a0 0x7f9ac4158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac939c700 1 -- 172.21.15.131:0/38814327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac40b0160 msgr2=0x7f9ac40ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac939c700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac40b0160 0x7f9ac40ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac939c700 1 -- 172.21.15.131:0/38814327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ac41596a0 msgr2=0x7f9ac4158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac939c700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ac41596a0 0x7f9ac4158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.556 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac939c700 1 -- 172.21.15.131:0/38814327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc009ae0 con 0x7f9ac4159a70 2023-12-26T23:13:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac939c700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac4159a70 0x7f9ac40c75e0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4089710 tx=0x7f9abc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac0ff9700 1 -- 172.21.15.131:0/38814327 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc008ad0 con 0x7f9ac4159a70 2023-12-26T23:13:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac0ff9700 1 -- 172.21.15.131:0/38814327 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc008c30 con 0x7f9ac4159a70 2023-12-26T23:13:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac40c4b90 con 0x7f9ac4159a70 2023-12-26T23:13:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.552+0000 7f9ac0ff9700 1 -- 172.21.15.131:0/38814327 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc026460 con 0x7f9ac4159a70 2023-12-26T23:13:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.553+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac40cf4f0 con 0x7f9ac4159a70 2023-12-26T23:13:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.553+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac40657e0 con 0x7f9ac4159a70 2023-12-26T23:13:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.554+0000 7f9ac0ff9700 1 -- 172.21.15.131:0/38814327 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9abc003040 con 0x7f9ac4159a70 2023-12-26T23:13:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.555+0000 7f9ac0ff9700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9aa8066f30 0x7f9aa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.555+0000 7f9ac0ff9700 1 -- 172.21.15.131:0/38814327 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f9abc086520 con 0x7f9ac4159a70 2023-12-26T23:13:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.555+0000 7f9ac8b9b700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9aa8066f30 0x7f9aa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.556+0000 7f9ac8b9b700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9aa8066f30 0x7f9aa80693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009db0 tx=0x7f9ab80091a0 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.560 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.560+0000 7f9ac0ff9700 1 -- 172.21.15.131:0/38814327 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9abc0535b0 con 0x7f9ac4159a70 2023-12-26T23:13:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.753+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac40b3540 con 0x7f9aa8066f30 2023-12-26T23:13:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.754+0000 7f9ac0ff9700 1 -- 172.21.15.131:0/38814327 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9ac40b3540 con 0x7f9aa8066f30 2023-12-26T23:13:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9aa8066f30 msgr2=0x7f9aa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9aa8066f30 0x7f9aa80693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009db0 tx=0x7f9ab80091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac4159a70 msgr2=0x7f9ac40c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac4159a70 0x7f9ac40c75e0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4089710 tx=0x7f9abc018000 comp rx=0 tx=0).stop 2023-12-26T23:13:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 shutdown_connections 2023-12-26T23:13:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f9aa8066f30 0x7f9aa80693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ac4159a70 0x7f9ac40c75e0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ac40b0160 0x7f9ac40ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 --2- 172.21.15.131:0/38814327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ac41596a0 0x7f9ac4158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 >> 172.21.15.131:0/38814327 conn(0x7f9ac406ece0 msgr2=0x7f9ac40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 shutdown_connections 2023-12-26T23:13:02.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.756+0000 7f9acadff700 1 -- 172.21.15.131:0/38814327 wait complete. 2023-12-26T23:13:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 1 to deep-scrub 2023-12-26T23:13:02.774 INFO:tasks.ceph:Scrubbing osd.2 2023-12-26T23:13:02.774 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-12-26T23:13:02.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.969+0000 7f3da7dff700 1 Processor -- start 2023-12-26T23:13:02.969 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.969+0000 7f3da7dff700 1 -- start start 2023-12-26T23:13:02.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.970+0000 7f3da7dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3da0156360 0x7f3da006e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.970+0000 7f3da7dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156730 0x7f3da006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.970+0000 7f3da7dff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da00acfc0 con 0x7f3da00c8490 2023-12-26T23:13:02.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.970+0000 7f3da7dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da006f1f0 con 0x7f3da0156360 2023-12-26T23:13:02.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.970+0000 7f3da7dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00c91f0 con 0x7f3da0156730 2023-12-26T23:13:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.971+0000 7f3da539a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156730 0x7f3da006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.971+0000 7f3da5b9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3da0156360 0x7f3da006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.971+0000 7f3da639c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f3da00c8490 0x7f3da00c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.131:45846/0 (socket says 172.21.15.131:45846) 2023-12-26T23:13:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.971+0000 7f3da639c700 1 -- 172.21.15.131:0/1929944097 learned_addr learned my addr 172.21.15.131:0/1929944097 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.971+0000 7f3da539a700 1 -- 172.21.15.131:0/1929944097 >> v2:172.21.15.190:3300/0 conn(0x7f3da0156360 msgr2=0x7f3da006e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.971+0000 7f3da539a700 1 --2- 172.21.15.131:0/1929944097 >> v2:172.21.15.190:3300/0 conn(0x7f3da0156360 0x7f3da006e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.971+0000 7f3da539a700 1 -- 172.21.15.131:0/1929944097 >> v1:172.21.15.190:6789/0 conn(0x7f3da00c8490 legacy=0x7f3da00c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.971+0000 7f3da539a700 1 -- 172.21.15.131:0/1929944097 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00c94d0 con 0x7f3da0156730 2023-12-26T23:13:02.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.971+0000 7f3da539a700 1 --2- 172.21.15.131:0/1929944097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156730 0x7f3da006ecb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d900098e0 tx=0x7f3d900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ada521b5b32d3a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da4b99700 1 -- 172.21.15.131:0/1929944097 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000b640 con 0x7f3da0156730 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da4b99700 1 -- 172.21.15.131:0/1929944097 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9000b7a0 con 0x7f3da0156730 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da4b99700 1 -- 172.21.15.131:0/1929944097 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000bb40 con 0x7f3da0156730 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da7dff700 1 -- 172.21.15.131:0/1929944097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156730 msgr2=0x7f3da006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da7dff700 1 --2- 172.21.15.131:0/1929944097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156730 0x7f3da006ecb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d900098e0 tx=0x7f3d900091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da7dff700 1 -- 172.21.15.131:0/1929944097 shutdown_connections 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da7dff700 1 --2- 172.21.15.131:0/1929944097 >> v2:172.21.15.190:3300/0 conn(0x7f3da0156360 0x7f3da006e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da7dff700 1 --2- 172.21.15.131:0/1929944097 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156730 0x7f3da006ecb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da7dff700 1 -- 172.21.15.131:0/1929944097 >> 172.21.15.131:0/1929944097 conn(0x7f3da00675b0 msgr2=0x7f3da00679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da7dff700 1 -- 172.21.15.131:0/1929944097 shutdown_connections 2023-12-26T23:13:02.974 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.972+0000 7f3da7dff700 1 -- 172.21.15.131:0/1929944097 wait complete. 2023-12-26T23:13:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.974+0000 7f3da7dff700 1 Processor -- start 2023-12-26T23:13:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.974+0000 7f3da7dff700 1 -- start start 2023-12-26T23:13:02.977 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da7dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c8490 0x7f3da015dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da7dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156360 0x7f3da006d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da7dff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3da0156730 0x7f3da0150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da7dff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da015e480 con 0x7f3da0156730 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da7dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da015e760 con 0x7f3da00c8490 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da7dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da015ea40 con 0x7f3da0156360 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da639c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3da0156730 0x7f3da0150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da539a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156360 0x7f3da006d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da5b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c8490 0x7f3da015dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da5b9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c8490 0x7f3da015dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.131:60562/0 (socket says 172.21.15.131:60562) 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.975+0000 7f3da5b9b700 1 -- 172.21.15.131:0/1976606046 learned_addr learned my addr 172.21.15.131:0/1976606046 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:02.978 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3da5b9b700 1 -- 172.21.15.131:0/1976606046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156360 msgr2=0x7f3da006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3da5b9b700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156360 0x7f3da006d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3da5b9b700 1 -- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3da0156730 msgr2=0x7f3da0150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3da5b9b700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3da0156730 0x7f3da0150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3da5b9b700 1 -- 172.21.15.131:0/1976606046 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d90009420 con 0x7f3da00c8490 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3da639c700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3da0156730 0x7f3da0150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3da539a700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0156360 0x7f3da006d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3da5b9b700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c8490 0x7f3da015dc50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00ede0 tx=0x7f3d9c00b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3d95ffb700 1 -- 172.21.15.131:0/1976606046 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00eee0 con 0x7f3da00c8490 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3d95ffb700 1 -- 172.21.15.131:0/1976606046 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c00ddc0 con 0x7f3da00c8490 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da015ed20 con 0x7f3da00c8490 2023-12-26T23:13:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.976+0000 7f3d95ffb700 1 -- 172.21.15.131:0/1976606046 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c010650 con 0x7f3da00c8490 2023-12-26T23:13:02.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.977+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da006dc00 con 0x7f3da00c8490 2023-12-26T23:13:02.980 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.977+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3da00c31f0 con 0x7f3da00c8490 2023-12-26T23:13:02.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.979+0000 7f3d95ffb700 1 -- 172.21.15.131:0/1976606046 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3d9c005a00 con 0x7f3da00c8490 2023-12-26T23:13:02.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.979+0000 7f3d95ffb700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3d84066f30 0x7f3d840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.981 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.979+0000 7f3d95ffb700 1 -- 172.21.15.131:0/1976606046 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f3d9c014030 con 0x7f3da00c8490 2023-12-26T23:13:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.980+0000 7f3da539a700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3d84066f30 0x7f3d840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.980+0000 7f3d95ffb700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f3d8406d6d0 0x7f3d8406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.980+0000 7f3d95ffb700 1 -- 172.21.15.131:0/1976606046 --> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d84070170 con 0x7f3d8406d6d0 2023-12-26T23:13:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.980+0000 7f3d95ffb700 1 -- 172.21.15.131:0/1976606046 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=215) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d9c035ea0 con 0x7f3da00c8490 2023-12-26T23:13:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.980+0000 7f3da639c700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f3d8406d6d0 0x7f3d8406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.980+0000 7f3da539a700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3d84066f30 0x7f3d840693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d900093f0 tx=0x7f3d90017000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.980+0000 7f3da639c700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f3d8406d6d0 0x7f3d8406fb40 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.981+0000 7f3d95ffb700 1 -- 172.21.15.131:0/1976606046 <== osd.2 v2:172.21.15.131:6800/39608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3d84070170 con 0x7f3d8406d6d0 2023-12-26T23:13:02.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.995+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 --> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3da015e8a0 con 0x7f3d8406d6d0 2023-12-26T23:13:02.996 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3d95ffb700 1 -- 172.21.15.131:0/1976606046 <== osd.2 v2:172.21.15.131:6800/39608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3da015e8a0 con 0x7f3d8406d6d0 2023-12-26T23:13:02.996 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:13:02.996 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-26T23:13:02.996 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:13:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f3d8406d6d0 msgr2=0x7f3d8406fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3da7dff700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6800/39608,v1:172.21.15.131:6801/39608] conn(0x7f3d8406d6d0 0x7f3d8406fb40 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:02.997 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3d84066f30 msgr2=0x7f3d840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3da7dff700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3d84066f30 0x7f3d840693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d900093f0 tx=0x7f3d90017000 comp rx=0 tx=0).stop 2023-12-26T23:13:02.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c8490 msgr2=0x7f3da015dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:02.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3da7dff700 1 --2- 172.21.15.131:0/1976606046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c8490 0x7f3da015dc50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00ede0 tx=0x7f3d9c00b430 comp rx=0 tx=0).stop 2023-12-26T23:13:02.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3da639c700 1 -- 172.21.15.131:0/1976606046 reap_dead start 2023-12-26T23:13:02.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 shutdown_connections 2023-12-26T23:13:02.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.996+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 >> 172.21.15.131:0/1976606046 conn(0x7f3da00675b0 msgr2=0x7f3da01515f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:02.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.997+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 shutdown_connections 2023-12-26T23:13:02.998 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:02.997+0000 7f3da7dff700 1 -- 172.21.15.131:0/1976606046 wait complete. 2023-12-26T23:13:03.013 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-12-26T23:13:03.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.204+0000 7fde217ff700 1 Processor -- start 2023-12-26T23:13:03.205 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.204+0000 7fde217ff700 1 -- start start 2023-12-26T23:13:03.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde217ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c159a70 0x7fde1c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde217ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fde1c0b2a90 0x7fde1c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde217ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1c0afcd0 con 0x7fde1c1596a0 2023-12-26T23:13:03.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde217ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c0b3e70 con 0x7fde1c0b2a90 2023-12-26T23:13:03.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde217ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b0f20 con 0x7fde1c159a70 2023-12-26T23:13:03.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde1affd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fde1c1596a0 0x7fde1c0b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43986/0 (socket says 172.21.15.131:43986) 2023-12-26T23:13:03.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde1affd700 1 -- 172.21.15.131:0/1978967582 learned_addr learned my addr 172.21.15.131:0/1978967582 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:03.208 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde1b7fe700 1 --2- 172.21.15.131:0/1978967582 >> v2:172.21.15.131:3300/0 conn(0x7fde1c0b2a90 0x7fde1c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde1a7fc700 1 --2- 172.21.15.131:0/1978967582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c159a70 0x7fde1c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde19ffb700 1 -- 172.21.15.131:0/1978967582 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014566773 0 0) 0x7fde1c0afcd0 con 0x7fde1c1596a0 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde19ffb700 1 -- 172.21.15.131:0/1978967582 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde04003570 con 0x7fde1c1596a0 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde1b7fe700 1 -- 172.21.15.131:0/1978967582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c159a70 msgr2=0x7fde1c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde1b7fe700 1 --2- 172.21.15.131:0/1978967582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c159a70 0x7fde1c0b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde1b7fe700 1 -- 172.21.15.131:0/1978967582 >> v1:172.21.15.131:6789/0 conn(0x7fde1c1596a0 legacy=0x7fde1c0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.206+0000 7fde1b7fe700 1 -- 172.21.15.131:0/1978967582 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0b3540 con 0x7fde1c0b2a90 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.207+0000 7fde1b7fe700 1 --2- 172.21.15.131:0/1978967582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c0b2a90 0x7fde1c0b06a0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e60 tx=0x7fde14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac1d0d569e70f18 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.207+0000 7fde19ffb700 1 -- 172.21.15.131:0/1978967582 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1400e070 con 0x7fde1c0b2a90 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.207+0000 7fde19ffb700 1 -- 172.21.15.131:0/1978967582 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1400bab0 con 0x7fde1c0b2a90 2023-12-26T23:13:03.209 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.207+0000 7fde19ffb700 1 -- 172.21.15.131:0/1978967582 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1400be70 con 0x7fde1c0b2a90 2023-12-26T23:13:03.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.207+0000 7fde217ff700 1 -- 172.21.15.131:0/1978967582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c0b2a90 msgr2=0x7fde1c0b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.207+0000 7fde217ff700 1 --2- 172.21.15.131:0/1978967582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c0b2a90 0x7fde1c0b06a0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e60 tx=0x7fde14009780 comp rx=0 tx=0).stop 2023-12-26T23:13:03.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.208+0000 7fde217ff700 1 -- 172.21.15.131:0/1978967582 shutdown_connections 2023-12-26T23:13:03.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.208+0000 7fde217ff700 1 --2- 172.21.15.131:0/1978967582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c0b2a90 0x7fde1c0b06a0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.208+0000 7fde217ff700 1 --2- 172.21.15.131:0/1978967582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c159a70 0x7fde1c0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.208+0000 7fde217ff700 1 -- 172.21.15.131:0/1978967582 >> 172.21.15.131:0/1978967582 conn(0x7fde1c06ece0 msgr2=0x7fde1c06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:03.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.208+0000 7fde217ff700 1 -- 172.21.15.131:0/1978967582 shutdown_connections 2023-12-26T23:13:03.210 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.208+0000 7fde217ff700 1 -- 172.21.15.131:0/1978967582 wait complete. 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.209+0000 7fde217ff700 1 Processor -- start 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.209+0000 7fde217ff700 1 -- start start 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.210+0000 7fde217ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde1c0b2a90 0x7fde1c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.210+0000 7fde217ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c1596a0 0x7fde1c1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.210+0000 7fde217ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c159a70 0x7fde1c0cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.210+0000 7fde217ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c15d830 con 0x7fde1c159a70 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.210+0000 7fde217ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c155290 con 0x7fde1c0b2a90 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.210+0000 7fde217ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c159e40 con 0x7fde1c1596a0 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.210+0000 7fde1b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c159a70 0x7fde1c0cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c159a70 0x7fde1c0cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36958/0 (socket says 172.21.15.131:36958) 2023-12-26T23:13:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1b7fe700 1 -- 172.21.15.131:0/1262219271 learned_addr learned my addr 172.21.15.131:0/1262219271 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1affd700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde1c0b2a90 0x7fde1c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1a7fc700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c1596a0 0x7fde1c1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1b7fe700 1 -- 172.21.15.131:0/1262219271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c1596a0 msgr2=0x7fde1c1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1b7fe700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c1596a0 0x7fde1c1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1b7fe700 1 -- 172.21.15.131:0/1262219271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde1c0b2a90 msgr2=0x7fde1c15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1b7fe700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde1c0b2a90 0x7fde1c15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1b7fe700 1 -- 172.21.15.131:0/1262219271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde14009a20 con 0x7fde1c159a70 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde1b7fe700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c159a70 0x7fde1c0cc700 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fde1c089710 tx=0x7fde14019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde02ffd700 1 -- 172.21.15.131:0/1262219271 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14012e50 con 0x7fde1c159a70 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde02ffd700 1 -- 172.21.15.131:0/1262219271 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde14027470 con 0x7fde1c159a70 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde02ffd700 1 -- 172.21.15.131:0/1262219271 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14027830 con 0x7fde1c159a70 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.211+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c154710 con 0x7fde1c159a70 2023-12-26T23:13:03.214 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.212+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c15a330 con 0x7fde1c159a70 2023-12-26T23:13:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.213+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c0657e0 con 0x7fde1c159a70 2023-12-26T23:13:03.215 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.213+0000 7fde02ffd700 1 -- 172.21.15.131:0/1262219271 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fde14024070 con 0x7fde1c159a70 2023-12-26T23:13:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.214+0000 7fde02ffd700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fde08066f60 0x7fde08069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.214+0000 7fde02ffd700 1 -- 172.21.15.131:0/1262219271 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fde14086d20 con 0x7fde1c159a70 2023-12-26T23:13:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.215+0000 7fde1affd700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fde08066f60 0x7fde08069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.215+0000 7fde1affd700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fde08066f60 0x7fde08069410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fde0c007fd0 tx=0x7fde0c009040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.218 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.218+0000 7fde02ffd700 1 -- 172.21.15.131:0/1262219271 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde14053db0 con 0x7fde1c159a70 2023-12-26T23:13:03.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.412+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fde1c0cac70 con 0x7fde08066f60 2023-12-26T23:13:03.413 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.413+0000 7fde02ffd700 1 -- 172.21.15.131:0/1262219271 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fde1c0cac70 con 0x7fde08066f60 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fde08066f60 msgr2=0x7fde08069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fde08066f60 0x7fde08069410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fde0c007fd0 tx=0x7fde0c009040 comp rx=0 tx=0).stop 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c159a70 msgr2=0x7fde1c0cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c159a70 0x7fde1c0cc700 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fde1c089710 tx=0x7fde14019000 comp rx=0 tx=0).stop 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 shutdown_connections 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fde08066f60 0x7fde08069410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde1c159a70 0x7fde1c0cc700 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde1c0b2a90 0x7fde1c15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 --2- 172.21.15.131:0/1262219271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde1c1596a0 0x7fde1c1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.415+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 >> 172.21.15.131:0/1262219271 conn(0x7fde1c06ece0 msgr2=0x7fde1c157e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.416+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 shutdown_connections 2023-12-26T23:13:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.416+0000 7fde217ff700 1 -- 172.21.15.131:0/1262219271 wait complete. 2023-12-26T23:13:03.418 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 2 to deep-scrub 2023-12-26T23:13:03.433 INFO:tasks.ceph:Scrubbing osd.3 2023-12-26T23:13:03.433 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-12-26T23:13:03.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.628+0000 7fd5eabff700 1 Processor -- start 2023-12-26T23:13:03.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.628+0000 7fd5eabff700 1 -- start start 2023-12-26T23:13:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5eabff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5e41572b0 0x7fd5e40ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5eabff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5e4157680 0x7fd5e40c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5eabff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e40ae180 con 0x7fd5e40c9120 2023-12-26T23:13:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5eabff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e40c9540 con 0x7fd5e41572b0 2023-12-26T23:13:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5eabff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e40ad890 con 0x7fd5e4157680 2023-12-26T23:13:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5e899b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd5e41572b0 0x7fd5e40ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5e919c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd5e40c9120 0x7fd5e415b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43992/0 (socket says 172.21.15.131:43992) 2023-12-26T23:13:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5e919c700 1 -- 172.21.15.131:0/4089595507 learned_addr learned my addr 172.21.15.131:0/4089595507 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5e3fff700 1 --2- 172.21.15.131:0/4089595507 >> v2:172.21.15.190:3300/0 conn(0x7fd5e4157680 0x7fd5e40c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5e899b700 1 -- 172.21.15.131:0/4089595507 >> v2:172.21.15.190:3300/0 conn(0x7fd5e4157680 msgr2=0x7fd5e40c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5e899b700 1 --2- 172.21.15.131:0/4089595507 >> v2:172.21.15.190:3300/0 conn(0x7fd5e4157680 0x7fd5e40c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5e899b700 1 -- 172.21.15.131:0/4089595507 >> v1:172.21.15.131:6789/0 conn(0x7fd5e40c9120 legacy=0x7fd5e415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.630+0000 7fd5e899b700 1 -- 172.21.15.131:0/4089595507 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e415bde0 con 0x7fd5e41572b0 2023-12-26T23:13:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.631+0000 7fd5e899b700 1 --2- 172.21.15.131:0/4089595507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 0x7fd5e40ae610 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4069440 tx=0x7fd5d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28761a202186313 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.631+0000 7fd5e37fe700 1 -- 172.21.15.131:0/4089595507 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800b6b0 con 0x7fd5e41572b0 2023-12-26T23:13:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.631+0000 7fd5e37fe700 1 -- 172.21.15.131:0/4089595507 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d800b810 con 0x7fd5e41572b0 2023-12-26T23:13:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.631+0000 7fd5e37fe700 1 -- 172.21.15.131:0/4089595507 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800bbb0 con 0x7fd5e41572b0 2023-12-26T23:13:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.631+0000 7fd5eabff700 1 -- 172.21.15.131:0/4089595507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 msgr2=0x7fd5e40ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.631+0000 7fd5eabff700 1 --2- 172.21.15.131:0/4089595507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 0x7fd5e40ae610 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4069440 tx=0x7fd5d80091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.632+0000 7fd5eabff700 1 -- 172.21.15.131:0/4089595507 shutdown_connections 2023-12-26T23:13:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.632+0000 7fd5eabff700 1 --2- 172.21.15.131:0/4089595507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 0x7fd5e40ae610 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.632+0000 7fd5eabff700 1 --2- 172.21.15.131:0/4089595507 >> v2:172.21.15.190:3300/0 conn(0x7fd5e4157680 0x7fd5e40c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.632+0000 7fd5eabff700 1 -- 172.21.15.131:0/4089595507 >> 172.21.15.131:0/4089595507 conn(0x7fd5e40675b0 msgr2=0x7fd5e40679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.632+0000 7fd5eabff700 1 -- 172.21.15.131:0/4089595507 shutdown_connections 2023-12-26T23:13:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.632+0000 7fd5eabff700 1 -- 172.21.15.131:0/4089595507 wait complete. 2023-12-26T23:13:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.634+0000 7fd5eabff700 1 Processor -- start 2023-12-26T23:13:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.634+0000 7fd5eabff700 1 -- start start 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.635+0000 7fd5eabff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40c9120 0x7fd5e4151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.635+0000 7fd5eabff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 0x7fd5e4169ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.635+0000 7fd5eabff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e4157680 0x7fd5e40c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.635+0000 7fd5eabff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e4151b80 con 0x7fd5e41572b0 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.635+0000 7fd5eabff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e416ab00 con 0x7fd5e4157680 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.635+0000 7fd5eabff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e416a540 con 0x7fd5e40c9120 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.635+0000 7fd5e3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 0x7fd5e4169ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.635+0000 7fd5e3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 0x7fd5e4169ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36982/0 (socket says 172.21.15.131:36982) 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.635+0000 7fd5e3fff700 1 -- 172.21.15.131:0/912375630 learned_addr learned my addr 172.21.15.131:0/912375630 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e919c700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e4157680 0x7fd5e40c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e899b700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40c9120 0x7fd5e4151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e3fff700 1 -- 172.21.15.131:0/912375630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40c9120 msgr2=0x7fd5e4151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e3fff700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40c9120 0x7fd5e4151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e3fff700 1 -- 172.21.15.131:0/912375630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e4157680 msgr2=0x7fd5e40c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e3fff700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e4157680 0x7fd5e40c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e3fff700 1 -- 172.21.15.131:0/912375630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d8009420 con 0x7fd5e41572b0 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e919c700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e4157680 0x7fd5e40c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e3fff700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 0x7fd5e4169ab0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000d860 tx=0x7fd5d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e0ff9700 1 -- 172.21.15.131:0/912375630 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00125f0 con 0x7fd5e41572b0 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e0ff9700 1 -- 172.21.15.131:0/912375630 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d0012750 con 0x7fd5e41572b0 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e416a820 con 0x7fd5e41572b0 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5e0ff9700 1 -- 172.21.15.131:0/912375630 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0012b10 con 0x7fd5e41572b0 2023-12-26T23:13:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.636+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e40a9d40 con 0x7fd5e41572b0 2023-12-26T23:13:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.637+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5e40a98c0 con 0x7fd5e41572b0 2023-12-26T23:13:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.638+0000 7fd5e0ff9700 1 -- 172.21.15.131:0/912375630 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd5d001e460 con 0x7fd5e41572b0 2023-12-26T23:13:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.639+0000 7fd5e0ff9700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5c8066f30 0x7fd5c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.639+0000 7fd5e0ff9700 1 -- 172.21.15.131:0/912375630 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fd5d0084cd0 con 0x7fd5e41572b0 2023-12-26T23:13:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.639+0000 7fd5e899b700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5c8066f30 0x7fd5c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.639+0000 7fd5e0ff9700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7fd5c806d6d0 0x7fd5c806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.639+0000 7fd5e0ff9700 1 -- 172.21.15.131:0/912375630 --> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5c8070170 con 0x7fd5c806d6d0 2023-12-26T23:13:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.639+0000 7fd5e0ff9700 1 -- 172.21.15.131:0/912375630 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=215) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d008a220 con 0x7fd5e41572b0 2023-12-26T23:13:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.640+0000 7fd5e919c700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7fd5c806d6d0 0x7fd5c806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.640+0000 7fd5e899b700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5c8066f30 0x7fd5c80693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80093f0 tx=0x7fd5d8017000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.640+0000 7fd5e919c700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7fd5c806d6d0 0x7fd5c806fb40 crc :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.641 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.641+0000 7fd5e0ff9700 1 -- 172.21.15.131:0/912375630 <== osd.3 v2:172.21.15.131:6817/39609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd5c8070170 con 0x7fd5c806d6d0 2023-12-26T23:13:03.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.655+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 --> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd5e40717f0 con 0x7fd5c806d6d0 2023-12-26T23:13:03.655 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.655+0000 7fd5e0ff9700 1 -- 172.21.15.131:0/912375630 <== osd.3 v2:172.21.15.131:6817/39609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd5e40717f0 con 0x7fd5c806d6d0 2023-12-26T23:13:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:13:03.656 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-26T23:13:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:13:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7fd5c806d6d0 msgr2=0x7fd5c806fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6817/39609,v1:172.21.15.131:6820/39609] conn(0x7fd5c806d6d0 0x7fd5c806fb40 crc :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5c8066f30 msgr2=0x7fd5c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd5c8066f30 0x7fd5c80693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80093f0 tx=0x7fd5d8017000 comp rx=0 tx=0).stop 2023-12-26T23:13:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 msgr2=0x7fd5e4169ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 --2- 172.21.15.131:0/912375630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e41572b0 0x7fd5e4169ab0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000d860 tx=0x7fd5d0008040 comp rx=0 tx=0).stop 2023-12-26T23:13:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5e919c700 1 -- 172.21.15.131:0/912375630 reap_dead start 2023-12-26T23:13:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 shutdown_connections 2023-12-26T23:13:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 >> 172.21.15.131:0/912375630 conn(0x7fd5e40675b0 msgr2=0x7fd5e406d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:03.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 shutdown_connections 2023-12-26T23:13:03.658 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.656+0000 7fd5eabff700 1 -- 172.21.15.131:0/912375630 wait complete. 2023-12-26T23:13:03.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-12-26T23:13:03.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.865+0000 7f4ed17ff700 1 Processor -- start 2023-12-26T23:13:03.865 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.865+0000 7f4ed17ff700 1 -- start start 2023-12-26T23:13:03.866 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.866+0000 7f4ed17ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ecc0b0750 0x7f4ecc0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.866+0000 7f4ed17ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0c9400 0x7f4ecc0c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.866+0000 7f4ed17ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc156940 con 0x7f4ecc0cb320 2023-12-26T23:13:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.866+0000 7f4ed17ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0c8cc0 con 0x7f4ecc0b0750 2023-12-26T23:13:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.866+0000 7f4ed17ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0ad700 con 0x7f4ecc0c9400 2023-12-26T23:13:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.866+0000 7f4eca7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ecc0b0750 0x7f4ecc0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.866+0000 7f4ecaffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4ecc0cb320 0x7f4ecc069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43998/0 (socket says 172.21.15.131:43998) 2023-12-26T23:13:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.866+0000 7f4ecaffd700 1 -- 172.21.15.131:0/1259828790 learned_addr learned my addr 172.21.15.131:0/1259828790 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4ecb7fe700 1 --2- 172.21.15.131:0/1259828790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0c9400 0x7f4ecc0c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.868 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4ec9ffb700 1 -- 172.21.15.131:0/1259828790 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125745090 0 0) 0x7f4ecc156940 con 0x7f4ecc0cb320 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4eca7fc700 1 -- 172.21.15.131:0/1259828790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0c9400 msgr2=0x7f4ecc0c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4eca7fc700 1 --2- 172.21.15.131:0/1259828790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0c9400 0x7f4ecc0c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4eca7fc700 1 -- 172.21.15.131:0/1259828790 >> v1:172.21.15.131:6789/0 conn(0x7f4ecc0cb320 legacy=0x7f4ecc069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4eca7fc700 1 -- 172.21.15.131:0/1259828790 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0ae500 con 0x7f4ecc0b0750 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4eca7fc700 1 --2- 172.21.15.131:0/1259828790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0b0750 0x7f4ecc0b0bf0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098e0 tx=0x7f4ec00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759789dc895b701a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4ec9ffb700 1 -- 172.21.15.131:0/1259828790 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000b640 con 0x7f4ecc0b0750 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4ec9ffb700 1 -- 172.21.15.131:0/1259828790 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec000b7a0 con 0x7f4ecc0b0750 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.867+0000 7f4ec9ffb700 1 -- 172.21.15.131:0/1259828790 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000bb40 con 0x7f4ecc0b0750 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.868+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1259828790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0b0750 msgr2=0x7f4ecc0b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.868+0000 7f4ed17ff700 1 --2- 172.21.15.131:0/1259828790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0b0750 0x7f4ecc0b0bf0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098e0 tx=0x7f4ec00091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.868+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1259828790 shutdown_connections 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.868+0000 7f4ed17ff700 1 --2- 172.21.15.131:0/1259828790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0c9400 0x7f4ecc0c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.868+0000 7f4ed17ff700 1 --2- 172.21.15.131:0/1259828790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0b0750 0x7f4ecc0b0bf0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.869 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.868+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1259828790 >> 172.21.15.131:0/1259828790 conn(0x7f4ecc068040 msgr2=0x7f4ecc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:03.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.868+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1259828790 shutdown_connections 2023-12-26T23:13:03.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.868+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1259828790 wait complete. 2023-12-26T23:13:03.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.869+0000 7f4ed17ff700 1 Processor -- start 2023-12-26T23:13:03.870 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.870+0000 7f4ed17ff700 1 -- start start 2023-12-26T23:13:03.871 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.870+0000 7f4ed17ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0b0750 0x7f4ecc16d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.870+0000 7f4ed17ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc0c9400 0x7f4ecc15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ed17ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cb320 0x7f4ecc1438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ed17ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0c53a0 con 0x7f4ecc0cb320 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ed17ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0c7d60 con 0x7f4ecc0c9400 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ed17ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0c8040 con 0x7f4ecc0b0750 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ecb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cb320 0x7f4ecc1438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ecb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cb320 0x7f4ecc1438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37000/0 (socket says 172.21.15.131:37000) 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ecb7fe700 1 -- 172.21.15.131:0/1078565 learned_addr learned my addr 172.21.15.131:0/1078565 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ecaffd700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0b0750 0x7f4ecc16d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4eca7fc700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc0c9400 0x7f4ecc15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ecb7fe700 1 -- 172.21.15.131:0/1078565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0b0750 msgr2=0x7f4ecc16d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ecb7fe700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0b0750 0x7f4ecc16d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ecb7fe700 1 -- 172.21.15.131:0/1078565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc0c9400 msgr2=0x7f4ecc15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:03.872 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ecb7fe700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc0c9400 0x7f4ecc15d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:03.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.871+0000 7f4ecb7fe700 1 -- 172.21.15.131:0/1078565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0009420 con 0x7f4ecc0cb320 2023-12-26T23:13:03.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.872+0000 7f4ecb7fe700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cb320 0x7f4ecc1438f0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009e90 tx=0x7f4ec400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.873 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.872+0000 7f4eb2ffd700 1 -- 172.21.15.131:0/1078565 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400cb70 con 0x7f4ecc0cb320 2023-12-26T23:13:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.872+0000 7f4eb2ffd700 1 -- 172.21.15.131:0/1078565 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec400f040 con 0x7f4ecc0cb320 2023-12-26T23:13:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.872+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc0c55b0 con 0x7f4ecc0cb320 2023-12-26T23:13:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.872+0000 7f4eb2ffd700 1 -- 172.21.15.131:0/1078565 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4014480 con 0x7f4ecc0cb320 2023-12-26T23:13:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.872+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc0c79c0 con 0x7f4ecc0cb320 2023-12-26T23:13:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.873+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc0657e0 con 0x7f4ecc0cb320 2023-12-26T23:13:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.874+0000 7f4eb2ffd700 1 -- 172.21.15.131:0/1078565 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4ec4014830 con 0x7f4ecc0cb320 2023-12-26T23:13:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.875+0000 7f4eb2ffd700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4eb8066f30 0x7f4eb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.875+0000 7f4eb2ffd700 1 -- 172.21.15.131:0/1078565 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f4ec4085040 con 0x7f4ecc0cb320 2023-12-26T23:13:03.875 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.875+0000 7f4ecaffd700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4eb8066f30 0x7f4eb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:03.876 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.876+0000 7f4ecaffd700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4eb8066f30 0x7f4eb80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc007160 tx=0x7f4ebc007f50 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:03.879 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:03.879+0000 7f4eb2ffd700 1 -- 172.21.15.131:0/1078565 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ec4052170 con 0x7f4ecc0cb320 2023-12-26T23:13:04.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.071+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ecc0c8200 con 0x7f4eb8066f30 2023-12-26T23:13:04.071 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.071+0000 7f4eb2ffd700 1 -- 172.21.15.131:0/1078565 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ecc0c8200 con 0x7f4eb8066f30 2023-12-26T23:13:04.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4eb8066f30 msgr2=0x7f4eb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4eb8066f30 0x7f4eb80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc007160 tx=0x7f4ebc007f50 comp rx=0 tx=0).stop 2023-12-26T23:13:04.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cb320 msgr2=0x7f4ecc1438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cb320 0x7f4ecc1438f0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009e90 tx=0x7f4ec400b040 comp rx=0 tx=0).stop 2023-12-26T23:13:04.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 shutdown_connections 2023-12-26T23:13:04.074 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4eb8066f30 0x7f4eb80693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ecc0cb320 0x7f4ecc1438f0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ecc0b0750 0x7f4ecc16d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 --2- 172.21.15.131:0/1078565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ecc0c9400 0x7f4ecc15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 >> 172.21.15.131:0/1078565 conn(0x7f4ecc068040 msgr2=0x7f4ecc157430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:04.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 shutdown_connections 2023-12-26T23:13:04.075 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.074+0000 7f4ed17ff700 1 -- 172.21.15.131:0/1078565 wait complete. 2023-12-26T23:13:04.075 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 3 to deep-scrub 2023-12-26T23:13:04.092 INFO:tasks.ceph:Scrubbing osd.4 2023-12-26T23:13:04.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-12-26T23:13:04.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.284+0000 7f60403ff700 1 Processor -- start 2023-12-26T23:13:04.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.284+0000 7f60403ff700 1 -- start start 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.285+0000 7f60403ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6038069c30 0x7f60380abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.285+0000 7f60403ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60380c7430 con 0x7f60380ae080 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.285+0000 7f60403ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60380ac460 con 0x7f60380adcb0 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.285+0000 7f60403ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60380c6820 con 0x7f6038069c30 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603e99c700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f60380ae080 0x7f603806df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44000/0 (socket says 172.21.15.131:44000) 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603e99c700 1 -- 172.21.15.131:0/811271153 learned_addr learned my addr 172.21.15.131:0/811271153 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603d99a700 1 --2- 172.21.15.131:0/811271153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6038069c30 0x7f60380abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603d199700 1 -- 172.21.15.131:0/811271153 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099767188 0 0) 0x7f60380c7430 con 0x7f60380ae080 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603d199700 1 -- 172.21.15.131:0/811271153 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6024003570 con 0x7f60380ae080 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603d199700 1 -- 172.21.15.131:0/811271153 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254782649 0 0) 0x7f60380ac460 con 0x7f60380adcb0 2023-12-26T23:13:04.287 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603d199700 1 -- 172.21.15.131:0/811271153 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60380c7430 con 0x7f60380adcb0 2023-12-26T23:13:04.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603d199700 1 -- 172.21.15.131:0/811271153 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2605217024 0 0) 0x7f6024003570 con 0x7f60380ae080 2023-12-26T23:13:04.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603d99a700 1 -- 172.21.15.131:0/811271153 >> v1:172.21.15.190:6789/0 conn(0x7f60380adcb0 legacy=0x7f6038069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603d99a700 1 -- 172.21.15.131:0/811271153 >> v1:172.21.15.131:6789/0 conn(0x7f60380ae080 legacy=0x7f603806df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.286+0000 7f603d99a700 1 -- 172.21.15.131:0/811271153 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60380c6b00 con 0x7f6038069c30 2023-12-26T23:13:04.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.287+0000 7f603d99a700 1 --2- 172.21.15.131:0/811271153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6038069c30 0x7f60380abf20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f602800aac0 tx=0x7f602800a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b4ec88195955ad server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.287+0000 7f603d199700 1 -- 172.21.15.131:0/811271153 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300017f0 con 0x7f6038069c30 2023-12-26T23:13:04.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.287+0000 7f603d199700 1 -- 172.21.15.131:0/811271153 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602800c660 con 0x7f6038069c30 2023-12-26T23:13:04.288 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.287+0000 7f603d199700 1 -- 172.21.15.131:0/811271153 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800ca00 con 0x7f6038069c30 2023-12-26T23:13:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.287+0000 7f60403ff700 1 -- 172.21.15.131:0/811271153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6038069c30 msgr2=0x7f60380abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.287+0000 7f60403ff700 1 --2- 172.21.15.131:0/811271153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6038069c30 0x7f60380abf20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f602800aac0 tx=0x7f602800a330 comp rx=0 tx=0).stop 2023-12-26T23:13:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.287+0000 7f60403ff700 1 -- 172.21.15.131:0/811271153 shutdown_connections 2023-12-26T23:13:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.287+0000 7f60403ff700 1 --2- 172.21.15.131:0/811271153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6038069c30 0x7f60380abf20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.287+0000 7f60403ff700 1 -- 172.21.15.131:0/811271153 >> 172.21.15.131:0/811271153 conn(0x7f60380675b0 msgr2=0x7f60380679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.288+0000 7f60403ff700 1 -- 172.21.15.131:0/811271153 shutdown_connections 2023-12-26T23:13:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.288+0000 7f60403ff700 1 -- 172.21.15.131:0/811271153 wait complete. 2023-12-26T23:13:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.289+0000 7f60403ff700 1 Processor -- start 2023-12-26T23:13:04.289 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.289+0000 7f60403ff700 1 -- start start 2023-12-26T23:13:04.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.290+0000 7f60403ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6038069c30 0x7f60380cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.291 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.290+0000 7f60403ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60380adcb0 0x7f603814d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.290+0000 7f60403ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60380ae080 0x7f60380af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.290+0000 7f60403ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60380cbbc0 con 0x7f60380adcb0 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.290+0000 7f60403ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60380c9d10 con 0x7f6038069c30 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.290+0000 7f60403ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60380c9f60 con 0x7f60380ae080 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.290+0000 7f603d99a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60380adcb0 0x7f603814d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.290+0000 7f603d99a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60380adcb0 0x7f603814d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37016/0 (socket says 172.21.15.131:37016) 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.290+0000 7f603d99a700 1 -- 172.21.15.131:0/3041844369 learned_addr learned my addr 172.21.15.131:0/3041844369 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603e19b700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6038069c30 0x7f60380cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603e99c700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60380ae080 0x7f60380af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603d99a700 1 -- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60380ae080 msgr2=0x7f60380af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603d99a700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60380ae080 0x7f60380af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603d99a700 1 -- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6038069c30 msgr2=0x7f60380cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603d99a700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6038069c30 0x7f60380cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603d99a700 1 -- 172.21.15.131:0/3041844369 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602800a5d0 con 0x7f60380adcb0 2023-12-26T23:13:04.292 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603e99c700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60380ae080 0x7f60380af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603e19b700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6038069c30 0x7f60380cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f603d99a700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60380adcb0 0x7f603814d790 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f60280030c0 tx=0x7f6028019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f602e7fc700 1 -- 172.21.15.131:0/3041844369 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028000d10 con 0x7f60380adcb0 2023-12-26T23:13:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f602e7fc700 1 -- 172.21.15.131:0/3041844369 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602800c7d0 con 0x7f60380adcb0 2023-12-26T23:13:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60380ca240 con 0x7f60380adcb0 2023-12-26T23:13:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.291+0000 7f602e7fc700 1 -- 172.21.15.131:0/3041844369 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028022420 con 0x7f60380adcb0 2023-12-26T23:13:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.292+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60380ca800 con 0x7f60380adcb0 2023-12-26T23:13:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.292+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60380717f0 con 0x7f60380adcb0 2023-12-26T23:13:04.293 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.293+0000 7f602e7fc700 1 -- 172.21.15.131:0/3041844369 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6028020070 con 0x7f60380adcb0 2023-12-26T23:13:04.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.294+0000 7f602e7fc700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f601c066f30 0x7f601c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.294+0000 7f602e7fc700 1 -- 172.21.15.131:0/3041844369 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f6028085850 con 0x7f60380adcb0 2023-12-26T23:13:04.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.294+0000 7f603e19b700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f601c066f30 0x7f601c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.295+0000 7f602e7fc700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f601c06d560 0x7f601c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.295+0000 7f602e7fc700 1 -- 172.21.15.131:0/3041844369 --> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f601c070000 con 0x7f601c06d560 2023-12-26T23:13:04.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.295+0000 7f602e7fc700 1 -- 172.21.15.131:0/3041844369 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=215) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6028085b40 con 0x7f60380adcb0 2023-12-26T23:13:04.295 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.295+0000 7f603e19b700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f601c066f30 0x7f601c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6034003990 tx=0x7f6034009040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.295+0000 7f603e99c700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f601c06d560 0x7f601c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.296+0000 7f603e99c700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f601c06d560 0x7f601c06f9d0 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.296 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.297+0000 7f602e7fc700 1 -- 172.21.15.131:0/3041844369 <== osd.4 v2:172.21.15.190:6801/40010 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f601c070000 con 0x7f601c06d560 2023-12-26T23:13:04.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.311+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 --> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f60380c6de0 con 0x7f601c06d560 2023-12-26T23:13:04.311 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.311+0000 7f602e7fc700 1 -- 172.21.15.131:0/3041844369 <== osd.4 v2:172.21.15.190:6801/40010 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f60380c6de0 con 0x7f601c06d560 2023-12-26T23:13:04.312 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:13:04.312 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-26T23:13:04.312 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:13:04.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f601c06d560 msgr2=0x7f601c06f9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.312 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.190:6801/40010,v1:172.21.15.190:6803/40010] conn(0x7f601c06d560 0x7f601c06f9d0 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f601c066f30 msgr2=0x7f601c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f601c066f30 0x7f601c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6034003990 tx=0x7f6034009040 comp rx=0 tx=0).stop 2023-12-26T23:13:04.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60380adcb0 msgr2=0x7f603814d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 --2- 172.21.15.131:0/3041844369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60380adcb0 0x7f603814d790 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f60280030c0 tx=0x7f6028019000 comp rx=0 tx=0).stop 2023-12-26T23:13:04.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f603e99c700 1 -- 172.21.15.131:0/3041844369 reap_dead start 2023-12-26T23:13:04.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 shutdown_connections 2023-12-26T23:13:04.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 >> 172.21.15.131:0/3041844369 conn(0x7f60380675b0 msgr2=0x7f60380ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:04.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 shutdown_connections 2023-12-26T23:13:04.313 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.312+0000 7f60403ff700 1 -- 172.21.15.131:0/3041844369 wait complete. 2023-12-26T23:13:04.329 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-12-26T23:13:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.521+0000 7fb7a41ff700 1 Processor -- start 2023-12-26T23:13:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.521+0000 7fb7a41ff700 1 -- start start 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.522+0000 7fb7a41ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0cc810 0x7fb79c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.522+0000 7fb7a41ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79c0686c0 con 0x7fb79c06bfd0 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.522+0000 7fb7a41ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79c069520 con 0x7fb79c13c510 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.522+0000 7fb7a41ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c068910 con 0x7fb79c0cc810 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a179a700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb79c06bfd0 0x7fb79c0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44014/0 (socket says 172.21.15.131:44014) 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a179a700 1 -- 172.21.15.131:0/802628053 learned_addr learned my addr 172.21.15.131:0/802628053 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a279c700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0cc810 0x7fb79c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962636409 0 0) 0x7fb79c0686c0 con 0x7fb79c06bfd0 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb788003570 con 0x7fb79c06bfd0 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757360638 0 0) 0x7fb788003570 con 0x7fb79c06bfd0 2023-12-26T23:13:04.524 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0cc810 msgr2=0x7fb79c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0cc810 0x7fb79c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 >> v1:172.21.15.190:6789/0 conn(0x7fb79c13c510 legacy=0x7fb79c06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c068bf0 con 0x7fb79c06bfd0 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7fb78c001470 con 0x7fb79c06bfd0 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 >> v1:172.21.15.131:6789/0 conn(0x7fb79c06bfd0 legacy=0x7fb79c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb788003a90 0x7fb788003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0cc810 0x7fb788004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.523+0000 7fb7a0f99700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb788004d50 0x7fb7880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c068910 con 0x7fb788004d50 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78800bb40 con 0x7fb788003a90 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78800bd00 con 0x7fb79c0cc810 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a179a700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0cc810 0x7fb788004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a279c700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb788004d50 0x7fb7880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.525 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a1f9b700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb788003a90 0x7fb788003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a279c700 1 -- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0cc810 msgr2=0x7fb788004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a279c700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0cc810 0x7fb788004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a279c700 1 -- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb788003a90 msgr2=0x7fb788003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a279c700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb788003a90 0x7fb788003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a279c700 1 -- 172.21.15.131:0/802628053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb788003570 con 0x7fb788004d50 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.524+0000 7fb7a279c700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb788004d50 0x7fb7880051c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fb79400e210 tx=0x7fb79400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4f86e02d03f62a server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79400eac0 con 0x7fb788004d50 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79400f040 con 0x7fb788004d50 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a0f99700 1 -- 172.21.15.131:0/802628053 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79400ee70 con 0x7fb788004d50 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a41ff700 1 -- 172.21.15.131:0/802628053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb788004d50 msgr2=0x7fb7880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a41ff700 1 --2- 172.21.15.131:0/802628053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb788004d50 0x7fb7880051c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fb79400e210 tx=0x7fb79400b040 comp rx=0 tx=0).stop 2023-12-26T23:13:04.527 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a279c700 1 -- 172.21.15.131:0/802628053 reap_dead start 2023-12-26T23:13:04.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a41ff700 1 -- 172.21.15.131:0/802628053 shutdown_connections 2023-12-26T23:13:04.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a41ff700 1 -- 172.21.15.131:0/802628053 >> 172.21.15.131:0/802628053 conn(0x7fb79c0ce3e0 msgr2=0x7fb79c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:04.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a41ff700 1 -- 172.21.15.131:0/802628053 shutdown_connections 2023-12-26T23:13:04.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.525+0000 7fb7a41ff700 1 -- 172.21.15.131:0/802628053 wait complete. 2023-12-26T23:13:04.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.527+0000 7fb7a41ff700 1 Processor -- start 2023-12-26T23:13:04.528 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.527+0000 7fb7a41ff700 1 -- start start 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a41ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb788004d50 0x7fb79c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a41ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb79c1521b0 0x7fb79c139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a41ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0c3e70 0x7fb79c0c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a41ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0c4850 con 0x7fb79c1521b0 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a41ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0b2b90 con 0x7fb788004d50 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a41ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0b2e70 con 0x7fb79c0c3e70 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a179a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb79c1521b0 0x7fb79c139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a179a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb79c1521b0 0x7fb79c139ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37030/0 (socket says 172.21.15.131:37030) 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a179a700 1 -- 172.21.15.131:0/24401214 learned_addr learned my addr 172.21.15.131:0/24401214 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a1f9b700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb788004d50 0x7fb79c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a279c700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0c3e70 0x7fb79c0c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.530 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a179a700 1 -- 172.21.15.131:0/24401214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0c3e70 msgr2=0x7fb79c0c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a179a700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0c3e70 0x7fb79c0c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a179a700 1 -- 172.21.15.131:0/24401214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb788004d50 msgr2=0x7fb79c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a179a700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb788004d50 0x7fb79c0b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.528+0000 7fb7a179a700 1 -- 172.21.15.131:0/24401214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c0b3150 con 0x7fb79c1521b0 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.529+0000 7fb7a179a700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb79c1521b0 0x7fb79c139ea0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fb78c007110 tx=0x7fb78c001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.529+0000 7fb791ffb700 1 -- 172.21.15.131:0/24401214 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00f610 con 0x7fb79c1521b0 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.529+0000 7fb791ffb700 1 -- 172.21.15.131:0/24401214 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c00f770 con 0x7fb79c1521b0 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.529+0000 7fb791ffb700 1 -- 172.21.15.131:0/24401214 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00fb30 con 0x7fb79c1521b0 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.529+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79c0b3430 con 0x7fb79c1521b0 2023-12-26T23:13:04.531 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.529+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79c13a3e0 con 0x7fb79c1521b0 2023-12-26T23:13:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.530+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79c0b37a0 con 0x7fb79c1521b0 2023-12-26T23:13:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.531+0000 7fb791ffb700 1 -- 172.21.15.131:0/24401214 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb78c01d460 con 0x7fb79c1521b0 2023-12-26T23:13:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.532+0000 7fb791ffb700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb780066f30 0x7fb7800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.532+0000 7fb791ffb700 1 -- 172.21.15.131:0/24401214 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fb78c084820 con 0x7fb79c1521b0 2023-12-26T23:13:04.532 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.532+0000 7fb7a1f9b700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb780066f30 0x7fb7800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.533 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.533+0000 7fb7a1f9b700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb780066f30 0x7fb7800693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb798007ab0 tx=0x7fb798009040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.536 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.536+0000 7fb791ffb700 1 -- 172.21.15.131:0/24401214 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb78c0518b0 con 0x7fb79c1521b0 2023-12-26T23:13:04.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.729+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb79c0657e0 con 0x7fb780066f30 2023-12-26T23:13:04.729 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.729+0000 7fb791ffb700 1 -- 172.21.15.131:0/24401214 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb79c0657e0 con 0x7fb780066f30 2023-12-26T23:13:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.731+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb780066f30 msgr2=0x7fb7800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.731+0000 7fb7a41ff700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb780066f30 0x7fb7800693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb798007ab0 tx=0x7fb798009040 comp rx=0 tx=0).stop 2023-12-26T23:13:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb79c1521b0 msgr2=0x7fb79c139ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb79c1521b0 0x7fb79c139ea0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fb78c007110 tx=0x7fb78c001da0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 shutdown_connections 2023-12-26T23:13:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fb780066f30 0x7fb7800693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb79c0c3e70 0x7fb79c0c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb788004d50 0x7fb79c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 --2- 172.21.15.131:0/24401214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb79c1521b0 0x7fb79c139ea0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 >> 172.21.15.131:0/24401214 conn(0x7fb79c0ce3e0 msgr2=0x7fb79c15f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 shutdown_connections 2023-12-26T23:13:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.732+0000 7fb7a41ff700 1 -- 172.21.15.131:0/24401214 wait complete. 2023-12-26T23:13:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 4 to deep-scrub 2023-12-26T23:13:04.750 INFO:tasks.ceph:Scrubbing osd.5 2023-12-26T23:13:04.750 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-12-26T23:13:04.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.943+0000 7f495a3ff700 1 Processor -- start 2023-12-26T23:13:04.944 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.944+0000 7f495a3ff700 1 -- start start 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.945+0000 7f495a3ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49540adcb0 0x7f49540abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.945+0000 7f495a3ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f49540ae080 0x7f4954069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.945+0000 7f495a3ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49540c7430 con 0x7f4954069a60 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.945+0000 7f495a3ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49540ac460 con 0x7f49540adcb0 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.945+0000 7f495a3ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4954069e80 con 0x7f49540ae080 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.945+0000 7f4953fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49540adcb0 0x7f49540abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f4953fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49540adcb0 0x7f49540abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47848/0 (socket says 172.21.15.131:47848) 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f4953fff700 1 -- 172.21.15.131:0/1729393422 learned_addr learned my addr 172.21.15.131:0/1729393422 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f494bfff700 1 --2- 172.21.15.131:0/1729393422 >> v2:172.21.15.190:3300/0 conn(0x7f49540ae080 0x7f4954069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f4953fff700 1 -- 172.21.15.131:0/1729393422 >> v2:172.21.15.190:3300/0 conn(0x7f49540ae080 msgr2=0x7f4954069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f4953fff700 1 --2- 172.21.15.131:0/1729393422 >> v2:172.21.15.190:3300/0 conn(0x7f49540ae080 0x7f4954069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f4953fff700 1 -- 172.21.15.131:0/1729393422 >> v1:172.21.15.190:6789/0 conn(0x7f4954069a60 legacy=0x7f495406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f4953fff700 1 -- 172.21.15.131:0/1729393422 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49540c6820 con 0x7f49540adcb0 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f494bfff700 1 --2- 172.21.15.131:0/1729393422 >> v2:172.21.15.190:3300/0 conn(0x7f49540ae080 0x7f4954069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f4953fff700 1 --2- 172.21.15.131:0/1729393422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 0x7f49540abf20 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f4954071110 tx=0x7f4944009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a660365753a9aa1 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.946+0000 7f49537fe700 1 -- 172.21.15.131:0/1729393422 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494400b860 con 0x7f49540adcb0 2023-12-26T23:13:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f49537fe700 1 -- 172.21.15.131:0/1729393422 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494400b9c0 con 0x7f49540adcb0 2023-12-26T23:13:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f49537fe700 1 -- 172.21.15.131:0/1729393422 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494400bd60 con 0x7f49540adcb0 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f495a3ff700 1 -- 172.21.15.131:0/1729393422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 msgr2=0x7f49540abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f495a3ff700 1 --2- 172.21.15.131:0/1729393422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 0x7f49540abf20 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f4954071110 tx=0x7f4944009250 comp rx=0 tx=0).stop 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f495a3ff700 1 -- 172.21.15.131:0/1729393422 shutdown_connections 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f495a3ff700 1 --2- 172.21.15.131:0/1729393422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 0x7f49540abf20 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f495a3ff700 1 --2- 172.21.15.131:0/1729393422 >> v2:172.21.15.190:3300/0 conn(0x7f49540ae080 0x7f4954069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f495a3ff700 1 -- 172.21.15.131:0/1729393422 >> 172.21.15.131:0/1729393422 conn(0x7f49540675b0 msgr2=0x7f49540679b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f495a3ff700 1 -- 172.21.15.131:0/1729393422 shutdown_connections 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.947+0000 7f495a3ff700 1 -- 172.21.15.131:0/1729393422 wait complete. 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.949+0000 7f495a3ff700 1 Processor -- start 2023-12-26T23:13:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.949+0000 7f495a3ff700 1 -- start start 2023-12-26T23:13:04.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f495a3ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4954069a60 0x7f49540aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f495a3ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 0x7f49540a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.951 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f495a3ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49540ae080 0x7f49540b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f495a3ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49540b12e0 con 0x7f49540adcb0 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f495a3ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49540aaf40 con 0x7f4954069a60 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f495a3ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49540a5dc0 con 0x7f49540ae080 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f494bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 0x7f49540a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f494bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 0x7f49540a5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47864/0 (socket says 172.21.15.131:47864) 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f494bfff700 1 -- 172.21.15.131:0/2788827776 learned_addr learned my addr 172.21.15.131:0/2788827776 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f4953fff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4954069a60 0x7f49540aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f494bfff700 1 -- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49540ae080 msgr2=0x7f49540b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f495899c700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49540ae080 0x7f49540b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f494bfff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49540ae080 0x7f49540b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f494bfff700 1 -- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4954069a60 msgr2=0x7f49540aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f494bfff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4954069a60 0x7f49540aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f494bfff700 1 -- 172.21.15.131:0/2788827776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440094f0 con 0x7f49540adcb0 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f4953fff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4954069a60 0x7f49540aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.950+0000 7f495899c700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49540ae080 0x7f49540b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:13:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.951+0000 7f494bfff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 0x7f49540a5880 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e8a0 tx=0x7f493c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.951+0000 7f4950ff9700 1 -- 172.21.15.131:0/2788827776 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c00c430 con 0x7f49540adcb0 2023-12-26T23:13:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.951+0000 7f4950ff9700 1 -- 172.21.15.131:0/2788827776 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c00c590 con 0x7f49540adcb0 2023-12-26T23:13:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.951+0000 7f4950ff9700 1 -- 172.21.15.131:0/2788827776 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c00c950 con 0x7f49540adcb0 2023-12-26T23:13:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.951+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49540b06d0 con 0x7f49540adcb0 2023-12-26T23:13:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.951+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49540b0c90 con 0x7f49540adcb0 2023-12-26T23:13:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.952+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49540717f0 con 0x7f49540adcb0 2023-12-26T23:13:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.953+0000 7f4950ff9700 1 -- 172.21.15.131:0/2788827776 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f493c00cd00 con 0x7f49540adcb0 2023-12-26T23:13:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.954+0000 7f4950ff9700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4934066f30 0x7f49340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.954+0000 7f4950ff9700 1 -- 172.21.15.131:0/2788827776 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f493c084480 con 0x7f49540adcb0 2023-12-26T23:13:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.954+0000 7f4953fff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4934066f30 0x7f49340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.954+0000 7f4950ff9700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7f493406d6d0 0x7f493406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.954+0000 7f4950ff9700 1 -- 172.21.15.131:0/2788827776 --> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4934070170 con 0x7f493406d6d0 2023-12-26T23:13:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.954+0000 7f4950ff9700 1 -- 172.21.15.131:0/2788827776 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=215) v2 ==== 24+0+0 (secure 0 0 0) 0x7f493c086980 con 0x7f49540adcb0 2023-12-26T23:13:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.955+0000 7f495899c700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7f493406d6d0 0x7f493406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.955+0000 7f4953fff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4934066f30 0x7f49340693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4954071110 tx=0x7f4944017000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.955+0000 7f495899c700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7f493406d6d0 0x7f493406fb40 crc :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:04.956 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.956+0000 7f4950ff9700 1 -- 172.21.15.131:0/2788827776 <== osd.5 v2:172.21.15.190:6817/40012 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4934070170 con 0x7f493406d6d0 2023-12-26T23:13:04.970 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.970+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 --> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f49540c6b00 con 0x7f493406d6d0 2023-12-26T23:13:04.971 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f4950ff9700 1 -- 172.21.15.131:0/2788827776 <== osd.5 v2:172.21.15.190:6817/40012 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f49540c6b00 con 0x7f493406d6d0 2023-12-26T23:13:04.971 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:13:04.971 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-26T23:13:04.971 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7f493406d6d0 msgr2=0x7f493406fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.190:6817/40012,v1:172.21.15.190:6818/40012] conn(0x7f493406d6d0 0x7f493406fb40 crc :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4934066f30 msgr2=0x7f49340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f4934066f30 0x7f49340693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4954071110 tx=0x7f4944017000 comp rx=0 tx=0).stop 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 msgr2=0x7f49540a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 --2- 172.21.15.131:0/2788827776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49540adcb0 0x7f49540a5880 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e8a0 tx=0x7f493c008040 comp rx=0 tx=0).stop 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495899c700 1 -- 172.21.15.131:0/2788827776 reap_dead start 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 shutdown_connections 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 >> 172.21.15.131:0/2788827776 conn(0x7f49540675b0 msgr2=0x7f49540c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:04.972 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 shutdown_connections 2023-12-26T23:13:04.973 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:04.971+0000 7f495a3ff700 1 -- 172.21.15.131:0/2788827776 wait complete. 2023-12-26T23:13:04.988 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-12-26T23:13:05.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.178+0000 7fec3a5ff700 1 Processor -- start 2023-12-26T23:13:05.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.178+0000 7fec3a5ff700 1 -- start start 2023-12-26T23:13:05.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.179+0000 7fec3a5ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fec340c8820 0x7fec340ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.179+0000 7fec3a5ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fec340c8bf0 0x7fec340b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.179+0000 7fec3a5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec34069d10 0x7fec340b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.179+0000 7fec3a5ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec34157a80 con 0x7fec340c8bf0 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.179+0000 7fec3a5ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec340cac50 con 0x7fec340c8820 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.179+0000 7fec3a5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec340caf30 con 0x7fec34069d10 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.179+0000 7fec337fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fec340c8bf0 0x7fec340b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.179+0000 7fec337fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fec340c8bf0 0x7fec340b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47878/0 (socket says 172.21.15.131:47878) 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec337fe700 1 -- 172.21.15.131:0/3799796515 learned_addr learned my addr 172.21.15.131:0/3799796515 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec33fff700 1 --2- 172.21.15.131:0/3799796515 >> v2:172.21.15.190:3300/0 conn(0x7fec340c8820 0x7fec340ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec38b9c700 1 --2- 172.21.15.131:0/3799796515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec34069d10 0x7fec340b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec337fe700 1 -- 172.21.15.131:0/3799796515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec34069d10 msgr2=0x7fec340b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec337fe700 1 --2- 172.21.15.131:0/3799796515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec34069d10 0x7fec340b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec337fe700 1 -- 172.21.15.131:0/3799796515 >> v2:172.21.15.190:3300/0 conn(0x7fec340c8820 msgr2=0x7fec340ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec337fe700 1 --2- 172.21.15.131:0/3799796515 >> v2:172.21.15.190:3300/0 conn(0x7fec340c8820 0x7fec340ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec337fe700 1 -- 172.21.15.131:0/3799796515 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec340cb210 con 0x7fec340c8bf0 2023-12-26T23:13:05.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec337fe700 1 --2- 172.21.15.131:0/3799796515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec340c8bf0 0x7fec340b12e0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fec34064ea0 tx=0x7fec28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1499514c6924ea6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec32ffd700 1 -- 172.21.15.131:0/3799796515 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2800b640 con 0x7fec340c8bf0 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec32ffd700 1 -- 172.21.15.131:0/3799796515 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2800b7a0 con 0x7fec340c8bf0 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.180+0000 7fec32ffd700 1 -- 172.21.15.131:0/3799796515 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2800bb60 con 0x7fec340c8bf0 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.181+0000 7fec3a5ff700 1 -- 172.21.15.131:0/3799796515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec340c8bf0 msgr2=0x7fec340b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.181+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/3799796515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec340c8bf0 0x7fec340b12e0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fec34064ea0 tx=0x7fec28009250 comp rx=0 tx=0).stop 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.181+0000 7fec3a5ff700 1 -- 172.21.15.131:0/3799796515 shutdown_connections 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.181+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/3799796515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec34069d10 0x7fec340b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.181+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/3799796515 >> v2:172.21.15.190:3300/0 conn(0x7fec340c8820 0x7fec340ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.181+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/3799796515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec340c8bf0 0x7fec340b12e0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.181+0000 7fec3a5ff700 1 -- 172.21.15.131:0/3799796515 >> 172.21.15.131:0/3799796515 conn(0x7fec34068040 msgr2=0x7fec34069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.181+0000 7fec3a5ff700 1 -- 172.21.15.131:0/3799796515 shutdown_connections 2023-12-26T23:13:05.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.181+0000 7fec3a5ff700 1 -- 172.21.15.131:0/3799796515 wait complete. 2023-12-26T23:13:05.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.182+0000 7fec3a5ff700 1 Processor -- start 2023-12-26T23:13:05.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.183+0000 7fec3a5ff700 1 -- start start 2023-12-26T23:13:05.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.183+0000 7fec3a5ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec34069d10 0x7fec3416d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.183+0000 7fec3a5ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec340c8820 0x7fec34155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec3a5ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec340c8bf0 0x7fec340a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec3a5ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34155840 con 0x7fec34069d10 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec3a5ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec340a5a50 con 0x7fec340c8820 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec3a5ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3416d5f0 con 0x7fec340c8bf0 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec33fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec34069d10 0x7fec3416d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec33fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec34069d10 0x7fec3416d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47890/0 (socket says 172.21.15.131:47890) 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec33fff700 1 -- 172.21.15.131:0/1413172181 learned_addr learned my addr 172.21.15.131:0/1413172181 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec337fe700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec340c8820 0x7fec34155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec38b9c700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec340c8bf0 0x7fec340a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec33fff700 1 -- 172.21.15.131:0/1413172181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec340c8bf0 msgr2=0x7fec340a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec33fff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec340c8bf0 0x7fec340a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec33fff700 1 -- 172.21.15.131:0/1413172181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec340c8820 msgr2=0x7fec34155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec33fff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec340c8820 0x7fec34155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec33fff700 1 -- 172.21.15.131:0/1413172181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280094f0 con 0x7fec34069d10 2023-12-26T23:13:05.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.184+0000 7fec33fff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec34069d10 0x7fec3416d010 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fec2400ecb0 tx=0x7fec240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.185+0000 7fec1bfff700 1 -- 172.21.15.131:0/1413172181 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2400c850 con 0x7fec34069d10 2023-12-26T23:13:05.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.185+0000 7fec1bfff700 1 -- 172.21.15.131:0/1413172181 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2400c9b0 con 0x7fec34069d10 2023-12-26T23:13:05.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.185+0000 7fec1bfff700 1 -- 172.21.15.131:0/1413172181 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2400cd70 con 0x7fec34069d10 2023-12-26T23:13:05.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.185+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3416d8d0 con 0x7fec34069d10 2023-12-26T23:13:05.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.185+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3416de90 con 0x7fec34069d10 2023-12-26T23:13:05.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.186+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec340657e0 con 0x7fec34069d10 2023-12-26T23:13:05.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.187+0000 7fec1bfff700 1 -- 172.21.15.131:0/1413172181 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fec24013680 con 0x7fec34069d10 2023-12-26T23:13:05.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.188+0000 7fec1bfff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fec20066f30 0x7fec200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.188+0000 7fec1bfff700 1 -- 172.21.15.131:0/1413172181 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fec24084880 con 0x7fec34069d10 2023-12-26T23:13:05.188 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.188+0000 7fec337fe700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fec20066f30 0x7fec200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.189+0000 7fec337fe700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fec20066f30 0x7fec200693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec34064ea0 tx=0x7fec28017000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.192 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.192+0000 7fec1bfff700 1 -- 172.21.15.131:0/1413172181 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec24051910 con 0x7fec34069d10 2023-12-26T23:13:05.385 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.385+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fec340cb4f0 con 0x7fec20066f30 2023-12-26T23:13:05.386 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.386+0000 7fec1bfff700 1 -- 172.21.15.131:0/1413172181 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fec340cb4f0 con 0x7fec20066f30 2023-12-26T23:13:05.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fec20066f30 msgr2=0x7fec200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fec20066f30 0x7fec200693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec34064ea0 tx=0x7fec28017000 comp rx=0 tx=0).stop 2023-12-26T23:13:05.388 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec34069d10 msgr2=0x7fec3416d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec34069d10 0x7fec3416d010 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fec2400ecb0 tx=0x7fec240091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 shutdown_connections 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fec20066f30 0x7fec200693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec340c8bf0 0x7fec340a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec34069d10 0x7fec3416d010 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 --2- 172.21.15.131:0/1413172181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec340c8820 0x7fec34155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 >> 172.21.15.131:0/1413172181 conn(0x7fec34068040 msgr2=0x7fec34158750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 shutdown_connections 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.388+0000 7fec3a5ff700 1 -- 172.21.15.131:0/1413172181 wait complete. 2023-12-26T23:13:05.389 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 5 to deep-scrub 2023-12-26T23:13:05.406 INFO:tasks.ceph:Scrubbing osd.6 2023-12-26T23:13:05.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-12-26T23:13:05.599 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.599+0000 7fe14c9ff700 1 Processor -- start 2023-12-26T23:13:05.600 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.599+0000 7fe14c9ff700 1 -- start start 2023-12-26T23:13:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.600+0000 7fe14c9ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe148142f00 0x7fe148069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.600+0000 7fe14c9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1480cc590 0x7fe1480cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14c9ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe148069640 con 0x7fe1480cb340 2023-12-26T23:13:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14c9ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1480cb7c0 con 0x7fe148142f00 2023-12-26T23:13:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14c9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe148068750 con 0x7fe1480cc590 2023-12-26T23:13:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe146d9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe1480cb340 0x7fe1480c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46684/0 (socket says 172.21.15.131:46684) 2023-12-26T23:13:05.601 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14659c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe148142f00 0x7fe148069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe146d9d700 1 -- 172.21.15.131:0/3478074514 learned_addr learned my addr 172.21.15.131:0/3478074514 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:05.602 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe145d9b700 1 --2- 172.21.15.131:0/3478074514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1480cc590 0x7fe1480cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14559a700 1 -- 172.21.15.131:0/3478074514 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727162775 0 0) 0x7fe148069640 con 0x7fe1480cb340 2023-12-26T23:13:05.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14559a700 1 -- 172.21.15.131:0/3478074514 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe130003570 con 0x7fe1480cb340 2023-12-26T23:13:05.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14659c700 1 -- 172.21.15.131:0/3478074514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1480cc590 msgr2=0x7fe1480cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:05.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14659c700 1 --2- 172.21.15.131:0/3478074514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1480cc590 0x7fe1480cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.603 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14659c700 1 -- 172.21.15.131:0/3478074514 >> v1:172.21.15.131:6789/0 conn(0x7fe1480cb340 legacy=0x7fe1480c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.601+0000 7fe14659c700 1 -- 172.21.15.131:0/3478074514 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe148068a30 con 0x7fe148142f00 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.602+0000 7fe14659c700 1 --2- 172.21.15.131:0/3478074514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 0x7fe148069ad0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fe1380098e0 tx=0x7fe1380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d2d04cd787f966 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.602+0000 7fe14559a700 1 -- 172.21.15.131:0/3478074514 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138011630 con 0x7fe148142f00 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.602+0000 7fe14559a700 1 -- 172.21.15.131:0/3478074514 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138011790 con 0x7fe148142f00 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.602+0000 7fe14559a700 1 -- 172.21.15.131:0/3478074514 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138011b30 con 0x7fe148142f00 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.602+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3478074514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 msgr2=0x7fe148069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.602+0000 7fe14c9ff700 1 --2- 172.21.15.131:0/3478074514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 0x7fe148069ad0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fe1380098e0 tx=0x7fe1380091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.603+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3478074514 shutdown_connections 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.603+0000 7fe14c9ff700 1 --2- 172.21.15.131:0/3478074514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 0x7fe148069ad0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.603+0000 7fe14c9ff700 1 --2- 172.21.15.131:0/3478074514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1480cc590 0x7fe1480cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.603+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3478074514 >> 172.21.15.131:0/3478074514 conn(0x7fe148063cd0 msgr2=0x7fe14806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.603+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3478074514 shutdown_connections 2023-12-26T23:13:05.604 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.603+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3478074514 wait complete. 2023-12-26T23:13:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.605+0000 7fe14c9ff700 1 Processor -- start 2023-12-26T23:13:05.605 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.605+0000 7fe14c9ff700 1 -- start start 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.606+0000 7fe14c9ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1480cb340 0x7fe1480c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.606+0000 7fe14c9ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1480cc590 0x7fe14813fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.606+0000 7fe14c9ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 0x7fe1480c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.606+0000 7fe14c9ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480c5a30 con 0x7fe148142f00 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.606+0000 7fe14c9ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14816d5d0 con 0x7fe1480cc590 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.606+0000 7fe14c9ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14816d8b0 con 0x7fe1480cb340 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.606+0000 7fe146d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 0x7fe1480c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.606+0000 7fe146d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 0x7fe1480c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47906/0 (socket says 172.21.15.131:47906) 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.606+0000 7fe146d9d700 1 -- 172.21.15.131:0/3655102508 learned_addr learned my addr 172.21.15.131:0/3655102508 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe14659c700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1480cb340 0x7fe1480c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe145d9b700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1480cc590 0x7fe14813fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.608 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe146d9d700 1 -- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1480cb340 msgr2=0x7fe1480c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe146d9d700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1480cb340 0x7fe1480c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe146d9d700 1 -- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1480cc590 msgr2=0x7fe14813fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe146d9d700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1480cc590 0x7fe14813fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe146d9d700 1 -- 172.21.15.131:0/3655102508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe138009420 con 0x7fe148142f00 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe145d9b700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1480cc590 0x7fe14813fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe146d9d700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 0x7fe1480c31a0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fe140001460 tx=0x7fe140001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe1367fc700 1 -- 172.21.15.131:0/3655102508 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14000a940 con 0x7fe148142f00 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe1367fc700 1 -- 172.21.15.131:0/3655102508 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14000aaa0 con 0x7fe148142f00 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14816db90 con 0x7fe148142f00 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.607+0000 7fe1367fc700 1 -- 172.21.15.131:0/3655102508 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14000ae20 con 0x7fe148142f00 2023-12-26T23:13:05.609 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.608+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1480a98b0 con 0x7fe148142f00 2023-12-26T23:13:05.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.608+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1480717f0 con 0x7fe148142f00 2023-12-26T23:13:05.610 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.609+0000 7fe1367fc700 1 -- 172.21.15.131:0/3655102508 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe140007300 con 0x7fe148142f00 2023-12-26T23:13:05.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.610+0000 7fe1367fc700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe128066f30 0x7fe1280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.610+0000 7fe1367fc700 1 -- 172.21.15.131:0/3655102508 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fe140084b40 con 0x7fe148142f00 2023-12-26T23:13:05.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.610+0000 7fe1367fc700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7fe12806d6d0 0x7fe12806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.610+0000 7fe14659c700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe128066f30 0x7fe1280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.610+0000 7fe1367fc700 1 -- 172.21.15.131:0/3655102508 --> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe128070170 con 0x7fe12806d6d0 2023-12-26T23:13:05.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.610+0000 7fe1367fc700 1 -- 172.21.15.131:0/3655102508 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=215) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe140086ea0 con 0x7fe148142f00 2023-12-26T23:13:05.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.611+0000 7fe145d9b700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7fe12806d6d0 0x7fe12806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.611 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.611+0000 7fe14659c700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe128066f30 0x7fe1280693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1380091e0 tx=0x7fe138017000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.611+0000 7fe145d9b700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7fe12806d6d0 0x7fe12806fb40 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.612 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.612+0000 7fe1367fc700 1 -- 172.21.15.131:0/3655102508 <== osd.6 v2:172.21.15.190:6816/40035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe128070170 con 0x7fe12806d6d0 2023-12-26T23:13:05.627 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.627+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 --> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe148068d10 con 0x7fe12806d6d0 2023-12-26T23:13:05.628 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe1367fc700 1 -- 172.21.15.131:0/3655102508 <== osd.6 v2:172.21.15.190:6816/40035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe148068d10 con 0x7fe12806d6d0 2023-12-26T23:13:05.628 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:13:05.628 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-26T23:13:05.628 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:13:05.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7fe12806d6d0 msgr2=0x7fe12806fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe14c9ff700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.190:6816/40035,v1:172.21.15.190:6819/40035] conn(0x7fe12806d6d0 0x7fe12806fb40 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe128066f30 msgr2=0x7fe1280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe14c9ff700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe128066f30 0x7fe1280693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1380091e0 tx=0x7fe138017000 comp rx=0 tx=0).stop 2023-12-26T23:13:05.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 msgr2=0x7fe1480c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe14c9ff700 1 --2- 172.21.15.131:0/3655102508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe148142f00 0x7fe1480c31a0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fe140001460 tx=0x7fe140001ae0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.629 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe146d9d700 1 -- 172.21.15.131:0/3655102508 reap_dead start 2023-12-26T23:13:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 shutdown_connections 2023-12-26T23:13:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.628+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 >> 172.21.15.131:0/3655102508 conn(0x7fe148063cd0 msgr2=0x7fe14806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.629+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 shutdown_connections 2023-12-26T23:13:05.630 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.629+0000 7fe14c9ff700 1 -- 172.21.15.131:0/3655102508 wait complete. 2023-12-26T23:13:05.645 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-12-26T23:13:05.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.835+0000 7fc9ebbff700 1 Processor -- start 2023-12-26T23:13:05.835 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.835+0000 7fc9ebbff700 1 -- start start 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.836+0000 7fc9ebbff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc9e4159820 0x7fc9e40abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.836+0000 7fc9ebbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e40aa480 0x7fc9e406dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.836+0000 7fc9ebbff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9e406e2b0 0x7fc9e415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.836+0000 7fc9ebbff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e415ad50 con 0x7fc9e4159820 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.836+0000 7fc9ebbff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e406e730 con 0x7fc9e406e2b0 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.836+0000 7fc9ebbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e415a8c0 con 0x7fc9e40aa480 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc9e4159820 0x7fc9e40abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e999b700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc9e4159820 0x7fc9e40abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47918/0 (socket says 172.21.15.131:47918) 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e999b700 1 -- 172.21.15.131:0/2304479315 learned_addr learned my addr 172.21.15.131:0/2304479315 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9ea19c700 1 --2- 172.21.15.131:0/2304479315 >> v2:172.21.15.190:3300/0 conn(0x7fc9e406e2b0 0x7fc9e415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.838 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e999b700 1 -- 172.21.15.131:0/2304479315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e40aa480 msgr2=0x7fc9e406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e919a700 1 --2- 172.21.15.131:0/2304479315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e40aa480 0x7fc9e406dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e999b700 1 --2- 172.21.15.131:0/2304479315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e40aa480 0x7fc9e406dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e999b700 1 -- 172.21.15.131:0/2304479315 >> v2:172.21.15.190:3300/0 conn(0x7fc9e406e2b0 msgr2=0x7fc9e415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e999b700 1 --2- 172.21.15.131:0/2304479315 >> v2:172.21.15.190:3300/0 conn(0x7fc9e406e2b0 0x7fc9e415d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e999b700 1 -- 172.21.15.131:0/2304479315 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e4159f90 con 0x7fc9e4159820 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9ea19c700 1 --2- 172.21.15.131:0/2304479315 >> v2:172.21.15.190:3300/0 conn(0x7fc9e406e2b0 0x7fc9e415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e919a700 1 --2- 172.21.15.131:0/2304479315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e40aa480 0x7fc9e406dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e999b700 1 --2- 172.21.15.131:0/2304479315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e4159820 0x7fc9e40abdb0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fc9e406cbd0 tx=0x7fc9e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9e672f32988ae9 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.837+0000 7fc9e8999700 1 -- 172.21.15.131:0/2304479315 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000b820 con 0x7fc9e4159820 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9e8999700 1 -- 172.21.15.131:0/2304479315 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e000b980 con 0x7fc9e4159820 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9e8999700 1 -- 172.21.15.131:0/2304479315 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000bd20 con 0x7fc9e4159820 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9ebbff700 1 -- 172.21.15.131:0/2304479315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e4159820 msgr2=0x7fc9e40abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.839 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/2304479315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e4159820 0x7fc9e40abdb0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fc9e406cbd0 tx=0x7fc9e0009250 comp rx=0 tx=0).stop 2023-12-26T23:13:05.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9ebbff700 1 -- 172.21.15.131:0/2304479315 shutdown_connections 2023-12-26T23:13:05.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/2304479315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e4159820 0x7fc9e40abdb0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/2304479315 >> v2:172.21.15.190:3300/0 conn(0x7fc9e406e2b0 0x7fc9e415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/2304479315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e40aa480 0x7fc9e406dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.840 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9ebbff700 1 -- 172.21.15.131:0/2304479315 >> 172.21.15.131:0/2304479315 conn(0x7fc9e406c290 msgr2=0x7fc9e40ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:05.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9ebbff700 1 -- 172.21.15.131:0/2304479315 shutdown_connections 2023-12-26T23:13:05.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.838+0000 7fc9ebbff700 1 -- 172.21.15.131:0/2304479315 wait complete. 2023-12-26T23:13:05.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.840+0000 7fc9ebbff700 1 Processor -- start 2023-12-26T23:13:05.841 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.840+0000 7fc9ebbff700 1 -- start start 2023-12-26T23:13:05.842 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9ebbff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e406e2b0 0x7fc9e40c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9ebbff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e40aa480 0x7fc9e40b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9ebbff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e4159820 0x7fc9e40af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9ebbff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40b1750 con 0x7fc9e406e2b0 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9ebbff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40b0170 con 0x7fc9e40aa480 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9ebbff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40b0450 con 0x7fc9e4159820 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9e999b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e406e2b0 0x7fc9e40c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9e999b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e406e2b0 0x7fc9e40c3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47920/0 (socket says 172.21.15.131:47920) 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9e999b700 1 -- 172.21.15.131:0/4275008421 learned_addr learned my addr 172.21.15.131:0/4275008421 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9e919a700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e40aa480 0x7fc9e40b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9ea19c700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e4159820 0x7fc9e40af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9e999b700 1 -- 172.21.15.131:0/4275008421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e4159820 msgr2=0x7fc9e40af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9e999b700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e4159820 0x7fc9e40af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9e999b700 1 -- 172.21.15.131:0/4275008421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e40aa480 msgr2=0x7fc9e40b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:05.843 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.841+0000 7fc9e999b700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e40aa480 0x7fc9e40b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:05.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.842+0000 7fc9e999b700 1 -- 172.21.15.131:0/4275008421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e00094f0 con 0x7fc9e406e2b0 2023-12-26T23:13:05.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.842+0000 7fc9e999b700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e406e2b0 0x7fc9e40c3580 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00094c0 tx=0x7fc9e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.842+0000 7fc9d9ffb700 1 -- 172.21.15.131:0/4275008421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0024440 con 0x7fc9e406e2b0 2023-12-26T23:13:05.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.842+0000 7fc9d9ffb700 1 -- 172.21.15.131:0/4275008421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e00245c0 con 0x7fc9e406e2b0 2023-12-26T23:13:05.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.842+0000 7fc9d9ffb700 1 -- 172.21.15.131:0/4275008421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0024980 con 0x7fc9e406e2b0 2023-12-26T23:13:05.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.842+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e40b0730 con 0x7fc9e406e2b0 2023-12-26T23:13:05.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.842+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e40b0cf0 con 0x7fc9e406e2b0 2023-12-26T23:13:05.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.843+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e40b1060 con 0x7fc9e406e2b0 2023-12-26T23:13:05.844 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.844+0000 7fc9d9ffb700 1 -- 172.21.15.131:0/4275008421 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc9e001a420 con 0x7fc9e406e2b0 2023-12-26T23:13:05.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.845+0000 7fc9d9ffb700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc9c8066f30 0x7fc9c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:05.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.845+0000 7fc9d9ffb700 1 -- 172.21.15.131:0/4275008421 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fc9e0086500 con 0x7fc9e406e2b0 2023-12-26T23:13:05.845 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.845+0000 7fc9e919a700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc9c8066f30 0x7fc9c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:05.846 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.846+0000 7fc9e919a700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc9c8066f30 0x7fc9c80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4006fd0 tx=0x7fc9d4008040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:05.850 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:05.850+0000 7fc9d9ffb700 1 -- 172.21.15.131:0/4275008421 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9e0053630 con 0x7fc9e406e2b0 2023-12-26T23:13:06.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.042+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9e40657e0 con 0x7fc9c8066f30 2023-12-26T23:13:06.042 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.042+0000 7fc9d9ffb700 1 -- 172.21.15.131:0/4275008421 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc9e40657e0 con 0x7fc9c8066f30 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc9c8066f30 msgr2=0x7fc9c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc9c8066f30 0x7fc9c80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4006fd0 tx=0x7fc9d4008040 comp rx=0 tx=0).stop 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e406e2b0 msgr2=0x7fc9e40c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e406e2b0 0x7fc9e40c3580 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00094c0 tx=0x7fc9e0017000 comp rx=0 tx=0).stop 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 shutdown_connections 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fc9c8066f30 0x7fc9c80693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e4159820 0x7fc9e40af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9e406e2b0 0x7fc9e40c3580 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 --2- 172.21.15.131:0/4275008421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e40aa480 0x7fc9e40b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.044+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 >> 172.21.15.131:0/4275008421 conn(0x7fc9e406c290 msgr2=0x7fc9e41568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.045+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 shutdown_connections 2023-12-26T23:13:06.045 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.045+0000 7fc9ebbff700 1 -- 172.21.15.131:0/4275008421 wait complete. 2023-12-26T23:13:06.046 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 6 to deep-scrub 2023-12-26T23:13:06.062 INFO:tasks.ceph:Scrubbing osd.7 2023-12-26T23:13:06.062 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-12-26T23:13:06.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.255+0000 7f8201fff700 1 Processor -- start 2023-12-26T23:13:06.255 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.255+0000 7f8201fff700 1 -- start start 2023-12-26T23:13:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.256+0000 7f8201fff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81fc159810 0x7f81fc0c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.256+0000 7f8201fff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc0cbb00 con 0x7f81fc0cec80 2023-12-26T23:13:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.256+0000 7f8201fff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc0ce040 con 0x7f81fc0cd7b0 2023-12-26T23:13:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.256+0000 7f8201fff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc0cb2a0 con 0x7f81fc159810 2023-12-26T23:13:06.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.256+0000 7f81fb7fe700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81fc159810 0x7f81fc0c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.256+0000 7f81faffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f81fc0cec80 0x7f81fc0cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46698/0 (socket says 172.21.15.131:46698) 2023-12-26T23:13:06.257 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.256+0000 7f81faffd700 1 -- 172.21.15.131:0/3389121637 learned_addr learned my addr 172.21.15.131:0/3389121637 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:06.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.257+0000 7f81fa7fc700 1 -- 172.21.15.131:0/3389121637 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411932247 0 0) 0x7f81fc0cbb00 con 0x7f81fc0cec80 2023-12-26T23:13:06.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.257+0000 7f81fa7fc700 1 -- 172.21.15.131:0/3389121637 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81e4003570 con 0x7f81fc0cec80 2023-12-26T23:13:06.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.257+0000 7f81fb7fe700 1 -- 172.21.15.131:0/3389121637 >> v1:172.21.15.190:6789/0 conn(0x7f81fc0cd7b0 legacy=0x7f81fc0cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:06.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.257+0000 7f81fb7fe700 1 -- 172.21.15.131:0/3389121637 >> v1:172.21.15.131:6789/0 conn(0x7f81fc0cec80 legacy=0x7f81fc0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.257+0000 7f81fb7fe700 1 -- 172.21.15.131:0/3389121637 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc159be0 con 0x7f81fc159810 2023-12-26T23:13:06.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.257+0000 7f81fb7fe700 1 --2- 172.21.15.131:0/3389121637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 0x7f81fc0c5440 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0098d0 tx=0x7f81ec009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86520167ca6eb3bd server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.258 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.257+0000 7f81fa7fc700 1 -- 172.21.15.131:0/3389121637 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec005c10 con 0x7f81fc159810 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.257+0000 7f81fa7fc700 1 -- 172.21.15.131:0/3389121637 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec005d70 con 0x7f81fc159810 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.257+0000 7f81fa7fc700 1 -- 172.21.15.131:0/3389121637 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec006280 con 0x7f81fc159810 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.258+0000 7f8201fff700 1 -- 172.21.15.131:0/3389121637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 msgr2=0x7f81fc0c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.258+0000 7f8201fff700 1 --2- 172.21.15.131:0/3389121637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 0x7f81fc0c5440 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0098d0 tx=0x7f81ec009120 comp rx=0 tx=0).stop 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.258+0000 7f8201fff700 1 -- 172.21.15.131:0/3389121637 shutdown_connections 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.258+0000 7f8201fff700 1 --2- 172.21.15.131:0/3389121637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 0x7f81fc0c5440 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.258+0000 7f8201fff700 1 -- 172.21.15.131:0/3389121637 >> 172.21.15.131:0/3389121637 conn(0x7f81fc099950 msgr2=0x7f81fc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.258+0000 7f8201fff700 1 -- 172.21.15.131:0/3389121637 shutdown_connections 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.258+0000 7f8201fff700 1 -- 172.21.15.131:0/3389121637 wait complete. 2023-12-26T23:13:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.259+0000 7f8201fff700 1 Processor -- start 2023-12-26T23:13:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.260+0000 7f8201fff700 1 -- start start 2023-12-26T23:13:06.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.260+0000 7f8201fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81fc0cd7b0 0x7f81fc0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.260+0000 7f8201fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81fc0cec80 0x7f81fc0ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.260+0000 7f8201fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 0x7f81fc169890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.260+0000 7f8201fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc15d4f0 con 0x7f81fc159810 2023-12-26T23:13:06.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.260+0000 7f8201fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0c99c0 con 0x7f81fc0cec80 2023-12-26T23:13:06.262 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.260+0000 7f8201fff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0c8ff0 con 0x7f81fc0cd7b0 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 0x7f81fc169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 0x7f81fc169890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47932/0 (socket says 172.21.15.131:47932) 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fbfff700 1 -- 172.21.15.131:0/4091104384 learned_addr learned my addr 172.21.15.131:0/4091104384 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fb7fe700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81fc0cd7b0 0x7f81fc0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81faffd700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81fc0cec80 0x7f81fc0ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fbfff700 1 -- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81fc0cd7b0 msgr2=0x7f81fc0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fbfff700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81fc0cd7b0 0x7f81fc0c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fbfff700 1 -- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81fc0cec80 msgr2=0x7f81fc0ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fbfff700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81fc0cec80 0x7f81fc0ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fbfff700 1 -- 172.21.15.131:0/4091104384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0093c0 con 0x7f81fc159810 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.261+0000 7f81fbfff700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 0x7f81fc169890 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c930 tx=0x7f81f40026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.262+0000 7f81e37fe700 1 -- 172.21.15.131:0/4091104384 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4012630 con 0x7f81fc159810 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.262+0000 7f81e37fe700 1 -- 172.21.15.131:0/4091104384 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f40127d0 con 0x7f81fc159810 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.262+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc0c83e0 con 0x7f81fc159810 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.262+0000 7f81e37fe700 1 -- 172.21.15.131:0/4091104384 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4012b90 con 0x7f81fc159810 2023-12-26T23:13:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.262+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc0c89a0 con 0x7f81fc159810 2023-12-26T23:13:06.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.263+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f81fc0717f0 con 0x7f81fc159810 2023-12-26T23:13:06.264 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.264+0000 7f81e37fe700 1 -- 172.21.15.131:0/4091104384 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f81f401a3b0 con 0x7f81fc159810 2023-12-26T23:13:06.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.264+0000 7f81e37fe700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f81e8066f30 0x7f81e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.265+0000 7f81e37fe700 1 -- 172.21.15.131:0/4091104384 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f81f4084450 con 0x7f81fc159810 2023-12-26T23:13:06.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.265+0000 7f81fb7fe700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f81e8066f30 0x7f81e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.265+0000 7f81e37fe700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7f81e806d6d0 0x7f81e806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.265+0000 7f81e37fe700 1 -- 172.21.15.131:0/4091104384 --> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81e8070170 con 0x7f81e806d6d0 2023-12-26T23:13:06.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.265+0000 7f81e37fe700 1 -- 172.21.15.131:0/4091104384 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=215) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81f40869b0 con 0x7f81fc159810 2023-12-26T23:13:06.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.265+0000 7f81fb7fe700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f81e8066f30 0x7f81e80693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009390 tx=0x7f81ec015000 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.265 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.265+0000 7f81faffd700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7f81e806d6d0 0x7f81e806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.266 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.266+0000 7f81faffd700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7f81e806d6d0 0x7f81e806fb40 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.267 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.267+0000 7f81e37fe700 1 -- 172.21.15.131:0/4091104384 <== osd.7 v2:172.21.15.190:6800/40011 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f81e8070170 con 0x7f81e806d6d0 2023-12-26T23:13:06.281 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.281+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 --> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f81fc0c8ce0 con 0x7f81e806d6d0 2023-12-26T23:13:06.282 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f81e37fe700 1 -- 172.21.15.131:0/4091104384 <== osd.7 v2:172.21.15.190:6800/40011 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f81fc0c8ce0 con 0x7f81e806d6d0 2023-12-26T23:13:06.282 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2023-12-26T23:13:06.282 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-26T23:13:06.282 INFO:teuthology.orchestra.run.smithi131.stdout:} 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7f81e806d6d0 msgr2=0x7f81e806fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.190:6800/40011,v1:172.21.15.190:6802/40011] conn(0x7f81e806d6d0 0x7f81e806fb40 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f81e8066f30 msgr2=0x7f81e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f81e8066f30 0x7f81e80693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009390 tx=0x7f81ec015000 comp rx=0 tx=0).stop 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 msgr2=0x7f81fc169890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 --2- 172.21.15.131:0/4091104384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81fc159810 0x7f81fc169890 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c930 tx=0x7f81f40026e0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f81fbfff700 1 -- 172.21.15.131:0/4091104384 reap_dead start 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 shutdown_connections 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 >> 172.21.15.131:0/4091104384 conn(0x7f81fc099950 msgr2=0x7f81fc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 shutdown_connections 2023-12-26T23:13:06.284 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.282+0000 7f8201fff700 1 -- 172.21.15.131:0/4091104384 wait complete. 2023-12-26T23:13:06.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-12-26T23:13:06.490 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.490+0000 7f09c53ff700 1 Processor -- start 2023-12-26T23:13:06.491 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.491+0000 7f09c53ff700 1 -- start start 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09c53ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09c0159a70 0x7f09c00b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09c53ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09c00b2a90 0x7f09c00b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09c53ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c00afcd0 con 0x7f09c01596a0 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09c53ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c00b3e70 con 0x7f09c0159a70 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09c53ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c00b0f20 con 0x7f09c00b2a90 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09be7fc700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09c0159a70 0x7f09c00b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09beffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f09c01596a0 0x7f09c00b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46710/0 (socket says 172.21.15.131:46710) 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09beffd700 1 -- 172.21.15.131:0/826855793 learned_addr learned my addr 172.21.15.131:0/826855793 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09bf7fe700 1 --2- 172.21.15.131:0/826855793 >> v2:172.21.15.190:3300/0 conn(0x7f09c00b2a90 0x7f09c00b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09be7fc700 1 -- 172.21.15.131:0/826855793 >> v2:172.21.15.190:3300/0 conn(0x7f09c00b2a90 msgr2=0x7f09c00b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09be7fc700 1 --2- 172.21.15.131:0/826855793 >> v2:172.21.15.190:3300/0 conn(0x7f09c00b2a90 0x7f09c00b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09be7fc700 1 -- 172.21.15.131:0/826855793 >> v1:172.21.15.131:6789/0 conn(0x7f09c01596a0 legacy=0x7f09c00b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.492+0000 7f09be7fc700 1 -- 172.21.15.131:0/826855793 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00b3540 con 0x7f09c0159a70 2023-12-26T23:13:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.493+0000 7f09be7fc700 1 --2- 172.21.15.131:0/826855793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159a70 0x7f09c00b0160 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f09c00b66b0 tx=0x7f09b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8d46efd623b157 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.493+0000 7f09bdffb700 1 -- 172.21.15.131:0/826855793 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b400b6b0 con 0x7f09c0159a70 2023-12-26T23:13:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.493+0000 7f09bdffb700 1 -- 172.21.15.131:0/826855793 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b400b810 con 0x7f09c0159a70 2023-12-26T23:13:06.494 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.493+0000 7f09bdffb700 1 -- 172.21.15.131:0/826855793 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b400bbb0 con 0x7f09c0159a70 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.493+0000 7f09c53ff700 1 -- 172.21.15.131:0/826855793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159a70 msgr2=0x7f09c00b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.493+0000 7f09c53ff700 1 --2- 172.21.15.131:0/826855793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159a70 0x7f09c00b0160 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f09c00b66b0 tx=0x7f09b40091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.494+0000 7f09c53ff700 1 -- 172.21.15.131:0/826855793 shutdown_connections 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.494+0000 7f09c53ff700 1 --2- 172.21.15.131:0/826855793 >> v2:172.21.15.190:3300/0 conn(0x7f09c00b2a90 0x7f09c00b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.494+0000 7f09c53ff700 1 --2- 172.21.15.131:0/826855793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c0159a70 0x7f09c00b0160 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.494+0000 7f09c53ff700 1 -- 172.21.15.131:0/826855793 >> 172.21.15.131:0/826855793 conn(0x7f09c006ece0 msgr2=0x7f09c006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.494+0000 7f09c53ff700 1 -- 172.21.15.131:0/826855793 shutdown_connections 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.494+0000 7f09c53ff700 1 -- 172.21.15.131:0/826855793 wait complete. 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.495+0000 7f09c53ff700 1 Processor -- start 2023-12-26T23:13:06.495 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.495+0000 7f09c53ff700 1 -- start start 2023-12-26T23:13:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.496+0000 7f09c53ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c00b2a90 0x7f09c015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.496+0000 7f09c53ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c01596a0 0x7f09c00c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.496+0000 7f09c53ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c0159a70 0x7f09c00c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.496+0000 7f09c53ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00c6fd0 con 0x7f09c01596a0 2023-12-26T23:13:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.496+0000 7f09c53ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c015ac10 con 0x7f09c00b2a90 2023-12-26T23:13:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.496+0000 7f09c53ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c015afb0 con 0x7f09c0159a70 2023-12-26T23:13:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.496+0000 7f09be7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c01596a0 0x7f09c00c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.496+0000 7f09be7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c01596a0 0x7f09c00c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47958/0 (socket says 172.21.15.131:47958) 2023-12-26T23:13:06.497 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.496+0000 7f09be7fc700 1 -- 172.21.15.131:0/812296492 learned_addr learned my addr 172.21.15.131:0/812296492 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:06.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09beffd700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c00b2a90 0x7f09c015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09bf7fe700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c0159a70 0x7f09c00c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09be7fc700 1 -- 172.21.15.131:0/812296492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c0159a70 msgr2=0x7f09c00c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09be7fc700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c0159a70 0x7f09c00c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09be7fc700 1 -- 172.21.15.131:0/812296492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c00b2a90 msgr2=0x7f09c015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09be7fc700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c00b2a90 0x7f09c015a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09be7fc700 1 -- 172.21.15.131:0/812296492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b4009420 con 0x7f09c01596a0 2023-12-26T23:13:06.498 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09bf7fe700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c0159a70 0x7f09c00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-26T23:13:06.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09be7fc700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c01596a0 0x7f09c00c5f50 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f09c0089710 tx=0x7f09b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09a6ffd700 1 -- 172.21.15.131:0/812296492 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4011420 con 0x7f09c01596a0 2023-12-26T23:13:06.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09a6ffd700 1 -- 172.21.15.131:0/812296492 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b4011580 con 0x7f09c01596a0 2023-12-26T23:13:06.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c00c6490 con 0x7f09c01596a0 2023-12-26T23:13:06.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.497+0000 7f09a6ffd700 1 -- 172.21.15.131:0/812296492 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4011940 con 0x7f09c01596a0 2023-12-26T23:13:06.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.498+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c00c6a10 con 0x7f09c01596a0 2023-12-26T23:13:06.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.499+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c015b120 con 0x7f09c01596a0 2023-12-26T23:13:06.499 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.499+0000 7f09a6ffd700 1 -- 172.21.15.131:0/812296492 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f09b4011cf0 con 0x7f09c01596a0 2023-12-26T23:13:06.500 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.500+0000 7f09a6ffd700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f09ac066f30 0x7f09ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.500+0000 7f09a6ffd700 1 -- 172.21.15.131:0/812296492 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f09b40860c0 con 0x7f09c01596a0 2023-12-26T23:13:06.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.501+0000 7f09beffd700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f09ac066f30 0x7f09ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.501 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.501+0000 7f09beffd700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f09ac066f30 0x7f09ac0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f09b0001820 tx=0x7f09b0009040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.504 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.505+0000 7f09a6ffd700 1 -- 172.21.15.131:0/812296492 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09b4053150 con 0x7f09c01596a0 2023-12-26T23:13:06.694 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.694+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f09c00657e0 con 0x7f09ac066f30 2023-12-26T23:13:06.695 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.694+0000 7f09a6ffd700 1 -- 172.21.15.131:0/812296492 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f09c00657e0 con 0x7f09ac066f30 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f09ac066f30 msgr2=0x7f09ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f09ac066f30 0x7f09ac0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f09b0001820 tx=0x7f09b0009040 comp rx=0 tx=0).stop 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c01596a0 msgr2=0x7f09c00c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c01596a0 0x7f09c00c5f50 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f09c0089710 tx=0x7f09b4017000 comp rx=0 tx=0).stop 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 shutdown_connections 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f09ac066f30 0x7f09ac0693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c0159a70 0x7f09c00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09c00b2a90 0x7f09c015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 --2- 172.21.15.131:0/812296492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c01596a0 0x7f09c00c5f50 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 >> 172.21.15.131:0/812296492 conn(0x7f09c006ece0 msgr2=0x7f09c00cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 shutdown_connections 2023-12-26T23:13:06.698 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.697+0000 7f09c53ff700 1 -- 172.21.15.131:0/812296492 wait complete. 2023-12-26T23:13:06.699 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 7 to deep-scrub 2023-12-26T23:13:06.716 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:13:06.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.905+0000 7fe5b85ff700 1 Processor -- start 2023-12-26T23:13:06.905 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.905+0000 7fe5b85ff700 1 -- start start 2023-12-26T23:13:06.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.906+0000 7fe5b85ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe5b013c510 0x7fe5b00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.906+0000 7fe5b85ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe5b00cc810 0x7fe5b006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.906+0000 7fe5b85ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b00686c0 con 0x7fe5b006b8c0 2023-12-26T23:13:06.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.906+0000 7fe5b85ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b006c280 con 0x7fe5b00cc810 2023-12-26T23:13:06.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.906+0000 7fe5b85ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b0069520 con 0x7fe5b013c510 2023-12-26T23:13:06.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.906+0000 7fe5b6b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe5b00cc810 0x7fe5b006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.907+0000 7fe5b6b9c700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe5b00cc810 0x7fe5b006bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47972/0 (socket says 172.21.15.131:47972) 2023-12-26T23:13:06.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.907+0000 7fe5b6b9c700 1 -- 172.21.15.131:0/1707290892 learned_addr learned my addr 172.21.15.131:0/1707290892 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:06.907 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.907+0000 7fe5b639b700 1 --2- 172.21.15.131:0/1707290892 >> v2:172.21.15.190:3300/0 conn(0x7fe5b013c510 0x7fe5b00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.907+0000 7fe5b6b9c700 1 -- 172.21.15.131:0/1707290892 >> v2:172.21.15.190:3300/0 conn(0x7fe5b013c510 msgr2=0x7fe5b00c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.907+0000 7fe5b6b9c700 1 --2- 172.21.15.131:0/1707290892 >> v2:172.21.15.190:3300/0 conn(0x7fe5b013c510 0x7fe5b00c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.907+0000 7fe5b6b9c700 1 -- 172.21.15.131:0/1707290892 >> v1:172.21.15.190:6789/0 conn(0x7fe5b006b8c0 legacy=0x7fe5b00cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.907+0000 7fe5b6b9c700 1 -- 172.21.15.131:0/1707290892 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b0068910 con 0x7fe5b00cc810 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.907+0000 7fe5b6b9c700 1 --2- 172.21.15.131:0/1707290892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b00cc810 0x7fe5b006bd40 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fe5b006fcc0 tx=0x7fe5a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ee752e12e7d433 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.907+0000 7fe5b5399700 1 -- 172.21.15.131:0/1707290892 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800e070 con 0x7fe5b00cc810 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b5399700 1 -- 172.21.15.131:0/1707290892 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a800ba20 con 0x7fe5b00cc810 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b5399700 1 -- 172.21.15.131:0/1707290892 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80092e0 con 0x7fe5b00cc810 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b85ff700 1 -- 172.21.15.131:0/1707290892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b00cc810 msgr2=0x7fe5b006bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b85ff700 1 --2- 172.21.15.131:0/1707290892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b00cc810 0x7fe5b006bd40 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fe5b006fcc0 tx=0x7fe5a8009840 comp rx=0 tx=0).stop 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b85ff700 1 -- 172.21.15.131:0/1707290892 shutdown_connections 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b85ff700 1 --2- 172.21.15.131:0/1707290892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b00cc810 0x7fe5b006bd40 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b85ff700 1 --2- 172.21.15.131:0/1707290892 >> v2:172.21.15.190:3300/0 conn(0x7fe5b013c510 0x7fe5b00c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b85ff700 1 -- 172.21.15.131:0/1707290892 >> 172.21.15.131:0/1707290892 conn(0x7fe5b00ce3e0 msgr2=0x7fe5b00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b85ff700 1 -- 172.21.15.131:0/1707290892 shutdown_connections 2023-12-26T23:13:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.908+0000 7fe5b85ff700 1 -- 172.21.15.131:0/1707290892 wait complete. 2023-12-26T23:13:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.910+0000 7fe5b85ff700 1 Processor -- start 2023-12-26T23:13:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.910+0000 7fe5b85ff700 1 -- start start 2023-12-26T23:13:06.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.910+0000 7fe5b85ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b006b8c0 0x7fe5b013be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.910+0000 7fe5b85ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b00cc810 0x7fe5b015fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.911 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b85ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b013c510 0x7fe5b0159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b85ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b013c3b0 con 0x7fe5b013c510 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b85ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00b31f0 con 0x7fe5b006b8c0 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b85ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00b25e0 con 0x7fe5b00cc810 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b6b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b013c510 0x7fe5b0159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b6b9c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b013c510 0x7fe5b0159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47982/0 (socket says 172.21.15.131:47982) 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b6b9c700 1 -- 172.21.15.131:0/2617024798 learned_addr learned my addr 172.21.15.131:0/2617024798 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b639b700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b006b8c0 0x7fe5b013be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b5b9a700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b00cc810 0x7fe5b015fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b6b9c700 1 -- 172.21.15.131:0/2617024798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b00cc810 msgr2=0x7fe5b015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b6b9c700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b00cc810 0x7fe5b015fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b6b9c700 1 -- 172.21.15.131:0/2617024798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b006b8c0 msgr2=0x7fe5b013be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b6b9c700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b006b8c0 0x7fe5b013be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b6b9c700 1 -- 172.21.15.131:0/2617024798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a8009ae0 con 0x7fe5b013c510 2023-12-26T23:13:06.912 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.911+0000 7fe5b6b9c700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b013c510 0x7fe5b0159d20 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fe5b006fe40 tx=0x7fe5a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.912+0000 7fe5a67fc700 1 -- 172.21.15.131:0/2617024798 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8008a80 con 0x7fe5b013c510 2023-12-26T23:13:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.912+0000 7fe5a67fc700 1 -- 172.21.15.131:0/2617024798 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a8008be0 con 0x7fe5b013c510 2023-12-26T23:13:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.912+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b00b28c0 con 0x7fe5b013c510 2023-12-26T23:13:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.912+0000 7fe5a67fc700 1 -- 172.21.15.131:0/2617024798 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8026460 con 0x7fe5b013c510 2023-12-26T23:13:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.912+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b00b2e80 con 0x7fe5b013c510 2023-12-26T23:13:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.913+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b00657e0 con 0x7fe5b013c510 2023-12-26T23:13:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.914+0000 7fe5a67fc700 1 -- 172.21.15.131:0/2617024798 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe5a8003040 con 0x7fe5b013c510 2023-12-26T23:13:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.915+0000 7fe5a67fc700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe594066f30 0x7fe5940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.915+0000 7fe5a67fc700 1 -- 172.21.15.131:0/2617024798 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fe5a8086520 con 0x7fe5b013c510 2023-12-26T23:13:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.915+0000 7fe5b639b700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe594066f30 0x7fe5940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:06.915 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.915+0000 7fe5b639b700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe594066f30 0x7fe5940693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009da0 tx=0x7fe5ac009120 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:06.919 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:06.919+0000 7fe5a67fc700 1 -- 172.21.15.131:0/2617024798 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5a80535b0 con 0x7fe5b013c510 2023-12-26T23:13:07.110 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.110+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5b0068bf0 con 0x7fe594066f30 2023-12-26T23:13:07.112 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.112+0000 7fe5a67fc700 1 -- 172.21.15.131:0/2617024798 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50621 (secure 0 0 0) 0x7fe5b0068bf0 con 0x7fe594066f30 2023-12-26T23:13:07.113 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe594066f30 msgr2=0x7fe5940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe594066f30 0x7fe5940693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009da0 tx=0x7fe5ac009120 comp rx=0 tx=0).stop 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b013c510 msgr2=0x7fe5b0159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b013c510 0x7fe5b0159d20 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fe5b006fe40 tx=0x7fe5a8018000 comp rx=0 tx=0).stop 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 shutdown_connections 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fe594066f30 0x7fe5940693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b013c510 0x7fe5b0159d20 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b006b8c0 0x7fe5b013be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 --2- 172.21.15.131:0/2617024798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b00cc810 0x7fe5b015fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.114+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 >> 172.21.15.131:0/2617024798 conn(0x7fe5b00ce3e0 msgr2=0x7fe5b00c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.115+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 shutdown_connections 2023-12-26T23:13:07.115 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:07.115+0000 7fe5b85ff700 1 -- 172.21.15.131:0/2617024798 wait complete. 2023-12-26T23:13:07.116 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:13:07.133 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-12-26T23:13:05.245049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2461928,"kb_used_data":22184,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752512792,"statfs":{"total":773094113280,"available":770573099008,"internally_reserved":0,"allocated":22716416,"data_stored":12028816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001864"},"pg_stats":[{"pgid":"1.0","version":"213'370","reported_seq":1236,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-26T23:12:55.797881+0000","last_change":"2023-12-26T21:36:38.969614+0000","last_active":"2023-12-26T23:12:55.797881+0000","last_peered":"2023-12-26T23:12:55.797881+0000","last_clean":"2023-12-26T23:12:55.797881+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:12:55.797881+0000","last_undegraded":"2023-12-26T23:12:55.797881+0000","last_fullsized":"2023-12-26T23:12:55.797881+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-26T21:36:37.567326+0000","last_clean_scrub_stamp":"2023-12-26T21:36:37.567326+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.98199999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52100000000000002}]},{"osd":1,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.498,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.72199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.0449999999999999},"last":0.55200000000000005}]},{"osd":2,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.94899999999999995,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.495},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0760000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.44500000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.77100000000000002}]},{"osd":6,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.5,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.59699999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":1.028,"15min":1.028},"last":0.78100000000000003}]}]},{"osd":6,"up_from":15,"seq":64424510582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.32800000000000001},"max":{"1min":0.873,"5min":0.998,"15min":1.1080000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73499999999999999,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0680000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":1.022,"15min":1.046},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.95999999999999996,"15min":1.0580000000000001},"last":0.627}]},{"osd":2,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91500000000000004,"15min":1.071},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":0.89800000000000002,"5min":0.97899999999999998,"15min":1.0680000000000001},"last":0.82499999999999996}]},{"osd":3,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":0.96699999999999997,"15min":1.024},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":1.0149999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":1.018,"15min":1.1599999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.84599999999999997,"5min":0.92100000000000004,"15min":1.111},"last":0.877},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.71799999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.92000000000000004}]}]},{"osd":1,"up_from":15,"seq":64424510582,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300496,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071344,"statfs":{"total":96636764160,"available":96329056256,"internally_reserved":0,"allocated":3284992,"data_stored":1946378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.497},{"interface":"front","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.39000000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.33900000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.84099999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.272,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.441}]},{"osd":4,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.53600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.103},"last":0.749},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":0.95499999999999996,"5min":1.258,"15min":1.258},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.57899999999999996,"5min":0.56000000000000005,"15min":0.51300000000000001},"max":{"1min":1.0329999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82099999999999995,"15min":0.82999999999999996},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.85599999999999998}]},{"osd":6,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":0.82799999999999996},"min":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0609999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.83399999999999996,"15min":0.84099999999999997},"min":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.091,"5min":1.1559999999999999,"15min":1.2090000000000001},"last":0.872}]},{"osd":7,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.82299999999999995},"min":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.218},"last":0.88500000000000001}]}]},{"osd":0,"up_from":15,"seq":64424510582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308108,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063732,"statfs":{"total":96636764160,"available":96321261568,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.27600000000000002},"max":{"1min":0.60699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.372},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82999999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.91800000000000004,"15min":1.012},"last":0.41199999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.2250000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.49299999999999999},"max":{"1min":1.02,"5min":1.0649999999999999,"15min":1.113},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.157},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.137,"5min":1.1459999999999999,"15min":1.282},"last":0.63}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.085,"5min":1.121,"15min":1.151},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79600000000000004,"15min":0.80300000000000005},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93200000000000005,"5min":1.167,"15min":1.167},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0640000000000001,"5min":1.1439999999999999,"15min":1.256},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.81200000000000006},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54000000000000004},"max":{"1min":1.042,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.69699999999999995}]}]},{"osd":2,"up_from":14,"seq":60129543287,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64100000000000001,"5min":0.67800000000000005,"15min":0.84599999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.46300000000000002}]},{"osd":1,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64700000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.624,"15min":0.625},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.73399999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75,"5min":0.89300000000000002,"15min":1.056},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77600000000000002,"15min":0.79500000000000004},"min":{"1min":0.56299999999999994,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83299999999999996,"5min":1.8,"15min":1.8},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.085,"5min":1.222,"15min":1.222},"last":0.66000000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.79600000000000004},"min":{"1min":0.60099999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1020000000000001,"5min":1.923,"15min":1.923},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95599999999999996,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.69099999999999995}]},{"osd":6,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89400000000000002,"5min":0.97799999999999998,"15min":1.272},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.69599999999999995,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0449999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.77600000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.81299999999999994},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1619999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.65600000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.175,"5min":1.841,"15min":1.841},"last":0.67800000000000005}]}]},{"osd":3,"up_from":15,"seq":64424510583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":304012,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067828,"statfs":{"total":96636764160,"available":96325455872,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.48699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56999999999999995,"5min":0.746,"15min":0.76500000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47099999999999997,"15min":0.47799999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.753},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.34100000000000003},"max":{"1min":0.69399999999999995,"5min":0.77100000000000002,"15min":0.90400000000000003},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.628,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.0309999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.113,"5min":1.159,"15min":1.177},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.93400000000000005,"5min":0.999,"15min":1.081},"last":0.73799999999999999}]},{"osd":5,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.58399999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.028,"5min":1.046,"15min":1.0980000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.79900000000000004},"min":{"1min":0.53500000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.751}]},{"osd":6,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.61399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.999,"5min":1.202,"15min":1.202},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":0.84399999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.177,"15min":1.177},"last":0.91900000000000004}]},{"osd":7,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.53300000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.214},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0549999999999999,"5min":1.292,"15min":1.292},"last":0.77300000000000002}]}]},{"osd":4,"up_from":14,"seq":60129543286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.70199999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.877,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.746},{"interface":"front","average":{"1min":0.622,"5min":0.68400000000000005,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":0.94599999999999995,"15min":1.008},"last":0.65900000000000003}]},{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.437,"15min":0.437},"max":{"1min":0.84399999999999997,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85299999999999998,"5min":0.89000000000000001,"15min":1.0329999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82699999999999996,"5min":0.92900000000000005,"15min":0.998},"last":0.68200000000000005}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.52300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72199999999999998,"15min":0.75600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81100000000000005,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.76100000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.75,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.36799999999999999},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":0.98999999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"max":{"1min":0.81899999999999995,"5min":1.002,"15min":1.024},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86599999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.54400000000000004,"5min":0.53700000000000003,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.79200000000000004}]}]},{"osd":5,"up_from":15,"seq":64424510581,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300496,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071344,"statfs":{"total":96636764160,"available":96329056256,"internally_reserved":0,"allocated":3284992,"data_stored":1946378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.60799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.40899999999999997},"max":{"1min":0.91800000000000004,"5min":0.92700000000000005,"15min":1.0629999999999999},"last":0.77200000000000002}]},{"osd":1,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.016},"last":0.83799999999999997}]},{"osd":2,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.012,"15min":1.022},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.99199999999999999},"last":0.74399999999999999}]},{"osd":3,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":0.997,"15min":0.997},"last":0.79600000000000004}]},{"osd":4,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93000000000000005,"15min":0.95199999999999996},"last":0.85899999999999999}]},{"osd":6,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":0.97799999999999998,"15min":1.042},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.90600000000000003,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.81599999999999995}]},{"osd":7,"last update":"Tue Dec 26 23:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68999999999999995,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.90500000000000003,"15min":0.996},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.90000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:13:07.136 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-12-26T21:36:37.567326+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=26, tm_hour=21, tm_min=36, tm_sec=37, tm_wday=1, tm_yday=360, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=26, tm_hour=23, tm_min=13, tm_sec=0, tm_wday=1, tm_yday=360, tm_isdst=0) 2023-12-26T23:13:07.136 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-12-26T23:13:16.566 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:13:16.573 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:13:27.137 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-26T23:13:27.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.340+0000 7f1de13ff700 1 Processor -- start 2023-12-26T23:13:27.339 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.340+0000 7f1de13ff700 1 -- start start 2023-12-26T23:13:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.341+0000 7f1de13ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ddc0b19f0 0x7f1ddc1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.341+0000 7f1de13ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ddc0caa10 con 0x7f1ddc0c8bf0 2023-12-26T23:13:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.341+0000 7f1de13ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ddc157a80 con 0x7f1ddc0c8820 2023-12-26T23:13:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.341+0000 7f1de13ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0cac50 con 0x7f1ddc0b19f0 2023-12-26T23:13:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.341+0000 7f1dda7fc700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1ddc0c8bf0 0x7f1ddc0b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49284/0 (socket says 172.21.15.131:49284) 2023-12-26T23:13:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.341+0000 7f1dda7fc700 1 -- 172.21.15.131:0/816451853 learned_addr learned my addr 172.21.15.131:0/816451853 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1ddb7fe700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ddc0b19f0 0x7f1ddc1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430630334 0 0) 0x7f1dd0001110 con 0x7f1ddc0c8bf0 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc4003570 con 0x7f1ddc0c8bf0 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 938688791 0 0) 0x7f1dc4003570 con 0x7f1ddc0c8bf0 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ddc0b19f0 msgr2=0x7f1ddc1620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ddc0b19f0 0x7f1ddc1620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 >> v1:172.21.15.190:6789/0 conn(0x7f1ddc0c8820 legacy=0x7f1ddc0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc0caf30 con 0x7f1ddc0c8bf0 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3666430693 0 0) 0x7f1dd0001600 con 0x7f1ddc0c8bf0 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 >> v1:172.21.15.131:6789/0 conn(0x7f1ddc0c8bf0 legacy=0x7f1ddc0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc4003a90 0x7f1dc4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ddc0b19f0 0x7f1dc4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc4004d50 0x7f1dc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0cac50 con 0x7f1dc4004d50 2023-12-26T23:13:27.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc400bbf0 con 0x7f1dc4003a90 2023-12-26T23:13:27.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc400bdb0 con 0x7f1ddc0b19f0 2023-12-26T23:13:27.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1ddaffd700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc4003a90 0x7f1dc4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.342+0000 7f1ddb7fe700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc4004d50 0x7f1dc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1dda7fc700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ddc0b19f0 0x7f1dc4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.346 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1ddb7fe700 1 -- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ddc0b19f0 msgr2=0x7f1dc4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1ddb7fe700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ddc0b19f0 0x7f1dc4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1ddb7fe700 1 -- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc4003a90 msgr2=0x7f1dc4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1ddb7fe700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc4003a90 0x7f1dc4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1ddb7fe700 1 -- 172.21.15.131:0/816451853 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc4003570 con 0x7f1dc4004d50 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1ddb7fe700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc4004d50 0x7f1dc40051c0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc070fa0 tx=0x7f1dd400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2643b697b74af6 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd400ea60 con 0x7f1dc4004d50 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd400f040 con 0x7f1dc4004d50 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.343+0000 7f1dd9ffb700 1 -- 172.21.15.131:0/816451853 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd400ee10 con 0x7f1dc4004d50 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.344+0000 7f1de13ff700 1 -- 172.21.15.131:0/816451853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc4004d50 msgr2=0x7f1dc40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.347 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.344+0000 7f1de13ff700 1 --2- 172.21.15.131:0/816451853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc4004d50 0x7f1dc40051c0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc070fa0 tx=0x7f1dd400b040 comp rx=0 tx=0).stop 2023-12-26T23:13:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.344+0000 7f1ddb7fe700 1 -- 172.21.15.131:0/816451853 reap_dead start 2023-12-26T23:13:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.344+0000 7f1de13ff700 1 -- 172.21.15.131:0/816451853 shutdown_connections 2023-12-26T23:13:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.344+0000 7f1de13ff700 1 -- 172.21.15.131:0/816451853 >> 172.21.15.131:0/816451853 conn(0x7f1ddc068040 msgr2=0x7f1ddc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.344+0000 7f1de13ff700 1 -- 172.21.15.131:0/816451853 shutdown_connections 2023-12-26T23:13:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.344+0000 7f1de13ff700 1 -- 172.21.15.131:0/816451853 wait complete. 2023-12-26T23:13:27.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.345+0000 7f1de13ff700 1 Processor -- start 2023-12-26T23:13:27.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.346+0000 7f1de13ff700 1 -- start start 2023-12-26T23:13:27.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.346+0000 7f1de13ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc4004d50 0x7f1ddc1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.346+0000 7f1de13ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ddc14fe00 0x7f1ddc14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.346+0000 7f1de13ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ddc158180 0x7f1ddc1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.346+0000 7f1de13ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc06d8f0 con 0x7f1ddc14fe00 2023-12-26T23:13:27.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.346+0000 7f1de13ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc158b10 con 0x7f1ddc158180 2023-12-26T23:13:27.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.346+0000 7f1de13ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc1505a0 con 0x7f1dc4004d50 2023-12-26T23:13:27.352 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1dda7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ddc14fe00 0x7f1ddc14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1dda7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ddc14fe00 0x7f1ddc14dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41298/0 (socket says 172.21.15.131:41298) 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1dda7fc700 1 -- 172.21.15.131:0/1331284649 learned_addr learned my addr 172.21.15.131:0/1331284649 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1ddaffd700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc4004d50 0x7f1ddc1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1ddb7fe700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ddc158180 0x7f1ddc1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1dda7fc700 1 -- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc4004d50 msgr2=0x7f1ddc1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1dda7fc700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc4004d50 0x7f1ddc1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1dda7fc700 1 -- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ddc158180 msgr2=0x7f1ddc1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1dda7fc700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ddc158180 0x7f1ddc1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1dda7fc700 1 -- 172.21.15.131:0/1331284649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc150880 con 0x7f1ddc14fe00 2023-12-26T23:13:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1ddaffd700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc4004d50 0x7f1ddc1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1ddb7fe700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ddc158180 0x7f1ddc1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.347+0000 7f1dda7fc700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ddc14fe00 0x7f1ddc14dcc0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0007ae0 tx=0x7f1dd00079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.348+0000 7f1dc2ffd700 1 -- 172.21.15.131:0/1331284649 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0007e30 con 0x7f1ddc14fe00 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.348+0000 7f1dc2ffd700 1 -- 172.21.15.131:0/1331284649 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd000f8a0 con 0x7f1ddc14fe00 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.348+0000 7f1dc2ffd700 1 -- 172.21.15.131:0/1331284649 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd000fc60 con 0x7f1ddc14fe00 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.348+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ddc150b60 con 0x7f1ddc14fe00 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.348+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc150270 con 0x7f1ddc14fe00 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.349+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ddc0657e0 con 0x7f1ddc14fe00 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.350+0000 7f1dc2ffd700 1 -- 172.21.15.131:0/1331284649 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1dd0013600 con 0x7f1ddc14fe00 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.350+0000 7f1dc2ffd700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1dc8066f30 0x7f1dc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.354 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.350+0000 7f1dc2ffd700 1 -- 172.21.15.131:0/1331284649 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f1dd0084730 con 0x7f1ddc14fe00 2023-12-26T23:13:27.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.351+0000 7f1ddaffd700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1dc8066f30 0x7f1dc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.351+0000 7f1ddaffd700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1dc8066f30 0x7f1dc80693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc000c00 tx=0x7f1dcc009040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:27.355 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.355+0000 7f1dc2ffd700 1 -- 172.21.15.131:0/1331284649 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dd00517c0 con 0x7f1ddc14fe00 2023-12-26T23:13:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.543+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 --> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ddc0caa10 con 0x7f1dc8066f30 2023-12-26T23:13:27.545 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.545+0000 7f1dc2ffd700 1 -- 172.21.15.131:0/1331284649 <== mgr.10398 v2:172.21.15.131:6836/92228 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50537 (secure 0 0 0) 0x7f1ddc0caa10 con 0x7f1dc8066f30 2023-12-26T23:13:27.546 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:13:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.547+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1dc8066f30 msgr2=0x7f1dc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.548 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1dc8066f30 0x7f1dc80693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc000c00 tx=0x7f1dcc009040 comp rx=0 tx=0).stop 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ddc14fe00 msgr2=0x7f1ddc14dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ddc14fe00 0x7f1ddc14dcc0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0007ae0 tx=0x7f1dd00079d0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 shutdown_connections 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f1dc8066f30 0x7f1dc80693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ddc158180 0x7f1ddc1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc4004d50 0x7f1ddc1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 --2- 172.21.15.131:0/1331284649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ddc14fe00 0x7f1ddc14dcc0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 >> 172.21.15.131:0/1331284649 conn(0x7f1ddc068040 msgr2=0x7f1ddc0c1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 shutdown_connections 2023-12-26T23:13:27.549 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.548+0000 7f1de13ff700 1 -- 172.21.15.131:0/1331284649 wait complete. 2023-12-26T23:13:27.550 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2023-12-26T23:13:27.565 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-26T23:13:27.249194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2461928,"kb_used_data":22184,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752512792,"statfs":{"total":773094113280,"available":770573099008,"internally_reserved":0,"allocated":22716416,"data_stored":12028816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002186"},"pg_stats":[{"pgid":"1.0","version":"215'371","reported_seq":1247,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-26T23:13:04.991706+0000","last_change":"2023-12-26T23:13:04.991681+0000","last_active":"2023-12-26T23:13:04.991706+0000","last_peered":"2023-12-26T23:13:04.991706+0000","last_clean":"2023-12-26T23:13:04.991706+0000","last_became_active":"2023-12-26T21:36:38.968550+0000","last_became_peered":"2023-12-26T21:36:38.968550+0000","last_unstale":"2023-12-26T23:13:04.991706+0000","last_undegraded":"2023-12-26T23:13:04.991706+0000","last_fullsized":"2023-12-26T23:13:04.991706+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'371","last_scrub_stamp":"2023-12-26T23:13:04.991615+0000","last_deep_scrub":"215'371","last_deep_scrub_stamp":"2023-12-26T23:13:04.991615+0000","last_clean_scrub_stamp":"2023-12-26T23:13:04.991615+0000","objects_scrubbed":2,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-28T00:24:10.821024+0000","scrub_duration":0.0059436769999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543290,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.372},"max":{"1min":0.98199999999999998,"5min":0.98799999999999999,"15min":1.048},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93400000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66100000000000003}]},{"osd":1,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.498,"5min":0.39700000000000002,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.72199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.39600000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.0449999999999999},"last":0.81100000000000005}]},{"osd":2,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.55800000000000005,"5min":0.53500000000000003,"15min":0.48899999999999999},"max":{"1min":0.94899999999999995,"5min":0.96199999999999997,"15min":1.0089999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1},"last":0.752}]},{"osd":3,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0760000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.98199999999999998},"last":0.76300000000000001}]},{"osd":4,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.44500000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90900000000000003,"15min":0.93000000000000005},"last":0.505}]},{"osd":5,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Tue Dec 26 23:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.5,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.59699999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.996,"5min":1.028,"15min":1.028},"last":0.70699999999999996}]}]},{"osd":6,"up_from":15,"seq":64424510586,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67800000000000005,"15min":0.70999999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95999999999999996,"5min":0.998,"15min":1.1080000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72499999999999998,"15min":0.73999999999999999},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63300000000000001}]},{"osd":1,"last update":"Tue Dec 26 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.439},"max":{"1min":1.03,"5min":1.03,"15min":1.046},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72199999999999998,"15min":0.745},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0580000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Tue Dec 26 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.52600000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.53600000000000003,"5min":0.49299999999999999,"15min":0.47199999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Dec 26 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73899999999999999,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.748,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Dec 26 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88600000000000001,"5min":1.018,"15min":1.1599999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63600000000000001,"15min":0.65300000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40899999999999997},"max":{"1min":0.876,"5min":0.94599999999999995,"15min":1.0669999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Dec 26 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64800000000000002,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.38300000000000001},"max":{"1min":0.877,"5min":0.92100000000000004,"15min":1.111},"last":0.496},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70699999999999996,"15min":0.71599999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.45500000000000002},"max":{"1min":0.95199999999999996,"5min":1.103,"15min":1.103},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.72499999999999998},"min":{"1min":0.45300000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91600000000000004,"5min":0.98399999999999999,"15min":1.1419999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.42299999999999999},"max":{"1min":0.92000000000000004,"5min":0.97099999999999997,"15min":1.1970000000000001},"last":0.71499999999999997}]}]},{"osd":1,"up_from":15,"seq":64424510586,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300496,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071344,"statfs":{"total":96636764160,"available":96329056256,"internally_reserved":0,"allocated":3284992,"data_stored":1946378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.28399999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.755},{"interface":"front","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.33900000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.84099999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.272,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.60699999999999998}]},{"osd":4,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.53600000000000003},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.103},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":0.95499999999999996,"5min":1.258,"15min":1.258},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.57899999999999996,"5min":0.56000000000000005,"15min":0.51300000000000001},"max":{"1min":1.0329999999999999,"5min":1.0549999999999999,"15min":1.22},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82099999999999995,"15min":0.82999999999999996},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58499999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.875}]},{"osd":6,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":0.82799999999999996},"min":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0609999999999999,"5min":1.1399999999999999,"15min":1.1950000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.83399999999999996,"15min":0.84099999999999997},"min":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.54200000000000004},"max":{"1min":1.091,"5min":1.1559999999999999,"15min":1.2090000000000001},"last":0.83499999999999996}]},{"osd":7,"last update":"Tue Dec 26 23:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"max":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.82299999999999995},"min":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.218},"last":0.85999999999999999}]}]},{"osd":0,"up_from":15,"seq":64424510586,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308108,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063732,"statfs":{"total":96636764160,"available":96321261568,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.45100000000000001,"15min":0.45400000000000001},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.27600000000000002},"max":{"1min":0.60699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.434},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.95499999999999996,"15min":1.1060000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82999999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.622}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.95499999999999996,"15min":1.3839999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.91800000000000004,"15min":1.012},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.2250000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.49299999999999999},"max":{"1min":1.02,"5min":1.0649999999999999,"15min":1.113},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.157},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.137,"5min":1.1459999999999999,"15min":1.282},"last":0.749}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.085,"5min":1.121,"15min":1.151},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79600000000000004,"15min":0.80300000000000005},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93200000000000005,"5min":1.167,"15min":1.167},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0640000000000001,"5min":1.1439999999999999,"15min":1.256},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.81200000000000006},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54000000000000004},"max":{"1min":1.042,"5min":1.2030000000000001,"15min":1.3360000000000001},"last":0.69999999999999996}]}]},{"osd":2,"up_from":14,"seq":60129543291,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64100000000000001,"5min":0.67800000000000005,"15min":0.84599999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.57799999999999996}]},{"osd":1,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64700000000000002,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.438}]},{"osd":3,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.624,"15min":0.625},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.73399999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75,"5min":0.89300000000000002,"15min":1.056},"last":0.65100000000000002}]},{"osd":4,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77600000000000002,"15min":0.79500000000000004},"min":{"1min":0.56299999999999994,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83299999999999996,"5min":1.8,"15min":1.8},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.085,"5min":1.222,"15min":1.222},"last":0.88200000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.79600000000000004},"min":{"1min":0.60099999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1020000000000001,"5min":1.923,"15min":1.923},"last":0.996},{"interface":"front","average":{"1min":0.753,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.95599999999999996,"5min":1.1599999999999999,"15min":1.2130000000000001},"last":0.84099999999999997}]},{"osd":6,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89400000000000002,"5min":0.97799999999999998,"15min":1.272},"last":0.87},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.69599999999999995,"5min":0.60299999999999998,"15min":0.53300000000000003},"max":{"1min":1.0449999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.90100000000000002}]},{"osd":7,"last update":"Tue Dec 26 23:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.81299999999999994},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1619999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.65600000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.175,"5min":1.841,"15min":1.841},"last":0.85099999999999998}]}]},{"osd":3,"up_from":15,"seq":64424510587,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":304012,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067828,"statfs":{"total":96636764160,"available":96325455872,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.48699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56999999999999995,"5min":0.746,"15min":0.76500000000000001},"last":0.60099999999999998}]},{"osd":1,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47099999999999997,"15min":0.47799999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.753},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.34100000000000003},"max":{"1min":0.69399999999999995,"5min":0.77100000000000002,"15min":0.90400000000000003},"last":0.58399999999999996}]},{"osd":2,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.628,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.41499999999999998},"max":{"1min":0.74299999999999999,"5min":0.93500000000000005,"15min":1.0309999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.748,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.113,"5min":1.159,"15min":1.177},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.54000000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.93400000000000005,"5min":0.999,"15min":1.081},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.58399999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.028,"5min":1.046,"15min":1.0980000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.79900000000000004},"min":{"1min":0.53500000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80400000000000005,"15min":0.82899999999999996},"min":{"1min":0.61399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.999,"5min":1.202,"15min":1.202},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":0.84399999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.177,"15min":1.177},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Dec 26 23:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.53300000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.214},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0549999999999999,"5min":1.292,"15min":1.292},"last":0.69699999999999995}]}]},{"osd":4,"up_from":14,"seq":60129543290,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312204,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059636,"statfs":{"total":96636764160,"available":96317067264,"internally_reserved":0,"allocated":2691072,"data_stored":1356010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.70199999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.41599999999999998},"max":{"1min":0.877,"5min":0.91800000000000004,"15min":1.0489999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.68400000000000005,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":0.94599999999999995,"15min":1.008},"last":0.83299999999999996}]},{"osd":1,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.437,"15min":0.437},"max":{"1min":0.84399999999999997,"5min":0.98699999999999999,"15min":1.0249999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85299999999999998,"5min":0.89000000000000001,"15min":1.0329999999999999},"last":0.75}]},{"osd":2,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82699999999999996,"5min":0.92900000000000005,"15min":0.998},"last":0.81799999999999995}]},{"osd":3,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.52300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72199999999999998,"15min":0.75600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81100000000000005,"5min":0.97299999999999998,"15min":1.0589999999999999},"last":0.78500000000000003}]},{"osd":5,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.41299999999999998,"5min":0.40200000000000002,"15min":0.39300000000000002},"max":{"1min":0.75,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.44800000000000001,"5min":0.41299999999999998,"15min":0.36799999999999999},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":0.98999999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"max":{"1min":0.81899999999999995,"5min":1.002,"15min":1.024},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.98799999999999999},"last":0.84899999999999998}]},{"osd":7,"last update":"Tue Dec 26 23:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86599999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.54400000000000004,"5min":0.53700000000000003,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":1.1599999999999999,"15min":1.1799999999999999},"last":0.88400000000000001}]}]},{"osd":5,"up_from":15,"seq":64424510585,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300496,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071344,"statfs":{"total":96636764160,"available":96329056256,"internally_reserved":0,"allocated":3284992,"data_stored":1946378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 26 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69999999999999996,"15min":0.71899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.75},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":1.0629999999999999},"last":0.68300000000000005}]},{"osd":1,"last update":"Tue Dec 26 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.71699999999999997,"15min":0.71899999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94599999999999995,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.45200000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.016},"last":0.73999999999999999}]},{"osd":2,"last update":"Tue Dec 26 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.504,"5min":0.45600000000000002,"15min":0.41699999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.022},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.44800000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.76500000000000001}]},{"osd":3,"last update":"Tue Dec 26 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73799999999999999,"15min":0.751},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.878,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.86099999999999999,"5min":0.96899999999999997,"15min":0.997},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Dec 26 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88300000000000001,"5min":0.93899999999999995,"15min":1.0309999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.95199999999999996},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Dec 26 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.626,"15min":0.65700000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":1.042},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.47999999999999998,"5min":0.45800000000000002,"15min":0.441},"max":{"1min":0.84299999999999997,"5min":0.99299999999999999,"15min":1.0820000000000001},"last":0.71699999999999997}]},{"osd":7,"last update":"Tue Dec 26 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69099999999999995,"15min":0.71699999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90000000000000002,"5min":0.97499999999999998,"15min":1.0920000000000001},"last":0.52900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-26T23:13:27.567 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-12-26T23:13:27.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.752+0000 7fd6b13ff700 1 Processor -- start 2023-12-26T23:13:27.752 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.752+0000 7fd6b13ff700 1 -- start start 2023-12-26T23:13:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.753+0000 7fd6b13ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac0b07f0 0x7fd6ac1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.753+0000 7fd6b13ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6ac159cf0 0x7fd6ac155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6b13ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac0c8bf0 con 0x7fd6ac159920 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6b13ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ac155f50 con 0x7fd6ac159cf0 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6b13ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0c8e60 con 0x7fd6ac0b07f0 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6aaffd700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd6ac159920 0x7fd6ac0b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49292/0 (socket says 172.21.15.131:49292) 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6aaffd700 1 -- 172.21.15.131:0/2105785380 learned_addr learned my addr 172.21.15.131:0/2105785380 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6ab7fe700 1 --2- 172.21.15.131:0/2105785380 >> v2:172.21.15.131:3300/0 conn(0x7fd6ac159cf0 0x7fd6ac155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6aa7fc700 1 --2- 172.21.15.131:0/2105785380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac0b07f0 0x7fd6ac1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6a9ffb700 1 -- 172.21.15.131:0/2105785380 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13245127 0 0) 0x7fd6ac0c8bf0 con 0x7fd6ac159920 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6a9ffb700 1 -- 172.21.15.131:0/2105785380 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd694003570 con 0x7fd6ac159920 2023-12-26T23:13:27.756 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6ab7fe700 1 -- 172.21.15.131:0/2105785380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac0b07f0 msgr2=0x7fd6ac1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6ab7fe700 1 --2- 172.21.15.131:0/2105785380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac0b07f0 0x7fd6ac1554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6ab7fe700 1 -- 172.21.15.131:0/2105785380 >> v1:172.21.15.131:6789/0 conn(0x7fd6ac159920 legacy=0x7fd6ac0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.754+0000 7fd6ab7fe700 1 -- 172.21.15.131:0/2105785380 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0c9140 con 0x7fd6ac159cf0 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6ab7fe700 1 --2- 172.21.15.131:0/2105785380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac159cf0 0x7fd6ac155a10 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400d0d0 tx=0x7fd6a400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cd117c1b39dcef server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6a9ffb700 1 -- 172.21.15.131:0/2105785380 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4010070 con 0x7fd6ac159cf0 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6a9ffb700 1 -- 172.21.15.131:0/2105785380 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a400da80 con 0x7fd6ac159cf0 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6a9ffb700 1 -- 172.21.15.131:0/2105785380 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a400de40 con 0x7fd6ac159cf0 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2105785380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac159cf0 msgr2=0x7fd6ac155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6b13ff700 1 --2- 172.21.15.131:0/2105785380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac159cf0 0x7fd6ac155a10 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400d0d0 tx=0x7fd6a400b990 comp rx=0 tx=0).stop 2023-12-26T23:13:27.757 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2105785380 shutdown_connections 2023-12-26T23:13:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6b13ff700 1 --2- 172.21.15.131:0/2105785380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac159cf0 0x7fd6ac155a10 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6b13ff700 1 --2- 172.21.15.131:0/2105785380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac0b07f0 0x7fd6ac1554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2105785380 >> 172.21.15.131:0/2105785380 conn(0x7fd6ac068040 msgr2=0x7fd6ac069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.755+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2105785380 shutdown_connections 2023-12-26T23:13:27.758 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.756+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2105785380 wait complete. 2023-12-26T23:13:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.757+0000 7fd6b13ff700 1 Processor -- start 2023-12-26T23:13:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.757+0000 7fd6b13ff700 1 -- start start 2023-12-26T23:13:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6b13ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac159920 0x7fd6ac1699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6b13ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6ac159cf0 0x7fd6ac145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6b13ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac14f2a0 0x7fd6ac14d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6b13ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac14ee10 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6b13ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac146320 con 0x7fd6ac159cf0 2023-12-26T23:13:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6b13ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac14e200 con 0x7fd6ac159920 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac14f2a0 0x7fd6ac14d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac14f2a0 0x7fd6ac14d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41314/0 (socket says 172.21.15.131:41314) 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6ab7fe700 1 -- 172.21.15.131:0/2136442472 learned_addr learned my addr 172.21.15.131:0/2136442472 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6aaffd700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac159920 0x7fd6ac1699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.758+0000 7fd6aa7fc700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6ac159cf0 0x7fd6ac145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd6ab7fe700 1 -- 172.21.15.131:0/2136442472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac159920 msgr2=0x7fd6ac1699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd6ab7fe700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac159920 0x7fd6ac1699b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd6ab7fe700 1 -- 172.21.15.131:0/2136442472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6ac159cf0 msgr2=0x7fd6ac145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd6ab7fe700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6ac159cf0 0x7fd6ac145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd6ab7fe700 1 -- 172.21.15.131:0/2136442472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a400bc30 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd6aa7fc700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6ac159cf0 0x7fd6ac145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-26T23:13:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd6ab7fe700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac14f2a0 0x7fd6ac14d9b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0688f0 tx=0x7fd6a401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd692ffd700 1 -- 172.21.15.131:0/2136442472 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a401f670 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd692ffd700 1 -- 172.21.15.131:0/2136442472 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a401f7f0 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac14e4e0 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd692ffd700 1 -- 172.21.15.131:0/2136442472 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a401fbb0 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.759+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac14eaa0 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.760+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac14efc0 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.763 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.761+0000 7fd692ffd700 1 -- 172.21.15.131:0/2136442472 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd6a4003040 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.762+0000 7fd692ffd700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd698066f30 0x7fd6980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.762+0000 7fd692ffd700 1 -- 172.21.15.131:0/2136442472 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fd6a4086fa0 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.762+0000 7fd6aaffd700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd698066f30 0x7fd6980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.763+0000 7fd6aaffd700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd698066f30 0x7fd6980693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd69c007fd0 tx=0x7fd69c009040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:27.765 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.766+0000 7fd692ffd700 1 -- 172.21.15.131:0/2136442472 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6ac14efc0 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.957 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.957+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fd6ac0657e0 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.962+0000 7fd692ffd700 1 -- 172.21.15.131:0/2136442472 <== mon.0 v2:172.21.15.131:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd6a4054030 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.961 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.962+0000 7fd692ffd700 1 -- 172.21.15.131:0/2136442472 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v76)=0 v76) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd6a4057ac0 con 0x7fd6ac14f2a0 2023-12-26T23:13:27.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd698066f30 msgr2=0x7fd6980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd698066f30 0x7fd6980693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd69c007fd0 tx=0x7fd69c009040 comp rx=0 tx=0).stop 2023-12-26T23:13:27.965 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac14f2a0 msgr2=0x7fd6ac14d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac14f2a0 0x7fd6ac14d9b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0688f0 tx=0x7fd6a401c000 comp rx=0 tx=0).stop 2023-12-26T23:13:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 shutdown_connections 2023-12-26T23:13:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7fd698066f30 0x7fd6980693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac14f2a0 0x7fd6ac14d9b0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6ac159920 0x7fd6ac1699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 --2- 172.21.15.131:0/2136442472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6ac159cf0 0x7fd6ac145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.965+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 >> 172.21.15.131:0/2136442472 conn(0x7fd6ac068040 msgr2=0x7fd6ac0c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.966+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 shutdown_connections 2023-12-26T23:13:27.966 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:27.966+0000 7fd6b13ff700 1 -- 172.21.15.131:0/2136442472 wait complete. 2023-12-26T23:13:27.982 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-26T23:13:27.983 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-26T23:13:28.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.173+0000 7f3a991ff700 1 Processor -- start 2023-12-26T23:13:28.172 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.173+0000 7f3a991ff700 1 -- start start 2023-12-26T23:13:28.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.174+0000 7f3a991ff700 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3a940b0750 0x7f3a940b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:28.175 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.174+0000 7f3a991ff700 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a94156940 con 0x7f3a940cb320 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.174+0000 7f3a991ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a940ae500 con 0x7f3a940c9400 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.174+0000 7f3a991ff700 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a940c8cc0 con 0x7f3a940b0750 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.174+0000 7f3a92d9d700 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3a940cb320 0x7f3a94069ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49296/0 (socket says 172.21.15.131:49296) 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.174+0000 7f3a92d9d700 1 -- 172.21.15.131:0/4293391207 learned_addr learned my addr 172.21.15.131:0/4293391207 (peer_addr_for_me v1:172.21.15.131:0/0) 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.174+0000 7f3a9259c700 1 --2- 172.21.15.131:0/4293391207 >> v2:172.21.15.131:3300/0 conn(0x7f3a940b0750 0x7f3a940b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.175+0000 7f3a91d9b700 1 -- 172.21.15.131:0/4293391207 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778607546 0 0) 0x7f3a94156940 con 0x7f3a940cb320 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.175+0000 7f3a91d9b700 1 -- 172.21.15.131:0/4293391207 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a7c003570 con 0x7f3a940cb320 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.175+0000 7f3a9259c700 1 -- 172.21.15.131:0/4293391207 >> v1:172.21.15.190:6789/0 conn(0x7f3a940c9400 legacy=0x7f3a940c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.175+0000 7f3a9259c700 1 -- 172.21.15.131:0/4293391207 >> v1:172.21.15.131:6789/0 conn(0x7f3a940cb320 legacy=0x7f3a94069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:28.176 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.175+0000 7f3a9259c700 1 -- 172.21.15.131:0/4293391207 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a940b0280 con 0x7f3a940b0750 2023-12-26T23:13:28.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.175+0000 7f3a9259c700 1 --2- 172.21.15.131:0/4293391207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940b0750 0x7f3a940b0bf0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f3a880098e0 tx=0x7f3a880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b27350c497cd3bf server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:28.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.175+0000 7f3a91d9b700 1 -- 172.21.15.131:0/4293391207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800b640 con 0x7f3a940b0750 2023-12-26T23:13:28.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.175+0000 7f3a91d9b700 1 -- 172.21.15.131:0/4293391207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3a8800b7a0 con 0x7f3a940b0750 2023-12-26T23:13:28.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.175+0000 7f3a91d9b700 1 -- 172.21.15.131:0/4293391207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800bb60 con 0x7f3a940b0750 2023-12-26T23:13:28.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.176+0000 7f3a991ff700 1 -- 172.21.15.131:0/4293391207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940b0750 msgr2=0x7f3a940b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:28.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.176+0000 7f3a991ff700 1 --2- 172.21.15.131:0/4293391207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940b0750 0x7f3a940b0bf0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f3a880098e0 tx=0x7f3a880091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:28.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.176+0000 7f3a991ff700 1 -- 172.21.15.131:0/4293391207 shutdown_connections 2023-12-26T23:13:28.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.176+0000 7f3a991ff700 1 --2- 172.21.15.131:0/4293391207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940b0750 0x7f3a940b0bf0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:28.177 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.176+0000 7f3a991ff700 1 -- 172.21.15.131:0/4293391207 >> 172.21.15.131:0/4293391207 conn(0x7f3a94068040 msgr2=0x7f3a94069590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:28.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.176+0000 7f3a991ff700 1 -- 172.21.15.131:0/4293391207 shutdown_connections 2023-12-26T23:13:28.178 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.176+0000 7f3a991ff700 1 -- 172.21.15.131:0/4293391207 wait complete. 2023-12-26T23:13:28.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.177+0000 7f3a991ff700 1 Processor -- start 2023-12-26T23:13:28.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.178+0000 7f3a991ff700 1 -- start start 2023-12-26T23:13:28.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.178+0000 7f3a991ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a940b0750 0x7f3a94146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:28.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.178+0000 7f3a991ff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940c9400 0x7f3a941528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:28.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.178+0000 7f3a991ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940cb320 0x7f3a94151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:28.181 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.178+0000 7f3a991ff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a94147270 con 0x7f3a940c9400 2023-12-26T23:13:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.178+0000 7f3a991ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a94153340 con 0x7f3a940cb320 2023-12-26T23:13:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.178+0000 7f3a991ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a94153620 con 0x7f3a940b0750 2023-12-26T23:13:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940c9400 0x7f3a941528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940c9400 0x7f3a941528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41318/0 (socket says 172.21.15.131:41318) 2023-12-26T23:13:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9259c700 1 -- 172.21.15.131:0/397010555 learned_addr learned my addr 172.21.15.131:0/397010555 (peer_addr_for_me v2:172.21.15.131:0/0) 2023-12-26T23:13:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a92d9d700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a940b0750 0x7f3a94146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9359e700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940cb320 0x7f3a94151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9259c700 1 -- 172.21.15.131:0/397010555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a940b0750 msgr2=0x7f3a94146d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9259c700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a940b0750 0x7f3a94146d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9259c700 1 -- 172.21.15.131:0/397010555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940cb320 msgr2=0x7f3a94151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9259c700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940cb320 0x7f3a94151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9259c700 1 -- 172.21.15.131:0/397010555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a88009420 con 0x7f3a940c9400 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.179+0000 7f3a9259c700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940c9400 0x7f3a941528f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f3a940688f0 tx=0x7f3a880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.180+0000 7f3a7affd700 1 -- 172.21.15.131:0/397010555 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88000bc0 con 0x7f3a940c9400 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.180+0000 7f3a7affd700 1 -- 172.21.15.131:0/397010555 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3a8800b910 con 0x7f3a940c9400 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.180+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a94153900 con 0x7f3a940c9400 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.180+0000 7f3a7affd700 1 -- 172.21.15.131:0/397010555 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88000d20 con 0x7f3a940c9400 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.180+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a94151d00 con 0x7f3a940c9400 2023-12-26T23:13:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.181+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a94157d20 con 0x7f3a940c9400 2023-12-26T23:13:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.182+0000 7f3a7affd700 1 -- 172.21.15.131:0/397010555 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 116) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3a88023070 con 0x7f3a940c9400 2023-12-26T23:13:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.183+0000 7f3a7affd700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3a80066f20 0x7f3a800693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-26T23:13:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.183+0000 7f3a7affd700 1 -- 172.21.15.131:0/397010555 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f3a88085820 con 0x7f3a940c9400 2023-12-26T23:13:28.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.183+0000 7f3a92d9d700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3a80066f20 0x7f3a800693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-26T23:13:28.185 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.184+0000 7f3a92d9d700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3a80066f20 0x7f3a800693d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a84007fd0 tx=0x7f3a84009040 comp rx=0 tx=0).ready entity=mgr.10398 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-26T23:13:28.187 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.187+0000 7f3a7affd700 1 -- 172.21.15.131:0/397010555 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a880529d0 con 0x7f3a940c9400 2023-12-26T23:13:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.427+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a941579c0 con 0x7f3a940c9400 2023-12-26T23:13:28.427 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.428+0000 7f3a7affd700 1 -- 172.21.15.131:0/397010555 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+3544 (secure 0 0 0) 0x7f3a941579c0 con 0x7f3a940c9400 2023-12-26T23:13:28.427 INFO:teuthology.orchestra.run.smithi131.stdout: 2023-12-26T23:13:28.427 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":100,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8372,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2599990460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2599990460},{"type":"v1","addr":"172.21.15.190:6835","nonce":2599990460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":9565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/3465421941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3465421941},{"type":"v1","addr":"172.21.15.131:6834","nonce":3465421941}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93},{"gid":9940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2025943505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2025943505},{"type":"v1","addr":"172.21.15.190:6834","nonce":2025943505}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":10122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3595865527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3595865527},{"type":"v1","addr":"172.21.15.131:6835","nonce":3595865527}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[]} 2023-12-26T23:13:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3a80066f20 msgr2=0x7f3a800693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3a80066f20 0x7f3a800693d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a84007fd0 tx=0x7f3a84009040 comp rx=0 tx=0).stop 2023-12-26T23:13:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940c9400 msgr2=0x7f3a941528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-26T23:13:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940c9400 0x7f3a941528f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f3a940688f0 tx=0x7f3a880091a0 comp rx=0 tx=0).stop 2023-12-26T23:13:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 shutdown_connections 2023-12-26T23:13:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:6836/92228,v1:172.21.15.131:6837/92228] conn(0x7f3a80066f20 0x7f3a800693d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a940cb320 0x7f3a94151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a940b0750 0x7f3a94146d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 --2- 172.21.15.131:0/397010555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a940c9400 0x7f3a941528f0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-26T23:13:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 >> 172.21.15.131:0/397010555 conn(0x7f3a94068040 msgr2=0x7f3a94069510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-26T23:13:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 shutdown_connections 2023-12-26T23:13:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2023-12-26T23:13:28.430+0000 7f3a991ff700 1 -- 172.21.15.131:0/397010555 wait complete. 2023-12-26T23:13:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 100 2023-12-26T23:13:28.448 DEBUG:tasks.cephfs.filesystem:already dead... 2023-12-26T23:13:28.448 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-26T23:13:28.448 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-26T23:13:28.449 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.462 INFO:tasks.ceph.mds.a:Stopped 2023-12-26T23:13:28.462 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-26T23:13:28.462 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.516 INFO:tasks.ceph.mds.c:Stopped 2023-12-26T23:13:28.516 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-26T23:13:28.516 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.534 INFO:tasks.ceph.mds.b:Stopped 2023-12-26T23:13:28.534 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-26T23:13:28.534 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.590 INFO:tasks.ceph.mds.d:Stopped 2023-12-26T23:13:28.590 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-26T23:13:28.590 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-26T23:13:28.590 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.643 INFO:tasks.ceph.osd.0:Stopped 2023-12-26T23:13:28.644 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-26T23:13:28.644 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.699 INFO:tasks.ceph.osd.1:Stopped 2023-12-26T23:13:28.699 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-26T23:13:28.700 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.755 INFO:tasks.ceph.osd.2:Stopped 2023-12-26T23:13:28.755 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-26T23:13:28.755 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.806 INFO:tasks.ceph.osd.3:Stopped 2023-12-26T23:13:28.806 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-26T23:13:28.806 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.854 INFO:tasks.ceph.osd.4:Stopped 2023-12-26T23:13:28.854 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-26T23:13:28.854 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.902 INFO:tasks.ceph.osd.5:Stopped 2023-12-26T23:13:28.902 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-12-26T23:13:28.902 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:28.954 INFO:tasks.ceph.osd.6:Stopped 2023-12-26T23:13:28.954 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-26T23:13:28.954 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:29.005 INFO:tasks.ceph.osd.7:Stopped 2023-12-26T23:13:29.005 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-26T23:13:29.005 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-26T23:13:29.006 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:29.073 INFO:tasks.ceph.mgr.y:Stopped 2023-12-26T23:13:29.073 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-26T23:13:29.074 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:29.135 INFO:tasks.ceph.mgr.x:Stopped 2023-12-26T23:13:29.135 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-26T23:13:29.136 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-26T23:13:29.136 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:29.167 INFO:tasks.ceph.mon.a:Stopped 2023-12-26T23:13:29.168 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-26T23:13:29.168 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:29.192 INFO:tasks.ceph.mon.b:Stopped 2023-12-26T23:13:29.192 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-26T23:13:29.192 INFO:teuthology.orchestra.run:waiting for 300 2023-12-26T23:13:29.257 INFO:tasks.ceph.mon.c:Stopped 2023-12-26T23:13:29.257 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-26T23:13:29.257 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-26T23:13:29.340 INFO:teuthology.orchestra.run.smithi131.stdout:1703626748.1495957 mon.a (mon.0) 324 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-12-26T23:13:29.341 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-12-26T23:13:29.341 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-26T23:13:29.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-26T23:13:29.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-26T23:13:29.582 INFO:teuthology.orchestra.run.smithi131.stdout:1703626748.1495957 mon.a (mon.0) 324 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-12-26T23:13:29.583 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T23:13:29.583 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-26T23:13:29.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T23:13:29.799 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-26T23:13:29.919 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T23:13:29.919 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-26T23:13:30.054 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T23:13:30.054 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-26T23:13:30.180 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T23:13:30.180 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-26T23:13:30.460 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T23:13:30.460 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-26T23:13:30.578 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T23:13:30.578 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-26T23:13:30.692 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T23:13:30.692 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-12-26T23:13:30.840 INFO:tasks.ceph:Archiving mon data... 2023-12-26T23:13:30.841 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/data/mon.a.tgz 2023-12-26T23:13:30.841 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2023-12-26T23:13:30.887 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.EPO2oTO2YS 2023-12-26T23:13:30.887 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.EPO2oTO2YS 2023-12-26T23:13:31.398 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.EPO2oTO2YS 2023-12-26T23:13:31.512 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.EPO2oTO2YS is 2MB 2023-12-26T23:13:31.652 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.EPO2oTO2YS 2023-12-26T23:13:31.696 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/data/mon.b.tgz 2023-12-26T23:13:31.696 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-12-26T23:13:31.741 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.HeplcmmfDG 2023-12-26T23:13:31.741 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.HeplcmmfDG 2023-12-26T23:13:32.291 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.HeplcmmfDG 2023-12-26T23:13:32.406 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.HeplcmmfDG is 3MB 2023-12-26T23:13:32.545 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.HeplcmmfDG 2023-12-26T23:13:32.589 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/data/mon.c.tgz 2023-12-26T23:13:32.589 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-12-26T23:13:32.672 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.uJY6oGNp4E 2023-12-26T23:13:32.672 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.uJY6oGNp4E 2023-12-26T23:13:33.169 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.uJY6oGNp4E 2023-12-26T23:13:33.284 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.uJY6oGNp4E is 3MB 2023-12-26T23:13:33.423 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.uJY6oGNp4E 2023-12-26T23:13:33.466 INFO:tasks.ceph:Cleaning ceph cluster... 2023-12-26T23:13:33.467 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-26T23:13:33.472 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-26T23:13:33.574 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-12-26T23:13:33.574 DEBUG:teuthology.orchestra.run.smithi131:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-26T23:13:33.579 DEBUG:teuthology.orchestra.run.smithi190:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-26T23:13:33.655 INFO:teuthology.orchestra.run.smithi190.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-26T23:13:33.688 INFO:teuthology.orchestra.run.smithi131.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-26T23:13:33.704 INFO:tasks.ceph:Archiving crash dumps... 2023-12-26T23:13:33.706 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/remote/smithi131/crash 2023-12-26T23:13:33.707 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-26T23:13:33.774 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/remote/smithi190/crash 2023-12-26T23:13:33.775 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-26T23:13:33.842 INFO:tasks.ceph:Shutting down logrotate 2023-12-26T23:13:33.842 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:13:33.848 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-26T23:13:33.949 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-26T23:13:34.016 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-26T23:13:34.084 INFO:tasks.ceph:Compressing logs... 2023-12-26T23:13:34.084 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-26T23:13:34.089 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-26T23:13:34.152 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91282.log 2023-12-26T23:13:34.153 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104135.log 2023-12-26T23:13:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.91282.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91282.log.gz 2023-12-26T23:13:34.154 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90442.log 2023-12-26T23:13:34.155 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46197.log 2023-12-26T23:13:34.155 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104135.log.gz 2023-12-26T23:13:34.156 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-12-26T23:13:34.157 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95434.log 2023-12-26T23:13:34.157 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.90442.log.gz 2023-12-26T23:13:34.158 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45157.log 2023-12-26T23:13:34.158 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39849.log 2023-12-26T23:13:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46197.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46197.log.gz 2023-12-26T23:13:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90962.log 2023-12-26T23:13:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95434.log: gzip 88.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.95434.log.gz--verbose 2023-12-26T23:13:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:-- /var/log/ceph/ceph-client.admin.88920.log 2023-12-26T23:13:34.159 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90962.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90962.log.gz 2023-12-26T23:13:34.160 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95250.log 2023-12-26T23:13:34.162 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69356.log 89.5% 2023-12-26T23:13:34.163 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88920.log.gz 2023-12-26T23:13:34.163 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100524.log 2023-12-26T23:13:34.163 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95250.log.gz 2023-12-26T23:13:34.163 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57222.log 2023-12-26T23:13:34.163 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69356.log.gz 2023-12-26T23:13:34.164 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.1.45157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45157.log.gz 2023-12-26T23:13:34.164 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39561.log 2023-12-26T23:13:34.165 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.39849.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39849.log.gz 2023-12-26T23:13:34.165 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-12-26T23:13:34.165 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.39561.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39561.log.gz 2023-12-26T23:13:34.165 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39663.log 2023-12-26T23:13:34.165 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-26T23:13:34.165 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.39663.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39663.log.gz 2023-12-26T23:13:34.165 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39753.log 2023-12-26T23:13:34.167 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-12-26T23:13:34.167 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.39753.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39753.log.gz 2023-12-26T23:13:34.167 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39801.log 2023-12-26T23:13:34.169 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99490.log 2023-12-26T23:13:34.169 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100524.log.gz 2023-12-26T23:13:34.169 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57222.log: 88.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.57222.log.gz--verbose 2023-12-26T23:13:34.169 INFO:teuthology.orchestra.run.smithi131.stderr:-- /var/log/ceph/ceph-client.admin.76203.log 2023-12-26T23:13:34.169 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99490.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.99490.log.gz-5 2023-12-26T23:13:34.169 INFO:teuthology.orchestra.run.smithi131.stderr:--verbose -- /var/log/ceph/ceph-client.admin.89936.log 2023-12-26T23:13:34.169 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45475.log 2023-12-26T23:13:34.170 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76203.log.gz 2023-12-26T23:13:34.170 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89936.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.89936.log.gz-5 2023-12-26T23:13:34.170 INFO:teuthology.orchestra.run.smithi131.stderr:--verbose -- /var/log/ceph/ceph-client.admin.60415.log 2023-12-26T23:13:34.170 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45475.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45475.log.gz 2023-12-26T23:13:34.170 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85139.log 2023-12-26T23:13:34.170 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85485.log 2023-12-26T23:13:34.170 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60415.log.gz 2023-12-26T23:13:34.172 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-26T23:13:34.172 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.39801.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39801.log.gz 2023-12-26T23:13:34.172 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-12-26T23:13:34.172 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-12-26T23:13:34.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85139.log: gzip -5 --verbose 88.4%-- /var/log/ceph/ceph-client.admin.87736.log -- replaced with /var/log/ceph/ceph-client.admin.85139.log.gz 2023-12-26T23:13:34.173 INFO:teuthology.orchestra.run.smithi131.stderr: 2023-12-26T23:13:34.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85485.log: gzip -5 --verbose -- 87.7%/var/log/ceph/ceph-client.admin.77828.log 2023-12-26T23:13:34.174 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.85485.log.gz 2023-12-26T23:13:34.174 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87736.log: gzip 87.9%-5 -- replaced with /var/log/ceph/ceph-client.admin.87736.log.gz--verbose 2023-12-26T23:13:34.174 INFO:teuthology.orchestra.run.smithi131.stderr:-- /var/log/ceph/ceph-client.admin.47647.log 2023-12-26T23:13:34.174 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46902.log 2023-12-26T23:13:34.174 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77828.log.gz 2023-12-26T23:13:34.174 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47647.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47647.log.gz 2023-12-26T23:13:34.174 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65619.log 2023-12-26T23:13:34.175 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-12-26T23:13:34.176 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46902.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46902.log.gz 2023-12-26T23:13:34.176 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60571.log 2023-12-26T23:13:34.176 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65619.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65619.log.gz 2023-12-26T23:13:34.177 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68224.log 2023-12-26T23:13:34.177 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67346.log 2023-12-26T23:13:34.177 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60571.log.gz 2023-12-26T23:13:34.178 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68224.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68224.log.gzgzip 2023-12-26T23:13:34.178 INFO:teuthology.orchestra.run.smithi131.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.72445.log 2023-12-26T23:13:34.178 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67346.log: gzip 87.7%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.67346.log.gz-- 2023-12-26T23:13:34.179 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53693.log 2023-12-26T23:13:34.179 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-mon.b.log: 93.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-26T23:13:34.179 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39609.log 2023-12-26T23:13:34.180 INFO:teuthology.orchestra.run.smithi190.stderr: 93.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-26T23:13:34.181 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.39609.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39609.log.gz 2023-12-26T23:13:34.181 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39895.log 2023-12-26T23:13:34.182 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72445.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72445.log.gz 2023-12-26T23:13:34.182 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76295.log 2023-12-26T23:13:34.182 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53693.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2023-12-26T23:13:34.183 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53008.log 2023-12-26T23:13:34.183 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101263.log 2023-12-26T23:13:34.183 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.76295.log.gz 2023-12-26T23:13:34.183 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87131.log 2023-12-26T23:13:34.183 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53008.log.gz 2023-12-26T23:13:34.184 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.39895.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39895.log.gz 2023-12-26T23:13:34.184 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-12-26T23:13:34.185 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95064.log 2023-12-26T23:13:34.185 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87131.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.101263.log.gz 2023-12-26T23:13:34.185 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87131.log.gz 2023-12-26T23:13:34.185 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104670.log 2023-12-26T23:13:34.185 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95064.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95064.log.gz 2023-12-26T23:13:34.185 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96302.log 2023-12-26T23:13:34.186 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53142.log 2023-12-26T23:13:34.187 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104670.log.gz 2023-12-26T23:13:34.187 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.96302.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96302.log.gzgzip 2023-12-26T23:13:34.187 INFO:teuthology.orchestra.run.smithi131.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.71520.log 2023-12-26T23:13:34.188 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.45603.log 2023-12-26T23:13:34.188 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53142.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53142.log.gz 2023-12-26T23:13:34.188 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51385.log 2023-12-26T23:13:34.189 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71520.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.71520.log.gz-5 2023-12-26T23:13:34.189 INFO:teuthology.orchestra.run.smithi131.stderr:--verbose -- /var/log/ceph/ceph-client.admin.73667.log 2023-12-26T23:13:34.189 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39709.log 2023-12-26T23:13:34.190 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51385.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51385.log.gz 2023-12-26T23:13:34.190 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96351.log 2023-12-26T23:13:34.191 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.3.45603.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-12-26T23:13:34.192 INFO:teuthology.orchestra.run.smithi190.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.45603.log.gz 2023-12-26T23:13:34.192 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88598.log 2023-12-26T23:13:34.192 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73667.log.gz 2023-12-26T23:13:34.192 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.96351.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96351.log.gz 2023-12-26T23:13:34.192 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75546.log 2023-12-26T23:13:34.193 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88598.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88598.log.gz 2023-12-26T23:13:34.193 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96000.log 2023-12-26T23:13:34.193 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75546.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75546.log.gz 2023-12-26T23:13:34.194 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87039.log 2023-12-26T23:13:34.194 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.96000.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96000.log.gz 2023-12-26T23:13:34.194 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2023-12-26T23:13:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87039.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87039.log.gz 2023-12-26T23:13:34.196 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73213.log 2023-12-26T23:13:34.196 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.39709.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39709.log.gz 2023-12-26T23:13:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53276.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz 2023-12-26T23:13:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90074.log 2023-12-26T23:13:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73213.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73213.log.gz 2023-12-26T23:13:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54894.log 2023-12-26T23:13:34.198 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-12-26T23:13:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90074.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90074.log.gz 2023-12-26T23:13:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45287.log 2023-12-26T23:13:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54894.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.54894.log.gz-5 2023-12-26T23:13:34.200 INFO:teuthology.orchestra.run.smithi131.stderr:--verbose -- /var/log/ceph/ceph-client.admin.94416.log 2023-12-26T23:13:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45287.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45287.log.gz 2023-12-26T23:13:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86405.log 2023-12-26T23:13:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94416.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94416.log.gz 2023-12-26T23:13:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75834.log 2023-12-26T23:13:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86405.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86405.log.gz 2023-12-26T23:13:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64472.log 2023-12-26T23:13:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46566.log 2023-12-26T23:13:34.203 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75834.log.gz 2023-12-26T23:13:34.204 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64472.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64472.log.gz 2023-12-26T23:13:34.204 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45669.log 2023-12-26T23:13:34.205 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46566.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46566.log.gz 2023-12-26T23:13:34.205 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-12-26T23:13:34.206 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45669.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45669.log.gz 2023-12-26T23:13:34.206 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66139.log 2023-12-26T23:13:34.207 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103260.log 2023-12-26T23:13:34.208 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66139.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.66139.log.gz 2023-12-26T23:13:34.208 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73775.log 2023-12-26T23:13:34.210 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103260.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103260.log.gz 2023-12-26T23:13:34.210 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86548.log 2023-12-26T23:13:34.211 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53787.log 2023-12-26T23:13:34.211 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73775.log.gz 2023-12-26T23:13:34.212 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58891.log 2023-12-26T23:13:34.212 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86548.log.gz 2023-12-26T23:13:34.213 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53787.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53787.log.gz 2023-12-26T23:13:34.213 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78434.log 2023-12-26T23:13:34.214 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58891.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58891.log.gz 2023-12-26T23:13:34.214 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55998.log 2023-12-26T23:13:34.215 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44837.log 2023-12-26T23:13:34.216 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78434.log.gz 2023-12-26T23:13:34.216 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55998.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55998.log.gz 2023-12-26T23:13:34.217 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59120.log 2023-12-26T23:13:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44837.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44837.log.gz 2023-12-26T23:13:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86181.log 2023-12-26T23:13:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59120.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59120.log.gzgzip 2023-12-26T23:13:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.79023.log 2023-12-26T23:13:34.220 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86181.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86181.log.gz 2023-12-26T23:13:34.220 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49863.log 2023-12-26T23:13:34.221 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100848.log 2023-12-26T23:13:34.221 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79023.log.gz 2023-12-26T23:13:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49863.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.49863.log.gz 2023-12-26T23:13:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.69494.log 2023-12-26T23:13:34.223 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100848.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100848.log.gz 2023-12-26T23:13:34.223 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46532.log 2023-12-26T23:13:34.225 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69494.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69494.log.gz 2023-12-26T23:13:34.225 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68568.log 2023-12-26T23:13:34.226 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47233.log 2023-12-26T23:13:34.226 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46532.log.gz 2023-12-26T23:13:34.227 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68568.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68568.log.gz 2023-12-26T23:13:34.227 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50339.log 2023-12-26T23:13:34.228 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47233.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47233.log.gz 2023-12-26T23:13:34.228 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68988.log 2023-12-26T23:13:34.229 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50339.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50339.log.gz 2023-12-26T23:13:34.229 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73821.log 2023-12-26T23:13:34.230 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68988.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68988.log.gz 2023-12-26T23:13:34.230 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103583.log 2023-12-26T23:13:34.232 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73821.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73821.log.gz 2023-12-26T23:13:34.232 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64265.log 2023-12-26T23:13:34.233 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46817.log 2023-12-26T23:13:34.233 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103583.log.gz 2023-12-26T23:13:34.234 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64265.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64265.log.gz 2023-12-26T23:13:34.234 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93898.log 2023-12-26T23:13:34.235 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46817.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46817.log.gz 2023-12-26T23:13:34.235 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71045.log 2023-12-26T23:13:34.236 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93898.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93898.log.gz 2023-12-26T23:13:34.236 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84980.log 2023-12-26T23:13:34.237 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71045.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71045.log.gz 2023-12-26T23:13:34.237 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69218.log 2023-12-26T23:13:34.238 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.84980.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84980.log.gz 2023-12-26T23:13:34.239 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79773.log 2023-12-26T23:13:34.239 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69218.log: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.69218.log.gz--verbose 2023-12-26T23:13:34.239 INFO:teuthology.orchestra.run.smithi131.stderr:-- /var/log/ceph/ceph-client.admin.47279.log 2023-12-26T23:13:34.241 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45899.log 2023-12-26T23:13:34.241 INFO:teuthology.orchestra.run.smithi131.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.79773.log.gz 2023-12-26T23:13:34.242 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47279.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47279.log.gz 2023-12-26T23:13:34.242 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45148.log 2023-12-26T23:13:34.243 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70173.log 2023-12-26T23:13:34.243 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45899.log.gz 2023-12-26T23:13:34.244 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45148.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45148.log.gz 2023-12-26T23:13:34.244 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54940.log 2023-12-26T23:13:34.245 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70173.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70173.log.gz 2023-12-26T23:13:34.245 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68132.log 2023-12-26T23:13:34.246 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54940.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54940.log.gz 2023-12-26T23:13:34.246 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59796.log 2023-12-26T23:13:34.247 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69402.log 2023-12-26T23:13:34.247 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68132.log.gz 2023-12-26T23:13:34.248 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59796.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59796.log.gz 2023-12-26T23:13:34.249 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49265.log 2023-12-26T23:13:34.250 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69402.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69402.log.gz 2023-12-26T23:13:34.250 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54618.log 2023-12-26T23:13:34.251 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49265.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49265.log.gz 2023-12-26T23:13:34.251 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94369.log 2023-12-26T23:13:34.252 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59842.log 2023-12-26T23:13:34.252 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54618.log.gz 2023-12-26T23:13:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94369.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94369.log.gz 2023-12-26T23:13:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99749.log 2023-12-26T23:13:34.254 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59842.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59842.log.gz 2023-12-26T23:13:34.255 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69540.log 2023-12-26T23:13:34.255 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99749.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99749.log.gz 2023-12-26T23:13:34.256 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75360.log 2023-12-26T23:13:34.257 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69540.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69540.log.gz 2023-12-26T23:13:34.257 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89656.log 2023-12-26T23:13:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75360.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75360.log.gz 2023-12-26T23:13:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72929.log 2023-12-26T23:13:34.259 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89656.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.89656.log.gz 2023-12-26T23:13:34.259 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51061.log 2023-12-26T23:13:34.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72929.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72929.log.gz 2023-12-26T23:13:34.260 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74930.log 2023-12-26T23:13:34.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51061.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.51061.log.gz-5 2023-12-26T23:13:34.262 INFO:teuthology.orchestra.run.smithi131.stderr:--verbose -- /var/log/ceph/ceph-client.admin.79403.log 2023-12-26T23:13:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74930.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74930.log.gz 2023-12-26T23:13:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88136.log 2023-12-26T23:13:34.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79403.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79403.log.gz 2023-12-26T23:13:34.264 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93601.log 2023-12-26T23:13:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100476.log 2023-12-26T23:13:34.265 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88136.log.gz 2023-12-26T23:13:34.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93601.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93601.log.gz 2023-12-26T23:13:34.266 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76766.log 2023-12-26T23:13:34.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100476.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100476.log.gz 2023-12-26T23:13:34.268 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65711.log 2023-12-26T23:13:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76766.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76766.log.gz 2023-12-26T23:13:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58140.log 2023-12-26T23:13:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65711.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65711.log.gz 2023-12-26T23:13:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78026.log 2023-12-26T23:13:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58140.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58140.log.gz 2023-12-26T23:13:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66961.log 2023-12-26T23:13:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78026.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78026.log.gz 2023-12-26T23:13:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80127.log 2023-12-26T23:13:34.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77416.log 2023-12-26T23:13:34.274 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66961.log.gz 2023-12-26T23:13:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.80127.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80127.log.gz 2023-12-26T23:13:34.275 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78480.log 2023-12-26T23:13:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77416.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77416.log.gz 2023-12-26T23:13:34.276 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72042.log 2023-12-26T23:13:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78480.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78480.log.gz 2023-12-26T23:13:34.277 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66047.log 2023-12-26T23:13:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72042.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72042.log.gz 2023-12-26T23:13:34.278 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87177.log 2023-12-26T23:13:34.279 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66047.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66047.log.gz 2023-12-26T23:13:34.279 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94895.log 2023-12-26T23:13:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87177.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.87177.log.gz 2023-12-26T23:13:34.281 INFO:teuthology.orchestra.run.smithi131.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.88340.log 2023-12-26T23:13:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94895.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94895.log.gz 2023-12-26T23:13:34.282 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66277.log 2023-12-26T23:13:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88340.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88340.log.gz 2023-12-26T23:13:34.283 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74976.log 2023-12-26T23:13:34.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66277.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66277.log.gz 2023-12-26T23:13:34.284 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80293.log 2023-12-26T23:13:34.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74976.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74976.log.gz 2023-12-26T23:13:34.285 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101333.log 2023-12-26T23:13:34.286 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.80293.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80293.log.gz 2023-12-26T23:13:34.286 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65757.log 2023-12-26T23:13:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101333.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101333.log.gz 2023-12-26T23:13:34.288 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102780.log 2023-12-26T23:13:34.288 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65757.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65757.log.gz 2023-12-26T23:13:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46554.log 2023-12-26T23:13:34.290 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102780.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102780.log.gzgzip 2023-12-26T23:13:34.290 INFO:teuthology.orchestra.run.smithi131.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.59934.log 2023-12-26T23:13:34.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46554.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46554.log.gz 2023-12-26T23:13:34.291 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55448.log 2023-12-26T23:13:34.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59934.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59934.log.gz 2023-12-26T23:13:34.292 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46195.log 2023-12-26T23:13:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55448.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55448.log.gz 2023-12-26T23:13:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78344.log 2023-12-26T23:13:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68408.log 2023-12-26T23:13:34.294 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46195.log.gz 2023-12-26T23:13:34.295 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78344.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78344.log.gz 2023-12-26T23:13:34.295 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100025.log 2023-12-26T23:13:34.297 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68408.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68408.log.gz 2023-12-26T23:13:34.297 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45761.log 2023-12-26T23:13:34.298 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100025.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100025.log.gz 2023-12-26T23:13:34.298 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60941.log 2023-12-26T23:13:34.299 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93496.log 2023-12-26T23:13:34.299 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45761.log.gz 2023-12-26T23:13:34.300 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60941.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60941.log.gz 2023-12-26T23:13:34.300 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101733.log 2023-12-26T23:13:34.301 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93496.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93496.log.gz 2023-12-26T23:13:34.301 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60275.log 2023-12-26T23:13:34.302 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101733.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101733.log.gz 2023-12-26T23:13:34.302 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84840.log 2023-12-26T23:13:34.303 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60275.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60275.log.gz 2023-12-26T23:13:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90745.log 2023-12-26T23:13:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.84840.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84840.log.gz 2023-12-26T23:13:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50634.log 2023-12-26T23:13:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.90745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.90745.log.gz 2023-12-26T23:13:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78664.log 2023-12-26T23:13:34.307 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50634.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50634.log.gz 2023-12-26T23:13:34.307 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-12-26T23:13:34.308 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78664.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78664.log.gz 2023-12-26T23:13:34.308 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70053.log 2023-12-26T23:13:34.309 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104227.log 2023-12-26T23:13:34.310 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70053.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70053.log.gz 2023-12-26T23:13:34.310 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69126.log 2023-12-26T23:13:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104227.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104227.log.gz 2023-12-26T23:13:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60094.log 2023-12-26T23:13:34.312 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69126.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69126.log.gz 2023-12-26T23:13:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66769.log 2023-12-26T23:13:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60094.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60094.log.gz 2023-12-26T23:13:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65168.log 2023-12-26T23:13:34.315 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66769.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66769.log.gz 2023-12-26T23:13:34.315 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58752.log 2023-12-26T23:13:34.316 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65168.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65168.log.gz 2023-12-26T23:13:34.316 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70900.log 2023-12-26T23:13:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58752.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58752.log.gz 2023-12-26T23:13:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99556.log 2023-12-26T23:13:34.318 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70900.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70900.log.gz 2023-12-26T23:13:34.319 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101617.log 2023-12-26T23:13:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99556.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99556.log.gz 2023-12-26T23:13:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55580.log 2023-12-26T23:13:34.321 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101617.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101617.log.gz 2023-12-26T23:13:34.321 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88755.log 2023-12-26T23:13:34.323 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55580.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55580.log.gz 2023-12-26T23:13:34.323 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75068.log 2023-12-26T23:13:34.324 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88755.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88755.log.gz 2023-12-26T23:13:34.324 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-osd.4.log: 93.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-12-26T23:13:34.325 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49080.log 2023-12-26T23:13:34.326 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75068.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75068.log.gz 2023-12-26T23:13:34.326 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76065.log 2023-12-26T23:13:34.327 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49080.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49080.log.gz 2023-12-26T23:13:34.327 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69986.log 2023-12-26T23:13:34.328 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76065.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76065.log.gz 2023-12-26T23:13:34.329 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100756.log 2023-12-26T23:13:34.330 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69986.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69986.log.gz 2023-12-26T23:13:34.330 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85387.log 2023-12-26T23:13:34.331 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100756.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100756.log.gz 2023-12-26T23:13:34.332 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103214.log 2023-12-26T23:13:34.333 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85387.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85387.log.gz 2023-12-26T23:13:34.333 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87269.log 2023-12-26T23:13:34.334 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103214.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103214.log.gz 2023-12-26T23:13:34.334 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103306.log 2023-12-26T23:13:34.336 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45991.log 2023-12-26T23:13:34.336 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.87269.log.gz 2023-12-26T23:13:34.336 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103306.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103306.log.gz 2023-12-26T23:13:34.337 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67677.log 2023-12-26T23:13:34.338 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45991.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45991.log.gz 2023-12-26T23:13:34.338 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51433.log 2023-12-26T23:13:34.339 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67677.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67677.log.gz 2023-12-26T23:13:34.339 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53390.log 2023-12-26T23:13:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51433.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51433.log.gz 2023-12-26T23:13:34.341 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104365.log 2023-12-26T23:13:34.342 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53390.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53390.log.gz 2023-12-26T23:13:34.342 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70451.log 2023-12-26T23:13:34.343 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99028.log 2023-12-26T23:13:34.343 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104365.log.gz 2023-12-26T23:13:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70451.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.70451.log.gz 2023-12-26T23:13:34.344 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90842.log 2023-12-26T23:13:34.345 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99028.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99028.log.gz 2023-12-26T23:13:34.346 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88547.log 2023-12-26T23:13:34.346 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90842.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90842.log.gz 2023-12-26T23:13:34.346 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93254.log 2023-12-26T23:13:34.348 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88547.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88547.log.gz 2023-12-26T23:13:34.348 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85552.log 2023-12-26T23:13:34.349 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93254.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93254.log.gz 2023-12-26T23:13:34.349 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95783.log 2023-12-26T23:13:34.350 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85552.log: gzip 89.6%-5 -- replaced with /var/log/ceph/ceph-client.admin.85552.log.gz--verbose 2023-12-26T23:13:34.350 INFO:teuthology.orchestra.run.smithi131.stderr:-- /var/log/ceph/ceph-client.admin.49909.log 2023-12-26T23:13:34.350 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.95783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.95783.log.gz 2023-12-26T23:13:34.351 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104503.log 2023-12-26T23:13:34.352 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49909.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49909.log.gz 2023-12-26T23:13:34.352 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67625.log 2023-12-26T23:13:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104503.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104503.log.gz 2023-12-26T23:13:34.353 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94277.log 2023-12-26T23:13:34.355 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67625.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67625.log.gz 2023-12-26T23:13:34.355 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47555.log 2023-12-26T23:13:34.356 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94277.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94277.log.gz 2023-12-26T23:13:34.356 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94185.log 2023-12-26T23:13:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100430.log 2023-12-26T23:13:34.357 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47555.log.gz 2023-12-26T23:13:34.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94185.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94185.log.gz 2023-12-26T23:13:34.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72192.log 2023-12-26T23:13:34.359 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100430.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100430.log.gz 2023-12-26T23:13:34.359 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102422.log 2023-12-26T23:13:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72192.log.gz 2023-12-26T23:13:34.360 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93848.log 2023-12-26T23:13:34.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102422.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102422.log.gz 2023-12-26T23:13:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45715.log 2023-12-26T23:13:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93848.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93848.log.gz 2023-12-26T23:13:34.362 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45427.log 2023-12-26T23:13:34.364 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45715.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45715.log.gz 2023-12-26T23:13:34.364 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59367.log 2023-12-26T23:13:34.365 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45427.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45427.log.gz 2023-12-26T23:13:34.365 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69034.log 2023-12-26T23:13:34.366 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59367.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59367.log.gz 2023-12-26T23:13:34.366 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50109.log 2023-12-26T23:13:34.367 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69034.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69034.log.gz 2023-12-26T23:13:34.367 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85837.log 2023-12-26T23:13:34.368 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50109.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50109.log.gz 2023-12-26T23:13:34.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103951.log 2023-12-26T23:13:34.369 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99439.log 2023-12-26T23:13:34.370 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85837.log.gz 2023-12-26T23:13:34.371 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103951.log: gzip 88.1%-5 -- replaced with /var/log/ceph/ceph-client.admin.103951.log.gz--verbose 2023-12-26T23:13:34.371 INFO:teuthology.orchestra.run.smithi131.stderr:-- /var/log/ceph/ceph-client.admin.49126.log 2023-12-26T23:13:34.372 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99439.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99439.log.gz 2023-12-26T23:13:34.372 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65122.log 2023-12-26T23:13:34.373 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49126.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49126.log.gz 2023-12-26T23:13:34.373 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55382.log 2023-12-26T23:13:34.374 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65122.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65122.log.gz 2023-12-26T23:13:34.374 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88201.log 2023-12-26T23:13:34.375 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55382.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55382.log.gz 2023-12-26T23:13:34.375 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90796.log 2023-12-26T23:13:34.376 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100618.log 2023-12-26T23:13:34.377 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88201.log.gz 2023-12-26T23:13:34.377 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90796.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90796.log.gz 2023-12-26T23:13:34.377 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99795.log 2023-12-26T23:13:34.379 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100618.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100618.log.gz 2023-12-26T23:13:34.379 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74165.log 2023-12-26T23:13:34.379 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99795.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99795.log.gz 2023-12-26T23:13:34.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60757.log 2023-12-26T23:13:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93445.log 2023-12-26T23:13:34.381 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74165.log.gz 2023-12-26T23:13:34.382 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60757.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60757.log.gz 2023-12-26T23:13:34.382 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104319.log 2023-12-26T23:13:34.383 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93445.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93445.log.gz 2023-12-26T23:13:34.383 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74006.log 2023-12-26T23:13:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104319.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104319.log.gz 2023-12-26T23:13:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61241.log 2023-12-26T23:13:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98731.log 2023-12-26T23:13:34.386 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74006.log.gz 2023-12-26T23:13:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61241.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61241.log.gz 2023-12-26T23:13:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59750.log 2023-12-26T23:13:34.388 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.98731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86947.log 2023-12-26T23:13:34.388 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98731.log.gz 2023-12-26T23:13:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59750.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59750.log.gz 2023-12-26T23:13:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46187.log 2023-12-26T23:13:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86947.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86947.log.gz 2023-12-26T23:13:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60803.log 2023-12-26T23:13:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46187.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46187.log.gz 2023-12-26T23:13:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67822.log 2023-12-26T23:13:34.393 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60803.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60803.log.gz 2023-12-26T23:13:34.393 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95110.log 2023-12-26T23:13:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67822.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67822.log.gz 2023-12-26T23:13:34.394 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53075.log 2023-12-26T23:13:34.395 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46545.log 2023-12-26T23:13:34.395 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95110.log.gz 2023-12-26T23:13:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53075.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53075.log.gz 2023-12-26T23:13:34.396 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73379.log 2023-12-26T23:13:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102976.log 2023-12-26T23:13:34.398 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46545.log.gz 2023-12-26T23:13:34.398 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73379.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73379.log.gz 2023-12-26T23:13:34.399 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51847.log 2023-12-26T23:13:34.400 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99342.log 2023-12-26T23:13:34.400 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102976.log.gz 2023-12-26T23:13:34.401 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51847.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51847.log.gz 2023-12-26T23:13:34.401 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89410.log 2023-12-26T23:13:34.402 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99342.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99342.log.gz 2023-12-26T23:13:34.402 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65805.log 2023-12-26T23:13:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89410.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89410.log.gz 2023-12-26T23:13:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70123.log 2023-12-26T23:13:34.404 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65805.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65805.log.gz 2023-12-26T23:13:34.405 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104580.log 2023-12-26T23:13:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70123.log: gzip 89.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.70123.log.gz 2023-12-26T23:13:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:--verbose -- /var/log/ceph/ceph-client.admin.58446.log 2023-12-26T23:13:34.407 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93800.log 2023-12-26T23:13:34.407 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104580.log.gz 2023-12-26T23:13:34.408 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58446.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58446.log.gz 2023-12-26T23:13:34.408 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52242.log 2023-12-26T23:13:34.409 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93800.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-12-26T23:13:34.409 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93800.log.gz 2023-12-26T23:13:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.52242.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52242.log.gz 2023-12-26T23:13:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90028.log 2023-12-26T23:13:34.411 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51709.log 2023-12-26T23:13:34.412 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90028.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90028.log.gz 2023-12-26T23:13:34.413 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95550.log 2023-12-26T23:13:34.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44933.log 2023-12-26T23:13:34.414 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51709.log.gz 2023-12-26T23:13:34.415 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95550.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95550.log.gz 2023-12-26T23:13:34.415 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95158.log 2023-12-26T23:13:34.416 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102474.log 2023-12-26T23:13:34.417 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44933.log.gz 2023-12-26T23:13:34.418 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95158.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95158.log.gz 2023-12-26T23:13:34.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49449.log 2023-12-26T23:13:34.419 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83654.log 2023-12-26T23:13:34.419 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102474.log.gz 2023-12-26T23:13:34.420 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49449.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49449.log.gz 2023-12-26T23:13:34.420 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69632.log 2023-12-26T23:13:34.422 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.83654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61784.log 2023-12-26T23:13:34.422 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83654.log.gz 2023-12-26T23:13:34.423 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68011.log 2023-12-26T23:13:34.423 INFO:teuthology.orchestra.run.smithi131.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.69632.log.gz 2023-12-26T23:13:34.424 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61784.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61784.log.gz 2023-12-26T23:13:34.424 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77877.log 2023-12-26T23:13:34.426 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85083.log 2023-12-26T23:13:34.426 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68011.log.gz 2023-12-26T23:13:34.427 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77877.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77877.log.gz 2023-12-26T23:13:34.428 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61592.log 2023-12-26T23:13:34.428 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85083.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85083.log.gz 2023-12-26T23:13:34.429 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59704.log 2023-12-26T23:13:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61592.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61592.log.gz 2023-12-26T23:13:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89226.log 2023-12-26T23:13:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59704.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59704.log.gz 2023-12-26T23:13:34.432 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60665.log 2023-12-26T23:13:34.433 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89226.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89226.log.gz 2023-12-26T23:13:34.433 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85975.log 2023-12-26T23:13:34.435 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60665.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60665.log.gz 2023-12-26T23:13:34.435 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50155.log 2023-12-26T23:13:34.436 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85975.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85975.log.gz 2023-12-26T23:13:34.437 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49219.log 2023-12-26T23:13:34.437 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50155.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50155.log.gz 2023-12-26T23:13:34.438 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93947.log 2023-12-26T23:13:34.439 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49219.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49219.log.gz 2023-12-26T23:13:34.440 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59072.log 2023-12-26T23:13:34.440 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93947.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93947.log.gz 2023-12-26T23:13:34.441 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100572.log 2023-12-26T23:13:34.442 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59072.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59072.log.gz 2023-12-26T23:13:34.443 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60711.log 2023-12-26T23:13:34.444 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100572.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100572.log.gz 2023-12-26T23:13:34.444 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64367.log 2023-12-26T23:13:34.445 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60711.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60711.log.gz 2023-12-26T23:13:34.446 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86901.log 2023-12-26T23:13:34.447 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64367.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64367.log.gz 2023-12-26T23:13:34.447 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52459.log 2023-12-26T23:13:34.448 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86901.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86901.log.gz 2023-12-26T23:13:34.449 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70590.log 2023-12-26T23:13:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52459.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52459.log.gz 2023-12-26T23:13:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94323.log 2023-12-26T23:13:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89570.log 2023-12-26T23:13:34.452 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70590.log.gz 2023-12-26T23:13:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94323.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94323.log.gz 2023-12-26T23:13:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103675.log 2023-12-26T23:13:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89570.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89570.log.gz 2023-12-26T23:13:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99079.log 2023-12-26T23:13:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93552.log 2023-12-26T23:13:34.456 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103675.log.gz 2023-12-26T23:13:34.457 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99079.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99079.log.gz 2023-12-26T23:13:34.457 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102629.log 2023-12-26T23:13:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93552.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93552.log.gz 2023-12-26T23:13:34.459 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79589.log 2023-12-26T23:13:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102629.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102629.log.gz 2023-12-26T23:13:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46531.log 2023-12-26T23:13:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79589.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79589.log.gz 2023-12-26T23:13:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68838.log 2023-12-26T23:13:34.463 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46531.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46531.log.gz 2023-12-26T23:13:34.463 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51755.log 2023-12-26T23:13:34.465 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68838.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68838.log.gz 2023-12-26T23:13:34.465 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74316.log 2023-12-26T23:13:34.466 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51755.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-12-26T23:13:34.466 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51755.log.gz 2023-12-26T23:13:34.467 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92136.log 2023-12-26T23:13:34.468 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74316.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74316.log.gz 2023-12-26T23:13:34.468 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66369.log 2023-12-26T23:13:34.469 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.92136.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92136.log.gz 2023-12-26T23:13:34.469 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72090.log 2023-12-26T23:13:34.470 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103859.log 2023-12-26T23:13:34.470 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66369.log.gz 2023-12-26T23:13:34.471 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72090.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72090.log.gz 2023-12-26T23:13:34.471 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71428.log 2023-12-26T23:13:34.472 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103859.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103859.log.gz 2023-12-26T23:13:34.472 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-12-26T23:13:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71428.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71428.log.gz 2023-12-26T23:13:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67448.log 2023-12-26T23:13:34.475 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85791.log 2023-12-26T23:13:34.476 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67448.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67448.log.gz 2023-12-26T23:13:34.476 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65306.log 2023-12-26T23:13:34.477 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79843.log 2023-12-26T23:13:34.478 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85791.log.gz 2023-12-26T23:13:34.478 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65306.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65306.log.gz 2023-12-26T23:13:34.479 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92802.log 2023-12-26T23:13:34.479 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80076.log 2023-12-26T23:13:34.480 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79843.log.gz 2023-12-26T23:13:34.481 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.92802.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92802.log.gz 2023-12-26T23:13:34.481 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72399.log 2023-12-26T23:13:34.482 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.80076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64672.log 2023-12-26T23:13:34.482 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80076.log.gz 2023-12-26T23:13:34.483 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72399.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72399.log.gz 2023-12-26T23:13:34.483 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55250.log 2023-12-26T23:13:34.484 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50247.log 2023-12-26T23:13:34.485 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64672.log.gz 2023-12-26T23:13:34.486 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55250.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55250.log.gz 2023-12-26T23:13:34.486 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100206.log 2023-12-26T23:13:34.487 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45194.log 2023-12-26T23:13:34.487 INFO:teuthology.orchestra.run.smithi131.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50247.log.gz 2023-12-26T23:13:34.489 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85692.log 2023-12-26T23:13:34.489 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100206.log.gz 2023-12-26T23:13:34.490 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45194.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45194.log.gz 2023-12-26T23:13:34.490 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-26T23:13:34.491 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85692.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85692.log.gz 2023-12-26T23:13:34.491 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68792.log 2023-12-26T23:13:34.493 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53531.log 2023-12-26T23:13:34.494 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68792.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68792.log.gz 2023-12-26T23:13:34.494 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74715.log 2023-12-26T23:13:34.496 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72721.log 2023-12-26T23:13:34.496 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53531.log.gz 2023-12-26T23:13:34.496 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74715.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.74715.log.gz 2023-12-26T23:13:34.497 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46563.log 2023-12-26T23:13:34.498 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71095.log 2023-12-26T23:13:34.498 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72721.log.gz 2023-12-26T23:13:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46563.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46563.log.gz 2023-12-26T23:13:34.500 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71772.log 2023-12-26T23:13:34.501 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71095.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71095.log.gz 2023-12-26T23:13:34.501 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52339.log 2023-12-26T23:13:34.502 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-26T23:13:34.503 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94708.log 2023-12-26T23:13:34.503 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71772.log.gz 2023-12-26T23:13:34.503 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52339.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52339.log.gz 2023-12-26T23:13:34.504 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88442.log 2023-12-26T23:13:34.505 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89029.log 2023-12-26T23:13:34.505 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94708.log.gz 2023-12-26T23:13:34.506 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88442.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.88442.log.gz 2023-12-26T23:13:34.506 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49679.log 2023-12-26T23:13:34.508 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85929.log 2023-12-26T23:13:34.508 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89029.log.gz 2023-12-26T23:13:34.509 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49679.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49679.log.gz 2023-12-26T23:13:34.509 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70642.log 2023-12-26T23:13:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102176.log 2023-12-26T23:13:34.510 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85929.log.gz 2023-12-26T23:13:34.511 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70642.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70642.log.gz 2023-12-26T23:13:34.512 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66185.log 2023-12-26T23:13:34.513 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99135.log 2023-12-26T23:13:34.513 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102176.log.gz 2023-12-26T23:13:34.514 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66185.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66185.log.gz 2023-12-26T23:13:34.514 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50856.log 2023-12-26T23:13:34.515 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68270.log 2023-12-26T23:13:34.516 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99135.log.gz 2023-12-26T23:13:34.517 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50856.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50856.log.gz 2023-12-26T23:13:34.517 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95388.log 2023-12-26T23:13:34.518 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66093.log 2023-12-26T23:13:34.518 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68270.log.gz 2023-12-26T23:13:34.519 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95388.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95388.log.gz 2023-12-26T23:13:34.519 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66889.log 2023-12-26T23:13:34.520 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2023-12-26T23:13:34.521 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66093.log.gz 2023-12-26T23:13:34.521 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66889.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66889.log.gz 2023-12-26T23:13:34.522 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64523.log 2023-12-26T23:13:34.522 INFO:teuthology.orchestra.run.smithi190.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-12-26T23:13:34.523 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78076.log 2023-12-26T23:13:34.524 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2023-12-26T23:13:34.524 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64523.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64523.log.gz 2023-12-26T23:13:34.524 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94042.log 2023-12-26T23:13:34.525 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53343.log 2023-12-26T23:13:34.526 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78076.log.gz 2023-12-26T23:13:34.527 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94042.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94042.log.gz 2023-12-26T23:13:34.527 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88390.log 2023-12-26T23:13:34.528 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104273.log 2023-12-26T23:13:34.528 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53343.log.gz 2023-12-26T23:13:34.529 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88390.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88390.log.gz 2023-12-26T23:13:34.530 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99286.log 2023-12-26T23:13:34.531 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89364.log 2023-12-26T23:13:34.531 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104273.log.gz 2023-12-26T23:13:34.532 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99286.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99286.log.gz 2023-12-26T23:13:34.532 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102581.log 2023-12-26T23:13:34.533 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90908.log 2023-12-26T23:13:34.534 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.89364.log.gz 2023-12-26T23:13:34.534 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102581.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102581.log.gz 2023-12-26T23:13:34.534 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-12-26T23:13:34.536 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70749.log 2023-12-26T23:13:34.536 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90908.log.gz 2023-12-26T23:13:34.537 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56304.log 2023-12-26T23:13:34.539 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71950.log 2023-12-26T23:13:34.539 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70749.log.gz 2023-12-26T23:13:34.540 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56304.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56304.log.gz 2023-12-26T23:13:34.541 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61171.log 2023-12-26T23:13:34.542 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71950.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71950.log.gz 2023-12-26T23:13:34.542 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71336.log 2023-12-26T23:13:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79357.log 2023-12-26T23:13:34.545 INFO:teuthology.orchestra.run.smithi131.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61171.log.gz 2023-12-26T23:13:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71336.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71336.log.gz 2023-12-26T23:13:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103352.log 2023-12-26T23:13:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79357.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79357.log.gz 2023-12-26T23:13:34.547 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71996.log 2023-12-26T23:13:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103352.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103352.log.gz 2023-12-26T23:13:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65076.log 2023-12-26T23:13:34.550 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71996.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71996.log.gz 2023-12-26T23:13:34.550 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85745.log 2023-12-26T23:13:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65076.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65076.log.gz 2023-12-26T23:13:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103813.log 2023-12-26T23:13:34.553 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85745.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85745.log.gz 2023-12-26T23:13:34.553 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53437.log 2023-12-26T23:13:34.554 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103813.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103813.log.gz 2023-12-26T23:13:34.555 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74058.log 2023-12-26T23:13:34.556 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53437.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53437.log.gz 2023-12-26T23:13:34.557 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-12-26T23:13:34.558 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74058.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74058.log.gz 2023-12-26T23:13:34.558 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74511.log 2023-12-26T23:13:34.560 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74818.log 2023-12-26T23:13:34.561 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74511.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74511.log.gz 2023-12-26T23:13:34.561 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101566.log 2023-12-26T23:13:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74818.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74818.log.gz 2023-12-26T23:13:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77721.log 2023-12-26T23:13:34.564 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.101566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101566.log.gz 2023-12-26T23:13:34.564 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75594.log 2023-12-26T23:13:34.566 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77721.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77721.log.gz 2023-12-26T23:13:34.567 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75594.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75594.log.gz 2023-12-26T23:13:34.568 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101078.log 2023-12-26T23:13:34.570 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46921.log 2023-12-26T23:13:34.571 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101078.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101078.log.gz 2023-12-26T23:13:34.571 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48531.log 2023-12-26T23:13:34.572 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46921.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46921.log.gz 2023-12-26T23:13:34.574 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48531.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48531.log.gz 2023-12-26T23:13:34.575 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46176.log 2023-12-26T23:13:34.576 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78572.log 2023-12-26T23:13:34.578 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46176.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46176.log.gz 2023-12-26T23:13:34.579 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78572.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78572.log.gz 2023-12-26T23:13:34.579 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59170.log 2023-12-26T23:13:34.581 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57528.log 2023-12-26T23:13:34.583 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59170.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59170.log.gz 2023-12-26T23:13:34.584 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57528.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57528.log.gz 2023-12-26T23:13:34.584 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85032.log 2023-12-26T23:13:34.586 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93750.log 2023-12-26T23:13:34.588 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85032.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85032.log.gz 2023-12-26T23:13:34.589 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93750.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93750.log.gz 2023-12-26T23:13:34.589 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68178.log 2023-12-26T23:13:34.591 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52293.log 2023-12-26T23:13:34.592 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68178.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68178.log.gz 2023-12-26T23:13:34.594 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52293.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52293.log.gz 2023-12-26T23:13:34.594 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39286.log 2023-12-26T23:13:34.596 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49173.log 2023-12-26T23:13:34.597 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39286.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.39286.log.gz 2023-12-26T23:13:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49173.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49173.log.gz 2023-12-26T23:13:34.599 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99887.log 2023-12-26T23:13:34.601 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47839.log 2023-12-26T23:13:34.602 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99887.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99887.log.gz 2023-12-26T23:13:34.604 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47839.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47839.log.gz 2023-12-26T23:13:34.604 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90350.log 2023-12-26T23:13:34.606 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100664.log 2023-12-26T23:13:34.607 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90350.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.90350.log.gz 2023-12-26T23:13:34.609 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100664.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100664.log.gz 2023-12-26T23:13:34.609 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51337.log 2023-12-26T23:13:34.611 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45623.log 2023-12-26T23:13:34.612 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51337.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51337.log.gz 2023-12-26T23:13:34.614 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45623.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45623.log.gz 2023-12-26T23:13:34.614 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103721.log 2023-12-26T23:13:34.616 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95342.log 2023-12-26T23:13:34.617 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103721.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103721.log.gz 2023-12-26T23:13:34.619 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95342.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95342.log.gz 2023-12-26T23:13:34.619 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64315.log 2023-12-26T23:13:34.621 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89888.log 2023-12-26T23:13:34.622 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64315.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64315.log.gz 2023-12-26T23:13:34.624 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89888.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89888.log.gz 2023-12-26T23:13:34.624 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73329.log 2023-12-26T23:13:34.626 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89982.log 2023-12-26T23:13:34.627 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73329.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73329.log.gz 2023-12-26T23:13:34.628 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74109.log 2023-12-26T23:13:34.629 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89982.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89982.log.gz 2023-12-26T23:13:34.630 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55692.log 2023-12-26T23:13:34.631 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74109.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74109.log.gz 2023-12-26T23:13:34.632 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55692.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55692.log.gz 2023-12-26T23:13:34.632 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86313.log 2023-12-26T23:13:34.635 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102262.log 2023-12-26T23:13:34.636 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75406.log 2023-12-26T23:13:34.636 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86313.log.gz 2023-12-26T23:13:34.638 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102262.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102262.log.gz 2023-12-26T23:13:34.638 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80640.log 2023-12-26T23:13:34.638 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75406.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75406.log.gz 2023-12-26T23:13:34.639 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94846.log 2023-12-26T23:13:34.640 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.80640.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80640.log.gz 2023-12-26T23:13:34.641 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75926.log 2023-12-26T23:13:34.642 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94846.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94846.log.gz 2023-12-26T23:13:34.642 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101783.log 2023-12-26T23:13:34.644 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75926.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75926.log.gz 2023-12-26T23:13:34.644 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101217.log 2023-12-26T23:13:34.646 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101783.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101783.log.gz 2023-12-26T23:13:34.646 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64118.log 2023-12-26T23:13:34.647 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101217.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101217.log.gz 2023-12-26T23:13:34.647 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46916.log 2023-12-26T23:13:34.649 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64118.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64118.log.gz 2023-12-26T23:13:34.650 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62060.log 2023-12-26T23:13:34.650 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46916.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46916.log.gz 2023-12-26T23:13:34.651 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51479.log 2023-12-26T23:13:34.652 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62060.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62060.log.gz 2023-12-26T23:13:34.653 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-12-26T23:13:34.654 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51479.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51479.log.gz 2023-12-26T23:13:34.654 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66672.log 2023-12-26T23:13:34.656 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99236.log 2023-12-26T23:13:34.657 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.66672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66672.log.gz 2023-12-26T23:13:34.658 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104043.log 2023-12-26T23:13:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99236.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99236.log.gz 2023-12-26T23:13:34.661 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102130.log 2023-12-26T23:13:34.664 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104043.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104043.log.gz 2023-12-26T23:13:34.664 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86500.log 2023-12-26T23:13:34.664 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102130.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102130.log.gz 2023-12-26T23:13:34.665 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54756.log 2023-12-26T23:13:34.666 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86500.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86500.log.gz 2023-12-26T23:13:34.668 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80593.log 2023-12-26T23:13:34.669 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54756.log.gz 2023-12-26T23:13:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70947.log 2023-12-26T23:13:34.670 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.80593.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80593.log.gz 2023-12-26T23:13:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76696.log 2023-12-26T23:13:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70947.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70947.log.gz 2023-12-26T23:13:34.674 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78977.log 2023-12-26T23:13:34.675 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76696.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76696.log.gz 2023-12-26T23:13:34.675 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79543.log 2023-12-26T23:13:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78977.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78977.log.gz 2023-12-26T23:13:34.678 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69702.log 2023-12-26T23:13:34.679 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79543.log.gz 2023-12-26T23:13:34.681 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65665.log 2023-12-26T23:13:34.682 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69702.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69702.log.gz 2023-12-26T23:13:34.683 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46539.log 2023-12-26T23:13:34.685 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65665.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65665.log.gz 2023-12-26T23:13:34.686 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90212.log 2023-12-26T23:13:34.686 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46539.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46539.log.gz 2023-12-26T23:13:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72583.log 2023-12-26T23:13:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90212.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90212.log.gz 2023-12-26T23:13:34.690 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64943.log 2023-12-26T23:13:34.690 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72583.log.gz 2023-12-26T23:13:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61546.log 2023-12-26T23:13:34.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64943.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64943.log.gz 2023-12-26T23:13:34.693 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67574.log 2023-12-26T23:13:34.696 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61546.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61546.log.gz 2023-12-26T23:13:34.696 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 /var/log/ceph/ceph-client.admin.67574.log: --verbose -- /var/log/ceph/ceph-client.admin.79115.log 2023-12-26T23:13:34.696 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67574.log.gz 2023-12-26T23:13:34.697 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67774.log 2023-12-26T23:13:34.700 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79115.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79115.log.gz 2023-12-26T23:13:34.700 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87670.log 2023-12-26T23:13:34.702 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67774.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67774.log.gz 2023-12-26T23:13:34.703 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 /var/log/ceph/ceph-client.admin.87670.log: --verbose -- /var/log/ceph/ceph-client.admin.72537.log 2023-12-26T23:13:34.703 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87670.log.gz 2023-12-26T23:13:34.705 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102083.log 2023-12-26T23:13:34.705 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72537.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72537.log.gz 2023-12-26T23:13:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102083.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102083.log.gz 2023-12-26T23:13:34.709 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80173.log 2023-12-26T23:13:34.710 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71858.log 2023-12-26T23:13:34.711 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.80173.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80173.log.gz 2023-12-26T23:13:34.714 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71858.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71858.log.gz 2023-12-26T23:13:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59466.log 2023-12-26T23:13:34.716 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49725.log 2023-12-26T23:13:34.717 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59466.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59466.log.gz 2023-12-26T23:13:34.720 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49725.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49725.log.gz 2023-12-26T23:13:34.720 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89480.log 2023-12-26T23:13:34.722 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70997.log 2023-12-26T23:13:34.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89480.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89480.log.gz 2023-12-26T23:13:34.725 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86359.log 2023-12-26T23:13:34.725 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70997.log.gz 2023-12-26T23:13:34.727 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77927.log 2023-12-26T23:13:34.728 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86359.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86359.log.gz 2023-12-26T23:13:34.729 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74363.log 2023-12-26T23:13:34.730 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77927.log.gz 2023-12-26T23:13:34.733 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100384.log 2023-12-26T23:13:34.734 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74363.log.gz 2023-12-26T23:13:34.735 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101855.log 2023-12-26T23:13:34.735 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100384.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100384.log.gz 2023-12-26T23:13:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101855.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101855.log.gz 2023-12-26T23:13:34.739 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90512.log 2023-12-26T23:13:34.742 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90512.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.90512.log.gz 2023-12-26T23:13:34.743 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77522.log 2023-12-26T23:13:34.745 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77522.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77522.log.gz 2023-12-26T23:13:34.746 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49541.log 2023-12-26T23:13:34.749 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49541.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49541.log.gz 2023-12-26T23:13:34.749 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78124.log 2023-12-26T23:13:34.751 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78124.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78124.log.gz 2023-12-26T23:13:34.752 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51801.log 2023-12-26T23:13:34.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51801.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51801.log.gz 2023-12-26T23:13:34.755 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86623.log 2023-12-26T23:13:34.758 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86623.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.86623.log.gz 2023-12-26T23:13:34.758 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76111.log 2023-12-26T23:13:34.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76111.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76111.log.gz 2023-12-26T23:13:34.761 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74771.log 2023-12-26T23:13:34.765 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74771.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74771.log.gz 2023-12-26T23:13:34.765 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91056.log 2023-12-26T23:13:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65999.log 2023-12-26T23:13:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.91056.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91056.log.gz 2023-12-26T23:13:34.769 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95296.log 2023-12-26T23:13:34.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65999.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65999.log.gz 2023-12-26T23:13:34.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95296.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95296.log.gz 2023-12-26T23:13:34.772 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72138.log 2023-12-26T23:13:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72138.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72138.log.gz 2023-12-26T23:13:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79311.log 2023-12-26T23:13:34.778 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79311.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79311.log.gz 2023-12-26T23:13:34.778 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99933.log 2023-12-26T23:13:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99933.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99933.log.gz 2023-12-26T23:13:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75138.log 2023-12-26T23:13:34.784 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55052.log 2023-12-26T23:13:34.786 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75138.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75138.log.gz 2023-12-26T23:13:34.786 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55052.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55052.log.gz 2023-12-26T23:13:34.787 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103767.log 2023-12-26T23:13:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103767.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103767.log.gz 2023-12-26T23:13:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75498.log 2023-12-26T23:13:34.793 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75498.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75498.log.gz 2023-12-26T23:13:34.794 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80365.log 2023-12-26T23:13:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.80365.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80365.log.gz 2023-12-26T23:13:34.797 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65953.log 2023-12-26T23:13:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65953.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65953.log.gz 2023-12-26T23:13:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76816.log 2023-12-26T23:13:34.803 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76816.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76816.log.gz 2023-12-26T23:13:34.803 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66439.log 2023-12-26T23:13:34.806 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66439.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66439.log.gz 2023-12-26T23:13:34.807 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70848.log 2023-12-26T23:13:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70848.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70848.log.gz 2023-12-26T23:13:34.810 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74461.log 2023-12-26T23:13:34.813 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74461.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74461.log.gz 2023-12-26T23:13:34.813 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71230.log 2023-12-26T23:13:34.816 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71230.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71230.log.gz 2023-12-26T23:13:34.816 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-26T23:13:34.820 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87862.log 2023-12-26T23:13:34.823 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46194.log 2023-12-26T23:13:34.824 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87862.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.87862.log.gz 2023-12-26T23:13:34.824 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46194.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46194.log.gz 2023-12-26T23:13:34.824 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86855.log 2023-12-26T23:13:34.828 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86855.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86855.log.gz 2023-12-26T23:13:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60367.log 2023-12-26T23:13:34.834 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67396.log 2023-12-26T23:13:34.834 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66323.log 2023-12-26T23:13:34.834 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60367.log.gz 2023-12-26T23:13:34.834 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-26T23:13:34.835 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67396.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67396.log.gz 2023-12-26T23:13:34.836 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65260.log 2023-12-26T23:13:34.838 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66323.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66323.log.gz 2023-12-26T23:13:34.840 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65260.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65260.log.gz 2023-12-26T23:13:34.840 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75880.log 2023-12-26T23:13:34.843 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50587.log 2023-12-26T23:13:34.843 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75880.log.gz 2023-12-26T23:13:34.847 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50587.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50587.log.gz 2023-12-26T23:13:34.847 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88090.log 2023-12-26T23:13:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88090.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88090.log.gz 2023-12-26T23:13:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79257.log 2023-12-26T23:13:34.854 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79257.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79257.log.gz 2023-12-26T23:13:34.854 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72813.log 2023-12-26T23:13:34.855 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102680.log 2023-12-26T23:13:34.856 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72813.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72813.log.gz 2023-12-26T23:13:34.859 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102680.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102680.log.gz 2023-12-26T23:13:34.859 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75022.log 2023-12-26T23:13:34.860 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71144.log 2023-12-26T23:13:34.864 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75022.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75022.log.gz 2023-12-26T23:13:34.864 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94754.log 2023-12-26T23:13:34.864 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71144.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71144.log.gz 2023-12-26T23:13:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47793.log 2023-12-26T23:13:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73430.log 2023-12-26T23:13:34.868 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.94754.log.gz 2023-12-26T23:13:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47793.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47793.log.gz 2023-12-26T23:13:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60182.log 2023-12-26T23:13:34.871 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73430.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73430.log.gz 2023-12-26T23:13:34.872 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60182.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60182.log.gz 2023-12-26T23:13:34.872 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54710.log 2023-12-26T23:13:34.876 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54710.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54710.log.gz 2023-12-26T23:13:34.876 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75972.log 2023-12-26T23:13:34.879 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75972.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75972.log.gz 2023-12-26T23:13:34.879 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64622.log 2023-12-26T23:13:34.882 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64622.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64622.log.gz 2023-12-26T23:13:34.882 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46180.log 2023-12-26T23:13:34.885 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46180.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46180.log.gz 2023-12-26T23:13:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88853.log 2023-12-26T23:13:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88853.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88853.log.gz 2023-12-26T23:13:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53599.log 2023-12-26T23:13:34.892 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53599.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53599.log.gz 2023-12-26T23:13:34.892 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101032.log 2023-12-26T23:13:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101032.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101032.log.gz 2023-12-26T23:13:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104089.log 2023-12-26T23:13:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49771.log 2023-12-26T23:13:34.900 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87572.log 2023-12-26T23:13:34.901 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49771.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49771.log.gz 2023-12-26T23:13:34.901 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104089.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104089.log.gz 2023-12-26T23:13:34.901 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104457.log 2023-12-26T23:13:34.905 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.87572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.87572.log.gz 2023-12-26T23:13:34.906 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46932.log 2023-12-26T23:13:34.906 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104457.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104457.log.gz 2023-12-26T23:13:34.907 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46932.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46932.log.gz 2023-12-26T23:13:34.907 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53740.log 2023-12-26T23:13:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53740.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53740.log.gz 2023-12-26T23:13:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53209.log 2023-12-26T23:13:34.913 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53209.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53209.log.gz 2023-12-26T23:13:34.913 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95480.log 2023-12-26T23:13:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53834.log 2023-12-26T23:13:34.918 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.95480.log.gz 2023-12-26T23:13:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89748.log 2023-12-26T23:13:34.921 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69264.log 2023-12-26T23:13:34.921 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53834.log.gz 2023-12-26T23:13:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72859.log 2023-12-26T23:13:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89748.log: /var/log/ceph/ceph-client.admin.69264.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89748.log.gz 2023-12-26T23:13:34.923 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69264.log.gz 2023-12-26T23:13:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79681.log 2023-12-26T23:13:34.928 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45945.log 2023-12-26T23:13:34.928 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72859.log.gz 2023-12-26T23:13:34.929 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79681.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79681.log.gz 2023-12-26T23:13:34.929 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94486.log 2023-12-26T23:13:34.932 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45945.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45945.log.gz 2023-12-26T23:13:34.932 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94486.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94486.log.gz 2023-12-26T23:13:34.933 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60229.log 2023-12-26T23:13:34.936 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60229.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60229.log.gz 2023-12-26T23:13:34.936 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68362.log 2023-12-26T23:13:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68362.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68362.log.gz 2023-12-26T23:13:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86452.log 2023-12-26T23:13:34.943 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86452.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86452.log.gz 2023-12-26T23:13:34.943 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70224.log 2023-12-26T23:13:34.946 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75742.log 2023-12-26T23:13:34.947 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70224.log.gz 2023-12-26T23:13:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94964.log 2023-12-26T23:13:34.948 INFO:teuthology.orchestra.run.smithi190.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-12-26T23:13:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54986.log 2023-12-26T23:13:34.949 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75742.log.gz 2023-12-26T23:13:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94964.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94964.log.gz 2023-12-26T23:13:34.954 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76157.log 2023-12-26T23:13:34.954 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54986.log.gz 2023-12-26T23:13:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75452.log 2023-12-26T23:13:34.957 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76157.log.gz 2023-12-26T23:13:34.961 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75452.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75452.log.gz 2023-12-26T23:13:34.961 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51153.log 2023-12-26T23:13:34.965 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51153.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51153.log.gz 2023-12-26T23:13:34.965 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88658.log 2023-12-26T23:13:34.968 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88658.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88658.log.gz 2023-12-26T23:13:34.968 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75648.log 2023-12-26T23:13:34.971 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75648.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75648.log.gz 2023-12-26T23:13:34.971 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102928.log 2023-12-26T23:13:34.974 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102928.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102928.log.gz 2023-12-26T23:13:34.974 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99841.log 2023-12-26T23:13:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62013.log 2023-12-26T23:13:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99841.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99841.log.gz 2023-12-26T23:13:34.979 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63992.log 2023-12-26T23:13:34.980 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62013.log.gz 2023-12-26T23:13:34.981 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46551.log 2023-12-26T23:13:34.982 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63992.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63992.log.gz 2023-12-26T23:13:34.983 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86717.log 2023-12-26T23:13:34.984 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68478.log 2023-12-26T23:13:34.985 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46551.log.gz 2023-12-26T23:13:34.986 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86717.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86717.log.gz 2023-12-26T23:13:34.987 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61495.log 2023-12-26T23:13:34.989 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68478.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68478.log.gz 2023-12-26T23:13:34.989 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.61495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61495.log.gz 2023-12-26T23:13:34.990 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103168.log 2023-12-26T23:13:34.993 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103168.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103168.log.gz 2023-12-26T23:13:34.993 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50948.log 2023-12-26T23:13:34.996 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50948.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50948.log.gz 2023-12-26T23:13:34.996 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90258.log 2023-12-26T23:13:34.999 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90258.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90258.log.gz 2023-12-26T23:13:35.000 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69310.log 2023-12-26T23:13:35.003 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69310.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69310.log.gz 2023-12-26T23:13:35.003 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47141.log 2023-12-26T23:13:35.006 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47141.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47141.log.gz 2023-12-26T23:13:35.006 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76249.log 2023-12-26T23:13:35.009 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76249.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76249.log.gz 2023-12-26T23:13:35.009 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89794.log 2023-12-26T23:13:35.012 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64423.log 2023-12-26T23:13:35.014 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89794.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89794.log.gz 2023-12-26T23:13:35.014 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50680.log 2023-12-26T23:13:35.017 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64423.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64423.log.gz 2023-12-26T23:13:35.017 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45807.log 2023-12-26T23:13:35.017 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50680.log.gz 2023-12-26T23:13:35.018 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49311.log 2023-12-26T23:13:35.020 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45807.log.gz 2023-12-26T23:13:35.020 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47747.log 2023-12-26T23:13:35.024 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49311.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49311.log.gz 2023-12-26T23:13:35.024 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85883.log 2023-12-26T23:13:35.024 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47747.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47747.log.gz 2023-12-26T23:13:35.025 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75314.log 2023-12-26T23:13:35.026 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85883.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85883.log.gz 2023-12-26T23:13:35.027 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51015.log 2023-12-26T23:13:35.028 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75314.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75314.log.gz 2023-12-26T23:13:35.029 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86763.log 2023-12-26T23:13:35.030 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51015.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51015.log.gz 2023-12-26T23:13:35.030 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100894.log 2023-12-26T23:13:35.032 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86763.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86763.log.gz 2023-12-26T23:13:35.033 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85238.log 2023-12-26T23:13:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100894.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100894.log.gz 2023-12-26T23:13:35.035 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45853.log 2023-12-26T23:13:35.036 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85238.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85238.log.gz 2023-12-26T23:13:35.037 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67724.log 2023-12-26T23:13:35.038 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45853.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45853.log.gz 2023-12-26T23:13:35.038 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51245.log 2023-12-26T23:13:35.042 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67724.log: /var/log/ceph/ceph-client.admin.51245.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67724.log.gz 2023-12-26T23:13:35.042 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51245.log.gz 2023-12-26T23:13:35.042 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61712.log 2023-12-26T23:13:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45529.log 2023-12-26T23:13:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61712.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61712.log.gz 2023-12-26T23:13:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45529.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45529.log.gz 2023-12-26T23:13:35.047 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50385.log 2023-12-26T23:13:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64770.log 2023-12-26T23:13:35.050 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50385.log.gz 2023-12-26T23:13:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68746.log 2023-12-26T23:13:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101125.log 2023-12-26T23:13:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64770.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64770.log.gz 2023-12-26T23:13:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68746.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68746.log.gz 2023-12-26T23:13:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47509.log 2023-12-26T23:13:35.057 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101125.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101125.log.gz 2023-12-26T23:13:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47509.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47509.log.gz 2023-12-26T23:13:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75788.log 2023-12-26T23:13:35.062 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101171.log 2023-12-26T23:13:35.063 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75788.log.gz 2023-12-26T23:13:35.063 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90120.log 2023-12-26T23:13:35.066 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75696.log 2023-12-26T23:13:35.066 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101171.log.gz 2023-12-26T23:13:35.067 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90120.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90120.log.gz 2023-12-26T23:13:35.067 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51617.log 2023-12-26T23:13:35.069 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75696.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75696.log.gz 2023-12-26T23:13:35.070 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51617.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51617.log.gz 2023-12-26T23:13:35.070 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86021.log 2023-12-26T23:13:35.073 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86021.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86021.log.gz 2023-12-26T23:13:35.074 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93393.log 2023-12-26T23:13:35.077 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93393.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93393.log.gz 2023-12-26T23:13:35.077 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46933.log 2023-12-26T23:13:35.080 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46933.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46933.log.gz 2023-12-26T23:13:35.081 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98837.log 2023-12-26T23:13:35.083 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.98837.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98837.log.gz 2023-12-26T23:13:35.084 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61662.log 2023-12-26T23:13:35.086 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50063.log 2023-12-26T23:13:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61662.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61662.log.gz 2023-12-26T23:13:35.088 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89134.log 2023-12-26T23:13:35.090 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50063.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50063.log.gz 2023-12-26T23:13:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89134.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89134.log.gz 2023-12-26T23:13:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59319.log 2023-12-26T23:13:35.093 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78911.log 2023-12-26T23:13:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59319.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59319.log.gz 2023-12-26T23:13:35.096 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95204.log 2023-12-26T23:13:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78911.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78911.log.gz 2023-12-26T23:13:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95204.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95204.log.gz 2023-12-26T23:13:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51107.log 2023-12-26T23:13:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51107.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51107.log.gz 2023-12-26T23:13:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50431.log 2023-12-26T23:13:35.104 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50431.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50431.log.gz 2023-12-26T23:13:35.105 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85435.log 2023-12-26T23:13:35.106 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76888.log 2023-12-26T23:13:35.109 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71382.log 2023-12-26T23:13:35.109 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85435.log.gz 2023-12-26T23:13:35.109 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57834.log 2023-12-26T23:13:35.111 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76888.log: /var/log/ceph/ceph-client.admin.71382.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76888.log.gz 2023-12-26T23:13:35.111 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71382.log.gz 2023-12-26T23:13:35.113 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71682.log 2023-12-26T23:13:35.115 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57834.log.gz 2023-12-26T23:13:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54572.log 2023-12-26T23:13:35.118 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71682.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71682.log.gz 2023-12-26T23:13:35.118 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54572.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54572.log.gz 2023-12-26T23:13:35.118 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79451.log 2023-12-26T23:13:35.121 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79451.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79451.log.gz 2023-12-26T23:13:35.122 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65573.log 2023-12-26T23:13:35.125 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65573.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65573.log.gz 2023-12-26T23:13:35.125 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74413.log 2023-12-26T23:13:35.128 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74413.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74413.log.gz 2023-12-26T23:13:35.128 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47325.log 2023-12-26T23:13:35.131 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47325.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47325.log.gz 2023-12-26T23:13:35.132 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71474.log 2023-12-26T23:13:35.133 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79497.log 2023-12-26T23:13:35.136 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66231.log 2023-12-26T23:13:35.136 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71474.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71474.log.gz 2023-12-26T23:13:35.137 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102829.log 2023-12-26T23:13:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79497.log: /var/log/ceph/ceph-client.admin.66231.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79497.log.gz 2023-12-26T23:13:35.138 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66231.log.gz 2023-12-26T23:13:35.140 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98976.log 2023-12-26T23:13:35.142 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102829.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102829.log.gz 2023-12-26T23:13:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.98976.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98976.log.gz 2023-12-26T23:13:35.143 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47371.log 2023-12-26T23:13:35.146 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47371.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47371.log.gz 2023-12-26T23:13:35.146 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88499.log 2023-12-26T23:13:35.149 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88499.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88499.log.gz 2023-12-26T23:13:35.150 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70798.log 2023-12-26T23:13:35.153 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70798.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70798.log.gz 2023-12-26T23:13:35.153 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73867.log 2023-12-26T23:13:35.156 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73867.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73867.log.gz 2023-12-26T23:13:35.156 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46896.log 2023-12-26T23:13:35.159 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46896.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46896.log.gz 2023-12-26T23:13:35.160 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60525.log 2023-12-26T23:13:35.163 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65030.log 2023-12-26T23:13:35.163 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60525.log.gz 2023-12-26T23:13:35.164 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86809.log 2023-12-26T23:13:35.167 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46935.log 2023-12-26T23:13:35.167 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65030.log.gz 2023-12-26T23:13:35.168 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86809.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86809.log.gz 2023-12-26T23:13:35.168 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45241.log 2023-12-26T23:13:35.170 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46935.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46935.log.gz 2023-12-26T23:13:35.172 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45241.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45241.log.gz 2023-12-26T23:13:35.172 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47463.log 2023-12-26T23:13:35.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47463.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47463.log.gz 2023-12-26T23:13:35.175 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54802.log 2023-12-26T23:13:35.178 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54802.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54802.log.gz 2023-12-26T23:13:35.178 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68654.log 2023-12-26T23:13:35.181 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68654.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68654.log.gz 2023-12-26T23:13:35.181 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45379.log 2023-12-26T23:13:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45379.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45379.log.gz 2023-12-26T23:13:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49357.log 2023-12-26T23:13:35.188 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49357.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49357.log.gz 2023-12-26T23:13:35.188 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50902.log 2023-12-26T23:13:35.192 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74264.log 2023-12-26T23:13:35.192 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50902.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50902.log.gz 2023-12-26T23:13:35.192 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74264.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74264.log.gz 2023-12-26T23:13:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49013.log 2023-12-26T23:13:35.196 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79069.log 2023-12-26T23:13:35.197 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49013.log.gz 2023-12-26T23:13:35.197 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.48962.log 2023-12-26T23:13:35.199 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92182.log 2023-12-26T23:13:35.200 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79069.log.gz 2023-12-26T23:13:35.201 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.2.48962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.48962.log.gz 2023-12-26T23:13:35.201 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68700.log 2023-12-26T23:13:35.203 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.92182.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92182.log.gz 2023-12-26T23:13:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68700.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68700.log.gz 2023-12-26T23:13:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92735.log 2023-12-26T23:13:35.208 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.92735.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.92735.log.gz 2023-12-26T23:13:35.208 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89272.log 2023-12-26T23:13:35.211 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89272.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89272.log.gz 2023-12-26T23:13:35.212 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100338.log 2023-12-26T23:13:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100338.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100338.log.gz 2023-12-26T23:13:35.215 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92848.log 2023-12-26T23:13:35.218 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.92848.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92848.log.gz 2023-12-26T23:13:35.218 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86267.log 2023-12-26T23:13:35.221 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86267.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86267.log.gz 2023-12-26T23:13:35.221 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71612.log 2023-12-26T23:13:35.225 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71612.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71612.log.gz 2023-12-26T23:13:35.225 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72238.log 2023-12-26T23:13:35.226 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76598.log 2023-12-26T23:13:35.229 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66835.log 2023-12-26T23:13:35.229 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72238.log.gz 2023-12-26T23:13:35.230 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61079.log 2023-12-26T23:13:35.231 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.76598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76598.log.gz 2023-12-26T23:13:35.231 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66835.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66835.log.gz 2023-12-26T23:13:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61079.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61079.log.gz 2023-12-26T23:13:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103537.log 2023-12-26T23:13:35.237 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103537.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103537.log.gz 2023-12-26T23:13:35.238 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51291.log 2023-12-26T23:13:35.241 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51291.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51291.log.gz 2023-12-26T23:13:35.241 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93651.log 2023-12-26T23:13:35.243 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59015.log 2023-12-26T23:13:35.247 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93651.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93651.log.gz 2023-12-26T23:13:35.247 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47601.log 2023-12-26T23:13:35.247 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59015.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59015.log.gz 2023-12-26T23:13:35.248 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47601.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47601.log.gz 2023-12-26T23:13:35.248 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72491.log 2023-12-26T23:13:35.251 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72491.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72491.log.gz 2023-12-26T23:13:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103905.log 2023-12-26T23:13:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71904.log 2023-12-26T23:13:35.256 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103905.log.gz 2023-12-26T23:13:35.256 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56916.log 2023-12-26T23:13:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59658.log 2023-12-26T23:13:35.259 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71904.log.gz 2023-12-26T23:13:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56916.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56916.log.gz 2023-12-26T23:13:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102877.log 2023-12-26T23:13:35.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59658.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59658.log.gz 2023-12-26T23:13:35.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102877.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102877.log.gz 2023-12-26T23:13:35.264 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88803.log 2023-12-26T23:13:35.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56610.log 2023-12-26T23:13:35.268 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88803.log.gz 2023-12-26T23:13:35.268 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85187.log 2023-12-26T23:13:35.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51199.log 2023-12-26T23:13:35.271 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56610.log.gz 2023-12-26T23:13:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85187.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85187.log.gz 2023-12-26T23:13:35.272 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50523.log 2023-12-26T23:13:35.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51199.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51199.log.gz 2023-12-26T23:13:35.275 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50523.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50523.log.gz 2023-12-26T23:13:35.276 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79727.log 2023-12-26T23:13:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79727.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79727.log.gz 2023-12-26T23:13:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87085.log 2023-12-26T23:13:35.281 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65907.log 2023-12-26T23:13:35.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80243.log 2023-12-26T23:13:35.283 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87085.log.gz 2023-12-26T23:13:35.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65907.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65907.log.gz 2023-12-26T23:13:35.285 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45063.log 2023-12-26T23:13:35.287 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.80243.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80243.log.gz 2023-12-26T23:13:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45063.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45063.log.gz 2023-12-26T23:13:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50201.log 2023-12-26T23:13:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50201.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50201.log.gz 2023-12-26T23:13:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79209.log 2023-12-26T23:13:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79209.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79209.log.gz 2023-12-26T23:13:35.295 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46196.log 2023-12-26T23:13:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46196.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46196.log.gz 2023-12-26T23:13:35.299 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99703.log 2023-12-26T23:13:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47698.log 2023-12-26T23:13:35.302 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69935.log 2023-12-26T23:13:35.303 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99703.log.gz 2023-12-26T23:13:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47698.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47698.log.gz 2023-12-26T23:13:35.306 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.69935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69935.log.gz 2023-12-26T23:13:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87223.log 2023-12-26T23:13:35.309 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85290.log 2023-12-26T23:13:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87223.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87223.log.gz 2023-12-26T23:13:35.311 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69080.log 2023-12-26T23:13:35.313 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85290.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.85290.log.gz 2023-12-26T23:13:35.314 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69080.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69080.log.gz 2023-12-26T23:13:35.314 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99391.log 2023-12-26T23:13:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86671.log 2023-12-26T23:13:35.318 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99391.log.gz 2023-12-26T23:13:35.318 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100986.log 2023-12-26T23:13:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100710.log 2023-12-26T23:13:35.321 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86671.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86671.log.gz 2023-12-26T23:13:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100986.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100986.log.gz 2023-12-26T23:13:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76019.log 2023-12-26T23:13:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100710.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100710.log.gz 2023-12-26T23:13:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76019.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76019.log.gz 2023-12-26T23:13:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73259.log 2023-12-26T23:13:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73259.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73259.log.gz 2023-12-26T23:13:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68886.log 2023-12-26T23:13:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68886.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68886.log.gz 2023-12-26T23:13:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51571.log 2023-12-26T23:13:35.336 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51571.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51571.log.gz 2023-12-26T23:13:35.336 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64720.log 2023-12-26T23:13:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64720.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64720.log.gz 2023-12-26T23:13:35.340 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59270.log 2023-12-26T23:13:35.343 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59270.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59270.log.gz 2023-12-26T23:13:35.343 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52941.log 2023-12-26T23:13:35.346 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52941.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52941.log.gz 2023-12-26T23:13:35.347 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52409.log 2023-12-26T23:13:35.350 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52409.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52409.log.gz 2023-12-26T23:13:35.350 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60321.log 2023-12-26T23:13:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60321.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60321.log.gz 2023-12-26T23:13:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65214.log 2023-12-26T23:13:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78780.log 2023-12-26T23:13:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65214.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65214.log.gz 2023-12-26T23:13:35.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52009.log 2023-12-26T23:13:35.360 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78780.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78780.log.gz 2023-12-26T23:13:35.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52009.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52009.log.gz 2023-12-26T23:13:35.361 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68316.log 2023-12-26T23:13:35.364 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68316.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68316.log.gz 2023-12-26T23:13:35.365 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103629.log 2023-12-26T23:13:35.368 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78241.log 2023-12-26T23:13:35.369 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103629.log.gz 2023-12-26T23:13:35.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58964.log 2023-12-26T23:13:35.372 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71566.log 2023-12-26T23:13:35.372 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78241.log.gz 2023-12-26T23:13:35.374 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58964.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58964.log.gz 2023-12-26T23:13:35.374 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47993.log 2023-12-26T23:13:35.375 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-12-26T23:13:35.375 INFO:teuthology.orchestra.run.smithi131.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71566.log.gz 2023-12-26T23:13:35.376 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102525.log 2023-12-26T23:13:35.377 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47993.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47993.log.gz 2023-12-26T23:13:35.377 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87624.log 2023-12-26T23:13:35.379 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102525.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102525.log.gz 2023-12-26T23:13:35.379 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77979.log 2023-12-26T23:13:35.381 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87624.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87624.log.gz 2023-12-26T23:13:35.381 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39732.log 2023-12-26T23:13:35.383 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77979.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77979.log.gz 2023-12-26T23:13:35.383 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95880.log 2023-12-26T23:13:35.384 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39732.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39732.log.gz 2023-12-26T23:13:35.384 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60987.log 2023-12-26T23:13:35.386 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95880.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95880.log.gz 2023-12-26T23:13:35.386 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85338.log 2023-12-26T23:13:35.388 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60987.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60987.log.gz 2023-12-26T23:13:35.388 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48470.log 2023-12-26T23:13:35.389 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.85338.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85338.log.gz 2023-12-26T23:13:35.390 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78618.log 2023-12-26T23:13:35.391 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.48470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48470.log.gz 2023-12-26T23:13:35.391 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79161.log 2023-12-26T23:13:35.393 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78618.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78618.log.gz 2023-12-26T23:13:35.393 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78174.log 2023-12-26T23:13:35.395 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79161.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79161.log.gz 2023-12-26T23:13:35.395 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73162.log 2023-12-26T23:13:35.397 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78174.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78174.log.gz 2023-12-26T23:13:35.397 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67939.log 2023-12-26T23:13:35.398 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.73162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73162.log.gz 2023-12-26T23:13:35.398 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49587.log 2023-12-26T23:13:35.400 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67939.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67939.log.gz 2023-12-26T23:13:35.400 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104411.log 2023-12-26T23:13:35.401 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49587.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49587.log.gz 2023-12-26T23:13:35.402 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-12-26T23:13:35.404 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104411.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104411.log.gz 2023-12-26T23:13:35.404 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61033.log 2023-12-26T23:13:35.407 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.61033.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61033.log.gz 2023-12-26T23:13:35.407 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55514.log 2023-12-26T23:13:35.410 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100116.log 2023-12-26T23:13:35.414 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53646.log 2023-12-26T23:13:35.414 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55514.log.gz 2023-12-26T23:13:35.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100116.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100116.log.gz 2023-12-26T23:13:35.415 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53646.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53646.log.gz 2023-12-26T23:13:35.415 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95834.log 2023-12-26T23:13:35.418 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95834.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95834.log.gz 2023-12-26T23:13:35.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74884.log 2023-12-26T23:13:35.421 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74884.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74884.log.gz 2023-12-26T23:13:35.422 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49817.log 2023-12-26T23:13:35.425 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60849.log 2023-12-26T23:13:35.426 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49817.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49817.log.gz 2023-12-26T23:13:35.426 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60849.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60849.log.gz 2023-12-26T23:13:35.427 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72675.log 2023-12-26T23:13:35.430 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72675.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72675.log.gz 2023-12-26T23:13:35.430 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55184.log 2023-12-26T23:13:35.433 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55184.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55184.log.gz 2023-12-26T23:13:35.433 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72767.log 2023-12-26T23:13:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72767.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72767.log.gz 2023-12-26T23:13:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94231.log 2023-12-26T23:13:35.439 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94231.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94231.log.gz 2023-12-26T23:13:35.440 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33000.log 2023-12-26T23:13:35.443 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.tmp-client.admin.33000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77669.log 2023-12-26T23:13:35.444 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33000.log.gz 2023-12-26T23:13:35.445 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103122.log 2023-12-26T23:13:35.447 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77669.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77669.log.gz 2023-12-26T23:13:35.448 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103122.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103122.log.gz 2023-12-26T23:13:35.448 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54848.log 2023-12-26T23:13:35.451 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54848.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54848.log.gz 2023-12-26T23:13:35.452 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87790.log 2023-12-26T23:13:35.455 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87790.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87790.log.gz 2023-12-26T23:13:35.455 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90166.log 2023-12-26T23:13:35.458 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90166.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90166.log.gz 2023-12-26T23:13:35.458 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55316.log 2023-12-26T23:13:35.461 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64837.log 2023-12-26T23:13:35.463 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55316.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55316.log.gz 2023-12-26T23:13:35.463 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79635.log 2023-12-26T23:13:35.465 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64837.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64837.log.gz 2023-12-26T23:13:35.466 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.79635.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79635.log.gz 2023-12-26T23:13:35.466 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67207.log 2023-12-26T23:13:35.469 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67207.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67207.log.gz 2023-12-26T23:13:35.470 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101663.log 2023-12-26T23:13:35.473 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.101663.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101663.log.gz 2023-12-26T23:13:35.473 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59612.log 2023-12-26T23:13:35.476 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59612.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59612.log.gz 2023-12-26T23:13:35.476 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86091.log 2023-12-26T23:13:35.479 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86091.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86091.log.gz 2023-12-26T23:13:35.480 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47417.log 2023-12-26T23:13:35.483 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47417.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47417.log.gz 2023-12-26T23:13:35.483 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75228.log 2023-12-26T23:13:35.486 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75228.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75228.log.gz 2023-12-26T23:13:35.487 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52807.log 2023-12-26T23:13:35.489 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52807.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52807.log.gz 2023-12-26T23:13:35.490 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52874.log 2023-12-26T23:13:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52874.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52874.log.gz 2023-12-26T23:13:35.493 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51893.log 2023-12-26T23:13:35.496 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51893.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51893.log.gz 2023-12-26T23:13:35.497 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67526.log 2023-12-26T23:13:35.498 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102732.log 2023-12-26T23:13:35.502 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90304.log 2023-12-26T23:13:35.502 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.102732.log: /var/log/ceph/ceph-client.admin.67526.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102732.log.gz 2023-12-26T23:13:35.502 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67526.log.gz 2023-12-26T23:13:35.503 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72284.log 2023-12-26T23:13:35.507 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90304.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90304.log.gz 2023-12-26T23:13:35.507 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55118.log 2023-12-26T23:13:35.507 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72284.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72284.log.gz 2023-12-26T23:13:35.508 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55118.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55118.log.gz 2023-12-26T23:13:35.509 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55646.log 2023-12-26T23:13:35.511 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55646.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55646.log.gz 2023-12-26T23:13:35.512 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50810.log 2023-12-26T23:13:35.515 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50810.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50810.log.gz 2023-12-26T23:13:35.515 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76365.log 2023-12-26T23:13:35.518 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76365.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76365.log.gz 2023-12-26T23:13:35.519 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59418.log 2023-12-26T23:13:35.521 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65376.log 2023-12-26T23:13:35.522 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59418.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59418.log.gz 2023-12-26T23:13:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65376.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65376.log.gz 2023-12-26T23:13:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52553.log 2023-12-26T23:13:35.529 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52553.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52553.log.gz 2023-12-26T23:13:35.529 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69586.log 2023-12-26T23:13:35.532 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69586.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69586.log.gz 2023-12-26T23:13:35.532 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51939.log 2023-12-26T23:13:35.534 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94662.log 2023-12-26T23:13:35.535 INFO:teuthology.orchestra.run.smithi190.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-12-26T23:13:35.537 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51939.log: /var/log/ceph/ceph-client.admin.94662.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.51939.log.gz 2023-12-26T23:13:35.537 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61125.log 2023-12-26T23:13:35.537 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.94662.log.gz 2023-12-26T23:13:35.540 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61125.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61125.log.gz 2023-12-26T23:13:35.540 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45333.log 2023-12-26T23:13:35.541 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64575.log 2023-12-26T23:13:35.545 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72629.log 2023-12-26T23:13:35.545 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64575.log: /var/log/ceph/ceph-client.admin.45333.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64575.log.gz 2023-12-26T23:13:35.545 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45333.log.gz 2023-12-26T23:13:35.547 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72629.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72629.log.gz 2023-12-26T23:13:35.547 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103997.log 2023-12-26T23:13:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.103997.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103997.log.gz 2023-12-26T23:13:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49403.log 2023-12-26T23:13:35.553 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74214.log 2023-12-26T23:13:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49403.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49403.log.gz 2023-12-26T23:13:35.557 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90396.log 2023-12-26T23:13:35.557 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74214.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74214.log.gz 2023-12-26T23:13:35.558 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.90396.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90396.log.gz 2023-12-26T23:13:35.558 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46198.log 2023-12-26T23:13:35.561 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46198.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46198.log.gz 2023-12-26T23:13:35.562 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50477.log 2023-12-26T23:13:35.565 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50477.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50477.log.gz 2023-12-26T23:13:35.565 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74578.log 2023-12-26T23:13:35.568 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74578.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74578.log.gz 2023-12-26T23:13:35.568 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104624.log 2023-12-26T23:13:35.571 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104624.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104624.log.gz 2023-12-26T23:13:35.572 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83608.log 2023-12-26T23:13:35.575 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.83608.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83608.log.gz 2023-12-26T23:13:35.575 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104181.log 2023-12-26T23:13:35.578 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.104181.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104181.log.gz 2023-12-26T23:13:35.579 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72332.log 2023-12-26T23:13:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72332.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72332.log.gz 2023-12-26T23:13:35.582 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49633.log 2023-12-26T23:13:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49633.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49633.log.gz 2023-12-26T23:13:35.585 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77137.log 2023-12-26T23:13:35.588 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77137.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77137.log.gz 2023-12-26T23:13:35.589 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89318.log 2023-12-26T23:13:35.592 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89318.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89318.log.gz 2023-12-26T23:13:35.592 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51525.log 2023-12-26T23:13:35.595 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51525.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51525.log.gz 2023-12-26T23:13:35.595 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99183.log 2023-12-26T23:13:35.598 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99183.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99183.log.gz 2023-12-26T23:13:35.599 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100292.log 2023-12-26T23:13:35.602 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100292.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100292.log.gz 2023-12-26T23:13:35.602 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67872.log 2023-12-26T23:13:35.605 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67872.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67872.log.gz 2023-12-26T23:13:35.606 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65527.log 2023-12-26T23:13:35.608 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65527.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65527.log.gz 2023-12-26T23:13:35.609 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60895.log 2023-12-26T23:13:35.612 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60895.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60895.log.gz 2023-12-26T23:13:35.612 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70693.log 2023-12-26T23:13:35.615 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70693.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70693.log.gz 2023-12-26T23:13:35.615 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59222.log 2023-12-26T23:13:35.618 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59222.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59222.log.gz 2023-12-26T23:13:35.619 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68934.log 2023-12-26T23:13:35.620 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39606.log 2023-12-26T23:13:35.624 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66723.log 2023-12-26T23:13:35.624 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39606.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39606.log.gz 2023-12-26T23:13:35.625 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66723.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66723.log.gz 2023-12-26T23:13:35.626 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68934.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68934.log.gz 2023-12-26T23:13:35.627 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100802.log 2023-12-26T23:13:35.630 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78526.log 2023-12-26T23:13:35.631 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100802.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100802.log.gz 2023-12-26T23:13:35.631 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78526.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78526.log.gz 2023-12-26T23:13:35.631 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49495.log 2023-12-26T23:13:35.635 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49495.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49495.log.gz 2023-12-26T23:13:35.635 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54664.log 2023-12-26T23:13:35.638 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54664.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54664.log.gz 2023-12-26T23:13:35.638 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65853.log 2023-12-26T23:13:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65853.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65853.log.gz 2023-12-26T23:13:35.642 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89840.log 2023-12-26T23:13:35.645 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94596.log 2023-12-26T23:13:35.646 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89840.log.gz 2023-12-26T23:13:35.647 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99979.log 2023-12-26T23:13:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76650.log 2023-12-26T23:13:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.99979.log: /var/log/ceph/ceph-client.admin.94596.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99979.log.gz 2023-12-26T23:13:35.650 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94596.log.gz 2023-12-26T23:13:35.653 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76650.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76650.log.gz 2023-12-26T23:13:35.653 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60617.log 2023-12-26T23:13:35.657 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60617.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60617.log.gz 2023-12-26T23:13:35.657 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87339.log 2023-12-26T23:13:35.659 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.87339.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87339.log.gz 2023-12-26T23:13:35.660 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94800.log 2023-12-26T23:13:35.663 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.94800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88705.log 2023-12-26T23:13:35.666 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94800.log.gz 2023-12-26T23:13:35.667 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60463.log 2023-12-26T23:13:35.667 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.88705.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88705.log.gz 2023-12-26T23:13:35.667 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60463.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60463.log.gz 2023-12-26T23:13:35.667 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89180.log 2023-12-26T23:13:35.671 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89180.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89180.log.gz 2023-12-26T23:13:35.671 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86993.log 2023-12-26T23:13:35.674 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.86993.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86993.log.gz 2023-12-26T23:13:35.675 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95018.log 2023-12-26T23:13:35.677 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95018.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95018.log.gz 2023-12-26T23:13:35.678 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95950.log 2023-12-26T23:13:35.681 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.95950.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95950.log.gz 2023-12-26T23:13:35.681 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60004.log 2023-12-26T23:13:35.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60004.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60004.log.gz 2023-12-26T23:13:35.685 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77772.log 2023-12-26T23:13:35.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94139.log 2023-12-26T23:13:35.688 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77772.log.gz 2023-12-26T23:13:35.690 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53484.log 2023-12-26T23:13:35.693 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78710.log 2023-12-26T23:13:35.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53484.log: /var/log/ceph/ceph-client.admin.94139.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53484.log.gz 2023-12-26T23:13:35.693 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94139.log.gz 2023-12-26T23:13:35.695 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.78710.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78710.log.gz 2023-12-26T23:13:35.696 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69172.log 2023-12-26T23:13:35.699 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69172.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69172.log.gz 2023-12-26T23:13:35.699 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59888.log 2023-12-26T23:13:35.702 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59888.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59888.log.gz 2023-12-26T23:13:35.703 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89702.log 2023-12-26T23:13:35.706 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.89702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96074.log 2023-12-26T23:13:35.707 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89702.log.gz 2023-12-26T23:13:35.708 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45577.log 2023-12-26T23:13:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.96074.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96074.log.gz 2023-12-26T23:13:35.711 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47187.log 2023-12-26T23:13:35.712 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45577.log.gz 2023-12-26T23:13:35.712 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93703.log 2023-12-26T23:13:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100940.log 2023-12-26T23:13:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47187.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47187.log.gz 2023-12-26T23:13:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.93703.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93703.log.gz 2023-12-26T23:13:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51663.log 2023-12-26T23:13:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.100940.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100940.log.gz 2023-12-26T23:13:35.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51663.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51663.log.gz 2023-12-26T23:13:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69448.log 2023-12-26T23:13:35.723 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69448.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69448.log.gz 2023-12-26T23:13:35.799 INFO:teuthology.orchestra.run.smithi190.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-12-26T23:13:36.871 INFO:teuthology.orchestra.run.smithi131.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-12-26T23:13:36.956 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-12-26T23:13:37.147 INFO:teuthology.orchestra.run.smithi190.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-12-26T23:13:37.205 INFO:teuthology.orchestra.run.smithi131.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-12-26T23:13:37.351 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-12-26T23:13:37.561 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-12-26T23:13:37.565 INFO:teuthology.orchestra.run.smithi190.stderr: 2023-12-26T23:13:37.565 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m3.435s 2023-12-26T23:13:37.565 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m15.186s 2023-12-26T23:13:37.565 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m1.352s 2023-12-26T23:13:37.640 INFO:teuthology.orchestra.run.smithi131.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-12-26T23:13:37.825 INFO:teuthology.orchestra.run.smithi131.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-12-26T23:13:37.908 INFO:teuthology.orchestra.run.smithi131.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-12-26T23:13:38.236 INFO:teuthology.orchestra.run.smithi131.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-12-26T23:13:38.241 INFO:teuthology.orchestra.run.smithi131.stderr: 2023-12-26T23:13:38.241 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m4.115s 2023-12-26T23:13:38.241 INFO:teuthology.orchestra.run.smithi131.stderr:user 0m17.924s 2023-12-26T23:13:38.241 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m3.403s 2023-12-26T23:13:38.242 INFO:tasks.ceph:Archiving logs... 2023-12-26T23:13:38.242 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/remote/smithi131/log 2023-12-26T23:13:38.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-26T23:13:40.977 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/remote/smithi190/log 2023-12-26T23:13:40.978 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-26T23:13:42.742 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-26T23:13:42.751 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-26T23:13:42.751 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-26T23:13:42.756 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-26T23:13:42.881 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-26T23:13:42.889 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-26T23:13:42.890 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-26T23:13:42.894 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-26T23:13:42.932 INFO:teuthology.orchestra.run.smithi131.stderr:bash: ntpq: command not found 2023-12-26T23:13:42.935 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2023-12-26T23:13:42.939 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-26T23:13:42.940 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2023-12-26T23:13:42.940 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 178 +142us[ +150us] +/- 77ms 2023-12-26T23:13:42.940 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv02.front.sepia.ceph.com 3 7 377 106 +201us[ +202us] +/- 49ms 2023-12-26T23:13:42.940 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 104 -261us[ -261us] +/- 83ms 2023-12-26T23:13:42.940 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T23:13:42.943 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-26T23:13:42.943 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-12-26T23:13:42.943 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 111 +81us[ +81us] +/- 78ms 2023-12-26T23:13:42.943 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv02.front.sepia.ceph.com 3 8 377 235 +220us[ +218us] +/- 79ms 2023-12-26T23:13:42.943 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 102 -303us[ -303us] +/- 83ms 2023-12-26T23:13:42.943 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-12-26T23:13:42.944 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-26T23:13:42.952 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-26T23:13:42.953 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-26T23:13:42.960 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-26T23:13:43.135 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-26T23:13:43.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-26T23:13:43.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T23:13:43.338 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-26T23:13:43.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T23:13:43.409 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-26T23:13:43.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-26T23:13:43.427 INFO:teuthology.task.internal:Duration was 6535.736685 seconds 2023-12-26T23:13:43.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-26T23:13:43.435 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-26T23:13:43.435 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-26T23:13:43.439 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-26T23:13:43.551 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-26T23:13:43.557 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-26T23:13:43.930 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-26T23:13:43.930 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2023-12-26T23:13:43.930 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-26T23:13:43.988 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2023-12-26T23:13:43.988 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-26T23:13:44.046 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-26T23:13:44.046 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-26T23:13:44.050 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-26T23:13:44.130 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-26T23:13:44.131 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-26T23:13:44.135 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-26T23:13:44.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-26T23:13:44.268 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-26T23:13:44.268 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-26T23:13:44.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-26T23:13:44.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-26T23:13:44.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-26T23:13:44.418 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-26T23:13:44.480 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2023-12-26T23:13:44.485 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2023-12-26T23:13:44.521 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-26T23:13:44.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T23:13:44.562 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-26T23:13:44.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-26T23:13:44.603 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-26T23:13:44.612 INFO:teuthology.task.internal:Transferring archived files... 2023-12-26T23:13:44.612 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/remote/smithi131 2023-12-26T23:13:44.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-26T23:13:44.689 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800/remote/smithi190 2023-12-26T23:13:44.689 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-26T23:13:44.763 INFO:teuthology.task.internal:Removing archive directory... 2023-12-26T23:13:44.763 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-26T23:13:44.767 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-26T23:13:44.809 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-26T23:13:44.820 INFO:teuthology.task.internal:Not uploading archives. 2023-12-26T23:13:44.820 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-26T23:13:44.829 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-26T23:13:44.830 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-26T23:13:44.850 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-26T23:13:44.889 INFO:teuthology.orchestra.run.smithi131.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 26 23:13 /home/ubuntu/cephtest 2023-12-26T23:13:44.893 INFO:teuthology.orchestra.run.smithi190.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 26 23:13 /home/ubuntu/cephtest 2023-12-26T23:13:44.896 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-26T23:13:44.906 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-26T23:13:44.924 INFO:teuthology.nuke:Checking targets against current locks 2023-12-26T23:13:44.951 DEBUG:teuthology.nuke:shortname: smithi131 2023-12-26T23:13:44.951 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-26T23:13:44.974 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-26 21:13:34.091589', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvWSIYGN42nYXiZs+QFU5n975Xqni3CVSzG4cs89fCCP7TwSr65VsCey5e0Y0vLDmCeAeWE8TtPLR38WHbqHJ8='} 2023-12-26T23:13:44.977 DEBUG:teuthology.nuke:shortname: smithi190 2023-12-26T23:13:44.977 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-26T23:13:44.998 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501800', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-26 21:13:34.089700', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBElAKLCJzJNC0gYvfEhSyz+aEWg0uonEKHK9wzdREnEf7++OXkZtvR7zKpar+37n0Ax4xVUToUYupuA29ji5gfA='} 2023-12-26T23:13:45.027 INFO:teuthology.orchestra.console.smithi131:Power off 2023-12-26T23:13:45.027 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-26T23:13:45.056 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2023-12-26T23:13:45.056 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-26T23:13:45.070 INFO:teuthology.orchestra.console.smithi190:Power off 2023-12-26T23:13:45.070 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-26T23:13:45.081 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-12-26T23:13:45.094 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2023-12-26T23:13:45.094 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-26T23:13:45.115 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-12-26T23:13:49.083 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-26T23:13:49.116 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-26T23:13:49.193 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-12-26T23:13:49.225 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-12-26T23:13:53.194 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-26T23:13:53.226 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-26T23:13:53.303 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-12-26T23:13:53.333 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-12-26T23:13:53.404 INFO:teuthology.orchestra.console.smithi131:Power off completed 2023-12-26T23:13:57.338 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-26T23:13:57.449 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-12-26T23:13:57.549 INFO:teuthology.orchestra.console.smithi190:Power off completed 2023-12-26T23:13:57.655 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 623393c9d50c04c7ea1d88824b8e92712e3e5611 client.1-kernel-sha1: 623393c9d50c04c7ea1d88824b8e92712e3e5611 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6535.736684560776 failure_reason: '"1703626748.1495957 mon.a (mon.0) 324 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-12-26T23:13:57.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-26T23:13:57.735 INFO:teuthology.run:FAIL